Desktop agent for MWS?

Developers
#1

Hi,

Since MWS would allow for big files to be uploaded and also to create shared spaces between users, I wonder if could be used as a document management system.

Use case example:

  • UserA and UserB share a bag, and UserA uploads a .doc file in it.
  • On upload a new tiddler is created which also contains metadata of the file (author, file title, upload date, …)
  • If UserB wants to edit the file, it can click on the tiddler and a TW desktop agent opens the file in Microsoft Word in .tmp mode, and on save, the changed .doc is uploaded to TW.

Desktop Agent role (redacted with some AI assistance):

  • Role: A background app on the user’s computer that connects the browser to the local operating system.
  • Functions:
    • Listens for an edit command from the TiddlyWiki frontend via a WebSocket.
    • Checks Out: When commanded, it locks the file on the server, downloads it as a temporary file (e.g., document.tmp), and opens it with the default local application like Word.
    • Monitors: Watches the temporary file for save events using fs.watch.
    • Checks In: When the file is saved, the agent uploads the new version to the server, unlocks it, and deletes the temporary file.

This Desktop Agent allows to surpass the sandbox limitations of browsers, it can interact with the OS and I think this new capabilities might allow for interesting usecases.

Is there something like this planed on the development roadmap?
Do you think this is a path that might be worth exploring it?

#2

I do not want to be negative, but I have to be “the devil’s advocate” here

I think MWS is not designed to be a “binary file management” system. It is designed to manage meta-data text files about binary files. Those tiddlers “link” to the binary files.

@Arlen22 – As fare as I know MWS has no special handling for huge 3rd party files at the moment. Saving them inside the database will still kill the database performance over time. – IMO Huge binary files have to have special treatment. – Only the meta data should go into the database.

Syncing in a consistent way is extremely hard. There are whole software packages, that do only sync. Nothing else. There are several things that are important for document management.

  1. Data must not be lost
  2. After changing - A rollback must be possible
  3. Data must not be blocked

While a locking mechanism is relatively simple, failing is inevitable. Eg:

A common one:

  • User A checks out the document
  • User A goes on holiday or is ill for 2 weeks
  • The document is blocked

or Tricky one:

  • User A checks out the document
  • Lock is written back to database
  • Software crashes → Nothing written to disk or to database
  • Neither User A nor User B can access the document, in a regular way, because it is locked
  • Admin is on holiday – No not on holiday – but has time in 2 days

or An “impossible” one:

  • User A and User B check out the document at the same time
  • Do not tell me that’s not possible. According to Murphy’s law it is possible.
  • User A lock wins
  • User B thinks he can write
  • This problem is not easy to resolve, without low level manipulating the locking mechanism. Or manually resolving the problem, users copying files manually back and forth. Doing the more work twice. …

or A fatal one:

  • User A and user B happily work with the document
  • It is in the database and it is .doc format
  • The document is fine and sits in the database for 5 years
  • User A and user B changed company
  • User C has the latest PC and software stack. C checks out the document
  • Result: The .doc file, can not be opened anymore because the software can not handle the 5 year old proprietary data format

and so on and so on.

Trust me those (and many more) problems are real. I have experienced them all using Visual SourceSafe in the `90s and early 2000th. IMO - A check-out and locking system does not work.

I did not even discuss important point 2. A rollback must be possible.

  • I think document management needs a version control system
  • Which is extremely disk space hungry, if binary files are used
    • So again extra handling for binary data

@Arlen22 – As far as I know MWS does not have version control yet. But I think it’s planned. – BUT versioning for text based tiddlers is easy, compared to binary data.

Just my thoughts.
-m

#3

I’m not sure I understand how this use case relates to TiddlyWiki first of all.

There are already many existing documents storage and document syncing solutions available for desktop and mobile. There are both commercial and open source solutions available.

I can appreciate the feeling of wanting to use tiddlywiki for everything, and I wouldn’t be surprised at all if someone would write a tool that syncs regular files into MWS as tiddlers.

I think the direction we’re headed with MWS is more in the direction of it being a wiki. So we’ll probably have text-based conflict resolution, edit histories, and stuff like that that we’re already familiar with from WikiPedia and other wiki software.

It would be far easier to just integrate MWS as a wiki option in some kind of existing open source solution than to try to integrate document storage into MWS.

MWS is focused on solving the wiki use case for TiddlyWiki. The browser sandbox is not an issue for MWS because it is server-based like any other website. A generic binary document storage solution (aka cloud drive) is very different from a markup storage solution (aka wiki).

There are existing tools in the TW5 verse which allow you to save your wikis to desktop, and those can then be synced to any cloud drive using their normal syncing tools. So it feels like that use-case has been a long-solved scenario and we’re more focused on the wiki use-case.

1 Like
#4

Well I clearly did not account for the caveats mentioned by @pmario , probably makes more sense to limit TW usage for frontend stuff and manage blob files with a dedicated backend storage solution, and integrate it via APIs.

why Desktop Agent?:
I was recently working with an html5 DMS written in Java. It was interesting to me that eventhough I was in a web browser, when I clicked the edit button next to the title of an uploaded .doc file, a Microsoft Word instance was launched in my OS (I thought that browsers where isolated from the OS, like a sandbox environment… This behavior was possible because of Java applets).

I then asked an AI about how to replicate this functionality with a NodeJs TW, and It recommended to install a TW desktop agent in my computer so it acts as a bridge between the browsed TW and the OS.

Apparently the use of desktop agents is a common practice, used by Spotify (to skip songs with keyboard buttons even when spotify is running in a different tab) and Slack (to pass notifications to the OS taskbar) among others.

So I thought that a Desktop Agent for TW might allow for some interesting use cases that surpass the sandbox limitation of browsers.

About creating a DMS, how feasible might that be to integrate a MWS frontend with a backend storage solution via APIs?

I would like the frontend to be in TW because it allows for very flexible and customizable data driven UI. But at the same time I wish the UI allows to manage .docs, .xml files and so on. I guess @poc2go insights might be relevant here too because of your expertise on combining TW with other tools.

#5

I too, like the idea of “TW Desktop Agent” to overcome the sandbox limitation of browsers, and think it’s worth discussing. The term “Desktop Agent” is so apt, I am going to steal it !

“I then asked an AI about how to replicate this functionality with a NodeJs TW, and It recommended to install a TW desktop agent in my computer so it acts as a bridge between the browsed TW and the OS.”

I’m not a TW node.js user, but I believe if node.js is your desktop, then node.js is capable of the Desktop Agent role i.e. receive commands from frontend TW instance to launch desktop file or interface with backend storage solution etc. But if it’s anything like my other experiences with TW, this was probably already done a decade back, became a norm, people move on and you can’t find any trace of it on a new forum like talk.tiddlywiki.org :wink:

I did test a proof-of-concept of TW Desktop Agent and shared at https://talk.tiddlywiki.org/t/exploring-tw-desktop-integration-using-tm-http-request. This is meant to be just a small tool to allow TW’s TM-HTTP-REQUEST to inform the desktop agent to perform (via user scripts) those desktop tasks that the TW frontend cannot.

I didn’t work on it since. I think the utility will be more obvious if I have made the demo a TW desktop file browser that can launch files at where the desktop agent is planted. However, that will need more time. For now, I’m just collecting info. Today, I read that Android’s Tasker app is not an Android TV app, but it can be sideloaded on Android TV and use, with some difficulties. Tasker supports the XML-HTTP-REQUEST protocol used in TM-HTTP-REQUEST so there is that slim, slim possibility of controlling your Android TV from TW using TW desktop agent :wink: