Hi everyone,
I am very close to releasing a public beta of my “live multiplayer” plugin: TW5-Yjs
I started out by writing an update to the Bob server sync mechanism, but it quickly turned out that re-writing the underlaying logic was turning the tool into its own “thing”.
As such, I wanted to touch base with the community on the key features I am focusing on, and ask if there are any ideas/suggestions/etc for tools along these lines.
::Aside:: After watching Saq’s excellent presentation on the File-Uploads plugin, and studying the Yjs library further, I think I will end up splitting this up into a Yjs “core” and a set of “provider” (the Yjs term) plugins that hook everything together. This current prerelease under discussion would be TW5-Yjs and TW5-YjsWebsockets.
Key Features
- Multi User Server
- MultiServer extends the native TW5 Server
- adds an “admin” Authorization type that controls who has administrative access to the server functions
- and can be configured to Simple or Header based Authentication
- and can support randomized Anonymous Usernames for Anon users
- and has a secure “Read-Only” mode with changes to Yjs docs from “readers” ignored by the server.
- Multi Wiki Server
- with each wiki considered a “room” with a unique server path
- with a main “RootWiki” at the root server path that allows admin users to:
- create and delete Wikis
- modify any served wiki’s “tiddlywiki.info” file from the RootWiki
- LIVE Multiplayer experience
- All updates to a wiki are sent to all connected users live via a WebSocket Client-Server model
- “Updates” trade Yjs info about the snapshot/state of the wiki document and the “Confict Free Replicated Datatype” logic ensures that “all users sync to the same state… eventually”
- All fields handled as “text strings”. For now, field values in the Yjs-doc “copy” of the tiddler are of “string type”, and replaced by the incoming value. Eventually, the full “Yjs Text type” will be used, which will allow sharing of individual insert/delete actions, tracking Undos, showing active User’s cursors with different colors, etc. Eventually, other types might be added, with field-name prefixes allowing a “grow-only/field-name” as a grow-only integer, etc. This is advanced Yjs/CRDT stuff.
Some things to consider
- FileWatcher system?
- Bob has a functional system to watch the filesystem for changes, and update a wiki-in memory.
- Yjs files as “source of truth”
- Can we simply trust a “server address + $:/SiteTitle” to identify a unique wiki?
- Bob uses an internal
$tw.WikiName
variable which becomes part of the “path” to the wiki and is exposed in a$:/status/WikiName
tiddler. - Is this a good place for a UUID?
- Bob uses an internal
- While there are good examples on how to save “Yjs Documents” to persistent storage, they are mostly Database focused (browser & server).
- This means that any wiki loaded from another source (a single-file or a node-folder) would need to be compared to a Yjs document, with the “incoming” wiki changes being used to update the Yjs doc (this in turn updates all connected clients). Server-restarts, loading from a single-file (or other source, say Fission) but connecting to a WebsocketServer, etc. This all needs examining.
- Can we simply trust a “server address + $:/SiteTitle” to identify a unique wiki?
There are many technical details about this setup, but feel free to just shout out ideas.
One pain point noticed so far: The Storylist UI does not differentiate from “close this open tiddler” and “discard/delete this open draft”. “Closing” an open draft tiddler immediately deletes it from the wiki, which can interrupt others editing their own drafts. A simple update of the icon for the “discard/delete this draft” to say, a trash-can, and adding a “miniminze/remove draft from storylist” button with the “X” would be a good start (thanks @pmario for this suggestion).
::Another Aside:: Would this be a set of features that the community would get behind as an OpenCollective task?