Sharing & Synchronizing TiddlyWikis with a CRDT protocol

Developers
#1

I’m thinking of using TiddlyWiki as a shared, distributed wiki - running in the browser - with all the basic features of WikiMedia (a talk page & change history behind each page/tiddler).

I’m thinking of organizing a TiddlyWiki much like an NNTP newsreader & spool, or an RSS blogroll. Groups/Feeds - each consisting of messages that get auto assembled into threads, and from there, sorted & merged into documents.

Which leads me to wonder about the syncer.js protocol & refresh/sync mechanism.

What I’d like to do is post additions & edits to message-like feeds (RSS, SMTP lists, NNTP, DIS, HLA, DDS, XMPP, syslog) - that can be distributed via broadcast, multicast, publish-subscribe, and/or gossip mechanisms). And then have subscribing clients (tiddlywikis) integrate changes as they come in. I’d also like to be able to write directly to the feed with AtomPub. (All with OAUTH style authorization & key management - UCAN style.)

Clearly, this requires some kind of eventual consistency (CRDT) or automatic merging/versioning mechanism - as in PouchDB/CouchDB, or a network of git repositories, or IPFS.

Which brings me to several questions about the core update mechanisms & protocols used by TiddlyWiki:

  • does the current update mechanism operate like a CRDT - allowing for asynchronous updates from multiple sources? And/or,
  • does the current mechanism provide for versioning, as with WikiMedia, or git?
  • is there a restful interface and/or message passing interface?
  • how do copies of a TiddlyWiki connect to an update stream (i.e., via a URL vs. downloading a copy from a master on TiddlyHost)

And… to what extent is this set of features already built into a TiddlyWiki derivative?

Thanks Very Much, Miles Fidelman

Multi User Alternative to TiddlyWiki
#2

Hi @mfidelman, and welcome!

CRDT-related things like y.js and automerge have come up here a couple of times - you might want to search for “y.js” and “yjs” among the existing topics, and you also might want to talk to @joshuafontany, who (IIRC) has spent some time experimenting with y.js and TiddlyWiki.

-Rob

#3

[TW5-Yjs] Real Time Sync Brainstorm looks especially relevant!

#4

Probably you can get some value out of Composer, which is an open source initiative to share and synchronize documents with the CRDT protocol.

#5

Because I cant find any definition of the acronym CRDT (Not the best communication) I asked for a description from ChatGPT.

Does this reflect what is being discussed?

CRDT stands for Conflict-free Replicated Data Type.

It’s a specialized data structure designed for distributed computing environments, where multiple users or systems simultaneously modify the same data without coordination. CRDTs automatically ensure consistency among replicas, eventually converging to the same state across all nodes, even after conflicting updates.

Key Characteristics:

  • Conflict-free: No conflicts arise that require manual resolution.
  • Replicated: Copies of data exist on multiple nodes.
  • Convergent: Different nodes independently merge updates, eventually reaching the same state (eventual consistency).
  • Commutative: Operations can be applied in any order.

Types of CRDTs:

  1. Operation-based CRDTs (CmRDT):

    • Broadcast operations immediately.
    • Examples: Collaborative text editors (Google Docs, real-time collaboration tools).

  2. State-based CRDTs (CvRDT):

    • Nodes periodically exchange entire states or deltas.
    • Example: Counters, sets (G-sets, 2P-sets, OR-sets), registers.

Common Applications:

  • Collaborative applications (real-time editing).
  • Distributed databases and caching systems.
  • Offline-first and synchronization applications.

Simple Example:

  • G-counter:
    A counter CRDT where each node maintains its own local increment value. To get the total, sum increments from all nodes. No matter the order of updates, the total count remains consistent.

Why use CRDTs?

  • Provide automatic conflict resolution, simplifying distributed application design.
  • Allow users to work offline and sync later without complex logic.
  • Improve performance and scalability in distributed environments.

In short, CRDTs make distributed and collaborative systems simpler, more robust, and more user-friendly by ensuring data consistency without requiring explicit synchronization or conflict resolution logic.