Solutions for syncing the same TiddlyWiki across many devices?

#17

To be clear, I’m not here looking for solutions or advice for saving across platforms/locations! I’m quite pleased with TiddlyHost (often using auto-save), especially now that TiddlyHost can alert me if/when I do trip up and try to save new changes from a stale instance. (And now that TiddlyHost. also keeps a great backup archive!)

My comments above were simply emphasizing that while I’m not personally a likely customer for a single-author check-in check-out approach, I am very excited about the prospect of getting a multi-user solution at TiddlyHost. For that kind of special project, even I would bring myself to work deliberately within check-in-check-out parameters. :upside_down_face:

1 Like
#18

I use git and have a backup automatically run every minute or so on my computer. On my phone, I have a shortcut that runs git sync commands in the background (and pushes whenever I exit Quine, my mobile TiddlyWiki client). This works well (and has the added benefit that you can “roll back” your Wiki easily).

1 Like
#19

Resilio Sync is the original mutable torrents tool, and by far the most performant of its class. You’ll still face race conditions during syncing (not a big deal by hand, but it can matter if you’ve more automation in your wiki construction workflow). I’ve run into trouble synchronizing multiple old Bob instances, but theoretically it should work.

1 Like
#20

I use git and have a backup automatically run every minute or so on my computer

Interesting! So do each save operation - automatic or initiated by you - create a commit to a target Git repository? If so:

  1. Do the actual contents of each commit, and the diff from the previous commit, look sensible? And does it matter? And what would the commit message be??
  2. Does this Git-based solution allow dealing with sync conflicts?
1 Like
Advice on how to host/backup/store my Wiki
#21

Thanks @session. I’ve heard about Hubzilla but didn’t know it comes with WebDAV. I can see how that might be another way to sync TiddlyWiki files, in addition to other tools that have been mentioned.

But I can’t figure out how to find a reliable Hubzilla instance to sign up for, or even find out the features and limitations (e.g. storage quota, etc.) of instances. Or am I expected to host my own Hubzilla instance and federate with others???

#22

finding a reliable instance is always the problem in the Fedivese :slight_smile: if you want to do something serious you actually really should considering self hosting an instance… Hubzilla does make it quid easy…

But for the start there are quid some public instances you can use for a try out.

https://talk.tiddlywiki.org/t/option-for-hosting-tiddlywikis-with-an-integrated-access-control-permission-system-and-fediverse-connectivity

under /siteinfo you find some information about installed addons - yes storage can be limited - but mostly there is plenty storage granted

2 Likes
#23

For the last couple of months, I have been using a set up in which I have copies of a single file TiddlyWiki for notes on my desktop, tablet and phone, and the three copies of the same wiki synchronize with each other on a tiddler level.

The sync is via an in-development plugin (Harmony) that synchronizes tiddlers to a private GitHub repo which essentially serves as a free sync server, with the user not requiring any knowledge of git. Any changes to a wiki configured to sync to a given repo are automatically propagated to other wikis syncing to the same repo.

Essentially this is the same model used by apps such as Evernote, where you have different instances of the app on different devices that can all work separately and offline, but they also sync with each other via a server. Here, the private GitHub repo plays the role of a no configuration necessary free server, and you retain full control over your data. Creating a private GitHub repo is very easy even for non-technical users, whereas setting up your own server securely can be a complex undertaking.

This setup allows for use cases like synchronizing a wiki between devices at the tiddler level; ensuring that changes in one tab aren’t overwritten by an instance of the same wiki open in another tab or device; or being offline for an extended period and syncing any changes made to other devices once online.

The driver for this design choice is that I need offline access to my wikis at all times, so online hosting solutions are a non-starter for me. Were online hosting an option, I would likely choose TiddlyPWA.

Currently the plugin works robustly enough that I have been using it on a daily basis since February. It would need a rewrite to make it usable for a wider audience, which I will admit that I am currently struggling to find motivation for given that it already works very well for my very specific use cases, and I am not sure that many others would find this useful.

PS: a useful though unplanned benefit of this approach is that you get a per tiddler history of changes in the GitHub repo and it would theoretically be possible to revert a wiki back to a specific time in the history of changes.

3 Likes
#24

Looking forward to see this however;

I was hoping MWS Multi-wiki server could do this to an internet hosted SQL Server, that can be placed on any apatche/cpanel host.

1 Like
#25

You would also need to deploy MWS TiddlyWiki on node.js on the server to interact with the SQL database.

1 Like
#26

Wow @saqimtiaz - that’s almost my exact need as well and I haven’t been able to solve it. I’m offline such a big chunk of the time as I move between work and home (long train commute) and trying to keep things all in sync is very difficult. If that ever became public I would jump all over it. I’m not very technical, but do have a github that I’ve just used for TiddlyWiki page hosting etc.

3 Likes
#27

Do the actual contents of each commit, and the diff from the previous commit, look sensible? And does it matter? And what would the commit message be??

They’re pretty atomic. I save the Wiki every time TiddlyDesktop loses key window status (so if I’m switching apps), which generally is a good time to commit the wiki and push it. My commit message is “auto commit from ___”, which fills in what device (phone / computer) did the commit. I forgot to mention that this script is also auto-git pulling at a similar interval.

Does this Git-based solution allow dealing with sync conflicts?

It does, but only superficially. I also have TiddlyDesktop quit when I sleep my machine (and relaunch when I wake it), because I’ve had issues like this:

  1. Edit something on my phone
  2. Go back to my computer, where TiddlyDesktop is still running with contents prior to (1)
  3. Edit the wiki, which saves, and clobbers the phone changes from (1)
1 Like
#28

I understand this is still in development. But once ready, it sounds like it could be ideal as a multi-user solution (for cases where a wiki is shared among familiar trusted users who cannot connect on the same local network). Is that right? If it requires all devices to interact over the same GitHub account, that could be mildly inconvenient (if at least one user also wants to do other things with GitHub that would not be shared), but not necessarily a deal-breaker.

I’d love to see multi-user experiments of as many kinds as possible. And any per-tiddler sync solution is going to have a great advantage (in minimizing potential conflicts) over single-html solutions. Reduced bandwidth demand is also very attractive!

2 Likes
#29

I imagine it doesn’t. A GitHub repo allows you to name collaborators. Each user would need a GH id, and be given permission to the repository, but that should do it.

2 Likes
#30

I knew someone would make the leap to multi-user setups :slight_smile:
I am actually unsure how well that would work. The present sync logic assumes that multiple devices are not editing the same wiki in parallel, but rather one after another as is the case when a user switches between devices. There are certain limitations to what the GitHub API allows in terms of retrieving data and how often one can talk to the API, so while improvements in this area might be possible, it isn’t something I have investigated at present.

2 Likes
#31

One of the challenges here is that if you switch from device A to device B when moving from the office to the train, you would need to open the wiki on the device B to let it sync before going offline. Otherwise you would risk editing having edited a tiddler on device A and then without that change propagating to device B, editing the same tiddler again and losing earlier changes from device A.

I would be happy to discuss this further privately or in another thread to understand if Harmony would be useful for you to test out.

1 Like
#32

That’s very informative thanks @session. I’ll be sure to check out /siteinfo when exploring Hubzilla instances. Hopefully Hubzilla is still actively developed/maintained that it’s around for the long term!

#33

As the others have said, this sounds exciting especially if it enables multiuser edits. Please share in case you every decide to publish this plugin!

#34

As far as I know, TiddlyPWA only requires online access the first time you access it from a given browser/device. After that it can always be accessed offline. It has a service worker for offline access of the app wiki and a local copy of the tiddlers are stored in the browser’s indexdb. Are your offline requirements even more strict than that?

1 Like
#35

Past experience has shown me that I cannot trust browsers to persist data over longer periods of time, which is a deal breaker for my particular needs. In addition to that the ability to easily move the wiki to another device while offline has come in handy in the past as well. For others with a bit more flexibility in this regard, TiddlyPWA would indeed be an option for offline usage as well.

2 Likes
#36

Neat, I have also been considering using GitHub as the “tiddler store backup” for my sync cases.

I’m also still exploring Yjs for real-time sync. With Yjs local-first apps, you would need the internet to load the initial “wiki/page/room”, but then all edits once there are “non-deterministically conflict free” with all other copies of that doc ID. While there could be “have a REALLY old copy of a tiddler open somewhere, and edit that after a bunch of newer edits elsewhere” race-condition, but at least you would have a clean backup-history if it was also persisted in plain text to GitHub.

The final sync to github could be at a slower rate, if you are saving all live edits to browser indexeddb AND broadcasting to any listening wikis (local or remote networks). Consider real-time multi-user mouse-cursor tracking, like on https://kinopio.club - where certain things like mouse position, or tiddler-card-being-dragged xyz-position, should be streamed in realtime to other clients, but should NOT be persisted to an edit-history/backup.

Great conversation. Would love to check out Harmony when you feel ready.

1 Like