Solutions for syncing the same TiddlyWiki across many devices?

#15

:rofl:

I’m not offended, but I must say it’s clear you’re on a different planet from me (a more stable one, surely) in terms of the rhythm of starting and stopping work on a project.

(For me, tabs are always open for multitasking, I’m catching moments here and there to bring this and that wiki up to date in between different people stopping by my office unannounced, and then I’m often running out exactly when I realize I’m already late for a thing. Home isn’t much different, with interruptions from human and non-human family members. Closing all TiddlyWiki tabs before I’m switching location — let alone quitting the browser! — needs to be aspirational, not mandatory.)

I do have great respect for people who have clear-cut intervals of uninterrupted and decisive work time, and who actually have an end-of-shift routine. Great respect. Still, I’ve accepted that I’m not one of them. :grin:

5 Likes
#16

I understand where you are coming from, but at the same time I learned that the end of the day is distinctly different. In Tiddlywiki classic I had to save to USB or Dropbox. It may take seconds, but it always felt excruciatingly slow.

  • I had to lock my laptop away, turn off power and clear my desk.

However, I believe there are ways to help you here, if you really can’t add another step to your departure regime, one I have researched is to make a timed autosave/checkin. You could also privilege one device to not need checkout/in but it can’t save if another holds the checkout.

  • I have always felt there would be an advantage if we had a form of logout on wikis, because various processes belong in the logout, as do somethings things in the startup.
  • I shut down my computer at the end of day, and especially Fridays, due to the environmental impact of using power when I am not there.

However for you the idea of autosave to a mobile device which is always with you may be the best approach.

  • If you already have another tab open from last time, at home or work, The trick then is to reload the wiki on any other device before making changes.
    • I have explored some ways to do this automatically. For example if the last change was saved and occurred more than 30mins ago reload the wiki when the tab gets focus.
  • Tiddlyhost can take the place of the wiki on a mobile, as long as you have internet access most of the time, and if not local storage can help.
1 Like
#17

To be clear, I’m not here looking for solutions or advice for saving across platforms/locations! I’m quite pleased with TiddlyHost (often using auto-save), especially now that TiddlyHost can alert me if/when I do trip up and try to save new changes from a stale instance. (And now that TiddlyHost. also keeps a great backup archive!)

My comments above were simply emphasizing that while I’m not personally a likely customer for a single-author check-in check-out approach, I am very excited about the prospect of getting a multi-user solution at TiddlyHost. For that kind of special project, even I would bring myself to work deliberately within check-in-check-out parameters. :upside_down_face:

1 Like
#18

I use git and have a backup automatically run every minute or so on my computer. On my phone, I have a shortcut that runs git sync commands in the background (and pushes whenever I exit Quine, my mobile TiddlyWiki client). This works well (and has the added benefit that you can “roll back” your Wiki easily).

1 Like
#19

Resilio Sync is the original mutable torrents tool, and by far the most performant of its class. You’ll still face race conditions during syncing (not a big deal by hand, but it can matter if you’ve more automation in your wiki construction workflow). I’ve run into trouble synchronizing multiple old Bob instances, but theoretically it should work.

1 Like
#20

I use git and have a backup automatically run every minute or so on my computer

Interesting! So do each save operation - automatic or initiated by you - create a commit to a target Git repository? If so:

  1. Do the actual contents of each commit, and the diff from the previous commit, look sensible? And does it matter? And what would the commit message be??
  2. Does this Git-based solution allow dealing with sync conflicts?
1 Like
Advice on how to host/backup/store my Wiki
#21

Thanks @session. I’ve heard about Hubzilla but didn’t know it comes with WebDAV. I can see how that might be another way to sync TiddlyWiki files, in addition to other tools that have been mentioned.

But I can’t figure out how to find a reliable Hubzilla instance to sign up for, or even find out the features and limitations (e.g. storage quota, etc.) of instances. Or am I expected to host my own Hubzilla instance and federate with others???

#22

finding a reliable instance is always the problem in the Fedivese :slight_smile: if you want to do something serious you actually really should considering self hosting an instance… Hubzilla does make it quid easy…

But for the start there are quid some public instances you can use for a try out.

https://talk.tiddlywiki.org/t/option-for-hosting-tiddlywikis-with-an-integrated-access-control-permission-system-and-fediverse-connectivity

under /siteinfo you find some information about installed addons - yes storage can be limited - but mostly there is plenty storage granted

2 Likes
#23

For the last couple of months, I have been using a set up in which I have copies of a single file TiddlyWiki for notes on my desktop, tablet and phone, and the three copies of the same wiki synchronize with each other on a tiddler level.

The sync is via an in-development plugin (Harmony) that synchronizes tiddlers to a private GitHub repo which essentially serves as a free sync server, with the user not requiring any knowledge of git. Any changes to a wiki configured to sync to a given repo are automatically propagated to other wikis syncing to the same repo.

Essentially this is the same model used by apps such as Evernote, where you have different instances of the app on different devices that can all work separately and offline, but they also sync with each other via a server. Here, the private GitHub repo plays the role of a no configuration necessary free server, and you retain full control over your data. Creating a private GitHub repo is very easy even for non-technical users, whereas setting up your own server securely can be a complex undertaking.

This setup allows for use cases like synchronizing a wiki between devices at the tiddler level; ensuring that changes in one tab aren’t overwritten by an instance of the same wiki open in another tab or device; or being offline for an extended period and syncing any changes made to other devices once online.

The driver for this design choice is that I need offline access to my wikis at all times, so online hosting solutions are a non-starter for me. Were online hosting an option, I would likely choose TiddlyPWA.

Currently the plugin works robustly enough that I have been using it on a daily basis since February. It would need a rewrite to make it usable for a wider audience, which I will admit that I am currently struggling to find motivation for given that it already works very well for my very specific use cases, and I am not sure that many others would find this useful.

PS: a useful though unplanned benefit of this approach is that you get a per tiddler history of changes in the GitHub repo and it would theoretically be possible to revert a wiki back to a specific time in the history of changes.

3 Likes
#24

Looking forward to see this however;

I was hoping MWS Multi-wiki server could do this to an internet hosted SQL Server, that can be placed on any apatche/cpanel host.

1 Like
#25

You would also need to deploy MWS TiddlyWiki on node.js on the server to interact with the SQL database.

1 Like
#26

Wow @saqimtiaz - that’s almost my exact need as well and I haven’t been able to solve it. I’m offline such a big chunk of the time as I move between work and home (long train commute) and trying to keep things all in sync is very difficult. If that ever became public I would jump all over it. I’m not very technical, but do have a github that I’ve just used for TiddlyWiki page hosting etc.

3 Likes
#27

Do the actual contents of each commit, and the diff from the previous commit, look sensible? And does it matter? And what would the commit message be??

They’re pretty atomic. I save the Wiki every time TiddlyDesktop loses key window status (so if I’m switching apps), which generally is a good time to commit the wiki and push it. My commit message is “auto commit from ___”, which fills in what device (phone / computer) did the commit. I forgot to mention that this script is also auto-git pulling at a similar interval.

Does this Git-based solution allow dealing with sync conflicts?

It does, but only superficially. I also have TiddlyDesktop quit when I sleep my machine (and relaunch when I wake it), because I’ve had issues like this:

  1. Edit something on my phone
  2. Go back to my computer, where TiddlyDesktop is still running with contents prior to (1)
  3. Edit the wiki, which saves, and clobbers the phone changes from (1)
1 Like
#28

I understand this is still in development. But once ready, it sounds like it could be ideal as a multi-user solution (for cases where a wiki is shared among familiar trusted users who cannot connect on the same local network). Is that right? If it requires all devices to interact over the same GitHub account, that could be mildly inconvenient (if at least one user also wants to do other things with GitHub that would not be shared), but not necessarily a deal-breaker.

I’d love to see multi-user experiments of as many kinds as possible. And any per-tiddler sync solution is going to have a great advantage (in minimizing potential conflicts) over single-html solutions. Reduced bandwidth demand is also very attractive!

2 Likes
#29

I imagine it doesn’t. A GitHub repo allows you to name collaborators. Each user would need a GH id, and be given permission to the repository, but that should do it.

2 Likes
#30

I knew someone would make the leap to multi-user setups :slight_smile:
I am actually unsure how well that would work. The present sync logic assumes that multiple devices are not editing the same wiki in parallel, but rather one after another as is the case when a user switches between devices. There are certain limitations to what the GitHub API allows in terms of retrieving data and how often one can talk to the API, so while improvements in this area might be possible, it isn’t something I have investigated at present.

2 Likes
#31

One of the challenges here is that if you switch from device A to device B when moving from the office to the train, you would need to open the wiki on the device B to let it sync before going offline. Otherwise you would risk editing having edited a tiddler on device A and then without that change propagating to device B, editing the same tiddler again and losing earlier changes from device A.

I would be happy to discuss this further privately or in another thread to understand if Harmony would be useful for you to test out.

1 Like
#32

That’s very informative thanks @session. I’ll be sure to check out /siteinfo when exploring Hubzilla instances. Hopefully Hubzilla is still actively developed/maintained that it’s around for the long term!

#33

As the others have said, this sounds exciting especially if it enables multiuser edits. Please share in case you every decide to publish this plugin!

#34

As far as I know, TiddlyPWA only requires online access the first time you access it from a given browser/device. After that it can always be accessed offline. It has a service worker for offline access of the app wiki and a local copy of the tiddlers are stored in the browser’s indexdb. Are your offline requirements even more strict than that?

1 Like