Solutions for syncing the same TiddlyWiki across many devices?

#1

Dear Talk TiddlyWiki,

I am a fairly new user of TiddlyWiki. So far, I’m going through the excellent Grok TiddlyWiki text book, and have set up an instance of the also excellent TiddlyPWA by @valpackett to tinker on Glitch.

Before diving deeper into TiddlyWiki, I’d like to understand the current big picture of ways to read/write the same TiddlyWiki across all my devices.

Ideally the set up should be:

  • able to sync changes from different devices;
  • less brittle/more reliable with fewer moving parts that might break down;
  • able to handle the addition and removal of lots of plugins, themes, etc. without breaking.

Here are the solutions I’ve found so far:

My questions are:

  1. Are there any options known to be especially problematic (i.e. unmaintained or high risk of data loss/corruption) that I should avoid???
  2. Any other notable methods other than those listed above?

Thank you in advance! :bowing_woman:

P.S. I use Mozilla Firefox in case that matters.

2 Likes
#2

I want to put in a quick vote for TiddlyHost. Unless there is some reason you need not to be working with an online server, it’s simply a beautifully-developed hosting solution. And I love the fact that all my wikis are online (even beyond the ones that I design for my students); I can be on the road visiting a friend, or stepping into a co-worker’s office, and realize I want to share an example, and it’s just right there. Make public and private sites, with or without encryption. Also check out their external-core option that reduces storage and bandwidth demands.

Advantage #1: The online version just is the authoritative one. (Occasionally I used to get “stung” when — in going back and forth between home and work — I accidentally interacted with a stale tab (one that had loaded an earlier version, and that I had left open by accident), and overwrote work that I had done at the other location. TiddlyHost now offers some protection against that, in the form of a warning, when you try to save new work done on a stale instance, that there’s a more recent version on the server. So then you can load the online wiki, drag over any tiddlers that you’ve just created/modified, and you’re back on track.)

Advantage #2: Trouble-free backup history: If you support them with an optional modest monthly subscription, you’ll have a no-fuss history of prior saves from which you can restore. It’s a fantastic way of outsourcing the clutter of backups.

By all means, set up a routine download to a local machine as well. As a daily user over many years, I have very very rarely noticed an outage to the server (as with any server), but TiddlyHost has been admirably well maintained and @simon has also been responsive here at the forum.

5 Likes
#3

TidGi mobile app on Android device can sync with TidGi-Desktop on same wifi, and is tested to work with up to 30k tiddlers or 2GB of wiki size.

TidGi-Desktop itself sync with other PC’s TidGi-Desktop using github.

1 Like
#4

I don’t see Syncthing on your list. This allows you to easily carry a local, offline copy of TW.

3 Likes
#5

I support @Springer’s argument for tiddlyhost. I do however feel we could add a stronger interface for this kind of thing.

One idea would be for on loading a wiki from tiddlyhost or other single file host, check the wiki has not being opened (a check a tiddler),

  • if already open (elsewhere) give some details as to where it is open, so you can go there and save. Click to reload and try again.
  • If not already open, present a dialogue to open read only or read update. If for update save back to host with the checkout details (device/browser and user?).
    • Hide/stop auto save and rely only on local storage to save changes.
    • On save to host check it back in, and reload.

I think I have all I need to implement this, but previous attempts have led me down multiple rabbit holes, such that I never created a finished result.

  • The lesson I learned is we need to make this as simple as possible.
1 Like
#6

I’m confused about how this would be possible. When I open a tiddlyhost file in a browser tab, loading it is a simple matter of retrieving the html file. Later, I can use my browser to interact again with the server (so long as I’m logged in) to save an updated version. It seems the tiddlyhost server has no way of knowing whether there’s a stale copy still lingering in a browser tab from a prior load.

For tiddlyhost to have a way to “know” this, it would need something like the ability to “ping” files that are loaded in a “live” browser window. But this kind of server-originated handshake seems quite contrary to the self-contained html concept, no?

The most extreme version of this challenge: any public TiddlyHost site can be loaded as a public site before the owner logs into tiddlyhost. If I load and make changes to a public site that I own and have loaded this way, then a request to save my updated version will succeed exactly if I am logged in by that time (of attempting to save). So, knowing whether there is a likely copy “out there” (potentially with unsaved changes) therefore means trying to keep “tabs” :wink: on even the instances when my public file has been loaded, without login, over the web.

Are you imagining some way for private wikis to “phone home” with some kind of specially-encoded “I’m still hanging out here” ping, sent exactly when anything is modified but not saved (and then sent every 10 minutes that the page is still loaded and “live” without being saved)? Otherwise I see nothing that could help TiddlyHost recognize the difference between a file loaded, modified, saved, and closed and a file loaded, modified, saved, but modified again (without closing) before laptop is closed, or desktop at work dropping into offline sleep mode.

If so, it sounds like the initial “hey I’m a loaded instance hanging out here with new content” signal would be almost exactly the same as an auto-save, except that instead of saving, the server just gets a “there’s something here that could be saved” communication… But then why not just insist on using auto-save (with the benefit of a backup history)?

Or do you know some trick that I’m not even imagining here? :thinking:

1 Like
#7

My idea is to build a layer in tiddlywiki that disables save unless the wiki is available then save back to tiddlyhost, that the current device/browser has it. Other device browsers will be discouraged from saving if another has it, and can see which browser device has it.

This is about one user multiple devices. You have to check back in to make it available on another device. The user id could be added on top, for multiuser checkout.

1 Like
#8

This is because you only use saver. With sync-adaptor, this is not a problem. It sync single tiddler to save bandwidth and prevent conflict.

That’s why I create TidGi-Mobile, instead of continue use Tiddloid.

1 Like
#9

Thank you everyone for your kind responses!!

@Springer @TW_Tones: Thanks for the comments about Tiddlyhost, I should probably give it a try!

@Mark_S Thanks for mentioning Syncthing. Conceptually it seems to be the same as the first option I listed, i.e. save to HTML file, and sync the HTML using a service, where instead of using Nextcloud I’d use Syncthing? Or does Syncthing provide something unique compared to other generic file-syncing solutions?

@linonetwo TIL TidGi-Desktop! Thank you.

#10

Syncthing is much less complex than using Nextcloud, and is free, as in beer, which I’ve never actually encountered, BTW. You don’t have to set up a domain, or rent space on a server, or worry about all the layers of security that you would for setting up your own cloud.

2 Likes
#11

Gosh, since multi-talking and distraction is the reason my everyday TiddlyWiki browser tabs sometimes get left open (rather than closed at the end of the day) at work, I don’t think adding a required step of “checking back in” for each wiki before going home would avoid much nuisance. It may even create more nuisance, since I could arrive home and realize that a wiki (or two or three) would be still “checked out” at work (even when I am confident that unsaved work is minor or nonexistent), so now I can’t save from home at all. :grimacing: If on the other hand the “protection” could be reduced to a heads-up confirmation dialogue (“Hey, this project was opened by Springer from IP xxx.xx.xxx.x today at 12:59, last saved at 14:16, but not subsequently closed out. Begin editing session with TiddlyHost’s last-saved (14:16) version?”) then that would just more information, which is not a problem.

This possibility, on the other hand, is golden. Collaborative projects are the frontier I’m most excited about, and needing to “check out” (in library sense) to do specific tasks, and then return it to “circulation” (for other users) would be a game-changer.

For what it’s worth, the fediverse / hubzilla solution being explored by @session looks promising at the moment for multi-user access (but perhaps without much per-tiddler independence for simultaneous editing). I’m sure y’all will be getting reports soon about those developments.

1 Like
#12

I think there was a mention that it was possible to use Tailscale to make your own node.js instance available across the net. Never looked into it to see how feasible that was. Just throwing this out there for completeness.

1 Like
#13

Hubzilla works also with WebDAV - which makes it easy to sync any kind of file from all kind of devices… e.g. there is RCX for android RCX - Rclone for Android | F-Droid - Free and Open Source Android App Repository

and FolderSync with automatic sync function https://foldersync.io/

Rclone for the Desktop https://rclone.org/

there are many other apps which work with WebDAV

Hubzilla also has a function which is called “nomadic identity” - this function can clone all your files to an other server where you can have a clone of your Hubzilla Account

2 Likes
#14

@Springer there are way to extend these ideas to make them simpler to use. For example closing your browser will prompt each unsaved tab and if saving is equal to check in it becomes available to another device.

  • If this is all you have to do before you leave it’s simple

Additional effort could be made to allow changes on a copy not checked out, and save changes in a seperate file. When you have the wiki you could import and review changes.

Another trick for a single user access multiple devices is to carry the wiki with you. On a mobile and at home or work access the wiki on the mobile or via a USB or wifi file connection.

1 Like
#15

:rofl:

I’m not offended, but I must say it’s clear you’re on a different planet from me (a more stable one, surely) in terms of the rhythm of starting and stopping work on a project.

(For me, tabs are always open for multitasking, I’m catching moments here and there to bring this and that wiki up to date in between different people stopping by my office unannounced, and then I’m often running out exactly when I realize I’m already late for a thing. Home isn’t much different, with interruptions from human and non-human family members. Closing all TiddlyWiki tabs before I’m switching location — let alone quitting the browser! — needs to be aspirational, not mandatory.)

I do have great respect for people who have clear-cut intervals of uninterrupted and decisive work time, and who actually have an end-of-shift routine. Great respect. Still, I’ve accepted that I’m not one of them. :grin:

5 Likes
#16

I understand where you are coming from, but at the same time I learned that the end of the day is distinctly different. In Tiddlywiki classic I had to save to USB or Dropbox. It may take seconds, but it always felt excruciatingly slow.

  • I had to lock my laptop away, turn off power and clear my desk.

However, I believe there are ways to help you here, if you really can’t add another step to your departure regime, one I have researched is to make a timed autosave/checkin. You could also privilege one device to not need checkout/in but it can’t save if another holds the checkout.

  • I have always felt there would be an advantage if we had a form of logout on wikis, because various processes belong in the logout, as do somethings things in the startup.
  • I shut down my computer at the end of day, and especially Fridays, due to the environmental impact of using power when I am not there.

However for you the idea of autosave to a mobile device which is always with you may be the best approach.

  • If you already have another tab open from last time, at home or work, The trick then is to reload the wiki on any other device before making changes.
    • I have explored some ways to do this automatically. For example if the last change was saved and occurred more than 30mins ago reload the wiki when the tab gets focus.
  • Tiddlyhost can take the place of the wiki on a mobile, as long as you have internet access most of the time, and if not local storage can help.
1 Like
#17

To be clear, I’m not here looking for solutions or advice for saving across platforms/locations! I’m quite pleased with TiddlyHost (often using auto-save), especially now that TiddlyHost can alert me if/when I do trip up and try to save new changes from a stale instance. (And now that TiddlyHost. also keeps a great backup archive!)

My comments above were simply emphasizing that while I’m not personally a likely customer for a single-author check-in check-out approach, I am very excited about the prospect of getting a multi-user solution at TiddlyHost. For that kind of special project, even I would bring myself to work deliberately within check-in-check-out parameters. :upside_down_face:

1 Like
#18

I use git and have a backup automatically run every minute or so on my computer. On my phone, I have a shortcut that runs git sync commands in the background (and pushes whenever I exit Quine, my mobile TiddlyWiki client). This works well (and has the added benefit that you can “roll back” your Wiki easily).

2 Likes
#19

Resilio Sync is the original mutable torrents tool, and by far the most performant of its class. You’ll still face race conditions during syncing (not a big deal by hand, but it can matter if you’ve more automation in your wiki construction workflow). I’ve run into trouble synchronizing multiple old Bob instances, but theoretically it should work.

1 Like
#20

I use git and have a backup automatically run every minute or so on my computer

Interesting! So do each save operation - automatic or initiated by you - create a commit to a target Git repository? If so:

  1. Do the actual contents of each commit, and the diff from the previous commit, look sensible? And does it matter? And what would the commit message be??
  2. Does this Git-based solution allow dealing with sync conflicts?
1 Like
Advice on how to host/backup/store my Wiki