TiddlyPWA: Secure offline storage and cross-device synchronization solution

Look at the .env file in the Glitch editor, there should be a command to hash the password using the terminal.

2 Likes

Yes this works. So one can go directly to the Glitch link in my earlier post, open the

.env file (left sidebar) and terminal (bottom) and paste

.deno/bin/deno run https://codeberg.org/valpackett/tiddlypwa/raw/branch/release/server/hash-admin-password.ts

Into the terminal to get new values for hash and salt and enter them in the

.env.

Nice work. You’ve implemented a lot of nice features here!

Some notes on my current understanding.

Browser storage

Comparison to the browser storage plugin:

  1. browser storage plugin is implemented using local storage which allows very small amounts of storage ~5MB. This quota is shared for everything within the same domain, so having multiple TW instances on the same domain can make it very tight. TiddlyPWA uses indexdb and the storage quota is much, much higher depending on how much your device has available.
  2. the most recent version of the browser storage plugin got eviction protection permission feature added. TiddlyPWA has this also.
  3. browser storage plugin does not broadcast changes to other tabs in the same browser (requires reloading the page to get the changes from other tabs). TiddlyPWA does broadcast changes…very nice!
  4. browser storage plugin can store and load all tiddlers, even plugins. Even plugins containing javascript. TiddlyPWA is implemented as a syncadapter (probably the most widely used syncadapters are the nodejs web adapter and filesystem adapters…these do not handle plugins). As such, TiddlyPWA does not store plugins in indexdb browser storage. Adding/changing plugins requires saving a new “app wiki”. TiddlyPWA has an interface for saving a new app wiki to a sync server, but I haven’t quite figured out the process yet.

Considering just the browser storage feature of TiddlyPWA, it comes out on top for every feature except plugins.

A random thought I just had. The browser storage plugin is compatible with other syncadapters and savers. Theoretically you could install this plugin in with TiddlyPWA and configure it to only store plugins. That way you get the best features of both the browser storage plugin and the indexdb syncer.

Sync server

  1. I was able to get the glitch version working and it was pretty smooth. I’m just experimenting so I went with the “no-account, will expire after 5 days” version.
  2. I was able to get the local version working. It was plain HTTP, I didn’t want to mess with encryption. I’m not sure what feature I miss out here. It worked enough to play around with.
  3. The sync server has an admin interface. This wasn’t apparent to me at first. Nothing can be synced to the server until a new slot is created using the admin interface (I think).

Encryption and passwords

  1. The password used when launching the sync server is different from the password for syncing a wiki. The launch password is only for accessing the sync server admin interface. This is obvious once you use it more, but I didn’t figure it out right away.
  2. Which password to use where can be confusing. I still don’t have it straight and will have to play around more. Is the same password used for the local storage sync as for the remote sync? Or only if you use the same password…otherwise they will be different.
  3. There are quite a few different workflows and the exact sequence of steps to use them are not yet clear to me.
  4. From what I can tell the actual passwords are not stored anywhere by TiddlyPWA. This is good for security but that means you’d better not lose or forget your password. Otherwise your data will be irrecoverable. Do I have that right?
5 Likes

Wow, I wasn’t aware of that at all :eyes: Interesting. It should theoretically be possible to do the same thing it does. However, not doing that has the advantage of guaranteed easy recovery from installing a bad plugin that screws everything up. Note that we can’t quite use safe mode for such a recovery because—oops—disabling all plugins means disabling TiddlyPWA as one of them!!

I’m glad someone noticed this!! :3

If you use sync itself over plain HTTP (localhost or over a VPN, hopefully not over the public internet) while the app wiki itself is running e.g. from tiddly.packett.cool (or any HTTPS server), you don’t miss out on anything.

If you tried to load an app wiki from a sync server over plain unencrypted HTTP… nothing would work :slight_smile: Features restricted to secure contexts in browsers include even the Web Crypto API.

“local storage sync” is not a concept that exists. The wiki password isn’t used for sync, not directly anyway. The password basically only ever gets turned into encryption keys for storage. That encrypted data can be synced to sync servers; which wiki “slot” to sync with is determined by the token. To avoid mixing up wikis encrypted with different keys (which would just explode), the password-derived keys are also used to prove that it’s the same wiki.

Yeah, it’s quite difficult to document something so flexible; I’ve tried to promote an easy “happy path” with hosting the wiki on the sync server, but evidently there are still ways to improve the documentation. More guidance on what to do after the server is hosted is definitely something to add.

Yes. Thanks for bringing that up to my attention; I made the encryption doc all about the paranoid nerd stuff and forgot to put something as basic as an all-caps “DON’T LOSE YOUR PASSWORD” in the beginning. To be fair to myself, I’ve also recommended doing “export all” type backups everywhere e.g. in GettingStarted :smiley:

4 Likes

Released 0.1.3, improved documentation including more guidance towards hosting wikis on sync servers and basic password info (and even a Caddy+Tailscale HTTPS mention). In the plugin itself, a bunch of alerts have been made less annoying, and some improvements have been made to set-up processes. UpdateInPlace as always.

5 Likes

Hi,

i am really not techincal in any away , so forgive my ignorance, but what does this Plugin do for the average pedestrian like myself :slight_smile: ?

@paulgilbert2000 from what I gather, it will let you use a tiddlywiki from a browser, and:

  • store the data locally, in the machine where you are using the browser; so saving will feel really quick
  • sync the data seamlessly with a server/“the cloud”

but the really nice thing is that it will allow you to check the same tiddlywiki from another browser/device, and have it synchronized automatically; you can do edits on both tiddlywikis at the same time, and won’t lose data [1].

1: except when you create an edition conflict, by modifying the same tiddler at the same time; in that case, the most recent change overwrites the previous one, I think.

6 Likes

Correct, for now the only conflict resolution is “newer version is picked client-side”; in the future I’ll try to add a way to view the older one and merge them…

5 Likes

Spent another hour tonight on a non-work machine to see if I could figure this out, still stuck, have multiple accounts but don’t understand the steps to set up a wiki and have access to that wiki on both my laptop and iPhone. If anybody who’se figured this out could spin up a screen-record video or something for how to do this - “dummies version” - it’d be greatly appreciated :slight_smile: @Mohammad - you’re great at this kind of thing, have you played with this?

Hi @stobot
Not yet, but I would share once I had time.

@valpackett

  1. It would be nice if non-technical people can follow your instruction for setting up TiddlyPWA.

Side note (ignore if not relevant): Can you suggest a better name for TiddlyPWA? like TW5-PWA or something better? I (myself) prefer not to use Tiddly!

1 Like

What is the difference between this approach and having the Tiddlywiki on Tiddlyhost with password (encryption) enabled?

(I could see that it is faster as the saving happens locally and syncing is decoupled from the save to the local files)

if you open the same wiki on two devices at the same time, and then:

  • edit on one device, then save
  • edit on another device, then save

saving on the second device will destroy all changes done on the first device

with TiddlyPWA, what happens is:

  • edit on the first device, then save
    • TiddlyPWA on the first device sends the changes to its sync point on the cloud
    • TiddlyPWA on the second device notices the changes (without having to reload the wiki) and downloads them
  • edit on the second device, then save, will keep the changes made on both devices.
3 Likes

Exactly: you can easily work on different tiddlers on various devices including working offline and everything would be reconciled smoothly when they all sync. Considering a whole-file saving thing would give me tremendous paranoia about losing changes like that :smiley:

(However editing the same tiddler on two devices with TiddlyPWA is… not quite google docs :slight_smile: You technically can work on the same draft in real time on both devices but you have to be very careful to wait when switching devices until you see the changes reflected. As mentioned in the docs, there is no manual/smart conflict resolution yet (which is probably the largest reason this is version 0.1.x) – conflicts are resolved just by “newest whole version wins” on the device side.)

3 Likes

amazing stuff @valpackett !

is there currently a way of handling modified plugins (i.e. plugins where the user has modified plugin files)? i remember seeing somewhere in the docs the plugins are served separately to the user-modified tiddlers? or maybe i didn’t import in the right way for my user-modified versions of command palette and section editor to work? thanks again for this great tool!

PS iphone safari users might benefit from setting font-size to 16px to reduce auto-zooming (more details at makiaea ); users of ios/macos lockdown mode will need to disable lockdown mode for the specific sites https://tiddly.packett.cool and glitch.com for them to work

1 Like

thanks!

Yes, if you want to customize JavaScript shadow tiddlers, you have to save the app wiki to apply the changes as if you were just installing a plugin – except for now you won’t get a notification that you need to do that, and it will be redundantly stored as a regular tiddler as well, I should try to improve that.

Things that will be saved into the app wiki are defined by the $:/plugins/valpackett/tiddlypwa/app-filter filter and you can see them in the ‘which tiddlers are saved in the app wiki’ thingy, as you can see such overrides do appear there:

1 Like

Thanks for clarifying this for me. :+1:

Thank you too. For the explanation and for the wonderful tiddlywiki enhancement.

TWPWA was indeed the last stepping stone of making my tiddlywiki experience complete for my use case.

I have in addition a maybe offtopic question. I dont know if its related to TiddlywikiPWA or my mobile browser.

I am using Mull (hardened fennec/firefox mobile browser) with Ublock origin enabled and surfing the tiddlywikiPWA page gives me following error:

Welcome to TiddlyPWA

Oops…

WebAssembly is unavailable, we cannot unlock the wiki without it.

I am only experiencing this in Mull and not in firefox focus or duckduckgo browser on android.
I dont know if webassembly has to be distributed with mull or if mull has an issue using the android webassembly (as, I suppose, every other mobile browser).

therefore I wanted to ask if this issue has something to do with Mull settings or elsewhere. Maybe you or someone else have a clue of where to start investigating this issue.

PS: it is not crucial for me to work with TWPWA in Mull however I never had problems with mull and thought this could be a bug if PWA?

Thanks in advance.

B.R
TTT

Hi, I’m giving a try to this, looks good so far :+1:

I’ve created a docker image to self host it on my server : Docker

1 Like

Can I add it to a nodejs wiki? (for example, TidGi desktop app), and open that wiki on a LAN mobile phone, so it can replace Tiddloid completely?

Will I need additional setup on the nodejs tiddlywiki server? Or just installing the plugin and it works?

Also hope it won’t prevent other sync-adaptor working, because I’ve write a IPC-sync-adaptor, so on a PC it is faster to load content from nodejs to election’s browser.

1 Like

No. TiddlyPWA is itself a syncadaptor that ‘syncs’ to IndexedDB in the browser (encrypting the tiddlers!!) and then separately syncs that using a custom protocol to a custom server. It completely replaces the nodejs server mode.

2 Likes