TiddlyPWA: Secure offline storage and cross-device synchronization solution

thanks!

Yes, if you want to customize JavaScript shadow tiddlers, you have to save the app wiki to apply the changes as if you were just installing a plugin – except for now you won’t get a notification that you need to do that, and it will be redundantly stored as a regular tiddler as well, I should try to improve that.

Things that will be saved into the app wiki are defined by the $:/plugins/valpackett/tiddlypwa/app-filter filter and you can see them in the ‘which tiddlers are saved in the app wiki’ thingy, as you can see such overrides do appear there:

2 Likes

Thanks for clarifying this for me. :+1:

Thank you too. For the explanation and for the wonderful tiddlywiki enhancement.

TWPWA was indeed the last stepping stone of making my tiddlywiki experience complete for my use case.

I have in addition a maybe offtopic question. I dont know if its related to TiddlywikiPWA or my mobile browser.

I am using Mull (hardened fennec/firefox mobile browser) with Ublock origin enabled and surfing the tiddlywikiPWA page gives me following error:

Welcome to TiddlyPWA

Oops…

WebAssembly is unavailable, we cannot unlock the wiki without it.

I am only experiencing this in Mull and not in firefox focus or duckduckgo browser on android.
I dont know if webassembly has to be distributed with mull or if mull has an issue using the android webassembly (as, I suppose, every other mobile browser).

therefore I wanted to ask if this issue has something to do with Mull settings or elsewhere. Maybe you or someone else have a clue of where to start investigating this issue.

PS: it is not crucial for me to work with TWPWA in Mull however I never had problems with mull and thought this could be a bug if PWA?

Thanks in advance.

B.R
TTT

Hi, I’m giving a try to this, looks good so far :+1:

I’ve created a docker image to self host it on my server : Docker

1 Like

Can I add it to a nodejs wiki? (for example, TidGi desktop app), and open that wiki on a LAN mobile phone, so it can replace Tiddloid completely?

Will I need additional setup on the nodejs tiddlywiki server? Or just installing the plugin and it works?

Also hope it won’t prevent other sync-adaptor working, because I’ve write a IPC-sync-adaptor, so on a PC it is faster to load content from nodejs to election’s browser.

1 Like

No. TiddlyPWA is itself a syncadaptor that ‘syncs’ to IndexedDB in the browser (encrypting the tiddlers!!) and then separately syncs that using a custom protocol to a custom server. It completely replaces the nodejs server mode.

2 Likes

IMPORTANT: everyone please update to 0.1.4! This version disables lazy-loading, which was (of course…) causing more problems than anticipated: for example tiddlers weren’t fully loaded when making exports/backups, leaving those without the actual data. Sorry about that :confused:

https://tiddly.packett.cool/#UpdateInPlace

There’s also a server update in this version: lowering memory usage on big syncs. Instructions on updating the server have been added to the hosting docs.

7 Likes

Is it safe to upgrade the core to 5.3.1?

1 Like

Released 0.2.0:

  • added an always-visible sync indicator in the toolbar #3 (oop, it looks a bit funny when spinning, I’ll replace the icon in the next update)
  • now saving while a long slow sync is going on will re-trigger a sync after that first sync is done, so you wouldn’t end up with an unsynced write if you quickly opened the wiki and made a change while your Glitch-hosted server was still waking up (something I’ve noticed myself but no one reported, heh)
  • fixed importing wikis with plugins / generally “content tiddlers not being saved after a plugin/module has been imported” #28
  • added warning when a tiddler deleted from the database remains in the app file #6
  • core updated to 5.3.1 on the website (of course it was safe @danielw2904! generally we don’t expect any breakage, especially not in minor releases)
  • it is now possible to purchase a “supporter key” from my Patreon “shop” and save it to the wiki to see a cute message in the control panel :smiley: (shhh don’t tell Patreon staff but this is basically a trick to allow (something morally equivalent to) one-time donations, without the whole subscribe-and-unsubscribe dance)

As always, update using https://tiddly.packett.cool/#UpdateInPlace.

Coming up next is probably a cool feature proposed by a supporter: an API for writing to the wiki from scripts/webhooks as discussed #29! And I have promised whole-server backup buttons #18 a couple months ago but it was a bit frustrating to see that SQLite doesn’t have a dump-as-SQL-text option in the library (it’s only in the shell binary) and I kinda gave up due to that but I promise I’ll come back to solving that.

7 Likes

Hi @valpackett it’s great to see these updates, thank you. TiddlyPWA is very usable, and the implementation is elegant and flexible. For a lot of users, I think it is now the most approachable way to use TiddlyWiki without sacrificing privacy.

It reminds me that we’re not currently linking to TiddlyPWA from tiddlywiki.com. We need a tiddler tagged “Saving” to appear in “GettingStarted”. I’ll attend to that when I get chance, but would welcome a PR to the tiddlywiki-com branch if you’d like to do it.

I’m also thinking about replacing the “Desktop” card in the QuickStart of “HelloThere” with one for TiddlyPWA. We’ve had a number of problems recently with anti-virus software flagging TiddlyDesktop (apparently it uses a component that is also used by some malware). Issues like that are intrinsic to delivering native software, and make developing TiddlyDesktop expensive and dispiriting. I’ll give it some more thought but would welcome any other views.

8 Likes

Hi @valpackett, I’d just like to say this is amazing work you’ve done here. I was actually in the middle of working on something like this for my own project, but had to shelve it due to day job obligations. My apologies if this is redundant question: I assume TPWA is able to handle plugins containing JavaScript, correct? Many thanks. We should definitely promote this more to new users seeking to get started with tiddlywiki quickly without much fuss. Incorporating more APIs for different backend services would also be smashing.

1 Like

Yes, of course there’s no problem with plugins. If you host the app wiki directly on a sync server (i.e. use the suggested workflow) you even get easy plugin installation! :slight_smile:

2 Likes

Do you know if TWPWA allows you to use the RawMarkup tag? I tried it and it wasn’t working, but I doubt I was setting everything up correctly on TPWA. I use that tag to pull in some custom CSS from a CDN location.

What does the browser dev console say? We don’t currently set any CSP restrictions or anything on the wiki app pages served from the sync server… (however, not a fan of pulling things from CDNs at all)

Normally you are right, but I pull in the Tailwind styles this way (not ideal, but it works) and it gives me limitless styling capability in TiddlyWiki.

There are no errors in the console. The $:/tags/RawMarkup works if I just put in a style section or something, but if I try and put in <script src="https://cdn.tailwindcss.com"></script> it doesn’t work.

This does work on TW hosted on TiddlyHost or local however.

1 Like

Ohh right. RawMarkup is used for regenerating the wiki HTML, you need to save the app wiki to apply those, but currently only modules (plugins/themes) would be included. You can override $:/plugins/valpackett/tiddlypwa/app-filter adding for example [tag[$:/tags/RawMarkup]] to the end, then your tiddler would be included when generating the new app HTML.

1 Like

I am really impressed with this solution @valpackett, I had done a lot of research on this in the past, tried to encourage a PWA version to get past the saving issues with tiddlywiki, no click and use. But my skills were not broad enough in this area. Thank you.

But my own attempts to follow the instructions get lost due to jargon or need to know information (I dont have) to set up the server.

Please tell me If I am wrong;

The architecture of a PWA is that it is hosted at a location online, once you visit that website you can elect to install a local copy on your machine. It works and saves locally out of the box.

  • Many PWA’s also allow the local copy to be synchronised online so that the same data become available if you run the same PWA on different devices or over time.

It seems to me we could make use of two PWA types

  1. PWA with server sync as described above, I will call this a “Roaming PWA”.
  2. A simpler version that can be obtained from a URL and installed locally and only used locally. Perhaps a “Local only PWA”
    2.1 This is somewhat in keeping with owning your own data that a local tiddlywiki does so well, also if you have a node server you control.

The blockage for me is understanding how to implement the first “Roaming PWA” and understand how we could handle a lot of people subscribing to out PWA and the server implications.

I would also be keen to see if we can publish a tiddlywiki at a simple URL that people could then make it their own through a local PWA and solve their saving issues. This may not need additional internet resources or demand and ongoing “subscription”.

This is so important to me, and the ongoing success of tiddlywiki, I will offer any help needed to build documentation or custom solutions for making use of this technology. But of course until I can do it myself this will be difficult.

  • A final issue to consider is users having PWA in which you can drop files and media to make use of the file uploads plugin approach, to save files externally (but local) and provide an external reference to them.
    • For example to use ones local SSD/HDD/LAN to store the media and keep the wiki light. This local data storage can be much cheaper than online storage.
    • This is not compatible with a roaming PWA, but does not depend on a server.

These solutions have the potential to go down in TiddlyWiki History as the day it became really available to everyone.

1 Like

That worked perfectly. Thanks!

Wow, the ability to add a thing to the wiki via a webhook would mean I could write an iOS shortcut which added a tiddler, put it in the share sheet, and my bookmarking wiki would suddenly be on a par with (e.g.) Pocket as far as convenience!!

1 Like

Over the last week I have been mulling over an idea for cross tab/browser/device sync to address my own use cases, where the concept involves using IndexedDB storage and Broadcast Channels. While I had tested TiddlyPWA as soon as it was announced, it was only when my search for others working with Broadcast Channels and IndexedDB in TW led me back to TiddlyPWA that I first looked at the code.

Very impressive, elegant and thorough work @valpackett!

I need to find the time to investigate how TiddlyPWA interacts with savers. The overall usage pattern that interests me personally is having copies of the same wiki locally on different machines that can save themselves using a saver, and can also sync with each other using a sync server.

Splitting out the IndexedDB syncer implementation as a separate plugin would have value for users in of itself to protect against data loss when using other saving mechanisms, though I understand that decoupling it so that it could be used with or without TiddlyPWA would be tricky.

I noticed that you had to work around some core design decisions in places. Please do open Issues for such things on GitHub or start threads for them here in the Developer category, and wherever backwards compatibility allows we should address them in the core.

5 Likes