TiddlyPWA: Secure offline storage and cross-device synchronization solution

Plugins
, , ,
#1

:wave: Hello everyone,

so a year ago I’ve tried TiddlyWiki again, didn’t like any existing storage/hosting solutions and started making my own. Now, it’s finally ready to be announced to the public!

Introducing TiddlyPWA :tada: :tada: :tada:

Basically, :notebook_with_decorative_cover: TiddlyPWA turns TiddlyWiki5 into an offline-first Progressive Web App with encrypted local persistent storage and efficient synchronization with a server that can easily be hosted for free.

It’s TiddlyWiki turned into a modern notes app, with the modern user experience you’d expect from that, without sacrificing anything that makes TiddlyWiki great: you can easily install plugins and themes, upgrade the TW core, and so on – when hosting the app wiki on the sync server, saving it is a one click operation.

It all started with seeing a prototype of an IndexedDB syncadaptor, looking at the syncadaptor API made me realize the potential and have a whole Vision of how it should work… now here it is! It’s very much a beta version as of right now, but with your support it could grow into something truly reliable and amazing.

Thanks :3

23 Likes
Tools for task management and GTD other than TiddlyWiki
Highlights not to be missed in 2023
Various question about tiddly wiki
#2

My preliminary review concludes: Wow!! - Well done!

Thank you for sharing this.

Best:
Hans

2 Likes
#3

@valpackett how to use pre existing wikis using this method?

1 Like
#4

The easiest way would be to export all tiddlers as JSON (that’s under the sidebar → Tools), drag the JSON file into the TiddlyPWA wiki and click “Import”. That’s how I was importing all of tiddlywiki.com for testing something large :slight_smile:

#5

I will check that out when I am back on my desktop.

Does this work with tiddlyclip plug in which works on single file html wikis and require browser add ons.

1 Like
#6

Of course, any regular plugin that doesn’t mess with storage should work! Just went ahead and tested TiddlyClip, works great indeed.

Note that you need to host the app wiki (the HTML wiki with all the plugins including the TiddlyPWA plugin itself) on a sync server to be able to easily install plugins.

#7

This is super cool, @valpackett!
Maybe just for the less technically inclined (like me) who might take a longer time to set up, here is what I did to self-host and it seems to work:

  1. Set up Tailscale on “server” (my home pc or a vm) and “clients” (my phone)
  2. Install Caddy (I used brew in the mac)
  3. Go to TiddlyPWA and obtain the hash and salt
  4. Create a script called pwa_tiddly.sh with the following content:
ADMIN_PASSWORD_HASH=REDACTED_REDACTED # <- first line of previous step
ADMIN_PASSWORD_SALT=REDACTED_REDACTED # <- second line of previous step
DB_PATH=/Path/To/Database/Location/pwa.db \
deno run --unstable --allow-env \
--allow-read=/Path/To/Database/Location --allow-write=/Path/To/Database/Location \
--allow-net=:8000 \
https://codeberg.org/valpackett/tiddlypwa/raw/branch/release/server/run.ts \
--host 0.0.0.0 \
--port 8000

4.1. Make the script executable (chmod +x pwa_tiddly.sh)
4.2. Run the script with ./pwa_tiddly.sh

  1. Run Caddy reverse proxy where YOUR.SERVER.IP is the Tailscale IP of the machine that runs the script from the previous step. It looks something like 100.12.3.45 or 100.123.45.678. You can also run Caddy without the ip (just --from :9000 etc) but then you won’t get automatic HTTPS which is necessary.
caddy reverse-proxy --from YOUR.SERVER.IP:9000 --to YOUR.SERVER.IP:8000
  1. On the “clients” go to HTTPS://YOUR.SERVER.IP:9000. I got a warning since the https certificate is not signed but Ignored it and it worked (Safari on iOS). (Possibly better solution would be to sign the cretificate but no idea how)
6 Likes
#8

Yay! I’m glad you like it :slight_smile:

Hah, I used to run various services on a home server but I’m completely burned out on being a sysadmin, so being able to use an easy, zero-maintenance public hosting service like Glitch was an explicit goal here. (Being able to do it safely, even: client-side encryption is always used so that e.g. the admins of such a service wouldn’t be able to snoop on my notes.)

better solution would be to sign the cretificate but no idea how

One of Caddy’s biggest selling points is in fact that it can get you a TLS certificate automatically from Let’s Encrypt; that’s mostly intended for things that are on the public internet, but as long as you use a real public DNS name you can use it for intranet things, and there is DNS challenge support in Caddy. There is of course also the whole “have your own CA that you entrust on all your devices” option of course, back in the day I even made a little program for “running” one :smiley:

2 Likes
#9

Thank you! Maybe I should set up Glitch but I also use this as a learning experience. :slight_smile:

I discovered that Tailscale supports https using the .ts.net domain and Caddy has special support for this domain

Special cases:

Enabling https in tailscale and using my .ts.net domain instead of the ip did the trick :slight_smile:

1 Like
#10

EDIT 2: This is resolved see below

I’m experiencing an issue that my macros (tiddlers tagged $:/tags/Macro) are not available until I open them manually. Any ideas of how to trigger them to be available?

EDIT: This is my workaround e.g., for TiddlyTools I add the following to my default tiddler

<p hidden>
{{TiddlyTools/Macros/edit-list}}
</p>
1 Like
#11

Thanks for the report! Hah, I thought that I made it so that anything with any tag that starts with $: is always fully loaded on start (I wasn’t even thinking about macros when I did that, haha). Turns out I misunderstood how tags are stored in the serialized JSON form, expecting an array instead of a string. Oops.

Funnily enough I thought I was seeing that issue when testing with the tiddlywiki.com contents, but actually it was just the macro tiddlers not being included in the JSON export; the real issue wouldn’t occur there, because the macro tiddlers in the docs themselves start with $: and that’s handled correctly.

I have released version 0.1.2 with the fix, you can drag the updated plugin from #UpdateInPlace into your wiki and update it – hopefully that goes smoothly :slight_smile:

2 Likes
#12

Update went smoothly and everything is working now! Amazing work!

#13

Maybe just for clarification: Should one update all three files (core, TiddlyPWA, Web App Manifest)? Core is blocked since it is the same version and I updated the other two.

#14

Yes, update what updates :slight_smile: This time the only real update was TiddlyPWA, I bumped both versions just for consistency.

3 Likes
#15

I’m excited to try this - sounds impressive! Unfortunately I’ll have to wait until someone provides a really simplified set of instructions as I got lost immediately. Am I right that if I get to the end of this process I’ll be able to access a single wiki both on my laptop and my phone - and both offline? That’s a scenario in TiddlyWiki broadly I haven’t figured out yet.

All I have is my Windows work computer and personal iPhone. tiddly.packett.cool is blocked on my laptop but glitch.com isn’t, so I’m hopeful that once this is setup I can still operate. I opened a glitch account, but don’t know what to do next.

2 Likes
#16

This sounds amazing! I’m going to try to play around with it this weekend.

#17

@valpackett this may be quite revolutionary, like @stobot I am still struggling with some of the jargon and finding it hard to plummet the implications and possibilities, despite the recognition by myself, some years ago, that a PWA Version of tiddlywiki was most likely possible, but also highly desirable. Ultimately I see it being used to publish “tiddlywiki application solutions” across multiple platforms with little or no complexity for non-technical users. This has the potential to empower TiddlyWiki designers to provide real solutions to a general audience. This would I believe, solve some long standing “make it your own” or distribution issues that have long held tiddlywiki back.

  • Anyway thank you for your work and I will investigate deeply as time permits, and will certainly consider a financial contribution sooner or later.
  • Although it will take time for me to understand the methods and details, when I research something I consider serious, as I do this, I tend to move towards a deep understanding, with the intent to consider its strategic implications. That is why It may take sometime for me.

Perhaps I could ask you @valpackett to share what you think is the scope of possibilities this will enable?

Bravo, Bravo, Bravo.

1 Like
#18

I also set up Glitch and the easiest way is to go here and enter a password (that will be hashed and added to the login for your wiki)

https://tiddly.packett.cool/#ServerHosting/Glitch

But not sure what to do if this site is blocked. I entered “Test” and copied the link

It looks like one could hash the password manually and use the glitch link directly but not sure how to do that.

#19

Obviously also dependent on the same website TiddlyPWA — TiddlyWiki Storage & Sync Solution

#20

Yes that’s correct. You get a glitch link you use to access the wiki. On iOS a saved that to the Home Screen via safari and it works offline (enable persistent storage in the save menu!)

2 Likes