TiddlyPWA: Secure offline storage and cross-device synchronization solution

Plugins
, , ,
#65

I was about to say that I got TiddlyPWA to work on my Mac and iOS devices which now sync instantly. Impressive!
Thanks @Newbie for your detailed description!

  1. Is the procedure you (Newbie) describe above necessary? In principle one should be able to drag and drop the content tiddlers and the tiddlers needed for the particular wiki, e.g. for Groktiddlywiki there are
  • GrokTiddlyWiki
  • TakeAway
  • TiddlyRemember

as per Grok TiddlyWiki — Build a deep, lasting understanding of TiddlyWiki

However, in practice I ran into problems even when deselecting tiddlers that were already in existence: a banner obstructed the top menu bar of the wiki and some more issues.
So, should the drag and drop work at least in principle and if not, I would like to better understand why not.

  1. How does the syncing work? It appears way too fast to be exchanging an entire HTML file each save. What is the granularity of the saving/syncing mechanism and what is lurking in the shadows (if anything)? So far the experience is very good indeed but I have not tested large wikis with complicated content!

  2. On my Mac, what needs to be done to use TiddlyPWA with Tiddly Desktop or should I access the wiki through Safari?

  3. Where does the wiki handled by TiddlyPWA reside in the file system?

Thanks everyone and especially @valpackett, of course!!!

#66

To be honest, I do not think is necessary but convenient for wikis with a large set of tiddlers. I only used it as it was the recommended method by @valpackett on this post

The plugin drag and drop step exists because the plugins are not exported into the .json.

TiddlyPWA was born specially to be an alternative that didn’t submit a whole .html each time (among other things), as the original blogpost mentions. For what I can understand from Synchronization and Encryption tiddlers and from @valpackett posts, it creates two categories for tiddlers, one with the “app” information (plugins, core, etc), and another as “data”, and then syncs the detected changes from “data” tiddlers into the browser’s IndexedDB (A kind of Cookie Jar on “steroids” that exists on modern browsers) and then upload the changes into the server (anyone with more expertise feel free to correct me :sweat_smile:).

What you suggest sounds like a good experiment. Having the wiki on TiddlyDesktop would make it easier to detach the wiki from other browser cache, although I’m not sure how well would play. Maybe we could just drag and drop the “TiddlyPWA” and “Web App Manifest” plugins into a TiddlyDesktop generated wiki and check how well it plays after setting up the sync server(I will try as soon as I have a chance :grin:).

If you refer on what you are currently seeing when the wiki is open then on the browser’s cache.
If you refer to where the TiddlyPWA Sync server is saving the “data” tiddlers, it is doing so into the sqlite3 file inside the hidden .data folder on the project’s root folder.

Last but not least, take all I wrote on this answer with a pinch of salt, as I’m learning how it works myself and I’m highly prone to errors and missunderstadings :sweat_smile:

#67

@valpackett I have long argued for a PWA implementation and I am excited that one is presented here, I would love to jump on the bandwagon, however I am currently time limited and in attempting to follow the instructions here, there is a lot of assumed knowledge or hit Brick walls for example I try “Try It Out – Right Here, Right Now” I get “Could not decode the salt”.

Perhaps we could consolidate, simply or extend the guidance to allow a larger audience to adopt it?

I would like to see this migrated into an easy to use solution that we can use to market the value of TiddlyWiki to a much broader audience, but for now, even with my technical knowledge I am not getting any where. Let alone to get a holistic understanding so I can compare it with other implementations.

2 Likes
#68

Through Safari (or Firefox etc.) Trying to mix other solutions might lead to everything exploding :smiley:

Well, the default (Glitch dot com oriented) is .data/tiddly.db relative to where the server is started at, but the path can be configured via the --db flag or the DB_PATH environment variable.

#69

That… sounds strange. Ummm are you using password manager autofill? It might have autofilled something into the salt field under the “If you are going to sync a pre-existing wiki into this one, click here” dropdown >_< Yeah, the message is definitely confusing if the dropdown is actually closed. I’ll change it and try to add some anti-unexpected-autofill measures as well.

2 Likes
#70

Silly question perhaps but does TiddlyPWA have access to the file system? I am on macOS and Safari atm, if that is of importance.
If not, how would I out-reference image files that I do not want included in the wiki itself?

#71

Nope! That’s not currently possible.

With current filesystem APIs I think it would be possible to sync external files without copying them to IndexedDB, but implementing that sounds like a bit of a nightmare. Starting with UX type decisions even. If we have add one external file on one device, what happens on other synced devices? TiddlyPWA is not Syncthing.

If you just want local references that don’t sync the files themselves… That should be a separate plugin actually (:

#72

Like @TW_Tones , i’m both really keen on this most promising solution to what seems quite a popular problem, but am also finding its implementation a bit overly-challenging for a non-developer-type to adopt. Specifics of my problem were posted here (rather off-topic; sorry!), and answered by @valpackett here (quickly, too: much appreciated!).

Still, i’m not yet seeing how this works at the most fundamental level -for example saving. I am logged-in to the same test wiki i set up in 2 different browsers (Chrome instances, both); i’ve made an edit in one, it’s been saved and shows up in Recent Changes in that instance, but not in the other. I haven’t (yet) got access to a cloud server (still don’t know how to do that), and neither has this TW instance been downloaded to my desktop (as TW does by default w/o some cloud storage), so… Where does this instance with my 1st Test Tiddler reside?

Perhaps this software is not for the likes of a non-developer user like me, but i’m keen enough, i’m willing to invest some effort in this; i just have to ask: should i stay away until there’s a more n00b-friendly UI/UX about it? Or is it OK to bumble along thusly in this channel? Or is there a better channel for new-user-supportive conversation?

#73

Activly Bumble!

They are a friendly bunch and generally quite helpful. :slight_smile:

#74

Thanks @valpackett, but I was not thinking about a syncing solution (which TiddlyPWA is not) but merely the ability to

  • link to files that exist locally on the computer’s filesystem
  • maybe even embed images present in the filesystem without having to import the image files themselves to the wiki

I am clear about having to sync files myself.

#75

If your wiki is powered by nodejs there is a feature that does exactly what you are looking for.

#76

Thanks @valpackett , I really like TiddlyPWA. I first tried it on glitch, now trying to setup in a docker container.
When I try to start the container - all env variables set and correct deno command, I think - I get:

....
Download https://deno.land/x/sqlite@v3.8/src/function.ts
Download https://deno.land/x/sqlite@v3.8/build/vfs.js
Download https://deno.land/x/lz4@v0.1.2/mod.ts
Download https://deno.land/std@0.159.0/encoding/ascii85.ts
Download https://deno.land/x/lz4@v0.1.2/wasm.js
error: Uncaught (in promise) Error: Run Deno with --unstable!
	if (!('BroadcastChannel' in globalThis)) throw new Error('Run Deno with --unstable!');
	                                               ^
    at listen (https://codeberg.org/valpackett/tiddlypwa/raw/branch/release/server/run.ts:9:49)
    at https://codeberg.org/valpackett/tiddlypwa/raw/branch/release/server/run.ts:32:29

Any idea?

In docker I’m using Deno 2.0.0-rc.5 and I think --unstable is deprecated, and for instance Deno.seekSync (function) as well. Maybe TiddlyPWA needs Deno 1.x.y?

#78

Yep, the current flag is --unstable-broadcast-channel. Updated the docs and released an update that upgrades the sqlite dependency to a new version that works on Deno 2 (and includes a previous decorator fix that didn’t get into the release branch), so now it should work well!

4 Likes
#79

Hi @valpackett , I tested, it works ! Thanks a lot.

#80

@valpackett thank you, thank you for your great work! I’ve provisionally transferred three of my TW to TWPWA to see how it works for me.

Is there a way to change the “app” icon? All three of them have the same one and since I’m using the text-less option in iOS, I need to remember which one is which. It be even nicer if it could pick each wiki’s own defined icon (for example like regular bookmarks show the bookmarked TiddlyWiki’s .ico resource).

Here’s what I’m seeing right now:

IMG_7798
IMG_77981179×1470 44.4 KB

#81

You can set the favicon by saving an image as $:/favicon.ico tiddler, more info here: https://tiddlywiki.com/#Setting%20a%20favicon. This is what TiddlyPWA does,
Don’t be confused by the “.ico” in the tiddler title, it’s just a title. The image doesn’t have to be in ICO format. PNG is usually the best choice for compatibility with various browsers.
After changing the favicon, you have to remove and add the PWA desktop shortcut again for it to reflect the change.

1 Like
#82

Huh, I knew about the $:/favicon.ico mechanism and I was already using it in the original wiki that I transferred into the PWA version. When I did not see my favicon in the new version I thought it was there but it wasn’t used.

I guess favicons are part of the things that are not exported by the “Export All” Tool. I also had the same issue with my custom ViewTemplate ordering, I had to recreate it by hand.

I wish TiddlyPWA had a function similar to TiddlyHost’s “Upload file into this wiki” that would get everything from the source wiki.

#83

After a bit of experimenting, I realized my best bet to carry everything over was to start from scratch. Instead of using the Export Tool, I dragged the whole HTML file into the PWA version and everything carried over properly. Tiddlers, plugins, settings, the whole thing.

#84

$:/favicon.ico is used by default, but you can also further customize the icons part of the app manifest by adding tiddlers with the $:/tags/ManifestIcon tag. type/sizes/purpose fields will be passed through to the manifest.

3 Likes
#85

Hi there again, hopefully this “necro” is seen with good eyes :crossed_fingers: .

I have been a happy user of TiddlyPWA over glitch since I entered this community. Life didn’t allow me to contribute back enough, but TiddlyPWA has been there to support me.

Lately, as I stumbled across an increasing “503” errors from glitch, I learnt that they are shutting down the service, as it was also echoed on the forums last month.

Does exists any similar site that we can migrate our TiddlyPWA to? Will be there any tutorials for the migrations? And more important than anything else, can we help with the transition in any way?

Thanks in advance as always!