This is kind of what @jeremyruston is experimenting with in the Multi Wiki Store branch using SQLite. Might be worth following or getting your input. 
As to Syncing, I’ve come around to the idea that there should be a “fast-sync” layer that gets active wikis in memory in sync with each other (and could optionally also sync user details like cursor positions), and a “slow sync” layer that is actually writing to a shared data store that new wikis loaded into memory get “initialized” from.
Hard problem, especially around “source of truth” and possibly CRDT live multiplayer code. (Yjs also uses IndexedDB and WebRTC in some clients). Getting close to something tho.
Hi @valpackett
The mismatch here is that IndexedDB only offers an asynchronous API, while the core wiki store object has to be synchronous. That’s really what a syncadaptor does: allows an asynchronous API as backing store. In that sort of situation one can think of the wiki object in memory as a cache.
I have experimented with pluggable alternate implementations of the core wiki object. See the “demo-alternate-store” branch at GitHub - Jermolene/TiddlyWiki5 at demo-alternate-store
1 Like
Hi there. Did anyone tried to migrate a tiddlywiki with custom content into TiddlyPWA? It is feasible?.
The use case I think of is using TiddlyPWA to host the Grok Tiddlywiki book, so I can easily track my progress on whichever device I am without the hassle of hosting a private page on a place that allows static sites and saving the wiki on a Source Control Forge (gitlab, github, etc).
I read that it could be possible to export all the tiddlers as json and then import them into TiddlyPWA, but I think it will miss the really useful TiddlyRemember plugin cards.
Did anyone tried?
Thanks in advance.
You can import plugins as well!
Thanks! That made me confident enough to give it a try 
. I’m still hestitant of having the admin page on a internet-front-facing for personal notes, but for keeping the Grok TiddlyWiki TakeAways and the Tutorial-built wiki seems great.
Here’s the steps I followed to migrate both wikis (Grok and tutorial-built wiki) I had on TiddlyDesktop into TiddlyPWA:
- Export the wiki you want to import into TiddlyPWA as .json
- Login to glitch.com
- Go to ServerHosting/Glitch TiddlyPWA’s tiddler
- Write the password you want to use to access the wikis admin panel
- Click on the remix button, the glitch.com for the site will open with the readme
- Click on “Preview > New Window” to open the admin panel
- Use the password from Step3, the admin menu should unlock
- Click “Create new wiki” button and write the name to identify the wiki information
- Go to “TiddlyPWA — TiddlyWiki Storage & Sync Solution” and paste the URL from the admin page and the hash for the wiki you just created (Listed on the wiki list at the admin page) and click “Upload”
- Go to the given URL by “~UploadApp”, copy the token used on UploadApp and set a new password. Only works on non-private tabs due to using the browser memory database.
- (Optional) If the wiki has extra plugins, drag and drop them from “More > Plugins > Plugins” on the original wiki tab into the TiddlyPWA tab. Will require using the “Save” button so the new tiddlywikiapp.html is saved on Glitch
- Click on “Import” from the “Wiki More options > import” and select the exported JSON from the original wiki
- Click on “Import” button from the list of tiddlers shown to be imported
- Reload the website and login if you did not save the password on the browsers
- Sadly, the wiki configuration is not exported into the JSON, so just try to mimic the original wiki options, navigating from all the places on $:/ControlPanel and setting everything to the same values as the original wiki
Here’s the final result, looks like the images are exported as base64 on the JSON. It would be awesome to be able to also export configuration options on Wikis that rely heavily on it. Also I can not see the “Create your own” and “Help” buttons on for TakeAways but maybe is just a setting I missed setting up 
.
I did not see the usual “Install” button that other PWAs have, but I’m unsure if this is expected for all PWAs or not, from my complete ignorance on the subject:
Currently I’m using the clever save and export into Gitlab Pages from @boris for personal notes, which allows leveraging the private gitlab pages feature to put a barrier to the Wiki from the wide internet. It has it’s drawbacks, like having to reload the site between browsers restarts to ensure you are not overwritting your notes, or using loads of space if you do not clear the commits from the repository (I think this is due to the whole HTML being uploaded and it being encripted, currently is using almost 1GB and the wiki only has an small photo tiddler 
).
Having the tiddlers auto-update and refresh is almost magical, and looks almost like the dream I have of tiddlywiki commiting just the tiddlers changes/creations/deletions into the chosen SCM for data backup from the hosted static site (Maybe someone already did this and I’m totally unaware 
).
Hopefully this helps others to give TiddlyPWA a try as well 
I was about to say that I got TiddlyPWA to work on my Mac and iOS devices which now sync instantly. Impressive!
Thanks @Newbie for your detailed description!
- Is the procedure you (Newbie) describe above necessary? In principle one should be able to drag and drop the content tiddlers and the tiddlers needed for the particular wiki, e.g. for Groktiddlywiki there are
- GrokTiddlyWiki
- TakeAway
- TiddlyRemember
as per Grok TiddlyWiki — Build a deep, lasting understanding of TiddlyWiki
However, in practice I ran into problems even when deselecting tiddlers that were already in existence: a banner obstructed the top menu bar of the wiki and some more issues.
So, should the drag and drop work at least in principle and if not, I would like to better understand why not.
-
How does the syncing work? It appears way too fast to be exchanging an entire HTML file each save. What is the granularity of the saving/syncing mechanism and what is lurking in the shadows (if anything)? So far the experience is very good indeed but I have not tested large wikis with complicated content!
-
On my Mac, what needs to be done to use TiddlyPWA with Tiddly Desktop or should I access the wiki through Safari?
-
Where does the wiki handled by TiddlyPWA reside in the file system?
Thanks everyone and especially @valpackett, of course!!!
To be honest, I do not think is necessary but convenient for wikis with a large set of tiddlers. I only used it as it was the recommended method by @valpackett on this post
The plugin drag and drop step exists because the plugins are not exported into the .json.
TiddlyPWA was born specially to be an alternative that didn’t submit a whole .html each time (among other things), as the original blogpost mentions. For what I can understand from Synchronization and Encryption tiddlers and from @valpackett posts, it creates two categories for tiddlers, one with the “app” information (plugins, core, etc), and another as “data”, and then syncs the detected changes from “data” tiddlers into the browser’s IndexedDB (A kind of Cookie Jar on “steroids” that exists on modern browsers) and then upload the changes into the server (anyone with more expertise feel free to correct me ).
What you suggest sounds like a good experiment. Having the wiki on TiddlyDesktop would make it easier to detach the wiki from other browser cache, although I’m not sure how well would play. Maybe we could just drag and drop the “TiddlyPWA” and “Web App Manifest” plugins into a TiddlyDesktop generated wiki and check how well it plays after setting up the sync server(I will try as soon as I have a chance 
).
If you refer on what you are currently seeing when the wiki is open then on the browser’s cache.
If you refer to where the TiddlyPWA Sync server is saving the “data” tiddlers, it is doing so into the sqlite3 file inside the hidden .data folder on the project’s root folder.
Last but not least, take all I wrote on this answer with a pinch of salt, as I’m learning how it works myself and I’m highly prone to errors and missunderstadings
@valpackett I have long argued for a PWA implementation and I am excited that one is presented here, I would love to jump on the bandwagon, however I am currently time limited and in attempting to follow the instructions here, there is a lot of assumed knowledge or hit Brick walls for example I try “Try It Out – Right Here, Right Now” I get “Could not decode the salt”.
Perhaps we could consolidate, simply or extend the guidance to allow a larger audience to adopt it?
I would like to see this migrated into an easy to use solution that we can use to market the value of TiddlyWiki to a much broader audience, but for now, even with my technical knowledge I am not getting any where. Let alone to get a holistic understanding so I can compare it with other implementations.
2 Likes
Through Safari (or Firefox etc.) Trying to mix other solutions might lead to everything exploding 
Well, the default (Glitch dot com oriented) is .data/tiddly.db relative to where the server is started at, but the path can be configured via the --db flag or the DB_PATH environment variable.
That… sounds strange. Ummm are you using password manager autofill? It might have autofilled something into the salt field under the “If you are going to sync a pre-existing wiki into this one, click here” dropdown >_< Yeah, the message is definitely confusing if the dropdown is actually closed. I’ll change it and try to add some anti-unexpected-autofill measures as well.
2 Likes
Silly question perhaps but does TiddlyPWA have access to the file system? I am on macOS and Safari atm, if that is of importance.
If not, how would I out-reference image files that I do not want included in the wiki itself?
Nope! That’s not currently possible.
With current filesystem APIs I think it would be possible to sync external files without copying them to IndexedDB, but implementing that sounds like a bit of a nightmare. Starting with UX type decisions even. If we have add one external file on one device, what happens on other synced devices? TiddlyPWA is not Syncthing.
If you just want local references that don’t sync the files themselves… That should be a separate plugin actually (:
Like @TW_Tones , i’m both really keen on this most promising solution to what seems quite a popular problem, but am also finding its implementation a bit overly-challenging for a non-developer-type to adopt. Specifics of my problem were posted here (rather off-topic; sorry!), and answered by @valpackett here (quickly, too: much appreciated!).
Still, i’m not yet seeing how this works at the most fundamental level -for example saving. I am logged-in to the same test wiki i set up in 2 different browsers (Chrome instances, both); i’ve made an edit in one, it’s been saved and shows up in Recent Changes in that instance, but not in the other. I haven’t (yet) got access to a cloud server (still don’t know how to do that), and neither has this TW instance been downloaded to my desktop (as TW does by default w/o some cloud storage), so… Where does this instance with my 1st Test Tiddler reside?
Perhaps this software is not for the likes of a non-developer user like me, but i’m keen enough, i’m willing to invest some effort in this; i just have to ask: should i stay away until there’s a more n00b-friendly UI/UX about it? Or is it OK to bumble along thusly in this channel? Or is there a better channel for new-user-supportive conversation?
Activly Bumble!
They are a friendly bunch and generally quite helpful.
Thanks @valpackett, but I was not thinking about a syncing solution (which TiddlyPWA is not) but merely the ability to
- link to files that exist locally on the computer’s filesystem
- maybe even embed images present in the filesystem without having to import the image files themselves to the wiki
I am clear about having to sync files myself.
If your wiki is powered by nodejs there is a feature that does exactly what you are looking for.
Thanks @valpackett , I really like TiddlyPWA. I first tried it on glitch, now trying to setup in a docker container.
When I try to start the container - all env variables set and correct deno command, I think - I get:
....
Download https://deno.land/x/sqlite@v3.8/src/function.ts
Download https://deno.land/x/sqlite@v3.8/build/vfs.js
Download https://deno.land/x/lz4@v0.1.2/mod.ts
Download https://deno.land/std@0.159.0/encoding/ascii85.ts
Download https://deno.land/x/lz4@v0.1.2/wasm.js
error: Uncaught (in promise) Error: Run Deno with --unstable!
if (!('BroadcastChannel' in globalThis)) throw new Error('Run Deno with --unstable!');
^
at listen (https://codeberg.org/valpackett/tiddlypwa/raw/branch/release/server/run.ts:9:49)
at https://codeberg.org/valpackett/tiddlypwa/raw/branch/release/server/run.ts:32:29
Any idea?
In docker I’m using Deno 2.0.0-rc.5 and I think --unstable is deprecated, and for instance Deno.seekSync (function) as well. Maybe TiddlyPWA needs Deno 1.x.y?
Yep, the current flag is --unstable-broadcast-channel. Updated the docs and released an update that upgrades the sqlite dependency to a new version that works on Deno 2 (and includes a previous decorator fix that didn’t get into the release branch), so now it should work well!
4 Likes