Using syncadaptor and Github saver in parallel

,
#1

Hello all,

I’m experimenting with the idea of using a local indexeddb syncadaptor and the GitHub/cloud saver handler in parallel. To that end, I’ve created a demo for you all to play with. There may be bugs as this is an early build so don’t do anything I wouldn’t do:

https://abesamma.github.io/oneplaybook-syncadaptor-demo/

What do all think? Do you think this is a useful way of using TiddlyWiki? You get the benefit of fast local saves which are automatic and manual cloud sync (which can take slightly longer to persist using the inbuilt savers) when you click the save button. Do let me know if you see anything break. Already noted some small issues that I will iron out over the coming days.

Link to repo: GitHub - abesamma/oneplaybook-syncadaptor-demo: A demonstration of the syncadaptor for TiddlyWiki made by and for Oneplaybook.

3 Likes
#2

Yes.

IMO “browser stores” are emerging as increasingly important because of the LOSS of easy access to the OS following the “apocalypse / clampdown” all browsers went through for security reasons.

My one worry is only you can get “locked-in” to a specific browser instance.

But, overall, it is interesting and potentially very useful.

TT

1 Like
#3

That is mitigated by the ability to sync to your cloud account via one of the inbuilt savers. You may basically transfer your work to different browsers via this. Since they still work in parallel to the syncadaptor, wouldn’t that be enough? That’s the question.

1 Like
#4

I think it may, finally need, a few more docs explaining synch?

TT

1 Like
#5

I think it may, finally need, a few more docs explaining synch?

Most certainly yes. I will incorporate this into the Oneplaybook repo so I might add it there under the technical notes section.

1 Like
#6

You are one clever fella! :smiley:

#7

Is this different from the Browser Storage plugin? Is there an upper storage limit? Are there any security concerns re local file addresses as there are (mildly) with the browser storage plugin?

Thanks!

1 Like
#8

I must confess, I haven’t tried the browser storage plugin so I must say IDK here. Concerning the upper limit, it really depends on your device and browser. So yeah, there might be. However, those limits are quite generous and you will be asked by your browser if you’d like to raise the limit when you’re close to hitting them I believe.

We’re working on a some way for a user to authenticate access to data and encrypting them at rest wherever they’re persisted (cloud or local) to mitigate some of these concerns. The early public build may not have that but that’s a goal we aspire to before we hit v1.0.0. The ultimate objective is to persist single tid files to any cloud provider of your choosing instead of a big html file.

1 Like
#9

Hi, TW has an official plugin, that seems to do the same thing: https://tiddlywiki.com/#BrowserStorage%20Plugin It allows you to host a wiki at eg: GitHub and if you don’t save it, it uses the browser local storage to save changed tiddlers. …

It is an opt-in mechanism. So you have to activate the right filter setting, if you want to save every tiddler to the browser storage.

There is a readme in the plugin, which also contains the setting. I did play with it at: My TiddlyWiki — a non-linear personal web notebook, which is set up to automatically save every tiddler. …

WARNING: It is a personal test-wiki, which may change at any time, without warning. So if you want to keep your data, create your own wiki at GitHub.

#10

Our solution uses PouchDB, which in turn uses IndexedDB, whose limit is calculated based on the hard disk space available (usually 50%). LocalStorage (which is what BrowserStorage uses) is hardcapped at 5mb.

Functionally they do the same thing. But ours has more wiggle room in terms of storage. I wish more browsers supported the File system API. That would be neat!

1 Like
#11

@abesamma I think this is an important step in the use of tiddlywiki, the former noteself did something similar but rather than github uses a hosted CouchDB.

When using such a DB locally it has the advantage of being able to maintain versions of tiddlers even if you do not write them back to the host.

I am not sure how your solution may help the two key issues that remain in my view with saving tiddlers/tiddlywikis.

  • How does it work If I access the same wiki from more than one tab/browser/device with the write credentials?
  • How can we have users have their own version of the wiki + their changes independent of the master wiki (This should be close with your solution and others)
  • How to we publish something on the internet such that more than one user can contribute to the master wiki without data loss. Bob does this on Node but securely publishing online is not simple and I am unsure how secure.

See also Saving on Browser with File System Access API

2 Likes
#12

One problem with Note Self, which also used Pouchdb, was that it didn’t seem to be able to cleanly purge itself from memory. It had a configuration to remove itself, but when you went back to reinstall, the reinstall failed because it could detect something there. So, anyway, is it possible to purge your local stored objects?

1 Like
#13

@Mark_S I just checked for the IndexDB in the above solution, It is possible to use the developer tools to locate the DB and delete it.

A Javascript button could be made to do this is my Guess.

Post Script - you must reload the wiki.

#14

@abesamma

I found on your demosite in Chrome “developer tools” the following;

  1. Page layout may be unexpected due to Quirks Mode

  2. One or more documents in this page is in Quirks Mode, which will render the affected document(s) with quirks incompatible with the current HTML and CSS specifications.

Quirks Mode exists mostly due to historical reasons. If this is not intentional, you can add or modify the DOCTYPE to be <!DOCTYPE html> to render the page in No Quirks Mode.

Document in the DOM tree Mode URL
document Quirks Mode Oneplaybook — Syncadaptor demo

I do not know if it is important.

Incidentally, Copying content in table form from Chrome dev tools creates tables valid in both discourse and tiddlywiki. as above.

#15

Won’t that delete everything, even information held by other applications and web pages?

#16

You only need delete the named indexdb for the URL in question.

#17

Very good questions to consider @TW_Tones. I don’t have all the answers now but I am sure we can work progressively towards answering most of them. Especially that last one.

1 Like
#18

Yes indeed, but we don’t bother to take advantage of that at least in this working version. We’re staying KISS for now and simply want to achieve reliable and robust CRUD operations and achieving the goal of persisting tiddlers as individual files to any service, local or cloud, such that it is compatible with TW for nodesjs tiddler folder setup using the usual HTML 5 file setup.

However, that is something I would like to experiment in the future.

2 Likes
#19

@abesamma it would also be nice to access to a list of tiddler titles that are stored in the local IndexDB. This would allow a user to backup changes or send the changed tiddlers in a JSON package or generate a free standing wiki from the hosted wiki plus changes (Eg if access to the host is lost)

Just mentioning it now since you may have recently coded the part that you use to retrieve tiddler titles in the IndexDB.

Good work, I am impressed.

1 Like
#20

That’s interesting. I did have a short look at the plugin code. … It seems there are 2 js files that include the same libraries 2 times.