[tw5] Major updates to Chromium native file system saver plugin

Google Group (Read Only)
#22

Dylon,

One of the reasons for introducing the save button at startup is so the plugin can quickly read the original file and compute its file hash as soon as possible.

In some of my own experiments, in a startup action if the wiki is available for editing I immediately save and reload it to indicate this session “owns it”. Similarly when finished the final save needs to release it.

  • With a splash screen advising its loading this may be ok because in need only be done once per session (that has the wiki for edit/save) even on larger wikis.

I have come across two issues;

  • The browser session needs a finger print to be saved in local storage, so this can be compared with the loaded copy of the wiki to check ownership
    • I have found an almost unique fingerprint
  • If the wiki is on file:// I have not found how to get a seperate finger print for each tab, so it could be opened in more than one tab and save will only save the current tabs changes.

Have you found a way to open a file:// wiki in two tabs and not confuse the session?

#23

It’s not really a locking system like you describe. Instead, it basically works by comparing the contents of the data that was saved last time to what it reads off of the disk. If it doesn’t match what it expects, an error is reported. It uses hashing to make this take less memory, but that doesn’t make that much of a difference besides reducing memory consumption.

My suggested route for recovery if an error is reported there is to attempt to save again. It will fall back to the download saver and you will have to merge the two versions manually using tiddler import/export.

I’ve uploaded a YouTube video demonstrating how the check works. Note that since the plugin only cares about contents, the wiki needs to be non-identical (usually happens immediately due to modification timestamps).

It would definitely be possible to store a session fingerprint in local storage or IndexedDB to check for issues locally. However, that doesn’t prevent issues with concurrent modifications from different browsers or on different computers via network drives.

1 Like
#24

Dyllon,

[…]

In terms of stressing if you picked the right file, assuming you used the modal button, you don’t need to stress out too much since the consistency check should catch that.

The stress I describe in steps 1-7 below is only about using the modal button. During the first save, the consistency check is not done, right? Therefore, that first save during each session is a potentially damaging operation. That’s the source of the stress.

#25

Folks,

I am watching this with “bated breath” ( in a nervous and excited state anticipating what will happen) however as I do not understand the technical part of the saver I am finding it hard to contribute.

It seems to me this is approaching the “Nirvana” of savable wikis without any additional installation of servers, savers or Browser addons. This will transform how people can acquire, use and distribute wikis to naive users. It just needs a little more work.

What follows is some ‘innovative’ solutions I have developed that may be able to smooth the path to a solution with the least “resistance” of not almost transparent.

  • You may also find these tips and tricks powerful in their own right.
  • ‘Innovative because’ “I see few if anyone using them”

I believe I can add a lot here but there remains a slight technical barrier for me. What I want to see is to minimize user interaction including between reloads, and when forced interaction is necessary, improve and guide the user for minimum complexity. Here are a few techniques that may help overcome the barriers.

  • Copy path to clipboard: If we want to save a tiddlywiki to an existing location, as may be forced when reloading the wiki or browser, we already have the full path and filename available in the browser, it just needs to be available to the save as dialogue.

    • I have created a save button with additional actions that copies the contents of $:/info/url/full to the clipboard, where ctrl-V pastes the full path and filename and I just click save.
    • This would save the need to remember the full path and filename and reduce the error of selecting the wrong file
    • Remember also
      • the full path tells you if the wiki is at a file location which is a useful piece of information
      • If we have a modal pop up as mentioned below in “Bait and switch”, this can be explained for the user, perhaps a variation for the detected OS.

  • Custom Target/tab: If we want to protect against opening the same wiki in a tab, one trick is after the first save (knowing the full path and filename) we can construct a html link with an appropriate target attribute, then navigate to that (if possible closing the original tab), thus the wiki is now running in a tab with a named target.

    • If we bookmark or pin the tab this remains the target. If clicked in future it will open the very same tab, never another new tab.

      • Warns if unsaved changes exist in the tab.

    • This should help given the global file:// issues

  • Bookmarklet: Perhaps extending the value of Custom Target/tab and because there is value using it, we can consider the use of a bookmarklet both with a target and a payload if necessary. An example may be so if the user returned to the source site, published on the internet the bookmarklette can recall you already have a local copy and with a click. When saving from the source URL for the first time we could prompt the user to use their saved bookmark if they want to access their local copy, to not create a new one, or override and existing one.

  • Bait and switch: Using the existing save button or a customized copy which does more work than the default save button, on this occasion opening a modal with an explanation rather than adding another tiddler and set of instructions would be wise. One could also allow the full path top be copied to the clipbord etc…

    • However this modal would not be displayed if it can go ahead and save automatically.

  • Local storage: As long as one does not tax the local storage much, it can act as a fairly safe space to store changes, and if when closing the tab the dirty indicator triggered the Native file save, then all tiddlers are saved to file and the local storage vacated.

    • The point here is automated save is not necessary but can be optional. The user should be able to choose when they invoke the file saving.

  • File Associations: Most browsers on opening a tiddlywiki file, what ever its name, detect its html and opens it as expected. This allows us to save tiddlywikis as for example a empty.tw file. Helping distinguish it from all other html files. Then we can provide (yes install) a file association to force such files to be opened in a particular browser, in this case the one with local storage for that wiki such as the one containing the IndexDB. Perhaps a .twc (tiddlywiki chrome) or .twf firefox or .twe tiddlywiki edge would be useful.

    • Browsers also enable handlers for given extension/mime types to be set, which may be another avenue to improve user interaction more eg .tw*

  • Note: For me, if my default browser, or all browsers have the Timimi extension installed it does not matter in which browser I open the wiki it can save to the local filename.

    • Currently testing this solution I am using Edge as it is my only browser without Timimi installed. Timimi once installed is almost transparent at saving, its only problem is it requires both a local install and an addon for each browser.

    • There would be value if it were possible to choose or avoid the Native saver on file based wikis in browsers running Timimi.

#26

[…]

I believe I can add a lot here but there remains a slight technical barrier for me.

What I want to see is to minimize user interaction including between reloads, and when forced interaction is necessary, improve and guide the user for minimum complexity. Here are a few techniques that may help overcome the barriers.

  • Copy path to clipboard: If we want to save a tiddlywiki to an existing location, as may be forced when reloading the wiki or browser, we already have the full path and filename available in the browser, it just needs to be available to the save as dialogue.

Does this one have any technical barrier for you? Sounds like you have it figured out. Sounds like a useful approach as long as the tiddlywiki is loaded from a file url.
[…]

  • Custom Target/tab: If we want to protect against opening the same wiki in a tab, one trick is after the first save (knowing the full path and filename) we can construct a html link with an appropriate target attribute, then navigate to that (if possible closing the original tab), thus the wiki is now running in a tab with a named target.

What needs to know the full path and filename here? The user will know and the browser will know, but the functionality running in the web page (i.e. the tiddlywiki instance) will not know. It only has access to the filename minus the directory location. The directory location is hidden from it.

The user has the information required to construct the html link you mention, but the tiddlywiki page does not.
[…]

    • This should help given the global file:// issues

I’m not sure what you mean here.

  • Bookmarklet: Perhaps extending the value of Custom Target/tab and because there is value using it, we can consider the use of a bookmarklet both with a target and a payload if necessary.

What do you mean by “payload”? Could you give an example?

  • An example may be so if the user returned to the source site, published on the internet the bookmarklette can recall you already have a local copy and with a click. When saving from the source URL for the first time we could prompt the user to use their saved bookmark if they want to access their local copy, to not create a new one, or override and existing one.

I’m not sure how the bookmarklet would know you already have a local copy
[…]

#27

Brian;

  • Copy path to clipboard: If we want to save a tiddlywiki to an existing location, as may be forced when reloading the wiki or browser, we already have the full path and filename available in the browser, it just needs to be available to the save as dialogue.

Does this one have any technical barrier for you? Sounds like you have it figured out. Sounds like a useful approach as long as the tiddlywiki is loaded from a file url.

The only issue is the user needs to be told and/or reminded they can use crtl-v in the file dialogue. Its a trick and not commonly used, although easy to use. This is why if we can rely on local storage for small changes (no auto-save) and trigger a modal when saving we can present a modal to guide the user and address the need for the button click.

[…]

  • Custom Target/tab: If we want to protect against opening the same wiki in a tab, one trick is after the first save (knowing the full path and filename) we can construct a html link with an appropriate target attribute, then navigate to that (if possible closing the original tab), thus the wiki is now running in a tab with a named target.

What needs to know the full path and filename here? The user will know and the browser will know, but the functionality running in the web page (i.e. the tiddlywiki instance) will not know. It only has access to the filename minus the directory location. The directory location is hidden from it.

The full path and directory is known by a tiddlywiki loaded from file. If a user stores the full path and filename in a wiki it also known to that wiki (eg store in a cookie)

The user has the information required to construct the html link you mention, but the tiddlywiki page does not.
[…]

Unless it is given and stores the full path and filename, however if we have a bookmarklet it can store the the full path and filename including a target tab. Clicking the same bookmark will always open in the same tab avoiding the opening of multiple tabs and the global file:// issues(see next notes)

    • This should help given the global file:// issues

I’m not sure what you mean here.

Just the “problem” with all file tiddlywikis having access to the same file:// domain. However it could be used for communication between wikis if we wanted. However as a rule we want a file:// tiddly wiki opened only once in a browser (or any browser) so there is not “contention”

  • Bookmarklet: Perhaps extending the value of Custom Target/tab and because there is value using it, we can consider the use of a bookmarklet both with a target and a payload if necessary.

What do you mean by “payload”? Could you give an example?

  • An example may be so if the user returned to the source site, published on the internet the bookmarklette can recall you already have a local copy and with a click. When saving from the source URL for the first time we could prompt the user to use their saved bookmark if they want to access their local copy, to not create a new one, or override and existing one.

I’m not sure how the bookmarklet would know you already have a local copy

The bookmarklet only exists if you created it from a local copy, it contains the link and target but can include a payload ie a tiddler containing details about the local copy. Thus the bookmarklete knows you have a local copy.

A trick that can be used is on arriving at an internet site we want to save a local copy we use the existing mechanism to save the local copy, then we can open that copy (ideally closing the internet tab #1). Once you have opened the local copy, you have access to the full path. Now so we do not loose the link to the local copy we can provide a link (we build it and can add a payload) for users to drag into their favorites and bookmarks.

#1 - before closing the internet tab

  • Save a cookie (or more) for that site indicating the existence of a local copy.

  • When our internet wiki loads it can test for these cookies, and advise the user a local copy exists.

    • It can then ask ONCE the bookmark to be clicked, dropping the “payload tiddler”, then the internet site can also save the full path into the cookie(s)

  • Subsequent visits to the internet site in the same browser can list all local copies of itself and provide links to its target tab OR allow new copies to be saved locally.
    I know my replies are somewhat convoluted but this is a somewhat complex problem with substantial features if we can achieve it. I believe we just need a little more tweaking and a strong guided workflow.

#28

Yes, though assuming you use this plugin in another writer, it should be detected on the other side. The only way to really run into problems is with very rapid race conditions. Something like:

  1. Window 1: Initiate a save, confirm the original value to be X.
  2. Window 1 stops executing due to OS/browser scheduler.
  3. Window 2: Initiate save, confirm original value to be X.
  4. Window 1: Finishes write Y.
  5. Window 2: Finishes write Z.
    While unlikely, this could happen and any changes in Y would be overwritten with Z.

I did some investigation, and it looks like it is possible to serialize the wiki for comparison before the first save. I’m investigating that, but there are some weird scenarios where the first save is slightly different from subsequent saves.

1 Like
#29

  • Copy path to clipboard: That’s a good idea and should be possible when loaded as file://. UI design is a consideration here to make it clear to users when copying the path.

  • Custom Target/tab: Do you mean something like Open Links in a New Tab, Or Re-Use Already Existing Tab (usefulangle.com)? I think this only works when clicking on links from a common page.

  • Local storage: Not really all that safe since we can’t guarantee that changes will be flushed to disk before the browser closes. If going with that kind of approach, I would rather just use a JS lock which can work with tabs in the same browser window.

  • File Associations: Might work, though that requires some platform specific code such as an installer to setup the associations.

For Timimi, there is obviously the solution of disabling the saver via a setting, but that obviously sucks if you have multiple people using the wiki. It’d be nice if there was a way to specify a priority for savers, but I don’t think there is one right now. I don’t want to add a bunch of custom logic to this saver for other specific savers. I could add in some special field containing JS that can be modified for determining whether to enable this saver or not, but that seems like a lot of complexity for users.

1 Like
#30
  • Copy path to clipboard: That’s a good idea and should be possible when loaded as file://. UI design is a consideration here to make it clear to users when copying the path.

It can be hidden within a custom/or customized save button once we are confident the user is informed.

yes, as per your link, a vanilla browser target.The trick here is once establishing that the user wants to revisit a site, and or have a local copy that we use the target parameter when constructing a link to the address we want in a named tab. Following that link once makes it a named tab, and future links to the same target open the same tab.

Ideally we may be able to name the current tabs target (not sure we can), worst case we open a new tab, closing the original tab. The key is to trap it into A bookmark to open the site in future.

Saving on Browser with File System Access API
~HelloThere

  • Local storage: Not really all that safe since we can’t guarantee that changes will be flushed to disk before leaving the browser
  • If going with that kind of approach, I would rather just use a JS lock which can work with tabs in the same browser window.

I do not know what JS Lock is to compare. Can you enlighten me please.

  • File Associations: Might work, though that requires some platform specific code such as an installer to setup the associations.

For windows this is a simple registry entry. A very simple install, done once for all tiddlywikis. But the truth is we should build some installers for each OS.

For Timimi, there is obviously the solution of disabling the saver via a setting, but that obviously sucks if you have multiple people using the wiki. It’d be nice if there was a way to specify a priority for savers, but I don’t think there is one right now. I don’t want to add a bunch of custom logic to this saver for other specific savers. I could add in some special field containing JS that can be modified for determining whether to enable this saver or not, but that seems like a lot of complexity for users.

I need to test this as the two may be possible at the same time, but it could be a matter of manual disabling “Saving on Browser with File System Access API” buy asking the user on first save, "un-Check if you have Timimi installed" as its not required. Its all about finessing the interface and options.

  • Is there a simple way to disable “Saving on Browser with File System Access API” with a click?, or could you build one in? For Timimi, there is obviously the solution of disabling the saver via a setting, but that obviously sucks if you have multiple people using the wiki. It’d be nice if there was a way to specify a priority for savers, but I don’t think there is one right now. I don’t want to add a bunch of custom logic to this saver for other specific savers. I could add in some special field containing JS that can be modified for determining whether to enable this saver or not, but that seems like a lot of complexity for users.
    I just had a thought, can we interfere with the reload button to open the wiki in a “target tab”? Otherwise we just ask;

  • Please open the following link and bookmark the site, and pin the tab (if desired) closing this existing tab.

  • Perhaps also there is a possibility to make use of the “new Window” open and close features of TW 5.2.2
    Regards
    Tones

#31

Custom Target/tab:

I have now tested opening the current wiki in a new tab with the target set, so that if bookmarked, subsequent visits with the bookmark open the same named tab. I knew this worked however;

  • At the very same time a link opens a new tab, the current tab can initiate a modal with no exit, requesting the user closes the tab.
  • If combined with the other interactions demanded by the “Chromium native file system saver plugin” the complexity is further reduced.
  • I re-tested the fact that bookmarks and/or pinned tabs (with a target set) retain knowledge of the target name used to open them

Just to clarify my suggestions in this thread relate to using the " Chromium native file system saver plugin" to make a “wiki your own” with local saving - non-multiuser, it is multi-user in a sense each user ends up with their own wiki.
I am also endeavoring to find solutions to make using this “saver” as simple to use as possible so it works for new users.

See for more on requirements for multi-user my comments at Multi User Alternative to TiddlyWiki - #24 by TW_Tones

#32

Saver modules do export a name and a priority. I think the lower the lowest priority-number wins. … But I’m not sure.

A saver also exports a .canSave() function, which is used by the saver-handler to see if a saver can be used. … The first saver that returns .canSave() === true sorted by priority will be used to save.

IMO the canSave() function can be used to dynamically enable/disable a saver.

See: TiddlyWiki5/core/modules/savers at master · Jermolene/TiddlyWiki5 · GitHub