WebDAV for saving single file wiki

[willw]

I’ve been experimenting for a few days, running rclone locally to serve tw5 files over webdav from google drive. It took around 1/2 hour to follow a tutorial on configuring Google drive with a client_id, then:

$ rclone serve webdav gdrive://wikis
NOTICE: Google drive root 'wikis': WebDav Server started on
                  http://localhost:8080/

Saving is very slow, around 10 seconds between ‘Starting to save’ and ‘Saved’. The empty wiki is ~4MB with selected plugins. I’m using pre-release 5.2.3 (don’t know if it’s using PUT saver; frankly, I’m pretty vague on what that is, even after spending a while searching around the feature commit).

The idea is to collaborate with a few colleagues, via the shared google drive. I doubt it’ll stand up to simultaneous editing. We’ll see.

If this works out then the plan to use the file upload plugin to slurp pdfs into a gdrive folder alongside, then we’ll have full text search of pdfs via drive (integrating with tw search would be cool).

Incidentally, while looking around google’s cloud stuff, I came across a way to ‘publish’ and serve the tiddlywiki html file using a single Google Apps Script function

render-html-file-in-drive-in-a-google-apps-script-web-app

Simpler than TiddlyDrive, and less effort than a Google App Engine version.

[Mark_S]

Ten seconds isn’t all that bad.

I haven’t used google drive with webdav, but I have used rclone to copy files to gdrive. The additional parameters I use to improve speed are:

--fast-list --transfers=40 --checkers=40 --tpslimit=10 --drive-chunk-size=1M --max-backlog 200000

Don’t know if these have effect in server mode, but it might be worth checking out. Also, be sure to have your own unique gdrive id (client_id ?) and not use the demo one provided at rclone.org – that account is throttled because of it’s heavy usage. Sorry if I’m vague – it’s been awhile since I set it up. You might check the rclone forum to see if they have hints on optimisation.

[amreus]

If anyone uses the scoop installer, I have created a “bucket” that contains dave, webdav, and widdler programs. You can add it here: http://github.com/amreus/bucket

If you are unfamiliar with scoop, it makes installing, updating, and uninstalling apps fairly quick and painless.

rclone, which can also be used to serve wikis using webdav, is available in one of the default app repositories, which are referred to as “buckets.”

The basic scoop commands to try out widdler for example is:

> scoop bucket add amreus https://github.com/amreus/bucket
> scoop update
> scoop install widdler

The main scoop site is http://scoop.sh

If you aware of any related apps, I’d be glad to try to add them also.

[Joost]

I addressed the possible use of the Hacdias webdav server (Windows) and WebDavNav (OSX) server a while ago. Let me share 2 tips that may help you in using them.

1. Hacdias webdav server

If you use multiple tiddlywikis, you may want to use the following example script for quickly starting webdav and opening a particular wiki. It is an AutoHotKey script which you can use with the free AutoHotKey tool. You can even compile it into a Windows executable (.exe) if you like. This example expects the webdav executable and the script in the same directory (change the script if you want to use a different approach).

Script:

SetWorkingDir %A_ScriptDir%
FileSelectFile, SelectedFile, 3, , Open a tiddlywiki file, Tiddlywiki Documents (*.htm; *.html)
if (SelectedFile = "") {
    MsgBox, You didn't select any file
	ExitApp
	}
else {
    Run, webdav -c config.yaml    
    Run, brave.exe -new-tab "127.0.0.1:9999/%SelectedFile%
    }

Replace brave.exe with the executable of your browser and the settings you use for webdav. The script will remember the wiki file location you used the last time you used the script.

Now you can start any wiki in one go (select and start)…

2. WebDavNav

Chrome (or Chromium related) browsers will not accept using a self signed certificate within the latest OSX versions. Enable the certificate in the OSX keychain (selecting only SSL for the certificate is sufficient).

Maybe this is of use to anyone. Enjoy.

[stokito]

I’m new to the TW and just tried the widdler and I’m a bit confused.
So the TW page is edited and saved entirely with PUT. E.g. even if was added a short note the 2 mb will be transferred over network.
As far I understood the widdler is just a regular webdav server and the only difference that it creates files automatically on attempt to access it. I’m not sure how the feature is important.

Maybe it’s possible just to make a index.html page that with a JS will list all files in directory (PROPFIND) and render them. The files will be small notes in raw text or markdown.
Then the index page can be just cached in browser and any changes will be small and atomic with a small transfer over network.
Sorry for asking the stupid question I just doesn’t have time to read all docs.

Also please add the link to Awesome WebDAV list GitHub - fstanis/awesome-webdav: A curated list of awesome apps that support WebDAV and tools related to it.
There is a list of all main servers, clients and articles.

Also please check the WebDAV Share with Lighttpd - instruction for OpenWrt WiFi routers firmware. You can just setup a small webdav server just on your router (which is anyway always online, just put a USB stick). Then you’ll have a very cheap self hosted solution to store your data.

[jeremyruston]

Hi @stokito welcome to TiddlyWiki!

@stokito

So the TW page is edited and saved entirely with PUT. E.g. even if was added a short note the 2 mb will be transferred over network.

That’s correct.

In TiddlyWiki parlance, Widdler (GitHub - qbit/widdler: A WebDAV server for TiddlyWikis) operates as a “saver”, meaning it saves the entire wiki file each time. TiddlyWiki also supports “syncers” that synchronise individual tiddlers back to a server (that’s the architecture used with TiddlyWiki’s built-in server side). I think it would be possible to build a WebDAV-basd syncer for TiddlyWiki, I don’t think it has been done.

[stokito]

Thank you. It would be great to have a pure webdav only server. Then it’s not needed for the nodejs based server side. The lighttpd webdav is really small and fast but more difficult to configure.
BTW the WebDAV itself hat PROPPATCH and PROPFIND methods that may be used for tags or additional taxonomy PROPPATCH Method | Microsoft Learn

If the TW turned to a PWA it should become more useful.

[TW_Tones]

Welcome to the community @stokito

@stokito

If the TW turned to a PWA it should become more useful.

• I agree there, do have a go. It would make TiddlyWiki use and adoption seamless.

@stokito

the 2 mb will be transferred over network.

Do have a look at Using the external JavaScript template for further optimisation of the save process (even without a server). When you compare the size of a single file wiki to common media files and images today, it is at the low end on the “size spectrum”.

@stokito

You can just setup a small webdav server just on your router (which is anyway always online, just put a USB stick). Then you’ll have a very cheap self hosted solution to store your data.

• People are also using Raspberry Pi etc… for this.
  Yes, I have long thought of tiddlywiki as a way to give devices, their own website, for documentation and tools on the Intranet.
• A little bit of data, in the right place, especially on an interactive webpage/app such as tiddlywiki can make a big difference.
  • Rather than just files a TiddlyWiki, kind of provides an “operating system” and Graphical user interface to those files, if you have the universal client, a browser handy.
• Even if offline, stick a cheap thumb drive to a device containing a TiddlyWiki and use an OTG cable from a smartphone to access the devices TiddlyWiki.
  • better yet, if its on the network.
  • There is a Thumb drive in my Car with all the manuals on it, and curated access in a tiddlywiki, for example.

Welcome to all TiddlyWiki’s possibilities.

[stokito]

Just for the record: there is also a remoteStorage protocol which is oauth+ something like webdav. And it looks like there is even a plugin for the TW GitHub - fiatjaf/tiddlywiki-remotestorage: a syncadaptor plugin that saves tiddlers on custom remoteStorage directories
The plugin is outdated and from issues it looks like not working anymore.

Anyway, there is a JS library that allows to easily sync data with Google Drive, DropBox the RS protocol and in future will support the raw WebDAV.
https://remotestoragejs.readthedocs.io/en/latest/

So the TW may just start using the RemoteStorage.JS and delegate this question of data saving to it.

[stokito]

ok, it’s already discussed here "Universal plugin for tiddlywiki backup service" [concept, discussions]