WebDAV for saving single file wiki

[saqimtiaz]

@TW_Tones

I would like to discuss further work on the
https://github.com/hacdias/webdav server and its use with tiddlywiki.

I think a separate post might make for better engagement.

[saqimtiaz]

There is a relatively new custom WebDAV server for TiddlyWiki, with elements built in similar to my wikifarm set up for creating new wikis:

Features

• TiddlyWikis are served over WebDav so you can save directly from the browser.
• Automatically create new wiki files by browsing to a non-existent html file.
• Built in .htpasswd management (Adding users).
• Password protection via HTTP Basic Authentication.
• Multiple users (adding another user to the .htaccess file creates a new user namespace).
• Optional TLS support.

GitHub - qbit/widdler: A WebDAV server for TiddlyWikis

[Mohammad]

Saq,
It seems it needs the Go language to be installed! Am I right?

[saqimtiaz]

@Mohammad yes that is correct, though it should be possible to create pre-compiled binaries for it, just as is the case for the other two Go based implementations (micromata/dave and hacdias/webdav).

Personally I prefer to have the wiki creation/management stuff in TiddlyWiki itself as is the case in my set up, but having this in the WebDAV server is useful for those not willing or able to go the DIY route.

[saqimtiaz]

This may be of interest for those of you using WebDAV to save your wikis:

File Uploads plugin: updates

October 3rd, 2021: Changelog: FileUploads plugin v0.1.14: requires TiddlyWiki 5.2.0 now better integrates uploaders in plugins not using the Commons tiddler namespace adds support for disabling auto file uploads and manually uploading selected tiddlers using the tm-upload-tiddlers message. This completes my backlog of intended changes to the FileUploads plugin. Feedback and potential bug reports are welcome. The one remaining issue that I am tracking is that tiddler titles are used verb…

[TW_Tones]

I just wqanted to add to this discussion the externalising of Javascript

Using the external JavaScript template with the single file configuration

If not already, it could reduce the total size of a webdav package of more than one TiddlyWiki

[digitalap3]

@saqimtiaz

it will automatically detect and act as a WebDAV client

Hello this is mostly for @saqimtiaz but of course any help is appreciated - referencing the above quote and looking at your wikifarm webdav TW I can’t figure out how you are getting that filesystem tiddler.

I have wikis on a webdav server and it saves as expected. I can connect to and interact with the server with a client on my local machine.

Could it be that I am not ‘localhost’ but connecting to an external domain via https? That doesn’t seem to matter with ‘normal’ webdav saving.

Thanks for this work I am really starting to appreciate webdav and TW.

edit: I am working with a new 523-pre TW that I dragged your wikifarm over to and imported all tiddlers.

[saqimtiaz]

As the writeup and videos you linked to state, the wikifarm is implemented using custom code that is not published as of yet.

The ability to save over WebDAV is a native TW feature.

[digitalap3]

Yeah I figured there was something in there I was missing. No matter how many times I read things I know I am missing the answer. Thank you for your reply. I hope that becomes public some day it is a really nice feature!!! I don’t know why it’s taken me this long to realize how sweet the webdav features are.

[Mya]

@Mark_S It’s been a while since you posted this, but thank you! A (very) recent switch to Mac has me pretty much questioning everything instead of my usual ‘dive I and see what happens’ approach. Hopefully that level of mastery will come to me once again quickly… In the meantime I appreciate the lifeline you threw out here

[Mohammad]

Hi @saqimtiaz

Is the below Tiddlywiki saver categorized under WEBDAV saver?

pearigee/tiddly: A small TiddlyWiki 5 server that supports PUT (DAV) saving. (github.com)

It seems this is a light weight solution! But I am not sure after compile and build it needs Rust or can produce single executable?

[saqimtiaz]

I am not sure. Don’t we have a lot of lightweight WebDAV sever options now? What makes this particular one more appealing than others?

One thing to watch out for is that a some of these custom WebDAV implementations don’t really support the entire WebDAV spec and just support enough methods for the TiddlyWiki Put saver to work. That is sufficient if all you want to do is save, but falls short if you also want to create backups, create backup directories if they do not exist, etc.

[Mohammad]

@saqimtiaz

I am not sure. Don’t we have a lot of lightweight WebDAV sever options now? What makes this particular one more appealing than others?

Just noted to small footprint! nothing more!

@saqimtiaz

but falls short if you also want to create backups, create backup directories if they do not exist, etc.

Thank you for clarification!

[amreus]

So does reclone support the copy request?

I’ve been toying with redbean even though it lacks a key feature on Windows. It’s only a few lines of code to implement a PUT method for saving. I looked into the RFC for the copy method and it seems like it would be fairly simple to implement also. Can you share a Wiki file that send the copy request?

[willw]

I’ve been experimenting for a few days, running rclone locally to serve tw5 files over webdav from google drive. It took around 1/2 hour to follow a tutorial on configuring Google drive with a client_id, then:

$ rclone serve webdav gdrive://wikis
NOTICE: Google drive root 'wikis': WebDav Server started on
                  http://localhost:8080/

Saving is very slow, around 10 seconds between ‘Starting to save’ and ‘Saved’. The empty wiki is ~4MB with selected plugins. I’m using pre-release 5.2.3 (don’t know if it’s using PUT saver; frankly, I’m pretty vague on what that is, even after spending a while searching around the feature commit).

The idea is to collaborate with a few colleagues, via the shared google drive. I doubt it’ll stand up to simultaneous editing. We’ll see.

If this works out then the plan to use the file upload plugin to slurp pdfs into a gdrive folder alongside, then we’ll have full text search of pdfs via drive (integrating with tw search would be cool).

Incidentally, while looking around google’s cloud stuff, I came across a way to ‘publish’ and serve the tiddlywiki html file using a single Google Apps Script function

render-html-file-in-drive-in-a-google-apps-script-web-app

Simpler than TiddlyDrive, and less effort than a Google App Engine version.

[Mark_S]

Ten seconds isn’t all that bad.

I haven’t used google drive with webdav, but I have used rclone to copy files to gdrive. The additional parameters I use to improve speed are:

--fast-list --transfers=40 --checkers=40 --tpslimit=10 --drive-chunk-size=1M --max-backlog 200000

Don’t know if these have effect in server mode, but it might be worth checking out. Also, be sure to have your own unique gdrive id (client_id ?) and not use the demo one provided at rclone.org – that account is throttled because of it’s heavy usage. Sorry if I’m vague – it’s been awhile since I set it up. You might check the rclone forum to see if they have hints on optimisation.

[amreus]

If anyone uses the scoop installer, I have created a “bucket” that contains dave, webdav, and widdler programs. You can add it here: http://github.com/amreus/bucket

If you are unfamiliar with scoop, it makes installing, updating, and uninstalling apps fairly quick and painless.

rclone, which can also be used to serve wikis using webdav, is available in one of the default app repositories, which are referred to as “buckets.”

The basic scoop commands to try out widdler for example is:

> scoop bucket add amreus https://github.com/amreus/bucket
> scoop update
> scoop install widdler

The main scoop site is http://scoop.sh

If you aware of any related apps, I’d be glad to try to add them also.

[Joost]

I addressed the possible use of the Hacdias webdav server (Windows) and WebDavNav (OSX) server a while ago. Let me share 2 tips that may help you in using them.

1. Hacdias webdav server

If you use multiple tiddlywikis, you may want to use the following example script for quickly starting webdav and opening a particular wiki. It is an AutoHotKey script which you can use with the free AutoHotKey tool. You can even compile it into a Windows executable (.exe) if you like. This example expects the webdav executable and the script in the same directory (change the script if you want to use a different approach).

Script:

SetWorkingDir %A_ScriptDir%
FileSelectFile, SelectedFile, 3, , Open a tiddlywiki file, Tiddlywiki Documents (*.htm; *.html)
if (SelectedFile = "") {
    MsgBox, You didn't select any file
	ExitApp
	}
else {
    Run, webdav -c config.yaml    
    Run, brave.exe -new-tab "127.0.0.1:9999/%SelectedFile%
    }

Replace brave.exe with the executable of your browser and the settings you use for webdav. The script will remember the wiki file location you used the last time you used the script.

Now you can start any wiki in one go (select and start)…

2. WebDavNav

Chrome (or Chromium related) browsers will not accept using a self signed certificate within the latest OSX versions. Enable the certificate in the OSX keychain (selecting only SSL for the certificate is sufficient).

Maybe this is of use to anyone. Enjoy.