(easy) webdav saving à la Timimi

There has been a lot of discussion about Timimi lately because as of June it will no longer work in Chrome (for now it can still be reactivated if Chrome disables it, I read. If you remove it Chrome will not allow you to reinstall it though). The following zsh script automatically saves TiddlyWiki files on change and makes a timestamped backup copy each time they are saved, and may thus serve as a simple replacement (see also notes at the end). It was inspired by @btheado’s post WebDAV for saving single file wiki - #19 by btheado. Non shell-type users: while it can be run in the zsh (macOS Terminal) it does not need to be run in the shell–see below.

#!/bin/zsh

twdir='/Users/jan/Library/Mobile Documents/com~apple~CloudDocs/m'
twwebdavsaverdir=$twdir/twwebdavsaver
pidfile=$twwebdavsaverdir/twwebdavsaver.pid
logfile=$twwebdavsaverdir/twwebdavsaver.log

exec > $logfile 2>&1

cd $twdir

rclone serve webdav --addr 127.0.0.1:8080 . &
echo $! > $pidfile

while true; do
  find . -type d -o -name "*.html" | entr -ndps '
    if [[ -f $0 && $(grep "TiddlyWiki created by Jeremy Ruston" $0) ]]; then
      HISTORYDIR=$(dirname $0)/history
      [[ -d $HISTORYDIR ]] || mkdir $HISTORYDIR
      cp $0 "$HISTORYDIR/${$(basename $0):r}"-$(date +%Y-%m-%d--%H-%M-%S).html
    fi
  ' &
  PID=$!
  echo $PID >> $pidfile
  wait $PID
  # kill -9
  [[ $? -eq 137 ]] && break
done

The script works for local TiddlyWiki files in the directory specified in twdir (/Users/jan/Library/Mobile Documents/com~apple~CloudDocs/m in the example), which you will need to change, and in all of its subdirectories.

On change it updates the TiddlyWiki file and makes a timestamped copy in the directory specified in HISTORYDIR, which in the example is the subdirectory history of the directory in which he TiddlyWiki file resides. If this subdirectory does not exist the script will create it. You may want to use another name than history.

You also may want to change the format of the time stamp (-$(date +%Y-%m-%d--%H-%M-%S)) as well as the port number in the URL (8080).

Installation and running

Installation my sound complicated, but once installed usage is transparant.

Shell users will know that they can create the script wherever they like and that it needs to be made executable. Non-shell users can use Automator–see below.

The script uses the utilities rclone and entr which are not installed by default on macOS and thus need to be installed (also if you use Automator). Both can be installed using brew with the following shell (Terminal) commands:

brew install rclone
brew install entr

You can test if brew is installed by typing

which brew

in Terminal.

If it is installed Terminal (the shell) will tell you where it found it, usually

/usr/local/bin/brew

If brew is not installed it will say

brew not found

You can install it by following the instructions on https://brew.sh/ (again the shell!).

The script can be run with macOS Automator (no shell!). For this to work it needs to be modified slightly. The line

export PATH=$PATH:/usr/local/bin

needs to be added at the top and

#!/bin/zsh

may be removed.

Instructions for automating execution of a shell script with Automator can be found on e.g. Automating Login and Startup Events in macOS (Option 1). Copy the modified version of the script and make sure /bin/zsh is selected in the Shell dropdown. Then save (at a location of your choice; I use the name twwebdavsaver). Running is then a simple matter of clicking the icon of the saved application.

If desired you can change the icon of the generated application. See e.g., Using Automator.

If you use Automator you also need a script to stop the saver (e.g., named twwebdavsaver-stop), created as a second Automator application with the following content:

twdir='/your/directory/here'
twwebdavsaverdir=$twdir/twwebdavsaver
pidfile=$twwebdavsaverdir/twwebdavsaver.pid
logfile=$twwebdavsaverdir/twwebdavsaver.log

exec >> $logfile 2>&1

kill -9 $(cat $pidfile | tr '\n' ' ')
echo "$(date +%Y/%m/%d\ %H:%M:%S): twwebdavsaver stopped"

The directories must be the same as in the twwebdavsaver script.

When run the scripts (both the shell script and the Automator script) create two files in the specified directory (twwebdavsaverdir): twwebdavsaver.pid and twwebdavsaver.log. The pid file contains the process IDs of the rclone and entr utilities, which the stop script needs to stop them. Note that there may be many PIDs in the pid file. This is because entr stops and restarts when it detects a change in the contents of the directories it monitors (creation, deletion of files, etc.). The log file logs entr's and the stop script’s messages.

Please note the following:

The backup algorithm is extremely simple (a backup copy each time a file is saved). It is possible to implement more sophisticated algorithms though.

The script will not make backups of non-TiddlyWiki html files it finds. It identifies TiddlyWiki files by looking for the string TiddlyWiki created by Jeremy Ruston, which all TiddlyWiki files contain.

The script is very general in that it can handle not only TiddlyWiki files in the directory specified in twdir but also in all subdirectories of this directory, however deep. If all TiddlyWiki files reside in the same directory or if only one TiddlyWiki file needs to be monitored the script can be greatly simplified.

It is possible to add the Automator application to your login items. It is then started automatically on login and you will not have to start and stop it manually every time.

The stop script kills only the rclone and entr instances started by the saver. Other instances are not touched.

I may be possible to implement something similar on Windows, which would make the approach useful to more users (I have no access to Windows).

While I have been using the above script for some time without problems (so far) it is wise to MAKE A BACKUP BEFORE YOU TRY IT, especially since the script on the first save will first update the existing file and only then make a backup copy. On occasion I have noticed that an instance of entr may still be running after stopping the saver. This can be detected with the following command:

pgrep entr

If this command returns a process ID or pid (e.g., 19624), then kill the instance with

kill -9 theprocessidreturnedbythepgrepcommand
1 Like

Some TiddlyWiki users may find handy to extend this script and have a full history of changes saved in version control as well (it is ugly for fully encrypted wikis, but works nice for plain text ones).

Just like running a file watcher (entr in this case, inotifywait in Linux is a common option as well) in a loop, and copying files when they change, one may as well autocommit the changes to a version control repository.

Here is an example script:

#!/bin/bash

abspath=$(readlink -f "$1")

while true; do
  inotifywait -e modify "$abspath" |\
    git add "$abspath" &&\
    now=$(date) &&\
    git commit -m "$now" "$abspath"
done
1 Like

Interesting (also discussed in @btheado’s post mentioned above).
I find diffing single file TiddlyWikis a pain though.
For TiddlyWikis for which version control is crucial, I use the node.js version, which stores all tiddlers as individual, human readable files, which makes diffing much easier.
I may implement additional backup algorithms as I go.