Nodejs and backups

Hello I’m trying out tiddlywiki5 on nodejs for what right now would be a personal notes/journal.

I’m not really seeing anything about backups really on the main site (other than BACKUP BACKUP BACKUP :slight_smile: ) or in the forums more than questions on it though.

Someone said backup code may already be set up, and it would just be a case of adding rename_old_tiddler func before the (over)write to get multiple versions?

I had just watched a git video the other day and that may be a really good option but I’m afraid if it’s not super simple I might lose data. I’m hoping maybe I can get some versions and then backup off the server somehow. Any ideas? Anyone have a working model? Is the code in plaintext on the server somewhere to take a look at or would it need to be recompiled?

In my opinion, git is your best option when it comes to Tiddlywiki. Data will eventually get lost, sometimes saving errors will wipe out the text of a tiddler, etc, so you really want versioning. I’m not a Windows user, so if you are on Windows, I am not familiar if there is a non-technical, GUI interface for Windows to use git for those that are not comfortable with the command line, but you really should learn at least the most basic git commands. Every server I run is on Linux, and for Tiddlywiki, I have setup cronjobs to run scripts to make a commit every 6 hours to offsite git servers using Gitea git server.

You can always get away with using other tools, like rsync, then a script to make a backup zip of the rsynced changes. There are a lot of ways to do backups, even as basic as just zipping the folder, but that makes for large backups, as the entire Tiddlywiki is backedup every time rather than just the changes.

You do not want to use local backup versioning without also syncing off the machine that is running Tiddlywiki, or even that same location, such as your house. If the harddrive fails or some other catastrophic event, local backups to the same machine or same building are as useless as not backing up at all.

But, there are no built in backup functions or features in Tiddlywiki, that is up to you to do on your own.

If you use Github, Gitea or Gogs to backup offsite, all the files are fully readable online so you can recover lost data, copy and paste, etc.

Gitea - https://gitea.com
Gogs - https://gogs.io

Basic Commands:
(these are the commands for linux terminal. Not sure if it is the same on Windows terminal)

Setup your commit info:

Add Name to Config

git config --global user.name "<your firstname> <your last name>"

Add Email to Config

git config --global user.email <your@email>

Initialize your New Git:

change to the “tiddler” folder of your Tiddly server and type:

git init

Stage all files for versioning

git add .

Commit Changes with date message

git commit -m "$(date -R)"

I would recommend pushing the changes to a remote git server.
I use my own git server running Gitea. If you use a remote server,
you want to add the server info to your local git. Not going to go into
the setup of the server or the generation of ssh keys. There is plenty
of info online for how to do this, and you find more info in Gitea and Gogs
documentation.

In the same “tiddler” directorty, type the following:

Add Remote Server to git

git remote add origin ssh://git@<remote address>:<remote port>/path/to/your.git

Push Changes

git push -u origin master

I would create a backup script that does these commands all at once:

cd <path to your tiddler folder>
git add .
git commit -m "$(date -R)"
git push -u origin master

And schedule it to run at intervals, like very 6 hours, etc. It will only make a commit
if there are actual changes.

2 Likes

Hi @brandonholland, welcome to talk.tiddlywiki!!

I’m a Node-and-git user, something fairly similar to what @CasperBooWisdom is suggesting. But if trying to use git with your node server seems too complex, you might try for a simpler setup and skip Node altogether.

There are two main advantages to Node, as I see it, and they’re closely related. First, it stores the wiki as separate tiddler files. If, like me, you occasionally want to perform some transformations on your tiddlers outside of what TW gives you out of the box, then this is immensely helpful. I can operate on tiddler files rather than the whole wiki.

Second, it makes it easier to save backups as lists of small changes, and not as entire wikis. I do this with git, but there are other tools that could do the same thing. This is also made much easier by the multiple files when running Node.

But if these advantages are not important to you, then you can easily use a single-file wiki, served up from the filesystem or from any of the seventeen zillion tools that create a miniature web server from the file-system. Whenever you want a backup, you can simply copy the file to your backup storage location. The only downside I know to this style is that when you save changes from Tiddlywiki, the OS will suggest that you save your file as something like myWiki.html - Copy or myWiki.html - Copy 2 on Windows or myWiki.1.html, myWiki.2.html on *nix, and you will have to correct it every time to save as myWiki.html every time. I don’t find that a big deal, but it is a minor inconvenience.

Welcome @brandonholland, I just installed the updated bob.exe which uses node and allows you to have multiple users, including your self by managing tiddler edit locking.

  • A quick look through that allows you to backup your node wiki as a single file html.
  • Also you could just export all tiddlers, from inside the wiki, and reimport them to restore as a backup.
    • You could automate this further if desired
  • Lets see if more node users have their own suggestions

I am looking to utilize git to back up specific wiki folders, is there anything that should be added to .gitignore from a standard nodejs wiki install or can I just push the whole wiki directory structure?

I don’t have a .gitignore in any of my wiki backups. I run my tiddlywiki’s through Docker, so I have one main folder called tiddlywiki and under that I have separate folders for each of my wikis, so I just do “git init” in each of those sub folders that have their own tiddlywiki.info file and the tiddlers folder, as everything in there is just text files, images, pdfs, etc. Although you could make one git repo for all of them in a root folder that contains each wiki in sub folders, I would suggest making each wiki its own repo.

2 Likes

Now that I am almost ready to commit, I see that I will need to use .gitignore for the roles.csv file in each wiki directory other than that I should be good to go.

I developed TidGi (short for Tiddly Git) for auto git backup, it uses GitHub - tiddly-gittly/git-sync-js: JS implementation for Git-Sync, a script that backup notes in git repo to the remote. , so I just need to double click open the app, and use it like using Notion, no need to touch dirty things under the hood (terminal and git cli).

Here is a script I use to back up my long running Node.js TiddlyWiki. I made a directory in my wiki directory called wiki_backups and put that script in there. Then I run that from cron daily.

This keeps daily copies up to 12 latest of the wiki.

#!/bin/bash

dir=$(cd $(dirname $0) && pwd)
backup_file=$(date "+wiki-%Y-%m-%d.tar.gz")

pushd "$dir/.." >/dev/null
tar c wiki | gzip > "$dir/$backup_file"
echo "Backup saved to $backup_file"
popd >/dev/null

pushd "$dir" >/dev/null
ls -t1 wiki-*.tar.gz | sort -rn | tail -n +12 | xargs rm -fv
popd >/dev/null
1 Like

using git wit tiddlywiki on node is a good idea but if you have separated data file and meta files for each tiddler, then you will get some annoying things to do by yourself when you rename a tiddler. for instance for the fubar tiddler, you will get something like fubar.tw5 and fubar.tw5.meta. If you rename fubar into horror you will have to rename the two files. But you should have renamed with git mv (and restarted node) and if you’ve done it with tiddlywiki, then you have to make it manually.

This is why I don’t git the tiddlers directory, but I maintain a git-operated copy of it where I manually put all the change in sync. That’s not very pleasant.

For my use-cases, that’s not a problem. Any such changes I do programmatically, so if I need a rename, I can do it for the matching .meta file alongside the main tidder.

I don’t quite follow what you do here, but it doesn’t sound pleasant! My workflow seems relatively simple for my usecases.

What does this mean? Why not just auto commit anything? My git is 2GB after importing the Evernote, so I don’t care about a single mess in it.

Why are you renaming files in your git repo? Git commits will snapshot your files at a specific time, and if you rename them later in tiddlywiki, or even manually outside of tiddlywiki, then the next git commit will show those files renamed. I just don’t understand what you are doing with git, because git commits should generally not be messed with once committed. Any changes should be done with a new commit, so I am just baffled about what you are referring to. Why are you using the git mv command. Those commands are there for those that really need them for specific purposes, but generally, especially when backing up a tiddlywiki, you should not be altering your commits, but rather simply making new commits with the changes. The way you are doing your backups makes me wonder why you are even using git in the first place. You might as well just do manual archive backups.

You can use the git mv (move) command to move files into subdirectories. So it does not mess with the commit history.

There is nothing wrong with git mv.
After moving a file git status identifies the action as renamed

PS E:\temp\asdf> git status
On branch master
Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
        renamed:    test.txt -> aa/test.txt

If you move a file with the file-explorer from one directory to an other git status identifies it as 1 file “deleted” and a new file “untracked”:

PS E:\temp\asdf> git status
On branch master
Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        deleted:    new.txt

Untracked files:
  (use "git add <file>..." to include in what will be committed)
        aa/new.txt

So technically there is a difference. BUT in the end, in a visual history, git identifies both actions as “renaming” a file. So practically there is no difference

So @jypre a git mv is not necessary. You can move your files around, using the file explorer. It should be much less manual work.

Then 2 git commands should be just fine.

git add .
git commit -m "some useful description"

hope that helps
-m

Yes, I am aware of that, which is why I said

Those commands are there for those that really need them for specific purposes, but generally, especially when backing up a tiddlywiki, you should not be altering your commits.

I completely agree. There is no practical reason to make changes to files using git commands when backing up a Tiddlywiki. It just adds another unnecessary manual step, which the OP said is: