In-Wiki Incremental Revisions -- Now Even Better!

Developers
#1

Update 2023-03-23: Revision functionality has been completely overhauled and now contains a viewer for previous revisions, a number of settings and statistics, as well as pruning functionality to keep the number of kept revisions small. See also this comment.

You can play around with it on the demo site.

I’ve been playing a bit with the idea I outlined in this thread, and made a proof-of-concept for an in-wiki backup that uses the new prerelease makepatches and applypatches filter operators to create a backups of tiddlers during save that use as little space as possible.

The actual backup macro is surprisingly short, but took even more surprising amounts of time to get right. :sweat_smile:

The macro is called each time a tiddler is saved. For that to work, I used the configurable messagecatcher outlined here to intercept the tm-save-tiddler message and run the macro before passing along the message. For that I created and activated the “Yaisog” PageTemplate that uses $genesis to include a messagecatcher if configured.

The macro can be called manually, too, e.g. with a button. Since it uses the modified date to separate backups, no additional backups will be created by calling the macro repeatedly if no changes were made.

There is also a barebones Patches Viewer which will list tiddlers and their backups (I call them patches, from the filters used) and lets you choose one for viewing. The patches are all saved into the $:/patches namespace. They are associated with tiddlers via their created field – this way renames should be properly tracked and archived.

My vision for the next steps is a button on the ViewToolbar that pops up a list of backups for the current tiddler which can then be chosen for viewing.

Also desperately needed are controls for deleting and pruning older backups, per tiddler and wiki-wide. I can imagine a button that will delete all backups before day X – which means that all patches must be combined into the backup “origin” to which the differential patches are applied. This “origin” is currently created whenever a tiddler is modified for the first time.

There should also be a calculation somewhere to determine the space taken up by the patches.

So much to do, so little time…

Have a nice day
Yaisog

13 Likes
How do you save?
Documenting Changes
Dealing with changes to core tags macro
#2

That is interesting @Yaisog, thank you for sharing.

I have played around with patches in the context of creating undo/redo functionality for user actions that modify multiple tiddlers, for which I create patches of all tiddlers that change during the action. The concept is functional but requires a fair bit more work in terms of UI affordances.

For the approach you are pursuing, you might find UI inspiration in JD’s revisions plugin. I believe the key difference is that this plugin saves the entirety of each version of a tiddler:
http://j.d.revisions.tiddlyspot.com/

2 Likes
#3

Thanks for the link. I had a nagging feeling in the back of my mind that I had seen something like this before, but a search for “backup” returned without what I was looking for. Should’ve searched for “revisions” instead…
I’ll have a look for some inspiration. I’m not sure I’ll make a full plugin out of this one day, though. It would probably be easier for JD to switch to patching…

1 Like
#4

Note to myself about currently missing functionality:

  • Handle deletions of tiddlers → what happens to its patches?
  • Buttons to revert tiddlers to their currently viewed version (from within the history view mechanism)
  • View differences (that’ll have to wait until $diff-text can handle patches, maybe)
  • Control behavior via a specific field, like JD’s keep-rev
  • PageControls button to globally pause/unpause backup generation
#5

After considerable rework, the revision functionality is now at a point where I use it in my production wikis. In the demo site linked to in the OP, there is now also a Viewer for previous revisions with diff view, the option to restore previous (or later) revisions, and selective deletion of revisions.

There are new ControlPanel settings to control the behavior and also initiate pruning of older revisions (with optional prior export of removed revisions), as well as statistics for tiddlers with the most revisions or most used storage space for revisions.

https://yaisog-patches.tiddlyhost.com/

Please let me know if you find something not working.

4 Likes
#6

This is phenomenal!

I’m curious as to whether you’re creating a patch on every save, regardless of whether there are changes? Your demo seems to have a number of revisions for which the viewer isn’t highlighting any diffs. Or is this triggered off the modified date?

If the latter, perhaps it would make sense to not save a patch – or to replace the most recent patch – if the only change is to the modification date. I don’t know about others, but I often use the save button instead of the cancel button when I’m done looking at the source of a tiddler, even if I made no changes.

#7

Hi @Scott_Sauyet, thanks.

Yes, a new revision is created on every save. There actually is a check for the modified date and no revision is created if the date hasn’t changed (a corresponding patch tiddler would already exist), but save does update the modified field even if there were no changes.

I do plan to put in a check if anything but modified has changed. The tricky part is that when save is pressed, not the current draft has to be compared to the original, but the original has to be compared to the previous revision. So that revision has to be rebuilt by successively applying all patches. This might slow down saving a bit when there are a lot of revisions (in my dev wiki I have 100+ revisions for e.g. the tiddler with the macro). Might not be relevant for “normal” use, though.

On the other hand, since I’m intercepting tm-save-tiddler anyway, I might just compare the draft to the original and pass tm-cancel-tiddler upwards instead of tm-save-tiddler when nothing has changed. That way, modified will not change… :thinking:

I often use the save button instead of the cancel button when I’m done looking at the source of a tiddler, even if I made no changes.

Same here, though this has changed somewhat since I started using revisions. It is the reason why some revisions show no differences in the demo wiki, though. :wink:
Also, changes in title or the other fields are not highlighted and are a little more difficult to recognize.

#8

If you do, I might use this tool for that effect alone, regardless of the actual revision-tracking purpose!

I really think this should be in core.

Yeah, I’ve been trying to think how that might be done. No good answers yet, but I’ll keep thinking!

#9

Hello @Scott_Sauyet, I just updated the tiddler $:/plugins/yaisog/ui/PageTemplate/MessageHandler/tm-save-tiddler to check if any fields have changed and if they didn’t send cancel instead of save. The contents of this tiddler is executed whenever the $messagecatcher in the Page Template intercepts a tm-save-tiddler message.

That one was actually pretty tricky, as we have to iterate over a variable number of fields, get the value for each from two different tiddlers, and compare them. What made it extra tricky is that we also have to differentiate between field does not exist and field is empty, while the get operator will give an empty result in both cases. And everything has to be done as a single filtered transclusion, because we cannot use $list here. I’m not sure I found the most elegant solution, but it’s one that works.

Feel free to play around with it and let me know if something doesn’t work. In case that cancel is sent instead of save, there will be a message in the browser console, so you’ll know what’s going on.

I think this would have been much, much, much easier in JavaScript, but then it wouldn’t have been a challenge, would it? :wink:

#10

Hi @Yaisog, I’ll take a look, but it probably won’t be until tomorrow at the earliest.

#11

Don’t worry, @Scott_Sauyet, it is not going anywhere…

1 Like
#12
  1. Long borders are outside the borders
  2. I migrate all tiddlers from $:/plugins/yaisog and $:/core/modules/filters/format/timestamp.js but nothing appears when I save.
    sdhsdh
    sdhsdh1304×583 70.2 KB
#13

Hi @moosh, you probably need to activate the PageTemplate first: Press Ctrl-Shift-L and select the new PageTemplate. Alternatively, select it in ControlPanel → Appearance → Layout.

I really do need to add a short description of how to install it in another wiki, or alternatively package it as a plugin…

Regarding the linebreaks, go to ControlPanel → Appearance → Theme Tweaks, and set “Wrap long lines in code blocks” to “Yes”.

Have a nice day
Yaisog

1 Like
#14

It was enabled, apparently the problem is in the browser, since is displayed normally in Firefox.

Is it possible to enable the ability to save revisions to other buttons? For example, to the add comment plugin Mehregan — personal knowledge management app

It makes sense to make it possible to add a short description to the revision, which will be displayed with the name in the viewer.

#15

Hello @moosh, the creation of revisions is done by calling the global macro make-incremental-backup with the title of the tiddler to be revisioned as a parameter, e.g.

<$macrocall $name="make-incremental-backup" title=<<currentTiddler>> />

This can be added to any button, to be performed on-demand.

It makes sense to make it possible to add a short description to the revision

You mean like in e.g. a code commit or a manual snapshot? That’s a slightly different application from what I had in mind. I don’t think it would take a lot of work to change the macros and templates to add this, but it would be another plugin with a different purpose? It would probably require some tweaking at other ends, e.g. the prune-backups macro should never prune manually created revisions, and these should also be visually distinguishable in the list.

While I currently have no plans to add more functionality beyond the basics on the proof-of-concept site, I might pick up on the idea at some point. But, like with all things TiddlyWiki, it is possible for anyone to just take my code and extend it.

#16

Hi @Scott_Sauyet (and maybe other), there was a typo in $:/plugins/yaisog/ui/PageTemplate/MessageHandler/tm-save-tiddler that led to unexpected behavior when deleting fields. This is now corrected on the demo site.

#17

Just a caption, done. Thanks for the great plugins.

1235
1235913×521 67.8 KB

#18

Again, amazing work. I’m not quite sure where I will want to use this yet, but I simply know I will.

One bug: there seems to be a mismatch between what’s reported in statistics and the number of revisions shown on the viewer.

#19

Hi @Scott_Sauyet, regarding the bug: The viewer always also shows the current state of the tiddler, which in most cases is not a revision (unless you go back to an older revision, then a copy of the current state is made beforehand). There is a note at the top when viewing the current state:

image
image812×89 9.25 KB

I did think about showing only “current” in the list instead of the full date, to make it clearer, but then I thought that knowing the last modification date of the tiddler might also be beneficial. I can see how this would be a little confusing, though.

Maybe I’ll put the last modified date into the explanatory note…?

#20

I don’t know if I was stuck in some old version by accident or in some intermediate state. The issue I was seeing is totally gone and everything seems to be working the way I’d expect.

I don’t know when and where I’ll use this (most of my main TW work is on Node and Git; where I already have a useful versioning mechanism) but I do know that I will be using this at some point. Thanks for the useful tool!