[RFC] Running a difference check on whole of tiddler including fields

[RFC] Request for comment

Arising from Importing a large json file containing hundreds of new tiddlers - #7 by TW_Tones

Has anyone previously made a process to compare two tiddlers, with different titles or one being a shadow and the other an edited shadow tiddler including, comparing all or most field values?

  • The DiffTextWidget seems to have sufficient functionality to do this
  • You may want to compare all fields except modified and possibly created, creator and modifier
  • Ideally one could do this with a filter and detect those that are effectively identical so one of the two can be deleted.

Perhaps a preview field differences would be helpful

  • Even more interesting if one of the tiddlers was within a JSON tiddler (non-plugin)
1 Like

The TW preview can handle that for the tiddler text

Thanks @pmario I was aware of that.

I have also noted that the input process allows examination of differences as well as for fields. Although in both cases they do so graphically. I am keen instead to check all fields and text for differences (excluding created and modified) as these are almost inevitably, ideally with a count, any changes above 0 would mean they are not identical.

This all reminded me of an opportunity to make better use of the preview tools. I have built some interesting tools I will share soon.

  • Including making use of other previews I created as well.

When investigating the difference tool I learned we use this library $:/core/modules/utils/diff-match-patch/diff_match_patch.js a public library with API · google/diff-match-patch Wiki · GitHub I suspect there are more functions we could provide access to with a little javascript.

  • This includes generating patch content and more.
  • See the DiffTextWidget and the note that Within it, the variable diff-count is available.

This is in part what triggered my interest Importing a large json file containing hundreds of new tiddlers but I am pursuing multiple lines of investigation and potential uses. Thus why I am asking;

Another Question is also is it sufficient to exclude title, created and modified dates and possibly creator and modifier?

The key feature may be given two tiddlers is there 0 or more differences in any field including text but except a list of fields eg; title, created and modified fields

  • This would permit a programmatic check between;
    • Importing tiddler and existing one
    • Shadow + tiddler
    • tiddler and source-tiddler

This post may be interesting for you too

Yes very useful documentation @pmario.

I have already started by writing a difference checker between two different tiddler names, by simply comparing each field in both tiddlers with exceptions.

I am trying to build one that

  • Returns true/false in the comparison
    • I come across edited shadow tiddlers that in fact have no differences, if I knew this I could delete the tiddler.
    • I have one wiki where hundreds have this state due to an upgrade mistake
  • Another a count of fields changed/added/removed. I am not bothering with the differences within a changed field just the fact it is different.
  • Writing a JS macro that leverages $:/core/modules/utils/diff-match-patch/diff_match_patch.js would be smarter.
    • but I am not that smart :nerd_face: (yet)
  • It would be useful to show such tools against plugins and the import process
    • Just imagine being told what field(s) have changed, or not, rather than having to go looking.

Anyone else know of “prior art” with this.

Note: The Prerelease 5.2.26 introduces New Diff-Match-Patch Primitives and levenshtein Operator will allow this to be developed, thanks @jeremyruston

tony

in an old wiki 5 15 of mine(March 2018)

you can import your old tiddlers

then when you import the new versions

it allows a preview of all the diffs in $:/import (this is before actually importing)

it can show all the diffs you requested in tabs (delta fields=values, delta shadows)

see http://twjam5beta.tiddlyspot.com/

WimM

@wjamoe very comprehensive. I will keep this reference for when I need it. However what I am looking for is a simple test of each field (except nominated fields) with a change to use programmatically rather than visually.