[RFC] Running a difference check on whole of tiddler including fields

[TW_Tones]

[RFC] Request for comment

Arising from Importing a large json file containing hundreds of new tiddlers - #7 by TW_Tones

Has anyone previously made a process to compare two tiddlers, with different titles or one being a shadow and the other an edited shadow tiddler including, comparing all or most field values?

• The DiffTextWidget seems to have sufficient functionality to do this
• You may want to compare all fields except modified and possibly created, creator and modifier
• Ideally one could do this with a filter and detect those that are effectively identical so one of the two can be deleted.

Perhaps a preview field differences would be helpful

• Even more interesting if one of the tiddlers was within a JSON tiddler (non-plugin)

[pmario]

@TW_Tones

Has anyone previously made a process to compare two tiddlers, with different titles or one being a shadow and the other an edited shadow tiddler including, comparing all or most field values

The TW preview can handle that for the tiddler text

[TW_Tones]

Thanks @pmario I was aware of that.

I have also noted that the input process allows examination of differences as well as for fields. Although in both cases they do so graphically. I am keen instead to check all fields and text for differences (excluding created and modified) as these are almost inevitably, ideally with a count, any changes above 0 would mean they are not identical.

This all reminded me of an opportunity to make better use of the preview tools. I have built some interesting tools I will share soon.

• Including making use of other previews I created as well.

When investigating the difference tool I learned we use this library $:/core/modules/utils/diff-match-patch/diff_match_patch.js a public library with API · google/diff-match-patch Wiki · GitHub I suspect there are more functions we could provide access to with a little javascript.

• This includes generating patch content and more.
• See the DiffTextWidget and the note that Within it, the variable diff-count is available.

This is in part what triggered my interest Importing a large json file containing hundreds of new tiddlers but I am pursuing multiple lines of investigation and potential uses. Thus why I am asking;

@TW_Tones

Has anyone previously made a process to compare two tiddlers, with different titles or one being a shadow and the other an edited shadow tiddler including, comparing all or most field values?

Another Question is also is it sufficient to exclude title, created and modified dates and possibly creator and modifier?

[TW_Tones]

The key feature may be given two tiddlers is there 0 or more differences in any field including text but except a list of fields eg; title, created and modified fields

• This would permit a programmatic check between;
  • Importing tiddler and existing one
  • Shadow + tiddler
  • tiddler and source-tiddler

[pmario]

This post may be interesting for you too

HowTo Compare the Content of a Shadow Tiddler With the Content of an Overwritten Shadow

At Tiddlers with JS-prototype tiddlers Scott asks this question after a very compelling story. Check the link and read it Every plugin creates shadow tiddlers at wiki startup. Users can overwrite those shadows so they become normal tiddlers. Check for Overwritten Shadows There is a filter, that is used in the $:/ControlPanel → Basics → Number of overridden shadow tiddlers if you click the “Search Icon” Show all overwritten shadow tiddlers … [is[tiddler]is[shadow]sort[title]] …

[TW_Tones]

Yes very useful documentation @pmario.

I have already started by writing a difference checker between two different tiddler names, by simply comparing each field in both tiddlers with exceptions.

• Your documentation will help me apply this to shadow/tiddler pairs.
• Later I would like to apply it to tiddlers inside a JSON (not just a plugin), more than one JSON format.
  • Clarification needed on what is a JSON tiddler - technical view
  • How do TWs JSON Formats Look Like

I am trying to build one that

• Returns true/false in the comparison
  • I come across edited shadow tiddlers that in fact have no differences, if I knew this I could delete the tiddler.
  • I have one wiki where hundreds have this state due to an upgrade mistake
• Another a count of fields changed/added/removed. I am not bothering with the differences within a changed field just the fact it is different.
• Writing a JS macro that leverages $:/core/modules/utils/diff-match-patch/diff_match_patch.js would be smarter.
  • but I am not that smart (yet)
• It would be useful to show such tools against plugins and the import process
  • Just imagine being told what field(s) have changed, or not, rather than having to go looking.

Anyone else know of “prior art” with this.

[TW_Tones]

Note: The Prerelease 5.2.26 introduces New Diff-Match-Patch Primitives and levenshtein Operator will allow this to be developed, thanks @jeremyruston

[wjamoe]

tony

in an old wiki 5 15 of mine(March 2018)

you can import your old tiddlers

then when you import the new versions

it allows a preview of all the diffs in $:/import (this is before actually importing)

it can show all the diffs you requested in tabs (delta fields=values, delta shadows)

see http://twjam5beta.tiddlyspot.com/

WimM

[TW_Tones]

@wjamoe very comprehensive. I will keep this reference for when I need it. However what I am looking for is a simple test of each field (except nominated fields) with a change to use programmatically rather than visually.