[RFC] Fundamental TiddlyWiki design research - capture add/change of field values

#1

I want to know if there is a practical way to capture all occasions where a field is added to a tiddler, or the value in any tiddlers field is changed?

  • We could all this a “tiddler fields history”
  • There are numerous ways to add/change or delete fields but I am wondering if there is a way such as with an event catcher or message catcher widget that could help us do this globally for a wiki?
  • The worst case scenario is I would have to add this everywhere I manipulate fields customise buttons, intercept changes in the edit template etc… This is what I am trying to avoid.
  • A list of fields that are exceptions to this history may be nominated such as the text field.

[Edited] Update: There appears to be no existing mechanism’s we can leverage, so to address this Original Topic objective I have started to compile a specific solution in this reply and are still open to ideas and alternative ways to achive this.

Background;

I am personally trying to find innovative solutions to “everyday design” needs in tiddlywiki to expose sophisticated solutions using very simple to understand methods. This helps me build even more sophisticated solutions while not hitting cognitively complex and time consuming methods, it also stands to help less experienced users.

It is a result of my research in this area that I have realised if I can “capture all occasions where a field is added to a tiddler, or the value in a field is changed”, or it is deleted, I can introduce some useful features.

  • To minimise the impact on the wiki these changes can be logged in a field within each tiddler itself so if the tiddler is deleted so is its “fields history”
    • unlike if I used a “global fields history tiddler” I do not need to store the tiddler name where the field change occurs because the history relates only to the current tiddler.
    • I would set timestamp=no here so the history mechanism does not interfere with the modified date.
  • This idea is also based on an idea that many tiddlers are either;
    • Static or don’t change much after creation like code tiddlers
    • OR Change a lot, but only within a short lifetime then become dormant or deleted such as a task tiddler.
    • OR change a little gradually overtime and there is value being able trace these changes.
  • I can see in a multi-use wiki that we may also want to track the user who made such a change.

Edited post script:

  • Another possible technical hurdle here is being able to add index or json tiddler entries in a field other than the text field.

Thanks in advanced for your considered response.

How to get transclusion depth?
#2

So, the “what” is audit/history/logging per field.
The “where” is by tiddler field.

But what is the why? I’m not getting it… :confused:

#3

Here is the short version by example.

All it takes to define something that needs to be done is give it an empty done-date. It will be done as soon as it has a value today or earlier.

Lets concider a shopping list item “buy rice” add an empty done-date and it will be outstanding. When you do buy the rice you set the done-date. Then three weeks later you want to “buy rice” again. Just clear the done-date and its outstanding again.

This is a reoccuring item. But each time, you clear or stamp the field you loose its history. If you log these two types of changes you can later determin when and exactly how often you decided to and bought rice. You could even determin how long it took from reordering and buying it, not to mention a summary of each shopping event for every shopping list.

Now concider this method applied to most tiddler status, workflows, changes or reporting.

Note how no other tags or status handling is needed and in fact the verb “buy” may even be what puts it into the shopping list which on listing hides the "buy " prefix.

This whole process is easy to reuse time and again for any field or tiddler prefix without coding and could come with buttons to generate list, sidebars, buttons and other features.

Is that sufficient @CodaCoder ?

#4

Does “tiddler fields” include the “text” and “title” fields? I think any history should include the whole tiddler, which of course will generate a lot of data.

Some (most?) todo apps implement this by making copies of the previous or original item, with some kind of link to the master or meta item.

Maybe the “history” or “log” could be implemented by copying the whole tiddlers?

Let’s say you have a book tiddler, and each time you read it, you want a tiddler with start and end dates, your thoughts for that reading and so on.

“List” the book-reading-tiddlers from the book-tiddler, and you could have a history of every reading.

#5

Thanks for your feedback @cdaven

There is no reason not to make this an option but by default along with modified modifier text title and type I think these should be left out. the intention is to make this available to user or designer designed fields.

  • There are the other ways when you want to clone the whole tiddler
  • The edit and draft mechanisms are designed for this in many ways.

This is in fact something I want to avoid. Keep in mind one tiddler could have multiple fields and this would require a copy for each field change, when instead you just can trace field changes with this idea.

When you read but don’t change a tiddler, such can be logged in a number of ways even using this solution, but this solution enables even simpler ways to achieve similar things, with multiple fields.

#6

I would just add that fields that may indicate a project name or domain (work/Personal) tend to be quite static but retaining a history when and if these values change on a tiddler is very useful.

This is why I would like to see if we can globaly

capture add/change of field values

I am still seeking assistance please.

#7

Bump, I have explained why I need this, but so far I do not have any suggestions as to how we could log the change in field contents.

For example is message catcher on Action set field enough?

#8

Hi @TW_Tones I don’t think that there is currently any mechanism that would allow us to maintain the history of changes to a field.

However, it is something that I’ve been exploring in the background for a few years. The approach I’m interested in would be to change the way that tiddlers are stored to use an append-only log that records the history of every tiddler and field. It’s a significant core change, and requires extensive rework in JavaScript. But the advantages would be compelling: the ability to roll back changes arbitrarily, or browse a wiki at a particular point in time.

#9

This would come at the cost of potentially HUGE file sizes, though. So, ideally there would be snapshots and the ability to delete everything before a certain date. Or a (selectable) maximum number of versions.

I’m looking forward to see what your explorations will yield.

Have a nice day
Yaisog

#10

I think it would be worth to explore the hypercore protocol, that stores data in an append only log by default. It would need a new syncer and some helper functions for TW. But it wouldn’t need other big changes to the TW core itself.

About the OT. … If that “backend” would be used, a cryptographically signed history, would be included “for free”.

… I do have a draft post “an alternative multi-user concept” laying around for some time … So that’s a reminder

1 Like
#11

Suddenly, every twiddlerphile on the planet went…

image

#12

Just to clarify here I am more looking for a mechanism, that I/ we can use to do so if required.

  • I am interested in the larger Idea you and Mario propose but I think there is a much smaller target that would achieve what I suggest.
  • I only want a mechanism to target specific fields and the designer can take responsibility for the resulting logic and log sizes.
  • I would be unlikely to do this on the text field for example.

I am not thinking about anything as comprehensive as you describe. All I want is the mechanism to detect changes to named fields. It is a little like how I can use the message catcher to detect all tiddler delete messages.

  • Imagine for a moment I have a status field on many but not all tiddlers. All I want to do is to log when the “status” field changes value by storing the new value. If I store the logged action in the current tiddler, I do not even need a full text reference such as "tiddlername!!fieldname: new value"
    I can just use "status: new value" because the current tiddler is implied.
  • I would sometimes/most likely use a timestamp as the key as knowing when it happened is helpful.

The solution required I am suggesting would simply need to have a method to trigger a named action macro in all the places a change to a named field can be made.

  • The designer would then code an action that recorded the change.
  • eg actionsetfield , the field editor, and when an edit or edit text widget sets a field.
  • This would be useful even if the designer needs to store the last value to test if the new value is a change. eg; status-last not equal to stauts, then once detected set last-status to current.

One of the key design innovations I am looking at is using date fields in an advanced way such as if something needs to be done, give it an empty done-date (no todo tags or any other logic). When we do it, we stamp done-date with now. In this case we know when it was done because done-date contains when.

  • but what if this was a reoccurring task and I reset the done-date field to blank so it will need doing again?
  • It is here I would like to log the tiddler was first done, cleared and done again.

The log for changes in done-date would read as follows

  • Note I also log when the field was cleared

Stamp | fieldname | new value

logdate done-date "first done timestamp"
logdate done-date ""
logdate done-date "second done timestamp"

I could even define a field called done-date-log to store the log and use;

Stamp | new value

logdate "first done timestamp"
logdate  ""
logdate  "second done timestamp"

[Edited] One strength of this “in tiddler log” is it disappears when the tiddler is deleted, or exported to an archive.

Please ask more questions of clarification if needed.

#13

The problem is, that we don’t know what other users will do. … The motto is “hope for the best and prepare for the worst” … case scenario.

The biggest problem I see, is that the wiki will get bloated. … Then after some time users demand compression algorithms, to make the file size smaller. … and so on and so on.

So I would advice to have a look at git-scm and learn how to use it for local single file wikis. … Whenever you made a major change to your wiki you commit that change to the local repository. Since git only saves “diffs” the repo size should be manageable.

There are some GUIs, eg: GitHub Desktop or SourceTree from Atlassian, which seems to be a cool app with some nice features that could probably help managing several versions of single file wikis.

  • The advantage of such a system is, that it would allow to use a free backup system in the cloud using private repos.
  • TW can be used out of the box.
  • harddisk space requirement are lower than having a lot of backups.
  • repos allow tagging - for major milestones
  • If major changes are planned - feature branches can be created, without modifying the main wiki

and so on …

I did “play with” SourceTree for about 30 minutes, but it looks cool. … It requires to create an account at BitBucket. … Which I already had. … The only problem was finding the PW :slight_smile:

1 Like
#14

Hi @TW_Tones there is no current mechanism that can do what you have asked. While you can indeed trap messages like tm-delete-tiddler that mechanism won’t trap all tiddler deletions (in particular it will miss tiddlers deleted with <$action-deletetiddler>).

“Simply” is doing a lot of work here! It’s certainly conceptually simple, but there is nothing like that mechanism present in TW at the moment, and so you are proposing something new that is not currently supported.

#15

Thanks that is what I needed to know, so I can stop looking.

The next question for me is can it be built?

Well yes it can, all I need to do is to intercept the field editor on selected fields and provide a set of actions to any user interface items that I write that edit that field and not use methods that don’t allow interception.

Thanks Mario for your feedback

  • We already “do not know” and in a few minutes, I can do something with a widget and a macro to do all but destroy my wiki.
  • There are also many ways to cause bloating in tiddlywiki already and we rely on the designer, even user, not to go too far.
  • You may have missed my targeted field approach which I clarified later, the idea is to be able “to do it on any field, but not every field”, perhaps even only a few fields.
    • The designer can control the life of the log data they generate
    • If stored in the tiddler its life ends with the deleting of the tiddler
  • Also note that to get close to what I propose at the moment requires cloning the whole tiddler which consumes more space, than what I am seeking here.

@pmario the diff management platforms like Git and source tree are an opportunity as you say but too me they stray too far from " capture add/change of field values" to provide an immediate solution, and a single file solution. Although I will keep an eye out should this be developed further.

To all!

Based on the feedback so far, I am starting to see other ways to achieve this. I will update the thread shortly but would appreciate any ideas how to do this effectively and/or leverage existing methods.

#16

Triggering an action when fields change in draft mode.

  • This should be simple as the original tiddler fields can be compared to the draft tiddler fields on save. Additional actions can be added to the Done button to first compare the fields and if a change has occurred trigger a log change action.

  • When using select, checkbox and edit widgets ouit side of the edit mode, to alter a field the designer needs to first capture the existing value so the result can be compared to any changes.

    • Using edit or edit-text widgets will detect a change as soon as the user starts typing a value, so a more sophisticated method is required.
    • Perhaps an update mode to edit one or more fields on the current tiddler could make a temporary whole tiddler backup providing the original values against which to compare field values?.

Once a field change is detected

  • We have a tiddlername, a field name, its original and its new values available to us.
    • If the existing value was logged previously we need only log the new value with sufficient information to identify when the change occurred, the tiddler and fieldname.
      • By choosing the context correctly we may not need to log more than when/and the new value eg;
        • $:/fieldchanges/tiddlername/fieldname (contains log)
        • Or store within the current tiddler a field “fieldname-log”
  • One set of actions that is passed tiddlername, a field name, its original and its new values with a parameter would be sufficient to respond to the different log methods; ie the parameters determine what and where to log the change.
    • We need to include clearing the field as a change
#17

Actually yes, we could but only temporarily, for a tiddler being updated so we can compare the original and new values of each field and detect a change. As currently happens in Draft mode, see

But instead we build an equivalent for a tiddler in update mode

Update Mode;

  • When viewing a tiddler from the view template we may wish to edit one or more values / fields without entering the Edit mode.
  • The user can select Update mode which;
    • Makes a copy of the current tiddler
    • when in update mode various fields are made editable in update mode
  • The user can then deselect Update mode which;
    • Determines which fields changed and triggers a log action
    • Delete the temporary copy of the current tiddler
  • Optionally allows any or all changes to be canceled and restore the temporary copy of the current tiddler.

The triggered log action

  • Will be as described in;

If anyone is still trying to follow me can you see a better way?

#18

Bump:

As I have responded here I am looking to develop a mechanism to act on changes to selected fields and give the designer the ability to manage the volume or otherwise. I have outlined how this is a somewhat simple approach and is basically about how does none achieve this although we know there are no existing methods available.

I still hope you can provide some knowledge or ideas here.

2 Likes