How to disable all lists update on every click to make Tiddlywiki work fast with 10000+ tiddlers?

[Yaisog]

Hey @saqimtiaz, thanks for the $refresh widget. I took a crack at improving it in a way that it would (alternatively) allow for an arbitrary filter to control refreshing, not just a subfilter with the list of changed tiddlers as input. A bit like the way that $action-listops works. This way it’s maybe a bit more straightforward to couple refreshing to the existence of some state tiddler. Towards that goal I also added a parameter that inverts the logic and refreshes when the filter evaluates to an empty result.

$ _yaisog_modules_widgets_conditional-refresh.js.json (2.4 KB)

Attribute	Description
subfilter	An optional subfilter expression, which takes the list of changed tiddlers as input. The content of the widget refreshes only if the subfilter produces any output.
filter	An optional filter expression. The content of the widget only refreshes if the filter produces any output.
invertFilter	Optionally set to “yes” to make the widget refresh if the filter does not produce any output. Does not apply to the subfilter parameter.

For dependence on a single state tiddler $:/state/toggle-refresh to not exist, use:

<$conditional-refresh filter="[[$:/state/toggle-refresh]!is[tiddler]]">
    <$log message="Freshly refreshed." />
</$conditional-refresh>

To display the log message only when no state tiddler with the prefix $:/state/refresh/ exists, you could use

<$conditional-refresh filter="[prefix[$:/state/refresh/]]" invertFilter="yes">
    <$log message="Freshly refreshed." />
</$conditional-refresh>

The !prefix[] syntax wouldn’t work here, as it would return a large number of tiddlers.

A beautiful application of this widget will be (I hope) to pause refreshes for the invisible tiddlers in the zoomin storyview, instead of folding / unfolding them, to make the UI more responsive. All I need to do (I think) is include $conditional-refresh in the ViewTemplate and tie it via the filter parameter to the current-tiddler field of $:/HistoryList.
Folding has the disadvantage that the tiddlers need to be rendered anew when they are shown again, which would be slower than refreshing, so switching between tiddlers should become somewhat faster.

Have a nice day
Yaisog

PS: Your code won’t allow to use variables like <currentTiddler> in the subfilter expression. I think you need to change this line:

filteredChanges = this.wiki.filterTiddlers(this.filter,null,this.wiki.makeTiddlerIterator(changedTiddlerTitles));

and replace null with this. That made it work for me.

[saqimtiaz]

Glad to hear you got something working to your liking.

@Yaisog

I took a crack at improving it in a way that it would (alternatively) allow for an arbitrary filter to control refreshing, not just a subfilter with the list of changed tiddlers as input.

You can also simply ignore the input list of changed tiddlers and start your filter with all[tiddlers] or all[shadows+tiddlers]

@Yaisog

The !prefix[] syntax wouldn’t work here, as it would return a large number of tiddlers.

I would probably use something along these lines: [prefix[$:/state/refresh/]count[]match[0]]

@Yaisog

Folding has the disadvantage that the tiddlers need to be rendered anew when they are shown again, which would be slower than refreshing, so switching between tiddlers should become somewhat faster.

The refresh mechanism passes each widget the list of tiddlers that have changed and allows the widget to determine if it needs to refresh or re-render in response to changes in those particular tiddlers. Unless you cache the titles of changed tiddlers for which the refresh was not carried out and force a custom refresh using that data when needed, you would need to re-render anyway.

[linonetwo]

For developers, I have an approach to reduce refresh My approach to allow widget to debounce refresh · Discussion #6654 · Jermolene/TiddlyWiki5 · GitHub

My wiki has 15000+ tiddlers too, so performance is on my mind too.

[Yaisog]

Hi Saq, good to have you back. And in tip-top shape I see…

@saqimtiaz

You can also simply ignore the input list of changed tiddlers and start your filter with all[tiddlers]

I did think about this, but wanted to differentiate the same way that $action-listops does. More semantic, I guess?

@saqimtiaz

I would probably use something along these lines: [prefix[$:/state/refresh/]count[]match[0]]

Oh man, how did I not come up with that?
This is good enough to get rid of the ugly invertFilter parameter. BUT, I’ll use [prefix[$:/state/refresh/]count[]compare:number:eq[0]].

@saqimtiaz

Unless you cache the titles of changed tiddlers for which the refresh was not carried out and force a custom refresh […]

You mean that tiddlers that were refresh-suppressed for a while can be outdated, and just calling a refresh on the tiddler itself will not retroactively apply previous updates? Ah, dang. If I need to re-render anyways, I might as well just stick with folding/unfolding…

[saqimtiaz]

@Yaisog

You mean that tiddlers that were refresh-suppressed for a while can be outdated, and just calling a refresh on the tiddler itself will not retroactively apply previous updates?

Not unless you cache and provide a list of tiddlers that have changed since the last refresh. Each widget refreshes or re-renders itself depending on which tiddlers have changed in the wiki.

[Yaisog]

@saqimtiaz

Not unless you cache and provide a list of tiddlers that have changed since the last refresh.

That would be too much work and too error-prone for the gains to be had, I think. Too bad.

[TW_Tones]

It sounds as if we had a queueing mechanism into which you could push titles then trigger actions on those titles later would be a useful code pattern.

• In this case it would be a refresh tiddler queue.
• However there are many other applications of such a queue we could use if we extended this to push and pop LIFO/FIFO

The key here is we need tools that bridge the gap between variables (memory) and saving to tiddler fields (requires triggers). Imagine pushing and popping items on a memory based queue throughout a session only saving or discarding before closing the wiki.

• The Key difference to any mechanism’s I am familiar with, is this queue can accumulate across time and multiple processes could access it. Without the need to commit with a trigger.