Repeat expression [n] times? A Recursive Experiment

[pmario]

@TW_Tones

Perhaps if we could improve something as important as the $list widget,

Recursive procedures do not necessarily contain list-widgets. So it does not really make sense to add an exit condition to the list widget. It will not solve the underlaying problem. The number of list elements is determined by the filter-expression.

Handling 100000 elements internally is not the real problem. It seems as soon as we do create that many DOM elements “in one go”, we do get a problem.

At the moment there are some pull requests underway, which try to explore the problem a bit closer.

@TW_Tones

• For a recursive process, where one macro (containing a list) calls itself, we could count a depth variable and use that in the exist condition

  • Eg; exit-condition="[<depth>compare:number:gt[100]]
  • But can we find a way to exit all levels? I expect you can in JavaScript.

As I wrote. No list-widgets are involved.

Only the widget-prototype is involved. Eg. The most common problem new users face is {{}} which is the shortcut for <$transclude/> in the tiddler preview pain → It reads its own tiddler content and renders it. So it reads <$transclude/> and renders → and so on.

Internally behaviour like this creates a “nested” widget-tree. So transclusion inside transclusion inside transclusion … The widget-depth can be counted.

At the moment the maximum number of nested transclusions is hard-coded to 1000. That’s OK for the TW UI, which needs about a depth of 60 - maximum. So there is plenty of headroom. – But –

Recursive procedures like the one, discussed in this thread or the Fibonacci series are designed to “behave well”.

They “only” need more time (and memory). So the hardcoded max of 1000 may not be enough.

The main problem is, that the underlaying widget-prototype functions are performance sensitive. Even adding a new if(){} command, at the wrong place, can have performance related side effects.

I did suggest tv-max-widget-tree-depth in an earlier post. That cannot happen.

It did create a general slowdown of creating the right sidebar → More → Tags list at tw-com from ~260ms to >310ms. That’s about 20% increase for “normal” use on my PC.

A different way, that does not have a negative performance impact, introduces a mechanism, that we (the devs) do not really want. It may limit future development “freedom”, because once published we cannot get rid of it anymore, for backwards compatibility reasons.

So a completely new mechanism will be evaluated, which should avoid the need to “publish” an internal “private” variable.

hope that makes sense
-mario

[pmario]

@TiddlyTweeter

For instance the Collatz (above) could be run a long time with pauses? Also the “recommencement number” could be established after, say, 100 loops—so that, in TW, it freshly restarts the loop at that number (retaining the loop-count)? So that potential infinity is postponed?

TW is designed to be executed “synchronous” - What you describe is an “asynchronous” behaviour, which brings a whole new set of challenges. - So yes it could be done that way, but we do not have any APIs in the core, that would allow us to do stuff like this one asynchronously. - Except -

The tm-http-* messages work asynchronously because that cannot be avoided.

[jeremyruston]

Hi @TW_Tones your two opening paragraphs are a lucid exposition of the differences between conventional procedural programming and the declarative programming seen in TiddlyWiki.

@TW_Tones

As we already have the counter variable, consider if we could set in the list widget parameters an exit condition filter, such as if we use the attribute value pair “counter=item”;

The list widget already as a limit attribute for cutting off the list at a given number of entries. How would this be different?

[TW_Tones]

@pmario

Recursive procedures do not necessarily contain list-widgets. So it does not really make sense to add an exit condition to the list widget. It will not solve the underlaying problem. The number of list elements is determined by the filter-expression.

I understand there are various ways to get into a “loop” including transclusions. The idea was that be it in a list widget, or even a special list widget, that the designer could chose to design with the exit limit in mind. This not so much a global solution but an option when the code is at risk of running too long.

• I think most recursions may be able to be “written as nested lists” if needed.

@pmario

The main problem is, that the underlaying widget-prototype functions are performance sensitive. Even adding a new if(){} command, at the wrong place, can have performance related side effects.

I am thinking to use it like a debug or performance monitor where we can choose this code pattern in cases where we may need it. In effect making it possible if not global.

@jeremyruston

The list widget already as a limit attribute for cutting off the list at a given number of entries.

Good point Jeremy, this limit is a good option for many cases but this does limit the input only. If the code pattern then leads to excessive loops or recursion it will not nessasarily make a difference.

• However perhaps I/we should document this use of the limit operator.

Example documentation note

Filters can generate, none, one or many items so be carful when writing TiddlyWiki Script because you could “include” a very large number of titles that may take a long time to process. This may appear to effectively freeze your tiddlywiki.

• One way to combat this is to use the limit operator to ensure the output does not exceed a give number of items. For example if you expect not more than 100 items, putting a limit of 10,000 +[limit[10000]] may protect you from a logical error that generates a long list.

@jeremyruston

How would this be different?

• The limit can be a more advanced filter that acts on any variable or other parameter, such as a recursion depth.
• There is no logical way to determine if the exit condition occurred.
• The filter would be independent of the main filter, and trivial to add or remove. Allowing such a condition to be kept out of the listing, when not in use.
• it’s just an idea to see if we can at least have the opportunity to build solutions that can be coded to test for possible “run away”, or long processing. I am perfectly open to any approach that may help reduce this problem.
  • The simplest version may be a limit number to test against the counter variable, but this is almost equivalent to using the limit operator.