Recursive related items using list field

#1

There has been a lot of good discussion closely related to, but not related to my need. I need to very often generate flat list of related tiddlers in a hierarchy, but not using tags where “parents” are identified but rather using list where “children” are identified. Due to this difference, I’m using kin filters but keep hearing about how there are faster ways which I’m trying to take advantage of. Based on a comment that @EricShulman recently posted, I thought maybe that would be what I was looking for, but alas it’s actually slower than kin in my testing.

If anyone has something faster, please let me know!

This is an example of what my outline looks like using the list methodology. Due to the arbitrary nature of these sprawling outlines, I’m inclined to keep using lists instead of tags.
image

Producing this as a flat list using kin takes about 20ms

<$list filter="[[2023-07-29]kin:list:from[]!title[2023-07-29]]">
<<currentTiddler>><br>
</$list>

But doing what I think is the fastest native equivalent - a recursive list takes about 25ms

\define outline-list()
<$list filter="[list<currentTiddler>]">
<<currentTiddler>>
<<outline-list>>
</$list>
\end

<$tiddler tiddler="2023-07-29">
<$wikify name="ordered-list" text=<<outline-list>>>
<$list filter="[<ordered-list>enlist-input[]]">
<<currentTiddler>><br>
</$list>
</$wikify>
</$tiddler>

I keep hearing about the taggingtree being so much faster and I’m trying to get a piece of that! Note that I do need a flat list for navigational purposes (pressing up or down while editing has to go in a “linear” order through the hierarchy).

For testing purposes, here are tiddlers to go along with the macros.

tiddlers.json (1.6 KB)

#2

Have a look at @pmarios TocP tools. It also comes with filter operators.

#3

I’m looking at it ( tocP — Parent based - Table of Content (wikilabs.github.io)) now and it / they seem similar to the other tag variants that the structure is based on a structure of storing “parents” for each “child” like tag systems works, whereas my structure is the opposite in that I’m storing “children” for each “parent”. So I can’t really test it out fully. Do you have any reason to think the performance would be better with an approach here that maybe I can incorporate? I’m not seeing anything in the code that indicates big shortcuts, but I could be definitely overlooking something.

I guess one thing I haven’t tested is if the reverse-structure I’m using is faster or slower than the other. Something I can check for sure, but it’s important that I’m able to retain manual re-ordering which is why I’m setup the way I am.

1 Like
#4

Your second code example (using wikitext-based macro recursion) has some extra overhead that is unnecessary. Instead of using $wikify to capture the ordered-list followed by [<ordered-list>enlist-input[]] (which could be more efficiently written as [enlist<ordered-list>]) to iterate over the result, try this simplified code for a more accurate speed test:

\define outline-list()
<$list filter="[list<currentTiddler>]">
<<currentTiddler>><br>
<<outline-list>>
</$list>
\end

<$tiddler tiddler="2023-07-29"><<outline-list>></$tiddler>
1 Like
#5

If you use or build your own recurcive macros you could parametise the filter so for each tiddler you itterate the child list field. If your know the heirachy is valid you dont need to worry about loops. Other wise you need to make sure you cant be your own grandfather etc…

#6

If you have performance problems in lists you should not use the wikify widget. It’s heavy and the new functions should do a decent job. … But I did not have a closer look at the code. It was just the wikify widget that jumped out.

#7

That’s correct. tocP uses a “parent” field by default instead of the “tags” field. So the structure is built very similar to a tag-based tree.

Your usecase seems to be different – As I wrote in the other post. I did only skim your OP atm.

#8

For things like finding the next or previous item for navigational purposes, it might be more efficient to only find the immediate next and previous as needed rather than calculating the entire list.

#9

@stobot back to your OT the tagging tree operator uses the list tags of current tiddler model;

  • I know you want a flat list of members using your alternate filter where the children are named.
  • Just to illustrate the recurcive list with a different “child” filter I responded below.
  • Wikify can be used to turn this into a flat list, but I hope we could find a better way, but perhaps we could make a custom operator, I am not sure yet.

Noting that taggingtree and tagstree generate the list of member of a heirachy (specifical a tags heirachy) in a single filter, you can do it with a recusive list widget. Only the Kin operator can do it with a different filter.

Here is an example using TW 5.3.0 and nested procedures

\procedure recurse(tiddler filter)
\procedure each-item()
<ol>
<$list filter=<<filter>> >
<li><$link/></li>
<<each-item>>
</$list>
</ol>
\end each-item
<$set name=currentTiddler value=<<tiddler>> emptyValue=<<currentTiddler>> >
Recurse <<currentTiddler>> with <$text text=<<filter>>/><br>
<<each-item>>
<hr>
\end recurse
  • This is designed to have a “recurse” initial / final then each-items, so you can do things before and after the list, as well as for each-item.
  • If the tiddler is not provided it uses the current tiddler.
    • thats why I name the filter: parameter in case “tiddler” is not provided.
  • Because each-item uses a new <ol> or ordered list each level is numbered and indented, removing this and li will produce a flat list.
    • As I raised with the author of taggingtree I would like the result to be as in this example for the TOC
  • It does not check for loops
    • I would love to identify the minimalist loop test I can find.
<<recurse TableOfContents filter:"[tag<currentTiddler>]">>
  • This would replicate the TOC macros use of the tiddlers tagged with each tiddler.
<<recurse Top filter:"[all[current]enlist{!!child-list}]">>
  • This one uses a different filter to iterate the clild-list field if it exists

In my small test case the result looks like this;

Snag_77fa5de

#10

I appreciate the help @TW_Tones , but I’m not stuck on how to build the list (a simplified one was in my OP) - I was looking for methods that are more performant than the ones I’ve already tried.

Removing the wikify (thanks @EricShulman & @pmario ) did get the time down to the times being essentially the same. As I endeavor to be minimalist with plugins, that’s enough to get rid of the kin filter and use what I have.

I suspect that further overall performance gains are to be made based on decisions of pre-calculating and storing some lists (similar to what you’re saying @saqimtiaz) . Right now, “navigating” down an item forces all of the filters to be recalculated. For the expensive ones it might be worth calculating and saving the list in a temp area to save some calculation, though that makes it much more complicated to ensure it doesn’t get used when it should be saved. I’ll post the total code later once I convert it from kin to make it less abstract.