Recursive filter operators to show all tiddlers beneath a tag and all tags above a tiddler

Tips & Tricks
#42

A child is the direct descendant of a parent. A descendant is anyone, no matter how many generations, who is a descendant.

I am not the child of my grandfather, but I am a descendant of my grandfather.

To use CSS as a comparison, it’s like the difference between

div.level1 div.level2 { }

and

div.level1 > div.level2 { }

They are rather unattractive characters, aren’t they? But they are part of one of the longest running broadcast series. They are owned by Fo x Broadcasting, which is worth an estimated $18 billion.

That is why I would urge caution in using their images.

#43

@Yaisog I was using the taggingtree operator for a path or route through all items in the tiddlywiki.com TableOfContents and realise the output of taggingtree does not maintain, or flatten the structure. Imagin you were to step through the top level items first, starting at the top one, now follow all its children, and their children in the order they are found until exhausted now move onto the next item in the top level.

  • The output of taggingtree is not in the above order, making it less useful navigating trees.

I don’t know if this achievable in your code but if you could output the list of titles in the “natural” and “flattened” order, it will have even more uses. The current order is not of much use, so the new order can be default and not need any switches.

#44

Here’s my TiddlyTools <<toc-list>> macro (see https://tiddlytools.com/#TiddlyTools%2FTOC), which walks a TOC tree structure in the manner you have described (“depth-first traversal”) and produces a “flat list” output of links to all tiddlers in the TOC:

\define toc-list(here,max,exclude,level:"1",field:"tags")
<!-- SHOW FLAT LIST -->
<$list filter="""[contains:$field$[$here$]sortby{$here$!!list}] -[subfilter<__exclude__>] -[[$here$]]""">
   <$text text="[["/><<currentTiddler>><$text text="]]"/><br>
   <$reveal default="$level$" type="nomatch" text="$max$">
      <$macrocall $name="toc-list" here=<<currentTiddler>> max="$max$" exclude="""$exclude$ [[$here$]]""" level={{{ [[$level$]add[1]] }}} field=<<__field__>>/>
   </$reveal>
</$list>
\end

Notes:

  • Only the here parameter is required, where here is the top-level starting tag.
    Example: <<toc-list "TableOfContents">>
  • max (optional) allows you to limit the depth of traversal.
    Example: <<toc-list here:"TableOfContents" max:3>>
  • exclude (optional) can be used to provide a filter that ignores specified tiddlers. It is also used internally to prevent infinite recursive loops by ignoring any tiddlers that have already been visited in the current TOC tree branch.
  • level is used internally (along with max) to track the current depth.
  • field:fieldname (optional, default=“tags”) allows you to define the tree structure using a field other than “tags” (e.g., field:parent)
  • You can use <$wikify name="flatlist" text="<<toc-list TableOfContents>>">...</$wikify> to capture the output into a variable that can then be used via [enlist<flatlist>] to perform operations on the resultant tiddler list (e.g., previous/next navigation via <$link to={{{ [enlist<flatlist>before<currentTiddler>] }}}>previous</$link> and <$link to={{{ [enlist<flatlist>after<currentTiddler>] }}}>next</$link>

The macro can also be simplified a bit by removing the handling for max, level and field parameters, like this:

\define toc-list(here,exclude)
<$list filter="""[tag[$here$]sortby{$here$!!list}] -[subfilter<__exclude__>] -[[$here$]]""">
   <$text text="[["/><<currentTiddler>><$text text="]]"/><br>
   <$macrocall $name="toc-list" here=<<currentTiddler>> exclude="""$exclude$ [[$here$]]"""/>
</$list>
\end

enjoy,
-e

2 Likes
#45

Thanks @EricShulman you have documented how to achieve what I was asking for, however I was aware of this approach, and was wondering if the taggingtree filter operator could return the flat list, making it a lot simpler. It is so close to doing this already and with 5.3.0 it could be used in a function.

  • Until taggingtree has this output order, the way you show is the only way to achive this so I am sure it will help others finding this thread by search.
#46

Thank you @TW_Tones for the suggestion. It makes sense to have a returned tiddler order that itself has a use. I had a quick look at the code, and I think one only needs to change

$tw.utils.pushTop(results,intermediate);
intermediate.forEach(function(title) {
	gettagsrecursively(title,results,options);
});

to

intermediate.forEach(function(title) {
	$tw.utils.pushTop(results,title);
	gettagsrecursively(title,results,options);
});

so that the hierarchy is “logged” into the results array as it’s traversed. The traversal already happens “depth first”.
One thing to consider – before I test and change this – is that duplicates are removed. So, when you have tiddlers with multiple parent tags within the hierarchy, they will only show up once when they are encountered first.
Also, how do you plan to make use of this? The returned tiddlers carry no information about where within the hierarchy they are located or at what level, regardless of their ordering, so you would have to have some additional machinery in place to figure this out, which will probably kill the performance gains you get from using taggingtree[].
On the other hand, @EricShulman’s solution will let you easily and selectively perform additional operations while traversing the hierarchy, without necessarily operating on a whole family of tiddlers at once.

Have a nice day
Yaisog

#47

If they list is in the order of traversal, depth first, you can step through the list and visit every tiddler is in its structured order and guarentees you visit every tiddler.

  • the first thing we can determin is position in the list.
  • but lets not forget while navigating this the user will be at a position in the list and we can retrieve additional info from the current tiddler in addition to next and previous.
    • thus we can find parents and siblings etc… Of the current tiddler
  • another key value of the heirachy list counting or computing something for all tiddlers in the heirachy. With the tiddlers in order I may be able determin some subtotals whilst retaining whole of heirachy totals. Such as sum the current tiddlers children

With time I am confident we can achive more, simply by ensuring we flatten the list into a deterministic depth first list. But perhaps there is value having a width first option.

#48

Hi @TW_Tones, even with a depth-first list you wouldn’t know when you reach the end of the hierachy of one parent and the hierarchy of that parent’s sibling starts, would you? It’s all still just one long list of tiddlers, just sorted differently. There is no information in that list about the level within the hierarchy that the current tiddler is located at…
Anyway, I suggest you try changing the code in taggingtree.js as outlined above and see if that’s what you are looking for. If you can, give us a code example or two of how you would use the depth-first list. I’d be happy to add a suffix that controls sorting if it proves useful.

Have a nice day
Yaisog

PS: In the * taggingtree and tagstree Example* at https://yaisog.tiddlyhost.com/, the current-version output list for Jackie is
Marge Patty Selma Bart Lisa Maggie Ling
while with depth-first it would be
Marge Bart Lisa Maggie Patty Selma Ling
How would you know – just from this list – whether Patty is a another child or sibling of Marge?

1 Like
#49

Thanks I have some code patterns I will share. For any given tiddler in a list you can go back to that actual tiddler and discover or count its children and parent, even siblings all over again, and this can just in time.

However the key value of a flat depth first list is you can step through every tiddler exhastivly in that heirachy in an appropriate order.

  • my argument is the current order is even less useful.

I will look at that modification thanks. If I get it working I will share and example.

#50

@Yaisog thanks for your help here, I replaced the code all at the top to all the code in the bottom, however it throws a red error;

After reload this happens

Internal JavaScript Error

Well, this is embarrassing. It is recommended that you restart TiddlyWiki by refreshing your browser

ReferenceError: gettagsrecursively is not defined

Unfortunately my JavaScript is not strong enough to resolve, yet.

And I had to edit the wiki and remove module-type value “filteroperator”

#51

Once I have the depth first order, it will be more practical but here is an example of obtaining additional info;

<ol>
<$list filter="[[TableOfContents]taggingtree[]!has[draft.of]]">
   <$list filter="[<currentTiddler>is[tag]]">
       <li><$link/></li>
   </$list>
</$list>
</ol>

So in the inner list I can determine if the current tiddler has children or not, that is “is a leaf” on the tree, Just by using is[tag]] or !is[tag].

  • From here I could identify the last child and allow the user to move after that (in the full list) to the next sibling/parent/grandparent, without re-walking the tree, just working off the depth first list.
  • Similarly for any tiddler in the tree we can look to see if it has more than one tag, which may indicate a cross hierarchy relationship, or two parents etc…
#52

Hi @TW_Tones, it works for me and it does what it should. Maybe you left out a semicolon somewhere?
Here is a link to a sharing edition TW from where you can drag-n-drop the modified taggingtree.js:
taggingtree.js

You wouldn’t know if it is a parent or grandparent, though.

Acting on this information (by using the current tiddler is the other parent’s tree) sounds rather complicated. I think that many of the edge cases will make taggingtree much harder to use (and to read!) than @EricShulman’s recursive macro above – especially if you want to do it all within a filter run.

Please keep us updated with the advances you make with its application while I contemplate how to best integrate an optional alternate ordering while keeping the filter operator codebase small.

Have a nice day
Yaisog

1 Like
#53

I wonder if we could split off this detailed discussion from the OT?

The update works very well thanks @Yaisog

Although one could work it out, that is not the point. The depth first simply provides a full list of items in the hierarchies “natural order” and as a filter this can generate a list, and be saved in a variable for reuse. As a list you can use the order operators, or the $set index.

  • Using such a list one can then move to the logical next or previous including up or down the hierarchy.
  • There may be other information in the tiddlers who are members of this hierarchy that are used for logic.

I am curious if anyone wants to make use of the default order, breadth-first, it currently returns?

  • With breadth-first I can only see value as an unordered list, eg to calculate something on every tiddler.
  • I reviewed this whole thread to see if anyone mentioned, or made use specifically of the default order. No one did. I think they just want an exhaustive list.
  • I don’t think you need to do anything but make this new version the published one.

I believe the new order in the working version you gave me could be the default and there is no need to provide any further options.

  • Although I am open to alternative views, I don’t expect anyone would want breadth-first for a flat list. Although I really want a depth first, even in a flat list.

Thanks so much for your help @Yaisog, I will return.

#54

Just closing on the use of taggingtree
Both the width first and depth first versions of taggingtree, support the following;

  • Ability to list all tiddlers in a tag hierarchy
  • Save them in a variable or with a button field/tiddler
  • Compare more than one list derived from a heirachy with another
  • East of searching for a title or text in any tiddler within a heirachy but not outside
  • Test for and accumulate values in all of some tiddlers only in the heirachy
  • and more I am sure.
  • allow you to navigate to or step through all tiddlers in the heirachy, width first is not intuitive
  • Test if a tiddler title is anywhere in the heirachy
    *Use tagstree

the depth first

  • All of the above
  • Step through all tiddlers in a suitable order
  • Whilst stepping through in this order one can also determine parents, children etc… but has the advantage of matching the order that got you here.

Perhaps one day we could have a more generic solution we can provide the filter to, not just tagging.

#55

Just a big thank you!! I went to write this for my own use today, stopped, checked the forum, and found that someone had written exactly what I needed. This is perfect for my use!

#56

A post was split to a new topic: Which Method do you use to Monitor Performance