Filter Question: Return all TOC Tiddler Titles as a Plain List

Using the recursive macro, it is possible to retrieve all tiddler titles in a TOC no matter how many branches and leaves are existed.

With the status of modern filter queries in Tiddlywiki 5.2.5 is there any equivalent filter or set of filter runs?

The kin filter by Bimlas does precisely this. It can retrieve a full list of tiddlers from any node down, up, or the whole tree.

  • It is also possible to subtract one set of tiddlers from the other for example to identify all cousins without counting siblings etc…

I think you could do this with core code, the “cheat” being that you know in advance the maximum number of tagging levels you want.

Hi Mark,
If this is not a recursive macro, then please let me know how it works.

It is a recursive macro, but there is some code, that prevents “endless loops”. eg: A tagged B and B tagged A. …

The macro takes care of that “issue” by tracking nodes, that have already been visited higher up the “tree”. …

Thank you, Mario! I have one and it is a recursive macro.
I was looking to see if this is possible with pure filters and no macro, so I can use it from $:/AdvancedSearch.

1 Like

Update: The initial :map:flat that I had in the example was not necessary, I think. The simple chaining of subfilter calls should suffice.

Check out this discussion and this one from a while back.
That was before :map:flat or :reduce was available, so some of the content may be outdated. But then again, this could mean that it can be made to work nowadays with core filters… (I haven’t read through that old thread again completely).

If you know how many levels deep you want to go at maximum, you could use something like

\define tagger() [all[]] [tagging[]]

<$list filter="[[WikiText]]subfilter<tagger>subfilter<tagger>subfilter<tagger>]" />

This will give you all tagged tiddlers, 3 levels deep. Add more :map for more levels. Add operators in the tagger if you want to filter out some results.

Have a nice day
Yaisog

PS: If I remember correctly, the subfilter (\define) replacements are done before the whole filter is evaluated. Therefore, you cannot have “true” recursion in a filter, as it wouldn’t know beforehand where to stop.

PPS: I think I said that in the other threads, but kin is sloooow for large wikis, especially if you go full recursive with no limit on the levels. That’s why I was looking for other solutions, too.

1 Like

Thank you @Yaisog.

Seems no real recursion is possible with filters.

I disagree with this, see Fun with soft filter parameters and run prefixes - #9 by TW_Tones but there is also this problem with trying to force things into a single filter which is simply unnecessary.

Tiddlywiki may not be a procedural language, and we could even say it is list driven, however the standard global coding structures of sequence, iteration, selection and recursion can all be represented with a key feature being nesting.

  • When we, I and Bimlas, saw there was no mechanism to iterate a full hierarchy within a single filter we developed the Kin operator (his coding).
  • Pushing the ability to iterate complex trees into a filter is one of convenience, because it can already be achieved.
  • The more you cram into a single filter, the less access you have to intermediate values and the opportunity to intervein at different parts of the hierarchy.

Here is a modification of my code found in Fun with soft filter parameters and run prefixes - #9 by TW_Tones that demonstrates how the list from even a “recursive nested macro/filter” can be used to set a variable containing all members of a hierarchy. see <<full-list>>.

\define each-other-level(filter)
<$link to=<<currentTiddler>> ><$text text=<<currentTiddler>>/></$link>
<$list filter="$filter$">
   <<each-other-level $filter$>>
</$list>
\end
\define first-level(filter)
<$list filter="$filter$">
   <<each-other-level $filter$>>
</$list>
\end

Start in Filter Operators<br>
<$tiddler tiddler="Filter Operators">

<$set name=full-list value=<<first-level "[is[current]tagging[]]">> >

<<full-list>>

</$set>

</$tiddler>

Post Script: There are cases where you may need to first wikify the variable full-list before use.

Not unless you write your own filter operators which can do that recursion internally (albeit only for a very specific purpose). Kin does that, but is a bit overpowered and doesn’t work well for large wikis. Take a look at https://talk.tiddlywiki.org/t/recursive-filter-operators-to-show-all-tiddlers-beneath-a-tag-and-all-tags-above-a-tiddler/3814 for two very light filters which do exactly what you need, but nothing else.

I also checked again how far one can take a pure filter approach with TW features that came out since then and came up with this:

\define iteration-depth() 3
\define iteration-source() WikiText

\define reduce-subfilter() [enlist<accumulator>] [enlist<accumulator>tagging[]] +[format:titlelist[]join[ ]]

<$list filter="[range<iteration-depth>] +[reduce<reduce-subfilter>,<iteration-source>enlist-input[]]" />

This uses the reduce filter operator to run the subfilter a given number of times, as determined by the input to reduce. The input itself is not used and could be anything, only the number of items matters. The root tiddler for which to get the tag hierarchy is passed as initial value of the accumulator variable (second filter parameter). The subfilter takes the current accumulator, concatenates all tiddlers that are tagged with one of those listed there, and then packages everything up into a string again. Repeat for the next iteration.

This looks nicer than a string of subfilter<tagger> calls and has the advantage that the depth can be set programmatically – you could call it with different depths and see at which point the output doesn’t change anymore. The disadvantage is that you have to pass the root tiddler as variable instead of as filter input, and cannot just call e.g. [[WikiText]reduce.... For something like that to work we have to use currentTiddler instead of iteration-source and wrap everything in a filter iterator that sets the currentTiddler variable for its content, e.g. :map:

\define iteration-depth() 3

\define reduce-subfilter() [enlist<accumulator>] [enlist<accumulator>tagging[]] +[format:titlelist[]join[ ]]

\define map-subfilter() [range<iteration-depth>] +[reduce<reduce-subfilter>,<currentTiddler>enlist-input[]]

<$list filter="[[WikiText]] :map:flat[subfilter<map-subfilter>]" />

Not so pretty anymore, but this way you can get the tag hierarchies for multiple root tiddlers in a single filter call:

<$list filter="[[WikiText]] [HelloThere]] [[JeremyRuston]] :map:flat[subfilter<map-subfilter>] +[unique[]]" />

Each of the (sub)filters is not very complicated, but together they perform a pretty complex set of operations. And we finally found a use for these new filter operators and prefixes! Yay! Next stop: $genesis. :sweat_smile:

Have a nice day
Yaisog

2 Likes

This is pretty cool! I am playing with the filter and return to you!
I think setting the depth is not a big deal. Testing with depth=3, 4, 10 is very fast on tiddlywiki.com for root=TableOfContents. All 1273 tiddlers is returned (assuming the root tag is returned)