I recently ran into a problem that I haven’t solved yet. But another question that comes to mind is what’s the most complex filter expression you’ve ever tried. How did you discover it, or realise that it required a complex filter expression, and how was this filter expression eventually written. What can we learn from that. Be aware that there is very little discussion of complex filter expressions in the documentation, and very few examples. Either that or they are so complex that they are almost impossible to learn. So if you have a good example, feel free to share it.
The first ones that come to mind are used to generate attendance reports for a chess club wiki. You pick a start date and an end date and I find all meetings occurring between them. Some meetings are online, others are in-person. I calculate all sorts of things, including lines like this:
<$let max-in-person-total={{{ [tag[Meeting]!has[draft.of]] :filter[get[meeting-date]compare:string:gteq<start-date>] :filter[get[meeting-date]compare:string:lteq<end-date>] :filter[{!!meeting-location}match[The Exchange]] :map[get[attending]enlist-input[]count[]add{!!guests}] +[maxall[]] }}}>
It’s not all that complex, but it’s a little intimidating at first glance:
[
tag[Meeting]!has[draft.of]] <!-- all meetings -->
:filter[get[meeting-date]compare:string:gteq<start-date>] <!-- on/after start date -->
:filter[get[meeting-date]compare:string:lteq<end-date>] <!-- on/before end date -->
:filter[{!!meeting-location}match[The Exchange]] <!-- at correct location -->
:map[ <!-- convert each by -->
get[attending] <!-- getting "attending" field -->
enlist-input[] <!-- turning that into a list -->
count[] <!-- counting the results -->
add{!!guests} <!-- and including non-members -->
] +[maxall[] <!-- taking the maximum of list -->
]
I’m sure if I tried I could dig up some more complex ones in my wikis.
2 Likes
This is crazy, very good example, but maybe it could be explained a little more. There’s a lot that I still don’t understand because of some of the usage in there that I didn’t figure out.Like the code after map.
1 Like
It might help to have an example Meeting tiddler. Here is a recent one, with some irrelevant fields omitted, and names anonymized:
attending: [[Boo-Boo Bear]] [[Yogi Bear]] [[Captain Caveman]] [[Fred Flintstone]] [[Pebbles Flintstong]] [[Wilma Flintstone]] [[Huckleberry Hound]] [[George Jetson]] [[Jane Jetson]] [[Bamm-Bamm Rubble]] [[Barney Rubble]] [[Betty Rubble]] [[Ranger Smith]]
caption: May 21, 2025 lunchtime in-person session
guests: 6
meeting-date: 2025-05-21
meeting-location: The Exchange
meeting-time-EST: 12:00
tags: Meeting
title: Meeting/2025-05-21
tournament-type: Friendly
So, assuming the initial tag selection, the rejection of draft tiddlers and the date and location filters are fairly clear, let’s look at the :map section. First of all the map filter run prefix runs a process against every element in your input list. In our case, that process is get[attending]enlist-input[]count[]add(!!guests}. We:
-
get[attending]by getting the fieldattending(which above has all those Hannah-Barbera cartoon characters) -
enlist-input[], by turning that string into a list of strings (which are themselves character names.) In retrospect, I could have combined the first two steps with the simpler,enlist({!!attending}, but I didn’t think of it -
count[], which counts the number of items in that list -
add(!!guests}, by extracting the numericguestsfield and adding the value to the count from the previous clause.
Because of the :map we run this against all the Meeting tiddlers we had chosen earlier, and then we call maxall[] on the result, giving us the maximum value computed from these tiddlers.
I should know this is one of many calculations done on the Meeting data, and the tiddler combines them all into something like this:
1 Like
I don’t know if this is the MOST complex filter I’ve used, but this is one of my favorite complex filters:
[<currentTiddler>fields[]]
:map[<..currentTiddler>get<currentTiddler>else[]length[]]
[<currentTiddler>fields[]length[]]
+[sum[]]
What it does is add up the total number of bytes used to store all the fields of the currentTiddler. It consists of four filter runs:
- The first filter run gets the names of all the fields in the current tiddler
- The second filter run changes these field names to the corresponding length of each field’s contents.
- Note the
else[]operator… this ensures that empty field values produce a length value of 0
- Note the
- The third filter run gets the length of the field names themselves
- The final filter run adds together all the values from the second and third filter runs
You can see this filter, along with other filters for calculating the number of lines and number of whitespace-separated words, in TiddlyTools/Templates/Size. If you copy this tiddler to your own TiddlyWiki file and then edit any tiddler, you will see a line like:
38 lines, 121 words, 1889 bytes
has been added to the EditTemplate, just below the "Draft of ..." heading. Note that these calculations are automatically updated as you make changes to the field contents. In addition, if you click on this line, it will display a popup table that shows the individual lines/words/bytes value for each field (as well as the field names themselves).
enjoy,
-e
6 Likes
One of the most epic headwreckers I’ve had with TiddlyWiki was in my [[Bookcase]], when trying to get a filter to reverse whatever the current sort list is. I had some help from folks in the Discord which I am terribly grateful for.
For context, I use a system tiddler as a staging area for the way the [[Bookcase]] gets displayed. [[Bookcase]] transcludes the contents of this tiddler.
<$list filter="[<thisTiddler>contains:view[$:/templates/bookcase/list-display]]">
<ul>
<$list filter="[tag[Bookcase]tag{$:/templates/bookcase/full-list!!tagquery}subfilter{$:/templates/bookcase/full-list!!sortquery}] ~[tag[Bookcase]subfilter{$:/templates/bookcase/full-list!!sortquery}] +[!is[system]]">
<$transclude $tiddler={{{ [<thisTiddler>get[view]] }}} />
</$list>
</ul>
</$list>
And here’s the reverse filter that gave so much trouble. It is in an action-setfield widget in [[Bookcase]] that sets the sortview query
{{{ [{!!sortquery}search[!sort]search-replace[!sort],[sort]] :else[{!!sortquery}search-replace[sort],[!sort]] }}}
Some buttons have a reverse sort (!sort[]) and some buttons have non-reversed sort (sort[]) … so this filter checks for either or and then uses search-replace to make the desired change.
Another gotcha was having to manually set the currentTiddler
<$tiddler tiddler="$:/templates/bookcase/full-list">
2 Likes
This is a hard question to answer because almost always when a filter expression gets ungainly, I realize I need to maintain readability for troubleshooting purposes, so I start setting up clearly-named variables, custom functions, and/or subfilters.
Subfilters (and these other related tools) are often a game-changer. In any given wiki, there are going to be typical filter patterns. Consolidating a pattern into a “chunk” in the form of a subfilter is great, because then you can tweak the subfilter just once in order to have your modification propagate across all the places where that pattern happens.
3 Likes
Seems like a question worth expanding on. Filters are like friends, we all feel differently. We could compile a list of our personal favourite filter expressions, the filter expressions that we personally find the most complex, the ugliest filter expressions, the most incomprehensible filter expressions, and the most valuable filter expressions.
I myself only encounter a lot of complex filter problems when developing plugins. For general everyday use, I haven’t built too complex filters. Me. So I currently I can’t name my favourite filter expression. But I can name a favourite TiddlyWiki improvement, and that’s the conditional shortcut syntax.
The conditional shortcut syntax has really eased a lot of work for me, and thinking about problems has become a lot easier. The Future Maybe the conditional shortcut syntax can be further enhanced in the future.
Thank you all for the discussion and answers, it’s been really great.
1 Like
Not exactly complex, but long
\define makeSearchExpression(arr, composer, id, instruments, key, movement, movementType, opus, performers, work, workType,)
[tag[movement]regexp:_mvt-arr.[(?i)$arr$]regexp:_mvt-composer[(?i)$composer$]regexp:_mvt-id[$id$]regexp:_mvt-key[$key$]regexp:_mvt-instruments[(?i)$instruments$]regexp:_mvt-movement[(?i)$movement$]regexp:_mvt-movementType[(?i)$movementType$]regexp:_mvt-op.[(?i)$opus$]regexp:_mvt-performers[(?i)$performers$]regexp:_mvt-work[(?i)$work$]regexp:_mvt-workType[(?i)$workType$]]
\end
It constructs a search expression for searching individual movements of classical music works (ripped CDs) in my music player.
If I recall correctly, it’s been a while, the filter didn’t work when split over multiple lines…
3 Likes
Interesting question, and I like @Springer’s answer. However I always act to reduce the apparent complexity in my filters so my best filters do not appear complex M.I.S.S “Make it simple sweetheart” is much better than the KISS principal.
A real boon to this simplification process is now available with functions and custom operators. This allows compound filters to be written in an easy to read way. I consistently try to generalise functions for reuse and move them into global functions and procedures. Then subsequent filters are built on top of those utility functions.
I have also found it quite easy to present ChatGPT with the JavaScript found in a core filteroperator module and ask it to “write one” for me. It is amazing how reducing filters to simpler meaningful ones can make complex filter authorship much easier. We have discussed this in the past and its about reducing the cognitive load when composing solutions.
Eg; Consider balance.fields[creditfieldname],[debitfieldname] to subtract debitfieldname from creditfieldname for all titles given to the filter and that have the fields.
For me the most complex filters are the ones I do not yet understand. See recursive functions in this conversation by @stobot and @saqimtiaz
1 Like
You all haven’t lived until you’ve tried recursive functions. Ever tried to get all the tags of a tiddler, along with all the tags of those tags? and tags of those tags? and so on?
Without the risk of infinite loops?
I don’t have a working example on hand to show because I blocked out the memories of when I did this. They were too painful. But I know I’ve done stuff like this a few times.