How to add or subtract selective subfilters from a filter

#1

I’m hoping that I can explain what I’m trying to do clearly enough. Please let me know if clarification is needed. I want to be able to filter my tiddlers tagged Railroads, but be able to use something like a checkbox to include or exclude a selection of subfilters. For instance, initially my filter looks like this:

[tag[Railroads]prefix[F]]

But then I only want to look at street railways so I check the include box “Street” and the filter turns into this:

[tag[Railroads]search:rrfield[Street]]

But then I want to look want to look at everything else but street railways so I uncheck the include box and check the exclude box “Street” and the filter turns into this:

[tag[Railroads]!search:rrfield[Street]]

The above is a simplified example as I have many options that I would like to choose from. I could build up a filter that looks like this:

[tag[Railroads]prefix[F]subfiltersubfiltersubfilter]

and then feed the variables with fields. But the problem is these variables cannot be empty and when the checkbox is unchecked it would be empty. If any subfilter variable is blank, the whole filter fails.

I’ve tried different ways of concatentation and macro usage, but I’ve had no luck so far. I’m stumped at this point.

Is what I want to do even possible? If so, how would I approach it?

Thanks in advance!

#2

One way to concatenate filters is with seperate Filter runs, the most common I use is the subtract;

  • prefix -, output titles are removed from the filter’s output (if such tiddlers exist)
    • Eg first filter -[subfiter<mysubfilter>]
    • Where mysubfilter is a variable contining a fiull and valid filter that says what to remove from the earlier results.
  • see Filter Exprelssion for more
  • Eg +[sort[]]

You also mention to add selective subfilters, they are even simpler, just concatenate multiple filter runs and use the [subfiter<mysubfilter>] if you want.

Remember;

  • Remember a filter run is that inside the first “layer” of square brackets
    [ filter run ]
  • you can use the Advanced Search Filter tab to test your filters.
#3

Yes, it’s possible! Here’s how I’d approach it:

  • Use List Mode checkboxes, so that the state of each checkbox will be stored in the same field. Something like this:
<$checkbox tiddler="FilterTiddler" listField="rrfield" checked="" unchecked="Street" default="" invertTag=yes> Street</$checkbox>
  • Construct your filter so that you’re always removing tiddlers you don’t want to see from the broader category.
  • This is why we used invertTag in the $checkbox widget - to indicate that all possible values of rrfield are included by default.
[tag[Railroads]] :filter[{FilterTiddler!!rrfield}!match[]then{!!title}!search:rrfield:some{FilterTiddler!!rrfield}]

What this says:

  1. Start with all tiddlers tagged “Railroad”
  2. if FilterTiddler!!rrfield is not blank, then…
  3. for each title returned by the previous filter run…
  4. keep it only if its rrfield does not contain any of the values in FilterTiddler!!rrfield - i.e., any of the unchecked checkboxes.

I’m assuming from your example that all the values of interest are contained in rrfield, but you should be able to extend the filter to account for multiple field names by using the same sort of filter run.

This is untested and all from memory of the last time I did something similar, so I can’t guarantee it works, but I hope it will be enough to get you started!

1 Like
#4

“when the checkbox is unchecked it would be empty”

Instead of setting the variables to empty subfilters, you could use filters that act like a “pass through”, such as “[is[tiddler]]” (assuming you aren’t looking for shadow tiddlers, of course) or “[has[title]]” (if you also want to include shadow tiddlers).

Note that the pass through filter should be something that has minimal processing overhead, so that it doesn’t adversely impact the performance of the whole filter. Perhaps for your use-case, the pass through filter could be “[tag[Railroads]]” since the tag[...] filter is highly optimized. You’ll have to experiment with some larger datasets to see which pass through filter performs the best for your data.

Let me know how it goes…

enjoy,
-e

#5

The all[] operator with no parameter, by definition, passes the input through to the output unchanged. So it should be usable as an “identity function” in all cases.

#6

What for me works best in such situations is a combination of some of the above ideas:

  • use a :not filter run prefix to remove unwanted results
  • use a $set widget to define a subfilter depending on some condition (e.g. the value of state tiddler)
    • the “pass-through” for :not is just an empty filter run (i.e. remove nothing)
  • use subfilter<> to (progressively) remove the unwanted titles

Because of :not, one sometimes has to use double negation and thus maybe think a little more… :wink:

<$set name="excludeFilter" filter="[[$:/state/search/street]match[exclude]]" value="[field:rrfield[Street]]" emptyValue="[[]]">
  <$set name="onlyFilter" filter="[[$:/state/search/street]match[only]]" value="[!field:rrfield[Street]]" emptyValue="[[]]">
    <$set name="results" filter="[tag[Railroads]] :not[subfilter<excludeFilter>] :not[subfilter<onlyFilter>]">
      <!-- do something with the results... -->
    </$set>
  </$set>
</$set>

If the tiddler $:/state/search/street is set to exclude, tiddlers with the value Street in field rrfield will be removed. If it is set to only, tiddlers without the value Street will be removed. For any other value, the original results will simply be passed through.

Removing nothing is maybe the fastest way to pass through results (in some previous testing, all[] seemed to be slower than I had thought it would be).

Have a nice day
Yaisog

PS: Instead of :not[subfilter<...>] you could also use !subfilter<...> within the same filter run. Separating it into multiple runs looks maybe a bit more tidy…

#7

Thanks to everyone for the very helpful responses. I haven’t had a chance to try anything yet due to intermittent internet (getting fixed tomorrow) and celebrating our 40th anniversary. Hopefully will get a chance tomorrow.

#8

So after experimenting a while, I ended up going with a solution very similar to what @etardiff suggested. It was the simplest, overall solution because of the number of choices I was trying to have. I ended up searching directly for the listField of the checkbox widget. Works flawlessly!

Here’s the final filter:

[tag[Railroads]subfilter{$:/state/subfilter}] :filter[{$:/state/modifiers!!rrfield}!match[]then{!!title}search:rrfield:some{$:/state/modifiers!!rrfield}]

And here’s a screenshot of my interface:

image
image1424×1039 197 KB

#9

Handsome! Looks like a project that might serve very well to showcase how TiddlyWiki can function as a nimble database.

1 Like