Conditional Filtering

I have this dynamic table set up, but I cannot figure out how to incorporate a certain variable in the filter. I have one tiddler that has a field ‘PMI’ and it has ‘x’ in it. How do I have it show up in the list if it fits all the other criteria AND the user selects Yes in the PMI dropdown, but not show up if the user selects No?

Program: <$select tiddler="$:/state/myProgramInput" field="text">
            <option value="Fixed">Fixed</option>
            <option value="ARM">ARM</option>
            <option value="HELOC">HELOC</option>
            <option value="Construction">Construction</option>
        </$select>
Package: <$select tiddler="$:/state/myPackageInput" field="text">
            <option value="Prequal">Prequal</option>
            <option value="Initial">Initial</option>
            <option value="Redisclosure">Redisclosure</option>
            <option value="Pre-Closing">Pre-Closing</option>
        </$select>
PMI: <$select tiddler="$:/state/myPMIInput" field="text">
            <option value="x">Yes</option>
            <option value="">No</option>
        </$select>

<hr>

    <$vars ProgramInput={{$:/state/myProgramInput!!text}}>
    <$vars PackageInput={{$:/state/myPackageInput!!text}}>
    <$vars PMIInput={{$:/state/myPMIInput!!text}}>

<table>
  <thead>
    <tr>
<th> </th>
      <th>Name</th>
      <th>Code</th>
      <th>Stack</th>
    </tr>
  </thead>
  <tbody>
<$list variable=stack filter="[{StackingOrder}enlist-input[]]">
    <$list filter=" [tag[Documents]has:field<ProgramInput>has:field<PackageInput>stack<stack>] " variable="tiddler">
      <tr>
<td><$checkbox></$checkbox></td>
        <td>
          <$link to=<<tiddler>>>
            <$text text={{{ [<tiddler>] }}} />
          </$link>
<$text text={{{ [<tiddler>get[caption]] }}} />
        </td>
         <td>
            <$text text={{{ [<tiddler>get[code]] }}} />
        </td>
        <td>
            <$text text={{{ [<tiddler>get[stack]] }}} />
        </td>
      </tr>
    </$list>
</$list>
  </tbody>
</table>

    </$vars>
    </$vars>

I’ve searched the forum and the tiddlywiki site but I’m still too new at this to get it I guess :frowning:

Welcome to the talk community @mswho42! (Any relation to mswhatsit? :wink: )

I am not positive I understand your requirements, but my best guess says that this would fix it:

<$list filter="""
  [tag[Documents]has:field<ProgramInput>has:field<PackageInput>stack<stack>]
  :filter[get[PMI]match<PMIInput>]
""" variable="tiddler">

All I’m doing is adding the filter run :filter[get[PMI]match<PMIInput>] to the end of your filter expression. You’ve already done the work of converting the Yes into an x for your variable, so we can do a simple match test. (Note that the triple quotes and the spread into multiple lines is only to make the change more obvious. This would work just fine on a single line.)

Please let us know if that captures your requirements correctly. Good luck!

Second cousins, twice removed I believe? LOL :wink:

The filter run isn’t working for me, but I’m sure I didn’t explain the situation clearly enough. I created an example site: https://mswho42.tiddlyhost.com/. Tiddler ’ Worst Case Amortization Schedule’ should show up when the Program is Fixed and the Package is Initial, but only when PMI is also Yes.

Thank you for putting up the example. It almost always helps, and is worth it if it doesn’t require a huge effort.

Again, I think I get it. I would approach this with two filter runs in one expression. The first includes the Document if it does not include the PMI field. The second one includes it if has the field and its value matches your PMIInput variable:

    <$list filter=""" 
        [tag[Documents]has:field<ProgramInput>has:field<PackageInput>stack<stack>!has[PMI]] 
        [tag[Documents]has:field<ProgramInput>has:field<PackageInput>stack<stack>PMI<PMIInput>]
    """ variable="tiddler">

I added the PMI field to the “Worst Case” tiddler, with value of x, and it hides and shows as you toggle the PMI dropdown. I’m pretty sure that this captures some of the cases.

But I don’t know exactly what you want in the case the tiddler has the field PMI with a blank value. If that can’t happen, then this may not matter. Right now, it would show that document regardless of the toggle. We could change the !has[PMI] to !has:field[PMI], and then it would show depending on the state of the toggle. If neither of these behaviors is right, then we might need to fill in this chart to determine what to do:

PMI field Toggle Include?
(no field) Yes No
(no field) No No
x Yes Yes
x No No
(blank) Yes ?
(blank) No ?
(other) Yes ?
(other) No ?

The (blank) and (other) can likely be combined into other, but I’d want to be sure before making that call.

Our two cases should be mutually exclusive, but if for some reason they are not, we can add the filter run +[unique[]] to the end of the filter expression…

2 Likes

Wow, yeah it would be helpful to have the PMI field on the Worst Case tiddler wouldn’t it? :roll_eyes: That’s my bad.
Tiddlers should only have the PMI field if it’s going to have an x in it, so your code looks like it’s going to work flawlessly!! Simple and brilliant!! I can’t thank you enough!

2 Likes

Clearly, you’ve got yourself set up with html tables, framed with variables in such a way as to output dynamic content. And @Scott_Sauyet has helped out with your specific challenge.

In case it’s useful, I’ll point out that your phrase “dynamic table” always makes me think of this gold-standard tool: Shiraz dynamic tables.

In case you’re interested, the dynamic tables in Shiraz plugin allow super-fast creation of (sortable, live-editable) tables in row=tiddler column=field structure. So in your case all you’d need to type is something like:

<<table-dynamic 
caption:"Shiraz style of dynamic table"
filter:"[tag[Documents]has:field<ProgramInput>has:field<PackageInput>]"
fields:"tbl-expand tbl-checkbox title caption code stack"
>>

which, for your mockup data which I dragged to Shiraz demo site, looks like this (with your html tables below for comparison):

The advantage is that you can change the filter or the set of fields on the fly — or even dynamically! — and sort by any of the columns by clicking on its header. Details for any row can be expanded with a tbl-expand toggle. It’s also got some summary footer options. And, anything can be edited in place.

Lots of customizability is possible (in how things appear, what the templates are for column headers, cells, etc.), so even if something doesn’t look exactly as you’d want, odds are that this tool can be tweaked so that it gets you the look you want, AND allows intuitive modifications once configured…

[Edit to add: as noted on another thread, one tricky frontier with dynamic tables is how to get something like your column for the stack field to sort according to a custom “brute” order stored elsewhere (without converting that sequence into something machine-sortable, say by adding numbers to each line). I can get a simple(ish) function to do what you want, but still can’t (yet) get the dynamic table to sort by that order without adding layers…]

Of course this may not be the right tool for your task, and you’ve got a working solution already!

But I always give a shoutout to Shiraz as one of the best packages for upgrading the interface of a TiddlyWiki and getting spreadsheet-like harnessing of fields.

4 Likes

It really is a great tool! I came across it while trying to build my table initially, but honestly it felt above my current skill level. It probably can do everything I need and then some, so it’s still on my radar for sure.

Later on after @Scott_Sauyet posted the answer I realized it unfortunately is not perfect. It does solve the problem I had, but in a way created another one. The ‘Worst Case Amortization Schedule’ tiddler needs to always show if the program is ARM, regardless of PMI status. The PMI field is only a factor on Fixed programs. So if Program = ARM and PMI = No, the tiddler isn’t showing up but should. I’m researching on whether there’s a simple If/Then way of doing filters but keep finding stuff on a higher level again LOL.

1 Like

If you haven’t gotten it by then, I’ll try to look tonight. But this should have a similar technique to what I did above. Just add another filter run which captures your new case. (And possibly add +[unique[]] to the end in case one is selected by multiple rules.)

(It might help if you added a few more test-cases to your example as well.)

I cloned the Doc Package Check to tinker on. I’m struggling but I’m trying :slight_smile: I also added some more documents but honestly the Worst Case tiddler is the only real issue. At least that I’ve come across haha

I’m sorry if I added to your workload by suggesting more Documents. I misunderstood the rules in my quick reading earlier and thought we’d need more tests.

I got a version I think is working. But there’s magic involved that I don’t quite understand. I had a fundamental misunderstanding that I didn’t recognize until after I had developed this version.

<$let
    doc1={{{ [tag[Documents]has:field<ProgramInput>has:field<PackageInput>stack<stack>!has[PMI]format:titlelist[]join[ ]] }}}
    doc2={{{ [tag[Documents]has:field<ProgramInput>has:field<PackageInput>stack<stack>Fixed[x]PMI<PMIInput>format:titlelist[]join[ ]] }}}
    doc3={{{ [<ProgramInput>match[ARM]] :map[all[tiddlers]tag[Documents]has:field<ProgramInput>has:field<PackageInput>stack<stack>format:titlelist[]join[ ]] }}}
>
    <$list filter="[enlist<doc1>] [enlist<doc2>] [enlist<doc3>] +[unique[]]" variable="tiddler">
      <tr> <!-- ... --> </tr>
    </$list>
</$let>

The magic had to do with this: I had ignored the whole stack mechanism and thought each of my filter runs as selecting a group of documents. So in my <$let>, I formatted the elements of the group and joined them into a title list. Then when I used them in a filter, I used enlist to reconstitute the individual items from these lists.

However, once I realized that I was only choosing an individual item, I tried to remove the titlelist and enlist, and it broke things. So I restored them, but they feel odd. I haven’t spent the time to determine why it broke, since I really think that regardless of whether Shiraz is used, @Springer is right that this could be better done by selecting all matching documents, and them sorting them appropriately using your StackingOrder. So I will turn my attention to that thread for now.

But first, I do want to point out that in doing this, I made a discovery in doing this that makes me very happy. I’ve often wanted in a filter some sort of conditional check that’s doesn’t involve the current titles. I’ve never figured out how to do this. I managed it here: I wanted the results of a filter only if the
value of the variable ProgramInput was ARM. I used :map to make that work, starting :map's filter with [all[tiddlers]]:

[<ProgramInput>match[ARM]] :map[all[tiddlers]...more[criteria]]

This is a problem that’s stumped me for some time. I’m very glad to have a technique for it. So thank you for your interesting challenge!

I’ll just caution that this will work if and only if you only want the first result of the :map filter, as “vanilla” :map returns only one output per input value. Your working example uses format:titlelist[]join[ ] to join all the results of the :map run into a single title list, so this didn’t become an issue — but I wanted to mention it in any case, lest someone else try it and find they’re missing most of their expected results.

Luckily, you can recapture all the results of the :map run by using :map:flat instead… or (my preference) switching to :then in place of :map. I made a quick demo with a simpler filter to illustrate these differences:

Code
`[[A]match[A]] :then[tag[HelloThere]]`

{{{ [[A]match[A]] :then[tag[HelloThere]] }}}

`[[A]match[A]] :map[all[tiddlers]tag[HelloThere]]`

{{{ [[A]match[A]] :map[all[tiddlers]tag[HelloThere]] }}}

`[[A]match[A]] :map:flat[all[tiddlers]tag[HelloThere]]`

{{{ [[A]match[A]] :map:flat[all[tiddlers]tag[HelloThere]] }}}

As an an additional bonus, you can omit the all[tiddlers] step when using :then, as (unlike :map) it doesn’t take the output(s) of the previous run as its default starting values.

2 Likes

Oh, even better! Clearly I didn’t really understand why I got it to work. I was just so happy when I did, as I’ve missed this arbitrary-predicate-in-a-filter mechanism for some time. Thank you for the much improved version!

3 Likes

Just for closure on my own rabbit-hole here:

I realized that I wanted to articulate how to generate @mswho42’s custom-sorted list more intuitively (that is, rather than using two nested lists as you did, with the outer one serving only to generate sort order).

So, here’s what I landed on (though @etardiff and @Scott_Sauyet may have even more elegant solutions!):

Have a tiddler tagged $:/tags/Global with the following content:

<$set name="stackwise" filter="[{StackingOrder}enlist-input[]listed[stack]is[tiddler]]">

Then in any tiddler the following kind of thing will get a filtered list of tiddlers sorted according to that list:

<$list filter="[tag[Documents]sortby<stackwise>]"/>

(Replace tag[Documents] with any desired filter expression.)

I still don’t have a solution for sorting Shiraz dynamic tables according to any column/field’s appropriate custom sort-order (where different columns can use different custom sort-orders), but that’s a longer-term itch!

2 Likes

No worries at all, because it took, like, 2 minutes. Thank goodness for drag and drop tiddlers! LOL!

Wow you all are amazing!! I’m going to be studying this for a while so please be patient while I catch up. I want to understand it all before I implement it/ask questions. Thank you all so much for the quick replies, too!!

I was curious about how to get this done. If you want to see how I managed it, you can expand the following. If not, I won’t be offended. It is a combination of many ideas discussed in this thread, with my own input plus that of @etardiff and @Springer

Scott's version

Here is an alternate version of your tiddler:

Doc Package Check Alternate.json (1.9 KB)

You can download this and drag the resulting file onto your wiki. It’s got a new name and shouldn’t overlay anything. It’s also self-contained; there are no changes elsewhere.

I made a number of changes:

Inlined variables

This was actually my last step after everything else was working. Your version has

    <$vars ProgramInput={{$:/state/myProgramInput!!text}}>
    <$vars PackageInput={{$:/state/myPackageInput!!text}}>
    <$vars PMIInput={{$:/state/myPMIInput!!text}}>

    </$vars>
    </$vars>
    </$vars>

First of all these could probably be better handled with a single LetWidget. But I found that point moot, because at the end I only used each variable once. To me that means that the only reason for declaring these variables is if it makes the code more understandable. That happens often enough, but here, I found that it was more understandable to replace <ProgramInput> in the filter with {$:/state/myProgramInput!!text} And because text is the default field, we can skip that and just use {$:/state/myProgramInput}. This is a choice that I tend to make, but there is absolutely no objection if you still prefer the variables.

Selection logic for PMI

The reason for your question was to handle the additional complexities in document selection. Here’s my current understanding of the special rules: If the Program selection is Fixed, then a document that has PMI: x is included only if the PMI dropdown is Yes. If the PMI dropdown is No, then it should not be included. However, if the Program selection is something other than Fixed, then a document with PMI: x is included regardless of the value of the PMI dropdown.

This seems to affect only one document in the list (Worst Case Amortization Schedule)Doc Package Check Alternate.json (1.9 KB)
, but that doesn’t matter to the logic.

Here’s how I handled it:

    <$let maybePMI={{{
      [{$:/state/myProgramInput}match[Fixed]] 
      :then[has:field[Fixed]!PMI{$:/state/myPMIInput}] 
    }}} >
      <$list filter="""
        [tag[Documents]has:field{$:/state/myProgramInput}has:field{$:/state/myPackageInput}] 
        -[<maybePMI>]
        +[sortby<stackwise>]
      """ variable="tiddler">
        <tr> <!-- ... --> </tr>
      </$list>
    </$let>

We’ll discuss the sort bit below.

What we want to do is to include all potentially relevant documents, then remove Worst Case if it meets our somewhat complex condition. However, to handle that complex condition, the only way I found was to use two filter runs, and the filter syntax doesn’t have any parentheses to group those two runs. So I define it ahead of time as a variable. (maybePMI. You might have a more appropriate name; I called it foo until I came to write up this post, because I didn’t have a name for it.) The way that definition works is based on the realization I mentioned in a previous post, corrected by @etardiff’s explanations: We test if the current ProgramInput is Fixed in one filter run, and in the next one, we use the filter prefix :then to select the tiddler we want to remove.

Then in the filter for the <$list>, we select all potentially relevant documents and remove the oddball one. You’ll notice that we don’t use stack in here. That’s coming up next.

I split the filters for both the <$let> and the <$list across multiple lines. And when I did so, I replaced the one double-quote for my attribute with triple double-quotes (""".) I don’t know if others do the same thing, but I used the triple quotes with multiple lines just as a visible reminder that the attribute goes across multiple lines. This could be plain double-quotes, and it could all be on one line; your choice.

Finally, there is a bug in here. It only works completely correctly because there is only one document to remove. We might be able to patch this up with <$set> in place of <$let>, and possibly some enlisting, but I won’t try that. @etardiff made useful suggestions before. Perhaps she can offer clean-up here.

Sorting

The biggest structural change was offered by @Springer. The original mechanism was awkward: We iterated through all the stacks, and reran our filters each time, including also a test for matching the current stack. It would be much cleaner to select the right tiddlers and then run a sort on the result.

I was trying another approach for the sorting than what @Springer suggested. I thought @etardiff’s indexof was gong to be enough, or my version of the same, but I never did manage to get that version working properly. However, the version from @Springer works perfectly.

Instead of her suggestion of putting our sorter in a $:/tags/Global context, I just use it locally. If you need it in multiple places, then do move it to a global location.

All that was necessary was to wrap the contents inside a a

<$set name="stackwise" filter="[{StackingOrder}enlist-input[]listed[stack]is[tiddler]]">
<!-- contents here -->
</$set>

and to sort the filtered document tiddlers with

        +[sortby<stackwise>]

If that sorter is not clear enough, please ask. I’m sure @Springer could explain. Or I could, although it took me a few minutes at first to understand. I still want an indexof one working correctly though. Perhaps one day.