How to Delimit more than one filter in a string or field?

Folks,

I have come across a case where I want a field or variable to contain a list of filters/filter runs. Yes more than one filter/filter run!

  • I am making a macro that will take this field or variable and split it into one or more filters and act on each.

Question

What is the best way to delimit each filter so they can be split apart and dealt with one “filter run” at a time?

  • For example we may seperate them with a comma [filter],[filter2],[filter3] +[sort[]] however we may want to use a comma in the filter.
  • I know for a fact we can’t use “|” in filters, I call it “pipe”, as I need to \define pipe() | to use is as <pipe> in filters, so I could use pipe as the delimiter eg; [filter]|[filter2] | [filter3] +[sort[]]|[tag[test]] but then the filter will not be useful unless it is spit.
    • This may be good or bad depending on the case, in filtered transclusions or transcludins this format clashed with the template format
  • Perhaps I could use a unicode character or & symbol?
    • See the idea of a new filter run in more

More
  • This is specifically to drive actions through a macro and the ActionListopsWidget
  • The idea is a single string, field or variable could specify a set of actions to perform, cool idea don’t you think?
  • I note at Named Filter Run Prefix we could “create a new filter run prefix”
    • so what if I did create a new filter run prefix of “:&” to delimit filter runs like some command line tools use this to delimit a set of commands on one line.
    • Thus I would delimit each filter as follows :&[filter] :&[filter2] :&[filter3] +[sort[]]|[tag[test]] that is I split on “:&”
      • Would this filter run work? &[filter3] +[sort[]] because really it is two runs?
      • and perhaps we can improve this filter run later?

I don’t quite understand what you’re after, but why can a space character not be the delimiter? And if not that, then maybe ][, i.e the end of one filter and the start of next, can be used as the identifier of where one ends and the other starts.

Sure! BTW, many widgets with an action attribute demands that the actions are written as one string.

Thanks for your reply but.

I know of filters that contain spaces inside a tiddlername, field or variable name. Perhaps we could use ][ or ] [ but then we cant delimit filters containing more than one run, a list of titles is multiple filter runs and some do not need [].

Yes thats why I want to be able split a set of filters in to more than one single action string.

An example may be changing a to do task with a due date into a reference tiddler, removing due. where this step has one or more operations.

  • I am still trying to work out specifying tiddler name, althought current tiddler is usualy ok
  • and operating on more than one fieldname because we often need to provide the fieldname to a $field parameter. Perhaps my delimiter can contain a fieldname to which to apply the action?

Tony

I regularly use :::: as my delimiter.

Reasonably safe (I haven’t yet run into conflicts), easy to enter because it is just a keyboard character, and it sticks out fairly well.

I will look at your suggestion of :::: @Charlie_Veniot and test it a bit thanks.

Interestingly it is treated as a tiddler title.

<$list filter="[tag[$:/tags/Macro]]::::[tag[$:/tags/ViewTemplate]]">

</$list>

But the idea of a multi-character even word isd interesting.

As you found out splitting filters is complex. Even more so if it should be backwards compatible and future proof (forward compatible).

A simple filter run following below will break all your suggestions for “how to achieve” a split, since all possibilities discussed in the OP will fail.

[[tid with spaces]] tid-1|x [enlist[tid-2,something a&b :&tid-name]]

The only thing possible would be a new filter prefix … BUT … it will only work for your specific usecase, since it would need to be applied to every element above as an “identifier” to split. … So that will only add complexity for no real win.

On the other hand the core already has a parser that identifies filters as created above. If you copy my example into the AdvancedSearch → Filter tab it will be resolved perfectly fine.

If I do understand your OP right you want to transfer a filter string like this

[[tid with spaces]] tid-1|x [enlist[tid-2,something a&b :&tid-name]]

into something like this

tid with spaces 
tid-1|x 
[enlist[tid-2,something a&b :&tid-name]]

Where every line if used with a list-widget would create a valid “subfilter” string. … Is that right?

Thanks all, this is a work in progress and evolving.

On further reflection,

  • lets call these “filter lists”, ie a list of filters.
  • and when a filter list is “parsed” it just results in a number of “filters”.

On considering @twMat and @Charlie_Veniot’s suggestions I have come to realise that any “unlikely set of characters” even “a word” can be used to delimit a set of filters, and in fact if used in error at worst they become a tiddler title.

Keep in mind I hope to create a string, field value or variable that contains one or more filters (that is why they need to be delimited) that can be pragmatically converted to generate an action, as in the ActionListopsWidget, I now realise the following

  • I need to also provide a fieldname along with “each filter” in the Filter List.
  • This is because given the way filters are used, and specifically the filter and subfilter parameters in the ActionListopsWidget . I have come to realise that I will need to pass a fieldname along with each filter in a filter list.

For example

The filter [[done]] [[archive]] -[[todo]] along with the fieldname tags will add two tags and remove another.

  • Then also consider the field reference-date the filter [<now YYYY0MM0DD0SS0hh0mm0ss0XXX>] will stamp the date now on reference-date.
  • So I want a string something like (tags)[[done]] [[archive]] -[[todo]] (reference-date)[<now YYYY0MM0DD0SS0hh0mm0ss0XXX>]

At the following tiddlyhost site I have my POC, you can see where I am taking this.
https://sandbox-by-tones.tiddlyhost.com/#Test%20action%20listops

  • So far I am only applying single filters, some with multiple runs, and naming the field in another parameter.
  • almost, but I expect the above illustrates this better now.
  • what you show is dividing it into “filter runs”, which is a useful method to understand.
  • however imagine I want the first line to be done archive or [[done]] [[archive]] which in the listops subfilter will add these two tags. I need to delimit one or more “filter runs”

[Edited] To either blow you mind or drive you bat crazy I can now see also further delimiting a complex filter into one or more buttons.

⦅reference⦆(tags)[[done]] -[[todo]] (reference-date)[<now YYYY0MM0DD0SS0hh0mm0ss0XXX>] ⦅archive⦆(tags)-[[done]] [[archive]] -[[todo]] (archive-date)[<now YYYY0MM0DD0SS0hh0mm0ss0XXX>]

  • Which would result in a reference and archive button and their actions.
  • An optional display when filter also comes to mind.
  • I just need to build my string parsing tools

@pmario I would like to know the code to “seperate filter runs” as in your example however.

It needs some core changes to be able to deal with filter expressions as shown in my example. At the moment the core doesn’t provide an API to split filter stings like this… It’s a “special” enlist-like filter operator, that is able to create those lists.

The problem is, that all core draggable list-macros use the action-listops widget with a hadcoded enlist[xxxx.

So my changes would need to be a plugin and duplicate the code for draggable macros with new names (my usecase), … I didn’t see a possibility to get it merged to the core without a long discussion about its usefullness.

My usecase is to be able to create drag & dropable lists, that create valid filter-strings, that can be stored in fields. The filter strings should be compatible, so they work in the AdvancedSearch → Filter tab out of the box.

I want to use that mechanism to create a story-editor where 1 tiddler can contain several “list-like” fields which represent different stories.

IMO stories would be handy with the newly proposed thread here at talk. Drawing inspiration from Kodemo

I do have the finished code, but my implementation of the story-editor UI turned out to be completely unusable for most users.

2 Likes

I don’t know what you’re doing or why, but I wouldn’t do that.

For me, delimiters would be useful in this way:

<$let filterBig="[tag[$:/tags/Macro]]::::[tag[$:/tags/ViewTemplate]]"
           filterPart1={{{ [<filterBig>split[::::]nth[1]] }}}
           filterPart2={{{ [<filterBig>split[::::]nth[2]] }}} >

! Big List

<$list filter={{{ [<filterBig>split[::::]join[ ]] }}}>
{{!!title}}; 
</$list>

! Part 1 List

<$list filter={{{ [<filterPart1>] }}}>
{{!!title}}; 
</$list>

! Part 2 List

<$list filter={{{ [<filterPart2>] }}}>
{{!!title}}; 
</$list>

</$let>

@Charlie_Veniot thanks for you review, but yes I did not explain enough for you to understand and that example would mislead you.

  • This was only to demonstrate what would happen if a “list of filters” delimited by you suggestion :::: was used directly as a filter. It adds another list item [[::::]], and its just another filter run

My intention is to use a “list of delimited filters” to pass into a macro which separates them and acts on each independently. But if used incorrectly I just want a bad listitem at most, not an error in the filter. in this case [[::::]] will appear in the list.

A preview can be found here Test action listops , where each button demonstrates passing a “single filter” into a macro that creates a button that does what the filter asks, using the action listops widget.

  • Once I decide on a delimiter. I will create a macro to split the filters and use them to define one action each. Allowing for example for tags to be changed and a date stamp applied etc…
  • This would in effect create one button to perform multiple actions specified in a single string/field/variable.
  • However, as mentioned, I realise I somehow need to label each filter with the fieldname it relates to, because we need to set the $field parameter in the ActionListOps widget.
    • I am still thinking about this issue.

If you follow and have an idea, let me know, but I would understand if my notes are too hard to follow, sorry.

When I have a working solution I hope it will prove simple to use and very powerful.

If you want delimiters, then the macro (or whatever else) should always check for delimiters. If there are none, then handle that. If there are delimiters, then handle that.

With the delimiters, remove them and replace with spaces if you want the entire guacamole for the filter, or use the delimiters to split apart the large filter into pieces.

I’ll have to let somebody else try and help you out on this, because I am clearly not following at all, so whatever ideas I’m having are junk.

Hi all,

Thanks for your feedback. I have answered my own question inspired by your suggestions.

I am using two delimiters, one to seperate multiple filters “&”, and one to seperate a target fieldname from a filter ::. I then just split this as needed in a macro and generate the appropriate action widget. eg:

field name:: [<now [UTC]YYYY/MM/DD>] & tags::[tag[$:/tags/Macro]]&tags::+[toggle[tagname],[off]]

with
<<split-filter "field name:: [<now [UTC]YYYY/MM/DD>] & tags::[tag[$:/tags/Macro]]&tags::+[toggle[tagname],[off]]">>

If you want to see the experimental code its here Sandbox by Tones — On the TiddlyWiki platform V5.2.3

@TW_Tones, I know you marked this discussion as solved, but I just came across it and decided to add my 2 cents.

From your original problem statement it sounds like you wanted to write a filter whose output will be a list of filters and your chosen approach is to embed a separator character and then use the split filter operator to convert the string into a list of filters.

The cascade filter operator is another place where it is required to provide a filter which generates a list of filters. For typical uses of that operator, the filters are stored in separate tiddler fields and a filter which selects the contents of those tiddler fields serves the purpose of being the “filter generating filter”.

Last year when I wrote the cascade filter extended example, I wanted to be able to use a single string to generate multiple filters. I thought of using a delimiter with split like you are doing, but I worried it would complicate the explanation. I settled on storing the filters in separate tiddler fields.

But just last week I discovered the syntax for filters allows for a list of filters to be concretely specified in a single string. The trick is that when filter runs are delimited by single or double quotes, then what is inside those quotes will not be further parsed for filter syntax. That means you can include square braces and other punctuation required for specifying filters.

To be concrete, pretend you want to generate this list of 3 (not particularly useful) filters:

  1. [all[]] :map[split[]first[1]] - returns the first character of each input title
  2. [all[]] :map[split[]last[1]] - returns the last character of each input title
  3. [length[]] - returns the character length of each input title.

Using quote characters you can write a filter which will return the above three filters like this:

"[all[]] :map[split[]first[1]]" "[all[]] :map[split[]last[1]]" "[length[]]"

Paste the above into the filter tab of the advanced search and you will get the 3 filters as output. All without using any “home-grown” delimiter. The quotes are the delimiters and tiddlywiki filter parsing natively supports them.

In later posts you reveal that you also want to associate each filter with the name of a field to operate on. IMO to “go with the grain” of tiddlywiki, it is best to keep those names as a separate list and use the setmultiplevariables widget to tie the names together with the filters.

For example, the code for your “reference” button could look like this:

<!-- For convenience of this example, set variables using the let
widget. Maybe in your real use-case the variables would be input
parameters to a macro -->
<$let
  referenceFiltersFilter='
    "[[done]] [[archive]] -[[todo]]"
    "[<now YYYY0MM0DD0SS0hh0mm0ss0XXX>]"
  '
  referenceFieldsFilter='tags reference-date'
>
<$setmultiplevariables
  $names=<<referenceFieldsFilter>>
  $values=<<referenceFiltersFilter>>
>
<$list filter=<<referenceFieldsFilter>>>

Insert your own action list-ops widget here which will set the
field ''<$text text=<<currentTiddler>>/>'' using the filter
''<$text text={{{[<currentTiddler>getvariable[]]}}}/>''

</$list>
</$setmultiplevariables>
</$let>

Output of the above looks like this:

Insert your own action list-ops widget here which will set the field tags using the filter [[done]] [[archive]] -[[todo]]

Insert your own action list-ops widget here which will set the field reference-date using the filter [<now YYYY0MM0DD0SS0hh0mm0ss0XXX>]

There are pros and cons to this approach and it might not fit your use case. I just wanted to share my new-found way of using quote delimiters to write filter-generating filters.

2 Likes

@btheado thanks very much for discovering this quote delimited list of filters mechanism. It is a surprising fact, thinking not so much outside the box, but outside the delimiter :nerd_face:

  • That in itself will provide hours of fun.

Yes thanks for taking this further and providing the working example with $names and $values. Although I am still not too content, as I am a little shy of the “setmultiplevariables”.

  • I believe setmultiplevariables came about in part because of my pointing out some limitations in what we could do.
  • But the way it is implemented, across two filters needing the results to pair correctly to work makes it a little awkward.
  • For example the number of filters in referenceFiltersFilter must numerically pair with those fields in referenceFieldsFilter
    • and things get complicated If I wanted a filter to generate the fieldnames rather than the literals as in 'tags reference-date'
    • This limits the possibility of filters whose result is missing or is[blank] either in the $name or $value.

Not withstanding these grumbles what you have shared is inspired, thank you. I will take this new knowledge and see where I can take it.

1 Like

Thanks again @btheado

Perhaps as a minimalist case of this “delimiting filters” the following demonstrates how we can "store more than one filter run in a filter “statement”.

  • In this case I still use the standard double quotes for the whole filter but the filter runs within them are delimited by single quotes.
<$list filter="'[tag[$:/tags/ViewTemplate]] [tag[$:/tags/Macro]] +[sort[]]' '[tag[$:/tags/Macro]] +[limit[2]]'" variable=filter>
   <$list filter=<<filter>> >

   </$list>
  <hr>
</$list>

The following is just further exploration for those interested.

But this also works if the set of filters is stored in a variable;

<$set name=set-of-filters value="'[tag[$:/tags/ViewTemplate]] [tag[$:/tags/Macro]] +[!sort[]]' '[tag[$:/tags/Macro]] +[limit[2]]'">
<$list filter=<<set-of-filters>> variable=filter>
   <$list filter=<<filter>> >

   </$list>
  <hr>
</$set>
  • I think using such a structure containing actionwidgets could prove useful, but it is just a hunch.

Now using the list widgets counter variable we can get both the nth filter and the nth item from the filter

<$list filter="'[tag[$:/tags/ViewTemplate]] [tag[$:/tags/Macro]] +[!sort[]]' '[tag[$:/tags/Macro]] +[limit[2]]'" variable=filter counter=item>
   Filter: <<item>><br>
   <$list filter=<<filter>> counter=item>
        <<item>> <$link/><br>
   </$list>
  <hr>
</$list>

Interesting things may be possible selecting the Nth filter as needed.

  • Maybe even use this to identify the fieldname to which to apply some action listops widgets.
1 Like

Amazing! This worth to be under Tips and Tricks and a paragraph to official docs!

1 Like