Filter operators: inverting input and parameters

Developers
,
#1

I wonder if variable (like in the reduce operator) could be a way to devise a filter operator that could make it possible to have such possibility:

[foo] → [foo and bar]

where bar would be computed from the foo tiddler.

for instance, I have a “tables” dictionary where the “t3” key has a “p4 p7” value.

what I would like to get is something like [t3]mystery[tables] or [tables]mystery[t3] that would result in a [p4 and p7 are on table t3]. All of this within a single filter run (which could be composed of several filter runs).

And also, could operator variables allow something like:

[[t3]%getindex[tables]] equivalent to [[tables]getindex[t3]]

Because sometimes t3 is in fact the result I’ve got earlier inside the filter. I I could use it immediately, it could allow for some clever things done in the same filter. And this could simplify a lot of things in some cases, instead of having variables and several filters while having to write field(s) with your results and later exploit those fields.

Note that % would be a convention to stipulate such a behaviour, in the like of !. I have already noticed that this % would require checking every macro with ! capacity because often the code for checking ! is bit too hasty/optimist and would break.

#2

<$list filter="t3 :map[[data]getindex<currentTiddler>split[ ]join[ and ]addsuffix[ are on table ]addsuffix<currentTiddler>]"/>

1 Like
Clarification: How filter operator, :filter, and :map run prefixes work?
#3

Hank you very much @Mark_S ! I overlooked filter prefix and especially their variables, of which I was not aware until recently.

With you help, I’ve been able to produce this code:

<$vars turn={{!!turn}} table=t4
namePlayers="[indexing[players]split[|]first[]addprefix[ (]addprefix<currentTiddler>addsuffix[)]]"

tell="[join[ and ]addsuffix[ play on table ]addsuffix<table>addsuffix[ for the ]addsuffix{!!caption}addsuffix[.]]"

decode="[!is[blank]] :map[subfilter<namePlayers>] +[subfilter<tell>]"
>
<$set select=0 name=opps filter="[[turn]addsuffix{!!turn}]
:map[getindex<table>split[ ]subfilter<decode>]
">
<<opps>>
</$set>
</$vars>

sample otuput: “p3 (Louis) and p4 (Paul) play on table t4 for the fourth round.”

the code is within a tiddle rhaving the followinf field: “turn” worth “4”, the “turn4” tiddler is a dictionnary of table (like “t4”) which hlod encounters (like “p3 p4”) and has a “caption” field valued “fourth round”. “players” is a dictionary tiddler with key like “p4” and value like “Paul|c14” (where “c14” is a key for a “clubs” dictionary).

Now, the [!is[blank]]bit at the beginning of the “decode” subfilter is a kludge. It does nothing (in this context), but it is the beginning of a filter run without any prefix and its presence was mandatory for the whole code to work; I was unable to make it work directly with the “:map” prefixed run. This is surely a bug, isn’t it?

The nice thing is that with :map I,ve been able to have a map within a map without having to resort to map filter operator, which can be replace by a subfilter<mapped-run>if within a :maprun which itself has proven itself to be reluctant to be something like a sequence of run (things like [[fist-run-code] [second-run-code]]` were not accepted/operated.

#4

:map is a filter – it doesn’t generate anything unless given input. You could probably use [all[tiddlers]] instead of [!is[blank]] at the start. This might be more intuitive.

#5

@Mark_S But there is indeed input at the beginning. this input is 2 tiddlers, “p3” and “p4”, fed by the first filter run.

I can use [!is[system]] which is better has being identified as a kludge and never has an effect for the kind of data there is.

but its preence is man,datory otherwise the filter breaks (no output).

#6

There is no input to the below filter run since it is called via the subfilter operator:

decode="[!is[blank]] :map[subfilter<namePlayers>] +[subfilter<tell>]"

The behaviour is precisely as expected. A :map filter run prefix with no input, outputs nothing.

As documented, the subfilter operator is a selection constructor and ignores its input: https://tiddlywiki.com/#Selection%20Constructors

Subfilter operator is only sometimes a selection constructor
#7

@saqimtiaz The subfilter was here because I saw no other mean to call my filter run. Initially I tried :map<namePlayers> but it did not work. How would you have done it (with keeping my namePlayers variable)?

All what both of you tell me about my errors tell me an other story as well, one telling that we must improve the technical documentation to help people know of these subtleties first and then be able to assimilate them. This is not trivial. And I do not criticize the current state of the doc, buts merely conclude that we need to make it even better.

I’ll give an exampe related to the present topic: when you read the doc about the subfilter operator, you don’t read anything about it being a //selection constructor// or a description of what a a selection constructor does or doesn’t. You just have it tagged as //selection constructor//. This is hardly optimal and easily not even seen. Spotting those problems will be a devoted task. And then you have to decide what the fix should be.

#8

There are too many things I don’t understand about your data, your approach and what drives it. I can take a look and advise if you can:

  • provide tiddlers with data that illustrate the problem
  • explain the desired output
  • explain why you want to have all these filters in separate variables, but still want to assemble your final output in a single variable

I personally find the filter documentation to be very good, with a few exceptions like the Filter Expression tiddler which is due for a complete rewrite since we have added so many filter run prefixes. The problem is that no one reads the documentation on filters that explains the different components and how they interact. They skip directly to an operator and then wonder why they don’t understand.

I came very late to TiddlyWiki5 and struggled with filters until I dedicated the time to read the entire filter documentation starting from https://tiddlywiki.com/#Filters and in particular from https://tiddlywiki.com/#Filter%20Syntax
I have not had any difficulties with understanding filters after that.

Which isn’t to say that there isn’t room for improvement in documentation and we all learn differently as well. As always, PRs for improvements to documentation are very welcome.

#9

@saqimtiaz a lot to tell but here is the most easyly told:

  • my desired output has been shown previously as well as what is the tiddlers involved in a previous post of this subject. maybe I should include a json export of the lot?

  • I like to name filter run so as to explain what they are trying to do. it helps building them and maintain them especially if I refactor.

  • filter are difficult. I had many difficulties in doing anything with them. I gradually improved but, yes, there is still many a cliff to climb for me! It will take time.

  • the technical points are surely told for the most of them. But that does not mean everyone will notice them. Some, and I’m among them, will struggle. Yes, a PR (and not telling it’s a pull request in all letters is not making PR clear and this is yet again an example showing that yes you technically tell it but it is also true that not everyone will read it especially if it doesn’t do git and no that’s not a taken and again this includes me until recently). So yes, one has to have a try at it and proposes its work for approbation with a pull request. and this includes myself of course.

  • I’m sure wikipedia would have failed if its updates required PR. couldn’t we update our technical doc as wikipedia and not like a source code? Couldn’t we agree to have a try at this idea? And if yes, then let’s see how we can do this in real life.

#10

Yes that would be the way to go and is generally recommended when asking for assistance. If your tiddlers depend on third party plugins then it is perhaps better to share a TiddlyWiki file with a minimum set of tiddlers and those plugins.

I felt comfortable abbreviating since you and I have had this particular conversation before :slight_smile:

As you have seen in another thread, work on this is already underway. The technical challenge has been resolved and hopefully the community will work together over time to figure out the best user workflow and UX for it.

#11

@saqimtiaz Here are all the tiddlers needed for recreating my wiki and its problem. Import all the tiddlers from the joint json files, save the wiki and reload them for there are javascript filter operators included.

Then open third turn tiddler and click on ''check opp`` after having enabled the JS console. See the trace. The error on display will be:

  • unknown player buzz. unknown player himself. unknown player p71. unknown player p21
  • t78 has less than two players. t1 has more than two players. t0y is not a correct table identifier.
  • 8 players compete but there are too many tables.
  • p4 appears more than once.

The code is in the checkOpp tiddler.

All of this is part of a wiki aimed at organising a tournament of wargames between two players per game.

The code as is is running fine. I post it so as to answers yours questions about my own questions about the programming issues I initially had and especially about my style of programming filters.

Iit is tested with a tw 5.2.0 version.

BTW tw5.20 is most often unable to save my changes when I modify the text field of a tiddler. Anything else (other fields, name, type and tags) is always OK. But the most strange is that sometimes’ the text is well recorded once. Never again after that (at least for the tiddler in question). I shall open a ticket for that once I have dug a little deeper into that problem.

partial-export.json (9.2 KB)

#12

I might be missing something here. I was expecting a set of tiddlers where I can test the code you posted above, however that does not seem to render anything even as written. Or perhaps you can provide a minimum test case based on the tiddler you provided, that illustrates the problem?

#13

but… this is a json export of a set of tiddlers. can’t you import them? and do as I said then?

this code is working though. you can have a peek if you wish. But from what you say, I really want to be able to provide you what you need, for I have a need of it. for my next programming question.

#14

I did so and the debugging output is indecipherable to me. It would take far too much time for me to try and understand the complete logic of your application and what the debug is trying to show.

What is needed is a working minimal test case which illustrates the problem you are facing or area on which you would like input.

What I can do if you would like, is given a set of tiddlers where I can test the code below which you have earlier posted, try to advise on alternative ways to approach the filters:

<$vars turn={{!!turn}} table=t4
namePlayers="[indexing[players]split[|]first[]addprefix[ (]addprefix<currentTiddler>addsuffix[)]]"

tell="[join[ and ]addsuffix[ play on table ]addsuffix<table>addsuffix[ for the ]addsuffix{!!caption}addsuffix[.]]"

decode="[!is[blank]] :map[subfilter<namePlayers>] +[subfilter<tell>]"
>
<$set select=0 name=opps filter="[[turn]addsuffix{!!turn}]
:map[getindex<table>split[ ]subfilter<decode>]
">
<<opps>>
</$set>
</$vars>
#15

OK’ so the tiddler export is fine. I will minimize my export and repost it then.

#16

While minimizing, I stumbled upon that mystery:

{{{ p3 p4 :map[[players]getindex<currentTiddler>] }}}

worrk fine. But not so the code below

{{{ p3 p4 :map[[players]getindex<currentTiddler>split[|]] }}}

where I simply added split[|] before the last ]. that produces:

Filter error: Missing closing bracket in filter expression

as a link to a tiddler of the same name. this linking is normal, but why this error message?

please import t3.json and open test3 tiddler to ree the problem.
t3.json (612 Bytes)

#17

The triple braces syntax is for filtered transclusions and not a generic filter execution mechanism. In filtered transclusions the pipe character is reserved for specifying templates and a tooltip. The tooltip option in particular is what causes this issue and is one of those things that we are forced to maintain only due to backwards compatibility constraints. I am in a rush at present so don’t have the opportunity to refer you to the appropriate documentation.

For just executing a filter and displaying output this might work for you:

<$text text={{{ p3 p4 :map[[players]getindex<currentTiddler>split[|]] }}} />
1 Like
#18

I didn’t realize a pipe could break a filtered transclusion!

Another workaround is to put the offending character in it’s own variable:

<$vars pipe="|">
 {{{ p3 p4 :map[[players]getindex<currentTiddler>split<pipe>] }}}
</$vars>
#19

Or to change the pipe for a less dangerous character. I selected ^ which is even more readible, if an original choice for the task.

Many thanks @saqimtiaz for your explanation! I shall stop this thread here.

#20

I like this approach too, I tend to use the subfilter operator for this, especially for additional filters eg; from memory.

\define active-todos() [!tag[done]]

{{{ [all[]tag[todo]subfilter<active-todos>] }}}
or
{{{ [all[]tag[todo]!subfilter<active-todos>] }}}

I only did a cursory test on this method above.z The idea is there, just it may not be accurate.

The subfilter may include [!tag[done]!tag[archive]!tag[hold] etc… to define more sophisticate alternative tags.

I little trick here is “active is what it is not”, so the active-todos filter contains what it would not be. Because a filter “removes” the candidate’s, what remains is what I want and that is the filter name that is “give me active todos” so to negate it “!” is to “give me inactive todos” (As I said not recently tested.

Now with the filter operator we can do this;

\define todos() [tag[todo]!tag[done]]
\define donedos() [tag[todo]tag[done]]

{{{ [filter<todos>limit[10]] }}}
{{{ [filter<donedos>limit[10]] }}}

or

\define todos() [tag[todo]]
\define done() [tag[done]]

{{{ [filter<todos>subfilter<done>limit[10]] }}}

{{{ [filter<todos>!subfilter<done>limit[10]] }}}
1 Like