An "in" filter operator would be useful

Developers
,
#1

An “in” operator that knew how to work with lists would be useful in situations where you want the status of something to be stored somewhere other than the original tiddler. It would not require you to list or enlist and thus interrupt your current filter stream.

So something like

<$vars mylist="[[Ten reasons to switch to TiddlyWiki]] Examples">
<$list filter="[tag[HelloThere]in<mylist>]">
This item is tagged by "HelloThere" and is in my list: <<currentTiddler>><br/>
</$list>
</$vars>

Edit: … not require …

#2

Did you try https://tiddlywiki.com/prerelease/#contains%20Operator:[[contains%20Operator]]%20[[contains%20Operator%20(Examples)]] ?

#3

I think @Mark_S is proposing the reverse of that operator. contains filters the list of inputs based on whether a field on those inputs contains the parameter. in would filter the list of inputs based on whether the parameter contains the input tiddler.

#4

Indeed. @Mark_S is asking for a string/array operator more than a field sniffer.

#5

Is this use case not satisfactorily catered for by the :intersection filter run prefix?

<$vars mylist="[[Ten reasons to switch to TiddlyWiki]] Examples">
<$list filter="[tag[HelloThere]] :intersection[enlist<mylist>]">
This item is tagged by "HelloThere" and is in my list: <<currentTiddler>><br/>
</$list>
</$vars>
1 Like
#6

Thanks saq! I’ll have to try your solution and see if it works. I think the problem with it is that it is inserting a 2nd run. That might not always be possible if, as in my case, you’re trying to not change existing code.

In general though, wouldn’t it be nice to have the whole suite of affordances (seems to be a popular term these days) that we would expect, say, in Javascript? Looking for something in an array (title list) is a natural thing to want to do.

Edit: Saq’s suggestion does work!

#7

It certainly is. Not sure if this has legs, but here’s the dream I had:

[[some-list]es[my_fn]]

es: “Escape to EcmaScript”

my-fn is a loaded js function (macro). It receives the list at that point and returns a list (array).

Perhaps that’s how operators already work, I don’t know. :confused:

#8

I guess you could say that in TW a js macro that takes an array is called a filter operator.

If you have a macro, why not rather write it as a filter operator and thus your filter becomes

[[some-list]es[]my-fn[]]

#9

Only that es becomes the generic escape mechanism, my_fn handles the specifics. If that runs counter to the TW “way”, then that’s okay.

#10

It’s an attractive idea, but there are a number of issues with “escaping to JavaScript [within wikitext]” that led to me discarding it quite early on in TW5’s development, in favour of the current approach that requires JS code to be packaged in module tiddlers.

  1. It isn’t in general possible to “sandbox” user written JS in order to protect from it doing malicious or inadvertent things, or to detect and interrupt runaway infinite loops in user written JS
  2. Philosophically, there is an interesting tension: TiddlyWiki5 is intended for use by people who are not JS developers, but we want it to be a productive environment for JS developers. The concern is not wanting to alienate users who are not (and don’t want to become) JS developers

In contrast, TW5 native filters have some nice characteristics – not least that they are guaranteed to always complete execution, and it is not possible to accidentally create infinitive loops

There is now actually a little progress on (1): there’s an emerging technique to run JS safely by transpiling it to add instrumentation, and to run it in an iframe. The iframe runs as a separate process container, and so cannot interfere with the host page. The added instrumentation code would do things like send a heartbeat message to the hosting page so that it can detect infinite loops.

As usual, these observations come from the perspective of the core. There’s absolutely nothing to stop somebody making a plugin that added your ES operator for those users that understand the risks.

#11

I was expecting that my_fn would be… which leads me to think: es[] becomes “what’s the point?”

Anyway, wasn’t intending to hijack Mark’s thread…

#12

@Mark_S I am always in favour of adding more operators as necessary to make filters more expressive. It does help however to understand and have an example or two of specific problems that cannot be overcome by current affordances, or are too convoluted. That helps not only with writing the code but the tests and documentation as well.

Do you happen to have any examples in mind where multiple filter runs cannot be used?

I believe all widgets that accept filters accept a filter expression, which can be one more or more runs. Even filter operators that accept subfilters will accept multiple filter runs in the same expression. This is one of the things that can make a filter operator more powerful than its filter run prefix equivalent, for example filter[] vs :filter

I know there are some core macros that accept filter steps that could be problematic but I cannot remember which they are off the top of my head.

I assumed the example and pseudocode you posted was a simplified example so the solution I suggested was the most flexible one in which there was most flexibility in deriving the two lists between which to check for overlap. Depending on your use case there may be more performant approaches too.

For example taking the literal example of the code you posted, it could also be written as the following which could be faster:

<$vars mylist="[[Ten reasons to switch to TiddlyWiki]] Examples">
<$list filter="[enlist<mylist>tag[HelloThere]]">
This item is tagged by "HelloThere" and is in my list: <<currentTiddler>><br/>
</$list>
</$vars>

or

<$list filter="[[Ten reasons to switch to TiddlyWiki]] Examples +[tag[HelloThere]]">
This item is tagged by "HelloThere" and is in my list: <<currentTiddler>><br/>
</$list>
#13

Mark’s wish for an in[] operator is innately expressive, IMV. Plus, might I suggest, expected to be there?

1 Like
#14

In this case I’m substituting for the “sort” attribute of the TOC macro. So I can’t change where the tag[mytag] filter occurs.

For me the issue is discoverability. Yes, your approach worked, and that’s great. Thanks! But I don’t think most people would come across this very easily. I use wikitext all the time, and it didn’t occur to me. The first place I looked was the list of filter operators. I looked at list, enlist, filter, subfilter, and contains. I think that is the approach most people would have – look for a tool that matches the problem. There was no tool that matched this exact problem, even though an in tool is common place in most code languages these days.

#15

I don’t know which is more expressive in English
[tag[X]in<mylist>] vs [enlist<mylist>tag[X]]

[in<mylist>tag[X]] it also sounds fine for me. It also sounds fine for you, maybe a second name (in) for enlist operator could be created.

IMV, it is also more expressive something like [in<mylist>tagged[X]]