Filter to check if an input item is contained in the parameter list

[Yaisog]

This is sort of a continuation of this discussion:

Filter to look up in dictionary tiddlers

Another example for this would be to check if an input title is contained in a list which itself is saved in a variable, I think. The other way around would simply use match<title> to check if the title stored in is contained in the input list. I believe the current solution for this is :filter, together with enlist[] and match[], which performs the enlisting for each input title, which is probably not very efficient. This is something I run across every now and then, since I usally save the …

also possibly touched upon here:

Inverse of `contains` operator: check if value is one of a list

I’m looking for a filter operator that, it seems, probably doesn’t exist. You can think of it as the inverse of a contains operator. contains takes as input a list of items and has a parameter. If any one of the items matches the parameter, then contains succeeds. I want one that takes a single item a input (likely used in a :filter call) and has a parameter that captures a list of values. It should succeed if that parameter matches any on of the items. Perhaps it might be called contained…

I think it’s a (probably) little known fact that the enlist operator can check if the input items are not contained in its parameter list by using the ! prefix. I surely didn’t know about that. This behavior is quite different from the non-inverted enlist, which ignores its input and does something completely different, and thus maybe unexpected.

Now it would be consequent to have a mode that is the inverse of this inverse operation, i.e. a mode that checks if the input is contained in the parameter list. To me, one does not make sense without the other. Otherwise I’d need to get very creative in order to use the !enlist to do what I need.

Looking at the source code for enlist, it seems pretty straightforward to add this inverse-inverse operation, and I have it running in a developer wiki. The question is, what would be a consistent way of defining this behavior? Currently, I have added a third suffix contain (in addition to dedupe and raw), but I’m not sure this is the best way.

Comments and suggestions are welcome. Maybe one day this can proposed as core feature, if done right.

Yaisog

PS: These are the lines added to enlist.js:

	} else {
		if(operator.suffix === "contain") {
			var results = [];
			source(function(tiddler,title) {
				if(list.indexOf(title) !== -1) {
					results.push(title);
				}
			});
			return results;
		} else {
			return list;
		}
	}

which is basically a copy of the ! prefix definition, with the === replaced by a !==.

[Yaisog]

You’d use it like this:

<$let titlelist={{{ [ some condition ] +[format:titlelist[]join[ ]] }}} >
    <% if [<currentTiddler>enlist:contain<titlelist>] %>
        The tiddler <<currentTiddler>> fulfills the condition.
    <% endif %>
</$let>

This is just a simple example. You wouldn’t use it like that, but just put the condition into the if.

So just imagine that the condition is very computationally intensive and the titlelist is checked in multiple places within the $let widget, so that you wouldn’t want to evaluate it anew every time it is used.

[Scott_Sauyet]

@Yaisog

I think it’s a (probably) little known fact that the enlist operator can check if the input items are not contained in its parameter list by using the ! prefix.

It’s certainly news to me. I would never have thought of looking there.

@Yaisog

Looking at the source code for enlist, it seems pretty straightforward to add this inverse-inverse operation, and I have it running in a developer wiki.

I think the code is fine. But for the same reason that I didn’t find it under enlist the first time, I think it would be hard to find as a suffix there. I’d much prefer this to be its own operator.

Still, well done! Maybe now I’ll remember this and be able to find it next time I need it.

@Yaisog

				if(list.indexOf(title) !== -1) {

Man, I long for the day when TW core goes to modern JS. if (list.includes(title)) reads so much better.

[Yaisog]

Funnily, I was going to create a new filter operator when I looked at enlist to “borrow” some code and saw that my work was 98% done already…

[TW_Tones]

@Yaisog

Now it would be consequent to have a mode that is the inverse of this inverse operation, i.e. a mode that checks if the input is contained in the parameter list.

Forgive my simple brain, but this is not easy to follow, a comon weakness of human brains is when we inverse the invers the inverse. Just look at how many people say the opposite of what them mean with “double negatives”.

However are you not just asking for the intersection?

Consider this;

\function list1() a b d f g +[format:titlelist[]join[ ]]
\function list2() a e f k +[format:titlelist[]join[ ]]

#{{{ [enlist<list1>] }}}
#{{{ [enlist<list2>] }}}
# {{{ [enlist<list1>] :intersection[enlist<list2>] }}} <!-- in input list1, and list 2 -->

• Ignore the way I created the two lists, it could be any filter or literals

The result;

But then also consider

# {{{ [enlist<list1>!enlist<list2>] }}}
# {{{ [enlist<list2>!enlist<list1>] }}}
# {{{ [enlist<list2>] [enlist<list1>] +[sort[]] }}}

The result;

[Yaisog]

A solution based on an intersection approach was also proposed by @saqimtiaz in the second discussion referenced above.

It does work, but working with filter run prefixes often makes filters longer and harder to read. One could also imagine a :map construct that uses enlist / match inside. Literal search with the right options might also work. Both might be even harder to figure out when I look at the code a month from now.

There’s always been different ways to do things in TW, which is part of its appeal: as a programmer I do not need to figure out the one way.

Since adding the desired functionality to enlist basically amounts to changing a === to a !== I think it might be worthwhile to think about adding another way to enable such a basic piece of data manipulation. I mean even JS has a separate method for such things (see @Scott_Sauyet’s post above). It does seem to be advantageous to create a new filter operator, e.g. includes (like the JS method) or included, to make it easier to find / remember. In my own wikis I will run with a custom enlist suffix for now, which on second thought I will rename to enlist:included.

[TW_Tones]

@Yaisog

There’s always been different ways to do things in TW, which is part of its appeal: as a programmer I do not need to figure out the one way.

I agree it is similar from my perspective, but its the larger audience, the inexperienced users that may find it difficult choosing between different approaches. They may wonder if they are using the “correct” way, especialy when its not yet working for them.

• Even if there are many ways it would help if we have a “canonical form”, with others available.

In mathematics and computer science, a canonical, normal, or standard form of a mathematical object is a standard way of presenting that object as a mathematical expression.

• We should avoid exceptional and develop the systematic.

[CodaCoder]

What about https://tiddlywiki.com/#contains%20Operator ?

[Scott_Sauyet]

@CodaCoder

What about https://tiddlywiki.com/#contains%20Operator ?

You might want to see the paragraph of mine quoted in the OP.

It’s the difference of subject versus object, that is, between

['Friday', 'Saturday', 'Sunday'] contains Today

and

Today is contained in ['Friday', 'Saturday', 'Sunday']

[etardiff]

I somehow missed this thread the first time around; thanks for calling attention to it again, @Yaisog! I had no idea that !enlist could be used this way, and I like what you’ve done with the extended suffix. Honestly, this kind of complement to contains is probably THE operator whose absence I feel most often, and I’d love to see it make it into the core—either as a suffix or a standalone contained operator.

In the meantime, I think I may experiment with modifying my enlist as you’ve done. Have you encountered any issues in the interim? Anything you’d do differently?

[Mohammad]

@TW_Tones

# {{{ [enlist<list1>] :intersection[enlist<list2>] }}} <!-- in input list1, and list 2 -->

If you use functions, then no need to use enlist. Try this one

\function list1() a b d f g
\function list2() a e f k


#{{{ [function[list1]] }}}
#{{{ [function[list2]] }}}
# {{{ [function[list1]] :intersection[function[list2]] }}} <!-- in input list1, and list 2 -->

Results in:

1. abdfg
2. aefk
3. af

[Mohammad]

Another example:

\function list1() Sunday Monday Tuesday Wednesday
\function list2() Friday Saturday Sunday Monday



#{{{ [function[list1]] ||formatlist }}}
#{{{ [function[list2]] ||formatlist }}}
# {{{ [function[list1]] :intersection[function[list2]] || formatlist }}} <!-- in input list1, and list 2 -->

Where the formatlist is a tiddler (in TW terminology a template) with below contents:

<$link/>&nbsp;

This result in:

1. Sunday Monday Tuesday Wednesday
2. Friday Saturday Sunday Monday
3. Sunday Monday

[Yaisog]

Hi Emily,
in my implementation I changed the suffix from contains to includes – I think I wanted to make it consistent with something else, but can’t remember what.
I have not had any trouble with it at all, it works as expected. It was a very minor change after all.
The full code of my enlist.js is this, for copy-pasting:

/*\
title: $:/core/modules/filters/enlist.js
type: application/javascript
module-type: filteroperator

Filter operator returning its operand parsed as a list

\*/
(function(){

/*jslint node: true, browser: true */
/*global $tw: false */
"use strict";

/*
Export our filter function
*/
exports.enlist = function(source,operator,options) {
	var allowDuplicates = false;
	switch(operator.suffix) {
		case "raw":
			allowDuplicates = true;
			break;
		case "dedupe":
			allowDuplicates = false;
			break;
	}
	var list = $tw.utils.parseStringArray(operator.operand,allowDuplicates);
	if(operator.prefix === "!") {
		var results = [];
		source(function(tiddler,title) {
			if(list.indexOf(title) === -1) {
				results.push(title);
			}
		});
		return results;
	} else {
		if(operator.suffix === "includes") {
			var results = [];
			source(function(tiddler,title) {
				if(list.indexOf(title) !== -1) {
					results.push(title);
				}
			});
			return results;
		} else {
			return list;
		}
	}
};

})();

[Scott_Sauyet]

@Yaisog

in my implementation I changed the suffix from contains to includes – I think I wanted to make it consistent with something else, but can’t remember what.

Probably the JS includes methods for Strings and Arrays, which in an event similar to smooshgate (for flatten), the obvious name of contains was replaced with includes to avoid breaking the web.