The meaning of `:all`

#1

In a solved discussion, @saqimtiaz gave this nice simple answer:

I would like to find out more about this :all syntax, but am surprised by how little documentation is available. There are only two hits on the main page. One is for an example (see the last inline example in that tiddler) which demonstrates avoiding unwanted de-duplication – for instance a list field might be allowed to hold multiple copies of the string “abc”.

The other, in Filter Expression, has this:

Run Equivalent named prefix Interpretation Output
run :or[run] de-duplicated union of sets … OR run
=run :all[run] union of sets without de-duplication … OR run
+run :and[run] accumulation of filter steps … AND run
… … … …

And I understand the description as “union of sets without de-duplication”. What I don’t understand is how this also preserves spaces. I don’t mean how it works technically; I tested, altering the "add abc" example to "add abc def" instead, and each button press adds a copy of the string [[abc def]] to the field in question. That’s clear enough. But I can’t see how this behavior is related to the un-de-duplicated (!) description. Nor can I see how one would ever be expected to find out about this behavior. Am I missing something fundamental?

I few months ago, I put aside an issue that sounded possible, but simply seemed beyond my then-skill-level. I’m not sure, but I think this :all prefix would have given me some chance to solve it, and when I go back to that work, I will definitely try it out.

So, is this a fundamental part of what :all is for? Or is it some accident of the implementation that happened to have useful consequences? Is there some way that it might be documented more explicitly?

1 Like
Append an variable with space into a list
Append an variable with space into a list
#2

In your particular example, you didn’t need the :all. But if there had been a duplicate in the list, you wouldn’t have gotten an appended item:

<$vars variable="new day">
<$list filter="[[Monday]] [[new day]] [<variable>]">
<$link/>
<br/>
</$list>
</$vars>
#3

Is this how you altered the example?

<$action-listops $field="myfield" $filter="[enlist:raw{!!myfield}] :all[[abc def]]" />

If so, I think you may be conflating what the filter does (preserve all the results of a filter run even if some or all them duplicate the results of prior runs) and what the filter run [[abc def]] does (return the single literal value “abc def”. It is the double bracket title format, not the :all, that is preserving spaces.

I think the format of the example in that thread may have misled you somewhat. If the question had been only “how to add a variable that includes spaces”, the answer would not require :all, as the [<variable>] format already indicates that spaces in the title should be preserved.

<$vars variable="new day"
<$list filter="[[Monday]] [[Tuesday]] [<variable>]">
<$link/>
<br/>
</$list>
</$vars>

The issue comes in with the append operator. If you try this modification of one of the append examples, you’ll see that it circumvents deduplication and simply adds “Monday” to the end of the list:

[list[Days of the Week]append[Monday]]

But the append operator expects a list: if you use [append<variable>] where <<variable>> is a space-separated list rather than a single word, it will append each word in the list as if it were a single-word title, without preserving spaces. Saq suggested using :all[<variable>] to circumvent this behavior: in this case, <<variable>> will be treated as a single title, because it’s the only thing in its filter run.

As a side note, I tend to see = used more often than :all, so you may find more results searching that way.

1 Like
#4

Thank you both for setting me straight. It looked as though saq’s answer – as simple as it was – seemed to introduce new magic. And after my frustration in trying to find the documentation for it, I imagined there was some craziness.

Sorry for the waste of time. But thank you both for answering. Anyone else who reads the example that way will at least have some good answers.

1 Like
#5

As you can see at https://tiddlywiki.com/#Filter%20Expression the = prefix is there since v5.1.20 … So it’s not an “accident” …

It’s mainly needed and was invented for mathematical operators, because they need it eg: to sum up the same values several times.

Also see: https://tiddlywiki.com/#Dominant%20Append:[[Dominant%20Append]]%20[[enlist%20Operator]]

We started to implement more and more “filter run prefixes”, and they started to be more and more cryptic. eg: + - ~ = … So with TW v5.1.23 there was the decision to add “named filter runs”

1 Like
#6

I suppose we can think of ALL as “All Values” without the default deduplication.

There are multiple ways a title gets into a filter, and as a title we typically want the list to be deduplicated because a title is a title, a key to a tiddler, and duplicates would just make it harder to handle, there would be many occasions we would need to deduplicate it and this would be difficult mid-filter. As a result deduplication is the default.

Mathematics came some time later, It was only once we introduced numbers, where deduplication would interfear, that the = / All was introduced. Imagin a set of numbers in which a duplicates arose eg 42 43 42 you can’t change that to 42 43 (deduplicated) and expect the correct results.

  • Ideally you would place it against any value you don’t want to be removed because it is a duplicate, but in the OT it was only called for as a result of the last “filter run” :all[<variable>]

= / all could also be described as “include even if duplicate”. or “all values - even if duplicated”.

Perhaps we should spell it out in the documentation a little more.

#7

I misunderstood what :all was doing. I knew it was avoiding de-duplication. But I thought it was also preserving spaces. People have already set me straight on that. But here by “accident” I meant something like “happy accident”, when something unexpected and useful is implemented, but not on purpose.

I founded and still (occasionally) maintain the JavaScript functional programming library Ramda. At one point we created the function applySpec, which we intended to be used like this:

const myFunc = applySpec ({
  foo: add(1)
  bar: multiply(5)
}

myFunc (2) //=> {foo: 3, bar: 10}

But at some point a user pointed out to us that its implementation – just by chance – happened to also work for arrays:

const anotherFunc = applySpec ([
  add(1), 
  multiply(5)
])

anotherFunc (2) //=> [3, 10]

This is a “happy accident” – a useful, but unplanned result of the implementation.

Since I’ve learned that it was not :all that preserved spaces, this turned out to simply be a bad guess. (It also means that this has nothing to do with solving the earlier issue I mentioned, but I believe this discussion has still given me an idea for when I can spare the time for that.)

#8

I think that would be a good idea. It was hard to find – I searched first for “all” and not “:all”, and there are far too many hits for “all”. I did think to look for it as a Filter Run Prefix, but that tag has only six tiddlers: :cascade, :filter, :intersection, :map, :reduce, and :sort. To my mind, it would be useful if the others, :or :all, :and, :except, and :else each had their own page, even if it’s a one-liner with a link back to Filter Expression and that tag. I will put it on my documentation TODO list; the main work is still a ways away, but I might move this one up.