All I want for Christmas

[CodaCoder]

<!-- Assign a list to a variable v -->

<$set name=test1 filter="[range[1],[3]let[v]]">

<!-- v gets a length (or count) property -->
<<v.length>> -> 3

<!-- v contains the list (so far, in longer expressions) -->
<<v>>  ->  1 2 3

<!-- No dotted indices available -->
<<v.N>>  ->  (blank)

~~~~~~

<!-- Assign a list to a variable v -->
<!-- Create a dotted variable per list item using spread notation, "v..." -->
<$set name=test2 filter="[range[1],[3]let[v...]]">

<!-- v gets a length (or count) property -->
<<v.length>>  ->  3

<!-- v contains the list (so far, in longer expressions) -->
<<v>>  ->  1 2 3

<!-- Use dotted member notation (v.N) to resolve each item -->
<<v.1>>  ->  1 
<<v.2>>  ->  2
<<v.3>>  ->  3

~~~~~~

<!-- Filtered transclusions should work similarly -->
<$let test3={{{ [range[1],[3]let[v...]] }}}>  ->  v.1 v.2 v.3 

<$let test4={{{ [<v.2>add[1]] }}}  ->  3

~~~~~~

<!-- Spread notation limited to 10 items -->

<$let test5={{{ [range[1],[50]let[v...10]] }}}>  ->  v.1 .. v.10 


~~~~~~

<!-- Apply list of names to members (alongside numeric indices) -->

<$let names="one two three"
      test6={{{ [range[1],[3]let[v...],<names>] }}}  ->  v.one .. v.three 
      test7={{{ [<v.three>subtract[1]] }}}  ->  2
      test8={{{ [<v.3>subtract[2]] }}}  ->  1

Thoughts @etardiff @jeremyruston @pmario @saqimtiaz @Scott_Sauyet ?

[buggyj]

maybe reuse the field notation eg <<v!!1>>

[Scott_Sauyet]

Very interesting ideas!

@CodaCoder

<!-- Assign a list to a variable v -->

I’m not sure there is much advantage in this first section over just using the test1 variable, which already holds a list. It certainly would avoid many enlist calls, but that’s about it.

---

@CodaCoder

<!-- Create a dotted variable per list item using spread notation, "v..." -->

@CodaCoder

`<!-- Filtered transclusions should work similarly -->`

But these sections ar much more interesting. They add a lot of new convenience.

---

@CodaCoder

<!-- Spread notation limited to 10 items -->

I’m not sure this is worth the additional syntactic sugar, given that I would expect this to do the same:

<$let test5={{{ [range[1],[50]first[10]let[v...]] }}}>  ->  v.1 .. v.10 
<!--                          ^^^^^^^^^                -->

---

@CodaCoder

`<!-- Apply list of names to members (alongside numeric indices) -->`

I can’t tell how much I would use this. I guess it would make sense when the list is a tuple in disguise.

---

So overall, some very nice ideas.

However, I think it may be a little late. I assume that the following is planned for 5.4.0:

Introducing Multi-valued Variables

It is syntactically very different but seems to cover much of the same ground as this proposal.

---

[CodaCoder]

Thanks, Scott.

@Scott_Sauyet

I’m not sure there is much advantage in this first section over just using the test1 variable, which already holds a list. It certainly would avoid many enlist calls, but that’s about it.

Since I left the “discussion” as code comments, perhaps some of it was left open to interpretation. The key component (left unmentioned) is a proposed new operator let[]. Let is a pass-thru operator, akin to @Yaisog’s debug operator both of which can be sprinkled throughout the expression without interrupting the logic in any way:

  [<list-stuff>let[a]op[]] :map[foo[]let[b...]op[]let[c...]op[]let[d]]

Whereas ..currentTiddler is tied to (partial) outcomes, let[v] would allow arbitrary access to states midway (or, indeed anywhere) in the chain.

Plus, of course, that first section is more about getting the ball rolling – and $set needs to be syntactically correct. Much the same with using range[],[] – a dumb list constructor good enough for starting the conversation.

[CodaCoder]

@Scott_Sauyet

I assume that the following is planned for 5.4.0:

It was in my mind as I wrote it out. Though I don’t recall anything about in-situ variable assignment in filters.

@buggyj Yes, of course. My syntax choices are placeholders at this stage. I would need convincing that “!!” “field accessor” is valid here.

[CodaCoder]

@Scott_Sauyet

I’m not sure this is worth the additional syntactic sugar, given that I would expect this to do the same:

I’d argue that’s both valid/expected (in that context) but also a side-effect – whereas v...10 is direct and clear.

@Yaisog in the light of XXX[], what’s your take?

[Scott_Sauyet]

@CodaCoder

Let is a pass-thru operator, akin to @Yaisog’s debug operator both of which can be sprinkled throughout the expression without interrupting the logic in any way

Ah yes, of course. And then available inside the rest of the filter, right, not just outside as test1 would be. Makes sense.

@CodaCoder

It was in my mind as I wrote it out. Though I don’t recall anything about in-situ variable assignment in filters.

An example from that thread is

[my.custom.function<param>,[2]] =>index [<data>jsonget[top],<index>]

So it’s here more like a filter-run suffix, and not inline the way your suggestion is. Got it.

@CodaCoder

I’d argue that’s both valid/expected (in that context) but also a side-effect – whereas v...10 is direct and clear.

You’re absolutely right. I was missing a fundamental point on my first reading, that this is entirely pass-though. I’m liking it more and more!

---

I’m still not sure I follow the advantage of having both the let[v] and the let[v...] syntaxes. How would you use the former in a way that the latter would not work just as well? I see (now! thank you) that let[v...10] would be different and necessary, but I’m still not seeing the rationale for the let[v] format.

[CodaCoder]

@Scott_Sauyet

How would you use the former in a way that the latter would not work just as well?

Good question.

It depends on the context – in what I’ve shared already, I’ve focused on lists and the application of let[] to lists.

Now consider any point in a filter chain where you’d like to capture just “x” → let[x] because you know there is only one value at that point but whose value you want (need!) to capture and, as you pointed out, reuse later in the filter:

[...let[x]...] +[op[]addprefix<x>...]

I’m struggling to think of meaningful use-cases

[Scott_Sauyet]

@CodaCoder

I’m struggling to think of meaningful use-cases

I’m guessing that’s difficult because our TW usage really only deals with one structure, a string. We often split that string into a meaningful list of substrings and iterate on them, but we mostly don’t distinguish between such a title-list containing only one title and a string containing that title. (Some strange dark feeling is starting to overwhelm me… I think there’s a demon near… yes, yes, and the demon’s name is … jQuery!)¹

---

¹JFTR, I think jQuery did an awful lot right, and its use of selector results in a uniform, chainable, list-like structure was one of them. That it also did a lot wrong doesn’t change that.

[Mark_S]

“Let there be ‘Let’ on Earth, and let it begin Wiki”

[Yaisog]

@CodaCoder

@Yaisog in the light of XXX[], what’s your take?

I have often wished for variable assignment inside filters and “multi-valued variables” (arrays or objects). These are right at the top on my Christmas wishlist, too. They could make many filters shorter and eliminate the need for extra subfilters or functions and such.

As @Scott_Sauyet mentioned, there is a lot of overlap of your proposal with @jeremyruston’s work in that direction and it seems that significant progress in that direction has already been made, though that was 6 months ago. Currently, the PR is marked as “in progress”…

It probably doesn’t make much of a difference if let is an operator or run prefix (MVV), preferably both.

To be able to access list / array items directly sure is nice, but there should be a way to iterate over the list (another run prefix maybe) and to use some counter variable to access a dynamically chosen item. The syntax for this might get complicated.

Personally, I don’t think I have much use for a “first n” functionality. A check if the list is empty will usually suffice.

[TW_Tones]

I am not sure I follow this thread but I suspect it overlaps some inroads into named parameters and key value pairs. for example widgets that use the $names and $values parameters. we also have as a result of the $params parameter in the $parameters widget, which as I understand is a json array.

for handling larger arrays a set of keyword value pairs or permitting the value to be a list and perhaps some head/tail operators to simplify processing such lists would go a long way.

I fear a lack of use and understanding of these existing features, let alone, not developing them further, will result in increased complexity by alternative solutions being developed or unnecessary loss of backwards compatibility. I also fear the inherent complexity alternative approaches will introduce.

All I can do right now is raise my concerns, but I will work with anyone who asks, to throw light on this bleeding edge.

[Scott_Sauyet]

@TW_Tones

I am not sure I follow this thread

Attempted explanation below.

@TW_Tones

but I suspect it overlaps some inroads into named parameters and key value pairs. for example widgets that use the $names and $values parameters.

Maybe a bit, conceptually, but there’s probably very little practical overlap.

What this is about

I’m going to demonstrate by example. I think that should be sufficient to show why this would be useful

Imagine you have two functions, calc.points that uses age, experience and skills parameters; and calc.threshold that uses a category parameter. (This is made up, but I’m sure we can imagine a scenario relatively close to this.

Now you want to report on whether the calculated points reaches that calculated threshold. We know how to do this in wikitext, usually easiest with a $let wrapper:

<$let 
  points={{{ [calc.points{!!age},{!!experience},{!!skills}] }}} 
  threshold={{{ [calc.threshold{!!category}] }}}
>
  {{{ [<points>compare:number:gteq<threshold>then[passes]else[fails]] }}}
</$let>

But what if we wanted to do this in a filter? Although there are surely others, the most obvious case where we need to confine ourselves to a single filter is in a cascade. So let’s say we have a tiddler tagged $:/tags/ViewTemplateBodyFilter, that chooses our PassThresholdBodyTemplate in the case that the points for current tiddler passes the threshold. There’s no obvious way to do this.¹ But with @CodaCoder’s suggestion, we could do this in a single filter like this:

title: $:/_/my/cascades/body/pass-threshold
list-before: $:/config/ViewTemplateBodyFilters/default

[calc.points{!!age},{!!experience},{!!skills}let[points]]                 <!--
                                             ^^^^^^^^^^^                   -->
[calc.threshold{!!category}let[threshold]]                                <!--
                           ^^^^^^^^^^^^^^                                  -->
[<points>compare:number:gteq<threshold>then[PassThresholdBodyTemplate]]   <!--  
 ^^^^^^^^                   ^^^^^^^^^^^                                    -->

(I’m pretty sure we can’t actually nest comments in a filter like that, but the point should be clear.)

Here we assign variables inside the filter for use later in the filter. That’s the core of the proposal. It extends to allowing us to work with lists held by such variables, and one step further to intentionally limit ourselves to an initial slice of such a list. But the core is simply to allow in-filter variable assignment.

The discussion about this being a pass-through operator simply means that it doesn’t change the input context. Although it assigns a variable, it doesn’t interfere with the rest of the operation.

[[1]] [[2]] [[3]] [[4]] [[5]]  +[multiply[2]let[evens]subtract[1]]                                <!--
                                            ^^^^^^^^^^,         '+ current filter value: 1 3 5 7 9
                                                      '--but `evens` has: 2 4 6 8 10               -->

Comparison with Multi-Values Variables

Introducing Multi-valued Variables is a proposal @jeremyruston created, stemming out of his work with improving color handling, but with much broader implications. Thematically, it overlaps a lot with the OP; both deal with assignments of lists of values to a single variable and both offer in-filter ways of assigning such variables.

Syntactically, they are very different, though. I think the biggest difference from the discussion above is that MVV offers an assignment only as a filter-run prefix (syntactically, possibly a suffix), where as the OP here allows it almost anywhere you can have an operator.

¹ Yes, I’m sure we can do this be creating still another function that takes age, experience, skills and category parameters, but this suggestion is about how to make things simpler for users, not how to make them possible.

[Ste_W]

I know it’s off topic, but every time I see the title this song starts in my head and I don’t see why I should suffer alone.

- YouTube

[Scott_Sauyet]

@Ste_W

every time I see the title this song starts in my head and I don’t see why I should suffer alone.

You’re just lucky that these things don’t autoplay here. I’m extremely subject to earworms. When they’re songs I like, fine. I can listen to Elvis Costello’s Radio Radio for hours (weeks!) at a time. But if a Christmas song gets in there – especially in October! – I’m liable to do anything: launch the bombs, steal your girlfriend, mix up the salad and dessert forks, really anything.

You’ve been warned!

[TW_Tones]

I would not want to discorage variable in filters, but to do it will need a lot of wisdom, and user testing. So keep working folks.

However: I think we are often trying to drive functionality into filters when is is simpler and easier to use nested structures be they list widgets or let widgets. At the very least we should support the ease of use outside filters before we then try and drive it into filters.

• If we look at the functional and structural advantaged of the if/else/endif statements we can see how the syntax can provide a much easier way to represent real world algorithiums. Both nesting conditions and automatic responses to the opposite the else condition.

Personaly I think we could introduce another structure designed to assist the designer with multi-variable problems. Perhaps a while/wend and similar structures?

There may already be opportunities with functions, custom widgets and Genesis widget. I have a strong feeling with all these advances of recent years that we HAVE NOT yet explored the possibility space these gave us. Plenty of answers already exist within these tools, and it we can find some hard limitations it is those we need to address, not follow to many other divergent paths.

My views here are based on a carrer of communicating/translating training and teaching ICT, I always want to frame that as how hard would it be for a naive readers to understand, or myself after months of not using something.

[EricShulman]

While I agree that it is often “simpler and easier to use nested structures” for many wikiscript purposes… consider the relevant issue that @Scott_Sauyet has identified:

@Scott_Sauyet

Although there are surely others, the most obvious case where we need to confine ourselves to a single filter is in a cascade

Cascade definitions, by their very nature, are not able to use $list or $let widgets (or any other surrounding widget syntax for that matter). Thus, regardless of the general advantages of using “nested structures” to define and manage “intermediate” variables, there is also a clear purpose for being able to “assign variables inside the filter for use later in the filter”.

Although it might never result in any significant changes to the TWCore filter processing, it is certainly worth it to continue to discuss these ideas.

-e

[TW_Tones]

@EricShulman

Cascade definitions, by their very nature, are not able to use

This confuses me. Custom functions by definition return the first result so one can code a cascade therein. We also have the “Cascade Filter Run Prefix”.

• Infact filter runs already handle currentTiddler and …currentTiddler
• The whole wiki is driven by cascades, not embeded in single filters.

@EricShulman

Although it might never result in any significant changes to the TWCore filter processing, it is certainly worth it to continue to discuss these ideas.

I would never discorage discussion, but I think there are things we should be discussing that we are not.

For example in a similar way to the filter runs if there were an operator given a key word return its value from a keyword=value pair (list there of) and we had easy tools to create such keyword=value pairs we could easily have unlimited parameter values available to a filter.

• Actually I made a tool to convert all non-standard fields into fieldname=value pairs for this very purpose.