varlist reminds me of the getvariable operator, with the distinction that the latter operates on its parameter and ignores its input. Perhaps we should keep that in mind when thinking of appropriate nomenclature in the interest of consistency.
What would be the intended behaviour when accessing a variable-list using getvariable[]? I would expect it to be the first member of the list.
a b c :let:alphabet[all[]] [getvariable[alphabet]]
My preferred name choices would be:
:let:x[...] → :collect:x[...]
varlist[x] → spread[x]
The names have complementary meanings, and spread is also the name of JavaScript’s ... operator.
For the <$letlist> widget, I couldn’t find a better name. Besides, there’s already a variety of widgets to set variables in multiple, slightly different ways. Adding yet another widget type to that mix feels somehow wrong.
It might not be technically possible, but what about extending <$let> and filtered attributes, so that <$letlist x="filter"> could be written as <$let x={{{ :collect[filter] }}}> or <$let x={{{ :let[filter] }}}>, passing the special list value from the filter run to the attribute using a hidden default variable? This would be more orthogonal and composable and could prove useful in other situations, too.
Anyways, I think even without any further changes this already is a huge improvement.
To play devil’s advocate, as someone without any coding background beyond TW, I actually really like that :let and $letlist are clearly related to $let, a widget I already understand and use frequently. One recurring complaint I see from new and intermediate users is that TW has so many widgets and filter operators that the names are difficult to remember. collect and spread may be memorable to those with some JS background; to me, though, they’re brand new terms without any inherent connection to variable handling. I think it would be a mistake to introduce names that differ significantly from the terms we already use when we could instead build on users’ understanding of extant tools that perform related functions.
Also, from a purely practical standpoint, :let is shorter. 
Of the three, I think varlist is the weakest name (and this is not helped by the fact that, like @Yaisog, I intuitively want to use enlist<name> to enlist any variable I’ve saved as name, regardless of the widget/function/filter I used to do it). But varlist does at least evoke a list of variables, so it’s already doing significantly better than spread. If anything, I’d anticipate some confusion about whether it’s varlist or varslist, as older wikis may still be using the $vars widget.
I’d had some similar thoughts myself. It would be convenient to be able to mix single-value and multi-value variable definitions within a single widget, as you can currently do if you’re defining “multi-value” title lists with name={{{ a b c +[format:titlelist[]join[ ]] }}}. Otherwise, I think I’d often need both $let and $letlist, just as I currently find myself using $let and $set. But I assume this is unlikely to happen, as it would increase the complexity and likelihood of errors when using $let.
I have to admit this took me off guard.
One recurring complaint I see from new and intermediate users is that TW has so many widgets and filter operators that the names are difficult to remember.
This is correct, my feeling is that this is done for providing mechanisms to fill the gaps related to basic tools offered by “traditional” programming languages. For example the $list widget fills the gap of a missing explicit loop construction like for. The <%if> shortcut makes a step towards “programming” by offering an alternative to using then and else filter operators - a syntax that is clearly more “readable” (for me at least) than a piece of code in the middle of a long complex filter etc.
Even this whole thread and the pull request behind it triggers this controversial feeling. Getting better tools for lists handling is great, this is tangential to my older question Working with lists in TiddlyWiki . Yet this is one more thing to learn and start using. Plus the need to refactor old codebases to lower down technical debt. Plus the need to refactor old knowledge sources to recommend new “best practices”.
I kind of understand this is a tradeoof though. Wikitext is trying to fill the gap between “traditional” programming languages and pure markup/templating languages, by being both, and this is not easy at all.
what about using varlist for all 3 purposes. varlist sets a variable which contains a list. so :varlist:test creates a variable named test which contains a list, <$varlist> sets a variable which contains a list and then use <$let-varlist >to set multiple variables that contain lists. to me this would help make code readable as whenever you see varlist, you know that the variable set by that is a variable that contains a list.
Thinking through my historic pain points and where I’ve discussed the need for some kind of storing variable (this thread for example), the way it felt is that only having a single variable (currentTiddler) within the filter was the limit I was trying to overcome.
I’m realizing that I actually had something even more ambitious in mind. This veers off this topic a little bit, but list has always been 1-dimensional in nature - it exposes a single variable, and so other computed values need to be re-calculated within the loop of the widget. If we are filtering real tiddlers then we have syntax shortcuts and efficiencies using the {!!field} notation meaning I can sort by or filter on a field value, but we can’t do that by computed values that are also like properties of the <<currentTiddler>> “object” / record.
And then what if we let these pseudo-fields map within the widget? We kind of get to something more like a SQL situation where the output has “rows” and “columns”. Most of the time I use <$list> it’s within a <table> so this feels very natural.
Today example: Producing a list of tiddlers sorted by their length with both name and computed value put a filter and sort just to show comparison to pseudo-code below.
<$list filter="[prefix[A]] :filter[get[text]length[]compare:number:gt[20]] :sort:number[get[text]length[]]">
<<currentTiddler>>: <$text text={{{ [<currentTiddler>get[text]length[]] }}}/><br>
</$list>
One of the things I’m trying to illustrate here is that get[text]length[] is run not only multiple times within the filter, but then for each entry within the list widget. If that piece is “expensive” / “slow” then this is not ideal.
Pseudo-code below: Storing a computed value in the filter, and then spitting it out of the list widget along with the <<currentTiddler>> variable. I’ll use the word store instead of let to acknowledge that these really do different things.
<$list filter="[prefix[A]] :store:length[get[text]length[]] :filter[<length>compare:number:gt[20]] :and[sort<length>]">
<<currentTiddler>>: <<length>><br>
</$list>
What I’m trying to show in the above is using in-filter notation to store the computed values somewhat as properties of the currentTiddler variable, so that they can be used like they are. It might even be interesting if the syntax was <!!length> in the filter and <<!!length>> within the widget just to illustrate the near-field-like usage.
Maybe this is entirely separate from “multi-valued-variables” but I think there’s some overlap. The above actually doesn’t require variables to be multi-value, so maybe they can be both done with different names.
Thank you for the feedback and encouragement.
I have just implemented one minor change that I think goes a little way to reducing confusion. Originally, the <$letlist> widget was intended to be used with string attributes that were interpreted as a filter:
<$letlist
subcategory=":let:category[tf.makecat<currentTiddler>,[8]] [{schema}jsonget[root],<category>]"
>
...
</$letlist>
Now, I’ve updated things so that the attribute value uses the familiar {{{}}} syntax to specify the filter:
<$letlist
subcategory={{{ :let:category[tf.makecat<currentTiddler>,[8]] [{schema}jsonget[root],<category>] }}}
>
...
</$letlist>
It’s a small change, but I think it reduces confusion, and makes the <$letlist> widget be a much more precise analogue of the <$let> widget, with the only difference being the use of multi-valued variables.
As a bonus, this change also allows the <$letlist> widget to be used to assign the full result list of a function call directly to a multivalued variable:
\function myfunc() [all[tiddlers]sort[]]
<$letlist varname=<<myfunc>>>
<$text text={{{ [varlist[varname]] +[join[-]] }}}/>
</$letlist>
I will address the other feedback as soon as I can.
Thanks @saqimtiaz I think we could drop the varlistoperator and instead overload the getvariable operator. It would be backwards compatible because there’s no way for existing code to produce a multi-valued variable. I’ll look into it further, but it does seem desirable to reduce the number of novel entities introduced by this change.
We may yet get there; the next step for TiddlyWiki might well be to support lists of lists, which would allow us to model both tables and other interesting structures.
Interesting idea. It may be possible to find an approach that would let us do this. We’d have to take a slightly different approach for filtered attributes versus situations like the <$list> widget where the widget itself computes the filter. I’ll give it some thought.
Sadly I don’t think this is possible. The existing semantics are that [[nosuchvariable]getvariable[]] returns a result list of a single blank item. However, in the case of a multi-valued variable, we need to be able to represent an empty list, and so it would be natural for [[nosuchvariable]getvariable[]] to return an empty list.
Hi @jeremyruston, could enlist:varlist[name] work, with maybe a better suffix name? The first item of the parameter (per definition a title list) is taken as the name of the variable and enlisted as a multi-valued variable.
Another way to reuse enlist could be enlist:‹name›[], where ‹name› is the name of the multi-valued variable, with the boundary condition that it cannot be “dedupe” or “raw”.
I’ve made some further updates that make things a little simpler:
<$letlist> widget in favour of updating the existing <$let> widget so that it actually assigns the complete result list of evaluating the attribute to a multi-valued variable. This is backwards compatible because existing usages will only see a single value for the variable:let filter run prefix so that the values to be assigned are taken from the input title list and the name of the variable is taken from the filter run itselfI’m pleased with both these changes, but I think there’s still a little way to go. I’d like to find a better name for the :let filter run prefix, and I’d like to figure out an alternative to the varlist operator.
These are GREAT changes, love the simplicity and that syntax is MUCH cleaner. I would love to be able to get to <myvar> within the filter rather than varlist[myvar] for some extra functionality if we can get there, but these recent changes are still big improvements. Oh, and I like the :let naming myself for what it’s worth, Thanks!
I also like :let, but :set might be a good alternative if we need one. As I said above, I think it’d be helpful to leverage users’ knowledge of existing TW terminology, and many people already associate let and set with assigning variable names.
I’d also love to be able to use enlist<myvar> as an alternative to varlist[myvar], if that’s at all possible.
Thanks for the feedback @stobot @etardiff.
With the latest updates, assignments like <$let a={{{ [all[tiddlers]] }}} > will now create a multi-valued variable containing the complete results instead of the usual single valued variable. We therefore need to make sure that it always behave as a single valued variable for all existing usages of variables. That means that all of those existing usages cannot be changed.
In particular, the problem with allowing <myvar> to return a result list instead of a single value is that it would mean that an assignment like the example above would no longer behave in the same way as at present: because it would now be interpreted as an assignment to a multi-valued variable, all the values would be retrieved by the <myvar> syntax instead of just the first.
One way around these issues would be to introduce a new syntax for accessing multi-valued variables as operator operands. It seems like it might be useful: it would allow us to simply pass lists to user defined functions, and introduce much simpler list operations like combine, intersect etc.
The syntax would have to be a pair of one or more characters that can be used to quote the variable name. There aren’t many open/close character pairs that we haven’t used and that are easy to type on ordinary keyboards. The only obvious one is round brackets:
[my.function(multivaluedvarname),<anothervarname>,(anothermultivaluedvarname)]
That seems inconsistent with our existing usage of <varname>. Another alternative would be to use double angle brackets:
[my.function<<multivaluedvarname>>,<anothervarname>,<<anothermultivaluedvarname>>]
Neither of these proposals is 100% backwards compatible, but it seems unlikely that anyone is using variable names that start with < and end with >, and so I wouldn’t think it would be a problem.
It might be worth mentioning a related idea that I have considered: to introduce quote symbols for using an entire sub filter as an operator operand. It’s obviously crazy because it leads to an even more unintelligible soup of similar looking symbols. For example:
[my.function{{{ [<something>addsuffix<anotherthing>] }}}]
Again, maddeningly, I don’t think we can do this with a significant break in backwards compatibility. The problem is essentially the same as above, but it can be restated in simpler terms. If myvar was assigned as <$let a={{{ [all[tiddlers]] }}} > then it would behind the scenes be a multivalued variable. The proposal here is that because the variable is multi-valued then the enlist operator would return all the values in the variable, which would not be compatible with the existing usage which would take the first result in the list and treat it as a title list.
This case could be fixed with an explicit suffix to the enlist operator: enlist:all[] perhaps, or enlist:multi[]. However, that does seem a cop out, and rather undermines the goal of introducing as little new syntax as possible.
With the second proposal above of using double angle brackets to indicate access to a multi-valued variable, it would be possible to use [<<multivar>>] within a filter to obtain the complete result list from a multi-valued variable.
Hmmm.
Personally, I’d really prefer single round brackets to double angle brackets for this usage. We spend so much time telling people not to use constructions like my.function<<multivaluedvarname>>; I imagine that confusion will only increase if new users are encountering that sort of syntax.
I’m also rather partial to enlist:all, though; as new syntax goes it’s certainly better than varlist, and looks less like an error than the double angle brackets.
Even more time is spent forming the habit after being told how to do it the right way. Indeed this would be confusing and what makes the matters worse is that there’s no linter that could produce verbose error messages in these situations, so yes, I’m afraid it is only going to make it harder.
Maybe we could have a special naming convention that, if used, treats MVVs differently, similar to functions and the period. Myself, I never use function[my-function] in a filter, but rather .my-function[]. Similarly, if we “mark” MVVs with something like a plus in their name, [<my+variable>] would be a shortcut for varlist[my+variable] – since the latter would seldom be used, its unwieldy name is of secondary concern.
Also, similar to function syntax, this might help WikiText programmers to identify which variable is an MVV and which is not. I would probably implement some kind of naming convention anyway, to preserve what’s left of my sanity, and would welcome some guidance that directly derives from core functionality.
I am with @etardiff and @vuk – I think [<<multivar>>] is confusing. Especially, since there are many existing posts here in the forum, which tell our users a different story.
I would prefer [(mulitvar)]
@Yaisog suggested a special+naming, which I think is also confusing.
For the reasons stated here, I definitely agree that the round braces would read better.
But I’m curious as to whether we would have an obvious parallel for double parentheses outside a filter.
| Brackets | Inside [filter run]
|
Outside filter run |
|---|---|---|
[ … ]
|
act.on[title] |
[[title]] |
{ … }
|
use{transclusion!!var} |
{{transclusion!!var}} |
< … >
|
work.with<my-variable> |
<<my-variable>> |
( … )
|
use.all(multi-val) |
(( ??? )) |
While such a parallel clearly is not essential, it would be nice to have this sort of consistency. Is there a logical candidate?