Introducing Multi-valued Variables

Even more time is spent forming the habit after being told how to do it the right way. Indeed this would be confusing and what makes the matters worse is that there’s no linter that could produce verbose error messages in these situations, so yes, I’m afraid it is only going to make it harder.

2 Likes

Maybe we could have a special naming convention that, if used, treats MVVs differently, similar to functions and the period. Myself, I never use function[my-function] in a filter, but rather .my-function[]. Similarly, if we “mark” MVVs with something like a plus in their name, [<my+variable>] would be a shortcut for varlist[my+variable] – since the latter would seldom be used, its unwieldy name is of secondary concern.
Also, similar to function syntax, this might help WikiText programmers to identify which variable is an MVV and which is not. I would probably implement some kind of naming convention anyway, to preserve what’s left of my sanity, and would welcome some guidance that directly derives from core functionality.

@jeremyruston

I am with @etardiff and @vuk – I think [<<multivar>>] is confusing. Especially, since there are many existing posts here in the forum, which tell our users a different story.

I would prefer [(mulitvar)]

@Yaisog suggested a special+naming, which I think is also confusing.

For the reasons stated here, I definitely agree that the round braces would read better.

But I’m curious as to whether we would have an obvious parallel for double parentheses outside a filter.

Brackets Inside [filter run] Outside filter run
[] act.on[title] [[title]]
{} use{transclusion!!var} {{transclusion!!var}}
<> work.with<my-variable> <<my-variable>>
() use.all(multi-val) (( ??? ))

While such a parallel clearly is not essential, it would be nice to have this sort of consistency. Is there a logical candidate?

2 Likes

I’ve pushed an update to add support for single round brackets, which also allows the “varlist” operator to be removed. Once again, it seems like a big win.

Very good question. Perhaps ((my variable)) should take a cue from {{{ filter }}} and display all of the results with wikification. On the other hand, I never liked the behaviour of {{{ filter }}} and so we probably shouldn’t perpetuate it.

Maybe instead of just concatenating the results, ((variable) should wikify them with a double line break in between each item. That would generally mean that paragraphs would be generated, which would make things much more readable. It would still largely be a tool for debugging, I suspect.

1 Like

I can’t help dreaming of the possibilities offered by a full fledged syntax like:

((variable||template|parameter1|parameter2))

Fred

I’ve made a further improvement that simplifies things even more: a shortcut syntax => for the let filter run prefix:

[my.custom.function<param>,[2]] =>index [<data>jsonget[top],<index>]

Using the shortcut syntax also avoids having to use double square brackets around the name of the variable.

1 Like

While not strictly WikiText, double parentheses are used for keyboard shortcut references.

1 Like

Jeremy and all,

I hope nobody would be relying on a solution with filters that include angle bracket characters like this, but in theory someone could have backward compatibility problems if their wiki had something like this:

{{{ =<< Fake Guillemets =>> +[join[ ]]}}}

(Above does yield << Fake Guillemets >> on oficial releases up till now, but with the proposed shortcut syntax it would yield nothing, until user makes fix, as in {{{ =[[<<]] Fake Guillemets =[[>>]] +[join[ ]]}}})

Luckily the more likely occasion for => unwittingly appearing in someone’s existing wiki would involve having a space after, and this does not seem to cause a problem (since shortcut is triggered only when => is immediately followed by non-space character(s)):

{{{ =0 =< =1 =< =2 => =1 => =0 +[join[ ]]}}} still behaves as expected in new build preview, to yield
0 < 1 < 2 > 1 > 0

I have no perspective on whether reasonable backward compatibility needs to anticipate such bizarre use-cases as the sloppy Fake Guillemets. Just making note in case further discussion is warranted.

UPDATE: I realize a somewhat less outlandish case is where people are tempted to use >> as a kind of shorthand arrow, as in:

{{{ Family =>> Genus =>> Species =>> Population +[join[ ]] }}}
to yield Family >> Genus >> Species >> Population.

Again, this would be unwise usage (double-angle brackets should never be used as graphic elements by anyone familiar with TiddlyWIki!), and it would be easy to fix once flagged… Conceivably it’s out there somewhere though…

As you say, there are some edge cases that will no longer work as before. My thinking was that these cases involve the use of angle brackets, which most users would be inclined to avoid given the existing usages of angle brackets by the core.

I’m open to suggestion on this, and there is certainly a case to be made for holding this change back until a subsequent release that is planned to break or bend backwards compatibility.

Especially as an R user I like the => notation as that’s really close to the assignment operator there. While a very small thing, it’s VERY notable to me that it’s two characters, whereas all of the other (shortcut) prefixes are single characters. So, while it’s NOT new syntax, it somewhat feels like it is…

what about enlist:vars[]
or enlist:titles[]

I have seen a lot of code where this syntax is used for returning a variable as the currentTiddler in a list result. and I have definitely used this method too:

<$list fitler=" [{$:/config/mysetting}!match[yes]then[<<display-results>>]else[<<hide-results>>]" >
<<currentTiddler>>
</$list>

Apologies @VikingMage I had failed to update the top post with the latest changes. The PR now uses round brackets to retrieve the full list of results from a multi-valued variable. There are more details in the top post, and in the preview:

When I try the above on the preview site, I get Filter error: Missing [ in filter expression. If I change it to [my.function(multivaluedvarname),<anothervarname>,[a constant]] then I don’t get any error (displays all tiddler titles since my.function is undefined). Maybe the current code is only handling the parens syntax for the first argument?

I’m trying the above because I’m skeptical about using special syntax for this. To me it seems better to use an operator to handle the MVV (though difficult to come up with a good name for it).

Unless I misunderstand the code, in order for any javascript operator to make use of an MVV, code has to be explicitly added as you’ve done for the title operator (i.e. [(myvar)] is shorthand for [title(myvar)]).

Do you envision use cases beyond just the title operator for this? If so then will the special case code be added? If not, then it seem like a very big decision to lock in new syntax for not much gain.

Maybe there are gains to be had with this syntax and user defined functions? That’s why I was trying the syntax I quoted from your post. To try to understand the benefits from the user defined function perspective.

Adding this as an operator would involve much less “lock in”. In fact, even with the special syntax, a dedicated operator might still be worthwhile since an operator can have variables as arguments.

Contrived example:

function concat.vars(a1, a2, a3)
[varlist<a1>] :all[varlist<a2>] :all[varlist<a3>]
\end

{{{ a b c :let[[v1]] [concat.vars[v1],[v1],[v1]] }}}

=> a b c a b c a b c

Did you consider the possibility of implementing let as an operator rather than a filter run prefix?

Thanks @btheado funnily enough I just fixed that bug here.

Newly written operators can choose to just use the operand property that gives multi-valued operands, and avoid special cases.

Writing that made me realise the title operator doesn’t use that technique so I pushed an update here.

Reviewing the current list of operators I haven’t come up with any more existing operators that would seem to benefit from multi-valued operands, but I’d welcome suggestions.

However, there is plenty of scope for new operators that take advantage of multi-values - for example, some revised, more concise list operators for manipulating multiple lists at once.

I’ve also just pushed an update allowing multivalued parameters for user defined functions.

With the latest update your example works as expected when updated to use the latest syntax:

\function concat.vars(a1, a2, a3)
[(a1)] :all[(a2)] :all[(a3)]
\end

{{{ a b c :let[[v1]] [concat.vars(v1),(v1),(v1)] }}}

I am dubious about allowing filter operators to have side effects. It’s bad enough that this PR introduces a filter run prefix that has side effects.

There are also ergonomic considerations. I find that the :let filter run prefix or the prefix jumps out much more prominently than operators when visually scanning the text of a filter.

The implementation would be tricky because of backwards compatibility. Filter operators currently return an array of results or an iterator. There would have to be some form of overloading to allow filter operators to also return variables that need to be set for the scope of the remainder of the filter.

3 Likes

The count[] operator could be a candidate for special MVV handling.

Sadly, I don’t think we can make count(var) do anything useful because it conflicts with the existing semantics.

However, counting the number of items in a multi-valued variable is still very simple:

<$let a={{{ [all[tiddlers]] }}}>
<$text text={{{ [(a)count[]] }}}/>
</$let>
1 Like

In order to be able to test and adapt to MVVs in wikis that I don’t want to fully upgrade to the current prerelease, I made a plugin from the core tiddlers in the merged PR. This way I can activate and deactivate MVVs simply by enabling and disabling the plugin in $:/ControlPanel.

It’s a bit late, maybe, but the 5.4.0 release is still a ways off, so it might still be useful for some folks.

$__plugins_yaisog_multi-valued-variables.json (99.3 KB)

PS: Just to throw it out there: Might it be a good idea to automatically generate plugins from PRs, for targeted testing and such?

I think there is no need to make it configurable. The changes should be backwards compatible. If you do not need MVVs, just do not use them. There is a completely new syntax to use them. If you use standard variable syntax there should be no changes.

For new functionality, that is intended to be a plugin → yes.

For generic functions, that will be part of the core → no.

We can not guarantee that those temporary-plugins go away after the new TW version is released. They will cause problems, which will be hard to find and maintain. We do not have the capacity and the mechanisms to get rid of outdated stuff.

Every PR can be tested in isolation with the Netlfy PR preview functionality see: Introduce multi-valued variables and let filter run prefix by Jermolene · Pull Request #8972 · TiddlyWiki/TiddlyWiki5 · GitHub