Parameterised Transclusion Introduction

Announcements
#1

Despite the fact that v5.2.4 has not yet been released, there is now a pre-pre-release of v5.3.0 available:

https://tiddlywiki.com/prerelease/parameterised-transclusions/

The purpose of the pre-pre-release is to give everyone an opportunity to study the significant improvements that we collectively call “parameterised transclusion”, and to kick off a discussion about them. (v5.3.0 also includes all of the enhancements in v5.2.4 that you can read about at https://tiddlywiki.com/prerelease).

I won’t try to fully explain the new features here because it’s more useful to point people at the documentation and use it as an opportunity to identify areas that can be improved.

There is a brief introduction to the new features in the release note that comes up when you visit the v5.3.0 pre-pre-release, including links to the key entry points for the new documentation. There is also a high level summary of the changes aimed at end users at the top of the GitHub pull request.

I will pick out one small example of the usage of these new features. It’s to resolve the long standing problem that transcluding one of the core images does not allow direct control over the size of the image, instead requiring a CSS rule to resize these images.

I think @twMat once made a suggestion that it would be useful to be able to somehow pass the required size as part of the transclusion. With parameterised transclusion, that’s exactly what we can now do:

{{$:/core/images/close-button|12px|12px}}

Note the use of a single vertical bar to delimit the parameters that are passed to the transclusion.

To make that work, we need a small modification to $:/core/images/close-button:

title: $:/core/images/close-button
tags: $:/tags/Image

<$parameters width="22pt" height="22pt">
<svg width=<<width>> height=<<height>> class="tc-image-close-button tc-image-button" viewBox="0 0 128 128"><path fill-rule="evenodd" d="M65.086 75.41l-50.113 50.113c-3.121 3.121-8.192 3.126-11.316.002-3.118-3.118-3.123-8.19.002-11.316l50.114-50.114L3.659 13.982C.538 10.86.533 5.79 3.657 2.666c3.118-3.118 8.19-3.123 11.316.002l50.113 50.114L115.2 2.668c3.121-3.121 8.192-3.126 11.316-.002 3.118 3.118 3.123 8.19-.002 11.316L76.4 64.095l50.114 50.114c3.121 3.121 3.126 8.192.002 11.316-3.118 3.118-8.19 3.123-11.316-.002L65.086 75.409z"/></svg>
</$parameters>

Note the provision of default values for the width and height in the <$parameters> widget; this ensures that transcluding the image without specifying any parameters continues to work as it does today.

Please note that these enhanced, parameterised images are not currently in v5.3.0, but I do hope to include them once the main PR is merged.

The plan is that v5.2.4 will be released soon (hopefully in November), and then if all goes well we will immediately merge the parameterised transclusion pull request into what will then become the pre-release of v5.3.0.

Why all of this added complexity to our release process? The reason is that the work on parameterised transclusion is fundamental to the future of TiddlyWiki, and so it is important that we get the best feedback we can before we make any final decisions.

At this point the changes are provisional, and explicitly subject to change. We’ve already had a lot of very helpful feedback on GitHub that has helped improve the quality and flexibility of these changes. But now that we have the basic reference documentation in place it is easier for non-developers to understand the changes and follow progress. (Documentation is like a bottomless put. I’ve spent a long time preparing a good deal of new material, but it is still thin in places, and I hope we’ll improve it together.)

I’ll emphasise again that feedback is welcomed on all aspects of these changes, but there are some specific open questions that I’d like to share here – most of them won’t make sense until after reading the documentation.

  • The documentation introduces a number of new terms to TiddlyWiki. These terms are carefully defined, and are used consistently. However, I am not confident that we’ve arrived at the best choice of words for some of these new concepts. In particular:
    • Is “procedure” confusing for the replacement for macros? It’s definitely quite long to type…
    • Is “function” the best word for custom, parameterised filter operators stored in variables?
    • Is $$ the best prefix for custom widgets?
    • Is . (period) the best prefix for custom filter operators?
    • Are the new <$slot> and <$fill> widgets the best names?

Questions or comments are welcome.

10 Likes
[tw5] Talk.TiddlyWiki has strange flagging rules
#2

Maybe the $ prefix would make more sense, since it’s the sign used for custom widgets ? Using a period could also create confusion with css classes.

To define custom operator, instead of \function, maybe \operator or \filter would be more intuitive?
There is the Array.prototype.filter() method in javascript so filter would kinda make sense …

And instead of procedure, \function or \fn to be shorter (maybe support both syntax) ? Could it be possible to use the arrow syntax somehow ?

\template would also make sense since macros are mostly used for string interpolation (template literals).

They makes sense to me. Slot is used in the official mozilla doc for defining new html element so I find this very appropriate. Maybe the above terms also come from some javascript concept that I do not know ?

#3

My interpretation is that $$ is only used to prefix the widget name in the widget definition - correct? …but this makes it redundant since the widget definition is already “prefixed” with the pragma name itself, i.e \widget ? Or why is there a need for this prefix in the definition? I’m happy that custom widgets are not invoked with this prefix but if they were, then it would make some sense also in the definition.

In my mind the $ character has always signalled “system” and a layer away from the end user. But custom widgets bring the implementation forth to the user, just like other procedures, so it is no longer “behind the scenes”.

The use of a single $ prefix to overwrite core widgets is sensible though!

It would indeed be pretty but I also see potential confusion with css classes. Maybe the prefix could be _ ?

I’ve previously explained my stance on this but it is interesting to see that @telumire also thinks other terms would be more intuitive for what essentially will be a filter expression. Here are some points I didn’t bring up previously:

As both @telumire and I note, an intuitive name for this would be \filter. But given how it would be used, the name would conflict with the filter operator… but so I must ask: Couldn’t this be solved with some enhancement to the filter operator to distinguish the cases, e.g a suffix? After all, we are just talking about where to define a filter expression, right?

Or, still for the filter operator, could perhaps the parameter be treated as for other functionality in TW, i.e if the parameter not defined then the operator falls back to look for a filter defined externally (i.e the pragma). Additionally, if the filter parameter is followed by multiple parameters (comma separated) then it knows right away that it is the pragma (and with the other parameters to be used as parameters for the pragma).

…

Otherwise, if \filterjust ain’t possible, then I think \expression (perhaps with the synonym \exp) would be good. (This also hints at the term “regexp” which might just turn out to be a popular use case for this whole thing. ) Alternatively we could consider \filterop (which uses an abbreviation we’re already using for listops). Actually, the full \filteroperator would probably also be fine. I mean, how many of these things will you write anyhow? (But for macros, which I assume is the most popular pragma, it would be neat with very short synonyms.)

Thanx.

5 Likes
#4

For now, a function is used like this: [function[myfn],[value],...] and the filter operator is used like that : [filter<my-filter>]

If the name is changed from function to filter, this shouldn’t cause a conflict : [filter[myfilter],[value],...] is a different syntax, but then instead of a whole new concept these new filters could be seen as an extension of the filter operator.

This might be easier to learn ? Otherwise, the name “operator” would be pretty clear : [operator[myoperator],[value],...]

1 Like
#5

In my opinion having macro, function, and procedure at the same time is confusing and difficult to follow, especially for new commers. It seems for backward compatibility the macro will be kept and new term procedure is added. I thought can we use def like Python

\def name()   ...

So, this covers both: the old macro, and the new procedure.

This even can replace functions, as function are filter expression, so TW core can check the \def name() ... to see if it contains a filter expression or not! If from performance point of view this has any impact, then we can use a special naming conversion like below

\def _name()  [tag[HelloThere]]

The above definition states that a pragma name has single underscore, so it is a filter expression.
We can think to better prefix.
I know defining function in this way will affect the [function[myfn],[value],...] syntax, but then why not to use [_myfun[value],...]

NOTE: Like Python, I mean here we can use name mangling and this can allow use to have different functionality with simple \def name() ...

2 Likes
#6

If we can avoid $$, it is better to avoid. Like @twMat I think $ signals system and more $$, makes them scary for non-developers. So, if technically it is possible use the single $ and just use some name mangling here again.

I like the . too much. Small, and pretty. As end user or developer, we don’t use CSS class with the prefix (.) inside filters, or as widget attributes. There are exceptions (I think, one place . is used with CSS class is tm-focus-selector), but I can understand @telumire concern in this regard.

1 Like
#7

I like this!

\operator myoperator() ...

...

<$list filter="[operator[myoperator]]"/>

The operator term matches existing documentation, which could be completed with a link to “custom operators”…

Fred

1 Like
#8

I agree with @telumire and @twMat in their doubts and their proposal for new syntax look good. But how @Mohammad had pointed the new syntax could be confusing for many users and he has a gread idea, pointing to the use of the abbreviation.

For me, thinking about syntax and functionality:

  • macros are smart text/html templates
  • procedure looks like a macro, but it is better.
  • function is a smart template for filters
  • widget is also like a template, but the new commers won’t really use it, IMHO.

My conclusions/ideas:

  • Mantain define for macros and use def for procedures (“Procedures are the modern replacement for macros”).
  • We could also use the same pragma def for functions. But if we want differenciate it, we could use a symbol as the pragma widget does. Something like:
 \def $add-tax(rate:"1.10") [multiply<rate>]

I have took the $ from widget syntax, but it could be any other symbol like @. I don’t know if this symbol could be used in the filter syntax, but it adds visibility to the custom filter operator. For comparasi
[<total>.add-tax[1.125]]
[<total>_add-tax[1.125]]
[<total>$add-tax[1.125]]
[<total>@add-tax[1.125]]

  • I have a doubt about the use of fuction in [function[myfn],[value],...], it more complex that the use as custom filter operator [.myfn[value]] and the use of macro with parameters in filters. So I will prefer use filter<myfn "value"> if it isn’t a complete filter expression.
1 Like
#9

I like it!

It “calls” for the external operator - I like it!

#10

Realization:
Significantly simplify the syntax by reusing the same structure for all three “cases”.

Here are three gradually simpler syntaxes. I’m not sure what is possible to implement. They all show the following three things:

the definition

the direct call to the operator function in a filter

the call to the "operator" filter operator in a filter

Thus:


Joint parameter list everywhere, like in the pragma:


Joint parameters and soft brackets, like in the pragma:


There can be little disagreement that the latter is the simplest to learn and remember, but I can imagine this syntax is just not possible for some very good reasons.

#11

I have another doubt, it is about if for have global use of procedures, functions and widgets, we will have to use their own tag or we will use $:/tags/Macro. In this first case (several tag), IMO, this would be overcomplicating the functionality. It will be a point in fovar of the solution of same pragma (\def) and same tag ($:/tags/Macro).

#12

Good point. I have no idea what Jeremy has in mind. One idea could be a general tag for global stuff, e.g $:/tags/Global or something similar… even if I personally think the idea to use tags to control tiddler functionality is a bit quirky. It seems more like a setting for the tiddler that, rather than manually typed, should be selected more akin to how the tiddler type is selected.

TW v5.3.0-pre: It is proposed to add $:/tags/Global for global procedures, functions, widgets, macros
#13

I have only skimmed the conversation but I’d like to voice my strong opposition to having both \def and \define and having them do different things. It’s super confusing and people will keep mixing them up. if anything I think going verbose is better than adding another layer of arcane symbols that do magic things without any regard to what the symbol means in a language or culture.

I don’t have much of an opinion for using $$ for custom widget beyond not fully understanding why - I presume it’s for future-compatibility to avoid future core widgets accidentally getting overridden by custom ones? I’d wonder if a better option wouldn’t be to just use single $ and require explicit overriding, something like:

\widget $myCustomWidget()
// vs
\widget $codeblock()
// vs
\widget override codeblock()

The first one would define a custom widget, the second would do nothing and show a warning to the user of the mistake, the third would actually and intentionally override a built-in widget. Buuuuut… I am a very hands-on and technical person who likes to keep things perfectly clean and pristine. I imagine for a regular user the warning could be confusing. Than anything.

I am a little surprised with the name of the <$genesis> widget. I don’t think it matters for non-programmers what’s the name ultimately, and for programmers <$super> or <$parent> would’ve been much more obvious (and both are a simpler word than genesis which might be tougher to non native English speakers who are not fluent).

I like the idea of functions and I think it’s fine to keep them named that way to avoid the confusion with filters in general, but I don’t see any mention of how they behave with input if you invoke them inside a filter, though I am going to guess they just execute the same way - once per input.

I don’t quite understand the parameterised transclusion though, the word doesn’t appear in documentation and the only difference… I reread the docs and understand it now. The fill/slot feels kinda clunky to use but I’d put the blame on this being HTML, I don’t think there’s a better way to handle it.

#14

Please note I’ve updated my syntax proposal here above. I’d love to hear which ones are at all possible.

#15

In order to retain the ability to use variables or text references for operator names and parameters, my preferred syntax would rather be:

    \operator myoperator(param1, param2) ...

    [...@myoperator[val1],[val2 ]...]

    [...operator[myoperator],[val1],[val2]...]

Please note the slight variation in the operator filter operator invocation, where nested square brackets are avoided.

With this proposal one could use this legit call:

[...@myoperator<param1var>,{param2ref}...]

where the parameters values can be retrieved in variables or text references, or:

[...operator<namevar>,<param1var>,<param2var>...]

where even the name of the user-defined operator can be parameterized.

Fred

1 Like
#16

That is quite true! In new 5.3.0+ we should use the new syntax. The reason \define is kept is backward compatibility!
So, in new release we should not use mixed of them and doc shall be updated accordingly.

Side Note:
One issue with TiddlyWiki doc (tiddlywiki.com) is, it shows the doc from 5.0.0 up to 5.2.3 and this confusing! For example while we have $let, the $var shall be not shown in the doc and it should be left up to the version $let introduced.

#17

@jeremyruston

Is the introduction of Parametrized Tranclusions an opportunity to harmonize how parameter values are appointed in the pragma definition (as default values) and for widget attributes in invocations? I’m referring to the use of both : and = i.e:

\define foo(param:default)
<<foo param:myvalue>>
<$macrocall $name="foo" param=myval />
<$widget param=myvalue />

Note that it even isn’t enough to distinguish between definitions and calls to know what sign is to be used but it really depends on what kind of invocation you use. And with the introduction of custom widgets this discrepancy will stick out even more.

(BTW, I was surprised to realize that in javascript functions, default parameter values are actually set with = so I can’t help but wonder what made TW settle for : in the past?)

#18

There is a very early discussion somewhere at github and it seems we made the wrong decision

\define test(x:default)  ... IMO ":" is OK here, probably could be "=" too

<<test x:10>> ... : .. IMO was the wrong decision
\newenvironment for custom definitions
#19

@jeremyruston

‍1) It seems the transcludewidget will become a very versatile tool, both allowing to present content from transclusions and evaluate procedure calls by replacing the macrocall widget.

I like it! But it stretches the concept of “transclusion”. Maybe we should reconsider the widget name? “Transclusion” is not a term most people are familiar with. The way it is used with this OP I think the widget could instead be called e.g the “presentation” widget.

‍2) It would be useful with a clear overview of what the OP “renders superfluous”. I envision a table with two columns for side-by-side comparison of “the legacy way” vs “the recommended way”.

#20

I think we should point to Transclusion - Wikipedia to explain it in more detail. Especially the “parameterized” section is exactly to the point of the new functionality the transclusion widget has now.

1 Like