Macros, Filter Expressions and WikiText Parsing

[Yaisog]

I am often confused by how to get a string of WikiText displayed with (inline) parsing applied. I am always trying around until I find something that works. Since the number of options has recently increased with the introduction of filter functions and procedures, I made a reference tiddler for my personal use that might be of use to others, too, maybe.

The title of the Tiddler is New ''Tiddler'' and contains the following definitions at the top:

\define modified-title-define() {{{ [<currentTiddler>uppercase[]] }}}
\procedure modified-title-proc() {{{ [<currentTiddler>uppercase[]] }}}
\function .modified-title-function() [<currentTiddler>uppercase[]]

I used this title to generate a string that is readily available in currentTiddler, but I want to modify it with a filter function, here uppercase for demonstration purposes. Otherwise I’d just use <<currentTiddler>>.

I am posting a screenshot, since it’s difficult to transfer the tiddler output to TWTalk.

The output I want is the unlinked string (because the modified tiddler doesn’t exist) with the word Tiddler in bold and uppercase, same as $wikify outputs – but noone uses $wikify if not absolutely necessary. The only other way to get this seems to be using $let / $set.

If someone has other ideas and/or examples let me know.

Here is a link to a sharing edition with the tiddler.

[Yaisog]

The only place in the official docs that I found where something related is discussed is a tiddler called Places where the parser ignores WikiText. That isn’t really of much help here, though.

[CodaCoder]

No harm in trying…

<$text text=`${ [.modified-title-function[]]  }$` />

Nope.

[TW_Tones]

@Yaisog I am quite familular with all the forms you have used above but “reading between the lines” I suspect I know the origin of this question.

First try this {{!!title}}

• I understand your examples are for illustration only but I would not include markup in the title.
  • it is important to seperate the operation of your tiddlywiki from its presentation.
• an example closer to everyday use would be using markup in the caption field. For example core buttons include icons in the caption field. However this is a display step.
  • the caption field can contain markup and it will be parsed when you transclude the caption field.

Background
I hope this helps. This is for anyone passing this way, struggling with parsing questions.

Being a wiki and markup/markdown platform tiddlywiki has a step you do not see on other platforms. I/we call this the “render” process.

This is the last thing that happens before you see the result on screen. Basically it converts the output to html and displays it. This happens automatically and you don’t need to do anything.

• In the same process just before the conversion to HTML the parser runs through your wiki markup, tiddlywiki script etc… to determine what to display. In many ways with the exception of trigger events like buttons and actions this is the only time tiddlywiki is “processing” content. It even finds out what else it needs to update on the screen and does that.
  • I tend to call this the “evaluation” step,
    • This is when the TiddlyWiki script, wikitext parsing is done and the DOM is populated.
    • Only then is it converted to HTML

As designers, programmers and tiddlywiki enthusiasts we tend to think and live in the world of tiddlywiki script and markup, we tend to think of this all as a programing language, and it is very close to that, but it has this render step at the end, we often forget.

• Your question is some ways relates to this functionality. Perhaps you want to render the output, before the final automatic step?
• I often want to get a “rendered” result and use that in my tiddlywiki script.
• Finally, as I have learned recently as a general rule as soon as you use a Widget you are determining an outcome to be displayed.
  • It is difficult to use the output of a widget as the input to tiddlywiki script, because it (the widget) is a display “feature”
    • This includes {{!!caption}} because is is a shortcut to $transclude widget, it can be used as a parameter but it “only” gets evaluated before the widget/macro/procedure that uses it, is evaluated.

Just to reassure you tiddlywiki is decades old now, and most gaps have being filled, but if it already has a good solution we tend not to change it. Transcluding may be the current solution to your current problem. As is using the new function to evaluate a filter to use in subsequent tiddlywiki script.

• There are other workarounds we can discuss.

It is important to gain a conceptual understanding of the rendering process to put these questions in context.

I invite any well informed people like @pmario and @jeremyruston to comment or edit this reply for clarity.

[Yaisog]

Hi @TW_Tones, I think you might have misunderstood what I was trying to say. My example is very far from my real-world usage and only there to illustrate the point without much distraction.

First off, it’s not about using formatting markup in the title. This will not be displayed correctly in most places, especially the ViewTemplate. This was done only to be able to use currentTiddler in the code, which is a generally well-known variable, instead of other constructs that are harder to understand. Yes, caption might be a good field to use markup in, but in the “real world” I might just be using any string containing markup in a variable.

Also, I want to apply some filter to that string before output. So {{!!title}} will not do. Again, uppercase is just an easily understood example and has nothing to do with my real-world problem.

The output difference between inline filters, macro calls of inline filters or macro calls of variables was not clear to me beforehand.

@TW_Tones

the caption field can contain markup and it will be parsed when you transclude the caption field

This sounds like the caption field gets special treatment, which is probably not what you’re trying to say. Whether something gets parsed or not should not depend on the source (field), but only on the method used to transclude the contents. In my examples, instead of <currentTiddler> I should be able to use {!!caption} with the same result (if the contents of the caption field is the same).

Anyway, my point was to create a reference for myself and others to show how to get inline formatting applied when displaying a computed string. It is possible without using $wikify, by using $let, as I’m trying to avoid $wikify for such purposes if at all possible. This is the only other way I found, there might be others, though.
I am not suggesting that anything should be changed about that.

Yaisog

[TW_Tones]

@Yaisog

I think you might have misunderstood what I was trying to say. My example is very far from my real-world usage and only there to illustrate the point without much distraction.

As I said;

@TW_Tones

I understand your examples are for illustration only

Please also consider I said;

@TW_Tones

This is for anyone passing this way, struggling with parsing questions.

Because the title of this topic is “Macros, Filter Expressions and WikiText Parsing”

@Yaisog

in the “real world” I might just be using any string containing markup in a variable.

• Perhaps you can find a real world and minimal example.

@Yaisog

Also, I want to apply some filter to that string before output. So {{!!title}} will not do. Again, uppercase is just an easily understood example and has nothing to do with my real-world problem.

• Related to this I recently asked Why cant we also use the view widget for variables? because this has a baring on what you raised.

@Yaisog

I should be able to use {!!caption} with the same result

This means you are using this inside filter, the result is the output will typically be treated as a title, unless you take steps to do otherwise.

• Using the format and or join operators, can help here, as well as using variables that contain markup symbols you want to include.
• The new substitute operator may also help place variables within a prepared string of markup.

@Yaisog

string of WikiText displayed with (inline) parsing applied

I am sure I can help, whether I find a way or be in a position to request a specific change.

• For example I see you are not using the function in some of its new forms, or as a custom operator.
• Also depending on what you are trying to achieve a custom widget, or an existing widget could be created to both “assemble a string, and have it parsed as wikitext”.
  • Not withstanding Why cant we also use the view widget for variables? which is an identified gap.
    • If this were possible you could construct wiki text for formatting in a variable, then have it displayed with the parsing of that taking place without using wikify.

Finally, regardless of your belief I may have misunderstood you, I am confident if you understood what I outlined in my reply (and perhaps you do), that you would know enough to;

• Reduce your confusion
• Solve your own problem
• Be able to describe your current problem with a better example.

Yours sincerely, with a view to being helpful.

[pmario]

I think there is a misunderstanding of the “filtered transclusion” in the first place, that causes problems with the understanding of functions and procedures later in the process.

{{{ [[your filter]] }}} is a shortcut for <$lilst filter="[[your filter]]"/>

which internally is converted to a widget-tree like this:

<$list filter="[[your filter]]">
<$link/>
</$list>

If you open the widget-tree for all 3 versions, you’ll see the same structure.

So {{{ xxx }}} will always create a link as an output. Sometimes, that’s not the desired outcome. So we had to do this:

<$text text={{{ [[your filter]] }}}/>
or
<$text text={{{ [[your filter]format:titlelist[]] }}}/>

Where the later version also shows the braces like: [[your filter]]

Since the filtered transclusion syntax could not be changed in a backwards compatible way, the \function syntax was invented, which behaves in the same way, but will always create text output.

\function f.xx() [[your filter]]
\function f.yy() [[your filter]format:titlelist[]]

<<f.xx>>
<<f.yy>>

---

\define and \procedure are very similar. They only behave different with text-susbstitution and variable handling.

• define can do this: \define test(xxx) $xxx$ - <<__xxx__>>
• procedure uses variables like this: \procedure test(xxx) <<xxx>> … procedures don’t do text substitutions

hope that makes sense.
-mario

[Yaisog]

Hi @pmario, thanks for the explanations. And like I said, I am not complaining that anything doesn’t work as it should, but merely chronicling how it does. The bit about what the filtered transclusion expands to when used in normal WikiText (as opposed to filtered attribute values) was very helpful, though, in understanding the observed behavior.

I do think, in light of this, the docs tiddler Transclusion in WikiText is incomplete. It only expands the regular transclusion shortcut syntax. I’ll propose adding a paragraph with your input.

[TW_Tones]

[Edited for completeness] as I made an error.

We can use \define test(xxx) $(var)$ $xxx$ <<__xxx__>> <<var2>> can also access previously defined variables <<var2>>

Amazing how quickly I forgot the old macros with the new features.

[pmario]

@pmario

• define can do this: \define test(xxx) $xxx$ - <<__xxx__>>

@TW_Tones – This code is intentional. The first $xxx$ is a “find replace” and <<__xxx__>> is a special local variable, that gives us access to the the variable defined in the \define text(xxx)

In many cases we had to cover the substituted variable within tripple double quotes to avoid problems, if the variable also contained a single or double youtes.: \define test(xxx) """$xxx$""" - <<__xxx__>>

There is no problem with the local variable <<__xxx__>>. It does not have a problem with quotes in the text.

[Yaisog]

So, as I was editing the doc tiddler I inspected the widget tree for the examples given therein. What I noticed was that the example given in Generated Widgets, {{MyTiddler||MyTemplate!!myField}} does not work. The generated $transclude widget is not

<$transclude $tiddler="MyTemplate" $field="myField"/>

but

<$transclude $tiddler="MyTemplate!!myField"/>

which transcludes nothing. Similarly, {{MyTiddler!!field||MyTemplate}} does also not work (it only transcludes the text field). It seems that text references and templates cannot be combined. I’m not sure if this is a bug or a doc error.

[TW_Tones]

I have bumped into this limitation before. Which was very close to the reason I said this How to reference a field in another tiddler - #2 by TW_Tones in the documentation in Transclusion in WikiText you can see it is divided into transclusions AND separately “Transcluding Text References”, it makes no mention of the hybrid form.

• That is, it is only documented as such by the fact this hybrid form is not documented directly.
• Although this mentions ParametersWidget what I do below, which is an alternative, and provides a little more info.

@Yaisog

{{MyTiddler||MyTemplate!!myField}}

• So we are not explicitly told this is a valid syntax, apparently it is not.

However;

Keep in mind you could now create a tiddler that you transclude, that includes a parameter, and this parameter could be your fieldname, and the transuded tiddler “gets the field value”. eg {{MyTiddler||TemplateTitle|fieldname}}

The exact syntax and method to use is not easy to find on tiddlywiki.com, and is not mentioned in a few places it could be.

For example in the tiddler “transcluding” add a caption and description then

{{||transcludeme}}

{{||transcludeme|description}}

• Thus

In “transcludeme” tiddler;

\parameters (fieldname:"caption")

{{{ [all[current]get<fieldname>] }}}

• You can also use the $parameters widget inside the transcludeme

<$parameters fieldname=caption> 
{{{ [all[current]get<fieldname>] }}}
<$parameters>

Thus for your example

• {{MyTiddler||MyTemplate!!myField}}
• instead use {{MyTiddler||MyTemplate|myField}} where MyTemplate included code similar to my “transcludeme” tiddler.

[Yaisog]

@Yaisog

I’m not sure if this is a bug or a doc error.

@saqimtiaz @pmario @jeremyruston: Core developer opinions explicitly invited.

[saqimtiaz]

This seems like an error in the very least, barring any historical quirks that I may not be aware of, and even so documentation should show the recommended approach which would be to use the $field attribute.