Introduce Multi-valued Variables and Let Filter Run Prefix - Examples

#1

Jeremy recently added a new fiter rub prefix to TiddlyWiki Filter Language. See

He stated that:

This PR introduces a new filter run prefix :let that assigns the result of the filter run to a variable that is made available for the remaining filter runs of the filter expression.

It solves the problem that otherwise it is impossible to compute values for filter operator parameters; parameters can only be a literal string, text reference or variable reference. The :let filter run prefix provides a solution:

[my.custom.function<param>,[2]] :let[[index]] [<data>jsonget[top],<index>]

The shortcut syntax => can also be used:

[my.custom.function<param>,[2]] =>index [<data>jsonget[top],<index>]

Using the shortcut syntax also avoids having to use double square brackets around the name of the variable.

This example first sorts all the input tiddlers by length, and then uses the length of the shortest title as the second index of a JSON lookup:

[all[tiddlers]] :sort:number[get[text]length[]] :let[[index]] [<data>jsonget[top],<index>]

This PR also introduces the ability for multi-valued variables which contain list of results, not just single strings. They can be assigned with the new :let filter run prefix, or the existing <$let> widget. The full list of values can be retrieved using round brackets instead of the usual angle brackets. In all other contexts only the first item in the list is used as the variable value.

This seems exciting. You can evaluate this new feature on TiddlyWiki Pre-release ā€” 5.4.0-prerelease

See also official doc: https://tiddlywiki.com/prerelease/#Let%20Filter%20Run%20Prefix
My question is: Can you give some simple examples can be use by an intermediate TW user?

Examples

These examples are tested on TiddlyWiki Pre-release ā€” 5.4.0-prerelease

Eaxample 01

<$let diff= {{{ [tag[Learning]get[text]length[]sum[]]   :let[[smL]] 
                [tag[HelloThere]get[text]length[]sum[]] :let[[smH]]
                [<smL>subtract<smH>]
			    	}}} >

The difference is: <<diff>>
</$let>

Produces: The difference is: 49437. Here the smL and smH are two variables store intermediate results.

Example 02

You may open TiddlyWiki Pre-release ā€” 5.4.0-prerelease and create few tiddlers tagged with idea. There are some tiddlers with task tag.

<$let mood = {{{ 
    [tag[task]count[]] :let[[numTasks]]
    [tag[idea]count[]] :let[[numIdeas]]
    [<numTasks>subtract<numIdeas>] :let[[diff]] 
    [<diff>match[0]then[[Balanced mindset]]
    [<diff>compare:integer:gt[0]then[Youā€™re actionā€‘oriented]]
    [<diff>compare:integer:lt[0]then[Youā€™re in a creative phase]]
}}}>

<p>Your current mood: <<mood>></p>

</$let>

Produces: Your current mood: Youā€™re actionā€‘oriented

Example 03

<$let weekendMessage={{{ 
    [<now>format:date[DDD]] :let[[today]]
    [<today>match[Saturday]then[Go for shopping!]]
    [<today>match[Sunday]then[Go for exercise!]]
    :else[[Keep focused, itā€™s a weekday.]]
}}}>

Today is: ''<<now format:"DDD">>''<br/>
Message: ''<<weekendMessage>>''

</$let>

Produces:

Today is: Thursday
Message: Keep focused, itā€™s a weekday.

5 Likes
#2

This is awesome! Do I understand it right that this can replace some uses of the <$let> widget? If so, I wonder how well you could break up filter run prefixes with newlines.

Iā€™ll often do something like this:

<$let
something={{{ (filter here) }}}
another={{{ (some other filter here) }}}
>

where having newlines can be helpful.

#3

I am confident new lines are permitted in a filter especially between filter runs.

However this is not so common for myself as I tend to break complex filters into functions

1 Like
#4

Yes, you can store intermediate results into new variables and use them later inside you filter runs.
See edited OP for examples.

1 Like
#5

The original post is a wiki now.
Few examples have been added.

Please add more examples.

1 Like
#6

Examples with subtract[] or divide[] are better than those with add[], since for the latter you can use sum[] in most cases (e.g. Example 1), which has no equivalent for subtraction or division.

1 Like
#7

Thank you @Yaisog. I revised the Example.
By the way have you noticed the new :let ignores error in filter? For example:

[Tested on TiddlyWiki Pre-release ā€” 5.4.0-prerelease]

<$let diff= {{{ [tag[Learning]get[text]length[]sum[]]   :let[[smL]] 
                [tag[HelloThere]get[text]length[]sum[]] :let[[smH]]
		[<smL>subtract<smH>]
			    	}}} >

The difference is: <<diff>>
</$let>

Produces: The difference is: 49437

Now, intentionally remove a bracked from second filter expression, e.g. instead of [tag[HelloThere]get[text]length[]sum[]] :let[[smH]] use tag[HelloThere]get[text]length[]sum[]] :let[[smH]] still it works, I expect a filter error here. Look at the below one:

<$let sample= {{{ tag[Learning]get[text]length[]sum[]]   :let[[smL]] 
				  [<smL>addsuffix[/somthing]]
			    	}}} >

<<sample>>
</$let>

Produces tag/somthing

#8

I am playing with the new filter ā€œrun letā€, and the [(varname)] form to to ā€œget a measureā€ of what this enables, and which of the current challenges it stands to overcome.

  • This is the ā€œmulti-valued variableā€ part not yet illustrated above.

I think this is because of;

The :let filter run prefix always clears the current result list.

I would hope we can illustrate what it enables for a more general audience.

#9

I wonder if thatā€™s intentional. Filters are already hard to debug with only the means provided by the core, not getting an error message makes this harder. Iā€™d think itā€™s less predictable what you would actually get from a faulty filter. Maybe @saqimtiaz or @jeremyruston care to comment.

#10

Please create an issue at GitHub, otherwise it will be forgotten here.

#11

If you put this in a tiddler in 5.3.8 (not having a leading bracket):

{{{ tag[Learning]get[text]length[]sum[]] }}}

you get

tag
0

There is no error, just unexpected output. So maybe :let doesnā€™t ignore errors after allā€¦

#12

Yes, you are right. I checked the example on TiddlyWiki Pre-release ā€” 5.4.0-prerelease

<$list filter="[tag[Learning]get[text]length[]sum[]]    
               tag[HelloThere]get[text]length[]sum[]]" variable=diff>
							 
The difference is: <<diff>>
</$list>

Produces:

The difference is: 59333

The difference is: tag

So it seems TW here ignore the filter error too.

#13

Hi @Mohammad, here is a real-world example for why MVVs and :let are so fantastic:

\function .transfer-function(tapsData,sampleRate:1e9)
	[[{"x":$1$,"y":$2$}]] =>[[data-point-template]]
	[<tapsData>get[text]enlist-input:raw[]] =>[[filterTaps]]
	[(filterTaps)count[]log[2]ceil[]] :map[[2]power<currentTiddler>] [[2048]] +[maxall[]] =>[[outputLength]]
	[(filterTaps)fft[magnitude],<outputLength>] =>[[transferMagnitude]]
	[<outputLength>divide[2]negate[]] =>[[negativeSamples]]
	[<outputLength>divide[2]subtract[1]] =>[[positiveSamples]]
	[range<negativeSamples>,<positiveSamples>divide<outputLength>multiply<sampleRate>] =>[[fValues]]
	[(transferMagnitude)last<negativeSamples>] =[(transferMagnitude)first<negativeSamples>] +[log[10]multiply[20]search-replace[Infinity],[1000]] =>[[HValues]]
	[<data-point-template>substitute(fValues),(HValues)join[,]]
\end

Itā€™s a function that computes the filter transfer function for an FIR filter whose (possibly complex-valued) taps are saved in the tapsData tiddler. It outputs a JSON array (minus the outer brackets) of objects, one for each frequency point, ready for plotting with e.g. Observable Plot.

It takes the taps, calculates the next highest power of 2 of their count, computes and shifts the FFT, collects data for both axes and then puts it all together.

This could have been done (and initially was) with procedures, functions, lots of format:titlelist[]join[] and enlist[], but is now all within a single function thatā€™s easy to copy-paste without missing half of the definitions. Marvellous!

Unfortunately, it canā€™t be used as example on tiddlywiki.com, because it uses a custom fft[] operator and a modified substitute[] operator that can handle MVV parameters (generate a templated output item for each item in the parameter lists). But man, are MVVs and => nice additions.
:grin:

Final output looks like this (uses a custom Observable Plot widget):

image
image954Ɨ546 38.7 KB

4 Likes
#14

Wow! This is amazing! I canā€™t believe such a complex operation can be done with just one function.
Thank you!

#15

Thanks for this excellent example of what you can do. I just thought it worth pointing out even if few filter operators handle the ā€œmultivalue variableā€ that fact is filters are already good at handling lists. So as long as a list can be stored let and retrieved [(varlist)] then 90% of the possibilities are realised.

Although areas for enhancement that come to mind are;

  • Enabling parameters to such functions, that generate lists [(func.name tagname)] however this can be achieved another way, just as a function.
  • Provide a shortform that can return the list (for a quick peep) <(varlist)> or <<(varlist)>>

I could see there there would be value the community brainstorming filters and syntaxes that we may be able to add to the Multi-values variables handling.

Some play

\function input.list() [all[shadows+tiddlers]tag[$:/tags/ViewTemplate]]
\function 2nd.func() [(input.list)addprefix[tag: ]format:titlelist[]join[ ]]

{{{ [(input.list)] }}}

{{{ [2nd.func[]] }}}

<<input.list>>

<<2nd.func>>

{{{ [<2nd.func>enlist-input[]] }}}
#16

One of the things that I think let will make cleaner is when you need to compare the values that result from multiple :or situations. Using map only keeps the result for one more filter run, so if you need an :or in a later comparison, then you have to start using helper functions.

In TiddlyWiki, there are 
<$text text={{{ [prefix[A]] [prefix[B]] +[count[]] =>AB 
                [prefix[X]] [prefix[Y]] +[count[]] =>XY 
				[<AB>compare:number:gt<XY>then[more]else[fewer]] }}}/> 
tiddlers that start with A or B than there are that start with X or Y.

image

BONUS: If you use a font with coding ligatures (In modern Windows, try ā€œCascadia Codeā€) then the shorthand operator is an actual arrow which looks great!

image

I might suggest that Cascadia Code be added somewhere in the list of default fonts for editing ahead of Consolas - Iā€™m not sure how often this is revisited, but Cascadia Code is now the default font for Windows Terminal. Iā€™m using it as my primary TiddlyWiki font these days.

image

2 Likes
Cascadia Code ligatures
#17

Itā€™s also available as a google webfont: https://fonts.google.com/specimen/Cascadia+Code

In case any of yā€™all havenā€™t seen my site detailing use of google fonts for TiddlyWiki (tips, tricks, draggable WYSIWYG per-font tiddlers once you grab the setup tiddlers), hereā€™s a little plug: Google fonts ā€” for TiddlyWiki

EDIT TO ADD: Itā€™s not easy to find the list of ligatures supported by Cascadia Code. I did find it here. Pasting for reference:

image

2 Likes