Tricks for refactoring ugly long filters?

#1

I have a filter that converts a string by replacing each char that is a digit with a string consisting of dots as long as that number:

<$let str="8x7x6x5x4x3x2x1">
{{{ [<str>
search-replace[8],[........]
search-replace[7],[.......]
search-replace[6],[......]
search-replace[5],[.....]
search-replace[4],[....]
search-replace[3],[...]
search-replace[2],[..]
search-replace[1],[.]
] }}}
</$let>

I’ve split it to multiple lines to show that intuitively there’s potential for refactoring here - this is just a lot of copypasted code. But I have no idea how to approach it: I barely start to understand the idea of list as a loop, while here I see a nested, two level loop.

A second question is about performance. Does my ugly filter above do one additional pass for each search-replace operator? I could probably implement this algorithm in one pass using a Python oneliner.

Edit: not sure it’s one pass, but it’s indeed an oneliner:

>>> ''.join([''.join(['.'] * int(x)) if x in '87654321' else x for x in '8x7x6x5x4x3x2x1'])
'........x.......x......x.....x....x...x..x.'
1 Like
#2

Here’s one way to do it. Whether this is less ugly or not is probably a matter of opinion, but it is at least less visually repetitive.

\function convert.num() [regexp[\D]] :else[range{!!title}search-replace::regexp[\d],[.]]

<$let str="8x7x6x5x4x3x2x1">
{{{ [<str>split[]] :map:flat[convert.num[]] +[join[]] }}}
</$let>

Steps:

  • split between each character
  • The filter with the :map prefix will be run once per input value. Since I needed a two-part filter to make use of the :else condition, I had to move both parts into the function/subfilter convert.num, which becomes the sole content of my :map run.
    • By default, :map runs yield one output per input value. The :flat suffix switches it to a (potentially) one-to-many relationship, which will allow us to turn one digit into many periods.
    • I like functions, personally, but you could also do this with the subfilter operator, if you prefer. You’d need to make the following changes:
      • \function convert.num() → \define convert.num()
      • :map:flat[convert.num[]] → :map:flat[subfilter<convert.num>]
  • Now, looking at convert.num itself…
    • [regexp[\D]] = search for a non-digit. If this filter run returns any (non-digit) value, the rest of the function will be ignored, and the character is preserved as-is.
    • :else…
      • use range to generate a list of numbers beginning with 1 and ending with {!!title} (= the number). For instance, range[8] will give me 1 2 3 4 5 6 7 8.
      • Use search-replace::regexp[\d],[.] to replace each number in the range output with a period.
  • Finally, +[join[]] everything back into a single string.

And my generalized advice for long, ugly filters: if you find yourself reusing the same string of operators with different parameters, move it into a function instead!

Honestly, I might not have ordinarily bothered in a case like this, since your original is far more transparent at a glance and only ~30% longer. But this was a fun challenge, thanks!

5 Likes
Understanding the filter run prefix concept
Understanding the filter run prefix concept
Rating macro where parameter defines number of time a value is repeated
#3

Thank you.

I must confess that I find your code less intimidating in this form:

\function convert.num(num) [regexp[\D]] :else[range<num>search-replace::regexp[\d],[.]]

<$let str="8x7x6x5x4x3x2x1">
{{{ [<str>split[]] :map:flat[convert.num<currentTiddler>] +[join[]] }}}
</$let>

Maybe https://tiddlywiki.com/#Functions should include a second example with this {!!title} trick when functions are used with no parameter signature, because it really saves typing for explicitly passing currentTiddler as parameter to the function. The existing example that uses an explicitly declared parameter is forming a habit that feels painful to break.

#4

Oops, the one bit I’d neglected to explain!

This trick actually relies on the use of the :map prefix, not any inherent property of functions. For comparison:
image

You can see how the {!!title}/<currentTiddler> reference in .func changes in the second example vs. the first.

  • Example #2 works because :map remaps <<currentTiddler>> to each input value—a bit like the way <<currentTiddler>> changes when you nest a $list in a $list.
  • In Example #1, {{!!title}} continues to equal Draft of ‘New Tiddler 1’, which was where I took this screenshot.

I agree that it would make intuitive sense if <currentTiddler>/{!!title} in a function did refer to the input value received by that function; this would be more in line with the way the filter and subfilter operators work. But I suspect, for backwards-compatibility reasons, we’re not likely to see that change.

If you do need a current-value “placeholder” in a function (for instance, if you want to use :map itself inside a function), the pass-through operators all[] or is[] will do the trick. I demonstrated all[] in Example #3 above. It’s not technically necessary in this specific function — \function .func2() [lowercase[]] would have the same effect — but you can see the difference vs. .func.

2 Likes
#5

I was just merging the wisdom from this thread into my personal knowledge base and since the examples are attached as image, I had to type them.

I was very surprised to see that I’m not getting the same result for Example #3. After multiple visual passes I was already typing a complaint forum post, switched to my wiki to copypaste the code

> {{{ [enlist[A B C].func2[]] }}} -> abc

here. Then I finally (yet purely accidentally, since the function definition was at the top of the tiddler and not in the visible area) noticed that I somehow managed to type it like this:

\func .func2() [all[]lowercase[]]

Hey TiddlyWiki, this was a rookie mistake, but why couldn’t I get a big red scary error message about mistyping the pragma? That’s just not fair! :laughing:

#6

Sorry to necro-post (is that the correct use, asks old fogey?)…

I just noticed that for the sake of modeling (and coming here from the recent rating-stars question), it would be ideal if the function worked for multi-digit numbers. The solution above treats a 10 as if it were just a 1, and 12 as a one and a two (etc.).

I tried seeing if I could make a quick fix, but it seems the crux of the problem requires handling multi-digit sequences in the regex part, which I’m not confident with…

1 Like
#7

You’re right, and I had the same thought at the time, but since @vuk’s original filter didn’t include any multi-digit numbers, I decided not to worry about it. Lazy of me! Here’s a slightly messier version that should handle integers >9:

\function num.map() [all[]] :map:flat[range{!!title}] :map:flat[[.]] +[join[]] 
\function convert.num() [regexp[\D]] ~[num.map[]]

<$let str="12x2x1">
{{{ [<str>search-replace:g:regexp[(\d+)],[ $1 ]enlist-input:raw[]] :map[convert.num[]] +[join[]] }}}
</$let>

Both this approach and my initial suggestion are designed to handle an arbitrary string <<str>> which may contain both digits and any non-numeral characters. But if the numbers in 8x7x6x5x4x3x2x1 are the important part and the x is actually just a spacer character (i.e., it’s the only non-number that will ever appear in the string), here’s a simpler solution:

\function convert.num() [range{!!title}] :map:flat[[*]] +[join[]]

<$let str="12x2x1">
{{{ [<str>split[x]] :map[convert.num[]] +[join[x]] }}}
</$let>

I only know a few regex tricks myself… and all the clever ones I learned from @EricShulman, who probably knows a neater way to do this. :slight_smile:

1 Like