How to do a more semantic join on a list

I’m so happy for the relatively recent join attribute on the ListWidget. But it’s a bit too naive for my tastes. It give this behavior:

A B C D: “A, B, C, D”
A B C: “A, B, C”
A B: “A, B”
A: “A”

When what I really want is this:

A B C D: “A, B, C, and D”
A B C: “A, B, and C”
A B: “A and B”
A: “A”

(and yes, I like Oxford commas! :stuck_out_tongue:)

I can write a procedure to do this:

\procedure add-commas(filter)
<$list filter=<<filter>> counter="counter"><% if [<counter-first>!match[yes]] %><% if [<counter-last>match[yes]] %><% if [<counter>match[2]] %><$text text=" and "/><% else %><$text text=", and "/><% endif %><% else %><$text text=", "/><% endif %><% endif %>{{!!title}}</$list>
\end

Question 1: That’s fairly ugly. Is there a cleaner way to do this?

Question 2: It would still need to be extended to accept a procedure/macro to handle the display of the list item—we won’t always want just {{!!title}}. I’ve done that before with macros and <$macrocall>, but my quick attempt to do that with <$transclude> didn’t work, and nor did it work properly when I fall back to macros. Do you have any pointers for me?

Question 3: Does anyone have a good way to lay out <% if %> clauses for inline use (meaning that they generate no extraneous spaces.) I did the following but am not real happy with it:

\procedure add-commas(filter)
<$list filter=<<filter>> counter="counter"><% 
    if [<counter-first>!match[yes]] 
%><% 
        if [<counter-last>match[yes]] 
%><% 
            if [<counter>match[2]] 
               %><$text text=" and " /><% 
            else 
                %><$text text=", and "/><% 
            endif 
%><% 
        else 
            %><$text text=", "/><% 
        endif 
%><% 
    endif 
%>{{!!title}}</$list>
\end

Do you have a cleaner suggestion? (Yes, I do know that space collapse in HTML, but I would likely want to use this inside preformatted sections as well.)


You can see these in action, including my broken attempt at passing a macro by downloading the following and dragging it to any old wiki.

add-commas.json (2.9 KB)

You can include the \whitespace trim pragma at the top of the procedure, and it should resolve this issue:

\procedure add-commas(filter)
\whitespace trim

Prior to the welcome introduction of $list join=, I’d started using a modified version of Ben Webber’s list-inline macro, and it’s still my go-to when I need something more complicated than a basic comma join. I’ve added a number of optional parameters as I needed them, so my version now includes slots for a prefix, suffix, emptyMessage, and “fill” - i.e. the content of the list item. I haven’t rewritten it to 5.3.0+ standards, though… perhaps I should.

I’m not sure I quite understand the issue you were having, but I just tried adding a “fill” parameter to your procedure as I’d done with my macro, and it seemed to work all right.

\procedure add-commas(filter, fill:"{{!!title}}")
\whitespace trim
<$list filter=<<filter>> counter="counter">
	<% if [<counter-first>!match[yes]] %>
		<% if [<counter-last>match[yes]] %>
			<% if [<counter>match[2]] %>
				<$text text=" and "/>
			<% else %>
				<$text text=", and "/>
			<% endif %>
		<% else %>
			<$text text=", "/>
		<% endif %>
	<% endif %>
	<<fill>>
</$list>
\end

For example:

<<add-commas "[tag[HelloThere]]" fill:"{{!!title}} CONTENT <<tag>>">>

<<add-commas "[tag[HelloThere]]" fill:"<<counter>>) {{!!title}}">>
1 Like

I’ve seen that all over the place in the source and in others’ code, but for some reason, never really thought to use it myself. Too many bad memories of HTML tags are still affecting me! Thank you.

Looks like I’ve half-reinvented it. I’ll have to look at the implementation.

Yes, that does fine. I think I was confused by the complexity of the last time I did this, where I was passing additional—fairly complex— macros to another macro, and then including them by name using a value extracted from other data. This is much simpler. Thank you very much!

2 Likes

I do think this is now somewhat trivial with functions, the requirment for the and is something I have not done yet. jonining with comas is trivial, If my regex and tw regex skills, eg search and replace operator, were better I would replace the last comma (if any) with " and " then I would have a complete solution in at most two functions.

I’m struggling to imagine how this would work while preserving Scott’s desired functionality (including Oxford commas and permitting list item templates other than static text). It would be pretty simple if the join attribute had access to the counter variable, and you could do something like this…

NONFUNCTIONAL CODE

\procedure add-commas(filter, fill:"{{!!title}}")
\function join() [<counter-last>match[yes]] :then[<counter>match[2]then[ and ]else[, and]] ~[[, ]]
\whitespace trim
<$list filter=<<filter>> counter="counter" join=<<join>>>
	<<fill>>
</$list>
\end

… but it doesn’t seem to. I’d be curious to see your approach!

Edit: Answering my own question:

\procedure add-commas(filter, fill:"{{!!title}}")
\function join()  [<counter-first>match[yes]then[]] ~[<counter-last>match[yes]then<last>] ~[[, ]]
\function last() [<counter>match[2]then[ and ]] ~[[, and ]]
\whitespace trim
<$list filter=<<filter>> counter="counter">
	<<join>><<fill>>
</$list>
\end

Relatedly:

Here’s the flexible version I ended up with for my own use:

\procedure list-inline(filter, fill:"{{!!title}}" join:", ", last:" and ", joinLast:"yes", prefix, suffix, empty)
\whitespace trim
<% if [subfilter<filter>] %>
	<<prefix>>
	<$list filter="[subfilter<filter>butlast[]]" join=<<join>>>
		<<fill>>
	</$list>
	<% if [<joinLast>match:caseinsensitive[yes]] %>
		<<join>>
	<% endif %>
	<<last>>	
	<$list filter="[subfilter<filter>last[]]">
		<<fill>>
	</$list>
	<<suffix>>
<% else %>
	<<empty>>
<% endif %>
\end

Normally I’d try to avoid using [subfilter<filter>] more than once. I did initially try to take advantage of the <<condition>> variable available with the <% if %> syntax, and was using something more like this:

<% if [subfilter<filter>] +[format:titlelist[]join[ ]] %>
	...
	<$list filter="[enlist<condition>butlast[]]" join=<<join>>>
	...

But I realized that the enlist step was producing some unwanted results when the <<filter>> outputs themselves included wikitext with links. (In my case, I was retrieving field values with [[mixed-format text]] that shouldn't be split up at [[word boundaries]].) Outside this edge case, I’d expect enlist<condition> to work fine.

I’m not sure how my approach compares to Scott’s, performance-wise; I’ve avoided the counter attribute, but introduced some inefficiency with three runs of [subfilter<filter>]. I imagine it would probably depend on the relative complexity of the filter vs. the number of results it returns.

  • I have not investigated this yet. How would it look? Are they single or multiline?

I have this working;

  • No external macros needed
\define data()
A B C D
A B C
A B
A
X Y Z A B
\end data
\function list.data() [<data>splitregexp[\n]] :map[add.joiners[]] +[format:titlelist[]join[ ]] 
\function add.joiners() [split[ ]butlast[]join[, ]else[]] [split[ ]nth[2]then[ and ]] [split[ ]last[]] +[join[]]


<!-- <<list.data>> each row joined and made into titles -->

<$list filter="[enlist<list.data>]" >

</$list>
  • I am not sure exactly how the nth[2] is working in [split[ ]nth[2]then[ and ]]

Result

You’ve also added functionality, so it’s hard to compare performance. My version did not have your prefix, suffix, and empty, nor does it offer a way to skip the Oxford comma. (But really, who would ever want to?! :stuck_out_tongue:)

I did edit my previous post with a function-based solution inspired by your initial suggestion, in case you missed it! Personally, I’d want a flexible slot that could accommodate something like this:

<<add-commas "[tag[HelloThere]]" fill:"<<counter>>) <$link/>">>

Not me! Honestly, that’s a hold-over from the pre $list join days, when I was using the macro for both semantic joins with “and” or “or” and simple comma lists. And Ben’s macro uses it, so I kept it for feature parity. :slight_smile:

Ideally, they’d look any way you wanted. Think of what <$list filter="..." join=", ">anything here</$list> can do: joining any arbitrary content created per item with commas. I’d want the same thing, except that I could specify a final list join differently, and handle two-item lists somewhat differently. My attempt at this flopped miserably, but Emily showed it was much simpler than I imagined.

My feeling here is a two step approach,

  • A simple join each with string eg ,<space>
    • Although there may be sense in using an obscure joiner eg |
  • A subsequent replace last string (joiner eg ,<space> with alternate eg <space>and<space>
    • Or a butlast and last joiner.
    • One may want to join with other strings eg then
    • All within a function or two

There is a clear difference between joining lists and words (a form of list) within a title.

I’m not sure if you consider that a win, but it doesn’t do what I was looking for: I want something that serves as a simple extension of what <$list ... join=...> can do. Instead of

Item 1, Item 2, Item 3, Item 4

I want it to return

Item 1, Item 2, Item 3, and Item 4

And instead of

Item 1, Item 2

I would like to get

Item 1 and Item 2

Here the various Items could be plain text, but could also involve arbitrary markup, perhaps

<button icon="Icon for Item 1"><action ...handle item 1... />Item 1</button>,
<button icon="Icon for Item 2"><action ...handle item 2... />Item 2</button>, and
<button icon="Icon for Item 1"><action ...handle item 3... />Item 1</button>

Note the commas and “and” at the ends of those lines.

So I want a macro/procedure at the end, because I want to call it like this:

<<add-commas "[tag[HelloThere]]" "<<my-formatter>>">>

Note that this is missing the Oxford comma that I want, but, as Emily noted, is easy enough to make optional.

The trouble with an after-the-fact replacement is that there is no reason to believe that the joiner won’t appear elsewhere, or even that in replacing it you won’t replace an important bit of the HTML.

Imagine joining “Help Grammar Commas” with > into

<a href="#Help">Help</a> > <a href="#Grammar">Grammar</a> > <a href="# Commas">Commas</a>

to display as

Help > Grammar > Commas

and then replacing the final “>” with “and”, to get

<a href="#Help">Help</a> > <a href="#Grammar">Grammar</a> > <a href="# Commas>Commas</a and 

Things probably won’t go well then.

I think I have the same objective, but now I understand where the oxford comma is, I assumed it was the and but in my solution above it was trivial to use , and.

  • I do this in long sentences myself

Snag_1f3bc319

I will attempt to read this more deeply, but I feel I have the answer to the Original Topic Data, now I need to look at how to do this for anything?

  • And generalise it in other ways.

[Edited] Now reading you desire to do this with html I understand why you think my current approach may be a problem, but with any content I would choose an appropriate “joiner” for the content, and most likely delimit it differently, perhaps by modularisation such as turning each link into its own string, then the joiners are outside those strings.

Just want to re-highlight this edit I snuck in, which I believe honors both Scott’s needs and Tony’s love of functions :wink: :

You could also parameterize the joiners for more flexibility:

\procedure add-commas(filter, fill:"{{!!title}}", joinWith:", ", beforeLast:" and ")
\function join()  [<counter-first>match[yes]then[]] ~[<counter-last>match[yes]then<last>] ~[<joinWith>]
\function last() [<counter>!match[2]then<joinWith>] [<beforeLast>] +[join[]]
\whitespace trim
<$list filter=<<filter>> counter="counter">
	<<join>><<fill>>
</$list>
\end

<<add-commas "[tag[HelloThere]]">>

<<add-commas "[tag[HelloThere]]" fill:"<$link/>" joinWith:" & " beforeLast:" last but not least, ">>
1 Like

I did see and and very much like the last version, although, I might stick for now with your version that fixed mine. Even with the nested ifs it feels like simple code.

1 Like

In this case I have found a way to treat each html “line” as an item I list, and render them.

I store the html items in a define macro so it can be rendered directly and the only thing that could interfere are other wikifyable content, buit this is also an advantage.

To assist I created a procedure that accepts as its input a function name list.html.data that contains our list of items, in this case delimited by new line.

I also included a function join.item that if the numeric variables item, items and penultimate are available, it will determine the correct joiner. This is simply placed after each list item.

  • I wrapped all output to hide it if there is no function name provided because its junk
  • I wrapped the list in <kbd></kbd> for fun.
  • I added a title/tooltip that is wikified for fun on commas2, just mouse over

The data

\define html.data() 
<a href="#Help">Help</a>
<a href="#Grammar">Grammar</a>
<a href="#Commas">Commas</a>
<a title=<<transclusion>> href="#Commas2">Commas2</a>
\end html.data
\function list.html.data() [<html.data>splitregexp[\n]]
  • The data can be defined instead as a procedure, I now use \define to represent static content.
    • The data can be replaced with a transclusion to another tiddler eg {{HTML-links}}
  • The function list.html.data has the desired result of evaluating, like wikify, so the content is available in a list for further processing without using wikify.

The Procedure (can globalise)

  • You could also globalise join.item function as long as the variables are available.
\procedure output(function joiner:" >" last-joiner:" and" end:".")
   \function join.item() [<item>compare:number:lt<penultimate>then<joiner>] :else[<item>compare:number:eq<penultimate>then[ and]else<end>] [<item>compare:number:eq<items>then[, and]]

<$list filter="[<function>!is[blank]]" variable=~>
<!-- html.data: <<html.data>> <hr>-->
<$let items={{{ [function<function>count[]] }}} penultimate={{{ [<items>subtract[1]] }}}>
<!-- <<items>> <<penultimate>> -->
<kbd>
<$list filter="[function<function>]" counter=item>
   <<currentTiddler>><<join.item>>
</$list>
</kbd>
\end output

The invocation

<<output "list.html.data">>

The Result

Snag_1f80e6bf

Conclusion

The above should work when given any list via a function name, yet to be tested, but it should accept all kinds of data as long as it’s delimited by new lines, perhaps with the exception of pragmas.

  • This is suited to inline outputs, a block form could be created but you may not want the same joiners.
  • We could even modify the data to have a parameter to set the domain portion of the links.

Interesting stuff you’re doing. I don’t think it’s that closely related to my goal, which was to slightly extend what we could do with <$list ... join=...>. Several versions do this well, including Emily’s latest:

\procedure create-link() <a href={{{ [{!!title}addprefix[#]] }}}>{{!!title}}</a>

<<add-commas "Help Grammar Commas" >>

<<add-commas "Help Grammar Commas" fill:"<div style='border: 2px solid #0f0; background: #dfd; display: inline-block; padding: .25em; '>{{!!title}}</div>" joinWith:" :: "  beforeLast:"" >>

<<add-commas "Help Grammar Commas" fill:"<<create-link>>" joinWith:" > "  beforeLast:"" >>

yields output like this:

Note that I don’t start with HTML, but just with the text "Help Grammar Commas".

Of course it’s not a big deal that you’re pursuing a different problem from the one I’m working on. But it should be noted.

add-commas demo.json (1.4 KB)