You mean that you want to include the a shortened version of https://tiddlywiki.com/#Filter%20Parameter into the documentation for every filter parameter? Why not just link to that tiddler in the “parameter” heading of the summary table? Oh wait, that’s already being done. 
Mine thought too, FWIW. Wrapping in [ ] at the end is a bit like dotting my 'I’s and 't’s at the end of writing a word. I don’t actually write my filters like that, but that’s how a parse them.
Of course these do not work if used apart, the issue Jeremy and many others of us face is our knowledge of filters is similar to toys we remember from childhood. So precise its hard to view with naive eyes/minds.
eg tag«something» and get«fieldname» and joined; [tag«something»get«fieldname»]
Not really, “I think want”, the syntax to read range«begin»,«end» and direct the user to documentation and examples that show the use of “Hard/Soft-Direct/indirect” or the more user friendly “literal/transclusion and variable”.
- Ideally with a tooltip on the syntax
1 Like
I’ve been watching this topic for a bit now, and I have a simple suggestion about the variable markers that I’m surprised nobody’s brought up.
Couldn’t we just use all-capital letters for the variable markers, such as BEGIN / END / STEP?
As pointed out in previous posts:
- Color poses a problem for colorblind users.
- Bracket characters like
< >are part of regular TW bracket use, so we should avoid using them as they’re confusing for new/light users, as Scott points out in the OP. - Double-chevron characters like
« »aren’t part of all standard keyboard layouts and may be difficult/annoying for some people to insert. They might also be confused for other standard TW bracket use by new users, and certainly don’t help with the “no double brackets” rule they’re trying to remember.
Additionally:
- The “bold and italic text generators” Yaisog mentioned earlier may not all output screen-reader friendly glyphs. I’ve previously seen people pointing out such a side effect out on other sites: when their screen reader runs into those “fancy” italic/black-letter/cursive strings of text that you see some people using on social media, it can start spitting out illegible “garbage” for them.
- Applying basic styling like italics could pose a problem for users who override page CSS for visual reasons. Their variable markers could disappear entirely!
But capital letters:
- are generally not currently used in our documentation, so they’d stand out as “something is different here”
- can be typed on any keyboard layout
- don’t add additional characters (thus keeping the docs’ size down)
- use no bracket characters so there’s no chance of confusing users as to whether the brackets are supposed to be included or not
- pose no problem with CSS styling or any overrides to it
- are perfectly accessible to people using screen readers (as far as I am aware, correct me if I’m wrong)
- and most importantly, should(?) be an easy find-and-replace on the existing documentation
Especially if paired with a standardized note prefacing the examples such as “Terms in ALL CAPS are placeholders for your values”, then wouldn’t an all-caps method be both easy to create and simple to understand?
[range[END]]
[range[BEGIN],[END]]
[range[BEGIN],[END],[STEP]]
Personally, I use all-caps as a variable placeholder all the time in my personal documentation notes for both TW and Linux, because it’s instantly obvious how I have to substitute my actual value into a string when I see UUID-GOES-HERE or [[YOUR_FILTER_RUN]].
*Edited to simplify the prefacing “note”
4 Likes
I do like that idea. Will be worth a test.
I think that is definitely worth considering.
But there is still a flaw: What needs to be substituted is not END, but [END] (with the brackets). While this could be substituted with [42], it could also be substituted with {!!last-value} or <my-counter>. And that is hard to convey if we continue to use a bracket that is valid syntax.
This is not a problem for the examples, if @Springer and I get our way and lists of example contain several of these options. But for the syntax overview, I would really like to see a fix.
I don’t have the answer. I don’t really like the guillemets. Unless you’re French, « and » are probably difficult to type. But I’m struggling to find something that works which actually is easy to type. The only obvious alternative – ( and ) – is claimed in a current PR.
I think I see what you’re trying to say about the substitution, but if I’m reading it right, it’s not as much of a issue with the usage of all-caps themselves as it is with the explanation text I included, correct? (“Terms in ALL CAPS are placeholders for your values or variables.”)
I just tossed that phrase in there as a general placeholder explanation, but I see now that it would be smarter to omit the “or variables” at the end. Otherwise it could confuse new users who, as you pointed out, don’t know yet that you also need to replace the [ ] around the “END” when you replace the value with a variable. (Update: I’ve edited the note in my prior post accordingly.)
- Improving the documentation about variables to include a section on how to use a variable as an operator parameter – which is strangely missing from “Variables in Filters”! – is another topic entirely, but would cover the above new-user help, because then they’d know they need to substitute
[END]instead of justEND.
If you tell people that they have to replace the uppercase text, they will understand that [range[5],[10]] is likely legitimate. But there’s no reason that they would recognize that [range<stop>,<start>] is also legitimate, as is [range{My Contants!!first},{My Constants!!last}]. That’s the problem with using [ - ] or < - > or { - } to designate the parameters that need to be changed.
Most TW users do not read some linear tutorial about using filters, but first poke around trying to solve their immediate problems. A user might happen to see someone’s use of range[5], understand what it does and visit the range Operator documentation to figure out how to extend it to have both a start and end parameter. If they want to use their variables, they might well try this version:
<!-- BROKEN -->
[range[<start>],[<stop>]]
because they’re substituting the uppercase text of BEGIN and END with their variable references. If they want to try again, perhaps they’ll notice the small link Learn more about how to use Filters, and visit Filters, which is too high level for them, but its list of links to follow starts with Introduction to filter notation, which sounds helpful,. When they get there, though, they see an introduction and a second section, which together have thirteen filter examples, every one of which uses literal values in square brackets. Only if they’re intrepid will they go onto the second section, titled “Special parameters”. There’s a good chance they will have given up on this as not helpful by now, and “special parameters” doesn’t sound like it’s going to help.
And that’s why I think we need to somehow make it more explicit that any brackets meant to be replaced by real ones are dummies. But again, this is a large change, time consuming and fiddly.
1 Like
clearly the solution is to have all the brackets at once! ⦕ ⦖ (or is the correct order ⦖ ⦕ ?! Ok ok, what about 〖 〗
(seriously, I remain a fan of « » in part because they’re trivial to type under linux at least (compose+<+<) but even so, the danger of any unicode exploration is difficulty and confusion, another example of which is that the following are all different characters, but may look identical (or very very similar) depending on your local fonts «《 ⟪. Crucially, to the casual explanation, all would be described the same and I can easily imagine them all ending up in documentation without extra care 
)
A quick additional thought:
It’s not ideal for printed or screen-reader purposes, BUT:
a hover effect on parameters might be very helpful, where
[range[18],[65]]
is what you see, but if you hover over [18] the expression [18] gets a background-highlight effect, and the tooltip shows
[18](hard parameter)
{!!minimum-age}(indirect)
<changecount>(variable)
or something like that. Just brainstorming here…
Certainly something like this can help. In practice, still, the problem is that mid-level users often are uncertain exactly where/how a variable can be swapped in. Having filter parameters jump out somehow, with clarity about the common ways that curly and angle brackets look in practice, would be ideal.
2 Likes
While I’m not generally a fan of obscuring TW mechanisms behind documentation macros, I do think that a procedure (or perhaps a <$docs.filters> widget, as @Scott_Sauyet suggested) might be the best way to balance ease of input with the need for more sophisticated (and ideally, transparent) display. Here’s a quick, procedure-based mockup of the sort of dynamic display I would find most useful myself, borrowing some inspiration from @EricShulman’s earlier suggestion:
\function .docs-param-config() [[$:/temp/docs-config/]] [<storyTiddler>] +[join[]]
\procedure .docs-param(label)
\function .param-type() [<.docs-param-config>get[text]] ~string
\function .param-display()
[<.param-type>match[variable]then[<XX>]]
~[<.param-type>match[transclusion]then[{!!XX}]]
~"[XX]"
+[search-replace[XX],<label>]
\end .param-display
<<.param-display>>
\end
\procedure .docs-param-select()
<$select tiddler=<<.docs-param-config>> class="tc-float-right" default="string">
<option>string</option>
<option>variable</option>
<option>transclusion</option>
</$select>
\end
<pre><<.docs-param-select>>
[range<<.docs-param end>>]
[range<<.docs-param begin>>,<<.docs-param end>>]
[range<<.docs-param begin>>,<<.docs-param end>>,<<.docs-param step>>]
</pre>
Sample output:
Obviously that <pre>...</pre> section is terrible to type or read; my main goal here was simply to show the sort of dynamic display I was envisioning. Ideally the code needed to turn
[range[<begin>]]
[range[<begin>],[<end>]]
[range[<begin>],[<end>],[<step>]]
(or similar) into a <pre> block with the appropriate macros could itself be contained in a hypothetical <$docs.filters> widget, but I haven’t done much work with custom widgets myself and am not very confident with the $slot mechanism, so I’ll leave that as an exercise for anyone who feels inspired. I took a crack at this in a later post in this thread.
Edit: I think @Springer’s concern about screen readers may apply here too… while the filters themselves would be plain-text (and could be easily copy-pasted, etc.) the selection mechanism might be less transparent…
3 Likes
I’ve been thinking along the same lines, but keep getting stuck on how to mark this up so that this would work. It also increases the burden on docs maintainers, and there’s already a dearth of that. But as I was also planning on making the opposite point in response to @Nemo, perhaps I’m still mostly confused.
I’m less concerned about ease-of-typing here than some others. Documentation is meant to be read much, much more than it’s written. It feels like a fairly minor thing.
I feel a little like, “Scott, meet Scott” here. Sorry.
Took me a while to research how to get them on a (non-french) mac:
⌥ \ (option + backslash) = «
add shift to above for »
It occurs to me that if all filter parameters were typed this way on tw-com, they could still by default DISPLAY as square brackets (!), but with some kind of css that helps us easily overlay info about how anything within that parameter expression (internally coded as «18» or whatever) responds to these 3 very important ways that parameters can be specified.
This is fantastic! I think this is the best pointer for the correct direction we’ve seen yet.
While I wouldn’t mind making a stab at a widget-based version, I can’t make any commitments; I haven’t done much with them either, and I don’t know what time I’ll have.
One nasty problem. We really would like to show each of the following:
[range{tid1},{tid2}]
[range{!!fld1},{!!fld2}]
[range{tid1!!fld1},{tid2!!fld2}]
but that would require an additional string for each parameter name.
Not to mention the possibilities of mix-and-match:
[range<start>,[100],{!!count}]
My head’s hurting!
1 Like
Hmm, why? That feels entirely backward to me. For me, the whole point is to not confuse users into thinking that our “replace-me” tokens are legitimate syntax, or, worse, that they are the only legitimate syntax.
I also want to stress again that there are two distinct uses of filters in the operator documentation. Here, I’m worried about the syntax descriptions. But there are also examples. For those, I feel like the solution is obvious, if tedious: just add enough of them to get a feeling of the variety of options that are available. But they need none of this special handling. People should be able to cut and paste them directly.
2 Likes
Technically it suffices to do the third, right?
The first and second just default to text field and currentTiddler. Granted the long form might be intimidating, since it’s rarely actually needed, and is visually noisy compared to {!!fieldvalue} or {tidreference}
Right, so I was thinking about the relevant CSS (that would swoop in when there’s a parameter, expressed as «18» (or whatever) behind the scenes… so that [18] is displayed, while ALSO adding GUI styling that cues the viewer to recognie that whole expression as a unit, AND adds a hover-tooltip with suggestions about indirect and variable expressions.
After someone sees that kind of thing and explores it a couple times, they might recognize the visual style of a parameter, and get used to thinking that square brackets around parameters are valid, but not like square brackets at the edge of a filter expression. They always open up into these more nuanced possibilities.
I don’t like the idea of actually using «» characters in a way that can be seen / copied though … because I think that whatever code we have should be copy-pasteable and should just … “work”. (And like you I’m worried about confusion… new users thinking they should be figuring out how to type « and » 
)
Is there a smart way to get tw-com to overlay a kind of “smart syntax” about where what’s being shown is a filter-operator parameter and somehow make a tooltip there?
1 Like
I was thinking about that too. I’d initially considered providing a select widget on a per-param basis, so you could change the display of a given param “in place”… but that would sacrifice the ability to copy an entire filter line, and I think it would be far less screen reader-friendly. I ultimately concluded that providing just one realistic sample per category would still be better than what we’ve got currently, and hopefully readers will understand that they can mix and match. (Though if we’re handling most of the display with procedures and/or widgets anyway, it wouldn’t be difficult to add a note pointing this out explicitly, or even a few links to other relevant tiddlers.)
As for the various types of transclusion, it wouldn’t be too much trouble to add a line item for each; we’d just have to decide which default tiddler title to use. I did consider adding an option for HelloThere!!field; I’d decided against it as I feared it might suggest that “HelloThere” actually had the specified field and would produce a meaningful result.
1 Like
Even apart from the whole test-case tiddler mechanism, surely a couple common-sense tiddlers, with recognizable names and field values, would be helpful at tw-com, right?
So, {{SolarSystem!!planet-count}} or {{JeremyRuston!!favorite-color}} or {{MotovunJack!!birthyear}} would then be directly available as field value references. 
1 Like
Absolutely. This is my top concern as well.
I think procedures are the simplest way to do this without adding any new syntax to the parser. For instance, I can quickly tweak the code I shared above to add some (ugly) highlighting to my example:
\procedure .docs-param(label)
\function .param-type() [<.docs-param-config>get[text]] ~string
\function .param-display()
[<.param-type>match[variable]then[<XX>]]
~[<.param-type>match[transclusion]then[{!!XX}]]
~"[XX]"
+[search-replace[XX],<label>]
\end .param-display
@@<<.param-display>>@@
\end
Of course, I’d want something a little less eye-searing for the actual docs! But I do like the idea of using CSS classes to provide some additional visual cues; it emphasizes that the brackets are part of the parameter. And if we were using <span> instead of @@, it’d be easy to add tooltips and aria-labels, too.
1 Like
I said I wasn’t going to attempt a custom widget, but apparently I couldn’t help myself. Here’s a self-contained example (raw code follows at the end of the post): Sample $docs.filters.tid (2.0 KB) (Now revised as discussed in post #55, below)
It converts code like this…
<$docs.filters>
[range«end»]
[range«begin»,«end»]
[range«begin»,«end»,«step»]
</$docs.filters>
into output like this:
I have to confess that I had no idea how to type guillemets myself. (For my fellow Windows users, it’s Alt-174 for « and Alt-175 for »; the numbers must be typed on your numpad.) I’d really prefer a more standard character for use on TW-com, but I copy-pasted a few of them anyway to use as quick delimiters; they are, if nothing else, easy to read, and unlikely to appear in sample filters.
Other notes:
- I used $wikify to capture the value of
<$slot $name=ts-raw />for use in filtering, but surely (hopefully?) this isn’t best practice…? - I used
<<color primary>>to add some quick, palette-responsive color to the parameters. For a more polished version, we might prefer a more thoughtfully chosen palette. (Perhaps a different color for each type of parameter?) - I didn’t put a lot of effort into styling the $select widget, but IMO, it would look better either outside the bounds of the
<pre>block or inline with the first line of code. - I added a quick tooltip to the $select to help clarify its use, but obviously, a
<<lingo>>macro would be preferable. We could also include more explanatory text before or after the codeblock; here, I was trying to keep the display as clean as possible.
Full sample code:
\widget $docs.filters()
\function .docs-param-config() [[$:/temp/docs-config/]] [<storyTiddler>] +[join[]]
\procedure .docs-param(label)
\function .param-type() [<.docs-param-config>get[text]] ~string
\function .trim-label() [<.param-type>!match[string]] :then[<label>trim[]] ~[<label>]
\function .param-display()
[<.param-type>match[variable]then[<XX>]]
~[<.param-type>match[transclusion]then[{!!XX}]]
~"[XX]"
+[search-replace[XX],<.trim-label>]
:map[<.param-type>match[variable]then{!!title}search-replace:g[ ],[-]else{!!title}]
\end .param-display
<span class="tc-param"><<.param-display>></span>
\end .docs-param
\procedure .docs-param-select()
<$select
tiddler=<<.docs-param-config>>
default="string"
class="tc-float-right"
tooltip="Select a type of parameter to see it used in the following filters.
You may mix and match parameter types within a single filter run."
>
<option>string</option>
<option>variable</option>
<option>transclusion</option>
</$select>
\end .docs-param-select
\whitespace trim
<style>
.tc-param { color: <<color primary>>; }
</style>
<<.docs-param-select>>
<pre>
<$wikify name=content text="<$slot $name=ts-raw />">
<$list
variable="output"
filter="""[<content>search-replace:g[«],[<<.docs-param "]search-replace:g[»],[">>]]"""
>
<<output>>
</$list>
</$wikify>
</pre>
\end
<$docs.filters>
[range«end»]
[range«My Tiddler»]
[range«begin»,«end»]
[range«begin»,«end»,«step»]
</$docs.filters>
3 Likes