Feedback on documentation of filter operators

#21

Geeks Warning !

For users that can read JavaScript or C-like languages and are willing to dig into the TW test code, it is always worth to have a closer look at the TW test edition

In this particular case: each test coverage is at: TiddlyWiki5/editions/test/tiddlers/tests/test-filters.js at 028c80782d105beb90f5d58a7f22e865c7e8c6f4 · TiddlyWiki/TiddlyWiki5 · GitHub

But it is not very detailed atm. There is an improved version in the new PR – which may help advanced users.

Many of the test in the test-edition are part of the TW documentation. But not all of them.

This should be no excuse for missing documentation, but it can help to improve the documentation for those who are able and willing to dig the “rabbit hole”.

-m

#22

You are right. That’s an inconsistency which has potential for more confusion. – It may or may not be fixed. – At the moment I am not sure if it should be fixed.

The implementation of the each-operator is a bit tricky at the moment. It contains nested if()s that are hard to read and debug. From my point of view it should be rewritten.

That’s a very interesting catch, but understandable. Running each[] does all it’s work in 1 run. So if used in the right way it can be an improvement for slow filters that need unique lists.

The unique[] filter needs to go through a potentially large list 2 times. We already know that it can be a performance killer.

#23

IMO, it doesn’t need to be. It’s a little odd that each:value[title] is the one exception. But it is consistent with both

  • each[] = each[title] (and the current docs, where title is explicitly called out as the default parameter for each)
  • and the way TW uses title as equivalent to “the input string”, generally speaking: [title[missing title]] returns “missing title”, even though [field:title[missing title]] does not.

I think it’s a pretty obscure exception, anyway. I didn’t remember having seen each:value in the wild before (in fact, I had forgotten it existed at all before this thread prompted me to read the docs more closely), and a Google search for “each:value[” tiddlywiki just now only turned up two or three real results, including this thread.

This was my intuition too, but I’m glad to have some numbers to support it—particularly because I also find unique[] much more semantically obvious, and I’ve never seen each:value[] explicitly recommended instead, even though it would be a more efficient 1-to-1 substitution.

If each does get rewritten, I hope the better performance can be preserved!

#24

@pmario Thanks for the diligence. It’ very much appreciated. :clap:

Having spent some time digging into the links you provided (thank you!), here’s my take on what I’ve seen. And this I think speaks to the issues that confound readers of the documentation:

The core source documentation, and the user documentation (for each[] in this case) bring about a problem often seen when terms are not clearly defined. It seems that the root cause is the core source documentation, whereby its notion of what things are called and what things are, are not set out clearly. In the case of each[], that same issue seems to have been copied across to the user documentation.

Here are the terms that seem to be sometimes synonyms, and sometimes not:

tiddler → title → value

tiddler is an object with properties { <data> }
title is a string "my title"
value is whatever you say it is in the coding context you are working. It could be a tiddler, or a title, or an array element/member of a list (a list whose properties are unknown outside of the core code’s context). It really could be anything with any possible JavaScript type.

Sadly, value is transplanted from the core source documentation into the user documentation as though the context where it made sense would come right along with it. Bang. Without it, users have no clue (other than guessing) what value truly refers to.

Muddling up tiddler and title happens across user documentation on tw .com. Old-timers like you and I can generally mush them together and deduce what is actually meant. In most cases, we do it without thinking about it. But that’s not what happens in a newbie’s head – which is unfortunate, to say the least

And if this point is lost on anyone, consider any “list of tiddlers” you might think you have in your wikitext/code – it is in actual fact a “list of (tiddler) titles”.

Aside: Where “group” came from, I have no idea. I still don’t get what it means, what the author’s intention was, what it’s referring to or… anything. :exploding_head:

Yes, I know PRs for docs are always welcome. But if i don’t understand it, I can’t offer any help.

^Z

#25

I created the combobox stylesheet back in 2021, tiddlywiki has changed how the core svg is handled now, so the demo will no longer work. Here’s an updated version that works with the latest version: combobox

2 Likes
#26

I did close the PR since the exact issue has been discussed several years ago. I did miss that one. As so often backwards compatibility is the argument here.

There is one thing we can be certain. Whenever we do have a problem to explain a functionality in a consistent and logical way, there is a 90% chance that the underlaying concept has issues and “backwards compatibility” adds to our technical dept.

#27

In my own involvement with tiddlywiki, I think these issues are often the result of the documentation being minimalistic when a new feature was released. At the time, to those who introduce something it makes sense and is consistent. The problem is in time the unwritten meanings are forgotten.

  • If the each operator is an example, I have always being comfortable with it, in part because of what I was trying to do when I first encountered it.
  • Its initial design was very focused on field values each[fieldname] and it is not generic, the keyword each was then expanded with each:list-item and each:values, It is worth reading examples and following the link to GroupedLists because this is almost the why of its existence.

To start with we should expand on changes in the releases and expand the documentation a little. If a little history helps then it may be worth while eg;

The each operator was introduced in its each[fieldname] for finding unique values of fieldnames, and returning a/one title containing that value. Thus a common way to use is it to retrive the list of values of a fieldname each[fieldname]get[fieldname].

  • Since each is a generic word there was value expanding the each operator through the operator suffixes list-item or value

However the documentation of these operator suffixes are simply too brief. I would make a documentation change but to be responcible I would need to confirm through tests my understanding and that will take some time.

Can I suggest the participants in this thread add just a little to the documentation to improve it?

#28

I wonder if this is helpful. I have a start to the updating the documentation for each especially including some examples, written using the testcase widget. I have also slightly modified the descriptions in the each Operator description tiddler.

You can see this by downloading the following and dragging it onto tiddlywik.com:

EachDocs.json (6.7 KB)

image
image1024×731 51.6 KB

I would like it if these examples could all be shown inside a tabbed interface, but when I tried in several ways, they always showed all the separate parts of the CompoundTiddler in one. I’ll try to figure that out later, but first I want to know if people see this as a useful approach.

I would also love to be using the .operator-example mechanism with its dynamic “Try It” button, but I don’t like that this puts a temporary tiddler inside the CompoundTiddler. That confuses things. But I haven’t spend much time on it.

So, is this worth pursuing?

5 Likes
#29

Trust yourself. That’s of enormous benefit. :clap:

#30

I’m not it’s biggest fan by a long way. The worst part is, “view source” is useless. Put it all where users can find it without any need to perform rocket surgery.

#31

I couldn’t tell if you’re talking about the TestCaseWidget or the .operator-example mechanism.

I like the TestCaseWidget a lot. I think it will be a boon for writing documentation that allows for examples drawn from whatever style of wiki the users want, and not just from a documentation wiki. But it still has quite a few quirks that I’m hoping to see addressed. I’ll probably be writing those up sometime soon if I go down this path.

The .operator-example function/macro/proc/whatever is a minor convenience. I could certainly live without it. It makes sense to reduce duplication using something like it. If it weren’t there, I would probably invent my own. When I tried it here, though, on clicking the button, a new tiddler was added to the tabs on the left, with a long system name. That was a major turn-off. Again, if I do pursue this, I may look to create a more appropriate implementation of “Try it” for testcases.

#32

Sorry, the latter. The unnecessary obfuscation of examples on TW .com needs to end, IMV.

#33

This is helpful, in the sense that it’s a description of the algorithm that is written in user friendly human language rather than being too abstract (like current docs). One more improvement in this direction that you could consider is gently tempting the reader to use each. Examples of its usage like the one in the screenshot above answer HOW to use the each operator. I think a finishing touch at the end would be a couple more examples of WHEN to use it. Not even some generic practical scenarios, but scenarios when using each is better than solving the problem in a different way without it - be it shorter queries or faster running ones. I, as a learner with no years of knowledge bias under belt, need a couple of dogmas for each operator. A la “you put the rook on an open line, because there it is most efficient” in a chess context.

#34

That’s because you (and I, sometimes) are well versed in “how things work” – we’re privileged. Not so the average reader.

I do similar things all over my wikis, but they’re not put out there for public consumption. The power of transclusion^n shouldn’t be a barrier to inclusion (of people) and the communication of methods and approaches. It’s so unnecessary and, frankly, foolhardy for a documentation site to obfuscate the “howto” side of things.

You’re aware I work in Test (SDET as they like to call it these day) so I’m a user champion first and the dev’s “friend” ~last.

I think you know what you’re doing is sorely needed (even if this thread is the only evidence for it). So please keep it open and transparent.

1 Like
#35

Do you have any specific examples in mind? When I asked above (and that was not rhetorical; I really had none myself), the only response was @etardiff’s use of [each[field]get[field]. I cover that in the second example. But not in a particularly motivated way, just using colors of the rainbow. These sorts of test cases should lead themselves to motivated examples. I was expecting to do a third one to show a two-level list. Perhaps books grouped by author.

#36

Do you have any specific examples in mind?

Absolutely no, hence my struggle. My mind is spinning around the idea that for many tiddlers with same field value, and the operator grabbing the first one, while not controlling the order of tiddlers, it really looks like a distant cousin of a randomizer. Why should I use each instead of explicit random picking?

#37

I have mixed feelings about that. In the early days of the web, “View Source” was one of the best tools available for learning HTML. But as the platform has gotten more sophisticated, there are much better HTML-learning resources – all written in HTML. “View Source” is only occasionally helpful, and it’s extremely rare that a tutorial for HTML even considers how its own source can be used to learn it. I don’t think that it’s a big deal for TiddlyWiki documentation to work similarly. But… tiddlywiki.com serves many purposes, and one of them is to be a playground for testing things. For that, obscurity in internal implementation is a major red flag to me. And it is hard to tease the two apart.

But I personally would prefer to separate out the documentation into its own site, which doesn’t sound likely to ever happen, so I may not be a good person to speak on this.

For now, I will probably try to continue with what I’m doing and see if we can get a more user-friendly version of the each documentation. Then we can see where it might go from there.

#38

I have a kind of practical example of the each operator here: Demos — Q&A and tools for tiddlywiki

image
image1368×492 30.1 KB

This list each tiddlers located in the same countries. A maybe more useful setup would be to list each song of a particular music album, or each chapters of a book in a wiki containing a library of books …

2 Likes
#39

I have no idea whether this is a common practice, but I have a lot of fields that I use like the tags field — that is, title-list formatted connections to other tiddlers (or potential tiddlers) that I want categorized in appropriately named fields for ease of use in filters and ViewTemplates. When I first started out, I tended to use tags for everything, but that quickly led to a lot of tag pollution… and the more tags you have on a given tiddler, the less useful they become, particularly if you’re mixing structural and informational tags.

In any case, if you do structure your wiki this way, [each:list-item[fieldname]] fills a similar role to the more specialized [tags[]]. I think I end up using it this way far more often than vanilla [each[fieldname]] or even [each[fieldname]get[fieldname]].

2 Likes
#40

Another draft: EachDocs.json (8.8 KB)

This one includes a two-level list: Show each author and a list of books by that author:

image
image2012×1944 196 KB

I was working on a third example myself which is similar, using books and authors. I’ll look at this more closely when I try a :list-item example.

I keep thinking I may be headed that way too, although I still mostly use Tags. But again, this will be helpful when I switch to :list-item.

Would either of you be interested in writing up a small sample testcase that could be used the way my (now three) examples work?

2 Likes