Introducing the testcase widget

The upcoming v5.3.4 release includes a new <$testcase> widget that is intended to help solve a couple of related problems:

  • To make self-documenting, interactive examples for learning wikitext, widgets and filters
  • To make automated unit tests that verify that the elements of a TiddlyWiki application are working correctly

You can try it out here – (this is a permalink to the new documentation tiddlers)

Making Documentation Examples with the Testcase Widget

The core has long used some basic custom macros to present examples in the documentation. These macros are workable for simple, self-contained examples, but can be extremely hard to follow in cases where the examples are also using additional tiddlers. The <$testcase> widget solves that problem by displaying interactive examples that show the output together with a tabbed display of the constituent tiddlers that produce it.

Here’s an example of a testcase:

The tabs on the left give the source of each tiddler in the test case, and the panel on the right shows the output.

The source tiddlers can be edited, with the significant limitation that edits will be lost if the widget is refreshed.

Test cases can specify their expected results, allowing them to be run as unit tests. Here’s an example of a failing testcase:

Making Unit Tests with the Testcase Widget

The core testing framework has included support for test case tiddlers for a few years. The core is gradually switching over to them in preference to our original JavaScript tests, and we already have more than 130 test cases in the test edition. The new thing in this release is being able to run the test cases interactively in the browser, unlocking their potential as a learning tool.

Unit tests can also be useful for wikitext authors. For example, if you are working on a custom function you might create a series of test cases that illustrate the ways that it works. By including the expected results, the system can automatically verify that a unit test passes. This can be done on the command line or via the control panel. It then becomes much easier to modify and improve that function because you can rely on the test cases to find anything that breaks.

Inside the Testcase Widget

The testcase widget is essentially a lightweight version of the existing <$innerwiki> widget. It constructs an entirely separate wiki store object, fills it with the required tiddlers, and then renders a template that by default provides the tabbed view of the source tiddlers alongside the output.

The example above was generated with:


<$testcase>

<$data title="Description" text="How to calculate 2 plus 2"/>

<$data title="FirstNumber" text="2"/>

<$data title="SecondNumber" text="2"/>

<$data title="Output" text="<$text text={{{ [{FirstNumber}add{SecondNumber}] }}}/>"/>

</$testcase>

As part of these improvements, the <$data> widget that originated as a component of the <$innerwiki> plugin is moved into the core. It is also extended with support for reading tiddlers from “compound tiddlers”, which are the special format originally used by the TiddlyWiki test suite for embedding multiple tiddlers within a single tiddler.

Testcase Tiddlers

Testcase tiddlers are single tiddlers containing a payload of multiple tiddlers that together comprise a self-contained test. There is a special view template cascade that causes testcase tiddlers to be displayed by the testcase widget. Here’s an example.

The compound tiddler format packs multiple payload tiddlers into the text field, separated by + on a line by itself. For example:

title: First
tags: one two

This is the first tiddler
+
title: Second
tags: three four

This is the second tiddler
+
title: third
tags: five six

This is the third tiddler

Testcase tiddlers

12 Likes

This is fantastic!

I do have a few questions

  • Is the difference between a documentation-style output and a test-case one triggered by inclusion of an ExpectedResult tiddler? Or is there something else going on?
  • I see the +-line delimiter. Is this going to have (the likely very minor) effect of not allowing a single + on a line within the text of an included (micro-)tiddler?
  • If we’re switching to this style of testing for internals, should we change on not-yet-merged PRs? (I’m thinking specifically of 7990, but I imagine others are also affected.)

Thank you! It is something I’ve wanted to do for some time, and it’s come out nicely.

That is correct, you just need to include a tiddler called “ExpectedResult” in the payload, containing the HTML of the expected output.

That’s correct. It would be a serious limitation if these tiddlers were intended to contain arbitrary user data, and there aren’t any attractive ways to “fix” it. But it is not a problem because the entire goal of these compound tiddlers is to make test cases easy to write, without having to switch between multiple tiddlers.

It will be a very gradual transition. In the case of PRs like 7990 perhaps there might be some advantages in switching, particularly the possibility of using the test cases in the documentation, but at this point I don’t think there are hard and fast rules.

Thank you, this is magnificent!
I can think of a dozen use cases for me.
It’s amazing how TiddlyWiki still evolves, great!

Thank you, much appreciated.

It would be interesting to hear more about the use cases that have occurred to you. The community has already been very inventive with the <$innerwiki> plugin.

I’ll get back to you as soon as I’ve finished something.
At the moment I usually have more ideas than time to actually realise them - unfortunately. Tiddlywiki is so incredibly versatile and I regret not having discovered it earlier.

Thanks for this @jeremyruston it is delivering some real features designers need.

The following question is asked before I look at the widget closely;

I am wondering if there is a way to use the test case widget to “document” actively some of the current core processes, like pulling together the tiddlers involved in the view template or cascade etc…

  • This would make a great example of the testcase widget, and suggest a path to documenting such processes?

Has anyone made tools to pack or unpack tiddlers defined in this particular compound tiddler format? Or should we?

I see testActions, but, if this is still 5.3.4-prerelease, can I propose:

preTestActions: Actions that can be executed prior to the test.
postTestActions: Actions that can be executed after the test.

testPreActions and testPostActions would be fine, too, if you prefer to stick with a common naming style.

And…

testLogTo: Title of tiddler used as a logger (typical timestamped log lines).

Obviously, postTestActions could be used to achieve the same.

1 Like

It looks great @jeremyruston! It will make documentation of TW and of own creations much easier to handle.
I have a couple of questions and comments.

1: The description of the $testcase widget is not a link in example-usage (for documentation/demonstration only, not testing, no expected result) and links to “Output” tiddler (in host wiki) in the test-usage. Is this intended?
I would propose that the description links to the TestCaseTiddler used, otherwise it is not a link; regardless of whether it is a test-usage or example usage.

1A: Side comment: maybe we should agree on some official terms for what I call test-usage and example-usage? These are obviously separate things, with different application and styling. I think the documentation could be clearer if we explicitly name these.

2: The mechanism of TestCaseTiddlers seems to be made only with test-usage of the $testcase in mind, not with example-usage.
Would it make sense to have a new tag, perhaps $:/tags/wiki-test-example, which would cause the tiddler tagged with it to render as $testwidget with no expected output (“example-usage”)? The tiddlers tagged so would not be included in ControlPanel, since they are not tests.
This would be intended to make creating examples/demonstrations using TestCaseTiddlers easier to work with.

2A: Side question, what does the “-spec” in $:/tags/wiki-test-spec and $:/tags/wiki-test-spec-failing stand for, and why not to skip it at all?

3: Wouldn’t text/vnd.tiddlywiki-compound type be less confusing for the Compound Tiddlers, instead of text/vnd.tiddlywiki-multiple?

4: It seems the description field of TestCaseTiddlers replaces the functionality and overrides the Description payload tiddler (if present). I can imagine reasons behind this, it would be clear to have this fact stated in the docs.

5: I can’t find the documentation of the testcase procedure.

6: I could help with documentation (see 4. and 5., there are some minor typos too). Is it appropriate to commit at this point in the master branch? Or is it better to commit them to the tw-com branch after v5.3.4 release?

7: I can’t achieve the example-usage look (without the :heavy_check_mark: / :x: sign and error message if not passed) using the testcase procedure. This is possible using the testcase widget, as seen in the presented examples. Seeing that the procedure doesn’t do anything but transclude the testcase template, I think the issue is with the template itself rather than the procedure.

8: I find the current styling, particularly the blue background and the inner shadow in the rendered output a bit distracting. It looks alright with the Vanilla palette and a couple of other light core palettes, but quite out-of-place in most, particularly in the dark ones.
The nested testcase example has a yellow background, is this intended? Was it meant for all failed test, also the non-nested ones?
I could help here too.

Very cool!

Small nudges for naive readers!

  • A comma (or parentheses) would prevent a syntactic red herring here :laughing: (and I actually did read it wrong for about 20 puzzled seconds, parsing the bolded words together):

This test case intentionally fails to show how failures are displayed.

This test case intentionally fails, to show how failures are displayed.

This test case intentionally fails (in order to show how failures are displayed).

As one grammar website puts it: “Let’s eat, Grandma!” is just a comma away from “Let’s eat Grandma!”

  • Also: it’s bit odd (for any beginner) if the title of the mock-tiddler (like “Simple transclusion”) behaves like a link, but in fact yields a missing tiddler called “Output”.

At any rate, tools for dynamic and interactive documentation are fantastic, and much appreciated!

2 Likes

Would this widget make more sense as part of the jasmine plugin, or some other core plugin, rather than as a core widget?

I do not think so. The testcase widget should have a lot of potential for documentation, which needs several additional “content” tiddlers, to really show the potential of widgets or eg: filter operators in a wider context.

Managing such examples within 1 tiddler imo is a big advantage.

But how often do people make TiddlyWikis with documentation? It seems like in those cases, it wouldn’t be hard to include a tiddlywiki/documentation plugin.

I’m not planning to die on this hill. Only suggesting that this idea be considered to help limit potential feature bloat. A plugin would be easier to deprecate or change down the lane if the need were to arise without introducing widgets to core that must forever be supported.

Edit: Don’t get me wrong. I definitely see the benefit to these widgets. And if they’re included, I may use them in my Relink and Uglify demos, but those demo TiddlyWikis can easily include a core plugin to do so. This functionality wouldn’t be needed for the actual Relink/Uglify plugins end-users would install in their own TWs.

1 Like

+1 to a postTestActions param!

With game making tools like tw-gamification plugin, we can create a playground that add “Points” when user complete a wikitext puzzle or tutorial.

Hi @Flibbles an excellent point, and I should have mentioned before that the motivation for including the testcase widget in the core is so that plugins can use it in their documentation without having to require the user to install another plugin before the documentation works.

An example is the geospatial plugin which introduces quite complex new primitives and the docs benefit enormously from the test cases. See the preview build.

1 Like

I should have clarified that I mean the documentation that is built into the plugin, and accessible as tabs in the plugin view:

2 Likes

Could I ask someone with familiarity of this new widget to give an example how one may include function and procedure definitions, perhaps there values possibly independent of the output, basically allowing one to see the default value of such before the output uses them?

  • This would illustrate some of its functionality, and or may point to a minor improvement?
  • It would allow a kind of debug tiddler to be presented to the user, the intermediate results of supporting code.

Hi @TW_Tones could you expand on that? Function and procedure definitions are included in testcases as payload tiddlers. The tabs on the left allow the values of payload tiddlers to be inspected independently of the output.

Thanks @Springer

Thank you, that is a bug, which should be fixed now.