Introducing the testcase widget

[vilc]

It looks great @jeremyruston! It will make documentation of TW and of own creations much easier to handle.
I have a couple of questions and comments.

1: The description of the $testcase widget is not a link in example-usage (for documentation/demonstration only, not testing, no expected result) and links to “Output” tiddler (in host wiki) in the test-usage. Is this intended?
I would propose that the description links to the TestCaseTiddler used, otherwise it is not a link; regardless of whether it is a test-usage or example usage.

1A: Side comment: maybe we should agree on some official terms for what I call test-usage and example-usage? These are obviously separate things, with different application and styling. I think the documentation could be clearer if we explicitly name these.

2: The mechanism of TestCaseTiddlers seems to be made only with test-usage of the $testcase in mind, not with example-usage.
Would it make sense to have a new tag, perhaps $:/tags/wiki-test-example, which would cause the tiddler tagged with it to render as $testwidget with no expected output (“example-usage”)? The tiddlers tagged so would not be included in ControlPanel, since they are not tests.
This would be intended to make creating examples/demonstrations using TestCaseTiddlers easier to work with.

2A: Side question, what does the “-spec” in $:/tags/wiki-test-spec and $:/tags/wiki-test-spec-failing stand for, and why not to skip it at all?

3: Wouldn’t text/vnd.tiddlywiki-compound type be less confusing for the Compound Tiddlers, instead of text/vnd.tiddlywiki-multiple?

4: It seems the description field of TestCaseTiddlers replaces the functionality and overrides the Description payload tiddler (if present). I can imagine reasons behind this, it would be clear to have this fact stated in the docs.

5: I can’t find the documentation of the testcase procedure.

6: I could help with documentation (see 4. and 5., there are some minor typos too). Is it appropriate to commit at this point in the master branch? Or is it better to commit them to the tw-com branch after v5.3.4 release?

7: I can’t achieve the example-usage look (without the / sign and error message if not passed) using the testcase procedure. This is possible using the testcase widget, as seen in the presented examples. Seeing that the procedure doesn’t do anything but transclude the testcase template, I think the issue is with the template itself rather than the procedure.

8: I find the current styling, particularly the blue background and the inner shadow in the rendered output a bit distracting. It looks alright with the Vanilla palette and a couple of other light core palettes, but quite out-of-place in most, particularly in the dark ones.
The nested testcase example has a yellow background, is this intended? Was it meant for all failed test, also the non-nested ones?
I could help here too.

[Springer]

Very cool!

Small nudges for naive readers!

• A comma (or parentheses) would prevent a syntactic red herring here (and I actually did read it wrong for about 20 puzzled seconds, parsing the bolded words together):

This test case intentionally fails to show how failures are displayed.

This test case intentionally fails, to show how failures are displayed.

This test case intentionally fails (in order to show how failures are displayed).

As one grammar website puts it: “Let’s eat, Grandma!” is just a comma away from “Let’s eat Grandma!”

• Also: it’s bit odd (for any beginner) if the title of the mock-tiddler (like “Simple transclusion”) behaves like a link, but in fact yields a missing tiddler called “Output”.

At any rate, tools for dynamic and interactive documentation are fantastic, and much appreciated!

[Flibbles]

Would this widget make more sense as part of the jasmine plugin, or some other core plugin, rather than as a core widget?

[pmario]

I do not think so. The testcase widget should have a lot of potential for documentation, which needs several additional “content” tiddlers, to really show the potential of widgets or eg: filter operators in a wider context.

Managing such examples within 1 tiddler imo is a big advantage.

[Flibbles]

But how often do people make TiddlyWikis with documentation? It seems like in those cases, it wouldn’t be hard to include a tiddlywiki/documentation plugin.

I’m not planning to die on this hill. Only suggesting that this idea be considered to help limit potential feature bloat. A plugin would be easier to deprecate or change down the lane if the need were to arise without introducing widgets to core that must forever be supported.

Edit: Don’t get me wrong. I definitely see the benefit to these widgets. And if they’re included, I may use them in my Relink and Uglify demos, but those demo TiddlyWikis can easily include a core plugin to do so. This functionality wouldn’t be needed for the actual Relink/Uglify plugins end-users would install in their own TWs.

[linonetwo]

+1 to a postTestActions param!

With game making tools like tw-gamification plugin, we can create a playground that add “Points” when user complete a wikitext puzzle or tutorial.

[jeremyruston]

@Flibbles

Would this widget make more sense as part of the jasmine plugin, or some other core plugin, rather than as a core widget?

Hi @Flibbles an excellent point, and I should have mentioned before that the motivation for including the testcase widget in the core is so that plugins can use it in their documentation without having to require the user to install another plugin before the documentation works.

An example is the geospatial plugin which introduces quite complex new primitives and the docs benefit enormously from the test cases. See the preview build.

[jeremyruston]

@jeremyruston

so that plugins can use it in their documentation

I should have clarified that I mean the documentation that is built into the plugin, and accessible as tabs in the plugin view:

[TW_Tones]

Could I ask someone with familiarity of this new widget to give an example how one may include function and procedure definitions, perhaps there values possibly independent of the output, basically allowing one to see the default value of such before the output uses them?

• This would illustrate some of its functionality, and or may point to a minor improvement?
• It would allow a kind of debug tiddler to be presented to the user, the intermediate results of supporting code.

[jeremyruston]

@TW_Tones

Could I ask someone with familiarity of this new widget to give an example how one may include function and procedure definitions, perhaps there values possibly independent of the output, basically allowing one to see the default value of such before the output uses them?

Hi @TW_Tones could you expand on that? Function and procedure definitions are included in testcases as payload tiddlers. The tabs on the left allow the values of payload tiddlers to be inspected independently of the output.

[jeremyruston]

@Springer

A comma (or parentheses) would prevent a syntactic red herring here (and I actually did read it wrong for about 20 puzzled seconds, parsing the bolded words together):

Thanks @Springer

@Springer

Also: it’s bit odd (for any beginner) if the title of the mock-tiddler (like “Simple transclusion”) behaves like a link, but in fact yields a missing tiddler called “Output”.

Thank you, that is a bug, which should be fixed now.

[jeremyruston]

@CodaCoder

I see testActions, but, if this is still 5.3.4-prerelease, can I propose:

preTestActions: Actions that can be executed prior to the test.
postTestActions: Actions that can be executed after the test.

Hi @CodaCoder would there be a great advantage over concatenating the actions into a single string?

[jeremyruston]

@TW_Tones

I am wondering if there is a way to use the test case widget to “document” actively some of the current core processes, like pulling together the tiddlers involved in the view template or cascade etc…

Yes, we can use the testcase widget to create simplified setups for explanatory purposes. I would have thought that in the case of view templates and cascades it would generally be more instructive to explore the main wiki.

@TW_Tones

Has anyone made tools to pack or unpack tiddlers defined in this particular compound tiddler format? Or should we?

Hi @TW_Tones the compound tiddler format is designed to make it easy to author testcases by hand. If you’ve got a bunch of existing tiddlers that you want to make into a testcase then generally it would be easier to use those tiddlers directly as payload for the testcase widget, rather than copying them into a compound tiddler.

[jeremyruston]

@vilc

1: The description of the $testcase widget is not a link in example-usage (for documentation/demonstration only, not testing, no expected result) and links to “Output” tiddler (in host wiki) in the test-usage. Is this intended?
I would propose that the description links to the TestCaseTiddler used, otherwise it is not a link; regardless of whether it is a test-usage or example usage.

Thanks @vilc that bug is now fixed.

@vilc

1A: Side comment: maybe we should agree on some official terms for what I call test-usage and example-usage? These are obviously separate things, with different application and styling. I think the documentation could be clearer if we explicitly name these.

I am not sure that they should be considered separate things. I quite like the idea that if I am reading about some arcane and complex operator that I can click through to see the full set of testcases in the test edition, and understand the obscure the edge cases.

@vilc

The mechanism of TestCaseTiddlers seems to be made only with test-usage of the $testcase in mind, not with example-usage.

At the moment, the presence of the expected results determines whether the tests are run, and causes the cross or tick icon to be displayed. Are there other visual changes that you are thinking of?

@vilc

2A: Side question, what does the “-spec” in $:/tags/wiki-test-spec and $:/tags/wiki-test-spec-failing stand for, and why not to skip it at all?

Perhaps it would be clearer still to use $:/tags/testcase and $:/tags/testcase-failing?

@vilc

4: It seems the description field of TestCaseTiddlers replaces the functionality and overrides the Description payload tiddler (if present). I can imagine reasons behind this, it would be clear to have this fact stated in the docs.

Indeed, the initial reason was backwards compatibility with existing tests, and then I realised that it was useful to be able to access the description without loading up the testcase and extracting the tiddler.

I’ve added a note to the docs here

@vilc

5: I can’t find the documentation of the testcase procedure.

Thank you that is missing – I would welcome PRs to help with this. At this point they should just target the master branch.

@vilc

7: I can’t achieve the example-usage look (without the / sign and error message if not passed) using the testcase procedure. This is possible using the testcase widget, as seen in the presented examples. Seeing that the procedure doesn’t do anything but transclude the testcase template, I think the issue is with the template itself rather than the procedure.

Very strange. Please could you post the code you tried? It may be worth moving this part of the discussion to a GitHub issue.

@vilc

8: I find the current styling, particularly the blue background and the inner shadow in the rendered output a bit distracting. It looks alright with the Vanilla palette and a couple of other light core palettes, but quite out-of-place in most, particularly in the dark ones.

I’d welcome improvements. It would certainly be good to make sure things look good in the core palettes.

@vilc

The nested testcase example has a yellow background, is this intended? Was it meant for all failed test, also the non-nested ones?

It is intentional that nested testcases get a different colour; otherwise I found them difficult to read. There are actually three custom colours defined in the palette.

[jeremyruston]

@linonetwo

With game making tools like tw-gamification plugin , we can create a playground that add “Points” when user complete a wikitext puzzle or tutorial.

Interesting idea. These would have to be actions performed in the host wiki, not the embedded sub wiki.

I think a more fundamental obstacle here is the transient nature of user modifications made within the testcase widget. We would need to add some way of saving the current subwiki state within the host wiki so that we can re-render it properly if there is a refresh.

[CodaCoder]

@jeremyruston

@CodaCoder

I see testActions, but, if this is still 5.3.4-prerelease, can I propose:

preTestActions: Actions that can be executed prior to the test.
postTestActions: Actions that can be executed after the test.

Hi @CodaCoder would there be a great advantage over concatenating the actions into a single string?

Sorry, I don’t follow.

As originally specified, you (we) have the ability to specify actions prior to the test (“setup”). I was looking to provide actions after the test (“teardown”).

As to your question, I don’t follow where you’re going with that.

[jeremyruston]

@CodaCoder

I was looking to provide actions after the test (“teardown”).

What would be the use of these teardown actions in this context? I’m familiar with them in the context of test frameworks for programming languages, but here we’re essentially working with a VM, so it’s not clear what teardown would be needed.

[vilc]

@jeremyruston

Are there other visual changes that you are thinking of?

I was thinking of the description of the testcase acting as a link, but that has been sorted out with the fix.

@jeremyruston

Perhaps it would be clearer still to use $:/tags/testcase and $:/tags/testcase-failing?

I think this will be much clearer.

@jeremyruston

Very strange. Please could you post the code you tried? It may be worth moving this part of the discussion to a GitHub issue.

I’ll post it to GH too. For anyone reading here, replication:

• clone TestCases/TranscludeWidget/SimpleTransclusion
• rename to Payload
• remove the ExprectedResult tiddler from the payload
• use Payload though the testcase procedure and through the testcase/data widgets, only the latter shows no fail message (as expected when there is no ExpectedResult):

This bug actually leads to one of my previous suggestions, which I have to reformat a bit now.

Currently, if a tiddler tagged $:/tags/wiki-test-spec (or in future $:/tags/testcase) and it contains no ExpectedResult, the “test failed” frame and the icon are shown. Is this the expected behavior?
Wouldn’t it make more sense that a compound tiddler tagged $:/tags/wiki-test-spec/$:/tags/testcase but not containing ExpectedResult be displayed without the fail message?
It might be that both this and issue about using testcase procedure are actually both issues with the template.

[vilc]

I have created a GH issue: [BUG] testcase procedure: “test failed” message displayed even though no ExpectedResult is defined · Issue #8218 · Jermolene/TiddlyWiki5 (github.com)

[CodaCoder]

@jeremyruston

What would be the use of these teardown actions in this context?

In a word, “encapsulation” – of both actions prior and actions after.

Could that be achieved by other means? Of course.