Introducing the testcase widget

[jeremyruston]

@CodaCoder

I see testActions, but, if this is still 5.3.4-prerelease, can I propose:

preTestActions: Actions that can be executed prior to the test.
postTestActions: Actions that can be executed after the test.

Hi @CodaCoder would there be a great advantage over concatenating the actions into a single string?

[jeremyruston]

@TW_Tones

I am wondering if there is a way to use the test case widget to “document” actively some of the current core processes, like pulling together the tiddlers involved in the view template or cascade etc…

Yes, we can use the testcase widget to create simplified setups for explanatory purposes. I would have thought that in the case of view templates and cascades it would generally be more instructive to explore the main wiki.

@TW_Tones

Has anyone made tools to pack or unpack tiddlers defined in this particular compound tiddler format? Or should we?

Hi @TW_Tones the compound tiddler format is designed to make it easy to author testcases by hand. If you’ve got a bunch of existing tiddlers that you want to make into a testcase then generally it would be easier to use those tiddlers directly as payload for the testcase widget, rather than copying them into a compound tiddler.

[jeremyruston]

@vilc

1: The description of the $testcase widget is not a link in example-usage (for documentation/demonstration only, not testing, no expected result) and links to “Output” tiddler (in host wiki) in the test-usage. Is this intended?
I would propose that the description links to the TestCaseTiddler used, otherwise it is not a link; regardless of whether it is a test-usage or example usage.

Thanks @vilc that bug is now fixed.

@vilc

1A: Side comment: maybe we should agree on some official terms for what I call test-usage and example-usage? These are obviously separate things, with different application and styling. I think the documentation could be clearer if we explicitly name these.

I am not sure that they should be considered separate things. I quite like the idea that if I am reading about some arcane and complex operator that I can click through to see the full set of testcases in the test edition, and understand the obscure the edge cases.

@vilc

The mechanism of TestCaseTiddlers seems to be made only with test-usage of the $testcase in mind, not with example-usage.

At the moment, the presence of the expected results determines whether the tests are run, and causes the cross or tick icon to be displayed. Are there other visual changes that you are thinking of?

@vilc

2A: Side question, what does the “-spec” in $:/tags/wiki-test-spec and $:/tags/wiki-test-spec-failing stand for, and why not to skip it at all?

Perhaps it would be clearer still to use $:/tags/testcase and $:/tags/testcase-failing?

@vilc

4: It seems the description field of TestCaseTiddlers replaces the functionality and overrides the Description payload tiddler (if present). I can imagine reasons behind this, it would be clear to have this fact stated in the docs.

Indeed, the initial reason was backwards compatibility with existing tests, and then I realised that it was useful to be able to access the description without loading up the testcase and extracting the tiddler.

I’ve added a note to the docs here

@vilc

5: I can’t find the documentation of the testcase procedure.

Thank you that is missing – I would welcome PRs to help with this. At this point they should just target the master branch.

@vilc

7: I can’t achieve the example-usage look (without the / sign and error message if not passed) using the testcase procedure. This is possible using the testcase widget, as seen in the presented examples. Seeing that the procedure doesn’t do anything but transclude the testcase template, I think the issue is with the template itself rather than the procedure.

Very strange. Please could you post the code you tried? It may be worth moving this part of the discussion to a GitHub issue.

@vilc

8: I find the current styling, particularly the blue background and the inner shadow in the rendered output a bit distracting. It looks alright with the Vanilla palette and a couple of other light core palettes, but quite out-of-place in most, particularly in the dark ones.

I’d welcome improvements. It would certainly be good to make sure things look good in the core palettes.

@vilc

The nested testcase example has a yellow background, is this intended? Was it meant for all failed test, also the non-nested ones?

It is intentional that nested testcases get a different colour; otherwise I found them difficult to read. There are actually three custom colours defined in the palette.

[jeremyruston]

@linonetwo

With game making tools like tw-gamification plugin , we can create a playground that add “Points” when user complete a wikitext puzzle or tutorial.

Interesting idea. These would have to be actions performed in the host wiki, not the embedded sub wiki.

I think a more fundamental obstacle here is the transient nature of user modifications made within the testcase widget. We would need to add some way of saving the current subwiki state within the host wiki so that we can re-render it properly if there is a refresh.

[CodaCoder]

@jeremyruston

@CodaCoder

I see testActions, but, if this is still 5.3.4-prerelease, can I propose:

preTestActions: Actions that can be executed prior to the test.
postTestActions: Actions that can be executed after the test.

Hi @CodaCoder would there be a great advantage over concatenating the actions into a single string?

Sorry, I don’t follow.

As originally specified, you (we) have the ability to specify actions prior to the test (“setup”). I was looking to provide actions after the test (“teardown”).

As to your question, I don’t follow where you’re going with that.

[jeremyruston]

@CodaCoder

I was looking to provide actions after the test (“teardown”).

What would be the use of these teardown actions in this context? I’m familiar with them in the context of test frameworks for programming languages, but here we’re essentially working with a VM, so it’s not clear what teardown would be needed.

[vilc]

@jeremyruston

Are there other visual changes that you are thinking of?

I was thinking of the description of the testcase acting as a link, but that has been sorted out with the fix.

@jeremyruston

Perhaps it would be clearer still to use $:/tags/testcase and $:/tags/testcase-failing?

I think this will be much clearer.

@jeremyruston

Very strange. Please could you post the code you tried? It may be worth moving this part of the discussion to a GitHub issue.

I’ll post it to GH too. For anyone reading here, replication:

• clone TestCases/TranscludeWidget/SimpleTransclusion
• rename to Payload
• remove the ExprectedResult tiddler from the payload
• use Payload though the testcase procedure and through the testcase/data widgets, only the latter shows no fail message (as expected when there is no ExpectedResult):

This bug actually leads to one of my previous suggestions, which I have to reformat a bit now.

Currently, if a tiddler tagged $:/tags/wiki-test-spec (or in future $:/tags/testcase) and it contains no ExpectedResult, the “test failed” frame and the icon are shown. Is this the expected behavior?
Wouldn’t it make more sense that a compound tiddler tagged $:/tags/wiki-test-spec/$:/tags/testcase but not containing ExpectedResult be displayed without the fail message?
It might be that both this and issue about using testcase procedure are actually both issues with the template.

[vilc]

I have created a GH issue: [BUG] testcase procedure: “test failed” message displayed even though no ExpectedResult is defined · Issue #8218 · Jermolene/TiddlyWiki5 (github.com)

[CodaCoder]

@jeremyruston

What would be the use of these teardown actions in this context?

In a word, “encapsulation” – of both actions prior and actions after.

Could that be achieved by other means? Of course.

[xcazin]

Hi @jeremyruston, thank you very much for this spectacular feature! It occurred to me that the test fail/success could be used in documentation to provide interactive challenges to the user, but it doesn’t seem to work as I expected: if I edit the Output text of the example below to, say, The sum is <$text text={{{ [[2]add[6]] }}}/>, I would obtain The sum is 8 as a result but the test would still fail, which is a bit deceptive

@jeremyruston

Is there a way to configure <$testcase> so that the OutputHTML variable gets updated each time the output result is modified by the user? That would mean that the testOutput text defined in <$testcase> would act more like a default value for the test.

[jeremyruston]

@vilc

Currently, if a tiddler tagged $:/tags/wiki-test-spec (or in future $:/tags/testcase) and it contains no ExpectedResult, the “test failed” frame and the icon are shown. Is this the expected behavior?

Hi @vilc no, that is not intended. As you noted elsewhere, it is a restriction of the testcase tiddler format, and the underlying testcase widget works as intended.

[jeremyruston]

@CodaCoder

In a word, “encapsulation” – of both actions prior and actions after.

Hi @CodaCoder could you give an example of the need for teardown actions? As in what specific actions might be needed? Given that the wiki doesn’t need clearing up because it is not persistent I am struggling to come up with an example.

[jeremyruston]

@xcazin

Hi @jeremyruston, thank you very much for this spectacular feature! It occurred to me that the test fail/success could be used in documentation to provide interactive challenges to the user, but it doesn’t seem to work as I expected: if I edit the Output text of the example below to, say, The sum is <$text text={{{ [[2]add[6]] }}}/>, I would obtain The sum is 8 as a result but the test would still fail, which is a bit deceptive

At the moment the testcase widget performs the test using an offscreen rendering that is independent of the rendering used for display. It is done this way because we need to be able to run the tests without displaying anything when using the option to only display failing tests. But there may be a possibility to do it differently, I’ll give it some thought.

[CodaCoder]

@jeremyruston

Given that the wiki doesn’t need clearing up because it is not persistent

Fair point. YAGNI.

[linonetwo]

A todo on my plugin