A first go at unit testing

,
#1

I spend a lot of time fixing my typos and other mistakes. I have to use tests to ease it up.

So I remembered my java coding time and I came with the idea of twtest (same idea as jtest).

Here is what I achieved in two days:

twtest-sample
twtest-sample1299Ă—737 56.3 KB

This is only a tool for testing filter runs, especially for testing functions. So far, it is meant for functions returning a single elements. (*void^*) is my convention to tell there is no tiddler in the result.

This is made by a small CSS sheet and a single template. Here is that one.

\parameters(label:"no label" filter:"[[no filter]]", value:"no value")

\function .wrong(label, awaited)
  [dump[input for .wrong]]
  +[first[]]
  :map[[<dl class="wrong"><dt>$(label)$</dt><dd>ERROR</dd><dd>GOT: <samp class="got"><$text text="""$(currentTiddler)$"""/></samp></dd><dd>AWAITED: <samp class="awaited"><$text text="""$(awaited)$"""/></samp></dd></dl>]substitute[]]
\end .wrong

\function .wrongVoid(label, awaited)
  [[<dl class="wrong"><dt>$(label)$</dt><dd>ERROR</dd><dd>GOT: <samp class="got">(* void! *)</samp></dd><dd>AWAITED: <samp class="awaited"><$text text="""$(awaited)$"""/></samp></dd></dl>]substitute[]dump[.wrongVoid result]]
\end .wrongVoid

\function .correct(label)
	+[first[]]
  [[<dl class="correct"><dt>$(label)$</dt><dd>SUCCESS</dd></dl>]substitute[]]
\end .correct

\function .verdict(label, value)
  [all[]match<value>]
  :then[.correct<label>]
  :else[.wrong<label>,<value>]
\end .verdict

\function .verdictIfVoid(label, value)
  [<value>first[]]
  :then[.wrongVoid<label>,<value>]
  :else[.correct<label>]
\end .verdictIfVoid

\function .test(label, filter, value)
  [subfilter<filter>]
  :map[.verdict<label>,<value>]
   else[.verdictIfVoid<label>,<value>]
\end .test

<$let res = {{{ [.test<label>,<filter>,<value>] }}}
  real = {{{ [subfilter<filter>dump[real value]] }}}
>
<!--
* filter: <code><<filter>></code>
* awaited value: <samp><$text text=<<value>>/></samp>
* real value: <samp><$text text=<<real>>/></samp>
* verdict text: <samp><$text text=<<res>>/></samp>
-->
<<res>>
</$let>

And an example of use, corresponding to the image shown.

<$let project=bdmsda model=phpdat>
<$transclude $tiddler="$:/user/twtest/templates/ftest.template"
  label="`.getLocation` standard success case"
  filter="[.getLocation[holes]]"
  value="$:/user/data/pcdgen/holes/$(model)$"
/>
<$transclude $tiddler="$:/user/twtest/templates/ftest.template"
  label="`.getLocation` with unknown reference"
  filter="[.getLocation[dummy]]"
  value=""
/>
<$transclude $tiddler="$:/user/twtest/templates/ftest.template"
  label="`.getLocation` test filter that is broken (to test .wrong)"
  filter="[.getLocation[holes]]"
  value="$:/user/data/pcdgen/hole/$(model)$"
/>
<$transclude $tiddler="$:/user/twtest/templates/ftest.template"
  label="`.getLocation` test filter that can't work as intended (to test .wrongVoid)"
  filter="[.getLocation[hole]]"
  value="$:/user/data/pcdgen/hole/$(model)$"
/>
<$transclude $tiddler="$:/user/twtest/templates/ftest.template"
  label="`.getLocation` improper input"
  filter="[.getLocation[hole]]"
  value=""
/>
</$let>

What do you think of it? I know these are a lot of problem that will arise. Especially, mocking might be tricky. Here I am using the real stuff for testing, which not what should be done. My first task was to see how I could test a filter run.That is done. So, how would you try to handle this mocking issue?

1 Like
#2

That’s a very interesting approach.

Some time ago I did create the “trigger-widget” to create automated tests, that can be run from inside TW.

  • “<$trigger” is a widget, that triggers an action macro, whenever it is rendered in the story river.

  • The plugin contains a log-macro, which writes a timestamp and some user defined text into a data-tiddler

  • This data-tiddler is shown right below the story river and is always visible.

  • After performing the actions, it uses the wikify and diff-text widgets to compare the state / content of eg: $:/StoryList and / or the $:/HistoryList with a predefined content of an “expected” JSON tiddler.

  • If there is no difference, everything is fine and “0 errors” is logged

  • If there are differences, “x errors” are logged.

After a test is evaluated it can open the “next” test-tiddler - and so on …

It works as expected, but the main problem was, that it runs the tests so fast, that users cannot follow it. That’s OK for a real test. But for an “example wiki” some user interaction is needed.

That’s why I did create some [previous] and [next] buttons, to run 1 test after the other and give the users some time to read the info.

  • The linked wiki runs a test for every function the “navigator-widget” can handle.

  • Only click [next] or [previous] buttons. If something else is clicked, it will log errors

    • Do not mess with tiddlers in view- or edit-mode other then clicking “next” or “previous”

  • The external window - button click - will always log errors

There is no real documentation other than this post or the edition code itself.

Warning: – Backup first - The trigger-widget has the potential to brick your wiki if used wrong.

have fun!

PS: Trigger — an interactive test framework

3 Likes
#3

Just FYI…

Some years ago, I used Katalon (free edition) to write UI test cases for my wikis.

Katalon provides a high level API over Selenium/WebDriver APIs accessed via Groovy/Java code.

You can use JavaScript, too, but not as a first-class citizen – you can only inject JS and execute it in the page context using Katalon/Selenium APIs. Workable and quite useful, but not ideal.

The whole thing worked fine, to a point, but since the wikis were constantly evolving, the amount of work to maintain the test cases became unmanageable, time wise.

#4

Sorry for this very late reply, but I got ill and ceased all work on tw matter.

I am not looking into GUI test like what you’ve done. Really, above all, I want to be able to check that my functions are doing OK especially as I undertake refactoring them.

@pmario I discovered logger.js thanks to your effort. Thank you, it’s both useful and interesting.

And just today I imported my early test framework into a new wkiki, including my example, but no the tested function. And Then I saw some suceesfull tests when I was looking for an empty result. And so I have to say I would like to things:

  • being able to tell if my.function exists (and, possibly, return the title of the tiddler where it is found (and being executed);
  • having the possibility to stop interpreting a function/filter run when an unknown function/dariable is called (as a function).

The latter would allow {{{ [[4]match[2]then[same]else[other]] }}} to evaluate to other and {{{ [[4]unknown.match[2]then[same]else[other]] }}} to evaluate to an empty list (as of now, it is evaluated to other).

1 Like
#5

@jypre I am glad you are better now!

I am very interested in unit testing and other insight to tiddlywiki operation, However I must admit I don’t follow your or @pmario’s replies above. Although I would like to.

  • Given I intentionally focus on tiddlywiki from a super user perspective, I am curious on what these ideas add to normal defensive code patterns, that test for missing results etc…?
  • Although this may be based on my own ignorance.

But the key reason for replying is to let you know about a related discussion developing a test case widget.

#6

@TW_Tones I spend too much time fixing macros or modifying them after that. I just want a tool to get reliability and spending less time at the same time.

My aim is just a bare OK/cancel resault.

A for the $testcase widget: that’s quite interesting, but that’s not what I want. I shall
look at it in closer detail, though, because surely something has to be done for documenting mechanisms in TW.

As for things like normal defensive code patterns, they are just a way to build things. But not a way to formally check them. I want a unit testing framework to just be sure that I do what I want to do (and document it by the way) and know immediately when something is broken. It has to be manually called (showing the tiddler testing stuff, that is).

#7

I want this to, but the way I code does not seem to risk the errors I think you are discussing.

  • Although If I understood what is your greater challenges, I possibly have ways to avoid or see them occurring, I could share with you and the community.

However as I share the same objective, I find the greater problem is matching braces and other single character syntactic errors.

  • A TiddlyWiki Script Highlighter would save more grief for me.

However I have raised questions about loop detection and exits and other issues that may help with this “Problem with TiddlyWiki” and I applaud your desire to assist with debugging tiddlywiki script.

#8

@TW_Tones Please share your experience of safe programming in a separate thread or in a dedicated online tiddjywiki. I have done some little things to help out example coding for my own documertation BTW.

And I agree that syntax errors are a real pin, and most of them are due to typos. We really need a lint for tiddlywiki! One for wikitext, and one dedicated for filter run and functions (surely that would be the same tool…). And a good syntax highlighter would be a good thing, too (but not a replacement for a lint tool).

#9

Sure, but I was only commenting to assist with your;

This seem similar to recent discussions about loop cancels and time outs for runaway processes, but I may be wrong.