A first go at unit testing

,
#1

I spend a lot of time fixing my typos and other mistakes. I have to use tests to ease it up.

So I remembered my java coding time and I came with the idea of twtest (same idea as jtest).

Here is what I achieved in two days:

twtest-sample
twtest-sample1299Ă—737 56.3 KB

This is only a tool for testing filter runs, especially for testing functions. So far, it is meant for functions returning a single elements. (*void^*) is my convention to tell there is no tiddler in the result.

This is made by a small CSS sheet and a single template. Here is that one.

\parameters(label:"no label" filter:"[[no filter]]", value:"no value")

\function .wrong(label, awaited)
  [dump[input for .wrong]]
  +[first[]]
  :map[[<dl class="wrong"><dt>$(label)$</dt><dd>ERROR</dd><dd>GOT: <samp class="got"><$text text="""$(currentTiddler)$"""/></samp></dd><dd>AWAITED: <samp class="awaited"><$text text="""$(awaited)$"""/></samp></dd></dl>]substitute[]]
\end .wrong

\function .wrongVoid(label, awaited)
  [[<dl class="wrong"><dt>$(label)$</dt><dd>ERROR</dd><dd>GOT: <samp class="got">(* void! *)</samp></dd><dd>AWAITED: <samp class="awaited"><$text text="""$(awaited)$"""/></samp></dd></dl>]substitute[]dump[.wrongVoid result]]
\end .wrongVoid

\function .correct(label)
	+[first[]]
  [[<dl class="correct"><dt>$(label)$</dt><dd>SUCCESS</dd></dl>]substitute[]]
\end .correct

\function .verdict(label, value)
  [all[]match<value>]
  :then[.correct<label>]
  :else[.wrong<label>,<value>]
\end .verdict

\function .verdictIfVoid(label, value)
  [<value>first[]]
  :then[.wrongVoid<label>,<value>]
  :else[.correct<label>]
\end .verdictIfVoid

\function .test(label, filter, value)
  [subfilter<filter>]
  :map[.verdict<label>,<value>]
   else[.verdictIfVoid<label>,<value>]
\end .test

<$let res = {{{ [.test<label>,<filter>,<value>] }}}
  real = {{{ [subfilter<filter>dump[real value]] }}}
>
<!--
* filter: <code><<filter>></code>
* awaited value: <samp><$text text=<<value>>/></samp>
* real value: <samp><$text text=<<real>>/></samp>
* verdict text: <samp><$text text=<<res>>/></samp>
-->
<<res>>
</$let>

And an example of use, corresponding to the image shown.

<$let project=bdmsda model=phpdat>
<$transclude $tiddler="$:/user/twtest/templates/ftest.template"
  label="`.getLocation` standard success case"
  filter="[.getLocation[holes]]"
  value="$:/user/data/pcdgen/holes/$(model)$"
/>
<$transclude $tiddler="$:/user/twtest/templates/ftest.template"
  label="`.getLocation` with unknown reference"
  filter="[.getLocation[dummy]]"
  value=""
/>
<$transclude $tiddler="$:/user/twtest/templates/ftest.template"
  label="`.getLocation` test filter that is broken (to test .wrong)"
  filter="[.getLocation[holes]]"
  value="$:/user/data/pcdgen/hole/$(model)$"
/>
<$transclude $tiddler="$:/user/twtest/templates/ftest.template"
  label="`.getLocation` test filter that can't work as intended (to test .wrongVoid)"
  filter="[.getLocation[hole]]"
  value="$:/user/data/pcdgen/hole/$(model)$"
/>
<$transclude $tiddler="$:/user/twtest/templates/ftest.template"
  label="`.getLocation` improper input"
  filter="[.getLocation[hole]]"
  value=""
/>
</$let>

What do you think of it? I know these are a lot of problem that will arise. Especially, mocking might be tricky. Here I am using the real stuff for testing, which not what should be done. My first task was to see how I could test a filter run.That is done. So, how would you try to handle this mocking issue?

1 Like
#2

That’s a very interesting approach.

Some time ago I did create the “trigger-widget” to create automated tests, that can be run from inside TW.

  • “<$trigger” is a widget, that triggers an action macro, whenever it is rendered in the story river.

  • The plugin contains a log-macro, which writes a timestamp and some user defined text into a data-tiddler

  • This data-tiddler is shown right below the story river and is always visible.

  • After performing the actions, it uses the wikify and diff-text widgets to compare the state / content of eg: $:/StoryList and / or the $:/HistoryList with a predefined content of an “expected” JSON tiddler.

  • If there is no difference, everything is fine and “0 errors” is logged

  • If there are differences, “x errors” are logged.

After a test is evaluated it can open the “next” test-tiddler - and so on …

It works as expected, but the main problem was, that it runs the tests so fast, that users cannot follow it. That’s OK for a real test. But for an “example wiki” some user interaction is needed.

That’s why I did create some [previous] and [next] buttons, to run 1 test after the other and give the users some time to read the info.

  • The linked wiki runs a test for every function the “navigator-widget” can handle.

  • Only click [next] or [previous] buttons. If something else is clicked, it will log errors

    • Do not mess with tiddlers in view- or edit-mode other then clicking “next” or “previous”

  • The external window - button click - will always log errors

There is no real documentation other than this post or the edition code itself.

Warning: – Backup first - The trigger-widget has the potential to brick your wiki if used wrong.

have fun!

PS: Trigger — an interactive test framework

3 Likes
#3

Just FYI…

Some years ago, I used Katalon (free edition) to write UI test cases for my wikis.

Katalon provides a high level API over Selenium/WebDriver APIs accessed via Groovy/Java code.

You can use JavaScript, too, but not as a first-class citizen – you can only inject JS and execute it in the page context using Katalon/Selenium APIs. Workable and quite useful, but not ideal.

The whole thing worked fine, to a point, but since the wikis were constantly evolving, the amount of work to maintain the test cases became unmanageable, time wise.