The $testcase widget and CompoundTiddlers

Developers
#1

Along with other updates, the $testcase widget introduces the idea of CompoundTiddlers which affords the functionality in test cases.

I value this increased functionality but feel to be really useful we need to make use of and mature this solution further. In particular compound tiddlers could be used in a number of other cases but could also be provided with a set of tools to make them more usable in the testcases.

The following should be quite easy todo so I present them here for others to consider;

  • Ability to take a result of a filter, or selected tiddler and copy it to the clipboard in this compound tiddler format.
  • Ability to expand a compound tiddler into real tiddlers
  • Ability to selectively insert or remove a compound tiddler entry from a compound tiddler
  • Ability to export a set of tiddlers as a compound tiddler, which can be reimported and held as is or expanded as above.

The ability to store tiddlers in a fashion where they are not actually tiddlers, unless expanded, or used in a test case within a compound tiddler provides the possibility of tools to;

  • Quickly select from a filter or advanced search the tiddlers to store in a compoundTiddler
  • Provide a rapid way to take a problem and put it in a test case and or share that test case.
  • A way of publishing a set of tiddlers on talk.tiddlywiki that are simple to read but can also be placed into a running test case, innerwiki or expanded into a test wiki.
  • Trash tiddler(s) into a compound tiddler, then recover it.
  • Take tiddlers into an offline compound tiddler such as a version backup
  • Use compoundTiddlers against Innerwiki as well.
    • Could have a large repository of tiddlers to produce different wiki editions through the innerwiki
  • Distribute tiddlers as a compound tiddler then allow selective restore or expansion.
  • Allow a compound tiddler to be used as a set of reference tiddlers so they can be restored to original content.

For many users this will be more practical than using JSON and Plugin forms of a collection of tiddlers.

1 Like
Mixed thoughts on how to improve stuff for new users
#2

Agreed on nearly all points.

One thing I would like from these above all else is to allow us to build up documentation examples drawn from domains other than that of “A Homepage for Tiddlywiki”.

In a recent private conversation, I said

I think the main site is trying to do too much, especially using its own content for documentation examples.

and the person I was speaking to asked

Hm, I would have thought that is an advantage? So that the user can se actual examples. What is not good about it?

This was my response:

The trouble is that we can mostly choose examples drawn from one domain, that of a documentation wiki. Imagine trying to document an answer to the question, “How do I list all the tiddlers tagged with a tag that itself has a specified tag?” You could try to find some example in the documentation where there is a reasonable-sized output for this, or you could just use the TestCaseWidget with these tiddlers:

title: Apple
tags: Red

title: Banana
tags: Yellow

title: Cherry
tags: Red

title: Horse
tags: Animal

title: Red
tags: Color

title: Watermelon
tags: Red

title: Yellow
tags: Color

and demonstrate that [tag[Color]tagging[]sort[]] will select Apple, Banana, Cherry, and Watermelon but not Horse, Red, or Yellow. Understanding this does not require you to figure out what say, Filter Filter Run Prefix (Examples) represents.

I would love to have 5 - 10 small wikis with different types of content and different structures that we could use over and over to explain the various techniques.

The existing system is extremely clever. But clever is rarely an advantage in documentation.

1 Like
#3

Totally agree

Again totally agree, we tend to shoehorn the documentation into service as test data and it it is not perfect and although complex, fairly limited in structure and functionality.

I could also see the primary documentation providing links into an “examples test wiki” which addresses the macro/widget/plugin using one or more data sources in that “examples test wiki”.

  • Within that we could include a range of data sources including but not limited to Compound tiddlers and test cases.
  • This wiki could also include the ability to submit new tiddlers and edits to existing tiddlers in a similar manner to tiddlywiki.com with a larger group permitted to accept changes through GitHub

Some of the areas this community would benefit from, that this approach would assist;

  • Give people data to illustrate their problem using test cases rather than their own data
  • Have an off tiddlywiki repository of;
    • Test data
    • Examples
    • Test cases
  • A single site on which to test and share code not made complicated by the inclusion of documentation tiddlers
    • I could explain why the current documentation macros are a barrier to improving the documentation
  • A repository of data, against which users can experiment in their learning journey.

I agree we may make use of more than one for structuraly different data sets or examples however think we should start and try and focus on a single wiki containing multiple data/test cases.

  • I won’t spell it out here but this viewpoint is informed by a broad set of ideas I have about data tiddlers
  • This is ultimately a continuation of the desire to provide the test data which resulted in the periodical table data I released earlier. I had hoped to inspire others to publish similar, somewhat static, example data sets, against which examples and now test cases could be written, both for education and debugging.

As this Topic proposes such tools could be used, on a “examples test wiki”, to assist users develop examples and test cases that they could easily share, that others can install on the “examples test wiki”.

  • It may be worth including local storage on this wiki for a little more persistence.
#4

That’s definitely taking it further than I’m envisioning. I think that $testcase and Compound Tiddlers – with only one small addition – could give us everything we need to improve the documentation right within tiddlywiki.com. That one addition is the ability to import the tiddlers from one Compound Tiddler into various distinct $testcases.

We can imagine a Compound Tiddler full of tiddlers having to do with, say Nutrition, including a bunch of them for foods:

title: Apple
tags: Fruit
images: images/RedApple images/AppleTree
serving-size: 1 medium unpeeled Apple (~180 grams)
calories: 94
water: 156    <!-- Note measurements except calories are in grams -->
protein: 0.4
carbs: 25
sugar: 19
fiber: 4.4
fat: 0.3
vitamins: C
minerals: Potassium

Apples are among the world’s most popular fruits.

They grow on the apple tree (Malus domestica), originally from Central Asia.

Apples are high in fiber, vitamin C, and various antioxidants. They are also very filling, 
considering their low calorie count. Studies show that eating apples can have multiple 
benefits for your health.

The various bits of documentation could include this set of tiddlers to show things like constructing conditional filters (“Find all foods with fiber at least 2 grams no more than 1.5 grams of fat”), building a ViewTemplate (add a footer below the text with nutritional information), or constructing a TOC (it might include Food > Fruit > Apple for instance).

I think we’re saying the same thing. I’m definitely thinking of this for a documentation wiki—one that I would hope could eventually fold into tiddlywiki.com if it proves itself out—and I would want that to be a single wiki, but I would want that wiki to contain separate Compound Tiddlers holding different mini-wikis. Does that match your thinking?

#5

This will be possible with the next TW version. If you have a look a the preview wiki the TestCaseTiddlers has a new field import-compound which allows testcase tiddlers to import data from other testcase tiddlers.

So 1 testcase can handle 1 example using the same data over and over, without polluting TW-com with test-data.

2 Likes
#6

Oh, thank you for the info. That’s perfect. Going to bed now, but will look this over tomorrow.

#7
  • Perhaps but I am not sure we need call them mini-wikis but keep them as “testcases” some of which may have enough for a mini-wiki (If by that you mean an independent wiki), otherwise just aa demo/test cases;
  • such as
    • listing nested data via fields rather than tags (This data does not currently exist in tiddlywiki.com)
    • Targeting a range of tiddlers with a filter to alter their CSS. A Better example.
#8

That’s already possible.

image
image2069Ă—931 112 KB

#9

Hi @TW_Tones @Scott_Sauyet the crucial thing to understand about compound tiddlers is that the reason for their existence is to give users an easy and minimal way to manually type the definitions of a number of tiddlers in a single text box.

The alternative would have been the existing plugin format, which already allows multiple tiddlers to be packed together into one text field. Right now we have good support for read only access to the subtiddlers within a plugin, but we don’t (yet) have support for dynamically writing a subtiddler into a plugin.

But the trouble with the plugin format is that it is based on JSON, and so is unwieldy for manual authorship.

Hence the introduction of the compound tiddler format. The goal of being easy to type led to a very simple syntax: it is basically multiple tiddlers in the existing .tid file format joined together with lines consisting of a single +. For example:

title: First Tiddler
tags: Apple Banana

This is the text of the first tiddler
+
title: Second Tiddler
tags: Banana Kiwi

This is the text of the second tiddler

A cursory analysis of the format should show that it is not a universal format: there are tiddlers that simple cannot be expressed in this format. For example, any tiddler with a single + on a line by itself will be interpreted as a break between tiddlers.

That problem rules out most of the ideas in the OP. But I don’t think that’s a problem; the same ideas can be achieved using the existing plugin format.

This isn’t something we can fix except by adopting a different format but in any case this behaviour is by design.

All over TiddlyWiki we find the same tension between the competing attractions of machine friendly formats like JSON and human formats like .tid for representing the same thing. The core tries to provide good tooling for both so that users can build hybrid solutions.

#10

Does it rule out my thought on sharing these compound tiddlers across multiple testcases/examples? Mario seemed to imply otherwise, but I won’t be able to chase that down while still on my mobile device.

#11

No, the sharing of compound tiddlers has always been supported when using the <$testcase> widget directly and from v5.3.6 will also be supported by compound tiddlers. See Data widget add compound filter parameter by pmario · Pull Request #8447 · TiddlyWiki/TiddlyWiki5 · GitHub

To be clear, compound tiddlers work in situations where the content is manually authored by somebody with an awareness of the limitations. The situation where they will not work correctly is when one is trying to store arbitrary tiddlers within them: they will fail with tiddlers that have a single line containing + or with have field values containing line breaks.

1 Like
#12

I think this desire is very valid. We did already discuss the possibility to copy / pasted tiddlers from this forum into TW and vice versa in .tid format.

The compound format is not much different. It has a type: text/vnd.tiddlywiki-multiple and that’s basically it.

So on the TW side we would need an extension to the import-from-clipboard mechanism that analyses the incoming text and if it looks like a standard .tid file format we can import it as a valid tiddler. The problem here is, that without a dedicated info that it is a .tid file, we can have “false positives” – and the import will be messed up.

Seeing type: text/vnd.tiddlywiki-multiple in the incoming text will make this type analysis deterministic and much less error-prone.

But

There are still 2 usecases.

  1. Importing the TestCaseTiddler as a TestCaseTiddler
  2. Importing testcase tiddler and split them into several tiddlers

IMO there is no algorithm I can think of at the moment, that can interpret the “intent” of the user.

So we will need a 2 step import. 1) import as TestCaseTiddler → 2) manually “split” that tiddler into several tiddlers.

#13

Hi @pmario the fundamental problem is that not all tiddlers can be expressed within a compound tiddler: problems arise with those that have fields containing newlines, and those containing a single “+” on a line by itself.

The compound tiddler format is designed to be written by humans, and is not suitable to be written by code unless it is known in advance that the constraints of the format will not be a problem.

1 Like
#14

I understand these limitations but still think this additional functionality would be useful in building and testing test cases to be shared. Many of which are going to be somewhat simple. Example include simple test data and the widgets to list them.

  • Once a set of tiddlers are encoded in a compound tiddler or available to do this with the author can quickly build a test case and share it.
  • With minor changes a new test case be built from another to illustrate something different.

We can first build this as a plugin and one it matures move it to the core so all wikis can accept and build test cases.