Custom assert widget for filters does not work as expected

#1

Similar to the $assert.equal widget from here My "unit test" fails , I am trying to write an assert widget for filters.

Here’s my code so far:

<!-- True/False assertion for filters: if filter evaluation is a non-empty list, then True, else False -->
\widget $assert.filter(filter)
<%if [<filter>count[]compare:number:gt[0]] %>
OK
<%else%>
FAIL
<%endif%>
\end

[[TiddlerWithEmptyLines]] is used as fixture containing test data

<$vars tiddler="TiddlerWithEmptyLines" rexpr="[\s\t]*">
    <$list filter="[<tiddler>get[text]splitregexp[\n]]" counter="i">
		    <$let filter="[<currentTiddler>regexp<rexpr>]">
		        <<i>>: <<currentTiddler>> : <$assert.filter filter=<<filter>> />
				    <br/>
				</$let>
    </$list>
</$vars>

Something doesn’t work, because all tests (for each line of the tiddler containing test data) pass.

I am attaching this code + the fixture tiddler containing text data.

Note: the tiddler containing test data is not copy/paste friendly, since it contains empty lines. It is not render-friendly either. Please inspect its content in Edit Mode: empty lines are either just “\n”, or contain whitespaces, or tabs, or both.

tiddlers.json (973 Bytes)

#2

Your filter widget treats the filter parameter as normal variable, it is never evaluated as a filter, and therefore your tests always pass.

Have a look at the subfilter operator. and you don’t need the count and compare since an empty result from evaluating the provided filter will always trigger the else clause.

#3

I am still stuck. To be able to use subfilter operator, I have to craft the filter string into a variable, so I can pass it as parameter to my custom widget. But to be able to use regexp operator while having square brackets in the regular expression, the docs tell me to use a variable, which is the opposite to hardcoding the filter string :man_facepalming:

Current version of code:

<!-- True/False assertion for filters: if filter evaluation is a non-empty list, then True, else False -->
\widget $assert.filter(filter)
<%if [subfilter<filter>] %>
OK
<%else%>
FAIL
<%endif%>
\end

[[TiddlerWithEmptyLines]] is used as fixture containing test data

<$vars tiddler="TiddlerWithEmptyLines">
    <$set name="rexpr" value="[\s\t]*">
        <$list filter="[<tiddler>get[text]splitregexp[\n]]" counter="i">
		        <$let filter="[[<<currentTiddler>>]regexp[<<rexpr>>]">
				        <<filter>> : <<i>> : <<currentTiddler>> : <$assert.filter filter=<<filter>> />
    				    <br/>
            </$let>
        </$list>
    </$set>
</$vars>
#4

I don’t follow. Every parameter to a custom widget is already available as a variable within the widget. You can pass it to the widget as a literal string, a variable, or a text reference. All of them will work. https://tiddlywiki.com/#Widget%20Attributes

I highly recommend that you spend some time on the filter syntax documentation to internalize the basic way of using variable parameters within a filter.
See also: https://tiddlywiki.com/#Filter%20Parameter

The newer version of your code mixes the correct way to reference a variable as a parameter to a filter step (with single angle brackets operator<varname>) with using double angle brackets within square brackets (operator[<<varname>>]).

It would also be easier to provide assistance if you explained the structure of your test data, ideally without needing to download files. As I currently see it, you aren’t filtering out blank lines or white space only lines which will give false positives. Is each whitespace separated section of text meant to be a filter? That wont work for realistic real world scenarios as filter expression can contain whitespace. When storing multiple filters in a single tiddler I usually split on line breaks and filter out lines that are empty or whitespace only, as well as lines prefixed with characters that I want to use to escape comments.

With the above caveats in mind about not understanding the structure of your test data, this might be closer to what you have in mind:

<!-- True/False assertion for filters: if filter evaluation is a non-empty list, then True, else False -->
\widget $assert.filter(filter)
<%if [subfilter<filter>] %>
OK
<%else%>
FAIL
<%endif%>
\end

[[TiddlerWithEmptyLines]] is used as fixture containing test data

<$let tiddler="TiddlerWithEmptyLines" rexpr="[\s\t]*">
    <$list filter="[<tiddler>get[text]splitregexp[\n]]" counter="i">
		    <$let filter="[<currentTiddler>regexp<rexpr>trim[]!is[blank]]">
		        <<i>>: <<currentTiddler>> : <$assert.filter filter=<<filter>> />
				    <br/>
				</$let>
    </$list>
</$let>
#5

Apparently I need more iterations of this. And more practical coding, which I started to do only recently after months of learning the basics of wikitext mostly in read-only mode. I’m just not confident about it, which is curious by itself. I don’t get that “I don’t understand this at all” feeling, which would just make me abandon the quest with no regrets. But as soon as I try to write a practically useful snippet of wikitext code, I get stuck.

The newer version of your code mixes the correct way to reference a variable as a parameter to a filter step (with single angle brackets operator<varname> ) with using double angle brackets within square brackets (operator[<<varname>>] ).

I tried to use double brackets because that’s how variables are referenced when used to build other variables in context of a <$let> widget. Since the filter (the one that is going to be passed to the custom assert widget) is stored in a variable, I tried to get the regexp (which contains the trouble causing square brackets itself) into the filter this way. Looks like it was a bad idea.

I hoped this

would be good enough. My test tiddler contains:

  1. lines of text
  2. empty lines (as in at the end of a non-empty line I hit Enter, cursor goes to the beginning of new line, I hit Enter again, I get an empty line above the cursor). this I referenced as “\n” above.
  3. lines that contain only a mix of zero or more whitespace characters (produced by pressing the Spacebar) and tab characters (produced by hitting the Tab key). hence the [\s\t]* regexp. visually, they look “empty” as well, but their size (in characters) is greater than zero. yet I treat both 2. and 3. as opposite to 1.

I believe this is the trim[]!is[blank]] part of the filter. I never thought of doing that. I’ve never used trim filter operator before, yet I still don’t understand why is this needed. Well, you explain why:

which will give false positives

but I still don’t understand why I’m going to get false positives.

Is each whitespace separated section of text meant to be a filter?

Not at all. My initial goal (for which I would’ve happily used a core filter assertion widget, if there was one, but there isn’t, hence I’m trying to write one to help me with writing unit tests, since the assert.equal widget from another thread isn’t good at answering the “does this filter return at least one item? tell me by comparing two hardcoded values” question) was to write a filter that would let me distinguish between lines like 1. and lines like 2. or 3. (see above). The ultimate practical problem I want to solve is that I have tiddlers with content written in a hurry that have multiple consecutive “empty” (as in either 2. or 3.) lines of text, between lines of actual text (1.) and I want to find them with a script, to make text in those tiddlers look “nicer” by replacing multiple “empty” lines with just one, and sometimes entirely remove a single “empty” (as in 2.) one.

Your version of code produces the expected tests results with my test data. The biggest confusion about it, as I already said above, is why removing the trim[]!is[blank]] part from the filter breaks everything and makes all tests pass (which is wrong).

#6

In TiddlyWiki filters, a result that returns an empty string or a whitespace only string is not the same as no result. Therefore, if you want to exclude lines that are only whitespace, you need to do so before passing in the variable to the widget, or change the logic of the widget to handle it there. Try this in $:/AdvancedSearch, in the filter tab, and note the number of matches returned: [[]]

2 Likes