What tests can we do on a filter syntax and regex challenge?

#1

Folks,

I was thinking it could be quite simple to provide a way to enter a filter test various aspects of its syntax. A “filter checker”. We could past a problem filter, or an experiment when writing one, in a field and get feedback or list its results (with a +[limit[]]) so it does not run to all tiddlers or infinity.

Simple feedback request:
I am starting this discussion to seek the communities feedback on different valid tests we can do on a filter. Please provide any you think of and I will incorporate them below, and also point out any exceptions if any you find any. Don’t worry if you propose something already listed.

Regex challenge;
For those of you who know regex, filters could be interrogated for their syntax and a far more advanced way. If you do share some “regular expressions” these please spell out what the regex determines in plain English.

Syntax Checks:

  1. Always match a “[” with a “]”
  2. Always match a “{” with a “}”
  3. Always match a “<” with a “>”
  4. Reject “{{” or “}}” “{{{” or “}}}”
    4.1 Exception unless it is the outer
  5. If “!” follows “{!” ensure it is “{!!”
  6. Display the result of the filter if broken
  7. Propose a closing “]” at the end of the filter if “[” mismatch occurs (common error of mine)
  8. If a filter run prefix such as “:intersection” is used check it is one of the documented names, and is a prefix “only”
    8.1 except if inside a <:> or “[:]”
  9. If a filter run prefix such as “+ - ~” is used check it is one of the documented names, and is a prefix “only”. Provide a translation to a full name is available
  10. Check for the cases where a “[” or “]” wraps a “<”, “>”,"{, “}” because these are always/often? invalid
  11. If we see a “!!” it proceeds a fieldname

;Syntax checker features

  1. Provide the ability to set an “outer” variable, or filter
  2. Provide for the testing of nested filters
    Provide for the testing of filtered transclusions, even show both a list and filtered transclusion for a given filter.
  3. Automatically set +[limit[]] toggle off
  4. Simultaneously retrieve a count of the filter result +[count[]] and display
  5. If above a threshold number hide output until you choose to review.
  6. Allow you to list a set of tag pills to reorder ones used in your filter
  7. Provide a list of “words” found in your filter which are most often tags, fieldnames, titles or variables. Test each to see what it may be (one or more)
  8. Provide a set of variables for values that cant be used directly inside a filter because they will break filters.
  9. Provide a set of variables with special characters that can not be entered from the keyboard.
    9.1 Also allow a charcode or html entity to be the value of a variable

If someone wants to build or collaborate on the “filter checker” please let me know here or private message, I would prefer a wikitext/widget solution than javascript unless there was a good reason.

#2

Folks, If you contribute to this topic or the “filter checker” you will no doubt be honing your “filter skills”, so why not make your learning journey or experience also contribute to the community? I am!

#3

No takers on this Challenge?

#4

Hi @TW_Tones we had a conversation with @saqimtiaz last year that led to some experiments with producing a visualisation of the structure of a filter:

I don’t think it’s viable to try to simulate the filter parser in wikitext; any small errors will render the whole tool useless. Better would be to add a new primitive that decomposes a filter into a bundle of interlinked tiddlers that can be readily rendered in wikitext.

1 Like
#5

@jeremyruston as I said

Folks, If you contribute to this topic or the “filter checker” you will no doubt be honing your “filter skills”, so why not make your learning journey or experience also contribute to the community? I am!

I am not suggesting this at all, although the filter visualizer/helper tool is an interesting direction. Including the filter builder idea.

I am just asking what tests we can undertake on a filter because with a robust set of these we can guide someone when their filter is not behaving and assist in learning. I am looking at “high level rules only” and more for beginner’s or tired experts.

As you can see from my original list it is more from the the perspective of common pitfalls see item 4. One being addressed in the syntax checker features, to set a limit of results while experimenting with filters, because incomplete ones have the habit of generating long lists and effectively stopping interaction.

My post was an invitation to demonstrate what people know, valid tests they can see, and build some guidance. To actually make a syntax checker is just a possible outcome.