TiddlyWiki Filter Language

TiddlyWiki Filter language

A collaborative text on Filter Language


[Updated] Some “prose” that may help new and developing users.

TiddlyWiki filters are a key feature in tiddlywiki that grant substantial power to users and designers. Becomming proficent in writting filters is very rewarding. To help come to terms with how they work consider the following;


filters

  • Filters are a long established metaphore which refers to selectivly allowing some members of a larger list or set through, to form a new list, according to a set of conditions “The filter”. It can also be thought of as removing some members of a larger list or set.
    • To demonstrate this, there is an implied all tiddlers at the begining of filters which includes all tiddlers, then the filter you write applies a selective process to this list. All referes to regular tiddlers (that are not system tiddlers, or not shadow tiddlers)

Lists and titles

  • Filters are primarily used to filter lists of tiddler titles, but can be used to filter any list or the simpler cases of zero, one or more.
    • So a basic filter is to filter out from the list of all tiddlers with a specific tag [tag[tagname]] here we assume the input is a list of all tiddlers and the filter says “if it has the tag tagname”. The result will be all tiddlers with the tag tagname (which can be zero or more).
    • A common list of one item is the current tiddler, [all[current]] which replaces the implied [all[tiddlers]] such that, if the currentTiddler value is available, all that is returned is the currentTiddlers title. So if we combine this with the last filter [all[current]tag[tagname]] the currentTiddlers title will only “pass through the filter” if the current tiddler is tagged tagname.
  • Since lists are commonly tiddler titles, it is important top be aware of the default of No repeated titles, please see Dominant Append.

No repeated titles

For the convenience of users and designers of filters, the result of a filter is automatically “deduplicated”, meaning repeated titles are remove keeping the list a set of unique titles.

True and False

Filters also can be considered to return true or false,

  • if nothing is returned, the filters the result can be considered false,
  • if anything at all is returned, the filters result can be considered true
    • Although it may be true more than once (returning multiple titles), there are methods such as adding +[limit[1]] at the end to ensure its only true once, that is we limit the results, if any, to 1.
  • Often when we are testing for a condition such as true OR false we do not actually need the resulting value, and we certainly don’t want the currentTiddler variable changed. So we tend to set the result to an arbitrary variable name.
<$list filter="[tag[tagname]] +[limit[1]]" variable=~>
Content used, if there is even one tiddler in the wiki with the tag "tagname". 
</$list>
  • In the above example, since we do not care which tiddler is the first with this tag, we set the variable name to “~” <<~>>
    • so it does not set the variable “currentTiddler” <<currentTiddler>>.

Titles and strings

A very simple filter is one or more titles such as one [[tiddler two]] tiddler-c.

The [[ ]] is used because there is a space in the tiddlers name.

  • This results in a list of three titles one,tiddler two and tiddler-c.
  • This way of naming titles in a filter is in effect a shorthand for [title[one]] [title[tiddler two]] [title[tiddler-c]]
  • If you lookup the title Operator you will see this form [title[one]] is a “constructor” this means the implied [all[]] is no longer applied and the result is only a list of three titles.

Note: The above very simple filter shows how a filter can be created with one or more arbitary titles wether or not an actual tiddler exists for each title.

  • This demonstrates how we can introduce ‘any values’ to a filter to which we can then apply various filters, thus allowing maths or formating to be applied to any input.

Beyond literals and strings

Using <variables>,{!!fields} and <<macros>> in filters

When using filters we quickly discover we want to write filters that respond to the values in variables, fields and other tiddlers, even macros. To support the writting of such tiddlers we can use the following in filters;

  • <variablename>, <macroname macroparam>, {!!fieldname}, {tiddlername}, {tiddlername!!fieldname}.
  • For example [<variablename>match{!!myfield}] note the outer [ ] are used to make this a Filter Run.
    • since each of these are already “delimited” by their own "braces such as <name> {name} it is unnessasary to use [ or ] such as [<name>] or [{name}] in filters, unless you want this to be used as a Filter Run
  • If using variables, fields and other tiddlers in filters, there is a posibility they will return more than one title, string or value. As a result you may need to use one of the following operators to help deal with the vaues therein list[], enlist[], enlist-input[], subfilter[]

+ Other ways to use or set a filter

We typicaly think of filters as a parameter to a widget such as the $list widget filter="yourfilter" however there are otherways to use a filter.

  • If the whole filter is available in a fieldname, you can set the filter directly filter={{!!filterfield}}.
    • Or similarly a variable filter=<<variable>>.
  • But perhaps the most valuable is the Filtered Transclusion where we use the “tripple curly braces” {{{ yourfilter }}} where we have no need for any widget.
    • For example {{{ [tag[tagname]] }}} will produce a list of tiddlers with the tagname, directly where it is written.
    • Or can also be used as an attribute value name={{{ [tag[tagname]first[]] }}}
    • Most filters are valid as parameters and filtered transclusions, and thus filtered transclusions provide a simple way to document alternative filters.

Strings and combinations thereof

Because filters can manipulate abitary strings of characters, in addition to tiddler titles, we can use filters to format, combine (concatinate) any set of characters/strings.

Common examples include;

  • {{{ [[string or title]addprefix[This is a ]] }}}
  • {{{ [[string or title]addsuffix[ concatinated]] }}}
  • {{{ [<variable>] [{!!fieldname}] [<macroname param>]] }}} produces a list and will Not concatinate the values.
    • However we can join them as follows;
    • {{{ [<variable>] [{!!fieldname}] [<macroname param>]] +[join[ ]] }}} Note we are joining them with a single space.
  • We can also remove a prefix or a suffix and various other string operations see String Operators in Filter Operators

Note: It is importiant to remember what was covered in the section No repeated titles when manipulating strings. There are plenty of workarounds.


[Edited] I continue to develop this off line but invite people to nominate a subject area about filters, you may want such documentation to address.
2 Likes
[@mario here] -- Copied this text into the wiki above, so it can be edited there

[Updated] Some “prose” that may help new and developing users.

TiddlyWiki filters are a key feature in tiddlywiki that grant substantial power to users and designers. Becomming proficent in writting filters is very rewarding. To help come to terms with how they work concider the following;


filters

  • Filters are a long established metaphore which refers to selectivly allowing some members of a larger list or set through, to form a new list, according to a set of conditions “The filter”. It can also be thought of as removing some members of a larger list or set.
    • To demonstrate this, there is an implied all tiddlers at the begining of filters which includes all tiddlers, then the filter you write applies a selective process to this list. All referes to regular tiddlers (that are not system tiddlers, or not shadow tiddlers)

Lists and titles

  • Filters are primarily used to filter lists of tiddler titles, but can be used to filter any list or the simpler cases of zero, one or more.
    • So a basic filter is to filter out from the list of all tiddlers with a specific tag [tag[tagname]] here we assume the input is a list of all tiddlers and the filter says “if it has the tag tagname”. The result will be all tiddlers with the tag tagname (which can be zero or more).
    • A common list of one item is the current tiddler, [all[current]] which replaces the implied [all[tiddlers]] such that, if the currentTiddler value is available, all that is returned is the currentTiddlers title. So if we combine this with the last filter [all[current]tag[tagname]] the currentTiddlers title will only “pass through the filter” if the current tiddler is tagged tagname.
  • SInce lists are commonly tiddler titles it is important top be aware of the default of No repeated titles, please see dedicate section.

No repeated titles

For the convenience of users and designers of filters, the result of a filter is automaticaly “deduplicated”, meaning repeated titles are remove keeping the list a set of unique titles.

True and False

Filters also can be concidered to return true or false,

  • if nothing is returned, the filters the result can be concidered false,
  • if anything at all is returned, the filters result can be concidered true
    • Although it may be true more than once (returning multiple titles), there are methods such as adding +[limit[1]] at the end to ensure its only true once, that is we limit the results, if any, to 1.
  • Often when we are testing for a condition such as true OR false we do not actualy need the resulting value, and we certianly don’t want the currentTiddler variable changed. So we tend to set the result to an arbitary variable name.
<$list filter="[tag[tagname]] +[limit[1]]" variable=~>
Content used, if there is even one tiddler in the wiki with the tag "tagname". 
</$list>
  • In the abvove example, since we do not care which tiddler is the first with this tag, we set the variable name to “~” <<~>>
    • so it doesn not set the variable “currentTiddler” <<currentTiddler>>.

Titles and strings

A very simple filter is one or more titles such as one [[tiddler two]] tiddler-c.

The [[ ]] is used because there is a space in the tiddlers name.

  • This results in a list of three titles one,tiddler two and tiddler-c.
  • This way of naming titles in a filter is in effect a shorthand for [title[one]] [title[tiddler two]] [title[tiddler-c]]
  • If you lookup the title Operator you will see this form [title[one]] is a “constructor” this means the implied [all[]] is no longer applied and the result is only a list of three titles.

Note: The above very simple filter shows how a filter can be created with one or more arbitary titles wether or not an actual tiddler exists for each title.

  • This demonstrates how we can introduce ‘any values’ to a filter to which we can then apply various filters, thus allowing maths or formating to be applied to any input.

Beyond literals and strings

Using <variables>,{!!fields} and <<macros>> in filters

When using filters we quickly discover we want to write filters that respond to the values in variables, fields and other tiddlers, even macros. To support the writting of such tiddlers we can use the following in filters;

  • <variablename>, <macroname macroparam>, {!!fieldname}, {tiddlername}, {tiddlername!!fieldname}.
  • For example [<variablename>match{!!myfield}] note the outer [ ] are used to make this a Filter Run.
    • since each of these are already “delimited” by their own "braces such as <name> {name} it is unnessasary to use [ or ] such as [<name>] or [{name}] in filters, unless you want this to be used as a Filter Run
  • If using variables, fields and other tiddlers in filters, there is a posibility they will return more than one title, string or value. As a result you may need to use one of the following operators to help deal with the vaues therein list[], enlist[], enlist-input[], subfilter[]

+ Other ways to use or set a filter

We typicaly think of filters as a parameter to a widget such as the $list widget filter="yourfilter" however there are otherways to use a filter.

  • If the whole filter is available in a fieldname, you can set the filter directly filter={{!!filterfield}}.
    • Or similarly a variable filter=<<variable>>.
  • But perhaps the most valuable is the Filtered Transclusion where we use the “tripple curly braces” {{{ yourfilter }}} where we have no need for any widget.
    • For example {{{ [tag[tagname]] }}} will produce a list of tiddlers with the tagname, directly where it is written.
    • Or can also be used as an attribute value name={{{ [tag[tagname]first[]] }}}
    • Most filters are valid as parameters and filtered transclusions, and thus filtered transclusions provide a simple way to document alternative filters.

Strings and combinations thereof

Because filters can manipulate abitary strings of characters, in addition to tiddler titles, we can use filters to format, combine (concatinate) any set of characters/strings.

Common examples include;

  • {{{ [[string or title]addprefix[This is a ]] }}}
  • {{{ [[string or title]addsuffix[ concatinated]] }}}
  • {{{ [<variable>] [{!!fieldname}] [<macroname param>]] }}} produces a list and will Not concatinate the values.
    • However we can join them as follows;
    • {{{ [<variable>] [{!!fieldname}] [<macroname param>]] +[join[ ]] }}} Note we are joining them with a single space.
  • We can also remove a prefix or a suffix and various other string operations see String Operators in Filter Operators

Note: It is importiant to remember what was covered in the section No repeated titles when manipulating strings. There are plenty of workarounds.


[Edited] I continue to develop this off line but invite people to nominate a subject area about filters, you may want such documentation to address.
5 Likes

Excellent writeup!
I hope other join us and help to this wiki thread, later we can publish this as a separate TW on tiddlyhost (e.g. filter-language.tiddlyhost.com)

I am thinking to add some other parts as below

  • Syntax
  • Control characters (symbols, e.g brackets: [, {, <,...)
  • Examples (like cases to illustrate how an operator works, these are simple)
  • Case studies (real life examples)
1 Like

In relation to case studies it seems to me there is possibly a dozen common filters, up to 50 regular filters and not more than 50 more to cover most cases.

In particular we need to expand examples to filters that use real world variables, fields and macros. Too many on tiddlywiki were documented in a rush to explain an operator and are somewhat artificial.

I see value in developing a package of standard real world data both in plugins and json files, against which such examples can act on. The result is people become familular with the data which helps with understanding the examples. If the data can be understood through general knowledge all the better.

  • if well designed, examples can help people build solutions such as some dummy contacts tiddlers.

Data Set ideas;

  • Contact tiddlers (Forms)
  • Recipe tiddlers (multi-tiddlers, ingredients/shoppinglist, alternate search)
  • HTML Colour names (select and set)
  • Invoicing
1 Like

The official documentation have a good point (for me), it is concise. But it doesn’t give any hint to guide you, the following tiddler to read is not clear.

For description we can do a modification of https://tiddlywiki.com/#Filters , somethin like (with better English)

You can think of TiddlyWiki as a database in which the records are tiddlers. A database typically provides a way of discovering which records match a given pattern, and in TiddlyWiki this is done with filters.

A filter is a concise notation used for return a particular result, known as its output, from the content of the TiddlyWiki (where it is used). This output can be a selection of data stored in your TiddlyWiki. You can also obtain a (list of) new value(s) generated from the original data.

The filter can be used inside of the different mechanisms of TiddlyWiki, which are used to add content in a tiddler. These are transclusion, macros and widgets.

A filter output can change as tiddlers are added and deleted in the wiki. TiddlyWiki recalculates on the fly, automatically updating any filter-based counts or lists as well.

1 Like

@TW_Tones, are you still maintaining this? Could/should this not be merged with @Brian_Radspinner Those pesky brackets… page and possibly the Filters and Transclusions page in GROK?

bobj

Sorry worry what is this?

The write up on filter language