Extra Filter Operators v1.1.0 - Smarter searching

Version 1.1.0 is out, well that was a quick increase of a minor version:

  • susearch now supports strict flag which will cause it to only match if every word from the query is found in an input.

Is the query treated as a search-phrase or a list of search-terms? If the latter, does the order matter?

Thanks @Maurycy - that helped me :slight_smile:

I did small modification, that the link is also shown, when Do marking option is unchecked:

		<li>
			<$list filter="[{$:/temp/SusearchLiveExample!!do-marking}match[1]]" emptyMessage="""<$link><$text text=<<currentTiddler>>/></$link>
            <span style="font-size: 0.75em; color: rgb(144, 238, 144)" >&nbsp; &nbsp;
            <$view field="published" format="date" template="DD. MMM YYYY"/>
            </span>""" variable="_">	
            <$link><$transclude tiddler={{{ [<currentTiddler>susearch-mark<query>] }}} field="title" /></$link>
            <span style="font-size: 0.75em; color: rgb(144, 238, 144)" >&nbsp; &nbsp;
            <$view field="published" format="date" template="DD. MMM YYYY"/>
            </span>
				<$list filter="[<field>match[text]]" variable="_">
					<blockquote><$transclude tiddler={{{
						[<currentTiddler>get[text]susearch-mark:raw-strip<query>focus-on:regexp<regexp-query>]
					}}} field="title" /></blockquote>
				</$list>
			</$list>
		</li>

One additional issue:
the search text will be shown in the result list only once - even if there much more there…

I think it’s already explained in the structure section of the operator – if it’s not clear enough I’d love to know how to improve it:

  • Suffixes:
    • :field(optional) Field through which to search. Defaults to title
    • :flags (optional) Multiple flags can be provided by delimiting them with a comma ,.
      • raw-strip will exclude all WikiText and HTML while matching. Most useful when searching through the contents of tiddlers.
      • strict will only match if the whole phrase or all words are found.
  • Operands:
    • [query] – The text to search for. It will match:
      • The whole phrase
      • Every word in the phrase as-is
      • Every word in the phrase with stripped special characters, while also stripping the same special characters from the searched text.

I’ve bolded the relevant sections.

That’s intended, see this reply. If you want to display the whole contents with highlights you don’t need to use focus-on. I didn’t really see a use case for making focus-on focus on all occurrences so that’ why I didn’t implement it – you think you’d find it useful?

You tell me – is this an improvement?

Strict means strict, therefore stripping special characters should not happen. Perhaps add another flag/switch?

“strict will only match […] or all words” could be taken to mean any order.

“The whole phrase” implies perhaps “the phrase as a whole as written” but doesn’t say that explicitly.

“Every word in the phrase as-is” runs counter to “Every word […] with stripped characters” in what follows.

I don’t offer any real improvement because I don’t know for sure what’s intended – hence my question(s).

Why not try it and see? :slight_smile:

Because that’s getting the cart before the horse. :sunglasses:

Active focus-on make sense.

That means, more hits are shown when they within the focus length of 128 (=default) - right?

The problem with strict is that it’s so context and past-experiences dependent I shouldn’t have used it. all-words would be much better candidate and I’ll change it to that.

As for the rest there are multiple opposing needs here: brevity, clarity, precision. Then there is also other things like English being my second language and neurodivergence which makes some things that are normally difficult (documentation) even harder.


But you also make another great point I didn’t realize until I started writing the reply, because I didn’t think there is a way the order of words can matter because we’re just filtering, not sorting… But of course the order of words can still matter.

“Every word in the phrase as-is ” runs counter to “Every word […] with stripped characters” in what follows.

The stripped version is for situation like searching for either can't or cant will still match the other.

Anyhow I’ve got some ideas about how to improve the documentation now, thanks :).

Yes, exactly.

focus-on essentially combines two things:

  • Find the first index of a match
  • Substring a bit of the text before that match and some after (+ optional rudimentary handling for HTML tags)

So if multiple matches are found in the first substring range they’ll still be included. The target for that functionality was a situation that I see in many apps that support search, including Google, where there is limited screen space for a preview so we need a small slice of the text:

I can see focus-on being very useful! Would it be difficult to allow it to set a focus length based on number of words rather than characters?

It would be a bit of a pain and would make it orders of magnitude slower (not that it matters much, it would still be fast enough) but there are no technical limitations. I just wouldn’t use it myself as I feel like the contexts in which focus-on is mostly useful deal with limited about of space so you are more interested in number of characters which varies less than if it were the number of words.

I can add it to the roadmap as a possible “I might work on this someday” but unless there’s an actual interest from multiple people I’m unlikely to go down that rabbit hole, sorry :slight_smile:.

I took my sweet, sweet time but version 1.2.0 is now out. I made the mistake of getting distracted mid development so coming back to 80% finished code was a challenge to figure out but I think stuff should be working fine.

I mainly reworked the susearch filter to be a bit easier to use.

1 Like

@Maurycy
In this live search example tiddler, how to get Do search Do sort Do marking checked by default Also how to keep All tiddlers and Search in text as default option for searches.

I tried to keep those a default values, but it was not working correctly.

Can susearch-sort support multiple fields?

susearch-sort:title,caption,text,keywords:raw-strip[query]

Seems keywords are not being matched.

Also this doc is great, would you want to port this macro to tw official doc, to make tw official filter docs looks like this?

Whoops, I somehow missed replying to these!

I sure wouldn’t mind, assuming there is interest in the core developer team/community to change the docs to look like that. It’d certainly require ironing out some kinks.

No, it only supports searching through a single field. It’s something that could potentially be added, though it’d make the already-confusing scoring system even more confusing.

Sorry, I am afraid I don’t understand the question. The checkboxes control the generated filter (with the exception of Do marking, which is handled in a separate step when displaying the results).

@Maurycy I want those checkboxes to be in checked state by default. I tried some modifications to the code after reading about checkbox widget, but it was not successful.

Also I want All tiddlers and Search in text as default option for select widget on the side of the search box.