Extra Filter Operators v1.1.0 - Smarter searching

Plugins
,
#1

The Sales Pitch

Search is hard. Thankfully the extreme brains working for me have donated their gray matter to develop an incredibly smart solution that will solve all your problems¹. How?

With extra filter operators of course! Coming to your house now!

¹ - No promise of actual problem solving is being made here .

Documentation · Demo · Install: Extra Operators

And a bit more seriously

As part of working on an unannounced project I wanted a search mechanism that has more functionality than you can squeeze from vanilla TW. I didn’t like the idea of shipping those with the plugin and so the idea was born to share them.

As it usually happens with my ideas, what was one simple filter turned into 4 complex filters that should cover all your searching needs together with new and exceptional documentation, unit tests guaranteeing no regressions and powerful live example.

There are four filter operators available as of now:

  • susearch – search that’s slightly more powerful than the good-old search; among others it can strip away wikitext to avoid false positives caused by the structure.
  • susearch-sort – sort the results in an intelligent way that for the most part should feel just right and natural.
  • susearch-mark – for highlighting the matches.
  • focus-on – for extracting a part of the input, specifically designed for partial previews of the matched text.

Documentation · Demo · Install: Extra Operators

Screenshots

Well isn’t that documentation just glorious?

image
image1461×989 89.6 KB

These examples are to die for

image
image1231×825 39.7 KB

The live demo feels so alive!

image
image1321×829 79.6 KB

Version history:

  • 1.1.0 – susearch now supports strict flag which will cause it to only match if every word from the query is found in an input.
  • 1.0.2 – susearch, susearch-sort and susearch-mark no longer ignore spaces when matching words (ie. sand no longer matches s and).
  • 1.0.1 – focus-on fixed a bug where the last character would be trimmed if the whole text was supposed to be displayed
9 Likes
Any precise advanced topic based search operator?
#2

Now that I finally got it off my plate I can go back to Auto Complete and go through the GitHub issues and suggestions reported. Or something!

2 Likes
#3

Eagerly waiting for this one

#4

Great job! Your plugins and additions are really useful. You are on the right track and hope to see more and more useful tools from you!

I would like to add few comments may be useful for future development

  • there was some efforts in the past to include fuzzy search (I mean here to get better results)
  • a discussion here has some good points for improvement search results. @pmario has given some good hints there
  • Searching in links with the word search brings some good previous efforts

Thank you again for all your efforts.

#5

I am aware of this one but I wanted a low-cost solution that can seamlessly integrate and be multi-functional. Simple filters seemed like the best bet for me :).

I’ve seen that thread though I haven’t responded myself. I think my solution is kind of similar to what @pmario had in mind, in that most of the heavy lifting is done in the dedicated sort operator (susearch-sort) and in general it prefers fuller and earlier matches.

#6

@Maurycy

I’ve been playing around with your extra operators and I have some questions. I feel like I’m not using them quite correctly.

In my Kansas Railroads TW, all of my railroads are tagged with Railroads. In trying to use the focus-on operator, I entered in the filter search tab of advanced search [tag[Railroads]focus-on[Pratt]]. This should have yielded a handful of tiddlers. Instead the results list was 1836 matches which is the number of tiddlers tagged with Railroads, but the links are all showing as missing because the last character of the title is missing. (see screenshot below)

image
image966×789 90.8 KB

The susearch operator seems to work just fine. It yielded 4 results which are the ones with Pratt in the title.

image
image968×416 27.7 KB

The susearch-mark operator returns the same number of tiddlers as the focus-on operator, but the last character isn’t missing.

image
image909×840 80 KB

the susearch-sort operator yields all 1836 tiddlers, but the ones with Pratt in the title are listed first.

image
image1031×767 71.1 KB

Give how robust your plugins are, I feel like I’m not doing some correctly. What might I be doing wrong?

#7

Oh dear, I’d hope the descriptions, documentation and examples would be enough to explain how to use those but I never ran them by anyone else so it appears they might not be doing their job.

Essentially the way to use those oeprators is as follows:

  • Start the with a list of titles you want to search through.
  • susearch will search through the tiddlers, removing any that don’t match the search.
  • Then susearch-sort will sort the results such that the more relevant the result is, the higher it appears in the list.

With this list of results you can now display them:

  • Use susearch-mark to add highlights – you shouldn’t do that in the original filter, because then you won’t be able to extract the original title. So you can link to <<currentTiddler>> and then use <$transclude filter={{{ [<currentTiddler>susearch-mark<query>] }}} field="title"/> to display the highlighted result.
  • Use focus-on if you do search in the text of the tiddler and you want to display a short preview of the relevant passage in the tiddler.

Go to the live demo, mark all three options, select “All Tiddlers” + “Search in text” and you can see all the operators in use:

  • susearch and susearch-sort are used to filter and sort the results
  • susearch-mark is used to add highlights both in the title and the preview
  • focus-on is used to generate the preview of the body, extracting the part of the tiddler’s text that contains the first mark + its context.

As a side note I see I broke something last minute, will fix it today!

#8

New version 1.0.1 released:

  • focus-on fixed a bug where the last character would be trimmed if the whole text was supposed to be displayed

I also made the three big changes to the showcase:

  • Fixed the live search example not working correctly for titles, Ducks and Idioms.
  • Expanded documentation for susearch-mark so that it explains how to get the marked text to display
  • Added explanation to how the search in the live example is done.
#9

Hi @Maurycy, I think I may have found an issue. Is the following working as you would expect?

image
image814×771 58.8 KB

It’s finding…

   things (A|a)nd something

not just “sand”.

#10

Great catch! I’ll fix it later today. This was caused by the mechanism which does the simplified match (step which omits special characters).

#11

Thanks for the tips. Got it figured out I think. One question. Does the focus-on operator only find the first occurrence in the text or will it find all?

#12

Hello @Maurycy

I tagged the Tiddler Susearch Live Example with $:/tags/AdvancedSearch

  • Checking an option in Susearch Live Example will not sync it in $:/AdvancedSearch - why?

  • the search results are different:

grafik
grafik1849×981 83.3 KB

#13

1.0.2 was released:

  • susearch, susearch-sort and susearch-mark no longer ignore spaces when matching words (ie. sand no longer matches s and).

Fixed in 1.0.2 now!

It only finds the first occurrence. Do you have a use case for displaying more? I guess it could return each match as a separate title in the output, if you’d actually use this functionality I’ll add it to my roadmap as an optional flag.

You have hallo in the first screenshot and hello in the second, that explains the discrepancy between the results :slight_smile: . I added the tag and it appears to be working fine, can you explain what you mean by “will not sync it”? I’ve opened both tiddlers at once and any change to either makes the same change to the other (It stores its data in the temp tiddler $:/temp/SusearchLiveExample without any qualify, so it should appear the same in both)

#14

I do. Same as Ctrl+F in the browser offers a “highlight all” switch.

#15

But that’s what susearch-mark accomplishes, it highlights all results already. I don’t recall ever seeing a tool with search that offers mini-preview of a search result also showing multiple matches for it – it’s always the first one (or any one, I am not sure). Not that there is anything wrong with upsetting the status quo, I just don’t want to do work for the sake of doing work (I say that but I am already working on one other change I need to susearch so who knows if I won’t do this one too in one go).

#16

Sorry. That’s what I was thinking of. Cool.

#17

image
image1672×1000 56.9 KB

Just a suggestion - Is it possible to make complex queries as shown in the image using the plug in, may be in the future ? (Image is taken from an app called Remnote - selections were made just for demo purpose)

This is the closest example I could find in TW.

#18

It depends. If you’re asking for “Can you make complex queries like that?” then you can do them even without this plugin, since it mostly just expands on text-search.

If you’re asking “Can you make a query builder with this plugin?” then the answer is no, it’s just a collection of filter operators. Search query builder like the one you linked sounds like an interesting challenge but it’s one that’s too far from my needs to work on it.

#19

Sorry - my fault :frowning:

exact - thats working now :slight_smile:

I tried to show the tiddler as a link in the list with an additional field:

So I replaced

<$transclude tiddler={{{ [<currentTiddler>susearch-mark<query>] }}} field="title" />

with

<$transclude tiddler=<$link/>{{{ [<currentTiddler>susearch-mark<query>] }}} <span style="font-size: 0.75em; color: rgb(144, 238, 144)" >&nbsp; &nbsp;<$view field="published" format="date" template="DD. MMM YYYY"/>&nbsp; &nbsp;</span>

Result:

  • it works only in Do marking option
  • not highlighted in Search in title

grafik
grafik1122×494 88.5 KB

How can that be fixed?
Thanks in advance.

#20

If you want to link the result you’ll want to do something like this (split into multiple lines for clarity):

<$link>
  <$transclude tiddler={{{ [<currentTiddler>susearch-mark<query>] }}} field="title" />
</$link>
<span style="font-size: 0.75em; color: rgb(144, 238, 144)" >
&nbsp; &nbsp;
<$view field="published" format="date" template="DD. MMM YYYY"/>
&nbsp; &nbsp;
</span>

This way you’ll wrap the output in the link. The way you wrote it Tiddlywiki most likely interpreted it like this:

<$transclude tiddler=<$link/>
\__________/ -- The widget name
             \______/ -- Set the attribute `tiddler`
                     \____/ -- to `<$link`
                           \/ -- Self-closed transclude widget

Sot it got completely confused.