@filter-wizards operator for searching "everymention"?

#1

Taking the search operator as a starting point

[all[tiddlers+system]search:text[my-word]] --> 53 tiddlers

Is there a way I can get a count of all mentions?

Pseudo-codally speaking,

[all[tiddlers+system]search-all:text[my-word]count[]] --> 42,123 mentions

OR

[all[tiddlers+system]search:text:everymention[my-word]count[]] --> 42,123 mentions

Is there an existing op that can do that?

TIA!

#2

The approach to something similar I have done in the past is, first to obtain the list of tiddlers containing one or more mentions then iterate these counting the number of occurrences therein. You could even store the count for each tiddler in the tiddler or elsewhere with a batch button however you can ‘‘reduce’’ it with an accumulator.

  • Even if in the end you find a single filter to do this, I find it easier to use nested list widgets in the first go.
  • One more method is required, I am looking to locate it now, the actual string count.
#3

@CodaCoder here is my work in progress I expect you have the skills to complete;

Use on TiddlyWiki.com

\function tiddlers-count() [enlist<tiddlers>count[]]
\procedure everymention(string)
<$set name=tiddlers filter="[all[tiddlers+system]search:text<string>]">

;"<<string>>" found in <<tiddlers-count>> tiddlers.

<$list filter="[subfilter<tiddlers>!is[system]limit[10]]" >
   tiddler=<$link/><br>
   {{{ [{!!text}splitregexp[\s]lowercase[]match<string>] }}}<br>
</$list>

</$set>
\end everymention

<<everymention filter>>
  • The search finds tiddlers containing search term
  • For each tiddler we spilt on whitespace and match with the string
    • Later we would count this
      • but it needs a better regular expression because it misses “filter” in "[[Filter|filter]] and $filter
  • Once the correct test/count is done you can count it and join them with a reduce

I should be doing other things, best of luck :nerd_face:

#4

Try this:

find: <$edit-text field="find"/><$list filter="[{!!find}!match[]]" variable="find">

<$set name="tids" filter="[all[tiddlers+shadows]search:text:casesensitive<find>]">

<$list filter="[enlist<tids>]"><$link/> (<$text text={{{ [{!!text}split<find>count[]subtract[1]] }}}/>)<br></$list>

Total = {{{ [enlist<tids>] :map[get[text]split<find>count[]subtract[1]] +[sum[]] }}}

Notes:

  • The first line provides a simple input field for the entering the target text. If the input is not blank, then it gets the target text and sets a “find” variable for further use.
  • The second line gets a list of all tiddlers that contain at least one occurence of the target text using a case-sensitive search. This is necessary because the split<find> filter that is used later on is case-sensistive.
  • The third line displays a link to each matching tiddler along with the number of occurences within that tiddler’s text field. It does this by splitting the field value using the target text and then subtracting 1 (e.g., if the text field contains 2 occurences of the target, then it will be split into 3 pieces.)
  • The fourth line uses the same split<find> technique to map each matching tiddler title into the count of occurences, and then applies +[sum[]] to get the total for all tiddlers.

enjoy,
-e

1 Like
#5

I could not keep away,

  • I made use of my new found favorite, functions, because they present another way to build compound filters, it has a macro with a hardcoded search string.
  • It presents more results for each string than Erics, perhaps because I am using the regular expression which is greedy?

Works on tiddlywiki

\function tiddlers-count() [enlist<tiddlers>count[]]
\function total-count() [subfilter<tiddlers>get[text]lowercase[]splitregexp<string>count[]]
\function search-regexp() [<string>addprefix[/]addsuffix[/gi]]
\function count-string-in-text() [all[current]get[text]lowercase[]splitregexp<string>count[]]

\procedure everymention(string)
<$set name=tiddlers filter="[all[tiddlers]!is[system]search:text<string>]">

;"<<string>>" found in <<tiddlers-count>> tiddlers text field, Total occurrences <<total-count>>

<details><summary>All tiddlers and count of occurrences</summary>

<$list filter="[subfilter<tiddlers>]" counter=item>
   <<item>>. (<<count-string-in-text>>) <$link/>  <br>
</$list>

</details>


</$set>
\end everymention

<<everymention delete>>

Snag_2f27e5d2
Snag_2f27e5d21778×812 44.8 KB

This is an interesting exercise in comparing approaches :nerd_face:

1 Like
#6

Guys, thank you! And sorry, I got busy on a humdinger of a bug (-e should raise his eyebrows if I post the hacky workaround I came up with).

@TW_Tones Sorry, I need pre-5.3.x at this time. :frowning: But I’ll try it soon as I’ve upgraded (my dozens of wikis).

@EricShulman maaan, exceptional diligence as usual. Danke. Merci. Grassy Ass. Cheers mate. :dart:
Enjoy, indeed.

But… all I really want is a number, e.g., 42123. Finding where they are is easy. This is just a “stats thing”.

By the time I upgrade, I suspect Tones will have made his custom filter-op do it in one shot (I’m hopeful, anyway).

Thanks again.

#7

I am not sure about this and it needs validating

\function occurrences(string) [all[tiddlers]!is[system]search:text<string>get[text]lowercase[]split<string>count[]]

# delete occurrences=<<occurrences delete>>
# filter occurrences=<<occurrences filter>>

or < 5.3.x

<$let string="delete">
{{{ [all[tiddlers]!is[system]search:text<string>get[text]lowercase[]split<string>count[]] }}}
</$let>
#8

I’m a little curious as to what’s even meant by this in the context of transcluded material. If TiddlerA, TiddlerB, and TiddlerC all transclude TiddlerD, and TiddlerX transcludes TiddlerA, TiddlerB, and TiddlerC, and both TiddlerX and TiddlerD have single mentions of my-word, how many occurrences would that entail? Two is a reasonable answer, as only TiddlerX and TiddlerD have it in their raw markup. One can argue for four as TiddlerX is somehow the shell containing the others and it will have four occurrences. And of course there is a strong argument for eight, as A, B, C, and D each have one and X has four.

And it gets more confusing if this value is in the title of a transcluded tiddler!

#9

Agreed. However, that isn’t the context. The context is simply “straight text”, in my case, voluminous amounts of the stuff, like, say, Lord Of The Rings: “how many times is Mithrandir mentioned?” – where the entire text is contained in one wiki, where each chapter is split into “sections”, where each section is a tiddler.

That, is my use-case. :slight_smile:

1 Like