RC-1: search-variables, dumpvariables rewritten to power $:/AdvancedSearch - Variables Tab

[pmario]

Hi Folks,
I’m happy to announce RC1 for the AdvancedSearch - Variables Tab. I was using the new possibilities a lot, while creating several PRs that I did lately.

The latest preview can be found at: https://tiddlywiki5-git-fork-pmario-dumpvari-87399a-jermolenes-projects.vercel.app/

Goals

• Get a compact overview of global variables
• It should be easy to “narrow” down the resulting list
• It should have fast access to the variable content
• It should link to the tiddlers, where the variables are defined
• There should be an easy way to find out where the variables are used
• Have fun!

New AdvancedSearch Variable UI

1. New Variables tab – in $:/AdvancedSearch reachable with keyboard navigation
2. Type Radio Buttons – by default it will ist all variables visible in AdvancedSearch
3. Sort Selector – alphabetical / raw. Raw finds the variables as they are in the variables “prototype chain”. So the nearest “scope” will at the top.
4. Expand / Fold All – Search results can be expanded to see more info
5. Number of global variables
6. Variables – formatted according to their definition signature
7. Search input – Filters the list. So it will reduce the number of matches
8. Clear Search Input
9. Show Exclude Text Input

9. Show Exclude Text Input (star indicates, that there is an active exclude filter )
10. Show Saved Exclusion Configurations dropdown
11. Select an exclude configuration
12. Create a new exclude filter configuration
13. Edit an existing configuration

14. Show Variable Details
15. Search Variable Definition Tiddler
16. Show Variable Content

15. Search Variable Definition Tiddler
16. Show Variable Content
17. Search for 19) in all shadows+tiddlers text and tag
18. Text from 19) found in tiddler x
19. If nothing is found users can remove some info in the search field, to make the search less specific.
20. Reset Details Search Text

New or Extended Core Macros

• Completely New search-variables Macro which is used in the AdvancedSearch tab.
• search-variables also has type, sort, subfilter, format parameters to be used with a limited subset of results.
• dumpvariables Macro got an update and new parameters
  • type, sort, subfilter, format
  • This macro is there for backwards compatibility reasons. Usability is slightly improved because of the new parameters

Important: The search-variables macro can also be used with predefined type, sort and subfilter parameters, which allows us to drastically limit the available variable list.

New or Extended Filter Operators

• variable Operator got a new suffix and parameter
• format Operator got a new VariableFormat: suffix: variable
• VariableFormat: defaults to: $type$ $name$($params$) $firstLine$
• getvariables Operator is updated with new parameters: text, value needed for info about \functions

Feedback

Here or at GH: Dumpvariables rewrite new syntax plus search by pmario · Pull Request #7992 · Jermolene/TiddlyWiki5 · GitHub

have fun!
mario

[pmario]

reserved for future updates

[Scott_Sauyet]

Wow, that is an impressive amount of work, and I look forward to trying it soon! (Vercel is blocked at GigantiCorp, my day job.) I supposed I could grab and test from your branch, but lunchbreak is too short for that. If my evening commitments end early enough, I’ll check it out then.

[CodaCoder]

This is looking very useful to have, @pmario. Bravo.

[TW_Tones]

Excellent work @pmario you have completely changed the way a designer developer can work with TiddlyWiki with these tools, and likely will also help new users coming to terms with how tiddlywiki works.

Wow comments

• So often I tried to teach people how to research how to do something through tiddlywikis own methods. But this was always too complicated to easily share. It will be much easier with these tools.
  • Even doing my own research in tiddlywiki
• Once included in the core, it will be possible to use new or Extended Filter operators to support self documentation
• I think I can extend the plugin view template to report on such variables defined within the plugin shadow tiddlers, a great way to learn how a plugin works.
  • And get coding inspiration because variables/procedures etc… are the main way features are made available, in TiddlyWiki.

Feedback

• I wonder if published wikis may better hide this tab, as it will not server users much, only enthusiast and wiki designers?
• I am working on an annotated version of the tiddlywiki.com and documentation tiddlers, this makes me wonder if a feature you could add is the ability to save a note against any element this tool can find, then when people find something of interest they can generate an annotation to assist later.
  • Example Variables> Var > Open ignore variable > open some tiddlers using ignore
  • Then observing “it is used as a placeholder for variable on list widgets” one could create an annotation against the ignore variable.
  • If nothing else perhaps you could make (the) list item templates so designers/users can do something like this?
  • another example would be to place in the annotation for tv-story-list the link to “tv-story-list Variable” tiddler, as the “canonical” definition.
• I think we may be able to place more information along side previews of a tiddler, a preview in a tiddler or difference previews, maybe even inside the Import tiddlers?.

I will keep on playing;

[Edited] I will add any more observations here

• Of course in this case the widget variables only relate to custom widgets defined with \widget however it makes me wonder about finding core widgets and macros etc… even if we excluded javascript defined ones?
  • How is it treating shadow tiddlers be they in the core or a plugin? Could we use a checkbox to include these as well, or only?
• Would it be possible to transclude the variables tool to put it in a particular context, to see what is valid their?

If we look in the procedures and go to \procedure .doc-tab-link(text, target, tooltip, class) we see the parameters. Then we open to see the definition;

• I wonder if we should reproduce the first line of the definition as annotated here, above the definition?, if not just the dropdown?
  
  

Observation;

This will make it easy for us to search for an use existing macros rather than necessarily creating new ones (keeping in mind the dependencies).

Ideas

• I wonder if a button to copy the full definition would be helpful?
• What if we could construct or annotate tiddlers with one or more invocations, that can be copied to clipboard.
  • eg for \procedure .link-badge-here(link, colour:"#d88e63") we would give;
  • <<.link-badge-here link:"" colour:"#d88e63">>
  • <$macrocall $name=.link-badge-here link="" colour="#d88e63"/> be aware that macrocall works without loss for most procedures and reads better for users, in my opinion.
    • <$transclude $variable=.link-badge-here link="" colour="#d88e63"/>
• The current nature of the display of variables makes it a little difficult to highlight part of the definition to copy the macroname or parmkater etc…
• Inside a variable definition view, the search is global, what about searching within the current variable definition, someone may be looking to see how a particular parameter is being used within the definition.

Sorry I am giving such a large reply, hopefully the things I rase a somewhat simple, but feel free to ask me for help if needed.

• Procedures and macros are only visible if global, this is understandable. But consider this definition.

\procedure proc-with-define()
   \define in-proc() Nothing
no much
\end

• Even with the $:/tags/Global macro, the in-proc macro will not be listed, again understandable, however if we allows a full wiki search, not just global definitions we may catch it.
  • Also show non global definitions
  • If nothing else a clear statement of global and non-embedded being excluded would be wise.

[pmario]

@TW_Tones

Of course in this case the widget variables only relate to custom widgets defined with \widget however it makes me wonder about finding core widgets and macros etc… even if we excluded javascript defined ones?

Core widgets are all javascript. They are not part of the “variables chain”. So they are not visible. Extracting the core widget signatures from the source code would be a completely different project.

@TW_Tones

How is it treating shadow tiddlers be they in the core or a plugin? Could we use a checkbox to include these as well, or only?

18 and 19 will find variables that are defined in shadow tiddlers. Only variables that are used will be shown using the variables operator.

@TW_Tones

Would it be possible to transclude the variables tool to put it in a particular context, to see what is valid their?

Yes it can be called using <<dumpvariables>> or <<search-variables>> in a tiddler to be used for documentation purpose.

Debugging should be done using the existing <$log> or <$action-log /> widgets

@TW_Tones

I wonder if we should reproduce the first line of the definition as annotated here, above the definition?, if not just the dropdown?

Not really. That would make it copy/paste able, which will not produce working code.

If no tiddler is shown, we need to remove some text from (19) eg: \procedure .doc-tab-link, so it will show us where the definition comes from.

The “definition” information is useful to get an overview, but it can not be used to copy/paste code. Many macros will not work if copy pasted from variable content, because some needed code is defined in other variables.

The details search is intended to find the tiddlers where the macros are defined. Copy / pasting code needs to be done from those tiddlers. Neither dumpvariables nor search-variables do look up anything from tiddlers.

Everything show in the Variables tab is created using variables, format and getvariables operators. Everything comes from internal memory.

@TW_Tones

I wonder if a button to copy the full definition would be helpful?

As I wrote, most of the code will not work if copy / pasted from there.

@TW_Tones

What if we could construct or annotate tiddlers with one or more invocations, that can be copied to clipboard.

Same thing here. Will not really work. We need to go to the tiddlers, where the code is defined by changing the details search string to: \procedure .link-badge-here – For me removing some text is much simpler, that adding it. …

I’ll have a closer look and I’ll make sure, that all detail-searches do produce links to makros. So I’ll probably have to remove the parameters from the search string.

@TW_Tones

The current nature of the display of variables makes it a little difficult to highlight part of the definition to copy the macroname or parmkater etc…

That’s intended.

@TW_Tones

Inside a variable definition view, the search is global, what about searching within the current variable definition, someone may be looking to see how a particular parameter is being used within the definition.

That’s not possible with empty.html. We would need the core dynannotate-plugin for that.

If users used the uglify-plugin, the code shown it the details section will be completely unreadable.

I did not think about that. I’m not sure if I can detect uglified code. May be if there is only 1 line-feed and the line is longer than 120 chars. … I’ll cut the code there … not sure yet.

@TW_Tones

Sorry I am giving such a large reply, hopefully the things I rase a somewhat simple, but feel free to ask me for help if needed.

No problem. I can see how others want to use it.

My intention was to quickly find the tiddlers, where code is defined. So I could see, what has to be rewritten. As you can see, all the doc-macros starting with a dot have been rewritten to TW v5.3.x syntax already. This tool helped to find out, if I did miss something.

@TW_Tones

Even with the $:/tags/Global macro, the in-proc macro will not be listed, again understandable, however if we allows a full wiki search, not just global definitions we may catch it.

• Also show non global definitions
• If nothing else a clear statement of global and non-embedded being excluded would be wise.

As I wrote. Everything comes form memory. The variables tab searches in-memory only.

It will be shown like this. So the link to the definition tiddler is the important thing to be found.

My Conclusion

From the presented information, there seems to be a strong desire to copy/paste content, which will result in non-working code.

So I will probably have to change the “variable content” (16) in a way, that makes it clear that it is a hint only. - Not sure yet. - I’ll try and cut it of after 120 chars or so. I’ll see it that makes sense

I’ll also make sure, that the details search will show links.

Thanks for the feedback.

[TW_Tones]

Thanks Mario,

I will respond with alternative or demos in response to some of your comments. I know the value of coping text is sometimes limited, however there is value letting it be possible, some macros and function especially are somewhat trivial and it is useful.

• But I will review it again in light of your comments.

As I said this is a substantial utility to using and researching tiddlywiki.

I see now that you are only working in everything from memory. I suppose the next question, that I am not asking you to answer, is extending this into the world of local or in tiddler definitions.

[pmario]

@TW_Tones

I suppose the next question, that I am not asking you to answer, is extending this into the world of local or in tiddler definitions.

hihi, … As you did not ask

TW variables are “nested” so the following code will give us a test-variable with 2 different values, depending on the “scope”.

The <<transclusion>> variable gives us the the tiddler context.

transclusion variable: <<transclusion>>

<$let test="abc">
test: <<test>><br>
	<$let test="inner">
		inner test: <<test>><br>
	</$let>
test: <<test>>
</$let>

• Whenever a tiddler is transcluded the core defines the “transclusion” variable and all the content in the tiddler is stored as “children”.

• So, the “outer test” internally are stored as a child of “transclusion”

• The “inner test” variable is also named test, but it is stored in a different place as a child of “outer test”

(That’s not 100% correct but it should give you an idea)

So there is a possibility to dig out all internal variables that are part of a tiddler. – But I do have no clue atm, how we could represent that in a reasonable way.

The existing “widget-tree” tiddler preview pane does show this info. But it’s way to complex to be useful.

[TW_Tones]

I think finding the \define \procedure \parameters and \function pragma are more important, than the nested variables that are not one of these, unless we get the ability to redefine these, Having those in the dumpvrariables is sufficient.

Even when pragma are inside other pragma like my example;

@TW_Tones

\procedure proc-with-define()
   \define in-proc() Nothing
no much
\end

Are themselves less important because even if this definition was imported or at the top of the tiddler, the inner in-proc is only valid within proc-with-define which is almost intended to be hidden. As a result its very unlikely to be sought, and can easily be duplicated.