Proposal: TOC-macros Rewrite using IF, Functions and Procedures (+new fuctionality)

[pmario]

I did just update and basically rewrite all the toc-macros with functions, procedures and the conditional if function.

I think the code is much more approachable now, since there should be no weird “inverse logic” anymore. I did eliminate all the emptyMessage stuff, which is super hard to debug because of the “twisting logic” it requires (most of the time ;).

With the new code it’s easy to create a new function, that will expand all the branches with 1 button and 1 new “useGlobalState” variable. It’s usable for small tocs, with may be 100 tiddlers open at the same time. But at tw-com it fails, because it opens 1000++ of elements and users will loose the context.

So I’ll try to implement a version, that allows us to expand from a “level 1” link and not from the root.

More experiments are needed. I hope I’ll be able to publish a “preview” this weekend.

have fun!
mario

[baiguai]

Wow! Excellent. Thank you. This is very exciting

[TW_Tones]

Mario I am keen to add a couple of things to the standard TOC so as you approach completion could you consider them?

• For each node in the tree if it contains a condition field use that to determine if that node (and thus its children are to be displayed)
• For each node in the tree if it contains a child-filter use that not the tags to list the children, then for the children use their condition/child filter or return to the tag method.

In both cases if the fieldnames default but can be overridden eg condition/toc-condition or child-filter/toc-child-filter

• A condition could hide a branch of items such as when in read-only mode
• A child-filter could list incomplete todo items in oldest first order
  • Or allow listing with a completely different filter to the TOC

If you dont find the WHY “self evident” let me know and I will put the full argument.

[pmario]

@TW_Tones

• For each node in the tree if it contains a condition field use that to determine if that node (and thus its children are to be displayed)
• For each node in the tree if it contains a child-filter use that not the tags to list the children, then for the children use their condition/child filter or return to the tag method.

I think that makes sense. We already do have a toc-link: yes/no option, which allows us to disable linking to those tiddlers.

I can think of a toc-exclude filter-field, which removes the (1) item. It would be similar to the exclude macro-parameter, which is global. So IMO it’s a more fine-grained version of exclude

If there is toc-exclude, IMO a toc-include filter-field would make sense too.

I think both of them could be implemented as ´[subfilter{!!fieldName}]´ – Not sure how complex the new filter string will be. Some experiments needed, but I could see the usecases.

At the moment I think toc-exclude will take precedence. So the pseudocode workflow will be:

• [tag<tag>] filter will be evaluated first - then
• [subfilter<toc-include>] filter will be able to add new elements - then
• -[subfilter<toc-exclude>] filter will remove elements again

IMO toc-exclude: yes should be a possibility for -[subfilter<currentTiddler>], which will remove the possibillity to exclude a tiddler with the title: yes. Not sure about this one. What do you think?

With this mechanism toc-exclude will be able to remove / hide everything if needed.

Show some code for experimenting

<$let
	myFilter="[tag[one]]"
	toc-exclude-yes="[<currentTiddler>]"
	toc-exclude="asdf"
	toc-include="[tag[test]]"

	toc-exclude={{{ [<toc-exclude>match[yes]then<toc-exclude-yes>else<toc-exclude>] }}}
>

<$list filter="[subfilter<myFilter>] [subfilter<toc-include>] -[subfilter<toc-exclude>]"/>

just some thoughts.
-m

[TW_Tones]

@pmario

I think both of them could be implemented as ´[subfilter{!!fieldName}]´ – Not sure how complex the new filter string will be. Some experiments needed, but I could see the usecases.

• I have done a suit of POC examples on conditions I may one day argue be added to the core for toc, tabs, toolbars, even tags and view template items over here https://conditional-elements.tiddlyhost.com
• However I still feel there may be even better ways with custom operators rather than subfilter, in many ways they are the same.

@pmario

So the pseudocode workflow will be:

• [tag<tag>] filter will be evaluated first - then
• [subfilter<toc-include>] filter will be able to add new elements - then
• -[subfilter<toc-exclude>] filter will remove elements again

• I totally agree in this order, then include can be an even bugger or sophisticated filter.

In your code examples you are primarily setting variables to filter, but I am also keen to set the fieldnames to be used and place exceptions in this fieldnames so every node can be different.

• Also have a look at my design-mode condition to toggle multiple conditions output.

[pmario]

TLDR; Jump to Anyway at the end

---

@TW_Tones

In your code examples you are primarily setting variables to filter, …

Yes, that’s just to play around with some variables to test the filter commands in a simple setting, where I have control over everything. That’s basically a test-bench to develop filter strings.

@TW_Tones

… but I am also keen to set the fieldnames to be used and place exceptions in this fieldnames so every node can be different.

I think having different field-names, that can contain filters is overkill. It will increase the complexity exponentially.

As I wrote: TOCs are recursive. So eg:

• If I do have 2 new tiddlers named: aaa and [[b b]] - both only tagged with: aaa and
• Tiddler FirstOne gets a field: toc-include: [tag[aaa]] the outputted toc will look like this:

That is the equivalent of aaa being tagged: FirstOne instead.

---

Anyway. In the way it is implemented at the moment, you could use the following code, which would read the filter string from toc-xxx instead. I think, that should be flexible enough, in a “probably” backwards compatible way.

\function tf.toc-include() [<currentTiddler>get[toc-xxx]]

<div class="tc-table-of-contents">
<<toc "Contents">>
</div>

[TW_Tones]

@pmario

I think having different field-names, that can contain filters is overkill. It will increase the complexity exponentially.

Not at all.

I may leave all my TOC as is, but now under each project I may list task to be done. Adopt a system so exceptions are trivial.

But it does not matter I can make my own TOC macro, I just think it would be a winner. For example each user could have their own TOC branch. Thats only one exception for each user.

[pmario]

@TW_Tones – I did just upload a working POC to GH: Work In Progress - Toc v5.3.x rewrite by pmario · Pull Request #7961 · Jermolene/TiddlyWiki5 · GitHub

Important: Read the info at GH. Only the toc macros uses the new fields atm.

<div class="tc-table-of-contents">
<<toc "Contents">>
</div>

If something does not work please post at GitHub. This thread should “stay” as a general discussion about the possibilities. Issues should be posted at GH

[Scott_Sauyet]

There is one feature I would love to have built-in to the TOCs: Differential sorting. AFAICT, there is only one sort at a time for the TOC. I would love it if, for instance, when I have a list of Person tiddlers nested somewhere in my TOC, I could sort them by last-name then first-name, rather than the default alphabetic sort running in the rest of the TOC.

I’m not sure where the configuration for this would belong: Does the Person tag have a toc-sort property that might be fed to sortsub? Or does the call to the TOC macro include a sort-Person property? While I slightly prefer the latter, spaces in the tag name would take some special handling.

@pmario: I don’t know if this would fit at all with what you’re doing, which I think is more focused on modernizing the TOC code than adding features. But it’s something I would love to have!

[TW_Tones]

@pmario

@TW_Tones – I did just upload a working POC to GH: Work In Progress - Toc v5.3.x rewrite by pmario · Pull Request #7961 · Jermolene/TiddlyWiki5 · GitHub

@pmario excellent work rewriting these to a much more readable and “modern” code layout, especially given all the “inherited features”

• It does need extensive testing and I do hope to spend time on it.
• It may very well make it easier for people to understand in future, although is is packed with features and complexity as a result.

As we say in Australia

Bloody good work, mate!

Notes:

• I really appreciate the use of fields within the toc nodes and while I fully support the concept of include and exclude fields/filters;
• I am still not sure how to replicate the behaviours I intended for the condition and child-filter fields, I can see how we may effectively do this but it is a little more cognitively challenging dealing with includes and excludes.
  • Perhaps you could write a paragraph on each to say how?
• The code patterns we (you in particular) establish now, in the TOC macro, will of course be translated into all the toc macros so its worth a robust discussion now.

Avoiding a lost opportunity

• About the condition possibly toc-condition field, Please note, I did not expect you to follow my link to https://conditional-elements.tiddlyhost.com/ before, because I know its a rabbit hole, however it is more important now. I have done considerable work to look at introducing the condition field, (already present in my design) for editor toolbar buttons, tabs/toolbars/tags and the last step was the TOC you are now working on.
  • I think there is a strong argument to continue with this code pattern already found in the editor toolbar buttons, to these other aspects of the core and into the TOC macros.
  • This condition field can be in addition to you current proposals, as could the child-filter (by any other name) toc-children. the key difference being they replace the normal behaviour when encountered.
  • I have written such code primarily in the filters, and happy to help.
• Using the condition for tiddlers with a given tag, I hope to introduce a condition to the iteration of $:/tags/ViewTemplate* tags and $:/tags/EditTemplate tag.

@Scott_Sauyet

Differential sorting.

This should be able to be done, adding a sort, within the optional fields on a node within the TOC, it may be achievable in the proposed toc-include and toc-exclude.

• This is one reason I favoured a field that replaces the standard toc filter eg [tag<currentTiddler>] with the a “child-filter” for specific nodes (ie not via include and exclude)
  *This would allow the easy addition of sort operator, perhaps even sortby.

[Scott_Sauyet]

@TW_Tones

@Scott_Sauyet

Differential sorting.

This should be able to be done, adding a sort, within the optional fields on a node within the TOC, it may be achievable in the proposed toc-include and toc-exclude.

Are you saying that this feature should already be available in TW, or that it is something that could reasonably be added?

[TW_Tones]

@Scott_Sauyet

Are you saying that this feature should already be available in TW, or that it is something that could reasonably be added?

• There is a sort parameter in the current Table-of-Contents Macros but I belive this applied to the whole TOC, see below
• It is something you may be able to do with @pmario TOC macro rewrites discussed here.
• It is definitely something that could easily be done with the code pattern above I am promoting.

The current toc macros

tag
The root tag that identifies the top level of the tree

sort
An optional extra filter step, e.g. sort[title]

These two parameters are combined into a single filter expression like this: [tag[$tag$]$sort$]

• See how it involves including a sort “filter”?, you may be able “shoe horn” into the $sort$ parameter some kind of conditional alternative sort.
• Needs some filter wizardry.

[Scott_Sauyet]

@TW_Tones

There is a sort parameter in the current Table-of-Contents Macros but I belive this applied to the whole TOC, see below

Yes, that’s for the entire TOC.

@TW_Tones

It is something you may be able to do with @pmario TOC macro rewrites discussed here.

I hope to look at the code more carefully soon, but a casual glance didn’t show it as a possibility so far. I just didn’t know if it could be easily added.

@TW_Tones

It is definitely something that could easily be done with the code pattern above I am promoting.

I will have to read that more closely soon.

Thanks for the response!

[pmario]

@TW_Tones

• For each node in the tree if it contains a condition field use that to determine if that node (and thus its children are to be displayed)
  [. . .]
• A condition could hide a branch of items such as when in read-only mode

I did rename your condition → toc-exclude. That’s it.

• The easiest way to hide a node is set the toc-exclude: yes. It will be hidden

• If you want to hide a branch if the “read-only-mode” tiddler contains yes, you can do this:
  toc-exclude: [{read-only-mode}match[yes]then<currentTiddler>]

If I did understand your 2 points above right, that’s as you described your “condition” field.

@TW_Tones

• For each node in the tree if it contains a child-filter use that not the tags to list the children, then for the children use their condition/child filter or return to the tag method.

• A child-filter could list incomplete todo items in oldest first order

I did rename “child-filter” → toc-include

Using your todo example it will be: [tag[todo]!tag[done]sort[created]]

[pmario]

@Scott_Sauyet

[…] I would love it if, for instance, when I have a list of Person tiddlers nested somewhere in my TOC, I could sort them by last-name then first-name, rather than the default alphabetic sort running in the rest of the TOC.

That’s already possible. Eg: Create the following tiddlers:

• “Contacts sorted by last name” – field: toc-include: [tag[person]sort[last-name]] – tagged: root

• “Contacts sorted by first name” – field: toc-include: [tag[person]sort[first-name]] – tagged: root

• pmario – fields: first-name: Mario, last-name:Pietsch – tagged: person

• Scott_Sauyet – fields: first-name:Scott, last-name: Sauyet – tagged: person

• test-contacts: as shown below

title: test-contacts

<div class="tc-table-of-contents">
<<toc "root">>
</div>

That should do what you suggest.

As you can see, “Contacts sorted…” and “person” tiddlers are not connected by tags. They are indirectly connected by the toc-include filter, which contains [tag[person]]

[TW_Tones]

OK, I think I see where we may be at variance;

• In your proposal we can invoke a TOC macro that includes this additional behaviour.
• In my proposal the standard toc does as it always does, unless it comes across condition/child-filter or equivalent fieldname
  • ideally default or configured fieldnames
  • Thus we need only add such fields to a tiddler that appears in “any toc” for the behaviour to occur.

Another thing I am unclear on @pmario is this is driven by this list and filter as follows;

<$list filter=`[all[shadows+tiddlers]tag<tag>!has[draft.of]$(sort)$] [subfilter<tf.toc-include>] -[<tag>] -[subfilter<exclude>]`>

• Does this not mean when crafting my include I have to craft my exclude to avoid both listing?
• I appreciate the input to this filter is all tiddlers [all[shadows+tiddlers] which means exclusion is most likely in custom filters.
  *Unlike a “condition” field that replaces the filter above for all children, I will have to specifically rule back in tiddlers that are excluded by tag<tag>!has[draft.of]

The following is untested examples of the filter(s) additions I may have here;

The condition test

... :filter[subfilter{!!condition}else[true]]

• if !!cdondition exists use it, otherwise make the first part of the filter result in true.

The child listing;

<$list 
   filter=`[subfilter{!!child-filter}] 
  :else[all[shadows+tiddlers]tag<tag>!has[draft.of]$(sort)$] [subfilter<tf.toc-include>] -[<tag>] -[subfilter<exclude>]`>
...

• if child-filter has a value “on the current node” use it otherwise continue with your proposed filter.

Special note

Unlike your proposal setting the name of the fields tf.toc-include and toc-include I was thinking to provide the field names “condition” and “child-filter” as paramters, to the TOC macro with default set. I could then itterate the same toc using a different set of fieldnames, thus permitting different views of the same content (not unlike your method)

A result what ever is done.

It would be quite easy to update the current tiddlywiki TOC to have simplified and technical views, perhaps avoiding overwhelming new users, but providing a comprehensive TOC for learning or experienced users.

[pmario]

@TW_Tones

This is one reason I favoured a field that replaces the standard toc filter eg [tag<currentTiddler>] with the a “child-filter” for specific nodes (ie not via include and exclude)
*This would allow the easy addition of sort operator, perhaps even sortby.

I think replacing the [all[shadows+tiddlers]tag<tag>!has[draft.of]$(sort)$] section in the toc-filters is against the nature of the TOC-macros. They are primarily tag based, because it’s easy to understand.

With the new toc-include field we only need 1 tag to start the mechanism. eg: root

Contacts → tagged: root … From there on you can entirely go with toc-include fields. As soon as users add an additional tag, the default behaviour kicks in – additionally.

[TW_Tones]

@pmario

They are primarily tag based, because it’s easy to understand.

• I only intend this alternative for specific nodes found in the tag tree, I am not replacing the default behaviour, this is just how we code the filter to do what I am looking for.

I am not currently questioning this tag based toc.

However I have come across many cases where the use of recursion is raised because people are not satisfied with only tag based TOC’s;

• Some of your own work like TocP illustrates this.
• Sometimes they want a field based TOC or use an alternative tag field.

I have a knowledge based wiki containing a lot of plan language tiddler titles and currently use tags to organise them, I can drag the title from the TOC or a seperate index and drop the title onto any tiddler to tag it and thus including it in the toc/index.

• As this gets larger I would prefer to move these into an alternative tags field and retain tags for high level subjects that do not participate in the TOC hierarchy.

This suggests to me - a generic hierarchical list macro

Once a revised TOC is in place, what ever its features, maybe the very next thing to do is to provide tiddlywiki with a “generic hierarchical list macro” that includes many of the features discussed here, but driven by any filter(s) which user chooses.

• Although don’t get me wrong I really want these features in the built in TOC.

[pmario]

@TW_Tones

Sometimes they want a field based TOC or use an alternative tag field.

I could live with a new tagField macro parameter, similar to the one I did implement for the tag-picker macro, which is on halt atm.

I think the more functionality we add to the toc macros, the lower the possibility will be, to get it merged.

[pmario]

I did just push a new version

where all toc-macros should work now. If they do not work it’s probably a bug.

As some may know, I’m not a fan of “hidden information”.
So there are 2 new CSS classes assigned now, with which we can mark parents, that have hidden nodes.

• toc-item-included … is assigned to every element, that comes from toc-include filter
• toc-item-excluded … is assigned to a “parent” node, that has any hidden child-nodes defined by a filter in the parent

There’s some CSS inline STYLE to play with:

<style>
.toc-item-included  > a,
.toc-item-included > .tc-toc-caption
{
  color: green;
}

.toc-item-included > button > svg {
  fill: green;
}

.toc-item-excluded > a,
.toc-item-excluded > .tc-toc-caption
{
  color: red;
}

.toc-item-excluded  > button > svg {
  fill: red;
}
</style>