Is there a way to improve performance of many nested transclusions?

If a <$list> is producing more than a couple dozen links (or $buttons - anything that uses event wireup), I use $eventcatcher.

Every.

Time.

Ah, that’s because I was just jumping straight in with a view template for testing, bypassing the <% if %> framing. Actually, in the big picture I’d still bypass the conditional shortcut framing around the edges and bypass the tag[Chapter] start to the list filter, using just a cascade condition (rather than $:/tags/ViewTemplate) to target the chapter tiddlers. And the book template is already applying this transcluded bit only to chapter-tiddlers.

Note that replacing <currentTiddler>tagging[] with tag<currentTiddler> at the beginning of a filter run should be faster as the latter is indexed:

Whichever of <currentTiddler>tagging[] (or tag<currentTiddler>) and tag[Verse] is expected to return fewer results should be at the start of the filter run.

Agreed.

2 Likes

Oh my, I had assumed that <currentTiddler>tagging[] would somehow draw on the same tag-indexing efficiency as tag<currentTiddler>. Learning stuff here!

You know what? I just looked at the code and it does. Even if our documentation on performance does not state so, the underlying code path is indeed the same! Thank you for prompting me to look at that.

2 Likes

Not that I necessarily object, but could one of you explain why? I don’t yet have any real intuitive understanding of the dis/advantages of the two techniques.

But if anything, I would have thought of moving in the opposite direction, adding additional ViewTemplates; these templates would appear underneath the default body template – and moving the verse content from text to a custom field. This would allow users to claim the text field for their own annotations at all levels: Verse, Chapter, and Book.

That makes sense, and it’s definitely the former. Chapters are structured collection of subsets of the Verses.

1 Like

I’ve done it with buttons. I don’t know why it seems harder with links. But I’m sure I’ll figure it out.

Something along these lines:

\procedure clickactions()
<$action-navigate $to=<<dom-data-target-tiddler>>/>
\end

<$eventcatcher selector=".mylinks" $click=<<clickactions>> >
<a class="mylinks" data-target-tiddler="this is what I want to link to">my text</a>
</$eventcatcher>

Thank you. I’ll try that soon. I was attempting tm-navigate and having issues (which I’m not worried about now.) Is there a clear choice of when to use tm-navigate verus $action-navigate?

tm-navigate is really designed to be issued from JavaScript, where it allows passing on the other navigateFrom* parameters. $action-navigate itself generates a tm-navigate message with the necessary parameters, so is ideal for usage from wikitext.

Got it. Thank you for all the help!

I realized soon after posting that my trouble was not with buttons versus links but with the navigate. I could easily send a notification on click. But I was clearly not using the right parameter for tm-navigate (which I’ll figure out at some other time!)

1 Like

Hm, I would highly disrecommend moving the text out of the text field. By all means, add a notes field, with its own view template… but the text field is so easily transcluded, searched, etc. All sorts of third-party plugins and view templates etc are going to be optimized for the text field… tucking the real content of verses into any other field seems like asking for bewilderment, especially to non–power-user folks. (It’s true that beginners may have a head-scratching moment when they go to edit a chapter tiddler, and find the text field empty… but at least the content of verses is in the text field in your current version, so search tools, etc., will locate text with zero struggle.)

Others may have different ideas. For me, the body template cascade is for the idea that the core presentation of tiddler content might vary according to its type. If you have the thought “chapters should look like this,” then the chapter template should be applied, via a cascade condition, to tiddlers tagged chapter. Then the template itself is a bit leaner and has less noise (and still can be transcluded within the book template, which is already transcluding based on a condition like tag<currentTiddler>tag[Chapter]. So again your template doesn’t have to waste an “if” or list-condition step.

Meanwhile, things tagged $:/tags/ViewTemplate should be for elements that usually should be shown, above/below/beyond the body element of all (or most) tiddlers (or, on any subset of tiddlers, but in a way that does not want to displace the body/primary display element). For example, something like a “notes” template — which might attach to any number of categories or scopes of tiddler content — would surely be best displayed (and perhaps edited live/inline) in a view template (perhaps with minimal visual footprint when empty, until hover shows “add notes” button, or whatever). But having to tell lots of view templates to disappear whenever they fail to match the storyTiddler’s category (each of those templates being framed with with some reveal or list widget, or shortcut conditional) strikes me as brittle and inefficient.

So, the determining question for me is: does this template serve as the “protein and potatoes” for some major category of content in the wiki (especially a tag-driven category), and should it be off the table for direct (non-transcluded) display in other whole categories of tiddlers? If so, use the cascade conditional structure to serve it up.

One further reason for being frugal with view templates appeals to me as a novice tinkerer. When I’m troubleshooting css or using browser inspector, I love having as few logical layers in a tiddler’s storyview “sandwich” as possible. A bit of stray padding or coding error in a view template that is supposed to be hidden can be a headache to track down if every tiddler has many suppressed view templates (for book, and for chapter, and for verse, for tag-node, etc.)

That said, cascades feel too fiddly for something that I’m still “trying on” and developing. I usually develop any template as a viewtemplate… Then I usually move it to the cascade condition (with list-before field as needed) only once I’m done developing. That way, I can even have multiple variations tagged $:/tags/ViewTemplate while tinkering.

2 Likes

It was mostly a passing thought as I wrote the previous post. It seemed like a good idea for at least a few moments.

Sold! You had me at “searched”! :wink:

I think I can handle that with another ViewTemplate above the text entry. You’re right; I need to do something.

Thank you. That gives me a lot to think about. Cascades came later for me than templates, and I’ve really only reached for them when I needed something I couldn’t manage with templates, partly Your approach is probably more sensible. But I do need to think it through.

Indeed. That may be the only reason I think of them as for more advanced cases.

Thank you for the feedback. It’s been incredibly helpful!

1 Like

Thank you

A gigantic thank you to all of you who took the time to help with this. I learned a great deal from all of you, and ended up with a much more efficient implementation.

So thank you @EricShulman, @saqimtiaz, @pmario, @springer, and @CodaCoder! This has been invaluable.

In the new version, I use the basic restructuring Springer suggested, plus the eventcatcher suggestion from Saq and the cached filter from Eric. I noticed speedups with each of those suggestions. All this means that even the larger books—such as Genesis, Isaiah, and Psalms—open quickly enough that I have no current concerns. (Although I would always love further suggestions.)

Again, thank you all very much!

3 Likes

Yes, it’s performing very responsively now! :slight_smile:

Two small changes you might want to consider, though neither is substantial:

In $:/_/bible/templates/book you can simplify the list widget filter to:

[tag[Chapter]] :filter[book<..currentTiddler>] +[nsort[seq]]

In $:/_/bible/templates/chapter you can avoid an extra div around the chapters by giving the one created by the $eventcatcher the appropriate class:

\whitespace trim
<% if [<currentTiddler>tag[Chapter]] %>
<$eventcatcher selector="a" $click=<<open-verse>> tag="div" class="chapter">
<$set name=verses filter="[<currentTiddler>tagging[]tag[Verse]]">
<$list filter="[enlist<verses>get[para]unique[]nsort[]]" variable="thisPara">
  <p><$list filter="[enlist<verses>para<thisPara>sort[seq]]">
    <span class="verse">
      <sup class="verse-number">
        <a class="tc-tiddlylink" data-verse=<<currentTiddler>> href=`#$(currentTiddler)`
>{{!!verse}}</a>
      </sup>&nbsp;<span class="text">{{!!text}}&ensp;</span>
    </span>
  </$list></p>
</$list>
</$set>
</$eventcatcher>
<% endif %>
1 Like

Oh, of course. Thank you. I tend to expand these during debugging and too often forget to compress them.

Thank you. I didn’t realize that, and it’s nice because I was a bit bothered by having a <div> wrapped in a <span>.

Both fixes are in the latest (still unpublished) version.

– Scott

1 Like

It seems to me there are two issues with many nested transclusions,

  • There is the rendering of the transcluded content
  • There is the listing via filters to choose the order and items to render, typicaly nested.

When you stop to think about it, the filters are a design convenience, and help new content to be categorised, however once complete they are perhaps unnecessary processing to run each time.

  • Lets face it once complete the bible should have a fixed content and structure.

Perhaps an approach could be designed so rather than sophisticated filters you just iterate static lists of tiddlers, if the static list is present iterate it, if the static list is not present then use the full filter.

  • You could create a batch process to generate and store the static lists, and it may only need to be run once.
  • If you have notes and annotations keep them out of this process
1 Like

Hmm, I may have been so focused on how that I didn’t really stop and consider whether!

This is a really good point.

I tend to work with a large number of data tiddlers, the sort of design exemplified by #10120. And with such a design, the trick is getting the basic tiddlers right, so that I can combine them in all sorts of interesting ways. But here the combinations are simple: The Bible is made of Books, Books are made of Chapters, and Chapters are made of Verses. There are some slight complexities about paragraphs and some odd additional handling for Psalms. But all of it really is static.

I think you’re probably right, and I certainly can perform the static generation. Nonetheless, I feel uncomfortable with doing so. I need to figure out why. I will probably first proceed with what I’m doing, as I believe there is only one significant step remaining. But then I may well continue on from there to try your approach. One thing I wanted to do anyway was to do a separate conversion of the verse-based data I’ve been starting with into a simple nested JSON structure without all the repetition. That would mesh nicely with this approach.

Thank you very much for your response and your insight. I expect I will try this at at some point.

There may be value in developing a generic approach to this, as I suggested a filter that

  • looks for the static representation and uses it if available
  • If not available then iterates the necessary list using the logical filter (the time consuming one) offer to save it in a static list.
  • If the filter static list exists, in an authorship mode, click to forcibly iterate the filter and regenerate the static list (If new content was added/deleted, and log the changes if any) this data can be used for new “in this release” lists.

I am just hinting here at a deeper but generic solution for saving iterated lists, into static lists.

1 Like