Is there a way to improve performance of many nested transclusions?

Thank you. I’ll try that soon. I was attempting tm-navigate and having issues (which I’m not worried about now.) Is there a clear choice of when to use tm-navigate verus $action-navigate?

tm-navigate is really designed to be issued from JavaScript, where it allows passing on the other navigateFrom* parameters. $action-navigate itself generates a tm-navigate message with the necessary parameters, so is ideal for usage from wikitext.

Got it. Thank you for all the help!

I realized soon after posting that my trouble was not with buttons versus links but with the navigate. I could easily send a notification on click. But I was clearly not using the right parameter for tm-navigate (which I’ll figure out at some other time!)

1 Like

Hm, I would highly disrecommend moving the text out of the text field. By all means, add a notes field, with its own view template… but the text field is so easily transcluded, searched, etc. All sorts of third-party plugins and view templates etc are going to be optimized for the text field… tucking the real content of verses into any other field seems like asking for bewilderment, especially to non–power-user folks. (It’s true that beginners may have a head-scratching moment when they go to edit a chapter tiddler, and find the text field empty… but at least the content of verses is in the text field in your current version, so search tools, etc., will locate text with zero struggle.)

Others may have different ideas. For me, the body template cascade is for the idea that the core presentation of tiddler content might vary according to its type. If you have the thought “chapters should look like this,” then the chapter template should be applied, via a cascade condition, to tiddlers tagged chapter. Then the template itself is a bit leaner and has less noise (and still can be transcluded within the book template, which is already transcluding based on a condition like tag<currentTiddler>tag[Chapter]. So again your template doesn’t have to waste an “if” or list-condition step.

Meanwhile, things tagged $:/tags/ViewTemplate should be for elements that usually should be shown, above/below/beyond the body element of all (or most) tiddlers (or, on any subset of tiddlers, but in a way that does not want to displace the body/primary display element). For example, something like a “notes” template — which might attach to any number of categories or scopes of tiddler content — would surely be best displayed (and perhaps edited live/inline) in a view template (perhaps with minimal visual footprint when empty, until hover shows “add notes” button, or whatever). But having to tell lots of view templates to disappear whenever they fail to match the storyTiddler’s category (each of those templates being framed with with some reveal or list widget, or shortcut conditional) strikes me as brittle and inefficient.

So, the determining question for me is: does this template serve as the “protein and potatoes” for some major category of content in the wiki (especially a tag-driven category), and should it be off the table for direct (non-transcluded) display in other whole categories of tiddlers? If so, use the cascade conditional structure to serve it up.

One further reason for being frugal with view templates appeals to me as a novice tinkerer. When I’m troubleshooting css or using browser inspector, I love having as few logical layers in a tiddler’s storyview “sandwich” as possible. A bit of stray padding or coding error in a view template that is supposed to be hidden can be a headache to track down if every tiddler has many suppressed view templates (for book, and for chapter, and for verse, for tag-node, etc.)

That said, cascades feel too fiddly for something that I’m still “trying on” and developing. I usually develop any template as a viewtemplate… Then I usually move it to the cascade condition (with list-before field as needed) only once I’m done developing. That way, I can even have multiple variations tagged $:/tags/ViewTemplate while tinkering.

2 Likes

It was mostly a passing thought as I wrote the previous post. It seemed like a good idea for at least a few moments.

Sold! You had me at “searched”! :wink:

I think I can handle that with another ViewTemplate above the text entry. You’re right; I need to do something.

Thank you. That gives me a lot to think about. Cascades came later for me than templates, and I’ve really only reached for them when I needed something I couldn’t manage with templates, partly Your approach is probably more sensible. But I do need to think it through.

Indeed. That may be the only reason I think of them as for more advanced cases.

Thank you for the feedback. It’s been incredibly helpful!

1 Like

Thank you

A gigantic thank you to all of you who took the time to help with this. I learned a great deal from all of you, and ended up with a much more efficient implementation.

So thank you @EricShulman, @saqimtiaz, @pmario, @springer, and @CodaCoder! This has been invaluable.

In the new version, I use the basic restructuring Springer suggested, plus the eventcatcher suggestion from Saq and the cached filter from Eric. I noticed speedups with each of those suggestions. All this means that even the larger books—such as Genesis, Isaiah, and Psalms—open quickly enough that I have no current concerns. (Although I would always love further suggestions.)

Again, thank you all very much!

3 Likes

Yes, it’s performing very responsively now! :slight_smile:

Two small changes you might want to consider, though neither is substantial:

In $:/_/bible/templates/book you can simplify the list widget filter to:

[tag[Chapter]] :filter[book<..currentTiddler>] +[nsort[seq]]

In $:/_/bible/templates/chapter you can avoid an extra div around the chapters by giving the one created by the $eventcatcher the appropriate class:

\whitespace trim
<% if [<currentTiddler>tag[Chapter]] %>
<$eventcatcher selector="a" $click=<<open-verse>> tag="div" class="chapter">
<$set name=verses filter="[<currentTiddler>tagging[]tag[Verse]]">
<$list filter="[enlist<verses>get[para]unique[]nsort[]]" variable="thisPara">
  <p><$list filter="[enlist<verses>para<thisPara>sort[seq]]">
    <span class="verse">
      <sup class="verse-number">
        <a class="tc-tiddlylink" data-verse=<<currentTiddler>> href=`#$(currentTiddler)`
>{{!!verse}}</a>
      </sup>&nbsp;<span class="text">{{!!text}}&ensp;</span>
    </span>
  </$list></p>
</$list>
</$set>
</$eventcatcher>
<% endif %>
1 Like

Oh, of course. Thank you. I tend to expand these during debugging and too often forget to compress them.

Thank you. I didn’t realize that, and it’s nice because I was a bit bothered by having a <div> wrapped in a <span>.

Both fixes are in the latest (still unpublished) version.

– Scott

1 Like

It seems to me there are two issues with many nested transclusions,

  • There is the rendering of the transcluded content
  • There is the listing via filters to choose the order and items to render, typicaly nested.

When you stop to think about it, the filters are a design convenience, and help new content to be categorised, however once complete they are perhaps unnecessary processing to run each time.

  • Lets face it once complete the bible should have a fixed content and structure.

Perhaps an approach could be designed so rather than sophisticated filters you just iterate static lists of tiddlers, if the static list is present iterate it, if the static list is not present then use the full filter.

  • You could create a batch process to generate and store the static lists, and it may only need to be run once.
  • If you have notes and annotations keep them out of this process
1 Like

Hmm, I may have been so focused on how that I didn’t really stop and consider whether!

This is a really good point.

I tend to work with a large number of data tiddlers, the sort of design exemplified by #10120. And with such a design, the trick is getting the basic tiddlers right, so that I can combine them in all sorts of interesting ways. But here the combinations are simple: The Bible is made of Books, Books are made of Chapters, and Chapters are made of Verses. There are some slight complexities about paragraphs and some odd additional handling for Psalms. But all of it really is static.

I think you’re probably right, and I certainly can perform the static generation. Nonetheless, I feel uncomfortable with doing so. I need to figure out why. I will probably first proceed with what I’m doing, as I believe there is only one significant step remaining. But then I may well continue on from there to try your approach. One thing I wanted to do anyway was to do a separate conversion of the verse-based data I’ve been starting with into a simple nested JSON structure without all the repetition. That would mesh nicely with this approach.

Thank you very much for your response and your insight. I expect I will try this at at some point.

There may be value in developing a generic approach to this, as I suggested a filter that

  • looks for the static representation and uses it if available
  • If not available then iterates the necessary list using the logical filter (the time consuming one) offer to save it in a static list.
  • If the filter static list exists, in an authorship mode, click to forcibly iterate the filter and regenerate the static list (If new content was added/deleted, and log the changes if any) this data can be used for new “in this release” lists.

I am just hinting here at a deeper but generic solution for saving iterated lists, into static lists.

1 Like

That’s only partially true. It’s nice to have the possibility to match the different elements in any way you need to. Usually paragraphs are used in different contexts. So if there is one book it’s hard to reference the right places.

Since we have a granularity of verses / tiddlers it’s easy to move “connect” them to different contexts and interpretations.

I’m not certain that this is how Tony meant it, but to me this would not involve full renderings of everything. I would keep the verse tiddlers, but I might have collections tiddler that look like this:

title: King James Bible
tags: TableOfContents

<<book Genesis>>
<<book Exodus>>
<<book Leviticus>>
<!-- ... -->
title: 1 John
tags: Book

<<chapter "1 John 1">>
<<chapter "1 John 2">>
<<chapter "1 John 3">>
<<chapter "1 John 4">>
<<chapter "1 John 5">>
title: 1 John 4
tags: Chapter

<p><<verse "1 John 4:1">><<verse "1 John 4:2">><<verse "1 John 4:3">></p>
<p><<verse "1 John 4:4">><<verse "1 John 4:5">><<verse "1 John 4:6">></p>
<p><<verse "1 John 4:7">><<verse "1 John 4:8">><<verse "1 John 4:9"><<verse "1 John 4:10"><<verse "1 John 4:11"><<verse "1 John 4:12"><<verse "1 John 4:13">></p>
<!-- ... -->

quite possibly simplifying the <<book>>,<<chapter>>, and <<verse>> names to <<b>>, <<c>>, and <<v>>.

This would not store the renderings anywhere but would store the results of the current filtering. This would not prevent someone from writing other filters to combine things in interesting ways.

It does strike me as a very useful alternate way of thinking about this. But it’s a bit alien to how I’ve been doing things lately, and I’m not yet sure where the tradeoffs are.

In The Sense of Style, Pinker argues/claims that there is no such thing as a paragraph – only the paragraph break.

Without disputing the claim outright, I remain optimistic but (so far) unconvinced the distinction is useful. Yet, since reading it, it has stuck in my brain for whatever reason. I do wonder whether, in coding and/or text processing, it might be useful for something… one day.

:man_shrugging:

Search: pinker no such thing as a paragraph.

I think I disagree. But he’s bright enough and interesting enough that I’ll give it more thought. And I just put the book on my to-buy list.

Grammatically, I think of a paragraph as an exploration of a single idea over the course of one to a handful of consecutive sentences. From a web-dev perspective, a paragraph is a visual grouping of consecutive sentences, which can be targeted as a group for display or behavioral changes.

But I am interested to read his thoughts. I especially would like to see if similar logic would say the same for sentence/sentence break, chapter/chapter break, etc.

I away this is true that we don’t use a paragraph symbol, two new lines results in a logical paragraph and the parser wraps it in p tags.

So paragraphs are a a set of sentences that are wrapped together. Of course with CSS you can style such paragraphs.

Afterthought! Looking at the docs for a totally different reason, spurred by an old comment by @etardiff, I realized that I want to propose a variant that is even more semantically elegant:

"[enlist<verses>each[para]get[para]nsort[]]"

I repeatedly forget about the each operator! It bundles in the unique function without (yet) having to drill down to focus on the field values themselves… So it can be useful when (unlike your task here, @Scott_Sauyet) you don’t want to actually get the field values, but to gather the first within each kind/value. So it would be fantastic for producing, say, a list of the first few words of each paragraph.

Oh that is nice. I haven’t used each before. I’d glanced at it and thought it equivalent to JS’s forEach, which I mostly try to avoid. But it’s quite different. I can see several interesting uses for this.

Thank you.

1 Like