Macro performance issue

#1

Hey all,

I was hoping to get some advice on a performance issue I’m having in my Kansas Railroads wiki.

I have a tiddler that displays in a tab in the sidebar that shows a list of the predecessors of certain railroads (see included code). The Predecessors of Railroads is the one that is in the sidebar which calls the New TOC macro. It has a select widget which selects which railroad to show the predecessors for. The select widget is followed by a table that shows the current status of what is selected ($:/state/fieldforstatus is simply a subfilter to check if the current railroad has the matching status or not - note: I don’t think this is where the problem lies because I use this identically in a flat list of railroads with no issue).

If this sidebar tab with Predecessors of Railroads is opened for the first time, it can take up to 1.2 seconds for displaying a list of only 126 tiddlers. The refresh takes on the order of 920 milliseconds. This seems horribly excessive to me for such a small list of tiddlers.

I’ve thought about using suggestions from previous posts concerning throttling refresh, but this seems much more fundamental than a refresh issue for a large number of tiddlers (126 is not large) and that something in the code could be done vastly more efficiently.

I’m suspecting that at least part of the issue is my use of the kin filter in declaring a couple of variables. I am using the kin filter to determine the railroads that have the current railroad as a tag. I probably could replace the kin filter with tagging[] here instead. Would that help performance in and of itself? I haven’t actually experimented witht this yet because I just thought of it as I was composing this message.

I’m also wondering if the toc-new macro has some inefficiencies in it.

Thanks for any advance. It’s all welcome.

New TOC macro.tid (2.9 KB)
Predecessors of Railroads.tid (4.7 KB)

#2

Not sure if it’s a good enough test, but the Predecessors of Railroads displayed instantly on tiddlywiki.com

I don’t use TOCs so I don’t know how to test it.

#3

Yeah unfortunately that’s to be expected when not populated with the 1800+ railroads in my wiki (back to the copyright issues I’ve mentioned before).

I need to try to see if I can generate an example with enough tiddlers to demonstrate the issue without trampling on any copyright concerns.

#4

Is it public?  

#5

No because much of the content of the full wiki is copyright protected.

I will see about creating a version with just enough content to demonstrate the issue and then post it on Tiddlyhost.

#6

@CodaCoder I know his railroads was not public. It is quite a large Database which is valuable Intellectual Property.

@HistoryBuff perhaps you do as we did once and enlist a trusted party with deep TiddlyWiki skills to look with you. Perhaps you could even provide remote control through a trusted link, so someone can explore a little and investigate with you watching and talking.

  • Perhaps you could just share the code, in the list, with a little explanation.

Some notes;

  • The final list size may not represent the lists used to retrieve the final list. Commonly for example we use all[tiddler] or it is implied. But perhaps you can alter the order of the filter to eliminate more titles first.
  • Sometimes writing a fresh list and filter rather than using already existing macros may be a way to contain the complexity of what you are asking. Test it in another tiddler to see if its quicker.
  • Look at any filters in the process and check they are using starting or limiting filters as documented in Performance and other notes there in.
  • If lists are not too big, but you need access to it more than once it may be better to save a working list in a variable, and reuse that variable, such as with the $set filter then the filter is evaluated once.
  • The Kin operator is very powerful and featured perhaps you can search for and use TocP or tagstree or taggingtree (More recently became available) which are relatively more limited functionally and most likely faster.
  • If you are sourcing this list by walking a tree this is a great example of when to saae the result in a variable, list field or tiddler and only refresh when the tree content is modified.
  • Try a really basic list with little or no additional information in it and see if that works better. If it improves, it is likely an issue in the display, not the listing.
  • If the list does not change much or often, perhaps use my snapshot tool (to html) to capture the result and just view the result, ie: not re evaluate each time #1
  • look for the performance tools they may be able to indicate where the processes are using more time (I now you can, not how)

#1 I may need to adapt this to work for something in the sidebar, but a work around is open the tiddler, snapshot it, hide it from the sidebar, create a new sidebar tab and transclude the shapshot tiddler eg {{$:/snapshot/original title}}

#7

Someone else holds the copyright? Fair enough.

Yep. That’ll help.

#8

It seems, you use the kin filter, which is known to cause performance problems.

Did you try to create your lists without it?

#9

As I said @pmario but then you don’t read long posts.

#10

All,

I have uploaded a limited wiki to my tiddlyhost site. You can find it here: Kansas Railroads.

Feel free to play around with it.

I haven’t had a chance to take in some of the suggestions in the previous posts. However, I did make a huge improvement in performance just by removing the status bar logic at the top of the predecessors list. This is what uses the kin operator. Removing that cut out 600-800 milliseconds. I would like to try to operators tagstree or taggingtree that @TW_Tones suggested instead of the kin operator.

Please let me know if you have any further questions or comments.

#11

Before I set off on the complete wrong track…

Am I meant to be looking at this?

image
image582×593 24.6 KB

Initial findings:

  1. You have a glut of plugins installed.

  2. Everything in your UI is presenting quite lengthy lists. The lists are full of links (HTML <a> elements). That’s a ton of wiring the browser needs to do just to show you the UI. That can certainly be optimized by using $eventcatcher.

#12

Yes - you are looking at exactly the right thing. Thanks very much for the initial input.

I agree, I have a lot of plugins. I’ve been meaning to go through them and remove some that I can do without. It’s one of those things on the ever growing to do list.

I’ll have to investigate the $eventcatcher. I’ve not looked at that before. I will say that I’m okay generally with the performance once it initally loads. The predecessors selection is by far the slowest (which, as I said previously has a lot to do with the kin operator). Having said that, you’re right in that there are certainly better ways to do things and that’s kind of what I’m looking for. I am not a coder and it probably shows.

#13

Yes, but the whole UX is sluggish, to say the least. Here’s a quick stab at the $eventcatcher improvement…

\procedure do-rr-link()
<$action-navigate $to=<<dom-rr-href>>/>
\end

<$eventcatcher tag="div" selector=".rr-link" $click=<<do-rr-link>>>
<$list filter="[tag[Railroads]]" variable=rr>
 <div class="rr-link tc-tiddlylink tc-tiddlylink-resolves" rr-href=<<rr>> title=<<rr>>><<rr>></div>
</$list>
</$eventcatcher>

I think you’ll notice how much faster that is to list all [tag[Railroads]].

1 Like
#14

RR-TEST.json (625 Bytes)

#15

Thanks! Gave that a quick try. My goodness, that does make a difference. I’m going to have to study that a bit and then apply that to the many lists that I do have.

#16

Knew you’d like it. That’s over 1600 links removed and 1 link replaces them all. If you look in FF devtools at your links, each one has a event attached to it ← they take an enormous amount of time (relatively speaking) to produce. If you look at one of “mine”, scroll up, you’ll eventually find one event handler attached to the eventcatcher div.

The key thing you need to grasp is how your link attributes are used and handled inside the procedure (or macro). All the attributes you create are turned into variables prefixed by dom-.

So…

<some-tag attr-1="one" attr-two="two" ...>

becomes…

<<dom-attr-1>>
<<dom-attr-2>>

in the procedure.

The example at the bottom of https://tiddlywiki.com/#EventCatcherWidget should help get you going.

#17

Concentrate first on the lists you keep visible in the UI – Sidebar, definitely.

Then the lengthier ones that become visible when someone clicks on something “innocent looking”.

Fair warning: I can’t help you with the TOC thing. Never used it. If you hit a stumbling block there, all I can say is, try the same approach. If they are just <a ....> elements, the same overall structure should work.

Maybe someone reading this has already optimized the toc stuff to work with $eventcatcher?

#18

So the first thing I did today was to replace the kin operator with taggingtree. That alone easily cut the delay down by 75%. (This was all in that progress bar at the top of the list.)

Now I’m going to explore the $eventcatcher and convert to that.

Thanks @CodaCoder and @TW_Tones

#19

@CodaCoder

One question about the div element within the eventcatcher widget. Below part of the code you posted:

<div class="rr-link tc-tiddlylink tc-tiddlylink-resolves" rr-href=<<rr>> title=<<rr>>><<rr>></div>

I see that the actual tiddler that is navigated to is rr as defined by rr-href=<> and dom-rr-href in the action-navigate widget. I am not clear on how this works as I haven’t seen this kind of thing within a div tag before. Is this basically a custom attribute to the div tag? Also, what is the purpose of the title attribute within the div tag? From my googling, I thought custom attributes had to start with data-*.

I’m hoping the above aren’t stupid questions, but this is going a tad beyond my basic understanding.

#20

Yes, which is what I tried to draw your attention to above. You can call your attributes anything you like (though you should try to precede them all with data-, but I promise you, the HTML police aren’t looking!)

And… I could have left the link as an <a> tag… I wanted to make the point that you can link via any tag that makes sense. <div> made sense to me since it’s automatically, sans CSS, a block element and I wanted one link per line. Make sense?

You can add as many attrs as makes sense to you. For example, you could add founded-by="George Porgie" and founded-year=1895 if it makes sense to have them there and if you can think of a reason to have them available inside the click handler. So just remember, TW/eventcatcher will add dom- to each of them, even your class attribute (if you have one) becomes dom-class in the click handler.

Focusing on the example I gave you, I chose rr for Railroad. I could have left href as just href but it’s sometimes a good idea to make clear (even to yourself) that something different is going on when next you look at the code (which might be 6 months from now).

title attribute: The div is now behaving as a link. All good links have tooltips. In HTML, that’s usually the job of the title attribute. Necessary? Your call.

Tips:

  1. Use the Inspector in devtools and look at your $eventcatcher div. Just for clarity.

  2. Use <$action-log/> in your click handler and look at your dom-* variables. Again, just for clarity:

\procedure do-rr-link()
<$action-log $$message=show-me-everything-dude $$all=yes />
<$action-navigate $to=<<dom-rr-href>>/>
\end

Nonsense. None of us were born knowing this stuff. To you, right now, this is a new pattern. Follow the two tips above and it’ll all make sense. Promise.