How to disable all lists update on every click to make Tiddlywiki work fast with 10000+ tiddlers?

1

My Tiddlywiki on-line stores become big, and $list in them are very long too.
Long ago Bj wrote plugin for me to disable all $list refresh. here it is Интернет-магазин Heeg.html загружается..... — Heeg.html
but this plugin don’t work in new TW5 so big stores with 10000+ items work slow
for example Интернет-магазин Heeg.html загружается..... — Heeg.html

may be somebody knows how this plugin works an can write new version for 5.1.2
to disable refresh (or make refresh all lists on special button click) all $list and {{{[…[…]]}}}

2

@Siniy-Kit I have posted for you a quick modification of a widget of mine to selectively control refresh via a subfilter, and to never refresh if there is no filter specified: https://gist.github.com/saqimtiaz/e9611a5dd37ac247fc16aa3d4dfb3e23

Usage Examples:
To never refresh a list:

<$refresh>
<$list filter="[!is[system]count[]]"/>
</$refresh>

To refresh only when the subfilter returns a match in the list of tiddlers that have changed, in this case when tiddler xyz changes:

<$refresh subfilter="[match[xyz]]">
<$list filter="[!is[system]count[]]"/>
</$refresh>
3

Thank you, Saqimtiaz. But i don’t understand, how to use it. I put your code to my TW here

but it didn’t start

4

@Siniy-Kit you need to give the tiddler with the JavaScript that I posted the field “module-type” with value “widget”

5

OK. Now we can test
https://heeg.ru/shop2_3_refresh.html?id=1Q9VrNSsCznmyi6y9B0Ht94rBGrUc7HHQPZjK-8H_ZOQ#Petli
https://heeg.ru/shop2_3.html?id=1Q9VrNSsCznmyi6y9B0Ht94rBGrUc7HHQPZjK-8H_ZOQ#Petli

it works a little faster with “refresh off”, but not very fast. How can we see what processes make Tiddlywiki work slow? How can I make JavaScript speed test?

6

It is best to do in chrome

7

You should consider an alternative to using the zoomin storyview. With the zoomin storyview, when you navigate to a new tiddler, the previous tiddlers are still rendered and refreshed.

8

@Siniy-Kit another approach is to take a “html” snapshot of a tiddler in html and display that rather than rendering the whole tiddlers and lists therein, again, each time the tiddler is opened.

Attached is my snapshot tool that illustrates this, but expect you may need to test and customise for your use. Please try on tiddlywiki.com before installing in your wiki.

snapshot.json (6.2 KB)

9

Thank you Tones. I’ll test it. I think my problem is not many lists refresh. The main problem is LONG lists. For example here is my experiment
many short lists Интернет-магазин Heeg.html загружается..... — Heeg.html

and the same TW but long list open Интернет-магазин Heeg.html загружается..... — Heeg.html

10

@Siniy-Kit when I follow your second link I only see 4 columns with short lists Information/Catalogue/Our address and Call us. So I can see a long list with a view to using the shapshot tool.

This snapshot approach will not stop all list updates, only those in a tiddler (or part there of) with a snapshot.

  • I assume this is a single file wiki, not node?

I think there would be value getting some metrics to determine if what you ask for is an actual performance bottleneck before looking for a solution.

I am not sure if this approach is going to help you, however I have a hunch it can.

A few thoughts;

  • The snapshot (by the Wiki Author) takes a copy the the HTML rendered for a particular tiddler and stores it in another tiddler. When viewing the original tiddler the “pre-rendered” HTML is displayed rather than its body, this means the list widget need not be rendered at “run time”.
    • Theoretically this allows you to pre-render and save the result for visitors, however the tiddler view will/may not respond to changes unless another snapshot is generated.
  • In the case of an online shop I am not sure how a html snapshot will work with interactive products eg add to cart component. However it works a bit like a static html tiddler while still being inside the tiddlywiki.
  • In TiddlyWiki 5.2.x the new cascade mechanism for the view templates may help implement this is a smart way. The html snapshot can be rendered without any macros and other code be included and render time.

Ideas;

  • The concept of pre-rendering tiddlers makes use of the standard rendering through the body template. However it is possible to have an alternate body template designed to support pre-rendering of tiddlers in particular ways, this may include pulling content such as pre-rendered lists but include other interactive content.
    • The question is will this help meet your objectives to enhance performance?
    • Perhaps a mechanism to transclude html snapshot’s as parts of a tiddler makes sense?
  • Pre-rendering could actually pull html from the file system, which would allow it to be externalised and cached separately whilst existing within the interactive tiddlywiki.
  • I can imagine the idea of navigating to snapshot tiddlers by default with only the designer being in a position to rerender the tiddlers snapshot using the wikitext and $list widgets etc… If a tiddler need not be re-rendered on the live site the source tiddler could be removed from the published wiki.

I hope I am helping, not confusing you, but it seems this is an area that has a number of innovative possibilities.

11

second link has a mistake. here is write link Интернет-магазин Heeg.html загружается..... — Heeg.html

is it possible to make automatically “html” snapshot of all !is[system] tiddlers “on open” my TW?

12

@Siniy-Kit I can drop my package on your site and it seems ready to work, but as a rule it takes a snapshot of the result of rendering the body, on the link you showed me most of the content is rendered via some other method such as the view template.

I think it could be modified to capture the body and result after applying the view template, but I admit I struggle with the other language on the site.

Here is another package of two buttons I use to open a new window with the full rendering, which demonstrates what could be done with the snapshot tool, by opening the tiddler in a window including with the view template applied.

new-windows.json (13.3 KB)

Again this can be dropped on your site.

I imagine the answer is yes, but I need to know more about your solution. And I suspect there may be better ways.

  • Is this a single-file wiki or node?
  • does the user only have read-only access?
  • do you only update it when you open it?
  • How does a users order get saved/dispatched?
  • What evidence do you have that it is the list widget causing slowness?, I can’t see any slowing!
13

Unfortunately snapshot is not working on my site
here demo with snapshot Обувь и всё для обуви — Уход за обувью
and here is snapshot tiddler Обувь и всё для обуви — Уход за обувью

it is empty, because my tiddlers text field is emty. And all data to it is put by template Обувь и всё для обуви — Уход за обувью
with special tag $:/tags/ViewTemplate
so all content is not in tiddler-body and there is nothing to snapshot…

and if I put my data to text field of ONE tiddler my tiddlywiki file become 2mb large, and works slow too Обувь и всё для обуви — Уход за обувью

and snapshot is not empty Обувь и всё для обуви — Уход за обувью
but very big. So , our problem is big DOM tree?

14

As I said;

Thats why I suggested this;

However I asked

See if you can make use of my code, and ideas and feel free to direct any questions to me however I can’t help much more right now due to;

  • The use of other language’s on your site (Its hard for me to follow/explore)
  • Not enough info from you (See questions asked you have not yet answered)
  • Personally busy.
15

I’m not in a position to try this right now, but I was wondering if using the select widget for selecting a tiddler from a large list would have any better performance than selecting from a displayed list. Just curious.

16

I managed to do a little testing today with a displayed list versus the select widget and found no significant performance difference. I was expecting this to some degree since the same list is being generated. I was curious to see if the fact that the list wasn’t always displayed in the select widget played a performance role. Answered.

17

I just wanted to point out people are using the new event catcher widget to reduce the complexity of the HTML generated using the list widget. Apparently it has performance advantages.

Perhaps this could help?

Also consider reworking the code so when you click on a list item it updates something that is not visible and does not cause a need to refresh the tiddler containing the list. For example add the item to the cart, but don’t indicate it is in the cart in the original list.

18

@Siniy-Kit what part of the site are you experiencing performance problems with:

  1. How long it takes to show a long list the first time?
  2. How long it takes when you click on something in the list to show the item?
  3. How long it takes to go forwards and back in the long list?
19

Attached is a very quick experiment at implementing a new layout for performance testing purposes that avoids the need for the zoomin view, it also removes the sidebar. Please try this on your site and report back if it helps with performance.

  • Import all the tiddlers
  • Ctrl+Shift+L and choose the single tiddler layout.

You could also try combining this with the refresh disabling widget, setting a filter such that the refresh is only disabled when single tiddler layout is active.

single-tiddler-layout.json (2.6 KB)

The next thing to try would be to simplify the templates used to render the product details.

20

Thank you, Saqimtiaz. I’ll Try it.

Here I make experiment with 2500 tiddler list

and it takes 4-5 seconds to open Recent tiddlers on my 10 year old computer and low price android smartphone and 1 second on (not my) new computer.

Absolutely the same result I see on long list in on-line stores people make on my tiddlywiki template Интернет-магазин Heeg.html загружается..... — Heeg.html my computer opens list on this link 6 seconds (to many)

So I make my experiment. I render static cut of tiddler with long list
Косметика для обуви it opens 1 cecond on my old computer…
and tiddlywiki variant Интернет-магазин Heeg.html загружается..... — Heeg.html

9-10 seconds full page load