Speed comparison of data tiddler lookup

Out of curiosity I did a speed comparison for various ways to access data in DataTiddlers. This is the result as measured with Maurycy’s Advanced Performance Plugin:

Filter Total time Uses Longest run Average time Median time
[<jsondata>jsonget[807810887407]] 6942.00ms 1000 17.00ms 6.94ms 6.00ms
[{New Tiddler 4}jsonget[807810887407]] 6935.00ms 1000 27.00ms 6.93ms 6.00ms
[[New Tiddler 3]getindex[807810887407]] 62.00ms 1000 1.00ms 0.06ms 0.00ms
[[New Tiddler 4]getindex[807810887407]] 15.00ms 1000 1.00ms 0.01ms 0.00ms

jsonget[] is comparatively slow for simple name:value DataTiddlers. The faster getindex[] is 4× slower for application/x-tiddler-dictionary than for application/json, both being in the microseconds per lookup.


Here’s how it was measured:
Tiddler New Tiddler 3 is a dictionary tiddler with 18075 entries of random numbers. The last lines look like this:

[...]
133007463997:142745435157
176656710595:430569764937
133014656883:396363251674
474946751459:374715567685
807810887407:631723026204

Tiddler New Tiddler 4 is a JSON Tiddler with the same data in JSON format. The last lines look like this:

	[...]
	"133007463997": "142745435157",
	"176656710595": "430569764937",
	"133014656883": "396363251674",
	"474946751459": "374715567685",
	"807810887407": "631723026204"
}

In another tiddler I ran a lookup of the last data point (with the name 807810887407) 1000 times:

<$let jsondata={{New Tiddler 4}}>
<$list filter="[range[1000]]">
<$let a={{{ [[New Tiddler 3]getindex[807810887407]] }}}
		  b={{{ [[New Tiddler 4]getindex[807810887407]] }}}
			c={{{ [{New Tiddler 4}jsonget[807810887407]] }}}
			d={{{ [<jsondata>jsonget[807810887407]] }}}>
</$let>
</$list>
</$let>

and measured its render time.
Just in case that the elapsed time for jsonget[] was due to loading the text data from the JSON tiddler, I also assigned it to a variable and read it from there. There was not much of a difference, though, see the table above.

I have no idea if the getindex[] lookups are cached and therefore faster when always looking up the same key.

Also, I explicitly don’t want to make jsonget[] look bad or anything – it’s much more powerful than getindex[] after all. For simple data tiddlers, I’ll stick with getindex[]. Not that I have many of those left, since Jeremy wants to deprecate them and all…

Have a nice day
Yaisog

1 Like

The current incarnation of the JSON filter operators work on arbitrary text rather than tiddlers and as such there is no caching, which is present for data tiddlers, and accounts for the difference in performance. There is a longer term goal for how the JSON operators can be made more performance, but it is helpful to explore practical real world use cases before devoting further effort in that distraction.

Apologies for the brevity: I am on the road for a few weeks.