Typing frozen for 45 seconds when changing title, huge number of Relink orphans

#1

@Flibbles Thank you so much for Relink, It is a crucial part of the Tiddlywiki experience.

1) However, recently when I start typing in the title field to change a title, my typing is frozen for 45 seconds before the wiki allows me to resume typing. Strangely, as long as I don’t save it, the problem doesn’t happen again if I change the title again, but once I save the tiddler the problem will recur if I open the tiddler and try to change the title again.

2) While investigating I discovered I had 11398 “Relink orphans”. Is this normal? (My wiki has 13,637 tiddlers.) Did I mess something up when I updated my Tiddlywiki? Do I need to do anything about the orphans?

  1. Is it normal to have 4905 “Relink Missing”?

Background:

After updating my Tiddlywiki (to the one just before 5-3-3), I started noticing weird things happening with Relink. (So sorry I can not remember what they were. I vaguely seem to remember that maybe the function disappeared completely for a while and I finally noticed that titles I was editing were not getting relinked.)

So I checked my Relink plugin and discovered it was not the latest version 2.4.2.

Around the same time I also saw a warning on Tiddlywiki Talk to update Tiddlywiki again because of a problem with Relink.

So I updated my Tiddlywiki again, this time to 5-3-3, and I also updated my Relink plugin to 2.4.2 by going to the Relink website and drag and dropping. I believe things worked fine for a short while after that, but not sure because I was very busy off and on these past two months.

However now, as mentioned above, every time I start typing in the title field to change a title, the wiki freezes up for 45 seconds before it lets me continue typing.

#2

I would have suggested you made sure it is the latest version of the plugin?, 5.3.x has problems unless you use the latest version.

But since you have, perhaps there are some legacy configuration or overridden shadows from the earlier version.

Use a fresh empty version with the new plugin and see if you can reproduce it.

#3

Relink has no problem with the empty version.

Maybe it has something to do with my thousands of Relink orphans? I remember I used to have a similar problem typing in the tag field until I was told to set something to not search until I had typed my 4th character or so.

Or maybe I messed up my wiki in that inbetween stage when I upgraded to 5.3.2 (?) but hadn’t yet upgraded the Relink or upgraded the wiki further to the fix in 5.3.3 either

#4

That’s because of Relink’s caching. In a browser instance, Relink has an indexer which will cache all connections that exist in your wiki, so that it can quickly determine references later. This cache is in memory, so when you close and reopen your wiki, it’ll need to replenish.

Unfortunately, populating the cache has to be expensive. There may be ways to improve it, but utlimately Relink has to go through every tiddler looking for links so it can know what it has to change when you update titles.

It’s easy for something to be classified as an orphan. It just means something doesn’t reference it directly. But if for instance, a tiddler has a tag, and something utilizes the tiddler through that tag rather than directly referencing it, it’s still considered an orphan. You should notice that you have less Relink orphans than you do garden variety orphans.

That’s a lot. Without seeing your setup, I can’t say why. If you click on the title in the “missing” list, it’ll give you an idea where it’s being referenced. If you’re using Relink-titles and you don’t have the latest versions of everything, then it’s probably considering all uninstantiated root directories of your tiddlers “missing”. Like, if you have path/to/my/tiddler. Then path, path/to, and path/to/my are all considered missing if they don’t exist. The latest Relink version is better about this.

Glad to hear it. I work hard to make sure Relink is robust, but the ground keeps shifting underneath it with every release. Issues keep slipping in as Tiddlywiki progresses.

Yeah… Tiddlywiki in general isn’t very efficient for wikis with 10,000+ tiddlers. I’ve considered some solutions like having Relink populate its index in the background at startup, but this kind of multithreaded solution will drive me straight to bug city, so I’ve been holding off for the time being.

#5

Okay hold on. Just thought about this. 45 seconds? That’s… a little much. For 13,000 tiddlers, I’d expect about 5 seconds on my 10-year-old laptop. There is something weird about it. Are you able to share more details? How slow is Tiddlywiki in general? What are most of those tiddlers? Is your computer paleolithic? I don’t suppose you’re able to share your wiki with me?

#6

@Flibbles Before upgrading, my relink was instant (as opposed to 45 seconds). I had just as many tiddlers then, too (13576)

What I mean by instant is, I just now opened up my last tiddlywiki from before upgrading and started typing to change a title and instantly the relink notification appears below the title field. I didn’t even have time to start the stopwatch. I don’t have to wait 45 seconds for the relink notification to kick in and appear so that I can safely save my tiddler.

In fact I can’t do anything else at all with my wiki until the 45 seconds are up. The whole wiki is frozen until the Relink notification appears.

As for how slow my wiki is in general, here are some stats comparing the speed of my wiki between 5.1.22 (the old) and the new 5.3.1 (I upgraded again to 5.3.3 probably a week later but didn’t do any comparisons then):

opening wiki
old 10 sec 69 ms
new 12 sec 03 ms

opening first tiddler of session - a new blank tiddler
old 3:27
new 1:74

saving the first tiddler of the session - that previous new blank tiddler
old 58:79
new 51:11

second save of the session (after typing hello in the previous blank tiddler)
old 9:07
new 7:04

opening another tiddler (a journal index tiddler)
old 2:36
new 6:14

after typing hello and saving and closing that second tiddler
old 9:30
new 10:62

#7

If you disable relink and or other plugins, save and reload does the performance problem go?

This may also eliminate a relinking configuration setting that is introducing the delay. If relink off not problem, relink on a problem you can be more confident. But you can do the same with all plugins to see if the fault is caused by one of them.

1 Like
#8

I too would like to know if there’s a difference, because there’s clearly a problem, and if it’s Relink, I must fix it. These delays are egregious.

#9

@Flibbles I disabled Relink, reloaded the wiki, and the problem went away: On typing in the title field, the wiki did not freeze, the “update tags and lists of other tiddlers” notification instantly appeared, I could immediately click the save and close button.

I then re-enabled Relink, re-loaded the wiki, and the problem is back: On typing in the title field the wiki freezes for 45 seconds before the relink notification appears.

I don’t know if this means anything but I looked in the cotents tab of the Relink plugin. Every item in the list of shadow tiddlers is bold except for this one: $:/plugins/flibbles/relink/Filters/Orphans

#10

Perhaps copy that tiddler rename and remove tags or module type. The delete the overwritten tiddler and see if that fixes it

#11

@TW_Tones, Resetting the Orphan’s filter shouldn’t really make a difference to the load.

@Sapphireslinger, when you say the problem was fixed, do you mean everything was snappy fast then, or did you still have 5-10 second delays with basic operations like opening tiddlers? Was the 45-second one-time delay the only thing that disappeared?

#12

@Flibbles The 45 second one-time delay on typing to change a title was the only thing that disappeared when disabling Relink.

I ran some more tests. It may not have any bearing but I discovered that :

Opening the first tiddler of the session

  • If it is a new blank tiddler it opens fast 2:20
  • But If the first tiddler to be opened is an already existing tiddler it opens slow 45:65. After that, saving open tiddlers is fast, and opening new tiddlers of either type after that is fast also…
  • However, if the first tiddler of the session is a new blank tiddler and I do not open any more tiddlers but save that new one, it will save slow (49 seconds). After that hurdle however, opening an already existing tiddler will be fast and opening and saving tiddlers of either type will be fast.
  • But if the first tiddler of the session is a new blank tiddler (opens fast) and then I leave it open and go open an already existing tiddler, the already existing tiddler will open slow. Then I return and save the first tiddler and it saves fast instead of slow, and opening and saving tiddlers is fast after that.

However none of that solved the main problem (the change-an-existing-title-delay), that remains constant.

Some simple “Relink Enabled” versus “Relink Disabled” stats.

(A slight difference of a few seconds should not mean anything because I’m working with my smartphone’s stopwatch and it is harder to hit the button just right, than with a real stopwatch.)

opening wiki:
Relink Enabled: 6:38
Relink Disabled: 6:43

opening first tiddler of session - if a new blank tiddler :
Relink Enabled: 1:74
Relink Disabled: 2:20

saving first tiddler of the session - if that same new blank tiddler
Relink Enabled: 48:51
Relink Disabled: 49:65

opening second tiddler of session (an already existing one):
Relink Enabled: 1:72
Relink Disabled: 1:66

saving second tiddler of session (an already existing one):
Relink Enabled: 5:70
Relink Disabled: 5:81

A bit of another scenario:

opening first tiddler of the session - if already existing tiddler
Relink Enabled: 45:37
Relink Disabled: 45:65

saving first tiddler of the session - if that same already existing tiddler
Relink Enabled: 5:87
Relink Disabled: 5:94

More Stats:

I am using Linux Mint, and Firefox.

#13

I see. Wow. Yeah, it looks like Relink is only responsible for that first 45 second delay, specifically concerning changing a character in a tiddler’s title (which I could only mitigate by pre-caching in the background or something). Everything else is probably within the margin of error. Which means, in general, Tiddlywiki is just super slow for you. I honestly don’t know how you’re tolerating it.

It’s been well known that Tiddlywiki becomes super sluggish once you start cresting 10,000 tiddlers. We’re a club of people all in the same helpless boat. I would love for the Tiddlywiki core team to really take performance seriously. I would be the first to jump and start profiling code and tackling big offenders.

#14

A lot of us (core team and super users) have in the past, and one reason this is somewhat vexed is tiddlywiki gives you enough rope to hang yourself. If you shortened the rope to stop people hanging themself, you would loose a lot of functionality.

I encourage people with performance problems to simp[ly ask for help, ideally with a link to an example, because there is a number of “low hanging fruit” and a lot of performance improvement methods.

  • Experience shows if someone makes an example to share they often solve their problem while doing so and if not someone else solves it quickly.

This is quality in many areas of software, for example SQL allows you to ask stupid questions that cause serious performance issues because it is crontrary to the structure of the data, it will still do it for you, but you should not have asked :nerd_face:

#15

You’re right. I may be being overly harsh here. I’m really only speaking from my experience where getting performance PRs merged takes months of convincing.

When I say taking performance seriously, I mean taking on some serious tasks, such as underlying changes to the way widget hierarchies are handled, or adopting lazy filter operators to reduce memory thrashing. Big fruit, Not low hanging. But they’re things I and a few others have been pushing for for a while.

2 Likes
#16

@Flibbles and @TW_Tones I think the Relink delay problem has been partially solved.

I didn’t think it was the large number of tiddlers that was causing the Relink problem because before I updated to 5.3.3 I had the same number of tiddlers and no Relink problem.

And I didn’t know how to begin investigating which of my modifications to my wiki was clashing with the update to cause the problem because I hadn’t kept track over the years and didn’t know where to find them.

With my own wiki so messed up, I thought it was as good a time as any to try switching over to a completely different edition, the beautiful Timimi site.

But asking the question how to import all my non-system tiddlers into another tiddlywiki CleverNote (the 1.0.49 version at the Timimi site) got me thinking again - Why not try that with the standard empty.html to solve this problem here?

Previously I only knew how to update my tiddlywiki the standard way - dragging it onto the update page. It never occurred to me that It was possible to download an empty.html first and then export all my non-system tiddlers over to it, thus getting rid of all my modifications at once.

Once I realized there was this alternative way to update, I searched in this forum for exporting non-system tiddlers and this came up: Simplest way to port tiddlers to a new TiddlyWiki where Telumire says:

You can be more selective by exporting your tiddlers with the advanced search, you could write

[all[tiddlers]!is[system]]

for example to only get non system tiddlers, then export in json. To import the tiddlers in the new wiki, drag and drop the json.

I had had no idea all these years that was what one did with jsons - drag and drop them like plugins. I remember one time laboriously opening up a json and trying to recreate it in my wiki ^o^ >_< (Or make a tiddler and put the json contents inside or something.) If I ever did know at one time, I must have promptly forgotten it.

Now I have this broad idea that a json is just code for a bunch of tiddlers, like a plugin. And @TW_Tones, now I can understand more of your tips here.

So did this alternative way of updating and sloughing off all mods solve the Relink problem?

Well so far, now I only have the Relink problem on the first tiddler of the session. After that, relinking is snappy! I can live with that!

Now I will import all the rest of the plugins and mods again as I feel or discover that I need them and update you on any changes.

Thank you both for all the help.

1 Like
#17

@Flibbles and @TW_Tones I’m so sorry I was mistaken, and gave incorrect information in my last post. Starting with an empty tiddlywiki and porting over only non-system tiddlers, did not correct the problem at all, not even temporarily.

I thought I was running those tests with the Relink plugin installed, when in fact Relink was NOT installed. (After drag and dropping the Relink plugin, I had clicked the “save” and “reload” buttons up in the notification bar instead of clicking the “save” button down in the Tiddlywiki and closing out the tab. So the relink plugin hadn’t gotten installed.)

After truly installing Relink

Today I installed Relink properly, and this time made sure it was showing up in the plugins before running the tests. The long delay remains constant.

I then deleted Relink and ran the tests again. This time there was only a long delay on the first rename, subsequent renames are fast.

Additional Information

My tiddlywiki used to be 37 mb. After transferring all my non-system tiddlers over to an empty, the size is only 32.4 mb.

Sorry I can’t give you a copy to delve into and fix, it’s a personal wiki.

#18

I don’t know how much of a difference it will make, but I recently pushed a new version of Relink which does better about caching. It probably won’t fix your problem, but it does resolve a caching issue that was slowing things down.

1 Like
#19

You are right, there is still the delay.

#20

@Flibbles

  1. I would like to return to using my snappy older version of Relink (1.9.1) Are there any really bad bugs I need to look out for if using it with Tiddlywiki 5-3-3?

I saw the mention of a bug here. I could be careful about not having other stuff open when relinking.

  1. Where can I find the Relink versions between 1.9.1 and 2.4.2? I would like to test to see if there is one later than 1.9.1 that doesn’t cause the long delay.

Why I Ask

I was combing through all the past backups of my Tiddlywiki to see exactly when the problem emerged. (I opened a copy of each backup and tried to relink something to see how long it took, and took note of the Relink version.)

My Tiddlywiki was upgraded to 5.3.3 by December 25th. And I continued to use Relink 1.9.1 until sometime around February 29 when I upgraded to Relink 2.4.2. The backup labeled 2024-02-29 is where I first see Relink 2.4.2 and a long delay. The backup before that labeled 2024-02-27 has Relink 1.9.1 and no delay.

  • With Tiddlywiki 5.1.22 Relink 1.9.1 was lightning fast.
  • With Tiddlywiki 5.3.3 Relink 1.9.1 had maybe a half-a-second delay. Still very fast.
  • With Tiddlywiki 5.3.3 and Relink 2.4.2 the 45-second delay emerges.

So is there any reason I should not go back to Relink 1.9.1?