Winner of the Prize for the Most Interesting Thread by a New User
This has been a fantastic thread to read!
I haven’t built large enough wikis to run into any issues with scaling. But I’m guessing for the most part Node won’t make much of a difference with this. Although there is some Lazy Loading capability; I think that will help mostly with tiddlers that have large bodies, especially images. Other than that, Node wikis still load the entire content in one go. Their benefits have to do with resilience, more discrete chunks, easier saving, and other things. But I don’t think they offer much more in the way of per-wiki capacity.
An alternative might be to work with virtual tiddlers. I can imagine a central list of sentences, and a startup (or build-time) process that splits those sentences into words. Those words are then either actual tiddlers (if you’ve created them) or virtual tiddlers, that when linked to can list all the sentences which include them.
So the virtual sentence tiddler, s14861 might have the content “She sold the cauliflower to him at the market”, might (after removing the words from some stop list) link to “she”, “sold”, “cauliflower”, “him”, and “market”. And although you already have tiddlers for “she”, “him”, and “market”, the words “sold” and “cauliflower” are for the moment virtual tiddlers. “Sold” might link to four or five different sentences, and “cauliflower” only to this one.
This style might let you include a fair bit more content than a tiddler-per-sentence design. Searching will likely be slow when this grows very large, but plain navigation might be ok.
Note well: I’ve never tried anything like this! I may be crazy here.