Tiddlywiki Connection with Database

Until your browser will crash. There are wikis out there, with 30k+ tiddlers.

But – It depends, what you want to achieve?

TiddlyWiki was designed to be a self-contained wiki. So it’s 1 HTML file, that contains all the data. So if you want to store text-based content it will be fine. If you want to store binary data it’s probably the wrong tool.

1 Like

In my case data is not constant, it will vary from the options I select and data can be updated by someone else. So data is constant at all.
For instance , if I did the same now. But there is a chance that someone might have changed the data. So I need the recent data rather than the older one.

I don’t have full control over the Database, I am just using it from a 3rd party.
Since the data is private can’t expose into API calls from tiddly.

Really appreciate the effort.
Currently I’m running a nodejs as a middleware to get the queries and retrieve the data.

I have some binary and hex coded data even more complex data rather than text. I will keep the data in db rather than storing in Tiddlywiki itself.

OK. So you use a Node.js server to retrieve data from the private database and create tiddlers with that data, so TW can understand it – right?

OK So TW shows some meta data about the binary data and provides some links to the “real” data if needed. – right?

1 Like

Yeah… exactly. The same you explained. I was checking whether tiddlywiki can handle some powerful queries inbuilt , then it would be amazing.

I retrive the data and converting to json , so that TW can easily understand. Data in the DB is pretty complex and it’s a graphDb.

Really appreciate for the wonderful insight.
Great explanation.

OK. I think that’s the right approach. The TW server has a possibility to “lazy-load” the tiddler body “content”. So the title and all fields are transferred to the browser. So you can efficiently filter your lists.

As soon as a tiddler is show, the content will be dynamically loaded from the TW server. This can speed up the initial load time significantly.

@linonetwo has a huge wiki that works that way. May be he can jump in here?

1 Like

I’m just online, so the link is 50M large / huge size of TiddlyWiki, performance remains good. Also a HugeWiki plugin for developers

Import it to NodeJS will show it works smoothly. But my own wiki’s UI might be slow due to some slow filters like Too slow! · Issue #1 · bimlas/tw5-kin-filter · GitHub . Not using them will make wiki fast.

1 Like

Thank you . I got more from those answers.

I used the filter and had some logic in the nodejs to reduce the loading issue. Till now UI works perfectly,

I am following this sqlite3 WebAssembly & JavaScript Documentation Index and I hope that by the time I learn the basics of the filter language, somebody who is much smarter than me will release an sqlite TiddlyWiki plugin which will provide an alternative to said filter language for managing tiddlers :smiley: Albeit I have no idea about the size inflation rate of a single file wiki that would bundle such a plugin.

1 Like

Yeah …I started to learn about Database using MySQL . Now I am using some powerful graphDB rather than normal RelationalDB.

There is a SQLite Node.js backend plugin in the works, which should also allow multi user access.

It does not change the internal data structure in the browser. It adds a new “syncadaptor” to the plugin.

@pmario
I have a scenario such that, tm-http-request shows CORS issue, but when I try the endpoint with postman it’s working ? Does Tiddlywiki have any method to fix the CORS?

Not necessarily. It depends on the GraphDB. Depending on the API that is available there it should be possible to directly access data from within TW.

There is the tm-http-request message, that will allow us to send requests directly to external DBs which provide an API, that returns JSON data.

On the other hand allowing direct access to a private database may be problematic from a privacy perspective.

So also here it depends on your privacy requirements.

Currently API endpoint doesn’t have much features yet. And mostly I’m encountering CORS issue

CORS settings have to be adjusted on the server side. So you probably have to go with a Node.js “middleware”, since you know that works.

How does the GraphDB api look like?

The API currently supports only GET , and they created some custom apis to retrieve some documents.

I even tried with bypass proxy , still it was showing the CORS from tiddlyside.
Currently implemented nodejs middleware to bypass the CORS issue and it will act as a proxy.