TW AI - a self-hosted RAG for TW content

I’d like to get feedback on my TW AI project: A self-hosted RAG for TW documentation (including GrokTiddlyWiki, thanks to @sobjornstad having graciously granted his permission).

Of course, it was vibe-coded. (I’m now addicted to Claude Code.)

But the way it’s designed, it should be easy to modify to provide a RAG over any NodeJS-hosted TW wiki. Just search the code for the original websites (example: tiddlywiki.com), and it should be straightforward to plug in the URLs for the wikis you want to include in your RAG.

In this case, it grabs the latest copies of three websites from their public URLs, hosts them all locally with NodeJS, then uses the Node API to run filter queries on them to get the list of tiddlers in each. (I had to add some unsafe options to get this to work with GrokTiddlyWiki’s content, so don’t expose these sites to the world.) Then it pulls the content of each returned tiddler, embedding and indexing each in the vector database. At that point the RAG database is ready for queries.

Running this will require some familiarity with development tools like Docker, npm, and python. And you will need to supply your own OpenAI API key. (These are free to get, easy to “load” with your credit card, and won’t renew automatically. I’m still running on the $20 USD I loaded onto mine back in August.)

I’m very open to feedback. But I was excited (and a bit surprised) to get it working with less than 40 hours total invested in the effort, especially since it’s so easy to adapt to other TW wikis if needed.

Screenshot below.

1 Like