Serving multiple TWs from a single Node app

#1

I’m wondering if anyone has experience running multiple wikis from a single Node application.

I am starting to work on some personal stuff that I would like to host in the cloud. I want these to be available to me at home and at work, but Giganticorp, where I work, in its infinite wisdom, blocks Tiddlyhost. So I figure that I’m on my own.

Rather than hosting multiple apps in a cloud provider, I was thinking it would be useful to have a single application running to manage my various wikis (Start, Stop, Create, Delete) with each wiki having a dedicated folder, its own git backup, and a regular cadence for synching with git. The main application would handle security and routing to these wikis as well as any administrative tasks.

I would expect to serve, say, the recipes folder as example.com/recipes. This would let me also support static versions of these which others could view, clone, and modify their own copies of: at, say, example.com/recipes.html.

I’m guessing that the existing Node version is far too tied to its single wiki to offer much help here. (Please correct me if I’m wrong!!) But I should be able to start each wiki in its own process and pipe streams from my main application. (I guess I would probably need some way to check if each process is working as expected. Does whatever API TW Node uses between its client and server have a simple heath-check route to report, “Yes, I’m still running fine”?)

Has anyone done this? I’d love to look at existing work before deciding whether to embark on this.

And even if not, does anyone have advice on how to go about this?

#2

If you are using node to host, the simplest and easiest way is to install bob.exe or you can get a plugin version. The default wiki allows you to create multiple wikis below and handle plugins however you wish, the added benefit is tiddler level contention resolution enabling true multi-user edits (with tiddler granularity).

If you find a way to host node like this, on the Internet, with the security I would love to know the steps. So far it seems somewhat in accessible requiring many steps and special knowledge.

1 Like
#3

It looks like I’m trying to reinvent Bob! I thought there must be something. I’ll try it out tomorrow.

Thank you for the pointer.

2 Likes
#4

Though not as recently updated you should also take a look at TiddlyServer just to see if it’s something you would rather work with.

That being said I am a big fan of the Bob plugin.

2 Likes
#5

Thank you. I will. And I may play with my own version, even after looking into both of these, as it sounds like an interesting challenge. (I’d still probably use one of those unless mine had some killer feature otherwise missing.)

#6

Yes Bob is good - the current binary releases are for TW 5.2.3 but it’s easy to make your own Bob.exe with the latest release (currently 5.2.5) following the instructions. Easy but fiddly. I made Linux, macOS and windows versions on a Linux system. I linked to a windows binary which I built in a recent post.

#7

I expect to play with these this evening. I have two features in mind that I would love to have and don’t know if either of these has: versioning, so that I can support multiple semver versions of the same wiki in the same system; and git integration, so that I can back up my wiki in a more granular way than whole-file diffing. But I have about six TW things on my plate at the moment, so even if I decide to roll my own, it won’t be soon.

#8

Not quite sure what you mean by multiple semver versions, but I Bob can generate backups automatically and I also run a cron job to autocommit any changes.

Crucially, it seems pretty stable - running for 4 days on a Windows 2019 server (at work, I hasten to add) without incident.

#9

I imagine serving things like this:

But I might also want to have rules that let me, say, offer only up the latest version of every major version before the current one, and only the latest of each point release of the current major version, and every patch version of the current point release of the current major version. So, for instance, if the current version is 3.4.5, I might keep around 3.4.4, 3.4.3, 3.4.2, 3.4.1, 3.4.0, 3.3.6, 3.2.1, 3.1.8, 3.0.5, 2.4.7, 1.9.3, and 0.12.2, if those are the latest versions in their respective lanes.

(Or, I might not bother with that at all!)

I’d love to automate that as part of the app and not separate jobs.

That sounds like a pretty good runtime for Windows before something goes wrong! :slight_smile: I’m hoping this would be a long-runnning Linux process, where I simply don’t need to think about it for months. But we’ll see. I don’t have a lot of experience building those.

#10

Bob has the tools to export a node site to a html single file wiki. In fact you can do this manually as well with a download that bypasses the server saving mechanism.

  • You could do this when you make major version changes to keep a history of releases with the current always the node wiki.
  • If you had to, you can open any single file wiki and drag a tiddler back to the current one to restore it.
  • This can also be done from the command line so you can automate it there if you wish.

However I would be more inclined to modularise content, appearance and tools and save versions of those, rather than the whole wiki, thus you are set up to reverse to a previous module rather than the whole wiki.

  • Perhaps exporting only new major version
  • Perhaps a different regime for different wikis.
1 Like
#11

I’ve been playing with Bob, and am impressed with what it can do. It seems to be very much the reverse of what I was considering, with its Tiddlywiki UI central, working somehow with a server. But I haven’t dug into the implementation at all, and I may be missing its core idea. My thought was a central server that coordinated the work of multiple Node.js Tiddlywiki servers, each responsible for managing a single wiki. I haven’t dug into TiddlyServer yet, but will play with that soon. Very interesting stuff here!

I have mixed feelings here. For my day job, I’m regularly saving the folder structure of a Node TW back to GitLab. I’m very, very happy with that. For the two plugins I’ve been working on, I do something similar, but also generate and store the current version as the main one as well as in its own static wiki. I really like this output, but am still missing several automation steps, so my workflow is not yet where I’d like it. What I was imagining here is a third approach that also automates the git saving, leaving only a release process to kick off manually. And while I was picturing a trivial UI for managing this, Bob might inspire me to actually write that in TW as well, something I hadn’t even considered before seeing Bob.

For my personal stuff, I would most likely never publish a version, just work with the main branch all along, using granular backups to a git repo. But things like plugins, I would prefer to keep all announced versions live in static homes.

Absolutely. This is a must.

1 Like
#12

I wrote a node app for this long ago but I haven’t maintained it for a long time. Found it was easier to just run multiple node apps each with a unique port.

Any way here for reference: GitHub - sukima/tw-bookcase: A TiddlyWiki management/proxy server

#13

Trying to install as local app (versus global)

PS C:\TW\tw-bookcase>
# npm link
npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm WARN deprecated cross-spawn-async@2.2.5: cross-spawn no longer requires a build toolchain, use it instead
npm WARN prepublish-on-install As of npm@5, `prepublish` scripts are deprecated.
npm WARN prepublish-on-install Use `prepare` for build steps and `prepublishOnly` for upload-only.
npm WARN prepublish-on-install See the deprecation note in `npm help scripts` for more information.

I got some warnings.

#14
  • I will declare I am not a GitHub user (much) and its still another language to me.
  • Incorrect, I would word it as follows

Bob is server that coordinates the serving of multiple wikis under that server, and can manage plugins for those wikis from one place.

  • One server for multiple wikis will be more efficient than multiple
    “tiddlywiki/node servers”.

  • Not withstanding the above, one machine could run multiple instances of bob (different folders, listening on different IP Addresses), which would be multiple servers.

  • You could choose to modify the Bob Master wiki to look like a single TiddlyWiki and hide it’s settings etc… and just forget the subWikis.

As I read it Bob and Bob.exe (Contains all Node.js) is a single server, but node can publish multiple tiddlywikis. The Bob Master wiki in bob is in effect the Root wiki, with “folders used” for subwikis, otherwise in a SubWiki you could be on your own.

  • however in keeping with TiddlyWikis Node server allows plugins to be installed in two ways,
    • One where all wikis and subwikis inherit the plugins in a folder.
    • The Other you can still install by drag and drop/import a plugin in any subwiki but it will remain only within that wiki.
  • The other seriously strong advantage is tiddler level locks.

There are other features available via bob not listed here

1 Like
#15

The download feature of Bob is a major plus for me! Not that node TW lacks this (“save offline snapshot”)

#16

Well, “my thought” here referred to what I was thinking of doing, not what Bob is doing.

Yes, but also with more possibilities of an issue one place causing issues elsewhere. A thin layer wrapped over multiple servers won’t fail if one server has a hiccup. Or at least, I hope it wouldn’t. :wink:

I’ve seen various ways of achieving that. I’ve never seen the advantage. My wikis tend to be fairly discrete, and I wouldn’t want to share much between them but the core, and whatever plugins I choose at that time to install.

I haven’t seen that yet. I’ll check into it.

1 Like
#17

Hey, Scott. Only you can decide for you… but that comment/response makes me think TiddlyServer (link up-thread) is likely the choice you’d prefer.

I’ve never used Bob (never allowed myself the time to wrap my head around it) but I’ve used TS for a few years now. I’m serving node-based and single file-based wikis on n.n.n.n:ports, some separate and some shared. “It just works”.

#18

I’ll look at it soon. Thanks!

That’s what I do for my local development. I’m just looking for a quick self-managed alternative to TiddlyHost, which is blocked by my work. I know TiddlyHost itself is open source, but I think it’s trying to solve a much larger problem than mine.

It’s very difficult these days to install anything from NPM without warnings, unless it is dependency-free, as is Tiddlywiki.

This is a bad situation, as it teaches people to ignore warnings. But I have no idea how it will get fixed.

#19

Just out of curiosity, I’ve retested, and Giganticorp, where I work, has unblocked tiddlyhost.

So I’m personally putting this on hold for a while, as TH will handle my needs… now to go see if there’s a paid tier…

Sorry if I’ve wasted people’s time. I may still come back at this as an interesting project, but not before I get certain other work done.

#20

Nope. Open-ended but worthwhile discussion. :clap: