Running multiple wikis on node.js with shared tiddlers

The following describes a setup to run multiple individual wikis that share one or more sets of tiddlers between them. In most use-cases the shared tiddlers would be config tiddlers, template tiddlers or, more generally, customized system tiddlers that would be common for two or more wikis.

The basic file structure is assumed as follows

/
├── wiki1/
│   ├── tiddlywiki.info
│   └── tiddlers/
│       └── shared/
│           └── tiddlywiki.files
├── wiki2/
│   ├── tiddlywiki.info
│   └── tiddlers/
│       └── shared/
│           └── tiddlywiki.files
├── wiki3/
│   ├── tiddlywiki.info
│   └── tiddlers/
│       └── shared/
│           └── tiddlywiki.files
└── shared/
    └── tiddlers/

The tiddlywiki.info files are the basic configuration files, loading plugins, etc. as for any node.js wiki.

tiddlywiki.files

The tiddlywiki.files files tell TW to load tiddlers from somewhere else (see tiddlywiki.files Files. For the case described above they all look identical:

{
    "directories": [
        {
            "path": "../../../shared/tiddlers",
            "isTiddlerFile": true,
            "isEditableFile": true,
            "searchSubdirectories": true,
            "fields": {
                "source": "shared"
            }
        }
    ]
}

Note that for each tiddler that is loaded from the shared folder, a field source is added with the value shared to help with identifying the tiddler source from within the wiki. At this point it has no other function. The value can be set to anything, e.g. to distinguish multiple shared sources (we might call them bags).

For any tiddlers that should be shared between wikis, the corresponding files are put into the respective shared folder (and should be deleted from the folders of all wikis that load them from the shared source.
Multiple shared folders can be set up this way, and for each a subfolder within the tiddlers folder of the wiki must be created with a corresponding tiddlywiki.files file. These subfolders are a recipe of what bags to load for each wiki.

Propagating updates

If shared tiddlers are edited in one wiki, or if files are moved manually into/from the shared folder, tiddlywiki must be restarted in node.js and the wiki refreshed in the browser to propagate changes.

The same holds true when wikis are synchronized between multiple machines to propagate the changes to all of them. However, compared to single-file wikis, sync conflicts usually only affect a small number of tiddlers / files and can usually be identified easily.

Using tiddlyserver

tiddlyserver can be used to serve multiple node.js wikis simultaneously and to also serve static files like images and fonts (and single-file wikis).
The config file settings.json in the root folder of the folder structure shown above could look like this:

{
  "tree": {
    "wiki1": "./wiki1",
    "wiki2": "./wiki2",
    "wiki3": "./wiki3",
    "media":"./media"
  },
  "_datafolderserver":"C:/Users/user/AppData/Roaming/npm/node_modules/tiddlywiki",
  "bindInfo": {
    "bindAddress": ["127.0.0.1"],
    "port": 8080,
    "filterBindAddress": true,
  },
  "$schema": "./settings-2-2.schema.json"
}

The _datafolderserver tells tiddlyserver where to find the tiddlywiki core files.

The command line to serve all wikis is then

tiddlyserver --config .\settings.json

The URL for wiki1 would then be http://127.0.0.1:8080/wiki1, and so forth.

Using $:/config/FileSystemPaths

When using $:/config/FileSystemPaths to create a folder hierarchy corresponding to tiddler namespaces (see Customising Tiddler File Naming) like $:/core, care must be taken that files from are not automatically saved back to the local wiki folder. This is where the source field can help.

[is[draft]search-replace:g:regexp[/|\\],[_]addprefix[drafts/]]
[!field:source[shared]removeprefix[$:/yaisog]addprefix[system/yaisog/]]
[!field:source[shared]removeprefix[$:/core]addprefix[system/core/]]
[!field:source[shared]removeprefix[$:/config]addprefix[system/config/]]

The rules in $:/config/FileSystemPaths would cause loaded shared tiddlers to be saved according to any matching rules. Adding the source field check avoids these rules to match and thus the tiddler is saved back to the shared directory.

New files will be saved to the local wiki folder, which would be the topmost bag. Tiddlers in that folder override any loaded shared tiddlers.

5 Likes

Thanks @Yaisog

I’m having trouble locating tiddlyserver.json. The only similar file is settings.json mentioned here. Which is also the only place where _datafolderserver is mentioned.

Are you working with a different set of docs?

Hi @CodaCoder, the config file can have any name when you pass it via the --config parameter to tiddlyserver, as was done above. If you keep the filename “settings.json” you probably don’t need to use the parameter. I did rename it to get better clarity of the root folder contents, “settings.json” seemed too generic.

To avoid more confusion on this already confusing topic, I have changed the filename in the instructions above to “settings.json”. Thanks for pointing it out.

1 Like

Thanks for sharing, I was using ln -s soft link to link folders together, now I think using tiddlywiki.files might be a better option.

"fields": {
                "source": "shared"
            }

will this field be saved to .tid file? What happened if you modify or delete this field in the wiki? And seems it is only be used to classify some system tiddlers, so maybe I could not using this field like you do?

Hi @linonetwo, I don’t know more about the field handling than what is written at tiddlywiki.files Files. The field seems to be written back to the .tid files. The documentation says that it’s an override, so your own modifications might be lost.

However, it seems possible to add a prefix to the saved value of the field instead, so your own value would not be overwritten. Also, you don’t have to use source as field name. Actually, you don’t have to add any custom fields automatically, unless you intend to make use of it, e.g. in $:/config/FileSystemPaths.

The field is added to any tiddlers loaded from the specified location. These can be any tiddlers, not just system tiddlers. Furthermore, you don’t have to use a static string as above, but you can populate the field with a number of automatically generated values. This is all described in the doc.

2 Likes

Thanks @Yaisog, slowly, the fog is starting to clear.

I’ve always installed my TiddlyServers as “portable” – node.exe executable stored locally in each server folder.

The assumption below, I believe, is for non-portable, global (npm ... -g) installations:

 "_datafolderserver":"C:/Users/user/AppData/Roaming/npm/node_modules/tiddlywiki",

What should I set _datafolderserver to in my case? I fancy it can be left out completely and that @Arlen22 is looking first for a tiddlywiki folder in the server folder which is where I place it.

Aside: it might be a good idea to point out (perhaps under the heading Using tiddlyserver) that TiddlyServer supports delivery of single-file TiddlyWikis as well as wikis where the tiddlers are stored as .tid files.

Once again, thanks for doing this :clap: :heart:

As far as I understand it, it’s an optional property. If you leave it out, tiddlyserver will use a version that comes bundled with it (but was very outdated when I installed it). You set it to the directory that npm installed tiddlywiki in. npm list or npm list -g should tell you where that is (depending on if you used the global switch during install).