Part of the problem is before you read the “chicken to the egg” instructions perhaps instead you need to choose if its a “Bantum to the egg” problem.
With hosting on the internet I would first look at your available hosting, can you do this with single file wikis. Is tiddlyhost enough, use TW-Receiver if you have a php site. Can you securly host node on the Internet? Once you decide how you are hosting then you need the detailed instructions.
Thanks. Truly. I’m about to carry on here, please don’t take this the wrong way.
Frequently people who post (or partial post) assume you know about docker, nginix, etc.
“docker”, now. I hadn’t found that to be presumed. I know what it is, but, what’s special about using it in this context? Metaphorical question… unless.
When I go searching, what I find is posts from, oh, 2017 or so. That makes me think I must have missed something; surely by now, it’s been done enough times that the then-new posts are out of date (thinks I), and practice has given birth to colloquialisms! Which is why my un-grokked googling… ho ho, ha ha.
In this context, do docker and monit overlap, for instance? Should I use both? FWIW, my project is much smaller than many, and my repetition tool of choice is Salt. Does that imply overlap, somehow? No idea and I don’t expect anybody else to do the work of answering that, unless.
Would it be enough, for example, as the absolute minimum, to have nginx holding the SSL keys and treating TiddlyWiki as a hosted app? My guess is no, there’s some further piece (node.js?), but I’m not sure why, and obviously I don’t yet understand why other tools (docker, monit, etc.) are necessary in this context.
Blame Apache. For all things. Praise be.
Currently there’s a relevant discussion in the top of the GG : https://groups.google.com/g/tiddlywiki/c/hJftdxHMQ6I
So that might help.
Yes! Thanks again. But again I gripe; I’m still reading there, and the question I still have is, what are the abstract pieces we need to do this? TiddlyWiki is an application, so we need some web server to host it, nginx is preferred. HTTPS is (way) better than HTTP, nginx does that. Would that by itself be enough for one’s first go-round?
With hosting on the internet I would first look at your available hosting, can you do this with single file wikis. Is tiddlyhost enough, use TW-Receiver if you have a php site.
Currently, PHP isn’t in the mix at all. TiddlyWiki is a demonstration of what HTML5 can do, all by itself. (Right?)
Can you securly host node on the Internet?
Yeah. I myself am root.
Once you decide how you are hosting then you need the detailed instructions.
Sort of. What I need is an abstract guideline. All I know for sure is that I want to provide two-way communication between TiddlyWiki and users via HTTPS. TiddlyWiki has its own account system, so that’s handy – Curly can be the boss, Moe and Shemp have accounts. (Correct me if that’s wrong.) Now I just want to make that pass over HTTPS instead of in-house on one server.
It’s not needed to go with docker, if you don’t want to start many of them. The nice part though is, that it’s easy to create “throw away” containers for testing, without the need to install all the TW related stuff at the main system.
On the other hand docker has to be installed on the main system 
…
If you use Salt as your orchestration system, you will be able to use it to orchestrate docker. So Salt imo is a layer above it. I would search for docker at: Salt Table of Contents if you consider it.
It’s possible to use monit with docker, but docker or docker-compose should handle restarting crashed containers already.
I think so.
You can use nginx in a basic configuration as a file server to host “read only” single file wikis over HTTPS … As Tony mentioned, with a bit of PHP it would be possible to save them back.
You can also use nginx and WebDav to save back single file wikis. TiddlyWiki has a WebDav saver built in. The WebDav saver uses ETags to check if the wiki can be saved back. Handling ETags can be a bit ticky, if you enable server side compression.
So for the first run I would send the wiki uncompressed until I’m sure that everything works. And then experiment with server side compression, to minimize data transfer.
We – the devs – use nodejs to build TiddlyWiki. There is a development server, that can also be used to save single tiddlers back to the filesystem. This mechanism is know as “TiddlyWiki nodejs server”. It’s perfectly fine to use it on your private network. BUT as soon as you create a web-facing version you need to take care about server security.
If you consider to use it, you should configure nginx as a reverse proxy, that handles all the web requests, the certificate stuff and serving additional files.
As I wrote a relatively easy way would be to use nginx as a WebDav server and single file wikis which can be saved back to directories. If you have additional assets like images, nginx can handle them fine as a file-server.
hope that makes sense
Mario
PS: I assume, you know, that on a web server all incoming ports must be closed, except those you need for your app. AND The app needs authentication and authorisation.
PPS:
As we did some development with the WebDav saver, I did use a docker image for local testing, because it’s easy to “throw away” when I don’t need it anymore. See: https://github.com/ionelmc/docker-webdav … The version I used was older. I didn’t test the latest one yet. … I wouldn’t use this for production, but it can show you a working nginx WebDav configuration.
PPPS:
Additional info about WebDav here at Talk: Use WebDAV for saving a single file wiki
Hi Michael,
It is relatively easy to use TiddlyWiki through a remote HTTPS service, but it is important to distinguish the two following kinds of service, since TiddlyWiki has two different natures:
In its standalone file avatar, the standard way to serve a modifiable version of a TiddlyWiki file is through the WebDAV protocol: as Mark suggested, we recently had a thread on the Google group on this type of service. Many people in the community use WebDAV to serve their wikis so you should find more by searching the Google group.There is also a wiki document by Saq on using WebDAV with TW.
In the “Node.js” avatar, which is inherently modifiable, TiddlyWiki exists simultaneously in two forms: a web page in your browser, and a collection of files on the server host, maintained by a JavaScript application —tiddlywiki.js— fired by the Node.js server program. The code that resides in your browser discusses with tiddlywiki.js at the other end, in order to synchronise your local changes. Note that it currently doesn’t sync the other way round.
While the first form obviously needs an external server program (like rclone or nginx or apache) to be served remotely, the NodeJS form of TiddlyWiki is a full service by itself that can natively handle HTTPS. You can install it on your server host with npm install -g tiddlywiki. It doesn’t require any external server in order to do what you want. You’ll only need to first setup credentials as described here: https://tiddlywiki.com/#WebServer%20Authorization then create the HTTPS certificate and key files as described here: https://tiddlywiki.com/#Using%20HTTPS. Then you can run on your server host a command like:
tiddlywiki mywikifolder --listen “host=0.0.0.0:8998” “credentials=myusers.csv” “readers=(authenticated)” writers=mary “tls-cert=~/tls/server.crt” “tls-key=~/tls/key.pem”
That should be enough to serve TiddlyWiki on your local network through HTTPS with a reasonable level of security. If your local server host has the IP 192.168.1.3, then the wiki will be reachable at https://192.168.1.3:8998 . Note that the 0.0.0.0 notation is a convention to say that connections are accepted beyond the local host. This is not an actual IP address of any host anywhere.
At this stage, I should probably mention a variant of the Node.js tiddlywiki server that is called TiddlyServer. It has the advantage of being able to serve both the standalone and the Node.js form.
If you want to access your brand new service from outside your local network, you’ll need to connect to your gateway router and set a NAT rule that says that a request to a certain port of your gateway should be redirected to the local server host and port. For instance, if your gateway has the global address 185.199.108.153, and you set a rule that says that a request to 8997 should be redirected to your local server host (192.168.1.3) and port 8998, any connection to https://185.199.108.153:8997 from outside your network will get to your wiki.
Now you see the problem in the phrase any connection. If it works for you, it will work for thousands of bots cruising the Internet in search of an open door to get in. Most of them will give up after a few attempts, but some will insist; I don’t think there is a mechanism in TW that closes the connection after too many attempts. One thing to make sure of is to never launch tiddlywiki as root. If a bot gets in, at least it won’t be able to invade your system.
Another level of protection is to use a proxy HTTP(S) server. Typically using an Apache or NGINX server configured to further redirect connections to a running tiddlywiki. Note that this is a completely different setup than serving a standalone file as mentioned above. These very common HTTP(S) servers can also be configured to reject connections based on a variety of criteria. One of the advantages of these widespread tools is that Let’s Encrypt tools know how to automatically edit their configuration files and upgrade them to HTTPS, certified by Let’s Encrypt. With such a certificate, your browser won’t complain anymore that your service is self-signed.
It certainly would be nice to have a full HowTo for that proxying process applied to tiddlywiki, but I’m afraid it doesn’t exist yet. Maybe because opening a certified Node.js TiddlyWiki access outside a local network doesn’t really make sense before TiddlyWiki becomes natively multi-user? If you want to explore this route, have a look at the Bob variant.
–Xavier.
Depending on the exact setup you’d like there is a tutorial here that covers starting TiddlyWiki with nodejs, watching and auto starting the process with pm2, putting that all behind nginx as a proxy, and securing with Let’s Encrypt. You can find the tutorial here: How to: TiddlyWiki on NodeJS + nginx proxy + LetsEncrypt
I’d always vote for this option, because those servers also are much better to serve other file assets in a performant way. The TW server wasn’t designed with security in mind.
Hi mdeabreu,
Depending on the exact setup you’d like there is a tutorial here that covers starting TiddlyWiki with nodejs, watching and auto starting the process with pm2, putting that all behind nginx as a proxy, and securing with Let’s Encrypt. You can find the tutorial here: How to: TiddlyWiki on NodeJS + nginx proxy + LetsEncrypt
Oh I didn’t know this HowTo. It is really helpful indeed, thank you!
–Xavier.
I’ve been slow on this end, clearly have reading to do, probably will come back with questions, but: thanks much, all. This is tremendously helpful.
Resurrecting this after a few weeks.
Having read what’s been posted here (thanks, all), it turns out that I was already doing most of what’s being described, but I’ve hit a snag.
I have a NodeJS-based instance of TiddlyWiki. On the same system, we run a little shell script:
#!/bin/sh
cd /where/my/wiky/sits && tiddlywiki mywiki --listen host=localhost port=8087
And, within the nginx config file, inside the SSL section:
[…]
location /dm/ {
proxy_pass http://127.0.0.1:8087/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
[…]
My web server was already running on a set of keys I’d installed previously. So, no issues there.
This works partly. nginx now passes instructions to anything under “/dm/” along to the running TiddlyWiki service, and the requested content does come back, but: with that, there are three error messages; the button that offers a writing instance doesn’t appear; and, the button that usually offers saving the current content stays rotating in red.
Those three error messages are:
syncer-browser-tiddlyweb - 13th March 2022 at 4:06pm
Get Status Error: [XMLHttpRequest](https://[my server]/dm/#XMLHttpRequest) error code: 404
syncer-browser-tiddlyweb - 13th March 2022 at 4:24pm (count: 19)
Error retrieving skinny tiddler list: [XMLHttpRequest](https://[my server]/dm/#XMLHttpRequest) error code: 404
syncer-browser-tiddlyweb - 13th March 2022 at 4:25pm (count: 224)
Sync error while processing save of ‘[$:/StoryList](https://[my server]/dm/#%24%3A%2FStoryList)’: [XMLHttpRequest](https://[my server]/dm/#XMLHttpRequest) error code: 405
We see from the two “(count. …)” pages, the returned page constantly tries to update. So, it looks like “syncer-browser-tiddlyweb” is a process expecting to be called, with different content than I have here, but I haven’t been able to track that down.
If I then click to some other location on the web server, that triggers an “are you sure?” popup; further indication that what’s on this page is waiting for something that isn’t arriving.
Hints?
Thanks.
Related: pmario refers above to WebDAV. I took that to mean it would be (at least potentially) providing TiddlyWiki to accept new input from the user side. But, I read that WebDAV is now regarded as obsolete, with references to standard processes like ssh and rsync.
Along with the error messages I posted above, I’m starting to wonder: could it be correct to say that TiddlyWiki doesn’t – ever – accept input if it’s being accessed from within another server? Is that what we mean when we talk about the “single page” version of TW?
Am I going about this the wrong way?
My goal is really small. I want to share a small set of TW instances, among a small group of people. The whole reason for TW, here, is its interactive feature. But of course, encryption is necessary across an insecure (open Internet) process.
Is there some other way to encrypt TW traffic, that doesn’t involve wrapping it inside some other server?
Thanks.
Just a guess. I would have to recreate a similar setup to be sure.
It seems you access the TW server with a path-prefix /dm/ and you tell nginx about it. But you didn’t tell the the rest of the system about it. Some references to the docs You may have a closer look at it.
Aha! That contained the hint I needed! Thanks.
My main mistake: I didn’t realize that it was a problem to stage the wiki at someplace other than root. Turns out, if I just set that to “/”, with no other changes, my TiddlyWiki works fine. If I change that to something else, the initial startup of the wiki does happen, but only partially, as I described. My full solution below.
Adding “path-prefix=[whatever]” wasn’t necessary in my case.
From one of the pages you linked, “Using a custom path prefix with the client-server edition”:
Configure the client by creating a tiddler called $:/config/tiddlyweb/host that contains $protocol$//$host$/path/to/my/wiki/
That was the missing piece I needed, and I never would have found it on my own, so, much thanks.
Now, in case it helps anybody else: the whole process of making TiddlyWiki available via nginx under SSL, with some arbitrary path name – to allow for more than one TiddlyWiki instance, or whatever else.
First some details/caveats:
That said:
Let’s assume that all of our TiddlyWiki instances are stored under one base directory, like:
/some/place
For example, say we have:
/some/place/foo
/some/place/bar
/some/place/[…]
For each TW instance, we have a TW shell script. There’s nothing unusual for this; if a wiki is stored at “/some/place/foo/”, then:
#!/bin/sh
cd /some/place && tiddlywiki foo --listen host=localhost port=[PORT]
Important details I’m leaving out:
Moving on:
Make an entry in nginx’s config, under the appropriate ‘server’ section. There’s no reason we can’t put this under HTTPS, so within the “listen 443 ssl;” section:
location /foo/ {
proxy_pass http://127.0.0.1:[PORT]/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forewarded-For $proxy_add_x_for_for;
}
On both the shell script and the “location” entry, change “[PORT]” to whatever you choose, as long as it’s consistent. Remember that each instance of the wiki needs its own number, you know what to do.
Then, in the wiki’s source, in the directory ‘/tiddlers/’: make a page, named:
$__config_tiddlyweb_host.tid
So in our example we have:
/some/place/foo/tiddlers/$__config_tiddlyweb_host.tid
Contents of that file:
title: $:/config/tiddlyweb/host
type: text/vnd.tiddlywiki$protocol$//$host$/[NAME]/
…in this example, [NAME] should be “foo”:
$protocol$//$host$/foo/
When the file is in place, check its permissions. Your details may vary; in our case we have:
-rw-rw-r-- 1 mjinks nginx 148 Mar 16 23:37 '/some/place/foo/tiddlers/$__config_tiddlyweb_host.tid'
The important part is that the nginx process needs read and write on the file. That probably isn’t handled automatically, so (again, your details may vary):
$ sudo chgrp nginx \$__config_tiddlyweb_host.tid
$ sudo chmod 664 \$__config_tiddlyweb_host.tid
$ ls -l \$__config_tiddlyweb_host.tid
-rw-rw-r-- 1 mjinks nginx 148 Mar 16 23:37 '$__config_tiddlyweb_host.tid'
From here on we’re just verifying that our setup is complete and correct.
Make sure the shell script is running. Restart it. Arrange for the script to be started at boot time; how will depend on your system.
Double check your nginx config. Restart that too.
Now with the service up and running we should be able to go to:
On its TW interface, click through to ‘More → System → $:/config/tiddlyweb/host’. In it we should find content something like:
$protocol$//$host$/foo/
…and that’s it! A TiddlyWiki instance upon teh (secure) Intarweb. Maybe. Correct me if I’m wrong, and if I’m not wrong, is there a good place to post this?
It would be great if you wrote up a summary and posted it in the Tips & Tricks category in this forum, with a link back to this original thread for any further discussion.
The only field needed to create a valid tiddler is title. So you can remove fields that are empty. It won’t cause any problems for the system.
In the right sidebar, the tiddlers are sorted by the modified field. So if there is no modified field, they will be listed at the end. That shouldn’t be a problem either for your usecase.
Okay, thanks. Sometime in my fiddling, I did have those lines removed. Then, right when I started testing, there was failure. Now I’m sure that was a coincidence, some other part was wrong.
I’ll adjust. Thanks again.
Thank you! I have added a reply there directing people to this thread for further discussion or questions.
I personally would start the tiddlywiki-thread as a different user, that only has write access to the xxx/tiddlers directory and no where else in the system
I did change the wiki text quite a bit.
I did try to give it more structure.
I did change the “writer perspective” from “I” to “We” and “The wiki…”
I did add some more points to the “assumptions” about the existing setup and security.
That’s it for the beginning. The “assumptions” points would make good topics for more wikis. Every point can be 1 wiki.
IMO HTTPS doesn’t make a server secure. It only improves security for the communication to and from the server.
SSL Server Test (Powered by Qualys SSL Labs) shows you some info about your https settings. If it doesn’t show an A grade … There is some work to do