More on using stackblitz webcontainers to run node.js tiddlywiki in the browser

[btheado]

@cdaven

Very cool! You can click this link to get your own Node.js instance in your browser: Node.js (forked) - StackBlitz

I’ve played with this some that last few days and here are some things I’ve found out. If you don’t care how it works and you just want to try it out, then skip to the links toward the end.

How it works

Stackblitz webcontainers use wasm to run nodejs directly inside the browser.

Wasm is a technology which allows running languages other than Javascript in web browsers. There are also many uses for it outside the web browser.

Node.js is a program which can run javascript outside of the browser. Node.js is not written in javascript and the stackblitz folks have compiled it to wasm so it can run in the browser. Dealing with node.js projects also means needing to install packages (usually with npm). Stackblitz also makes an npm client program available in the browser by compiling it to wasm.

Node.js and npm need to be able to read files on a filesystem in order to operate. Apparently there are ways of simulating a sandboxed filesystem inside of wasm. Maybe this filesystem is just stored in memory? I’m not sure. In any case, node.js and npm are able to read and write files inside of this sandbox.

Npm also requires network access in order to fetch the packages. Code running in wasm is permitted to fetch urls just like any other javascript. From looking at the network requests in developer tools, it looks like the packages are fetched through a stackblitz url (maybe not from npm directly due to CORS issues?).

To be useful for, the node.js program typically needs to be able to operate as a web server. Modern browsers have the service worker feature which allows network requests to be intercepted. It looks to me like requests from the browser to the tiddlywiki server running in node.js are intercepted by the service worker and fed directly to the wasm node.js. The response from node.js is returned by the service worker to the browser page and it looks just like a real server provided the response.

Limitations

As @cdaven noticed, saving individual tiddlers via the PUT saver fails. The failed network requests in the developer tools network tab do not have any useful information on the reason for the failure. Maybe there is a way further debug the service worker to see why it is failing. Maybe there is something about the PUT request that the stackblitz service worker implementation doesn’t handle properly. Or maybe it doesn’t handle PUTs altogether?

Launching official TiddlyWiki editions using npm commands

As @cdaven demonstrated in the other thread, a plain node.js webcontainer along with npm install tiddlywiki can be used to bring up a nodejs tiddlywiki instance. Since the tiddlywiki npm package includes all the editions, you can easily launch any of them.

1. Visit Node.js (forked) - StackBlitz and type npm install tiddlywiki in the terminal
2. Run one of the following commands
   a. cd node_modules/tiddlywiki/editions/tw5.com
   b. cd node_modules/tiddlywiki/editions/tw5.com-server (but as mentioned before saving individual tiddlers via PUT fails)
   c. cd node_modules/tiddlywiki/editions/dynaviewdemo
   d. cd node_modules/tiddlywiki/editions/full
   e. cd node_modules/tiddlywiki/editions/innerwikidemo
   f. etc.
3. Run this command: tiddlywiki --listen

Launching other TW projects from github (single click!)

Stackblitz also has the feature of being able to launch any nodejs project directly from github. Their webcontainer does not have a git client, so somehow there must be something on their server side which aids in cloning the git repositories into the browser. I’m not sure how this part works, but it is very flexible and even allows you to launch from a specific branch in the respository.

But it means any tiddlywiki github repository that has the following two things can easily be launched via a single click url (npm and node will automatically run):

1. A package.json file
2. A proper tiddlywiki edition folder containing a tiddlywiki.info file

Here are some single-click, launch from github examples which work. Node.js will automatically be launched and a frame containing the rendered tiddlywiki edition will be displayed. These all work:

1. Cdruan - Tw5 Captivate - StackBlitz
2. Btheado - Tw Ftlist - StackBlitz (terminal parameter required in url since the package.json doesn’t have either a “start” or “dev” script entry.
3. Tiddly-gittly - Tw Echarts - StackBlitz
4. Jasonmhoule - Editor Autolist Markdown - StackBlitz
5. Crazko - Krystal - StackBlitz

@saqimtiaz’s projects have the required files, but they seem to fail on the npm install step. It looks to me like he has package-lock.json files in his repo and they refer to old bleeding edge pre-release versions which are no longer available on npm? I’m not 100% sure on that, but these all fail with “failed to fetch dependencies” errors:

1. Saqimtiaz - Streams - StackBlitz
2. Saqimtiaz - Tw5 File Uploads - StackBlitz
3. Saqimtiaz - Tw5 Plugins Sandbox - StackBlitz

Many other TW projects on github are missing either the package.json file or the tiddlywiki.info file or both.

Bummer: the main TiddlyWiki can’t be cloned for some reason

But the biggest letdown of the github feature is that it doesn’t seem to be able to clone the main TiddlyWiki repo. No useful error message is given. I’m wondering if it has something to do with the size of the repository or maybe it doesn’t like some file in the repository.

The best usecase I see for this stackblitz is to be able to launch any fork and any branch of the main TiddlyWiki respository. That way an author doesn’t have to build a standalone tiddlywiki file and publish it on the web in order to easily share what their changes look like. Instead they can just construct and share a stackblitz link.

But since the repository can’t clone, that can’t be done. Here is a link which shows the git clone failure: Jermolene - Tiddly Wiki5 - StackBlitz

[TW_Tones]

Very interesting, thanks for sharing. I am not sure I have the skills to make use of this, yet, but it seems to me to have great potential to support developers and designers, but perhaps ultimately support general users to make tiddlywiki their own. Thanks for putting in the effort on this @btheado

[sukima]

Given that stackblitz runs in the browser what advantage does this have over the other typical means of running TW? Saves would be meaningless over service workers as there is no persistence in stackblitz. Even so what was wrong with a single HTML file saved to a web server or github pages?

I like stackblitz as an idea on its own especially for showing SPA’s the need compilation. Or Node specific apps. But I’m not really seeing the huge advantage in the context of a static site generator or static HTML files like TW is. Seems just a bit complicated for a build system that os so self contained and easily ran locally or with no build pipeline in the case of a shared HTML file over email or hosted on dropbox.

[btheado]

Welcome!

@sukima

Given that stackblitz runs in the browser what advantage does this have over the other typical means of running TW?

For the general, common use cases, I don’t see any advantage. In the specific cases such as the one I mentioned, there is less friction using stackblitz (or there would be if stackblitz were able to clone the tiddlywiki repository ):

@btheado

The best usecase I see for this stackblitz is to be able to launch any fork and any branch of the main TiddlyWiki respository. That way an author doesn’t have to build a standalone tiddlywiki file and publish it on the web in order to easily share what their [pull request] changes look like. Instead they can just construct and share a stackblitz link.

Creating and clicking a stackblitz link is easier than publishing an HTML file, IMO.

It might also be useful for someone to dip their toes in the TiddlyWiki node.js waters via stackblitz before diving into the installation of node.js and git on their own computer.

[saqimtiaz]

@btheado

@saqimtiaz’s projects have the required files, but they seem to fail on the npm install step. It looks to me like he has package-lock.json files in his repo and they refer to old bleeding edge pre-release versions which are no longer available on npm? I’m not 100% sure on that, but these all fail with “failed to fetch dependencies” errors:

Interestingly, this may be the same issue as checking out the TW repo as that is what happens under the covers to resolve the TW pre-release dependencies.

[sukima]

@btheado

Creating and clicking a stackblitz link is easier than publishing an HTML file

Dear lord what has this world come to?!

@saqimtiaz

that is what happens under the covers to resolve the TW pre-release dependencies.

What dependencies. The package.json dependencies is empty.

I think what I was saying is IMO I found stackblitz way more complicated and confusing when it came to projects compared to just installing node via my preferred package manager. In cases of complicated build pipelines like React, Ember, Svelte, etc. sure having a pre running example in stackblitz does help but in the case of TW which is dependency free maybe I’m missing the advantage. Again this is subjective so if it works for everyone else great.

[sukima]

@btheado

fail on the npm install step

Does stackblitz block via CORS? Are there docs on why it can’t reach NPM?

I know stackblitz won’t allow firefox only chrome. One of the reasons I got frustrated with them in the past.

[sukima]

I just tried and am also surprised it didn’t work. Seems stackblitz is right out broken.

{
  "name": "node-starter",
  "version": "0.0.0",
  "scripts": {
    "start": "tiddlywiki wiki --listen"
  },
  devDependencies": {
    "tiddlywiki": "5.2.3"
  }
}

[btheado]

@saqimtiaz

Interestingly, this may be the same issue as checking out the TW repo as that is what happens under the covers to resolve the TW pre-release dependencies.

I think it is two separate issues. The stackblitz server is used to clone a project from github. But for npm packages it is all on the client side in the browser. It looks like the npm client depends on the git binary which is not present. This is what I get when I manually invoke npm using a package url similar to the one your package-lock.json file:

❯ npm install git+ssh://git@github.com/Jermolene/TiddlyWiki5
┌ [1/4] 🔍  Resolving dependencies
└ 𐄂 Failed
ERR!: No git binary found in $PATH

error Command failed with exit code 1

[btheado]

@sukima

I know stackblitz won’t allow firefox only chrome. One of the reasons I got frustrated with them in the past.

I only used firefox. The links I shared worked for me.