Computational notebooks in TW

[Offray]

Hi all and happy end of year holidays.

I was organizing my browser tabs (yes, that’s one of my holidays pleasures) and found the GitHub thread about Add support for running external shell tasks which states:

It can be difficult for experienced developers to become proficient in working with TiddlyWiki’s code: besides requiring a good understanding of JavaScript, the internals are built around relatively unusual concepts and techniques that are likely to be unfamiliar to most.

However, many software developers are very familiar with using and creating text-based command line tools that adhere to the Unix philosophy of using stdin/stdout for input and output, allowing tools to be chained together in powerful ways.

The proposed external task mechanism allows Unix-style tasks to be registered and executed by TiddlyWiki under Node.js. Execution can be triggered via a server command or via a message propagated from the browser.

and I was thinking that maybe some kind of reactive notebook like Julia’s Pluto.jl, Python’s marimo or Javascript’s Observable but using TiddlyWiki as a base, could help to bring the gap between TW and wider audiences including, but beyond developers, like scientist, journalist or hacktivists (that was my intend with the Pharo powered computational notebooks in Grafoscopio).

In the a fore mentioned thread, @linonetwo talked about his usage of zx as a TidGi-Desktop plugin and I was wondering if could be possible to add support for other languages beyond JS, like Lua, adding a Fengari (repo) plugin for TW. That could enable tiddlers with the application/lua type, that can be executed in a reactive fashion and connected with visualization toolkits like Echarts or other functionality/data provided by TW.

The path of having interactive reactive web notebooks is something I’m personally interested to explore now that Hypermedia Systems, like HTMX and Data Star, are enabling the possibility to create rich web apps with minimal/no developer facing JS and from a wide variety of server side languages. I wonder if something similar, but client side could be possible with things like Fengari and Lua.

The first question to explore that path would be, how to package Fengari as a TiddlyWiki plugin? Of course, there are many extra things and any constructive idea/critic is welcomed.

Cheers,

[vuk]

@Offray

The first question to explore that path would be, how to package Fengari as a TiddlyWiki plugin?

There’s a short introduction here https://tiddlywiki.com/dev/#Module%20System

Also see How to include external js and css files?

Trying to package a complex JavaScript codebase as TiddlyWiki plugin can be a challenging task, if you’re not fluent in JavaScript (hey, that’s me). I have tried it once and failed, but I don’t regret spending that time. A simpler intermediary step to understand how things work is to implement something simple from scratch in JavaScript - a custom macro for example. I found this blog post quite helpful Making a "Hello World" Javascript Macro in Tiddlywiki ~ Adithya's Lair

[pmario]

@Offray

The first question to explore that path would be, how to package Fengari as a TiddlyWiki plugin? Of course, there are many extra things and any constructive idea/critic is welcomed.

Fengari (Moon in greek) is the Lua VM written in JavaScript. It uses JavaScript’s garbage collector so that interoperability with the DOM is non-leaky.

So – Fengari is a Lua interpreter written in Javascript.

The Lua itself is a language that is equally complex as Javascript, while being similar enough to learn one or the other.

There is a step by step TW Widget Tutorial in the Dev edition, that tells you exactly, how to create a TW widget in a TW way in a language that is native to the browser.

In step 3 TW Hello World widget is created using JavaScript.

Where hello.js is 12 lines of JS code – 26 lines if you count the comments. It creates a TW native widget, with all the necessary utilities to several of them at the same time and all of them survive re-render events that are triggered if you edit a tiddler in the wiki store.

None of those mechanisms are available in Fengari. – Nada

Tutorials 4 to 6 describe the TW refresh mechanism and how to use it.
Tutorials 7 to 9 describe how to handle widget attributes.

That’s the basics about the TW specialities. Sure the “devil is in the details”, but that’s even more true for an “alien” browser language.

The rest is JavaScript knowledge as described at MDN (Mozilla Developer Network) or as described in the Book: Eloquent Javascript, which is a free online book.

The second edition (pdf) uses the coding style the TW core uses at the moment.

Just some thoughts.
-mario

[pmario]

@Offray

The path of having interactive reactive web notebooks is something I’m personally interested to explore now that Hypermedia Systems , like HTMX and Data Star , are enabling the possibility to create rich web apps with minimal/no developer facing JS and from a wide variety of server side languages. I wonder if something similar, but client side could be possible with things like Fengari and Lua.

All of this can be created using TW wikitext, where you do not need any server-side backend and no JavaScript at all.

IMO even the checkbox-widget intermediate example is more powerful than what’s shown in the data-star TODO example. That’s all pure wikitext.

---

HTMLX – is almost the same thing as our widgets. Only the syntax is a bit different. TW5 exists since 2014 and probably is equally or even more powerful.

just some more thoughts
-mario

[linonetwo]

It works out of box in TidGi app, it supports GitHub - tiddly-gittly/zx-script: Run Javascript inside tiddlywiki like jupyter notebook plugin.

[Offray]

Thanks for your answer and references. They allow me to get a better picture of TiddlyWiki internals and development workflow.

@pmario

So – Fengari is a Lua interpreter written in Javascript.

The Lua itself is a language that is equally complex as Javascript, while being similar enough to learn one or the other.

Yes, kind of the point of many languages that run in the browser, like ClojureScript, PureScript, and even Fengari, is to avoid the monolingual dangers and design flaws that have made JavaScript (in)famous in so many memes and literature. So, if I’m going to learn a language that runs in the browser, hopefully that time/effort will not be spent JS, as I barely know it (more on that in the answer to your next comment).

Even Douglas Crawford, the author of “JavaScript the good parts” and one of the prominent figures behind its education and promotion, has been advocating for something else, because of the congenital defects of JS.

TW is my to go SPA (Single Page Application) dev platform for several projects. But I use a diverse ecosystem of tools, including HedgeDoc, Pharo, GToolkit, Fossil, Markdown, Markdeep, among others. The idea of having interactive web experiences programmed from several languages and beyond JS is compelling to me now that Hypermedia Systems are opening the path for a more diverse web development experience. Lua is a good embeddable language to put in other places, from microcontrollers to game engines and that’s why I think of it as a first to put inside TW.

For the moment, my explorations will start in the server (more details in the next answer). But I was wondering about the client counterpart of such exploration.

[Offray]

@pmario

All of this can be created using TW wikitext, where you do not need any server-side backend and no JavaScript at all.

@pmario

HTMLX – is almost the same thing as our widgets. Only the syntax is a bit different. TW5 exists since 2014 and probably is equally or even more powerful.

Thanks for the checkbox example. It shows the kind of power and flexibility I like in TW.

Despite of TW’s power as an offline/serverles SPA, I need a server when the knowledge management problems I usually tackle get more complex. For example, recently we use TW for linguistic revitalizing in the Colombian Amazonas (I will share more details in a proper thread, if needed, but here are some generalities). We use the concept of interpersonal wikis, so each researcher has his/her own TW and we synchronized them and visualized the wikis evolution, using Fossil and Pharo, via a custom data story and Domain Specific Language, developed for that purpose. I can not imagine myself doing that from JavaScript or TiddlyWiki alone, as I would loose all the fluency that Pharo/GToolkit provide me as a development platform.

So, given that I already know Pharo Smalltalk, with almost all my projects done in such platform and I have little to no knowledge/interest in JavaScript and the server and external languages/tools are needed anyway, finding Hypermedia Systems is pretty liberating, as I can create interactive experiences for the web, with the knowledge I already have without being forced too deep into JS or its framework of the week (I felt the so called JavaScript fatigue just by trying to map what was needed to learn it and by seeing at the syntax inconsistencies). I imagine that making the web a multilingual platform is what made HTMX so popular and a viable alternative to React or Angular, even despite of being developed solo for long time a by a small community just recently, after its popularity explosion.

Maybe a little bit of my motivations/path to explore Hypermedia Systems, can clarify how I see following my “web development” future story and my wondering about how TW would fit in that path. But that could be too detailed or long to post it here. So I’ll update this thread with a link to a bliki post with more details.

Ps: By the way, the idea of interpersonal wikis came from the inspiring limitations on the multiuser story behind TW, its excellent support for drag and drop and our experiments with detailed changes history, using Fossil. Now that TW will support multiuser, I think that some of the practice behind the concept would be still valuable. A real time multiuser TW via libraries like yjs or Automerge would be a dream come true.

[Offray]

@Offray

Maybe a little bit of my motivations/path to explore Hypermedia Systems, can clarify how I see following my “web development” future story and my wondering about how TW would fit in that path. But that could be too detailed or long to post it here. So I’ll update this thread with a link to a bliki post with more details.

Here are the details, in this bliki post titled My path to become a future “web developer”.