Project 2036: the future of TiddlyWiki

Jeremy

Can this thread be “pinned”.

I find it important. And needs highlighting for a few months, IMO, to get maximal feedback.

Best, TT

4 Likes

As this thread became a general TW feedback thread, I’d like to note that in this forum all feedback will come from TW enthusiasts. That is, if a person looks at TW, finds it to be complex, and moves on to use some other tool, then chances of them stumbling into this thread are extremely slim.

Maybe worth submitting a message in the Obsidian forum to get some of those users’ feedback (that is, users that looked at TW but ended up using Obsidian)

7 Likes

First, you know, others don’t, I am not code literate.
But, readers, keep that in mind.

TW is more than bare coding. It is a kind of philosophy too.
In the past I have praised it for being “agnostic”–that still works.
Most everything else is not agnostic in it’s application.
The agnosticism of TW on uses is a WINNER.

TW let’s users (albeit experts already) MAKE what they desire without limits.

A comment, TT.

4 Likes

Probably a topic for another thread, but this doesn’t work as I’d expect:

tiddlywiki --verbose newwiki --load somewhere/some_wiki.html

doesn’t make newwiki in my directory. Creating and loading also doesn’t make files for any of the tiddlers:

tiddlywiki --verbose newwiki --init
tiddlywiki --verbose newwiki --load somewhere/some_wiki.html

(for any folks reading - I think you need to import the wiki)

This is where I think the command line interface could be improved for TiddlyWiki. Having one-liners to “explode” and “implode” a wiki would go a long way.

I would just say there are good thoughts on this topic: TW5+TW6 informal discussion - including my own.

Vision

I have some thoughts and experience that from an organisational and values perspective, which I have DM’d you about.

I will comment on the product’s vision. I think of TiddlyWiki as one of the closest realisations of the concept of hypertext, all the way through the application’s design.

Ward Cunningham’s Wiki Design Principles combine with the Tiddler Philosophy for a working and reading environment that enables and encourages the use of hypertext at every turn.

Distribution (more related to TWX)

I also think having some more ‘batteries included’ would help, perhaps having a few core extensions that are very limited in scope, where they won’t be extended further and create ecosystems of their own. I am thinking of GNU Emacs here. In Emacs, is the ‘vanilla’ toolkit that has to be turned on & customised to be better (e.g. icomplete), with competing third-party suites that have their own extension ecosystem (helm, ivy, vertico) that are at varying levels of integration with the core. Emacs very conservatively expands its core offering, while never breaking compatibility.

Unfortunately, this leaves the user with an uncomfortable uncertainty on if they’re truly getting ‘the best’. It also means that if you are heavily invested in one of those toolkits, you’d have to rework your entire stack of Emacs packages* if you want a specific new feature.
*very similar to TW plugins, with updaters and all.

I would encourage you to look into the world of Emacs as I think the way it splits off and has its own ecosystems of packages around a fairly consistent core is very similar to TiddlyWiki. Emacs also has popular distributions like Doom Emacs, which introduce users to the Emacs ecosystem, while still relying on the same core code.

You could also take some notes from Linux distributions in this regard. There is a full graphical desktop that’s the default, most prominent link, but also a minimal release that is often ‘headless’ and intended for servers. Most of the packages are the same, with a last 300 or so totally optional packages. As long as you have a good idea of how large the core should be, shipping some high-quality packages alongside would be reasonable to get more users through the proverbial door.

Let TW be like Fedora, which offers:

  • A standard ‘semi-skimmed’ release
  • A ‘skimmed’ release for special cases (Server, CoreOS)
  • Distributions downstream of it can be more akin to ‘whole milk’ (Nobara, Ultramarine, Universal Blue/Bazzite).

There has been talk over and over about how different distributions of TW could work and I think there is more yet to happen. For me, I would encourage you to consider keeping the core in a similar scope as it is now, while officially supporting and shipping some quality-of-life extensions by default. As some examples, I believe Relink, Favourites, Context Search (though it needs an update) and possibly Tiddler Commander would be good candidates for inclusion in a standard distribution of TW.

TWX interface & markup language in particular

Beyond what I suggested in the older topic:

If you are prepared to do breaking changes, you should probably move to a Markdown-like syntax. It is far my favourite LML (that is between Asciidoc 2.0+ and Textile), and TW’s Wikitext is a good markup language. But for whatever reason, it is a serious stopping point for many users. I switch between Markdown, TW Wikitext and Confluence/Jira markup every single day and don’t have a problem. Still, you have to go with what’s popular.

Base Markdown is not sufficient and needs extensions.
I also think it’s important to be close to HTML, and let us use raw HTML in tiddlers like we can now. This means you should probably be closer to the original implementation of Markdown in some ways, for example in having a single linebreak still be treated as part of the same <p>, without inserting a <br> like Discourse does.

On that note, Outline Wiki has finally managed to integrate Markdown and WYSIWYG in a compelling way, which should be considered as part of my suggestion in the older topic to have viewing and editing modes be closer to each other.

5 Likes

This post resonated a lot with me, and the fact that TWs versatility is both its biggest strength and its biggest weakness has come up before in several discussions.

Abandoning its very powerful but rather technical root would be a mistake, I think, we can’t really live and build all we have now without it; but without appealing to the masses who need a simple note taking or tasks tool, it is also hard to grow in popularity.

Jeremy is probably stretched thin, a single person can’t possibly attend alone to the core, desktop, mobile, MWS, end users, UI and still be able to significantly advance all simultaneously.

Maybe a potential solution is for Jeremy to focus solely on the core, a powerful unified, backwards-compatibility-breaking core, versatile enough for all the programmer, power users, and wonderful plugins we have amassed.

Then on top of that solid foundation, leverage the community, let all the contributors build the variety of plugins, flexible mobile UIs, WYSWYG, and all the end-user friendly ready-made solutions for the less technical masses. The larger community is also more likely or able to be able to cover and refine all the different variations and myriad of particular requirements the public will naturally have.

We’d “just” need a friendly way for the users to access the various “flavours”, which the current plugin library already does acceptably well.

2 Likes

In 11 years from now, I would like TW to be the no-brainer ecosystem to go to when designing data driven applications.

Not being a developer myself I looked at others with this “magical powers” and wished I had the tools and skills to materialize my ideas, TW in its current state allows me to create wikis and demos in an interface that does not require to have to write code (but does not limit you if you have the skills to create custom plugins) however, I wish for a deeper customization.

I would like to trespass the wiki realm, therefore I specially like the efforts of @poc2go with Tw5-node-red continued to lower the barriers to materialize projects for non-tech folks like me and opens the door to make our wikis to talk with IOT devices, Telegram bots, mail servers, and whatever you imagine by using Node-Red.

I guess In the coming years I want TW to be so standardized and interoperable that allows you to not just create wikis but whatever custom software solution you need.

1 Like

I’d love to see a list of what these TiddlyDesktop on macOS capabilities are.

[ext[the caption|the file path]] lets you do two things. If the path is a folder, it opens the Finder to that folder. If the path is a file, it opens the file in whatever app that you have set (with the Finder’s Info panel). As my work on a show progresses, I use one tiddler to take notes on a piece I’m composing that includes many different files, the scoring file, the original audio files from workshops which are the source material for the lyrics, the lyrics as they progress are written directly in the tiddler, the pdf files of the score that I send out to the performers with their input and transposition needs attached to each, the DAW files I record the different versions into, rehearsal recordings, and finally videos of performances. All these files are in different places. I used to make aliases and then put the aliases into folders along with text snippets, or a descriptive text file. TiddlyDesktop makes this so much easier and more flexible.

The other tool is the iframe: <iframe width="100%" height="600" src="/path-to-folder"></iframe> for example. I see the contents of a folder directly in the tiddler without having to switch to the Finder and then open the right folder to list the contents. As an example, a single tiddler can group any number of such “open folders” together to give a bird’s eye view of work that has been done with or for a given person. And because it’s TiddlyWiki, you can write explanatory notes, include links to other tiddlers, or use transclusion all in the same place where the open FInder windows are.

My main wiki runs under node.js on an old mac mini and broadcasts itself on a port I’ve opened in my internet box/router so that I can work with it from anywhere on any device. But the TiddlyDesktop wiki is all about keeping track of local files. Hope this helps.

1 Like

Thank you for this.

I am currently considering what my next main desktop working environment should be what with my seven year old Windows box going out of fashion and having acquired a Mac Mini slightly unintentionally. As such I am also interested in tiddlers containing

  • message: URLs. I know Mail.app kind of does this right less so Thunderbird (cb_thunderlink and derivatives)
  • Page references for PDFs which should be done by adding #page=n to the URL
  • References to source files possibly handled by emacsclient in my case

Re: Thunderbird email links:
The cb_thunderlink plugin apparently only works with older versions of Thunderbird.

What works for me is saving the message you want to link to as an .eml file. For example, without the user defined caption for the sake of clarity, [ext[/Users/m/Desktop/email with TD/[Talk TW] Project 2036_ the future of TiddlyWiki - Jon Schneider via Talk TW <noreply@talk.tiddlywiki.org> - 2025-04-09 1125.eml]] opens the email in a fresh (undistracted) Thunderbird window. To do the save I used right click Save As… on the message; and then opened the “Format: All FIles” dropdown menu to get to choose “Mail Files”. Your message takes 10 KB. Once it’s path is referenced in TiddlyWiki, it will be so much easier to find (and to take notes on) than lost in the endless meanders of the mail app itself.

Re: Page refs for PDFs
“#page=n” works fine for me. Long pdfs where page references are useful don’t go into my local TiddlyDesktop wiki because I don’t write anything that long that needs to be shared as a pdf. I have them on my main node.js wiki. Since I want to access them from anywhere, I put them on a CDN server; but they could just as well be on the old mac mini which serves my wiki. The code which is of type “application/pdf” in a field called “_canonical_uri:” is like this (for example):

https://TWpull.b-cdn.net/101%20Montunos.pdf#page=41

Re: source files
If you set the finder’s info panel to open the file with emacs, and copy the path, the ext syntax as explained previously will open the file with emacs.

1 Like

Just to follow up on this, what I like about TW is its technical strength in terms of what can be done using macros, widgets, variables, transcludes and so on. For me, the downside is that the wikitext-based “programming language” is overly complex and not well documented. Every time I come back to do some serious dev work in TW after a month’s break, I forget the differences between $var$, $(var)$, <<var>>, <$var>, {{var|1}}, {{var||1}}, {{{[[var]]}}}. I also forget when and in which context they should be used. It would be great if there is some way to unify and simplify all these different syntax.

On a similar note, I don’t know the history of the ListWidget but I imagine long ago it probably started as a simple way to produce a list of tiddlers. Today, however, so much has been added to ListWidget that it no longer just produce a list of tiddlers but can do so much more. Its output is often no longer just a list of tiddlers but can also be a number, plaintext, htmltext, a list of text strings and so on. I have two suggestions here. First, ListWidget needs a better name to reflect what it currently does, and; Second, there needs to be a way to directly specify the type of output be it a list of tiddlers, a number, a datetime, htmltext plaintext, etc…

3 Likes

Software Democracy

I have a belief that the existing TiddlyWiki demonstrates something I would call “Software democracy” giving everyone access to build, customise and publish applications, Websites and tools, including generators of such things, while at the same time using the standards on which it is built. HTML, CSS and Javascript.

Computers are everywhere but being able to make software to suit any purpose is not accessible to the vast majority of humans on earth. TiddlyWiki provides a gateway drug into this field of making computers do what you want to satisfy real worlds needs.

I believe any new version should maintain and enhance these possibilities of Software Democracy.

I also use the word Platform, and I think calling it “TiddlyWiki Platform” puts paid to complaints of the apparent triviality of TiddlyWiki as a name, no platform or framework is trivial.

I think what you raise here points to the following that the current TiddlyWiki and TWX could do;

Embrace HTML, CSS and Javascript

Smoothing out when and where and how to use these standards on top of and in TiddlyWiki would allow users to leverage these technologies more. Even tools for reference, syntax, design and importing open source projects outside TiddlyWiki into TiddlyWiki seems to be a big bang for the buck.

8 Likes

Hi Jeremy,
First I’d thank you and all the developers of TW for the great job! Awesome. TW became an important part of my life organising all the small but important notes. Wtf?

I totally agree to focus on the future of TW.
Please keep the stability as high as now.
Please differenciate better between experiments, half developed software and stable plugins.

Some plugins I’d like to see with full integration, because they matter for me most.

  • TW Plugin TiddlyJam
  • TW Plugin for E-Book (TWpub plugin)
  • TW Plugin for Node Red
  • TW Plugin for maps und maptiles ( leaflet plugin)
  • TW Plugin for draw.io
  • TW Plugin tiddlymaps
  • TW Plugin timeline plugin

The above extensions are so valuable for me, a better and more systematic integration would be great!
I’m not a programmer, maybe I can help with UI testing?

Chris

1 Like

Welcome to Talk TW Chris69, Thanks for contributing.

When you say “full intergration” what do you mean. Each of these plugins have being developed by different contributors and placing them in the core would consume bytes when not everyone may use them. Or do you mean something else?

  • If you think these could be in common use we could package them into an “edition” of tiddlywiki.

I wonder if TWX could include additional tools for data manipulation?

The tiddler / field as a record is powerful and can be easily used to structure data and tables. It would be great if we had a set of tools that allow a more intuitive way for solutions to be built on top of tiddlers but appearing as if they were records, tables or a collection of tables a database including using tiddler titles as keys/forign keys.

  • This would allow those comming to tiddlywiki from skills as broad as database, spreadsheet, tables to use tiddlywiki from the state by using there own expierence.
  • By providing these tools we also encorage the “correct” use of tiddlywiki.

Similarly I would like to see tools, standards and or guidence that helps with dealing with self-referential lists.

  • A key example is when sorting by a camputed value. It would be nice if we could generate a list in memory with multiple columns, sort on a nominated colum then return the list (of titles), then purging that working table (Structured variable?) from memory.

Hi,
I really appreciate your demand.

All of the above mentioned plugins fit perfectly in my work environment. And I’d really love to make TW a more universal tool for me. The potential of TW with integrating of all above plugins is tempting

But, some of the plugins seem to be more of a temporay playground to test ideas, not more. They generally work, but no activity for years…

I’d wish, that mature plugins are marked as such and can all be installed from the same official repository and in the same easy manner.

I’m willing to contribute in this direction. Not in a programming manner, but more as a user with UI focus.

Chris

2 Likes

Right. Given the massive use of plugins this sounds like a good, relatively simple to acheive, objective.

I still use some plugins made years ago that are no longer maintained (death of author; abandoned; experimental etc. )

I always worry they will stop working. But @jeremyruston has done an amazing job on backwards compatibility so the failures on v. old plugins I routinely use is still pretty low.

But, the point, I think, is the one you imply: there is a case to be made for a somewhat more Centralized source for, what one might call, Canonical Plugins.

An affirmative thought, TT

Exactly!

Probably this case touches one aspect of the thoughts of Jeremy, talking about the future of TW.

Chris

1 Like