Project 2036: the future of TiddlyWiki

#13

I was prompted to take a look back at TiddlyWikiClassic to remember your motivations to launch TW5. Surprisingly I find its still well maintained site with tiiddlers being added even today. So a few points come to my mind:

  • Break compatibility if you feel you must (and looking at some commenst so far I think you may need to). Others will bridge the gap by maintaining TW5 as they have TWC.
  • That said - maybe add some focus at the conversion or plugins needed to consolidate content into the new TWX. Your community of supporters may be split into these different dialects and hence not giving you the scale of a single community needed to keep driving forward as easily.
  • Some analytics for knowing what features are being used and how big your community is may be helpful in driving conversion funcion priorities and keeping the core lightweight (i.e what might be better moved to plugin libraries)
  • As with TWC - I think others will step into maintaining each project you mentioned if you want to step back from ‘so many fronts’ (which is understandable) … However, what is needed most from you is a vision for what those projects are and what the goals of TWX is.
  • Just to be controversial - would you start to commercialise an application while still providing a free/open source tier… Looking briefly at Obsidian - it seems like far less flexible in the markup (e.g. no widgets) and plugins (only via javascript). Is this maybe a MVP for a free version ?
  • I remember at TW5 launch - JQuery was a inspring new tech that helped shape the direction… Now we have more tech like React, JSX ( which could help inspire how to simplify syntax IMHO) and aybe help with building desktop and mobile apps.

:slight_smile:

3 Likes
#14

I think it worthwhile to update the javascript dialect used by the core, so that we can replace some obsolete code and implement features like regex sticky flag. For backward capability, we had better drop support IE in v5.4.0.

#15

… yes! to all that @Bob_Jansen says, especially about Apple HyperCard (and before that, Xerox PARC NoteCards) … and yes! to all that @jeremyruston and everyone else says!

… and from a user-not-developer TiddlyWiki viewpoint (my personal perspective), it might be valuable to gather and share simple “User Stories” to help inspire bits of the future TWX – e.g.:

  • “As a writer, I want to create a new deck of cards with links between them, so that i can organize and share a network of ideas with my readers.”
  • “As a reader, I want to search for cards that mention specific items, so that I can find answers to my questions.”
  • “As a collaborator, I want to add comments and pictures and links to pre-existing decks of cards, so that I can enhance the value of those decks.”
  • “As a novice-level developer, I want to add simple automated features to pre-existing decks of cards, so that I can perform well-defined boring mechanical repetitive actions (such as jumping to a random card with certain characteristics, or counting how many cards have certain characteristics).”

etc, etc – all super-obvious stuff to developers, and all “magical powers” to 99% of the future TiddlyWiki user base! :innocent:

4 Likes
#16

As an ordinary user, I hope that the editing functions can be enhanced. Currently, most note-taking software is evolving towards being more like an editor, allowing users to start editing notes immediately upon opening the software. However, when we open TiddlyWiki, we still need to click the edit button to enter the editing mode. Don’t think that just one extra step doesn’t matter; it can actually be quite annoying. In other words, TiddlyWiki might need to consider this aspect. Because this could potentially be the real issue that breaks TiddlyWiki’s backward compatibility.

Another feature I’m looking forward to is that when we can directly start editing as soon as we open TiddlyWiki, we should be able to write our wikitext, especially macro code, in the editing area, and then have the corresponding functions generated automatically. Just like what Obsidian does now, when you use # followed by some content, you can create a tag, and then a tag button will be generated. When you click on this content, a search interface will pop up immediately, showing other entries that have this tag. I don’t mean that we have to replicate this exact feature. In fact, it’s not difficult to implement such a function with a macro. The challenge lies in achieving this effect within the editing area. In other words, we need a more hybrid editing approach, a way that incorporates the presentation of interface content into the editing process. I think this is why outline note-taking software like logseq or others are becoming popular.

There is also another aspect: we can further reduce the user’s learning cost. No one doubts the power of TiddlyWiki, but its learning cost is really too high, which makes it difficult for other users to stay with it. Using TiddlyWiki is like entering a high-level school for these users. They go through a lot of hardship, learn some skills, and finally leave for other products to shine. I think this is rather strange.

An important aspect of improving the editing method is to optimize the long-form content editing experience. I understand that the product concept of TiddlyWiki is to write content as a combination of different notes. But there is still a wide demand for writing long articles. Currently, I don’t use TiddlyWiki to write long articles. The article I posted on the forum last time was written using Siyuan Notes, not TiddlyWiki.

4 Likes
#17

@dongrentianyu’s comments on writing chunks of text took me back to my AI research days . I got a wonderful book,

Mapping Hypertext: analysis, linkage and display of knowledge for the next generation of on-line text and graphics by Robert Horn from the Lexington Institute, 1989.

The book is written as a hypertext, pages and pages of chunks of text and graphics, all hyperlinked by textual links to other chunks of stuff.

Completely unreadable!

Random access a chunk and follow links, OK, but ‘reading’ as we know it, a waste of time. You get lost in hyperspace very quickly. The undoubted knowledge embedded in the content is scrambled and gets lost very quickly.

TW suffers from this as well as you can see by trying to find stuff in the documentation except the authors have tried very hard to organise the content to varying degrees of success.

Got the book out of my bookshelf today and openened it again to a wonderful world of reminiscing.

If we are anticipating a new version of TW, it might pay the designers to have a look at this book. IT does contain lost of useful snippets related to writing and structuring content, ways that might for a basis for content organisation facilities in the new version.

Still available, seemingly, from Amazon, $461

https://www.amazon.com.au/Mapping-Hypertext-Organization-Generation-Line/dp/0962556505/ref=sr_1_1?crid=3B3LOWBBP9IFQ&dib=eyJ2IjoiMSJ9.rvAG1N2MuHBOA3yk4GZ0x8amuVuap1wQ3IGk_J8Xi2RvJ5DaguuLRfmGORd5l3ljP1HsFaq-GSzoz-5jn6nyyn_-dqpn5b6lUa_R1Z2fwIjVEtbQ-7j7jfvzT29-zRX1.m4hc9_4D8JV2RwRL0T6N3KbWTljdwTA8AnZWxmJ0hEA&dib_tag=se&keywords=mapping+hypertext&qid=1742944053&sprefix=mapping+hypertext%2Caps%2C292&sr=8-1

bobj

#18

Internet Archive has a version available for browsing here: https://archive.org/details/mappinghypertext0000horn

Looks interesting, I’ll give it a look.

TWX in 2036

I rediscovered TiddlyWiki back in December while looking for a platform or framework to make simple data-driven applications. I am in love with it. It meets almost all my needs. If we are thinking about TWX in only 11 years, here’s what I would want to see:

  • Better handling of structured data: I am planning to spend some time with MWS and its SQLite backend. I would love to have SQLite power accessible if possible, either a core function of the framework or via a plugin. That could remove the need for data tiddlers, which @jeremyruston is on record saying were probably a bad idea.
  • Simplify metaphors: mentioned above, there is some cruft around procedures and macros that can be a little confusing. Simplify the fundamental data types and components of the framework as much as possible.
  • Better support for collaboration and external media: MWS is promising and I hope the ideas being tested there are part of TWX. I also saw a thread about embedding external media playback with annotation. I think that’s extremely powerful and would love to see that as a first-class feature

All in all, if TWX never comes to be I’m very happy with what TW5 provides. Here’s to another 25 years!

1 Like
#19

If we’re talking about breaking compatibility, then the first suggestion from me is either shifting to Markdown as the default markup language, or adding support for Markdown as an fully functional alternative to WikiText.

1 Like
#20

I think this is a topic where TW’s flexibility is both a boon and a bane; which user-base should the future be geared towards?

Developers, from what I can glean as a non-programmer, want something that is more similar to other software solutions.

Product Users who want a mostly ready to use product would benefit from a more modern UI and a little more “linearity” but I think the problem with both is that they can mean different things to different users.

Tinkerers are the most likely to roll with TW’s quirks regardless. My biased view as a tinkerer is that this is the only tool that exists for those without full on developer skills but are still interested in building and not just using, so keeping things open to non-developers is important.

I was alive in the 90’s when the internet came about and remember the chatter about what it could be used for, and then seeing what it turned out to be. I hear the chatter about AI now and I see how the early usage by the general public (non-tech people) is already veering from what the tech people think AI is going to be used like. Your average person isn’t going to be integrating AI with TW for their needs, they’re going to grab a premade fully finished product, possibly one made with TW, but at that point the tinkering and flexibility of TW isn’t what they were looking for in the first place.

My day job is in metal fabrication. I work with everything from modern machining to processes that are fundamentally the same as they were when discovered in The Bronze Age. Even as a non-programmer, I can see that AI in programming is yet another tool and not a means to an end.

Your average person can’t be bothered to run a home 3D printer despite technology being as cheap and easy as it is nowadays, but they will use Amazon Alexa to do shopping. They’re not going to bother with AI beyond finished consumer applications. I think hypothesizing about that potential user-base to attract misses the point of what makes TW unique.

7 Likes
#21

100% agreed to this.

My first TW usage was 2009 with TW Classic, and the doubleclick-to-edit made for a very low barrier (at the time) to move between reading/editing.

In the last few years, my note taking has mainly been in tomboy/gnote, which does as you describe - provide a richly formatted visual output in the window which is editable at any time. There is no distinction between a reading and editing mode.

Returning to TW in the last year and picking up TW5, the loss of doubleclick-to-edit I was originally disappointed with, though there is a plugin which implements it and I’ve only set it on one of the several TW I have. The problem being that sometimes I want to doubleclick-to-select, so adds frustration in another direction. (“doubleclick = edit only if no text is selected” is an idea I’ve thought of, but I need to learn JS coding before seeing if I can tweak the existing for that)

Anyway, for my taste, tomboy/gnote style “the viewer IS the editor at all times” distinction would be wonderful - and keep a dedicated “edit the markup directly” mode for the power user gurus who want to get their hands that little bit dirtier!

(I do think an always-an-editor mode needs a more powerful undo/view-old-revisions history though, since the chance of accidentally “I typed in the wrong window and didn’t realise it till later” is much higher. I use TW via node so I get tiddlers as individual files, and shovel the whole thing into local git, but a more integrated solution would be great (and indeed, some of David Bovill’s demonstration (on the "Hitchhikers TiddlyWiki All-Nighter) of running local shell code via TW feels like a potential great step in that direction)

1 Like
#22

That’s how my TWs work, for the most part.

Edit your tiddlers through another tiddler[1] – call it your “master editor”. Let it navigate through your tiddlers, selecting them for editing. Viewing them in the story, they update “live” as you type in your editor.

Whoever it was that said, “the only limit is your imagination,” was dead right. Stop looking at the way TW is presented on tiddlywiki .com as “the only way”. TW is not hardware – it was built to be modified. TiddlyWiki puts the soft back in software.

[1] Take great care modifying macros and css live. And by "great care" I mean don't do it.
2 Likes
#23

Correct me if I’m wrong, but what you describe sounds like the editor is still separate to the display though, but instead of needing a preview pane to the editor, the edits are reflected live instantly? It definitely sounds neat and I’m interested to know how you did that, but it’s not what I imagine as the ideal.

#24

Many of the comments here make good points. Things I’d like to see TiddlyWiki improve on going forward:

  • Live WYSIWIG editing like @linonetwo’s slate-write. This is “table stakes” for apps in this category today.
  • A more official shared / community plugin repository system. The TiddlyWiki plugin repository could point to CPL, Relink, Kookma, Fibbles, EvidentlyCube, etc. - the plugin repositories that contain so many handy utilities. Maybe some of these could become “Recommended” by the core (Relink or TW-Commander, for instance).
  • Document and extend the tiddlywiki command line interface. Give examples, and simplify common operations (for example, rendering a tiddler or filter for a single-line wiki should be as simple as tiddlywiki my_wiki.html --render "Index" and write to stdout).
    • Blur the line between single file and NodeJS wikis. It’d be great if I could trivially “explode” a single file wiki, edit the files, and then “implode” it back.

I’d also like many of the features that have open PRs - dark mode customization, AI integration, AST node serialization - to land, but I know that this takes time.

#25

To import a single file wiki into a NodeJS one, you can do

$ tiddlywiki newwiki --load path/to/file.html

It works even if the single file wiki is password protected:

$ tiddlywiki newwiki --password 123 --load path/to/file.html

I have created an older thread regarding this months ago, expressing my concern that this leaves the plain text password in the shell history file though.

To export it back, after editing the files, you can do

$ tiddlywiki newwiki --build index

Am I misunderstanding this part and you mean something else?

1 Like
#27

My Viewpoint on TiddlyWiki

TiddlyWiki is a scalable framework that adapts not only to the user’s skill level but also to a wide range of use cases. Hear me out.

Let’s say I want a simple journal. TiddlyWiki handles that effortlessly. As I get more comfortable, I can explore fields and transclusions ({{||}}, {{!!}}, etc.), learning in small increments or keeping things relatively unchanged. If I’m more technically inclined—like I am—I can experiment with HTML, CSS, and even bits of JavaScript.

What I love most is pushing TiddlyWiki beyond its typical use case. Is been turned it into a PC desktop clone, an Excel-style spreadsheet, and even a desktop application framework. It scales with both the user’s knowledge and creativity.

The few limits I’ve encountered are minor and often feel like gaps in my own knowledge rather than flaws in TiddlyWiki itself. I refer to it as a framework because it doesn’t function like any other tool I know. With plugin support, I genuinely can’t think of any major constraints—it operates much like game engines or other frameworks, except it’s built on HTML, CSS, and JavaScript.

What sets TiddlyWiki apart is its unique scalability. Whether you’re my grandmother or a JavaScript cyber-god, both can use it comfortably without a steep learning curve. While some features could be more user-friendly, its progress and stability are mind-blowing :exploding_head:. Most frameworks break old plugins with every minor update, but TiddlyWiki remains impressively robust.

If a few suggestions could be made, a more robust editing, highlighting is problematic, the “”" function isn’t by default a button, text scale and font are not available on a per tiddler basis without knowing @@values or css, canonical_uri is a “hidden feature” many fields are not easily understood at a glance things like caption & description. Documentation is very lacking, it takes a long time for docs to catch-up with features or user’s end up making wikis to further explain syntax.

3 Likes
#28

Jeremy

Can this thread be “pinned”.

I find it important. And needs highlighting for a few months, IMO, to get maximal feedback.

Best, TT

3 Likes
#29
#30

As this thread became a general TW feedback thread, I’d like to note that in this forum all feedback will come from TW enthusiasts. That is, if a person looks at TW, finds it to be complex, and moves on to use some other tool, then chances of them stumbling into this thread are extremely slim.

Maybe worth submitting a message in the Obsidian forum to get some of those users’ feedback (that is, users that looked at TW but ended up using Obsidian)

6 Likes
#31

First, you know, others don’t, I am not code literate.
But, readers, keep that in mind.

TW is more than bare coding. It is a kind of philosophy too.
In the past I have praised it for being “agnostic”–that still works.
Most everything else is not agnostic in it’s application.
The agnosticism of TW on uses is a WINNER.

TW let’s users (albeit experts already) MAKE what they desire without limits.

A comment, TT.

3 Likes
#32

Probably a topic for another thread, but this doesn’t work as I’d expect:

tiddlywiki --verbose newwiki --load somewhere/some_wiki.html

doesn’t make newwiki in my directory. Creating and loading also doesn’t make files for any of the tiddlers:

tiddlywiki --verbose newwiki --init
tiddlywiki --verbose newwiki --load somewhere/some_wiki.html

(for any folks reading - I think you need to import the wiki)

This is where I think the command line interface could be improved for TiddlyWiki. Having one-liners to “explode” and “implode” a wiki would go a long way.

#33

I would just say there are good thoughts on this topic: TW5+TW6 informal discussion - including my own.

Vision

I have some thoughts and experience that from an organisational and values perspective, which I have DM’d you about.

I will comment on the product’s vision. I think of TiddlyWiki as one of the closest realisations of the concept of hypertext, all the way through the application’s design.

Ward Cunningham’s Wiki Design Principles combine with the Tiddler Philosophy for a working and reading environment that enables and encourages the use of hypertext at every turn.

Distribution (more related to TWX)

I also think having some more ‘batteries included’ would help, perhaps having a few core extensions that are very limited in scope, where they won’t be extended further and create ecosystems of their own. I am thinking of GNU Emacs here. In Emacs, is the ‘vanilla’ toolkit that has to be turned on & customised to be better (e.g. icomplete), with competing third-party suites that have their own extension ecosystem (helm, ivy, vertico) that are at varying levels of integration with the core. Emacs very conservatively expands its core offering, while never breaking compatibility.

Unfortunately, this leaves the user with an uncomfortable uncertainty on if they’re truly getting ‘the best’. It also means that if you are heavily invested in one of those toolkits, you’d have to rework your entire stack of Emacs packages* if you want a specific new feature.
*very similar to TW plugins, with updaters and all.

I would encourage you to look into the world of Emacs as I think the way it splits off and has its own ecosystems of packages around a fairly consistent core is very similar to TiddlyWiki. Emacs also has popular distributions like Doom Emacs, which introduce users to the Emacs ecosystem, while still relying on the same core code.

You could also take some notes from Linux distributions in this regard. There is a full graphical desktop that’s the default, most prominent link, but also a minimal release that is often ‘headless’ and intended for servers. Most of the packages are the same, with a last 300 or so totally optional packages. As long as you have a good idea of how large the core should be, shipping some high-quality packages alongside would be reasonable to get more users through the proverbial door.

Let TW be like Fedora, which offers:

  • A standard ‘semi-skimmed’ release
  • A ‘skimmed’ release for special cases (Server, CoreOS)
  • Distributions downstream of it can be more akin to ‘whole milk’ (Nobara, Ultramarine, Universal Blue/Bazzite).

There has been talk over and over about how different distributions of TW could work and I think there is more yet to happen. For me, I would encourage you to consider keeping the core in a similar scope as it is now, while officially supporting and shipping some quality-of-life extensions by default. As some examples, I believe Relink, Favourites, Context Search (though it needs an update) and possibly Tiddler Commander would be good candidates for inclusion in a standard distribution of TW.

TWX interface & markup language in particular

Beyond what I suggested in the older topic:

If you are prepared to do breaking changes, you should probably move to a Markdown-like syntax. It is far my favourite LML (that is between Asciidoc 2.0+ and Textile), and TW’s Wikitext is a good markup language. But for whatever reason, it is a serious stopping point for many users. I switch between Markdown, TW Wikitext and Confluence/Jira markup every single day and don’t have a problem. Still, you have to go with what’s popular.

Base Markdown is not sufficient and needs extensions.
I also think it’s important to be close to HTML, and let us use raw HTML in tiddlers like we can now. This means you should probably be closer to the original implementation of Markdown in some ways, for example in having a single linebreak still be treated as part of the same <p>, without inserting a <br> like Discourse does.

On that note, Outline Wiki has finally managed to integrate Markdown and WYSIWYG in a compelling way, which should be considered as part of my suggestion in the older topic to have viewing and editing modes be closer to each other.

2 Likes