Is backwards compatibility all it is cracked up to be?

There is an interesting long running discussion comparing TiddlyWiki with Obsidian. A leitmotif that is shared with a recent GitHub post by twMat is a well-articulated concern about the experience of TiddlyWiki for a first time user.

Reflecting on this, I have realised that perhaps over the years our ruthless focus on backwards compatibility may have led to us putting the needs of existing users above the needs of prospective new users.

If that’s a reasonable analysis, and if we really do want TiddlyWiki to be more popular, perhaps we need to shift our priorities to focus equally ruthlessly on improving the new user experience. This discussion shows that many in the community are invested enough in TiddlyWiki that they would love to see it become more popular.

If were to take this direction after v5.4.0, the sort of things we might do include:

  • Form a working group of people who can be advocates for new users, setting the vision and priorities for the effort
  • Make a concerted, coordinated effort to improve our reference documentation, filling in some of the gaps
  • If @sobjornstad is willing, to adopt GrokTiddlyWiki as the official user guide for TiddlyWiki
  • If @simon is willing, adopt TiddlyHost as our official community hosting site
  • Figure out ways to help us measure improvements in popularity, to provide ongoing motivation. We wouldn’t ever want to use industry standard tracking techniques that are invasive and do not respect privacy, but I hope we could figure something out. Maybe just providing simple feedback buttons that users can opt to click if they want their voice to be heard
  • Do some usability analysis with real users. For example, we could work on a standardised task list, and invite people in the community to try it on their friends and family, observing the areas that prove difficult
  • Reevaluate trade-offs between backwards compatibility and first user experience, leaning more towards the latter. There was a discussion that touched on some of this a couple of years ago
  • Radically improve the experience for first time visitors to tiddlywiki.com by figuring out a better way to give a familiar experience of a lively landing page while still showcasing TiddlyWiki. There is a PR with some experiments on this, with a preview available here
  • Figure out how we will communicate breaks in backwards compatibility to users to help them make the upgrade. For example, we might take two or three releases to progressively fully deprecate a feature, at first just issuing warnings

And perhaps eventually we might indeed go as far as a name change. The ultimate break in backwards compatibility is to excise the word “tiddler” from the system, given how prevalent it is.

Looking at the above list of ideas, it is does seem that much of the effort in an initiative like this would not be software development tasks, but rather things like writing, organising, planning, and figuring out funding etc. This would have to be something that involves the wider community: I believe that there are people here with the skills we need, but we need to do a better job of coordinating work together to achieve these sorts of ambitious goals.

Just to be clear, all I’m doing at this stage is floating an idea for discussion. We need to think through the implications, decide what we want the future to be, and then work together on formulating the new policies.

One footnote is that this could be a topic where @springer’s professional skills might help us reason about some of the trade offs. I think I’ve overheard philosophical discussions that centre on the tension between the needs of the lives of the people currently alive vs. those who will be born in the future. If that’s right, perhaps people have already discovered principles that could be useful.

12 Likes

This is a great discussion and I’d like to share my thoughts on it.

Firstly, I don’t think there is a necessary link between breaking backwards compatibility and helping new users with TiddlyWiki. We don’t break backward compatibility to help new users use TiddlyWiki better, we break backward compatibility out of TiddlyWiki’s own development needs. It may be difficult for TiddlyWiki to develop further without breaking backward compatibility.

But on the other hand, as Jeremy mentioned before, there are no features that need to be removed from TiddlyWiki at this point, even marco, which is retained and only no longer recommended. This is an important point. Because TiddlyWiki has a lot of old plugins and old ways of using them that exist in users’ TiddlyWiki. This makes many plugins from a decade ago work just fine now. Other note-taking software rarely does, and may be obsolete after four or five years as the software’s architecture is upgraded and users have to deal with it all over again. This is also Whereas That’s why I always recommend that users use wikitext in TiddlyWiki instead of markdown, because markdown is unreliable, and different software has different tags. joe Armstrong has already pointed this out in his blog, the link to which I’ll put at the bottom.

Further, I’ll be recommending that new users come to TiddlyWiki, not because TiddlyWiki is now going to break backwards compatibility, but I’m going to proudly show them that TiddlyWiki5, which has been running for twenty years, is completely trustworthy. They can migrate their knowledge base to TiddlyWiki, keep it and use it forever, untouched by anyone, anything.

Secondly, to promote and develop TiddlyWiki, focus on promoting it on internet platforms rather than recommending it to people specifically around them. Because Internet platforms are more persistent, and some bloggers with huge followings are more likely to promote TiddlyWiki. This is very evident on software like Obsidian and notion, where there are a lot of people creating videos spontaneously and more people creating videos for revenue. The most recent I know of is that Sönke Ahrens created a video using Obsidian for note taking. But there are very few videos on youtube about TiddlyWiki, many of them are a few years old, which doesn’t reflect the growth of TiddlyWiki. So much so that many new users don’t know CPL exists, and installing plugins is still only possible by going to the plugin website and drag and drop importing. So if you have a friend who makes and posts videos on youtube, consider asking him to help introduce TiddlyWiki. the other side of the coin is that if you create a good thing with TiddlyWiki, you can also post videos on youtube to promote and introduce it. This is far more appealing to new users than if we were to discuss it on a forum.

Thirdly, to make TiddlyWiki more accessible to new users, there should be more documentation written from a new user’s perspective, and we can recommend configurations to help new users make the transition. There are several plugins that can be installed as pre-configured versions of TiddlyWiki. You can install some of the best plugins recognised by the community to help new users use TiddlyWiki better, such as relink plugin and Commander plugin. It is also necessary to install the CPL or future official release of the plugin library. Some seasoned veterans can continue to use the blank version, but the configured version is still recommended for new users. And of course maybe a good platform needs to be recommended to them in order to make sure they can upgrade TiddlyWiki properly without any surprises.

That’s all I can think of at the moment, I’ll add more when I have other ideas. Feel free to continue the discussion.

On the other hand, I’ve been a teacher before, but I wouldn’t recommend TiddlyWiki to my students, because in China, they don’t have so much time to learn a note-taking software, and they always have too much homework to write. Chinese college students, however, seem to enjoy studying note-taking software. But they end up choosing Obsidian or Siyuan Notes because they don’t even know TiddlyWiki exists.

5 Likes

I do not insist on anything and do not even suggest anything, I am just trying to learn how to formulate prompts for different AI bots and decided to share, taking advantage of the opportunity, what GPT Chat gave me for my next prompt ChatGPT - Go ASON Jinaga Unison And I emphasize that I am a very active user. Not that “I can’t go a day without tiddlywiki”, but I live with tiddlywiki files for hours every day. It is a pity that the tiddlymap plugin is not being developed and there is no way to search for all author’s tiddlywiki files on tiddlyhost and, in general, there is no way to make a public presentation of files in the style of a common project page, that is, to repeat this interface for an individual user. Maybe no one is interested in this, but I am not a supporter of collective creation of information, for example, Wikipedia pages, but this is just by the way.In short, less is better. The main thing is not to lose uniqueness. I noticed that all bots impose storing tiddlers in large special databases, and I think this is a mistake. We need to look at the ecosystem as just a distributed system of TW files and this is the strength of the project! As few dependencies as possible!

I’ll start with an answer to the question in the title. YES.

This means you care. It’s respect. It’s a core value.

This is important. Don’t give this up. Even so, I’ve seen TW upgrades break TW plugins which then break my apps. Fortunately, so far, authors of these plugins have quickly repaired their plugins. It is these actions that keep me alive, to continue upgrading TW, even though I may not need it.

From my point of view… I wasn’t looking for a note taking app when I researched for TW. I was looking for content management system (CMS). I was not looking for something like WordPress or a Wiki. I was looking for something like Drupal, but an offline/personal solution. Capturing a note is usually a small part of the content to be captured in my solutions.

Drupal has the concept of content types, a way to organize and manage different types of information. They define structure of the content… all configurable.

TiddlyWiki has a content type field. However this is a content format type. So, instead I use tags to define content type. Future projects I might create a content-type field for this. But, it would better if this is a core TW field.

Aside: Drupal has had backward compatibility issues. Usually the compatibility ends at each major version.

Drupal provides the tools to manage content types. TW does not. I had to do that myself.

My conclusion, TW is (almost) perfect for me as a personal CMS, not a note taker. So, I don’t compare TW with note taking apps.

3 Likes

Introducing procedures as a replacement of macros has been the most beneficial change from my experience. Even as an experienced user I was very frustrated with subtleties between $param$, $(param)$, <<__param__>>, or $(__param__)$ !!! So, good ridance to macros and its /define and <$macro-call>… I will not miss the hours spent trying to debug you or work around you ! Let it be the first to trial whatever phase out policy you like.

If you can identify other improvements in user experience that simplyify TW then do so without fear… For backward compatibility goes - anyone who wants say macros could get a plugin that re-introduces them or not upgrade a wiki-file that crucially relies on it … Its more a question of how not if.

There a couple of candidates in my mind that revolve around syntax and hence user experience:

  • The widgets vs procedures discussion - that asks that if one was chosen then a simpler syntax of <<widget>> could be preferable to the mix of <$my.widget> vs <<my.proc>>… Personally I think you could go further and just say there is a priority order (custom widget then TW widget then HTML) that makes just <widget> a possibility.
  • I think filter suffixes are over complicated (why both : and , delimiters) and are redundant now that multiple parameters are supported … and time for the depreciated regex syntax to be purged.

I think you have also raised some good ideas and definately should move on them… with a couple of suggestions from me regarding these items:

Make sure to expand this beyond the echo chamber of this community - some real UX designers from other sites (maybe other open source communities - or a collab with you-tubers that do website redesigns)… But key is to get ideas from people who are not already “indoctrinated” :smiley: with TW’s current ways.

As long as people can opt in and out - I dont think industry mechansisms would be that intrusive… But I think it is key to know how many users are there that benefit from what features… or for that matter use macros as an example that informs this discussion. Another example is knowing what plugins are out there in the ecosystem so that they can be promoted to new users (another popular discussion topic I’ve seen)
I think this deserves its own thread to get it happening and make it possibly an ongoing activity. I’ll draft something soon.

Cheers: CB

To address the question posed in the thread title rather than any of the specific points raised thus far… I think it depends on how much of your existing userbase you want to retain. To me, one of the greatest strengths of TiddlyWiki has always been the incredible breadth and depth of knowledge of its users, and their incredible patience and generosity in sharing that knowledge. I’ve never encountered another community (of any size!) that’s so quick to provide specific, often custom-built solutions for new users and veterans alike — and the fact that the TW community accomplishes this with a cast of active contributors numbering in the hundreds, at most, makes this all the more impressive. So I suppose my plea is this: Don’t pare away the features that attracted your biggest fans in the name of appealing to hypothetical new users.

I admit I don’t have the data to support or refute this fear. I didn’t discover TiddlyWiki until 2021, long after the transition from TWC, so I don’t know what percentage of the Classic power users or its broader userbase made the switch to TW5. (On the flip side, I have spoken to long-time TWC users who began delving into TW5 only within the last year, and got the impression that it was not particularly easier for them than it had been for me, as an absolute beginner… and the points at which they struggled were broadly similar.)

But to speak purely for myself, I’ve invested 4+ years into my wikis at this point, and built more bespoke procedures (and yes, macros) than I can count. My most complex wikis would not function in a hypothetical TWX that eschewed $macrocall, filter suffixes, or the <$widget/>/<<procedure>> syntax, to pick a few examples from @Christian_Byron’s list. So, rather than face the task of converting several years’ worth of complicated wikitext to a new format, I think I would be tempted to stay with the last backwards-compatible version and carry on with the functional system I already have, incorporating new ideas from whatever fraction of the community chose to stay with TW5. And without any practical reason to familiarize myself with the way TWX does things, I wouldn’t be very well equipped to advise new users — much as I’ve enjoyed giving back to the community that has been so helpful to me.

Of course, there are degrees of backwards compatibility, and some breaks would be far more disruptive than others. For instance, I wouldn’t mind in the slightest if TW stopped supporting IE, and I suspect I’m not alone on that front! But if you do want to maintain continuity of the community, I think it’s worth asking developers and power users:

  1. Do you expect to adopt TWX, no matter what, when it becomes available?
  2. What features of TW5 could you not afford to lose? Or, put another way: what would dissuade or prevent you from using a new version of TW?

Unless you could add a plugin that restored those features - which is very possible and probably only adds a few seconds to an upgrade process (possibly only as a once off). Or a conversion tool was available that changed every \define to \procedure for you ? My point was there are many ways to solve any compatibility issue beyond just avoiding it.

If supporting macro calls in the core meant redirecting work from new features or making them impossible … or put new users off because they did not know which approach to use … or they wasted time like I did struggling with the old tech and gave up instead (btw - that is possibly a good start to remove macros from documentation asap) … wouldnt it be better to look at these pain points ?

Perhaps I’m missing something, or perhaps this is meant to be a stand-in for a generalized “supporting X”… but macrocalls are already in the core, so it’s difficult for me to understand how keeping them there could require as much work as implementing a new feature.

Re: your conversion tool suggestion: That’s certainly possible, and may even be an adequate solution for many (most?) users. I’d be wary of running my largest wiki (currently ~30MB not including the external core, and 25k+ tiddlers) through a search-and-replace tool; I think it would be almost guaranteed to freeze my browser. But I recognize that I may be an outlier in this regard.

Edit: On the other hand, building a converter would also take time that could otherwise be spent on the core… and some older syntax like $variable$ doesn’t have an easy modern equivalent. In fact, I did convert the majority of my custom macros to procedures over a few months after the release of 5.3.0; the ones that have survived were the ones that couldn’t be converted without complicated workarounds. And I wasn’t using $var$ nearly as heavily as many of the older plugins I’ve looked at.

Otherwise: I appreciate your points, but I don’t think we’re likely to convince each other, so I’ll just say…

Please don’t apply this suggestion to TW5! As @dongrentianyu noted, many useful and popular plugins are now 5+ years old, and many of their authors have since left the community. I think it’s a mistake to remove any wikitext that users may encounter in a TiddlyWiki “in the wild” from the official documentation: it is and ought to remain the ultimate source of truth.

I don’t understand what problem for newcomers might be fixed by breaking backwards compatibility. In fact it might cause some problems.

What really needs to happen is that all the key features that people expect in a modern note-taking system need to somehow be folded into the unified TiddlyWiki experience, without sending the user on a scavenger hunt.

  • Web capture
  • Android and IOS support
  • Nearly WYSIWYG (Doesn’t have to be full rich text – most others aren’t)
  • Timimi - like saving
  • Markdown support
  • Easy insert and management of images and other media
3 Likes

A key aspect of this new approach is to establish a clean and consistent syntax/grammar for TW 5.4.0. Currently, we have syntax, macros, widgets, and settings dating back to 2005, alongside numerous new ones added over time. While some older elements have been deprecated, they are still available in TW 5.3.7 and included in official documentation. This often leaves users confused about what they should use when trying to accomplish something.

I recommend using a clean set of modern features (the latest grammar, syntax, widgets, etc.) in TW 5.4.0 and keeping all older elements in TW 5.3.7, with a note for users who might be curious about using those older features, possibly for compatibility with their existing plugins.

1 Like

I like to ask one question before diving into above discussion.

Do we need to update all our old TiddlyWiki versions every time a new one is announced? Do you usually upgrade your Wiki from a 2010 version to the latest one, like in 2025? Why?

In most cases, I think the answer is NO! A TiddlyWiki is a self-sufficient, independent tool that works on its own. I have old TiddlyWikis built on TW 5.1.12, and they work great! The only dependency is a browser to open my wiki. I can add or remove plugins because TiddlyWiki always allows access to older official plugins.

Python programmers are likely familiar with this concept. Python behaves similarly. A Python environment enables you to work without worrying about updates or new versions, as you can maintain an older version with all its packages intact and consistent.

TiddlyWiki is not like MS Word, where you might face issues opening a 2010 doc file on a new laptop. Even the latest laptop/computer in 2025 can open a TiddlyWiki from 2005. So why should I worry about backward compatibility breaking with TW 5.4.0?

I highly recommend upgrading to TW 5.4.0 for its modern, user-friendly interface and polished, simpler, and cleaner grammar. Take Fortran as an example—a software that still supports code from the 1970s. By default, Fortran only uses modern syntax and grammar, but its compiler includes an option to enable running code written in Fortran 77.

4 Likes

The sort of backwards compatibility breaks that I am thinking of here are exemplified by the changes in v5.4.0. The breaks are generally minor edge cases that would only affect a small number of users. We’ve deliberately stacked them all together in the hope that it will reduce the disruption, and make it easier for users to understand when there might be a backwards compatibility issue that affects them.

Under the proposal here, we would have merged the changes in v5.4.0 when they were ready, rather than stacking them all up together and waiting until the accumulation of potential improvements reaches a threshold.

So, I am not thinking about things as radical as deprecating macros, precisely because it would break so many community resources. In any case, we’ve already extensively discussed the technical aspects of deprecating features and moving them into a plugin.

I think perhaps I’ve presented this as a radical change, but really it’s better seen as a relatively minor tweak to the policies and ambitions that inform the hundreds of little decisions that we make as we work on the project.

The other aspect of my original post was rebalancing the focus of our work between new users and existing users, and a part of that is trying to address more of the tasks that are not around software development, which means mobilising more of the community to work together on these improvements.

That’s exactly the kind of change I want to see. The things you propose could have been done at any time in the last ten years, but we haven’t done them. Part of the reason is that the project is overly dependent on me, and my focus has always been on iterating TiddlyWiki to make it as useful as possible. I haven’t had the bandwidth to do everything that is needed, and we haven’t as a community made a concerted effort to change the status quo.

Best wishes

Jeremy

3 Likes

My understanding of the 5.4.0 objective is to only break with backward-compatitility when necessary to support improvements or new features that have otherwise been previously impossible to implement, and to still retain as much backward-compatibiity as possible.

To my way of thinking, this does NOT include the wholesale abandonment of deprecated syntax (such as the \define pragma or the $macrocall widget) that are still widely used in numerous add-ons, but rather an acceptance that certain aspects of how those older features perform MIGHT need to change in some (hopefully) minor ways that minimize the amout of “breakage” that occurs.

This is of great concern to me, as my TiddlyTools.com add-ons (currently 184 of them), contain a total of approximately 1.2MB of wikitext source code, much of which still uses deprecated syntax in order to permit them to seamlessly work with older versions of TiddlyWiki (e.g., using \define instead of \procedure, [subfilter<variable>] instead of [<function>], and $list widgets instead of %if... %elseif%... %endif% conditional syntax.)

I will, of course, use the more modern syntax elements when there is no other way to achieve the results I want, but if it can still be done with older syntax, that is what I will use. I hope I NEVER have to put a note on an add-on that says “requires TW5.3.x or below” and only infrequently need a note that says “requires TW5.4.0 or above”.

-e

6 Likes

Very important point: most users assume TiddlyWiki behaves like other common software and that an update might break their wiki. But TiddlyWiki is different—it has zero dependencies and can keep working indefinitely, as long as a browser can open it! Of course, users might be curious to explore new features in the latest versions, but there’s no pressure—their existing wiki continues to work perfectly without needing to worry about compatibility or updates.

Even better, older versions remain fully functional and never become obsolete. TiddlyWiki doesn’t force upgrades, and there’s no risk of new versions interfering with or overwriting your current setup. You can keep your personal wiki exactly as it is, knowing it will keep running smoothly on any modern browser.

TiddlyWiki.com also supports access to the plugin library for any version—past or present—so you can always get the tools you need without having to migrate or modify your wiki. This thoughtful design ensures that ongoing development and modernization won’t disrupt existing users or break older wikis.

TiddlyWiki is future-friendly by nature: the platform evolves, but it never leaves anyone behind.

4 Likes

I really appreciate your thoughtful approach. You’ve made it possible for the entire userbase to continue using TiddlyTools without worry, while also introducing modern, more intuitive logic—like using %if for conditionals instead of $list. It’s great to see development moving forward without leaving anyone behind. And I have to say, I love seeing ‘requires TW5.4.0 or above’—that makes me smile. :blush:

The only thing that worries me a little is seeing a mix of old and modern features in new tools. While I understand the need for backward compatibility, it can sometimes create confusion—especially for newer users trying to grasp best practices. Still, I trust your careful design choices will continue to strike a balance between legacy support and modern simplicity.

2 Likes

I have to admit that it is always new features instead of freedom and flexibility that attracts new users, that’s why I don’t think being conservative is a good idea if TiddlyWiki aims at attracting new users. For example, If this feature was implememnted and merged, then users can implement TOC in tiddlers, block level links and perhaps outline view of the article.

After all, IMO Obsidian and Tiddlywiki is fundamentally different. Obsidian relies on software, and don’t need to care backward compatibility (and Electron is already breaking it, two years ago they dropped support for Windows 8.1). Its plugins are also rolling release. While Tiddlywiki relies on browser, which varies from computer to computer.

IMO we may not be too conservative if we need to attract more new users, as @Mohammad points out that many Tiddlywiki users don’t upgrades too often.

2 Likes

An idea for an addition to the commentary suddenly arose.ChatGPT - TiddlyWiki Go AGI интеграция

To a new user, all features are new. It’s not having the ones that they would expect with any other application that is the problem.

I’m a bit different, I always like to use the latest version of TiddlyWiki. Sometimes I even go to the pre-release versions and see what new features I need in them. Even those preview versions that haven’t been incorporated into the core yet. Using the latest version of TiddlyWiki avoids a lot of unnecessary problems and gives access to a lot of new features. And I highly recommend upgrading the wiki as TiddlyWiki is upgraded, which can be seen as a test, both of TiddlyWiki and of yourself wiki. On the other hand, when upgrading TiddlyWiki across many versions, it can be difficult to solve problems that arise, because it’s not clear if the problem is brought by the new version or by one of the previous version updates.

And, the plugin will mostly use the latest syntax of TiddlyWiki. I can’t get away from shortcut syntax expressions anymore, both in the little things I’ve written myself and in my plugins, which heavily include shortcut syntax expressions. I’m sure other plugins will be the same. Of course we can stay in one version forever, and plugins stay in that version forever. But plugins, or TiddlyWiki, are not everything. If there’s a problem you want to solve, then maybe the plugin author has already solved it, and an update to the plugin will make it happen, so why not?

TiddlyWiki’s syntax is indeed a bit complex and confusing. This is probably because define and procedure share an internal set of variable definitions. I.e. the usage of $x$ can also be used in procedure, which may be worth discussing in more depth. At this point, my view is that it’s good to find what you’re used to using, as long as it’s not deprecated and not recommended.

Agree break the backward compability, we can even start a working group to help migrate outdated popular plugins, like GitHub - flibbles/tw5-graph: Dynamic and customizable graphing library for Tiddlywiki5 do for GitHub - felixhayashi/TW5-TiddlyMap: Map drawing and topic visualization for your wiki .

It was boring to understand other programmers. But with AI, it is much easier. I’m not the creator of GitHub - tiddly-gittly/TiddlyWiki-CPL: TiddlyWiki5 Plugin Library for TiddlyWiki Global Communities, but I already update it with AI for several new features without understanding most of its tedious long code. I thought it would be unrealiable, but I find we could ask it to write test, so there will be no regression.

Another brighter future is that, software / plugins could be dynamic generated when needed. So if some of plugins are outdated, very possible that a people will stand and generate a new one with similar feature for the new TiddlyWiki platfrom version. Old plugins’s value will fade out even more faster.