Tiddlywiki as a note taking tool - Positive and negative aspects - Lets discuss

Inability to scale.

Interesting. I used to use a tree-type note taker for 20+ years. I had about 200 “nodes” in that program, each node corresponds to 1 tiddler. Then I found Tiddlywiki. I take notes about many programming languages and tools I have to use, and dozens of projects I work on over the years. So I would have 100-200 tiddlers. I don’t think I would need 30,000 tiddlers. TW works for my use case, so that’s why I use it.

The TW search facility is much better than in my other note taking app. The first tiddlers it lists are the ones that match the title of the tiddler, which I like.

The problem with the tree-type note taker is it had to be recompiled to keep up with 64-bit architecture, and support had stopped years before 64 bit compilers existed. With TW it works in any browser. So I don’t have to worry about TW becoming obsolete.

Also:

  • It’s not that hard to upgrade TW when needed.
  • It supports the Markdown plugin which I like.
  • I don’t think TW won’t have to be recompiled if we get to 128 bit architecture.
2 Likes

Just my rather long three cents regarding the OP. It raises many important issues, which made me rethink if TW is really the best tool to use in my case. But I have come to conclusion that even though it has many problems, I know no better solution in my case. The most important feature of TW that other solutions lack, is its extreme portability. Comparing to the popular Obsidian, I would probably be happy with its smaller set of featuers/hackability, but requiring the installed application and a rather costly sync subscription are a major drawback.

I would add another another issue of TW from my perspective of note taking for personal and work purposes.

  • The relative difficulty in storing attachments (images, documents) in TW. It is not feasible in single file TW. Node.js on a server (for online access) is difficult to set up for a non experienced user. In case of an open knowledge wiki, this can be overcome by using links to public resources hosted independently of TW. But in case of private content, well, I can link to my cloud storage where I keep the documents, but then I can’t see it in TW (well maybe some API of e.g. Google Drive would allow it, but it requires work to set up). In Google Keep/ OneNote/ Evernote/ Apple Notes I can just throw a heavy PDF in and not worry about it. If that is a good way of archiving data and documents is another question of course.

And here are some counterarguments to those from link in OP:

I would ask the long-time Tiddlywiki users to stop and consider for a moment if Tiddlywiki is actually aiding expansion of their knowledge, or is it merely satisfying their urge to tinker with stuff.

  • Both for me. I surely spend much more time on tweaking it than I need for efficient note taking, but at the same time I am taking proper and useful notes in it. Prior to discovering TW I didn’t take much work-related notes (because I couldn’t find a tool where it would be worth doing) and didn’t pay much attention to organizing personal notes.

Tiddlywiki UI is outdated

  • Yes, it does look outdated, I agree. But there is no specific argument made as to what does the current design lack or what it cannot do properly. I presume one of the reasons why no one bothered to change it much, is that it simply works good. There is plenty of popular software, both commercial and free, that has not changed UI much since a long time, take office suites or browsers for example.
3 Likes

Thanks @vilc for you view point. I had a thought when reading this comment.

In some ways this is similtaniously true and false, and it comes about due to the versatility of tiddlywiki.

If you treat tiddlywiki as a platform or compare to a cloud server you can use webdav or node to break the single file model storing media in seperate files. But not using the single file model.

  • Note that tiddlyhost is not the above, it is cloud delivered single file wikis.

If you treat tiddlywiki as a single file smart document its has enought capacity if not more, as most single file documents to handle media except perhaps for the most media rich contents. Compare it to word, powerpoint, excel even a combination of these. Like such documents it can be cloned, posted, shared, emailed.

  • Quite importiantly you do not need any service to host it and can just have local files, only you access, and all for free.

If we do want to move to the server or cloud platform model, we quickly see a range of divergent solutions, many technical, or with some limitation or other that makes choosing difficult. But for most users the line between “served or single file” is blured, so it is easy to be confused, or make unfair comparisons.

  • Do Notion and obsidian etc… run on a single file model, without a server and fees? I dont think so.

Having used TiddlyWiki for more than a decade now and informed by my expierence in Information and Knowledge management, I primarily use a large number of single file wikis and happy with lots of intergration between wikis (I will not detail for brevity).

  • I have a large complex Single files wiki that ties everything together which does not include much media.

If I were a “betting man”, I am not, I would think if we could serve TiddlyWiki multifile on PHP or WebDav and permit TiddlerLevel locks (as in Bob) for multi-user, and external files (like upoload plugin) then it would be much easier to set up and deploy scalable internet facing cloud and multi-user team solutions.

  • There would be nothing holding TiddlyWiki back.
  • Unfortunatly the server side is dominated by Node, which is difficult to get Internet facing, and secure.
  • Since beyond a single file wiki there are so many platforms and operating systems etc… involved, the community ends up fractured and divergent and its hard to coral people back towards a somewhat universal hosting solution.

Just my personal perspective, and hopefruly a direction to move.

1 Like

It’s not that difficult if you can embed media(images,videos and pdf) as local files and reference them using relative path. I use this method using tiddlyclip plug in and save media addon. File upload plug in also might do this.

This is what I do for images.

These local media files load fast and can be accessed in mobile also easily if saved in a cloud storage of choice. Also the wiki can be kept light.

2 Likes

I think it’s totally fair to lean heavily into the tooling. I see the long term fiddling similar to fiddling with emacs, of which I am currently doing. But I think it really depends on how you are using TiddyWiki and where. I have worked at 2 aviation related companies where I leaned hard into using TiddlyWiki to organize stuff related to my job. I don’t work at those companies anymore. But I had very little control over my computing environment and so the idea of being able to take my notes with me in one single file really appeals to me.

The independence of TiddlyWiki in the single file version is really useful if you can’t install anything on your computer, or if you don’t want to store anything outside of your corporate network. The freedom to carry everything in a single HTML file can motivate you to want to use a tool and you adapt your workflows to it.

3 Likes

@Mark_S Some of the items I have kept in those modals overflow beyond the margins of the phone’s display. I should use two separate quick capture tiddlers - one for desktop and other for mobile - which gets automatically displayed depending on the device used.

It’s is good to know that, despite the critical approach to TW, Riz is committed to maintain Timimi, as it’s one of the easiest path to storing your TW locally and the route we follow here after having a public wiki in TiddlyHost.

Regarding Riz’s criticisms I don’t share the starting premise: the idea of wanting to have single TW with everything on it as an stressing point in my use cases. And I found splitting my wikis by topic and audience a powerful knowledge management mechanism (personal private/public, our table role playing group, the data activism grassroots community, my general classes, wiki/hipertext learning, among others).

That being said, I share some of the criticisms. I share the one regarding the default UI feeling kind of old, but that have been addressed in several theme and ui tweaking discussions, so I will address other two.

NodeJS seems overkilling when HTML files are not enough

There are lighter self containd alternatives like TW5-Server. But, part of the JS culture creates this maximalist approaches (if plain JS is not enough, go with the overcomplex framework of the week, headless full browser Electron or huge NodeJS and its dependencies). And while each culture has its caveats and advantages, I think that, as a TW community, we should explore and make more visible more minimalist, non JS nor full blown solutions. With TiddlyWikiPharo we are trying that approach, and because we use Nim for some external optimizations, maybe we’ll use TW5-Server in the future.

Operators/filters knowledge seems too specific and difficult to recontextualize elsewhere

I like the way TW created a continuum that blurs the distinction between user and developer, instead of those high barriers being put in other wiki software, where the jump between being a “content creator” and a “functionality creator” is pretty big. In contrast, in TW you go from formating wiki text, to macros, to filters and extra operators in a mostly pretty fluent way. I have seen that with some of my learners in my classes and workshops.

That being said, the more complex the filters and customizations go, the more you need the particular ways of TW, knowing that they will not be transferable elsewhere which, at least in my case, make me practice them in pretty constrain settings and do most of my data manipulation in Pharo (as shown in the data stories about migrations from Hugo and from Wordpress to TW).

What I would like is to be able to extend TW in several languages, once its data has been exposed in JSON. This is the approach we use in TiddlyWikiPharo by parsing (with Nim optimizations) the HTML file, looking for the JSON tiddlers. But something easier should be provided by default, kind of the /tiddlers.json address provided by TiddlyHost, if we want an easier cross-language extension mechanism that doesn’t require a full NodeJS server behind to work, but it is provided by the default HTML single file (because it already knows how to store and read itself as JSON).

Exposing the JSON storage as a easy prepackaged default will open possibilities to use TW not only from external languages and frameworks beyond JS/NodeJS (as we do with Pharo and Nim), but also from “internal” ones, which would create more extension mechanisms to traverse/blur the gap between content and functionality creators.

For example, I imagine adding a Fengari plugin included in some stand alone version of TW that makes TW extensible from Lua using some kind of Lua powered domain specific language (DSL) for tiddlers’ querying, composing, building and styling. The language can be similar to the one used in Typst “hashtag” functions but using Lua tables/functions to query and compose tiddlers. So a functionality like the one in this filter, could be expressed in an imaginary Lua DSL:

--||
all()
|> select()
|> taggedAs("member")
|> ramdomized()
|> styled("MemberTemplate")
--|

The user would loose the compactness of <$list filter="[tag[member]choose-random:all[]]"> of the initial tiddler example, in this imaginary Lua DSL (influenced by Elixir’s pipe operator). But, by using functions to compose a tiddler’s query, randomize its result and injecting that into view, s/he would gain exposure to functional programming concepts (listed below), while keeping them simple in an easy to learn language. In this way, the usage of alter filters outside TiddlyWiki is high, as it is bringing back knowledge outside of web programming (e.g. from the diverse Lua ecosystem) into TW.

For example, in the previous code, the all() |> select () could be replaced by selectAll(), which would be defined in some “Lua tiddler macro” as something like:

function select(reach)
  if reach == "all" then
    return selectAll()
  end
  -- ... other selection cases follow
end


function selectAll()
    return select(all())
end

And so, filters could be an example of DSLs, pipe operators and functions, function composition and a bridge to the Lua language, that would have a gentle learning curve, as we do with TW operators today, but with more reusability potential.

Of course, this is easier said than done and there is a long path ahead to follow this, including parser changes to support other languages and DSLs. But this could be easily started by improving the exposing of the JSON storage.

As a working conclusion…

While TW’s criticisms and their validity depend on the context where they arise and despite TW providing a fluent experience when is “used as intended” (and its many intended uses behind), it is easily stressed when is not: large wikis, complex theming, non JS or Node. Some of the criticisms can be addressed, kind of swiftly, by exposing TW storage internals so more languages and frameworks, avoiding the incidental current complexity and lack of recontextualization that is behind many of such criticisms.

How to traverse and open the possibilities, as a community, is still an open question, as more experimentation is needed (I’ll do my part by sharing my experiences of extending TW mostly outside of TW).

Ps: The code for the imaginary Lua TiddlyWiki DSL could also be written as the equivalent:

--||
taggedAs("member")
|> ramdomized()
|> styled("MemberTemplate")
--|

which shows that once a sound extension DSL have been implemented, economy of language and composition becomes easier… of course is even easiest in the plane of imaginary extension DSL and details are in the implementation.

3 Likes

That’s an interesting point. It is so from my perspective, when I think about it. I have basic knowledge of some scripting languages and html/css, so I think I can manage my TW better than “an average Joe” (but rather not better than “an average TW user”, as we seem to be a rather technical crowd). But I have no experience in JS or in web hosting and so on, so using an online-accessible TW in Node variant is too much effort for the gains it offers. It would be interesting to have an option “in between”, something more flexible than single html file, but not as difficult to maintain and scary for newbies as Node. Maybe it would be easier to implement in services such as Tiddlyhost as well. But as you said, it probably would require quite a lot of development work, that may be better to invest in other improvements.

1 Like

In that case, TW5-Server, listed above, seems a good alternative to NodeJS and it has parallel implementations in two easy to learn languages: Nim and Python. You just download and run a script or binary.

Probably, we are going to explore the Nim implementation in detail in the future, as we would like to have self hosted wikis, while avoiding the incidental complexity of NodeJS.

Add some pro/con in Obsidian vs Tiddlywiki compare site?

Why not add pro/con here?

Those websites has better SEO than our forum, post things there will better introduce tw To new users , other than just sharing information Between us who already know lots about tw.

I just searched for Tiddlywiki as a note taking tool - Positive and negative in Google and got this 10day old conversation.

2 Likes

Although I haven’t studied the linked material in depth I can attribute my own experiences. For context I searched for several decades for a journaling system. I tried things like OmniFocus, TaskWarrior, Things, and even Markdown files. Nothing stuck. Nothing I did kept me active on the system daily. It wasn’t till I started using TW as my daily note-taking and to do tracking system at my current job did it stick. For me TW is a fantastic tool that I have depended on every day for four years consistently.

The points the OP is making seem pedantic to me. Though technically correct in their observations I feel they don’t apply to my own experiences and needs of a daily notes system. Perhaps these concerns would be more noticeable to me if I were to use TW in a different kind of note taking (college classes, book research)—but in this case I think I would not associate TW as the tool for the job as my outcome might likely be more a text book then a wiki—I mean there is MediaWiki for that kind of scale and seriously who is taking notes at the scale of Wikipedia?!

I also take issue at the idea that notes I take on a daily basis would have need to back-link and full text search so far back as to need 30K tiddlers! I figure in about five years of making tiddlers daily I will move the > 5 years ones to an archive. Easy under Node as they are all text files on disk. Speaking of which if I need to full text search that I’d either use grep or feed the text files into Elastic Search.

I feel like they were expecting TW to solve more problems then it is designed to do. TW works well for me and I am ever thankful it is available to me.

6 Likes

I use TiddlyWiki for note-taking. I started a few years back (before the current Obsidian/Logseq goldrush) for a work project. Use then expanded to all of work and then to everything. In essence, I started a commonplace book by accident.

This means that the first objection, the on again / off again use, doesn’t resonate with me. I’ve been using one wiki for in excess of 8 years with my usage growing rather than becoming sporadic.

I also don’t tend to spend a lot of time tinkering. Here and there I’ll look for a plugin or fiddle with the aesthetics, but mostly I use it to write hypertext.

I have started moving my usage over to Markdown with time. While it is true that the default is wikitext, it isn’t hard to get most defaults moved over to markdown. I’ve wondered if having a good Markdown Edition (as opposed to a demo) would help.

5 Likes

i appreciate the ability to tinker and manipulate my notes as it is fun, helps to streamline my future workflow, gives me something to do when things are slow, and most importantly gives me more opportunities to look at and learn about my and how everything connects in different ways - which helps inspire new ideas!

3 Likes

7 posts were split to a new topic: The TW Filter Syntax Documentation – There is Room for Improvements

I’ve been using TiddlyWiki since 2017 and my understanding it was quite basic to begin with but I was able to get a lot of use out of it. Grok helped my understanding a lot though I’ve only partially gone through the course. There gets a point where I struggled with it and my brain kept glazing over. (I blame tiredness and my own lack of programming understanding more than anything else on this front.) I’ve tried going back to it again and plan to again in the future. I’ve already got a lot out of it though and more from checking this forum daily for problems/ solutions that may be of use and for ideas of what else I can do with my various wiki instances.

I don’t use TiddlyWiki as a note taking tool though I did attempt to use it this way when I first found it. I have other tools at hand that work well for note taking that are quick, easily accessible and not so easily lost if I accidently refresh my screen. I do use it as a note keeping tool though which I thinki is an important distinction. Notes taken by other methods are not so easily found after and spread out through various different locations. If previously written notes have been found important or useful enough to keep for future reference I would take the time to draft some tiddler articles based on them. I’ve also adapted some documentation into the wiki to make the information more accessible.

3 Likes

To get things back on topic, i can offer a few pros and cons

+ notes can be formatted in various ways and formats using css that other notes apps dont offer

+ templates let you design your tidders to look however you need them, be it kanban, dossier, checklist, table, or cornell

+ design custom snippets, macros, and keybinds to make your workflow quicker and better fit to you

+ community made solutions tend to be well designed

- the steep learning curve makes it challenging to make those things for your usecase

- some of the naming of operators, fields and classses limit the users choice in some cases.

- the large amount of things tw can do also means Learning how to do those things, which can be challenging enough to take more time to figure out than to use for notetaking

I use my tw as notepads more than notebooks nowadays. I often export my tiddlers into jaon to sort in my ssd or import specific ones to use as reference.

2 Likes

on “5. issues with search” i really like @clsturgeon 's rsort e.g. Auto complete with sorting by relevance and am using it with command palette after reading @etardiff 's Navigating to a missing tiddler with the Command Palette? - #7 by etardiff

i can’t access https://philosopher.life/ to read the “audit note by the author” (raised in “1. inability to scale”, which implies Riz’s note is from a while back) but i suffered data loss until i realised that browsers by default will cache (single-file) TW, and started to only use non-cached “private” windows/browser settings to always load the latest version instead of a cached version that might not contain my latest changes. if you haven’t fallen foul of this particular scenario, you will likely never realise it happened, and it’s super difficult to diagnose/appreciate without a particular set of circumstances.

but any saving system for any notetaking system has gotchas (e.g. cloud saving), this is just one that happened to affect my usage of tiddlywiki and happily is now solved. i’m not sure if this is the issue that affected the “audit note by the author” as that note is not accessible to me, but it could be a candidate.

i agree with @Mark_S and others, i also very highly appreciate TW’s semi-“independence” of being able to run a “simple” long-term reliable tool by decent authors, which is a point not mentioned in Riz’s note. a commercial product brings a commercial bias, which carries substantial negative risk that we likely can’t evaluate.

1 Like

There’s a discussion about this author, site, and disappearance, along with a link to an archive of the site, here:

I have this problem daily on my android device. I always have to reload once or twice to get the latest version of a synched TW file. I’ll have to see if private mode will fix it.

1 Like

It may be possible to not cache tw using meta data -

1 Like