Not wanting to be negative. And frustrated right now! - please ignore this post.
Elephant in the room… been wasting hours! - missed a ‘"’ in a TiddlyWiki markup - really! Could have produced so much more in that time!
I hear all the time - the greatest strength of TiddlyWiki is also it’s greatest weakness. The complexity!
Are there any developments going on that reduces it’s complexity for devs? Or can I expect in the future - not a ‘"’ or a ‘""’ or a ‘"""’ but a ‘’""""’. Not to mention the ‘{’, ‘{{’, ‘{{{’ markup - would not be surprised if four embedded ‘{{{{’ is coming…
Yes - a tiddler list - is simple. I realize that TiddlyWiki needs to have additional features to stay ahead of the game - but adding to it’s already existing markup isn’t helping - makes insanely complex markup even more complex and out of reach all of us every day ‘user’ devs (well - at least me).
But adding but yet more complex markup, embedded in markup that is embedded in markup, is not helping. I spend more time debugging markup formatting errrors and getting the markup to work - than actually building my TiddlyWiki app!
Might be just me - guess I not keeping up with current dev complexities.
Whoa, you have mistaken me for one of the big boys… which might just explain why I agree with you.
I think one contributing reason is the idea to strive for backward compatibility. It obviously has some advantages but it also means you can’t easily redo core stuff but must come up with workarounds. This leads to further complexity even if some things are “deprecated away”.
Another reason is that wikitext is just one of many languages that build up a TW. So as you get more advanced and demanding then you need to look into the non-wikitext stuff. And I would guess that the dependency on other languages causes quirks in wikitext. And then there’s the performance aspect that also steers how you should use stuff… but maybe that’s true for all systems.
the greatest strength of TiddlyWiki is also it’s greatest weakness. The complexity!
Actually, I think it is the configuaribilty or flexibility. The “complexity” is more a consequence from this.
With that said; Do come with concrete ideas for how to simplify things!
The below is not to negate your experiences which I think are very valid.
I’ve been doing some fairly complex stuff with Tiddlywiki and I haven’t really had any situation where a markup issue like this was a big deal - it’s almost always quite evident where the issue is. At the same time I’ve been programming for close to 2 decades now and I always had this knack for being precise with my braces — might be why :).
I don’t know how long you’ve been doing TW but perhaps this is something that’ll get easier with time.
But I do agree that when you do anything remotely complex in TW it can get very messy very quickly. A lot of core tiddlers don’t have any kind of formatting or spacing so it all looks like one big ball of macros, filters and whatnots. I… used to write code like that long time ago so maybe that’s another reason why it comes fairly easy to me to parse through and understand :).
Unfortunately I don’t think there is anything that can be truly done about it - as @twMat had said, a lot of it is a matter of backwards compatibility. I am sure if Jeremy could start over the syntax and functionalities would be cleaner and less arcane, though on the other hand if part of the goal was to still maintain compatibility with markup languages like Wikitext or Markdown it’d always be kind of messy.
I think one possible improvement that would be backwards compatible would be to have some errors in a tiddler. Stuff like:
Calling a widget without enclosing every string argument in quotation marks (which I believe might be allowed but is also more prone to errors)
Forgetting to close or open a quotation mark
Forgetting to close > a widget or macro
Syntax errors in filters
Missing the final closing square bracket in a filter which is something I keep doing D:
And just report those potential errors in a visible way.
I thought about the same problem for a while. The only solution I could come up with is to introduce symbols which are not on standard keyboards (ex. ∑, ∆, ∈, ⊡, ▣) and then use the editor toolbar as sort of a keyboard specific to TW (which it already is in a way). This could theoretically simplify things… but in the long-term it may make things even more complex.
I like to refer to a similar situation! I did this before.
I use Fortran alot in heavy computation. It is almost the oldest computer language still compiles and runs codes from 1965. This is absolutely more than great. It is unbelievable! BUT
The compiler has switches to turn off all those legacy stuffs (why Tiddlywiki has not?)
The TiddlyWiki google forum/talk and docs while are great sources of tips and tricks and Q&A, BUT, they are a big source of confusion and complexity. You can do the same job using features from 5.0.0 to 5.2.3 in different ways! Then a newcomer looking for solution, cannot understand that solution belong to which version and cannot realize he/she may find a more semantic/simpler solution for the same job.
I believe preparing a good practice for WIkiTexting fo 5.2.2+ would help you to experience, how simple and pleasant TidlyWiki is.
This leads to a more complex codebase which tends to be a big problem.
I am going to bet a lot of core is still using a lot of legacy features.
More so plugins, of which a lot still work perfectly fine but are no longer maintained.
As someone who never uses the mouse when writing TiddlyWiki script, I am very much against the idea . That being said, I think part of the reason why the existing syntax is fairly easy to work with if you have any web developer experience (or Wiki/forum experience) is because it uses both existing paradigms (<$ instead of < for invoking a macro; BBCode used to be popular was also similar to HTML; a lot of the markup is known) and also uses things you can name. How do I succinctly name ▣ in a way that my brain knows how to interpret D:
The idea would be to use symbols which visually represent the function they perform. For example, in my note-taking system I have developed a symbolic system based on square symbols because they represent rationality/epistemology/categorisation/putting things into boxes/the process of learning. Thus, I have symbols such as:
□ for questions because the empty square symbolises the unknown/unanswered
⊡ for ideas because a dot in the middle symbolises a seed which has been planted (but it’s validity is yet to be determined)
▣ for (contextual) facts because the question has been answered or the idea has been confirmed, but the fact that it’s not full (■) reminds me that it’s embedded within the unknown/a context which could completely change the fact’s meaning.
These symbols visually represent the scientific process with minimal distortion and thus instead of remembering tags/words/letters I find that my mind works more efficiently with them.
I have symilar systems build with circles (⨀, ⊕, 〇) and colours ().
The initial idea I had when I have found TW was that if these or similar “symbolic visual languages” could be integrated into TW they would allow more efficient thinking and utilisation at least for me. Whether this would work generally, I don’t know.
P.S. For those interested at the moment I use Keyboard designer for input.
Like any flexible system it doesn’t do much unless you tell it what to do.
That makes it frustrating/discouraging to beginners.
But if you get through this initial phase it will probably be worth it.
For missing brackets and stuff (which still happens to me sometimes some more times)
I use the CodeMirror editor with an additional plugin. Saves me some more time.
The documentation is not very well structured. One of the problems is that you need to know some TW language before you get the results you’re after.
That makes it frustrating/discouraging to beginners.
But if you get through this initial phase it will probably be worth it.
It would be nice if new/additional features were always provided with an explanation and example.
This forum is pretty good (and friendly) at addressing calls for help.
no matter where you are at at the learning curve.
One thing I’d like to point out is that one of the stronger points of TW is it’s backward compatibility.
After all my hard work, diligent study and countless searches I am confident that my work will not be destroyed by ever demanding innovations.
In fact, when looking back, I sometimes don’t know how I did it.
I am a user not a developer, I’m using TW all of the time but develop only some of the time…
…but at least I don’t have to worry about continuity.
Yes, the markup can be confusing at the beginning, but with the time you will have a mental cheatsheet and other shortcut to avoid the errors.
For example, when I see the {{{ (filtered transclusion) I think about it as basic list widget. And I avoid to write the bracket together to the rest of the expression. A single space is good to me.
From {{{[MyFilterExpression]}}}
to {{{ [MyFilterExpression] }}}
Then I easily can focus on the filter expression and its errors.
Also the { is the special use of transclusion ( {{ ) in filters.
Because Fortran is a compiler and turning off compiler switches doesn’t have negative side effects to the runtime system. The compiler does the optimization.
For web pages JS is executed in different ways, but all of them have to be done on the client at runtime. So adding code to the core for swtching on or off features will add complexity only. There isn’t much to win here.
Hi poc2go
Sorry to read that. … I did create a TW cheat-sheet attached to this post, which is also part of a pending pull-request for the TW documentation edition at tiddlywiki.com
The main tiddler is: Wikitext Cheat-Sheet … which also contains a button to export the tiddler and all related tiddlers that are tagged “Wikitext Cheat-Sheet” …
So users can add their own info to the table and can export / import it to other wikis for fast reference. The table also contains links to the more detailed wikitext tiddlers at tw-com.
It won’t immediately fix your specific problem, since Filters in TW should probably have their own cheat-sheet. …
But it may be of use for future readers of this thread.
I empathise with the frustration you are finding and whilst I would never say TiddlyWiki is perfect, I would suggest “the frustration will pass”. You will get to see it all afresh on a new day.
However having being involved with TiddlyWiki over a decade the use of quotes and braces you raise are no longer problems for me, in fact I would be prepared to argue, the quotes are as they are out of logical necessity and no more complex than almost any other language out there. It’s all about the necessary delimiters, Because tiddlywiki uses transclusion the curly braces are required to help there.
Well I would, in fact be surprised, I am confident here this area is unlikely to change, the use of delimiters in tiddlywiki are largely well designed and fixed. In fact I don’t think that that is needed even possible.
Remember “On one Hand” that these delimiter rules are all about giving designers opportunities to include delimiters in strings that also need to be delimited. The thing is if you did not have these possibilities, you would be restricted in the kind of strings you can use.
The fact is “computers suck at understanding humans”, and “humans suck and asking computers what to do”, so languages need to help both parties agree on what they are talking about. As a result using a language means you are forced to come to terms with syntax demands that are logically required.
There is however an approach that can make you life much simpler and that is to be aware of the details but use your own de facto standards. The problem is it takes time and experience to develop and learn de facto standards. However 99% of all the delimiters in Wikitext are the same as that which I used in Advanced Basic, COBOL, and numerous other code I have written over the years.
name=no-space
name=“with space”
name="Need to ‘quote within’ "
name='Need to “quote within” ’ <= I avoid this although it is valid
In the above you can see as a rule I never use " in names, strings or titles
name=""“any complex delimiters including ’ " just use the most enveloping”""
The fact is if you have a value you can’t control then assume it has a ’ or " in it and wrap the code in triple quotes.
Of course I freely use double " Quotes in Prose as I have here. SO If I write code for the manipulation of text it is best to be very clear in you delimiter use.
So you may want to rage against the machine, and “I empathise with the frustrations”, but I think you are misguided in targeting delimiters and nesting for your fury, because they are in fact “not just essential” but also “grant you the power” of tiddlywiki and most environments, that give you the designer the power you need to achieve almost anything.
Another area that confuses people
TiddlyWiki is very permissive and allows HTML to accompany wikitext so when writing wiki text we need to ensure there is no doubt about what we a writing so to accommodate <html></html> tags and <$widgets/> variables <<varname>> and transclusions {{tiddlername}} are all necessary in wikitext.
However;
Filters logically do not allow html or widgets in them so we can use a reduced form within filters, and when delimited we do not need to add more delimiters, so in filters we use <varname>, {!!fieldname} and {tiddlername}. So I;
Avoid spaces in field, macro or variable names, join with “-”
Thus I need not use complex delimiters except when I must.
Every rule can be broken and I do so when necessary for the problem at hand (rare) and such rule braking becomes clear and obvious.
I too are concerned about this and as such raised this in relation to “parametised transclusions”
However I think documentation, learning materials and real world guidance and examples is most likely to help because most complexity in TiddlyWiki is in effect logically and functionally necessary.
Thanks to everyone for their thoughtful posts, which together present a helpful analysis of where we are and how we got here.
The first thing to say is that I personally frequently get the same feeling. I look at some of the wikitext code I write and see that soup of brackets, and think how overwhelming it must be for beginners. I’m acutely aware of the complexity inherent in flexible, open-ended tools like TiddlyWiki, and strive to improve things.
As others have noted, the basic reason is that TiddlyWiki has evolved over time with strict backwards compatibility constraints. A lot of that evolution has been directed towards improving TiddlyWiki’s wikitext and filter languages so that more of TiddlyWiki’s internal logic can be hackable. That’s why things like map and reduce were introduced, even though no end user had ever asked for them.
TiddlyWiki’s vision is that everyone has slightly different needs, and that we can only universally and respectfully cater to those needs by allowing every aspect of the experience and functionality to be changed or improved. That’s why the long term goal for TiddlyWiki is that the core becomes an ever smaller sliver of JavaScript, with more and more of the core implemented in wikitext.
We do have a reasonable plan that should let us improve things: at some point in the future we deprecate obsolete features, perhaps moving them into an optional plugin, and strip the implementation out of the core.
My feeling is that it would make sense to do that once the parameterised transclusion work is merged, and also the core templates have been updated to use the new features.
Many elements of the core templates use “old code style” IMO because of: “If it ain’t broken, don’t fix it”.
Using new filter operators for core templates we have the possibility to simplify filter expressions, improve readability. But we also have to test the whole thing, which can be a hell lot of work.
For some of my latest pull-requests I try to update UI elements bit by bit, using tiny steps. eg improving the human readability by adding indentation only in the “first step”. So it is easier to test. … The downside is, that this also slows down the whole process … It’s not easy to find the balance between correct code, backwards compatibility and the desire for fast iterations.
As an end user I think this is essential. I’m in no position to figure these things out. Although I have asked in a way for this. I didn’t know I was asking for map. I also didn’t understand the discussions about it…
This sounds great but has only value if the end user is able to understand what kind of hacking can be done. It will take more effort to give people insight in TW. One can create technically sound systems but, if poorly understood by end users, it will not be of much help.
Many people have probably experienced that things easy to them are sometimes hard to code. While hard things are sometimes easy to put into code.
Clear and concise steps should be undertaken to assure that the end user really becomes an end user.
Going by my experience, the biggest issues to me in adopting TW were the following (and I’ll expand on them in a bit):
Not understanding its capabilities.
Feeling discouraged by the theme/visual design.
Needs some kind of self-hosting.
I think I’ve been stumbling upon TW from time to time over the last few years, I don’t exactly recall if what I saw was TW but it certainly worked similarly (in that it was a wiki inside a browser entirely stored in memory). I always left feeling “okay, that’s neat but I want an actual CMS/Wiki”. What it took for me was to:
Start taking notes for a campaign in Realmforge
Get angry at Realmforge for being so slow
Waste days looking for an alternative that worked the same but was faster
Waste weeks writing my own alternative in React
Get dissilusioned with how much work it is
Find someone on reddit mentioning that they’re building their world in TiddlyWiki and give it an honest attempt
And just how YNAB caused me to expound to everyone I meet the benefits of proper budgeting, TiddlyWiki has caused me to realize that it’s the single most powerful and amazing tool for all kinds of note taking there is. I world-build for my DnD campaign. I use it to track which LEGO pieces I am still missing when recollecting old sets. I use it to keep track of my daily work at my day job. It completely replaced Nimbus Note, which replaced Notion (because Notion was too slow and loved to randomly, in the middle of a DnD session, log me out) which replaced EverNote (which was too bloated and in the middle of a clusterduck of a rewrite) which replaced self-hosted DokuWiki (because I wanted better mobile access). I can’t imagine I’ll ever switch, I just need to figure out how to better upload images so that they are stored externally.
But I digress.
Feeling discouraged by the theme/visual design
When you visit tiddlywiki.com what you see doesn’t look modern. And while chasing modern for the sake of chasing modern, it’s difficult to sell to people how great of a tool TW is if you have to tell them “Check out TiddlyWiki but don’t mind the website looks a little dated”.
On one hand it’s awesome that TW’s website is written in TW. On the other hand I think two things would be great for it to have better reach and adoption and just keep people’s attention for long enough for them to see its power:
A new main theme that hides most of the complexity and gives the vibe of a modern web tool.
A simpler front page, which can also be made in TW, that gives a shorter overview of its power and is more to the point.
I am pretty sure #1 could be solved with some money and a good web designer. #2 though is something much more complicated.
(I expect a lot of pushback towards this, because of my experiences in DROD community where whenever I mention that the forums and the game’s website look extremely dated there is a trong pushback)
Not understanding its capabilities
It took me a while of using TW to understand its power that’s partially in just how simple it is (and how fast it works because hey, backend is optional) and partially in the power of macros and widgets. I don’t really have any idea how to explain it better, there are as many usecases as users.
For me a great thing is I can very quickly create interactive mini-pages that allow my inner-perfectionist to have everything look and feel the same.
Perhaps having a set of varied and real use-cases prominently displayed on the front page be a good idea. Or maybe a wizard that interactively tries to determine what you need it for.
Needs some kind of self-hosting
While the idea of being able to just hit ctrl+s and save the HTML file somewhere on your disk, in the world of SaaS and mobile apps it feels like it’s lacking a lot. Setting up your own self hosting is a big pain, even if all you do is just use GitHub saver. The myriad of options doesn’t help (which is something that someone has already mentioned).
An integrated TW hosting service would be great.
Okay, that came out as a wall of text. I wasn’t trying to sound too negative, I am still fairly new in this community so I hope I am not overstepping my bounds, just wanted to list what’s actually pained me in getting to love TW.
I think most regular users of TW are in love with this piece of software and have experienced the pains you mentioned in your post at some point of their usage. Once someone has experienced the beauty of TW, it’s addictive and is difficult to leave it.
Since this thread has gone on beyond @jeremyruston’s brilliant summary post I thought, well, I make a few comments.
IMO we want TW for “DOERS” primarily. Meaning: Folk who just write about their thing without ever having to tweak anything. Where “we” help is merely direct them to what version they need to do their thing in. Finish. Basta.
The vast opera that TW code discussion has become is something of a weird hinterland to me a lot of the time. There is, from my point of view, frequent confusion of EXTENSION with CENTRAL (core) that messes them up.
What am I saying? Merely that definition of “procedural objectives” matters.
).