Elephant in the room

#9

Yes, the markup can be confusing at the beginning, but with the time you will have a mental cheatsheet and other shortcut to avoid the errors.

For example, when I see the {{{ (filtered transclusion) I think about it as basic list widget. And I avoid to write the bracket together to the rest of the expression. A single space is good to me.

From {{{[MyFilterExpression]}}}
to {{{ [MyFilterExpression] }}}
Then I easily can focus on the filter expression and its errors.

Also the { is the special use of transclusion ( {{ ) in filters.

1 Like
#10

Wow. Very interesting! I always love to know how smarter people organize their knowledge and thoughts. Please feel free to share more of this!

#11

Because Fortran is a compiler and turning off compiler switches doesn’t have negative side effects to the runtime system. The compiler does the optimization.

For web pages JS is executed in different ways, but all of them have to be done on the client at runtime. So adding code to the core for swtching on or off features will add complexity only. There isn’t much to win here.

#12

Hi poc2go
Sorry to read that. … I did create a TW cheat-sheet attached to this post, which is also part of a pending pull-request for the TW documentation edition at tiddlywiki.com

The main tiddler is: Wikitext Cheat-Sheet … which also contains a button to export the tiddler and all related tiddlers that are tagged “Wikitext Cheat-Sheet” …

So users can add their own info to the table and can export / import it to other wikis for fast reference. The table also contains links to the more detailed wikitext tiddlers at tw-com.

It won’t immediately fix your specific problem, since Filters in TW should probably have their own cheat-sheet. …

But it may be of use for future readers of this thread.

Wikitext Cheat-Sheet.json (16.0 KB)

6 Likes
#13

I empathise with the frustration you are finding and whilst I would never say TiddlyWiki is perfect, I would suggest “the frustration will pass”. You will get to see it all afresh on a new day.

However having being involved with TiddlyWiki over a decade the use of quotes and braces you raise are no longer problems for me, in fact I would be prepared to argue, the quotes are as they are out of logical necessity and no more complex than almost any other language out there. It’s all about the necessary delimiters, Because tiddlywiki uses transclusion the curly braces are required to help there.

@poc2go you say

Well I would, in fact be surprised, I am confident here this area is unlikely to change, the use of delimiters in tiddlywiki are largely well designed and fixed. In fact I don’t think that that is needed even possible.

Remember “On one Hand” that these delimiter rules are all about giving designers opportunities to include delimiters in strings that also need to be delimited. The thing is if you did not have these possibilities, you would be restricted in the kind of strings you can use.

The fact is “computers suck at understanding humans”, and “humans suck and asking computers what to do”, so languages need to help both parties agree on what they are talking about. As a result using a language means you are forced to come to terms with syntax demands that are logically required.

There is however an approach that can make you life much simpler and that is to be aware of the details but use your own de facto standards. The problem is it takes time and experience to develop and learn de facto standards. However 99% of all the delimiters in Wikitext are the same as that which I used in Advanced Basic, COBOL, and numerous other code I have written over the years.

  • name=no-space
  • name=“with space”
  • name="Need to ‘quote within’ "
  • name='Need to “quote within” ’ <= I avoid this although it is valid
    • In the above you can see as a rule I never use " in names, strings or titles
  • name=""“any complex delimiters including ’ " just use the most enveloping”""
    • The fact is if you have a value you can’t control then assume it has a ’ or " in it and wrap the code in triple quotes.
  • Of course I freely use double " Quotes in Prose as I have here. SO If I write code for the manipulation of text it is best to be very clear in you delimiter use.

So you may want to rage against the machine, and “I empathise with the frustrations”, but I think you are misguided in targeting delimiters and nesting for your fury, because they are in fact “not just essential” but also “grant you the power” of tiddlywiki and most environments, that give you the designer the power you need to achieve almost anything.

Another area that confuses people
TiddlyWiki is very permissive and allows HTML to accompany wikitext so when writing wiki text we need to ensure there is no doubt about what we a writing so to accommodate <html></html> tags and <$widgets/> variables <<varname>> and transclusions {{tiddlername}} are all necessary in wikitext.

However;
Filters logically do not allow html or widgets in them so we can use a reduced form within filters, and when delimited we do not need to add more delimiters, so in filters we use <varname>, {!!fieldname} and {tiddlername}. So I;

  • Avoid spaces in field, macro or variable names, join with “-”
  • Thus I need not use complex delimiters except when I must.

Every rule can be broken and I do so when necessary for the problem at hand (rare) and such rule braking becomes clear and obvious.

  • I too are concerned about this and as such raised this in relation to “parametised transclusions”

However I think documentation, learning materials and real world guidance and examples is most likely to help because most complexity in TiddlyWiki is in effect logically and functionally necessary.

  • Additional tools and syntax checking would help
#14

Thanks to everyone for their thoughtful posts, which together present a helpful analysis of where we are and how we got here.

The first thing to say is that I personally frequently get the same feeling. I look at some of the wikitext code I write and see that soup of brackets, and think how overwhelming it must be for beginners. I’m acutely aware of the complexity inherent in flexible, open-ended tools like TiddlyWiki, and strive to improve things.

As others have noted, the basic reason is that TiddlyWiki has evolved over time with strict backwards compatibility constraints. A lot of that evolution has been directed towards improving TiddlyWiki’s wikitext and filter languages so that more of TiddlyWiki’s internal logic can be hackable. That’s why things like map and reduce were introduced, even though no end user had ever asked for them.

TiddlyWiki’s vision is that everyone has slightly different needs, and that we can only universally and respectfully cater to those needs by allowing every aspect of the experience and functionality to be changed or improved. That’s why the long term goal for TiddlyWiki is that the core becomes an ever smaller sliver of JavaScript, with more and more of the core implemented in wikitext.

We do have a reasonable plan that should let us improve things: at some point in the future we deprecate obsolete features, perhaps moving them into an optional plugin, and strip the implementation out of the core.

My feeling is that it would make sense to do that once the parameterised transclusion work is merged, and also the core templates have been updated to use the new features.

12 Likes
TiddlyWiki is almost 20 years old!
#15

No rant – Just some thoughts.

Many elements of the core templates use “old code style” IMO because of: “If it ain’t broken, don’t fix it”.

Using new filter operators for core templates we have the possibility to simplify filter expressions, improve readability. But we also have to test the whole thing, which can be a hell lot of work.

For some of my latest pull-requests I try to update UI elements bit by bit, using tiny steps. eg improving the human readability by adding indentation only in the “first step”. So it is easier to test. … The downside is, that this also slows down the whole process … It’s not easy to find the balance between correct code, backwards compatibility and the desire for fast iterations.

#16

As an end user I think this is essential. I’m in no position to figure these things out. Although I have asked in a way for this. I didn’t know I was asking for map. I also didn’t understand the discussions about it…

This sounds great but has only value if the end user is able to understand what kind of hacking can be done. It will take more effort to give people insight in TW. One can create technically sound systems but, if poorly understood by end users, it will not be of much help.

Many people have probably experienced that things easy to them are sometimes hard to code. While hard things are sometimes easy to put into code.

Clear and concise steps should be undertaken to assure that the end user really becomes an end user.

#17

Going by my experience, the biggest issues to me in adopting TW were the following (and I’ll expand on them in a bit):

  • Not understanding its capabilities.
  • Feeling discouraged by the theme/visual design.
  • Needs some kind of self-hosting.

I think I’ve been stumbling upon TW from time to time over the last few years, I don’t exactly recall if what I saw was TW but it certainly worked similarly (in that it was a wiki inside a browser entirely stored in memory). I always left feeling “okay, that’s neat but I want an actual CMS/Wiki”. What it took for me was to:

  • Start taking notes for a campaign in Realmforge
  • Get angry at Realmforge for being so slow
  • Waste days looking for an alternative that worked the same but was faster
  • Waste weeks writing my own alternative in React
  • Get dissilusioned with how much work it is
  • Find someone on reddit mentioning that they’re building their world in TiddlyWiki and give it an honest attempt

And just how YNAB caused me to expound to everyone I meet the benefits of proper budgeting, TiddlyWiki has caused me to realize that it’s the single most powerful and amazing tool for all kinds of note taking there is. I world-build for my DnD campaign. I use it to track which LEGO pieces I am still missing when recollecting old sets. I use it to keep track of my daily work at my day job. It completely replaced Nimbus Note, which replaced Notion (because Notion was too slow and loved to randomly, in the middle of a DnD session, log me out) which replaced EverNote (which was too bloated and in the middle of a clusterduck of a rewrite) which replaced self-hosted DokuWiki (because I wanted better mobile access). I can’t imagine I’ll ever switch, I just need to figure out how to better upload images so that they are stored externally.

But I digress.

Feeling discouraged by the theme/visual design

When you visit tiddlywiki.com what you see doesn’t look modern. And while chasing modern for the sake of chasing modern, it’s difficult to sell to people how great of a tool TW is if you have to tell them “Check out TiddlyWiki but don’t mind the website looks a little dated”.
On one hand it’s awesome that TW’s website is written in TW. On the other hand I think two things would be great for it to have better reach and adoption and just keep people’s attention for long enough for them to see its power:

  1. A new main theme that hides most of the complexity and gives the vibe of a modern web tool.
  2. A simpler front page, which can also be made in TW, that gives a shorter overview of its power and is more to the point.

I am pretty sure #1 could be solved with some money and a good web designer. #2 though is something much more complicated.

(I expect a lot of pushback towards this, because of my experiences in DROD community where whenever I mention that the forums and the game’s website look extremely dated there is a trong pushback)

Not understanding its capabilities

It took me a while of using TW to understand its power that’s partially in just how simple it is (and how fast it works because hey, backend is optional) and partially in the power of macros and widgets. I don’t really have any idea how to explain it better, there are as many usecases as users.

For me a great thing is I can very quickly create interactive mini-pages that allow my inner-perfectionist to have everything look and feel the same.

Perhaps having a set of varied and real use-cases prominently displayed on the front page be a good idea. Or maybe a wizard that interactively tries to determine what you need it for.

Needs some kind of self-hosting

While the idea of being able to just hit ctrl+s and save the HTML file somewhere on your disk, in the world of SaaS and mobile apps it feels like it’s lacking a lot. Setting up your own self hosting is a big pain, even if all you do is just use GitHub saver. The myriad of options doesn’t help (which is something that someone has already mentioned).

An integrated TW hosting service would be great.

Okay, that came out as a wall of text. I wasn’t trying to sound too negative, I am still fairly new in this community so I hope I am not overstepping my bounds, just wanted to list what’s actually pained me in getting to love TW.

5 Likes
Pluggable backends for uploading files
Pluggable backends for uploading files
#18

I think most regular users of TW are in love with this piece of software and have experienced the pains you mentioned in your post at some point of their usage. Once someone has experienced the beauty of TW, it’s addictive and is difficult to leave it.

2 Likes
#19

I think tiddlyhost.com is as close as we get at the moment.

#20

Since this thread has gone on beyond @jeremyruston’s brilliant summary post I thought, well, I make a few comments.

IMO we want TW for “DOERS” primarily. Meaning: Folk who just write about their thing without ever having to tweak anything. Where “we” help is merely direct them to what version they need to do their thing in. Finish. Basta.

The vast opera that TW code discussion has become is something of a weird hinterland to me a lot of the time. There is, from my point of view, frequent confusion of EXTENSION with CENTRAL (core) that messes them up.

What am I saying? Merely that definition of “procedural objectives” matters.

Just a side comment, TT

#21

Bingo! And I think to the very good.

Kudos to YOU!

TW IS unusual in it’s evolution over time WITH pretty amazing backwards compatibility.

1 Like
#22

I’m my own worse enemy. Probably making filters more complicated than it really is.

I have used TiddlyWiki for over a decade - but not the TiddlyWiki ‘Personal Notebook’ app so much. My approach to building web apps is using the TiddlyWiki JavaScript codebase and framework; boot.js, utils.js, the deserializers, etc. Has a lot of functions that handle common tasks that come up when building a custom web application.

It has all that is needed to build and interact with the DOM and data. Everything being a tiddler solves the data management problem where data can be correlated and interconnected using tags, backlinks, types, the ‘list’ and so forth. Is very complete so don’t really need any other modules to handle server/client/user interaction. Most modules I add are specific to interface to corp databases, financial institutions, and stuff provided by the customer that whats the site built.

Has a lot of useful source code and is easy to tweak. Especially when handling data coming in of different formats; REST, CSV, XML, Worksheets. Standardize into tiddlers, display dynamic pages (domMaker!), process the data, and output as needed.

I rarely used WikiText or filters. JS macros called $tw.utils, $tw.wiki and $tw.tiddler functions to manage the data. In most cases had tiddlers organized in multiple $tw.wiki instances.

It seems TiddlyWiki was designed to not only build a personal notebook app but also to be used as a core framework to develop custom web based systems. One would not even know the framework is TiddlyWiki - a completely different site. Is an awesome tool.

So am a WikiText beginner - know the rules, the syntax, if I can remember them. But hasn’t quite ‘clicked’ yet. Am in that frustrating time when know enough to be dangerous - but that light at the end of the tunnel is still just out of reach. Years ago had similar experience with JS regular expressions which now just come naturally.

What is happening to me, know what I want to do - but my filters end up being these long sequences of steps and runs and conditionals - knowing will look back some day and laugh - being clueless!

Would like to mention the filter syntax makes sense. The documentation is fine, more examples would be helpful. But there are 140+ filter operators. Bunch of categories, parameters (which can be hard or soft (which can be indirect or variable)) Some can be negated, some not. There just is a lot to it. Seems somewhere in there is a simpler language that’s screaming to get out - but what is there is what you got to have.

The other components - widgets, actions, macros, pragmas, plugins, styles for me are pretty straight forward. Just bumping my head on those filters. More than likely, some night will suddenly wake up and out of the blue - ‘I got it!’.

7 Likes
#23

Yep!

Although we’re complete opposites, we arrive at the same conclusion. (I’m definitely no dev.)
That is actually a compliment to TW.
It appeals to vastly different audiences for infinitely diverging reasons.

1 Like
#24

Thanks @Maurycy for your feedback to the community.

With the right local or internet hosting look at the file uploads plugin work

  • This interesting because I took no notice of this at all, I just wanted to know the capabilities behind it. Key is the level of User Interface customisation to make it how you want.

  • Whilst a more fashionable look may help people who are “aesthetically sensitive” the look has to be an option for users, not a bespoke site.

  • I favour the idea the tiddlywiki.com become such a showcase, perhaps with more than one fashionable layout people can select, and not a documentation resource, with the current (and improved) tiddlywiki.com hosted at tiddlywiki.org

  • This would allow us to seperate the sometime competing issues of looking good and being a turbocharged documentation resource.

  • This is an ongoing issue, for one simple reason, it has so many possibilities.

  • This has come up many times and the conclusion was start publishing editions with different use cases.

  • It may be wishful thinking but I persist in a belief I have not yet realised that there are ways to explain its capabilities to a general audience. Perhaps with editions displayed in the prose that demonstrates this.

  • How does tiddlyhost fail to meet this?

It is feedback like yours @Maurycy we need, never apologise for being honest.

  • I cant agree more but I think it is important to seperate this capability from that which is presented to naïve or non-technical users lest their “brain explodes”.

  • I hope you can make your method’s publicly available, perhaps in the developer thread.

  • Actually as a “techo” myself using tiddlywiki for its platform, IDE, SDK, repository, code library, code generator, site generator, pastebox etc… I am often also at risk of my “brain exploding”.

2 Likes
Derailing elephant - comparative analysis of TW front page
#25

@jeremyruston I just want to bring this to your attention, I believe you have said something similar in the past.

By showcase I and the discussion suggests a highly designed and fashionable website (based on a tiddlywiki option) that optimises catching peoples interest. Emphasis on “marketing”, in a non financial way, the Idea of tiddlywiki.

  • I am confident the community has the skills and experience, but by spitting the competing interests between presentation (tiddlywiki.com) and documentation (tiddlywiki.org) it will be a somewhat simpler task.
#26

@TW_Tones Let me try to address some of your points.

I’d argue that huge majority of users are aesthetically sensitive in one way or another, otherwise commercial websites wouldn’t invest so much money and time into optimizing every little detail and doing A/B tests of seemingly pointless stuff. They of course do that to optimize sales, but that’s no different then to optimize for TW users.

I’ve made this huge post about my thoughts on what could be possibly improved from the UX/Web Design perspective by comparing how similar commercial products design their front page - Derailing elephant - comparative analysis of TW front page

This actually plays into another problem with TW, community fragmentation. I initially started with just a standalone file thinking it’ll be good enough. By the time I wanted to host it in some capacity, I’ve stumbled upon the information that TiddlySpot is dead and I think for a while there was no TiddlyHost, and I must’ve read somewhere in some older Google Group thread that there is no hosting service currently, so when I stumbled upon TiddlyHost I just incorrectly either assumend it’s not working or it’s not worth the time.

Looking at it now, and looking from the perspective of an informed developer, I see the following things that would stop be from using it:

  • No easily visible Pricing information – if it’s free I’d like to understand the extends of its freedom and if it’s donation-based or selling-your-data-based.
  • No clear information about the limits - what’s the max allowed file size, transfer limits etc.
  • The front page is a little bare in a way that in my inspires lack of confidence in either quality or consistency of the service. It’s difficult to explain, probably something to do with how it feels more lika a companion piece to TW than its own standalone hosting service, if that makes sense.
  • I also saw some posts about Auto Save not working which sounds to me like it always sends the whole wiki upon save, and I really like having it save only whatever changes.

Right now the reason I wouldn’t switch is because I like having full control over my wikis and already made the time and money investment. I’d be happy to switch if some things were there, but maybe I shouldn’t derail this thread even more.

#27

It is other problem, maybe the spliting your filters can help you, and then you can use little filter expression within filter and subfilter operator then you have more readable complex filter expression when you store them in variables/macros. See the examples of its use in docs https://tiddlywiki.com/#filter%20Operator:[[filter%20Operator]]%20[[subfilter%20Operator]]

1 Like
#28

I want to highlight this, again.

IMO, it is currently inevitable that will happen with textual-based markup for advanced uses. Every system I looked at that evolves over time becomes symbol baroque (see: 3) eventually.

I dunno (not being an “on-the-current-programming-model-ball” person) what the solution is.

There may not be one?

Just a comment
TT