TiddlyWIki Syntax highlighting

#1

This has being raised before and there are quite a few Google Groups posts relating to this but we continue to have a serious gap here. A number of highlighters are available but not for TiddlyWiki itself.

  • If anyone has found a reasonable solution lets make it our own, document it or make a core plugin to support it.

I personally find that 90% of the difficult to solve errors when writing in Wiki Text and TiddlyWiki script, that is with macros, widgets, html etc…, (not in the Javascript) are single character errors such as delimiters such as quotes, closing a widget or tag. This can occur when copying code from elsewhere or typing up from memory.

  • Unfortunately tiddlywiki is not always good at reporting errors in code, but rather than a sophisticated solution, what about just having wikitext characters highlighted along with matching quote, braces and tags closure?
  • perhaps a code preview that highlights miss matches?

It should be possible to adapt a HTML highlighter that displays non-matching opens and close tags, to understand macro calls and widgets and unbalanced quotes. Sure there will be some devil in the details but with the Regex Gurus and coders around, if we got this started I don’t think it would be long before we had a finished result.

  • Then in time we can leverage the same code for the wikitext parser to highlight wikitext characters like ! !! ; * etc… as well.

What do you think?

5 Likes
#2

Relevant: Language contributor checklist — highlight.js 11.7.0 documentation

1 Like
#3

I would suggest if someone proceeds with this, we start with a “TiddlyWiki Wiki Text/markup” then move on to a “TiddlyWiki Script” which includes the markup syntax plus html, widget and macro syntaxes.

#4

The language repos on https://github.com/highlightjs use a few sample files for test cases. So I think we would need to create a relatively small set (2 - 4?) of different files that together capture all the features we would want the highlighter to note.

I’m making the assumption that we would use the same highlighter as used in the official plugin. Perhaps that’s not warranted.

#5

Mkes sense,

If someone can kick this along I will be happy to contribute where I can.

#6

Bump,

I was just reviewing the syntax highlighting here https://highlightjs.tiddlyhost.com/ thanks to @Mohammad and remember this is an outstanding issue and gap in TiddlyWiki. If we had this we could;

  • quickly identify wikitext characters,
  • unclosed braces { {{ ( [ [[ etc…
  • The scope of widgets
  • See parameter values
  • Map a macro name to a define pragma
  • and more?

This need will be even greater with the release of TiddlyWiki 5.3.x due to a substantial increase in the syntax options.

Such highlighting, if only in a preview;

  • Would help new and experienced users avoid simple mistakes
    • perhaps even reducing the need to ask for help
  • Assist when sharing code snipits in talk.tiddlywiki
  • Assist when writing more complex TiddlyWiki script
  • Assist community members read, understand and support other community members TiddlyWiki script
  • Assist in reading and understanding “TiddlyWiki Script” in plugins and the core, thus assisting self documentation, and users learning from TiddlyWiki itself.

How to?

What we want to find is how to write a highlight “brush” for tiddlywiki wiki text and script (Widgets and macros, pragma etc…"), I expect it would be a hybrid between markdown and html brushes.

  • Once we have the method, I think we can crowdsource the various highlight rules.
  • However, Someone with strong regular expression skills, and knowledge of the tiddlywiki parser, may also find such a task somewhat simple, because the parsing logic is close to the syntax logic.
  • As HTML is also valid within wikitext we could start with the HTML highlighter,
  • Then using a markdown syntax if it exists, then modify the markdown part to the wikitext part, where many of the symbols used are interchangeable.

See also the relationship to mime types Is there any reason to avoid switching exclusively to Markdown tiddlers? - #4 by TW_Tones and Typed Blocks in WikiText and the codeblock widget and its language type CodeBlockWidget.

I would like to mention @jeremyruston, @pmario, @TiddlyTweeter and @saqimtiaz here to name a few.

2 Likes
#7

I’m pretty sure the “official reply” to this request is “CodeMirror plugin” but, yeah, I would also like for some built in bracket highlighting because brackets play a central role in wikitext, already for beginners, and it is a common source of error.

#8

Although I more often than not find syntax highlighting extremely annoying, whoever wants to set that up, maybe the following can be one of many helpful resources:

How TO - Create a Syntax Highlighter


My only hopes: please make it easy to turn off syntax highlighting; and: ability to easily adjust colours, would be cool.
#9

@twMat please tell me if I am wrong,

  • but I don’t think the CodeMirror plugin has this syntax highlighting, or only a small subset.
  • I am not sure but think codemirrors syntax highlighting uses the same highlighting code, if we have it in one we can have it in another.
    • If only code mirror, then that is better than nothing
#10

I see

  • CodeMirror Mode Markdown: Markdown highlighting mode for CodeMirror
  • and CodeMirror Mode TiddlyWiki Classic: Tiddlywiki Classic highlighting mode for CodeMirror
#11

I don’t know. I only tried the Codemirror plugin briefly years ago and, if I recall, it felt like overkill for my needs - which are more aligned with what you request here.

#12

Hi.

I was working on a wikitext brush over a year ago for Rob Hoelz’s brush loader for the highlight.js plugin. It’s a hack of the markdown brush. It was never in a state I was happy to make public and, being a non-programmer, it’s still very janky. Since I’ve come across this thread, I’ve brushed it up (no pun intended) and uploaded to tiddlyhost in the hopes that someone finds it useful and make a more competent highlighter.

https://tw-wikitext-brush.tiddlyhost.com/

It uses regex lookahead and lookbehind, the latter is not supported on safari. It’s unusable on internet explorer.

Thanks for a wonderful tool and a wonderful community.

2 Likes
#13

I know there are two or three small projects already tried to add TiddlyWiki to highlight.js language brushes. Some are brought here:

  1. GitHub - adithya-badidey/TW5-codemirror-plus: An attempt to make a better writing experience for TW using codemirror. by Adithya
  2. GitHub - Gk0Wk/TW5-CodeMirror-Enhanced: An enhanced for CodeMirror framework in TiddlyWiki, including TW5 highlight, WikiLink auto-completion, expandable hint, snippets, etc. by @Sttot (see demo: TW5CME — CodeMirror Editor Enhanced Framework for TiddlyWiki5)

And also worthes to note the VSCode project:

  1. [Howto] Use TW5-Syntax plugin for VSCode.dev by @joshuafontany

See also CodeMirrorPlus vs CodeMirrorEnhanced - #2 by Mohammad

The second (TW5-CodeMirror-Enhanced) is the most promising. It has

  • syntax highlighting
  • autocomplete
  • live help/tip on TW commands (widgets, macro, …)

So may be the TW5-CodeMirror-Enhanced project by @Sttot should be supported to cover what else you asked here!

TW5-CodeMirror-Enhanced implements a swift and rich extension framework for TiddlyWiki5’s CodeMirror editor, including TiddlyWiki5 (text/vnd.tiddlywiki ) syntax highlighting, Wiki link auto-completion, clickable links, Tiddler preview, and more. More features ( syntax tree, syntax completion , WYSIWYG mode , quick template input , etc. ) is under development . This framework is open source framework , anyone is welcome to join the development , the documentation is being written .

4 Likes
#14

Hi, is there some news about CodeMirror-Enhanced ?

I tried it, I get link or header highlighting, but nothing for widgets, macro, etc…

Also I get an automatic hint for completion of link (without the Ctrl-Space), but still nothing for widgets, macro, or simple text.

And the codeblock highlight doesn’t work either.

#15

Have you looked at @oeyoews work on [Release] Codemirror6 Plugin it expect there may be something already in the extensive enhancements first by @BurningTreeC making the plugin and @oeyoews’s extensive enhancements, almost too many to keep track of (a good thing).

1 Like
#16

No I completely missed that, this seems quite good :slightly_smiling_face:

Thanks, I will try it.

#17

In fact, codemirror already supports tiddlywiki, but I haven’t figured out yet how to combine the following two files.

#18

Well, I found a way to use tiddlywiki legacy-mode in cm6, but unfortunately, it has some functional conflicts with cm6. It may still be necessary to rewrite legacy-mode.

#19

To me I personally think codemirror, perhaps in a future release could be integrated into tiddlywiki differently so that it uses a “view template body” cascade (or similar) rather than replacing the standard editor which it does now.

  • I don’t think this is any more complex, may be even easier than the original integration, when we did not have features like cascades.
  • I would be happy to get involved with a project to do this, as a tiddlywiki superuser not as a Js developer.
  • We could then introduce a filter to determine on which tiddlers codemirror is active, with an opt in or opt out like a more sophisticated code body
  • This would also leave room for additional editors, even non-operative editors for read only wikis etc…
    • Including the ability to invoke the codemirror and other editors where we want to.
    • Alternative editors may include wysiwyg, html, plan text, external etc…

It is just a matter of leveraging new features and hackability in tiddlywiki, something which “codemirror”, not the plugin, knows nothing about except in its intergration with tiddlywiki.