Railway Syntax Format

This follows on from my first post in the area, Plain English in the Documentation, in which @TW_Tones raised some interesting points we can develop here. That’s actually the starting point on my original posting which follows, and which I’ll develop below it.
===========Original post ====================
If there’s one thing I found frustrating about finally having time to catch up on TW5, it’s the very poor documentation. Yes, I’ve seen Grok, but it’s a programmer’s wet dream, not something I can roll out to an end user.
That started me thinking about using TW in its data collection mode, to roll-my-own. Macros are a particular issue, I need a DOM to know what to call, and half the examples given don’t work straight-out-of-the-box: all[tiddlers] gives me nothing, for some reason. That in turn had me thinking about syntax documentation, and the “railway” format - which then brought me full circle, because Jeremy Ruston’s included it as an add-in.
Catching up on actually coding in it revealed its ancient origins, I haven’t seen a ::= object assignment in 50 years! That’s the flavour used in Tab Atkins’ library, I presume the same’s true here.
Does anyone else use it? Please don’t tell me there’s a complete “railway” syntax write-up hidden in the sidings somewhere? Or rather, point me in the right direction!

Post scriptum: I’m extending my initial post (ruled off to preserve it)
=================Addendum continues==============
In the first posting, it became clear this is certainly a useful contribution, and as @TW_Tones suggests, we might find a home for it in the wider documentation. You can see I’ve already started to address the issue, and it may well be I’m the ideal author, coming in with a fresh eye and needing to document my erudition, or lack thereof.
Let me first extend the original posting with my place-keeping comment to @pmario, full self-documentation on the Railroad format as implemented is on Tad Atkin’s site, which the plugin author (Jeremy Ruston?) links to. This is part of a wider family of Extended Backus–Naur form meta-syntax notations the ISO and W3C have made a mess of (this was my approach path, backwards, in passing, to landing here: it was a real pleasure to find TiddlyWiki awaiting as I disembarked!), to give some wider context.

In my earlier posting, @TW_Tones suggests there is an earlier taxonomy, but didn’t point towards it. My own suggestion is equally received, but I’m not too happy with the one used in GrokTW as it is excessively Techie, in my opinion. If @TW_Tones’ one can be located, I’ve no major reason not to use it, considering it’s the Group Construct of a lot of people who know far more about the subject than I do. I will of course reserve my author’s right to decide, if it’s also too techie.

The third question is, where to store the work-in-progress? I’m obviously talking online multi-user. I’d prefer to add to it visibly, so it can be made instantly available even if unfinished. It therefore needs a beta-test mentor, and could well be tagged at skill-level.

Just for some background, I date back before the Ark in computing terms. I started on the Lyons Leo in 1962, which was entirely machine code, with memory cards containing one bit, a single transistor and resistor, hand-soldered! Integrated circuits were in development. In 1968, my class teacher won the New Scientist competition-of-the-year, a year’s free use of the Imperial College IBM 360. At the end of tge year, he came into class in a high state of dudgeon. He’d been writing “a program that writes programs” (one of the first compilers), which obviously needed a starting point, and he’d intended using Start() or Begin(), but IBM had just snaffled them as file-handling functions. This was perhaps 15 years before scope was added! My name’s Jeremy Main, so I jested, “Why not name it after me, Sir?”. He looked sideways, muttered, “D’you know, that’s notva bad idea at that.” About 15 years later, I was in contact with Bjorn Sjoustrup about a detail in a financial interface, and asked him if Bell Labs had looked at the previous drafts. It seems they had, so it really wasn’t a bad idea at that. Poor Buddha, I automated the matra. Billions of machines invoking MAIN() every whipstitch. You’ve gotta laugh.
My last major venture into systems was as a beta-tester on the UK transfer of it’s Statute Law corpus online.

It would be nice to have a link to the “example” you are referring to, so we have the chance to fix it. all[tiddlers] alone isn’t a valid filter.

The filter syntax always starts and ends with [ and ]. So a valid filter is [all[tiddlers]]


TW allows us to create permalinks, which can be posted here. Which makes it easier for us to help.

1 Like

The documentation for the Railroad Plugin is very sparse.

It explains it’s own syntax using railroad diagrams, which is tricky but possible if some time is invested.

There are no explicit examples, except the diagrams used in TW itself. They pretty much show all the possibilities. Railroad diagrams are mainly used to describe the filter syntax (linked in the other post)

We will welcome every contribution, that will help us to improve the documentation for this plugin. The best way would probably be an edition, that exclusively explains this plugin.

2 Likes

@pmario
Thanks for the replies, I’m now using this as an enhanced continuation of my first posting, when this group started, which was more of the nature of an enquiry if there was interest. It seems there is, but I can’t go where @TW_tones suggests, for reasons I’ve suggested on my earlier post. I’m now going to tie the two together, so forgive me if my edits rehash your foundations. I’m going to open the question up.
In facr, as I said, I came in a massive circle looking for a tool to use, and found it embedded in TW! I enjoy a fine case of Divine Providence, and this is one. As a stop-gap, the Atkins site is self-documented, so that’s a starting point.

I have played with the railway diagrams myself but did not find them as suitable for my needs, just inclosing remember there has being many diagraming solutions over the years, search for “diagram tiddlywiki” or diagram at links.tiddlywiki.org other terms include image and visualisations eg ECharts - Bringing amazing visualization tools to TiddlyWiki!

Some dead links will come up but many of these implement diagramming that is separately documented in other open source projects.

You’ll find this now connects back to your suggestions in March. I explored the Atkins system before coming here, and it does include beneath-the-bonnet subdefinitions. As long as we use the modularity to the maximum, it should work. I’ve a crude suspicion it’s there because it was used in the design process, so rather than reinvent the wheel, perhaps we should check to see if there’s a funny round thing in the attic somewhere!

Other bullet points from March are:

  • Taxonomy
  • Online storage - PS You cover some options in the post I’m replying to, but which would you use if you were doing this, please? I’ll be picking brains to the max.
  • Moderation and collaborative support

One of the reasons for choosing Railroad is that a picture paints a thousand words. What I will need support on is examples. It could well be your other solutions could be useful in overview, though, for example in the Intro or Summary of each section.

Although on the whole I’d prefer to document systematically, begining at the beginning and ending at the index, it might be wise to bootstrap by documenting the Add-in first. In that respect, what footnotes, bibliography and index tools exist now, please?

The parser source code comment-section contains some info, that may be the complete syntax overview. I didn’t check it, but it may be useful.

title: $:/plugins/tiddlywiki/railroad/parser.js
type: application/javascript
module-type: library

Parser for the source of a railroad diagram.

[:x]			optional, normally included
[x]				optional, normally omitted
{x}				one or more
{x +","}		one or more, comma-separated
[{:x}]			zero or more, normally included
[{:x +","}]		zero or more, comma-separated, normally included
[{x}]			zero or more, normally omitted
[{x +","}]		zero or more, comma-separated, normally omitted
x y z			sequence
<-x y z->		explicit sequence
(x|y|z)			alternatives
(x|:y|z)		alternatives, normally y
"x"				terminal
<"x">			nonterminal
/"blah"/		comment
-				dummy
[[x|"tiddler"]]	link
{{"tiddler"}}	transclusion

"x" can also be written 'x' or """x"""

pragmas:
	\arrow yes|no
	\debug yes|no
	\start single|double|none
	\end single|double|none

There’s certainly more to the instruction set than that, though, as it doesn’t include the path logic elements on Tad Atkins’ site. Tad also uses the ancient assignment code ::= used to differentiate from a logical =.
I suspect the link code may be a straight lift from Tad’s site, and right now my energy’s low, sorry. If it’s something you can use, go and have a play over there, he includes his source code so we can compare it with the plugin.

For anyone coming in this, the ISO have a standard for the generalised documentation of command sequences, which somehow made it into TW5 as the Railroad addin. That’s it’s nickname, as it looks a bit like the stops on a railway timetable, shapes and lines, in the same general domain as flowchart logic but more specifically focused on the user interface, specifically.
The general idea is that by reading left-right, the user can get a good idea of what a successful code element looks like.

I’ve rummaged in the attic and found the forgotten wheel. We need more than one to do better than a wheelbarrow!

In tiddlyWiki.com:
Select Reference - Wikitext - Macro Definitions in Wikitext - Macro Definition Syntax
and lo and behold…

A search for it reveals the Plugin’s author is Astrid Elocson, and it was included in Release 5.1.8 17.4.2015. I presume it’s therefore licit for Help texts - can we link to the relevant pages?

astrid’s thread in gg had some historical detail https://groups.google.com/u/1/g/tiddlywiki/c/b57cfTZoI5w/m/3isWJMzH5F4J especially her demo http://ae-railroad.tiddlyspot.com which i found useful as a new user to railroad diagrams

1 Like

I’ve been playing, and it’s remarkably simple. Much like macros aren’t

Anyway, the result of my playing’s a crib tiddler. the essence of it is:

  1. You need the Railroad add-in
  2. Your coding’s always contained in a macro envelope header and footer:
    <$railroad text="""
    Your coding
    """/>
  3. I’ve chosen to use the term “Instruction” for a code object, which can be nested
  4. There were some suggestions that these can be linked to tiddlers, which isn’t at first sight true. Perhaps a CodeMeister can check, and, if possible, add it to an upgrade wishlist.
    4a. This is a subclass of a wider issue, that if Process A has a Process B as part of it, Ie when you load an App, it has a menu. That’s Process A. Process B explains what each menu item does. You can see how this fits a Tiddler structure. What I think this is pointing towards is another instantiation of the tiddler data block concept with a different UI presentation. The work started in TiddlyWiki.com subordinates it to Tiddler data, whereas it could, and probably should, be at a higher level. OK, enough metaphysics.

To anyone new, the Railroad format’s a way of documenting the user path into some kind of process, “How to skin a rabbit”. It offers a flow process read left-to-right showing the logic, for example how data extraction macros use a “Core Macros” => “Filters” structure, with a more defined start point of “[all[tiddlers]]” and “list-links” as specific examples of the former, which things like tag selection further refines.

A favourite macro, <<list-links filter:"[tag]+[!tag[Complete]]+[nsort[priority]]">> lists all tiddler with a tag matching the calling tiddler name, ToDo being a classic example - it also eliminates tiddlers tagged Complete from the list, and sorts the output by a user-defined sequence. Sorry about the drift off-subject, my objective was to genericise this to be a mastery of macros.

Returning to the subject at hand, the add-in recognises four different process types, namely linear, loop, optional and alternatives.

  • The linear process type just gives MonkeySee MonkeyDo, without brackets.
  • The loop process type uses curly brackets { MonkeySee + MonkeyDo } to put the reaction into the return side of the loop.
  • The optional process type uses square brackets with a colon after the opening bracket to indicate whether the text is in-line or branched [ MonkeySee ] and [: MonkeySee ].
  • The alternatives process type uses round brackets and the “|” symbol to delimit ( ThrowCoconut | ThrowBanana ) - of course, this is where MonkeyDo is subclassified into possible choices.

You’ll also observe that because it uses whitespace as a delimiter between instructions, CamelCase is forced on us, as quotes interfere with macro processing and square brackets have a specific overloaded meaning in this subjargon. I haven’t spotted a way round this yet.

And so, to the meat: would anyone like the descriptive tiddler, and if so, where? I’d rather pass it to a GitHub specialist than dabble, I’ve other fish to fry.

@pmario Thanks for the parser, I’ll see if that adds more elucidation.

As a side comment, the Terminal and Non-Terminal concepts were inherited from earlier thinking, to clearly mark process termination in the middle of a complex network. In HTML, the former would de END.
Amusingly, many moons ago, I got my surname into code - my real name’s Jeremy Main. Way on back in 1969, my school class teacher, John Jaworski, won the New Scientist Competition of the Year, a year’s free use of the Imperial College IBM360. We pitched in on bit-mapped fonts and bubble-sorting coding. Anyway, apart from demonstrating the need for a SysOp user override (his code CRLF-SysClock Zero-Loop drowned the room in paper from the fast lineprinter, before they hit the switch!), he worked on “a program that writes programs”, we call them compilers now. He was beta-testing when IBM released BASIC 1, with just 26 integer variables - and snaffled Start() and Begin() as file-handling. No local domains back then. To his fury, he had to look for something else, so I jested, “Why not call it after me, Sir?” He glanced sideways, muttered, “D’you know, it’s not such a bad idea at that” and life moved on. 1974, Kernigan and Ritchie, The C Programming Language - whoa! Can’t be. Another decade later, I’m coding an accounting interface to a Financial Options system, Bjorn Shoustrup’s on the line, he’s now working for Chase, so after dealing with what was going on, I asked him if Bell had looked at earlier compilers - they had - and was Jaworsky’s one - he thought so. And that, dear friends, is how you outbid Buddha. Where he has seried ranks of shaven-headed priests, I’ve rank on rank of microchips… :woozy_face:
Anyway, to conclude for a bit, our implementation doesn’t have a Terminal in quite the way intended, instead it uses a T dead-end.

1 Like