Odd behavior with \end and html comments

[DBH]

I’ve recently noticed something odd that happens with a specific pattern of html comments and pragmas. What I see is not a problem for me but it gives me the feeling that it might be revealing something that is worth mentioning. I’m posting this in case someone who knows enough about how wikitext parsing works might find it interesting or an indication of something not quite right happening. The wiki I am working on is using 5.3.0.

To be clear, it works just fine to have html comments within a procedure or macro. It works just fine to “comment out” a procedure definition. The following, however, does not work-

\define outer()
<!--
\procedure exampleProcedure()
Some example wikitext...
\end

Some commented out instructions...

-->

Some live wikitext...

\end 

The result of that is that the \end for the commented out procedure terminates the outer macro causing everything else that is supposed to be commented out to render as if it was not in an html comment block. That includes rendering the → that should have terminated the comment block. Then everything else in the macro renders as if it was not in a “pragma block.” It does not seem to matter if macros or procedures are used or, in the case I have, if they are mixed.

This isn’t a real problem for me, so I don’t need a fix. I have just changed the \end for my example procedure to \end exampleProcedure and then it is no longer recognized as being for the outer macro and the comment block terminates with the → as it should but it is still valid if I edit the example and move it outside the comment block. Anything that makes the \end for the example invalid as an end of the macro fixes things, for example, \endQQQQ fixes the problem but would need to be changed if I edit the example and move it out of the comment block.

Here are a couple screenshots to show what I mean. First nested macros that show the behavior-

Here is how basically the same setup works well when the inner macro is replaced with a procedure that is terminated by an \end followed by the procedure name-

Skippable background:

In case anyone wonders why I would do something that probably looks pretty strange-

I’ve got a button that creates new tiddlers with some standard wikitext. After creating a tiddler I need to edit it and give some values to some fields. I’m creating a lot of these tiddlers now but eventually it is the kind of thing that I might only need to do once every few months. To make it as painless as possible to do this once I have half forgotten what needs to be done, one of the things that gets put into the newly created tiddler is a set of instructions inside HTML comments (<!-- instructions... -->) Since the final tiddler also needs to contain a procedure that follows a certain structure but is different for each of these tiddlers, I thought it would be nice to include a sample procedure in the commented out instructions. I put all of this into a good old fashioned macro, since it is just text and then set the text of the new tiddler to the value of that macro.

[pmario]

IMO the end of the exampleProcedure needs to look like this: \end exampleProcedure

The code I tested:

\define outer()
<!--
\procedure exampleProcedure()
Some example wikitext...
\end exampleProcedure

Some commented out instructions...

-->

<<exampleProcedure>>

Some live wikitext...

\end

<<outer>>

[DBH]

Thanks for looking a this. My second screenshot shows that \end exampleProcedure avoids the problem, so I agree. (The <<exampleProcedure>> isn’t needed because the tiddler that defines it doesn’t make direct use of it. It only needs to define the procedure for another tiddler to use. That part works just fine for me.)

I guess my point is more that it seems strange to me that the when it is parsed the <!-- is encountered and then the inner \end is acted upon before the --> is encountered. Naively I would think that once <!-- is encountered all that would happen is that each line would be examined looking for a --> and that anything before that would be ignored. Admittedly this is a very niche thing to do but it feels like the parsing is getting “confused” and I just want to throw this out there in case someone thinks it is a sign of something more important.

[TW_Tones]

I do think this is a minor problem if they outer is closed by and ‘\end’ inside a comment. The problem is there will be cases where it will be hard to find the cause or understand the symptoms.

Using comments has being enhanced recently along with the ability to nest definitions and name the ‘\end’ so it is not Supprising that such an edge case would occur.

Fortunatly if I understand this topic, until resolved simply ensuring you use named \end removes the ambiguity, and specifically when nesting even inside comments.

[CodaCoder]

@DBH

I guess my point is more that it seems strange to me that the when it is parsed the <!-- is encountered and then the inner \end is acted upon before the --> is encountered. Naively I would think that once <!-- is encountered all that would happen is that each line would be examined looking for a --> and that anything before that would be ignored.

100% agree. You can stop second-guessing yourself. IMO, it’s a flat-out bug. You should report it on Github.

[DBH]

You’re right, named \end avoids the problem, as long as that is an option (my possibly wrong understanding is that named \end only works for procedures and though I’ve only tested procedures and macros, I suspect that it will happen with any “pragma block.”)

I think you hit the nail on the head when you say that though it feels minor “The problem is there will be cases where it will be hard to find the cause or understand the symptoms.”

[DBH]

Thanks for the input. I’ll report it on GitHub.

[pmario]

@DBH

I guess my point is more that it seems strange to me that the when it is parsed the <!-- is encountered and then the inner \end is acted upon before the --> is encountered.

The TW parser does not work from line to line. It works from “outer structure” to “inner structure”

So if it detects \define xx() it’s starts to search for the first \end it finds. That’s the reason, why nested widgets must have a named end eg: \end abc

Once the outer-structure is found it takes all the content in between start and end and passes it on to a new parser instance.

So I think it will be hard to detect the problem you pointed out.

And it’s a known problem with something like this, where the end-bold is outside the second div

Some text <div>''bold text</div> text end''

[TW_Tones]

A syntax checker preview that matches braces would go a long way to help avoid this kind of error.