Best way to handle lists/items (ul, ol, li)?

#1

I love to utilize lists in various situations, and usually I ended up using raw html codes instead of markdowns TW provides. I wonder if there is a better way to handle the following use cases:

  1. When I want several paragraphs or other blocks (say a math block – this almost always appears in academic notes) then I always have to manually enclose them by <div>. Why can’t the wrapper be smart enough to detect such hierarchy by indentations or so?

  2. Styling. Say I want to give a style to the ul, then @@.style [[lists]] @@ does not work. Or, if I want to give a style to a specific item, *item 1 \n @@.stlye *item2 @@ breaks the listing. What is the best solution which makes the document still pleasing to read for humans?

Yes, this is missing in TiddlyWiki wikitext
#2

What do you mean with “the wrapper” and even what do you mean with “paragraphs or other blocks”? Do you mean, respectively, the parser and content of html tags?

Typically you need an empty row before @@ commands to ensure block mode. See if that helps.

While I agree it doesn’t work properly, it is unclear if you with \n mean <br> or do you mean that it is on a new line but just didn’t want to type it as such here?

As far as I know, multiline content within a ul or ol has to be enclosed in a div.

#3

By wrapper, I meant something that parse the markdown and wrap it as html codes. For the latter, it would be clear from the following use cases:

  • A
    B
  • C
    or
  • A
    $$ latex $$
  • B

Humans can understand the above without any additional syntax. Why not the parser/wapper?

Yeah \n was some pseudocode not to waste the paper, but sorry if it confused you. That was not my point though. Most of my tiddler are very messy with huge number of <div> syntax and at some point I decided to use the raw html code. Or if I need to style one item, I have to use the full ul/li synax which makes the situation worse.

#4

I think one problem is that linebreaks are not interpreted as hard linebreaks but demands an empty row. It would be interesting if soft linebreaks could be accepted in bullets to break the line, but I suspect this is not backwards compatible so it won’t be accepted. But I would like this because I also suffer repeatedly from the problems you bring up.

But fellow @pmario has created space-space-newline which does enable soft linebreaks inside lists so maybe this will be of use!

#5

I think it can be made backward-compatible if naively implementing how humans understand the list structure.

For example, (below, all line breaks are hard)

  • A

B

  • C

is easily undestood by both the parser and humans that there are two lists A/C and an intermediate paragrah B. On the other hand, let’s say we have

  • A

    B

  • C

D

  • E

Humans interpret this as there are two lists ABC (while A and B are the same item as they have the same indentation although separately by a linebreak)/E, with intermediate paragraph D. The reason why we think C is the continuation of the previous list although there is a hard linebreak, is because it is extremely unlikely that two distinct lists are consecutive without any intermediate paragraph in any human writing.

I can modify the current wrapper to work as intended, that I will probably do. But it still doesn’t solve the styling issue.

#6

With respect @minjaep I think you are yet to fully understand the implications of using tiddlywiki’s markdown or wikitext standard.

As you said yourself;

It would help if first we return to seeing if we can show you how to get what you want from Wikitext, before looking for more hacky solutions. See the internals plugin below.

Well, first it is already very smart interpreting wikitext based on a long history of needs. In fact when making lists HTML demands we wrap a number of line items <li>an item in the list</li> inside a <ol><ol> or <ul></ul> the wikitext parser is doing all this work for us.

So rather than answer a question you ask I would like to share two methods that show you perhaps it is not as complex as you think.

One. Most wikitext symbols like # ! * etc… can be followed by .classname. so in a stylesheet define classname

.classname {
   various css and styles
}

So you can do this

* one
*.classname two
* three

Two. Now consider the # rather than the * because we may want the numbers to continue. This illustrates how in a list we need continuity, note clear when we use <ul>.

# one<br>Second line 
# two<div> Div within `<li>` </div>
# three<p>A whole paragraph</p>
# Last item after a paragraph within the last item

This is in fact demanded by logic. If you want something to be nested within something you need to somehow nest it, be it a div or otherwise, or as in my example a <br>. For something to be within a list item it must be within the tags <li>here</li>

Now all this comes about because wiki text is converted to HTML to display in the browser.

If you understand wikitext and a little of HTML , both to assist learning and deal with more complex cases I suggest installing the Internals Tool already installed on TiddlyWiki.com (or just use it there) then select the preview Raw HTML to see what is occuring. This gives you insight into what the wikitext parser does and how to get it to accommodate other html.

Side note;
I have long struggled to get the core developers to recognise we need a wikitext character to indicate a sentence, or div ie it is incorrect to use the wikitext “;” to do this and it gets bolded and a blank line follows it anyway. There should be a way to indicate an otherwise single line is a sentence (and the subsequent lines (\n) will not be wrapped up (in a paragraph) and which which we can add the .classname.

Much of this is addressed with the Custom Markup plugin, however this an not the solution here.

1 Like
#7

Thanks for reminding about it. I was forgetting about some way to handle styles so that probably solved my issue.

I don’t understand your point with this example. It is not ambiguous at all that there are 4 items continued in one oredered list, and the first of them has two lines. As said, we can use indetation to distinguish if the second line is continued from the first line in your example. That means,

# one
Second line

and

# one
  Second line

can be handlede differently. Or even

# one

  Second line

can be understood as this second line is continued from the first item. I don’t understand why I should do

# <div>one

Second line</div>

to achieve what I want, thinking about the philosophy of using markdown instead of html.

#8

Consider

# one
# two

We move on to the next item using new line - ie the end of the line says this is the end of the item.

Rather than use <div> consider the other possibility I suggested <br> this says

introduce a line break but don’t end the current item.

# one<br> Second line 
# two
# three
Is this a new paragraph or line? or part of list item 3.?

renders as

   1. one
      Second line
   2. two
   3. three

Is this a new paragraph or line? or part of list item 3.?

As I suggested before there are real and meaningful reasons in the wikitext rendering process that your proposed method is not the way it works. For example the last line of the above list, how can wikitext determine the bullet/number list has finished? Is it a new paragraph or part of the lists item? Currently it uses the <newline> to determine this.

FYI: There is an editor toolbar button available to click and insert <br> available.

I think we can call the <br> a soft line break, from memory word processors also implement this with a “ctrl-enter” rather than “enter”.

#9

I don’t know if you intentionally ignore it, but I keep saying we can use indentation to determine such things. For me, your example must treat the last line as a line and a part of the third item, which is consistent with the first line.

There are three different things which can be interpreted differently without ambiguity.
Case 1:

# one<br> Second line 
# two
# three
Is this a new paragraph or line? or part of list item 3.?

The last line should be parsed as

# three<br>Is this a new paragraph or line? or part of list item 3.?

Case 2:

# one<br> Second line 
# two
# three

Is this a new paragraph or line? or part of list item 3.?

The last line should be parsed as a new paragraph after ending the previous list.

<p>Is this a new paragraph or line? or part of list item 3.?</p>

Case 3:

# one<br> Second line 
# two
# three

  Is this a new paragraph or line? or part of list item 3.?

The last line should be parsed as

<li><p>three</p>
<p>Is this a new paragraph or line? or part of list item 3.?</p></li>

because it is indented as the previous item in the list. There is no need to introduce fancy wikitext to distinguish these cases.

#10

P.S. You may argue the following is ambigous:

# one-first paragraph

  one-second paragraph

# two

if the two should be the first item of the new list. What I’m arguing is that it must be interpreted as the second item still, as I have never seen a writing style using consecutive ordered lists without any intermediate paragraph explaining them.

So what I’m expecting is something like

<ol>
<li><p>one-first</p><p>one-second</p></li>
<li>two</li>
</ol>
#11

Styling lists is simple.

*.myClass some text

You only need to define a tiddler tagged $:/tags/Stylesheet and add this text

.myClass {
  background: yellow;
}
#12

Not at all intentional, please forgive me. In fact I did not so much as ignore it as understand simple indentation (leading spaces) are ignored by the parser as a rule.

The indentation method is not currently in use and in fact a lot of the wiki text depends on ignoring it, ie; leading spaces on a line, especially when you indent widget code etc… It ignores indentation so it does not introduce unwanted white space, but the code can be formatted in a readable way.

If you install the codemirror plugin you can, I believe, indent with tabs, perhaps a tab character at the beginning of a line to trigger the response you ask for. But I expect this too could have unwanted side effects.

What you suggest is a good idea, but not necessarily in keeping with the current wikitext markup. Perhaps some of our developer’s could consider if it were possible, perhaps the ability to enter a soft line break like other word processors ctrl-enter, and introduce an invisible soft line break rather than the <br>, so it also displays as such in the editor…

#13

Hi minjaep,

You are right, there are some inconsistencies in list handling. While it’s easy for humans to interpret your examples in the right way. It’s not easy for a simple parser like ours. The parser has no info about “user intent”, “probabilities” and “context”.

Usually our parser works with “binary” decisions like that.

Rule 0: … do things …
Rule 1: check start of the line for a special marker like: “! !! * # ; : >”
Rule 2: If marker found: check for next “new line” → that’s the end of the element.
   Rule 2.a: check the text of the element for “bold”, “italic” formatting and apply
Rule 3: If no marker found: check for “next” 2 “new lines” → that’s a paragraph
   Rule 2.a: check the text of the element for “bold”, “italic” formatting and apply
Rule 4: at current position start with Rule 0

As you can see, there is a difference between paragraph handling and “special marker” handling. While 1 nl is OK for headings, it could have been different for list elements. … BUT … that would have changed the wikitext syntax quite a bit.

If 2 nl are used to end a list element. Users would have been forced to create them like that:

* line 1

** line 2

* line 3

While for me and may be you, that would make sense, we would have had a lot of other users that would have called us crazy.

Some users prefer lists like that

*element1
**element2
*element3

I personally think, that’s completely unreadable. … We are not able to change that handling in a backwards compatible way.

… we would be able to define some new list elements, that behave differently.

BUT it would not save your “Case 3” problem, because that would need information about “user intent” and “context”, which we don’t have.

#14

Our parser doesn’t “look back” for performance reasons. It only “walks” forward … in a single run. So there is no “second” run to change any output.

#15

I’m thinking of something, but I’m not sure, if it has negative side effects. … A lot of testing would be needed.

eg: If you write a list as follows in standard wikitext syntax.

* element 1

* element 2

* element 3

It will work. So it would be possible to copy/paste some code from your “special” wiki to a “standard” wiki and it still works.

We could create a new list parser named eg: mylist and force new elements to have 2 line-breaks. So the above format would be a new standard. … for your wikis only.

But it would allow you to do something like this:

*.hard-linebreaks line 1
line 2
line 3 in the same list element

* line 1 in next element

where hard-linebreaks is a style definition like this:

.h, .hl, .hard-linebreaks {
  word-break: normal;
  word-wrap: break-word;
  white-space: pre-wrap;
}

and the result would look like this

grafik
grafik1399×609 17.5 KB

I’m not sure, if it would work with KaTex elements.

To activate the mylist wikirule it would be needed to write something like this:

\rules except list

*.hard-linebreaks line 1
line 2
line 3 in the same list element

* line 1 in next element

So the default list-parser will be deactivated and mylist will take over. … wikitext parsers are activated alphabetically. So if list is deactivated mylist will get the text to analyse. …

What do you think?

#16

I have been a bit too enthusiastic. … We can’t do this, without modifying the core :confused: … More investigation is needed

#17

Thanks for sharing your insight! And thanks for the temporary solution you suggested.

But I don’t agree it needs a second run to take care of Case 3. Whenever you face with #, you put <li> and then read the texts until you face with 2nl. Currently, the parser ends the whole list by putting </li></ol> at this point.

What I’m asking is just read it a bit more, to see if there is indentation (say &nbsp&nbsp right after 2nl. If that is the case, the parser does not end the current <li> and try to find the next 2nl without such indentation. It is just a simple logic (and I will modify my core wikitext parser this way) without any performance drawback.

Also, you can make your parser so that it puts </ol> only when you see some non-special characters after 2nl. This will make the change of logic backward-compatible. You may use either

# a
# b

or

#a

#b

Because the parser in the latter case, it will not close the list as you see # after the 2nl, which is considered as continuation of the previous list.

#18

You don’t need to read the indentation everywhere. You can only read it while the parser detect a list. I suggested an algorithm above that I think easy to implement and backward-compatible. Maybe it is better for me to write some plugin for it as the proof of concept.

#19

Hi @minjaep and welcome to the community.

If you have a backwards compatible change in mind for the wikitext list parser that would improve the versatility of the markup, a PR would be very welcome and would allow for better discussion of the implementation:

Edit: see also https://tiddlywiki.com/#Contributing

1 Like
#20

I have thought of this a little more last weekend, and decided to use Markdown plugin instead. As explained at https://tiddlywiki.com/plugins/tiddlywiki/markdown/ the lists are already implemented as I desired. They indeed use indentations (tabs) to distinguish paragraphs/items within lists.

Another reason I chose Markdown over Wikitext is that the former is now becoming a standard for modern text/markdown editors. Of course there are several variants of Markdown but they are mostly compatible, e.g. most use 1. instead of # for the ordered lists.

As the above plugin tosses the result to Wikitext parser for the TW specified features like macros, I found no reason not to use it. Maybe it has performance disadvantage that I cannot recognize, but I use KaTeX anyway which parses the text before Wikitext anyway.