Tools to build outline-like tiddlers

Question

Are there tools out there to help build tiddlers that often have outline-like structures, but do not have a clean, consistent outline format? I’m trying to build this so each section/subsection/sub-subsection has its own URL, so I’m breaking many of them down fairly granularly. But I’m finding it tedious to create them. When I hand it over to a non-TiddlyWiki-savvy user, I would really like this to be cleaner to use.

Background

I’ve been very slowly working on building a wiki to gather together the 150 or so PDF policies for a regional school system (of which I’m a Board member) and covert them into a single wiki. The main goal is simply consolidation into a single file, but an important secondary goal would be to have everything someone might want to point to have its own URL.

But the policies do not have a consistent format. Some are just a few paragraphs, with no further breakdown. Some are full-fledged outlines with sections named, e.g. IV.A.2.c. But many of them fall in between, with some outline-structured sections, and other unstructured blocks of text. And the numbering for these varies from one policy to another. Sometimes the top-level breakdown is I / II / III / IV, sometimes it’s A / B / C / D, and sometimes it’s 1 / 2 / 3 / 4. For example see Policy 3265 to understand the sorts of structural complexity I must be able to handle.

I think I have a reasonable way to break these down and display them, but I don’t have good tools to help me with that, or to help the eventual maintainer.

Current markup

Right now, the trick is that each outline level includes its own content, and then calls a procedure to display all of its children, and then includes any final direct content. Children know their parents through a parent field. If they’re in an outline-like section, they also have a marker field that might contain VIII or C or 7, for instance. (This is probably necessary, since it’s quite possibly that a later revision will remove C, but not renumber D - G; I can’t use the parent’s type to generate a sequential set.)

For instance, Section 2 (my own designation, not in the actual document) of Policy 1520, looks like this:

title: Policy1520(s2)
tags: Policy1520
caption: Guidelines
parent: Policy1520

!!! Outside Funded Athletic Teams

Any athletic team which is funded in whole or in part by any Booster club or other outside 
organization for the benefit of Region 8 students must adhere to the following guidelines in
order for the program to be considered a Region 8 athletic program.

<<children list>>

(The tags in there are for the current TOC. It is duplicative of parent, and I will probably eventually remove such tags.)

So for this one, there is introductory material and the call <<children list>> The parameter to children specifies which of list styles to use. Currently there are six, and I expect a handful more by the time this is completed. list is one of the most simple of them.

One of its children (#6) looks like this:

title: Policy1520(s2)(6)
tags: Policy1520(s2)
caption: Operational expenses
marker: 6
parent: Policy1520(s2)

All operational expenses as described by the administration will be paid out of student activity fund.

The children procedure looks pretty much like this:

\procedure children(type)

  \procedure definition()
    <div class="definition">
      <div class="term"><$link>{{!!marker}}. {{!!caption}}</$link>: </div>
      <$transclude $mode="block"/>
    </div>
  \end definition

  \procedure outline()
    <div class="level">
      <div class="heading"><$link><% if [<currentTiddler>has[marker]] %>{{!!marker}}. <% endif %>{{!!caption}}</$link></div>
      <$transclude $mode="block"/>
    </div>
  \end outline

  \procedure list()
    <div class="list-item">
      <div class="marker"><$link>{{!!marker}}</$link>. </div>
      <$transclude $mode="block"/>
    </div>
  \end list

  <!-- ... more here ... -->

  <$list filter="[parent<currentTiddler>!has[draft.of]sortan[]]" >
    <$transclude $variable=<<type>> $mode="block" />
  </$list>
\end

Tooling

All that works fine to display a policy broken down into bite-sized, addressable pieces. But it does not make it easy to create or modify one. I’m hoping people have suggestions for tools to look to for inspiration on this. I don’t expect there is a ready-made solution, though. There are several possibilities I can imagine, but they all revolve around adding new subsections to the current section, or as siblings to the current one. It would be really nice if they would allow me to insert into the middle of a list, automatically renumbering the subsequent entries

Streams

The elephant in the room, obviously, is Streams. I’ve never used it before, never really having the need. But I’ve looked at it many times, and I just reviewed it more carefully. I haven’t dug into the implementation yet, and I don’t have any sense of how difficult it would be to adapt to my needs.

What I love

  • the inline editing of multiple tiddlers,
  • the easy changing of levels,
  • the straightforward drag-and-drop.

What I’m missing

  • control over list styles and markers
  • the edit toolbar (not for me – I really don’t use it – but for eventual maintainers.)

What worries me:

  • I don’t like this style:

    • each node that has children, contains their titles in a list in the field stream-list
    • each node that has a parent, contains the parent’s title in the field parent

    I feel we should never do both of these, as they too often lead to conflicts. Of course this is harder for Streams, which doesn’t have much besides that list for sorting. For my case, I have titles and markers that can offer the correct sorting (once I get those pesky Roman Numerals sorted out.)

  • Most development seems to have stopped three or four years ago. I don’t know if @saqimtiaz is planning on supporting this and making changes.

Overall

There is a lot to be learned from here. But it’s not as though I could simply make a few tweaks to Streams and get the behavior I want.

Other tools

I don’t know of any. Does anyone have suggestions? Again, I’m not looking for exact matches to my requirements (although those suggestions would be extremely welcome!) I’m just looking for wikis that have overlapping ideas that I could steal investigate.

Streams works, it has toolbar, or use GitHub - tiddly-gittly/streams-outliner-lib: Lightweight Outliner component extracted from Streams plugin, for use in other plugins. as a library to build your own UI, you could customize what it shows.

It certainly does work. It’s an extremely impressive feat!

The toolbar is there only if you drill down into the specific tiddler you want to change, and then the context is lost. This is true also of my current process, but Streams thoroughly blurs the lines between view and edit, in a very useful way. But the edit toolbar is not available in what I think of as the main editing interface, which is technically a view. For my own usecase, I’m trying to imagine how it might be made available.

Ahh, seeing it active in your version made me search again in Streams, and I simply missed that there was a configuration option for it. Nice!

Thank you. I will dig into it. But my first impressions are confused. The screenshot below shows how the plugin’s README looks, when I try to expand a few of the items:

Is that what you intended? Most of the “This renders as” sections show only bullets… or bullets and breadcrumbs. And the breadcrumbs seem to double up:


As far as I can tell, there are two main differences between my approach and Streams/Outliner:

  • My various put-children-here procedures offer differentiation on list styles, so I can nest I / II / III inside item B as necessary. I also think I have more styles available, built-in. (This is due to the source documents being developed by various people over several decades; there is no overall consistency to them)

  • I use the parent node on sub-elements to select the children to show and use the marker or title fields to sort them. Streams and Outliner use a list field on the parent to select and order the children.

The first point is essential to the output I want to create. But on the second one, I am quite flexible. Being able to use/develop tooling for one or the other would probably be the deciding factor.

Note that https://charter2024.andoverct.info, which uses an earlier version of these techniques, is pretty simple to maintain, because it has a very consistent hierarchy. It’s mostly the mixed hierarchies in the current document that are causing concern.

1 Like

Hi Scott, the current plan is for a rewrite of Streams from the ground up after the TW v5.4.0 release.

I understand the concern. In practice for users who author and manipulate their content via Streams this has proven to be robust but is indeed error-prone if you want to manipulate your data otherwise. As you mention, the list field is needed to store the order of the children of a node. The parent field is somewhat redundant but helps signifcantly with performance when rendering breadcrumbs and for keyboard navigation in very large streams with hundreds of nodes.

As such there are no current plans to move away from this dual linking at present. I have long been considering the merits of storing this relational information in a config tiddler to avoid modifying a tiddler in order to annotate it with Streams. The caveat is of course that this makes exporting the component tiddlers of a stream to another wiki significantly more complex.

Overall, note that a lot of the complexity in Streams comes from trying to offer a fluent keyboard-driven writing and editing experience. If the intent is just to present pre-authored content that is not going to be edited often, it is likely more complex than needed.

1 Like

That’s good to hear. I haven’t had a real use for Streams in my TW work so far, but it’s really nice to know it’s available if I do need it. I’m glad it will get more attention.

Are there planned features for 5.4.0 that you think will make this codebase simpler?

Yes, and my usage of TW has all been about manipulating data in numerous ways. It’s not something I find problematic for Streams to do, but it’s also not something I really want to replicate.

I have no need for that in the current project, but I have done that successfully in a few cases, and I’m considering it for an upcoming one. It’s a nice technique when it works. Indirection can be extremely powerful (as who says, Wheeler, Stroustrop, Koenig, Lampson, all of them? :slight_smile: )

Certainly. There are overlapping ideas between what I’m trying and what Streams does. I’ve never considered incorporating streams. Too many of the ideas are incompatible. But it’s been very useful to look to for inspiration.


I have completed enough of the content conversions that I am stepping back to see if my current techniques can be refactored. I have come to the conclusion that some complexity I was trying to replicate is accidental, and that I probably don’t need to replicate it, which loosens a number of design constraints for me.

Part of this refactoring will need to incorporate useful maintenance features. One this whole thing is complete, if the Board chooses to use this (it’s entirely built on spec), the maintenance will fall to people without any TW knowledge, so I have to think pretty hard about how to make it as user-friendly as possible.

I’m struggling to decide if I can reduce all my duplicative work, and do everything from the title structure of something like Policy3265(V)(s1)(B), inferring all parent-child relationships and ordering from that string. Clearly it’s a child of Policy3265(V)(s1), and it comes after Policy3265(V)(s1)(A) and before Policy3265(V)(s1)(C). (I’ll have to finish my Roman Numerals thing for that, but it’s already close to what I want.)

Thank you for the reply. And thank you for Streams… and for all your work in the community!