Design question: how to configure tool to alter wikis with a specific structure? (Long post!)

#1

Background

I’m still working on the bible wikis discussed in several previous threads. I’m trying to figure out an extension mechanism. The wikis themselves are supposed to be bare-bones. I’m hoping to find a way to make it easy to write extensions which can be enhanced with some simple language packs. And I’d love some advice.

The goal is that a user can write an extension: imagine an annotation tool, a prev/next navigation widget, a favorite verses tracker, a chapter-of-the-day reading list, or a random passage selector. And then this extension (plus perhaps a language pack) could be installed in any of these similarly-structured wikis.

Current extension

For this discussion, I’ll use a simple extension that adds information about the books. The basic wikis have just a flat list of books. But traditionally, the books are broken down into Old and New Testaments, which are further broken down groups such as Law, History, Poetry, Minor Prophets, Gospels, Epistles, etc. This extension1 would add such information to the wiki. Let’s assume that we want a new top-level entry in our default TableOfContents sidebar tab, called (in English) “Sections”, with Old Testament and New Testament underneath it, and Law, History, Gospels and the like under those, and the books themselves categorized one level further down. The “In English” there is important. I want this to allow for simple language packs, in this case, for "Section/s", "Old Testament", "Major Prophet/s", "Gospel/s", and the like. (Note that the main wiki already has a language mechanism, covering among other things, the names of the biblical books. The extension can just use that.)

You can see this output in a demo version of the ongoing work. It looks like this:

image
image1487×1030 137 KB

Design Goals

While an extension may well add many new tiddlers, new templates, new stylesheets, and so forth, it may also add/remove tags or other fields from existing tiddlers or delete them altogether. I would prefer to be able to do this declaratively. I want the extension developer to be able to include something that doesn’t try to directly use Actions or Buttons, but simply says the user wants to:

  • remove the tag “TableOfContents” from tiddler “Book”
  • add tiddler “Section”, with caption “Sections” and the tag “TableOfContents”
  • add tiddler “New Testament”, with tag “Section”
  • add tiddler “Gospel” with caption “Gospels” and the tag “[[New Testament]]”
  • add tag “Gospel” to the existing tiddler “Matthew”
  • add tag “Gospel” to the existing tiddler “Mark”
    …

And the language pack loaded for this library would override such text as “Section”, and “New Testament”, while the overrides for “Matthew” and “Mark” would come from the $:/language/* tiddlers loaded with the language of their wiki.

I’m also hoping such a configuration would be reversible, so parallel to the install button, there would also be an uninstall one. This is simple enough for those things that add information: delete the new tiddler, remove the new tag. But I would have to introduce some cache for those things that delete or overwrite information. That’s not impossible, but it’s complex and I will probably simply ignore that for the moment.

Potential Mechanism

JSON

I started attempting this with JSON that looks like this:

{
  "title": "Section",
  "tags": "TableOfContents",
  "caption": "Sections",
  "_children": [
    {
      "title": "Old Testament",
      "_children": [{
          "title": "Law", 
          "_children": ["Genesis", "Exodus", "Leviticus", "Numbers", "Deuteronomy"]
        },
        {
          "title": "History (OT)",
          "caption": "History",
          "_children": ["Joshua", "Judges", "Ruth", "1 Samuel", "2 Samuel", "1 Kings",
                        "2 Kings", "1 Chronicles", "2 Chronicles", "Ezra", "Nehemiah", "Esther"]
        },
        /* ... */
      ]
    },
    {
      "title": "New Testament",
      "_children": [/* ... */]
    }
  ]
}
Full File

{
“title”: “Section”,
“tags”: “TableOfContents”,
“caption”: “Sections”,
“_children”: [
{
“title”: “Old Testament”,
“_children”: [
{
“title”: “Law”,
“_children”: [“Genesis”, “Exodus”, “Leviticus”, “Numbers”, “Deuteronomy”]
},
{
“title”: “History (OT)”,
“caption”: “History”,
“_children”: [
“Joshua”, “Judges”, “Ruth”, “1 Samuel”, “2 Samuel”, “1 Kings”,
“2 Kings”, “1 Chronicles”, “2 Chronicles”, “Ezra”, “Nehemiah”, “Esther”
]
},
{
“title”: “Wisdom”,
“_children”: [“Job”, “Psalms”, “Proverbs”, “Ecclesiastes”, “Song of Solomon”]
},
{
“title”: “Major Prophet”,
“caption”: “Major Prophets”,
“_children”: [“Isaiah”, “Jeremiah”, “Lamentations”, “Ezekiel”, “Daniel”]
},
{
“title”: “Minor Prophet”,
“caption”: “Minor Prophets”,
“_children”: [
“Hosea”, “Joel”, “Amos”, “Obadiah”, “Jonah”, “Micah”, “Nahum”,
“Habakkuk”, “Zephaniah”, “Haggai”, “Zechariah”, “Malachi”
]
}
]
},
{
“title”: “New Testament”,
“_children”: [
{
“title”: “Gospel”,
“caption”: “Gospels”,
“_children”: [“Matthew”, “Mark”, “Luke”, “John”]
},
{
“title”: “History (NT)”,
“caption”: “History”,
“_children”: [“Acts”]
},
{
“title”: “Epistle”,
“caption”: “Epistles”,
“_children”: [
“Romans”, “1 Corinthians”, “2 Corinthians”, “Galatians”,
“Ephesians”, “Philippians”, “Colossians”, “1 Thessalonians”,
“2 Thessalonians”, “1 Timothy”, “2 Timothy”, “Titus”,
“Philemon”, “Hebrews”, “James”, “1 Peter”, “2 Peter”, “1 John”,
“2 John”, “3 John”, “Jude”
]
},
{
“title”: “Prophecy”,
“_children”: [“Revelations”]
}
]
}
]
}

I had two problems with this approach. First, my JSON-in-TW skills are mediocre, and I was having difficulty looping over array indices with <$list> widgets. I’m sure I could eventually figure that out, but second, I was not happy with the built-in assumption that Old Testament, nested this way inside Section would automatically get a tag of "Section".

As I tried to untangle that, I ended up considering a second approach:

Structured Field-Names

Thinking more on the lines of the declarative list above, I thought about something more like this, a tiddler with a number of fields:

section.title: Section
section.tags: TableOfContents
section.caption: Sections
old-testament.title: Old Testament
old-testament.tags: Section
law.title: Law
law.tags: [[Old Testament]]
Genesis.tags+: Law
Exodus.tags+: Law
Leviticus.tags+: Law
Numbers.tags+: Law
Deuteronomy.tags+: Law
history-ot.title: History (OT)
history-ot.caption: History
history-ot.tags: [[Old Testament]]
Joshua.tags+: [[History (OT)]]
Judges.tags+: [[History (OT)]]
Ruth.tags+: [[History (OT)]]
1 Samuel.tags+: [[History (OT)]]
2 Samuel.tags+: [[History (OT)]]
...
Full File 
title: Sections Config
tags: TiddlerActions
section.title: Section
section.tags: TableOfContents
section.caption: Sections
old-testament.title: Old Testament
old-testament.tags: Section
law.title: Law
law.tags: [[Old Testament]]
Genesis.tags+: Law
Exodus.tags+: Law
Leviticus.tags+: Law
Numbers.tags+: Law
Deuteronomy.tags+: Law
history-ot.title: History (OT)
history-ot.caption: History
history-ot.tags: [[Old Testament]]
Joshua.tags+: [[History (OT)]]
Judges.tags+: [[History (OT)]]
Ruth.tags+: [[History (OT)]]
1 Samuel.tags+: [[History (OT)]]
2 Samuel.tags+: [[History (OT)]]
1 Kings.tags+: [[History (OT)]]
2 Kings.tags+: [[History (OT)]]
1 Chronicles.tags+: [[History (OT)]]
2 Chronicles.tags+: [[History (OT)]]
Ezra.tags+: [[History (OT)]]
Nehemiah.tags+: [[History (OT)]]
Esther.tags+: [[History (OT)]]
wisdom.title: Wisdom
wisdom.tags: [[Old Testament]]
Job.tags+: Widsdom
Psalms.tags+: Widsdom
Proverbs.tags+: Widsdom
Ecclesiastes.tags+: Widsdom
Song of Solomon.tags+: Widsdom
major-prophets.title: [[Major Prophets]]
major-prophets.tags: [[Old Testament]]
Isaiah.tags+: [[Major Prophets]]
Jeremiah.tags+: [[Major Prophets]]
Lamentations.tags+: [[Major Prophets]]
Ezekiel.tags+: [[Major Prophets]]
Daniel.tags+: [[Major Prophets]]
minor-prophets.title: Minor Prophets
minor-prophets.tags: [[Old Testament]]
Hosea.tags+: [[Minor Prophets]]
Joel.tags+: [[Minor Prophets]]
Amos.tags+: [[Minor Prophets]]
Obadiah.tags+: [[Minor Prophets]]
Jonah.tags+: [[Minor Prophets]]
Micah.tags+: [[Minor Prophets]]
Nahum.tags+: [[Minor Prophets]]
Habakkuk.tags+: [[Minor Prophets]]
Zephaniah.tags+: [[Minor Prophets]]
Haggai.tags+: [[Minor Prophets]]
Zechariah.tags+: [[Minor Prophets]]
Malachi.tags+: [[Minor Prophets]]
new-testament.title: New Testament
new-testament.tags: Section
gospels.title: Gospel
gospels.caption: Gospels
gospels.tags: [[New Testament]]
Matthew.tags+: [[Gospel]]
Mark.tags+: [[Gospel]]
Luke.tags+: [[Gospel]]
John.tags+: [[Gospel]]
history-nt.title: History (NT)
history-nt.caption: History
history-nt.tags: [[New Testament]]
Acts.tags+: [[History (NT)]]
epistles.title: Epistle
epistles.caption: Epistles
epistles.tags: [[New Testament]]
Romans.tags+: Epistle
1 Corinthians.tags+: Epistle
2 Corinthians.tags+: Epistle
Galatians.tags+: Epistle
Ephesians.tags+: Epistle
Philippians.tags+: Epistle
Colossians.tags+: Epistle
1 Thessalonians.tags+: Epistle
2 Thessalonians.tags+: Epistle
1 Timothy.tags+: Epistle
2 Timothy.tags+: Epistle
Titus.tags+: Epistle
Philemon.tags+: Epistle
Hebrews.tags+: Epistle
James.tags+: Epistle
1 Peter.tags+: Epistle
2 Peter.tags+: Epistle
1 John.tags+: Epistle
2 John.tags+: Epistle
3 John.tags+: Epistle
Jude.tags+: Epistle
prophecy.title: Prophecy
prophecy.caption: Prophecy
prophecy.tags: [[New Testament]]
Revelations.tags+: Prophecy

and then some code that is supplied this tiddler collects all this information. For instance:

section.title: Section
section.tags: TableOfContents
section.caption: Sections

would be gathered together by their shared “section” prefix, and because of the existence of .title would be a new tiddler, with the given tags and caption. (Those prefixes are arbitrary, they could just as easily be numbers.)

And a line like this:

Joshua.tags+: [[History (OT)]]

would simply add a tag to the existing 'Joshua' tiddler. I would presumably allow a tags- suffix as well to remove one, I might also make the new tiddler signal more like section.title+, as it would make the parallel removal, section.title- more obvious and more consistent. I have three initial concerns over this. First, the data is somewhat more repetitive than the JSON version, which I never appreciate. Second, I’ve generally tried to be very conservative in my field-naming convention, generally preferring lower-case letters and hyphens only. This would add spaces, full stops, and plus signs as well. I don’t think that’s a problem, but I know there used to be issues with this, and I worry. Third, this is squeezing structured data into a flat format; it may simply feel awkward. None of these concerns are overwhelming, though.

I don’t know how to do all of this yet, but I don’t see any roadblocks. I’d likely do a POC in JavaScript first, then consider a wikitext version. But I’ve come up with one more interesting possible mechanism.

Bundle the tiddler deltas into a CompoundTiddler

A third possibility would be to use actual tiddlers with all the change information, stored together in a CompoundTiddler, and when this is consumed, new tiddlers are cloned into the main wiki, and other have their fields updated base on this information.

I don’t have any mockups of this, and haven’t thought it through as carefully. Moreover, I’m not even sure if it’s possible. Another recent topic asks about that. If it is possible, though, it would be the solution perhaps easiest for someone familiar with TiddlyWiki to use.

Question

Does any one of these seem like a better design? Does one of them feel more in-tune with TiddlyWiki? Is there another technique that I should be considering? Any other suggestions?




1 I’m using “extension” and not “plugin” because this probably can’t be comfortably done via the plugin mechanism. These tools will add tiddlers that should not be shadows; they will alter existing tiddlers to, say, add or remove tags.

#2

I would suggest creating a config tiddler with fields and buttons.
You can then use actions to update the tiddlers which need updating.
I’d create an Extensions Config tiddler tagged with $:/tags/ControlPanel/SettingsTab so that your users can access via the Settings tab in the control panel.
You could then have the Extensions Config tiddler read from json tiddlers tagged with the tag for your extensions config. Perhaps have one of the json variables configured so that users can write custom actions that can be used to extend the actions in the Extensions Config and another for actions to reverse the changes. a simple filter could exclude extension files that don’t have undo actions (not sure how easy it would be to validate that the undo actions work correctly).
If you want to enforce undo functionality then you could create a set of action templates that can use variables to customise the actions and pre-code the undo version of the actions.

#3

I can’t believe I wrote that entire long post and didn’t mention the UI I was considering to render and act on whatever configuration I came up with. The idea is pretty simple:

image
image1000×744 58.5 KB

However the configuration works, a similar UI should be able to accept it. Or at least that’s the theory.

Yes, that was the plan. I don’t expect that part to be difficult.

Oh, that’s a good idea! I hadn’t really thought about it, but that makes sense.

Yes, that’s the plan exactly. Although whether that configuration is JSON, tiddler fields, or a CompoundTiddler is still a big question.

I’d like to stay away from that. I think an extension will consist of two things: the usual tiddlers, templates, stylesheets, etc, that are necessary to operate the extension, and this sort of declarative configuration. That configuration is mostly about modifying the tiddlers of the underlying wiki and to create tiddlers we need to internationalize. The tiddlers I create won’t be called “Section”, “New Testament”, “Gospel”, etc. in Spanish or French or Chinese. But the templates and such would likely not be handled by this mechanism.

In general, nearly impossible. But here we can give the user some control over these actions, even if it’s just a checkbox that dictates whether to take this particular step. The biggest concern is an action for the deletion of a tiddler. (This particular extension might conceivably want that. There is a tiddler in the baseline wikis, a TagTiddler called (in English) "Book". It is there only to serves as a tag and to be used in the Table Of Contents. This Sections extension serves the same role in the TOC, and might want to delete that tiddler.) But if a tiddler is deleted entirely, and the user decides to remove the extension, then there is no way to restore the wiki to its pre-extension status. So I think there might have to be a way to keep such tiddlers in a temporary store, where they don’t interact with the current wiki, but from which they could be revived if necessary. I’m simply not ready to deal with that complexity until I get the basics working.

Thank you very much for the great feedback!

1 Like
#4

this is why I suggested:

you could have a “delete function” which archives it using some method. and then have an uninstall button which reverses the install actions including restoring the deleted tiddler from the archive… like you said, probably to complicated for your immediate needs