Curious discovery - subtiddlers in non-plugins

Developers
#1

For the curious, technical and tiddlywiki hacker, I just discovered something interesting, that provides additional code patterns to pack tiddlers.

  • Don’t bother reading unless you want to hack, once I understand it better if any “cool tools” can be developed I will publish separately.
  • I am possibly “late to the party”, but thought I would share anyway
  • It involves being able to address and import tiddlers from within another tiddler.

Background;

  • Plugins actually contain tiddlers in a JSON format. They have a plugin-type field of plugin, but they are just tiddlers, containing “subtiddlers”.
    • Normally we access the tiddlers within plugins as shadow tiddlers.
    • If we edit shadow tiddlers, they become tiddlers, with the same name but outside the plugin.
  • You can list the shadow tiddlers within a plugin tiddler with the plugintiddlers Operator
  • A few widgets allow the reference to a named subtiddler such as transclude and view widget.
    • You need to give the tiddler name for which it is a subtiddler of.
    • These also accept a field parameter to target any field within a shadow tiddler
  • There are plugintiddlers and subtiddlerfields operators as well, to access shadow tiddlers in a plugin.

My simple discovery;

A tiddler containing subtiddlers need not be a plugin, it can be a regular tiddler (containing the subtiddlers) ie not have plugin-type=plugin (if you change this field back it again becomes a plugin).

  • The above widgets and operators can still work against such a tiddler.
  • The tiddlers in such a “package” are not accessible as shadows, but they are via the above operators, as subtiddlers. Note if the original plugin is in the wiki they will still be shadows due to that (needs research)
  • Another way to generate a tiddler containing subtiddlers is to import or drop some tiddlers on your wiki, rename $:/import and delete its fields (especially plugin-type)
  • You can Import all packed tiddlers as “tiddlers”, as follows, note it will replace the original package tiddler, with a list of imported tiddlers as does the $:/Import process.
<$button message=tm-perform-import param="packagetitle">
Import packagetitle
</$button>

Why is this interesting?

  • Its a way to pack tiddlers into single tiddler for access from your code, to the field level if needed.
    • Including without turning them into tiddlers.
  • Its another form of data tiddler
  • You can clone or rename it to $:/Import and set plugin-type to import, to its contents.
  • You can clone or rename it set plugin-type=plugin to access its contents as shadow tiddlers.

Questions arising;

  • How can we save / pack a filtered list of tiddlers into a new package tiddler, with or without plugin-type set
  • How can we extract a single tiddler from such a package without making it a plugin first?
  • I believe the format of subtiddlers is somewhat different to a JSON file of tiddlers, which can also be imported, and raw JSON can be imported using the deserializer option on the browse widget <$browse deserializer="text/plain"/> will import it as a tiddler type text/plain and not import the tiddlers within.
    • However I can not access its subtiddlers or perform and import from that tiddler
    • It uses a different format.
#2

See the following post. It describes the export-json format and also shows the format of $:/Import and discusses them.

I may have some more thoughts later.
-mario

#3

I remembered and reviewed that, it is a nice overview of the workings and a valuable resource, but I could not extract from that sufficient understanding to learn how to move content “into and out of, or between the various formats”, nor could I understand the templates to use, to generate a tiddler containing multiple tiddlers in one or more of those formats, or subsequently extract them, in wiki, not just on import.

  • Because tiddlers are easily interchanged between wikis, via drag and drop and export import, I figure we can reuse these formats to get different outcomes.
  • The ability to understand these formats and develop solutions that leverage them should not need much, or any new code.

Examples may include;

  • Archiving tiddlers to another tiddler but not appear as shadows
  • Identify tiddlers changes in a plugin tiddler (ie edited) and insert the updated tiddlers back into the plugin, or add delete tiddlers in the “plugin”.
  • Retain N multiple versions of a given tiddler in such a tiddler (in wiki tiddler backup)
  • Archive tiddlers such as they are searchable in the archive, but not standard search.
    • Similar to a trashbin without relying on system tiddlers.
  • Restore test data to a known state from a plugin or package
  • I would love to be able to package tiddlers and save them as JSON files in wiki, and start to avoid using the file system, perhaps even make a virtual filesystem within a wiki, for tiddlers, json, data tiddlers and other wiki “files” if not for other common formats.
    • With export to the file system as needed. eg; Powershell scripts, YAML, xml formats that MS Word can read, email files, ZIP files, PDF that the wiki can display and more.
    • Pre or post tiddlywiki rendering.

I am looking forward to that :nerd_face:

[Vision] A Re-Play Configuration of Wiki UI States Synced to a Video
#4

I wonder if @Mohammad ’s Trash bin plugin could be repackaged as a more generic JSON archive creator

#5

That’s right. If you import a bundle.json file into a wiki it is converted from the “Array of tiddlers” into a plugin-like $:/Import tiddler, as you pointed out.

So clearly the core contains the functionality to convert from and to those structures. But full control over these functions is only available for plugin authors, at the moment. ATM The documentation is “read the code”, which makes it a bit hard to learn :wink:

That’s one usecase I also wanted to explore for quite some time. … Since TW UI can be 100% controlled with state-tiddlers, it’s possible to re-store different “UI states” by manipulating several state tiddlers at the same time. … OR as you wrote: “Restore a known state”.

But I do want to go much further. … Inception like … Like a bundle, that contains a plugin-like tiddler that contains a series of plugin-like tiddlers.

More brainstorming details to come in a different thread, so I don’t want to derail this one.

1 Like
[Vision] A Re-Play Configuration of Wiki UI States Synced to a Video
#6

A bit more about an idea, that haunts me from time to time, which imo is related to your thoughts here.
See: [Vision] A Re-Play Configuration of Wiki UI States Synced to a Video

#7

could this be a solution to those wanting “sections” of text within a tiddler to be accessed / transcluded separately? ie: a new tiddler type (“segmented” or w/e) where:

  • a new line makes a subtiddler, named serially (“ParentTiddler_1”, “ParentTiddler_2”, etc)
  • the subtiddlers remain accessible in the edit pane and can be navigated between (ideally with the keyboard)
  • the subtiddlers are transcluded in the view template so it looks the same as normal tiddlers BUT,
  • the sections can be transcluded/manipulated in other tiddlers with references!

this way, one tiddler still contains all the data from the sections, but it is broken up so as to be accessible via transclude without having to transclude the whole thing

thoughts? has this been done already in this way?

#8

Nice idea. … But there are some pitfalls. Nothing within a plugin tiddler is visible out of the box.

Have a look at my post HowTo Compare the Content of a Shadow Tiddler With the Content of an Overwritten Shadow and you’ll see that it’s relatively complex to access content within a plugin-tiddler.

That’s by design. Plugins are only needed at startup to create shadow Tiddlers. So you don’t get any optimizations like: search, backlinks, tags, filters or internal caches and indexes, that can speed up tiddler handling.

So imo that’s not really an option for that special usecase.

1 Like
#9

I have been using ‘non-standard’ plugin as a type of container since the get-go. I use them to store (contain) web-clips, that way I cannot get confused with my creations and the work of others.

They can be seen in use as ‘closedlists’ here:

https://closedlistdemo.tiddlyhost.com/

The ckeditor plugin supports direct in place editing of subtiddlers:

<$edit-html tiddler=<<parentTiddler>> subtiddler=<<currentTiddler>>mode="block"/>

And I have custom filters such as;

[plugin<plugintiddler>tagsub[license]]
#10

Yes, I think this is a strong possibility.

  • But this only needs a custom view template or body cascade to remedy. Although admittedly a cognitive challenge.
#11

Sadly it’s also a technical challenge. Internally the wiki.store keeps track of every tiddler that has been parsed. The initial conversion from wikitext to the parse-tree is one of the most expensive functions in terms of performance. So the wiki.store caches the tiddler parse-tree together with the tiddler object. So consecutive rendering of a tiddler can use the parse-tree cache.

Subtiddlers have no representation in the active wiki.store. At the moment no caching can happen. The more subtiddlers will be used the slower everything will get.

So it’s not as easy as it seems.

1 Like
#12

I did not know about that plugin: TestClipList — a non-linear personal web notebook … Thanks for the info

From a very short look, it seems to have a lot of dependencies and the plugin namespace is a bit confusing. Will have a closer look at the code.

#13

I know nothing about the tech here. So I have a question: could having json tiddlers packed in a regular tiddler be something that would allow “static” htmls to become “dynamic” htmls?

#14

No. For dynamic HTMLs we would need some sort of JavaScript to be part of the html-file. The JSON tiddlers would need to be stored on the server and the js-code would need to dynamically load it, convert and render it. … So in the end we would have a system that is like the TW app as it is now. …

#15

ok thanks Mario, it was worth asking!