Possible to make a custom upgrader?

#1

So i had a thought that I’m not entirely sure how to approach.

Lets say you use two tiddlywiki, one as a template to build improvements on, and one to do your notetaking, journaling, etc.

Well, how do you move those improvements over? you can collect all the minute changes you’ve made and copy paste them, but that’s not exactly optimal.

So, would it be possible to modify the upgrader that is used for tiddlywiki version upgrades to do this sort of this? and I guess before that, I should probably ask, How does the upgrader itself work?

because currently, I find myself exporting all my notes from an old version to import to a new version if I think the amount of improvements made are too involved to invest in a 2nd time.

I’d imagine the end result would work like a plugin in your development tw, and when you are ready, you take your production tw, in my case the notebook/journal I use, and drag it into the dev tw, and it compares all tiddlers in both, and copies the changes from the development version, to allow you to download the new one, maybe with its own version seperate to the tiddlywiki core version?

1 Like
#2

There are 2 system upgraders. One to check plugin versions on import and the other one to block some system tiddlers from import.

In the whole TW history only one upgrader was created to convert theme-tweak settings from v5.0.13-beta to 5.0.14-beta. – As I wrote they are highly specific.

I think upgraders are not really the right way to go, except you have users, that also need to update to a completely different data-structure.

If I personally develop something new I use my bundler plugin, to store a configuration of all tiddlers that I change. In that way it is easy to export milestones, which limits the risk to mess up something that already worked. I can go back to a milestone. I only need to be careful, not to miss an important tiddler to bundle.

Once a proof of concept works, I switch to a Node.js development system, which allows me to use git versioning which is much more powerful.

Did you ever consider to use a Node.js client server configuration for development.

1 Like
#3

I did, but I wasn’t a fan of the workflow, I found myself preferring to use tiddlywiki in its .html format, since I can just open it on any browser and use syncthing to keep it the same on my laptop and phone.

If there were a way to convert the node.js version of tiddlywiki into a standalone android app or windows executable with tiddlers saved in an accompanying folder, preferrably in JSON, without the reliance on downloading additional files, I would probably go that route.

(I did not see anything like that when looking at how node was set up, but I admit I am not the most familiar with it.)

I would probably still want to go the route of making a custom upgrader even with a node instance, as I often make custom system tiddlers or modify existing system tiddlers.

#4

Me too.

@Justin_H, based on everything you said, you should go with bundler.

#5

I’ll check it out then :blush:

I’m also going to continue to look into modifying the upgrader plugin too, just in case it isn’t what I’m looking for.

#6

There are many ways and reasons to do what you ask, but the key is establishing how you identify the tiddlers you would save and transfer. try and keep them such you have a filter to list them and do what you want with them. I tend to use advanced search export as json then import into the destination. or in some cases a bookmarklet. for small repeating installs.

I tend to put code and templates under my brand $:/PSaT/project/ as a naming standard.

but you can use tags or fieldnames to organise.

#7

Possibly related issue:

#8

that is one of the hurdles I’ve come up against, since I want to avoid modifying system tiddlers outside of the minor adjustments made.

If I added a tag like, “Improvements” to something like a viewTemplate tiddler or a custom field of “revision” with a different number, it could then add either of those to any tiddlers that are dependent upon them, or get pulled into filters that I otherwise would not want to include them in. (I am aware I can filter out system tiddlers, but there are times where I have had the need to include system tiddlers, so I would need to filter individual tiddlers out and, etc)

So I thought, okay, what if i package these changes into a plugin tiddler that I can easily revert and not need to tag or name differently for tiddlers that need to have the same title. I liked this, but the initial organization of tiddlers into the plugin was tedious, working with plugins in general can be a bit frustrating, specifically upgrading them or adding to them.

An alternative I considered was just making a list of every change, but if your making many minute adjustments, going back and forth from your changelog to the tiddlers you are working on becomes just as much if not more of a hassle the more modifications you make or revise. If it were less troublesome to work with plugins, this would 100% be the way I would want to approach it.

I then thought, okay, what about a dynamic list of changes? so I modified the recent tab to include system tiddlers, and then to build a list off of that, and export as JSON… and that is where it left me with a list of tiddlers that had been changed but I encountered a different issue.

What do you do if you are making edits to a tiddler that has been edited a different way to what you just did? Well, I can’t export individual tiddlers as JSON each if there are a bunch of edits, and even if I did I would be left with a bunch of tiddlers without an idea of which ones I want to keep, which need to be merged, etc.

Which is why i thought it would be a good idea to piggyback off of the upgrader, to compare all tiddlers in the current one with the upgrades, show which ones are different and use that to either determine it fine to override them or to look into them and merge them myself.

Sorry for the tangent, but it’s been something I’ve been working around for a bit, and the use of a customized upgrader is just a new approach to it lol

#9

@Justin_H

You’ve nailed all the pitfalls in the current system capabilities available. I’ll say it again, bundler addresses these the best.

For my largest “system bundle” (currently 1294 tiddlers), something that helps me is to keep a list of tiddlers I need to purge. These arise because there is no way in a bundle to handle things like renamed tiddlers: OldName → BetterName.

After having imported a saved bundle from wiki A into another wiki B, both OldName and BetterName will now exist in wiki B (and C and D, etc). As part of the bundle I have a piece of code that checks for the existence of everything in the PurgeList. When found, a button appears alongside OldName that allows me to delete it.

Of course, it is down to me to remember when creating BetterName to add OldName to the PurgeList. But that’s where we are, some of this stuff has to rely on human activity. It’s practical if not ideal.

#10

No, don’t worry. These are your experiences, and many of us share these. I have made lists in the sidebar using the list-links draggable, because you can drop other titles on to the list. I used the technical name of manifest. But otherwise I make all the tiddlers under a names space link $:/PSAT/project/ and add a field package-filter that says [prefix[$:/PSAT/project]] but if I have other tiddlers I add them to this filter. The Bundler plugin also allows filter bundles that do something similar.

  • Doing it as you go is the best way I have found to do this.
  • Although I have made a number of supporting macros and templates that display on any tiddler with the package-filter field
#11

yea :sweat_smile: just one of tiddlywikis quirks in the long run.
I think itd be best for now that I do give the bundler plugin a chance, rather than trying to build my own ‘solution’.

atleast, until someone comes around with a more fluid plugin builder and modifier. (tinka my beloved, its time I see other plugins.)

#12

The bundler internally does already support plugin creation, but there is no generalized interface at the moment. I did use the functionality with a custom UI for the Lucide Custom Icon Editions.

It is planned:

  • Bundle configurations can be used to create plugins
    • The bundler configuration will be the “source of truth”
  • It will have a “dry-run” feature, that allows users to get some info, what will happen if a plugin will be created.
    • That’s a verify step for peace of mind
    • The dry-run output will be the same as the plugin creation log
    • The dry-run log can be used as a minimal version history, if the user copy / pastes it into the version history-tiddler, which should be part of the plugin
  • The UI will be minimal, so it fits my personal needs
  • The bundle configuration will be part of the plugin
    • So it is possible to drag and drop a plugin into the bundler-edition, and everything needed will be there to modify a plugin
  • Creating a new plugin will be a 1-button-click experience
    • Metadata needed for a plugin will be auto-created, but can be modified later (verify step)
    • That makes testing plugins extremely fast
  • Plugin version numbers will be increased automatically, so there will be no problem with the core import dialogue
    • The “verify” / “dry-run” step will allow users to set their own plugin-name and version-number
  • The content will be hashed and the hash stored as meta data.
    • So it is possible to see, if a plugin has the exact same content as an other plugin. Just the meta data is different
  • Tiddlers can be in “standby” mode.
    • So tiddler titles are part of the bundler configuration, but are not included in plugins.
    • That’s handy, to “mark” related tiddlers, that are not ready for production at the moment, but they belong to the bundle
    • Stand by tiddlers will be exported with the bundle, but not with plugins … (that’s the idea atm)

80% of this functionality is already there. The bundler js-code supports all those features. Only the generic UI is missing atm.

have fun!
Mario

2 Likes
#13

What an absolutely marvelous idea. Kudos, @pmario

:trophy: