So, circling back to the impetus for this thead’s speculation: the tags field is a list field (albeit one that anomalously shakes off attempts at custom ordering!)… and the tag tiddler HAS a list field…
But the list-before and list-after fields on a tag’s tag-children are often doing “smarter” work (and more portable across wikis) than the list-field that lives in the tag-parent tiddler.
So, my line of thought in this thread is to help people see when they want to avoid messing with the list-field in the tag-parent, if/when what we really want to do (especially for custom cascade conditions and such) is make sure that the priority-juxtapositions travel with the custom content, not the tag-parent.
Perhaps when the tags are ordered as desired, capturing that order in another field (not list) on the tags tiddler, or somewhere else?. Then using that to sortby the tags on the destination. In both cases saving, and resorting could be facilitated by tag pill dropdown items. Although it will still need an intervention.
Using sortby the saved ordered list, is once removed from an exact list of tags, so additional tagged items, can be incorporated.
I would love to reimagine our drag-and-drop to support the more explicit list-before/after fields rather than a list on the tag tiddler. But I don’t even have a good design in my head, never mind an initial implementation.
That’s an exercise I do all the time for any complex project, especially ones that have been around for a long time.
Without considering how I could get there, I imagine what the system would be like if I were building it from scratch with all the information gathered since it started. TiddlyWiki has developed a lot of cruft in the name of backward compatibility. That backward compatibility is essential for long-lived projects, but but it does lead to systems with a dozen ways to do one basic thing, and no ways to do one obvious things because doing so would break one popular plugin written a dozen years ago and not updated for nine years.
I have many things that I would do differently if starting from scratch, from extremely serious to trivial. I’m guessing that anyone who has worked with it long has such a list. @jeremyruston’s list must be gigantic!
Just a few:
There are inconsistent naming schemes everywhere: title, _canonical_uri, currentTiddler, tm-edit-tiddler, $field. Or think of how to reserve certain names for use in the core, we use a $ prefix, but to reserve certain other names for use outside the core we use a . character.
We use of one field (title) for both the unique id for the tiddler and its default display name.
We can write tiddlers to shadow entire other tiddlers, but we can’t so this at the field level.
We have functions and procedures… but still have their macro predecessors.
I’m a little confused by the question, since drag-and-drop is one chief way we maintain the explicit lists in a bespoke order.
I would prefer that we could use drag-and-drop or something similar to @vilc’s new offering to allow us to maintain list-before / list-after on the individual tags rather than list fields on the tag tiddler.
Actually, I was hoping you would do it for this tags issue, how do you logically combine drag and drop with set orders in list-before and list-after. In some ways they seem contradictory.
I genuinely want to understand this.
Start opinion!
I don’t actually see the problems you do with tiddlywiki. I feel the focus on backwards compatibility has informed the design of solutions that don’t compromise the past or the future as much as many people seem to think. Sure it may increase the complexity sometimes when trying to change the underlying behaviour, filling gaps we find as our use gets more sophisticated, or when learning how it works, but it actually is this “priority” that allows most users to “not know how it works, under the hood”, but to rely on the published practices.
If backwards compatibility was not assured you would always be having to know how it works under the hood, to avoid unexpected consequences.
I have always being interested in IT because it has sat at the forefront, bleeding edge, of human understanding, and I always push the limits of a system I get to know. Sure I am always pushing the limits, but that does not change how few limits are imposed on us by what we have today in tiddlywiki.
A new wiki, one that you don’t plan to distribute, can break every rule in the TiddlyVerse if it wants, so if it means enough to you, you need not design for the future, but fork it into a bespoke solution. It seems to me with tiddlywiki, the question is “are you going to generate technical debt for the future”, not how much technical debt tiddlywiki already has.
The list-before and list-after fields have been introduced, to allow users to add new tagged elements into ViewTemplates, without the need to modify shadow tiddlers. Eg: $:/tags/ViewTemplate is a shadow tiddler, that has a predefined list-field.
Let’s assume users use list-before, list-after in a tiddler that comes from a plugin. Those tiddlers are also shadow tiddlers.
This mechanism works well, except, when you start to drag and drop sort the order with the tag-pill dropdown. As soon as you do this all those shadows are converted to system tiddlers. $:/tags/ViewTemplate will be converted and the list field will be updated. The shadows, that contain the list-* fields will be overwritten and the list-before, -after fields will be removed.
So instead of not modifying shadows, the complete opposite happens.
If users actively change a shadow tiddler in the story river, we do display a dialogue and tell them what happens. So we (the core devs) can say: We told you, if there are problems.
To be consistent, we would need to show a dialogue if a tag-pill drag&drop happens. The problem with that dialogue is, that it would cause more problems, than silently overwriting shadows.
We (the core devs) have to make decisions, knowing that there will always be someone, which has a problem with that decision. We can only hope, that for 90% of our end users the decision is right and we will never hear from them
I think you misunderstand. Perhaps I didn’t explain well enough.
I love the backward compatibility TW strives for!
But it involves a lot of cruft. Much of the technical debt can never be repaid. That’s the result of relentless backward compatibility.
So it’s a fun exercise to imagine what things might look like if we were to start over and build on the best ideas, ignoring the cruft. I don’t expect it to happen, not before TWX, anyway. But thinking about it can be quite enjoyable.
I do understand the problem of trying to design for many different users, and I also understand both how the current tag-pill drag-and-drop works, and the design constraints that it operates under. Nonetheless, it’s always possible there could be improvements.
I think the conversation @springer opened here suggests some real possibilities for a more useful UI. I’m far from sure that it could be done in a backward compatible manner, but I think it’s worth investigating. I don’t know if/when I’ll have time to play with it, but I believe it’s a useful conversation.
To be clear: My thought in this post (which @Scott_Sauyet rightly took from initial location) was NOT really a complaint about the way TW developers chose to handle ordering within tags. There’s a lot of considerations to juggle!
It’s a realization that I myself find the “bottom-up” way of tweaking order (using list-before and list-after fields) to be much more powerful — especially with the increasing role of cascades.
So, I’m imagining a plugin solution that might be helpful for myself and others.
The visual cues in my proof-of-concept are 100% backwards-compatible. This way, when I drop down a tagpill, I can quickly see WHY the things are where they are. This will help me make better decisions (even though opening the tag-child to add a field is considerably less convenient than drag-and-drop within the tagpill).
My stretch-vision involves making some kind of alternate “surgical drag” solution. Maybe with a modifier key? Or maybe using a marginal drag area that is otherwise easy to avoid? This would enable a more convenient access to modifying the children’s fields (rather than the list field of tag parent).
Even this would also be backwards-compatible and entirely optional, perhaps useful just for geeks who share a similar experience to mine (about the occasional downsides of the standard behavior for tagpill drag-and-drop).
We start with this, where only the lock icon and the title are actually shown. The others are just for this explanation. The right-hand column shows other fields on the tiddler in question:
The important thing to note here is that the drag-and drop didn’t change the ViewTemplate's list field, only the list-before, list-after of dragged child.
And by default, the shadows are locked and cannot be dragged. You would be able to unlock them, but only while the tag-pill is open.
Of course this might also require updating the logic of sorting the items based on list fields, list-after, and list-before. And that’s already pretty complex logic. It is also possible that this wouldn’t be necessary; what’s there might just work.
I don’t know if there would be some additional trigger necessary to launch this, such as a modification key while dragging. It would be really nice, if we can get it to work, to make this the default, but it seems likely not to be backwardly compatible.
Currently, tag ordering is determined by applying a combination of the list field on the tag tiddler plus any list-before or list-after fields on the individual tagged tiddlers. In addition, if a tagged tiddler is not in the tag tiddler’s list and does not have a list-before or list-after field, it defaults to being added to the end of the list (alpha sorted).
This existing hybrid approach allows tiddlers to be explicitly positioned using list-before or list-after fields and is commonly used by 3rd-party creators to provide an initial placement for imported tiddlers (and especially for ViewTemplate or EditTemplate tiddlers).
Then, as soon as you drag-and-drop an individual tiddler to change the tag ordering, the list-before and list-after fields are removed from all the tagged tiddlers and the tag tiddler’s list field is updated to contain all those tagged tiddlers (including those that were previously just alpha appended to the end of the list).
Now, consider the following “minimal change” strategy that preserves any existing list-before or list-after fields:
If a tiddler has a list-before or list-after field, just update that field to contain the title of the tiddler after/before it, but do NOT update the tag tiddler’s list field.
As a special case, if a tiddler is dragged to the start or end of a list and a modifier key (e.g., ctrl) is pressed, then it would get a list-before or list-after field with an empty value. This allows it to remain at the start or end of the list, even if the tiddler that is before/after it is moved.
If a tiddler is already in the tag tiddler’s list or does not have a list-before or list-after field, update the tag tiddler’s list to set the tiddler’s new position (as is currently done).
In all cases, do NOT change any other tagged tiddlers (i.e., leave their list-before and list-after fields intact).
I’m not certain, but this approach might even be (mostly) backward compatible.
That’s precisely what I was imagining, although I hadn’t yet considered the blank list-before/after yet. I think my technique could be extended to just add one extra dropzone at the top or bottom, and not require a modifier key.
Exactly. that was the goal.
I think that if every place that does this sorting in the core goes through this new process, it will be very close to entirely backward compatible. The only exception would be if custom code depended on the tag tiddler’s list field for other processing, which would be odd, since that would only work right now if they always sorted the tag pill with drag and drop, and never set list-before/after manually. So it sounds low-risk. But it would require the users to get used to a slightly different UI.
I will try to update that tiddler to also show the top and bottom drop-zones. I should have time this evening.
Are we close enough to a design that I should open a GH issue? I’m not ready to commit to creating a PR for it. There’s a lot of code in that area I know nothing about. But I would like to try if I can.
I absolutely love this as the easiest and most intuitive element here.
For things like stylesheets, it’s just so often that I care not a whit about anything except making a certain tiddler drop to the end (so that it trumps whatever other old stuff or plugin-installed stuff was interfering). And with cascade conditions, it’s so often that my interest is in nothing more than making sure that this new custom condition not be stuck (as it is by default) at the end of the cascade. (And usually if I’m building the condition, I’m already in control of making it conditionally savvy enough to work well at the top of the cascade.)
]In some prior back-and-forth, I thought I recall @jeremyruston offering some reason why it really needs to happen that if there’s reordering in the tagpill, that existing list-before and list-after fields needed to be zapped in the process. But I no longer recall the details.
But I did just think of a wrinkle. And it might be why we have the current drop-everything-and-rebuild-the-entire-list mechanism in the first place.
If we have
Template1
{list-before: 'Template2'}
Template2
{}
and we now drag Template2 to the list-before drop-zone of Template1, we would get:
Template2
{list-before: 'Template1'}
Template1
{list-before: 'Template2'}
It would be nice to prevent this, by removing the list-before from Template1. But that would contradict
In all cases, do NOT change any other tagged tiddlers (i.e., leave their list-before and list-after fields intact).
Perhaps that’s not a concern. We could certainly cause the same ambiguity by manually updating our list-before/after fields, and the algorithm will still manage. But it does feel untidy.
I misread a little bit. My idea is slightly different from this:
I would expect the core list fields to remain intact, and only work with list-before / list-after for our custom tiddlers. Plugins could override the tag tiddler’s list field, but I would expect them to usually use list-before/after. The locking mechanism in my sample UI is meant to make it a very deliberate move to adjust any shadow tiddler’s placement. Move your own tiddlers amidst the existing shadow ones, but don’t (often) move those shadows around one another. Obviously you could unlock, and move, say, $:/core/ui/ViewTemplate/tags above $:/core/ui/ViewTemplate/title. I imagine that would be rare, but we need to support it.
So I would expect that the ones in the list field of the tag tiddler would be locked, and not likely to move. And even if they are, I think we can still use the list-before/after fields for them. The sorter needs to already be able to deal with this, so I don’t think it’s an additional burden.
But unlocking can be accompanied by a “You’re overriding a shadow. Proceed?” message, to help drive the point home.
There is another idea I mentioned in the conversation that would not be backwardly compatible: turning the list-before/after fields into list fields (in theory unordered sets, not actually ordered lists, but that’s an implementation detail.)
is that we could now drag Template 1 to another wiki, and expect it to simply work, and be placed in the right spot, *regardless of whether we also dragged Template 2.
But I wouldn’t focus on this now, as I think we can do the backwardly compatible version on its own, and this will be much more controversial.
I am in favour of changing the default behaviour. I do not really like it, that enlist and enlist-input do not see the before-* fields.
See the problem with list-before/after if we use [enlist{$:/tags/ViewTemplate!!list}] or [{$:/tags/ViewTemplate!!list}enlist-input[]] we get a different list than with [[$:/tags/ViewTemplate]tagging[]]
All of those inconsistencies should also be “fixed” if the behaviour is changed in a backwards incompatible way.
Also see: Extended Listops Filters all of them will probably need to know about the new behaviour.
The list-before/after handling is hardcoded in wiki.js at exports.sortByList().
There needs to be a solid concept, before anything is coded. Otherwise we will cause more inconsistencies as we have at the moment.
I don’t know how to fix that. The enlist operator should not know where its data comes from. If we can update tagging[] in a backward compatible manner, and use it—or its underlying implementation—everywhere the core needs to order a list of tags, I think that’s probably the best we can do. If we are using enlist somewhere we should be using tagging[], then we can fix those, with a risk to backward incompatibility. But I don’t think we can change them in user-land.
The only thing I see as a possibility, but I think it’s a terrible one, would be to change property access to handle list differently. Uggh!
But the big thing here is that the changes we’re proposing should not break much of anything, and I believe shouldn’t require any update to tagging[]. Essentially, we have two methods to modify the order of tag lists, two methods that will work together. All we’re doing is changing which of the two is used by default by the drag-and-drop sorting.
I have being giving this a little through and think we have the filter operators to process the list-before and list-after fields so if these values remained we could always capture the intent of the tiddler author(s). Although I am yet to do this.
That is not unexpected. This “sort by” highlights the direction I was thinking and rather than change the existing mechanisms we can find a way to override them, So often our lists are seen in different orders such as by title, by a date or status. Its on the UI where this becomes an issue, so why not improved how that works.
Get the part of the core that iterates the tag to use an alternate sortby list if it exists. The user altering the order can snapshot the alternate sort order.
The order can be
stored in an additional list field on the tag tiddler
