Stumped on complex sort

#1

Inspired by this thread (with variable-responsive html tables), I wanted to mock up a shiraz-style dynamic table that does the same things.

But I realized (in mocking up my suggestion) that I was stumped by the sorting needs of @mswho42 — which seem to combine :sortsub-style need (retrieve the value in the stack field, since that value is the basis of sort order) with sortby-style recourse to a separate tiddler (that contains the custom sort order for those values).

For a simpler example, documentation shows how to do
{{{ Friday Monday +[sortby{Days of the Week!!list}] }}}

… But if I have a list of tiddlers each with a day field, how do I get that list to sort based on their day field, but in the custom order? Seems to need a sortsub move, but I’m having trouble getting that play nice with sortby…?

<$vars compare-by-day="[get[day]sortby{Days of the Week!!list}]">

<$list filter="[[Amanda]] [[Jane]] +[sortsub<compare-by-day>]">

{{!!title}} — {{!!day}}

</$list>

The result is

Amanda — Friday
Jane — Monday

but of course I’m hoping to get it the other way `round … (which then should map onto the more complex problem in the other post)

Conditional Filtering
#2

???

#3

Sorry, keyboard mishit posted before I finished.

1 Like
#4

I don’t have time now to consider how we might get this right, but I can explain why it’s not meeting your expectations.

sortby<whatever> takes a list, and sorts that entire list based on their positions in whatever.

sortsub<converter> takes a list, uses the runnable converter (function, procedure, macro) to create an index for each element, then sorts the originals by that index.

The latter is more general, as we could derive sortby from sortsub in a fairly straightforward manner, but not vice versa.

Think of sortsub like this:

Amanda Jane  +[sortsub:number<letter-count>]
(calculate `Amanda -> 6` and `Jane -> 4`)
sort the pairs (Amanda, 6), (Jane, 4) numerically by their second element to get:
(Jane, 4), (Amanda, 6)
then extract the first components, to get:
Jane Amanda

I doubt this is the exact mechanism, but it should show the idea.

I will often use sortsub:string<by-last> with something like

\procedure by-last(tid) [<tid>get[last-name]] [<tid>get[first-name]] +join[~]]

Aside: TW seems to be missing the most powerful of JS’s sorting tools: supplying a comparator, which for any pair A and B returns a number which is negative if A should come before B, zero if it doesn’t matter, and positive if B should come before A. That’s harder to use, but quite powerful.

#5

I mostly solved my own problem… Unsurprisingly, it’s actually easy to get the sort needed here — just so long as we go at it from the other end (which is pretty much the logic @mswho42 used in that other thread, though that approach employed nested list widgets):

\function sortby.StackOrder()
[{StackingOrder}enlist-input[]listed[stack]is[tiddler]]
\end

{{{ [sortby.StackOrder[]] +[format:titlelist[]] }}}

(And further conditions can be integrated into that same filter run.)

Or in the simplified example I started with, it might look like:

\function sortby.weekday()
[{Days of the Week!!list}enlist-input[]listed[day]is[tiddler]]
\end

<$list filter="[sortby.weekday[]]">
{{!!title}} — {{!!day}}<br>
</$list>

This does get the required order.

Alas, there’s still no “hook” that is easily harnessed by shiraz dynamic tables… which was the impetus — hoping I could get it set up so that the table could sort tiddlers according to a column/field with a proprietary (indirectly-established) order.

#6

I’m glad to hear it, as I’ve gotten stumped myself.

I really feel there are one or two missing operators that would help here, notably indexof:

[enlist[A B C D E]indexof[D]]

would yield 4.

With something like that, I think we could quickly write a filter that would help us do these types of sorting.

I can write it in JS easily enough, but it’s more annoying in wikitext.

Still, this should work with @mswho42 's custom tables, allowing us to run a filter or three to choose the inputs and just sort them with sortsub,

It’s bedtime now, and I can’t go any further right now, but here’s one barely tested JS module:

title: $:/core/modules/filters/indexof.js
type: application/javascript
module-type: filteroperator

/*\
title: $:/core/modules/filters/indexof.js
type: application/javascript
module-type: filteroperator

Filter operator for the index of a value in list

\*/

"use strict";

/*
Export our filter function
*/
exports.indexof = function(source,operator,options) {
	var results = [], index = 0, found = false;
	source(function(tiddler, title) {
		index += 1;
		if(!found && String(title) == operator.operand) {
			results.push(String(index));
			found = true;
		}
	});
	if (!found) {results.push(String(Number.MAX_SAFE_INTEGER));}
	return results;
};

If it’s not found, we return a very large number. Traditionally a not-found would yield -1, but this version feels better for the type of sorting we’re discussing.

#7

I see you have a working JS solution already, but I think you could do something similar with a function operator:

\function index.of(item) [allbefore:include<item>count[]]
\function zindex.of(item) [allbefore<item>count[]]

{{{ [enlist[A B C D E]index.of[D]] }}}

{{{ [enlist[A B C D E]zindex.of[D]] }}}

As I was writing this it occurred to me that index.of is essentially the inverse of the existing nth operator, so just for fun, I added zindex.of to mirror zth.

Both return “0” for any parameter not present in the input list, e.g. {{{ [enlist[A B C D E]index.of[Q]] }}} = 0. I don’t read JS well enough to understand how your custom operator handles missing items, but I suspect the function could also be tweaked to provide your preferred output — as a rough guess, possibly something like

\function index.of(item) [match<item>] :then[allbefore:include<item>count[]] ~[[missing value]]

{{{ [enlist[A B C D E]index.of[Q]] }}}

where “missing value” could be “-1” if you prefer.

Edit: Or the following, which will bump any missing value to the end of the list rather than the beginning:

\function index.of(item) [match<item>] :then[allbefore:include<item>count[]] ~[count[]add[1]]
2 Likes
#8

Nice!

I don’t think I’ve even seen allbefore!

#9

Indeed, I confess I’m so stubbornly attached to those dynamic tables that I find myself wondering whether the next frontier (for myself — slow! — or @mohammad if he’s interested :slight_smile: ) would be designing an enhancement so that each dynamic-table column readily employs a cascade condition to determine the appropriate sort-order for that field/column [edit-to-add: (Shiraz dynamic-table macro currently wants to apply one sortOp criterion to the whole table.)]… :thinking:

Of course I’ve already been approaching fieldnames as nodes that can become increasingly more useful — both virtually and as stored JSON tiddlers with field-specific info…

Having a custom sort-order associated with a fieldname (like stack for @mswho42) might be something that could be hooked not just by shiraz dynamic tables but by lots of different things!

For example, LC card-catalog numbers sort in their own specific way, task priority field might sort in a specific non-alphabetic way that @Mohammad already recognizes as calling for a klugey solution of using explicitly number-prefixed status values, etc.

A tiddler for that field would be the natural place to point toward a sort-order, whether it’s one of the several alpha/numeric options (case-sensitive or not, numbers-first or not, etc.), brutely list-based (like StackingOrder or Days of the Week), or regex-based (like sorting book titles with initial stopwords bumped to the end). Someone could even want an intuitive system for sorting color values (!).

Now that I’m obsessing about info associated with a fieldname (or more granular field use-patterns, if the same fieldname could be pressed into multiple uses)… we’re getting back into the cluster of “usual suspect” topics that it feels like a subset of us keep mulling on (@etardiff, @Scott_Sauyet, @TW_Tones).

1 Like
#10

I did consider something more flexible in #12176. I have had little need for it myself, but I can easily imagine wanting it at some point. I was thinking of mine as an alternative to Shiraz tables, but would be honored, if I could get it to work, to have it folded into Shiraz.

There are three sorts of flexibility that I would love to add:

  • The ability to derive column values from one or many fields, and not just use field values directly.
  • The ability to easily create a sorter and associate it with any field. While there is something in place now in Shiraz, it only lets you associate a field name with a single sorter, across all tables in the wiki. I want configuration for the table to say that for this table use the following sorter for this specific column.
  • The ability to sort on multiple fields. Thus I want to sort on, say, political-party, ascending, then within that by score, descending; then by last-name ascending, and finally by first-name, ascending. I imagine the UI for this as simply clicking on headers, with each header cycling through none / ascending / descending states, but the other sorts that we’ve already established being used for a preliminary sort on this field. So for the choice described, we would click first-name, click last-name, click score twice (for descending) and then click political-party. This would be stored somewhere as "first-name last-name !score political-party", and the table configuration could easily default to the preferred sort-view. I made a start at doing this in JS, and just now updated it to have the three-cycle of none/ascending/descending. But the second section (filter output) would not combine well right now with the notion of fields deriving from columns.

I don’t know if I’ll ever get back to work on this, but I’m intrigued once again.

#11

Is this so? If so I’ve missed it/forgotten; I see only that the whole table can get a sortOp parameter declared (defaults to sort) for any instance of the macro.

I do think this is possible with a template for the body of a cell. I did lots of custom cell templates at one point… The catch (if I recall correctly) is that you can’t then sort on the actual value, because the sort can’t see into whatever the content is, when the displayed content is calculated.

This is already on @Mohammad’s to-do list — or at least a secondary sort.

#12

This is fairly dense reading, but it does give the information I used to create some custom column templates. The limitation I was noting is that these templates have a list field with the names of columns which use them. This means I have to use the same formatter for, say, total even though in one table it might represent an integer count, and in another, a currency amount.

That’s part of what I want to capture. But it’s complicated, because I might want to sort birthdays by their MM-DD format but display April 16 and January 26. So I want to able to easily separate the display from the sort. By default, I would expect to sort by the display value.

Oh, that’s great to know.

#13

I too have created custom column templates (based on reading that documentation). I have seen nothing that helps me specify a sort order specific to a column (not based on one of the sortOp types, which again seems to be baked into the whole table).

Nope, the date column sorts by the “real” date, even though the template can be modified to display the date however you need:

Modify $:/plugins/kookma/shiraz/templates/body/date to show desired format:

<td class="shiraz-dtable-col-fixedsize">
<$view 
  tiddler=<<currentRecord>> 
  field=<<currentColumn>> 
  format="date" 
  template="MMM 0DD YYYY"
/>
</td>

Then put this in a tiddler and see the sort behave appropriately:

<<table-dynamic filter:"[has[modified]limit[10]]" fields:"title modified">>
#14

Sorry, faulty memory circuits.

Except that if I want to sort by birthday (05-08 < 07-01) rather than date-of-birth (1966-07-01 < 1996-05-08), even though they may operate on similar fields., I need a different sorter.

#15

Indeed!

And if you created a column called “yearless-birthday” with contents that extract the month and day with a filter expression (not just a date format trick)… this will also display fine, but will not sort properly.

(This would be an ideal candidate for a custom-sort pattern that uses regexp to ignore the digits for the year, similar to sorting book titles through a custom subsort (based on dropping the stop words).)