Sorting Hint - multiple sort criteria

#1

This is a tip for those who have trouble doing more complex sorts. I haven’t noticed this mentioned, although I haven’t looked thoroughly.

If you need a multipart sort, you can do it with separate steps.

[some[selection]of[tiddlers]sort[first-name]sort[last-name]]
                            \______________/\______________/
            secondary sort /                                \ primary sort

Note that the more important feature you’re sorting for comes last.

This is probably not as efficient as what you might do with sortsub, but it’s simpler to work with, and it’s more powerful. sortsub will generate keys from your items and sort by those keys. That works great for some cases, but it doesn’t work well, for instance, with mixed number and string types as you would have with street addresses. This technique covers those as well.

Note that you can of course extend this to three sort criteria or more. Just put them in reverse order, so that the most significant criterion comes last, and the least significant one comes first, etc.

If there are other simple ways to do this built into TW, I’d love to hear more!

Technical Explanation

This runs two separate sort operations. First we sort the inputs by first-name. Now, on that new sorted list, we run a second sort operation on last-name. But in all modern JavaScript implementations, sorts are stable, that is, values that the comparator says are equal (say, having the same last name) are returned in the same order they were in the original list.

So after sorting by first name, we might get:

Jane Roe
Jane Doe
John Q. Public
John Doe
John Roe

then when we sort by last name, we never alter the relative order of the two Roe’s we’ve already done, so Jane Roe will end up before John Roe, so we will get

Jane Doe
John Doe
John Q. Public
Jane Roe
Jane Doe

There are other good sorting algorithms that are not stable. But being stable, which is now specified
for JS engines, gives us the ability to do this consistently.

Example

The Address issue was my use-case. I have some Address objects that I want to appear in custom search results. They have a number of fields, including ones like these:

address:142 Bunker Hill Rd - Apt B
apt: B
street: Bunker Hill Rd
street-number:	142

as well as city, state, postal code fields, and latitude, longitude, altitude ones. If I sort my search results by the address field above, I get an ordering like this:

10 Hickory Hill Dr
104 Bunker Hill Rd
105 Boston Hill Rd
109 Bunker Hill Rd
109 Long Hill Rd
11 Hickory Hill Dr
112 Bunker Hill Rd
…
84 Long Hill Rd
85 Long Hill Rd
93 Boston Hill Rd
95 Bunker Hill Rd
97 Long Hill Rd

(This was a search for “hill”, which in my case also lists the people with the last names “Hill”, “Hills”, “Phillips”, and “Schilling”.)

That is definitely not the order I want. I would much rather get

93 Boston Hill Rd
105 Boston Hill Rd
95 Bunker Hill Rd
104 Bunker Hill Rd
109 Bunker Hill Rd
112 Bunker Hill Rd
10 Hickory Hill Dr
11 Hickory Hill Dr
…
84 Long Hill Rd
85 Long Hill Rd
97 Long Hill Rd
109 Long Hill Rd

In other words, I want to sort by street name first, and then ,if the streets match, by street number. I want numeric sorting on the street number, so that 23 comes before 100, which is different from the JS standard lexicographic sorting, which sorts character-by-character.

Using the technique above, I wrote:

[search[for]matching[addresses]nsort[street-number]sort[street]
                               \__________________/\__________/
                secondary sort /                              \ primary sort

And it sorts them the way I choose.

2 Likes
#2

Nice. There is one typo if someone were to do a copy and paste of the pattern from this post. You have square brackets in your first example, around the first sort:

[some[selection]of[tiddlers][sort][first-name]sort[last-name]

instead of

[some[selection]of[tiddlers]sort[first-name]sort[last-name]

#3

Oops, thank you! Fixed.

1 Like
#4

And now, if you add the final trailing bracket…

:wink:

Good stuff, Scott!

#5

Oh boy! I do sometimes long for StackOverflow’s fix-it-if-you-see-it capability. Thank you. This one is fixed as well.

#6

Thanks @Scott_Sauyet for researching this and writing it up for us. A very helpful resource.

Building on what you have done you could take the sort and place it inside a function such as sort.address[] and reuse it where needed.

This discussion of sort reminds me if my earlyier days programming always striving to optimize sorts.

  • in fact I became quite adept at heirachical programming
  • inspired by your post I may see if I can bring some of those old techniques into tiddlywiki. Particularly totals and subtotals on change of groups within the sort eg number of addresses in the same street, town etc…

I recently crafted a custom list.report widget that you give a filter and or alternative settings to the various elements.

  • I will see how I can incorporate your method. Perhaps just providing a function name like sort.address to the list.report widget.
  • just ask if you want to see it.

Thanks again

#7

That makes sense. If I find a need to reuse it, I will probably do so. I would love then to also figure out how to make the field names to use configurable. I regret titling the field street, and wish I had used street-name in parallel with street-number. It wouldn’t be hard to change that, as the 1200 address tiddlers (hey, it’s a small town!) are generated from other sources, and I probably will do that sometime soon. But it would be nice if the function defaulted the preferred field names of street-number and street-name, but allowed me to override, with something like:

[<my-search-results>sort.addresses::street[]]

It sounds to me like my bad old days of COBOL and RPG!

I remember you sharing either the implementation or a discussion about the idea. While I understood how it could be useful, I had no immediate use for it, and filed it away as something to remember when the time came.

I would love to see it!

#8

image

I hope I need say no more…

1 Like
#9

Like, uh… we want to see it.

#10

Capitalization Of Boilerplate Oriented Language (COBOL)

#11

list-reportWidget.json (8.2 KB) try on TiddlyWiki.com

Without yet incorporating the “multiple sort criteria”, which could just be added to the filter, here is my $list.report widget. It is delivered in a JSON bundle, see for some examples, $:/PSaT/list-reportWidget/examples

  • It uses defaults unless you provide it with alternates through one or more named $fill widgets, as documented.
  • It is still a little rough around the edges and feedback appreciated. I can start a new thread if too many questions arise.
  • Be aware this only lists the filter and des not provide the ability to “break on changes”, nested lists, as discussed above.
  • It is not written in COBOL, although I do miss some aspects of it :nerd_face:
#12

I don’t think I’ve ever actually ROFLed, but that got me pretty close.

#13

I wish I could claim credit. James Iry’s whole article is brilliant!

#14

This looks interesting, but I think it’s only because I once did report-writing software that I have some sense of what it’s for. I think it would be really good to include a more full-fledged example that shows what this will help you accomplish more easily than you can do in plain wikitext. Your example is far too bare-bones for that.

I think if you can do that, you might have a very useful tool on your hand!

1 Like
#15

My list report was presented to illustrate how easily you could turn your multiple sort criteria into an operator and use it in other tools. I should have spelt it out.

  • I only write such comprehencive responces to be helpful and informative and share substantial ideas.

The basic use of the $list.report I shared is as simple as <$list.report filter="your filter"/>, and you gain a pile of functinality, so we can see using this with the multiple sort as;

\function sort.address() [sort[first-name]sort[last-name]]

<$list.report filter="[tag[TableOfContents]sort.address[]]"/>
  • I have not tested this exact sort, because I do not have a suitable set of test data at hand. But it works otherwise.

My widget then provides a default header and contains the logic to add a range of elements to your “report” (using fill widgets), as documented in $:/PSaT/list-reportWidget, and coded in $:/PSaT/list-reportWidget/global

  • Its easy to add odd-item, even-item, each-item, footer etc…
  • Includes defaults, you could modify.
  • Close and customise $list.report as needed, from the code pattern given.

I have done a lot of work with functions and custom widgets, and the $list.report was a prototype developed halfway through this journey, by looking at it closely an enthusiast can see a number of useful code patterns and start to understand how custom widgets can be used. I hope this stimulates collaboration to develop some well known code patterns.

  • One key value is capturing complexity in the widget, to make its use very simple. I think I demonstrate that, above.
  • Something I learned, I will add to the next version, since I wrote this is to include in the parameters widget $params=params then use setmultiple variables ($name $values) or a genesis widget to convert all parameters (even unnamed inside the widget) to variable=value.
    • The key value of this is you can pass arbitary parameter value pairs into the widget and use them in side the widget (via fill widgets), but importiantly your parameters can be many things;
      • parameter=“literal”
      • parameter=“variable-name” procedure or function, see
      • parameter=<<variable>> procedure or function.
      • parameter={{!!fieldname}}
      • parameter={{tiddlername}}
      • parameter=the new backtick formats
    • The result is you can have much simpler code because you dont need to code as many variables outside the widget, just have them evaluated when you call the widget.
#16

I have actualy done a proof of concept before 5.3.x but basicaly you pass the sort fields as parameters eg `“country state region street”.

  • From there you call N nested or recursive list widgets.

[Update]

I was working on a new way of building recurcive functions with 5.3.x for example itterating the table of conents. One number I have is the depth number, indicating how far into the tree deep you are. In this case of course I am walking the tree, however if we concider a multilevel sort, with two or more “values within values”, eg country, state, region It just struck me if we;

  • Structure our data correctly it is already sorted
    • Perhaps we could just apply a structure then walk the tree?
  • We can use this recurcive process to do our input data, and for each change in level, do totals.

Food for thought.