Is there a way to use the journal-date field to organize journal entries with nested subsumes?

#1

I have a bunch of journal entries that are tagged “Journal” and have a journal-date field with the date like so: YEAR-MM-DD_HHMM_SS000 for example 2023-11-14_1130_00000

(I had to make it human-readable, it was too frustrating making sure I had filled it in right otherwise.)

QUESTION: Is there a way to use the journal-date field, to organize all the tiddlers that are tagged with “Journal” and have a journal-date field, so that I see a list of subsumes like so:

etc
Journal 2020
Journal 2021
Journal 2022
Journal 2023
etc

And when I click on one (for instance, Journal 2023) it shows a list of subsumes like so:

2023 January
2023 February
2023 March
2023 April
2023 May
2023 June
2023 July
2023 August
2023 September
2023 October
2023 November

And when I click on one (for instance, 2023 January) it displays the results of a list filter for all entries in January perhaps using this list filter:


<$list filter="[journal-field[I DON'T HOW TO INDICATE THE PART OF THE JOURNAL-DATE FIELD THAT INDICATES THAT IT IS JANUARY]!sort[journal-date]]">
<h2><$link><$transclude field="title" mode="block"/></$link></h2>
<$transclude field="text" mode="block"/>
</$list>`

I have wondered how to do this for a long time now.

And also I am so grateful for giffmex’s subsume plugin. I use it all the time.

P.S. My version of Tiddlywiki is 5.1.22. (I’ve tried twice to update in the past but that’s another topic for the future.)

1 Like
#2

I think you would have gained some value here if you got past this issue and used a date picker, It is now more complex especialy since you are not using 0MM 0DD 0hh 0mm 0ss.

The most direct way is to sort your tiddlers by date then list them and use nested list widgets to group the tiddlers.

To save us needing to learn about subsume in this case please explain it and/or link to it, otherwise you are forcing us to research it, just to help.

I may return with an example however be aware tiddlywiki already does something similar with the recent tab $:/core/ui/SideBar/Recent, and the time line macro, however I belive you would be better writing nested list widgets.

1 Like
#3

OK, I went and researched using a list widget, or nested list widgets and re-discovered the difficulty in handling multiple breaks within a list, eg break on year, month and day of month etc… that you are asking for.

  • However this has long being an objective of mine and I finally achieved it.
  • I hope to use it in some advanced custom widgets sooner or later.

I have found a new approach, and will try and simplify and improve it however it is a bit too complex to detail here (for now at least), it also makes use of standard tiddlywiki serial dates. But here it is if you would like to see the methods. Note it needs tiddlywiki 5.3.x Group-dates procedure.json (1.9 KB)

Drop it on tiddlywiki.com, then in your own tiddler use the following;

<<group-dates filter:"" date-field:"">>

  • eg <<group-dates "[tag[TableOfContents]]" created>>

Inside the groups dates procedure you will see a list of functions that define various filters, however after them you will see four procedures new-year, new-month, new-doy (day of year) and listItem, Changes in these allow you to change the output data and format, including do nothing.

Technical innovation?

  • I use the filter and date-field to store the list of ordered titles in a variable
  • I use a range filter to step through each item by changing an index.
    • This means it is trivial to access the last title (index -1) to make comparisons with the current title and see if a change in year/month or day of year occurs.
    • When a change is detected a procedure is called to reflect that before listing the next item(s)
#4

The subsume plugin:

I am using: https://giffmex.org/gifts/subsume.html

Using a datepicker:

I found <$edit-date field=“lastSync” showTime=“yes” showSeconds=“yes” use24hour=“yes”> from: Date picker + Action widget - #4 by linonetwo (edited it to read field=“journal-date”)

I also found this: Use date picker in a field while creating a tiddler - #4 by EricShulman and implemented it, changing the necessary field names to fit my case.

So now I am using YYYY0MM0DD0hh0mm0ss0XXX in the journal-date field. Is this format OK?

Oh I just saw your new reply. I will check it out.

#5

I upgraded my tiddlywiki to 5.3.1 and dropped in your Group-dates procedure.json.

Then I put this in a tiddler:

<<group-dates “[tag[Journal]has[journal-date]sort[journal-date]]” journal-date>>

But no subsumes appeared. Just a long list of entries that keep crashing my tiddlywiki tab. They do have headings like Year 2023 and January 2023 but then an extra heading appears (1st of January) and the whole bunch of entries from all months and days in the year.

One reason I want to open my journal in stages, like clicking on a year subsume and then a month subsume, is to keep from opening a tiddler with millions of entries; this way only a month of entries opens at any one time.

I know I could simply tag each journal entry with and “[year] [month]” e.g. “2023 November” and make a tiddler for each month like so:

<$list filter="[tag[2023 November]sort[journal-date]]">
<$macrocall $name=subsume tid=<<currentTiddler>>/>
</$list>

And then tag all those month tiddlers with “Journal 2023” and make a year tiddler like so:

<$list filter="[tag[Journal 2023]sort[journal-date]]">
<$macrocall $name=subsume tid=<<currentTiddler>>/>
</$list>

And tag all those year tiddlers with “Journal” and collect them all in a journal tiddler like so:

<$list filter="[tag[Journal]sort[journal-date]]">
<$macrocall $name=subsume tid=<<currentTiddler>>/>
</$list>
  • I love the way subsumes hook up with other subsumes, for some reason it reminds me intensely of crotcheting.
  • I believe it was a post of Mohammed’s that introduced this way to structure a journal. It was not my idea.

It can feel satisfyingly straightforward to use that method, but the problem with using tags to get the subsume structure is that:

  • I have to remember to make a month tiddler every month for the rest of my life, and even if I did them all up in advance …
  • I thought tags were ideally for browsing and fields for organizing. Using “(YEAR) (month)” tags to structure requires every single journal entry to have the additional tag “(YEAR) (month)” on it instead of just a “Journal” tag and a journal-date field.

For a long time I’ve been wondering if it was possible to do it all instead via “parsing”(?) the journal-date field.

Is there some code that can nest those subsumes all at once? For instance, this is just a WILD horribly wrong guess on my part:

<$list filter="[tag[Journal]sort entries into list of subsumes by years[via first four digits in the journal-date field]]">
<$macrocall $name=subsume tid=<<currentTiddler>>/>

 <$list filter="[results of previous run sorted into list of subsumes by month[via digits 5 and 6 in the journal-date field]sort[journal-date]]"> it would do this over and over for each year
<$macrocall $name=subsume tid=<<currentTiddler>>/>
</$list>

</$list>
#6

If first you drop it on tiddlywiki.com you can see how it works. Then you would need to modify the procedures within for each level and to suit your needs.

It seems the order year/month break is not your main requirement but that it is collapsible.

Subsume may be helpfull but its seems it is just one of a number of ways to open content.

I will reflect on this and see if I can give an answer. The wikis not online is it?

1 Like
#7

Thank you for reminding me to test it on Tiddlywiki.com.

I went and dropped the json file on tiddlywiki.com and the code was easier to see.

I made a new tiddler with your example filter and experimented with removing lines from the end.

You are correct, I’m really hoping it can be collapsible. A tiddler containing years of journal entries would be unwieldy if I couldn’t open it in sections.

I also tried dropping the subsume plugin on tiddlywiki.com and adding parts of it to your code like:

<$list filter="[range[0],<top.index>]" variable=position counter=item>**<$macrocall $name=subsume tid=<<currentTiddler>>/>**
      <$list filter="[current.year[]!match<last.year>]" variable=year><<new-year>></$list>
      <$list filter="[current.month[]!match<last.month>]" variable=month><<new-month>></$list>
      <$list filter="[current.doy[]!match<last.doy>]" variable=doy><<new-doy>></$list>
      <<listItem>>
</$list>

But that didn’t work : (

To answer your last question: Alas, the wiki, being a private journal, is not online.

#8

@Sapphireslinger my solution above allowed headers for each section without using nested lists. To make ìt all collapsible it may be easier to use nested lists in will share a simple example here soon.

#9

Thank you. Looking forward to it!

I learned a tiny bit today. I figured out how to use regexp on the journal-date field to make a journal month tiddler without having to use a month tag:


<$list filter="[tag[Journal]has[journal-date]regexp:journal-date[^202204]sort[journal-date]]">
<$macrocall $name=subsume tid=<<currentTiddler>>/>
</$list>

But it doesn’t help the main goal which is to not have to manually make month tiddlers. I imagine it would be better to sliderize the dates list that your group-dates.json produces.

Meanwhile every so often I open up your group-dates json and stare at it. Someday it will no longer be greek to me. ^o^

#10

Hi, I have only now got to review the suitability of my previous procedure Group-dates procedure.json to deliver your requirement, and I note that it is may not be suitable for your colapsable details view, because that most likely needs a nested structure, where my procedure avoids nesting in favor of detecting the change in year/month day.

  • Some CSS Wizz may be able to help.

I am keen on building a general solution for nested reports, but this is chalenging to design if it allows handling any number of nestings, that the user may specify. Although year/month/day is a common requirement.

  • I am sure you dont need to make month tiddlers, but hey if you can make it work, go for it.
  • In a way the method I used will “be somewhat difficult to understand by many” because it takes a whole new approach, so don’t be hard on yourself. It aims to be useable by everyone, not nessasarily understood by everyone (although some additional documentation would help).

The next step

I will start to investigate a solution to your problem and such problems in general. I expect it may still use a similar mechanisium, or a different structure all together. However here I am trying to write a general solution, innovate where possible and leverage some new tiddlywiki features to achive this, so my answer may not come too soon.

Here is a code pattern you may want to investitage in the mean time;

<$list filter="each year" variable=year>
Year header
   <$list filter="each month in year" variable=moy>
   Month header
      <$list filter="each day in month  of year" variable=dom>
         Day header
         <$list filter="each item" variable=item>
             Item header
                item template
             Item footer
         </$list>
         Day footer
      </$list>
   Month footer
   </$list>
Year Footer
</$list>

This could also be turned inside out item > dom > moy > year

  • The header an footer of each level could use the details widget (search for this and other options) to make everything inside it collaps.

The date functions inside my last procedure may prove useful for us both, and in future versions.

[Post script] It seems evident that breaking dates into years, months and even day of month is a very common need so extracting these to groups for nested lists would make sense as a subset of a more general nested list solution. The difference between each date group is as little as YYYY 0MM and 0DD.

  • Although there may be cases such as quarters, hours and minutes.
#11

Thank you so much for the code pattern!

I added the detail widgets. That appeared to work nicely.

Then I searched for “extract year” in the forum and found List by year/month? with your code for extracting years and months from a field:

{{{ [{!!created}split[]first[4]join[]] }}}
{{{ [{!!created}split[]nth[5]] [{!!created}split[]nth[6]] +[join[]] }}}
{{{ [{!!created}split[]nth[7]] [{!!created}split[]nth[8]] +[join[]] }}}

So I tried adding <$list filter=“tag[Journal]has[journal-date][{!!journal-date}split[]first[4]join[]]” variable=year> but nothing happened.

<$list filter="tag[Journal]has[journal-date][{!!journal-date}split[]first[4]join[]]" variable=year>
<$details summary="Year header">
   <$list filter="each month in year" variable=moy>
<$details summary="Month header">
      <$list filter="each day in month  of year" variable=dom>
         <$details summary="Day header">
         <$list filter="each item" variable=item>
             <$details summary="Item header">
                item template
             </$details>
         </$list>
         </$details>
      </$list>
   </$details>
   </$list>
</$details>
</$list>
#12

Your filter syntax is a bit borked! Try this:
<$list filter="[tag[Journal]get[journal-date]split[]first[4]join[]]" ...
Notes:

  • The filter was missing the initial open square bracket
  • The reference to {!!journal-date} will always refer to that field within the tiddler that contains the wikitext, rather than the tiddlers that match the preceding tag[Journal] filter operator.
  • Instead, use get[journal-date] to fetch the field value from the matched tiddlers. Note that has[journal-date] isn’t needed, since get[journal-date] only gives results for tiddlers that have a non-blank journal-date field… thus, has[...fieldname...] is automatically implied when get[...fieldname...] is used.

let me know how it goes…

enjoy,
-e

1 Like
#13

That was a while ago, I would perhaps do this now in 5.3.x

\function get.year() [format:date[YYYY]]
\function get.month() [format: date[0MM]]
\function get.dom() [format: date[0DD]]

demo
* {{{ [{!!created}get.year[YYYY]] }}}
* {{{ [{!!created}get.month[0MM]] }}}
* {{{ [{!!created}get.dom[0DD]] }}}
  • Here you get the tiddlywiki date serial from where ever it is, in this case the created field {!!created} then use one of the custom functions/operators. Note the format can be anything here
#14

@EricShulman thank you for elucidating the “get”.

However, it still did not work.

I did this:

<$list filter="[tag[Journal]get[journal-date]split[]first[4]join[]]" variable=year>
<$details summary="Year header">
   <$list filter="tag[Journal]get[journal-date]split[]nth[5]] tag[Journal]get[journal-date]split[]nth[6]] +[join[]]" variable=moy>
<$details summary="Month header">
      <$list filter="tag[Journal]get[journal-date]split[]nth[7]] tag[Journal]get[journal-date]split[]nth[8]] +[join[]]" variable=dom>
         <$details summary="Day header">
         <$list filter="each item" variable=item>
             <$details summary="Item header">
                item template
             </$details>
         </$list>
         </$details>
      </$list>
   </$details>
   </$list>
</$details>
</$list>

And got this:
experiment 02

Notes:

  • after changing the code: only one year header and one month month header and one day header and two item headers to choose from. (I should have multiple years and months and days of entries.)
  • before changing the code: the results were still erroneous, only showing two year headers to choose from, 4 month headers in each year to choose from, 6 day headers in each year to choose from, and 2 item headers (see below)

Before changing the code:

experiment 01
experiment 01598×622 23.4 KB

#15

@TW_Tones, thank you for the updated code to try:

I tried this:

\function get.year() [format:date[YYYY]]
\function get.month() [format: date[0MM]]
\function get.dom() [format: date[0DD]]

<$list filter="[tag[Journal]get[journal-date]get.year[YYYY]]" variable=year>
<$details summary="Year header">
   <$list filter="tag[Journal]get[journal-date]get.month[0MM]]" variable=moy>
<$details summary="Month header">
      <$list filter="tag[Journal]get[journal-date]get.dom[0DD]]" variable=dom>
         <$details summary="Day header">
         <$list filter="each item" variable=item>
             <$details summary="Item header">
                item template
             </$details>
         </$list>
         </$details>
      </$list>
   </$details>
   </$list>
</$details>
</$list>

And got this:

experiment 03
experiment 03602×677 24.4 KB

  • I got over a hundred year headers, but I know I don’t have journal entries spanning a hundred years.
  • There are only two month headers appearing in each year, and only two day headers appearing in each month and two item headers in each day.
#16

@EricShulman I found some old code that you helped me with 3 years ago: https://groups.google.com/g/tiddlywiki/c/H-teh7Cv0kg/m/VmhQqCTdBgAJ

I tried to repurpose it for here but that last list filter especially has me stumped. I hardly know how to formulate it. Any illumination you can give is appreciated.

Note: I no longer tag my journal entries with a year and month but they do have the journal-date field with this format YYYY0MM0DD0hh0mm0ssXXX

Note: My tiddlywiki is still 5.1.22 I did try out upgrading it recently for this project but the upgrade slows my whole wiki way down even after working on it for awhile so I went back to the older version for the present.

<$list filter="[range[2021,2023]]" variable="year">
   <$list filter="[tag[Journal]get[journal-date]split[]first[4]join[]limit[1]]">
      <<year>><br>
      <$list filter="[range[1,12]addprefix[$:/language/Date/Long/Month/]get[text]]" variable="month">
         <$list filter="[tag[Journal]get[journal-date]split[]nth[5]] [tag[Journal]get[journal-date]split[]nth[6]] +[join[]limit[1]]">
            &emsp;<<month>><br>
            <$list filter="[tag[Journal]search:journal-date[first[4]=<year>]sort[journal-date]] [tag[Journal]get[journal-date]split[]nth[5]] [tag[Journal]get[journal-date]split[]nth[6]] +[join[]=<month>">
               &emsp;&emsp;<$link /> <br>
            </$list>
         </$list>
      </$list>
   </$list>
</$list>
#17

Just an update on bits of code I am trying that haven’t worked. Thanks for letting me put it here:

It occured to me that if <<year>> and <<month>> were what I needed the first six digits of the journal-date field to match then I could try this:

<$list filter="[tag[Journal]regexp:field:journal-date[^<<year>><<month>>]sort[journal-date]]">

But I’m sure the format is all off and it did not work.

Brave Leo AI suggested this:

<$list filter="[tag[Journal]]">
  <<year>>: <<month>>
  <<if $year == $currentTiddler.journal-date.substring(0, 4) && $month == $currentTiddler.journal-date.substring(5, 6)]>>
    <<$currentTiddler>>
  <<else>>
    <</if>>
</$list>

Note: <<year>>: <<month>> is probably nothing I want.

Note: I changed the 0,6 that was suggested to 0,4 and the 6,8 to 5,6.

Note: <<if $year == $currentTiddler.journal-date.substring(0, 4) && $month == $currentTiddler.journal-date.substring(5, 6)]>> seems like it’s so close to what I want. I really don’t want to bother with date per se, but just treat it as “Are the first six digits in the journal-date field matching the output of the year and month variables?” if that makes sense.

Note: I’m trying to solve this part of the code before I add any more code like reveal widgets.

#18

Unfortunately, most LLMs don’t seem to know much about TW syntax, and they won’t tell you this, either. None of this is meaningful in TW:

<<if $year == $currentTiddler.journal-date.substring(0, 4) && $month == $currentTiddler.journal-date.substring(5, 6)]>>
    <<$currentTiddler>>
  <<else>>
    <</if>>

This is a good instinct! Here’s one way to do it:

<$let match=`$(year)$$(month)$`>
<$list filter="[tag[Journal]search:journal-date:anchored<match>]">

</$list>
</$let>

Notes:

  • I know the documentation can be a bit confusing on this point, but filter operators can only take an operand in one set of [ ] { } or < >, with no internal nesting.
    • Within a filter, we use…
      • [text] to refer to a literal value text (as you did with tag[Journal])
      • <text> to refer to the value of the “text” variable, defined outside the filter
      • { } to refer to a value being transcluded from a field (so {!!journal-date} will take the full content of the <<currentTiddler>>'s journal-date field, and {journal-date} will take the text field of the tiddler called “journal-date”, if it exists.)
  • This means we need to build the <<year>><<month>> combination before we can use it in a search.
    • Here, I’m using the $let widget to do it, because it lets me use substituted attribute values. Within the backticks, we have to use $(year)$ instead of <<year>>, but it’s easier to combine variables (and/or other text).
    • An “old-fashioned” way to do this might use <$let match={{{ [<year>] [<month>] +[join[]] }}}> or <$let match={{{ [<year>addsuffix<month>] }}}> instead. Here, we’re using filtered transclusion to assign the first result of a filter to the variable “match”. It’s obviously a bit less concise, since we have to use filter syntax to do it.
  • Once we’ve defined the target string and assigned it to a variable, we can use it in the filter.
    • regexp doesn’t really help us here, since it expects a regular expression, not a TW variable.
    • Instead, I used the search operator with the anchored flag, so it will look only at the start of the journal-date field.

There are other ways to do this (you could set the value of <<match>> with \define or \function, for instance) but I think this is probably one of the simplest. Let me know how it goes!

1 Like
#19

If you have the date from which you would obtain year and months use this approach;

[{!!journal-date}format:date[YYYY0MM]] and use this for your tests.

#20

As someone who is not familiar with RegEx, I figured I would give this a shot:

NestedDetails
NestedDetails615×597 20.3 KB 

<$list filter="[tag[Journal]get[journal-date]format:date[YYYY]each:value[]sort[title]]" variable="YEAR">
<details>
<summary><strong>Journal <<YEAR>></strong></summary>

<$list filter="[tag[Journal]get[journal-date]prefix<YEAR>format:date[YYYY MMM]each:value[]sort[title]]" variable="YEARMONTH">

<details>
<summary><<YEARMONTH>></summary>

<ul>
<$list filter="[tag[Journal]] :filter[get[journal-date]format:date[YYYY MMM]match<YEARMONTH>]" variable="ENTRY">
<li><$link to=<<ENTRY>> />&emsp;(<em><$view tiddler=<<ENTRY>> field="journal-date" format="date" template="YYYY-0MM-0DD"  /></em>)
<br><$transclude tiddler=<<ENTRY>> /></li>
</$list>
</ul>
</details>
</$list>

I don’t know if this will hold up when using a lot of entries, but it might give you some ideas.

1 Like