Relative date between two given dates, not now - native tiddlywiki

#5

Yes, I think so. I’ve started looking at it.

Great! I think this belongs in the core, though, so I hope you’re ok with me “looking” at your code and creating a new official DateFormat token for this?

As a programmer, Unix timestamps are great for simple datetime calculations, so I think it belongs in the core of any language.

I’m sure you know this, but Unix timestamps are always “in UTC”, which is one of its advantages. It’s like a global, stricly increasing counter, and the same point in time around the world has the same timestamp value.

I see that both tokens “ss” (seconds), and “XXX” (milliseconds) are taken, and since most systems use “S” for Unix timestamps, I think we should too. There’s always a small risk of confusing different tokens, but I don’t think picking “X” instead will be any better. Any thoughts?

#6

Sorting dates, comparing dates etc… are fine but I believe we need to get add and and subtract date functions even if it is only days. Do any of these dates allow us to convert, add/subtract and convert back without error?

#7

My convertdate[...] filter isn’t just for converting to/from “epoch” timestamps. It can also be used to convert an ISO8601-compliant formatted datetime input into another formatted datetime output by using TWCore standard date codes.

As such, it can accept a local datetime text value entered by a user and output another formatted datetime text value or an epoch timestamp that represents the same local datetime. It can also automatically convert a local datetime input into a UTC datetime or epoch timestamp by adding/subtracting the appropriate timezone offset. This offers significantly more utility than simply converting to/from UTC epoch timestamp values, and I’d hate to see any new extension of TWCore datetime functionality limited to just handling UTC epoch timestamps.

-e

#8

Yes, you can take a “regular” date, convert it to Unix time, do mathematics with it, and convert back to a “regular” date. That’s the beauty of it. Javascript has this built in, and takes timezones and daylight savings into account when converting.

I’m not sure I follow. I’m talking about adding a missing token to DateFormat, to make it possible to format a datetime as Unix time.

What is it that you don’t like about that?

#9

When you have a solution, plugin or core please provide instructions on how to do this if possible.

Or if @EricShulman timer tools can do it, what is the minimal package, I can install to get this date addition/subtraction feature, it is this we need in the core. It’s like having a car (tiddlywiki) that you can’t schedule for it next service.

  • Then ideally we get this into the core. I suspect this code is somewhere deep in the core already just not exposed.
#10

All you need is
https://tiddlytools.com/timer.html#TiddlyTools%2FTime%2FConvertDate%2FFilter

Then you can write filter syntax like this:

<$let fmt="MMM DDth YYYY at 0hh:0mm:0ss.0XXX"
  oneyear={{{ [[365]multiply[24]multiply[60]multiply[60]multiply[1000]] }}}>
<$edit-text field="start_date"/><br>
One year later is:
{{{ [{!!start_date}convertdate[timestamp]add<oneyear>convertdate<fmt>] }}}
</$let>

Notes:

  • fmt and oneyear are provided for readability in this example
  • You can enter a date in any ISO8601-compliant format (see ISO 8601 - Wikipedia)
  • The filter converts the input to an numeric “epoch time” value using convertdate[timestamp]
  • Then it add one year (in milliseconds)
  • Then it converts to the desired output format
#11

The first issue is that adding a new token to the DateFormat handling (see formatDateString in $:/core/modules/utils/utils.js) is problematic with regard to backward compatibility. In the current TWCore code, any unrecognized characters in the format string are “passed through unchanged”, and adding a new token could result in previously unchanged “pass through” text being processed and replaced with a formatted datetime value.

We ran into this problem in the past when we added the “day of year” format. We had initially chosen “dd” and “0dd” as the formatting codes, but this resulted in text such as “tiddler” being converted to “ti365ler”. To avoid this, we changed to using “ddddd” and “0ddddd”, reasoning that those sequences of 5 "d"s, while certainly more awkward, were unlikely to occur in normal text.

Thus, using a simple format token like “S” (or “X”) will break existing handling for passing through plain text (e.g., formatting “Something YYYY/0MM/0DD” could result in “1640995200000omething 2022/07/16”).

This is why my TiddlyTools ConvertDate filter uses timestamp as a stand-alone keyword (not to be combined with any other DateFormat tokens) and also why I opted to create a completely separate filter (convertdate[...]) rather than than attempting to add another token to the DateFormat handling.

A second issue is that the current format:date[] filter only accepts input values that use the TWCore 17-digit “system date” format of “YYYY0MM0DD0hh0mm0ss0XXX” and cannot handle any other formatted date text. In contrast, my convertdate[...] filter also handles conversion from any ISO8601-compliant formatted datetime text representation to any TWCore date-formatted text representation, enabling usage like this:

Enter your birthdate:<br>
<$edit-text field="test-date"/> {{{ [{!!test-date}convertdate[was a DDD]] }}}<br>
<$let years=60 msec={{{ [<years>multiply[365.25]multiply[24]multiply[3600000]] }}}>
You will be <<years>> years old on 
{{{ [{!!test-date}convertdate[timestamp]add<msec>convertdate[DDD, MMM DDth YYYY]] }}}
</$let>

Try entering your birthdate as formatted date text (e.g., “July 24 1962”)

-e

1 Like
#12

@EricShulman certianly deserves the title

TiddlyWiki Timelord

:nerd_face:

#13

Great information, thanks for sharing! I understand your point, and agree with you that a new token maybe isn’t the best way to solve this problem.

Another thing that I realized is that formatting a date as Unix time is not the right way to think about it. Some languages really do format to Unix time, but not e.g. .NET, where you have other ways to convert to and from Unix time.

Why? Because C# is a typed language, and you want your Unix time as a long, not a string – and formatting always returns strings. (In TiddlyWiki, it probably wouldn’t matter, since we parse strings like ‘123’ to a number before doing maths on it.)

So, does this mean we want the convertdate filter operator in the TiddlyWiki core?

#14

I see now that convertdate only handles ISO8601-compliant datetimes, and not TiddlyWiki’s own format, and my own shorthand formats (omitting dashes and spaces).

None of these work as I would expect them to:

*{{{ [{!!created}convertdate[timestamp]] }}}
*{{{ [{!!created}convertdate[]] }}}
*{{{ [[202207180910]convertdate[timestamp]] }}}
*{{{ [[202207180910]convertdate[]] }}}
*{{{ [[20220718]convertdate[timestamp]] }}}
*{{{ [[20220718]convertdate[]] }}}
#15

Here’s some of the internal code for the convertdate[] filter:

if (format=="timestamp") {
   results.push(new Date(title.replace(re,"$1")).getTime().toString());
} else {
   if (title.match(/^-?\d+$/)) {
      results.push($tw.utils.formatDateString(new Date(Number(title)),format));
   } else {
      results.push($tw.utils.formatDateString(new Date(title.replace(re,"$1")),format));
   }
}

As you can see, the code uses new Date(...) to turn the input to a JS date object. This standard JS method only accepts ISO8601-compliant text or epoch date signed integers as input, and thus does not handle the TWCore’s 17-digit “system date” format (or any abbreviated forms thereof).

Also, take note that, if the input is an integer value (as detected by the regexp pattern “/^-?\d+$/”), then it is assumed to be an epoch date value and is passed directly to new Date() to turn it into a JS date object.

In order to achieve the results you want, you would first need to format the input to an ISO8601-compliant form by using the TWCore’s existing format:date[...] filter, like this:

*{{{ [{!!created}format:date[YYYY 0MM 0DD 0hh:0mm:0ss.0XXX]convertdate[timestamp]] }}}
*{{{ [{!!created}format:date[YYYY 0MM 0DD 0hh:0mm:0ss.0XXX]convertdate[]] }}}
*{{{ [[202207180910]format:date[YYYY 0MM 0DD 0hh:0mm:0ss.0XXX]convertdate[timestamp]] }}}
*{{{ [[202207180910]format:date[YYYY 0MM 0DD 0hh:0mm:0ss.0XXX]convertdate[]] }}}
*{{{ [[20220718]format:date[YYYY 0MM 0DD 0hh:0mm:0ss.0XXX]convertdate[timestamp]] }}}
*{{{ [[20220718]format:date[YYYY 0MM 0DD 0hh:0mm:0ss.0XXX]convertdate[]] }}}

Perhaps I can add a filter suffix (e.g., “convertdate:system[…]”) to indicate that the numeric input is a TWCore “system date” (or an abbreviated form of system date) rather than an epoch date signed integer value. This suffix would then automatically apply the format:date[...] handling internally, so that the filter syntax could be simplified to something like this:

*{{{ [{!!created}convertdate:system[timestamp]] }}}
*{{{ [{!!created}convertdate:system[]] }}}
*{{{ [[202207180910]convertdate:system[timestamp]] }}}
*{{{ [[202207180910]convertdate:system[]] }}}
*{{{ [[20220718]convertdate:system[timestamp]] }}}
*{{{ [[20220718]convertdate:system[]] }}}

Let me experiment a bit to see what I can come up with…

-e

#16

Yes, that works, but is cumbersome.

My reading journal now calculates how many days it took to read each book, and even the number of pages read per day! :nerd_face:

<$list filter="[<currentTiddler>status[complete]]">
<$list filter="[<currentTiddler>has[start-date]]">
<$list filter="[<currentTiddler>has[end-date]]">
	<$let
		start={{{ [<currentTiddler>get[start-date]format:date[YYYY-0MM-0DD 0hh:0mm:0ss.0XXX]convertdate[timestamp]] }}}
		end={{{ [<currentTiddler>get[end-date]format:date[YYYY-0MM-0DD 0hh:0mm:0ss.0XXX]convertdate[timestamp]] }}}
		days={{{ [<end>subtract<start>divide[86400000]fixed[0]] }}}
	>
		<tr>
			<th align="right">Reading time</th>
			<td><<days>> days
			<$list filter="[<currentTiddler>has[pages]]">
			<$let ppd={{{ [{!!pages}divide<days>fixed[0]] }}}>
				(<<ppd>> pages per day)
			</$let>
			</$list>
			</td>
		</tr>
	</$let>
</$list>
</$list>
</$list>

There are three types of inputs, right?

  1. Unix time (any number!) – like 1658155612
  2. “TiddlyWiki datetimes” (numbers that look like ISO datetimes) – like 20220718164703, but also 20220718
  3. Javascript-Date()'s accepted inputs – like “2022-07-18 16:47:16” or “Mon Jul 18 2022” or “12:34:56”

The first two overlap, but not the third one. A Unix time can also be a valid TiddlyWiki datetime.

My suggestion would be to consider #2 and #3 the defaults/expected inputs, and #1 the special case.

Then you would have to type convertdate:timestamp[…] when you’re coming from a Unix timestamp, which also is symmetrical with convertdate[timestamp], right?

#17

I’ve updated https://tiddlytools.com/timer.html#TiddlyTools%2FTime%2FConvertDate%2FFilter to add the :timestamp operator suffix as we have discussed.

Syntax now supports:

  • convertdate[timestamp] - converts ISO8601 OR TWCore datetime text value to posix time epoch signed integer
  • convertdate[outputformat] - converts ISO8601 or TWCore datetime text value to TWCore Date Formatted text output
  • convertdate:timestamp[outputformat] - converts posix time epoch signed integer to TWCore Date Formatted text output

enjoy,
-e

1 Like
#18

We have an outstanding PR for the core for a parseDate operator which has a very narrow scope. It would be helpful @EricShulman and @cdaven if you reviewed it and provided any relevant feedback: https://github.com/Jermolene/TiddlyWiki5/pull/6512

#19

I’m not sure whether I should put my feedback here or there, but anyway:

As I understand it, the use case for parsedate is this:

  • DatetimeJS → DatetimeTW

You can convert a datetime from some formats that Javascript supports (a subset of ISO, for one) to the TWCore format.

convertdate supports this as well (the test cases from that PR give the same results). It also adds these use cases:

  • DatetimeJS → DatetimeCustom
  • DatetimeJS → DatetimeUNIX
  • DatetimeTW → DatetimeCustom
  • DatetimeTW → DatetimeUNIX
  • DatetimeUNIX → DatetimeCustom
  • DatetimeUNIX → DatetimeTW

It has three acceptable input formats: JS/ISO, TWCore, Unix time

And three output formats: TWCore, custom format/template string, Unix time

Unless I’m missing something, I would say that convertdate is a much better candidate for inclusion in TiddlyWiki core. Well done, @EricShulman!

(Note that “DatetimeTW → DatetimeCustom” is the same thing (?) as format:date, which could also be obsoleted by convertdate. Again, unless I’m missing something… Chesterton’s Fence, and all that.)

#20

hmm… I’m thinking if my convertdate filter is to be a candidate for core inclusion perhaps I should change the filter operator name to “parsedate”, and use “number” as the keyword suffix/operand for conversion to/from “unix date”:

parsedate[...]
parsedate[number]
parsedate:number[...]

where “…” is any TW Date Format string which, if omitted defaults to “[UTC]YYYY0MM0DD0hh0mm0ss0XXX”

-e

#21

This is all very promising nice work team, @cdaven and “tiddlywiki timelord Eric”.

I just ask that you do not loose site of the OT and the high demand need to add and subtract time. Erics previous example makes this achievable and could be simplified with prepared macros/and variables like “oneyear” “onemonth” or ultimately an “Nth-timeunit” etc…

  • I mention this now because it would possibly not take much to go a little further and solve this long standing gap of simple date maths.
  • We must keep an eye on making this simple where possible for the range of skill levels in our community.

Love your work folks.

#22

I’ve just renamed
https://tiddlytools.com/timer.html#TiddlyTools%2FTime%2FConvertDate%2FFilter
to
https://tiddlytools.com/timer.html#TiddlyTools%2FTime%2FParseDate

and changed the filter definition to use
parsedate[...] - converts ISO8601 or TWCore datetime text to TWCore Date Formatted output
parsedate[number] - converts ISO8601 or TWCore datetime text to “unix time” signed integer value
parsedate:number[...] - converts “unix time” signed integer value to TWCore Date Formatted output

note: “:number” operator suffix and “[number]” operand value are both literal keywords

-e

1 Like
#23

I liked convertdate, but I don’t mind parsedate either. “Parse” to me is taking a string that’s not really a date and creating a date from it, while “convert” is saying that the input must be a date of some kind.

If I’m thinking out loud, maybe “number” should be “milliseconds”, and we could add “seconds” as well?:

  • parsedate[seconds]
  • parsedate[milliseconds]

Is that more understandable than “number”? At least it would give some clue as to why you’re dividing by 3600 or 3600000 and so on.

But then again, “number” is more abstract, and I think maybe using concrete terms like “seconds” or even “hours” could be confusing. I mean, a date and a time converted into “seconds”…

In C#, the DateTime class has methods AddDays(), AddHours() and AddSeconds(). If you want to subtract, you use negative numbers.

What if we could do something like this?

[{!!created}adddays[1]] for adding or subtracting days, hours, seconds. It would first convert/parse the input format, then do the arithmetic, then convert/parse back to the original format, I think.

And possibly [{!!modified}subtractdate{!!created}] to get the time difference in seconds or milliseconds?

#24

Date(), and therefore parsedate, also supports RFC 2822 datetimes, as @pmario noted here: https://github.com/Jermolene/TiddlyWiki5/pull/6512#issuecomment-1065894935