Created dates maintained by core, can we make a minor change? Uniquify

Developers
#1

As you know when “timestamps are on”. tiddlywiki will assign a created date to tiddler once, as its created or if it has no created date and is edited. This date stamp should remain unchanged for the life of that tiddler. Lets ignore intentional change or deletion of the created time stamp.

  • There is an assumption that such a date time stamp could be the same for more than one tiddler

I was wondering if and how we may go about modifying the created date time stamps for tiddlers created in a given wiki to be unique? simply by adding a millisecond until it is unique.

This would then allow the created date to act as a (mostly) unique identifier, and hashed versions ie reducing the decimal timestamp to a base 64 version to reference, that date using fewer bytes.

For example you may have a list or data tiddler that can uniquely list title references this can be converted back to decimal then search for the “only” tiddler with that “unique” created date. Remember the created date will not change as a rule.

Why?

  • It is an interesting experiment
  • Its a type of semi-unique tiddler serial number or unique ID
  • We get a unique alias for tiddlers we apply this to
  • We can list titles using this alias with less bytes while leaning on an existing value in tiddlers

Rather than argue the relative merits of this, could I first get your advice on how I could make this happen? Through that mechanism we will discover more about its merits and opportunities and limitations.

[After words]
I plan to allow a created date to be modified to be unique “just in time”, when we are looking to list a tiddler with its unique created date reference. We need only check for uniqueness when Creating tiddlers and or/referencing them using the “unique” created date. Otherwise we don’t care what the created dates are.

#2

From my point of view, if something is designed to be changed, It can not be used as an ID anymore – because it can be changed.

That problem was introduced when the “Timestamp” $:/config/TimestampDisable configuration was introduced. Created can not be “trusted” anymore. Created and modified are just fields that are used for “user-specific” sorting. So it looks nice in the UI. That’s their new use-case now.

If we need a UID it has to be designed to be created once. One rule needs to be: There is only one place in the core that defines that value, then it is never touched again.

Request - Make Timestamp On / Off More Configurable
#3

Mario, If I think through the use of my proposal I can see it being of value as a partial, defacto unique id, if a tiddler is created with timestamps off, it just can’t be used this way.

  • No extra bytes required
  • Can be (re)validated when using it first as a reference ID
  • unlikely to have many clashes - time and date needs to be the same and have the milliseconds identical 1 in 1,000 so the probability of a clash is in the millons or greater.

But if on then the created date will be applied on the next edit (I believe) the thing is if the core sets the created date by making it unique (by adding milliseconds) normal creation is captured. If we import tiddlers they are statistically unlikely to have the same date.

Please respect my Request;

Now lets just assume a given tiddler has a unique created date, at the time it was created in wiki, and I decide to add this “unique code” to some data tiddler, as a link etc… I can first test if it is unique still, and address this before relying on it. If it proves to be a problem we could add this uniqueness mechanism to import.

I am now looking at seeing If I can identify how with ChatGPT

I am having some success with this, where we register newly created or modified tiddlers that have been forcibly given unique created date timestamps.

  • I am working on the import process now
#4

I think your ideas are good, but I think they lack scalability.

And that’s the crux. IMO The assumption is wrong, because it does not scale well. If something should land in the core, we have to have the bigger picture in mind. TW does not win a popularity contest, but we can assume that there are 10000 users / wikis out there.

The question for collision probability here is: How many tiddlers do we need in 2 wikis, where we have a probability from 1 out of x that there is a collision.

The answer for x is 448, with the following assumptions:

  • There are 10000 wikis, that use this mechanism
  • 30 Tiddlers per day are produced
  • We have a 9 hours per day, where tiddlers are produced.
    • Since TW stores the date format with UTC format,
    • Tiddler creation times are when users are awake. So there will be ranges which are the same world wide. (Increasing it to 14h per day - does not matter as much as I thought. See the details.)
  • We look at the probability of a collision after 10 years from now

The full details can be found at https://claude.ai/share/6f86c7b9-e7fe-4764-9b41-02fa8d8544ce … I am not a mathematician, but the calculations look reasonable.

The main problem with collision probabilities is the Birthday problem, which states:

From wikipedia:

The birthday problem asks for the probability that, in a set of n randomly chosen people, at least two will share the same birthday.

Here the number is 23 for a 50% chance of a “collision”. This number is incredibly low, compared to the 356 numbers of day per year.

Why this matters? - The created date down to the millisecond can be considered the birthday of a tiddler. Our ranges are much higher considered we have much higher possible number of keys.

But still with some reasonable assumptions: We have 10000 wikis, that produce 30 tiddlers a day. Ranges created within 9h per day with 250 working days.

You can see the details in the linked conversation with Claude near the end.

  • Claude does suggest to use more 4 digits to be “safer”. … But I would say:
  • That’s still not good enough.

Even with the early numbers, where we did consider the whole future number range for 130 years, it would not be good enough in my books. Where we have 1 in 1000 with 12000 tiddlers per wiki. That’s “concerning” to use the LLMs words.

Here is the conversation as an MD file, that you can feed own LLM, to verify or “to slate” it.

timestamp_collision_conversation.md.tid (8.4 KB)

From your replies I can see, that you want to use the created date field, since it is already there. So no extra “space” needed. So tiddler size does not increase.

  • But with 4 extra digits as Claude suggested, we do need more space
    • As I wrote, I think that is not good enough.
  • If we do already need more space we can make it bullet-proof

Some time ago I did post a similar “train of thoughts” at GitHub: Using New UUID v7 as a Tiddler ID for MWS Related Data Storages · TiddlyWiki/TiddlyWiki5 · Discussion #9035 · GitHub

Which also suggests to use a new created field with a really unique format. UUID v7

  • The properties are the same as with your changes
  • It’s transparent for users - The do not even see it
  • Implementation should be straight forward
  • We can do it at import time
  • We can create an automatic upgrader, so users do not need to do anything

The OP is long, but I think the implementation done in the right places will only be a view lines of code.

The crux with my suggestion

Off course it also has one (or more :wink: UUIDs are horrible to “remember” by users.
The UUID contains this timestamp: 2026-03-12 16:40:23.524 UTC in the bold section. The rest are random numbers in the italic section. 7 is the version number.

UUID-v7: 019ce2eb-b2e4-739b-bd29-4f9b77c76b04
vs.
created: 20260312164023524

But if you want to use your “uniquified” date as links to tiddlers, it also is hard to read by humans.

Aliasing those IDs is an interesting idea, that we may follow. … But this will cost additional “space”

Just some thoughts.

have fun!
Mario

#5

If the system you’re proposing would add a UUID field with a unique value either at new tiddler creation or during the import process, would this make it a good candidate for a plugin? While I understand why someone might want tiddler UUIDs, they (and any architecture needed to support them) would take up more space, as you say… and personally, I’d really like to avoid adding a new field I’ll never use to tens of thousands of tiddlers. (I even delete most of my created fields!)

Request - Make Timestamp On / Off More Configurable
#6

I think it can be done as a plugin. We are brainstorming here

For my personal usecase, it would not add an UUID field, it would replace the created date field. I personally do not really want to carry it around anymore. It can be affected by the “Timestamp setting”, so I can not trust it anymore.

I’d remove it - same as you - but for a different reason :slight_smile:

The existing created field in a .tid file in sum uses 26 chars:

created: 20260312164023524 … 26 characters including “:” and “space”

A new c7 field would use 40 chars.

c7: 019ce2eb-b2e4-739b-bd29-4f9b77c76b04 … 40 characters including “:” and “space”

So it’s 14 characters more per tiddler in a file.

  • All the time related functionality will be the same
    • Search, Sort …
  • It will be transparent for the user
  • UUID can be used and shown if we need to

@TW_Tones … Alias Idea

I can see that.

With my proposal no checks needed. We know it is an universal ID.

Yes. The upgrader can handle that. Plus other things we may want.

Now the cool stuff follows. As I showed already. Neither the current date nor a new UUID will be human friendly. But I do like the idea of using a human memorable “phrase” to identify them. There are existing mechanisms to encode hex values in long “seed phrases”. But we do not need them to be long.

  • For our usecase, they could be 2 or 3 words to remember to get a 100% hit
  • With 1 word, we could cover a 9 hour window. With the second one 16 second

So if you type in 1 word into a created search input field, it will list all tiddlers from within a 9 hour range … if my math is right.

Since the “word list” can be fixed, it would not even be needed to create a data-tiddler, for the lookup. There can be an algorithm that calculates the phrases / aliases as you called them.

So if I have memorised “2 words” from my own wiki for a certain time period, it will work for every tiddler for the same time period. … That’s interesting.

Choosing the right words for the right bits. We could have: Show all tiddlers from the year of the “horse” or “dragon” if we want to. … Just dreaming :wink:

-m

#7

If that’s a concern. You probably should also make sure your type: text/vnd.tiddlywiki (which is default if missing or empty) is removed from TW syntax tiddlers. And fields that are empty, which tags: sometimes is.

Just a thought.

#8

Yes, I have a clean-up procedure I run regularly.

#9

I was writing a substantial and informative reply on my mobile and lost it.

I will have to return later. please be aware I have found a number of things that mitigate the issues that may arise so see real value proceeding. so again focusing on the mechanisms I need

  • import tiddler uniqueness
  • new tiddler uniqueness
  • modified tiddler check and uniqify once
  • when adding a tiddler to a list or table via the created value ensure uniqueness ie we tolerate duplicates in the greater set of tiddlers because we can ensure uniqueness only when required / and registered.

you may notice to achieve the above I need a registry of created dates to make the efficient and allowing me to record those dates ensured unique. only imported, modified and new tiddlers need to be checked against this, or registered.

  • I am currently using this registry to store the title at creation however only the key date is needed to test uniqueness and register dates.
  • given this mechanism I need not proactively change existing dates in the wiki.

so I have all this working including a cache of created dates for quick check and registration, except the import process ATM.

As usual for tiddlywiki this design is fault tolerant, and just in time, permitting use in the real world.

#10

I expected the claims above to generate :thinking: so;

First Let state the Obvious

Tiddler Titles are unique! but not over time,

  • we openly can rename them and want to.

The Proposed locally unique references are;

Locally Unique over ALL time

I will just expand a little, ensuring a tiddler only requires a modification of the created so no additional bytes, but for efficiency I am storing a list of items granted a unique created date allowing efficient testing of new (rarely clash as in the future) and incoming tiddlers created dates. So only for “uniquified” created fields/tiddlers do I store double the created date.

  • In addition we store the original title as a historical record of uniqifcation, which we could engineer into relink if we wanted, or dump altogether.
  • Clashes may occur in the future for rapid batch creation but we intervein to stop this.

My intentions are

  • only in wiki uniqueness
    • because I commonly have long titles (this can shorten a list of titles) by instead using the locally unique reference.
    • Thus something like my flags solution could list these unique created dates rather than the actual title. Typically saving space as I tend to use a lot of long prose like titles.
  • As a rule I do not modify or use tools that modify the created date, to me it is sacrosanct. And I tend never to defeat the modified date either.

Other experiments present themselves with “this in place”;

This needs to be explored further once we have a working solution. Below is brainstorming;

  • The URL to a site acts as a universally unique address
    • Generate a permalink that goes via the unique ID rather then the title, that can change (POC done)
    • Obscure content in link with a UID reference
      • Also stopping the need for % encoded URL’s which also get long and unreadable.
      • Also Reduce the size of permaviews
  • Obscure titles with answers to questions on a quiz site
    • Could be further obscured by using a reversable mathematical encoding of the created date.
  • Maintains true serial dates ie no created date need be the same as another, thus we know the exact order of creation even with batch processes
  • More power in relationships by dates tools, such as via @EricShulman calendar tools.
  • Interestingly a unique link using a created date in the link provides a small piece of information to the location storing that link (another tiddlywiki), ie they can identify the created date of the destination in the link :thinking:
  • Using the created serial when linking tiddlers with relationships allowing reduced bytes, obscuring and no need for relink if titles are renamed.
  • Logging and tracking content using the locally unique created date as the key, reducing bytes
    required and permitting export and deletion, allowing reimport if needed because the keys remain unique, adding a degree of obfuscation (hiding the title).
    • Could include field values before change,
      • allowing rollbacks per field.
      • Or shove it into diff match patch allowing selective update
        • Read only Reviewers can generate JSON file(s) of these to email to wiki owner. In this case even renames and possibly deletion suggestions.

This is only necessary for Universal Unique ID’s. I am only ensuring local unique ID’s and intervein on import. For me Imports are usually not “content tiddlers” they tend to be macros and plugins (usually my own) and I just need to avoid a clash with my registered tiddler, not others.

  • I am happy to leverage the publish domain/folder as the unique wiki prefix which has the result of turning locally unique tiddlers into universally unique.

My argument here is we could cause and force uniqueness of created dates just as we now apply created dates, particularly helpful on batch generation or bulk import, because unique means serial and separable order of creation/import.

  • A subset of which we may register to make use of that uniqueness and simultaneously guarantee from existing created dates.

  • Perhaps introduce an Import date field that does not change time stamps, except for created date uniquification.

What we call a birthday is a month/day of any year and from a set of maximum size 365 I am including years and hours, minutes and microseconds.

Note it can be read as a date/time stamp which is understandable or translated to the current titles name carrying that created value.

  • But this unreadability can be useful.

How can this be so?, With “Timestamp setting off” new tiddlers can be created without timestamps, they will not be unique, created will be blank. Only those we want to refer to as unique need be set, as long as creation (future dates) and imports (not future dates) stop future tiddlers clashing.

@pmario perhaps we can use my solution allowing tiddlers to be enrolled in a locally registered unique created date, then add a layer that allows them to become universally unique. This can be achieved separately or in an integrated method where we may be able to save further bytes. We could also design this perhaps to generate UUIDs only on export or request, but update the local UUID to make it so only on export (unique option?)

  • This is in keeping with incrementally developed solutions often found in TiddlyWiki

Why @etardiff is it simply to reduce bytes?

  • I would expect this should be only in good use cases.
  • I personally would not do this as information available in Created and Modified dates are critical real world and system fields, that may prove useful in retrospect.
  • I am tempted to add an import date/time stamp silently and uniquely in case one day this may prove useful eg; to identify changes that result in a fault from a particular date.

This makes me wonder if a log of modified date / locally unique ID could inform us of the multiple modifications organised by date stamp against a particular tiddler. Even if the actual change on each date is lost in the fog of time?

Modifier and Creator

I would also note I think we totally under utilise the “Modifier and Creator” fields, at least I do. I rarely change the $:/status/UserName and if we used short codes import batches or times, project names etc… we could even hide creators using a stamp. All we need to do is extend the uses for modifier or creator while maintaining is original use. eg a modifier could be a concatenated current user + import batch, or the Creator = the creator + a small uniquification string.

  • Note creator never changes unless there is an intervention so is a useful field on its own. Arguable blank creators are an opportunity to set something forever

What do we need?

  • Intervein in the provision of created date at a low level to ensure new tiddlers unique - successfully POC’ed
  • Intervein in the created date on import to ensure they don’t clash with existing created dates - should be somewhat easy
    • May allow toggle setting of import date, also identifying imported tiddlers

Warnings

  • Tiddlers installed by Bookmarklets may bypass this facility.
    • My bookmarklet site could be designed to copy created date to a bookmarklet date and clear the created date, perhaps setting the creator to the bookmarklet name. Or use the creator field?
    • My bookmarklet tool could include the bookmarklet tiddler allowing access to the details and set its installed date.

I hope someone cares about my work here?

#11

I’ve been trying to catch up on the interesting threads taking place while I’ve been heads-down elsewhere. This is definitely among the interesting ones!

Yes, that’s how I would go about it. But I probably wouldn’t do it now.

I do a lot of conversions to TW from outside sources, and I usually just generate tiddlers from external sources in Node. It used to be that I would give them all created/modified dates, and sort them in whatever order I wanted by starting with a Unix timestamp and continually incrementing it for each file. It worked, but there was really no good reason to do that, as @pmario taught me several years ago. I now rarely worry about created date at all, and only use modified date for ad hoc searches of my own recent changes.

Agreed, it is very interesting. I think one early design mistake of TiddlyWiki was to use a mutable title as the unique key; we would have been much better served with a UUID. But it’s also far too late to correct that, in my opinion.

I haven’t read @pmario’s AI dialog, but I did run a calculation of the Birthday Paradox for a single day, and found the 50% threshold for the 84,600,000 milliseconds in a day at 10,905 tiddlers, the 1% threshold at 1,319, and 0.1% at just 417 – and .1% per day is still far too high a probability for me. This means that this absolutely is unsuitable for a global UUID. Unsurprising. But depending on your tolerance for risk, this is probably fine for a local unique id, especially since id collisions aren’t fatal, just requiring you to increment and check again.

One very, very minor correction:

UUID-v7: 019ce2eb-b2e4-739b-bd29-4f9b77c76b04
//                          ^---- only half-random: can only be 8, 9, a, or b

I have had almost no use for them myself, especially created. Besides this current proposal, how much to do you use it? And for what?

#12

Tiddlymap is an example of a “unique identifier.”

#13

With what I propose there are no risks I can see, I am proposing interviening on creation and import only if created is non blank and clashes with a subset of created dates that have been previously registered. The same check is done on tiddlers added to this registry, I plan batch and ad hoc.

That means only a subset of tiddlers are registered via their creation date and I have prevented incoming duplicates. the same with new tiddler but they will be created in the future of the last registrations and unlikely to overlap.

once I have this I plan to explore the possibilities it enables.

#14

I feel as though that would limit the usefulness. I would expect that at a minimum all user-created tiddlers would have this if we wanted to use it as any sort of id. But if you’re planning a batch mode, it should be easy enough to update a wiki to add that for any missing tiddlers.

One other thing I forgot to mention in my previous post is that the git model might be very appropriate. Commit hashes are UUIDs (I don’t know exactly what format.) But they use a much-shorted version of it for many purposes; it’s unique enough for a given repo to not worry much about collisions. So if your commit id is actually 5e36569402efeafa65562fecadf373433ee4746c, in many places, including URLs, you can just use the first 28 bits (7 characters): 5e36569.

This would be an interesting approach. If we used that UUID v7 format, and used the full format for any sorting, but chose the last n bytes, which are more truly random, as local aliases, we might get a lot of benefit. Above that might be ee4746c.

It’s still probably not something I’m very likely to want. I still prefer urls like http://wiki.example.com/#Horse+Sense, even if that could be shortened to http://wiki.example.com/#ee4746c. But it’s very interesting.

#15

GitHub shows you 7 characters. But the link itself contains the full hash of the commit.

That’s exactly the direction I was going. For me ee4746c is impossible to remember. It’s only seven characters, but I can not care enough to remember it. – I don’t want to.

On the other hand if a triplet of human readable words and it’s appearance in a wiki is random enough, it could be used to find a tiddler with a good certainty of uniqueness. Similar to Tony’s approach.

eg. calm+month for tiddlers created with an experimental plugin would be good to list all tiddlers created within about 1 1/2 days (today and tomorrow). Since uuid7 is a timestamp they can share phrases.

So some duplets could be used for links that filter for a certain time range. – That’s an interesting side effect

A triplet eg: warm elm binds … Will probably be random (16 bits) enough to find the exact tiddler (if it exists) in the wiki, even if it is renamed. We still have the birthday problem here.

Every additional word that you are willing to remember, will give you more uniqueness. eg: jay, cave, brave, sage will be 23 bits that are part of a tiddlers UUID7 phrase representation.

There is some trade of between accuracy and (mental) cost to remember the “address” / “birthday” or however you want to name it.

I am not sure, if this is practical at all, but it’s a nice experiment. – I’ll publish an experimental plugin today. It does not contain an URL catcher that would let you use http://wiki.example.com/#calm+month

Just some thoughts

#16

A post was split to a new topic: Request - Make Timestamp On / Off More Configurable