What's best practice for handling complex meta-data?

#1

I currently store some meta data about various tiddlers in separate “meta data tiddlers” by using the tiddler titles as field names. Here are two such meta-data tiddlers with metadata about three other tiddlers:

title: Abbreviations meta data tiddler
HelloThere: Hell
GettingStarted: GetSta
Community: Com

title: Title-letter-count meta data tiddler
HelloThere: 10
GettingStarted: 14
Community: 8

But it would be better if all meta-data could be stored in a single meta-data tiddler. What would be a good solution for this?

In reality, each meta tiddler can have arbitrary numbers of “entries” (the above both have 3) and an entry can have an empty value. Entries (i.e tiddler titles) are actively added/deleted and their values must be easy to modify.

Spontaneously, I’m feel skeptical about the following two variants (but maybe you have a different experience?)

title: my single meta-data tiddler
HelloThere: Hell 10
GettingStarted: GetSta 14
Community: Comm 8

(Problem: If, say, “GetSta” is missing then “14” can be mistaken to be the abbreviation.)

title: my single meta-data tiddler
HelloThere---abbreviation: Hell
HelloThere---lettercount: 10
GettingStarted---abbreviation: GetSta
GettingStarted---lettercount: 14
Community---abbreviation: Comm
Community---lettercount: 8

(Problem: Lots of string fiddling to add suffixes or split them off, and concatenation macros)

I’m guessing the “proper” way to treat this is to use a JSON tiddler - right? As implied, the meta-data is only “one level deep” (i.e the entries have various values, but the values themselves don’t have further values).

…but I need to be able to easily add, edit, and delete entries and property values. For example, can I edit a value directly in an EditTextWidget? Something like:

<$edit-text tiddler=MyMeta jsonindex=HelloThere /> (…except there exists no jsonindex property for this widget)


What would you say is best (or at least good) practice for this situation?

3 Likes
#2

I’m perplexed, at least with the kind of metadata you offer here, as to why one wouldn’t just store the metadata in the tiddlers themselves… Less risk of having obsolete references (from deleted/renamed tiddlers, etc…? Generate the abbreviation at the same time as you generate the tiddler, etc…?

What do you do with this metadata that makes it attractive to keep the metadata separate?

-Springer

#3

The actual use case is a much more complex plugin I’m working on - and the examples above are just dummy data. (I hope to present it within a few days.) The reason to store the metadata in a separate tiddler is because it is set by the user, leaving all other tiddlers un-modified. If the metadata is stored in a single tiddler, it can be easily transferred between wikis. The nature of the plugin in such that you might want to do this.

2 Likes
#4

@twMat I understand exactly where you are coming from on this matter, and have already given it some thought. There seems to be to be two approaches;

  1. The development of bespoke tiddlers for metadata for a particular solution
  2. The development of a defacto approach that multiple solutions could share.

However there is from what I can see two different types of metadata, which most likely need a different approach.

  1. one is something with a one to one relationship with the tiddler for which we are storing this metadata.
  2. The other is a one to many relationship such as say multiple activity/log data as metadata for the subject tiddler.

I would think we could start with;
The development of a defacto approach that multiple solutions could share, with a one to one relationship with the tiddler for which we are storing this metadata.

  • The seperate metadata tiddler for each regular tiddler can have a known namespace
  • We can make an "opt in/out export or drag and drop facility that will include the meta data tiddlers if they exist.

I will keep my eyes open

I am also happy to continue discussing the OT " What’s best practice for handling complex meta-data?"

#5

Yeah, thanks. I’d appreciate if this not becoming a general discussion before the OP is resolved since I’m stuck on this problem.

I think it will be a useful plugin. If nothing else, it is a PoC for a concept targeting a factual problem in TW.

#6

Do tell me if the following seems on track to you @twMat

Of note, having a seperate metadata tiddler includes;

  • metadata for shadow tiddlers without overwriting them
  • allowing the same tiddler with different metadata in a different wikis
  • not changing the subject tiddler eg modified date, when changing metadata

If we can “fit all one to one tiddler metadata in one tiddler” we can have a standard “synthetic title” in which we place most metadata. The name “shadow” is taken so my preference is a “meta tiddler”.

  • That is each tiddler has a meta tiddler of the form $:/meta/tiddlername in which we can store additional metadata relating to tiddlername.
  • we can have a global macro equivalent to the qualify macro called meta-tiddler
    \define meta-tiddler() $:/meta/$(currentTiddler)$
    • There may be an issue here with having to wikify this, so perhaps a javascript macro is needed.
  • Relink can be configured to handle the rename of tiddlername and its matching meta tiddler.

Metadata in the meta tiddler
With a reliable one to one relationship between a tiddlername and its meta tiddler we can reliably store metadata in the meta tiddler (not qualified state tiddlers), for example it could contain;

  • state-statename fields
  • temp-tempname fields
  • notes

What kind of metadata are we likely to store in meta tiddler?

  • fields with a specific name
  • list fields containing one or more values/titles
  • fields based on a tiddler title with a value

I do think “one to many” metadata belongs in other tiddlers, perhaps with a key or prefix of the current tiddler.

  • Keep in mind with large one to many metadata we may instead use a data tiddler with shared or unique to the current tiddler. However we could have a convention that stores information about such metadata tiddlers in the meta tiddler.

I hope I am making sense.

#7

I prefer the second solution. This may take a little space on storage, but it is very clear and easy to maintain and follow. I do not think you need a separator like ---- instead use a simple period .

title: my single meta-data tiddler

HelloThere.abr: Hell
HelloThere.lnt: 10
GettingStarted.abr: GetSta
GettingStarted.lnt: 14

.
.

Here abr stands for abbreviation and lnt for length.

#8

Tones, that is an appealing general approach but, as I really try to make clear in the OP, I want a single meta tiddler to hold the all meta data for all the other tiddlers. I am creatinga a plugin and a main reason for the ideally single meta-data tiddler is so users can easily transfer this meta-data between wikis. Such a transfer can hypothetically be achieved even if there are multiple meta-data tiddlers but for my particular case I find it more suitable and neat if it was a single one.

#9

Yeah, I’ve used it before and I’ve seen others use it but I’m still wondering if it can be considered “best practice” or if a JSON solution would be the “appropriate” way to do it… but with the additional demands I have that it should be easy to modify the data therein (e.g via EditTextWidgets).

#10
  • Happy to help if I can, but at the moment “best practice” is in my mind what I related above. I am sure we can create an easy package to transfer all metadata.

Never the less some ideas that may help;

  • Perhaps storing `“key”=“value” pairs in fields? tiddlernamefield=“fieldname/value fieldname/value fieldname/value”
  • If not then perhaps the best approach a JSON Data Tiddler with a key of tiddlername and multiple fields? Again in a key/value arrangement.
    • Why? because typically a fieldname tends to store not more than a title or list of titles
  • You could try and stuff fields with JSON arrays?

I too am interested in using tiddler names as fieldnames in particular cases especially relationships but I shied away from this because I could see us “saturating the fieldname space”, the “new field drop down” would be full of tiddlernames.

I thought the JSON mangler plugin and others in the past allowed for nested and structured JSON data?

#11

My friend, your heart is in the right place - but my problem is not that I can’t solve this. I can. I’m explicitly asking what is “best practice” which should clearly imply that I’m asking people who know, or at least have a strong arguments for why one option is better. With that said, it may well be that the description in my OP is insufficient to answer the question.

#12

@twMat I think you may be over-thinking this.

TiddlyWiki (as you well know) is a JSON store. Why not just use tiddlers, suitably name-spaced, and add your data as titles+fields? Magically, they can all be surfaced and managed as you would any other tiddler, with all the CRUD power(s) that TW brings to you…

Just think of the namespace as “the thing you think you need to create”.

Sorry if I completely missed the point.

#13

Thanks @CodaCoder - it just seems individual tiddlers for each item that has metadata will mean a lot of extra kilobytes. Each tiddler brings an overhead of… not sure how much, but it adds up.

Well, well. I’ll release my thing within a few days and maybe I’ll get some feedback when people see it in action.

#14

And I would argue “I do know something and put it already”, according to the OT. “What’s best practice for handling complex meta-data?” and in fact I feel I gave a “well informed view” however you have qualified the OT by saying;

I want a single meta tiddler to hold the all meta data for all the other tiddlers .

  • The question now becomes is this best practice or not?

I will stand aside and see what others suggest, but please don’t underestimate my contribution, it is based on professional knowledge.

#15

Thanks. I’ll pause pursuing the question further. My current implementation uses the structure that is outlined in the OPs first paragraph, and which I’m not satisfied with, but with all the references to those tiddlers in my code I’ll just stick with it for now. After it is published, I’ll see if people come with better ideas (where your concepts just may be the optimal solutions).