Data tiddlers are forced into type=application/json

Developers
#1

Folks,

I note that when you use an index to add a key/value to a data tiddler, that the first time you do this it sets the type=application/json on the data tiddler. To me this makes a lot of sense, well at least the first time if the type field is empty, as it is by default.

With further use I have discovered this is actually done every time you add an item via an index, even if the tiddler has a different type, it is simply overridden by application/json.

This to me is too opinionated. If I set the destination tiddler to another type I would like it to remain so, or at least if I could say what type that data tiddler is (would require more changes).

I do not know where this would be occurring perhaps within the actionsetfield widget and perhaps elsewhere when ever the index parameter is available.

  • I would like this process not to be so opinionated and leave the type field alone if it already has a value.
  • This I believe requires a core change, and should be backward compatible?
    • If potentially not backward compatible provide, a way to stop this behaviour when using the index parameter.

Why?

  • Even if I use an index to populate the text field of a tiddler, if that tiddler is of type=text/vnd.tiddlywiki or blank it is wikifield. If such entries contain a link to a tiddler title eg [[this title]] or [[oneword]] title, this title is now listed as a missing tiddler, and all titles in that data tiddler can be listed with {{{ [[data-tiddler-title]links[]] }}}.

What are your thoughts before I raise an issue?

#2

data-tiddlers can have 2 types:

  • application/json and
  • application/x-tiddler-dictionary

The second one allows you to use key : value pairs on a line by line basis.

Both of them are rendered as text/plain in the view template

#3

That is true, I want to change these to text/vnd.tiddlywiki and leave it that way, not resetting the type every time I add/change a record.

  • It is fair on first creation and it will stay that way unless changed. But I want to change it and have a good reason, but the way it is implemented I cant.
#4

That’s not possible. If the tiddler type is wrong, the internal functions do not know, how to process the text-field.

If it is application/json, the core needs to treat the text-field as a JSON and use the browser json stringify function to create the text field.

If it is application/x-tiddler-dictionary it needs to write key:values into the text-field.

Those 2 possibilities are fundamentally different. If you change the type field of a data-tiddler you will cause problems.

Every widget, that has an index parameter relies on the type field, to know how to treat the data-tiddler. The type-field of the data-tiddler is the “source of truth”. So it cannot be user defined.

#5

Just bring the type when adding a data tiddler

#6

@linonetwo could you explain what you are suggesting here in plain english?, or another language I can auto translate, I seriously value your contributions and ideas however your brevity often means I can’t use it. Please treat me as a dummy :nerd_face:

#7

Hi @TW_Tones , being brief often means speaker don’t know much :stuck_out_tongue_winking_eye: I haven’t use action widget to update data tiddler, so I cound’t say much.

I only know it works as expected if I create an empty tiddler with type ‘application/x-tiddler-dictionary’ , and add the data later.(because I read the source code and find

)

And I was using JS to do that.

#8

Keep in mind the reader may not know much as well. There needs to be a minimum amount of information or it’s not very useful.

  • Your last reply helped me understand, thanks.

To be clear, I can create a tiddler and give it what ever type I want, but even when the tiddler exists, and has a type “writing with an index”, it will force it to “application/x-tiddler-dictionary”.

If it has no type and “writing with an index” it will force it to "“application/x-tiddler-dictionary”.

I know the reason this is happening but I do not like it. I would prefer to be able to use index to set a value in a data tiddler without changing the type. Sure I may have to provide the type each time “I set a value in a tiddler”, but I don’t have to change the tiddler type each time.

  • This is not currently possible because of prior design decisions
  • I think this should change.
#9

One reason I am doing this is I want to write [[tiddler titles]] into a data tiddler, yet give it the type text/vnd.tiddlywiki.

If I can do this all titles named in the data tiddler become “missing” or virtual tiddlers. see Virtual tiddlers - #21 by TW_Tones

#10

If you want to store links in data-tiddlers, you may want to have a closer look at my “script-manager” editions and bundles: Wikilabs Home — A home for: Plugins, Editions and Themes!

I use it as my main edition to collect ideas and links to external and internal sources.

#11

Can you give me a little more info about where to look please Mario?

1 Like
#12

There is a detailed intro at google groups: https://groups.google.com/g/tiddlywiki/c/_VQw8yIKWn8/m/vxJU13slBQAJ from 6+ years ago.

I did update the whole thing to TW v5.3.3, so the old links there should not be used anymore. I did create a whole video series about every step and how all the templates have been created.

The main element is a new tiddler icon in the right sidebar, which creates new tiddlers tagged: script

Those tiddlers use their own ViewTemplate, which has a big green button: “Create Links Container”

image
image2222×560 55 KB

This button creates a new data-tiddler, which can be managed using the “script” tiddler.
Once the container is created, elements can be added, edited and removed

image
image2252×777 78.7 KB

That’s basically it.

Script Manager latest: Script Manager Edition — manage information of all types

have fun!
mario

#13

Thanks Mario. I remember this from way back then. I do follow all the videos, and I know it was interesting to me back then.

  • I understand this and think that there should be another way to establish the source of truth when writing to the tiddler. If nothing else allow the write action to say whilst writing this is a “application/x-tiddler-dictionary” or “application/json”.
  • To me it seems we have an “opinionated source of truth”, that the type in the destination tiddler is the source of truth and as the coder there is only one way for me to provide this truth.
    • Proof of this argument is regardless of what I set it to the write action will change it back.
    • In fact the type field field is not the source truth because the write process determines this truth and changes it (insists on it). It is the write process that is the source of truth.
    • It is making an assumption it is or must be a particular type, if I do not give it one it sees to be correct.

I will continue to review the links and videos to try and glean the detail of you example, and I think I partially understand how they relate to what I am after, but I do not yet understand.

However it has given me a lead on what I am after;

The ability to have titles stored in a data tiddler, to also be considered missing tiddlers, if they do not exist.

  • If for example there existed a tv-data-tiddler-type that would be used rather than depend on, or reset the data tiddlers type field. All would be fine.

I have just stumbled upon another issue of using the full time stamp including microseconds writing to a data tiddler, its too fast and writes with the same time key, thus overwrite prior entries in the same microsecond.

  • All I need to do is introduce a microsecond delay between each write.
<$button >
<$list filter="[tag[TableOfContents]]">
<$action-setfield $tiddler="DataTiddler" $index=<<now YYYY0MM0DD0hh0mm0ss0XXX>> $value=<<currentTiddler>>/>
</$list>
Add to data
</$button>

<$list filter="[tag[TableOfContents]]">

</$list>
#14

That’s a possibility. We would need to change the core code, to be implemented that way.

But there are several problems, that will increase the complexity quite a bit.

  1. Let’s assume a user starts to write to a vnd.tiddlywiki tiddler with a write-index command using the x-tiddler-dictionary format.

  2. Now the user creates a new write-index action using json format.

The no. 2. write action now has a problem: It does not know about the previous format, because type says: vnd.tiddlywiki.

So what should happen?

Assuming, that vnd.tiddlywiki is wrong, is the wrong assumption, because test: value is a valid vnd.tiddlywiki and a valid x-tiddler-dictionary format. So with this information alone, the no. 2. write action cannot decide, what to do.

What we definitely know is: Loosing data is a no go

  • So we will need to create a “write-index” action, that has a new parameter for the read-type: x-tiddler-dictionary to read the existing data
  • And we need a write-type: json to write the new data.

That’s cool. right? - No

Those parameters will break, if I hit the no. 2. actions 2 times in a row. –

The second activation says: read-type: x-tiddler-dictionary and write-type: json – Which is wrong now, since it change to JSON with the first activation.

There probably is a reason, why core assumptions have to be consistent.

That’s right. That’s how it is coded at the moment. You can raise an issue at GitHub.

But you have to provide a consistent workflow to solve the problem described above.

Especially if the user wants to use both “write-index” actions. No. 1 and no. 2. have to work and there has to be documentation in a way, that users can understand and use it.

I did not even talk about export, import a vnd.tiddlywiki tiddler, that contains json or x-tiddler-dictionary format, without any information about it.

just some thoughts.
-m

#15

If the type of a tiddler impacts how TiddlyWiki core handles a tiddler, then I’m thinking we ought not use the type field in a way that might trip up the core.

It would be better to have your own custom field to do your own custom things.

Essentially, I’m saying “go ahead, support your needs by customizing your TiddlyWiki as you need (via the standard methods of doing so). On the flip side of that coin, I’m saying " please, do not foist this high-risk/low-reward(?) request on the core such that it impacts all users’ TiddlyWiki instances going forward, whatever the impact(s) may be.”

#16

Hi @TW_Tones the process of updating a data tiddler needs to know the correct MIME type for the tiddler. If you wanted tiddlers with a different MIME type to act as though they were data tiddlers then we’d need some way to tell the core that that type should be treated as either “application/json” or “application/x-tiddler-dictionary”.

The “type” field is used for a specific purpose in TiddlyWiki, and drives many mechanisms. I don’t see the case for more complexity in this area.

#17

Of course. I can work through an example, but I can already see how to do it.

I would argue there is a fault here, because if I want to set the type field for a specific purpose, in this case to recognise missing tiddlers, I can not, It will cause writing to it via an index, will reset it.

  • The solution is complex.

I can see how this could be fixed, actually fixing my problem and reducing complexity, by providing appropriate flexibility.

I understand it was a design decision some time ago, I am simply suggesting a better one.

#18

I think that would be a regrettable mistake.

Better for you to create a custom field to handle your purpose. A custom field would also make it possible for you to report on a tiddler that has a value in your custom field that used to match the “type” field, but suddenly no longer matches.

1 Like
#19

I may do that @Charlie_Veniot however I can already see a simple solution which is similar to a pattern already implemented elsewhere, allow a type parameter or variable, when using index. And stop the forced change of the tiddlers type field if its already set when given the type.

  • from my perspective its simply a better design approach and should be backward compatible.

Edited

Actualy I think a fix to them above is a good idea, however if I return to the root cause. The desire to have tiddler titles in a data tiddler, to be recognized as links or missing tiddlers may have other solutions.

  • I may design my own kind of data tiddler that does this.
#20

When the internals of a product make use of a particular field for however many mechanisms, I think it is a risky proposition to start using that field in “unorthodox”(?) ways.

But hey, no guts no glory.

I don’t know why I chimed in. Kind of strange that I bothered, since I’m increasingly having less interest in budging away from 5.2.3. Huh. Fascinating…