How do I set up an action widget to add to a nested JSON tiddler WITHOUT overwriting it entirely?

[Greyed]

I can write simple index/value pairs into a JSON tid easily, but properly adding to nested JSON has become the bane of my sanity!

Given an example JSON structure like this in a tiddler named “SomeTid”:

{
    "sale1": {
        "discount": "50",
        "price": "10.50"
    },
    "sale2": {
        "discount": "60",
        "price": "8.00"
    }
}

I managed to figure out how to add a new entry of “sale3: discount=60, price=5.99” with this action widget:

<$action-setfield $tiddler="SomeTid" $field="text" $value={{{ [[SomeTid]get[text]jsonextract[]jsonset:object[sale3]jsonset[sale3],[discount],[60]jsonset[sale3],[price],[5.99]] }}} />

…but this ugly ‘sledgehammer’ method forcibly overwrites the whole text field of the JSON tid – which in clumsy hands (like mine!) has very high potential to cause the complete loss of its data. (It also squishes the human-readable “pretty” formatting into a single, compact line that’s terrible to read or hand-correct.)

With simple, non-nested JSON, I can use the $index mode of $action-setfield or $action-listops and it safely adds the new index and value, leaving the existing ones alone – and it also doesn’t “break” the tid’s appearance.

The “Modifying JSON tiddlers” documentation only says to use either of those widgets to modify JSON, but I can’t figure out how to make them add nested JSON without a full-overwrite? I’m not sure what the $value (or $filter and/or $subfilter) need to be to make it work. Everything I’ve tried so far has resulted in the JSON being escaped in either the index or value when it does anything at all. For example, this:

<$action-listops $tiddler="SomeTid" $index="sale3" $filter="[[]jsonset[discount],[60]]" />

…correctly inserts a new “sale3” index like I want, but the value it writes is "{\"discount\":\"60\"}" with the JSON-escaping, which is no good.

Can anyone pretty please lend me a hand to get this working, or tell me if it’s just not possible? I know I’m probably missing something painfully obvious, but I’ve searched the TW docs up and down for weeks now trying to figure out what it, and I think I’ve officially fried my poor noggin at this point.

[Scott_Sauyet]

@Greyed

…but this ugly ‘sledgehammer’ method forcibly overwrites the whole text field of the JSON tid – which in clumsy hands (like mine!) has very high potential to cause the complete loss of its data.

It’s easy to forget that JSON is a string format. While it is meant to carry a payload of a simple object, array, or primitive value, it is not those values; it is a string that can be deserialized into those values. So anything that is going to update the payload in that nested value will necessarily create a new string.

I don’t know the TW JSON tools well enough to offer alternate ways to do what you do here, but any tools that do exist will still have to replace the string value.

@Greyed

It also squishes the human-readable “pretty” formatting into a single, compact line that’s terrible to read or hand-correct.

That is solvable:

<$action-setfield $tiddler="SomeTid" $field="text" $value={{{ 
    [[SomeTid]get[text]jsonextract[]] 
    +[jsonset:object[sale3]]
    +[jsonset[sale3],[discount],[60]]
    +[jsonset[sale3],[price],[5.99]]
}}} />

AFAICT, there is no way to get the result back into a prettified format using wikitext. Even if we could parse it into a JS object, the stringify operator doesn’t seem to offer an indent parameter.

Edit:

Found it, it’s part of the format Operator

<$action-setfield $tiddler="SomeTid" $field="text" $value={{{ 
    [[SomeTid]get[text]jsonextract[]] 
    +[jsonset:object[sale3]]
    +[jsonset[sale3],[discount],[60]]
    +[jsonset[sale3],[price],[5.99]]
    +[format:json[4]]
}}} />

[jeremyruston]

Hi @Greyed

@Greyed

It also squishes the human-readable “pretty” formatting into a single, compact line that’s terrible to read or hand-correct

That’s a very good point. Perhaps there is a case for the JSON operators to always return formatted JSON. There would perhaps be a modest performance impact, but it would improve readability and debuggability. Alternatively, we could re just add a suffix to the jsonextract operator to request formatted JSON: jsonextract:formatted[foo],[bar].

[Greyed]

@Scott_Sauyet

It’s easy to forget that JSON is a string format. While it is meant to carry a payload of a simple object, array, or primitive value, it is not those values; it is a string that can be deserialized into those values. So anything that is going to update the payload in that nested value will necessarily create a new string.

Mm, I’m new to using JSON and don’t have any formal compsci education, so I’m still working on properly cementing the concepts into my mind. Gotta get this one set in post-haste.

I do still find it odd that when using a simple JSON structure instead of a nested one, I can write just the single index/value pair with no risk to the remainder of the source tiddler? Half of my testing with nesting values left me with completely deleted source text when I just wanted to append a new item…if I hadn’t been using a temptid generated by a button click as my test data, I could have wiped out a lot of hard work by accident and would need to restore a backup.

So that’s why I’m sure I must be missing something; surely there must be some way to append/insert/edit nested data that doesn’t have a potential side effect of becoming Godzilla and rampaging about destroying Tokyo – er, the rest of the tiddler! Perhaps one of our wonderful TW gurus has an idea.

@Scott_Sauyet

Found it, it’s part of the format Operator

Arrrrgh, I can’t believe I forgot about the format operator, thank you for the reminder! I knew I’d seen something with a way to specify JSON space-indent level but I couldn’t for the life of me remember what it was. The workaround method is helpful as well!

@jeremyruston

Perhaps there is a case for the JSON operators to always return formatted JSON.

That sounds great as a default for the average use case if there’s also an option to leave its output compacted, like when working with a very large JSON tiddler.

For example, a pet project I’m trying out imports a 234k+ line JSON file which I have trimmed down from an even larger API export. Without any indentation, this trimmed file is 2.5 MB, but with 4-space indents it balloons to 6.8 MB due to a lot of nesting. Using the compact format is very helpful there, though I assume a situation like this is an edge case. (Even I plan to keep that tid read-only!)