Are stringify and jsonstringify operators duplicates?

Developers
#1

Does anyone know whether there is any difference in the functionality of the stringify and jsonstringify filter operators?

I’ve been working the docs for these filters in this draft PR: Improve jsonstringify and stringify operators docs by mateuszwilczek · Pull Request #7650 · Jermolene/TiddlyWiki5 (github.com).

Currently these operators have identical documentation and examples (permalink). The only difference is in the table with filter operators: stringify applies JS formatting, jsonstringify applies JSON formatting (but isn’t that the same thing?). Please mind the typos in the docs: there are redundant backslashes \ before some escaped characters, already taken care of in my PR.

I don’t have much understanding of TW’s JS internals, but from what I have explored, both operators, or associated JS functions, seem to be defined identically. stringify was introduced in v5.1.14, jsonstringify in v5.1.15, the rawunicode suffix was added to both of them in v5.1.23. Also, jsonstringify function seems to be defined only to be used through the filter operator, whereas the older stringify function is used internally in a couple of places in the core. It seems to me that jsonstringify is just an unnecessary duplication of stringify.

Does anyone know if there is any difference between these operators? Or can anyone confirm, that this is in fact an overlook, and jsonstringify is unnecessary?

Edit: typos, added links do docs.

#2

I’m bumping the topic in hope that someone knowing the origins of jsonstringify will see it.

The points listed in OP lead me to think, that jsonstringify is an overlooked duplication of stringify. I will update the documentation accordingly.

As far as I understand the core code I looked through, it should be relatively easy to remove jsonstringify filter and JS functions. Any other components depending on this functionality use stringify anyway.
However, this could create backwards incompatibility if many plugins are using jsonstringify instead of stringify.

#3

Hi @vilc apologie for the late reply. I did look at this in detail a few weeks ago, but could not satisfactorily explain the two different implementations. The docs can perhaps say that the jsonstringify operator is deprecated. I don’t think we should change the implementation.

#4

Just reviewing the documentation there is a different encoding for Characters from 0x00 to 0x1f in jsonstringify its \\u#### and stringinfy is \\x## but since V5.1.23 becomes the \u#### encoding. Which appears to make them the same?

However jsonstringify is the older and other plugin designers may have used this, so it needs to remain for backwards compatibility,. as is possibly the case for stringify now.

#5

I can see in the source code that they are not identical. A quick test demonstrates:

====================================================
raw:           "foobar"
stringify:     "foobar"
jsonstringify: "foobar"
====================================================
raw:           "獅子、老虎和熊，天哪"
stringify:     "\u7345\u5B50\u3001\u8001\u864E\u548C\u718A\uFF0C\u5929\u54EA"
jsonstringify: "\u7345\u5B50\u3001\u8001\u864E\u548C\u718A\uFF0C\u5929\u54EA"
====================================================
raw:           "string	with	tabs"
stringify:     "string\x09with\x09tabs"
jsonstringify: "string\twith\ttabs"
====================================================
raw:           "string with a    backspace "
stringify:     "string with a   \x08 backspace "
jsonstringify: "string with a   \b backspace "
====================================================
raw:           "string with a    formfeed "
stringify:     "string with a   \x0C formfeed "
jsonstringify: "string with a   \f formfeed "
====================================================

I have not investigated their differnet use-cases.

#6

Thanks @TW_Tones and @Scott_Sauyet for noticing the details. I will look into it again and try to write the docs so that the similarities and differences of the operators are more apparent.

I would welcome any hints as to the actual use cases of these differences. I guess the jsonstringify should now be considered the default if in doubt, as TW operates on JSON encoded strings in many situations; am I right?

#7

I can confirm that (stringify changing 0x00 - 0x1f characters to \x## format, jsonstringify changing to \t and so on or \u####), at leas with newline and tab characters, I seem to be unable to enter any other control characters using TW in browser only.

This information was actually partially there in the docs (before I started to mess with it), but intertwined with some inaccurate info.

I have discovered some more quirks, which I think deserve explanation in the docs as well:

  • stringify changes tab to \x09, but newline to \n. I wasn’t able to test it with other control characters.
  • stringify escapes single quote as \', whereas jsonstringify does not. Makes sense I guess, since JS allows quoting strings with both ' and " (JSON allows only ").
  • none of them escapes the forward slash /. If I understand correctly it is not obligatory to escape it, but allowed and sometimes handy in JSON.
#8

Thank you @vilc I believe you’ve identified the crucial difference between the operators. The consequence is that using jsonstringify is marginally more efficient for encoding JSON strings.

#9

I’ve made a new docs PR: Improve jsonstringify and stringify operators docs: part 2 by mateuszwilczek · Pull Request #7748 · Jermolene/TiddlyWiki5 (github.com)

@TW_Tones and @Scott_Sauyet if you find time, please take a look at the PR or its Vercel preview and let know if the new documentation of those operators is consistent with your observations and generally makes sense.

#10

It makes sense to me, it’s easily understood, and from my inexperienced TW eye, looks like a well-written PR. I did raise a question, but only because it was sparked in reading this.

No critique here, just a question for the core team. How long do we keep these “New in 5.1.23” annotations around? There’s been the entire 5.2.x train as well as 5.3.0 and 5.3.1, and this would likely appear in 5.3.2. Is there some official shelf-life?

Looks good to me.

#11

We should always keep the “new in 5.x.y” annotations in the documentation. This is because not everyone upgrades their TiddlyWiki files, so they may be using a very much older revision and will still need to know if a particular widget parameter or macro argument can be used with that specific version of the TWCore.

1 Like
#12

I’ve seen "Since 5.1.23" style annotations often enough. “New” feels like it should have a limited life-span. Not a big deal, but it does seem surprising to me to continue to see “new” nearly two years after its release.

#13

In <<.fromversion "5.3.2">> we could change the annotation text and styling depending on the version number. We could highlight the most recent release with a bright “NEW IN v5.3.2” badge, but mark older versions with less eye catching “introduced in v5.2.3” text.

5 Likes
#14

Or if we wanted to get fancy, keep the “new” around for n versions (perhaps 3?), and the actual version number in the badge could be a link to the relevant release notes.

I’m curious that there seems to be very little effort to expose older versions, things like https://tiddlywiki.com/archive/full/TiddlyWiki-5.1.23.html. I’m assuming this is intentional, but I don’t know why. If someone is still using 5.1.23, for whatever reason, I would think that it would be easier to use the documentation from that version. I only discovered it by looking at the docs github repo. Are there good reasons for this?

2 Likes
#15

Linking to the release notes from the badge is very clever!
I’m not sure about having a fixed amount of versions of what counts as new, it might be better to adjust it manually, especially after bigger releases like v5.3.0. But relying on a fixed amount of releases back requires less effort in the future of course, even if not always optimal.

As to the “snapshots” of old docs, they would be obviously stuck at an older version of the docs. So they would not contain any fixes and improvements to the things that the older versions included. For example the operators from OP - their documentation was incomplete and partially wrong until now.

This would be useful if docs were complete for the previous version at the release of the next one, but it’s a lot of work until we get there.

In the meantime, the solution proposed above, with two different styles of badges for new and not-so-new features sounds good and easy to implement to me.

#16

Right. I wasn’t thinking of using it for this case. It was just a tangential question. I remembered having seen them at some point looking at the source code. It took a little doing to find them again. I was just wondering why. I think it would be great if they were easier to locate. Even better would be if they each had a semi-prominent notice that they are out-of-date, with a link to the main site.

Just a thought.