URL encoding using a `+` instead of `%20` for a space

Developers
#41

@Scott_Sauyet to cut a long story short, if you are prepared to use the underscore instead, I would be happier, it is to do with encodeURI and encodeURIComponent standards. The plus + is not inside published standards, even in the fragment, although it will work in many cases.

  • Ideally this would be an optin or optout because I can forsee cases where the existing permalink encoding method is needed.

    • The following characters need not be encoded both with encodeURI and encodeURIComponent A–Z a–z 0–9 - _ . ! ~ * ' ( ) note the _ underscore is among them, + is not.
    • The following additional characters also valid when encoding with encodeURI ; / ? : @ & = + $ , # the + symbol and others are in this subset. So if we modify your proposed change to “not encode these additional characters” the permalinks will look even easier to read.
      • However I have read that not all systems may consider this a valid URI even if only used in the fragment. Thus I believe we need to allow people to opt out or in.

  • I have described this issue back in this thread but happy to give more details if you request so.

I have an alternative way to patch this to achieve the above rather than using a replace with + I can show you, we just need to add an opting/out process.

#42

The latest commit switches to underscores. I agree that it makes for a much more readable URL. You can see my comment on it on GitHub.

This is not correct. The relevant parts of the specification are these:

   fragment      = *( pchar / "/" / "?" )
   pchar         = unreserved / pct-encoded / sub-delims / ":" / "@"
   pct-encoded   = "%" HEXDIG HEXDIG
   unreserved    = ALPHA / DIGIT / "-" / "." / "_" / "~"
   sub-delims    = "!" / "$" / "&" / "'" / "(" / ")"
                 / "*" / "+" / "," / ";" / "="

with these in a different spec included by reference:

    ALPHA        =  %x41-5A / %x61-7A   ; A-Z / a-z
    DIGIT        =  %x30-39  ; 0-9
    HEXDIG       =  DIGIT / "A" / "B" / "C" / "D" / "E" / "F"

This means that the characters allowed in a fragment, besides alphanumeric and percent-encoded ones are these:

- . _ ~ ! $ & ' ( ) * + , ; = : @ / ?

So this is a legitimate URL:

https://tiddlywiki.com/#A_(surprising?)_tiddler_with_various_*special/unusual*_characters,_&_more!

Note that encodeURI and encodeURIComponent are not normative as to what forms a legitimate URL. Instead, they are tools to turn URL-like strings into guaranteed-legitimate URLs. There are plenty of legitimate URLs they could never produce. That doesn’t mean those URLs are not legitimate.

That’s in the latest update.

That’s entirely possible, and we might have to see how that could play out. Do you have any references?

How do you envision that working? Multiple permalink/permaview buttons? A control panel setting? Something else?

This thread has gotten long. I would appreciate a refresher or a pointer to the specifics you’ve already posted.

Thank you as always for your feedback!

#43

In the GitHub discussion I address at least a part of this, the fact that many programs consider sentence-ending punctuation like ., ?, and ! at the end of a url not to be part of the link, but rather part of the surrounding text.

You can test it in its current state at https://tiddlywiki5-2vg2v0leq-jermolene.vercel.app/

1 Like
#44

@Scott_Sauyet for a totally unrelated tool I wanted to get a url to a tiddler, constructed. I could replace spaces with _ but is there another way to get the “permalink format” as we do not have a permalink operator?

  • Perhaps a compatible operator or additional format:permalink[] would be helpful?
#45

If there’s support for this proposal, it would be easy enough to refactor what I’ve written, offering this formatting for both purposes.

It’s not clear, though, if there is any support. This discussion has been mostly you and me, with some technical details from @pmario, some interesting interjections from @oeyoews, and a somewhat mixed-signals message from @jeremyruston. The only interactions on the actual pull request have been a few well-appreciated corrections and suggestions from @pmario. I’m not sure whether to take this as discouragement or only a sign that things may move slowly around here.

However, if there is no support for this proposal, it would be easy enough to extract the relevant code to make a permalink formatter of some sort. It could be stand-alone or submitted for inclusion in the core. The trouble would be that those permalink, at least as I’ve been doing them, would not actually be useful if they weren’t being decoded when loading.

#46

Actually in this case it would be simple enough to make a plugin, minimise the number of core tiddlers you replace, such as writing an alternative permalink button that does as desired. You add functionality rather than modify it.

  • If it proves to be useful it may then become a core addition.
  • This would suit me as I have a hunch, no “educated guess”, that this being a default may break something.
  • Yet at the same time it would be nice to not encode the other permitted characters in the fragment as well.

The only thing I don’t know is how when such a link is followed, how it opens the required tiddler.

  • I would like to understand this.

Although not withstanding the above, I do have a custom permalink button that creates a permalink tiddler, allowing the original to change title and the previously published permalink still “work”. It also allows you to see which permalinks you generated and thus published.

  • In this case there is no need to redirect underscore containing tiddlers.
  • This works more like sophisticated link managers such as yoast in WordPress, it helps maintain your “SEO juice”.

In a related matter I am building a way to generate links to other wikis so they are opened in named window/tabs so you can maintain one tab per wiki even after following multiple links.

#47

Therein lies the rub..

The whole idea of permalinks are that they lead you to the desired resource. So encoding must be paired with decoding.

That’s why this has to be a core change. The code in question is buried in the be middle of one of the most central tiddlers, boot.js.

That sounds fascinating. I have questions, but I’m typing on my phone on a bouncing bus, so I’ll save them for later.

#48
  • Not with my approach
  • will have a look. now thanks

The idea is rather than a permalink, drag or copy a html a tag, containing the link, pretty link, the target window name set, and maybe more information (eg browser brand). Primarily we put such links into a tiddlywiki as a link to an external resource.

  • You can also build a method to do the same thing from a href in any wiki, but it requires more assumptions than getting it from the source wiki.
#49

You can see my changes to that in the pull request.

Is this related to the earlier discussion about URL shorteners? That’s something I thought I wanted, but eventually realized will not satisfy my use-cases, where there are many read-only users and only a small number of editors for a wiki. It certainly has many uses, though.

Or does it have to do with what you suggested earlier, about creating a new tiddler with a simpler permalink URL that transcludes the target tiddler? While that doesn’t appeal much to me, I still look forward to seeing how it goes.

Or is it something different altogether?

I’m not quite following, although I look forward to seeing what you come up with.

From your earlier post, I thought you were spawning new windows/tabs from one wiki to another and then intercommunicating among them. I haven’t done much multi-window/multi-tab work in years, but I remember the great number of complexities involved, and I was wondering how you were handling them. It sounds as though this is somewhat different.

#50
  • Thanks
  • Indirectly because it gives an option to add this feature.
  • Keep in mind tiddlywiki already uses many additional tiddlers with tiddlywiki.com is using 4,000+ and you only need additional ones for the external links and/or short links you publish. The way tiddlywiki works they get loaded into memory effectively caching them.
  • It is also easy to keep them out of searches etc, so you would not know they are there.
  • Yes, but I think you will be surprised, they can look exactly like the tiddler they are masquerading as, most will not know the difference, and they do not appear in searches.

I have most of this 90%+ completed so will share next week sometime at the latest.

  • Thats a different project, focused on proving a window manager for tiddlers opened in a new window. I have what I need to proceed, just not the time.
  • I have considered interwiki communications for all in the file: “domain” but not further yet.
#51

The previous PR for this was closed as a backward compatibility question was raised just before 5.2.0 was released. There is now a new version on GitHub. This version returns to using + characters for space, as initially discussed. The _ character turned out to cause problems.

1 Like