URL encoding using a `+` instead of `%20` for a space

Developers
#1

Would there be any interest from the community – and any support from the core team – for changing the URL fragment encoding to allow + instead of %20 to encode a space?

It’s simply a matter of prettier URLs, but to me that is likely enough of a reason. I would rather look at this:

https://tiddlywiki.com/#The+First+Rule+of+Using+TiddlyWiki

than this:

https://tiddlywiki.com/#The%20First%20Rule%20of%20Using%20TiddlyWiki

Technically, this is slightly problematic, because I’m pretty sure that according to spec, this is only valid in the query string, not the fragment identifier. But since TW is already hijacking the usual behavior of the fragment, and since these fragments will not be interpreted by anything other than TW, I don’t see it as a serious issue.

Of course other special characters (including the + sign itself) would still be %-encoded, but it would prettify a large number of URLs. And of course any existing %20-encoded URLs should still work.

I would expect this to be easy to code, although finding all the spots where such changes would need to be applied might possibly be trickier.

So, are there others who would appreciate this? Are there technical reasons it cannot or should not be done? Or is this just a ridiculous idea, as TW would prefer to stick to web standards even when they might not be very relevant?

Addendum: I’ve used a number of sites that do this, either in the fragment part or the path part, although at the moment the only one that comes to mind is Atlassian’s Confluence.

Doubts regarding framing of tiddler titles without spaces
#2

@Scott_Sauyet I have looked into the complexities around links and encodings and I have found it hard to find all the possibilities vs the encoding standards.

However In fact we can often use links with spaces in them in Chrome/Firefox and edge without encoding as long as the whole address is delimited to know where it starts and ends.

For example [[https://tiddlywiki.com/#The First Rule of Using TiddlyWiki]] not only reads well, works (just did FireFox), and displays well in the browser address bar.

  • Just recently I made a link that uses a url and adds to the end plus “+ delimited” words from the tiddler title to search google. In effect it replaces spaces with the required + anyway, so no need to encode.
#3

There is a fair bit to the standards. But did you find that it would be particularly difficult in TW? I haven’t looked yet.

Yes, I do the same. I guess I didn’t say so, but my point was mostly regarding where TW generates local urls to export. Permalink and Permaview are the only ones I know of, but there may be others. Here, I would love the URL that is auto-copied to the user’s clipboard to have the +-encoding. We couldn’t use literal spaces, because, as you note, often enough the usage is not delimited.

It’s not so pretty in Brave. It works just fine, but it does so by URL-encoding, and all those spaces become %20.

Cool. I’d love to see it in action. But “so no need to encode” is incorrect: you are encoding this yourself! You’re just using the encoding similar to what I’m suggesting rather than the canonical URI encoding. Watch out, or we’ll make a script kiddie out of you yet! :wink:

#4

That’s right.

It is a serious issue, since eg: “a+b” is a valid tiddler title. But in your request it would be needed to replace “+” with a “space” which would break existing links to a+b.

So in terms of backwards compatibility there is a very low chance, that the OP will be implemented.

1 Like
#5

No, I thought it difficult understanding what the universe expects, as you point out brave is displaying %20 and Firefox/edge and chrome are not. This is all before looking at tiddlywiki, which has comprehensive support.

Without meaning to be facetious perhaps stop using Brave, or accepts its limitations?

I renamed the tiddler “The First Rule of Using TiddlyWiki” to “The+First+Rule+of+Using+TiddlyWiki” and the permalink encodes %2B.
See here an old prelease on tiddlyhost

Any url you construct needs to obey the laws of the URI/URL rules and even then browsers will display it differently. Firefox does not decode %2B back to + but it does %20 to space.

Can you state your desired outcome separate from any details about a way to achieve it?

Is it?

Have a look here TiddlyWiki Pre-release — 5.3.0-prerelease uploaded 27/05/2023

  • Where I do use + and . named tiddlers
  • I also use + to format a google query from the words in the tiddler title
<a title=<<currentTiddler>> href={{{ [[https://google.com/search?q=]] [<currentTiddler>] +[join[+]] }}}>Search Title</a>

<a title=<<currentTiddler>> href={{{ [[https://google.com/search?q=]] [[site:tiddlywiki.com]] [<currentTiddler>] +[join[+]] }}}>Search Title</a>

I have some solutions for prettier URLs I can give more details for;

  • Choose a tiddler title that is URL friendly in th first place that does not encode poorly https://prerelease-play.tiddlyhost.com/#The.First.Rule.of.Using.TiddlyWiki
  • If you are unhappy with the tiddler looking like The.First.Rule.of.Using.TiddlyWiki in tiddlywiki I have plenty of methods around that;
    • An extra tiddler with an alternate title that transcludes the original, give two names to link to. My tiddler masquerade tool.
    • Use the view template title cascade to alter the way the titile is displayed
      • Use caption
      • use my alt-title which displayes an alternate title while keeping the real title
      • Use my tiddler masquerade tool.
  • We could modify the permalink to both generate a nicer link and a tiddler masquarding the original.
#6

This proposal wouldn’t change how that’s encoded, namely as "a%2Bb". So I don’t think that’s an issue.

The code would be trivial:

const encode = (title) => encodeURIComponent(title).replaceAll('%20', '+')
const decode = (text) => decodeURIComponent(text.replaceAll('+', '%20'))

encode('a+b') //=> 'a%2Bb'
decode('a%2Bb') //=> 'a+b'

encode('The First Rule of Using Tiddlywiki') //=> 'The+First+Rule+of+Using+Tiddlywiki'
decode('The+First+Rule+of+Using+Tiddlywiki') //=> 'The First Rule of Using Tiddlywiki'
#7

Oh wait, never mind, I see. If we come into http://wiki.example.com/Foo+Bar, that should address an already existing Foo+Bar tidder. Although the new scheme would never create such a fragment, there might be existing urls out there already in use. Although, again, they would not have been created even by the current scheme, which encodes the + as %2B.

That’s too bad, though.

#8

I’m sorry, I think we’re talking past each other here. I care only a little about the URL as it appears in the browser bar. And I don’t care much about how I type the link within the wiki text. But when I’m reading where I’m going to visit, I’d like it as readable as possible. If in an email or text, I point someone to the page for a cartoon character, I’d rather it read http://cartoons.example.com/#Bugs+Bunny than http://cartoons.example.com/#Bugs%20Bunny. That was my only concern: I want the canonical URL for the tiddler to be more readable when possible.

But @pmario is probably right. There could be someone already pointing to ...#Bugs+Bunny for an existing tiddler with the + embedded in the name. While that seems fairly unlikely, it’s probably enough to kill this idea.

#9

Actually, it’s not. The fragment identifier allows

  • ALPHA
  • DIGIT
  • pct-encoded
  • -
  • .
  • _
  • ~
  • !
  • $
  • &
  • '
  • (
  • )
  • *
  • +
  • ,
  • ;
  • =
  • @

To the point here, it allows +.

#10

That’s the same issue as %20. It does display %2B which is also ugly. IMO it’s not a fix. It’s kind of a hack.

#11

I’m not sure I follow. There is very little we can do to handle arbitrary punctuation for the fragment identifiers without doing significantly more work – more difficult work, at that.

But my suggestion can be captured in those two snippets of code above. When we create Permalinks/Permaviews, and maybe in other places where we turn a tiddler title into a URL fragment, we might make for prettier URLs by replacing %20 with +. And clearly we’d need to handle the reverse when loading the page with one of those URL fragments.

But while I’d really like this, I do recognize that it’s likely a non-starter because of a potential backward incompatibility for someone who manually created a URL including a +.

#12

Is it true that that you are unhappy with the encoding that comes from the permalink button?

The following solution will make links in tiddlywiki read;

  • http://cartoons.example.com/#Bugs Bunny
  • Forget any alt titles. + %20 or %2B

But remember;

That button is here $:/core/ui/Buttons/permalink and works by calling the message=“tm-permalink” which also sets the address bar and copies an encoded link to the clipboard.

Here I cloned the above button and modified it, to copy “a pretty permalink” without encoding the tiddler name, or changing the address bar.

  • eg; https://tiddlywiki.com/#Working with TiddlyWiki

Requires 5.3.x+ prettyurl.json (1.0 KB)

Once you paste the link somewhere else you need to make sure it is delimited because it contains spaces.

  • eg [[https://tiddlywiki.com/#Working with TiddlyWiki]]
  • or href="https://tiddlywiki.com/#Working with TiddlyWiki"

Links from the Above load in the FireFox address bar, with the spaces and not the ugly encoding and work. Unfortunatly crome is adding the %20's once you follow the link, but it works.

  • However your desire to read links in tiddlywiki without encoding works (always did).
  • The replacement permalink copies the “permalink” to the clipboard with spaces, unencoded.

Notes

  • This button will no longer respond to language settings.
  • If desired let me know and I can make this button dragable to drop the pretty url into an edit field, perhaps wrapped in [[ ]] or " "
  • If there are spaces in the link before the tiddler titles, these are being encoded, such as on file:// links. We may be able to fix that.

Advanced.
My own work is planned to copy to the clipboard a full url in a html a tag, including a pretty link, auto-generated target (tab name), tooltip and possibly more.

  • This will make links to external wikis open in the same tab.
  • This will allow me to hide the protocol https:// etc…
#13

Yes.

But that won’t do for what I want. I want a legal URL, one that I can cut and paste anywhere, and not worry about delimiting it. The current situation already has that. But I want one more thing: readability.

The work you’re doing is interesting, possibly even inspired, but doesn’t have much to do with my use-case.

That’s not my desire at all. The point is not how it looks in the internals of TW or how it looks in the browser bar, although both of those would be nice side benefits. The point is that I like readable urls, and this:

https://tiddlywiki.com/#Drag+and+Drop

is much more readable than this:

https://tiddlywiki.com/#Drag%20and%20Drop

and it is a legal URL that I can use wherever I want to point to an individual tiddler.

Spaces are a problem to me. First of all, this is no longer a legal URL. And if I paste it into a chat window, it will not link to the right place.

While I might spend some time looking at what you’re doing, and implement my idea for my own wikis, I do feel like it would be an improvement to TW in general. That’s why I raised the issue.

#14

@Scott_Sauyet the answers I gave you were because have not being able to understand your core requirement. Although that has “now come to light”.

  • Because I was not sure I gave some quite general answers as well, to help you no matter what your requirement proved to be. I do think there are details in my previouse replies that would help.
  • Respectfuly, oh yes it does. It has everything to do with your use case.

Yes, browsers, not particularly tiddlywiki, when copying a link containing spaces will encode them. But as you point out more characters are valid within the fragment (which is the tiddler title) part of a link. You appear happy if at least this part is more readable?

Sentence+Tiddler+title

  • Standard permalink gives you https://shared.tiddlyhost.com/#Sentence%2BTiddler%2Btitle
  • My permalink gives you https://shared.tiddlyhost.com/#Sentence+Tiddler+title

Sentence_Tiddler_title - I think I favour the underscore because no coding is nessasary, and it looks ok.

  • Standard permalink gives you https://shared.tiddlyhost.com/#Sentence_Tiddler_title
  • My permalink gives you https://shared.tiddlyhost.com/#Sentence_Tiddler_title
    • note above that if you used underscore rather than + the core permalink is the same as my custom permalink

All the above links arrive at their destination, and you can post into chats etc…

But wait there is more…

Using my aformationed alt-title package alternate-title.json (1.6 KB) you can name a tiddler with underscores Url friendly names or (https://shared.tiddlyhost.com/#Url_friendly_names but set the alt-title field to a title with spaces.

  • to prove this follow the above link to see the tiddler.

But wait there is even more…

Any tiddler you create effectivly has a matching URL automaticaly, as we have noticed this gets encoded, including the fragment. If the title/fragment does not need encoding, it need not be. My custom permalink will not encode the fragment at all.

  • Perhaps we could improve it to encode characters not in your list, above.

However on the same wiki https://shared.tiddlyhost.com/ I have installed my masqurade button. Although its triviual to do this manualy.

  • I went to $:/ControlPanel whose permalink is normaly https://shared.tiddlyhost.com/#%24%3A%2FControlPanel
  • Clicked the masquerade button and named it ControlPanel (you could name it with + or _ if you wanted. Just use any name that will not get the encoding)
  • And set the alt-title to anything you want to see in the wiki, or delete it to use the tiddler name.
    • Save the masquerading tiddler.
    • The permalink is now un-encoded.
  • Follow this link to see https://shared.tiddlyhost.com/#ControlPanel
    • note no encoding?
  • The masqurade also allows you to create a tiddler title you can search for in standard search that displays a system tiddler
  • For you convience I switched on the indicatior for masquraded tiddlers

In closing

Because we can have as many tiddlers we want if we want to craft how our links read, we just change the name of the tiddler, or create a new one and transclude the original.

  • The default permalink has one way of doing it and encodes the fragment at the same time, so don’t use it.

See the contents or recent tabs for related tiddlers.

#15

Most files are named like this for better readability

xxxxx-xxxx-xxxx

#16

Post Script;

If we could put together formal details about how fragments and thus titles have a different encodeing scheme to the URL address we could request the tm-permalink encode the two parts of the permalink differently (if nessasary).

  • I was thinking of writting a new encode operator to match your list above when I discovered in encodeuri Operator if you follow the link to encodeURI() does exactly that.
  • Again we still need to see if the universe it happy with this but it look like it.
1 Like
#17

https://github.com/Jermolene/TiddlyWiki5/blob/ceee20fd5970e1b75c2117d2522c998a6c5054f3/core/modules/startup/story.js#L181 You can refer to this document and write one that meets your needs.

#18

I am not a javascript dev so I can’t Are you just linking me to the source that would need to be modified to;

  • If so thanks, or please give more details.
#19

I have made changes to $:/core/modules/startup/story.js in a copy of tiddlywiki.com to use the more permissive encodeURI for the tiddlername only, both in the permalink and permaview, rather than the default encodeURIComponent().

  • The result is as expected

However with ChatGPT it seems to recomend the reverse, which we are not interested in.

As you can see, encodeURI() does not encode the & character, which could be problematic if you’re trying to create a well-formed, unambiguous URI.

So while you can use encodeURI() for a fragment, using encodeURIComponent() provides a more foolproof way to ensure that the fragment is a valid URI component without special characters that could interfere with its interpretation.

ChatGPT aggrees there is less risk if the start of the URI is well formed, and in tiddlywikis case it is, but suggests there may be edge cases and future standards that will clash with this, and some apps that will not handle it.

Anyway I have a fix coded and tested if anyone wants to try it is now on https://shared.tiddlyhost.com/#Home

Post script, using my mascerade tool you could create a tiddler with a very short name such as !1 resulting in a permalink of https://shared.tiddlyhost.com/#!1

#20

I think what we would need is a new permaview and permalink format that is similar to the POST - application/x-www-form-urlencoded format as described at: encodeURIComponent() - JavaScript | MDN

Important: The links below do not work with TW v5.3.1

Permalink

https://tiddlywiki.com/#New+Tiddler+with+Spaces

Which would show a valid link here in the forum. So it is a valid URL

https://tiddlywiki.com/#New+Tiddler+with+Spaces

  • # … fragment identifyer
  • New+Tiddler+with+Spaces … Title with spaces

Permaview

https://tiddlywiki.com/#first=New+Tiddler&New+Tiddler&encodeuricomponent+Operator

Which would show a valid link here in the forum. So it is a valid URL

https://tiddlywiki.com/#first=New+Tiddler&New+Tiddler&encodeuricomponent+Operator

  • # … fragment identifyer
  • first=New+Tiddler … optional – Would be the first tiddler shown in the story river.
  • & … separator between tiddler titles
  • New+Tiddler … Title with spaces

IMO the permaview is only a bit more readable than the existing syntax.

Problem

With this syntax TW has no chance to decide if this is the “new” or the “old” syntax. So we would need more info. May be

https://tiddlywiki.com/#!New+Tiddler+with+Spaces … Which would block tiddler titles starting with !

and

https://tiddlywiki.com/#?tag=New+Tiddler&thisOne&one+with+spaces

Where #! … Would be the new-syntax indicator
and #? … Would be the indicator for a tag query, that opens every tiddler tagged xxx

just brainstorming