New Uglifier update. Now compress wikitext!

On top of uglifying javascript and css, the TW5-Uglifier now compresses wikitext!

Now we can all feel free to pretty print our plugins as much as we want, since we know now that we can publish compressed versions of our source, or the end-users can compress plugins themselves with the easy-to-use wizard. Don’t worry! It doesn’t touch your own tiddlers. Only plugins.

Check it out, guys. It’s kinda fun just to see what your wikitext turns into when you feed it in, and the plugin has a nice demo page.

Also, if you think this is super cool, give it a star in github so that more people can see this. The more developers that know about this, the less they’ll publish their plugins in cryptic unpretty forms like jerks, because now they’ll know there are easy ways to shrink their work down.

Just a few highlights:

  • You can now customize what tiddler types Uglify compresses. For instance, maybe you want your wikitext uglified, but not your javascript? Sure thing. The wizard has a simple customize dropdown, and NodeJS servers only need a few config tiddlers.
  • The wikitext uglifying applies many tricks, such as getting rid of whitespace, removing unnecessary tags, and changing attribute and parameter quotation to its most efficient alternative. All while making sure the generated html from a page is identical to what it used to be.
  • Nearly 400 unit tests to ensure that uglifying does things right.
  • Once Tiddlywiki V5.2.1 releases, and the new $let widget is released, there will be even more rules applied to compress wikitext.

So yeah, check it out. Also, I’ll be working with the core code over the next few weeks to optimize it for uglification (and also just improving it in general).

-Flibbles

3 Likes

WOW! Absolutely amazing!

1 Like

@flibbles I have two colleagues that are enormously grateful for your work on the Uglifier, thank you.

1 Like

Some statistics

  • Empty.html 5.2.1 (2021.11.21) ~2.240MB
    • empty.html minified: 1.33MB
  • Empty with my essential plugins (my own selection = 22 plugins including codemirror, highlight.js kookma essentials, fontawesome, relink, relink-title, …) 4.02MB
    • This minified = 2.82MB

This is amazing!

3 Likes

It is very impressive! Couple of points …

1 - Particularly for on-line TWs I’d like to know whether it also improves “load-time”?

2 - Over some days I will test if I can add it to any wiki and it runs robust. If so, then I’ll add it to everything.

Good stuff!

TT

1 Like

Are you sure those numbers are right? It looks like you’re barely trimming anything extra off of your “essential plugin” mix. 0.9MB is about what you’d expect trimmed from just the core.

Anyway, the secret to good wikitext compression is \whitespace trim. Uglify’s results will get better as I introduce it more consistently to the core code.

Thank you!

If you think you find any wikitext patterns which might be buggy, you can plug them into the demo’s wikitext example, and it’ll tell you if it’s messing up or not.

2 Likes

I think yes! Lets repeat the test. I update this post within 10 minutes.

@Flibbles
I repeated the test, this time they were 22 plugins (two more subplugins from CodeMirror).
I used the drag and drop and then delete uglifier. For 4.02MB ->2.82Mb around 1.2MB.

I would appreciate if other users can share such numbers!

Hmm. That is underwhelming. I would have expected more. May I have a list of some of those plugins so I can see why they aren’t shrinking as much as I’d expect?

I suspect they just don’t take advantage of “\whitespace trim”, which most won’t.

See the list of plugins in uglifier test: (most from official lib kookma lib , Relink lib)


Hmm. Well, I looked at some of them. And it seems like they’re mostly a bunch of small tiddlers. And some aren’t optimized with “\whitespace trim”, so there’s only so much Uglify can do. Ah well. The core can still compress more; I’m working on that. And there are still some rules I can add that’ll help a bit.

I follow your work in progress on GitHub! May be I can optimize some plugins for better compression!

Hi Flibbles,
thanks a lot
it would be great to have a beautifier if ever I want to edit my uglyfied plugins (and do not find the last version befor uglifying - I know this could happen to me;-)
The service also would be nice if it could simply do the update to the newest version of TW.
Best wishes Jan

If you use Tiddlywiki on Node.JS, then Uglify only affects what’s sent to the server. You could disable it in its config tiddler, reload, and get pretty content again.

For updating your tiddlywiki to the latest version, I’d recommend the updater instead. The only reason Uglify updates your project at all is because it needs a minimum version in order to function.

1 Like

Great! Astonishing that this is not listed prominently on the tiddlywiki.com site.
A good position for the link could be next to the get empty button. So far, I always got an empty file to upgrade - Ah, stupid me, I found it under upgrading.

And alas I am not using node.js yet, because I need plenty of php-backend functions. I wonder whether this is compatible.