Lightweight Edition of Tiddlywiki

#9

I like exploring the minimums and maximums of my tools however on balance I don’t feel there is an imperative, for me at least, to reduce TiddlyWikis size, and these are a few of the factors.

Of course empty.html is our minimal wiki and if you are using a server you can split out core code into a separate reusable set. Perhaps we could call smaller versions sub-empty.

It seems to me the simple desire for a smaller minimal solution possible has as many reasons as the people who ask for it. The question is why?

  • For lower memory devices
  • For lower storage
  • Benefits if potential users are large eg 100,000’s of user x per user size
  • For easy interchange
    • You could build a content or changes only interchange method to help with this.
  • For less complexity
    • For whom, yourself, the user?
  • For simpler user interface
    • You can hide as much of the UI as you want from users of course.
  • To prove it can be done
  • Because the audience are sensitive to slow bandwidth.

You can hack tiddlywiki into a lower “size” by removing components, and you could in most cases search for core dependencies’ as you do so, so you don’t break it totally.

  • But this effort needs some reason or payback.

However if you review this site for some insight Understanding file sizes | GreenNet you can see that a 5MB file for which by comparison tiddlywiki.com uses < 6.7MB and empty.html < 2.3Mb these sizes are similar to;

  • A three-minute MP3 audio at a very high bitrate (256kpbs);
  • 1 minute of low-resolution video, or of streaming from a video-sharing site;
  • all the Wikileaks cablegate files released by mid-Dec 2010;
  • a 20-page PDF which might include a badly-chosen cover photo;
  • the complete works of Shakespeare (uncompressed)

Also if you build the server infrastructure with javascript and other externalised components then the per-unit space required including server savers etc… is reduced, but arguably you would need a few before they save more in total than the server bytes, even if the per wiki space is lower.

Just a few ideas from somewhat of a contrary position.

2 Likes
#10

So as it happens, a feature I added to Uglify was one where you could use filters to specify swaths of core tiddlers which you wanted to strip away in your uglified version to remove features you didn’t want.

For instance, remove all themes. Or remove the ability to edit tiddlers (if you were publishing your TiddlyWiki), remove the encryption stuff, etc…

Though the features is 80% done, I discontinued work because I wasn’t sure there would be enough interest.

Is there? (And not just from you @Mohammad. I already know you’re interested.)

4 Likes
#11

I think, comparing those 2 apps is not really fair. It’s a bit like to compare a “Word” with “Notepad”.

IMO it depends, what is lightweight for you?

If TW is sent in a compressed way over the wire the core.js is about 390kByte and the data.html is about 25kByte (data.html) … Including full functionality.

So if data volume, that needs to be transferred is a problem, it’s mainly a configuration setting of the server host.

If data stored on the disk is a problem. I do have a question? … On a mobile phone. How many “snapshot” images do users store? … If the answer is 3. Then it’s about the same size as the wiki at tiddlywiki.com including all the content.

So if 2 images are a problem, then yes. TW in the current format is a problem.

It is possible. If someone would pay for the development, it should be doable. But I think it wouldn’t make much sense. …

As you know, an empty wiki minified is about 1.3 MByte in size. …
empty.html is 2.4 MByte

The following values are plain text. … So not minified.

  • removing the encryption library would save us 25 kByte
  • There are about 100 button images as SVG, where each of them is less then 1kByte
    • So removing 50% of them for (view mode only) can save 50kByte
  • language tiddlers contain about 80kByte of text.
    • So compiling a “super custom version” which only contains the target language for the UI text will save us 80kByte
    • There are 18 pallet CSS tiddlers 4kByte each. … If we only add 2. 1 dark, 1 light. I will save us about 60kByte
  • Removing all the node-js server command line commands will give us about 40kByte
  • Removing all the redundant default saver modules (~2k each) will save us 25 kByte
    • default and WebDav saver only
  • Removing all the nodejs - server code gives us 29 kByte
  • publishing with classic-storyview only will save us 11 kByte (zooming view needs 8k)
  • The whole TW UI tiddlers are about 300 kByte
    • Compression (make it unreadable for humans) could give us about 100k
  • Removing the editor code will give us 40kByte
    • Yes. The core editor specific code is probably a bit more than 40 kByte.
    • In TW everything is highly reusable.

That’s basically it. The rest of the code is widgets. We can’t really remove them, because the UI would be not be possible anymore.

So in sum, I think there is a potential of saving 400kByte – plain text code – which will result in a highly “crippled” version of TW. … No edit, 2 pallets, No theme switching,

Probably 3 Month of full-time work to make it work again and 3 month of changing the docs, to describe what doesn’t work anymore. … The result will be an incompatible TW version.

So if a user adds 1 image which is 500kByte in size, all the optimizations are gone. For reference an image on a today’s phone is at least: 2MByte. JPG can not be compressed. … BUT It can be “optimized” for the web (see more later). …

From my point of view, optimizing user images for the web and telling users how to do so, will be a much better investment. It will need 1 or 2 hours to tell them.

So my question is: Is it worth the effort, or are there “low hanging fruits” somewhere else?

As I wrote above. TW code itself is highly modular. Most of the code is reused. The main difference between a nodejs version and a browser version is “boot”, “startup” and “fake-dom” code. This code is 120k plain text. So may be 30k could be saved, since more than 1/2 is shared with the browser version.

Styles, themes are part of editions already. empty.html doesn’t contain them.

Most users make their wiki big using plugins, they don’t really need or use … Eg. IMO A published read only wiki, doesn’t need codemirror or code highlighting. …

Using optimized JPEGs, PNGs and SVGs will result in much bigger wins.

Squoosh and SVGOMG will need some “digging”, but the results can be impressive in combination with _canonical_uri

SVGs are scaleable, so they are still usable if eg: printed. … JPGs and PNGs for printing need to be different to images optimized for the web.

Extra info, which may be interesting

There is a possibility to externalize the TW core. BUT … but it will add complexity to the maintenence. Since it’s not a “single file wiki” anymore.

On the other hand, if you use GitHub as a platform it can highly improve the “save to the web” speed. Since it only saves the wiki content back to the server.

Assuming no user-plugins are used, since they would go as content by default.

It would be possible to include plugins in the core part, with a custom built wiki.

just some thoughts.

#12

Incorporating an optional image size optimization using Squoosh or similar was part of the plan for the FileUploads workflow, that has been postponed until the syncers and savers refactoring that will make it easier to have multiple background daemons working together so that a workflow could look like:

  • import image
  • image gets compressed if it meets predefined criteria
  • once compressed, the image is uploaded and a canonical URI tiddler created.

Alternatively you could upload the full fat image and save the compressed version or a thumbnail in the wiki.

3 Likes
#13

I did think about it, how we could make maintenance easier … It may be possible to create a browser AddOn, that contains the TW core code. … So if the externalized core can’t be found locally or online, the boot mechanism may be able to use the addOn instead.

I think this would be worth some experiments. There will be some new versioning problems but it would still be interesting.

A second option may be a browser AddOn, that contains the core and all core plugins. With a simple UI it should be possible to create and save an externalized and configurable core.

So if the user has a “stripped down” data.html only, they may be able to create an external core, that could be used by the data.html …

The addOns won’t be light weight, but the data.html file would.

just some brainstorming.

1 Like
#14

About publishing.

An other possibility, that would be worth it, to invest time, is the TW static file creation. There is an all tiddlers html file, which is “non linear”. … Making it “linear” would be relatively straight forward and worth some effort.

Doing so may also result in much better accessibility. IFF we would make that requirement a number 1 priority. Having a “screen reader” friendly publishing option, that could read a “linear” version of a wiki in the background, while doing something else, IMO would be a “killer function”.

Improving TW output for the browser “reader mode” (F9 with FireFox) would be an extremely nice to have for me. It would be simple to experiment with that new browser mode and give feedback. … It’s time consuming, but it may be worth it.

3 Likes
#15

Hi @pmario
Many thanks for your complete explanation. These are very useful for one likes make a customized smaller size TW.

As TW is a very flexible tool, what I raised above was to see possibility of making custom, minimal editions for special purposes. Examples:

  1. A searchable help file distributed with a Python code
  2. A searchable html with TOC contains a research essay of a graduate student
  3. A digital catalogue distributed with a device
  4. A homework where teacher can add comments and grade it

Not all of them needs to be full 2.5MB and distributed through emails, …

So, while I am agree that sizes like 5MB for today devices is not a problem at all, but when I have 10kb information, it does not make sense to distribute it as 2.4MB

I appreciate your answer and I am aware of features, flexibility of full version, this is just adding extra flexibility to TW, having option to have a lightweight version with some functionality

#16

This would be very useful! In the the current state, TW exports a 2kb tiddler to a html file with 130kB.

#17

Almost all of it is the stylesheet; the static export includes a snapshot of all of the wiki CSS. With a different template one can specify more minimal styles.

2 Likes
#18

I believe we could further reduce the size of SVG icons by :

  • Only storing the path content (d= …) and using a template to transclude each path
  • Going one step further and manually optimizing the path with tools like SvgPathEditor can lead to slightly better compression
  • Using CSS SVG icon rather than transclusion (only need to set a class to the button rather than transcluding the title)

It’s probably not worth the effort, but I will do some tests and report back my findings

2 Likes
#19

Most TW icons are already hand optimized in a way, that we can still edit them with eg. Inkscape. If we add wikitext, then those tools can’t edit them anymore.

#20

What if we made them tiny and packaged TW with a magnifying glass? Less pixels means less space right?

#21

Which made me think of, paraphrasing, “I’m crushing your SVG, I’m crushing your SVG…”

#22

We could use a template to display the wikified SVG icons + a button to export the SVG code, that way they can be edited in a svg editor easily ?

EDIT : Here are some other ways to optimize SVG / Images

#23

Just to point out that we already optimise SVG icons with https://npmjs.org/svgo:

At the moment the processing is manual, so any unoptimised SVGs are a bug and should be reported.

#24

I spotted a few path some times ago that could be further optimized, however I dont think that’s a bug but rather that SVGO does not always find the best optimization possible. It’s not much, about 50 char gained by manually tweaking the path. I will try to find these icons and submit an improved version if applicable.

EDIT: For example, the path in $:/core/images/add-comment is 329 char. Using SvgPathEditor, it’s possible to get it down to 265 char. by using the “round” option, without visual difference in the output.

optimized :

image
image1382×978 165 KB

vs original :

image
image1383×980 165 KB

#25

I compress my wiki and GitHub - tiddly-gittly/Tiddlywiki-NodeJS-Github-Template: Template to generate a auto-sync-to-github, privatcy-in-mind, freely-deployed TiddlyWiki. using terser GitHub - terser/terser: 🗜 JavaScript parser, mangler and compressor toolkit for ES6+

#26

And removing writing-related plugins is a good option when building read-only blog site

1 Like
#27

When considering a Lightweight Edition of Tiddlywiki as usual I would step back and ask why, as well and what you are prepared to give up and what is the reason for this?

Tiddlywiki as Site Generator or Generate alternative tiddlywiki editions

Something that will help you if you find a clear reason to do so is to keep in mind;

  • TiddlyWiki is a powerful publishing tool, whether it’s different versions of itself such as plain text, static html and more. You can craft the output for publishing, it can be a site generator if you want - step aside WIX and Squarespace. So perhaps generate output rather than minimise.
  • It’s so east to generate alternative tiddlywiki editions either for publishing or designed to publish.
    • The Innerwiki plugin can generate tiddlywiki’s in an iframe including or excluding whatever you want from the parent wiki. If the result is viable you can save a copy. If not viable it wont work.

So tiddlywiki can be a Software Development Environment for versions of itself, another connection with “non-trivial Quines”.

In my view the “raison d’etre” for tiddlywiki is its User Interface allowing it to edit itself and maintain relationships between elements that are always up to date. If you don’t want these then use tiddlywiki to publish different content, or use another tool.

  • Using another tool like feather wiki is not so bad because there are a lot of ways to interact between full wikis and say feather wiki.
2 Likes
#28

I think that’s an excellent thing to discuss.

Although my pairing is TiddlyWiki and Feather Wiki (well, I’ve got an ecosystem of supporting light tools), the same reasoning would apply to TiddlyWiki paired with a lightweight version of TiddlyWiki for my workflows.

Which is the same reasoning as: why would anybody want both Notepad and Microsoft Word on a Windows computer? There are some tasks just better suited for the lighter tool.

When I want to do some serious intertwingularity mapping, I grab TiddlyWiki.

When I want to just capture a quick note and need some quick and simple formatting to rapidly organise information at that moment as it is flowing fast, I will reach for the lighter tool every time. Feather Wiki does it for me because not only is it tiny and fast, I don’t have to put up with markup, which slows me down when I’m trying to rapidly organise/emphasise the text I’m taking down.

Which I will intertwingularity map the daylights out of later with TiddlyWiki.

I may not be comparing apples with apples, or oranges, or whatever fruit, but I imagine the case for a lightweight TiddlyWiki edition as sidekick for the regular edition could be similar.

Aside:

If anybody is toying with the thought: A “Linux From Scratch” kind of thing for TiddlyWiki, the cost benefit doesn’t make any sense to me.

Recipes for taking (what we have now) empty TiddlyWiki and turning into a solution for something, now that floats my boat something silly.

2 Likes