Externalising Image Files with JSzip using tools you probably already have

,
#1

Theory and motivation

Images take up space. In general, the more space taken up inside a TW file, the worse the performance. It would be better to outsource images to the local file system, or a remote file system. The solution is to externalise files. Externalisation means exporting the image tiddlers to an external image file, emptying their binary content and populating the _canonical_uri field with the path to the external image. (_canonical_uri was chosen because of its memorability). There are 3 main ways of accomplishing this task:

  1. By hand
    You can do all these steps by hand, but it’s slow and error-prone.

  2. Using Node.js
    There are instructions on Tiddlywiki.com for externalising files using the ‘‘node.js’’ version of TW. The problem is that you need a node.js installation, and it doesn’t work with stand-alone files. So you would have to convert your TW file to tiddlers, process the images, and then convert back

  3. File Upload plugin
    The File Upload Plugin can automatically externalize your image files. The main complication here is that you need to have your TW file served by a WebDav server. If your normal method of accessing TW is not via WebDav, this might be complicated.

Here is a different approach – a set of tools that allow you to zip up your image files into one convenient download which you can then extract wherever you want. You can specify paths that get stored in the zip file and have the extraction recreate them when you unzip the archive. Every major platform and most smartphones now have a zip tool built in!

At the same time that the zip file is made, the image tiddlers have their new _canonical_uri field set. For this reason, it’s absolutely essential to verify that the zip file has been successfully created before saving your TW file.

Note that some output image names may have to be changed in order to accommodate allowed file paths.

I’ve only tested on Linux. Pathing might need to be tweaked for Windows.

External Images using JSZip

2 Likes
Generating static pages but not for upload at GitHub
Towards a friendlier workflow for images in notes
Easily link images as external content
#2

Nice work. I will test it on windows today.

Thanks this is very helpful as most images I put on tiddlywiki are one off clips in documentation and training. They dont exist anywhere other than in tiddlywiki which uses and documents images internaly and I need not even rename them.

To periodically externalise images keeping wiki size lower is fantastic.

#3

Thanks @Mark_S

This seems to be technically working for me, packaging Images, updating image tiddlers etc… however somehow the external images are not displaying. I will continue to experiment and document what I learn.

  • This on wikis at a file address.

Some notes;

  • Would it be ok for me to package this with additional user interface elements and features? with acknowledgement?
  • I am curious why you used a JavaScript module and if its possible another way
  • I wonder if we can get folder/path information from $:/info tiddlers just to automate it a little further.
    • With more than one local file wiki in a single folder I see value in placing images for each wiki in a /wikiname-images folder rather than just /images.
#4

Compare working addresses with generated addresses. Maybe windows/browser wants file:// or something, for instance.

Sure, as long as you’re supporting :wink:

I am not aware of any other way to automate the exporting of image files. The javascript module simply adds one more message to the messages offered by the ziplib plugin.

What information is available? I’m pretty sure the browser doesn’t expose information about the file system.

An experiment I didn’t try was if you have a tiddler like myotherfiles/myimage.png, whether it would retain the / in the zip. It probably should if it doesn’t. So that would be one way of specifying different paths. The code is pretty simple – you could just add a selection to specify whatever file path you wanted. Or have the path stored in each TW. Whatever works best. But for people just trying it out, using an edit field is simplest.

Thanks!

1 Like
#5
  • Of course - thanks :nerd_face:

Yes and no, open $:/info/url/full on any wiki and you have the address of the wiki, on file its the full path. Of course this allows us to construct a default fullpath/images and when this can’t be injected into the process we can copy to clipboard and paste trhe full path into the save dialogue, you don’t even need to navigate folders just save with the full path and filename.

  • The converse is also true in the save and open dialogues you can copy to clipboard a filename and/or the full filepath to a file and past into a field back in the wiki.
  • This may help workflow if a folder name is inside the zip, and you want to save the zip in the folder above etc… for expansion there.

Suggestion;

  • Perhaps you should add a delete $:/temp/_ZipTiddler tiddler step since it could be quite large