Best way to organise local files for a single file html wiki

I have tons of images, videos and pdf which I embed in my wikis as local files using relative paths. I want to organise them properly so that I can retrieve them easily when I need in the future

So my question is can I create multiple sub folders (of course all these will be in subdirectory of the TW file ) to organise the files hierarchically.
Will this create problem if I shift to node js wiki in the future. Or should I save everything in a single folder in the TW subdirectory

I currently have subfolders underneath the main folder where I have my single file wiki. I have a separate subfolder for each type of external file (images, documents, etc.). I’ve had no issues with this at all.

I can’t really answer the question about a move to a node.js version. I’ve never tried to do that and know nothing about it.

I think a lot of the organization is going to be what makes most sense to you, the way you think and your work flow. If you’re using relative links, as far as I know, Tiddlywiki won’t care how you organize it.

2 Likes

I use the /files subfolder on nodejs version.

https://tiddlywiki.com/#Using%20the%20integrated%20static%20file%20server

1 Like

By coincidence I just needed to fix my paths to be able to use them with a SPA, a Node.js configuration and sending them to github.

The solution is relatively simple.

Single Page Configuration

In the directory, where you have your wiki.html

  • create a ./files/images directory for your images
  • ./files/pdf/ for your pdf’s – and so on

The _canonical_uri: will be ./files/images/image.png

Which will work for a single file wiki, a Node.js configuration and if you want to send it to a GitHub, GitLab and any other providers page, with the exact same structure.

Images can have more subdirectories, if you need them. But that also makes your _canonical_uri field harder to type.


GitHub Pages

You can have a look at the structure

Cheasheet demo: Cheat Sheet — Compact Wikitext Overview
GitHub Pages repo: https://github.com/wikilabs/wikilabs.github.io/tree/master/editions/cheatsheet
Dev Page image meta data: https://github.com/wikilabs/editions/tree/master/cheatsheet/tiddlers

For consistency reasons I did choose the meta-data dir to be ./tiddlers/files/images/image.png.tid. It can be any other directory as long as the _canonical_uri field is ./files/images/...


Dev Server Node.js

For my dev-server I use: https://github.com/wikilabs/editions/tree/master/cheatsheet-server which only contains the Node.js server configuration, the build scripts and the actual images in ./files/images/ direcotry.

The ./files/images/ dir needs to be at the same level as the tiddlywiki.info file. That’s a necessity

I hope that makes sense.

This configuration works for me. For others the whole directory structure may be different. But as I wrote.

All imges are visible for all 3 configurations, with only 1 source of truth.

./editions/cheatsheet and ./editions/cheatsheet-server, which can be used to dynamically build the SPA, the GitHub pages and it can be used to test and develop everything.

2 Likes

As others have illustrated tiddlywiki can reference files in its folder and below. The main point I want to make is TiddlyWiki is just another HTML Page from this perspective and not only accomodates standard references to files, but perhaps your decision about files and folder should be driven more by Internet and website standards than anything specific to tiddlywiki.

  • For example if you name the tiddlywiki index.html it will be opened by the browser automaticaly, on navigating to the address and/or folder. In that folder a favicon file can be read etc…
1 Like

In node, it appears that you can clone and modify the get-file route to add additional paths. I seem to recall, for instance that BJ was using a “media” image directory.

1 Like

i’m trying out a different approach for your reference. I use absolute path on single file TW, but the path is store in two parts: a device-tree path to a directory tree and a relative path within a directory tree.

The relative path (e.g. /books/nonfiction/ ) is store with the tiddlers.

The device-treeroot path (e.g. /storage/emulated/0/My Files/books/ ) is retrieved from a dictionary tiddler using the device-type and the root of the directory tree (i.e. “books” in /storage/emulated/0/My Files/books/ directory tree) as the key. The device-type (e.g. “windows”,“chromebook”,“android1”, ,“android2”) is determine by a auto-startup tiddler using environment parameters such as the TW file path and screen resolution.

i’m doing this now as my file directory trees can be from different storage media on different machines, such as a Windows machine with multiple drives, and an android device with (limited) internal storage and sdcard storage, which is how my files are currently stored. Note that the files themselves (especially on Android’s external storage) may not be accessible from within TW due to browser’s and Android’s restrictions, even if the file references are correct and can be used in other app.

I don’t use node.js, not entirely sure how this will work out if I want to use it in the future. If I need to relocate the file directory trees or switch to relative path, I should be able to update the device-treeroot dictionary without changing all the tiddlers.

1 Like

Thanks everyone for sharing your inputs.
@pmario I may need one more level of subdirectory than what you are using. Also I dont use _canonical_uri: field. Instead I prefer to use the [img[relative-path]] for images since there a tons of images in my wiki. Creating separate tiddler for each wiki is not feasible I think.

Are you referring to the media directory used by savemedia addon by buggyj ?
If yes, BJ has made it configurable - we can use files folder instead of media if needed.

@jacng This looks interetsing. Do you have to demo wiki to take a look at this. I want to see how the data tiddler for device-treeroot looks like.

Don’t have a demo. I will see if i can extract some relevant wikitext to share. A sample of the device-treeroot Dictionary tiddler below for info.

title: devicePathTable
type: application/x-tiddler-dictionary
text:

treeroots: [[[BigBooks]] [[BigPictures]] [[Camera]]]

windows-default: /C:/My%20Files
windows-BigBooks: /E:/big
windows-BigPictures: /E:/big
windows-Camera: /D:/DCIM

android1-default: /storage/emulated/0/My%20Files
android1-BigBooks: /storage/C633-1B02/big
android1-BigPictures: /storage/C633-1B02/big
android1-Camera: /storage/emulated/0/DCIM

chromebook-default: /media/fuse/android_files/My%20Files
chromebook-BigBooks: /media/fuse/android_files/big
chromebook-BigPictures: /media/removable/X128
chromebook-Camera: /media/fuse/android_files/DCIM

device: android1
device-default: /storage/emulated/0/My%20Files
base-BigBooks: /storage/C633-1B02/big
base-BigPictures: /storage/C633-1B02/big
base-Camera: /storage/emulated/0/DCIM

Note: Blank lines added in dictionary definition for readability.

In this example, there are 3 directory trees that are explicitly defined: "BigBooks, “BigPictures” and “Camera”.

The last group of defintions (“device”, “device-default” “base-xxx”) are set by the auto-startup tiddler based on the device-type.

default device-treeroot path
There is a default path for each device (e.g. “windows-default: /C:/My%20Files”) which is returned whenever a key (e.g.“windows-Books”) is not found. I have several directory trees (/My Files/Books, /My Files/Articles, /My Files/Pictures, …) using the default path (/My Files/) and hence need not be explicitly defined.

1 Like

What’s the advantage of using this approach…I store most of my media files in the onedrive…thats why I am using relative path so that I can acces them in both desktop and mobiles

Oh, probably not as useful for your application then.

For my usage, where the files are local and the file paths are different on different devices, this approach (using absolute path with a configurable device-root path prefix) manages the file paths for me automatically when switching between devices.

How do you use the same files in desktop and mobile without a cloud server ?

Then how do you use this path? With a macro to access the images/files?

I’ve used this approach. The only problem is that then you are using a non-standard macro to retrieve files.

The other approach is to overwrite the _canonical_uri field of resources. The advantage here is then that references inside of tiddlers can be made with standard wikitext syntax. The disadvantage is that the _canonical_uri field of maybe hundreds of images/files gets modified depending on platform.

To Arunnbabu and Mark:

On file reference
For my purpose, the file reference is useful by itself to know where the file is located. TW allows me to keep info and make notes about the files. I can do searches, sort and filter on these files using the file info and notes.

For a file “Sleepy Hippo.gif” in folder “My Files/Pictures”, i will end up with these absolute file paths:

Windows “/C:/My%20Files/Pictures/misc/Sleepy Hippo.gif”
Android “/storage/emulated/0/My%20Files/Pictures/misc/Sleepy%20Hippo.gif”
Chromebook “/media/fuse/android_files/My%20Files/Pictures/misc/Sleepy%20Hippo.gif”

On accessing the files from TW
It is useful also to be able to open the files from within TW.

To display the image in TW, I use html syntax :
<img src="/media/fuse/android_files/My%20Files/Pictures/misc/Sleepy%20Hippo.gif" />

To provide a file reference to launch the image file:
<a href="file:/media/fuse/android_files/My%20Files/Pictures/misc/Sleepy%20Hippo.gif">Sleepy Hippo.gif</a>

For images, this mostly works for me so far. One exception I encountered is images on external storage on Android Chromebook.

i don’t have much experience with using canonical_uri. It doubles the number of tiddlers :sweat_smile: I suspect I have not yet discover the usefulness of canonical_uri. So far, I’m not particular with the placement of images in my application.

I do the same for other files like PDF, TXT, EPUB etc, with varying success depending on the file types, OS, browser, apps installed, and for Android devices, even the particular models due to manufacturer customization. Generally, Windows has the most success in opening a file. Often, a browser will download the file instead. Sometimes, the browser will download the file and then immediately launch it successfully, giving the appearance that it supports direct access to the file.

Anyway, I normally use other apps to open the files (EPUB, PDF, TXT), so launching the files from within TW is more of a convenience and not a showstopper for my purpose. I guess that may not be the case for your usage.

For Chrome browser on Android, TW files must be in Chrome Download folder
Oh, I just remember one motivation for me to use absolute path. On Android, Chrome browser requires html file itself, such as TW file, to be in Chrome Download directory, which is “/storage/emulated/0/Android/data/com.android.chrome/files/Download/” on my phone now. My TW files on Android are in Chrome Download directory. You can get around Chrome by using Tiddloid or webdav to launch TW.

External media files (i.e. images, video, MP3 ) elsewhere are still accessible using absolute path when the TW file itself is in Chrome Download directory. Relative path won’t work unless my external files are all under Chrome Download directory which is no-go for me. There are so many caveats for external files :sweat_smile:

I’m not sure what is a non-standard macro nor the problem with it ???

I’m using macro like this:

\define showFileRef( plain, extf )
<a href="file:$extf$">$plain$</a>
\end

\define showImage( imgfile )
<img src="$imgfile$" />
\end

<$let ...
	  imagefile={{{[[file://]] [<deviceRootPath>decodeuri[]] [<relativeFullPath>decodeuricomponent[]encodeuri[]] +[join[]]}}}
      extfile   ={{{[[//]] [<path>] [<relativeFullPath>decodeuricomponent[]encodeuri[]] +[join[]]}}}
      plainfile={{{[<path>decodeuri[]] [<relativeFullPath>decodeuricomponent[]] +[join[]]}}}
>
<$macrocall $name="showFileRef" plain=<<plainfile>> extf=<<extfile>> /><br>
<$macrocall $name="showImage" imgfile=<<imagefile>> />
</$let>

Are there hidden issues with this approach ?

It’s about future-proofing. This comes out from our experience of going between TiddlyWiki Classic and TiddlyWiki5. None of our old macros or plugins worked!

This is acceptable and unavoidable for special tasks like cloud tags, data visualisations, Twitter-raking, etc. But for basic things like images and PDFs it’s problematic.

You would really like (or at least I would), your basic tiddler to be re-usable on some other platform or future version of TiddlyWiki. But there is no possibility of this happening when your images are inserted using your own unique macro calls.

That’s why you might prefer to insert images as transclusions which has syntax that is recognised on other platforms and certainly in future versions of TiddlyWiki.

Ah, I see. Thanks for the sharing !!! That’s not something that would come across my mind at this stage of my usage :grin:

I can’t recall now exactly why I choose these macro constructs, except that there were considerable trials and errors just to get the images to show up on various devices. If there is a straight forward wikitext syntax that could consistently display the images like these macros, I could use it. The other approach you hinted regarding canonical_uri_ field doesn’t seem like it. I guess I would have to give it a pass :sweat_smile: