Organizing computer data files and matching to TiddlyWiki

[wikster]

I recently started thinking about how I could organize my life. This is a very broad aim on which everyone has a different perspective. In my case, I identified the following “spheres” of interest. In no order of importance these are:

Learn
Do
Socialize
Health & Well-being
Home
Entertainment
Travel
People
Financial & Taxes

I visualize each sphere as an orange and like an orange a sphere consists of sub-parts (slices). There are no airtight spheres and therefore they may be interrelated (like having oranges inside the same basket).
Please note that no work-related spheres are included in the above. That’s a different universe!

My main aim with TiddlyWiki is for it to be a useful instrument to record relevant activity in each sphere or slices of it. Ideally this should be reflected to the greatest possible extent in my organizing my digital life.

For the sake of illustration, let’s say that “Home” includes regular maintenance tasks for which I need to set a reminder. I’ll set one or more reminders as necessary in TiddlyWiki (using the excellent TiddlyTools Time plugins) and keep a record in TiddlyWiki of who the maintenance engineer was, when the maintenance jobs were carried out, contract details etc. I’ll link the tiddler(s) to my contract and other documents stored on my PC.

Thus TiddlyWiki will be my first point of access to most of my data on my PC and I’m optimistic that this method will be useful in organizing my data on my PC. I’m thinking how best to use tags in TiddlyWiki and relate them to directories on my PC, but I’m finding it difficult to see how to actually implement it.

I would welcome any suggestions or ideas you may wish to share about using TiddlyWiki based on your own experience.

[seasamin]

I started the same process again a year ago when I upgraded to TW5. I had many TW2 HTML files for different life aspects. The lesson I learnt was that TW5 along with its available plugins (particularly Projectify and Stickies To Dos) meant that it handled more content and task management much better.

I found some life aspects like Home maintenance and management whilst involved enough for their own TW they also were so deeply integrated with regular life that I need task control along with reference notes through my primary life TW.

It is very easy to drag & drop tasks/tiddlers between TW files but maintaining separate TWs for some areas was just inefficient unless there cross-over was limited.

The overriding need was to have one task management TW to manage life and then build on great plugins to simplify that separation of life aspects using the Projectify plugin system for separation and then integration in current activity by using the Stickies plugin to bring focus to current needs across a Project or Cross-Projects issues like shopping for materials etc.

The Sticky Table allows sifting current sticky tasks by filters so if you use additional tags or fields to help like the Projectify field for “priority” (limited to up to level 3 for most urgent) and others you may create to separate table display needs. Your best friend in that process is the “Tiddler Commander” tool to quickly create and reassign tags, fields or other variables.

I also find tiddler story display too confusing and prefer the Tiddlers displayed as Tabs but what ever works for you.

Some Life Aspects may be distinctly separate needing their own TW such as Tech Maintenance, Investment Research or Health Issues that would overload your main Life TW file and may also want to share content with others.

So during the last year I decided to reintegrate some separate Life Aspect TWs as categories in Projectify that can have Projects and Sub Projects in each Projectify Category. As I did tiddlers on research, solutions & completion issues I change tags to keep them in the project but as tagged Context-“LifeAspectName” collections.

I started my TW5 conversion last year by making a pro-forma TW with all my preferred tools and layout. So if I need to extract a whole project or parts in a separate reference TW it is not too hard but it can be a little tedious for a day or so to get it right.

[wikster]

Thank you very much for your comprehensive reply. It’s very useful for me.

I had a look at Projectify a few months ago, but I decided that it was much more than what I need for my own circumstances. It is nevertheless quite impressive and I almost wish that I could use it!

I like the idea of the Sticky ToDo plugin, but I’m unsure if I really need it. I’m intending to write a Journal every day where I may write down random thoughts, but also to add a list of things to do as they come to my mind. I would deal with those To Dos later in the day either by doing them or adding them to my Calendar as future tasks. It reminds me of a similar procedure with another application (Logseq) that I used until very recently. Therefore this plugin could be useful to me after all.

[seasamin]

Projectify is not as complex as at first look. But for a journal maybe just a good tagging system may do.

Sticky To Dos I saw on first look it could be powerful but did not really realize how to apply to my TW structure.

The completed Sticky by a checkbox tick adds good journal tool of when that Sticky was dealt with recorded by pop-up date completed in the original tiddler. Making a Sticky is simple by just highlight a phrase or list item and click the Sticky button then track relevant Stickys across all tiddlers with Sticky Table layouts.

As you apply TW principle of breaking down topics into smaller tiddlers then tags and fields for those tiddlers gives power to sorting relevance of Stickys to related tags and/or fields.

So one Social tiddler on a single get-together activity may relate to social issues, bookings, events, friends, finances etc and adding tags or fields for them allows the Table to focus on displaying and processing unfinished issues grouped by different topics like those. Sorting turned out very simple once I saw the power of the Sticky Table with applying filters and using Tiddler Commander to do grunt work of tag and field usage across many tiddlers.

Good luck with it all !

TW power allows ongoing development as you use it and that makes it more powerful than most other software for its price or often at any price. Your methodology is your own and not controlled by third party software layouts.

[TW_Tones]

@wikster if you use TiddlyDesktop or the aging hta version you can open links and even programs from tiddlywiki. This allows a tighter integration with tiddlywiki. You can import folders and paths and use file links to access them.

[wikster]

Thanks. TiddlyDesktop is what I use and I’ll certainly have a number of links to my files/directories inside my TiddlyWiki.

[pmario]

For the links to your files I would go with _canonical_uri. See the second option at: How to embed PDF and other documents

I’m working on a little function at the moment that will let us drag & drop import canonical uri’s from the file system using the $browser-widget with a new message.

The best way is, to use relative paths, which makes it possible to move files and keep the _canonical_uri value intact.

[TW_Tones]

I often use the open or save dialogues to copy a path to a file or folder eg r-click. Then I can paste the path into my tiddlywiki. The file path is then available for the wiki to use.

• in windows we can also copy, delete and rename in these file dialogues.

But if course I need to cancel the open or save activity.

I wonder if it is posible to trigger a file dialogue to navigate around file systems to select files folders or r-click functions but not for import, just to get the path or filenames or manage files?

Keeping in mind this grants no access rights the existing dialogues dont already have. This is in part because we can paste the result into tiddlywiki even if the dialogue will not capture the path info.

I will just add I have import and save buttons to trigger an import or save which first copies the full path and file name stored in the wiki to the clipboard. Then in the dialogue I used ctrl-V to paste the full address with no need to navigate just open.

[TW_Tones]

See also A small tool to import links to files instead of files - #12 by TW_Tones

[tw-FRed]

Hi Tones,

As far as I can tell, what you want to do isn’t possible in a browser for security reasons. At most you’d get a filename from the <input type="file"> element, but not a full path.

Source: MDN (see note #2)

Fred

[TW_Tones]

In windows 11 I use this dialogue to find the file or folder and r-click copy as path. Then paste the result where I need it.

[markkerrigan]

I know this might seem obvious, but I suggest putting all of your TiddlyWikis in one single folder, ie

tw/wiki.html

Or tw/notes/wiki.html

That way you can create a common “reference” folder or “photos” folder and have it be shared amongst all of your wikis.

[dionisos]

I think that naming and organizing stuffs is quite difficult, uncertain, and a subjective task.

First, I think we can explicitly state the goal of both : Quickly finding and accessing stuffs.

We can start with different information about the file or tiddler we are searching for :

1. We know a complete path to it.
2. We know the name of what we are searching for but not the path.
3. We don’t know the name, but have many hypotheses about what this name could be.
4. We just don’t know the name, but know something specific about the content.
5. We neither know the name nor the content, we just have some [fuzzy] conceptual link to it.
6. You don’t even know you want to find it.

There are also many situations in which we can be :

• A+/-) We are already more or less near the thing we are looking for.
• B+/-) We want to access the thing more or less often.
• C+/-) We want to find something more or less specifics.

Examples :

• 1B+ ) I want to open my todo list, mostly I will know the path to it givens I open it each day, the only real challenge is that I want to access it quickly given I do it often.
  My solution here, is just to have a “bookmark” tag on it, and have a tab with my list of bookmarks (so, I access it in at most two clicks)
  But I only bookmark things I really open often (or often in the situation I am in), otherwise it woulds get too big and counter-productive to the actual goal (currently, I have only 5 links in it).
• 2 ) This situation is very convenient in TiddlyWiki, I just use the standard search bar and it is quick. (it is less quick for a file on the computer, but it seems to be one of the point of accessing it through TiddlyWiki)
• 3 ) There is some “bug” in Emacs I already created a tiddler about, but did I name it “bug_emacs”, “BugEmacs” or “ProblemsInEmacs” or even “ProblèmesInEmacs”(the French word).
  One way to be more often In 2) than in 3), is to have the stricter naming conventions possibles, and to follow it.
  I also have a field “keywords” in some tiddlers, and I modified the standard search bar for also searching in this field, I use it where there are many synonymous for the same things with no obvious better one.
  Otherwise, either you will have to try many possibilities, or use another way to find the tiddler/file.
• 4 ) Very easy in TiddlyWiki with a quick search, way too slow for a file if you don’t know its extension and at least a partial path to it (sometime still the better way, if you are out of luck).
• 5A+C- ) I want to see some photos of my 2020 holidays, here I certainly don’t know the full path of each file, nor the content(or it isn’t really searchable), but it is still easy as long as I know a partial path, because I don’t want something specific. I also want quick access to other photos as soon as I open the first one. This is what directories or lists in TiddlyWiki excel at.
  In TiddlyWiki you can make tiddlers with lists for anything that makes senses, for files, you are restricted to a tree (or you can use [symbolic] link, but it is way more bothersome than to add a tag on the concerned tiddlers).
• 5C- ) This is the worst situation, and you don’t want to be in it, or if you are in it, you want it to be as retrievable as possible (this is after-all, one of the main goal of all of this).
  Having many ways to access to the same thing, or/and using partial paths from the context you are in, help (if it is specific enough for it to not be too long to go through).
• 6 ) If it is something you want to remember at a specific time, obviously you want a reminder at this specific time. For tiddlers I know I will forget to even look at in a particular context, I use a special tag on my tiddler (I name it “serendipity” not a very good name, but I stick with it). So when I am in a particular context, I try to remember to look at a list of Context+Serendipity. For instance, before going outside, it gives me a small list of stuffs I sometime to forget to take with me, my shopping list, and if it is a holiday day or not (yes, I am a little dumb and head in the clouds ^^).

Here are some methodologies I think can help.

• Having conventions for names and where to put stuffs, it helps both to not waste too much time when you sort stuffs (you don’t want to have to do too many choices each time), and to find it backs (because the choices you did are more constants and you build habits about them).
• Having a directory/tag for all the things which are still not correctly named/tagged/put where it should be. (you will not always have the time or will to do it correctly, and you want easy access to everything which should still be sorted)
• A description of what to do when something should be in many disparate categories : In TiddlyWiki it is easy, you put a tag for all of them, for files in directories, you will need a rule to choose (and choose quickly and consistently).
• You need a rule for the different levels of specificity, for files, there are not many reasonable choices, you put it in the more specific directory possible, but I think it should also be the rule for tags : If I have a tiddler about Emacs, I tag it with “Emacs”, but not “Software”, because “Emacs” is a strict sub-category of “Software” (if I think it isn’t a strict sub-category, I still use both tags).
• As a meta-rule, when I have to do a difficult choice about something, I add the general rules about how I made this choice in my list of rules (and why), so I use it when I encounter a similar problem (because I think consistency is one of the key).
• As another meta-rule, rules should not be too hard to follow, otherwise I will not follow them, or I will lose time trying.
• I often ask myself “how will I probably try to find it in 2 years ?”, when I have to take a difficult decision. If in two years I find it somewhat hard to find because I don’t try in the way I thought I would, I can even think about where my expectation about myself were wrong.

Sorry it was long and badly organized (this is sort of ironic ^^), and I probably forgot a lot of stuffs, there is so much to say about this subject.

[TW_Tones]

I am confident we can find answers for you as far as is possible. The truth is the information must be available and unless you catagorise, rename or annotate images inside tiddlywiki or outside the information will not be there to be found.

• The truth is this is an expancive problem that needs an expancive solution because most of it is about things in the “external world”.

One idea that comes to mind is using AI to analyse the images and provide descriptions of the images and place this in a tiddler referencing each image.

• perhaps we can semi-automate it to get descrptions to save in tiddlywiki.

Here I uploadeed the attached image to ChatGPT (sunbscription) and asked for a description.

Can you describe this image?

The image depicts a night scene with a clear view of the sky. Dominating the sky is a large, reddish celestial body, which appears to be a “blood moon” — a term often used to describe the reddish coloration of a lunar eclipse. The sky around the moon is dark, but not completely black, suggesting the presence of ambient light or a long exposure. Below the moon, there are scattered clouds, and the landscape is characterized by silhouettes of trees and vegetation, indicating a suburban or semi-rural setting. There’s a house visible through the trees, and it’s dark with no discernible lights, adding to the nocturnal ambiance. The overall scene is serene and quite typical of a night during a lunar eclipse, with the blood moon being the central feature of interest.

If you saved this against a tiddler pointing to this image you have more than enough keywords to help find it.

[dionisos]

Yes you are perfectly right, I was too quick when I said the content of images wasn’t really searchable. It was an approximation because I thought it would be hard to do, and also I rarely would have a use for it.
But yes, it seems like with the new AI technologies, it is quite doable (and this is impressive).

[TW_Tones]

I just tried this again with another image and also asked for a short name. All we need now is to intergrate this with tiddlywiki

Please describe this picture and generate a suitable short name

The picture shows a patch of green clover leaves covering the ground. Among the mostly three-leaf clovers, there is a distinctive four-leaf clover near the center, which is often considered a symbol of good luck. The clovers have a vibrant green color, and there is some variation in the coloration, with some leaves having lighter green patches. The ground is not visible due to the density of the clovers, and there are some small flowers and other grasses interspersed between the clover leaves.

A suitable short name for this picture could be “Lucky Clover Patch”.

I did not realise but the filename I uploaded had the name “fourleafeclover.jpg” which may have given it a clue.