Tools to Import and Sync directory tree of file links with Tiddlywiki

I’m working on storing directory file links in Tiddlywiki, and keeping it updated by importing and syncing with json file of directory listing. This seems to be working finally, so I will share it and get some feedbacks.

There was an earlier discussion on this topic Directory listing plugin, this solution is using Powershell script.

I’m not conversant with Powershell, just learning it along the way. The Powershell command for listing a directory is Get-childitem. It has command parameters to include/exclude files using file mask etc. Some common Powershell command for directory listing:

  • Get a plain directory file listing:
    Get-ChildItem -path "C:\Downloads\DirFile TEST\" -file -Recurse -Name
    Output:
    Sleepy hippo { [ test ] } .png
    directory secondary\Notfavorites_2_26_23.html

  • Formatted directory output of file attributes :
    Get-ChildItem -path "C:\Downloads\DirFile TEST\" -file -Recurse| ForEach-Object { "File: $($_.fullname) Length: $($_.length)"}
    Output:
    File: C:\Downloads\DirFile TEST\Sleepy hippo { [ test ] } .png Length: 1105366
    File: C:\Downloads\DirFile TEST\directory secondary\Notfavorites_2_26_23.html Length: 505503

In this attachment DirFIles2TW.json (27.5 KB), there is a longer Powershell script to generate a directory listing and output to a Tiddlywiki json file, like this:

[  {    "title":  "(file) C:/Downloads/test Now/IMG_20230326_094935.jpg :20230616054717801:",
        "tags":  "fdir:file",
        "type":  "text/vnd.tiddlywiki",
        "text":  "",
        "fdir.name":  "/C:/Downloads/test%20Now/IMG_20230326_094935.jpg",
        "fdir.status":  "import",
        "fdir.filename":  "IMG_20230326_094935.jpg",
        "fdir.path":  "C:/Downloads/test Now/",
        "fdir.created":  "20230416144608713",
        "fdir.modified":  "20230326014936000",
        "fdir.size":  "3184329"
   } ]

More details is in the Readme tiddler in the attachment.

The file directory json file is fine for one-time import of directory listing into Tiddlywiki . For ongoing updating, I tried writing a sync job using wikitext to update the existing file tiddlers with a more recent directory json file. This gives me some grief with wikitext actions seemingly not executing in order. It was eventually resolved with “tv-action-refresh-policy() always”. Wikitext is weird :sweat_smile:

After each sync, a summary breakdown of file changes is shown, like this:

The sync job is capturing all the file changes I made to the file directory, so it seems to be working. However, it is taking a long time to execute, often with “This page isn’t responding, wait or exit page.” warning, especially when syncing directory with more than a thousand files. It does eventually complete the sync but there is some inefficient wikitext to be resolved. A thousand files is good enough for my purpose though and I have spent far too much time on this, so I will push on with my TW application and revisit this later.

I use wikitext for the sync job instead of Powershell as I hope to eventually adopt Saq’ WebDAV solution to retrieve a similiar file directory json listing from a WebDAV server. That is a more universal solution to access a file directory and the same json output can be processed using wikitext.

Lastly, I came across Tiddlywiki’s built-in tree macro and adapted it as a directory browser for inspecting file tiddlers. Tree macros is a delightful surprise to me.

The Directory Import and Sync tool is provided as-is. I can’t commit to maintain it but I’m happy to answer query and help whenever I can. Your feedback is welcome. Cheers!

3 Likes

If you are using TidGi Desktop, you can use zx-script plugin to read fs and add tiddler to the wiki, just inside the wiki. (see its readme for demos)

I think nowadays, zx-script is a better choice than shell script.

And TidGi now supports click [[file://D:\xxx]] link to open the folder.

1 Like

Thanks for the tip. I have not heard of zx-script, will have a look at it.

TiddlyDesktop and TidGi Desktop are still some way distant in my future. I’m a decade behind the community. I’m using single file TW, still learning and prodding along, hence fiddling with file directory import and such, which was historically done on TW Classic :sweat_smile: I will look into NodeJS app-based TW when I get more comfortable with TW and my needs goes beyond single file TW.