Importing local file links by copying browser's directory listing screen content into Tiddlywiki

Hi all,

This is a sharing of (yet) another way to import local file links ! This one works by copying the screen content from a Web browser’s directory listing screen into Tiddlywiki.

Desktop browser can browse files in local file directories using a URL such as “file:///C:/Downloads/”. The screen content of the browser’s directory listing screen can be copied into Tiddlywiki by selecting the entire screen or some file entries, copy (’‘with Ctrl-C, not drag-and-drop which will copy the files themselves instead of the screen content’’), and paste into an empty region in Tiddlywiki. Tiddlywiki will import the content as “text/html” and “text/plain” files which contain the respective html and text contents of the browser’s web page.

Although the imported “text/html” in TW looks identical to the browser directory screen and the file links even work when you click on it, both file formats are not immediately usable for manipulation in TW. The file entries info needs to be extracted to be useful in TW.

Extracting Chrome’s directory listing from “text/plain” file

The sample wikitext below extracts Chrome-formatted directory list info from an imported “Untitled text/plain 123” tiddler into “filenameList” and “fullpathList” variables:

\function slash.prefix.for.windows() [all[]] :map[<currentTiddler>!prefix[/]then[/]else[]addsuffix<currentTiddler>]
\function line.feed() [charcode[10]]
<$let rawDirList  ={{{[[Untitled text/plain 123]get[text]splitregexp[\r\n]format:titlelist[]join[ ]]}}}
	  filepath    ={{{[enlist<rawDirList>first[]search-replace[Index of ],[]search-replace:g[\],[/]slash.prefix.for.windows[]]}}}
	  filenameList={{{[enlist<rawDirList>butfirst[3]] :map[splitregexp[\t]first[]format:titlelist[]] +[join[ ]]}}}
	  fullpathList={{{[enlist<filenameList>addprefix<filepath>format:titlelist[]] +[join[ ]]}}}
>

The directory pathname on browser’s directory screen is shown only at the top of the screen so you need to copy the screen content from the top down for this to work.

Extracting Chrome’s directory listing from “text/html” file

The sample wikitext below extracts Chrome-formatted directory list info from an imported “Untitled text/html 34” tiddler:

\procedure pattern_chromeTblHeader() <tbody id="tbody">
\procedure pattern_chromeRowSeparator() </td></tr>
\procedure pattern_chromeTableRow() ^.*href="([^"]+?)([^"/]*)".*data-value="([0-9]+)".*data-value="([0-9]+)".*$
\function line.feed() [charcode[10]]
\procedure dir_contents() $1┋$2┋$3┋$4

{{{[[Untitled text/html 34]get[text]splitregexp<pattern_chromeTblHeader>butfirst[]splitregexp<pattern_chromeRowSeparator>butlast[]search-replace:g:regexp<pattern_chromeTableRow>,<dir_contents>format:titlelist[]join<line.feed>]}}}

into this intermediate format I’m using for storing multiple pieces of file info:

( Path ┋Filename┋file size ┋date modified )

/C:/Downloads/┋Filename1.pdf┋2847575┋1685023056 
/C:/Downloads/┋Filename2.png┋1861073┋1686443160 

You can use your own format, such as “$1$2” for full pathname, or “[ext[$2|$1$2]”, by changing the line \procedure dir_contents() $1┋$2┋$3┋$4 accordingly.

Sample app to import Chrome browser’s directory listing

Attached is an example app J.DirDrop.json (9.9 KB)
that uses the wikitext above for “text/html” to import Chrome browser directory listing. This works only on Chrome for now because I didn’t have time to do one for Firefox browser. I find this method to be quite efficient using mouse and keyboard for importing a single directory of file links. You can customise it to do your own processing on imported file links. I didn’t do a thorough test so there might be edge cases where the importing didn’t work. Usual caveats apply regarding using it with care and saving your files.

Have fun !

2 Likes

I have updated the example app DirDropV2.json (12.3 KB)
to work for both Chrome and Firefox browsers. Adapting for Firefox browser seems straightforward as the firefox “text/html” import format is “cleaner” compared to Chrome browser’s. This turned out to be harder than I thought. I ended up using the “text/plain” contents instead of wrestling with it !

For single file TW, I find this to be a relatively easy way to import local file links. It works entirely within the browser.

It’s smoother if you use the local directory links (in cyan in the screenshot above) to open the browser’s directory listing. The example app now allows selection of imported file links for further processing. You will have to write your own button actions to do the processing, whether to create individual file tiddlers for each file link, or to convert the file links into relative links, or whatever.

have fun!

3 Likes

Another update. This version,J.DirDropV3.json (22.3 KB), supports import of additional directory listing formats. The directory listing formats now supported are:

  • Windows plain text directory listing
    This is the “bare” directory listing from the command prompt using “dir /s /b”. You can copy the text file using Windows Explorer or copy the text content of the file listing, and paste into the dropzone. Supports import of sub-directories.
  • File links from Windows Explorer:
    In Windows Explorer, you can multi-select files and directories, “copy as path” and paste (CTRL-V) them into the dropzone.
  • Linux plain text directory listing:
    This is the Linux equivalence of Windows “bare” directory listing using “find /path/to/directory/” command in shell prompt. Supports import of sub-directories.
    For Android mobile, I use “Total Commander” File manger app to create this directory listing, then paste into the new “Text input field” (see below) to import into TW.
  • Android Total Commander 's Directory listing:
    On Chromebook, Google’s default “Files” file manager app does not support copying of file links, unlike Windows Explorer. I use Android “Total Commander” app (running on Chrome OS) to create and import this directory listing list by selecting/finding the files/directories, copy (CTRL-C) and paste (CTRL-V) them into the dropzone. More details in the Readme in the attached json file.
  • URL list:
    An example of a URL list is bookmarked URLs that can be copied from the browser’s Favorites/Bookmark Manager and paste into the dropzone. I have no practical use for this at the moment.
  • Chrome and Firefox Browser’s Directory listing
    This is the browser’s directory listing screens that are already supported and documented previously, included here again for completeness.

"Text input field" for directory import on Android mobile
On Android Chrome browser, there seems to be no way to drop text into Tiddlywiki Dropzone using the touch interface without a keyboard. My clunky workaround is a text input field next to the Dropzone solely to support pasting of directory listing text on Android mobile. It supports the same text input as the Dropzone, except that the Dropzone also accepts files that contain directory listing text. Maybe somebody has a better idea ?

2 Likes

Yet another update. For this minor update TW_DirDrop 3.1.json (34.5 KB), the focus is on import of Bookmarks links, instead of local file links:

1. URL List from Chrome/Firefox Bookmarks Manager

The dropzone now accepts, and then ignores, the presence of folder names within the URL list, as Chrome/Firefox browser uses folders for Bookmarks organization.

Note 1: Importing of bookmarks is useful for retrieving the list of opened tabs in Chrome/Firefox browser. This can be accomplished by “Bookmarking all tabs” (Ctrl-Shift-D on Chrome browser) to a new folder, then in Bookmark Manager (Ctrl-Shift-O on Chrome browser), copying all the newly created bookmarks in the saved folder and pasting into the dropzone (or directly into TW, if you only need a plain URL list).

Note 2: Import of Favorites is NOT supported on Windows Edge browser as Edge doesn’t seem to support direct copying of Favorites to Clipboard.

2. Sync tabs URLs from chrome:history/syncedTabs/

On Chrome browser, the dropzone can capture the list of synchronized tabs on other Chrome devices into TW. Access this page using “chrome:history/syncedTabs/”, or by selecting “Tabs from other devices” on Chrome History page (Ctrl-H), then copy the list INCLUDING the “sync device” at the head of each Synced Tabs list, and paste into the dropzone.

3. Import of HTML and Text contents from clipboard

Imports from clipboard into TW may contain “text/html” and/or “text/plain” contents, among others. The dropzone now internally checks both the HTML and Text contents of import for contents it can accept. Compared to the Text contents, the HTML contents often contain additional embedded information, such as embedded links or descriptions. For example, synced URLs are found only in the imported HTML contents of Chrome Synced tabs page.

Firefox browser has a constraint: imported TEXT contents may displace the HTML contents if both are available. As a workaround, the dropzone now has a selection list to import “HTML content only” for better support on Firefox browser.

4. On Android device, use Text input field instead of Dropzone

On non-Android device, Dropzone will be used for capturing Directory Listings. But on Android device, the Text input field is used instead. Determination of Android or non-Android devices is based on presence of Android style prefix ("/storage/emulated/0/") in TW URL.

2 Likes

This is another update of J.DirDrop. Earlier versions of J.DirDrop support mass import of file links and URL links into TW. This version focuses on file links and goes further to create file tiddlers using imported file links, and also synchronize them with existing file tiddlers. This maintains a set of file link tiddlers that corresponds to actual files in file directories, and is useful for keeping notes linked to the files.

“J.DirDrop V4” JSON is attached here J.DirDropV4.json (52.4 KB). An online version is available at https://j-dirdrop.tiddlyhost.com/ for testing. Details of the changes can be found in J.DirDrop V4 Readme at “https://j-dirdrop.tiddlyhost.com/##J.DirDrop_V4”.

Some highlights:

  • Supports multiple ways to import file links:

    • From a single directory from Chrome/Firefox/Edge browser’s file directory screen
    • From a single directory from Windows explorer using “copy as path”
    • From one/multiple directories in a directory listing file with “.txt” file extension, such as from the captured output of dir /s /b \file\directory\path command in Windows, copy and paste as a file onto the dropzone
    • From one/multiple directories copy and paste as text, such as from the captured output of dir /s /b \file\directory\path command in Windows.
  • File sync
    File synchronization will create file tiddlers only for new file links. Existing file tiddlers will not be overwritten by new file tiddlers. For my purpose, these file tiddlers are used directly to store notes on my picture files, ebook files, pdf files etc.

  • Speed of importing and sync
    A pleasant surprise, for me, is the speed and ease with which more than 10K file links in an entire directory tree can be imported as file tiddlers and sync in a few seconds using wikitext implementation, even on a mid-range Android phone.

  • Change filepath prefix
    A search and replace can be performed on file links during import to change from absolute to relative file path, or from one OS file system where the file links are captured to another where the file links are referenced.

Some caveats:

  • In case this is not clear, J.DirDrop only processes file links. The actual files in the directories are untouched and not changed in any way.

  • J.DirDrop V4 imports file links and creates basic file tiddlers containing the file path in the title. You will need your own TW app to access, update and manipulate these file tiddlers.

  • Local file links don’t work on Tiddlyhost. While copying and importing local file links to J.DirDrop can be tested on Tiddlyhost, local files are inaccessible from Tiddlyhost web site. To test local file links, download and test J.DirDrop locally on the machine.

Edit: Added a gif video to demonstrate copying of file links from browser directory screen to create file link tiddlers J.DirDropV4 capturing Chrome browser directory

1 Like

Looks really useful. Good stuff.