Get a list of files inside a directory in a TiddlyWiki Node.js edition?

In TiddlyWiki Node.js edition.

I created a symlink “files” to ~/Twassets in my ~/wikis/notes wiki so it’s ~wikis/notes/files.

In ~/TWassets I created a symlink to a directory named /home/wikster/Documents/Learn/Library/Manuals/ and named it Link-to-Manuals.

I tried [ext[Manuals|/files/Link-to-Manuals]] but it does not work as I would want which is to list the contents of that directory.

Please note that the symlink to Manuals in ~TWassets works in Krusader, that is it opens the /home/wikster/Twassets/Link-to-Manuals directory.

Final objective: I can link to individual files successfully inside the /home/wikster/Documents/Learn/Library/Manuals/ directory, but I want a way to show a list of files inside this directory.

That will never happen with a browser, for privacy reasons. Browsers do not allow JavaScript code to read directory info from the users file system.

Thank you for the information. As I wrote, I can link to individual files, but not a directory in order to see the contents. Also I could misremember it, but I somehow recall that this was possible with TiddlyWiki Classic.

While TWClassic did have some ability to read a local directory, the issue isn’t TW5 vs TWClassic… it’s the browsers.

They used to allow filesystem access after prompting the user for permission, but that ability was completely purged from all modern browser applications due to “security concerns”.

The problem is that, with filesystem access, 3rd-party javascript code could potentially write to your local drive (or at least see the directory structure and file names), and thus look for vulnerabilities and potential “virus vectors” and report that information back to a remote server.

Browsers do permit limited local filesystem READ access that requires user interaction to select a file, and then it only provides the selected filename and a “blob” of file contents, but not the directory information. TiddlyWiki5 uses this “browse” mechanism to implement the “import” button in the Sidebar>Tools tab.

Modern browsers also allow limited local filesystem WRITE access via the “HTML5 download” mechanism which either defaults to the configured “Downloads” folder or requires user interaction to select a target directory and filename. TiddlyWiki5 uses this download process as the default “HTML5 download saver” and for writing selected tiddler content (as JSON, HTML, CSV, etc.) via the “export tiddler” menu item.

Even so, there are some clever techniques to DO enable some limited forms of direct file I/O. Most of these rely upon either browser extensions or external applications that you must explicitly install and configure to permit TiddlyWiki to save changes to your local TiddlyWiki files without requiring bothersome user interactions.

Documentation for many of these extensions and applications can be found by using the checkboxes in the GettingStarted tiddler.

-e

1 Like

I have a plugin for TiddlyWiki on node.js that adds the ability to display directory listings amongst other things, that I am hoping to get get cleaned up, documented and published this winter.

3 Likes

Thank you for the information. It’s interesting to see how the World has changed over the years!

I “consulted” Microsoft Copilot and it gave me a solution including using an index.html file inside the directory whose content I need to see.

It’s a script which I have to make executable and run inside Terminal.

#!/usr/bin/env bash
# update-manuals-index.sh
# Idempotent generator for Manuals index.html with change reporting.
# Usage: update-manuals-index.sh [--manuals PATH] [--name NAME] [--preview]
set -euo pipefail

MANUALS_LINK="${MANUALS_LINK:-$HOME/Twassets/Link-to-Manuals}"
INDEX_NAME="index.html"
PREVIEW=false

while [[ $# -gt 0 ]]; do
  case "$1" in
    --manuals) MANUALS_LINK="$2"; shift 2 ;;
    --name) INDEX_NAME="$2"; shift 2 ;;
    --preview) PREVIEW=true; shift ;;
    -h|--help) cat <<EOF
update-manuals-index.sh  Create or update index.html inside Manuals target.

Options:
  --manuals PATH    Path to the symlink or directory (default: $HOME/Twassets/Link-to-Manuals)
  --name NAME       Filename to create/update inside target (default: index.html)
  --preview         Print generated HTML to stdout and show change summary without writing file
EOF
  exit 0 ;;
    *) echo "Unknown arg: $1"; exit 1 ;;
  esac
done

# Resolve target dir (follow symlink)
if [ -L "$MANUALS_LINK" ] || [ -e "$MANUALS_LINK" ]; then
  TARGET_DIR="$(readlink -f -- "$MANUALS_LINK")"
else
  TARGET_DIR="$MANUALS_LINK"
fi

if [ ! -d "$TARGET_DIR" ]; then
  echo "❌ Manuals target not found or not a directory: $TARGET_DIR"
  exit 1
fi

# Helpers
url_escape() {
  # single argument required
  python3 - <<'PY'
import urllib.parse,sys
print(urllib.parse.quote(sys.argv[1]))
PY
}

# Produce new listing (non-recursive) as sorted newline list (skip hidden, skip index)
NEW_LIST_TMP="$(mktemp)"
trap 'rm -f "$NEW_LIST_TMP"' EXIT

# Use a safe loop to handle arbitrary filenames including spaces and newlines
(
  cd -- "$TARGET_DIR"
  find . -maxdepth 1 -mindepth 1 -printf '%P\n' 2>/dev/null | \
    while IFS= read -r f; do
      [ "$f" = "$INDEX_NAME" ] && continue
      [ "${f:0:1}" = "." ] && continue
      if [ -d "$f" ]; then
        if [ -f "$f/$INDEX_NAME" ]; then
          printf '%s\n' "$f/$INDEX_NAME"
        fi
      else
        printf '%s\n' "$f"
      fi
    done
) | sort > "$NEW_LIST_TMP"

# Read current list from existing index.html if present
CUR_LIST_TMP="$(mktemp)"
if [ -f "$TARGET_DIR/$INDEX_NAME" ]; then
  # extract href values (best-effort), remove leading ./ or /
  grep -oP 'href="\K[^"]+' "$TARGET_DIR/$INDEX_NAME" 2>/dev/null | sed 's#^\./##;s#^/##' | sort > "$CUR_LIST_TMP" || true
else
  : > "$CUR_LIST_TMP"
fi

# Determine added and removed
ADDED="$(comm -13 "$CUR_LIST_TMP" "$NEW_LIST_TMP" | sed '/^$/d' || true)"
REMOVED="$(comm -23 "$CUR_LIST_TMP" "$NEW_LIST_TMP" | sed '/^$/d' || true)"

# Build new index.html into a temp file (safe write)
TMP_INDEX="$(mktemp)"
cat > "$TMP_INDEX" <<'HTML_HEAD'
<!doctype html>
<meta charset="utf-8">
<title>Manuals Index</title>
<style>
body { font-family: system-ui, -apple-system, "Segoe UI", Roboto, "Helvetica Neue", Arial; margin: 1.5rem; color: #111; }
h1 { font-size: 1.25rem; margin-bottom: .5rem; }
ul { line-height: 1.6; padding-left: 1.1rem; }
li { margin: .2rem 0; }
.small { color: #666; font-size: .9rem; margin-bottom: .8rem; }
</style>
<body>
<h1>Manuals</h1>
<p class="small">Auto-generated on:
HTML_HEAD

date -u +"%Y-%m-%d %H:%M UTC" >> "$TMP_INDEX"
cat >> "$TMP_INDEX" <<'HTML_MID'
</p>
<ul>
HTML_MID

# Append file links from NEW_LIST_TMP (URL-escape each path safely)
while IFS= read -r entry; do
  [ -z "$entry" ] && continue
  # Use python -c with quoted arg to avoid IndexError and handle arbitrary names
  esc="$(python3 -c "import urllib.parse,sys;print(urllib.parse.quote(sys.argv[1]))" "$entry")"
  printf '  <li><a href="%s">%s</a></li>\n' "$esc" "$(printf '%s' "$entry" | sed 's/&/\&amp;/g; s/</\&lt;/g; s/>/\&gt;/g')" >> "$TMP_INDEX"
done < "$NEW_LIST_TMP"

cat >> "$TMP_INDEX" <<'HTML_FOOT'
</ul>
</body>
</html>
HTML_FOOT

# If preview mode: print generated HTML and show change summary
if $PREVIEW; then
  echo "=== Generated index (preview) ==="
  cat "$TMP_INDEX"
  echo
  echo "=== Change summary ==="
  if [ -z "$ADDED" ] && [ -z "$REMOVED" ]; then
    echo "ℹ️ No changes would be made to $TARGET_DIR/$INDEX_NAME"
  else
    if [ -n "$ADDED" ]; then
      echo "➕ Files to be added:"
      echo "$ADDED" | sed 's/^/   • /'
    fi
    if [ -n "$REMOVED" ]; then
      echo "➖ Files to be removed:"
      echo "$REMOVED" | sed 's/^/   • /'
    fi
  fi
  rm -f "$TMP_INDEX"
  exit 0
fi

# Write only if different
DEST="$TARGET_DIR/$INDEX_NAME"
if [ -f "$DEST" ]; then
  if cmp -s "$TMP_INDEX" "$DEST"; then
    echo "ℹ️ index.html is already up to date at $DEST"
    if [ -z "$ADDED" ] && [ -z "$REMOVED" ]; then
      echo "ℹ️ No files would be added or removed."
    else
      if [ -n "$ADDED" ]; then
        echo "➕ Files that would be added:"
        echo "$ADDED" | sed 's/^/   • /'
      fi
      if [ -n "$REMOVED" ]; then
        echo "➖ Files that would be removed:"
        echo "$REMOVED" | sed 's/^/   • /'
      fi
    fi
    rm -f "$TMP_INDEX"
    exit 0
  fi
fi

mv "$TMP_INDEX" "$DEST"
trap - EXIT
echo "✅ index.html updated at $DEST"
if [ -n "$ADDED" ]; then
  echo "➕ Added:"
  echo "$ADDED" | sed 's/^/   • /'
fi
if [ -n "$REMOVED" ]; then
  echo "➖ Removed:"
  echo "$REMOVED" | sed 's/^/   • /'
fi

Preview (no write): ~/scripts/update-manuals-index.sh --preview --manuals /home/wikster/Documents/Learn/Library/Manuals

Update now: ~/scripts/update-manuals-index.sh --manuals /home/wikster/Documents/Learn/Library/Manuals

It’s a convenient solution as the directory’s content doesn’t change frequently and I’ll use it until something better becomes available.

Thank you for the heads up. I look forward to seeing it!