# TiddlyDesktop Overhaul
This branch (`tiddlydesktop-overhaul`) is a major fe…ature and infrastructure update for TiddlyDesktop. It upgrades the runtime, dramatically improves perceived startup time, adds four new end-user features to the wiki list (plugin management, wiki format conversion, wiki-folder creation, dark mode), fixes cross-browser drag-and-drop imports, and ships an entirely new bundled plugin that brings **real-time collaborative editing** (Yjs/CodeMirror 6 over an authenticated WebSocket relay, with **end-to-end encryption**, an encrypted LAN fast path, tiddler sharing, and chat) to TiddlyDesktop.
The complete diff (including all new files) is in `full-diff.patch` (~7,300 lines).
---
## 1. Platform / build changes
| Change | Files |
|---|---|
| NW.js upgraded **0.108.0 → 0.112.0** | `bld.sh`, `download-nwjs.sh`, `nwjs-version.txt` |
| App version bumped **0.0.22 → 0.0.23** | `source/package.json` |
| New npm dependency: **`ws` ^8.18.0** (Node WebSocket client/server, used by the collab transport) | `package.json` |
| Build script copies `ws` into `source/node_modules/` so NW.js can `require()` it at runtime | `bld.sh` |
| Build script copies the new `codemirror-6-collab-nwjs` plugin into the bundled TiddlyWiki plugins | `bld.sh` |
## 2. Faster, smoother startup (loading splash + deferred boot)
Previously the synchronous TiddlyWiki boot blocked the main process before any window appeared. Now:
- **`source/js/main.js`** — If no wiki path is passed on the command line, the wiki list window is opened *immediately*, before TiddlyWiki boots. The heavy `$tw.boot.boot()` is deferred to the next tick (`setTimeout(..., 0)`) so the splash window can paint first. After boot completes, all windows opened pre-boot are rendered via the new `tryRender()` mechanism, and `$tw.desktop.bootComplete` is set. Command-line argument processing is unchanged in behaviour, just relocated inside the deferred boot.
- **`source/html/backstage-tiddler-window.html`** — New inline-styled loading splash (`td-loading-splash`) with a CSS spinner animation and "Loading TiddlyDesktop…" text, shown until real content renders.
- **`source/js/backstage-window.js`** — Window rendering is split into two phases tracked by `windowLoaded` / `rendered` flags:
- `onloaded()` now only exposes `$tw`, shows/focuses the window, and calls `tryRender()`.
- New idempotent **`tryRender()`** renders the TiddlyWiki content only once both the window has loaded *and* `$tw` has finished booting (checked by a new `isBootComplete()` helper). It then installs link trapping, popup handling, devtools (F12), the menu bar, renders the content, and removes the splash.
- `onclose()` is now guarded so closing a window *before* boot completes (no change listener registered yet) no longer throws.
- New `$tw.desktop.pluginHooks` array in `main.js` lets plugins register hooks that run when wiki windows finish loading.
- `$tw.desktop.utils` now also exposes Node's `ws`, `https`, and `http` modules (in both `main.js` and `wiki-folder-main.js`) for use by plugins such as the collab transport.
## 3. Wiki list: convert between single-file and folder wikis
Users can now convert any listed wiki between the two storage formats, non-destructively:
- **`plugins/tiddlydesktop/WikiListRow.tid`** — Each wiki row gains a context-sensitive convert button: "**to folder**" on `wikifile://` entries, "**to file**" on `wikifolder://` entries (sends new `tiddlydesktop-convert-wiki` message).
- **`plugins/tiddlydesktop/modules/startup/handlers.js`** — New `tiddlydesktop-convert-wiki` handler: computes a sensible destination path (`wiki.html` for folder→file; strips the extension or appends `-folder` for file→folder), asks for confirmation (showing source/destination and noting the original is kept), then delegates to `convertWiki()` and alerts on failure.
- **`source/js/window-list.js`** — New **`WindowList.prototype.convertWiki(sourceUrl, destPath, callback)`**: forks the bundled `tiddlywiki.js` as a child process —
- folder → file: `tiddlywiki <folder> --rendertiddler $:/core/save/all <dest> text/plain`
- file → folder: `tiddlywiki --load <file> --savewikifolder <dest>`
- On success, the old wiki-list entry tiddler is replaced with a new entry pointing at the converted wiki (correct `wikifile`/`wikifolder` tag). stderr is captured and surfaced in the error message on non-zero exit.
## 4. Wiki list: create a new wiki *folder*
- **`plugins/tiddlydesktop/toolbar/CreateWikiFolder.tid`** (new) — "Create new wiki folder" toolbar button using a directory save dialog (`$browse nwdirectory nwsaveas`), added to the main toolbar list in `toolbar/toolbar.tid`.
- **`plugins/tiddlydesktop/modules/startup/handlers.js`** — New `tiddlydesktop-create-wiki-folder` handler.
- **`source/js/window-list.js`** — New **`createWikiFolderAtPath(dest)`**: refuses if the path exists, otherwise scaffolds a standard Node wiki folder (`tiddlywiki.info` with `tiddlywiki/filesystem` + `tiddlywiki/server` plugins and vanilla/snowwhite themes, empty `tiddlers/` directory) and immediately opens it as a `wikifolder://` window.
## 5. Wiki list: Plugin Manager ("plugins" button + chooser dialog)
A full plugin install/remove UI for any listed wiki, without opening it:
- **`plugins/tiddlydesktop/WikiListRow.tid`** — New "**plugins**" button per wiki row (sends `tiddlydesktop-open-plugin-chooser`).
- **`plugins/tiddlydesktop/PluginChooser.tid`** (new) — Modal overlay dialog (transcluded into `WikiListWindow.tid`) showing the target wiki path with a file/folder icon, a live search filter, the scrollable plugin list with checkboxes, plugin titles/descriptions/versions/type badges, an empty-state message, a status line, and Apply/Cancel buttons. Shows a warning banner if the target wiki is currently open (changes are blocked until it's closed).
- **`plugins/tiddlydesktop/modules/startup/plugin-manager.js`** (new, ~370 lines) — Startup module implementing the logic:
- **Enumeration**: scans all plugin search paths returned by `$tw.getLibraryItemSearchPaths()` — i.e. the plugins bundled with the embedded TiddlyWiki *plus* any paths in `TIDDLYWIKI_PLUGIN_PATH` — reading each `plugin.info` for title, description, version, and plugin-type. `$:/core` and the TiddlyDesktop plugin itself are hidden.
- **Installed detection**: for single-file wikis, parses the `tiddlywiki-tiddler-store` JSON out of the HTML; for folder wikis, reads the `plugins` array from `tiddlywiki.info`.
- **Apply (single-file wikis)**: makes a timestamped backup first (honouring the `$:/TiddlyDesktop/BackupPath` template), then rewrites the tiddler store JSON — removing deselected plugins and injecting newly selected ones via TW's own `$tw.loadPluginFolder()`. Carefully escapes `</script>` inside the JSON and uses a function replacer so `$` characters in plugin code aren't mangled by `String.replace`.
- **Apply (folder wikis)**: edits the `plugins` array in `tiddlywiki.info`.
- **Safety rails**: `$:/core` can never be removed; folder wikis additionally protect `tiddlywiki/tiddlyweb` and `tiddlywiki/filesystem`; applying is refused while the wiki's window is open.
- All dialog state lives in `$:/temp/TiddlyDesktop/PluginChooser/*` tiddlers and is fully cleaned up on close.
- **`plugins/tiddlydesktop/styles/wikilist.tid`** — ~230 new lines of CSS for the chooser overlay/dialog/rows/badges/status and the new convert/plugins buttons, all palette-aware.
## 6. Cross-browser drag-and-drop import fix
- **`source/js/utils/dragdrop.js`** (new, ~190 lines) — Works around Chromium's cross-application drag-data sanitiser on Linux, which strips the `text/vnd.tiddler` MIME type (and most others) when a drag crosses app boundaries, so tiddlers dragged from e.g. Firefox previously arrived as plain HTML and lost their fields. The interceptor:
- probes a list of MIME types (`text/uri-list`, `text/x-moz-url`, `text/vnd.tiddler`, `text/html`, JSON variants, …) even when not enumerated in `dataTransfer.types`;
- detects and decodes Chromium's mangled UTF-16LE-as-Latin-1 payloads;
- extracts `data:text/vnd.tiddler,...` URIs whether they arrive as a bare URI line or embedded in HTML markup, plus raw JSON payloads (including a self-identifying `{__type, fields}` envelope);
- skips drops onto text inputs/contenteditable and file drops;
- dispatches `tm-import-tiddlers` on the wiki's navigator widget (found by walking *down* the widget tree, since `rootWidget` has no parent to bubble to), so the standard `$:/Import` preview opens with full tiddler fields intact.
- Installed in **both** window types: on the iframe document of wiki-file windows (`source/js/wiki-file-window.js`) and on the window document of wiki-folder windows (`source/js/wiki-folder-main.js`).
## 7. Dark mode support
- **`plugins/tiddlydesktop/styles/dark-mode.tid`** (new) — Sets the CSS `color-scheme` property on `:root` from the active palette's `color-scheme` field, so native UI (scrollbars, form controls, etc.) follows dark palettes.
## 8. NW.js bridges for sandboxed (nwdisable) iframes
Single-file wikis run inside an `nwdisable` iframe where Node.js is stripped and network I/O initiated from the iframe's call stack is suppressed. **`source/js/wiki-file-window.js`** now injects two parent-owned bridges after the iframe loads (used by the collab plugin, available to any plugin):
- **HTTP queue bridge** — the parent runs a `setInterval` that drains `_nwjsHttpQueue` (requests pushed by the iframe as plain objects) using Node `http`/`https` entirely from the parent's event loop, writing JSON results into `_nwjsHttpResults` which the iframe polls. No cross-context function calls during async operations. Signals readiness via `_nwjsHttpQueueReady()`.
- **WebSocket bridge** — same queue-drain pattern: the iframe pushes `create`/`send`/`terminate` commands to `_nwjsWsCmdQueue`; the parent owns the `ws` sockets and pushes `open`/`message`/`close`/`error` events to `_nwjsWsEventQueue`, dispatched to the iframe's `_nwjsWsOnEvent`. Exposes `_nwjsWsCreate/_nwjsWsSend/_nwjsWsTerminate`, signals readiness via `_nwjsWsBridgeReady()`.
- **`_nwjsOpenExternal(url)`** — opens URLs in the system browser via `nw.Shell` (used for OAuth).
- All timers and sockets are cleaned up on window close; any registered `$tw.desktop.pluginHooks` are invoked when the iframe finishes loading.
## 9. New plugin: `codemirror-6-collab-nwjs` — real-time collaboration
A brand-new bundled plugin (`$:/plugins/tiddlywiki/codemirror-6-collab-nwjs`, ~6,100 lines) providing real-time collaborative editing for TiddlyDesktop. Works in both wiki-folder windows (direct Node access) and single-file wiki windows (via the §8 bridges).
### 9.1 Collaborative editing engine (`files/collab.js` + `files/lib/yjs-collab.js`)
- Bundles **Yjs + y-codemirror.next + y-protocols/awareness** (esbuild bundle) as a TW library module.
- A `codemirror6-plugin` module that wires CodeMirror 6 editors to Y.Text documents: character-level real-time sync, remote selections/carets rendered with name badges, and per-user deterministic colours (12-colour palette hashed from the username, overridable in settings).
- Implements the CM6 ↔ Y.Text sync ViewPlugin directly against the editor core's own `ViewPlugin` class (instead of `yCollab`) to avoid silent module-identity mismatches.
- Handles awareness updates (base64-encoded binary over the transport), lifecycle/teardown per tiddler, live username updates, and surfacing edit conflicts via the ConflictDialog.
### 9.2 Transport (`modules/startup/transport.js`)
Two simultaneous delivery channels with message-id deduplication (each message processed exactly once, whichever channel wins):
- **Relay channel** — `wss://` connection to a user-configured relay server with `Authorization: Bearer` token; carries control messages and serves as the data fallback. Auto-reconnects with backoff; reconnects automatically when relay URL / room code / token config changes. Never auto-connects on load — connection requires an explicit user action (Connect button or applying an invite).
- **LAN channel** — direct `ws://` between peers on the same network (ports 45700–45710), encrypted with **ChaCha20-Poly1305** using a session key derived via **X25519 ECDH → HKDF-SHA256**. The X25519 public keys are announced through the relay, but the announcement is itself end-to-end encrypted (see below) and the room content key is folded into the session-key HKDF — so in strong mode a malicious relay cannot MITM the key exchange. Shown as a "LAN ⚡" badge in the UI.
- Device identity: persistent device ID plus an ephemeral per-window session suffix, so two windows of the same wiki act as distinct peers. Publishes connection status, E2E strength, and room-member presence (including "who is editing what") to `$:/temp/collab/*` tiddlers, and exposes the `window.TiddlyDesktop.collab` API consumed by the editor engine, sharing, and chat modules.
### 9.2.1 End-to-end encryption of the relay channel
Every peer-to-peer message routed through the relay (collab updates, awareness, presence, tiddler sharing, chat, LAN announcements) is encrypted client-side with **AES-256-GCM via WebCrypto** before it leaves the process; the relay only ever sees an opaque `{type:"enc", deviceId, iv, ct}` envelope, with the sender's `deviceId` bound as authenticated data. WebCrypto is used (rather than Node's `crypto`) so the same path works inside the sandboxed nwdisable iframe of single-file wikis, where Node is unavailable. The content key is derived with **HKDF-SHA256**:
- **Room token set → strong E2E.** The token is **never transmitted to the relay** (the old `X-Room-Token` header and the `roomToken`/display-name fields in the `join` message were removed), so the relay cannot derive the key. Content is confidential even against a fully malicious relay operator.
- **No token → still always encrypted**, with the key derived from the room code. This protects against passive network eavesdroppers and other peers, but since the relay knows the room code it can derive this key. The UI labels this case honestly (`🔓 encrypted (room code)`) rather than overclaiming.
Hardening: **anti-downgrade** (once connected, cleartext peer messages are dropped; only relay-origin control frames may be plaintext); refusal to connect at all if WebCrypto is unavailable (never falls back to plaintext); async encryption is serialised through send/receive promise chains to preserve wire order. **Relay compatibility note:** because the token is no longer sent, relay room access must rest on the OAuth identity + room code; a relay that hard-requires `X-Room-Token` will reject these clients. Since content is E2E encrypted, an unauthorized joiner only ever receives ciphertext.
### 9.3 OAuth sign-in (`modules/startup/oauth.js`)
- Sign-in with **GitHub, GitLab, or any OIDC provider**, discovered dynamically from the relay's `/api/auth/providers` endpoint.
- Flow: open the system browser at the provider's authorize URL (with the relay's server-side callback as `redirect_uri`), then poll the relay's `/api/auth/result` until the token arrives (5-minute timeout) — no deep-link handling needed on the desktop.
- Stores token/provider/username/user-id in config tiddlers (token changes trigger an automatic transport reconnect), re-verifies the token on every relay connection, and signs out automatically on 401.
- **Invite codes**: generate a `collab1:`-prefixed base64 code bundling relay URL + room code + room token; pasting one (or a bare room code) configures everything and connects immediately. Leave-room support included. **Secure by default**: if no room token is set when an invite is generated, a 256-bit cryptographically random one is minted automatically and carried in the invite (shared out-of-band, never via the relay), so invited collaborators get true end-to-end encryption without any extra steps.
### 9.4 Tiddler sharing (`modules/startup/sharing.js`)
A room-level share/subscribe protocol on top of the transport:
- Any member can **share** a tiddler; the first peer to share a title owns the claim (duplicate claims are dropped until the owner unshares). Drafts are never broadcast.
- Other members see shared tiddlers in the sidebar and click **Get** to fetch the current content and subscribe to future updates (propagated via a `$tw.wiki` change listener). Renames by the owner follow automatically on subscribers.
- **Catch-up conflict resolution** when reconnecting: newer remote wins silently, newer local is pushed to the room, and a same-timestamp text divergence opens the ConflictDialog (use shared / keep local).
- Owned and subscribed titles persist across wiki saves/reloads in config tiddlers; manifests are exchanged on join so late arrivals see the full share list.
### 9.5 Chat (`modules/startup/chat.js`)
- Lightweight room chat over the same transport: transient messages (cleared each new session), timestamps, sender names, Enter-to-send, and an unread-count badge on the chat toggle that clears when the panel opens.
### 9.6 UI (`ui/*.tid`, `styles/collab-ui.tid`)
- **Collab sidebar tab** — connection status dot + LAN badge, error notices, member list with live "editing …" indicators, conflict notices, the shared-tiddler list with Get/Got buttons and "yours" badges, room panel (Connect/Disconnect/Invite/Leave, invite-code box with copy-to-clipboard, join-by-code input), and a collapsible settings panel (relay URL, room code, room token, display name, colour picker, OAuth sign-in/out).
- **Settings tiddler** — full settings page with a "How it works" walkthrough and an accurate **Encryption** section explaining the strong (token) vs room-code levels.
- **E2E status badges** — sidebar and status bar show `🔒 end-to-end encrypted` when a room token is set, or `🔓 encrypted (room code) — set a room token for E2E` otherwise, driven by the new `e2e` field on the status tiddler.
- **Status bar** (page template) — room, connection dot, LAN badge, E2E badge, and signed-in username.
- **Chat panel** (page template) — floating toggle + panel.
- **Share buttons** in both the view toolbar and edit toolbar (system tiddlers excluded; edit-toolbar version resolves `draft.of`).
- **"Shared" banner** in the edit template showing whether the open tiddler is yours or shared by a named peer.
- **Page-controls Collab button** with a live status dot.
- **ConflictDialog** — modal side-by-side comparison of local vs. shared content with explicit resolution buttons.
- **Notifications** for missing relay URL / room code, and ~860 lines of palette-aware CSS for all of the above.
## 10. Documentation & licensing
- **`readme.md`** — new **Troubleshooting** section, including a Linux/Wayland note: run `nw --ozone-platform=x11` (or `OZONE_PLATFORM=x11`) to fix Wayland glitches such as broken drag & drop, odd window frames, and misplaced dialogs; plus notes on Windows UNC shares and resetting configuration via `--user-data-dir`.
- **`LICENSE`** (new) — TiddlyDesktop's BSD 3-Clause license plus a **Third-party components** section reproducing the license notices for TiddlyWiki5 (BSD), nw.js (MIT, noting its bundled Chromium/Node notices), the `ws` module (MIT), and the bundled Yjs libraries (MIT).
## 11. Minor cleanups
- `plugins/tiddlydesktop/styles/buttons.tid` — indentation normalised (spaces → tabs).
- `source/js/wiki-file-window.js` — stray unindented `console.log` fixed; trailing whitespace removed.
---
## File inventory
**Modified (18):** `bld.sh`, `download-nwjs.sh`, `nwjs-version.txt`, `package.json`, `source/package.json`, `readme.md`, `plugins/tiddlydesktop/{WikiListRow,WikiListWindow}.tid`, `plugins/tiddlydesktop/modules/startup/handlers.js`, `plugins/tiddlydesktop/styles/{buttons,wikilist}.tid`, `plugins/tiddlydesktop/toolbar/toolbar.tid`, `source/html/backstage-tiddler-window.html`, `source/js/{backstage-window,main,wiki-file-window,wiki-folder-main,window-list}.js`
(The collab plugin's `transport.js`, `oauth.js`, and the `Settings`/`Sidebar`/`StatusBar` UI tiddlers + `collab-ui.tid` stylesheet additionally carry the end-to-end encryption changes described in §9.2.1.)
**Added (28):** `LICENSE`, `plugins/tiddlydesktop/PluginChooser.tid`, `plugins/tiddlydesktop/modules/startup/plugin-manager.js`, `plugins/tiddlydesktop/styles/dark-mode.tid`, `plugins/tiddlydesktop/toolbar/CreateWikiFolder.tid`, `source/js/utils/dragdrop.js`, and the 22 files of `plugins/codemirror-6-collab-nwjs/` (plugin.info, config, transport/oauth/sharing/chat startup modules, collab engine + bundled Yjs library, 12 UI tiddlers, stylesheet).