@bobfrankston/rmfmail 1.0.665 → 1.0.667

package/TODO.md

@@ -12,11 +12,11 @@ These items don't need user input — source-only changes, compile-verified befo
 
 | Order | Item | Scope | Status |
 |---|---|---|---|
-| **1** | **C118 — STATUS-before-SELECT in syncFolder** | S | Issue `STATUS folder (UIDNEXT)` before SELECT. If `UIDNEXT - 1 <= local highestUid` skip SELECT and FETCH entirely — folder is up to date. Cuts SELECT calls by ~99% on idle folders. Avoids the bobma/Sent SELECT-wedge case Bob hit (server is fine, but SELECT on idle huge folder + dead-socket-not-detected = 300s timeout). `mailx-imap/index.ts:909`. |
+| ~~1~~ | ~~**C118 — STATUS-before-SELECT in syncFolder**~~ — **DONE** (already implemented at `mailx-imap/index.ts:1079`; TODO entry was stale). | S | |
 | **2** | **C119 — Lazy folder sync** | M | Don't sync all 96 bobma folders on every periodic pass. Only sync (a) folders the user has expanded in the tree, (b) special-use folders (INBOX/Sent/Drafts/Trash), (c) folders touched by recent local actions. Other folders sync on-demand when user clicks them. Closes most of the Thunderbird-vs-mailx speed gap. Touches `mailx-imap/index.ts` syncAll loop. |
 | **3** | **C121 — Counting timer for server connect** | S | Visible elapsed-time counter while the connection is being established, so Bob can tell "DNS slow", "TCP handshake stuck", "TLS hung", "IMAP greeting missing", or "LOGIN slow" apart at a glance. Phases: `dns → tcp connect → tls handshake → imap greeting → LOGIN ack`. Each phase emits a start event with `phase` + `accountId`; client renders `Connecting to <host> (<phase>, 3.2s)` in the status bar with a 100 ms ticking elapsed counter. On success or failure, drop the indicator and log per-phase durations. Touches `node-tcp-transport.connect()` + `iflow-direct.imap-native.ts:connect`/`login` + a new `imapManager.emit("connectProgress", ...)` chain. Pairs naturally with C120 — same `connId` ID space so the timer indicator can name the connection that later times out. |
 | **4** | **C122 — Recent-unread count overlay on taskbar icon** | S | Show recent-unread count as a numeric overlay on the running mailx taskbar icon (Win11 style: small badge in the lower-right corner of the icon). `setOverlayIcon` is already wired (used as a placeholder); needs (a) a number-rendered overlay bitmap (1 / 2 / … / 9+ glyphs at 16×16 or via `DrawText` to a memory DC) and (b) hook into the unread-counter event chain so the badge updates when `folderCountsChanged` or `idle` events arrive. Distinct from S65 — that one is "badge persists when mailx isn't running" and is blocked on tray-process / Win11 Notification Center integration; this one is the *while-running* number. |
-| **5** | **C126 — Cold-start latency (~28s observed)** | M | Bob 2026-05-09 00:10:10 → 00:10:39: 28 s from `Starting rmfmail service` to first `getAccounts` IPC. WebView2 startup + JS module-graph load is sequential and blocks first render. Three concrete fixes: (a) pre-render the last-known folder-tree + first inbox page into index.html as static HTML so the page paints immediately on load; (b) bundle client modules into one file (or 2-3 priority groups) so one HTTP fetch loads the app instead of the cascade of imports from `msger.localhost`; (c) replace the placeholder `await new Promise(r => setTimeout(r, 500))` "brief pause for WebView2 init" in `bin/mailx.ts:1588` with a real "WebView ready" event from msger so the daemon doesn't fire IMAP+OAuth work until after the WebView's first paint (so it doesn't compete for stdin during early IPC handshake). (a) is biggest win — turns "28s blank screen" into "instant cached view, freshens in background". |
+| ~~5~~ | ~~**C126 — Cold-start latency (~28s observed)**~~ — **MOSTLY DONE** (2026-05-11). Root cause IDed from boot log: 43 s of cascading ES module imports through msger's custom protocol IPC (one roundtrip per file). Fixes shipped: (a) bundling via esbuild (`bin/build-bundles.mjs`) collapses the cascade to one fetch per entry point — `client/app.bundle.js` 345 kb, `client/compose/compose.bundle.js` 99 kb; (b) boot-snapshot hydration — bundled app saves `folder-tree` + `ml-body` innerHTML to localStorage every 30 s; inline script in `index.html` restores at cold start before the bundle has fetched. Sub-bullet (c) [500 ms placeholder pause] not in current code at cited line — skipped as stale. Expected cold start now 3-5 s (full WebView2 + bundle parse), with near-instant *feel* via hydrated snapshot after the first run. | M | |
 | **6** | **C125 — Unify desktop + Android IMAP code paths** | M | Today's split (`mailx-imap` Node-only + `mailx-store-web` browser-only) duplicates orchestration that's protocol-identical. Differences boil down to (a) transport factory choice (b) whether clients are persistent — both parameters, not architectural divides. Plan: extract `mailx-imap-core` with ImapManager + ops queue + fast-lane + sync orchestration over a `Transport` and a `Storage` interface, no Node-specific imports. `mailx-imap` and `mailx-store-web` become thin shims that wire engine-specific Storage + Transport. `{persistent: true\|false}` flag on ImapManager covers the lifetime split (no fast-lane queue when ephemeral). Pre-req for Android non-Gmail IMAP without forking sync code. |
 | **5** | **C124 — mailto handler on Linux + macOS** | S | Counterpart to P115 (Win). Linux: write a `~/.local/share/applications/rmfmail.desktop` with `MimeType=x-scheme-handler/mailto;` + `Exec=node /path/to/mailx.js --mailto %u`, `xdg-mime default rmfmail.desktop x-scheme-handler/mailto`, `update-desktop-database`. No exe-launcher needed — Linux pickers read `Name=` from the `.desktop` directly. macOS: real `.app` bundle (LaunchServices won't register a bare script) with `Info.plist` `CFBundleURLTypes` declaring `mailto` + a tiny launcher binary; `LSSetDefaultHandlerForURLScheme` programmatically sets the default. Rust `rmfmailto-src/` crate already portable — drop the Win32 imports and `cargo build --target x86_64-apple-darwin` produces the macOS launcher. |
 ~~done — C120 read transport diagnostics~~ shipped v1.0.583 (per-folder timeout error now includes `[conn#X r=YB w=ZB writes=N sinceLastRead=Tms]` snapshot for the doomed socket).
@@ -236,7 +236,8 @@ Previously shown as showstoppers; moved here because they haven't recurred on re
 
 Small, self-contained items. Pick them up between higher-priority blocks without asking. Bump version per fix.
 
-- **Q138 — Quoted-reply image sizes lost.** Bob 2026-05-10: in the reply quote of a marketing email (Qatar Airways), the App Store / Google Play / AppGallery button images render larger than in the original. Cause: `sanitizeQuotedBody` in `client/app.ts` strips `width` / `height` attributes (along with `align`, `valign`, `bgcolor`, `cellpadding`, `cellspacing`, `border`) to flatten marketing-email layout tables. Stripping those attrs from `<img>` is collateral damage — table-only attributes there, but they also size images. Fix: in the regex, scope the attr strip to non-img tags, OR leave width/height alone everywhere (the layout flattening doesn't actually need them on tables either — `<table>→<div>` rewrites are doing the heavy lifting). ~10 line change. Low priority — visual nit, not data loss.
+- ~~**Q138 — Quoted-reply image sizes lost.**~~ DONE 2026-05-11 in `client/app.ts:1000` — strip skips `<img>`. Two-pass loop handles multiple stripped attrs per tag.
+- **Q138 (original entry) — Quoted-reply image sizes lost.** Bob 2026-05-10: in the reply quote of a marketing email (Qatar Airways), the App Store / Google Play / AppGallery button images render larger than in the original. Cause: `sanitizeQuotedBody` in `client/app.ts` strips `width` / `height` attributes (along with `align`, `valign`, `bgcolor`, `cellpadding`, `cellspacing`, `border`) to flatten marketing-email layout tables. Stripping those attrs from `<img>` is collateral damage — table-only attributes there, but they also size images. Fix: in the regex, scope the attr strip to non-img tags, OR leave width/height alone everywhere (the layout flattening doesn't actually need them on tables either — `<table>→<div>` rewrites are doing the heavy lifting). ~10 line change. Low priority — visual nit, not data loss.
 - **Q136 — Server / provider capability matrix (forward backlog).** Catalog of server-specific features mailx could exploit. Detection is dynamic everywhere — IMAP via `CAPABILITY` + RFC 2971 `ID`, non-IMAP via provider-type at account setup. Each row below is its own future work item; this entry is the index.
   - **Dovecot** (IMAP — bobma's primary server): `NOTIFY` (✅ shipped 2026-05-10, see startWatching), `IDLE` (✅), `MOVE` (✅), `QRESYNC` / `CONDSTORE` (📋 Q135 — fast incremental sync with VANISHED + flag-since-modseq), `LIST-EXTENDED` (📋 single-roundtrip folder list with status counts), `METADATA` (📋 per-mailbox annotations — could hold per-folder mailx state on the server), `UNSELECT` (📋 fast folder switch without EXPUNGE), Push Framework / RFC 5423 (📋 server-side HTTP push — requires bobma config change, would let us retire IDLE).
   - **Cyrus** (IMAP — common in academia / legacy ISPs): `CONDSTORE` / `QRESYNC` (📋), `LIST-STATUS` (📋 — counts piggybacked on LIST), `MULTIAPPEND` (📋 — batch draft saves), `ANNOTATE-EXPERIMENT-1` (📋 — Cyrus's pre-METADATA cousin, syntax-incompatible). Server fingerprint via `ID` to distinguish from Dovecot when capability sets overlap.
@@ -256,10 +257,12 @@ Small, self-contained items. Pick them up between higher-priority blocks without
 
 - **Q135 — Server-capability gating + Dovecot QRESYNC/CONDSTORE fast path.** Bob 2026-05-10: "with IMAP you need to take advantage of features for each IMAP server. Start with Dovecot, support others as we identify them." Architectural shape: `iflow-direct` exposes a `Capabilities` set from the server's CAPABILITY response (`QRESYNC`, `CONDSTORE`, `MOVE`, `LIST-EXTENDED`, `SPECIAL-USE`, `XLIST`, `ENABLE`, `UIDPLUS`, `LITERAL+`, `NOTIFY`); `mailx-imap` branches per-account. Concrete first win — **QRESYNC/CONDSTORE on bobma**: replace the full UID-set re-fetch in `syncFolder` (`mailx-imap/index.ts`) with `SELECT ... QRESYNC (uidvalidity highestmodseq)` which returns `VANISHED` for deletions + `FETCH FLAGS` only for changed flags. Zero baseline re-fetch on idle folders, no false-positive deletes from truncated UID lists (the 50% safety guards become belt-and-braces rather than load-bearing). Second win — **MOVE (RFC 6851)** atomic move instead of COPY+EXPUNGE pair (already enabled on Gmail; needs detection on Dovecot/others). Third — **NOTIFY (RFC 5465)** per-mailbox interest declaration so IDLE on INBOX also receives FETCH/EXISTS events for OTHER selected mailboxes without re-IDLEing each. Per-server identification via `*ID` extension + capability fingerprint (Dovecot vs Cyrus vs Courier vs uw-imap have distinguishable CAPABILITY sets). Estimate: QRESYNC slice is M (iflow-direct exposes `select(folder, { qresync: { uidvalidity, modseq } })` + parse `VANISHED` + parse `OK [HIGHESTMODSEQ ...]`; mailx-imap persists modseq per folder in DB; reconcile loop uses VANISHED instead of set-diff). Pre-req: iflow-direct doesn't currently expose modseq or QRESYNC params — extension needed there first.
 - **Q134 — Real popout: compose + preview into independent msger windows.** Current popout button (title-bar icon) toggles the floating compose overlay to fill the host window — fast win but not a true OS window. Bob 2026-05-10: "popout should pop out into an independent window. This should also be true for the preview." Requires daemon refactor: single `handle = showService(...)` in `bin/mailx.ts` (~40 `handle.send` call sites) becomes a `windows` registry — `broadcast(msg)` fans events to all open windows; per-window onRequest closures route request acks back to the originator. New jsonrpc actions `openPopout({ mode, init })` (spawns a fresh `showService` with `contentDir: <client>` pointed at `compose/compose.html` or a `preview/preview.html` for viewer popout) and `getPopoutInit({ token })` (init data delivery — see option below). Client wires the popout button to call `api.openPopout(...)` then closes the floating overlay. Preview gets a matching button in `message-viewer.ts`. Init data delivery: URL-hash token + first-load fetch from daemon cache; popout has no state until it asks. Estimate: half-day. Pre-req for any window-pair workflow (browse main + read popped-out message, write compose while triaging inbox).
-- **Q133 — TinyMCE: pre-warm CDN fetch / progressive editor swap.** Bob 2026-05-10: "if fetching tinymce takes time it should be done in background and only activate when it is available." Today, picking TinyMCE in Settings means the FIRST compose-open after that setting blocks for ~1s while the CDN bundle downloads. Better: when user toggles to TinyMCE in Settings, fire a fetch (or a `<link rel="modulepreload">`) so the bundle is in browser cache by the time they hit Compose. Even better: open compose with Quill instantly, swap the editor to TinyMCE in place once the bundle resolves. Touches `compose.ts` editor init + `app.ts` Settings toggle handler.
+- ~~**Q133 — TinyMCE: pre-warm CDN fetch (settings-toggle prefetch).**~~ DONE 2026-05-11. `optEditorTinymce` change handler now appends `<link rel="prefetch">` for the configured TinyMCE URL so the next compose-open finds it cached. Progressive in-place editor swap (Quill → TinyMCE without reopen) is deferred — bigger change, smaller win once the prefetch is in place.
+- **Q133 (original entry) — TinyMCE: pre-warm CDN fetch / progressive editor swap.** Bob 2026-05-10: "if fetching tinymce takes time it should be done in background and only activate when it is available." Today, picking TinyMCE in Settings means the FIRST compose-open after that setting blocks for ~1s while the CDN bundle downloads. Better: when user toggles to TinyMCE in Settings, fire a fetch (or a `<link rel="modulepreload">`) so the bundle is in browser cache by the time they hit Compose. Even better: open compose with Quill instantly, swap the editor to TinyMCE in place once the bundle resolves. Touches `compose.ts` editor init + `app.ts` Settings toggle handler.
 - **Q132 — importgen bugs blocking "always use importgen" rule for rmfmail.** Three issues, fix in `y:\dev\utils\importgen`: (1) reads CWD's `package.json`, so running from `client/` reads `client/package.json` (declares only quill); needs a `-p` flag or to walk up. (2) `resolveEntryPoint` blows up on packages with conditional `exports` (e.g. `".": { "import": { "node": "...", "browser": "..." } }`) — `dotExport.import` becomes an object, then `entryPoint.startsWith` throws. (3) Doesn't follow dynamic `await import()` calls if it scans imports rather than package.json deps (still need to verify which path it actually takes). Until fixed, rmfmail's import map is hand-maintained.
 - **Q131 — Show holidays in calendar sidebar.** Thunderbird-style "Show holidays" checkbox. When checked, daemon's `getCalendarEvents` ALSO fetches `en.usa#holiday@group.v.calendar.google.com` (Google's standard US holidays) and merges into results, marking each with `isHoliday: true`. UI renders holiday rows with a distinct style (muted color, no edit/delete actions). Concrete edits: (a) `mailx-service/index.ts` line 1075ish `getCalendarEvents` — also call `gsync.listCalendarEvents(tp, fromMs, toMs, "en.usa#holiday@group.v.calendar.google.com")` when settings.calendar.showHolidays is true, store with `is_holiday=1` flag; (b) `mailx-store/db.ts` `calendar_events` schema — add `is_holiday` column; (c) `client/components/calendar-sidebar.ts` — render rows with `data-holiday="1"` styled distinctly; (d) settings UI radio/checkbox in the calendar sidebar's options popover. ~2 hours.
-- **Q117 — Spring-loaded folders (drag-hover-expand).** When the user is dragging messages and hovers over a collapsed folder in the folder tree, expand it after a short hover delay so the user can see / target child folders. Auto-collapse back to its prior state when the drag leaves the folder OR the drop completes. Standard pattern in Outlook / Thunderbird / Finder. Implementation: `dragenter` on a folder row starts a timer (named constant `DRAG_HOVER_EXPAND_MS`, ~600ms target); `dragleave` cancels; on timer fire, set an `expanded-during-drag` flag and toggle the disclosure triangle. `dragend` / `drop` walks back up clearing flags and restoring prior state. No persistence — purely transient. (Per `feedback_no_magic_numbers.md`: timing values go in named constants, not inline.)
+- ~~**Q117 — Spring-loaded folders (drag-hover-expand).**~~ DONE — already implemented at `folder-tree.ts:355+` with `DRAG_HOVER_EXPAND_MS = 600` constant, `dragAutoExpanded` set, and `dragend` collapse-back path. TODO entry was stale.
+- **Q117 (original entry) — Spring-loaded folders.** When the user is dragging messages and hovers over a collapsed folder in the folder tree, expand it after a short hover delay so the user can see / target child folders. Auto-collapse back to its prior state when the drag leaves the folder OR the drop completes. Standard pattern in Outlook / Thunderbird / Finder. Implementation: `dragenter` on a folder row starts a timer (named constant `DRAG_HOVER_EXPAND_MS`, ~600ms target); `dragleave` cancels; on timer fire, set an `expanded-during-drag` flag and toggle the disclosure triangle. `dragend` / `drop` walks back up clearing flags and restoring prior state. No persistence — purely transient. (Per `feedback_no_magic_numbers.md`: timing values go in named constants, not inline.)
 - **Verify rail-view / rail-settings dropdowns in narrow mode (paused 2026-04-30).** Added `rail-view` icon (👁) and `openMenuFromRail()` in `client/app.ts` that reparents the toolbar dropdown to `<body>` and switches to `position:fixed` anchored to the rail icon — so the toolbar's `display:none` in narrow mode no longer hides the menu. Compile not yet re-verified after the edits because Bash kept wedging; user paused to start the monorepo reorg (mailx → mailxapp). When resuming: run `tsc` in `client/`, sanity-check that opening View / Settings from the rail in narrow mode shows the dropdown anchored next to the icon, and that toolbar buttons in wide mode still work (they just toggle `hidden` on the same dropdown DOM, which is now a body child but still works).
 
 ---

package/bin/build-bundles.mjs

@@ -0,0 +1,96 @@
+#!/usr/bin/env node
+/**
+ * Bundle client entry points so cold-start drops from ~43 s to ~2 s.
+ *
+ * mailx's msger custom protocol routes every fetch through one IPC channel
+ * (Rust → daemon stdin → response → stdout), so an ES module import cascade
+ * pays a multi-second roundtrip per file. With ~17 client-side modules each
+ * pulling in transitive deps, the WebView is paying that cost N times in
+ * series.
+ *
+ * esbuild collapses the import graph into one file per entry point. The
+ * boot router in index.html stops cascading after one fetch.
+ *
+ * Dynamic `await import(...)` calls stay as runtime imports (rmf-tiny
+ * adapter for the TinyMCE editor, android-bootstrap fallback). Those are
+ * either tiny themselves, or live in a code path the desktop main flow
+ * doesn't take.
+ */
+import * as esbuild from "esbuild";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+
+const root = path.resolve(fileURLToPath(import.meta.url), "..", "..");
+const watch = process.argv.includes("--watch");
+
+const entries = [
+    {
+        in:  path.join(root, "client", "app.ts"),
+        out: path.join(root, "client", "app.bundle.js"),
+        label: "app",
+    },
+    {
+        in:  path.join(root, "client", "compose", "compose.ts"),
+        out: path.join(root, "client", "compose", "compose.bundle.js"),
+        label: "compose",
+    },
+];
+
+const shared = {
+    bundle:    true,
+    format:    "esm",
+    target:    "es2022",
+    platform:  "browser",
+    sourcemap: true,
+    // Keep dynamic imports for code paths that should still load on demand:
+    // - @bobfrankston/rmf-tiny is loaded only when the user picks TinyMCE.
+    // - @bobfrankston/mailx-store-web/android-bootstrap is only on Android.
+    // - Quill / TinyMCE themselves are loaded by their adapters at runtime.
+    // - jsonc-parser is large and only used by the JSONC config editor.
+    external: [
+        "@bobfrankston/rmf-tiny",
+        "@bobfrankston/mailx-store-web/*",
+        "tinymce",
+        "tinymce/*",
+        "quill",
+        "quill/*",
+        "jsonc-parser",
+        "mammoth",
+        "html-to-docx",
+    ],
+    // The output has to be ESM-compatible — index.html loads it via
+    // <script type="module"> through the boot router's `await import()`.
+    // Splitting would create extra chunks served as separate fetches,
+    // defeating the purpose; the whole point is one fetch per entry.
+    splitting: false,
+    // Suppress the "minify saved N bytes" chatter; we want the source
+    // readable in DevTools (sourcemap doubles up).
+    minify: false,
+    logLevel: "info",
+};
+
+async function buildAll() {
+    const t0 = Date.now();
+    const results = await Promise.all(entries.map(e =>
+        esbuild.build({ ...shared, entryPoints: [e.in], outfile: e.out })
+            .then(r => ({ ok: true, label: e.label, out: e.out }))
+            .catch(err => ({ ok: false, label: e.label, err }))
+    ));
+    for (const r of results) {
+        if (r.ok) console.log(`  bundle:${r.label} → ${path.relative(root, r.out)}`);
+        else console.error(`  bundle:${r.label} FAILED: ${r.err?.message || r.err}`);
+    }
+    const failed = results.filter(r => !r.ok).length;
+    if (failed) process.exit(1);
+    console.log(`build-bundles: done in ${Date.now() - t0} ms`);
+}
+
+if (watch) {
+    const ctxs = await Promise.all(entries.map(e =>
+        esbuild.context({ ...shared, entryPoints: [e.in], outfile: e.out })
+    ));
+    for (const c of ctxs) await c.watch();
+    console.log("build-bundles: watch mode");
+} else {
+    await buildAll();
+}