alvin-bot 4.9.2 → 4.9.4

package/CHANGELOG.md

@@ -2,6 +2,85 @@
 
 All notable changes to Alvin Bot are documented here.
 
+## [4.9.4] — 2026-04-13
+
+### 🔌 Web UI fully decoupled from main bot — port conflicts no longer crash anything
+
+Colleague feedback (WhatsApp voice note, 2026-04-13):
+> *"The gateway binds to port 3100 like OpenClaw. When the bot restarts,
+> the port is often still held → catastrophic crash. I ended up
+> decoupling the gateway process completely, because the actual bot
+> runs independently of the gateway — it can still answer Telegram
+> even if the web endpoint isn't reachable yet. It's weird that the
+> main routine crashes when the port is busy. It should just run in
+> the background, watch for the port to become free, and connect
+> then. Zero impact on the main routine."*
+
+He was right. My v4.9.0 `stopWebServer()` fix was *prevention* — it stopped the bot itself from holding 3100 across restarts. But it didn't cover the *resilience* side: a foreign process holding 3100 (another dev server, an OpenClaw-style orphan, a TIME_WAIT race after SIGKILL) still crashed the boot, because `startWebServer()` was synchronous and the `uncaught exception` from `server.listen()` escaped to the main event loop.
+
+**Complete rewrite of the bind loop:**
+
+- **`src/web/bind-strategy.ts` (new) — pure decision helper.** `decideNextBindAction(err, attempt, opts)` returns either `{type: "retry-port", port, attempt}` (climb the ladder) or `{type: "retry-background", delayMs, port}` (back off, retry the original port in 30 s). EADDRINUSE with attempts remaining → ladder. EADDRINUSE exhausted → background. Any other error → background. 8 unit tests covering every branch + purity.
+
+- **`src/web/server.ts` startWebServer — non-blocking, fresh-server-per-attempt.** Returns `void` synchronously, NEVER throws, NEVER blocks on bind. Each attempt creates a new `http.Server` (no state-recycling bugs) and attaches its own error handler. On failure, cleans up and calls `decideNextBindAction` to decide the next move. If the ladder is exhausted, schedules a 30 s background retry at the original port — the Telegram bot keeps running the whole time, the web UI just isn't reachable yet.
+
+- **`src/web/server.ts` WebSocketServer attached POST-bind.** The `ws` library's `WebSocketServer` constructor installs its own event plumbing on the underlying `http.Server` and — crucially — causes EADDRINUSE errors to escape as uncaught exceptions when attached pre-listen. Debugging this chewed an hour on 2026-04-13. Fix: only `new WebSocketServer({ server })` AFTER `listen()` has fired its callback. The unit-test `test/web-server-integration.test.ts "when the primary port is taken"` pins this behaviour.
+
+- **`src/web/server.ts` error handler: `on` not `once`.** Previous version used `.once("error", handler)` and a node edge case where a single bind failure emits TWO error events left the second one uncaught. Handler is now `on` with a `handled` guard — idempotent, and a post-bind quiet logger replaces it on success.
+
+- **`src/web/server.ts` defensive try/catch around `server.listen()`.** In the wild Node sometimes throws synchronously for edge-case binds (already-listening, invalid backlog, kernel race). The catch funnels sync throws through the same `handleBindFailure` path as async error events.
+
+- **`src/web/server.ts` `closeHttpServerGracefully(server)` + `stopWebServer()`.** The old `stopWebServer(server)` took an explicit server arg; it's been split into a low-level helper (`closeHttpServerGracefully(server)`, exported for tests) and a stateful top-level (`stopWebServer()`, no args, cleans up `currentServer` + `wsServerRef` + `bindRetryTimer`). Safe to call before start, safe to call twice, cancels pending background retries.
+
+- **`src/index.ts` call sites adjusted.** `const webServer = startWebServer()` → `startWebServer()`. `stopWebServer(webServer)` → `stopWebServer()`. The comment above the call explains the decoupling so nobody accidentally re-couples it in a future "clean up" refactor.
+
+**Testing: 186 → 201 (+15 new).**
+
+- `test/web-server-resilience.test.ts` — 8 unit tests for `decideNextBindAction`
+- `test/web-server-integration.test.ts` — 7 real-server integration tests: startWebServer returns void, binds, stops, is idempotent, survives primary-port conflict by climbing the ladder, closes servers with hanging sockets.
+- **Live-verified on the maintainer's machine**: `launchctl unload` + dual-stack Node hog on port 3100 + `launchctl load` → bot booted cleanly → out.log contained `[web] port 3100 busy (EADDRINUSE) — trying 3101` → `🌐 Web UI: http://localhost:3101   (Port 3100 was busy, using 3101 instead)` → Telegram responsive throughout. Exactly what the colleague described.
+
+**Non-goals / intentionally unchanged:**
+- Timeouts stay unlimited (v4.8.8 behaviour preserved).
+- The primary port is still `WEB_PORT || 3100` — no config schema change.
+- When the bot binds on a non-primary port (e.g. 3101), the README permalink still points at 3100. Users hitting a ladder-climbed bot should check the startup log; this is rare and temporary.
+
+## [4.9.3] — 2026-04-11
+
+### 🛠 Two UX bugs found in production after v4.9.2 — now closed
+
+Ali triggered `/cron run Daily Job Alert` after the v4.9.2 deploy and saw 13 minutes of chat silence followed by nothing. Forensics on the live bot revealed two distinct problems on top of an already-successful run:
+
+**1. `subagent-delivery` has been silently dropping every banner for days.** Err.log: `GrammyError: Call to 'sendMessage' failed! (400: Bad Request: can't parse entities: Can't find end of the entity starting at byte offset 2636)`. The daily-job-alert sub-agent produces markdown-dense output (`|` tables, `**bold**`, `\|` escapes, mixed asterisks). Telegram's Markdown parser refuses it, `api.sendMessage(..., parse_mode: "Markdown")` throws, and the bare try/catch in `deliverSubAgentResult` logs + bails. **Result: the user has never seen a sub-agent-delivery banner, even when the underlying run succeeded perfectly and emailed the HTML report correctly.**
+
+Fix in `src/services/subagent-delivery.ts`: new `sendWithMarkdownFallback()` helper that detects the "can't parse entities" pattern and retries the SAME text without `parse_mode`. All three code paths (file-upload case, single-message case, chunked case) now flow through the helper. 3 new tests drive the happy path, non-parse errors, and the chunked path.
+
+**2. `/cron run` had zero proof-of-life for 13 minutes.** The handler used to `await runJobNow(...)` synchronously and reply only when finished. Telegram's typing indicator expires after 5s. Users saw: command sent → typing indicator blip → nothing → nothing → (much later, if at all) result. For cron jobs that take 10-15 min (daily job alert, Perseus health, Polyseus P&L), this is indistinguishable from a dead bot.
+
+Fix — new handler flow:
+
+```
+bot:  🚀 Started *Daily Job Alert* — working…          ← instant ack
+bot:  🔄 Running *Daily Job Alert* · 1m 0s elapsed…    ← edit every 60s
+bot:  🔄 Running *Daily Job Alert* · 2m 0s elapsed…    ← edit
+...
+bot:  ✅ Done — *Daily Job Alert* · 13m 17s             ← final edit
+bot:  ✅ *Daily Job Alert* completed · 13m · 2.6M/28k  ← subagent-delivery
+       [full report body, Markdown-safe with plain-text fallback]
+```
+
+The ticker uses a single `editMessageText` call per minute on the same message — zero notification spam, clean visual progress. Every edit is wrapped with `isHarmlessTelegramError` so the inevitable "message is not modified" races stay silent. The ack itself falls back to plain text if the first `reply` hits a parse error, and the final edit falls back to a fresh plain message if the edit fails.
+
+New module: `src/handlers/cron-progress.ts` with pure helpers — `formatElapsed`, `escapeMarkdown`, `buildTickerText`, `buildDoneText`. 8 tests cover the formatting rules and markdown-safety escapes so future cron jobs with weird names (`weird_job*name`) can't break the ticker.
+
+**186 tests total** (+11 new). All green. Timeouts remain unlimited.
+
+**What you see after this upgrade:**
+- Instant "🚀 Started" ack on `/cron run`
+- Live elapsed-time ticker every minute
+- Final "✅ Done" when the sub-agent finishes
+- A separate banner+body message with the full report — **this time actually delivered**, even when the body contains broken Markdown
+
 ## [4.9.2] — 2026-04-11
 
 ### 🔍 Post-review polish: three edge cases from the strict audit

package/README.md

@@ -114,7 +114,18 @@ That's it. The setup wizard validates everything:
 
 **Requires:** Node.js 18+ ([nodejs.org](https://nodejs.org)) · Telegram bot token ([@BotFather](https://t.me/BotFather)) · Your Telegram user ID ([@userinfobot](https://t.me/userinfobot))
 
-Free AI providers available — no credit card needed.
+Free AI providers available — no credit card needed. **Privacy-first?** Pick the 🔒 **Offline — Gemma 4 E4B** option in setup for a fully local LLM via Ollama (macOS/Linux: automated install; Windows: manual).
+
+### 📘 First-time setup walkthroughs
+
+Step-by-step guides with screenshots and screen-for-screen instructions:
+
+| Platform | PDF (printable) |
+|---|---|
+| 🍎 **macOS** (with `launchd` background service) | [Download PDF](https://github.com/alvbln/Alvin-Bot/releases/latest/download/Alvin-Bot-macOS-Setup-Guide.pdf) |
+| 🪟 **Windows** (with Task Scheduler / Startup folder) | [Download PDF](https://github.com/alvbln/Alvin-Bot/releases/latest/download/Alvin-Bot-Windows-Setup-Guide.pdf) |
+
+Both guides cover: Node.js install · Telegram bot creation · first-time `setup` · foreground test · background service · offline Gemma 4 mode · troubleshooting. ~15 min end-to-end for a first-time user.
 
 ### macOS: use `launchd` instead of pm2 (recommended)
 

package/dist/handlers/commands.js

@@ -16,6 +16,9 @@ import { getMCPStatus, getMCPTools, callMCPTool } from "../services/mcp.js";
 import { listCustomTools, executeCustomTool } from "../services/custom-tools.js";
 import { screenshotUrl, extractText, generatePdf, hasPlaywright } from "../services/browser.js";
 import { listJobs, createJob, deleteJob, toggleJob, runJobNow, formatNextRun, humanReadableSchedule } from "../services/cron.js";
+import { resolveJobByNameOrId } from "../services/cron-resolver.js";
+import { buildTickerText, buildDoneText, escapeMarkdown } from "./cron-progress.js";
+import { isHarmlessTelegramError } from "../util/telegram-error-filter.js";
 import { storePassword, revokePassword, getSudoStatus, verifyPassword } from "../services/sudo.js";
 import { config } from "../config.js";
 import { BOT_VERSION } from "../version.js";
@@ -1442,11 +1445,25 @@ export function registerCommands(bot) {
             return;
         }
         // /cron run <name-or-id>
+        //
+        // UX contract:
+        //   1. Instantly post a "🚀 Started …" message so the user knows
+        //      the command was received.
+        //   2. Every 60s edit that message with the elapsed-time ticker
+        //      so the chat shows proof-of-life during 10+ min sub-agent
+        //      runs (the Daily Job Alert takes ~13 min in production).
+        //   3. When runJobNow returns, edit the same message into a
+        //      final "✅ Done" / "❌ error" / "⏳ already running" state.
+        //   4. The heavy lifting (banner + full body + chunking) stays in
+        //      subagent-delivery.ts — which now has a Markdown→plain-text
+        //      fallback so it actually reaches the user.
         if (arg.startsWith("run ")) {
             const nameOrId = arg.slice(4).trim();
-            await ctx.api.sendChatAction(ctx.chat.id, "typing");
-            const outcome = await runJobNow(nameOrId);
-            if (outcome.status === "not-found") {
+            // Resolve up-front so we can show the real job name in the
+            // "Started" ack, and so we handle the not-found case BEFORE
+            // spending a Telegram round-trip on a pointless placeholder.
+            const resolved = resolveJobByNameOrId(listJobs(), nameOrId);
+            if (!resolved) {
                 const jobs = listJobs();
                 const hint = jobs.length > 0
                     ? `\n\nAvailable:\n${jobs.slice(0, 10).map(j => `• ${j.name}`).join("\n")}`
@@ -1454,15 +1471,80 @@ export function registerCommands(bot) {
                 await ctx.reply(`❌ No job matches <code>${nameOrId}</code>.${hint}`, { parse_mode: "HTML" });
                 return;
             }
-            if (outcome.status === "already-running") {
-                await ctx.reply(`⏳ Job "${outcome.job.name}" is already running — not starting a duplicate. ` +
-                    `Wait for the current run to finish, or /subagents cancel to abort it.`);
-                return;
+            const jobName = resolved.name;
+            const startedAt = Date.now();
+            // Post initial ack — we'll edit THIS message for the ticker and
+            // the final state.
+            let ackMessageId = null;
+            try {
+                const ack = await ctx.reply(`🚀 Started *${escapeMarkdown(jobName)}* — working…`, { parse_mode: "Markdown" });
+                ackMessageId = ack.message_id;
+            }
+            catch (err) {
+                // If even the initial ack fails, fall back to plain text so
+                // the user still knows we received the command.
+                try {
+                    const ack = await ctx.reply(`🚀 Started ${jobName} — working…`);
+                    ackMessageId = ack.message_id;
+                }
+                catch { /* give up on the ack — run still fires below */ }
+            }
+            const chatId = ctx.chat.id;
+            // Progress ticker: edit the ack message with elapsed time every
+            // 60s. Errors from editMessageText (including the harmless
+            // "message is not modified") are swallowed via the central filter.
+            const ticker = setInterval(async () => {
+                if (ackMessageId === null)
+                    return;
+                const elapsed = Math.floor((Date.now() - startedAt) / 1000);
+                try {
+                    await ctx.api.editMessageText(chatId, ackMessageId, buildTickerText(jobName, elapsed), { parse_mode: "Markdown" });
+                }
+                catch (err) {
+                    if (!isHarmlessTelegramError(err)) {
+                        console.warn(`[cron:run] ticker edit failed:`, err);
+                    }
+                }
+            }, 60_000);
+            let outcome;
+            try {
+                outcome = await runJobNow(nameOrId);
+            }
+            finally {
+                clearInterval(ticker);
+            }
+            // Final state — edit the ack message one last time.
+            const elapsed = Math.floor((Date.now() - startedAt) / 1000);
+            const finalText = (() => {
+                if (outcome.status === "not-found") {
+                    // Shouldn't happen — we already resolved successfully above —
+                    // but handle it for completeness.
+                    return `❌ ${escapeMarkdown(jobName)} — not found (race?)`;
+                }
+                if (outcome.status === "already-running") {
+                    return buildDoneText(outcome.job.name, elapsed, { ok: true, skipped: true });
+                }
+                return buildDoneText(outcome.job.name, elapsed, {
+                    ok: !outcome.error,
+                    error: outcome.error,
+                });
+            })();
+            if (ackMessageId !== null) {
+                try {
+                    await ctx.api.editMessageText(chatId, ackMessageId, finalText, { parse_mode: "Markdown" });
+                }
+                catch (err) {
+                    if (!isHarmlessTelegramError(err)) {
+                        // Last-ditch fallback: post as a new plain message so the
+                        // user sees the result even if the edit failed.
+                        await ctx.reply(finalText).catch(() => { });
+                    }
+                }
+            }
+            else {
+                // We never got an ack message id — just post fresh
+                await ctx.reply(finalText, { parse_mode: "Markdown" }).catch(() => ctx.reply(finalText));
             }
-            const output = outcome.output
-                ? `\`\`\`\n${outcome.output.slice(0, 2000)}\n\`\`\``
-                : "(no output)";
-            await ctx.reply(`🔧 Job "${outcome.job.name}" executed:\n${output}${outcome.error ? `\n\n❌ ${outcome.error}` : ""}`, { parse_mode: "Markdown" });
             return;
         }
         await ctx.reply("Unknown cron command. Use /cron for help.");

package/dist/handlers/cron-progress.js

@@ -0,0 +1,52 @@
+/**
+ * Pure helpers for the /cron run progress ticker.
+ *
+ * Separated from commands.ts so the formatting and safety rules can be
+ * unit-tested without standing up the entire grammy Context. The command
+ * handler wires these into a setInterval that edits a single Telegram
+ * message once per tick, giving the user visible proof-of-life during
+ * long-running (10+ min) cron jobs.
+ *
+ * See test/cron-progress-ticker.test.ts for the contract.
+ */
+/** Human-readable elapsed time — adapts unit to magnitude. */
+export function formatElapsed(seconds) {
+    if (seconds < 60)
+        return `${seconds}s`;
+    const minutes = Math.floor(seconds / 60);
+    const remSec = seconds % 60;
+    if (minutes < 60)
+        return `${minutes}m ${remSec}s`;
+    const hours = Math.floor(minutes / 60);
+    const remMin = minutes % 60;
+    return `${hours}h ${remMin}m`;
+}
+/**
+ * Escape Markdown-breaking characters in untrusted display strings so
+ * an edit-message call can safely use `parse_mode: Markdown` without
+ * triggering "can't parse entities" — the exact bug that killed every
+ * daily-job-alert banner for days.
+ *
+ * We use Telegram Markdown (v1) escape rules: only `*`, `_`, `[`, `` ` ``.
+ * The rest flow through unchanged.
+ */
+export function escapeMarkdown(text) {
+    return text.replace(/([*_[\]`])/g, "\\$1");
+}
+/** Intermediate ticker text: "🔄 Running *name* · 2m 5s elapsed…" */
+export function buildTickerText(jobName, elapsedSeconds) {
+    const safe = escapeMarkdown(jobName);
+    return `🔄 Running *${safe}* · ${formatElapsed(elapsedSeconds)} elapsed…`;
+}
+/** Final ticker state: "✅ Done — *name* · 13m 17s" (or ❌ / ⏳). */
+export function buildDoneText(jobName, elapsedSeconds, outcome) {
+    const safe = escapeMarkdown(jobName);
+    if (outcome.skipped) {
+        return `⏳ *${safe}* is already running — not starting a duplicate`;
+    }
+    if (!outcome.ok) {
+        const errLine = outcome.error ? `\n\n${outcome.error.slice(0, 500)}` : "";
+        return `❌ *${safe}* — ${formatElapsed(elapsedSeconds)}${errLine}`;
+    }
+    return `✅ Done — *${safe}* · ${formatElapsed(elapsedSeconds)}`;
+}

package/dist/index.js

@@ -267,7 +267,7 @@ const shutdown = async () => {
     }
     // Release :3100 so the next launchd boot doesn't hit EADDRINUSE.
     // Must happen before exit — see src/web/server.ts stopWebServer() comment.
-    await stopWebServer(webServer).catch((err) => console.warn("[shutdown] stopWebServer failed:", err));
+    await stopWebServer().catch((err) => console.warn("[shutdown] stopWebServer failed:", err));
     await unloadPlugins().catch(() => { });
     await disconnectMCP().catch(() => { });
     // Tear down any bot-managed local runners (Ollama, LM Studio, …) so VRAM
@@ -404,8 +404,13 @@ async function startOptionalPlatforms() {
     }
 }
 startOptionalPlatforms().catch(err => console.error("Platform startup error:", err));
-// Start Web UI (ALWAYS — regardless of Telegram/AI config)
-const webServer = startWebServer();
+// Start Web UI (ALWAYS — regardless of Telegram/AI config).
+// startWebServer is now non-blocking and will never throw: if port 3100
+// is busy (foreign process, TIME_WAIT, another bot instance), it climbs
+// the port ladder up to 3119 and then enters a background retry loop
+// at 3100 every 30s. The Telegram bot runs independently — Web UI is a
+// feature, not core. See src/web/bind-strategy.ts for the retry rules.
+startWebServer();
 // Start Cron Scheduler — route notifications through delivery queue for reliability
 setNotifyCallback(async (target, text) => {
     if (target.platform === "web") {

package/dist/services/subagent-delivery.js

@@ -10,6 +10,35 @@
  * module with a fake bot via __setBotApiForTest.
  */
 import { getVisibility } from "./subagents.js";
+/**
+ * Telegram's Markdown parser rejects unbalanced or unexpected entities
+ * (stray `*`, `_`, un-escaped `|` in tables, etc.). Sub-agent outputs
+ * mix all of these. When we hit one of these errors, retry the same
+ * content as plain text so the user still sees the result instead of
+ * a silent drop.
+ */
+function isTelegramParseError(err) {
+    if (!err || typeof err !== "object")
+        return false;
+    const e = err;
+    const haystack = `${e.message ?? ""} ${e.description ?? ""}`;
+    return /can't parse entities|can't find end of the entity/i.test(haystack);
+}
+/**
+ * Send a Markdown message with an automatic plain-text retry on parse
+ * errors. Any other error propagates to the caller's outer catch.
+ */
+async function sendWithMarkdownFallback(api, chatId, text) {
+    try {
+        await api.sendMessage(chatId, text, { parse_mode: "Markdown" });
+    }
+    catch (err) {
+        if (!isTelegramParseError(err))
+            throw err;
+        console.warn(`[subagent-delivery] Markdown parse failed, retrying as plain text`);
+        await api.sendMessage(chatId, text);
+    }
+}
 const MAX_TG_CHUNK = 3800; // below Telegram's 4096 limit with headroom
 const FILE_UPLOAD_THRESHOLD = 20_000; // switch to .md file upload above this
 let injectedApi = null;
@@ -243,7 +272,7 @@ export async function deliverSubAgentResult(info, result, opts = {}) {
     try {
         // Case 1: very long output → file upload with a short banner
         if (body.length > FILE_UPLOAD_THRESHOLD) {
-            await api.sendMessage(info.parentChatId, banner, { parse_mode: "Markdown" });
+            await sendWithMarkdownFallback(api, info.parentChatId, banner);
             try {
                 const { InputFile } = await import("grammy");
                 const buf = Buffer.from(body, "utf-8");
@@ -257,12 +286,14 @@ export async function deliverSubAgentResult(info, result, opts = {}) {
         }
         // Case 2: fits in a single message → banner + body joined
         if (body.length + banner.length + 2 <= MAX_TG_CHUNK) {
-            await api.sendMessage(info.parentChatId, `${banner}\n\n${body}`, { parse_mode: "Markdown" });
+            await sendWithMarkdownFallback(api, info.parentChatId, `${banner}\n\n${body}`);
             return;
         }
         // Case 3: medium output → banner as its own message, body chunked
-        await api.sendMessage(info.parentChatId, banner, { parse_mode: "Markdown" });
+        await sendWithMarkdownFallback(api, info.parentChatId, banner);
         for (let i = 0; i < body.length; i += MAX_TG_CHUNK) {
+            // Body chunks are always sent as plain text — markdown across
+            // arbitrary chunk boundaries would be inconsistent anyway.
             await api.sendMessage(info.parentChatId, body.slice(i, i + MAX_TG_CHUNK));
         }
     }

package/dist/web/bind-strategy.js

@@ -0,0 +1,42 @@
+/**
+ * Pure decision helper for the web-server bind loop.
+ *
+ * Decouples the "what should happen next" logic from the side-effect
+ * spaghetti of real http.Server binding so it can be unit-tested in
+ * isolation. See test/web-server-resilience.test.ts for the contract.
+ *
+ * Why this exists: the v4.8.x and earlier implementations crashed the
+ * entire bot when port 3100 was held by a foreign process. A colleague
+ * running an OpenClaw fork hit the same bug years ago and ended up
+ * decoupling the web server completely — the main bot should never be
+ * gated on a web-UI bind. This helper encodes the decision logic so
+ * the new startWebServer() can just act on the returned action.
+ */
+/**
+ * Decide what the bind loop should do next after a failed listen().
+ *
+ * Rule of thumb:
+ *   - EADDRINUSE AND attempts remaining  → climb the port ladder.
+ *   - EADDRINUSE AND ladder exhausted    → background retry at original port.
+ *   - any other error (EACCES, listen-called-twice, etc.) → background retry.
+ *
+ * PURE: no timers, no I/O, no mutation of inputs. Safe to call from tests.
+ */
+export function decideNextBindAction(err, attempt, opts) {
+    const code = err?.code;
+    if (code === "EADDRINUSE" && attempt < opts.maxPortTries - 1) {
+        return {
+            type: "retry-port",
+            port: opts.originalPort + attempt + 1,
+            attempt: attempt + 1,
+        };
+    }
+    // EADDRINUSE with no attempts left, OR any non-EADDRINUSE error:
+    // don't walk the port ladder further, just back off and retry the
+    // original port in the background.
+    return {
+        type: "retry-background",
+        delayMs: opts.backgroundRetryMs,
+        port: opts.originalPort,
+    };
+}