switchroom 0.5.0

package/telegram-plugin/docs/pinned-progress-card-reliability.md

@@ -0,0 +1,144 @@
+# Pinned progress card — reliability spec
+
+Status: **documenting existing system + closing gaps.** The pin/unpin machinery already exists (see §3). This spec formalizes the invariants it must hold, enumerates failure modes, specifies the test matrix, and lists the residual gaps that still need closing to hit "insanely reliable UX."
+
+## 1. Goal
+
+When the agent is working on a turn, the user always sees a **single, live-updating, pinned** status message for that turn. When the turn ends, that message is either marked `✅ Done` and unpinned, or (for very fast turns) never shown at all. No stale pins, no orphan pins, no duplicate pins, no silent failures.
+
+"Insanely reliable" = every one of the invariants in §4 holds under crash, kill, restart, rate-limit, race, and parallel-turn conditions, with automated tests covering each.
+
+## 2. Non-goals
+
+- Per-tool progress granularity beyond what the event-driven card already renders.
+- Pinning arbitrary bot messages (only the per-turn progress card).
+- Fighting the user for pin real-estate: a user-pinned message is always a barrier (see I8).
+- Pinning in group chats with many admins (Telegram restricts `pinChatMessage` to bots with `can_pin_messages`; failure is logged and swallowed, card still updates in place).
+
+## 3. Existing implementation (ground truth)
+
+Files and their load-bearing roles:
+
+| File | Role |
+|---|---|
+| `progress-card.ts` | Pure reducer + renderer. Turn-scoped state; event → HTML. |
+| `progress-card-driver.ts` | Cadence controller. Coalesce + min-interval + heartbeat + zombie ceiling. Fires `emit` with `isFirstEmit` flag and `onTurnComplete` with `turnKey`. |
+| `active-pins.ts` | Sidecar (`$AGENT_DIR/.active-pins.json`) — add/remove/read/write. Atomic rename on write. Shape-validated reads. |
+| `active-pins-sweep.ts` | Two sweeps: (a) `sweepActivePins` drains the sidecar and unpins each; (b) `sweepBotAuthoredPins` walks `getChat().pinned_message` and unpins anything authored by our bot, stopping at the first user-authored pin. |
+| `server.ts` (streamMode='checklist' block) | Wires driver → Telegram API. Owns `progressPinnedMsgIds`, `unpinnedTurnKeys`, and the idempotent `unpinProgressCard` closure. Boot-time + pre-restart sweeps wired. |
+
+Current lifecycle:
+
+```
+enqueue / startTurn
+  └─► driver allocates turnKey (chatId:threadId:seq)
+      └─► render(state) → emit(isFirstEmit=true)
+          └─► handleStreamReply creates message → messageId
+              ├─► addActivePin(sidecar)     ← WRITE BEFORE API CALL
+              └─► pinChatMessage(disable_notification=true)
+                   └─► on failure: removeActivePin (roll back)
+
+...live edits via coalesced flush (400ms) + heartbeat (5s)...
+
+turn_end   OR   stream_reply(done=true)   OR   reply(final=true)
+  └─► unpinProgressCard(turnKey) — idempotent via unpinnedTurnKeys
+      └─► unpinChatMessage
+           └─► finally: removeActivePin (regardless of outcome)
+```
+
+Crash / kill paths:
+- `SIGKILL` mid-turn → sidecar retains entry → next boot's `sweepActivePins` unpins.
+- `/restart`, `/update`, `/reconcile --restart` → proactive pre-SIGTERM sweep (see server.ts:2841).
+- Sidecar lost but pin still on Telegram → next boot's `sweepBotAuthoredPins` walks `getChat` and removes bot-authored pins until a user-authored pin acts as barrier.
+
+## 4. Invariants
+
+All must be tested (see §7). `I*` numbers are referenced elsewhere in this spec.
+
+| ID | Invariant |
+|---|---|
+| **I1** | Every `pinChatMessage` call is preceded by a successful `addActivePin` sidecar write. |
+| **I2** | Every successful pin produces exactly one `unpinChatMessage` call over the card's lifetime (across in-session unpin + restart sweep). |
+| **I3** | `unpinProgressCard(turnKey, …)` is idempotent — first call fires the API, all subsequent calls for the same `turnKey` are no-ops. |
+| **I4** | On process start, any sidecar entry from a prior session is consumed (attempt unpin, then clear) before new traffic is accepted. |
+| **I5** | The final render of the card (before unpin) shows `stage === 'done'` → `✅ Done` header. |
+| **I6** | Turns that complete faster than `initialDelayMs` (default 30s) produce **no** pin and **no** card — suppressed, not deferred-then-cancelled. |
+| **I7** | Parallel active turns on the same `(chatId, threadId)` each have independent `turnKey`, `pin`, `unpin`, and `sidecar` entries. The second `enqueue` force-closes the first (including its unpin) before creating the new pin. |
+| **I8** | `sweepBotAuthoredPins` stops at the first non-bot pinned message for a chat — never unpins a user's pin. |
+| **I9** | Zombie ceiling: a card whose `lastEventAt` is older than `maxIdleMs` (5 min) is force-closed via the same path as `turn_end` — unpin + `onTurnComplete` + state delete. |
+| **I10** | `pinChatMessage` failure does not leave the sidecar polluted — `removeActivePin` is called in the failure branch. |
+| **I11** | Completion notification (`✅ Done — <summary>` top-level message) only sent in forum-topic turns (`threadId != null`); never in plain DMs. |
+
+## 5. Failure modes & mitigations
+
+| # | Failure mode | Current mitigation | Residual gap |
+|---|---|---|---|
+| F1 | SIGKILL between `addActivePin` and `pinChatMessage` | Boot-time `sweepBotAuthoredPins` (no pin to remove, sidecar cleared) | None. |
+| F2 | SIGKILL between `pinChatMessage` success and `turn_end` | Boot-time `sweepActivePins` | None. |
+| F3 | SIGKILL between `unpinChatMessage` in-flight and `removeActivePin` | Next boot re-attempts unpin; Telegram unpin is idempotent (400 is harmless) | **Gap:** no structured telemetry distinguishes "real stale pin swept" from "redundant sweep on already-unpinned message" — both log generic failure. |
+| F4 | `pinChatMessage` 429 (rate limit) | Error logged + sidecar rolled back; card continues to live-update in place (unpinned) | **Gap:** no retry with backoff. User silently loses the pin for that turn. Should at least surface status-reaction signal differently. |
+| F5 | `unpinChatMessage` 429 | Error logged + sidecar cleared via `.finally()`; next boot will see stale pin via `sweepBotAuthoredPins` | **Gap:** mid-session stale pin persists until next restart. Consider best-effort retry (1 retry after 1s) before giving up. |
+| F6 | Bot lacks `can_pin_messages` in group | Error logged and swallowed; card still live-updates inline | None — graceful degradation. |
+| F7 | Session restart while turn still live | Pre-restart sweep unpins; new process's startup sweep is redundant but harmless; the resumed turn creates a fresh pin via the `--continue` path | **Gap:** user sees "Done → unpin → new pin" flicker during ~1–3s restart. Consider deferring pre-restart unpin until the new process confirms it has taken over (out of scope for this spec; needs handoff-protocol work). |
+| F8 | Two parallel turns on same `chatId:threadId` | `turnKey` allocator + `isSync` guards in enqueue handler | None (tests cover). |
+| F9 | Duplicate enqueue echoes from session-tail (JSONL rotation, reconnect) | `seenEnqueueMsgIds` 60s dedup + `pendingSyncEchoes` sync marker | None — well tested. |
+| F10 | Heartbeat keeps ticking a card whose `turn_end` was dropped | `maxIdleMs` zombie ceiling (5 min) force-closes | **Gap:** 5 min is long. Surface a warning in the card header after ~2 min of no events: `⚠️ No events for 2m — likely stuck.` |
+| F11 | User manually unpins the card mid-turn | Next `pinChatMessage` in this session is never called (pin is one-shot per turn); sidecar holds stale entry until `onTurnComplete` fires `removeActivePin` after a harmless `unpinChatMessage` 400 | **Gap:** card stops being pinned but user has no visual indication the card is still live. Low priority — if they unpinned it they chose to. |
+| F12 | Two bots in the chat both managing pins | `sweepBotAuthoredPins` filters by `botUserId` | None. |
+| F13 | `getChat().pinned_message` returns only the top pin, so a stack of bot pins requires iteration | `sweepBotAuthoredPins` loops up to `maxPerChat=32` | None. |
+
+## 6. Observability requirements
+
+Current state: `process.stderr.write` lines for pin/unpin failure. That's insufficient for "insanely reliable."
+
+Required:
+
+1. **Structured log event per pin/unpin**, one line JSON on stderr with prefix `pin-event:`. Fields: `event` (`pin|unpin|sweep-pin|sweep-auth`), `chatId`, `messageId`, `turnKey`, `outcome` (`ok|fail|rate-limited|forbidden`), `error?`, `durationMs`.
+2. **`/pins-status` admin command** (or extend `/status`): report current sidecar entries + in-memory `progressPinnedMsgIds` + any divergence.
+3. **Weekly self-audit** (or on boot): call `sweepBotAuthoredPins` in read-only mode across allowlisted chats and report count of bot pins not tracked in sidecar. Alarm if > 0 after a steady-state period.
+4. **Metric: pin-to-first-edit latency** — time from `pinChatMessage` returning to the first subsequent `editMessageText`. Should stay under ~1s; breach indicates rate-limit pressure.
+5. **Metric: orphan sweep frequency** — count of pins cleaned up by startup / bot-authored sweep per boot. Steady-state should be 0.
+
+## 7. Test matrix
+
+Existing tests to keep (enumerate and reference in CI):
+- `active-pins.test.ts` — sidecar add/remove/read/write/idempotency/corruption.
+- `active-pins-sweep.test.ts` — timeout bounds, barrier semantics, max-per-chat loop.
+- `progress-card.test.ts` — reducer covers all `turn_end` paths, renderer produces `✅ Done`.
+- `progress-card-driver.test.ts` — `isFirstEmit` fires exactly once, `onTurnComplete` fires exactly once, `initialDelayMs` suppression, parallel-turn force-close.
+
+New tests required for this spec:
+
+| ID | Test | Covers |
+|---|---|---|
+| T1 | Integration: simulate pin API failure → assert `removeActivePin` called and no stale sidecar entry | I10 |
+| T2 | Integration: simulate unpin API failure → assert sidecar cleared in `.finally()`, assert next-boot sweep picks up Telegram-side stale pin via `sweepBotAuthoredPins` | F3, F5 |
+| T3 | Unit: `sweepBotAuthoredPins` stops on first user-authored pin (barrier) | I8 |
+| T4 | Integration: two parallel `startTurn` calls on same `chatId:threadId` → two distinct `turnKey`s, two pins, two unpins, no orphan sidecar entries at end | I7 |
+| T5 | Integration: `turn_end` before `initialDelayMs` → zero emits, zero pins, sidecar untouched | I6 |
+| T6 | Integration: heartbeat ticks 2 min past last event → header shows stuck-warning; 5 min → zombie close fires unpin | F10, I9 |
+| T7 | Integration: boot with non-empty sidecar → sweep runs before first inbound message is processed | I4 |
+| T8 | Integration: rate-limit simulation — 20 rapid turns → each gets pin + unpin, no 429 surfaces to user visibly; degraded path logs structured event | F4, F5, §6.1 |
+| T9 | Structured log assertion: every pin/unpin emits exactly one `pin-event:` JSON line with all required fields | §6.1 |
+| T10 | Self-audit: boot-time read-only sweep reports 0 orphan pins on a clean chat | §6.3 |
+
+## 8. Implementation plan (gap-closing)
+
+Order of work, smallest-first:
+
+1. **Structured pin-event logging** (~30 LOC in `server.ts` + one helper) — closes §6.1, enables T9.
+2. **T1–T5 tests** — no production code changes, just formalizes existing behavior.
+3. **Stuck-warning in card header** (~15 LOC in `progress-card.ts` renderer + driver signal) — closes F10 lower tier.
+4. **Unpin retry (single attempt, 1s backoff)** (~20 LOC in `unpinProgressCard`) — closes F5.
+5. **`/pins-status` admin command** (~40 LOC) — closes §6.2.
+6. **Boot-time read-only audit + metric** (~30 LOC) — closes §6.3–6.5.
+7. **T6–T10 tests.**
+
+Total estimate: ~150 LOC production + ~400 LOC tests. No schema changes. No config migration. Backwards compatible with existing sidecars.
+
+## 9. Out of scope (future work)
+
+- Handoff-protocol for restart flicker (F7): needs new-process-confirms-takeover handshake. Large.
+- Multi-pin stacking UX (one pin per sub-agent task): current model is one pin per parent turn; changing it requires reworking `turnKey` allocation.
+- Pinning arbitrary user-selected bot messages.
+- Fallback to a "sticky last message" non-pin display when `can_pin_messages` is absent.

package/telegram-plugin/docs/stream-json-daemon-mode.md

@@ -0,0 +1,477 @@
+# stream-json daemon mode — architectural design
+
+Status: **DRAFT — design only, no code**.
+Author: investigation + writeup by `assistant` agent, 2026-05-01.
+Companion: `docs/streaming-deterministic.md` (April 2026 research notes that
+informed the current `stream_reply` heuristic — superseded by the findings here
+for the per-token streaming question).
+
+---
+
+## 0. TL;DR
+
+Switchroom agents currently run interactive Claude Code (`--dangerously-load-development-channels server:switchroom-telegram`) and observe model output via two derived sources: session-tail (whole assistant messages, no token-level granularity) and PTY-tail (parses the TUI rendering, structurally fragile). Per-token streaming of the assistant's reply text — the Claude.ai/mobile UX Ken asked for — is impossible from either source.
+
+Verified live in this investigation: Claude Code's `--print --output-format=stream-json --include-partial-messages` mode emits raw Anthropic-API `content_block_delta` events to stdout in real time, while supporting MCP servers, sub-agents, slash commands, and skills. Multi-turn input over stdin works (`--input-format=stream-json`).
+
+**The catch**: `--dangerously-load-development-channels server:NAME` does not surface the channel's MCP tools (`reply`, `stream_reply`, `react`, ...) to the model when the session runs in `--print` mode. Re-registering the same plugin via `--mcp-config` exposes the tools but loses the channels-as-inbound integration — there's no longer a path for Telegram messages to reach the agent.
+
+Path C — "migrate switchroom agents to stream-json daemon mode" — is therefore a re-architecture, not a flag change. The bridge becomes the inbound + outbound IO loop that today is split between `--channels` (inbound) and the MCP tool calls (outbound). Everything that touches the agent lifecycle (sub-agents, hooks, permission prompts, OAuth flow, slash commands, vault, restart semantics, slot-pool failover) needs revalidation under the new IO shape.
+
+This document captures what's true today, what would change, what's at risk, and how a spike could de-risk the unknowns before committing to a migration.
+
+---
+
+## 1. What was validated in this investigation
+
+### 1.1 Claude Code surfaces (verified against `claude --version` 2.1.126)
+
+| Flag | Behaviour |
+|---|---|
+| `--print` | One-shot mode (originally), but accepts multi-turn input when paired with `--input-format=stream-json` |
+| `--input-format text\|stream-json` | Text default; stream-json reads `{"type":"user","message":...}` JSON lines from stdin and processes each as a turn |
+| `--output-format text\|json\|stream-json` | stream-json emits one event per line to stdout |
+| `--include-partial-messages` | Emits `stream_event` lines containing Anthropic-API `content_block_delta` events (per-token text deltas) |
+| `--include-hook-events` | Emits hook lifecycle events (`PreToolUse`, `PostToolUse`, `Stop`, etc.) on the same stdout stream |
+| `--replay-user-messages` | Re-emits user messages back on stdout for ack tracking |
+| `--mcp-config <file>` | Loads MCP servers as regular tools the model can call |
+| `--strict-mcp-config` | Restricts to MCP servers from `--mcp-config` only (excludes user/global/.mcp.json) |
+| `--dangerously-load-development-channels server:NAME` | Loads an MCP plugin **as a channel** (inbound notifications + chat) |
+
+### 1.2 Per-token streaming verified
+
+```
+$ echo "say one word" | claude --print --verbose \
+    --output-format stream-json --include-partial-messages "say hi"
+{"type":"system","subtype":"init","tools":[...],"mcp_servers":[...],...}
+{"type":"stream_event","event":{"type":"message_start",...}}
+{"type":"stream_event","event":{"type":"content_block_delta",
+  "delta":{"type":"text_delta","text":"Hi there"}},...}
+{"type":"stream_event","event":{"type":"content_block_delta",
+  "delta":{"type":"text_delta","text":", how's it going?"}},...}
+{"type":"stream_event","event":{"type":"message_stop"},...}
+{"type":"result","subtype":"success","result":"Hi there, how's it going?",...}
+```
+
+Two `content_block_delta` events for a single sentence. Real per-token streaming, not whole-message atomic writes. This is the Anthropic API SSE shape forwarded verbatim to stdout.
+
+### 1.3 Multi-turn over stdin verified
+
+```
+$ printf '{"type":"user","message":{"role":"user","content":"say one"}}\n{"type":"user","message":{"role":"user","content":"say two"}}\n' \
+  | claude --print --verbose --output-format stream-json --input-format stream-json
+# Both turns get processed; per-token deltas emit for each.
+```
+
+### 1.4 The `--channels` blocker (negative finding)
+
+When the same MCP server is registered as `--dangerously-load-development-channels server:switchroom-telegram` rather than `--mcp-config`:
+
+- `init.tools` only contains built-ins + claude.ai HTTP MCP. **No `reply`, `stream_reply`, `react`, etc. exposed to the model.**
+- The model, asked directly: *"No `switchroom-telegram` MCP server is connected to this session."*
+
+Re-registering via `--mcp-config /tmp/test-mcp.json --strict-mcp-config`:
+
+- `mcp_servers` lists `switchroom-telegram status=failed`.
+- The MCP server crashes on boot — likely because `server.ts` looks for the gateway IPC socket and short-circuits when missing, and the test environment has no live gateway socket.
+
+So: no single flag invocation today both (a) surfaces the channel's tools to the model AND (b) routes Telegram inbound to it via stdin. Today's `--channels` mode does both for interactive sessions but neither path is wired for `--print --output-format=stream-json`.
+
+### 1.5 What the on-disk JSONL contains in stream-json mode
+
+Inspected after the test run:
+
+```
+$ jq -r 'type' < ~/.claude/projects/.../.../<id>.jsonl | sort -u
+queue-operation
+user
+attachment
+ai-title
+assistant
+last-prompt
+```
+
+**No `stream_event` lines on disk.** Token deltas are stdout-only. `session-tail` watching the JSONL cannot become a streaming source no matter what flags get passed at invocation. This confirms `session-tail.ts:16` is right that "per-token text deltas are NOT in this file" — but `session-tail.ts:5` is wrong about the *reason*. Streaming events exist, just on a different transport.
+
+### 1.6 Codebase docstrings to update if/when this lands
+
+`telegram-plugin/session-tail.ts:5-8` and `telegram-plugin/pty-tail.ts:5-9` both
+state Claude Code doesn't support `--output-format stream-json` in `--channels`
+mode. The corrected nuance: it doesn't expose stream-json from a `--channels`
+**interactive** session, but `--print` mode does support stream-json end-to-end
+when MCP servers are registered via `--mcp-config`. Path C is exactly the
+migration to that mode.
+
+---
+
+## 2. The architectural change in detail
+
+### 2.1 Current architecture (today)
+
+```
+┌────────────────────────────┐     ┌──────────────────────────┐
+│ switchroom-{name}.service  │     │ switchroom-{name}-       │
+│                            │     │ gateway.service          │
+│  start.sh                  │     │                          │
+│  └─ exec claude            │     │  bun gateway.ts          │
+│      --channels server:    │     │  ├─ Telegram bot         │
+│      switchroom-telegram   │     │  ├─ progress card        │
+│      ...                   │     │  ├─ slot banner          │
+│      └─ child MCP plugin   │◀────┤  └─ IPC socket           │
+│         (server.ts dynamic │ IPC │                          │
+│          imports bridge.ts │     │                          │
+│          when gateway      │     │                          │
+│          socket detected)  │     │                          │
+└────────────────────────────┘     └──────────────────────────┘
+        │                                    ▲
+        ▼                                    │
+   service.log (TUI capture)         Telegram inbound
+   ├─ session-tail watches JSONL    polls bot.getUpdates()
+   │  (whole messages)
+   └─ pty-tail watches log
+      (TUI extraction — broken
+       on current Claude Code)
+```
+
+Inbound Telegram → gateway polls → IPC → bridge.ts → MCP NotificationHandler → claude session.
+Outbound: model calls `reply` MCP tool → server.ts/bridge.ts → IPC → gateway → Telegram API.
+
+The MCP plugin lives inside Claude Code as a child process. It has dual identity — as a "channel" it receives inbound notifications; as an "MCP server" it exposes the `reply`/`stream_reply`/`react` tools. Both come from the same `server.ts` codebase, switched on whether the gateway socket is reachable (when reachable: act as bridge; when not: legacy monolith).
+
+### 2.2 Proposed architecture (Path C)
+
+```
+┌────────────────────────────┐     ┌──────────────────────────┐
+│ switchroom-{name}-         │     │ switchroom-{name}-       │
+│ daemon.service             │     │ gateway.service          │
+│                            │     │                          │
+│  bridge-daemon.ts          │     │  bun gateway.ts          │
+│  ├─ spawns claude --print  │     │  ├─ Telegram bot         │
+│  │  --input-format         │     │  ├─ progress card        │
+│  │   stream-json           │     │  ├─ slot banner          │
+│  │  --output-format        │◀────┤  └─ IPC socket           │
+│  │   stream-json           │ IPC │                          │
+│  │  --include-partial-     │     │                          │
+│  │   messages              │     │                          │
+│  │  --include-hook-events  │     │                          │
+│  │  --mcp-config <path>    │     │                          │
+│  │                         │     │                          │
+│  ├─ claude.stdin           │     │                          │
+│  │  ◀── Telegram inbound   │     │                          │
+│  │      (via gateway IPC)  │     │                          │
+│  │                         │     │                          │
+│  └─ claude.stdout          │     │                          │
+│     ──→ stream events      │     │                          │
+│         ├─ content_block_  │     │                          │
+│         │  delta → live    │     │                          │
+│         │  text streaming  │     │                          │
+│         ├─ tool_use →      │     │                          │
+│         │  progress card   │     │                          │
+│         └─ result →        │     │                          │
+│            turn complete   │     │                          │
+└────────────────────────────┘     └──────────────────────────┘
+```
+
+Key change: a new **bridge-daemon** process owns the claude lifecycle. It spawns claude with stream-json mode, pumps Telegram inbound to its stdin, and consumes stream events from its stdout. The MCP plugin is loaded via `--mcp-config` so the model still has `reply`/`stream_reply`/`react` tools — but those tools now only push to Telegram via the gateway IPC (no channels-as-inbound path needed because the daemon owns inbound).
+
+Per-token streaming: the bridge-daemon edits the pre-allocated Telegram draft on each `content_block_delta` event. The model's eventual `reply`/`stream_reply` tool call is the canonical "this is the answer" signal that finalizes the message; until then, the deltas drive a live preview.
+
+### 2.3 The model-vs-deltas coordination problem
+
+The model emits text via `content_block_delta` (every token) AND eventually calls `reply(text="<full text>")` or `stream_reply(text="<full text>", done=true)`. The bridge has the same text from two sources. Choices:
+
+**A. Deltas-only.** Render text as it streams; ignore the eventual `reply`/`stream_reply` text content (just observe the tool call as "the model has decided this turn is done"). Risk: the model's tool call is the explicit "send this" signal — bypassing it changes semantics. Some agents may rely on the tool's `format`/`reply_to`/`message_thread_id` parameters for correctness (e.g. forum-topic threading). Lose those if we ignore the tool.
+
+**B. Tool-call-only.** Stream nothing live; wait for the model's final `reply`/`stream_reply`. Loses the streaming UX entirely — defeats the whole exercise.
+
+**C. Deltas-as-preview, tool-call-as-finalize.** Stream deltas into the placeholder draft as they arrive; when the model calls `reply`/`stream_reply`, finalize the draft with that tool's text + parameters (potentially identical text, but with the tool's `format`/`reply_to`/etc. applied). This is the Claude.ai pattern. Pre-existing draft consumption logic in `stream-reply-handler.ts` is close to what's needed.
+
+Recommendation: **C**. Most flexible, preserves tool semantics, gives the streaming UX.
+
+Edge case: model emits a partial reply via deltas, then *doesn't* call `reply` (e.g. interrupted, error, refused). Need a turn-end watchdog that converts the partial preview into a finalized message (or deletes the placeholder). The existing orphan-cleanup logic on `turn_end` handles a similar case today.
+
+### 2.4 Inbound message format
+
+Claude Code expects `--input-format=stream-json` lines like:
+
+```json
+{"type":"user","message":{"role":"user","content":"hello"}}
+```
+
+The bridge-daemon translates Telegram inbound to this format. Image/document attachments need to be inlined as base64 or referenced by file path (Anthropic API supports both). The MCP `download_attachment` tool currently handles this lazily; the daemon needs to either inline at inbound time or keep the tool.
+
+Special cases the existing `--channels` integration handles that need explicit migration:
+
+- **Forum topic threading** (`message_thread_id` is a Telegram concept; Claude has no native field for it). Today wrapped in the channel's metadata. Daemon needs to inject it into the prompt as an XML wrapper or system note so the model knows what topic to reply to.
+- **Reply-to context** (Telegram quote-reply). Same treatment.
+- **Multi-message coalescing** (gateway buffers rapid messages). Already happens upstream of the daemon.
+
+### 2.5 Why the bridge-daemon needs to be a NEW service
+
+The existing `bridge.ts` is a child process of `claude` (loaded via dynamic import from `server.ts` when the gateway socket is detected). It can't spawn the parent claude — that's the wrong direction. The daemon mode requires reversing the parent/child relationship: the bridge owns claude, claude is the child.
+
+Practical implication: switchroom adds a third systemd unit per agent (`switchroom-{name}-daemon.service`), distinct from both the gateway and the legacy agent service. Per-agent restart semantics need rework.
+
+---
+
+## 3. Per-feature impact analysis
+
+Each switchroom feature is currently implemented somewhere along the path `Telegram → gateway → IPC → bridge → claude → MCP tool → IPC → gateway → Telegram`. Migrating to stream-json daemon mode shifts who owns what. This table is the work.
+
+| Feature | Current home | Daemon-mode home | Risk |
+|---|---|---|---|
+| **Inbound Telegram → claude** | `--channels` MCP NotificationHandler | bridge-daemon writes stream-json to claude's stdin | Medium — wrap forum-topic + reply-to context as prompt prefix |
+| **Outbound `reply` / `stream_reply` / `react` tools** | MCP server inside claude's child process | Same MCP server, registered via `--mcp-config`; communicates with gateway via existing IPC | Low — gateway IPC contract unchanged |
+| **Status reactions** (👀 🤔 👨‍💻 👍) | Driven by session-tail JSONL events | Driven by stream-json `tool_use` / `result` events from claude's stdout | Low — equivalent event shapes, different source |
+| **Pre-alloc placeholder** (`🔵 thinking`) | Gateway pre-allocates on inbound | Same — gateway still owns chat-id-keyed pre-alloc map | Low |
+| **Per-token streaming** | Doesn't work | bridge-daemon edits placeholder on each `content_block_delta` | New capability, low risk |
+| **`update_placeholder` IPC** (recall.py) | Hindsight hook → IPC → gateway | Unchanged — Hindsight hook still fires via `--include-hook-events` consumption OR directly via IPC | Low |
+| **Progress card** | session-tail `tool_use` events → reducer → renderer | stream-json `tool_use` events → same reducer → same renderer | Low — change input source, same reducer |
+| **Sub-agents (Task)** | Spawned as child claude processes inheriting parent env | Verify they inherit stream-json mode; verify `parent_tool_use_id` correlation works | **MEDIUM — needs spike validation** |
+| **Hooks (Pre/PostToolUse, Stop)** | Run in claude's process, fire to disk + IPC | `--include-hook-events` exposes lifecycle on stream | Low — better visibility |
+| **Slash commands** | Typed in TUI, executed by claude | Need to verify how they're invoked via stream-json input | **MEDIUM — needs spike** |
+| **Permission prompts** | Interactive TUI → user types y/n | stream-json must emit a permission_request event the bridge can route to Telegram inline-keyboard, then write back the answer to stdin | **HIGH — needs spike, may not exist** |
+| **OAuth flow** (login, reauth, code paste) | `claude` invokes browser, user pastes code into TUI | Daemon-mode browsers? Pasting into stdin? | **HIGH — likely needs separate `--print` invocation for OAuth, or daemon kept on legacy mode for auth flows** |
+| **Slot-pool failover** (auto-fallback on quota) | Restart agent service with new slot env | Restart bridge-daemon with new slot env (same shape) | Low |
+| **Vault** (passphrase entry, secret materialization) | Telegram inbound → MCP plugin → vault CLI | Same — vault is separate process; daemon just routes | Low |
+| **OAuth code redaction** (PR #490) | MCP plugin's `redactAuthCodeMessage` after exchange | Same — runs in MCP server which still exists | Low |
+| **`/restart`, `/reset`, `/new` commands** | Gateway intercepts, sends signal to agent service | Same intercept, restart bridge-daemon instead | Low |
+| **Resume / continue** | `claude --continue` re-attaches to most recent session in cwd | Verify `--continue` works for stream-json daemon (session_id in init event suggests yes, but unverified) | **MEDIUM — needs spike** |
+| **Sub-agent SQLite registry** | `bridge.ts` watches sub-agent JSONLs | Daemon emits sub-agent events on stream OR daemon also runs the JSONL watchers | Medium — coordinate event sources |
+| **`SWITCHROOM_HANDOFF_SHOW_LINE`** (handoff briefing) | Read by start.sh, baked into `--append-system-prompt` | Same — daemon constructs the same prompt prefix | Low |
+
+Five rows flagged MEDIUM or HIGH risk. Each needs a spike or a design decision before commitment.
+
+---
+
+## 4. Migration strategy options
+
+### 4.1 Big-bang migration
+
+All agents flip to daemon mode in one release. New systemd unit replaces old. Rollback = revert + restart.
+
+**Pros**: clean codebase post-migration; no dual-mode bookkeeping.
+**Cons**: blast radius is the entire fleet; one regression breaks every agent simultaneously; harder to A/B compare.
+
+### 4.2 Per-agent opt-in via config
+
+Add `agent.streaming_mode: 'classic' | 'daemon'` to `switchroom.yaml`. New agents default to daemon; existing agents stay classic until manually flipped.
+
+**Pros**: incremental rollout; easy to revert per-agent; can validate on one chat-class agent before rolling to all.
+**Cons**: codebase has to support both modes for a long time; tests need to cover both; config knob is one more thing to misuse.
+
+### 4.3 Side-by-side A/B (parallel daemon for streaming preview only)
+
+Keep classic agent doing all the work; spawn a SECOND `claude --print` process per agent JUST for the streaming text preview. The preview stream feeds the placeholder; the canonical message comes from the classic agent's `reply` tool.
+
+**Pros**: zero risk to existing functionality; pure additive.
+**Cons**: ~2× compute per agent (two model sessions); coordination is fragile (the two sessions can drift); MCP state would need to mirror; rejected on cost grounds.
+
+**Recommended: 4.2 (per-agent opt-in)** with a chat-class agent (e.g. `clerk` or `klanker`) as the first migrant. After 1-2 weeks of stability, expand. After 4-6 weeks of the daemon being the default for all new agents and most existing ones, deprecate classic mode in a release that bumps the major version.
+
+---
+
+## 5. Spike plan — what must work end-to-end
+
+Before committing to a migration, build a spike that proves the core path. Spike acceptance:
+
+1. **Spawn-and-converse**: bridge-daemon spawns claude in stream-json mode, sends a Telegram-equivalent inbound to its stdin, receives `content_block_delta` events on stdout, sends an outbound to Telegram via the MCP `reply` tool. Round-trip works on a real Telegram bot.
+
+2. **Per-token preview**: as the model generates, the placeholder draft updates character-by-character in Telegram (verified by client screenshots/recording). The 600ms throttle from `draft-stream.ts` keeps under Telegram's edit-rate limit.
+
+3. **Tool-call finalize**: when the model calls `reply` or `stream_reply` after streaming via deltas, the placeholder is finalized with the tool's text + format. No duplicate messages.
+
+4. **Sub-agent dispatch**: model dispatches a `Task(subagent_type='worker', ...)` and gets a result. The sub-agent's events (parent_tool_use_id correlation) flow correctly.
+
+5. **Hook firing**: a UserPromptSubmit hook (recall.py shape) fires and emits an event the bridge can act on.
+
+6. **Permission prompt**: a tool requiring permission triggers something the bridge can present to Telegram (inline keyboard) and feed an answer back.
+
+7. **`--continue` resumption**: kill the bridge-daemon mid-conversation, restart it with `--continue`, send a new turn — model has prior context.
+
+8. **OAuth flow**: log in, paste code, restart, agent runs. May require keeping classic mode for the auth flow itself (daemon mode after auth is established).
+
+If all 8 work in a 1-2 week spike, Path C is feasible and the per-agent migration can begin. If 6 (permissions) or 8 (OAuth) hit a hard blocker, Path C is shelved or scoped down to "streaming-only sidecar" patterns.
+
+---
+
+## 6. Open questions / unknowns
+
+These need answers before commit:
+
+1. **Does `--include-hook-events` emit hooks as inline events that can replace the file-based hook lifecycle?** Or do hooks still need to run as subprocesses and write to disk? If the latter, the daemon doesn't simplify hooks.
+2. **What does a permission prompt look like in stream-json?** Need to test interactively or find docs. If there's no event for it, switchroom needs `--permission-mode auto` or `bypassPermissions` for daemon-mode agents (loss of safety surface for some users).
+3. **How does `Task(subagent_type)` interact with stream-json?** Does the sub-agent get its own stream? Does the parent see the sub-agent's events? `parent_tool_use_id` in the init event suggests yes but unverified.
+4. **What's the real performance / cost difference?** Stream-json mode emits more events to stdout — does that increase per-turn latency or memory usage? Probably negligible but unmeasured.
+5. **What happens on session compaction?** Claude Code automatically compacts long sessions; verify the daemon survives compaction without losing context.
+6. **What happens if the Hindsight HTTP server isn't ready when claude boots?** Today `start.sh` has a `HINDSIGHT_WAIT` loop. The daemon needs the same.
+7. **What about IDE integration?** Switchroom doesn't need it but verifying it doesn't conflict.
+8. **Does `switchroom auth code` still work?** OAuth code paste likely needs to remain on a non-daemon code path.
+
+---
+
+## 7. Risks
+
+| Risk | Likelihood | Impact | Mitigation |
+|---|---|---|---|
+| Permission prompts can't be plumbed through stream-json | Medium | High — would force `bypassPermissions` for daemon agents | Spike early; if blocked, daemon mode is opt-in for sandboxed agents only |
+| Sub-agent correlation breaks | Low | High — sub-agent UX is core | Spike early; existing `parent_tool_use_id` field suggests this works |
+| OAuth flow incompatible with daemon mode | High | Medium — auth happens once, can be a separate code path | Keep classic mode for auth flows; daemon mode after auth is established |
+| Claude Code stream-json shape changes between versions | Medium | Medium — Anthropic occasionally refines | Pin to a Claude Code version; integration test against captured fixtures (per `HARNESS.md` Pattern 6) |
+| Daemon process crash leaves orphaned session | Medium | Low — `--continue` recovers | systemd `Restart=on-failure` + boot-time `--continue` (already pattern) |
+| Telegram edit-rate limit hits during fast streams | Low | Low — existing 600ms throttle handles | Reuse `draft-stream.ts` throttle; same code path |
+| Coordination between deltas + model's `reply` call ships duplicate messages | Medium | Medium — primary UX bug class | "Deltas as preview, reply as finalize" pattern (3.3 option C); regression tests |
+| Migration breaks existing PR review queue (open branches assume classic mode) | Medium | Low | Coordinate with active PR authors before flipping defaults |
+
+---
+
+## 8. Decision points
+
+Before this PR could be merged, the following need to be decided:
+
+1. **Is per-token streaming worth this scope?** If "good enough" UX comes from Path A (heartbeat placeholder with semantic chunks every 3-5s), Path C may be over-engineering for the stated goal.
+2. **Is switchroom willing to own the IO loop?** Today switchroom inherits Claude Code's TUI lifecycle and just instruments around it. Path C makes switchroom the parent process — closer to OpenClaw's model. Architectural shift.
+3. **Per-agent opt-in vs big-bang?** §4 lays out the options; recommendation is opt-in.
+4. **How long does the spike get?** Suggest 1-2 weeks for the 8 acceptance items in §5; if they all work, commit; if not, retreat to Path A with confidence.
+5. **What's the deprecation timeline for classic mode?** If we're keeping it long-term, code complexity stays; if we're aiming to delete it, deprecation announcement is a separate user-facing comm.
+
+---
+
+## 9. Alternatives considered
+
+### 9.1 Path A — heartbeat placeholder (RECOMMENDED for now)
+
+50 LOC, no architectural change. Edit the placeholder every 3-5s with elapsed time + last tool used. User sees `🔵 thinking · 5s · reading CLAUDE.md` etc. Doesn't deliver per-token streaming but never silent for >5s — matches the OpenClaw "no silent gap > 2s" rule from #303.
+
+**Why prefer over Path C**: ships in a day; zero migration risk; achieves the stated UX goal ("not silent for so long"); doesn't preclude Path C later.
+
+### 9.2 Path B — stream-json sidecar reader
+
+Run a SECOND claude process per agent in `--print --output-format=stream-json` mode just to capture stream events for streaming UX. Verified during this investigation: this doesn't work because there's no way to attach a stream reader to an existing interactive session — the secondary instance would be a separate conversation, not an observer of the primary.
+
+**Rejected**.
+
+### 9.3 Fix V1Extractor for new TUI format
+
+Reverse-engineer current Claude Code TUI rendering, update `pty-tail.ts:111-216` to match. Would restore PTY-based streaming.
+
+**Rejected**: structurally fragile; the test added in PR #507 already documents this pattern as the wrong tool. Even fixed, breaks again on next Claude Code TUI change.
+
+### 9.4 Force model to use `stream_reply` via stricter prompting / hooks
+
+Already the default `profiles/default/CLAUDE.md` instruction. Model compliance is variable. A `Stop` hook could veto turns that didn't call `stream_reply` — heavyweight, brittle, and PR #483 already removed the rejection that punished it.
+
+**Rejected as standalone**, but compatible with Path A or Path C.
+
+### 9.5 OpenClaw model — custom runtime
+
+Re-implement Claude's agent loop with full SDK control. Switchroom explicitly chose NOT to do this (`docs/vs-openclaw.md`); reversing that decision is a much bigger conversation than streaming UX.
+
+**Out of scope** — would not be the same product.
+
+---
+
+## 10. Recommended path forward
+
+1. **Ship Path A (heartbeat placeholder) immediately** — closes the user-visible gap with low risk. Tracks in a separate small PR.
+2. **File this design doc as an architectural issue** linking to all the validation evidence. Solicit feedback before any spike.
+3. **Schedule a 1-2 week spike** if/when the team commits to Path C, against the 8 acceptance items in §5. Spike outcome is binary: green-light migration, or shelve and stick with Path A long-term.
+4. **Update the codebase docstrings** (`session-tail.ts:5`, `pty-tail.ts:5`) with the corrected nuance about stream-json availability — independent of any migration commitment.
+5. **Capture this investigation in `HARNESS.md`** as a worked example of the "validate assumptions against the upstream surface" pattern.
+
+The validation finding that Claude Code DOES support per-token streaming changes the long-term architecture conversation. Whether to act on it now is a strategic question, not a tactical one.
+
+---
+
+## 11. Spike results — 2026-05-01
+
+Spike executed against the 8 acceptance items in §5. Each test was a real
+shell invocation against `claude` 2.1.126 with shell-captured output.
+Findings supersede earlier theory.
+
+### 11.1 Pass/fail summary
+
+| # | Item | Result | Notes |
+|---|---|---|---|
+| 1 | Spawn-and-converse | ✅ PASS | Multi-turn over stdin works; init event lists tools+MCP servers correctly |
+| 2 | Per-token preview | ✅ PASS | 200-word reply: 21 deltas at ~3/sec, ~50 chars/delta. With 600ms throttle = ~1-2 edits/sec, well under Telegram's 30/sec hard cap |
+| 3 | Tool-call finalize coordination | ✅ PASS | `content_block_start.event.content_block.type` distinguishes text/thinking/tool_use blocks → bridge can stream the right content |
+| 4 | Sub-agents (Task) | ✅ PASS | `parent_tool_use_id` correlates sub-agent events to their parent dispatch; full event lifecycle observable |
+| 5 | Hooks fire and emit events | ✅ PASS | `--include-hook-events` produces `system/hook_started` + `system/hook_response` with full stdout/stderr/exit. Richer than today's IPC-only path |
+| 6 | **Permission prompts** | ❌ **HARD BLOCKER** | `--print --permission-mode default` auto-denies tools requiring permission. NO event in stream surfaces the request. The `notifications/claude/channel/permission_request` side-channel does NOT fire when running in `--print` mode, even with `--dangerously-load-development-channels` set |
+| 7 | `--resume <session_id>` | ✅ PASS | Conversation context survives across `claude --print` invocations. Tested: set "favourite color is teal" in turn 1, killed session, restarted with `--resume`, model recalled "teal" correctly |
+| 8 | OAuth flow | ⚠ DEFERRED | Likely requires keeping classic mode for the auth flow itself. Not blocking the architectural decision but adds operational complexity |
+
+**6 of 7 testable items pass. 1 hard blocker (#6).**
+
+### 11.2 Spike #6 — the permission blocker in detail
+
+Test setup: `claude --print --verbose --output-format=stream-json --input-format=stream-json --include-partial-messages --include-hook-events --dangerously-load-development-channels server:switchroom-telegram --permission-mode default` with a prompt asking the model to write a file via the Write tool.
+
+Result:
+```
+permission_denials: [{
+  "tool_name": "Write",
+  "tool_use_id": "toolu_...",
+  "tool_input": {"file_path": "...", "content": "hello spike"}
+}]
+
+tool_result.is_error: true
+tool_result.content: "Claude requested permissions to write to ...,
+  but you haven't granted it yet."
+```
+
+The Write tool was auto-denied. There was NO event in the stream that:
+- Asked for user approval
+- Allowed the bridge to route the request to Telegram
+- Took an answer back to grant permission
+
+The existing switchroom Telegram inline-keyboard permission flow at `server.ts:1192-1228` depends on the `notifications/claude/channel/permission_request` MCP notification. **That notification doesn't fire in `--print` mode** — even when `--dangerously-load-development-channels` is set. The denial happens upstream of the channel notification, so there's no way to surface the prompt to the user.
+
+### 11.3 What this means for Path C
+
+The blocker is real but not necessarily fatal. Three response strategies, in order of practical viability:
+
+**Strategy A — `--permission-mode bypassPermissions` for daemon-mode agents**
+- Simplest: agents in daemon mode skip all permission checks.
+- Acceptable if all agents run on a trusted single-user host and tools are limited to a known-safe set (Telegram MCP, Hindsight MCP, Read/Grep/Glob).
+- Real safety regression for agents that might use Bash/Edit/Write with broad access. Today's interactive Telegram approval flow stops the model from accidentally `rm -rf` or sending stupid emails; bypassPermissions removes that gate.
+- For switchroom's chat-class agents (clerk, klanker, gymbro, finn — mostly use MCP tools, rarely Bash/Edit) this is probably fine.
+- For development-y agents that legitimately edit code or run arbitrary commands, this is a meaningful loss.
+
+**Strategy B — pre-approve every needed tool in `settings.json`**
+- `permissions.allow` array can pre-approve tool patterns: `["Read", "Grep", "Bash(git status)", ...]`
+- Agent never asks the user because everything is pre-approved.
+- High maintenance burden — every new tool/command needs to be added explicitly.
+- Defeats the dynamic-approval benefit but preserves "no surprise approvals" safety.
+
+**Strategy C — file an upstream feature request to Anthropic**
+- Ask Claude Code to emit `permission_request` as a stream event in `--print` mode that an external tool (the bridge) can answer via stdin.
+- This is the architecturally correct fix. Other tools using `--print --output-format=stream-json` (the agent SDK pattern) would benefit too.
+- Realistic timeline: weeks-to-months, with non-zero risk of "won't ship."
+- Recommended regardless of which path switchroom takes locally.
+
+### 11.4 Updated verdict on Path C
+
+**Pre-spike confidence**: 70% reliable, conditional on 3 unknowns clearing.
+**Post-spike confidence**: ~85% for chat-class agents in `bypassPermissions` mode. ~50% for general-purpose agents without an upstream change.
+
+The remaining items (sub-agents, --resume, hooks) all came in cleaner than expected. Hooks in particular are a meaningful upgrade — `system/hook_started` + `system/hook_response` events expose the lifecycle on the wire, replacing the file-based IPC patterns with first-class stream observability.
+
+But the permission-prompt blocker is a real architectural cost. Path C's quality depends on which of A/B/C in §11.3 the team accepts.
+
+### 11.5 Revised recommendation (post-spike)
+
+1. **Ship Path A (heartbeat placeholder) immediately** — unchanged from §10.
+2. **Defer Path C migration** until either:
+   - Anthropic adds permission events to stream-json (Strategy C in §11.3), OR
+   - Switchroom commits to `bypassPermissions` mode for chat-class agents and accepts the safety tradeoff.
+3. **Update the architectural risk register** — the design doc previously listed permission prompts as MEDIUM risk requiring spike validation. Spike confirms it's HIGH risk requiring an explicit policy decision before commitment.
+
+The spike was worth running: it converted a HIGH-risk unknown into a HIGH-risk known, which is exactly what spikes are for. Path C remains feasible but the quality bar is now lower than originally projected, and the chat-class-only scope is more honest than "all agents."
+
+### 11.6 What's been actively retired by this validation
+
+- The PTY-tail / V1Extractor approach is now confirmed as the wrong architectural direction (it predates the per-token streaming surface that already exists).
+- The "session-tail can become a streaming source" idea is also retired — verified by inspecting the on-disk JSONL: it contains `assistant`/`user`/`system` events but no `stream_event` lines, regardless of `--include-partial-messages` flag at invocation.
+- The middle-ground "heartbeat + tool labels via session-tail" hybrid is the WORST option (paying maintenance cost for moderate UX). Either pure heartbeat (no extractor) or commit to Path C.