clawdbot 2026.1.4-1

package/docs/refactor/canvas-a2ui.md

@@ -0,0 +1,93 @@
+---
+summary: "Refactor: host A2UI from the Gateway (HTTP), remove app-bundled shells"
+read_when:
+  - Refactoring Canvas/A2UI ownership or assets
+  - Moving UI rendering from native bundles into the Gateway
+  - Updating node canvas navigation or A2UI command flows
+---
+
+# Canvas / A2UI — HTTP-hosted from Gateway
+
+Status: Implemented · Date: 2025-12-20
+
+## Goal
+- Make the **Gateway (TypeScript)** the single owner of A2UI.
+- Remove **app-bundled A2UI shells** (macOS, iOS, Android).
+- A2UI renders only when the **Gateway is reachable** (acceptable failure mode).
+
+## Decision
+All A2UI HTML/JS assets are **served by the Gateway canvas host** on
+`canvasHost.port` (default `18793`), bound to the **bridge interface**. Nodes
+(mac/iOS/Android) **navigate to the advertised `canvasHostUrl`** before applying
+A2UI messages. No local custom-scheme or bundled fallback remains.
+
+## Why
+- One source of truth (TS) for A2UI rendering.
+- Faster iteration (no app release required for A2UI updates).
+- iOS/Android/macOS all behave identically.
+
+## New behavior (summary)
+1) `canvas.a2ui.*` on any node:
+   - Ensure Canvas is visible.
+   - Navigate the node WebView to the Gateway A2UI URL.
+   - Apply/reset A2UI messages once the page is ready.
+2) If Gateway is unreachable:
+   - A2UI fails with an explicit error (no fallback).
+
+## Gateway changes
+
+### Serve A2UI assets
+Add A2UI HTML/JS to the Gateway Canvas host (standalone HTTP server on
+`canvasHost.port`), e.g.:
+
+```
+/__clawdbot__/a2ui/           -> index.html
+/__clawdbot__/a2ui/a2ui.bundle.js -> bundled A2UI runtime
+```
+
+Serve Canvas files at `/__clawdbot__/canvas/` and A2UI at `/__clawdbot__/a2ui/`.
+Use the shared Canvas host handler (`src/canvas-host/server.ts`) to serve these
+assets and inject the action bridge + live reload if desired.
+
+### Canonical host URL
+The Gateway exposes a **canonical** `canvasHostUrl` in hello/bridge payloads
+so nodes don’t need to guess.
+
+## Node changes (mac/iOS/Android)
+
+### Navigation path
+Before applying A2UI:
+- Navigate to `${canvasHostUrl}/__clawdbot__/a2ui/`.
+
+### Remove bundled shells
+Remove all fallback logic that serves A2UI from local bundles:
+- macOS: remove custom-scheme fallback for `/__clawdbot__/a2ui/`
+- iOS/Android: remove packaged A2UI assets and "default scaffold" assumptions
+
+### Error behavior
+If `canvasHostUrl` is missing or unreachable:
+- `canvas.a2ui.push/reset` returns a clear error:
+  - `A2UI_HOST_UNAVAILABLE` or `A2UI_HOST_NOT_CONFIGURED`
+
+## Security / transport
+- For non-TLS Gateway URLs (http), iOS/Android will need ATS exceptions.
+- For TLS (https), prefer WSS + HTTPS with a valid cert.
+
+## Implementation plan
+1) Gateway
+   - Add A2UI assets under `src/canvas-host/`.
+   - Serve them at `/__clawdbot__/a2ui/` (align with existing naming).
+   - Serve Canvas files at `/__clawdbot__/canvas/` on `canvasHost.port`.
+   - Expose `canvasHostUrl` in handshake + bridge hello payloads.
+2) Node runtimes
+   - Update `canvas.a2ui.*` to navigate to `canvasHostUrl`.
+   - Remove custom-scheme A2UI fallback and bundled assets.
+3) Tests
+   - TS: verify `/__clawdbot__/a2ui/` responds with HTML + JS.
+   - Node: verify A2UI fails when host is unreachable and succeeds when reachable.
+4) Docs
+   - Update `docs/mac/canvas.md`, `docs/ios/spec.md`, `docs/android/connect.md`
+     to remove local fallback assumptions and point to gateway-hosted A2UI.
+
+## Notes
+- iOS/Android may still require ATS exceptions for `http://` canvas hosts.

package/docs/refactor/cli-unification.md

@@ -0,0 +1,64 @@
+---
+summary: "Refactor: unify on the clawdbot CLI + gateway-first control; retire clawdbot-mac"
+read_when:
+  - Removing or replacing the macOS CLI helper
+  - Adding node capabilities or permissions metadata
+  - Updating macOS app packaging/install flows
+---
+
+# CLI unification (clawdbot-only)
+
+Status: active refactor · Date: 2025-12-20
+
+## Goals
+- **Single CLI**: use `clawdbot` for all automation (local + remote). Retire `clawdbot-mac`.
+- **Gateway-first**: all agent actions flow through the Gateway WebSocket + node.invoke.
+- **Permission awareness**: nodes advertise permission state so the agent can decide what to run.
+- **No duplicate paths**: remove macOS control socket + Swift CLI surface.
+
+## Non-goals
+- Keep legacy `clawdbot-mac` compatibility.
+- Support agent control when no Gateway is running.
+
+## Key decisions
+1) **No Gateway → no control**
+   - If the macOS app is running but the Gateway is not, remote commands (canvas/run/notify) are unavailable.
+   - This is acceptable to keep one network surface.
+
+2) **Remove ensure-permissions CLI**
+   - Permissions are **advertised by the node** (e.g., screen recording granted/denied).
+   - Commands will still fail with explicit errors when permissions are missing.
+
+3) **Mac app installs/symlinks `clawdbot`**
+   - Bundle a standalone `clawdbot` binary in the app (bun-compiled).
+   - Install/symlink that binary to `/usr/local/bin/clawdbot` and `/opt/homebrew/bin/clawdbot`.
+   - No `clawdbot-mac` helper remains.
+
+4) **Canvas parity across node types**
+   - Use `node.invoke` commands consistently (`canvas.present|navigate|eval|snapshot|a2ui.*`).
+   - The TS CLI provides convenient wrappers so agents never have to craft raw `node.invoke` calls.
+
+## Command surface (new/normalized)
+- `clawdbot nodes invoke --command canvas.*` remains valid.
+- New CLI wrappers for convenience:
+  - `clawdbot canvas present|navigate|eval|snapshot|a2ui push|a2ui reset`
+- New node commands (mac-only initially):
+  - `system.run` (shell execution)
+  - `system.notify` (local notifications)
+
+## Permission advertising
+- Node hello/pairing includes a `permissions` map:
+  - Example keys: `screenRecording`, `accessibility`, `microphone`, `notifications`, `speechRecognition`.
+  - Values: boolean (`true` = granted, `false` = not granted).
+- Gateway `node.list` / `node.describe` surfaces the map.
+
+## Gateway mode + config
+- Gateways should only auto-start when explicitly configured for **local** mode.
+- When config is missing or explicitly remote, `clawdbot gateway` should refuse to auto-start unless forced.
+
+## Implementation checklist
+- Add bun-compiled `clawdbot` binary to macOS app bundle; update codesign + install flows.
+- Remove `ClawdbotCLI` target and control socket server.
+- Add node command(s) for `system.run` and `system.notify` on macOS.
+- Add permission map to node hello/pairing + gateway responses.
+- Update TS CLI + docs to use `clawdbot` only.

package/docs/refactor/gateway-client.md

@@ -0,0 +1,31 @@
+---
+summary: "Refactor notes for the macOS gateway client typed API migration (Dec 2025)."
+read_when:
+  - Refactoring macOS gateway client or typed gateway methods
+  - Auditing agent routing or channel semantics
+---
+
+# Gateway Client Refactor (Dec 2025)
+
+Goal: remove stringly-typed gateway calls from the macOS app, centralize routing/channel semantics, and improve error handling.
+
+## Progress
+
+- [x] Fold legacy “AgentRPC” into `GatewayConnection` (single layer; no separate client object).
+- [x] Typed gateway API: `GatewayConnection.Method` + `requestDecoded/requestVoid` + typed helpers (status/agent/chat/cron/etc).
+- [x] Centralize agent routing/channel semantics via `GatewayAgentChannel` + `GatewayAgentInvocation`.
+- [x] Improve gateway error model (structured `GatewayResponseError` + decoding errors include method).
+- [x] Migrate mac call sites to typed helpers (leave only intentionally dynamic forwarding paths).
+- [x] Convert remaining UI raw channel strings to `GatewayAgentChannel` (Cron editor).
+- [x] Cleanup naming: rename remaining tests/docs that still reference “RPC/AgentRPC”.
+
+### Notes
+
+- Intentionally string-based:
+  - `BridgeServer` dynamic request forwarding (method is data-driven).
+  - `ControlChannel` request wrapper (generic escape hatch).
+
+## Notes / Non-goals
+
+- No functional behavior changes intended (beyond better errors and removing “magic strings”).
+- Keep changes incremental: introduce typed APIs first, then migrate call sites, then remove old helpers.

package/docs/refactor/gateway.md

@@ -0,0 +1,99 @@
+---
+summary: "Refactor notes for the macOS gateway client: single shared websocket + follow-ups"
+read_when:
+  - Investigating duplicate/stale Gateway WS connections
+  - Refactoring macOS gateway client architecture
+  - Debugging noisy reconnect storms on gateway restart
+---
+# Gateway Refactor Notes (macOS client)
+
+Last updated: 2025-12-12
+
+This document captures the rationale and outcome of the macOS app’s Gateway client refactor: **one shared websocket connection per app process**, with an in-process event bus for server push frames.
+
+Related docs:
+- `docs/refactor/new-arch.md` (overall gateway protocol/server plan)
+- `docs/gateway.md` (gateway operations/runbook)
+- `docs/presence.md` (presence semantics and dedupe)
+- `docs/mac/webchat.md` (WebChat surfaces and debugging)
+
+---
+
+## Background: what was wrong
+
+Symptoms:
+- Restarting the gateway produced a *storm* of reconnects/log spam (`gateway/ws in connect`, `hello`, `hello-ok`) and elevated `clients=` counts.
+- Even with “one panel open”, the mac app could hold tens of websocket connections to `ws://127.0.0.1:18789`.
+
+Root cause (historical bug):
+- The mac app was repeatedly “reconfiguring” a gateway client on a timer (via health polling), creating a new websocket owner each time.
+- Old websocket owners were not fully torn down and could keep watchdog/tick tasks alive, leading to **connection accumulation** over time.
+
+---
+
+## What changed
+
+- **One socket owner:** `GatewayConnection.shared` is the only supported entry point for gateway RPC.
+- **No global notifications:** server push frames are delivered via `GatewayConnection.shared.subscribe(...) -> AsyncStream<GatewayPush>` (no `NotificationCenter` fan-out).
+- **No tunnel side effects:** `GatewayConnection` does not create/ensure SSH tunnels in remote mode; it consumes the already-established forwarded port.
+
+---
+
+## Current architecture (as of 2025-12-12)
+
+Goal: enforce the invariant **“one gateway websocket per app process (per effective config)”**.
+
+Key elements:
+- `GatewayConnection.shared` owns the one websocket and is the *only* supported entry point for app code that needs gateway RPC.
+- Consumers (e.g. Control UI, agent invocations, SwiftUI WebChat) call `GatewayConnection.shared.request(...)` and do not create their own sockets.
+- If the effective connection config changes (local ↔ remote tunnel port, token change), `GatewayConnection` replaces the underlying connection.
+- The transport (`GatewayChannelActor`) is an internal detail and forwards push frames back into `GatewayConnection`.
+- Server-push frames are delivered via `GatewayConnection.shared.subscribe(...) -> AsyncStream<GatewayPush>` (in-process event bus).
+
+Notes:
+- Remote mode requires an SSH control tunnel. `GatewayConnection` **does not** start tunnels; it consumes the already-established forwarded port (owned by `ConnectionModeCoordinator` / `RemoteTunnelManager`).
+
+---
+
+## Design constraints / principles
+
+- **Single ownership:** Exactly one component owns the actual socket and reconnect policy.
+- **Explicit config changes:** Recreate/reconnect only when config changes, not as a side effect of periodic work.
+- **No implicit fan-out sockets:** Adding new UI features must not accidentally add new persistent gateway connections.
+- **Testable seams:** Connection config and websocket session creation should be overridable in tests.
+
+---
+
+## Status / remaining work
+
+- ✅ One shared websocket per app process (per config)
+- ✅ Event streaming moved into `GatewayConnection` (`AsyncStream<GatewayPush>`) and replays latest snapshot to new subscribers
+- ✅ `NotificationCenter` removed for in-process gateway events (ControlChannel / Instances / WebChatSwiftUI)
+- ✅ Remote tunnel lifecycle is not started implicitly by random RPC calls
+- ✅ Payload decoding helpers extracted so UI adapters stay thin
+- ✅ Dedicated resolved-endpoint publisher for remote mode (`GatewayEndpointStore`)
+
+---
+
+## Testing strategy (what we want to cover)
+
+Minimum invariants:
+- Repeated requests under the same config do **not** create additional websocket tasks.
+- Concurrent requests still create **exactly one** websocket and reuse it.
+- Shutdown prevents any reconnect loop after failures.
+- Config changes (token / endpoint) cancel the old socket and reconnect once.
+
+Nice-to-have integration coverage:
+- Multiple “consumers” (Control UI + agent invocations + SwiftUI WebChat) all call through the shared connection and still produce only one websocket.
+
+Additional coverage added (macOS):
+- Subscribing after connect replays the latest snapshot.
+- Sequence gaps emit an explicit `GatewayPush.seqGap(...)` before the corresponding event.
+
+---
+
+## Debug notes (operational)
+
+When diagnosing “too many connections”:
+- Prefer counting actual TCP connections on port 18789 and grouping by PID to see which process is holding sockets.
+- Gateway `--verbose` prints *every* connect/hello and event broadcast; use it only when needed and filter output if you’re just sanity-checking.

package/docs/refactor/new-arch.md

@@ -0,0 +1,171 @@
+---
+summary: "Implementation plan for the new gateway architecture and protocol"
+read_when:
+  - Executing the gateway refactor
+---
+# New Gateway Architecture – Implementation Plan (detailed)
+
+Last updated: 2025-12-09
+
+Goal: replace legacy gateway/stdin/TCP control with a single WebSocket Gateway, typed protocol, and first-frame snapshot. No backward compatibility.
+
+---
+
+## Phase 0 — Foundations
+- **Naming**: CLI subcommand `clawdbot gateway`; internal namespace `Gateway`.
+- **Protocol folder**: create `protocol/` for schemas and build artifacts. ✅ `src/gateway/protocol`.
+- **Schema tooling**:
+  - Prefer **TypeBox** (or ArkType) as source-of-truth types. ✅ TypeBox in `schema.ts`.
+  - `pnpm protocol:gen`: emits JSON Schema (`dist/protocol.schema.json`). ✅
+  - `pnpm protocol:gen:swift`: generates Swift `Codable` models (`apps/macos/Sources/ClawdbotProtocol/GatewayModels.swift`). ✅
+  - AJV compile step for server validators. ✅
+- **CI**: add a job that fails if schema or generated Swift is stale. ✅ `pnpm protocol:check` (runs gen + git diff).
+
+## Phase 1 — Protocol specification
+- Frames (WS text JSON, all with explicit `type`):
+  - `req {type:"req", id, method:"connect", params:{minProtocol,maxProtocol,client:{name,version,platform,deviceFamily?,modelIdentifier?,mode,instanceId}, caps, auth:{token?}, locale?, userAgent?}}`
+  - `res {type:"res", id, ok:true, payload: hello-ok }` (or `ok:false` then close)
+  - `hello-ok {type:"hello-ok", protocol:<chosen>, server:{version,commit,host,connId}, features:{methods,events}, snapshot:{presence[], health, stateVersion:{presence,health}, uptimeMs}, policy:{maxPayload, maxBufferedBytes, tickIntervalMs}}`
+  - `req {type:"req", id, method, params?}`
+  - `res {type:"res", id, ok, payload?, error?}` where `error` = `{code,message,details?,retryable?,retryAfterMs?}`
+  - `event {type:"event", event, payload, seq?, stateVersion?}` (presence/tick/shutdown/agent)
+  - `close` (standard WS close codes; policy uses 1008 for slow consumer/unauthorized, 1012/1001 for restart)
+- Payload types:
+  - `PresenceEntry {host, ip, version, platform?, deviceFamily?, modelIdentifier?, mode, lastInputSeconds?, ts, reason?, tags?[], instanceId?}`
+  - `HealthSnapshot` (match existing `clawdbot health --json` fields)
+  - `AgentEvent` (streamed tool/output; `{runId, seq, stream, data, ts}`)
+  - `TickEvent {ts}`
+  - `ShutdownEvent {reason, restartExpectedMs?}`
+  - Error codes: `NOT_LINKED`, `AGENT_TIMEOUT`, `INVALID_REQUEST`, `UNAVAILABLE`.
+- Error shape: `{code, message, details?, retryable?, retryAfterMs?}`
+- Rules:
+  - First frame must be `req` with `method:"connect"`; otherwise close. Add handshake timeout (e.g., 3s) for silent clients.
+  - Negotiate protocol: server picks within `[minProtocol,maxProtocol]`; if none, reply `res ok:false` and close.
+  - Protocol version bump on breaking changes; `hello-ok` must include `minClient` when needed.
+  - `stateVersion` increments for presence/health to drop stale deltas.
+  - Stable IDs: client sends `instanceId`; server issues per-connection `connId` in `hello-ok`; presence entries may include `instanceId` to dedupe reconnects.
+  - Token-based auth: bearer token in `auth.token`; required except for loopback development.
+  - Presence is primarily connection-derived; client may add hints (e.g., lastInputSeconds); entries expire via TTL to keep the map bounded (e.g., 5m TTL, max 200 entries).
+  - Idempotency keys: required for `send` and `agent` to safely retry after disconnects.
+  - Size limits: bound first-frame size by `maxPayload`; reject early if exceeded.
+  - Close on any non-JSON or wrong `type` before connect.
+  - Per-op idempotency keys: client SHOULD supply an explicit key per `send`/`agent`; if omitted, server may derive a scoped key from `instanceId+connId`, but explicit keys are safer across reconnects.
+  - Locale/userAgent are informational; server may log them for analytics but must not rely on them for access control.
+
+## Phase 2 — Gateway WebSocket server
+- New module `src/gateway/server.ts`:
+  - Bind 127.0.0.1:18789 (configurable).
+  - On connect: validate `connect` params, send snapshot payload, start event pump.
+  - Per-connection queues with backpressure (bounded; drop oldest non-critical).
+  - WS-level caps: set `maxPayload` to cap frame size before JSON parse.
+  - Emit `tick` every N seconds when idle (or WS ping/pong if adequate).
+  - Emit `shutdown` before exit; then close sockets.
+- Methods implemented:
+  - `health`, `status`, `system-presence`, `system-event`, `send`, `agent`.
+  - Optional: `set-heartbeats` removed/renamed if heartbeat concept is retired.
+- Events implemented:
+  - `agent`, `presence` (deltas, with `stateVersion`), `tick`, `shutdown`.
+  - All events include `seq` for loss/out-of-order detection.
+- Logging: structured logs on connect/close/error; include client fingerprint.
+- Slow consumer policy:
+  - Per-connection outbound queue limit (bytes/messages). If exceeded, drop non-critical events (presence/tick) or close with a policy violation / retryable code; clients reconnect with backoff.
+- Handshake edge cases:
+  - Close on handshake timeout.
+  - Close on over-limit first frame (maxPayload).
+  - Close immediately on non-JSON or wrong `type` before connect.
+  - Default guardrails: `maxPayload` ~512 KB, handshake timeout ~3 s, outbound buffered amount cap ~1.5 MB (tune as you implement).
+- Dedupe cache: bound TTL (~5m) and max size (~1000 entries); evict oldest first (LRU) to prevent memory growth.
+
+## Phase 3 — Gateway CLI entrypoint
+- Add `clawdbot gateway` command in CLI program:
+  - Reads config (port, WS options).
+  - Foreground process; exit non-zero on fatal errors.
+  - Flags: `--port`, `--no-tick` (optional), `--log-json` (optional).
+- System supervision docs for launchd/systemd (see `gateway.md`).
+
+## Phase 4 — Presence/health snapshot & stateVersion
+- `hello-ok.snapshot` includes:
+  - `presence[]` (current list)
+  - `health` (full snapshot)
+  - `stateVersion {presence:int, health:int}`
+  - `uptimeMs`
+  - `policy {maxPayload, maxBufferedBytes, tickIntervalMs}`
+- Emit `presence` deltas with updated `stateVersion.presence`.
+- Emit `tick` to indicate liveness when no other events occur.
+- Keep `health` method for manual refresh; not required after connect.
+ - Presence expiry: prune entries older than TTL; enforce a max map size; include `stateVersion` in presence events.
+
+## Phase 5 — Clients migration
+- **macOS app**:
+  - Replace stdio/SSH RPC with WS client (tunneled via SSH/Tailscale for remote). ✅ GatewayConnection/ControlChannel now use Gateway WS.
+  - Implement handshake, snapshot hydration, subscriptions to `presence`, `tick`, `agent`, `shutdown`. ✅ snapshot + presence events broadcast to InstancesStore; agent events still to wire to UI if desired.
+  - Remove immediate `health/system-presence` fetch on connect. ✅ presence hydrated from snapshot; periodic refresh kept as fallback.
+  - Handle connect failures (`res ok:false`) and retry with backoff if version/token mismatched. ✅ macOS GatewayChannel reconnects with exponential backoff.
+- **CLI**:
+- Add lightweight WS client helper for `status/health/send/agent` when Gateway is up. ✅ `gateway` subcommands use the Gateway over WS.
+  - Consider a “local only” flag to avoid accidental remote connects. (optional; not needed with tunnel-first model.)
+- **WebChat backend**:
+  - Single WS to Gateway; seed UI from snapshot; forward `presence/tick/agent` to browser. ✅ implemented via the WebChat gateway client in `webchat/server.ts`.
+  - Fail fast if handshake fails; no fallback transports. ✅ (webchat returns gateway unavailable)
+
+## Phase 6 — Send/agent path hardening
+- Ensure only the Gateway can open Baileys; no IPC fallback.
+- `send` executes in-process; respond with explicit result/error, not via heartbeat.
+- `agent` spawns Pi; respond quickly with `{runId,status:"accepted"}` (ack); stream `event:agent {runId, seq, stream, data, ts}`; final `res:agent {runId, status:"ok"|"error", summary}` completes request (idempotent via key).
+- Idempotency: side-effecting methods (`send`, `agent`) accept an idempotency key; keep a short-lived dedupe cache to avoid double-send on client retries. Client retry flow: on timeout/close, retry with same key; Gateway returns cached result when available; cache TTL ~5m and bounded.
+- Agent stream ordering: enforce monotonic `seq` per runId; if gap detected by server, terminate stream with error; if detected by client, issue a retry with same idempotency key.
+ - Send response shape: `{messageId?, toJid?, error?}` and always include `runId` when available for traceability.
+
+## Phase 7 — Keepalive and shutdown semantics
+- Keepalive: `tick` events (or WS ping/pong) at fixed interval; clients treat missing ticks as disconnect and reconnect.
+- Shutdown: send `event:shutdown {reason, restartExpectedMs?}` then close sockets; clients auto-reconnect.
+- Restart semantics: close sockets with a standard retryable close code; on reconnect, `hello-ok` snapshot must be sufficient to rebuild UI without event replay.
+  - Use a standard close code (e.g., 1012 service restart or 1001 going away) for planned restart; 1008 policy violation for slow consumers.
+  - Include `policy` in `hello-ok` so clients know the tick interval and buffer limits to tune their expectations.
+
+## Phase 8 — Cleanup and deprecation
+- Retire `clawdbot rpc` as default path; keep only if explicitly requested (documented as legacy).
+- Remove reliance on `src/infra/control-channel.ts` for new clients; mark as legacy or delete after migration. ✅ file removed; mac app now uses Gateway WS.
+- Update README, docs (`architecture.md`, `gateway.md`, `webchat.md`) to final shapes; remove `control-api.md` references if obsolete.
+- Presence hygiene:
+  - Presence derived primarily from connection (server-fills host/ip/version/connId/instanceId); allow client hints (e.g., lastInputSeconds).
+  - Add TTL/expiry; prune to keep map bounded (e.g., 5m TTL, max 200 entries).
+
+## Edge cases and ordering
+- Event ordering: all events carry `seq`; clients detect gaps and should re-fetch snapshot (or targeted refresh) on gap.
+- Partial handshakes: if client connects and never sends `req:connect`, server closes after handshake timeout.
+- Garbage/oversize first frame: bounded by `maxPayload`; server closes immediately on parse failure.
+- Duplicate delivery on reconnect: clients must send idempotency keys; Gateway dedupe cache prevents double-send/agent execution.
+- Snapshot sufficiency: `hello-ok.snapshot` must contain enough to render UI after reconnect without event replay.
+- Client reconnect guidance: exponential backoff with jitter; reuse same `instanceId` across reconnects to avoid duplicate presence; resend idempotency keys for in-flight sends/agents; on seq gap, issue `health`/`system-presence` refresh.
+- Presence TTL/defaults: set a concrete TTL (e.g., 5 minutes) and prune periodically; cap the presence map size with LRU if needed.
+- Replay policy: if seq gap detected, server does not replay; clients must pull fresh `health` + `system-presence` and continue.
+
+## Phase 9 — Testing & validation
+- Unit: frame validation, handshake failure, auth/token, stateVersion on presence events, agent stream fanout, send dedupe. ✅
+- Integration: connect → snapshot → req/res → streaming agent → shutdown. ✅ Covered in gateway WS tests (connect/health/status/presence, agent ack+final, shutdown broadcast).
+- Load: multiple concurrent WS clients; backpressure behavior under burst. ✅ Basic fanout test with 3 clients receiving presence broadcast; heavier soak still recommended.
+- Mac app smoke: presence/health render from snapshot; reconnect on tick loss. (Manual: open Instances tab, verify snapshot after connect, induce seq gap by toggling wifi, ensure UI refreshes.)
+- WebChat smoke: snapshot seed + event updates; tunnel scenario. ✅ Offline snapshot harness in `src/webchat/server.test.ts` (mock gateway) now passes; live tunnel still recommended for manual.
+- Idempotency tests: retry send/agent with same key after forced disconnect; expect deduped result. ✅ send + agent dedupe + reconnect retry covered in gateway tests.
+- Seq-gap handling: ✅ clients now detect seq gaps (WebChat gateway client + mac `GatewayConnection/GatewayChannel`) and refresh health/presence (webchat) or trigger UI refresh (mac). Load-test still optional.
+
+## Phase 10 — Rollout
+- Version bump; release notes: breaking change to control plane (WS only).
+- Ship launchd/systemd templates for `clawdbot gateway`.
+- Recommend Tailscale/SSH tunnel for remote access; no additional auth layer assumed in this model.
+
+---
+
+- Quick checklist
+- [x] Protocol types & schemas (TS + JSON Schema + Swift via quicktype)
+- [x] AJV validators wired
+- [x] WS server with connect → snapshot → events
+- [x] Tick + shutdown events
+- [x] stateVersion + presence deltas
+- [x] Gateway CLI command
+- [x] macOS app WS client (Gateway WS for control; presence events live; agent stream UI pending)
+- [x] WebChat WS client
+- [x] Remove legacy stdin/TCP paths from default flows (file removed; mac app/CLI on Gateway)
+- [x] Tests (unit/integration/load) — unit + integration + basic fanout/reconnect; heavier load/soak optional
+- [x] Docs updated and legacy docs flagged

package/docs/refactor/tui.md

@@ -0,0 +1,26 @@
+---
+summary: "Refactor plan: Gateway TUI parity with pi-mono interactive UI"
+read_when:
+  - Building or refactoring the Gateway TUI
+  - Syncing TUI slash commands with Clawdbot behavior
+---
+# Gateway TUI refactor plan
+
+Updated: 2026-01-03
+
+## Goals
+- Match pi-mono interactive TUI feel (editor, streaming, tool cards, selectors).
+- Keep Clawdbot semantics: Gateway WS only, session store owns state, no branching/export.
+- Work locally or remotely via Gateway URL/token.
+
+## Non-goals
+- Branching, export, OAuth flows, or hook UIs.
+- File-system operations on the Gateway host from the TUI.
+
+## Checklist
+- [x] Protocol + server: sessions.patch supports model overrides; agent events include tool results (text-only payloads).
+- [x] Gateway TUI client: add session/model helpers + stricter typing.
+- [x] TUI UI kit: theme + components (editor, message feed, tool cards, selectors).
+- [x] TUI controller: keybindings + Clawdbot slash commands + history/stream wiring.
+- [x] Docs + changelog updated for the new TUI behavior.
+- [x] Gate: lint, build, tests, docs list.

package/docs/refactor/web-gateway-troubleshooting.md

@@ -0,0 +1,37 @@
+---
+summary: "Troubleshooting guide for the web gateway/Baileys stack"
+read_when:
+  - Diagnosing web gateway socket or login issues
+---
+# Web Gateway Troubleshooting (Nov 26, 2025)
+
+## Symptoms & quick fixes
+- **Stream Errored / Conflict / status 409–515:** WhatsApp closed the socket because another session is active or creds went stale. Run `clawdbot logout`, then `clawdbot login`, then restart the Gateway.
+- **Logged out:** Console prints “session logged out”; re-link with `clawdbot login`.
+- **Repeated retries then exit:** Tune reconnect behavior via config `web.reconnect` and restart the Gateway.
+- **No inbound messages:** Ensure the QR-linked account is online in WhatsApp, and check logs for `web-heartbeat` to confirm auth age/connection.
+- **Status 515 right after pairing:** The QR login flow now auto-restarts once; you should not need a manual gateway restart after scanning.
+- **Fast nuke:** From an allowed WhatsApp sender you can send `/restart` to request a supervised restart (launchd/mac app setups); wait a few seconds for it to come back.
+
+## Helpful commands
+- Start the Gateway: `clawdbot gateway --verbose`
+- Logout (clear creds): `clawdbot logout`
+- Relink (show QR): `clawdbot login --verbose`
+- Tail logs (default): `tail -f /tmp/clawdbot/clawdbot-*.log`
+
+## Reading the logs
+- `web-reconnect`: close reasons, retry/backoff, max-attempt exit.
+- `web-heartbeat`: connectionId, messagesHandled, authAgeMs, uptimeMs (every 60s by default).
+- `web-auto-reply`: inbound/outbound message records with correlation IDs.
+
+## When to tweak knobs
+- High churn networks: increase `web.reconnect.maxAttempts`.
+- Slow links: raise `web.reconnect.maxMs` to give more headroom before bailing.
+- Chatty monitors: increase `web.heartbeatSeconds` if log volume is high.
+
+## If it keeps failing
+1) `clawdbot logout` → `clawdbot login` (fresh QR link).
+2) Ensure no other device/browser is using the same WA Web session.
+3) Check WhatsApp mobile app is online and not in low-power mode.
+4) If status is 515, let the client restart once after pairing (already handled automatically).
+5) Capture the last `web-reconnect` entry and the status code before escalating.

package/docs/refactor/webagent-session.md

@@ -0,0 +1,46 @@
+---
+summary: "WebChat session migration notes (Gateway WS-only)"
+read_when:
+  - Changing WebChat Gateway methods/events
+---
+<!-- {% raw %} -->
+# WebAgent session migration (WS-only)
+
+Context: web chat currently lives in a WKWebView that loads the pi-web bundle. Sends go over HTTP `/rpc` to the webchat server, and updates come from `/socket` snapshots based on session JSONL file changes. The Gateway itself already speaks WebSocket to the webchat server, and Pi writes the session JSONL files. This doc tracks the plan to move WebChat to a single Gateway WebSocket and drop the HTTP shim/file-watching.
+
+## Target state
+- Gateway WS adds methods:
+  - `chat.history { sessionKey }` → `{ sessionKey, messages[], thinkingLevel }` (reads the existing JSONL + session store).
+  - `chat.send { sessionKey, message, attachments?, thinking?, deliver?, timeoutMs<=30000, idempotencyKey }` → `res { runId, status:"accepted" }` or `res ok:false` on validation/timeout.
+- Gateway WS emits `chat` events `{ runId, sessionKey, seq, state:"delta"|"final"|"error", message?, errorMessage?, usage?, stopReason? }`. Streaming is optional; minimum is a single `state:"final"` per send.
+- Client consumes only WS: bootstrap via `chat.history`, send via `chat.send`, live updates via `chat` events. No file watchers.
+- Health gate: client subscribes to `health` and blocks send when health is not OK; 30s client-side timeout for sends.
+- Tunneling: only the Gateway WS port needs to be forwarded; HTTP server remains for static assets but no RPC endpoints.
+
+## Server work (Node)
+- Implement `chat.history` and `chat.send` handlers in `src/gateway/server.ts`; update protocol schemas/tests.
+- Emit `chat` events by plumbing `agentCommand`/`emitAgentEvent` outputs; include assistant text/tool results.
+- Remove `/rpc` and `/socket` routes + file-watch broadcast from `src/webchat/server.ts`; leave static host only.
+
+## Client work (pi-web bundle)
+- Replace `NativeTransport` with a Gateway WS client:
+  - `connect` → `chat.history` for initial state.
+  - Listen to `chat/presence/tick/health`; update UI from events only.
+  - Send via `chat.send`; mark pending until `chat state:final|error`.
+  - Enforce health gate + 30s timeout.
+- Remove reliance on session file snapshots and `/rpc`.
+
+## Persistence
+- Keep passing `--session <.../.clawdbot/sessions/{{SessionId}}.jsonl>` to Pi so it continues writing JSONL. The WS history reader uses the same file; no new store introduced.
+
+## Docs to update when shipping
+- `docs/webchat.md` (WS-only flow, methods/events, health gate, tunnel WS port).
+- `docs/mac/webchat.md` (WKWebView now talks Gateway WS; `/rpc`/file-watch removed).
+- `docs/architecture.md` / `typebox.md` if protocol methods are listed.
+- Optional: add a concise Gateway chat protocol appendix if needed.
+
+## Open decisions
+- Streaming granularity: start with `state:"final"` only, or include token/tool deltas immediately?
+- Attachments over WS: text-only initially is OK; confirm before wiring binary/upload path.
+- Error shape: use `res ok:false` for validation/timeout, `chat state:"error"` for model/runtime failures.
+<!-- {% endraw %} -->