@quantumclaw/quantumclaw 2026.3.22

package/docs/concepts/presence.md

@@ -0,0 +1,102 @@
+---
+summary: "How QuantumClaw presence entries are produced, merged, and displayed"
+read_when:
+  - Debugging the Instances tab
+  - Investigating duplicate or stale instance rows
+  - Changing gateway WS connect or system-event beacons
+title: "Presence"
+---
+
+# Presence
+
+QuantumClaw “presence” is a lightweight, best‑effort view of:
+
+- the **Gateway** itself, and
+- **clients connected to the Gateway** (mac app, WebChat, CLI, etc.)
+
+Presence is used primarily to render the macOS app’s **Instances** tab and to
+provide quick operator visibility.
+
+## Presence fields (what shows up)
+
+Presence entries are structured objects with fields like:
+
+- `instanceId` (optional but strongly recommended): stable client identity (usually `connect.client.instanceId`)
+- `host`: human‑friendly host name
+- `ip`: best‑effort IP address
+- `version`: client version string
+- `deviceFamily` / `modelIdentifier`: hardware hints
+- `mode`: `ui`, `webchat`, `cli`, `backend`, `probe`, `test`, `node`, ...
+- `lastInputSeconds`: “seconds since last user input” (if known)
+- `reason`: `self`, `connect`, `node-connected`, `periodic`, ...
+- `ts`: last update timestamp (ms since epoch)
+
+## Producers (where presence comes from)
+
+Presence entries are produced by multiple sources and **merged**.
+
+### 1) Gateway self entry
+
+The Gateway always seeds a “self” entry at startup so UIs show the gateway host
+even before any clients connect.
+
+### 2) WebSocket connect
+
+Every WS client begins with a `connect` request. On successful handshake the
+Gateway upserts a presence entry for that connection.
+
+#### Why one-off CLI commands do not show up
+
+The CLI often connects for short, one‑off commands. To avoid spamming the
+Instances list, `client.mode === "cli"` is **not** turned into a presence entry.
+
+### 3) `system-event` beacons
+
+Clients can send richer periodic beacons via the `system-event` method. The mac
+app uses this to report host name, IP, and `lastInputSeconds`.
+
+### 4) Node connects (role: node)
+
+When a node connects over the Gateway WebSocket with `role: node`, the Gateway
+upserts a presence entry for that node (same flow as other WS clients).
+
+## Merge + dedupe rules (why `instanceId` matters)
+
+Presence entries are stored in a single in‑memory map:
+
+- Entries are keyed by a **presence key**.
+- The best key is a stable `instanceId` (from `connect.client.instanceId`) that survives restarts.
+- Keys are case‑insensitive.
+
+If a client reconnects without a stable `instanceId`, it may show up as a
+**duplicate** row.
+
+## TTL and bounded size
+
+Presence is intentionally ephemeral:
+
+- **TTL:** entries older than 5 minutes are pruned
+- **Max entries:** 200 (oldest dropped first)
+
+This keeps the list fresh and avoids unbounded memory growth.
+
+## Remote/tunnel caveat (loopback IPs)
+
+When a client connects over an SSH tunnel / local port forward, the Gateway may
+see the remote address as `127.0.0.1`. To avoid overwriting a good client‑reported
+IP, loopback remote addresses are ignored.
+
+## Consumers
+
+### macOS Instances tab
+
+The macOS app renders the output of `system-presence` and applies a small status
+indicator (Active/Idle/Stale) based on the age of the last update.
+
+## Debugging tips
+
+- To see the raw list, call `system-presence` against the Gateway.
+- If you see duplicates:
+  - confirm clients send a stable `client.instanceId` in the handshake
+  - confirm periodic beacons use the same `instanceId`
+  - check whether the connection‑derived entry is missing `instanceId` (duplicates are expected)

package/docs/concepts/queue.md

@@ -0,0 +1,89 @@
+---
+summary: "Command queue design that serializes inbound auto-reply runs"
+read_when:
+  - Changing auto-reply execution or concurrency
+title: "Command Queue"
+---
+
+# Command Queue (2026-01-16)
+
+We serialize inbound auto-reply runs (all channels) through a tiny in-process queue to prevent multiple agent runs from colliding, while still allowing safe parallelism across sessions.
+
+## Why
+
+- Auto-reply runs can be expensive (LLM calls) and can collide when multiple inbound messages arrive close together.
+- Serializing avoids competing for shared resources (session files, logs, CLI stdin) and reduces the chance of upstream rate limits.
+
+## How it works
+
+- A lane-aware FIFO queue drains each lane with a configurable concurrency cap (default 1 for unconfigured lanes; main defaults to 4, subagent to 8).
+- `runEmbeddedPiAgent` enqueues by **session key** (lane `session:<key>`) to guarantee only one active run per session.
+- Each session run is then queued into a **global lane** (`main` by default) so overall parallelism is capped by `agents.defaults.maxConcurrent`.
+- When verbose logging is enabled, queued runs emit a short notice if they waited more than ~2s before starting.
+- Typing indicators still fire immediately on enqueue (when supported by the channel) so user experience is unchanged while we wait our turn.
+
+## Queue modes (per channel)
+
+Inbound messages can steer the current run, wait for a followup turn, or do both:
+
+- `steer`: inject immediately into the current run (cancels pending tool calls after the next tool boundary). If not streaming, falls back to followup.
+- `followup`: enqueue for the next agent turn after the current run ends.
+- `collect`: coalesce all queued messages into a **single** followup turn (default). If messages target different channels/threads, they drain individually to preserve routing.
+- `steer-backlog` (aka `steer+backlog`): steer now **and** preserve the message for a followup turn.
+- `interrupt` (legacy): abort the active run for that session, then run the newest message.
+- `queue` (legacy alias): same as `steer`.
+
+Steer-backlog means you can get a followup response after the steered run, so
+streaming surfaces can look like duplicates. Prefer `collect`/`steer` if you want
+one response per inbound message.
+Send `/queue collect` as a standalone command (per-session) or set `messages.queue.byChannel.discord: "collect"`.
+
+Defaults (when unset in config):
+
+- All surfaces → `collect`
+
+Configure globally or per channel via `messages.queue`:
+
+```json5
+{
+  messages: {
+    queue: {
+      mode: "collect",
+      debounceMs: 1000,
+      cap: 20,
+      drop: "summarize",
+      byChannel: { discord: "collect" },
+    },
+  },
+}
+```
+
+## Queue options
+
+Options apply to `followup`, `collect`, and `steer-backlog` (and to `steer` when it falls back to followup):
+
+- `debounceMs`: wait for quiet before starting a followup turn (prevents “continue, continue”).
+- `cap`: max queued messages per session.
+- `drop`: overflow policy (`old`, `new`, `summarize`).
+
+Summarize keeps a short bullet list of dropped messages and injects it as a synthetic followup prompt.
+Defaults: `debounceMs: 1000`, `cap: 20`, `drop: summarize`.
+
+## Per-session overrides
+
+- Send `/queue <mode>` as a standalone command to store the mode for the current session.
+- Options can be combined: `/queue collect debounce:2s cap:25 drop:summarize`
+- `/queue default` or `/queue reset` clears the session override.
+
+## Scope and guarantees
+
+- Applies to auto-reply agent runs across all inbound channels that use the gateway reply pipeline (WhatsApp web, Telegram, Slack, Discord, Signal, iMessage, webchat, etc.).
+- Default lane (`main`) is process-wide for inbound + main heartbeats; set `agents.defaults.maxConcurrent` to allow multiple sessions in parallel.
+- Additional lanes may exist (e.g. `cron`, `subagent`) so background jobs can run in parallel without blocking inbound replies.
+- Per-session lanes guarantee that only one agent run touches a given session at a time.
+- No external dependencies or background worker threads; pure TypeScript + promises.
+
+## Troubleshooting
+
+- If commands seem stuck, enable verbose logs and look for “queued for …ms” lines to confirm the queue is draining.
+- If you need queue depth, enable verbose logs and watch for queue timing lines.

package/docs/concepts/retry.md

@@ -0,0 +1,69 @@
+---
+summary: "Retry policy for outbound provider calls"
+read_when:
+  - Updating provider retry behavior or defaults
+  - Debugging provider send errors or rate limits
+title: "Retry Policy"
+---
+
+# Retry policy
+
+## Goals
+
+- Retry per HTTP request, not per multi-step flow.
+- Preserve ordering by retrying only the current step.
+- Avoid duplicating non-idempotent operations.
+
+## Defaults
+
+- Attempts: 3
+- Max delay cap: 30000 ms
+- Jitter: 0.1 (10 percent)
+- Provider defaults:
+  - Telegram min delay: 400 ms
+  - Discord min delay: 500 ms
+
+## Behavior
+
+### Discord
+
+- Retries only on rate-limit errors (HTTP 429).
+- Uses Discord `retry_after` when available, otherwise exponential backoff.
+
+### Telegram
+
+- Retries on transient errors (429, timeout, connect/reset/closed, temporarily unavailable).
+- Uses `retry_after` when available, otherwise exponential backoff.
+- Markdown parse errors are not retried; they fall back to plain text.
+
+## Configuration
+
+Set retry policy per provider in `~/.quantumclaw/quantumclaw.json`:
+
+```json5
+{
+  channels: {
+    telegram: {
+      retry: {
+        attempts: 3,
+        minDelayMs: 400,
+        maxDelayMs: 30000,
+        jitter: 0.1,
+      },
+    },
+    discord: {
+      retry: {
+        attempts: 3,
+        minDelayMs: 500,
+        maxDelayMs: 30000,
+        jitter: 0.1,
+      },
+    },
+  },
+}
+```
+
+## Notes
+
+- Retries apply per request (message send, media upload, reaction, poll, sticker).
+- Composite flows do not retry completed steps.

package/docs/concepts/session-pruning.md

@@ -0,0 +1,121 @@
+---
+title: "Session Pruning"
+summary: "Session pruning: tool-result trimming to reduce context bloat"
+read_when:
+  - You want to reduce LLM context growth from tool outputs
+  - You are tuning agents.defaults.contextPruning
+---
+
+# Session Pruning
+
+Session pruning trims **old tool results** from the in-memory context right before each LLM call. It does **not** rewrite the on-disk session history (`*.jsonl`).
+
+## When it runs
+
+- When `mode: "cache-ttl"` is enabled and the last Anthropic call for the session is older than `ttl`.
+- Only affects the messages sent to the model for that request.
+- Only active for Anthropic API calls (and OpenRouter Anthropic models).
+- For best results, match `ttl` to your model `cacheRetention` policy (`short` = 5m, `long` = 1h).
+- After a prune, the TTL window resets so subsequent requests keep cache until `ttl` expires again.
+
+## Smart defaults (Anthropic)
+
+- **OAuth or setup-token** profiles: enable `cache-ttl` pruning and set heartbeat to `1h`.
+- **API key** profiles: enable `cache-ttl` pruning, set heartbeat to `30m`, and default `cacheRetention: "short"` on Anthropic models.
+- If you set any of these values explicitly, QuantumClaw does **not** override them.
+
+## What this improves (cost + cache behavior)
+
+- **Why prune:** Anthropic prompt caching only applies within the TTL. If a session goes idle past the TTL, the next request re-caches the full prompt unless you trim it first.
+- **What gets cheaper:** pruning reduces the **cacheWrite** size for that first request after the TTL expires.
+- **Why the TTL reset matters:** once pruning runs, the cache window resets, so follow‑up requests can reuse the freshly cached prompt instead of re-caching the full history again.
+- **What it does not do:** pruning doesn’t add tokens or “double” costs; it only changes what gets cached on that first post‑TTL request.
+
+## What can be pruned
+
+- Only `toolResult` messages.
+- User + assistant messages are **never** modified.
+- The last `keepLastAssistants` assistant messages are protected; tool results after that cutoff are not pruned.
+- If there aren’t enough assistant messages to establish the cutoff, pruning is skipped.
+- Tool results containing **image blocks** are skipped (never trimmed/cleared).
+
+## Context window estimation
+
+Pruning uses an estimated context window (chars ≈ tokens × 4). The base window is resolved in this order:
+
+1. `models.providers.*.models[].contextWindow` override.
+2. Model definition `contextWindow` (from the model registry).
+3. Default `200000` tokens.
+
+If `agents.defaults.contextTokens` is set, it is treated as a cap (min) on the resolved window.
+
+## Mode
+
+### cache-ttl
+
+- Pruning only runs if the last Anthropic call is older than `ttl` (default `5m`).
+- When it runs: same soft-trim + hard-clear behavior as before.
+
+## Soft vs hard pruning
+
+- **Soft-trim**: only for oversized tool results.
+  - Keeps head + tail, inserts `...`, and appends a note with the original size.
+  - Skips results with image blocks.
+- **Hard-clear**: replaces the entire tool result with `hardClear.placeholder`.
+
+## Tool selection
+
+- `tools.allow` / `tools.deny` support `*` wildcards.
+- Deny wins.
+- Matching is case-insensitive.
+- Empty allow list => all tools allowed.
+
+## Interaction with other limits
+
+- Built-in tools already truncate their own output; session pruning is an extra layer that prevents long-running chats from accumulating too much tool output in the model context.
+- Compaction is separate: compaction summarizes and persists, pruning is transient per request. See [/concepts/compaction](/concepts/compaction).
+
+## Defaults (when enabled)
+
+- `ttl`: `"5m"`
+- `keepLastAssistants`: `3`
+- `softTrimRatio`: `0.3`
+- `hardClearRatio`: `0.5`
+- `minPrunableToolChars`: `50000`
+- `softTrim`: `{ maxChars: 4000, headChars: 1500, tailChars: 1500 }`
+- `hardClear`: `{ enabled: true, placeholder: "[Old tool result content cleared]" }`
+
+## Examples
+
+Default (off):
+
+```json5
+{
+  agents: { defaults: { contextPruning: { mode: "off" } } },
+}
+```
+
+Enable TTL-aware pruning:
+
+```json5
+{
+  agents: { defaults: { contextPruning: { mode: "cache-ttl", ttl: "5m" } } },
+}
+```
+
+Restrict pruning to specific tools:
+
+```json5
+{
+  agents: {
+    defaults: {
+      contextPruning: {
+        mode: "cache-ttl",
+        tools: { allow: ["exec", "read"], deny: ["*image*"] },
+      },
+    },
+  },
+}
+```
+
+See config reference: [Gateway Configuration](/gateway/configuration)

package/docs/concepts/session-tool.md

@@ -0,0 +1,242 @@
+---
+summary: "Agent session tools for listing sessions, fetching history, and sending cross-session messages"
+read_when:
+  - Adding or modifying session tools
+title: "Session Tools"
+---
+
+# Session Tools
+
+Goal: small, hard-to-misuse tool set so agents can list sessions, fetch history, and send to another session.
+
+## Tool Names
+
+- `sessions_list`
+- `sessions_history`
+- `sessions_send`
+- `sessions_spawn`
+
+## Key Model
+
+- Main direct chat bucket is always the literal key `"main"` (resolved to the current agent’s main key).
+- Group chats use `agent:<agentId>:<channel>:group:<id>` or `agent:<agentId>:<channel>:channel:<id>` (pass the full key).
+- Cron jobs use `cron:<job.id>`.
+- Hooks use `hook:<uuid>` unless explicitly set.
+- Node sessions use `node-<nodeId>` unless explicitly set.
+
+`global` and `unknown` are reserved values and are never listed. If `session.scope = "global"`, we alias it to `main` for all tools so callers never see `global`.
+
+## sessions_list
+
+List sessions as an array of rows.
+
+Parameters:
+
+- `kinds?: string[]` filter: any of `"main" | "group" | "cron" | "hook" | "node" | "other"`
+- `limit?: number` max rows (default: server default, clamp e.g. 200)
+- `activeMinutes?: number` only sessions updated within N minutes
+- `messageLimit?: number` 0 = no messages (default 0); >0 = include last N messages
+
+Behavior:
+
+- `messageLimit > 0` fetches `chat.history` per session and includes the last N messages.
+- Tool results are filtered out in list output; use `sessions_history` for tool messages.
+- When running in a **sandboxed** agent session, session tools default to **spawned-only visibility** (see below).
+
+Row shape (JSON):
+
+- `key`: session key (string)
+- `kind`: `main | group | cron | hook | node | other`
+- `channel`: `whatsapp | telegram | discord | signal | imessage | webchat | internal | unknown`
+- `displayName` (group display label if available)
+- `updatedAt` (ms)
+- `sessionId`
+- `model`, `contextTokens`, `totalTokens`
+- `thinkingLevel`, `verboseLevel`, `systemSent`, `abortedLastRun`
+- `sendPolicy` (session override if set)
+- `lastChannel`, `lastTo`
+- `deliveryContext` (normalized `{ channel, to, accountId }` when available)
+- `transcriptPath` (best-effort path derived from store dir + sessionId)
+- `messages?` (only when `messageLimit > 0`)
+
+## sessions_history
+
+Fetch transcript for one session.
+
+Parameters:
+
+- `sessionKey` (required; accepts session key or `sessionId` from `sessions_list`)
+- `limit?: number` max messages (server clamps)
+- `includeTools?: boolean` (default false)
+
+Behavior:
+
+- `includeTools=false` filters `role: "toolResult"` messages.
+- Returns messages array in the raw transcript format.
+- When given a `sessionId`, QuantumClaw resolves it to the corresponding session key (missing ids error).
+
+## Gateway session history and live transcript APIs
+
+Control UI and gateway clients can use the lower level history and live transcript surfaces directly.
+
+HTTP:
+
+- `GET /sessions/{sessionKey}/history`
+- Query params: `limit`, `cursor`, `includeTools=1`, `follow=1`
+- Unknown sessions return HTTP `404` with `error.type = "not_found"`
+- `follow=1` upgrades the response to an SSE stream of transcript updates for that session
+
+WebSocket:
+
+- `sessions.subscribe` subscribes to all session lifecycle and transcript events visible to the client
+- `sessions.messages.subscribe { key }` subscribes only to `session.message` events for one session
+- `sessions.messages.unsubscribe { key }` removes that targeted transcript subscription
+- `session.message` carries appended transcript messages plus live usage metadata when available
+- `sessions.changed` emits `phase: "message"` for transcript appends so session lists can refresh counters and previews
+
+## sessions_send
+
+Send a message into another session.
+
+Parameters:
+
+- `sessionKey` (required; accepts session key or `sessionId` from `sessions_list`)
+- `message` (required)
+- `timeoutSeconds?: number` (default >0; 0 = fire-and-forget)
+
+Behavior:
+
+- `timeoutSeconds = 0`: enqueue and return `{ runId, status: "accepted" }`.
+- `timeoutSeconds > 0`: wait up to N seconds for completion, then return `{ runId, status: "ok", reply }`.
+- If wait times out: `{ runId, status: "timeout", error }`. Run continues; call `sessions_history` later.
+- If the run fails: `{ runId, status: "error", error }`.
+- Announce delivery runs after the primary run completes and is best-effort; `status: "ok"` does not guarantee the announce was delivered.
+- Waits via gateway `agent.wait` (server-side) so reconnects don't drop the wait.
+- Agent-to-agent message context is injected for the primary run.
+- Inter-session messages are persisted with `message.provenance.kind = "inter_session"` so transcript readers can distinguish routed agent instructions from external user input.
+- After the primary run completes, QuantumClaw runs a **reply-back loop**:
+  - Round 2+ alternates between requester and target agents.
+  - Reply exactly `REPLY_SKIP` to stop the ping‑pong.
+  - Max turns is `session.agentToAgent.maxPingPongTurns` (0–5, default 5).
+- Once the loop ends, QuantumClaw runs the **agent‑to‑agent announce step** (target agent only):
+  - Reply exactly `ANNOUNCE_SKIP` to stay silent.
+  - Any other reply is sent to the target channel.
+  - Announce step includes the original request + round‑1 reply + latest ping‑pong reply.
+
+## Channel Field
+
+- For groups, `channel` is the channel recorded on the session entry.
+- For direct chats, `channel` maps from `lastChannel`.
+- For cron/hook/node, `channel` is `internal`.
+- If missing, `channel` is `unknown`.
+
+## Security / Send Policy
+
+Policy-based blocking by channel/chat type (not per session id).
+
+```json
+{
+  "session": {
+    "sendPolicy": {
+      "rules": [
+        {
+          "match": { "channel": "discord", "chatType": "group" },
+          "action": "deny"
+        }
+      ],
+      "default": "allow"
+    }
+  }
+}
+```
+
+Runtime override (per session entry):
+
+- `sendPolicy: "allow" | "deny"` (unset = inherit config)
+- Settable via `sessions.patch` or owner-only `/send on|off|inherit` (standalone message).
+
+Enforcement points:
+
+- `chat.send` / `agent` (gateway)
+- auto-reply delivery logic
+
+## sessions_spawn
+
+Spawn a sub-agent run in an isolated session and announce the result back to the requester chat channel.
+
+Parameters:
+
+- `task` (required)
+- `label?` (optional; used for logs/UI)
+- `agentId?` (optional; spawn under another agent id if allowed)
+- `model?` (optional; overrides the sub-agent model; invalid values error)
+- `thinking?` (optional; overrides thinking level for the sub-agent run)
+- `runTimeoutSeconds?` (defaults to `agents.defaults.subagents.runTimeoutSeconds` when set, otherwise `0`; when set, aborts the sub-agent run after N seconds)
+- `thread?` (default false; request thread-bound routing for this spawn when supported by the channel/plugin)
+- `mode?` (`run|session`; defaults to `run`, but defaults to `session` when `thread=true`; `mode="session"` requires `thread=true`)
+- `cleanup?` (`delete|keep`, default `keep`)
+- `sandbox?` (`inherit|require`, default `inherit`; `require` rejects spawn unless the target child runtime is sandboxed)
+- `attachments?` (optional array of inline files; subagent runtime only, ACP rejects). Each entry: `{ name, content, encoding?: "utf8" | "base64", mimeType? }`. Files are materialized into the child workspace at `.quantumclaw/attachments/<uuid>/`. Returns a receipt with sha256 per file.
+- `attachAs?` (optional; `{ mountPath? }` hint reserved for future mount implementations)
+
+Allowlist:
+
+- `agents.list[].subagents.allowAgents`: list of agent ids allowed via `agentId` (`["*"]` to allow any). Default: only the requester agent.
+- Sandbox inheritance guard: if the requester session is sandboxed, `sessions_spawn` rejects targets that would run unsandboxed.
+
+Discovery:
+
+- Use `agents_list` to discover which agent ids are allowed for `sessions_spawn`.
+
+Behavior:
+
+- Starts a new `agent:<agentId>:subagent:<uuid>` session with `deliver: false`.
+- Sub-agents default to the full tool set **minus session tools** (configurable via `tools.subagents.tools`).
+- Sub-agents are not allowed to call `sessions_spawn` (no sub-agent → sub-agent spawning).
+- Always non-blocking: returns `{ status: "accepted", runId, childSessionKey }` immediately.
+- With `thread=true`, channel plugins can bind delivery/routing to a thread target (Discord support is controlled by `session.threadBindings.*` and `channels.discord.threadBindings.*`).
+- After completion, QuantumClaw runs a sub-agent **announce step** and posts the result to the requester chat channel.
+  - If the assistant final reply is empty, the latest `toolResult` from sub-agent history is included as `Result`.
+- Reply exactly `ANNOUNCE_SKIP` during the announce step to stay silent.
+- Announce replies are normalized to `Status`/`Result`/`Notes`; `Status` comes from runtime outcome (not model text).
+- Sub-agent sessions are auto-archived after `agents.defaults.subagents.archiveAfterMinutes` (default: 60).
+- Announce replies include a stats line (runtime, tokens, sessionKey/sessionId, transcript path, and optional cost).
+
+## Sandbox Session Visibility
+
+Session tools can be scoped to reduce cross-session access.
+
+Default behavior:
+
+- `tools.sessions.visibility` defaults to `tree` (current session + spawned subagent sessions).
+- For sandboxed sessions, `agents.defaults.sandbox.sessionToolsVisibility` can hard-clamp visibility.
+
+Config:
+
+```json5
+{
+  tools: {
+    sessions: {
+      // "self" | "tree" | "agent" | "all"
+      // default: "tree"
+      visibility: "tree",
+    },
+  },
+  agents: {
+    defaults: {
+      sandbox: {
+        // default: "spawned"
+        sessionToolsVisibility: "spawned", // or "all"
+      },
+    },
+  },
+}
+```
+
+Notes:
+
+- `self`: only the current session key.
+- `tree`: current session + sessions spawned by the current session.
+- `agent`: any session belonging to the current agent id.
+- `all`: any session (cross-agent access still requires `tools.agentToAgent`).
+- When a session is sandboxed and `sessionToolsVisibility="spawned"`, QuantumClaw clamps visibility to `tree` even if you set `tools.sessions.visibility="all"`.