loudmouth-ai 0.1.0

package/docs/start/wizard.md

@@ -0,0 +1,318 @@
+---
+summary: "CLI onboarding wizard: guided setup for gateway, workspace, channels, and skills"
+read_when:
+  - Running or configuring the onboarding wizard
+  - Setting up a new machine
+---
+
+# Onboarding Wizard (CLI)
+
+The onboarding wizard is the **recommended** way to set up Clawdbot on macOS,
+Linux, or Windows (via WSL2; strongly recommended).
+It configures a local Gateway or a remote Gateway connection, plus channels, skills,
+and workspace defaults in one guided flow.
+
+Primary entrypoint:
+
+```bash
+clawdbot onboard
+```
+
+Follow‑up reconfiguration:
+
+```bash
+clawdbot configure
+```
+
+Recommended: set up a Brave Search API key so the agent can use `web_search`
+(`web_fetch` works without a key). Easiest path: `clawdbot configure --section web`
+which stores `tools.web.search.apiKey`. Docs: [Web tools](/tools/web).
+
+## QuickStart vs Advanced
+
+The wizard starts with **QuickStart** (defaults) vs **Advanced** (full control).
+
+**QuickStart** keeps the defaults:
+- Local gateway (loopback)
+- Workspace default (or existing workspace)
+- Gateway port **18789**
+- Gateway auth **Token** (auto‑generated, even on loopback)
+- Tailscale exposure **Off**
+- Telegram + WhatsApp DMs default to **allowlist** (you’ll be prompted for your phone number)
+
+**Advanced** exposes every step (mode, workspace, gateway, channels, daemon, skills).
+
+## What the wizard does
+
+**Local mode (default)** walks you through:
+  - Model/auth (OpenAI Code (Codex) subscription OAuth, Anthropic API key (recommended) or setup-token (paste), plus MiniMax/GLM/Moonshot/AI Gateway options)
+- Workspace location + bootstrap files
+- Gateway settings (port/bind/auth/tailscale)
+- Providers (Telegram, WhatsApp, Discord, Google Chat, Mattermost (plugin), Signal)
+- Daemon install (LaunchAgent / systemd user unit)
+- Health check
+- Skills (recommended)
+
+**Remote mode** only configures the local client to connect to a Gateway elsewhere.
+It does **not** install or change anything on the remote host.
+
+To add more isolated agents (separate workspace + sessions + auth), use:
+
+```bash
+clawdbot agents add <name>
+```
+
+Tip: `--json` does **not** imply non-interactive mode. Use `--non-interactive` (and `--workspace`) for scripts.
+
+## Flow details (local)
+
+1) **Existing config detection**
+   - If `~/.clawdbot/clawdbot.json` exists, choose **Keep / Modify / Reset**.
+   - Re-running the wizard does **not** wipe anything unless you explicitly choose **Reset**
+     (or pass `--reset`).
+   - If the config is invalid or contains legacy keys, the wizard stops and asks
+     you to run `clawdbot doctor` before continuing.
+   - Reset uses `trash` (never `rm`) and offers scopes:
+     - Config only
+     - Config + credentials + sessions
+     - Full reset (also removes workspace)
+
+2) **Model/Auth**
+   - **Anthropic API key (recommended)**: uses `ANTHROPIC_API_KEY` if present or prompts for a key, then saves it for daemon use.
+   - **Anthropic OAuth (Claude Code CLI)**: on macOS the wizard checks Keychain item "Claude Code-credentials" (choose "Always Allow" so launchd starts don't block); on Linux/Windows it reuses `~/.claude/.credentials.json` if present.
+   - **Anthropic token (paste setup-token)**: run `claude setup-token` on any machine, then paste the token (you can name it; blank = default).
+   - **OpenAI Code (Codex) subscription (Codex CLI)**: if `~/.codex/auth.json` exists, the wizard can reuse it.
+   - **OpenAI Code (Codex) subscription (OAuth)**: browser flow; paste the `code#state`.
+     - Sets `agents.defaults.model` to `openai-codex/gpt-5.2` when model is unset or `openai/*`.
+   - **OpenAI API key**: uses `OPENAI_API_KEY` if present or prompts for a key, then saves it to `~/.clawdbot/.env` so launchd can read it.
+   - **OpenCode Zen (multi-model proxy)**: prompts for `OPENCODE_API_KEY` (or `OPENCODE_ZEN_API_KEY`, get it at https://opencode.ai/auth).
+   - **API key**: stores the key for you.
+   - **Vercel AI Gateway (multi-model proxy)**: prompts for `AI_GATEWAY_API_KEY`.
+   - More detail: [Vercel AI Gateway](/providers/vercel-ai-gateway)
+   - **MiniMax M2.1**: config is auto-written.
+   - More detail: [MiniMax](/providers/minimax)
+   - **Synthetic (Anthropic-compatible)**: prompts for `SYNTHETIC_API_KEY`.
+   - More detail: [Synthetic](/providers/synthetic)
+   - **Moonshot (Kimi K2)**: config is auto-written.
+   - **Kimi Code**: config is auto-written.
+   - More detail: [Moonshot AI (Kimi + Kimi Code)](/providers/moonshot)
+   - **Skip**: no auth configured yet.
+   - Pick a default model from detected options (or enter provider/model manually).
+   - Wizard runs a model check and warns if the configured model is unknown or missing auth.
+  - OAuth credentials live in `~/.clawdbot/credentials/oauth.json`; auth profiles live in `~/.clawdbot/agents/<agentId>/agent/auth-profiles.json` (API keys + OAuth).
+   - More detail: [/concepts/oauth](/concepts/oauth)
+
+3) **Workspace**
+   - Default `~/clawd` (configurable).
+   - Seeds the workspace files needed for the agent bootstrap ritual.
+   - Full workspace layout + backup guide: [Agent workspace](/concepts/agent-workspace)
+
+4) **Gateway**
+   - Port, bind, auth mode, tailscale exposure.
+   - Auth recommendation: keep **Token** even for loopback so local WS clients must authenticate.
+   - Disable auth only if you fully trust every local process.
+   - Non‑loopback binds still require auth.
+
+5) **Channels**
+  - WhatsApp: optional QR login.
+  - Telegram: bot token.
+  - Discord: bot token.
+  - Google Chat: service account JSON + webhook audience.
+  - Mattermost (plugin): bot token + base URL.
+   - Signal: optional `signal-cli` install + account config.
+   - iMessage: local `imsg` CLI path + DB access.
+  - DM security: default is pairing. First DM sends a code; approve via `clawdbot pairing approve <channel> <code>` or use allowlists.
+
+6) **Daemon install**
+   - macOS: LaunchAgent
+     - Requires a logged-in user session; for headless, use a custom LaunchDaemon (not shipped).
+   - Linux (and Windows via WSL2): systemd user unit
+     - Wizard attempts to enable lingering via `loginctl enable-linger <user>` so the Gateway stays up after logout.
+     - May prompt for sudo (writes `/var/lib/systemd/linger`); it tries without sudo first.
+   - **Runtime selection:** Node (recommended; required for WhatsApp/Telegram). Bun is **not recommended**.
+
+7) **Health check**
+   - Starts the Gateway (if needed) and runs `clawdbot health`.
+   - Tip: `clawdbot status --deep` adds gateway health probes to status output (requires a reachable gateway).
+
+8) **Skills (recommended)**
+   - Reads the available skills and checks requirements.
+   - Lets you choose a node manager: **npm / pnpm** (bun not recommended).
+   - Installs optional dependencies (some use Homebrew on macOS).
+
+9) **Finish**
+   - Summary + next steps, including iOS/Android/macOS apps for extra features.
+  - If no GUI is detected, the wizard prints SSH port-forward instructions for the Control UI instead of opening a browser.
+  - If the Control UI assets are missing, the wizard attempts to build them; fallback is `pnpm ui:build` (auto-installs UI deps).
+
+## Remote mode
+
+Remote mode configures a local client to connect to a Gateway elsewhere.
+
+What you’ll set:
+- Remote Gateway URL (`ws://...`)
+- Token if the remote Gateway requires auth (recommended)
+
+Notes:
+- No remote installs or daemon changes are performed.
+- If the Gateway is loopback‑only, use SSH tunneling or a tailnet.
+- Discovery hints:
+  - macOS: Bonjour (`dns-sd`)
+  - Linux: Avahi (`avahi-browse`)
+
+## Add another agent
+
+Use `clawdbot agents add <name>` to create a separate agent with its own workspace,
+sessions, and auth profiles. Running without `--workspace` launches the wizard.
+
+What it sets:
+- `agents.list[].name`
+- `agents.list[].workspace`
+- `agents.list[].agentDir`
+
+Notes:
+- Default workspaces follow `~/clawd-<agentId>`.
+- Add `bindings` to route inbound messages (the wizard can do this).
+- Non-interactive flags: `--model`, `--agent-dir`, `--bind`, `--non-interactive`.
+
+## Non‑interactive mode
+
+Use `--non-interactive` to automate or script onboarding:
+
+```bash
+clawdbot onboard --non-interactive \
+  --mode local \
+  --auth-choice apiKey \
+  --anthropic-api-key "$ANTHROPIC_API_KEY" \
+  --gateway-port 18789 \
+  --gateway-bind loopback \
+  --install-daemon \
+  --daemon-runtime node \
+  --skip-skills
+```
+
+Add `--json` for a machine‑readable summary.
+
+Gemini example:
+
+```bash
+clawdbot onboard --non-interactive \
+  --mode local \
+  --auth-choice gemini-api-key \
+  --gemini-api-key "$GEMINI_API_KEY" \
+  --gateway-port 18789 \
+  --gateway-bind loopback
+```
+
+Z.AI example:
+
+```bash
+clawdbot onboard --non-interactive \
+  --mode local \
+  --auth-choice zai-api-key \
+  --zai-api-key "$ZAI_API_KEY" \
+  --gateway-port 18789 \
+  --gateway-bind loopback
+```
+
+Vercel AI Gateway example:
+
+```bash
+clawdbot onboard --non-interactive \
+  --mode local \
+  --auth-choice ai-gateway-api-key \
+  --ai-gateway-api-key "$AI_GATEWAY_API_KEY" \
+  --gateway-port 18789 \
+  --gateway-bind loopback
+```
+
+Moonshot example:
+
+```bash
+clawdbot onboard --non-interactive \
+  --mode local \
+  --auth-choice moonshot-api-key \
+  --moonshot-api-key "$MOONSHOT_API_KEY" \
+  --gateway-port 18789 \
+  --gateway-bind loopback
+```
+
+Synthetic example:
+
+```bash
+clawdbot onboard --non-interactive \
+  --mode local \
+  --auth-choice synthetic-api-key \
+  --synthetic-api-key "$SYNTHETIC_API_KEY" \
+  --gateway-port 18789 \
+  --gateway-bind loopback
+```
+
+OpenCode Zen example:
+
+```bash
+clawdbot onboard --non-interactive \
+  --mode local \
+  --auth-choice opencode-zen \
+  --opencode-zen-api-key "$OPENCODE_API_KEY" \
+  --gateway-port 18789 \
+  --gateway-bind loopback
+```
+
+Add agent (non‑interactive) example:
+
+```bash
+clawdbot agents add work \
+  --workspace ~/clawd-work \
+  --model openai/gpt-5.2 \
+  --bind whatsapp:biz \
+  --non-interactive \
+  --json
+```
+
+## Gateway wizard RPC
+
+The Gateway exposes the wizard flow over RPC (`wizard.start`, `wizard.next`, `wizard.cancel`, `wizard.status`).
+Clients (macOS app, Control UI) can render steps without re‑implementing onboarding logic.
+
+## Signal setup (signal-cli)
+
+The wizard can install `signal-cli` from GitHub releases:
+- Downloads the appropriate release asset.
+- Stores it under `~/.clawdbot/tools/signal-cli/<version>/`.
+- Writes `channels.signal.cliPath` to your config.
+
+Notes:
+- JVM builds require **Java 21**.
+- Native builds are used when available.
+- Windows uses WSL2; signal-cli install follows the Linux flow inside WSL.
+
+## What the wizard writes
+
+Typical fields in `~/.clawdbot/clawdbot.json`:
+- `agents.defaults.workspace`
+- `agents.defaults.model` / `models.providers` (if Minimax chosen)
+- `gateway.*` (mode, bind, auth, tailscale)
+- `channels.telegram.botToken`, `channels.discord.token`, `channels.signal.*`, `channels.imessage.*`
+- Channel allowlists (Slack/Discord/Matrix/Microsoft Teams) when you opt in during the prompts (names resolve to IDs when possible).
+- `skills.install.nodeManager`
+- `wizard.lastRunAt`
+- `wizard.lastRunVersion`
+- `wizard.lastRunCommit`
+- `wizard.lastRunCommand`
+- `wizard.lastRunMode`
+
+`clawdbot agents add` writes `agents.list[]` and optional `bindings`.
+
+WhatsApp credentials go under `~/.clawdbot/credentials/whatsapp/<accountId>/`.
+Sessions are stored under `~/.clawdbot/agents/<agentId>/sessions/`.
+
+Some channels are delivered as plugins. When you pick one during onboarding, the wizard
+will prompt to install it (npm or a local path) before it can be configured.
+
+## Related docs
+
+- macOS app onboarding: [Onboarding](/start/onboarding)
+- Config reference: [Gateway configuration](/gateway/configuration)
+- Providers: [WhatsApp](/channels/whatsapp), [Telegram](/channels/telegram), [Discord](/channels/discord), [Google Chat](/channels/googlechat), [Signal](/channels/signal), [iMessage](/channels/imessage)
+- Skills: [Skills](/tools/skills), [Skills config](/tools/skills-config)

package/docs/testing.md

@@ -0,0 +1,349 @@
+---
+summary: "Testing kit: unit/e2e/live suites, Docker runners, and what each test covers"
+read_when:
+  - Running tests locally or in CI
+  - Adding regressions for model/provider bugs
+  - Debugging gateway + agent behavior
+---
+
+# Testing
+
+Clawdbot has three Vitest suites (unit/integration, e2e, live) and a small set of Docker runners.
+
+This doc is a “how we test” guide:
+- What each suite covers (and what it deliberately does *not* cover)
+- Which commands to run for common workflows (local, pre-push, debugging)
+- How live tests discover credentials and select models/providers
+- How to add regressions for real-world model/provider issues
+
+## Quick start
+
+Most days:
+- Full gate (expected before push): `pnpm lint && pnpm build && pnpm test`
+
+When you touch tests or want extra confidence:
+- Coverage gate: `pnpm test:coverage`
+- E2E suite: `pnpm test:e2e`
+
+When debugging real providers/models (requires real creds):
+- Live suite (models + gateway tool/image probes): `pnpm test:live`
+
+Tip: when you only need one failing case, prefer narrowing live tests via the allowlist env vars described below.
+
+## Test suites (what runs where)
+
+Think of the suites as “increasing realism” (and increasing flakiness/cost):
+
+### Unit / integration (default)
+
+- Command: `pnpm test`
+- Config: `vitest.config.ts`
+- Files: `src/**/*.test.ts`
+- Scope:
+  - Pure unit tests
+  - In-process integration tests (gateway auth, routing, tooling, parsing, config)
+  - Deterministic regressions for known bugs
+- Expectations:
+  - Runs in CI
+  - No real keys required
+  - Should be fast and stable
+
+### E2E (gateway smoke)
+
+- Command: `pnpm test:e2e`
+- Config: `vitest.e2e.config.ts`
+- Files: `src/**/*.e2e.test.ts`
+- Scope:
+  - Multi-instance gateway end-to-end behavior
+  - WebSocket/HTTP surfaces, node pairing, and heavier networking
+- Expectations:
+  - Runs in CI (when enabled in the pipeline)
+  - No real keys required
+  - More moving parts than unit tests (can be slower)
+
+### Live (real providers + real models)
+
+- Command: `pnpm test:live`
+- Config: `vitest.live.config.ts`
+- Files: `src/**/*.live.test.ts`
+- Default: **enabled** by `pnpm test:live` (sets `CLAWDBOT_LIVE_TEST=1`)
+- Scope:
+  - “Does this provider/model actually work *today* with real creds?”
+  - Catch provider format changes, tool-calling quirks, auth issues, and rate limit behavior
+- Expectations:
+  - Not CI-stable by design (real networks, real provider policies, quotas, outages)
+  - Costs money / uses rate limits
+  - Prefer running narrowed subsets instead of “everything”
+  - Live runs will source `~/.profile` to pick up missing API keys
+  - Anthropic key rotation: set `CLAWDBOT_LIVE_ANTHROPIC_KEYS="sk-...,sk-..."` (or `CLAWDBOT_LIVE_ANTHROPIC_KEY=sk-...`) or multiple `ANTHROPIC_API_KEY*` vars; tests will retry on rate limits
+
+## Which suite should I run?
+
+Use this decision table:
+- Editing logic/tests: run `pnpm test` (and `pnpm test:coverage` if you changed a lot)
+- Touching gateway networking / WS protocol / pairing: add `pnpm test:e2e`
+- Debugging “my bot is down” / provider-specific failures / tool calling: run a narrowed `pnpm test:live`
+
+## Live: model smoke (profile keys)
+
+Live tests are split into two layers so we can isolate failures:
+- “Direct model” tells us the provider/model can answer at all with the given key.
+- “Gateway smoke” tells us the full gateway+agent pipeline works for that model (sessions, history, tools, sandbox policy, etc.).
+
+### Layer 1: Direct model completion (no gateway)
+
+- Test: `src/agents/models.profiles.live.test.ts`
+- Goal:
+  - Enumerate discovered models
+  - Use `getApiKeyForModel` to select models you have creds for
+  - Run a small completion per model (and targeted regressions where needed)
+- How to enable:
+  - `pnpm test:live` (or `CLAWDBOT_LIVE_TEST=1` if invoking Vitest directly)
+- Set `CLAWDBOT_LIVE_MODELS=modern` (or `all`, alias for modern) to actually run this suite; otherwise it skips to keep `pnpm test:live` focused on gateway smoke
+- How to select models:
+  - `CLAWDBOT_LIVE_MODELS=modern` to run the modern allowlist (Opus/Sonnet/Haiku 4.5, GPT-5.x + Codex, Gemini 3, GLM 4.7, MiniMax M2.1, Grok 4)
+  - `CLAWDBOT_LIVE_MODELS=all` is an alias for the modern allowlist
+  - or `CLAWDBOT_LIVE_MODELS="openai/gpt-5.2,anthropic/claude-opus-4-5,..."` (comma allowlist)
+- How to select providers:
+  - `CLAWDBOT_LIVE_PROVIDERS="google,google-antigravity,google-gemini-cli"` (comma allowlist)
+- Where keys come from:
+  - By default: profile store and env fallbacks
+  - Set `CLAWDBOT_LIVE_REQUIRE_PROFILE_KEYS=1` to enforce **profile store** only
+- Why this exists:
+  - Separates “provider API is broken / key is invalid” from “gateway agent pipeline is broken”
+  - Contains small, isolated regressions (example: OpenAI Responses/Codex Responses reasoning replay + tool-call flows)
+
+### Layer 2: Gateway + dev agent smoke (what “@clawdbot” actually does)
+
+- Test: `src/gateway/gateway-models.profiles.live.test.ts`
+- Goal:
+  - Spin up an in-process gateway
+  - Create/patch a `agent:dev:*` session (model override per run)
+  - Iterate models-with-keys and assert:
+    - “meaningful” response (no tools)
+    - a real tool invocation works (read probe)
+    - optional extra tool probes (exec+read probe)
+    - OpenAI regression paths (tool-call-only → follow-up) keep working
+- Probe details (so you can explain failures quickly):
+  - `read` probe: the test writes a nonce file in the workspace and asks the agent to `read` it and echo the nonce back.
+  - `exec+read` probe: the test asks the agent to `exec`-write a nonce into a temp file, then `read` it back.
+  - image probe: the test attaches a generated PNG (cat + randomized code) and expects the model to return `cat <CODE>`.
+  - Implementation reference: `src/gateway/gateway-models.profiles.live.test.ts` and `src/gateway/live-image-probe.ts`.
+- How to enable:
+  - `pnpm test:live` (or `CLAWDBOT_LIVE_TEST=1` if invoking Vitest directly)
+- How to select models:
+  - Default: modern allowlist (Opus/Sonnet/Haiku 4.5, GPT-5.x + Codex, Gemini 3, GLM 4.7, MiniMax M2.1, Grok 4)
+  - `CLAWDBOT_LIVE_GATEWAY_MODELS=all` is an alias for the modern allowlist
+  - Or set `CLAWDBOT_LIVE_GATEWAY_MODELS="provider/model"` (or comma list) to narrow
+- How to select providers (avoid “OpenRouter everything”):
+  - `CLAWDBOT_LIVE_GATEWAY_PROVIDERS="google,google-antigravity,google-gemini-cli,openai,anthropic,zai,minimax"` (comma allowlist)
+- Tool + image probes are always on in this live test:
+  - `read` probe + `exec+read` probe (tool stress)
+  - image probe runs when the model advertises image input support
+  - Flow (high level):
+    - Test generates a tiny PNG with “CAT” + random code (`src/gateway/live-image-probe.ts`)
+    - Sends it via `agent` `attachments: [{ mimeType: "image/png", content: "<base64>" }]`
+    - Gateway parses attachments into `images[]` (`src/gateway/server-methods/agent.ts` + `src/gateway/chat-attachments.ts`)
+    - Embedded agent forwards a multimodal user message to the model
+    - Assertion: reply contains `cat` + the code (OCR tolerance: minor mistakes allowed)
+
+Tip: to see what you can test on your machine (and the exact `provider/model` ids), run:
+
+```bash
+clawdbot models list
+clawdbot models list --json
+```
+
+## Live: Anthropic setup-token smoke
+
+- Test: `src/agents/anthropic.setup-token.live.test.ts`
+- Goal: verify Claude Code CLI setup-token (or a pasted setup-token profile) can complete an Anthropic prompt.
+- Enable:
+  - `pnpm test:live` (or `CLAWDBOT_LIVE_TEST=1` if invoking Vitest directly)
+  - `CLAWDBOT_LIVE_SETUP_TOKEN=1`
+- Token sources (pick one):
+  - Profile: `CLAWDBOT_LIVE_SETUP_TOKEN_PROFILE=anthropic:setup-token-test`
+  - Raw token: `CLAWDBOT_LIVE_SETUP_TOKEN_VALUE=sk-ant-oat01-...`
+- Model override (optional):
+  - `CLAWDBOT_LIVE_SETUP_TOKEN_MODEL=anthropic/claude-opus-4-5`
+
+Setup example:
+
+```bash
+clawdbot models auth paste-token --provider anthropic --profile-id anthropic:setup-token-test
+CLAWDBOT_LIVE_SETUP_TOKEN=1 CLAWDBOT_LIVE_SETUP_TOKEN_PROFILE=anthropic:setup-token-test pnpm test:live src/agents/anthropic.setup-token.live.test.ts
+```
+
+## Live: CLI backend smoke (Claude Code CLI or other local CLIs)
+
+- Test: `src/gateway/gateway-cli-backend.live.test.ts`
+- Goal: validate the Gateway + agent pipeline using a local CLI backend, without touching your default config.
+- Enable:
+  - `pnpm test:live` (or `CLAWDBOT_LIVE_TEST=1` if invoking Vitest directly)
+  - `CLAWDBOT_LIVE_CLI_BACKEND=1`
+- Defaults:
+  - Model: `claude-cli/claude-sonnet-4-5`
+  - Command: `claude`
+  - Args: `["-p","--output-format","json","--dangerously-skip-permissions"]`
+- Overrides (optional):
+  - `CLAWDBOT_LIVE_CLI_BACKEND_MODEL="claude-cli/claude-opus-4-5"`
+  - `CLAWDBOT_LIVE_CLI_BACKEND_MODEL="codex-cli/gpt-5.2-codex"`
+  - `CLAWDBOT_LIVE_CLI_BACKEND_COMMAND="/full/path/to/claude"`
+  - `CLAWDBOT_LIVE_CLI_BACKEND_ARGS='["-p","--output-format","json","--permission-mode","bypassPermissions"]'`
+  - `CLAWDBOT_LIVE_CLI_BACKEND_CLEAR_ENV='["ANTHROPIC_API_KEY","ANTHROPIC_API_KEY_OLD"]'`
+  - `CLAWDBOT_LIVE_CLI_BACKEND_IMAGE_PROBE=1` to send a real image attachment (paths are injected into the prompt).
+  - `CLAWDBOT_LIVE_CLI_BACKEND_IMAGE_ARG="--image"` to pass image file paths as CLI args instead of prompt injection.
+  - `CLAWDBOT_LIVE_CLI_BACKEND_IMAGE_MODE="repeat"` (or `"list"`) to control how image args are passed when `IMAGE_ARG` is set.
+  - `CLAWDBOT_LIVE_CLI_BACKEND_RESUME_PROBE=1` to send a second turn and validate resume flow.
+- `CLAWDBOT_LIVE_CLI_BACKEND_DISABLE_MCP_CONFIG=0` to keep Claude Code CLI MCP config enabled (default disables MCP config with a temporary empty file).
+
+Example:
+
+```bash
+CLAWDBOT_LIVE_CLI_BACKEND=1 \
+  CLAWDBOT_LIVE_CLI_BACKEND_MODEL="claude-cli/claude-sonnet-4-5" \
+  pnpm test:live src/gateway/gateway-cli-backend.live.test.ts
+```
+
+### Recommended live recipes
+
+Narrow, explicit allowlists are fastest and least flaky:
+
+- Single model, direct (no gateway):
+  - `CLAWDBOT_LIVE_MODELS="openai/gpt-5.2" pnpm test:live src/agents/models.profiles.live.test.ts`
+
+- Single model, gateway smoke:
+  - `CLAWDBOT_LIVE_GATEWAY_MODELS="openai/gpt-5.2" pnpm test:live src/gateway/gateway-models.profiles.live.test.ts`
+
+- Tool calling across several providers:
+  - `CLAWDBOT_LIVE_GATEWAY_MODELS="openai/gpt-5.2,anthropic/claude-opus-4-5,google/gemini-3-flash-preview,zai/glm-4.7,minimax/minimax-m2.1" pnpm test:live src/gateway/gateway-models.profiles.live.test.ts`
+
+- Google focus (Gemini API key + Antigravity):
+  - Gemini (API key): `CLAWDBOT_LIVE_GATEWAY_MODELS="google/gemini-3-flash-preview" pnpm test:live src/gateway/gateway-models.profiles.live.test.ts`
+  - Antigravity (OAuth): `CLAWDBOT_LIVE_GATEWAY_MODELS="google-antigravity/claude-opus-4-5-thinking,google-antigravity/gemini-3-pro-high" pnpm test:live src/gateway/gateway-models.profiles.live.test.ts`
+
+Notes:
+- `google/...` uses the Gemini API (API key).
+- `google-antigravity/...` uses the Antigravity OAuth bridge (Cloud Code Assist-style agent endpoint).
+- `google-gemini-cli/...` uses the local Gemini CLI on your machine (separate auth + tooling quirks).
+- Gemini API vs Gemini CLI:
+  - API: Clawdbot calls Google’s hosted Gemini API over HTTP (API key / profile auth); this is what most users mean by “Gemini”.
+  - CLI: Clawdbot shells out to a local `gemini` binary; it has its own auth and can behave differently (streaming/tool support/version skew).
+
+## Live: model matrix (what we cover)
+
+There is no fixed “CI model list” (live is opt-in), but these are the **recommended** models to cover regularly on a dev machine with keys.
+
+### Modern smoke set (tool calling + image)
+
+This is the “common models” run we expect to keep working:
+- OpenAI (non-Codex): `openai/gpt-5.2` (optional: `openai/gpt-5.1`)
+- OpenAI Codex: `openai-codex/gpt-5.2` (optional: `openai-codex/gpt-5.2-codex`)
+- Anthropic: `anthropic/claude-opus-4-5` (or `anthropic/claude-sonnet-4-5`)
+- Google (Gemini API): `google/gemini-3-pro-preview` and `google/gemini-3-flash-preview` (avoid older Gemini 2.x models)
+- Google (Antigravity): `google-antigravity/claude-opus-4-5-thinking` and `google-antigravity/gemini-3-flash`
+- Z.AI (GLM): `zai/glm-4.7`
+- MiniMax: `minimax/minimax-m2.1`
+
+Run gateway smoke with tools + image:
+`CLAWDBOT_LIVE_GATEWAY_MODELS="openai/gpt-5.2,openai-codex/gpt-5.2,anthropic/claude-opus-4-5,google/gemini-3-pro-preview,google/gemini-3-flash-preview,google-antigravity/claude-opus-4-5-thinking,google-antigravity/gemini-3-flash,zai/glm-4.7,minimax/minimax-m2.1" pnpm test:live src/gateway/gateway-models.profiles.live.test.ts`
+
+### Baseline: tool calling (Read + optional Exec)
+
+Pick at least one per provider family:
+- OpenAI: `openai/gpt-5.2` (or `openai/gpt-5-mini`)
+- Anthropic: `anthropic/claude-opus-4-5` (or `anthropic/claude-sonnet-4-5`)
+- Google: `google/gemini-3-flash-preview` (or `google/gemini-3-pro-preview`)
+- Z.AI (GLM): `zai/glm-4.7`
+- MiniMax: `minimax/minimax-m2.1`
+
+Optional additional coverage (nice to have):
+- xAI: `xai/grok-4` (or latest available)
+- Mistral: `mistral/`… (pick one “tools” capable model you have enabled)
+- Cerebras: `cerebras/`… (if you have access)
+- LM Studio: `lmstudio/`… (local; tool calling depends on API mode)
+
+### Vision: image send (attachment → multimodal message)
+
+Include at least one image-capable model in `CLAWDBOT_LIVE_GATEWAY_MODELS` (Claude/Gemini/OpenAI vision-capable variants, etc.) to exercise the image probe.
+
+### Aggregators / alternate gateways
+
+If you have keys enabled, we also support testing via:
+- OpenRouter: `openrouter/...` (hundreds of models; use `clawdbot models scan` to find tool+image capable candidates)
+- OpenCode Zen: `opencode/...` (auth via `OPENCODE_API_KEY` / `OPENCODE_ZEN_API_KEY`)
+
+More providers you can include in the live matrix (if you have creds/config):
+- Built-in: `openai`, `openai-codex`, `anthropic`, `google`, `google-vertex`, `google-antigravity`, `google-gemini-cli`, `zai`, `openrouter`, `opencode`, `xai`, `groq`, `cerebras`, `mistral`, `github-copilot`
+- Via `models.providers` (custom endpoints): `minimax` (cloud/API), plus any OpenAI/Anthropic-compatible proxy (LM Studio, vLLM, LiteLLM, etc.)
+
+Tip: don’t try to hardcode “all models” in docs. The authoritative list is whatever `discoverModels(...)` returns on your machine + whatever keys are available.
+
+## Credentials (never commit)
+
+Live tests discover credentials the same way the CLI does. Practical implications:
+- If the CLI works, live tests should find the same keys.
+- If a live test says “no creds”, debug the same way you’d debug `clawdbot models list` / model selection.
+
+- Profile store: `~/.clawdbot/credentials/` (preferred; what “profile keys” means in the tests)
+- Config: `~/.clawdbot/clawdbot.json` (or `CLAWDBOT_CONFIG_PATH`)
+
+If you want to rely on env keys (e.g. exported in your `~/.profile`), run local tests after `source ~/.profile`, or use the Docker runners below (they can mount `~/.profile` into the container).
+
+## Deepgram live (audio transcription)
+
+- Test: `src/media-understanding/providers/deepgram/audio.live.test.ts`
+- Enable: `DEEPGRAM_API_KEY=... DEEPGRAM_LIVE_TEST=1 pnpm test:live src/media-understanding/providers/deepgram/audio.live.test.ts`
+
+## Docker runners (optional “works in Linux” checks)
+
+These run `pnpm test:live` inside the repo Docker image, mounting your local config dir and workspace (and sourcing `~/.profile` if mounted):
+
+- Direct models: `pnpm test:docker:live-models` (script: `scripts/test-live-models-docker.sh`)
+- Gateway + dev agent: `pnpm test:docker:live-gateway` (script: `scripts/test-live-gateway-models-docker.sh`)
+- Onboarding wizard (TTY, full scaffolding): `pnpm test:docker:onboard` (script: `scripts/e2e/onboard-docker.sh`)
+- Gateway networking (two containers, WS auth + health): `pnpm test:docker:gateway-network` (script: `scripts/e2e/gateway-network-docker.sh`)
+- Plugins (custom extension load + registry smoke): `pnpm test:docker:plugins` (script: `scripts/e2e/plugins-docker.sh`)
+
+Useful env vars:
+
+- `CLAWDBOT_CONFIG_DIR=...` (default: `~/.clawdbot`) mounted to `/home/node/.clawdbot`
+- `CLAWDBOT_WORKSPACE_DIR=...` (default: `~/clawd`) mounted to `/home/node/clawd`
+- `CLAWDBOT_PROFILE_FILE=...` (default: `~/.profile`) mounted to `/home/node/.profile` and sourced before running tests
+- `CLAWDBOT_LIVE_GATEWAY_MODELS=...` / `CLAWDBOT_LIVE_MODELS=...` to narrow the run
+- `CLAWDBOT_LIVE_REQUIRE_PROFILE_KEYS=1` to ensure creds come from the profile store (not env)
+
+## Docs sanity
+
+Run docs checks after doc edits: `pnpm docs:list`.
+
+## Offline regression (CI-safe)
+
+These are “real pipeline” regressions without real providers:
+- Gateway tool calling (mock OpenAI, real gateway + agent loop): `src/gateway/gateway.tool-calling.mock-openai.test.ts`
+- Gateway wizard (WS `wizard.start`/`wizard.next`, writes config + auth enforced): `src/gateway/gateway.wizard.e2e.test.ts`
+
+## Agent reliability evals (skills)
+
+We already have a few CI-safe tests that behave like “agent reliability evals”:
+- Mock tool-calling through the real gateway + agent loop (`src/gateway/gateway.tool-calling.mock-openai.test.ts`).
+- End-to-end wizard flows that validate session wiring and config effects (`src/gateway/gateway.wizard.e2e.test.ts`).
+
+What’s still missing for skills (see [Skills](/tools/skills)):
+- **Decisioning:** when skills are listed in the prompt, does the agent pick the right skill (or avoid irrelevant ones)?
+- **Compliance:** does the agent read `SKILL.md` before use and follow required steps/args?
+- **Workflow contracts:** multi-turn scenarios that assert tool order, session history carryover, and sandbox boundaries.
+
+Future evals should stay deterministic first:
+- A scenario runner using mock providers to assert tool calls + order, skill file reads, and session wiring.
+- A small suite of skill-focused scenarios (use vs avoid, gating, prompt injection).
+- Optional live evals (opt-in, env-gated) only after the CI-safe suite is in place.
+
+## Adding regressions (guidance)
+
+When you fix a provider/model issue discovered in live:
+- Add a CI-safe regression if possible (mock/stub provider, or capture the exact request-shape transformation)
+- If it’s inherently live-only (rate limits, auth policies), keep the live test narrow and opt-in via env vars
+- Prefer targeting the smallest layer that catches the bug:
+  - provider request conversion/replay bug → direct models test
+  - gateway session/history/tool pipeline bug → gateway live smoke or CI-safe gateway mock test