loudmouth-ai 0.1.0

package/docs/concepts/agent-workspace.md

@@ -0,0 +1,231 @@
+---
+summary: "Agent workspace: location, layout, and backup strategy"
+read_when:
+  - You need to explain the agent workspace or its file layout
+  - You want to back up or migrate an agent workspace
+---
+# Agent workspace
+
+The workspace is the agent's home. It is the only working directory used for
+file tools and for workspace context. Keep it private and treat it as memory.
+
+This is separate from `~/.clawdbot/`, which stores config, credentials, and
+sessions.
+
+**Important:** the workspace is the **default cwd**, not a hard sandbox. Tools
+resolve relative paths against the workspace, but absolute paths can still reach
+elsewhere on the host unless sandboxing is enabled. If you need isolation, use
+[`agents.defaults.sandbox`](/gateway/sandboxing) (and/or per‑agent sandbox config).
+When sandboxing is enabled and `workspaceAccess` is not `"rw"`, tools operate
+inside a sandbox workspace under `~/.clawdbot/sandboxes`, not your host workspace.
+
+## Default location
+
+- Default: `~/clawd`
+- If `CLAWDBOT_PROFILE` is set and not `"default"`, the default becomes
+  `~/clawd-<profile>`.
+- Override in `~/.clawdbot/clawdbot.json`:
+
+```json5
+{
+  agent: {
+    workspace: "~/clawd"
+  }
+}
+```
+
+`clawdbot onboard`, `clawdbot configure`, or `clawdbot setup` will create the
+workspace and seed the bootstrap files if they are missing.
+
+If you already manage the workspace files yourself, you can disable bootstrap
+file creation:
+
+```json5
+{ agent: { skipBootstrap: true } }
+```
+
+## Extra workspace folders
+
+Older installs may have created `~/clawdbot`. Keeping multiple workspace
+directories around can cause confusing auth or state drift, because only one
+workspace is active at a time.
+
+**Recommendation:** keep a single active workspace. If you no longer use the
+extra folders, archive or move them to Trash (for example `trash ~/clawdbot`).
+If you intentionally keep multiple workspaces, make sure
+`agents.defaults.workspace` points to the active one.
+
+`clawdbot doctor` warns when it detects extra workspace directories.
+
+## Workspace file map (what each file means)
+
+These are the standard files Clawdbot expects inside the workspace:
+
+- `AGENTS.md`
+  - Operating instructions for the agent and how it should use memory.
+  - Loaded at the start of every session.
+  - Good place for rules, priorities, and "how to behave" details.
+
+- `SOUL.md`
+  - Persona, tone, and boundaries.
+  - Loaded every session.
+
+- `USER.md`
+  - Who the user is and how to address them.
+  - Loaded every session.
+
+- `IDENTITY.md`
+  - The agent's name, vibe, and emoji.
+  - Created/updated during the bootstrap ritual.
+
+- `TOOLS.md`
+  - Notes about your local tools and conventions.
+  - Does not control tool availability; it is only guidance.
+
+- `HEARTBEAT.md`
+  - Optional tiny checklist for heartbeat runs.
+  - Keep it short to avoid token burn.
+
+- `BOOT.md`
+  - Optional startup checklist executed on gateway restart when internal hooks are enabled.
+  - Keep it short; use the message tool for outbound sends.
+
+- `BOOTSTRAP.md`
+  - One-time first-run ritual.
+  - Only created for a brand-new workspace.
+  - Delete it after the ritual is complete.
+
+- `memory/YYYY-MM-DD.md`
+  - Daily memory log (one file per day).
+  - Recommended to read today + yesterday on session start.
+
+- `MEMORY.md` (optional)
+  - Curated long-term memory.
+  - Only load in the main, private session (not shared/group contexts).
+
+See [Memory](/concepts/memory) for the workflow and automatic memory flush.
+
+- `skills/` (optional)
+  - Workspace-specific skills.
+  - Overrides managed/bundled skills when names collide.
+
+- `canvas/` (optional)
+  - Canvas UI files for node displays (for example `canvas/index.html`).
+
+If any bootstrap file is missing, Clawdbot injects a "missing file" marker into
+the session and continues. Large bootstrap files are truncated when injected;
+adjust the limit with `agents.defaults.bootstrapMaxChars` (default: 20000).
+`clawdbot setup` can recreate missing defaults without overwriting existing
+files.
+
+## What is NOT in the workspace
+
+These live under `~/.clawdbot/` and should NOT be committed to the workspace repo:
+
+- `~/.clawdbot/clawdbot.json` (config)
+- `~/.clawdbot/credentials/` (OAuth tokens, API keys)
+- `~/.clawdbot/agents/<agentId>/sessions/` (session transcripts + metadata)
+- `~/.clawdbot/skills/` (managed skills)
+
+If you need to migrate sessions or config, copy them separately and keep them
+out of version control.
+
+## Git backup (recommended, private)
+
+Treat the workspace as private memory. Put it in a **private** git repo so it is
+backed up and recoverable.
+
+Run these steps on the machine where the Gateway runs (that is where the
+workspace lives).
+
+### 1) Initialize the repo
+
+If git is installed, brand-new workspaces are initialized automatically. If this
+workspace is not already a repo, run:
+
+```bash
+cd ~/clawd
+git init
+git add AGENTS.md SOUL.md TOOLS.md IDENTITY.md USER.md HEARTBEAT.md memory/
+git commit -m "Add agent workspace"
+```
+
+### 2) Add a private remote (beginner-friendly options)
+
+Option A: GitHub web UI
+
+1. Create a new **private** repository on GitHub.
+2. Do not initialize with a README (avoids merge conflicts).
+3. Copy the HTTPS remote URL.
+4. Add the remote and push:
+
+```bash
+git branch -M main
+git remote add origin <https-url>
+git push -u origin main
+```
+
+Option B: GitHub CLI (`gh`)
+
+```bash
+gh auth login
+gh repo create clawd-workspace --private --source . --remote origin --push
+```
+
+Option C: GitLab web UI
+
+1. Create a new **private** repository on GitLab.
+2. Do not initialize with a README (avoids merge conflicts).
+3. Copy the HTTPS remote URL.
+4. Add the remote and push:
+
+```bash
+git branch -M main
+git remote add origin <https-url>
+git push -u origin main
+```
+
+### 3) Ongoing updates
+
+```bash
+git status
+git add .
+git commit -m "Update memory"
+git push
+```
+
+## Do not commit secrets
+
+Even in a private repo, avoid storing secrets in the workspace:
+
+- API keys, OAuth tokens, passwords, or private credentials.
+- Anything under `~/.clawdbot/`.
+- Raw dumps of chats or sensitive attachments.
+
+If you must store sensitive references, use placeholders and keep the real
+secret elsewhere (password manager, environment variables, or `~/.clawdbot/`).
+
+Suggested `.gitignore` starter:
+
+```gitignore
+.DS_Store
+.env
+**/*.key
+**/*.pem
+**/secrets*
+```
+
+## Moving the workspace to a new machine
+
+1. Clone the repo to the desired path (default `~/clawd`).
+2. Set `agents.defaults.workspace` to that path in `~/.clawdbot/clawdbot.json`.
+3. Run `clawdbot setup --workspace <path>` to seed any missing files.
+4. If you need sessions, copy `~/.clawdbot/agents/<agentId>/sessions/` from the
+   old machine separately.
+
+## Advanced notes
+
+- Multi-agent routing can use different workspaces per agent. See
+  [Channel routing](/concepts/channel-routing) for routing configuration.
+- If `agents.defaults.sandbox` is enabled, non-main sessions can use per-session sandbox
+  workspaces under `agents.defaults.sandbox.workspaceRoot`.

package/docs/concepts/agent.md

@@ -0,0 +1,117 @@
+---
+summary: "Agent runtime (embedded p-mono), workspace contract, and session bootstrap"
+read_when:
+  - Changing agent runtime, workspace bootstrap, or session behavior
+---
+# Agent Runtime 🤖
+
+Clawdbot runs a single embedded agent runtime derived from **p-mono**.
+
+## Workspace (required)
+
+Clawdbot uses a single agent workspace directory (`agents.defaults.workspace`) as the agent’s **only** working directory (`cwd`) for tools and context.
+
+Recommended: use `clawdbot setup` to create `~/.clawdbot/clawdbot.json` if missing and initialize the workspace files.
+
+Full workspace layout + backup guide: [Agent workspace](/concepts/agent-workspace)
+
+If `agents.defaults.sandbox` is enabled, non-main sessions can override this with
+per-session workspaces under `agents.defaults.sandbox.workspaceRoot` (see
+[Gateway configuration](/gateway/configuration)).
+
+## Bootstrap files (injected)
+
+Inside `agents.defaults.workspace`, Clawdbot expects these user-editable files:
+- `AGENTS.md` — operating instructions + “memory”
+- `SOUL.md` — persona, boundaries, tone
+- `TOOLS.md` — user-maintained tool notes (e.g. `imsg`, `sag`, conventions)
+- `BOOTSTRAP.md` — one-time first-run ritual (deleted after completion)
+- `IDENTITY.md` — agent name/vibe/emoji
+- `USER.md` — user profile + preferred address
+
+On the first turn of a new session, Clawdbot injects the contents of these files directly into the agent context.
+
+Blank files are skipped. Large files are trimmed and truncated with a marker so prompts stay lean (read the file for full content).
+
+If a file is missing, Clawdbot injects a single “missing file” marker line (and `clawdbot setup` will create a safe default template).
+
+`BOOTSTRAP.md` is only created for a **brand new workspace** (no other bootstrap files present). If you delete it after completing the ritual, it should not be recreated on later restarts.
+
+To disable bootstrap file creation entirely (for pre-seeded workspaces), set:
+
+```json5
+{ agent: { skipBootstrap: true } }
+```
+
+## Built-in tools
+
+Core tools (read/exec/edit/write and related system tools) are always available,
+subject to tool policy. `apply_patch` is optional and gated by
+`tools.exec.applyPatch`. `TOOLS.md` does **not** control which tools exist; it’s
+guidance for how *you* want them used.
+
+## Skills
+
+Clawdbot loads skills from three locations (workspace wins on name conflict):
+- Bundled (shipped with the install)
+- Managed/local: `~/.clawdbot/skills`
+- Workspace: `<workspace>/skills`
+
+Skills can be gated by config/env (see `skills` in [Gateway configuration](/gateway/configuration)).
+
+## p-mono integration
+
+Clawdbot reuses pieces of the p-mono codebase (models/tools), but **session management, discovery, and tool wiring are Clawdbot-owned**.
+
+- No p-coding agent runtime.
+- No `~/.pi/agent` or `<workspace>/.pi` settings are consulted.
+
+## Sessions
+
+Session transcripts are stored as JSONL at:
+- `~/.clawdbot/agents/<agentId>/sessions/<SessionId>.jsonl`
+
+The session ID is stable and chosen by Clawdbot.
+Legacy Pi/Tau session folders are **not** read.
+
+## Steering while streaming
+
+When queue mode is `steer`, inbound messages are injected into the current run.
+The queue is checked **after each tool call**; if a queued message is present,
+remaining tool calls from the current assistant message are skipped (error tool
+results with "Skipped due to queued user message."), then the queued user
+message is injected before the next assistant response.
+
+When queue mode is `followup` or `collect`, inbound messages are held until the
+current turn ends, then a new agent turn starts with the queued payloads. See
+[Queue](/concepts/queue) for mode + debounce/cap behavior.
+
+Block streaming sends completed assistant blocks as soon as they finish; it is
+**off by default** (`agents.defaults.blockStreamingDefault: "off"`).
+Tune the boundary via `agents.defaults.blockStreamingBreak` (`text_end` vs `message_end`; defaults to text_end).
+Control soft block chunking with `agents.defaults.blockStreamingChunk` (defaults to
+800–1200 chars; prefers paragraph breaks, then newlines; sentences last).
+Coalesce streamed chunks with `agents.defaults.blockStreamingCoalesce` to reduce
+single-line spam (idle-based merging before send). Non-Telegram channels require
+explicit `*.blockStreaming: true` to enable block replies.
+Verbose tool summaries are emitted at tool start (no debounce); Control UI
+streams tool output via agent events when available.
+More details: [Streaming + chunking](/concepts/streaming).
+
+## Model refs
+
+Model refs in config (for example `agents.defaults.model` and `agents.defaults.models`) are parsed by splitting on the **first** `/`.
+
+- Use `provider/model` when configuring models.
+- If the model ID itself contains `/` (OpenRouter-style), include the provider prefix (example: `openrouter/moonshotai/kimi-k2`).
+- If you omit the provider, Clawdbot treats the input as an alias or a model for the **default provider** (only works when there is no `/` in the model ID).
+
+## Configuration (minimal)
+
+At minimum, set:
+- `agents.defaults.workspace`
+- `channels.whatsapp.allowFrom` (strongly recommended)
+
+---
+
+*Next: [Group Chats](/concepts/group-messages)* 🦞

package/docs/concepts/architecture.md

@@ -0,0 +1,122 @@
+---
+summary: "WebSocket gateway architecture, components, and client flows"
+read_when:
+  - Working on gateway protocol, clients, or transports
+---
+# Gateway architecture
+
+Last updated: 2026-01-22
+
+## Overview
+
+- A single long‑lived **Gateway** owns all messaging surfaces (WhatsApp via
+  Baileys, Telegram via grammY, Slack, Discord, Signal, iMessage, WebChat).
+- Control-plane clients (macOS app, CLI, web UI, automations) connect to the
+  Gateway over **WebSocket** on the configured bind host (default
+  `127.0.0.1:18789`).
+- **Nodes** (macOS/iOS/Android/headless) also connect over **WebSocket**, but
+  declare `role: node` with explicit caps/commands.
+- One Gateway per host; it is the only place that opens a WhatsApp session.
+- A **canvas host** (default `18793`) serves agent‑editable HTML and A2UI.
+
+## Components and flows
+
+### Gateway (daemon)
+- Maintains provider connections.
+- Exposes a typed WS API (requests, responses, server‑push events).
+- Validates inbound frames against JSON Schema.
+- Emits events like `agent`, `chat`, `presence`, `health`, `heartbeat`, `cron`.
+
+### Clients (mac app / CLI / web admin)
+- One WS connection per client.
+- Send requests (`health`, `status`, `send`, `agent`, `system-presence`).
+- Subscribe to events (`tick`, `agent`, `presence`, `shutdown`).
+
+### Nodes (macOS / iOS / Android / headless)
+- Connect to the **same WS server** with `role: node`.
+- Provide a device identity in `connect`; pairing is **device‑based** (role `node`) and
+  approval lives in the device pairing store.
+- Expose commands like `canvas.*`, `camera.*`, `screen.record`, `location.get`.
+
+Protocol details:
+- [Gateway protocol](/gateway/protocol)
+
+### WebChat
+- Static UI that uses the Gateway WS API for chat history and sends.
+- In remote setups, connects through the same SSH/Tailscale tunnel as other
+  clients.
+
+## Connection lifecycle (single client)
+
+```
+Client                    Gateway
+  |                          |
+  |---- req:connect -------->|
+  |<------ res (ok) ---------|   (or res error + close)
+  |   (payload=hello-ok carries snapshot: presence + health)
+  |                          |
+  |<------ event:presence ---|
+  |<------ event:tick -------|
+  |                          |
+  |------- req:agent ------->|
+  |<------ res:agent --------|   (ack: {runId,status:"accepted"})
+  |<------ event:agent ------|   (streaming)
+  |<------ res:agent --------|   (final: {runId,status,summary})
+  |                          |
+```
+
+## Wire protocol (summary)
+
+- Transport: WebSocket, text frames with JSON payloads.
+- First frame **must** be `connect`.
+- After handshake:
+  - Requests: `{type:"req", id, method, params}` → `{type:"res", id, ok, payload|error}`
+  - Events: `{type:"event", event, payload, seq?, stateVersion?}`
+- If `CLAWDBOT_GATEWAY_TOKEN` (or `--token`) is set, `connect.params.auth.token`
+  must match or the socket closes.
+- Idempotency keys are required for side‑effecting methods (`send`, `agent`) to
+  safely retry; the server keeps a short‑lived dedupe cache.
+- Nodes must include `role: "node"` plus caps/commands/permissions in `connect`.
+
+## Pairing + local trust
+
+- All WS clients (operators + nodes) include a **device identity** on `connect`.
+- New device IDs require pairing approval; the Gateway issues a **device token**
+  for subsequent connects.
+- **Local** connects (loopback or the gateway host’s own tailnet address) can be
+  auto‑approved to keep same‑host UX smooth.
+- **Non‑local** connects must sign the `connect.challenge` nonce and require
+  explicit approval.
+- Gateway auth (`gateway.auth.*`) still applies to **all** connections, local or
+  remote.
+
+Details: [Gateway protocol](/gateway/protocol), [Pairing](/start/pairing),
+[Security](/gateway/security).
+
+## Protocol typing and codegen
+
+- TypeBox schemas define the protocol.
+- JSON Schema is generated from those schemas.
+- Swift models are generated from the JSON Schema.
+
+## Remote access
+
+- Preferred: Tailscale or VPN.
+- Alternative: SSH tunnel
+  ```bash
+  ssh -N -L 18789:127.0.0.1:18789 user@host
+  ```
+- The same handshake + auth token apply over the tunnel.
+- TLS + optional pinning can be enabled for WS in remote setups.
+
+## Operations snapshot
+
+- Start: `clawdbot gateway` (foreground, logs to stdout).
+- Health: `health` over WS (also included in `hello-ok`).
+- Supervision: launchd/systemd for auto‑restart.
+
+## Invariants
+
+- Exactly one Gateway controls a single Baileys session per host.
+- Handshake is mandatory; any non‑JSON or non‑connect first frame is a hard close.
+- Events are not replayed; clients must refresh on gaps.

package/docs/concepts/channel-routing.md

@@ -0,0 +1,114 @@
+---
+summary: "Routing rules per channel (WhatsApp, Telegram, Discord, Slack) and shared context"
+read_when:
+  - Changing channel routing or inbox behavior
+---
+# Channels & routing
+
+
+Clawdbot routes replies **back to the channel where a message came from**. The
+model does not choose a channel; routing is deterministic and controlled by the
+host configuration.
+
+## Key terms
+
+- **Channel**: `whatsapp`, `telegram`, `discord`, `slack`, `signal`, `imessage`, `webchat`.
+- **AccountId**: per‑channel account instance (when supported).
+- **AgentId**: an isolated workspace + session store (“brain”).
+- **SessionKey**: the bucket key used to store context and control concurrency.
+
+## Session key shapes (examples)
+
+Direct messages collapse to the agent’s **main** session:
+
+- `agent:<agentId>:<mainKey>` (default: `agent:main:main`)
+
+Groups and channels remain isolated per channel:
+
+- Groups: `agent:<agentId>:<channel>:group:<id>`
+- Channels/rooms: `agent:<agentId>:<channel>:channel:<id>`
+
+Threads:
+
+- Slack/Discord threads append `:thread:<threadId>` to the base key.
+- Telegram forum topics embed `:topic:<topicId>` in the group key.
+
+Examples:
+
+- `agent:main:telegram:group:-1001234567890:topic:42`
+- `agent:main:discord:channel:123456:thread:987654`
+
+## Routing rules (how an agent is chosen)
+
+Routing picks **one agent** for each inbound message:
+
+1. **Exact peer match** (`bindings` with `peer.kind` + `peer.id`).
+2. **Guild match** (Discord) via `guildId`.
+3. **Team match** (Slack) via `teamId`.
+4. **Account match** (`accountId` on the channel).
+5. **Channel match** (any account on that channel).
+6. **Default agent** (`agents.list[].default`, else first list entry, fallback to `main`).
+
+The matched agent determines which workspace and session store are used.
+
+## Broadcast groups (run multiple agents)
+
+Broadcast groups let you run **multiple agents** for the same peer **when Clawdbot would normally reply** (for example: in WhatsApp groups, after mention/activation gating).
+
+Config:
+
+```json5
+{
+  broadcast: {
+    strategy: "parallel",
+    "120363403215116621@g.us": ["alfred", "baerbel"],
+    "+15555550123": ["support", "logger"]
+  }
+}
+```
+
+See: [Broadcast Groups](/broadcast-groups).
+
+## Config overview
+
+- `agents.list`: named agent definitions (workspace, model, etc.).
+- `bindings`: map inbound channels/accounts/peers to agents.
+
+Example:
+
+```json5
+{
+  agents: {
+    list: [
+      { id: "support", name: "Support", workspace: "~/clawd-support" }
+    ]
+  },
+  bindings: [
+    { match: { channel: "slack", teamId: "T123" }, agentId: "support" },
+    { match: { channel: "telegram", peer: { kind: "group", id: "-100123" } }, agentId: "support" }
+  ]
+}
+```
+
+## Session storage
+
+Session stores live under the state directory (default `~/.clawdbot`):
+
+- `~/.clawdbot/agents/<agentId>/sessions/sessions.json`
+- JSONL transcripts live alongside the store
+
+You can override the store path via `session.store` and `{agentId}` templating.
+
+## WebChat behavior
+
+WebChat attaches to the **selected agent** and defaults to the agent’s main
+session. Because of this, WebChat lets you see cross‑channel context for that
+agent in one place.
+
+## Reply context
+
+Inbound replies include:
+- `ReplyToId`, `ReplyToBody`, and `ReplyToSender` when available.
+- Quoted context is appended to `Body` as a `[Replying to ...]` block.
+
+This is consistent across channels.

package/docs/concepts/compaction.md

@@ -0,0 +1,49 @@
+---
+summary: "Context window + compaction: how Clawdbot keeps sessions under model limits"
+read_when:
+  - You want to understand auto-compaction and /compact
+  - You are debugging long sessions hitting context limits
+---
+# Context Window & Compaction
+
+Every model has a **context window** (max tokens it can see). Long-running chats accumulate messages and tool results; once the window is tight, Clawdbot **compacts** older history to stay within limits.
+
+## What compaction is
+Compaction **summarizes older conversation** into a compact summary entry and keeps recent messages intact. The summary is stored in the session history, so future requests use:
+- The compaction summary
+- Recent messages after the compaction point
+
+Compaction **persists** in the session’s JSONL history.
+
+## Configuration
+See [Compaction config & modes](/concepts/compaction) for the `agents.defaults.compaction` settings.
+
+## Auto-compaction (default on)
+When a session nears or exceeds the model’s context window, Clawdbot triggers auto-compaction and may retry the original request using the compacted context.
+
+You’ll see:
+- `🧹 Auto-compaction complete` in verbose mode
+- `/status` showing `🧹 Compactions: <count>`
+
+Before compaction, Clawdbot can run a **silent memory flush** turn to store
+durable notes to disk. See [Memory](/concepts/memory) for details and config.
+
+## Manual compaction
+Use `/compact` (optionally with instructions) to force a compaction pass:
+```
+/compact Focus on decisions and open questions
+```
+
+## Context window source
+Context window is model-specific. Clawdbot uses the model definition from the configured provider catalog to determine limits.
+
+## Compaction vs pruning
+- **Compaction**: summarises and **persists** in JSONL.
+- **Session pruning**: trims old **tool results** only, **in-memory**, per request.
+
+See [/concepts/session-pruning](/concepts/session-pruning) for pruning details.
+
+## Tips
+- Use `/compact` when sessions feel stale or context is bloated.
+- Large tool outputs are already truncated; pruning can further reduce tool-result buildup.
+- If you need a fresh slate, `/new` or `/reset` starts a new session id.