@symerian/symi 3.0.20 → 3.0.22

package/docs/concepts/agent-loop.md

@@ -1,148 +0,0 @@
----
-summary: "Agent loop lifecycle, streams, and wait semantics"
-read_when:
-  - You need an exact walkthrough of the agent loop or lifecycle events
-title: "Agent Loop"
----
-
-# Agent Loop (Symi)
-
-An agentic loop is the full “real” run of an agent: intake → context assembly → model inference →
-tool execution → streaming replies → persistence. It’s the authoritative path that turns a message
-into actions and a final reply, while keeping session state consistent.
-
-In Symi, a loop is a single, serialized run per session that emits lifecycle and stream events
-as the model thinks, calls tools, and streams output. This doc explains how that authentic loop is
-wired end-to-end.
-
-## Entry points
-
-- Gateway RPC: `agent` and `agent.wait`.
-- CLI: `agent` command.
-
-## How it works (high-level)
-
-1. `agent` RPC validates params, resolves session (sessionKey/sessionId), persists session metadata, returns `{ runId, acceptedAt }` immediately.
-2. `agentCommand` runs the agent:
-   - resolves model + thinking/verbose defaults
-   - loads skills snapshot
-   - calls `runEmbeddedPiAgent` (pi-agent-core runtime)
-   - emits **lifecycle end/error** if the embedded loop does not emit one
-3. `runEmbeddedPiAgent`:
-   - serializes runs via per-session + global queues
-   - resolves model + auth profile and builds the pi session
-   - subscribes to pi events and streams assistant/tool deltas
-   - enforces timeout -> aborts run if exceeded
-   - returns payloads + usage metadata
-4. `subscribeEmbeddedPiSession` bridges pi-agent-core events to Symi `agent` stream:
-   - tool events => `stream: "tool"`
-   - assistant deltas => `stream: "assistant"`
-   - lifecycle events => `stream: "lifecycle"` (`phase: "start" | "end" | "error"`)
-5. `agent.wait` uses `waitForAgentJob`:
-   - waits for **lifecycle end/error** for `runId`
-   - returns `{ status: ok|error|timeout, startedAt, endedAt, error? }`
-
-## Queueing + concurrency
-
-- Runs are serialized per session key (session lane) and optionally through a global lane.
-- This prevents tool/session races and keeps session history consistent.
-- Messaging channels can choose queue modes (collect/steer/followup) that feed this lane system.
-  See [Command Queue](/concepts/queue).
-
-## Session + workspace preparation
-
-- Workspace is resolved and created; sandboxed runs may redirect to a sandbox workspace root.
-- Skills are loaded (or reused from a snapshot) and injected into env and prompt.
-- Bootstrap/context files are resolved and injected into the system prompt report.
-- A session write lock is acquired; `SessionManager` is opened and prepared before streaming.
-
-## Prompt assembly + system prompt
-
-- System prompt is built from Symi’s base prompt, skills prompt, bootstrap context, and per-run overrides.
-- Model-specific limits and compaction reserve tokens are enforced.
-- See [System prompt](/concepts/system-prompt) for what the model sees.
-
-## Hook points (where you can intercept)
-
-Symi has two hook systems:
-
-- **Internal hooks** (Gateway hooks): event-driven scripts for commands and lifecycle events.
-- **Plugin hooks**: extension points inside the agent/tool lifecycle and gateway pipeline.
-
-### Internal hooks (Gateway hooks)
-
-- **`agent:bootstrap`**: runs while building bootstrap files before the system prompt is finalized.
-  Use this to add/remove bootstrap context files.
-- **Command hooks**: `/new`, `/reset`, `/stop`, and other command events (see Hooks doc).
-
-See [Hooks](/automation/hooks) for setup and examples.
-
-### Plugin hooks (agent + gateway lifecycle)
-
-These run inside the agent loop or gateway pipeline:
-
-- **`before_model_resolve`**: runs pre-session (no `messages`) to deterministically override provider/model before model resolution.
-- **`before_prompt_build`**: runs after session load (with `messages`) to inject `prependContext`/`systemPrompt` before prompt submission.
-- **`before_agent_start`**: legacy compatibility hook that may run in either phase; prefer the explicit hooks above.
-- **`agent_end`**: inspect the final message list and run metadata after completion.
-- **`before_compaction` / `after_compaction`**: observe or annotate compaction cycles.
-- **`before_tool_call` / `after_tool_call`**: intercept tool params/results.
-- **`tool_result_persist`**: synchronously transform tool results before they are written to the session transcript.
-- **`message_received` / `message_sending` / `message_sent`**: inbound + outbound message hooks.
-- **`session_start` / `session_end`**: session lifecycle boundaries.
-- **`gateway_start` / `gateway_stop`**: gateway lifecycle events.
-
-See [Plugins](/tools/plugin#plugin-hooks) for the hook API and registration details.
-
-## Streaming + partial replies
-
-- Assistant deltas are streamed from pi-agent-core and emitted as `assistant` events.
-- Block streaming can emit partial replies either on `text_end` or `message_end`.
-- Reasoning streaming can be emitted as a separate stream or as block replies.
-- See [Streaming](/concepts/streaming) for chunking and block reply behavior.
-
-## Tool execution + messaging tools
-
-- Tool start/update/end events are emitted on the `tool` stream.
-- Tool results are sanitized for size and image payloads before logging/emitting.
-- Messaging tool sends are tracked to suppress duplicate assistant confirmations.
-
-## Reply shaping + suppression
-
-- Final payloads are assembled from:
-  - assistant text (and optional reasoning)
-  - inline tool summaries (when verbose + allowed)
-  - assistant error text when the model errors
-- `NO_REPLY` is treated as a silent token and filtered from outgoing payloads.
-- Messaging tool duplicates are removed from the final payload list.
-- If no renderable payloads remain and a tool errored, a fallback tool error reply is emitted
-  (unless a messaging tool already sent a user-visible reply).
-
-## Compaction + retries
-
-- Auto-compaction emits `compaction` stream events and can trigger a retry.
-- On retry, in-memory buffers and tool summaries are reset to avoid duplicate output.
-- See [Compaction](/concepts/compaction) for the compaction pipeline.
-
-## Event streams (today)
-
-- `lifecycle`: emitted by `subscribeEmbeddedPiSession` (and as a fallback by `agentCommand`)
-- `assistant`: streamed deltas from pi-agent-core
-- `tool`: streamed tool events from pi-agent-core
-
-## Chat channel handling
-
-- Assistant deltas are buffered into chat `delta` messages.
-- A chat `final` is emitted on **lifecycle end/error**.
-
-## Timeouts
-
-- `agent.wait` default: 30s (just the wait). `timeoutMs` param overrides.
-- Agent runtime: `agents.defaults.timeoutSeconds` default 600s; enforced in `runEmbeddedPiAgent` abort timer.
-
-## Where things can end early
-
-- Agent timeout (abort)
-- AbortSignal (cancel)
-- Gateway disconnect or RPC timeout
-- `agent.wait` timeout (wait-only, does not stop agent)

package/docs/concepts/agent-workspace.md

@@ -1,234 +0,0 @@
----
-summary: "Agent workspace: location, layout, and backup strategy"
-read_when:
-  - You need to explain the agent workspace or its file layout
-  - You want to back up or migrate an agent workspace
-title: "Agent Workspace"
----
-
-# Agent workspace
-
-The workspace is the agent's home. It is the only working directory used for
-file tools and for workspace context. Keep it private and treat it as memory.
-
-This is separate from `~/.symi/`, which stores config, credentials, and
-sessions.
-
-**Important:** the workspace is the **default cwd**, not a hard sandbox. Tools
-resolve relative paths against the workspace, but absolute paths can still reach
-elsewhere on the host unless sandboxing is enabled. If you need isolation, use
-[`agents.defaults.sandbox`](/gateway/sandboxing) (and/or per‑agent sandbox config).
-When sandboxing is enabled and `workspaceAccess` is not `"rw"`, tools operate
-inside a sandbox workspace under `~/.symi/sandboxes`, not your host workspace.
-
-## Default location
-
-- Default: `~/.symi/workspace`
-- If `SYMI_PROFILE` is set and not `"default"`, the default becomes
-  `~/.symi/workspace-<profile>`.
-- Override in `~/.symi/symi.json`:
-
-```json5
-{
-  agent: {
-    workspace: "~/.symi/workspace",
-  },
-}
-```
-
-`symi onboard`, `symi configure`, or `symi setup` will create the
-workspace and seed the bootstrap files if they are missing.
-
-If you already manage the workspace files yourself, you can disable bootstrap
-file creation:
-
-```json5
-{ agent: { skipBootstrap: true } }
-```
-
-## Extra workspace folders
-
-Older installs may have created `~/symi`. Keeping multiple workspace
-directories around can cause confusing auth or state drift, because only one
-workspace is active at a time.
-
-**Recommendation:** keep a single active workspace. If you no longer use the
-extra folders, archive or move them to Trash (for example `trash ~/symi`).
-If you intentionally keep multiple workspaces, make sure
-`agents.defaults.workspace` points to the active one.
-
-`symi doctor` warns when it detects extra workspace directories.
-
-## Workspace file map (what each file means)
-
-These are the standard files Symi expects inside the workspace:
-
-- `AGENTS.md`
-  - Operating instructions for the agent and how it should use memory.
-  - Loaded at the start of every session.
-  - Good place for rules, priorities, and "how to behave" details.
-
-- `SYMICORE.md`
-  - Persona, tone, and boundaries.
-  - Loaded every session.
-
-- `USER.md`
-  - Who the user is and how to address them.
-  - Loaded every session.
-
-- `IDENTITY.md`
-  - The agent's name, vibe, and emoji.
-  - Created/updated during the bootstrap ritual.
-
-- `TOOLS.md`
-  - Notes about your local tools and conventions.
-  - Does not control tool availability; it is only guidance.
-
-- `SYMIPULSE.md`
-  - Optional tiny checklist for heartbeat runs.
-  - Keep it short to avoid token burn.
-
-- `BOOT.md`
-  - Optional startup checklist executed on gateway restart when internal hooks are enabled.
-  - Keep it short; use the message tool for outbound sends.
-
-- `BOOTSTRAP.md`
-  - One-time first-run ritual.
-  - Only created for a brand-new workspace.
-  - Delete it after the ritual is complete.
-
-- `memory/YYYY-MM-DD.md`
-  - Daily memory log (one file per day).
-  - Recommended to read today + yesterday on session start.
-
-- `MEMORY.md` (optional)
-  - Curated long-term memory.
-  - Only load in the main, private session (not shared/group contexts).
-
-See [Memory](/concepts/memory) for the workflow and automatic memory flush.
-
-- `skills/` (optional)
-  - Workspace-specific skills.
-  - Overrides managed/bundled skills when names collide.
-
-- `canvas/` (optional)
-  - Canvas UI files for node displays (for example `canvas/index.html`).
-
-If any bootstrap file is missing, Symi injects a "missing file" marker into
-the session and continues. Large bootstrap files are truncated when injected;
-adjust limits with `agents.defaults.bootstrapMaxChars` (default: 20000) and
-`agents.defaults.bootstrapTotalMaxChars` (default: 150000).
-`symi setup` can recreate missing defaults without overwriting existing
-files.
-
-## What is NOT in the workspace
-
-These live under `~/.symi/` and should NOT be committed to the workspace repo:
-
-- `~/.symi/symi.json` (config)
-- `~/.symi/credentials/` (OAuth tokens, API keys)
-- `~/.symi/agents/<agentId>/sessions/` (session transcripts + metadata)
-- `~/.symi/skills/` (managed skills)
-
-If you need to migrate sessions or config, copy them separately and keep them
-out of version control.
-
-## Git backup (recommended, private)
-
-Treat the workspace as private memory. Put it in a **private** git repo so it is
-backed up and recoverable.
-
-Run these steps on the machine where the Gateway runs (that is where the
-workspace lives).
-
-### 1) Initialize the repo
-
-If git is installed, brand-new workspaces are initialized automatically. If this
-workspace is not already a repo, run:
-
-```bash
-cd ~/.symi/workspace
-git init
-git add AGENTS.md SYMICORE.md TOOLS.md IDENTITY.md USER.md SYMIPULSE.md memory/
-git commit -m "Add agent workspace"
-```
-
-### 2) Add a private remote (beginner-friendly options)
-
-Option A: GitHub web UI
-
-1. Create a new **private** repository on GitHub.
-2. Do not initialize with a README (avoids merge conflicts).
-3. Copy the HTTPS remote URL.
-4. Add the remote and push:
-
-```bash
-git branch -M main
-git remote add origin <https-url>
-git push -u origin main
-```
-
-Option B: GitHub CLI (`gh`)
-
-```bash
-gh auth login
-gh repo create symi-workspace --private --source . --remote origin --push
-```
-
-Option C: GitLab web UI
-
-1. Create a new **private** repository on GitLab.
-2. Do not initialize with a README (avoids merge conflicts).
-3. Copy the HTTPS remote URL.
-4. Add the remote and push:
-
-```bash
-git branch -M main
-git remote add origin <https-url>
-git push -u origin main
-```
-
-### 3) Ongoing updates
-
-```bash
-git status
-git add .
-git commit -m "Update memory"
-git push
-```
-
-## Do not commit secrets
-
-Even in a private repo, avoid storing secrets in the workspace:
-
-- API keys, OAuth tokens, passwords, or private credentials.
-- Anything under `~/.symi/`.
-- Raw dumps of chats or sensitive attachments.
-
-If you must store sensitive references, use placeholders and keep the real
-secret elsewhere (password manager, environment variables, or `~/.symi/`).
-
-Suggested `.gitignore` starter:
-
-```gitignore
-.DS_Store
-.env
-**/*.key
-**/*.pem
-**/secrets*
-```
-
-## Moving the workspace to a new machine
-
-1. Clone the repo to the desired path (default `~/.symi/workspace`).
-2. Set `agents.defaults.workspace` to that path in `~/.symi/symi.json`.
-3. Run `symi setup --workspace <path>` to seed any missing files.
-4. If you need sessions, copy `~/.symi/agents/<agentId>/sessions/` from the
-   old machine separately.
-
-## Advanced notes
-
-- Multi-agent routing can use different workspaces per agent. See
-  [Channel routing](/channels/channel-routing) for routing configuration.
-- If `agents.defaults.sandbox` is enabled, non-main sessions can use per-session sandbox
-  workspaces under `agents.defaults.sandbox.workspaceRoot`.

package/docs/concepts/agent.md

@@ -1,123 +0,0 @@
----
-summary: "Agent runtime (embedded pi-mono), workspace contract, and session bootstrap"
-read_when:
-  - Changing agent runtime, workspace bootstrap, or session behavior
-title: "Agent Runtime"
----
-
-# Agent Runtime 🤖
-
-Symi runs a single embedded agent runtime derived from **pi-mono**.
-
-## Workspace (required)
-
-Symi uses a single agent workspace directory (`agents.defaults.workspace`) as the agent’s **only** working directory (`cwd`) for tools and context.
-
-Recommended: use `symi setup` to create `~/.symi/symi.json` if missing and initialize the workspace files.
-
-Full workspace layout + backup guide: [Agent workspace](/concepts/agent-workspace)
-
-If `agents.defaults.sandbox` is enabled, non-main sessions can override this with
-per-session workspaces under `agents.defaults.sandbox.workspaceRoot` (see
-[Gateway configuration](/gateway/configuration)).
-
-## Bootstrap files (injected)
-
-Inside `agents.defaults.workspace`, Symi expects these user-editable files:
-
-- `AGENTS.md` — operating instructions + “memory”
-- `SYMICORE.md` — persona, boundaries, tone
-- `TOOLS.md` — user-maintained tool notes (e.g. `imsg`, `sag`, conventions)
-- `BOOTSTRAP.md` — one-time first-run ritual (deleted after completion)
-- `IDENTITY.md` — agent name/vibe/emoji
-- `USER.md` — user profile + preferred address
-
-On the first turn of a new session, Symi injects the contents of these files directly into the agent context.
-
-Blank files are skipped. Large files are trimmed and truncated with a marker so prompts stay lean (read the file for full content).
-
-If a file is missing, Symi injects a single “missing file” marker line (and `symi setup` will create a safe default template).
-
-`BOOTSTRAP.md` is only created for a **brand new workspace** (no other bootstrap files present). If you delete it after completing the ritual, it should not be recreated on later restarts.
-
-To disable bootstrap file creation entirely (for pre-seeded workspaces), set:
-
-```json5
-{ agent: { skipBootstrap: true } }
-```
-
-## Built-in tools
-
-Core tools (read/exec/edit/write and related system tools) are always available,
-subject to tool policy. `apply_patch` is optional and gated by
-`tools.exec.applyPatch`. `TOOLS.md` does **not** control which tools exist; it’s
-guidance for how _you_ want them used.
-
-## Skills
-
-Symi loads skills from three locations (workspace wins on name conflict):
-
-- Bundled (shipped with the install)
-- Managed/local: `~/.symi/skills`
-- Workspace: `<workspace>/skills`
-
-Skills can be gated by config/env (see `skills` in [Gateway configuration](/gateway/configuration)).
-
-## pi-mono integration
-
-Symi reuses pieces of the pi-mono codebase (models/tools), but **session management, discovery, and tool wiring are Symi-owned**.
-
-- No pi-coding agent runtime.
-- No `~/.pi/agent` or `<workspace>/.pi` settings are consulted.
-
-## Sessions
-
-Session transcripts are stored as JSONL at:
-
-- `~/.symi/agents/<agentId>/sessions/<SessionId>.jsonl`
-
-The session ID is stable and chosen by Symi.
-Legacy Pi/Tau session folders are **not** read.
-
-## Steering while streaming
-
-When queue mode is `steer`, inbound messages are injected into the current run.
-The queue is checked **after each tool call**; if a queued message is present,
-remaining tool calls from the current assistant message are skipped (error tool
-results with "Skipped due to queued user message."), then the queued user
-message is injected before the next assistant response.
-
-When queue mode is `followup` or `collect`, inbound messages are held until the
-current turn ends, then a new agent turn starts with the queued payloads. See
-[Queue](/concepts/queue) for mode + debounce/cap behavior.
-
-Block streaming sends completed assistant blocks as soon as they finish; it is
-**off by default** (`agents.defaults.blockStreamingDefault: "off"`).
-Tune the boundary via `agents.defaults.blockStreamingBreak` (`text_end` vs `message_end`; defaults to text_end).
-Control soft block chunking with `agents.defaults.blockStreamingChunk` (defaults to
-800–1200 chars; prefers paragraph breaks, then newlines; sentences last).
-Coalesce streamed chunks with `agents.defaults.blockStreamingCoalesce` to reduce
-single-line spam (idle-based merging before send). Non-Telegram channels require
-explicit `*.blockStreaming: true` to enable block replies.
-Verbose tool summaries are emitted at tool start (no debounce); Control UI
-streams tool output via agent events when available.
-More details: [Streaming + chunking](/concepts/streaming).
-
-## Model refs
-
-Model refs in config (for example `agents.defaults.model` and `agents.defaults.models`) are parsed by splitting on the **first** `/`.
-
-- Use `provider/model` when configuring models.
-- If the model ID itself contains `/` (OpenRouter-style), include the provider prefix (example: `openrouter/moonshotai/kimi-k2`).
-- If you omit the provider, Symi treats the input as an alias or a model for the **default provider** (only works when there is no `/` in the model ID).
-
-## Configuration (minimal)
-
-At minimum, set:
-
-- `agents.defaults.workspace`
-- `channels.whatsapp.allowFrom` (strongly recommended)
-
----
-
-_Next: [Group Chats](/channels/group-messages)_ 🦞

package/docs/concepts/architecture.md

@@ -1,135 +0,0 @@
----
-summary: "WebSocket gateway architecture, components, and client flows"
-read_when:
-  - Working on gateway protocol, clients, or transports
-title: "Gateway Architecture"
----
-
-# Gateway architecture
-
-Last updated: 2026-01-22
-
-## Overview
-
-- A single long‑lived **Gateway** owns all messaging surfaces (WhatsApp via
-- Control-plane clients (macOS app, CLI, web UI, automations) connect to the
-  Gateway over **WebSocket** on the configured bind host (default
-  `127.0.0.1:18789`).
-- **Nodes** (macOS/iOS/Android/headless) also connect over **WebSocket**, but
-  declare `role: node` with explicit caps/commands.
-- One Gateway per host; it is the only place that opens a WhatsApp session.
-- The **canvas host** is served by the Gateway HTTP server under:
-  - `/__symi__/canvas/` (agent-editable HTML/CSS/JS)
-  - `/__symi__/a2ui/` (A2UI host)
-    It uses the same port as the Gateway (default `18789`).
-
-## Components and flows
-
-### Gateway (daemon)
-
-- Maintains provider connections.
-- Exposes a typed WS API (requests, responses, server‑push events).
-- Validates inbound frames against JSON Schema.
-- Emits events like `agent`, `chat`, `presence`, `health`, `heartbeat`, `cron`.
-
-### Clients (mac app / CLI / web admin)
-
-- One WS connection per client.
-- Send requests (`health`, `status`, `send`, `agent`, `system-presence`).
-- Subscribe to events (`tick`, `agent`, `presence`, `shutdown`).
-
-### Nodes (macOS / iOS / Android / headless)
-
-- Connect to the **same WS server** with `role: node`.
-- Provide a device identity in `connect`; pairing is **device‑based** (role `node`) and
-  approval lives in the device pairing store.
-- Expose commands like `canvas.*`, `camera.*`, `screen.record`, `location.get`.
-
-Protocol details:
-
-- [Gateway protocol](/gateway/protocol)
-
-### WebChat
-
-- Static UI that uses the Gateway WS API for chat history and sends.
-- In remote setups, connects through the same SSH/Tailscale tunnel as other
-  clients.
-
-## Connection lifecycle (single client)
-
-```mermaid
-sequenceDiagram
-    participant Client
-    participant Gateway
-
-    Client->>Gateway: req:connect
-    Gateway-->>Client: res (ok)
-    Note right of Gateway: or res error + close
-    Note left of Client: payload=hello-ok<br>snapshot: presence + health
-
-    Gateway-->>Client: event:presence
-    Gateway-->>Client: event:tick
-
-    Client->>Gateway: req:agent
-    Gateway-->>Client: res:agent<br>ack {runId, status:"accepted"}
-    Gateway-->>Client: event:agent<br>(streaming)
-    Gateway-->>Client: res:agent<br>final {runId, status, summary}
-```
-
-## Wire protocol (summary)
-
-- Transport: WebSocket, text frames with JSON payloads.
-- First frame **must** be `connect`.
-- After handshake:
-  - Requests: `{type:"req", id, method, params}` → `{type:"res", id, ok, payload|error}`
-  - Events: `{type:"event", event, payload, seq?, stateVersion?}`
-- If `SYMI_GATEWAY_TOKEN` (or `--token`) is set, `connect.params.auth.token`
-  must match or the socket closes.
-- Idempotency keys are required for side‑effecting methods (`send`, `agent`) to
-  safely retry; the server keeps a short‑lived dedupe cache.
-- Nodes must include `role: "node"` plus caps/commands/permissions in `connect`.
-
-## Pairing + local trust
-
-- All WS clients (operators + nodes) include a **device identity** on `connect`.
-- New device IDs require pairing approval; the Gateway issues a **device token**
-  for subsequent connects.
-- **Local** connects (loopback or the gateway host’s own tailnet address) can be
-  auto‑approved to keep same‑host UX smooth.
-- **Non‑local** connects must sign the `connect.challenge` nonce and require
-  explicit approval.
-- Gateway auth (`gateway.auth.*`) still applies to **all** connections, local or
-  remote.
-
-Details: [Gateway protocol](/gateway/protocol), [Pairing](/channels/pairing),
-[Security](/gateway/security).
-
-## Protocol typing and codegen
-
-- TypeBox schemas define the protocol.
-- JSON Schema is generated from those schemas.
-- Swift models are generated from the JSON Schema.
-
-## Remote access
-
-- Preferred: Tailscale or VPN.
-- Alternative: SSH tunnel
-
-  ```bash
-  ssh -N -L 18789:127.0.0.1:18789 user@host
-  ```
-
-- The same handshake + auth token apply over the tunnel.
-- TLS + optional pinning can be enabled for WS in remote setups.
-
-## Operations snapshot
-
-- Start: `symi gateway` (foreground, logs to stdout).
-- Health: `health` over WS (also included in `hello-ok`).
-- Supervision: launchd/systemd for auto‑restart.
-
-## Invariants
-
-- Exactly one Gateway controls a single Baileys session per host.
-- Handshake is mandatory; any non‑JSON or non‑connect first frame is a hard close.
-- Events are not replayed; clients must refresh on gaps.