@symerian/symi 3.0.21 → 3.0.22

package/docs/concepts/compaction.md

@@ -1,61 +0,0 @@
----
-summary: "Context window + compaction: how Symi keeps sessions under model limits"
-read_when:
-  - You want to understand auto-compaction and /compact
-  - You are debugging long sessions hitting context limits
-title: "Compaction"
----
-
-# Context Window & Compaction
-
-Every model has a **context window** (max tokens it can see). Long-running chats accumulate messages and tool results; once the window is tight, Symi **compacts** older history to stay within limits.
-
-## What compaction is
-
-Compaction **summarizes older conversation** into a compact summary entry and keeps recent messages intact. The summary is stored in the session history, so future requests use:
-
-- The compaction summary
-- Recent messages after the compaction point
-
-Compaction **persists** in the session’s JSONL history.
-
-## Configuration
-
-Use the `agents.defaults.compaction` setting in your `symi.json` to configure compaction behavior (mode, target tokens, etc.).
-
-## Auto-compaction (default on)
-
-When a session nears or exceeds the model’s context window, Symi triggers auto-compaction and may retry the original request using the compacted context.
-
-You’ll see:
-
-- `🧹 Auto-compaction complete` in verbose mode
-- `/status` showing `🧹 Compactions: <count>`
-
-Before compaction, Symi can run a **silent memory flush** turn to store
-durable notes to disk. See [Memory](/concepts/memory) for details and config.
-
-## Manual compaction
-
-Use `/compact` (optionally with instructions) to force a compaction pass:
-
-```
-/compact Focus on decisions and open questions
-```
-
-## Context window source
-
-Context window is model-specific. Symi uses the model definition from the configured provider catalog to determine limits.
-
-## Compaction vs pruning
-
-- **Compaction**: summarises and **persists** in JSONL.
-- **Session pruning**: trims old **tool results** only, **in-memory**, per request.
-
-See [/concepts/session-pruning](/concepts/session-pruning) for pruning details.
-
-## Tips
-
-- Use `/compact` when sessions feel stale or context is bloated.
-- Large tool outputs are already truncated; pruning can further reduce tool-result buildup.
-- If you need a fresh slate, `/new` or `/reset` starts a new session id.

package/docs/concepts/context.md

@@ -1,161 +0,0 @@
----
-summary: "Context: what the model sees, how it is built, and how to inspect it"
-read_when:
-  - You want to understand what “context” means in Symi
-  - You are debugging why the model “knows” something (or forgot it)
-  - You want to reduce context overhead (/context, /status, /compact)
-title: "Context"
----
-
-# Context
-
-“Context” is **everything Symi sends to the model for a run**. It is bounded by the model’s **context window** (token limit).
-
-Beginner mental model:
-
-- **System prompt** (Symi-built): rules, tools, skills list, time/runtime, and injected workspace files.
-- **Conversation history**: your messages + the assistant’s messages for this session.
-- **Tool calls/results + attachments**: command output, file reads, images/audio, etc.
-
-Context is _not the same thing_ as “memory”: memory can be stored on disk and reloaded later; context is what’s inside the model’s current window.
-
-## Quick start (inspect context)
-
-- `/status` → quick “how full is my window?” view + session settings.
-- `/context list` → what’s injected + rough sizes (per file + totals).
-- `/context detail` → deeper breakdown: per-file, per-tool schema sizes, per-skill entry sizes, and system prompt size.
-- `/usage tokens` → append per-reply usage footer to normal replies.
-- `/compact` → summarize older history into a compact entry to free window space.
-
-See also: [Slash commands](/tools/slash-commands), [Token use & costs](/reference/token-use), [Compaction](/concepts/compaction).
-
-## Example output
-
-Values vary by model, provider, tool policy, and what’s in your workspace.
-
-### `/context list`
-
-```
-🧠 Context breakdown
-Workspace: <workspaceDir>
-Bootstrap max/file: 20,000 chars
-Sandbox: mode=non-main sandboxed=false
-System prompt (run): 38,412 chars (~9,603 tok) (Project Context 23,901 chars (~5,976 tok))
-
-Injected workspace files:
-- AGENTS.md: OK | raw 1,742 chars (~436 tok) | injected 1,742 chars (~436 tok)
-- SYMICORE.md: OK | raw 912 chars (~228 tok) | injected 912 chars (~228 tok)
-- TOOLS.md: TRUNCATED | raw 54,210 chars (~13,553 tok) | injected 20,962 chars (~5,241 tok)
-- IDENTITY.md: OK | raw 211 chars (~53 tok) | injected 211 chars (~53 tok)
-- USER.md: OK | raw 388 chars (~97 tok) | injected 388 chars (~97 tok)
-- SYMIPULSE.md: MISSING | raw 0 | injected 0
-- BOOTSTRAP.md: OK | raw 0 chars (~0 tok) | injected 0 chars (~0 tok)
-
-Skills list (system prompt text): 2,184 chars (~546 tok) (12 skills)
-Tools: read, edit, write, exec, process, browser, message, sessions_send, …
-Tool list (system prompt text): 1,032 chars (~258 tok)
-Tool schemas (JSON): 31,988 chars (~7,997 tok) (counts toward context; not shown as text)
-Tools: (same as above)
-
-Session tokens (cached): 14,250 total / ctx=32,000
-```
-
-### `/context detail`
-
-```
-🧠 Context breakdown (detailed)
-…
-Top skills (prompt entry size):
-- frontend-design: 412 chars (~103 tok)
-- oracle: 401 chars (~101 tok)
-… (+10 more skills)
-
-Top tools (schema size):
-- browser: 9,812 chars (~2,453 tok)
-- exec: 6,240 chars (~1,560 tok)
-… (+N more tools)
-```
-
-## What counts toward the context window
-
-Everything the model receives counts, including:
-
-- System prompt (all sections).
-- Conversation history.
-- Tool calls + tool results.
-- Attachments/transcripts (images/audio/files).
-- Compaction summaries and pruning artifacts.
-- Provider “wrappers” or hidden headers (not visible, still counted).
-
-## How Symi builds the system prompt
-
-The system prompt is **Symi-owned** and rebuilt each run. It includes:
-
-- Tool list + short descriptions.
-- Skills list (metadata only; see below).
-- Workspace location.
-- Time (UTC + converted user time if configured).
-- Runtime metadata (host/OS/model/thinking).
-- Injected workspace bootstrap files under **Project Context**.
-
-Full breakdown: [System Prompt](/concepts/system-prompt).
-
-## Injected workspace files (Project Context)
-
-By default, Symi injects a fixed set of workspace files (if present):
-
-- `AGENTS.md`
-- `SYMICORE.md`
-- `TOOLS.md`
-- `IDENTITY.md`
-- `USER.md`
-- `SYMIPULSE.md`
-- `BOOTSTRAP.md` (first-run only)
-
-Large files are truncated per-file using `agents.defaults.bootstrapMaxChars` (default `20000` chars). Symi also enforces a total bootstrap injection cap across files with `agents.defaults.bootstrapTotalMaxChars` (default `150000` chars). `/context` shows **raw vs injected** sizes and whether truncation happened.
-
-## Skills: what’s injected vs loaded on-demand
-
-The system prompt includes a compact **skills list** (name + description + location). This list has real overhead.
-
-Skill instructions are _not_ included by default. The model is expected to `read` the skill’s `SKILL.md` **only when needed**.
-
-## Tools: there are two costs
-
-Tools affect context in two ways:
-
-1. **Tool list text** in the system prompt (what you see as “Tooling”).
-2. **Tool schemas** (JSON). These are sent to the model so it can call tools. They count toward context even though you don’t see them as plain text.
-
-`/context detail` breaks down the biggest tool schemas so you can see what dominates.
-
-## Commands, directives, and “inline shortcuts”
-
-Slash commands are handled by the Gateway. There are a few different behaviors:
-
-- **Standalone commands**: a message that is only `/...` runs as a command.
-- **Directives**: `/think`, `/verbose`, `/reasoning`, `/elevated`, `/model`, `/queue` are stripped before the model sees the message.
-  - Directive-only messages persist session settings.
-  - Inline directives in a normal message act as per-message hints.
-- **Inline shortcuts** (allowlisted senders only): certain `/...` tokens inside a normal message can run immediately (example: “hey /status”), and are stripped before the model sees the remaining text.
-
-Details: [Slash commands](/tools/slash-commands).
-
-## Sessions, compaction, and pruning (what persists)
-
-What persists across messages depends on the mechanism:
-
-- **Normal history** persists in the session transcript until compacted/pruned by policy.
-- **Compaction** persists a summary into the transcript and keeps recent messages intact.
-- **Pruning** removes old tool results from the _in-memory_ prompt for a run, but does not rewrite the transcript.
-
-Docs: [Session](/concepts/session), [Compaction](/concepts/compaction), [Session pruning](/concepts/session-pruning).
-
-## What `/context` actually reports
-
-`/context` prefers the latest **run-built** system prompt report when available:
-
-- `System prompt (run)` = captured from the last embedded (tool-capable) run and persisted in the session store.
-- `System prompt (estimate)` = computed on the fly when no run report exists (or when running via a CLI backend that doesn’t generate the report).
-
-Either way, it reports sizes and top contributors; it does **not** dump the full system prompt or tool schemas.

package/docs/concepts/features.md

@@ -1,51 +0,0 @@
----
-summary: "Symi capabilities across channels, routing, media, and UX."
-read_when:
-  - You want a full list of what Symi supports
-title: "Features"
----
-
-## Highlights
-
-<Columns>
-  <Card title="Channels" icon="message-square">
-  </Card>
-  <Card title="Plugins" icon="plug">
-    Add Mattermost and more with extensions.
-  </Card>
-  <Card title="Routing" icon="route">
-    Multi-agent routing with isolated sessions.
-  </Card>
-  <Card title="Media" icon="image">
-    Images, audio, and documents in and out.
-  </Card>
-  <Card title="Apps and UI" icon="monitor">
-    Web Control UI and macOS companion app.
-  </Card>
-  <Card title="Mobile nodes" icon="smartphone">
-    iOS and Android nodes with Canvas support.
-  </Card>
-</Columns>
-
-## Full list
-
-- WhatsApp integration via WhatsApp Web (Baileys)
-- Telegram bot support (grammY)
-- Mattermost bot support (plugin)
-- iMessage integration via local imsg CLI (macOS)
-- Agent bridge for Pi in RPC mode with tool streaming
-- Streaming and chunking for long responses
-- Multi-agent routing for isolated sessions per workspace or sender
-- Subscription auth for Anthropic and OpenAI via OAuth
-- Sessions: direct chats collapse into shared `main`; groups are isolated
-- Group chat support with mention based activation
-- Media support for images, audio, and documents
-- Optional voice note transcription hook
-- WebChat and macOS menu bar app
-- iOS node with pairing and Canvas surface
-- Android node with pairing, Canvas, chat, and camera
-
-<Note>
-Legacy Claude, Codex, Gemini, and Opencode paths have been removed. Pi is the only
-coding agent path.
-</Note>

package/docs/concepts/markdown-formatting.md

@@ -1,128 +0,0 @@
----
-summary: "Markdown formatting pipeline for outbound channels"
-read_when:
-  - You are changing markdown formatting or chunking for outbound channels
-  - You are adding a new channel formatter or style mapping
-  - You are debugging formatting regressions across channels
-title: "Markdown Formatting"
----
-
-# Markdown formatting
-
-Symi formats outbound Markdown by converting it into a shared intermediate
-representation (IR) before rendering channel-specific output. The IR keeps the
-source text intact while carrying style/link spans so chunking and rendering can
-stay consistent across channels.
-
-## Goals
-
-- **Consistency:** one parse step, multiple renderers.
-- **Safe chunking:** split text before rendering so inline formatting never
-  breaks across chunks.
-- **Channel fit:** map the same IR to Slack mrkdwn, Telegram HTML, and Signal
-  style ranges without re-parsing Markdown.
-
-## Pipeline
-
-1. **Parse Markdown -> IR**
-   - IR is plain text plus style spans (bold/italic/strike/code/spoiler) and link spans.
-   - Offsets are UTF-16 code units so Signal style ranges align with its API.
-   - Tables are parsed only when a channel opts into table conversion.
-2. **Chunk IR (format-first)**
-   - Chunking happens on the IR text before rendering.
-   - Inline formatting does not split across chunks; spans are sliced per chunk.
-3. **Render per channel**
-   - **Slack:** mrkdwn tokens (bold/italic/strike/code), links as `<url|label>`.
-   - **Telegram:** HTML tags (`<b>`, `<i>`, `<s>`, `<code>`, `<pre><code>`, `<a href>`).
-   - **Signal:** plain text + `text-style` ranges; links become `label (url)` when label differs.
-
-## IR example
-
-Input Markdown:
-
-```markdown
-Hello **world** — see [docs](https://docs.symi.ai).
-```
-
-IR (schematic):
-
-```json
-{
-  "text": "Hello world — see docs.",
-  "styles": [{ "start": 6, "end": 11, "style": "bold" }],
-  "links": [{ "start": 19, "end": 23, "href": "https://docs.symi.ai" }]
-}
-```
-
-## Where it is used
-
-- Slack, Telegram, and Signal outbound adapters render from the IR.
-  their own formatting rules, with Markdown table conversion applied before
-  chunking when enabled.
-
-## Table handling
-
-Markdown tables are not consistently supported across chat clients. Use
-`markdown.tables` to control conversion per channel (and per account).
-
-- `code`: render tables as code blocks (default for most channels).
-- `bullets`: convert each row into bullet points (default for Signal + WhatsApp).
-- `off`: disable table parsing and conversion; raw table text passes through.
-
-Config keys:
-
-```yaml
-channels:
-  markdown:
-    tables: code
-  accounts:
-    work:
-      markdown:
-        tables: off
-```
-
-## Chunking rules
-
-- Chunk limits come from channel adapters/config and are applied to the IR text.
-- Code fences are preserved as a single block with a trailing newline so channels
-  render them correctly.
-- List prefixes and blockquote prefixes are part of the IR text, so chunking
-  does not split mid-prefix.
-- Inline styles (bold/italic/strike/inline-code/spoiler) are never split across
-  chunks; the renderer reopens styles inside each chunk.
-
-If you need more on chunking behavior across channels, see
-[Streaming + chunking](/concepts/streaming).
-
-## Link policy
-
-- **Slack:** `[label](url)` -> `<url|label>`; bare URLs remain bare. Autolink
-  is disabled during parse to avoid double-linking.
-- **Telegram:** `[label](url)` -> `<a href="url">label</a>` (HTML parse mode).
-- **Signal:** `[label](url)` -> `label (url)` unless label matches the URL.
-
-## Spoilers
-
-Spoiler markers (`||spoiler||`) are parsed only for Signal, where they map to
-SPOILER style ranges. Other channels treat them as plain text.
-
-## How to add or update a channel formatter
-
-1. **Parse once:** use the shared `markdownToIR(...)` helper with channel-appropriate
-   options (autolink, heading style, blockquote prefix).
-2. **Render:** implement a renderer with `renderMarkdownWithMarkers(...)` and a
-   style marker map (or Signal style ranges).
-3. **Chunk:** call `chunkMarkdownIR(...)` before rendering; render each chunk.
-4. **Wire adapter:** update the channel outbound adapter to use the new chunker
-   and renderer.
-5. **Test:** add or update format tests and an outbound delivery test if the
-   channel uses chunking.
-
-## Common gotchas
-
-- Slack angle-bracket tokens (`<@U123>`, `<#C123>`, `<https://...>`) must be
-  preserved; escape raw HTML safely.
-- Telegram HTML requires escaping text outside tags to avoid broken markup.
-- Signal style ranges depend on UTF-16 offsets; do not use code point offsets.
-- Preserve trailing newlines for fenced code blocks so closing markers land on
-  their own line.