durar-ai 2026.4.4

package/docs/concepts/agent-workspace.md

@@ -0,0 +1,246 @@
+---
+summary: "Agent workspace: location, layout, and backup strategy"
+read_when:
+  - You need to explain the agent workspace or its file layout
+  - You want to back up or migrate an agent workspace
+title: "Agent Workspace"
+---
+
+# Agent workspace
+
+The workspace is the agent's home. It is the only working directory used for
+file tools and for workspace context. Keep it private and treat it as memory.
+
+This is separate from `~/.Durar/`, which stores config, credentials, and
+sessions.
+
+**Important:** the workspace is the **default cwd**, not a hard sandbox. Tools
+resolve relative paths against the workspace, but absolute paths can still reach
+elsewhere on the host unless sandboxing is enabled. If you need isolation, use
+[`agents.defaults.sandbox`](/gateway/sandboxing) (and/or per‑agent sandbox config).
+When sandboxing is enabled and `workspaceAccess` is not `"rw"`, tools operate
+inside a sandbox workspace under `~/.Durar/sandboxes`, not your host workspace.
+
+## Default location
+
+- Default: `~/.Durar/workspace`
+- If `Durar_PROFILE` is set and not `"default"`, the default becomes
+  `~/.Durar/workspace-<profile>`.
+- Override in `~/.Durar/Durar.json`:
+
+```json5
+{
+  agent: {
+    workspace: "~/.Durar/workspace",
+  },
+}
+```
+
+`Durar onboard`, `Durar configure`, or `Durar setup` will create the
+workspace and seed the bootstrap files if they are missing.
+Sandbox seed copies only accept regular in-workspace files; symlink/hardlink
+aliases that resolve outside the source workspace are ignored.
+
+If you already manage the workspace files yourself, you can disable bootstrap
+file creation:
+
+```json5
+{ agent: { skipBootstrap: true } }
+```
+
+## Extra workspace folders
+
+Older installs may have created `~/Durar`. Keeping multiple workspace
+directories around can cause confusing auth or state drift, because only one
+workspace is active at a time.
+
+**Recommendation:** keep a single active workspace. If you no longer use the
+extra folders, archive or move them to Trash (for example `trash ~/Durar`).
+If you intentionally keep multiple workspaces, make sure
+`agents.defaults.workspace` points to the active one.
+
+`Durar doctor` warns when it detects extra workspace directories.
+
+## Workspace file map (what each file means)
+
+These are the standard files Durar expects inside the workspace:
+
+- `AGENTS.md`
+  - Operating instructions for the agent and how it should use memory.
+  - Loaded at the start of every session.
+  - Good place for rules, priorities, and "how to behave" details.
+
+- `SOUL.md`
+  - Persona, tone, and boundaries.
+  - Loaded every session.
+  - Guide: [SOUL.md Personality Guide](/concepts/soul)
+
+- `USER.md`
+  - Who the user is and how to address them.
+  - Loaded every session.
+
+- `IDENTITY.md`
+  - The agent's name, vibe, and emoji.
+  - Created/updated during the bootstrap ritual.
+
+- `TOOLS.md`
+  - Notes about your local tools and conventions.
+  - Does not control tool availability; it is only guidance.
+
+- `HEARTBEAT.md`
+  - Optional tiny checklist for heartbeat runs.
+  - Keep it short to avoid token burn.
+
+- `BOOT.md`
+  - Optional startup checklist executed on gateway restart when internal hooks are enabled.
+  - Keep it short; use the message tool for outbound sends.
+
+- `BOOTSTRAP.md`
+  - One-time first-run ritual.
+  - Only created for a brand-new workspace.
+  - Delete it after the ritual is complete.
+
+- `memory/YYYY-MM-DD.md`
+  - Daily memory log (one file per day).
+  - Recommended to read today + yesterday on session start.
+
+- `MEMORY.md` (optional)
+  - Curated long-term memory.
+  - Only load in the main, private session (not shared/group contexts).
+
+See [Memory](/concepts/memory) for the workflow and automatic memory flush.
+
+- `skills/` (optional)
+  - Workspace-specific skills.
+  - Highest-precedence skill location for that workspace.
+  - Overrides project agent skills, personal agent skills, managed skills, bundled skills, and `skills.load.extraDirs` when names collide.
+
+- `canvas/` (optional)
+  - Canvas UI files for node displays (for example `canvas/index.html`).
+
+If any bootstrap file is missing, Durar injects a "missing file" marker into
+the session and continues. Large bootstrap files are truncated when injected;
+adjust limits with `agents.defaults.bootstrapMaxChars` (default: 20000) and
+`agents.defaults.bootstrapTotalMaxChars` (default: 150000).
+`Durar setup` can recreate missing defaults without overwriting existing
+files.
+
+## What is NOT in the workspace
+
+These live under `~/.Durar/` and should NOT be committed to the workspace repo:
+
+- `~/.Durar/Durar.json` (config)
+- `~/.Durar/agents/<agentId>/agent/auth-profiles.json` (model auth profiles: OAuth + API keys)
+- `~/.Durar/credentials/` (channel/provider state plus legacy OAuth import data)
+- `~/.Durar/agents/<agentId>/sessions/` (session transcripts + metadata)
+- `~/.Durar/skills/` (managed skills)
+
+If you need to migrate sessions or config, copy them separately and keep them
+out of version control.
+
+## Git backup (recommended, private)
+
+Treat the workspace as private memory. Put it in a **private** git repo so it is
+backed up and recoverable.
+
+Run these steps on the machine where the Gateway runs (that is where the
+workspace lives).
+
+### 1) Initialize the repo
+
+If git is installed, brand-new workspaces are initialized automatically. If this
+workspace is not already a repo, run:
+
+```bash
+cd ~/.Durar/workspace
+git init
+git add AGENTS.md SOUL.md TOOLS.md IDENTITY.md USER.md HEARTBEAT.md memory/
+git commit -m "Add agent workspace"
+```
+
+### 2) Add a private remote (beginner-friendly options)
+
+Option A: GitHub web UI
+
+1. Create a new **private** repository on GitHub.
+2. Do not initialize with a README (avoids merge conflicts).
+3. Copy the HTTPS remote URL.
+4. Add the remote and push:
+
+```bash
+git branch -M main
+git remote add origin <https-url>
+git push -u origin main
+```
+
+Option B: GitHub CLI (`gh`)
+
+```bash
+gh auth login
+gh repo create Durar-workspace --private --source . --remote origin --push
+```
+
+Option C: GitLab web UI
+
+1. Create a new **private** repository on GitLab.
+2. Do not initialize with a README (avoids merge conflicts).
+3. Copy the HTTPS remote URL.
+4. Add the remote and push:
+
+```bash
+git branch -M main
+git remote add origin <https-url>
+git push -u origin main
+```
+
+### 3) Ongoing updates
+
+```bash
+git status
+git add .
+git commit -m "Update memory"
+git push
+```
+
+## Do not commit secrets
+
+Even in a private repo, avoid storing secrets in the workspace:
+
+- API keys, OAuth tokens, passwords, or private credentials.
+- Anything under `~/.Durar/`.
+- Raw dumps of chats or sensitive attachments.
+
+If you must store sensitive references, use placeholders and keep the real
+secret elsewhere (password manager, environment variables, or `~/.Durar/`).
+
+Suggested `.gitignore` starter:
+
+```gitignore
+.DS_Store
+.env
+**/*.key
+**/*.pem
+**/secrets*
+```
+
+## Moving the workspace to a new machine
+
+1. Clone the repo to the desired path (default `~/.Durar/workspace`).
+2. Set `agents.defaults.workspace` to that path in `~/.Durar/Durar.json`.
+3. Run `Durar setup --workspace <path>` to seed any missing files.
+4. If you need sessions, copy `~/.Durar/agents/<agentId>/sessions/` from the
+   old machine separately.
+
+## Advanced notes
+
+- Multi-agent routing can use different workspaces per agent. See
+  [Channel routing](/channels/channel-routing) for routing configuration.
+- If `agents.defaults.sandbox` is enabled, non-main sessions can use per-session sandbox
+  workspaces under `agents.defaults.sandbox.workspaceRoot`.
+
+## Related
+
+- [Standing Orders](/automation/standing-orders) — persistent instructions in workspace files
+- [Heartbeat](/gateway/heartbeat) — HEARTBEAT.md workspace file
+- [Session](/concepts/session) — session storage paths
+- [Sandboxing](/gateway/sandboxing) — workspace access in sandboxed environments

package/docs/concepts/agent.md

@@ -0,0 +1,129 @@
+---
+summary: "Agent runtime, workspace contract, and session bootstrap"
+read_when:
+  - Changing agent runtime, workspace bootstrap, or session behavior
+title: "Agent Runtime"
+---
+
+# Agent Runtime
+
+Durar runs a single embedded agent runtime.
+
+## Workspace (required)
+
+Durar uses a single agent workspace directory (`agents.defaults.workspace`) as the agent’s **only** working directory (`cwd`) for tools and context.
+
+Recommended: use `Durar setup` to create `~/.Durar/Durar.json` if missing and initialize the workspace files.
+
+Full workspace layout + backup guide: [Agent workspace](/concepts/agent-workspace)
+
+If `agents.defaults.sandbox` is enabled, non-main sessions can override this with
+per-session workspaces under `agents.defaults.sandbox.workspaceRoot` (see
+[Gateway configuration](/gateway/configuration)).
+
+## Bootstrap files (injected)
+
+Inside `agents.defaults.workspace`, Durar expects these user-editable files:
+
+- `AGENTS.md` — operating instructions + “memory”
+- `SOUL.md` — persona, boundaries, tone
+- `TOOLS.md` — user-maintained tool notes (e.g. `imsg`, `sag`, conventions)
+- `BOOTSTRAP.md` — one-time first-run ritual (deleted after completion)
+- `IDENTITY.md` — agent name/vibe/emoji
+- `USER.md` — user profile + preferred address
+
+On the first turn of a new session, Durar injects the contents of these files directly into the agent context.
+
+Blank files are skipped. Large files are trimmed and truncated with a marker so prompts stay lean (read the file for full content).
+
+If a file is missing, Durar injects a single “missing file” marker line (and `Durar setup` will create a safe default template).
+
+`BOOTSTRAP.md` is only created for a **brand new workspace** (no other bootstrap files present). If you delete it after completing the ritual, it should not be recreated on later restarts.
+
+To disable bootstrap file creation entirely (for pre-seeded workspaces), set:
+
+```json5
+{ agent: { skipBootstrap: true } }
+```
+
+## Built-in tools
+
+Core tools (read/exec/edit/write and related system tools) are always available,
+subject to tool policy. `apply_patch` is optional and gated by
+`tools.exec.applyPatch`. `TOOLS.md` does **not** control which tools exist; it’s
+guidance for how _you_ want them used.
+
+## Skills
+
+Durar loads skills from these locations (highest precedence first):
+
+- Workspace: `<workspace>/skills`
+- Project agent skills: `<workspace>/.agents/skills`
+- Personal agent skills: `~/.agents/skills`
+- Managed/local: `~/.Durar/skills`
+- Bundled (shipped with the install)
+- Extra skill folders: `skills.load.extraDirs`
+
+Skills can be gated by config/env (see `skills` in [Gateway configuration](/gateway/configuration)).
+
+## Runtime boundaries
+
+The embedded agent runtime is built on the Pi agent core (models, tools, and
+prompt pipeline). Session management, discovery, tool wiring, and channel
+delivery are Durar-owned layers on top of that core.
+
+## Sessions
+
+Session transcripts are stored as JSONL at:
+
+- `~/.Durar/agents/<agentId>/sessions/<SessionId>.jsonl`
+
+The session ID is stable and chosen by Durar.
+Legacy session folders from other tools are not read.
+
+## Steering while streaming
+
+When queue mode is `steer`, inbound messages are injected into the current run.
+Queued steering is delivered **after the current assistant turn finishes
+executing its tool calls**, before the next LLM call. Steering no longer skips
+remaining tool calls from the current assistant message; it injects the queued
+message at the next model boundary instead.
+
+When queue mode is `followup` or `collect`, inbound messages are held until the
+current turn ends, then a new agent turn starts with the queued payloads. See
+[Queue](/concepts/queue) for mode + debounce/cap behavior.
+
+Block streaming sends completed assistant blocks as soon as they finish; it is
+**off by default** (`agents.defaults.blockStreamingDefault: "off"`).
+Tune the boundary via `agents.defaults.blockStreamingBreak` (`text_end` vs `message_end`; defaults to text_end).
+Control soft block chunking with `agents.defaults.blockStreamingChunk` (defaults to
+800–1200 chars; prefers paragraph breaks, then newlines; sentences last).
+Coalesce streamed chunks with `agents.defaults.blockStreamingCoalesce` to reduce
+single-line spam (idle-based merging before send). Non-Telegram channels require
+explicit `*.blockStreaming: true` to enable block replies.
+Verbose tool summaries are emitted at tool start (no debounce); Control UI
+streams tool output via agent events when available.
+More details: [Streaming + chunking](/concepts/streaming).
+
+## Model refs
+
+Model refs in config (for example `agents.defaults.model` and `agents.defaults.models`) are parsed by splitting on the **first** `/`.
+
+- Use `provider/model` when configuring models.
+- If the model ID itself contains `/` (OpenRouter-style), include the provider prefix (example: `openrouter/moonshotai/kimi-k2`).
+- If you omit the provider, Durar tries an alias first, then a unique
+  configured-provider match for that exact model id, and only then falls back
+  to the configured default provider. If that provider no longer exposes the
+  configured default model, Durar falls back to the first configured
+  provider/model instead of surfacing a stale removed-provider default.
+
+## Configuration (minimal)
+
+At minimum, set:
+
+- `agents.defaults.workspace`
+- `channels.whatsapp.allowFrom` (strongly recommended)
+
+---
+
+_Next: [Group Chats](/channels/group-messages)_ 🦞

package/docs/concepts/architecture.md

@@ -0,0 +1,156 @@
+---
+summary: "WebSocket gateway architecture, components, and client flows"
+read_when:
+  - Working on gateway protocol, clients, or transports
+title: "Gateway Architecture"
+---
+
+# Gateway architecture
+
+## Overview
+
+- A single long‑lived **Gateway** owns all messaging surfaces (WhatsApp via
+  Baileys, Telegram via grammY, Slack, Discord, Signal, iMessage, WebChat).
+- Control-plane clients (macOS app, CLI, web UI, automations) connect to the
+  Gateway over **WebSocket** on the configured bind host (default
+  `127.0.0.1:18789`).
+- **Nodes** (macOS/iOS/Android/headless) also connect over **WebSocket**, but
+  declare `role: node` with explicit caps/commands.
+- One Gateway per host; it is the only place that opens a WhatsApp session.
+- The **canvas host** is served by the Gateway HTTP server under:
+  - `/__Durar__/canvas/` (agent-editable HTML/CSS/JS)
+  - `/__Durar__/a2ui/` (A2UI host)
+    It uses the same port as the Gateway (default `18789`).
+
+## Components and flows
+
+### Gateway (daemon)
+
+- Maintains provider connections.
+- Exposes a typed WS API (requests, responses, server‑push events).
+- Validates inbound frames against JSON Schema.
+- Emits events like `agent`, `chat`, `presence`, `health`, `heartbeat`, `cron`.
+
+### Clients (mac app / CLI / web admin)
+
+- One WS connection per client.
+- Send requests (`health`, `status`, `send`, `agent`, `system-presence`).
+- Subscribe to events (`tick`, `agent`, `presence`, `shutdown`).
+
+### Nodes (macOS / iOS / Android / headless)
+
+- Connect to the **same WS server** with `role: node`.
+- Provide a device identity in `connect`; pairing is **device‑based** (role `node`) and
+  approval lives in the device pairing store.
+- Expose commands like `canvas.*`, `camera.*`, `screen.record`, `location.get`.
+
+Protocol details:
+
+- [Gateway protocol](/gateway/protocol)
+
+### WebChat
+
+- Static UI that uses the Gateway WS API for chat history and sends.
+- In remote setups, connects through the same SSH/Tailscale tunnel as other
+  clients.
+
+## Connection lifecycle (single client)
+
+```mermaid
+sequenceDiagram
+    participant Client
+    participant Gateway
+
+    Client->>Gateway: req:connect
+    Gateway-->>Client: res (ok)
+    Note right of Gateway: or res error + close
+    Note left of Client: payload=hello-ok<br>snapshot: presence + health
+
+    Gateway-->>Client: event:presence
+    Gateway-->>Client: event:tick
+
+    Client->>Gateway: req:agent
+    Gateway-->>Client: res:agent<br>ack {runId, status:"accepted"}
+    Gateway-->>Client: event:agent<br>(streaming)
+    Gateway-->>Client: res:agent<br>final {runId, status, summary}
+```
+
+## Wire protocol (summary)
+
+- Transport: WebSocket, text frames with JSON payloads.
+- First frame **must** be `connect`.
+- After handshake:
+  - Requests: `{type:"req", id, method, params}` → `{type:"res", id, ok, payload|error}`
+  - Events: `{type:"event", event, payload, seq?, stateVersion?}`
+- `hello-ok.features.methods` / `events` are discovery metadata, not a
+  generated dump of every callable helper route.
+- Shared-secret auth uses `connect.params.auth.token` or
+  `connect.params.auth.password`, depending on the configured gateway auth mode.
+- Identity-bearing modes such as Tailscale Serve
+  (`gateway.auth.allowTailscale: true`) or non-loopback
+  `gateway.auth.mode: "trusted-proxy"` satisfy auth from request headers
+  instead of `connect.params.auth.*`.
+- Private-ingress `gateway.auth.mode: "none"` disables shared-secret auth
+  entirely; keep that mode off public/untrusted ingress.
+- Idempotency keys are required for side‑effecting methods (`send`, `agent`) to
+  safely retry; the server keeps a short‑lived dedupe cache.
+- Nodes must include `role: "node"` plus caps/commands/permissions in `connect`.
+
+## Pairing + local trust
+
+- All WS clients (operators + nodes) include a **device identity** on `connect`.
+- New device IDs require pairing approval; the Gateway issues a **device token**
+  for subsequent connects.
+- Direct local loopback connects can be auto-approved to keep same-host UX
+  smooth.
+- Durar also has a narrow backend/container-local self-connect path for
+  trusted shared-secret helper flows.
+- Tailnet and LAN connects, including same-host tailnet binds, still require
+  explicit pairing approval.
+- All connects must sign the `connect.challenge` nonce.
+- Signature payload `v3` also binds `platform` + `deviceFamily`; the gateway
+  pins paired metadata on reconnect and requires repair pairing for metadata
+  changes.
+- **Non‑local** connects still require explicit approval.
+- Gateway auth (`gateway.auth.*`) still applies to **all** connections, local or
+  remote.
+
+Details: [Gateway protocol](/gateway/protocol), [Pairing](/channels/pairing),
+[Security](/gateway/security).
+
+## Protocol typing and codegen
+
+- TypeBox schemas define the protocol.
+- JSON Schema is generated from those schemas.
+- Swift models are generated from the JSON Schema.
+
+## Remote access
+
+- Preferred: Tailscale or VPN.
+- Alternative: SSH tunnel
+
+  ```bash
+  ssh -N -L 18789:127.0.0.1:18789 user@host
+  ```
+
+- The same handshake + auth token apply over the tunnel.
+- TLS + optional pinning can be enabled for WS in remote setups.
+
+## Operations snapshot
+
+- Start: `Durar gateway` (foreground, logs to stdout).
+- Health: `health` over WS (also included in `hello-ok`).
+- Supervision: launchd/systemd for auto‑restart.
+
+## Invariants
+
+- Exactly one Gateway controls a single Baileys session per host.
+- Handshake is mandatory; any non‑JSON or non‑connect first frame is a hard close.
+- Events are not replayed; clients must refresh on gaps.
+
+## Related
+
+- [Agent Loop](/concepts/agent-loop) — detailed agent execution cycle
+- [Gateway Protocol](/gateway/protocol) — WebSocket protocol contract
+- [Queue](/concepts/queue) — command queue and concurrency
+- [Security](/gateway/security) — trust model and hardening

package/docs/concepts/compaction.md

@@ -0,0 +1,122 @@
+---
+summary: "How Durar summarizes long conversations to stay within model limits"
+read_when:
+  - You want to understand auto-compaction and /compact
+  - You are debugging long sessions hitting context limits
+title: "Compaction"
+---
+
+# Compaction
+
+Every model has a context window -- the maximum number of tokens it can process.
+When a conversation approaches that limit, Durar **compacts** older messages
+into a summary so the chat can continue.
+
+## How it works
+
+1. Older conversation turns are summarized into a compact entry.
+2. The summary is saved in the session transcript.
+3. Recent messages are kept intact.
+
+When Durar splits history into compaction chunks, it keeps assistant tool
+calls paired with their matching `toolResult` entries. If a split point lands
+inside a tool block, Durar moves the boundary so the pair stays together and
+the current unsummarized tail is preserved.
+
+The full conversation history stays on disk. Compaction only changes what the
+model sees on the next turn.
+
+## Auto-compaction
+
+Auto-compaction is on by default. It runs when the session nears the context
+limit, or when the model returns a context-overflow error (in which case
+Durar compacts and retries). Typical overflow signatures include
+`request_too_large`, `context length exceeded`, `input exceeds the maximum
+number of tokens`, `input token count exceeds the maximum number of input
+tokens`, `input is too long for the model`, and `ollama error: context length
+exceeded`.
+
+<Info>
+Before compacting, Durar automatically reminds the agent to save important
+notes to [memory](/concepts/memory) files. This prevents context loss.
+</Info>
+
+## Manual compaction
+
+Type `/compact` in any chat to force a compaction. Add instructions to guide
+the summary:
+
+```
+/compact Focus on the API design decisions
+```
+
+## Using a different model
+
+By default, compaction uses your agent's primary model. You can use a more
+capable model for better summaries:
+
+```json5
+{
+  agents: {
+    defaults: {
+      compaction: {
+        model: "openrouter/anthropic/claude-sonnet-4-6",
+      },
+    },
+  },
+}
+```
+
+## Compaction start notice
+
+By default, compaction runs silently. To show a brief notice when compaction
+starts, enable `notifyUser`:
+
+```json5
+{
+  agents: {
+    defaults: {
+      compaction: {
+        notifyUser: true,
+      },
+    },
+  },
+}
+```
+
+When enabled, the user sees a short message (for example, "Compacting
+context...") at the start of each compaction run.
+
+## Compaction vs pruning
+
+|                  | Compaction                    | Pruning                          |
+| ---------------- | ----------------------------- | -------------------------------- |
+| **What it does** | Summarizes older conversation | Trims old tool results           |
+| **Saved?**       | Yes (in session transcript)   | No (in-memory only, per request) |
+| **Scope**        | Entire conversation           | Tool results only                |
+
+[Session pruning](/concepts/session-pruning) is a lighter-weight complement that
+trims tool output without summarizing.
+
+## Troubleshooting
+
+**Compacting too often?** The model's context window may be small, or tool
+outputs may be large. Try enabling
+[session pruning](/concepts/session-pruning).
+
+**Context feels stale after compaction?** Use `/compact Focus on <topic>` to
+guide the summary, or enable the [memory flush](/concepts/memory) so notes
+survive.
+
+**Need a clean slate?** `/new` starts a fresh session without compacting.
+
+For advanced configuration (reserve tokens, identifier preservation, custom
+context engines, OpenAI server-side compaction), see the
+[Session Management Deep Dive](/reference/session-management-compaction).
+
+## Related
+
+- [Session](/concepts/session) — session management and lifecycle
+- [Session Pruning](/concepts/session-pruning) — trimming tool results
+- [Context](/concepts/context) — how context is built for agent turns
+- [Hooks](/automation/hooks) — compaction lifecycle hooks (before_compaction, after_compaction)