durar-ai 2026.4.4

package/docs/gateway/index.md

@@ -0,0 +1,367 @@
+---
+summary: "Runbook for the Gateway service, lifecycle, and operations"
+read_when:
+  - Running or debugging the gateway process
+title: "Gateway Runbook"
+---
+
+# Gateway runbook
+
+Use this page for day-1 startup and day-2 operations of the Gateway service.
+
+<CardGroup cols={2}>
+  <Card title="Deep troubleshooting" icon="siren" href="/gateway/troubleshooting">
+    Symptom-first diagnostics with exact command ladders and log signatures.
+  </Card>
+  <Card title="Configuration" icon="sliders" href="/gateway/configuration">
+    Task-oriented setup guide + full configuration reference.
+  </Card>
+  <Card title="Secrets management" icon="key-round" href="/gateway/secrets">
+    SecretRef contract, runtime snapshot behavior, and migrate/reload operations.
+  </Card>
+  <Card title="Secrets plan contract" icon="shield-check" href="/gateway/secrets-plan-contract">
+    Exact `secrets apply` target/path rules and ref-only auth-profile behavior.
+  </Card>
+</CardGroup>
+
+## 5-minute local startup
+
+<Steps>
+  <Step title="Start the Gateway">
+
+```bash
+Durar gateway --port 18789
+# debug/trace mirrored to stdio
+Durar gateway --port 18789 --verbose
+# force-kill listener on selected port, then start
+Durar gateway --force
+```
+
+  </Step>
+
+  <Step title="Verify service health">
+
+```bash
+Durar gateway status
+Durar status
+Durar logs --follow
+```
+
+Healthy baseline: `Runtime: running` and `RPC probe: ok`.
+
+  </Step>
+
+  <Step title="Validate channel readiness">
+
+```bash
+Durar channels status --probe
+```
+
+With a reachable gateway this runs live per-account channel probes and optional audits.
+If the gateway is unreachable, the CLI falls back to config-only channel summaries instead
+of live probe output.
+
+  </Step>
+</Steps>
+
+<Note>
+Gateway config reload watches the active config file path (resolved from profile/state defaults, or `Durar_CONFIG_PATH` when set).
+Default mode is `gateway.reload.mode="hybrid"`.
+After the first successful load, the running process serves the active in-memory config snapshot; successful reload swaps that snapshot atomically.
+</Note>
+
+## Runtime model
+
+- One always-on process for routing, control plane, and channel connections.
+- Single multiplexed port for:
+  - WebSocket control/RPC
+  - HTTP APIs, OpenAI compatible (`/v1/models`, `/v1/embeddings`, `/v1/chat/completions`, `/v1/responses`, `/tools/invoke`)
+  - Control UI and hooks
+- Default bind mode: `loopback`.
+- Auth is required by default. Shared-secret setups use
+  `gateway.auth.token` / `gateway.auth.password` (or
+  `Durar_GATEWAY_TOKEN` / `Durar_GATEWAY_PASSWORD`), and non-loopback
+  reverse-proxy setups can use `gateway.auth.mode: "trusted-proxy"`.
+
+## OpenAI-compatible endpoints
+
+Durar’s highest-leverage compatibility surface is now:
+
+- `GET /v1/models`
+- `GET /v1/models/{id}`
+- `POST /v1/embeddings`
+- `POST /v1/chat/completions`
+- `POST /v1/responses`
+
+Why this set matters:
+
+- Most Open WebUI, LobeChat, and LibreChat integrations probe `/v1/models` first.
+- Many RAG and memory pipelines expect `/v1/embeddings`.
+- Agent-native clients increasingly prefer `/v1/responses`.
+
+Planning note:
+
+- `/v1/models` is agent-first: it returns `Durar`, `Durar/default`, and `Durar/<agentId>`.
+- `Durar/default` is the stable alias that always maps to the configured default agent.
+- Use `x-Durar-model` when you want a backend provider/model override; otherwise the selected agent's normal model and embedding setup stays in control.
+
+All of these run on the main Gateway port and use the same trusted operator auth boundary as the rest of the Gateway HTTP API.
+
+### Port and bind precedence
+
+| Setting      | Resolution order                                              |
+| ------------ | ------------------------------------------------------------- |
+| Gateway port | `--port` → `Durar_GATEWAY_PORT` → `gateway.port` → `18789` |
+| Bind mode    | CLI/override → `gateway.bind` → `loopback`                    |
+
+### Hot reload modes
+
+| `gateway.reload.mode` | Behavior                                   |
+| --------------------- | ------------------------------------------ |
+| `off`                 | No config reload                           |
+| `hot`                 | Apply only hot-safe changes                |
+| `restart`             | Restart on reload-required changes         |
+| `hybrid` (default)    | Hot-apply when safe, restart when required |
+
+## Operator command set
+
+```bash
+Durar gateway status
+Durar gateway status --deep   # adds a system-level service scan
+Durar gateway status --json
+Durar gateway install
+Durar gateway restart
+Durar gateway stop
+Durar secrets reload
+Durar logs --follow
+Durar doctor
+```
+
+`gateway status --deep` is for extra service discovery (LaunchDaemons/systemd system
+units/schtasks), not a deeper RPC health probe.
+
+## Multiple gateways (same host)
+
+Most installs should run one gateway per machine. A single gateway can host multiple
+agents and channels.
+
+You only need multiple gateways when you intentionally want isolation or a rescue bot.
+
+Useful checks:
+
+```bash
+Durar gateway status --deep
+Durar gateway probe
+```
+
+What to expect:
+
+- `gateway status --deep` can report `Other gateway-like services detected (best effort)`
+  and print cleanup hints when stale launchd/systemd/schtasks installs are still around.
+- `gateway probe` can warn about `multiple reachable gateways` when more than one target
+  answers.
+- If that is intentional, isolate ports, config/state, and workspace roots per gateway.
+
+Detailed setup: [/gateway/multiple-gateways](/gateway/multiple-gateways).
+
+## Remote access
+
+Preferred: Tailscale/VPN.
+Fallback: SSH tunnel.
+
+```bash
+ssh -N -L 18789:127.0.0.1:18789 user@host
+```
+
+Then connect clients to `ws://127.0.0.1:18789` locally.
+
+<Warning>
+SSH tunnels do not bypass gateway auth. For shared-secret auth, clients still
+must send `token`/`password` even over the tunnel. For identity-bearing modes,
+the request still has to satisfy that auth path.
+</Warning>
+
+See: [Remote Gateway](/gateway/remote), [Authentication](/gateway/authentication), [Tailscale](/gateway/tailscale).
+
+## Supervision and service lifecycle
+
+Use supervised runs for production-like reliability.
+
+<Tabs>
+  <Tab title="macOS (launchd)">
+
+```bash
+Durar gateway install
+Durar gateway status
+Durar gateway restart
+Durar gateway stop
+```
+
+LaunchAgent labels are `ai.Durar.gateway` (default) or `ai.Durar.<profile>` (named profile). `Durar doctor` audits and repairs service config drift.
+
+  </Tab>
+
+  <Tab title="Linux (systemd user)">
+
+```bash
+Durar gateway install
+systemctl --user enable --now Durar-gateway[-<profile>].service
+Durar gateway status
+```
+
+For persistence after logout, enable lingering:
+
+```bash
+sudo loginctl enable-linger <user>
+```
+
+Manual user-unit example when you need a custom install path:
+
+```ini
+[Unit]
+Description=Durar Gateway
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+ExecStart=/usr/local/bin/Durar gateway --port 18789
+Restart=always
+RestartSec=5
+TimeoutStopSec=30
+TimeoutStartSec=30
+SuccessExitStatus=0 143
+KillMode=control-group
+
+[Install]
+WantedBy=default.target
+```
+
+  </Tab>
+
+  <Tab title="Windows (native)">
+
+```powershell
+Durar gateway install
+Durar gateway status --json
+Durar gateway restart
+Durar gateway stop
+```
+
+Native Windows managed startup uses a Scheduled Task named `Durar Gateway`
+(or `Durar Gateway (<profile>)` for named profiles). If Scheduled Task
+creation is denied, Durar falls back to a per-user Startup-folder launcher
+that points at `gateway.cmd` inside the state directory.
+
+  </Tab>
+
+  <Tab title="Linux (system service)">
+
+Use a system unit for multi-user/always-on hosts.
+
+```bash
+sudo systemctl daemon-reload
+sudo systemctl enable --now Durar-gateway[-<profile>].service
+```
+
+Use the same service body as the user unit, but install it under
+`/etc/systemd/system/Durar-gateway[-<profile>].service` and adjust
+`ExecStart=` if your `Durar` binary lives elsewhere.
+
+  </Tab>
+</Tabs>
+
+## Multiple gateways on one host
+
+Most setups should run **one** Gateway.
+Use multiple only for strict isolation/redundancy (for example a rescue profile).
+
+Checklist per instance:
+
+- Unique `gateway.port`
+- Unique `Durar_CONFIG_PATH`
+- Unique `Durar_STATE_DIR`
+- Unique `agents.defaults.workspace`
+
+Example:
+
+```bash
+Durar_CONFIG_PATH=~/.Durar/a.json Durar_STATE_DIR=~/.Durar-a Durar gateway --port 19001
+Durar_CONFIG_PATH=~/.Durar/b.json Durar_STATE_DIR=~/.Durar-b Durar gateway --port 19002
+```
+
+See: [Multiple gateways](/gateway/multiple-gateways).
+
+### Dev profile quick path
+
+```bash
+Durar --dev setup
+Durar --dev gateway --allow-unconfigured
+Durar --dev status
+```
+
+Defaults include isolated state/config and base gateway port `19001`.
+
+## Protocol quick reference (operator view)
+
+- First client frame must be `connect`.
+- Gateway returns `hello-ok` snapshot (`presence`, `health`, `stateVersion`, `uptimeMs`, limits/policy).
+- `hello-ok.features.methods` / `events` are a conservative discovery list, not
+  a generated dump of every callable helper route.
+- Requests: `req(method, params)` → `res(ok/payload|error)`.
+- Common events include `connect.challenge`, `agent`, `chat`,
+  `session.message`, `session.tool`, `sessions.changed`, `presence`, `tick`,
+  `health`, `heartbeat`, pairing/approval lifecycle events, and `shutdown`.
+
+Agent runs are two-stage:
+
+1. Immediate accepted ack (`status:"accepted"`)
+2. Final completion response (`status:"ok"|"error"`), with streamed `agent` events in between.
+
+See full protocol docs: [Gateway Protocol](/gateway/protocol).
+
+## Operational checks
+
+### Liveness
+
+- Open WS and send `connect`.
+- Expect `hello-ok` response with snapshot.
+
+### Readiness
+
+```bash
+Durar gateway status
+Durar channels status --probe
+Durar health
+```
+
+### Gap recovery
+
+Events are not replayed. On sequence gaps, refresh state (`health`, `system-presence`) before continuing.
+
+## Common failure signatures
+
+| Signature                                                      | Likely issue                                                                    |
+| -------------------------------------------------------------- | ------------------------------------------------------------------------------- |
+| `refusing to bind gateway ... without auth`                    | Non-loopback bind without a valid gateway auth path                             |
+| `another gateway instance is already listening` / `EADDRINUSE` | Port conflict                                                                   |
+| `Gateway start blocked: set gateway.mode=local`                | Config set to remote mode, or local-mode stamp is missing from a damaged config |
+| `unauthorized` during connect                                  | Auth mismatch between client and gateway                                        |
+
+For full diagnosis ladders, use [Gateway Troubleshooting](/gateway/troubleshooting).
+
+## Safety guarantees
+
+- Gateway protocol clients fail fast when Gateway is unavailable (no implicit direct-channel fallback).
+- Invalid/non-connect first frames are rejected and closed.
+- Graceful shutdown emits `shutdown` event before socket close.
+
+---
+
+Related:
+
+- [Troubleshooting](/gateway/troubleshooting)
+- [Background Process](/gateway/background-process)
+- [Configuration](/gateway/configuration)
+- [Health](/gateway/health)
+- [Doctor](/gateway/doctor)
+- [Authentication](/gateway/authentication)

package/docs/gateway/local-models.md

@@ -0,0 +1,163 @@
+---
+summary: "Run Durar on local LLMs (LM Studio, vLLM, LiteLLM, custom OpenAI endpoints)"
+read_when:
+  - You want to serve models from your own GPU box
+  - You are wiring LM Studio or an OpenAI-compatible proxy
+  - You need the safest local model guidance
+title: "Local Models"
+---
+
+# Local models
+
+Local is doable, but Durar expects large context + strong defenses against prompt injection. Small cards truncate context and leak safety. Aim high: **≥2 maxed-out Mac Studios or equivalent GPU rig (~$30k+)**. A single **24 GB** GPU works only for lighter prompts with higher latency. Use the **largest / full-size model variant you can run**; aggressively quantized or “small” checkpoints raise prompt-injection risk (see [Security](/gateway/security)).
+
+If you want the lowest-friction local setup, start with [Ollama](/providers/ollama) and `Durar onboard`. This page is the opinionated guide for higher-end local stacks and custom OpenAI-compatible local servers.
+
+## Recommended: LM Studio + large local model (Responses API)
+
+Best current local stack. Load a large model in LM Studio (for example, a full-size Qwen, DeepSeek, or Llama build), enable the local server (default `http://127.0.0.1:1234`), and use Responses API to keep reasoning separate from final text.
+
+```json5
+{
+  agents: {
+    defaults: {
+      model: { primary: “lmstudio/my-local-model” },
+      models: {
+        “anthropic/claude-opus-4-6”: { alias: “Opus” },
+        “lmstudio/my-local-model”: { alias: “Local” },
+      },
+    },
+  },
+  models: {
+    mode: “merge”,
+    providers: {
+      lmstudio: {
+        baseUrl: “http://127.0.0.1:1234/v1”,
+        apiKey: “lmstudio”,
+        api: “openai-responses”,
+        models: [
+          {
+            id: “my-local-model”,
+            name: “Local Model”,
+            reasoning: false,
+            input: [“text”],
+            cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
+            contextWindow: 196608,
+            maxTokens: 8192,
+          },
+        ],
+      },
+    },
+  },
+}
+```
+
+**Setup checklist**
+
+- Install LM Studio: [https://lmstudio.ai](https://lmstudio.ai)
+- In LM Studio, download the **largest model build available** (avoid “small”/heavily quantized variants), start the server, confirm `http://127.0.0.1:1234/v1/models` lists it.
+- Replace `my-local-model` with the actual model ID shown in LM Studio.
+- Keep the model loaded; cold-load adds startup latency.
+- Adjust `contextWindow`/`maxTokens` if your LM Studio build differs.
+- For WhatsApp, stick to Responses API so only final text is sent.
+
+Keep hosted models configured even when running local; use `models.mode: "merge"` so fallbacks stay available.
+
+### Hybrid config: hosted primary, local fallback
+
+```json5
+{
+  agents: {
+    defaults: {
+      model: {
+        primary: "anthropic/claude-sonnet-4-6",
+        fallbacks: ["lmstudio/my-local-model", "anthropic/claude-opus-4-6"],
+      },
+      models: {
+        "anthropic/claude-sonnet-4-6": { alias: "Sonnet" },
+        "lmstudio/my-local-model": { alias: "Local" },
+        "anthropic/claude-opus-4-6": { alias: "Opus" },
+      },
+    },
+  },
+  models: {
+    mode: "merge",
+    providers: {
+      lmstudio: {
+        baseUrl: "http://127.0.0.1:1234/v1",
+        apiKey: "lmstudio",
+        api: "openai-responses",
+        models: [
+          {
+            id: "my-local-model",
+            name: "Local Model",
+            reasoning: false,
+            input: ["text"],
+            cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
+            contextWindow: 196608,
+            maxTokens: 8192,
+          },
+        ],
+      },
+    },
+  },
+}
+```
+
+### Local-first with hosted safety net
+
+Swap the primary and fallback order; keep the same providers block and `models.mode: "merge"` so you can fall back to Sonnet or Opus when the local box is down.
+
+### Regional hosting / data routing
+
+- Hosted MiniMax/Kimi/GLM variants also exist on OpenRouter with region-pinned endpoints (e.g., US-hosted). Pick the regional variant there to keep traffic in your chosen jurisdiction while still using `models.mode: "merge"` for Anthropic/OpenAI fallbacks.
+- Local-only remains the strongest privacy path; hosted regional routing is the middle ground when you need provider features but want control over data flow.
+
+## Other OpenAI-compatible local proxies
+
+vLLM, LiteLLM, OAI-proxy, or custom gateways work if they expose an OpenAI-style `/v1` endpoint. Replace the provider block above with your endpoint and model ID:
+
+```json5
+{
+  models: {
+    mode: "merge",
+    providers: {
+      local: {
+        baseUrl: "http://127.0.0.1:8000/v1",
+        apiKey: "sk-local",
+        api: "openai-responses",
+        models: [
+          {
+            id: "my-local-model",
+            name: "Local Model",
+            reasoning: false,
+            input: ["text"],
+            cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
+            contextWindow: 120000,
+            maxTokens: 8192,
+          },
+        ],
+      },
+    },
+  },
+}
+```
+
+Keep `models.mode: "merge"` so hosted models stay available as fallbacks.
+
+Behavior note for local/proxied `/v1` backends:
+
+- Durar treats these as proxy-style OpenAI-compatible routes, not native
+  OpenAI endpoints
+- native OpenAI-only request shaping does not apply here: no
+  `service_tier`, no Responses `store`, no OpenAI reasoning-compat payload
+  shaping, and no prompt-cache hints
+- hidden Durar attribution headers (`originator`, `version`, `User-Agent`)
+  are not injected on these custom proxy URLs
+
+## Troubleshooting
+
+- Gateway can reach the proxy? `curl http://127.0.0.1:1234/v1/models`.
+- LM Studio model unloaded? Reload; cold start is a common “hanging” cause.
+- Context errors? Lower `contextWindow` or raise your server limit.
+- Safety: local models skip provider-side filters; keep agents narrow and compaction on to limit prompt injection blast radius.

package/docs/gateway/logging.md

@@ -0,0 +1,113 @@
+---
+summary: "Logging surfaces, file logs, WS log styles, and console formatting"
+read_when:
+  - Changing logging output or formats
+  - Debugging CLI or gateway output
+title: "Gateway Logging"
+---
+
+# Logging
+
+For a user-facing overview (CLI + Control UI + config), see [/logging](/logging).
+
+Durar has two log “surfaces”:
+
+- **Console output** (what you see in the terminal / Debug UI).
+- **File logs** (JSON lines) written by the gateway logger.
+
+## File-based logger
+
+- Default rolling log file is under `/tmp/Durar/` (one file per day): `Durar-YYYY-MM-DD.log`
+  - Date uses the gateway host's local timezone.
+- The log file path and level can be configured via `~/.Durar/Durar.json`:
+  - `logging.file`
+  - `logging.level`
+
+The file format is one JSON object per line.
+
+The Control UI Logs tab tails this file via the gateway (`logs.tail`).
+CLI can do the same:
+
+```bash
+Durar logs --follow
+```
+
+**Verbose vs. log levels**
+
+- **File logs** are controlled exclusively by `logging.level`.
+- `--verbose` only affects **console verbosity** (and WS log style); it does **not**
+  raise the file log level.
+- To capture verbose-only details in file logs, set `logging.level` to `debug` or
+  `trace`.
+
+## Console capture
+
+The CLI captures `console.log/info/warn/error/debug/trace` and writes them to file logs,
+while still printing to stdout/stderr.
+
+You can tune console verbosity independently via:
+
+- `logging.consoleLevel` (default `info`)
+- `logging.consoleStyle` (`pretty` | `compact` | `json`)
+
+## Tool summary redaction
+
+Verbose tool summaries (e.g. `🛠️ Exec: ...`) can mask sensitive tokens before they hit the
+console stream. This is **tools-only** and does not alter file logs.
+
+- `logging.redactSensitive`: `off` | `tools` (default: `tools`)
+- `logging.redactPatterns`: array of regex strings (overrides defaults)
+  - Use raw regex strings (auto `gi`), or `/pattern/flags` if you need custom flags.
+  - Matches are masked by keeping the first 6 + last 4 chars (length >= 18), otherwise `***`.
+  - Defaults cover common key assignments, CLI flags, JSON fields, bearer headers, PEM blocks, and popular token prefixes.
+
+## Gateway WebSocket logs
+
+The gateway prints WebSocket protocol logs in two modes:
+
+- **Normal mode (no `--verbose`)**: only “interesting” RPC results are printed:
+  - errors (`ok=false`)
+  - slow calls (default threshold: `>= 50ms`)
+  - parse errors
+- **Verbose mode (`--verbose`)**: prints all WS request/response traffic.
+
+### WS log style
+
+`Durar gateway` supports a per-gateway style switch:
+
+- `--ws-log auto` (default): normal mode is optimized; verbose mode uses compact output
+- `--ws-log compact`: compact output (paired request/response) when verbose
+- `--ws-log full`: full per-frame output when verbose
+- `--compact`: alias for `--ws-log compact`
+
+Examples:
+
+```bash
+# optimized (only errors/slow)
+Durar gateway
+
+# show all WS traffic (paired)
+Durar gateway --verbose --ws-log compact
+
+# show all WS traffic (full meta)
+Durar gateway --verbose --ws-log full
+```
+
+## Console formatting (subsystem logging)
+
+The console formatter is **TTY-aware** and prints consistent, prefixed lines.
+Subsystem loggers keep output grouped and scannable.
+
+Behavior:
+
+- **Subsystem prefixes** on every line (e.g. `[gateway]`, `[canvas]`, `[tailscale]`)
+- **Subsystem colors** (stable per subsystem) plus level coloring
+- **Color when output is a TTY or the environment looks like a rich terminal** (`TERM`/`COLORTERM`/`TERM_PROGRAM`), respects `NO_COLOR`
+- **Shortened subsystem prefixes**: drops leading `gateway/` + `channels/`, keeps last 2 segments (e.g. `whatsapp/outbound`)
+- **Sub-loggers by subsystem** (auto prefix + structured field `{ subsystem }`)
+- **`logRaw()`** for QR/UX output (no prefix, no formatting)
+- **Console styles** (e.g. `pretty | compact | json`)
+- **Console log level** separate from file log level (file keeps full detail when `logging.level` is set to `debug`/`trace`)
+- **WhatsApp message bodies** are logged at `debug` (use `--verbose` to see them)
+
+This keeps existing file logs stable while making interactive output scannable.