durar-ai 2026.4.4

package/docs/logging.md

@@ -0,0 +1,389 @@
+---
+summary: "Logging overview: file logs, console output, CLI tailing, and the Control UI"
+read_when:
+  - You need a beginner-friendly overview of logging
+  - You want to configure log levels or formats
+  - You are troubleshooting and need to find logs quickly
+title: "Logging Overview"
+---
+
+# Logging
+
+Durar has two main log surfaces:
+
+- **File logs** (JSON lines) written by the Gateway.
+- **Console output** shown in terminals and the Gateway Debug UI.
+
+The Control UI **Logs** tab tails the gateway file log. This page explains where
+logs live, how to read them, and how to configure log levels and formats.
+
+## Where logs live
+
+By default, the Gateway writes a rolling log file under:
+
+`/tmp/Durar/Durar-YYYY-MM-DD.log`
+
+The date uses the gateway host's local timezone.
+
+You can override this in `~/.Durar/Durar.json`:
+
+```json
+{
+  "logging": {
+    "file": "/path/to/Durar.log"
+  }
+}
+```
+
+## How to read logs
+
+### CLI: live tail (recommended)
+
+Use the CLI to tail the gateway log file via RPC:
+
+```bash
+Durar logs --follow
+```
+
+Useful current options:
+
+- `--local-time`: render timestamps in your local timezone
+- `--url <url>` / `--token <token>` / `--timeout <ms>`: standard Gateway RPC flags
+- `--expect-final`: agent-backed RPC final-response wait flag (accepted here via the shared client layer)
+
+Output modes:
+
+- **TTY sessions**: pretty, colorized, structured log lines.
+- **Non-TTY sessions**: plain text.
+- `--json`: line-delimited JSON (one log event per line).
+- `--plain`: force plain text in TTY sessions.
+- `--no-color`: disable ANSI colors.
+
+When you pass an explicit `--url`, the CLI does not auto-apply config or
+environment credentials; include `--token` yourself if the target Gateway
+requires auth.
+
+In JSON mode, the CLI emits `type`-tagged objects:
+
+- `meta`: stream metadata (file, cursor, size)
+- `log`: parsed log entry
+- `notice`: truncation / rotation hints
+- `raw`: unparsed log line
+
+If the local loopback Gateway asks for pairing, `Durar logs` falls back to
+the configured local log file automatically. Explicit `--url` targets do not
+use this fallback.
+
+If the Gateway is unreachable, the CLI prints a short hint to run:
+
+```bash
+Durar doctor
+```
+
+### Control UI (web)
+
+The Control UI’s **Logs** tab tails the same file using `logs.tail`.
+See [/web/control-ui](/web/control-ui) for how to open it.
+
+### Channel-only logs
+
+To filter channel activity (WhatsApp/Telegram/etc), use:
+
+```bash
+Durar channels logs --channel whatsapp
+```
+
+## Log formats
+
+### File logs (JSONL)
+
+Each line in the log file is a JSON object. The CLI and Control UI parse these
+entries to render structured output (time, level, subsystem, message).
+
+### Console output
+
+Console logs are **TTY-aware** and formatted for readability:
+
+- Subsystem prefixes (e.g. `gateway/channels/whatsapp`)
+- Level coloring (info/warn/error)
+- Optional compact or JSON mode
+
+Console formatting is controlled by `logging.consoleStyle`.
+
+### Gateway WebSocket logs
+
+`Durar gateway` also has WebSocket protocol logging for RPC traffic:
+
+- normal mode: only interesting results (errors, parse errors, slow calls)
+- `--verbose`: all request/response traffic
+- `--ws-log auto|compact|full`: pick the verbose rendering style
+- `--compact`: alias for `--ws-log compact`
+
+Examples:
+
+```bash
+Durar gateway
+Durar gateway --verbose --ws-log compact
+Durar gateway --verbose --ws-log full
+```
+
+## Configuring logging
+
+All logging configuration lives under `logging` in `~/.Durar/Durar.json`.
+
+```json
+{
+  "logging": {
+    "level": "info",
+    "file": "/tmp/Durar/Durar-YYYY-MM-DD.log",
+    "consoleLevel": "info",
+    "consoleStyle": "pretty",
+    "redactSensitive": "tools",
+    "redactPatterns": ["sk-.*"]
+  }
+}
+```
+
+### Log levels
+
+- `logging.level`: **file logs** (JSONL) level.
+- `logging.consoleLevel`: **console** verbosity level.
+
+You can override both via the **`Durar_LOG_LEVEL`** environment variable (e.g. `Durar_LOG_LEVEL=debug`). The env var takes precedence over the config file, so you can raise verbosity for a single run without editing `Durar.json`. You can also pass the global CLI option **`--log-level <level>`** (for example, `Durar --log-level debug gateway run`), which overrides the environment variable for that command.
+
+`--verbose` only affects console output and WS log verbosity; it does not change
+file log levels.
+
+### Console styles
+
+`logging.consoleStyle`:
+
+- `pretty`: human-friendly, colored, with timestamps.
+- `compact`: tighter output (best for long sessions).
+- `json`: JSON per line (for log processors).
+
+### Redaction
+
+Tool summaries can redact sensitive tokens before they hit the console:
+
+- `logging.redactSensitive`: `off` | `tools` (default: `tools`)
+- `logging.redactPatterns`: list of regex strings to override the default set
+
+Redaction affects **console output only** and does not alter file logs.
+
+## Diagnostics + OpenTelemetry
+
+Diagnostics are structured, machine-readable events for model runs **and**
+message-flow telemetry (webhooks, queueing, session state). They do **not**
+replace logs; they exist to feed metrics, traces, and other exporters.
+
+Diagnostics events are emitted in-process, but exporters only attach when
+diagnostics + the exporter plugin are enabled.
+
+### OpenTelemetry vs OTLP
+
+- **OpenTelemetry (OTel)**: the data model + SDKs for traces, metrics, and logs.
+- **OTLP**: the wire protocol used to export OTel data to a collector/backend.
+- Durar exports via **OTLP/HTTP (protobuf)** today.
+
+### Signals exported
+
+- **Metrics**: counters + histograms (token usage, message flow, queueing).
+- **Traces**: spans for model usage + webhook/message processing.
+- **Logs**: exported over OTLP when `diagnostics.otel.logs` is enabled. Log
+  volume can be high; keep `logging.level` and exporter filters in mind.
+
+### Diagnostic event catalog
+
+Model usage:
+
+- `model.usage`: tokens, cost, duration, context, provider/model/channel, session ids.
+
+Message flow:
+
+- `webhook.received`: webhook ingress per channel.
+- `webhook.processed`: webhook handled + duration.
+- `webhook.error`: webhook handler errors.
+- `message.queued`: message enqueued for processing.
+- `message.processed`: outcome + duration + optional error.
+
+Queue + session:
+
+- `queue.lane.enqueue`: command queue lane enqueue + depth.
+- `queue.lane.dequeue`: command queue lane dequeue + wait time.
+- `session.state`: session state transition + reason.
+- `session.stuck`: session stuck warning + age.
+- `run.attempt`: run retry/attempt metadata.
+- `diagnostic.heartbeat`: aggregate counters (webhooks/queue/session).
+
+### Enable diagnostics (no exporter)
+
+Use this if you want diagnostics events available to plugins or custom sinks:
+
+```json
+{
+  "diagnostics": {
+    "enabled": true
+  }
+}
+```
+
+### Diagnostics flags (targeted logs)
+
+Use flags to turn on extra, targeted debug logs without raising `logging.level`.
+Flags are case-insensitive and support wildcards (e.g. `telegram.*` or `*`).
+
+```json
+{
+  "diagnostics": {
+    "flags": ["telegram.http"]
+  }
+}
+```
+
+Env override (one-off):
+
+```
+Durar_DIAGNOSTICS=telegram.http,telegram.payload
+```
+
+Notes:
+
+- Flag logs go to the standard log file (same as `logging.file`).
+- Output is still redacted according to `logging.redactSensitive`.
+- Full guide: [/diagnostics/flags](/diagnostics/flags).
+
+### Export to OpenTelemetry
+
+Diagnostics can be exported via the `diagnostics-otel` plugin (OTLP/HTTP). This
+works with any OpenTelemetry collector/backend that accepts OTLP/HTTP.
+
+```json
+{
+  "plugins": {
+    "allow": ["diagnostics-otel"],
+    "entries": {
+      "diagnostics-otel": {
+        "enabled": true
+      }
+    }
+  },
+  "diagnostics": {
+    "enabled": true,
+    "otel": {
+      "enabled": true,
+      "endpoint": "http://otel-collector:4318",
+      "protocol": "http/protobuf",
+      "serviceName": "Durar-gateway",
+      "traces": true,
+      "metrics": true,
+      "logs": true,
+      "sampleRate": 0.2,
+      "flushIntervalMs": 60000
+    }
+  }
+}
+```
+
+Notes:
+
+- You can also enable the plugin with `Durar plugins enable diagnostics-otel`.
+- `protocol` currently supports `http/protobuf` only. `grpc` is ignored.
+- Metrics include token usage, cost, context size, run duration, and message-flow
+  counters/histograms (webhooks, queueing, session state, queue depth/wait).
+- Traces/metrics can be toggled with `traces` / `metrics` (default: on). Traces
+  include model usage spans plus webhook/message processing spans when enabled.
+- Set `headers` when your collector requires auth.
+- Environment variables supported: `OTEL_EXPORTER_OTLP_ENDPOINT`,
+  `OTEL_SERVICE_NAME`, `OTEL_EXPORTER_OTLP_PROTOCOL`.
+
+### Exported metrics (names + types)
+
+Model usage:
+
+- `Durar.tokens` (counter, attrs: `Durar.token`, `Durar.channel`,
+  `Durar.provider`, `Durar.model`)
+- `Durar.cost.usd` (counter, attrs: `Durar.channel`, `Durar.provider`,
+  `Durar.model`)
+- `Durar.run.duration_ms` (histogram, attrs: `Durar.channel`,
+  `Durar.provider`, `Durar.model`)
+- `Durar.context.tokens` (histogram, attrs: `Durar.context`,
+  `Durar.channel`, `Durar.provider`, `Durar.model`)
+
+Message flow:
+
+- `Durar.webhook.received` (counter, attrs: `Durar.channel`,
+  `Durar.webhook`)
+- `Durar.webhook.error` (counter, attrs: `Durar.channel`,
+  `Durar.webhook`)
+- `Durar.webhook.duration_ms` (histogram, attrs: `Durar.channel`,
+  `Durar.webhook`)
+- `Durar.message.queued` (counter, attrs: `Durar.channel`,
+  `Durar.source`)
+- `Durar.message.processed` (counter, attrs: `Durar.channel`,
+  `Durar.outcome`)
+- `Durar.message.duration_ms` (histogram, attrs: `Durar.channel`,
+  `Durar.outcome`)
+
+Queues + sessions:
+
+- `Durar.queue.lane.enqueue` (counter, attrs: `Durar.lane`)
+- `Durar.queue.lane.dequeue` (counter, attrs: `Durar.lane`)
+- `Durar.queue.depth` (histogram, attrs: `Durar.lane` or
+  `Durar.channel=heartbeat`)
+- `Durar.queue.wait_ms` (histogram, attrs: `Durar.lane`)
+- `Durar.session.state` (counter, attrs: `Durar.state`, `Durar.reason`)
+- `Durar.session.stuck` (counter, attrs: `Durar.state`)
+- `Durar.session.stuck_age_ms` (histogram, attrs: `Durar.state`)
+- `Durar.run.attempt` (counter, attrs: `Durar.attempt`)
+
+### Exported spans (names + key attributes)
+
+- `Durar.model.usage`
+  - `Durar.channel`, `Durar.provider`, `Durar.model`
+  - `Durar.sessionKey`, `Durar.sessionId`
+  - `Durar.tokens.*` (input/output/cache_read/cache_write/total)
+- `Durar.webhook.processed`
+  - `Durar.channel`, `Durar.webhook`, `Durar.chatId`
+- `Durar.webhook.error`
+  - `Durar.channel`, `Durar.webhook`, `Durar.chatId`,
+    `Durar.error`
+- `Durar.message.processed`
+  - `Durar.channel`, `Durar.outcome`, `Durar.chatId`,
+    `Durar.messageId`, `Durar.sessionKey`, `Durar.sessionId`,
+    `Durar.reason`
+- `Durar.session.stuck`
+  - `Durar.state`, `Durar.ageMs`, `Durar.queueDepth`,
+    `Durar.sessionKey`, `Durar.sessionId`
+
+### Sampling + flushing
+
+- Trace sampling: `diagnostics.otel.sampleRate` (0.0–1.0, root spans only).
+- Metric export interval: `diagnostics.otel.flushIntervalMs` (min 1000ms).
+
+### Protocol notes
+
+- OTLP/HTTP endpoints can be set via `diagnostics.otel.endpoint` or
+  `OTEL_EXPORTER_OTLP_ENDPOINT`.
+- If the endpoint already contains `/v1/traces` or `/v1/metrics`, it is used as-is.
+- If the endpoint already contains `/v1/logs`, it is used as-is for logs.
+- `diagnostics.otel.logs` enables OTLP log export for the main logger output.
+
+### Log export behavior
+
+- OTLP logs use the same structured records written to `logging.file`.
+- Respect `logging.level` (file log level). Console redaction does **not** apply
+  to OTLP logs.
+- High-volume installs should prefer OTLP collector sampling/filtering.
+
+## Troubleshooting tips
+
+- **Gateway not reachable?** Run `Durar doctor` first.
+- **Logs empty?** Check that the Gateway is running and writing to the file path
+  in `logging.file`.
+- **Need more detail?** Set `logging.level` to `debug` or `trace` and retry.
+
+## Related
+
+- [Gateway Logging Internals](/gateway/logging) — WS log styles, subsystem prefixes, and console capture
+- [Diagnostics](/gateway/configuration-reference#diagnostics) — OpenTelemetry export and cache trace config

package/docs/nav-tabs-underline.js

@@ -0,0 +1,100 @@
+(() => {
+  const NAV_TABS_SELECTOR = ".nav-tabs";
+  const ACTIVE_UNDERLINE_SELECTOR = ".nav-tabs-item > div.bg-primary";
+  const UNDERLINE_CLASS = "nav-tabs-underline";
+  const READY_CLASS = "nav-tabs-underline-ready";
+
+  let navTabs = null;
+  let navTabsObserver = null;
+  let lastX = null;
+  let lastWidth = null;
+
+  const ensureUnderline = (tabs) => {
+    let underline = tabs.querySelector(`.${UNDERLINE_CLASS}`);
+    if (!underline) {
+      underline = document.createElement("div");
+      underline.className = UNDERLINE_CLASS;
+      tabs.appendChild(underline);
+    }
+    return underline;
+  };
+
+  const getActiveTab = (tabs) => {
+    const activeUnderline = tabs.querySelector(ACTIVE_UNDERLINE_SELECTOR);
+    return activeUnderline?.closest(".nav-tabs-item") ?? null;
+  };
+
+  const updateUnderline = () => {
+    if (!navTabs) {
+      return;
+    }
+
+    ensureUnderline(navTabs);
+
+    const activeTab = getActiveTab(navTabs);
+    if (!activeTab) {
+      navTabs.classList.remove(READY_CLASS);
+      return;
+    }
+
+    const navRect = navTabs.getBoundingClientRect();
+    const tabRect = activeTab.getBoundingClientRect();
+    const left = tabRect.left - navRect.left;
+
+    navTabs.style.setProperty("--nav-tab-underline-x", `${left}px`);
+    navTabs.style.setProperty("--nav-tab-underline-width", `${tabRect.width}px`);
+    navTabs.classList.add(READY_CLASS);
+
+    lastX = left;
+    lastWidth = tabRect.width;
+  };
+
+  const scheduleUpdate = () => {
+    requestAnimationFrame(updateUnderline);
+  };
+
+  const setupNavTabsObserver = (tabs) => {
+    if (!tabs || tabs === navTabs) {
+      return;
+    }
+
+    navTabs = tabs;
+    ensureUnderline(navTabs);
+    if (lastX !== null && lastWidth !== null) {
+      navTabs.style.setProperty("--nav-tab-underline-x", `${lastX}px`);
+      navTabs.style.setProperty("--nav-tab-underline-width", `${lastWidth}px`);
+      navTabs.classList.add(READY_CLASS);
+    }
+    navTabsObserver?.disconnect();
+    navTabsObserver = new MutationObserver(scheduleUpdate);
+    navTabsObserver.observe(navTabs, {
+      subtree: true,
+      attributes: true,
+      attributeFilter: ["class"],
+    });
+
+    scheduleUpdate();
+  };
+
+  const setupObservers = () => {
+    const tabs = document.querySelector(NAV_TABS_SELECTOR);
+    if (tabs) {
+      setupNavTabsObserver(tabs);
+    }
+  };
+
+  const rootObserver = new MutationObserver(setupObservers);
+
+  if (document.readyState === "loading") {
+    document.addEventListener("DOMContentLoaded", () => {
+      setupObservers();
+      rootObserver.observe(document.body, { childList: true, subtree: true });
+    });
+  } else {
+    setupObservers();
+    rootObserver.observe(document.body, { childList: true, subtree: true });
+  }
+
+  window.addEventListener("resize", scheduleUpdate);
+  void document.fonts?.ready?.then(scheduleUpdate, () => {});
+})();

package/docs/network.md

@@ -0,0 +1,69 @@
+---
+summary: "Network hub: gateway surfaces, pairing, discovery, and security"
+read_when:
+  - You need the network architecture + security overview
+  - You are debugging local vs tailnet access or pairing
+  - You want the canonical list of networking docs
+title: "Network"
+---
+
+# Network hub
+
+This hub links the core docs for how Durar connects, pairs, and secures
+devices across localhost, LAN, and tailnet.
+
+## Core model
+
+Most operations flow through the Gateway (`Durar gateway`), a single long-running process that owns channel connections and the WebSocket control plane.
+
+- **Loopback first**: the Gateway WS defaults to `ws://127.0.0.1:18789`.
+  Non-loopback binds require a valid gateway auth path: shared-secret
+  token/password auth, or a correctly configured non-loopback
+  `trusted-proxy` deployment.
+- **One Gateway per host** is recommended. For isolation, run multiple gateways with isolated profiles and ports ([Multiple Gateways](/gateway/multiple-gateways)).
+- **Canvas host** is served on the same port as the Gateway (`/__Durar__/canvas/`, `/__Durar__/a2ui/`), protected by Gateway auth when bound beyond loopback.
+- **Remote access** is typically SSH tunnel or Tailscale VPN ([Remote Access](/gateway/remote)).
+
+Key references:
+
+- [Gateway architecture](/concepts/architecture)
+- [Gateway protocol](/gateway/protocol)
+- [Gateway runbook](/gateway)
+- [Web surfaces + bind modes](/web)
+
+## Pairing + identity
+
+- [Pairing overview (DM + nodes)](/channels/pairing)
+- [Gateway-owned node pairing](/gateway/pairing)
+- [Devices CLI (pairing + token rotation)](/cli/devices)
+- [Pairing CLI (DM approvals)](/cli/pairing)
+
+Local trust:
+
+- Direct local loopback connects can be auto-approved for pairing to keep
+  same-host UX smooth.
+- Durar also has a narrow backend/container-local self-connect path for
+  trusted shared-secret helper flows.
+- Tailnet and LAN clients, including same-host tailnet binds, still require
+  explicit pairing approval.
+
+## Discovery + transports
+
+- [Discovery & transports](/gateway/discovery)
+- [Bonjour / mDNS](/gateway/bonjour)
+- [Remote access (SSH)](/gateway/remote)
+- [Tailscale](/gateway/tailscale)
+
+## Nodes + transports
+
+- [Nodes overview](/nodes)
+- [Bridge protocol (legacy nodes, historical)](/gateway/bridge-protocol)
+- [Node runbook: iOS](/platforms/ios)
+- [Node runbook: Android](/platforms/android)
+
+## Security
+
+- [Security overview](/gateway/security)
+- [Gateway config reference](/gateway/configuration)
+- [Troubleshooting](/gateway/troubleshooting)
+- [Doctor](/gateway/doctor)