nodmix 2026.5.25

package/docs/gateway/opentelemetry.md

@@ -0,0 +1,404 @@
+---
+summary: "Export Nodmix diagnostics to any OpenTelemetry collector via the diagnostics-otel plugin (OTLP/HTTP)"
+title: "OpenTelemetry export"
+read_when:
+  - You want to send Nodmix model usage, message flow, or session metrics to an OpenTelemetry collector
+  - You are wiring traces, metrics, or logs into Grafana, Datadog, Honeycomb, New Relic, Tempo, or another OTLP backend
+  - You need the exact metric names, span names, or attribute shapes to build dashboards or alerts
+---
+
+Nodmix exports diagnostics through the official `diagnostics-otel` plugin
+using **OTLP/HTTP (protobuf)**. Any collector or backend that accepts OTLP/HTTP
+works without code changes. For local file logs and how to read them, see
+[Logging](/logging).
+
+## How it fits together
+
+- **Diagnostics events** are structured, in-process records emitted by the
+  Gateway and bundled plugins for model runs, message flow, sessions, queues,
+  and exec.
+- **`diagnostics-otel` plugin** subscribes to those events and exports them as
+  OpenTelemetry **metrics**, **traces**, and **logs** over OTLP/HTTP.
+- **Provider calls** receive a W3C `traceparent` header from Nodmix's
+  trusted model-call span context when the provider transport accepts custom
+  headers. Plugin-emitted trace context is not propagated.
+- Exporters only attach when both the diagnostics surface and the plugin are
+  enabled, so the in-process cost stays near zero by default.
+
+## Quick start
+
+For packaged installs, install the plugin first:
+
+```bash
+nodmix plugins install clawhub:@nodmix/diagnostics-otel
+```
+
+```json5
+{
+  plugins: {
+    allow: ["diagnostics-otel"],
+    entries: {
+      "diagnostics-otel": { enabled: true },
+    },
+  },
+  diagnostics: {
+    enabled: true,
+    otel: {
+      enabled: true,
+      endpoint: "http://otel-collector:4318",
+      protocol: "http/protobuf",
+      serviceName: "nodmix-gateway",
+      traces: true,
+      metrics: true,
+      logs: true,
+      sampleRate: 0.2,
+      flushIntervalMs: 60000,
+    },
+  },
+}
+```
+
+You can also enable the plugin from the CLI:
+
+```bash
+nodmix plugins enable diagnostics-otel
+```
+
+<Note>
+`protocol` currently supports `http/protobuf` only. `grpc` is ignored.
+</Note>
+
+## Signals exported
+
+| Signal      | What goes in it                                                                                                                                         |
+| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| **Metrics** | Counters and histograms for token usage, cost, run duration, message flow, Talk events, queue lanes, session state/recovery, exec, and memory pressure. |
+| **Traces**  | Spans for model usage, model calls, harness lifecycle, tool execution, exec, webhook/message processing, context assembly, and tool loops.              |
+| **Logs**    | Structured `logging.file` records exported over OTLP when `diagnostics.otel.logs` is enabled.                                                           |
+
+Toggle `traces`, `metrics`, and `logs` independently. All three default to on
+when `diagnostics.otel.enabled` is true.
+
+## Configuration reference
+
+```json5
+{
+  diagnostics: {
+    enabled: true,
+    otel: {
+      enabled: true,
+      endpoint: "http://otel-collector:4318",
+      tracesEndpoint: "http://otel-collector:4318/v1/traces",
+      metricsEndpoint: "http://otel-collector:4318/v1/metrics",
+      logsEndpoint: "http://otel-collector:4318/v1/logs",
+      protocol: "http/protobuf", // grpc is ignored
+      serviceName: "nodmix-gateway",
+      headers: { "x-collector-token": "..." },
+      traces: true,
+      metrics: true,
+      logs: true,
+      sampleRate: 0.2, // root-span sampler, 0.0..1.0
+      flushIntervalMs: 60000, // metric export interval (min 1000ms)
+      captureContent: {
+        enabled: false,
+        inputMessages: false,
+        outputMessages: false,
+        toolInputs: false,
+        toolOutputs: false,
+        systemPrompt: false,
+      },
+    },
+  },
+}
+```
+
+### Environment variables
+
+| Variable                                                                                                          | Purpose                                                                                                                                                                                                                                    |
+| ----------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| `OTEL_EXPORTER_OTLP_ENDPOINT`                                                                                     | Override `diagnostics.otel.endpoint`. If the value already contains `/v1/traces`, `/v1/metrics`, or `/v1/logs`, it is used as-is.                                                                                                          |
+| `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` / `OTEL_EXPORTER_OTLP_METRICS_ENDPOINT` / `OTEL_EXPORTER_OTLP_LOGS_ENDPOINT` | Signal-specific endpoint overrides used when the matching `diagnostics.otel.*Endpoint` config key is unset. Signal-specific config wins over signal-specific env, which wins over the shared endpoint.                                     |
+| `OTEL_SERVICE_NAME`                                                                                               | Override `diagnostics.otel.serviceName`.                                                                                                                                                                                                   |
+| `OTEL_EXPORTER_OTLP_PROTOCOL`                                                                                     | Override the wire protocol (only `http/protobuf` is honored today).                                                                                                                                                                        |
+| `OTEL_SEMCONV_STABILITY_OPT_IN`                                                                                   | Set to `gen_ai_latest_experimental` to emit the latest experimental GenAI span attribute (`gen_ai.provider.name`) instead of the legacy `gen_ai.system`. GenAI metrics always use bounded, low-cardinality semantic attributes regardless. |
+| `NODMIX_OTEL_PRELOADED`                                                                                         | Set to `1` when another preload or host process already registered the global OpenTelemetry SDK. The plugin then skips its own NodeSDK lifecycle but still wires diagnostic listeners and honors `traces`/`metrics`/`logs`.                |
+
+## Privacy and content capture
+
+Raw model/tool content is **not** exported by default. Spans carry bounded
+identifiers (channel, provider, model, error category, hash-only request ids)
+and never include prompt text, response text, tool inputs, tool outputs, or
+session keys.
+Talk metrics export only bounded event metadata such as mode, transport,
+provider, and event type. They do not include transcripts, audio payloads,
+session ids, turn ids, call ids, room ids, or handoff tokens.
+
+Outbound model requests may include a W3C `traceparent` header. That header is
+generated only from Nodmix-owned diagnostic trace context for the active model
+call. Existing caller-supplied `traceparent` headers are replaced, so plugins or
+custom provider options cannot spoof cross-service trace ancestry.
+
+Set `diagnostics.otel.captureContent.*` to `true` only when your collector and
+retention policy are approved for prompt, response, tool, or system-prompt
+text. Each subkey is opt-in independently:
+
+- `inputMessages` - user prompt content.
+- `outputMessages` - model response content.
+- `toolInputs` - tool argument payloads.
+- `toolOutputs` - tool result payloads.
+- `systemPrompt` - assembled system/developer prompt.
+
+When any subkey is enabled, model and tool spans get bounded, redacted
+`nodmix.content.*` attributes for that class only.
+
+## Sampling and flushing
+
+- **Traces:** `diagnostics.otel.sampleRate` (root-span only, `0.0` drops all,
+  `1.0` keeps all).
+- **Metrics:** `diagnostics.otel.flushIntervalMs` (minimum `1000`).
+- **Logs:** OTLP logs respect `logging.level` (file log level). They use the
+  diagnostic log-record redaction path, not console formatting. High-volume
+  installs should prefer OTLP collector sampling/filtering over local sampling.
+- **File-log correlation:** JSONL file logs include top-level `traceId`,
+  `spanId`, `parentSpanId`, and `traceFlags` when the log call carries a valid
+  diagnostic trace context, which lets log processors join local log lines with
+  exported spans.
+- **Request correlation:** Gateway HTTP requests and WebSocket frames create an
+  internal request trace scope. Logs and diagnostic events inside that scope
+  inherit the request trace by default, while agent run and model-call spans are
+  created as children so provider `traceparent` headers stay on the same trace.
+
+## Exported metrics
+
+### Model usage
+
+- `nodmix.tokens` (counter, attrs: `nodmix.token`, `nodmix.channel`, `nodmix.provider`, `nodmix.model`, `nodmix.agent`)
+- `nodmix.cost.usd` (counter, attrs: `nodmix.channel`, `nodmix.provider`, `nodmix.model`)
+- `nodmix.run.duration_ms` (histogram, attrs: `nodmix.channel`, `nodmix.provider`, `nodmix.model`)
+- `nodmix.context.tokens` (histogram, attrs: `nodmix.context`, `nodmix.channel`, `nodmix.provider`, `nodmix.model`)
+- `gen_ai.client.token.usage` (histogram, GenAI semantic-conventions metric, attrs: `gen_ai.token.type` = `input`/`output`, `gen_ai.provider.name`, `gen_ai.operation.name`, `gen_ai.request.model`)
+- `gen_ai.client.operation.duration` (histogram, seconds, GenAI semantic-conventions metric, attrs: `gen_ai.provider.name`, `gen_ai.operation.name`, `gen_ai.request.model`, optional `error.type`)
+- `nodmix.model_call.duration_ms` (histogram, attrs: `nodmix.provider`, `nodmix.model`, `nodmix.api`, `nodmix.transport`, plus `nodmix.errorCategory` and `nodmix.failureKind` on classified errors)
+- `nodmix.model_call.request_bytes` (histogram, UTF-8 byte size of the final model request payload; no raw payload content)
+- `nodmix.model_call.response_bytes` (histogram, UTF-8 byte size of streamed model response events; no raw response content)
+- `nodmix.model_call.time_to_first_byte_ms` (histogram, elapsed time before the first streamed response event)
+
+### Message flow
+
+- `nodmix.webhook.received` (counter, attrs: `nodmix.channel`, `nodmix.webhook`)
+- `nodmix.webhook.error` (counter, attrs: `nodmix.channel`, `nodmix.webhook`)
+- `nodmix.webhook.duration_ms` (histogram, attrs: `nodmix.channel`, `nodmix.webhook`)
+- `nodmix.message.queued` (counter, attrs: `nodmix.channel`, `nodmix.source`)
+- `nodmix.message.received` (counter, attrs: `nodmix.channel`, `nodmix.source`)
+- `nodmix.message.dispatch.started` (counter, attrs: `nodmix.channel`, `nodmix.source`)
+- `nodmix.message.dispatch.completed` (counter, attrs: `nodmix.channel`, `nodmix.outcome`, `nodmix.reason`, `nodmix.source`)
+- `nodmix.message.dispatch.duration_ms` (histogram, attrs: `nodmix.channel`, `nodmix.outcome`, `nodmix.reason`, `nodmix.source`)
+- `nodmix.message.processed` (counter, attrs: `nodmix.channel`, `nodmix.outcome`)
+- `nodmix.message.duration_ms` (histogram, attrs: `nodmix.channel`, `nodmix.outcome`)
+- `nodmix.message.delivery.started` (counter, attrs: `nodmix.channel`, `nodmix.delivery.kind`)
+- `nodmix.message.delivery.duration_ms` (histogram, attrs: `nodmix.channel`, `nodmix.delivery.kind`, `nodmix.outcome`, `nodmix.errorCategory`)
+
+### Talk
+
+- `nodmix.talk.event` (counter, attrs: `nodmix.talk.event_type`, `nodmix.talk.mode`, `nodmix.talk.transport`, `nodmix.talk.brain`, `nodmix.talk.provider`)
+- `nodmix.talk.event.duration_ms` (histogram, attrs: same as `nodmix.talk.event`; emitted when a Talk event reports duration)
+- `nodmix.talk.audio.bytes` (histogram, attrs: same as `nodmix.talk.event`; emitted for Talk audio frame events that report byte length)
+
+### Queues and sessions
+
+- `nodmix.queue.lane.enqueue` (counter, attrs: `nodmix.lane`)
+- `nodmix.queue.lane.dequeue` (counter, attrs: `nodmix.lane`)
+- `nodmix.queue.depth` (histogram, attrs: `nodmix.lane` or `nodmix.channel=heartbeat`)
+- `nodmix.queue.wait_ms` (histogram, attrs: `nodmix.lane`)
+- `nodmix.session.state` (counter, attrs: `nodmix.state`, `nodmix.reason`)
+- `nodmix.session.stuck` (counter, attrs: `nodmix.state`; emitted only for stale session bookkeeping with no active work)
+- `nodmix.session.stuck_age_ms` (histogram, attrs: `nodmix.state`; emitted only for stale session bookkeeping with no active work)
+- `nodmix.session.turn.created` (counter, attrs: `nodmix.agent`, `nodmix.channel`, `nodmix.trigger`)
+- `nodmix.session.recovery.requested` (counter, attrs: `nodmix.state`, `nodmix.action`, `nodmix.active_work_kind`, `nodmix.reason`)
+- `nodmix.session.recovery.completed` (counter, attrs: `nodmix.state`, `nodmix.action`, `nodmix.status`, `nodmix.active_work_kind`, `nodmix.reason`)
+- `nodmix.session.recovery.age_ms` (histogram, attrs: same as the matching recovery counter)
+- `nodmix.run.attempt` (counter, attrs: `nodmix.attempt`)
+
+### Session liveness telemetry
+
+`diagnostics.stuckSessionWarnMs` is the no-progress age threshold for session
+liveness diagnostics. A `processing` session does not age toward this threshold
+while Nodmix observes reply, tool, status, block, or ACP runtime progress.
+Typing keepalives are not counted as progress, so a silent model or harness can
+still be detected.
+
+Nodmix classifies sessions by the work it can still observe:
+
+- `session.long_running`: active embedded work, model calls, or tool calls are
+  still making progress.
+- `session.stalled`: active work exists, but the active run has not reported
+  recent progress. Stalled embedded runs stay observe-only at first, then
+  abort-drain after `diagnostics.stuckSessionAbortMs` with no progress so queued
+  turns behind the lane can resume. When unset, the abort threshold defaults to
+  the safer extended window of at least 5 minutes and 3x
+  `diagnostics.stuckSessionWarnMs`.
+- `session.stuck`: stale session bookkeeping with no active work. This releases
+  the affected session lane immediately.
+
+Recovery emits structured `session.recovery.requested` and
+`session.recovery.completed` events. Diagnostic session state is marked idle
+only after a mutating recovery outcome (`aborted` or `released`) and only if the
+same processing generation is still current.
+
+Only `session.stuck` emits the `nodmix.session.stuck` counter, the
+`nodmix.session.stuck_age_ms` histogram, and the `nodmix.session.stuck`
+span. Repeated `session.stuck` diagnostics back off while the session remains
+unchanged, so dashboards should alert on sustained increases rather than every
+heartbeat tick. For the config knob and defaults, see
+[Configuration reference](/gateway/configuration-reference#diagnostics).
+
+### Harness lifecycle
+
+- `nodmix.harness.duration_ms` (histogram, attrs: `nodmix.harness.id`, `nodmix.harness.plugin`, `nodmix.outcome`, `nodmix.harness.phase` on errors)
+
+### Exec
+
+- `nodmix.exec.duration_ms` (histogram, attrs: `nodmix.exec.target`, `nodmix.exec.mode`, `nodmix.outcome`, `nodmix.failureKind`)
+
+### Diagnostics internals (memory and tool loop)
+
+- `nodmix.memory.heap_used_bytes` (histogram, attrs: `nodmix.memory.kind`)
+- `nodmix.memory.rss_bytes` (histogram)
+- `nodmix.memory.pressure` (counter, attrs: `nodmix.memory.level`)
+- `nodmix.tool.loop.iterations` (counter, attrs: `nodmix.toolName`, `nodmix.outcome`)
+- `nodmix.tool.loop.duration_ms` (histogram, attrs: `nodmix.toolName`, `nodmix.outcome`)
+
+## Exported spans
+
+- `nodmix.model.usage`
+  - `nodmix.channel`, `nodmix.provider`, `nodmix.model`
+  - `nodmix.tokens.*` (input/output/cache_read/cache_write/total)
+  - `gen_ai.system` by default, or `gen_ai.provider.name` when the latest GenAI semantic conventions are opted in
+  - `gen_ai.request.model`, `gen_ai.operation.name`, `gen_ai.usage.*`
+- `nodmix.run`
+  - `nodmix.outcome`, `nodmix.channel`, `nodmix.provider`, `nodmix.model`, `nodmix.errorCategory`
+- `nodmix.model.call`
+  - `gen_ai.system` by default, or `gen_ai.provider.name` when the latest GenAI semantic conventions are opted in
+  - `gen_ai.request.model`, `gen_ai.operation.name`, `nodmix.provider`, `nodmix.model`, `nodmix.api`, `nodmix.transport`
+  - `nodmix.errorCategory` and optional `nodmix.failureKind` on errors
+  - `nodmix.model_call.request_bytes`, `nodmix.model_call.response_bytes`, `nodmix.model_call.time_to_first_byte_ms`
+  - `nodmix.provider.request_id_hash` (bounded SHA-based hash of the upstream provider request id; raw ids are not exported)
+- `nodmix.harness.run`
+  - `nodmix.harness.id`, `nodmix.harness.plugin`, `nodmix.outcome`, `nodmix.provider`, `nodmix.model`, `nodmix.channel`
+  - On completion: `nodmix.harness.result_classification`, `nodmix.harness.yield_detected`, `nodmix.harness.items.started`, `nodmix.harness.items.completed`, `nodmix.harness.items.active`
+  - On error: `nodmix.harness.phase`, `nodmix.errorCategory`, optional `nodmix.harness.cleanup_failed`
+- `nodmix.tool.execution`
+  - `gen_ai.tool.name`, `nodmix.toolName`, `nodmix.errorCategory`, `nodmix.tool.params.*`
+- `nodmix.exec`
+  - `nodmix.exec.target`, `nodmix.exec.mode`, `nodmix.outcome`, `nodmix.failureKind`, `nodmix.exec.command_length`, `nodmix.exec.exit_code`, `nodmix.exec.timed_out`
+- `nodmix.webhook.processed`
+  - `nodmix.channel`, `nodmix.webhook`
+- `nodmix.webhook.error`
+  - `nodmix.channel`, `nodmix.webhook`, `nodmix.error`
+- `nodmix.message.processed`
+  - `nodmix.channel`, `nodmix.outcome`, `nodmix.reason`
+- `nodmix.message.delivery`
+  - `nodmix.channel`, `nodmix.delivery.kind`, `nodmix.outcome`, `nodmix.errorCategory`, `nodmix.delivery.result_count`
+- `nodmix.session.stuck`
+  - `nodmix.state`, `nodmix.ageMs`, `nodmix.queueDepth`
+- `nodmix.context.assembled`
+  - `nodmix.prompt.size`, `nodmix.history.size`, `nodmix.context.tokens`, `nodmix.errorCategory` (no prompt, history, response, or session-key content)
+- `nodmix.tool.loop`
+  - `nodmix.toolName`, `nodmix.outcome`, `nodmix.iterations`, `nodmix.errorCategory` (no loop messages, params, or tool output)
+- `nodmix.memory.pressure`
+  - `nodmix.memory.level`, `nodmix.memory.heap_used_bytes`, `nodmix.memory.rss_bytes`
+
+When content capture is explicitly enabled, model and tool spans can also
+include bounded, redacted `nodmix.content.*` attributes for the specific
+content classes you opted into.
+
+## Diagnostic event catalog
+
+The events below back the metrics and spans above. Plugins can also subscribe
+to them directly without OTLP export.
+
+**Model usage**
+
+- `model.usage` - tokens, cost, duration, context, provider/model/channel,
+  session ids. `usage` is provider/turn accounting for cost and telemetry;
+  `context.used` is the current prompt/context snapshot and can be lower than
+  provider `usage.total` when cached input or tool-loop calls are involved.
+
+**Message flow**
+
+- `webhook.received` / `webhook.processed` / `webhook.error`
+- `message.queued` / `message.processed`
+- `message.delivery.started` / `message.delivery.completed` / `message.delivery.error`
+
+**Queue and session**
+
+- `queue.lane.enqueue` / `queue.lane.dequeue`
+- `session.state` / `session.long_running` / `session.stalled` / `session.stuck`
+- `run.attempt` / `run.progress`
+- `diagnostic.heartbeat` (aggregate counters: webhooks/queue/session)
+
+**Harness lifecycle**
+
+- `harness.run.started` / `harness.run.completed` / `harness.run.error` -
+  per-run lifecycle for the agent harness. Includes `harnessId`, optional
+  `pluginId`, provider/model/channel, and run id. Completion adds
+  `durationMs`, `outcome`, optional `resultClassification`, `yieldDetected`,
+  and `itemLifecycle` counts. Errors add `phase`
+  (`prepare`/`start`/`send`/`resolve`/`cleanup`), `errorCategory`, and
+  optional `cleanupFailed`.
+
+**Exec**
+
+- `exec.process.completed` - terminal outcome, duration, target, mode, exit
+  code, and failure kind. Command text and working directories are not
+  included.
+
+## Without an exporter
+
+You can keep diagnostics events available to plugins or custom sinks without
+running `diagnostics-otel`:
+
+```json5
+{
+  diagnostics: { enabled: true },
+}
+```
+
+For targeted debug output without raising `logging.level`, use diagnostics
+flags. Flags are case-insensitive and support wildcards (e.g. `telegram.*` or
+`*`):
+
+```json5
+{
+  diagnostics: { flags: ["telegram.http"] },
+}
+```
+
+Or as a one-off env override:
+
+```bash
+NODMIX_DIAGNOSTICS=telegram.http,telegram.payload nodmix gateway
+```
+
+Flag output goes to the standard log file (`logging.file`) and is still
+redacted by `logging.redactSensitive`. Full guide:
+[Diagnostics flags](/diagnostics/flags).
+
+## Disable
+
+```json5
+{
+  diagnostics: { otel: { enabled: false } },
+}
+```
+
+You can also leave `diagnostics-otel` out of `plugins.allow`, or run
+`nodmix plugins disable diagnostics-otel`.
+
+## Related
+
+- [Logging](/logging) - file logs, console output, CLI tailing, and the Control UI Logs tab
+- [Gateway logging internals](/gateway/logging) - WS log styles, subsystem prefixes, and console capture
+- [Diagnostics flags](/diagnostics/flags) - targeted debug-log flags
+- [Diagnostics export](/gateway/diagnostics) - operator support-bundle tool (separate from OTEL export)
+- [Configuration reference](/gateway/configuration-reference#diagnostics) - full `diagnostics.*` field reference

package/docs/gateway/operator-scopes.md

@@ -0,0 +1,111 @@
+---
+summary: "Operator roles, scopes, and approval-time checks for Gateway clients"
+read_when:
+  - Debugging missing operator scope errors
+  - Reviewing device or node pairing approvals
+  - Adding or classifying Gateway RPC methods
+title: "Operator scopes"
+---
+
+Operator scopes define what a Gateway client may do after it authenticates.
+They are a control-plane guardrail inside one trusted Gateway operator domain,
+not hostile multi-tenant isolation. If you need strong separation between
+people, teams, or machines, run separate Gateways under separate OS users or
+hosts.
+
+Related: [Security](/gateway/security), [Gateway protocol](/gateway/protocol),
+[Gateway pairing](/gateway/pairing), [Devices CLI](/cli/devices).
+
+## Roles
+
+Gateway WebSocket clients connect with one role:
+
+- `operator`: control-plane clients such as CLI, Control UI, automation, and
+  trusted helper processes.
+- `node`: capability hosts such as macOS, iOS, Android, or headless nodes that
+  expose commands through `node.invoke`.
+
+Operator RPC methods require the `operator` role. Node-originated methods
+require the `node` role.
+
+## Scope levels
+
+| Scope                   | Meaning                                                                                                                                                                               |
+| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `operator.read`         | Read-only status, lists, catalog, logs, session reads, and other non-mutating control-plane calls.                                                                                    |
+| `operator.write`        | Normal mutating operator actions such as sending messages, invoking tools, updating talk/voice settings, and node command relay. Also satisfies `operator.read`.                      |
+| `operator.admin`        | Administrative control-plane access. Satisfies every `operator.*` scope. Required for config mutation, updates, native hooks, sensitive reserved namespaces, and high-risk approvals. |
+| `operator.pairing`      | Device and node pairing management, including listing, approving, rejecting, removing, rotating, and revoking pairing records or device tokens.                                       |
+| `operator.approvals`    | Exec and plugin approval APIs.                                                                                                                                                        |
+| `operator.talk.secrets` | Reading Talk configuration with secrets included.                                                                                                                                     |
+
+Unknown future `operator.*` scopes require an exact match unless the caller has
+`operator.admin`.
+
+## Method scope is only the first gate
+
+Each Gateway RPC has a least-privilege method scope. That method scope decides
+whether the request can reach the handler. Some handlers then apply stricter
+approval-time checks based on the concrete thing being approved or mutated.
+
+Examples:
+
+- `device.pair.approve` is reachable with `operator.pairing`, but approving an
+  operator device can only mint or preserve scopes the caller already holds.
+- `node.pair.approve` is reachable with `operator.pairing`, then derives extra
+  approval scopes from the pending node command list.
+- `chat.send` is normally a write-scoped method, but persistent `/config set`
+  and `/config unset` require `operator.admin` at command level.
+
+This lets lower-scope operators perform low-risk pairing actions without making
+all pairing approval admin-only.
+
+## Device pairing approvals
+
+Device pairing records are the durable source of approved roles and scopes.
+Already paired devices do not get broader access silently: reconnects that ask
+for a broader role or broader scopes create a new pending upgrade request.
+
+When approving a device request:
+
+- A request with no operator role does not need operator token scope approval.
+- A request for `operator.read`, `operator.write`, `operator.approvals`,
+  `operator.pairing`, or `operator.talk.secrets` requires the caller to hold
+  those scopes, or `operator.admin`.
+- A request for `operator.admin` requires `operator.admin`.
+- A repair request with no explicit scopes can inherit the existing operator
+  token scopes. If that existing token is admin-scoped, approval still requires
+  `operator.admin`.
+
+For paired-device token sessions, management is self-scoped unless the caller
+also has `operator.admin`: non-admin callers see only their own pairing entries,
+can approve or reject only their own pending request, and can rotate, revoke, or
+remove only their own device entry.
+
+## Node pairing approvals
+
+Legacy `node.pair.*` uses a separate Gateway-owned node pairing store. WS nodes
+use device pairing with `role: node`, but the same approval-level vocabulary
+applies.
+
+`node.pair.approve` uses the pending request command list to derive additional
+required scopes:
+
+- Commandless request: `operator.pairing`
+- Non-exec node commands: `operator.pairing` + `operator.write`
+- `system.run`, `system.run.prepare`, or `system.which`:
+  `operator.pairing` + `operator.admin`
+
+Node pairing establishes identity and trust. It does not replace the node's
+own `system.run` exec approval policy.
+
+## Shared-secret auth
+
+Shared gateway token/password auth is treated as trusted operator access for
+that Gateway. OpenAI-compatible HTTP surfaces, `/tools/invoke`, and HTTP session
+history endpoints restore the normal full operator default scope set for
+shared-secret bearer auth, even if a caller sends narrower declared scopes.
+
+Identity-bearing modes, such as trusted proxy auth or private-ingress `none`,
+can still honor explicit declared scopes. Use separate Gateways for real trust
+boundary separation.