nodmix 2026.5.25

package/docs/gateway/index.md

@@ -0,0 +1,383 @@
+---
+summary: "Runbook for the Gateway service, lifecycle, and operations"
+read_when:
+  - Running or debugging the gateway process
+title: "Gateway runbook"
+---
+
+Use this page for day-1 startup and day-2 operations of the Gateway service.
+
+<CardGroup cols={2}>
+  <Card title="Deep troubleshooting" icon="siren" href="/gateway/troubleshooting">
+    Symptom-first diagnostics with exact command ladders and log signatures.
+  </Card>
+  <Card title="Configuration" icon="sliders" href="/gateway/configuration">
+    Task-oriented setup guide + full configuration reference.
+  </Card>
+  <Card title="Secrets management" icon="key-round" href="/gateway/secrets">
+    SecretRef contract, runtime snapshot behavior, and migrate/reload operations.
+  </Card>
+  <Card title="Secrets plan contract" icon="shield-check" href="/gateway/secrets-plan-contract">
+    Exact `secrets apply` target/path rules and ref-only auth-profile behavior.
+  </Card>
+</CardGroup>
+
+## 5-minute local startup
+
+<Steps>
+  <Step title="Start the Gateway">
+
+```bash
+nodmix gateway --port 18789
+# debug/trace mirrored to stdio
+nodmix gateway --port 18789 --verbose
+# force-kill listener on selected port, then start
+nodmix gateway --force
+```
+
+  </Step>
+
+  <Step title="Verify service health">
+
+```bash
+nodmix gateway status
+nodmix status
+nodmix logs --follow
+```
+
+Healthy baseline: `Runtime: running`, `Connectivity probe: ok`, and `Capability: ...` that matches what you expect. Use `nodmix gateway status --require-rpc` when you need read-scope RPC proof, not just reachability.
+
+  </Step>
+
+  <Step title="Validate channel readiness">
+
+```bash
+nodmix channels status --probe
+```
+
+With a reachable gateway this runs live per-account channel probes and optional audits.
+If the gateway is unreachable, the CLI falls back to config-only channel summaries instead
+of live probe output.
+
+  </Step>
+</Steps>
+
+<Note>
+Gateway config reload watches the active config file path (resolved from profile/state defaults, or `NODMIX_CONFIG_PATH` when set).
+Default mode is `gateway.reload.mode="hybrid"`.
+After the first successful load, the running process serves the active in-memory config snapshot; successful reload swaps that snapshot atomically.
+</Note>
+
+## Runtime model
+
+- One always-on process for routing, control plane, and channel connections.
+- Single multiplexed port for:
+  - WebSocket control/RPC
+  - HTTP APIs (`/v1/models`, `/v1/embeddings`, `/v1/chat/completions`, `/v1/responses`, `/tools/invoke`)
+  - Plugin HTTP routes, such as optional `/api/v1/admin/rpc`
+  - Control UI and hooks
+- Default bind mode: `loopback`.
+- Auth is required by default. Shared-secret setups use
+  `gateway.auth.token` / `gateway.auth.password` (or
+  `NODMIX_GATEWAY_TOKEN` / `NODMIX_GATEWAY_PASSWORD`), and non-loopback
+  reverse-proxy setups can use `gateway.auth.mode: "trusted-proxy"`.
+
+## OpenAI-compatible endpoints
+
+Nodmix's highest-leverage compatibility surface is now:
+
+- `GET /v1/models`
+- `GET /v1/models/{id}`
+- `POST /v1/embeddings`
+- `POST /v1/chat/completions`
+- `POST /v1/responses`
+
+Why this set matters:
+
+- Most Open WebUI, LobeChat, and LibreChat integrations probe `/v1/models` first.
+- Many RAG and memory pipelines expect `/v1/embeddings`.
+- Agent-native clients increasingly prefer `/v1/responses`.
+
+Planning note:
+
+- `/v1/models` is agent-first: it returns `nodmix`, `nodmix/default`, and `nodmix/<agentId>`.
+- `nodmix/default` is the stable alias that always maps to the configured default agent.
+- Use `x-nodmix-model` when you want a backend provider/model override; otherwise the selected agent's normal model and embedding setup stays in control.
+
+All of these run on the main Gateway port and use the same trusted operator auth boundary as the rest of the Gateway HTTP API.
+
+Admin HTTP RPC (`POST /api/v1/admin/rpc`) is a separate, default-off plugin route for host tooling that cannot use WebSocket RPC. See [Admin HTTP RPC](/plugins/admin-http-rpc).
+
+### Port and bind precedence
+
+| Setting      | Resolution order                                              |
+| ------------ | ------------------------------------------------------------- |
+| Gateway port | `--port` → `NODMIX_GATEWAY_PORT` → `gateway.port` → `18789` |
+| Bind mode    | CLI/override → `gateway.bind` → `loopback`                    |
+
+Installed gateway services record the resolved `--port` in supervisor metadata. After changing `gateway.port`, run `nodmix doctor --fix` or `nodmix gateway install --force` so launchd/systemd/schtasks starts the process on the new port.
+
+Gateway startup uses the same effective port and bind when it seeds local
+Control UI origins for non-loopback binds. For example, `--bind lan --port 3000`
+seeds `http://localhost:3000` and `http://127.0.0.1:3000` before runtime
+validation runs. Add any remote browser origins, such as HTTPS proxy URLs, to
+`gateway.controlUi.allowedOrigins` explicitly.
+
+### Hot reload modes
+
+| `gateway.reload.mode` | Behavior                                   |
+| --------------------- | ------------------------------------------ |
+| `off`                 | No config reload                           |
+| `hot`                 | Apply only hot-safe changes                |
+| `restart`             | Restart on reload-required changes         |
+| `hybrid` (default)    | Hot-apply when safe, restart when required |
+
+## Operator command set
+
+```bash
+nodmix gateway status
+nodmix gateway status --deep   # adds a system-level service scan
+nodmix gateway status --json
+nodmix gateway install
+nodmix gateway restart
+nodmix gateway stop
+nodmix secrets reload
+nodmix logs --follow
+nodmix doctor
+```
+
+`gateway status --deep` is for extra service discovery (LaunchDaemons/systemd system
+units/schtasks), not a deeper RPC health probe.
+
+## Multiple gateways (same host)
+
+Most installs should run one gateway per machine. A single gateway can host multiple
+agents and channels.
+
+You only need multiple gateways when you intentionally want isolation or a rescue bot.
+
+Useful checks:
+
+```bash
+nodmix gateway status --deep
+nodmix gateway probe
+```
+
+What to expect:
+
+- `gateway status --deep` can report `Other gateway-like services detected (best effort)`
+  and print cleanup hints when stale launchd/systemd/schtasks installs are still around.
+- `gateway probe` can warn about `multiple reachable gateways` when more than one target
+  answers.
+- If that is intentional, isolate ports, config/state, and workspace roots per gateway.
+
+Checklist per instance:
+
+- Unique `gateway.port`
+- Unique `NODMIX_CONFIG_PATH`
+- Unique `NODMIX_STATE_DIR`
+- Unique `agents.defaults.workspace`
+
+Example:
+
+```bash
+NODMIX_CONFIG_PATH=~/.nodmix/a.json NODMIX_STATE_DIR=~/.nodmix-a nodmix gateway --port 19001
+NODMIX_CONFIG_PATH=~/.nodmix/b.json NODMIX_STATE_DIR=~/.nodmix-b nodmix gateway --port 19002
+```
+
+Detailed setup: [/gateway/multiple-gateways](/gateway/multiple-gateways).
+
+## Remote access
+
+Preferred: Tailscale/VPN.
+Fallback: SSH tunnel.
+
+```bash
+ssh -N -L 18789:127.0.0.1:18789 user@host
+```
+
+Then connect clients locally to `ws://127.0.0.1:18789`.
+
+<Warning>
+SSH tunnels do not bypass gateway auth. For shared-secret auth, clients still
+must send `token`/`password` even over the tunnel. For identity-bearing modes,
+the request still has to satisfy that auth path.
+</Warning>
+
+See: [Remote Gateway](/gateway/remote), [Authentication](/gateway/authentication), [Tailscale](/gateway/tailscale).
+
+## Supervision and service lifecycle
+
+Use supervised runs for production-like reliability.
+
+<Tabs>
+  <Tab title="macOS (launchd)">
+
+```bash
+nodmix gateway install
+nodmix gateway status
+nodmix gateway restart
+nodmix gateway stop
+```
+
+Use `nodmix gateway restart` for restarts. Do not chain `nodmix gateway stop` and `nodmix gateway start` as a restart substitute.
+
+On macOS, `gateway stop` uses `launchctl bootout` by default — this removes the LaunchAgent from the current boot session without persisting a disable, so KeepAlive auto-recovery still works after unexpected crashes and `gateway start` re-enables cleanly. To persistently suppress auto-respawn across reboots, pass `--disable`: `nodmix gateway stop --disable`.
+
+LaunchAgent labels are `ai.nodmix.gateway` (default) or `ai.nodmix.<profile>` (named profile). `nodmix doctor` audits and repairs service config drift.
+
+  </Tab>
+
+  <Tab title="Linux (systemd user)">
+
+```bash
+nodmix gateway install
+systemctl --user enable --now nodmix-gateway[-<profile>].service
+nodmix gateway status
+```
+
+For persistence after logout, enable lingering:
+
+```bash
+sudo loginctl enable-linger <user>
+```
+
+Manual user-unit example when you need a custom install path:
+
+```ini
+[Unit]
+Description=Nodmix Gateway
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+ExecStart=/usr/local/bin/nodmix gateway --port 18789
+Restart=always
+RestartSec=5
+TimeoutStopSec=30
+TimeoutStartSec=30
+SuccessExitStatus=0 143
+KillMode=control-group
+
+[Install]
+WantedBy=default.target
+```
+
+  </Tab>
+
+  <Tab title="Windows (native)">
+
+```powershell
+nodmix gateway install
+nodmix gateway status --json
+nodmix gateway restart
+nodmix gateway stop
+```
+
+Native Windows managed startup uses a Scheduled Task named `Nodmix Gateway`
+(or `Nodmix Gateway (<profile>)` for named profiles). If Scheduled Task
+creation is denied, Nodmix falls back to a per-user Startup-folder launcher
+that points at `gateway.cmd` inside the state directory.
+
+  </Tab>
+
+  <Tab title="Linux (system service)">
+
+Use a system unit for multi-user/always-on hosts.
+
+```bash
+sudo systemctl daemon-reload
+sudo systemctl enable --now nodmix-gateway[-<profile>].service
+```
+
+Use the same service body as the user unit, but install it under
+`/etc/systemd/system/nodmix-gateway[-<profile>].service` and adjust
+`ExecStart=` if your `nodmix` binary lives elsewhere.
+
+Do not also let `nodmix doctor --fix` install a user-level gateway service for the same profile/port. Doctor refuses that automatic install when it finds a system-level Nodmix gateway service; use `NODMIX_SERVICE_REPAIR_POLICY=external` when the system unit owns the lifecycle.
+
+  </Tab>
+</Tabs>
+
+## Dev profile quick path
+
+```bash
+nodmix --dev setup
+nodmix --dev gateway --allow-unconfigured
+nodmix --dev status
+```
+
+Defaults include isolated state/config and base gateway port `19001`.
+
+## Protocol quick reference (operator view)
+
+- First client frame must be `connect`.
+- Gateway returns `hello-ok` snapshot (`presence`, `health`, `stateVersion`, `uptimeMs`, limits/policy).
+- `hello-ok.features.methods` / `events` are a conservative discovery list, not
+  a generated dump of every callable helper route.
+- Requests: `req(method, params)` → `res(ok/payload|error)`.
+- Common events include `connect.challenge`, `agent`, `chat`,
+  `session.message`, `session.operation`, `session.tool`, `sessions.changed`,
+  `presence`, `tick`, `health`, `heartbeat`, pairing/approval lifecycle events,
+  and `shutdown`.
+
+Agent runs are two-stage:
+
+1. Immediate accepted ack (`status:"accepted"`)
+2. Final completion response (`status:"ok"|"error"`), with streamed `agent` events in between.
+
+See full protocol docs: [Gateway Protocol](/gateway/protocol).
+
+## Operational checks
+
+### Liveness
+
+- Open WS and send `connect`.
+- Expect `hello-ok` response with snapshot.
+
+### Readiness
+
+```bash
+nodmix gateway status
+nodmix channels status --probe
+nodmix health
+```
+
+### Gap recovery
+
+Events are not replayed. On sequence gaps, refresh state (`health`, `system-presence`) before continuing.
+
+## Common failure signatures
+
+| Signature                                                      | Likely issue                                                                    |
+| -------------------------------------------------------------- | ------------------------------------------------------------------------------- |
+| `refusing to bind gateway ... without auth`                    | Non-loopback bind without a valid gateway auth path                             |
+| `another gateway instance is already listening` / `EADDRINUSE` | Port conflict                                                                   |
+| `Gateway start blocked: set gateway.mode=local`                | Config set to remote mode, or local-mode stamp is missing from a damaged config |
+| `unauthorized` during connect                                  | Auth mismatch between client and gateway                                        |
+
+For full diagnosis ladders, use [Gateway Troubleshooting](/gateway/troubleshooting).
+
+## Safety guarantees
+
+- Gateway protocol clients fail fast when Gateway is unavailable (no implicit direct-channel fallback).
+- Invalid/non-connect first frames are rejected and closed.
+- Graceful shutdown emits `shutdown` event before socket close.
+
+---
+
+Related:
+
+- [Troubleshooting](/gateway/troubleshooting)
+- [Background Process](/gateway/background-process)
+- [Configuration](/gateway/configuration)
+- [Health](/gateway/health)
+- [Doctor](/gateway/doctor)
+- [Authentication](/gateway/authentication)
+
+## Related
+
+- [Configuration](/gateway/configuration)
+- [Gateway troubleshooting](/gateway/troubleshooting)
+- [Remote access](/gateway/remote)
+- [Secrets management](/gateway/secrets)

package/docs/gateway/local-model-services.md

@@ -0,0 +1,205 @@
+---
+summary: "Start local model servers on demand before Nodmix model requests"
+read_when:
+  - You want Nodmix to start a local model server only when its model is selected
+  - You run ds4, inferrs, vLLM, llama.cpp, MLX, or another OpenAI-compatible local server
+  - You need to control cold start, readiness, and idle shutdown for local providers
+title: "Local model services"
+---
+
+`models.providers.<id>.localService` lets Nodmix start a provider-owned local
+model server on demand. It is provider-level config: when the selected model
+belongs to that provider, Nodmix probes the service, starts the process if the
+endpoint is down, waits for readiness, then sends the model request.
+
+Use it for local servers that are expensive to keep running all day, or for
+manual setups where model selection should be enough to bring the backend up.
+
+## How it works
+
+1. A model request resolves to a configured provider.
+2. If that provider has `localService`, Nodmix probes `healthUrl`.
+3. If the probe succeeds, Nodmix uses the existing server.
+4. If the probe fails, Nodmix starts `command` with `args`.
+5. Nodmix polls readiness until `readyTimeoutMs` expires.
+6. The model request is sent through the normal provider transport.
+7. If Nodmix started the process and `idleStopMs` is positive, the process is
+   stopped after the last in-flight request has been idle for that long.
+
+Nodmix does not install launchd, systemd, Docker, or a daemon for this. The
+server is a child process of the Nodmix process that first needed it.
+
+## Config shape
+
+```json5
+{
+  models: {
+    providers: {
+      local: {
+        baseUrl: "http://127.0.0.1:8000/v1",
+        apiKey: "local-model",
+        api: "openai-completions",
+        timeoutSeconds: 300,
+        localService: {
+          command: "/absolute/path/to/server",
+          args: ["--host", "127.0.0.1", "--port", "8000"],
+          cwd: "/absolute/path/to/working-dir",
+          env: { LOCAL_MODEL_CACHE: "/absolute/path/to/cache" },
+          healthUrl: "http://127.0.0.1:8000/v1/models",
+          readyTimeoutMs: 180000,
+          idleStopMs: 0,
+        },
+        models: [
+          {
+            id: "my-local-model",
+            name: "My Local Model",
+            reasoning: false,
+            input: ["text"],
+            cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
+            contextWindow: 131072,
+            maxTokens: 8192,
+          },
+        ],
+      },
+    },
+  },
+}
+```
+
+## Fields
+
+- `command`: absolute executable path. Shell lookup is not used.
+- `args`: process arguments. No shell expansion, pipes, globbing, or quoting
+  rules are applied.
+- `cwd`: optional working directory for the process.
+- `env`: optional environment variables merged over the Nodmix process
+  environment.
+- `healthUrl`: readiness URL. If omitted, Nodmix appends `/models` to
+  `baseUrl`, so `http://127.0.0.1:8000/v1` becomes
+  `http://127.0.0.1:8000/v1/models`.
+- `readyTimeoutMs`: startup readiness deadline. Default: `120000`.
+- `idleStopMs`: idle shutdown delay for Nodmix-started processes. `0` or
+  omitted keeps the process alive until Nodmix exits.
+
+## Inferrs example
+
+Inferrs is a custom OpenAI-compatible `/v1` backend, so the same local service
+API works with the `inferrs` provider entry.
+
+```json5
+{
+  agents: {
+    defaults: {
+      model: { primary: "inferrs/google/gemma-4-E2B-it" },
+    },
+  },
+  models: {
+    mode: "merge",
+    providers: {
+      inferrs: {
+        baseUrl: "http://127.0.0.1:8080/v1",
+        apiKey: "inferrs-local",
+        api: "openai-completions",
+        timeoutSeconds: 300,
+        localService: {
+          command: "/opt/homebrew/bin/inferrs",
+          args: [
+            "serve",
+            "google/gemma-4-E2B-it",
+            "--host",
+            "127.0.0.1",
+            "--port",
+            "8080",
+            "--device",
+            "metal",
+          ],
+          healthUrl: "http://127.0.0.1:8080/v1/models",
+          readyTimeoutMs: 180000,
+          idleStopMs: 0,
+        },
+        models: [
+          {
+            id: "google/gemma-4-E2B-it",
+            name: "Gemma 4 E2B (inferrs)",
+            reasoning: false,
+            input: ["text"],
+            cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
+            contextWindow: 131072,
+            maxTokens: 4096,
+            compat: {
+              requiresStringContent: true,
+            },
+          },
+        ],
+      },
+    },
+  },
+}
+```
+
+Replace `command` with the result of `which inferrs` on the machine running
+Nodmix.
+
+## ds4 example
+
+For the full setup, context sizing guidance, and verification commands, see
+[ds4](/providers/ds4).
+
+```json5
+{
+  models: {
+    providers: {
+      ds4: {
+        baseUrl: "http://127.0.0.1:18000/v1",
+        apiKey: "ds4-local",
+        api: "openai-completions",
+        timeoutSeconds: 300,
+        localService: {
+          command: "<DS4_DIR>/ds4-server",
+          args: [
+            "--model",
+            "<DS4_DIR>/ds4flash.gguf",
+            "--host",
+            "127.0.0.1",
+            "--port",
+            "18000",
+            "--ctx",
+            "32768",
+            "--tokens",
+            "128",
+          ],
+          cwd: "<DS4_DIR>",
+          healthUrl: "http://127.0.0.1:18000/v1/models",
+          readyTimeoutMs: 300000,
+          idleStopMs: 0,
+        },
+        models: [],
+      },
+    },
+  },
+}
+```
+
+## Operational notes
+
+- One Nodmix process manages the child it started. Another Nodmix process
+  that sees the same health URL already live will reuse it without adopting it.
+- Startup is serialized per provider command and argument set, so concurrent
+  requests do not spawn duplicate servers for the same config.
+- Active streaming responses hold a lease; idle shutdown waits until response
+  body handling is complete.
+- Use `timeoutSeconds` on slow local providers so cold starts and long generations
+  do not hit the default model request timeout.
+- Use an explicit `healthUrl` if your server exposes readiness somewhere other
+  than `/v1/models`.
+
+## Related
+
+<CardGroup cols={2}>
+  <Card title="Local models" href="/gateway/local-models" icon="server">
+    Local model setup, provider choices, and safety guidance.
+  </Card>
+  <Card title="Inferrs" href="/providers/inferrs" icon="cpu">
+    Run Nodmix through the inferrs OpenAI-compatible local server.
+  </Card>
+</CardGroup>