nodmix 2026.5.25

package/docs/providers/inferrs.md

@@ -0,0 +1,272 @@
+---
+summary: "Run Nodmix through inferrs (OpenAI-compatible local server)"
+read_when:
+  - You want to run Nodmix against a local inferrs server
+  - You are serving Gemma or another model through inferrs
+  - You need the exact Nodmix compat flags for inferrs
+title: "Inferrs"
+---
+
+[inferrs](https://github.com/ericcurtin/inferrs) can serve local models behind an OpenAI-compatible `/v1` API. Nodmix works with `inferrs` through the generic `openai-completions` path.
+
+| Property           | Value                                                              |
+| ------------------ | ------------------------------------------------------------------ |
+| Provider id        | `inferrs` (custom; configure under `models.providers.inferrs`)     |
+| Plugin             | none — `inferrs` is not a bundled Nodmix provider plugin         |
+| Auth env var       | Optional. Any value works if your inferrs server has no auth       |
+| API                | OpenAI-compatible (`openai-completions`)                           |
+| Suggested base URL | `http://127.0.0.1:8080/v1` (or wherever your inferrs server lives) |
+
+<Note>
+  `inferrs` is currently best treated as a custom self-hosted OpenAI-compatible backend, not a dedicated Nodmix provider plugin. You configure it through `models.providers.inferrs` rather than an onboarding choice flag. If you need a true bundled plugin with auto-discovery, see [SGLang](/providers/sglang) or [vLLM](/providers/vllm).
+</Note>
+
+## Getting started
+
+<Steps>
+  <Step title="Start inferrs with a model">
+    ```bash
+    inferrs serve google/gemma-4-E2B-it \
+      --host 127.0.0.1 \
+      --port 8080 \
+      --device metal
+    ```
+  </Step>
+  <Step title="Verify the server is reachable">
+    ```bash
+    curl http://127.0.0.1:8080/health
+    curl http://127.0.0.1:8080/v1/models
+    ```
+  </Step>
+  <Step title="Add an Nodmix provider entry">
+    Add an explicit provider entry and point your default model at it. See the full config example below.
+  </Step>
+</Steps>
+
+## Full config example
+
+This example uses Gemma 4 on a local `inferrs` server.
+
+```json5
+{
+  agents: {
+    defaults: {
+      model: { primary: "inferrs/google/gemma-4-E2B-it" },
+      models: {
+        "inferrs/google/gemma-4-E2B-it": {
+          alias: "Gemma 4 (inferrs)",
+        },
+      },
+    },
+  },
+  models: {
+    mode: "merge",
+    providers: {
+      inferrs: {
+        baseUrl: "http://127.0.0.1:8080/v1",
+        apiKey: "inferrs-local",
+        api: "openai-completions",
+        models: [
+          {
+            id: "google/gemma-4-E2B-it",
+            name: "Gemma 4 E2B (inferrs)",
+            reasoning: false,
+            input: ["text"],
+            cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
+            contextWindow: 131072,
+            maxTokens: 4096,
+            compat: {
+              requiresStringContent: true,
+            },
+          },
+        ],
+      },
+    },
+  },
+}
+```
+
+## On-demand startup
+
+Inferrs can also be started by Nodmix only when an `inferrs/...` model is
+selected. Add `localService` to the same provider entry:
+
+```json5
+{
+  models: {
+    providers: {
+      inferrs: {
+        baseUrl: "http://127.0.0.1:8080/v1",
+        apiKey: "inferrs-local",
+        api: "openai-completions",
+        timeoutSeconds: 300,
+        localService: {
+          command: "/opt/homebrew/bin/inferrs",
+          args: [
+            "serve",
+            "google/gemma-4-E2B-it",
+            "--host",
+            "127.0.0.1",
+            "--port",
+            "8080",
+            "--device",
+            "metal",
+          ],
+          healthUrl: "http://127.0.0.1:8080/v1/models",
+          readyTimeoutMs: 180000,
+          idleStopMs: 0,
+        },
+        models: [
+          {
+            id: "google/gemma-4-E2B-it",
+            name: "Gemma 4 E2B (inferrs)",
+            reasoning: false,
+            input: ["text"],
+            cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
+            contextWindow: 131072,
+            maxTokens: 4096,
+            compat: {
+              requiresStringContent: true,
+            },
+          },
+        ],
+      },
+    },
+  },
+}
+```
+
+`command` must be absolute. Use `which inferrs` on the Gateway host and put that
+path in config. For the full field reference, see
+[Local model services](/gateway/local-model-services).
+
+## Advanced configuration
+
+<AccordionGroup>
+  <Accordion title="Why requiresStringContent matters">
+    Some `inferrs` Chat Completions routes accept only string
+    `messages[].content`, not structured content-part arrays.
+
+    <Warning>
+    If Nodmix runs fail with an error like:
+
+    ```text
+    messages[1].content: invalid type: sequence, expected a string
+    ```
+
+    set `compat.requiresStringContent: true` in your model entry.
+    </Warning>
+
+    ```json5
+    compat: {
+      requiresStringContent: true
+    }
+    ```
+
+    Nodmix will flatten pure text content parts into plain strings before sending
+    the request.
+
+  </Accordion>
+
+  <Accordion title="Gemma and tool-schema caveat">
+    Some current `inferrs` + Gemma combinations accept small direct
+    `/v1/chat/completions` requests but still fail on full Nodmix agent-runtime
+    turns.
+
+    If that happens, try this first:
+
+    ```json5
+    compat: {
+      requiresStringContent: true,
+      supportsTools: false
+    }
+    ```
+
+    That disables Nodmix's tool schema surface for the model and can reduce prompt
+    pressure on stricter local backends.
+
+    If tiny direct requests still work but normal Nodmix agent turns continue to
+    crash inside `inferrs`, the remaining issue is usually upstream model/server
+    behavior rather than Nodmix's transport layer.
+
+  </Accordion>
+
+  <Accordion title="Manual smoke test">
+    Once configured, test both layers:
+
+    ```bash
+    curl http://127.0.0.1:8080/v1/chat/completions \
+      -H 'content-type: application/json' \
+      -d '{"model":"google/gemma-4-E2B-it","messages":[{"role":"user","content":"What is 2 + 2?"}],"stream":false}'
+    ```
+
+    ```bash
+    nodmix infer model run \
+      --model inferrs/google/gemma-4-E2B-it \
+      --prompt "What is 2 + 2? Reply with one short sentence." \
+      --json
+    ```
+
+    If the first command works but the second fails, check the troubleshooting section below.
+
+  </Accordion>
+
+  <Accordion title="Proxy-style behavior">
+    `inferrs` is treated as a proxy-style OpenAI-compatible `/v1` backend, not a
+    native OpenAI endpoint.
+
+    - Native OpenAI-only request shaping does not apply here
+    - No `service_tier`, no Responses `store`, no prompt-cache hints, and no
+      OpenAI reasoning-compat payload shaping
+    - Hidden Nodmix attribution headers (`originator`, `version`, `User-Agent`)
+      are not injected on custom `inferrs` base URLs
+
+  </Accordion>
+</AccordionGroup>
+
+## Troubleshooting
+
+<AccordionGroup>
+  <Accordion title="curl /v1/models fails">
+    `inferrs` is not running, not reachable, or not bound to the expected
+    host/port. Make sure the server is started and listening on the address you
+    configured.
+  </Accordion>
+
+  <Accordion title="messages[].content expected a string">
+    Set `compat.requiresStringContent: true` in the model entry. See the
+    `requiresStringContent` section above for details.
+  </Accordion>
+
+  <Accordion title="Direct /v1/chat/completions calls pass but nodmix infer model run fails">
+    Try setting `compat.supportsTools: false` to disable the tool schema surface.
+    See the Gemma tool-schema caveat above.
+  </Accordion>
+
+  <Accordion title="inferrs still crashes on larger agent turns">
+    If Nodmix no longer gets schema errors but `inferrs` still crashes on larger
+    agent turns, treat it as an upstream `inferrs` or model limitation. Reduce
+    prompt pressure or switch to a different local backend or model.
+  </Accordion>
+</AccordionGroup>
+
+<Tip>
+For general help, see [Troubleshooting](/help/troubleshooting) and [FAQ](/help/faq).
+</Tip>
+
+## Related
+
+<CardGroup cols={2}>
+  <Card title="Local models" href="/gateway/local-models" icon="server">
+    Running Nodmix against local model servers.
+  </Card>
+  <Card title="Local model services" href="/gateway/local-model-services" icon="play">
+    Starting local model servers on demand for configured providers.
+  </Card>
+  <Card title="Gateway troubleshooting" href="/gateway/troubleshooting#local-openai-compatible-backend-passes-direct-probes-but-agent-runs-fail" icon="wrench">
+    Debugging local OpenAI-compatible backends that pass probes but fail agent runs.
+  </Card>
+  <Card title="Model selection" href="/concepts/model-providers" icon="layers">
+    Overview of all providers, model refs, and failover behavior.
+  </Card>
+</CardGroup>

package/docs/providers/inworld.md

@@ -0,0 +1,120 @@
+---
+summary: "Inworld streaming text-to-speech for Nodmix replies"
+read_when:
+  - You want Inworld speech synthesis for outbound replies
+  - You need PCM telephony or OGG_OPUS voice-note output from Inworld
+title: "Inworld"
+---
+
+Inworld is a streaming text-to-speech (TTS) provider. In Nodmix it
+synthesizes outbound reply audio (MP3 by default, OGG_OPUS for voice notes)
+and PCM audio for telephony channels such as Voice Call.
+
+Nodmix posts to Inworld's streaming TTS endpoint, concatenates the
+returned base64 audio chunks into a single buffer, and hands the result to
+the standard reply-audio pipeline.
+
+| Property      | Value                                                           |
+| ------------- | --------------------------------------------------------------- |
+| Provider id   | `inworld`                                                       |
+| Plugin        | bundled, `enabledByDefault: true`                               |
+| Contract      | `speechProviders` (TTS only)                                    |
+| Auth env var  | `INWORLD_API_KEY` (HTTP Basic, Base64 dashboard credential)     |
+| Base URL      | `https://api.inworld.ai`                                        |
+| Default voice | `Sarah`                                                         |
+| Default model | `inworld-tts-1.5-max`                                           |
+| Output        | MP3 (default), OGG_OPUS (voice notes), PCM 22050 Hz (telephony) |
+| Website       | [inworld.ai](https://inworld.ai)                                |
+| Docs          | [docs.inworld.ai/tts/tts](https://docs.inworld.ai/tts/tts)      |
+
+## Getting started
+
+<Steps>
+  <Step title="Set your API key">
+    Copy the credential from your Inworld dashboard (Workspace > API Keys)
+    and set it as an env var. The value is sent verbatim as the HTTP Basic
+    credential, so do not Base64-encode it again or convert it to a bearer
+    token.
+
+    ```
+    INWORLD_API_KEY=<base64-credential-from-dashboard>
+    ```
+
+  </Step>
+  <Step title="Select Inworld in messages.tts">
+    ```json5
+    {
+      messages: {
+        tts: {
+          auto: "always",
+          provider: "inworld",
+          providers: {
+            inworld: {
+              voiceId: "Sarah",
+              modelId: "inworld-tts-1.5-max",
+            },
+          },
+        },
+      },
+    }
+    ```
+  </Step>
+  <Step title="Send a message">
+    Send a reply through any connected channel. Nodmix synthesizes the
+    audio with Inworld and delivers it as MP3 (or OGG_OPUS when the channel
+    expects a voice note).
+  </Step>
+</Steps>
+
+## Configuration options
+
+| Option        | Path                                         | Description                                                       |
+| ------------- | -------------------------------------------- | ----------------------------------------------------------------- |
+| `apiKey`      | `messages.tts.providers.inworld.apiKey`      | Base64 dashboard credential. Falls back to `INWORLD_API_KEY`.     |
+| `baseUrl`     | `messages.tts.providers.inworld.baseUrl`     | Override Inworld API base URL (default `https://api.inworld.ai`). |
+| `voiceId`     | `messages.tts.providers.inworld.voiceId`     | Voice identifier (default `Sarah`).                               |
+| `modelId`     | `messages.tts.providers.inworld.modelId`     | TTS model id (default `inworld-tts-1.5-max`).                     |
+| `temperature` | `messages.tts.providers.inworld.temperature` | Sampling temperature `0..2` (optional).                           |
+
+## Notes
+
+<AccordionGroup>
+  <Accordion title="Authentication">
+    Inworld uses HTTP Basic auth with a single Base64-encoded credential
+    string. Copy it verbatim from the Inworld dashboard. The provider sends
+    it as `Authorization: Basic <apiKey>` without any further encoding, so
+    do not Base64-encode it yourself and do not pass a bearer-style token.
+    See [TTS auth notes](/tools/tts#inworld-primary) for the same callout.
+  </Accordion>
+  <Accordion title="Models">
+    Supported model ids: `inworld-tts-1.5-max` (default),
+    `inworld-tts-1.5-mini`, `inworld-tts-1-max`, `inworld-tts-1`.
+  </Accordion>
+  <Accordion title="Audio outputs">
+    Replies use MP3 by default. When the channel target is `voice-note`
+    Nodmix asks Inworld for `OGG_OPUS` so the audio plays as a native
+    voice bubble. Telephony synthesis uses raw `PCM` at 22050 Hz to feed
+    the telephony bridge.
+  </Accordion>
+  <Accordion title="Custom endpoints">
+    Override the API host with `messages.tts.providers.inworld.baseUrl`.
+    Trailing slashes are stripped before requests are sent.
+  </Accordion>
+</AccordionGroup>
+
+## Related
+
+<CardGroup cols={2}>
+  <Card title="Text-to-speech" href="/tools/tts" icon="waveform-lines">
+    TTS overview, providers, and `messages.tts` config.
+  </Card>
+  <Card title="Configuration" href="/gateway/configuration" icon="gear">
+    Full config reference including `messages.tts` settings.
+  </Card>
+  <Card title="Providers" href="/providers" icon="grid">
+    All bundled Nodmix providers.
+  </Card>
+  <Card title="Troubleshooting" href="/help/troubleshooting" icon="wrench">
+    Common issues and debugging steps.
+  </Card>
+</CardGroup>

package/docs/providers/kilocode.md

@@ -0,0 +1,135 @@
+---
+summary: "Use Kilo Gateway's unified API to access many models in Nodmix"
+title: "Kilo Gateway"
+read_when:
+  - You want a single API key for many LLMs
+  - You want to run models via Kilo Gateway in Nodmix
+---
+
+Kilo Gateway provides a **unified API** that routes requests to many models behind a single
+endpoint and API key. It is OpenAI-compatible, so most OpenAI SDKs work by switching the base URL.
+
+| Property | Value                              |
+| -------- | ---------------------------------- |
+| Provider | `kilocode`                         |
+| Auth     | `KILOCODE_API_KEY`                 |
+| API      | OpenAI-compatible                  |
+| Base URL | `https://api.kilo.ai/api/gateway/` |
+
+## Getting started
+
+<Steps>
+  <Step title="Create an account">
+    Go to [app.kilo.ai](https://app.kilo.ai), sign in or create an account, then navigate to API Keys and generate a new key.
+  </Step>
+  <Step title="Run onboarding">
+    ```bash
+    nodmix onboard --auth-choice kilocode-api-key
+    ```
+
+    Or set the environment variable directly:
+
+    ```bash
+    export KILOCODE_API_KEY="<your-kilocode-api-key>" # pragma: allowlist secret
+    ```
+
+  </Step>
+  <Step title="Verify the model is available">
+    ```bash
+    nodmix models list --provider kilocode
+    ```
+  </Step>
+</Steps>
+
+## Default model
+
+The default model is `kilocode/kilo/auto`, a provider-owned smart-routing
+model managed by Kilo Gateway.
+
+<Note>
+Nodmix treats `kilocode/kilo/auto` as the stable default ref, but does not
+publish a source-backed task-to-upstream-model mapping for that route. Exact
+upstream routing behind `kilocode/kilo/auto` is owned by Kilo Gateway, not
+hard-coded in Nodmix.
+</Note>
+
+## Built-in catalog
+
+Nodmix dynamically discovers available models from the Kilo Gateway at startup. Use
+`/models kilocode` to see the full list of models available with your account.
+
+Any model available on the gateway can be used with the `kilocode/` prefix:
+
+| Model ref                                | Notes                              |
+| ---------------------------------------- | ---------------------------------- |
+| `kilocode/kilo/auto`                     | Default — smart routing            |
+| `kilocode/anthropic/claude-sonnet-4`     | Anthropic via Kilo                 |
+| `kilocode/openai/gpt-5.5`                | OpenAI via Kilo                    |
+| `kilocode/google/gemini-3.1-pro-preview` | Google via Kilo                    |
+| ...and many more                         | Use `/models kilocode` to list all |
+
+<Tip>
+At startup, Nodmix queries `GET https://api.kilo.ai/api/gateway/models` and merges
+discovered models ahead of the static fallback catalog. The bundled fallback always
+includes `kilocode/kilo/auto` (`Kilo Auto`) with `input: ["text", "image"]`,
+`reasoning: true`, `contextWindow: 1000000`, and `maxTokens: 128000`.
+</Tip>
+
+## Config example
+
+```json5
+{
+  env: { KILOCODE_API_KEY: "<your-kilocode-api-key>" }, // pragma: allowlist secret
+  agents: {
+    defaults: {
+      model: { primary: "kilocode/kilo/auto" },
+    },
+  },
+}
+```
+
+<AccordionGroup>
+  <Accordion title="Transport and compatibility">
+    Kilo Gateway is documented in source as OpenRouter-compatible, so it stays on
+    the proxy-style OpenAI-compatible path rather than native OpenAI request shaping.
+
+    - Gemini-backed Kilo refs stay on the proxy-Gemini path, so Nodmix keeps
+      Gemini thought-signature sanitation there without enabling native Gemini
+      replay validation or bootstrap rewrites.
+    - Kilo Gateway uses a Bearer token with your API key under the hood.
+
+  </Accordion>
+
+  <Accordion title="Stream wrapper and reasoning">
+    Kilo's shared stream wrapper adds the provider app header and normalizes
+    proxy reasoning payloads for supported concrete model refs.
+
+    <Warning>
+    `kilocode/kilo/auto` and other proxy-reasoning-unsupported hints skip reasoning
+    injection. If you need reasoning support, use a concrete model ref such as
+    `kilocode/anthropic/claude-sonnet-4`.
+    </Warning>
+
+  </Accordion>
+
+  <Accordion title="Troubleshooting">
+    - If model discovery fails at startup, Nodmix falls back to the bundled static catalog containing `kilocode/kilo/auto`.
+    - Confirm your API key is valid and that your Kilo account has the desired models enabled.
+    - When the Gateway runs as a daemon, ensure `KILOCODE_API_KEY` is available to that process (for example in `~/.nodmix/.env` or via `env.shellEnv`).
+
+  </Accordion>
+</AccordionGroup>
+
+## Related
+
+<CardGroup cols={2}>
+  <Card title="Model selection" href="/concepts/model-providers" icon="layers">
+    Choosing providers, model refs, and failover behavior.
+  </Card>
+  <Card title="Configuration reference" href="/gateway/configuration-reference" icon="gear">
+    Full Nodmix configuration reference.
+  </Card>
+  <Card title="Kilo Gateway" href="https://app.kilo.ai" icon="arrow-up-right-from-square">
+    Kilo Gateway dashboard, API keys, and account management.
+  </Card>
+</CardGroup>