nodmix 2026.5.25

package/docs/providers/glm.md

@@ -0,0 +1,137 @@
+---
+summary: "GLM model family overview and how to use it in Nodmix"
+read_when:
+  - You want GLM models in Nodmix
+  - You need the model naming convention and setup
+title: "GLM (Zhipu)"
+---
+
+GLM is a model family (not a company) available through the [Z.AI](https://z.ai) platform. In Nodmix, GLM models are accessed through the bundled `zai` provider with refs like `zai/glm-5.1`.
+
+| Property            | Value                                                                       |
+| ------------------- | --------------------------------------------------------------------------- |
+| Provider id         | `zai`                                                                       |
+| Plugin              | bundled, `enabledByDefault: true`                                           |
+| Auth env vars       | `ZAI_API_KEY` or `Z_AI_API_KEY`                                             |
+| Onboarding choices  | `zai-api-key`, `zai-coding-global`, `zai-coding-cn`, `zai-global`, `zai-cn` |
+| API                 | OpenAI-compatible                                                           |
+| Default base URL    | `https://api.z.ai/api/paas/v4`                                              |
+| Suggested default   | `zai/glm-5.1`                                                               |
+| Default image model | `zai/glm-4.6v`                                                              |
+
+## Getting started
+
+<Steps>
+  <Step title="Choose an auth route and run onboarding">
+    Pick the onboarding choice that matches your Z.AI plan and region. The generic `zai-api-key` choice auto-detects the matching endpoint from the key shape; use the explicit regional choices when you want to force a specific Coding Plan or general API surface.
+
+    | Auth choice         | Best for                                            |
+    | ------------------- | --------------------------------------------------- |
+    | `zai-api-key`       | Generic API key with endpoint auto-detection        |
+    | `zai-coding-global` | Coding Plan users (global)                          |
+    | `zai-coding-cn`     | Coding Plan users (China region)                    |
+    | `zai-global`        | General API (global)                                |
+    | `zai-cn`            | General API (China region)                          |
+
+    <CodeGroup>
+
+```bash Auto-detect
+nodmix onboard --auth-choice zai-api-key
+```
+
+```bash Coding Plan (global)
+nodmix onboard --auth-choice zai-coding-global
+```
+
+```bash Coding Plan (China)
+nodmix onboard --auth-choice zai-coding-cn
+```
+
+```bash General API (global)
+nodmix onboard --auth-choice zai-global
+```
+
+```bash General API (China)
+nodmix onboard --auth-choice zai-cn
+```
+
+    </CodeGroup>
+
+  </Step>
+  <Step title="Set GLM as the default model">
+    ```bash
+    nodmix config set agents.defaults.model.primary "zai/glm-5.1"
+    ```
+  </Step>
+  <Step title="Verify models are available">
+    ```bash
+    nodmix models list --provider zai
+    ```
+  </Step>
+</Steps>
+
+## Config example
+
+```json5
+{
+  env: { ZAI_API_KEY: "sk-..." },
+  agents: { defaults: { model: { primary: "zai/glm-5.1" } } },
+}
+```
+
+<Tip>
+  `zai-api-key` lets Nodmix detect the matching Z.AI endpoint from the key shape and apply the correct base URL automatically. Use the explicit regional choices when you want to pin a specific Coding Plan or general API surface.
+</Tip>
+
+## Built-in catalog
+
+The bundled `zai` provider seeds 13 GLM model refs. All entries support reasoning unless marked otherwise; `glm-5v-turbo` and `glm-4.6v` accept image input as well as text.
+
+| Model ref            | Notes                                              |
+| -------------------- | -------------------------------------------------- |
+| `zai/glm-5.1`        | Default model. Reasoning, text only, 202k context. |
+| `zai/glm-5`          | Reasoning, text only, 202k context.                |
+| `zai/glm-5-turbo`    | Reasoning, text only, 202k context.                |
+| `zai/glm-5v-turbo`   | Reasoning, text + image, 202k context.             |
+| `zai/glm-4.7`        | Reasoning, text only, 204k context.                |
+| `zai/glm-4.7-flash`  | Reasoning, text only, 200k context.                |
+| `zai/glm-4.7-flashx` | Reasoning, text only.                              |
+| `zai/glm-4.6`        | Reasoning, text only.                              |
+| `zai/glm-4.6v`       | Reasoning, text + image. Default image model.      |
+| `zai/glm-4.5`        | Reasoning, text only.                              |
+| `zai/glm-4.5-air`    | Reasoning, text only.                              |
+| `zai/glm-4.5-flash`  | Reasoning, text only.                              |
+| `zai/glm-4.5v`       | Reasoning, text + image.                           |
+
+<Note>
+  GLM versions and availability can change. Run `nodmix models list --provider zai` to see the catalog rows known to your installed version, and check Z.AI's docs for newly added or deprecated models.
+</Note>
+
+## Advanced configuration
+
+<AccordionGroup>
+  <Accordion title="Endpoint auto-detection">
+    When you use the `zai-api-key` auth choice, Nodmix inspects the key shape to determine the correct Z.AI base URL. Explicit regional choices (`zai-coding-global`, `zai-coding-cn`, `zai-global`, `zai-cn`) override auto-detection and pin the endpoint directly.
+  </Accordion>
+
+  <Accordion title="Provider details">
+    GLM models are served by the `zai` runtime provider. For full provider configuration, regional endpoints, and additional capabilities, see the [Z.AI provider page](/providers/zai).
+  </Accordion>
+</AccordionGroup>
+
+## Related
+
+<CardGroup cols={2}>
+  <Card title="Z.AI provider" href="/providers/zai" icon="server">
+    Full Z.AI provider configuration and regional endpoints.
+  </Card>
+  <Card title="Model providers" href="/concepts/model-providers" icon="layers">
+    Choosing providers, model refs, and failover behavior.
+  </Card>
+  <Card title="Thinking modes" href="/tools/thinking" icon="brain">
+    `/think` levels for the reasoning-capable GLM family.
+  </Card>
+  <Card title="Models FAQ" href="/help/faq-models" icon="circle-question">
+    Auth profiles, switching models, and resolving "no profile" errors.
+  </Card>
+</CardGroup>

package/docs/providers/google.md

@@ -0,0 +1,472 @@
+---
+summary: "Google Gemini setup (API key + OAuth, image generation, media understanding, TTS, web search)"
+title: "Google (Gemini)"
+read_when:
+  - You want to use Google Gemini models with Nodmix
+  - You need the API key or OAuth auth flow
+---
+
+The Google plugin provides access to Gemini models through Google AI Studio, plus
+image generation, media understanding (image/audio/video), text-to-speech, and web search via
+Gemini Grounding.
+
+- Provider: `google`
+- Auth: `GEMINI_API_KEY` or `GOOGLE_API_KEY`
+- API: Google Gemini API
+- Runtime option: provider/model `agentRuntime.id: "google-gemini-cli"`
+  reuses Gemini CLI OAuth while keeping model refs canonical as `google/*`.
+
+## Getting started
+
+Choose your preferred auth method and follow the setup steps.
+
+<Tabs>
+  <Tab title="API key">
+    **Best for:** standard Gemini API access through Google AI Studio.
+
+    <Steps>
+      <Step title="Run onboarding">
+        ```bash
+        nodmix onboard --auth-choice gemini-api-key
+        ```
+
+        Or pass the key directly:
+
+        ```bash
+        nodmix onboard --non-interactive \
+          --mode local \
+          --auth-choice gemini-api-key \
+          --gemini-api-key "$GEMINI_API_KEY"
+        ```
+      </Step>
+      <Step title="Set a default model">
+        ```json5
+        {
+          agents: {
+            defaults: {
+              model: { primary: "google/gemini-3.1-pro-preview" },
+            },
+          },
+        }
+        ```
+      </Step>
+      <Step title="Verify the model is available">
+        ```bash
+        nodmix models list --provider google
+        ```
+      </Step>
+    </Steps>
+
+    <Tip>
+    The environment variables `GEMINI_API_KEY` and `GOOGLE_API_KEY` are both accepted. Use whichever you already have configured.
+    </Tip>
+
+  </Tab>
+
+  <Tab title="Gemini CLI (OAuth)">
+    **Best for:** reusing an existing Gemini CLI login via PKCE OAuth instead of a separate API key.
+
+    <Warning>
+    The `google-gemini-cli` provider is an unofficial integration. Some users
+    report account restrictions when using OAuth this way. Use at your own risk.
+    </Warning>
+
+    <Steps>
+      <Step title="Install the Gemini CLI">
+        The local `gemini` command must be available on `PATH`.
+
+        ```bash
+        # Homebrew
+        brew install gemini-cli
+
+        # or npm
+        npm install -g @google/gemini-cli
+        ```
+
+        Nodmix supports both Homebrew installs and global npm installs, including
+        common Windows/npm layouts.
+      </Step>
+      <Step title="Log in via OAuth">
+        ```bash
+        nodmix models auth login --provider google-gemini-cli --set-default
+        ```
+      </Step>
+      <Step title="Verify the model is available">
+        ```bash
+        nodmix models list --provider google
+        ```
+      </Step>
+    </Steps>
+
+    - Default model: `google/gemini-3.1-pro-preview`
+    - Runtime: `google-gemini-cli`
+    - Alias: `gemini-cli`
+
+    Gemini 3.1 Pro's Gemini API model id is `gemini-3.1-pro-preview`. Nodmix accepts the shorter `google/gemini-3.1-pro` as a convenience alias and normalizes it before provider calls.
+
+    **Environment variables:**
+
+    - `NODMIX_GEMINI_OAUTH_CLIENT_ID`
+    - `NODMIX_GEMINI_OAUTH_CLIENT_SECRET`
+
+    (Or the `GEMINI_CLI_*` variants.)
+
+    <Note>
+    If Gemini CLI OAuth requests fail after login, set `GOOGLE_CLOUD_PROJECT` or
+    `GOOGLE_CLOUD_PROJECT_ID` on the gateway host and retry.
+    </Note>
+
+    <Note>
+    If login fails before the browser flow starts, make sure the local `gemini`
+    command is installed and on `PATH`.
+    </Note>
+
+    `google-gemini-cli/*` model refs are legacy compatibility aliases. New
+    configs should use `google/*` model refs plus the `google-gemini-cli`
+    runtime when they want local Gemini CLI execution.
+
+  </Tab>
+</Tabs>
+
+## Capabilities
+
+| Capability             | Supported                     |
+| ---------------------- | ----------------------------- |
+| Chat completions       | Yes                           |
+| Image generation       | Yes                           |
+| Music generation       | Yes                           |
+| Text-to-speech         | Yes                           |
+| Realtime voice         | Yes (Google Live API)         |
+| Image understanding    | Yes                           |
+| Audio transcription    | Yes                           |
+| Video understanding    | Yes                           |
+| Web search (Grounding) | Yes                           |
+| Thinking/reasoning     | Yes (Gemini 2.5+ / Gemini 3+) |
+| Gemma 4 models         | Yes                           |
+
+## Web search
+
+The bundled `gemini` web-search provider uses Gemini Google Search grounding.
+Configure a dedicated search key under `plugins.entries.google.config.webSearch`,
+or let it reuse `models.providers.google.apiKey` after `GEMINI_API_KEY`:
+
+```json5
+{
+  plugins: {
+    entries: {
+      google: {
+        config: {
+          webSearch: {
+            apiKey: "AIza...", // optional if GEMINI_API_KEY or models.providers.google.apiKey is set
+            baseUrl: "https://generativelanguage.googleapis.com/v1beta", // falls back to models.providers.google.baseUrl
+            model: "gemini-2.5-flash",
+          },
+        },
+      },
+    },
+  },
+}
+```
+
+Credential precedence is dedicated `webSearch.apiKey`, then `GEMINI_API_KEY`,
+then `models.providers.google.apiKey`. `webSearch.baseUrl` is optional and
+exists for operator proxies or compatible Gemini API endpoints; when omitted,
+Gemini web search reuses `models.providers.google.baseUrl`. See
+[Gemini search](/tools/gemini-search) for the provider-specific tool behavior.
+
+<Tip>
+Gemini 3 models use `thinkingLevel` rather than `thinkingBudget`. Nodmix maps
+Gemini 3, Gemini 3.1, and `gemini-*-latest` alias reasoning controls to
+`thinkingLevel` so default/low-latency runs do not send disabled
+`thinkingBudget` values.
+
+`/think adaptive` keeps Google's dynamic thinking semantics instead of choosing
+a fixed Nodmix level. Gemini 3 and Gemini 3.1 omit a fixed `thinkingLevel` so
+Google can choose the level; Gemini 2.5 sends Google's dynamic sentinel
+`thinkingBudget: -1`.
+
+Gemma 4 models (for example `gemma-4-26b-a4b-it`) support thinking mode. Nodmix
+rewrites `thinkingBudget` to a supported Google `thinkingLevel` for Gemma 4.
+Setting thinking to `off` preserves thinking disabled instead of mapping to
+`MINIMAL`.
+</Tip>
+
+## Image generation
+
+The bundled `google` image-generation provider defaults to
+`google/gemini-3.1-flash-image-preview`.
+
+- Also supports `google/gemini-3-pro-image-preview`
+- Generate: up to 4 images per request
+- Edit mode: enabled, up to 5 input images
+- Geometry controls: `size`, `aspectRatio`, and `resolution`
+
+To use Google as the default image provider:
+
+```json5
+{
+  agents: {
+    defaults: {
+      imageGenerationModel: {
+        primary: "google/gemini-3.1-flash-image-preview",
+      },
+    },
+  },
+}
+```
+
+<Note>
+See [Image Generation](/tools/image-generation) for shared tool parameters, provider selection, and failover behavior.
+</Note>
+
+## Video generation
+
+The bundled `google` plugin also registers video generation through the shared
+`video_generate` tool.
+
+- Default video model: `google/veo-3.1-fast-generate-preview`
+- Modes: text-to-video, image-to-video, and single-video reference flows
+- Supports `aspectRatio` (`16:9`, `9:16`) and `resolution` (`720P`, `1080P`); audio output is not supported by Veo today
+- Supported durations: **4, 6, or 8 seconds** (other values snap to the nearest allowed value)
+
+To use Google as the default video provider:
+
+```json5
+{
+  agents: {
+    defaults: {
+      videoGenerationModel: {
+        primary: "google/veo-3.1-fast-generate-preview",
+      },
+    },
+  },
+}
+```
+
+<Note>
+See [Video Generation](/tools/video-generation) for shared tool parameters, provider selection, and failover behavior.
+</Note>
+
+## Music generation
+
+The bundled `google` plugin also registers music generation through the shared
+`music_generate` tool.
+
+- Default music model: `google/lyria-3-clip-preview`
+- Also supports `google/lyria-3-pro-preview`
+- Prompt controls: `lyrics` and `instrumental`
+- Output format: `mp3` by default, plus `wav` on `google/lyria-3-pro-preview`
+- Reference inputs: up to 10 images
+- Session-backed runs detach through the shared task/status flow, including `action: "status"`
+
+To use Google as the default music provider:
+
+```json5
+{
+  agents: {
+    defaults: {
+      musicGenerationModel: {
+        primary: "google/lyria-3-clip-preview",
+      },
+    },
+  },
+}
+```
+
+<Note>
+See [Music Generation](/tools/music-generation) for shared tool parameters, provider selection, and failover behavior.
+</Note>
+
+## Text-to-speech
+
+The bundled `google` speech provider uses the Gemini API TTS path with
+`gemini-3.1-flash-tts-preview`.
+
+- Default voice: `Kore`
+- Auth: `messages.tts.providers.google.apiKey`, `models.providers.google.apiKey`, `GEMINI_API_KEY`, or `GOOGLE_API_KEY`
+- Output: WAV for regular TTS attachments, Opus for voice-note targets, PCM for Talk/telephony
+- Voice-note output: Google PCM is wrapped as WAV and transcoded to 48 kHz Opus with `ffmpeg`
+
+Google's batch Gemini TTS path returns generated audio in the completed
+`generateContent` response. For lowest-latency spoken conversations, use the
+Google realtime voice provider backed by the Gemini Live API instead of batch
+TTS.
+
+To use Google as the default TTS provider:
+
+```json5
+{
+  messages: {
+    tts: {
+      auto: "always",
+      provider: "google",
+      providers: {
+        google: {
+          model: "gemini-3.1-flash-tts-preview",
+          voiceName: "Kore",
+          audioProfile: "Speak professionally with a calm tone.",
+        },
+      },
+    },
+  },
+}
+```
+
+Gemini API TTS uses natural-language prompting for style control. Set
+`audioProfile` to prepend a reusable style prompt before the spoken text. Set
+`speakerName` when your prompt text refers to a named speaker.
+
+Gemini API TTS also accepts expressive square-bracket audio tags in the text,
+such as `[whispers]` or `[laughs]`. To keep tags out of the visible chat reply
+while sending them to TTS, put them inside a `[[tts:text]]...[[/tts:text]]`
+block:
+
+```text
+Here is the clean reply text.
+
+[[tts:text]][whispers] Here is the spoken version.[[/tts:text]]
+```
+
+<Note>
+A Google Cloud Console API key restricted to the Gemini API is valid for this
+provider. This is not the separate Cloud Text-to-Speech API path.
+</Note>
+
+## Realtime voice
+
+The bundled `google` plugin registers a realtime voice provider backed by the
+Gemini Live API for backend audio bridges such as Voice Call and Google Meet.
+
+| Setting               | Config path                                                         | Default                                                                               |
+| --------------------- | ------------------------------------------------------------------- | ------------------------------------------------------------------------------------- |
+| Model                 | `plugins.entries.voice-call.config.realtime.providers.google.model` | `gemini-2.5-flash-native-audio-preview-12-2025`                                       |
+| Voice                 | `...google.voice`                                                   | `Kore`                                                                                |
+| Temperature           | `...google.temperature`                                             | (unset)                                                                               |
+| VAD start sensitivity | `...google.startSensitivity`                                        | (unset)                                                                               |
+| VAD end sensitivity   | `...google.endSensitivity`                                          | (unset)                                                                               |
+| Silence duration      | `...google.silenceDurationMs`                                       | (unset)                                                                               |
+| Activity handling     | `...google.activityHandling`                                        | Google default, `start-of-activity-interrupts`                                        |
+| Turn coverage         | `...google.turnCoverage`                                            | Google default, `only-activity`                                                       |
+| Disable auto VAD      | `...google.automaticActivityDetectionDisabled`                      | `false`                                                                               |
+| Session resumption    | `...google.sessionResumption`                                       | `true`                                                                                |
+| Context compression   | `...google.contextWindowCompression`                                | `true`                                                                                |
+| API key               | `...google.apiKey`                                                  | Falls back to `models.providers.google.apiKey`, `GEMINI_API_KEY`, or `GOOGLE_API_KEY` |
+
+Example Voice Call realtime config:
+
+```json5
+{
+  plugins: {
+    entries: {
+      "voice-call": {
+        enabled: true,
+        config: {
+          realtime: {
+            enabled: true,
+            provider: "google",
+            providers: {
+              google: {
+                model: "gemini-2.5-flash-native-audio-preview-12-2025",
+                voice: "Kore",
+                activityHandling: "start-of-activity-interrupts",
+                turnCoverage: "only-activity",
+              },
+            },
+          },
+        },
+      },
+    },
+  },
+}
+```
+
+<Note>
+Google Live API uses bidirectional audio and function calling over a WebSocket.
+Nodmix adapts telephony/Meet bridge audio to Gemini's PCM Live API stream and
+keeps tool calls on the shared realtime voice contract. Leave `temperature`
+unset unless you need sampling changes; Nodmix omits non-positive values
+because Google Live can return transcripts without audio for `temperature: 0`.
+Gemini API transcription is enabled without `languageCodes`; the current Google
+SDK rejects language-code hints on this API path.
+</Note>
+
+<Note>
+Control UI Talk supports Google Live browser sessions with constrained one-use
+tokens. Backend-only realtime voice providers can also run through the generic
+Gateway relay transport, which keeps provider credentials on the Gateway.
+</Note>
+
+For maintainer live verification, run
+`OPENAI_API_KEY=... GEMINI_API_KEY=... node --import tsx scripts/dev/realtime-talk-live-smoke.ts`.
+The smoke also covers OpenAI backend/WebRTC paths; the Google leg mints the same
+constrained Live API token shape used by Control UI Talk, opens the browser
+WebSocket endpoint, sends the initial setup payload, and waits for
+`setupComplete`.
+
+## Advanced configuration
+
+<AccordionGroup>
+  <Accordion title="Direct Gemini cache reuse">
+    For direct Gemini API runs (`api: "google-generative-ai"`), Nodmix
+    passes a configured `cachedContent` handle through to Gemini requests.
+
+    - Configure per-model or global params with either
+      `cachedContent` or legacy `cached_content`
+    - If both are present, `cachedContent` wins
+    - Example value: `cachedContents/prebuilt-context`
+    - Gemini cache-hit usage is normalized into Nodmix `cacheRead` from
+      upstream `cachedContentTokenCount`
+
+    ```json5
+    {
+      agents: {
+        defaults: {
+          models: {
+            "google/gemini-2.5-pro": {
+              params: {
+                cachedContent: "cachedContents/prebuilt-context",
+              },
+            },
+          },
+        },
+      },
+    }
+    ```
+
+  </Accordion>
+
+  <Accordion title="Gemini CLI JSON usage notes">
+    When using the `google-gemini-cli` OAuth provider, Nodmix normalizes
+    the CLI JSON output as follows:
+
+    - Reply text comes from the CLI JSON `response` field.
+    - Usage falls back to `stats` when the CLI leaves `usage` empty.
+    - `stats.cached` is normalized into Nodmix `cacheRead`.
+    - If `stats.input` is missing, Nodmix derives input tokens from
+      `stats.input_tokens - stats.cached`.
+
+  </Accordion>
+
+  <Accordion title="Environment and daemon setup">
+    If the Gateway runs as a daemon (launchd/systemd), make sure `GEMINI_API_KEY`
+    is available to that process (for example, in `~/.nodmix/.env` or via
+    `env.shellEnv`).
+  </Accordion>
+</AccordionGroup>
+
+## Related
+
+<CardGroup cols={2}>
+  <Card title="Model selection" href="/concepts/model-providers" icon="layers">
+    Choosing providers, model refs, and failover behavior.
+  </Card>
+  <Card title="Image generation" href="/tools/image-generation" icon="image">
+    Shared image tool parameters and provider selection.
+  </Card>
+  <Card title="Video generation" href="/tools/video-generation" icon="video">
+    Shared video tool parameters and provider selection.
+  </Card>
+  <Card title="Music generation" href="/tools/music-generation" icon="music">
+    Shared music tool parameters and provider selection.
+  </Card>
+</CardGroup>