nodmix 2026.5.25

package/docs/nodes/media-understanding.md

@@ -0,0 +1,469 @@
+---
+summary: "Inbound image/audio/video understanding (optional) with provider + CLI fallbacks"
+read_when:
+  - Designing or refactoring media understanding
+  - Tuning inbound audio/video/image preprocessing
+title: "Media understanding"
+sidebarTitle: "Media understanding"
+---
+
+Nodmix can **summarize inbound media** (image/audio/video) before the reply pipeline runs. It auto-detects when local tools or provider keys are available, and can be disabled or customized. If understanding is off, models still receive the original files/URLs as usual.
+
+Vendor-specific media behavior is registered by vendor plugins, while Nodmix core owns the shared `tools.media` config, fallback order, and reply-pipeline integration.
+
+## Goals
+
+- Optional: pre-digest inbound media into short text for faster routing + better command parsing.
+- Preserve original media delivery to the model (always).
+- Support **provider APIs** and **CLI fallbacks**.
+- Allow multiple models with ordered fallback (error/size/timeout).
+
+## High-level behavior
+
+<Steps>
+  <Step title="Collect attachments">
+    Collect inbound attachments (`MediaPaths`, `MediaUrls`, `MediaTypes`).
+  </Step>
+  <Step title="Select per-capability">
+    For each enabled capability (image/audio/video), select attachments per policy (default: **first**).
+  </Step>
+  <Step title="Choose model">
+    Choose the first eligible model entry (size + capability + auth).
+  </Step>
+  <Step title="Fallback on failure">
+    If a model fails or the media is too large, **fall back to the next entry**.
+  </Step>
+  <Step title="Apply success block">
+    On success:
+
+    - `Body` becomes `[Image]`, `[Audio]`, or `[Video]` block.
+    - Audio sets `{{Transcript}}`; command parsing uses caption text when present, otherwise the transcript.
+    - Captions are preserved as `User text:` inside the block.
+
+  </Step>
+</Steps>
+
+If understanding fails or is disabled, **the reply flow continues** with the original body + attachments.
+
+## Config overview
+
+`tools.media` supports **shared models** plus per-capability overrides:
+
+<AccordionGroup>
+  <Accordion title="Top-level keys">
+    - `tools.media.models`: shared model list (use `capabilities` to gate).
+    - `tools.media.image` / `tools.media.audio` / `tools.media.video`:
+      - defaults (`prompt`, `maxChars`, `maxBytes`, `timeoutSeconds`, `language`)
+      - provider overrides (`baseUrl`, `headers`, `providerOptions`)
+      - Deepgram audio options via `tools.media.audio.providerOptions.deepgram`
+      - audio transcript echo controls (`echoTranscript`, default `false`; `echoFormat`)
+      - optional **per-capability `models` list** (preferred before shared models)
+      - `attachments` policy (`mode`, `maxAttachments`, `prefer`)
+      - `scope` (optional gating by channel/chatType/session key)
+    - `tools.media.concurrency`: max concurrent capability runs (default **2**).
+
+  </Accordion>
+</AccordionGroup>
+
+```json5
+{
+  tools: {
+    media: {
+      models: [
+        /* shared list */
+      ],
+      image: {
+        /* optional overrides */
+      },
+      audio: {
+        /* optional overrides */
+        echoTranscript: true,
+        echoFormat: '📝 "{transcript}"',
+      },
+      video: {
+        /* optional overrides */
+      },
+    },
+  },
+}
+```
+
+### Model entries
+
+Each `models[]` entry can be **provider** or **CLI**:
+
+<Tabs>
+  <Tab title="Provider entry">
+    ```json5
+    {
+      type: "provider", // default if omitted
+      provider: "openai",
+      model: "gpt-5.5",
+      prompt: "Describe the image in <= 500 chars.",
+      maxChars: 500,
+      maxBytes: 10485760,
+      timeoutSeconds: 60,
+      capabilities: ["image"], // optional, used for multi-modal entries
+      profile: "vision-profile",
+      preferredProfile: "vision-fallback",
+    }
+    ```
+  </Tab>
+  <Tab title="CLI entry">
+    ```json5
+    {
+      type: "cli",
+      command: "gemini",
+      args: [
+        "-m",
+        "gemini-3-flash",
+        "--allowed-tools",
+        "read_file",
+        "Read the media at {{MediaPath}} and describe it in <= {{MaxChars}} characters.",
+      ],
+      maxChars: 500,
+      maxBytes: 52428800,
+      timeoutSeconds: 120,
+      capabilities: ["video", "image"],
+    }
+    ```
+
+    CLI templates can also use:
+
+    - `{{MediaDir}}` (directory containing the media file)
+    - `{{OutputDir}}` (scratch dir created for this run)
+    - `{{OutputBase}}` (scratch file base path, no extension)
+
+  </Tab>
+</Tabs>
+
+## Defaults and limits
+
+Recommended defaults:
+
+- `maxChars`: **500** for image/video (short, command-friendly)
+- `maxChars`: **unset** for audio (full transcript unless you set a limit)
+- `maxBytes`:
+  - image: **10MB**
+  - audio: **20MB**
+  - video: **50MB**
+
+<AccordionGroup>
+  <Accordion title="Rules">
+    - If media exceeds `maxBytes`, that model is skipped and the **next model is tried**.
+    - Audio files smaller than **1024 bytes** are treated as empty/corrupt and skipped before provider/CLI transcription; inbound reply context receives a deterministic placeholder transcript so the agent knows the note was too small.
+    - If the model returns more than `maxChars`, output is trimmed.
+    - `prompt` defaults to simple "Describe the {media}." plus the `maxChars` guidance (image/video only).
+    - If the active primary image model already supports vision natively, Nodmix skips the `[Image]` summary block and passes the original image into the model instead.
+    - If a Gateway/WebChat primary model is text-only, image attachments are preserved as offloaded `media://inbound/*` refs so the image/PDF tools or configured image model can still inspect them instead of losing the attachment.
+    - Explicit `nodmix infer image describe --model <provider/model>` requests are different: they run that image-capable provider/model directly, including Ollama refs such as `ollama/qwen2.5vl:7b`.
+    - If `<capability>.enabled: true` but no models are configured, Nodmix tries the **active reply model** when its provider supports the capability.
+
+  </Accordion>
+</AccordionGroup>
+
+### Auto-detect media understanding (default)
+
+If `tools.media.<capability>.enabled` is **not** set to `false` and you haven't configured models, Nodmix auto-detects in this order and **stops at the first working option**:
+
+<Steps>
+  <Step title="Active reply model">
+    Active reply model when its provider supports the capability.
+  </Step>
+  <Step title="agents.defaults.imageModel">
+    `agents.defaults.imageModel` primary/fallback refs (image only).
+    Prefer `provider/model` refs. Bare refs are qualified from configured image-capable provider model entries only when the match is unique.
+  </Step>
+  <Step title="Local CLIs (audio only)">
+    Local CLIs (if installed):
+
+    - `sherpa-onnx-offline` (requires `SHERPA_ONNX_MODEL_DIR` with encoder/decoder/joiner/tokens)
+    - `whisper-cli` (`whisper-cpp`; uses `WHISPER_CPP_MODEL` or the bundled tiny model)
+    - `whisper` (Python CLI; downloads models automatically)
+
+  </Step>
+  <Step title="Gemini CLI">
+    `gemini` using `read_many_files`.
+  </Step>
+  <Step title="Provider auth">
+    - Configured `models.providers.*` entries that support the capability are tried before the bundled fallback order.
+    - Image-only config providers with an image-capable model auto-register for media understanding even when they are not a bundled vendor plugin.
+    - Ollama image understanding is available when selected explicitly, for example through `agents.defaults.imageModel` or `nodmix infer image describe --model ollama/<vision-model>`.
+
+    Bundled fallback order:
+
+    - Audio: OpenAI → Groq → xAI → Deepgram → OpenRouter → Google → SenseAudio → ElevenLabs → Mistral
+    - Image: OpenAI → Anthropic → Google → MiniMax → MiniMax Portal → Z.AI
+    - Video: Google → Qwen → Moonshot
+
+  </Step>
+</Steps>
+
+To disable auto-detection, set:
+
+```json5
+{
+  tools: {
+    media: {
+      audio: {
+        enabled: false,
+      },
+    },
+  },
+}
+```
+
+<Note>
+Binary detection is best-effort across macOS/Linux/Windows; ensure the CLI is on `PATH` (we expand `~`), or set an explicit CLI model with a full command path.
+</Note>
+
+### Proxy environment support (provider models)
+
+When provider-based **audio** and **video** media understanding is enabled, Nodmix honors standard outbound proxy environment variables for provider HTTP calls:
+
+- `HTTPS_PROXY`
+- `HTTP_PROXY`
+- `ALL_PROXY`
+- `https_proxy`
+- `http_proxy`
+- `all_proxy`
+
+If no proxy env vars are set, media understanding uses direct egress. If the proxy value is malformed, Nodmix logs a warning and falls back to direct fetch.
+
+## Capabilities (optional)
+
+If you set `capabilities`, the entry only runs for those media types. For shared lists, Nodmix can infer defaults:
+
+- `openai`, `anthropic`, `minimax`: **image**
+- `minimax-portal`: **image**
+- `moonshot`: **image + video**
+- `openrouter`: **image + audio**
+- `google` (Gemini API): **image + audio + video**
+- `qwen`: **image + video**
+- `mistral`: **audio**
+- `zai`: **image**
+- `groq`: **audio**
+- `xai`: **audio**
+- `deepgram`: **audio**
+- Any `models.providers.<id>.models[]` catalog with an image-capable model: **image**
+
+For CLI entries, **set `capabilities` explicitly** to avoid surprising matches. If you omit `capabilities`, the entry is eligible for the list it appears in.
+
+## Provider support matrix (Nodmix integrations)
+
+| Capability | Provider integration                                                                                                         | Notes                                                                                                                                                                                                                                   |
+| ---------- | ---------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| Image      | OpenAI, OpenAI Codex OAuth, Codex app-server, OpenRouter, Anthropic, Google, MiniMax, Moonshot, Qwen, Z.AI, config providers | Vendor plugins register image support; `openai-codex/*` uses OAuth provider plumbing; `codex/*` uses a bounded Codex app-server turn; MiniMax and MiniMax OAuth both use `MiniMax-VL-01`; image-capable config providers auto-register. |
+| Audio      | OpenAI, Groq, xAI, Deepgram, OpenRouter, Google, SenseAudio, ElevenLabs, Mistral                                             | Provider transcription (Whisper/Groq/xAI/Deepgram/OpenRouter STT/Gemini/SenseAudio/Scribe/Voxtral).                                                                                                                                     |
+| Video      | Google, Qwen, Moonshot                                                                                                       | Provider video understanding via vendor plugins; Qwen video understanding uses the Standard DashScope endpoints.                                                                                                                        |
+
+<Note>
+**MiniMax note**
+
+- `minimax`, `minimax-cn`, `minimax-portal`, and `minimax-portal-cn` image understanding comes from the plugin-owned `MiniMax-VL-01` media provider.
+- Automatic image routing keeps using `MiniMax-VL-01` even if legacy MiniMax M2.x chat metadata claims image input.
+
+</Note>
+
+## Model selection guidance
+
+- Prefer the strongest latest-generation model available for each media capability when quality and safety matter.
+- For tool-enabled agents handling untrusted inputs, avoid older/weaker media models.
+- Keep at least one fallback per capability for availability (quality model + faster/cheaper model).
+- CLI fallbacks (`whisper-cli`, `whisper`, `gemini`) are useful when provider APIs are unavailable.
+- `parakeet-mlx` note: with `--output-dir`, Nodmix reads `<output-dir>/<media-basename>.txt` when output format is `txt` (or unspecified); non-`txt` formats fall back to stdout.
+
+## Attachment policy
+
+Per-capability `attachments` controls which attachments are processed:
+
+<ParamField path="mode" type='"first" | "all"' default="first">
+  Whether to process the first selected attachment or all of them.
+</ParamField>
+<ParamField path="maxAttachments" type="number" default="1">
+  Cap the number processed.
+</ParamField>
+<ParamField path="prefer" type='"first" | "last" | "path" | "url"'>
+  Selection preference among candidate attachments.
+</ParamField>
+
+When `mode: "all"`, outputs are labeled `[Image 1/2]`, `[Audio 2/2]`, etc.
+
+<AccordionGroup>
+  <Accordion title="File-attachment extraction behavior">
+    - Extracted file text is wrapped as **untrusted external content** before it is appended to the media prompt.
+    - The injected block uses explicit boundary markers like `<<<EXTERNAL_UNTRUSTED_CONTENT id="...">>>` / `<<<END_EXTERNAL_UNTRUSTED_CONTENT id="...">>>` and includes a `Source: External` metadata line.
+    - This attachment-extraction path intentionally omits the long `SECURITY NOTICE:` banner to avoid bloating the media prompt; the boundary markers and metadata still remain.
+    - If a file has no extractable text, Nodmix injects `[No extractable text]`.
+    - If a PDF falls back to rendered page images in this path, the media prompt keeps the placeholder `[PDF content rendered to images; images not forwarded to model]` because this attachment-extraction step forwards text blocks, not the rendered PDF images.
+
+  </Accordion>
+</AccordionGroup>
+
+## Config examples
+
+<Tabs>
+  <Tab title="Shared models + overrides">
+    ```json5
+    {
+      tools: {
+        media: {
+          models: [
+            { provider: "openai", model: "gpt-5.5", capabilities: ["image"] },
+            {
+              provider: "google",
+              model: "gemini-3-flash-preview",
+              capabilities: ["image", "audio", "video"],
+            },
+            {
+              type: "cli",
+              command: "gemini",
+              args: [
+                "-m",
+                "gemini-3-flash",
+                "--allowed-tools",
+                "read_file",
+                "Read the media at {{MediaPath}} and describe it in <= {{MaxChars}} characters.",
+              ],
+              capabilities: ["image", "video"],
+            },
+          ],
+          audio: {
+            attachments: { mode: "all", maxAttachments: 2 },
+          },
+          video: {
+            maxChars: 500,
+          },
+        },
+      },
+    }
+    ```
+  </Tab>
+  <Tab title="Audio + video only">
+    ```json5
+    {
+      tools: {
+        media: {
+          audio: {
+            enabled: true,
+            models: [
+              { provider: "openai", model: "gpt-4o-mini-transcribe" },
+              {
+                type: "cli",
+                command: "whisper",
+                args: ["--model", "base", "{{MediaPath}}"],
+              },
+            ],
+          },
+          video: {
+            enabled: true,
+            maxChars: 500,
+            models: [
+              { provider: "google", model: "gemini-3-flash-preview" },
+              {
+                type: "cli",
+                command: "gemini",
+                args: [
+                  "-m",
+                  "gemini-3-flash",
+                  "--allowed-tools",
+                  "read_file",
+                  "Read the media at {{MediaPath}} and describe it in <= {{MaxChars}} characters.",
+                ],
+              },
+            ],
+          },
+        },
+      },
+    }
+    ```
+  </Tab>
+  <Tab title="Image-only">
+    ```json5
+    {
+      tools: {
+        media: {
+          image: {
+            enabled: true,
+            maxBytes: 10485760,
+            maxChars: 500,
+            models: [
+              { provider: "openai", model: "gpt-5.5" },
+              { provider: "anthropic", model: "claude-opus-4-6" },
+              {
+                type: "cli",
+                command: "gemini",
+                args: [
+                  "-m",
+                  "gemini-3-flash",
+                  "--allowed-tools",
+                  "read_file",
+                  "Read the media at {{MediaPath}} and describe it in <= {{MaxChars}} characters.",
+                ],
+              },
+            ],
+          },
+        },
+      },
+    }
+    ```
+  </Tab>
+  <Tab title="Multi-modal single entry">
+    ```json5
+    {
+      tools: {
+        media: {
+          image: {
+            models: [
+              {
+                provider: "google",
+                model: "gemini-3.1-pro-preview",
+                capabilities: ["image", "video", "audio"],
+              },
+            ],
+          },
+          audio: {
+            models: [
+              {
+                provider: "google",
+                model: "gemini-3.1-pro-preview",
+                capabilities: ["image", "video", "audio"],
+              },
+            ],
+          },
+          video: {
+            models: [
+              {
+                provider: "google",
+                model: "gemini-3.1-pro-preview",
+                capabilities: ["image", "video", "audio"],
+              },
+            ],
+          },
+        },
+      },
+    }
+    ```
+  </Tab>
+</Tabs>
+
+## Status output
+
+When media understanding runs, `/status` includes a short summary line:
+
+```
+📎 Media: image ok (openai/gpt-5.4) · audio skipped (maxBytes)
+```
+
+This shows per-capability outcomes and the chosen provider/model when applicable.
+
+## Notes
+
+- Understanding is **best-effort**. Errors do not block replies.
+- Attachments are still passed to models even when understanding is disabled.
+- Use `scope` to limit where understanding runs (e.g. only DMs).
+
+## Related
+
+- [Configuration](/gateway/configuration)
+- [Image & media support](/nodes/images)

package/docs/nodes/talk.md

@@ -0,0 +1,154 @@
+---
+summary: "Talk mode: continuous speech conversations across local STT/TTS and realtime voice"
+read_when:
+  - Implementing Talk mode on macOS/iOS/Android
+  - Changing voice/TTS/interrupt behavior
+title: "Talk mode"
+---
+
+Talk mode has two runtime shapes:
+
+- Native macOS/iOS/Android Talk uses local speech recognition, Gateway chat, and `talk.speak` TTS. Nodes advertise the `talk` capability and declare the `talk.*` commands they support.
+- Browser Talk uses `talk.client.create` for client-owned `webrtc` and `provider-websocket` sessions, or `talk.session.create` for Gateway-owned `gateway-relay` sessions. `managed-room` is reserved for Gateway handoff and walkie-talkie rooms.
+- Android Talk can opt into Gateway-owned realtime relay sessions with `talk.realtime.mode: "realtime"` and `talk.realtime.transport: "gateway-relay"`. Otherwise it stays on native speech recognition, Gateway chat, and `talk.speak`.
+- Transcription-only clients use `talk.session.create({ mode: "transcription", transport: "gateway-relay", brain: "none" })`, then `talk.session.appendAudio`, `talk.session.cancelTurn`, and `talk.session.close` when they need captions or dictation without an assistant voice response.
+
+Native Talk is a continuous voice conversation loop:
+
+1. Listen for speech
+2. Send transcript to the model through the active session
+3. Wait for the response
+4. Speak it via the configured Talk provider (`talk.speak`)
+
+Browser realtime Talk forwards provider tool calls through `talk.client.toolCall`; browser clients do not call `chat.send` directly for realtime consults.
+
+Transcription-only Talk emits the same common Talk event envelope as realtime and STT/TTS sessions, but uses `mode: "transcription"` and `brain: "none"`. It is for captions, dictation, and observe-only speech capture; one-shot uploaded voice notes still use the media/audio path.
+
+## Behavior (macOS)
+
+- **Always-on overlay** while Talk mode is enabled.
+- **Listening → Thinking → Speaking** phase transitions.
+- On a **short pause** (silence window), the current transcript is sent.
+- Replies are **written to WebChat** (same as typing).
+- **Interrupt on speech** (default on): if the user starts talking while the assistant is speaking, we stop playback and note the interruption timestamp for the next prompt.
+
+## Voice directives in replies
+
+The assistant may prefix its reply with a **single JSON line** to control voice:
+
+```json
+{ "voice": "<voice-id>", "once": true }
+```
+
+Rules:
+
+- First non-empty line only.
+- Unknown keys are ignored.
+- `once: true` applies to the current reply only.
+- Without `once`, the voice becomes the new default for Talk mode.
+- The JSON line is stripped before TTS playback.
+
+Supported keys:
+
+- `voice` / `voice_id` / `voiceId`
+- `model` / `model_id` / `modelId`
+- `speed`, `rate` (WPM), `stability`, `similarity`, `style`, `speakerBoost`
+- `seed`, `normalize`, `lang`, `output_format`, `latency_tier`
+- `once`
+
+## Config (`~/.nodmix/nodmix.json`)
+
+```json5
+{
+  talk: {
+    provider: "elevenlabs",
+    providers: {
+      elevenlabs: {
+        voiceId: "elevenlabs_voice_id",
+        modelId: "eleven_v3",
+        outputFormat: "mp3_44100_128",
+        apiKey: "elevenlabs_api_key",
+      },
+      mlx: {
+        modelId: "mlx-community/Soprano-80M-bf16",
+      },
+      system: {},
+    },
+    speechLocale: "ru-RU",
+    silenceTimeoutMs: 1500,
+    interruptOnSpeech: true,
+    realtime: {
+      provider: "openai",
+      providers: {
+        openai: {
+          apiKey: "openai_api_key",
+          model: "gpt-realtime-2",
+          voice: "cedar",
+        },
+      },
+      instructions: "Speak warmly and keep answers brief.",
+      mode: "realtime",
+      transport: "webrtc",
+      brain: "agent-consult",
+    },
+  },
+}
+```
+
+Defaults:
+
+- `interruptOnSpeech`: true
+- `silenceTimeoutMs`: when unset, Talk keeps the platform default pause window before sending the transcript (`700 ms on macOS and Android, 900 ms on iOS`)
+- `provider`: selects the active Talk provider. Use `elevenlabs`, `mlx`, or `system` for the macOS-local playback paths.
+- `providers.<provider>.voiceId`: falls back to `ELEVENLABS_VOICE_ID` / `SAG_VOICE_ID` for ElevenLabs (or first ElevenLabs voice when API key is available).
+- `providers.elevenlabs.modelId`: defaults to `eleven_v3` when unset.
+- `providers.mlx.modelId`: defaults to `mlx-community/Soprano-80M-bf16` when unset.
+- `providers.elevenlabs.apiKey`: falls back to `ELEVENLABS_API_KEY` (or gateway shell profile if available).
+- `consultThinkingLevel`: optional thinking level override for the full Nodmix agent run behind realtime `nodmix_agent_consult` calls.
+- `consultFastMode`: optional fast-mode override for realtime `nodmix_agent_consult` calls.
+- `realtime.provider`: selects the active browser/server realtime voice provider. Use `openai` for WebRTC, `google` for provider WebSocket, or a bridge-only provider through Gateway relay.
+- `realtime.providers.<provider>` stores provider-owned realtime config. The browser receives only ephemeral or constrained session credentials, never a standard API key.
+- `realtime.providers.openai.voice`: built-in OpenAI Realtime voice id. Current `gpt-realtime-2` voices are `alloy`, `ash`, `ballad`, `coral`, `echo`, `sage`, `shimmer`, `verse`, `marin`, and `cedar`; `marin` and `cedar` are recommended for best quality.
+- `realtime.transport`: `webrtc` and `provider-websocket` are browser realtime transports. Android uses realtime relay only when this is `gateway-relay`; otherwise Android Talk uses its native STT/TTS loop.
+- `realtime.brain`: `agent-consult` routes realtime tool calls through Gateway policy; `direct-tools` is legacy direct-tool compatibility behavior; `none` is for transcription or external orchestration.
+- `realtime.instructions`: appends provider-facing system instructions to Nodmix's built-in realtime prompt. Use it for voice style and tone; Nodmix keeps the default `nodmix_agent_consult` guidance.
+- `talk.catalog` exposes each provider's valid modes, transports, brain strategies, realtime audio formats, and capability flags so first-party Talk clients can avoid unsupported combinations.
+- Streaming transcription providers are discovered through `talk.catalog.transcription`. The current Gateway relay uses the Voice Call streaming provider config until the dedicated Talk transcription config surface is added.
+- `speechLocale`: optional BCP 47 locale id for on-device Talk speech recognition on iOS/macOS. Leave unset to use the device default.
+- `outputFormat`: defaults to `pcm_44100` on macOS/iOS and `pcm_24000` on Android (set `mp3_*` to force MP3 streaming)
+
+## macOS UI
+
+- Menu bar toggle: **Talk**
+- Config tab: **Talk Mode** group (voice id + interrupt toggle)
+- Overlay:
+  - **Listening**: cloud pulses with mic level
+  - **Thinking**: sinking animation
+  - **Speaking**: radiating rings
+  - Click cloud: stop speaking
+  - Click X: exit Talk mode
+
+## Android UI
+
+- Voice tab toggle: **Talk**
+- Manual **Mic** and **Talk** are mutually exclusive runtime capture modes.
+- Manual Mic stops when the app leaves the foreground or the user leaves the Voice tab.
+- Talk Mode keeps running until toggled off or the Android node disconnects, and uses Android's microphone foreground-service type while active.
+
+## Notes
+
+- Requires Speech + Microphone permissions.
+- Native Talk uses the active Gateway session and only falls back to history polling when response events are unavailable.
+- Browser realtime Talk uses `talk.client.toolCall` for `nodmix_agent_consult` instead of exposing `chat.send` to provider-owned browser sessions.
+- Transcription-only Talk uses `talk.session.create`, `talk.session.appendAudio`, `talk.session.cancelTurn`, and `talk.session.close`; clients subscribe to `talk.event` for partial/final transcript updates.
+- The gateway resolves Talk playback through `talk.speak` using the active Talk provider. Android falls back to local system TTS only when that RPC is unavailable.
+- macOS local MLX playback uses the bundled `nodmix-mlx-tts` helper when present, or an executable on `PATH`. Set `NODMIX_MLX_TTS_BIN` to point at a custom helper binary during development.
+- `stability` for `eleven_v3` is validated to `0.0`, `0.5`, or `1.0`; other models accept `0..1`.
+- `latency_tier` is validated to `0..4` when set.
+- Android supports `pcm_16000`, `pcm_22050`, `pcm_24000`, and `pcm_44100` output formats for low-latency AudioTrack streaming.
+
+## Related
+
+- [Voice wake](/nodes/voicewake)
+- [Audio and voice notes](/nodes/audio)
+- [Media understanding](/nodes/media-understanding)