fengming 0.3.10 → 0.3.12

package/docs/nodes/media-understanding.md

@@ -1,495 +0,0 @@
----
-summary: "Inbound image/audio/video understanding (optional) with provider + CLI fallbacks"
-read_when:
-  - Designing or refactoring media understanding
-  - Tuning inbound audio/video/image preprocessing
-title: "Media understanding"
-sidebarTitle: "Media understanding"
----
-
-FengMing can **summarize inbound media** (image/audio/video) before the reply pipeline runs. It auto-detects when local tools or provider keys are available, and can be disabled or customized. If understanding is off, models still receive the original files/URLs as usual.
-
-Vendor-specific media behavior is registered by vendor plugins, while FengMing core owns the shared `tools.media` config, fallback order, and reply-pipeline integration.
-
-## Goals
-
-- Optional: pre-digest inbound media into short text for faster routing + better command parsing.
-- Preserve original media delivery to the model (always).
-- Support **provider APIs** and **CLI fallbacks**.
-- Allow multiple models with ordered fallback (error/size/timeout).
-
-## High-level behavior
-
-<Steps>
-  <Step title="Collect attachments">
-    Collect inbound attachments (`MediaPaths`, `MediaUrls`, `MediaTypes`).
-  </Step>
-  <Step title="Select per-capability">
-    For each enabled capability (image/audio/video), select attachments per policy (default: **first**).
-  </Step>
-  <Step title="Choose model">
-    Choose the first eligible model entry (size + capability + auth).
-  </Step>
-  <Step title="Fallback on failure">
-    If a model fails or the media is too large, **fall back to the next entry**.
-  </Step>
-  <Step title="Apply success block">
-    On success:
-
-    - `Body` becomes `[Image]`, `[Audio]`, or `[Video]` block.
-    - Audio sets `{{Transcript}}`; command parsing uses caption text when present, otherwise the transcript.
-    - Captions are preserved as `User text:` inside the block.
-
-  </Step>
-</Steps>
-
-If understanding fails or is disabled, **the reply flow continues** with the original body + attachments.
-
-## Config overview
-
-`tools.media` supports **shared models** plus per-capability overrides:
-
-<AccordionGroup>
-  <Accordion title="Top-level keys">
-    - `tools.media.models`: shared model list (use `capabilities` to gate).
-    - `tools.media.image` / `tools.media.audio` / `tools.media.video`:
-      - defaults (`prompt`, `maxChars`, `maxBytes`, `timeoutSeconds`, `language`)
-      - provider overrides (`baseUrl`, `headers`, `providerOptions`)
-      - Deepgram audio options via `tools.media.audio.providerOptions.deepgram`
-      - audio transcript echo controls (`echoTranscript`, default `false`; `echoFormat`)
-      - optional **per-capability `models` list** (preferred before shared models)
-      - `attachments` policy (`mode`, `maxAttachments`, `prefer`)
-      - `scope` (optional gating by channel/chatType/session key)
-    - `tools.media.concurrency`: max concurrent capability runs (default **2**).
-
-  </Accordion>
-</AccordionGroup>
-
-```json5
-{
-  tools: {
-    media: {
-      models: [
-        /* shared list */
-      ],
-      image: {
-        /* optional overrides */
-      },
-      audio: {
-        /* optional overrides */
-        echoTranscript: true,
-        echoFormat: '📝 "{transcript}"',
-      },
-      video: {
-        /* optional overrides */
-      },
-    },
-  },
-}
-```
-
-### Model entries
-
-Each `models[]` entry can be **provider** or **CLI**:
-
-<Tabs>
-  <Tab title="Provider entry">
-    ```json5
-    {
-      type: "provider", // default if omitted
-      provider: "openai",
-      model: "gpt-5.5",
-      prompt: "Describe the image in <= 500 chars.",
-      maxChars: 500,
-      maxBytes: 10485760,
-      timeoutSeconds: 60,
-      capabilities: ["image"], // optional, used for multi-modal entries
-      profile: "vision-profile",
-      preferredProfile: "vision-fallback",
-    }
-    ```
-  </Tab>
-  <Tab title="CLI entry">
-    ```json5
-    {
-      type: "cli",
-      command: "gemini",
-      args: [
-        "-m",
-        "gemini-3-flash",
-        "--allowed-tools",
-        "read_file",
-        "Read the media at {{MediaPath}} and describe it in <= {{MaxChars}} characters.",
-      ],
-      maxChars: 500,
-      maxBytes: 52428800,
-      timeoutSeconds: 120,
-      capabilities: ["video", "image"],
-    }
-    ```
-
-    CLI templates can also use:
-
-    - `{{MediaDir}}` (directory containing the media file)
-    - `{{OutputDir}}` (scratch dir created for this run)
-    - `{{OutputBase}}` (scratch file base path, no extension)
-
-  </Tab>
-</Tabs>
-
-### Provider credentials (`apiKey`)
-
-Provider media understanding uses the same provider auth resolution as normal
-model calls: auth profiles, environment variables, then
-`models.providers.<providerId>.apiKey`.
-
-`tools.media.*.models[]` entries do not accept an inline `apiKey` field. The
-`provider` value in a media model entry, such as `openai` or `moonshot`, must
-have credentials available through one of the standard provider auth sources.
-
-Minimal example:
-
-```json5
-{
-  models: {
-    providers: {
-      openai: { apiKey: "<OPENAI_API_KEY>" },
-      moonshot: { apiKey: "<MOONSHOT_API_KEY>" },
-    },
-  },
-}
-```
-
-For the full provider auth reference, including profiles, environment
-variables, and custom base URLs, see [Tools and custom providers](/gateway/config-tools).
-
-## Defaults and limits
-
-Recommended defaults:
-
-- `maxChars`: **500** for image/video (short, command-friendly)
-- `maxChars`: **unset** for audio (full transcript unless you set a limit)
-- `maxBytes`:
-  - image: **10MB**
-  - audio: **20MB**
-  - video: **50MB**
-
-<AccordionGroup>
-  <Accordion title="Rules">
-    - If media exceeds `maxBytes`, that model is skipped and the **next model is tried**.
-    - Audio files smaller than **1024 bytes** are treated as empty/corrupt and skipped before provider/CLI transcription; inbound reply context receives a deterministic placeholder transcript so the agent knows the note was too small.
-    - If the model returns more than `maxChars`, output is trimmed.
-    - `prompt` defaults to simple "Describe the {media}." plus the `maxChars` guidance (image/video only).
-    - If the active primary image model already supports vision natively, FengMing skips the `[Image]` summary block and passes the original image into the model instead.
-    - If a Gateway/WebChat primary model is text-only, image attachments are preserved as offloaded `media://inbound/*` refs so the image/PDF tools or configured image model can still inspect them instead of losing the attachment.
-    - Explicit `fengming infer image describe --model <provider/model>` requests are different: they run that image-capable provider/model directly, including Ollama refs such as `ollama/qwen2.5vl:7b`.
-    - If `<capability>.enabled: true` but no models are configured, FengMing tries the **active reply model** when its provider supports the capability.
-
-  </Accordion>
-</AccordionGroup>
-
-### Auto-detect media understanding (default)
-
-If `tools.media.<capability>.enabled` is **not** set to `false` and you haven't configured models, FengMing auto-detects in this order and **stops at the first working option**:
-
-<Steps>
-  <Step title="Active reply model">
-    Active reply model when its provider supports the capability.
-  </Step>
-  <Step title="agents.defaults.imageModel">
-    `agents.defaults.imageModel` primary/fallback refs (image only).
-    Prefer `provider/model` refs. Bare refs are qualified from configured image-capable provider model entries only when the match is unique.
-  </Step>
-  <Step title="Local CLIs (audio only)">
-    Local CLIs (if installed):
-
-    - `sherpa-onnx-offline` (requires `SHERPA_ONNX_MODEL_DIR` with encoder/decoder/joiner/tokens)
-    - `whisper-cli` (`whisper-cpp`; uses `WHISPER_CPP_MODEL` or the bundled tiny model)
-    - `whisper` (Python CLI; downloads models automatically)
-
-  </Step>
-  <Step title="Gemini CLI">
-    `gemini` using `read_many_files`.
-  </Step>
-  <Step title="Provider auth">
-    - Configured `models.providers.*` entries that support the capability are tried before the bundled fallback order.
-    - Image-only config providers with an image-capable model auto-register for media understanding even when they are not a bundled vendor plugin.
-    - Ollama image understanding is available when selected explicitly, for example through `agents.defaults.imageModel` or `fengming infer image describe --model ollama/<vision-model>`.
-
-    Bundled fallback order:
-
-    - Audio: OpenAI → Groq → xAI → Deepgram → OpenRouter → Google → SenseAudio → ElevenLabs → Mistral
-    - Image: OpenAI → Anthropic → Google → MiniMax → MiniMax Portal → Z.AI
-    - Video: Google → Qwen → Moonshot
-
-  </Step>
-</Steps>
-
-To disable auto-detection, set:
-
-```json5
-{
-  tools: {
-    media: {
-      audio: {
-        enabled: false,
-      },
-    },
-  },
-}
-```
-
-<Note>
-Binary detection is best-effort across macOS/Linux/Windows; ensure the CLI is on `PATH` (we expand `~`), or set an explicit CLI model with a full command path.
-</Note>
-
-### Proxy environment support (provider models)
-
-When provider-based **audio** and **video** media understanding is enabled, FengMing honors standard outbound proxy environment variables for provider HTTP calls:
-
-- `HTTPS_PROXY`
-- `HTTP_PROXY`
-- `ALL_PROXY`
-- `https_proxy`
-- `http_proxy`
-- `all_proxy`
-
-If no proxy env vars are set, media understanding uses direct egress. If the proxy value is malformed, FengMing logs a warning and falls back to direct fetch.
-
-## Capabilities (optional)
-
-If you set `capabilities`, the entry only runs for those media types. For shared lists, FengMing can infer defaults:
-
-- `openai`, `anthropic`, `minimax`: **image**
-- `minimax-portal`: **image**
-- `moonshot`: **image + video**
-- `openrouter`: **image + audio**
-- `google` (Gemini API): **image + audio + video**
-- `qwen`: **image + video**
-- `mistral`: **audio**
-- `zai`: **image**
-- `groq`: **audio**
-- `xai`: **audio**
-- `deepgram`: **audio**
-- Any `models.providers.<id>.models[]` catalog with an image-capable model: **image**
-
-For CLI entries, **set `capabilities` explicitly** to avoid surprising matches. If you omit `capabilities`, the entry is eligible for the list it appears in.
-
-## Provider support matrix (FengMing integrations)
-
-| Capability | Provider integration                                                                                                         | Notes                                                                                                                                                                                                                                       |
-| ---------- | ---------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| Image      | OpenAI, OpenAI Codex OAuth, Codex app-server, OpenRouter, Anthropic, Google, MiniMax, Moonshot, Qwen, Z.AI, config providers | Vendor plugins register image support; `openai/*` can use API-key or Codex OAuth routing; `codex/*` uses a bounded Codex app-server turn; MiniMax and MiniMax OAuth both use `MiniMax-VL-01`; image-capable config providers auto-register. |
-| Audio      | OpenAI, Groq, xAI, Deepgram, OpenRouter, Google, SenseAudio, ElevenLabs, Mistral                                             | Provider transcription (Whisper/Groq/xAI/Deepgram/OpenRouter STT/Gemini/SenseAudio/Scribe/Voxtral).                                                                                                                                         |
-| Video      | Google, Qwen, Moonshot                                                                                                       | Provider video understanding via vendor plugins; Qwen video understanding uses the Standard DashScope endpoints.                                                                                                                            |
-
-<Note>
-**MiniMax note**
-
-- `minimax`, `minimax-cn`, `minimax-portal`, and `minimax-portal-cn` image understanding comes from the plugin-owned `MiniMax-VL-01` media provider.
-- Automatic image routing keeps using `MiniMax-VL-01` even if legacy MiniMax M2.x chat metadata claims image input.
-
-</Note>
-
-## Model selection guidance
-
-- Prefer the strongest latest-generation model available for each media capability when quality and safety matter.
-- For tool-enabled agents handling untrusted inputs, avoid older/weaker media models.
-- Keep at least one fallback per capability for availability (quality model + faster/cheaper model).
-- CLI fallbacks (`whisper-cli`, `whisper`, `gemini`) are useful when provider APIs are unavailable.
-- `parakeet-mlx` note: with `--output-dir`, FengMing reads `<output-dir>/<media-basename>.txt` when output format is `txt` (or unspecified); non-`txt` formats fall back to stdout.
-
-## Attachment policy
-
-Per-capability `attachments` controls which attachments are processed:
-
-<ParamField path="mode" type='"first" | "all"' default="first">
-  Whether to process the first selected attachment or all of them.
-</ParamField>
-<ParamField path="maxAttachments" type="number" default="1">
-  Cap the number processed.
-</ParamField>
-<ParamField path="prefer" type='"first" | "last" | "path" | "url"'>
-  Selection preference among candidate attachments.
-</ParamField>
-
-When `mode: "all"`, outputs are labeled `[Image 1/2]`, `[Audio 2/2]`, etc.
-
-<AccordionGroup>
-  <Accordion title="File-attachment extraction behavior">
-    - Extracted file text is wrapped as **untrusted external content** before it is appended to the media prompt.
-    - The injected block uses explicit boundary markers like `<<<EXTERNAL_UNTRUSTED_CONTENT id="...">>>` / `<<<END_EXTERNAL_UNTRUSTED_CONTENT id="...">>>` and includes a `Source: External` metadata line.
-    - This attachment-extraction path intentionally omits the long `SECURITY NOTICE:` banner to avoid bloating the media prompt; the boundary markers and metadata still remain.
-    - If a file has no extractable text, FengMing injects `[No extractable text]`.
-    - If a PDF falls back to rendered page images in this path, the media prompt keeps the placeholder `[PDF content rendered to images; images not forwarded to model]` because this attachment-extraction step forwards text blocks, not the rendered PDF images.
-
-  </Accordion>
-</AccordionGroup>
-
-## Config examples
-
-<Tabs>
-  <Tab title="Shared models + overrides">
-    ```json5
-    {
-      tools: {
-        media: {
-          models: [
-            { provider: "openai", model: "gpt-5.5", capabilities: ["image"] },
-            {
-              provider: "google",
-              model: "gemini-3-flash-preview",
-              capabilities: ["image", "audio", "video"],
-            },
-            {
-              type: "cli",
-              command: "gemini",
-              args: [
-                "-m",
-                "gemini-3-flash",
-                "--allowed-tools",
-                "read_file",
-                "Read the media at {{MediaPath}} and describe it in <= {{MaxChars}} characters.",
-              ],
-              capabilities: ["image", "video"],
-            },
-          ],
-          audio: {
-            attachments: { mode: "all", maxAttachments: 2 },
-          },
-          video: {
-            maxChars: 500,
-          },
-        },
-      },
-    }
-    ```
-  </Tab>
-  <Tab title="Audio + video only">
-    ```json5
-    {
-      tools: {
-        media: {
-          audio: {
-            enabled: true,
-            models: [
-              { provider: "openai", model: "gpt-4o-mini-transcribe" },
-              {
-                type: "cli",
-                command: "whisper",
-                args: ["--model", "base", "{{MediaPath}}"],
-              },
-            ],
-          },
-          video: {
-            enabled: true,
-            maxChars: 500,
-            models: [
-              { provider: "google", model: "gemini-3-flash-preview" },
-              {
-                type: "cli",
-                command: "gemini",
-                args: [
-                  "-m",
-                  "gemini-3-flash",
-                  "--allowed-tools",
-                  "read_file",
-                  "Read the media at {{MediaPath}} and describe it in <= {{MaxChars}} characters.",
-                ],
-              },
-            ],
-          },
-        },
-      },
-    }
-    ```
-  </Tab>
-  <Tab title="Image-only">
-    ```json5
-    {
-      tools: {
-        media: {
-          image: {
-            enabled: true,
-            maxBytes: 10485760,
-            maxChars: 500,
-            models: [
-              { provider: "openai", model: "gpt-5.5" },
-              { provider: "anthropic", model: "claude-opus-4-6" },
-              {
-                type: "cli",
-                command: "gemini",
-                args: [
-                  "-m",
-                  "gemini-3-flash",
-                  "--allowed-tools",
-                  "read_file",
-                  "Read the media at {{MediaPath}} and describe it in <= {{MaxChars}} characters.",
-                ],
-              },
-            ],
-          },
-        },
-      },
-    }
-    ```
-  </Tab>
-  <Tab title="Multi-modal single entry">
-    ```json5
-    {
-      tools: {
-        media: {
-          image: {
-            models: [
-              {
-                provider: "google",
-                model: "gemini-3.1-pro-preview",
-                capabilities: ["image", "video", "audio"],
-              },
-            ],
-          },
-          audio: {
-            models: [
-              {
-                provider: "google",
-                model: "gemini-3.1-pro-preview",
-                capabilities: ["image", "video", "audio"],
-              },
-            ],
-          },
-          video: {
-            models: [
-              {
-                provider: "google",
-                model: "gemini-3.1-pro-preview",
-                capabilities: ["image", "video", "audio"],
-              },
-            ],
-          },
-        },
-      },
-    }
-    ```
-  </Tab>
-</Tabs>
-
-## Status output
-
-When media understanding runs, `/status` includes a short summary line:
-
-```
-📎 Media: image ok (openai/gpt-5.4) · audio skipped (maxBytes)
-```
-
-This shows per-capability outcomes and the chosen provider/model when applicable.
-
-## Notes
-
-- Understanding is **best-effort**. Errors do not block replies.
-- Attachments are still passed to models even when understanding is disabled.
-- Use `scope` to limit where understanding runs (e.g. only DMs).
-
-## Related
-
-- [Configuration](/gateway/configuration)
-- [Image & media support](/nodes/images)

package/docs/nodes/talk.md

@@ -1,160 +0,0 @@
----
-summary: "Talk mode: continuous speech conversations across local STT/TTS and realtime voice"
-read_when:
-  - Implementing Talk mode on macOS/iOS/Android
-  - Changing voice/TTS/interrupt behavior
-title: "Talk mode"
----
-
-Talk mode has two runtime shapes:
-
-- Native macOS/iOS/Android Talk uses local speech recognition, Gateway chat, and `talk.speak` TTS. Nodes advertise the `talk` capability and declare the `talk.*` commands they support.
-- Browser Talk uses `talk.client.create` for client-owned `webrtc` and `provider-websocket` sessions, or `talk.session.create` for Gateway-owned `gateway-relay` sessions. `managed-room` is reserved for Gateway handoff and walkie-talkie rooms.
-- Android Talk can opt into Gateway-owned realtime relay sessions with `talk.realtime.mode: "realtime"` and `talk.realtime.transport: "gateway-relay"`. Otherwise it stays on native speech recognition, Gateway chat, and `talk.speak`.
-- Transcription-only clients use `talk.session.create({ mode: "transcription", transport: "gateway-relay", brain: "none" })`, then `talk.session.appendAudio`, `talk.session.cancelTurn`, and `talk.session.close` when they need captions or dictation without an assistant voice response.
-
-Native Talk is a continuous voice conversation loop:
-
-1. Listen for speech
-2. Send transcript to the model through the active session
-3. Wait for the response
-4. Speak it via the configured Talk provider (`talk.speak`)
-
-Browser realtime Talk forwards provider tool calls through `talk.client.toolCall`; browser clients do not call `chat.send` directly for realtime consults.
-While a realtime consult is active, Talk clients can use `talk.client.steer` or
-`talk.session.steer` to classify spoken input as `status`, `steer`, `cancel`, or
-`followup`. Accepted steering is queued into the active embedded run; rejected
-steering returns a structured reason such as `no_active_run`, `not_streaming`,
-or `compacting`.
-
-Transcription-only Talk emits the same common Talk event envelope as realtime and STT/TTS sessions, but uses `mode: "transcription"` and `brain: "none"`. It is for captions, dictation, and observe-only speech capture; one-shot uploaded voice notes still use the media/audio path.
-
-## Behavior (macOS)
-
-- **Always-on overlay** while Talk mode is enabled.
-- **Listening → Thinking → Speaking** phase transitions.
-- On a **short pause** (silence window), the current transcript is sent.
-- Replies are **written to WebChat** (same as typing).
-- **Interrupt on speech** (default on): if the user starts talking while the assistant is speaking, we stop playback and note the interruption timestamp for the next prompt.
-
-## Voice directives in replies
-
-The assistant may prefix its reply with a **single JSON line** to control voice:
-
-```json
-{ "voice": "<voice-id>", "once": true }
-```
-
-Rules:
-
-- First non-empty line only.
-- Unknown keys are ignored.
-- `once: true` applies to the current reply only.
-- Without `once`, the voice becomes the new default for Talk mode.
-- The JSON line is stripped before TTS playback.
-
-Supported keys:
-
-- `voice` / `voice_id` / `voiceId`
-- `model` / `model_id` / `modelId`
-- `speed`, `rate` (WPM), `stability`, `similarity`, `style`, `speakerBoost`
-- `seed`, `normalize`, `lang`, `output_format`, `latency_tier`
-- `once`
-
-## Config (`~/.fengming/fengming.json`)
-
-```json5
-{
-  talk: {
-    provider: "elevenlabs",
-    providers: {
-      elevenlabs: {
-        voiceId: "elevenlabs_voice_id",
-        modelId: "eleven_v3",
-        outputFormat: "mp3_44100_128",
-        apiKey: "elevenlabs_api_key",
-      },
-      mlx: {
-        modelId: "mlx-community/Soprano-80M-bf16",
-      },
-      system: {},
-    },
-    speechLocale: "ru-RU",
-    silenceTimeoutMs: 1500,
-    interruptOnSpeech: true,
-    realtime: {
-      provider: "openai",
-      providers: {
-        openai: {
-          apiKey: "openai_api_key",
-          model: "gpt-realtime-2",
-          voice: "cedar",
-        },
-      },
-      instructions: "Speak warmly and keep answers brief.",
-      mode: "realtime",
-      transport: "webrtc",
-      brain: "agent-consult",
-    },
-  },
-}
-```
-
-Defaults:
-
-- `interruptOnSpeech`: true
-- `silenceTimeoutMs`: when unset, Talk keeps the platform default pause window before sending the transcript (`700 ms on macOS and Android, 900 ms on iOS`)
-- `provider`: selects the active Talk provider. Use `elevenlabs`, `mlx`, or `system` for the macOS-local playback paths.
-- `providers.<provider>.voiceId`: falls back to `ELEVENLABS_VOICE_ID` / `SAG_VOICE_ID` for ElevenLabs (or first ElevenLabs voice when API key is available).
-- `providers.elevenlabs.modelId`: defaults to `eleven_v3` when unset.
-- `providers.mlx.modelId`: defaults to `mlx-community/Soprano-80M-bf16` when unset.
-- `providers.elevenlabs.apiKey`: falls back to `ELEVENLABS_API_KEY` (or gateway shell profile if available).
-- `consultThinkingLevel`: optional thinking level override for the full FengMing agent run behind realtime `fengming_agent_consult` calls.
-- `consultFastMode`: optional fast-mode override for realtime `fengming_agent_consult` calls.
-- `realtime.provider`: selects the active browser/server realtime voice provider. Use `openai` for WebRTC, `google` for provider WebSocket, or a bridge-only provider through Gateway relay.
-- `realtime.providers.<provider>` stores provider-owned realtime config. The browser receives only ephemeral or constrained session credentials, never a standard API key.
-- `realtime.providers.openai.voice`: built-in OpenAI Realtime voice id. Current `gpt-realtime-2` voices are `alloy`, `ash`, `ballad`, `coral`, `echo`, `sage`, `shimmer`, `verse`, `marin`, and `cedar`; `marin` and `cedar` are recommended for best quality.
-- `realtime.transport`: `webrtc` and `provider-websocket` are browser realtime transports. Android uses realtime relay only when this is `gateway-relay`; otherwise Android Talk uses its native STT/TTS loop.
-- `realtime.brain`: `agent-consult` routes realtime tool calls through Gateway policy; `direct-tools` is legacy direct-tool compatibility behavior; `none` is for transcription or external orchestration.
-- `realtime.consultRouting`: `provider-direct` preserves the provider's direct reply when it skips `fengming_agent_consult`; `force-agent-consult` makes Gateway relay route finalized user transcripts through FengMing instead.
-- `realtime.instructions`: appends provider-facing system instructions to FengMing's built-in realtime prompt. Use it for voice style and tone; FengMing keeps the default `fengming_agent_consult` guidance.
-- `talk.catalog` exposes each provider's valid modes, transports, brain strategies, realtime audio formats, and capability flags so first-party Talk clients can avoid unsupported combinations.
-- Streaming transcription providers are discovered through `talk.catalog.transcription`. The current Gateway relay uses the Voice Call streaming provider config until the dedicated Talk transcription config surface is added.
-- `speechLocale`: optional BCP 47 locale id for on-device Talk speech recognition on iOS/macOS. Leave unset to use the device default.
-- `outputFormat`: defaults to `pcm_44100` on macOS/iOS and `pcm_24000` on Android (set `mp3_*` to force MP3 streaming)
-
-## macOS UI
-
-- Menu bar toggle: **Talk**
-- Config tab: **Talk Mode** group (voice id + interrupt toggle)
-- Overlay:
-  - **Listening**: cloud pulses with mic level
-  - **Thinking**: sinking animation
-  - **Speaking**: radiating rings
-  - Click cloud: stop speaking
-  - Click X: exit Talk mode
-
-## Android UI
-
-- Voice tab toggle: **Talk**
-- Manual **Mic** and **Talk** are mutually exclusive runtime capture modes.
-- Manual Mic stops when the app leaves the foreground or the user leaves the Voice tab.
-- Talk Mode keeps running until toggled off or the Android node disconnects, and uses Android's microphone foreground-service type while active.
-
-## Notes
-
-- Requires Speech + Microphone permissions.
-- Native Talk uses the active Gateway session and only falls back to history polling when response events are unavailable.
-- Browser realtime Talk uses `talk.client.toolCall` for `fengming_agent_consult` instead of exposing `chat.send` to provider-owned browser sessions.
-- Transcription-only Talk uses `talk.session.create`, `talk.session.appendAudio`, `talk.session.cancelTurn`, and `talk.session.close`; clients subscribe to `talk.event` for partial/final transcript updates.
-- The gateway resolves Talk playback through `talk.speak` using the active Talk provider. Android falls back to local system TTS only when that RPC is unavailable.
-- macOS local MLX playback uses the bundled `fengming-mlx-tts` helper when present, or an executable on `PATH`. Set `FENGMING_MLX_TTS_BIN` to point at a custom helper binary during development.
-- `stability` for `eleven_v3` is validated to `0.0`, `0.5`, or `1.0`; other models accept `0..1`.
-- `latency_tier` is validated to `0..4` when set.
-- Android supports `pcm_16000`, `pcm_22050`, `pcm_24000`, and `pcm_44100` output formats for low-latency AudioTrack streaming.
-
-## Related
-
-- [Voice wake](/nodes/voicewake)
-- [Audio and voice notes](/nodes/audio)
-- [Media understanding](/nodes/media-understanding)