fengming 0.3.9 → 0.3.11

package/docs/providers/ds4.md

@@ -1,309 +0,0 @@
----
-summary: "Run FengMing through ds4, a local DeepSeek V4 Flash OpenAI-compatible server"
-read_when:
-  - You want to run FengMing against antirez/ds4
-  - You want a local DeepSeek V4 Flash backend with tool calls
-  - You need the FengMing config for ds4-server
-title: "ds4"
----
-
-[ds4](https://github.com/antirez/ds4) serves DeepSeek V4 Flash from a local
-Metal backend with an OpenAI-compatible `/v1` API. FengMing connects to ds4
-through the generic `openai-completions` provider family.
-
-ds4 is not a bundled FengMing provider plugin. Configure it under
-`models.providers.ds4`, then select `ds4/deepseek-v4-flash`.
-
-- Provider id: `ds4`
-- Plugin: none
-- API: OpenAI-compatible Chat Completions (`openai-completions`)
-- Suggested base URL: `http://127.0.0.1:18000/v1`
-- Model id: `deepseek-v4-flash`
-- Tool calls: supported through OpenAI-style `tools` and `tool_calls`
-- Reasoning: DeepSeek-style `thinking` and `reasoning_effort`
-
-## Requirements
-
-- macOS with Metal support.
-- A working ds4 checkout with `ds4-server` and the DeepSeek V4 Flash GGUF file.
-- Enough memory for the context you choose. Larger `--ctx` values allocate more
-  KV memory when the server starts.
-
-<Warning>
-FengMing agent turns include tool schemas and workspace context. A tiny context
-such as `--ctx 4096` can pass direct curl tests but fail full agent runs with
-`500 prompt exceeds context`. Use at least `--ctx 32768` for agent and tool
-smoke tests. Use `--ctx 393216` only when you have enough memory and want ds4
-Think Max behavior.
-</Warning>
-
-## Quickstart
-
-<Steps>
-  <Step title="Start ds4-server">
-    Replace `<DS4_DIR>` with your ds4 checkout path.
-
-    ```bash
-    <DS4_DIR>/ds4-server \
-      --model <DS4_DIR>/ds4flash.gguf \
-      --host 127.0.0.1 \
-      --port 18000 \
-      --ctx 32768 \
-      --tokens 128
-    ```
-
-  </Step>
-  <Step title="Verify the OpenAI-compatible endpoint">
-    ```bash
-    curl http://127.0.0.1:18000/v1/models
-    ```
-
-    The response should include `deepseek-v4-flash`.
-
-  </Step>
-  <Step title="Add the FengMing provider config">
-    Add the config from [Full config](#full-config), then run a one-shot model
-    check:
-
-    ```bash
-    fengming infer model run \
-      --local \
-      --model ds4/deepseek-v4-flash \
-      --thinking off \
-      --prompt "Reply with exactly: fengming-ds4-ok" \
-      --json
-    ```
-
-  </Step>
-</Steps>
-
-## Full config
-
-Use this config when ds4 is already running on `127.0.0.1:18000`.
-
-```json5
-{
-  agents: {
-    defaults: {
-      model: { primary: "ds4/deepseek-v4-flash" },
-      models: {
-        "ds4/deepseek-v4-flash": {
-          alias: "DS4 local",
-        },
-      },
-    },
-  },
-  models: {
-    mode: "merge",
-    providers: {
-      ds4: {
-        baseUrl: "http://127.0.0.1:18000/v1",
-        apiKey: "ds4-local",
-        api: "openai-completions",
-        timeoutSeconds: 300,
-        models: [
-          {
-            id: "deepseek-v4-flash",
-            name: "DeepSeek V4 Flash (ds4)",
-            reasoning: true,
-            input: ["text"],
-            cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
-            contextWindow: 32768,
-            maxTokens: 128,
-            compat: {
-              supportsUsageInStreaming: true,
-              supportsReasoningEffort: true,
-              maxTokensField: "max_tokens",
-              supportsStrictMode: false,
-              thinkingFormat: "deepseek",
-              supportedReasoningEfforts: ["low", "medium", "high", "xhigh"],
-            },
-          },
-        ],
-      },
-    },
-  },
-}
-```
-
-Keep `contextWindow` aligned with the `ds4-server --ctx` value. Keep `maxTokens`
-aligned with `--tokens` unless you intentionally want FengMing to request less
-output than the server default.
-
-## On-demand startup
-
-FengMing can start ds4 only when a `ds4/...` model is selected. Add
-`localService` to the same provider entry:
-
-```json5
-{
-  models: {
-    providers: {
-      ds4: {
-        baseUrl: "http://127.0.0.1:18000/v1",
-        apiKey: "ds4-local",
-        api: "openai-completions",
-        timeoutSeconds: 300,
-        localService: {
-          command: "<DS4_DIR>/ds4-server",
-          args: [
-            "--model",
-            "<DS4_DIR>/ds4flash.gguf",
-            "--host",
-            "127.0.0.1",
-            "--port",
-            "18000",
-            "--ctx",
-            "32768",
-            "--tokens",
-            "128",
-          ],
-          cwd: "<DS4_DIR>",
-          healthUrl: "http://127.0.0.1:18000/v1/models",
-          readyTimeoutMs: 300000,
-          idleStopMs: 0,
-        },
-        models: [
-          {
-            id: "deepseek-v4-flash",
-            name: "DeepSeek V4 Flash (ds4)",
-            reasoning: true,
-            input: ["text"],
-            cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
-            contextWindow: 32768,
-            maxTokens: 128,
-            compat: {
-              supportsUsageInStreaming: true,
-              supportsReasoningEffort: true,
-              maxTokensField: "max_tokens",
-              supportsStrictMode: false,
-              thinkingFormat: "deepseek",
-              supportedReasoningEfforts: ["low", "medium", "high", "xhigh"],
-            },
-          },
-        ],
-      },
-    },
-  },
-}
-```
-
-`command` must be an absolute executable path. Shell lookup and `~` expansion are
-not used. See [Local model services](/gateway/local-model-services) for every
-`localService` field.
-
-## Think Max
-
-ds4 applies Think Max only when both conditions are true:
-
-- `ds4-server` starts with `--ctx 393216` or higher.
-- The request uses `reasoning_effort: "max"` or the equivalent ds4 effort field.
-
-If you run that large context, update both the server flags and FengMing model
-metadata:
-
-```json5
-{
-  contextWindow: 393216,
-  maxTokens: 384000,
-  compat: {
-    supportsUsageInStreaming: true,
-    supportsReasoningEffort: true,
-    maxTokensField: "max_tokens",
-    supportsStrictMode: false,
-    thinkingFormat: "deepseek",
-    supportedReasoningEfforts: ["low", "medium", "high", "xhigh", "max"],
-  },
-}
-```
-
-## Test
-
-Start with a direct HTTP check:
-
-```bash
-curl http://127.0.0.1:18000/v1/chat/completions \
-  -H 'content-type: application/json' \
-  -d '{"model":"deepseek-v4-flash","messages":[{"role":"user","content":"Reply with exactly: ds4-ok"}],"max_tokens":16,"stream":false,"thinking":{"type":"disabled"}}'
-```
-
-Then test FengMing model routing:
-
-```bash
-fengming infer model run \
-  --local \
-  --model ds4/deepseek-v4-flash \
-  --thinking off \
-  --prompt "Reply with exactly: fengming-ds4-ok" \
-  --json
-```
-
-For a full agent and tool-call smoke, use a context of at least 32768:
-
-```bash
-fengming agent \
-  --local \
-  --session-id ds4-tool-smoke \
-  --model ds4/deepseek-v4-flash \
-  --thinking off \
-  --message "Use the shell command pwd once, then reply exactly: tool-ok <output>" \
-  --json \
-  --timeout 240
-```
-
-Expected result:
-
-- `executionTrace.winnerProvider` is `ds4`
-- `executionTrace.winnerModel` is `deepseek-v4-flash`
-- `toolSummary.calls` is at least `1`
-- `finalAssistantVisibleText` starts with `tool-ok`
-
-## Troubleshooting
-
-<AccordionGroup>
-  <Accordion title="curl /v1/models cannot connect">
-    ds4 is not running or not bound to the host and port in `baseUrl`. Start
-    `ds4-server`, then retry:
-
-    ```bash
-    curl http://127.0.0.1:18000/v1/models
-    ```
-
-  </Accordion>
-
-  <Accordion title="500 prompt exceeds context">
-    The configured `--ctx` is too small for the FengMing turn. Raise
-    `ds4-server --ctx`, then update `models.providers.ds4.models[].contextWindow`
-    to match. Full agent turns with tools need substantially more context than a
-    direct one-message curl request.
-  </Accordion>
-
-  <Accordion title="Think Max does not activate">
-    ds4 only uses Think Max when `--ctx` is at least `393216` and the request
-    asks for `reasoning_effort: "max"`. Smaller contexts fall back to high
-    reasoning.
-  </Accordion>
-
-  <Accordion title="The first request is slow">
-    ds4 has a cold Metal residency and model warmup phase. Use
-    `localService.readyTimeoutMs: 300000` when FengMing starts the server on
-    demand.
-  </Accordion>
-</AccordionGroup>
-
-## Related
-
-<CardGroup cols={2}>
-  <Card title="Local model services" href="/gateway/local-model-services" icon="play">
-    Start local model servers on demand before model requests.
-  </Card>
-  <Card title="Local models" href="/gateway/local-models" icon="server">
-    Choose and operate local model backends.
-  </Card>
-  <Card title="Model providers" href="/concepts/model-providers" icon="layers">
-    Configure provider refs, auth, and failover.
-  </Card>
-  <Card title="DeepSeek" href="/providers/deepseek" icon="brain">
-    Native DeepSeek provider behavior and thinking controls.
-  </Card>
-</CardGroup>

package/docs/providers/elevenlabs.md

@@ -1,130 +0,0 @@
----
-summary: "Use ElevenLabs speech, Scribe STT, and realtime transcription with FengMing"
-read_when:
-  - You want ElevenLabs text-to-speech in FengMing
-  - You want ElevenLabs Scribe speech-to-text for audio attachments
-  - You want ElevenLabs realtime transcription for Voice Call or Google Meet
-title: "ElevenLabs"
----
-
-FengMing uses ElevenLabs for text-to-speech, batch speech-to-text with Scribe
-v2, and streaming STT with Scribe v2 Realtime.
-
-| Capability               | FengMing surface                                                     | Default                  |
-| ------------------------ | -------------------------------------------------------------------- | ------------------------ |
-| Text-to-speech           | `messages.tts` / `talk`                                              | `eleven_multilingual_v2` |
-| Batch speech-to-text     | `tools.media.audio`                                                  | `scribe_v2`              |
-| Streaming speech-to-text | Voice Call streaming or Google Meet `realtime.transcriptionProvider` | `scribe_v2_realtime`     |
-
-## Authentication
-
-Set `ELEVENLABS_API_KEY` in the environment. `XI_API_KEY` is also accepted for
-compatibility with existing ElevenLabs tooling.
-
-```bash
-export ELEVENLABS_API_KEY="..."
-```
-
-## Text-to-speech
-
-```json5
-{
-  messages: {
-    tts: {
-      providers: {
-        elevenlabs: {
-          apiKey: "${ELEVENLABS_API_KEY}",
-          speakerVoiceId: "pMsXgVXv3BLzUgSXRplE",
-          modelId: "eleven_multilingual_v2",
-        },
-      },
-    },
-  },
-}
-```
-
-Set `modelId` to `eleven_v3` to use ElevenLabs v3 TTS. FengMing keeps
-`eleven_multilingual_v2` as the default for existing installs.
-
-Discord voice channels use ElevenLabs' streaming TTS endpoint when ElevenLabs is
-the selected `voice.tts`/`messages.tts` provider. Playback starts from the
-returned audio stream instead of waiting for FengMing to download and write the
-whole audio file first. `latencyTier` maps to ElevenLabs'
-`optimize_streaming_latency` query parameter for models that accept it; FengMing
-omits that parameter for `eleven_v3`, which rejects it.
-
-## Speech-to-text
-
-Use Scribe v2 for inbound audio attachments and short recorded voice segments:
-
-```json5
-{
-  tools: {
-    media: {
-      audio: {
-        enabled: true,
-        models: [{ provider: "elevenlabs", model: "scribe_v2" }],
-      },
-    },
-  },
-}
-```
-
-FengMing sends multipart audio to ElevenLabs `/v1/speech-to-text` with
-`model_id: "scribe_v2"`. Language hints map to `language_code` when present.
-
-## Streaming STT
-
-The bundled `elevenlabs` plugin registers Scribe v2 Realtime for Voice Call and
-Google Meet agent-mode streaming transcription.
-
-| Setting         | Config path                                                               | Default                                           |
-| --------------- | ------------------------------------------------------------------------- | ------------------------------------------------- |
-| API key         | `plugins.entries.voice-call.config.streaming.providers.elevenlabs.apiKey` | Falls back to `ELEVENLABS_API_KEY` / `XI_API_KEY` |
-| Model           | `...elevenlabs.modelId`                                                   | `scribe_v2_realtime`                              |
-| Audio format    | `...elevenlabs.audioFormat`                                               | `ulaw_8000`                                       |
-| Sample rate     | `...elevenlabs.sampleRate`                                                | `8000`                                            |
-| Commit strategy | `...elevenlabs.commitStrategy`                                            | `vad`                                             |
-| Language        | `...elevenlabs.languageCode`                                              | (unset)                                           |
-
-```json5
-{
-  plugins: {
-    entries: {
-      "voice-call": {
-        config: {
-          streaming: {
-            enabled: true,
-            provider: "elevenlabs",
-            providers: {
-              elevenlabs: {
-                apiKey: "${ELEVENLABS_API_KEY}",
-                audioFormat: "ulaw_8000",
-                commitStrategy: "vad",
-                languageCode: "en",
-              },
-            },
-          },
-        },
-      },
-    },
-  },
-}
-```
-
-<Note>
-Voice Call receives Twilio media as 8 kHz G.711 u-law. The ElevenLabs realtime
-provider defaults to `ulaw_8000`, so telephony frames can be forwarded without
-transcoding.
-</Note>
-
-For Google Meet agent mode, set
-`plugins.entries.google-meet.config.realtime.transcriptionProvider` to
-`"elevenlabs"` and configure the same provider block under
-`plugins.entries.google-meet.config.realtime.providers.elevenlabs`.
-
-## Related
-
-- [Text-to-speech](/tools/tts)
-- [Google Meet](/plugins/google-meet)
-- [Model selection](/concepts/model-providers)

package/docs/providers/fal.md

@@ -1,240 +0,0 @@
----
-summary: "fal image, video, and music generation setup in FengMing"
-title: "Fal"
-read_when:
-  - You want to use fal image generation in FengMing
-  - You need the FAL_KEY auth flow
-  - You want fal defaults for image_generate, video_generate, or music_generate
----
-
-FengMing ships a bundled `fal` provider for hosted image, video, and music
-generation.
-
-| Property | Value                                                         |
-| -------- | ------------------------------------------------------------- |
-| Provider | `fal`                                                         |
-| Auth     | `FAL_KEY` (canonical; `FAL_API_KEY` also works as a fallback) |
-| API      | fal model endpoints                                           |
-
-## Getting started
-
-<Steps>
-  <Step title="Set the API key">
-    ```bash
-    fengming onboard --auth-choice fal-api-key
-    ```
-  </Step>
-  <Step title="Set a default image model">
-    ```json5
-    {
-      agents: {
-        defaults: {
-          imageGenerationModel: {
-            primary: "fal/fal-ai/flux/dev",
-          },
-        },
-      },
-    }
-    ```
-  </Step>
-</Steps>
-
-## Image generation
-
-The bundled `fal` image-generation provider defaults to
-`fal/fal-ai/flux/dev`.
-
-| Capability     | Value                                                              |
-| -------------- | ------------------------------------------------------------------ |
-| Max images     | 4 per request; Krea 2: 1 per request                               |
-| Edit mode      | Flux: 1 reference image; GPT Image 2: 10; Nano Banana 2: 14        |
-| Style refs     | Krea 2: up to 10 style references via `image` / `images`           |
-| Size overrides | Supported                                                          |
-| Aspect ratio   | Supported for generate, Krea 2, and GPT Image 2/Nano Banana 2 edit |
-| Resolution     | Supported                                                          |
-| Output format  | `png` or `jpeg`                                                    |
-
-<Warning>
-Flux image-to-image requests do **not** support `aspectRatio` overrides. GPT
-Image 2 and Nano Banana 2 edit requests use fal's `/edit` endpoint and accept
-aspect-ratio hints. Nano Banana 2 also accepts extra-native wide/tall ratios
-such as `4:1`, `1:4`, `8:1`, and `1:8`; Krea 2 validates its own smaller
-aspect-ratio subset.
-</Warning>
-
-Krea 2 models use fal's native Krea payload schema. FengMing sends
-`aspect_ratio`, `creativity`, and `image_style_references` instead of the
-generic `image_size` / edit-endpoint payload used by Flux. The model refs are:
-
-- `fal/krea/v2/medium/text-to-image`
-- `fal/krea/v2/large/text-to-image`
-
-Use Medium for faster expressive illustration, anime, painting, and artistic
-styles. Use Large for slower photoreal, raw texture, film grain, and detailed
-looks. Krea defaults to `fal.creativity: "medium"`; supported values are
-`raw`, `low`, `medium`, and `high`.
-
-Krea 2 exposes aspect ratio, not `image_size`, in fal's request schema. Prefer
-`aspectRatio`; FengMing maps `size` to the closest supported Krea aspect ratio
-and rejects `resolution` for Krea rather than dropping it.
-
-Use `outputFormat: "png"` when you want PNG output from fal models that expose
-`output_format`. fal does not declare an explicit transparent-background
-control in FengMing, so `background: "transparent"` is reported as an ignored
-override for fal models.
-Krea 2 endpoints do not expose an `output_format` request field through fal, so
-FengMing rejects `outputFormat` overrides for Krea requests.
-
-To use fal as the default image provider:
-
-```json5
-{
-  agents: {
-    defaults: {
-      imageGenerationModel: {
-        primary: "fal/fal-ai/flux/dev",
-      },
-    },
-  },
-}
-```
-
-To use Krea 2 Medium:
-
-```json5
-{
-  agents: {
-    defaults: {
-      imageGenerationModel: {
-        primary: "fal/krea/v2/medium/text-to-image",
-      },
-    },
-  },
-}
-```
-
-## Video generation
-
-The bundled `fal` video-generation provider defaults to
-`fal/fal-ai/minimax/video-01-live`.
-
-| Capability | Value                                                              |
-| ---------- | ------------------------------------------------------------------ |
-| Modes      | Text-to-video, single-image reference, Seedance reference-to-video |
-| Runtime    | Queue-backed submit/status/result flow for long-running jobs       |
-
-<AccordionGroup>
-  <Accordion title="Available video models">
-    **HeyGen video-agent:**
-
-    - `fal/fal-ai/heygen/v2/video-agent`
-
-    **Seedance 2.0:**
-
-    - `fal/bytedance/seedance-2.0/fast/text-to-video`
-    - `fal/bytedance/seedance-2.0/fast/image-to-video`
-    - `fal/bytedance/seedance-2.0/fast/reference-to-video`
-    - `fal/bytedance/seedance-2.0/text-to-video`
-    - `fal/bytedance/seedance-2.0/image-to-video`
-    - `fal/bytedance/seedance-2.0/reference-to-video`
-
-  </Accordion>
-
-  <Accordion title="Seedance 2.0 config example">
-    ```json5
-    {
-      agents: {
-        defaults: {
-          videoGenerationModel: {
-            primary: "fal/bytedance/seedance-2.0/fast/text-to-video",
-          },
-        },
-      },
-    }
-    ```
-  </Accordion>
-
-  <Accordion title="Seedance 2.0 reference-to-video config example">
-    ```json5
-    {
-      agents: {
-        defaults: {
-          videoGenerationModel: {
-            primary: "fal/bytedance/seedance-2.0/fast/reference-to-video",
-          },
-        },
-      },
-    }
-    ```
-
-    Reference-to-video accepts up to 9 images, 3 videos, and 3 audio references
-    through the shared `video_generate` `images`, `videos`, and `audioRefs`
-    parameters, with at most 12 total reference files.
-
-  </Accordion>
-
-  <Accordion title="HeyGen video-agent config example">
-    ```json5
-    {
-      agents: {
-        defaults: {
-          videoGenerationModel: {
-            primary: "fal/fal-ai/heygen/v2/video-agent",
-          },
-        },
-      },
-    }
-    ```
-  </Accordion>
-</AccordionGroup>
-
-## Music generation
-
-The bundled `fal` plugin also registers a music-generation provider for the
-shared `music_generate` tool.
-
-| Capability    | Value                                                                                                  |
-| ------------- | ------------------------------------------------------------------------------------------------------ |
-| Default model | `fal/fal-ai/minimax-music/v2.6`                                                                        |
-| Models        | `fal-ai/minimax-music/v2.6`, `fal-ai/ace-step/prompt-to-audio`, `fal-ai/stable-audio-25/text-to-audio` |
-| Runtime       | Synchronous request plus generated audio download                                                      |
-
-Use fal as the default music provider:
-
-```json5
-{
-  agents: {
-    defaults: {
-      musicGenerationModel: {
-        primary: "fal/fal-ai/minimax-music/v2.6",
-      },
-    },
-  },
-}
-```
-
-`fal-ai/minimax-music/v2.6` supports explicit lyrics and instrumental mode.
-ACE-Step and Stable Audio are prompt-to-audio endpoints; choose them with the
-`model` override when you want those model families.
-
-<Tip>
-Use `fengming models list --provider fal` to see the full list of available fal
-models, including any recently added entries.
-</Tip>
-
-## Related
-
-<CardGroup cols={2}>
-  <Card title="Image generation" href="/tools/image-generation" icon="image">
-    Shared image tool parameters and provider selection.
-  </Card>
-  <Card title="Video generation" href="/tools/video-generation" icon="video">
-    Shared video tool parameters and provider selection.
-  </Card>
-  <Card title="Music generation" href="/tools/music-generation" icon="music">
-    Shared music tool parameters and provider selection.
-  </Card>
-  <Card title="Configuration reference" href="/gateway/config-agents#agent-defaults" icon="gear">
-    Agent defaults including image, video, and music model selection.
-  </Card>
-</CardGroup>