fengming 0.3.9 → 0.3.11

package/docs/providers/vllm.md

@@ -1,407 +0,0 @@
----
-summary: "Run FengMing with vLLM (OpenAI-compatible local server)"
-read_when:
-  - You want to run FengMing against a local vLLM server
-  - You want OpenAI-compatible /v1 endpoints with your own models
-title: "vLLM"
----
-
-vLLM can serve open-source (and some custom) models via an **OpenAI-compatible** HTTP API. FengMing connects to vLLM using the `openai-completions` API.
-
-FengMing can also **auto-discover** available models from vLLM when you opt in with `VLLM_API_KEY` (any value works if your server does not enforce auth). Use `vllm/*` in `agents.defaults.models` to keep discovery dynamic when you also configure a custom vLLM base URL.
-
-FengMing treats `vllm` as a local OpenAI-compatible provider that supports
-streamed usage accounting, so status/context token counts can update from
-`stream_options.include_usage` responses.
-
-| Property         | Value                                    |
-| ---------------- | ---------------------------------------- |
-| Provider ID      | `vllm`                                   |
-| API              | `openai-completions` (OpenAI-compatible) |
-| Auth             | `VLLM_API_KEY` environment variable      |
-| Default base URL | `http://127.0.0.1:8000/v1`               |
-
-## Getting started
-
-<Steps>
-  <Step title="Start vLLM with an OpenAI-compatible server">
-    Your base URL should expose `/v1` endpoints (e.g. `/v1/models`, `/v1/chat/completions`). vLLM commonly runs on:
-
-    ```
-    http://127.0.0.1:8000/v1
-    ```
-
-  </Step>
-  <Step title="Set the API key environment variable">
-    Any value works if your server does not enforce auth:
-
-    ```bash
-    export VLLM_API_KEY="vllm-local"
-    ```
-
-  </Step>
-  <Step title="Select a model">
-    Replace with one of your vLLM model IDs:
-
-    ```json5
-    {
-      agents: {
-        defaults: {
-          model: { primary: "vllm/your-model-id" },
-        },
-      },
-    }
-    ```
-
-  </Step>
-  <Step title="Verify the model is available">
-    ```bash
-    fengming models list --provider vllm
-    ```
-  </Step>
-</Steps>
-
-## Model discovery (implicit provider)
-
-When `VLLM_API_KEY` is set (or an auth profile exists) and you **do not** define `models.providers.vllm`, FengMing queries:
-
-```
-GET http://127.0.0.1:8000/v1/models
-```
-
-and converts the returned IDs into model entries.
-
-<Note>
-If you set `models.providers.vllm` explicitly, FengMing uses your declared models by default. Add `"vllm/*": {}` to `agents.defaults.models` when you want FengMing to query that configured provider's `/models` endpoint and include all advertised vLLM models.
-</Note>
-
-## Explicit configuration (manual models)
-
-Use explicit config when:
-
-- vLLM runs on a different host or port
-- You want to pin `contextWindow` or `maxTokens` values
-- Your server requires a real API key (or you want to control headers)
-- You connect to a trusted loopback, LAN, or Tailscale vLLM endpoint
-
-```json5
-{
-  models: {
-    providers: {
-      vllm: {
-        baseUrl: "http://127.0.0.1:8000/v1",
-        apiKey: "${VLLM_API_KEY}",
-        api: "openai-completions",
-        timeoutSeconds: 300, // Optional: extend connect/header/body/request timeout for slow local models
-        models: [
-          {
-            id: "your-model-id",
-            name: "Local vLLM Model",
-            reasoning: false,
-            input: ["text"],
-            cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
-            contextWindow: 128000,
-            maxTokens: 8192,
-          },
-        ],
-      },
-    },
-  },
-}
-```
-
-To keep this provider dynamic without manually listing every model, add a provider
-wildcard to the visible model catalog:
-
-```json5
-{
-  agents: {
-    defaults: {
-      models: {
-        "vllm/*": {},
-      },
-    },
-  },
-}
-```
-
-## Advanced configuration
-
-<AccordionGroup>
-  <Accordion title="Proxy-style behavior">
-    vLLM is treated as a proxy-style OpenAI-compatible `/v1` backend, not a native
-    OpenAI endpoint. This means:
-
-    | Behavior | Applied? |
-    |----------|----------|
-    | Native OpenAI request shaping | No |
-    | `service_tier` | Not sent |
-    | Responses `store` | Not sent |
-    | Prompt-cache hints | Not sent |
-    | OpenAI reasoning-compat payload shaping | Not applied |
-    | Hidden FengMing attribution headers | Not injected on custom base URLs |
-
-  </Accordion>
-
-  <Accordion title="Qwen thinking controls">
-    For Qwen models served through vLLM, set
-    `compat.thinkingFormat: "qwen-chat-template"` on the configured provider
-    model row when the server expects Qwen chat-template kwargs. Models
-    configured this way expose a binary `/think` profile (`off`, `on`) because
-    Qwen template thinking is an on/off request flag, not an OpenAI-style effort
-    ladder.
-
-    ```json5
-    {
-      models: {
-        providers: {
-          vllm: {
-            models: [
-              {
-                id: "Qwen/Qwen3-8B",
-                name: "Qwen3 8B",
-                reasoning: true,
-                compat: { thinkingFormat: "qwen-chat-template" },
-              },
-            ],
-          },
-        },
-      },
-    }
-    ```
-
-    FengMing maps `/think off` to:
-
-    ```json
-    {
-      "chat_template_kwargs": {
-        "enable_thinking": false,
-        "preserve_thinking": true
-      }
-    }
-    ```
-
-    Non-`off` thinking levels send `enable_thinking: true`. If your endpoint
-    expects DashScope-style top-level flags instead, use
-    `compat.thinkingFormat: "qwen"` to send `enable_thinking` at the request
-    root.
-
-  </Accordion>
-
-  <Accordion title="Nemotron 3 thinking controls">
-    vLLM/Nemotron 3 can use chat-template kwargs to control whether reasoning is
-    returned as hidden reasoning or visible answer text. When an FengMing session
-    uses `vllm/nemotron-3-*` with thinking off, the bundled vLLM plugin sends:
-
-    ```json
-    {
-      "chat_template_kwargs": {
-        "enable_thinking": false,
-        "force_nonempty_content": true
-      }
-    }
-    ```
-
-    To customize these values, set `chat_template_kwargs` under the model params.
-    If you also set `params.extra_body.chat_template_kwargs`, that value has
-    final precedence because `extra_body` is the last request-body override.
-
-    ```json5
-    {
-      agents: {
-        defaults: {
-          models: {
-            "vllm/nemotron-3-super": {
-              params: {
-                chat_template_kwargs: {
-                  enable_thinking: false,
-                  force_nonempty_content: true,
-                },
-              },
-            },
-          },
-        },
-      },
-    }
-    ```
-
-  </Accordion>
-
-  <Accordion title="Qwen tool calls appear as text">
-    First make sure vLLM was started with the right tool-call parser and chat
-    template for the model. For example, vLLM documents `hermes` for Qwen2.5
-    models and `qwen3_xml` for Qwen3-Coder models.
-
-    Symptoms:
-
-    - skills or tools never run
-    - the assistant prints raw JSON/XML such as `{"name":"read","arguments":...}`
-    - vLLM returns an empty `tool_calls` array when FengMing sends
-      `tool_choice: "auto"`
-
-    Some Qwen/vLLM combinations return structured tool calls only when the
-    request uses `tool_choice: "required"`. For those model entries, force the
-    OpenAI-compatible request field with `params.extra_body`:
-
-    ```json5
-    {
-      agents: {
-        defaults: {
-          models: {
-            "vllm/Qwen-Qwen2.5-Coder-32B-Instruct": {
-              params: {
-                extra_body: {
-                  tool_choice: "required",
-                },
-              },
-            },
-          },
-        },
-      },
-    }
-    ```
-
-    Replace `Qwen-Qwen2.5-Coder-32B-Instruct` with the exact id returned by:
-
-    ```bash
-    fengming models list --provider vllm
-    ```
-
-    You can apply the same override from the CLI:
-
-    ```bash
-    fengming config set agents.defaults.models '{"vllm/Qwen-Qwen2.5-Coder-32B-Instruct":{"params":{"extra_body":{"tool_choice":"required"}}}}' --strict-json --merge
-    ```
-
-    This is an opt-in compatibility workaround. It makes every model turn with
-    tools require a tool call, so use it only for a dedicated local model entry
-    where that behavior is acceptable. Do not use it as a global default for all
-    vLLM models, and do not use a proxy that blindly converts arbitrary
-    assistant text into executable tool calls.
-
-  </Accordion>
-
-  <Accordion title="Custom base URL">
-    If your vLLM server runs on a non-default host or port, set `baseUrl` in the explicit provider config:
-
-    ```json5
-    {
-      models: {
-        providers: {
-          vllm: {
-            baseUrl: "http://192.168.1.50:9000/v1",
-            apiKey: "${VLLM_API_KEY}",
-            api: "openai-completions",
-            timeoutSeconds: 300,
-            models: [
-              {
-                id: "my-custom-model",
-                name: "Remote vLLM Model",
-                reasoning: false,
-                input: ["text"],
-                contextWindow: 64000,
-                maxTokens: 4096,
-              },
-            ],
-          },
-        },
-      },
-    }
-    ```
-
-  </Accordion>
-</AccordionGroup>
-
-## Troubleshooting
-
-<AccordionGroup>
-  <Accordion title="Slow first response or remote server timeout">
-    For large local models, remote LAN hosts, or tailnet links, set a
-    provider-scoped request timeout:
-
-    ```json5
-    {
-      models: {
-        providers: {
-          vllm: {
-            baseUrl: "http://192.168.1.50:8000/v1",
-            apiKey: "${VLLM_API_KEY}",
-            api: "openai-completions",
-            timeoutSeconds: 300,
-            models: [{ id: "your-model-id", name: "Local vLLM Model" }],
-          },
-        },
-      },
-    }
-    ```
-
-    `timeoutSeconds` applies to vLLM model HTTP requests only, including
-    connection setup, response headers, body streaming, and the total
-    guarded-fetch abort. Prefer this before increasing
-    `agents.defaults.timeoutSeconds`, which controls the whole agent run.
-
-  </Accordion>
-
-  <Accordion title="Server not reachable">
-    Check that the vLLM server is running and accessible:
-
-    ```bash
-    curl http://127.0.0.1:8000/v1/models
-    ```
-
-    If you see a connection error, verify the host, port, and that vLLM started with the OpenAI-compatible server mode.
-    For explicit loopback, LAN, or Tailscale endpoints, FengMing trusts the
-    exact configured `models.providers.vllm.baseUrl` origin for guarded model
-    requests. Metadata/link-local origins remain blocked without explicit
-    opt-in. Set `models.providers.vllm.request.allowPrivateNetwork: true` only
-    when vLLM requests must reach another private origin, and set it to `false`
-    to opt out of exact-origin trust.
-
-  </Accordion>
-
-  <Accordion title="Auth errors on requests">
-    If requests fail with auth errors, set a real `VLLM_API_KEY` that matches your server configuration, or configure the provider explicitly under `models.providers.vllm`.
-
-    <Tip>
-    If your vLLM server does not enforce auth, any non-empty value for `VLLM_API_KEY` works as an opt-in signal for FengMing.
-    </Tip>
-
-  </Accordion>
-
-  <Accordion title="No models discovered">
-    Auto-discovery requires `VLLM_API_KEY` to be set. If you have defined `models.providers.vllm`, FengMing uses only your declared models unless `agents.defaults.models` includes `"vllm/*": {}`.
-  </Accordion>
-
-  <Accordion title="Tools render as raw text">
-    If a Qwen model prints JSON/XML tool syntax instead of executing a skill,
-    check the Qwen guidance in Advanced configuration above. The usual fix is:
-
-    - start vLLM with the correct parser/template for that model
-    - confirm the exact model id with `fengming models list --provider vllm`
-    - add a dedicated per-model `params.extra_body.tool_choice: "required"`
-      override only if `tool_choice: "auto"` still returns empty or text-only
-      tool calls
-
-  </Accordion>
-</AccordionGroup>
-
-<Warning>
-More help: [Troubleshooting](/help/troubleshooting) and [FAQ](/help/faq).
-</Warning>
-
-## Related
-
-<CardGroup cols={2}>
-  <Card title="Model selection" href="/concepts/model-providers" icon="layers">
-    Choosing providers, model refs, and failover behavior.
-  </Card>
-  <Card title="OpenAI" href="/providers/openai" icon="bolt">
-    Native OpenAI provider and OpenAI-compatible route behavior.
-  </Card>
-  <Card title="OAuth and auth" href="/gateway/authentication" icon="key">
-    Auth details and credential reuse rules.
-  </Card>
-  <Card title="Troubleshooting" href="/help/troubleshooting" icon="wrench">
-    Common issues and how to resolve them.
-  </Card>
-</CardGroup>

package/docs/providers/volcengine.md

@@ -1,199 +0,0 @@
----
-summary: "Volcano Engine setup (Doubao models, coding endpoints, and Seed Speech TTS)"
-title: "Volcengine (Doubao)"
-read_when:
-  - You want to use Volcano Engine or Doubao models with FengMing
-  - You need the Volcengine API key setup
-  - You want to use Volcengine Speech text-to-speech
----
-
-The Volcengine provider gives access to Doubao models and third-party models
-hosted on Volcano Engine, with separate endpoints for general and coding
-workloads. The same bundled plugin can also register Volcengine Speech as a TTS
-provider.
-
-| Detail     | Value                                                      |
-| ---------- | ---------------------------------------------------------- |
-| Providers  | `volcengine` (general + TTS) + `volcengine-plan` (coding)  |
-| Model auth | `VOLCANO_ENGINE_API_KEY`                                   |
-| TTS auth   | `VOLCENGINE_TTS_API_KEY` or `BYTEPLUS_SEED_SPEECH_API_KEY` |
-| API        | OpenAI-compatible models, BytePlus Seed Speech TTS         |
-
-## Getting started
-
-<Steps>
-  <Step title="Set the API key">
-    Run interactive onboarding:
-
-    ```bash
-    fengming onboard --auth-choice volcengine-api-key
-    ```
-
-    This registers both the general (`volcengine`) and coding (`volcengine-plan`) providers from a single API key.
-
-  </Step>
-  <Step title="Set a default model">
-    ```json5
-    {
-      agents: {
-        defaults: {
-          model: { primary: "volcengine-plan/ark-code-latest" },
-        },
-      },
-    }
-    ```
-  </Step>
-  <Step title="Verify the model is available">
-    ```bash
-    fengming models list --provider volcengine
-    fengming models list --provider volcengine-plan
-    ```
-  </Step>
-</Steps>
-
-<Tip>
-For non-interactive setup (CI, scripting), pass the key directly:
-
-```bash
-fengming onboard --non-interactive \
-  --mode local \
-  --auth-choice volcengine-api-key \
-  --volcengine-api-key "$VOLCANO_ENGINE_API_KEY"
-```
-
-</Tip>
-
-## Providers and endpoints
-
-| Provider          | Endpoint                                  | Use case       |
-| ----------------- | ----------------------------------------- | -------------- |
-| `volcengine`      | `ark.cn-beijing.volces.com/api/v3`        | General models |
-| `volcengine-plan` | `ark.cn-beijing.volces.com/api/coding/v3` | Coding models  |
-
-<Note>
-Both providers are configured from a single API key. Setup registers both automatically.
-</Note>
-
-## Built-in catalog
-
-<Tabs>
-  <Tab title="General (volcengine)">
-    | Model ref                                    | Name                            | Input       | Context |
-    | -------------------------------------------- | ------------------------------- | ----------- | ------- |
-    | `volcengine/doubao-seed-1-8-251228`          | Doubao Seed 1.8                 | text, image | 256,000 |
-    | `volcengine/doubao-seed-code-preview-251028` | doubao-seed-code-preview-251028 | text, image | 256,000 |
-    | `volcengine/kimi-k2-5-260127`                | Kimi K2.5                       | text, image | 256,000 |
-    | `volcengine/glm-4-7-251222`                  | GLM 4.7                         | text, image | 200,000 |
-    | `volcengine/deepseek-v3-2-251201`            | DeepSeek V3.2                   | text, image | 128,000 |
-  </Tab>
-  <Tab title="Coding (volcengine-plan)">
-    | Model ref                                         | Name                     | Input | Context |
-    | ------------------------------------------------- | ------------------------ | ----- | ------- |
-    | `volcengine-plan/ark-code-latest`                 | Ark Coding Plan          | text  | 256,000 |
-    | `volcengine-plan/doubao-seed-code`                | Doubao Seed Code         | text  | 256,000 |
-    | `volcengine-plan/glm-4.7`                         | GLM 4.7 Coding           | text  | 200,000 |
-    | `volcengine-plan/kimi-k2-thinking`                | Kimi K2 Thinking         | text  | 256,000 |
-    | `volcengine-plan/kimi-k2.5`                       | Kimi K2.5 Coding         | text  | 256,000 |
-    | `volcengine-plan/doubao-seed-code-preview-251028` | Doubao Seed Code Preview | text  | 256,000 |
-  </Tab>
-</Tabs>
-
-## Text-to-speech
-
-Volcengine TTS uses the BytePlus Seed Speech HTTP API and is configured
-separately from the OpenAI-compatible Doubao model API key. In the BytePlus
-console, open Seed Speech > Settings > API Keys and copy the API key, then set:
-
-```bash
-export VOLCENGINE_TTS_API_KEY="byteplus_seed_speech_api_key"
-export VOLCENGINE_TTS_RESOURCE_ID="seed-tts-1.0"
-```
-
-Then enable it in `fengming.json`:
-
-```json5
-{
-  messages: {
-    tts: {
-      auto: "always",
-      provider: "volcengine",
-      providers: {
-        volcengine: {
-          apiKey: "byteplus_seed_speech_api_key",
-          voice: "en_female_anna_mars_bigtts",
-          speedRatio: 1.0,
-        },
-      },
-    },
-  },
-}
-```
-
-For voice-note targets, FengMing asks Volcengine for provider-native
-`ogg_opus`. For normal audio attachments, it asks for `mp3`. Provider aliases
-`bytedance` and `doubao` also resolve to the same speech provider.
-
-The default resource id is `seed-tts-1.0` because that is what BytePlus grants
-to newly created Seed Speech API keys in the default project. If your project
-has TTS 2.0 entitlement, set `VOLCENGINE_TTS_RESOURCE_ID=seed-tts-2.0`.
-
-<Warning>
-`VOLCANO_ENGINE_API_KEY` is for the ModelArk/Doubao model endpoints and is not a
-Seed Speech API key. TTS needs a Seed Speech API key from the BytePlus Speech
-Console, or a legacy Speech Console AppID/token pair.
-</Warning>
-
-Legacy AppID/token auth remains supported for older Speech Console applications:
-
-```bash
-export VOLCENGINE_TTS_APPID="speech_app_id"
-export VOLCENGINE_TTS_TOKEN="speech_access_token"
-export VOLCENGINE_TTS_CLUSTER="volcano_tts"
-```
-
-## Advanced configuration
-
-<AccordionGroup>
-  <Accordion title="Default model after onboarding">
-    `fengming onboard --auth-choice volcengine-api-key` currently sets
-    `volcengine-plan/ark-code-latest` as the default model while also registering
-    the general `volcengine` catalog.
-  </Accordion>
-
-  <Accordion title="Model picker fallback behavior">
-    During onboarding/configure model selection, the Volcengine auth choice prefers
-    both `volcengine/*` and `volcengine-plan/*` rows. If those models are not
-    loaded yet, FengMing falls back to the unfiltered catalog instead of showing an
-    empty provider-scoped picker.
-  </Accordion>
-
-  <Accordion title="Environment variables for daemon processes">
-    If the Gateway runs as a daemon (launchd/systemd), make sure model and TTS
-    env vars such as `VOLCANO_ENGINE_API_KEY`, `VOLCENGINE_TTS_API_KEY`,
-    `BYTEPLUS_SEED_SPEECH_API_KEY`, `VOLCENGINE_TTS_APPID`, and
-    `VOLCENGINE_TTS_TOKEN` are available to that process (for example, in
-    `~/.fengming/.env` or via `env.shellEnv`).
-  </Accordion>
-</AccordionGroup>
-
-<Warning>
-When running FengMing as a background service, environment variables set in your
-interactive shell are not automatically inherited. See the daemon note above.
-</Warning>
-
-## Related
-
-<CardGroup cols={2}>
-  <Card title="Model selection" href="/concepts/model-providers" icon="layers">
-    Choosing providers, model refs, and failover behavior.
-  </Card>
-  <Card title="Configuration" href="/gateway/configuration" icon="gear">
-    Full config reference for agents, models, and providers.
-  </Card>
-  <Card title="Troubleshooting" href="/help/troubleshooting" icon="wrench">
-    Common issues and debugging steps.
-  </Card>
-  <Card title="FAQ" href="/help/faq" icon="circle-question">
-    Frequently asked questions about FengMing setup.
-  </Card>
-</CardGroup>