fengming 0.3.9 → 0.3.11

package/docs/tools/music-generation.md

@@ -1,372 +0,0 @@
----
-summary: "Generate music via music_generate across ComfyUI, fal, Google Lyria, MiniMax, and OpenRouter workflows"
-read_when:
-  - Generating music or audio via the agent
-  - Configuring music-generation providers and models
-  - Understanding the music_generate tool parameters
-title: "Music generation"
-sidebarTitle: "Music generation"
----
-
-The `music_generate` tool lets the agent create music or audio through the
-shared music-generation capability with configured providers — ComfyUI,
-fal, Google, MiniMax, and OpenRouter today.
-
-For session-backed agent runs, FengMing starts music generation as a
-background task, tracks it in the task ledger, then wakes the agent again
-when the track is ready so the agent can tell the user and attach the
-finished audio. The completion agent follows the session's normal visible-reply
-mode: automatic final reply delivery when configured, or `message(action="send")`
-when the session requires the message tool. If the requester session is
-inactive or its active wake fails, and some generated audio is still missing
-from the completion reply, FengMing sends an idempotent direct fallback with
-only the missing audio.
-
-<Note>
-The built-in shared tool only appears when at least one music-generation
-provider is available. If you do not see `music_generate` in your agent's
-tools, configure `agents.defaults.musicGenerationModel` or set up a
-provider API key.
-</Note>
-
-## Quick start
-
-<Tabs>
-  <Tab title="Shared provider-backed">
-    <Steps>
-      <Step title="Configure auth">
-        Set an API key for at least one provider — for example
-        `GEMINI_API_KEY` or `MINIMAX_API_KEY`.
-      </Step>
-      <Step title="Pick a default model (optional)">
-        ```json5
-        {
-          agents: {
-            defaults: {
-              musicGenerationModel: {
-                primary: "google/lyria-3-clip-preview",
-              },
-            },
-          },
-        }
-        ```
-      </Step>
-      <Step title="Ask the agent">
-        _"Generate an upbeat synthpop track about a night drive through a
-        neon city."_
-
-        The agent calls `music_generate` automatically. No tool
-        allow-listing needed.
-      </Step>
-    </Steps>
-
-    For direct synchronous contexts without a session-backed agent run,
-    the built-in tool still falls back to inline generation and returns
-    the final media path in the tool result.
-
-  </Tab>
-  <Tab title="ComfyUI workflow">
-    <Steps>
-      <Step title="Configure the workflow">
-        Configure `plugins.entries.comfy.config.music` with a workflow
-        JSON and prompt/output nodes.
-      </Step>
-      <Step title="Cloud auth (optional)">
-        For Comfy Cloud, set `COMFY_API_KEY` or `COMFY_CLOUD_API_KEY`.
-      </Step>
-      <Step title="Call the tool">
-        ```text
-        /tool music_generate prompt="Warm ambient synth loop with soft tape texture"
-        ```
-      </Step>
-    </Steps>
-  </Tab>
-</Tabs>
-
-Example prompts:
-
-```text
-Generate a cinematic piano track with soft strings and no vocals.
-```
-
-```text
-Generate an energetic chiptune loop about launching a rocket at sunrise.
-```
-
-## Supported providers
-
-| Provider   | Default model                | Reference inputs | Supported controls                                    | Auth                                   |
-| ---------- | ---------------------------- | ---------------- | ----------------------------------------------------- | -------------------------------------- |
-| ComfyUI    | `workflow`                   | Up to 1 image    | Workflow-defined music or audio                       | `COMFY_API_KEY`, `COMFY_CLOUD_API_KEY` |
-| fal        | `fal-ai/minimax-music/v2.6`  | None             | `lyrics`, `instrumental`, `durationSeconds`, `format` | `FAL_KEY` or `FAL_API_KEY`             |
-| Google     | `lyria-3-clip-preview`       | Up to 10 images  | `lyrics`, `instrumental`, `format`                    | `GEMINI_API_KEY`, `GOOGLE_API_KEY`     |
-| MiniMax    | `music-2.6`                  | None             | `lyrics`, `instrumental`, `format=mp3`                | `MINIMAX_API_KEY` or MiniMax OAuth     |
-| OpenRouter | `google/lyria-3-pro-preview` | Up to 1 image    | `lyrics`, `instrumental`, `durationSeconds`, `format` | `OPENROUTER_API_KEY`                   |
-
-### Capability matrix
-
-The explicit mode contract used by `music_generate`, contract tests, and the
-shared live sweep:
-
-| Provider   | `generate` | `edit` | Edit limit | Shared live lanes                                                         |
-| ---------- | :--------: | :----: | ---------- | ------------------------------------------------------------------------- |
-| ComfyUI    |     ✓      |   ✓    | 1 image    | Not in the shared sweep; covered by `extensions/comfy/comfy.live.test.ts` |
-| fal        |     ✓      |   —    | None       | `generate`                                                                |
-| Google     |     ✓      |   ✓    | 10 images  | `generate`, `edit`                                                        |
-| MiniMax    |     ✓      |   —    | None       | `generate`                                                                |
-| OpenRouter |     ✓      |   ✓    | 1 image    | `generate`, `edit`                                                        |
-
-Use `action: "list"` to inspect available shared providers and models at
-runtime:
-
-```text
-/tool music_generate action=list
-```
-
-Use `action: "status"` to inspect the active session-backed music task:
-
-```text
-/tool music_generate action=status
-```
-
-Direct generation example:
-
-```text
-/tool music_generate prompt="Dreamy lo-fi hip hop with vinyl texture and gentle rain" instrumental=true
-```
-
-## Tool parameters
-
-<ParamField path="prompt" type="string" required>
-  Music generation prompt. Required for `action: "generate"`.
-</ParamField>
-<ParamField path="action" type='"generate" | "status" | "list"' default="generate">
-  `"status"` returns the current session task; `"list"` inspects providers.
-</ParamField>
-<ParamField path="model" type="string">
-  Provider/model override (e.g. `google/lyria-3-pro-preview`,
-  `comfy/workflow`).
-</ParamField>
-<ParamField path="lyrics" type="string">
-  Optional lyrics when the provider supports explicit lyric input.
-</ParamField>
-<ParamField path="instrumental" type="boolean">
-  Request instrumental-only output when the provider supports it.
-</ParamField>
-<ParamField path="image" type="string">
-  Single reference image path or URL.
-</ParamField>
-<ParamField path="images" type="string[]">
-  Multiple reference images (up to 10 on supporting providers).
-</ParamField>
-<ParamField path="durationSeconds" type="number">
-  Target duration in seconds when the provider supports duration hints.
-</ParamField>
-<ParamField path="format" type='"mp3" | "wav"'>
-  Output format hint when the provider supports it.
-</ParamField>
-<ParamField path="filename" type="string">Output filename hint.</ParamField>
-
-<Note>
-Not all providers support all parameters. FengMing still validates hard
-limits such as input counts before submission. When a provider supports
-duration but uses a shorter maximum than the requested value, FengMing
-clamps to the closest supported duration. Truly unsupported optional hints
-are ignored with a warning when the selected provider or model cannot honor
-them. Tool results report applied settings; `details.normalization`
-captures any requested-to-applied mapping.
-</Note>
-
-Provider request timeouts are operator configuration only. FengMing uses
-`agents.defaults.musicGenerationModel.timeoutMs` when configured, raises values
-below 120000ms to 120000ms, and otherwise defaults provider requests to
-300000ms.
-
-## Async behavior
-
-Session-backed music generation runs as a background task:
-
-- **Background task:** `music_generate` creates a background task, returns a
-  started/task response immediately, and posts the finished track later in
-  a follow-up agent message.
-- **Duplicate prevention:** while a task is `queued` or `running`, later
-  `music_generate` calls in the same session return task status instead of
-  starting another generation. Use `action: "status"` to check explicitly.
-- **Status lookup:** `fengming tasks list` or `fengming tasks show <taskId>`
-  inspects queued, running, and terminal status.
-- **Completion wake:** FengMing injects an internal completion event back
-  into the same session so the model can write the user-facing follow-up
-  itself.
-- **Prompt hint:** later user/manual turns in the same session get a small
-  runtime hint when a music task is already in flight, so the model does
-  not blindly call `music_generate` again.
-- **No-session fallback:** direct/local contexts without a real agent
-  session run inline and return the final audio result in the same turn.
-
-### Task lifecycle
-
-| State       | Meaning                                                                                        |
-| ----------- | ---------------------------------------------------------------------------------------------- |
-| `queued`    | Task created, waiting for the provider to accept it.                                           |
-| `running`   | Provider is processing (typically 30 seconds to 3 minutes depending on provider and duration). |
-| `succeeded` | Track ready; the agent wakes and posts it to the conversation.                                 |
-| `failed`    | Provider error or timeout; the agent wakes with error details.                                 |
-
-Check status from the CLI:
-
-```bash
-fengming tasks list
-fengming tasks show <taskId>
-fengming tasks cancel <taskId>
-```
-
-## Configuration
-
-### Model selection
-
-```json5
-{
-  agents: {
-    defaults: {
-      musicGenerationModel: {
-        primary: "google/lyria-3-clip-preview",
-        fallbacks: ["fal/fal-ai/minimax-music/v2.6", "minimax/music-2.6"],
-      },
-    },
-  },
-}
-```
-
-### Provider selection order
-
-FengMing tries providers in this order:
-
-1. `model` parameter from the tool call (if the agent specifies one).
-2. `musicGenerationModel.primary` from config.
-3. `musicGenerationModel.fallbacks` in order.
-4. Auto-detection using auth-backed provider defaults only:
-   - current default provider first;
-   - remaining registered music-generation providers in provider-id order.
-
-If a provider fails, the next candidate is tried automatically. If all
-fail, the error includes details from each attempt.
-
-Set `agents.defaults.mediaGenerationAutoProviderFallback: false` to use only
-explicit `model`, `primary`, and `fallbacks` entries.
-
-## Provider notes
-
-<AccordionGroup>
-  <Accordion title="ComfyUI">
-    Workflow-driven and depends on the configured graph plus node mapping
-    for prompt/output fields. The bundled `comfy` plugin plugs into the
-    shared `music_generate` tool through the music-generation provider
-    registry.
-  </Accordion>
-  <Accordion title="fal">
-    Uses fal model endpoints through the shared provider auth path. The
-    bundled provider defaults to `fal-ai/minimax-music/v2.6` and also exposes
-    `fal-ai/ace-step/prompt-to-audio` and
-    `fal-ai/stable-audio-25/text-to-audio` for prompt-to-audio requests.
-  </Accordion>
-  <Accordion title="Google (Lyria 3)">
-    Uses Lyria 3 batch generation. The current bundled flow supports
-    prompt, optional lyrics text, and optional reference images.
-  </Accordion>
-  <Accordion title="MiniMax">
-    Uses the batch `music_generation` endpoint. Supports prompt, optional
-    lyrics, instrumental mode, and mp3 output through either `minimax`
-    API-key auth or `minimax-portal` OAuth.
-  </Accordion>
-  <Accordion title="OpenRouter">
-    Uses OpenRouter chat completions audio output with streaming enabled. The
-    bundled provider defaults to `google/lyria-3-pro-preview` and also exposes
-    `openrouter/google/lyria-3-clip-preview`.
-  </Accordion>
-</AccordionGroup>
-
-## Choosing the right path
-
-- **Shared provider-backed** when you want model selection, provider
-  failover, and the built-in async task/status flow.
-- **Plugin path (ComfyUI)** when you need a custom workflow graph or a
-  provider that is not part of the shared bundled music capability.
-
-If you are debugging ComfyUI-specific behavior, see
-[ComfyUI](/providers/comfy). If you are debugging shared provider
-behavior, start with [fal](/providers/fal), [Google (Gemini)](/providers/google),
-[MiniMax](/providers/minimax), or [OpenRouter](/providers/openrouter).
-
-## Provider capability modes
-
-The shared music-generation contract supports explicit mode declarations:
-
-- `generate` for prompt-only generation.
-- `edit` when the request includes one or more reference images.
-
-New provider implementations should prefer explicit mode blocks:
-
-```typescript
-capabilities: {
-  generate: {
-    maxTracks: 1,
-    supportsLyrics: true,
-    supportsFormat: true,
-  },
-  edit: {
-    enabled: true,
-    maxTracks: 1,
-    maxInputImages: 1,
-    supportsFormat: true,
-  },
-}
-```
-
-Legacy flat fields such as `maxInputImages`, `supportsLyrics`, and
-`supportsFormat` are **not** enough to advertise edit support. Providers
-should declare `generate` and `edit` explicitly so live tests, contract
-tests, and the shared `music_generate` tool can validate mode support
-deterministically.
-
-## Live tests
-
-Opt-in live coverage for the shared bundled providers:
-
-```bash
-FENGMING_LIVE_TEST=1 pnpm test:live -- extensions/music-generation-providers.live.test.ts
-```
-
-Repo wrapper:
-
-```bash
-pnpm test:live:media music
-```
-
-This live file uses already-exported provider env vars ahead of stored auth
-profiles by default, and runs both `generate` and declared `edit` coverage when
-the provider enables edit mode. Coverage today:
-
-- `google`: `generate` plus `edit`
-- `fal`: `generate` only
-- `minimax`: `generate` only
-- `openrouter`: `generate` plus `edit`
-- `comfy`: separate Comfy live coverage, not the shared provider sweep
-
-Opt-in live coverage for the bundled ComfyUI music path:
-
-```bash
-FENGMING_LIVE_TEST=1 COMFY_LIVE_TEST=1 pnpm test:live -- extensions/comfy/comfy.live.test.ts
-```
-
-The Comfy live file also covers comfy image and video workflows when those
-sections are configured.
-
-## Related
-
-- [Background tasks](/automation/tasks) — task tracking for detached `music_generate` runs
-- [ComfyUI](/providers/comfy)
-- [Configuration reference](/gateway/config-agents#agent-defaults) — `musicGenerationModel` config
-- [Google (Gemini)](/providers/google)
-- [MiniMax](/providers/minimax)
-- [Models](/concepts/models) — model configuration and failover
-- [Tools overview](/tools)

package/docs/tools/ollama-search.md

@@ -1,153 +0,0 @@
----
-summary: "Ollama Web Search via a local Ollama host or the hosted Ollama API"
-read_when:
-  - You want to use Ollama for web_search
-  - You want a key-free web_search provider
-  - You want to use hosted Ollama Web Search with OLLAMA_API_KEY
-  - You need Ollama Web Search setup guidance
-title: "Ollama web search"
----
-
-FengMing supports **Ollama Web Search** as a bundled `web_search` provider. It
-uses Ollama's web-search API and returns structured results with titles, URLs,
-and snippets.
-
-For local or self-hosted Ollama, this setup does not need an API key by
-default. It does require:
-
-- an Ollama host that is reachable from FengMing
-- `ollama signin`
-
-For direct hosted search, set the Ollama provider base URL to `https://ollama.com`
-and provide a real `OLLAMA_API_KEY`.
-
-## Setup
-
-<Steps>
-  <Step title="Start Ollama">
-    Make sure Ollama is installed and running.
-  </Step>
-  <Step title="Sign in">
-    Run:
-
-    ```bash
-    ollama signin
-    ```
-
-  </Step>
-  <Step title="Choose Ollama Web Search">
-    Run:
-
-    ```bash
-    fengming configure --section web
-    ```
-
-    Then select **Ollama Web Search** as the provider.
-
-  </Step>
-</Steps>
-
-If you already use Ollama for models, Ollama Web Search reuses the same
-configured host.
-
-## Config
-
-```json5
-{
-  tools: {
-    web: {
-      search: {
-        provider: "ollama",
-      },
-    },
-  },
-}
-```
-
-Optional Ollama host override:
-
-```json5
-{
-  plugins: {
-    entries: {
-      ollama: {
-        config: {
-          webSearch: {
-            baseUrl: "http://ollama-host:11434",
-          },
-        },
-      },
-    },
-  },
-}
-```
-
-If you already configure Ollama as a model provider, the web-search provider can
-reuse that host instead:
-
-```json5
-{
-  models: {
-    providers: {
-      ollama: {
-        baseUrl: "http://ollama-host:11434",
-      },
-    },
-  },
-}
-```
-
-The Ollama model provider uses `baseUrl` as the canonical key. The web-search provider also honors `baseURL` on `models.providers.ollama` for compatibility with OpenAI SDK-style config examples.
-
-If no explicit Ollama base URL is set, FengMing uses `http://127.0.0.1:11434`.
-
-If your Ollama host expects bearer auth, FengMing reuses
-`models.providers.ollama.apiKey` (or the matching env-backed provider auth)
-for requests to that configured host.
-
-Direct hosted Ollama Web Search:
-
-```json5
-{
-  models: {
-    providers: {
-      ollama: {
-        baseUrl: "https://ollama.com",
-        apiKey: "OLLAMA_API_KEY",
-      },
-    },
-  },
-  tools: {
-    web: {
-      search: {
-        provider: "ollama",
-      },
-    },
-  },
-}
-```
-
-## Notes
-
-- No web-search-specific API key field is required for this provider.
-- If the Ollama host is auth-protected, FengMing reuses the normal Ollama
-  provider API key when present.
-- If `baseUrl` is `https://ollama.com`, FengMing calls
-  `https://ollama.com/api/web_search` directly and sends the configured Ollama
-  API key as bearer auth.
-- If the configured host does not expose web search and `OLLAMA_API_KEY` is set,
-  FengMing can fall back to `https://ollama.com/api/web_search` without sending
-  that env key to the local host.
-- FengMing warns during setup if Ollama is unreachable or not signed in, but
-  it does not block selection.
-- Runtime auto-detect can fall back to Ollama Web Search when no higher-priority
-  credentialed provider is configured.
-- Local Ollama daemon hosts use the local proxy endpoint
-  `/api/experimental/web_search`, which signs and forwards to Ollama Cloud.
-- `https://ollama.com` hosts use the public hosted endpoint
-  `/api/web_search` directly with bearer API-key auth.
-
-## Related
-
-- [Web Search overview](/tools/web) -- all providers and auto-detection
-- [Ollama](/providers/ollama) -- Ollama model setup and cloud/local modes