nodmix 2026.5.25

package/docs/providers/comfy.md

@@ -0,0 +1,362 @@
+---
+summary: "ComfyUI workflow image, video, and music generation setup in Nodmix"
+title: "ComfyUI"
+read_when:
+  - You want to use local ComfyUI workflows with Nodmix
+  - You want to use Comfy Cloud with image, video, or music workflows
+  - You need the bundled comfy plugin config keys
+---
+
+Nodmix ships a bundled `comfy` plugin for workflow-driven ComfyUI runs. The plugin is entirely workflow-driven, so Nodmix does not try to map generic `size`, `aspectRatio`, `resolution`, `durationSeconds`, or TTS-style controls onto your graph.
+
+| Property        | Detail                                                                           |
+| --------------- | -------------------------------------------------------------------------------- |
+| Provider        | `comfy`                                                                          |
+| Models          | `comfy/workflow`                                                                 |
+| Shared surfaces | `image_generate`, `video_generate`, `music_generate`                             |
+| Auth            | None for local ComfyUI; `COMFY_API_KEY` or `COMFY_CLOUD_API_KEY` for Comfy Cloud |
+| API             | ComfyUI `/prompt` / `/history` / `/view` and Comfy Cloud `/api/*`                |
+
+## What it supports
+
+- Image generation from a workflow JSON
+- Image editing with 1 uploaded reference image
+- Video generation from a workflow JSON
+- Video generation with 1 uploaded reference image
+- Music or audio generation through the shared `music_generate` tool
+- Output download from a configured node or all matching output nodes
+
+## Getting started
+
+Choose between running ComfyUI on your own machine or using Comfy Cloud.
+
+<Tabs>
+  <Tab title="Local">
+    **Best for:** running your own ComfyUI instance on your machine or LAN.
+
+    <Steps>
+      <Step title="Start ComfyUI locally">
+        Make sure your local ComfyUI instance is running (defaults to `http://127.0.0.1:8188`).
+      </Step>
+      <Step title="Prepare your workflow JSON">
+        Export or create a ComfyUI workflow JSON file. Note the node IDs for the prompt input node and the output node you want Nodmix to read from.
+      </Step>
+      <Step title="Configure the provider">
+        Set `mode: "local"` and point at your workflow file. Here is a minimal image example:
+
+        ```json5
+        {
+          plugins: {
+            entries: {
+              comfy: {
+                config: {
+                  mode: "local",
+                  baseUrl: "http://127.0.0.1:8188",
+                  image: {
+                    workflowPath: "./workflows/flux-api.json",
+                    promptNodeId: "6",
+                    outputNodeId: "9",
+                  },
+                },
+              },
+            },
+          },
+        }
+        ```
+      </Step>
+      <Step title="Set the default model">
+        Point Nodmix at the `comfy/workflow` model for the capability you configured:
+
+        ```json5
+        {
+          agents: {
+            defaults: {
+              imageGenerationModel: {
+                primary: "comfy/workflow",
+              },
+            },
+          },
+        }
+        ```
+      </Step>
+      <Step title="Verify">
+        ```bash
+        nodmix models list --provider comfy
+        ```
+      </Step>
+    </Steps>
+
+  </Tab>
+
+  <Tab title="Comfy Cloud">
+    **Best for:** running workflows on Comfy Cloud without managing local GPU resources.
+
+    <Steps>
+      <Step title="Get an API key">
+        Sign up at [comfy.org](https://comfy.org) and generate an API key from your account dashboard.
+      </Step>
+      <Step title="Set the API key">
+        Provide your key through one of these methods:
+
+        ```bash
+        # Environment variable (preferred)
+        export COMFY_API_KEY="your-key"
+
+        # Alternative environment variable
+        export COMFY_CLOUD_API_KEY="your-key"
+
+        # Or inline in config
+        nodmix config set plugins.entries.comfy.config.apiKey "your-key"
+        ```
+      </Step>
+      <Step title="Prepare your workflow JSON">
+        Export or create a ComfyUI workflow JSON file. Note the node IDs for the prompt input node and the output node.
+      </Step>
+      <Step title="Configure the provider">
+        Set `mode: "cloud"` and point at your workflow file:
+
+        ```json5
+        {
+          plugins: {
+            entries: {
+              comfy: {
+                config: {
+                  mode: "cloud",
+                  image: {
+                    workflowPath: "./workflows/flux-api.json",
+                    promptNodeId: "6",
+                    outputNodeId: "9",
+                  },
+                },
+              },
+            },
+          },
+        }
+        ```
+
+        <Tip>
+        Cloud mode defaults `baseUrl` to `https://cloud.comfy.org`. You only need to set `baseUrl` if you use a custom cloud endpoint.
+        </Tip>
+      </Step>
+      <Step title="Set the default model">
+        ```json5
+        {
+          agents: {
+            defaults: {
+              imageGenerationModel: {
+                primary: "comfy/workflow",
+              },
+            },
+          },
+        }
+        ```
+      </Step>
+      <Step title="Verify">
+        ```bash
+        nodmix models list --provider comfy
+        ```
+      </Step>
+    </Steps>
+
+  </Tab>
+</Tabs>
+
+## Configuration
+
+Comfy supports shared top-level connection settings plus per-capability workflow sections (`image`, `video`, `music`):
+
+```json5
+{
+  plugins: {
+    entries: {
+      comfy: {
+        config: {
+          mode: "local",
+          baseUrl: "http://127.0.0.1:8188",
+          image: {
+            workflowPath: "./workflows/flux-api.json",
+            promptNodeId: "6",
+            outputNodeId: "9",
+          },
+          video: {
+            workflowPath: "./workflows/video-api.json",
+            promptNodeId: "12",
+            outputNodeId: "21",
+          },
+          music: {
+            workflowPath: "./workflows/music-api.json",
+            promptNodeId: "3",
+            outputNodeId: "18",
+          },
+        },
+      },
+    },
+  },
+}
+```
+
+### Shared keys
+
+| Key                   | Type                   | Description                                                                           |
+| --------------------- | ---------------------- | ------------------------------------------------------------------------------------- |
+| `mode`                | `"local"` or `"cloud"` | Connection mode.                                                                      |
+| `baseUrl`             | string                 | Defaults to `http://127.0.0.1:8188` for local or `https://cloud.comfy.org` for cloud. |
+| `apiKey`              | string                 | Optional inline key, alternative to `COMFY_API_KEY` / `COMFY_CLOUD_API_KEY` env vars. |
+| `allowPrivateNetwork` | boolean                | Allow a private/LAN `baseUrl` in cloud mode.                                          |
+
+### Per-capability keys
+
+These keys apply inside the `image`, `video`, or `music` sections:
+
+| Key                          | Required | Default  | Description                                                                  |
+| ---------------------------- | -------- | -------- | ---------------------------------------------------------------------------- |
+| `workflow` or `workflowPath` | Yes      | --       | Path to the ComfyUI workflow JSON file.                                      |
+| `promptNodeId`               | Yes      | --       | Node ID that receives the text prompt.                                       |
+| `promptInputName`            | No       | `"text"` | Input name on the prompt node.                                               |
+| `outputNodeId`               | No       | --       | Node ID to read output from. If omitted, all matching output nodes are used. |
+| `pollIntervalMs`             | No       | --       | Polling interval in milliseconds for job completion.                         |
+| `timeoutMs`                  | No       | --       | Timeout in milliseconds for the workflow run.                                |
+
+The `image` and `video` sections also support:
+
+| Key                   | Required                             | Default   | Description                                         |
+| --------------------- | ------------------------------------ | --------- | --------------------------------------------------- |
+| `inputImageNodeId`    | Yes (when passing a reference image) | --        | Node ID that receives the uploaded reference image. |
+| `inputImageInputName` | No                                   | `"image"` | Input name on the image node.                       |
+
+## Workflow details
+
+<AccordionGroup>
+  <Accordion title="Image workflows">
+    Set the default image model to `comfy/workflow`:
+
+    ```json5
+    {
+      agents: {
+        defaults: {
+          imageGenerationModel: {
+            primary: "comfy/workflow",
+          },
+        },
+      },
+    }
+    ```
+
+    **Reference-image editing example:**
+
+    To enable image editing with an uploaded reference image, add `inputImageNodeId` to your image config:
+
+    ```json5
+    {
+      plugins: {
+        entries: {
+          comfy: {
+            config: {
+              image: {
+                workflowPath: "./workflows/edit-api.json",
+                promptNodeId: "6",
+                inputImageNodeId: "7",
+                inputImageInputName: "image",
+                outputNodeId: "9",
+              },
+            },
+          },
+        },
+      },
+    }
+    ```
+
+  </Accordion>
+
+  <Accordion title="Video workflows">
+    Set the default video model to `comfy/workflow`:
+
+    ```json5
+    {
+      agents: {
+        defaults: {
+          videoGenerationModel: {
+            primary: "comfy/workflow",
+          },
+        },
+      },
+    }
+    ```
+
+    Comfy video workflows support text-to-video and image-to-video through the configured graph.
+
+    <Note>
+    Nodmix does not pass input videos into Comfy workflows. Only text prompts and single reference images are supported as inputs.
+    </Note>
+
+  </Accordion>
+
+  <Accordion title="Music workflows">
+    The bundled plugin registers a music-generation provider for workflow-defined audio or music outputs, surfaced through the shared `music_generate` tool:
+
+    ```text
+    /tool music_generate prompt="Warm ambient synth loop with soft tape texture"
+    ```
+
+    Use the `music` config section to point at your audio workflow JSON and output node.
+
+  </Accordion>
+
+  <Accordion title="Backward compatibility">
+    Existing top-level image config (without the nested `image` section) still works:
+
+    ```json5
+    {
+      plugins: {
+        entries: {
+          comfy: {
+            config: {
+              workflowPath: "./workflows/flux-api.json",
+              promptNodeId: "6",
+              outputNodeId: "9",
+            },
+          },
+        },
+      },
+    }
+    ```
+
+    Nodmix treats that legacy shape as the image workflow config. You do not need to migrate immediately, but the nested `image` / `video` / `music` sections are recommended for new setups.
+
+    <Tip>
+    If you only use image generation, the legacy flat config and the new nested `image` section are functionally equivalent.
+    </Tip>
+
+  </Accordion>
+
+  <Accordion title="Live tests">
+    Opt-in live coverage exists for the bundled plugin:
+
+    ```bash
+    NODMIX_LIVE_TEST=1 COMFY_LIVE_TEST=1 pnpm test:live -- extensions/comfy/comfy.live.test.ts
+    ```
+
+    The live test skips individual image, video, or music cases unless the matching Comfy workflow section is configured.
+
+  </Accordion>
+</AccordionGroup>
+
+## Related
+
+<CardGroup cols={2}>
+  <Card title="Image Generation" href="/tools/image-generation" icon="image">
+    Image generation tool configuration and usage.
+  </Card>
+  <Card title="Video Generation" href="/tools/video-generation" icon="video">
+    Video generation tool configuration and usage.
+  </Card>
+  <Card title="Music Generation" href="/tools/music-generation" icon="music">
+    Music and audio generation tool setup.
+  </Card>
+  <Card title="Provider Directory" href="/providers/index" icon="layers">
+    Overview of all providers and model refs.
+  </Card>
+  <Card title="Configuration reference" href="/gateway/config-agents#agent-defaults" icon="gear">
+    Full config reference including agent defaults.
+  </Card>
+</CardGroup>

package/docs/providers/deepgram.md

@@ -0,0 +1,184 @@
+---
+summary: "Deepgram transcription for inbound voice notes"
+read_when:
+  - You want Deepgram speech-to-text for audio attachments
+  - You want Deepgram streaming transcription for Voice Call
+  - You need a quick Deepgram config example
+title: "Deepgram"
+---
+
+Deepgram is a speech-to-text API. In Nodmix it is used for inbound
+audio/voice-note transcription through `tools.media.audio` and for Voice Call
+streaming STT through `plugins.entries.voice-call.config.streaming`.
+
+For batch transcription, Nodmix uploads the complete audio file to Deepgram
+and injects the transcript into the reply pipeline (`{{Transcript}}` +
+`[Audio]` block). For Voice Call streaming, Nodmix forwards live G.711
+u-law frames over Deepgram's WebSocket `listen` endpoint and emits partial or
+final transcripts as Deepgram returns them.
+
+| Detail        | Value                                                      |
+| ------------- | ---------------------------------------------------------- |
+| Website       | [deepgram.com](https://deepgram.com)                       |
+| Docs          | [developers.deepgram.com](https://developers.deepgram.com) |
+| Auth          | `DEEPGRAM_API_KEY`                                         |
+| Default model | `nova-3`                                                   |
+
+## Getting started
+
+<Steps>
+  <Step title="Set your API key">
+    Add your Deepgram API key to the environment:
+
+    ```
+    DEEPGRAM_API_KEY=dg_...
+    ```
+
+  </Step>
+  <Step title="Enable the audio provider">
+    ```json5
+    {
+      tools: {
+        media: {
+          audio: {
+            enabled: true,
+            models: [{ provider: "deepgram", model: "nova-3" }],
+          },
+        },
+      },
+    }
+    ```
+  </Step>
+  <Step title="Send a voice note">
+    Send an audio message through any connected channel. Nodmix transcribes it
+    via Deepgram and injects the transcript into the reply pipeline.
+  </Step>
+</Steps>
+
+## Configuration options
+
+| Option            | Path                                                         | Description                           |
+| ----------------- | ------------------------------------------------------------ | ------------------------------------- |
+| `model`           | `tools.media.audio.models[].model`                           | Deepgram model id (default: `nova-3`) |
+| `language`        | `tools.media.audio.models[].language`                        | Language hint (optional)              |
+| `detect_language` | `tools.media.audio.providerOptions.deepgram.detect_language` | Enable language detection (optional)  |
+| `punctuate`       | `tools.media.audio.providerOptions.deepgram.punctuate`       | Enable punctuation (optional)         |
+| `smart_format`    | `tools.media.audio.providerOptions.deepgram.smart_format`    | Enable smart formatting (optional)    |
+
+<Tabs>
+  <Tab title="With language hint">
+    ```json5
+    {
+      tools: {
+        media: {
+          audio: {
+            enabled: true,
+            models: [{ provider: "deepgram", model: "nova-3", language: "en" }],
+          },
+        },
+      },
+    }
+    ```
+  </Tab>
+  <Tab title="With Deepgram options">
+    ```json5
+    {
+      tools: {
+        media: {
+          audio: {
+            enabled: true,
+            providerOptions: {
+              deepgram: {
+                detect_language: true,
+                punctuate: true,
+                smart_format: true,
+              },
+            },
+            models: [{ provider: "deepgram", model: "nova-3" }],
+          },
+        },
+      },
+    }
+    ```
+  </Tab>
+</Tabs>
+
+## Voice Call streaming STT
+
+The bundled `deepgram` plugin also registers a realtime transcription provider
+for the Voice Call plugin.
+
+| Setting         | Config path                                                             | Default                          |
+| --------------- | ----------------------------------------------------------------------- | -------------------------------- |
+| API key         | `plugins.entries.voice-call.config.streaming.providers.deepgram.apiKey` | Falls back to `DEEPGRAM_API_KEY` |
+| Model           | `...deepgram.model`                                                     | `nova-3`                         |
+| Language        | `...deepgram.language`                                                  | (unset)                          |
+| Encoding        | `...deepgram.encoding`                                                  | `mulaw`                          |
+| Sample rate     | `...deepgram.sampleRate`                                                | `8000`                           |
+| Endpointing     | `...deepgram.endpointingMs`                                             | `800`                            |
+| Interim results | `...deepgram.interimResults`                                            | `true`                           |
+
+```json5
+{
+  plugins: {
+    entries: {
+      "voice-call": {
+        config: {
+          streaming: {
+            enabled: true,
+            provider: "deepgram",
+            providers: {
+              deepgram: {
+                apiKey: "${DEEPGRAM_API_KEY}",
+                model: "nova-3",
+                endpointingMs: 800,
+                language: "en-US",
+              },
+            },
+          },
+        },
+      },
+    },
+  },
+}
+```
+
+<Note>
+Voice Call receives telephony audio as 8 kHz G.711 u-law. The Deepgram
+streaming provider defaults to `encoding: "mulaw"` and `sampleRate: 8000`, so
+Twilio media frames can be forwarded directly.
+</Note>
+
+## Notes
+
+<AccordionGroup>
+  <Accordion title="Authentication">
+    Authentication follows the standard provider auth order. `DEEPGRAM_API_KEY` is
+    the simplest path.
+  </Accordion>
+  <Accordion title="Proxy and custom endpoints">
+    Override endpoints or headers with `tools.media.audio.baseUrl` and
+    `tools.media.audio.headers` when using a proxy.
+  </Accordion>
+  <Accordion title="Output behavior">
+    Output follows the same audio rules as other providers (size caps, timeouts,
+    transcript injection).
+  </Accordion>
+</AccordionGroup>
+
+## Related
+
+<CardGroup cols={2}>
+  <Card title="Media tools" href="/tools/media-overview" icon="photo-film">
+    Audio, image, and video processing pipeline overview.
+  </Card>
+  <Card title="Configuration" href="/gateway/configuration" icon="gear">
+    Full config reference including media tool settings.
+  </Card>
+  <Card title="Troubleshooting" href="/help/troubleshooting" icon="wrench">
+    Common issues and debugging steps.
+  </Card>
+  <Card title="FAQ" href="/help/faq" icon="circle-question">
+    Frequently asked questions about Nodmix setup.
+  </Card>
+</CardGroup>

package/docs/providers/deepinfra.md

@@ -0,0 +1,87 @@
+---
+summary: "Use DeepInfra's unified API to access the most popular open source and frontier models in Nodmix"
+read_when:
+  - You want a single API key for the top open source LLMs
+  - You want to run models via DeepInfra's API in Nodmix
+title: "DeepInfra"
+---
+
+DeepInfra provides a **unified API** that routes requests to the most popular open source and frontier models behind a single
+endpoint and API key. It is OpenAI-compatible, so most OpenAI SDKs work by switching the base URL.
+
+## Getting an API key
+
+1. Go to [https://deepinfra.com/](https://deepinfra.com/)
+2. Sign in or create an account
+3. Navigate to Dashboard / Keys and generate a new API key or use the auto created one
+
+## CLI setup
+
+```bash
+nodmix onboard --deepinfra-api-key <key>
+```
+
+Or set the environment variable:
+
+```bash
+export DEEPINFRA_API_KEY="<your-deepinfra-api-key>" # pragma: allowlist secret
+```
+
+## Config snippet
+
+```json5
+{
+  env: { DEEPINFRA_API_KEY: "<your-deepinfra-api-key>" }, // pragma: allowlist secret
+  agents: {
+    defaults: {
+      model: { primary: "deepinfra/deepseek-ai/DeepSeek-V3.2" },
+    },
+  },
+}
+```
+
+## Supported Nodmix surfaces
+
+The bundled plugin registers all DeepInfra surfaces that match current
+Nodmix provider contracts:
+
+| Surface                  | Default model                      | Nodmix config/tool                                     |
+| ------------------------ | ---------------------------------- | -------------------------------------------------------- |
+| Chat / model provider    | `deepseek-ai/DeepSeek-V3.2`        | `agents.defaults.model`                                  |
+| Image generation/editing | `black-forest-labs/FLUX-1-schnell` | `image_generate`, `agents.defaults.imageGenerationModel` |
+| Media understanding      | `moonshotai/Kimi-K2.5` for images  | inbound image understanding                              |
+| Speech-to-text           | `openai/whisper-large-v3-turbo`    | inbound audio transcription                              |
+| Text-to-speech           | `hexgrad/Kokoro-82M`               | `messages.tts.provider: "deepinfra"`                     |
+| Video generation         | `Pixverse/Pixverse-T2V`            | `video_generate`, `agents.defaults.videoGenerationModel` |
+| Memory embeddings        | `BAAI/bge-m3`                      | `agents.defaults.memorySearch.provider: "deepinfra"`     |
+
+DeepInfra also exposes reranking, classification, object-detection, and other
+native model types. Nodmix does not currently have first-class provider
+contracts for those categories, so this plugin does not register them yet.
+
+## Available models
+
+Nodmix dynamically discovers available DeepInfra models at startup. Use
+`/models deepinfra` to see the full list of models available.
+
+Any model available on [DeepInfra.com](https://deepinfra.com/) can be used with the `deepinfra/` prefix:
+
+```
+deepinfra/MiniMaxAI/MiniMax-M2.5
+deepinfra/deepseek-ai/DeepSeek-V3.2
+deepinfra/moonshotai/Kimi-K2.5
+deepinfra/zai-org/GLM-5.1
+...and many more
+```
+
+## Notes
+
+- Model refs are `deepinfra/<provider>/<model>` (e.g., `deepinfra/Qwen/Qwen3-Max`).
+- Default model: `deepinfra/deepseek-ai/DeepSeek-V3.2`
+- Base URL: `https://api.deepinfra.com/v1/openai`
+- Native video generation uses `https://api.deepinfra.com/v1/inference/<model>`.
+
+## Related
+
+- [Model providers](/concepts/model-providers)
+- [All providers](/providers/index)