comfyui-mcp 0.9.1 → 0.9.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -6,6 +6,18 @@ All notable changes to this project are documented here. This project adheres to
6
6
 
7
7
  ## Unreleased
8
8
 
9
+ ### Fixed
10
+
11
+ - **Docker build hang on rate-limited CI (e.g. Glama)** — `npm ci` in the
12
+ Dockerfile no longer runs the `cloudflared` postinstall, which fetches a
13
+ ~40 MB binary from GitHub releases over an `https.get()` call with no
14
+ timeout. On networks where GitHub rate-limits (or otherwise stalls)
15
+ unauthenticated requests, that fetch hung indefinitely and blocked image
16
+ builds. Install scripts are now skipped with `--ignore-scripts` and the
17
+ two native deps we actually need (`better-sqlite3`, `sharp`) are rebuilt
18
+ explicitly. The runtime tunnel helper already downloads the cloudflared
19
+ binary lazily on first use, so no functionality is lost.
20
+
9
21
  ### Added
10
22
 
11
23
  - **`get_job_status` cloud-mode coverage** — when `COMFYUI_API_KEY` is set,
package/Dockerfile CHANGED
@@ -18,11 +18,22 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
18
18
  python3 make g++ \
19
19
  && rm -rf /var/lib/apt/lists/*
20
20
 
21
- # Install against the lockfile. scripts/ is needed because our (safe) postinstall
22
- # runs during install; copying it first keeps the dependency layer cacheable.
21
+ # Install against the lockfile. scripts/ is copied first so the dep layer stays
22
+ # cacheable when only src/ changes.
23
+ #
24
+ # We pass --ignore-scripts to skip ALL install hooks, then explicitly rebuild
25
+ # the native deps we actually need. Why: the optional `cloudflared` package's
26
+ # postinstall downloads a ~40 MB binary from GitHub releases over an
27
+ # https.get() call with no timeout. On rate-limited CI networks (notably
28
+ # Glama's build sandbox) that request hangs indefinitely and the whole image
29
+ # build stalls. The runtime tunnel helper in src/services/tunnel.ts already
30
+ # downloads the binary lazily on first use, so dropping the install-time
31
+ # fetch is safe. better-sqlite3 + sharp still need their `install` scripts
32
+ # to fetch / build their native bindings, hence the explicit rebuild.
23
33
  COPY package.json package-lock.json ./
24
34
  COPY scripts ./scripts
25
- RUN npm ci
35
+ RUN npm ci --ignore-scripts \
36
+ && npm rebuild better-sqlite3 sharp
26
37
 
27
38
  # Compile TypeScript -> dist/
28
39
  COPY tsconfig.json ./
@@ -103,13 +103,42 @@ A Node HTTP server on `localhost:PORT`:
103
103
 
104
104
  ## 6. Build order
105
105
  0. **v2 authoring skill** (enabler — write the extension correctly).
106
- 1. **Tunnel helper** — port `tunnel-manager` into our server (`startQuickTunnel(port) → url`), behind a flag.
107
- 2. **AI SDK chat endpoint** — `/api/chat` with one server-side tool (`generate_image`) end-to-end.
108
- 3. **Sidebar skeleton** — `defineSidebarTab` + `useChat` hitting the tunnel; render stream.
106
+ 1. **Tunnel helper** — port `tunnel-manager` into our server (`startQuickTunnel(port) → url`), behind a flag. ✅ done (`src/services/tunnel.ts`).
107
+ 2. **AI SDK chat endpoint** — `/api/chat` with one server-side tool (`generate_image`) end-to-end. ✅ done (`src/experimental/{agent-poc,chat-handler}.ts`).
108
+ 3. **Sidebar skeleton** — sidebar tab + chat UI hitting the tunnel; render stream. ✅ done **as a v1 extension** (see §7).
109
109
  4. **Live edit** — one client-side tool (`set_widget_value`) applied via `WidgetHandle`; prove the loop.
110
110
  5. **Wire comfyui-mcp** as the server-side tool surface (MCP client); expand client-side graph tools.
111
111
  6. **Provider switch** (Claude/Codex/Gemini) + connection/key UX + polish into a shippable node pack.
112
112
 
113
+ ## 7. Panel implementation status — v1 now, v2 later
114
+
115
+ `@comfyorg/extension-api` (the v2 package the rest of this doc assumes) is **not yet on npm** as of 2026-06 — PRs #12142–#12145 are still in review and there is no published ETA. We therefore shipped the panel against the **v1 extension API** that every existing ComfyUI extension uses today, and tagged every v1-specific call site `// TODO(v2):` for the upgrade.
116
+
117
+ What lives in the repo now:
118
+
119
+ - `web/extensions/comfyui-mcp-agent-panel/comfyui-mcp-agent-panel.js` — single-file drop-in extension. Vanilla DOM (no framework, no bundler). Registers via `window.app.registerExtension(...)` and mounts a sidebar tab via `app.extensionManager.registerSidebarTab({...})`.
120
+ - `web/extensions/comfyui-mcp-agent-panel/README.md` — install + connection-config instructions; explains backend URL / bearer token settings (stored in `localStorage`).
121
+ - `src/experimental/ui-message-stream-parser.ts` + matching vitest suite — the AI SDK UI message stream consumer (text-start/delta/end, tool-input-available, tool-output-available, finish). The panel JS inlines a byte-equivalent copy of this parser since it ships unbundled.
122
+
123
+ The panel currently implements:
124
+
125
+ - **Connection UX** — paste tunnel URL + bearer token, persisted under `comfyui-mcp.agent-panel.*` localStorage keys.
126
+ - **Chat stream** — POSTs `{ messages: UIMessage[] }` to `<backendUrl>/api/chat`, parses the SSE stream, and renders streaming assistant text plus tool cards for `generate_image` (the POC server-side tool).
127
+ - **Abort on unmount** — the panel cancels any in-flight `fetch` on `destroy()`.
128
+
129
+ ### Migration to v2 (when the package ships)
130
+
131
+ | v1 (today) | v2 (after `@comfyorg/extension-api` is on npm) |
132
+ |------------|-------------------------------------------------|
133
+ | `window.app.registerExtension({ name, setup })` | `defineExtension({ name, setup() { ... } })` |
134
+ | `app.extensionManager.registerSidebarTab({ id, title, icon, type:'custom', render, destroy })` | `defineSidebarTab({ id, title, icon, type:'custom', render, destroy })` |
135
+ | Inlined `parseUiMessageStream` JS in the panel | Real ESM import from `src/experimental/...` once a build step enters the picture |
136
+ | Read `window.app` at module scope | Pure `import` from `@comfyorg/extension-api`; no globals |
137
+
138
+ Cross-reference for the full pattern map: `plugin/skills/comfyui-frontend-extensions/references/migrate-v1-to-v2.md`.
139
+
140
+ Step 4 (live graph edits) is **deferred until v2** unless we decide it's worth writing v1-shim wrappers around `LiteGraph` directly — the v2 `WidgetHandle.setValue` path is much cleaner and the POC works end-to-end without it.
141
+
113
142
  ## References
114
143
  - Ungate (MIT): https://github.com/orchidfiles/ungate — clone at `~/code/ungate`.
115
144
  - `cloudflared` npm: https://www.npmjs.com/package/cloudflared
@@ -0,0 +1,21 @@
1
+ export interface UiMessageStreamChunk {
2
+ type: string;
3
+ [k: string]: unknown;
4
+ }
5
+ export interface ParseResult {
6
+ chunks: UiMessageStreamChunk[];
7
+ /** Unterminated trailing fragment; pass to the next call. */
8
+ remainder: string;
9
+ /** Whether the `[DONE]` sentinel was seen. */
10
+ done: boolean;
11
+ }
12
+ /**
13
+ * Parse a slice of an AI SDK UI message stream. Stateless — the caller owns
14
+ * the remainder buffer.
15
+ *
16
+ * Frame grammar: events are separated by `\n\n`; each event is one or more
17
+ * `data: <payload>` lines. The terminator is the literal `[DONE]`. We
18
+ * tolerate `\r\n\n` and stray blank lines.
19
+ */
20
+ export declare function parseUiMessageStream(buffer: string): ParseResult;
21
+ //# sourceMappingURL=ui-message-stream-parser.d.ts.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"ui-message-stream-parser.d.ts","sourceRoot":"","sources":["../../src/experimental/ui-message-stream-parser.ts"],"names":[],"mappings":"AAsBA,MAAM,WAAW,oBAAoB;IACnC,IAAI,EAAE,MAAM,CAAC;IAIb,CAAC,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC;CACtB;AAED,MAAM,WAAW,WAAW;IAC1B,MAAM,EAAE,oBAAoB,EAAE,CAAC;IAC/B,6DAA6D;IAC7D,SAAS,EAAE,MAAM,CAAC;IAClB,8CAA8C;IAC9C,IAAI,EAAE,OAAO,CAAC;CACf;AAED;;;;;;;GAOG;AACH,wBAAgB,oBAAoB,CAAC,MAAM,EAAE,MAAM,GAAG,WAAW,CA2ChE"}
@@ -0,0 +1,73 @@
1
+ // ---------------------------------------------------------------------------
2
+ // AI SDK UI Message Stream Protocol — consumer-side parser.
3
+ //
4
+ // `POST /api/chat` returns a Server-Sent Events stream whose every `data:`
5
+ // frame is a JSON-encoded `UIMessageChunk` (the AI SDK v6 UI message stream
6
+ // protocol — see `node_modules/ai/dist/index.d.ts` for the full union). The
7
+ // terminator frame is the literal `data: [DONE]`.
8
+ //
9
+ // This module provides a single pure function that turns an arbitrary slice of
10
+ // the raw text body (i.e. however much arrived in the latest `reader.read()`)
11
+ // into:
12
+ // - a list of decoded chunk objects, and
13
+ // - a `remainder` string (an unterminated partial event) the caller must
14
+ // prepend to the NEXT slice before re-invoking.
15
+ //
16
+ // Keeping the parser pure (no state, no DOM, no fetch) makes it cheap to test
17
+ // against canned byte streams in vitest. The exact same logic is *also*
18
+ // inlined into the browser-side panel (web/extensions/...) because that file
19
+ // is dropped directly into ComfyUI's `web/extensions/` directory with no
20
+ // bundler; the two implementations are kept byte-equivalent.
21
+ // ---------------------------------------------------------------------------
22
+ /**
23
+ * Parse a slice of an AI SDK UI message stream. Stateless — the caller owns
24
+ * the remainder buffer.
25
+ *
26
+ * Frame grammar: events are separated by `\n\n`; each event is one or more
27
+ * `data: <payload>` lines. The terminator is the literal `[DONE]`. We
28
+ * tolerate `\r\n\n` and stray blank lines.
29
+ */
30
+ export function parseUiMessageStream(buffer) {
31
+ const chunks = [];
32
+ let done = false;
33
+ // Normalize line endings so the split below works regardless of transport.
34
+ const normalized = buffer.replace(/\r\n/g, "\n");
35
+ // Split on the blank-line frame boundary. The trailing element is whatever
36
+ // is *after* the last `\n\n` — i.e. an unterminated frame in progress.
37
+ const parts = normalized.split("\n\n");
38
+ const remainder = parts.pop() ?? "";
39
+ for (const frame of parts) {
40
+ // A frame can contain multiple `data:` lines (SSE spec joins them with
41
+ // `\n`). In practice AI SDK emits a single line per frame, but we honor
42
+ // the spec to keep the parser interoperable.
43
+ const dataLines = [];
44
+ for (const line of frame.split("\n")) {
45
+ if (!line.startsWith("data:"))
46
+ continue;
47
+ // Strip `data:` then a single optional leading space (per SSE spec).
48
+ let payload = line.slice(5);
49
+ if (payload.startsWith(" "))
50
+ payload = payload.slice(1);
51
+ dataLines.push(payload);
52
+ }
53
+ if (dataLines.length === 0)
54
+ continue;
55
+ const payload = dataLines.join("\n");
56
+ if (payload === "[DONE]") {
57
+ done = true;
58
+ continue;
59
+ }
60
+ try {
61
+ const parsed = JSON.parse(payload);
62
+ if (parsed && typeof parsed === "object" && typeof parsed.type === "string") {
63
+ chunks.push(parsed);
64
+ }
65
+ }
66
+ catch {
67
+ // Malformed frame — skip silently. The AI SDK never emits invalid
68
+ // JSON; this is just defensive for non-conforming proxies.
69
+ }
70
+ }
71
+ return { chunks, remainder, done };
72
+ }
73
+ //# sourceMappingURL=ui-message-stream-parser.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"ui-message-stream-parser.js","sourceRoot":"","sources":["../../src/experimental/ui-message-stream-parser.ts"],"names":[],"mappings":"AAAA,8EAA8E;AAC9E,4DAA4D;AAC5D,EAAE;AACF,2EAA2E;AAC3E,4EAA4E;AAC5E,4EAA4E;AAC5E,kDAAkD;AAClD,EAAE;AACF,+EAA+E;AAC/E,8EAA8E;AAC9E,QAAQ;AACR,2CAA2C;AAC3C,2EAA2E;AAC3E,oDAAoD;AACpD,EAAE;AACF,8EAA8E;AAC9E,wEAAwE;AACxE,6EAA6E;AAC7E,yEAAyE;AACzE,6DAA6D;AAC7D,8EAA8E;AAkB9E;;;;;;;GAOG;AACH,MAAM,UAAU,oBAAoB,CAAC,MAAc;IACjD,MAAM,MAAM,GAA2B,EAAE,CAAC;IAC1C,IAAI,IAAI,GAAG,KAAK,CAAC;IAEjB,2EAA2E;IAC3E,MAAM,UAAU,GAAG,MAAM,CAAC,OAAO,CAAC,OAAO,EAAE,IAAI,CAAC,CAAC;IAEjD,2EAA2E;IAC3E,uEAAuE;IACvE,MAAM,KAAK,GAAG,UAAU,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC;IACvC,MAAM,SAAS,GAAG,KAAK,CAAC,GAAG,EAAE,IAAI,EAAE,CAAC;IAEpC,KAAK,MAAM,KAAK,IAAI,KAAK,EAAE,CAAC;QAC1B,uEAAuE;QACvE,wEAAwE;QACxE,6CAA6C;QAC7C,MAAM,SAAS,GAAa,EAAE,CAAC;QAC/B,KAAK,MAAM,IAAI,IAAI,KAAK,CAAC,KAAK,CAAC,IAAI,CAAC,EAAE,CAAC;YACrC,IAAI,CAAC,IAAI,CAAC,UAAU,CAAC,OAAO,CAAC;gBAAE,SAAS;YACxC,qEAAqE;YACrE,IAAI,OAAO,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC;YAC5B,IAAI,OAAO,CAAC,UAAU,CAAC,GAAG,CAAC;gBAAE,OAAO,GAAG,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC;YACxD,SAAS,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC;QAC1B,CAAC;QACD,IAAI,SAAS,CAAC,MAAM,KAAK,CAAC;YAAE,SAAS;QACrC,MAAM,OAAO,GAAG,SAAS,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;QAErC,IAAI,OAAO,KAAK,QAAQ,EAAE,CAAC;YACzB,IAAI,GAAG,IAAI,CAAC;YACZ,SAAS;QACX,CAAC;QACD,IAAI,CAAC;YACH,MAAM,MAAM,GAAG,IAAI,CAAC,KAAK,CAAC,OAAO,CAAyB,CAAC;YAC3D,IAAI,MAAM,IAAI,OAAO,MAAM,KAAK,QAAQ,IAAI,OAAO,MAAM,CAAC,IAAI,KAAK,QAAQ,EAAE,CAAC;gBAC5E,MAAM,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;YACtB,CAAC;QACH,CAAC;QAAC,MAAM,CAAC;YACP,kEAAkE;YAClE,2DAA2D;QAC7D,CAAC;IACH,CAAC;IAED,OAAO,EAAE,MAAM,EAAE,SAAS,EAAE,IAAI,EAAE,CAAC;AACrC,CAAC"}
@@ -0,0 +1,75 @@
1
+ # Cloudflare Worker — docs proxy
2
+
3
+ Worker that proxies `https://comfyui-mcp.artokun.io/docs*` to the Mintlify
4
+ deployment at `artokun.mintlify.dev`. Bare-root requests 302 to `/docs`;
5
+ anything else on the host returns 404 (this host is docs-only).
6
+
7
+ - Worker source: `docs-proxy.js`
8
+ - Wrangler config: `wrangler.jsonc`
9
+ - Production URL: <https://comfyui-mcp.artokun.io/docs>
10
+ - Mintlify origin: <https://artokun.mintlify.dev/docs>
11
+
12
+ ## Prerequisites (must all be true for the public URL to work)
13
+
14
+ 1. **DNS record** — `comfyui-mcp` AAAA/A or CNAME record exists on the
15
+ `artokun.io` zone in Cloudflare, **with the proxy (orange cloud) enabled**.
16
+ Without this record the Worker route never fires and clients get NXDOMAIN /
17
+ `curl: (6) Could not resolve host`.
18
+ 2. **Worker deployed** — `comfyui-mcp-docs-proxy` is deployed on the account
19
+ matching `account_id` in `wrangler.jsonc`.
20
+ 3. **Worker route attached** — `comfyui-mcp.artokun.io/*` on `artokun.io` zone
21
+ (declared in `wrangler.jsonc`, applied at deploy time).
22
+ 4. **Mintlify custom domain** — Mintlify project has
23
+ `comfyui-mcp.artokun.io/docs` configured so generated asset/link URLs use
24
+ the `/docs` prefix.
25
+
26
+ ## Deploy
27
+
28
+ ```bash
29
+ cd infra/cloudflare
30
+ npx wrangler deploy
31
+ ```
32
+
33
+ Wrangler must be authenticated against the Cloudflare account that owns the
34
+ `artokun.io` zone (account id `208c358f58d75a3fc684695473f431dd`). Verify with
35
+ `wrangler whoami` — if it shows a different account, run
36
+ `wrangler logout && wrangler login` and pick the right account in the OAuth
37
+ flow.
38
+
39
+ ## Recovery: `comfyui-mcp.artokun.io` returns NXDOMAIN
40
+
41
+ Symptoms:
42
+
43
+ ```text
44
+ $ curl -I https://comfyui-mcp.artokun.io/docs
45
+ curl: (6) Could not resolve host: comfyui-mcp.artokun.io
46
+ $ dig comfyui-mcp.artokun.io @drake.ns.cloudflare.com +short # empty
47
+ ```
48
+
49
+ Cause: the DNS record for the `comfyui-mcp` subdomain has been deleted from
50
+ the `artokun.io` zone, or its proxy was disabled and the underlying target
51
+ is unreachable. The Worker can be deployed and the route attached, but
52
+ without the DNS record the hostname has no IP.
53
+
54
+ Fix (Cloudflare dashboard):
55
+
56
+ 1. Dashboard → `artokun.io` zone → **DNS → Records**.
57
+ 2. Add a record:
58
+ - Type: `AAAA`
59
+ - Name: `comfyui-mcp`
60
+ - IPv6 address: `100::` (RFC 6666 discard prefix — value is irrelevant
61
+ because the request never leaves Cloudflare; the Worker route intercepts
62
+ it). An `A 192.0.2.1` placeholder works equally well.
63
+ - **Proxy status: Proxied (orange cloud)** — this is mandatory.
64
+ - TTL: Auto.
65
+ 3. Save. Propagation on Cloudflare's own resolvers is near-instant.
66
+ 4. Verify:
67
+
68
+ ```bash
69
+ dig comfyui-mcp.artokun.io @drake.ns.cloudflare.com +short # should now return Cloudflare IPs
70
+ curl -I https://comfyui-mcp.artokun.io/docs # HTTP/2 200
71
+ curl -I https://comfyui-mcp.artokun.io/ # HTTP/2 302 → /docs
72
+ ```
73
+
74
+ If `curl` returns 404 from a Cloudflare server, the DNS is fixed but the
75
+ Worker route is missing — re-run `wrangler deploy`.
@@ -0,0 +1,139 @@
1
+ # comfyui-mcp — Install Guide for AI Agents
2
+
3
+ This document tells an AI agent (Cline, Claude Code, Cursor, etc.) how to
4
+ install and configure **comfyui-mcp** in one shot. It is a focused subset of
5
+ the project [README](./README.md) — see that file for full documentation.
6
+
7
+ ## What you are installing
8
+
9
+ An MCP server that lets the user's AI assistant drive
10
+ [ComfyUI](https://github.com/comfyanonymous/ComfyUI): generate images, run and
11
+ author workflows, manage models and custom nodes, and control the server.
12
+ **86 MCP tools** across the categories shown in
13
+ [docs/tools/](https://comfyui-mcp.artokun.io/docs/tools/image-generation).
14
+
15
+ ## Prerequisites
16
+
17
+ - **Node.js ≥ 22.** Confirm with `node --version`; install via
18
+ [nodejs.org](https://nodejs.org/) or `nvm` if missing.
19
+ - The package is published to npm as **`comfyui-mcp`** and runs via `npx` — no
20
+ global install required.
21
+ - The server needs a ComfyUI to talk to. **Three options**, pick one with the
22
+ user (ask if unclear):
23
+
24
+ 1. **Local ComfyUI** — the user is running ComfyUI on the same machine. The
25
+ server auto-detects the install path and port (8188 → 8000 fallback). No
26
+ extra config needed.
27
+ 2. **Remote ComfyUI** — the user runs ComfyUI on a different host
28
+ (RunPod, VPS, LAN box). Pass `--comfyui-url <url>` or set
29
+ `COMFYUI_URL` env. When the host is non-loopback, local-FS auto-detection
30
+ is suppressed.
31
+ 3. **Comfy Cloud** — the user has a [cloud.comfy.org](https://cloud.comfy.org)
32
+ API key. Set `COMFYUI_API_KEY` env. The server routes HTTP primitives to
33
+ the cloud; local-only tools throw `CLOUD_UNSUPPORTED`.
34
+
35
+ ## Add to the MCP client config
36
+
37
+ ### Claude Code / Claude Desktop (`~/.claude/settings.json`)
38
+
39
+ **Local ComfyUI** (most common):
40
+
41
+ ```json
42
+ {
43
+ "mcpServers": {
44
+ "comfyui": {
45
+ "command": "npx",
46
+ "args": ["-y", "comfyui-mcp"]
47
+ }
48
+ }
49
+ }
50
+ ```
51
+
52
+ **Remote ComfyUI:**
53
+
54
+ ```json
55
+ {
56
+ "mcpServers": {
57
+ "comfyui": {
58
+ "command": "npx",
59
+ "args": ["-y", "comfyui-mcp", "--comfyui-url", "https://my-comfy.example.com"]
60
+ }
61
+ }
62
+ }
63
+ ```
64
+
65
+ **Comfy Cloud:**
66
+
67
+ ```json
68
+ {
69
+ "mcpServers": {
70
+ "comfyui": {
71
+ "command": "npx",
72
+ "args": ["-y", "comfyui-mcp"],
73
+ "env": {
74
+ "COMFYUI_API_KEY": "<ask the user for their cloud.comfy.org key>"
75
+ }
76
+ }
77
+ }
78
+ }
79
+ ```
80
+
81
+ ### Cline / Cursor / generic MCP
82
+
83
+ Use the same `command` + `args` shape (Cline expects `command` + `args` in
84
+ its `cline_mcp_settings.json`; Cursor expects similar in its MCP settings
85
+ panel).
86
+
87
+ ## Optional environment variables
88
+
89
+ Set in the `env` block above. None are required for the local-default flow.
90
+
91
+ - `COMFYUI_HOST` / `COMFYUI_PORT` — override host/port (defaults: auto-detect)
92
+ - `COMFYUI_PATH` — explicit ComfyUI install path (auto-detected on Mac / Linux
93
+ / Windows when unset)
94
+ - `COMFYUI_DOWNLOAD_CACHE_DIR` — model download cache (default
95
+ `~/.comfyui-mcp/cache`)
96
+ - `COMFYUI_LRU_CACHE_SIZE_GB` — cap the cache; `0` disables eviction
97
+ - `CIVITAI_API_TOKEN`, `HUGGINGFACE_TOKEN`, `GITHUB_TOKEN` — for gated
98
+ downloads and higher API rate limits
99
+ - `REGISTRY_ACCESS_TOKEN` — Comfy Registry API key for `publish_custom_node`
100
+ - `COMFY_API_KEY` — comfy.org API key for hosted partner nodes (different
101
+ from `COMFYUI_API_KEY`, which is for Comfy Cloud)
102
+ - `COMFYUI_CLOUD_URL` — override the Comfy Cloud endpoint
103
+ (default `https://cloud.comfy.org`)
104
+
105
+ Full reference: [docs/configuration](https://comfyui-mcp.artokun.io/docs/configuration).
106
+
107
+ ## Verify
108
+
109
+ After updating the settings file, **restart the MCP client** (Claude Code: run
110
+ `/mcp` to reconnect; Cline: toggle the server). Then ask the assistant:
111
+
112
+ > What ComfyUI tools do you have?
113
+
114
+ It should list ~86 tools across generation, workflow execution/authoring,
115
+ models, custom nodes, etc. If the user wants a quick smoke test, ask:
116
+
117
+ > Generate a 1024×1024 image of a red apple on a wooden table.
118
+
119
+ That exercises the `generate_image` tool end-to-end (auto-selects a local
120
+ checkpoint or uses defaults; returns an `asset_id` you can `view_image` to
121
+ see).
122
+
123
+ ## Common issues
124
+
125
+ - **"ComfyUI not detected on ports 8188, 8000"** — ComfyUI isn't running. Tell
126
+ the user to start it (Desktop app or `python main.py`).
127
+ - **`CLOUD_UNSUPPORTED` errors** — `COMFYUI_API_KEY` is set, so the server is
128
+ in cloud mode and a local-only tool was called. Either unset the key (to
129
+ use a local install) or stick to cloud-compatible tools.
130
+ - **Empty model lists** — `extra_model_paths.yaml` is misconfigured. Run
131
+ `health_check` for a diagnostic.
132
+
133
+ ## License + repo
134
+
135
+ - **License:** [MIT](./LICENSE)
136
+ - **Repo:** https://github.com/artokun/comfyui-mcp
137
+ - **npm:** https://www.npmjs.com/package/comfyui-mcp
138
+ - **Docs:** https://comfyui-mcp.artokun.io/docs
139
+ - **Issues:** https://github.com/artokun/comfyui-mcp/issues
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "comfyui-mcp",
3
- "version": "0.9.1",
3
+ "version": "0.9.3",
4
4
  "mcpName": "io.github.artokun/comfyui-mcp",
5
5
  "description": "MCP server for ComfyUI — workflow execution, visualization, composition, registry, and skill generation",
6
6
  "homepage": "https://comfyui-mcp.artokun.io/docs",
@@ -0,0 +1,97 @@
1
+ # comfyui-mcp Agent Panel — v1 ComfyUI extension
2
+
3
+ A drop-in sidebar tab for ComfyUI that hosts a chat UI talking to the
4
+ [experimental agent backend](../../../src/experimental/agent-poc.ts) shipped
5
+ with `comfyui-mcp`. This is the **v1 implementation** — built against today's
6
+ `app.registerExtension(...)` API. It will be ported to the v2
7
+ `@comfyorg/extension-api` package the moment that ships
8
+ (Comfy-Org PRs #12142–#12145); v1-specific call sites are tagged
9
+ `// TODO(v2):` in the source.
10
+
11
+ ## Install
12
+
13
+ 1. Copy **`comfyui-mcp-agent-panel.js`** (single file, no dependencies) into
14
+ one of these locations — ComfyUI auto-loads any `.js` it finds inside them:
15
+ - `<ComfyUI>/web/extensions/comfyui-mcp-agent-panel.js`, or
16
+ - `<ComfyUI>/custom_nodes/<your-pack>/web/comfyui-mcp-agent-panel.js`.
17
+ 2. Hard-reload the ComfyUI page (`Cmd/Ctrl+Shift+R`).
18
+ 3. A new **Agent** tab (chat icon) appears in the left sidebar.
19
+
20
+ The README and this directory layout exist for repo hygiene only — ComfyUI
21
+ does **not** consume the README.
22
+
23
+ ## Start the backend
24
+
25
+ In a separate terminal at the repo root:
26
+
27
+ ```bash
28
+ COMFYUI_MCP_AGENT_POC=1 \
29
+ COMFYUI_MCP_AGENT_TUNNEL=1 \
30
+ ANTHROPIC_API_KEY=sk-ant-... # or OPENAI_API_KEY / GOOGLE_GENERATIVE_AI_API_KEY
31
+ npm run dev:agent-poc
32
+ ```
33
+
34
+ The server prints two values you need:
35
+
36
+ - `chat server listening on http://127.0.0.1:8765/api/chat` — the local URL,
37
+ fine if ComfyUI runs on the same machine over `http://`.
38
+ - `public URL: https://<random>.trycloudflare.com` — required if ComfyUI is
39
+ served over HTTPS or on a remote host (browser mixed-content would block a
40
+ plain-`localhost` POST otherwise).
41
+ - `session token: <hex>` — the bearer token.
42
+
43
+ ## Configure the panel
44
+
45
+ Open the **Agent** tab → **Connection** section, then paste:
46
+
47
+ - **Backend URL** — `https://<random>.trycloudflare.com` (or
48
+ `http://localhost:8765` for same-machine HTTP setups).
49
+ - **Bearer token** — the hex string from the server stdout.
50
+
51
+ Click **Save** (or just hit **Send** — the panel persists the live input
52
+ values whenever a request is dispatched, so explicit Save is optional).
53
+ Both values persist in `window.localStorage` under
54
+ `comfyui-mcp.agent-panel.backendUrl` / `comfyui-mcp.agent-panel.token`.
55
+
56
+ > **Backend URL forms accepted.** The panel normalizes `…/`, `…/api`, and
57
+ > `…/api/chat` suffixes — paste whatever the server logged.
58
+
59
+ > **Security:** the bearer token can spend on your provider API keys.
60
+ > `localStorage` is per-origin readable by every script on the ComfyUI page —
61
+ > rotate the token (restart the POC) if you suspect leakage.
62
+
63
+ ## What it can do (POC scope)
64
+
65
+ - Stream chat replies from the configured provider (Claude / Codex / Gemini —
66
+ picked server-side via `src/experimental/provider-registry.ts`).
67
+ - Surface a tool card whenever the model calls the POC `generate_image` tool;
68
+ the backend stubs the actual ComfyUI workflow execution and returns a
69
+ placeholder result.
70
+
71
+ Live graph edits (`set_widget_value`, `add_node`, etc.) are **not yet** wired
72
+ — that's step 4 in `design/embedded-agent-panel.md` and depends on either
73
+ the v2 `extension-api` `WidgetHandle` surface or our own v1 shims around
74
+ `LiteGraph`.
75
+
76
+ ## Files
77
+
78
+ - `comfyui-mcp-agent-panel.js` — the entire extension (one file, no build).
79
+ - `README.md` — this file.
80
+
81
+ The SSE parser embedded in the panel JS is byte-equivalent to
82
+ `src/experimental/ui-message-stream-parser.ts` (which has vitest coverage at
83
+ `src/__tests__/experimental/ui-message-stream-parser.test.ts`). Keep them in
84
+ sync if you change either.
85
+
86
+ ## V1 → V2 migration plan
87
+
88
+ When `@comfyorg/extension-api` lands on npm:
89
+
90
+ | v1 (this file) | v2 |
91
+ |----------------|----|
92
+ | `window.app.registerExtension({ name, setup })` | `defineExtension({ name, setup() {} })` |
93
+ | `app.extensionManager.registerSidebarTab({ id, title, icon, type:'custom', render, destroy })` | `defineSidebarTab({ id, title, icon, type:'custom', render, destroy })` |
94
+ | Embedded `parseUiMessageStream` JS | `import { parseUiMessageStream } from '...'` (a real bundler enters the picture) |
95
+
96
+ See `plugin/skills/comfyui-frontend-extensions/references/migrate-v1-to-v2.md`
97
+ for the full mapping.
@@ -0,0 +1,604 @@
1
+ // =============================================================================
2
+ // comfyui-mcp Agent Panel — v1 ComfyUI frontend extension.
3
+ //
4
+ // Registers a sidebar tab inside ComfyUI that hosts a chat UI talking to our
5
+ // experimental backend (src/experimental/agent-poc.ts). Drop this single file
6
+ // into ComfyUI's `web/extensions/` directory (or the user's
7
+ // `ComfyUI/custom_nodes/<pack>/web/` dir) and reload the page.
8
+ //
9
+ // Wire format: the backend's `POST /api/chat` returns an AI SDK v6 UI Message
10
+ // Stream (Server-Sent Events). We parse `text-start`/`text-delta`/`text-end`
11
+ // to append streaming text, and `tool-input-available` /
12
+ // `tool-output-available` to render a tool card.
13
+ //
14
+ // Settings (panel UI; persisted via window.localStorage under
15
+ // `comfyui-mcp.agent-panel.*`):
16
+ // - `backendUrl` — public URL the panel POSTs to (e.g. the cloudflared
17
+ // trycloudflare.com URL or `http://localhost:8765`).
18
+ // - `token` — bearer token printed on the server's stdout.
19
+ // Both are required before the first message can be sent.
20
+ //
21
+ // SECURITY NOTE: localStorage is per-origin readable by any script on the
22
+ // ComfyUI page. The bearer token grants spend on the user's provider keys —
23
+ // don't share workflow JSON containing it, and rotate it (restart the POC) if
24
+ // you suspect leakage.
25
+ //
26
+ // V1→V2 MIGRATION: this file uses `window.app.registerExtension(...)` (v1) and
27
+ // `app.extensionManager.registerSidebarTab(...)`. When the v2 npm package
28
+ // `@comfyorg/extension-api` (PRs #12142–#12145) ships, the equivalent calls
29
+ // are `defineExtension({ setup() { ... } })` + `defineSidebarTab({ id, title,
30
+ // type: 'custom', icon, render, destroy })`. Every v1-specific call below is
31
+ // marked `// TODO(v2):` — see
32
+ // `plugin/skills/comfyui-frontend-extensions/references/migrate-v1-to-v2.md`.
33
+ // =============================================================================
34
+
35
+ // ---------------------------------------------------------------------------
36
+ // AI SDK UI Message Stream parser. Kept byte-equivalent to the TS module at
37
+ // `src/experimental/ui-message-stream-parser.ts` (which has vitest coverage).
38
+ // We can't import that TS module here because this file is loaded directly
39
+ // by ComfyUI's browser with no bundler.
40
+ // ---------------------------------------------------------------------------
41
+ function parseUiMessageStream(buffer) {
42
+ const chunks = [];
43
+ let done = false;
44
+ const normalized = buffer.replace(/\r\n/g, "\n");
45
+ const parts = normalized.split("\n\n");
46
+ const remainder = parts.pop() ?? "";
47
+
48
+ for (const frame of parts) {
49
+ const dataLines = [];
50
+ for (const line of frame.split("\n")) {
51
+ if (!line.startsWith("data:")) continue;
52
+ let payload = line.slice(5);
53
+ if (payload.startsWith(" ")) payload = payload.slice(1);
54
+ dataLines.push(payload);
55
+ }
56
+ if (dataLines.length === 0) continue;
57
+ const payload = dataLines.join("\n");
58
+ if (payload === "[DONE]") {
59
+ done = true;
60
+ continue;
61
+ }
62
+ try {
63
+ const parsed = JSON.parse(payload);
64
+ if (parsed && typeof parsed === "object" && typeof parsed.type === "string") {
65
+ chunks.push(parsed);
66
+ }
67
+ } catch {
68
+ // ignore malformed frames
69
+ }
70
+ }
71
+ return { chunks, remainder, done };
72
+ }
73
+
74
+ // ---------------------------------------------------------------------------
75
+ // localStorage-backed settings (small, sync, plenty for the POC).
76
+ // ---------------------------------------------------------------------------
77
+ const STORAGE_KEY_BACKEND = "comfyui-mcp.agent-panel.backendUrl";
78
+ const STORAGE_KEY_TOKEN = "comfyui-mcp.agent-panel.token";
79
+
80
+ function loadSettings() {
81
+ try {
82
+ return {
83
+ backendUrl: window.localStorage.getItem(STORAGE_KEY_BACKEND) ?? "",
84
+ token: window.localStorage.getItem(STORAGE_KEY_TOKEN) ?? "",
85
+ };
86
+ } catch {
87
+ return { backendUrl: "", token: "" };
88
+ }
89
+ }
90
+
91
+ function saveSettings(s) {
92
+ try {
93
+ window.localStorage.setItem(STORAGE_KEY_BACKEND, s.backendUrl ?? "");
94
+ window.localStorage.setItem(STORAGE_KEY_TOKEN, s.token ?? "");
95
+ } catch {
96
+ // localStorage may be unavailable in private/locked-down browsers; the
97
+ // panel just becomes session-scoped in that case.
98
+ }
99
+ }
100
+
101
+ // ---------------------------------------------------------------------------
102
+ // Tiny id helper for UIMessage ids. The AI SDK accepts any unique string.
103
+ // ---------------------------------------------------------------------------
104
+ function uid() {
105
+ if (typeof crypto !== "undefined" && crypto.randomUUID) return crypto.randomUUID();
106
+ return `m_${Date.now().toString(36)}_${Math.random().toString(36).slice(2, 10)}`;
107
+ }
108
+
109
+ // ---------------------------------------------------------------------------
110
+ // Build the panel DOM. Returns { root, destroy } so the host can mount/unmount.
111
+ // ---------------------------------------------------------------------------
112
+ function buildPanel() {
113
+ const root = document.createElement("div");
114
+ root.className = "comfyui-mcp-agent-panel";
115
+ root.style.cssText = `
116
+ display: flex; flex-direction: column; height: 100%;
117
+ padding: 8px; gap: 8px; box-sizing: border-box;
118
+ font: 13px/1.4 -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
119
+ color: var(--input-text, #ddd); background: var(--comfy-menu-bg, #222);
120
+ `;
121
+
122
+ // ---- Settings strip ------------------------------------------------------
123
+ const settingsBox = document.createElement("details");
124
+ settingsBox.style.cssText = "border: 1px solid #444; border-radius: 4px; padding: 6px;";
125
+ const settingsSummary = document.createElement("summary");
126
+ settingsSummary.textContent = "Connection";
127
+ settingsSummary.style.cssText = "cursor: pointer; user-select: none; font-weight: 600;";
128
+ settingsBox.appendChild(settingsSummary);
129
+
130
+ const settings = loadSettings();
131
+ settingsBox.open = !settings.backendUrl || !settings.token;
132
+
133
+ const makeRow = (labelText, type, value, placeholder) => {
134
+ const row = document.createElement("div");
135
+ row.style.cssText = "display: flex; flex-direction: column; gap: 2px; margin-top: 6px;";
136
+ const label = document.createElement("label");
137
+ label.textContent = labelText;
138
+ label.style.cssText = "font-size: 11px; opacity: 0.7;";
139
+ const input = document.createElement("input");
140
+ input.type = type;
141
+ input.value = value;
142
+ input.placeholder = placeholder;
143
+ input.style.cssText = `
144
+ width: 100%; padding: 4px 6px; border: 1px solid #555; border-radius: 3px;
145
+ background: var(--comfy-input-bg, #181818); color: inherit; box-sizing: border-box;
146
+ `;
147
+ row.append(label, input);
148
+ return { row, input };
149
+ };
150
+
151
+ const { row: urlRow, input: urlInput } = makeRow(
152
+ "Backend URL",
153
+ "url",
154
+ settings.backendUrl,
155
+ "https://<random>.trycloudflare.com",
156
+ );
157
+ const { row: tokenRow, input: tokenInput } = makeRow(
158
+ "Bearer token",
159
+ "password",
160
+ settings.token,
161
+ "from server stdout: 'session token: ...'",
162
+ );
163
+
164
+ const saveBtn = document.createElement("button");
165
+ saveBtn.type = "button";
166
+ saveBtn.textContent = "Save";
167
+ saveBtn.style.cssText =
168
+ "margin-top: 8px; padding: 4px 10px; cursor: pointer; align-self: flex-start;";
169
+ saveBtn.addEventListener("click", () => {
170
+ saveSettings({
171
+ backendUrl: urlInput.value.trim(),
172
+ token: tokenInput.value.trim(),
173
+ });
174
+ appendSystem("Connection saved.");
175
+ settingsBox.open = false;
176
+ });
177
+
178
+ settingsBox.append(urlRow, tokenRow, saveBtn);
179
+ root.appendChild(settingsBox);
180
+
181
+ // ---- Message log ---------------------------------------------------------
182
+ const log = document.createElement("div");
183
+ log.style.cssText = `
184
+ flex: 1 1 auto; overflow-y: auto; padding: 6px;
185
+ border: 1px solid #444; border-radius: 4px;
186
+ display: flex; flex-direction: column; gap: 6px;
187
+ `;
188
+ root.appendChild(log);
189
+
190
+ // ---- Input row -----------------------------------------------------------
191
+ const form = document.createElement("form");
192
+ form.style.cssText = "display: flex; gap: 6px;";
193
+ const input = document.createElement("textarea");
194
+ input.placeholder = "Ask the agent... (Enter to send, Shift+Enter for newline)";
195
+ input.rows = 2;
196
+ input.style.cssText = `
197
+ flex: 1; padding: 6px; border: 1px solid #555; border-radius: 3px;
198
+ background: var(--comfy-input-bg, #181818); color: inherit; resize: vertical;
199
+ font: inherit;
200
+ `;
201
+ const sendBtn = document.createElement("button");
202
+ sendBtn.type = "submit";
203
+ sendBtn.textContent = "Send";
204
+ sendBtn.style.cssText = "padding: 6px 12px; cursor: pointer;";
205
+ form.append(input, sendBtn);
206
+ root.appendChild(form);
207
+
208
+ // ---- DOM helpers ---------------------------------------------------------
209
+ const messages = []; // UIMessage[] for /api/chat history.
210
+
211
+ function makeBubble(role) {
212
+ const bubble = document.createElement("div");
213
+ bubble.style.cssText = `
214
+ padding: 6px 8px; border-radius: 4px; max-width: 95%;
215
+ white-space: pre-wrap; word-wrap: break-word;
216
+ `;
217
+ if (role === "user") {
218
+ bubble.style.background = "#2a4d6e";
219
+ bubble.style.alignSelf = "flex-end";
220
+ } else if (role === "system") {
221
+ bubble.style.background = "#3a3a3a";
222
+ bubble.style.fontStyle = "italic";
223
+ bubble.style.opacity = "0.8";
224
+ bubble.style.alignSelf = "center";
225
+ bubble.style.fontSize = "11px";
226
+ } else {
227
+ bubble.style.background = "#333";
228
+ bubble.style.alignSelf = "flex-start";
229
+ }
230
+ log.appendChild(bubble);
231
+ log.scrollTop = log.scrollHeight;
232
+ return bubble;
233
+ }
234
+
235
+ function appendUser(text) {
236
+ const bubble = makeBubble("user");
237
+ bubble.textContent = text;
238
+ }
239
+
240
+ function appendSystem(text) {
241
+ const bubble = makeBubble("system");
242
+ bubble.textContent = text;
243
+ }
244
+
245
+ function appendAssistantStub() {
246
+ const bubble = makeBubble("assistant");
247
+ bubble.dataset.role = "assistant";
248
+ return bubble;
249
+ }
250
+
251
+ function appendToolCard({ toolCallId, toolName, input: toolInput, output }) {
252
+ const card = document.createElement("div");
253
+ card.style.cssText = `
254
+ align-self: flex-start; padding: 6px 8px; border-radius: 4px;
255
+ background: #2c2c2c; border-left: 3px solid #6aa84f;
256
+ font-family: ui-monospace, SFMono-Regular, Menlo, monospace; font-size: 11px;
257
+ max-width: 95%; white-space: pre-wrap; word-wrap: break-word;
258
+ `;
259
+ const head = document.createElement("div");
260
+ head.style.cssText = "font-weight: 600; margin-bottom: 4px;";
261
+ head.textContent = `tool ${toolName ?? "?"} (${toolCallId.slice(0, 8)}…)`;
262
+ card.appendChild(head);
263
+ if (toolInput !== undefined) {
264
+ const inDiv = document.createElement("div");
265
+ inDiv.textContent = `input: ${safeStringify(toolInput)}`;
266
+ card.appendChild(inDiv);
267
+ }
268
+ if (output !== undefined) {
269
+ const outDiv = document.createElement("div");
270
+ outDiv.style.opacity = "0.85";
271
+ outDiv.textContent = `output: ${safeStringify(output)}`;
272
+ card.appendChild(outDiv);
273
+ }
274
+ log.appendChild(card);
275
+ log.scrollTop = log.scrollHeight;
276
+ return card;
277
+ }
278
+
279
+ function safeStringify(v) {
280
+ try {
281
+ return JSON.stringify(v);
282
+ } catch {
283
+ return String(v);
284
+ }
285
+ }
286
+
287
+ // ---- Send pipeline -------------------------------------------------------
288
+ let inFlight = null; // AbortController for the current request.
289
+
290
+ /** Read the live connection settings, preferring the form inputs over storage
291
+ * so that a user who types into Connection and hits Send (without clicking
292
+ * Save first) gets the expected behavior — and we silently persist the
293
+ * values they implicitly approved by sending. */
294
+ function readConnection() {
295
+ const liveUrl = urlInput.value.trim();
296
+ const liveToken = tokenInput.value.trim();
297
+ const stored = loadSettings();
298
+ const backendUrl = liveUrl || stored.backendUrl;
299
+ const token = liveToken || stored.token;
300
+ if (
301
+ backendUrl &&
302
+ token &&
303
+ (backendUrl !== stored.backendUrl || token !== stored.token)
304
+ ) {
305
+ saveSettings({ backendUrl, token });
306
+ }
307
+ return { backendUrl, token };
308
+ }
309
+
310
+ /** Normalize a user-pasted backend URL into a `/api/chat` endpoint.
311
+ * Accepts forms like:
312
+ * https://abc.trycloudflare.com
313
+ * https://abc.trycloudflare.com/
314
+ * https://abc.trycloudflare.com/api
315
+ * https://abc.trycloudflare.com/api/chat
316
+ * and returns the canonical `<origin>/api/chat`. */
317
+ function toChatUrl(raw) {
318
+ // Strip whitespace + trailing slashes.
319
+ let s = raw.trim().replace(/\/+$/, "");
320
+ // Strip a trailing `/api/chat` or `/api` segment if the user copied either.
321
+ s = s.replace(/\/api\/chat$/i, "").replace(/\/api$/i, "");
322
+ return s + "/api/chat";
323
+ }
324
+
325
+ async function sendMessage(text) {
326
+ const { backendUrl, token } = readConnection();
327
+ if (!backendUrl || !token) {
328
+ appendSystem("Set the backend URL and bearer token in the Connection section first.");
329
+ settingsBox.open = true;
330
+ return;
331
+ }
332
+
333
+ const userMsg = {
334
+ id: uid(),
335
+ role: "user",
336
+ parts: [{ type: "text", text }],
337
+ };
338
+ messages.push(userMsg);
339
+ appendUser(text);
340
+
341
+ sendBtn.disabled = true;
342
+ input.disabled = true;
343
+ const assistantBubble = appendAssistantStub();
344
+ let assistantText = "";
345
+ // Track open tool calls so we can fill in their output when it arrives,
346
+ // AND so the assistant message we persist to history includes the
347
+ // dynamic-tool parts the model actually emitted (otherwise multi-turn
348
+ // tool conversations lose the tool context — the model can't see what
349
+ // it called or got back on the next turn).
350
+ const toolCards = new Map();
351
+ const toolParts = new Map(); // toolCallId -> dynamic-tool UIMessagePart
352
+
353
+ // Single chunk-dispatcher; closes over the per-request mutable state.
354
+ // Declared as a `const` so its binding is unambiguously available before
355
+ // the read loop runs (block-scoped function decls in `try` are spec-fuzzy
356
+ // across engines and strict mode).
357
+ const processChunk = (chunk) => {
358
+ switch (chunk.type) {
359
+ case "text-start":
360
+ // Nothing to render; bubble is already in place.
361
+ break;
362
+ case "text-delta":
363
+ if (typeof chunk.delta === "string") {
364
+ assistantText += chunk.delta;
365
+ assistantBubble.textContent = assistantText;
366
+ log.scrollTop = log.scrollHeight;
367
+ }
368
+ break;
369
+ case "text-end":
370
+ break;
371
+ case "tool-input-available": {
372
+ const id = String(chunk.toolCallId ?? uid());
373
+ const toolName = String(chunk.toolName ?? "");
374
+ const card = appendToolCard({
375
+ toolCallId: id,
376
+ toolName,
377
+ input: chunk.input,
378
+ });
379
+ toolCards.set(id, card);
380
+ // Build the assistant-message part so a follow-up turn can see
381
+ // this call. Mirrors the AI SDK's DynamicToolUIPart shape.
382
+ toolParts.set(id, {
383
+ type: "dynamic-tool",
384
+ toolName,
385
+ toolCallId: id,
386
+ state: "input-available",
387
+ input: chunk.input,
388
+ });
389
+ break;
390
+ }
391
+ case "tool-output-available": {
392
+ // The backend POC `generate_image` tool resolves server-side and
393
+ // the result rides this chunk. We just surface the payload in
394
+ // the corresponding card (or open a new one if we missed the
395
+ // input event for some reason). This is the "one tool-execution
396
+ // path end-to-end" required by the build order.
397
+ const id = String(chunk.toolCallId ?? uid());
398
+ let card = toolCards.get(id);
399
+ if (!card) {
400
+ card = appendToolCard({ toolCallId: id, toolName: "(tool)" });
401
+ toolCards.set(id, card);
402
+ }
403
+ const outDiv = document.createElement("div");
404
+ outDiv.style.opacity = "0.85";
405
+ outDiv.textContent = `output: ${safeStringify(chunk.output)}`;
406
+ card.appendChild(outDiv);
407
+ log.scrollTop = log.scrollHeight;
408
+ // Promote the persisted part to output-available so multi-turn
409
+ // history carries the tool result back to the model.
410
+ const prior = toolParts.get(id) ?? {
411
+ type: "dynamic-tool",
412
+ toolName: "(tool)",
413
+ toolCallId: id,
414
+ };
415
+ toolParts.set(id, {
416
+ ...prior,
417
+ state: "output-available",
418
+ output: chunk.output,
419
+ });
420
+ break;
421
+ }
422
+ case "tool-output-error": {
423
+ const id = String(chunk.toolCallId ?? uid());
424
+ const card = toolCards.get(id);
425
+ const errDiv = document.createElement("div");
426
+ errDiv.style.color = "#f08";
427
+ errDiv.textContent = `error: ${chunk.errorText ?? "tool failed"}`;
428
+ (card ?? appendToolCard({ toolCallId: id, toolName: "(tool)" })).appendChild(
429
+ errDiv,
430
+ );
431
+ const prior = toolParts.get(id) ?? {
432
+ type: "dynamic-tool",
433
+ toolName: "(tool)",
434
+ toolCallId: id,
435
+ };
436
+ toolParts.set(id, {
437
+ ...prior,
438
+ state: "output-error",
439
+ errorText: String(chunk.errorText ?? "tool failed"),
440
+ });
441
+ break;
442
+ }
443
+ case "error":
444
+ assistantBubble.textContent = String(chunk.errorText ?? "stream error");
445
+ assistantBubble.style.background = "#5a2828";
446
+ break;
447
+ case "finish":
448
+ // Loop exits on `[DONE]`.
449
+ break;
450
+ default:
451
+ // Unknown chunk types (data-*, reasoning-*, etc.) are silently
452
+ // ignored for the POC.
453
+ break;
454
+ }
455
+ };
456
+
457
+ inFlight = new AbortController();
458
+ try {
459
+ const res = await fetch(toChatUrl(backendUrl), {
460
+ method: "POST",
461
+ headers: {
462
+ "content-type": "application/json",
463
+ authorization: `Bearer ${token}`,
464
+ },
465
+ body: JSON.stringify({ messages }),
466
+ signal: inFlight.signal,
467
+ });
468
+
469
+ if (!res.ok || !res.body) {
470
+ const errText = await res.text().catch(() => "");
471
+ assistantBubble.textContent = `Error ${res.status}: ${errText || res.statusText}`;
472
+ assistantBubble.style.background = "#5a2828";
473
+ return;
474
+ }
475
+
476
+ const reader = res.body.getReader();
477
+ const decoder = new TextDecoder();
478
+ let buffer = "";
479
+ let streamDone = false;
480
+
481
+ while (!streamDone) {
482
+ const { value, done } = await reader.read();
483
+ if (done) {
484
+ // Flush any pending multi-byte UTF-8 bytes the decoder is buffering
485
+ // (a final TCP read could split a multi-byte char in half; without
486
+ // this flush the trailing bytes are silently dropped).
487
+ buffer += decoder.decode();
488
+ const tail = parseUiMessageStream(buffer);
489
+ buffer = tail.remainder;
490
+ if (tail.done) streamDone = true;
491
+ for (const chunk of tail.chunks) processChunk(chunk);
492
+ break;
493
+ }
494
+ buffer += decoder.decode(value, { stream: true });
495
+ const result = parseUiMessageStream(buffer);
496
+ buffer = result.remainder;
497
+ if (result.done) streamDone = true;
498
+
499
+ for (const chunk of result.chunks) processChunk(chunk);
500
+ }
501
+
502
+ // Persist the assistant message so subsequent turns include it. We
503
+ // record any tool parts FIRST (matching the AI SDK's typical part
504
+ // order — tool invocations precede the final text summary) and only
505
+ // emit a message if the model produced any content this turn.
506
+ const parts = [];
507
+ for (const part of toolParts.values()) parts.push(part);
508
+ if (assistantText) parts.push({ type: "text", text: assistantText });
509
+ if (parts.length > 0) {
510
+ messages.push({ id: uid(), role: "assistant", parts });
511
+ }
512
+ } catch (err) {
513
+ if (err && err.name === "AbortError") {
514
+ assistantBubble.textContent += "\n[aborted]";
515
+ } else {
516
+ const msg = err && err.message ? err.message : String(err);
517
+ assistantBubble.textContent = `Request failed: ${msg}`;
518
+ assistantBubble.style.background = "#5a2828";
519
+ }
520
+ } finally {
521
+ inFlight = null;
522
+ sendBtn.disabled = false;
523
+ input.disabled = false;
524
+ input.focus();
525
+ }
526
+ }
527
+
528
+ form.addEventListener("submit", (ev) => {
529
+ ev.preventDefault();
530
+ const text = input.value.trim();
531
+ if (!text || inFlight) return;
532
+ input.value = "";
533
+ void sendMessage(text);
534
+ });
535
+
536
+ input.addEventListener("keydown", (ev) => {
537
+ // Enter sends; Shift+Enter inserts a newline.
538
+ if (ev.key === "Enter" && !ev.shiftKey) {
539
+ ev.preventDefault();
540
+ form.requestSubmit();
541
+ }
542
+ });
543
+
544
+ return {
545
+ root,
546
+ destroy() {
547
+ try {
548
+ inFlight?.abort();
549
+ } catch {}
550
+ root.remove();
551
+ },
552
+ };
553
+ }
554
+
555
+ // ---------------------------------------------------------------------------
556
+ // v1 registration. We reach for `window.app` lazily — at module-eval time
557
+ // `app` may not yet be on `window`, but `registerExtension` itself queues.
558
+ // ---------------------------------------------------------------------------
559
+ const app = window.app ?? globalThis.app;
560
+ if (!app || typeof app.registerExtension !== "function") {
561
+ console.error(
562
+ "[comfyui-mcp] window.app.registerExtension is unavailable. " +
563
+ "This extension targets the v1 ComfyUI frontend API.",
564
+ );
565
+ } else {
566
+ // TODO(v2): replace with `defineExtension({ name, setup() {...} })`.
567
+ app.registerExtension({
568
+ name: "comfyui-mcp.agent-panel",
569
+ async setup() {
570
+ const tabId = "comfyui-mcp.agent";
571
+ let mounted = null; // { root, destroy }
572
+
573
+ const tabSpec = {
574
+ id: tabId,
575
+ title: "Agent",
576
+ // ComfyUI ships PrimeIcons; `pi-comments` is the closest "chat" glyph.
577
+ icon: "pi pi-comments",
578
+ tooltip: "comfyui-mcp Agent",
579
+ type: "custom",
580
+ render: (container) => {
581
+ if (mounted) mounted.destroy();
582
+ mounted = buildPanel();
583
+ container.appendChild(mounted.root);
584
+ },
585
+ destroy: () => {
586
+ mounted?.destroy();
587
+ mounted = null;
588
+ },
589
+ };
590
+
591
+ // TODO(v2): replace with `defineSidebarTab({ id, title, type: 'custom',
592
+ // icon, render, destroy })` imported from '@comfyorg/extension-api'.
593
+ const mgr = app.extensionManager;
594
+ if (mgr && typeof mgr.registerSidebarTab === "function") {
595
+ mgr.registerSidebarTab(tabSpec);
596
+ } else {
597
+ console.error(
598
+ "[comfyui-mcp] app.extensionManager.registerSidebarTab is unavailable; " +
599
+ "the agent panel cannot mount. Update ComfyUI to a version that exposes the extension manager.",
600
+ );
601
+ }
602
+ },
603
+ });
604
+ }