dsh-comfyui 0.3.0-beta.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +160 -0
- package/README.zh.md +160 -0
- package/client/client.js +3089 -0
- package/client/client.js.map +1 -0
- package/cordis.patch.yml +4 -0
- package/lib/analyze.d.ts +61 -0
- package/lib/analyze.js +96 -0
- package/lib/comfyui.d.ts +195 -0
- package/lib/comfyui.js +392 -0
- package/lib/config.d.ts +71 -0
- package/lib/config.js +28 -0
- package/lib/convert.d.ts +35 -0
- package/lib/convert.js +289 -0
- package/lib/host-hint.d.ts +29 -0
- package/lib/host-hint.js +73 -0
- package/lib/http.d.ts +15 -0
- package/lib/http.js +41 -0
- package/lib/index.d.ts +24 -0
- package/lib/index.js +344 -0
- package/lib/params.d.ts +63 -0
- package/lib/params.js +537 -0
- package/lib/progress.d.ts +27 -0
- package/lib/progress.js +98 -0
- package/lib/proxy.d.ts +13 -0
- package/lib/proxy.js +89 -0
- package/lib/queue.d.ts +50 -0
- package/lib/queue.js +103 -0
- package/lib/routes.d.ts +13 -0
- package/lib/routes.js +800 -0
- package/lib/skill.d.ts +12 -0
- package/lib/skill.js +72 -0
- package/lib/store.d.ts +120 -0
- package/lib/store.js +193 -0
- package/lib/templates.d.ts +36 -0
- package/lib/templates.js +84 -0
- package/lib/tools.d.ts +165 -0
- package/lib/tools.js +493 -0
- package/package.json +80 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 dsh-comfyui contributors
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,160 @@
|
|
|
1
|
+
# dsh-comfyui
|
|
2
|
+
|
|
3
|
+
> Version 0.2.0 — see [Roadmap & design boundaries](#roadmap--design-boundaries-v02) for this phase's confirmed scope.
|
|
4
|
+
|
|
5
|
+
ComfyUI integration for DeepSeek Harness (DH): let an agent generate and process images and videos directly against your ComfyUI server — submit workflows, run built-in templates, watch results render in the chat, and manage everything from a right-docked UI panel.
|
|
6
|
+
|
|
7
|
+
## Features
|
|
8
|
+
|
|
9
|
+
- **Agent tools**
|
|
10
|
+
- `comfyui_run` — submit a ComfyUI API-format workflow, or pick a built-in template, and get the generated media back. Two modes: `sync` (wait and return media) and `async` (start a background job, collect with `job_output` — ideal for video).
|
|
11
|
+
- `comfyui_object_info` — list the node definitions of your ComfyUI server so the agent can build valid workflows on the fly.
|
|
12
|
+
- `comfyui_workflow` — list and run runnable workflows from the plugin library. `action: list` also reports graph workflows you saved on the ComfyUI server and whether each has been **extracted** into runnable workflows; unextracted graphs are flagged so the agent tells you to extract them in the panel first.
|
|
13
|
+
- **UI panel** (right-docked; open from the sidebar rail) — three tabs:
|
|
14
|
+
- **Workflows** — the plugin library of runnable (API) workflows: create / edit / run / delete, plus "Import file" to load an API-format `.json` directly. The **ComfyUI 端保存** section auto-detects graph workflows you saved in ComfyUI, shows which runnable workflows were extracted from each, and offers **提取** with per-graph analysis: a canvas often holds several independent flows, so you choose 整体提取 / 按分量提取 / 只提取主流程. Workflows can be classified with **tags** (see below).
|
|
15
|
+
- **Assets** — everything the plugin generated, newest first, with a detail view and download links.
|
|
16
|
+
- **Queue** — a task center over ComfyUI's unified jobs API (`/api/jobs`): every task across the live queue **and** history in five states (pending / in progress / completed / failed / cancelled), filterable by state, with per-task progress bars (plugin-submitted jobs), preview thumbnails, failure messages and duration, plus actions — delete, interrupt, rerun, clear queue/history, free memory. Tasks the plugin queued are marked with their workflow name.
|
|
17
|
+
- **Load area(加载区)** — an image loader in the style of ComfyUI's LoadImage node, sitting at the top of the Workflows tab:
|
|
18
|
+
- Shows the **current source image** as a large preview; clicking opens a picker window with a nav bar (全部 / 已导入 / 已生成), a type filter, a paste/upload drop zone on the right, and a masonry grid of every image in the ComfyUI `input` directory plus everything the plugin generated. Picking an image closes the window and makes it the current source.
|
|
19
|
+
- The current source image is the **default input image**: any workflow run that has an unset image-type parameter uses it automatically — the agent does not need to name a file.
|
|
20
|
+
- **Resolution auto-match**: uploads record their pixel size; when a run leaves `width`/`height` unset, they default to the source image's actual size.
|
|
21
|
+
- **Hash naming + dedup**: uploads are renamed to `原名_短哈希.ext` (SHA-256 first 10 hex); re-uploading an identical file reuses the existing name instead of storing a duplicate. The picker refreshes live after uploads.
|
|
22
|
+
- Selecting a **generated** image copies it from the output directory into `input` on the fly, so image-loading nodes can use it.
|
|
23
|
+
- **Workflow tags** — classify runnable workflows with preset categories (图生图 / 文生图 / 文生视频 / 图生视频 / 参考生视频 / 文生音频 / 参考生音频) plus any custom tags. Tags are edited in the workflow editor, shown on the top-right of each card, and the library list has a filter bar with per-tag counts.
|
|
24
|
+
- **Built-in templates** — `txt2img` (SDXL text-to-image), `img2img` (SDXL image-to-image), `video` (Wan 2.1 text-to-video via ComfyUI-WanVideoWrapper). Template node ids are documented in the tool description so the agent overrides the right inputs.
|
|
25
|
+
- **Media proxy** — generated files stream through a same-origin route (`/comfyui/media`), so the browser never talks to ComfyUI directly: no CORS, no mixed-content, no API key in the page, and remote ComfyUI installs work unchanged.
|
|
26
|
+
- **Tool card** — results render as a media wall (images/videos with download links) right in the chat, including background-job status.
|
|
27
|
+
- **Settings page** — a ComfyUI section in the DH settings where you can edit the server URL, API-key env var, data directory, and asset cap, test the connection, and switch the plugin UI language (中文 / English — stored in the browser, applies to the whole plugin UI), all without touching `cordis.yml`.
|
|
28
|
+
- **Companion skill** — a runtime skill (`dsh-comfyui-workflows`) registered through `ctx.skills.register`: the agent learns the graph-vs-runnable model, canvas analysis rules (connected components, bypassed groups, dangling nodes), when to ask you about extract mode, and the extraction tech rules.
|
|
29
|
+
|
|
30
|
+
### Graph workflows vs runnable workflows (提取执行流)
|
|
31
|
+
|
|
32
|
+
ComfyUI works in two layers:
|
|
33
|
+
|
|
34
|
+
- **Graph workflow(衍生主题)** — the UI graph you save in ComfyUI (nodes/links/widgets). It is a source, not directly runnable, and a single canvas is often a **test bench holding several independent flows at once**. Visual `groups` are just rectangles — the real executable unit is a **connected component** over the links, with bypassed (`mode 4`) and dangling nodes excluded.
|
|
35
|
+
- **Runnable workflow(运行主题)** — an API-format prompt, the actual execution unit. You get one by **extracting** it from a graph (1 graph → N runnable workflows) or by pasting/importing an API `.json` directly.
|
|
36
|
+
|
|
37
|
+
The **提取** flow analyzes the canvas first (components with their node counts and group membership, bypassed/dangling counts) and lets you pick:
|
|
38
|
+
|
|
39
|
+
- **整体提取** — merge every component into one runnable workflow (everything executes together).
|
|
40
|
+
- **按分量提取** (recommended) — one runnable workflow per independent flow.
|
|
41
|
+
- **只提取主流程** — the largest component only (usually the flow under test).
|
|
42
|
+
|
|
43
|
+
Extraction follows the live `/object_info`: rewires Reroute/bypass pass-throughs, maps widget values (including dynamic sub-widgets and `control_after_generate`), inlines primitives, drops stale slot references with a warning, skips components with no output node, and fails loudly when a required input was never wired — every extracted workflow is validated by `POST /prompt` (zero `node_errors`) before it is stored.
|
|
44
|
+
|
|
45
|
+
### Adjustable parameters
|
|
46
|
+
|
|
47
|
+
Every runnable workflow carries an adjustable **parameter set** (`parameters`) so one workflow can produce different results per run:
|
|
48
|
+
|
|
49
|
+
- **Auto-detected (conservative)**: prompt text inputs, resolution (`EmptyLatentImage` width/height), sampler steps (`KSampler.steps`), and seed (`KSampler.seed`, randomized per run by default). Sampling-related inputs (cfg/denoise) and model selections stay as authored.
|
|
50
|
+
- **Advanced parameters**: the panel's workflow editor can expose any node input as a custom parameter (pick node → pick input → name it), and edit each parameter's name, label, default, and random toggle.
|
|
51
|
+
- **Agent-facing**: the parameter list is written into the workflow's input notes (`inputs` field) and shown by `comfyui_workflow` `action: list`; `action: run` accepts `parameters: {"prompt": "...", "seed": 42}` overrides — explicit values win over random/default, omitted parameters use their defaults.
|
|
52
|
+
- **Load-area integration**: an unset image-type parameter is filled with the load-area's current source image, and unset `width`/`height` are auto-matched to that image's recorded pixel size. Explicit values always win, so the agent can still override both.
|
|
53
|
+
|
|
54
|
+
## Requirements
|
|
55
|
+
|
|
56
|
+
- DeepSeek Harness (web profile) — the plugin targets the `web` profile (`@deepseek-ai/dsh-web-app` ≥ 0.1.0-rc.6).
|
|
57
|
+
- A running [ComfyUI](https://github.com/comfystack/ComfyUI) server (default `http://127.0.0.1:8188`).
|
|
58
|
+
- For the `video` template: the [ComfyUI-WanVideoWrapper](https://github.com/kijai/ComfyUI-WanVideoWrapper) custom nodes and Wan 2.1 model files.
|
|
59
|
+
|
|
60
|
+
## Installation
|
|
61
|
+
|
|
62
|
+
```sh
|
|
63
|
+
dsh plugin --profile web add dsh-comfyui
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
Then restart the web server (host-side rows mount at boot). The panel trigger appears in the sidebar rail, the settings section "ComfyUI" appears in the settings page, and the agent gains `comfyui_run`, `comfyui_object_info`, `comfyui_workflow`, and the `dsh-comfyui-workflows` skill immediately.
|
|
67
|
+
|
|
68
|
+
### API key (remote servers)
|
|
69
|
+
|
|
70
|
+
For a remote ComfyUI behind an authenticating proxy, provide the key through the credentials store or an environment variable named by `apiKeyEnv` (default `COMFYUI_API_KEY`). The key is resolved per request on the host and never sent to the browser.
|
|
71
|
+
|
|
72
|
+
## Usage
|
|
73
|
+
|
|
74
|
+
Ask the agent, e.g.:
|
|
75
|
+
|
|
76
|
+
- "用 ComfyUI 画一张红猫的图" / "Generate a red cat image with ComfyUI"
|
|
77
|
+
- "把这幅图转成赛博朋克风格" (img2img with an input image filename)
|
|
78
|
+
- "把加载区这张动漫图转成真人照片,分辨率跟原图一致" (load-area source image + resolution auto-match)
|
|
79
|
+
- "生成一段 5 秒的短视频:日落下的城市" (video; needs the Wan wrapper)
|
|
80
|
+
- "用我之前在 ComfyUI 里保存的 Krea-Afterlight 跑一下" — the agent lists server-side graphs; if yours is not extracted yet it will tell you to click **提取** in the panel first (a canvas may contain several independent flows, so you can extract all as one, per component, or the main flow only).
|
|
81
|
+
|
|
82
|
+
The agent picks a template, inspects your server with `comfyui_object_info`, or runs a library workflow with `comfyui_workflow`.
|
|
83
|
+
|
|
84
|
+
### Configuration
|
|
85
|
+
|
|
86
|
+
The plugin reads a `comfyui` section from `cordis.yml` (or the settings page):
|
|
87
|
+
|
|
88
|
+
```yaml
|
|
89
|
+
# cordis.yml
|
|
90
|
+
- id: comfyui
|
|
91
|
+
name: dsh-comfyui
|
|
92
|
+
config:
|
|
93
|
+
baseUrl: http://127.0.0.1:8188
|
|
94
|
+
apiKeyEnv: COMFYUI_API_KEY
|
|
95
|
+
timeoutMs: 180000
|
|
96
|
+
maxMediaItems: 12
|
|
97
|
+
dataDir: ''
|
|
98
|
+
maxAssets: 200
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
| Key | Default | Description |
|
|
102
|
+
| --- | --- | --- |
|
|
103
|
+
| `baseUrl` | `http://127.0.0.1:8188` | ComfyUI HTTP server base URL |
|
|
104
|
+
| `apiKeyEnv` | `COMFYUI_API_KEY` | Env-variable / credential name for the optional API key |
|
|
105
|
+
| `connectTimeoutMs` | `10000` | Per-request connect/read timeout |
|
|
106
|
+
| `timeoutMs` | `180000` | Sync generation wait budget (raise for video) |
|
|
107
|
+
| `pollIntervalMs` | `1000` | History polling interval while waiting |
|
|
108
|
+
| `maxMediaItems` | `12` | Max media items returned per workflow |
|
|
109
|
+
| `maxMediaBytes` | `67108864` | Max bytes the media proxy streams per file |
|
|
110
|
+
| `dataDir` | *(DSH data dir)* | Where the workflow library and asset index live (`$DSH_HOME/data/dsh-comfyui` by default) |
|
|
111
|
+
| `maxAssets` | `200` | Max entries kept in the asset index |
|
|
112
|
+
|
|
113
|
+
## Roadmap & design boundaries (v0.2)
|
|
114
|
+
|
|
115
|
+
Confirmed scope decisions for this phase:
|
|
116
|
+
|
|
117
|
+
- **Image-to-video / reference-to-video / audio variants** are not plugin-side features: they share the same upload node as image-to-image, so any such workflow (extracted or imported) works as-is. Add the workflow, not plugin code.
|
|
118
|
+
- **No front-end parameter presets/favorites.** Advanced customization happens in the workflow editor (edit parameter defaults, or add an advanced parameter for any node input). This keeps one source of truth.
|
|
119
|
+
- **Model-strength knobs** (e.g. `ref_boost`) are handled through the advanced-parameter mechanism: expose the node input, label its effect, and the agent can tune it per run.
|
|
120
|
+
- **Planned**: auto-commit of run parameters — after a successful run, save the used parameter values as that workflow's new defaults, so the next run opens with the previous session's values instead of the authored ones.
|
|
121
|
+
|
|
122
|
+
## Security
|
|
123
|
+
|
|
124
|
+
- The tools only ever connect to the **configured** `baseUrl` — the agent cannot name arbitrary targets (SSRF containment).
|
|
125
|
+
- The API key lives on the host (credentials store / environment), resolved per request; `/comfyui/config` reports only `hasApiKey`.
|
|
126
|
+
- Media is capped by size; config writes require same-origin requests.
|
|
127
|
+
- Workflows extracted from ComfyUI are validated (non-empty `class_type`, object inputs, zero `node_errors` on the server) before they are stored.
|
|
128
|
+
- Headless profiles without a web server keep the tools and skip the routes silently.
|
|
129
|
+
|
|
130
|
+
## Architecture
|
|
131
|
+
|
|
132
|
+
A single npm package with two halves, following the DH plugin conventions:
|
|
133
|
+
|
|
134
|
+
- `src/index.ts` — host entry: `inject: ['tools']`; registers the tools and the companion skill (`ctx.skills.register`, optional service), and mounts routes + media proxy on a `webServer` sub-fiber (`ctx.inject`) so concurrent entry settling can never skip the mounts.
|
|
135
|
+
- `src/comfyui.ts` — minimal ComfyUI HTTP client (queue prompt, poll history, object_info, system_stats, interrupt, view download, userdata list/read).
|
|
136
|
+
- `src/analyze.ts` — canvas analysis: connected components over active nodes, group membership, dangling/isolated nodes, bypassed counts.
|
|
137
|
+
- `src/convert.ts` — graph → API extraction: rewires links (Reroute / bypass pass-through), derives widget order from the graph's own input array plus object_info (including `control_after_generate` and dynamic sub-widgets), inlines primitive values, drops stale slot references with a warning, and fails loudly when a required input was never wired.
|
|
138
|
+
- `src/skill.ts` — the `dsh-comfyui-workflows` companion skill body.
|
|
139
|
+
- `src/params.ts` — parameter application: fills an unset image parameter with the load-area source image, auto-matches `width`/`height` to the source's recorded size, and re-syncs DynamicCombo parent/child pairs.
|
|
140
|
+
- `src/store.ts` — the workflow library and asset index on disk (`workflows.json` + `assets.json`), plus the load-area records (`current-image.json`, `media-sizes.json`, `media-hashes.json`).
|
|
141
|
+
- `src/queue.ts` — tracks prompts the plugin queued and moves completed ones into the asset index (sweep-on-read, no timers).
|
|
142
|
+
- `src/tools.ts` — `ToolDefinition`s registered through `ctx.tools.register`; results carry `presentationMeta` so the client card renders from the session log.
|
|
143
|
+
- `src/routes.ts` — same-origin HTTP routes for the panel (config, workflows, ComfyUI-side graphs + analyze/extract, assets, queue, run, load area, upload with hash dedup + size recording).
|
|
144
|
+
- `src/client/` — browser half: the `shell.overlay` right-docked panel + `sidebar.footer.action` trigger, the `tool.call.toolview` card (key `comfyui_run`), and the `settings.section` page (id `comfyui`).
|
|
145
|
+
- `cordis.patch.yml` — the `dsh.bundle.patch` layer that inserts the `comfyui` row into a profile.
|
|
146
|
+
|
|
147
|
+
## Development
|
|
148
|
+
|
|
149
|
+
```sh
|
|
150
|
+
pnpm install
|
|
151
|
+
npm run typecheck # host + client
|
|
152
|
+
npm run build # tsc (host lib/) + tsdown (client bundle)
|
|
153
|
+
npm pack --dry-run # inspect the publish contents
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
To test against a local profile: `dsh plugin --profile web add <path-to-this-repo>` (pnpm links the directory), restart the web server, and rebuild with `npm run build` after changes.
|
|
157
|
+
|
|
158
|
+
## License
|
|
159
|
+
|
|
160
|
+
MIT
|
package/README.zh.md
ADDED
|
@@ -0,0 +1,160 @@
|
|
|
1
|
+
# dsh-comfyui
|
|
2
|
+
|
|
3
|
+
> 版本 0.2.0 —— 本阶段确认的功能范围见 [Roadmap 与设计边界](#roadmap-与设计边界v02)。
|
|
4
|
+
|
|
5
|
+
DeepSeek Harness (DH) 的 ComfyUI 集成插件:让 Agent 直接调用你的 ComfyUI 服务器生成和处理图像、视频——提交工作流、使用内置模板、在对话里直接看到生成结果,并通过右侧停靠的 UI 面板管理一切。
|
|
6
|
+
|
|
7
|
+
## 功能
|
|
8
|
+
|
|
9
|
+
- **Agent 工具**
|
|
10
|
+
- `comfyui_run` —— 提交 ComfyUI API 格式的工作流,或选用内置模板,返回生成的媒体。两种模式:`sync`(等待并返回媒体)与 `async`(后台任务,用 `job_output` 收集结果——视频生成强烈建议)。
|
|
11
|
+
- `comfyui_object_info` —— 列出你的 ComfyUI 服务器支持的节点定义,让 Agent 能当场构造合法的工作流。
|
|
12
|
+
- `comfyui_workflow` —— 列出并运行插件库中的可运行工作流。`action: list` 还会报告你在 ComfyUI 端保存的图工作流以及每个是否已**提取**出可运行执行流;未提取的会明确标注,Agent 会先转告你在面板里点"提取"。
|
|
13
|
+
- **UI 面板**(右侧停靠;从侧边栏轨道打开)——三个页签:
|
|
14
|
+
- **工作流** —— 插件库(可运行的 API 工作流):新建 / 编辑 / 运行 / 删除,支持"导入文件"直接加载 API 格式 `.json`。"ComfyUI 端保存"分区自动检测你在 ComfyUI 里保存的图工作流,显示每个已提取出的执行流,并提供**提取**:画布上往往躺着多个独立流程,可整体提取 / 按分量提取 / 只提取主流程。工作流可用**标签**分类(见下)。
|
|
15
|
+
- **资产** —— 插件生成的所有结果,最新在前,带详情视图和下载链接。
|
|
16
|
+
- **队列** —— 基于 ComfyUI 统一任务 API(`/api/jobs`)的任务中心:实时队列**和**历史里所有任务按五态展示(待生成 / 生成中 / 已完成 / 失败 / 已取消),可按状态筛选;插件提交的任务带进度条(进度来自 ComfyUI WS 的 progress 事件),终态任务带预览缩略图、失败原因与耗时;支持操作:删除、中断、重跑、清空队列/历史、释放内存。插件提交的任务标注工作流名。
|
|
17
|
+
- **加载区** —— 仿 ComfyUI LoadImage 节点的图像加载器,位于工作流页顶部:
|
|
18
|
+
- 当前源图以大图展示;点击打开加载窗口:上方导航条(全部 / 已导入 / 已生成)、类型筛选、右侧粘贴/上传区、下方瀑布流网格展示 ComfyUI `input` 目录全部图片 + 插件生成的全部结果。点击图像即选定并关闭窗口,成为当前源图。
|
|
19
|
+
- 当前源图即**默认输入图**:任何含图片类参数(未显式指定)的工作流运行都会自动使用它——无需指定文件名。
|
|
20
|
+
- **分辨率自动匹配**:上传时记录像素尺寸;运行时不传 `width`/`height` 则自动用源图实际分辨率。
|
|
21
|
+
- **哈希命名 + 去重**:上传重命名为 `原名_短哈希.ext`(SHA-256 前 10 位十六进制);重复上传相同文件直接复用已有文件名,不产生重复存储。上传后列表实时刷新。
|
|
22
|
+
- 选中**已生成**的图时会自动从输出目录复制到 `input`,图像加载节点即可使用。
|
|
23
|
+
- **工作流标签** —— 用预设分类(图生图 / 文生图 / 文生视频 / 图生视频 / 参考生视频 / 文生音频 / 参考生音频)+ 自定义标签给可运行工作流分类。标签在编辑工作流时勾选/添加,显示在卡片右上角,列表顶部有带计数的标签筛选条。
|
|
24
|
+
- **内置模板** —— `txt2img`(SDXL 文生图)、`img2img`(SDXL 图生图)、`video`(Wan 2.1 文生视频,需要 ComfyUI-WanVideoWrapper)。模板节点 id 写在工具描述里,Agent 会覆盖正确的输入。
|
|
25
|
+
- **媒体代理** —— 生成文件经同源路由(`/comfyui/media`)转发,浏览器不直接接触 ComfyUI:没有 CORS、没有混合内容、页面里不出现 API Key,远程 ComfyUI 部署也可直接使用。
|
|
26
|
+
- **工具卡片** —— 结果在对话里渲染成媒体墙(图片/视频带下载链接),后台任务也有状态提示。
|
|
27
|
+
- **设置页** —— DH 设置里新增 "ComfyUI" 分区:改服务器地址、API Key 环境变量名、数据目录、资产上限、测试连接,并可切换插件界面语言(中文 / English——存于浏览器,作用于整个插件 UI),无需改动 `cordis.yml`。
|
|
28
|
+
- **配套 skill** —— 通过 `ctx.skills.register` 注册的运行时 skill(`dsh-comfyui-workflows`):让 Agent 掌握图工作流 vs 执行流的概念、画布分析规则(连通分量、绕过组、悬空节点)、何时该询问你提取方式,以及图→API 提取的技术规则。
|
|
29
|
+
|
|
30
|
+
### 图工作流 vs 可运行工作流(提取执行流)
|
|
31
|
+
|
|
32
|
+
ComfyUI 分两层:
|
|
33
|
+
|
|
34
|
+
- **图工作流(衍生主题)** —— 你在 ComfyUI 里保存的 UI 图(nodes/links/widgets),是"源",不能直接运行。一个画布常常是**同时测试多个独立流程的试验台**;视觉 `groups` 只是矩形,真正可执行单元是**连通分量**(按连线连通,排除绕过与悬空节点)。
|
|
35
|
+
- **可运行工作流(运行主题)** —— API 格式的 prompt,真正的执行单元。从图里**提取**出来(1 图 → N 个执行流),或直接粘贴/导入 API `.json`。
|
|
36
|
+
|
|
37
|
+
**提取**流程会先分析画布(各分量的节点数与所在组、绕过/悬空计数),然后让你选择:
|
|
38
|
+
|
|
39
|
+
- **整体提取** —— 所有分量合成一个执行流(运行时全部执行)。
|
|
40
|
+
- **按分量提取**(推荐)—— 每个独立流程一个执行流。
|
|
41
|
+
- **只提取主流程** —— 只取最大分量(通常是当前测试区块)。
|
|
42
|
+
|
|
43
|
+
提取严格对照实时 `/object_info`:重排 Reroute/bypass 直通、映射 widget 值(含动态子 widget 与 `control_after_generate`)、内联 Primitive、丢弃失效的输出槽位引用并警告、跳过无输出节点的分量、必需输入缺失时明确报错——每个提取出的执行流都会先经 `POST /prompt` 校验(`node_errors` 为空)才入库。
|
|
44
|
+
|
|
45
|
+
### 可调参数
|
|
46
|
+
|
|
47
|
+
每个可运行工作流带一组**可调参数**(`parameters`),让同一流程按需生成不同结果:
|
|
48
|
+
|
|
49
|
+
- **自动识别(保守集)**:提取时自动提取 提示词(文本输入节点)、分辨率(`EmptyLatentImage` 宽高)、采样步数(`KSampler.steps`)、种子(`KSampler.seed`,默认**每次运行随机**)。采样相关(cfg/denoise)与模型选择不暴露,保持工作流原样。
|
|
50
|
+
- **高级参数**:面板"编辑工作流"里可手动暴露任意节点的任意输入为参数(选节点 → 选输入 → 命名),并调整每个参数的名称、显示名、默认值、是否随机。
|
|
51
|
+
- **Agent 感知**:参数清单自动写入工作流的"参数说明"(`inputs` 字段),`comfyui_workflow` 工具的 `action: list` 会展示;`action: run` 接受 `parameters: {"prompt": "...", "seed": 42}` 覆盖——显式传值优先于随机/默认,未传参数用默认值。
|
|
52
|
+
- **加载区联动**:未显式指定的图片类参数自动填入加载区当前源图;未传的 `width`/`height` 自动匹配该图记录的像素尺寸。显式传值始终优先,Agent 仍可覆盖两者。
|
|
53
|
+
|
|
54
|
+
## 环境要求
|
|
55
|
+
|
|
56
|
+
- DeepSeek Harness(web profile)—— 本插件面向 `web` profile(`@deepseek-ai/dsh-web-app` ≥ 0.1.0-rc.6)。
|
|
57
|
+
- 一个运行中的 [ComfyUI](https://github.com/comfystack/ComfyUI) 服务器(默认 `http://127.0.0.1:8188`)。
|
|
58
|
+
- 使用 `video` 模板需要 [ComfyUI-WanVideoWrapper](https://github.com/kijai/ComfyUI-WanVideoWrapper) 自定义节点和 Wan 2.1 模型文件。
|
|
59
|
+
|
|
60
|
+
## 安装
|
|
61
|
+
|
|
62
|
+
```sh
|
|
63
|
+
dsh plugin --profile web add dsh-comfyui
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
然后重启 Web 服务(Host 端行在启动时挂载)。侧边栏轨道出现面板入口,设置页出现 "ComfyUI" 分区,Agent 立即获得 `comfyui_run`、`comfyui_object_info`、`comfyui_workflow` 与 `dsh-comfyui-workflows` skill。
|
|
67
|
+
|
|
68
|
+
### API Key(远程服务器)
|
|
69
|
+
|
|
70
|
+
远程 ComfyUI 若位于需要鉴权的代理之后,通过凭据存储或 `apiKeyEnv` 指定的环境变量(默认 `COMFYUI_API_KEY`)提供密钥。密钥在 Host 端按请求解析,绝不发给浏览器。
|
|
71
|
+
|
|
72
|
+
## 使用
|
|
73
|
+
|
|
74
|
+
直接告诉 Agent,例如:
|
|
75
|
+
|
|
76
|
+
- "用 ComfyUI 画一张红猫的图"
|
|
77
|
+
- "把这幅图转成赛博朋克风格"(img2img,需要输入图片文件名)
|
|
78
|
+
- "把加载区这张动漫图转成真人照片,分辨率跟原图一致"(加载区源图 + 分辨率自动匹配)
|
|
79
|
+
- "生成一段 5 秒的短视频:日落下的城市"(video,需要 Wan 插件)
|
|
80
|
+
- "用我之前在 ComfyUI 里保存的 Krea-Afterlight 跑一下" —— Agent 会列出服务器端的图工作流;如果你指的那个还没提取,它会先转告你去面板里点**提取**(画布可能含多个独立流程,可选整体/按分量/主流程)。
|
|
81
|
+
|
|
82
|
+
Agent 会选用模板,或用 `comfyui_object_info` 探查你的服务器,或用 `comfyui_workflow` 运行插件库里已保存的执行流。
|
|
83
|
+
|
|
84
|
+
### 配置
|
|
85
|
+
|
|
86
|
+
插件读取 `cordis.yml` 中的 `comfyui` 段(或通过设置页修改):
|
|
87
|
+
|
|
88
|
+
```yaml
|
|
89
|
+
# cordis.yml
|
|
90
|
+
- id: comfyui
|
|
91
|
+
name: dsh-comfyui
|
|
92
|
+
config:
|
|
93
|
+
baseUrl: http://127.0.0.1:8188
|
|
94
|
+
apiKeyEnv: COMFYUI_API_KEY
|
|
95
|
+
timeoutMs: 180000
|
|
96
|
+
maxMediaItems: 12
|
|
97
|
+
dataDir: ''
|
|
98
|
+
maxAssets: 200
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
| 键 | 默认值 | 说明 |
|
|
102
|
+
| --- | --- | --- |
|
|
103
|
+
| `baseUrl` | `http://127.0.0.1:8188` | ComfyUI HTTP 服务器地址 |
|
|
104
|
+
| `apiKeyEnv` | `COMFYUI_API_KEY` | 可选 API Key 的环境变量/凭据名 |
|
|
105
|
+
| `connectTimeoutMs` | `10000` | 单次请求连接/读取超时 |
|
|
106
|
+
| `timeoutMs` | `180000` | 同步生成的等待预算(视频请调大) |
|
|
107
|
+
| `pollIntervalMs` | `1000` | 等待完成时的历史轮询间隔 |
|
|
108
|
+
| `maxMediaItems` | `12` | 每个工作流最多返回的媒体条数 |
|
|
109
|
+
| `maxMediaBytes` | `67108864` | 媒体代理单文件流式传输上限(字节) |
|
|
110
|
+
| `dataDir` | *(DSH 数据目录)* | 工作流库与资产索引存放位置(默认 `$DSH_HOME/data/dsh-comfyui`) |
|
|
111
|
+
| `maxAssets` | `200` | 资产索引最多保留的条数 |
|
|
112
|
+
|
|
113
|
+
## Roadmap 与设计边界(v0.2)
|
|
114
|
+
|
|
115
|
+
本阶段确认的范围决策:
|
|
116
|
+
|
|
117
|
+
- **图生视频 / 参考生视频 / 音视频变体**不是插件侧功能:它们与图生图共用同一个上传节点,任何这类工作流(提取或导入)开箱即用。缺的是工作流本身,不是插件代码。
|
|
118
|
+
- **不做前端参数预设/收藏**。高级定制在工作流编辑里完成(修改参数默认值,或把任意节点输入添加为高级参数)。保持唯一事实来源。
|
|
119
|
+
- **模型强度类参数**(如 `ref_boost`)走高级参数机制:暴露节点输入、标注作用,Agent 即可按次调节。
|
|
120
|
+
- **规划中**:运行参数自动回写——运行成功后把本次使用的参数值保存为该工作流的新默认值,下次打开就是上一次会话用的值,而不是作者初始值。
|
|
121
|
+
|
|
122
|
+
## 安全
|
|
123
|
+
|
|
124
|
+
- 工具只会连接**配置好的** `baseUrl` —— Agent 无法指定任意目标(SSRF 遏制)。
|
|
125
|
+
- API Key 只存在于 Host(凭据存储/环境变量),按请求解析;`/comfyui/config` 只回传 `hasApiKey`。
|
|
126
|
+
- 媒体有大小上限;配置写入要求同源请求。
|
|
127
|
+
- 从 ComfyUI 提取的执行流会先校验(非空 `class_type`、对象型 inputs、服务器 `node_errors` 为空)再入库。
|
|
128
|
+
- 无 Web 服务器的 headless profile 只保留工具,路由静默跳过。
|
|
129
|
+
|
|
130
|
+
## 架构
|
|
131
|
+
|
|
132
|
+
一个 npm 包、双端实现,遵循 DH 插件约定:
|
|
133
|
+
|
|
134
|
+
- `src/index.ts` —— Host 入口:`inject: ['tools']`;注册工具与配套 skill(`ctx.skills.register`,可选服务),并把路由与媒体代理挂到 `webServer` 子 fiber(`ctx.inject`)上,避免并发 settle 导致挂载被静默跳过。
|
|
135
|
+
- `src/comfyui.ts` —— 精简的 ComfyUI HTTP 客户端(排队、轮询历史、object_info、system_stats、interrupt、view 下载、userdata 列表/读取)。
|
|
136
|
+
- `src/analyze.ts` —— 画布分析:激活节点的连通分量、组归属、悬空/孤立节点、绕过计数。
|
|
137
|
+
- `src/convert.ts` —— 图 → API 提取:重排连线(Reroute / bypass 直通)、按图自身输入顺序 + object_info 推导 widget 顺序(含 `control_after_generate` 与动态子 widget)、内联 Primitive 值、丢弃失效的输出槽位引用并警告、必需输入缺失时明确报错。
|
|
138
|
+
- `src/skill.ts` —— `dsh-comfyui-workflows` 配套 skill 正文。
|
|
139
|
+
- `src/params.ts` —— 参数应用:未指定的图片参数填入加载区源图、`width`/`height` 自动匹配源图记录尺寸、DynamicCombo 父子联动同步。
|
|
140
|
+
- `src/store.ts` —— 工作流库与资产索引落盘(`workflows.json` + `assets.json`),以及加载区记录(`current-image.json`、`media-sizes.json`、`media-hashes.json`)。
|
|
141
|
+
- `src/queue.ts` —— 追踪插件提交的 prompt,完成后移入资产索引(读时清扫,无定时器)。
|
|
142
|
+
- `src/tools.ts` —— 通过 `ctx.tools.register` 注册的 `ToolDefinition`;结果携带 `presentationMeta`,客户端卡片从会话日志渲染。
|
|
143
|
+
- `src/routes.ts` —— 面板用的同源 HTTP 路由(配置、工作流、ComfyUI 端图 + 分析/提取、资产、队列、运行、加载区、带哈希去重与尺寸记录的上传)。
|
|
144
|
+
- `src/client/` —— 浏览器端:`shell.overlay` 右侧停靠面板 + `sidebar.footer.action` 触发器、`tool.call.toolview` 卡片(key `comfyui_run`)、`settings.section` 设置页(id `comfyui`)。
|
|
145
|
+
- `cordis.patch.yml` —— `dsh.bundle.patch` 补丁层,把 `comfyui` 行插入 profile。
|
|
146
|
+
|
|
147
|
+
## 开发
|
|
148
|
+
|
|
149
|
+
```sh
|
|
150
|
+
pnpm install
|
|
151
|
+
npm run typecheck # host + client
|
|
152
|
+
npm run build # tsc(host lib/)+ tsdown(client bundle)
|
|
153
|
+
npm pack --dry-run # 检查发布内容
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
本地测试:`dsh plugin --profile web add <本仓库路径>`(pnpm 链接目录),重启 Web 服务,改动后 `npm run build` 重新构建。
|
|
157
|
+
|
|
158
|
+
## License
|
|
159
|
+
|
|
160
|
+
MIT
|