@mokuyoaxis/dsh-iris 0.2.0 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.dsh/skills/iris-compose-media/SKILL.md +10 -5
- package/.dsh/skills/iris-compose-media/references/workflows.md +7 -1
- package/.dsh/skills/iris-verify-ui/SKILL.md +2 -1
- package/CHANGELOG.md +47 -2
- package/README.en.md +65 -29
- package/README.md +65 -29
- package/bin/dsh-iris.js +222 -21
- package/docs/AGENT_QUICKSTART.md +10 -6
- package/docs/ARCHITECTURE.md +25 -18
- package/docs/ARTIFACT_LIBRARY.md +5 -5
- package/docs/ARTIFACT_MANIFEST.md +1 -1
- package/docs/CLI_MANAGEMENT.md +181 -0
- package/docs/COMPOSITE_VISION.md +35 -0
- package/docs/CORE_RUNTIME_CONTRACT.md +8 -7
- package/docs/DSH_CORE_MIGRATION.md +48 -19
- package/docs/DSH_RC2_ADAPTATION.md +4 -2
- package/docs/FAULT_INJECTION.md +12 -2
- package/docs/HEADLESS_CLI.md +13 -7
- package/docs/HOST_ADAPTER_CONTRACT.md +10 -10
- package/docs/HOST_DOCTOR.md +3 -3
- package/docs/MODEL_PORT_CONTRACT.md +19 -14
- package/docs/OCR_MODEL.md +39 -0
- package/docs/PROMPT_OPTIMIZER.md +47 -0
- package/docs/PROVIDER_ADAPTER_CONTRACT.md +6 -4
- package/docs/PROVIDER_HEALTH.md +22 -5
- package/docs/PROVIDER_RUNTIME_HARNESS.md +1 -1
- package/docs/PROVIDER_SUBMISSION_CONTRACT.md +2 -2
- package/docs/ROADMAP.md +29 -19
- package/docs/SECURITY.md +5 -5
- package/docs/TASK_SEMANTICS.md +2 -2
- package/docs/VISION_CLI.md +132 -0
- package/docs/VISION_MODEL.md +51 -0
- package/docs/WORKBENCH_ARTIFACTS.md +67 -0
- package/docs/file-access-across-environments.md +8 -5
- package/docs/releases/0.2.1.md +56 -0
- package/lib/actions.js +187 -270
- package/lib/adapters.js +133 -22
- package/lib/api.js +49 -7
- package/lib/chromium-browser.js +117 -0
- package/lib/client.js +671 -101
- package/lib/command-service.js +121 -7
- package/lib/composite-vision-routing.js +42 -0
- package/lib/config.js +89 -10
- package/lib/core-artifact-store.js +45 -13
- package/lib/core-maintenance.js +213 -0
- package/lib/core-tasks.js +12 -5
- package/lib/core-user-projection.js +1 -0
- package/lib/dsh-core-adapter.js +63 -9
- package/lib/dsh-text-model-adapter.js +222 -0
- package/lib/dsh-vision-model-adapter.js +75 -0
- package/lib/generation-input.js +36 -5
- package/lib/headless-vision.js +181 -0
- package/lib/host-runtime.js +1 -0
- package/lib/http-vision-model-adapter.js +187 -0
- package/lib/image-edit-input.js +34 -0
- package/lib/index.js +205 -175
- package/lib/locate.js +86 -28
- package/lib/media-probe.js +2 -1
- package/lib/model-call-runtime.js +45 -0
- package/lib/model-invoker.js +4 -2
- package/lib/model-port-contract.js +13 -5
- package/lib/models.js +10 -5
- package/lib/ocr-model-routing.js +26 -0
- package/lib/ocr.js +110 -64
- package/lib/prompt-optimizer-config.js +2 -1
- package/lib/prompt-optimizer-core.js +121 -0
- package/lib/prompt-optimizer.js +37 -193
- package/lib/provider-adapters.js +85 -9
- package/lib/provider-catalog.js +19 -7
- package/lib/provider-config-service.js +236 -0
- package/lib/provider-contract.js +3 -2
- package/lib/provider-health.js +80 -1
- package/lib/provider-protocol.js +23 -0
- package/lib/provider-task-runner.js +58 -16
- package/lib/summarize.js +31 -20
- package/lib/summary-input.js +52 -0
- package/lib/task-wait.js +40 -0
- package/lib/video-input.js +30 -0
- package/lib/vision-core.js +42 -0
- package/lib/vision-image-input.js +83 -0
- package/lib/vision-input.js +31 -0
- package/lib/vision-model-routing.js +180 -0
- package/lib/vision.js +3 -2
- package/lib/workbench-artifacts.js +52 -0
- package/lib/workbench-management.js +151 -0
- package/lib/workbench-zip.js +39 -0
- package/package.json +1 -1
- package/user_guide.md +52 -22
|
@@ -35,11 +35,13 @@ Do not add an analysis, transcription, or review step merely because a tool exis
|
|
|
35
35
|
|
|
36
36
|
## Resolve media inputs
|
|
37
37
|
|
|
38
|
-
-
|
|
38
|
+
- For edits to an existing Core image, pass its ID as `source_artifact_id` to `iris_edit_image` with the requested change. This creates a new Task and Artifact while keeping the source intact; it requires a configured chat image model.
|
|
39
|
+
- PNG, JPEG and WebP can be used directly; do not convert a generated PNG to JPEG merely to call vision or OCR. Core IDs and host paths remain separate inputs.
|
|
40
|
+
- Use `iris_look_at_image` for a host-visible absolute `image_path` or an existing Core image `artifact_id` (choose one).
|
|
39
41
|
- Use `iris_relook_attachment` for an image attachment already present in the session or produced by Iris.
|
|
40
|
-
- Use `iris_long_ocr` when exact text from an image matters more than a visual description.
|
|
41
|
-
- Video and audio
|
|
42
|
-
- Keep every returned attachment ID, task ID, and output path that a later step needs.
|
|
42
|
+
- Use `iris_long_ocr` when exact text from an image matters more than a visual description; choose exactly one of `image_path`, Core `artifact_id`, or session `attachment_id`. `iris_locate` accepts the same three sources for original-pixel bounding boxes.
|
|
43
|
+
- Video and audio file inputs require host-visible absolute `video_path` or `audio_path` values. Video summaries can instead consume existing Core `frame_artifact_ids`. A browser-local path, `content://` URI, or ordinary web URL is not a host path; request an upload or exported host path.
|
|
44
|
+
- Keep every returned Core Artifact ID, attachment ID, task ID, and output path that a later step needs. Core `artifact_id` and session `attachment_id` are distinct; do not use either as a file path or invent identifiers.
|
|
43
45
|
|
|
44
46
|
An attachment returned by `iris_draw_image` can be passed as `first_frame_attachment_id` to `iris_generate_video`. An existing host image uses `first_frame_path`. The local path returned by `iris_speak_text` can be passed as `audio_path` for S2V.
|
|
45
47
|
|
|
@@ -48,6 +50,7 @@ An attachment returned by `iris_draw_image` can be passed as `first_frame_attach
|
|
|
48
50
|
Choose the minimum chain, then read only the matching section of [references/workflows.md](references/workflows.md):
|
|
49
51
|
|
|
50
52
|
- inspect an image, then draw: `iris_look_at_image` or `iris_relook_attachment` → `iris_draw_image`;
|
|
53
|
+
- edit a Core image, then inspect: `iris_edit_image` → `iris_look_at_image` using the new Artifact ID;
|
|
51
54
|
- generate/reuse a still, then animate: optional `iris_draw_image` → `iris_generate_video`;
|
|
52
55
|
- summarize a video, then narrate: `iris_media_summarize` → `iris_speak_text`;
|
|
53
56
|
- create an S2V talking video: resolve a first frame and audio → `iris_generate_video`;
|
|
@@ -55,6 +58,8 @@ Choose the minimum chain, then read only the matching section of [references/wor
|
|
|
55
58
|
|
|
56
59
|
Use `iris_video_frames` only for explicit frame extraction or a custom frame-level workflow. Do not load every workflow section when one route is sufficient.
|
|
57
60
|
|
|
61
|
+
If frames are already saved by `iris_video_frames`, pass its returned `frame_artifact_ids` to `iris_media_summarize` instead of extracting again. This reads the original timestamps and frame order without the video file or ffmpeg. Omit `video_path`, `max_frames`, `target_width`, and `transcribe:true`; supply existing speech as `transcribe_text` when available.
|
|
62
|
+
|
|
58
63
|
## Control cost and iteration
|
|
59
64
|
|
|
60
65
|
- Create one new Iris generation **Task** per requested artifact by default. A Provider Attempt is an internal candidate submission inside that Task; do not count failover attempts as user-authorized new Tasks.
|
|
@@ -72,7 +77,7 @@ When a generation tool returns a Task, or the user asks to retry, recover, cance
|
|
|
72
77
|
- Observation failure resumes observation; delivery failure resumes delivery. Neither authorizes regeneration. If that recovery action is available only in the Iris workbench, say so and preserve the Task ID.
|
|
73
78
|
- If the user cancels, stop the chain and do not launch downstream generation. Cancellation requested or unknown is not proof that no provider work occurred.
|
|
74
79
|
- Preserve partial outputs. Report which step succeeded, which failed or became uncertain, and whether accepted work may remain active.
|
|
75
|
-
- If an attachment is missing, request a re-upload or host path. If `ffmpeg`/`ffprobe` is unavailable,
|
|
80
|
+
- If an attachment is missing, request a re-upload or host path. If `ffmpeg`/`ffprobe` is unavailable, new video frame extraction cannot run; summaries can still use existing Core frame IDs.
|
|
76
81
|
- If vision is unavailable but the creation prompt is complete, skip source analysis only when the requested result still remains valid.
|
|
77
82
|
|
|
78
83
|
## Report the result
|
|
@@ -9,6 +9,12 @@ Read only the section matching the chosen workflow. Keep paid steps bounded by t
|
|
|
9
9
|
3. Call `iris_draw_image` once for each requested artifact.
|
|
10
10
|
4. Inspect the result only when verification or iteration was requested. A mismatch is evidence to report, not automatic permission to regenerate.
|
|
11
11
|
|
|
12
|
+
## Edit a Core image, then inspect
|
|
13
|
+
|
|
14
|
+
1. Resolve the source Core Artifact ID; one static PNG, JPEG or WebP image up to 20 MiB is supported.
|
|
15
|
+
2. Call `iris_edit_image` with `source_artifact_id` and a focused modification prompt, optionally an explicit model. The original stays intact and the edited Artifact records its source.
|
|
16
|
+
3. Inspect the new Artifact ID only when review is requested. Preserve the Task ID if acceptance or delivery is uncertain; do not repeat the edit automatically.
|
|
17
|
+
|
|
12
18
|
## Generate or reuse a still, then animate
|
|
13
19
|
|
|
14
20
|
1. Create a still with `iris_draw_image`, reuse an Iris-produced first-frame attachment, or obtain a host-visible path. Do not pass an ordinary session attachment as `first_frame_attachment_id`.
|
|
@@ -18,7 +24,7 @@ Read only the section matching the chosen workflow. Keep paid steps bounded by t
|
|
|
18
24
|
|
|
19
25
|
## Summarize a video, then narrate
|
|
20
26
|
|
|
21
|
-
1. Call `iris_media_summarize` with `transcribe=true` unless the user wants visual-only analysis.
|
|
27
|
+
1. Call `iris_media_summarize` with `video_path` and `transcribe=true` unless the user wants visual-only analysis. If Core frames already exist, use their `frame_artifact_ids` instead and optionally provide `transcribe_text`; omit `transcribe=true` and sampling options for existing frames.
|
|
22
28
|
2. Do not also call `iris_video_frames` or `iris_transcribe_audio` by default; the summary action already samples frames and can transcribe audio.
|
|
23
29
|
3. Write narration for the requested audience and length without presenting inference as visible fact.
|
|
24
30
|
4. Call `iris_speak_text`; return the summary, contact-sheet attachment, and saved audio path.
|
|
@@ -22,6 +22,7 @@ Ask one concise question only when a missing reference, target state, theme, or
|
|
|
22
22
|
## Prepare the inputs
|
|
23
23
|
|
|
24
24
|
1. Identify each image source:
|
|
25
|
+
- Existing Core image: pass `artifact_id` to look/locate/OCR. Crop and pixel diff still need a supported path or attachment; export the Artifact explicitly if those steps require it.
|
|
25
26
|
- Host absolute path: pass `image_path`.
|
|
26
27
|
- Current-session or Iris-produced attachment: pass `attachment_id`.
|
|
27
28
|
- Browser-local path, `content://` URI, or ordinary web URL: do not assume the host can read it. Ask for an upload or a host-accessible file.
|
|
@@ -45,7 +46,7 @@ When raw HTML needs capture, call `iris_html_screenshot` with the same HTML wrap
|
|
|
45
46
|
|
|
46
47
|
### 2. Inspect semantics
|
|
47
48
|
|
|
48
|
-
Inspect the current and reference images separately. Call `iris_look_at_image` for a host path and `iris_relook_attachment` for an attachment.
|
|
49
|
+
Inspect the current and reference images separately. Call `iris_look_at_image` for a Core Artifact ID or host path and `iris_relook_attachment` for an attachment.
|
|
49
50
|
|
|
50
51
|
Each vision call sees only one image. Ask the same structured questions for both images:
|
|
51
52
|
|
package/CHANGELOG.md
CHANGED
|
@@ -4,7 +4,51 @@
|
|
|
4
4
|
|
|
5
5
|
## [Unreleased]
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
后续未归入版本的变更在此记录。
|
|
8
|
+
|
|
9
|
+
## [0.2.1] - 2026-10-09
|
|
10
|
+
|
|
11
|
+
0.2.1 的更新重点是 **完成 OpenAI 兼容图片与视觉工作流**:生成图片 → Core Artifact → 按 ID 看图/OCR → 聊天改图 → 新 Artifact,CLI 与 DSH 工作台共用同一链路。模型发现与能力分类、模型级图片协议、视觉输入和作品管理共同支持这套流程。要求仍为 Node.js ≥ 22.0.0,DSH 插件仅支持 `0.2.0-rc.2`;包名、配置版本和生产依赖保持不变。完整范围见 [0.2.1 发布说明](docs/releases/0.2.1.md)。
|
|
12
|
+
|
|
13
|
+
### Added
|
|
14
|
+
|
|
15
|
+
- 生图模型可独立设置 `imageProtocol`,分别选择 DashScope、OpenAI Images、聊天生图或 Responses 生图;省略或 `auto` 时继承账号默认。同账号的模型可使用不同接口,生成、实测和恢复按实际协议绑定,不扫描接口或自动探测。
|
|
16
|
+
- 聊天生图兼容非流式 Chat Completions 图片消息;Responses 支持同步单图、已完成的 `image_generation_call` 及网关图片消息。生成结果按真实字节保存为 PNG/JPEG/WebP Core Artifact,MIME、后缀和内容哈希一致。纯文字 200、缺图或未完成响应不会宣称成功或自动重提。
|
|
17
|
+
- Artifact ID 聊天改图:DSH `iris_edit_image`、Core 图片卡“改图”与 CLI `run image` 的 `source_artifact_id` 共用 `openai-chat-images`。修改版另存为新 Artifact,记录 `derived-from` 及来源引用,原图保留;编辑池只选择聊天生图模型。详见 [Artifact 聊天改图](docs/CLI_MANAGEMENT.md#artifact-聊天改图)。
|
|
18
|
+
- 独立视觉 CLI:`vision look / locate / ocr / summarize`,支持严格显式模型选择、JSON/文本、结果文件和摘要拼图输出;DSH 与 CLI 复用共享视觉业务。纯视觉不创建 Task/Artifact,主动音轨转写使用 Core。
|
|
19
|
+
- 看图、定位和 OCR 支持 `artifact_id`;视频摘要支持 `frame_artifact_ids`。只读核验 Core 原字节、MIME 和哈希,不要求原文件或临时导出;已有视频帧无需再次抽帧或 ffmpeg。
|
|
20
|
+
- 账号与模型可配置 `visionInput.maxBytes/maxDimension`,模型逐字段覆盖账号,默认发送预算 8 MiB。工作台和 CLI `models vision-input` 可设置或恢复继承;看图、OCR 分块、定位和聊天改图按实际候选准备发送副本,Core 原图、下载与哈希不变。
|
|
21
|
+
- 工作台统一 Core/旧版图片、视频、音频和文本,每页 24 项,支持媒体、来源和产物类型过滤、真实匹配计数。Core 图片卡直接看图/OCR/改图,识别结果可取消、复制和下载,并明确 OCR 部分完成。
|
|
22
|
+
- 作品详情、显式跨页选择、ID 复制与混合来源 ZIP 下载,单次最多 200 项、下载上限 128 MiB。Core 删除先预览,再确认移入可恢复回收区;恢复不覆盖冲突文件。详见 [工作台作品管理](docs/WORKBENCH_ARTIFACTS.md)。
|
|
23
|
+
- CLI 补齐 S2V 数字人视频、HTML 截图、账号/模型/能力分配管理、模型发现与显式实测、有界 `task wait`、分页过滤、批量查询/导出,以及 JSON 文件和 stdin 输入。
|
|
24
|
+
- CLI `core delete/cleanup/transactions/restore` 提供默认预览、引用和活跃任务保护、明确选择后的隔离、哈希核对恢复及失败补偿;不提供永久 purge。
|
|
25
|
+
- 对话泡泡增加本次改写规则、原样前缀/后缀、“只组装”、可编辑结果和原文/改动对照。规则按次传入,v1 配置保持不变;未扩展后续独立提示词优化系统。
|
|
26
|
+
|
|
27
|
+
### Changed
|
|
28
|
+
|
|
29
|
+
- 提示词优化、单图看图/重看、OCR、定位、视频拼图摘要及生成后自述迁入共享 Text/Vision Model Ports。准备与候选共用整体预算,严格检查正常终态;取消、超时、截断或协议错误停止后续生成,超长正文明确失败。
|
|
30
|
+
- OCR 整次操作最多 32 块、64 次生成、120 秒;先按转正后的长图切片,再按候选预算处理每块,记录实际发送尺寸与字节数,缩小时提示细字风险,区分完成、部分完成和失败。
|
|
31
|
+
- 定位显式要求模型返回 0–1000 归一化坐标,依据实际发送尺寸和 EXIF 方向换算回原图像素,向外取整供裁剪;接受经核验的 DSH 附件规范化,不猜测返回坐标域。
|
|
32
|
+
- 模型路由区分短时限流和明确耗尽。短时 429 按 `Retry-After`(默认 60 秒)冷却,以琥珀色显示;到期只恢复候选资格,不自动验证。明确免费额度或预算耗尽以红色持续停用至手动实测成功,重启后仍跳过。DSH、CLI、OCR 共用账号 × 模型规则。
|
|
33
|
+
- S2V 首帧与音频迁入共享 Core Attempt 上传链,各候选使用自己的账号;HTML 截图经独立 Chromium Browser Port 或 DSH 宿主保存 Core PNG。配置写入使用私有备份、原子替换和竞争写保护。
|
|
34
|
+
- 修改模型图片协议只使对应生图验证失效,不解除限流或耗尽停用;发现保留手工能力、协议覆盖和输入预算。原图引用参与 Core 删除与清理保护。
|
|
35
|
+
|
|
36
|
+
### Fixed
|
|
37
|
+
|
|
38
|
+
- 修复 4K 生图响应的 base64 校验触发调用栈溢出,保留原字节交付与字符、padding 校验。
|
|
39
|
+
- 修复视觉探针图片过小造成的能力误判,改用 128×128 红色 PNG。
|
|
40
|
+
- 修复 OCR 显式 `overlap: 0` 被入口默认值覆盖,以及失败块导致后续原段号丢失。
|
|
41
|
+
- 修复视频摘要时间戳标签未绘制、十帧以上按文件名字典序排列造成的时序错乱;交付与模型输入使用同一张拼图。
|
|
42
|
+
- 保护提示词写回时的新草稿、忙碌状态和结构化引用,旧请求迟到结果不覆盖新操作;不支持的显式 reasoning effort 在生成前拒绝。
|
|
43
|
+
- 修正跨平台 CLI 测试的文件 URL、路径、权限与信号断言;缺少可选 ffmpeg/ffprobe 时只跳过相应视频段。
|
|
44
|
+
|
|
45
|
+
### 已知限制
|
|
46
|
+
- 浏览器/移动端完整作品管理,以及真实异步任务重启后的 CLI 接管尚未完成验收;已有本地测试和 rc.2 服务隔离检查不替代这些验证。
|
|
47
|
+
- Responses 当前仅同步单图;聊天改图不支持 mask、多参考图或连续编辑历史。独立 Images 编辑与原生 Gemini/Fal 协议未在本版接入。
|
|
48
|
+
|
|
49
|
+
### 后续方向
|
|
50
|
+
|
|
51
|
+
- 继续适配其他协议,并规划 Music 音乐生成功能;均属于后续工作,不纳入 0.2.1。详见 [路线图](docs/ROADMAP.md#后续功能方向)。
|
|
8
52
|
|
|
9
53
|
## [0.2.0] - 2026-10-01
|
|
10
54
|
|
|
@@ -223,7 +267,8 @@ Iris 0.2.0 要求 Node.js ≥ 22.0.0,DSH 插件仅支持 `0.2.0-rc.2`;旧宿
|
|
|
223
267
|
- 上传采用流式限额、临时 `.part` 文件和原子落盘,失败时会清理未完成文件。
|
|
224
268
|
- 修改状态的路由拒绝明确的跨站请求,媒体文件继续使用随机能力令牌访问。
|
|
225
269
|
|
|
226
|
-
[Unreleased]: https://github.com/mokuyoaxis/dsh-iris/compare/v0.2.
|
|
270
|
+
[Unreleased]: https://github.com/mokuyoaxis/dsh-iris/compare/v0.2.1...HEAD
|
|
271
|
+
[0.2.1]: https://github.com/mokuyoaxis/dsh-iris/compare/v0.2.0...v0.2.1
|
|
227
272
|
[0.2.0]: https://github.com/mokuyoaxis/dsh-iris/compare/v0.1.4...v0.2.0
|
|
228
273
|
[0.1.4]: https://github.com/mokuyoaxis/dsh-iris/compare/v0.1.3...v0.1.4
|
|
229
274
|
[0.1.3]: https://github.com/mokuyoaxis/dsh-iris/compare/v0.1.2...v0.1.3
|
package/README.en.md
CHANGED
|
@@ -8,7 +8,7 @@
|
|
|
8
8
|
|
|
9
9
|
<h1 align="center">Iris Media for DSH</h1>
|
|
10
10
|
|
|
11
|
-
<p align="center"><strong>A multimodal media production runtime ·
|
|
11
|
+
<p align="center"><strong>A multimodal media production runtime · DSH plugin and standalone CLI</strong></p>
|
|
12
12
|
|
|
13
13
|
<p align="center">
|
|
14
14
|
<a href="#deepseek-harness-adaptation"><img alt="DeepSeek Harness compatible" src="https://img.shields.io/badge/DeepSeek%20Harness-compatible-4D6BFE.svg?style=flat-square"></a>
|
|
@@ -18,7 +18,7 @@
|
|
|
18
18
|
<a href="LICENSE"><img alt="MIT License" src="https://img.shields.io/badge/license-MIT-1689FF.svg?style=flat-square"></a>
|
|
19
19
|
</p>
|
|
20
20
|
|
|
21
|
-
dsh-iris gives agents
|
|
21
|
+
dsh-iris gives agents, the Iris workbench and a standalone CLI image, video, speech and visual understanding capabilities. Each image model can use DashScope, OpenAI Images, chat image generation or Responses. Shared Core Artifacts connect generation, vision, OCR and chat editing. The DSH workbench provides configuration and task management, while the 🫧 bubble offers draft optimization in the composer.
|
|
22
22
|
|
|
23
23
|
## A real generation example
|
|
24
24
|
|
|
@@ -44,18 +44,34 @@ These captures come from an Android browser connected to DSH under Termux/PRoot
|
|
|
44
44
|
|
|
45
45
|
The project is at an early stage; interfaces and configuration formats may still change between releases.
|
|
46
46
|
|
|
47
|
-
Iris
|
|
47
|
+
Iris 0.2.1 provides a DSH plugin and a headless CLI sharing tasks, artifacts and model routing, completing the OpenAI-compatible image and vision workflow. Core internals have not been frozen as a public SDK.
|
|
48
48
|
|
|
49
|
-
> Version support: Iris `0.2.
|
|
49
|
+
> Version support: Iris `0.2.x` requires Node.js `>=22.0.0` and its DSH plugin supports only the exact host version `0.2.0-rc.2`. Older DSH hosts should pin `@mokuyoaxis/dsh-iris@0.1.4` as described below instead of following npm latest. Iris and DSH have independent version numbers.
|
|
50
|
+
|
|
51
|
+
## What's new in 0.2.1
|
|
52
|
+
|
|
53
|
+
**Complete the OpenAI-compatible image and vision workflow:** image generation → Core Artifact → vision/OCR by ID → chat editing → new Artifact, shared by the CLI and DSH workbench.
|
|
54
|
+
|
|
55
|
+
- **Models and protocols:** discover models and classify capabilities, then choose image protocols per model; models within one account can use Images, chat or Responses interfaces.
|
|
56
|
+
- **Image delivery:** preserve PNG/JPEG/WebP bytes and handle large base64 image responses.
|
|
57
|
+
- **Artifact editing:** edit from a Core image card, `iris_edit_image`, or the CLI; save a linked new image while preserving the source.
|
|
58
|
+
- **Standalone vision:** look, locate, OCR and video summaries with explicit model selection and direct image/frame Artifact inputs.
|
|
59
|
+
- **Input budgets:** configure account/model sending limits, preserve originals, report OCR resizing and map locate coordinates to source pixels.
|
|
60
|
+
- **Library and CLI management:** pagination, filters, selection across pages, ZIP downloads and recoverable Core deletion; S2V, HTML screenshots, configuration and bounded waiting.
|
|
61
|
+
- **Model recovery:** amber for temporary 429 cooldown; red for confirmed quota or budget exhaustion until a manual test succeeds.
|
|
62
|
+
|
|
63
|
+
See the [0.2.1 release notes](docs/releases/0.2.1.md) and [CHANGELOG](CHANGELOG.md) for scope and pending checks.
|
|
64
|
+
|
|
65
|
+
Future work includes additional protocol adapters and **music generation**; see the [roadmap](docs/ROADMAP.md#后续功能方向).
|
|
50
66
|
|
|
51
67
|
## Two ways to use Iris
|
|
52
68
|
|
|
53
|
-
| | Path A: DSH plugin
|
|
69
|
+
| | Path A: DSH plugin | Path B: Headless CLI |
|
|
54
70
|
|---|---|---|
|
|
55
71
|
| Who it's for | DeepSeek Harness users who want agent tools, the workbench, and in-chat prompt optimization | Users who want media generation, task tracking, and artifact export from the command line without DSH |
|
|
56
72
|
| Entry point | Choose a pinned release under "Quickest start" below | `dsh-iris run ...` — see [Headless CLI](docs/HEADLESS_CLI.md) |
|
|
57
73
|
| Data location | `$DSH_HOME/iris/v1/` (Core data root `core-v0`) | Any absolute path passed via `--data-root` |
|
|
58
|
-
| Automatic observation | Yes (bounded DSH host ticks + restart takeover) |
|
|
74
|
+
| Automatic observation | Yes (bounded DSH host ticks + restart takeover) | Explicit `task observe`; 0.2.1 also offers bounded `task wait` |
|
|
59
75
|
| Dependencies | DSH `0.2.0-rc.2` and Node.js ≥ 22 | Node.js ≥ 22 and `sharp` only (`ffmpeg` for video frames) |
|
|
60
76
|
|
|
61
77
|
Both paths read and write the same Core Task/Artifact facts: point the CLI at the `core-v0` data root of a DSH profile to inspect/export DSH-generated artifacts, and vice versa. Only one writer is allowed per data root.
|
|
@@ -74,7 +90,7 @@ Check the host with `dsh --version` and ensure `pnpm` is on your PATH. Iris and
|
|
|
74
90
|
|
|
75
91
|
| Your DSH | Iris choice | Source |
|
|
76
92
|
|---|---|---|
|
|
77
|
-
| `0.2.0-rc.2` | `0.2.
|
|
93
|
+
| `0.2.0-rc.2` | `0.2.1` | Pinned npm release |
|
|
78
94
|
| `>=0.1.2-rc.1 <0.1.3-0` or `0.1.5-rc.1` | Stable `0.1.4` | Pinned npm release |
|
|
79
95
|
| Other versions | No current support claim | Check host adaptation before installing |
|
|
80
96
|
|
|
@@ -95,24 +111,33 @@ npm pack @mokuyoaxis/dsh-iris@0.1.4
|
|
|
95
111
|
|
|
96
112
|
This writes a `.tgz` to the current directory. Use `npm install @mokuyoaxis/dsh-iris@0.1.4` if you only want a regular npm dependency; that does not add a DSH profile layer. Pinning `@0.1.4` keeps selecting the old stable release after latest changes. See [npm pack](https://docs.npmjs.com/cli/v11/commands/npm-pack/). Version 0.1.4 does not contain the 0.2.0 Core media CLI or the later DSH rc.2 adaptation.
|
|
97
113
|
|
|
98
|
-
### DSH 0.2.0-rc.2: install 0.2.
|
|
114
|
+
### DSH 0.2.0-rc.2: install 0.2.1
|
|
99
115
|
|
|
100
|
-
Install the pinned
|
|
116
|
+
Install the pinned release into your Web profile:
|
|
101
117
|
|
|
102
118
|
```bash
|
|
103
|
-
dsh plugin --profile web add @mokuyoaxis/dsh-iris@0.2.
|
|
119
|
+
dsh plugin --profile web add @mokuyoaxis/dsh-iris@0.2.1
|
|
104
120
|
dsh web
|
|
105
121
|
```
|
|
106
122
|
|
|
107
|
-
The plugin loads the next
|
|
123
|
+
The plugin loads on the next DSH start. The web UI defaults to `http://127.0.0.1:3080`; open the complete URL if startup output includes authentication parameters. Reload the backend and refresh the web client after updating.
|
|
124
|
+
|
|
125
|
+
### Standalone CLI: install 0.2.1
|
|
126
|
+
|
|
127
|
+
Install in your own project without DSH:
|
|
128
|
+
|
|
129
|
+
```bash
|
|
130
|
+
npm install @mokuyoaxis/dsh-iris@0.2.1
|
|
131
|
+
npx dsh-iris --help
|
|
132
|
+
```
|
|
108
133
|
|
|
109
134
|
### Choosing a version in dsh-market
|
|
110
135
|
|
|
111
|
-
The marketplace entry should resolve to the full package name `@mokuyoaxis/dsh-iris`. Check both its displayed Iris version and your local `dsh --version` before installing: 0.2.
|
|
136
|
+
The marketplace entry should resolve to the full package name `@mokuyoaxis/dsh-iris`. Check both its displayed Iris version and your local `dsh --version` before installing: 0.2.1 is for DSH rc.2; older hosts should use the pinned `@0.1.4` commands above. Catalog and npm information may be cached. The actual npm package's `dsh.engines.dsh` and Node engines determine its declared requirements; a visible listing does not imply support for every DSH release.
|
|
112
137
|
|
|
113
138
|
> The unscoped `dsh-iris` on npm belongs to a different plugin. Always keep the fully scoped name `@mokuyoaxis/dsh-iris` when installing or updating Iris Media.
|
|
114
139
|
|
|
115
|
-
After the first start, open "Settings → Iris workbench → + Add provider", enter your
|
|
140
|
+
After the first start, open "Settings → Iris workbench → + Add provider", enter your provider base URL and API key, then click "Discover models". A single provider can start with automatic capability assignment. Full steps, image protocol selection, and failover notes are in the [user guide](user_guide.md) (Chinese).
|
|
116
141
|
|
|
117
142
|
Once configured, minimal instructions can be as simple as:
|
|
118
143
|
|
|
@@ -142,26 +167,27 @@ For a copyable installation and usage prompt, see [Agent quickstart prompt](docs
|
|
|
142
167
|
|
|
143
168
|
## Prerequisites
|
|
144
169
|
|
|
145
|
-
- A Node.js version that satisfies your DeepSeek Harness release's requirement; Iris 0.2.
|
|
170
|
+
- A Node.js version that satisfies your DeepSeek Harness release's requirement; Iris 0.2.1 requires at least 22.0.0, while the published npm 0.1.4 historically requires 20.10.0
|
|
146
171
|
- A working DeepSeek Harness environment with `pnpm` on the PATH
|
|
147
172
|
- At least one supported media or vision provider
|
|
148
|
-
- `ffmpeg` and `ffprobe`
|
|
173
|
+
- `ffmpeg` and `ffprobe` for extracting frames or summarizing a video file; summaries of existing Core frames need neither
|
|
149
174
|
|
|
150
175
|
After installing the plugin, add a provider in the Iris workbench and assign models to the capabilities you need. Provider keys are kept in Iris's host-side configuration; on POSIX systems, Iris's own directories and files are tightened to `0700` and `0600` respectively, and neither the UI nor the API ever returns full keys. On Windows, file modes are not equivalent to ACLs — keep relying on your user directory and system account permissions.
|
|
151
176
|
|
|
152
177
|
## DeepSeek Harness adaptation
|
|
153
178
|
|
|
154
|
-
dsh-iris remains a native DSH plugin.
|
|
179
|
+
dsh-iris remains a native DSH plugin. Iris 0.2.1 server registers 15 agent tools, including image editing, and uses host-provided routing, attachment, model, and lifecycle services. The web client contributes settings, composer, and global-overlay UI. Iris does not start another service or listen on an extra port.
|
|
155
180
|
|
|
156
181
|
| Iris version | DSH range | Verification |
|
|
157
182
|
|---|---|---|
|
|
158
183
|
| Published stable `0.1.4` | `>=0.1.2-rc.1 <0.1.3-0` | Historical clean and daily Web profiles on Linux ARM64 |
|
|
159
184
|
| Published stable `0.1.4` | `0.1.5-rc.1` | Historical daily Android/Linux profile checks |
|
|
160
185
|
| `0.2.0` | Only `0.2.0-rc.2` | Installed-service isolation checks, Core diff, and SlotRegistry; user demonstration reported successful, with detailed browser actions and asynchronous restart/rollback verification continuing |
|
|
186
|
+
| `0.2.1` | Only `0.2.0-rc.2` | Installed-service checks cover 15 tools and 31 boundaries; complete browser/mobile and real asynchronous restart acceptance remain pending |
|
|
161
187
|
|
|
162
188
|
See the [rc.2 adaptation notes](docs/DSH_RC2_ADAPTATION.md) for scope. The published npm 0.1.4 package does not contain these later changes.
|
|
163
189
|
|
|
164
|
-
Iris 0.2.
|
|
190
|
+
Iris 0.2.x no longer claims older DSH support; historical validation does not automatically cover this version. DSH `0.2.0` stable and other rc versions are also outside its support range. Automated checks cover server loading, tool and Skill registration, routes, the client bundle, and UI slots separately so Doctor can identify which boundary failed.
|
|
165
191
|
|
|
166
192
|
Core and the DSH Adapter are being separated: DSH keeps loading the same npm package, while the Adapter maps host capabilities into Core. A DSH API change should require an Adapter or client-bridge fix, not a rewrite of Task or Artifact semantics. Version 0.2.0 includes the headless CLI. The `dsh` prefix will be removed after a second real Host Adapter is implemented and accepted, with a new npm package at that point; see the [roadmap](docs/ROADMAP.md).
|
|
167
193
|
|
|
@@ -172,6 +198,7 @@ Core and the DSH Adapter are being separated: DSH keeps loading the same npm pac
|
|
|
172
198
|
| Tool | Purpose |
|
|
173
199
|
|---|---|
|
|
174
200
|
| `iris_draw_image` | Generate images from a prompt |
|
|
201
|
+
| `iris_edit_image` | Edit a source Artifact into a new image with a chat image model, preserving provenance |
|
|
175
202
|
| `iris_generate_video` | Text-to-video, image-to-video, or S2V digital-human video |
|
|
176
203
|
| `iris_speak_text` | Synthesize speech from text |
|
|
177
204
|
| `iris_transcribe_audio` | Transcribe audio to text |
|
|
@@ -198,6 +225,12 @@ Core and the DSH Adapter are being separated: DSH keeps loading the same npm pac
|
|
|
198
225
|
|
|
199
226
|
## File inputs
|
|
200
227
|
|
|
228
|
+
Look, locate and OCR accept a local path, session attachment or Core image `artifact_id`; summaries can reuse `frame_artifact_ids`. Artifact inputs are read-only and checked against their MIME type and content hash. PNG/JPEG/WebP inputs need no manual JPEG conversion. See [vision CLI](docs/VISION_CLI.md).
|
|
229
|
+
|
|
230
|
+
Look, OCR chunks, locate and chat editing prepare sending copies using account/model budgets, with an 8 MiB default. Core originals, downloaded dimensions and hashes stay intact. OCR reports resizing and small-text risk; locate returns source-pixel coordinates. See [input budgets](docs/CLI_MANAGEMENT.md#看图输入预算).
|
|
231
|
+
|
|
232
|
+
Chat editing uses a source Artifact ID and saves a linked new image; it requires a model configured for chat image generation. See [Artifact editing](docs/CLI_MANAGEMENT.md#artifact-聊天改图) and [workbench management](docs/WORKBENCH_ARTIFACTS.md) for filtering, selection, downloads and recovery.
|
|
233
|
+
|
|
201
234
|
Iris accepts three file sources, in the recommended order:
|
|
202
235
|
|
|
203
236
|
1. **Browser upload** — best for local files and cross-environment access; 64 MB per file.
|
|
@@ -208,6 +241,8 @@ Uploads are stored under `$DSH_HOME/iris/v1/uploads/` and kept for 7 days by def
|
|
|
208
241
|
|
|
209
242
|
## Conversation prompt optimization
|
|
210
243
|
|
|
244
|
+
The 0.2.1 adds per-operation rules, assembly without a model call and editable comparisons. Rules are not persisted and no standalone prompt CLI is added. See the [prompt optimizer guide](docs/PROMPT_OPTIMIZER.md).
|
|
245
|
+
|
|
211
246
|
Iris adds a borderless, text-free “🫧” control directly to the DSH composer, so the workbench does not need to be open. It opens a translucent glass panel with background blur and soft depth; on narrow screens and Android browsers it becomes a safe-area-aware, internally scrollable bottom sheet to prevent overlap and overflow. It reads only the current unsent plain-text draft and offers general, image, video, and start-to-end-frame video targets. The result is previewed first and is written back only after confirmation; Iris never sends it automatically. Drafts containing structured `@` or `/` references are left untouched for now so their identities are not lost.
|
|
212
247
|
|
|
213
248
|
By default, the optimizer uses the model selected for the current session and falls back to DSH's default model when the session has none. The request carries no chat history, tools, attachments, or workspace content—only the current draft and target template. To prevent reasoning tokens from exhausting the generation budget before visible text is complete, `generation.reasoningEffort` defaults to `off-if-supported`: reasoning is disabled only when DSH model metadata explicitly advertises such an effort; otherwise the provider default is used, and the session High/Low setting is not inherited. JSON may select `provider-default`, `inherit`, or an exact effort ID advertised by the model. For predictable cost and latency, use a `fixed` route with a lightweight non-reasoning model. You can also export `prompt-optimizer.json`, change `route.mode` to `fixed`, name another `provider` and `model` already registered in DSH, and import it again. “Reset to defaults” restores Iris's built-in prompt, target templates, session-model routing, and generation settings. The composer control can be disabled independently and re-enabled from the Iris workbench; disabling it does not stop the workbench, agent tools, or task runtime. An optimization call may incur charges from the selected text model.
|
|
@@ -226,9 +261,9 @@ Transcription is a separate `transcribe` capability and does not consume the TTS
|
|
|
226
261
|
|
|
227
262
|
## Capability health and model verification
|
|
228
263
|
|
|
229
|
-
The workbench and main bubble
|
|
264
|
+
The workbench and main bubble show gray for unconfigured, blue for unverified, green for success within seven days, and muted red for authentication or permission failures. The 0.2.1 also shows amber for temporary 429 cooldown and bright red for confirmed free quota or budget exhaustion, with a reason and recovery condition.
|
|
230
265
|
|
|
231
|
-
Iris performs no background probes.
|
|
266
|
+
Iris performs no background probes. Temporary 429 responses use `Retry-After`, or a 60-second default cooldown. Expiry restores candidate eligibility without a verification request. Confirmed free quota or budget exhaustion persists across restarts until an explicit model test succeeds; test buttons remain available. Vision, image, and TTS have built-in test samples; video and transcription tests require real input through the CLI. Network errors, 5xx responses, content-safety failures, cancellation, and unknown acceptance do not overwrite recent success evidence. Host Doctor only checks loading and host ports. See [Provider and capability health](docs/PROVIDER_HEALTH.md).
|
|
232
267
|
|
|
233
268
|
## Offline diagnostics
|
|
234
269
|
|
|
@@ -241,7 +276,7 @@ npx @mokuyoaxis/dsh-iris doctor --json
|
|
|
241
276
|
|
|
242
277
|
The installed binary is `dsh-iris`. Exit code `0` is healthy, `1` means warnings, and `2` means hard errors. Offline Doctor checks Node.js, sharp, ffmpeg/ffprobe, a real but cleaned-up storage write probe, configuration, models/assignments, task semantics, temporary files, and orphaned or missing artifacts.
|
|
243
278
|
|
|
244
|
-
A running DSH instance also provides a “Host diagnostics” card in the Iris workbench and exposes the authenticated `/iris/api/doctor` JSON. It checks the DSH version, plugin,
|
|
279
|
+
A running DSH instance also provides a “Host diagnostics” card in the Iris workbench and exposes the authenticated `/iris/api/doctor` JSON. It checks the DSH version, plugin, current tools, two Skills, four route groups, Browser/attachment/model capabilities, client version, and four UI Slots. It reads only safe snapshots and registration evidence; it never calls Browser, models, or providers. See [Host Doctor](docs/HOST_DOCTOR.md).
|
|
245
280
|
|
|
246
281
|
## Composed workflow example
|
|
247
282
|
|
|
@@ -259,18 +294,19 @@ Runtime data lives under `$DSH_HOME/iris/v1/` by default:
|
|
|
259
294
|
|---|---|
|
|
260
295
|
| `providers.json` | Providers, capability assignments, and redacted health evidence; file mode 0600 |
|
|
261
296
|
| `prompt-optimizer.json` | Imported optimizer prompt, target templates, model route, and generation settings; file mode 0600 |
|
|
262
|
-
| `
|
|
263
|
-
| `
|
|
264
|
-
| `
|
|
297
|
+
| `core-v0/` | New tasks, attempts, Artifact objects, manifests and recoverable quarantine transactions |
|
|
298
|
+
| `tasks.json` | Legacy task metadata and attachment indexes; up to 500 records |
|
|
299
|
+
| `artifacts.json` | Legacy work-library index and random access tokens; no prompt, provider, or task relationship |
|
|
300
|
+
| `outputs/` | Legacy generated and processed media |
|
|
265
301
|
| `uploads/` | Temporary copies of browser uploads |
|
|
266
302
|
|
|
267
303
|
On first load, 0.1.1 tightens POSIX permissions across an existing `$DSH_HOME/iris/v1/` tree without modifying file contents or following symlinks. Large media downloads stream to a private temporary file and are moved into place atomically, instead of loading a whole video into memory.
|
|
268
304
|
|
|
269
305
|
Asynchronous tasks are polled in the background. After a plugin restart, Iris can re-adopt tasks that are still running remotely. Cancellation signals propagate through local waits and polling; whether the remote computation itself can be cancelled depends on the provider's API.
|
|
270
306
|
|
|
271
|
-
The work library has its own lifecycle: clearing terminal task history does not delete works, and older media can be recovered by re-indexing `outputs/`.
|
|
307
|
+
The work library has its own lifecycle: clearing terminal task history does not delete works, and older media can be recovered by re-indexing `outputs/`. Legacy deletion removes actual files with separate confirmation; see [Work library v0](docs/ARTIFACT_LIBRARY.md). Core deletion uses preview and recoverable quarantine; see [workbench management](docs/WORKBENCH_ARTIFACTS.md). Favorites, tags and search remain future work.
|
|
272
308
|
|
|
273
|
-
|
|
309
|
+
Legacy media links carry random capability tokens. Core media uses random Artifact IDs and host/browser origin guards. Images are, wherever possible, also saved as DSH durable attachments so they stay usable in the conversation.
|
|
274
310
|
|
|
275
311
|
## Security notes
|
|
276
312
|
|
|
@@ -280,13 +316,13 @@ Audio and video are served through Iris media links that carry random capability
|
|
|
280
316
|
- `IRIS_TRUSTED_HOSTS` is not an authentication mechanism. When exposing DSH to the public internet or an untrusted network, configure authentication and HTTPS at the reverse proxy or host layer.
|
|
281
317
|
- Paid model probes run one capability at a time and always confirm before a real provider call; video and transcription never submit empty-sample paid probes automatically.
|
|
282
318
|
- HTML screenshots render inside a sandboxed page without same-origin privileges; scripts and external network access are disabled by default.
|
|
283
|
-
-
|
|
319
|
+
- Legacy media links use random capability tokens; Core media reads verify Artifact content hashes. A host allowlist does not replace user authentication.
|
|
284
320
|
- Iris provides no account system of its own; multi-user isolation and access control are the responsibility of the DeepSeek Harness deployment.
|
|
285
321
|
|
|
286
322
|
## Known limitations
|
|
287
323
|
|
|
288
324
|
- `sharp` bundles native components; glibc Linux, Windows, and macOS are the primary targets. Non-standard environments such as bare Termux may need extra work.
|
|
289
|
-
- Video frame extraction and
|
|
325
|
+
- Video-file frame extraction and summaries require system-installed `ffmpeg` and `ffprobe`; summaries of existing Core frames need neither.
|
|
290
326
|
- HTML screenshots load no remote scripts, fonts, or images; inline the resources you need first.
|
|
291
327
|
- Under WSL, containers, and remote deployments, browser paths usually cannot be handed to the host directly — prefer uploading files.
|
|
292
328
|
- Provider model lists are candidate sets only; whether a model truly supports a capability is settled by real calls.
|
|
@@ -300,7 +336,7 @@ When the plugin is enabled, it automatically registers the two bundled skills th
|
|
|
300
336
|
| [`iris-verify-ui`](.dsh/skills/iris-verify-ui/SKILL.md) | Combine screenshots, semantic inspection, element grounding, cropping, and pixel comparison into evidence-based UI acceptance |
|
|
301
337
|
| [`iris-compose-media`](.dsh/skills/iris-compose-media/SKILL.md) | Chain two or more Iris tools into media workflows: inspect-then-draw, image-to-video, video summary with narration, S2V, and more |
|
|
302
338
|
|
|
303
|
-
From 0.1.2, as long as the host provides the DSH skill registry, regular npm installs can discover and load both skills from any project directory — no need to clone this repository or configure a skill search path. When the project directory contains same-name skills, DSH's native precedence keeps using the project versions; if the skill registry is unavailable, the
|
|
339
|
+
From 0.1.2, as long as the host provides the DSH skill registry, regular npm installs can discover and load both skills from any project directory — no need to clone this repository or configure a skill search path. When the project directory contains same-name skills, DSH's native precedence keeps using the project versions; if the skill registry is unavailable, the Iris tools still load on their own.
|
|
304
340
|
|
|
305
341
|
For deterministic selection, invoke either skill with DSH's `/name` form:
|
|
306
342
|
|
|
@@ -322,7 +358,7 @@ Tests cover config merging, model identity, task lifecycle, generation actions,
|
|
|
322
358
|
|
|
323
359
|
The test scheduler runs on Node.js, launching one process per file and stopping at the first failure — no Bash involved. Temporary directories use system APIs and are cleaned up on exit. GitHub Actions defines a Linux/Windows × Node.js 22.0.0/latest 22.x matrix, with repeated natural-exit checks on the minimum version, and a pre-release hook re-runs the full suite. DSH host smoke tests on native Windows/WSL are still pending real-machine verification.
|
|
324
360
|
|
|
325
|
-
Releases are triggered by `v*` tags through `.github/workflows/release.yml`: the full test suite runs first, then the tarball is packaged, a GitHub Release is created with the workflow's built-in `GITHUB_TOKEN` (tarball attached; release notes taken from the matching CHANGELOG section), and npm publish happens when an `NPM_TOKEN` secret is configured. No local GitHub credentials are needed.
|
|
361
|
+
Releases are triggered by `v*` tags through `.github/workflows/release.yml`: the full test suite runs first, then the tarball is packaged, a GitHub Release is created with the workflow's built-in `GITHUB_TOKEN` (tarball attached; release notes taken first from `docs/releases/<version>.md`, otherwise from the matching CHANGELOG section), and npm publish happens when an `NPM_TOKEN` secret is configured. No local GitHub credentials are needed.
|
|
326
362
|
|
|
327
363
|
## Documentation
|
|
328
364
|
|