@mokuyoaxis/dsh-iris 0.2.2 → 0.2.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (63) hide show
  1. package/.dsh/skills/iris-compose-media/SKILL.md +23 -73
  2. package/.dsh/skills/iris-compose-media/agents/openai.yaml +1 -1
  3. package/.dsh/skills/iris-compose-media/references/task-v2-recovery.md +23 -22
  4. package/.dsh/skills/iris-compose-media/references/workflows.md +25 -30
  5. package/.dsh/skills/iris-use/SKILL.md +41 -0
  6. package/.dsh/skills/iris-use/agents/openai.yaml +4 -0
  7. package/.dsh/skills/iris-use/references/media-contract.md +73 -0
  8. package/.dsh/skills/iris-verify-ui/SKILL.md +9 -7
  9. package/.dsh/skills/iris-verify-ui/agents/openai.yaml +1 -1
  10. package/.dsh/skills/iris-verify-ui/references/verdict-and-degradation.md +3 -0
  11. package/CHANGELOG.md +26 -2
  12. package/README.en.md +39 -32
  13. package/README.md +37 -31
  14. package/bin/dsh-iris.js +27 -6
  15. package/docs/AGENT_QUICKSTART.md +9 -9
  16. package/docs/ARCHITECTURE.md +1 -1
  17. package/docs/CORE_RUNTIME_CONTRACT.md +3 -1
  18. package/docs/DSH_CORE_MIGRATION.md +6 -2
  19. package/docs/FAULT_INJECTION.md +1 -0
  20. package/docs/HEADLESS_CLI.md +8 -4
  21. package/docs/HOST_DOCTOR.md +3 -3
  22. package/docs/MODEL_PORT_CONTRACT.md +6 -2
  23. package/docs/MUSIC.md +77 -0
  24. package/docs/PROMPT_OPTIMIZER.md +48 -10
  25. package/docs/PROVIDER_ADAPTER_CONTRACT.md +2 -0
  26. package/docs/ROADMAP.md +14 -25
  27. package/docs/SECURITY.md +3 -2
  28. package/docs/SYNC_DELIVERY_RECOVERY.md +39 -0
  29. package/docs/TASK_SEMANTICS.md +4 -4
  30. package/docs/releases/0.2.3.md +54 -0
  31. package/lib/actions.js +40 -3
  32. package/lib/adapters.js +1 -0
  33. package/lib/api.js +14 -5
  34. package/lib/bundled-skills.js +1 -1
  35. package/lib/capability.js +2 -0
  36. package/lib/client.js +111 -26
  37. package/lib/command-service.js +12 -3
  38. package/lib/config.js +26 -2
  39. package/lib/core-artifact-store.js +44 -6
  40. package/lib/core-delivery-cache.js +145 -0
  41. package/lib/core-maintenance.js +12 -1
  42. package/lib/core-tasks.js +19 -5
  43. package/lib/core-user-projection.js +2 -2
  44. package/lib/dsh-core-adapter.js +14 -4
  45. package/lib/dsh-text-model-adapter.js +15 -2
  46. package/lib/generation-input.js +10 -0
  47. package/lib/host-runtime.js +4 -2
  48. package/lib/index.js +23 -4
  49. package/lib/models.js +4 -1
  50. package/lib/music-generation.js +21 -0
  51. package/lib/music-options.js +18 -0
  52. package/lib/music-transport.js +46 -0
  53. package/lib/prompt-optimizer-config.js +92 -21
  54. package/lib/prompt-optimizer-core.js +51 -6
  55. package/lib/prompt-optimizer.js +20 -14
  56. package/lib/provider-adapter.js +1 -0
  57. package/lib/provider-adapters.js +24 -1
  58. package/lib/provider-catalog.js +18 -7
  59. package/lib/provider-config-service.js +13 -1
  60. package/lib/provider-protocol.js +12 -0
  61. package/lib/provider-task-runner.js +80 -5
  62. package/package.json +4 -2
  63. package/user_guide.md +30 -16
@@ -1,93 +1,43 @@
1
1
  ---
2
2
  name: iris-compose-media
3
- description: "Compose two or more dsh-iris tools into goal-driven image, video, speech, transcription, and multimodal workflows. Use when a request requires analyzing supplied media and creating a derivative, generating then reviewing an image, turning a video summary into narration, preparing image and audio inputs for video, or coordinating asynchronous Iris tasks. Do not use for a single direct media operation. 用于看图后绘图、生成后自检、视频总结配旁白、图像加语音生成视频等多步骤媒体任务。"
3
+ description: "Complete multi-step Iris creative requests and coordinated media assets: image revisions, image-to-video, video summaries with narration, talking videos, and optional music. 用于看图后绘图、图片修改与版本检查、视频总结配旁白、图片视频素材与可选配乐等创作流程。单步媒体操作用 iris-use,界面还原与验收用 iris-verify-ui。"
4
4
  ---
5
5
 
6
- # Compose Media with Iris
6
+ # 用 Iris 完成媒体创作
7
7
 
8
- Turn a media goal into the shortest reliable chain of Iris tools. Preserve useful attachments and paths between steps, keep paid generation bounded, and distinguish an accepted background task from a finished artifact.
8
+ 围绕用户要交付的作品组织必要步骤,复用已有素材,保留可以继续修改的版本。本 skill 适用于多步骤创作或多个配套产物;一次生成、改图、识别或查询由 `iris-use` 指导,界面还原与验收由 `iris-verify-ui` 指导。
9
9
 
10
- ## Decide whether to use this skill
10
+ ## 确定本次作品
11
11
 
12
- Use this skill only when the request needs two or more dependent Iris operations. Typical triggers include:
12
+ 从用户要求确定已有素材、最终产物和关键完成条件。仅在缺失信息会实质改变作品时询问,其余采用合理假设继续。
13
13
 
14
- - analyze an image, then create a new image from the findings;
15
- - generate an image, inspect it, and revise only if requested;
16
- - summarize a video, then write and synthesize narration;
17
- - generate or reuse a still image, synthesize speech, then create an S2V video;
18
- - transcribe or OCR source media, then turn its content into another artifact.
14
+ 不要因工具存在而追加分析、转写、检查或配乐。已有明确修改指令和Core原图时,可以直接编辑;不必先描述整张图。用户要求检查或修正时,按其要求复核与继续修改,保留每个有效版本。
19
15
 
20
- For one direct operation such as drawing, describing, OCR, transcription, speech synthesis, or video generation, do not use this skill; call the matching tool directly. For screenshot comparison or frontend visual acceptance, use `iris-verify-ui` instead.
16
+ 调用参数、模型能力和素材身份按[共享能力和产物说明](../iris-use/references/media-contract.md)。当前工具与配置决定实际可用路线,目录中的模型名不能证明功能可用。
21
17
 
22
- Ask one concise question only when a missing source, target format, voice, aspect ratio, or intended audience would materially change the result. Otherwise make a conservative assumption and state it in the result.
18
+ ## 选择创作流程
23
19
 
24
- ## Plan the chain
20
+ 按需要阅读[创作流程参考](references/workflows.md)的对应部分:
25
21
 
26
- Before calling tools, identify:
22
+ - 图片创作与版本修改:文字或参考素材 → `iris_draw_image` / `iris_edit_image` → 按要求看图或OCR → 交付选定版本。
23
+ - 图片视频素材:生成或复用首帧 → `iris_generate_video`;有源图时优先保留并复用。
24
+ - 视频内容与旁白:`iris_media_summarize` → 整理旁白稿 → `iris_speak_text`。
25
+ - 数字人口播:首帧 + 已完成的人声音频文件 → `iris_generate_video` 的S2V入口。
26
+ - 信息再创作:`iris_long_ocr` / `iris_transcribe_audio` → 核对可用文字 → 用户要求的新素材。
27
+ - 配套音乐:仅在用户要求或选择含配乐的流程时加入 `iris_generate_music`,和图片/视频分别保存与交付。
27
28
 
28
- 1. the source media and how Iris can read it;
29
- 2. the requested final artifact;
30
- 3. the minimum dependent tool chain;
31
- 4. which steps can incur generation cost;
32
- 5. whether a downstream step needs a file path, attachment ID, text, or completed task.
29
+ 视频分析优先复用已有 `frame_artifact_ids` 或 `transcribe_text`,无需再调用 `iris_video_frames` 或转写同一素材。依赖步骤顺序执行;独立素材可以在宿主工具允许时并行,不据此承诺供应商限额或更高成功率。
33
30
 
34
- Do not add an analysis, transcription, or review step merely because a tool exists. Parallelize only independent read-only work; dependent media operations must remain ordered.
31
+ ## 完成素材衔接
35
32
 
36
- ## Resolve media inputs
33
+ 每步保留实际返回的Task、Artifact和附件ID。下一步需要Core ID时直接复用;需要宿主文件时先明确导出。TTS与Music当前返回Core音频及播放链接,不能把链接作为 `audio_path`。
37
34
 
38
- - For edits to an existing Core image, pass its ID as `source_artifact_id` to `iris_edit_image` with the requested change. This creates a new Task and Artifact while keeping the source intact; it requires a configured chat image model.
39
- - PNG, JPEG and WebP can be used directly; do not convert a generated PNG to JPEG merely to call vision or OCR. Core IDs and host paths remain separate inputs.
40
- - Use `iris_look_at_image` for a host-visible absolute `image_path` or an existing Core image `artifact_id` (choose one).
41
- - Use `iris_relook_attachment` for an image attachment already present in the session or produced by Iris.
42
- - Use `iris_long_ocr` when exact text from an image matters more than a visual description; choose exactly one of `image_path`, Core `artifact_id`, or session `attachment_id`. `iris_locate` accepts the same three sources for original-pixel bounding boxes.
43
- - Video and audio file inputs require host-visible absolute `video_path` or `audio_path` values. Video summaries can instead consume existing Core `frame_artifact_ids`. A browser-local path, `content://` URI, or ordinary web URL is not a host path; request an upload or exported host path.
44
- - Keep every returned Core Artifact ID, attachment ID, task ID, and output path that a later step needs. Core `artifact_id` and session `attachment_id` are distinct; do not use either as a file path or invent identifiers.
35
+ 完成一个步骤后再启动依赖它的生成。排队、运行中或交付失败都不代表文件已经存在。已有完整作品的补交付、人工重试和Music内置502重试的区别,按需阅读[任务恢复说明](references/task-v2-recovery.md)。
45
36
 
46
- An attachment returned by `iris_draw_image` can be passed as `first_frame_attachment_id` to `iris_generate_video`. An existing host image uses `first_frame_path`. The local path returned by `iris_speak_text` can be passed as `audio_path` for S2V.
37
+ ## 按目标检查与交付
47
38
 
48
- ## Choose a workflow
39
+ 检查创作要求中的具体内容:图片的主体、文字和构图;视频的动作与片段用途;旁白的内容与声音。音乐风格与听感由试听判断。像素差异用于可比图片的测量,不是通用创作品质分数。
49
40
 
50
- Choose the minimum chain, then read only the matching section of [references/workflows.md](references/workflows.md):
41
+ 音乐为默认不执行的可选步骤。可选音乐失败时保留其他完成素材并报告缺失;用户明确要求带配乐成品时,缺少音乐意味着该要求尚未完成。当前Iris没有自动混音/合成工具,分别交付素材不能表述为已经完成配乐成片。
51
42
 
52
- - inspect an image, then draw: `iris_look_at_image` or `iris_relook_attachment` → `iris_draw_image`;
53
- - edit a Core image, then inspect: `iris_edit_image` → `iris_look_at_image` using the new Artifact ID;
54
- - generate/reuse a still, then animate: optional `iris_draw_image` → `iris_generate_video`;
55
- - summarize a video, then narrate: `iris_media_summarize` → `iris_speak_text`;
56
- - create an S2V talking video: resolve a first frame and audio → `iris_generate_video`;
57
- - recognize, then create: `iris_long_ocr` or `iris_transcribe_audio` → the requested creation tool.
58
-
59
- Use `iris_video_frames` only for explicit frame extraction or a custom frame-level workflow. Do not load every workflow section when one route is sufficient.
60
-
61
- If frames are already saved by `iris_video_frames`, pass its returned `frame_artifact_ids` to `iris_media_summarize` instead of extracting again. This reads the original timestamps and frame order without the video file or ffmpeg. Omit `video_path`, `max_frames`, `target_width`, and `transcribe:true`; supply existing speech as `transcribe_text` when available.
62
-
63
- ## Control cost and iteration
64
-
65
- - Create one new Iris generation **Task** per requested artifact by default. A Provider Attempt is an internal candidate submission inside that Task; do not count failover attempts as user-authorized new Tasks.
66
- - The original request authorizes its requested artifact. Review alone does not authorize regeneration.
67
- - Create at most 2 new generation Tasks per artifact, and only when the first Task has a known terminal outcome and the user explicitly requested iteration. Respect a lower or higher bound stated by the user.
68
- - Do not probe models or create speculative alternatives. Honor an explicit `providerId::modelId`; otherwise use Iris capability assignment and bounded failover.
69
- - Automatic failover is allowed only while the current Provider Attempt has `acceptance=not_accepted`. Accepted, acceptance-unknown, or outcome-unknown work must never trigger another Provider Attempt or Task automatically.
70
-
71
- ## Handle Task v2, cancellation, and failures
72
-
73
- When a generation tool returns a Task, or the user asks to retry, recover, cancel, or deliver one, read [references/task-v2-recovery.md](references/task-v2-recovery.md). Its stop rules are mandatory.
74
-
75
- - Continue immediately only from a completed attachment, text, or local path.
76
- - Use `iris_task_status` to observe an existing background Task when a dependent next step needs the result. Never describe queued/running work as complete.
77
- - Observation failure resumes observation; delivery failure resumes delivery. Neither authorizes regeneration. If that recovery action is available only in the Iris workbench, say so and preserve the Task ID.
78
- - If the user cancels, stop the chain and do not launch downstream generation. Cancellation requested or unknown is not proof that no provider work occurred.
79
- - Preserve partial outputs. Report which step succeeded, which failed or became uncertain, and whether accepted work may remain active.
80
- - If an attachment is missing, request a re-upload or host path. If `ffmpeg`/`ffprobe` is unavailable, new video frame extraction cannot run; summaries can still use existing Core frame IDs.
81
- - If vision is unavailable but the creation prompt is complete, skip source analysis only when the requested result still remains valid.
82
-
83
- ## Report the result
84
-
85
- Return a compact workflow record containing:
86
-
87
- 1. the completed chain and any material assumptions;
88
- 2. attachment IDs and host paths needed to use the outputs;
89
- 3. task IDs and their last known status;
90
- 4. any omitted or failed step and its effect on the final artifact;
91
- 5. whether further generation would create another potentially billable Task.
92
-
93
- Never hide that a summary omitted audio, that a review was skipped, or that the final artifact differs from the request.
43
+ 按本次授权的产物、修改范围和预算执行,不固定所有流程的Task数量。不要为一次失败再添加外层重试;用户取消时停止后续生成。报告已完成作品及可用链接/ID、部分结果、关键假设和未完成条件;进一步生成的费用与受理事实以任务记录为准。
@@ -1,4 +1,4 @@
1
1
  interface:
2
2
  display_name: "Iris Compose Media"
3
3
  short_description: "Compose multi-step media workflows with Iris"
4
- default_prompt: "Use $iris-compose-media to analyze the supplied media and complete the requested multi-step creation workflow."
4
+ default_prompt: "Use $iris-compose-media to complete my creative workflow, reuse existing assets, and add music only when requested."
@@ -1,30 +1,31 @@
1
- # Iris Task v2 recovery
1
+ # Iris 任务恢复与重新生成
2
2
 
3
- Read this reference only after a generation tool has returned a Task, or when the user asks about retrying, cancelling, recovering, or delivering an existing Task.
3
+ 本文件保留原路径,供Iris skills共用。已有任务需要等待、恢复、取消或重试时阅读;普通生成无需逐项执行恢复流程。
4
4
 
5
- ## Keep the identities separate
5
+ ## 三类操作
6
6
 
7
- - A **Task** represents one user-authorized artifact request.
8
- - A **Provider Attempt** is an internal candidate submission inside that Task.
9
- - Automatic failover may create another Provider Attempt only while acceptance is explicitly `not_accepted`; it does not authorize a new Task.
10
- - A new Task is a new potentially billable user operation. Create it only for the original requested artifact or after explicit, informed retry/iteration approval.
7
+ **查询/观察**检查已有任务;**补交付**取回已经生成的作品;**重新生成**创建新Task并可能消耗费用/额度。Provider Attempt是Task内的候选提交,不能把它与Agent再次调用工具混为一谈。
11
8
 
12
- ## Decide from facts
9
+ 通用候选只在acceptance明确为not_accepted时切换。已受理或受理未知时不由Agent自动换模型或重新提交。用户已经授权的作品修改按该范围执行;单纯查看或检查不增加重新生成授权。
13
10
 
14
- | Observed fact | Agent action | Forbidden action |
15
- |---|---|---|
16
- | `acceptance=not_accepted` | Let Iris continue its bounded provider failover inside the same Task | Creating a replacement Task merely to imitate failover |
17
- | accepted or `acceptance=unknown` | Preserve the Task ID and stop new generation | Resubmitting to another provider/model |
18
- | queued/running | Observe the existing Task only when the next step depends on it | Claiming the artifact exists or starting a duplicate |
19
- | `watching_paused` / observation failure | Tell the user to resume/re-observe the same Task in the Iris workbench | Starting generation again |
20
- | succeeded but delivery unavailable | Tell the user to retry delivery for the same Task in the Iris workbench | Regenerating the media |
21
- | `needs_attention` / outcome unknown | Report the known facts and request human review | Guessing failure, success, or billing outcome |
22
- | cancellation requested/unknown | Stop downstream work and preserve the Task ID | Treating cancellation as proof that the provider did no work |
23
- | terminal failure with known non-acceptance | Report failure and the exhausted attempts | Silently starting a new Task |
24
- | terminal result reviewed and user explicitly requests another version | Create at most one additional Task unless the user sets a larger bound | Treating review itself as approval to regenerate |
11
+ ## 根据实际状态继续
25
12
 
26
- `iris_task_status` observes the existing Task. If an action is not exposed as an Iris Agent tool, direct the user to the existing Task in the Iris workbench; never simulate recovery by calling a generation tool again.
13
+ | 事实 | 操作 |
14
+ |---|---|
15
+ | 排队/运行中 | 用 `iris_task_status` 查询;依赖结果未完成前不启动下一步 |
16
+ | 观察暂停/观察失败 | 通过工作台“重新观察”或CLI task observe继续同一Task,不重新生成 |
17
+ | outcome=succeeded、deliveryState=failed | 对原Task补交付;同步完整结果使用已有缓存,不再生成 |
18
+ | acceptance/outcome未知 | 保留Task与未知事实,不推定上游没有生成或没有扣额度 |
19
+ | 用户请求取消 | 停止后续步骤;本地停止等待不等于远端取消成功 |
20
+ | 终态且未成功交付,用户知情要求重试 | 用工作台重试或CLI task retry创建关联的新Task,确认可能重复计费并重新提供输入 |
21
+ | 已完成作品,用户要求修改/另一版 | 按已授权要求生成新版本,保留已有产物;不是旧Task的恢复 |
27
22
 
28
- ## Report uncertainty
23
+ `iris_task_status`本身只查询事实,不执行重新观察或补交付。恢复动作目前主要在工作台/API和CLI,不能靠再次调用生成工具模拟恢复。CLI task retry需要confirm-billing;新任务记录retriedFrom,旧Task保持。
29
24
 
30
- Always include the Task ID, last observed state, acceptance/outcome knowledge, whether a remote operation may still be active, and the exact safe next action. Do not collapse “could not observe” into “generation failed.”
25
+ ## Music 的有限502策略
26
+
27
+ Music工具一次调用可按参数对HTTP502重新生成同一模型:默认额外2次、每次新Task并记录retriedFrom。502可能发生在上游已生成后,因此受理未知仍保留,重试可能额外扣额度。它是已配置的功能行为,不是通用候选切换,也不授权Agent再包一层重试。
28
+
29
+ Music超时、其他HTTP状态或无音频响应不套用该策略;有完整音频的交付失败先补交付。Music网关没有已证实的远端查询/取消接口,不为它编造可轮询ID或撤销成功。普通task retry只重新生成一次,不接收Music重试参数。
30
+
31
+ 报告Task ID、已知状态、可用产物、未知事实和下一步。用户已明确授权的操作不重复征求同一授权;任务范围扩大或新的知情重试确有必要时再说明影响。
@@ -1,44 +1,39 @@
1
- # Iris composition workflow details
1
+ # Iris 创作流程参考
2
2
 
3
- Read only the section matching the chosen workflow. Keep paid steps bounded by the main Skill and Task v2 recovery rules.
3
+ 仅阅读本次目标对应的部分。参数、Core ID、附件与文件路径的共同约定见主文链接的共享说明。
4
4
 
5
- ## Inspect, then draw
5
+ ## 图片创作与版本修改
6
6
 
7
- 1. Inspect the source with `iris_look_at_image` or `iris_relook_attachment`. Ask about subject, composition, palette, style, visible text, and the requested transformation.
8
- 2. Turn observations into a self-contained prompt. Separate visible facts from requested changes and preserve exact text.
9
- 3. Call `iris_draw_image` once for each requested artifact.
10
- 4. Inspect the result only when verification or iteration was requested. A mismatch is evidence to report, not automatic permission to regenerate.
7
+ 已有Core原图并明确要求修改时,直接用 `iris_edit_image(source_artifact_id, prompt)`。写清改变和保留的内容;当前仅支持单张Core静态图片输入。源图只有附件或路径时,说明改图入口需要Core作品,不把附件ID当作Core ID,也不虚构导入工具。
11
8
 
12
- ## Edit a Core image, then inspect
9
+ 需要理解参考素材后再创作时,使用 `iris_look_at_image` 或 `iris_relook_attachment`,提取与目标有关的主体、构图、文字和风格,再调用 `iris_draw_image`。保留用户给出的准确文字;生成后的自述可以提供线索,不需要默认再做一次相同的全图描述。
13
10
 
14
- 1. Resolve the source Core Artifact ID; one static PNG, JPEG or WebP image up to 20 MiB is supported.
15
- 2. Call `iris_edit_image` with `source_artifact_id` and a focused modification prompt, optionally an explicit model. The original stays intact and the edited Artifact records its source.
16
- 3. Inspect the new Artifact ID only when review is requested. Preserve the Task ID if acceptance or delivery is uncertain; do not repeat the edit automatically.
11
+ 用户要求检查时针对具体标准使用看图或 `iris_long_ocr`。已有授权的修改继续用新Artifact作为下一版来源,不从头重画源作品。交付选定版本及其真实ID,保留原版本;模型描述不能证明画面精确满足每一项要求。
17
12
 
18
- ## Generate or reuse a still, then animate
13
+ ## 图片视频素材
19
14
 
20
- 1. Create a still with `iris_draw_image`, reuse an Iris-produced first-frame attachment, or obtain a host-visible path. Do not pass an ordinary session attachment as `first_frame_attachment_id`.
21
- 2. Write a motion prompt that covers camera movement, subject motion, timing, and invariants.
22
- 3. Call `iris_generate_video` with `first_frame_attachment_id` or `first_frame_path`. Use only controls supported by the selected t2v/i2v model.
23
- 4. Observe a returned Task before claiming that the video exists.
15
+ 复用Iris生成的首帧附件作为 `first_frame_attachment_id`,或使用真实宿主图片的 `first_frame_path`。其他会话附件不能直接充当该首帧ID。没有首帧且任务需要时才调用 `iris_draw_image`。
24
16
 
25
- ## Summarize a video, then narrate
17
+ 将运动、镜头、时序及需保留的画面特征写入prompt,调用支持该模式的 `iris_generate_video`。首帧和视频分别保留;等待未完成时报告Task,不宣称已有成片。
26
18
 
27
- 1. Call `iris_media_summarize` with `video_path` and `transcribe=true` unless the user wants visual-only analysis. If Core frames already exist, use their `frame_artifact_ids` instead and optionally provide `transcribe_text`; omit `transcribe=true` and sampling options for existing frames.
28
- 2. Do not also call `iris_video_frames` or `iris_transcribe_audio` by default; the summary action already samples frames and can transcribe audio.
29
- 3. Write narration for the requested audience and length without presenting inference as visible fact.
30
- 4. Call `iris_speak_text`; return the summary, contact-sheet attachment, and saved audio path.
19
+ ## 视频内容与旁白
31
20
 
32
- Use `iris_video_frames` separately only for individual frames or a custom frame workflow. Use `iris_transcribe_audio` separately for a full transcript or an audio-only input.
21
+ 已有Core帧时调用 `iris_media_summarize(frame_artifact_ids, transcribe_text?)`,不重复抽帧,也不传video_path、采样参数或transcribe:true。只有视频文件时按目标决定是否需要音轨;纯画面分析明确transcribe:false,已有转写直接提供。
33
22
 
34
- ## Create an S2V talking video
23
+ 整理用户需要的摘要和旁白稿,再用 `iris_speak_text`。交付摘要、联系表和音频Artifact/播放链接;只有后续处理需要音频文件时才导出。转写未完成或摘要仅包含画面时说明范围,不补造台词。
35
24
 
36
- 1. Resolve or generate a suitable first-frame portrait.
37
- 2. Use existing host audio or call `iris_speak_text` and keep its absolute output path.
38
- 3. Confirm that the audio is a clear human voice in WAV/MP3, smaller than 15 MB, and shorter than 20 seconds.
39
- 4. Call `iris_generate_video` with an S2V model, first-frame input, `audio_path`, and `480P` or `720P`. Do not pass t2v/i2v-only `size` or `duration` controls.
40
- 5. Observe the returned Task; do not submit a duplicate while it is running.
25
+ ## 数字人口播
41
26
 
42
- ## Turn recognized content into a new artifact
27
+ 复用或生成首帧。已有可用宿主人声音频则直接使用;TTS返回Core音频时先按共享说明导出为文件,再传 `audio_path`。音频应是清晰人声WAV/MP3、小于15 MB、时长小于20秒。
43
28
 
44
- Use `iris_long_ocr` for exact image text or `iris_transcribe_audio` for speech, then pass only necessary checked facts into the selected generation tool. Preserve names, numbers, and quoted wording when accuracy matters.
29
+ 使用支持S2V的模型和resolution=480P/720P,不传文生/图生专用size/duration。Music不是通用口播输入;音乐与人声混合成新的音轨需要另一个实际可用、已授权的处理步骤。
30
+
31
+ ## 信息再创作
32
+
33
+ `iris_long_ocr`用于准确图片文字,`iris_transcribe_audio`用于说话音频。OCR为partial时保留已识别部分并说明缺口,按目标决定是否需要补充;不要把部分识别当作完整原文。将必要事实交给用户要求的创作步骤,保留姓名、数量、引文与来源ID。
34
+
35
+ ## 可选配套音乐
36
+
37
+ 用户要求歌曲、配乐或选定含音乐的创作目标时,使用 `iris_generate_music`,把情绪、年代、乐器、人声、语言及期望时长写入prompt。图片和音乐可以共用创作要求,但分别保存,不宣称二者已自动关联或混音。
38
+
39
+ 用户仅要求图片或视频时不添加音乐。可选音乐失败时交付其他完成素材并说明;带配乐成品的要求未满足则明确留为未完成。一次Music工具调用可以按配置包含有限502重试,不再用Agent循环追加调用,也不换成TTS或程序生成节拍。
@@ -0,0 +1,41 @@
1
+ ---
2
+ name: iris-use
3
+ description: "Use dsh-iris for a single media operation or an existing task: image creation/editing, vision/OCR, video, speech, music, and artifact delivery. 用于 Iris 单步媒体操作、音乐生成、作品取回与任务状态查询;多步骤创作选 iris-compose-media,界面对照验收选 iris-verify-ui。"
4
+ ---
5
+
6
+ # 使用 Iris 媒体能力
7
+
8
+ 把用户要求交给对应的 Iris 功能,使用当前配置分配的模型,并交付实际产物。单独生成一首音乐、修改一张图片、识别文字或查询已有任务都适用本 skill。
9
+
10
+ ## 选择入口
11
+
12
+ | 用户目标 | Iris 工具 |
13
+ |---|---|
14
+ | 生成图片 | `iris_draw_image` |
15
+ | 修改已有 Core 图片 | `iris_edit_image`,输入 `source_artifact_id` |
16
+ | 理解图片 | `iris_look_at_image`;会话附件用 `iris_relook_attachment` |
17
+ | 识别图片文字、定位对象 | `iris_long_ocr`、`iris_locate` |
18
+ | 生成视频 | `iris_generate_video` |
19
+ | 合成说话声音、转写音频 | `iris_speak_text`、`iris_transcribe_audio` |
20
+ | 生成歌曲或配乐 | `iris_generate_music` |
21
+ | 裁剪、图片比较、静态 HTML 截图、视频抽帧 | `iris_crop`、`iris_pixel_diff`、`iris_html_screenshot`、`iris_video_frames` |
22
+ | 理解视频或已有帧 | `iris_media_summarize` |
23
+ | 查询已有任务 | `iris_task_status` |
24
+
25
+ 多步骤作品与素材组合由 `iris-compose-media` 指导;界面还原、截图对照和视觉验收由 `iris-verify-ui` 指导。一次操作无需追加另一项 skill。
26
+
27
+ ## 按当前能力执行
28
+
29
+ - 当前工具列表是可调用入口的依据;skill 可见不代表插件已经装载新工具。工具缺失、服务端/客户端不一致或模型未配置时,说明具体缺口。必要时查看 Iris 工作台的宿主诊断与能力分配,不以生成请求探测配置。
30
+ - 模型名称、供应商名称和模型目录不能证明能力。尊重用户指定的 `providerId::modelId`,其余按功能配置选择;Music 必须有明确分配,并绑定音乐协议。
31
+ - 参数与输入输出的详细区别见[共享能力和产物说明](references/media-contract.md)。仅阅读当前操作相关部分。
32
+ - 请求歌曲或配乐时调用 Music;TTS 用于说话声音。音乐未配置或调用失败时报告原因,不擅自用 NumPy、节拍合成或 TTS 替代。用户明确要求程序化合成时按该要求处理。
33
+ - 音乐生成的时长、歌词、演唱语言和风格当前由 prompt 表达;不把它们当作保证生效的结构化控制。
34
+
35
+ ## 交付与继续已有工作
36
+
37
+ 交付实际返回的图片附件、Artifact ID、播放链接或文本,并保留 Task ID。Core ID、会话附件 ID、HTTP 播放链接和宿主文件路径是不同输入,按下一步需要使用或导出,不推测存储路径。
38
+
39
+ 已有任务的失败、恢复、取消或重试,按需阅读[任务恢复说明](../iris-compose-media/references/task-v2-recovery.md)。Music 工具内部的有限502重试由参数控制;Agent 不在外层再包重试循环。
40
+
41
+ 用户只请求生成时无需额外调用检查工具。确需复核时按原要求检查;音乐风格与听感由试听判断,不由音频格式或转写结果代替。未完成或未交付的任务如实报告,已有产物继续保留。
@@ -0,0 +1,4 @@
1
+ interface:
2
+ display_name: "Iris Use"
3
+ short_description: "Use Iris media tools and deliver existing artifacts"
4
+ default_prompt: "Use $iris-use to choose the available Iris tool for my media request and return its actual artifact or task result."
@@ -0,0 +1,73 @@
1
+ # Iris 能力与产物衔接
2
+
3
+ 三项 Iris skills 共用本说明。按当前工具 schema 调用;下表描述 Iris 0.2.3 的入口,是否可用须看实际工具列表。不要为阅读本说明调用模型或读取供应商凭据。
4
+
5
+ ## 工具、输入和输出
6
+
7
+ | 工具 | 主要输入 | 完成后的输出 |
8
+ |---|---|---|
9
+ | `iris_draw_image` | prompt,可选 size、n、model | Task、Core 图片 Artifact、DSH 图片附件;可能附有图片自述 |
10
+ | `iris_edit_image` | source_artifact_id、修改 prompt | 新图片 Artifact 与附件,保留原图和来源关系 |
11
+ | `iris_look_at_image` | image_path 或 artifact_id 二选一 | 图片理解文字 |
12
+ | `iris_relook_attachment` | attachment_id | 会话图片理解文字 |
13
+ | `iris_long_ocr` | image_path、artifact_id、attachment_id 三选一 | 识别文字和 complete/partial/failed 状态;部分结果不能当作全文 |
14
+ | `iris_locate` | 同上,加 target | 原文件像素 bbox 或未找到;模型定位不是像素真值 |
15
+ | `iris_generate_video` | 文本、首帧或 S2V 首帧加 audio_path | 完成后视频 Artifact/播放链接;等待未完成则返回 Task |
16
+ | `iris_speak_text` | text、可选 voice、model | Core 音频 Artifact 与播放链接,当前 Core 分支不返回本地文件路径 |
17
+ | `iris_generate_music` | prompt、可选 model、timeout_ms、retry_502、retry_delay_ms | MP3/WAV Artifact 与播放链接;有限重试可创建关联的新 Task |
18
+ | `iris_transcribe_audio` | 宿主 audio_path;CLI/Action另支持audio_url | Core 转写任务、文本产物;不保证有逐词时间轴 |
19
+ | `iris_task_status` | 可选 task_id | 任务事实及完成后的 Artifact/播放链接;查询不代替重新观察或交付 |
20
+ | `iris_crop` | image_path 或 attachment_id,加裁剪区域 | Core 裁剪图片与 DSH 附件 |
21
+ | `iris_pixel_diff` | 两侧各自使用图片路径或会话附件 | 差异统计、Core 热力图和 DSH 附件 |
22
+ | `iris_html_screenshot` | 静态 HTML 字符串 | Core 截图和 DSH 图片附件 |
23
+ | `iris_video_frames` | video_path | Core frame_artifact_ids、时间戳和会话图片附件 |
24
+ | `iris_media_summarize` | video_path 或 frame_artifact_ids 二选一 | 摘要和带时间戳的联系表附件,可消费现有转写文字 |
25
+
26
+ 不要用工具名之外的虚构入口,例如 Iris 当前没有 Agent 提示词优化工具或自动音视频混音工具。提示词预览属于工作台/HTTP功能;Agent 可以直接整理已授权的创作要求,不必为了每次生成追加优化调用。
27
+
28
+ ## 功能与模型选择
29
+
30
+ 生图、视频、TTS、转写和视觉沿用当前功能配置。改图只接受带图编辑的聊天图片协议,当前是 `openai-chat-images`,源图为一个 Core 静态 PNG/JPEG/WebP,最多20 MiB;不提供蒙版或多张参考图输入。
31
+
32
+ Music 为独立 `music-gen` 功能,仅使用明确分配名单。模型须配置 `musicProtocol: gemini-web-music`;它不改变账号图片默认协议。其他 Music Adapter 尚未实现。当前正式参数控制客户端超时与502重试;曲长、人声、歌词、语言等是描述意图。
33
+
34
+ 通用候选仅在明确未受理时切换。Music 的HTTP502策略是按配置重新生成同一模型,不是换供应商:默认额外2次、间隔5000ms,每次新Task并记录retriedFrom,可能额外消耗额度。Agent 调用一次工具后消费其结果;工具失败不授权另一层自动重试。已有完整结果优先补交付,详见各 skill 链接的任务恢复说明。
35
+
36
+ ## 保持输入身份
37
+
38
+ - Core Artifact ID 指向当前数据根中的持久作品;会话 attachment_id 属于宿主附件。二者不可互换。
39
+ - `/iris/api/core/artifact/.../media` 是播放/下载链接,不是宿主 `audio_path` 或 `video_path`。
40
+ - `content://`、浏览器本地路径与普通网络URL不能直接充当宿主文件路径。已经有可用Core作品时优先复用,确需文件输入时才导出。
41
+ - 保存每步实际返回的ID。图片编辑保留原作品并创建新版本;不要为继续操作重新生成源素材。
42
+
43
+ ## 需要文件时导出
44
+
45
+ 导出消费已有作品,不调用生成模型。DSH工具没有通用Artifact导出入口,可用工作台下载,或在具备Shell与相同Core数据根时使用CLI。以下路径和ID必须替换为已确认的实际值;输出文件不能覆盖已有文件。
46
+
47
+ ```bash
48
+ dsh-iris artifact export artifact_ID \
49
+ --data-root /absolute/path/iris-core \
50
+ --output /absolute/path/narration.wav
51
+ ```
52
+
53
+ 按实际Artifact媒体类型选择后缀,导出不会转换格式。TTS → S2V:取回已完成音频Artifact → 导出 → 确认宿主文件可读 → 将该路径传给 `audio_path`。不得从ID或播放链接拼出文件路径。
54
+
55
+ ## Core图片的本地处理
56
+
57
+ Agent crop/diff 工具使用路径或会话附件;CLI 可直接消费Core图片ID,无需先导出。CLI与Agent的参数表不能混用:
58
+
59
+ ```bash
60
+ dsh-iris run crop --data-root /absolute/path/iris-core \
61
+ --input '{"artifact_id":"artifact_ID","left":10,"top":20,"width":200,"height":100}'
62
+
63
+ dsh-iris media diff --data-root /absolute/path/iris-core \
64
+ --input '{"image_a_artifact_id":"artifact_A","image_b_artifact_id":"artifact_B"}'
65
+ ```
66
+
67
+ ## 视频与声音的区别
68
+
69
+ - 文生/图生视频使用prompt、首帧及模型支持的size/duration。首帧附件限Iris生成图片的附件;其他图片应使用宿主文件路径。
70
+ - S2V为数字人口播:首帧加清晰的人声WAV/MP3,文件小于15 MB、时长小于20秒,resolution为480P或720P。不传文生/图生专用size/duration。
71
+ - 音乐不是S2V口播音轨的通用替代输入。配乐可作为独立素材交付;最终混音/合成只有在相应工具可用且任务要求时执行。
72
+ - 视频摘要已有Core帧时使用frame_artifact_ids,不重复抽帧;已有转写用transcribe_text。此模式不传video_path、抽帧选项或transcribe:true。
73
+ - 从视频文件抽帧需ffmpeg/ffprobe;复用Core帧不需要。Agent的视频文件摘要可以转写音轨;仅需画面时明确transcribe:false。
@@ -1,12 +1,14 @@
1
1
  ---
2
2
  name: iris-verify-ui
3
- description: "Verify web UIs, HTML, and interface screenshots with dsh-iris through repeatable capture, semantic inspection, element location, cropping, pixel-diff analysis, and bounded rechecks. Use when comparing a UI with a reference image, locating visual regressions, validating frontend changes, or matching an implementation to a design. 用于对照参考图检查界面、定位视觉偏差、验证前端修改或比较两张 UI 截图。"
3
+ description: "Reproduce and verify web interfaces against reference screenshots or acceptance criteria using Iris vision, OCR, capture, grounding and pixel comparison. 用于对照参考图检查界面、还原设计、定位视觉偏差与验证前端修改;单步图片理解或媒体创作用 iris-use / iris-compose-media。"
4
4
  ---
5
5
 
6
6
  # Verify UI with Iris
7
7
 
8
8
  Use vision models to understand and locate problems. Use deterministic pixel tools to measure them. Establish comparable capture conditions before editing, then recheck under the same conditions. Do not declare success from visual impression alone.
9
9
 
10
+ Choose checks that answer the actual acceptance criteria. Read the relevant part of the [shared capability and artifact contract](../iris-use/references/media-contract.md) when an input or handoff is unclear; a standalone description or image generation belongs to `iris-use`.
11
+
10
12
  ## Choose the shortest path
11
13
 
12
14
  Select a path from the evidence provided:
@@ -22,7 +24,7 @@ Ask one concise question only when a missing reference, target state, theme, or
22
24
  ## Prepare the inputs
23
25
 
24
26
  1. Identify each image source:
25
- - Existing Core image: pass `artifact_id` to look/locate/OCR. Crop and pixel diff still need a supported path or attachment; export the Artifact explicitly if those steps require it.
27
+ - Existing Core image: pass `artifact_id` to look/locate/OCR. Agent crop/diff need a path or attachment; CLI `run crop` and `media diff` accept Core IDs directly. Use the shared contract's examples, or export only when the chosen consumer needs a file.
26
28
  - Host absolute path: pass `image_path`.
27
29
  - Current-session or Iris-produced attachment: pass `attachment_id`.
28
30
  - Browser-local path, `content://` URI, or ordinary web URL: do not assume the host can read it. Ask for an upload or a host-accessible file.
@@ -46,7 +48,7 @@ When raw HTML needs capture, call `iris_html_screenshot` with the same HTML wrap
46
48
 
47
49
  ### 2. Inspect semantics
48
50
 
49
- Inspect the current and reference images separately. Call `iris_look_at_image` for a Core Artifact ID or host path and `iris_relook_attachment` for an attachment.
51
+ When semantic interpretation is needed, inspect the current and reference images separately. Call `iris_look_at_image` for a Core Artifact ID or host path and `iris_relook_attachment` for an attachment. Use `iris_long_ocr` when exact interface text matters; preserve its partial-result status.
50
52
 
51
53
  Each vision call sees only one image. Ask the same structured questions for both images:
52
54
 
@@ -58,7 +60,7 @@ Compare the two observation sets and prioritize the differences. Treat vision an
58
60
 
59
61
  ### 3. Measure the whole image
60
62
 
61
- Call `iris_pixel_diff` with the current and reference images. Record:
63
+ When quantitative screenshot comparison serves the criteria, call `iris_pixel_diff` with the current and reference images, or CLI `media diff` for Core IDs. Record:
62
64
 
63
65
  - diff ratio
64
66
  - changed-pixel count and normalized dimensions
@@ -87,7 +89,7 @@ Run `iris_pixel_diff` on the two crop attachments to verify whether the problem
87
89
 
88
90
  Edit source files only when the task authorizes changes. Fix the highest-impact deterministic issue first, keep the patch focused, and preserve existing user changes.
89
91
 
90
- After each modification:
92
+ After an authorized modification, repeat the relevant checks under the same conditions:
91
93
 
92
94
  1. Capture again under the baseline conditions.
93
95
  2. Rerun the whole-image diff.
@@ -95,11 +97,11 @@ After each modification:
95
97
  4. Record before-and-after diff ratios and bounding-box changes.
96
98
  5. Confirm that the fix did not move the difference elsewhere.
97
99
 
98
- Run at most 3 modify-and-recheck rounds by default. Stop as soon as the user's criteria pass. Also stop when one round gives no improvement, capture conditions cannot be aligned, or remaining differences come from the environment. Explain the stopping reason instead of looping indefinitely.
100
+ Use the task's revision scope and budget. Stop when the criteria pass; if progress stops, capture conditions cannot be aligned, or differences come from the environment, report that limit instead of repeating the same loop. Visual checks do not prove browser interactions or application behavior; verify those with the project's actual browser/test workflow when required.
99
101
 
100
102
  ## Assign a verdict and degrade safely
101
103
 
102
- Read [references/verdict-and-degradation.md](references/verdict-and-degradation.md) when assigning the final verdict or when Browser, vision, location, offline HTML, or comparable capture conditions are unavailable. Use exactly **Pass**, **Partial pass**, or **Indeterminate**, and apply the reference's evidence limits.
104
+ Read [references/verdict-and-degradation.md](references/verdict-and-degradation.md) when assigning the final verdict or when Browser, vision, location, offline HTML, or comparable capture conditions are unavailable. Use **Pass**, **Partial pass**, **Fail**, or **Indeterminate** according to the evidence. Confirmed unmet requirements are Fail; missing evidence is Indeterminate.
103
105
 
104
106
  ## Report the result
105
107
 
@@ -1,4 +1,4 @@
1
1
  interface:
2
2
  display_name: "Iris Verify UI"
3
3
  short_description: "Verify UI changes with visual evidence"
4
- default_prompt: "Use $iris-verify-ui to compare the supplied UI with its reference and return a bounded evidence-based verdict."
4
+ default_prompt: "Use $iris-verify-ui to check my interface against its reference or acceptance criteria and report a verdict supported by evidence."
@@ -8,8 +8,11 @@ Use exactly one verdict:
8
8
 
9
9
  - **Pass**: all objective requirements are satisfied and comparable evidence supports the conclusion.
10
10
  - **Partial pass**: the primary goal is satisfied, but localized differences or environmental noise remain.
11
+ - **Fail**: comparable evidence confirms a required criterion is unmet. Describe the failed criterion; do not replace a known failure with uncertainty.
11
12
  - **Indeterminate**: a reference is missing, capture conditions differ, a required tool is unavailable, or evidence is insufficient.
12
13
 
14
+ A reference screenshot is needed for reference-matching claims; explicit acceptance criteria can still support a verdict without one. Use Partial pass only for remaining optional/minor items when the required goal passes. A required criterion that fails takes precedence over missing evidence for other criteria; report those uncertainties alongside Fail.
15
+
13
16
  Report:
14
17
 
15
18
  1. Conditions: inputs, dimensions, theme, capture method, and known noise.
package/CHANGELOG.md CHANGED
@@ -4,7 +4,30 @@
4
4
 
5
5
  ## [Unreleased]
6
6
 
7
- 后续未归入版本的变更在此记录。
7
+ ## [0.2.3] - 2026-10-14
8
+
9
+ ### Added
10
+
11
+ - 独立音乐生成,支持模型分配、Chat Completions 内嵌MP3/WAV、CLI/Agent工具/API和工作台试听下载;可配置有限502重试,重发保留独立任务关系。见 [音乐生成](docs/MUSIC.md)。
12
+ - 随包 `iris-use`,指导单步媒体操作、音乐、任务查询与作品交付;三项Skill共用能力和产物说明。
13
+ - 同步媒体结果的私有交付缓存。本地交付失败可沿原任务重新交付,不重新生成;完整字节可离线恢复,多产物中断复用已有作品。见 [同步交付恢复](docs/SYNC_DELIVERY_RECOVERY.md)。
14
+
15
+ ### Changed
16
+
17
+ - 音乐协议选项采用通用名称“Chat Completions(内嵌音频)”,保留现有配置标识。
18
+ - 媒体编排按创作目标选择步骤,配乐保持可选;UI验收补充失败结论,移除固定调用次数限制。宿主诊断包含16工具与三项Skill。
19
+ - 用户文档补齐音乐配置、任务恢复与三项Skill,统一安装和升级说明;个人网关实验与审计记录保留在内部资料中。
20
+
21
+ ### Fixed
22
+
23
+ - 提示词再次优化失败或取消时保留旧预览和手工编辑,成功迟到不覆盖等待期间的新编辑。单片段忠实整理无需模型调用,预览可查看实际指令。
24
+ - 改写使用正文交付协议,拒绝可识别的新增JSON外壳、外层围栏或整体序列化字符串,保留原稿中的合法格式。
25
+ - 显示文本模型选择原因;无效显式路由、元数据查询失败或明确缺少文本输入能力在生成前拒绝。
26
+ - 配置异常与被忽略的字段给出明确反馈。“仅恢复默认指令”保留模型、生成参数和入口开关;恢复全部设置说明完整范围,读取异常时保护原文件。见 [提示词优化](docs/PROMPT_OPTIMIZER.md)。
27
+
28
+ ### Security
29
+
30
+ - 交付缓存仅保存在私有数据根,签名URL和媒体正文不进入公开Task/Artifact响应或普通日志;缓存随显式任务隔离和恢复管理。
8
31
 
9
32
  ## [0.2.2] - 2026-10-11
10
33
 
@@ -288,7 +311,8 @@ Iris 0.2.0 要求 Node.js ≥ 22.0.0,DSH 插件仅支持 `0.2.0-rc.2`;旧宿
288
311
  - 上传采用流式限额、临时 `.part` 文件和原子落盘,失败时会清理未完成文件。
289
312
  - 修改状态的路由拒绝明确的跨站请求,媒体文件继续使用随机能力令牌访问。
290
313
 
291
- [Unreleased]: https://github.com/mokuyoaxis/dsh-iris/compare/v0.2.2...HEAD
314
+ [Unreleased]: https://github.com/mokuyoaxis/dsh-iris/compare/v0.2.3...HEAD
315
+ [0.2.3]: https://github.com/mokuyoaxis/dsh-iris/compare/v0.2.2...v0.2.3
292
316
  [0.2.2]: https://github.com/mokuyoaxis/dsh-iris/compare/v0.2.1...v0.2.2
293
317
  [0.2.1]: https://github.com/mokuyoaxis/dsh-iris/compare/v0.2.0...v0.2.1
294
318
  [0.2.0]: https://github.com/mokuyoaxis/dsh-iris/compare/v0.1.4...v0.2.0