dsh-image-gen 0.2.0 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.en.md +171 -171
- package/README.md +181 -187
- package/README.zh-CN.md +171 -171
- package/lib/index.js +111 -32
- package/lib/types/dashscope.d.ts +2 -2
- package/lib/types/google.d.ts +2 -2
- package/lib/types/openai-compatible.d.ts +1 -1
- package/lib/types/reference-image.d.ts +17 -0
- package/lib/types/seedream.d.ts +2 -2
- package/package.json +131 -131
package/README.zh-CN.md
CHANGED
|
@@ -1,171 +1,171 @@
|
|
|
1
|
-
<div align="center">
|
|
2
|
-
|
|
3
|
-
<img src="docs/assets/hero-poster.webp" alt="dsh-image-gen 宣传海报" width="100%" style="max-width: 860px; border-radius: 12px;" />
|
|
4
|
-
|
|
5
|
-
<br /><br />
|
|
6
|
-
|
|
7
|
-
# 🎨 dsh-image-gen
|
|
8
|
-
|
|
9
|
-
**让 DeepSeek Harness 像 ChatGPT 一样在对话中直接生成图片,支持画廊汇总,全屏预览、快捷复制与一键下载。**
|
|
10
|
-
|
|
11
|
-
[](https://www.npmjs.com/package/dsh-image-gen)
|
|
12
|
-
[](https://github.com/deepseek-ai)
|
|
13
|
-
[](https://www.npmjs.com/package/dsh-image-gen)
|
|
14
|
-
[](LICENSE)
|
|
15
|
-
|
|
16
|
-
[English](README.en.md) | **简体中文**
|
|
17
|
-
|
|
18
|
-
<br />
|
|
19
|
-
|
|
20
|
-
<p align="center">💬 <b>直接对你的 DeepSeek Harness Agent 发送以下提示词:</b></p>
|
|
21
|
-
|
|
22
|
-
```text
|
|
23
|
-
帮我安装生图插件,执行命令:pnpm dsh plugin --profile web add dsh-image-gen@latest
|
|
24
|
-
```
|
|
25
|
-
|
|
26
|
-
<p align="center"><sub>(也可以手动在终端执行:<code>pnpm dsh plugin --profile web add dsh-image-gen@latest</code>)</sub></p>
|
|
27
|
-
|
|
28
|
-
<br />
|
|
29
|
-
|
|
30
|
-
<p align="center">安装完成后,在 DSH 设置中填入自己的 API Key,就可以直接对 Agent 说:</p>
|
|
31
|
-
|
|
32
|
-
```text
|
|
33
|
-
帮我画一张雨夜霓虹街头的赛博朋克猫咪。
|
|
34
|
-
```
|
|
35
|
-
|
|
36
|
-
<p align="center">Agent 会自动调用 <code>generate_image</code>,生成图片并直接显示在当前对话中。</p>
|
|
37
|
-
|
|
38
|
-
<br />
|
|
39
|
-
|
|
40
|
-
<img src="docs/assets/chat-preview.png" alt="对话生图效果预览" width="820" />
|
|
41
|
-
|
|
42
|
-
</div>
|
|
43
|
-
|
|
44
|
-
---
|
|
45
|
-
|
|
46
|
-
## 💡 它解决什么问题?
|
|
47
|
-
|
|
48
|
-
**`dsh-image-gen` 是专为 DeepSeek Harness (DSH) 打造的开源图像生成插件。**
|
|
49
|
-
|
|
50
|
-
DeepSeek Harness 已经可以让 Agent 调用不同工具完成任务,本项目为它补上了原生的**多模态生图能力**:
|
|
51
|
-
|
|
52
|
-
```mermaid
|
|
53
|
-
graph LR
|
|
54
|
-
A[用户 Prompt] --> B[DeepSeek Harness Agent]
|
|
55
|
-
B --> C[generate_image 工具]
|
|
56
|
-
C --> D[Gemini / OpenAI / Seedream / DashScope]
|
|
57
|
-
D --> E[图片数据]
|
|
58
|
-
E --> F[当前 Conversation 对话流]
|
|
59
|
-
```
|
|
60
|
-
|
|
61
|
-
---
|
|
62
|
-
|
|
63
|
-
## 🚀 快速安装与使用
|
|
64
|
-
|
|
65
|
-
### 1. 安装插件
|
|
66
|
-
|
|
67
|
-
在你的 DeepSeek Harness 项目根目录下运行:
|
|
68
|
-
|
|
69
|
-
```bash
|
|
70
|
-
# 推荐方式:安装或升级到最新版本
|
|
71
|
-
pnpm dsh plugin --profile web add dsh-image-gen@latest
|
|
72
|
-
|
|
73
|
-
# 若已将 dsh 安装为系统全局命令:
|
|
74
|
-
dsh plugin --profile web add dsh-image-gen@latest
|
|
75
|
-
```
|
|
76
|
-
|
|
77
|
-
<details>
|
|
78
|
-
<summary><b>🛠️ 其他安装方式(Git 仓库直装 / 本地调试)</b></summary>
|
|
79
|
-
|
|
80
|
-
```bash
|
|
81
|
-
# 方式 B:从 GitHub 仓库直接安装最新代码
|
|
82
|
-
pnpm dsh plugin --profile web add git+https://github.com/shanliuling/dsh-image-gen.git
|
|
83
|
-
|
|
84
|
-
# 方式 C:本地克隆源码开发安装
|
|
85
|
-
git clone https://github.com/shanliuling/dsh-image-gen.git
|
|
86
|
-
pnpm dsh plugin --profile web add ./dsh-image-gen
|
|
87
|
-
```
|
|
88
|
-
|
|
89
|
-
</details>
|
|
90
|
-
|
|
91
|
-
### 2. 配置 API Key 与工作区设置
|
|
92
|
-
|
|
93
|
-
打开 DSH Web 页面(默认 `http://localhost:3080`):
|
|
94
|
-
|
|
95
|
-
1. 进入 **Settings → Plugins → Image generation**。
|
|
96
|
-
2. 选择 Provider,填写 API Key。
|
|
97
|
-
3. 可按需开启 **保存到工作区**(默认开启)并自定义子目录,点击 **保存** 即可。
|
|
98
|
-
|
|
99
|
-
<div align="center">
|
|
100
|
-
<img src="docs/assets/settings-preview.png?v=0.1.7" alt="设置面板预览" width="720" />
|
|
101
|
-
</div>
|
|
102
|
-
|
|
103
|
-
### 3. 开始对话生图
|
|
104
|
-
|
|
105
|
-
现在直接在聊天框输入:
|
|
106
|
-
|
|
107
|
-
```text
|
|
108
|
-
生成一张极简主义的现代建筑客厅插画。
|
|
109
|
-
```
|
|
110
|
-
|
|
111
|
-
当前 Agent 就会自动调用 `generate_image` 工具并在对话流中返回图片。
|
|
112
|
-
|
|
113
|
-
### 4. 查看原生生图画廊
|
|
114
|
-
|
|
115
|
-
点击会话顶栏的 **`[画廊]`** Tab,即可集中查看和搜索所有对话生成的历史图片:
|
|
116
|
-
|
|
117
|
-
<div align="center">
|
|
118
|
-
<img src="docs/assets/gallery-preview.png" alt="原生生图画廊预览" width="820" />
|
|
119
|
-
</div>
|
|
120
|
-
|
|
121
|
-
---
|
|
122
|
-
|
|
123
|
-
## ✨ 主要能力
|
|
124
|
-
|
|
125
|
-
- 💬 **对话中直接生图**:不需要切换到其他网站,也不需要手动复制 Prompt,直接告诉 Agent 你想画什么即可。
|
|
126
|
-
- 🖼️ **历史生图画廊**:顶栏自带「画廊」Tab,自动汇总所有历史生成的图片,支持关键词搜索、厂商筛选、单张删除(带防误触确认与持久化墓碑)与一键复制/下载。
|
|
127
|
-
- 🔍 **交互式图片工具**:支持点击全屏大图预览、一键复制图片到剪贴板、本地下载与新标签页打开。
|
|
128
|
-
- 🎨 **多 Provider 支持**:支持 Google Gemini、OpenAI Images、OpenAI Compatible API、ByteDance Seedream / 火山方舟以及阿里云 DashScope(通义万相 / Qwen-Image)。Provider、模型和 Endpoint 均可在设置中自由定制。
|
|
129
|
-
- 🔑 **BYOK (自带 Key)**:插件使用你自己的 API Key。API Key 通过 DeepSeek Harness 的 `credentials` 服务管理,采用写保护隔离,不需要写进项目源码或配置文件,前端不存明文。
|
|
130
|
-
- 🖼️ **图片跟随会话保存**:生成结果会接入 DeepSeek Harness 的 Attachment / Conversation 体系,重新打开历史会话后,仍然可以看到之前生成的图片。
|
|
131
|
-
- 💾 **生成图片自动落盘到工作区**:默认在每次生成后把图片文件保存到当前会话工作区,工具结果会返回文件的绝对路径;可在设置中关闭或修改保存目录。
|
|
132
|
-
- ⚙️ **原生设置界面**:Provider、API Key、模型和 Endpoint 都可以直接在 DSH Web 设置中修改,不需要手动编辑配置文件。
|
|
133
|
-
|
|
134
|
-
---
|
|
135
|
-
|
|
136
|
-
## 📦 支持的 Provider
|
|
137
|
-
|
|
138
|
-
| Provider | 默认模型 | 默认 Endpoint / Base URL |
|
|
139
|
-
| :--- | :--- | :--- |
|
|
140
|
-
| **Google Gemini** | `gemini-3.1-flash-image` | `https://generativelanguage.googleapis.com/v1beta/interactions` |
|
|
141
|
-
| **OpenAI Images** | `gpt-image-2` | `https://api.openai.com/v1` |
|
|
142
|
-
| **OpenAI Compatible** | 自定义 | 自定义 Base URL |
|
|
143
|
-
| **ByteDance Seedream / 火山方舟** | `doubao-seedream-5-0-260128` | `https://ark.cn-beijing.volces.com/api/v3` |
|
|
144
|
-
| **Aliyun DashScope / 通义万相** | `wanx2.1-t2i-turbo` | `https://dashscope.aliyuncs.com/api/v1` |
|
|
145
|
-
|
|
146
|
-
---
|
|
147
|
-
|
|
148
|
-
## 🛠️ 本地开发 (Development)
|
|
149
|
-
|
|
150
|
-
```bash
|
|
151
|
-
# 克隆仓库
|
|
152
|
-
git clone https://github.com/shanliuling/dsh-image-gen.git
|
|
153
|
-
cd dsh-image-gen
|
|
154
|
-
|
|
155
|
-
# 安装依赖与构建
|
|
156
|
-
pnpm install
|
|
157
|
-
pnpm run typecheck
|
|
158
|
-
pnpm run test
|
|
159
|
-
pnpm run build
|
|
160
|
-
|
|
161
|
-
# 检查 npm 打包内容
|
|
162
|
-
pnpm run pack:check
|
|
163
|
-
```
|
|
164
|
-
|
|
165
|
-
---
|
|
166
|
-
|
|
167
|
-
## 📄 开源协议 (License)
|
|
168
|
-
|
|
169
|
-
本项目基于 [MIT License](LICENSE) 开源。
|
|
170
|
-
|
|
171
|
-
如果这个插件对你有用,欢迎点一个 ⭐️ **Star** 支持!
|
|
1
|
+
<div align="center">
|
|
2
|
+
|
|
3
|
+
<img src="docs/assets/hero-poster.webp" alt="dsh-image-gen 宣传海报" width="100%" style="max-width: 860px; border-radius: 12px;" />
|
|
4
|
+
|
|
5
|
+
<br /><br />
|
|
6
|
+
|
|
7
|
+
# 🎨 dsh-image-gen
|
|
8
|
+
|
|
9
|
+
**让 DeepSeek Harness 像 ChatGPT 一样在对话中直接生成图片,支持画廊汇总,全屏预览、快捷复制与一键下载。**
|
|
10
|
+
|
|
11
|
+
[](https://www.npmjs.com/package/dsh-image-gen)
|
|
12
|
+
[](https://github.com/deepseek-ai)
|
|
13
|
+
[](https://www.npmjs.com/package/dsh-image-gen)
|
|
14
|
+
[](LICENSE)
|
|
15
|
+
|
|
16
|
+
[English](README.en.md) | **简体中文**
|
|
17
|
+
|
|
18
|
+
<br />
|
|
19
|
+
|
|
20
|
+
<p align="center">💬 <b>直接对你的 DeepSeek Harness Agent 发送以下提示词:</b></p>
|
|
21
|
+
|
|
22
|
+
```text
|
|
23
|
+
帮我安装生图插件,执行命令:pnpm dsh plugin --profile web add dsh-image-gen@latest
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
<p align="center"><sub>(也可以手动在终端执行:<code>pnpm dsh plugin --profile web add dsh-image-gen@latest</code>)</sub></p>
|
|
27
|
+
|
|
28
|
+
<br />
|
|
29
|
+
|
|
30
|
+
<p align="center">安装完成后,在 DSH 设置中填入自己的 API Key,就可以直接对 Agent 说:</p>
|
|
31
|
+
|
|
32
|
+
```text
|
|
33
|
+
帮我画一张雨夜霓虹街头的赛博朋克猫咪。
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
<p align="center">Agent 会自动调用 <code>generate_image</code>,生成图片并直接显示在当前对话中。</p>
|
|
37
|
+
|
|
38
|
+
<br />
|
|
39
|
+
|
|
40
|
+
<img src="docs/assets/chat-preview.png" alt="对话生图效果预览" width="820" />
|
|
41
|
+
|
|
42
|
+
</div>
|
|
43
|
+
|
|
44
|
+
---
|
|
45
|
+
|
|
46
|
+
## 💡 它解决什么问题?
|
|
47
|
+
|
|
48
|
+
**`dsh-image-gen` 是专为 DeepSeek Harness (DSH) 打造的开源图像生成插件。**
|
|
49
|
+
|
|
50
|
+
DeepSeek Harness 已经可以让 Agent 调用不同工具完成任务,本项目为它补上了原生的**多模态生图能力**:
|
|
51
|
+
|
|
52
|
+
```mermaid
|
|
53
|
+
graph LR
|
|
54
|
+
A[用户 Prompt] --> B[DeepSeek Harness Agent]
|
|
55
|
+
B --> C[generate_image 工具]
|
|
56
|
+
C --> D[Gemini / OpenAI / Seedream / DashScope]
|
|
57
|
+
D --> E[图片数据]
|
|
58
|
+
E --> F[当前 Conversation 对话流]
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
---
|
|
62
|
+
|
|
63
|
+
## 🚀 快速安装与使用
|
|
64
|
+
|
|
65
|
+
### 1. 安装插件
|
|
66
|
+
|
|
67
|
+
在你的 DeepSeek Harness 项目根目录下运行:
|
|
68
|
+
|
|
69
|
+
```bash
|
|
70
|
+
# 推荐方式:安装或升级到最新版本
|
|
71
|
+
pnpm dsh plugin --profile web add dsh-image-gen@latest
|
|
72
|
+
|
|
73
|
+
# 若已将 dsh 安装为系统全局命令:
|
|
74
|
+
dsh plugin --profile web add dsh-image-gen@latest
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
<details>
|
|
78
|
+
<summary><b>🛠️ 其他安装方式(Git 仓库直装 / 本地调试)</b></summary>
|
|
79
|
+
|
|
80
|
+
```bash
|
|
81
|
+
# 方式 B:从 GitHub 仓库直接安装最新代码
|
|
82
|
+
pnpm dsh plugin --profile web add git+https://github.com/shanliuling/dsh-image-gen.git
|
|
83
|
+
|
|
84
|
+
# 方式 C:本地克隆源码开发安装
|
|
85
|
+
git clone https://github.com/shanliuling/dsh-image-gen.git
|
|
86
|
+
pnpm dsh plugin --profile web add ./dsh-image-gen
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
</details>
|
|
90
|
+
|
|
91
|
+
### 2. 配置 API Key 与工作区设置
|
|
92
|
+
|
|
93
|
+
打开 DSH Web 页面(默认 `http://localhost:3080`):
|
|
94
|
+
|
|
95
|
+
1. 进入 **Settings → Plugins → Image generation**。
|
|
96
|
+
2. 选择 Provider,填写 API Key。
|
|
97
|
+
3. 可按需开启 **保存到工作区**(默认开启)并自定义子目录,点击 **保存** 即可。
|
|
98
|
+
|
|
99
|
+
<div align="center">
|
|
100
|
+
<img src="docs/assets/settings-preview.png?v=0.1.7" alt="设置面板预览" width="720" />
|
|
101
|
+
</div>
|
|
102
|
+
|
|
103
|
+
### 3. 开始对话生图
|
|
104
|
+
|
|
105
|
+
现在直接在聊天框输入:
|
|
106
|
+
|
|
107
|
+
```text
|
|
108
|
+
生成一张极简主义的现代建筑客厅插画。
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
当前 Agent 就会自动调用 `generate_image` 工具并在对话流中返回图片。
|
|
112
|
+
|
|
113
|
+
### 4. 查看原生生图画廊
|
|
114
|
+
|
|
115
|
+
点击会话顶栏的 **`[画廊]`** Tab,即可集中查看和搜索所有对话生成的历史图片:
|
|
116
|
+
|
|
117
|
+
<div align="center">
|
|
118
|
+
<img src="docs/assets/gallery-preview.png" alt="原生生图画廊预览" width="820" />
|
|
119
|
+
</div>
|
|
120
|
+
|
|
121
|
+
---
|
|
122
|
+
|
|
123
|
+
## ✨ 主要能力
|
|
124
|
+
|
|
125
|
+
- 💬 **对话中直接生图**:不需要切换到其他网站,也不需要手动复制 Prompt,直接告诉 Agent 你想画什么即可。
|
|
126
|
+
- 🖼️ **历史生图画廊**:顶栏自带「画廊」Tab,自动汇总所有历史生成的图片,支持关键词搜索、厂商筛选、单张删除(带防误触确认与持久化墓碑)与一键复制/下载。
|
|
127
|
+
- 🔍 **交互式图片工具**:支持点击全屏大图预览、一键复制图片到剪贴板、本地下载与新标签页打开。
|
|
128
|
+
- 🎨 **多 Provider 支持**:支持 Google Gemini、OpenAI Images、OpenAI Compatible API、ByteDance Seedream / 火山方舟以及阿里云 DashScope(通义万相 / Qwen-Image)。Provider、模型和 Endpoint 均可在设置中自由定制。
|
|
129
|
+
- 🔑 **BYOK (自带 Key)**:插件使用你自己的 API Key。API Key 通过 DeepSeek Harness 的 `credentials` 服务管理,采用写保护隔离,不需要写进项目源码或配置文件,前端不存明文。
|
|
130
|
+
- 🖼️ **图片跟随会话保存**:生成结果会接入 DeepSeek Harness 的 Attachment / Conversation 体系,重新打开历史会话后,仍然可以看到之前生成的图片。
|
|
131
|
+
- 💾 **生成图片自动落盘到工作区**:默认在每次生成后把图片文件保存到当前会话工作区,工具结果会返回文件的绝对路径;可在设置中关闭或修改保存目录。
|
|
132
|
+
- ⚙️ **原生设置界面**:Provider、API Key、模型和 Endpoint 都可以直接在 DSH Web 设置中修改,不需要手动编辑配置文件。
|
|
133
|
+
|
|
134
|
+
---
|
|
135
|
+
|
|
136
|
+
## 📦 支持的 Provider
|
|
137
|
+
|
|
138
|
+
| Provider | 默认模型 | 默认 Endpoint / Base URL |
|
|
139
|
+
| :--- | :--- | :--- |
|
|
140
|
+
| **Google Gemini** | `gemini-3.1-flash-image` | `https://generativelanguage.googleapis.com/v1beta/interactions` |
|
|
141
|
+
| **OpenAI Images** | `gpt-image-2` | `https://api.openai.com/v1` |
|
|
142
|
+
| **OpenAI Compatible** | 自定义 | 自定义 Base URL |
|
|
143
|
+
| **ByteDance Seedream / 火山方舟** | `doubao-seedream-5-0-260128` | `https://ark.cn-beijing.volces.com/api/v3` |
|
|
144
|
+
| **Aliyun DashScope / 通义万相** | `wanx2.1-t2i-turbo` | `https://dashscope.aliyuncs.com/api/v1` |
|
|
145
|
+
|
|
146
|
+
---
|
|
147
|
+
|
|
148
|
+
## 🛠️ 本地开发 (Development)
|
|
149
|
+
|
|
150
|
+
```bash
|
|
151
|
+
# 克隆仓库
|
|
152
|
+
git clone https://github.com/shanliuling/dsh-image-gen.git
|
|
153
|
+
cd dsh-image-gen
|
|
154
|
+
|
|
155
|
+
# 安装依赖与构建
|
|
156
|
+
pnpm install
|
|
157
|
+
pnpm run typecheck
|
|
158
|
+
pnpm run test
|
|
159
|
+
pnpm run build
|
|
160
|
+
|
|
161
|
+
# 检查 npm 打包内容
|
|
162
|
+
pnpm run pack:check
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
---
|
|
166
|
+
|
|
167
|
+
## 📄 开源协议 (License)
|
|
168
|
+
|
|
169
|
+
本项目基于 [MIT License](LICENSE) 开源。
|
|
170
|
+
|
|
171
|
+
如果这个插件对你有用,欢迎点一个 ⭐️ **Star** 支持!
|
package/lib/index.js
CHANGED
|
@@ -116,7 +116,7 @@ async function editDashScopeImage(options) {
|
|
|
116
116
|
model: options.model,
|
|
117
117
|
input: { messages: [{
|
|
118
118
|
role: "user",
|
|
119
|
-
content: [{ image: toDataUrl$1(
|
|
119
|
+
content: [...options.sourceImages.map((sourceImage) => ({ image: toDataUrl$1(sourceImage) })), { text: options.prompt }]
|
|
120
120
|
}] },
|
|
121
121
|
parameters: {
|
|
122
122
|
prompt_extend: true,
|
|
@@ -197,11 +197,11 @@ function editGoogleImage(input) {
|
|
|
197
197
|
interactionInput: [{
|
|
198
198
|
type: "text",
|
|
199
199
|
text: input.prompt
|
|
200
|
-
}, {
|
|
200
|
+
}, ...input.sourceImages.map((sourceImage) => ({
|
|
201
201
|
type: "image",
|
|
202
|
-
mime_type:
|
|
203
|
-
data: Buffer.from(
|
|
204
|
-
}]
|
|
202
|
+
mime_type: sourceImage.mediaType,
|
|
203
|
+
data: Buffer.from(sourceImage.data).toString("base64")
|
|
204
|
+
}))]
|
|
205
205
|
});
|
|
206
206
|
}
|
|
207
207
|
/** Shared Google request, response parsing, decoding, and size enforcement. */
|
|
@@ -309,30 +309,67 @@ async function readBoundedText$2(response, maxBytes, label) {
|
|
|
309
309
|
//#region lib/types/reference-image.js
|
|
310
310
|
/** Latest-DSH compatibility boundary for resolving conversation image references. */
|
|
311
311
|
/**
|
|
312
|
-
* Resolve
|
|
313
|
-
*
|
|
314
|
-
*
|
|
315
|
-
* messages or how nested model-facing content stores durable image refs.
|
|
312
|
+
* Resolve one or more edit references while keeping every DSH-specific detail
|
|
313
|
+
* behind this compatibility boundary. Explicit selectors preserve caller order;
|
|
314
|
+
* without selectors, all images in the newest image-bearing message are used.
|
|
316
315
|
*/
|
|
317
|
-
async function
|
|
318
|
-
|
|
319
|
-
|
|
320
|
-
|
|
316
|
+
async function resolveReferenceImages(input) {
|
|
317
|
+
const sourceAttachmentIds = mergeSelectors({
|
|
318
|
+
single: input.sourceAttachmentId,
|
|
319
|
+
multiple: input.sourceAttachmentIds,
|
|
320
|
+
singleName: "source_attachment_id",
|
|
321
|
+
multipleName: "source_attachment_ids",
|
|
322
|
+
equal: attachmentIdsEqual
|
|
323
|
+
});
|
|
324
|
+
const sourcePaths = mergeSelectors({
|
|
325
|
+
single: input.sourcePath,
|
|
326
|
+
multiple: input.sourcePaths,
|
|
327
|
+
singleName: "source_path",
|
|
328
|
+
multipleName: "source_paths",
|
|
329
|
+
equal: (left, right) => left.trim() === right.trim()
|
|
330
|
+
});
|
|
331
|
+
if (sourceAttachmentIds !== void 0 && sourcePaths !== void 0) throw new Error("edit_image accepts only one of source_attachment_id, source_attachment_ids, source_path, or source_paths");
|
|
332
|
+
if (sourcePaths !== void 0) return Promise.all(sourcePaths.map((sourcePath) => readWorkspaceReferenceImage({
|
|
333
|
+
sourcePath,
|
|
321
334
|
...input.agent?.session.header?.cwd === void 0 ? {} : { workspaceRoot: input.agent.session.header.cwd },
|
|
322
335
|
...input.maxBytes === void 0 ? {} : { maxBytes: input.maxBytes },
|
|
323
336
|
signal: input.signal
|
|
324
|
-
});
|
|
337
|
+
})));
|
|
325
338
|
if (input.agent === void 0) throw new Error("edit_image requires an active DSH agent session to resolve a reference image");
|
|
326
|
-
const
|
|
327
|
-
if (
|
|
328
|
-
if (
|
|
339
|
+
const refs = findReferenceImages(input.agent.session.deriveMessages(), sourceAttachmentIds);
|
|
340
|
+
if (refs.length === 0) {
|
|
341
|
+
if (sourceAttachmentIds !== void 0) throw new Error(`edit_image could not find image attachment ${sourceAttachmentIds[0]} in the current conversation`);
|
|
329
342
|
throw new Error("edit_image requires an image in the current conversation; upload or generate an image first");
|
|
330
343
|
}
|
|
331
|
-
|
|
332
|
-
|
|
333
|
-
|
|
334
|
-
|
|
335
|
-
|
|
344
|
+
if (sourceAttachmentIds !== void 0 && refs.length !== sourceAttachmentIds.length) {
|
|
345
|
+
const missing = sourceAttachmentIds.find((id) => !refs.some((ref) => attachmentIdsEqual(String(ref.attachmentId), id)));
|
|
346
|
+
throw new Error(`edit_image could not find image attachment ${missing ?? "unknown"} in the current conversation`);
|
|
347
|
+
}
|
|
348
|
+
return Promise.all(refs.map(async (ref) => {
|
|
349
|
+
const stored = await input.attachments.readImage(ref, input.signal);
|
|
350
|
+
if (input.maxBytes !== void 0 && stored.data.byteLength > input.maxBytes) throw new Error(`edit_image source image is too large (${stored.data.byteLength} bytes; maximum ${input.maxBytes})`);
|
|
351
|
+
return {
|
|
352
|
+
data: stored.data,
|
|
353
|
+
mediaType: stored.ref.mediaType
|
|
354
|
+
};
|
|
355
|
+
}));
|
|
356
|
+
}
|
|
357
|
+
/** Find images in caller order, or every image in the newest image-bearing message. */
|
|
358
|
+
function findReferenceImages(messages, sourceAttachmentIds) {
|
|
359
|
+
if (sourceAttachmentIds !== void 0) return sourceAttachmentIds.flatMap((id) => {
|
|
360
|
+
const ref = findReferenceImage(messages, id);
|
|
361
|
+
return ref === void 0 ? [] : [ref];
|
|
362
|
+
});
|
|
363
|
+
const latestHumanMessage = [...messages].reverse().find((message) => message.source?.kind === "user");
|
|
364
|
+
if (latestHumanMessage !== void 0) {
|
|
365
|
+
const refs = collectInBlocks(latestHumanMessage.content);
|
|
366
|
+
if (refs.length > 0) return refs;
|
|
367
|
+
}
|
|
368
|
+
for (let index = messages.length - 1; index >= 0; index -= 1) {
|
|
369
|
+
const refs = collectInBlocks(messages[index]?.content ?? []);
|
|
370
|
+
if (refs.length > 0) return refs;
|
|
371
|
+
}
|
|
372
|
+
return [];
|
|
336
373
|
}
|
|
337
374
|
/**
|
|
338
375
|
* Read an explicitly named workspace image without exposing filesystem or DSH
|
|
@@ -397,7 +434,7 @@ function findInBlocks(blocks, sourceAttachmentId) {
|
|
|
397
434
|
const block = blocks[index];
|
|
398
435
|
if (block === void 0) continue;
|
|
399
436
|
if (block.type === "image") {
|
|
400
|
-
if (sourceAttachmentId === void 0 || String(block.attachment.attachmentId)
|
|
437
|
+
if (sourceAttachmentId === void 0 || attachmentIdsEqual(String(block.attachment.attachmentId), sourceAttachmentId)) return block.attachment;
|
|
401
438
|
continue;
|
|
402
439
|
}
|
|
403
440
|
if (block.type === "tool-result") {
|
|
@@ -417,6 +454,33 @@ function parseDimensions(value) {
|
|
|
417
454
|
function imageMediaType$2(value) {
|
|
418
455
|
return value === "image/png" || value === "image/jpeg" || value === "image/webp" || value === "image/gif";
|
|
419
456
|
}
|
|
457
|
+
function collectInBlocks(blocks) {
|
|
458
|
+
const refs = [];
|
|
459
|
+
for (const block of blocks) {
|
|
460
|
+
if (block.type === "image") refs.push(block.attachment);
|
|
461
|
+
if (block.type === "tool-result") refs.push(...collectInBlocks(block.content));
|
|
462
|
+
}
|
|
463
|
+
return refs;
|
|
464
|
+
}
|
|
465
|
+
function attachmentIdsEqual(actual, requested) {
|
|
466
|
+
if (actual === requested) return true;
|
|
467
|
+
const actualDigest = sha256Digest(actual);
|
|
468
|
+
const requestedDigest = sha256Digest(requested);
|
|
469
|
+
return actualDigest !== void 0 && actualDigest === requestedDigest;
|
|
470
|
+
}
|
|
471
|
+
function sha256Digest(value) {
|
|
472
|
+
return /^(?:sha256:)?([0-9a-f]{64})$/i.exec(value.trim())?.[1]?.toLowerCase();
|
|
473
|
+
}
|
|
474
|
+
function mergeSelectors(input) {
|
|
475
|
+
if (input.multiple === void 0) return input.single === void 0 ? void 0 : [input.single];
|
|
476
|
+
if (input.multiple.length === 0) {
|
|
477
|
+
if (input.single !== void 0) return [input.single];
|
|
478
|
+
throw new Error(`edit_image ${input.multipleName} must not be empty`);
|
|
479
|
+
}
|
|
480
|
+
const single = input.single;
|
|
481
|
+
if (single !== void 0 && !input.multiple.some((value) => input.equal(single, value))) throw new Error(`edit_image ${input.singleName} must also appear in ${input.multipleName} when both are provided`);
|
|
482
|
+
return input.multiple;
|
|
483
|
+
}
|
|
420
484
|
function detectImageMediaType(data) {
|
|
421
485
|
if (startsWith(data, [
|
|
422
486
|
137,
|
|
@@ -538,8 +602,11 @@ async function generateOpenAICompatibleImage(input) {
|
|
|
538
602
|
}
|
|
539
603
|
async function editOpenAICompatibleImage(input) {
|
|
540
604
|
const form = new FormData();
|
|
541
|
-
const
|
|
542
|
-
|
|
605
|
+
const imageField = input.sourceImages.length === 1 ? "image" : "image[]";
|
|
606
|
+
input.sourceImages.forEach((sourceImage, index) => {
|
|
607
|
+
const uploadBytes = new Uint8Array(sourceImage.data);
|
|
608
|
+
form.append(imageField, new Blob([uploadBytes], { type: sourceImage.mediaType }), `reference-${index + 1}.${extensionOf(sourceImage.mediaType)}`);
|
|
609
|
+
});
|
|
543
610
|
form.append("prompt", input.prompt);
|
|
544
611
|
form.append("model", input.model);
|
|
545
612
|
if (input.size !== void 0 && input.size.length > 0) form.append("size", input.size);
|
|
@@ -684,7 +751,7 @@ async function editSeedreamImage(input) {
|
|
|
684
751
|
body: JSON.stringify({
|
|
685
752
|
model: input.model,
|
|
686
753
|
prompt: input.prompt,
|
|
687
|
-
image:
|
|
754
|
+
image: input.sourceImages.map(toDataUrl),
|
|
688
755
|
...input.size === void 0 || input.size.length === 0 ? {} : { size: input.size },
|
|
689
756
|
response_format: "b64_json"
|
|
690
757
|
})
|
|
@@ -1005,7 +1072,7 @@ function apply(ctx, config = {}) {
|
|
|
1005
1072
|
}));
|
|
1006
1073
|
ctx.tools.register(defineTool({
|
|
1007
1074
|
name: "edit_image",
|
|
1008
|
-
description: "Edit or restyle
|
|
1075
|
+
description: "Edit, combine, or restyle existing images with the configured provider. Images attached inline to the latest human message are already readable DSH attachments even when no workspace file exists. In that case, call edit_image immediately with prompt only; NEVER call read_image, glob, or shell to locate them, and NEVER invent @ paths. All inline images will be used in upload order. For specific older conversation images use source_attachment_id or source_attachment_ids; both canonical sha256: IDs and full bare SHA-256 digests are accepted. For files the user explicitly names in the workspace use source_path or source_paths. Provide exactly one selector field. Without a selector, images from the latest human message take priority; only when that message has no images does editing fall back to the newest conversation image.",
|
|
1009
1076
|
parameters: {
|
|
1010
1077
|
prompt: {
|
|
1011
1078
|
type: "string",
|
|
@@ -1016,10 +1083,20 @@ function apply(ctx, config = {}) {
|
|
|
1016
1083
|
type: "string",
|
|
1017
1084
|
description: "Optional attachment id of a specific image already present in the current conversation."
|
|
1018
1085
|
},
|
|
1086
|
+
source_attachment_ids: {
|
|
1087
|
+
type: "array",
|
|
1088
|
+
items: { type: "string" },
|
|
1089
|
+
description: "Optional ordered attachment ids of multiple images already present in the current conversation. Prompt references such as image 1 and image 2 follow this order."
|
|
1090
|
+
},
|
|
1019
1091
|
source_path: {
|
|
1020
1092
|
type: "string",
|
|
1021
1093
|
description: "Optional absolute or workspace-relative path of a specific image file inside the active session workspace. Prefer this when the user names a saved file."
|
|
1022
1094
|
},
|
|
1095
|
+
source_paths: {
|
|
1096
|
+
type: "array",
|
|
1097
|
+
items: { type: "string" },
|
|
1098
|
+
description: "Optional ordered absolute or workspace-relative paths of multiple image files inside the active session workspace."
|
|
1099
|
+
},
|
|
1023
1100
|
aspect_ratio: {
|
|
1024
1101
|
type: "string",
|
|
1025
1102
|
enum: [
|
|
@@ -1052,11 +1129,13 @@ function apply(ctx, config = {}) {
|
|
|
1052
1129
|
const active = resolveProvider(current());
|
|
1053
1130
|
const credential = await ctx.credentials.resolve(credentialRef(active.apiKeyEnv));
|
|
1054
1131
|
if (credential === void 0 || credential.value.length === 0) throw new Error(`edit_image requires the ${active.apiKeyEnv} credential; configure it in Settings > Plugins > Image generation.`);
|
|
1055
|
-
const
|
|
1132
|
+
const sourceImages = await resolveReferenceImages({
|
|
1056
1133
|
...exec.agent === void 0 ? {} : { agent: exec.agent },
|
|
1057
1134
|
attachments: ctx.attachments,
|
|
1058
1135
|
...typeof args.source_attachment_id === "string" ? { sourceAttachmentId: args.source_attachment_id } : {},
|
|
1136
|
+
...Array.isArray(args.source_attachment_ids) ? { sourceAttachmentIds: args.source_attachment_ids } : {},
|
|
1059
1137
|
...typeof args.source_path === "string" ? { sourcePath: args.source_path } : {},
|
|
1138
|
+
...Array.isArray(args.source_paths) ? { sourcePaths: args.source_paths } : {},
|
|
1060
1139
|
maxBytes: ctx.attachments.imageLimits.maxImageBytes,
|
|
1061
1140
|
signal: exec.signal
|
|
1062
1141
|
});
|
|
@@ -1068,7 +1147,7 @@ function apply(ctx, config = {}) {
|
|
|
1068
1147
|
endpoint: active.endpoint,
|
|
1069
1148
|
model: active.model,
|
|
1070
1149
|
prompt: args.prompt,
|
|
1071
|
-
|
|
1150
|
+
sourceImages,
|
|
1072
1151
|
aspectRatio,
|
|
1073
1152
|
imageSize,
|
|
1074
1153
|
maxBytes: ctx.attachments.imageLimits.maxImageBytes,
|
|
@@ -1081,7 +1160,7 @@ function apply(ctx, config = {}) {
|
|
|
1081
1160
|
baseURL: active.baseURL,
|
|
1082
1161
|
model: active.model,
|
|
1083
1162
|
prompt: args.prompt,
|
|
1084
|
-
|
|
1163
|
+
sourceImages,
|
|
1085
1164
|
size,
|
|
1086
1165
|
maxBytes: ctx.attachments.imageLimits.maxImageBytes,
|
|
1087
1166
|
signal: exec.signal
|
|
@@ -1091,7 +1170,7 @@ function apply(ctx, config = {}) {
|
|
|
1091
1170
|
baseURL: active.baseURL,
|
|
1092
1171
|
model: active.model,
|
|
1093
1172
|
prompt: args.prompt,
|
|
1094
|
-
|
|
1173
|
+
sourceImages,
|
|
1095
1174
|
size,
|
|
1096
1175
|
maxBytes: ctx.attachments.imageLimits.maxImageBytes,
|
|
1097
1176
|
signal: exec.signal
|
|
@@ -1101,7 +1180,7 @@ function apply(ctx, config = {}) {
|
|
|
1101
1180
|
endpoint: active.endpoint,
|
|
1102
1181
|
model: active.model,
|
|
1103
1182
|
prompt: args.prompt,
|
|
1104
|
-
|
|
1183
|
+
sourceImages,
|
|
1105
1184
|
size,
|
|
1106
1185
|
maxBytes: ctx.attachments.imageLimits.maxImageBytes,
|
|
1107
1186
|
signal: exec.signal
|
package/lib/types/dashscope.d.ts
CHANGED
|
@@ -10,10 +10,10 @@ export interface DashScopeImageOptions {
|
|
|
10
10
|
signal?: AbortSignal;
|
|
11
11
|
}
|
|
12
12
|
export interface DashScopeEditOptions extends DashScopeImageOptions {
|
|
13
|
-
|
|
13
|
+
sourceImages: Array<{
|
|
14
14
|
data: Uint8Array;
|
|
15
15
|
mediaType: ImageMediaType;
|
|
16
|
-
}
|
|
16
|
+
}>;
|
|
17
17
|
}
|
|
18
18
|
export declare function generateDashScopeImage(options: DashScopeImageOptions): Promise<{
|
|
19
19
|
data: Uint8Array;
|
package/lib/types/google.d.ts
CHANGED
|
@@ -22,9 +22,9 @@ export declare function generateGoogleImage(input: GoogleRequestBase & {
|
|
|
22
22
|
/** Send one native Google image-editing request using already-resolved bytes. */
|
|
23
23
|
export declare function editGoogleImage(input: GoogleRequestBase & {
|
|
24
24
|
prompt: string;
|
|
25
|
-
|
|
25
|
+
sourceImages: Array<{
|
|
26
26
|
data: Uint8Array;
|
|
27
27
|
mediaType: ImageMediaType;
|
|
28
|
-
}
|
|
28
|
+
}>;
|
|
29
29
|
}): Promise<GeneratedImage>;
|
|
30
30
|
export {};
|
|
@@ -23,7 +23,7 @@ export declare function editOpenAICompatibleImage(input: {
|
|
|
23
23
|
baseURL: string;
|
|
24
24
|
model: string;
|
|
25
25
|
prompt: string;
|
|
26
|
-
|
|
26
|
+
sourceImages: CompatibleReferenceImage[];
|
|
27
27
|
size?: string;
|
|
28
28
|
maxBytes: number;
|
|
29
29
|
signal: AbortSignal;
|