@anionex/dsh-vision-toolkit 0.1.30 → 0.1.31
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +16 -12
- package/README.zh.md +10 -6
- package/assets/skill/SKILL.md +2 -2
- package/assets/skill/UPSTREAM.json +5 -5
- package/assets/skill/references/restore-ui.md +6 -4
- package/docs/requirements-traceability/README.md +1 -1
- package/docs/requirements-traceability/README.zh.md +1 -1
- package/lib/exposure.js +18 -7
- package/lib/exposure.js.map +1 -1
- package/lib/index.js +4 -4
- package/lib/index.js.map +1 -1
- package/lib/skill.js +7 -7
- package/lib/skill.js.map +1 -1
- package/lib/types/exposure.d.ts +4 -0
- package/lib/types/exposure.d.ts.map +1 -1
- package/lib/types/index.d.ts +1 -1
- package/lib/types/skill.d.ts +4 -4
- package/lib/types/skill.d.ts.map +1 -1
- package/package.json +2 -1
- package/patches/vision-tools-dsh.patch +29 -13
- package/src/exposure.ts +22 -7
- package/src/index.ts +4 -4
- package/src/skill.ts +7 -7
package/README.md
CHANGED
|
@@ -6,9 +6,12 @@
|
|
|
6
6
|
|
|
7
7
|
# DSH Vision Toolkit
|
|
8
8
|
|
|
9
|
+
<a href="https://trendshift.io/repositories/149708?utm_source=trendshift-badge&utm_medium=badge&utm_campaign=badge-trendshift-149708" target="_blank" rel="noopener noreferrer"><img src="https://trendshift.io/api/badge/trendshift/repositories/149708/daily?language=TypeScript" alt="Anionex%2Fdsh-vision-toolkit | Trendshift" width="250" height="55"/></a>
|
|
10
|
+
|
|
9
11
|
[](https://dshfind.com/en/plugins/Anionex/dsh-vision-toolkit)
|
|
10
12
|
[](https://dshfind.com/en/plugins/Anionex/dsh-vision-toolkit)
|
|
11
13
|
[](https://www.theagenticleaderboard.com)
|
|
14
|
+
|
|
12
15
|
[](https://www.npmjs.com/package/@anionex/dsh-vision-toolkit)
|
|
13
16
|
[](LICENSE)
|
|
14
17
|
[](cordis.patch.yml)
|
|
@@ -26,14 +29,14 @@ If you use DeepSeek or another text-only model in DeepSeek Harness (DSH), you ma
|
|
|
26
29
|
|
|
27
30
|
🏆 This project is the first comprehensive vision-tool plugin in the DeepSeek Harness ecosystem: it was initiated before internal beta and built during the beta with reference to [`agent-vision-toolkit`](https://github.com/Anionex/agent-vision-toolkit).
|
|
28
31
|
|
|
29
|
-
> **Original work:** The system and division of responsibilities behind these visual tools, together with the `vision-
|
|
32
|
+
> **Original work:** The system and division of responsibilities behind these visual tools, together with the `vision-skills` Skill, were personally created and continuously refined by the author through long-term real-world use and repeated iteration.
|
|
30
33
|
|
|
31
34
|
## Highlights
|
|
32
35
|
|
|
33
36
|
- **Paste and use it immediately.** Paste an image in DSH Web and the text-only route switches to its `(Vision Toolkit)` variant without manual path copying or model changes.
|
|
34
37
|
- **A seamless image workflow.** Native thumbnails, session history, and workspace paths stay intact; Web can preview artifacts and Headless can continue using the same structured results.
|
|
35
38
|
- **One command to install.** The built-in free Gemini 3.7 Flash vision service is ready after installation, with no API key required.
|
|
36
|
-
- **Built-in free vision.** The shared service works immediately after installation
|
|
39
|
+
- **Built-in free vision.** The shared service works immediately after installation with a quota of **300 images per machine per day**.
|
|
37
40
|
- **Vision guided by intent.** The agent extracts evidence for the task at hand, such as “Where is the error?” or “Where is the button?”, instead of returning a generic caption.
|
|
38
41
|
- **A complete screenshot-to-verification loop.** Reference images, HTML screenshots, difference regions, and pixel comparison work together for UI restoration.
|
|
39
42
|
|
|
@@ -147,7 +150,7 @@ The first start prepares an isolated runtime: the plugin prefers a system Python
|
|
|
147
150
|
|
|
148
151
|
### 3. Paste an image and describe the outcome you want
|
|
149
152
|
|
|
150
|
-
Paste a screenshot into the conversation or place an image in the session workspace, then invoke `/vision-
|
|
153
|
+
Paste a screenshot into the conversation or place an image in the session workspace, then invoke `/vision-skills`. For example:
|
|
151
154
|
|
|
152
155
|
```text
|
|
153
156
|
Inspect this screenshot. Explain the error and tell me what to fix first.
|
|
@@ -197,7 +200,7 @@ The plugin keeps image understanding and deterministic local image processing in
|
|
|
197
200
|
|
|
198
201
|
```mermaid
|
|
199
202
|
flowchart LR
|
|
200
|
-
Image["Screenshot or local HTML"] --> Skill["vision-
|
|
203
|
+
Image["Screenshot or local HTML"] --> Skill["vision-skills Skill"]
|
|
201
204
|
Skill --> Agent["Text agent selects a task"]
|
|
202
205
|
Agent --> Vision["Use a vision model when image understanding is needed"]
|
|
203
206
|
Agent --> Local["Run crop, SVG, and pixel work locally"]
|
|
@@ -209,13 +212,13 @@ flowchart LR
|
|
|
209
212
|
|
|
210
213
|
The visual capabilities come from a packaged, pinned `agent-vision-toolkit` snapshot. The DSH plugin handles installation, session-scoped tool exposure, Credentials, path checks, cancellation, timeouts, result files, and Web presentation. The runtime never fetches upstream `main` in the background.
|
|
211
214
|
|
|
212
|
-
The `vision-
|
|
213
|
-
`SKILL.md` plus all five upstream playbooks. Tool
|
|
214
|
-
Artifact delivery, progressive exposure, and DSH
|
|
215
|
-
adapted; the upstream tool-selection rules,
|
|
216
|
-
remain intact. The exact upstream Skill
|
|
217
|
-
and reviewable adapter patch are
|
|
218
|
-
`patches/vision-tools-dsh.patch`.
|
|
215
|
+
The bundled `vision-skills` Skill is the DSH adapter of the upstream
|
|
216
|
+
`vision-tools` Skill: its `SKILL.md` plus all five upstream playbooks. Tool
|
|
217
|
+
names, argument syntax, Artifact delivery, progressive exposure, and DSH
|
|
218
|
+
path/lifecycle boundaries are adapted; the upstream tool-selection rules,
|
|
219
|
+
coarse-to-fine method, and task SOPs remain intact. The exact upstream Skill
|
|
220
|
+
commit, source hashes, adapted hashes, and reviewable adapter patch are
|
|
221
|
+
recorded in `assets/skill/UPSTREAM.json` and `patches/vision-tools-dsh.patch`.
|
|
219
222
|
|
|
220
223
|
For routes that DSH positively identifies as text-only, the plugin registers a sibling `<model> (Vision Toolkit)` variant. By default, pasting an image in DSH Web switches to that variant and gives the model both a reusable workspace path and a visual description focused on the current task.
|
|
221
224
|
|
|
@@ -239,6 +242,7 @@ This is a shared zero-configuration entry point, not an unlimited private endpoi
|
|
|
239
242
|
|
|
240
243
|
| Limit | Current value |
|
|
241
244
|
|---|---:|
|
|
245
|
+
| Daily quota | 300 images per machine per day |
|
|
242
246
|
| Images per request | Up to 5 |
|
|
243
247
|
| Image size | 4 MiB per image |
|
|
244
248
|
| Decoded pixels | 20,000,000 per image |
|
|
@@ -383,7 +387,7 @@ The updater revalidates the Profile before mutation, snapshots the original mani
|
|
|
383
387
|
|
|
384
388
|
| Problem | What to do |
|
|
385
389
|
|---|---|
|
|
386
|
-
| Pasting an image still says the model does not support image input | Restart the Web Profile, refresh the page, and confirm the selected route has the `(Vision Toolkit)` suffix. You can also place the image in the session workspace and invoke `/vision-
|
|
390
|
+
| Pasting an image still says the model does not support image input | Restart the Web Profile, refresh the page, and confirm the selected route has the `(Vision Toolkit)` suffix. You can also place the image in the session workspace and invoke `/vision-skills` |
|
|
387
391
|
| The free service returns 429 | Wait for the `Retry-After` interval, or switch to your own endpoint when you need stable higher volume |
|
|
388
392
|
| The image exceeds a size or pixel limit | Crop or resize it first; the error identifies whether bytes or decoded pixels caused the rejection |
|
|
389
393
|
| A custom Credential is missing | Enter the API key in **Settings → Vision Toolkit** and confirm the Credential name matches the provider configuration |
|
package/README.zh.md
CHANGED
|
@@ -6,9 +6,12 @@
|
|
|
6
6
|
|
|
7
7
|
# DSH Vision Toolkit
|
|
8
8
|
|
|
9
|
+
<a href="https://trendshift.io/repositories/149708?utm_source=trendshift-badge&utm_medium=badge&utm_campaign=badge-trendshift-149708" target="_blank" rel="noopener noreferrer"><img src="https://trendshift.io/api/badge/trendshift/repositories/149708/daily?language=TypeScript" alt="Anionex%2Fdsh-vision-toolkit | Trendshift" width="250" height="55"/></a>
|
|
10
|
+
|
|
9
11
|
[](https://dshfind.com/zh/plugins/Anionex/dsh-vision-toolkit)
|
|
10
12
|
[](https://dshfind.com/zh/plugins/Anionex/dsh-vision-toolkit)
|
|
11
13
|
[](https://www.npmjs.com/package/@anionex/dsh-vision-toolkit)
|
|
14
|
+
|
|
12
15
|
[](LICENSE)
|
|
13
16
|
[](cordis.patch.yml)
|
|
14
17
|
|
|
@@ -28,14 +31,14 @@
|
|
|
28
31
|
|
|
29
32
|
🏆 本项目为deepseek harness生态首个综合性视觉工具插件:内测前已立项,并在内测期间参考本人的[`agent-vision-toolkit`](https://github.com/Anionex/agent-vision-toolkit)做出
|
|
30
33
|
|
|
31
|
-
> **原创声明:** 这套视觉工具的体系和划分方式,以及 `vision-
|
|
34
|
+
> **原创声明:** 这套视觉工具的体系和划分方式,以及 `vision-skills` Skill,均由作者个人原创并持续打磨,相关工具、方法和工作流来自长期的真实使用与反复迭代。
|
|
32
35
|
|
|
33
36
|
## 亮点
|
|
34
37
|
|
|
35
38
|
- **粘贴即可使用。** 在 DSH Web 里粘贴图片,文本模型会自动切换到看图模式变体,不需要手动复制路径或更换模型。
|
|
36
39
|
- **无缝体验。** 图片保留原生缩略图、会话记录和工作区路径;Web 可以预览产物,Headless 也能继续使用同一份结构化结果。
|
|
37
40
|
- **一行命令安装即用。** 安装插件后默认使用内置免费 Gemini 3.7 Flash 视觉服务,不需要申请 API Key。
|
|
38
|
-
- **内置免费视觉。**
|
|
41
|
+
- **内置免费视觉。** 安装后即可直接使用共享服务,每台机器每天有 **300 张图** 的免费额度。
|
|
39
42
|
- **带着意图去看图。** Agent 不只生成通用描述,而是围绕“报错在哪里”“按钮在哪”等当前任务提取证据。
|
|
40
43
|
- **从截图到可验证结果。** 参考图、HTML 截图、差异定位和像素对比组成一条完整 UI 还原闭环。
|
|
41
44
|
|
|
@@ -150,7 +153,7 @@ dsh plugin --profile headless add @anionex/dsh-vision-toolkit
|
|
|
150
153
|
|
|
151
154
|
### 3. 粘贴图片,直接说你要做什么
|
|
152
155
|
|
|
153
|
-
在会话中粘贴截图,或把图片放进会话工作区,然后调用 `/vision-
|
|
156
|
+
在会话中粘贴截图,或把图片放进会话工作区,然后调用 `/vision-skills`。例如:
|
|
154
157
|
|
|
155
158
|
```text
|
|
156
159
|
看看这张截图,告诉我报错原因和最值得先修的地方。
|
|
@@ -200,7 +203,7 @@ dsh plugin --profile headless add @anionex/dsh-vision-toolkit
|
|
|
200
203
|
|
|
201
204
|
```mermaid
|
|
202
205
|
flowchart LR
|
|
203
|
-
Image["截图或本地 HTML"] --> Skill["vision-
|
|
206
|
+
Image["截图或本地 HTML"] --> Skill["vision-skills Skill"]
|
|
204
207
|
Skill --> Agent["文本 Agent 选择任务"]
|
|
205
208
|
Agent --> Vision["需要理解图片时调用视觉模型"]
|
|
206
209
|
Agent --> Local["裁剪、SVG、像素等任务在本地处理"]
|
|
@@ -212,7 +215,7 @@ flowchart LR
|
|
|
212
215
|
|
|
213
216
|
视觉能力来自打包的固定版本 `agent-vision-toolkit`。DSH 插件负责安装、会话级工具暴露、Credential、路径校验、取消、超时、结果文件和 Web 展示。运行时不会在后台拉取上游 `main`。
|
|
214
217
|
|
|
215
|
-
`vision-
|
|
218
|
+
`vision-skills` Skill(上游原名 `vision-tools`)现在以上游 `SKILL.md` 和全部 5 篇上游 SOP 为明确底稿:
|
|
216
219
|
只适配工具名、结构化参数、Artifact 交付、渐进式暴露,以及 DSH 的路径和生命周期边界;
|
|
217
220
|
上游的工具选择规则、由粗到细方法和任务流程保持不变。精确的上游 Skill commit、
|
|
218
221
|
源文件哈希、适配后哈希和可审查补丁分别记录在 `assets/skill/UPSTREAM.json` 与
|
|
@@ -240,6 +243,7 @@ API Key: https://agent-vision.anionex.me(自动填写)
|
|
|
240
243
|
|
|
241
244
|
| 限制 | 当前值 |
|
|
242
245
|
|---|---:|
|
|
246
|
+
| 每日额度 | 每台机器每天 300 张图 |
|
|
243
247
|
| 单次请求图片数 | 最多 5 张 |
|
|
244
248
|
| 单张图片大小 | 4 MiB |
|
|
245
249
|
| 单张图片像素 | 20,000,000 |
|
|
@@ -384,7 +388,7 @@ dsh plugin --profile web remove @anionex/dsh-vision-toolkit
|
|
|
384
388
|
|
|
385
389
|
| 问题 | 处理方式 |
|
|
386
390
|
|---|---|
|
|
387
|
-
| 粘贴图片后仍提示模型不支持图片 | 重启 Web Profile 并刷新页面,确认当前模型已切换到带 `(Vision Toolkit)` 的变体;也可以把图片先放进会话工作区,再调用 `/vision-
|
|
391
|
+
| 粘贴图片后仍提示模型不支持图片 | 重启 Web Profile 并刷新页面,确认当前模型已切换到带 `(Vision Toolkit)` 的变体;也可以把图片先放进会话工作区,再调用 `/vision-skills` |
|
|
388
392
|
| 免费服务提示 429 | 按错误中的 `Retry-After` 等待后重试;如果需要稳定高额度,切换到自己的视觉端点 |
|
|
389
393
|
| 图片过大或像素超限 | 先裁剪或缩放图片;错误会明确显示是字节还是像素限制 |
|
|
390
394
|
| 自定义 Credential 缺失 | 在 **设置 → 视觉工具** 填写 API Key,并确认 Credential 名称与配置一致 |
|
package/assets/skill/SKILL.md
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# vision-
|
|
1
|
+
# vision-skills
|
|
2
2
|
|
|
3
3
|
Ten native DSH tools give a text-only agent eyes. Use these structured tools
|
|
4
4
|
directly; do not shell out to the bundled Python scripts or reproduce their
|
|
@@ -7,7 +7,7 @@ plugin, so tool calls do not receive credentials.
|
|
|
7
7
|
|
|
8
8
|
The visual execution schemas are mounted only for the current Agent after this
|
|
9
9
|
Skill is loaded. A normal `skill` call activates them for the next model step.
|
|
10
|
-
If this content arrived through a direct `/vision-
|
|
10
|
+
If this content arrived through a direct `/vision-skills` invocation and the
|
|
11
11
|
visual tools are still absent, call `vision_toolkit_activate` once. Do not call
|
|
12
12
|
that bootstrap when the visual tools are already present.
|
|
13
13
|
|
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
"repository": "https://github.com/Anionex/agent-vision-toolkit",
|
|
4
4
|
"commit": "77c24ad5b5d7a123119862893129f939307f1d3f",
|
|
5
5
|
"patch": "patches/vision-tools-dsh.patch",
|
|
6
|
-
"patchSha256": "
|
|
6
|
+
"patchSha256": "ebbd49035945f094be9a582c5095d3e1689879d2501a7c82d0ca4517e38abf1b",
|
|
7
7
|
"sourceFiles": [
|
|
8
8
|
{
|
|
9
9
|
"path": "SKILL.md",
|
|
@@ -39,8 +39,8 @@
|
|
|
39
39
|
"adaptedFiles": [
|
|
40
40
|
{
|
|
41
41
|
"path": "SKILL.md",
|
|
42
|
-
"bytes":
|
|
43
|
-
"sha256": "
|
|
42
|
+
"bytes": 16070,
|
|
43
|
+
"sha256": "5e66863f14e85beaba46a6d148ab82304aa00d63ce2ae90cb25e3aad975f1b22"
|
|
44
44
|
},
|
|
45
45
|
{
|
|
46
46
|
"path": "references/gui.md",
|
|
@@ -64,8 +64,8 @@
|
|
|
64
64
|
},
|
|
65
65
|
{
|
|
66
66
|
"path": "references/restore-ui.md",
|
|
67
|
-
"bytes":
|
|
68
|
-
"sha256": "
|
|
67
|
+
"bytes": 10197,
|
|
68
|
+
"sha256": "8cd479a195c526098733ac518cb9928ffdafdbb3ad93e08864efe4fd7bd312b3"
|
|
69
69
|
}
|
|
70
70
|
]
|
|
71
71
|
}
|
|
@@ -37,8 +37,9 @@ typography, shadows, decorative details, and icon geometry.
|
|
|
37
37
|
or questions into the same round instead of waiting for each result before
|
|
38
38
|
starting the next call. Normally one or two rounds are enough.
|
|
39
39
|
4. Do not use `vision_trace`, foreground extraction, repeated color sampling,
|
|
40
|
-
|
|
41
|
-
consume the delivery window.
|
|
40
|
+
iterative `vision_pixel_diff` work, or hand-written SVG in fast mode. Those
|
|
41
|
+
are fidelity tools and will consume the delivery window. Icons and decorative
|
|
42
|
+
marks stay library-based or screenshot-backed; never hand-write SVG code.
|
|
42
43
|
|
|
43
44
|
### Build the approximation
|
|
44
45
|
|
|
@@ -47,8 +48,9 @@ typography, shadows, decorative details, and icon geometry.
|
|
|
47
48
|
when zoomed in.
|
|
48
49
|
2. Reuse the project's existing components and CSS tokens. If its frontend or
|
|
49
50
|
icon library contains a reasonably similar component or icon, use it
|
|
50
|
-
directly instead of recreating the reference.
|
|
51
|
-
|
|
51
|
+
directly instead of recreating the reference. Never hand-write SVG in fast
|
|
52
|
+
mode: use an approximate library icon, or extract the original pixels as a
|
|
53
|
+
screenshot-backed asset.
|
|
52
54
|
3. Use nearby existing palette tokens or visually similar CSS values. Exact
|
|
53
55
|
sampled hex values, gradients, subtle borders, and shadow opacity are out of
|
|
54
56
|
scope unless one of them defines the whole composition.
|
|
@@ -36,7 +36,7 @@ This reference maps the DSH Vision Toolkit product brief's committed P0/P1 requi
|
|
|
36
36
|
| Portability | Runtime and browser calls use argv vectors and Node filesystem/process APIs rather than POSIX shell composition. Managed preparation has `uv` and `venv`/pip paths; Python and Chrome discovery include platform-specific candidates. Package tests avoid machine-local dependency specifiers, and the checked-in fixture/profile flow is keyless. |
|
|
37
37
|
| Performance and cancellation | [`src/runtime.ts`](../../src/runtime.ts) applies one hard deadline, propagates `AbortSignal`, bounds per-session concurrency, rejects oversized decoded images before remote I/O, deduplicates repeated inputs within one glance operation, and keeps a one-entry content/configuration-keyed cache for the last successful identical glance in each live Session. [`src/index.ts`](../../src/index.ts) aborts plugin-owned calls before unregistering tools. Timeout, caller/plugin cancellation, cache-hit/miss, semaphore, and independent-session behavior are automated. |
|
|
38
38
|
| Observability | [`src/runtime.ts`](../../src/runtime.ts) logs bounded tool name, outcome, total/upstream duration, image count/bytes/pixels, cache hits, model, and error category while excluding base64, credentials, headers, and unbounded upstream output. |
|
|
39
|
-
| Model-context economy | The profile-global runtime publishes one small bootstrap, while [`src/exposure.ts`](../../src/exposure.ts) mounts the ten execution schemas only in an Agent that loads `vision-
|
|
39
|
+
| Model-context economy | The profile-global runtime publishes one small bootstrap, while [`src/exposure.ts`](../../src/exposure.ts) mounts the ten execution schemas only in an Agent that loads `vision-skills` and hides the bootstrap afterward. Other Agents remain unchanged. Health, connection, and version diagnostics stay exclusively on the Web Settings seam and never become model tools. Unit tests cover Agent isolation and restoration; every real-profile tool flow asserts the initial and activated schema sets. |
|
|
40
40
|
| Maintainability | Every tool calls one [`VisionToolkitRuntime`](../../src/runtime.ts); DSH-specific adaptation remains in [`src/tools.ts`](../../src/tools.ts), [`src/exposure.ts`](../../src/exposure.ts), [`src/index.ts`](../../src/index.ts), [`src/web.ts`](../../src/web.ts), and [`src/client/index.tsx`](../../src/client/index.tsx), while the pinned upstream remains the sole algorithm source. Runtime readiness, Skill/bootstrap publication, and Agent-scoped schemas belong to one lifecycle generation. |
|
|
41
41
|
| HTML screenshot isolation | The pinned screenshot guard uses a disposable Chrome profile, `--use-mock-keychain`, and `--incognito`, removes the profile after each call, and keeps network disabled. [`tests/html-screenshot-guard.spec.ts`](../../tests/html-screenshot-guard.spec.ts) captures the actual argv and cleanup behavior. |
|
|
42
42
|
| UI restoration acceptance | [`scripts/ui-restoration-example.ts`](../../scripts/ui-restoration-example.ts) performs reference → HTML render → pixel diff through the real runtime. [`examples/ui-restoration/README.md`](../../examples/ui-restoration/README.md) owns the reproducible procedure and checked-in evidence; [`tests/ui-restoration-example.spec.ts`](../../tests/ui-restoration-example.spec.ts) enforces it. |
|
|
@@ -36,7 +36,7 @@
|
|
|
36
36
|
| 可移植性 | 运行时和浏览器调用使用 argv 向量与 Node 文件系统/进程 API,不拼接 POSIX Shell。Managed 准备同时提供 `uv` 和 `venv`/pip 路径;Python 与 Chrome 探测包含平台候选。软件包测试拒绝机器本地依赖声明,已提交 fixture/Profile 流程无需真实 Key。 |
|
|
37
37
|
| 性能与取消 | [`src/runtime.ts`](../../src/runtime.ts) 应用一个硬截止时间、传递 `AbortSignal`、限制每个会话的并发数、在远程 I/O 前拒绝解码后过大的图片、在一次 glance 操作中对重复输入去重,并为每个活动会话保留一条按内容/配置键控的最近成功相同 glance 缓存。[`src/index.ts`](../../src/index.ts) 会在注销工具前中止插件拥有的调用。超时、调用方/插件取消、缓存命中/未命中、信号量和独立会话行为均有自动化覆盖。 |
|
|
38
38
|
| 可观察性 | [`src/runtime.ts`](../../src/runtime.ts) 记录有界的工具名、结果、总耗时/上游耗时、图片数量/字节/像素、缓存命中、模型和错误类别,同时排除 base64、Credential、鉴权头和无界上游输出。 |
|
|
39
|
-
| 模型上下文经济性 | Profile 级运行时只发布一个很小的引导工具,[`src/exposure.ts`](../../src/exposure.ts) 仅为加载 `vision-
|
|
39
|
+
| 模型上下文经济性 | Profile 级运行时只发布一个很小的引导工具,[`src/exposure.ts`](../../src/exposure.ts) 仅为加载 `vision-skills` 的 Agent 挂载 10 个执行 schema,并在成功后隐藏引导工具。其他 Agent 不受影响。健康检查、连接测试和版本诊断只存在于 Web Settings 边界,永远不会成为模型工具。单元测试覆盖 Agent 隔离与恢复;每条真实 Profile 工具流程都会断言初始和激活后的 schema 集合。 |
|
|
40
40
|
| 可维护性 | 每个工具都调用同一个 [`VisionToolkitRuntime`](../../src/runtime.ts);DSH 专用适配保留在 [`src/tools.ts`](../../src/tools.ts)、[`src/exposure.ts`](../../src/exposure.ts)、[`src/index.ts`](../../src/index.ts)、[`src/web.ts`](../../src/web.ts) 和 [`src/client/index.tsx`](../../src/client/index.tsx),固定上游继续作为算法唯一来源。运行时就绪、skill/引导工具发布和 Agent 级 schema 属于同一个生命周期 generation。 |
|
|
41
41
|
| HTML 截图隔离 | 固定截图 guard 使用一次性 Chrome Profile、`--use-mock-keychain` 和 `--incognito`,每次调用后删除 Profile,并保持网络禁用。[`tests/html-screenshot-guard.spec.ts`](../../tests/html-screenshot-guard.spec.ts) 捕获实际 argv 与清理行为。 |
|
|
42
42
|
| UI 还原验收 | [`scripts/ui-restoration-example.ts`](../../scripts/ui-restoration-example.ts) 通过真实运行时执行参考页面 → HTML 渲染 → 像素差异。[`examples/ui-restoration/README.md`](../../examples/ui-restoration/README.md) 负责可复现流程与已提交证据;[`tests/ui-restoration-example.spec.ts`](../../tests/ui-restoration-example.spec.ts) 强制执行该契约。 |
|
package/lib/exposure.js
CHANGED
|
@@ -6,18 +6,28 @@
|
|
|
6
6
|
* @module dsh-vision-toolkit/exposure
|
|
7
7
|
*/
|
|
8
8
|
import { defineTool } from '@deepseek-ai/dsh-tools';
|
|
9
|
-
import {
|
|
9
|
+
import { VISION_SKILLS_CONTENT, VISION_SKILLS_NAME } from "./skill.js";
|
|
10
10
|
import { VISION_TOOL_NAMES } from "./tools.js";
|
|
11
11
|
/** Small bootstrap tool retained only until the current Agent gains visual tools. */
|
|
12
12
|
export const VISION_TOOLKIT_ACTIVATE = 'vision_toolkit_activate';
|
|
13
|
+
/** Skill name used by releases before the rename to vision-skills. */
|
|
14
|
+
export const LEGACY_VISION_TOOLS_SKILL_NAME = 'vision-tools';
|
|
15
|
+
/** Unique pre-rename line in bundled instructions, kept for Session restore. */
|
|
16
|
+
export const LEGACY_VISION_TOOLS_SKILL_MARKER = 'If this content arrived through a direct `/vision-tools` invocation and the';
|
|
13
17
|
function renderJson(_args, value) {
|
|
14
18
|
return [{ type: 'text', text: JSON.stringify(value, null, 2) }];
|
|
15
19
|
}
|
|
16
20
|
function isRecord(value) {
|
|
17
21
|
return value !== null && typeof value === 'object' && !Array.isArray(value);
|
|
18
22
|
}
|
|
23
|
+
function isBundledSkillName(name) {
|
|
24
|
+
return name === VISION_SKILLS_NAME || name === LEGACY_VISION_TOOLS_SKILL_NAME;
|
|
25
|
+
}
|
|
26
|
+
function isBundledSkillContent(text) {
|
|
27
|
+
return text.includes(VISION_SKILLS_CONTENT) || text.includes(LEGACY_VISION_TOOLS_SKILL_MARKER);
|
|
28
|
+
}
|
|
19
29
|
function isVisionSkillArguments(value) {
|
|
20
|
-
return isRecord(value) && value.name
|
|
30
|
+
return isRecord(value) && isBundledSkillName(value.name);
|
|
21
31
|
}
|
|
22
32
|
function nativeSkillCall(raw) {
|
|
23
33
|
try {
|
|
@@ -31,12 +41,13 @@ function containsBundledSkillContent(blocks) {
|
|
|
31
41
|
return blocks.some(block => isRecord(block)
|
|
32
42
|
&& block.type === 'text'
|
|
33
43
|
&& typeof block.text === 'string'
|
|
34
|
-
&& block.text
|
|
44
|
+
&& isBundledSkillContent(block.text));
|
|
35
45
|
}
|
|
36
46
|
function isBundledSkillResult(value) {
|
|
37
47
|
return isRecord(value)
|
|
38
|
-
&& value.name
|
|
39
|
-
&& value.content ===
|
|
48
|
+
&& isBundledSkillName(value.name)
|
|
49
|
+
&& typeof value.content === 'string'
|
|
50
|
+
&& isBundledSkillContent(value.content);
|
|
40
51
|
}
|
|
41
52
|
/** Whether durable history proves that this Session loaded the bundled Skill. */
|
|
42
53
|
function hasLoadedVisionSkill(session) {
|
|
@@ -45,7 +56,7 @@ function hasLoadedVisionSkill(session) {
|
|
|
45
56
|
if (event.type === 'user/message') {
|
|
46
57
|
const source = event.data.source;
|
|
47
58
|
if (source.kind === 'skill-invocation'
|
|
48
|
-
&& source.name
|
|
59
|
+
&& isBundledSkillName(source.name)
|
|
49
60
|
&& containsBundledSkillContent(event.data.content))
|
|
50
61
|
return true;
|
|
51
62
|
continue;
|
|
@@ -96,7 +107,7 @@ export class VisionToolExposure {
|
|
|
96
107
|
this.activationTool = defineTool({
|
|
97
108
|
name: VISION_TOOLKIT_ACTIVATE,
|
|
98
109
|
description: `Activate the independent Vision Toolkit execution tools for this Agent: ${Object.values(VISION_TOOL_NAMES).join(', ')}. `
|
|
99
|
-
+ `Loading the ${
|
|
110
|
+
+ `Loading the ${VISION_SKILLS_NAME} Skill normally activates them automatically; call this once when the visual tools are still absent, then use them for image understanding, OCR, UI detection, and related tasks. `
|
|
100
111
|
+ 'It is safe to call before the Skill is loaded, and this activation tool disappears after success.',
|
|
101
112
|
parameters: {},
|
|
102
113
|
output: {
|
package/lib/exposure.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"exposure.js","sourceRoot":"","sources":["../src/exposure.ts"],"names":[],"mappings":"AAAA;;;;;;GAMG;AAKH,OAAO,EAAE,UAAU,EAAuB,MAAM,wBAAwB,CAAA;AAExE,OAAO,EAAE,
|
|
1
|
+
{"version":3,"file":"exposure.js","sourceRoot":"","sources":["../src/exposure.ts"],"names":[],"mappings":"AAAA;;;;;;GAMG;AAKH,OAAO,EAAE,UAAU,EAAuB,MAAM,wBAAwB,CAAA;AAExE,OAAO,EAAE,qBAAqB,EAAE,kBAAkB,EAAE,MAAM,YAAY,CAAA;AACtE,OAAO,EAAE,iBAAiB,EAAE,MAAM,YAAY,CAAA;AAE9C,qFAAqF;AACrF,MAAM,CAAC,MAAM,uBAAuB,GAAG,yBAAyB,CAAA;AAEhE,sEAAsE;AACtE,MAAM,CAAC,MAAM,8BAA8B,GAAG,cAAc,CAAA;AAE5D,gFAAgF;AAChF,MAAM,CAAC,MAAM,gCAAgC,GAAG,6EAA6E,CAAA;AAe7H,SAAS,UAAU,CAAC,KAAc,EAAE,KAAc;IAChD,OAAO,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC,KAAK,EAAE,IAAI,EAAE,CAAC,CAAC,EAAE,CAAC,CAAA;AACjE,CAAC;AAED,SAAS,QAAQ,CAAC,KAAc;IAC9B,OAAO,KAAK,KAAK,IAAI,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,CAAA;AAC7E,CAAC;AAED,SAAS,kBAAkB,CAAC,IAAa;IACvC,OAAO,IAAI,KAAK,kBAAkB,IAAI,IAAI,KAAK,8BAA8B,CAAA;AAC/E,CAAC;AAED,SAAS,qBAAqB,CAAC,IAAY;IACzC,OAAO,IAAI,CAAC,QAAQ,CAAC,qBAAqB,CAAC,IAAI,IAAI,CAAC,QAAQ,CAAC,gCAAgC,CAAC,CAAA;AAChG,CAAC;AAED,SAAS,sBAAsB,CAAC,KAAc;IAC5C,OAAO,QAAQ,CAAC,KAAK,CAAC,IAAI,kBAAkB,CAAC,KAAK,CAAC,IAAI,CAAC,CAAA;AAC1D,CAAC;AAED,SAAS,eAAe,CAAC,GAAW;IAClC,IAAI,CAAC;QACH,OAAO,sBAAsB,CAAC,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAA;IAChD,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,KAAK,CAAA;IACd,CAAC;AACH,CAAC;AAED,SAAS,2BAA2B,CAAC,MAA0B;IAC7D,OAAO,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC,QAAQ,CAAC,KAAK,CAAC;WACtC,KAAK,CAAC,IAAI,KAAK,MAAM;WACrB,OAAO,KAAK,CAAC,IAAI,KAAK,QAAQ;WAC9B,qBAAqB,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,CAAA;AACzC,CAAC;AAED,SAAS,oBAAoB,CAAC,KAAc;IAC1C,OAAO,QAAQ,CAAC,KAAK,CAAC;WACjB,kBAAkB,CAAC,KAAK,CAAC,IAAI,CAAC;WAC9B,OAAO,KAAK,CAAC,OAAO,KAAK,QAAQ;WACjC,qBAAqB,CAAC,KAAK,CAAC,OAAO,CAAC,CAAA;AAC3C,CAAC;AAED,iFAAiF;AACjF,SAAS,oBAAoB,CAAC,OAAgB;IAC5C,MAAM,WAAW,GAAG,IAAI,GAAG,EAAU,CAAA;IACrC,KAAK,MAAM,KAAK,IAAI,OAAO,CAAC,MAAM,EAAE,CAAC;QACnC,IAAI,KAAK,CAAC,IAAI,KAAK,cAAc,EAAE,CAAC;YAClC,MAAM,MAAM,GAAG,KAAK,CAAC,IAAI,CAAC,MAAM,CAAA;YAChC,IAAI,MAAM,CAAC,IAAI,KAAK,kBAAkB;mBACjC,kBAAkB,CAAC,MAAM,CAAC,IAAI,CAAC;mBAC/B,2BAA2B,CAAC,KAAK,CAAC,IAAI,CAAC,OAAO,CAAC;gBAAE,OAAO,IAAI,CAAA;YACjE,SAAQ;QACV,CAAC;QACD,IAAI,KAAK,CAAC,IAAI,KAAK,WAAW,EAAE,CAAC;YAC/B,IAAI,KAAK,CAAC,IAAI,CAAC,IAAI,KAAK,OAAO,IAAI,eAAe,CAAC,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,CAAC;gBACzE,WAAW,CAAC,GAAG,CAAC,MAAM,CAAC,KAAK,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC,CAAA;YAC5C,CAAC;YACD,SAAQ;QACV,CAAC;QACD,IAAI,KAAK,CAAC,IAAI,KAAK,aAAa,EAAE,CAAC;YACjC,MAAM,CAAC,KAAK,CAAC,GAAG,KAAK,CAAC,IAAI,CAAC,OAAO,CAAC,OAAO,CAAA;YAC1C,IAAI,KAAK,EAAE,IAAI,KAAK,aAAa;mBAC5B,KAAK,CAAC,OAAO,KAAK,IAAI;mBACtB,WAAW,CAAC,GAAG,CAAC,MAAM,CAAC,KAAK,CAAC,UAAU,CAAC,CAAC;mBACzC,2BAA2B,CAAC,KAAK,CAAC,OAAO,CAAC;gBAAE,OAAO,IAAI,CAAA;YAC5D,SAAQ;QACV,CAAC;QACD,IAAI,KAAK,CAAC,IAAI,KAAK,oBAAoB;eAClC,KAAK,CAAC,IAAI,CAAC,IAAI,KAAK,OAAO;eAC3B,KAAK,CAAC,IAAI,CAAC,OAAO,KAAK,KAAK;eAC5B,sBAAsB,CAAC,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC;eAC5C,2BAA2B,CAAC,KAAK,CAAC,IAAI,CAAC,OAAO,CAAC;YAAE,OAAO,IAAI,CAAA;IACnE,CAAC;IACD,OAAO,KAAK,CAAA;AACd,CAAC;AAED;;;;;GAKG;AACH,MAAM,OAAO,kBAAkB;IAUV;IACA;IAVV,cAAc,CAAgB;IACtB,MAAM,GAAG,IAAI,GAAG,EAAwB,CAAA;IACjD,SAAS,GAAG,KAAK,CAAA;IAEzB;;;OAGG;IACH,YACmB,GAAY,EACZ,WAAmC;QADnC,QAAG,GAAH,GAAG,CAAS;QACZ,gBAAW,GAAX,WAAW,CAAwB;QAEpD,IAAI,CAAC,cAAc,GAAG,UAAU,CAAC;YAC/B,IAAI,EAAE,uBAAuB;YAC7B,WAAW,EAAE,2EAA2E,MAAM,CAAC,MAAM,CAAC,iBAAiB,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,IAAI;kBACnI,eAAe,kBAAkB,oLAAoL;kBACrN,mGAAmG;YACvG,UAAU,EAAE,EAAE;YACd,MAAM,EAAE;gBACN,MAAM,EAAE;oBACN,IAAI,EAAE,QAAQ;oBACd,oBAAoB,EAAE,KAAK;oBAC3B,UAAU,EAAE;wBACV,SAAS,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,QAAQ,EAAE,IAAI,EAAE;wBAC9C,KAAK,EAAE,EAAE,IAAI,EAAE,OAAO,EAAE,KAAK,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,EAAE,QAAQ,EAAE,IAAI,EAAE;qBACpE;iBACF;gBACD,MAAM,EAAE,UAAU;aACnB;YACD,OAAO,EAAE,CAAC,KAAK,EAAE,IAAI,EAA0C,EAAE;gBAC/D,IAAI,IAAI,CAAC,KAAK,KAAK,SAAS,EAAE,CAAC;oBAC7B,MAAM,IAAI,KAAK,CAAC,GAAG,uBAAuB,gCAAgC,CAAC,CAAA;gBAC7E,CAAC;gBACD,OAAO,OAAO,CAAC,OAAO,CAAC,IAAI,CAAC,QAAQ,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAA;YACnD,CAAC;YACD,WAAW,EAAE,GAAG,EAAE,CAAC,CAAC,EAAE,IAAI,EAAE,SAAS,EAAE,KAAK,EAAE,uBAAuB,EAAE,IAAI,EAAE,SAAS,EAAE,CAAC;SAC1F,CAAC,CAAA;IACJ,CAAC;IAED,uEAAuE;IACvE,OAAO;QACL,IAAI,IAAI,CAAC,SAAS;YAAE,MAAM,IAAI,KAAK,CAAC,+DAA+D,CAAC,CAAA;QACpG,IAAI,CAAC,SAAS,GAAG,IAAI,CAAA;QACrB,MAAM,SAAS,GAAG;YAChB,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,eAAe,EAAE,CAAC,EAAE,KAAK,EAAE,EAAE,EAAE,GAAG,IAAI,CAAC,MAAM,CAAC,KAAK,CAAC,CAAA,CAAC,CAAC,CAAC;YACnE,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,gBAAgB,EAAE,CAAC,EAAE,KAAK,EAAE,EAAE,EAAE,GAAG,IAAI,CAAC,MAAM,CAAC,KAAK,CAAC,CAAA,CAAC,CAAC,CAAC;YACpE,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,eAAe,EAAE,CAAC,OAAO,EAAE,KAAK,EAAE,EAAE;gBAC9C,IAAI,KAAK,CAAC,IAAI,KAAK,UAAU;oBAAE,IAAI,CAAC,6BAA6B,CAAC,OAAO,CAAC,CAAA;YAC5E,CAAC,CAAC;YACF,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,cAAc,EAAE,CAAC,IAAI,EAAE,MAAM,EAAE,EAAE;gBAC3C,IAAI,MAAM,CAAC,OAAO,KAAK,KAAK;uBACvB,IAAI,CAAC,IAAI,KAAK,OAAO;uBACrB,IAAI,CAAC,KAAK,KAAK,SAAS;uBACxB,sBAAsB,CAAC,IAAI,CAAC,SAAS,CAAC;uBACtC,oBAAoB,CAAC,MAAM,CAAC,KAAK,CAAC,EAAE,CAAC;oBACxC,IAAI,CAAC,QAAQ,CAAC,IAAI,CAAC,KAAK,CAAC,CAAA;gBAC3B,CAAC;gBACD,OAAO,SAAS,CAAA;YAClB,CAAC,CAAC;SACH,CAAA;QACD,IAAI,CAAC;YACH,KAAK,MAAM,KAAK,IAAI,IAAI,CAAC,GAAG,CAAC,MAAM,CAAC,IAAI,EAAE;gBAAE,IAAI,CAAC,MAAM,CAAC,KAAK,CAAC,CAAA;QAChE,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACf,KAAK,MAAM,OAAO,IAAI,SAAS,CAAC,OAAO,EAAE;gBAAE,OAAO,EAAE,CAAA;YACpD,IAAI,CAAC,aAAa,EAAE,CAAA;YACpB,IAAI,CAAC,SAAS,GAAG,KAAK,CAAA;YACtB,MAAM,KAAK,CAAA;QACb,CAAC;QACD,OAAO,GAAG,EAAE;YACV,IAAI,CAAC,IAAI,CAAC,SAAS;gBAAE,OAAM;YAC3B,IAAI,CAAC,SAAS,GAAG,KAAK,CAAA;YACtB,KAAK,MAAM,OAAO,IAAI,SAAS,CAAC,OAAO,EAAE;gBAAE,OAAO,EAAE,CAAA;YACpD,IAAI,CAAC,aAAa,EAAE,CAAA;QACtB,CAAC,CAAA;IACH,CAAC;IAEO,MAAM,CAAC,KAAY;QACzB,IAAI,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,KAAK,CAAC;YAAE,OAAM;QAClC,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,KAAK,EAAE,EAAE,MAAM,EAAE,KAAK,EAAE,aAAa,EAAE,EAAE,EAAE,SAAS,EAAE,EAAE,EAAE,CAAC,CAAA;QAC3E,IAAI,oBAAoB,CAAC,KAAK,CAAC,OAAO,CAAC;YAAE,IAAI,CAAC,QAAQ,CAAC,KAAK,CAAC,CAAA;IAC/D,CAAC;IAEO,QAAQ,CAAC,KAAY;QAC3B,IAAI,CAAC,MAAM,CAAC,KAAK,CAAC,CAAA;QAClB,MAAM,KAAK,GAAG,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,KAAK,CAAC,CAAA;QACpC,wEAAwE;QACxE,IAAI,KAAK,KAAK,SAAS;YAAE,MAAM,IAAI,KAAK,CAAC,6BAA6B,MAAM,CAAC,KAAK,CAAC,EAAE,CAAC,wBAAwB,CAAC,CAAA;QAC/G,IAAI,KAAK,CAAC,MAAM;YAAE,OAAO,EAAE,SAAS,EAAE,KAAK,EAAE,KAAK,EAAE,CAAC,GAAG,KAAK,CAAC,SAAS,CAAC,EAAE,CAAA;QAE1E,MAAM,WAAW,GAAG,IAAI,CAAC,WAAW,EAAE,CAAA;QACtC,MAAM,aAAa,GAAsB,EAAE,CAAA;QAC3C,IAAI,CAAC;YACH,KAAK,MAAM,UAAU,IAAI,WAAW;gBAAE,aAAa,CAAC,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,KAAK,CAAC,QAAQ,CAAC,UAAU,CAAC,CAAC,CAAA;YAC9F,uEAAuE;YACvE,wEAAwE;YACxE,iEAAiE;YACjE,IAAI,CAAC,IAAI,CAAC,aAAa,CAAC,KAAK,CAAC,OAAO,CAAC;gBAAE,IAAI,CAAC,mBAAmB,CAAC,KAAK,CAAC,CAAA;QACzE,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACf,KAAK,MAAM,OAAO,IAAI,aAAa,CAAC,OAAO,EAAE;gBAAE,OAAO,EAAE,CAAA;YACxD,MAAM,KAAK,CAAA;QACb,CAAC;QAED,KAAK,CAAC,MAAM,GAAG,IAAI,CAAA;QACnB,KAAK,CAAC,aAAa,GAAG,aAAa,CAAA;QACnC,KAAK,CAAC,SAAS,GAAG,WAAW,CAAC,GAAG,CAAC,UAAU,CAAC,EAAE,CAAC,UAAU,CAAC,IAAI,CAAC,CAAA;QAChE,OAAO,EAAE,SAAS,EAAE,IAAI,EAAE,KAAK,EAAE,CAAC,GAAG,KAAK,CAAC,SAAS,CAAC,EAAE,CAAA;IACzD,CAAC;IAED,6EAA6E;IACrE,aAAa,CAAC,OAAgB;QACpC,OAAO,IAAI,CAAC,GAAG,CAAC,QAAQ,CAAC,GAAG,CAAC,OAAO,CAAC,EAAE,CAAC,KAAK,OAAO,CAAA;IACtD,CAAC;IAEO,6BAA6B,CAAC,OAAgB;QACpD,KAAK,MAAM,KAAK,IAAI,IAAI,CAAC,GAAG,CAAC,MAAM,CAAC,IAAI,EAAE,EAAE,CAAC;YAC3C,IAAI,KAAK,CAAC,OAAO,KAAK,OAAO,IAAI,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,KAAK,CAAC,EAAE,MAAM,KAAK,IAAI,EAAE,CAAC;gBACzE,IAAI,CAAC,mBAAmB,CAAC,KAAK,CAAC,CAAA;YACjC,CAAC;QACH,CAAC;IACH,CAAC;IAEO,mBAAmB,CAAC,KAAY;QACtC,MAAM,KAAK,GAAG,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,KAAK,CAAC,CAAA;QACpC,IAAI,KAAK,KAAK,SAAS,IAAI,KAAK,CAAC,eAAe,KAAK,SAAS;YAAE,OAAM;QACtE,KAAK,CAAC,eAAe,GAAG,KAAK,CAAC,GAAG,CAAC,KAAK,CAAC,QAAQ,CAAC,EAAE,IAAI,EAAE,CAAC,uBAAuB,CAAC,EAAE,CAAC,CAAA;IACvF,CAAC;IAEO,MAAM,CAAC,KAAY;QACzB,MAAM,KAAK,GAAG,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,KAAK,CAAC,CAAA;QACpC,IAAI,KAAK,KAAK,SAAS;YAAE,OAAM;QAC/B,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,KAAK,CAAC,CAAA;QACzB,IAAI,CAAC,YAAY,CAAC,KAAK,CAAC,CAAA;IAC1B,CAAC;IAEO,aAAa;QACnB,KAAK,MAAM,KAAK,IAAI,IAAI,CAAC,MAAM,CAAC,MAAM,EAAE;YAAE,IAAI,CAAC,YAAY,CAAC,KAAK,CAAC,CAAA;QAClE,IAAI,CAAC,MAAM,CAAC,KAAK,EAAE,CAAA;IACrB,CAAC;IAEO,YAAY,CAAC,KAAoB;QACvC,KAAK,CAAC,eAAe,EAAE,EAAE,CAAA;QACzB,KAAK,MAAM,OAAO,IAAI,KAAK,CAAC,aAAa,CAAC,OAAO,EAAE;YAAE,OAAO,EAAE,CAAA;IAChE,CAAC;CACF"}
|
package/lib/index.js
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
* @anionex/dsh-vision-toolkit — DSH Vision Toolkit profile bundle.
|
|
3
3
|
*
|
|
4
4
|
* Plugin lifecycle follows the documented readiness chain: verify the pinned
|
|
5
|
-
* upstream checkout, publish the vision-
|
|
5
|
+
* upstream checkout, publish the vision-skills Skill and its one-shot bootstrap,
|
|
6
6
|
* then mount the execution tools only in Agents that load that Skill or invoke
|
|
7
7
|
* the bootstrap. Any
|
|
8
8
|
* failure leaves no model capability behind, and disposal unregisters every
|
|
@@ -14,7 +14,7 @@ import { Config, VISION_TOOLKIT_SETTINGS_NAMESPACE, resolveConfig, } from "./con
|
|
|
14
14
|
import { VisionToolExposure } from "./exposure.js";
|
|
15
15
|
import { createPasteTakeoverResolver, installImageInputVariants } from "./image-input-variants.js";
|
|
16
16
|
import { VisionToolkitRuntimeManager } from "./runtime-manager.js";
|
|
17
|
-
import {
|
|
17
|
+
import { VISION_SKILLS_SKILL } from "./skill.js";
|
|
18
18
|
import { createVisionTools } from "./tools.js";
|
|
19
19
|
import { PLUGIN_VERSION } from "./version.js";
|
|
20
20
|
import { installVisionToolkitWeb, VisionToolkitWebBackend } from "./web.js";
|
|
@@ -47,7 +47,7 @@ export async function apply(ctx, config = {}) {
|
|
|
47
47
|
let skill;
|
|
48
48
|
try {
|
|
49
49
|
activationTool = ctx.tools.register(exposure.activationTool);
|
|
50
|
-
skill = ctx.skills.register(
|
|
50
|
+
skill = ctx.skills.register(VISION_SKILLS_SKILL);
|
|
51
51
|
exposureDisposer = exposure.install();
|
|
52
52
|
operationalDisposers = { activationTool, exposure: exposureDisposer, skill };
|
|
53
53
|
const info = manager.current().upstreamVersion;
|
|
@@ -67,7 +67,7 @@ export async function apply(ctx, config = {}) {
|
|
|
67
67
|
}
|
|
68
68
|
catch (error) {
|
|
69
69
|
const message = error instanceof Error ? error.message : String(error);
|
|
70
|
-
ctx.logger.error('dsh-vision-toolkit %s: runtime not ready; the vision-
|
|
70
|
+
ctx.logger.error('dsh-vision-toolkit %s: runtime not ready; the vision-skills skill, activation bootstrap, and Agent-scoped visual tools are NOT registered. Settings remain available for repair. %s', PLUGIN_VERSION, message);
|
|
71
71
|
}
|
|
72
72
|
const backend = new VisionToolkitWebBackend(ctx, manager, artifacts, ensureOperational);
|
|
73
73
|
const pastedImages = new PastedImageBackend(ctx, {
|
package/lib/index.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;GAUG;AAMH,OAAO,EAAE,wBAAwB,EAAE,wBAAwB,EAAE,MAAM,sBAAsB,CAAA;AACzF,OAAO,EACL,MAAM,EACN,iCAAiC,EACjC,aAAa,GAEd,MAAM,aAAa,CAAA;AACpB,OAAO,EAAE,kBAAkB,EAAE,MAAM,eAAe,CAAA;AAClD,OAAO,EAAE,2BAA2B,EAAE,yBAAyB,EAAE,MAAM,2BAA2B,CAAA;AAClG,OAAO,EAAE,2BAA2B,EAAE,MAAM,sBAAsB,CAAA;AAClE,OAAO,EAAE,
|
|
1
|
+
{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;GAUG;AAMH,OAAO,EAAE,wBAAwB,EAAE,wBAAwB,EAAE,MAAM,sBAAsB,CAAA;AACzF,OAAO,EACL,MAAM,EACN,iCAAiC,EACjC,aAAa,GAEd,MAAM,aAAa,CAAA;AACpB,OAAO,EAAE,kBAAkB,EAAE,MAAM,eAAe,CAAA;AAClD,OAAO,EAAE,2BAA2B,EAAE,yBAAyB,EAAE,MAAM,2BAA2B,CAAA;AAClG,OAAO,EAAE,2BAA2B,EAAE,MAAM,sBAAsB,CAAA;AAClE,OAAO,EAAE,mBAAmB,EAAE,MAAM,YAAY,CAAA;AAChD,OAAO,EAAE,iBAAiB,EAAE,MAAM,YAAY,CAAA;AAC9C,OAAO,EAAE,cAAc,EAAE,MAAM,cAAc,CAAA;AAC7C,OAAO,EAAE,uBAAuB,EAAE,uBAAuB,EAAE,MAAM,UAAU,CAAA;AAC3E,OAAO,EAAE,qBAAqB,EAAE,kBAAkB,EAAE,MAAM,mBAAmB,CAAA;AAE7E,MAAM,CAAC,MAAM,IAAI,GAAG,6BAA6B,CAAA;AAEjD,OAAO,EAAE,MAAM,EAAE,CAAA;AAEjB,MAAM,CAAC,MAAM,MAAM,GAAG,CAAC,OAAO,EAAE,aAAa,EAAE,QAAQ,EAAE,YAAY,EAAE,UAAU,EAAE,QAAQ,EAAE,UAAU,CAAC,CAAA;AAExG,qFAAqF;AACrF,MAAM,CAAC,KAAK,UAAU,KAAK,CAAC,GAAY,EAAE,SAA8B,EAAE;IACxE,2EAA2E;IAC3E,4EAA4E;IAC5E,0EAA0E;IAC1E,iDAAiD;IACjD,MAAM,QAAQ,GAAG,GAAG,CAAC,QAAQ,CAAC,QAAQ,CAAC,iCAAiC,EAAE,MAAM,EAAE;QAChF,IAAI,EAAE,MAAM;QACZ,OAAO,EAAE,MAAM;QACf,QAAQ,EAAE,CAAC,KAAK,EAAE,EAAE,GAAG,aAAa,CAAC,KAAK,CAAC,CAAA,CAAC,CAAC;KAC9C,CAAC,CAAA;IACF,MAAM,OAAO,GAAG,IAAI,2BAA2B,CAAC,GAAG,CAAC,CAAA;IACpD,MAAM,SAAS,GAAG,IAAI,wBAAwB,CAAC,MAAM,wBAAwB,EAAE,CAAC,CAAA;IAChF,MAAM,SAAS,GAAG,IAAI,eAAe,EAAE,CAAA;IACvC,MAAM,SAAS,GAAsB,EAAE,CAAA;IACvC,IAAI,oBAAyG,CAAA;IAE7G,MAAM,iBAAiB,GAAG,GAAS,EAAE;QACnC,IAAI,CAAC,OAAO,CAAC,KAAK,IAAI,oBAAoB,KAAK,SAAS;YAAE,OAAM;QAChE,MAAM,QAAQ,GAAG,IAAI,kBAAkB,CAAC,GAAG,EAAE,GAAG,EAAE,CAAC,iBAAiB,CAClE,GAAG,EAAE,CAAC,OAAO,CAAC,OAAO,EAAE,EACvB,KAAK,CAAC,EAAE,CAAC,SAAS,CAAC,gBAAgB,CAAC,KAAK,CAAC,EAC1C,SAAS,CAAC,MAAM,CACjB,CAAC,CAAA;QACF,IAAI,cAAwC,CAAA;QAC5C,IAAI,gBAA0C,CAAA;QAC9C,IAAI,KAA+B,CAAA;QACnC,IAAI,CAAC;YACH,cAAc,GAAG,GAAG,CAAC,KAAK,CAAC,QAAQ,CAAC,QAAQ,CAAC,cAAc,CAAC,CAAA;YAC5D,KAAK,GAAG,GAAG,CAAC,MAAM,CAAC,QAAQ,CAAC,mBAAmB,CAAC,CAAA;YAChD,gBAAgB,GAAG,QAAQ,CAAC,OAAO,EAAE,CAAA;YACrC,oBAAoB,GAAG,EAAE,cAAc,EAAE,QAAQ,EAAE,gBAAgB,EAAE,KAAK,EAAE,CAAA;YAC5E,MAAM,IAAI,GAAG,OAAO,CAAC,OAAO,EAAE,CAAC,eAAe,CAAA;YAC9C,GAAG,CAAC,MAAM,CAAC,IAAI,CACb,6DAA6D,EAC7D,cAAc,EACd,IAAI,CAAC,OAAO,EACZ,IAAI,CAAC,MAAM,EACX,IAAI,CAAC,IAAI,CACV,CAAA;QACH,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACf,gBAAgB,EAAE,EAAE,CAAA;YACpB,IAAI,KAAK,KAAK,SAAS;gBAAE,KAAK,EAAE,CAAA;YAChC,cAAc,EAAE,EAAE,CAAA;YAClB,MAAM,KAAK,CAAA;QACb,CAAC;IACH,CAAC,CAAA;IAED,IAAI,CAAC;QACH,MAAM,OAAO,CAAC,UAAU,CAAC,QAAQ,CAAC,GAAG,EAAE,CAAC,CAAA;QACxC,iBAAiB,EAAE,CAAA;IACrB,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,MAAM,OAAO,GAAG,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAA;QACtE,GAAG,CAAC,MAAM,CAAC,KAAK,CACd,qLAAqL,EACrL,cAAc,EACd,OAAO,CACR,CAAA;IACH,CAAC;IAED,MAAM,OAAO,GAAG,IAAI,uBAAuB,CAAC,GAAG,EAAE,OAAO,EAAE,SAAS,EAAE,iBAAiB,CAAC,CAAA;IACvF,MAAM,YAAY,GAAG,IAAI,kBAAkB,CAAC,GAAG,EAAE;QAC/C,cAAc,EAAE,GAAG,EAAE,CAAC,qBAAqB;KAC5C,CAAC,CAAA;IACF,2EAA2E;IAC3E,yEAAyE;IACzE,uCAAuC;IACvC,MAAM,QAAQ,GAAG,yBAAyB,CACxC,GAAG,EACH,GAAG,EAAE,CAAC,aAAa,CAAC,QAAQ,CAAC,GAAG,EAAE,CAAC,EACnC,GAAG,EAAE,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,OAAO,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,SAAS,CACpD,CAAA;IACD,uBAAuB,CACrB,GAAG,EACH,OAAO,EACP,SAAS,EACT,YAAY,EACZ,2BAA2B,CAAC,GAAG,EAAE,GAAG,EAAE,CAAC,aAAa,CAAC,QAAQ,CAAC,GAAG,EAAE,CAAC,CAAC,CACtE,CAAA;IACD,SAAS,CAAC,IAAI,CAAC,QAAQ,CAAC,OAAO,CAAC,CAAA;IAChC,SAAS,CAAC,IAAI,CAAC,QAAQ,CAAC,KAAK,CAAC,KAAK,EAAE,IAAI,EAAE,EAAE;QAC3C,IAAI,CAAC;YACH,MAAM,OAAO,CAAC,WAAW,CAAC,IAAI,CAAC,CAAA;YAC/B,iBAAiB,EAAE,CAAA;YACnB,QAAQ,CAAC,SAAS,EAAE,CAAA;QACtB,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACf,MAAM,OAAO,GAAG,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAA;YACtE,GAAG,CAAC,MAAM,CAAC,KAAK,CAAC,0FAA0F,EAAE,OAAO,CAAC,CAAA;QACvH,CAAC;IACH,CAAC,CAAC,CAAC,CAAA;IAEH,OAAO,GAAG,EAAE;QACV,SAAS,CAAC,KAAK,EAAE,CAAA;QACjB,IAAI,oBAAoB,KAAK,SAAS,EAAE,CAAC;YACvC,oBAAoB,CAAC,QAAQ,EAAE,CAAA;YAC/B,oBAAoB,CAAC,cAAc,EAAE,CAAA;YACrC,oBAAoB,CAAC,KAAK,EAAE,CAAA;YAC5B,oBAAoB,GAAG,SAAS,CAAA;QAClC,CAAC;QACD,KAAK,MAAM,OAAO,IAAI,SAAS,CAAC,OAAO,EAAE;YAAE,OAAO,EAAE,CAAA;IACtD,CAAC,CAAA;AACH,CAAC"}
|
package/lib/skill.js
CHANGED
|
@@ -5,21 +5,21 @@
|
|
|
5
5
|
import { readFileSync } from 'node:fs';
|
|
6
6
|
import { fileURLToPath } from 'node:url';
|
|
7
7
|
/** Stable catalog/invocation name shared with progressive tool exposure. */
|
|
8
|
-
export const
|
|
8
|
+
export const VISION_SKILLS_NAME = 'vision-skills';
|
|
9
9
|
/** Packaged resource root for the adapted upstream playbooks. */
|
|
10
|
-
export const
|
|
10
|
+
export const VISION_SKILLS_RESOURCE_BASE = fileURLToPath(new URL('../assets/skill/', import.meta.url));
|
|
11
11
|
/** Exact bundled instructions used as the progressive-exposure evidence marker. */
|
|
12
|
-
export const
|
|
12
|
+
export const VISION_SKILLS_CONTENT = readFileSync(new URL('../assets/skill/SKILL.md', import.meta.url), 'utf8');
|
|
13
13
|
/** Runtime skill registration mounted only after every native tool is ready. */
|
|
14
|
-
export const
|
|
15
|
-
name:
|
|
14
|
+
export const VISION_SKILLS_SKILL = {
|
|
15
|
+
name: VISION_SKILLS_NAME,
|
|
16
16
|
description: '把截图或设计图还原为 UI(网页、应用界面或组件),生成 HTML/CSS 或项目现有前端代码;也支持图片问答、长截图/聊天记录 OCR、元素定位与盘点、裁剪、前景提取、像素对比、取色、SVG 描摹和 HTML 转截图。当任务涉及图片理解、UI 还原/重建(rebuild or restore a UI from a screenshot)、视觉回归、像素坐标、基于截图的 GUI 操作、可复用图片/SVG 素材、图表还原或长截图 OCR 时使用。',
|
|
17
17
|
whenToUse: '任务依赖图片文字/内容、像素坐标、截图转 UI 重建、视觉回归、可复用图片/SVG 素材、图表还原、基于截图的 GUI 操作或长截图 OCR 时使用。',
|
|
18
18
|
source: 'runtime',
|
|
19
19
|
resourceBase: {
|
|
20
20
|
kind: 'directory',
|
|
21
|
-
path:
|
|
21
|
+
path: VISION_SKILLS_RESOURCE_BASE,
|
|
22
22
|
},
|
|
23
|
-
content:
|
|
23
|
+
content: VISION_SKILLS_CONTENT,
|
|
24
24
|
};
|
|
25
25
|
//# sourceMappingURL=skill.js.map
|
package/lib/skill.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"skill.js","sourceRoot":"","sources":["../src/skill.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH,OAAO,EAAE,YAAY,EAAE,MAAM,SAAS,CAAA;AACtC,OAAO,EAAE,aAAa,EAAE,MAAM,UAAU,CAAA;AAGxC,4EAA4E;AAC5E,MAAM,CAAC,MAAM,
|
|
1
|
+
{"version":3,"file":"skill.js","sourceRoot":"","sources":["../src/skill.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH,OAAO,EAAE,YAAY,EAAE,MAAM,SAAS,CAAA;AACtC,OAAO,EAAE,aAAa,EAAE,MAAM,UAAU,CAAA;AAGxC,4EAA4E;AAC5E,MAAM,CAAC,MAAM,kBAAkB,GAAG,eAAe,CAAA;AAEjD,iEAAiE;AACjE,MAAM,CAAC,MAAM,2BAA2B,GAAG,aAAa,CACtD,IAAI,GAAG,CAAC,kBAAkB,EAAE,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAC7C,CAAA;AAED,mFAAmF;AACnF,MAAM,CAAC,MAAM,qBAAqB,GAAG,YAAY,CAC/C,IAAI,GAAG,CAAC,0BAA0B,EAAE,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,EACpD,MAAM,CACP,CAAA;AAED,gFAAgF;AAChF,MAAM,CAAC,MAAM,mBAAmB,GAAsB;IACpD,IAAI,EAAE,kBAAkB;IACxB,WAAW,EAAE,mOAAmO;IAChP,SAAS,EAAE,6EAA6E;IACxF,MAAM,EAAE,SAAS;IACjB,YAAY,EAAE;QACZ,IAAI,EAAE,WAAW;QACjB,IAAI,EAAE,2BAA2B;KAClC;IACD,OAAO,EAAE,qBAAqB;CAC/B,CAAA"}
|
package/lib/types/exposure.d.ts
CHANGED
|
@@ -9,6 +9,10 @@ import { type ToolDefinition } from '@deepseek-ai/dsh-tools';
|
|
|
9
9
|
import type { Context } from '@deepseek-ai/cordis';
|
|
10
10
|
/** Small bootstrap tool retained only until the current Agent gains visual tools. */
|
|
11
11
|
export declare const VISION_TOOLKIT_ACTIVATE = "vision_toolkit_activate";
|
|
12
|
+
/** Skill name used by releases before the rename to vision-skills. */
|
|
13
|
+
export declare const LEGACY_VISION_TOOLS_SKILL_NAME = "vision-tools";
|
|
14
|
+
/** Unique pre-rename line in bundled instructions, kept for Session restore. */
|
|
15
|
+
export declare const LEGACY_VISION_TOOLS_SKILL_MARKER = "If this content arrived through a direct `/vision-tools` invocation and the";
|
|
12
16
|
/** Result returned by the one-shot activation transport. */
|
|
13
17
|
export interface VisionToolkitActivationResult {
|
|
14
18
|
activated: boolean;
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"exposure.d.ts","sourceRoot":"","sources":["../../src/exposure.ts"],"names":[],"mappings":"AAAA;;;;;;GAMG;AAKH,OAAO,EAAc,KAAK,cAAc,EAAE,MAAM,wBAAwB,CAAA;AACxE,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,qBAAqB,CAAA;AAIlD,qFAAqF;AACrF,eAAO,MAAM,uBAAuB,4BAA4B,CAAA;
|
|
1
|
+
{"version":3,"file":"exposure.d.ts","sourceRoot":"","sources":["../../src/exposure.ts"],"names":[],"mappings":"AAAA;;;;;;GAMG;AAKH,OAAO,EAAc,KAAK,cAAc,EAAE,MAAM,wBAAwB,CAAA;AACxE,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,qBAAqB,CAAA;AAIlD,qFAAqF;AACrF,eAAO,MAAM,uBAAuB,4BAA4B,CAAA;AAEhE,sEAAsE;AACtE,eAAO,MAAM,8BAA8B,iBAAiB,CAAA;AAE5D,gFAAgF;AAChF,eAAO,MAAM,gCAAgC,gFAAgF,CAAA;AAS7H,4DAA4D;AAC5D,MAAM,WAAW,6BAA6B;IAC5C,SAAS,EAAE,OAAO,CAAA;IAClB,KAAK,EAAE,MAAM,EAAE,CAAA;CAChB;AA8ED;;;;;GAKG;AACH,qBAAa,kBAAkB;IAU3B,OAAO,CAAC,QAAQ,CAAC,GAAG;IACpB,OAAO,CAAC,QAAQ,CAAC,WAAW;IAV9B,QAAQ,CAAC,cAAc,EAAE,cAAc,CAAA;IACvC,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAkC;IACzD,OAAO,CAAC,SAAS,CAAQ;IAEzB;;;OAGG;gBAEgB,GAAG,EAAE,OAAO,EACZ,WAAW,EAAE,MAAM,cAAc,EAAE;IA6BtD,uEAAuE;IACvE,OAAO,IAAI,MAAM,IAAI;IAoCrB,OAAO,CAAC,MAAM;IAMd,OAAO,CAAC,QAAQ;IA0BhB,6EAA6E;IAC7E,OAAO,CAAC,aAAa;IAIrB,OAAO,CAAC,6BAA6B;IAQrC,OAAO,CAAC,mBAAmB;IAM3B,OAAO,CAAC,MAAM;IAOd,OAAO,CAAC,aAAa;IAKrB,OAAO,CAAC,YAAY;CAIrB"}
|
package/lib/types/index.d.ts
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
* @anionex/dsh-vision-toolkit — DSH Vision Toolkit profile bundle.
|
|
3
3
|
*
|
|
4
4
|
* Plugin lifecycle follows the documented readiness chain: verify the pinned
|
|
5
|
-
* upstream checkout, publish the vision-
|
|
5
|
+
* upstream checkout, publish the vision-skills Skill and its one-shot bootstrap,
|
|
6
6
|
* then mount the execution tools only in Agents that load that Skill or invoke
|
|
7
7
|
* the bootstrap. Any
|
|
8
8
|
* failure leaves no model capability behind, and disposal unregisters every
|
package/lib/types/skill.d.ts
CHANGED
|
@@ -4,11 +4,11 @@
|
|
|
4
4
|
*/
|
|
5
5
|
import type { SkillRegistration } from '@deepseek-ai/dsh-skill';
|
|
6
6
|
/** Stable catalog/invocation name shared with progressive tool exposure. */
|
|
7
|
-
export declare const
|
|
7
|
+
export declare const VISION_SKILLS_NAME = "vision-skills";
|
|
8
8
|
/** Packaged resource root for the adapted upstream playbooks. */
|
|
9
|
-
export declare const
|
|
9
|
+
export declare const VISION_SKILLS_RESOURCE_BASE: string;
|
|
10
10
|
/** Exact bundled instructions used as the progressive-exposure evidence marker. */
|
|
11
|
-
export declare const
|
|
11
|
+
export declare const VISION_SKILLS_CONTENT: string;
|
|
12
12
|
/** Runtime skill registration mounted only after every native tool is ready. */
|
|
13
|
-
export declare const
|
|
13
|
+
export declare const VISION_SKILLS_SKILL: SkillRegistration;
|
|
14
14
|
//# sourceMappingURL=skill.d.ts.map
|
package/lib/types/skill.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"skill.d.ts","sourceRoot":"","sources":["../../src/skill.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAIH,OAAO,KAAK,EAAE,iBAAiB,EAAE,MAAM,wBAAwB,CAAA;AAE/D,4EAA4E;AAC5E,eAAO,MAAM,
|
|
1
|
+
{"version":3,"file":"skill.d.ts","sourceRoot":"","sources":["../../src/skill.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAIH,OAAO,KAAK,EAAE,iBAAiB,EAAE,MAAM,wBAAwB,CAAA;AAE/D,4EAA4E;AAC5E,eAAO,MAAM,kBAAkB,kBAAkB,CAAA;AAEjD,iEAAiE;AACjE,eAAO,MAAM,2BAA2B,QAEvC,CAAA;AAED,mFAAmF;AACnF,eAAO,MAAM,qBAAqB,QAGjC,CAAA;AAED,gFAAgF;AAChF,eAAO,MAAM,mBAAmB,EAAE,iBAUjC,CAAA"}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@anionex/dsh-vision-toolkit",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.31",
|
|
4
4
|
"description": "DeepSeek Harness-native integration for agent-vision-toolkit: image Q&A, OCR, grounding, UI restoration, pixel diff, Artifacts, and Web UI.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"deepseek",
|
|
@@ -10,6 +10,7 @@
|
|
|
10
10
|
"text-only-llm",
|
|
11
11
|
"vision-language-model",
|
|
12
12
|
"vision-tools",
|
|
13
|
+
"vision-skills",
|
|
13
14
|
"computer-vision",
|
|
14
15
|
"ocr",
|
|
15
16
|
"ui-restoration",
|
|
@@ -16,7 +16,8 @@ index 8f42552..f2ee3f2 100644
|
|
|
16
16
|
- image yourself when a description you were given lacks a detail.
|
|
17
17
|
----
|
|
18
18
|
-
|
|
19
|
-
|
|
19
|
+
-# vision-tools
|
|
20
|
+
+# vision-skills
|
|
20
21
|
|
|
21
22
|
-Five local CLIs that give a text-only agent eyes. They read one shared
|
|
22
23
|
-vision config (`VISION_API_KEY` / `VISION_BASE_URL` / `VISION_MODEL` /
|
|
@@ -29,7 +30,7 @@ index 8f42552..f2ee3f2 100644
|
|
|
29
30
|
+
|
|
30
31
|
+The visual execution schemas are mounted only for the current Agent after this
|
|
31
32
|
+Skill is loaded. A normal `skill` call activates them for the next model step.
|
|
32
|
-
+If this content arrived through a direct `/vision-
|
|
33
|
+
+If this content arrived through a direct `/vision-skills` invocation and the
|
|
33
34
|
+visual tools are still absent, call `vision_toolkit_activate` once. Do not call
|
|
34
35
|
+that bootstrap when the visual tools are already present.
|
|
35
36
|
|
|
@@ -802,10 +803,10 @@ index 3261fcf..68025be 100644
|
|
|
802
803
|
nodes, labels, groups, edges, directions, and edge labels.
|
|
803
804
|
6. Generate the Mermaid, Graphviz, or requested structure from that inventory.
|
|
804
805
|
diff --git a/references/restore-ui.md b/references/restore-ui.md
|
|
805
|
-
index f9ec11a..
|
|
806
|
+
index f9ec11a..69627895161fc1b57c6241665255fdf86ae1c1e5 100644
|
|
806
807
|
--- a/references/restore-ui.md
|
|
807
808
|
+++ b/references/restore-ui.md
|
|
808
|
-
@@ -28,
|
|
809
|
+
@@ -28,17 +28,18 @@
|
|
809
810
|
1. Inspect the existing frontend stack, component library, icon set, source
|
|
810
811
|
assets, and design tokens before writing code, but stop searching as soon as
|
|
811
812
|
a usable local primitive is found.
|
|
@@ -824,12 +825,27 @@ index f9ec11a..de97a02 100644
|
|
|
824
825
|
starting the next call. Normally one or two rounds are enough.
|
|
825
826
|
-4. Do not use `trace`, foreground extraction, repeated color sampling, or
|
|
826
827
|
- iterative `pixel_diff` work in fast mode. Those are fidelity tools and will
|
|
828
|
+
- consume the delivery window.
|
|
827
829
|
+4. Do not use `vision_trace`, foreground extraction, repeated color sampling,
|
|
828
|
-
+
|
|
829
|
-
|
|
830
|
+
+ iterative `vision_pixel_diff` work, or hand-written SVG in fast mode. Those
|
|
831
|
+
+ are fidelity tools and will consume the delivery window. Icons and decorative
|
|
832
|
+
+ marks stay library-based or screenshot-backed; never hand-write SVG code.
|
|
830
833
|
|
|
831
834
|
### Build the approximation
|
|
832
|
-
|
|
835
|
+
|
|
836
|
+
@@ -47,8 +48,9 @@
|
|
837
|
+
when zoomed in.
|
|
838
|
+
2. Reuse the project's existing components and CSS tokens. If its frontend or
|
|
839
|
+
icon library contains a reasonably similar component or icon, use it
|
|
840
|
+
- directly instead of recreating the reference. Prefer an approximate library
|
|
841
|
+
- icon over cropping, tracing, or hand-drawing a new one.
|
|
842
|
+
+ directly instead of recreating the reference. Never hand-write SVG in fast
|
|
843
|
+
+ mode: use an approximate library icon, or extract the original pixels as a
|
|
844
|
+
+ screenshot-backed asset.
|
|
845
|
+
3. Use nearby existing palette tokens or visually similar CSS values. Exact
|
|
846
|
+
sampled hex values, gradients, subtle borders, and shadow opacity are out of
|
|
847
|
+
scope unless one of them defines the whole composition.
|
|
848
|
+
@@ -57,7 +59,7 @@
|
|
833
849
|
|
|
834
850
|
### Render once, fix once, deliver
|
|
835
851
|
|
|
@@ -838,7 +854,7 @@ index f9ec11a..de97a02 100644
|
|
|
838
854
|
existing browser setup.
|
|
839
855
|
2. Inspect the screenshot once. If there is an obvious structural failure such
|
|
840
856
|
as a missing major region, broken wrapping, or a wildly wrong scale, make
|
|
841
|
-
@@ -66,7 +
|
|
857
|
+
@@ -66,7 +68,7 @@
|
|
842
858
|
small color, icon, font, shadow, radius, or spacing differences.
|
|
843
859
|
|
|
844
860
|
A practical time box is roughly 30 seconds for project inspection plus
|
|
@@ -847,7 +863,7 @@ index f9ec11a..de97a02 100644
|
|
|
847
863
|
rendering, one correction, and screenshot delivery. Dependency installation or
|
|
848
864
|
a project that does not already run may extend that target; do not compensate
|
|
849
865
|
by silently switching back to a long precision loop.
|
|
850
|
-
@@ -112,18 +
|
|
866
|
+
@@ -112,18 +114,19 @@
|
|
851
867
|
### 2. Establish the coordinate system
|
|
852
868
|
|
|
853
869
|
Record the reference image dimensions and the target viewport. Screenshots may
|
|
@@ -875,7 +891,7 @@ index f9ec11a..de97a02 100644
|
|
|
875
891
|
|
|
876
892
|
Treat model boxes as approximate handles. Use them to organize the page, but
|
|
877
893
|
do not mistake their final few pixels for measured boundaries.
|
|
878
|
-
@@ -141,15 +
|
|
894
|
+
@@ -141,15 +144,11 @@
|
|
879
895
|
extraction, tighten the final region to the visual's own ink; adjacent text or
|
|
880
896
|
rules inside the region can become foreground components too.
|
|
881
897
|
|
|
@@ -896,7 +912,7 @@ index f9ec11a..de97a02 100644
|
|
|
896
912
|
```
|
|
897
913
|
|
|
898
914
|
Inspect the transparent result before use. Confirm that the whole visual is
|
|
899
|
-
@@ -162,8 +
|
|
915
|
+
@@ -162,8 +161,8 @@
|
|
900
916
|
- Follow the project's existing framework and component patterns.
|
|
901
917
|
- Build content, interaction, responsive layout, borders, shadows, and simple
|
|
902
918
|
shapes natively.
|
|
@@ -907,7 +923,7 @@ index f9ec11a..de97a02 100644
|
|
|
907
923
|
- Place extracted assets with explicit logical dimensions. Preserve their
|
|
908
924
|
aspect ratio and avoid baking surrounding whitespace into the asset.
|
|
909
925
|
- Match structure and proportions before tuning small spacing.
|
|
910
|
-
@@ -172,10 +
|
|
926
|
+
@@ -172,10 +171,10 @@
|
|
911
927
|
|
|
912
928
|
For an existing implementation, start here.
|
|
913
929
|
|
|
@@ -920,7 +936,7 @@ index f9ec11a..de97a02 100644
|
|
|
920
936
|
spot or explain.
|
|
921
937
|
3. Fix material discrepancies: missing or wrong content, incorrect hierarchy,
|
|
922
938
|
broken wrapping, visibly wrong alignment or scale, wrong component state,
|
|
923
|
-
@@ -185,7 +
|
|
939
|
+
@@ -185,7 +184,7 @@
|
|
924
940
|
rasterization, antialiasing, subpixel placement, or another imperceptible
|
|
925
941
|
rendering detail.
|
|
926
942
|
|
package/src/exposure.ts
CHANGED
|
@@ -11,12 +11,18 @@ import type { ContentBlock } from '@deepseek-ai/dsh-llm'
|
|
|
11
11
|
import type { Session } from '@deepseek-ai/dsh-session'
|
|
12
12
|
import { defineTool, type ToolDefinition } from '@deepseek-ai/dsh-tools'
|
|
13
13
|
import type { Context } from '@deepseek-ai/cordis'
|
|
14
|
-
import {
|
|
14
|
+
import { VISION_SKILLS_CONTENT, VISION_SKILLS_NAME } from './skill.ts'
|
|
15
15
|
import { VISION_TOOL_NAMES } from './tools.ts'
|
|
16
16
|
|
|
17
17
|
/** Small bootstrap tool retained only until the current Agent gains visual tools. */
|
|
18
18
|
export const VISION_TOOLKIT_ACTIVATE = 'vision_toolkit_activate'
|
|
19
19
|
|
|
20
|
+
/** Skill name used by releases before the rename to vision-skills. */
|
|
21
|
+
export const LEGACY_VISION_TOOLS_SKILL_NAME = 'vision-tools'
|
|
22
|
+
|
|
23
|
+
/** Unique pre-rename line in bundled instructions, kept for Session restore. */
|
|
24
|
+
export const LEGACY_VISION_TOOLS_SKILL_MARKER = 'If this content arrived through a direct `/vision-tools` invocation and the'
|
|
25
|
+
|
|
20
26
|
interface AgentExposure {
|
|
21
27
|
active: boolean
|
|
22
28
|
liftRestriction?: () => void
|
|
@@ -38,8 +44,16 @@ function isRecord(value: unknown): value is Record<string, unknown> {
|
|
|
38
44
|
return value !== null && typeof value === 'object' && !Array.isArray(value)
|
|
39
45
|
}
|
|
40
46
|
|
|
47
|
+
function isBundledSkillName(name: unknown): boolean {
|
|
48
|
+
return name === VISION_SKILLS_NAME || name === LEGACY_VISION_TOOLS_SKILL_NAME
|
|
49
|
+
}
|
|
50
|
+
|
|
51
|
+
function isBundledSkillContent(text: string): boolean {
|
|
52
|
+
return text.includes(VISION_SKILLS_CONTENT) || text.includes(LEGACY_VISION_TOOLS_SKILL_MARKER)
|
|
53
|
+
}
|
|
54
|
+
|
|
41
55
|
function isVisionSkillArguments(value: unknown): boolean {
|
|
42
|
-
return isRecord(value) && value.name
|
|
56
|
+
return isRecord(value) && isBundledSkillName(value.name)
|
|
43
57
|
}
|
|
44
58
|
|
|
45
59
|
function nativeSkillCall(raw: string): boolean {
|
|
@@ -54,13 +68,14 @@ function containsBundledSkillContent(blocks: readonly unknown[]): boolean {
|
|
|
54
68
|
return blocks.some(block => isRecord(block)
|
|
55
69
|
&& block.type === 'text'
|
|
56
70
|
&& typeof block.text === 'string'
|
|
57
|
-
&& block.text
|
|
71
|
+
&& isBundledSkillContent(block.text))
|
|
58
72
|
}
|
|
59
73
|
|
|
60
74
|
function isBundledSkillResult(value: unknown): boolean {
|
|
61
75
|
return isRecord(value)
|
|
62
|
-
&& value.name
|
|
63
|
-
&& value.content ===
|
|
76
|
+
&& isBundledSkillName(value.name)
|
|
77
|
+
&& typeof value.content === 'string'
|
|
78
|
+
&& isBundledSkillContent(value.content)
|
|
64
79
|
}
|
|
65
80
|
|
|
66
81
|
/** Whether durable history proves that this Session loaded the bundled Skill. */
|
|
@@ -70,7 +85,7 @@ function hasLoadedVisionSkill(session: Session): boolean {
|
|
|
70
85
|
if (event.type === 'user/message') {
|
|
71
86
|
const source = event.data.source
|
|
72
87
|
if (source.kind === 'skill-invocation'
|
|
73
|
-
&& source.name
|
|
88
|
+
&& isBundledSkillName(source.name)
|
|
74
89
|
&& containsBundledSkillContent(event.data.content)) return true
|
|
75
90
|
continue
|
|
76
91
|
}
|
|
@@ -119,7 +134,7 @@ export class VisionToolExposure {
|
|
|
119
134
|
this.activationTool = defineTool({
|
|
120
135
|
name: VISION_TOOLKIT_ACTIVATE,
|
|
121
136
|
description: `Activate the independent Vision Toolkit execution tools for this Agent: ${Object.values(VISION_TOOL_NAMES).join(', ')}. `
|
|
122
|
-
+ `Loading the ${
|
|
137
|
+
+ `Loading the ${VISION_SKILLS_NAME} Skill normally activates them automatically; call this once when the visual tools are still absent, then use them for image understanding, OCR, UI detection, and related tasks. `
|
|
123
138
|
+ 'It is safe to call before the Skill is loaded, and this activation tool disappears after success.',
|
|
124
139
|
parameters: {},
|
|
125
140
|
output: {
|
package/src/index.ts
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
* @anionex/dsh-vision-toolkit — DSH Vision Toolkit profile bundle.
|
|
3
3
|
*
|
|
4
4
|
* Plugin lifecycle follows the documented readiness chain: verify the pinned
|
|
5
|
-
* upstream checkout, publish the vision-
|
|
5
|
+
* upstream checkout, publish the vision-skills Skill and its one-shot bootstrap,
|
|
6
6
|
* then mount the execution tools only in Agents that load that Skill or invoke
|
|
7
7
|
* the bootstrap. Any
|
|
8
8
|
* failure leaves no model capability behind, and disposal unregisters every
|
|
@@ -24,7 +24,7 @@ import {
|
|
|
24
24
|
import { VisionToolExposure } from './exposure.ts'
|
|
25
25
|
import { createPasteTakeoverResolver, installImageInputVariants } from './image-input-variants.ts'
|
|
26
26
|
import { VisionToolkitRuntimeManager } from './runtime-manager.ts'
|
|
27
|
-
import {
|
|
27
|
+
import { VISION_SKILLS_SKILL } from './skill.ts'
|
|
28
28
|
import { createVisionTools } from './tools.ts'
|
|
29
29
|
import { PLUGIN_VERSION } from './version.ts'
|
|
30
30
|
import { installVisionToolkitWeb, VisionToolkitWebBackend } from './web.ts'
|
|
@@ -65,7 +65,7 @@ export async function apply(ctx: Context, config: VisionToolkitConfig = {}): Pro
|
|
|
65
65
|
let skill: (() => void) | undefined
|
|
66
66
|
try {
|
|
67
67
|
activationTool = ctx.tools.register(exposure.activationTool)
|
|
68
|
-
skill = ctx.skills.register(
|
|
68
|
+
skill = ctx.skills.register(VISION_SKILLS_SKILL)
|
|
69
69
|
exposureDisposer = exposure.install()
|
|
70
70
|
operationalDisposers = { activationTool, exposure: exposureDisposer, skill }
|
|
71
71
|
const info = manager.current().upstreamVersion
|
|
@@ -90,7 +90,7 @@ export async function apply(ctx: Context, config: VisionToolkitConfig = {}): Pro
|
|
|
90
90
|
} catch (error) {
|
|
91
91
|
const message = error instanceof Error ? error.message : String(error)
|
|
92
92
|
ctx.logger.error(
|
|
93
|
-
'dsh-vision-toolkit %s: runtime not ready; the vision-
|
|
93
|
+
'dsh-vision-toolkit %s: runtime not ready; the vision-skills skill, activation bootstrap, and Agent-scoped visual tools are NOT registered. Settings remain available for repair. %s',
|
|
94
94
|
PLUGIN_VERSION,
|
|
95
95
|
message,
|
|
96
96
|
)
|
package/src/skill.ts
CHANGED
|
@@ -8,28 +8,28 @@ import { fileURLToPath } from 'node:url'
|
|
|
8
8
|
import type { SkillRegistration } from '@deepseek-ai/dsh-skill'
|
|
9
9
|
|
|
10
10
|
/** Stable catalog/invocation name shared with progressive tool exposure. */
|
|
11
|
-
export const
|
|
11
|
+
export const VISION_SKILLS_NAME = 'vision-skills'
|
|
12
12
|
|
|
13
13
|
/** Packaged resource root for the adapted upstream playbooks. */
|
|
14
|
-
export const
|
|
14
|
+
export const VISION_SKILLS_RESOURCE_BASE = fileURLToPath(
|
|
15
15
|
new URL('../assets/skill/', import.meta.url),
|
|
16
16
|
)
|
|
17
17
|
|
|
18
18
|
/** Exact bundled instructions used as the progressive-exposure evidence marker. */
|
|
19
|
-
export const
|
|
19
|
+
export const VISION_SKILLS_CONTENT = readFileSync(
|
|
20
20
|
new URL('../assets/skill/SKILL.md', import.meta.url),
|
|
21
21
|
'utf8',
|
|
22
22
|
)
|
|
23
23
|
|
|
24
24
|
/** Runtime skill registration mounted only after every native tool is ready. */
|
|
25
|
-
export const
|
|
26
|
-
name:
|
|
25
|
+
export const VISION_SKILLS_SKILL: SkillRegistration = {
|
|
26
|
+
name: VISION_SKILLS_NAME,
|
|
27
27
|
description: '把截图或设计图还原为 UI(网页、应用界面或组件),生成 HTML/CSS 或项目现有前端代码;也支持图片问答、长截图/聊天记录 OCR、元素定位与盘点、裁剪、前景提取、像素对比、取色、SVG 描摹和 HTML 转截图。当任务涉及图片理解、UI 还原/重建(rebuild or restore a UI from a screenshot)、视觉回归、像素坐标、基于截图的 GUI 操作、可复用图片/SVG 素材、图表还原或长截图 OCR 时使用。',
|
|
28
28
|
whenToUse: '任务依赖图片文字/内容、像素坐标、截图转 UI 重建、视觉回归、可复用图片/SVG 素材、图表还原、基于截图的 GUI 操作或长截图 OCR 时使用。',
|
|
29
29
|
source: 'runtime',
|
|
30
30
|
resourceBase: {
|
|
31
31
|
kind: 'directory',
|
|
32
|
-
path:
|
|
32
|
+
path: VISION_SKILLS_RESOURCE_BASE,
|
|
33
33
|
},
|
|
34
|
-
content:
|
|
34
|
+
content: VISION_SKILLS_CONTENT,
|
|
35
35
|
}
|