@anionex/dsh-vision-toolkit 0.1.6 → 0.1.8

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (41) hide show
  1. package/README.i18n.yaml +2 -2
  2. package/README.md +23 -11
  3. package/README.zh.md +23 -11
  4. package/lib/client.js +391 -32
  5. package/lib/client.js.map +1 -1
  6. package/lib/config.js +14 -0
  7. package/lib/config.js.map +1 -1
  8. package/lib/image-input-variants.js +615 -0
  9. package/lib/image-input-variants.js.map +1 -0
  10. package/lib/index.js +8 -1
  11. package/lib/index.js.map +1 -1
  12. package/lib/paste-images.js +6 -0
  13. package/lib/paste-images.js.map +1 -1
  14. package/lib/types/client/index.d.ts +13 -5
  15. package/lib/types/client/index.d.ts.map +1 -1
  16. package/lib/types/client/paste-images.d.ts +69 -0
  17. package/lib/types/client/paste-images.d.ts.map +1 -1
  18. package/lib/types/config.d.ts +26 -0
  19. package/lib/types/config.d.ts.map +1 -1
  20. package/lib/types/image-input-variants.d.ts +153 -0
  21. package/lib/types/image-input-variants.d.ts.map +1 -0
  22. package/lib/types/index.d.ts.map +1 -1
  23. package/lib/types/paste-images.d.ts +35 -0
  24. package/lib/types/paste-images.d.ts.map +1 -1
  25. package/lib/types/web-request.d.ts +7 -0
  26. package/lib/types/web-request.d.ts.map +1 -1
  27. package/lib/types/web.d.ts +18 -2
  28. package/lib/types/web.d.ts.map +1 -1
  29. package/lib/web-request.js +11 -2
  30. package/lib/web-request.js.map +1 -1
  31. package/lib/web.js +142 -11
  32. package/lib/web.js.map +1 -1
  33. package/package.json +2 -1
  34. package/src/client/index.tsx +95 -20
  35. package/src/client/paste-images.tsx +331 -15
  36. package/src/config.ts +40 -0
  37. package/src/image-input-variants.ts +688 -0
  38. package/src/index.ts +18 -1
  39. package/src/paste-images.ts +39 -0
  40. package/src/web-request.ts +12 -2
  41. package/src/web.ts +158 -11
package/README.i18n.yaml CHANGED
@@ -2,5 +2,5 @@
2
2
  # side as of the last confirmed-consistent state. Both languages carry equal authority;
3
3
  # after editing either side, bring the other along and re-record with:
4
4
  # pnpm run verify-translation-pairing --write dsh-vision-dark-theme/README.md
5
- README.md: 60cdb5fc0cd1cc9d6d5249fcdd021677c1618eff
6
- README.zh.md: 3d6d056e4ba2d1a86683b71b2a40dcb5e5231fef
5
+ README.md: 73215f08bb8e697ddc9c49cc4e1a5491a0f4c735
6
+ README.zh.md: 8a8979e5b3ea6e03e58b51dea8ce49b0d1e218ea
package/README.md CHANGED
@@ -2,9 +2,10 @@
2
2
 
3
3
  # DSH Vision Toolkit
4
4
 
5
+ [![Top at dshfind](https://img.shields.io/badge/top%20at-dshfind-FFD700?style=flat-square)](https://dshfind.com/)
5
6
  [![X (Twitter)](https://img.shields.io/badge/-@anion__ex-000000?style=flat-square&logo=x&logoColor=white)](https://x.com/anion_ex)
6
- [![Release v0.1.6](https://img.shields.io/badge/release-v0.1.6-5B4CF0?style=flat-square)](https://github.com/Anionex/dsh-vision-toolkit/releases/tag/v0.1.6)
7
- [![Verified: 162 tests](https://img.shields.io/badge/verified-162%20tests-2EA44F?style=flat-square)](tests)
7
+ [![Release v0.1.7](https://img.shields.io/badge/release-v0.1.7-5B4CF0?style=flat-square)](https://github.com/Anionex/dsh-vision-toolkit/releases/tag/v0.1.7)
8
+ [![Verified: 168 tests](https://img.shields.io/badge/verified-168%20tests-2EA44F?style=flat-square)](tests)
8
9
  [![License: MIT](https://img.shields.io/badge/license-MIT-0B7285?style=flat-square)](LICENSE)
9
10
  [![Node.js](https://img.shields.io/badge/Node.js-%5E22.19%20%7C%20%3E%3D24-339933?style=flat-square&logo=nodedotjs&logoColor=white)](package.json)
10
11
  [![Python](https://img.shields.io/badge/Python-3.11%2B-3776AB?style=flat-square&logo=python&logoColor=white)](runtime/requirements.lock)
@@ -145,6 +146,14 @@ Runtime readiness is profile-wide, but the ten visual execution schemas are Agen
145
146
 
146
147
  Health checks, connection testing, and plugin/upstream version inspection are administrative Web Settings operations. `vision_toolkit_health` and `vision_toolkit_version` are not model tools and never enter an Agent's schema, including after visual-tool activation.
147
148
 
149
+ ## Image-input variants for text-only models
150
+
151
+ Text-only model routes get sibling model-selector entries named `<model> (Vision Toolkit)` under a matching provider group. A variant declares image input, so pasted images keep the native attachment flow — composer thumbnail, durable session image, and history rendering — and the plugin rewrites every image block into a Vision Toolkit description only on the wire to the model, before the request reaches the upstream route. The session log is untouched; replay and the UI keep the real image.
152
+
153
+ A variant is registered automatically for every model the host positively declares text-only (for example the DeepSeek chat family). Paste handling is automatic: when the current model is confirmed text-only and its variant exists, the browser integration switches the session to the variant by itself (a short notice names the new model) and the paste then keeps the native flow; no manual model change is needed. The host's verdict uses the exact model route the browser read from the live model catalog, with the selector label as fallback; unconfirmed or image-capable routes always keep the native flow, and a text-only model without a variant (for example when variants are disabled) keeps the paste-to-path takeover, which copies the image into the session workspace and inserts its path as text.
154
+
155
+ Description conversion needs the configured vision provider and its credential; when the runtime is not ready or a read fails, the wire block degrades to an explanatory note instead of failing the turn. Disable variants with `imageInputVariants.enabled: false`, restrict the wrapped routes with `imageInputVariants.providers`, or keep the paste-to-path behavior for text-only models with `imageInputVariants.autoSwitch: false`.
156
+
148
157
  ## Requirements
149
158
 
150
159
  - DeepSeek Harness with a Web or Headless profile and `pnpm` available to `dsh plugin`.
@@ -191,7 +200,7 @@ dsh plugin --profile web remove @dsh-external/dsh-vision-toolkit
191
200
  dsh plugin --profile web add @anionex/dsh-vision-toolkit
192
201
  ```
193
202
 
194
- After restarting, Settings → Vision should report plugin version **0.1.6**.
203
+ After restarting, Settings → Vision should report plugin version **0.1.7**.
195
204
 
196
205
  For a registry installation, update the dependency through the profile package manager:
197
206
 
@@ -233,6 +242,10 @@ The bundle defaults to the managed runtime. A profile patch can override the pro
233
242
  runtime:
234
243
  mode: managed
235
244
  allowedDirs: []
245
+ imageInputVariants:
246
+ enabled: true
247
+ providers: []
248
+ autoSwitch: true
236
249
  ```
237
250
 
238
251
  ### Configuration fields
@@ -254,16 +267,15 @@ The bundle defaults to the managed runtime. A profile patch can override the pro
254
267
  | `runtime.agentVisionToolkitPath` | unset | Required in `external` mode; exported exact snapshot or clean pinned Git checkout |
255
268
  | `runtime.python` | unset | Optional Python 3.11+ bootstrap/interpreter override |
256
269
  | `allowedDirs` | `[]` | Additional realpath-resolved input roots; the session workspace is always allowed |
270
+ | `imageInputVariants.enabled` | `true` | Register image-input variant entries for text-only model routes in the model selector |
271
+ | `imageInputVariants.providers` | `[]` | Restrict wrapped upstream routes by provider id; empty wraps every eligible route |
272
+ | `imageInputVariants.autoSwitch` | `true` | Automatically switch a text-only session to its image-input variant on paste, so the image keeps the native flow; `false` keeps the paste-to-path takeover for text-only models |
257
273
 
258
274
  ### Credentials
259
275
 
260
- Create or replace the referenced secret through DSH Credentials:
261
-
262
- ```sh
263
- dsh credentials set VISION_API_KEY
264
- ```
276
+ The Web Settings page accepts the actual value in its write-only **API key** field. Leave that field blank to retain an existing key; saving a non-empty value writes it under the advanced **Credential name** reference, which defaults to `VISION_API_KEY`. Headless deployments can pre-provision the same reference in `$DSH_HOME/.credentials.yaml`.
265
277
 
266
- The reference is stored in Settings; the value is not. Remote operations resolve it once per call and inject it only into that subprocess environment. The plugin excludes user `.env` files, checkout `.env` files, `PYTHONPATH`, `PYTHONHOME`, `VIRTUAL_ENV`, and user site-packages so ambient Python or upstream configuration cannot override the selected DSH provider. Logs, errors, tool results, Artifact metadata, and Settings responses never contain the secret.
278
+ Settings store only the reference, never the value. The browser does not receive a stored value, and a successful save clears the field instead of echoing it. Remote operations resolve the reference once per call and inject the value only into that subprocess environment. The plugin excludes user `.env` files, checkout `.env` files, `PYTHONPATH`, `PYTHONHOME`, `VIRTUAL_ENV`, and user site-packages so ambient Python or upstream configuration cannot override the selected DSH provider. Logs, errors, tool results, Artifact metadata, and Settings responses never contain the secret.
267
279
 
268
280
  ### Managed and external runtimes
269
281
 
@@ -345,8 +357,8 @@ The committed evidence records an initial `6.04%` difference across six non-zero
345
357
 
346
358
  | Symptom | Resolution |
347
359
  |---|---|
348
- | `Model "..." does not support image input. (attachment-error)` | The image used DSH's native model-attachment channel, so a text-only model rejected the turn before the Skill or Vision Toolkit could run. Use DSH Paste Input's attachment button, paste, or drop flow so the file is copied into the session workspace and represented by a path, then invoke `/vision-tools`. Restart the Web profile and reload the page after installing or upgrading either browser plugin. |
349
- | Credential reported missing | Run `dsh credentials set <REF>`, ensure `provider.credential` names that reference, then rerun health. Local-only tools do not need it. |
360
+ | `Model "..." does not support image input. (attachment-error)` | The image used DSH's native model-attachment channel, so a text-only model rejected the turn before the Skill or Vision Toolkit could run. With image-input variants enabled this is rare: pasting normally auto-switches the session to the `<model> (Vision Toolkit)` variant. If variants are disabled or auto-switch is off, use DSH Paste Input's attachment button, paste, or drop flow so the file is copied into the session workspace and represented by a path, then invoke `/vision-tools`. Restart the Web profile and reload the page after installing or upgrading either browser plugin. |
361
+ | Credential reported missing | Paste the key into Web Settings **API key**, keep the advanced **Credential name** aligned with `provider.credential`, save, then rerun health. Headless deployments can provision the same reference in `$DSH_HOME/.credentials.yaml`. Local-only tools do not need it. |
350
362
  | Runtime preparation fails | Read the Settings runtime error, verify Python 3.11+, package-cache/network access, disk permissions, and the exact external pin. Save only after correcting the candidate; the active generation remains intact. |
351
363
  | Chrome is not found | Install Chrome, Chromium, or Edge or configure an environment where one is discoverable. Only `vision_html_screenshot` is unavailable. |
352
364
  | macOS displays a keychain dialog | Confirm the current built adapter is installed and no stale external `html_shot`/headless Chrome process is running. Current launches use a mock keychain and disposable profile; cancel the dialog rather than resetting the login keychain. |
package/README.zh.md CHANGED
@@ -2,9 +2,10 @@
2
2
 
3
3
  # DSH Vision Toolkit
4
4
 
5
+ [![推荐 at dshfind](https://img.shields.io/badge/%E6%8E%A8%E8%8D%90-dshfind-FFD700?style=flat-square)](https://dshfind.com/)
5
6
  [![X (Twitter)](https://img.shields.io/badge/-@anion__ex-000000?style=flat-square&logo=x&logoColor=white)](https://x.com/anion_ex)
6
- [![Release v0.1.6](https://img.shields.io/badge/release-v0.1.6-5B4CF0?style=flat-square)](https://github.com/Anionex/dsh-vision-toolkit/releases/tag/v0.1.6)
7
- [![Verified: 162 tests](https://img.shields.io/badge/verified-162%20tests-2EA44F?style=flat-square)](tests)
7
+ [![Release v0.1.7](https://img.shields.io/badge/release-v0.1.7-5B4CF0?style=flat-square)](https://github.com/Anionex/dsh-vision-toolkit/releases/tag/v0.1.7)
8
+ [![Verified: 168 tests](https://img.shields.io/badge/verified-168%20tests-2EA44F?style=flat-square)](tests)
8
9
  [![License: MIT](https://img.shields.io/badge/license-MIT-0B7285?style=flat-square)](LICENSE)
9
10
  [![Node.js](https://img.shields.io/badge/Node.js-%5E22.19%20%7C%20%3E%3D24-339933?style=flat-square&logo=nodedotjs&logoColor=white)](package.json)
10
11
  [![Python](https://img.shields.io/badge/Python-3.11%2B-3776AB?style=flat-square&logo=python&logoColor=white)](runtime/requirements.lock)
@@ -145,6 +146,14 @@ flowchart LR
145
146
 
146
147
  健康检查、连接测试以及插件/上游版本检查属于 Web Settings 管理操作。`vision_toolkit_health` 和 `vision_toolkit_version` 不是模型工具,即使视觉执行工具已经激活,也永远不会进入 Agent schema。
147
148
 
149
+ ## 纯文本模型的图片输入变体
150
+
151
+ 纯文本模型路由会获得同名的兄弟模型条目:`<模型名> (Vision Toolkit)`,挂在对应的提供方分组下。变体声明支持图片输入,因此粘贴的图片走原生附件流程——输入框缩略图、会话持久化图片与历史渲染全部保留——插件只在发往模型的请求链路上把每个图片块改写成 Vision Toolkit 描述文本,再转交上游路由。会话日志不被改动;回放与 UI 看到的始终是真实图片。
152
+
153
+ 插件会自动为宿主明确声明为纯文本的每个模型注册变体(例如 DeepSeek 对话家族)。粘贴处理是全自动的:当当前模型被确认为纯文本、且它的变体已注册时,浏览器端集成会自动把会话切换到变体(会有一条简短提示说明新模型名),随后粘贴走原生流程,无需手动切换模型。宿主依据浏览器从实时模型目录读到的精确模型路由来裁决,模型选择器标签作为兜底;无法确认或支持图片的路由一律保持原生流程,而"纯文本但没有变体"的模型(例如变体被关闭时)继续走"粘贴转路径":图片被复制进会话工作区,输入框里插入的是它的路径文本。
154
+
155
+ 描述转换需要已配置的视觉提供方及其 Credential;当运行时未就绪或读取失败时,请求链路上的图片块降级为说明文本,而不是让整轮失败。用 `imageInputVariants.enabled: false` 关闭变体,用 `imageInputVariants.providers` 限制被包装的路由,或用 `imageInputVariants.autoSwitch: false` 让纯文本模型继续走"粘贴转路径"。
156
+
148
157
  ## 运行要求
149
158
 
150
159
  - 启用 Web 或 Headless Profile 的 DeepSeek Harness,并确保 `dsh plugin` 可以使用 `pnpm`。
@@ -191,7 +200,7 @@ dsh plugin --profile web remove @dsh-external/dsh-vision-toolkit
191
200
  dsh plugin --profile web add @anionex/dsh-vision-toolkit
192
201
  ```
193
202
 
194
- 重启后,Settings → 视觉工具 应显示插件版本 **0.1.6**。
203
+ 重启后,Settings → 视觉工具 应显示插件版本 **0.1.7**。
195
204
 
196
205
  通过注册表安装时,使用 Profile 的包管理命令更新依赖:
197
206
 
@@ -233,6 +242,10 @@ Bundle 默认使用 managed 运行时。Profile patch 可以覆盖提供方与
233
242
  runtime:
234
243
  mode: managed
235
244
  allowedDirs: []
245
+ imageInputVariants:
246
+ enabled: true
247
+ providers: []
248
+ autoSwitch: true
236
249
  ```
237
250
 
238
251
  ### 配置字段
@@ -254,16 +267,15 @@ Bundle 默认使用 managed 运行时。Profile patch 可以覆盖提供方与
254
267
  | `runtime.agentVisionToolkitPath` | 未设置 | `external` 模式必填;必须是精确导出快照或固定 commit 的干净 Git checkout |
255
268
  | `runtime.python` | 未设置 | 可选的 Python 3.11+ 引导程序/解释器覆盖值 |
256
269
  | `allowedDirs` | `[]` | 额外的 realpath 解析输入根目录;会话工作区始终允许 |
270
+ | `imageInputVariants.enabled` | `true` | 为纯文本模型路由在模型选择器中注册图片输入变体条目 |
271
+ | `imageInputVariants.providers` | `[]` | 按提供方 id 限制被包装的上游路由;为空时包装所有符合条件的路由 |
272
+ | `imageInputVariants.autoSwitch` | `true` | 粘贴时自动把纯文本会话切换到其图片输入变体,让图片保持原生流程;`false` 时纯文本模型继续走"粘贴转路径" |
257
273
 
258
274
  ### Credential
259
275
 
260
- 通过 DSH Credentials 创建或替换引用指向的密钥:
261
-
262
- ```sh
263
- dsh credentials set VISION_API_KEY
264
- ```
276
+ Web 设置页的只写 **API 密钥** 输入框直接接收真实密钥。留空表示保留现有密钥;填写后保存,会把密钥写入高级设置中的 **凭据名称**,默认名称是 `VISION_API_KEY`。Headless 部署可以在 `$DSH_HOME/.credentials.yaml` 中预置同名引用。
265
277
 
266
- Settings 只保存引用,不保存值。每次远程操作都会重新解析引用,并只把值注入对应子进程环境。插件排除用户 `.env`、checkout `.env`、`PYTHONPATH`、`PYTHONHOME`、`VIRTUAL_ENV` 和用户 site-packages,避免环境中的 Python 或上游配置覆盖选定的 DSH 提供方。日志、错误、工具结果、产物元数据和 Settings 响应都不包含密钥。
278
+ Settings 只保存引用,不保存值。浏览器不会读取已保存的密钥,保存成功后输入框也会立即清空而不是回显。每次远程操作都会重新解析引用,并只把值注入对应子进程环境。插件排除用户 `.env`、checkout `.env`、`PYTHONPATH`、`PYTHONHOME`、`VIRTUAL_ENV` 和用户 site-packages,避免环境中的 Python 或上游配置覆盖选定的 DSH 提供方。日志、错误、工具结果、产物元数据和 Settings 响应都不包含密钥。
267
279
 
268
280
  ### Managed 与 external 运行时
269
281
 
@@ -345,8 +357,8 @@ npm run example:ui-restoration:write
345
357
 
346
358
  | 症状 | 解决方法 |
347
359
  |---|---|
348
- | `Model "..." does not support image input. (attachment-error)` | 图片走了 DSH 的模型原生附件通道,纯文本模型会在 Skill 或 Vision Toolkit 运行前拒绝该轮。请使用 DSH Paste Input 的附件按钮、粘贴或拖放流程,让文件先复制到会话工作区并以路径形式进入消息,再调用 `/vision-tools`。安装或升级任一浏览器插件后,需要重启 Web Profile 并刷新页面。 |
349
- | Credential 显示缺失 | 执行 `dsh credentials set <REF>`,确认 `provider.credential` 指向该引用,再重新运行健康检查。本地工具不需要它。 |
360
+ | `Model "..." does not support image input. (attachment-error)` | 图片走了 DSH 的模型原生附件通道,纯文本模型会在 Skill 或 Vision Toolkit 运行前拒绝该轮。启用图片输入变体时这很少发生:粘贴会自动把会话切换到 `<模型名> (Vision Toolkit)` 变体。若变体被关闭或自动切换被禁用,请使用 DSH Paste Input 的附件按钮、粘贴或拖放流程,让文件先复制到会话工作区并以路径形式进入消息,再调用 `/vision-tools`。安装或升级任一浏览器插件后,需要重启 Web Profile 并刷新页面。 |
361
+ | Credential 显示缺失 | 在 Web 设置页的 **API 密钥** 中粘贴密钥,确认高级设置中的 **凭据名称** 与 `provider.credential` 一致,保存后重新运行健康检查。Headless 部署可以在 `$DSH_HOME/.credentials.yaml` 中预置同名引用。本地工具不需要它。 |
350
362
  | 运行时准备失败 | 查看 Settings 中的运行时错误,检查 Python 3.11+、软件包缓存/网络、磁盘权限和精确 external 固定版本。修正候选后再保存;当前 generation 不受影响。 |
351
363
  | 找不到 Chrome | 安装 Chrome、Chromium 或 Edge,或让其中一个可被运行环境发现。只有 `vision_html_screenshot` 不可用。 |
352
364
  | macOS 弹出钥匙串对话框 | 确认安装的是当前构建产物,且没有遗留的外部 `html_shot`/headless Chrome 进程。当前启动使用 mock keychain 和一次性 profile;取消对话框,不要重置登录钥匙串。 |