dsh-vision-router 1.1.1 → 1.2.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -10,8 +10,13 @@
10
10
 
11
11
  <p align="center">
12
12
  <a href="https://awesome-dsh-plugin.com"><img src="https://awesome-dsh-plugin.com/badge.svg" alt="awesome · DSH plugin" /></a>
13
- <a href="https://github.com/ysr666/dsh-vision-router/releases/tag/v1.1.1"><img src="https://img.shields.io/badge/release-v1.1.1-5B4CF0?style=flat-square" alt="Release v1.1.1" /></a>
14
- <a href="tests"><img src="https://img.shields.io/badge/verified-101%20tests-2EA44F?style=flat-square" alt="Verified: 101 tests" /></a>
13
+ <a href="https://github.com/zp-home/dsh-recommend"><img src="https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fzp-home%2Fdsh-recommend%2Fmain%2Fdata%2Fbadges%2Fysr666__dsh-vision-router.certified.json&amp;style=flat-square" alt="dsh-recommend 🏅 精选认证" /></a>
14
+ <a href="https://github.com/zp-home/dsh-recommend"><img src="https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fzp-home%2Fdsh-recommend%2Fmain%2Fdata%2Fbadges%2Fysr666__dsh-vision-router.json&amp;style=flat-square" alt="dsh score 0.90" /></a>
15
+ </p>
16
+
17
+ <p align="center">
18
+ <a href="https://github.com/ysr666/dsh-vision-router/releases/tag/v1.2.1"><img src="https://img.shields.io/badge/release-v1.2.1-5B4CF0?style=flat-square" alt="Release v1.2.1" /></a>
19
+ <a href="tests"><img src="https://img.shields.io/badge/verified-149%20tests-2EA44F?style=flat-square" alt="Verified: 149 tests" /></a>
15
20
  <a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-2EA44F?style=flat-square" alt="License: MIT" /></a>
16
21
  <a href="package.json"><img src="https://img.shields.io/badge/Node.js-%3E%3D22-339933?style=flat-square&amp;logo=nodedotjs&amp;logoColor=white" alt="Node.js >=22" /></a>
17
22
  <img src="https://img.shields.io/badge/runtime-no%20Python-8A2BE2?style=flat-square" alt="No Python" />
@@ -20,10 +25,12 @@
20
25
 
21
26
  <p align="center">English · <a href="README.zh.md">中文</a></p>
22
27
 
28
+ <p align="center">💬 <strong>QQ community group: 1105463028</strong></p>
29
+
23
30
  > [!WARNING]
24
- > 📌 **Announcement (v1.1.1)**
31
+ > 📌 **Announcement (v1.2.1)**
25
32
  >
26
- > Existing DSH providers are now **auto-wrapped on install**: text-only routes gain paste-and-go vision automatically, while native multimodal models keep their original image input and use Vision Router tools only when useful. Windows sharp/libvips startup conflicts are fixed by sharing the host sharp; install/update docs now distinguish npm/npx from source-checkout pnpm usage.
33
+ > v1.2.1 hardens the pixel loop: all ten pixel tools now accept uploaded-image attachment ids directly (no more `cannot read …/sha256:…` round trips), artifact filenames carry collision-free fingerprints, `vision_ground` retries degenerate boxes, the model guide replays fully from step 1 (leaving the settings first), and the settings card scrolls smoothly even with hundreds of models per provider.
27
34
 
28
35
  <p align="center">
29
36
  <img src="assets/vision-demo.gif" width="640" alt="Demo: paste an image, the agent locates the send button with vision_ground / vision_crop / vision_pixel_diff and answers with coordinates" />
@@ -34,7 +41,7 @@
34
41
  Most DSH vision plugins bridge images to DeepSeek as *text descriptions* — lossy, one-shot, and blind to pixels. This plugin keeps the **original pixels on the vision model's side** and DeepSeek on the reasoning side, and makes looking at an image an **ordinary tool call**:
35
42
 
36
43
  - **One command install.** The package ships its own composition patch (`dsh.bundle.patch`): `dsh plugin add` wires the row, the admission wrapper and the attachment limits automatically — zero manual file edits. Taking over the official DeepSeek route is an optional setting (stealth mode, off by default).
37
- - **Free by default.** The vision chain starts with a built-in OVHcloud anonymous endpoint (`Qwen2.5-VL-72B-Instruct`, no account, no key, 2 req/min per IP). Paid chains (OpenRouter, Pi-AI providers, direct OpenAI-compatible endpoints) are optional upgrades.
44
+ - **Free by default.** Vision tools end with a five-model OVHcloud anonymous fallback: no account, no key, 2 requests/minute per IP per model, roughly 10 RPM in theory across independent buckets. User-provided vision models run first.
38
45
  - **No Python.** The whole pipeline — downscale, grounding, crop, pixel diff, palette, OCR, SVG trace, cutout, HTML screenshot — runs on sharp / potrace / tesseract / system Chrome.
39
46
  - **Continuous multi-step image work.** An image turn is a text turn that calls tools: `vision_ground` → `vision_crop` → `vision_describe` → `vision_pixel_diff` → fix → screenshot again. The agent keeps iterating until the work is done.
40
47
  - **DeepSeek stays the brain.** Text turns are untouched in model, cost and context. The vision model is only the eyes, called on demand; answers are cached by image content.
@@ -48,17 +55,19 @@ The closest alternative is [@anionex/dsh-vision-toolkit](https://github.com/Anio
48
55
  |---|---|---|
49
56
  | Image Q&A out of the box | ✅ Built-in free chain (anonymous OVHcloud endpoint) — no account, no key | Requires your own vision API key (local pixel tools work without one) |
50
57
  | Runtime | ✅ Node only — no Python | Python 3.11+ managed runtime |
51
- | Getting an image in | ✅ Paste it — the turn auto-routes to the vision chain and auto-mounts the tools | Workspace path + `/vision-tools` command, then explicit tool calls |
58
+ | Getting an image in | ✅ Pick a “+ Auto Vision” group once, then paste directly | Workspace path + `/vision-tools` command, then explicit tool calls |
52
59
  | Turn routing | ✅ Image turns switch to vision, text turns switch back to DeepSeek — optional stealth takeover keeps the model picker looking stock | Tool-driven; no whole-turn auto-routing |
53
60
  | Profiles | Web | Web + Headless |
54
61
  | Playbooks | The pixel loop: ground → crop → diff → fix → screenshot again | Richer case library (long-screenshot OCR, UI restoration, GUI automation) |
55
- | Tests | 86 | 162 |
62
+ | Tests | 144 | 162 |
56
63
  | Install | One command | One command (npm) |
57
64
 
58
65
  Both are MIT-licensed and one command away. Pick this plugin when you want images to *just work* with zero setup; pick theirs when you need headless profiles or the extended playbook library. (Feature comparison reflects their README as of 2026-08.)
59
66
 
60
67
  ## Quick start
61
68
 
69
+ ### 1. Install and load the plugin
70
+
62
71
  Recommended for normal npm/npx installs (the same launch style used by the DSH README):
63
72
 
64
73
  ```sh
@@ -74,14 +83,32 @@ pnpm dsh plugin --profile web add dsh-vision-router
74
83
  pnpm dsh web
75
84
  ```
76
85
 
77
- If you already installed the DSH CLI globally and `dsh` is on `PATH`, the shorter `dsh ...` form works too. Restart a long-lived Web profile after installation — done. Zero configuration:
86
+ If you already installed the DSH CLI globally and `dsh` is on `PATH`, the shorter `dsh ...` form works too.
87
+
88
+ > [!NOTE]
89
+ > If you install the plugin **into a Web process that was already running long-term**, let that DSH Web process reload once so the plugin bundle itself is discovered. After the plugin is loaded, adding/removing models or changing wrapper scope **hot-updates without further DSH restarts**.
90
+
91
+ ### 2. Switch to a “+ Auto Vision” model group in chat
92
+
93
+ Once loaded, the plugin discovers the model groups enabled under **Settings → Models** and creates same-name auto-vision entries. For example:
94
+
95
+ ```text
96
+ opencode-go ← original model group, unchanged
97
+ opencode-go + Auto Vision ← choose this when sending images
98
+ ```
99
+
100
+ > [!IMPORTANT]
101
+ > **Before sending an image, open the model selector in the lower-right corner of the chat composer and choose a group marked “+ Auto Vision”.**
102
+ >
103
+ > Vision Router deliberately **does not modify the original model group**. If the conversation still uses the original text-only opencode / DeepSeek route, DSH can reject the image with “the current model does not support images” *before Vision Router gets a chance to handle it*. That is a model-entry selection issue, not a broken vision backend.
104
+
105
+ The auto-vision group follows the live DSH model catalog. Adding models or changing wrapper scope does not require a restart.
106
+
107
+ ### 3. Paste or upload the image
78
108
 
79
- - the plugin's bundle patch mounts the row, adds the admission wrapper and relaxes attachment limits to 20 MB / 100 MP — pure-additive, it never touches the core rows; whether the official DeepSeek route is taken over is decided by the optional stealth setting (off by default);
80
- - the default vision chain is the built-in free endpoint;
81
- - custom/third-party routes (e.g. opencode) gain image input through **Extra vision wrappers**;
82
- - every setting is editable live in **Settings → Plugins → Plugin config → 视觉路由(自动识图)**.
109
+ After choosing the “+ Auto Vision” group, paste or upload an image normally. The agent auto-mounts the vision tools and can use `vision_describe`, `vision_ground`, `vision_crop`, and the rest across multiple steps when needed.
83
110
 
84
- Then just paste an image into a conversation. The agent mounts the vision tools automatically and looks at it through `vision_describe` (and friends) — multi-step if needed.
111
+ The built-in anonymous OVH vision fallback is already configured, so normal image use needs no signup or API key. **The lower-right chat picker selects only the brain/conversation model**; vision backends do not belong there. Advanced options live under **Settings → Plugins → Plugin config → 视觉路由(自动识图)**: each vision-backend row selects one image-capable user model already configured under **Settings → Models**. Leaving every user row empty is valid; the OVH chain remains the final fallback. `Vision HTTP` is an internal transport route, not a model group users should select.
85
112
 
86
113
  ### See it in action
87
114
 
@@ -103,9 +130,9 @@ Then just paste an image into a conversation. The agent mounts the vision tools
103
130
 
104
131
  ### Pixel loop in practice
105
132
 
106
- <p align="center">
107
- <img src="assets/pixel-loop.png" width="100%" alt="Reference design and final agent rebuild, verified with vision_pixel_diff at 2.54% final difference." />
108
- </p>
133
+ [![Reference design and final agent rebuild, verified with vision_pixel_diff at 2.54% final difference.](https://raw.githubusercontent.com/ysr666/dsh-vision-router/main/assets/pixel-loop.png)](https://raw.githubusercontent.com/ysr666/dsh-vision-router/main/assets/pixel-loop.png)
134
+
135
+ <p align="center"><sub>Click the image to open the full-resolution original.</sub></p>
109
136
 
110
137
  The agent rebuilt the UI from the reference image, then verified the final result with `vision_pixel_diff`: **2.54% final diff** (32,939 / 1,296,000 differing pixels, threshold 16/channel).
111
138
 
@@ -159,11 +186,14 @@ vision_long_screenshot_ocr image="chat-log.png" chunkHeight=1200 overlap=120
159
186
 
160
187
  ## Provider fallback chain
161
188
 
162
- The vision chain walks providers in order and only surfaces an error after every one failed:
189
+ The vision tools try backends in order and surface an error only after all of them fail:
163
190
 
164
- 1. the **built-in free endpoint** (`vision-http` → `ovh/Qwen2.5-VL-72B-Instruct`) — no key, best-effort, 2 req/min per IP;
165
- 2. configured `httpProviders` (direct OpenAI-compatible endpoints with optional `apiKeyEnv`);
166
- 3. configured `providers` / `provider` + `fallbacks` (any adapter-backed provider, e.g. a Pi-AI profile like OpenRouter or Zhipu).
191
+ 1. **User vision models**: one per settings row, top to bottom; only models under **Settings → Models** that explicitly declare image input are shown;
192
+ 2. **Advanced custom HTTP vision endpoints**: legacy/advanced `httpProviders`, when present, run after the user models;
193
+ 3. **Built-in anonymous OVH fallback**: always last and never exposed in a model picker. The current quality-first chain is `Qwen3.5-397B-A17B` → `Qwen2.5-VL-72B-Instruct` → `Qwen3.6-27B` → `Mistral-Small-3.2-24B-Instruct-2506` → `Qwen3.5-9B`. OVH anonymous limits are **2 requests/minute per IP per model**. The five models have independent buckets, so spreading requests across them is about **10 RPM in theory**, subject to OVH's actual rate limiting. No signup or API key is required.
194
+
195
+ > [!IMPORTANT]
196
+ > This “vision chain” is the **eyes** used by Vision Router: each settings row selects one user vision model, while the lower-right chat picker selects the **brain/conversation model**. The two are deliberately separate. Text-only DeepSeek/opencode models are filtered out of the vision-backend dropdown, and the internal `Vision HTTP` transport route is no longer exposed to users.
167
197
 
168
198
  > In the legacy `routing: true` mode, the whole-turn chain walks only `provider + fallbacks` — `httpProviders` (including the free fallback) do not participate there. The default `routing: false` (tools-first) tries everything.
169
199
 
@@ -183,21 +213,29 @@ With stealth on, the plugin takes over the official `deepseek-official` route: t
183
213
 
184
214
  With the stock row present, the plugin falls back to the visible wrapper entry. Conversely, with stealth off but the stock row still disabled, the plugin performs a keep-alive takeover so the DeepSeek models don't vanish (the settings card explains this); to restore the fully official route, flip the `disabled` above back to `false` and restart.
185
215
 
186
- > Stealth mode **only affects the official DeepSeek route**. Custom/third-party routes like opencode are unrelated — use **Extra vision wrappers** below to give them image input.
216
+ > Stealth mode **only affects the official DeepSeek route**. Custom/third-party routes such as opencode are auto-wrapped into “+ Auto Vision” groups by default.
217
+
218
+ ## Auto-vision model groups and manual wrappers
187
219
 
188
- ## Extra vision wrappers
220
+ `autoWrapProviders` is on by default. The plugin discovers the provider/model entries currently enabled under **Settings → Models** and registers a same-name “+ Auto Vision” model group for them. **The original group is never changed**: choose the auto-vision group for images, or keep using the original group for plain text. DSH `llm/adapters-updated` events are synced live, so adding/removing models does not require a restart.
189
221
 
190
- `wrappedProviders` registers an auto-vision twin for any third-party/custom text route: pick the twin in the model selector and send images; text turns delegate to the original route unchanged. The typical use case is a custom interface such as opencode — it only declares text input, and one wrapper row makes it image-ready. In the settings card, configure it with two dropdowns (provider + model); leaving the model empty wraps every model of that route.
222
+ `wrappedProviders` is an **optional manual scope control**, not a required setup step. Use it only when:
223
+
224
+ 1. automatic wrapping is off and you want to pick which provider/models receive an auto-vision entry; or
225
+ 2. automatic wrapping remains on but one provider should expose only selected models in its “+ Auto Vision” group.
226
+
227
+ The settings card uses provider + model dropdowns; an empty model means every model on that route. Add multiple rows to select multiple models. Changes apply immediately with no restart.
191
228
 
192
229
  ## Web settings
193
230
 
194
- The Web profile registers a **视觉路由(自动识图)** card under **Settings → Plugins → Plugin config**, styled like the built-in cards. It live-edits:
231
+ The Web profile registers a **视觉路由(自动识图)** card under **Settings → Plugins → Plugin config**. Its top callout spells out the only step most users need: **return to chat → lower-right model selector → choose a “+ Auto Vision” model group → send the image**. The remaining controls are advanced customization:
195
232
 
196
- - switches: whole-turn legacy routing, vision tools, image-block rewriting, stealth (official DeepSeek route only);
197
- - **extra vision wrappers**: provider + model dropdowns that register image-capable twin entries for custom routes such as opencode;
198
- - vision request timeout, wrapper/chain route names;
199
- - the **vision chain** (one `provider/model` per line, top-down fallback) and the text model;
200
- - every field shows an "overridden" badge with a one-click reset to the composition default, plus discard/save;
233
+ - **Auto-create “+ Auto Vision” model groups**: enabled by default; follows the live model catalog with no restart;
234
+ - **Manual auto-vision scope (optional)**: only for disabling auto-wrap or limiting selected models;
235
+ - **Vision backend chain**: the real image-capable models used by `vision_describe` and friends; the built-in free Qwen is normally enough, and text-only models should not be placed here;
236
+ - switches for legacy whole-turn routing, vision tools, image-block rewriting and stealth mode (official DeepSeek route only);
237
+ - timeout, wrapper/chain route names, proxy and other advanced parameters;
238
+ - every field shows an overridden badge with one-click reset plus discard/save;
201
239
  - a **Test connection** button probes the first vision provider and reports latency inline;
202
240
  - artifact-producing tools render dedicated call cards with result facts and open-file buttons.
203
241
 
@@ -213,15 +251,16 @@ Everything is optional; defaults work out of the box. Edit via the Web card or a
213
251
 
214
252
  | Field | Default | Meaning |
215
253
  |---|---|---|
216
- | `provider` / `model` | `vision-http` / `ovh/Qwen2.5-VL-72B-Instruct` | shorthand chain (adapter-backed provider + model) |
217
- | `fallbacks` | `[]` | backup models for the shorthand provider |
218
- | `providers` | built-in free `vision-http` pair | multi-provider chain `{ provider, model, fallbacks[] }`, tried in order; wins over the shorthand. The first row ships as the built-in free model |
254
+ | `provider` / `model` | `vision-http` / `ovh/Qwen2.5-VL-72B-Instruct` | shorthand **vision backend** route (adapter-backed provider + model that genuinely accepts images) |
255
+ | `fallbacks` | `[]` | backup image models for the shorthand vision provider |
256
+ | `providers` | built-in free `vision-http` pair | multi-provider **vision backend** chain `{ provider, model, fallbacks[] }`, tried in order; do not put text-only models here |
219
257
  | `httpProviders` | built-in OVH entry | direct OpenAI-compatible endpoints `{ name, baseURL, model, apiKeyEnv, maxTokens }` |
220
- | `wrappedProviders` | `[{ provider: 'deepseek-official', models: [] }]` | extra text routes to wrap as image-capable twins: `{ provider, models[] }` — registers an auto-vision twin for any custom/third-party route (e.g. opencode); in the card, provider + model dropdowns, empty model = wrap all. The pre-filled deepseek-official row marks the built-in wrapper and is a no-op; changes apply live |
258
+ | `autoWrapProviders` | `true` | discover enabled provider/models and live-sync same-name “+ Auto Vision” groups; original groups stay unchanged |
259
+ | `wrappedProviders` | `[{ provider: 'deepseek-official', models: [] }]` | optional manual wrapper scope `{ provider, models[] }`, used after disabling auto-wrap or to restrict one provider to selected models; changes apply live, no restart |
221
260
  | `routing` | `false` | legacy whole-turn chain routing (one-shot answer). `false` = tools-first flow (recommended) |
222
261
  | `reverseRouting` | `true` | with `routing: true`, route text turns back to `textProvider` |
223
262
  | `wrapperRoute` / `chainRoute` | `deepseek-vision` / `vision-chain` | admission wrapper route name / fallback chain route name (empty disables) |
224
- | `stealth` | `false` | take over the official `deepseek-official` route (official row only; custom routes use `wrappedProviders`) |
263
+ | `stealth` | `false` | take over the official `deepseek-official` route (official row only; custom routes are auto-wrapped by default) |
225
264
  | `textProvider` | `deepseek-official` / `deepseek-v4-pro` | the model that reasons (your daily model) |
226
265
  | `tool` / `progressiveTools` / `autoActivateOnImage` | `true` ×3 | vision tools on / progressive mounting / auto-mount on image turns |
227
266
  | `rewriteImages` | `true` | rewrite image blocks in the model input (cached description or tool-hint marker); the UI log keeps images |
@@ -257,7 +296,7 @@ pnpm dsh plugin --profile web add dsh-vision-router
257
296
  pnpm dsh --profile web --dump-config | grep vision-router
258
297
  ```
259
298
 
260
- Restart a long-lived Web profile. The host discovers the browser bundle through `dsh.client` at startup.
299
+ When first adding the plugin to an already long-lived Web profile, let that Web process reload the plugin bundle; the host discovers the browser bundle through `dsh.client` at startup. **After the plugin is loaded, model-catalog and wrapper-scope changes hot-update and do not require a restart.**
261
300
 
262
301
  ### Disable / re-enable
263
302
 
@@ -313,6 +352,37 @@ pnpm dsh plugin --profile web remove dsh-vision-router
313
352
 
314
353
  This removes the dependency and the bundle layer. If you disabled the stock DeepSeek row manually, re-enable it in your profile patch.
315
354
 
355
+ ## Troubleshooting
356
+
357
+ ### Startup fails with `Unexpected token ... is not valid JSON` (UTF-8 BOM)
358
+
359
+ **Symptom:** `dsh web` / `pnpm dsh web` exits immediately at startup:
360
+
361
+ ```
362
+ SyntaxError: Unexpected token ...
363
+ is not valid JSON
364
+ at JSON.parse (<anonymous>)
365
+ at readProfileManifest (packages/boot/app-boot/src/profile.ts)
366
+ ```
367
+
368
+ **Cause:** `~/.dsh/profiles/<profile>/package.json` was saved as **UTF-8 with BOM** by an editor. The invisible `\uFEFF` character at the start makes `JSON.parse` fail, because JSON does not allow it before the opening brace.
369
+
370
+ **Recommended fix:** run Vision Router's standalone repair command. It does not require DSH to boot first; it locates the profile, detects a UTF-8 BOM, removes only the three leading BOM bytes, and then validates the JSON again:
371
+
372
+ ```sh
373
+ npx dsh-vision-router repair --profile web
374
+ ```
375
+
376
+ To diagnose without changing the file:
377
+
378
+ ```sh
379
+ npx dsh-vision-router doctor --profile web
380
+ ```
381
+
382
+ Replace `web` if you use another profile, or omit `--profile` to scan all profiles.
383
+
384
+ Manual fallback: in VS Code, use “Save with Encoding” → `UTF-8` (without BOM). If `repair` removes the BOM but the JSON is still invalid, it will not guess or rewrite any other JSON content; inspect the file manually.
385
+
316
386
  ## Security notes
317
387
 
318
388
  - Image text is **untrusted evidence**: descriptions, OCR output and the auto-mount note all tell the agent never to execute instructions found inside images.
package/README.zh.md CHANGED
@@ -10,8 +10,13 @@
10
10
 
11
11
  <p align="center">
12
12
  <a href="https://awesome-dsh-plugin.com"><img src="https://awesome-dsh-plugin.com/badge.svg" alt="awesome · DSH plugin" /></a>
13
- <a href="https://github.com/ysr666/dsh-vision-router/releases/tag/v1.1.1"><img src="https://img.shields.io/badge/release-v1.1.1-5B4CF0?style=flat-square" alt="Release v1.1.1" /></a>
14
- <a href="tests"><img src="https://img.shields.io/badge/verified-101%20tests-2EA44F?style=flat-square" alt="Verified: 101 tests" /></a>
13
+ <a href="https://github.com/zp-home/dsh-recommend"><img src="https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fzp-home%2Fdsh-recommend%2Fmain%2Fdata%2Fbadges%2Fysr666__dsh-vision-router.certified.json&amp;style=flat-square" alt="dsh-recommend 🏅 精选认证" /></a>
14
+ <a href="https://github.com/zp-home/dsh-recommend"><img src="https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fzp-home%2Fdsh-recommend%2Fmain%2Fdata%2Fbadges%2Fysr666__dsh-vision-router.json&amp;style=flat-square" alt="dsh score 0.90" /></a>
15
+ </p>
16
+
17
+ <p align="center">
18
+ <a href="https://github.com/ysr666/dsh-vision-router/releases/tag/v1.2.1"><img src="https://img.shields.io/badge/release-v1.2.1-5B4CF0?style=flat-square" alt="Release v1.2.1" /></a>
19
+ <a href="tests"><img src="https://img.shields.io/badge/verified-149%20tests-2EA44F?style=flat-square" alt="Verified: 149 tests" /></a>
15
20
  <a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-2EA44F?style=flat-square" alt="License: MIT" /></a>
16
21
  <a href="package.json"><img src="https://img.shields.io/badge/Node.js-%3E%3D22-339933?style=flat-square&amp;logo=nodedotjs&amp;logoColor=white" alt="Node.js >=22" /></a>
17
22
  <img src="https://img.shields.io/badge/runtime-no%20Python-8A2BE2?style=flat-square" alt="No Python" />
@@ -20,10 +25,12 @@
20
25
 
21
26
  <p align="center"><a href="README.md">English</a> · 中文</p>
22
27
 
28
+ <p align="center">💬 <strong>QQ 用户交流群:1105463028</strong></p>
29
+
23
30
  > [!WARNING]
24
- > 📌 **公告(v1.1.1)**
31
+ > 📌 **公告(v1.2.1)**
25
32
  >
26
- > 现在安装后会**自动包装 DSH 已有模型**:纯文本路由自动获得粘贴即用的识图入口,原生多模态模型保留原图直传,仅在需要精确定位 / OCR / 像素验证时按需使用 Vision Router 工具。另修复 Windows sharp/libvips 启动冲突,并补齐 npm/npx 与源码 pnpm 两套安装/升级指引。
33
+ > v1.2.1 加固像素闭环:十个像素工具可直接接受上传图片的附件 ID(告别 `cannot read …/sha256:…` 的绕路),产物文件名带指纹不再互相覆盖,`vision_ground` 对退化框自动重试,模型引导支持从第 1 步完整重放(先退出设置页),设置卡片在数百模型目录下滚动依然流畅。
27
34
 
28
35
  <p align="center">
29
36
  <img src="assets/vision-demo.gif" width="640" alt="演示:粘贴图片,Agent 用 vision_ground / vision_crop / vision_pixel_diff 定位发送按钮并给出坐标" />
@@ -34,7 +41,7 @@
34
41
  大多数 DSH 视觉插件把图片“翻译”成一段文字描述再喂给 DeepSeek——有损、一次性、看不见像素。本插件把**原图像素留在视觉模型侧**、把推理留在 DeepSeek 侧,并把“看图”变成一次**普通的工具调用**:
35
42
 
36
43
  - **一条命令安装。** 包自带组合补丁(`dsh.bundle.patch`):`dsh plugin add` 自动完成插件行挂载、准入包装与附件限制放宽——不用手改任何文件。是否接管官方 DeepSeek 路由由「隐身模式」开关决定(默认关)。
37
- - **默认免费。** 视觉链内置 OVHcloud 匿名端点(`Qwen2.5-VL-72B-Instruct`,免注册、免 Key,每 IP 2 次/分钟)。付费链路(OpenRouter、Pi-AI 供应商、任意 OpenAI 兼容直连端点)是可选升级。
44
+ - **默认免费。** 视觉工具最终兜底为 5 个 OVHcloud 匿名视觉模型:免注册、免 Key,每 IP、每模型 2 次/分钟,独立限额理论合计约 10 次/分钟;用户自备视觉模型会优先调用。
38
45
  - **无 Python。** 整条管线——缩放、定位、裁剪、像素对比、取色、OCR、SVG 矢量化、抠图、HTML 截图——全部基于 sharp / potrace / tesseract / 系统 Chrome。
39
46
  - **可连续多步看图。** 图片轮 = 调用工具的文本轮:`vision_ground` → `vision_crop` → `vision_describe` → `vision_pixel_diff` → 修复 → 再截图,Agent 可以一直迭代到任务完成。
40
47
  - **DeepSeek 始终是大脑。** 文字轮在模型、成本、上下文上完全不动;视觉模型只当“眼睛”、按需调用,答案按图片内容缓存。
@@ -48,17 +55,19 @@
48
55
  |---|---|---|
49
56
  | 开箱图片问答 | ✅ 内置免费视觉链(OVHcloud 匿名端点),免注册免 Key | 远程工具需自备视觉 API Key(本地像素工具免 Key) |
50
57
  | 运行时 | ✅ 纯 Node,无需 Python | 需要 Python 3.11+ 受管运行时 |
51
- | 图片怎么进来 | ✅ 直接粘贴——轮次自动切视觉链并自动挂载工具 | 工作区路径 + `/vision-tools` 命令,再显式调用工具 |
58
+ | 图片怎么进来 | ✅ 选一次「+ 自动识图」模型组后直接粘贴 | 工作区路径 + `/vision-tools` 命令,再显式调用工具 |
52
59
  | 轮次路由 | ✅ 图片轮切视觉、文本轮切回 DeepSeek——可选隐身接管,模型选择器与官方一致 | 工具驱动,无整轮自动路由 |
53
60
  | 支持 profile | Web | Web + Headless |
54
61
  | 玩法库 | 像素循环:定位 → 裁剪 → 对比 → 修复 → 再截图 | 更丰富的案例库(长截图 OCR、UI 还原、GUI 自动化) |
55
- | 测试 | 86 | 162 |
62
+ | 测试 | 144 | 162 |
56
63
  | 安装 | 一条命令 | 一条命令(npm) |
57
64
 
58
65
  两者都是 MIT 许可、一条命令安装。想要图片**粘贴即用**、零配置就选本插件;需要 Headless 部署或更丰富的案例库,可以看 @anionex/dsh-vision-toolkit。(功能对比以其 README 2026-08 状态为准。)
59
66
 
60
67
  ## 快速开始
61
68
 
69
+ ### 1. 安装并让插件加载
70
+
62
71
  普通 npm / npx 安装方式推荐这样用(与 DSH 官方 README 的启动方式一致):
63
72
 
64
73
  ```sh
@@ -74,14 +83,32 @@ pnpm dsh plugin --profile web add dsh-vision-router
74
83
  pnpm dsh web
75
84
  ```
76
85
 
77
- 如果你已经全局安装 DSH CLI,并且终端里能直接执行 `dsh`,也可以继续使用较短的 `dsh ...` 写法。长期运行的 Web profile 安装后重启——完成,零配置:
86
+ 如果你已经全局安装 DSH CLI,并且终端里能直接执行 `dsh`,也可以继续使用较短的 `dsh ...` 写法。
87
+
88
+ > [!NOTE]
89
+ > 如果你是把插件**首次安装进一个已经长期运行的 Web 进程**,需要让 DSH Web 进程重新加载一次插件本体。插件加载完成后,新增/删除模型、修改自动识图包装范围都会**热更新,无需再重启 DSH**。
90
+
91
+ ### 2. 在聊天页切换到「+ 自动识图」模型组
92
+
93
+ 插件加载后会自动发现 **设置 → 模型** 里已启用的模型组,并为它们额外创建同名的自动识图入口。例如:
94
+
95
+ ```text
96
+ opencode-go ← 原模型组,保持不变
97
+ opencode-go + 自动识图 ← 发图片时选这个
98
+ ```
99
+
100
+ > [!IMPORTANT]
101
+ > **发图前,请点击聊天页输入区右下角的模型选择器,选择带「+ 自动识图」的模型组。**
102
+ >
103
+ > Vision Router 故意**不修改原模型组**。因此如果当前仍选着原来的纯文本 `opencode-go` / DeepSeek 路由,DSH 会在插件处理图片之前先提示“当前模型不支持图片”。这不是视觉后端配置失败,只是还没有切到自动识图入口。
104
+
105
+ 这个模型组的模型列表会跟随 DSH 的模型目录实时同步;新增模型或修改包装范围后无需重启。
106
+
107
+ ### 3. 直接粘贴或上传图片
78
108
 
79
- - 插件的 bundle 补丁自动挂载插件行、挂上准入包装并放宽附件限制到 20MB / 1 亿像素——纯增量、不碰核心行;是否接管官方 DeepSeek 路由由「隐身模式」开关决定(默认关);
80
- - 默认视觉链就是内置免费端点;
81
- - opencode 等自定义/第三方路由用「额外识图包装」获得发图能力;
82
- - 全部配置可在 **设置 → 插件 → 插件配置 → 视觉路由(自动识图)** 实时修改。
109
+ 选好「+ 自动识图」模型组后,直接往对话里贴图即可。Agent 会自动挂载视觉工具,通过 `vision_describe`、`vision_ground`、`vision_crop` 等工具看图,需要时连续多步操作。
83
110
 
84
- 然后直接往对话里贴一张图。Agent 自动挂载视觉工具,通过 `vision_describe`(以及其余 8 个工具)看图,需要时连续多步。
111
+ 默认已经有内置 OVH 匿名视觉兜底,无需注册、无需 Key。**聊天页右下角只选择“脑子/会话模型”**;视觉模型不要在那里选。高级配置在 **设置 → 插件 → 插件配置 → 视觉路由(自动识图)**:视觉后端链每一行只选择一个你在 **设置 → 模型** 中已经配置且支持图片输入的用户模型;一行都不填也可以,OVH 免费链会固定在最后兜底。插件内部的 `Vision HTTP` 只是传输实现,不是用户需要选择的模型组。
85
112
 
86
113
  ### 实际效果
87
114
 
@@ -103,9 +130,9 @@ pnpm dsh web
103
130
 
104
131
  ### 像素闭环实测
105
132
 
106
- <p align="center">
107
- <img src="assets/pixel-loop-zh.png" width="100%" alt="参考设计与 Agent 最终复刻,通过 vision_pixel_diff 实测最终差异为 2.54%。" />
108
- </p>
133
+ [![参考设计与 Agent 最终复刻,通过 vision_pixel_diff 实测最终差异为 2.54%。](assets/pixel-loop-zh.png)](assets/pixel-loop-zh.png)
134
+
135
+ <p align="center"><sub>点击图片可查看完整原图。</sub></p>
109
136
 
110
137
  Agent 仅根据参考图复刻 UI,再用 `vision_pixel_diff` 验证最终结果:**最终差异 2.54%**(32,939 / 1,296,000 个差异像素,threshold 16/channel)。
111
138
 
@@ -159,11 +186,14 @@ vision_long_screenshot_ocr image="chat-log.png" chunkHeight=1200 overlap=120
159
186
 
160
187
  ## 供应商降级链
161
188
 
162
- 视觉链按顺序逐个尝试,全部失败才报错:
189
+ 视觉工具按顺序逐个尝试,全部失败才报错:
163
190
 
164
- 1. **内置免费端点**(`vision-http` → `ovh/Qwen2.5-VL-72B-Instruct`)——免 Key、尽力而为、每 IP 2 次/分钟;
165
- 2. 配置的 `httpProviders`(OpenAI 兼容直连端点,可选 `apiKeyEnv`);
166
- 3. 配置的 `providers` / `provider` + `fallbacks`(任何有适配器的供应商,例如 Pi-AI 配置的 OpenRouter 或智谱)。
191
+ 1. **用户视觉模型**:设置卡里一行一个,从上到下;只显示 **设置 → 模型** 中明确声明支持 image 输入的模型;
192
+ 2. **高级自定义 HTTP 视觉端点**:如果旧配置/高级配置中存在 `httpProviders`,在用户模型之后尝试;
193
+ 3. **内置 OVH 匿名免费兜底**:固定最后尝试,不需要出现在任何模型选择器里。当前内置链按质量优先为 `Qwen3.5-397B-A17B` → `Qwen2.5-VL-72B-Instruct` → `Qwen3.6-27B` → `Mistral-Small-3.2-24B-Instruct-2506` → `Qwen3.5-9B`。OVH 匿名限额为 **每 IP、每模型 2 次/分钟**;5 个模型是独立限额,因此理论上分散请求可到约 **10 次/分钟**,实际仍以 OVH 当时的限流为准。免注册、免 Key。
194
+
195
+ > [!IMPORTANT]
196
+ > 这里的“视觉链”是 Vision Router 调用的**眼睛**:设置页里每一行只选一个用户视觉模型;聊天页右下角选择的是**脑子/会话模型**,两者完全分开。纯文本 DeepSeek / opencode 不会出现在视觉后端下拉里;内部 `Vision HTTP` 也不会再暴露给用户。
167
197
 
168
198
  > 在旧版 `routing: true` 模式下,整轮链只走 `provider + fallbacks`——`httpProviders`(含免费兜底)不参与。默认的 `routing: false`(工具优先)会尝试全部。
169
199
 
@@ -183,20 +213,28 @@ vision_long_screenshot_ocr image="chat-log.png" chunkHeight=1200 overlap=120
183
213
 
184
214
  官方行在场时,插件自动回退为可见包装入口。反过来,隐身模式关闭但官方行仍被禁用时,插件会做 keep-alive 兜底接管,保住 DeepSeek 模型(设置卡片会给出提示);想完全恢复官方原生行,把上面的 `disabled` 改回 `false` 再重启即可。
185
215
 
186
- > 隐身模式**只作用于官方 DeepSeek 路由**。opencode 等自定义/第三方文本路由与隐身模式无关——用「额外识图包装」让它们支持发图。
216
+ > 隐身模式**只作用于官方 DeepSeek 路由**。opencode 等自定义/第三方文本路由与隐身模式无关——默认会被自动包装成「+ 自动识图」模型组。
217
+
218
+ ## 自动识图模型组与手动包装
187
219
 
188
- ## 额外识图包装
220
+ 默认开启 `autoWrapProviders`:插件会自动发现 **设置 → 模型** 中当前已启用的 provider / model,并额外注册同名的「+ 自动识图」模型组。**原模型组完全不变**;发图片时选自动识图组,纯文字仍可继续用原组。DSH 的 `llm/adapters-updated` 变化会触发同步,所以新增/删除模型后无需重启。
189
221
 
190
- `wrappedProviders` 给任意第三方/自定义文本路由注册「自动识图」孪生条目:模型选择器里选中它就能发图,文字轮原样交给原路由处理。典型用法是 opencode 等接入的自定义接口——它们默认只声明文本输入,加一行包装即可直接发图。设置卡片里用两个下拉(provider + 模型)配置;模型留空 = 包装该路由的全部模型,同一 provider 要包装多个模型就添加多行。
222
+ `wrappedProviders` 是**可选的手动范围控制**,不是普通用户必须配置的步骤。只有两种情况需要它:
223
+
224
+ 1. 关闭了自动包装,想手动指定哪些 provider / model 获得自动识图入口;
225
+ 2. 自动包装保持开启,但只想让某个 provider 的部分模型出现在「+ 自动识图」组。
226
+
227
+ 设置卡片里用两个下拉(provider + 模型)配置;模型留空 = 包装该路由的全部模型,同一 provider 要限定多个模型就添加多行。修改即时生效,无需重启。
191
228
 
192
229
  ## Web 设置
193
230
 
194
- Web 配置页在 **设置 → 插件 → 插件配置** 下注册「视觉路由(自动识图)」卡片,样式与内置卡片一致,可实时修改:
231
+ Web 配置页在 **设置 → 插件 → 插件配置** 下注册「视觉路由(自动识图)」卡片,顶部会直接提示最重要的使用步骤:**回到聊天页 → 右下角模型选择器 → 选择「+ 自动识图」模型组 → 发图**。其余设置主要用于高级定制:
195
232
 
233
+ - **自动创建「+ 自动识图」模型组**:默认开启,自动发现已有模型;模型目录变化热更新,无需重启;
234
+ - **手动限定自动识图范围(可选)**:仅在需要关闭自动包装或限制部分模型时使用;
235
+ - **视觉后端链**:给 `vision_describe` 等视觉工具调用的真正图片模型,默认内置免费 Qwen 即可;不要填纯文本模型;
196
236
  - 开关:整轮自动路由(旧模式)、识图工具、图片块改写、隐身模式(仅官方 DeepSeek 路由);
197
- - **额外识图包装**:provider + 模型双下拉,给 opencode 等自定义路由注册可发图的孪生条目;
198
- - 视觉请求超时、包装/链路由名;
199
- - **视觉模型链**(每行一个 `provider/model`,自上而下降级)与文本模型;
237
+ - 视觉请求超时、包装/链路由名、代理等高级参数;
200
238
  - 每个字段都有「已覆盖」徽标与一键恢复组合默认,以及放弃/保存;
201
239
  - 「测试连接」按钮探测第一个视觉提供方并内联显示延迟/失败原因;
202
240
  - 产出制品的工具在对话里渲染专用调用卡(关键字段 + 打开文件按钮)。
@@ -213,15 +251,16 @@ Web 配置页在 **设置 → 插件 → 插件配置** 下注册「视觉路由
213
251
 
214
252
  | 字段 | 默认值 | 含义 |
215
253
  |---|---|---|
216
- | `provider` / `model` | `vision-http` / `ovh/Qwen2.5-VL-72B-Instruct` | 简写链路(有适配器的供应商 + 模型) |
217
- | `fallbacks` | `[]` | 简写供应商的备用模型 |
218
- | `providers` | 内置免费 `vision-http` 条目 | 多供应商链路 `{ provider, model, fallbacks[] }`,按序尝试;优先于简写形式。第一行开箱预置内置免费模型 |
254
+ | `provider` / `model` | `vision-http` / `ovh/Qwen2.5-VL-72B-Instruct` | 简写视觉后端链路(有适配器且真正支持图片输入的供应商 + 模型) |
255
+ | `fallbacks` | `[]` | 简写视觉供应商的备用图片模型 |
256
+ | `providers` | 内置免费 `vision-http` 条目 | 多供应商视觉后端链 `{ provider, model, fallbacks[] }`,按序尝试;优先于简写形式。不要填写纯文本模型 |
219
257
  | `httpProviders` | 内置 OVH 条目 | OpenAI 兼容直连端点 `{ name, baseURL, model, apiKeyEnv, maxTokens }` |
220
- | `wrappedProviders` | `[{ provider: 'deepseek-official', models: [] }]` | 额外识图包装:`{ provider, models[] }`,给 opencode 等任意第三方/自定义文本路由注册可发图的孪生条目(卡片里 provider + 模型双下拉;模型留空 = 包装全部)。预置的 deepseek-official 条目标记内置包装、无副作用;改动即时生效 |
258
+ | `autoWrapProviders` | `true` | 自动发现当前已启用 provider / model,并热更新同名「+ 自动识图」模型组;原模型组不变 |
259
+ | `wrappedProviders` | `[{ provider: 'deepseek-official', models: [] }]` | 可选的手动包装范围 `{ provider, models[] }`;用于关闭自动包装后手动指定,或限制某个 provider 只包装部分模型。改动即时生效,无需重启 |
221
260
  | `routing` | `false` | 旧版整轮链路由(一次性整轮回答)。`false` = 工具优先流程(推荐) |
222
261
  | `reverseRouting` | `true` | 开启 `routing` 时,文字轮路由回 `textProvider` |
223
262
  | `wrapperRoute` / `chainRoute` | `deepseek-vision` / `vision-chain` | 准入包装路由名 / 降级链路由名(置空关闭) |
224
- | `stealth` | `false` | 接管官方 `deepseek-official` 路由(仅官方行;自定义路由用 `wrappedProviders`) |
263
+ | `stealth` | `false` | 接管官方 `deepseek-official` 路由(仅官方行;自定义路由默认由自动包装处理) |
225
264
  | `textProvider` | `deepseek-official` / `deepseek-v4-pro` | 负责思考的模型(你的日常模型) |
226
265
  | `tool` / `progressiveTools` / `autoActivateOnImage` | `true` ×3 | 视觉工具开关 / 渐进式挂载 / 图片轮自动挂载 |
227
266
  | `rewriteImages` | `true` | 模型输入层改写图片块(缓存描述或工具提示标记);界面日志保留图片 |
@@ -257,7 +296,7 @@ pnpm dsh plugin --profile web add dsh-vision-router
257
296
  pnpm dsh --profile web --dump-config | grep vision-router
258
297
  ```
259
298
 
260
- 长期运行的 Web profile 需重启。宿主在启动时通过 `dsh.client` 声明发现浏览器端包。
299
+ 首次把插件装进已经长期运行的 Web profile 时,需要让 Web 进程重新加载插件本体;宿主在启动时通过 `dsh.client` 声明发现浏览器端包。**插件加载完成后,模型目录与包装范围的变化会热更新,不需要为这些变化重启。**
261
300
 
262
301
  ### 禁用 / 恢复
263
302
 
@@ -311,6 +350,37 @@ pnpm dsh plugin --profile web remove dsh-vision-router
311
350
 
312
351
  同时移除依赖与 bundle 层。若你曾手动禁用官方 DeepSeek 行,记得在 profile 补丁里恢复。
313
352
 
353
+ ## 故障排查
354
+
355
+ ### 启动报错 `Unexpected token ... is not valid JSON`(UTF-8 BOM)
356
+
357
+ **现象**:`dsh web` / `pnpm dsh web` 启动时直接退出:
358
+
359
+ ```
360
+ SyntaxError: Unexpected token ...
361
+ is not valid JSON
362
+ at JSON.parse (<anonymous>)
363
+ at readProfileManifest (packages/boot/app-boot/src/profile.ts)
364
+ ```
365
+
366
+ **原因**:`~/.dsh/profiles/<profile>/package.json` 被某些编辑器保存成了 **UTF-8 with BOM**。文件最前面多了一个不可见的 `\uFEFF` 字符,dsh 读取 manifest 时直接 `JSON.parse`,而 JSON 不允许在开头出现这个字符,于是解析失败。
367
+
368
+ **推荐修复**:直接运行 Vision Router 自带的独立修复命令。它不需要 DSH 先成功启动,会定位 profile、检测 UTF-8 BOM,只删除开头的三个 BOM 字节,然后重新验证 JSON:
369
+
370
+ ```sh
371
+ npx dsh-vision-router repair --profile web
372
+ ```
373
+
374
+ 只想检查、不修改文件时:
375
+
376
+ ```sh
377
+ npx dsh-vision-router doctor --profile web
378
+ ```
379
+
380
+ 如果你使用的不是 `web` profile,把 `web` 换成对应名称;也可以不传 `--profile`,让 doctor 扫描全部 profile。
381
+
382
+ 手动兜底方式:VS Code 右下角编码 → “通过编码保存” → 选择 `UTF-8`(无 BOM)。若 `repair` 去掉 BOM 后仍提示 JSON 非法,它不会猜测或重写其他 JSON 内容,请再手动检查文件。
383
+
314
384
  ## 安全说明
315
385
 
316
386
  - 图片中的文字是**不可信证据**:描述、OCR 输出与自动挂载提示都要求 Agent 绝不执行图片内出现的指令。
Binary file
Binary file
package/docs/doctor.md ADDED
@@ -0,0 +1,52 @@
1
+ # Vision Router doctor / repair
2
+
3
+ Vision Router ships a small standalone diagnostic CLI. It does not need DSH to boot first, so it can still run when DSH exits while parsing a broken profile manifest.
4
+
5
+ ## Normal installation stays unchanged
6
+
7
+ Use DSH's own plugin command:
8
+
9
+ ```sh
10
+ npx @deepseek-ai/dsh plugin --profile web add dsh-vision-router
11
+ npx @deepseek-ai/dsh web
12
+ ```
13
+
14
+ For a DeepSeek Harness source checkout:
15
+
16
+ ```sh
17
+ pnpm dsh plugin --profile web add dsh-vision-router
18
+ pnpm dsh web
19
+ ```
20
+
21
+ The doctor is a recovery/diagnostic tool, not a replacement installer.
22
+
23
+ ## Diagnose profiles
24
+
25
+ ```sh
26
+ npx dsh-vision-router doctor
27
+ ```
28
+
29
+ To inspect only the Web profile:
30
+
31
+ ```sh
32
+ npx dsh-vision-router doctor --profile web
33
+ ```
34
+
35
+ The command locates the DSH home (`$DSH_HOME` when set, otherwise `~/.dsh`), scans profile `package.json` files, reports UTF-8 BOM bytes, validates the JSON after ignoring a leading BOM for diagnosis, and reports whether `dsh-vision-router` is present as a profile dependency and bundle layer.
36
+
37
+ ## Repair the UTF-8 BOM startup failure
38
+
39
+ If DSH fails before plugins can load with an error such as:
40
+
41
+ ```text
42
+ SyntaxError: Unexpected token ... is not valid JSON
43
+ at readProfileManifest (.../profile.ts:...)
44
+ ```
45
+
46
+ run:
47
+
48
+ ```sh
49
+ npx dsh-vision-router repair --profile web
50
+ ```
51
+
52
+ `repair` removes only the three-byte UTF-8 BOM prefix (`EF BB BF`) when it is present, then validates the remaining JSON. It does not reformat, regenerate, or otherwise rewrite the profile contents. If JSON is still invalid for another reason, the command reports that and stops rather than guessing a repair.