dsh-vision-router 1.1.1 → 1.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +105 -35
- package/README.zh.md +103 -33
- package/assets/pixel-loop-zh.png +0 -0
- package/assets/pixel-loop.png +0 -0
- package/docs/doctor.md +52 -0
- package/docs/http-provider-compatibility.md +73 -0
- package/docs/update-check.md +42 -0
- package/docs/update-check.zh-CN.md +42 -0
- package/index.js +845 -127
- package/lib/client.js +945 -99
- package/lib/doctor-cli.js +113 -0
- package/lib/doctor.js +113 -0
- package/lib/http-compat.js +264 -0
- package/lib/self-update.js +165 -0
- package/lib/update-check.js +223 -0
- package/package.json +5 -2
package/README.md
CHANGED
|
@@ -10,8 +10,13 @@
|
|
|
10
10
|
|
|
11
11
|
<p align="center">
|
|
12
12
|
<a href="https://awesome-dsh-plugin.com"><img src="https://awesome-dsh-plugin.com/badge.svg" alt="awesome · DSH plugin" /></a>
|
|
13
|
-
<a href="https://github.com/
|
|
14
|
-
<a href="
|
|
13
|
+
<a href="https://github.com/zp-home/dsh-recommend"><img src="https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fzp-home%2Fdsh-recommend%2Fmain%2Fdata%2Fbadges%2Fysr666__dsh-vision-router.certified.json&style=flat-square" alt="dsh-recommend 🏅 精选认证" /></a>
|
|
14
|
+
<a href="https://github.com/zp-home/dsh-recommend"><img src="https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fzp-home%2Fdsh-recommend%2Fmain%2Fdata%2Fbadges%2Fysr666__dsh-vision-router.json&style=flat-square" alt="dsh score 0.90" /></a>
|
|
15
|
+
</p>
|
|
16
|
+
|
|
17
|
+
<p align="center">
|
|
18
|
+
<a href="https://github.com/ysr666/dsh-vision-router/releases/tag/v1.2.1"><img src="https://img.shields.io/badge/release-v1.2.1-5B4CF0?style=flat-square" alt="Release v1.2.1" /></a>
|
|
19
|
+
<a href="tests"><img src="https://img.shields.io/badge/verified-149%20tests-2EA44F?style=flat-square" alt="Verified: 149 tests" /></a>
|
|
15
20
|
<a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-2EA44F?style=flat-square" alt="License: MIT" /></a>
|
|
16
21
|
<a href="package.json"><img src="https://img.shields.io/badge/Node.js-%3E%3D22-339933?style=flat-square&logo=nodedotjs&logoColor=white" alt="Node.js >=22" /></a>
|
|
17
22
|
<img src="https://img.shields.io/badge/runtime-no%20Python-8A2BE2?style=flat-square" alt="No Python" />
|
|
@@ -20,10 +25,12 @@
|
|
|
20
25
|
|
|
21
26
|
<p align="center">English · <a href="README.zh.md">中文</a></p>
|
|
22
27
|
|
|
28
|
+
<p align="center">💬 <strong>QQ community group: 1105463028</strong></p>
|
|
29
|
+
|
|
23
30
|
> [!WARNING]
|
|
24
|
-
> 📌 **Announcement (v1.
|
|
31
|
+
> 📌 **Announcement (v1.2.1)**
|
|
25
32
|
>
|
|
26
|
-
>
|
|
33
|
+
> v1.2.1 hardens the pixel loop: all ten pixel tools now accept uploaded-image attachment ids directly (no more `cannot read …/sha256:…` round trips), artifact filenames carry collision-free fingerprints, `vision_ground` retries degenerate boxes, the model guide replays fully from step 1 (leaving the settings first), and the settings card scrolls smoothly even with hundreds of models per provider.
|
|
27
34
|
|
|
28
35
|
<p align="center">
|
|
29
36
|
<img src="assets/vision-demo.gif" width="640" alt="Demo: paste an image, the agent locates the send button with vision_ground / vision_crop / vision_pixel_diff and answers with coordinates" />
|
|
@@ -34,7 +41,7 @@
|
|
|
34
41
|
Most DSH vision plugins bridge images to DeepSeek as *text descriptions* — lossy, one-shot, and blind to pixels. This plugin keeps the **original pixels on the vision model's side** and DeepSeek on the reasoning side, and makes looking at an image an **ordinary tool call**:
|
|
35
42
|
|
|
36
43
|
- **One command install.** The package ships its own composition patch (`dsh.bundle.patch`): `dsh plugin add` wires the row, the admission wrapper and the attachment limits automatically — zero manual file edits. Taking over the official DeepSeek route is an optional setting (stealth mode, off by default).
|
|
37
|
-
- **Free by default.**
|
|
44
|
+
- **Free by default.** Vision tools end with a five-model OVHcloud anonymous fallback: no account, no key, 2 requests/minute per IP per model, roughly 10 RPM in theory across independent buckets. User-provided vision models run first.
|
|
38
45
|
- **No Python.** The whole pipeline — downscale, grounding, crop, pixel diff, palette, OCR, SVG trace, cutout, HTML screenshot — runs on sharp / potrace / tesseract / system Chrome.
|
|
39
46
|
- **Continuous multi-step image work.** An image turn is a text turn that calls tools: `vision_ground` → `vision_crop` → `vision_describe` → `vision_pixel_diff` → fix → screenshot again. The agent keeps iterating until the work is done.
|
|
40
47
|
- **DeepSeek stays the brain.** Text turns are untouched in model, cost and context. The vision model is only the eyes, called on demand; answers are cached by image content.
|
|
@@ -48,17 +55,19 @@ The closest alternative is [@anionex/dsh-vision-toolkit](https://github.com/Anio
|
|
|
48
55
|
|---|---|---|
|
|
49
56
|
| Image Q&A out of the box | ✅ Built-in free chain (anonymous OVHcloud endpoint) — no account, no key | Requires your own vision API key (local pixel tools work without one) |
|
|
50
57
|
| Runtime | ✅ Node only — no Python | Python 3.11+ managed runtime |
|
|
51
|
-
| Getting an image in | ✅
|
|
58
|
+
| Getting an image in | ✅ Pick a “+ Auto Vision” group once, then paste directly | Workspace path + `/vision-tools` command, then explicit tool calls |
|
|
52
59
|
| Turn routing | ✅ Image turns switch to vision, text turns switch back to DeepSeek — optional stealth takeover keeps the model picker looking stock | Tool-driven; no whole-turn auto-routing |
|
|
53
60
|
| Profiles | Web | Web + Headless |
|
|
54
61
|
| Playbooks | The pixel loop: ground → crop → diff → fix → screenshot again | Richer case library (long-screenshot OCR, UI restoration, GUI automation) |
|
|
55
|
-
| Tests |
|
|
62
|
+
| Tests | 144 | 162 |
|
|
56
63
|
| Install | One command | One command (npm) |
|
|
57
64
|
|
|
58
65
|
Both are MIT-licensed and one command away. Pick this plugin when you want images to *just work* with zero setup; pick theirs when you need headless profiles or the extended playbook library. (Feature comparison reflects their README as of 2026-08.)
|
|
59
66
|
|
|
60
67
|
## Quick start
|
|
61
68
|
|
|
69
|
+
### 1. Install and load the plugin
|
|
70
|
+
|
|
62
71
|
Recommended for normal npm/npx installs (the same launch style used by the DSH README):
|
|
63
72
|
|
|
64
73
|
```sh
|
|
@@ -74,14 +83,32 @@ pnpm dsh plugin --profile web add dsh-vision-router
|
|
|
74
83
|
pnpm dsh web
|
|
75
84
|
```
|
|
76
85
|
|
|
77
|
-
If you already installed the DSH CLI globally and `dsh` is on `PATH`, the shorter `dsh ...` form works too.
|
|
86
|
+
If you already installed the DSH CLI globally and `dsh` is on `PATH`, the shorter `dsh ...` form works too.
|
|
87
|
+
|
|
88
|
+
> [!NOTE]
|
|
89
|
+
> If you install the plugin **into a Web process that was already running long-term**, let that DSH Web process reload once so the plugin bundle itself is discovered. After the plugin is loaded, adding/removing models or changing wrapper scope **hot-updates without further DSH restarts**.
|
|
90
|
+
|
|
91
|
+
### 2. Switch to a “+ Auto Vision” model group in chat
|
|
92
|
+
|
|
93
|
+
Once loaded, the plugin discovers the model groups enabled under **Settings → Models** and creates same-name auto-vision entries. For example:
|
|
94
|
+
|
|
95
|
+
```text
|
|
96
|
+
opencode-go ← original model group, unchanged
|
|
97
|
+
opencode-go + Auto Vision ← choose this when sending images
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
> [!IMPORTANT]
|
|
101
|
+
> **Before sending an image, open the model selector in the lower-right corner of the chat composer and choose a group marked “+ Auto Vision”.**
|
|
102
|
+
>
|
|
103
|
+
> Vision Router deliberately **does not modify the original model group**. If the conversation still uses the original text-only opencode / DeepSeek route, DSH can reject the image with “the current model does not support images” *before Vision Router gets a chance to handle it*. That is a model-entry selection issue, not a broken vision backend.
|
|
104
|
+
|
|
105
|
+
The auto-vision group follows the live DSH model catalog. Adding models or changing wrapper scope does not require a restart.
|
|
106
|
+
|
|
107
|
+
### 3. Paste or upload the image
|
|
78
108
|
|
|
79
|
-
|
|
80
|
-
- the default vision chain is the built-in free endpoint;
|
|
81
|
-
- custom/third-party routes (e.g. opencode) gain image input through **Extra vision wrappers**;
|
|
82
|
-
- every setting is editable live in **Settings → Plugins → Plugin config → 视觉路由(自动识图)**.
|
|
109
|
+
After choosing the “+ Auto Vision” group, paste or upload an image normally. The agent auto-mounts the vision tools and can use `vision_describe`, `vision_ground`, `vision_crop`, and the rest across multiple steps when needed.
|
|
83
110
|
|
|
84
|
-
|
|
111
|
+
The built-in anonymous OVH vision fallback is already configured, so normal image use needs no signup or API key. **The lower-right chat picker selects only the brain/conversation model**; vision backends do not belong there. Advanced options live under **Settings → Plugins → Plugin config → 视觉路由(自动识图)**: each vision-backend row selects one image-capable user model already configured under **Settings → Models**. Leaving every user row empty is valid; the OVH chain remains the final fallback. `Vision HTTP` is an internal transport route, not a model group users should select.
|
|
85
112
|
|
|
86
113
|
### See it in action
|
|
87
114
|
|
|
@@ -103,9 +130,9 @@ Then just paste an image into a conversation. The agent mounts the vision tools
|
|
|
103
130
|
|
|
104
131
|
### Pixel loop in practice
|
|
105
132
|
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
133
|
+
[](https://raw.githubusercontent.com/ysr666/dsh-vision-router/main/assets/pixel-loop.png)
|
|
134
|
+
|
|
135
|
+
<p align="center"><sub>Click the image to open the full-resolution original.</sub></p>
|
|
109
136
|
|
|
110
137
|
The agent rebuilt the UI from the reference image, then verified the final result with `vision_pixel_diff`: **2.54% final diff** (32,939 / 1,296,000 differing pixels, threshold 16/channel).
|
|
111
138
|
|
|
@@ -159,11 +186,14 @@ vision_long_screenshot_ocr image="chat-log.png" chunkHeight=1200 overlap=120
|
|
|
159
186
|
|
|
160
187
|
## Provider fallback chain
|
|
161
188
|
|
|
162
|
-
The vision
|
|
189
|
+
The vision tools try backends in order and surface an error only after all of them fail:
|
|
163
190
|
|
|
164
|
-
1.
|
|
165
|
-
2.
|
|
166
|
-
3.
|
|
191
|
+
1. **User vision models**: one per settings row, top to bottom; only models under **Settings → Models** that explicitly declare image input are shown;
|
|
192
|
+
2. **Advanced custom HTTP vision endpoints**: legacy/advanced `httpProviders`, when present, run after the user models;
|
|
193
|
+
3. **Built-in anonymous OVH fallback**: always last and never exposed in a model picker. The current quality-first chain is `Qwen3.5-397B-A17B` → `Qwen2.5-VL-72B-Instruct` → `Qwen3.6-27B` → `Mistral-Small-3.2-24B-Instruct-2506` → `Qwen3.5-9B`. OVH anonymous limits are **2 requests/minute per IP per model**. The five models have independent buckets, so spreading requests across them is about **10 RPM in theory**, subject to OVH's actual rate limiting. No signup or API key is required.
|
|
194
|
+
|
|
195
|
+
> [!IMPORTANT]
|
|
196
|
+
> This “vision chain” is the **eyes** used by Vision Router: each settings row selects one user vision model, while the lower-right chat picker selects the **brain/conversation model**. The two are deliberately separate. Text-only DeepSeek/opencode models are filtered out of the vision-backend dropdown, and the internal `Vision HTTP` transport route is no longer exposed to users.
|
|
167
197
|
|
|
168
198
|
> In the legacy `routing: true` mode, the whole-turn chain walks only `provider + fallbacks` — `httpProviders` (including the free fallback) do not participate there. The default `routing: false` (tools-first) tries everything.
|
|
169
199
|
|
|
@@ -183,21 +213,29 @@ With stealth on, the plugin takes over the official `deepseek-official` route: t
|
|
|
183
213
|
|
|
184
214
|
With the stock row present, the plugin falls back to the visible wrapper entry. Conversely, with stealth off but the stock row still disabled, the plugin performs a keep-alive takeover so the DeepSeek models don't vanish (the settings card explains this); to restore the fully official route, flip the `disabled` above back to `false` and restart.
|
|
185
215
|
|
|
186
|
-
> Stealth mode **only affects the official DeepSeek route**. Custom/third-party routes
|
|
216
|
+
> Stealth mode **only affects the official DeepSeek route**. Custom/third-party routes such as opencode are auto-wrapped into “+ Auto Vision” groups by default.
|
|
217
|
+
|
|
218
|
+
## Auto-vision model groups and manual wrappers
|
|
187
219
|
|
|
188
|
-
|
|
220
|
+
`autoWrapProviders` is on by default. The plugin discovers the provider/model entries currently enabled under **Settings → Models** and registers a same-name “+ Auto Vision” model group for them. **The original group is never changed**: choose the auto-vision group for images, or keep using the original group for plain text. DSH `llm/adapters-updated` events are synced live, so adding/removing models does not require a restart.
|
|
189
221
|
|
|
190
|
-
`wrappedProviders`
|
|
222
|
+
`wrappedProviders` is an **optional manual scope control**, not a required setup step. Use it only when:
|
|
223
|
+
|
|
224
|
+
1. automatic wrapping is off and you want to pick which provider/models receive an auto-vision entry; or
|
|
225
|
+
2. automatic wrapping remains on but one provider should expose only selected models in its “+ Auto Vision” group.
|
|
226
|
+
|
|
227
|
+
The settings card uses provider + model dropdowns; an empty model means every model on that route. Add multiple rows to select multiple models. Changes apply immediately with no restart.
|
|
191
228
|
|
|
192
229
|
## Web settings
|
|
193
230
|
|
|
194
|
-
The Web profile registers a **视觉路由(自动识图)** card under **Settings → Plugins → Plugin config
|
|
231
|
+
The Web profile registers a **视觉路由(自动识图)** card under **Settings → Plugins → Plugin config**. Its top callout spells out the only step most users need: **return to chat → lower-right model selector → choose a “+ Auto Vision” model group → send the image**. The remaining controls are advanced customization:
|
|
195
232
|
|
|
196
|
-
-
|
|
197
|
-
- **
|
|
198
|
-
-
|
|
199
|
-
-
|
|
200
|
-
-
|
|
233
|
+
- **Auto-create “+ Auto Vision” model groups**: enabled by default; follows the live model catalog with no restart;
|
|
234
|
+
- **Manual auto-vision scope (optional)**: only for disabling auto-wrap or limiting selected models;
|
|
235
|
+
- **Vision backend chain**: the real image-capable models used by `vision_describe` and friends; the built-in free Qwen is normally enough, and text-only models should not be placed here;
|
|
236
|
+
- switches for legacy whole-turn routing, vision tools, image-block rewriting and stealth mode (official DeepSeek route only);
|
|
237
|
+
- timeout, wrapper/chain route names, proxy and other advanced parameters;
|
|
238
|
+
- every field shows an overridden badge with one-click reset plus discard/save;
|
|
201
239
|
- a **Test connection** button probes the first vision provider and reports latency inline;
|
|
202
240
|
- artifact-producing tools render dedicated call cards with result facts and open-file buttons.
|
|
203
241
|
|
|
@@ -213,15 +251,16 @@ Everything is optional; defaults work out of the box. Edit via the Web card or a
|
|
|
213
251
|
|
|
214
252
|
| Field | Default | Meaning |
|
|
215
253
|
|---|---|---|
|
|
216
|
-
| `provider` / `model` | `vision-http` / `ovh/Qwen2.5-VL-72B-Instruct` | shorthand
|
|
217
|
-
| `fallbacks` | `[]` | backup models for the shorthand provider |
|
|
218
|
-
| `providers` | built-in free `vision-http` pair | multi-provider chain `{ provider, model, fallbacks[] }`, tried in order;
|
|
254
|
+
| `provider` / `model` | `vision-http` / `ovh/Qwen2.5-VL-72B-Instruct` | shorthand **vision backend** route (adapter-backed provider + model that genuinely accepts images) |
|
|
255
|
+
| `fallbacks` | `[]` | backup image models for the shorthand vision provider |
|
|
256
|
+
| `providers` | built-in free `vision-http` pair | multi-provider **vision backend** chain `{ provider, model, fallbacks[] }`, tried in order; do not put text-only models here |
|
|
219
257
|
| `httpProviders` | built-in OVH entry | direct OpenAI-compatible endpoints `{ name, baseURL, model, apiKeyEnv, maxTokens }` |
|
|
220
|
-
| `
|
|
258
|
+
| `autoWrapProviders` | `true` | discover enabled provider/models and live-sync same-name “+ Auto Vision” groups; original groups stay unchanged |
|
|
259
|
+
| `wrappedProviders` | `[{ provider: 'deepseek-official', models: [] }]` | optional manual wrapper scope `{ provider, models[] }`, used after disabling auto-wrap or to restrict one provider to selected models; changes apply live, no restart |
|
|
221
260
|
| `routing` | `false` | legacy whole-turn chain routing (one-shot answer). `false` = tools-first flow (recommended) |
|
|
222
261
|
| `reverseRouting` | `true` | with `routing: true`, route text turns back to `textProvider` |
|
|
223
262
|
| `wrapperRoute` / `chainRoute` | `deepseek-vision` / `vision-chain` | admission wrapper route name / fallback chain route name (empty disables) |
|
|
224
|
-
| `stealth` | `false` | take over the official `deepseek-official` route (official row only; custom routes
|
|
263
|
+
| `stealth` | `false` | take over the official `deepseek-official` route (official row only; custom routes are auto-wrapped by default) |
|
|
225
264
|
| `textProvider` | `deepseek-official` / `deepseek-v4-pro` | the model that reasons (your daily model) |
|
|
226
265
|
| `tool` / `progressiveTools` / `autoActivateOnImage` | `true` ×3 | vision tools on / progressive mounting / auto-mount on image turns |
|
|
227
266
|
| `rewriteImages` | `true` | rewrite image blocks in the model input (cached description or tool-hint marker); the UI log keeps images |
|
|
@@ -257,7 +296,7 @@ pnpm dsh plugin --profile web add dsh-vision-router
|
|
|
257
296
|
pnpm dsh --profile web --dump-config | grep vision-router
|
|
258
297
|
```
|
|
259
298
|
|
|
260
|
-
|
|
299
|
+
When first adding the plugin to an already long-lived Web profile, let that Web process reload the plugin bundle; the host discovers the browser bundle through `dsh.client` at startup. **After the plugin is loaded, model-catalog and wrapper-scope changes hot-update and do not require a restart.**
|
|
261
300
|
|
|
262
301
|
### Disable / re-enable
|
|
263
302
|
|
|
@@ -313,6 +352,37 @@ pnpm dsh plugin --profile web remove dsh-vision-router
|
|
|
313
352
|
|
|
314
353
|
This removes the dependency and the bundle layer. If you disabled the stock DeepSeek row manually, re-enable it in your profile patch.
|
|
315
354
|
|
|
355
|
+
## Troubleshooting
|
|
356
|
+
|
|
357
|
+
### Startup fails with `Unexpected token ... is not valid JSON` (UTF-8 BOM)
|
|
358
|
+
|
|
359
|
+
**Symptom:** `dsh web` / `pnpm dsh web` exits immediately at startup:
|
|
360
|
+
|
|
361
|
+
```
|
|
362
|
+
SyntaxError: Unexpected token ...
|
|
363
|
+
is not valid JSON
|
|
364
|
+
at JSON.parse (<anonymous>)
|
|
365
|
+
at readProfileManifest (packages/boot/app-boot/src/profile.ts)
|
|
366
|
+
```
|
|
367
|
+
|
|
368
|
+
**Cause:** `~/.dsh/profiles/<profile>/package.json` was saved as **UTF-8 with BOM** by an editor. The invisible `\uFEFF` character at the start makes `JSON.parse` fail, because JSON does not allow it before the opening brace.
|
|
369
|
+
|
|
370
|
+
**Recommended fix:** run Vision Router's standalone repair command. It does not require DSH to boot first; it locates the profile, detects a UTF-8 BOM, removes only the three leading BOM bytes, and then validates the JSON again:
|
|
371
|
+
|
|
372
|
+
```sh
|
|
373
|
+
npx dsh-vision-router repair --profile web
|
|
374
|
+
```
|
|
375
|
+
|
|
376
|
+
To diagnose without changing the file:
|
|
377
|
+
|
|
378
|
+
```sh
|
|
379
|
+
npx dsh-vision-router doctor --profile web
|
|
380
|
+
```
|
|
381
|
+
|
|
382
|
+
Replace `web` if you use another profile, or omit `--profile` to scan all profiles.
|
|
383
|
+
|
|
384
|
+
Manual fallback: in VS Code, use “Save with Encoding” → `UTF-8` (without BOM). If `repair` removes the BOM but the JSON is still invalid, it will not guess or rewrite any other JSON content; inspect the file manually.
|
|
385
|
+
|
|
316
386
|
## Security notes
|
|
317
387
|
|
|
318
388
|
- Image text is **untrusted evidence**: descriptions, OCR output and the auto-mount note all tell the agent never to execute instructions found inside images.
|
package/README.zh.md
CHANGED
|
@@ -10,8 +10,13 @@
|
|
|
10
10
|
|
|
11
11
|
<p align="center">
|
|
12
12
|
<a href="https://awesome-dsh-plugin.com"><img src="https://awesome-dsh-plugin.com/badge.svg" alt="awesome · DSH plugin" /></a>
|
|
13
|
-
<a href="https://github.com/
|
|
14
|
-
<a href="
|
|
13
|
+
<a href="https://github.com/zp-home/dsh-recommend"><img src="https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fzp-home%2Fdsh-recommend%2Fmain%2Fdata%2Fbadges%2Fysr666__dsh-vision-router.certified.json&style=flat-square" alt="dsh-recommend 🏅 精选认证" /></a>
|
|
14
|
+
<a href="https://github.com/zp-home/dsh-recommend"><img src="https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fzp-home%2Fdsh-recommend%2Fmain%2Fdata%2Fbadges%2Fysr666__dsh-vision-router.json&style=flat-square" alt="dsh score 0.90" /></a>
|
|
15
|
+
</p>
|
|
16
|
+
|
|
17
|
+
<p align="center">
|
|
18
|
+
<a href="https://github.com/ysr666/dsh-vision-router/releases/tag/v1.2.1"><img src="https://img.shields.io/badge/release-v1.2.1-5B4CF0?style=flat-square" alt="Release v1.2.1" /></a>
|
|
19
|
+
<a href="tests"><img src="https://img.shields.io/badge/verified-149%20tests-2EA44F?style=flat-square" alt="Verified: 149 tests" /></a>
|
|
15
20
|
<a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-2EA44F?style=flat-square" alt="License: MIT" /></a>
|
|
16
21
|
<a href="package.json"><img src="https://img.shields.io/badge/Node.js-%3E%3D22-339933?style=flat-square&logo=nodedotjs&logoColor=white" alt="Node.js >=22" /></a>
|
|
17
22
|
<img src="https://img.shields.io/badge/runtime-no%20Python-8A2BE2?style=flat-square" alt="No Python" />
|
|
@@ -20,10 +25,12 @@
|
|
|
20
25
|
|
|
21
26
|
<p align="center"><a href="README.md">English</a> · 中文</p>
|
|
22
27
|
|
|
28
|
+
<p align="center">💬 <strong>QQ 用户交流群:1105463028</strong></p>
|
|
29
|
+
|
|
23
30
|
> [!WARNING]
|
|
24
|
-
> 📌 **公告(v1.
|
|
31
|
+
> 📌 **公告(v1.2.1)**
|
|
25
32
|
>
|
|
26
|
-
>
|
|
33
|
+
> v1.2.1 加固像素闭环:十个像素工具可直接接受上传图片的附件 ID(告别 `cannot read …/sha256:…` 的绕路),产物文件名带指纹不再互相覆盖,`vision_ground` 对退化框自动重试,模型引导支持从第 1 步完整重放(先退出设置页),设置卡片在数百模型目录下滚动依然流畅。
|
|
27
34
|
|
|
28
35
|
<p align="center">
|
|
29
36
|
<img src="assets/vision-demo.gif" width="640" alt="演示:粘贴图片,Agent 用 vision_ground / vision_crop / vision_pixel_diff 定位发送按钮并给出坐标" />
|
|
@@ -34,7 +41,7 @@
|
|
|
34
41
|
大多数 DSH 视觉插件把图片“翻译”成一段文字描述再喂给 DeepSeek——有损、一次性、看不见像素。本插件把**原图像素留在视觉模型侧**、把推理留在 DeepSeek 侧,并把“看图”变成一次**普通的工具调用**:
|
|
35
42
|
|
|
36
43
|
- **一条命令安装。** 包自带组合补丁(`dsh.bundle.patch`):`dsh plugin add` 自动完成插件行挂载、准入包装与附件限制放宽——不用手改任何文件。是否接管官方 DeepSeek 路由由「隐身模式」开关决定(默认关)。
|
|
37
|
-
- **默认免费。**
|
|
44
|
+
- **默认免费。** 视觉工具最终兜底为 5 个 OVHcloud 匿名视觉模型:免注册、免 Key,每 IP、每模型 2 次/分钟,独立限额理论合计约 10 次/分钟;用户自备视觉模型会优先调用。
|
|
38
45
|
- **无 Python。** 整条管线——缩放、定位、裁剪、像素对比、取色、OCR、SVG 矢量化、抠图、HTML 截图——全部基于 sharp / potrace / tesseract / 系统 Chrome。
|
|
39
46
|
- **可连续多步看图。** 图片轮 = 调用工具的文本轮:`vision_ground` → `vision_crop` → `vision_describe` → `vision_pixel_diff` → 修复 → 再截图,Agent 可以一直迭代到任务完成。
|
|
40
47
|
- **DeepSeek 始终是大脑。** 文字轮在模型、成本、上下文上完全不动;视觉模型只当“眼睛”、按需调用,答案按图片内容缓存。
|
|
@@ -48,17 +55,19 @@
|
|
|
48
55
|
|---|---|---|
|
|
49
56
|
| 开箱图片问答 | ✅ 内置免费视觉链(OVHcloud 匿名端点),免注册免 Key | 远程工具需自备视觉 API Key(本地像素工具免 Key) |
|
|
50
57
|
| 运行时 | ✅ 纯 Node,无需 Python | 需要 Python 3.11+ 受管运行时 |
|
|
51
|
-
| 图片怎么进来 | ✅
|
|
58
|
+
| 图片怎么进来 | ✅ 选一次「+ 自动识图」模型组后直接粘贴 | 工作区路径 + `/vision-tools` 命令,再显式调用工具 |
|
|
52
59
|
| 轮次路由 | ✅ 图片轮切视觉、文本轮切回 DeepSeek——可选隐身接管,模型选择器与官方一致 | 工具驱动,无整轮自动路由 |
|
|
53
60
|
| 支持 profile | Web | Web + Headless |
|
|
54
61
|
| 玩法库 | 像素循环:定位 → 裁剪 → 对比 → 修复 → 再截图 | 更丰富的案例库(长截图 OCR、UI 还原、GUI 自动化) |
|
|
55
|
-
| 测试 |
|
|
62
|
+
| 测试 | 144 | 162 |
|
|
56
63
|
| 安装 | 一条命令 | 一条命令(npm) |
|
|
57
64
|
|
|
58
65
|
两者都是 MIT 许可、一条命令安装。想要图片**粘贴即用**、零配置就选本插件;需要 Headless 部署或更丰富的案例库,可以看 @anionex/dsh-vision-toolkit。(功能对比以其 README 2026-08 状态为准。)
|
|
59
66
|
|
|
60
67
|
## 快速开始
|
|
61
68
|
|
|
69
|
+
### 1. 安装并让插件加载
|
|
70
|
+
|
|
62
71
|
普通 npm / npx 安装方式推荐这样用(与 DSH 官方 README 的启动方式一致):
|
|
63
72
|
|
|
64
73
|
```sh
|
|
@@ -74,14 +83,32 @@ pnpm dsh plugin --profile web add dsh-vision-router
|
|
|
74
83
|
pnpm dsh web
|
|
75
84
|
```
|
|
76
85
|
|
|
77
|
-
如果你已经全局安装 DSH CLI,并且终端里能直接执行 `dsh`,也可以继续使用较短的 `dsh ...`
|
|
86
|
+
如果你已经全局安装 DSH CLI,并且终端里能直接执行 `dsh`,也可以继续使用较短的 `dsh ...` 写法。
|
|
87
|
+
|
|
88
|
+
> [!NOTE]
|
|
89
|
+
> 如果你是把插件**首次安装进一个已经长期运行的 Web 进程**,需要让 DSH Web 进程重新加载一次插件本体。插件加载完成后,新增/删除模型、修改自动识图包装范围都会**热更新,无需再重启 DSH**。
|
|
90
|
+
|
|
91
|
+
### 2. 在聊天页切换到「+ 自动识图」模型组
|
|
92
|
+
|
|
93
|
+
插件加载后会自动发现 **设置 → 模型** 里已启用的模型组,并为它们额外创建同名的自动识图入口。例如:
|
|
94
|
+
|
|
95
|
+
```text
|
|
96
|
+
opencode-go ← 原模型组,保持不变
|
|
97
|
+
opencode-go + 自动识图 ← 发图片时选这个
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
> [!IMPORTANT]
|
|
101
|
+
> **发图前,请点击聊天页输入区右下角的模型选择器,选择带「+ 自动识图」的模型组。**
|
|
102
|
+
>
|
|
103
|
+
> Vision Router 故意**不修改原模型组**。因此如果当前仍选着原来的纯文本 `opencode-go` / DeepSeek 路由,DSH 会在插件处理图片之前先提示“当前模型不支持图片”。这不是视觉后端配置失败,只是还没有切到自动识图入口。
|
|
104
|
+
|
|
105
|
+
这个模型组的模型列表会跟随 DSH 的模型目录实时同步;新增模型或修改包装范围后无需重启。
|
|
106
|
+
|
|
107
|
+
### 3. 直接粘贴或上传图片
|
|
78
108
|
|
|
79
|
-
|
|
80
|
-
- 默认视觉链就是内置免费端点;
|
|
81
|
-
- opencode 等自定义/第三方路由用「额外识图包装」获得发图能力;
|
|
82
|
-
- 全部配置可在 **设置 → 插件 → 插件配置 → 视觉路由(自动识图)** 实时修改。
|
|
109
|
+
选好「+ 自动识图」模型组后,直接往对话里贴图即可。Agent 会自动挂载视觉工具,通过 `vision_describe`、`vision_ground`、`vision_crop` 等工具看图,需要时连续多步操作。
|
|
83
110
|
|
|
84
|
-
|
|
111
|
+
默认已经有内置 OVH 匿名视觉兜底,无需注册、无需 Key。**聊天页右下角只选择“脑子/会话模型”**;视觉模型不要在那里选。高级配置在 **设置 → 插件 → 插件配置 → 视觉路由(自动识图)**:视觉后端链每一行只选择一个你在 **设置 → 模型** 中已经配置且支持图片输入的用户模型;一行都不填也可以,OVH 免费链会固定在最后兜底。插件内部的 `Vision HTTP` 只是传输实现,不是用户需要选择的模型组。
|
|
85
112
|
|
|
86
113
|
### 实际效果
|
|
87
114
|
|
|
@@ -103,9 +130,9 @@ pnpm dsh web
|
|
|
103
130
|
|
|
104
131
|
### 像素闭环实测
|
|
105
132
|
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
133
|
+
[](assets/pixel-loop-zh.png)
|
|
134
|
+
|
|
135
|
+
<p align="center"><sub>点击图片可查看完整原图。</sub></p>
|
|
109
136
|
|
|
110
137
|
Agent 仅根据参考图复刻 UI,再用 `vision_pixel_diff` 验证最终结果:**最终差异 2.54%**(32,939 / 1,296,000 个差异像素,threshold 16/channel)。
|
|
111
138
|
|
|
@@ -159,11 +186,14 @@ vision_long_screenshot_ocr image="chat-log.png" chunkHeight=1200 overlap=120
|
|
|
159
186
|
|
|
160
187
|
## 供应商降级链
|
|
161
188
|
|
|
162
|
-
|
|
189
|
+
视觉工具按顺序逐个尝试,全部失败才报错:
|
|
163
190
|
|
|
164
|
-
1.
|
|
165
|
-
2.
|
|
166
|
-
3.
|
|
191
|
+
1. **用户视觉模型**:设置卡里一行一个,从上到下;只显示 **设置 → 模型** 中明确声明支持 image 输入的模型;
|
|
192
|
+
2. **高级自定义 HTTP 视觉端点**:如果旧配置/高级配置中存在 `httpProviders`,在用户模型之后尝试;
|
|
193
|
+
3. **内置 OVH 匿名免费兜底**:固定最后尝试,不需要出现在任何模型选择器里。当前内置链按质量优先为 `Qwen3.5-397B-A17B` → `Qwen2.5-VL-72B-Instruct` → `Qwen3.6-27B` → `Mistral-Small-3.2-24B-Instruct-2506` → `Qwen3.5-9B`。OVH 匿名限额为 **每 IP、每模型 2 次/分钟**;5 个模型是独立限额,因此理论上分散请求可到约 **10 次/分钟**,实际仍以 OVH 当时的限流为准。免注册、免 Key。
|
|
194
|
+
|
|
195
|
+
> [!IMPORTANT]
|
|
196
|
+
> 这里的“视觉链”是 Vision Router 调用的**眼睛**:设置页里每一行只选一个用户视觉模型;聊天页右下角选择的是**脑子/会话模型**,两者完全分开。纯文本 DeepSeek / opencode 不会出现在视觉后端下拉里;内部 `Vision HTTP` 也不会再暴露给用户。
|
|
167
197
|
|
|
168
198
|
> 在旧版 `routing: true` 模式下,整轮链只走 `provider + fallbacks`——`httpProviders`(含免费兜底)不参与。默认的 `routing: false`(工具优先)会尝试全部。
|
|
169
199
|
|
|
@@ -183,20 +213,28 @@ vision_long_screenshot_ocr image="chat-log.png" chunkHeight=1200 overlap=120
|
|
|
183
213
|
|
|
184
214
|
官方行在场时,插件自动回退为可见包装入口。反过来,隐身模式关闭但官方行仍被禁用时,插件会做 keep-alive 兜底接管,保住 DeepSeek 模型(设置卡片会给出提示);想完全恢复官方原生行,把上面的 `disabled` 改回 `false` 再重启即可。
|
|
185
215
|
|
|
186
|
-
> 隐身模式**只作用于官方 DeepSeek 路由**。opencode
|
|
216
|
+
> 隐身模式**只作用于官方 DeepSeek 路由**。opencode 等自定义/第三方文本路由与隐身模式无关——默认会被自动包装成「+ 自动识图」模型组。
|
|
217
|
+
|
|
218
|
+
## 自动识图模型组与手动包装
|
|
187
219
|
|
|
188
|
-
|
|
220
|
+
默认开启 `autoWrapProviders`:插件会自动发现 **设置 → 模型** 中当前已启用的 provider / model,并额外注册同名的「+ 自动识图」模型组。**原模型组完全不变**;发图片时选自动识图组,纯文字仍可继续用原组。DSH 的 `llm/adapters-updated` 变化会触发同步,所以新增/删除模型后无需重启。
|
|
189
221
|
|
|
190
|
-
`wrappedProviders`
|
|
222
|
+
`wrappedProviders` 是**可选的手动范围控制**,不是普通用户必须配置的步骤。只有两种情况需要它:
|
|
223
|
+
|
|
224
|
+
1. 关闭了自动包装,想手动指定哪些 provider / model 获得自动识图入口;
|
|
225
|
+
2. 自动包装保持开启,但只想让某个 provider 的部分模型出现在「+ 自动识图」组。
|
|
226
|
+
|
|
227
|
+
设置卡片里用两个下拉(provider + 模型)配置;模型留空 = 包装该路由的全部模型,同一 provider 要限定多个模型就添加多行。修改即时生效,无需重启。
|
|
191
228
|
|
|
192
229
|
## Web 设置
|
|
193
230
|
|
|
194
|
-
Web 配置页在 **设置 → 插件 → 插件配置**
|
|
231
|
+
Web 配置页在 **设置 → 插件 → 插件配置** 下注册「视觉路由(自动识图)」卡片,顶部会直接提示最重要的使用步骤:**回到聊天页 → 右下角模型选择器 → 选择「+ 自动识图」模型组 → 发图**。其余设置主要用于高级定制:
|
|
195
232
|
|
|
233
|
+
- **自动创建「+ 自动识图」模型组**:默认开启,自动发现已有模型;模型目录变化热更新,无需重启;
|
|
234
|
+
- **手动限定自动识图范围(可选)**:仅在需要关闭自动包装或限制部分模型时使用;
|
|
235
|
+
- **视觉后端链**:给 `vision_describe` 等视觉工具调用的真正图片模型,默认内置免费 Qwen 即可;不要填纯文本模型;
|
|
196
236
|
- 开关:整轮自动路由(旧模式)、识图工具、图片块改写、隐身模式(仅官方 DeepSeek 路由);
|
|
197
|
-
-
|
|
198
|
-
- 视觉请求超时、包装/链路由名;
|
|
199
|
-
- **视觉模型链**(每行一个 `provider/model`,自上而下降级)与文本模型;
|
|
237
|
+
- 视觉请求超时、包装/链路由名、代理等高级参数;
|
|
200
238
|
- 每个字段都有「已覆盖」徽标与一键恢复组合默认,以及放弃/保存;
|
|
201
239
|
- 「测试连接」按钮探测第一个视觉提供方并内联显示延迟/失败原因;
|
|
202
240
|
- 产出制品的工具在对话里渲染专用调用卡(关键字段 + 打开文件按钮)。
|
|
@@ -213,15 +251,16 @@ Web 配置页在 **设置 → 插件 → 插件配置** 下注册「视觉路由
|
|
|
213
251
|
|
|
214
252
|
| 字段 | 默认值 | 含义 |
|
|
215
253
|
|---|---|---|
|
|
216
|
-
| `provider` / `model` | `vision-http` / `ovh/Qwen2.5-VL-72B-Instruct` |
|
|
217
|
-
| `fallbacks` | `[]` |
|
|
218
|
-
| `providers` | 内置免费 `vision-http` 条目 |
|
|
254
|
+
| `provider` / `model` | `vision-http` / `ovh/Qwen2.5-VL-72B-Instruct` | 简写视觉后端链路(有适配器且真正支持图片输入的供应商 + 模型) |
|
|
255
|
+
| `fallbacks` | `[]` | 简写视觉供应商的备用图片模型 |
|
|
256
|
+
| `providers` | 内置免费 `vision-http` 条目 | 多供应商视觉后端链 `{ provider, model, fallbacks[] }`,按序尝试;优先于简写形式。不要填写纯文本模型 |
|
|
219
257
|
| `httpProviders` | 内置 OVH 条目 | OpenAI 兼容直连端点 `{ name, baseURL, model, apiKeyEnv, maxTokens }` |
|
|
220
|
-
| `
|
|
258
|
+
| `autoWrapProviders` | `true` | 自动发现当前已启用 provider / model,并热更新同名「+ 自动识图」模型组;原模型组不变 |
|
|
259
|
+
| `wrappedProviders` | `[{ provider: 'deepseek-official', models: [] }]` | 可选的手动包装范围 `{ provider, models[] }`;用于关闭自动包装后手动指定,或限制某个 provider 只包装部分模型。改动即时生效,无需重启 |
|
|
221
260
|
| `routing` | `false` | 旧版整轮链路由(一次性整轮回答)。`false` = 工具优先流程(推荐) |
|
|
222
261
|
| `reverseRouting` | `true` | 开启 `routing` 时,文字轮路由回 `textProvider` |
|
|
223
262
|
| `wrapperRoute` / `chainRoute` | `deepseek-vision` / `vision-chain` | 准入包装路由名 / 降级链路由名(置空关闭) |
|
|
224
|
-
| `stealth` | `false` | 接管官方 `deepseek-official`
|
|
263
|
+
| `stealth` | `false` | 接管官方 `deepseek-official` 路由(仅官方行;自定义路由默认由自动包装处理) |
|
|
225
264
|
| `textProvider` | `deepseek-official` / `deepseek-v4-pro` | 负责思考的模型(你的日常模型) |
|
|
226
265
|
| `tool` / `progressiveTools` / `autoActivateOnImage` | `true` ×3 | 视觉工具开关 / 渐进式挂载 / 图片轮自动挂载 |
|
|
227
266
|
| `rewriteImages` | `true` | 模型输入层改写图片块(缓存描述或工具提示标记);界面日志保留图片 |
|
|
@@ -257,7 +296,7 @@ pnpm dsh plugin --profile web add dsh-vision-router
|
|
|
257
296
|
pnpm dsh --profile web --dump-config | grep vision-router
|
|
258
297
|
```
|
|
259
298
|
|
|
260
|
-
|
|
299
|
+
首次把插件装进已经长期运行的 Web profile 时,需要让 Web 进程重新加载插件本体;宿主在启动时通过 `dsh.client` 声明发现浏览器端包。**插件加载完成后,模型目录与包装范围的变化会热更新,不需要为这些变化重启。**
|
|
261
300
|
|
|
262
301
|
### 禁用 / 恢复
|
|
263
302
|
|
|
@@ -311,6 +350,37 @@ pnpm dsh plugin --profile web remove dsh-vision-router
|
|
|
311
350
|
|
|
312
351
|
同时移除依赖与 bundle 层。若你曾手动禁用官方 DeepSeek 行,记得在 profile 补丁里恢复。
|
|
313
352
|
|
|
353
|
+
## 故障排查
|
|
354
|
+
|
|
355
|
+
### 启动报错 `Unexpected token ... is not valid JSON`(UTF-8 BOM)
|
|
356
|
+
|
|
357
|
+
**现象**:`dsh web` / `pnpm dsh web` 启动时直接退出:
|
|
358
|
+
|
|
359
|
+
```
|
|
360
|
+
SyntaxError: Unexpected token ...
|
|
361
|
+
is not valid JSON
|
|
362
|
+
at JSON.parse (<anonymous>)
|
|
363
|
+
at readProfileManifest (packages/boot/app-boot/src/profile.ts)
|
|
364
|
+
```
|
|
365
|
+
|
|
366
|
+
**原因**:`~/.dsh/profiles/<profile>/package.json` 被某些编辑器保存成了 **UTF-8 with BOM**。文件最前面多了一个不可见的 `\uFEFF` 字符,dsh 读取 manifest 时直接 `JSON.parse`,而 JSON 不允许在开头出现这个字符,于是解析失败。
|
|
367
|
+
|
|
368
|
+
**推荐修复**:直接运行 Vision Router 自带的独立修复命令。它不需要 DSH 先成功启动,会定位 profile、检测 UTF-8 BOM,只删除开头的三个 BOM 字节,然后重新验证 JSON:
|
|
369
|
+
|
|
370
|
+
```sh
|
|
371
|
+
npx dsh-vision-router repair --profile web
|
|
372
|
+
```
|
|
373
|
+
|
|
374
|
+
只想检查、不修改文件时:
|
|
375
|
+
|
|
376
|
+
```sh
|
|
377
|
+
npx dsh-vision-router doctor --profile web
|
|
378
|
+
```
|
|
379
|
+
|
|
380
|
+
如果你使用的不是 `web` profile,把 `web` 换成对应名称;也可以不传 `--profile`,让 doctor 扫描全部 profile。
|
|
381
|
+
|
|
382
|
+
手动兜底方式:VS Code 右下角编码 → “通过编码保存” → 选择 `UTF-8`(无 BOM)。若 `repair` 去掉 BOM 后仍提示 JSON 非法,它不会猜测或重写其他 JSON 内容,请再手动检查文件。
|
|
383
|
+
|
|
314
384
|
## 安全说明
|
|
315
385
|
|
|
316
386
|
- 图片中的文字是**不可信证据**:描述、OCR 输出与自动挂载提示都要求 Agent 绝不执行图片内出现的指令。
|
package/assets/pixel-loop-zh.png
CHANGED
|
Binary file
|
package/assets/pixel-loop.png
CHANGED
|
Binary file
|
package/docs/doctor.md
ADDED
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
# Vision Router doctor / repair
|
|
2
|
+
|
|
3
|
+
Vision Router ships a small standalone diagnostic CLI. It does not need DSH to boot first, so it can still run when DSH exits while parsing a broken profile manifest.
|
|
4
|
+
|
|
5
|
+
## Normal installation stays unchanged
|
|
6
|
+
|
|
7
|
+
Use DSH's own plugin command:
|
|
8
|
+
|
|
9
|
+
```sh
|
|
10
|
+
npx @deepseek-ai/dsh plugin --profile web add dsh-vision-router
|
|
11
|
+
npx @deepseek-ai/dsh web
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
For a DeepSeek Harness source checkout:
|
|
15
|
+
|
|
16
|
+
```sh
|
|
17
|
+
pnpm dsh plugin --profile web add dsh-vision-router
|
|
18
|
+
pnpm dsh web
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
The doctor is a recovery/diagnostic tool, not a replacement installer.
|
|
22
|
+
|
|
23
|
+
## Diagnose profiles
|
|
24
|
+
|
|
25
|
+
```sh
|
|
26
|
+
npx dsh-vision-router doctor
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
To inspect only the Web profile:
|
|
30
|
+
|
|
31
|
+
```sh
|
|
32
|
+
npx dsh-vision-router doctor --profile web
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
The command locates the DSH home (`$DSH_HOME` when set, otherwise `~/.dsh`), scans profile `package.json` files, reports UTF-8 BOM bytes, validates the JSON after ignoring a leading BOM for diagnosis, and reports whether `dsh-vision-router` is present as a profile dependency and bundle layer.
|
|
36
|
+
|
|
37
|
+
## Repair the UTF-8 BOM startup failure
|
|
38
|
+
|
|
39
|
+
If DSH fails before plugins can load with an error such as:
|
|
40
|
+
|
|
41
|
+
```text
|
|
42
|
+
SyntaxError: Unexpected token ... is not valid JSON
|
|
43
|
+
at readProfileManifest (.../profile.ts:...)
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
run:
|
|
47
|
+
|
|
48
|
+
```sh
|
|
49
|
+
npx dsh-vision-router repair --profile web
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
`repair` removes only the three-byte UTF-8 BOM prefix (`EF BB BF`) when it is present, then validates the remaining JSON. It does not reformat, regenerate, or otherwise rewrite the profile contents. If JSON is still invalid for another reason, the command reports that and stops rather than guessing a repair.
|