dsh-vision-router 1.7.7 → 2.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (71) hide show
  1. package/README.md +58 -62
  2. package/README.zh.md +65 -66
  3. package/docs/releases/v2.0.0.md +194 -0
  4. package/docs/releases/v2.0.1.md +17 -0
  5. package/docs/v2-capability-routing.md +191 -0
  6. package/entry.js +156 -132
  7. package/index.js +146 -5
  8. package/lib/catalog-corrections.js +2 -1
  9. package/lib/client-presentation-boundary-main.js +850 -0
  10. package/lib/client-presentation-boundary.js +17 -324
  11. package/lib/diagnostic-redaction.js +26 -0
  12. package/lib/doctor-cli.js +7 -0
  13. package/lib/doctor-vision-limits.js +72 -0
  14. package/lib/guide-vision-toggle-highlight.js +166 -0
  15. package/lib/html-script-marker.js +112 -0
  16. package/lib/legacy-core-vision-policy-bridge.js +84 -7
  17. package/lib/live-model-client-prelude.js +3 -1
  18. package/lib/live-model-discovery.js +4 -3
  19. package/lib/local-remote-settings-permission.js +2 -1
  20. package/lib/native-image-coexistence.js +5 -2
  21. package/lib/ollama-cold-start.js +2 -1
  22. package/lib/pi-ai-bridge-wire-compat.js +35 -4
  23. package/lib/public-entry.js +17 -0
  24. package/lib/remote-settings-bridge.js +10 -0
  25. package/lib/remote-settings-risk-confirmation.js +3 -1
  26. package/lib/runtime-config-normalizer.js +32 -7
  27. package/lib/runtime-reliability.js +2 -1
  28. package/lib/settings-client-rc8-lifecycle.js +3 -1
  29. package/lib/settings-ia-client-prelude.js +226 -0
  30. package/lib/settings-limit-client-prelude.js +110 -0
  31. package/lib/settings-migration.js +200 -0
  32. package/lib/settings-number-contract.js +29 -0
  33. package/lib/strict-live-model-client-prelude.js +137 -0
  34. package/lib/string-normalization.js +13 -0
  35. package/lib/structured-flow-hardening.js +8 -8
  36. package/lib/trusted-vision-hints.js +2 -1
  37. package/lib/update-check.js +2 -1
  38. package/lib/v2-settings-ia-integration.js +254 -0
  39. package/lib/version-range.js +13 -0
  40. package/lib/vision-background-benchmark.js +1015 -0
  41. package/lib/vision-background-stop-store.js +137 -0
  42. package/lib/vision-breaker-observer.js +27 -0
  43. package/lib/vision-breaker-shadow-health.js +99 -0
  44. package/lib/vision-capability-benchmark-client.js +437 -0
  45. package/lib/vision-capability-benchmark-hardening.js +150 -0
  46. package/lib/vision-capability-benchmark-service.js +965 -0
  47. package/lib/vision-capability-benchmark.js +524 -0
  48. package/lib/vision-capability-identity.js +73 -0
  49. package/lib/vision-capability-probe.js +429 -0
  50. package/lib/vision-capability-reference.js +135 -0
  51. package/lib/vision-capability-router.js +449 -0
  52. package/lib/vision-capability-shadow.js +576 -0
  53. package/lib/vision-exact-check-client.js +193 -0
  54. package/lib/vision-image-input-verdict.js +122 -0
  55. package/lib/vision-limit-diagnostics.js +202 -0
  56. package/lib/vision-model-visibility-boundary-main.js +469 -0
  57. package/lib/vision-model-visibility-boundary.js +28 -0
  58. package/lib/vision-resilience.js +34 -2
  59. package/lib/vision-routing-authority.js +70 -0
  60. package/lib/vision-routing-product.js +44 -0
  61. package/lib/vision-routing-settings-prelude.js +553 -0
  62. package/lib/vision-routing-topology-refresh.js +136 -0
  63. package/lib/vision-runtime-performance.js +304 -0
  64. package/lib/vision-toggle-root-hardening.js +484 -0
  65. package/lib/vision-tool-runtime-boundary.js +49 -5
  66. package/lib/vision-turn-budget-client-prelude.js +276 -0
  67. package/lib/web-capability-boundary.js +2 -6
  68. package/lib/wrapper-scope-client-prelude.js +15 -0
  69. package/package.json +6 -5
  70. package/lib/vision-backend-smoke-test-client.js +0 -249
  71. package/lib/vision-backend-smoke-test.js +0 -495
package/README.md CHANGED
@@ -4,9 +4,9 @@
4
4
 
5
5
  <h1 align="center">dsh-vision-router</h1>
6
6
 
7
- <p align="center"><strong>Paste an image and it just works — eyes for text-only agents on DeepSeek Harness. Free out of the box, no key, no Python, one command.</strong></p>
7
+ <p align="center"><strong>Turn vision on when you need it — eyes for text-only agents on DeepSeek Harness. Free out of the box, no key, no Python, one command.</strong></p>
8
8
 
9
- <p align="center">DeepSeek keeps thinking; the built-in free vision chain and fourteen deep tools do the seeing. Image turns behave like ordinary tool-calling turns — grounded, measurable, repeatable.</p>
9
+ <p align="center">DeepSeek keeps thinking; the built-in free vision chain and fourteen deep tools do the seeing. When an image matters, enable the composer’s “👁 Vision” control and use image turns like ordinary tool-calling turns — grounded, measurable, repeatable.</p>
10
10
 
11
11
  <p align="center">
12
12
  <a href="https://awesome-dsh-plugin.com"><img src="https://awesome-dsh-plugin.com/badge.svg" alt="awesome · DSH plugin" /></a>
@@ -16,8 +16,8 @@
16
16
  </p>
17
17
 
18
18
  <p align="center">
19
- <a href="https://github.com/ysr666/dsh-vision-router/releases/tag/v1.7.1"><img src="https://img.shields.io/badge/release-v1.7.1-5B4CF0?style=flat-square" alt="Release v1.7.1" /></a>
20
- <a href="tests"><img src="https://img.shields.io/badge/verified-657%20tests-2EA44F?style=flat-square" alt="Verified: 657 tests" /></a>
19
+ <a href="https://github.com/ysr666/dsh-vision-router/releases/tag/v2.0.0"><img src="https://img.shields.io/badge/release-v2.0.0-5B4CF0?style=flat-square" alt="Release v2.0.0" /></a>
20
+ <a href="tests"><img src="https://img.shields.io/badge/verified-Node%2022%20%2B%2024-2EA44F?style=flat-square" alt="Verified: Node 22 + 24" /></a>
21
21
  <a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-2EA44F?style=flat-square" alt="License: MIT" /></a>
22
22
  <a href="package.json"><img src="https://img.shields.io/badge/Node.js-%3E%3D22-339933?style=flat-square&amp;logo=nodedotjs&amp;logoColor=white" alt="Node.js >=22" /></a>
23
23
  <img src="https://img.shields.io/badge/runtime-no%20Python-8A2BE2?style=flat-square" alt="No Python" />
@@ -29,9 +29,9 @@
29
29
  <p align="center">💬 <strong>QQ community group: 1105463028</strong></p>
30
30
 
31
31
  > [!WARNING]
32
- > 📌 **Announcement (v1.7.1)**
32
+ > 📌 **Announcement (v2.0.0)**
33
33
  >
34
- > **v1.7.1:** Remote settings can now be enabled after an explicit risk confirmation.
34
+ > **v2.0.0:** Capability-aware Auto routing + benchmarks, explicit 👁 Vision, and Settings 2.0. [What’s new →](docs/releases/v2.0.0.md)
35
35
 
36
36
  <p align="center">
37
37
  <img src="assets/vision-demo.gif" width="640" alt="Demo: paste an image, the agent locates the send button with vision_ground / vision_crop / vision_pixel_diff and answers with coordinates" />
@@ -137,26 +137,30 @@ If you already installed the DSH CLI globally and `dsh` is on `PATH`, the shorte
137
137
  > [!NOTE]
138
138
  > If you install the plugin **into a Web process that was already running long-term**, let that DSH Web process reload once so the plugin bundle itself is discovered. After the plugin is loaded, adding/removing models or changing wrapper scope **hot-updates without further DSH restarts**.
139
139
 
140
- ### 2. Switch to a “+ Auto Vision” model group in chat
140
+ ### 2. Pick your normal model, then enable “👁 Vision” when needed
141
141
 
142
- Once loaded, the plugin discovers the model groups enabled under **Settings → Models** and creates same-name auto-vision entries. For example:
143
- ```text
144
- opencode-go ← original model group, unchanged
145
- opencode-go + Auto Vision ← choose this when sending images
146
- ```
142
+ The stock model selector in the lower-right corner still chooses your **brain/conversation model** — DeepSeek, Qwen, or any other ordinary route. Vision Router’s generated “+ Auto Vision” wrappers remain real Host routes for image admission, but are hidden from the stock picker and `/model` when ownership can be established safely.
143
+
144
+ When you need image input, explicitly click **“👁 Vision”** beside the composer:
145
+
146
+ - `👁 Vision`: the ordinary model is active and Vision is off;
147
+ - `👁 Vision ✓`: Vision Router has switched the session to that model’s internal vision wrapper;
148
+ - the mode persists across sends and **does not auto-reset**;
149
+ - turning it off switches back to the same ordinary model; choosing a different ordinary model turns Vision off;
150
+ - changing only reasoning effort keeps Vision on.
147
151
 
148
152
  > [!IMPORTANT]
149
- > **Before sending an image, open the model selector in the lower-right corner of the chat composer and choose a group marked “+ Auto Vision”.**
153
+ > **Pasting or uploading an image does not enable Vision automatically. Before sending an image, make sure the control shows `👁 Vision ✓`.**
150
154
  >
151
- > Vision Router deliberately **does not modify the original model group**. If the conversation still uses the original text-only opencode / DeepSeek route, DSH can reject the image with “the current model does not support images” *before Vision Router gets a chance to handle it*. That is a model-entry selection issue, not a broken vision backend.
152
-
153
- The auto-vision group follows the live DSH model catalog. Adding models or changing wrapper scope does not require a restart.
155
+ > The real wrapper route is still present underneath to satisfy DSH image admission. Hiding is presentation-only and fails open: if the browser cannot confidently prove a route belongs to Vision Router, that route remains visible rather than risking hiding a third-party provider.
154
156
 
155
157
  ### 3. Paste or upload the image
156
158
 
157
- After choosing the “+ Auto Vision” model group, paste or upload an image normally. By default the complete vision tool schema is stable from session start, so the agent can immediately use `vision_describe`, `vision_ground`, `vision_crop`, and the rest across multiple steps when needed.
159
+ With “👁 Vision” enabled, paste or upload an image normally. By default the complete vision tool schema is stable from session start, so the agent can immediately use `vision_describe`, `vision_ground`, `vision_crop`, and the rest across multiple steps when needed.
160
+
161
+ If the session already contains images, DSH may reject switching from a vision wrapper back to a text-only route. Vision Router does not bypass that Host rule: it shows a transient error using the same interaction style as the stock model selector, keeps the real current model unchanged, and leaves `👁 Vision ✓` reflecting the actual state so the session stays usable.
158
162
 
159
- The built-in anonymous OVH vision fallback is already configured, so normal image use needs no signup or API key. **The lower-right chat picker selects only the brain/conversation model**; vision backends do not belong there. Advanced options live under **Settings → Plugins → Plugin config → 视觉路由(自动识图)**: each vision-backend row may select any callable generative user model already configured under **Settings → Models**. DSH image-capability metadata is advisory only: undeclared or text-only-labelled models remain selectable and show a warning. At runtime Vision Router always tries the provider's registered DSH adapter first — including WebSocket, RPC and private transports — and falls through on a real failure. The direct compatibility bridge is used only when an http(s) OpenAI Chat Completions endpoint is positively identified. Leaving every user row empty is valid; the OVH chain remains the final fallback. `Vision HTTP` is an internal transport route, not a model group users should select.
163
+ The built-in anonymous OVH vision fallback is already configured, so normal image use needs no signup or API key. **The lower-right chat picker selects only the brain/conversation model**; vision backends do not belong there. Advanced options live under **Settings → Vision Router**: each vision-backend row may select any callable generative user model already configured under **Settings → Models**. DSH image-capability metadata is advisory only: undeclared or text-only-labelled models remain selectable and show a warning. At runtime Vision Router always tries the provider's registered DSH adapter first — including WebSocket, RPC and private transports — and falls through on a real failure. The direct compatibility bridge is used only when an http(s) OpenAI Chat Completions endpoint is positively identified. Leaving every user row empty is valid; the OVH chain remains the final fallback. `Vision HTTP` is an internal transport route, not a model group users should select.
160
164
 
161
165
  ### See it in action
162
166
 
@@ -190,6 +194,8 @@ Any of these channels can join the vision chain as an `httpProviders` entry (key
190
194
 
191
195
  ## Highlights
192
196
 
197
+ - **Capability-aware Auto routing.** Keep configured order for deterministic control, or explicitly enable Auto to prioritize already-configured models using measured capability evidence. Auto never infers capability from model names, and enabling Auto alone does not start benchmarks.
198
+ - **Verifiable model profiling.** Exact Test Vision sends one request to one exact model; Quick and Full benchmark OCR / general / structured / document / grounding capabilities. Background profiling is separately authorized and yields to real foreground vision work.
193
199
  - **Original pixels, real answers.** The vision chain reads the image at original resolution (auto-downscaled only to protect latency/quota); the agent's question travels with the image, so answers are about *your* question, not a generic description.
194
200
  - **Automatic failover with classified errors.** Region blocks, ToS filtering, 402 quota, 429 rate limits, context overflow, network failures — the chain walks providers one by one and only reports after all of them failed, with actionable advice. A 429 immediately advances to the next backend and opens a Retry-After-aware cooldown instead of sleeping inside the request.
195
201
  - **Image memory.** Vision answers are cached by attachment content hash; later text turns substitute the recorded description (marked as untrusted evidence), so DeepSeek genuinely remembers earlier images without re-spending vision calls.
@@ -280,7 +286,7 @@ Failures are classified (region / tos / quota / rate-limit / context / network)
280
286
 
281
287
  ## Stealth mode
282
288
 
283
- Stealth mode is **off by default** (explicit opt-in since issue #34): with it off, the official `deepseek-official` route stays untouched and image turns go through the visible "DeepSeek + 自动识图" wrapper entry in the picker.
289
+ Stealth mode is **off by default** (explicit opt-in since issue #34): with it off, the official `deepseek-official` route stays untouched. When you need images, the composer’s “👁 Vision” control switches to the internal DeepSeek wrapper, which is hidden from the stock picker and `/model` presentation by default.
284
290
 
285
291
  With stealth on, the plugin takes over the official `deepseek-official` route: the model picker looks exactly like stock (same DeepSeek group, same model names), but each entry is the auto-vision wrapper that declares image input and delegates text turns to a rebuilt native DeepSeek adapter (same `llm-deepseek` settings section and credentials). Old sessions keep working through the hidden `deepseek-vision` alias. The takeover requires the stock row to be absent — disable it in your profile patch layer (`~/.dsh/profiles/<profile>/cordis.patch.yml`):
286
292
 
@@ -290,52 +296,51 @@ With stealth on, the plugin takes over the official `deepseek-official` route: t
290
296
  disabled: true
291
297
  ```
292
298
 
293
- With the stock row present, the plugin falls back to the visible wrapper entry. Conversely, with stealth off but the stock row still disabled, the plugin performs a keep-alive takeover so the DeepSeek models don't vanish (the settings card explains this); to restore the fully official route, flip the `disabled` above back to `false` and restart.
299
+ With the stock row present, the plugin keeps the official route and uses the internal wrapper through “👁 Vision”. Conversely, with stealth off but the stock row still disabled, the plugin performs a keep-alive takeover so the DeepSeek models don't vanish (the settings card explains this); to restore the fully official route, flip the `disabled` above back to `false` and restart.
294
300
 
295
- > Stealth mode **only affects the official DeepSeek route**. Custom/third-party routes such as opencode are auto-wrapped into “+ Auto Vision” groups by default.
301
+ > Stealth mode **only affects the official DeepSeek route**. Custom/third-party routes such as opencode also receive internal vision wrappers by default, used through the composer toggle rather than a second user-facing model group.
296
302
 
297
- ## Auto-vision model groups and manual wrappers
303
+ ## Auto-vision wrappers and manual scope
298
304
 
299
- `autoWrapProviders` is on by default. The plugin discovers the provider/model entries currently enabled under **Settings → Models** and registers a same-name “+ Auto Vision” model group for them. **The original group is never changed**: choose the auto-vision group for images, or keep using the original group for plain text. DSH `llm/adapters-updated` events are synced live, so adding/removing models does not require a restart.
305
+ `autoWrapProviders` is on by default. The plugin discovers the provider/model entries currently enabled under **Settings → Models** and registers matching internal vision wrappers. **The original group is never changed.** Ordinary users do not need to find or manually select these routes: when ownership is confidently established, the wrappers are hidden from the stock picker and `/model`, and the composer’s “👁 Vision” control switches to them as needed. DSH `llm/adapters-updated` events are synced live, so adding/removing models does not require a restart.
300
306
 
301
307
  `wrappedProviders` is an **optional manual scope control**, not a required setup step. Use it only when:
302
308
 
303
- 1. automatic wrapping is off and you want to pick which provider/models receive an auto-vision entry; or
304
- 2. automatic wrapping remains on but one provider should expose only selected models in its “+ Auto Vision” group.
309
+ 1. automatic wrapping is off and you want to choose which provider/models can use “👁 Vision”; or
310
+ 2. automatic wrapping remains on but one provider should generate wrappers for only selected models.
305
311
 
306
- The settings card uses provider + model dropdowns; an empty model means every model on that route. Add multiple rows to select multiple models. Changes apply immediately with no restart.
312
+ The settings card uses provider + model dropdowns; an empty model means every model on that route. Add multiple rows to select multiple models. Changes apply immediately with no restart. If browser-side ownership or exact mirroring cannot be established, presentation hiding fails open so a third-party route is never hidden merely for cosmetic cleanliness.
307
313
 
308
314
  ## Web settings
309
315
 
310
- The Web profile registers a **视觉路由(自动识图)** card under **Settings → Plugins → Plugin config**. Its top callout spells out the only step most users need: **return to chat → lower-right model selector → choose a “+ Auto Vision” model group → send the image**. The remaining controls are advanced customization:
316
+ The Web profile registers a first-class **Settings → Vision Router** surface. Its General page keeps model choice and v2 routing authority together; Vision Strategy, Local & Device, Advanced and Diagnostics separate tool behavior, local backends, sensitive/performance controls and troubleshooting.
311
317
 
312
- - **Auto-create “+ Auto Vision” model groups**: enabled by default; follows the live model catalog with no restart;
313
- - **Manual auto-vision scope (optional)**: only for disabling auto-wrap or limiting selected models;
314
- - **Vision backend chain**: the real image-capable models used by `vision_describe` and friends; the built-in free Qwen is normally enough, and text-only models should not be placed here;
315
- - switches for legacy whole-turn routing, vision tools, image-block rewriting and stealth mode (official DeepSeek route only);
316
- - timeout, wrapper/chain route names, proxy and other advanced parameters;
317
- - every field shows an overridden badge with one-click reset plus discard/save;
318
- - a **Test connection** button prioritizes an enabled local backend, verifies that its configured model appears in `/v1/models`, and otherwise probes the first usable vision provider;
319
- - artifact-producing tools render dedicated call cards with result facts and open-file buttons.
318
+ - **Vision model chain**: the real image-capable models used by `vision_describe` and friends; the built-in free chain remains the final fallback;
319
+ - **Model selection**: keep the configured order, or explicitly enable capability-aware Auto with Balanced / Quality / Speed / Local preference;
320
+ - **Background capability data**: `off`, `local-free`, or `all`; this is separately authorized and never turns on merely because Auto was enabled;
321
+ - **Test Vision / Benchmark**: exact one-request image verification plus Quick (~3 requests, OCR + General) and Full (~6 requests, Structured + OCR + Document + Grounding + General) profiling; benchmark work continues if Settings is closed;
322
+ - **Local & Device**: Ollama / LM Studio and privacy-gated desktop screenshot controls;
323
+ - **Advanced / Diagnostics**: timeout, wrapper scope, proxy/network, compatibility, version, runtime status and troubleshooting.
320
324
 
321
325
  <p align="center">
322
- <img src="assets/vision-settings.png" width="72%" alt="The vision-router card in Settings > Plugins > Plugin config." />
326
+ <img src="assets/vision-settings.png" width="72%" alt="The Vision Router settings surface." />
323
327
  </p>
324
328
 
325
- > PR [#8](https://github.com/ysr666/dsh-vision-router/pull/8) upgrades the panel with catalog-driven model dropdowns, add/remove fallback rows, and proxy settings.
326
-
327
329
  ## Configuration
328
330
 
329
- Everything is optional; defaults work out of the box. Edit via the Web card or a profile patch:
331
+ Everything is optional; defaults work out of the box. Prefer **Settings → Vision Router**; profile overrides remain available for advanced deployments:
330
332
 
331
333
  | Field | Default | Meaning |
332
334
  |---|---|---|
335
+ | `routingMode` | `ordered` | `ordered` keeps the configured model-chain order; `auto` delegates prioritization to measured capability evidence. Auto is never enabled by migration |
336
+ | `routingPreference` | `balanced` | Auto preference: `balanced`, `quality`, `speed`, or `local`; changes ordering only among already-authorized candidates |
337
+ | `backgroundBenchmarking` | `off` | background capability profiling authority: `off`, `local-free`, or `all`; enabling Auto does not change it, and authorized background work runs only while Auto is active |
333
338
  | `provider` / `model` | `vision-http` / `ovh/Qwen2.5-VL-72B-Instruct` | shorthand **vision backend** route (adapter-backed provider + model that genuinely accepts images) |
334
339
  | `fallbacks` | `[]` | backup image models for the shorthand vision provider |
335
340
  | `providers` | built-in free `vision-http` pair | multi-provider **vision backend** chain `{ provider, model, fallbacks[] }`, tried in order; do not put text-only models here |
336
341
  | `httpProviders` | built-in OVH entry | direct OpenAI-compatible endpoints `{ name, baseURL, model, apiKeyEnv, maxTokens }` |
337
- | `autoWrapProviders` | `true` | discover enabled provider/models and live-sync same-name “+ Auto Vision” groups; original groups stay unchanged |
338
- | `wrappedProviders` | `[{ provider: 'deepseek-official', models: [] }]` | optional manual wrapper scope `{ provider, models[] }`, used after disabling auto-wrap or to restrict one provider to selected models; changes apply live, no restart |
342
+ | `autoWrapProviders` | `true` | discover enabled provider/models and live-sync their internal vision wrappers; confidently owned wrappers are hidden from the stock model picker while original groups stay unchanged |
343
+ | `wrappedProviders` | `[{ provider: 'deepseek-official', models: [] }]` | optional manual wrapper scope `{ provider, models[] }`, used after disabling auto-wrap or to restrict which models can enter an internal wrapper through “👁 Vision”; changes apply live, no restart |
339
344
  | `routing` | `false` | legacy whole-turn chain routing (one-shot answer). `false` = tools-first flow (recommended) |
340
345
  | `reverseRouting` | `true` | with `routing: true`, route text turns back to `textProvider` |
341
346
  | `wrapperRoute` / `chainRoute` | `deepseek-vision` / `vision-chain` | admission wrapper route name / fallback chain route name (empty disables) |
@@ -345,26 +350,21 @@ Everything is optional; defaults work out of the box. Edit via the Web card or a
345
350
  | `rewriteImages` | `true` | rewrite image blocks in the model input (cached description or tool-hint marker); the UI log keeps images |
346
351
  | `desktopScreenshot` | `false` | privacy opt-in for the model-callable `vision_screenshot` desktop-capture tool; checked live before every capture |
347
352
  | `freeFallback` | `true` | append the anonymous OVH models after explicit local/custom HTTP backends; turning this off never disables an explicitly configured local backend |
348
- | `localOllama` | `{ enabled: false, baseURL: 'http://127.0.0.1:11434/v1', model: 'qwen2.5vl', format: 'openai' }` | **Local vision backend (merged from dsh-vision)**: when enabled, `local-ollama` leads the HTTP vision chain; skipped automatically when Ollama is down; `format` selects `openai` (`/chat/completions`) or `anthropic` (`/messages`); optional `temperature` / `top_p` are sent only when explicitly set. v1.7 prewarms loopback models and renews a 30-minute residency so cold loading is not charged to the normal inference deadline |
349
- | `localLmStudio` | `{ enabled: false, baseURL: 'http://localhost:1234/v1', model: '', format: 'openai' }` | **Local LM Studio backend (merged from dsh-vision)**: follows Ollama and precedes custom/cloud HTTP backends; enabling it requires the real model identifier shown in LM Studio Developer or returned by `/v1/models`; supports the same optional sampling fields, while `format: 'anthropic'` requires LM Studio 0.4.1+ |
350
- | `instantDescribe` | `false` | **Instant local translation (merged from dsh-vision)**: when on and at least one local backend is usable, uncached image blocks are recognized before the first model step; Ollama is tried before LM Studio with a shared timeout budget, multi-image batches run concurrently (up to 3), and failures fall back to the static tool-hint marker |
351
- | `localDescribeStyle` | `plain` | **Local recognition output style (merged from dsh-vision)**: `plain` = flat description; `structured` = structured recognition (【初步判断】/【细节】/【空间结构】/【原图尺寸】), better for screenshot analysis |
353
+ | `localOllama` | `{ enabled: false, baseURL: 'http://127.0.0.1:11434/v1', model: 'qwen2.5vl', format: 'openai' }` | local vision backend; when enabled, `local-ollama` leads the HTTP vision chain, is skipped automatically when down, and supports OpenAI or Anthropic wire format |
354
+ | `localLmStudio` | `{ enabled: false, baseURL: 'http://localhost:1234/v1', model: '', format: 'openai' }` | local LM Studio backend after Ollama; enter the exact model identifier from LM Studio Developer or `/v1/models` |
355
+ | `visionTurnBudgetMs` | `0` | whole-turn vision wall-clock budget; `0` means unlimited. Concrete provider calls/tools still keep their own hard deadlines |
352
356
  | `downscale` / `downscaleMaxPixels` | `true` / `4000000` | pre-call downscale and its pixel budget (latency guard) |
353
357
  | `cache` / `cacheTtlSeconds` / `cacheMaxEntries` | `true` / `3600` / `200` | vision answer cache |
354
358
  | `timeoutMs` | `120000` | per vision call deadline |
355
359
  | `artifactsDir` | `.dsh-vision-router/artifacts` | artifact directory (relative to the session workspace) |
356
360
  | `proxy` / `proxyHosts` | `''` / openrouter hosts | optional proxy for vision provider hosts only |
357
- | `catalogCorrections` | `true` | built-in catalog-routing corrections: when the installed pi-ai catalog routes a known model to the wrong wire protocol (e.g. `opencode-go/qwen3.6-plus` to OpenAI chat completions while OpenCode Go only serves it on `/v1/messages`), the plugin answers that backend directly over the corrected protocol. Each correction disarms itself once the catalog is fixed upstream |
361
+ | `catalogCorrections` | `true` | built-in catalog-routing corrections for known upstream wire-protocol mismatches; each correction disarms itself once the catalog is fixed upstream |
358
362
 
359
363
  ### Local Ollama vision backend (merged from dsh-vision)
360
364
 
361
365
  > **Incremental author**: [shaoqiuyuavailable](https://github.com/shaoqiuyuavailable) (router local-vision increment)
362
366
  >
363
- > **Design credit**: the local vision backends (Ollama / LM Studio dual backends, instant recognition,
364
- > structured output, screenshot identification, same-image memory dedup, failure-fallback placeholder,
365
- > concurrency anti-snowball, timeout protection) inherit their design from
366
- > [dsh-vision](https://github.com/shaoqiuyuavailable/text-llm-vision/tree/dsh-vision) —
367
- > merged into the HTTP vision chain here, with per-level fallback and dual-protocol support added on top.
367
+ > **Design credit**: the local vision backends (Ollama / LM Studio dual backends, structured recognition, screenshot identification, same-image memory dedup, failure fallback, concurrency protection and timeout handling) inherit their design from [dsh-vision](https://github.com/shaoqiuyuavailable/text-llm-vision/tree/dsh-vision) — merged into the HTTP vision chain here, with per-level fallback and dual-protocol support added on top.
368
368
 
369
369
  An optional keyless local-first vision path for private, free, offline recognition. It plugs into the existing HTTP vision chain as `local-ollama`; if it fails, any configured cloud backends can still be tried unless you deliberately configure a local-only chain.
370
370
 
@@ -375,31 +375,27 @@ An optional keyless local-first vision path for private, free, offline recogniti
375
375
  ollama pull qwen2.5vl
376
376
  ```
377
377
 
378
- **2. Enable it** — in the settings card's "Local vision" group, or via a profile patch:
378
+ **2. Enable it** — under **Settings → Vision Router → Local & Device**, or via a profile patch:
379
379
 
380
380
  ```yaml
381
381
  - id: vision-router
382
382
  config:
383
383
  localOllama:
384
384
  enabled: true
385
- baseURL: 'http://127.0.0.1:11434/v1' # OpenAI-compatible endpoint
385
+ baseURL: 'http://127.0.0.1:11434/v1'
386
386
  model: 'qwen2.5vl'
387
- temperature: 0.5 # optional; low temperature is steadier for recognition
388
- top_p: 0.8 # optional; unset = server default
389
- instantDescribe: true # recognize images on the first model step
390
- localDescribeStyle: 'structured' # 'plain' | 'structured'
387
+ temperature: 0.5
388
+ top_p: 0.8
391
389
  ```
392
390
 
393
391
  **3. What happens**
394
392
 
395
393
  - When enabled, `local-ollama` heads the HTTP vision chain. For a strict local-only setup, remove cloud vision rows/custom HTTP endpoints and turn off `freeFallback`.
396
- - **v1.7 cold-start handling:** the selected loopback Ollama model is prewarmed through Ollama's native API and kept resident for 30 minutes. If it is cold when Ollama is the primary image backend, loading completes before the normal vision-task budget starts; a short `/api/ps` probe keeps a dead service on the fast fallback path. Remote Ollama URLs are never auto-warmed.
397
- - **LM Studio works the same way** — enable `localLmStudio` in the same "Local vision" group with its OpenAI-compatible endpoint (default `http://localhost:1234/v1`) and enter the exact model identifier shown in Developer or `/v1/models`. It sits after `local-ollama` and before custom/cloud HTTP backends.
394
+ - The selected loopback Ollama model is prewarmed through Ollama's native API and kept resident for 30 minutes. If it is cold when Ollama is the primary image backend, loading completes before the normal vision-task budget starts; a short `/api/ps` probe keeps a dead service on the fast fallback path. Remote Ollama URLs are never auto-warmed.
395
+ - **LM Studio works the same way** — enable `localLmStudio` with its OpenAI-compatible endpoint (default `http://localhost:1234/v1`) and enter the exact model identifier shown in Developer or `/v1/models`. It sits after `local-ollama` and before custom/cloud HTTP backends.
398
396
  - Each local backend can speak **OpenAI or Anthropic format** via `format` (default `openai`). Anthropic mode routes to `/v1/messages` with `anthropic-version` and base64 image sources; `x-api-key` is sent only when a key is configured. LM Studio needs version 0.4.1 or newer for this endpoint.
399
397
  - If a local backend is down or the call times out, its entry is skipped automatically and the chain falls through to the cloud backends — no call breaks.
400
- - `instantDescribe` tries enabled local backends in order (Ollama, then LM Studio) before the first model step. Multiple uncached images run concurrently (up to 3); one failed image does not block the others, and attachment-memory hits are reused without another local request.
401
398
  - `vision_screenshot` is disabled by default. After the separate Desktop screenshot opt-in, `identify=true` uses the same Ollama → LM Studio fallback.
402
- - Verify runtime decisions with `image turn — instantDescribe=… localBackends=…` and results with `instant local describe recognized N/M uncached image(s), C cached, F failed attempts` in the log.
403
399
 
404
400
  ## Requirements
405
401
 
@@ -581,4 +577,4 @@ Manual fallback: in VS Code, use “Save with Encoding” → `UTF-8` (without B
581
577
  <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/ysr666/dsh-vision-router/star-history/assets/star-history/star-history-dark.svg">
582
578
  <img alt="Star history chart" src="https://raw.githubusercontent.com/ysr666/dsh-vision-router/star-history/assets/star-history/star-history-light.svg" width="100%">
583
579
  </picture>
584
- </p>
580
+ </p>
package/README.zh.md CHANGED
@@ -4,9 +4,9 @@
4
4
 
5
5
  <h1 align="center">dsh-vision-router</h1>
6
6
 
7
- <p align="center"><strong>图片粘贴即用:给 DeepSeek Harness 的纯文本 Agent 装上“眼睛”——开箱免费、免 Key、无 Python、一条命令安装。</strong></p>
7
+ <p align="center"><strong>一键开启识图:给 DeepSeek Harness 的纯文本 Agent 装上“眼睛”——开箱免费、免 Key、无 Python、一条命令安装。</strong></p>
8
8
 
9
- <p align="center">DeepSeek 只负责思考,内置免费视觉链 + 14 个深看工具负责“看”;图片轮次就像普通工具调用一样自然、可定位、可验证。</p>
9
+ <p align="center">DeepSeek 只负责思考,内置免费视觉链 + 14 个深看工具负责“看”;需要看图时开启输入框旁的「👁 识图」,图片轮次就像普通工具调用一样自然、可定位、可验证。</p>
10
10
 
11
11
  <p align="center">
12
12
  <a href="https://awesome-dsh-plugin.com"><img src="https://awesome-dsh-plugin.com/badge.svg" alt="awesome · DSH plugin" /></a>
@@ -16,8 +16,8 @@
16
16
  </p>
17
17
 
18
18
  <p align="center">
19
- <a href="https://github.com/ysr666/dsh-vision-router/releases/tag/v1.7.1"><img src="https://img.shields.io/badge/release-v1.7.1-5B4CF0?style=flat-square" alt="Release v1.7.1" /></a>
20
- <a href="tests"><img src="https://img.shields.io/badge/verified-657%20tests-2EA44F?style=flat-square" alt="Verified: 657 tests" /></a>
19
+ <a href="https://github.com/ysr666/dsh-vision-router/releases/tag/v2.0.0"><img src="https://img.shields.io/badge/release-v2.0.0-5B4CF0?style=flat-square" alt="Release v2.0.0" /></a>
20
+ <a href="tests"><img src="https://img.shields.io/badge/verified-Node%2022%20%2B%2024-2EA44F?style=flat-square" alt="Verified: Node 22 + 24" /></a>
21
21
  <a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-2EA44F?style=flat-square" alt="License: MIT" /></a>
22
22
  <a href="package.json"><img src="https://img.shields.io/badge/Node.js-%3E%3D22-339933?style=flat-square&amp;logo=nodedotjs&amp;logoColor=white" alt="Node.js >=22" /></a>
23
23
  <img src="https://img.shields.io/badge/runtime-no%20Python-8A2BE2?style=flat-square" alt="No Python" />
@@ -29,9 +29,9 @@
29
29
  <p align="center">💬 <strong>QQ 用户交流群:1105463028</strong></p>
30
30
 
31
31
  > [!WARNING]
32
- > 📌 **公告(v1.7.1)**
32
+ > 📌 **公告(v2.0.0)**
33
33
  >
34
- > **v1.7.1:远程设置现可在风险确认后开启。**
34
+ > **v2.0.0:Auto 能力路由+测评、输入框识图、设置 2.0。** [查看完整更新 →](docs/releases/v2.0.0.md)
35
35
 
36
36
  <p align="center">
37
37
  <img src="assets/vision-demo.gif" width="640" alt="演示:粘贴图片,Agent 用 vision_ground / vision_crop / vision_pixel_diff 定位发送按钮并给出坐标" />
@@ -135,27 +135,30 @@ pnpm dsh plugin --profile web add dsh-vision-router
135
135
  > [!NOTE]
136
136
  > 如果你是把插件**首次安装进一个已经长期运行的 Web 进程**,需要让 DSH Web 进程重新加载一次插件本体。插件加载完成后,新增/删除模型、修改自动识图包装范围都会**热更新,无需再重启 DSH**。
137
137
 
138
- ### 2. 在聊天页切换到「+ 自动识图」模型组
138
+ ### 2. 选择日常模型,按需开启「👁 识图」
139
139
 
140
- 插件加载后会自动发现 **设置 → 模型** 里已启用的模型组,并为它们额外创建同名的自动识图入口。例如:
140
+ 聊天页右下角的原生模型选择器只负责选择你的**脑子/会话模型**,例如 DeepSeek、Qwen 或其他普通模型。Vision Router 生成的内部「+ 自动识图」wrapper 默认不会出现在原生模型列表和 `/model` 中。
141
141
 
142
- ```text
143
- opencode-go ← 原模型组,保持不变
144
- opencode-go + 自动识图 ← 发图片时选这个
145
- ```
142
+ 需要看图时,在输入框旁主动点击 **「👁 识图」**:
143
+
144
+ - `👁 识图`:当前普通模型,识图关闭;
145
+ - `👁 识图 ✓`:已切到该模型对应的 Vision Router 内部识图 route;
146
+ - 开启后会持续生效,发送消息后**不会自动复位**;
147
+ - 主动关闭会切回同一个普通模型;手动选择另一个普通模型会关闭识图;
148
+ - 只修改当前模型的 reasoning effort 不会关闭识图。
146
149
 
147
150
  > [!IMPORTANT]
148
- > **发图前,请点击聊天页输入区右下角的模型选择器,选择带「+ 自动识图」的模型组。**
151
+ > **上传 / 粘贴图片不会替你自动开启识图。发图前请先确认按钮处于 `👁 识图 ✓`。**
149
152
  >
150
- > Vision Router 故意**不修改原模型组**。因此如果当前仍选着原来的纯文本 `opencode-go` / DeepSeek 路由,DSH 会在插件处理图片之前先提示“当前模型不支持图片”。这不是视觉后端配置失败,只是还没有切到自动识图入口。
153
+ > Vision Router 仍保留真实 wrapper route 来通过 DSH 的图片准入,只是把它们作为内部实现隐藏起来。若浏览器端无法安全确认某个 route 属于 Vision Router,隐藏逻辑会 fail-open:宁可显示该 route,也不会误藏第三方模型。
151
154
 
152
- 这个模型组的模型列表会跟随 DSH 的模型目录实时同步;新增模型或修改包装范围后无需重启。
155
+ ### 3. 粘贴或上传图片
153
156
 
154
- ### 3. 直接粘贴或上传图片
157
+ 开启「👁 识图」后,直接往对话里贴图即可。默认情况下完整视觉工具表从会话开始就保持稳定,Agent 可直接调用 `vision_describe`、`vision_ground`、`vision_crop` 等工具看图,需要时连续多步操作。
155
158
 
156
- 选好「+ 自动识图」模型组后,直接往对话里贴图即可。默认情况下完整视觉工具表从会话开始就保持稳定,Agent 可直接调用 `vision_describe`、`vision_ground`、`vision_crop` 等工具看图,需要时连续多步操作。
159
+ 如果当前 session 已经包含图片,DSH 可能拒绝从识图 route 切回不接受图片的纯文本 route。此时 Vision Router 不绕过 Host 约束:会显示与原生模型选择器一致的临时错误提示,真实模型保持不变,`👁 识图 ✓` 也继续反映真实状态,可继续使用或稍后重试。
157
160
 
158
- 默认已经有内置 OVH 匿名视觉兜底,无需注册、无需 Key。**聊天页右下角只选择“脑子/会话模型”**;视觉模型不要在那里选。高级配置在 **设置 → 插件 → 插件配置 → 视觉路由(自动识图)**:视觉后端链每一行都可以选择 **设置 → 模型** 中任意可调用的生成式用户模型。DSH 的图片能力声明现在只作提示:未声明图片能力、甚至被标成仅文本的模型也会列出并给出警告。运行时永远先通过该供应商已注册的 DSH adapter 实际调用,因此 WebSocket、RPC 和私有协议都保留原生传输;只有明确识别为 http(s) OpenAI Chat Completions 的渠道才可能进入 HTTP 直连兼容兜底。实际调用失败后自动尝试下一后端;一行都不填也可以,OVH 免费链会固定在最后兜底。插件内部的 `Vision HTTP` 只是传输实现,不是用户需要选择的模型组。
161
+ 默认已经有内置 OVH 匿名视觉兜底,无需注册、无需 Key。**聊天页右下角只选择“脑子/会话模型”**;视觉模型不要在那里选。高级配置在 **设置 → Vision Router**:视觉后端链每一行都可以选择 **设置 → 模型** 中任意可调用的生成式用户模型。DSH 的图片能力声明现在只作提示:未声明图片能力、甚至被标成仅文本的模型也会列出并给出警告。运行时永远先通过该供应商已注册的 DSH adapter 实际调用,因此 WebSocket、RPC 和私有协议都保留原生传输;只有明确识别为 http(s) OpenAI Chat Completions 的渠道才可能进入 HTTP 直连兼容兜底。实际调用失败后自动尝试下一后端;一行都不填也可以,OVH 免费链会固定在最后兜底。插件内部的 `Vision HTTP` 只是传输实现,不是用户需要选择的模型组。
159
162
 
160
163
  ### 实际效果
161
164
 
@@ -178,7 +181,7 @@ opencode-go + 自动识图 ← 发图片时选这个
178
181
  | Intern AI(上海AI实验室) | `internvl-latest` · `internvl3.5-latest` | 30 RPM,**9000 万 token/月** | ✅ | chat.intern-ai.org.cn |
179
182
  | Groq | `meta-llama/llama-4-scout-17b-16e-instruct`(原生多模态,最多 5 张图) | 30 RPM / 14,400 次/天,免卡 | ❌ 需代理 | console.groq.com |
180
183
  | Google AI Studio | `gemini-2.5-flash` · `gemini-2.5-flash-lite` | 10–30 RPM / 500–1,500 次/天 | ❌ 需代理 | aistudio.google.com |
181
- | NVIDIA NIM | `meta-llama/llama-3.2-11b-vision-instruct` · `nvidia/nemotron-nano-12b-v2-vl` | 40 RPM,免卡 | ⚠️ | build.nvidia.com |
184
+ | NVIDIA NIM | `meta/llama-3.2-11b-vision-instruct` · `nvidia/nemotron-nano-12b-v2-vl` | 40 RPM,免卡 | ⚠️ | build.nvidia.com |
182
185
  | OpenCode Zen | `mimo-v2.5-free`(视觉 + 代码) | 30 RPM / 500 次/天 | ⚠️ | opencode.ai/zen |
183
186
  | OpenRouter | `google/gemma-4-26b-a4b-it:free` · `google/gemma-4-31b-it:free` | 未充值账户 50 次/天 | ❌ 需代理 | openrouter.ai |
184
187
 
@@ -189,6 +192,8 @@ opencode-go + 自动识图 ← 发图片时选这个
189
192
 
190
193
  ## 亮点
191
194
 
195
+ - **能力感知 Auto 路由。** 想要确定性就继续按配置顺序;想自动选择时再显式开启 Auto,只在已配置模型和已有实测证据上调整优先级。不会通过模型名猜能力,单纯开启 Auto 也不会自动发起测评。
196
+ - **可验证的模型测评。** 「测试识图」只向当前精确模型发一次请求;Quick / Full 分别测 OCR、通用理解,以及结构化、文档、定位等能力。后台能力数据是独立授权,并会给真实前台识图让路。
192
197
  - **原图像素,真实答案。** 视觉链按原始分辨率读图(仅为保护延迟/额度自动缩放);你的问题随图一起发送,答案围绕*你的问题*,而不是一段泛泛的描述。
193
198
  - **自动降级 + 分类报错。** 地区限制、ToS 风控、402 额度、429 限流、上下文超长、网络故障——链路逐供应商尝试,全部失败才报错并给出可操作的建议。遇到 429 会立即尝试下一后端,并按 Retry-After 开启冷却,不会在单次请求内睡眠等待。
194
199
  - **图片记忆。** 视觉答案按附件内容哈希缓存;后续文字轮用记录的描述替换历史图片(标注为不可信证据),DeepSeek 真正“记得”之前发过的图,且不重复消耗视觉调用。
@@ -220,7 +225,7 @@ Agent 仅根据参考图复刻 UI,再用 `vision_pixel_diff` 验证最终结
220
225
  <img src="assets/vision-tools-zh.svg" width="100%" alt="DSH Vision Router 的 11 个图像处理工具。" />
221
226
  </p>
222
227
 
223
- 图中展示 11 个图像处理工具;另有负责持久展示图片的 `vision_present` 与可选 1+x 结构化首遍识别的 `vision_bootstrap`,默认深看工具集共 13 个。若启动时显式开启隐私敏感的 `vision_screenshot`,则额外增加为第 14 个工具。
228
+ 图中展示 11 个图像处理工具;另有 `vision_materialize`、负责持久展示图片的 `vision_present` 与可选 1+x 结构化首遍识别的 `vision_bootstrap`,默认深看工具集共 14 个。若启动时显式开启隐私敏感的 `vision_screenshot`,则额外增加为第 15 个工具。
224
229
 
225
230
  | 工具 | 作用 | 产物 |
226
231
  |---|---|---|
@@ -264,7 +269,7 @@ vision_long_screenshot_ocr image="chat-log.png" chunkHeight=1200 overlap=120
264
269
 
265
270
  视觉工具按顺序逐个尝试,全部失败才报错:
266
271
 
267
- 1. **用户视觉模型**:设置卡里一行一个,从上到下;已启用供应商即使模型枚举部分失败也会保留在下拉中,可调用的生成式模型继续可选,图片能力声明只作提示,最终以运行时实际调用为准;
272
+ 1. **用户视觉模型**:设置页里一行一个,从上到下;已启用供应商即使模型枚举部分失败也会保留在下拉中,可调用的生成式模型继续可选,图片能力声明只作提示,最终以运行时实际调用为准;
268
273
  2. **本地 Ollama(可选,默认关)**:`localOllama.enabled` 开启后,通过本机 Ollama 做免 Key、离线识别(例如 qwen2.5vl);
269
274
  3. **本地 LM Studio(可选,默认关)**:`localLmStudio.enabled` 排在 Ollama 之后,模型名必须填写 LM Studio Developer 页或 `/v1/models` 返回的真实标识;
270
275
  4. **高级自定义 HTTP 视觉端点**:旧配置/高级配置中的 `httpProviders` 排在本地后端之后;
@@ -279,7 +284,7 @@ vision_long_screenshot_ocr image="chat-log.png" chunkHeight=1200 overlap=120
279
284
 
280
285
  ## 隐身模式
281
286
 
282
- 隐身模式默认**关闭**(issue #34 起显式 opt-in):关闭时官方 `deepseek-official` 路由原样保留,发图走选择器里可见的「DeepSeek + 自动识图」包装入口。
287
+ 隐身模式默认**关闭**(issue #34 起显式 opt-in):关闭时官方 `deepseek-official` 路由原样保留;需要看图时通过输入框旁的「👁 识图」切换到内部 DeepSeek wrapper。该 wrapper 默认从原生模型选择器和 `/model` 展示层隐藏。
283
288
 
284
289
  开启隐身模式后,插件接管官方 `deepseek-official` 路由:模型选择器看起来和原版完全一样(同一个 DeepSeek 组、同样的模型名),但每个条目背后都是声明了图片输入的自动识图包装;文字轮交给插件重建的原生 DeepSeek 适配器(读取同一个 `llm-deepseek` 设置段与凭据)。老会话通过隐藏的 `deepseek-vision` 别名继续工作。接管的前提是官方行不在场——在你的 profile 补丁层(`~/.dsh/profiles/<profile>/cordis.patch.yml`)禁用即可:
285
290
 
@@ -289,52 +294,51 @@ vision_long_screenshot_ocr image="chat-log.png" chunkHeight=1200 overlap=120
289
294
  disabled: true
290
295
  ```
291
296
 
292
- 官方行在场时,插件自动回退为可见包装入口。反过来,隐身模式关闭但官方行仍被禁用时,插件会做 keep-alive 兜底接管,保住 DeepSeek 模型(设置卡片会给出提示);想完全恢复官方原生行,把上面的 `disabled` 改回 `false` 再重启即可。
297
+ 官方行在场时,插件保留官方路由并使用内部 wrapper +「👁 识图」入口。反过来,隐身模式关闭但官方行仍被禁用时,插件会做 keep-alive 兜底接管,保住 DeepSeek 模型(设置页会给出提示);想完全恢复官方原生行,把上面的 `disabled` 改回 `false` 再重启即可。
293
298
 
294
- > 隐身模式**只作用于官方 DeepSeek 路由**。opencode 等自定义/第三方文本路由与隐身模式无关——默认会被自动包装成「+ 自动识图」模型组。
299
+ > 隐身模式**只作用于官方 DeepSeek 路由**。opencode 等自定义/第三方文本路由与隐身模式无关——默认也会生成内部识图 wrapper,由「👁 识图」按需使用。
295
300
 
296
- ## 自动识图模型组与手动包装
301
+ ## 自动识图包装与手动范围
297
302
 
298
- 默认开启 `autoWrapProviders`:插件会自动发现 **设置 → 模型** 中当前已启用的 provider / model,并额外注册同名的「+ 自动识图」模型组。**原模型组完全不变**;发图片时选自动识图组,纯文字仍可继续用原组。DSH 的 `llm/adapters-updated` 变化会触发同步,所以新增/删除模型后无需重启。
303
+ 默认开启 `autoWrapProviders`:插件会自动发现 **设置 → 模型** 中当前已启用的 provider / model,并为它们注册内部识图 wrapper。**原模型组完全不变**;普通用户不需要在模型选择器里寻找或手工选择这些 wrapper,它们会在能确认归属时默认隐藏,由聊天输入框旁的「👁 识图」负责切换。DSH 的 `llm/adapters-updated` 变化会触发同步,所以新增/删除模型后无需重启。
299
304
 
300
305
  `wrappedProviders` 是**可选的手动范围控制**,不是普通用户必须配置的步骤。只有两种情况需要它:
301
306
 
302
- 1. 关闭了自动包装,想手动指定哪些 provider / model 获得自动识图入口;
303
- 2. 自动包装保持开启,但只想让某个 provider 的部分模型出现在「+ 自动识图」组。
307
+ 1. 关闭了自动包装,想手动指定哪些 provider / model 可以使用「👁 识图」;
308
+ 2. 自动包装保持开启,但只想让某个 provider 的部分模型生成内部识图 wrapper。
304
309
 
305
- 设置卡片里用两个下拉(provider + 模型)配置;模型留空 = 包装该路由的全部模型,同一 provider 要限定多个模型就添加多行。修改即时生效,无需重启。
310
+ 设置页里用两个下拉(provider + 模型)配置;模型留空 = 包装该路由的全部模型,同一 provider 要限定多个模型就添加多行。修改即时生效,无需重启。若客户端无法确认 wrapper 归属或镜像关系不完整,展示层会 fail-open,不会为了“干净”而误隐藏第三方 route。
306
311
 
307
312
  ## Web 设置
308
313
 
309
- Web 配置页在 **设置 → 插件 → 插件配置** 下注册「视觉路由(自动识图)」卡片,顶部会直接提示最重要的使用步骤:**回到聊天页 → 右下角模型选择器 → 选择「+ 自动识图」模型组 → 发图**。其余设置主要用于高级定制:
314
+ Web profile 现在提供一级 **设置 → Vision Router** 页面。常规页把识图模型与 v2 路由授权放在一起;「识图策略 / 本地与设备 / 高级 / 诊断」分别承载工具行为、本地后端、敏感/性能设置和排障。
310
315
 
311
- - **自动创建「+ 自动识图」模型组**:默认开启,自动发现已有模型;模型目录变化热更新,无需重启;
312
- - **手动限定自动识图范围(可选)**:仅在需要关闭自动包装或限制部分模型时使用;
313
- - **视觉后端链**:给 `vision_describe` 等视觉工具调用的真正图片模型,默认内置免费 Qwen 即可;不要填纯文本模型;
314
- - 开关:整轮自动路由(旧模式)、识图工具、图片块改写、隐身模式(仅官方 DeepSeek 路由);
315
- - 视觉请求超时、包装/链路由名、代理等高级参数;
316
- - 每个字段都有「已覆盖」徽标与一键恢复组合默认,以及放弃/保存;
317
- - 「测试连接」按钮优先探测已启用的本地后端,并校验所填模型是否出现在 `/v1/models`;否则探测第一个可用视觉提供方;
318
- - 产出制品的工具在对话里渲染专用调用卡(关键字段 + 打开文件按钮)。
316
+ - **识图模型链**:`vision_describe` 等视觉工具真正调用的图片模型,内置免费链固定作为最终兜底;
317
+ - **模型选择方式**:继续按配置顺序,或显式开启能力感知 Auto,并选择「综合 / 质量 / 速度 / 本地」偏好;
318
+ - **后台补充能力数据**:`关闭 / 仅本地与免费 / 所有模型`,独立授权,不会因开启 Auto 自动开启;
319
+ - **测试识图 / 测评**:一次精确图片验证,以及 Quick(约3次,OCR+通用)/ Full(约6次,结构化+OCR+文档+定位+通用)能力测评;关闭设置页后任务仍继续;
320
+ - **本地与设备**:Ollama / LM Studio 与隐私敏感的桌面截屏开关;
321
+ - **高级 / 诊断**:超时、wrapper范围、代理/网络、兼容、版本、运行状态与排障。
319
322
 
320
323
  <p align="center">
321
- <img src="assets/vision-settings.png" width="72%" alt="设置 → 插件 → 插件配置 里的视觉路由卡片。" />
324
+ <img src="assets/vision-settings.png" width="72%" alt="Vision Router 一级设置页面。" />
322
325
  </p>
323
326
 
324
- > PR [#8](https://github.com/ysr666/dsh-vision-router/pull/8) 会把面板升级为目录驱动的模型下拉框、可增删的备用模型行与代理设置。
325
-
326
327
  ## 配置项
327
328
 
328
- 全部可选,默认即可用。通过 Web 卡片或 profile 补丁修改:
329
+ 全部可选,默认即可用。优先使用 **设置 → Vision Router**;高级部署仍可通过 profile 补丁覆盖:
329
330
 
330
331
  | 字段 | 默认值 | 含义 |
331
332
  |---|---|---|
333
+ | `routingMode` | `ordered` | `ordered` 按配置模型链执行;`auto` 把优先级委托给实测能力证据。升级不会自动开启 Auto |
334
+ | `routingPreference` | `balanced` | Auto 偏好:`balanced` / `quality` / `speed` / `local`;只在已授权候选之间改变顺序 |
335
+ | `backgroundBenchmarking` | `off` | 后台能力测评授权:`off` / `local-free` / `all`;开启 Auto 不会改变它,已授权后台任务只在 Auto 激活时运行 |
332
336
  | `provider` / `model` | `vision-http` / `ovh/Qwen2.5-VL-72B-Instruct` | 简写视觉后端链路(有适配器且真正支持图片输入的供应商 + 模型) |
333
337
  | `fallbacks` | `[]` | 简写视觉供应商的备用图片模型 |
334
- | `providers` | 内置免费 `vision-http` 条目 | 多供应商视觉后端链 `{ provider, model, fallbacks[] }`,按序尝试;优先于简写形式。不要填写纯文本模型 |
338
+ | `providers` | 内置免费 `vision-http` 条目 | 多供应商视觉后端链 `{ provider, model, fallbacks[] }`,按序尝试;不要填写纯文本模型 |
335
339
  | `httpProviders` | 内置 OVH 条目 | OpenAI 兼容直连端点 `{ name, baseURL, model, apiKeyEnv, maxTokens }` |
336
- | `autoWrapProviders` | `true` | 自动发现当前已启用 provider / model,并热更新同名「+ 自动识图」模型组;原模型组不变 |
337
- | `wrappedProviders` | `[{ provider: 'deepseek-official', models: [] }]` | 可选的手动包装范围 `{ provider, models[] }`;用于关闭自动包装后手动指定,或限制某个 provider 只包装部分模型。改动即时生效,无需重启 |
340
+ | `autoWrapProviders` | `true` | 自动发现当前已启用 provider / model,并热更新对应内部识图 wrapper;能确认归属时从原生模型选择器隐藏,原模型组不变 |
341
+ | `wrappedProviders` | `[{ provider: 'deepseek-official', models: [] }]` | 可选手动包装范围 `{ provider, models[] }`;用于关闭自动包装后手动指定,或限制某个 provider 只有部分模型可通过「👁 识图」进入 wrapper |
338
342
  | `routing` | `false` | 旧版整轮链路由(一次性整轮回答)。`false` = 工具优先流程(推荐) |
339
343
  | `reverseRouting` | `true` | 开启 `routing` 时,文字轮路由回 `textProvider` |
340
344
  | `wrapperRoute` / `chainRoute` | `deepseek-vision` / `vision-chain` | 准入包装路由名 / 降级链路由名(置空关闭) |
@@ -344,24 +348,23 @@ Web 配置页在 **设置 → 插件 → 插件配置** 下注册「视觉路由
344
348
  | `rewriteImages` | `true` | 模型输入层改写图片块(缓存描述或工具提示标记);界面日志保留图片 |
345
349
  | `desktopScreenshot` | `false` | 模型可调用的 `vision_screenshot` 桌面截屏隐私开关;每次截屏前实时检查 |
346
350
  | `freeFallback` | `true` | 在显式本地/自定义 HTTP 后端之后追加匿名 OVH 模型;关闭它不会停用用户明确配置的本地后端 |
347
- | `localOllama` | `{ enabled: false, baseURL: 'http://127.0.0.1:11434/v1', model: 'qwen2.5vl', format: 'openai' }` | **本地视觉后端(并入自 dsh-vision)**:开启后 local-ollama 排在 HTTP 视觉链最前;Ollama 未运行会自动跳过;`format` 可选 `openai`(`/chat/completions`)或 `anthropic`(`/messages`);可选的 `temperature` / `top_p` 只在显式填写时发送。v1.7 会预热本机 loopback 模型并续期 30 分钟驻留,冷加载时间不再计入正常识图截止时间 |
348
- | `localLmStudio` | `{ enabled: false, baseURL: 'http://localhost:1234/v1', model: '', format: 'openai' }` | **本地 LM Studio 后端(并入自 dsh-vision)**:排在 Ollama 之后、自定义/云 HTTP 后端之前;开启时必须填写 LM Studio Developer 页或 `/v1/models` 返回的真实模型标识;可选采样参数同 Ollama,`format: 'anthropic'` 需 LM Studio 0.4.1+ |
349
- | `instantDescribe` | `false` | **即时本地翻译(并入自 dsh-vision)**:开启且至少一个本地后端可用时,在第一模型步之前识别无缓存图片块;Ollama → LM Studio 共用总超时预算,多图并发上限 3,失败则回退静态工具标记 |
350
- | `localDescribeStyle` | `plain` | **本地识别输出风格(并入自 dsh-vision)**:`plain` = 平铺描述;`structured` = 结构化识别(【初步判断】/【细节】/【空间结构】/【原图尺寸】),截图分析质量更高 |
351
+ | `localOllama` | `{ enabled: false, baseURL: 'http://127.0.0.1:11434/v1', model: 'qwen2.5vl', format: 'openai' }` | 本地视觉后端;开启后排在 HTTP 视觉链前部,服务未运行会自动跳过,支持 OpenAI / Anthropic 协议 |
352
+ | `localLmStudio` | `{ enabled: false, baseURL: 'http://localhost:1234/v1', model: '', format: 'openai' }` | Ollama 之后的本地 LM Studio 后端;填写 Developer 页或 `/v1/models` 返回的真实模型 ID |
353
+ | `visionTurnBudgetMs` | `0` | 整轮视觉总墙钟预算;`0` = 不设整轮上限。具体 provider调用/工具仍有自己的硬超时 |
351
354
  | `downscale` / `downscaleMaxPixels` | `true` / `4000000` | 调用前压缩及其像素预算(延迟保护) |
352
355
  | `cache` / `cacheTtlSeconds` / `cacheMaxEntries` | `true` / `3600` / `200` | 视觉答案缓存 |
353
356
  | `timeoutMs` | `120000` | 单次视觉调用超时 |
354
357
  | `artifactsDir` | `.dsh-vision-router/artifacts` | 产物目录(相对会话工作区) |
355
358
  | `proxy` / `proxyHosts` | `''` / openrouter 域名 | 仅视觉供应商域名可选的本地代理 |
356
- | `catalogCorrections` | `true` | 内置目录纠错:当已安装的 pi-ai 目录把已知模型路由到错误协议时(例如 `opencode-go/qwen3.6-plus` 被指向 OpenAI chat completions,而 OpenCode Go 只在 `/v1/messages` 上提供该模型),插件直接按正确协议应答该后端;上游目录修复后每条纠错自动失效 |
359
+ | `catalogCorrections` | `true` | 内置目录纠错:当已安装目录把已知模型路由到错误协议时按正确协议应答;上游修复后对应纠错自动失效 |
357
360
 
358
361
  ### 本地 Ollama 视觉后端(并入自 dsh-vision)
359
362
 
360
363
  > **增量开发作者**:[shaoqiuyuavailable](https://github.com/shaoqiuyuavailable)(router 本地视觉增量)
361
364
  >
362
- > **思路来源**:本地视觉后端(Ollama / LM Studio 双后端、即时识别、结构化输出、截屏识别、同图去重记忆、失败降级占位、并发防雪崩、超时防护)的思路继承自 [dsh-vision](https://github.com/shaoqiuyuavailable/text-llm-vision/tree/dsh-vision)——本项目将其并入 HTTP 视觉链,并在此基础上扩展了逐级降级链与双协议支持。
365
+ > **思路来源**:Ollama / LM Studio 双本地后端、结构化识别、截屏识别、同图记忆、失败降级、并发保护与超时防护等设计继承自 [dsh-vision](https://github.com/shaoqiuyuavailable/text-llm-vision/tree/dsh-vision);本项目将其并入 HTTP 视觉链,并扩展逐级 fallback 与双协议支持。
363
366
 
364
- 可选的本地优先视觉路径:不需要 Key,支持隐私、零费用、离线识别。它作为 HTTP 视觉链里的 `local-ollama` 接入;若本地识别失败,除非用户明确配置纯本地链,否则仍可能继续尝试已配置的云后端。
367
+ 可选的本地优先视觉路径:不需要 Key,适合隐私、零费用、离线识别。它作为 HTTP 视觉链里的 `local-ollama` 接入;若本地识别失败,除非用户明确配置纯本地链,否则仍可能继续尝试已配置的云后端。
365
368
 
366
369
  **1. 安装 Ollama 并拉取视觉模型**
367
370
 
@@ -370,31 +373,27 @@ Web 配置页在 **设置 → 插件 → 插件配置** 下注册「视觉路由
370
373
  ollama pull qwen2.5vl
371
374
  ```
372
375
 
373
- **2. 开启** —— 设置卡片「本地视觉」组,或 profile patch:
376
+ **2. 开启** —— **设置 → Vision Router → 本地与设备**,或 profile patch:
374
377
 
375
378
  ```yaml
376
379
  - id: vision-router
377
380
  config:
378
381
  localOllama:
379
382
  enabled: true
380
- baseURL: 'http://127.0.0.1:11434/v1' # OpenAI 兼容端点
383
+ baseURL: 'http://127.0.0.1:11434/v1'
381
384
  model: 'qwen2.5vl'
382
- temperature: 0.5 # 可选;识别用低温更稳
383
- top_p: 0.8 # 可选;留空 = 服务端默认
384
- instantDescribe: true # 图片轮第一轮即本地识别
385
- localDescribeStyle: 'structured' # 'plain' | 'structured'
385
+ temperature: 0.5
386
+ top_p: 0.8
386
387
  ```
387
388
 
388
389
  **3. 行为说明**
389
390
 
390
- - 开启后 `local-ollama` 排在 HTTP 视觉链最前。若要严格纯本地,请移除云视觉行/自定义 HTTP 端点,并关闭 `freeFallback`。
391
- - **v1.7 冷启动处理:**选中的本机 loopback Ollama 模型会通过原生 API 预热并保持 30 分钟驻留。如果模型在 Ollama 作为首个图片后端时已经冷却,加载会在正常视觉任务预算开始之前完成;短 `/api/ps` 探测保证服务未运行/挂死时仍快速进入 fallback。远程 Ollama URL 不会自动预热。
392
- - **LM Studio 同理**——同一「本地视觉」组里开启 `localLmStudio`,填 OpenAI 兼容端点(默认 `http://localhost:1234/v1`),并使用 Developer 页或 `/v1/models` 返回的真实模型标识。它排在 `local-ollama` 之后、自定义/云 HTTP 后端之前。
391
+ - 开启后 `local-ollama` 排在 HTTP 视觉链前部。若要严格纯本地,请移除云视觉行/自定义 HTTP 端点,并关闭 `freeFallback`。
392
+ - 选中的本机 loopback Ollama 模型会通过原生 API 预热并保持 30 分钟驻留。如果模型在 Ollama 作为首个图片后端时已经冷却,加载会在正常视觉任务预算开始之前完成;短 `/api/ps` 探测保证服务未运行/挂死时仍快速进入 fallback。远程 Ollama URL 不会自动预热。
393
+ - **LM Studio 同理**——开启 `localLmStudio`,填 OpenAI 兼容端点(默认 `http://localhost:1234/v1`),并使用 Developer 页或 `/v1/models` 返回的真实模型标识。它排在 `local-ollama` 之后、自定义/云 HTTP 后端之前。
393
394
  - 每个本地后端可通过 `format` 选择 **OpenAI 或 Anthropic 格式**(默认 `openai`)。Anthropic 模式走 `/v1/messages`,带 `anthropic-version` 并把图片转为 base64 source;只有配置了 Key 才发送 `x-api-key`。LM Studio 需 0.4.1 或更高版本才提供该端点。
394
- - 任一本地后端未运行或调用超时时自动跳过,继续降级到云链——任何调用都不受影响。
395
- - `instantDescribe` 会在第一模型步之前按 Ollama → LM Studio 的顺序尝试已启用本地后端。多张无缓存图片并发识别(上限 3),单张失败不影响其余;命中附件记忆的图片不会再次请求本地服务。
395
+ - 任一本地后端未运行或调用超时时自动跳过,继续降级到云链。
396
396
  - `vision_screenshot` 默认关闭。单独开启「桌面截屏」隐私开关后,`identify=true` 使用同样的 Ollama → LM Studio 降级顺序。
397
- - 日志中的 `image turn — instantDescribe=… localBackends=…` 显示实时决策;`instant local describe recognized N/M uncached image(s), C cached, F failed attempts` 显示本轮结果。
398
397
 
399
398
  ## 环境要求
400
399
 
@@ -472,7 +471,7 @@ npx @deepseek-ai/dsh plugin --profile web add dsh-vision-router@<版本号>
472
471
  pnpm dsh plugin --profile web add dsh-vision-router@<版本号>
473
472
  ```
474
473
 
475
- 设置存放在 profile 的设置提供方里,升级不丢失。设置卡里的一键更新会自动显式安装 registry 已确认的版本,并在命令结束后核对实际安装版本——绝不只凭包管理器退出码就报成功。
474
+ 设置存放在 profile 的设置提供方里,升级不丢失。设置页的一键更新会自动显式安装 registry 已确认的版本,并在命令结束后核对实际安装版本——绝不只凭包管理器退出码就报成功。
476
475
 
477
476
  > **新版本一直不生效(`downloaded 0` / `added 0`):** pnpm v11 会拦下发布不足 24 小时的版本;按上面方式显式安装目标版本(pnpm 会自动写入豁免),或运行 `npx dsh-vision-router repair` 修复过期的带版本号豁免条目后,更新立即生效。
478
477
 
@@ -574,4 +573,4 @@ npx dsh-vision-router doctor --profile web
574
573
  <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/ysr666/dsh-vision-router/star-history/assets/star-history/star-history-dark.svg">
575
574
  <img alt="Star 历史趋势图" src="https://raw.githubusercontent.com/ysr666/dsh-vision-router/star-history/assets/star-history/star-history-light.svg" width="100%">
576
575
  </picture>
577
- </p>
576
+ </p>