@anionex/dsh-vision-toolkit 0.1.9 → 0.1.11

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.i18n.yaml CHANGED
@@ -2,5 +2,5 @@
2
2
  # side as of the last confirmed-consistent state. Both languages carry equal authority;
3
3
  # after editing either side, bring the other along and re-record with:
4
4
  # pnpm run verify-translation-pairing --write dsh-vision-dark-theme/README.md
5
- README.md: 026285db53ab41b5e70527060c692bf9f2785541
6
- README.zh.md: df34e29906568cd13f235dad56abe652a659a620
5
+ README.md: 0c81049d6e43d5c09056e7733349b45ea29fc22e
6
+ README.zh.md: 223b1d7f98b26eac72ab1ee0a76a83aed8b81673
package/README.md CHANGED
@@ -1,38 +1,66 @@
1
- ![DSH Vision Toolkit — native visual engineering for text-only DeepSeek Harness agents](assets/hero.png)
1
+ <p align="center">
2
+ <img src="assets/hero.png" alt="DSH Vision Toolkit — native visual engineering for text-only DeepSeek Harness agents" />
3
+ </p>
4
+
5
+ <h1 align="center">DSH Vision Toolkit</h1>
6
+
7
+ <p align="center">
8
+ English | <a href="https://github.com/Anionex/dsh-vision-toolkit/blob/main/README.zh.md">中文</a>
9
+ </p>
10
+
11
+ <p align="center">
12
+ <a href="https://dshfind.com/en/plugins/Anionex/dsh-vision-toolkit"><img src="https://img.shields.io/badge/recommended%20by-dshfind-FFD700?style=flat-square" alt="Recommended by dshfind" /></a>
13
+ <a href="https://dshfind.com/en/plugins/Anionex/dsh-vision-toolkit"><img src="https://img.shields.io/badge/dshfind%20score-94%20%7C%20highest--rated%20plugin-5B4CF0?style=flat-square" alt="dshfind score: 94 — highest-rated plugin" /></a>
14
+ <a href="https://x.com/anion_ex"><img src="https://img.shields.io/badge/-@anion__ex-000000?style=flat-square&amp;logo=x&amp;logoColor=white" alt="X: @anion_ex" /></a>
15
+ <a href="https://github.com/Anionex/dsh-vision-toolkit/releases/tag/v0.1.11"><img src="https://img.shields.io/badge/release-v0.1.11-5B4CF0?style=flat-square" alt="Release v0.1.11" /></a>
16
+ <a href="tests"><img src="https://img.shields.io/badge/verified-233%20tests-2EA44F?style=flat-square" alt="Verified: 233 tests" /></a>
17
+ </p>
18
+
19
+ <p align="center">
20
+ <a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-0B7285?style=flat-square" alt="License: MIT" /></a>
21
+ <a href="package.json"><img src="https://img.shields.io/badge/Node.js-%5E22.19%20%7C%20%3E%3D24-339933?style=flat-square&amp;logo=nodedotjs&amp;logoColor=white" alt="Node.js ^22.19 or >=24" /></a>
22
+ <a href="runtime/requirements.lock"><img src="https://img.shields.io/badge/Python-3.11%2B-3776AB?style=flat-square&amp;logo=python&amp;logoColor=white" alt="Python 3.11+" /></a>
23
+ <a href="cordis.patch.yml"><img src="https://img.shields.io/badge/DSH-Web%20%2B%20Headless-5B4CF0?style=flat-square" alt="DSH Web and Headless profiles" /></a>
24
+ </p>
2
25
 
3
- # DSH Vision Toolkit
26
+ ## Give your DSH agent eyes
4
27
 
5
- [![Recommended by dshfind](https://img.shields.io/badge/recommended%20by-dshfind-FFD700?style=flat-square)](https://dshfind.com/en/plugins/Anionex/dsh-vision-toolkit)
6
- [![dshfind score: 94 — highest-rated plugin](https://img.shields.io/badge/dshfind%20score-94%20%7C%20highest--rated%20plugin-5B4CF0?style=flat-square)](https://dshfind.com/en/plugins/Anionex/dsh-vision-toolkit)
7
- [![X (Twitter)](https://img.shields.io/badge/-@anion__ex-000000?style=flat-square&logo=x&logoColor=white)](https://x.com/anion_ex)
8
- [![Release v0.1.9](https://img.shields.io/badge/release-v0.1.9-5B4CF0?style=flat-square)](https://github.com/Anionex/dsh-vision-toolkit/releases/tag/v0.1.9)
9
- [![Verified: 230 tests](https://img.shields.io/badge/verified-230%20tests-2EA44F?style=flat-square)](tests)
10
- [![License: MIT](https://img.shields.io/badge/license-MIT-0B7285?style=flat-square)](LICENSE)
11
- [![Node.js](https://img.shields.io/badge/Node.js-%5E22.19%20%7C%20%3E%3D24-339933?style=flat-square&logo=nodedotjs&logoColor=white)](package.json)
12
- [![Python](https://img.shields.io/badge/Python-3.11%2B-3776AB?style=flat-square&logo=python&logoColor=white)](runtime/requirements.lock)
13
- [![DSH profiles](https://img.shields.io/badge/DSH-Web%20%2B%20Headless-5B4CF0?style=flat-square)](cordis.patch.yml)
28
+ Drop in a screenshot and let a text-only DeepSeek Harness agent inspect it, read it, locate elements, extract assets, rebuild interfaces, and measure whether the result matches.
14
29
 
15
- **Install:** `dsh plugin --profile web add @anionex/dsh-vision-toolkit`
30
+ DSH Vision Toolkit packages [`agent-vision-toolkit`](https://github.com/Anionex/agent-vision-toolkit) as a native DSH plugin. You get focused image Q&A, OCR, original-pixel coordinates, UI restoration, pixel comparison, downloadable results, and a Web Settings panel without assembling scripts by hand.
16
31
 
17
- **DSH Vision Toolkit brings [`agent-vision-toolkit`](https://github.com/Anionex/agent-vision-toolkit) into DeepSeek Harness as a native Profile Bundle.**
32
+ ```sh
33
+ dsh plugin --profile web add @anionex/dsh-vision-toolkit
34
+ ```
18
35
 
19
- Give text-only DSH agents eyes—and keep vision in the harness—with intent-aware image Q&A, OCR, original-pixel grounding, UI restoration, pixel verification, managed Artifacts, and Web Settings. Ten independent tools replace shell glue with structured schemas and Agent-scoped progressive exposure.
36
+ The npm package includes the visual toolkit snapshot and uses a managed runtime by default. **Normal installation does not require a source checkout or an `agentVisionToolkitPath`.**
20
37
 
21
38
  **Upstream toolkit:** [Anionex/agent-vision-toolkit](https://github.com/Anionex/agent-vision-toolkit) · **Project website:** [agent-vision.anionex.me](https://agent-vision.anionex.me)
22
39
 
23
- English | [中文](README.zh.md)
40
+ ## What you can do
24
41
 
25
- ## Why this exists
42
+ | Goal | What the agent can deliver |
43
+ |---|---|
44
+ | Understand a screenshot | Focused answers, visual descriptions, multi-image comparison, and OCR |
45
+ | Find an interface element | Original-image pixel coordinates with an optional labeled preview |
46
+ | Rebuild a page from a reference | Screenshot rendering, region-by-region diagnosis, and measurable iteration |
47
+ | Extract a usable asset | Cropped images, transparent foregrounds, dominant colors, or editable SVG |
48
+ | Read a long screenshot | Auditable chunks, Markdown output, manifests, and resumable OCR runs |
49
+ | Verify a visual result | A difference percentage, ranked mismatch regions, heatmap, and JSON report |
50
+
51
+ You can use remote vision only where it adds value. Cropping, tracing, pixel comparison, color analysis, foreground extraction, and HTML screenshots run locally.
26
52
 
27
- `agent-vision-toolkit` treats vision as an Agent-callable capability rather than a property of the base model. Its method carries the reason for looking into the visual request, moves from the whole image to targeted regions, and verifies coordinates, colors, geometry, and differences with focused tools instead of accepting a generic description as evidence.
53
+ ## See it in action
28
54
 
29
- DSH Vision Toolkit preserves that method while replacing CLI installation and Bash argument construction with native schemas, DSH Credentials, lifecycle-managed runtime preparation, structured Session-log results, previewable Artifacts, dedicated Web cards, and Settings. The Agent loads one versioned Skill and receives the ten visual schemas only when the current task needs them.
55
+ The first example is a live DSH Web view. The next two examples come from the same `agent-vision-toolkit` lineage packaged with this plugin, and the last shows the workflow inside a live DeepSeek Harness Web session. See the [asset provenance record](assets/upstream/README.md) for source details.
30
56
 
31
- The package delivers the committed P0 and P1 product scope. P2's stable `ctx.visionToolkit` service remains deliberately unpublished until an independent plugin becomes a real consumer; the internal runtime does not pretend that an unvalidated ecosystem API is stable.
57
+ ### DSH view example
32
58
 
33
- ## Proven use cases from agent-vision-toolkit
59
+ <p align="center">
60
+ <img src="assets/dsh-view-example.png" width="80%" alt="DSH Web session view in which a text-only DeepSeek-V4-Flash (Vision Toolkit) model answers a question about a pasted banner image." />
61
+ </p>
34
62
 
35
- The first two panels are official upstream reference runs from the same pinned `agent-vision-toolkit` lineage packaged by this bundle. The image Q&A and screenshot-guided debugging panel is a live DeepSeek Harness Web session, showing the same workflows through DSH. See the [asset provenance record](assets/upstream/README.md) for the upstream source images.
63
+ *A live DSH Web view: the user pastes a brand-banner screenshot, and the text-only model answers what the image contains through the `DeepSeek-V4-Flash (Vision Toolkit)` image-input variant.*
36
64
 
37
65
  ### Infographic restoration: screenshot to editable HTML/CSS
38
66
 
@@ -61,47 +89,60 @@ The first two panels are official upstream reference runs from the same pinned `
61
89
 
62
90
  *Left: intent-aware image Q&A in DSH Web. Right: a DSH Web screenshot-debugging turn that lists the concrete UI differences and continues toward `vision_pixel_diff`. The upstream workflow source is the same [`agent-vision-toolkit` reference](https://github.com/Anionex/agent-vision-toolkit/blob/c27d1a300962b553c0884993c575cd3e819465ce/README.md#real-world-effects).*
63
91
 
64
- DSH Vision Toolkit adds native tool schemas, versioned lifecycle, Credentials, structured Session results, Artifacts, Web presentation, Settings, and progressive exposure around these upstream capabilities. The next section is the reproducible proof executed and checked into this DSH repository.
92
+ DSH Vision Toolkit brings this workflow into DSH, where the result can become a file, a coordinate, a measured comparison, or the next step in the same session.
65
93
 
66
- ## DSH-native proof: reference-to-pixel verification
94
+ ## From a rough match to pixel-perfect
67
95
 
68
- The checked-in UI-restoration workflow renders an intentionally inaccurate HTML implementation, measures a `6.04%` pixel difference across six non-zero regions, iterates, and reaches an exact `0%` difference against the reference at `1200 × 720`.
96
+ The included UI-restoration workflow starts with an intentionally inaccurate HTML implementation. Vision Toolkit measures a `6.04%` difference, points to the worst regions, and helps drive the next iteration. The final render reaches an exact `0%` difference at `1200 × 720`.
69
97
 
70
98
  <p>
71
99
  <img src="examples/ui-restoration/assets/initial.png" width="49%" alt="Initial UI restoration candidate before Vision Toolkit iteration, with measurable layout and styling differences from the reference." />
72
100
  <img src="examples/ui-restoration/assets/implementation.png" width="49%" alt="Final UI restoration output reproduced by the checked-in workflow with zero pixel difference from the reference." />
73
101
  </p>
74
102
 
75
- | Verified surface | Evidence |
103
+ | Start | Result |
76
104
  |---|---|
77
- | Product scope | 10 independent visual tools, matching `vision-tools` Skill, Artifacts, dedicated Web cards, and live Settings |
78
- | Automated coverage | 17 Vitest files / 136 passing tests, plus a dependency-free portable package check |
79
- | Real profiles | Clean temporary Web and Headless installation, activation, disable, re-enable, and uninstall |
80
- | Visual acceptance | Reproducible HTML screenshot → pixel diff example with a final `0%` difference |
105
+ | Reference image | A working HTML implementation you can open and edit |
106
+ | First comparison | `6.04%` difference across the visible problem regions |
107
+ | Final comparison | `0%` difference at `1200 × 720` |
81
108
 
82
- ## Highlights
109
+ ## Why it feels different
83
110
 
84
- - **See images without bloating every prompt:** only `vision_toolkit_activate` is initially visible; loading `vision-tools` mounts ten independent schemas for that Agent and keeps version/health administration out of model context.
85
- - **Act on coordinates instead of parsing prose:** grounding and detection return original-image pixel boxes, while every model-visible result remains structured text or JSON.
86
- - **Deliver files, not temporary output:** crop, trace, OCR, pixel diff, foreground extraction, and HTML rendering produce described Artifacts that the Web client can preview, download, or open locally.
87
- - **Keep runtime and credentials controlled:** DSH Credentials hold API keys, managed mode prepares an exact isolated Python environment, and a failed Settings candidate cannot replace the serving generation.
88
- - **Close the visual verification loop:** local HTML rendering and pixel-diff ranking support reference → implementation → screenshot → measured iteration without a model-native image channel.
89
- - **Use the same bundle in Web and Headless profiles:** Web adds cards, previews, Settings, and health actions; Headless receives the same tool semantics and complete structured results.
111
+ - **Ask for the thing you need.** “Where is the submit button?” and “Why does this screenshot differ from the reference?” lead to focused visual work instead of a generic caption.
112
+ - **Get evidence you can use.** The agent returns coordinates, OCR, measurements, JSON, and files you can open or pass to the next step.
113
+ - **Keep the workflow in DSH.** Credentials, Settings, Artifacts, Web cards, and Headless results live alongside the rest of your session.
114
+ - **Use local tools when you can.** Crop, trace, pixel comparison, color analysis, foreground extraction, and HTML screenshots do not consume a vision API request.
115
+ - **Repeat the loop.** Reference image → implementation → screenshot → pixel diff gives UI work a measurable finish line.
90
116
 
91
- ## Quick start
117
+ ## Start in three steps
92
118
 
93
- Prerequisites: DeepSeek Harness `0.1.0-rc.6` or a compatible later `0.1.x` release, Python 3.11+, and `pnpm` available to `dsh plugin`. Install the published bundle from npm, add it to the profiles you use, and confirm the bundle row:
119
+ Use DeepSeek Harness `0.1.0-rc.6` or a compatible later `0.1.x` release. The plugin prepares its managed runtime on first use.
94
120
 
95
121
  ```sh
96
122
  dsh plugin --profile web add @anionex/dsh-vision-toolkit
97
123
  dsh plugin --profile headless add @anionex/dsh-vision-toolkit
98
- dsh --profile web --dump-config | grep vision-toolkit
99
- dsh --profile headless --dump-config | grep vision-toolkit
100
124
  ```
101
125
 
102
- Legacy profiles must use `nodeLinker: hoisted` and `autoInstallPeers: false` in their `pnpm-workspace.yaml`. An updated DSH launcher repairs these owned settings before `dsh plugin` runs; when using an older launcher, set them before installation so pnpm does not assemble a second Harness dependency graph inside the profile.
126
+ 1. Restart your Web profile and open **Settings → Vision Toolkit**.
127
+ 2. New installations use the built-in free Gemma 4 provider, so you can run **Test API connection** and **Test vision model** without an API key. To use another provider, edit the endpoint/model/protocol and provide its DSH Credential.
128
+ 3. In a conversation, paste an image or put it in the workspace, invoke `/vision-tools`, and ask for a concrete visual task.
129
+
130
+ If you use an older DSH launcher, the profile may need `nodeLinker: hoisted` and `autoInstallPeers: false` before installation. Current launchers repair these settings for you.
131
+
132
+ Local crop, trace, pixel, color, foreground, and HTML operations do not require a visual API credential.
133
+
134
+ ## Community Group
103
135
 
104
- Restart a running Web profile, open **Settings → Vision Toolkit**, select a DSH Credential for remote tools, run **Test API connection**, and then run **Test vision model** to verify one real image request. In a conversation, make an image available as a workspace path, invoke `/vision-tools`, and ask the Agent to call a specific `vision_*` tool. Local crop, trace, pixel, color, foreground, and HTML operations do not require a visual API credential.
136
+ Join the `agent-vision-toolkit` community group to exchange usage tips, share feedback, and suggest improvements.
137
+
138
+ <p align="center">
139
+ <img src="assets/community-group-qr.png" alt="QR code for the agent-vision-toolkit community group" width="260">
140
+ </p>
141
+
142
+ > **No local path is required.** Keep the default `runtime.mode: managed` for the normal npm installation. The optional `runtime.agentVisionToolkitPath` setting is only for developers or controlled deployments that deliberately use an external pinned checkout.
143
+
144
+ <details>
145
+ <summary><strong>Technical architecture</strong></summary>
105
146
 
106
147
  ## How it works
107
148
 
@@ -124,6 +165,8 @@ flowchart LR
124
165
 
125
166
  Tool definitions call one runtime; the runtime validates paths, limits, credentials, cancellation, and deadlines before dispatching to the pinned upstream snapshot or configured vision provider endpoint. Web presentation consumes the same structured results and Artifact descriptors, so it does not change Headless behavior. Health, connection testing, and version inspection stay in Settings rather than model tool schemas.
126
167
 
168
+ </details>
169
+
127
170
  ## Tools
128
171
 
129
172
  | Tool | Execution | Structured result | Artifact delivery |
@@ -141,6 +184,9 @@ Tool definitions call one runtime; the runtime validates paths, limits, credenti
141
184
 
142
185
  The plugin does not reimplement visual algorithms. Its DSH-owned layer validates paths and limits, resolves credentials, calls the pinned upstream scripts with argv vectors, parses their exact output contracts, classifies failures, describes files, and projects results to the model and Web client.
143
186
 
187
+ <details>
188
+ <summary><strong>Advanced model behavior</strong></summary>
189
+
144
190
  ## Progressive model exposure
145
191
 
146
192
  Runtime readiness is profile-wide, but the ten visual execution schemas are Agent-scoped. Before an Agent loads `vision-tools`, the plugin contributes only the small `vision_toolkit_activate` bootstrap; the visual tools are absent from that Agent's request schema. A successful call to the standard `skill` tool with `name="vision-tools"` mounts all ten tools automatically for the next model step and hides the bootstrap. A direct `/vision-tools` invocation injects the Skill instructions; if the visual tools are still absent, those instructions require one `vision_toolkit_activate` call. Activation affects only that Agent, restores when the Session contains durable evidence matching the bundled Skill version, and lasts until the Agent or plugin is disposed.
@@ -149,18 +195,20 @@ Health checks, connection testing, and plugin/upstream version inspection are ad
149
195
 
150
196
  ## Image-input variants for text-only models
151
197
 
152
- Text-only model routes get sibling model-selector entries named `<model> (Vision Toolkit)` under a matching provider group. A variant declares image input, so pasted images keep the native attachment flow — composer thumbnail, durable session image, and history rendering — and the plugin rewrites every image block into a Vision Toolkit description only on the wire to the model, before the request reaches the upstream route. The session log is untouched; replay and the UI keep the real image.
198
+ Text-only model routes get sibling model-selector entries named `<model> (Vision Toolkit)` under a matching provider group. DSH cannot pass a pasted attachment's local path through its native image block, so the default paste flow copies each image into the session workspace and inserts its absolute path into the model-visible message. The DSH model can then call `vision_glance` (or another visual tool) with that path, using the same focus-hinted bridge and `[vision model description]` channel markers as `agent-vision-toolkit`. The session log contains the durable path reference and the UI keeps the paste record.
199
+
200
+ A variant is still registered automatically for every model the host positively declares text-only (for example the DeepSeek chat family), but automatic switching is opt-in. With the default `autoSwitch: false`, a text-only session uses the paste-to-path flow so the DSH model receives a usable absolute path. If `autoSwitch: true` is explicitly enabled, the browser switches to `<model> (Vision Toolkit)` and the server-side image-input variant rewrites native image blocks into descriptions. The host's verdict uses the exact model route the browser read from the live model catalog, with the selector label as fallback; unconfirmed or image-capable routes keep the native flow.
153
201
 
154
- A variant is registered automatically for every model the host positively declares text-only (for example the DeepSeek chat family). Paste handling is automatic: when the current model is confirmed text-only and its variant exists, the browser integration switches the session to the variant by itself (a short notice names the new model) and the paste then keeps the native flow; no manual model change is needed. The host's verdict uses the exact model route the browser read from the live model catalog, with the selector label as fallback; unconfirmed or image-capable routes always keep the native flow, and a text-only model without a variant (for example when variants are disabled) keeps the paste-to-path takeover, which copies the image into the session workspace and inserts its path as text.
202
+ Description conversion needs the configured vision provider and its credential when the opt-in image-input variant is used; when the runtime is not ready or a read fails, the wire block degrades to the upstream-compatible `[vision unavailable: ...]` note instead of failing the turn. The bridge does not treat injected context files as the current user intent, and it uses the latest assistant paragraph when a tool-fetched image is being described. Disable variants with `imageInputVariants.enabled: false`, restrict the wrapped routes with `imageInputVariants.providers`, or opt into native attachment switching with `imageInputVariants.autoSwitch: true`.
155
203
 
156
- Description conversion needs the configured vision provider and its credential; when the runtime is not ready or a read fails, the wire block degrades to an explanatory note instead of failing the turn. Disable variants with `imageInputVariants.enabled: false`, restrict the wrapped routes with `imageInputVariants.providers`, or keep the paste-to-path behavior for text-only models with `imageInputVariants.autoSwitch: false`.
204
+ </details>
157
205
 
158
206
  ## Requirements
159
207
 
160
208
  - DeepSeek Harness with a Web or Headless profile and `pnpm` available to `dsh plugin`.
161
209
  - Python 3.11 or newer. Managed mode creates an isolated environment, so users do not install the upstream CLI or Python packages manually.
162
210
  - Network access on the first managed-runtime activation unless the exact packages in `runtime/requirements.lock` are already available in the configured package cache.
163
- - An OpenAI-compatible or Anthropic vision endpoint and DSH Credential for `vision_glance`, `vision_ground`, `vision_detect`, and non-split-only long-screenshot OCR. Local tools remain usable without that credential.
211
+ - The built-in free Gemma 4 provider is ready for `vision_glance`, `vision_ground`, `vision_detect`, and non-split-only long-screenshot OCR. A DSH Credential is required only when a custom OpenAI-compatible or Anthropic endpoint is configured. Local tools remain usable without either provider.
164
212
  - Chrome, Chromium, or Edge only for `vision_html_screenshot`; all other tools remain available when no supported browser is installed.
165
213
  - PNG, JPEG, GIF, or WebP inputs inside the session workspace or an explicitly configured `allowedDirs` root.
166
214
 
@@ -201,7 +249,7 @@ dsh plugin --profile web remove @dsh-external/dsh-vision-toolkit
201
249
  dsh plugin --profile web add @anionex/dsh-vision-toolkit
202
250
  ```
203
251
 
204
- After restarting, Settings → Vision should report plugin version **0.1.9**.
252
+ After restarting, Settings → Vision should report plugin version **0.1.11**. The built-in free provider is selected automatically; custom providers still use the configured DSH Credential.
205
253
 
206
254
  For a registry installation, update the dependency through the profile package manager:
207
255
 
@@ -229,16 +277,16 @@ The bundle defaults to the managed runtime. A profile patch can override the pro
229
277
  - id: vision-toolkit
230
278
  config:
231
279
  provider:
232
- baseUrl: https://api.inferera.com/v1
233
- credential: VISION_API_KEY
234
- model: gemini-3.6-flash
280
+ baseUrl: https://vision.anionex.me/v1
281
+ credential: ANIONEX_FREE_VISION
282
+ model: gemma-4-26b-a4b-it
235
283
  protocol: openai
236
284
  anthropicThinking: omit
237
285
  userAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36
238
286
  language: zh
239
287
  timeoutMs: 60000
240
- maxImageBytes: 10485760
241
- maxImagePixels: 40000000
288
+ maxImageBytes: 4194304
289
+ maxImagePixels: 20000000
242
290
  concurrency: 4
243
291
  runtime:
244
292
  mode: managed
@@ -246,23 +294,23 @@ The bundle defaults to the managed runtime. A profile patch can override the pro
246
294
  imageInputVariants:
247
295
  enabled: true
248
296
  providers: []
249
- autoSwitch: true
297
+ autoSwitch: false
250
298
  ```
251
299
 
252
300
  ### Configuration fields
253
301
 
254
302
  | Field | Default | Contract |
255
303
  |---|---|---|
256
- | `provider.baseUrl` | `https://api.inferera.com/v1` | Provider API base URL, normalized without trailing slashes; for Anthropic use a base ending in `/v1`, not the full `/messages` URL |
257
- | `provider.credential` | `VISION_API_KEY` | DSH Credential reference, never a secret value |
258
- | `provider.model` | `gemini-3.6-flash` | Multimodal model name sent to remote tools |
304
+ | `provider.baseUrl` | `https://vision.anionex.me/v1` | Built-in free OpenAI-compatible endpoint; custom providers may use another base URL, normalized without trailing slashes |
305
+ | `provider.credential` | `ANIONEX_FREE_VISION` | Read-only built-in reference for the free service; custom providers use a DSH Credential reference, never a secret value |
306
+ | `provider.model` | `gemma-4-26b-a4b-it` | Multimodal model name sent to remote tools |
259
307
  | `provider.protocol` | `openai` | `openai` sends Chat Completions requests; `anthropic` sends native Messages requests |
260
308
  | `provider.anthropicThinking` | `omit` | Anthropic thinking field. `omit` sends no thinking field and has the broadest compatibility. Use `disabled` or `adaptive` only when the selected model documents that mode; restore `omit` first if the provider returns HTTP 400. |
261
309
  | `provider.userAgent` | browser-compatible default | User-Agent sent by vision requests and explicit connection tests; override it for provider or proxy compatibility |
262
310
  | `language` | `zh` | Vision output language: `zh` or `en` |
263
311
  | `timeoutMs` | `60000` | Whole-operation deadline, 1000-600000 ms; each tool may request a narrower override |
264
- | `maxImageBytes` | `10485760` | Encoded-byte limit per input image |
265
- | `maxImagePixels` | `40000000` | Decoded-pixel limit per input image |
312
+ | `maxImageBytes` | `4194304` | Encoded-byte limit per input image; the built-in free service accepts up to 4 MiB |
313
+ | `maxImagePixels` | `20000000` | Decoded-pixel limit per input image; the built-in free service accepts up to 20,000,000 pixels |
266
314
  | `concurrency` | `4` | In-flight operations per session, 1-16 |
267
315
  | `runtime.mode` | `managed` | `managed` uses the packaged snapshot; `external` accepts only the exact pin |
268
316
  | `runtime.agentVisionToolkitPath` | unset | Required in `external` mode; exported exact snapshot or clean pinned Git checkout |
@@ -270,19 +318,34 @@ The bundle defaults to the managed runtime. A profile patch can override the pro
270
318
  | `allowedDirs` | `[]` | Additional realpath-resolved input roots; the session workspace is always allowed |
271
319
  | `imageInputVariants.enabled` | `true` | Register image-input variant entries for text-only model routes in the model selector |
272
320
  | `imageInputVariants.providers` | `[]` | Restrict wrapped upstream routes by provider id; empty wraps every eligible route |
273
- | `imageInputVariants.autoSwitch` | `true` | Automatically switch a text-only session to its image-input variant on paste, so the image keeps the native flow; `false` keeps the paste-to-path takeover for text-only models |
321
+ | `imageInputVariants.autoSwitch` | `false` | Opt into automatically switching a text-only session to its image-input variant on paste; the default `false` keeps the DSH-compatible paste-to-path flow |
274
322
 
275
323
  ### Credentials
276
324
 
277
- The Web Settings page accepts the actual value in its write-only **API key** field. Leave that field blank to retain an existing key; saving a non-empty value writes it under the advanced **Credential name** reference, which defaults to `VISION_API_KEY`. Headless deployments can pre-provision the same reference in `$DSH_HOME/.credentials.yaml`.
325
+ The built-in free provider uses the fixed `ANIONEX_FREE_VISION` reference and does not accept or store a user API key. If you change the endpoint, model, or protocol to a custom provider, the write-only **API key** field unlocks; saving a non-empty value writes it under the advanced **Credential name** reference. Headless deployments can pre-provision that custom reference in `$DSH_HOME/.credentials.yaml`.
278
326
 
279
327
  Settings store only the reference, never the value. The browser does not receive a stored value, and a successful save clears the field instead of echoing it. Remote operations resolve the reference once per call and inject the value only into that subprocess environment. The plugin excludes user `.env` files, checkout `.env` files, `PYTHONPATH`, `PYTHONHOME`, `VIRTUAL_ENV`, and user site-packages so ambient Python or upstream configuration cannot override the selected DSH provider. Logs, errors, tool results, Artifact metadata, and Settings responses never contain the secret.
280
328
 
281
- ### Managed and external runtimes
329
+ ### Built-in free service limits
330
+
331
+ The public service is shared and intended as a zero-configuration default, not an unlimited private endpoint. Limits are enforced by the proxy and returned as OpenAI-style errors with a reason code and readable message; rate-limit responses also include `Retry-After` and request-quota headers.
332
+
333
+ | Limit | Current value |
334
+ |---|---:|
335
+ | Per client | 100 requests per UTC day |
336
+ | Global service | 400 requests per UTC day |
337
+ | Burst | 20 requests per 60 seconds |
338
+ | Image bytes | 4 MiB per image |
339
+ | Decoded pixels | 20,000,000 per image |
340
+ | Output | 512 tokens maximum |
341
+
342
+ ### Managed runtime and optional external runtime
282
343
 
283
344
  Managed mode verifies `vendor/agent-vision-toolkit/UPSTREAM_MANIFEST.json`, prefers `uv`, falls back to `venv` plus pip, installs exact versions from `runtime/requirements.lock`, coordinates concurrent preparation with a heartbeat lock, and publishes a staged environment only after all probes pass.
284
345
 
285
- External mode is intended for development or controlled deployments:
346
+ Most users should stop at managed mode. It is included in the npm package and prepares the pinned Python environment for you.
347
+
348
+ The optional external mode is for plugin development or controlled deployments that already maintain the exact upstream checkout:
286
349
 
287
350
  ```yaml
288
351
  - id: vision-toolkit
@@ -299,6 +362,8 @@ The path must be an exported copy matching the packaged manifest or the root of
299
362
 
300
363
  The Web profile registers a Vision Toolkit Settings section for the provider URL, Credential reference, model, OpenAI/Anthropic protocol, Anthropic thinking mode, User-Agent, language, timeout, byte/pixel limits, concurrency, runtime mode, Python override, external source path, and allowed directories. It also shows plugin/upstream versions, the active runtime generation, non-secret Credential configured/source/writable facts, runtime paths, health results, and Artifact-route availability.
301
364
 
365
+ The **Plugin updates** card checks the profile's configured npm registry for a newer `@anionex/dsh-vision-toolkit` release. **Update and restart** installs that exact confirmed version into the current DSH profile, verifies the installed package, starts an independent restart helper, and gracefully restarts DSH Web; the open page waits for the replacement process and reloads after the new plugin version is serving. The action is same-origin, fixed to this package, serialized, and unavailable for `link:`, `file:`, workspace, git, URL, transitive, ambiguous, read-only, or missing-`pnpm` installations so local development sources are never overwritten. A restart can interrupt work that is currently running, so the UI requires an explicit confirmation.
366
+
302
367
  `Save and apply` validates the complete value, prepares the candidate Python/upstream runtime, commits the Settings revision, and only then atomically switches generations. A rejected candidate leaves the previous generation serving and is reported separately from a genuinely unavailable runtime. `Reload` always restores the authoritative saved value, even when its revision did not change, so a rejected browser draft is discarded. If initial startup cannot prepare a runtime, the Settings route remains available so a valid configuration can make the first generation operational. A stale browser revision receives a conflict instead of overwriting a newer save; reload before retrying. A read-only Settings provider allows inspection and health checks but disables saves.
303
368
 
304
369
  `Run health check` performs local checks only. `Test API connection` is an explicit action that sends the configured Credential to `GET /models`; OpenAI uses Bearer authentication, while Anthropic uses `x-api-key` and `anthropic-version`. That lightweight probe uploads no image and creates no completion. `Test vision model` separately sends the bundled `assets/vision-model-test.png` through the same multimodal runtime path as `vision_glance`; it creates one real completion and is the authoritative check that the selected endpoint, credential, model, protocol, and upstream account can process images. The Vision model health card displays a dedicated `Verified`, `Not tested`, or `Test failed` tag, so an HTTP 200 response from `/models` is not presented as a successful image test. Plugin load and ordinary Settings reads never make either request.
@@ -341,11 +406,6 @@ npm run example:ui-restoration:write
341
406
 
342
407
  The committed evidence records an initial `6.04%` difference across six non-zero worst regions and a final `0%` difference with no non-zero worst region. Check mode reproduces the tool path and verifies the committed assets; write mode intentionally refreshes the evidence.
343
408
 
344
- ## Communication group
345
-
346
- <img width="254" height="328" alt="image" src="https://github.com/user-attachments/assets/63c25c69-c3ba-4c47-8dee-98d60fe3954d" />
347
-
348
-
349
409
  ## Troubleshooting
350
410
 
351
411
  | Symptom | Resolution |
@@ -382,7 +442,14 @@ Update the upstream snapshot only through `pnpm run upstream:sync -- <checkout>`
382
442
 
383
443
  ## Project status and scope
384
444
 
385
- Version `0.1.9` is the current public npm release. P0 and P1 are product commitments in this package. P2 is a design threshold: no stable `ctx.visionToolkit` service, capability-discovery API, or provider ecosystem is published until at least one independent plugin consumes the internal capability shape. Web upload, drag-and-drop, camera/video/audio/document ingestion, interactive box editing, automatic GUI clicking, service clusters, model routing, model voting, and cross-session vision caches remain outside the current product.
445
+ Version `0.1.11` is the current public npm release. The product focuses on screenshot understanding, visual grounding, OCR, asset extraction, UI restoration, and pixel-level verification in DSH Web and Headless profiles. Web upload, drag-and-drop, camera/video/audio/document ingestion, interactive box editing, automatic GUI clicking, service clusters, model routing, model voting, and cross-session vision caches remain outside the current product.
446
+
447
+ <details>
448
+ <summary><strong>Maintainer scope note</strong></summary>
449
+
450
+ The stable `ctx.visionToolkit` service and capability-discovery API remain unpublished until an independent plugin becomes a real consumer. This keeps the public integration surface tied to a tested use case rather than an unvalidated ecosystem contract.
451
+
452
+ </details>
386
453
 
387
454
  ## Community and About
388
455
 
@@ -401,11 +468,3 @@ If you would like to follow my future work, [follow me on X](https://x.com/anion
401
468
  ## License
402
469
 
403
470
  The plugin is MIT-licensed. The packaged `agent-vision-toolkit` snapshot retains its upstream MIT license in `vendor/agent-vision-toolkit/LICENSE` and remains the sole implementation of its visual algorithms.
404
-
405
- ## Join the Community Group
406
-
407
- You are welcome to join the `agent-vision-toolkit` community group to exchange usage tips, share feedback, and suggest improvements. Scan the QR code below to join.
408
-
409
- <p align="center">
410
- <img src="assets/community-group-qr.png" alt="QR code for the agent-vision-toolkit community group" width="320">
411
- </p>