@anionex/dsh-vision-toolkit 0.1.9 → 0.1.10
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.i18n.yaml +2 -2
- package/README.md +40 -32
- package/README.zh.md +40 -27
- package/lib/client.js +22 -8
- package/lib/client.js.map +1 -1
- package/lib/config.js +20 -11
- package/lib/config.js.map +1 -1
- package/lib/defaults.js +6 -0
- package/lib/defaults.js.map +1 -0
- package/lib/image-input-variants.js +204 -23
- package/lib/image-input-variants.js.map +1 -1
- package/lib/runtime.js +8 -2
- package/lib/runtime.js.map +1 -1
- package/lib/types/client/index.d.ts +1 -1
- package/lib/types/client/index.d.ts.map +1 -1
- package/lib/types/config.d.ts +3 -0
- package/lib/types/config.d.ts.map +1 -1
- package/lib/types/defaults.d.ts +6 -0
- package/lib/types/defaults.d.ts.map +1 -0
- package/lib/types/image-input-variants.d.ts +2 -2
- package/lib/types/image-input-variants.d.ts.map +1 -1
- package/lib/types/runtime.d.ts +1 -1
- package/lib/types/runtime.d.ts.map +1 -1
- package/lib/types/web.d.ts.map +1 -1
- package/lib/web.js +7 -1
- package/lib/web.js.map +1 -1
- package/package.json +1 -1
- package/src/client/index.tsx +23 -8
- package/src/config.ts +30 -11
- package/src/defaults.ts +6 -0
- package/src/image-input-variants.ts +226 -25
- package/src/runtime.ts +8 -3
- package/src/web.ts +7 -0
package/README.i18n.yaml
CHANGED
|
@@ -2,5 +2,5 @@
|
|
|
2
2
|
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
|
3
3
|
# after editing either side, bring the other along and re-record with:
|
|
4
4
|
# pnpm run verify-translation-pairing --write dsh-vision-dark-theme/README.md
|
|
5
|
-
README.md:
|
|
6
|
-
README.zh.md:
|
|
5
|
+
README.md: 2a05d467a5c0cff82900d6074220392553f5d972
|
|
6
|
+
README.zh.md: 07cff26ce48a7f0df6791ac9723debaeb723d5f4
|
package/README.md
CHANGED
|
@@ -5,8 +5,8 @@
|
|
|
5
5
|
[](https://dshfind.com/en/plugins/Anionex/dsh-vision-toolkit)
|
|
6
6
|
[](https://dshfind.com/en/plugins/Anionex/dsh-vision-toolkit)
|
|
7
7
|
[](https://x.com/anion_ex)
|
|
8
|
-
[](https://github.com/Anionex/dsh-vision-toolkit/releases/tag/v0.1.10)
|
|
9
|
+
[](tests)
|
|
10
10
|
[](LICENSE)
|
|
11
11
|
[](package.json)
|
|
12
12
|
[](runtime/requirements.lock)
|
|
@@ -101,7 +101,15 @@ dsh --profile headless --dump-config | grep vision-toolkit
|
|
|
101
101
|
|
|
102
102
|
Legacy profiles must use `nodeLinker: hoisted` and `autoInstallPeers: false` in their `pnpm-workspace.yaml`. An updated DSH launcher repairs these owned settings before `dsh plugin` runs; when using an older launcher, set them before installation so pnpm does not assemble a second Harness dependency graph inside the profile.
|
|
103
103
|
|
|
104
|
-
Restart a running Web profile, open **Settings → Vision Toolkit**,
|
|
104
|
+
Restart a running Web profile, open **Settings → Vision Toolkit**, and run **Test API connection** followed by **Test vision model**. New installations use the built-in free Moondream provider automatically, so no API key or DSH Credential is required. To use another provider, edit the endpoint/model/protocol and provide its DSH Credential. In a conversation, make an image available as a workspace path, invoke `/vision-tools`, and ask the Agent to call a specific `vision_*` tool. Local crop, trace, pixel, color, foreground, and HTML operations do not require a visual API credential.
|
|
105
|
+
|
|
106
|
+
## Community Group
|
|
107
|
+
|
|
108
|
+
Join the `agent-vision-toolkit` community group to exchange usage tips, share feedback, and suggest improvements.
|
|
109
|
+
|
|
110
|
+
<p align="center">
|
|
111
|
+
<img src="assets/community-group-qr.png" alt="QR code for the agent-vision-toolkit community group" width="260">
|
|
112
|
+
</p>
|
|
105
113
|
|
|
106
114
|
## How it works
|
|
107
115
|
|
|
@@ -149,18 +157,18 @@ Health checks, connection testing, and plugin/upstream version inspection are ad
|
|
|
149
157
|
|
|
150
158
|
## Image-input variants for text-only models
|
|
151
159
|
|
|
152
|
-
Text-only model routes get sibling model-selector entries named `<model> (Vision Toolkit)` under a matching provider group. A variant declares image input, so pasted images keep the native attachment flow — composer thumbnail, durable session image, and history rendering — and the plugin rewrites every image block into a Vision Toolkit description only on the wire to the model, before the request reaches the upstream route. The session log is untouched; replay and the UI keep the real image.
|
|
160
|
+
Text-only model routes get sibling model-selector entries named `<model> (Vision Toolkit)` under a matching provider group. A variant declares image input, so pasted images keep the native attachment flow — composer thumbnail, durable session image, and history rendering — and the plugin rewrites every image block into a Vision Toolkit description only on the wire to the model, before the request reaches the upstream route. The vision prompt is focus-hinted with the latest user or assistant intent, using the same role, instruction, image-text policy, and `[vision model description]` channel markers as `agent-vision-toolkit`; the model receives task-relevant evidence instead of a broad generic description. The session log is untouched; replay and the UI keep the real image.
|
|
153
161
|
|
|
154
162
|
A variant is registered automatically for every model the host positively declares text-only (for example the DeepSeek chat family). Paste handling is automatic: when the current model is confirmed text-only and its variant exists, the browser integration switches the session to the variant by itself (a short notice names the new model) and the paste then keeps the native flow; no manual model change is needed. The host's verdict uses the exact model route the browser read from the live model catalog, with the selector label as fallback; unconfirmed or image-capable routes always keep the native flow, and a text-only model without a variant (for example when variants are disabled) keeps the paste-to-path takeover, which copies the image into the session workspace and inserts its path as text.
|
|
155
163
|
|
|
156
|
-
Description conversion needs the configured vision provider and its credential; when the runtime is not ready or a read fails, the wire block degrades to
|
|
164
|
+
Description conversion needs the configured vision provider and its credential; when the runtime is not ready or a read fails, the wire block degrades to the upstream-compatible `[vision unavailable: ...]` note instead of failing the turn. The bridge does not treat injected context files as the current user intent, and it uses the latest assistant paragraph when a tool-fetched image is being described. Disable variants with `imageInputVariants.enabled: false`, restrict the wrapped routes with `imageInputVariants.providers`, or keep the paste-to-path behavior for text-only models with `imageInputVariants.autoSwitch: false`.
|
|
157
165
|
|
|
158
166
|
## Requirements
|
|
159
167
|
|
|
160
168
|
- DeepSeek Harness with a Web or Headless profile and `pnpm` available to `dsh plugin`.
|
|
161
169
|
- Python 3.11 or newer. Managed mode creates an isolated environment, so users do not install the upstream CLI or Python packages manually.
|
|
162
170
|
- Network access on the first managed-runtime activation unless the exact packages in `runtime/requirements.lock` are already available in the configured package cache.
|
|
163
|
-
-
|
|
171
|
+
- The built-in free Moondream provider is ready for `vision_glance`, `vision_ground`, `vision_detect`, and non-split-only long-screenshot OCR. A DSH Credential is required only when a custom OpenAI-compatible or Anthropic endpoint is configured. Local tools remain usable without either provider.
|
|
164
172
|
- Chrome, Chromium, or Edge only for `vision_html_screenshot`; all other tools remain available when no supported browser is installed.
|
|
165
173
|
- PNG, JPEG, GIF, or WebP inputs inside the session workspace or an explicitly configured `allowedDirs` root.
|
|
166
174
|
|
|
@@ -201,7 +209,7 @@ dsh plugin --profile web remove @dsh-external/dsh-vision-toolkit
|
|
|
201
209
|
dsh plugin --profile web add @anionex/dsh-vision-toolkit
|
|
202
210
|
```
|
|
203
211
|
|
|
204
|
-
After restarting, Settings → Vision should report plugin version **0.1.
|
|
212
|
+
After restarting, Settings → Vision should report plugin version **0.1.10**. The built-in free provider is selected automatically; custom providers still use the configured DSH Credential.
|
|
205
213
|
|
|
206
214
|
For a registry installation, update the dependency through the profile package manager:
|
|
207
215
|
|
|
@@ -229,16 +237,16 @@ The bundle defaults to the managed runtime. A profile patch can override the pro
|
|
|
229
237
|
- id: vision-toolkit
|
|
230
238
|
config:
|
|
231
239
|
provider:
|
|
232
|
-
baseUrl: https://
|
|
233
|
-
credential:
|
|
234
|
-
model:
|
|
240
|
+
baseUrl: https://vision.anionex.me/v1
|
|
241
|
+
credential: ANIONEX_FREE_VISION
|
|
242
|
+
model: moondream-3.1
|
|
235
243
|
protocol: openai
|
|
236
244
|
anthropicThinking: omit
|
|
237
245
|
userAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36
|
|
238
246
|
language: zh
|
|
239
247
|
timeoutMs: 60000
|
|
240
|
-
maxImageBytes:
|
|
241
|
-
maxImagePixels:
|
|
248
|
+
maxImageBytes: 4194304
|
|
249
|
+
maxImagePixels: 20000000
|
|
242
250
|
concurrency: 4
|
|
243
251
|
runtime:
|
|
244
252
|
mode: managed
|
|
@@ -253,16 +261,16 @@ The bundle defaults to the managed runtime. A profile patch can override the pro
|
|
|
253
261
|
|
|
254
262
|
| Field | Default | Contract |
|
|
255
263
|
|---|---|---|
|
|
256
|
-
| `provider.baseUrl` | `https://
|
|
257
|
-
| `provider.credential` | `
|
|
258
|
-
| `provider.model` | `
|
|
264
|
+
| `provider.baseUrl` | `https://vision.anionex.me/v1` | Built-in free OpenAI-compatible endpoint; custom providers may use another base URL, normalized without trailing slashes |
|
|
265
|
+
| `provider.credential` | `ANIONEX_FREE_VISION` | Read-only built-in reference for the free service; custom providers use a DSH Credential reference, never a secret value |
|
|
266
|
+
| `provider.model` | `moondream-3.1` | Multimodal model name sent to remote tools |
|
|
259
267
|
| `provider.protocol` | `openai` | `openai` sends Chat Completions requests; `anthropic` sends native Messages requests |
|
|
260
268
|
| `provider.anthropicThinking` | `omit` | Anthropic thinking field. `omit` sends no thinking field and has the broadest compatibility. Use `disabled` or `adaptive` only when the selected model documents that mode; restore `omit` first if the provider returns HTTP 400. |
|
|
261
269
|
| `provider.userAgent` | browser-compatible default | User-Agent sent by vision requests and explicit connection tests; override it for provider or proxy compatibility |
|
|
262
270
|
| `language` | `zh` | Vision output language: `zh` or `en` |
|
|
263
271
|
| `timeoutMs` | `60000` | Whole-operation deadline, 1000-600000 ms; each tool may request a narrower override |
|
|
264
|
-
| `maxImageBytes` | `
|
|
265
|
-
| `maxImagePixels` | `
|
|
272
|
+
| `maxImageBytes` | `4194304` | Encoded-byte limit per input image; the built-in free service accepts up to 4 MiB |
|
|
273
|
+
| `maxImagePixels` | `20000000` | Decoded-pixel limit per input image; the built-in free service accepts up to 20,000,000 pixels |
|
|
266
274
|
| `concurrency` | `4` | In-flight operations per session, 1-16 |
|
|
267
275
|
| `runtime.mode` | `managed` | `managed` uses the packaged snapshot; `external` accepts only the exact pin |
|
|
268
276
|
| `runtime.agentVisionToolkitPath` | unset | Required in `external` mode; exported exact snapshot or clean pinned Git checkout |
|
|
@@ -274,10 +282,23 @@ The bundle defaults to the managed runtime. A profile patch can override the pro
|
|
|
274
282
|
|
|
275
283
|
### Credentials
|
|
276
284
|
|
|
277
|
-
The
|
|
285
|
+
The built-in free provider uses the fixed `ANIONEX_FREE_VISION` reference and does not accept or store a user API key. If you change the endpoint, model, or protocol to a custom provider, the write-only **API key** field unlocks; saving a non-empty value writes it under the advanced **Credential name** reference. Headless deployments can pre-provision that custom reference in `$DSH_HOME/.credentials.yaml`.
|
|
278
286
|
|
|
279
287
|
Settings store only the reference, never the value. The browser does not receive a stored value, and a successful save clears the field instead of echoing it. Remote operations resolve the reference once per call and inject the value only into that subprocess environment. The plugin excludes user `.env` files, checkout `.env` files, `PYTHONPATH`, `PYTHONHOME`, `VIRTUAL_ENV`, and user site-packages so ambient Python or upstream configuration cannot override the selected DSH provider. Logs, errors, tool results, Artifact metadata, and Settings responses never contain the secret.
|
|
280
288
|
|
|
289
|
+
### Built-in free service limits
|
|
290
|
+
|
|
291
|
+
The public service is shared and intended as a zero-configuration default, not an unlimited private endpoint. Limits are enforced by the proxy and returned as OpenAI-style errors with a reason code and readable message; rate-limit responses also include `Retry-After` and request-quota headers.
|
|
292
|
+
|
|
293
|
+
| Limit | Current value |
|
|
294
|
+
|---|---:|
|
|
295
|
+
| Per client | 30 requests per UTC day |
|
|
296
|
+
| Global service | 120 requests per UTC day |
|
|
297
|
+
| Burst | 6 requests per 60 seconds |
|
|
298
|
+
| Image bytes | 4 MiB per image |
|
|
299
|
+
| Decoded pixels | 20,000,000 per image |
|
|
300
|
+
| Output | 512 tokens maximum |
|
|
301
|
+
|
|
281
302
|
### Managed and external runtimes
|
|
282
303
|
|
|
283
304
|
Managed mode verifies `vendor/agent-vision-toolkit/UPSTREAM_MANIFEST.json`, prefers `uv`, falls back to `venv` plus pip, installs exact versions from `runtime/requirements.lock`, coordinates concurrent preparation with a heartbeat lock, and publishes a staged environment only after all probes pass.
|
|
@@ -341,11 +362,6 @@ npm run example:ui-restoration:write
|
|
|
341
362
|
|
|
342
363
|
The committed evidence records an initial `6.04%` difference across six non-zero worst regions and a final `0%` difference with no non-zero worst region. Check mode reproduces the tool path and verifies the committed assets; write mode intentionally refreshes the evidence.
|
|
343
364
|
|
|
344
|
-
## Communication group
|
|
345
|
-
|
|
346
|
-
<img width="254" height="328" alt="image" src="https://github.com/user-attachments/assets/63c25c69-c3ba-4c47-8dee-98d60fe3954d" />
|
|
347
|
-
|
|
348
|
-
|
|
349
365
|
## Troubleshooting
|
|
350
366
|
|
|
351
367
|
| Symptom | Resolution |
|
|
@@ -382,7 +398,7 @@ Update the upstream snapshot only through `pnpm run upstream:sync -- <checkout>`
|
|
|
382
398
|
|
|
383
399
|
## Project status and scope
|
|
384
400
|
|
|
385
|
-
Version `0.1.
|
|
401
|
+
Version `0.1.10` is the current public npm release. P0 and P1 are product commitments in this package. P2 is a design threshold: no stable `ctx.visionToolkit` service, capability-discovery API, or provider ecosystem is published until at least one independent plugin consumes the internal capability shape. Web upload, drag-and-drop, camera/video/audio/document ingestion, interactive box editing, automatic GUI clicking, service clusters, model routing, model voting, and cross-session vision caches remain outside the current product.
|
|
386
402
|
|
|
387
403
|
## Community and About
|
|
388
404
|
|
|
@@ -401,11 +417,3 @@ If you would like to follow my future work, [follow me on X](https://x.com/anion
|
|
|
401
417
|
## License
|
|
402
418
|
|
|
403
419
|
The plugin is MIT-licensed. The packaged `agent-vision-toolkit` snapshot retains its upstream MIT license in `vendor/agent-vision-toolkit/LICENSE` and remains the sole implementation of its visual algorithms.
|
|
404
|
-
|
|
405
|
-
## Join the Community Group
|
|
406
|
-
|
|
407
|
-
You are welcome to join the `agent-vision-toolkit` community group to exchange usage tips, share feedback, and suggest improvements. Scan the QR code below to join.
|
|
408
|
-
|
|
409
|
-
<p align="center">
|
|
410
|
-
<img src="assets/community-group-qr.png" alt="QR code for the agent-vision-toolkit community group" width="320">
|
|
411
|
-
</p>
|
package/README.zh.md
CHANGED
|
@@ -5,8 +5,8 @@
|
|
|
5
5
|
[](https://dshfind.com/zh/plugins/Anionex/dsh-vision-toolkit)
|
|
6
6
|
[](https://dshfind.com/zh/plugins/Anionex/dsh-vision-toolkit)
|
|
7
7
|
[](https://x.com/anion_ex)
|
|
8
|
-
[](https://github.com/Anionex/dsh-vision-toolkit/releases/tag/v0.1.10)
|
|
9
|
+
[](tests)
|
|
10
10
|
[](LICENSE)
|
|
11
11
|
[](package.json)
|
|
12
12
|
[](runtime/requirements.lock)
|
|
@@ -101,7 +101,15 @@ dsh --profile headless --dump-config | grep vision-toolkit
|
|
|
101
101
|
|
|
102
102
|
旧 Profile 的 `pnpm-workspace.yaml` 必须使用 `nodeLinker: hoisted` 和 `autoInstallPeers: false`。更新后的 DSH launcher 会在 `dsh plugin` 运行前修复这两个自有设置;使用旧 launcher 时,应在安装前手动设置,避免 pnpm 在 Profile 内组装第二套 Harness 依赖图。
|
|
103
103
|
|
|
104
|
-
安装后重启正在运行的 Web Profile,打开 **设置 →
|
|
104
|
+
安装后重启正在运行的 Web Profile,打开 **设置 → 视觉工具**,先执行**测试 API 连接**,再执行**测试视觉模型**。新安装会自动使用内置免费 Moondream 提供方,不需要 API Key 或 DSH Credential。若要使用其他提供方,请修改端点、模型或协议,并配置对应的 DSH Credential。在会话中把图片放进工作区路径,调用 `/vision-tools`,再让 Agent 使用明确的 `vision_*` 工具。本地裁剪、SVG、像素、颜色、前景和 HTML 操作不需要视觉 API Credential。
|
|
105
|
+
|
|
106
|
+
## 加入交流群
|
|
107
|
+
|
|
108
|
+
欢迎加入 `agent-vision-toolkit` 项目交流群,交流使用经验、反馈问题并提出建议。
|
|
109
|
+
|
|
110
|
+
<p align="center">
|
|
111
|
+
<img src="assets/community-group-qr.png" alt="agent-vision-toolkit 项目交流群二维码" width="260">
|
|
112
|
+
</p>
|
|
105
113
|
|
|
106
114
|
## 工作原理
|
|
107
115
|
|
|
@@ -149,18 +157,18 @@ flowchart LR
|
|
|
149
157
|
|
|
150
158
|
## 纯文本模型的图片输入变体
|
|
151
159
|
|
|
152
|
-
纯文本模型路由会获得同名的兄弟模型条目:`<模型名> (Vision Toolkit)`,挂在对应的提供方分组下。变体声明支持图片输入,因此粘贴的图片走原生附件流程——输入框缩略图、会话持久化图片与历史渲染全部保留——插件只在发往模型的请求链路上把每个图片块改写成 Vision Toolkit
|
|
160
|
+
纯文本模型路由会获得同名的兄弟模型条目:`<模型名> (Vision Toolkit)`,挂在对应的提供方分组下。变体声明支持图片输入,因此粘贴的图片走原生附件流程——输入框缩略图、会话持久化图片与历史渲染全部保留——插件只在发往模型的请求链路上把每个图片块改写成 Vision Toolkit 描述文本,再转交上游路由。视觉提示会携带最新的用户或助手意图,并与 `agent-vision-toolkit` 对齐角色提示、描述要求、图片文字策略以及 `[vision model description]` 等通道标记;模型获得的是与当前任务相关的证据,而不是宽泛的通用描述。会话日志不被改动;回放与 UI 看到的始终是真实图片。
|
|
153
161
|
|
|
154
162
|
插件会自动为宿主明确声明为纯文本的每个模型注册变体(例如 DeepSeek 对话家族)。粘贴处理是全自动的:当当前模型被确认为纯文本、且它的变体已注册时,浏览器端集成会自动把会话切换到变体(会有一条简短提示说明新模型名),随后粘贴走原生流程,无需手动切换模型。宿主依据浏览器从实时模型目录读到的精确模型路由来裁决,模型选择器标签作为兜底;无法确认或支持图片的路由一律保持原生流程,而"纯文本但没有变体"的模型(例如变体被关闭时)继续走"粘贴转路径":图片被复制进会话工作区,输入框里插入的是它的路径文本。
|
|
155
163
|
|
|
156
|
-
描述转换需要已配置的视觉提供方及其 Credential
|
|
164
|
+
描述转换需要已配置的视觉提供方及其 Credential;当运行时未就绪或读取失败时,请求链路上的图片块降级为与上游兼容的 `[vision unavailable: ...]` 提示,而不是让整轮失败。桥接不会把注入的上下文文件当作当前用户意图;如果图片来自工具调用,则使用最新的助手段落作为关注提示。用 `imageInputVariants.enabled: false` 关闭变体,用 `imageInputVariants.providers` 限制被包装的路由,或用 `imageInputVariants.autoSwitch: false` 让纯文本模型继续走"粘贴转路径"。
|
|
157
165
|
|
|
158
166
|
## 运行要求
|
|
159
167
|
|
|
160
168
|
- 启用 Web 或 Headless Profile 的 DeepSeek Harness,并确保 `dsh plugin` 可以使用 `pnpm`。
|
|
161
169
|
- Python 3.11 或更高版本。Managed 模式会创建隔离环境,用户无需手工安装上游 CLI(命令行界面)或 Python 包。
|
|
162
170
|
- 首次启用 managed 运行时需要联网;如果配置的软件包缓存已有 `runtime/requirements.lock` 中的精确版本,则无需联网。
|
|
163
|
-
- `vision_glance`、`vision_ground`、`vision_detect` 和非仅切分长截图 OCR
|
|
171
|
+
- 内置免费 Moondream 提供方可直接用于 `vision_glance`、`vision_ground`、`vision_detect` 和非仅切分长截图 OCR。只有改用自定义 OpenAI 兼容或 Anthropic 端点时才需要 DSH Credential;本地工具不依赖任何远程提供方。
|
|
164
172
|
- 只有 `vision_html_screenshot` 需要 Chrome、Chromium 或 Edge;未安装受支持浏览器时,其他工具保持可用。
|
|
165
173
|
- 输入必须是会话工作区或显式 `allowedDirs` 根目录内的 PNG、JPEG、GIF 或 WebP。
|
|
166
174
|
|
|
@@ -201,7 +209,7 @@ dsh plugin --profile web remove @dsh-external/dsh-vision-toolkit
|
|
|
201
209
|
dsh plugin --profile web add @anionex/dsh-vision-toolkit
|
|
202
210
|
```
|
|
203
211
|
|
|
204
|
-
重启后,Settings → 视觉工具 应显示插件版本 **0.1.
|
|
212
|
+
重启后,Settings → 视觉工具 应显示插件版本 **0.1.10**。内置免费提供方会自动选中;自定义提供方仍使用配置的 DSH Credential。
|
|
205
213
|
|
|
206
214
|
通过注册表安装时,使用 Profile 的包管理命令更新依赖:
|
|
207
215
|
|
|
@@ -229,16 +237,16 @@ Bundle 默认使用 managed 运行时。Profile patch 可以覆盖提供方与
|
|
|
229
237
|
- id: vision-toolkit
|
|
230
238
|
config:
|
|
231
239
|
provider:
|
|
232
|
-
baseUrl: https://
|
|
233
|
-
credential:
|
|
234
|
-
model:
|
|
240
|
+
baseUrl: https://vision.anionex.me/v1
|
|
241
|
+
credential: ANIONEX_FREE_VISION
|
|
242
|
+
model: moondream-3.1
|
|
235
243
|
protocol: openai
|
|
236
244
|
anthropicThinking: omit
|
|
237
245
|
userAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36
|
|
238
246
|
language: zh
|
|
239
247
|
timeoutMs: 60000
|
|
240
|
-
maxImageBytes:
|
|
241
|
-
maxImagePixels:
|
|
248
|
+
maxImageBytes: 4194304
|
|
249
|
+
maxImagePixels: 20000000
|
|
242
250
|
concurrency: 4
|
|
243
251
|
runtime:
|
|
244
252
|
mode: managed
|
|
@@ -253,16 +261,16 @@ Bundle 默认使用 managed 运行时。Profile patch 可以覆盖提供方与
|
|
|
253
261
|
|
|
254
262
|
| 字段 | 默认值 | 契约 |
|
|
255
263
|
|---|---|---|
|
|
256
|
-
| `provider.baseUrl` | `https://
|
|
257
|
-
| `provider.credential` | `
|
|
258
|
-
| `provider.model` | `
|
|
264
|
+
| `provider.baseUrl` | `https://vision.anionex.me/v1` | 内置免费 OpenAI 兼容端点;自定义提供方可改用其他基础 URL,使用时会去除结尾斜杠 |
|
|
265
|
+
| `provider.credential` | `ANIONEX_FREE_VISION` | 免费服务的只读内置引用;自定义提供方使用 DSH Credential 引用,而不是密钥值 |
|
|
266
|
+
| `provider.model` | `moondream-3.1` | 远程工具使用的多模态模型名 |
|
|
259
267
|
| `provider.protocol` | `openai` | `openai` 发送 Chat Completions 请求;`anthropic` 发送原生 Messages 请求 |
|
|
260
268
|
| `provider.anthropicThinking` | `omit` | Anthropic thinking 字段。`omit` 不发送 thinking 字段,兼容性最好;仅当所选模型明确支持时使用 `disabled` 或 `adaptive`,提供方返回 HTTP 400 时应先恢复 `omit`。 |
|
|
261
269
|
| `provider.userAgent` | 浏览器兼容默认值 | 视觉请求和显式连接测试发送的 User-Agent;可为提供方或代理兼容性覆盖 |
|
|
262
270
|
| `language` | `zh` | 视觉输出语言:`zh` 或 `en` |
|
|
263
271
|
| `timeoutMs` | `60000` | 完整操作截止时间,1000-600000 毫秒;每个工具可请求更窄的覆盖值 |
|
|
264
|
-
| `maxImageBytes` | `
|
|
265
|
-
| `maxImagePixels` | `
|
|
272
|
+
| `maxImageBytes` | `4194304` | 每张输入图片的编码字节上限;内置免费服务最多接受 4 MiB |
|
|
273
|
+
| `maxImagePixels` | `20000000` | 每张输入图片的解码像素上限;内置免费服务最多接受 20,000,000 像素 |
|
|
266
274
|
| `concurrency` | `4` | 每个会话内的并发操作数,1-16 |
|
|
267
275
|
| `runtime.mode` | `managed` | `managed` 使用打包快照;`external` 只接受精确固定版本 |
|
|
268
276
|
| `runtime.agentVisionToolkitPath` | 未设置 | `external` 模式必填;必须是精确导出快照或固定 commit 的干净 Git checkout |
|
|
@@ -274,10 +282,23 @@ Bundle 默认使用 managed 运行时。Profile patch 可以覆盖提供方与
|
|
|
274
282
|
|
|
275
283
|
### Credential
|
|
276
284
|
|
|
277
|
-
|
|
285
|
+
内置免费提供方使用固定的 `ANIONEX_FREE_VISION` 引用,不接受也不会保存用户 API Key。修改端点、模型或协议切换到自定义提供方后,只写的 **API 密钥** 输入框会自动解锁;填写后保存,会把密钥写入高级设置中的 **凭据名称** 引用。Headless 部署可以在 `$DSH_HOME/.credentials.yaml` 中预置该自定义引用。
|
|
278
286
|
|
|
279
287
|
Settings 只保存引用,不保存值。浏览器不会读取已保存的密钥,保存成功后输入框也会立即清空而不是回显。每次远程操作都会重新解析引用,并只把值注入对应子进程环境。插件排除用户 `.env`、checkout `.env`、`PYTHONPATH`、`PYTHONHOME`、`VIRTUAL_ENV` 和用户 site-packages,避免环境中的 Python 或上游配置覆盖选定的 DSH 提供方。日志、错误、工具结果、产物元数据和 Settings 响应都不包含密钥。
|
|
280
288
|
|
|
289
|
+
### 内置免费服务限制
|
|
290
|
+
|
|
291
|
+
公开服务是共享的零配置默认入口,不是无限量私有端点。限制由代理执行,并以 OpenAI 风格错误返回明确的原因代码和可读提示;限流响应还会携带 `Retry-After` 与请求额度响应头。
|
|
292
|
+
|
|
293
|
+
| 限制 | 当前值 |
|
|
294
|
+
|---|---:|
|
|
295
|
+
| 单客户端 | 每个 UTC 日 30 次 |
|
|
296
|
+
| 全局服务 | 每个 UTC 日 120 次 |
|
|
297
|
+
| 突发 | 60 秒内 6 次 |
|
|
298
|
+
| 图片字节 | 每张最多 4 MiB |
|
|
299
|
+
| 解码像素 | 每张最多 20,000,000 像素 |
|
|
300
|
+
| 输出 | 最多 512 tokens |
|
|
301
|
+
|
|
281
302
|
### Managed 与 external 运行时
|
|
282
303
|
|
|
283
304
|
Managed 模式会验证 `vendor/agent-vision-toolkit/UPSTREAM_MANIFEST.json`,优先使用 `uv`,回退到 `venv` 加 pip,按 `runtime/requirements.lock` 安装精确版本,通过 heartbeat 锁协调并发准备,并只在全部探针通过后发布 staging 环境。
|
|
@@ -390,7 +411,7 @@ pnpm pack --dry-run
|
|
|
390
411
|
|
|
391
412
|
## 项目状态与范围
|
|
392
413
|
|
|
393
|
-
版本 `0.1.
|
|
414
|
+
版本 `0.1.10` 是当前公开 npm 发布。P0 和 P1 是本包的产品承诺。P2 是设计门槛:至少一个独立插件消费内部能力形态前,不发布稳定 `ctx.visionToolkit` 服务、能力发现 API 或提供方生态。Web 上传、拖拽、摄像头/视频/音频/文档输入、交互式标注框编辑、GUI 自动点击、远程服务集群、模型路由、模型投票和跨会话视觉缓存不属于当前产品范围。
|
|
394
415
|
|
|
395
416
|
## 社区与关于
|
|
396
417
|
|
|
@@ -409,11 +430,3 @@ pnpm pack --dry-run
|
|
|
409
430
|
## 许可证
|
|
410
431
|
|
|
411
432
|
插件采用 MIT 许可。打包的 `agent-vision-toolkit` 快照在 `vendor/agent-vision-toolkit/LICENSE` 保留上游 MIT 许可证,并继续作为视觉算法的唯一实现。
|
|
412
|
-
|
|
413
|
-
## 加入交流群
|
|
414
|
-
|
|
415
|
-
欢迎加入 `agent-vision-toolkit` 项目交流群,交流使用经验、反馈问题并提出建议。请扫描下方二维码入群。
|
|
416
|
-
|
|
417
|
-
<p align="center">
|
|
418
|
-
<img src="assets/community-group-qr.png" alt="agent-vision-toolkit 项目交流群二维码" width="320">
|
|
419
|
-
</p>
|
package/lib/client.js
CHANGED
|
@@ -18,6 +18,10 @@ const NS = 'vision-toolkit';
|
|
|
18
18
|
const SETTINGS_ROUTE = '/_dsh/vision-toolkit/settings';
|
|
19
19
|
const PRESENTATION_META_KEY = '$dshVisionToolkit';
|
|
20
20
|
const DEFAULT_USER_AGENT = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36';
|
|
21
|
+
// Keep these browser defaults aligned with src/defaults.ts without importing server-side config.
|
|
22
|
+
const BUILT_IN_FREE_VISION_BASE_URL = 'https://vision.anionex.me/v1';
|
|
23
|
+
const BUILT_IN_FREE_VISION_CREDENTIAL = 'ANIONEX_FREE_VISION';
|
|
24
|
+
const BUILT_IN_FREE_VISION_MODEL = 'moondream-3.1';
|
|
21
25
|
const en = {
|
|
22
26
|
nav: 'Vision',
|
|
23
27
|
settingsTitle: 'Vision Toolkit',
|
|
@@ -34,7 +38,7 @@ const en = {
|
|
|
34
38
|
apiKeyBlank: 'The API key cannot contain only spaces.',
|
|
35
39
|
apiKeyInvalid: 'Paste only the key, without a variable name, quotes, spaces, or line breaks.',
|
|
36
40
|
credential: 'Credential name',
|
|
37
|
-
credentialHint: '
|
|
41
|
+
credentialHint: 'The built-in free provider needs no user key. For a custom provider, this is the DSH credential reference used to store its key.',
|
|
38
42
|
model: 'Model',
|
|
39
43
|
protocol: 'API protocol',
|
|
40
44
|
anthropicThinking: 'Anthropic thinking',
|
|
@@ -181,7 +185,7 @@ const zh = {
|
|
|
181
185
|
apiKeyBlank: 'API 密钥不能只包含空格。',
|
|
182
186
|
apiKeyInvalid: '请只粘贴密钥本身,不要包含变量名、引号、空格或换行。',
|
|
183
187
|
credential: '凭据名称',
|
|
184
|
-
credentialHint: '
|
|
188
|
+
credentialHint: '内置免费视觉服务无需用户密钥;切换到自定义服务时,此处是保存其密钥的 DSH 凭据名称。',
|
|
185
189
|
model: '模型名称',
|
|
186
190
|
protocol: 'API 协议',
|
|
187
191
|
anthropicThinking: 'Anthropic thinking',
|
|
@@ -641,16 +645,16 @@ class VisionSettingsController {
|
|
|
641
645
|
exports.VisionSettingsController = VisionSettingsController;
|
|
642
646
|
function draftOf(value) {
|
|
643
647
|
return {
|
|
644
|
-
baseUrl: value.provider?.baseUrl ??
|
|
645
|
-
credential: value.provider?.credential ??
|
|
646
|
-
model: value.provider?.model ??
|
|
648
|
+
baseUrl: value.provider?.baseUrl ?? BUILT_IN_FREE_VISION_BASE_URL,
|
|
649
|
+
credential: value.provider?.credential ?? BUILT_IN_FREE_VISION_CREDENTIAL,
|
|
650
|
+
model: value.provider?.model ?? BUILT_IN_FREE_VISION_MODEL,
|
|
647
651
|
protocol: value.provider?.protocol ?? 'openai',
|
|
648
652
|
anthropicThinking: value.provider?.anthropicThinking ?? 'omit',
|
|
649
653
|
userAgent: value.provider?.userAgent ?? DEFAULT_USER_AGENT,
|
|
650
654
|
language: value.language ?? 'zh',
|
|
651
655
|
timeoutMs: String(value.timeoutMs ?? 60000),
|
|
652
|
-
maxImageBytes: String(value.maxImageBytes ??
|
|
653
|
-
maxImagePixels: String(value.maxImagePixels ??
|
|
656
|
+
maxImageBytes: String(value.maxImageBytes ?? 4194304),
|
|
657
|
+
maxImagePixels: String(value.maxImagePixels ?? 20000000),
|
|
654
658
|
concurrency: String(value.concurrency ?? 4),
|
|
655
659
|
runtimeMode: value.runtime?.mode ?? 'managed',
|
|
656
660
|
toolkitPath: value.runtime?.agentVisionToolkitPath ?? '',
|
|
@@ -699,6 +703,12 @@ function valueOf(draft, t) {
|
|
|
699
703
|
allowedDirs: draft.allowedDirs.split(/\r?\n/).map(entry => entry.trim()).filter(Boolean),
|
|
700
704
|
};
|
|
701
705
|
}
|
|
706
|
+
function isBuiltInFreeVisionDraft(draft) {
|
|
707
|
+
return draft.baseUrl.trim().replace(/\/+$/, '') === BUILT_IN_FREE_VISION_BASE_URL
|
|
708
|
+
&& draft.credential.trim() === BUILT_IN_FREE_VISION_CREDENTIAL
|
|
709
|
+
&& draft.model.trim() === BUILT_IN_FREE_VISION_MODEL
|
|
710
|
+
&& draft.protocol === 'openai';
|
|
711
|
+
}
|
|
702
712
|
function Field({ label, children, hint }) {
|
|
703
713
|
return (0, jsx_runtime_1.jsxs)("label", { className: "dvt-field", children: [(0, jsx_runtime_1.jsx)("span", { children: label }), children, hint === undefined ? null : (0, jsx_runtime_1.jsx)("small", { children: hint })] });
|
|
704
714
|
}
|
|
@@ -840,7 +850,11 @@ function LoadedSettings({ controller, t }) {
|
|
|
840
850
|
};
|
|
841
851
|
const busy = state.action !== undefined;
|
|
842
852
|
const credentialMatchesSnapshot = draft.credential.trim() === snapshot.credential.ref;
|
|
843
|
-
const
|
|
853
|
+
const builtInCredentialChangedProvider = snapshot.credential.source === 'built-in-free'
|
|
854
|
+
&& !isBuiltInFreeVisionDraft(draft);
|
|
855
|
+
const keyLocked = credentialMatchesSnapshot
|
|
856
|
+
&& !snapshot.credential.writable
|
|
857
|
+
&& !builtInCredentialChangedProvider;
|
|
844
858
|
const canSave = snapshot.writable || (apiKey.length > 0 && !keyLocked);
|
|
845
859
|
const runtimeErrorTitle = snapshot.runtime.ready ? t('runtimeCandidateRejected') : t('runtimeUnavailable');
|
|
846
860
|
return ((0, jsx_runtime_1.jsxs)("div", { className: "dvt-settings", children: [(0, jsx_runtime_1.jsx)("div", { className: "dvt-alert notice", children: t('externalNotice') }), !snapshot.writable ? (0, jsx_runtime_1.jsx)("div", { className: "dvt-alert warning", children: t('readOnly') }) : null, draftError === undefined ? null : (0, jsx_runtime_1.jsx)("div", { className: "dvt-alert error", children: draftError }), state.error === undefined ? null : (0, jsx_runtime_1.jsx)("div", { className: "dvt-alert error", children: state.error }), state.message === 'saved' ? (0, jsx_runtime_1.jsx)("div", { className: "dvt-alert success", children: t('saved') }) : null, snapshot.runtime.lastError === undefined ? null : (0, jsx_runtime_1.jsxs)("div", { className: "dvt-alert error", children: [(0, jsx_runtime_1.jsx)("strong", { children: runtimeErrorTitle }), (0, jsx_runtime_1.jsx)("span", { children: snapshot.runtime.lastError })] }), (0, jsx_runtime_1.jsxs)("section", { className: "dvt-panel dvt-essential", children: [(0, jsx_runtime_1.jsxs)("div", { className: "dvt-panel-title", children: [(0, jsx_runtime_1.jsxs)("div", { children: [(0, jsx_runtime_1.jsx)("h3", { children: t('provider') }), (0, jsx_runtime_1.jsx)("p", { children: t('providerHint') })] }), (0, jsx_runtime_1.jsx)("span", { className: `dvt-badge ${snapshot.credential.configured ? 'ok' : 'error'}`, children: snapshot.credential.configured ? t('configured') : t('missing') })] }), (0, jsx_runtime_1.jsxs)("div", { className: "dvt-form-grid", children: [(0, jsx_runtime_1.jsx)(Field, { label: t('protocol'), children: (0, jsx_runtime_1.jsxs)("select", { disabled: !snapshot.writable || busy, value: draft.protocol, onChange: (event) => { update('protocol', event.target.value); }, children: [(0, jsx_runtime_1.jsx)("option", { value: "openai", children: "OpenAI Chat Completions" }), (0, jsx_runtime_1.jsx)("option", { value: "anthropic", children: "Anthropic Messages" })] }) }), (0, jsx_runtime_1.jsx)(Field, { label: t('baseUrl'), children: (0, jsx_runtime_1.jsx)(dsh_client_ui_primitives_1.Input, { disabled: !snapshot.writable || busy, value: draft.baseUrl, onChange: (event) => { update('baseUrl', event.target.value); } }) }), (0, jsx_runtime_1.jsx)(Field, { label: t('model'), children: (0, jsx_runtime_1.jsx)(dsh_client_ui_primitives_1.Input, { disabled: !snapshot.writable || busy, value: draft.model, onChange: (event) => { update('model', event.target.value); } }) }), (0, jsx_runtime_1.jsx)(Field, { label: t('apiKey'), hint: keyLocked ? t('apiKeyLocked') : snapshot.credential.source === undefined ? t('apiKeyHint') : `${t('apiKeyHint')} ${t('sourceHint', { source: t('source'), value: credentialSource(snapshot.credential.source, t) })}`, children: (0, jsx_runtime_1.jsx)(dsh_client_ui_primitives_1.Input, { "aria-label": t('apiKey'), type: "password", autoComplete: "new-password", disabled: busy || keyLocked, placeholder: snapshot.credential.configured ? t('apiKeyPlaceholderConfigured') : t('apiKeyPlaceholderMissing'), value: apiKey, onChange: (event) => { setApiKey(event.target.value); setDraftError(undefined); } }) })] })] }), (0, jsx_runtime_1.jsxs)("div", { className: "dvt-save-row", children: [(0, jsx_runtime_1.jsx)(dsh_client_ui_primitives_1.Button, { variant: "primary", disabled: !canSave || busy, onClick: save, children: state.action === 'save' ? t('saving') : t('save') }), (0, jsx_runtime_1.jsx)(dsh_client_ui_primitives_1.Button, { variant: "outline", disabled: busy, onClick: () => { void controller.load(); }, children: t('reload') })] }), (0, jsx_runtime_1.jsxs)("section", { className: "dvt-panel", children: [(0, jsx_runtime_1.jsxs)("div", { className: "dvt-panel-title", children: [(0, jsx_runtime_1.jsxs)("div", { children: [(0, jsx_runtime_1.jsx)("h3", { children: t('health') }), (0, jsx_runtime_1.jsx)("p", { children: t('connectionHint') })] }), (0, jsx_runtime_1.jsxs)("div", { className: "dvt-actions", children: [(0, jsx_runtime_1.jsx)(dsh_client_ui_primitives_1.Button, { size: "sm", variant: "outline", disabled: busy || !snapshot.runtime.ready, onClick: () => { void controller.runHealth('health'); }, children: state.action === 'health' ? t('testing') : t('runHealth') }), (0, jsx_runtime_1.jsx)(dsh_client_ui_primitives_1.Button, { size: "sm", variant: "outline", disabled: busy || !snapshot.runtime.ready, onClick: () => { void controller.runHealth('connection'); }, children: state.action === 'connection' ? t('testing') : t('testConnection') }), (0, jsx_runtime_1.jsx)(dsh_client_ui_primitives_1.Button, { size: "sm", variant: "primary", disabled: busy || !snapshot.runtime.ready, onClick: () => { void controller.runHealth('model'); }, children: state.action === 'model' ? t('testingModel') : t('testModel') })] })] }), (0, jsx_runtime_1.jsx)("p", { className: "dvt-muted", children: t('saveBeforeTesting') }), state.health === undefined ? (0, jsx_runtime_1.jsx)("p", { className: "dvt-muted", children: t('notTested') }) : (0, jsx_runtime_1.jsx)("div", { className: "dvt-health-grid", children: Object.entries(state.health.checks).map(([name, check]) => {
|