opencode-vision-analyze 0.1.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +73 -42
- package/README.zh.md +71 -42
- package/dist/index.d.ts +26 -0
- package/dist/index.js +333 -97
- package/package.json +4 -2
package/README.md
CHANGED
|
@@ -4,39 +4,20 @@
|
|
|
4
4
|
[](./LICENSE)
|
|
5
5
|
[](https://opencode.ai/docs/plugins)
|
|
6
6
|
|
|
7
|
+
English | [简体中文](./README.zh.md)
|
|
8
|
+
|
|
7
9
|
A tool-based vision routing plugin for [opencode](https://opencode.ai): when the main model can't see images, it calls the `vision_analyze` tool on demand — your dedicated vision model describes the image and the description flows straight back into the conversation. When the main model already supports images, pasted images pass through untouched and the tool short-circuits to return raw pixels.
|
|
8
10
|
|
|
9
11
|
**Zero runtime dependencies.** Only node builtins (`crypto`/`fs`/`path`) and type-only imports — nothing to install beyond the plugin itself.
|
|
10
12
|
|
|
11
|
-
##
|
|
12
|
-
|
|
13
|
-
There are already a few vision plugins in the ecosystem. The differences:
|
|
14
|
-
|
|
15
|
-
| | opencode-vision | opencode-vision-router | opencode-image-vision | **opencode-vision-analyze** |
|
|
16
|
-
|---|---|---|---|---|
|
|
17
|
-
| Mechanism | skill + subagent delegation | pointer + subagent delegation | direct SDK call (read-image / read-ocr) | **tool + plugin-managed sub-session** |
|
|
18
|
-
| Vision model source | auto-discovered image models | single `model` option | per-feature provider/model | single `model` option |
|
|
19
|
-
| Main-model capability detection | models.dev catalog + auth | `chat.params` live learning | name regex (fragile) | `config.providers()` capabilities (cached) |
|
|
20
|
-
| Image persistence | /tmp (session+part hash) | tmpDir | user dir / clipboard dir | `.opencode/vision/` content-addressed sha256 |
|
|
21
|
-
| Output | subagent answers itself | subagent answers itself | description / OCR text | description text (with cache) |
|
|
22
|
-
| Vision-capable main model | skip registration | skip routing (`force` to override) | skipModels / forceDescription | skip injection + **native fast path** (tool returns the raw image as attachment) |
|
|
23
|
-
| Request path | opencode session | opencode session | **direct third-party SDK** | opencode sub-session (unified auth, no extra credentials) |
|
|
24
|
-
| Failure visibility | via subagent tool chain | via subagent | tool output | tool output (never throws) |
|
|
25
|
-
|
|
26
|
-
Highlights:
|
|
13
|
+
## Features
|
|
27
14
|
|
|
28
15
|
- **Tool-based, not pre-analysis.** The turn starts immediately; the model decides when (and with which question) to look. No blocking on submit, failures are visible and retryable inside the agent loop. Same philosophy as production-proven agent designs.
|
|
29
16
|
- **Question-aware descriptions.** The model passes its own focused question to `vision_analyze` — not a one-shot generic caption computed at submit time.
|
|
30
17
|
- **Native fast path.** If the main model is vision-capable, `vision_analyze` skips the vision model entirely and returns the raw image as a tool attachment.
|
|
31
|
-
- **Content-addressed cache.** Images are stored as `<sha256>.<ext>` (deduped across sessions); descriptions are cached per `<image-hash>:<question>` — the same image with the same question is described exactly once.
|
|
18
|
+
- **Content-addressed cache.** Images are stored as content-addressed `<sha256>.<ext>` files (deduped across sessions within the same store); descriptions are cached per `<image-hash>:<question>` — the same image with the same question is described exactly once.
|
|
32
19
|
- **Unified auth.** The vision call runs through an opencode sub-session, so it reuses the provider credentials opencode already manages. No extra API key plumbing.
|
|
33
20
|
|
|
34
|
-
## Requirements
|
|
35
|
-
|
|
36
|
-
- [opencode](https://opencode.ai) (plugins are loaded with Bun; npm plugins are installed automatically at startup)
|
|
37
|
-
- a vision model you have access to, referenced as `provider/model` (e.g. `"openai/gpt-4o-mini"`, `"anthropic/claude-sonnet-4-5"`)
|
|
38
|
-
- supported image extensions: png / jpg / jpeg / gif / webp
|
|
39
|
-
|
|
40
21
|
## Installation
|
|
41
22
|
|
|
42
23
|
### Option A — npm (recommended)
|
|
@@ -45,7 +26,7 @@ Highlights:
|
|
|
45
26
|
// opencode.json (project or global)
|
|
46
27
|
{
|
|
47
28
|
"plugin": [
|
|
48
|
-
["opencode-vision-analyze", { "
|
|
29
|
+
["opencode-vision-analyze", { "models": ["openai/gpt-4o-mini"] }]
|
|
49
30
|
]
|
|
50
31
|
}
|
|
51
32
|
```
|
|
@@ -66,14 +47,14 @@ curl -fsSL https://raw.githubusercontent.com/MwumLi/opencode-vision-analyze/main
|
|
|
66
47
|
// opencode.json
|
|
67
48
|
{
|
|
68
49
|
"plugin": [
|
|
69
|
-
["./.opencode/vision-analyze.ts", { "
|
|
50
|
+
["./.opencode/vision-analyze.ts", { "models": ["openai/gpt-4o-mini"] }]
|
|
70
51
|
]
|
|
71
52
|
}
|
|
72
53
|
```
|
|
73
54
|
|
|
74
55
|
Notes for the curl path:
|
|
75
56
|
|
|
76
|
-
-
|
|
57
|
+
- The URL above points at the `main` branch (latest source); for a pinned release, swap `main` for a release tag (e.g. `v0.1.0`) and re-run curl.
|
|
77
58
|
- The file is TypeScript source — opencode loads plugins with Bun, so this works as-is.
|
|
78
59
|
- Options must be passed via the `plugin` tuple (the auto-discovered `.opencode/plugins/` directory can't carry options).
|
|
79
60
|
|
|
@@ -81,16 +62,44 @@ Notes for the curl path:
|
|
|
81
62
|
|
|
82
63
|
| Option | Required | Default | Description |
|
|
83
64
|
|---|---|---|---|
|
|
84
|
-
| `
|
|
85
|
-
| `
|
|
65
|
+
| `models` | no | — | Ordered candidate list of vision models (`provider/model`), tried one after another until one succeeds. A single vision model is written as `models: ["..."]`. When omitted (or an empty array) the plugin auto-discovers all image-capable models. |
|
|
66
|
+
| `unlisted_fallback` | no | `false` | When an explicit `models` chain is configured and it is exhausted, keep going with image-capable models that were not listed. |
|
|
67
|
+
| `free_first` | no | `false` | In auto-discovery, prefer anonymous/built-in free providers (`custom` source) ahead of config-defined ones — reverses the source-tier order. |
|
|
68
|
+
| `timeout_ms` | no | `60000` | Timeout (ms) budget for each individual `create`/`prompt` request inside the vision sub-session |
|
|
69
|
+
|
|
70
|
+
Supported image extensions: png / jpg / jpeg / gif / webp.
|
|
71
|
+
|
|
72
|
+
Image storage: inside a git project images live under `<project>/.opencode/vision`; in non-git directories they go to the user cache dir (`<cache>/opencode-vision-analyze/vision`) — mirroring opencode's own project/global session scoping. Defaults per platform: Linux `$XDG_CACHE_HOME || ~/.cache`, macOS `~/Library/Caches` (a `$XDG_CACHE_HOME` override is honored), Windows `%LOCALAPPDATA% || ~/AppData/Local`. An empty cache-root env var is treated as unset (falls back to the default). In git projects, add `.opencode/vision/` to your `.gitignore` if you don't want the cache tracked. The storage root is re-resolved on every write, so switching scope (e.g. after `git init`) takes effect on the next pasted image — though old session hints keep pointing at stale absolute paths until you re-paste.
|
|
73
|
+
|
|
74
|
+
An ordered-candidates example with auto-fallback and free-first discovery:
|
|
75
|
+
|
|
76
|
+
```jsonc
|
|
77
|
+
// opencode.json
|
|
78
|
+
{
|
|
79
|
+
"plugin": [
|
|
80
|
+
[
|
|
81
|
+
"opencode-vision-analyze",
|
|
82
|
+
{
|
|
83
|
+
"models": ["anthropic/claude-sonnet-4-5", "openai/gpt-4o-mini"],
|
|
84
|
+
"unlisted_fallback": true,
|
|
85
|
+
"free_first": true
|
|
86
|
+
}
|
|
87
|
+
]
|
|
88
|
+
]
|
|
89
|
+
}
|
|
90
|
+
```
|
|
86
91
|
|
|
87
92
|
## How it works
|
|
88
93
|
|
|
89
94
|
```
|
|
90
95
|
User pastes image + question
|
|
91
96
|
└─ chat.message hook (before persist)
|
|
97
|
+
├─ message model ∈ candidate chain → do nothing (recursion guard)
|
|
92
98
|
├─ main model has image input capability → do nothing (raw image goes to model)
|
|
93
|
-
|
|
99
|
+
├─ no image-capable model available → do nothing (no hint, no persist;
|
|
100
|
+
│ core's default image handling applies)
|
|
101
|
+
└─ text-only main model → persist image to the vision store
|
|
102
|
+
(<sha256>.<ext>; project-local inside a git repo, user cache otherwise)
|
|
94
103
|
and inject a synthetic hint (hidden in TUI, visible to model):
|
|
95
104
|
"use the vision_analyze tool with image_path: ..."
|
|
96
105
|
|
|
@@ -103,34 +112,31 @@ vision_analyze tool:
|
|
|
103
112
|
│ → return raw image as attachment (no vision model call)
|
|
104
113
|
├─ http(s) image URL → download (20 MB cap) → same disk path
|
|
105
114
|
├─ description cache hit (sha + question) → return cached text
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
115
|
+
│ (tagged with the model that produced it)
|
|
116
|
+
└─ candidate chain: sub-session under current session per candidate, in order —
|
|
117
|
+
parentID, all tools disabled, dedicated system prompt, image + question
|
|
118
|
+
sent to that vision model → first success returns → sub-session deleted
|
|
109
119
|
```
|
|
110
120
|
|
|
111
121
|
Key behaviors:
|
|
112
122
|
|
|
113
123
|
- **Capability gating** — queries `config.providers()` capabilities; results cached per process. A vision-capable main model never gets hints or routing.
|
|
114
|
-
- **
|
|
124
|
+
- **Candidate chain** — the `models` list is tried in order until one succeeds. Explicit models always head the chain. With no explicit config (or an empty `models` list) the plugin auto-discovers every image-capable model, ordered by provider source (config first, then env/api, then custom/anonymous; reversed with `free_first: true`). With `unlisted_fallback: true`, an exhausted explicit chain continues onto unlisted image-capable models.
|
|
125
|
+
- **Recursion guard (whole chain)** — messages from the candidate chain's own sub-sessions are never re-processed.
|
|
126
|
+
- **Empty chain degradation** — if no vision model is available at all, the plugin still loads: pasted images are left untouched (no hint injected) and the tool returns a clear error instead of routing.
|
|
127
|
+
- **Loginless free models** — auto-discovery uses `config.providers()`, the same source as the `/models` picker, so image-capable zen free models are found even without login (their provider is `custom` source → default last tier; put them first with `free_first: true`).
|
|
115
128
|
- **The tool never throws** — every failure returns readable text so the agent loop can retry, rephrase, or inform the user.
|
|
116
129
|
- **URL images** — `image_path` accepts `http(s)://...` URLs (must end in a supported image extension: png/jpg/jpeg/gif/webp).
|
|
117
130
|
|
|
118
131
|
## Known limitations
|
|
119
132
|
|
|
120
133
|
- **V1 session flow only** — hooks are attached to the V1 `SessionPrompt` path; if opencode's default interaction moves to the V2 session core, hooks won't fire (silently).
|
|
121
|
-
- **SSRF surface** — URL downloads follow redirects and don't block private-range / cloud-metadata addresses. Acceptable for a local single-user CLI; add address filtering before using in multi-tenant environments.
|
|
122
|
-
- **Abort doesn't propagate** — user aborts don't cancel in-flight downloads/sub-session requests; they run to their own deadlines (30s download, `timeout_ms` sub-session). After timeout/abort the sub-session is deleted, but the orphan turn may still be billed by the provider.
|
|
123
134
|
- **Historical images** — images from messages sent before the plugin was enabled can't be described (no hint, no path on disk).
|
|
124
|
-
- **Unbounded caches** — both the image store and description cache grow without eviction (
|
|
125
|
-
- **Single model, no fallback chain** — one explicit `model` option; if it fails, the tool returns an error message instead of trying other providers.
|
|
135
|
+
- **Unbounded caches** — both the image store and the description cache grow without eviction (image store: git-project-scoped or user-cache-scoped; description cache: per process).
|
|
126
136
|
|
|
127
137
|
## Roadmap
|
|
128
138
|
|
|
129
|
-
- [ ]
|
|
130
|
-
- [ ] Timeout wrapping for the capability query (`config.providers()`)
|
|
131
|
-
- [ ] Abort sub-session (`/session/{id}/abort`) before delete on timeout
|
|
132
|
-
- [ ] LRU / size cap for the description cache
|
|
133
|
-
- [ ] Optional private-address blocking for URL downloads
|
|
139
|
+
- [ ] Persistent description cache (content-addressed on disk, with LRU / size cap)
|
|
134
140
|
- [ ] Region cropping for zooming into image details
|
|
135
141
|
|
|
136
142
|
## Development
|
|
@@ -144,6 +150,31 @@ bun run build # tsc → dist/
|
|
|
144
150
|
|
|
145
151
|
The unit tests stub the plugin input/client — no running opencode instance is required.
|
|
146
152
|
|
|
153
|
+
## Release
|
|
154
|
+
|
|
155
|
+
Versioning is driven entirely by `npm version` — no manual `package.json` edits. It updates the version, creates a commit and an annotated `v<version>` tag, and (via hooks) runs a local gate then pushes to trigger the GitHub release workflow that publishes to npm.
|
|
156
|
+
|
|
157
|
+
```bash
|
|
158
|
+
npm version patch # 0.1.x → 0.1.(x+1): commit + tag v0.1.x, auto-push → release
|
|
159
|
+
npm version 1.2.0 # explicit full version
|
|
160
|
+
npm version prerelease --preid beta # beta smoke: 0.1.1 → 0.1.2-beta.0
|
|
161
|
+
```
|
|
162
|
+
|
|
163
|
+
Hooks configured in `package.json`:
|
|
164
|
+
|
|
165
|
+
- `preversion` — runs `typecheck && test && build` locally; if any fails the version is not bumped or tagged.
|
|
166
|
+
- `postversion` — `git push --follow-tags`; pushes the commit and its tag, which triggers the GitHub Actions `release.yml` (`on.push.tags: ["v*"]`) that runs the checks again and `npm publish --access public` using the `NPM_TOKEN` secret.
|
|
167
|
+
|
|
168
|
+
Beta smoke → stable flow:
|
|
169
|
+
|
|
170
|
+
```bash
|
|
171
|
+
npm version prerelease --preid beta # publish a beta to npm
|
|
172
|
+
# verify the beta on npm, then:
|
|
173
|
+
npm version patch # drops the pre-release and bumps to the stable version
|
|
174
|
+
```
|
|
175
|
+
|
|
176
|
+
Escape hatches: `npm version 1.2.3 --no-git-tag-version` (only bump the file) or `--ignore-scripts` (skip all hooks). `npm version` requires a clean working tree. If the `postversion` push fails, run `git push --follow-tags` manually.
|
|
177
|
+
|
|
147
178
|
## License
|
|
148
179
|
|
|
149
180
|
[MIT](./LICENSE)
|
package/README.zh.md
CHANGED
|
@@ -10,35 +10,14 @@
|
|
|
10
10
|
|
|
11
11
|
**零运行时依赖。** 只用 node 内置模块(`crypto`/`fs`/`path`)和纯类型导入——除插件本身外无需安装任何东西。
|
|
12
12
|
|
|
13
|
-
##
|
|
14
|
-
|
|
15
|
-
生态里已有若干视觉插件,差异如下:
|
|
16
|
-
|
|
17
|
-
| | opencode-vision | opencode-vision-router | opencode-image-vision | **opencode-vision-analyze** |
|
|
18
|
-
|---|---|---|---|---|
|
|
19
|
-
| 机制 | skill + 子代理委托 | 指针 + 子代理委托 | 直连 SDK(read-image / read-ocr) | **工具 + 插件自管子会话** |
|
|
20
|
-
| 视觉模型来源 | 自动发现的视觉模型 | 单一 `model` 选项 | 每功能独立 provider/model | 单一 `model` 选项 |
|
|
21
|
-
| 主模型能力判定 | models.dev 目录 + auth | `chat.params` 实时学习 | 名字正则(脆弱) | `config.providers()` 能力查询(缓存) |
|
|
22
|
-
| 图片落盘 | /tmp(会话+part 哈希) | tmpDir | 用户目录 / 剪贴板目录 | `.opencode/vision/` 内容寻址 sha256 |
|
|
23
|
-
| 产出 | 子代理自行作答 | 子代理自行作答 | 描述 / OCR 文本 | 描述文本(带缓存) |
|
|
24
|
-
| 有视觉主模型 | 跳过注册 | 跳过路由(`force` 可强制) | skipModels / forceDescription | 跳过注入 + **原生快速路径**(工具直接回传原图附件) |
|
|
25
|
-
| 请求路径 | opencode 会话 | opencode 会话 | **第三方 SDK 直连** | opencode 子会话(统一鉴权,无需额外密钥) |
|
|
26
|
-
| 失败可见性 | 经子代理工具链 | 经子代理 | 工具输出 | 工具输出(永不抛错) |
|
|
27
|
-
|
|
28
|
-
亮点:
|
|
13
|
+
## 特性
|
|
29
14
|
|
|
30
15
|
- **工具化,而非提交时预分析。** 轮次即时启动;模型自己决定何时看图、带着什么问题看。提交零阻塞,失败在 agent 循环里可见、可重试。
|
|
31
16
|
- **描述针对问题。** 模型把自己关注的问题传给 `vision_analyze`——而不是提交时预生成的一次性通用描述。
|
|
32
17
|
- **原生快速路径。** 主模型本身有视觉能力时,`vision_analyze` 完全跳过视觉模型,直接把原图作为工具附件返回。
|
|
33
|
-
- **内容寻址缓存。**
|
|
18
|
+
- **内容寻址缓存。** 图片按内容寻址 `<sha256>.<ext>` 落盘(跨会话、同一存储域内天然去重);描述按 `<图片哈希>:<问题>` 缓存——同图同问题只描述一次。
|
|
34
19
|
- **统一鉴权。** 视觉调用走 opencode 子会话,复用 opencode 已管理的 provider 凭据,无需额外配置 API Key。
|
|
35
20
|
|
|
36
|
-
## 运行前提
|
|
37
|
-
|
|
38
|
-
- [opencode](https://opencode.ai)(插件由 Bun 加载;npm 插件启动时自动安装)
|
|
39
|
-
- 一个你可用的视觉模型,以 `provider/model` 引用(如 `"openai/gpt-4o-mini"`、`"anthropic/claude-sonnet-4-5"`)
|
|
40
|
-
- 受支持的图片扩展名:png / jpg / jpeg / gif / webp
|
|
41
|
-
|
|
42
21
|
## 安装
|
|
43
22
|
|
|
44
23
|
### 方式 A —— npm(推荐)
|
|
@@ -47,7 +26,7 @@
|
|
|
47
26
|
// opencode.json(项目级或全局)
|
|
48
27
|
{
|
|
49
28
|
"plugin": [
|
|
50
|
-
["opencode-vision-analyze", { "
|
|
29
|
+
["opencode-vision-analyze", { "models": ["openai/gpt-4o-mini"] }]
|
|
51
30
|
]
|
|
52
31
|
}
|
|
53
32
|
```
|
|
@@ -68,14 +47,14 @@ curl -fsSL https://raw.githubusercontent.com/MwumLi/opencode-vision-analyze/main
|
|
|
68
47
|
// opencode.json
|
|
69
48
|
{
|
|
70
49
|
"plugin": [
|
|
71
|
-
["./.opencode/vision-analyze.ts", { "
|
|
50
|
+
["./.opencode/vision-analyze.ts", { "models": ["openai/gpt-4o-mini"] }]
|
|
72
51
|
]
|
|
73
52
|
}
|
|
74
53
|
```
|
|
75
54
|
|
|
76
55
|
curl 方式说明:
|
|
77
56
|
|
|
78
|
-
-
|
|
57
|
+
- 上方 URL 指向 `main` 分支(最新源码);如需固定版本,把 `main` 换成发布 tag(如 `v0.1.0`)再重新 curl。
|
|
79
58
|
- 文件是 TypeScript 源码——opencode 用 Bun 加载插件,直接可用。
|
|
80
59
|
- 选项必须通过 `plugin` 元组传入(`.opencode/plugins/` 自动发现目录无法携带选项)。
|
|
81
60
|
|
|
@@ -83,16 +62,44 @@ curl 方式说明:
|
|
|
83
62
|
|
|
84
63
|
| 选项 | 必填 | 默认值 | 说明 |
|
|
85
64
|
|---|---|---|---|
|
|
86
|
-
| `
|
|
87
|
-
| `
|
|
65
|
+
| `models` | 否 | — | 有序候选视觉模型数组(`provider/model`),逐个尝试直到成功即止;单个视觉模型写作 `models: ["..."]`。缺省或空数组时自动发现全部 image-capable 模型。 |
|
|
66
|
+
| `unlisted_fallback` | 否 | `false` | 显式 `models` 链耗尽后,自动续试未列入清单的 image-capable 模型。 |
|
|
67
|
+
| `free_first` | 否 | `false` | 自动发现时优先匿名/内置免费(`custom` 源)provider,置于 config 源之前——反转 source 档序。 |
|
|
68
|
+
| `timeout_ms` | 否 | `60000` | 子会话内每次 create/prompt 请求各自的超时预算(毫秒) |
|
|
69
|
+
|
|
70
|
+
受支持的图片扩展名:png / jpg / jpeg / gif / webp。
|
|
71
|
+
|
|
72
|
+
图片存储:git 项目内图片放在 `<项目>/.opencode/vision`;非 git 目录则放入用户级缓存目录(`<cache>/opencode-vision-analyze/vision`)——与 opencode 自身的项目/全局会话分域一致。各平台默认:Linux `$XDG_CACHE_HOME || ~/.cache`;macOS `~/Library/Caches`(亦接受 `$XDG_CACHE_HOME` 覆盖);Windows `%LOCALAPPDATA% || ~/AppData/Local`。缓存根 env 为空串视为未设置(回退默认)。git 项目中如不想跟踪缓存图片,请把 `.opencode/vision/` 加入 `.gitignore`。存储根在每次落盘时现算:切换存储范围(如执行 `git init`)后,从下一条贴图起即写入新域;旧会话 hint 里的绝对路径仍指向旧处,重新贴图即注入新 hint。
|
|
73
|
+
|
|
74
|
+
有序候选 + 自动续接 + 免费优先的配置示例:
|
|
75
|
+
|
|
76
|
+
```jsonc
|
|
77
|
+
// opencode.json
|
|
78
|
+
{
|
|
79
|
+
"plugin": [
|
|
80
|
+
[
|
|
81
|
+
"opencode-vision-analyze",
|
|
82
|
+
{
|
|
83
|
+
"models": ["anthropic/claude-sonnet-4-5", "openai/gpt-4o-mini"],
|
|
84
|
+
"unlisted_fallback": true,
|
|
85
|
+
"free_first": true
|
|
86
|
+
}
|
|
87
|
+
]
|
|
88
|
+
]
|
|
89
|
+
}
|
|
90
|
+
```
|
|
88
91
|
|
|
89
92
|
## 工作原理
|
|
90
93
|
|
|
91
94
|
```
|
|
92
95
|
用户贴图 + 提问
|
|
93
96
|
└─ chat.message 钩子(消息持久化前)
|
|
97
|
+
├─ 消息模型 ∈ 候选链 → 不做任何处理(递归防护)
|
|
94
98
|
├─ 主模型支持图片输入 → 不做任何处理(原图直发)
|
|
95
|
-
|
|
99
|
+
├─ 无任何 image-capable 模型 → 不做处理(不注入 hint、不落盘;
|
|
100
|
+
│ 交给核心对图片的默认处理)
|
|
101
|
+
└─ 纯文本主模型 → 图片落盘到 vision 存储
|
|
102
|
+
(<sha256>.<ext>;git 项目内 → 项目目录,非 git → 用户缓存目录)
|
|
96
103
|
并注入 synthetic 提示(TUI 隐藏、模型可见):
|
|
97
104
|
"用 vision_analyze 工具查看,image_path: ..."
|
|
98
105
|
|
|
@@ -105,34 +112,31 @@ vision_analyze 工具:
|
|
|
105
112
|
│ → 原图作为附件直接返回(不调视觉模型)
|
|
106
113
|
├─ http(s) 图片 URL → 下载(20 MB 上限)→ 统一磁盘路径
|
|
107
114
|
├─ 描述缓存命中(图片哈希 + 问题)→ 直接返回缓存文本
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
115
|
+
│ (标签沿用产出该描述的模型)
|
|
116
|
+
└─ 候选链:沿链逐候选建子会话(parentID 挂当前会话、禁用全部工具、
|
|
117
|
+
专用 system prompt,图片 + 问题发给该候选视觉模型)
|
|
118
|
+
→ 首个成功即返回 → 子会话删除
|
|
111
119
|
```
|
|
112
120
|
|
|
113
121
|
关键行为:
|
|
114
122
|
|
|
115
123
|
- **能力门控** —— 查询 `config.providers()` 能力字段,进程级缓存;有视觉能力的主模型永远不会收到提示或被路由。
|
|
116
|
-
-
|
|
124
|
+
- **候选链** —— `models` 列表按序逐个尝试直到成功。显式模型恒在链首;无显式配置(或 `models` 为空数组)时自动发现全部 image-capable 模型,按 provider 来源排序(config 最前 → env/api → custom/匿名;`free_first: true` 时反转)。`unlisted_fallback: true` 时显式链耗尽后会续试未列出的 image-capable 模型。
|
|
125
|
+
- **递归防护(整链)** —— 候选链子会话发起的消息不会被再次处理。
|
|
126
|
+
- **空链降级** —— 完全没有可用视觉模型时插件仍正常加载:贴图保持原样(不注入 hint),工具返回清晰错误而非路由。
|
|
127
|
+
- **免登录免费模型** —— 自动发现与 `/models` 选择器同源(`config.providers()`),免登录也可发现的 zen free 视觉模型会进入候选链(其 provider 为 `custom` 源 → 默认最末档;`free_first: true` 可提到最前)。
|
|
117
128
|
- **工具永不抛错** —— 所有失败都返回可读文字,agent 循环可以重试、换问题或告知用户。
|
|
118
129
|
- **URL 图片** —— `image_path` 接受 `http(s)://...` 地址(需以受支持的图片扩展名结尾:png/jpg/jpeg/gif/webp)。
|
|
119
130
|
|
|
120
131
|
## 已知限制
|
|
121
132
|
|
|
122
133
|
- **仅 V1 会话流** —— 钩子挂在 V1 `SessionPrompt` 路径上;若 opencode 默认交互切到 V2 会话核心,钩子不会触发(且不报错)。
|
|
123
|
-
- **SSRF 面** —— URL 下载跟随重定向、不拦截私网/云元数据地址。本地单用户 CLI 信任级别下可接受;多租户环境使用前应加地址过滤。
|
|
124
|
-
- **中止不传导** —— 用户中止不会取消进行中的下载/子会话请求,它们会跑到各自的 deadline(下载 30 秒、子会话 `timeout_ms`)。超时/中止后子会话虽被删除,但 provider 端的孤儿回合仍可能计费。
|
|
125
134
|
- **历史图片** —— 插件启用之前发送的图片无法被描述(无提示、磁盘上无路径)。
|
|
126
|
-
- **缓存无上限** ——
|
|
127
|
-
- **单模型无备选链** —— 只有一个显式 `model` 选项;失败时返回错误文字,不会尝试其他 provider。
|
|
135
|
+
- **缓存无上限** —— 图片存储与描述缓存均不淘汰(图片存储:git 项目级 / 用户级缓存目录;描述缓存:进程级)。
|
|
128
136
|
|
|
129
137
|
## Roadmap
|
|
130
138
|
|
|
131
|
-
- [ ]
|
|
132
|
-
- [ ] 能力查询(`config.providers()`)加超时保护
|
|
133
|
-
- [ ] 超时路径先中止子会话(`/session/{id}/abort`)再删除
|
|
134
|
-
- [ ] 描述缓存 LRU / 容量上限
|
|
135
|
-
- [ ] URL 下载可选私网地址拦截
|
|
139
|
+
- [ ] 描述缓存按内容 sha 落盘持久化(含 LRU / 容量上限)
|
|
136
140
|
- [ ] 区域裁剪(放大查看图片细节)
|
|
137
141
|
|
|
138
142
|
## 开发
|
|
@@ -146,6 +150,31 @@ bun run build # tsc → dist/
|
|
|
146
150
|
|
|
147
151
|
单元测试使用 stub 的插件输入/client——不需要运行中的 opencode。
|
|
148
152
|
|
|
153
|
+
## 发布
|
|
154
|
+
|
|
155
|
+
版本变更完全由 `npm version` 驱动——无需手改 `package.json`。它会更新版本号、创建 commit 与 annotated `v<版本>` tag,并通过钩子先跑本地门禁再自动推送,触发 GitHub release workflow 发布到 npm。
|
|
156
|
+
|
|
157
|
+
```bash
|
|
158
|
+
npm version patch # 0.1.x → 0.1.(x+1):commit + tag v0.1.x,自动推送 → 发布
|
|
159
|
+
npm version 1.2.0 # 显式指定完整版本
|
|
160
|
+
npm version prerelease --preid beta # beta 冒烟:0.1.1 → 0.1.2-beta.0
|
|
161
|
+
```
|
|
162
|
+
|
|
163
|
+
`package.json` 中配置的钩子:
|
|
164
|
+
|
|
165
|
+
- `preversion` —— 本地执行 `typecheck && test && build`;任一失败则不 bump / 不打 tag。
|
|
166
|
+
- `postversion` —— `git push --follow-tags`;推送 commit 及其 tag,触发 GitHub Actions `release.yml`(`on.push.tags: ["v*"]`),先复跑全部检查再用 `NPM_TOKEN` secret 执行 `npm publish --access public`。
|
|
167
|
+
|
|
168
|
+
beta 冒烟 → 正式两段式:
|
|
169
|
+
|
|
170
|
+
```bash
|
|
171
|
+
npm version prerelease --preid beta # 先发一个 beta 到 npm
|
|
172
|
+
# 在 npm 上验证 beta 无误后:
|
|
173
|
+
npm version patch # 去掉 pre 段并升到正式版本
|
|
174
|
+
```
|
|
175
|
+
|
|
176
|
+
逃逸舱:`npm version 1.2.3 --no-git-tag-version`(只改版本文件)或 `--ignore-scripts`(跳过全部钩子)。`npm version` 要求工作区干净;若 `postversion` 推送失败,手动执行 `git push --follow-tags`。
|
|
177
|
+
|
|
149
178
|
## 许可证
|
|
150
179
|
|
|
151
180
|
[MIT](./LICENSE)
|
package/dist/index.d.ts
CHANGED
|
@@ -1,4 +1,30 @@
|
|
|
1
1
|
import type { Plugin } from "@opencode-ai/plugin";
|
|
2
|
+
/**
|
|
3
|
+
* `config.providers()` 能力查询的超时预算(毫秒)。做成可改写对象(而非常量/选项):
|
|
4
|
+
* 避免选项膨胀;测试把 ms 调小即可缩短等待(TS 不允许对 import 的 let 绑定赋值)。
|
|
5
|
+
*/
|
|
6
|
+
export declare const providersTimeout: {
|
|
7
|
+
ms: number;
|
|
8
|
+
};
|
|
9
|
+
/**
|
|
10
|
+
* 判断目录是否位于 git 项目内:从 dir 向上(含自身)逐级找 `.git`
|
|
11
|
+
* (目录,或 worktree/submodule 的 `.git` 指针文件),到文件系统根为止。
|
|
12
|
+
* 纯同步、纯内置模块;作为命名导出便于单元测试注入真实临时目录验证。
|
|
13
|
+
*/
|
|
14
|
+
export declare function isInsideGitRepo(dir: string): boolean;
|
|
15
|
+
/**
|
|
16
|
+
* 用户级(非 git)图片缓存的平台根:cache 目录 + opencode-vision-analyze/vision。
|
|
17
|
+
* 空字符串 env 视为未设置(XDG/LOCALAPPDATA 官规:空值=未设置)——避免把 "" 当
|
|
18
|
+
* 有效根导致 path.join("",…) 产出相对路径、相对进程 cwd 落盘。
|
|
19
|
+
* darwin 亦接受 $XDG_CACHE_HOME 覆盖(跨平台统一 dotfiles 的宽容超集)。
|
|
20
|
+
*/
|
|
21
|
+
export declare function userVisionCacheRoot(env: Record<string, string | undefined>, platform: string, home: string): string;
|
|
22
|
+
/**
|
|
23
|
+
* 图片存储根:与 opencode 的项目/全局语义对齐——
|
|
24
|
+
* git 项目内 → <项目>/.opencode/vision(现状);非 git 目录 → 用户级缓存目录。
|
|
25
|
+
* 解析一次即可(纯函数,入参可注入以便三平台与 env 覆盖的单测)。
|
|
26
|
+
*/
|
|
27
|
+
export declare function resolveVisionDir(inputDir: string, env: Record<string, string | undefined>, platform: string, home: string): string;
|
|
2
28
|
declare const _default: {
|
|
3
29
|
id: string;
|
|
4
30
|
server: Plugin;
|
package/dist/index.js
CHANGED
|
@@ -2,26 +2,10 @@
|
|
|
2
2
|
* opencode-vision-analyze
|
|
3
3
|
*
|
|
4
4
|
* 为「不具备视觉能力的主模型」提供图片解读路由:当用户在消息中附带图片时,
|
|
5
|
-
* 插件把图片落盘到
|
|
6
|
-
*
|
|
7
|
-
*
|
|
8
|
-
*
|
|
9
|
-
*
|
|
10
|
-
* 安装方式一(npm):
|
|
11
|
-
* {
|
|
12
|
-
* "plugin": [["opencode-vision-analyze", { "model": "provider/vision-model" }]]
|
|
13
|
-
* }
|
|
14
|
-
*
|
|
15
|
-
* 安装方式二(curl 下载单文件,免 npm):
|
|
16
|
-
* mkdir -p .opencode
|
|
17
|
-
* curl -fsSL <raw-url>/src/index.ts -o .opencode/vision-analyze.ts
|
|
18
|
-
* {
|
|
19
|
-
* "plugin": [["./.opencode/vision-analyze.ts", { "model": "provider/vision-model" }]]
|
|
20
|
-
* }
|
|
21
|
-
*
|
|
22
|
-
* 选项:
|
|
23
|
-
* - model(必填):视觉模型的 "provider/model" 标识,例如 "anthropic/claude-sonnet-4-5"
|
|
24
|
-
* - timeout_ms:子会话请求的超时毫秒数(正数,默认 60000)
|
|
5
|
+
* 插件把图片落盘到 vision 目录(git 项目内 → <项目>/.opencode/vision;
|
|
6
|
+
* 非 git 目录 → 用户级缓存目录),并向模型注入一条 synthetic 提示
|
|
7
|
+
* (TUI 界面隐藏、模型可见),引导它通过 vision_analyze 工具让指定的视觉
|
|
8
|
+
* 模型描述图片。若主模型本身支持图片输入,则不做任何干预,原图直接发给主模型。
|
|
25
9
|
*
|
|
26
10
|
* 工作方式(vision_analyze 工具路径):主模型调用 vision_analyze 时,插件
|
|
27
11
|
* 创建一个 parentID 挂在当前会话下的临时子会话(不进会话列表、不生成
|
|
@@ -36,17 +20,13 @@
|
|
|
36
20
|
* 类型依赖仅 @opencode-ai/plugin 与 @opencode-ai/sdk 的 type import。
|
|
37
21
|
*
|
|
38
22
|
* 已知限制:
|
|
39
|
-
* - SSRF 面:downloadImage 的 fetch 跟随重定向、不拦截私网/云元数据地址。
|
|
40
|
-
* 本地单用户 CLI 的信任级别下可接受;生产多租户环境使用前应加私网
|
|
41
|
-
* 地址拦截。
|
|
42
|
-
* - 中止不传导:用户中止不会取消进行中的下载/子会话请求,最长空跑至
|
|
43
|
-
* 各自的 deadline(下载 30 秒、子会话 timeout_ms);超时/中止后子会话
|
|
44
|
-
* 虽被删除,但 provider 端已发出的孤儿回合仍可能计入用量。
|
|
45
23
|
* - 仅 V1 会话流有效:chat.message 钩子挂在 V1 SessionPrompt 路径上;
|
|
46
24
|
* 若交互默认切到 V2 Session 核心,本钩子不会触发(也不会报错)。
|
|
47
25
|
*/
|
|
48
26
|
import { createHash, randomUUID } from "node:crypto";
|
|
27
|
+
import { existsSync } from "node:fs";
|
|
49
28
|
import fs from "node:fs/promises";
|
|
29
|
+
import { homedir } from "node:os";
|
|
50
30
|
import path from "node:path";
|
|
51
31
|
/** 支持的图片扩展名 → MIME 类型(vision_analyze 加载磁盘图片时使用) */
|
|
52
32
|
const EXT_MIME = {
|
|
@@ -63,6 +43,22 @@ const MIME_EXT = {
|
|
|
63
43
|
"image/gif": ".gif",
|
|
64
44
|
"image/webp": ".webp",
|
|
65
45
|
};
|
|
46
|
+
/**
|
|
47
|
+
* 内部超时错误类型(name = "DeadlineError")。
|
|
48
|
+
* 与 AbortError 并列可判别:超时路径(providers 查询 / 子会话请求)据此判断
|
|
49
|
+
* "底层请求可能仍在飞",供调用方决定是否需要 abort 取消(见 attemptModel)。
|
|
50
|
+
*/
|
|
51
|
+
class DeadlineError extends Error {
|
|
52
|
+
constructor(message) {
|
|
53
|
+
super(message);
|
|
54
|
+
this.name = "DeadlineError";
|
|
55
|
+
}
|
|
56
|
+
}
|
|
57
|
+
/**
|
|
58
|
+
* `config.providers()` 能力查询的超时预算(毫秒)。做成可改写对象(而非常量/选项):
|
|
59
|
+
* 避免选项膨胀;测试把 ms 调小即可缩短等待(TS 不允许对 import 的 let 绑定赋值)。
|
|
60
|
+
*/
|
|
61
|
+
export const providersTimeout = { ms: 5000 };
|
|
66
62
|
/**
|
|
67
63
|
* 视觉子会话使用的系统提示词。
|
|
68
64
|
* 要求:精确转录图中文字,描述 UI/布局/对象/颜色等,优先回答用户问题,
|
|
@@ -77,6 +73,46 @@ const VISION_SYSTEM_PROMPT = [
|
|
|
77
73
|
].join("\n");
|
|
78
74
|
/** data URL 形如 data:<mime>;base64,<payload> */
|
|
79
75
|
const DATA_URL_PATTERN = /^data:([^;]+);base64,(.+)$/;
|
|
76
|
+
/**
|
|
77
|
+
* 判断目录是否位于 git 项目内:从 dir 向上(含自身)逐级找 `.git`
|
|
78
|
+
* (目录,或 worktree/submodule 的 `.git` 指针文件),到文件系统根为止。
|
|
79
|
+
* 纯同步、纯内置模块;作为命名导出便于单元测试注入真实临时目录验证。
|
|
80
|
+
*/
|
|
81
|
+
export function isInsideGitRepo(dir) {
|
|
82
|
+
let cur = path.resolve(dir);
|
|
83
|
+
for (;;) {
|
|
84
|
+
if (existsSync(path.join(cur, ".git")))
|
|
85
|
+
return true;
|
|
86
|
+
const parent = path.dirname(cur);
|
|
87
|
+
if (parent === cur)
|
|
88
|
+
return false; // 已到文件系统根
|
|
89
|
+
cur = parent;
|
|
90
|
+
}
|
|
91
|
+
}
|
|
92
|
+
/**
|
|
93
|
+
* 用户级(非 git)图片缓存的平台根:cache 目录 + opencode-vision-analyze/vision。
|
|
94
|
+
* 空字符串 env 视为未设置(XDG/LOCALAPPDATA 官规:空值=未设置)——避免把 "" 当
|
|
95
|
+
* 有效根导致 path.join("",…) 产出相对路径、相对进程 cwd 落盘。
|
|
96
|
+
* darwin 亦接受 $XDG_CACHE_HOME 覆盖(跨平台统一 dotfiles 的宽容超集)。
|
|
97
|
+
*/
|
|
98
|
+
export function userVisionCacheRoot(env, platform, home) {
|
|
99
|
+
const base = platform === "darwin"
|
|
100
|
+
? (env.XDG_CACHE_HOME || path.join(home, "Library", "Caches"))
|
|
101
|
+
: platform === "win32"
|
|
102
|
+
? (env.LOCALAPPDATA || path.join(home, "AppData", "Local"))
|
|
103
|
+
: (env.XDG_CACHE_HOME || path.join(home, ".cache"));
|
|
104
|
+
return path.join(base, "opencode-vision-analyze", "vision");
|
|
105
|
+
}
|
|
106
|
+
/**
|
|
107
|
+
* 图片存储根:与 opencode 的项目/全局语义对齐——
|
|
108
|
+
* git 项目内 → <项目>/.opencode/vision(现状);非 git 目录 → 用户级缓存目录。
|
|
109
|
+
* 解析一次即可(纯函数,入参可注入以便三平台与 env 覆盖的单测)。
|
|
110
|
+
*/
|
|
111
|
+
export function resolveVisionDir(inputDir, env, platform, home) {
|
|
112
|
+
if (isInsideGitRepo(inputDir))
|
|
113
|
+
return path.join(path.resolve(inputDir), ".opencode", "vision");
|
|
114
|
+
return userVisionCacheRoot(env, platform, home);
|
|
115
|
+
}
|
|
80
116
|
/**
|
|
81
117
|
* http(s) 下载图片的大小上限(20 MB)。提示注入可让模型指向超大图片,
|
|
82
118
|
* 下载不限长是成本/健壮性放大器:先按 content-length 头提前拒绝,
|
|
@@ -91,13 +127,35 @@ const MAX_DOWNLOAD_BYTES = 20 * 1024 * 1024;
|
|
|
91
127
|
*/
|
|
92
128
|
const plugin = async (input, optionsArg) => {
|
|
93
129
|
// ---- 选项解析与校验 ----------------------------------------------------
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
130
|
+
// 归一化规则:models 是唯一显式入口 —— 有序候选数组(保序去重,重复只留首个)。
|
|
131
|
+
// 缺省或空数组 → explicit 为空(进入自动发现,见 resolveChain)。传了 models 但
|
|
132
|
+
// 类型不对(非字符串数组)直接抛错,避免用户配错被静默当成自动模式。
|
|
133
|
+
const modelsOption = optionsArg?.models;
|
|
134
|
+
if (optionsArg?.models !== undefined && !Array.isArray(modelsOption)) {
|
|
135
|
+
throw new Error('opencode-vision-analyze option "models" must be an array of "provider/model" strings');
|
|
97
136
|
}
|
|
98
|
-
const
|
|
99
|
-
|
|
100
|
-
|
|
137
|
+
const hasModels = Array.isArray(modelsOption) && modelsOption.length > 0;
|
|
138
|
+
// 收集字符串候选并逐项校验 provider/model 格式(modelID 允许含 "/",按首个 "/" 切分)。
|
|
139
|
+
// 逐项 throw:元素非法报 label "models[i]";保序去重(重复只留首个)。
|
|
140
|
+
const raw = hasModels ? modelsOption : [];
|
|
141
|
+
const explicitModels = [];
|
|
142
|
+
const seenKeys = new Set();
|
|
143
|
+
raw.forEach((item, index) => {
|
|
144
|
+
const label = `models[${index}]`;
|
|
145
|
+
if (typeof item !== "string" || !item.includes("/")) {
|
|
146
|
+
throw new Error(`opencode-vision-analyze option "${label}" must be in "provider/model" format, got: ${JSON.stringify(item)}`);
|
|
147
|
+
}
|
|
148
|
+
const sep = item.indexOf("/");
|
|
149
|
+
if (seenKeys.has(item))
|
|
150
|
+
return; // 重复模型只保留首个
|
|
151
|
+
seenKeys.add(item);
|
|
152
|
+
explicitModels.push({ providerID: item.slice(0, sep), modelID: item.slice(sep + 1) });
|
|
153
|
+
});
|
|
154
|
+
// unlisted_fallback:显式链耗尽后是否自动续接未列出的 image-capable 模型(仅显式配置时
|
|
155
|
+
// 生效)。free_first:自动发现档序是否反转(匿名/内置 custom 优先,默认 config 优先)。
|
|
156
|
+
// 二者按严格布尔取真,非布尔值宽容忽略按 false 处理(与下方 timeout_ms 的宽容校验一致)。
|
|
157
|
+
const fallbackUnlisted = optionsArg?.unlisted_fallback === true;
|
|
158
|
+
const freeFirst = optionsArg?.free_first === true;
|
|
101
159
|
// 子会话请求的超时时间:timeout_ms 为正数时生效,默认 60 秒。
|
|
102
160
|
const timeoutOption = optionsArg?.timeout_ms;
|
|
103
161
|
const timeoutMs = typeof timeoutOption === "number" && Number.isFinite(timeoutOption) && timeoutOption > 0 ? timeoutOption : 60000;
|
|
@@ -106,7 +164,11 @@ const plugin = async (input, optionsArg) => {
|
|
|
106
164
|
const sessionModels = new Map();
|
|
107
165
|
/** "provider/model" → 是否具备图片输入能力(查询结果缓存,进程级) */
|
|
108
166
|
const imageCapable = new Map();
|
|
109
|
-
/**
|
|
167
|
+
/**
|
|
168
|
+
* 描述缓存:"<sha>:<question>" → 描述结果(同一张图 + 同一个问题只描述一次)。
|
|
169
|
+
* 值带 modelId:记录实际产出该描述的候选模型,缓存命中时标签沿用入库模型,
|
|
170
|
+
* 而不是用当前候选链链首近似(链配置变化或 fallback 命中次选时标签才真实)。
|
|
171
|
+
*/
|
|
110
172
|
const descriptions = new Map();
|
|
111
173
|
/** 本插件创建的子会话 ID 集合(正常路径用后即删,dispose 兜底清理残留) */
|
|
112
174
|
const subSessions = new Set();
|
|
@@ -118,40 +180,62 @@ const plugin = async (input, optionsArg) => {
|
|
|
118
180
|
return error;
|
|
119
181
|
return JSON.stringify(error) ?? String(error);
|
|
120
182
|
};
|
|
183
|
+
/**
|
|
184
|
+
* 无 ToolContext 的超时原语:到期以 DeadlineError(timeoutMessage) 拒绝。
|
|
185
|
+
* 与 withDeadline 的差别是它不感知 abort——能力查询(providers)等无 ctx 的
|
|
186
|
+
* 请求只关心"别永久挂起",不需要监听用户中止;子会话请求由 withDeadline 组合
|
|
187
|
+
* abort 信号后复用它,保证全插件只有一套计时机制。
|
|
188
|
+
*/
|
|
189
|
+
const withTimeout = async (promise, ms, timeoutMessage) => {
|
|
190
|
+
let timer;
|
|
191
|
+
const guard = new Promise((_, reject) => {
|
|
192
|
+
timer = setTimeout(() => reject(new DeadlineError(timeoutMessage)), ms);
|
|
193
|
+
});
|
|
194
|
+
try {
|
|
195
|
+
return await Promise.race([promise, guard]);
|
|
196
|
+
}
|
|
197
|
+
finally {
|
|
198
|
+
if (timer)
|
|
199
|
+
clearTimeout(timer);
|
|
200
|
+
}
|
|
201
|
+
};
|
|
121
202
|
/**
|
|
122
203
|
* 给子会话请求加超时与 abort 保护:任一触发即让 Promise 以错误结束,
|
|
123
|
-
*
|
|
204
|
+
* 不再等待底层请求;超时复用 withTimeout(DeadlineError),abort 仍以
|
|
205
|
+
* AbortError 拒绝;finally 中清理 timer 与监听器,避免泄漏。
|
|
124
206
|
*/
|
|
125
207
|
const withDeadline = (promise, ctx) => {
|
|
126
|
-
|
|
208
|
+
// ctx.abort 已中止时 abort 事件不会再触发,必须立即拒绝,
|
|
209
|
+
// 否则 race 只能干等 timer 超时。
|
|
210
|
+
if (ctx.abort.aborted)
|
|
211
|
+
return Promise.reject(new DOMException("Aborted", "AbortError"));
|
|
127
212
|
let onAbort;
|
|
128
|
-
const
|
|
129
|
-
// ctx.abort 已中止时 abort 事件不会再触发,必须立即拒绝,
|
|
130
|
-
// 否则 race 只能干等 timer 超时。
|
|
131
|
-
if (ctx.abort.aborted) {
|
|
132
|
-
reject(new DOMException("Aborted", "AbortError"));
|
|
133
|
-
return;
|
|
134
|
-
}
|
|
135
|
-
timer = setTimeout(() => reject(new Error(`vision model call timed out after ${timeoutMs}ms`)), timeoutMs);
|
|
213
|
+
const abortGuard = new Promise((_, reject) => {
|
|
136
214
|
onAbort = () => reject(new DOMException("Aborted", "AbortError"));
|
|
137
215
|
ctx.abort.addEventListener("abort", onAbort, { once: true });
|
|
138
216
|
});
|
|
139
|
-
return Promise.race([promise,
|
|
140
|
-
if (timer)
|
|
141
|
-
clearTimeout(timer);
|
|
217
|
+
return withTimeout(Promise.race([promise, abortGuard]), timeoutMs, `vision model call timed out after ${timeoutMs}ms`).finally(() => {
|
|
142
218
|
if (onAbort)
|
|
143
219
|
ctx.abort.removeEventListener("abort", onAbort);
|
|
144
220
|
});
|
|
145
221
|
};
|
|
222
|
+
/** 判断错误是否为中止信号(AbortError),供 attemptModel 标记 / 链循环中止整链。 */
|
|
223
|
+
const isAbortError = (error) => error instanceof Error && error.name === "AbortError";
|
|
224
|
+
/** 判断错误是否为本插件超时信号(DeadlineError)——与 AbortError 并列,表示"请求到期被本地掐断"。 */
|
|
225
|
+
const isDeadlineError = (error) => error instanceof Error && error.name === "DeadlineError";
|
|
146
226
|
/**
|
|
147
|
-
*
|
|
148
|
-
*
|
|
149
|
-
*
|
|
150
|
-
*
|
|
227
|
+
* 单个候选的尝试:创建子会话(parentID 挂当前会话)→ 用该候选模型描述 →
|
|
228
|
+
* 删除子会话。任何失败(创建 / 请求 / 超时 / 中止 / 底层抛错 / 无文本)都
|
|
229
|
+
* 收敛为 { ok: false } 返回而不向上抛——推进与否交给 describeWithChain 的
|
|
230
|
+
* 链循环决策;中止额外打 aborted 标记,链循环据此立即停整链。无论成败,
|
|
231
|
+
* finally 中都删除子会话——用后即删,不留孤儿。
|
|
151
232
|
*/
|
|
152
|
-
const
|
|
233
|
+
const attemptModel = async (candidate, image, question, ctx) => {
|
|
153
234
|
const dataURL = `data:${image.mime};base64,${image.bytes.toString("base64")}`;
|
|
154
235
|
let subID;
|
|
236
|
+
// "回合可能仍在飞"标记:请求被本地 deadline(超时)或用户 abort 掐断时置位,
|
|
237
|
+
// finally 据此先 abort 子会话(取消 provider 端孤儿回合)再 delete。
|
|
238
|
+
let endedByDeadline = false;
|
|
155
239
|
try {
|
|
156
240
|
const created = await withDeadline(input.client.session.create({ body: { parentID: ctx.sessionID, title: "vision analysis" } }), ctx);
|
|
157
241
|
if (created.error || !created.data) {
|
|
@@ -162,7 +246,7 @@ const plugin = async (input, optionsArg) => {
|
|
|
162
246
|
const response = await withDeadline(input.client.session.prompt({
|
|
163
247
|
path: { id: subID },
|
|
164
248
|
body: {
|
|
165
|
-
model: { providerID:
|
|
249
|
+
model: { providerID: candidate.providerID, modelID: candidate.modelID },
|
|
166
250
|
agent: "build",
|
|
167
251
|
// 子会话禁用全部工具:视觉模型只做纯文本描述,避免它反过来调用
|
|
168
252
|
// vision_analyze 形成递归,也避免任何副作用。
|
|
@@ -185,21 +269,90 @@ const plugin = async (input, optionsArg) => {
|
|
|
185
269
|
return { ok: false, error: "vision model returned no text" };
|
|
186
270
|
return { ok: true, text };
|
|
187
271
|
}
|
|
272
|
+
catch (error) {
|
|
273
|
+
// 异常(超时 / 中止 / 底层抛错)同样收敛为失败结果;aborted 标记交由链循环判断。
|
|
274
|
+
// 超时与中止都意味着底层请求可能仍在飞 → 需要先 abort 再 delete。
|
|
275
|
+
endedByDeadline = isDeadlineError(error) || isAbortError(error);
|
|
276
|
+
return { ok: false, error: errText(error), aborted: isAbortError(error) };
|
|
277
|
+
}
|
|
188
278
|
finally {
|
|
189
279
|
if (subID) {
|
|
190
280
|
subSessions.delete(subID);
|
|
281
|
+
// 先 abort(best-effort,取消 provider 端孤儿回合)再 delete;
|
|
282
|
+
// 成功/普通失败路径 turn 已自然结束,无需 abort。
|
|
283
|
+
if (endedByDeadline) {
|
|
284
|
+
await input.client.session.abort({ path: { id: subID } }).catch(() => { });
|
|
285
|
+
}
|
|
191
286
|
await input.client.session.delete({ path: { id: subID } }).catch(() => { });
|
|
192
287
|
}
|
|
193
288
|
}
|
|
194
289
|
};
|
|
195
290
|
/**
|
|
196
|
-
*
|
|
197
|
-
*
|
|
198
|
-
*
|
|
199
|
-
*
|
|
291
|
+
* 候选链描述:沿 resolveChain() 产出的候选链逐个尝试,首个成功即返回(附带
|
|
292
|
+
* 成功候选的引用键作标签来源);单个候选失败记录 `${key}: ${error}` 并推进
|
|
293
|
+
* 下一个;收到 abort(pre-abort 短路或尝试结果的 aborted 标记)立即中止整链;
|
|
294
|
+
* 全部失败聚合各候选原因;空链返回友好错误。本函数永不抛错。
|
|
295
|
+
*/
|
|
296
|
+
const describeWithChain = async (image, question, ctx) => {
|
|
297
|
+
// pre-abort 短路:不解析候选链、不建子会话,直接以 Aborted 收尾
|
|
298
|
+
if (ctx.abort.aborted)
|
|
299
|
+
return { ok: false, error: "Aborted" };
|
|
300
|
+
const chain = await resolveChain();
|
|
301
|
+
const failures = [];
|
|
302
|
+
for (const candidate of chain) {
|
|
303
|
+
const attempt = await attemptModel(candidate, image, question, ctx);
|
|
304
|
+
if (attempt.ok)
|
|
305
|
+
return { ok: true, text: attempt.text, modelId: modelRefKey(candidate) };
|
|
306
|
+
if (attempt.aborted)
|
|
307
|
+
return { ok: false, error: attempt.error }; // abort 中止整链
|
|
308
|
+
failures.push(`${modelRefKey(candidate)}: ${attempt.error}`);
|
|
309
|
+
}
|
|
310
|
+
if (failures.length === 0) {
|
|
311
|
+
return {
|
|
312
|
+
ok: false,
|
|
313
|
+
error: "no image-capable model configured (set the plugin models option or configure an image-capable provider model)",
|
|
314
|
+
};
|
|
315
|
+
}
|
|
316
|
+
return { ok: false, error: `all ${failures.length} candidate model(s) failed: ${failures.join("; ")}` };
|
|
317
|
+
};
|
|
318
|
+
/** 描述标签:标注图片文件名与产出描述的模型引用键(basename 运行时按图传入)。 */
|
|
319
|
+
const format = (basename, modelId, text) => `[Image: ${basename} — described by ${modelId}]\n${text}`;
|
|
320
|
+
/**
|
|
321
|
+
* 把图片字节内容原子落盘并返回最终 filepath。每次调用现算存储根
|
|
322
|
+
* (git 项目 → 项目 .opencode/vision;非 git → 用户级缓存,见 resolveVisionDir):
|
|
323
|
+
* 运行中存储范围变化(如 git init)从下一条图片起即时生效,无需重启。
|
|
324
|
+
* 先写 `<sha><ext>.tmp-<uuid>` 再 rename:共享目录(用户级/多实例)并发写同一 sha
|
|
325
|
+
* 时内容寻址下原子幂等;失败先清理临时文件再抛错,避免孤儿 tmp 累积。
|
|
326
|
+
*/
|
|
327
|
+
const persistImageBytes = async (bytes, ext) => {
|
|
328
|
+
const dir = resolveVisionDir(input.directory, process.env, process.platform, homedir());
|
|
329
|
+
const sha = createHash("sha256").update(bytes).digest("hex");
|
|
330
|
+
const filepath = path.join(dir, `${sha}${ext}`);
|
|
331
|
+
const tmpPath = path.join(dir, `${sha}${ext}.tmp-${randomUUID()}`);
|
|
332
|
+
await fs.mkdir(dir, { recursive: true, mode: 0o700 });
|
|
333
|
+
try {
|
|
334
|
+
await fs.writeFile(tmpPath, bytes);
|
|
335
|
+
await fs.rename(tmpPath, filepath);
|
|
336
|
+
}
|
|
337
|
+
catch (error) {
|
|
338
|
+
await fs.unlink(tmpPath).catch(() => { });
|
|
339
|
+
throw error;
|
|
340
|
+
}
|
|
341
|
+
return filepath;
|
|
342
|
+
};
|
|
343
|
+
/**
|
|
344
|
+
* 下载 http(s) URL 指向的图片并落盘(内容哈希命名天然去重,写盘细节见
|
|
345
|
+
* persistImageBytes)。扩展名不受支持、HTTP 非 2xx、网络失败(含 30 秒下载
|
|
346
|
+
* 超时)、超过 20 MB 下载上限(content-length 预检 + 读后复核)都返回 { error },
|
|
200
347
|
* 由调用方转成可读的错误文字。
|
|
348
|
+
*
|
|
349
|
+
* 中止传导:abort(用户中止)与 30 秒超时共同驱动一个 AbortController——
|
|
350
|
+
* 中止即刻断请求,不空跑满超时;pre-abort 直接放弃、不发请求。手动组合信号
|
|
351
|
+
* 而非 AbortSignal.any():engines node>=18(any 需 18.17+/20.3+),且与
|
|
352
|
+
* withDeadline 的 addEventListener 风格同构。错误以 AbortError/超时形式落入
|
|
353
|
+
* catch,统一转可读文字。
|
|
201
354
|
*/
|
|
202
|
-
const downloadImage = async (url) => {
|
|
355
|
+
const downloadImage = async (url, abort) => {
|
|
203
356
|
try {
|
|
204
357
|
// URL 解析与扩展名提取放在 try 内:畸形 URL 在 new URL 处抛错时,
|
|
205
358
|
// 错误以 "Image download failed" 前缀返回,而不是漏到外层的
|
|
@@ -208,24 +361,34 @@ const plugin = async (input, optionsArg) => {
|
|
|
208
361
|
const mime = EXT_MIME[ext];
|
|
209
362
|
if (!mime)
|
|
210
363
|
return { error: `unsupported image URL extension: ${ext || "(none)"}` };
|
|
211
|
-
|
|
212
|
-
if (
|
|
213
|
-
return { error:
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
364
|
+
// pre-abort:已中止则不必发请求,直接以 Aborted 收尾
|
|
365
|
+
if (abort.aborted)
|
|
366
|
+
return { error: "Aborted" };
|
|
367
|
+
const controller = new AbortController();
|
|
368
|
+
const onAbort = () => controller.abort();
|
|
369
|
+
abort.addEventListener("abort", onAbort, { once: true });
|
|
370
|
+
const timer = setTimeout(() => controller.abort(), 30_000);
|
|
371
|
+
try {
|
|
372
|
+
// 整个下载(fetch 响应头 + arrayBuffer 读 body)都在同一 guard 内:
|
|
373
|
+
// 30 秒预算与用户中止都覆盖到 body 读取阶段;收尾再清 timer/listener。
|
|
374
|
+
const response = await fetch(url, { signal: controller.signal });
|
|
375
|
+
if (!response.ok)
|
|
376
|
+
return { error: `HTTP ${response.status}` };
|
|
377
|
+
// 头字段缺失时 Number(null) 为 NaN,比较结果为 false,自然放行到读后复核。
|
|
378
|
+
if (Number(response.headers.get("content-length")) > MAX_DOWNLOAD_BYTES) {
|
|
379
|
+
return { error: "image exceeds 20 MB download limit" };
|
|
380
|
+
}
|
|
381
|
+
const bytes = Buffer.from(await response.arrayBuffer());
|
|
382
|
+
// 复核实际字节数:chunked 等无 content-length 的响应只有读后才能判大小。
|
|
383
|
+
if (bytes.length > MAX_DOWNLOAD_BYTES) {
|
|
384
|
+
return { error: "image exceeds 20 MB download limit" };
|
|
385
|
+
}
|
|
386
|
+
return { filepath: await persistImageBytes(bytes, ext) };
|
|
217
387
|
}
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
return { error: "image exceeds 20 MB download limit" };
|
|
388
|
+
finally {
|
|
389
|
+
clearTimeout(timer);
|
|
390
|
+
abort.removeEventListener("abort", onAbort);
|
|
222
391
|
}
|
|
223
|
-
const sha = createHash("sha256").update(bytes).digest("hex");
|
|
224
|
-
const dir = path.join(input.directory, ".opencode", "vision");
|
|
225
|
-
await fs.mkdir(dir, { recursive: true });
|
|
226
|
-
const filepath = path.join(dir, `${sha}${ext}`);
|
|
227
|
-
await fs.writeFile(filepath, bytes);
|
|
228
|
-
return { filepath };
|
|
229
392
|
}
|
|
230
393
|
catch (error) {
|
|
231
394
|
return { error: errText(error) };
|
|
@@ -258,7 +421,10 @@ const plugin = async (input, optionsArg) => {
|
|
|
258
421
|
try {
|
|
259
422
|
const question = args.question?.trim() || "Describe this image in full detail.";
|
|
260
423
|
// http(s) URL:先下载到本地 vision 目录,再统一走磁盘加载路径。
|
|
261
|
-
|
|
424
|
+
// ctx.abort 传入下载:用户中止即刻断下载(含 pre-abort 不再发请求)。
|
|
425
|
+
const download = /^https?:\/\//i.test(args.image_path)
|
|
426
|
+
? await downloadImage(args.image_path, ctx.abort)
|
|
427
|
+
: undefined;
|
|
262
428
|
if (download && "error" in download) {
|
|
263
429
|
return { title, output: `Image download failed: ${download.error}` };
|
|
264
430
|
}
|
|
@@ -285,15 +451,19 @@ const plugin = async (input, optionsArg) => {
|
|
|
285
451
|
}
|
|
286
452
|
// 描述缓存:内容哈希 + 问题作为 key,命中直接复用(title 标注 cached)。
|
|
287
453
|
const key = `${createHash("sha256").update(image.bytes).digest("hex")}:${question}`;
|
|
288
|
-
const format = (text) => `[Image: ${path.basename(imagePath)} — described by ${visionProviderID}/${visionModelID}]\n${text}`;
|
|
289
454
|
const cached = descriptions.get(key);
|
|
290
|
-
if (cached !== undefined)
|
|
291
|
-
|
|
292
|
-
|
|
455
|
+
if (cached !== undefined) {
|
|
456
|
+
// 命中时标签沿用入库时的模型(cached.modelId):即便此刻候选链链首
|
|
457
|
+
// 已与入库模型不同,也保持标签真实、不重写。
|
|
458
|
+
return { title: `${title} (cached)`, output: format(path.basename(imagePath), cached.modelId, cached.text) };
|
|
459
|
+
}
|
|
460
|
+
const result = await describeWithChain(image, question, ctx);
|
|
293
461
|
if (!result.ok)
|
|
294
462
|
return { title, output: `Image analysis failed: ${result.error}` };
|
|
295
|
-
|
|
296
|
-
|
|
463
|
+
// 入库带上实际产出描述的候选 modelId,供后续缓存命中还原真实标签
|
|
464
|
+
descriptions.set(key, { modelId: result.modelId, text: result.text });
|
|
465
|
+
// 成功标签直接用实际产出描述的候选引用键(而非链首近似)
|
|
466
|
+
return { title, output: format(path.basename(imagePath), result.modelId, result.text) };
|
|
297
467
|
}
|
|
298
468
|
catch (error) {
|
|
299
469
|
return { title, output: `Image analysis failed: ${errText(error)}` };
|
|
@@ -311,7 +481,7 @@ const plugin = async (input, optionsArg) => {
|
|
|
311
481
|
if (cached !== undefined)
|
|
312
482
|
return cached;
|
|
313
483
|
try {
|
|
314
|
-
const result = await input.client.config.providers();
|
|
484
|
+
const result = await withTimeout(input.client.config.providers(), providersTimeout.ms, `config.providers() timed out after ${providersTimeout.ms}ms`);
|
|
315
485
|
// HTTP 非 2xx 时 openapi-fetch 不抛错而是返回 { error }(data 为空)。
|
|
316
486
|
// 「查询失败」不能缓存成 false——那是一次瞬时故障而非「确认不支持」,
|
|
317
487
|
// 缓存会永久关闭能力门控;本次保守返回 false,下次再重试。
|
|
@@ -326,9 +496,73 @@ const plugin = async (input, optionsArg) => {
|
|
|
326
496
|
return false;
|
|
327
497
|
}
|
|
328
498
|
};
|
|
499
|
+
/** Provider.source → 自动发现档位:config 最优先,env/api 次之,custom 与未知值最末 */
|
|
500
|
+
const tierOfSource = (source) => source === "config" ? 0 : source === "env" || source === "api" ? 1 : 2;
|
|
501
|
+
/** "provider/model" 引用键(与 imageCapable 缓存的键格式一致) */
|
|
502
|
+
const modelRefKey = (c) => `${c.providerID}/${c.modelID}`;
|
|
503
|
+
// 候选链 memoize:chat.message 钩子与 vision_analyze 工具共享一次 providers 查询
|
|
504
|
+
let chainPromise;
|
|
329
505
|
/**
|
|
330
|
-
*
|
|
331
|
-
*
|
|
506
|
+
* 归一化产出最终候选链(统一数组,运行时只做逐个尝试):
|
|
507
|
+
* - 显式非空:显式链恒在链首(不受 free_first/档序影响);
|
|
508
|
+
* unlisted_fallback=true 时再追加未列出的 image-capable 模型
|
|
509
|
+
* - 显式为空:整链 = 自动发现(全部 image-capable 模型,按 source 档序)
|
|
510
|
+
*/
|
|
511
|
+
const resolveChain = () => {
|
|
512
|
+
chainPromise ??= (async () => {
|
|
513
|
+
// 自动模式:整链由自动发现决定
|
|
514
|
+
if (explicitModels.length === 0)
|
|
515
|
+
return listImageCapableModels();
|
|
516
|
+
// 显式模式:默认只用显式链;unlisted_fallback=true 时追加 inventory 中未列出的模型
|
|
517
|
+
if (!fallbackUnlisted)
|
|
518
|
+
return explicitModels;
|
|
519
|
+
const inventory = await listImageCapableModels();
|
|
520
|
+
const explicitSet = new Set(explicitModels.map(modelRefKey));
|
|
521
|
+
return [...explicitModels, ...inventory.filter((c) => !explicitSet.has(modelRefKey(c)))];
|
|
522
|
+
})();
|
|
523
|
+
return chainPromise;
|
|
524
|
+
};
|
|
525
|
+
/**
|
|
526
|
+
* 枚举 config.providers() 中全部 image-capable 模型,并按 Provider.source 档位
|
|
527
|
+
* 稳定排序(档内保持 providers 返回顺序):默认 config > env/api > custom;
|
|
528
|
+
* free_first=true 时档序反转(custom 优先)。顺带预填 imageCapable 缓存
|
|
529
|
+
* (与 imageSupport 同源,避免后续重复请求)。providers 查询瞬时失败返回空数组
|
|
530
|
+
* 并记日志:显式链仍可用(fallback 追加部分静默跳过),自动模式退化为空链——
|
|
531
|
+
* 因 resolveChain 的 memoize,本次空链会持续整个进程(见 docs/superpowers/specs/2026-09-05-opencode-vision-analyze-design.md 已知限制)。
|
|
532
|
+
*/
|
|
533
|
+
const listImageCapableModels = async () => {
|
|
534
|
+
try {
|
|
535
|
+
const result = await withTimeout(input.client.config.providers(), providersTimeout.ms, `config.providers() timed out after ${providersTimeout.ms}ms`);
|
|
536
|
+
if (!result.data)
|
|
537
|
+
return [];
|
|
538
|
+
const found = [];
|
|
539
|
+
for (const provider of result.data.providers ?? []) {
|
|
540
|
+
const tier = tierOfSource(provider.source);
|
|
541
|
+
for (const [modelID, model] of Object.entries(provider.models ?? {})) {
|
|
542
|
+
if (model?.capabilities?.input?.image !== true)
|
|
543
|
+
continue;
|
|
544
|
+
imageCapable.set(`${provider.id}/${modelID}`, true);
|
|
545
|
+
found.push({ providerID: provider.id, modelID, tier });
|
|
546
|
+
}
|
|
547
|
+
}
|
|
548
|
+
// 稳定排序:默认按档位升序(config 优先);free_first 反转成降序(custom 优先)
|
|
549
|
+
found.sort((a, b) => (freeFirst ? b.tier - a.tier : a.tier - b.tier));
|
|
550
|
+
return found.map(({ providerID, modelID }) => ({ providerID, modelID }));
|
|
551
|
+
}
|
|
552
|
+
catch (error) {
|
|
553
|
+
// 自动发现的关键 providers 查询失败要可观测:不静默吞掉,记一行日志便于定位。
|
|
554
|
+
// 显式链不受影响;自动模式按空链处理(本次进程内不再重试,见已知限制)。
|
|
555
|
+
console.error("[opencode-vision-analyze] config.providers() failed; auto vision discovery disabled", error);
|
|
556
|
+
return [];
|
|
557
|
+
}
|
|
558
|
+
};
|
|
559
|
+
/** 判断某 model 是否为当前候选链成员(chat.message 递归防护用) */
|
|
560
|
+
const isCandidateModel = async (model) => {
|
|
561
|
+
const chain = await resolveChain();
|
|
562
|
+
return chain.some((c) => c.providerID === model.providerID && c.modelID === model.modelID);
|
|
563
|
+
};
|
|
564
|
+
/**
|
|
565
|
+
* 把一个图片 file part 落盘(内容哈希命名去重,写盘细节见 persistImageBytes)。
|
|
332
566
|
* 返回落盘信息;MIME 不受支持或 URL 不是 base64 data URL 时返回 undefined。
|
|
333
567
|
*/
|
|
334
568
|
const persistImage = async (part) => {
|
|
@@ -340,12 +574,7 @@ const plugin = async (input, optionsArg) => {
|
|
|
340
574
|
return undefined;
|
|
341
575
|
try {
|
|
342
576
|
const bytes = Buffer.from(match[2], "base64");
|
|
343
|
-
|
|
344
|
-
const dir = path.join(input.directory, ".opencode", "vision");
|
|
345
|
-
await fs.mkdir(dir, { recursive: true });
|
|
346
|
-
const filepath = path.join(dir, `${sha}${ext}`);
|
|
347
|
-
await fs.writeFile(filepath, bytes);
|
|
348
|
-
return { filepath };
|
|
577
|
+
return { filepath: await persistImageBytes(bytes, ext) };
|
|
349
578
|
}
|
|
350
579
|
catch {
|
|
351
580
|
// fail-open 原则:图片落盘失败(EACCES/ENOSPC 等)只是少了 vision_analyze
|
|
@@ -358,11 +587,12 @@ const plugin = async (input, optionsArg) => {
|
|
|
358
587
|
* push 进去的 part 会一并入库)。
|
|
359
588
|
*
|
|
360
589
|
* 职责:
|
|
361
|
-
* 1.
|
|
362
|
-
* 2.
|
|
363
|
-
* 3.
|
|
590
|
+
* 1. 记录会话当前模型(先于递归防护:会话模型恰为视觉模型时也需记录);
|
|
591
|
+
* 2. 收集图片 part,没有图片则直接返回;
|
|
592
|
+
* 3. 递归防护——消息模型 ∈ 候选链全体成员(我们的描述子会话)则放行;
|
|
364
593
|
* 4. 能力门控——主模型本身能看图则不注入提示;
|
|
365
|
-
* 5.
|
|
594
|
+
* 5. 空链降级——没有任何可用视觉模型时不注入 hint;
|
|
595
|
+
* 6. 图片落盘,并注入一条 synthetic text part 引导模型使用 vision_analyze。
|
|
366
596
|
*/
|
|
367
597
|
const onChatMessage = async (hookInput, output) => {
|
|
368
598
|
// 记录会话当前模型,供后续 vision_analyze 快速路径与未显式指定 model 的
|
|
@@ -371,18 +601,24 @@ const plugin = async (input, optionsArg) => {
|
|
|
371
601
|
// 看不到该模型,会退化为子会话描述。
|
|
372
602
|
if (hookInput.model)
|
|
373
603
|
sessionModels.set(hookInput.sessionID, hookInput.model);
|
|
374
|
-
// 递归防护:视觉模型自身的 prompt(描述子会话)直接放行,
|
|
375
|
-
// 避免插件处理自己发起的消息造成循环。
|
|
376
|
-
if (hookInput.model?.providerID === visionProviderID && hookInput.model?.modelID === visionModelID)
|
|
377
|
-
return;
|
|
378
604
|
// 只处理 base64 图片附件;没有图片就没有副作用。
|
|
379
605
|
const images = output.parts.filter((part) => part.type === "file" && part.mime.startsWith("image/"));
|
|
380
606
|
if (images.length === 0)
|
|
381
607
|
return;
|
|
608
|
+
// 递归防护:防护集 = 整条候选链(而非单模型/链首)。描述子会话用的模型是
|
|
609
|
+
// 链上任意候选,只要消息模型命中任一成员就放行,避免插件处理自己发起的
|
|
610
|
+
// 消息形成循环。resolveChain 懒加载 + memoize,只在首次触发一次 providers 查询。
|
|
611
|
+
if (hookInput.model && (await isCandidateModel(hookInput.model)))
|
|
612
|
+
return;
|
|
382
613
|
// 能力门控:主模型有视觉能力时原图直发,不需要任何提示。
|
|
383
614
|
const current = hookInput.model ?? sessionModels.get(hookInput.sessionID);
|
|
384
615
|
if (current && (await imageSupport(current.providerID, current.modelID)))
|
|
385
616
|
return;
|
|
617
|
+
// 空链降级:没有任何可用视觉模型时不注入 hint、不落盘——落盘只会制造没有
|
|
618
|
+
// 视觉模型可消费的垃圾文件;交给核心 unsupportedParts 对图片 part 的默认处理。
|
|
619
|
+
const chain = await resolveChain();
|
|
620
|
+
if (chain.length === 0)
|
|
621
|
+
return;
|
|
386
622
|
// 每张图落盘并生成两行提示;任何一张落盘失败就跳过该图(不影响其余图片)。
|
|
387
623
|
const lines = [];
|
|
388
624
|
for (const part of images) {
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "opencode-vision-analyze",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.3.0",
|
|
4
4
|
"description": "OpenCode plugin: vision_analyze tool — describe images & image URLs with your own vision model for text-only main models; native-image fast path for multimodal models. Zero runtime dependencies.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "./dist/index.js",
|
|
@@ -18,7 +18,9 @@
|
|
|
18
18
|
"typecheck": "tsc --noEmit",
|
|
19
19
|
"build": "tsc",
|
|
20
20
|
"test": "bun test",
|
|
21
|
-
"prepublishOnly": "npm run build"
|
|
21
|
+
"prepublishOnly": "npm run build",
|
|
22
|
+
"preversion": "bun run typecheck && bun test && bun run build",
|
|
23
|
+
"postversion": "git push --follow-tags"
|
|
22
24
|
},
|
|
23
25
|
"keywords": [
|
|
24
26
|
"opencode",
|