opencode-vision-analyze 0.2.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +9 -9
- package/README.zh.md +7 -9
- package/dist/index.d.ts +26 -0
- package/dist/index.js +169 -79
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -4,6 +4,8 @@
|
|
|
4
4
|
[](./LICENSE)
|
|
5
5
|
[](https://opencode.ai/docs/plugins)
|
|
6
6
|
|
|
7
|
+
English | [简体中文](./README.zh.md)
|
|
8
|
+
|
|
7
9
|
A tool-based vision routing plugin for [opencode](https://opencode.ai): when the main model can't see images, it calls the `vision_analyze` tool on demand — your dedicated vision model describes the image and the description flows straight back into the conversation. When the main model already supports images, pasted images pass through untouched and the tool short-circuits to return raw pixels.
|
|
8
10
|
|
|
9
11
|
**Zero runtime dependencies.** Only node builtins (`crypto`/`fs`/`path`) and type-only imports — nothing to install beyond the plugin itself.
|
|
@@ -13,7 +15,7 @@ A tool-based vision routing plugin for [opencode](https://opencode.ai): when the
|
|
|
13
15
|
- **Tool-based, not pre-analysis.** The turn starts immediately; the model decides when (and with which question) to look. No blocking on submit, failures are visible and retryable inside the agent loop. Same philosophy as production-proven agent designs.
|
|
14
16
|
- **Question-aware descriptions.** The model passes its own focused question to `vision_analyze` — not a one-shot generic caption computed at submit time.
|
|
15
17
|
- **Native fast path.** If the main model is vision-capable, `vision_analyze` skips the vision model entirely and returns the raw image as a tool attachment.
|
|
16
|
-
- **Content-addressed cache.** Images are stored as `<sha256>.<ext>` (deduped across sessions); descriptions are cached per `<image-hash>:<question>` — the same image with the same question is described exactly once.
|
|
18
|
+
- **Content-addressed cache.** Images are stored as content-addressed `<sha256>.<ext>` files (deduped across sessions within the same store); descriptions are cached per `<image-hash>:<question>` — the same image with the same question is described exactly once.
|
|
17
19
|
- **Unified auth.** The vision call runs through an opencode sub-session, so it reuses the provider credentials opencode already manages. No extra API key plumbing.
|
|
18
20
|
|
|
19
21
|
## Installation
|
|
@@ -67,6 +69,8 @@ Notes for the curl path:
|
|
|
67
69
|
|
|
68
70
|
Supported image extensions: png / jpg / jpeg / gif / webp.
|
|
69
71
|
|
|
72
|
+
Image storage: inside a git project images live under `<project>/.opencode/vision`; in non-git directories they go to the user cache dir (`<cache>/opencode-vision-analyze/vision`) — mirroring opencode's own project/global session scoping. Defaults per platform: Linux `$XDG_CACHE_HOME || ~/.cache`, macOS `~/Library/Caches` (a `$XDG_CACHE_HOME` override is honored), Windows `%LOCALAPPDATA% || ~/AppData/Local`. An empty cache-root env var is treated as unset (falls back to the default). In git projects, add `.opencode/vision/` to your `.gitignore` if you don't want the cache tracked. The storage root is re-resolved on every write, so switching scope (e.g. after `git init`) takes effect on the next pasted image — though old session hints keep pointing at stale absolute paths until you re-paste.
|
|
73
|
+
|
|
70
74
|
An ordered-candidates example with auto-fallback and free-first discovery:
|
|
71
75
|
|
|
72
76
|
```jsonc
|
|
@@ -94,7 +98,8 @@ User pastes image + question
|
|
|
94
98
|
├─ main model has image input capability → do nothing (raw image goes to model)
|
|
95
99
|
├─ no image-capable model available → do nothing (no hint, no persist;
|
|
96
100
|
│ core's default image handling applies)
|
|
97
|
-
└─ text-only main model → persist image to
|
|
101
|
+
└─ text-only main model → persist image to the vision store
|
|
102
|
+
(<sha256>.<ext>; project-local inside a git repo, user cache otherwise)
|
|
98
103
|
and inject a synthetic hint (hidden in TUI, visible to model):
|
|
99
104
|
"use the vision_analyze tool with image_path: ..."
|
|
100
105
|
|
|
@@ -126,17 +131,12 @@ Key behaviors:
|
|
|
126
131
|
## Known limitations
|
|
127
132
|
|
|
128
133
|
- **V1 session flow only** — hooks are attached to the V1 `SessionPrompt` path; if opencode's default interaction moves to the V2 session core, hooks won't fire (silently).
|
|
129
|
-
- **SSRF surface** — URL downloads follow redirects and don't block private-range / cloud-metadata addresses. Acceptable for a local single-user CLI; add address filtering before using in multi-tenant environments.
|
|
130
|
-
- **Abort doesn't propagate** — user aborts don't cancel in-flight downloads/sub-session requests; they run to their own deadlines (30s download, `timeout_ms` sub-session). After timeout/abort the sub-session is deleted, but the orphan turn may still be billed by the provider.
|
|
131
134
|
- **Historical images** — images from messages sent before the plugin was enabled can't be described (no hint, no path on disk).
|
|
132
|
-
- **Unbounded caches** — both the image store and description cache grow without eviction (
|
|
135
|
+
- **Unbounded caches** — both the image store and the description cache grow without eviction (image store: git-project-scoped or user-cache-scoped; description cache: per process).
|
|
133
136
|
|
|
134
137
|
## Roadmap
|
|
135
138
|
|
|
136
|
-
- [ ]
|
|
137
|
-
- [ ] Abort sub-session (`/session/{id}/abort`) before delete on timeout
|
|
138
|
-
- [ ] LRU / size cap for the description cache
|
|
139
|
-
- [ ] Optional private-address blocking for URL downloads
|
|
139
|
+
- [ ] Persistent description cache (content-addressed on disk, with LRU / size cap)
|
|
140
140
|
- [ ] Region cropping for zooming into image details
|
|
141
141
|
|
|
142
142
|
## Development
|
package/README.zh.md
CHANGED
|
@@ -15,7 +15,7 @@
|
|
|
15
15
|
- **工具化,而非提交时预分析。** 轮次即时启动;模型自己决定何时看图、带着什么问题看。提交零阻塞,失败在 agent 循环里可见、可重试。
|
|
16
16
|
- **描述针对问题。** 模型把自己关注的问题传给 `vision_analyze`——而不是提交时预生成的一次性通用描述。
|
|
17
17
|
- **原生快速路径。** 主模型本身有视觉能力时,`vision_analyze` 完全跳过视觉模型,直接把原图作为工具附件返回。
|
|
18
|
-
- **内容寻址缓存。**
|
|
18
|
+
- **内容寻址缓存。** 图片按内容寻址 `<sha256>.<ext>` 落盘(跨会话、同一存储域内天然去重);描述按 `<图片哈希>:<问题>` 缓存——同图同问题只描述一次。
|
|
19
19
|
- **统一鉴权。** 视觉调用走 opencode 子会话,复用 opencode 已管理的 provider 凭据,无需额外配置 API Key。
|
|
20
20
|
|
|
21
21
|
## 安装
|
|
@@ -69,6 +69,8 @@ curl 方式说明:
|
|
|
69
69
|
|
|
70
70
|
受支持的图片扩展名:png / jpg / jpeg / gif / webp。
|
|
71
71
|
|
|
72
|
+
图片存储:git 项目内图片放在 `<项目>/.opencode/vision`;非 git 目录则放入用户级缓存目录(`<cache>/opencode-vision-analyze/vision`)——与 opencode 自身的项目/全局会话分域一致。各平台默认:Linux `$XDG_CACHE_HOME || ~/.cache`;macOS `~/Library/Caches`(亦接受 `$XDG_CACHE_HOME` 覆盖);Windows `%LOCALAPPDATA% || ~/AppData/Local`。缓存根 env 为空串视为未设置(回退默认)。git 项目中如不想跟踪缓存图片,请把 `.opencode/vision/` 加入 `.gitignore`。存储根在每次落盘时现算:切换存储范围(如执行 `git init`)后,从下一条贴图起即写入新域;旧会话 hint 里的绝对路径仍指向旧处,重新贴图即注入新 hint。
|
|
73
|
+
|
|
72
74
|
有序候选 + 自动续接 + 免费优先的配置示例:
|
|
73
75
|
|
|
74
76
|
```jsonc
|
|
@@ -96,7 +98,8 @@ curl 方式说明:
|
|
|
96
98
|
├─ 主模型支持图片输入 → 不做任何处理(原图直发)
|
|
97
99
|
├─ 无任何 image-capable 模型 → 不做处理(不注入 hint、不落盘;
|
|
98
100
|
│ 交给核心对图片的默认处理)
|
|
99
|
-
└─ 纯文本主模型 →
|
|
101
|
+
└─ 纯文本主模型 → 图片落盘到 vision 存储
|
|
102
|
+
(<sha256>.<ext>;git 项目内 → 项目目录,非 git → 用户缓存目录)
|
|
100
103
|
并注入 synthetic 提示(TUI 隐藏、模型可见):
|
|
101
104
|
"用 vision_analyze 工具查看,image_path: ..."
|
|
102
105
|
|
|
@@ -128,17 +131,12 @@ vision_analyze 工具:
|
|
|
128
131
|
## 已知限制
|
|
129
132
|
|
|
130
133
|
- **仅 V1 会话流** —— 钩子挂在 V1 `SessionPrompt` 路径上;若 opencode 默认交互切到 V2 会话核心,钩子不会触发(且不报错)。
|
|
131
|
-
- **SSRF 面** —— URL 下载跟随重定向、不拦截私网/云元数据地址。本地单用户 CLI 信任级别下可接受;多租户环境使用前应加地址过滤。
|
|
132
|
-
- **中止不传导** —— 用户中止不会取消进行中的下载/子会话请求,它们会跑到各自的 deadline(下载 30 秒、子会话 `timeout_ms`)。超时/中止后子会话虽被删除,但 provider 端的孤儿回合仍可能计费。
|
|
133
134
|
- **历史图片** —— 插件启用之前发送的图片无法被描述(无提示、磁盘上无路径)。
|
|
134
|
-
- **缓存无上限** ——
|
|
135
|
+
- **缓存无上限** —— 图片存储与描述缓存均不淘汰(图片存储:git 项目级 / 用户级缓存目录;描述缓存:进程级)。
|
|
135
136
|
|
|
136
137
|
## Roadmap
|
|
137
138
|
|
|
138
|
-
- [ ]
|
|
139
|
-
- [ ] 超时路径先中止子会话(`/session/{id}/abort`)再删除
|
|
140
|
-
- [ ] 描述缓存 LRU / 容量上限
|
|
141
|
-
- [ ] URL 下载可选私网地址拦截
|
|
139
|
+
- [ ] 描述缓存按内容 sha 落盘持久化(含 LRU / 容量上限)
|
|
142
140
|
- [ ] 区域裁剪(放大查看图片细节)
|
|
143
141
|
|
|
144
142
|
## 开发
|
package/dist/index.d.ts
CHANGED
|
@@ -1,4 +1,30 @@
|
|
|
1
1
|
import type { Plugin } from "@opencode-ai/plugin";
|
|
2
|
+
/**
|
|
3
|
+
* `config.providers()` 能力查询的超时预算(毫秒)。做成可改写对象(而非常量/选项):
|
|
4
|
+
* 避免选项膨胀;测试把 ms 调小即可缩短等待(TS 不允许对 import 的 let 绑定赋值)。
|
|
5
|
+
*/
|
|
6
|
+
export declare const providersTimeout: {
|
|
7
|
+
ms: number;
|
|
8
|
+
};
|
|
9
|
+
/**
|
|
10
|
+
* 判断目录是否位于 git 项目内:从 dir 向上(含自身)逐级找 `.git`
|
|
11
|
+
* (目录,或 worktree/submodule 的 `.git` 指针文件),到文件系统根为止。
|
|
12
|
+
* 纯同步、纯内置模块;作为命名导出便于单元测试注入真实临时目录验证。
|
|
13
|
+
*/
|
|
14
|
+
export declare function isInsideGitRepo(dir: string): boolean;
|
|
15
|
+
/**
|
|
16
|
+
* 用户级(非 git)图片缓存的平台根:cache 目录 + opencode-vision-analyze/vision。
|
|
17
|
+
* 空字符串 env 视为未设置(XDG/LOCALAPPDATA 官规:空值=未设置)——避免把 "" 当
|
|
18
|
+
* 有效根导致 path.join("",…) 产出相对路径、相对进程 cwd 落盘。
|
|
19
|
+
* darwin 亦接受 $XDG_CACHE_HOME 覆盖(跨平台统一 dotfiles 的宽容超集)。
|
|
20
|
+
*/
|
|
21
|
+
export declare function userVisionCacheRoot(env: Record<string, string | undefined>, platform: string, home: string): string;
|
|
22
|
+
/**
|
|
23
|
+
* 图片存储根:与 opencode 的项目/全局语义对齐——
|
|
24
|
+
* git 项目内 → <项目>/.opencode/vision(现状);非 git 目录 → 用户级缓存目录。
|
|
25
|
+
* 解析一次即可(纯函数,入参可注入以便三平台与 env 覆盖的单测)。
|
|
26
|
+
*/
|
|
27
|
+
export declare function resolveVisionDir(inputDir: string, env: Record<string, string | undefined>, platform: string, home: string): string;
|
|
2
28
|
declare const _default: {
|
|
3
29
|
id: string;
|
|
4
30
|
server: Plugin;
|
package/dist/index.js
CHANGED
|
@@ -2,33 +2,10 @@
|
|
|
2
2
|
* opencode-vision-analyze
|
|
3
3
|
*
|
|
4
4
|
* 为「不具备视觉能力的主模型」提供图片解读路由:当用户在消息中附带图片时,
|
|
5
|
-
* 插件把图片落盘到
|
|
6
|
-
*
|
|
7
|
-
*
|
|
8
|
-
*
|
|
9
|
-
*
|
|
10
|
-
* 安装方式一(npm):
|
|
11
|
-
* {
|
|
12
|
-
* "plugin": [["opencode-vision-analyze", { "models": ["provider/vision-model"] }]]
|
|
13
|
-
* }
|
|
14
|
-
*
|
|
15
|
-
* 安装方式二(curl 下载单文件,免 npm):
|
|
16
|
-
* mkdir -p .opencode
|
|
17
|
-
* curl -fsSL <raw-url>/src/index.ts -o .opencode/vision-analyze.ts
|
|
18
|
-
* {
|
|
19
|
-
* "plugin": [["./.opencode/vision-analyze.ts", { "models": ["provider/vision-model"] }]]
|
|
20
|
-
* }
|
|
21
|
-
*
|
|
22
|
-
* 选项:
|
|
23
|
-
* - models(可选,缺省/空数组 = 自动模式):有序视觉候选数组,如
|
|
24
|
-
* ["provider-a/m1", "provider-b/m2"];单模型写 ["provider/model"] 即可
|
|
25
|
-
* - unlisted_fallback(可选,默认 false):显式候选耗尽后自动续接未列出的 image-capable 模型
|
|
26
|
-
* - free_first(可选,默认 false):自动发现档序反转(custom/匿名免费源优先,默认 config 优先)
|
|
27
|
-
* - timeout_ms:单候选子会话请求的超时毫秒数(正数,默认 60000)
|
|
28
|
-
*
|
|
29
|
-
* 候选链语义:显式 models 恒在链首;缺省/空数组 → 自动发现全部 image-capable
|
|
30
|
-
* 模型并按 Provider.source 档序排列。链上候选逐个尝试,成功即止,全败聚合报错。
|
|
31
|
-
* 描述子会话的模型属于候选链,chat.message 递归防护以整链成员为集。
|
|
5
|
+
* 插件把图片落盘到 vision 目录(git 项目内 → <项目>/.opencode/vision;
|
|
6
|
+
* 非 git 目录 → 用户级缓存目录),并向模型注入一条 synthetic 提示
|
|
7
|
+
* (TUI 界面隐藏、模型可见),引导它通过 vision_analyze 工具让指定的视觉
|
|
8
|
+
* 模型描述图片。若主模型本身支持图片输入,则不做任何干预,原图直接发给主模型。
|
|
32
9
|
*
|
|
33
10
|
* 工作方式(vision_analyze 工具路径):主模型调用 vision_analyze 时,插件
|
|
34
11
|
* 创建一个 parentID 挂在当前会话下的临时子会话(不进会话列表、不生成
|
|
@@ -43,17 +20,13 @@
|
|
|
43
20
|
* 类型依赖仅 @opencode-ai/plugin 与 @opencode-ai/sdk 的 type import。
|
|
44
21
|
*
|
|
45
22
|
* 已知限制:
|
|
46
|
-
* - SSRF 面:downloadImage 的 fetch 跟随重定向、不拦截私网/云元数据地址。
|
|
47
|
-
* 本地单用户 CLI 的信任级别下可接受;生产多租户环境使用前应加私网
|
|
48
|
-
* 地址拦截。
|
|
49
|
-
* - 中止不传导:用户中止不会取消进行中的下载/子会话请求,最长空跑至
|
|
50
|
-
* 各自的 deadline(下载 30 秒、子会话 timeout_ms);超时/中止后子会话
|
|
51
|
-
* 虽被删除,但 provider 端已发出的孤儿回合仍可能计入用量。
|
|
52
23
|
* - 仅 V1 会话流有效:chat.message 钩子挂在 V1 SessionPrompt 路径上;
|
|
53
24
|
* 若交互默认切到 V2 Session 核心,本钩子不会触发(也不会报错)。
|
|
54
25
|
*/
|
|
55
26
|
import { createHash, randomUUID } from "node:crypto";
|
|
27
|
+
import { existsSync } from "node:fs";
|
|
56
28
|
import fs from "node:fs/promises";
|
|
29
|
+
import { homedir } from "node:os";
|
|
57
30
|
import path from "node:path";
|
|
58
31
|
/** 支持的图片扩展名 → MIME 类型(vision_analyze 加载磁盘图片时使用) */
|
|
59
32
|
const EXT_MIME = {
|
|
@@ -70,6 +43,22 @@ const MIME_EXT = {
|
|
|
70
43
|
"image/gif": ".gif",
|
|
71
44
|
"image/webp": ".webp",
|
|
72
45
|
};
|
|
46
|
+
/**
|
|
47
|
+
* 内部超时错误类型(name = "DeadlineError")。
|
|
48
|
+
* 与 AbortError 并列可判别:超时路径(providers 查询 / 子会话请求)据此判断
|
|
49
|
+
* "底层请求可能仍在飞",供调用方决定是否需要 abort 取消(见 attemptModel)。
|
|
50
|
+
*/
|
|
51
|
+
class DeadlineError extends Error {
|
|
52
|
+
constructor(message) {
|
|
53
|
+
super(message);
|
|
54
|
+
this.name = "DeadlineError";
|
|
55
|
+
}
|
|
56
|
+
}
|
|
57
|
+
/**
|
|
58
|
+
* `config.providers()` 能力查询的超时预算(毫秒)。做成可改写对象(而非常量/选项):
|
|
59
|
+
* 避免选项膨胀;测试把 ms 调小即可缩短等待(TS 不允许对 import 的 let 绑定赋值)。
|
|
60
|
+
*/
|
|
61
|
+
export const providersTimeout = { ms: 5000 };
|
|
73
62
|
/**
|
|
74
63
|
* 视觉子会话使用的系统提示词。
|
|
75
64
|
* 要求:精确转录图中文字,描述 UI/布局/对象/颜色等,优先回答用户问题,
|
|
@@ -84,6 +73,46 @@ const VISION_SYSTEM_PROMPT = [
|
|
|
84
73
|
].join("\n");
|
|
85
74
|
/** data URL 形如 data:<mime>;base64,<payload> */
|
|
86
75
|
const DATA_URL_PATTERN = /^data:([^;]+);base64,(.+)$/;
|
|
76
|
+
/**
|
|
77
|
+
* 判断目录是否位于 git 项目内:从 dir 向上(含自身)逐级找 `.git`
|
|
78
|
+
* (目录,或 worktree/submodule 的 `.git` 指针文件),到文件系统根为止。
|
|
79
|
+
* 纯同步、纯内置模块;作为命名导出便于单元测试注入真实临时目录验证。
|
|
80
|
+
*/
|
|
81
|
+
export function isInsideGitRepo(dir) {
|
|
82
|
+
let cur = path.resolve(dir);
|
|
83
|
+
for (;;) {
|
|
84
|
+
if (existsSync(path.join(cur, ".git")))
|
|
85
|
+
return true;
|
|
86
|
+
const parent = path.dirname(cur);
|
|
87
|
+
if (parent === cur)
|
|
88
|
+
return false; // 已到文件系统根
|
|
89
|
+
cur = parent;
|
|
90
|
+
}
|
|
91
|
+
}
|
|
92
|
+
/**
|
|
93
|
+
* 用户级(非 git)图片缓存的平台根:cache 目录 + opencode-vision-analyze/vision。
|
|
94
|
+
* 空字符串 env 视为未设置(XDG/LOCALAPPDATA 官规:空值=未设置)——避免把 "" 当
|
|
95
|
+
* 有效根导致 path.join("",…) 产出相对路径、相对进程 cwd 落盘。
|
|
96
|
+
* darwin 亦接受 $XDG_CACHE_HOME 覆盖(跨平台统一 dotfiles 的宽容超集)。
|
|
97
|
+
*/
|
|
98
|
+
export function userVisionCacheRoot(env, platform, home) {
|
|
99
|
+
const base = platform === "darwin"
|
|
100
|
+
? (env.XDG_CACHE_HOME || path.join(home, "Library", "Caches"))
|
|
101
|
+
: platform === "win32"
|
|
102
|
+
? (env.LOCALAPPDATA || path.join(home, "AppData", "Local"))
|
|
103
|
+
: (env.XDG_CACHE_HOME || path.join(home, ".cache"));
|
|
104
|
+
return path.join(base, "opencode-vision-analyze", "vision");
|
|
105
|
+
}
|
|
106
|
+
/**
|
|
107
|
+
* 图片存储根:与 opencode 的项目/全局语义对齐——
|
|
108
|
+
* git 项目内 → <项目>/.opencode/vision(现状);非 git 目录 → 用户级缓存目录。
|
|
109
|
+
* 解析一次即可(纯函数,入参可注入以便三平台与 env 覆盖的单测)。
|
|
110
|
+
*/
|
|
111
|
+
export function resolveVisionDir(inputDir, env, platform, home) {
|
|
112
|
+
if (isInsideGitRepo(inputDir))
|
|
113
|
+
return path.join(path.resolve(inputDir), ".opencode", "vision");
|
|
114
|
+
return userVisionCacheRoot(env, platform, home);
|
|
115
|
+
}
|
|
87
116
|
/**
|
|
88
117
|
* http(s) 下载图片的大小上限(20 MB)。提示注入可让模型指向超大图片,
|
|
89
118
|
* 下载不限长是成本/健壮性放大器:先按 content-length 头提前拒绝,
|
|
@@ -151,33 +180,49 @@ const plugin = async (input, optionsArg) => {
|
|
|
151
180
|
return error;
|
|
152
181
|
return JSON.stringify(error) ?? String(error);
|
|
153
182
|
};
|
|
183
|
+
/**
|
|
184
|
+
* 无 ToolContext 的超时原语:到期以 DeadlineError(timeoutMessage) 拒绝。
|
|
185
|
+
* 与 withDeadline 的差别是它不感知 abort——能力查询(providers)等无 ctx 的
|
|
186
|
+
* 请求只关心"别永久挂起",不需要监听用户中止;子会话请求由 withDeadline 组合
|
|
187
|
+
* abort 信号后复用它,保证全插件只有一套计时机制。
|
|
188
|
+
*/
|
|
189
|
+
const withTimeout = async (promise, ms, timeoutMessage) => {
|
|
190
|
+
let timer;
|
|
191
|
+
const guard = new Promise((_, reject) => {
|
|
192
|
+
timer = setTimeout(() => reject(new DeadlineError(timeoutMessage)), ms);
|
|
193
|
+
});
|
|
194
|
+
try {
|
|
195
|
+
return await Promise.race([promise, guard]);
|
|
196
|
+
}
|
|
197
|
+
finally {
|
|
198
|
+
if (timer)
|
|
199
|
+
clearTimeout(timer);
|
|
200
|
+
}
|
|
201
|
+
};
|
|
154
202
|
/**
|
|
155
203
|
* 给子会话请求加超时与 abort 保护:任一触发即让 Promise 以错误结束,
|
|
156
|
-
*
|
|
204
|
+
* 不再等待底层请求;超时复用 withTimeout(DeadlineError),abort 仍以
|
|
205
|
+
* AbortError 拒绝;finally 中清理 timer 与监听器,避免泄漏。
|
|
157
206
|
*/
|
|
158
207
|
const withDeadline = (promise, ctx) => {
|
|
159
|
-
|
|
208
|
+
// ctx.abort 已中止时 abort 事件不会再触发,必须立即拒绝,
|
|
209
|
+
// 否则 race 只能干等 timer 超时。
|
|
210
|
+
if (ctx.abort.aborted)
|
|
211
|
+
return Promise.reject(new DOMException("Aborted", "AbortError"));
|
|
160
212
|
let onAbort;
|
|
161
|
-
const
|
|
162
|
-
// ctx.abort 已中止时 abort 事件不会再触发,必须立即拒绝,
|
|
163
|
-
// 否则 race 只能干等 timer 超时。
|
|
164
|
-
if (ctx.abort.aborted) {
|
|
165
|
-
reject(new DOMException("Aborted", "AbortError"));
|
|
166
|
-
return;
|
|
167
|
-
}
|
|
168
|
-
timer = setTimeout(() => reject(new Error(`vision model call timed out after ${timeoutMs}ms`)), timeoutMs);
|
|
213
|
+
const abortGuard = new Promise((_, reject) => {
|
|
169
214
|
onAbort = () => reject(new DOMException("Aborted", "AbortError"));
|
|
170
215
|
ctx.abort.addEventListener("abort", onAbort, { once: true });
|
|
171
216
|
});
|
|
172
|
-
return Promise.race([promise,
|
|
173
|
-
if (timer)
|
|
174
|
-
clearTimeout(timer);
|
|
217
|
+
return withTimeout(Promise.race([promise, abortGuard]), timeoutMs, `vision model call timed out after ${timeoutMs}ms`).finally(() => {
|
|
175
218
|
if (onAbort)
|
|
176
219
|
ctx.abort.removeEventListener("abort", onAbort);
|
|
177
220
|
});
|
|
178
221
|
};
|
|
179
222
|
/** 判断错误是否为中止信号(AbortError),供 attemptModel 标记 / 链循环中止整链。 */
|
|
180
223
|
const isAbortError = (error) => error instanceof Error && error.name === "AbortError";
|
|
224
|
+
/** 判断错误是否为本插件超时信号(DeadlineError)——与 AbortError 并列,表示"请求到期被本地掐断"。 */
|
|
225
|
+
const isDeadlineError = (error) => error instanceof Error && error.name === "DeadlineError";
|
|
181
226
|
/**
|
|
182
227
|
* 单个候选的尝试:创建子会话(parentID 挂当前会话)→ 用该候选模型描述 →
|
|
183
228
|
* 删除子会话。任何失败(创建 / 请求 / 超时 / 中止 / 底层抛错 / 无文本)都
|
|
@@ -188,6 +233,9 @@ const plugin = async (input, optionsArg) => {
|
|
|
188
233
|
const attemptModel = async (candidate, image, question, ctx) => {
|
|
189
234
|
const dataURL = `data:${image.mime};base64,${image.bytes.toString("base64")}`;
|
|
190
235
|
let subID;
|
|
236
|
+
// "回合可能仍在飞"标记:请求被本地 deadline(超时)或用户 abort 掐断时置位,
|
|
237
|
+
// finally 据此先 abort 子会话(取消 provider 端孤儿回合)再 delete。
|
|
238
|
+
let endedByDeadline = false;
|
|
191
239
|
try {
|
|
192
240
|
const created = await withDeadline(input.client.session.create({ body: { parentID: ctx.sessionID, title: "vision analysis" } }), ctx);
|
|
193
241
|
if (created.error || !created.data) {
|
|
@@ -222,12 +270,19 @@ const plugin = async (input, optionsArg) => {
|
|
|
222
270
|
return { ok: true, text };
|
|
223
271
|
}
|
|
224
272
|
catch (error) {
|
|
225
|
-
// 异常(超时 / 中止 / 底层抛错)同样收敛为失败结果;aborted
|
|
273
|
+
// 异常(超时 / 中止 / 底层抛错)同样收敛为失败结果;aborted 标记交由链循环判断。
|
|
274
|
+
// 超时与中止都意味着底层请求可能仍在飞 → 需要先 abort 再 delete。
|
|
275
|
+
endedByDeadline = isDeadlineError(error) || isAbortError(error);
|
|
226
276
|
return { ok: false, error: errText(error), aborted: isAbortError(error) };
|
|
227
277
|
}
|
|
228
278
|
finally {
|
|
229
279
|
if (subID) {
|
|
230
280
|
subSessions.delete(subID);
|
|
281
|
+
// 先 abort(best-effort,取消 provider 端孤儿回合)再 delete;
|
|
282
|
+
// 成功/普通失败路径 turn 已自然结束,无需 abort。
|
|
283
|
+
if (endedByDeadline) {
|
|
284
|
+
await input.client.session.abort({ path: { id: subID } }).catch(() => { });
|
|
285
|
+
}
|
|
231
286
|
await input.client.session.delete({ path: { id: subID } }).catch(() => { });
|
|
232
287
|
}
|
|
233
288
|
}
|
|
@@ -263,13 +318,41 @@ const plugin = async (input, optionsArg) => {
|
|
|
263
318
|
/** 描述标签:标注图片文件名与产出描述的模型引用键(basename 运行时按图传入)。 */
|
|
264
319
|
const format = (basename, modelId, text) => `[Image: ${basename} — described by ${modelId}]\n${text}`;
|
|
265
320
|
/**
|
|
266
|
-
*
|
|
267
|
-
*
|
|
268
|
-
*
|
|
269
|
-
*
|
|
321
|
+
* 把图片字节内容原子落盘并返回最终 filepath。每次调用现算存储根
|
|
322
|
+
* (git 项目 → 项目 .opencode/vision;非 git → 用户级缓存,见 resolveVisionDir):
|
|
323
|
+
* 运行中存储范围变化(如 git init)从下一条图片起即时生效,无需重启。
|
|
324
|
+
* 先写 `<sha><ext>.tmp-<uuid>` 再 rename:共享目录(用户级/多实例)并发写同一 sha
|
|
325
|
+
* 时内容寻址下原子幂等;失败先清理临时文件再抛错,避免孤儿 tmp 累积。
|
|
326
|
+
*/
|
|
327
|
+
const persistImageBytes = async (bytes, ext) => {
|
|
328
|
+
const dir = resolveVisionDir(input.directory, process.env, process.platform, homedir());
|
|
329
|
+
const sha = createHash("sha256").update(bytes).digest("hex");
|
|
330
|
+
const filepath = path.join(dir, `${sha}${ext}`);
|
|
331
|
+
const tmpPath = path.join(dir, `${sha}${ext}.tmp-${randomUUID()}`);
|
|
332
|
+
await fs.mkdir(dir, { recursive: true, mode: 0o700 });
|
|
333
|
+
try {
|
|
334
|
+
await fs.writeFile(tmpPath, bytes);
|
|
335
|
+
await fs.rename(tmpPath, filepath);
|
|
336
|
+
}
|
|
337
|
+
catch (error) {
|
|
338
|
+
await fs.unlink(tmpPath).catch(() => { });
|
|
339
|
+
throw error;
|
|
340
|
+
}
|
|
341
|
+
return filepath;
|
|
342
|
+
};
|
|
343
|
+
/**
|
|
344
|
+
* 下载 http(s) URL 指向的图片并落盘(内容哈希命名天然去重,写盘细节见
|
|
345
|
+
* persistImageBytes)。扩展名不受支持、HTTP 非 2xx、网络失败(含 30 秒下载
|
|
346
|
+
* 超时)、超过 20 MB 下载上限(content-length 预检 + 读后复核)都返回 { error },
|
|
270
347
|
* 由调用方转成可读的错误文字。
|
|
348
|
+
*
|
|
349
|
+
* 中止传导:abort(用户中止)与 30 秒超时共同驱动一个 AbortController——
|
|
350
|
+
* 中止即刻断请求,不空跑满超时;pre-abort 直接放弃、不发请求。手动组合信号
|
|
351
|
+
* 而非 AbortSignal.any():engines node>=18(any 需 18.17+/20.3+),且与
|
|
352
|
+
* withDeadline 的 addEventListener 风格同构。错误以 AbortError/超时形式落入
|
|
353
|
+
* catch,统一转可读文字。
|
|
271
354
|
*/
|
|
272
|
-
const downloadImage = async (url) => {
|
|
355
|
+
const downloadImage = async (url, abort) => {
|
|
273
356
|
try {
|
|
274
357
|
// URL 解析与扩展名提取放在 try 内:畸形 URL 在 new URL 处抛错时,
|
|
275
358
|
// 错误以 "Image download failed" 前缀返回,而不是漏到外层的
|
|
@@ -278,24 +361,34 @@ const plugin = async (input, optionsArg) => {
|
|
|
278
361
|
const mime = EXT_MIME[ext];
|
|
279
362
|
if (!mime)
|
|
280
363
|
return { error: `unsupported image URL extension: ${ext || "(none)"}` };
|
|
281
|
-
|
|
282
|
-
if (
|
|
283
|
-
return { error:
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
|
|
364
|
+
// pre-abort:已中止则不必发请求,直接以 Aborted 收尾
|
|
365
|
+
if (abort.aborted)
|
|
366
|
+
return { error: "Aborted" };
|
|
367
|
+
const controller = new AbortController();
|
|
368
|
+
const onAbort = () => controller.abort();
|
|
369
|
+
abort.addEventListener("abort", onAbort, { once: true });
|
|
370
|
+
const timer = setTimeout(() => controller.abort(), 30_000);
|
|
371
|
+
try {
|
|
372
|
+
// 整个下载(fetch 响应头 + arrayBuffer 读 body)都在同一 guard 内:
|
|
373
|
+
// 30 秒预算与用户中止都覆盖到 body 读取阶段;收尾再清 timer/listener。
|
|
374
|
+
const response = await fetch(url, { signal: controller.signal });
|
|
375
|
+
if (!response.ok)
|
|
376
|
+
return { error: `HTTP ${response.status}` };
|
|
377
|
+
// 头字段缺失时 Number(null) 为 NaN,比较结果为 false,自然放行到读后复核。
|
|
378
|
+
if (Number(response.headers.get("content-length")) > MAX_DOWNLOAD_BYTES) {
|
|
379
|
+
return { error: "image exceeds 20 MB download limit" };
|
|
380
|
+
}
|
|
381
|
+
const bytes = Buffer.from(await response.arrayBuffer());
|
|
382
|
+
// 复核实际字节数:chunked 等无 content-length 的响应只有读后才能判大小。
|
|
383
|
+
if (bytes.length > MAX_DOWNLOAD_BYTES) {
|
|
384
|
+
return { error: "image exceeds 20 MB download limit" };
|
|
385
|
+
}
|
|
386
|
+
return { filepath: await persistImageBytes(bytes, ext) };
|
|
287
387
|
}
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
|
|
291
|
-
return { error: "image exceeds 20 MB download limit" };
|
|
388
|
+
finally {
|
|
389
|
+
clearTimeout(timer);
|
|
390
|
+
abort.removeEventListener("abort", onAbort);
|
|
292
391
|
}
|
|
293
|
-
const sha = createHash("sha256").update(bytes).digest("hex");
|
|
294
|
-
const dir = path.join(input.directory, ".opencode", "vision");
|
|
295
|
-
await fs.mkdir(dir, { recursive: true });
|
|
296
|
-
const filepath = path.join(dir, `${sha}${ext}`);
|
|
297
|
-
await fs.writeFile(filepath, bytes);
|
|
298
|
-
return { filepath };
|
|
299
392
|
}
|
|
300
393
|
catch (error) {
|
|
301
394
|
return { error: errText(error) };
|
|
@@ -328,7 +421,10 @@ const plugin = async (input, optionsArg) => {
|
|
|
328
421
|
try {
|
|
329
422
|
const question = args.question?.trim() || "Describe this image in full detail.";
|
|
330
423
|
// http(s) URL:先下载到本地 vision 目录,再统一走磁盘加载路径。
|
|
331
|
-
|
|
424
|
+
// ctx.abort 传入下载:用户中止即刻断下载(含 pre-abort 不再发请求)。
|
|
425
|
+
const download = /^https?:\/\//i.test(args.image_path)
|
|
426
|
+
? await downloadImage(args.image_path, ctx.abort)
|
|
427
|
+
: undefined;
|
|
332
428
|
if (download && "error" in download) {
|
|
333
429
|
return { title, output: `Image download failed: ${download.error}` };
|
|
334
430
|
}
|
|
@@ -385,7 +481,7 @@ const plugin = async (input, optionsArg) => {
|
|
|
385
481
|
if (cached !== undefined)
|
|
386
482
|
return cached;
|
|
387
483
|
try {
|
|
388
|
-
const result = await input.client.config.providers();
|
|
484
|
+
const result = await withTimeout(input.client.config.providers(), providersTimeout.ms, `config.providers() timed out after ${providersTimeout.ms}ms`);
|
|
389
485
|
// HTTP 非 2xx 时 openapi-fetch 不抛错而是返回 { error }(data 为空)。
|
|
390
486
|
// 「查询失败」不能缓存成 false——那是一次瞬时故障而非「确认不支持」,
|
|
391
487
|
// 缓存会永久关闭能力门控;本次保守返回 false,下次再重试。
|
|
@@ -436,7 +532,7 @@ const plugin = async (input, optionsArg) => {
|
|
|
436
532
|
*/
|
|
437
533
|
const listImageCapableModels = async () => {
|
|
438
534
|
try {
|
|
439
|
-
const result = await input.client.config.providers();
|
|
535
|
+
const result = await withTimeout(input.client.config.providers(), providersTimeout.ms, `config.providers() timed out after ${providersTimeout.ms}ms`);
|
|
440
536
|
if (!result.data)
|
|
441
537
|
return [];
|
|
442
538
|
const found = [];
|
|
@@ -466,8 +562,7 @@ const plugin = async (input, optionsArg) => {
|
|
|
466
562
|
return chain.some((c) => c.providerID === model.providerID && c.modelID === model.modelID);
|
|
467
563
|
};
|
|
468
564
|
/**
|
|
469
|
-
* 把一个图片 file part
|
|
470
|
-
* 文件名用内容哈希,天然去重(同一张图多次发送只落一份)。
|
|
565
|
+
* 把一个图片 file part 落盘(内容哈希命名去重,写盘细节见 persistImageBytes)。
|
|
471
566
|
* 返回落盘信息;MIME 不受支持或 URL 不是 base64 data URL 时返回 undefined。
|
|
472
567
|
*/
|
|
473
568
|
const persistImage = async (part) => {
|
|
@@ -479,12 +574,7 @@ const plugin = async (input, optionsArg) => {
|
|
|
479
574
|
return undefined;
|
|
480
575
|
try {
|
|
481
576
|
const bytes = Buffer.from(match[2], "base64");
|
|
482
|
-
|
|
483
|
-
const dir = path.join(input.directory, ".opencode", "vision");
|
|
484
|
-
await fs.mkdir(dir, { recursive: true });
|
|
485
|
-
const filepath = path.join(dir, `${sha}${ext}`);
|
|
486
|
-
await fs.writeFile(filepath, bytes);
|
|
487
|
-
return { filepath };
|
|
577
|
+
return { filepath: await persistImageBytes(bytes, ext) };
|
|
488
578
|
}
|
|
489
579
|
catch {
|
|
490
580
|
// fail-open 原则:图片落盘失败(EACCES/ENOSPC 等)只是少了 vision_analyze
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "opencode-vision-analyze",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.3.0",
|
|
4
4
|
"description": "OpenCode plugin: vision_analyze tool — describe images & image URLs with your own vision model for text-only main models; native-image fast path for multimodal models. Zero runtime dependencies.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "./dist/index.js",
|