@yanickxia/agent-vision 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +41 -28
- package/dist/analyze.d.ts.map +1 -1
- package/dist/analyze.js +3 -3
- package/dist/analyze.js.map +1 -1
- package/dist/cli.js +15 -8
- package/dist/cli.js.map +1 -1
- package/dist/config.d.ts +12 -5
- package/dist/config.d.ts.map +1 -1
- package/dist/config.js +203 -30
- package/dist/config.js.map +1 -1
- package/dist/provider.d.ts +8 -1
- package/dist/provider.d.ts.map +1 -1
- package/dist/provider.js +69 -9
- package/dist/provider.js.map +1 -1
- package/dist/server.d.ts +1 -1
- package/dist/server.js +2 -2
- package/dist/server.js.map +1 -1
- package/dist/video.d.ts +1 -1
- package/dist/video.d.ts.map +1 -1
- package/dist/video.js +8 -8
- package/dist/video.js.map +1 -1
- package/package.json +1 -1
- package/skills/agent-vision/SKILL.md +28 -44
- package/skills/agent-vision/references/configuration.md +51 -2
package/README.md
CHANGED
|
@@ -5,10 +5,10 @@ OpenAI-compatible 视觉模型理解图片和视频。
|
|
|
5
5
|
|
|
6
6
|
- 纯 Node.js / TypeScript,`npx` 自动下载运行
|
|
7
7
|
- MCP 工具:`analyze_image`、`analyze_video`
|
|
8
|
-
- Agent Skills 标准目录:`skills/agent-vision
|
|
8
|
+
- Agent Skills 标准目录:`skills/agent-vision/`,能力自适应:能看图的 Agent 直接看,纯文本 Agent 直接跑 CLI
|
|
9
9
|
- 图片:本地路径、HTTP(S) URL、Data URI
|
|
10
10
|
- 视频:本地路径或 URL,自动下载 npm 内的 ffmpeg 并均匀抽帧
|
|
11
|
-
-
|
|
11
|
+
- 多 provider 多模型并发竞速:所有 target 同时请求,谁先成功用谁,其余请求 abort
|
|
12
12
|
|
|
13
13
|
## 为什么做这个项目
|
|
14
14
|
|
|
@@ -31,13 +31,38 @@ ffmpeg 由 `@ffmpeg-installer/ffmpeg` 按平台自动安装,无需另行安装
|
|
|
31
31
|
|
|
32
32
|
## 配置
|
|
33
33
|
|
|
34
|
+
推荐用配置文件(一次配置,所有宿主共用):
|
|
35
|
+
|
|
34
36
|
```bash
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
37
|
+
mkdir -p ~/.config/agent-vision
|
|
38
|
+
cat > ~/.config/agent-vision/settings.json <<'EOF'
|
|
39
|
+
{
|
|
40
|
+
"providers": [
|
|
41
|
+
{
|
|
42
|
+
"name": "dashscope",
|
|
43
|
+
"baseUrl": "https://dashscope.aliyuncs.com/compatible-mode/v1",
|
|
44
|
+
"apiKey": "sk-...",
|
|
45
|
+
"models": ["qwen-vl-max", "qwen-vl-plus"]
|
|
46
|
+
},
|
|
47
|
+
{
|
|
48
|
+
"name": "openai",
|
|
49
|
+
"baseUrl": "https://api.openai.com/v1",
|
|
50
|
+
"apiKey": "sk-...",
|
|
51
|
+
"models": ["gpt-4o"]
|
|
52
|
+
}
|
|
53
|
+
]
|
|
54
|
+
}
|
|
55
|
+
EOF
|
|
56
|
+
chmod 600 ~/.config/agent-vision/settings.json
|
|
38
57
|
```
|
|
39
58
|
|
|
40
|
-
|
|
59
|
+
- 位置:`$AGENT_VISION_CONFIG` 显式路径,或 `$XDG_CONFIG_HOME/agent-vision/settings.json`(默认 `~/.config/agent-vision/settings.json`)
|
|
60
|
+
- 每个 provider × model 是一个 target;多个 target 时**并发请求,第一个成功者胜出**,其余请求 abort。注意成本:N 个 target 意味着每次分析最多 N 份上传与推理费用
|
|
61
|
+
- provider 字段:`baseUrl`(必填)、`models`(必填非空)、`name`/`apiKey`/`headers`(可选;`apiKey` 缺省回退环境变量)
|
|
62
|
+
- 顶层可选:`maxTokens`、`timeoutMs`、`headers`、`maxImageMb`、`maxVideoMb`、`allowPrivateUrls`、`ffmpegPath`
|
|
63
|
+
- 文件定义的字段优先于环境变量;含 key 的文件建议 `chmod 600`
|
|
64
|
+
|
|
65
|
+
环境变量作为回退仍然完整支持(无配置文件时):
|
|
41
66
|
|
|
42
67
|
| 设置 | 兼容回退 |
|
|
43
68
|
|---|---|
|
|
@@ -59,19 +84,14 @@ export AGENT_VISION_MODEL="your-vision-model"
|
|
|
59
84
|
|
|
60
85
|
## MCP 安装
|
|
61
86
|
|
|
62
|
-
### npm
|
|
87
|
+
### npm(推荐)
|
|
63
88
|
|
|
64
89
|
```json
|
|
65
90
|
{
|
|
66
91
|
"mcpServers": {
|
|
67
92
|
"agent-vision": {
|
|
68
93
|
"command": "npx",
|
|
69
|
-
"args": ["-y", "@yanickxia/agent-vision"]
|
|
70
|
-
"env": {
|
|
71
|
-
"AGENT_VISION_BASE_URL": "https://your-endpoint.example/v1",
|
|
72
|
-
"AGENT_VISION_API_KEY": "your-key",
|
|
73
|
-
"AGENT_VISION_MODEL": "your-vision-model"
|
|
74
|
-
}
|
|
94
|
+
"args": ["-y", "@yanickxia/agent-vision"]
|
|
75
95
|
}
|
|
76
96
|
}
|
|
77
97
|
}
|
|
@@ -84,17 +104,15 @@ export AGENT_VISION_MODEL="your-vision-model"
|
|
|
84
104
|
"mcpServers": {
|
|
85
105
|
"agent-vision": {
|
|
86
106
|
"command": "npx",
|
|
87
|
-
"args": ["-y", "github:yanickxia/agent-vision"]
|
|
88
|
-
"env": {
|
|
89
|
-
"AGENT_VISION_BASE_URL": "https://your-endpoint.example/v1",
|
|
90
|
-
"AGENT_VISION_API_KEY": "your-key",
|
|
91
|
-
"AGENT_VISION_MODEL": "your-vision-model"
|
|
92
|
-
}
|
|
107
|
+
"args": ["-y", "github:yanickxia/agent-vision"]
|
|
93
108
|
}
|
|
94
109
|
}
|
|
95
110
|
}
|
|
96
111
|
```
|
|
97
112
|
|
|
113
|
+
无需 env 块:配置从 `~/.config/agent-vision/settings.json` 读取(见上文「配置」)。
|
|
114
|
+
也可以继续用环境变量,在 `env` / `environment` 块里传入。
|
|
115
|
+
|
|
98
116
|
Claude Desktop、Claude Code、Cursor、Cline 等使用上面的标准
|
|
99
117
|
`mcpServers` 格式。
|
|
100
118
|
|
|
@@ -106,12 +124,7 @@ Claude Desktop、Claude Code、Cursor、Cline 等使用上面的标准
|
|
|
106
124
|
"agent-vision": {
|
|
107
125
|
"type": "local",
|
|
108
126
|
"command": ["npx", "-y", "@yanickxia/agent-vision"],
|
|
109
|
-
"enabled": true
|
|
110
|
-
"environment": {
|
|
111
|
-
"AGENT_VISION_BASE_URL": "https://your-endpoint.example/v1",
|
|
112
|
-
"AGENT_VISION_API_KEY": "your-key",
|
|
113
|
-
"AGENT_VISION_MODEL": "your-vision-model"
|
|
114
|
-
}
|
|
127
|
+
"enabled": true
|
|
115
128
|
}
|
|
116
129
|
}
|
|
117
130
|
}
|
|
@@ -123,9 +136,9 @@ Claude Desktop、Claude Code、Cursor、Cline 等使用上面的标准
|
|
|
123
136
|
npx skills add yanickxia/agent-vision --skill agent-vision -g -y
|
|
124
137
|
```
|
|
125
138
|
|
|
126
|
-
也可以复制 `skills/agent-vision/` 到 Agent 的 skills 目录。Skill
|
|
127
|
-
Agent
|
|
128
|
-
|
|
139
|
+
也可以复制 `skills/agent-vision/` 到 Agent 的 skills 目录。Skill 是能力自适应
|
|
140
|
+
的:Agent 自己能看图就直接看;看不了(或输入是视频)就由 Skill 指导 Agent
|
|
141
|
+
直接跑 CLI 完成分析。
|
|
129
142
|
|
|
130
143
|
## CLI
|
|
131
144
|
|
package/dist/analyze.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"analyze.d.ts","sourceRoot":"","sources":["../src/analyze.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAGhD,OAAO,
|
|
1
|
+
{"version":3,"file":"analyze.d.ts","sourceRoot":"","sources":["../src/analyze.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAGhD,OAAO,EAAyB,KAAK,gBAAgB,EAAE,MAAM,eAAe,CAAC;AAS7E,wBAAsB,YAAY,CAAC,OAAO,EAAE;IAC1C,MAAM,EAAE,MAAM,CAAC;IACf,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,MAAM,CAAC,EAAE,YAAY,CAAC;CACvB,GAAG,OAAO,CAAC,gBAAgB,CAAC,CAQ5B;AAED,wBAAsB,YAAY,CAAC,OAAO,EAAE;IAC1C,MAAM,EAAE,MAAM,CAAC;IACf,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,MAAM,CAAC,EAAE,YAAY,CAAC;CACvB,GAAG,OAAO,CAAC,gBAAgB,CAAC,CAe5B"}
|
package/dist/analyze.js
CHANGED
|
@@ -1,13 +1,13 @@
|
|
|
1
1
|
import { loadConfig } from "./config.js";
|
|
2
2
|
import { loadImage } from "./image.js";
|
|
3
|
-
import {
|
|
3
|
+
import { raceVisionCompletions } from "./provider.js";
|
|
4
4
|
import { extractVideoFrames } from "./video.js";
|
|
5
5
|
const DEFAULT_IMAGE_PROMPT = "Describe this image accurately. Include visible text, layout, objects, colors, and details relevant to understanding it. Do not invent details that are not visible.";
|
|
6
6
|
const DEFAULT_VIDEO_PROMPT = "Analyze these evenly sampled video frames as one timeline. Describe the sequence of events, scene or UI changes, actions, and visible text. Distinguish observations from uncertainty.";
|
|
7
7
|
export async function analyzeImage(options) {
|
|
8
8
|
const config = options.config ?? loadConfig();
|
|
9
9
|
const image = await loadImage(options.source, config);
|
|
10
|
-
return
|
|
10
|
+
return raceVisionCompletions({
|
|
11
11
|
config,
|
|
12
12
|
prompt: options.prompt?.trim() || DEFAULT_IMAGE_PROMPT,
|
|
13
13
|
images: [{ dataUrl: image.dataUrl, label: image.label }],
|
|
@@ -21,7 +21,7 @@ export async function analyzeVideo(options) {
|
|
|
21
21
|
config,
|
|
22
22
|
});
|
|
23
23
|
const userPrompt = options.prompt?.trim();
|
|
24
|
-
return
|
|
24
|
+
return raceVisionCompletions({
|
|
25
25
|
config,
|
|
26
26
|
prompt: userPrompt
|
|
27
27
|
? `${DEFAULT_VIDEO_PROMPT}\n\nUser question: ${userPrompt}`
|
package/dist/analyze.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"analyze.js","sourceRoot":"","sources":["../src/analyze.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AACzC,OAAO,EAAE,SAAS,EAAE,MAAM,YAAY,CAAC;AACvC,OAAO,EAAE,
|
|
1
|
+
{"version":3,"file":"analyze.js","sourceRoot":"","sources":["../src/analyze.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AACzC,OAAO,EAAE,SAAS,EAAE,MAAM,YAAY,CAAC;AACvC,OAAO,EAAE,qBAAqB,EAAyB,MAAM,eAAe,CAAC;AAC7E,OAAO,EAAE,kBAAkB,EAAE,MAAM,YAAY,CAAC;AAEhD,MAAM,oBAAoB,GACxB,sKAAsK,CAAC;AAEzK,MAAM,oBAAoB,GACxB,wLAAwL,CAAC;AAE3L,MAAM,CAAC,KAAK,UAAU,YAAY,CAAC,OAIlC;IACC,MAAM,MAAM,GAAG,OAAO,CAAC,MAAM,IAAI,UAAU,EAAE,CAAC;IAC9C,MAAM,KAAK,GAAG,MAAM,SAAS,CAAC,OAAO,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IACtD,OAAO,qBAAqB,CAAC;QAC3B,MAAM;QACN,MAAM,EAAE,OAAO,CAAC,MAAM,EAAE,IAAI,EAAE,IAAI,oBAAoB;QACtD,MAAM,EAAE,CAAC,EAAE,OAAO,EAAE,KAAK,CAAC,OAAO,EAAE,KAAK,EAAE,KAAK,CAAC,KAAK,EAAE,CAAC;KACzD,CAAC,CAAC;AACL,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,YAAY,CAAC,OAKlC;IACC,MAAM,MAAM,GAAG,OAAO,CAAC,MAAM,IAAI,UAAU,EAAE,CAAC;IAC9C,MAAM,MAAM,GAAG,MAAM,kBAAkB,CAAC;QACtC,MAAM,EAAE,OAAO,CAAC,MAAM;QACtB,SAAS,EAAE,OAAO,CAAC,SAAS,IAAI,CAAC;QACjC,MAAM;KACP,CAAC,CAAC;IACH,MAAM,UAAU,GAAG,OAAO,CAAC,MAAM,EAAE,IAAI,EAAE,CAAC;IAC1C,OAAO,qBAAqB,CAAC;QAC3B,MAAM;QACN,MAAM,EAAE,UAAU;YAChB,CAAC,CAAC,GAAG,oBAAoB,sBAAsB,UAAU,EAAE;YAC3D,CAAC,CAAC,oBAAoB;QACxB,MAAM,EAAE,MAAM;KACf,CAAC,CAAC;AACL,CAAC"}
|
package/dist/cli.js
CHANGED
|
@@ -44,19 +44,26 @@ async function runCli(args) {
|
|
|
44
44
|
const config = loadConfig();
|
|
45
45
|
let ffmpeg = "unavailable";
|
|
46
46
|
try {
|
|
47
|
-
ffmpeg = await ensureFfmpeg();
|
|
47
|
+
ffmpeg = await ensureFfmpeg(config.ffmpegPath);
|
|
48
48
|
}
|
|
49
49
|
catch {
|
|
50
50
|
// Report below without failing before the full doctor output is printed.
|
|
51
51
|
}
|
|
52
|
-
const
|
|
53
|
-
`
|
|
54
|
-
`base URL: ${config.baseUrl}`,
|
|
55
|
-
`API key: ${config.apiKey ? "configured" : "not configured (allowed for local endpoints)"}`,
|
|
56
|
-
`ffmpeg: ${ffmpeg}`,
|
|
52
|
+
const lines = [
|
|
53
|
+
`config file: ${config.configFile ?? "not found (using environment variables)"}`,
|
|
57
54
|
];
|
|
58
|
-
|
|
59
|
-
|
|
55
|
+
if (config.targets.length === 0) {
|
|
56
|
+
lines.push("targets: none configured");
|
|
57
|
+
}
|
|
58
|
+
else {
|
|
59
|
+
lines.push("targets:");
|
|
60
|
+
config.targets.forEach((target, index) => {
|
|
61
|
+
lines.push(` ${index + 1}. ${target.label} ${target.baseUrl} key: ${target.apiKey ? "configured" : "not configured"}`);
|
|
62
|
+
});
|
|
63
|
+
}
|
|
64
|
+
lines.push(`ffmpeg: ${ffmpeg}`);
|
|
65
|
+
process.stdout.write(`${lines.join("\n")}\n`);
|
|
66
|
+
if (config.targets.length === 0 || ffmpeg === "unavailable")
|
|
60
67
|
process.exitCode = 1;
|
|
61
68
|
return;
|
|
62
69
|
}
|
package/dist/cli.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"cli.js","sourceRoot":"","sources":["../src/cli.ts"],"names":[],"mappings":";AAEA,OAAO,EAAE,YAAY,EAAE,YAAY,EAAE,MAAM,cAAc,CAAC;AAC1D,OAAO,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AACzC,OAAO,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAC3C,OAAO,EAAE,cAAc,EAAE,gBAAgB,EAAE,MAAM,aAAa,CAAC;AAC/D,OAAO,EAAE,YAAY,EAAE,MAAM,YAAY,CAAC;AAE1C,MAAM,IAAI,GAAG,gBAAgB,cAAc;;;;;;;;;;;;;CAa1C,CAAC;AAEF,SAAS,MAAM,CAAC,IAAc,EAAE,IAAY,EAAE,KAAc;IAC1D,MAAM,MAAM,GAAG,GAAG,IAAI,GAAG,CAAC;IAC1B,MAAM,MAAM,GAAG,IAAI,CAAC,IAAI,CAAC,CAAC,GAAG,EAAE,EAAE,CAAC,GAAG,CAAC,UAAU,CAAC,MAAM,CAAC,CAAC,CAAC;IAC1D,IAAI,MAAM;QAAE,OAAO,MAAM,CAAC,KAAK,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC;IAC/C,MAAM,KAAK,GAAG,IAAI,CAAC,SAAS,CAAC,CAAC,GAAG,EAAE,EAAE,CAAC,GAAG,KAAK,IAAI,IAAI,CAAC,KAAK,IAAI,GAAG,KAAK,KAAK,CAAC,CAAC,CAAC;IAChF,OAAO,KAAK,IAAI,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;AAClD,CAAC;AAED,KAAK,UAAU,MAAM,CAAC,IAAc;IAClC,MAAM,OAAO,GAAG,IAAI,CAAC,CAAC,CAAC,CAAC;IACxB,IAAI,CAAC,OAAO,EAAE,CAAC;QACb,MAAM,gBAAgB,EAAE,CAAC;QACzB,OAAO;IACT,CAAC;IACD,IAAI,OAAO,KAAK,QAAQ,IAAI,OAAO,KAAK,IAAI,IAAI,OAAO,KAAK,MAAM,EAAE,CAAC;QACnE,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;QAC3B,OAAO;IACT,CAAC;IACD,IAAI,OAAO,KAAK,WAAW,IAAI,OAAO,KAAK,IAAI,EAAE,CAAC;QAChD,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,cAAc,IAAI,CAAC,CAAC;QAC5C,OAAO;IACT,CAAC;IACD,IAAI,OAAO,KAAK,QAAQ,EAAE,CAAC;QACzB,MAAM,MAAM,GAAG,UAAU,EAAE,CAAC;QAC5B,IAAI,MAAM,GAAG,aAAa,CAAC;QAC3B,IAAI,CAAC;YACH,MAAM,GAAG,MAAM,YAAY,
|
|
1
|
+
{"version":3,"file":"cli.js","sourceRoot":"","sources":["../src/cli.ts"],"names":[],"mappings":";AAEA,OAAO,EAAE,YAAY,EAAE,YAAY,EAAE,MAAM,cAAc,CAAC;AAC1D,OAAO,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AACzC,OAAO,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAC3C,OAAO,EAAE,cAAc,EAAE,gBAAgB,EAAE,MAAM,aAAa,CAAC;AAC/D,OAAO,EAAE,YAAY,EAAE,MAAM,YAAY,CAAC;AAE1C,MAAM,IAAI,GAAG,gBAAgB,cAAc;;;;;;;;;;;;;CAa1C,CAAC;AAEF,SAAS,MAAM,CAAC,IAAc,EAAE,IAAY,EAAE,KAAc;IAC1D,MAAM,MAAM,GAAG,GAAG,IAAI,GAAG,CAAC;IAC1B,MAAM,MAAM,GAAG,IAAI,CAAC,IAAI,CAAC,CAAC,GAAG,EAAE,EAAE,CAAC,GAAG,CAAC,UAAU,CAAC,MAAM,CAAC,CAAC,CAAC;IAC1D,IAAI,MAAM;QAAE,OAAO,MAAM,CAAC,KAAK,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC;IAC/C,MAAM,KAAK,GAAG,IAAI,CAAC,SAAS,CAAC,CAAC,GAAG,EAAE,EAAE,CAAC,GAAG,KAAK,IAAI,IAAI,CAAC,KAAK,IAAI,GAAG,KAAK,KAAK,CAAC,CAAC,CAAC;IAChF,OAAO,KAAK,IAAI,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;AAClD,CAAC;AAED,KAAK,UAAU,MAAM,CAAC,IAAc;IAClC,MAAM,OAAO,GAAG,IAAI,CAAC,CAAC,CAAC,CAAC;IACxB,IAAI,CAAC,OAAO,EAAE,CAAC;QACb,MAAM,gBAAgB,EAAE,CAAC;QACzB,OAAO;IACT,CAAC;IACD,IAAI,OAAO,KAAK,QAAQ,IAAI,OAAO,KAAK,IAAI,IAAI,OAAO,KAAK,MAAM,EAAE,CAAC;QACnE,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;QAC3B,OAAO;IACT,CAAC;IACD,IAAI,OAAO,KAAK,WAAW,IAAI,OAAO,KAAK,IAAI,EAAE,CAAC;QAChD,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,cAAc,IAAI,CAAC,CAAC;QAC5C,OAAO;IACT,CAAC;IACD,IAAI,OAAO,KAAK,QAAQ,EAAE,CAAC;QACzB,MAAM,MAAM,GAAG,UAAU,EAAE,CAAC;QAC5B,IAAI,MAAM,GAAG,aAAa,CAAC;QAC3B,IAAI,CAAC;YACH,MAAM,GAAG,MAAM,YAAY,CAAC,MAAM,CAAC,UAAU,CAAC,CAAC;QACjD,CAAC;QAAC,MAAM,CAAC;YACP,yEAAyE;QAC3E,CAAC;QACD,MAAM,KAAK,GAAG;YACZ,gBAAgB,MAAM,CAAC,UAAU,IAAI,yCAAyC,EAAE;SACjF,CAAC;QACF,IAAI,MAAM,CAAC,OAAO,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;YAChC,KAAK,CAAC,IAAI,CAAC,0BAA0B,CAAC,CAAC;QACzC,CAAC;aAAM,CAAC;YACN,KAAK,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC;YACvB,MAAM,CAAC,OAAO,CAAC,OAAO,CAAC,CAAC,MAAM,EAAE,KAAK,EAAE,EAAE;gBACvC,KAAK,CAAC,IAAI,CACR,KAAK,KAAK,GAAG,CAAC,KAAK,MAAM,CAAC,KAAK,KAAK,MAAM,CAAC,OAAO,UAAU,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC,YAAY,CAAC,CAAC,CAAC,gBAAgB,EAAE,CAC9G,CAAC;YACJ,CAAC,CAAC,CAAC;QACL,CAAC;QACD,KAAK,CAAC,IAAI,CAAC,WAAW,MAAM,EAAE,CAAC,CAAC;QAChC,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;QAC9C,IAAI,MAAM,CAAC,OAAO,CAAC,MAAM,KAAK,CAAC,IAAI,MAAM,KAAK,aAAa;YAAE,OAAO,CAAC,QAAQ,GAAG,CAAC,CAAC;QAClF,OAAO;IACT,CAAC;IACD,IAAI,OAAO,KAAK,OAAO,IAAI,OAAO,KAAK,OAAO,EAAE,CAAC;QAC/C,MAAM,IAAI,KAAK,CAAC,oBAAoB,OAAO,OAAO,IAAI,EAAE,CAAC,CAAC;IAC5D,CAAC;IACD,MAAM,MAAM,GAAG,IAAI,CAAC,CAAC,CAAC,CAAC;IACvB,IAAI,CAAC,MAAM,IAAI,MAAM,CAAC,UAAU,CAAC,GAAG,CAAC,EAAE,CAAC;QACtC,MAAM,IAAI,KAAK,CAAC,GAAG,OAAO,gCAAgC,CAAC,CAAC;IAC9D,CAAC;IACD,MAAM,MAAM,GAAG,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,UAAU,EAAE,IAAI,CAAC,CAAC;IACvD,IAAI,OAAO,KAAK,OAAO,EAAE,CAAC;QACxB,MAAM,MAAM,GAAG,MAAM,YAAY,CAAC,EAAE,MAAM,EAAE,GAAG,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC,CAAC;QAC7E,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,MAAM,CAAC,IAAI,IAAI,CAAC,CAAC;QACzC,OAAO;IACT,CAAC;IACD,MAAM,SAAS,GAAG,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,UAAU,CAAC,CAAC;IACpD,MAAM,SAAS,GAAG,SAAS,CAAC,CAAC,CAAC,MAAM,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;IAC5D,MAAM,MAAM,GAAG,MAAM,YAAY,CAAC;QAChC,MAAM;QACN,GAAG,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;QAC7B,GAAG,CAAC,SAAS,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,SAAS,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;KAClD,CAAC,CAAC;IACH,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,MAAM,CAAC,IAAI,IAAI,CAAC,CAAC;AAC3C,CAAC;AAED,MAAM,CAAC,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,KAAK,EAAE,EAAE;IAC5C,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC,iBAAiB,YAAY,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;IAC/D,OAAO,CAAC,QAAQ,GAAG,CAAC,CAAC;AACvB,CAAC,CAAC,CAAC"}
|
package/dist/config.d.ts
CHANGED
|
@@ -1,17 +1,24 @@
|
|
|
1
|
-
export interface
|
|
2
|
-
|
|
3
|
-
baseUrl: string;
|
|
1
|
+
export interface ModelTarget {
|
|
2
|
+
label: string;
|
|
4
3
|
model: string;
|
|
4
|
+
baseUrl: string;
|
|
5
|
+
apiKey?: string;
|
|
6
|
+
headers: Record<string, string>;
|
|
7
|
+
}
|
|
8
|
+
export interface VisionConfig {
|
|
9
|
+
targets: ModelTarget[];
|
|
10
|
+
configFile?: string;
|
|
5
11
|
maxTokens: number;
|
|
6
12
|
timeoutMs: number;
|
|
7
13
|
headers: Record<string, string>;
|
|
8
14
|
maxImageBytes: number;
|
|
9
15
|
maxVideoBytes: number;
|
|
10
16
|
allowPrivateUrls: boolean;
|
|
17
|
+
ffmpegPath?: string;
|
|
11
18
|
}
|
|
12
19
|
type Environment = Record<string, string | undefined>;
|
|
13
|
-
export
|
|
14
|
-
export declare function
|
|
20
|
+
export type SettingsReader = (filePath: string) => string | undefined;
|
|
21
|
+
export declare function loadConfig(env?: Environment, readSettings?: SettingsReader): VisionConfig;
|
|
15
22
|
export declare function chatCompletionsUrl(baseUrl: string): string;
|
|
16
23
|
export {};
|
|
17
24
|
//# sourceMappingURL=config.d.ts.map
|
package/dist/config.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"config.d.ts","sourceRoot":"","sources":["../src/config.ts"],"names":[],"mappings":"
|
|
1
|
+
{"version":3,"file":"config.d.ts","sourceRoot":"","sources":["../src/config.ts"],"names":[],"mappings":"AAKA,MAAM,WAAW,WAAW;IAC1B,KAAK,EAAE,MAAM,CAAC;IACd,KAAK,EAAE,MAAM,CAAC;IACd,OAAO,EAAE,MAAM,CAAC;IAChB,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,OAAO,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;CACjC;AAED,MAAM,WAAW,YAAY;IAC3B,OAAO,EAAE,WAAW,EAAE,CAAC;IACvB,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,SAAS,EAAE,MAAM,CAAC;IAClB,SAAS,EAAE,MAAM,CAAC;IAClB,OAAO,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IAChC,aAAa,EAAE,MAAM,CAAC;IACtB,aAAa,EAAE,MAAM,CAAC;IACtB,gBAAgB,EAAE,OAAO,CAAC;IAC1B,UAAU,CAAC,EAAE,MAAM,CAAC;CACrB;AAqBD,KAAK,WAAW,GAAG,MAAM,CAAC,MAAM,EAAE,MAAM,GAAG,SAAS,CAAC,CAAC;AAEtD,MAAM,MAAM,cAAc,GAAG,CAAC,QAAQ,EAAE,MAAM,KAAK,MAAM,GAAG,SAAS,CAAC;AAmNtE,wBAAgB,UAAU,CACxB,GAAG,GAAE,WAAyB,EAC9B,YAAY,GAAE,cAAsC,GACnD,YAAY,CAsFd;AAED,wBAAgB,kBAAkB,CAAC,OAAO,EAAE,MAAM,GAAG,MAAM,CAe1D"}
|
package/dist/config.js
CHANGED
|
@@ -1,3 +1,6 @@
|
|
|
1
|
+
import { readFileSync } from "node:fs";
|
|
2
|
+
import { homedir } from "node:os";
|
|
3
|
+
import path from "node:path";
|
|
1
4
|
import { VisionError } from "./errors.js";
|
|
2
5
|
function firstNonEmpty(env, names) {
|
|
3
6
|
for (const name of names) {
|
|
@@ -17,29 +20,171 @@ function positiveInteger(env, name, fallback) {
|
|
|
17
20
|
}
|
|
18
21
|
return value;
|
|
19
22
|
}
|
|
23
|
+
function stringRecord(value, where) {
|
|
24
|
+
if (!value || typeof value !== "object" || Array.isArray(value)) {
|
|
25
|
+
throw new VisionError("CONFIG_ERROR", `${where} must be an object`);
|
|
26
|
+
}
|
|
27
|
+
const headers = {};
|
|
28
|
+
for (const [key, entry] of Object.entries(value)) {
|
|
29
|
+
if (typeof entry !== "string") {
|
|
30
|
+
throw new VisionError("CONFIG_ERROR", `${where}.${key} must be a string`);
|
|
31
|
+
}
|
|
32
|
+
headers[key] = entry;
|
|
33
|
+
}
|
|
34
|
+
return headers;
|
|
35
|
+
}
|
|
20
36
|
function parseHeaders(raw) {
|
|
21
37
|
if (!raw?.trim())
|
|
22
38
|
return {};
|
|
23
39
|
try {
|
|
24
|
-
|
|
25
|
-
if (!value || Array.isArray(value) || typeof value !== "object") {
|
|
26
|
-
throw new Error("expected a JSON object");
|
|
27
|
-
}
|
|
28
|
-
const headers = {};
|
|
29
|
-
for (const [key, headerValue] of Object.entries(value)) {
|
|
30
|
-
if (typeof headerValue !== "string") {
|
|
31
|
-
throw new Error(`header ${key} must be a string`);
|
|
32
|
-
}
|
|
33
|
-
headers[key] = headerValue;
|
|
34
|
-
}
|
|
35
|
-
return headers;
|
|
40
|
+
return stringRecord(JSON.parse(raw), "AGENT_VISION_HEADERS");
|
|
36
41
|
}
|
|
37
42
|
catch (error) {
|
|
43
|
+
if (error instanceof VisionError)
|
|
44
|
+
throw error;
|
|
38
45
|
const detail = error instanceof Error ? error.message : String(error);
|
|
39
46
|
throw new VisionError("CONFIG_ERROR", `AGENT_VISION_HEADERS must be a JSON object of string values: ${detail}`);
|
|
40
47
|
}
|
|
41
48
|
}
|
|
42
|
-
|
|
49
|
+
function expandConfigPath(raw) {
|
|
50
|
+
if (raw === "~")
|
|
51
|
+
return homedir();
|
|
52
|
+
if (raw.startsWith("~/"))
|
|
53
|
+
return path.join(homedir(), raw.slice(2));
|
|
54
|
+
return path.resolve(raw);
|
|
55
|
+
}
|
|
56
|
+
function resolveSettingsPath(env) {
|
|
57
|
+
const explicit = env.AGENT_VISION_CONFIG?.trim();
|
|
58
|
+
if (explicit)
|
|
59
|
+
return { path: expandConfigPath(explicit), explicit: true };
|
|
60
|
+
const configRoot = env.XDG_CONFIG_HOME?.trim() || path.join(homedir(), ".config");
|
|
61
|
+
return { path: path.join(configRoot, "agent-vision", "settings.json"), explicit: false };
|
|
62
|
+
}
|
|
63
|
+
function defaultSettingsReader(filePath) {
|
|
64
|
+
try {
|
|
65
|
+
return readFileSync(filePath, "utf8");
|
|
66
|
+
}
|
|
67
|
+
catch (error) {
|
|
68
|
+
if (error.code === "ENOENT")
|
|
69
|
+
return undefined;
|
|
70
|
+
const detail = error instanceof Error ? error.message : String(error);
|
|
71
|
+
throw new VisionError("CONFIG_ERROR", `Could not read settings file ${filePath}: ${detail}`);
|
|
72
|
+
}
|
|
73
|
+
}
|
|
74
|
+
function positiveFileNumber(file, key, filePath) {
|
|
75
|
+
const raw = file[key];
|
|
76
|
+
if (raw === undefined)
|
|
77
|
+
return undefined;
|
|
78
|
+
if (typeof raw !== "number" || !Number.isSafeInteger(raw) || raw <= 0) {
|
|
79
|
+
throw new VisionError("CONFIG_ERROR", `${filePath}: ${key} must be a positive integer`);
|
|
80
|
+
}
|
|
81
|
+
return raw;
|
|
82
|
+
}
|
|
83
|
+
function parseProvider(entry, filePath) {
|
|
84
|
+
if (!entry || typeof entry !== "object" || Array.isArray(entry)) {
|
|
85
|
+
throw new VisionError("CONFIG_ERROR", `${filePath}: each provider must be an object`);
|
|
86
|
+
}
|
|
87
|
+
const record = entry;
|
|
88
|
+
const baseUrl = record.baseUrl;
|
|
89
|
+
if (typeof baseUrl !== "string" || !baseUrl.trim()) {
|
|
90
|
+
throw new VisionError("CONFIG_ERROR", `${filePath}: every provider needs a baseUrl string`);
|
|
91
|
+
}
|
|
92
|
+
let parsedUrl;
|
|
93
|
+
try {
|
|
94
|
+
parsedUrl = new URL(baseUrl);
|
|
95
|
+
}
|
|
96
|
+
catch {
|
|
97
|
+
throw new VisionError("CONFIG_ERROR", `${filePath}: invalid provider baseUrl: ${baseUrl}`);
|
|
98
|
+
}
|
|
99
|
+
if (parsedUrl.protocol !== "http:" && parsedUrl.protocol !== "https:") {
|
|
100
|
+
throw new VisionError("CONFIG_ERROR", `${filePath}: provider baseUrl must use http or https: ${baseUrl}`);
|
|
101
|
+
}
|
|
102
|
+
const models = record.models;
|
|
103
|
+
if (!Array.isArray(models) ||
|
|
104
|
+
models.length === 0 ||
|
|
105
|
+
models.some((model) => typeof model !== "string" || !model.trim())) {
|
|
106
|
+
throw new VisionError("CONFIG_ERROR", `${filePath}: every provider needs a non-empty models array of strings`);
|
|
107
|
+
}
|
|
108
|
+
const name = record.name;
|
|
109
|
+
if (name !== undefined && (typeof name !== "string" || !name.trim())) {
|
|
110
|
+
throw new VisionError("CONFIG_ERROR", `${filePath}: provider name must be a non-empty string`);
|
|
111
|
+
}
|
|
112
|
+
const apiKey = record.apiKey;
|
|
113
|
+
if (apiKey !== undefined && typeof apiKey !== "string") {
|
|
114
|
+
throw new VisionError("CONFIG_ERROR", `${filePath}: provider apiKey must be a string`);
|
|
115
|
+
}
|
|
116
|
+
const headers = record.headers === undefined
|
|
117
|
+
? undefined
|
|
118
|
+
: stringRecord(record.headers, `${filePath}: provider headers`);
|
|
119
|
+
return {
|
|
120
|
+
...(name !== undefined ? { name } : {}),
|
|
121
|
+
baseUrl,
|
|
122
|
+
...(apiKey !== undefined ? { apiKey } : {}),
|
|
123
|
+
models: models,
|
|
124
|
+
...(headers !== undefined ? { headers } : {}),
|
|
125
|
+
};
|
|
126
|
+
}
|
|
127
|
+
function parseSettings(raw, filePath) {
|
|
128
|
+
let parsed;
|
|
129
|
+
try {
|
|
130
|
+
parsed = JSON.parse(raw);
|
|
131
|
+
}
|
|
132
|
+
catch (error) {
|
|
133
|
+
const detail = error instanceof Error ? error.message : String(error);
|
|
134
|
+
throw new VisionError("CONFIG_ERROR", `${filePath} is not valid JSON: ${detail}`);
|
|
135
|
+
}
|
|
136
|
+
if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) {
|
|
137
|
+
throw new VisionError("CONFIG_ERROR", `${filePath} must contain a JSON object`);
|
|
138
|
+
}
|
|
139
|
+
const file = parsed;
|
|
140
|
+
let providers;
|
|
141
|
+
if (file.providers !== undefined) {
|
|
142
|
+
if (!Array.isArray(file.providers) || file.providers.length === 0) {
|
|
143
|
+
throw new VisionError("CONFIG_ERROR", `${filePath}: providers must be a non-empty array`);
|
|
144
|
+
}
|
|
145
|
+
providers = file.providers.map((entry) => parseProvider(entry, filePath));
|
|
146
|
+
}
|
|
147
|
+
const allowPrivateUrls = file.allowPrivateUrls;
|
|
148
|
+
if (allowPrivateUrls !== undefined && typeof allowPrivateUrls !== "boolean") {
|
|
149
|
+
throw new VisionError("CONFIG_ERROR", `${filePath}: allowPrivateUrls must be a boolean`);
|
|
150
|
+
}
|
|
151
|
+
const ffmpegPath = file.ffmpegPath;
|
|
152
|
+
if (ffmpegPath !== undefined && (typeof ffmpegPath !== "string" || !ffmpegPath.trim())) {
|
|
153
|
+
throw new VisionError("CONFIG_ERROR", `${filePath}: ffmpegPath must be a non-empty string`);
|
|
154
|
+
}
|
|
155
|
+
const maxTokens = positiveFileNumber(file, "maxTokens", filePath);
|
|
156
|
+
const timeoutMs = positiveFileNumber(file, "timeoutMs", filePath);
|
|
157
|
+
const headers = file.headers === undefined
|
|
158
|
+
? undefined
|
|
159
|
+
: stringRecord(file.headers, `${filePath}: headers`);
|
|
160
|
+
const maxImageMb = positiveFileNumber(file, "maxImageMb", filePath);
|
|
161
|
+
const maxVideoMb = positiveFileNumber(file, "maxVideoMb", filePath);
|
|
162
|
+
return {
|
|
163
|
+
...(providers !== undefined ? { providers } : {}),
|
|
164
|
+
...(maxTokens !== undefined ? { maxTokens } : {}),
|
|
165
|
+
...(timeoutMs !== undefined ? { timeoutMs } : {}),
|
|
166
|
+
...(headers !== undefined ? { headers } : {}),
|
|
167
|
+
...(maxImageMb !== undefined ? { maxImageMb } : {}),
|
|
168
|
+
...(maxVideoMb !== undefined ? { maxVideoMb } : {}),
|
|
169
|
+
...(allowPrivateUrls !== undefined ? { allowPrivateUrls } : {}),
|
|
170
|
+
...(ffmpegPath !== undefined ? { ffmpegPath } : {}),
|
|
171
|
+
};
|
|
172
|
+
}
|
|
173
|
+
function hostOf(baseUrl) {
|
|
174
|
+
return new URL(baseUrl).host;
|
|
175
|
+
}
|
|
176
|
+
export function loadConfig(env = process.env, readSettings = defaultSettingsReader) {
|
|
177
|
+
const settingsLocation = resolveSettingsPath(env);
|
|
178
|
+
const rawSettings = readSettings(settingsLocation.path);
|
|
179
|
+
let settings;
|
|
180
|
+
let configFile;
|
|
181
|
+
if (rawSettings !== undefined) {
|
|
182
|
+
settings = parseSettings(rawSettings, settingsLocation.path);
|
|
183
|
+
configFile = settingsLocation.path;
|
|
184
|
+
}
|
|
185
|
+
else if (settingsLocation.explicit) {
|
|
186
|
+
throw new VisionError("CONFIG_ERROR", `AGENT_VISION_CONFIG file not found: ${settingsLocation.path}`);
|
|
187
|
+
}
|
|
43
188
|
const apiKey = firstNonEmpty(env, [
|
|
44
189
|
"AGENT_VISION_API_KEY",
|
|
45
190
|
"VISION_API_KEY",
|
|
@@ -55,23 +200,51 @@ export function loadConfig(env = process.env) {
|
|
|
55
200
|
"VISION_MODEL",
|
|
56
201
|
"OPENAI_MODEL",
|
|
57
202
|
]) ?? "";
|
|
203
|
+
const maxTokens = settings?.maxTokens ?? positiveInteger(env, "AGENT_VISION_MAX_TOKENS", 4096);
|
|
204
|
+
const timeoutMs = settings?.timeoutMs ?? positiveInteger(env, "AGENT_VISION_TIMEOUT_MS", 120_000);
|
|
205
|
+
const headers = { ...parseHeaders(env.AGENT_VISION_HEADERS), ...settings?.headers };
|
|
206
|
+
const maxImageMb = settings?.maxImageMb ?? positiveInteger(env, "AGENT_VISION_MAX_IMAGE_MB", 20);
|
|
207
|
+
const maxVideoMb = settings?.maxVideoMb ?? positiveInteger(env, "AGENT_VISION_MAX_VIDEO_MB", 200);
|
|
208
|
+
const allowPrivateUrls = settings?.allowPrivateUrls ??
|
|
209
|
+
env.AGENT_VISION_ALLOW_PRIVATE_URLS?.trim().toLowerCase() === "true";
|
|
210
|
+
const ffmpegPath = settings?.ffmpegPath?.trim() || env.AGENT_VISION_FFMPEG_PATH?.trim() || undefined;
|
|
211
|
+
let targets;
|
|
212
|
+
if (settings?.providers?.length) {
|
|
213
|
+
const envApiKey = apiKey;
|
|
214
|
+
targets = settings.providers.flatMap((provider) => provider.models.map((model) => {
|
|
215
|
+
const targetApiKey = provider.apiKey ?? envApiKey;
|
|
216
|
+
return {
|
|
217
|
+
label: `${provider.name ?? hostOf(provider.baseUrl)}/${model}`,
|
|
218
|
+
model,
|
|
219
|
+
baseUrl: provider.baseUrl,
|
|
220
|
+
...(targetApiKey ? { apiKey: targetApiKey } : {}),
|
|
221
|
+
headers: { ...headers, ...provider.headers },
|
|
222
|
+
};
|
|
223
|
+
}));
|
|
224
|
+
}
|
|
225
|
+
else {
|
|
226
|
+
targets = model
|
|
227
|
+
? [{
|
|
228
|
+
label: model,
|
|
229
|
+
model,
|
|
230
|
+
baseUrl,
|
|
231
|
+
...(apiKey ? { apiKey } : {}),
|
|
232
|
+
headers,
|
|
233
|
+
}]
|
|
234
|
+
: [];
|
|
235
|
+
}
|
|
58
236
|
return {
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
237
|
+
targets,
|
|
238
|
+
...(configFile ? { configFile } : {}),
|
|
239
|
+
maxTokens,
|
|
240
|
+
timeoutMs,
|
|
241
|
+
headers,
|
|
242
|
+
maxImageBytes: maxImageMb * 1024 * 1024,
|
|
243
|
+
maxVideoBytes: maxVideoMb * 1024 * 1024,
|
|
244
|
+
allowPrivateUrls,
|
|
245
|
+
...(ffmpegPath ? { ffmpegPath } : {}),
|
|
68
246
|
};
|
|
69
247
|
}
|
|
70
|
-
export function requireModel(config) {
|
|
71
|
-
if (!config.model) {
|
|
72
|
-
throw new VisionError("CONFIG_ERROR", "Set AGENT_VISION_MODEL (or VISION_MODEL / OPENAI_MODEL)");
|
|
73
|
-
}
|
|
74
|
-
}
|
|
75
248
|
export function chatCompletionsUrl(baseUrl) {
|
|
76
249
|
let url;
|
|
77
250
|
try {
|
|
@@ -83,9 +256,9 @@ export function chatCompletionsUrl(baseUrl) {
|
|
|
83
256
|
if (url.protocol !== "http:" && url.protocol !== "https:") {
|
|
84
257
|
throw new VisionError("CONFIG_ERROR", "AGENT_VISION_BASE_URL must use http or https");
|
|
85
258
|
}
|
|
86
|
-
const
|
|
87
|
-
if (!
|
|
88
|
-
url.pathname = `${
|
|
259
|
+
const pathName = url.pathname.replace(/\/+$/, "");
|
|
260
|
+
if (!pathName.endsWith("/chat/completions")) {
|
|
261
|
+
url.pathname = `${pathName}/chat/completions`;
|
|
89
262
|
}
|
|
90
263
|
return url.toString();
|
|
91
264
|
}
|
package/dist/config.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"config.js","sourceRoot":"","sources":["../src/config.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,WAAW,EAAE,MAAM,aAAa,CAAC;AAgB1C,SAAS,aAAa,CAAC,GAAgB,EAAE,KAAe;IACtD,KAAK,MAAM,IAAI,IAAI,KAAK,EAAE,CAAC;QACzB,MAAM,KAAK,GAAG,GAAG,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,CAAC;QAChC,IAAI,KAAK;YAAE,OAAO,KAAK,CAAC;IAC1B,CAAC;IACD,OAAO,SAAS,CAAC;AACnB,CAAC;AAED,SAAS,eAAe,CACtB,GAAgB,EAChB,IAAY,EACZ,QAAgB;IAEhB,MAAM,GAAG,GAAG,GAAG,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,CAAC;IAC9B,IAAI,CAAC,GAAG;QAAE,OAAO,QAAQ,CAAC;IAC1B,MAAM,KAAK,GAAG,MAAM,CAAC,GAAG,CAAC,CAAC;IAC1B,IAAI,CAAC,MAAM,CAAC,aAAa,CAAC,KAAK,CAAC,IAAI,KAAK,IAAI,CAAC,EAAE,CAAC;QAC/C,MAAM,IAAI,WAAW,CAAC,cAAc,EAAE,GAAG,IAAI,6BAA6B,CAAC,CAAC;IAC9E,CAAC;IACD,OAAO,KAAK,CAAC;AACf,CAAC;AAED,SAAS,YAAY,CAAC,GAAuB;IAC3C,IAAI,CAAC,GAAG,EAAE,IAAI,EAAE;QAAE,OAAO,EAAE,CAAC;IAC5B,IAAI,CAAC;QACH,MAAM,KAAK,GAAY,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC;QACvC,IAAI,CAAC,KAAK,IAAI,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,IAAI,OAAO,KAAK,KAAK,QAAQ,EAAE,CAAC;YAChE,MAAM,IAAI,KAAK,CAAC,wBAAwB,CAAC,CAAC;QAC5C,CAAC;QACD,MAAM,OAAO,GAA2B,EAAE,CAAC;QAC3C,KAAK,MAAM,CAAC,GAAG,EAAE,WAAW,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,KAAK,CAAC,EAAE,CAAC;YACvD,IAAI,OAAO,WAAW,KAAK,QAAQ,EAAE,CAAC;gBACpC,MAAM,IAAI,KAAK,CAAC,UAAU,GAAG,mBAAmB,CAAC,CAAC;YACpD,CAAC;YACD,OAAO,CAAC,GAAG,CAAC,GAAG,WAAW,CAAC;QAC7B,CAAC;QACD,OAAO,OAAO,CAAC;IACjB,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,MAAM,MAAM,GAAG,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;QACtE,MAAM,IAAI,WAAW,CACnB,cAAc,EACd,gEAAgE,MAAM,EAAE,CACzE,CAAC;IACJ,CAAC;AACH,CAAC;AAED,MAAM,UAAU,UAAU,CAAC,MAAmB,OAAO,CAAC,GAAG;IACvD,MAAM,MAAM,GAAG,aAAa,CAAC,GAAG,EAAE;QAChC,sBAAsB;QACtB,gBAAgB;QAChB,gBAAgB;KACjB,CAAC,CAAC;IACH,MAAM,OAAO,GACX,aAAa,CAAC,GAAG,EAAE;QACjB,uBAAuB;QACvB,iBAAiB;QACjB,iBAAiB;KAClB,CAAC,IAAI,2BAA2B,CAAC;IACpC,MAAM,KAAK,GACT,aAAa,CAAC,GAAG,EAAE;QACjB,oBAAoB;QACpB,cAAc;QACd,cAAc;KACf,CAAC,IAAI,EAAE,CAAC;IAEX,OAAO;QACL,GAAG,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;QAC7B,OAAO;QACP,KAAK;QACL,SAAS,EAAE,eAAe,CAAC,GAAG,EAAE,yBAAyB,EAAE,IAAI,CAAC;QAChE,SAAS,EAAE,eAAe,CAAC,GAAG,EAAE,yBAAyB,EAAE,OAAO,CAAC;QACnE,OAAO,EAAE,YAAY,CAAC,GAAG,CAAC,oBAAoB,CAAC;QAC/C,aAAa,EACX,eAAe,CAAC,GAAG,EAAE,2BAA2B,EAAE,EAAE,CAAC,GAAG,IAAI,GAAG,IAAI;QACrE,aAAa,EACX,eAAe,CAAC,GAAG,EAAE,2BAA2B,EAAE,GAAG,CAAC,GAAG,IAAI,GAAG,IAAI;QACtE,gBAAgB,EACd,GAAG,CAAC,+BAA+B,EAAE,IAAI,EAAE,CAAC,WAAW,EAAE,KAAK,MAAM;KACvE,CAAC;AACJ,CAAC;AAED,MAAM,UAAU,YAAY,CAAC,MAAoB;IAC/C,IAAI,CAAC,MAAM,CAAC,KAAK,EAAE,CAAC;QAClB,MAAM,IAAI,WAAW,CACnB,cAAc,EACd,yDAAyD,CAC1D,CAAC;IACJ,CAAC;AACH,CAAC;AAED,MAAM,UAAU,kBAAkB,CAAC,OAAe;IAChD,IAAI,GAAQ,CAAC;IACb,IAAI,CAAC;QACH,GAAG,GAAG,IAAI,GAAG,CAAC,OAAO,CAAC,CAAC;IACzB,CAAC;IAAC,MAAM,CAAC;QACP,MAAM,IAAI,WAAW,CACnB,cAAc,EACd,kCAAkC,OAAO,EAAE,CAC5C,CAAC;IACJ,CAAC;IACD,IAAI,GAAG,CAAC,QAAQ,KAAK,OAAO,IAAI,GAAG,CAAC,QAAQ,KAAK,QAAQ,EAAE,CAAC;QAC1D,MAAM,IAAI,WAAW,CACnB,cAAc,EACd,8CAA8C,CAC/C,CAAC;IACJ,CAAC;IACD,MAAM,IAAI,GAAG,GAAG,CAAC,QAAQ,CAAC,OAAO,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC;IAC9C,IAAI,CAAC,IAAI,CAAC,QAAQ,CAAC,mBAAmB,CAAC,EAAE,CAAC;QACxC,GAAG,CAAC,QAAQ,GAAG,GAAG,IAAI,mBAAmB,CAAC;IAC5C,CAAC;IACD,OAAO,GAAG,CAAC,QAAQ,EAAE,CAAC;AACxB,CAAC"}
|
|
1
|
+
{"version":3,"file":"config.js","sourceRoot":"","sources":["../src/config.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,YAAY,EAAE,MAAM,SAAS,CAAC;AACvC,OAAO,EAAE,OAAO,EAAE,MAAM,SAAS,CAAC;AAClC,OAAO,IAAI,MAAM,WAAW,CAAC;AAC7B,OAAO,EAAE,WAAW,EAAE,MAAM,aAAa,CAAC;AA6C1C,SAAS,aAAa,CAAC,GAAgB,EAAE,KAAe;IACtD,KAAK,MAAM,IAAI,IAAI,KAAK,EAAE,CAAC;QACzB,MAAM,KAAK,GAAG,GAAG,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,CAAC;QAChC,IAAI,KAAK;YAAE,OAAO,KAAK,CAAC;IAC1B,CAAC;IACD,OAAO,SAAS,CAAC;AACnB,CAAC;AAED,SAAS,eAAe,CACtB,GAAgB,EAChB,IAAY,EACZ,QAAgB;IAEhB,MAAM,GAAG,GAAG,GAAG,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,CAAC;IAC9B,IAAI,CAAC,GAAG;QAAE,OAAO,QAAQ,CAAC;IAC1B,MAAM,KAAK,GAAG,MAAM,CAAC,GAAG,CAAC,CAAC;IAC1B,IAAI,CAAC,MAAM,CAAC,aAAa,CAAC,KAAK,CAAC,IAAI,KAAK,IAAI,CAAC,EAAE,CAAC;QAC/C,MAAM,IAAI,WAAW,CAAC,cAAc,EAAE,GAAG,IAAI,6BAA6B,CAAC,CAAC;IAC9E,CAAC;IACD,OAAO,KAAK,CAAC;AACf,CAAC;AAED,SAAS,YAAY,CAAC,KAAc,EAAE,KAAa;IACjD,IAAI,CAAC,KAAK,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,EAAE,CAAC;QAChE,MAAM,IAAI,WAAW,CAAC,cAAc,EAAE,GAAG,KAAK,oBAAoB,CAAC,CAAC;IACtE,CAAC;IACD,MAAM,OAAO,GAA2B,EAAE,CAAC;IAC3C,KAAK,MAAM,CAAC,GAAG,EAAE,KAAK,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,KAAK,CAAC,EAAE,CAAC;QACjD,IAAI,OAAO,KAAK,KAAK,QAAQ,EAAE,CAAC;YAC9B,MAAM,IAAI,WAAW,CAAC,cAAc,EAAE,GAAG,KAAK,IAAI,GAAG,mBAAmB,CAAC,CAAC;QAC5E,CAAC;QACD,OAAO,CAAC,GAAG,CAAC,GAAG,KAAK,CAAC;IACvB,CAAC;IACD,OAAO,OAAO,CAAC;AACjB,CAAC;AAED,SAAS,YAAY,CAAC,GAAuB;IAC3C,IAAI,CAAC,GAAG,EAAE,IAAI,EAAE;QAAE,OAAO,EAAE,CAAC;IAC5B,IAAI,CAAC;QACH,OAAO,YAAY,CAAC,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,EAAE,sBAAsB,CAAC,CAAC;IAC/D,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,IAAI,KAAK,YAAY,WAAW;YAAE,MAAM,KAAK,CAAC;QAC9C,MAAM,MAAM,GAAG,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;QACtE,MAAM,IAAI,WAAW,CACnB,cAAc,EACd,gEAAgE,MAAM,EAAE,CACzE,CAAC;IACJ,CAAC;AACH,CAAC;AAED,SAAS,gBAAgB,CAAC,GAAW;IACnC,IAAI,GAAG,KAAK,GAAG;QAAE,OAAO,OAAO,EAAE,CAAC;IAClC,IAAI,GAAG,CAAC,UAAU,CAAC,IAAI,CAAC;QAAE,OAAO,IAAI,CAAC,IAAI,CAAC,OAAO,EAAE,EAAE,GAAG,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC;IACpE,OAAO,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC;AAC3B,CAAC;AAED,SAAS,mBAAmB,CAAC,GAAgB;IAC3C,MAAM,QAAQ,GAAG,GAAG,CAAC,mBAAmB,EAAE,IAAI,EAAE,CAAC;IACjD,IAAI,QAAQ;QAAE,OAAO,EAAE,IAAI,EAAE,gBAAgB,CAAC,QAAQ,CAAC,EAAE,QAAQ,EAAE,IAAI,EAAE,CAAC;IAC1E,MAAM,UAAU,GAAG,GAAG,CAAC,eAAe,EAAE,IAAI,EAAE,IAAI,IAAI,CAAC,IAAI,CAAC,OAAO,EAAE,EAAE,SAAS,CAAC,CAAC;IAClF,OAAO,EAAE,IAAI,EAAE,IAAI,CAAC,IAAI,CAAC,UAAU,EAAE,cAAc,EAAE,eAAe,CAAC,EAAE,QAAQ,EAAE,KAAK,EAAE,CAAC;AAC3F,CAAC;AAED,SAAS,qBAAqB,CAAC,QAAgB;IAC7C,IAAI,CAAC;QACH,OAAO,YAAY,CAAC,QAAQ,EAAE,MAAM,CAAC,CAAC;IACxC,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,IAAK,KAA+B,CAAC,IAAI,KAAK,QAAQ;YAAE,OAAO,SAAS,CAAC;QACzE,MAAM,MAAM,GAAG,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;QACtE,MAAM,IAAI,WAAW,CACnB,cAAc,EACd,gCAAgC,QAAQ,KAAK,MAAM,EAAE,CACtD,CAAC;IACJ,CAAC;AACH,CAAC;AAED,SAAS,kBAAkB,CACzB,IAA6B,EAC7B,GAAW,EACX,QAAgB;IAEhB,MAAM,GAAG,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC;IACtB,IAAI,GAAG,KAAK,SAAS;QAAE,OAAO,SAAS,CAAC;IACxC,IAAI,OAAO,GAAG,KAAK,QAAQ,IAAI,CAAC,MAAM,CAAC,aAAa,CAAC,GAAG,CAAC,IAAI,GAAG,IAAI,CAAC,EAAE,CAAC;QACtE,MAAM,IAAI,WAAW,CAAC,cAAc,EAAE,GAAG,QAAQ,KAAK,GAAG,6BAA6B,CAAC,CAAC;IAC1F,CAAC;IACD,OAAO,GAAG,CAAC;AACb,CAAC;AAED,SAAS,aAAa,CAAC,KAAc,EAAE,QAAgB;IACrD,IAAI,CAAC,KAAK,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,EAAE,CAAC;QAChE,MAAM,IAAI,WAAW,CAAC,cAAc,EAAE,GAAG,QAAQ,mCAAmC,CAAC,CAAC;IACxF,CAAC;IACD,MAAM,MAAM,GAAG,KAAgC,CAAC;IAChD,MAAM,OAAO,GAAG,MAAM,CAAC,OAAO,CAAC;IAC/B,IAAI,OAAO,OAAO,KAAK,QAAQ,IAAI,CAAC,OAAO,CAAC,IAAI,EAAE,EAAE,CAAC;QACnD,MAAM,IAAI,WAAW,CACnB,cAAc,EACd,GAAG,QAAQ,yCAAyC,CACrD,CAAC;IACJ,CAAC;IACD,IAAI,SAAc,CAAC;IACnB,IAAI,CAAC;QACH,SAAS,GAAG,IAAI,GAAG,CAAC,OAAO,CAAC,CAAC;IAC/B,CAAC;IAAC,MAAM,CAAC;QACP,MAAM,IAAI,WAAW,CACnB,cAAc,EACd,GAAG,QAAQ,+BAA+B,OAAO,EAAE,CACpD,CAAC;IACJ,CAAC;IACD,IAAI,SAAS,CAAC,QAAQ,KAAK,OAAO,IAAI,SAAS,CAAC,QAAQ,KAAK,QAAQ,EAAE,CAAC;QACtE,MAAM,IAAI,WAAW,CACnB,cAAc,EACd,GAAG,QAAQ,8CAA8C,OAAO,EAAE,CACnE,CAAC;IACJ,CAAC;IACD,MAAM,MAAM,GAAG,MAAM,CAAC,MAAM,CAAC;IAC7B,IACE,CAAC,KAAK,CAAC,OAAO,CAAC,MAAM,CAAC;QACtB,MAAM,CAAC,MAAM,KAAK,CAAC;QACnB,MAAM,CAAC,IAAI,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,OAAO,KAAK,KAAK,QAAQ,IAAI,CAAC,KAAK,CAAC,IAAI,EAAE,CAAC,EAClE,CAAC;QACD,MAAM,IAAI,WAAW,CACnB,cAAc,EACd,GAAG,QAAQ,4DAA4D,CACxE,CAAC;IACJ,CAAC;IACD,MAAM,IAAI,GAAG,MAAM,CAAC,IAAI,CAAC;IACzB,IAAI,IAAI,KAAK,SAAS,IAAI,CAAC,OAAO,IAAI,KAAK,QAAQ,IAAI,CAAC,IAAI,CAAC,IAAI,EAAE,CAAC,EAAE,CAAC;QACrE,MAAM,IAAI,WAAW,CACnB,cAAc,EACd,GAAG,QAAQ,4CAA4C,CACxD,CAAC;IACJ,CAAC;IACD,MAAM,MAAM,GAAG,MAAM,CAAC,MAAM,CAAC;IAC7B,IAAI,MAAM,KAAK,SAAS,IAAI,OAAO,MAAM,KAAK,QAAQ,EAAE,CAAC;QACvD,MAAM,IAAI,WAAW,CAAC,cAAc,EAAE,GAAG,QAAQ,oCAAoC,CAAC,CAAC;IACzF,CAAC;IACD,MAAM,OAAO,GAAG,MAAM,CAAC,OAAO,KAAK,SAAS;QAC1C,CAAC,CAAC,SAAS;QACX,CAAC,CAAC,YAAY,CAAC,MAAM,CAAC,OAAO,EAAE,GAAG,QAAQ,oBAAoB,CAAC,CAAC;IAClE,OAAO;QACL,GAAG,CAAC,IAAI,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;QACvC,OAAO;QACP,GAAG,CAAC,MAAM,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;QAC3C,MAAM,EAAE,MAAkB;QAC1B,GAAG,CAAC,OAAO,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;KAC9C,CAAC;AACJ,CAAC;AAED,SAAS,aAAa,CAAC,GAAW,EAAE,QAAgB;IAClD,IAAI,MAAe,CAAC;IACpB,IAAI,CAAC;QACH,MAAM,GAAG,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC;IAC3B,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,MAAM,MAAM,GAAG,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;QACtE,MAAM,IAAI,WAAW,CAAC,cAAc,EAAE,GAAG,QAAQ,uBAAuB,MAAM,EAAE,CAAC,CAAC;IACpF,CAAC;IACD,IAAI,CAAC,MAAM,IAAI,OAAO,MAAM,KAAK,QAAQ,IAAI,KAAK,CAAC,OAAO,CAAC,MAAM,CAAC,EAAE,CAAC;QACnE,MAAM,IAAI,WAAW,CAAC,cAAc,EAAE,GAAG,QAAQ,6BAA6B,CAAC,CAAC;IAClF,CAAC;IACD,MAAM,IAAI,GAAG,MAAiC,CAAC;IAC/C,IAAI,SAAyC,CAAC;IAC9C,IAAI,IAAI,CAAC,SAAS,KAAK,SAAS,EAAE,CAAC;QACjC,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,IAAI,CAAC,SAAS,CAAC,IAAI,IAAI,CAAC,SAAS,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;YAClE,MAAM,IAAI,WAAW,CACnB,cAAc,EACd,GAAG,QAAQ,uCAAuC,CACnD,CAAC;QACJ,CAAC;QACD,SAAS,GAAG,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,aAAa,CAAC,KAAK,EAAE,QAAQ,CAAC,CAAC,CAAC;IAC5E,CAAC;IACD,MAAM,gBAAgB,GAAG,IAAI,CAAC,gBAAgB,CAAC;IAC/C,IAAI,gBAAgB,KAAK,SAAS,IAAI,OAAO,gBAAgB,KAAK,SAAS,EAAE,CAAC;QAC5E,MAAM,IAAI,WAAW,CACnB,cAAc,EACd,GAAG,QAAQ,sCAAsC,CAClD,CAAC;IACJ,CAAC;IACD,MAAM,UAAU,GAAG,IAAI,CAAC,UAAU,CAAC;IACnC,IAAI,UAAU,KAAK,SAAS,IAAI,CAAC,OAAO,UAAU,KAAK,QAAQ,IAAI,CAAC,UAAU,CAAC,IAAI,EAAE,CAAC,EAAE,CAAC;QACvF,MAAM,IAAI,WAAW,CACnB,cAAc,EACd,GAAG,QAAQ,yCAAyC,CACrD,CAAC;IACJ,CAAC;IACD,MAAM,SAAS,GAAG,kBAAkB,CAAC,IAAI,EAAE,WAAW,EAAE,QAAQ,CAAC,CAAC;IAClE,MAAM,SAAS,GAAG,kBAAkB,CAAC,IAAI,EAAE,WAAW,EAAE,QAAQ,CAAC,CAAC;IAClE,MAAM,OAAO,GAAG,IAAI,CAAC,OAAO,KAAK,SAAS;QACxC,CAAC,CAAC,SAAS;QACX,CAAC,CAAC,YAAY,CAAC,IAAI,CAAC,OAAO,EAAE,GAAG,QAAQ,WAAW,CAAC,CAAC;IACvD,MAAM,UAAU,GAAG,kBAAkB,CAAC,IAAI,EAAE,YAAY,EAAE,QAAQ,CAAC,CAAC;IACpE,MAAM,UAAU,GAAG,kBAAkB,CAAC,IAAI,EAAE,YAAY,EAAE,QAAQ,CAAC,CAAC;IACpE,OAAO;QACL,GAAG,CAAC,SAAS,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,SAAS,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;QACjD,GAAG,CAAC,SAAS,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,SAAS,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;QACjD,GAAG,CAAC,SAAS,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,SAAS,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;QACjD,GAAG,CAAC,OAAO,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;QAC7C,GAAG,CAAC,UAAU,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,UAAU,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;QACnD,GAAG,CAAC,UAAU,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,UAAU,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;QACnD,GAAG,CAAC,gBAAgB,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,gBAAgB,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;QAC/D,GAAG,CAAC,UAAU,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,UAAU,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;KACpD,CAAC;AACJ,CAAC;AAED,SAAS,MAAM,CAAC,OAAe;IAC7B,OAAO,IAAI,GAAG,CAAC,OAAO,CAAC,CAAC,IAAI,CAAC;AAC/B,CAAC;AAED,MAAM,UAAU,UAAU,CACxB,MAAmB,OAAO,CAAC,GAAG,EAC9B,eAA+B,qBAAqB;IAEpD,MAAM,gBAAgB,GAAG,mBAAmB,CAAC,GAAG,CAAC,CAAC;IAClD,MAAM,WAAW,GAAG,YAAY,CAAC,gBAAgB,CAAC,IAAI,CAAC,CAAC;IACxD,IAAI,QAAkC,CAAC;IACvC,IAAI,UAA8B,CAAC;IACnC,IAAI,WAAW,KAAK,SAAS,EAAE,CAAC;QAC9B,QAAQ,GAAG,aAAa,CAAC,WAAW,EAAE,gBAAgB,CAAC,IAAI,CAAC,CAAC;QAC7D,UAAU,GAAG,gBAAgB,CAAC,IAAI,CAAC;IACrC,CAAC;SAAM,IAAI,gBAAgB,CAAC,QAAQ,EAAE,CAAC;QACrC,MAAM,IAAI,WAAW,CACnB,cAAc,EACd,uCAAuC,gBAAgB,CAAC,IAAI,EAAE,CAC/D,CAAC;IACJ,CAAC;IAED,MAAM,MAAM,GAAG,aAAa,CAAC,GAAG,EAAE;QAChC,sBAAsB;QACtB,gBAAgB;QAChB,gBAAgB;KACjB,CAAC,CAAC;IACH,MAAM,OAAO,GACX,aAAa,CAAC,GAAG,EAAE;QACjB,uBAAuB;QACvB,iBAAiB;QACjB,iBAAiB;KAClB,CAAC,IAAI,2BAA2B,CAAC;IACpC,MAAM,KAAK,GACT,aAAa,CAAC,GAAG,EAAE;QACjB,oBAAoB;QACpB,cAAc;QACd,cAAc;KACf,CAAC,IAAI,EAAE,CAAC;IAEX,MAAM,SAAS,GACb,QAAQ,EAAE,SAAS,IAAI,eAAe,CAAC,GAAG,EAAE,yBAAyB,EAAE,IAAI,CAAC,CAAC;IAC/E,MAAM,SAAS,GACb,QAAQ,EAAE,SAAS,IAAI,eAAe,CAAC,GAAG,EAAE,yBAAyB,EAAE,OAAO,CAAC,CAAC;IAClF,MAAM,OAAO,GAAG,EAAE,GAAG,YAAY,CAAC,GAAG,CAAC,oBAAoB,CAAC,EAAE,GAAG,QAAQ,EAAE,OAAO,EAAE,CAAC;IACpF,MAAM,UAAU,GACd,QAAQ,EAAE,UAAU,IAAI,eAAe,CAAC,GAAG,EAAE,2BAA2B,EAAE,EAAE,CAAC,CAAC;IAChF,MAAM,UAAU,GACd,QAAQ,EAAE,UAAU,IAAI,eAAe,CAAC,GAAG,EAAE,2BAA2B,EAAE,GAAG,CAAC,CAAC;IACjF,MAAM,gBAAgB,GACpB,QAAQ,EAAE,gBAAgB;QAC1B,GAAG,CAAC,+BAA+B,EAAE,IAAI,EAAE,CAAC,WAAW,EAAE,KAAK,MAAM,CAAC;IACvE,MAAM,UAAU,GACd,QAAQ,EAAE,UAAU,EAAE,IAAI,EAAE,IAAI,GAAG,CAAC,wBAAwB,EAAE,IAAI,EAAE,IAAI,SAAS,CAAC;IAEpF,IAAI,OAAsB,CAAC;IAC3B,IAAI,QAAQ,EAAE,SAAS,EAAE,MAAM,EAAE,CAAC;QAChC,MAAM,SAAS,GAAG,MAAM,CAAC;QACzB,OAAO,GAAG,QAAQ,CAAC,SAAS,CAAC,OAAO,CAAC,CAAC,QAAQ,EAAE,EAAE,CAChD,QAAQ,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE;YAC5B,MAAM,YAAY,GAAG,QAAQ,CAAC,MAAM,IAAI,SAAS,CAAC;YAClD,OAAO;gBACL,KAAK,EAAE,GAAG,QAAQ,CAAC,IAAI,IAAI,MAAM,CAAC,QAAQ,CAAC,OAAO,CAAC,IAAI,KAAK,EAAE;gBAC9D,KAAK;gBACL,OAAO,EAAE,QAAQ,CAAC,OAAO;gBACzB,GAAG,CAAC,YAAY,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,YAAY,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;gBACjD,OAAO,EAAE,EAAE,GAAG,OAAO,EAAE,GAAG,QAAQ,CAAC,OAAO,EAAE;aAC7C,CAAC;QACJ,CAAC,CAAC,CACH,CAAC;IACJ,CAAC;SAAM,CAAC;QACN,OAAO,GAAG,KAAK;YACb,CAAC,CAAC,CAAC;oBACC,KAAK,EAAE,KAAK;oBACZ,KAAK;oBACL,OAAO;oBACP,GAAG,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;oBAC7B,OAAO;iBACR,CAAC;YACJ,CAAC,CAAC,EAAE,CAAC;IACT,CAAC;IAED,OAAO;QACL,OAAO;QACP,GAAG,CAAC,UAAU,CAAC,CAAC,CAAC,EAAE,UAAU,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;QACrC,SAAS;QACT,SAAS;QACT,OAAO;QACP,aAAa,EAAE,UAAU,GAAG,IAAI,GAAG,IAAI;QACvC,aAAa,EAAE,UAAU,GAAG,IAAI,GAAG,IAAI;QACvC,gBAAgB;QAChB,GAAG,CAAC,UAAU,CAAC,CAAC,CAAC,EAAE,UAAU,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;KACtC,CAAC;AACJ,CAAC;AAED,MAAM,UAAU,kBAAkB,CAAC,OAAe;IAChD,IAAI,GAAQ,CAAC;IACb,IAAI,CAAC;QACH,GAAG,GAAG,IAAI,GAAG,CAAC,OAAO,CAAC,CAAC;IACzB,CAAC;IAAC,MAAM,CAAC;QACP,MAAM,IAAI,WAAW,CAAC,cAAc,EAAE,kCAAkC,OAAO,EAAE,CAAC,CAAC;IACrF,CAAC;IACD,IAAI,GAAG,CAAC,QAAQ,KAAK,OAAO,IAAI,GAAG,CAAC,QAAQ,KAAK,QAAQ,EAAE,CAAC;QAC1D,MAAM,IAAI,WAAW,CAAC,cAAc,EAAE,8CAA8C,CAAC,CAAC;IACxF,CAAC;IACD,MAAM,QAAQ,GAAG,GAAG,CAAC,QAAQ,CAAC,OAAO,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC;IAClD,IAAI,CAAC,QAAQ,CAAC,QAAQ,CAAC,mBAAmB,CAAC,EAAE,CAAC;QAC5C,GAAG,CAAC,QAAQ,GAAG,GAAG,QAAQ,mBAAmB,CAAC;IAChD,CAAC;IACD,OAAO,GAAG,CAAC,QAAQ,EAAE,CAAC;AACxB,CAAC"}
|
package/dist/provider.d.ts
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import type { VisionConfig } from "./config.js";
|
|
1
|
+
import type { ModelTarget, VisionConfig } from "./config.js";
|
|
2
2
|
export interface VisionInputImage {
|
|
3
3
|
dataUrl: string;
|
|
4
4
|
label?: string;
|
|
@@ -9,6 +9,13 @@ export interface VisionCompletion {
|
|
|
9
9
|
usage?: Record<string, unknown>;
|
|
10
10
|
}
|
|
11
11
|
export declare function requestVisionCompletion(options: {
|
|
12
|
+
config: VisionConfig;
|
|
13
|
+
target: ModelTarget;
|
|
14
|
+
prompt: string;
|
|
15
|
+
images: VisionInputImage[];
|
|
16
|
+
signal?: AbortSignal;
|
|
17
|
+
}): Promise<VisionCompletion>;
|
|
18
|
+
export declare function raceVisionCompletions(options: {
|
|
12
19
|
config: VisionConfig;
|
|
13
20
|
prompt: string;
|
|
14
21
|
images: VisionInputImage[];
|
package/dist/provider.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"provider.d.ts","sourceRoot":"","sources":["../src/provider.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;
|
|
1
|
+
{"version":3,"file":"provider.d.ts","sourceRoot":"","sources":["../src/provider.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,WAAW,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAI7D,MAAM,WAAW,gBAAgB;IAC/B,OAAO,EAAE,MAAM,CAAC;IAChB,KAAK,CAAC,EAAE,MAAM,CAAC;CAChB;AAED,MAAM,WAAW,gBAAgB;IAC/B,IAAI,EAAE,MAAM,CAAC;IACb,KAAK,EAAE,MAAM,CAAC;IACd,KAAK,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;CACjC;AAiBD,wBAAsB,uBAAuB,CAAC,OAAO,EAAE;IACrD,MAAM,EAAE,YAAY,CAAC;IACrB,MAAM,EAAE,WAAW,CAAC;IACpB,MAAM,EAAE,MAAM,CAAC;IACf,MAAM,EAAE,gBAAgB,EAAE,CAAC;IAC3B,MAAM,CAAC,EAAE,WAAW,CAAC;CACtB,GAAG,OAAO,CAAC,gBAAgB,CAAC,CAqH5B;AAED,wBAAsB,qBAAqB,CAAC,OAAO,EAAE;IACnD,MAAM,EAAE,YAAY,CAAC;IACrB,MAAM,EAAE,MAAM,CAAC;IACf,MAAM,EAAE,gBAAgB,EAAE,CAAC;CAC5B,GAAG,OAAO,CAAC,gBAAgB,CAAC,CA6D5B"}
|
package/dist/provider.js
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
|
-
import { chatCompletionsUrl
|
|
2
|
-
import { VisionError } from "./errors.js";
|
|
1
|
+
import { chatCompletionsUrl } from "./config.js";
|
|
2
|
+
import { VisionError, errorMessage } from "./errors.js";
|
|
3
3
|
function responseText(content) {
|
|
4
4
|
if (typeof content === "string" && content.trim())
|
|
5
5
|
return content.trim();
|
|
@@ -18,7 +18,9 @@ function responseText(content) {
|
|
|
18
18
|
return parts.length > 0 ? parts.join("\n").trim() : undefined;
|
|
19
19
|
}
|
|
20
20
|
export async function requestVisionCompletion(options) {
|
|
21
|
-
|
|
21
|
+
if (!options.target.model) {
|
|
22
|
+
throw new VisionError("CONFIG_ERROR", "A vision model target is missing a model name");
|
|
23
|
+
}
|
|
22
24
|
if (options.images.length === 0) {
|
|
23
25
|
throw new VisionError("INVALID_SOURCE", "At least one image is required");
|
|
24
26
|
}
|
|
@@ -34,21 +36,26 @@ export async function requestVisionCompletion(options) {
|
|
|
34
36
|
image_url: { url: image.dataUrl, detail: "high" },
|
|
35
37
|
});
|
|
36
38
|
}
|
|
37
|
-
const headers = new Headers(options.
|
|
39
|
+
const headers = new Headers(options.target.headers);
|
|
38
40
|
headers.set("content-type", "application/json");
|
|
39
|
-
if (options.
|
|
40
|
-
headers.set("authorization", `Bearer ${options.
|
|
41
|
+
if (options.target.apiKey && !headers.has("authorization")) {
|
|
42
|
+
headers.set("authorization", `Bearer ${options.target.apiKey}`);
|
|
41
43
|
}
|
|
42
44
|
const controller = new AbortController();
|
|
43
45
|
const timer = setTimeout(() => controller.abort(), options.config.timeoutMs);
|
|
46
|
+
const external = options.signal;
|
|
47
|
+
const onExternalAbort = () => controller.abort();
|
|
48
|
+
if (external?.aborted)
|
|
49
|
+
controller.abort();
|
|
50
|
+
external?.addEventListener("abort", onExternalAbort, { once: true });
|
|
44
51
|
let response;
|
|
45
52
|
let raw;
|
|
46
53
|
try {
|
|
47
|
-
response = await fetch(chatCompletionsUrl(options.
|
|
54
|
+
response = await fetch(chatCompletionsUrl(options.target.baseUrl), {
|
|
48
55
|
method: "POST",
|
|
49
56
|
headers,
|
|
50
57
|
body: JSON.stringify({
|
|
51
|
-
model: options.
|
|
58
|
+
model: options.target.model,
|
|
52
59
|
messages: [{ role: "user", content }],
|
|
53
60
|
max_tokens: options.config.maxTokens,
|
|
54
61
|
stream: false,
|
|
@@ -58,6 +65,9 @@ export async function requestVisionCompletion(options) {
|
|
|
58
65
|
raw = await response.text();
|
|
59
66
|
}
|
|
60
67
|
catch (error) {
|
|
68
|
+
if (external?.aborted) {
|
|
69
|
+
throw new VisionError("API_ABORTED", "Request aborted because another vision model target finished first");
|
|
70
|
+
}
|
|
61
71
|
if (controller.signal.aborted) {
|
|
62
72
|
throw new VisionError("API_TIMEOUT", `Vision API timed out after ${options.config.timeoutMs} ms`);
|
|
63
73
|
}
|
|
@@ -67,6 +77,7 @@ export async function requestVisionCompletion(options) {
|
|
|
67
77
|
}
|
|
68
78
|
finally {
|
|
69
79
|
clearTimeout(timer);
|
|
80
|
+
external?.removeEventListener("abort", onExternalAbort);
|
|
70
81
|
}
|
|
71
82
|
let payload;
|
|
72
83
|
try {
|
|
@@ -106,10 +117,59 @@ export async function requestVisionCompletion(options) {
|
|
|
106
117
|
}
|
|
107
118
|
return {
|
|
108
119
|
text,
|
|
109
|
-
model: typeof record.model === "string" ? record.model : options.
|
|
120
|
+
model: typeof record.model === "string" ? record.model : options.target.model,
|
|
110
121
|
...(record.usage && typeof record.usage === "object"
|
|
111
122
|
? { usage: record.usage }
|
|
112
123
|
: {}),
|
|
113
124
|
};
|
|
114
125
|
}
|
|
126
|
+
export async function raceVisionCompletions(options) {
|
|
127
|
+
const targets = options.config.targets;
|
|
128
|
+
if (targets.length === 0) {
|
|
129
|
+
throw new VisionError("CONFIG_ERROR", "No vision model configured. Add providers to the settings file or set AGENT_VISION_MODEL (or VISION_MODEL / OPENAI_MODEL)");
|
|
130
|
+
}
|
|
131
|
+
const single = targets.length === 1 ? targets[0] : undefined;
|
|
132
|
+
if (single) {
|
|
133
|
+
return requestVisionCompletion({
|
|
134
|
+
config: options.config,
|
|
135
|
+
target: single,
|
|
136
|
+
prompt: options.prompt,
|
|
137
|
+
images: options.images,
|
|
138
|
+
});
|
|
139
|
+
}
|
|
140
|
+
const attempts = targets.map((target) => {
|
|
141
|
+
const controller = new AbortController();
|
|
142
|
+
return {
|
|
143
|
+
target,
|
|
144
|
+
controller,
|
|
145
|
+
promise: requestVisionCompletion({
|
|
146
|
+
config: options.config,
|
|
147
|
+
target,
|
|
148
|
+
prompt: options.prompt,
|
|
149
|
+
images: options.images,
|
|
150
|
+
signal: controller.signal,
|
|
151
|
+
}),
|
|
152
|
+
};
|
|
153
|
+
});
|
|
154
|
+
return new Promise((resolve, reject) => {
|
|
155
|
+
let pending = attempts.length;
|
|
156
|
+
const failures = [];
|
|
157
|
+
for (const attempt of attempts) {
|
|
158
|
+
attempt.promise.then((result) => {
|
|
159
|
+
for (const other of attempts) {
|
|
160
|
+
if (other !== attempt && !other.controller.signal.aborted) {
|
|
161
|
+
other.controller.abort();
|
|
162
|
+
}
|
|
163
|
+
}
|
|
164
|
+
resolve(result);
|
|
165
|
+
}, (error) => {
|
|
166
|
+
failures.push(`${attempt.target.label}: ${errorMessage(error)}`);
|
|
167
|
+
pending -= 1;
|
|
168
|
+
if (pending === 0) {
|
|
169
|
+
reject(new VisionError("API_ERROR", `All vision model targets failed:\n${failures.join("\n")}`));
|
|
170
|
+
}
|
|
171
|
+
});
|
|
172
|
+
}
|
|
173
|
+
});
|
|
174
|
+
}
|
|
115
175
|
//# sourceMappingURL=provider.js.map
|
package/dist/provider.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"provider.js","sourceRoot":"","sources":["../src/provider.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,kBAAkB,EAAE,
|
|
1
|
+
{"version":3,"file":"provider.js","sourceRoot":"","sources":["../src/provider.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,kBAAkB,EAAE,MAAM,aAAa,CAAC;AACjD,OAAO,EAAE,WAAW,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAaxD,SAAS,YAAY,CAAC,OAAgB;IACpC,IAAI,OAAO,OAAO,KAAK,QAAQ,IAAI,OAAO,CAAC,IAAI,EAAE;QAAE,OAAO,OAAO,CAAC,IAAI,EAAE,CAAC;IACzE,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,OAAO,CAAC;QAAE,OAAO,SAAS,CAAC;IAC9C,MAAM,KAAK,GAAG,OAAO;SAClB,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE;QACZ,IAAI,CAAC,IAAI,IAAI,OAAO,IAAI,KAAK,QAAQ;YAAE,OAAO,SAAS,CAAC;QACxD,MAAM,KAAK,GAAG,IAA+B,CAAC;QAC9C,OAAO,KAAK,CAAC,IAAI,KAAK,MAAM,IAAI,OAAO,KAAK,CAAC,IAAI,KAAK,QAAQ;YAC5D,CAAC,CAAC,KAAK,CAAC,IAAI;YACZ,CAAC,CAAC,SAAS,CAAC;IAChB,CAAC,CAAC;SACD,MAAM,CAAC,CAAC,IAAI,EAAkB,EAAE,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC;IACnD,OAAO,KAAK,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,SAAS,CAAC;AAChE,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,uBAAuB,CAAC,OAM7C;IACC,IAAI,CAAC,OAAO,CAAC,MAAM,CAAC,KAAK,EAAE,CAAC;QAC1B,MAAM,IAAI,WAAW,CAAC,cAAc,EAAE,+CAA+C,CAAC,CAAC;IACzF,CAAC;IACD,IAAI,OAAO,CAAC,MAAM,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QAChC,MAAM,IAAI,WAAW,CAAC,gBAAgB,EAAE,gCAAgC,CAAC,CAAC;IAC5E,CAAC;IAED,MAAM,OAAO,GAAmC;QAC9C,EAAE,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,OAAO,CAAC,MAAM,EAAE;KACvC,CAAC;IACF,KAAK,MAAM,CAAC,KAAK,EAAE,KAAK,CAAC,IAAI,OAAO,CAAC,MAAM,CAAC,OAAO,EAAE,EAAE,CAAC;QACtD,IAAI,KAAK,CAAC,KAAK,EAAE,CAAC;YAChB,OAAO,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,SAAS,KAAK,GAAG,CAAC,KAAK,KAAK,CAAC,KAAK,EAAE,EAAE,CAAC,CAAC;QAC7E,CAAC;QACD,OAAO,CAAC,IAAI,CAAC;YACX,IAAI,EAAE,WAAW;YACjB,SAAS,EAAE,EAAE,GAAG,EAAE,KAAK,CAAC,OAAO,EAAE,MAAM,EAAE,MAAM,EAAE;SAClD,CAAC,CAAC;IACL,CAAC;IAED,MAAM,OAAO,GAAG,IAAI,OAAO,CAAC,OAAO,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC;IACpD,OAAO,CAAC,GAAG,CAAC,cAAc,EAAE,kBAAkB,CAAC,CAAC;IAChD,IAAI,OAAO,CAAC,MAAM,CAAC,MAAM,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,eAAe,CAAC,EAAE,CAAC;QAC3D,OAAO,CAAC,GAAG,CAAC,eAAe,EAAE,UAAU,OAAO,CAAC,MAAM,CAAC,MAAM,EAAE,CAAC,CAAC;IAClE,CAAC;IAED,MAAM,UAAU,GAAG,IAAI,eAAe,EAAE,CAAC;IACzC,MAAM,KAAK,GAAG,UAAU,CAAC,GAAG,EAAE,CAAC,UAAU,CAAC,KAAK,EAAE,EAAE,OAAO,CAAC,MAAM,CAAC,SAAS,CAAC,CAAC;IAC7E,MAAM,QAAQ,GAAG,OAAO,CAAC,MAAM,CAAC;IAChC,MAAM,eAAe,GAAG,GAAG,EAAE,CAAC,UAAU,CAAC,KAAK,EAAE,CAAC;IACjD,IAAI,QAAQ,EAAE,OAAO;QAAE,UAAU,CAAC,KAAK,EAAE,CAAC;IAC1C,QAAQ,EAAE,gBAAgB,CAAC,OAAO,EAAE,eAAe,EAAE,EAAE,IAAI,EAAE,IAAI,EAAE,CAAC,CAAC;IACrE,IAAI,QAAkB,CAAC;IACvB,IAAI,GAAW,CAAC;IAChB,IAAI,CAAC;QACH,QAAQ,GAAG,MAAM,KAAK,CAAC,kBAAkB,CAAC,OAAO,CAAC,MAAM,CAAC,OAAO,CAAC,EAAE;YACjE,MAAM,EAAE,MAAM;YACd,OAAO;YACP,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC;gBACnB,KAAK,EAAE,OAAO,CAAC,MAAM,CAAC,KAAK;gBAC3B,QAAQ,EAAE,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,OAAO,EAAE,CAAC;gBACrC,UAAU,EAAE,OAAO,CAAC,MAAM,CAAC,SAAS;gBACpC,MAAM,EAAE,KAAK;aACd,CAAC;YACF,MAAM,EAAE,UAAU,CAAC,MAAM;SAC1B,CAAC,CAAC;QACH,GAAG,GAAG,MAAM,QAAQ,CAAC,IAAI,EAAE,CAAC;IAC9B,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,IAAI,QAAQ,EAAE,OAAO,EAAE,CAAC;YACtB,MAAM,IAAI,WAAW,CACnB,aAAa,EACb,oEAAoE,CACrE,CAAC;QACJ,CAAC;QACD,IAAI,UAAU,CAAC,MAAM,CAAC,OAAO,EAAE,CAAC;YAC9B,MAAM,IAAI,WAAW,CACnB,aAAa,EACb,8BAA8B,OAAO,CAAC,MAAM,CAAC,SAAS,KAAK,CAC5D,CAAC;QACJ,CAAC;QACD,MAAM,IAAI,WAAW,CAAC,WAAW,EAAE,gCAAgC,EAAE;YACnE,KAAK,EAAE,KAAK;SACb,CAAC,CAAC;IACL,CAAC;YAAS,CAAC;QACT,YAAY,CAAC,KAAK,CAAC,CAAC;QACpB,QAAQ,EAAE,mBAAmB,CAAC,OAAO,EAAE,eAAe,CAAC,CAAC;IAC1D,CAAC;IAED,IAAI,OAAgB,CAAC;IACrB,IAAI,CAAC;QACH,OAAO,GAAG,GAAG,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC;IACvC,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,GAAG,SAAS,CAAC;IACtB,CAAC;IACD,IAAI,CAAC,QAAQ,CAAC,EAAE,EAAE,CAAC;QACjB,MAAM,MAAM,GAAG,OAAO,IAAI,OAAO,OAAO,KAAK,QAAQ;YACnD,CAAC,CAAE,OAAmC;YACtC,CAAC,CAAC,SAAS,CAAC;QACd,MAAM,KAAK,GAAG,MAAM,EAAE,KAAK,CAAC;QAC5B,MAAM,UAAU,GAAG,KAAK,IAAI,OAAO,KAAK,KAAK,QAAQ;YACnD,CAAC,CAAE,KAAiC,CAAC,OAAO;YAC5C,CAAC,CAAC,SAAS,CAAC;QACd,MAAM,MAAM,GAAG,OAAO,UAAU,KAAK,QAAQ;YAC3C,CAAC,CAAC,UAAU;YACZ,CAAC,CAAC,GAAG,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,CAAC,IAAI,QAAQ,CAAC,UAAU,CAAC;QAC7C,MAAM,IAAI,WAAW,CACnB,WAAW,EACX,4BAA4B,QAAQ,CAAC,MAAM,KAAK,MAAM,EAAE,CACzD,CAAC;IACJ,CAAC;IACD,IAAI,CAAC,OAAO,IAAI,OAAO,OAAO,KAAK,QAAQ,EAAE,CAAC;QAC5C,MAAM,IAAI,WAAW,CAAC,oBAAoB,EAAE,kCAAkC,CAAC,CAAC;IAClF,CAAC;IACD,MAAM,MAAM,GAAG,OAAkC,CAAC;IAClD,MAAM,OAAO,GAAG,MAAM,CAAC,OAAO,CAAC;IAC/B,MAAM,WAAW,GAAG,KAAK,CAAC,OAAO,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;IACpE,MAAM,OAAO,GAAG,WAAW,IAAI,OAAO,WAAW,KAAK,QAAQ;QAC5D,CAAC,CAAE,WAAuC,CAAC,OAAO;QAClD,CAAC,CAAC,SAAS,CAAC;IACd,MAAM,YAAY,GAAG,OAAO,IAAI,OAAO,OAAO,KAAK,QAAQ;QACzD,CAAC,CAAE,OAAmC,CAAC,OAAO;QAC9C,CAAC,CAAC,SAAS,CAAC;IACd,MAAM,IAAI,GAAG,YAAY,CAAC,YAAY,CAAC,CAAC;IACxC,IAAI,CAAC,IAAI,EAAE,CAAC;QACV,MAAM,IAAI,WAAW,CACnB,oBAAoB,EACpB,2DAA2D,CAC5D,CAAC;IACJ,CAAC;IACD,OAAO;QACL,IAAI;QACJ,KAAK,EAAE,OAAO,MAAM,CAAC,KAAK,KAAK,QAAQ,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,OAAO,CAAC,MAAM,CAAC,KAAK;QAC7E,GAAG,CAAC,MAAM,CAAC,KAAK,IAAI,OAAO,MAAM,CAAC,KAAK,KAAK,QAAQ;YAClD,CAAC,CAAC,EAAE,KAAK,EAAE,MAAM,CAAC,KAAgC,EAAE;YACpD,CAAC,CAAC,EAAE,CAAC;KACR,CAAC;AACJ,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,qBAAqB,CAAC,OAI3C;IACC,MAAM,OAAO,GAAG,OAAO,CAAC,MAAM,CAAC,OAAO,CAAC;IACvC,IAAI,OAAO,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QACzB,MAAM,IAAI,WAAW,CACnB,cAAc,EACd,2HAA2H,CAC5H,CAAC;IACJ,CAAC;IACD,MAAM,MAAM,GAAG,OAAO,CAAC,MAAM,KAAK,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;IAC7D,IAAI,MAAM,EAAE,CAAC;QACX,OAAO,uBAAuB,CAAC;YAC7B,MAAM,EAAE,OAAO,CAAC,MAAM;YACtB,MAAM,EAAE,MAAM;YACd,MAAM,EAAE,OAAO,CAAC,MAAM;YACtB,MAAM,EAAE,OAAO,CAAC,MAAM;SACvB,CAAC,CAAC;IACL,CAAC;IAED,MAAM,QAAQ,GAAG,OAAO,CAAC,GAAG,CAAC,CAAC,MAAM,EAAE,EAAE;QACtC,MAAM,UAAU,GAAG,IAAI,eAAe,EAAE,CAAC;QACzC,OAAO;YACL,MAAM;YACN,UAAU;YACV,OAAO,EAAE,uBAAuB,CAAC;gBAC/B,MAAM,EAAE,OAAO,CAAC,MAAM;gBACtB,MAAM;gBACN,MAAM,EAAE,OAAO,CAAC,MAAM;gBACtB,MAAM,EAAE,OAAO,CAAC,MAAM;gBACtB,MAAM,EAAE,UAAU,CAAC,MAAM;aAC1B,CAAC;SACH,CAAC;IACJ,CAAC,CAAC,CAAC;IAEH,OAAO,IAAI,OAAO,CAAmB,CAAC,OAAO,EAAE,MAAM,EAAE,EAAE;QACvD,IAAI,OAAO,GAAG,QAAQ,CAAC,MAAM,CAAC;QAC9B,MAAM,QAAQ,GAAa,EAAE,CAAC;QAC9B,KAAK,MAAM,OAAO,IAAI,QAAQ,EAAE,CAAC;YAC/B,OAAO,CAAC,OAAO,CAAC,IAAI,CAClB,CAAC,MAAM,EAAE,EAAE;gBACT,KAAK,MAAM,KAAK,IAAI,QAAQ,EAAE,CAAC;oBAC7B,IAAI,KAAK,KAAK,OAAO,IAAI,CAAC,KAAK,CAAC,UAAU,CAAC,MAAM,CAAC,OAAO,EAAE,CAAC;wBAC1D,KAAK,CAAC,UAAU,CAAC,KAAK,EAAE,CAAC;oBAC3B,CAAC;gBACH,CAAC;gBACD,OAAO,CAAC,MAAM,CAAC,CAAC;YAClB,CAAC,EACD,CAAC,KAAc,EAAE,EAAE;gBACjB,QAAQ,CAAC,IAAI,CAAC,GAAG,OAAO,CAAC,MAAM,CAAC,KAAK,KAAK,YAAY,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC;gBACjE,OAAO,IAAI,CAAC,CAAC;gBACb,IAAI,OAAO,KAAK,CAAC,EAAE,CAAC;oBAClB,MAAM,CACJ,IAAI,WAAW,CACb,WAAW,EACX,qCAAqC,QAAQ,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAC3D,CACF,CAAC;gBACJ,CAAC;YACH,CAAC,CACF,CAAC;QACJ,CAAC;IACH,CAAC,CAAC,CAAC;AACL,CAAC"}
|
package/dist/server.d.ts
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
|
|
2
|
-
export declare const SERVER_VERSION = "0.
|
|
2
|
+
export declare const SERVER_VERSION = "0.2.0";
|
|
3
3
|
export declare function createServer(): McpServer;
|
|
4
4
|
export declare function startStdioServer(): Promise<void>;
|
|
5
5
|
//# sourceMappingURL=server.d.ts.map
|
package/dist/server.js
CHANGED
|
@@ -3,7 +3,7 @@ import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js"
|
|
|
3
3
|
import { z } from "zod";
|
|
4
4
|
import { analyzeImage, analyzeVideo } from "./analyze.js";
|
|
5
5
|
import { errorMessage } from "./errors.js";
|
|
6
|
-
export const SERVER_VERSION = "0.
|
|
6
|
+
export const SERVER_VERSION = "0.2.0";
|
|
7
7
|
function failure(error) {
|
|
8
8
|
return {
|
|
9
9
|
content: [{ type: "text", text: errorMessage(error) }],
|
|
@@ -17,7 +17,7 @@ export function createServer() {
|
|
|
17
17
|
});
|
|
18
18
|
server.registerTool("analyze_image", {
|
|
19
19
|
title: "Analyze image",
|
|
20
|
-
description: "Analyze an image from a local path, HTTP(S) URL, or base64 data URI. Use
|
|
20
|
+
description: "Analyze an image from a local path, HTTP(S) URL, or base64 data URI. Use ONLY when you cannot view the image yourself (no native multimodal/image input) — if you can see the image directly, answer from your own vision instead. For screenshots, UI, OCR, diagrams, photos, and visual debugging.",
|
|
21
21
|
inputSchema: z.object({
|
|
22
22
|
source: z.string().min(1).describe("Local path, HTTP(S) URL, or data:image/... URI"),
|
|
23
23
|
prompt: z.string().optional().describe("The user's specific question about the image"),
|
package/dist/server.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"server.js","sourceRoot":"","sources":["../src/server.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,SAAS,EAAE,MAAM,yCAAyC,CAAC;AACpE,OAAO,EAAE,oBAAoB,EAAE,MAAM,2CAA2C,CAAC;AACjF,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAC;AACxB,OAAO,EAAE,YAAY,EAAE,YAAY,EAAE,MAAM,cAAc,CAAC;AAC1D,OAAO,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAE3C,MAAM,CAAC,MAAM,cAAc,GAAG,OAAO,CAAC;AAEtC,SAAS,OAAO,CAAC,KAAc;IAC7B,OAAO;QACL,OAAO,EAAE,CAAC,EAAE,IAAI,EAAE,MAAe,EAAE,IAAI,EAAE,YAAY,CAAC,KAAK,CAAC,EAAE,CAAC;QAC/D,OAAO,EAAE,IAAI;KACd,CAAC;AACJ,CAAC;AAED,MAAM,UAAU,YAAY;IAC1B,MAAM,MAAM,GAAG,IAAI,SAAS,CAAC;QAC3B,IAAI,EAAE,cAAc;QACpB,OAAO,EAAE,cAAc;KACxB,CAAC,CAAC;IAEH,MAAM,CAAC,YAAY,CACjB,eAAe,EACf;QACE,KAAK,EAAE,eAAe;QACtB,WAAW,EACT,
|
|
1
|
+
{"version":3,"file":"server.js","sourceRoot":"","sources":["../src/server.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,SAAS,EAAE,MAAM,yCAAyC,CAAC;AACpE,OAAO,EAAE,oBAAoB,EAAE,MAAM,2CAA2C,CAAC;AACjF,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAC;AACxB,OAAO,EAAE,YAAY,EAAE,YAAY,EAAE,MAAM,cAAc,CAAC;AAC1D,OAAO,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAE3C,MAAM,CAAC,MAAM,cAAc,GAAG,OAAO,CAAC;AAEtC,SAAS,OAAO,CAAC,KAAc;IAC7B,OAAO;QACL,OAAO,EAAE,CAAC,EAAE,IAAI,EAAE,MAAe,EAAE,IAAI,EAAE,YAAY,CAAC,KAAK,CAAC,EAAE,CAAC;QAC/D,OAAO,EAAE,IAAI;KACd,CAAC;AACJ,CAAC;AAED,MAAM,UAAU,YAAY;IAC1B,MAAM,MAAM,GAAG,IAAI,SAAS,CAAC;QAC3B,IAAI,EAAE,cAAc;QACpB,OAAO,EAAE,cAAc;KACxB,CAAC,CAAC;IAEH,MAAM,CAAC,YAAY,CACjB,eAAe,EACf;QACE,KAAK,EAAE,eAAe;QACtB,WAAW,EACT,sSAAsS;QACxS,WAAW,EAAE,CAAC,CAAC,MAAM,CAAC;YACpB,MAAM,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC,gDAAgD,CAAC;YACpF,MAAM,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,8CAA8C,CAAC;SACvF,CAAC;KACH,EACD,KAAK,EAAE,EAAE,MAAM,EAAE,MAAM,EAAE,EAAE,EAAE;QAC3B,IAAI,CAAC;YACH,MAAM,MAAM,GAAG,MAAM,YAAY,CAAC,EAAE,MAAM,EAAE,GAAG,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC,CAAC;YAC7E,OAAO,EAAE,OAAO,EAAE,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,MAAM,CAAC,IAAI,EAAE,CAAC,EAAE,CAAC;QAC5D,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACf,OAAO,OAAO,CAAC,KAAK,CAAC,CAAC;QACxB,CAAC;IACH,CAAC,CACF,CAAC;IAEF,MAAM,CAAC,YAAY,CACjB,eAAe,EACf;QACE,KAAK,EAAE,eAAe;QACtB,WAAW,EACT,8IAA8I;QAChJ,WAAW,EAAE,CAAC,CAAC,MAAM,CAAC;YACpB,MAAM,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC,2BAA2B,CAAC;YAC/D,MAAM,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,8CAA8C,CAAC;YACtF,UAAU,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,uCAAuC,CAAC;SACzG,CAAC;KACH,EACD,KAAK,EAAE,EAAE,MAAM,EAAE,MAAM,EAAE,UAAU,EAAE,EAAE,EAAE;QACvC,IAAI,CAAC;YACH,MAAM,MAAM,GAAG,MAAM,YAAY,CAAC;gBAChC,MAAM;gBACN,GAAG,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;gBAC7B,GAAG,CAAC,UAAU,CAAC,CAAC,CAAC,EAAE,SAAS,EAAE,UAAU,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;aACjD,CAAC,CAAC;YACH,OAAO,EAAE,OAAO,EAAE,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,MAAM,CAAC,IAAI,EAAE,CAAC,EAAE,CAAC;QAC5D,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACf,OAAO,OAAO,CAAC,KAAK,CAAC,CAAC;QACxB,CAAC;IACH,CAAC,CACF,CAAC;IAEF,OAAO,MAAM,CAAC;AAChB,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,gBAAgB;IACpC,MAAM,MAAM,GAAG,YAAY,EAAE,CAAC;IAC9B,MAAM,SAAS,GAAG,IAAI,oBAAoB,EAAE,CAAC;IAC7C,MAAM,MAAM,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC;AAClC,CAAC"}
|
package/dist/video.d.ts
CHANGED
|
@@ -3,7 +3,7 @@ import type { VisionInputImage } from "./provider.js";
|
|
|
3
3
|
export interface VideoFrame extends VisionInputImage {
|
|
4
4
|
timestampSeconds: number;
|
|
5
5
|
}
|
|
6
|
-
export declare function ensureFfmpeg(): Promise<string>;
|
|
6
|
+
export declare function ensureFfmpeg(configuredPath?: string): Promise<string>;
|
|
7
7
|
export declare function extractVideoFrames(options: {
|
|
8
8
|
source: string;
|
|
9
9
|
maxFrames: number;
|
package/dist/video.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"video.d.ts","sourceRoot":"","sources":["../src/video.ts"],"names":[],"mappings":"AAMA,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAGhD,OAAO,KAAK,EAAE,gBAAgB,EAAE,MAAM,eAAe,CAAC;AAEtD,MAAM,WAAW,UAAW,SAAQ,gBAAgB;IAClD,gBAAgB,EAAE,MAAM,CAAC;CAC1B;AAoCD,wBAAgB,YAAY,
|
|
1
|
+
{"version":3,"file":"video.d.ts","sourceRoot":"","sources":["../src/video.ts"],"names":[],"mappings":"AAMA,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAGhD,OAAO,KAAK,EAAE,gBAAgB,EAAE,MAAM,eAAe,CAAC;AAEtD,MAAM,WAAW,UAAW,SAAQ,gBAAgB;IAClD,gBAAgB,EAAE,MAAM,CAAC;CAC1B;AAoCD,wBAAgB,YAAY,CAAC,cAAc,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,CAAC,CAkCrE;AAyGD,wBAAsB,kBAAkB,CAAC,OAAO,EAAE;IAChD,MAAM,EAAE,MAAM,CAAC;IACf,SAAS,EAAE,MAAM,CAAC;IAClB,MAAM,EAAE,YAAY,CAAC;IACrB,QAAQ,CAAC,EAAE,MAAM,CAAC;CACnB,GAAG,OAAO,CAAC,UAAU,EAAE,CAAC,CA8DxB"}
|
package/dist/video.js
CHANGED
|
@@ -37,9 +37,9 @@ async function bundledFfmpeg() {
|
|
|
37
37
|
return undefined;
|
|
38
38
|
}
|
|
39
39
|
}
|
|
40
|
-
export function ensureFfmpeg() {
|
|
40
|
+
export function ensureFfmpeg(configuredPath) {
|
|
41
41
|
ffmpegReady ??= (async () => {
|
|
42
|
-
const configured = process.env.AGENT_VISION_FFMPEG_PATH?.trim();
|
|
42
|
+
const configured = configuredPath?.trim() || process.env.AGENT_VISION_FFMPEG_PATH?.trim();
|
|
43
43
|
if (configured) {
|
|
44
44
|
const resolved = path.resolve(configured);
|
|
45
45
|
if (!(await usableFile(resolved))) {
|
|
@@ -70,8 +70,8 @@ function expandLocalPath(source) {
|
|
|
70
70
|
return path.join(homedir(), source.slice(2));
|
|
71
71
|
return path.resolve(source.startsWith("@") ? source.slice(1) : source);
|
|
72
72
|
}
|
|
73
|
-
async function runFfmpeg(args, allowFailure = false, timeoutMs = 120_000) {
|
|
74
|
-
const command = await ensureFfmpeg();
|
|
73
|
+
async function runFfmpeg(args, allowFailure = false, timeoutMs = 120_000, configuredPath) {
|
|
74
|
+
const command = await ensureFfmpeg(configuredPath);
|
|
75
75
|
return new Promise((resolve, reject) => {
|
|
76
76
|
const child = spawn(command, args, { stdio: ["ignore", "ignore", "pipe"] });
|
|
77
77
|
let stderr = "";
|
|
@@ -102,8 +102,8 @@ async function runFfmpeg(args, allowFailure = false, timeoutMs = 120_000) {
|
|
|
102
102
|
});
|
|
103
103
|
});
|
|
104
104
|
}
|
|
105
|
-
async function videoDuration(filePath, timeoutMs) {
|
|
106
|
-
const output = await runFfmpeg(["-hide_banner", "-i", filePath], true, timeoutMs);
|
|
105
|
+
async function videoDuration(filePath, timeoutMs, configuredPath) {
|
|
106
|
+
const output = await runFfmpeg(["-hide_banner", "-i", filePath], true, timeoutMs, configuredPath);
|
|
107
107
|
const match = output.match(/Duration:\s*(\d+):(\d+):(\d+(?:\.\d+)?)/);
|
|
108
108
|
if (!match?.[1] || !match[2] || !match[3]) {
|
|
109
109
|
throw new VisionError("UNSUPPORTED_VIDEO", "Could not determine video duration");
|
|
@@ -155,7 +155,7 @@ export async function extractVideoFrames(options) {
|
|
|
155
155
|
const workDir = await mkdtemp(prefix);
|
|
156
156
|
try {
|
|
157
157
|
const inputPath = await prepareInput(options.source.trim(), workDir, options.config);
|
|
158
|
-
const duration = await videoDuration(inputPath, options.config.timeoutMs);
|
|
158
|
+
const duration = await videoDuration(inputPath, options.config.timeoutMs, options.config.ffmpegPath);
|
|
159
159
|
const timestamps = Array.from({ length: options.maxFrames }, (_, index) => ((index + 0.5) * duration) / options.maxFrames);
|
|
160
160
|
for (const [index, timestamp] of timestamps.entries()) {
|
|
161
161
|
const outputPath = path.join(workDir, `frame-${String(index + 1).padStart(3, "0")}.jpg`);
|
|
@@ -175,7 +175,7 @@ export async function extractVideoFrames(options) {
|
|
|
175
175
|
"3",
|
|
176
176
|
"-y",
|
|
177
177
|
outputPath,
|
|
178
|
-
], false, options.config.timeoutMs);
|
|
178
|
+
], false, options.config.timeoutMs, options.config.ffmpegPath);
|
|
179
179
|
}
|
|
180
180
|
const frameNames = (await readdir(workDir))
|
|
181
181
|
.filter((name) => /^frame-\d+\.jpg$/.test(name))
|
package/dist/video.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"video.js","sourceRoot":"","sources":["../src/video.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,KAAK,EAAE,MAAM,oBAAoB,CAAC;AAC3C,OAAO,EAAE,MAAM,EAAE,KAAK,EAAE,QAAQ,EAAE,OAAO,EAAE,EAAE,EAAE,IAAI,EAAE,MAAM,kBAAkB,CAAC;AAC9E,OAAO,EAAE,SAAS,IAAI,WAAW,EAAE,MAAM,SAAS,CAAC;AACnD,OAAO,EAAE,OAAO,EAAE,MAAM,EAAE,MAAM,SAAS,CAAC;AAC1C,OAAO,IAAI,MAAM,WAAW,CAAC;AAC7B,OAAO,EAAE,OAAO,EAAE,MAAM,kBAAkB,CAAC;AAE3C,OAAO,EAAE,WAAW,EAAE,MAAM,aAAa,CAAC;AAC1C,OAAO,EAAE,kBAAkB,EAAE,MAAM,cAAc,CAAC;AAOlD,IAAI,WAAwC,CAAC;AAE7C,KAAK,UAAU,UAAU,CAAC,QAAgB;IACxC,IAAI,CAAC;QACH,MAAM,MAAM,CACV,QAAQ,EACR,OAAO,CAAC,QAAQ,KAAK,OAAO,CAAC,CAAC,CAAC,WAAW,CAAC,IAAI,CAAC,CAAC,CAAC,WAAW,CAAC,IAAI,CACnE,CAAC;QACF,OAAO,IAAI,CAAC;IACd,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,KAAK,CAAC;IACf,CAAC;AACH,CAAC;AAED,KAAK,UAAU,YAAY;IACzB,MAAM,cAAc,GAAG,OAAO,CAAC,QAAQ,KAAK,OAAO,CAAC,CAAC,CAAC,YAAY,CAAC,CAAC,CAAC,QAAQ,CAAC;IAC9E,KAAK,MAAM,SAAS,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,IAAI,IAAI,EAAE,CAAC,CAAC,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,CAAC;QACvE,IAAI,CAAC,SAAS;YAAE,SAAS;QACzB,MAAM,SAAS,GAAG,IAAI,CAAC,IAAI,CAAC,SAAS,EAAE,cAAc,CAAC,CAAC;QACvD,IAAI,MAAM,UAAU,CAAC,SAAS,CAAC;YAAE,OAAO,SAAS,CAAC;IACpD,CAAC;IACD,OAAO,SAAS,CAAC;AACnB,CAAC;AAED,KAAK,UAAU,aAAa;IAC1B,IAAI,CAAC;QACH,MAAM,QAAQ,GAAG,MAAM,MAAM,CAAC,0BAA0B,CAAC,CAAC;QAC1D,MAAM,SAAS,GAAG,QAAQ,CAAC,OAAwC,CAAC;QACpE,OAAO,SAAS,EAAE,IAAI,CAAC;IACzB,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,SAAS,CAAC;IACnB,CAAC;AACH,CAAC;AAED,MAAM,UAAU,YAAY;
|
|
1
|
+
{"version":3,"file":"video.js","sourceRoot":"","sources":["../src/video.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,KAAK,EAAE,MAAM,oBAAoB,CAAC;AAC3C,OAAO,EAAE,MAAM,EAAE,KAAK,EAAE,QAAQ,EAAE,OAAO,EAAE,EAAE,EAAE,IAAI,EAAE,MAAM,kBAAkB,CAAC;AAC9E,OAAO,EAAE,SAAS,IAAI,WAAW,EAAE,MAAM,SAAS,CAAC;AACnD,OAAO,EAAE,OAAO,EAAE,MAAM,EAAE,MAAM,SAAS,CAAC;AAC1C,OAAO,IAAI,MAAM,WAAW,CAAC;AAC7B,OAAO,EAAE,OAAO,EAAE,MAAM,kBAAkB,CAAC;AAE3C,OAAO,EAAE,WAAW,EAAE,MAAM,aAAa,CAAC;AAC1C,OAAO,EAAE,kBAAkB,EAAE,MAAM,cAAc,CAAC;AAOlD,IAAI,WAAwC,CAAC;AAE7C,KAAK,UAAU,UAAU,CAAC,QAAgB;IACxC,IAAI,CAAC;QACH,MAAM,MAAM,CACV,QAAQ,EACR,OAAO,CAAC,QAAQ,KAAK,OAAO,CAAC,CAAC,CAAC,WAAW,CAAC,IAAI,CAAC,CAAC,CAAC,WAAW,CAAC,IAAI,CACnE,CAAC;QACF,OAAO,IAAI,CAAC;IACd,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,KAAK,CAAC;IACf,CAAC;AACH,CAAC;AAED,KAAK,UAAU,YAAY;IACzB,MAAM,cAAc,GAAG,OAAO,CAAC,QAAQ,KAAK,OAAO,CAAC,CAAC,CAAC,YAAY,CAAC,CAAC,CAAC,QAAQ,CAAC;IAC9E,KAAK,MAAM,SAAS,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,IAAI,IAAI,EAAE,CAAC,CAAC,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,CAAC;QACvE,IAAI,CAAC,SAAS;YAAE,SAAS;QACzB,MAAM,SAAS,GAAG,IAAI,CAAC,IAAI,CAAC,SAAS,EAAE,cAAc,CAAC,CAAC;QACvD,IAAI,MAAM,UAAU,CAAC,SAAS,CAAC;YAAE,OAAO,SAAS,CAAC;IACpD,CAAC;IACD,OAAO,SAAS,CAAC;AACnB,CAAC;AAED,KAAK,UAAU,aAAa;IAC1B,IAAI,CAAC;QACH,MAAM,QAAQ,GAAG,MAAM,MAAM,CAAC,0BAA0B,CAAC,CAAC;QAC1D,MAAM,SAAS,GAAG,QAAQ,CAAC,OAAwC,CAAC;QACpE,OAAO,SAAS,EAAE,IAAI,CAAC;IACzB,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,SAAS,CAAC;IACnB,CAAC;AACH,CAAC;AAED,MAAM,UAAU,YAAY,CAAC,cAAuB;IAClD,WAAW,KAAK,CAAC,KAAK,IAAI,EAAE;QAC1B,MAAM,UAAU,GAAG,cAAc,EAAE,IAAI,EAAE,IAAI,OAAO,CAAC,GAAG,CAAC,wBAAwB,EAAE,IAAI,EAAE,CAAC;QAC1F,IAAI,UAAU,EAAE,CAAC;YACf,MAAM,QAAQ,GAAG,IAAI,CAAC,OAAO,CAAC,UAAU,CAAC,CAAC;YAC1C,IAAI,CAAC,CAAC,MAAM,UAAU,CAAC,QAAQ,CAAC,CAAC,EAAE,CAAC;gBAClC,MAAM,IAAI,WAAW,CACnB,sBAAsB,EACtB,+CAA+C,QAAQ,EAAE,CAC1D,CAAC;YACJ,CAAC;YACD,OAAO,QAAQ,CAAC;QAClB,CAAC;QACD,MAAM,UAAU,GAAG,CAAC,MAAM,YAAY,EAAE,CAAC,IAAI,CAAC,MAAM,aAAa,EAAE,CAAC,CAAC;QACrE,IAAI,CAAC,UAAU,EAAE,CAAC;YAChB,MAAM,IAAI,WAAW,CACnB,sBAAsB,EACtB,2DAA2D,CAC5D,CAAC;QACJ,CAAC;QACD,IAAI,OAAO,CAAC,QAAQ,KAAK,OAAO,EAAE,CAAC;YACjC,IAAI,CAAC;gBACH,MAAM,KAAK,CAAC,UAAU,EAAE,KAAK,CAAC,CAAC;YACjC,CAAC;YAAC,OAAO,KAAK,EAAE,CAAC;gBACf,MAAM,IAAI,WAAW,CACnB,sBAAsB,EACtB,wDAAwD,EACxD,EAAE,KAAK,EAAE,KAAK,EAAE,CACjB,CAAC;YACJ,CAAC;QACH,CAAC;QACD,OAAO,UAAU,CAAC;IACpB,CAAC,CAAC,EAAE,CAAC;IACL,OAAO,WAAW,CAAC;AACrB,CAAC;AAED,SAAS,eAAe,CAAC,MAAc;IACrC,IAAI,MAAM,KAAK,GAAG;QAAE,OAAO,OAAO,EAAE,CAAC;IACrC,IAAI,MAAM,CAAC,UAAU,CAAC,IAAI,CAAC;QAAE,OAAO,IAAI,CAAC,IAAI,CAAC,OAAO,EAAE,EAAE,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC;IAC1E,OAAO,IAAI,CAAC,OAAO,CAAC,MAAM,CAAC,UAAU,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC;AACzE,CAAC;AAED,KAAK,UAAU,SAAS,CACtB,IAAc,EACd,YAAY,GAAG,KAAK,EACpB,SAAS,GAAG,OAAO,EACnB,cAAuB;IAEvB,MAAM,OAAO,GAAG,MAAM,YAAY,CAAC,cAAc,CAAC,CAAC;IACnD,OAAO,IAAI,OAAO,CAAC,CAAC,OAAO,EAAE,MAAM,EAAE,EAAE;QACrC,MAAM,KAAK,GAAG,KAAK,CAAC,OAAO,EAAE,IAAI,EAAE,EAAE,KAAK,EAAE,CAAC,QAAQ,EAAE,QAAQ,EAAE,MAAM,CAAC,EAAE,CAAC,CAAC;QAC5E,IAAI,MAAM,GAAG,EAAE,CAAC;QAChB,IAAI,QAAQ,GAAG,KAAK,CAAC;QACrB,MAAM,KAAK,GAAG,UAAU,CAAC,GAAG,EAAE;YAC5B,QAAQ,GAAG,IAAI,CAAC;YAChB,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;QACxB,CAAC,EAAE,SAAS,CAAC,CAAC;QACd,KAAK,CAAC,MAAM,CAAC,WAAW,CAAC,MAAM,CAAC,CAAC;QACjC,KAAK,CAAC,MAAM,CAAC,EAAE,CAAC,MAAM,EAAE,CAAC,KAAa,EAAE,EAAE;YACxC,IAAI,MAAM,CAAC,MAAM,GAAG,SAAS;gBAAE,MAAM,IAAI,KAAK,CAAC;QACjD,CAAC,CAAC,CAAC;QACH,KAAK,CAAC,EAAE,CAAC,OAAO,EAAE,CAAC,KAAK,EAAE,EAAE;YAC1B,YAAY,CAAC,KAAK,CAAC,CAAC;YACpB,MAAM,CAAC,IAAI,WAAW,CAAC,cAAc,EAAE,wBAAwB,EAAE,EAAE,KAAK,EAAE,KAAK,EAAE,CAAC,CAAC,CAAC;QACtF,CAAC,CAAC,CAAC;QACH,KAAK,CAAC,EAAE,CAAC,OAAO,EAAE,CAAC,IAAI,EAAE,EAAE;YACzB,YAAY,CAAC,KAAK,CAAC,CAAC;YACpB,IAAI,QAAQ,EAAE,CAAC;gBACb,MAAM,CAAC,IAAI,WAAW,CAAC,gBAAgB,EAAE,0BAA0B,SAAS,KAAK,CAAC,CAAC,CAAC;gBACpF,OAAO;YACT,CAAC;YACD,IAAI,IAAI,KAAK,CAAC,IAAI,YAAY;gBAAE,OAAO,CAAC,MAAM,CAAC,CAAC;;gBAC3C,MAAM,CAAC,IAAI,WAAW,CAAC,cAAc,EAAE,MAAM,CAAC,IAAI,EAAE,CAAC,KAAK,CAAC,CAAC,IAAI,CAAC,IAAI,2BAA2B,IAAI,EAAE,CAAC,CAAC,CAAC;QAChH,CAAC,CAAC,CAAC;IACL,CAAC,CAAC,CAAC;AACL,CAAC;AAED,KAAK,UAAU,aAAa,CAC1B,QAAgB,EAChB,SAAiB,EACjB,cAAuB;IAEvB,MAAM,MAAM,GAAG,MAAM,SAAS,CAC5B,CAAC,cAAc,EAAE,IAAI,EAAE,QAAQ,CAAC,EAChC,IAAI,EACJ,SAAS,EACT,cAAc,CACf,CAAC;IACF,MAAM,KAAK,GAAG,MAAM,CAAC,KAAK,CAAC,yCAAyC,CAAC,CAAC;IACtE,IAAI,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,CAAC;QAC1C,MAAM,IAAI,WAAW,CAAC,mBAAmB,EAAE,oCAAoC,CAAC,CAAC;IACnF,CAAC;IACD,MAAM,QAAQ,GAAG,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,GAAG,IAAI,GAAG,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,GAAG,EAAE,GAAG,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC;IACpF,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,QAAQ,CAAC,IAAI,QAAQ,IAAI,CAAC,EAAE,CAAC;QAChD,MAAM,IAAI,WAAW,CAAC,mBAAmB,EAAE,2BAA2B,CAAC,CAAC;IAC1E,CAAC;IACD,OAAO,QAAQ,CAAC;AAClB,CAAC;AAED,KAAK,UAAU,YAAY,CACzB,MAAc,EACd,OAAe,EACf,MAAoB;IAEpB,IAAI,eAAe,CAAC,IAAI,CAAC,MAAM,CAAC,EAAE,CAAC;QACjC,MAAM,WAAW,GAAG,IAAI,CAAC,IAAI,CAAC,OAAO,EAAE,aAAa,CAAC,CAAC;QACtD,MAAM,kBAAkB,CAAC;YACvB,GAAG,EAAE,MAAM;YACX,WAAW;YACX,QAAQ,EAAE,MAAM,CAAC,aAAa;YAC9B,SAAS,EAAE,MAAM,CAAC,SAAS;YAC3B,gBAAgB,EAAE,MAAM,CAAC,gBAAgB;SAC1C,CAAC,CAAC;QACH,OAAO,WAAW,CAAC;IACrB,CAAC;IACD,IAAI,sBAAsB,CAAC,IAAI,CAAC,MAAM,CAAC,EAAE,CAAC;QACxC,MAAM,IAAI,WAAW,CAAC,gBAAgB,EAAE,kDAAkD,CAAC,CAAC;IAC9F,CAAC;IACD,MAAM,QAAQ,GAAG,eAAe,CAAC,MAAM,CAAC,CAAC;IACzC,IAAI,QAAQ,CAAC;IACb,IAAI,CAAC;QACH,QAAQ,GAAG,MAAM,IAAI,CAAC,QAAQ,CAAC,CAAC;IAClC,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,MAAM,IAAI,WAAW,CAAC,gBAAgB,EAAE,yBAAyB,QAAQ,EAAE,EAAE;YAC3E,KAAK,EAAE,KAAK;SACb,CAAC,CAAC;IACL,CAAC;IACD,IAAI,CAAC,QAAQ,CAAC,MAAM,EAAE,EAAE,CAAC;QACvB,MAAM,IAAI,WAAW,CAAC,gBAAgB,EAAE,eAAe,QAAQ,EAAE,CAAC,CAAC;IACrE,CAAC;IACD,IAAI,QAAQ,CAAC,IAAI,GAAG,MAAM,CAAC,aAAa,EAAE,CAAC;QACzC,MAAM,IAAI,WAAW,CACnB,kBAAkB,EAClB,YAAY,CAAC,QAAQ,CAAC,IAAI,GAAG,IAAI,GAAG,IAAI,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,iBAAiB,CAAC,MAAM,CAAC,aAAa,GAAG,IAAI,GAAG,IAAI,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,KAAK,CAC1H,CAAC;IACJ,CAAC;IACD,OAAO,QAAQ,CAAC;AAClB,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,kBAAkB,CAAC,OAKxC;IACC,IAAI,CAAC,MAAM,CAAC,SAAS,CAAC,OAAO,CAAC,SAAS,CAAC,IAAI,OAAO,CAAC,SAAS,GAAG,CAAC,IAAI,OAAO,CAAC,SAAS,GAAG,EAAE,EAAE,CAAC;QAC5F,MAAM,IAAI,WAAW,CAAC,kBAAkB,EAAE,2CAA2C,CAAC,CAAC;IACzF,CAAC;IACD,MAAM,MAAM,GAAG,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,QAAQ,IAAI,MAAM,EAAE,EAAE,eAAe,CAAC,CAAC;IACxE,MAAM,OAAO,GAAG,MAAM,OAAO,CAAC,MAAM,CAAC,CAAC;IACtC,IAAI,CAAC;QACH,MAAM,SAAS,GAAG,MAAM,YAAY,CAAC,OAAO,CAAC,MAAM,CAAC,IAAI,EAAE,EAAE,OAAO,EAAE,OAAO,CAAC,MAAM,CAAC,CAAC;QACrF,MAAM,QAAQ,GAAG,MAAM,aAAa,CAClC,SAAS,EACT,OAAO,CAAC,MAAM,CAAC,SAAS,EACxB,OAAO,CAAC,MAAM,CAAC,UAAU,CAC1B,CAAC;QACF,MAAM,UAAU,GAAG,KAAK,CAAC,IAAI,CAC3B,EAAE,MAAM,EAAE,OAAO,CAAC,SAAS,EAAE,EAC7B,CAAC,CAAC,EAAE,KAAK,EAAE,EAAE,CAAC,CAAC,CAAC,KAAK,GAAG,GAAG,CAAC,GAAG,QAAQ,CAAC,GAAG,OAAO,CAAC,SAAS,CAC7D,CAAC;QACF,KAAK,MAAM,CAAC,KAAK,EAAE,SAAS,CAAC,IAAI,UAAU,CAAC,OAAO,EAAE,EAAE,CAAC;YACtD,MAAM,UAAU,GAAG,IAAI,CAAC,IAAI,CAAC,OAAO,EAAE,SAAS,MAAM,CAAC,KAAK,GAAG,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,MAAM,CAAC,CAAC;YACzF,MAAM,SAAS,CACb;gBACE,cAAc;gBACd,WAAW;gBACX,OAAO;gBACP,KAAK;gBACL,SAAS,CAAC,OAAO,CAAC,CAAC,CAAC;gBACpB,IAAI;gBACJ,SAAS;gBACT,WAAW;gBACX,GAAG;gBACH,KAAK;gBACL,sDAAsD;gBACtD,MAAM;gBACN,GAAG;gBACH,IAAI;gBACJ,UAAU;aACX,EACD,KAAK,EACL,OAAO,CAAC,MAAM,CAAC,SAAS,EACxB,OAAO,CAAC,MAAM,CAAC,UAAU,CAC1B,CAAC;QACJ,CAAC;QACD,MAAM,UAAU,GAAG,CAAC,MAAM,OAAO,CAAC,OAAO,CAAC,CAAC;aACxC,MAAM,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,kBAAkB,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;aAC/C,IAAI,EAAE,CAAC;QACV,IAAI,UAAU,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;YAC5B,MAAM,IAAI,WAAW,CAAC,cAAc,EAAE,yCAAyC,CAAC,CAAC;QACnF,CAAC;QACD,OAAO,OAAO,CAAC,GAAG,CAChB,UAAU,CAAC,GAAG,CAAC,KAAK,EAAE,IAAI,EAAE,KAAK,EAAE,EAAE;YACnC,MAAM,SAAS,GAAG,UAAU,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;YACzC,MAAM,MAAM,GAAG,MAAM,QAAQ,CAAC,IAAI,CAAC,IAAI,CAAC,OAAO,EAAE,IAAI,CAAC,CAAC,CAAC;YACxD,OAAO;gBACL,OAAO,EAAE,0BAA0B,MAAM,CAAC,QAAQ,CAAC,QAAQ,CAAC,EAAE;gBAC9D,KAAK,EAAE,YAAY,SAAS,CAAC,OAAO,CAAC,CAAC,CAAC,UAAU;gBACjD,gBAAgB,EAAE,SAAS;aAC5B,CAAC;QACJ,CAAC,CAAC,CACH,CAAC;IACJ,CAAC;YAAS,CAAC;QACT,MAAM,EAAE,CAAC,OAAO,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,KAAK,EAAE,IAAI,EAAE,CAAC,CAAC;IACtD,CAAC;AACH,CAAC"}
|
package/package.json
CHANGED
|
@@ -1,64 +1,48 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: agent-vision
|
|
3
|
-
description:
|
|
3
|
+
description: Fallback vision for agents that CANNOT see images themselves. If you can read images natively (multimodal model, image-capable Read tool), do NOT use this skill — look at the image directly instead. Only use for image questions, OCR, UI inspection, or visual debugging when native vision is unavailable or failed, and for ALL video analysis (screen recordings, video summaries, timeline analysis), since no agent can watch video natively.
|
|
4
4
|
license: MIT
|
|
5
|
-
compatibility:
|
|
5
|
+
compatibility: Node.js 20+ for the CLI; the MCP server is optional.
|
|
6
6
|
---
|
|
7
7
|
|
|
8
8
|
# agent-vision
|
|
9
9
|
|
|
10
|
-
|
|
11
|
-
server is the execution layer; this skill explains when and how to call it.
|
|
10
|
+
## Look first — this gate decides everything
|
|
12
11
|
|
|
13
|
-
|
|
12
|
+
If you can read images natively (for example the Read tool in Claude Code or
|
|
13
|
+
built-in multimodal input), look at the image yourself and answer directly.
|
|
14
|
+
Do not load or run anything from this skill for an image you can already see.
|
|
15
|
+
This skill exists only for agents without native vision, or as a fallback when
|
|
16
|
+
native image reading failed.
|
|
14
17
|
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
| Screenshot, photo, diagram, UI, scanned text | `analyze_image` |
|
|
18
|
-
| Screen recording, demo, clip, animation timeline | `analyze_video` |
|
|
18
|
+
Videos are the exception: no agent can watch a video natively. Video analysis
|
|
19
|
+
always goes through the CLI below.
|
|
19
20
|
|
|
20
|
-
|
|
21
|
+
## Run the CLI
|
|
22
|
+
|
|
23
|
+
When you cannot see the input yourself (or it is a video), run the analysis
|
|
24
|
+
directly and read the plain-text output:
|
|
25
|
+
|
|
26
|
+
```bash
|
|
27
|
+
npx -y @yanickxia/agent-vision image "/absolute/path/to/error.png" --prompt "Read the error message and identify the likely cause."
|
|
28
|
+
npx -y @yanickxia/agent-vision video "/absolute/path/to/demo.mp4" --prompt "Summarize the UI flow in chronological order." --frames 8
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
If the package is installed locally, run `agent-vision image ...` instead of
|
|
32
|
+
`npx -y @yanickxia/agent-vision ...`.
|
|
21
33
|
|
|
22
34
|
## Calling rules
|
|
23
35
|
|
|
24
|
-
1. Pass the user's original question as
|
|
25
|
-
than a generic request for a description.
|
|
36
|
+
1. Pass the user's original question as `--prompt`. A focused question is
|
|
37
|
+
better than a generic request for a description.
|
|
26
38
|
2. Prefer an absolute local path. HTTP(S) URLs are also supported. Images may
|
|
27
39
|
additionally be supplied as a `data:image/...;base64,...` URI.
|
|
28
|
-
3. For videos, keep the default
|
|
40
|
+
3. For videos, keep the default `--frames 8` unless the user needs a faster
|
|
29
41
|
result or a denser timeline. The allowed range is 1–16.
|
|
30
42
|
4. If the result is uncertain, say so. Never invent visual details after a
|
|
31
|
-
failed
|
|
43
|
+
failed run.
|
|
32
44
|
5. Retry at most once for a transient API or timeout error. Configuration,
|
|
33
45
|
unsupported-format, and missing-file errors require correction, not retries.
|
|
34
46
|
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
```json
|
|
38
|
-
analyze_image({
|
|
39
|
-
"source": "/absolute/path/to/error.png",
|
|
40
|
-
"prompt": "Read the error message and identify the likely cause."
|
|
41
|
-
})
|
|
42
|
-
```
|
|
43
|
-
|
|
44
|
-
```json
|
|
45
|
-
analyze_video({
|
|
46
|
-
"source": "/absolute/path/to/demo.mp4",
|
|
47
|
-
"prompt": "Summarize the UI flow in chronological order.",
|
|
48
|
-
"max_frames": 8
|
|
49
|
-
})
|
|
50
|
-
```
|
|
51
|
-
|
|
52
|
-
## npx fallback
|
|
53
|
-
|
|
54
|
-
If the MCP tools are not available but shell execution is allowed, run:
|
|
55
|
-
|
|
56
|
-
```bash
|
|
57
|
-
npx -y @yanickxia/agent-vision image "/absolute/path/to/image.png" --prompt "user question"
|
|
58
|
-
npx -y @yanickxia/agent-vision video "/absolute/path/to/video.mp4" --prompt "user question" --frames 8
|
|
59
|
-
```
|
|
60
|
-
|
|
61
|
-
Before the npm package is available, replace the package name with
|
|
62
|
-
`github:yanickxia/agent-vision`.
|
|
63
|
-
|
|
64
|
-
Configuration details are in `references/configuration.md`.
|
|
47
|
+
Configuration — multiple providers and models raced concurrently — is
|
|
48
|
+
described in `references/configuration.md`.
|
|
@@ -1,6 +1,55 @@
|
|
|
1
1
|
# Configuration
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
agent-vision reads its configuration from a JSON settings file. Environment
|
|
4
|
+
variables still work as a fallback.
|
|
5
|
+
|
|
6
|
+
## Settings file
|
|
7
|
+
|
|
8
|
+
The first location that exists wins:
|
|
9
|
+
|
|
10
|
+
1. `$AGENT_VISION_CONFIG` — an explicit file path (an error if it is missing)
|
|
11
|
+
2. `$XDG_CONFIG_HOME/agent-vision/settings.json`
|
|
12
|
+
(default: `~/.config/agent-vision/settings.json`)
|
|
13
|
+
|
|
14
|
+
```json
|
|
15
|
+
{
|
|
16
|
+
"providers": [
|
|
17
|
+
{
|
|
18
|
+
"name": "dashscope",
|
|
19
|
+
"baseUrl": "https://dashscope.aliyuncs.com/compatible-mode/v1",
|
|
20
|
+
"apiKey": "sk-...",
|
|
21
|
+
"models": ["qwen-vl-max", "qwen-vl-plus"]
|
|
22
|
+
},
|
|
23
|
+
{
|
|
24
|
+
"name": "openai",
|
|
25
|
+
"baseUrl": "https://api.openai.com/v1",
|
|
26
|
+
"apiKey": "sk-...",
|
|
27
|
+
"models": ["gpt-4o"]
|
|
28
|
+
}
|
|
29
|
+
],
|
|
30
|
+
"maxTokens": 4096,
|
|
31
|
+
"timeoutMs": 120000
|
|
32
|
+
}
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
- Every provider × model combination is a target. With several targets, all
|
|
36
|
+
of them are requested **concurrently** and the first successful answer
|
|
37
|
+
wins; the remaining requests are aborted. Note the cost: N targets means
|
|
38
|
+
up to N paid completions per analysis.
|
|
39
|
+
- `name` is optional and only used in error messages; it defaults to the
|
|
40
|
+
baseUrl host.
|
|
41
|
+
- `apiKey` is optional per provider (for local endpoints without auth) and
|
|
42
|
+
falls back to the environment key.
|
|
43
|
+
- `headers` per provider merges over the top-level `headers`.
|
|
44
|
+
- Top-level knobs (all optional): `maxTokens`, `timeoutMs`, `headers`,
|
|
45
|
+
`maxImageMb`, `maxVideoMb`, `allowPrivateUrls`, `ffmpegPath`.
|
|
46
|
+
- Fields defined in the file take precedence over environment variables.
|
|
47
|
+
- The file may contain API keys: keep it out of version control and run
|
|
48
|
+
`chmod 600` on it.
|
|
49
|
+
|
|
50
|
+
## Environment variables (fallback)
|
|
51
|
+
|
|
52
|
+
Required when no settings file is used:
|
|
4
53
|
|
|
5
54
|
```bash
|
|
6
55
|
export AGENT_VISION_BASE_URL="https://your-openai-compatible-endpoint/v1"
|
|
@@ -12,7 +61,7 @@ export AGENT_VISION_MODEL="your-vision-model"
|
|
|
12
61
|
not require authentication. The server also recognizes `VISION_*` and
|
|
13
62
|
`OPENAI_*` aliases.
|
|
14
63
|
|
|
15
|
-
|
|
64
|
+
Optional variables:
|
|
16
65
|
|
|
17
66
|
- `AGENT_VISION_MAX_TOKENS` (default `4096`)
|
|
18
67
|
- `AGENT_VISION_TIMEOUT_MS` (default `120000`)
|