@liustack/modlens 2.7.3 → 2.7.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +23 -27
- package/README.zh-CN.md +23 -27
- package/dist/main.js +110 -30
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -11,13 +11,13 @@
|
|
|
11
11
|
|
|
12
12
|
DeepSeek-V4-Flash gives you a lot of model for very little money: fast, strong, and its one real flaw is no multimodal. And it's not just DeepSeek. Every text-only model running inside Codex, Claude Code, Pi Agent, or OpenClaw hits the same wall.
|
|
13
13
|
|
|
14
|
-
ModLens fixes this the lightest way possible
|
|
14
|
+
ModLens fixes this the lightest way possible: it never touches your config, never adds a local proxy, and is just a vision plug-in you can run as a CLI or install as an Agent Skill. What it hands back is not a one-line caption but structured visual evidence: text, layout, regions, entities, relations, visual clues. Five vision engines to pick from. The default one needs no key at all, and the fastest one runs on a free Gemini key whose image understanding embarrasses most flagships, Fable 5 included. How it works:
|
|
15
15
|
|
|
16
16
|
```text
|
|
17
17
|
text-only model in your agent harness ──▶ modlens skill (auto-triggers on images)
|
|
18
18
|
│
|
|
19
19
|
▼
|
|
20
|
-
|
|
20
|
+
vision engine · Gemini 3.6 Flash (free)
|
|
21
21
|
│
|
|
22
22
|
▼
|
|
23
23
|
structured JSON evidence ──▶ model answers with sight
|
|
@@ -25,49 +25,45 @@ text-only model in your agent harness ──▶ modlens skill (auto-triggers on
|
|
|
25
25
|
|
|
26
26
|
## You can just paste the image
|
|
27
27
|
|
|
28
|
-
|
|
28
|
+
Every other bridge makes you save the image to a file first, then mention the path in the chat. ModLens lets you paste it.
|
|
29
29
|
|
|
30
|
-
|
|
30
|
+
That is not laziness on their part. Pasting is handled end to end by the client: the moment an image lands in the chat box it is encoded and sent, and a vision MCP server never gets a chance to step in, which is why their docs can only tell you to save the file and report the path. ModLens takes the other route. Before those bytes are ever sent, the harness has already written them to local session storage, so the skill goes there, pulls them back out into a real file, and feeds that to the vision engine. You do nothing, and the model answers with the full image instead of asking you for a path.
|
|
31
31
|
|
|
32
|
-
Verified on real machines across
|
|
33
|
-
|
|
34
|
-
As far as we know, no other tool catches this paste yet. The usual answer is save the file, then report the path. ModLens's answer is just paste it.
|
|
32
|
+
Verified on real machines across four harnesses: Claude Code pinpoints the exact session from its injected session id, Pi stores sessions the same way, OpenCode swaps in SQLite, and Codex's pasted images already carry a temp path so the path-tag route handles them. Before touching anything, `recover-paste` works out which harness it is running inside, by walking the process ancestry and checking environment fingerprints, and reads only that harness's storage, so another tool's stale sessions can't impersonate it.
|
|
35
33
|
|
|
36
34
|
## Quick start
|
|
37
35
|
|
|
38
|
-
**1.
|
|
36
|
+
**1. Install the skill.** Just tell your agent (Claude Code, Codex, OpenClaw, Cursor, ...):
|
|
37
|
+
|
|
38
|
+
```text
|
|
39
|
+
Install the skill from https://github.com/liustack/modlens
|
|
40
|
+
```
|
|
39
41
|
|
|
40
|
-
|
|
42
|
+
or do it yourself:
|
|
41
43
|
|
|
42
44
|
```bash
|
|
43
|
-
|
|
44
|
-
modlens config set provider gemini-api
|
|
45
|
+
npx -y skills add liustack/modlens
|
|
45
46
|
```
|
|
46
47
|
|
|
47
|
-
|
|
48
|
+
Harnesses look for skills in different places: Claude Code reads `~/.claude/skills/`, Codex reads `~/.codex/skills/`, Pi and OpenCode read `~/.agents/skills/`. Symlinks work in all of them, so linking the skill folder once keeps every agent on the latest version.
|
|
48
49
|
|
|
49
|
-
**
|
|
50
|
+
**2. Wire up a vision engine.** Recommended: a free Gemini key from [aistudio.google.com](https://aistudio.google.com). Three minutes, no credit card, 5-10 seconds per image:
|
|
50
51
|
|
|
51
52
|
```bash
|
|
52
|
-
|
|
53
|
-
|
|
53
|
+
modlens config set gemini-api.apiKey <key>
|
|
54
|
+
modlens config set provider gemini-api
|
|
54
55
|
```
|
|
55
56
|
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
```text
|
|
59
|
-
Install the skill from https://github.com/liustack/modlens
|
|
60
|
-
```
|
|
57
|
+
Don't feel like typing those two lines? Tell your agent "set my Gemini key in modlens" and it runs them for you.
|
|
61
58
|
|
|
62
|
-
|
|
59
|
+
Skipping the sign-up is fine too: Antigravity CLI works with no key at all, it is just slower (15-40s) and its free quota is tight.
|
|
63
60
|
|
|
64
61
|
```bash
|
|
65
|
-
|
|
62
|
+
curl -fsSL https://antigravity.google/cli/install.sh | bash
|
|
63
|
+
agy # opens browser sign-in, then exit
|
|
66
64
|
```
|
|
67
65
|
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
**3. Use it.** Paste an image (or its path) and ask anything. The skill fires on its own.
|
|
66
|
+
**3. Use it.** Paste an image, or throw a path at it, and ask anything. The skill fires on its own.
|
|
71
67
|
|
|
72
68
|
## See it work
|
|
73
69
|
|
|
@@ -157,7 +153,7 @@ ModLens ships five vision providers. `antigravity-cli` stays the default: zero k
|
|
|
157
153
|
| `anthropic` | `ANTHROPIC_API_KEY` | a few seconds | Claude Haiku by default, schema via forced tool call |
|
|
158
154
|
| `claude-cli` | Claude Code signed in | 20-45s | no key, rides your Claude subscription, Read-only permissions |
|
|
159
155
|
|
|
160
|
-
`antigravity-cli`
|
|
156
|
+
`antigravity-cli` wins on needing no key and loses on both other fronts: it's slower (a full agent loop takes 15-40 seconds against 5-10 for `gemini-api` direct) and its quota is tight. The free tier is now a one-time weekly grant, pooled across the desktop app, the CLI, and the SDK, and parallel subagents drain it faster. Once it's gone you wait out the cycle: we hit that wall ourselves and the message read "94 hours until reset." Great for a first look, but `gemini-api` is what holds up day to day.
|
|
161
157
|
|
|
162
158
|
Config lives in `~/.modlens/config.json`. Environment variables override the file (`GEMINI_API_KEY`, `OPENAI_API_KEY`, `OPENAI_BASE_URL`, `ANTHROPIC_API_KEY`), and CLI flags override everything.
|
|
163
159
|
|
|
@@ -170,7 +166,7 @@ modlens config set provider gemini-api # switch the default provider
|
|
|
170
166
|
|
|
171
167
|
The free Gemini key takes three minutes at [aistudio.google.com](https://aistudio.google.com), no credit card.
|
|
172
168
|
|
|
173
|
-
|
|
169
|
+
You don't actually have to remember any of these commands. The skill ships a per-provider setup guide, so once it's installed you can just ask your agent: "how do I configure modlens," "set my Gemini key in modlens," "switch modlens to claude-cli." It reads the guide and runs them.
|
|
174
170
|
|
|
175
171
|
## Using it in Codex (DeepSeek and friends)
|
|
176
172
|
|
package/README.zh-CN.md
CHANGED
|
@@ -11,13 +11,13 @@
|
|
|
11
11
|
|
|
12
12
|
DeepSeek-V4-Flash 碗大又好吃,速度快,性能强,要说唯一的缺点就是没有多模态。不仅 DeepSeek-V4-Flash,只要是纯文本语言模型,跑在 Codex、Claude Code、Pi Agent、OpenClaw 中,都有这个问题。
|
|
13
13
|
|
|
14
|
-
ModLens
|
|
14
|
+
ModLens 用最轻的方式解决它:不动你的配置,不装本地代理,就是一个视觉外挂,CLI 和 skill 两种用法。它产出的不是一句话描述,是结构化的视觉证据:文字、版面、区块、实体、关系、视觉线索。视觉引擎有五个可选,默认那个零 key 就能跑,最快的那个用免费 Gemini key,识图能力连 Fable 5 都吊打。原理如下:
|
|
15
15
|
|
|
16
16
|
```text
|
|
17
17
|
Agent Harness 中的纯文本模型 ──▶ modlens skill(遇到图片自动触发)
|
|
18
18
|
│
|
|
19
19
|
▼
|
|
20
|
-
|
|
20
|
+
视觉引擎 · Gemini 3.6 Flash(免费)
|
|
21
21
|
│
|
|
22
22
|
▼
|
|
23
23
|
结构化 JSON 证据 ──▶ 模型带着视力回答
|
|
@@ -25,49 +25,45 @@ Agent Harness 中的纯文本模型 ──▶ modlens skill(遇到图片自动
|
|
|
25
25
|
|
|
26
26
|
## 你可以直接粘贴图片
|
|
27
27
|
|
|
28
|
-
|
|
28
|
+
别的方案让你先把图存成文件,再在对话里报一句路径。ModLens 让你直接粘贴。
|
|
29
29
|
|
|
30
|
-
ModLens
|
|
30
|
+
这不怪它们偷懒。粘贴这个动作从头到尾是客户端办的,图一进对话框就被转码发走,识图 MCP server 连插手的机会都没有,所以它们的文档只能教你存文件、报路径。ModLens 走的是另一条路:图片字节在发走之前,早被 harness 原样写进了本地会话存储,skill 直接去那里把它捞回来落成文件,再喂给视觉引擎。你什么都不用做,模型拿到的是完整图片,不是一句「麻烦告诉我路径」。
|
|
31
31
|
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
据我们所知,还没有第二个工具接住粘贴这一下。别的方案是先存文件、再报路径。ModLens 是你直接粘贴。
|
|
32
|
+
四家 harness 真机验证过:Claude Code 按注入的会话 ID 精确定位,Pi 的存储路数和它同构,OpenCode 换成了 SQLite,Codex 的粘贴图本来就带临时路径,走路径标签就行。动手之前 `recover-paste` 会先认清自己跑在哪一家(查进程祖先链,核对环境变量指纹),只读那一家的存储,别家的旧会话冒充不了。
|
|
35
33
|
|
|
36
34
|
## 快速开始
|
|
37
35
|
|
|
38
|
-
**1.
|
|
36
|
+
**1. 装 skill。** 直接告诉你的 agent(Claude Code、Codex、OpenClaw、Cursor 等):
|
|
37
|
+
|
|
38
|
+
```text
|
|
39
|
+
安装这个 skill https://github.com/liustack/modlens
|
|
40
|
+
```
|
|
39
41
|
|
|
40
|
-
|
|
42
|
+
或者自己动手:
|
|
41
43
|
|
|
42
44
|
```bash
|
|
43
|
-
|
|
44
|
-
modlens config set provider gemini-api
|
|
45
|
+
npx -y skills add liustack/modlens
|
|
45
46
|
```
|
|
46
47
|
|
|
47
|
-
|
|
48
|
+
各家 harness 找 skill 的位置不一样:Claude Code 读 `~/.claude/skills/`,Codex 读 `~/.codex/skills/`,Pi 和 OpenCode 读 `~/.agents/skills/`。软链接在哪家都好使,把 skill 目录链一次,各家永远用最新版。
|
|
48
49
|
|
|
49
|
-
|
|
50
|
+
**2. 接一个视觉引擎。** 推荐去 [aistudio.google.com](https://aistudio.google.com) 领个免费 Gemini key,三分钟,不要信用卡,出图 5-10 秒:
|
|
50
51
|
|
|
51
52
|
```bash
|
|
52
|
-
|
|
53
|
-
|
|
53
|
+
modlens config set gemini-api.apiKey <key>
|
|
54
|
+
modlens config set provider gemini-api
|
|
54
55
|
```
|
|
55
56
|
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
```text
|
|
59
|
-
安装这个 skill https://github.com/liustack/modlens
|
|
60
|
-
```
|
|
57
|
+
懒得敲这两行?跟 agent 说一句「帮我把 Gemini key 配进 modlens」,它自己会跑。
|
|
61
58
|
|
|
62
|
-
|
|
59
|
+
不想注册也行,装上 Antigravity CLI 就能零 key 开跑,代价是慢(15-40 秒),免费额度也紧:
|
|
63
60
|
|
|
64
61
|
```bash
|
|
65
|
-
|
|
62
|
+
curl -fsSL https://antigravity.google/cli/install.sh | bash
|
|
63
|
+
agy # 浏览器完成登录后退出
|
|
66
64
|
```
|
|
67
65
|
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
**3. 用起来。** 粘贴一张图(或者图片路径),随便问,skill 会自动触发。
|
|
66
|
+
**3. 用起来。** 粘贴一张图,或者甩个图片路径,随便问。skill 自己会触发。
|
|
71
67
|
|
|
72
68
|
## 看看效果
|
|
73
69
|
|
|
@@ -157,7 +153,7 @@ ModLens 内置五个视觉 provider,默认还是 `antigravity-cli`:零 key
|
|
|
157
153
|
| `anthropic` | `ANTHROPIC_API_KEY` | 几秒 | 默认 Claude Haiku,强制工具调用保 schema |
|
|
158
154
|
| `claude-cli` | Claude Code 已登录 | 20-45 秒 | 零 key,吃你的 Claude 订阅额度,只放行 Read 工具 |
|
|
159
155
|
|
|
160
|
-
`antigravity-cli`
|
|
156
|
+
`antigravity-cli` 胜在零 key,输在两头:慢(完整 agent 循环 15-40 秒,`gemini-api` 直连 5-10 秒),额度紧。它的免费档如今是一次性发放的周配额,桌面应用、CLI、SDK 共用一个池子,subagent 并行还加倍消耗,用超了得等下个周期(我们实测撞过一次,提示「94 小时后重置」)。所以它适合尝鲜,日常主力还是 `gemini-api` 稳。
|
|
161
157
|
|
|
162
158
|
配置放在 `~/.modlens/config.json`,环境变量能盖过它(`GEMINI_API_KEY`、`OPENAI_API_KEY`、`OPENAI_BASE_URL`、`ANTHROPIC_API_KEY`),CLI 参数最大。
|
|
163
159
|
|
|
@@ -170,7 +166,7 @@ modlens config set provider gemini-api # 换默认 provider
|
|
|
170
166
|
|
|
171
167
|
免费 Gemini key 去 [aistudio.google.com](https://aistudio.google.com) 领,三分钟,不要信用卡。
|
|
172
168
|
|
|
173
|
-
|
|
169
|
+
这些命令你其实一条都不用记。skill 自带一份分 provider 的配置手册,装完之后直接问你的 agent:「modlens 怎么配置」「帮我把 Gemini key 配进 modlens」「把默认 provider 切成 claude-cli」,它照着手册自己跑完。
|
|
174
170
|
|
|
175
171
|
## 在 Codex 里用(DeepSeek 等纯文本模型)
|
|
176
172
|
|
package/dist/main.js
CHANGED
|
@@ -36,8 +36,18 @@ function loadConfigFile(configPath = CONFIG_PATH) {
|
|
|
36
36
|
function defaultProviderName(config2) {
|
|
37
37
|
return config2.provider?.trim() || "antigravity-cli";
|
|
38
38
|
}
|
|
39
|
+
const PROVIDER_ALIASES = {
|
|
40
|
+
antigravity: "antigravity-cli",
|
|
41
|
+
agy: "antigravity-cli",
|
|
42
|
+
gemini: "gemini-api",
|
|
43
|
+
claude: "claude-cli"
|
|
44
|
+
};
|
|
39
45
|
function resolveProviderSettings(providerName, config2, env = process.env) {
|
|
40
|
-
const
|
|
46
|
+
const aliasNames = Object.entries(PROVIDER_ALIASES).filter(([, canonical]) => canonical === providerName).map(([alias]) => alias);
|
|
47
|
+
const fromFile = {
|
|
48
|
+
...Object.assign({}, ...aliasNames.map((alias) => config2.providers?.[alias] ?? {})),
|
|
49
|
+
...config2.providers?.[providerName] ?? {}
|
|
50
|
+
};
|
|
41
51
|
const bindings = ENV_BINDINGS[providerName] ?? {};
|
|
42
52
|
const settings = { ...fromFile };
|
|
43
53
|
for (const [field, envName] of Object.entries(bindings)) {
|
|
@@ -318,6 +328,7 @@ const SWITCH_HINT = `Or switch to a provider with its own quota and no interacti
|
|
|
318
328
|
modlens config set gemini-api.apiKey <key> # free key, no card: https://aistudio.google.com
|
|
319
329
|
modlens config set provider gemini-api`;
|
|
320
330
|
function describeAntigravityFailure(context) {
|
|
331
|
+
const since = context.startedAt ?? Date.now() - LOG_FRESHNESS_MS;
|
|
321
332
|
let envelope = null;
|
|
322
333
|
try {
|
|
323
334
|
envelope = parseEnvelope$1(context.stdout);
|
|
@@ -330,7 +341,7 @@ function describeAntigravityFailure(context) {
|
|
|
330
341
|
const agyError = typeof envelope?.error === "string" ? envelope.error.trim() : "";
|
|
331
342
|
const evidence = `${agyError}
|
|
332
343
|
${context.stderr}
|
|
333
|
-
${readRecentAgyLog()}`.toLowerCase();
|
|
344
|
+
${readRecentAgyLog(since)}`.toLowerCase();
|
|
334
345
|
if (evidence.includes("quota")) {
|
|
335
346
|
return [
|
|
336
347
|
agyError || "Antigravity CLI reported a quota error.",
|
|
@@ -359,14 +370,14 @@ function agyLogDir() {
|
|
|
359
370
|
return path.join(os.homedir(), ".gemini", "antigravity-cli", "log");
|
|
360
371
|
}
|
|
361
372
|
const LOG_FRESHNESS_MS = 2 * 60 * 1e3;
|
|
362
|
-
function readRecentAgyLog() {
|
|
373
|
+
function readRecentAgyLog(since) {
|
|
363
374
|
try {
|
|
364
375
|
const dir = agyLogDir();
|
|
365
376
|
const newest = fs.readdirSync(dir).filter((name) => name.endsWith(".log")).map((name) => {
|
|
366
377
|
const full = path.join(dir, name);
|
|
367
378
|
return { full, mtime: fs.statSync(full).mtimeMs };
|
|
368
379
|
}).sort((a, b) => b.mtime - a.mtime)[0];
|
|
369
|
-
if (!newest ||
|
|
380
|
+
if (!newest || newest.mtime < since) {
|
|
370
381
|
return "";
|
|
371
382
|
}
|
|
372
383
|
return fs.readFileSync(newest.full, "utf-8").slice(-8e3);
|
|
@@ -379,7 +390,8 @@ const antigravityCliProvider = {
|
|
|
379
390
|
defaultModel: DEFAULT_MODEL,
|
|
380
391
|
buildInvocation: buildAntigravityInvocation,
|
|
381
392
|
parseOutput: parseAntigravityOutput,
|
|
382
|
-
describeFailure: describeAntigravityFailure
|
|
393
|
+
describeFailure: describeAntigravityFailure,
|
|
394
|
+
hasInternalTimeout: true
|
|
383
395
|
};
|
|
384
396
|
const MIME_BY_EXT = {
|
|
385
397
|
".jpg": "image/jpeg",
|
|
@@ -732,9 +744,12 @@ Respond with ONE JSON object only, no markdown fences, no commentary. Fill this
|
|
|
732
744
|
throw new Error(`OpenAI-compatible API returned non-JSON output: ${truncate(text)}`);
|
|
733
745
|
}
|
|
734
746
|
const shaped = result;
|
|
735
|
-
|
|
747
|
+
const missing = ["summary", "ocr", "layout", "semantics", "visual", "uncertainty"].filter(
|
|
748
|
+
(field) => shaped[field] === void 0 || shaped[field] === null
|
|
749
|
+
);
|
|
750
|
+
if (missing.length > 0 || typeof shaped.summary !== "string" || typeof shaped.ocr !== "object" || !Array.isArray(shaped.uncertainty)) {
|
|
736
751
|
throw new Error(
|
|
737
|
-
`OpenAI-compatible API returned JSON that does not match the vision schema (missing
|
|
752
|
+
`OpenAI-compatible API returned JSON that does not match the vision schema${missing.length > 0 ? ` (missing: ${missing.join(", ")})` : ""}. Retry, or switch to gemini-api / anthropic for enforced schemas. Got: ${truncate(text)}`
|
|
738
753
|
);
|
|
739
754
|
}
|
|
740
755
|
return {
|
|
@@ -811,10 +826,11 @@ async function analyzeImage(options) {
|
|
|
811
826
|
parsed = await provider.execute(providerOptions);
|
|
812
827
|
} else if (provider.buildInvocation && provider.parseOutput) {
|
|
813
828
|
const invocation = provider.buildInvocation(providerOptions);
|
|
829
|
+
const backstop = provider.hasInternalTimeout ? timeoutMs + KILL_GRACE_MS : timeoutMs;
|
|
814
830
|
const commandResult = await runCommand(
|
|
815
831
|
provider.name,
|
|
816
832
|
invocation,
|
|
817
|
-
|
|
833
|
+
backstop,
|
|
818
834
|
provider.describeFailure
|
|
819
835
|
);
|
|
820
836
|
parsed = provider.parseOutput(commandResult.stdout);
|
|
@@ -861,11 +877,14 @@ function validateInputFile(filePath) {
|
|
|
861
877
|
}
|
|
862
878
|
}
|
|
863
879
|
function runCommand(providerName, invocation, timeoutMs, describeFailure) {
|
|
880
|
+
const runStartedAt = Date.now();
|
|
864
881
|
return new Promise((resolve, reject) => {
|
|
865
882
|
const child = spawn(invocation.command, invocation.args, {
|
|
866
883
|
cwd: invocation.cwd,
|
|
867
884
|
stdio: ["ignore", "pipe", "pipe"]
|
|
868
885
|
});
|
|
886
|
+
const outDecoder = new TextDecoder("utf-8");
|
|
887
|
+
const errDecoder = new TextDecoder("utf-8");
|
|
869
888
|
let stdout = "";
|
|
870
889
|
let stderr = "";
|
|
871
890
|
let timedOut = false;
|
|
@@ -890,7 +909,7 @@ function runCommand(providerName, invocation, timeoutMs, describeFailure) {
|
|
|
890
909
|
return;
|
|
891
910
|
}
|
|
892
911
|
if (code !== 0) {
|
|
893
|
-
const explained = describeFailure?.({ stdout, stderr, code }) ?? null;
|
|
912
|
+
const explained = describeFailure?.({ stdout, stderr, code, startedAt: runStartedAt }) ?? null;
|
|
894
913
|
reject(
|
|
895
914
|
new Error(
|
|
896
915
|
explained ?? `${providerName} provider failed with code ${code}.${stderr ? ` stderr: ${stderr.trim()}` : ""}`
|
|
@@ -910,11 +929,11 @@ function runCommand(providerName, invocation, timeoutMs, describeFailure) {
|
|
|
910
929
|
drainTimer = setTimeout(() => settle(exitCode), DRAIN_GRACE_MS);
|
|
911
930
|
};
|
|
912
931
|
child.stdout.on("data", (chunk) => {
|
|
913
|
-
stdout +=
|
|
932
|
+
stdout += outDecoder.decode(chunk, { stream: true });
|
|
914
933
|
restartDrain();
|
|
915
934
|
});
|
|
916
935
|
child.stderr.on("data", (chunk) => {
|
|
917
|
-
stderr +=
|
|
936
|
+
stderr += errDecoder.decode(chunk, { stream: true });
|
|
918
937
|
restartDrain();
|
|
919
938
|
});
|
|
920
939
|
child.on("error", (error) => {
|
|
@@ -925,9 +944,10 @@ function runCommand(providerName, invocation, timeoutMs, describeFailure) {
|
|
|
925
944
|
clearTimeout(timer);
|
|
926
945
|
clearTimeout(drainTimer);
|
|
927
946
|
if (error.code === "ENOENT") {
|
|
947
|
+
const missingCwd = !fs.existsSync(invocation.cwd);
|
|
928
948
|
reject(
|
|
929
949
|
new Error(
|
|
930
|
-
`Provider CLI not found: ${invocation.command}. Install
|
|
950
|
+
missingCwd ? `Working directory does not exist: ${invocation.cwd}` : `Provider CLI not found: ${invocation.command}. Install it and sign in first.`
|
|
931
951
|
)
|
|
932
952
|
);
|
|
933
953
|
return;
|
|
@@ -942,12 +962,39 @@ function runCommand(providerName, invocation, timeoutMs, describeFailure) {
|
|
|
942
962
|
child.on("close", (code) => settle(code));
|
|
943
963
|
});
|
|
944
964
|
}
|
|
965
|
+
function extensionFromMediaType(mediaType) {
|
|
966
|
+
const subtype = mediaType.split("/")[1]?.split("+")[0]?.replace(/[^a-z0-9]/gi, "");
|
|
967
|
+
return subtype ? subtype.toLowerCase() : "bin";
|
|
968
|
+
}
|
|
945
969
|
const EXT_BY_MIME = {
|
|
946
970
|
"image/png": "png",
|
|
947
971
|
"image/jpeg": "jpg",
|
|
948
972
|
"image/webp": "webp",
|
|
949
973
|
"image/gif": "gif"
|
|
950
974
|
};
|
|
975
|
+
function transcriptBelongsTo(filePath, cwd) {
|
|
976
|
+
const wanted = path.resolve(cwd);
|
|
977
|
+
let raw;
|
|
978
|
+
try {
|
|
979
|
+
raw = fs.readFileSync(filePath, "utf-8");
|
|
980
|
+
} catch {
|
|
981
|
+
return false;
|
|
982
|
+
}
|
|
983
|
+
for (const line of raw.split("\n")) {
|
|
984
|
+
if (!line.includes('"cwd"')) {
|
|
985
|
+
continue;
|
|
986
|
+
}
|
|
987
|
+
try {
|
|
988
|
+
const recorded = JSON.parse(line).cwd;
|
|
989
|
+
if (typeof recorded === "string") {
|
|
990
|
+
const resolved = path.resolve(recorded);
|
|
991
|
+
return resolved === wanted || resolved.startsWith(`${wanted}${path.sep}`);
|
|
992
|
+
}
|
|
993
|
+
} catch {
|
|
994
|
+
}
|
|
995
|
+
}
|
|
996
|
+
return true;
|
|
997
|
+
}
|
|
951
998
|
function forEachJsonLine(filePath, visit) {
|
|
952
999
|
let raw;
|
|
953
1000
|
try {
|
|
@@ -1007,6 +1054,9 @@ function jsonlAdapter(options) {
|
|
|
1007
1054
|
findNewest: (cwd) => {
|
|
1008
1055
|
let best = null;
|
|
1009
1056
|
for (const file of listJsonl(dirFor(cwd))) {
|
|
1057
|
+
if (!transcriptBelongsTo(file, cwd)) {
|
|
1058
|
+
continue;
|
|
1059
|
+
}
|
|
1010
1060
|
const timestamp = newestJsonlTimestamp(file, extractLine);
|
|
1011
1061
|
if (timestamp !== null && (!best || timestamp > best.timestamp)) {
|
|
1012
1062
|
best = { ref: jsonlSource(name, file, extractLine), timestamp };
|
|
@@ -1016,7 +1066,7 @@ function jsonlAdapter(options) {
|
|
|
1016
1066
|
},
|
|
1017
1067
|
findSession: (cwd, sessionId) => {
|
|
1018
1068
|
for (const file of listJsonl(dirFor(cwd))) {
|
|
1019
|
-
if (matchesSession(path.basename(file), sessionId)) {
|
|
1069
|
+
if (matchesSession(path.basename(file), sessionId) && transcriptBelongsTo(file, cwd)) {
|
|
1020
1070
|
return jsonlSource(name, file, extractLine);
|
|
1021
1071
|
}
|
|
1022
1072
|
}
|
|
@@ -1075,6 +1125,9 @@ const piAdapter = jsonlAdapter({
|
|
|
1075
1125
|
function opencodeDbPath() {
|
|
1076
1126
|
return path.join(os.homedir(), ".local", "share", "opencode", "opencode.db");
|
|
1077
1127
|
}
|
|
1128
|
+
function escapeLikePattern(value) {
|
|
1129
|
+
return value.replace(/[\\%_]/g, (char) => `\\${char}`);
|
|
1130
|
+
}
|
|
1078
1131
|
function opencodeQuery(dbPath, cwd, sessionId) {
|
|
1079
1132
|
let DatabaseSync;
|
|
1080
1133
|
try {
|
|
@@ -1088,8 +1141,10 @@ function opencodeQuery(dbPath, cwd, sessionId) {
|
|
|
1088
1141
|
const db = new DatabaseSync(dbPath, { readOnly: true });
|
|
1089
1142
|
try {
|
|
1090
1143
|
const resolved = path.resolve(cwd);
|
|
1091
|
-
const
|
|
1092
|
-
const
|
|
1144
|
+
const directoryFilter = `(session.directory = ? OR session.directory LIKE ? || '/%' ESCAPE '\\' OR ? LIKE session.directory || '/%' ESCAPE '\\')`;
|
|
1145
|
+
const escaped = escapeLikePattern(resolved);
|
|
1146
|
+
const sessionFilter = sessionId ? `AND ${directoryFilter} AND (session.id = ? OR session.slug = ?)` : `AND ${directoryFilter}`;
|
|
1147
|
+
const params = sessionId ? [resolved, escaped, resolved, sessionId, sessionId] : [resolved, escaped, resolved];
|
|
1093
1148
|
const rows = db.prepare(
|
|
1094
1149
|
`SELECT part.data AS data, part.time_created AS time_created, part.session_id AS session_id
|
|
1095
1150
|
FROM part
|
|
@@ -1182,8 +1237,16 @@ function harnessFromPsTable(psOutput, startPid) {
|
|
|
1182
1237
|
if (!proc) {
|
|
1183
1238
|
return null;
|
|
1184
1239
|
}
|
|
1185
|
-
|
|
1186
|
-
|
|
1240
|
+
const tokens = proc.command.trim().split(/\s+/);
|
|
1241
|
+
const candidates = [tokens[0]];
|
|
1242
|
+
if (/^(node|bun|deno)$/.test(path.basename(tokens[0] ?? ""))) {
|
|
1243
|
+
const script = tokens.slice(1).find((token) => !token.startsWith("-"));
|
|
1244
|
+
if (script) {
|
|
1245
|
+
candidates.push(script);
|
|
1246
|
+
}
|
|
1247
|
+
}
|
|
1248
|
+
for (const token of candidates) {
|
|
1249
|
+
const mapped = token ? HARNESS_BY_BASENAME[path.basename(token)] : void 0;
|
|
1187
1250
|
if (mapped) {
|
|
1188
1251
|
return mapped;
|
|
1189
1252
|
}
|
|
@@ -1220,26 +1283,29 @@ function detectHarness() {
|
|
|
1220
1283
|
return null;
|
|
1221
1284
|
}
|
|
1222
1285
|
const ADAPTERS = [claudeAdapter, piAdapter, opencodeAdapter];
|
|
1223
|
-
function sourceForExplicitPath(filePath) {
|
|
1224
|
-
|
|
1286
|
+
function sourceForExplicitPath(filePath, cwd, harness) {
|
|
1287
|
+
const declared = harness && harness !== "none" ? harness : void 0;
|
|
1288
|
+
if (declared === "opencode" || !declared && filePath.endsWith(".db")) {
|
|
1225
1289
|
return {
|
|
1226
1290
|
harness: "opencode",
|
|
1227
1291
|
location: filePath,
|
|
1228
|
-
extract: () => opencodeImagesFromRows(opencodeQuery(filePath,
|
|
1292
|
+
extract: () => opencodeImagesFromRows(opencodeQuery(filePath, cwd))
|
|
1229
1293
|
};
|
|
1230
1294
|
}
|
|
1231
|
-
if (filePath.includes(`${path.sep}.pi${path.sep}`)) {
|
|
1295
|
+
if (declared === "pi" || !declared && filePath.includes(`${path.sep}.pi${path.sep}`)) {
|
|
1232
1296
|
return jsonlSource("pi", filePath, piExtractLine);
|
|
1233
1297
|
}
|
|
1234
1298
|
return jsonlSource("claude-code", filePath, claudeExtractLine);
|
|
1235
1299
|
}
|
|
1236
1300
|
function locateSource(cwd, adapters = ADAPTERS) {
|
|
1237
1301
|
let best = null;
|
|
1302
|
+
const blockers = [];
|
|
1238
1303
|
for (const adapter of adapters) {
|
|
1239
1304
|
let candidate = null;
|
|
1240
1305
|
try {
|
|
1241
1306
|
candidate = adapter.findNewest(cwd);
|
|
1242
|
-
} catch {
|
|
1307
|
+
} catch (error) {
|
|
1308
|
+
blockers.push(`${adapter.name}: ${error instanceof Error ? error.message : String(error)}`);
|
|
1243
1309
|
}
|
|
1244
1310
|
if (candidate && (!best || candidate.timestamp > best.timestamp)) {
|
|
1245
1311
|
best = candidate;
|
|
@@ -1247,25 +1313,31 @@ function locateSource(cwd, adapters = ADAPTERS) {
|
|
|
1247
1313
|
}
|
|
1248
1314
|
if (!best) {
|
|
1249
1315
|
const dirs = adapters.map((a) => a.describe(cwd)).join(" , ");
|
|
1316
|
+
const blocked = blockers.length > 0 ? `
|
|
1317
|
+
Blocked: ${blockers.join(" | ")}` : "";
|
|
1250
1318
|
throw new Error(
|
|
1251
|
-
`No pasted images found in any session storage for this directory (looked in: ${dirs}). The user may not have pasted any, or the storage format changed; ask for a file path instead
|
|
1319
|
+
`No pasted images found in any session storage for this directory (looked in: ${dirs}). The user may not have pasted any, or the storage format changed; ask for a file path instead.${blocked}`
|
|
1252
1320
|
);
|
|
1253
1321
|
}
|
|
1254
1322
|
return best.ref;
|
|
1255
1323
|
}
|
|
1256
1324
|
function sourceForSession(cwd, sessionId, adapters = ADAPTERS) {
|
|
1325
|
+
const blockers = [];
|
|
1257
1326
|
for (const adapter of adapters) {
|
|
1258
1327
|
try {
|
|
1259
1328
|
const ref = adapter.findSession(cwd, sessionId);
|
|
1260
1329
|
if (ref) {
|
|
1261
1330
|
return ref;
|
|
1262
1331
|
}
|
|
1263
|
-
} catch {
|
|
1332
|
+
} catch (error) {
|
|
1333
|
+
blockers.push(`${adapter.name}: ${error instanceof Error ? error.message : String(error)}`);
|
|
1264
1334
|
}
|
|
1265
1335
|
}
|
|
1266
1336
|
const dirs = adapters.map((a) => a.describe(cwd)).join(" , ");
|
|
1337
|
+
const blocked = blockers.length > 0 ? `
|
|
1338
|
+
Blocked: ${blockers.join(" | ")}` : "";
|
|
1267
1339
|
throw new Error(
|
|
1268
|
-
`No session ${sessionId} with pasted images under this project (looked in: ${dirs}). Check --cwd, or drop --session to auto-locate by newest pasted image
|
|
1340
|
+
`No session ${sessionId} with pasted images under this project (looked in: ${dirs}). Check --cwd, or drop --session to auto-locate by newest pasted image.${blocked}`
|
|
1269
1341
|
);
|
|
1270
1342
|
}
|
|
1271
1343
|
function recoverPastedImages(options = {}) {
|
|
@@ -1285,7 +1357,7 @@ function recoverPastedImages(options = {}) {
|
|
|
1285
1357
|
const adapters = scoped ? ADAPTERS.filter((adapter) => adapter.name === scoped) : ADAPTERS;
|
|
1286
1358
|
let source = null;
|
|
1287
1359
|
if (options.transcript) {
|
|
1288
|
-
source = sourceForExplicitPath(options.transcript);
|
|
1360
|
+
source = sourceForExplicitPath(options.transcript, cwd, options.harness);
|
|
1289
1361
|
} else if (options.session) {
|
|
1290
1362
|
source = sourceForSession(cwd, options.session, adapters);
|
|
1291
1363
|
} else {
|
|
@@ -1307,14 +1379,22 @@ function recoverPastedImages(options = {}) {
|
|
|
1307
1379
|
`No pasted images found in ${source.location}. The user may not have pasted any, or the storage format changed; ask for a file path instead.`
|
|
1308
1380
|
);
|
|
1309
1381
|
}
|
|
1310
|
-
fs.mkdirSync(outDir, { recursive: true });
|
|
1382
|
+
fs.mkdirSync(outDir, { recursive: true, mode: 448 });
|
|
1383
|
+
try {
|
|
1384
|
+
fs.chmodSync(outDir, 448);
|
|
1385
|
+
} catch {
|
|
1386
|
+
}
|
|
1311
1387
|
const picked = all.slice(-count);
|
|
1312
1388
|
const images = picked.map((image) => {
|
|
1313
1389
|
const buffer = Buffer.from(image.data, "base64");
|
|
1314
1390
|
const hash = crypto.createHash("sha256").update(buffer).digest("hex").slice(0, 8);
|
|
1315
|
-
const ext = EXT_BY_MIME[image.mediaType] ??
|
|
1391
|
+
const ext = EXT_BY_MIME[image.mediaType] ?? extensionFromMediaType(image.mediaType);
|
|
1316
1392
|
const filePath = path.join(outDir, `paste-${hash}.${ext}`);
|
|
1317
|
-
fs.writeFileSync(filePath, buffer);
|
|
1393
|
+
fs.writeFileSync(filePath, buffer, { mode: 384 });
|
|
1394
|
+
try {
|
|
1395
|
+
fs.chmodSync(filePath, 384);
|
|
1396
|
+
} catch {
|
|
1397
|
+
}
|
|
1318
1398
|
const recovered = {
|
|
1319
1399
|
path: filePath,
|
|
1320
1400
|
mediaType: image.mediaType,
|
|
@@ -1332,7 +1412,7 @@ function recoverPastedImages(options = {}) {
|
|
|
1332
1412
|
return result;
|
|
1333
1413
|
}
|
|
1334
1414
|
const program = new Command();
|
|
1335
|
-
program.name("modlens").description("Plug-in vision for text-only LLMs: image in, structured JSON evidence out").version("2.7.
|
|
1415
|
+
program.name("modlens").description("Plug-in vision for text-only LLMs: image in, structured JSON evidence out").version("2.7.4");
|
|
1336
1416
|
program.command("analyze", { isDefault: true }).description("Analyze an image into structured JSON evidence (default command)").requiredOption("-i, --input <path|url>", "Input image path or https URL").option("-o, --output <path>", "Write result JSON to a file").option("-m, --model <name>", "Provider model name").option("-p, --provider <name>", `Vision provider (${listProviders().join(", ")})`).option("--prompt <text>", "Extra focus for this image").option("--timeout <ms>", "Provider timeout in milliseconds", "180000").option("--provider-bin <path>", "Provider binary path (default: agy)").option("--workdir <path>", "Working directory for the provider").action(async (options) => {
|
|
1337
1417
|
try {
|
|
1338
1418
|
const timeoutMs = Number.parseInt(options.timeout, 10);
|