@liustack/modlens 3.14.0 → 3.16.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +12 -0
- package/README.md +49 -2
- package/README.zh-CN.md +68 -17
- package/dist/main.js +55 -11
- package/docs/cli.md +2 -0
- package/docs/cli.zh-CN.md +98 -0
- package/docs/harness-setup.md +18 -4
- package/docs/harness-setup.zh-CN.md +67 -0
- package/docs/output-schema.md +2 -0
- package/docs/output-schema.zh-CN.md +74 -0
- package/docs/security.md +2 -0
- package/docs/security.zh-CN.md +43 -0
- package/docs/troubleshooting.md +2 -0
- package/docs/troubleshooting.zh-CN.md +210 -0
- package/dsh/client.js +87 -19
- package/dsh/index.js +329 -80
- package/package.json +1 -1
- package/skills/modlens/SKILL.md +5 -5
- package/skills/modlens/references/configure.md +4 -2
- package/skills/modlens/references/configure.zh-CN.md +177 -0
- package/skills/modlens/references/runtime.md +1 -1
- package/skills/modlens/scripts/run.ps1 +1 -1
- package/skills/modlens/scripts/run.sh +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,17 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 3.16.0 - 2026-08-14
|
|
4
|
+
|
|
5
|
+
- **dsh: whether a paste is taken over is now the host's call, made from real model metadata.** The browser half used to guess with a name regex, which read every vision model it did not recognize (Qwen2.5-VL, GPT-4o, ...) as text-only and hijacked its native paste. And turning `pasteToPath` off only removed the host route while the client kept capturing pastes into a 404. The client now asks `GET /modlens/paste?model=<selector label>` and the host answers from the provider registry's declared `inputModalities`, with every unknown answered conservatively: the label carries no provider id, so EVERY model whose name or id appears in it must be confirmed text-only. One image-capable match anywhere vetoes, an unreadable provider catalog vetoes (the vision twin could live there), and missing modality metadata counts as unknown, never as text-only. Verdicts are re-asked on every composer focus and every paste, the host empties its own cache on every provider-topology change (a same-named vision route mounting mid-session is seen within one round-trip), a 60-second hard age bound backstops both, and a route that vanishes mid-session costs only the pastes inside the one round-trip it takes the failure to come back, after which the client forgets its verdicts and stands down. Until a model is positively confirmed text-only, pastes stay native. The paste route's magic-byte table now matches the CLI's signature for signature (a plain `ftypmp42` video is refused instead of saved as `paste.heic`, real heic/heif brands pass), and the browser half gained its first test suite, loaded straight from `dsh/client.js`.
|
|
6
|
+
- **A provider that ignores SIGTERM no longer outlives the CLI.** The SIGKILL escalation timer was unref'd, so in a standalone run it died with the event loop and a stubborn provider process survived its parent indefinitely, found by independent review with a live repro. The timer is now ref'd and cleared by the child's own exit (normal runs pay nothing, the stubborn case at most the 2s grace), pinned by a test that drives the real spawn path from a genuinely independent parent process, because inside the test runner its own timers had been masking exactly this.
|
|
7
|
+
- **`config show` and error text stop leaking proxy credentials.** A proxy URL's userinfo (`http://alice:s3cr3t@proxy`) rendered verbatim in the one command whose contract is being safe to paste into an issue. It is now masked everywhere config show renders a proxy, and the shared redactor handles URL userinfo through the same WHATWG parser the runtime connects with, instead of a regex approximation: backslash authorities, slash runs, tabs inside the authority, and passwords containing `@` all get their credentials removed in normalized form, while scheme-less `//text@` prose and ordinary query `@`s stay verbatim.
|
|
8
|
+
- **The release pipeline can no longer publish first and fail later.** The tag workflow validated the CHANGELOG section only after `npm publish`. The notes are now extracted and verified before anything irreversible runs. And the release script's `git push --follow-tags` could deliver the tag while a stale main was rejected, triggering a release from a tree the remote branch never contained. It now syncs with origin first, refuses a behind or diverged main and an already-existing remote tag, and pushes branch and tag atomically, together or not at all.
|
|
9
|
+
- **The gates got honest.** `pnpm lint` always claimed to check `dsh` while the Biome config silently excluded it (now covered, with the plugin's own style rules); CI ran "Node 22" as latest-22.x while calling it the 22.19 floor (the matrix now runs the exact floor); and an Electron-runtime argv test pins the [#25](https://github.com/liustack/modlens/issues/25) fix against regression. Docs corrections ride along: both READMEs present the two paste flows as they actually ship, an empty `provider` is documented as the failover chain rather than a hardcoded antigravity pin, and the skill front matter carries the real 22.19 floor.
|
|
10
|
+
|
|
11
|
+
## 3.15.0 - 2026-08-14
|
|
12
|
+
|
|
13
|
+
- **dsh: the vision wrap discovers every eligible provider route ([#29](https://github.com/liustack/modlens/issues/29)).** The wrapper used to bind one hardcoded upstream, so a machine with several subscription packages (opencode-go, zai, ...) had to hand-pick a single route and lose the rest. With `upstream` unset the plugin now sweeps the llm registry and registers a `modlens-<provider>` wrapper for every route carrying wrappable text-only DeepSeek/GLM models, re-sweeping on the registry's own `llm/adapters-updated` notification so late-registering routes (llm-pi-ai mounts after settings load) are picked up without polling. A `discover` array narrows the set, `families` filters as before, setting `upstream` keeps exact legacy single-route behavior, wrappers never wrap themselves, and the deepseek-official wrap keeps its historical `deepseek-modlens` id so an upgraded machine's model selection survives. Design and behavior spec contributed by @zlycode01, who also published a reference implementation in a public fork; this is an independent upstream implementation of that spec, with the polling schedule replaced by the registry notification.
|
|
14
|
+
|
|
3
15
|
## 3.14.0 - 2026-08-14
|
|
4
16
|
|
|
5
17
|
- **dsh: pasting into a text-only model now just works — the paste becomes a file path.** The plugin grows a browser half (a hand-written bundle in dsh's client plugin protocol, zero dependencies, loaded automatically under the web profile). A capture-phase listener takes over image pastes before the composer's own intake: the bytes go to the plugin's `/modlens/paste` route on the dsh web server (loopback-bound, magic-byte checked, 25 MB cap, private 0600 temp file), and the composer receives the file path as plain text — the exact shape Pi, OpenCode, and Claude Code hand their models, and the modlens skill's primary trigger. Image admission never fires because the message carries no image attachment; verified end to end with a native text-only DeepSeek-V4-Flash, whose visible reasoning quoted the modlens skill rule and went for the path. The takeover is conditional: `(modlens vision)` variants and known vision models keep the native paste flow (thumbnails and all), and `pasteToPath: false` turns the feature off. The host route rides a scoped `ctx.inject(['webServer'], ...)`, so headless profiles never see any of it.
|
package/README.md
CHANGED
|
@@ -26,7 +26,7 @@
|
|
|
26
26
|
<img src="https://img.shields.io/badge/users-unknown-lightgrey?style=flat-square" alt="Users unknown">
|
|
27
27
|
</p>
|
|
28
28
|
|
|
29
|
-
DeepSeek and GLM
|
|
29
|
+
The flagship DeepSeek and GLM chat models are text-only and cannot read images. ModLens is a plug-in vision engine that gives a text-only model sight. **ModLens reads images pasted straight into the chat**, no saving to a file and passing a path first.
|
|
30
30
|
|
|
31
31
|
## Talk to us
|
|
32
32
|
|
|
@@ -34,10 +34,13 @@ Issues are welcome any time: [open one](https://github.com/liustack/modlens/issu
|
|
|
34
34
|
|
|
35
35
|
## Highlights
|
|
36
36
|
|
|
37
|
-
**🥇 The first vision plugin for DeepSeek Harness (dsh):** one command, `npx -y @deepseek-ai/dsh plugin --profile web add @liustack/modlens@latest`, and the text-only DeepSeek model behind dsh reads images through a native `read_image` tool. If dsh warns `declares no dsh.bundle`, pnpm's release-age gate installed an old version: see [troubleshooting](docs/troubleshooting.md#dsh-says-declares-no-dshbundle--installed-as-a-plain-dependency) for the one-line fix.
|
|
37
|
+
**🥇 The first vision plugin for DeepSeek Harness (dsh):** one command, `npx -y @deepseek-ai/dsh plugin --profile web add @liustack/modlens@latest`, and the text-only DeepSeek model behind dsh reads images through a native `read_image` tool. If dsh warns `declares no dsh.bundle`, pnpm's release-age gate installed an old version: see [troubleshooting](docs/troubleshooting.md#dsh-says-declares-no-dshbundle--installed-as-a-plain-dependency) for the one-line fix.
|
|
38
|
+
|
|
39
|
+
Pasting an image works two ways. **① Just paste.** On a text-only model the pasted image lands as a private temp file and its path enters the composer — the same interaction OpenCode and Pi ship — and the `read_image` tool takes it from there. **② Pick a `(modlens vision)` entry** in the model selector (it remembers your choice, so once is enough), then paste: the thumbnail stays visible in your message, closer to the Codex app feel, and the image is converted to structured evidence at request time, answered by the same underlying route. The plugin auto-discovers every provider route carrying text-only DeepSeek or GLM models and adds a wrapped entry per route (a stock install gets **`DeepSeek-V4-Flash (modlens vision)`** and **`DeepSeek-V4-Pro (modlens vision)`**; extra routes like opencode-go or zai get their own); the two families' own vision models are excluded automatically. Which paste route applies is the host's per-model call: only a model its metadata positively confirms text-only is taken over, anything unconfirmed is left alone, so vision models keep their native paste ([details](docs/harness-setup.md)).
|
|
38
40
|
|
|
39
41
|
**Paste an image and it reads it.** No saving to a file and passing a path first.
|
|
40
42
|
|
|
43
|
+
- **The lightest touch on the market.** No hooks, no wrappers, no local proxy daemon, not a single line changed in any harness config: on the skill harnesses it is exactly one skill folder, on dsh exactly one plugin. Uninstalling is deleting a folder, and your agents are back to stock.
|
|
41
44
|
- **Zero-config start.** Reuses what Claude Code, Codex, OpenCode, or Pi already have set up: the multimodal models on your machine go straight to work. Nothing at all? Antigravity CLI is a free no-key channel, and a free Gemini key brings a read down to 5-10 seconds.
|
|
42
45
|
- **Evidence, not imagination.** Full transcription, reading-order layout regions, entity and relation lists. The model quotes specifics.
|
|
43
46
|
- **Install once, use everywhere.** Verified on real machines in Claude Code, Codex, Pi, and OpenCode.
|
|
@@ -63,6 +66,50 @@ The install also inventories vision reachable through your other local harness C
|
|
|
63
66
|
|
|
64
67
|
Once installed, just chat. Paste an image or drop a path, ask anything, and the skill triggers on its own: the image goes to a vision engine and the answer comes back grounded in what it read.
|
|
65
68
|
|
|
69
|
+
## Vision engines: five built-in providers, four reusable CLIs, one failover chain
|
|
70
|
+
|
|
71
|
+
ModLens does not depend on any single vision service. Nine sources of vision in total: five built-in providers, any one of which is enough, plus four local agent CLIs whose logins can be reused. The built-ins:
|
|
72
|
+
|
|
73
|
+
| Provider | What it needs | Speed per read | Good for |
|
|
74
|
+
| :-- | :-- | :-- | :-- |
|
|
75
|
+
| `gemini-api` | a free Gemini API key ([3 minutes, no card](https://aistudio.google.com)) | 5-10s | the recommended default |
|
|
76
|
+
| `openai` | any OpenAI-compatible endpoint (key + baseUrl + model) | 5-10s | qwen-vl, GLM, self-hosted gateways |
|
|
77
|
+
| `anthropic` | an Anthropic API key | 5-10s | machines already holding one |
|
|
78
|
+
| `antigravity-cli` | the free `agy` CLI, one browser sign-in, no key | 15-45s | zero-signup starts |
|
|
79
|
+
| `claude-cli` | a signed-in Claude Code | 20-45s | riding your existing Claude subscription |
|
|
80
|
+
|
|
81
|
+
Without a pinned provider, every configured engine forms one failover chain: the fast API providers try first, the agent CLIs back them up, the first good result wins, and `meta.attempts` records every attempt so a fallback is never silent.
|
|
82
|
+
|
|
83
|
+
### `openai` is a universal socket, not just OpenAI
|
|
84
|
+
|
|
85
|
+
Any endpoint speaking the OpenAI chat-completions protocol with image input plugs straight in — that covers most of the vision-model world:
|
|
86
|
+
|
|
87
|
+
```bash
|
|
88
|
+
modlens config set openai.baseUrl https://dashscope.aliyuncs.com/compatible-mode/v1 # qwen-vl
|
|
89
|
+
modlens config set openai.apiKey <key>
|
|
90
|
+
modlens config set openai.model qwen3-vl-plus
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
The same three keys work for GLM's open platform, SiliconFlow, OpenRouter, a self-hosted vLLM/Ollama, or any gateway of your own. If your favorite vision model has an OpenAI-compatible API, ModLens can drive it.
|
|
94
|
+
|
|
95
|
+
### Reusing what your machine already has
|
|
96
|
+
|
|
97
|
+
Two more sources of vision need zero new keys, each behind one explicit consent recorded in config:
|
|
98
|
+
|
|
99
|
+
- **The harness you are talking in right now.** Running inside Claude Code with a subscription signed in? `claude-cli` reads images through it out of the box. The install flow asks the same question for whichever harness you install into.
|
|
100
|
+
- **Every other agent CLI on the machine.** `modlens doctor` discovers them, you grant per harness, and they join the same failover chain with no priority over your own keys. Every reused read is labeled in `meta.warnings` with whose quota it spent, so nothing is ever silently billed:
|
|
101
|
+
|
|
102
|
+
| Reused CLI | What it needs | Grant with | Rides as |
|
|
103
|
+
| :-- | :-- | :-- | :-- |
|
|
104
|
+
| Codex | a signed-in Codex CLI with a vision model | `config set reuse.codex true` | agent lane, 15-45s |
|
|
105
|
+
| OpenCode | a vision model configured in OpenCode | `config set reuse.opencode true` | agent lane, 15-45s |
|
|
106
|
+
| Pi | model credentials held by Pi | `config set reuse.pi true` | an API key upgrades to the 5-10s inline lane, OAuth drives Pi itself |
|
|
107
|
+
| Grok | a signed-in Grok CLI (SuperGrok) | `config set reuse.grok true` | agent lane, 15-45s |
|
|
108
|
+
|
|
109
|
+
### Picking and routing
|
|
110
|
+
|
|
111
|
+
Two knobs: `modlens config set provider <name>` states a preference (the chain still backs it up), `-p <name>` pins exactly one with no fallback. Machines behind a proxy set `HTTPS_PROXY` or `modlens config set proxy <url>` and the API providers route through it. Details: the [CLI manual](docs/cli.md) for defaults and flags, [Configuration](skills/modlens/references/configure.md) for every key, and [Security](docs/security.md) for who fetches what on remote URLs.
|
|
112
|
+
|
|
66
113
|
## See it work
|
|
67
114
|
|
|
68
115
|
Unedited runs, all driving a text-only DeepSeek-V4-Flash.
|
package/README.zh-CN.md
CHANGED
|
@@ -10,10 +10,10 @@
|
|
|
10
10
|
|
|
11
11
|
<p align="center">
|
|
12
12
|
<a href="./README.md">English</a> ·
|
|
13
|
-
<a href="docs/troubleshooting.md">故障排查</a> ·
|
|
14
|
-
<a href="skills/modlens/references/configure.md">配置</a> ·
|
|
15
|
-
<a href="docs/output-schema.md">输出契约</a> ·
|
|
16
|
-
<a href="docs/security.md">安全</a> ·
|
|
13
|
+
<a href="docs/troubleshooting.zh-CN.md">故障排查</a> ·
|
|
14
|
+
<a href="skills/modlens/references/configure.zh-CN.md">配置</a> ·
|
|
15
|
+
<a href="docs/output-schema.zh-CN.md">输出契约</a> ·
|
|
16
|
+
<a href="docs/security.zh-CN.md">安全</a> ·
|
|
17
17
|
<a href="https://github.com/liustack/modsearch">ModSearch(联网)</a>
|
|
18
18
|
</p>
|
|
19
19
|
|
|
@@ -26,7 +26,7 @@
|
|
|
26
26
|
<img src="https://img.shields.io/badge/users-unknown-lightgrey?style=flat-square" alt="Users unknown">
|
|
27
27
|
</p>
|
|
28
28
|
|
|
29
|
-
DeepSeek 和 GLM
|
|
29
|
+
DeepSeek 和 GLM 的主力对话模型是纯文本的,无法进行图片识别。ModLens 借助外挂视觉引擎,为纯文本模型补上视觉能力。**ModLens 支持直接粘贴图片识别**,无需先保存成文件再提供路径。
|
|
30
30
|
|
|
31
31
|
## 交流
|
|
32
32
|
|
|
@@ -34,10 +34,17 @@ DeepSeek 和 GLM 没有视觉能力,无法进行图片识别。ModLens 借助
|
|
|
34
34
|
|
|
35
35
|
## 亮点
|
|
36
36
|
|
|
37
|
-
**🥇 全网第一个支持 DeepSeek Harness(dsh)的外挂视觉识别插件:**一条命令 `npx -y @deepseek-ai/dsh plugin --profile web add @liustack/modlens@latest`,dsh 背后的纯文本 DeepSeek 模型即可通过原生 `read_image` 工具读图。如果 dsh 提示 `declares no dsh.bundle`,是 pnpm
|
|
37
|
+
**🥇 全网第一个支持 DeepSeek Harness(dsh)的外挂视觉识别插件:**一条命令 `npx -y @deepseek-ai/dsh plugin --profile web add @liustack/modlens@latest`,dsh 背后的纯文本 DeepSeek 模型即可通过原生 `read_image` 工具读图。如果 dsh 提示 `declares no dsh.bundle`,是 pnpm 的发布冷静期装了旧版,一行命令可解,见[故障排查](docs/troubleshooting.zh-CN.md#dsh-提示-declares-no-dshbundle--installed-as-a-plain-dependency)。
|
|
38
|
+
|
|
39
|
+
DeepSeek Harness 粘贴识图有两种玩法。
|
|
40
|
+
|
|
41
|
+
**① 直接粘贴** 贴进来的图片自主转换成文件路径进输入框(与 OpenCode、Pi 同款交互),`read_image` 工具接手读图。
|
|
42
|
+
|
|
43
|
+
**② 切到带 `(modlens vision)` 后缀的模型变体**(选择器有记忆,选一次就行)再粘贴:缩略图直接可见、所见即所得,体验更接近 Codex App。变体由插件自动发现生成:每条承载纯文本 DeepSeek 或 GLM 模型的 provider 路由各得一组包装条目(默认安装下就是 **`DeepSeek-V4-Flash (modlens vision)`** 和 **`DeepSeek-V4-Pro (modlens vision)`**,装了 opencode-go、zai 等额外路由的机器会各自多出一组),两家自己的视觉型号自动排除。走哪条通路由 host 依据真实模型元数据逐个裁决:只有被元数据确认纯文本的模型才会被接管,确认不了的一律不动,视觉模型因此保留原生贴图([细节](docs/harness-setup.zh-CN.md))。
|
|
38
44
|
|
|
39
45
|
**直接粘贴图片识别** 无需先保存成文件再提供路径。
|
|
40
46
|
|
|
47
|
+
- **全网最轻量。** 不用 hook,不套壳,不跑本地代理进程,不改任何 harness 配置的一行字:在 skill 类 harness 里它就是一个 skill 文件夹,在 dsh 里就是一个插件。卸载等于删个文件夹,你的 agent 立刻回到原样。
|
|
41
48
|
- **零配置起手。** 复用 Claude Code、Codex、OpenCode、Pi 已配置,直接复用你本机的其他多模态模型。什么都没有?Antigravity CLI 是免 key 的免费通道,配一个免费 Gemini key 可将识别耗时降至 5 到 10 秒。
|
|
42
49
|
- **基于证据而非想象。** 全文转录、按阅读顺序划分的版面区块、实体与关系列表,模型引用的是具体内容。
|
|
43
50
|
- **一次安装,多端可用。** Claude Code、Codex、Pi、OpenCode 均经真机验证。
|
|
@@ -65,12 +72,56 @@ agy # 浏览器完成
|
|
|
65
72
|
npx -y @deepseek-ai/dsh plugin --profile web add @liustack/modlens@latest
|
|
66
73
|
```
|
|
67
74
|
|
|
68
|
-
装完即有 `read_image` 工具,选「(modlens vision)」模型变体即可直接粘贴识图。引擎配置同样在 `~/.modlens`,详见[宿主接入](docs/harness-setup.md)。
|
|
75
|
+
装完即有 `read_image` 工具,选「(modlens vision)」模型变体即可直接粘贴识图。引擎配置同样在 `~/.modlens`,详见[宿主接入](docs/harness-setup.zh-CN.md)。
|
|
69
76
|
|
|
70
77
|
## 用法
|
|
71
78
|
|
|
72
79
|
装好之后不需要记任何命令。正常聊天,粘贴图片或给出图片路径,提问即可,skill 自动触发:图片交给视觉引擎,答案基于读到的内容返回。
|
|
73
80
|
|
|
81
|
+
## 视觉引擎:五个内置 provider,四家可复用 CLI,一条故障转移链
|
|
82
|
+
|
|
83
|
+
ModLens 不绑定任何单一视觉服务。视觉来源一共九个:五个内置 provider(配好任意一个就能用),加四家本机 agent CLI 的登录可以复用。先看内置的:
|
|
84
|
+
|
|
85
|
+
| Provider | 需要什么 | 单次识别耗时 | 适合谁 |
|
|
86
|
+
| :---------------- | :----------------------------------------------------------------------- | :----------- | :--------------------- |
|
|
87
|
+
| `gemini-api` | 免费 Gemini key([三分钟领取,无需信用卡](https://aistudio.google.com)) | 5-10 秒 | 推荐默认 |
|
|
88
|
+
| `openai` | 任意 OpenAI 兼容端点(key + baseUrl + model) | 5-10 秒 | qwen-vl、GLM、自建网关 |
|
|
89
|
+
| `anthropic` | Anthropic API key | 5-10 秒 | 手上已有 key 的机器 |
|
|
90
|
+
| `antigravity-cli` | 免费的 `agy` CLI,浏览器登录一次,无需 key | 15-45 秒 | 完全免注册起步 |
|
|
91
|
+
| `claude-cli` | 已登录的 Claude Code | 20-45 秒 | 复用现有 Claude 订阅 |
|
|
92
|
+
|
|
93
|
+
不钉死 provider 时,所有配好的引擎组成一条故障转移链:API 快车道先试,agent CLI 兜底,第一个可用结果胜出,`meta.attempts` 记录每次尝试,回退永远不是无声的。
|
|
94
|
+
|
|
95
|
+
### `openai` 是万能接口,不只是 OpenAI
|
|
96
|
+
|
|
97
|
+
任何讲 OpenAI chat-completions 协议、支持图片输入的端点都能直接插上,这基本覆盖了视觉模型的大半个世界:
|
|
98
|
+
|
|
99
|
+
```bash
|
|
100
|
+
modlens config set openai.baseUrl https://dashscope.aliyuncs.com/compatible-mode/v1 # qwen-vl
|
|
101
|
+
modlens config set openai.apiKey <key>
|
|
102
|
+
modlens config set openai.model qwen3-vl-plus
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
同样三个键,换成 GLM 开放平台、SiliconFlow、OpenRouter、自建 vLLM/Ollama 或你自己的网关都一样。你常用的视觉模型只要有 OpenAI 兼容 API,ModLens 就能驱动它。
|
|
106
|
+
|
|
107
|
+
### 复用你机器上已有的东西
|
|
108
|
+
|
|
109
|
+
还有两处现成的视觉能力,一个新 key 都不用配,每家都在你明确同意后才启用:
|
|
110
|
+
|
|
111
|
+
- **你正在对话的这个 harness 本身。**在登录了订阅的 Claude Code 里用?`claude-cli` 开箱即可借它读图。装进哪个 harness,安装流程就会问哪个 harness 的授权。
|
|
112
|
+
- **机器上其他的 agent CLI。**`modlens doctor` 会逐个发现,你按家授权,它们与你自己的 key 平级入链,不插队。每次复用都在 `meta.warnings` 里标明花的是谁的额度,绝不无声扣费:
|
|
113
|
+
|
|
114
|
+
| 复用来源 | 需要什么 | 授权命令 | 走哪条道 |
|
|
115
|
+
| :------- | :----------------------------- | :------------------------------- | :----------------------------------------------------- |
|
|
116
|
+
| Codex | 已登录且有视觉模型的 Codex CLI | `config set reuse.codex true` | agent 通道,15-45 秒 |
|
|
117
|
+
| OpenCode | OpenCode 里配好的视觉模型 | `config set reuse.opencode true` | agent 通道,15-45 秒 |
|
|
118
|
+
| Pi | Pi 持有的模型凭据 | `config set reuse.pi true` | API key 直接升级到 5-10 秒的快车道,OAuth 驱动 Pi 本体 |
|
|
119
|
+
| Grok | 已登录的 Grok CLI(SuperGrok) | `config set reuse.grok true` | agent 通道,15-45 秒 |
|
|
120
|
+
|
|
121
|
+
### 选择与路由
|
|
122
|
+
|
|
123
|
+
两个旋钮:`modlens config set provider <name>` 表达偏好(链继续兜底),`-p <name>` 钉死单个不回退。代理环境设 `HTTPS_PROXY` 或 `modlens config set proxy <url>`,API provider 自动走代理。细节见 [CLI 手册](docs/cli.zh-CN.md)(默认模型与参数)、[配置手册](skills/modlens/references/configure.zh-CN.md)(全部配置键)、[安全说明](docs/security.zh-CN.md)(远程 URL 由谁抓取)。
|
|
124
|
+
|
|
74
125
|
## 实测
|
|
75
126
|
|
|
76
127
|
以下均为原样记录,驱动的都是纯文本的 DeepSeek-V4-Flash。
|
|
@@ -97,16 +148,16 @@ Codex 桌面 App 中识别一张推文截图。作者、配文、照片内容(
|
|
|
97
148
|
|
|
98
149
|
## 文档
|
|
99
150
|
|
|
100
|
-
| 文档
|
|
101
|
-
|
|
|
102
|
-
| [安装手册](INSTALL.md)
|
|
103
|
-
| [CLI 手册](docs/cli.md) | skill 所驱动的 CLI:参数、配置与体检 |
|
|
104
|
-
| [故障排查](docs/troubleshooting.md) | 命令报错,查成因和解法 |
|
|
105
|
-
| [配置手册](skills/modlens/references/configure.md) | 配置 key、切换 provider、排查配置 |
|
|
106
|
-
| [输出契约](docs/output-schema.md) | 解析 JSON 或构建下游工具 |
|
|
107
|
-
| [宿主接入](docs/harness-setup.md) | 在 Codex、Claude Code、Pi、OpenCode 中配置 |
|
|
108
|
-
| [安全说明](docs/security.md) | 恢复文件的权限、图片内容作为不可信输入 |
|
|
109
|
-
| [更新日志](CHANGELOG.md)
|
|
151
|
+
| 文档 | 适用场景 |
|
|
152
|
+
| :------------------------------------------------------- | :----------------------------------------- |
|
|
153
|
+
| [安装手册](INSTALL.md) | 一步步安装 skill(为 agent 编写) |
|
|
154
|
+
| [CLI 手册](docs/cli.zh-CN.md) | skill 所驱动的 CLI:参数、配置与体检 |
|
|
155
|
+
| [故障排查](docs/troubleshooting.zh-CN.md) | 命令报错,查成因和解法 |
|
|
156
|
+
| [配置手册](skills/modlens/references/configure.zh-CN.md) | 配置 key、切换 provider、排查配置 |
|
|
157
|
+
| [输出契约](docs/output-schema.zh-CN.md) | 解析 JSON 或构建下游工具 |
|
|
158
|
+
| [宿主接入](docs/harness-setup.zh-CN.md) | 在 Codex、Claude Code、Pi、OpenCode 中配置 |
|
|
159
|
+
| [安全说明](docs/security.zh-CN.md) | 恢复文件的权限、图片内容作为不可信输入 |
|
|
160
|
+
| [更新日志](CHANGELOG.md) | 查询版本变更 |
|
|
110
161
|
|
|
111
162
|
## 参与方式
|
|
112
163
|
|
package/dist/main.js
CHANGED
|
@@ -355,6 +355,8 @@ async function fetchRemoteImageBase64(url, timeoutMs) {
|
|
|
355
355
|
if (response.status >= 300 && response.status < 400) {
|
|
356
356
|
const location = response.headers.get("location");
|
|
357
357
|
if (!location) {
|
|
358
|
+
await response.body?.cancel().catch(() => {
|
|
359
|
+
});
|
|
358
360
|
throw new Error(
|
|
359
361
|
`Redirect response (${response.status}) missing location header: ${safeUrl(current.toString())}`
|
|
360
362
|
);
|
|
@@ -367,12 +369,16 @@ async function fetchRemoteImageBase64(url, timeoutMs) {
|
|
|
367
369
|
continue;
|
|
368
370
|
}
|
|
369
371
|
if (!response.ok) {
|
|
372
|
+
await response.body?.cancel().catch(() => {
|
|
373
|
+
});
|
|
370
374
|
throw new Error(
|
|
371
375
|
`Failed to download image (${response.status}): ${safeUrl(current.toString())}`
|
|
372
376
|
);
|
|
373
377
|
}
|
|
374
378
|
const declaredLength = Number(response.headers.get("content-length"));
|
|
375
379
|
if (Number.isFinite(declaredLength) && declaredLength > MAX_REMOTE_IMAGE_BYTES) {
|
|
380
|
+
await response.body?.cancel().catch(() => {
|
|
381
|
+
});
|
|
376
382
|
throw new Error(
|
|
377
383
|
`Remote image is ${declaredLength} bytes, over the ${MAX_REMOTE_IMAGE_BYTES}-byte limit: ${safeUrl(current.toString())}`
|
|
378
384
|
);
|
|
@@ -385,9 +391,7 @@ async function fetchRemoteImageBase64(url, timeoutMs) {
|
|
|
385
391
|
}
|
|
386
392
|
throw new Error(`Too many redirects (max ${MAX_REDIRECTS}): ${safeUrl(url)}`);
|
|
387
393
|
} finally {
|
|
388
|
-
|
|
389
|
-
void dispatcher.close();
|
|
390
|
-
}
|
|
394
|
+
await Promise.allSettled(dispatchers.map((dispatcher) => dispatcher.close()));
|
|
391
395
|
}
|
|
392
396
|
}
|
|
393
397
|
function pinnedDispatcher(pinned) {
|
|
@@ -773,10 +777,29 @@ const TOKEN_SHAPES = [
|
|
|
773
777
|
/\beyJ[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{4,}\b/g,
|
|
774
778
|
// Auth headers: "Bearer xyz" / "Authorization: xyz" (space form is real).
|
|
775
779
|
/\b(?:bearer|authorization)\b[=:\s]+"?[A-Za-z0-9._~+/-]{12,}"?/gi,
|
|
776
|
-
// Labeled keys need an explicit = or : separator
|
|
780
|
+
// Labeled keys need an explicit = or : separator. Prose like
|
|
777
781
|
// "token limit_exceeded" is diagnostics, not a credential.
|
|
778
782
|
/\b(?:token|api[-_]?key)\b\s*[=:]\s*"?[A-Za-z0-9._~+/-]{12,}"?/gi
|
|
779
783
|
];
|
|
784
|
+
const URL_CANDIDATE = /\b[a-z][a-z0-9+.-]*:[^ ]*@[^ ]*/gi;
|
|
785
|
+
const RAW_USERINFO = /^([a-z][a-z0-9+.-]*:[\\/]{2,4})[^\s/?#]*@/i;
|
|
786
|
+
function parseUrl(candidate) {
|
|
787
|
+
try {
|
|
788
|
+
return new URL(candidate);
|
|
789
|
+
} catch {
|
|
790
|
+
return null;
|
|
791
|
+
}
|
|
792
|
+
}
|
|
793
|
+
function rebuildMasked(url, replacement) {
|
|
794
|
+
return `${url.protocol}//${replacement}@${url.host}${url.pathname}${url.search}${url.hash}`;
|
|
795
|
+
}
|
|
796
|
+
function maskUrlCredentials(url) {
|
|
797
|
+
const parsed = parseUrl(url);
|
|
798
|
+
if (parsed) {
|
|
799
|
+
return parsed.username !== "" || parsed.password !== "" ? rebuildMasked(parsed, "***") : url;
|
|
800
|
+
}
|
|
801
|
+
return url.replace(RAW_USERINFO, "$1***@");
|
|
802
|
+
}
|
|
780
803
|
function redactSecrets(text, knownSecrets = []) {
|
|
781
804
|
let out = text;
|
|
782
805
|
for (const secret of knownSecrets) {
|
|
@@ -787,6 +810,23 @@ function redactSecrets(text, knownSecrets = []) {
|
|
|
787
810
|
for (const shape of TOKEN_SHAPES) {
|
|
788
811
|
out = out.replace(shape, "[redacted]");
|
|
789
812
|
}
|
|
813
|
+
out = out.replace(URL_CANDIDATE, (token) => {
|
|
814
|
+
const pieces = token.split(/(?<=[^a-z0-9+.-])(?=[a-z][a-z0-9+.-]*:[\\/]{1,4})/i);
|
|
815
|
+
if (pieces.length === 1) {
|
|
816
|
+
const parsed = parseUrl(token);
|
|
817
|
+
if (parsed) {
|
|
818
|
+
return parsed.username !== "" || parsed.password !== "" ? rebuildMasked(parsed, "[redacted]") : token;
|
|
819
|
+
}
|
|
820
|
+
return token.replace(RAW_USERINFO, "$1[redacted]@");
|
|
821
|
+
}
|
|
822
|
+
return pieces.map((piece) => {
|
|
823
|
+
const parsed = parseUrl(piece);
|
|
824
|
+
if (parsed) {
|
|
825
|
+
return parsed.password !== "" ? rebuildMasked(parsed, "[redacted]") : piece;
|
|
826
|
+
}
|
|
827
|
+
return piece.replace(RAW_USERINFO, "$1[redacted]@");
|
|
828
|
+
}).join("");
|
|
829
|
+
});
|
|
790
830
|
return out;
|
|
791
831
|
}
|
|
792
832
|
const ANTHROPIC_DEFAULT_MODEL = "claude-haiku-4-5-20251001";
|
|
@@ -1498,7 +1538,7 @@ function renderEffectiveConfig(config2, env = process.env) {
|
|
|
1498
1538
|
const value = envValue ?? fileSettings[field];
|
|
1499
1539
|
const source = envValue ? "env" : fileSettings[field] !== void 0 ? "file" : null;
|
|
1500
1540
|
if (value !== void 0 && source) {
|
|
1501
|
-
const shown = field === "apiKey" ? maskKey(value) : value;
|
|
1541
|
+
const shown = field === "apiKey" ? maskKey(value) : field === "proxy" ? maskUrlCredentials(value) : value;
|
|
1502
1542
|
fields[field] = `${shown} (${source})`;
|
|
1503
1543
|
}
|
|
1504
1544
|
}
|
|
@@ -1516,9 +1556,10 @@ function renderEffectiveConfig(config2, env = process.env) {
|
|
|
1516
1556
|
effective.provider = config2.provider.trim();
|
|
1517
1557
|
}
|
|
1518
1558
|
if (config2.proxy?.trim()) {
|
|
1519
|
-
effective.proxy = `${config2.proxy.trim()} (file)`;
|
|
1559
|
+
effective.proxy = `${maskUrlCredentials(config2.proxy.trim())} (file)`;
|
|
1520
1560
|
} else if (env.HTTPS_PROXY || env.https_proxy || env.HTTP_PROXY || env.http_proxy) {
|
|
1521
|
-
|
|
1561
|
+
const raw = env.HTTPS_PROXY || env.https_proxy || env.HTTP_PROXY || env.http_proxy;
|
|
1562
|
+
effective.proxy = `${maskUrlCredentials(raw)} (env)`;
|
|
1522
1563
|
}
|
|
1523
1564
|
if (config2.guards) {
|
|
1524
1565
|
const guards = {};
|
|
@@ -2575,15 +2616,16 @@ function runCommand(providerName, invocation, timeoutMs, describeFailure) {
|
|
|
2575
2616
|
let timedOut = false;
|
|
2576
2617
|
let settled = false;
|
|
2577
2618
|
let drainTimer;
|
|
2619
|
+
let killTimer;
|
|
2578
2620
|
const timer = setTimeout(() => {
|
|
2579
2621
|
timedOut = true;
|
|
2580
2622
|
child.kill("SIGTERM");
|
|
2581
2623
|
settle(null);
|
|
2582
|
-
setTimeout(() => {
|
|
2624
|
+
killTimer = setTimeout(() => {
|
|
2583
2625
|
if (!exited) {
|
|
2584
2626
|
child.kill("SIGKILL");
|
|
2585
2627
|
}
|
|
2586
|
-
}, SIGKILL_GRACE_MS)
|
|
2628
|
+
}, SIGKILL_GRACE_MS);
|
|
2587
2629
|
}, timeoutMs);
|
|
2588
2630
|
const settle = (code) => {
|
|
2589
2631
|
if (settled) {
|
|
@@ -2638,6 +2680,7 @@ function runCommand(providerName, invocation, timeoutMs, describeFailure) {
|
|
|
2638
2680
|
settled = true;
|
|
2639
2681
|
clearTimeout(timer);
|
|
2640
2682
|
clearTimeout(drainTimer);
|
|
2683
|
+
clearTimeout(killTimer);
|
|
2641
2684
|
if (error.code === "ENOENT") {
|
|
2642
2685
|
const missingCwd = !fs.existsSync(invocation.cwd);
|
|
2643
2686
|
reject(
|
|
@@ -2652,6 +2695,7 @@ function runCommand(providerName, invocation, timeoutMs, describeFailure) {
|
|
|
2652
2695
|
child.on("exit", (code) => {
|
|
2653
2696
|
exitCode = code;
|
|
2654
2697
|
exited = true;
|
|
2698
|
+
clearTimeout(killTimer);
|
|
2655
2699
|
restartDrain();
|
|
2656
2700
|
});
|
|
2657
2701
|
child.on("close", (code) => settle(code));
|
|
@@ -3737,7 +3781,7 @@ function parsePositiveInt(raw, flag) {
|
|
|
3737
3781
|
}
|
|
3738
3782
|
return Number.parseInt(raw, 10);
|
|
3739
3783
|
}
|
|
3740
|
-
program.name("modlens").description("Plug-in vision for text-only LLMs: image in, structured JSON evidence out").version("3.
|
|
3784
|
+
program.name("modlens").description("Plug-in vision for text-only LLMs: image in, structured JSON evidence out").version("3.16.0");
|
|
3741
3785
|
program.command("analyze", { isDefault: true }).description("Analyze an image into structured JSON evidence (default command)").requiredOption("-i, --input <path|url>", "Input image path or https URL").option("-o, --output <path>", "Write result JSON to a file").option("-m, --model <name>", "Provider model name").option("-p, --provider <name>", `Vision provider (${listProviders().join(", ")})`).option("--prompt <text>", "Extra focus for this image").option("--timeout <ms>", "Provider timeout in milliseconds", "180000").option("--provider-bin <path>", "Provider binary path (default: agy)").option("--workdir <path>", "Working directory for the provider").option(
|
|
3742
3786
|
"--extra-body <json>",
|
|
3743
3787
|
`JSON merged into the API request body, e.g. '{"thinking":{"type":"disabled"}}'`
|
|
@@ -3788,7 +3832,7 @@ program.command("recover-paste").description(
|
|
|
3788
3832
|
"Recover images pasted into Claude Code, Pi, or OpenCode from local session storage (they never hit disk otherwise)"
|
|
3789
3833
|
).option("--count <n>", "How many recent pasted images to recover", "1").option("--out-dir <path>", "Directory to write recovered images to").option(
|
|
3790
3834
|
"--session <id>",
|
|
3791
|
-
"Claude Code session id for exact targeting (skills get it via ${
|
|
3835
|
+
"Claude Code session id for exact targeting (skills get it via ${CLAUDE_CODE_SESSION_ID})"
|
|
3792
3836
|
).option("--transcript <path>", "Explicit transcript .jsonl or .db (overrides --session)").option(
|
|
3793
3837
|
"--harness <name>",
|
|
3794
3838
|
"Force the storage scope: claude-code, pi, opencode, or none (default: auto-detect via process ancestry and env)"
|
package/docs/cli.md
CHANGED
|
@@ -0,0 +1,98 @@
|
|
|
1
|
+
---
|
|
2
|
+
summary: 'CLI 手册:参数、故障转移链、guard 与 doctor 子命令、配置键'
|
|
3
|
+
read_when:
|
|
4
|
+
- 不经 skill 直接手动运行 CLI
|
|
5
|
+
- 查某个参数、默认模型或子命令
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# ModLens CLI 手册
|
|
9
|
+
|
|
10
|
+
[English](cli.md) | 中文
|
|
11
|
+
|
|
12
|
+
skill 通过它的启动器驱动这个 CLI。本页讲的是直接手动运行。
|
|
13
|
+
|
|
14
|
+
## 直接使用
|
|
15
|
+
|
|
16
|
+
装好 skill 后不需要敲命令:粘贴图片或给出路径,随便提问,它会自动触发。手动运行:
|
|
17
|
+
|
|
18
|
+
```bash
|
|
19
|
+
modlens -i screenshot.png # local image
|
|
20
|
+
modlens -i https://example.com/chart.png # remote image
|
|
21
|
+
modlens -i chart.png --prompt "focus on axes" # extra focus
|
|
22
|
+
modlens recover-paste # pull a pasted image into a file
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
输出是固定的 JSON 结构:
|
|
26
|
+
|
|
27
|
+
```json
|
|
28
|
+
{
|
|
29
|
+
"image": "/path/to/screenshot.png",
|
|
30
|
+
"provider": "gemini-api",
|
|
31
|
+
"result": {
|
|
32
|
+
"summary": "A workflow diagram with four nodes connected by labeled arrows.",
|
|
33
|
+
"ocr": { "full_text": "/shaping\nBEFORE YOU BUILD\n...", "lines": [] },
|
|
34
|
+
"layout": { "regions": [{ "reading_order": 1, "type": "title", "text": "/shaping" }] },
|
|
35
|
+
"semantics": { "scene": "workflow diagram", "entities": [], "relations": [] },
|
|
36
|
+
"visual": { "dominant_colors": ["white", "black"], "style": "flat", "notes": [] },
|
|
37
|
+
"uncertainty": []
|
|
38
|
+
},
|
|
39
|
+
"meta": {
|
|
40
|
+
"generatedAt": "2026-08-06T12:00:00.000Z",
|
|
41
|
+
"model": "gemini-3.6-flash",
|
|
42
|
+
"conversationId": null,
|
|
43
|
+
"durationSeconds": 6.4,
|
|
44
|
+
"usage": { "promptTokenCount": 1234, "candidatesTokenCount": 567 },
|
|
45
|
+
"attempts": [{ "provider": "gemini-api", "ok": true, "durationSeconds": 6.4 }],
|
|
46
|
+
"warnings": []
|
|
47
|
+
}
|
|
48
|
+
}
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
`meta` 记录结果是怎么产生的:什么时间(`generatedAt`)、用了哪个 `model`、provider 有会话时的 `conversationId`、实际耗时 `durationSeconds`,以及 provider 上报的原始 `usage`(结构因 provider 而异,没有时为 `null`)。`attempts` 按顺序列出故障转移链尝试过的每个 provider 和失败原因。`warnings` 携带路由通知(故障转移、被忽略的 extraBody、自动模式下这次识别花了谁的额度)。
|
|
52
|
+
|
|
53
|
+
## 参数
|
|
54
|
+
|
|
55
|
+
`modlens analyze`(默认命令):
|
|
56
|
+
|
|
57
|
+
| 参数 | 含义 | 默认值 |
|
|
58
|
+
| :-- | :-- | :-- |
|
|
59
|
+
| `-i, --input <path\|url>` | 要分析的图片(必填) | |
|
|
60
|
+
| `-p, --provider <name>` | 钉死单个 provider,不回退 | 故障转移链(见下) |
|
|
61
|
+
| `-m, --model <name>` | provider 的模型 | 按 provider 而定(见下) |
|
|
62
|
+
| `-o, --output <path>` | 同时把 JSON 写入文件 | |
|
|
63
|
+
| `--prompt <text>` | 额外关注点 | |
|
|
64
|
+
| `--timeout <ms>` | provider 超时 | `180000` |
|
|
65
|
+
| `--provider-bin <path>` | provider 可执行文件路径 | `agy` / `claude` |
|
|
66
|
+
| `--workdir <path>` | provider 的工作目录 | 每次运行新建的隔离目录 |
|
|
67
|
+
| `--extra-body <json>` | 合并进 API 请求体的 JSON,如 `'{"thinking":{"type":"disabled"}}'` | 配置里该 provider 的 `extraBody` |
|
|
68
|
+
|
|
69
|
+
`--extra-body` 是厂商专属开关的通道,最常见的用途是关掉 thinking。它作用于三个 API provider,并在该次运行中替换配置里的 `extraBody`。各厂商的具体写法和它拒绝改动的字段见[配置手册](../skills/modlens/references/configure.zh-CN.md)。
|
|
70
|
+
|
|
71
|
+
`-m` 的默认模型取决于 provider:
|
|
72
|
+
|
|
73
|
+
| Provider | 默认模型 |
|
|
74
|
+
| :-- | :-- |
|
|
75
|
+
| `antigravity-cli`(默认) | `gemini-3.6-flash-low` |
|
|
76
|
+
| `gemini-api` | `gemini-3.6-flash` |
|
|
77
|
+
| `anthropic` | `claude-haiku-4-5-20251001` |
|
|
78
|
+
| `claude-cli` | `haiku` |
|
|
79
|
+
| `openai` | 无,必须传 `-m` |
|
|
80
|
+
|
|
81
|
+
`modlens recover-paste`:
|
|
82
|
+
|
|
83
|
+
| 参数 | 含义 | 默认值 |
|
|
84
|
+
| :-- | :-- | :-- |
|
|
85
|
+
| `--count <n>` | 恢复最近几张粘贴的图片 | `1` |
|
|
86
|
+
| `--out-dir <path>` | 恢复出的图片写到哪里 | 每次运行新建的私有 `<tmpdir>/modlens-paste-*` |
|
|
87
|
+
| `--session <id>` | 用会话 id 精确定位 | 自动检测 |
|
|
88
|
+
| `--transcript <path>` | 显式指定 transcript 的 `.jsonl` 或 `.db`(覆盖 `--session`) | |
|
|
89
|
+
| `--harness <name>` | 强制指定存储范围:`claude-code`、`pi`、`opencode`、`none` | 自动检测 |
|
|
90
|
+
| `--cwd <path>` | 粘贴图片时所在的项目目录 | 当前目录 |
|
|
91
|
+
|
|
92
|
+
共五个 provider:`antigravity-cli`(免 key)、`gemini-api`(最快的免费通道)、`openai`(任意 OpenAI 兼容的多模态端点)、`anthropic`,以及 `claude-cli`(复用你现有的 Claude 订阅)。不带 `-p` 时,一次运行会依次尝试每个已配好的 provider:API 快车道(inline API provider,不启动 agent、直接调 API 的引擎)先试(5-10 秒),agent 类兜底,第一个可用结果胜出,其余尝试记录在 `meta.attempts` 里。通过 `reuse.<harness>` 授权的 harness 会把复用来的引擎补进相同的区段(pi 的凭据算快车道,agent CLI 排在后面),不会插到你自己引擎的前面。细节和 `guards` 的 deny/allow 名单见[配置手册](../skills/modlens/references/configure.zh-CN.md)。
|
|
93
|
+
|
|
94
|
+
其他子命令:
|
|
95
|
+
|
|
96
|
+
- `modlens guard [--model <id>]`:判断当前激活的模型到底该不该运行引擎。退出码 0 表示放行,1 表示拒绝,判定结果以 JSON 输出。
|
|
97
|
+
- `modlens config <init|set|show>`:可用的键有 `provider`、`proxy`(API provider 的 HTTP/HTTPS 代理,也认 `HTTPS_PROXY`/`HTTP_PROXY`)、`reuse.<claude|codex|opencode|pi|grok>`、`guards.<denyModels|allowModels|denyWhenUnknown>`,以及 `<provider>.<apiKey|baseUrl|model|proxy|extraBody>`。
|
|
98
|
+
- `modlens doctor`:报告 Node 与 node:sqlite、各 provider 的就绪状态、本机的故障转移链、检测到的 harness、guard 规则和一次现场判定,以及 Reuse 一节里按 harness 的授权决定与发现的视觉能力。不花任何额度,`--json` 输出机器可读报告。
|
package/docs/harness-setup.md
CHANGED
|
@@ -8,6 +8,8 @@ read_when:
|
|
|
8
8
|
|
|
9
9
|
# Harness setup
|
|
10
10
|
|
|
11
|
+
English | [中文](harness-setup.zh-CN.md)
|
|
12
|
+
|
|
11
13
|
Where a pasted image ends up differs per harness, and modlens takes a different route in each. `recover-paste` detects which harness it runs inside (process ancestry, then environment fingerprints) and reads only that harness's storage.
|
|
12
14
|
|
|
13
15
|
## Codex
|
|
@@ -70,8 +72,20 @@ OpenCode, and Claude Code hand their models, and the modlens skill's and
|
|
|
70
72
|
`read_image` tool's primary trigger. Admission never fires because the message
|
|
71
73
|
carries no image attachment.
|
|
72
74
|
|
|
73
|
-
The takeover is conditional
|
|
74
|
-
|
|
75
|
+
The takeover is conditional, and the decision is the host's: the browser half
|
|
76
|
+
asks the plugin's route whether the currently selected model is text-only,
|
|
77
|
+
and the host answers from the provider registry's declared model metadata
|
|
78
|
+
(`inputModalities`), not from a name heuristic. A `(modlens vision)` variant
|
|
79
|
+
or any model that declares image input keeps its native paste flow (variants
|
|
75
80
|
convert at request time with the thumbnail preserved; vision models read
|
|
76
|
-
images themselves)
|
|
77
|
-
|
|
81
|
+
images themselves), and so does any model the host cannot resolve. Pastes
|
|
82
|
+
stay native until the host has confirmed a takeover is right. A model whose
|
|
83
|
+
metadata declares no input modalities counts as unresolved: absent metadata is
|
|
84
|
+
never read as "confirmed text-only". Verdicts also age out (60s), so a route
|
|
85
|
+
whose models changed mid-session is re-asked, not trusted forever.
|
|
86
|
+
`pasteToPath: false` in the plugin row turns the whole feature off: the
|
|
87
|
+
browser half stands down when the policy endpoint 404s. If the route vanishes
|
|
88
|
+
mid-session after a verdict already confirmed it, the pastes made in the
|
|
89
|
+
brief window before the failed upload comes back (one local round-trip) are
|
|
90
|
+
lost. The client then forgets its verdicts and every later paste goes
|
|
91
|
+
native.
|
|
@@ -0,0 +1,67 @@
|
|
|
1
|
+
---
|
|
2
|
+
summary: '宿主接入:图片在 Codex、Claude Code、Pi、OpenCode 中如何抵达模型'
|
|
3
|
+
read_when:
|
|
4
|
+
- 在某个具体的编码 agent 里安装配置 modlens
|
|
5
|
+
- 粘贴的图片没有抵达模型
|
|
6
|
+
- 了解 recover-paste 在各 harness 里分别做什么
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# 宿主接入
|
|
10
|
+
|
|
11
|
+
[English](harness-setup.md) | 中文
|
|
12
|
+
|
|
13
|
+
粘贴的图片最终落在哪里,每个 harness 都不一样,modlens 在每个 harness 里走的路线也不同。`recover-paste` 会检测自己运行在哪个 harness 里(先看进程祖先,再看环境变量指纹),只读取该 harness 的存储。
|
|
14
|
+
|
|
15
|
+
## Codex
|
|
16
|
+
|
|
17
|
+
粘贴的图片会落成真实的临时文件,消息里带着形如 `<image name=[Image #1] path="/tmp/xxxx.png">` 的标签。skill 直接从标签里读出路径。`recover-paste` 检测到 Codex 后会拒绝执行,并把你指回这个标签。
|
|
18
|
+
|
|
19
|
+
纯文本模型有一个坑:一旦 `models.json` 声明了 `input_modalities: ["text"]`,Codex TUI 会直接拦下 Ctrl+V 粘贴。改为把文件拖进终端、手动输入路径,或使用 `codex exec -i image.png "..."`。
|
|
20
|
+
|
|
21
|
+
## Claude Code、Pi、OpenCode
|
|
22
|
+
|
|
23
|
+
这三家都不像 Codex 那样递给模型一个可用的临时文件路径(较新的 Claude Code 版本确实会把粘贴写进自己的 `~/.claude/image-cache/`,但只在终端入口以路径行的形式注入),不过三者都会在网关剥离图片之前,把用户消息完整存在本地:
|
|
24
|
+
|
|
25
|
+
| Harness | 存储位置 | 说明 |
|
|
26
|
+
| :-- | :-- | :-- |
|
|
27
|
+
| Claude Code | `~/.claude/projects/<slug>/<session>.jsonl` | 图片以 base64 存储。注入的 `CLAUDE_CODE_SESSION_ID` 可精确定位当前 session |
|
|
28
|
+
| Pi | `~/.pi/agent/sessions/--<encoded-cwd>--/*.jsonl` | 结构与 Claude Code 相同 |
|
|
29
|
+
| OpenCode | `~/.local/share/opencode/opencode.db` | SQLite,图片以 data URL 存储(通过 `node:sqlite` 读取) |
|
|
30
|
+
|
|
31
|
+
在 Claude Code 里通过 `ANTHROPIC_BASE_URL` 接入纯文本模型时,粘贴的图片要么变成一个不带路径的 `[Unsupported Image]` 占位符(宽松的网关),要么直接让请求报错([#62009](https://github.com/anthropics/claude-code/issues/62009))。图片字节并没有丢,`recover-paste` 取回的就是它。
|
|
32
|
+
|
|
33
|
+
## skill 的存放位置
|
|
34
|
+
|
|
35
|
+
| Harness | skill 读取位置 |
|
|
36
|
+
| :-- | :-- |
|
|
37
|
+
| Claude Code | `~/.claude/skills/` |
|
|
38
|
+
| Codex | `~/.codex/skills/` |
|
|
39
|
+
| Pi、OpenCode | `~/.agents/skills/` |
|
|
40
|
+
|
|
41
|
+
这些位置都支持符号链接,把 skill 目录链接一次,每个 agent 用的就都是最新版本。
|
|
42
|
+
|
|
43
|
+
## 平台支持
|
|
44
|
+
|
|
45
|
+
macOS 和 Linux 完整支持,并在 CI 上以 Node 22 和 24 验证。
|
|
46
|
+
|
|
47
|
+
Windows 跑同一套 CI 矩阵。那里没有 `ps`,检测会跳过进程祖先这一步,退回到上面的环境变量指纹,所以一个什么指纹都不设的 harness 会被判为未检出(用 `--harness` 或 `MODLENS_HARNESS` 强制指定)。OpenCode 的粘贴恢复在 Windows 上有覆盖,包括 [#11](https://github.com/liustack/modlens/issues/11) 里的路径分隔符归一化:opencode 记录的 `session.directory` 用正斜杠,而那里的 `path.resolve` 返回反斜杠,匹配前两边都会归一化。JSONL 存储(Claude Code、Pi)以 `os.homedir()` 和各 harness 自己的磁盘 slug 为键,在 POSIX 上验证。外部引擎(Antigravity CLI、Claude CLI)只在有 Windows 版本的平台上运行。
|
|
48
|
+
|
|
49
|
+
## 网关配置
|
|
50
|
+
|
|
51
|
+
OpenCode 接 DeepSeek:执行 `opencode auth login`,选择 DeepSeek 并粘贴 key(会存进 `~/.local/share/opencode/auth.json`),然后在 `~/.config/opencode/opencode.jsonc` 里把默认模型设为 `deepseek/deepseek-v4-flash`。Pi 从 `~/.pi/agent/auth.json` 读取它的 key。
|
|
52
|
+
|
|
53
|
+
## DeepSeek Harness(dsh)
|
|
54
|
+
|
|
55
|
+
dsh 与其他 harness 不同:modlens 以原生工具的形式接入,而不是靠提示词触发的 skill。本包自身就是一个 dsh bundle,一条命令即可装进某个 profile:
|
|
56
|
+
|
|
57
|
+
```sh
|
|
58
|
+
npx -y @deepseek-ai/dsh plugin --profile web add @liustack/modlens@latest
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
这会注册一个 `read_image` 工具,它的 schema 随每次请求抵达模型(不靠触发启发式),运行同一个包里自带的 modlens CLI,并把结构化证据作为工具的标准 JSON 输出返回。引擎、复用授权和 guard 规则仍在 `~/.modlens/config.json` 里,与其他所有 harness 共享。dsh 还在开发者预览阶段,插件接口可能变化。这个插件刻意保持很小的接触面(原生工具注册、视觉变体所用的 llm 适配层、附件读取器,以及一个 agent 执行前钩子),其中任何一处变动,它都会大声报错而不是无声退化。
|
|
62
|
+
|
|
63
|
+
### 粘贴转路径(paste-to-path,web profile)
|
|
64
|
+
|
|
65
|
+
过去在 dsh Web UI 里,**纯文本模型**下粘贴图片会死在图片准入检查这一步。插件现在带了一个浏览器端半边(由 dsh 的客户端插件系统自动加载),恰好在这种情况下接管粘贴:图片字节发到插件在 dsh web 服务器上的 `/modlens/paste` 路由(仅回环地址,校验 magic byte,上限 25 MB),落成一个私有临时文件,输入框收到的则是纯文本的文件路径。这与 Pi、OpenCode、Claude Code 递给模型的形态一致,也正是 modlens skill 和 `read_image` 工具的首要触发条件。消息里不带图片附件,准入检查根本不会触发。
|
|
66
|
+
|
|
67
|
+
接管是有条件的,且裁决权在 host 一侧:浏览器半边先向插件路由询问当前选中的模型是否纯文本,host 用 provider 注册表里声明的模型元数据(`inputModalities`)回答,而不是靠名称猜。`(modlens vision)` 变体和任何声明支持图片输入的模型都保留原生粘贴流程(变体在发请求时转换且保留缩略图,视觉模型自己读图),host 认不出的模型同样不接管:在 host 确认该接管之前,粘贴一律走原生路径。模型元数据里没有声明输入模态的,一律算认不出:元数据缺失绝不当成「已确认纯文本」。裁决还有 60 秒时效,模型中途变了会重新问询,不会永远信旧答案。在插件配置行里设 `pasteToPath: false` 可整体关掉这个功能:策略端点 404 时浏览器半边彻底停手。若路由在裁决确认后中途消失,失败结果返回前那个短暂窗口(一次本地往返)内发生的粘贴会丢失,之后客户端清空全部裁决,后续粘贴一律走原生路径。
|