@liustack/modlens 3.13.0 → 3.15.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +9 -0
- package/README.md +44 -0
- package/README.zh-CN.md +56 -12
- package/dist/main.js +2 -2
- package/docs/cli.md +2 -0
- package/docs/cli.zh-CN.md +98 -0
- package/docs/harness-setup.md +20 -0
- package/docs/harness-setup.zh-CN.md +67 -0
- package/docs/output-schema.md +2 -0
- package/docs/output-schema.zh-CN.md +74 -0
- package/docs/security.md +2 -0
- package/docs/security.zh-CN.md +43 -0
- package/docs/troubleshooting.md +2 -0
- package/docs/troubleshooting.zh-CN.md +210 -0
- package/dsh/client.js +126 -0
- package/dsh/index.js +241 -55
- package/package.json +83 -77
- package/skills/modlens/SKILL.md +4 -4
- package/skills/modlens/references/configure.md +2 -0
- package/skills/modlens/references/configure.zh-CN.md +177 -0
- package/skills/modlens/references/runtime.md +1 -1
- package/skills/modlens/scripts/run.ps1 +1 -1
- package/skills/modlens/scripts/run.sh +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,14 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 3.15.0 - 2026-08-14
|
|
4
|
+
|
|
5
|
+
- **dsh: the vision wrap discovers every eligible provider route ([#29](https://github.com/liustack/modlens/issues/29)).** The wrapper used to bind one hardcoded upstream, so a machine with several subscription packages (opencode-go, zai, ...) had to hand-pick a single route and lose the rest. With `upstream` unset the plugin now sweeps the llm registry and registers a `modlens-<provider>` wrapper for every route carrying wrappable text-only DeepSeek/GLM models, re-sweeping on the registry's own `llm/adapters-updated` notification so late-registering routes (llm-pi-ai mounts after settings load) are picked up without polling. A `discover` array narrows the set, `families` filters as before, setting `upstream` keeps exact legacy single-route behavior, wrappers never wrap themselves, and the deepseek-official wrap keeps its historical `deepseek-modlens` id so an upgraded machine's model selection survives. Design and behavior spec contributed by @zlycode01, who also published a reference implementation in a public fork; this is an independent upstream implementation of that spec, with the polling schedule replaced by the registry notification.
|
|
6
|
+
|
|
7
|
+
## 3.14.0 - 2026-08-14
|
|
8
|
+
|
|
9
|
+
- **dsh: pasting into a text-only model now just works — the paste becomes a file path.** The plugin grows a browser half (a hand-written bundle in dsh's client plugin protocol, zero dependencies, loaded automatically under the web profile). A capture-phase listener takes over image pastes before the composer's own intake: the bytes go to the plugin's `/modlens/paste` route on the dsh web server (loopback-bound, magic-byte checked, 25 MB cap, private 0600 temp file), and the composer receives the file path as plain text — the exact shape Pi, OpenCode, and Claude Code hand their models, and the modlens skill's primary trigger. Image admission never fires because the message carries no image attachment; verified end to end with a native text-only DeepSeek-V4-Flash, whose visible reasoning quoted the modlens skill rule and went for the path. The takeover is conditional: `(modlens vision)` variants and known vision models keep the native paste flow (thumbnails and all), and `pasteToPath: false` turns the feature off. The host route rides a scoped `ctx.inject(['webServer'], ...)`, so headless profiles never see any of it.
|
|
10
|
+
- **The CLI survives Electron hosts ([#25](https://github.com/liustack/modlens/issues/25)).** In the packaged dsh desktop app, `process.execPath` is the Electron binary, and commander's Electron auto-detection then mis-slices argv so the script path lands as a stray positional (`too many arguments for 'analyze'`). The CLI now parses argv with explicit node semantics — it is always spawned script-first, whatever binary hosts it — and the plugin's spawns set `ELECTRON_RUN_AS_NODE` for good measure. Thanks to @hi-fangj for tracing it into commander's `_prepareUserArgs`.
|
|
11
|
+
|
|
3
12
|
## 3.13.0 - 2026-08-14
|
|
4
13
|
|
|
5
14
|
- **Proxy support actually works now ([#23](https://github.com/liustack/modlens/issues/23)).** 3.12.0's proxy path was broken on arrival, twice over: the bundled copy of undici had its internal `node:http2` references destroyed by bundling (the embedded `ProxyAgent` threw `http2.connect is not a function`), and handing any undici 8 dispatcher to the host's built-in fetch (a different undici major) fails with `UND_ERR_INVALID_ARG` regardless. undici is no longer bundled — it resolves from `node_modules`, shrinking the CLI bundle from 1.17 MB to 131 KB — and the proxied path now uses undici's own fetch so dispatcher and fetch are same-sourced, with the dispatcher closed after the response so its keep-alive pool cannot pin the process open. A new integration test drives the built CLI through a real local HTTP proxy, the exact coverage whose absence let 3.12.0 ship broken (and whose first draft repeated a classic mistake: `spawnSync` freezes the test's own fake servers, so the CLI must be spawned async); both the env-var and explicit-setting forms were also verified against a real LAN proxy. Independent review of the fix then caught the same cross-version boundary hiding in the no-proxy remote-image path — the IP-pinned download `Agent` was still handed to the host's fetch — so that path is now same-sourced too, and the Node floor rises to 22.19 (undici 8's own engine requirement, which externalizing made load-bearing). Thanks to @JooJeen for a diagnosis that had already isolated both layers.
|
package/README.md
CHANGED
|
@@ -63,6 +63,50 @@ The install also inventories vision reachable through your other local harness C
|
|
|
63
63
|
|
|
64
64
|
Once installed, just chat. Paste an image or drop a path, ask anything, and the skill triggers on its own: the image goes to a vision engine and the answer comes back grounded in what it read.
|
|
65
65
|
|
|
66
|
+
## Vision engines: five built-in providers, four reusable CLIs, one failover chain
|
|
67
|
+
|
|
68
|
+
ModLens does not depend on any single vision service. Nine sources of vision in total: five built-in providers, any one of which is enough, plus four local agent CLIs whose logins can be reused. The built-ins:
|
|
69
|
+
|
|
70
|
+
| Provider | What it needs | Speed per read | Good for |
|
|
71
|
+
| :-- | :-- | :-- | :-- |
|
|
72
|
+
| `gemini-api` | a free Gemini API key ([3 minutes, no card](https://aistudio.google.com)) | 5-10s | the recommended default |
|
|
73
|
+
| `openai` | any OpenAI-compatible endpoint (key + baseUrl + model) | 5-10s | qwen-vl, GLM, self-hosted gateways |
|
|
74
|
+
| `anthropic` | an Anthropic API key | 5-10s | machines already holding one |
|
|
75
|
+
| `antigravity-cli` | the free `agy` CLI, one browser sign-in, no key | 15-45s | zero-signup starts |
|
|
76
|
+
| `claude-cli` | a signed-in Claude Code | 20-45s | riding your existing Claude subscription |
|
|
77
|
+
|
|
78
|
+
Without a pinned provider, every configured engine forms one failover chain: the fast API providers try first, the agent CLIs back them up, the first good result wins, and `meta.attempts` records every attempt so a fallback is never silent.
|
|
79
|
+
|
|
80
|
+
### `openai` is a universal socket, not just OpenAI
|
|
81
|
+
|
|
82
|
+
Any endpoint speaking the OpenAI chat-completions protocol with image input plugs straight in — that covers most of the vision-model world:
|
|
83
|
+
|
|
84
|
+
```bash
|
|
85
|
+
modlens config set openai.baseUrl https://dashscope.aliyuncs.com/compatible-mode/v1 # qwen-vl
|
|
86
|
+
modlens config set openai.apiKey <key>
|
|
87
|
+
modlens config set openai.model qwen3-vl-plus
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
The same three keys work for GLM's open platform, SiliconFlow, OpenRouter, a self-hosted vLLM/Ollama, or any gateway of your own. If your favorite vision model has an OpenAI-compatible API, ModLens can drive it.
|
|
91
|
+
|
|
92
|
+
### Reusing what your machine already has
|
|
93
|
+
|
|
94
|
+
Two more sources of vision need zero new keys, each behind one explicit consent recorded in config:
|
|
95
|
+
|
|
96
|
+
- **The harness you are talking in right now.** Running inside Claude Code with a subscription signed in? `claude-cli` reads images through it out of the box. The install flow asks the same question for whichever harness you install into.
|
|
97
|
+
- **Every other agent CLI on the machine.** `modlens doctor` discovers them, you grant per harness, and they join the same failover chain with no priority over your own keys. Every reused read is labeled in `meta.warnings` with whose quota it spent, so nothing is ever silently billed:
|
|
98
|
+
|
|
99
|
+
| Reused CLI | What it needs | Grant with | Rides as |
|
|
100
|
+
| :-- | :-- | :-- | :-- |
|
|
101
|
+
| Codex | a signed-in Codex CLI with a vision model | `config set reuse.codex true` | agent lane, 15-45s |
|
|
102
|
+
| OpenCode | a vision model configured in OpenCode | `config set reuse.opencode true` | agent lane, 15-45s |
|
|
103
|
+
| Pi | model credentials held by Pi | `config set reuse.pi true` | an API key upgrades to the 5-10s inline lane, OAuth drives Pi itself |
|
|
104
|
+
| Grok | a signed-in Grok CLI (SuperGrok) | `config set reuse.grok true` | agent lane, 15-45s |
|
|
105
|
+
|
|
106
|
+
### Picking and routing
|
|
107
|
+
|
|
108
|
+
Two knobs: `modlens config set provider <name>` states a preference (the chain still backs it up), `-p <name>` pins exactly one with no fallback. Machines behind a proxy set `HTTPS_PROXY` or `modlens config set proxy <url>` and the API providers route through it. Details: the [CLI manual](docs/cli.md) for defaults and flags, [Configuration](skills/modlens/references/configure.md) for every key, and [Security](docs/security.md) for who fetches what on remote URLs.
|
|
109
|
+
|
|
66
110
|
## See it work
|
|
67
111
|
|
|
68
112
|
Unedited runs, all driving a text-only DeepSeek-V4-Flash.
|
package/README.zh-CN.md
CHANGED
|
@@ -10,10 +10,10 @@
|
|
|
10
10
|
|
|
11
11
|
<p align="center">
|
|
12
12
|
<a href="./README.md">English</a> ·
|
|
13
|
-
<a href="docs/troubleshooting.md">故障排查</a> ·
|
|
14
|
-
<a href="skills/modlens/references/configure.md">配置</a> ·
|
|
15
|
-
<a href="docs/output-schema.md">输出契约</a> ·
|
|
16
|
-
<a href="docs/security.md">安全</a> ·
|
|
13
|
+
<a href="docs/troubleshooting.zh-CN.md">故障排查</a> ·
|
|
14
|
+
<a href="skills/modlens/references/configure.zh-CN.md">配置</a> ·
|
|
15
|
+
<a href="docs/output-schema.zh-CN.md">输出契约</a> ·
|
|
16
|
+
<a href="docs/security.zh-CN.md">安全</a> ·
|
|
17
17
|
<a href="https://github.com/liustack/modsearch">ModSearch(联网)</a>
|
|
18
18
|
</p>
|
|
19
19
|
|
|
@@ -34,7 +34,7 @@ DeepSeek 和 GLM 没有视觉能力,无法进行图片识别。ModLens 借助
|
|
|
34
34
|
|
|
35
35
|
## 亮点
|
|
36
36
|
|
|
37
|
-
**🥇 全网第一个支持 DeepSeek Harness(dsh)的外挂视觉识别插件:**一条命令 `npx -y @deepseek-ai/dsh plugin --profile web add @liustack/modlens@latest`,dsh 背后的纯文本 DeepSeek 模型即可通过原生 `read_image` 工具读图。如果 dsh 提示 `declares no dsh.bundle`,是 pnpm 的发布冷静期把版本压旧了,一行配置可解,见[故障排查](docs/troubleshooting.md#dsh
|
|
37
|
+
**🥇 全网第一个支持 DeepSeek Harness(dsh)的外挂视觉识别插件:**一条命令 `npx -y @deepseek-ai/dsh plugin --profile web add @liustack/modlens@latest`,dsh 背后的纯文本 DeepSeek 模型即可通过原生 `read_image` 工具读图。如果 dsh 提示 `declares no dsh.bundle`,是 pnpm 的发布冷静期把版本压旧了,一行配置可解,见[故障排查](docs/troubleshooting.zh-CN.md#dsh-提示-declares-no-dshbundle--installed-as-a-plain-dependency)。要粘贴识图,把模型选择器切到插件新增的两个条目之一:**`DeepSeek-V4-Flash (modlens vision)`** 或 **`DeepSeek-V4-Pro (modlens vision)`**,贴图放行、发请求时转成证据(你的消息保留原生缩略图)、仍由原 DeepSeek 路由回答。包装只覆盖 DeepSeek 与 GLM 的文本模型,两家自己的视觉型号自动排除。
|
|
38
38
|
|
|
39
39
|
**直接粘贴图片识别** 无需先保存成文件再提供路径。
|
|
40
40
|
|
|
@@ -65,12 +65,56 @@ agy # 浏览器完成
|
|
|
65
65
|
npx -y @deepseek-ai/dsh plugin --profile web add @liustack/modlens@latest
|
|
66
66
|
```
|
|
67
67
|
|
|
68
|
-
装完即有 `read_image` 工具,选「(modlens vision)」模型变体即可直接粘贴识图。引擎配置同样在 `~/.modlens`,详见[宿主接入](docs/harness-setup.md)。
|
|
68
|
+
装完即有 `read_image` 工具,选「(modlens vision)」模型变体即可直接粘贴识图。引擎配置同样在 `~/.modlens`,详见[宿主接入](docs/harness-setup.zh-CN.md)。
|
|
69
69
|
|
|
70
70
|
## 用法
|
|
71
71
|
|
|
72
72
|
装好之后不需要记任何命令。正常聊天,粘贴图片或给出图片路径,提问即可,skill 自动触发:图片交给视觉引擎,答案基于读到的内容返回。
|
|
73
73
|
|
|
74
|
+
## 视觉引擎:五个内置 provider,四家可复用 CLI,一条故障转移链
|
|
75
|
+
|
|
76
|
+
ModLens 不绑定任何单一视觉服务。视觉来源一共九个:五个内置 provider(配好任意一个就能用),加四家本机 agent CLI 的登录可以复用。先看内置的:
|
|
77
|
+
|
|
78
|
+
| Provider | 需要什么 | 单次识别耗时 | 适合谁 |
|
|
79
|
+
| :-- | :-- | :-- | :-- |
|
|
80
|
+
| `gemini-api` | 免费 Gemini key([三分钟领取,无需信用卡](https://aistudio.google.com)) | 5-10 秒 | 推荐默认 |
|
|
81
|
+
| `openai` | 任意 OpenAI 兼容端点(key + baseUrl + model) | 5-10 秒 | qwen-vl、GLM、自建网关 |
|
|
82
|
+
| `anthropic` | Anthropic API key | 5-10 秒 | 手上已有 key 的机器 |
|
|
83
|
+
| `antigravity-cli` | 免费的 `agy` CLI,浏览器登录一次,无需 key | 15-45 秒 | 完全免注册起步 |
|
|
84
|
+
| `claude-cli` | 已登录的 Claude Code | 20-45 秒 | 复用现有 Claude 订阅 |
|
|
85
|
+
|
|
86
|
+
不钉死 provider 时,所有配好的引擎组成一条故障转移链:API 快车道先试,agent CLI 兜底,第一个可用结果胜出,`meta.attempts` 记录每次尝试,回退永远不是无声的。
|
|
87
|
+
|
|
88
|
+
### `openai` 是万能接口,不只是 OpenAI
|
|
89
|
+
|
|
90
|
+
任何讲 OpenAI chat-completions 协议、支持图片输入的端点都能直接插上,这基本覆盖了视觉模型的大半个世界:
|
|
91
|
+
|
|
92
|
+
```bash
|
|
93
|
+
modlens config set openai.baseUrl https://dashscope.aliyuncs.com/compatible-mode/v1 # qwen-vl
|
|
94
|
+
modlens config set openai.apiKey <key>
|
|
95
|
+
modlens config set openai.model qwen3-vl-plus
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
同样三个键,换成 GLM 开放平台、SiliconFlow、OpenRouter、自建 vLLM/Ollama 或你自己的网关都一样。你常用的视觉模型只要有 OpenAI 兼容 API,ModLens 就能驱动它。
|
|
99
|
+
|
|
100
|
+
### 复用你机器上已有的东西
|
|
101
|
+
|
|
102
|
+
还有两处现成的视觉能力,一个新 key 都不用配,每家都在你明确同意后才启用:
|
|
103
|
+
|
|
104
|
+
- **你正在对话的这个 harness 本身。**在登录了订阅的 Claude Code 里用?`claude-cli` 开箱即可借它读图。装进哪个 harness,安装流程就会问哪个 harness 的授权。
|
|
105
|
+
- **机器上其他的 agent CLI。**`modlens doctor` 会逐个发现,你按家授权,它们与你自己的 key 平级入链,不插队。每次复用都在 `meta.warnings` 里标明花的是谁的额度,绝不无声扣费:
|
|
106
|
+
|
|
107
|
+
| 复用来源 | 需要什么 | 授权命令 | 走哪条道 |
|
|
108
|
+
| :-- | :-- | :-- | :-- |
|
|
109
|
+
| Codex | 已登录且有视觉模型的 Codex CLI | `config set reuse.codex true` | agent 通道,15-45 秒 |
|
|
110
|
+
| OpenCode | OpenCode 里配好的视觉模型 | `config set reuse.opencode true` | agent 通道,15-45 秒 |
|
|
111
|
+
| Pi | Pi 持有的模型凭据 | `config set reuse.pi true` | API key 直接升级到 5-10 秒的快车道,OAuth 驱动 Pi 本体 |
|
|
112
|
+
| Grok | 已登录的 Grok CLI(SuperGrok) | `config set reuse.grok true` | agent 通道,15-45 秒 |
|
|
113
|
+
|
|
114
|
+
### 选择与路由
|
|
115
|
+
|
|
116
|
+
两个旋钮:`modlens config set provider <name>` 表达偏好(链继续兜底),`-p <name>` 钉死单个不回退。代理环境设 `HTTPS_PROXY` 或 `modlens config set proxy <url>`,API provider 自动走代理。细节见 [CLI 手册](docs/cli.zh-CN.md)(默认模型与参数)、[配置手册](skills/modlens/references/configure.zh-CN.md)(全部配置键)、[安全说明](docs/security.zh-CN.md)(远程 URL 由谁抓取)。
|
|
117
|
+
|
|
74
118
|
## 实测
|
|
75
119
|
|
|
76
120
|
以下均为原样记录,驱动的都是纯文本的 DeepSeek-V4-Flash。
|
|
@@ -100,12 +144,12 @@ Codex 桌面 App 中识别一张推文截图。作者、配文、照片内容(
|
|
|
100
144
|
| 文档 | 适用场景 |
|
|
101
145
|
| :------------------------------------------------- | :----------------------------------------- |
|
|
102
146
|
| [安装手册](INSTALL.md) | 一步步安装 skill(为 agent 编写) |
|
|
103
|
-
| [CLI 手册](docs/cli.md) | skill 所驱动的 CLI:参数、配置与体检 |
|
|
104
|
-
| [故障排查](docs/troubleshooting.md) | 命令报错,查成因和解法 |
|
|
105
|
-
| [配置手册](skills/modlens/references/configure.md) | 配置 key、切换 provider、排查配置 |
|
|
106
|
-
| [输出契约](docs/output-schema.md) | 解析 JSON 或构建下游工具 |
|
|
107
|
-
| [宿主接入](docs/harness-setup.md) | 在 Codex、Claude Code、Pi、OpenCode 中配置 |
|
|
108
|
-
| [安全说明](docs/security.md) | 恢复文件的权限、图片内容作为不可信输入 |
|
|
147
|
+
| [CLI 手册](docs/cli.zh-CN.md) | skill 所驱动的 CLI:参数、配置与体检 |
|
|
148
|
+
| [故障排查](docs/troubleshooting.zh-CN.md) | 命令报错,查成因和解法 |
|
|
149
|
+
| [配置手册](skills/modlens/references/configure.zh-CN.md) | 配置 key、切换 provider、排查配置 |
|
|
150
|
+
| [输出契约](docs/output-schema.zh-CN.md) | 解析 JSON 或构建下游工具 |
|
|
151
|
+
| [宿主接入](docs/harness-setup.zh-CN.md) | 在 Codex、Claude Code、Pi、OpenCode 中配置 |
|
|
152
|
+
| [安全说明](docs/security.zh-CN.md) | 恢复文件的权限、图片内容作为不可信输入 |
|
|
109
153
|
| [更新日志](CHANGELOG.md) | 查询版本变更 |
|
|
110
154
|
|
|
111
155
|
## 参与方式
|
package/dist/main.js
CHANGED
|
@@ -3737,7 +3737,7 @@ function parsePositiveInt(raw, flag) {
|
|
|
3737
3737
|
}
|
|
3738
3738
|
return Number.parseInt(raw, 10);
|
|
3739
3739
|
}
|
|
3740
|
-
program.name("modlens").description("Plug-in vision for text-only LLMs: image in, structured JSON evidence out").version("3.
|
|
3740
|
+
program.name("modlens").description("Plug-in vision for text-only LLMs: image in, structured JSON evidence out").version("3.15.0");
|
|
3741
3741
|
program.command("analyze", { isDefault: true }).description("Analyze an image into structured JSON evidence (default command)").requiredOption("-i, --input <path|url>", "Input image path or https URL").option("-o, --output <path>", "Write result JSON to a file").option("-m, --model <name>", "Provider model name").option("-p, --provider <name>", `Vision provider (${listProviders().join(", ")})`).option("--prompt <text>", "Extra focus for this image").option("--timeout <ms>", "Provider timeout in milliseconds", "180000").option("--provider-bin <path>", "Provider binary path (default: agy)").option("--workdir <path>", "Working directory for the provider").option(
|
|
3742
3742
|
"--extra-body <json>",
|
|
3743
3743
|
`JSON merged into the API request body, e.g. '{"thinking":{"type":"disabled"}}'`
|
|
@@ -3904,4 +3904,4 @@ config.command("show").description("Print the effective config (file merged with
|
|
|
3904
3904
|
process.exitCode = 1;
|
|
3905
3905
|
}
|
|
3906
3906
|
});
|
|
3907
|
-
await program.parseAsync();
|
|
3907
|
+
await program.parseAsync(process.argv, { from: "node" });
|
package/docs/cli.md
CHANGED
|
@@ -0,0 +1,98 @@
|
|
|
1
|
+
---
|
|
2
|
+
summary: 'CLI 手册:参数、故障转移链、guard 与 doctor 子命令、配置键'
|
|
3
|
+
read_when:
|
|
4
|
+
- 不经 skill 直接手动运行 CLI
|
|
5
|
+
- 查某个参数、默认模型或子命令
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# ModLens CLI 手册
|
|
9
|
+
|
|
10
|
+
[English](cli.md) | 中文
|
|
11
|
+
|
|
12
|
+
skill 通过它的启动器驱动这个 CLI。本页讲的是直接手动运行。
|
|
13
|
+
|
|
14
|
+
## 直接使用
|
|
15
|
+
|
|
16
|
+
装好 skill 后不需要敲命令:粘贴图片或给出路径,随便提问,它会自动触发。手动运行:
|
|
17
|
+
|
|
18
|
+
```bash
|
|
19
|
+
modlens -i screenshot.png # local image
|
|
20
|
+
modlens -i https://example.com/chart.png # remote image
|
|
21
|
+
modlens -i chart.png --prompt "focus on axes" # extra focus
|
|
22
|
+
modlens recover-paste # pull a pasted image into a file
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
输出是固定的 JSON 结构:
|
|
26
|
+
|
|
27
|
+
```json
|
|
28
|
+
{
|
|
29
|
+
"image": "/path/to/screenshot.png",
|
|
30
|
+
"provider": "gemini-api",
|
|
31
|
+
"result": {
|
|
32
|
+
"summary": "A workflow diagram with four nodes connected by labeled arrows.",
|
|
33
|
+
"ocr": { "full_text": "/shaping\nBEFORE YOU BUILD\n...", "lines": [] },
|
|
34
|
+
"layout": { "regions": [{ "reading_order": 1, "type": "title", "text": "/shaping" }] },
|
|
35
|
+
"semantics": { "scene": "workflow diagram", "entities": [], "relations": [] },
|
|
36
|
+
"visual": { "dominant_colors": ["white", "black"], "style": "flat", "notes": [] },
|
|
37
|
+
"uncertainty": []
|
|
38
|
+
},
|
|
39
|
+
"meta": {
|
|
40
|
+
"generatedAt": "2026-08-06T12:00:00.000Z",
|
|
41
|
+
"model": "gemini-3.6-flash",
|
|
42
|
+
"conversationId": null,
|
|
43
|
+
"durationSeconds": 6.4,
|
|
44
|
+
"usage": { "promptTokenCount": 1234, "candidatesTokenCount": 567 },
|
|
45
|
+
"attempts": [{ "provider": "gemini-api", "ok": true, "durationSeconds": 6.4 }],
|
|
46
|
+
"warnings": []
|
|
47
|
+
}
|
|
48
|
+
}
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
`meta` 记录结果是怎么产生的:什么时间(`generatedAt`)、用了哪个 `model`、provider 有会话时的 `conversationId`、实际耗时 `durationSeconds`,以及 provider 上报的原始 `usage`(结构因 provider 而异,没有时为 `null`)。`attempts` 按顺序列出故障转移链尝试过的每个 provider 和失败原因。`warnings` 携带路由通知(故障转移、被忽略的 extraBody、自动模式下这次识别花了谁的额度)。
|
|
52
|
+
|
|
53
|
+
## 参数
|
|
54
|
+
|
|
55
|
+
`modlens analyze`(默认命令):
|
|
56
|
+
|
|
57
|
+
| 参数 | 含义 | 默认值 |
|
|
58
|
+
| :-- | :-- | :-- |
|
|
59
|
+
| `-i, --input <path\|url>` | 要分析的图片(必填) | |
|
|
60
|
+
| `-p, --provider <name>` | 钉死单个 provider,不回退 | 故障转移链(见下) |
|
|
61
|
+
| `-m, --model <name>` | provider 的模型 | 按 provider 而定(见下) |
|
|
62
|
+
| `-o, --output <path>` | 同时把 JSON 写入文件 | |
|
|
63
|
+
| `--prompt <text>` | 额外关注点 | |
|
|
64
|
+
| `--timeout <ms>` | provider 超时 | `180000` |
|
|
65
|
+
| `--provider-bin <path>` | provider 可执行文件路径 | `agy` / `claude` |
|
|
66
|
+
| `--workdir <path>` | provider 的工作目录 | 每次运行新建的隔离目录 |
|
|
67
|
+
| `--extra-body <json>` | 合并进 API 请求体的 JSON,如 `'{"thinking":{"type":"disabled"}}'` | 配置里该 provider 的 `extraBody` |
|
|
68
|
+
|
|
69
|
+
`--extra-body` 是厂商专属开关的通道,最常见的用途是关掉 thinking。它作用于三个 API provider,并在该次运行中替换配置里的 `extraBody`。各厂商的具体写法和它拒绝改动的字段见[配置手册](../skills/modlens/references/configure.zh-CN.md)。
|
|
70
|
+
|
|
71
|
+
`-m` 的默认模型取决于 provider:
|
|
72
|
+
|
|
73
|
+
| Provider | 默认模型 |
|
|
74
|
+
| :-- | :-- |
|
|
75
|
+
| `antigravity-cli`(默认) | `gemini-3.6-flash-low` |
|
|
76
|
+
| `gemini-api` | `gemini-3.6-flash` |
|
|
77
|
+
| `anthropic` | `claude-haiku-4-5-20251001` |
|
|
78
|
+
| `claude-cli` | `haiku` |
|
|
79
|
+
| `openai` | 无,必须传 `-m` |
|
|
80
|
+
|
|
81
|
+
`modlens recover-paste`:
|
|
82
|
+
|
|
83
|
+
| 参数 | 含义 | 默认值 |
|
|
84
|
+
| :-- | :-- | :-- |
|
|
85
|
+
| `--count <n>` | 恢复最近几张粘贴的图片 | `1` |
|
|
86
|
+
| `--out-dir <path>` | 恢复出的图片写到哪里 | 每次运行新建的私有 `<tmpdir>/modlens-paste-*` |
|
|
87
|
+
| `--session <id>` | 用会话 id 精确定位 | 自动检测 |
|
|
88
|
+
| `--transcript <path>` | 显式指定 transcript 的 `.jsonl` 或 `.db`(覆盖 `--session`) | |
|
|
89
|
+
| `--harness <name>` | 强制指定存储范围:`claude-code`、`pi`、`opencode`、`none` | 自动检测 |
|
|
90
|
+
| `--cwd <path>` | 粘贴图片时所在的项目目录 | 当前目录 |
|
|
91
|
+
|
|
92
|
+
共五个 provider:`antigravity-cli`(免 key)、`gemini-api`(最快的免费通道)、`openai`(任意 OpenAI 兼容的多模态端点)、`anthropic`,以及 `claude-cli`(复用你现有的 Claude 订阅)。不带 `-p` 时,一次运行会依次尝试每个已配好的 provider:API 快车道(inline API provider,不启动 agent、直接调 API 的引擎)先试(5-10 秒),agent 类兜底,第一个可用结果胜出,其余尝试记录在 `meta.attempts` 里。通过 `reuse.<harness>` 授权的 harness 会把复用来的引擎补进相同的区段(pi 的凭据算快车道,agent CLI 排在后面),不会插到你自己引擎的前面。细节和 `guards` 的 deny/allow 名单见[配置手册](../skills/modlens/references/configure.zh-CN.md)。
|
|
93
|
+
|
|
94
|
+
其他子命令:
|
|
95
|
+
|
|
96
|
+
- `modlens guard [--model <id>]`:判断当前激活的模型到底该不该运行引擎。退出码 0 表示放行,1 表示拒绝,判定结果以 JSON 输出。
|
|
97
|
+
- `modlens config <init|set|show>`:可用的键有 `provider`、`proxy`(API provider 的 HTTP/HTTPS 代理,也认 `HTTPS_PROXY`/`HTTP_PROXY`)、`reuse.<claude|codex|opencode|pi|grok>`、`guards.<denyModels|allowModels|denyWhenUnknown>`,以及 `<provider>.<apiKey|baseUrl|model|proxy|extraBody>`。
|
|
98
|
+
- `modlens doctor`:报告 Node 与 node:sqlite、各 provider 的就绪状态、本机的故障转移链、检测到的 harness、guard 规则和一次现场判定,以及 Reuse 一节里按 harness 的授权决定与发现的视觉能力。不花任何额度,`--json` 输出机器可读报告。
|
package/docs/harness-setup.md
CHANGED
|
@@ -8,6 +8,8 @@ read_when:
|
|
|
8
8
|
|
|
9
9
|
# Harness setup
|
|
10
10
|
|
|
11
|
+
English | [中文](harness-setup.zh-CN.md)
|
|
12
|
+
|
|
11
13
|
Where a pasted image ends up differs per harness, and modlens takes a different route in each. `recover-paste` detects which harness it runs inside (process ancestry, then environment fingerprints) and reads only that harness's storage.
|
|
12
14
|
|
|
13
15
|
## Codex
|
|
@@ -57,3 +59,21 @@ npx -y @deepseek-ai/dsh plugin --profile web add @liustack/modlens@latest
|
|
|
57
59
|
```
|
|
58
60
|
|
|
59
61
|
This registers a `read_image` tool whose schema reaches the model on every request (no trigger heuristics), runs the modlens CLI shipped inside the same package, and returns the structured evidence as the tool's canonical JSON output. Engines, reuse grants, and guard rules stay in `~/.modlens/config.json`, shared with every other harness. dsh is in developer preview and its plugin surface may change; the plugin keeps its touch small (raw tool registration, the llm adapter surface for the vision variants, the attachment reader, and one agent pre-step hook) and degrades loudly if any of them moves.
|
|
62
|
+
|
|
63
|
+
### Paste-to-path (web profile)
|
|
64
|
+
|
|
65
|
+
Pasting an image into the dsh Web UI under a **text-only model** used to die at
|
|
66
|
+
image admission. The plugin now ships a browser half (loaded automatically by
|
|
67
|
+
dsh's client plugin system) that takes over the paste in exactly that case:
|
|
68
|
+
the image bytes go to the plugin's `/modlens/paste` route on the dsh web
|
|
69
|
+
server (loopback, magic-byte checked, 25 MB cap), land as a private temp file,
|
|
70
|
+
and the composer receives the file path as plain text — the same shape Pi,
|
|
71
|
+
OpenCode, and Claude Code hand their models, and the modlens skill's and
|
|
72
|
+
`read_image` tool's primary trigger. Admission never fires because the message
|
|
73
|
+
carries no image attachment.
|
|
74
|
+
|
|
75
|
+
The takeover is conditional: when the selected model is a `(modlens vision)`
|
|
76
|
+
variant or a known vision model, the native paste flow is left alone (variants
|
|
77
|
+
convert at request time with the thumbnail preserved; vision models read
|
|
78
|
+
images themselves). `pasteToPath: false` in the plugin row turns the whole
|
|
79
|
+
feature off.
|
|
@@ -0,0 +1,67 @@
|
|
|
1
|
+
---
|
|
2
|
+
summary: '宿主接入:图片在 Codex、Claude Code、Pi、OpenCode 中如何抵达模型'
|
|
3
|
+
read_when:
|
|
4
|
+
- 在某个具体的编码 agent 里安装配置 modlens
|
|
5
|
+
- 粘贴的图片没有抵达模型
|
|
6
|
+
- 了解 recover-paste 在各 harness 里分别做什么
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# 宿主接入
|
|
10
|
+
|
|
11
|
+
[English](harness-setup.md) | 中文
|
|
12
|
+
|
|
13
|
+
粘贴的图片最终落在哪里,每个 harness 都不一样,modlens 在每个 harness 里走的路线也不同。`recover-paste` 会检测自己运行在哪个 harness 里(先看进程祖先,再看环境变量指纹),只读取该 harness 的存储。
|
|
14
|
+
|
|
15
|
+
## Codex
|
|
16
|
+
|
|
17
|
+
粘贴的图片会落成真实的临时文件,消息里带着形如 `<image name=[Image #1] path="/tmp/xxxx.png">` 的标签。skill 直接从标签里读出路径。`recover-paste` 检测到 Codex 后会拒绝执行,并把你指回这个标签。
|
|
18
|
+
|
|
19
|
+
纯文本模型有一个坑:一旦 `models.json` 声明了 `input_modalities: ["text"]`,Codex TUI 会直接拦下 Ctrl+V 粘贴。改为把文件拖进终端、手动输入路径,或使用 `codex exec -i image.png "..."`。
|
|
20
|
+
|
|
21
|
+
## Claude Code、Pi、OpenCode
|
|
22
|
+
|
|
23
|
+
这三家都不像 Codex 那样递给模型一个可用的临时文件路径(较新的 Claude Code 版本确实会把粘贴写进自己的 `~/.claude/image-cache/`,但只在终端入口以路径行的形式注入),不过三者都会在网关剥离图片之前,把用户消息完整存在本地:
|
|
24
|
+
|
|
25
|
+
| Harness | 存储位置 | 说明 |
|
|
26
|
+
| :-- | :-- | :-- |
|
|
27
|
+
| Claude Code | `~/.claude/projects/<slug>/<session>.jsonl` | 图片以 base64 存储。注入的 `CLAUDE_CODE_SESSION_ID` 可精确定位当前 session |
|
|
28
|
+
| Pi | `~/.pi/agent/sessions/--<encoded-cwd>--/*.jsonl` | 结构与 Claude Code 相同 |
|
|
29
|
+
| OpenCode | `~/.local/share/opencode/opencode.db` | SQLite,图片以 data URL 存储(通过 `node:sqlite` 读取) |
|
|
30
|
+
|
|
31
|
+
在 Claude Code 里通过 `ANTHROPIC_BASE_URL` 接入纯文本模型时,粘贴的图片要么变成一个不带路径的 `[Unsupported Image]` 占位符(宽松的网关),要么直接让请求报错([#62009](https://github.com/anthropics/claude-code/issues/62009))。图片字节并没有丢,`recover-paste` 取回的就是它。
|
|
32
|
+
|
|
33
|
+
## skill 的存放位置
|
|
34
|
+
|
|
35
|
+
| Harness | skill 读取位置 |
|
|
36
|
+
| :-- | :-- |
|
|
37
|
+
| Claude Code | `~/.claude/skills/` |
|
|
38
|
+
| Codex | `~/.codex/skills/` |
|
|
39
|
+
| Pi、OpenCode | `~/.agents/skills/` |
|
|
40
|
+
|
|
41
|
+
这些位置都支持符号链接,把 skill 目录链接一次,每个 agent 用的就都是最新版本。
|
|
42
|
+
|
|
43
|
+
## 平台支持
|
|
44
|
+
|
|
45
|
+
macOS 和 Linux 完整支持,并在 CI 上以 Node 22 和 24 验证。
|
|
46
|
+
|
|
47
|
+
Windows 跑同一套 CI 矩阵。那里没有 `ps`,检测会跳过进程祖先这一步,退回到上面的环境变量指纹,所以一个什么指纹都不设的 harness 会被判为未检出(用 `--harness` 或 `MODLENS_HARNESS` 强制指定)。OpenCode 的粘贴恢复在 Windows 上有覆盖,包括 [#11](https://github.com/liustack/modlens/issues/11) 里的路径分隔符归一化:opencode 记录的 `session.directory` 用正斜杠,而那里的 `path.resolve` 返回反斜杠,匹配前两边都会归一化。JSONL 存储(Claude Code、Pi)以 `os.homedir()` 和各 harness 自己的磁盘 slug 为键,在 POSIX 上验证。外部引擎(Antigravity CLI、Claude CLI)只在有 Windows 版本的平台上运行。
|
|
48
|
+
|
|
49
|
+
## 网关配置
|
|
50
|
+
|
|
51
|
+
OpenCode 接 DeepSeek:执行 `opencode auth login`,选择 DeepSeek 并粘贴 key(会存进 `~/.local/share/opencode/auth.json`),然后在 `~/.config/opencode/opencode.jsonc` 里把默认模型设为 `deepseek/deepseek-v4-flash`。Pi 从 `~/.pi/agent/auth.json` 读取它的 key。
|
|
52
|
+
|
|
53
|
+
## DeepSeek Harness(dsh)
|
|
54
|
+
|
|
55
|
+
dsh 与其他 harness 不同:modlens 以原生工具的形式接入,而不是靠提示词触发的 skill。本包自身就是一个 dsh bundle,一条命令即可装进某个 profile:
|
|
56
|
+
|
|
57
|
+
```sh
|
|
58
|
+
npx -y @deepseek-ai/dsh plugin --profile web add @liustack/modlens@latest
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
这会注册一个 `read_image` 工具,它的 schema 随每次请求抵达模型(不靠触发启发式),运行同一个包里自带的 modlens CLI,并把结构化证据作为工具的标准 JSON 输出返回。引擎、复用授权和 guard 规则仍在 `~/.modlens/config.json` 里,与其他所有 harness 共享。dsh 还在开发者预览阶段,插件接口可能变化。这个插件刻意保持很小的接触面(原生工具注册、视觉变体所用的 llm 适配层、附件读取器,以及一个 agent 执行前钩子),其中任何一处变动,它都会大声报错而不是无声退化。
|
|
62
|
+
|
|
63
|
+
### 粘贴转路径(paste-to-path,web profile)
|
|
64
|
+
|
|
65
|
+
过去在 dsh Web UI 里,**纯文本模型**下粘贴图片会死在图片准入检查这一步。插件现在带了一个浏览器端半边(由 dsh 的客户端插件系统自动加载),恰好在这种情况下接管粘贴:图片字节发到插件在 dsh web 服务器上的 `/modlens/paste` 路由(仅回环地址,校验 magic byte,上限 25 MB),落成一个私有临时文件,输入框收到的则是纯文本的文件路径。这与 Pi、OpenCode、Claude Code 递给模型的形态一致,也正是 modlens skill 和 `read_image` 工具的首要触发条件。消息里不带图片附件,准入检查根本不会触发。
|
|
66
|
+
|
|
67
|
+
接管是有条件的:当选中的模型是 `(modlens vision)` 变体或已知的视觉模型时,原生粘贴流程不受影响(变体在发请求时转换且保留缩略图,视觉模型则自己读图)。在插件配置行里设 `pasteToPath: false` 可整体关掉这个功能。
|
package/docs/output-schema.md
CHANGED
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
---
|
|
2
|
+
summary: '输出契约:每次识别返回的 JSON 结构、result 字段与 meta'
|
|
3
|
+
read_when:
|
|
4
|
+
- 解析 modlens 输出或在它之上构建工具
|
|
5
|
+
- 查 meta.attempts 和 meta.warnings 的含义
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# ModLens 输出契约(v2)
|
|
9
|
+
|
|
10
|
+
[English](output-schema.md) | 中文
|
|
11
|
+
|
|
12
|
+
CLI 向 stdout 打印一个 JSON 对象:
|
|
13
|
+
|
|
14
|
+
```json
|
|
15
|
+
{
|
|
16
|
+
"image": "/abs/path/or/url",
|
|
17
|
+
"provider": "antigravity-cli",
|
|
18
|
+
"result": { "...": "see below" },
|
|
19
|
+
"meta": {
|
|
20
|
+
"generatedAt": "2026-08-01T12:00:00.000Z",
|
|
21
|
+
"model": "gemini-3.6-flash-low",
|
|
22
|
+
"conversationId": "string|null",
|
|
23
|
+
"durationSeconds": 25.4,
|
|
24
|
+
"usage": {},
|
|
25
|
+
"attempts": [{ "provider": "antigravity-cli", "ok": true, "durationSeconds": 25.4 }],
|
|
26
|
+
"warnings": []
|
|
27
|
+
}
|
|
28
|
+
}
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
`meta.attempts` 按顺序列出这次运行中故障转移链尝试过的每个 provider,失败时附带 `error` 字符串。`meta.warnings` 携带路由通知:故障转移、被忽略的 `extraBody`,以及自动模式下这次识别花了谁的额度。
|
|
32
|
+
|
|
33
|
+
只要 provider 支持,`result` 就由 JSON schema 强制约束(agent CLI 走 `--json-schema`,API provider 走 response-schema 字段或预填模板),CLI 返回前还会自己校验一遍结构,所以结构损坏的结果会触发故障转移,不会到你手上:
|
|
34
|
+
|
|
35
|
+
```json
|
|
36
|
+
{
|
|
37
|
+
"summary": "string",
|
|
38
|
+
"ocr": {
|
|
39
|
+
"full_text": "string",
|
|
40
|
+
"lines": [
|
|
41
|
+
{ "text": "string", "language": "string (optional)" }
|
|
42
|
+
]
|
|
43
|
+
},
|
|
44
|
+
"layout": {
|
|
45
|
+
"regions": [
|
|
46
|
+
{
|
|
47
|
+
"type": "title|subtitle|paragraph|list|table|chart|form|code|image|icon|other",
|
|
48
|
+
"reading_order": 1,
|
|
49
|
+
"text": "string"
|
|
50
|
+
}
|
|
51
|
+
]
|
|
52
|
+
},
|
|
53
|
+
"semantics": {
|
|
54
|
+
"scene": "string",
|
|
55
|
+
"intent": "string (optional)",
|
|
56
|
+
"entities": [
|
|
57
|
+
{ "name": "string", "type": "string", "evidence": "string (optional)" }
|
|
58
|
+
],
|
|
59
|
+
"relations": [
|
|
60
|
+
{ "subject": "string", "predicate": "string", "object": "string" }
|
|
61
|
+
]
|
|
62
|
+
},
|
|
63
|
+
"visual": {
|
|
64
|
+
"dominant_colors": ["string"],
|
|
65
|
+
"style": "string",
|
|
66
|
+
"notes": ["string"]
|
|
67
|
+
},
|
|
68
|
+
"uncertainty": ["string"]
|
|
69
|
+
}
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
必填字段:`summary`、`ocr`、`layout`、`semantics`、`visual`、`uncertainty`,也就是每一个顶层字段,`visual` 也不例外。(早期文档把 `visual` 写成可选,但强制执行的 schema 一直要求它,请以 schema 为准。)
|
|
73
|
+
|
|
74
|
+
相对 v1 的变化:删掉了像素级 `bbox` 坐标和数值型 `confidence` 分数。视觉模型会凭空编造这两样,v2 不再假装提供。`layout.regions[].type` 新增了 `code`。
|
package/docs/security.md
CHANGED
|
@@ -8,6 +8,8 @@ read_when:
|
|
|
8
8
|
|
|
9
9
|
# Security
|
|
10
10
|
|
|
11
|
+
English | [中文](security.zh-CN.md)
|
|
12
|
+
|
|
11
13
|
## Recovered images are private
|
|
12
14
|
|
|
13
15
|
Images pulled out of session storage are written 0600 into a 0700 directory. By default that directory is a fresh, unpredictable `<tmpdir>/modlens-paste-*` minted per run, so nobody on a shared machine can pre-create a known path (`recursive` mkdir leaves an existing directory's mode alone) and read the bytes. A pasted screenshot can hold anything. An explicit `--out-dir` is honoured but refused when unsafe: it must be a real directory, not a symlink, owned by you, with no group or world access.
|
|
@@ -0,0 +1,43 @@
|
|
|
1
|
+
---
|
|
2
|
+
summary: '安全:modlens 会运行什么、恢复出的图片如何受保护、图片内容属于不可信输入'
|
|
3
|
+
read_when:
|
|
4
|
+
- 审查这个工具会在你的机器上做什么
|
|
5
|
+
- 在共享机器上恢复粘贴的图片
|
|
6
|
+
- 判断视觉引擎的报告可信到什么程度
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# 安全
|
|
10
|
+
|
|
11
|
+
[English](security.md) | 中文
|
|
12
|
+
|
|
13
|
+
## 恢复出的图片是私有的
|
|
14
|
+
|
|
15
|
+
从会话存储里取出的图片以 0600 权限写入 0700 目录。默认这个目录是每次运行新建的、不可预测的 `<tmpdir>/modlens-paste-*`,共享机器上没人能预先创建一个已知路径来读取字节(`recursive` 的 mkdir 不会改动已存在目录的权限模式)。一张粘贴的截图里可能有任何东西。显式传入的 `--out-dir` 会被接受,但不安全时会被拒绝:它必须是真实目录,不是符号链接,归你所有,组和其他用户无任何权限。
|
|
16
|
+
|
|
17
|
+
恢复同样限定在单个项目内:检查的是 transcript 里记录的工作目录,不只是目录名,因为目录 slug 会撞车(`/tmp/a.b` 和 `/tmp/a-b` 生成同一个 slug)。相邻项目的图片绝不会被交出去。
|
|
18
|
+
|
|
19
|
+
## 传给引擎的权限
|
|
20
|
+
|
|
21
|
+
ModLens 调用 `agy` 时带上 `--dangerously-skip-permissions`,因为某些环境下 prompt 模式不带它就会失败。prompt 把 agent 限制为只读交给它的那一张图片,并指示它把图片内容严格当作数据。
|
|
22
|
+
|
|
23
|
+
`claude-cli` provider 只带 `--allowedTools Read` 运行,因此它能读本地文件,别的什么都做不了。
|
|
24
|
+
|
|
25
|
+
两个子进程 provider 还都运行在一个用完即弃的目录里,每次调用新建,结束后删除。本地图片时,目录里只有那张图片的一份私有副本,别无他物,而且是真实拷贝,绝不用硬链接,provider 往自己的临时路径写东西也碰不到原文件。远程图片时目录是空的,agent 把文件下载进去。没有这层隔离,图片里的文字就可能引导一个权限宽泛的 agent 去读原图旁边的文件,或读调用者恰好所在的项目。传 `--workdir` 即放弃这层隔离,在你指定的位置运行。
|
|
26
|
+
|
|
27
|
+
这是在缩小暴露面,不是操作系统沙箱:agent 仍然能读绝对路径、访问网络、启动进程。把它当作更收紧的默认值,而不是安全边界。不信任的图片优先用 `-p gemini-api`,它自己下载字节,不运行任何本地 agent。远程 URL 本来就优先走内联这一侧:远程 URL 的故障转移链先试内联的 API provider,agent 排在最后。远程 URL 实际由谁抓取因 provider 而异,而只有本地下载能被本地防护覆盖:
|
|
28
|
+
|
|
29
|
+
| Provider | 远程 URL 由谁抓取 | 本地防护 |
|
|
30
|
+
| :-- | :-- | :-- |
|
|
31
|
+
| `gemini-api` | modlens 自己下载,字节内联发送 | 私有地址防护、文件头魔数(magic bytes)检查、25 MB 上限 |
|
|
32
|
+
| `openai`、`anthropic` | URL 传给厂商,由厂商抓取 | 本地无,适用厂商自己的抓取策略 |
|
|
33
|
+
| `antigravity-cli`、agent CLI | agent 自行抓取 | 本地无 |
|
|
34
|
+
|
|
35
|
+
所以私有地址防护、文件头魔数检查和大小上限保护的恰好是 modlens 亲自下载的路径:每一次本地文件读取,加上 gemini-api 的远程抓取。显式 `-p` 钉死单个 provider,覆盖整条链。
|
|
36
|
+
|
|
37
|
+
## 图片内容是不可信输入
|
|
38
|
+
|
|
39
|
+
图片里的文字是不可信的,和网页一样。一张截图可能带着写给任何读取者的指令。prompt 里明确说了这一点,但那是缓解措施,不是保证:只分析你愿意打开的图片,图片来自别处时优先用沙箱化的工作目录。
|
|
40
|
+
|
|
41
|
+
## 证据,而非编造
|
|
42
|
+
|
|
43
|
+
引擎读不出来的内容会进入 `uncertainty`,而不是被脑补出来。v2 彻底去掉了像素坐标和置信度分数,因为这两个字段正是模型编造得最像真的。
|
package/docs/troubleshooting.md
CHANGED
|
@@ -8,6 +8,8 @@ read_when:
|
|
|
8
8
|
|
|
9
9
|
# Troubleshooting
|
|
10
10
|
|
|
11
|
+
English | [中文](troubleshooting.zh-CN.md)
|
|
12
|
+
|
|
11
13
|
Start with `modlens doctor`: it checks your Node version, which providers are ready, which one will be selected and why, and the detected harness, all without spending quota or making a network request. It catches most setup problems before you read any further.
|
|
12
14
|
|
|
13
15
|
Every message below is one modlens actually prints. Search this file for the words you saw.
|