@liustack/modlens 3.24.0 → 3.24.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +8 -0
- package/README.md +1 -1
- package/README.zh-CN.md +2 -2
- package/dist/main.js +2 -2
- package/docs/harness-setup.md +2 -2
- package/docs/harness-setup.zh-CN.md +2 -2
- package/docs/troubleshooting.md +2 -2
- package/docs/troubleshooting.zh-CN.md +2 -2
- package/dsh/index.js +205 -43
- package/package.json +1 -1
- package/skills/modlens/SKILL.md +4 -4
- package/skills/modlens/references/runtime.md +1 -1
- package/skills/modlens/scripts/run.ps1 +1 -1
- package/skills/modlens/scripts/run.sh +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,13 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 3.24.2 - 2026-08-24
|
|
4
|
+
|
|
5
|
+
- **dsh: repeated `modlens_read_image` calls reuse one image read ([#81](https://github.com/liustack/modlens/issues/81)).** A small host model can emit the same path-tool call repeatedly inside its thinking loop, and every call started a new CLI process because #68 cached attachment conversion but not the explicit tool. The tool now keeps a per-plugin LRU cache keyed by the CLI-normalized source identity and focus prompt. Identical concurrent calls join the same pending read. A changed local file or focus creates a fresh read, remote results expire after 60 seconds, and failures cool for 60 seconds before retrying. Returned evidence is cloned so one consumer cannot mutate the shared result, and the model-facing description explicitly says to call once and reuse the evidence. This prevents repeated tool decisions from becoming repeated vision-provider work. It cannot make a small model stop hallucinating calls or recover reliably from every tool failure, so duplicate call events may remain visible and hosts should still cap tool rounds where available.
|
|
6
|
+
|
|
7
|
+
## 3.24.1 - 2026-08-24
|
|
8
|
+
|
|
9
|
+
- **dsh bundle startup no longer logs `inactive context` before `llm` is ready ([#79](https://github.com/liustack/modlens/issues/79)).** Bundle loaders can invoke the plugin while required services are still inactive. The first auto-discovery sweep read `ctx.llm` immediately, logged one to three errors, then relied on later topology events to recover. Discovery now runs inside Cordis's injected `llm` lifecycle. Each activation binds its own service, invalidates pending probes before teardown, and releases its wrapper ownership. A Cordis effect boundary also stops Promise continuations that were already queued from registering or refreshing adapters after the scope starts unloading. Existing and late providers are still discovered exactly once, and preview hosts without injection retain their previous feature-detected path.
|
|
10
|
+
|
|
3
11
|
## 3.24.0 - 2026-08-22
|
|
4
12
|
|
|
5
13
|
- A single `gemini-api`, `openai`, or `anthropic` provider can now use multiple API keys. Both `providers.<name>.apiKey` and the corresponding environment variable accept a comma-separated list, trim each item, and ignore empty entries. Requests use the configured order and rotate only after authentication, rate-limit, or quota failures. Network, 5xx, and response-parsing failures skip the remaining keys and preserve the existing cross-provider failover. Quota cooldown is recorded per key, and a provider moves to the back only when every configured key is cooling. Legacy provider-level cooldown state still loads. Every sibling key is registered for redaction, `doctor` reports the key count, and the dsh settings card explains the syntax. Single-key and keyless attempts keep their previous JSON shape, while multi-key attempts add a zero-based `keyIndex`.
|
package/README.md
CHANGED
|
@@ -33,7 +33,7 @@ Issues are welcome any time: [open one](https://github.com/liustack/modlens/issu
|
|
|
33
33
|
|
|
34
34
|
## Highlights
|
|
35
35
|
|
|
36
|
-
**🥇 The most capable vision plugin for DeepSeek Harness (dsh):** install it instantly with one command: `npx -y @deepseek-ai/dsh plugin --profile web add @liustack/modlens@3.24.
|
|
36
|
+
**🥇 The most capable vision plugin for DeepSeek Harness (dsh):** install it instantly with one command: `npx -y @deepseek-ai/dsh plugin --profile web add @liustack/modlens@3.24.2`. See the [setup guide](docs/harness-setup.md) for installation and update details. If the command line is not your thing but you still want to try DSH, check out <a href="https://github.com/liustack/aimanager"><b>AIManager</b></a>, the lightest desktop wrapper for DeepSeek Harness. It gets you started with zero code or configuration and installs every dependency for you with one click.
|
|
37
37
|
|
|
38
38
|
Pasting an image works two ways. **① Just paste.** On a text-only model the pasted image lands as a private temp file and its path enters the composer — the same interaction OpenCode and Pi ship — and the `modlens_read_image` tool takes it from there. **② Pick a `(modlens vision)` entry** in the model selector (it remembers your choice, so once is enough), then paste: the thumbnail stays visible in your message, closer to the Codex app feel, and the image is converted to structured evidence at request time, answered by the same underlying route. The plugin auto-discovers every provider route carrying text-only DeepSeek or GLM models and adds a wrapped entry per route (a stock install gets **`DeepSeek-V4-Flash (modlens vision)`** and **`DeepSeek-V4-Pro (modlens vision)`**; extra routes like opencode-go or zai get their own); the two families' own vision models are excluded automatically. Which paste route applies is the host's per-model call: only a model its metadata positively confirms text-only is taken over, anything unconfirmed is left alone, so vision models keep their native paste ([details](docs/harness-setup.md)).
|
|
39
39
|
|
package/README.zh-CN.md
CHANGED
|
@@ -33,7 +33,7 @@ DeepSeek 和 GLM 的主力对话模型是纯文本的,无法进行图片识别
|
|
|
33
33
|
|
|
34
34
|
## 亮点
|
|
35
35
|
|
|
36
|
-
**🥇 全网最强的 DeepSeek Harness(dsh)外挂视觉识别插件:**一条命令即刻安装 `npx -y @deepseek-ai/dsh plugin --profile web add @liustack/modlens@3.24.
|
|
36
|
+
**🥇 全网最强的 DeepSeek Harness(dsh)外挂视觉识别插件:**一条命令即刻安装 `npx -y @deepseek-ai/dsh plugin --profile web add @liustack/modlens@3.24.2`。更多安装与更新细节参考 [配置手册](docs/harness-setup.zh-CN.md) 。如果用不惯命令行,也想想玩玩 DSH,推荐食用全网最轻量级的 DeepSeek Harness 桌面版封装 <a href="https://github.com/liustack/aimanager"><b> AIManager</b></a>,零代码零配置起手,一键帮你安装所有依赖环境。
|
|
37
37
|
|
|
38
38
|
DeepSeek Harness 粘贴识图有两种玩法。
|
|
39
39
|
|
|
@@ -71,7 +71,7 @@ agy # 浏览器完成
|
|
|
71
71
|
**DeepSeek Harness(dsh)用户不走 skill 流程**,本包就是原生 dsh 插件:
|
|
72
72
|
|
|
73
73
|
```sh
|
|
74
|
-
npx -y @deepseek-ai/dsh plugin --profile web add @liustack/modlens@3.24.
|
|
74
|
+
npx -y @deepseek-ai/dsh plugin --profile web add @liustack/modlens@3.24.2
|
|
75
75
|
```
|
|
76
76
|
|
|
77
77
|
装完即有 `modlens_read_image` 工具,选「(modlens vision)」模型变体即可直接粘贴识图。引擎配置同样在 `~/.modlens`,详见[宿主接入](docs/harness-setup.zh-CN.md)。
|
package/dist/main.js
CHANGED
|
@@ -5520,7 +5520,7 @@ function parsePositiveInt(raw, flag) {
|
|
|
5520
5520
|
}
|
|
5521
5521
|
return Number.parseInt(raw, 10);
|
|
5522
5522
|
}
|
|
5523
|
-
program.name("modlens").description("Plug-in vision for text-only LLMs: image in, structured JSON evidence out").version("3.24.
|
|
5523
|
+
program.name("modlens").description("Plug-in vision for text-only LLMs: image in, structured JSON evidence out").version("3.24.2");
|
|
5524
5524
|
program.command("analyze", { isDefault: true }).description("Analyze an image into structured JSON evidence (default command)").requiredOption("-i, --input <path|url>", "Input image path or https URL").option("-o, --output <path>", "Write result JSON to a file").option("-m, --model <name>", "Provider model name").option("-p, --provider <name>", `Vision provider (${listProviders().join(", ")})`).option("--prompt <text>", "Extra focus for this image").option("--timeout <ms>", "Provider timeout in milliseconds", "180000").option("--provider-bin <path>", "Provider binary path (default: agy)").option("--workdir <path>", "Working directory for the provider").option(
|
|
5525
5525
|
"--extra-body <json>",
|
|
5526
5526
|
`JSON merged into the API request body, e.g. '{"thinking":{"type":"disabled"}}'`
|
|
@@ -5631,7 +5631,7 @@ program.command("doctor").description(
|
|
|
5631
5631
|
configPath: CONFIG_PATH,
|
|
5632
5632
|
// Lets doctor name an installed skill copy that is older than
|
|
5633
5633
|
// the CLI reporting on it (issue #33).
|
|
5634
|
-
version: "3.24.
|
|
5634
|
+
version: "3.24.2"
|
|
5635
5635
|
});
|
|
5636
5636
|
const output = options.json ? JSON.stringify(report, null, 2) : renderDoctorReport(report);
|
|
5637
5637
|
process.stdout.write(`${output}
|
package/docs/harness-setup.md
CHANGED
|
@@ -55,7 +55,7 @@ OpenCode with DeepSeek: `opencode auth login`, pick DeepSeek and paste the key (
|
|
|
55
55
|
dsh is different from the other harnesses: modlens plugs in as a native tool, not a prompt-triggered skill. The package itself is a dsh bundle, so one command installs it into a profile:
|
|
56
56
|
|
|
57
57
|
```sh
|
|
58
|
-
npx -y @deepseek-ai/dsh plugin --profile web add @liustack/modlens@3.24.
|
|
58
|
+
npx -y @deepseek-ai/dsh plugin --profile web add @liustack/modlens@3.24.2
|
|
59
59
|
```
|
|
60
60
|
|
|
61
61
|
This registers a `modlens_read_image` tool whose schema reaches the model on every request (no trigger heuristics), runs the modlens CLI shipped inside the same package, and returns the structured evidence as the tool's canonical JSON output. Engines, reuse grants, and guard rules stay in `~/.modlens/config.json`, shared with every other harness. dsh is in developer preview and its plugin surface may change; the plugin keeps its touch small (raw tool registration, the llm adapter surface for the vision variants, the attachment reader, and one agent pre-step hook) and degrades loudly if any of them moves.
|
|
@@ -87,7 +87,7 @@ modlens ships often, and both install shapes freeze at whatever version they
|
|
|
87
87
|
got. On dsh, re-run the install with the version named:
|
|
88
88
|
|
|
89
89
|
```sh
|
|
90
|
-
npx -y @deepseek-ai/dsh plugin --profile <name> add @liustack/modlens@3.24.
|
|
90
|
+
npx -y @deepseek-ai/dsh plugin --profile <name> add @liustack/modlens@3.24.2
|
|
91
91
|
```
|
|
92
92
|
|
|
93
93
|
`npm view @liustack/modlens version` prints the current one, and this page is
|
|
@@ -55,7 +55,7 @@ OpenCode 接 DeepSeek:执行 `opencode auth login`,选择 DeepSeek 并粘贴
|
|
|
55
55
|
dsh 与其他 harness 不同:modlens 以原生工具的形式接入,而不是靠提示词触发的 skill。本包自身就是一个 dsh bundle,一条命令即可装进某个 profile:
|
|
56
56
|
|
|
57
57
|
```sh
|
|
58
|
-
npx -y @deepseek-ai/dsh plugin --profile web add @liustack/modlens@3.24.
|
|
58
|
+
npx -y @deepseek-ai/dsh plugin --profile web add @liustack/modlens@3.24.2
|
|
59
59
|
```
|
|
60
60
|
|
|
61
61
|
这会注册一个 `modlens_read_image` 工具,它的 schema 随每次请求抵达模型(不靠触发启发式),运行同一个包里自带的 modlens CLI,并把结构化证据作为工具的标准 JSON 输出返回。引擎、复用授权和 guard 规则仍在 `~/.modlens/config.json` 里,与其他所有 harness 共享。dsh 还在开发者预览阶段,插件接口可能变化。这个插件刻意保持很小的接触面(原生工具注册、视觉变体所用的 llm 适配层、附件读取器,以及一个 agent 执行前钩子),其中任何一处变动,它都会大声报错而不是无声退化。
|
|
@@ -73,7 +73,7 @@ dsh 的网页用户面前没有终端,所以引擎设置有一张卡片,在*
|
|
|
73
73
|
modlens 发布很频繁,而两种安装形态都会冻结在装进来的那个版本上。dsh 上重跑一遍安装即可,版本号要点名:
|
|
74
74
|
|
|
75
75
|
```sh
|
|
76
|
-
npx -y @deepseek-ai/dsh plugin --profile <name> add @liustack/modlens@3.24.
|
|
76
|
+
npx -y @deepseek-ai/dsh plugin --profile <name> add @liustack/modlens@3.24.2
|
|
77
77
|
```
|
|
78
78
|
|
|
79
79
|
`npm view @liustack/modlens version` 可以查到当前版本号,本页的版本号则由发布流程自动写入。
|
package/docs/troubleshooting.md
CHANGED
|
@@ -163,7 +163,7 @@ simply lands on an older one. Name the exact version instead, which pnpm treats
|
|
|
163
163
|
as a deliberate request rather than a resolution:
|
|
164
164
|
|
|
165
165
|
```sh
|
|
166
|
-
npx -y @deepseek-ai/dsh plugin --profile <name> add @liustack/modlens@3.24.
|
|
166
|
+
npx -y @deepseek-ai/dsh plugin --profile <name> add @liustack/modlens@3.24.2
|
|
167
167
|
```
|
|
168
168
|
|
|
169
169
|
`npm view @liustack/modlens version` prints the current one. pnpm 11 installs a named
|
|
@@ -178,7 +178,7 @@ file:
|
|
|
178
178
|
|
|
179
179
|
```yaml
|
|
180
180
|
minimumReleaseAgeExclude:
|
|
181
|
-
- '@liustack/modlens@3.24.
|
|
181
|
+
- '@liustack/modlens@3.24.2'
|
|
182
182
|
```
|
|
183
183
|
|
|
184
184
|
Or lift the gate for a single command, which lifts it for everything that
|
|
@@ -144,7 +144,7 @@ dsh profile 装到的是旧版 modlens。`dsh.bundle` 声明从 3.9.0 起才存
|
|
|
144
144
|
`@latest` 绕不开这一层,本页早先的说法是错的。冷静期先把候选版本过滤掉,dist-tag 才在剩下的里面解析,于是它直接落到了更旧的那个上。改成写死精确版本号,pnpm 会把它当作一次明确的指定,而不是一次解析:
|
|
145
145
|
|
|
146
146
|
```sh
|
|
147
|
-
npx -y @deepseek-ai/dsh plugin --profile <name> add @liustack/modlens@3.24.
|
|
147
|
+
npx -y @deepseek-ai/dsh plugin --profile <name> add @liustack/modlens@3.24.2
|
|
148
148
|
```
|
|
149
149
|
|
|
150
150
|
`npm view @liustack/modlens version` 可以查到当前版本号。pnpm 11 会装上被点名的版本,11.1.3 起还会把它作为一条已批准的例外写进该 profile 的 `pnpm-workspace.yaml`,其余所有包和 modlens 以后的版本仍然留在窗口后面。
|
|
@@ -153,7 +153,7 @@ npx -y @deepseek-ai/dsh plugin --profile <name> add @liustack/modlens@3.24.0
|
|
|
153
153
|
|
|
154
154
|
```yaml
|
|
155
155
|
minimumReleaseAgeExclude:
|
|
156
|
-
- '@liustack/modlens@3.24.
|
|
156
|
+
- '@liustack/modlens@3.24.2'
|
|
157
157
|
```
|
|
158
158
|
|
|
159
159
|
或者只为这一条命令解除冷静期,注意它解除的是这条命令解析到的所有包,不只 modlens:
|
package/dsh/index.js
CHANGED
|
@@ -10,9 +10,9 @@
|
|
|
10
10
|
// Loaded via the cordis.patch.yml row `@liustack/modlens/dsh` (see the
|
|
11
11
|
// package.json `dsh.bundle` manifest). Providers, reuse grants, and guard
|
|
12
12
|
// rules keep living in ~/.modlens/config.json, shared with every harness.
|
|
13
|
-
import { chmodSync, lstatSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs'
|
|
13
|
+
import { chmodSync, lstatSync, mkdirSync, readFileSync, statSync, writeFileSync } from 'node:fs'
|
|
14
14
|
import { homedir, tmpdir } from 'node:os'
|
|
15
|
-
import { dirname, join } from 'node:path'
|
|
15
|
+
import { dirname, join, resolve } from 'node:path'
|
|
16
16
|
import { fileURLToPath } from 'node:url'
|
|
17
17
|
import { spawnHidden } from './spawnHidden.js'
|
|
18
18
|
|
|
@@ -22,6 +22,90 @@ const CLI_PATH = fileURLToPath(new URL('../dist/main.js', import.meta.url))
|
|
|
22
22
|
const OUTPUT_SCHEMA = JSON.parse(readFileSync(new URL('./vision-schema.json', import.meta.url), 'utf8'))
|
|
23
23
|
|
|
24
24
|
const CLI_TIMEOUT_MS = 180_000
|
|
25
|
+
const TOOL_READ_CACHE_LIMIT = 256
|
|
26
|
+
const TOOL_READ_FAILURE_COOLDOWN_MS = 60_000
|
|
27
|
+
const TOOL_READ_REMOTE_TTL_MS = 60_000
|
|
28
|
+
// Shared by both evidence caches. Monotonic time keeps an NTP adjustment from
|
|
29
|
+
// extending or collapsing a retry window.
|
|
30
|
+
const monotonicNow = () => performance.now()
|
|
31
|
+
|
|
32
|
+
function toolReadSourceKey(rawSource) {
|
|
33
|
+
const source = rawSource.trim()
|
|
34
|
+
if (/^https?:\/\//i.test(source)) {
|
|
35
|
+
return `remote:${source}`
|
|
36
|
+
}
|
|
37
|
+
try {
|
|
38
|
+
const file = /^file:\/\//i.test(source) ? fileURLToPath(source) : resolve(source)
|
|
39
|
+
const stat = statSync(file, { bigint: true })
|
|
40
|
+
// A path is not an image identity. The inode and nanosecond timestamps
|
|
41
|
+
// make an overwrite or replacement a fresh read while unchanged bytes at
|
|
42
|
+
// the same path stay reusable during a thinking loop.
|
|
43
|
+
return `local:${file}:${stat.dev}:${stat.ino}:${stat.mode}:${stat.size}:${stat.mtimeNs}:${stat.ctimeNs}`
|
|
44
|
+
} catch {
|
|
45
|
+
// Let the CLI produce the canonical missing-file or invalid-path error.
|
|
46
|
+
// If the file appears later, the successful stat above changes the key.
|
|
47
|
+
return `local-unreadable:${source}`
|
|
48
|
+
}
|
|
49
|
+
}
|
|
50
|
+
|
|
51
|
+
function trimToolReadCache(cache) {
|
|
52
|
+
while (cache.size > TOOL_READ_CACHE_LIMIT) {
|
|
53
|
+
let victim
|
|
54
|
+
for (const [key, entry] of cache) {
|
|
55
|
+
if (entry.state !== 'pending') {
|
|
56
|
+
victim = key
|
|
57
|
+
break
|
|
58
|
+
}
|
|
59
|
+
}
|
|
60
|
+
// In-flight work stays joinable. The cap goes soft only while every
|
|
61
|
+
// candidate is pending, then settlement runs this trim again.
|
|
62
|
+
if (victim === undefined) return
|
|
63
|
+
cache.delete(victim)
|
|
64
|
+
}
|
|
65
|
+
}
|
|
66
|
+
|
|
67
|
+
function cachedToolRead(cache, key, remote, load) {
|
|
68
|
+
const now = monotonicNow()
|
|
69
|
+
let entry = cache.get(key)
|
|
70
|
+
if (entry && entry.expiresAt <= now) {
|
|
71
|
+
cache.delete(key)
|
|
72
|
+
entry = undefined
|
|
73
|
+
}
|
|
74
|
+
if (entry) {
|
|
75
|
+
// Map insertion order is the LRU order.
|
|
76
|
+
cache.delete(key)
|
|
77
|
+
cache.set(key, entry)
|
|
78
|
+
return entry.promise
|
|
79
|
+
}
|
|
80
|
+
|
|
81
|
+
entry = { state: 'pending', expiresAt: Number.POSITIVE_INFINITY, promise: undefined }
|
|
82
|
+
const promise = Promise.resolve()
|
|
83
|
+
.then(load)
|
|
84
|
+
.then(
|
|
85
|
+
(value) => {
|
|
86
|
+
if (cache.get(key) === entry) {
|
|
87
|
+
entry.state = 'success'
|
|
88
|
+
entry.expiresAt = remote ? monotonicNow() + TOOL_READ_REMOTE_TTL_MS : Number.POSITIVE_INFINITY
|
|
89
|
+
trimToolReadCache(cache)
|
|
90
|
+
}
|
|
91
|
+
return value
|
|
92
|
+
},
|
|
93
|
+
(error) => {
|
|
94
|
+
if (cache.get(key) === entry) {
|
|
95
|
+
// The same failure is free and stable during the cooldown. A later
|
|
96
|
+
// call probes again, so a recovered engine needs no restart.
|
|
97
|
+
entry.state = 'failure'
|
|
98
|
+
entry.expiresAt = monotonicNow() + TOOL_READ_FAILURE_COOLDOWN_MS
|
|
99
|
+
trimToolReadCache(cache)
|
|
100
|
+
}
|
|
101
|
+
throw error
|
|
102
|
+
},
|
|
103
|
+
)
|
|
104
|
+
entry.promise = promise
|
|
105
|
+
cache.set(key, entry)
|
|
106
|
+
trimToolReadCache(cache)
|
|
107
|
+
return promise
|
|
108
|
+
}
|
|
25
109
|
|
|
26
110
|
export const name = 'modlens'
|
|
27
111
|
export const inject = ['tools', 'agents', 'attachments', 'llm']
|
|
@@ -41,6 +125,10 @@ export function apply(ctx, config = {}) {
|
|
|
41
125
|
// whichever surface asks first (issue #68; auto-read used to bypass
|
|
42
126
|
// caching entirely and re-read every image on every step).
|
|
43
127
|
const evidenceCache = new Map()
|
|
128
|
+
// Explicit path-tool calls can repeat inside a small model's thinking loop.
|
|
129
|
+
// Keep their completed evidence beside the attachment cache so the model's
|
|
130
|
+
// repeated decision does not become repeated vision-provider work (#81).
|
|
131
|
+
const toolReadCache = new Map()
|
|
44
132
|
// Off by default since the vision provider converts at request time and
|
|
45
133
|
// keeps the durable log (and the UI thumbnail) intact; turn it on only for
|
|
46
134
|
// setups where images enter through a provider this plugin does not wrap.
|
|
@@ -55,7 +143,20 @@ export function apply(ctx, config = {}) {
|
|
|
55
143
|
// wrappers land, including the later sweeps, and read by the verdict.
|
|
56
144
|
const ownProviders = new Set()
|
|
57
145
|
if (config.visionProvider !== false) {
|
|
58
|
-
|
|
146
|
+
// Bundle loaders can call apply while this outer context is still waiting
|
|
147
|
+
// for its required services. Reading ctx.llm here then throws "inactive
|
|
148
|
+
// context" before the first discovery sweep can register any lifecycle
|
|
149
|
+
// work (#79). Put the whole provider registry inside an injected child
|
|
150
|
+
// scope: Cordis starts it only while llm is active, and tears its listeners
|
|
151
|
+
// and registrations down with that service. Preview hosts without inject
|
|
152
|
+
// keep the dependency-free plugin's former feature-detected path.
|
|
153
|
+
if (typeof ctx.inject === 'function') {
|
|
154
|
+
ctx.inject(['llm'], (scope) => {
|
|
155
|
+
return registerVisionProvider(scope, config, ownProviders, evidenceCache)
|
|
156
|
+
})
|
|
157
|
+
} else {
|
|
158
|
+
registerVisionProvider(ctx, config, ownProviders, evidenceCache)
|
|
159
|
+
}
|
|
59
160
|
}
|
|
60
161
|
// Paste-to-path: the browser half (dsh/client.js) intercepts image pastes
|
|
61
162
|
// and POSTs the bytes here; the file lands in a private temp dir and the
|
|
@@ -130,7 +231,7 @@ export function apply(ctx, config = {}) {
|
|
|
130
231
|
const readImageTool = (toolName) => ({
|
|
131
232
|
name: toolName,
|
|
132
233
|
description:
|
|
133
|
-
'Read an image through the modlens vision bridge. Use whenever a message references an image the current model cannot see: a local file path or an http(s) URL to a screenshot, photo, chart, diagram, or document scan. Returns structured evidence with every word transcribed (ocr.full_text), layout regions in reading order, semantics, and an uncertainty list
|
|
234
|
+
'Read an image through the modlens vision bridge. Use whenever a message references an image the current model cannot see: a local file path or an http(s) URL to a screenshot, photo, chart, diagram, or document scan. Returns structured evidence with every word transcribed (ocr.full_text), layout regions in reading order, semantics, and an uncertainty list. Quote the evidence instead of guessing. For the same image and focus, call this tool once and reuse its returned evidence instead of calling again. Requires a configured modlens engine (run `npx @liustack/modlens doctor` in a terminal to check).',
|
|
134
235
|
parameters: {
|
|
135
236
|
type: 'object',
|
|
136
237
|
properties: {
|
|
@@ -165,23 +266,31 @@ export function apply(ctx, config = {}) {
|
|
|
165
266
|
if (typeof args?.path !== 'string' || args.path.trim() === '') {
|
|
166
267
|
throw new Error(`${toolName} needs a non-empty string "path".`)
|
|
167
268
|
}
|
|
168
|
-
const
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
269
|
+
const sourceKey = toolReadSourceKey(args.path)
|
|
270
|
+
const cacheKey = JSON.stringify([sourceKey, typeof args.prompt === 'string' ? args.prompt : ''])
|
|
271
|
+
const pending = cachedToolRead(toolReadCache, cacheKey, sourceKey.startsWith('remote:'), async () => {
|
|
272
|
+
const cliArgs = [CLI_PATH, '-i', args.path, '--timeout', String(CLI_TIMEOUT_MS)]
|
|
273
|
+
if (args.prompt) {
|
|
274
|
+
cliArgs.push('--prompt', args.prompt)
|
|
275
|
+
}
|
|
276
|
+
// The shared read has the CLI's own deadline but not one caller's
|
|
277
|
+
// signal. A caller may stop waiting without cancelling every other
|
|
278
|
+
// concurrent caller that joined the same image read.
|
|
279
|
+
const { stdout, stderr, code } = await run(process.execPath, cliArgs, undefined)
|
|
280
|
+
if (code !== 0) {
|
|
281
|
+
throw new Error(`modlens failed (exit ${code}): ${(stderr || stdout).trim().slice(0, 500)}`)
|
|
282
|
+
}
|
|
283
|
+
let parsed
|
|
284
|
+
try {
|
|
285
|
+
parsed = JSON.parse(stdout)
|
|
286
|
+
} catch {
|
|
287
|
+
throw new Error(`modlens produced no JSON: ${stdout.trim().slice(0, 300)}`)
|
|
288
|
+
}
|
|
289
|
+
// The canonical value is the vision result itself; routing details
|
|
290
|
+
// (meta.attempts, whose quota a reused engine spent) stay operational.
|
|
291
|
+
return parsed.result
|
|
292
|
+
})
|
|
293
|
+
return structuredClone(await abortableWait(pending, exec.signal))
|
|
185
294
|
},
|
|
186
295
|
})
|
|
187
296
|
// A name of our own rather than the host's. dsh's registry is layered and
|
|
@@ -569,10 +678,50 @@ function registerVisionProvider(ctx, config, ownProviders, evidenceCache) {
|
|
|
569
678
|
if (Array.isArray(info?.inputModalities) && info.inputModalities.includes('image')) return false
|
|
570
679
|
return true
|
|
571
680
|
}
|
|
572
|
-
|
|
681
|
+
// Keep this activation bound to the exact service implementation that made
|
|
682
|
+
// it runnable. Cordis reuses the child context when llm is replaced, so
|
|
683
|
+
// looking the service up again after an await could otherwise move an old
|
|
684
|
+
// topology result into the new registry.
|
|
685
|
+
const llm = ctx.llm
|
|
686
|
+
if (typeof llm?.registerAdapter !== 'function' || typeof llm?.stream !== 'function') {
|
|
573
687
|
return
|
|
574
688
|
}
|
|
575
689
|
|
|
690
|
+
// Discovery promises are ordinary JavaScript work, not Cordis effects.
|
|
691
|
+
// The disposer invalidates this activation before the injected child is
|
|
692
|
+
// re-run, and clears the ownership facts whose actual adapter effects the
|
|
693
|
+
// framework tears down independently.
|
|
694
|
+
let active = true
|
|
695
|
+
const claimedProviders = new Set()
|
|
696
|
+
const deactivate = () => {
|
|
697
|
+
active = false
|
|
698
|
+
for (const providerId of claimedProviders) ownProviders?.delete(providerId)
|
|
699
|
+
claimedProviders.clear()
|
|
700
|
+
}
|
|
701
|
+
// Cordis marks a fiber UNLOADING before it runs activation disposers. A
|
|
702
|
+
// promise continuation already in the microtask queue can therefore see
|
|
703
|
+
// `active` before the disposer flips it. Creating and immediately releasing
|
|
704
|
+
// a zero-work effect is the framework's atomic liveness boundary: once it
|
|
705
|
+
// succeeds, the following synchronous registry mutation cannot race an
|
|
706
|
+
// unload. A lifecycle refusal cancels this activation without turning an
|
|
707
|
+
// expected teardown into a terminal diagnostic.
|
|
708
|
+
const activationCanCommit = () => {
|
|
709
|
+
if (!active) return false
|
|
710
|
+
if (typeof ctx.effect !== 'function') return true
|
|
711
|
+
try {
|
|
712
|
+
const release = ctx.effect(() => {})
|
|
713
|
+
if (typeof release === 'function') release()
|
|
714
|
+
return active
|
|
715
|
+
} catch (error) {
|
|
716
|
+
const inactive =
|
|
717
|
+
error?.code === 'INACTIVE_EFFECT' ||
|
|
718
|
+
/cannot create effect on inactive context/i.test(String(error?.message ?? error))
|
|
719
|
+
if (!inactive) throw error
|
|
720
|
+
deactivate()
|
|
721
|
+
return false
|
|
722
|
+
}
|
|
723
|
+
}
|
|
724
|
+
|
|
576
725
|
// dsh snapshots providerInfo and providerRetryPolicy at registration time.
|
|
577
726
|
// Keep the state and registration handle for each wrapper so an upstream
|
|
578
727
|
// replacement can refresh those snapshots instead of leaving a synthetic
|
|
@@ -582,6 +731,7 @@ function registerVisionProvider(ctx, config, ownProviders, evidenceCache) {
|
|
|
582
731
|
const policyKey = (policy) => (policy === undefined ? undefined : JSON.stringify(policy))
|
|
583
732
|
|
|
584
733
|
const registerWrapper = (upstream, providerId, displayName) => {
|
|
734
|
+
if (!activationCanCommit()) return false
|
|
585
735
|
const state = { displayName, retryPolicyKey: undefined }
|
|
586
736
|
const withVision = (info) => {
|
|
587
737
|
const inputModalities = Array.isArray(info?.inputModalities) ? [...info.inputModalities] : []
|
|
@@ -590,7 +740,7 @@ function registerVisionProvider(ctx, config, ownProviders, evidenceCache) {
|
|
|
590
740
|
return { ...info, provider: providerId, inputModalities }
|
|
591
741
|
}
|
|
592
742
|
try {
|
|
593
|
-
const registration =
|
|
743
|
+
const registration = llm.registerAdapter([providerId], {
|
|
594
744
|
// Duck-typing LlmAdapter: providerInfo/providerRetryPolicy are
|
|
595
745
|
// base-class defaults a plain object must supply itself (their
|
|
596
746
|
// absence is exactly the silent registration failure this catch
|
|
@@ -608,20 +758,20 @@ function registerVisionProvider(ctx, config, ownProviders, evidenceCache) {
|
|
|
608
758
|
// and the registration boundary below fails closed instead; the
|
|
609
759
|
// ordinary not-mounted-yet case never reaches this method, because
|
|
610
760
|
// reconcile waits for the upstream before registering (#66).
|
|
611
|
-
if (typeof
|
|
612
|
-
const policy =
|
|
761
|
+
if (typeof llm.providerRetryPolicy !== 'function') return undefined
|
|
762
|
+
const policy = llm.providerRetryPolicy(upstream)
|
|
613
763
|
state.retryPolicyKey = policyKey(policy)
|
|
614
764
|
return policy
|
|
615
765
|
},
|
|
616
766
|
async listModels(_provider, signal) {
|
|
617
|
-
const models = await
|
|
767
|
+
const models = await llm.listModels(upstream, signal)
|
|
618
768
|
return models.filter(shouldWrap).map((model) => ({
|
|
619
769
|
...withVision(model),
|
|
620
770
|
name: `${model.name ?? model.id} (modlens vision)`,
|
|
621
771
|
}))
|
|
622
772
|
},
|
|
623
773
|
async resolveModel(_provider, model, signal) {
|
|
624
|
-
const info = await
|
|
774
|
+
const info = await llm.resolveModelInfo(upstream, model, signal)
|
|
625
775
|
if (!shouldWrap(info)) {
|
|
626
776
|
// Refusing is right: wrapping a model that reads images itself
|
|
627
777
|
// would claim a bridge it does not need, hand it text evidence
|
|
@@ -662,7 +812,7 @@ function registerVisionProvider(ctx, config, ownProviders, evidenceCache) {
|
|
|
662
812
|
return (async function* () {
|
|
663
813
|
const converted = await convertImagesToEvidence(ctx, options.messages, options.signal, self)
|
|
664
814
|
const messages = restoreUpstreamSource(converted, providerId, upstream)
|
|
665
|
-
yield*
|
|
815
|
+
yield* llm.stream({ ...options, provider: upstream, messages })
|
|
666
816
|
})()
|
|
667
817
|
},
|
|
668
818
|
evidenceCache,
|
|
@@ -672,6 +822,7 @@ function registerVisionProvider(ctx, config, ownProviders, evidenceCache) {
|
|
|
672
822
|
// duplicate below means someone else holds that id, and skipping a
|
|
673
823
|
// provider we do not own would let a real vision model's paste be
|
|
674
824
|
// taken over, which is the bug the verdict exists to prevent.
|
|
825
|
+
claimedProviders.add(providerId)
|
|
675
826
|
ownProviders?.add(providerId)
|
|
676
827
|
return true
|
|
677
828
|
} catch (error) {
|
|
@@ -695,6 +846,7 @@ function registerVisionProvider(ctx, config, ownProviders, evidenceCache) {
|
|
|
695
846
|
const dropWrapper = (upstream, current) => {
|
|
696
847
|
registrations.delete(upstream)
|
|
697
848
|
wrapped.delete(upstream)
|
|
849
|
+
claimedProviders.delete(current.providerId)
|
|
698
850
|
ownProviders?.delete(current.providerId)
|
|
699
851
|
if (typeof current.registration === 'function') current.registration()
|
|
700
852
|
}
|
|
@@ -704,9 +856,9 @@ function registerVisionProvider(ctx, config, ownProviders, evidenceCache) {
|
|
|
704
856
|
// failed and what to do next depends on whether it failed before or after
|
|
705
857
|
// the host committed.
|
|
706
858
|
const routed = (providerId) => {
|
|
707
|
-
if (typeof
|
|
859
|
+
if (typeof llm.listProviders !== 'function') return false
|
|
708
860
|
try {
|
|
709
|
-
return
|
|
861
|
+
return llm.listProviders().some((info) => (typeof info === 'string' ? info : info?.id) === providerId)
|
|
710
862
|
} catch {
|
|
711
863
|
return false
|
|
712
864
|
}
|
|
@@ -724,7 +876,7 @@ function registerVisionProvider(ctx, config, ownProviders, evidenceCache) {
|
|
|
724
876
|
let nextPolicyKey
|
|
725
877
|
try {
|
|
726
878
|
nextPolicyKey =
|
|
727
|
-
typeof
|
|
879
|
+
typeof llm.providerRetryPolicy === 'function' ? policyKey(llm.providerRetryPolicy(upstream)) : undefined
|
|
728
880
|
} catch (error) {
|
|
729
881
|
dropWrapper(upstream, current)
|
|
730
882
|
console.error(`[modlens] vision provider refresh removed (${current.providerId}): ${error}`)
|
|
@@ -772,9 +924,9 @@ function registerVisionProvider(ctx, config, ownProviders, evidenceCache) {
|
|
|
772
924
|
// plumbing below could never correct it, because the name it compared
|
|
773
925
|
// against was a constant.
|
|
774
926
|
const upstreamName = () => {
|
|
775
|
-
if (typeof
|
|
927
|
+
if (typeof llm.listProviders !== 'function') return upstream
|
|
776
928
|
try {
|
|
777
|
-
const found =
|
|
929
|
+
const found = llm.listProviders().find((entry) => entry.id === upstream)
|
|
778
930
|
return found?.name ?? upstream
|
|
779
931
|
} catch {
|
|
780
932
|
return upstream
|
|
@@ -789,6 +941,7 @@ function registerVisionProvider(ctx, config, ownProviders, evidenceCache) {
|
|
|
789
941
|
// re-examined when the holder's route disappears.
|
|
790
942
|
let claimedElsewhere = false
|
|
791
943
|
const reconcile = () => {
|
|
944
|
+
if (!activationCanCommit()) return
|
|
792
945
|
if (reconciling) {
|
|
793
946
|
// dropWrapper's disposer makes the host emit adapters-updated while
|
|
794
947
|
// this very run is on the stack, and whatever that event announced
|
|
@@ -800,8 +953,8 @@ function registerVisionProvider(ctx, config, ownProviders, evidenceCache) {
|
|
|
800
953
|
try {
|
|
801
954
|
const current = registrations.get(upstream)
|
|
802
955
|
const available =
|
|
803
|
-
typeof
|
|
804
|
-
|
|
956
|
+
typeof llm.listProviders !== 'function' ||
|
|
957
|
+
llm.listProviders().some((info) => (typeof info === 'string' ? info : info?.id) === upstream)
|
|
805
958
|
if (!current) {
|
|
806
959
|
if (claimedElsewhere) {
|
|
807
960
|
if (routed(providerId)) return
|
|
@@ -848,7 +1001,7 @@ function registerVisionProvider(ctx, config, ownProviders, evidenceCache) {
|
|
|
848
1001
|
}
|
|
849
1002
|
reconcile()
|
|
850
1003
|
if (typeof ctx.on === 'function') ctx.on('llm/adapters-updated', reconcile)
|
|
851
|
-
return
|
|
1004
|
+
return deactivate
|
|
852
1005
|
}
|
|
853
1006
|
|
|
854
1007
|
// Auto-discovery. `wrapped` guards duplicates across sweeps and the
|
|
@@ -860,16 +1013,19 @@ function registerVisionProvider(ctx, config, ownProviders, evidenceCache) {
|
|
|
860
1013
|
// one promise chain so two can never interleave their probes at all.
|
|
861
1014
|
const discover = Array.isArray(config.discover) ? new Set(config.discover) : null
|
|
862
1015
|
const sweepOnce = async () => {
|
|
1016
|
+
if (!activationCanCommit()) return
|
|
863
1017
|
try {
|
|
864
1018
|
await sweepBody()
|
|
865
1019
|
} catch (error) {
|
|
1020
|
+
if (!active) return
|
|
866
1021
|
// A sweep failure must never become an unhandled rejection inside the
|
|
867
1022
|
// host process; the next topology notification simply tries again.
|
|
868
1023
|
console.error(`[modlens] vision provider discovery sweep failed: ${error}`)
|
|
869
1024
|
}
|
|
870
1025
|
}
|
|
871
1026
|
const sweepBody = async () => {
|
|
872
|
-
if (
|
|
1027
|
+
if (!active) return
|
|
1028
|
+
if (typeof llm.listProviders !== 'function') {
|
|
873
1029
|
// Older registry surface: fall back to the single legacy wrap once.
|
|
874
1030
|
if (!wrapped.has('__legacy_fallback__')) {
|
|
875
1031
|
wrapped.add('__legacy_fallback__')
|
|
@@ -877,15 +1033,18 @@ function registerVisionProvider(ctx, config, ownProviders, evidenceCache) {
|
|
|
877
1033
|
}
|
|
878
1034
|
return
|
|
879
1035
|
}
|
|
880
|
-
const providers =
|
|
1036
|
+
const providers = llm.listProviders()
|
|
1037
|
+
if (!active) return
|
|
881
1038
|
// Same tolerance as the pinned path: an entry may be a bare id string.
|
|
882
1039
|
const idOf = (info) => (typeof info === 'string' ? info : info?.id)
|
|
883
1040
|
const available = new Set(providers.map(idOf).filter(Boolean))
|
|
884
1041
|
for (const [upstream, current] of registrations) {
|
|
1042
|
+
if (!active) return
|
|
885
1043
|
if (available.has(upstream)) continue
|
|
886
1044
|
dropWrapper(upstream, current)
|
|
887
1045
|
}
|
|
888
1046
|
for (const info of providers) {
|
|
1047
|
+
if (!active) return
|
|
889
1048
|
const id = idOf(info)
|
|
890
1049
|
if (!id || String(id).startsWith('modlens-')) continue
|
|
891
1050
|
if (discover && !discover.has(id)) continue
|
|
@@ -900,13 +1059,19 @@ function registerVisionProvider(ctx, config, ownProviders, evidenceCache) {
|
|
|
900
1059
|
wrapped.add(id)
|
|
901
1060
|
let models = []
|
|
902
1061
|
try {
|
|
903
|
-
models = await
|
|
1062
|
+
models = await llm.listModels(id)
|
|
904
1063
|
} catch {
|
|
1064
|
+
if (!activationCanCommit()) return
|
|
905
1065
|
// Unreachable route today; release the claim so a later topology
|
|
906
1066
|
// change retries it.
|
|
907
1067
|
wrapped.delete(id)
|
|
908
1068
|
continue
|
|
909
1069
|
}
|
|
1070
|
+
// The promise can settle just before Cordis marks this fiber UNLOADING,
|
|
1071
|
+
// while the activation disposer is still one microtask away. Re-enter
|
|
1072
|
+
// the atomic lifecycle boundary before either continuing to refresh a
|
|
1073
|
+
// later registration or committing this provider's wrapper.
|
|
1074
|
+
if (!activationCanCommit()) return
|
|
910
1075
|
if (!models.some(shouldWrap)) {
|
|
911
1076
|
// No eligible models yet: release, the route may gain some later.
|
|
912
1077
|
wrapped.delete(id)
|
|
@@ -931,6 +1096,7 @@ function registerVisionProvider(ctx, config, ownProviders, evidenceCache) {
|
|
|
931
1096
|
void sweep()
|
|
932
1097
|
})
|
|
933
1098
|
}
|
|
1099
|
+
return deactivate
|
|
934
1100
|
}
|
|
935
1101
|
|
|
936
1102
|
// The same pasted attachment rides every later step of its session, and the
|
|
@@ -947,10 +1113,6 @@ function registerVisionProvider(ctx, config, ownProviders, evidenceCache) {
|
|
|
947
1113
|
// text forever.
|
|
948
1114
|
const EVIDENCE_CACHE_LIMIT = 256
|
|
949
1115
|
const EVIDENCE_FAILURE_COOLDOWN_MS = 60_000
|
|
950
|
-
// Monotonic, so an NTP step backwards cannot freeze a cooling failure for
|
|
951
|
-
// the size of the jump: a recovered engine is re-probed one cooldown after
|
|
952
|
-
// the failure, whatever the wall clock did in between.
|
|
953
|
-
const monotonicNow = () => performance.now()
|
|
954
1116
|
|
|
955
1117
|
/**
|
|
956
1118
|
* A cache key that survives replay: the same attachment serialized with a
|
package/package.json
CHANGED
package/skills/modlens/SKILL.md
CHANGED
|
@@ -20,11 +20,11 @@ powershell -ExecutionPolicy Bypass -File <skill-dir>\scripts\run.ps1 <args>
|
|
|
20
20
|
|
|
21
21
|
It resolves a working runtime (PATH `modlens`, then `npx`, then `bunx`) and forwards your arguments unchanged. Exit 78 means no runtime: relay the `nextSteps` from its stderr JSON instead of retrying.
|
|
22
22
|
|
|
23
|
-
If your harness forbids running scripts, reason through the same order by hand and run the first line that works (the pinned version is 3.24.
|
|
23
|
+
If your harness forbids running scripts, reason through the same order by hand and run the first line that works (the pinned version is 3.24.2):
|
|
24
24
|
|
|
25
|
-
1. A `modlens` on `PATH` whose major version is 3 and is at least 3.24.
|
|
26
|
-
2. Otherwise, if `npx` exists: `npx --yes --package @liustack/modlens@3.24.
|
|
27
|
-
3. Otherwise, if `bunx` exists: `bunx --bun @liustack/modlens@3.24.
|
|
25
|
+
1. A `modlens` on `PATH` whose major version is 3 and is at least 3.24.2: `modlens <args>`.
|
|
26
|
+
2. Otherwise, if `npx` exists: `npx --yes --package @liustack/modlens@3.24.2 modlens <args>`.
|
|
27
|
+
3. Otherwise, if `bunx` exists: `bunx --bun @liustack/modlens@3.24.2 <args>`.
|
|
28
28
|
4. Otherwise tell the user no JavaScript runtime was found and that installing Node 22.19+ (https://nodejs.org) or Bun (https://bun.sh) is the next step. Do not claim modlens itself failed.
|
|
29
29
|
|
|
30
30
|
`references/runtime.md` documents the pin and the diagnostic fields.
|
|
@@ -24,7 +24,7 @@ $ErrorActionPreference = 'Stop'
|
|
|
24
24
|
# package.json version, and the release script rewrites it on every bump.
|
|
25
25
|
$Package = '@liustack/modlens'
|
|
26
26
|
$Bin = 'modlens'
|
|
27
|
-
$Pinned = '3.24.
|
|
27
|
+
$Pinned = '3.24.2'
|
|
28
28
|
# -------------------------------------------------------------------------------
|
|
29
29
|
|
|
30
30
|
$NativeNote = 'no native artifact is published for this tool yet; phase A ships npm launch paths only'
|
|
@@ -22,7 +22,7 @@ set -eu
|
|
|
22
22
|
# package.json version, and the release script rewrites it on every bump.
|
|
23
23
|
PKG="@liustack/modlens"
|
|
24
24
|
BIN="modlens"
|
|
25
|
-
PINNED="3.24.
|
|
25
|
+
PINNED="3.24.2"
|
|
26
26
|
# -------------------------------------------------------------------------------
|
|
27
27
|
|
|
28
28
|
NATIVE_NOTE="no native artifact is published for this tool yet; phase A ships npm launch paths only"
|