@liustack/modlens 2.0.0 → 2.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,7 +1,7 @@
1
1
  <div align="center">
2
2
  <img src="https://raw.githubusercontent.com/liustack/modlens/main/assets/banner.jpg" width="100%" alt="ModLens, plug-in vision for text-only LLMs" />
3
3
  <h1>ModLens</h1>
4
- <p><b>Plug-in eyes for text-only LLMs. Free.</b></p>
4
+ <p><b>Free plug-in vision for your text-only LLM.</b></p>
5
5
  <p>
6
6
  <a href="https://www.npmjs.com/package/@liustack/modlens"><img src="https://img.shields.io/npm/v/@liustack/modlens" alt="npm"></a>
7
7
  <a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="License"></a>
@@ -9,12 +9,12 @@
9
9
  <p><a href="./README.zh-CN.md">简体中文</a></p>
10
10
  </div>
11
11
 
12
- DeepSeek-V4-Flash is one of the smartest cheap models you can run, and it's completely blind. Show it a screenshot and it just shrugs. Same dead end for every text-only model wired into Claude Code, OpenClaw, Codex, or any Agent Skills harness.
12
+ DeepSeek-V4-Flash gives you a lot of model for very little money: fast, strong, and its one real flaw is no multimodal. And it's not just DeepSeek. Every text-only model running inside Codex, Claude Code, Pi Agent, or OpenClaw hits the same wall.
13
13
 
14
- One command fixes that. Point ModLens at an image, a local path or a URL, and it hands back structured JSON evidence a text-only model can actually reason over: OCR text, layout regions in reading order, entities, relations, visual clues. The seeing itself happens in [Antigravity CLI](https://antigravity.google) (`agy`), so it runs on Google's free quota, not your API bill.
14
+ ModLens fixes this the lightest way possible. It never touches your config and never adds a local proxy. It's just a vision plug-in, usable as a CLI or as an Agent Skill, that turns any image into structured visual evidence: text, layout, regions, entities, relations, visual clues. Under the hood it runs on [Antigravity CLI](https://antigravity.google) (`agy`), whose vision comes from free-quota Gemini 3.6 Flash. And Gemini's image understanding is famously good, good enough to embarrass most flagships, Fable 5 included. How it works:
15
15
 
16
16
  ```text
17
- your text-only model ──▶ modlens skill (auto-triggers on images)
17
+ text-only model in your agent harness ──▶ modlens skill (auto-triggers on images)
18
18
  │
19
19
  ▼
20
20
  agy · Gemini 3.6 Flash (free quota)
@@ -23,8 +23,6 @@ your text-only model ──▶ modlens skill (auto-triggers on images)
23
23
  structured JSON evidence ──▶ model answers with sight
24
24
  ```
25
25
 
26
- Install the skill once and your agent starts handling images on its own. No model swap, no API key, no prompt surgery.
27
-
28
26
  ## Quick start
29
27
 
30
28
  **1. Install Antigravity CLI and sign in** (one-time):
@@ -34,7 +32,7 @@ curl -fsSL https://antigravity.google/cli/install.sh | bash
34
32
  agy # opens browser sign-in, then exit
35
33
  ```
36
34
 
37
- **2. Install the skill.** Tell your agent (Claude Code, Codex, OpenClaw, Cursor, ...):
35
+ **2. Install the skill.** Just tell your agent (Claude Code, Codex, OpenClaw, Cursor, ...):
38
36
 
39
37
  ```text
40
38
  Install the skill from https://github.com/liustack/modlens
@@ -46,7 +44,7 @@ or do it yourself:
46
44
  npx -y skills add liustack/modlens
47
45
  ```
48
46
 
49
- **3. Use it.** Drop an image path into the chat and ask anything. The skill fires automatically whenever your model needs eyes.
47
+ **3. Use it.** Paste an image path into the CLI and ask anything. The skill fires on its own.
50
48
 
51
49
  ## See it work
52
50
 
@@ -69,14 +67,28 @@ Real output, truncated:
69
67
  { "language": "en", "text": "BEFORE YOU BUILD" }
70
68
  ]
71
69
  },
72
- "layout": { "regions": [ { "reading_order": 1, "text": "/shaping BEFORE YOU BUILD", "type": "other" } ] },
70
+ "layout": {
71
+ "regions": [
72
+ {
73
+ "reading_order": 1,
74
+ "text": "/shaping BEFORE YOU BUILD",
75
+ "type": "other"
76
+ }
77
+ ]
78
+ },
73
79
  "uncertainty": []
74
80
  },
75
81
  "meta": { "model": "gemini-3.6-flash-low", "durationSeconds": 25.4 }
76
82
  }
77
83
  ```
78
84
 
79
- A run takes 15-40 seconds. The JSON shape is locked in by a schema at the provider level, so your agent never has to fish JSON out of markdown again.
85
+ Here is the whole loop inside the Codex desktop app: drop in a tweet screenshot, and a text-only DeepSeek-V4-Flash reads all of it through ModLens: the caption, the engagement numbers (2.9K replies, 270K likes, 5M views), even the image's alt text. Where the resolution runs out, it says so instead of guessing.
86
+
87
+ ![Text-only DeepSeek reading a tweet screenshot in full detail via ModLens](https://raw.githubusercontent.com/liustack/modlens/main/assets/demo-codex-app.png)
88
+
89
+ Batch mode works too: drop three illustrations at once, and the model announces it will read them one by one through ModLens, then delivers all three descriptions in 21 seconds, design intent included.
90
+
91
+ ![Text-only DeepSeek reading three images in one go via ModLens](https://raw.githubusercontent.com/liustack/modlens/main/assets/demo-codex-batch.png)
80
92
 
81
93
  ## CLI reference
82
94
 
@@ -97,16 +109,36 @@ modlens -i <image-path-or-url> [options]
97
109
 
98
110
  Reach for `-m gemini-3.1-pro-high` on dense screenshots or tricky documents. Output contract: [skills/modlens/references/output-schema.md](skills/modlens/references/output-schema.md).
99
111
 
112
+ ## Providers and config
113
+
114
+ ModLens ships four vision providers. `antigravity-cli` stays the default: zero keys, pure free quota.
115
+
116
+ | Provider | Needs | Typical speed | Notes |
117
+ | :-- | :-- | :-- | :-- |
118
+ | `antigravity-cli` (default) | `agy` signed in | 15-40s | free quota, full agent loop |
119
+ | `gemini-api` | free AI Studio key | 5-10s | fastest free route, schema enforced server-side |
120
+ | `openai` | baseUrl + apiKey + model | endpoint-dependent | any OpenAI-compatible multimodal endpoint (qwen-vl, GLM, ...) |
121
+ | `anthropic` | `ANTHROPIC_API_KEY` | a few seconds | Claude Haiku by default, schema via forced tool call |
122
+
123
+ Config lives in `~/.modlens/config.json`. Environment variables override the file (`GEMINI_API_KEY`, `OPENAI_API_KEY`, `OPENAI_BASE_URL`, `ANTHROPIC_API_KEY`), and CLI flags override everything.
124
+
125
+ ```bash
126
+ modlens config init # write a starter config
127
+ modlens config set gemini-api.apiKey <key> # saved with 0600 perms
128
+ modlens config show # keys come out masked
129
+ modlens config set provider gemini-api # switch the default provider
130
+ ```
131
+
132
+ The free Gemini key takes three minutes at [aistudio.google.com](https://aistudio.google.com), no credit card. Or skip the manual work entirely and tell your agent: "configure modlens with my Gemini API key".
133
+
100
134
  ## Using it in Codex (DeepSeek and friends)
101
135
 
102
- Codex speaks only the Responses API, and DeepSeek's official endpoint supports it natively. Start with the [official integration guide](https://api-docs.deepseek.com/quick_start/agent_integrations/codex/): its `models.json` declares deepseek-v4-flash as text-only (`input_modalities: ["text"]`), and that one line is what unlocks everything below.
136
+ Codex speaks only the Responses API, and DeepSeek's official endpoint supports it natively. Start with the [official integration guide](https://api-docs.deepseek.com/quick_start/agent_integrations/codex/): its `models.json` declares deepseek-v4-flash as text-only (`input_modalities: ["text"]`), and that one line is what unlocks the whole flow.
103
137
 
104
138
  One catch: once text-only is declared, the Codex TUI **blocks Ctrl+V image paste outright** (`Model deepseek-v4-flash does not support image inputs`). The gate sits in the input box itself, so the image never makes it into the message. Two moves get around it, both verified end to end with deepseek-v4-flash:
105
139
 
106
140
  - **Drag the image file into the terminal**, or type its path. The path lands as plain text, and the modlens skill picks it up from there.
107
- - Attach it with `codex exec -i image.png "..."`. Codex strips the pixels in core but leaves a `<image name=[Image #1] path="/tmp/....png">` text tag behind, and the skill reads the path out of that tag.
108
-
109
- Skip `models.json` (a bare custom-model config) and Codex assumes your model can see images, sending them raw, and whether that survives depends on the provider's patience. Dragging the file in is the one move that works everywhere, in every harness.
141
+ - Attach it with `codex exec -i image.png "..."`. The skill reads the path out of the message tag.
110
142
 
111
143
  ## Why a bridge instead of a multimodal model?
112
144
 
@@ -116,22 +148,22 @@ Skip `models.json` (a bare custom-model config) and Codex assumes your model can
116
148
 
117
149
  ModSearch, ModLens's sibling project, plays the same trick for web search and page fetching: [liustack/modsearch](https://github.com/liustack/modsearch).
118
150
 
119
- ## Built with liustack
151
+ ## Shameless plug
120
152
 
121
- ModLens v2 was shaped, coded, and shipped with **[liustack](https://github.com/liustack/liustack)**. Four Agent Skills, one loop: `shaping` before you build, `coding` while you build, `dig` when it breaks, `snapshot` when you hand off. A lighter, sharper alternative to Superpowers.
153
+ This project runs on LIUSTACK Skills. ModLens v2 was shaped, coded, and shipped with **[liustack](https://github.com/liustack/liustack)** end to end: `shaping` before you build, `coding` while you build, `dig` when it breaks, `snapshot` when you hand off. Lighter than Superpowers, and sharper.
122
154
 
123
- **ModLens gave your model eyes. liustack gives your whole workflow discipline:**
155
+ **ModLens gives your model eyes. LIUSTACK Skills gives your dev workflow wings:**
124
156
 
125
157
  ```bash
126
158
  npx -y skills add liustack/liustack -g
127
159
  ```
128
160
 
129
- ⭐ Like the idea? [Star ModLens](https://github.com/liustack/modlens) and [star liustack](https://github.com/liustack/liustack). Stars are how the next developer finds them.
161
+ ⭐ Like it? [Star ModLens](https://github.com/liustack/modlens) and [star liustack](https://github.com/liustack/liustack). Stars are how the next developer finds them.
130
162
 
131
163
  ## Security notes
132
164
 
133
- - ModLens runs `agy` with `--dangerously-skip-permissions`, because print mode skips every tool call without it. The prompt keeps the agent to reading that one image and tells it to treat image content as data, never as instructions. Even so, only point it at images you would open yourself, and run it inside a sandboxed workspace when you can.
134
- - Vision output is evidence, not gospel. Anything the engine cannot read lands in `uncertainty` instead of getting invented. Pixel bounding boxes and confidence scores were dropped in v2 because models fabricate them.
165
+ - ModLens runs `agy` with `--dangerously-skip-permissions`, because print mode can fail in some setups without it. The prompt keeps the agent to reading that one image and tells it to treat image content as data, never as instructions. Even so, only point it at images you would open yourself, and run it inside a sandboxed workspace when you can.
166
+ - Vision output is evidence. Anything the engine cannot read lands in `uncertainty` instead of getting invented. Pixel bounding boxes and confidence scores were dropped in v2 because models fabricate them.
135
167
 
136
168
  ## Disclaimer
137
169
 
package/README.zh-CN.md CHANGED
@@ -1,7 +1,7 @@
1
1
  <div align="center">
2
2
  <img src="https://raw.githubusercontent.com/liustack/modlens/main/assets/banner.jpg" width="100%" alt="ModLens,给纯文本 LLM 外挂视觉" />
3
3
  <h1>ModLens</h1>
4
- <p><b>给纯文本 LLM 外挂一双眼睛,免费。</b></p>
4
+ <p><b>免费给你的大语言模型(纯文本 LLM)外挂视觉能力。</b></p>
5
5
  <p>
6
6
  <a href="https://www.npmjs.com/package/@liustack/modlens"><img src="https://img.shields.io/npm/v/@liustack/modlens" alt="npm"></a>
7
7
  <a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="License"></a>
@@ -9,12 +9,12 @@
9
9
  <p><a href="./README.md">English</a></p>
10
10
  </div>
11
11
 
12
- DeepSeek-V4-Flash 便宜又能打,可惜是个瞎子。甩一张截图过去,只换来两手一摊。Claude Code、OpenClaw、Codex,随便哪个 Agent Skills 宿主里跑着的纯文本模型,都卡在这同一个死结上。
12
+ DeepSeek-V4-Flash 碗大又好吃,速度快,性能强,要说唯一的缺点就是没有多模态。不仅 DeepSeek-V4-Flash,只要是纯文本语言模型,跑在 Codex、Claude Code、Pi Agent、OpenClaw 中,都有这个问题。
13
13
 
14
- 一条命令解开这个死结。把 ModLens 指向任意图片(本地路径或 URL),它吐出纯文本模型真正用得上的结构化 JSON 证据:OCR 文字、按阅读顺序排好的版面区块、实体、关系、视觉线索。真正「看」图这件事,交给 [Antigravity CLI](https://antigravity.google)(`agy`)去干,用的是 Google 的免费额度,不碰你的 API 账单。
14
+ ModLens 用最轻量级方案解决这个问题。ModLens 不会入侵你的配置,也不会给你添加本地代理,ModLens 只是一个视觉外挂,有 cli 或 skill 两种模式。ModLens 能产出结构化的视觉证据:文字、版面、区块、实体、关系、视觉线索。ModLens 由 Antigravity [Antigravity CLI](https://antigravity.google)(`agy`)驱动,而 Antigravity 的视觉由免费额度的 Gemini 3.6 Flash 驱动。众所周知,Gemini 的识图能力,连 Fable 5 都吊打。原理如下:
15
15
 
16
16
  ```text
17
- 你的纯文本模型 ──▶ modlens skill(遇到图片自动触发)
17
+ Agent Harness 中的纯文本模型 ──▶ modlens skill(遇到图片自动触发)
18
18
  │
19
19
  ▼
20
20
  agy · Gemini 3.6 Flash(免费额度)
@@ -23,8 +23,6 @@ DeepSeek-V4-Flash 便宜又能打,可惜是个瞎子。甩一张截图过去
23
23
  结构化 JSON 证据 ──▶ 模型带着视力回答
24
24
  ```
25
25
 
26
- skill 装一次,你的 agent 以后见着图片自己就处理了。模型不用换,API key 不用要,提示词也不用改。
27
-
28
26
  ## 快速开始
29
27
 
30
28
  **1. 安装 Antigravity CLI 并登录**(一次性):
@@ -37,7 +35,7 @@ agy # 浏览器完成登录后退出
37
35
  **2. 安装 skill。** 直接告诉你的 agent(Claude Code、Codex、OpenClaw、Cursor 等):
38
36
 
39
37
  ```text
40
- Install the skill from https://github.com/liustack/modlens
38
+ 安装这个 skill https://github.com/liustack/modlens
41
39
  ```
42
40
 
43
41
  或者自己动手:
@@ -46,7 +44,7 @@ Install the skill from https://github.com/liustack/modlens
46
44
  npx -y skills add liustack/modlens
47
45
  ```
48
46
 
49
- **3. 用起来。** 往对话里丢一个图片路径,随便问。模型需要眼睛的时候,skill 自动触发。
47
+ **3. 用起来。** 在 cli 里粘贴个图片路径,随便问,skill 会自动触发。
50
48
 
51
49
  ## 看看效果
52
50
 
@@ -69,14 +67,28 @@ npx @liustack/modlens -i workflow.jpg
69
67
  { "language": "en", "text": "BEFORE YOU BUILD" }
70
68
  ]
71
69
  },
72
- "layout": { "regions": [ { "reading_order": 1, "text": "/shaping BEFORE YOU BUILD", "type": "other" } ] },
70
+ "layout": {
71
+ "regions": [
72
+ {
73
+ "reading_order": 1,
74
+ "text": "/shaping BEFORE YOU BUILD",
75
+ "type": "other"
76
+ }
77
+ ]
78
+ },
73
79
  "uncertainty": []
74
80
  },
75
81
  "meta": { "model": "gemini-3.6-flash-low", "durationSeconds": 25.4 }
76
82
  }
77
83
  ```
78
84
 
79
- 一次运行 15-40 秒。JSON 结构由 provider 层的 schema 硬性保证,你的 agent 不用再从 markdown 里抠 JSON 出来。
85
+ 整条链路在 Codex 桌面 App 里跑起来是这样:丢一张推文截图,纯文本的 DeepSeek-V4-Flash 通过 ModLens 读出了全部内容:配文、互动数据(2.9K 回复、270K 点赞、5M 浏览)、连图片的 alt 文字都没放过。分辨率不够的地方,它老实说读不清,不瞎编。
86
+
87
+ ![纯文本 DeepSeek 通过 ModLens 读出推文截图的全部细节](https://raw.githubusercontent.com/liustack/modlens/main/assets/demo-codex-app.png)
88
+
89
+ 批量也不在话下:一次丢三张插画进去,模型自己说「三张图我都用 modlens 视觉桥接逐张读取」,21 秒后逐张交卷,连画面的设计意图都点出来了。
90
+
91
+ ![纯文本 DeepSeek 通过 ModLens 一次读完三张图](https://raw.githubusercontent.com/liustack/modlens/main/assets/demo-codex-batch.png)
80
92
 
81
93
  ## CLI 参数
82
94
 
@@ -84,19 +96,41 @@ npx @liustack/modlens -i workflow.jpg
84
96
  modlens -i <图片路径或 URL> [选项]
85
97
  ```
86
98
 
87
- | 参数 | 含义 | 默认值 |
88
- | :-- | :-- | :-- |
89
- | `-i, --input <path\|url>` | 要解析的图片(必填) | |
90
- | `-o, --output <path>` | 同时把 JSON 写入文件 | |
91
- | `-m, --model <name>` | provider 模型 | `gemini-3.6-flash-low` |
92
- | `-p, --provider <name>` | 视觉 provider | `antigravity-cli` |
93
- | `--prompt <text>` | 额外关注点,比如 `"重点提取表格"` | |
94
- | `--timeout <ms>` | provider 超时 | `180000` |
95
- | `--provider-bin <path>` | provider 可执行文件 | `agy` |
96
- | `--workdir <path>` | provider 运行目录 | |
99
+ | 参数 | 含义 | 默认值 |
100
+ | :------------------------ | :-------------------------------- | :--------------------- |
101
+ | `-i, --input <path\|url>` | 要解析的图片(必填) | |
102
+ | `-o, --output <path>` | 同时把 JSON 写入文件 | |
103
+ | `-m, --model <name>` | provider 模型 | `gemini-3.6-flash-low` |
104
+ | `-p, --provider <name>` | 视觉 provider | `antigravity-cli` |
105
+ | `--prompt <text>` | 额外关注点,比如 `"重点提取表格"` | |
106
+ | `--timeout <ms>` | provider 超时 | `180000` |
107
+ | `--provider-bin <path>` | provider 可执行文件 | `agy` |
108
+ | `--workdir <path>` | provider 运行目录 | |
97
109
 
98
110
  截图信息密集或文档难啃,换成 `-m gemini-3.1-pro-high`。输出契约见 [skills/modlens/references/output-schema.md](skills/modlens/references/output-schema.md)。
99
111
 
112
+ ## Provider 与配置
113
+
114
+ ModLens 内置四个视觉 provider,默认还是 `antigravity-cli`:零 key,纯免费额度。
115
+
116
+ | Provider | 需要什么 | 速度 | 说明 |
117
+ | :-- | :-- | :-- | :-- |
118
+ | `antigravity-cli`(默认) | `agy` 登录过 | 15-40 秒 | 免费额度,完整 agent 循环 |
119
+ | `gemini-api` | 免费 AI Studio key | 5-10 秒 | 最快的免费路线,服务端强制 schema |
120
+ | `openai` | baseUrl + apiKey + model | 看端点 | 任何 OpenAI 兼容的多模态端点(qwen-vl、GLM 等) |
121
+ | `anthropic` | `ANTHROPIC_API_KEY` | 几秒 | 默认 Claude Haiku,强制工具调用保 schema |
122
+
123
+ 配置放在 `~/.modlens/config.json`,环境变量能盖过它(`GEMINI_API_KEY`、`OPENAI_API_KEY`、`OPENAI_BASE_URL`、`ANTHROPIC_API_KEY`),CLI 参数最大。
124
+
125
+ ```bash
126
+ modlens config init # 生成配置骨架
127
+ modlens config set gemini-api.apiKey <key> # 落盘即 0600 权限
128
+ modlens config show # key 打码显示
129
+ modlens config set provider gemini-api # 换默认 provider
130
+ ```
131
+
132
+ 免费 Gemini key 去 [aistudio.google.com](https://aistudio.google.com) 领,三分钟,不要信用卡。嫌麻烦就直接跟你的 agent 说一句「帮我把 Gemini key 配进 modlens」,让它自己跑命令。
133
+
100
134
  ## 在 Codex 里用(DeepSeek 等纯文本模型)
101
135
 
102
136
  Codex 只认 Responses API,DeepSeek 官方端点原生支持。先照着[官方集成文档](https://api-docs.deepseek.com/zh-cn/quick_start/agent_integrations/codex)配好:它的 `models.json` 把 deepseek-v4-flash 声明成纯文本(`input_modalities: ["text"]`),这一行就是打通下面整条链路的钥匙。
@@ -104,9 +138,7 @@ Codex 只认 Responses API,DeepSeek 官方端点原生支持。先照着[官
104
138
  有个坑要注意:声明纯文本之后,Codex TUI 会**直接拦下 Ctrl+V 粘贴图片**(报错 `Model deepseek-v4-flash does not support image inputs`),闸门卡在输入框那一层,图片压根到不了消息里。能用的招数有两个,都拿 deepseek-v4-flash 端到端验证过:
105
139
 
106
140
  - **把图片文件拖进终端**,或者手打路径。路径以纯文本形式落进消息,modlens skill 接着从这里接手。
107
- - 用 `codex exec -i 图片.png "..."` 附件方式:Codex 在内核层就把像素剥掉了,但留了一个 `<image name=[Image #1] path="/tmp/....png">` 文本标签,skill 从这个标签里把路径抠出来。
108
-
109
- 不配 `models.json`(裸的自定义模型配置)的话,Codex 会默认你的模型能看图,把图片原样发过去,能不能扛住全看服务商脾气好不好。拖文件这一招,换到哪个宿主都稳。
141
+ - 用 `codex exec -i 图片.png "..."` skill 从这里把路径抠出来。
110
142
 
111
143
  ## 为什么外挂,而不是换多模态模型?
112
144
 
@@ -116,11 +148,11 @@ Codex 只认 Responses API,DeepSeek 官方端点原生支持。先照着[官
116
148
 
117
149
  姊妹项目 ModSearch 用同一招补上联网搜索和网页抓取:[liustack/modsearch](https://github.com/liustack/modsearch)。
118
150
 
119
- ## 用 liustack 打造
151
+ ## 插入一条硬广告
120
152
 
121
- ModLens v2 从需求成形、编码到交付,全程用 **[liustack](https://github.com/liustack/liustack)** 跑完。四个 Agent Skills,一个闭环:动手前 `shaping` 捋清楚,编码时 `coding` 上纪律,出问题 `dig` 挖根因,交接时 `snapshot` 留快照。比 Superpowers 更轻,也更锋利。
153
+ 本项目由 LIUSTACK Skills 驱动,ModLens v2 从需求成形、编码到交付,全程用 **[liustack](https://github.com/liustack/liustack)** 驱动:动手前 `shaping` 捋清楚,编码时 `coding` 上纪律,出问题 `dig` 挖根因,交接时 `snapshot` 留快照。比 Superpowers 更轻,也更强。
122
154
 
123
- **ModLens 给你的模型装上眼睛,liustack 给你的整个工作流装上纪律:**
155
+ **ModLens 给你的模型装上眼睛,LIUSTACK Skills 给你的开发工作流装上翅膀:**
124
156
 
125
157
  ```bash
126
158
  npx -y skills add liustack/liustack -g
@@ -130,8 +162,8 @@ npx -y skills add liustack/liustack -g
130
162
 
131
163
  ## 安全说明
132
164
 
133
- - ModLens 调用 `agy` 时带上 `--dangerously-skip-permissions`,因为 print 模式不带这个参数就不执行工具调用。提示词已经把 agent 限定在只读那一张图,并要求把图片内容当数据看,绝不当指令执行。即便如此,也只解析你自己敢打开的图片,尽量在沙箱化的工作目录里跑。
134
- - 视觉输出是证据,不是圣旨。引擎读不准的地方会进 `uncertainty`,而不是被编出来凑数。v2 把像素坐标和置信度分数都删了,因为模型会瞎编这两样。
165
+ - ModLens 调用 `agy` 时带上 `--dangerously-skip-permissions`,因为 prompt/print 模式不带这个参数在某些场景会失败。提示词已经把 agent 限定在只读那一张图,并要求把图片内容当数据看,绝不当指令执行。即便如此,也只解析你自己敢打开的图片,尽量在沙箱化的工作目录里跑。
166
+ - 视觉输出是证据,引擎读不准的地方会进 `uncertainty`,而不是被编出来凑数。v2 把像素坐标和置信度分数都删了,因为模型会瞎编这两样。
135
167
 
136
168
  ## 免责声明
137
169
 
package/dist/main.js CHANGED
@@ -3,8 +3,120 @@ import { Command } from "commander";
3
3
  import * as fs from "fs";
4
4
  import * as path from "path";
5
5
  import { spawn } from "child_process";
6
+ import * as os from "os";
7
+ const CONFIG_DIR = path.join(os.homedir(), ".modlens");
8
+ const CONFIG_PATH = path.join(CONFIG_DIR, "config.json");
9
+ const ENV_BINDINGS = {
10
+ "gemini-api": { apiKey: "GEMINI_API_KEY" },
11
+ openai: { apiKey: "OPENAI_API_KEY", baseUrl: "OPENAI_BASE_URL" },
12
+ anthropic: { apiKey: "ANTHROPIC_API_KEY", baseUrl: "ANTHROPIC_BASE_URL" }
13
+ };
14
+ function loadConfigFile(configPath = CONFIG_PATH) {
15
+ let raw;
16
+ try {
17
+ raw = fs.readFileSync(configPath, "utf-8");
18
+ } catch {
19
+ return {};
20
+ }
21
+ try {
22
+ const parsed = JSON.parse(raw);
23
+ if (!parsed || typeof parsed !== "object") {
24
+ return {};
25
+ }
26
+ return parsed;
27
+ } catch (error) {
28
+ throw new Error(
29
+ `Failed to parse ${configPath}: ${error.message}. Fix or delete the file.`
30
+ );
31
+ }
32
+ }
33
+ function defaultProviderName(config2) {
34
+ return config2.provider?.trim() || "antigravity-cli";
35
+ }
36
+ function resolveProviderSettings(providerName, config2, env = process.env) {
37
+ const fromFile = config2.providers?.[providerName] ?? {};
38
+ const bindings = ENV_BINDINGS[providerName] ?? {};
39
+ const settings = { ...fromFile };
40
+ for (const [field, envName] of Object.entries(bindings)) {
41
+ const value = env[envName]?.trim();
42
+ if (value) {
43
+ settings[field] = value;
44
+ }
45
+ }
46
+ return settings;
47
+ }
48
+ function setConfigValue(dottedKey, value, configPath = CONFIG_PATH) {
49
+ const config2 = loadConfigFile(configPath);
50
+ if (dottedKey === "provider") {
51
+ config2.provider = value;
52
+ } else {
53
+ const dot = dottedKey.indexOf(".");
54
+ if (dot <= 0 || dot === dottedKey.length - 1) {
55
+ throw new Error(
56
+ `Invalid config key: ${dottedKey}. Use "provider" or "<provider>.<apiKey|baseUrl|model>".`
57
+ );
58
+ }
59
+ const providerName = dottedKey.slice(0, dot);
60
+ const field = dottedKey.slice(dot + 1);
61
+ if (!["apiKey", "baseUrl", "model"].includes(field)) {
62
+ throw new Error(`Unknown config field: ${field}. Use apiKey, baseUrl, or model.`);
63
+ }
64
+ config2.providers ??= {};
65
+ config2.providers[providerName] ??= {};
66
+ config2.providers[providerName][field] = value;
67
+ }
68
+ fs.mkdirSync(path.dirname(configPath), { recursive: true });
69
+ fs.writeFileSync(configPath, `${JSON.stringify(config2, null, 2)}
70
+ `, { mode: 384 });
71
+ try {
72
+ fs.chmodSync(configPath, 384);
73
+ } catch {
74
+ }
75
+ }
76
+ const CONFIG_TEMPLATE = {
77
+ provider: "antigravity-cli",
78
+ providers: {
79
+ "antigravity-cli": { model: "gemini-3.6-flash-low" },
80
+ "gemini-api": { apiKey: "", model: "gemini-3.6-flash" },
81
+ openai: { baseUrl: "", apiKey: "", model: "" },
82
+ anthropic: { apiKey: "", model: "claude-haiku-4-5-20251001" }
83
+ }
84
+ };
85
+ function initConfigFile(configPath = CONFIG_PATH, force = false) {
86
+ if (!force && fs.existsSync(configPath)) {
87
+ throw new Error(`${configPath} already exists. Use --force to overwrite.`);
88
+ }
89
+ fs.mkdirSync(path.dirname(configPath), { recursive: true });
90
+ fs.writeFileSync(configPath, `${JSON.stringify(CONFIG_TEMPLATE, null, 2)}
91
+ `, { mode: 384 });
92
+ try {
93
+ fs.chmodSync(configPath, 384);
94
+ } catch {
95
+ }
96
+ }
97
+ function renderConfig(config2) {
98
+ const masked = {
99
+ ...config2,
100
+ providers: Object.fromEntries(
101
+ Object.entries(config2.providers ?? {}).map(([name, settings]) => [
102
+ name,
103
+ {
104
+ ...settings,
105
+ ...settings.apiKey ? { apiKey: maskKey(settings.apiKey) } : {}
106
+ }
107
+ ])
108
+ )
109
+ };
110
+ return JSON.stringify(masked, null, 2);
111
+ }
112
+ function maskKey(key) {
113
+ if (key.length <= 8) {
114
+ return "****";
115
+ }
116
+ return `${key.slice(0, 6)}...${key.slice(-2)}`;
117
+ }
6
118
  function buildVisionPrompt(options) {
7
- const readInstruction = options.imageKind === "remote" ? `Fetch the image at this URL and analyze it: ${options.imageSource}` : `Read the image file at this path and analyze it: ${options.imageSource}`;
119
+ const readInstruction = options.imageKind === "inline" ? "Analyze the image attached to this message." : options.imageKind === "remote" ? `Fetch the image at this URL and analyze it: ${options.imageSource}` : `Read the image file at this path and analyze it: ${options.imageSource}`;
8
120
  const basePrompt = `${readInstruction}
9
121
 
10
122
  You are a vision parsing engine for a text-only LLM.
@@ -204,19 +316,320 @@ const antigravityCliProvider = {
204
316
  buildInvocation: buildAntigravityInvocation,
205
317
  parseOutput: parseAntigravityOutput
206
318
  };
319
+ const MIME_BY_EXT = {
320
+ ".jpg": "image/jpeg",
321
+ ".jpeg": "image/jpeg",
322
+ ".png": "image/png",
323
+ ".webp": "image/webp",
324
+ ".gif": "image/gif",
325
+ ".heic": "image/heic",
326
+ ".heif": "image/heif"
327
+ };
328
+ function mimeTypeFor(source) {
329
+ const ext = path.extname(new URL(source, "file:///").pathname).toLowerCase();
330
+ return MIME_BY_EXT[ext] ?? "image/jpeg";
331
+ }
332
+ function readLocalImageBase64(filePath) {
333
+ const data = fs.readFileSync(filePath).toString("base64");
334
+ return { data, mimeType: mimeTypeFor(filePath) };
335
+ }
336
+ async function fetchRemoteImageBase64(url, timeoutMs) {
337
+ const response = await fetch(url, { signal: AbortSignal.timeout(timeoutMs) });
338
+ if (!response.ok) {
339
+ throw new Error(`Failed to download image (${response.status}): ${url}`);
340
+ }
341
+ const buffer = Buffer.from(await response.arrayBuffer());
342
+ const contentType = response.headers.get("content-type")?.split(";")[0]?.trim();
343
+ return {
344
+ data: buffer.toString("base64"),
345
+ mimeType: contentType?.startsWith("image/") ? contentType : mimeTypeFor(url)
346
+ };
347
+ }
348
+ function extractJson(text) {
349
+ const trimmed = text.trim();
350
+ const direct = tryParse(trimmed);
351
+ if (direct !== null) {
352
+ return direct;
353
+ }
354
+ const fenced = /```(?:json)?\s*([\s\S]*?)```/i.exec(trimmed);
355
+ if (fenced) {
356
+ const parsed = tryParse(fenced[1].trim());
357
+ if (parsed !== null) {
358
+ return parsed;
359
+ }
360
+ }
361
+ const first = trimmed.indexOf("{");
362
+ const last = trimmed.lastIndexOf("}");
363
+ if (first >= 0 && last > first) {
364
+ return tryParse(trimmed.slice(first, last + 1));
365
+ }
366
+ return null;
367
+ }
368
+ function tryParse(text) {
369
+ try {
370
+ return JSON.parse(text);
371
+ } catch {
372
+ return null;
373
+ }
374
+ }
375
+ const ANTHROPIC_DEFAULT_MODEL = "claude-haiku-4-5-20251001";
376
+ const DEFAULT_BASE_URL$1 = "https://api.anthropic.com";
377
+ const TOOL_NAME = "report_vision_evidence";
378
+ async function executeAnthropicApi(options) {
379
+ const apiKey = options.settings?.apiKey;
380
+ if (!apiKey) {
381
+ throw new Error(
382
+ "anthropic provider needs an API key. Set ANTHROPIC_API_KEY, or run: modlens config set anthropic.apiKey <key>"
383
+ );
384
+ }
385
+ const model = options.model || options.settings?.model || ANTHROPIC_DEFAULT_MODEL;
386
+ const baseUrl = (options.settings?.baseUrl || DEFAULT_BASE_URL$1).replace(/\/$/, "");
387
+ const imageSource = options.imageKind === "remote" ? { type: "url", url: options.imageSource } : (() => {
388
+ const image = readLocalImageBase64(options.imageSource);
389
+ return {
390
+ type: "base64",
391
+ media_type: image.mimeType,
392
+ data: image.data
393
+ };
394
+ })();
395
+ const prompt = `${buildVisionPrompt({
396
+ imageSource: options.imageSource,
397
+ imageKind: "inline",
398
+ extraPrompt: options.extraPrompt
399
+ })}
400
+
401
+ Report your findings by calling the ${TOOL_NAME} tool.`;
402
+ const startedAt = Date.now();
403
+ const response = await fetch(`${baseUrl}/v1/messages`, {
404
+ method: "POST",
405
+ headers: {
406
+ "x-api-key": apiKey,
407
+ "anthropic-version": "2023-06-01",
408
+ "Content-Type": "application/json"
409
+ },
410
+ body: JSON.stringify({
411
+ model,
412
+ max_tokens: 4096,
413
+ tools: [
414
+ {
415
+ name: TOOL_NAME,
416
+ description: "Report the structured visual evidence extracted from the image.",
417
+ input_schema: VISION_RESULT_SCHEMA
418
+ }
419
+ ],
420
+ tool_choice: { type: "tool", name: TOOL_NAME },
421
+ messages: [
422
+ {
423
+ role: "user",
424
+ content: [
425
+ { type: "image", source: imageSource },
426
+ { type: "text", text: prompt }
427
+ ]
428
+ }
429
+ ]
430
+ }),
431
+ signal: AbortSignal.timeout(options.timeoutMs)
432
+ });
433
+ if (!response.ok) {
434
+ const body = await response.text();
435
+ throw new Error(`Anthropic API error ${response.status}: ${truncate$2(body)}`);
436
+ }
437
+ const payload = await response.json();
438
+ const toolUse = payload.content?.find((block) => block.type === "tool_use");
439
+ if (!toolUse?.input) {
440
+ throw new Error("Anthropic API returned no tool_use block.");
441
+ }
442
+ return {
443
+ result: toolUse.input,
444
+ meta: {
445
+ conversationId: null,
446
+ durationSeconds: (Date.now() - startedAt) / 1e3,
447
+ usage: payload.usage ?? null
448
+ }
449
+ };
450
+ }
451
+ function truncate$2(text) {
452
+ return text.length > 300 ? `${text.slice(0, 300)}...` : text;
453
+ }
454
+ const anthropicApiProvider = {
455
+ name: "anthropic",
456
+ defaultModel: ANTHROPIC_DEFAULT_MODEL,
457
+ execute: executeAnthropicApi
458
+ };
459
+ const GEMINI_API_DEFAULT_MODEL = "gemini-3.6-flash";
460
+ const DEFAULT_BASE_URL = "https://generativelanguage.googleapis.com";
461
+ async function executeGeminiApi(options) {
462
+ const apiKey = options.settings?.apiKey;
463
+ if (!apiKey) {
464
+ throw new Error(
465
+ "gemini-api provider needs an API key. Set GEMINI_API_KEY, or run: modlens config set gemini-api.apiKey <key> (free key: https://aistudio.google.com)"
466
+ );
467
+ }
468
+ const model = options.model || options.settings?.model || GEMINI_API_DEFAULT_MODEL;
469
+ const baseUrl = (options.settings?.baseUrl || DEFAULT_BASE_URL).replace(/\/$/, "");
470
+ const image = options.imageKind === "remote" ? await fetchRemoteImageBase64(options.imageSource, options.timeoutMs) : readLocalImageBase64(options.imageSource);
471
+ const prompt = buildVisionPrompt({
472
+ imageSource: options.imageSource,
473
+ imageKind: "inline",
474
+ extraPrompt: options.extraPrompt
475
+ });
476
+ const startedAt = Date.now();
477
+ const response = await fetch(`${baseUrl}/v1beta/models/${model}:generateContent`, {
478
+ method: "POST",
479
+ headers: {
480
+ "x-goog-api-key": apiKey,
481
+ "Content-Type": "application/json"
482
+ },
483
+ body: JSON.stringify({
484
+ contents: [
485
+ {
486
+ parts: [
487
+ { inline_data: { mime_type: image.mimeType, data: image.data } },
488
+ { text: prompt }
489
+ ]
490
+ }
491
+ ],
492
+ generationConfig: {
493
+ responseMimeType: "application/json",
494
+ responseJsonSchema: VISION_RESULT_SCHEMA
495
+ }
496
+ }),
497
+ signal: AbortSignal.timeout(options.timeoutMs)
498
+ });
499
+ if (!response.ok) {
500
+ const body = await response.text();
501
+ throw new Error(`Gemini API error ${response.status}: ${truncate$1(body)}`);
502
+ }
503
+ const payload = await response.json();
504
+ const text = payload.candidates?.[0]?.content?.parts?.map((part) => part.text ?? "").join("");
505
+ if (!text) {
506
+ throw new Error("Gemini API returned no text candidate.");
507
+ }
508
+ let result;
509
+ try {
510
+ result = JSON.parse(text);
511
+ } catch {
512
+ throw new Error(`Gemini API returned non-JSON output: ${truncate$1(text)}`);
513
+ }
514
+ return {
515
+ result,
516
+ meta: {
517
+ conversationId: null,
518
+ durationSeconds: (Date.now() - startedAt) / 1e3,
519
+ usage: payload.usageMetadata ?? null
520
+ }
521
+ };
522
+ }
523
+ function truncate$1(text) {
524
+ return text.length > 300 ? `${text.slice(0, 300)}...` : text;
525
+ }
526
+ const geminiApiProvider = {
527
+ name: "gemini-api",
528
+ defaultModel: GEMINI_API_DEFAULT_MODEL,
529
+ execute: executeGeminiApi
530
+ };
531
+ async function executeOpenaiCompat(options) {
532
+ const apiKey = options.settings?.apiKey;
533
+ const baseUrl = options.settings?.baseUrl?.replace(/\/$/, "");
534
+ const model = options.model || options.settings?.model;
535
+ if (!apiKey || !baseUrl || !model) {
536
+ throw new Error(
537
+ "openai provider needs baseUrl, apiKey, and model. Set OPENAI_BASE_URL and OPENAI_API_KEY, or run: modlens config set openai.baseUrl <url> / openai.apiKey <key> / openai.model <name>"
538
+ );
539
+ }
540
+ const imageUrl = options.imageKind === "remote" ? options.imageSource : toDataUrl(readLocalImageBase64(options.imageSource));
541
+ const prompt = `${buildVisionPrompt({
542
+ imageSource: options.imageSource,
543
+ imageKind: "inline",
544
+ extraPrompt: options.extraPrompt
545
+ })}
546
+
547
+ Respond with ONE JSON object only, no markdown fences, no commentary. Fill this exact structure with your findings from the image (do not repeat this template literally, replace every value):
548
+ {"summary":"one paragraph describing the image","ocr":{"full_text":"all visible text","lines":[{"text":"one line","language":"en"}]},"layout":{"regions":[{"type":"title|subtitle|paragraph|list|table|chart|form|code|image|icon|other","reading_order":1,"text":"region text"}]},"semantics":{"scene":"what kind of scene","intent":"what the image is for","entities":[{"name":"entity","type":"kind","evidence":"where seen"}],"relations":[{"subject":"a","predicate":"relates to","object":"b"}]},"visual":{"dominant_colors":["color"],"style":"visual style","notes":["notable visual detail"]},"uncertainty":["anything unreadable or ambiguous"]}`;
549
+ const startedAt = Date.now();
550
+ const response = await fetch(`${baseUrl}/chat/completions`, {
551
+ method: "POST",
552
+ headers: {
553
+ Authorization: `Bearer ${apiKey}`,
554
+ "Content-Type": "application/json"
555
+ },
556
+ body: JSON.stringify({
557
+ model,
558
+ messages: [
559
+ {
560
+ role: "user",
561
+ content: [
562
+ { type: "image_url", image_url: { url: imageUrl } },
563
+ { type: "text", text: prompt }
564
+ ]
565
+ }
566
+ ]
567
+ }),
568
+ signal: AbortSignal.timeout(options.timeoutMs)
569
+ });
570
+ if (!response.ok) {
571
+ const body = await response.text();
572
+ throw new Error(`OpenAI-compatible API error ${response.status}: ${truncate(body)}`);
573
+ }
574
+ const payload = await response.json();
575
+ const text = payload.choices?.[0]?.message?.content;
576
+ if (!text) {
577
+ throw new Error("OpenAI-compatible API returned no message content.");
578
+ }
579
+ const result = extractJson(text);
580
+ if (result === null) {
581
+ throw new Error(`OpenAI-compatible API returned non-JSON output: ${truncate(text)}`);
582
+ }
583
+ const shaped = result;
584
+ if (typeof shaped.summary !== "string" || typeof shaped.ocr !== "object") {
585
+ throw new Error(
586
+ `OpenAI-compatible API returned JSON that does not match the vision schema (missing summary/ocr). Retry, or switch to gemini-api / anthropic for enforced schemas. Got: ${truncate(text)}`
587
+ );
588
+ }
589
+ return {
590
+ result,
591
+ meta: {
592
+ conversationId: null,
593
+ durationSeconds: (Date.now() - startedAt) / 1e3,
594
+ usage: payload.usage ?? null
595
+ }
596
+ };
597
+ }
598
+ function toDataUrl(image) {
599
+ return `data:${image.mimeType};base64,${image.data}`;
600
+ }
601
+ function truncate(text) {
602
+ return text.length > 300 ? `${text.slice(0, 300)}...` : text;
603
+ }
604
+ const openaiCompatProvider = {
605
+ name: "openai",
606
+ defaultModel: "",
607
+ execute: executeOpenaiCompat
608
+ };
207
609
  const PROVIDERS = {
208
610
  "antigravity-cli": antigravityCliProvider,
209
611
  antigravity: antigravityCliProvider,
210
- agy: antigravityCliProvider
612
+ agy: antigravityCliProvider,
613
+ "gemini-api": geminiApiProvider,
614
+ gemini: geminiApiProvider,
615
+ openai: openaiCompatProvider,
616
+ "openai-compat": openaiCompatProvider,
617
+ anthropic: anthropicApiProvider,
618
+ claude: anthropicApiProvider
211
619
  };
212
620
  function resolveProvider(providerName = "antigravity-cli") {
213
621
  const normalized = providerName.trim().toLowerCase();
214
622
  const provider = PROVIDERS[normalized];
215
623
  if (!provider) {
216
- throw new Error(`Unsupported provider: ${providerName}`);
624
+ throw new Error(
625
+ `Unsupported provider: ${providerName}. Available: ${listProviders().join(", ")}`
626
+ );
217
627
  }
218
628
  return provider;
219
629
  }
630
+ function listProviders() {
631
+ return [...new Set(Object.values(PROVIDERS).map((provider) => provider.name))];
632
+ }
220
633
  const DEFAULT_TIMEOUT_MS = 18e4;
221
634
  const KILL_GRACE_MS = 3e4;
222
635
  async function analyzeImage(options) {
@@ -224,20 +637,35 @@ async function analyzeImage(options) {
224
637
  if (resolvedInput.kind === "local") {
225
638
  validateInputFile(resolvedInput.source);
226
639
  }
227
- const provider = resolveProvider(options.provider);
640
+ const config2 = options.config ?? loadConfigFile();
641
+ const provider = resolveProvider(options.provider || defaultProviderName(config2));
642
+ const settings = resolveProviderSettings(provider.name, config2);
228
643
  const timeoutMs = options.timeoutMs ?? DEFAULT_TIMEOUT_MS;
229
- const model = options.model || provider.defaultModel;
230
- const invocation = provider.buildInvocation({
644
+ const model = options.model || settings.model || provider.defaultModel;
645
+ const providerOptions = {
231
646
  imageSource: resolvedInput.source,
232
647
  imageKind: resolvedInput.kind,
233
648
  model,
234
649
  extraPrompt: options.prompt,
235
650
  providerBin: options.providerBin,
236
651
  workdir: options.workdir,
237
- timeoutMs
238
- });
239
- const commandResult = await runCommand(provider.name, invocation, timeoutMs + KILL_GRACE_MS);
240
- const parsed = provider.parseOutput(commandResult.stdout);
652
+ timeoutMs,
653
+ settings
654
+ };
655
+ let parsed;
656
+ if (provider.execute) {
657
+ parsed = await provider.execute(providerOptions);
658
+ } else if (provider.buildInvocation && provider.parseOutput) {
659
+ const invocation = provider.buildInvocation(providerOptions);
660
+ const commandResult = await runCommand(
661
+ provider.name,
662
+ invocation,
663
+ timeoutMs + KILL_GRACE_MS
664
+ );
665
+ parsed = provider.parseOutput(commandResult.stdout);
666
+ } else {
667
+ throw new Error(`Provider ${provider.name} implements neither execute nor buildInvocation.`);
668
+ }
241
669
  return {
242
670
  image: resolvedInput.source,
243
671
  provider: provider.name,
@@ -327,7 +755,8 @@ function runCommand(providerName, invocation, timeoutMs) {
327
755
  });
328
756
  }
329
757
  const program = new Command();
330
- program.name("modlens").description("Plug-in vision for text-only LLMs: image in, structured JSON evidence out").version("2.0.0").requiredOption("-i, --input <path|url>", "Input image path or https URL").option("-o, --output <path>", "Write result JSON to a file").option("-m, --model <name>", "Provider model name (default: gemini-3.6-flash-low)").option("-p, --provider <name>", "Vision provider name", "antigravity-cli").option("--prompt <text>", "Extra focus for this image").option("--timeout <ms>", "Provider timeout in milliseconds", "180000").option("--provider-bin <path>", "Provider binary path (default: agy)").option("--workdir <path>", "Working directory for the provider command").action(async (options) => {
758
+ program.name("modlens").description("Plug-in vision for text-only LLMs: image in, structured JSON evidence out").version("2.1.0");
759
+ program.command("analyze", { isDefault: true }).description("Analyze an image into structured JSON evidence (default command)").requiredOption("-i, --input <path|url>", "Input image path or https URL").option("-o, --output <path>", "Write result JSON to a file").option("-m, --model <name>", "Provider model name").option("-p, --provider <name>", `Vision provider (${listProviders().join(", ")})`).option("--prompt <text>", "Extra focus for this image").option("--timeout <ms>", "Provider timeout in milliseconds", "180000").option("--provider-bin <path>", "Provider binary path (default: agy)").option("--workdir <path>", "Working directory for the provider").action(async (options) => {
331
760
  try {
332
761
  const timeoutMs = Number.parseInt(options.timeout, 10);
333
762
  if (!Number.isFinite(timeoutMs) || timeoutMs <= 0) {
@@ -358,4 +787,46 @@ program.name("modlens").description("Plug-in vision for text-only LLMs: image in
358
787
  process.exit(1);
359
788
  }
360
789
  });
790
+ const config = program.command("config").description(`Manage ${CONFIG_PATH} (providers, keys, models)`);
791
+ config.command("init").description(`Create a starter config at ${CONFIG_PATH}`).option("--force", "Overwrite an existing config file").action((options) => {
792
+ try {
793
+ initConfigFile(CONFIG_PATH, Boolean(options.force));
794
+ process.stdout.write(
795
+ `Created ${CONFIG_PATH}
796
+ Fill in the apiKey fields you need, or use: modlens config set <provider>.apiKey <key>
797
+ `
798
+ );
799
+ } catch (error) {
800
+ process.stderr.write(
801
+ `Error: ${error instanceof Error ? error.message : String(error)}
802
+ `
803
+ );
804
+ process.exit(1);
805
+ }
806
+ });
807
+ config.command("set <key> <value>").description("Set a value, e.g. modlens config set gemini-api.apiKey <key>").action((key, value) => {
808
+ try {
809
+ setConfigValue(key, value);
810
+ process.stdout.write(`Saved ${key} to ${CONFIG_PATH}
811
+ `);
812
+ } catch (error) {
813
+ process.stderr.write(
814
+ `Error: ${error instanceof Error ? error.message : String(error)}
815
+ `
816
+ );
817
+ process.exit(1);
818
+ }
819
+ });
820
+ config.command("show").description("Print the effective config with API keys masked").action(() => {
821
+ try {
822
+ process.stdout.write(`${renderConfig(loadConfigFile())}
823
+ `);
824
+ } catch (error) {
825
+ process.stderr.write(
826
+ `Error: ${error instanceof Error ? error.message : String(error)}
827
+ `
828
+ );
829
+ process.exit(1);
830
+ }
831
+ });
361
832
  program.parse();
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@liustack/modlens",
3
- "version": "2.0.0",
3
+ "version": "2.1.0",
4
4
  "description": "Plug-in vision for text-only LLMs, powered by the free Antigravity CLI",
5
5
  "type": "module",
6
6
  "bin": {
@@ -22,23 +22,29 @@ Do not use this skill for:
22
22
 
23
23
  ```bash
24
24
  modlens --version
25
- agy --version
26
25
  ```
27
26
 
28
27
  If `modlens` is missing, run it via `npx @liustack/modlens` instead.
29
28
 
30
- If `agy` (Antigravity CLI) is missing:
29
+ ModLens supports four vision providers. Check what is configured:
31
30
 
32
31
  ```bash
33
- curl -fsSL https://antigravity.google/cli/install.sh | bash
32
+ modlens config show
34
33
  ```
35
34
 
36
- If `agy` is installed but not signed in, ask the user to run `agy` once in a terminal and complete the Google sign-in. This cannot be done non-interactively.
35
+ - **antigravity-cli** (default, no key needed): needs `agy` installed and signed in. If `agy --version` fails: `curl -fsSL https://antigravity.google/cli/install.sh | bash`, then ask the user to run `agy` once and complete the Google sign-in (cannot be done non-interactively).
36
+ - **gemini-api**: needs `GEMINI_API_KEY` env or `modlens config set gemini-api.apiKey <key>` (free key from https://aistudio.google.com).
37
+ - **openai**: any OpenAI-compatible multimodal endpoint; needs baseUrl + apiKey + model via env (`OPENAI_BASE_URL`, `OPENAI_API_KEY`) or `modlens config set openai.<field> <value>`.
38
+ - **anthropic**: needs `ANTHROPIC_API_KEY` env or config; defaults to Claude Haiku.
39
+
40
+ `modlens config init` writes a starter config to `~/.modlens/config.json` when none exists.
37
41
 
38
42
  ## Command
39
43
 
40
44
  ```bash
41
45
  modlens -i <image-path-or-url>
46
+ # pick a provider explicitly
47
+ modlens -i <image> -p gemini-api
42
48
  # or without a global install
43
49
  npx @liustack/modlens -i <image-path-or-url>
44
50
  ```
@@ -49,8 +55,7 @@ Optional flags:
49
55
  modlens -i <image> -o <output.json> -m <model> --prompt "<extra focus>" --timeout <ms>
50
56
  ```
51
57
 
52
- - Default model is `gemini-3.6-flash-low` (fastest, cheapest on quota). Use `-m gemini-3.1-pro-high` for dense or hard images.
53
- - A run typically takes 15-40 seconds. Do not treat silence as a hang before the timeout.
58
+ Speed expectations: `gemini-api` typically 5-10 seconds, `antigravity-cli` 15-40 seconds (full agent loop), `openai`/`anthropic` depend on the endpoint. For dense or hard images on antigravity-cli, try `-m gemini-3.1-pro-high`.
54
59
 
55
60
  ## Finding the image path in the chat
56
61
 
@@ -80,10 +85,11 @@ Top level: `{ image, provider, result, meta }`. Inside `result`:
80
85
  - `visual`: colors and style clues
81
86
  - `uncertainty[]`: what the vision engine was unsure about
82
87
 
83
- Structure is enforced by a JSON schema at the provider level. Full schema: `references/output-schema.md`.
88
+ Structure is enforced by schema on antigravity-cli (`--json-schema`), gemini-api (`responseJsonSchema`), and anthropic (forced tool call). The openai route uses a template prompt plus shape validation and fails loudly on mismatch.
84
89
 
85
90
  ## Failure Handling
86
91
 
87
- - Exit code 1 with `Provider CLI not found`: Antigravity CLI is not installed. Install it, then retry.
88
- - `no structured result` or auth-flavored errors: ask the user to run `agy` and sign in, or check quota.
92
+ - `Provider CLI not found`: Antigravity CLI is not installed. Install it, or switch provider: `-p gemini-api`.
93
+ - Missing key errors name the exact env var and `config set` command to run. Relay that to the user.
94
+ - `does not match the vision schema` on the openai route: retry once, then switch to `-p gemini-api` or `-p anthropic` for enforced schemas.
89
95
  - Timeouts: retry once with `--timeout 300000`. If it still fails, report the exact error instead of fabricating image content.