npm - @liustack/modlens - Versions diffs - 2.7.9 → 2.7.11 - Mend

@liustack/modlens 2.7.9 → 2.7.11

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (5) hide show

package/README.md CHANGED Viewed

@@ -9,75 +9,55 @@
   <p><a href="./README.zh-CN.md">简体中文</a></p>
 </div>
-DeepSeek-V4-Flash gives you a lot of model for very little money: fast, strong, and its one real flaw is no multimodal. And it's not just DeepSeek. Every text-only model running inside Codex, Claude Code, Pi Agent, or OpenClaw hits the same wall.
+DeepSeek-V4-Flash gives you a lot of model for very little money: fast, cheap, capable, and blind. It is not just DeepSeek either. Any text-only model running inside Codex, Claude Code, Pi, or OpenCode hits the same wall: you throw it a screenshot of an error and it sees nothing.
-ModLens fixes this the lightest way possible: it never touches your config, never adds a local proxy, and is just a vision plug-in you can run as a CLI or install as an Agent Skill. What it hands back is not a one-line caption but structured visual evidence: text, layout, regions, entities, relations, visual clues. Five vision engines to pick from. The default one needs no key at all, and the fastest one runs on a free Gemini key whose image understanding embarrasses most flagships, Fable 5 included. How it works:
+ModLens gives it sight, and **you just paste**. Every other bridge makes you save the image to a file and then mention the path in the chat. ModLens pulls the pasted image back out of session storage instead. What comes back is not "this is a screenshot of an error", it is evidence you can quote: every word in the image transcribed, the layout cut into regions in reading order, and the entities and relations on screen listed separately.
-![A text-only model hands an image to the vision engine through the modlens skill and gets structured JSON evidence back](https://raw.githubusercontent.com/liustack/modlens/main/assets/flow.en.png)
-- **You just paste.** Every other bridge makes you save a file and report its path. ModLens pulls the pasted image back out of session storage.
-- **Evidence, not an impression.** Full OCR text, layout regions in reading order, semantic entities, visual clues. Your model can quote specifics.
-- **It says when it cannot read something.** Anything uncertain lands in `uncertainty` rather than being invented. Pixel coordinates and confidence scores, the two things models fabricate most, were dropped in v2.
-- **No model swap, no config changes, no local proxy.** You picked your model for price and reasoning, not eyesight. That choice stays.
-- **Starts with no key, gets faster with a free one.** agy needs no key at all, and a free AI Studio Gemini key takes three minutes and reads an image in 5-10 seconds.
-- **Install once, works in four harnesses.** Verified in Claude Code, Codex, Pi, and OpenCode.
-**Requirements**: Node 18+ (22.5+ for OpenCode paste recovery), macOS or Linux. Hit a problem? [Troubleshooting](docs/troubleshooting.md) lists every error this CLI prints, with causes and fixes.
-## You can just paste the image
-Every other bridge makes you save the image to a file first, then mention the path in the chat. ModLens lets you paste it.
-That is not laziness on their part. Pasting is handled end to end by the client: the moment an image lands in the chat box it is encoded and sent, and a vision MCP server never gets a chance to step in, which is why their docs can only tell you to save the file and report the path. ModLens takes the other route. Before those bytes are ever sent, the harness has already written them to local session storage, so the skill goes there, pulls them back out into a real file, and feeds that to the vision engine. You do nothing, and the model answers with the full image instead of asking you for a path.
-Verified on real machines across four harnesses: Claude Code pinpoints the exact session from its injected session id, Pi stores sessions the same way, OpenCode swaps in SQLite, and Codex's pasted images already carry a temp path so the path-tag route handles them. Before touching anything, `recover-paste` works out which harness it is running inside, by walking the process ancestry and checking environment fingerprints, and reads only that harness's storage, so another tool's stale sessions can't impersonate it.
-Side by side:
-| | Swap in a multimodal model | Vision MCP servers | ModLens |
-| :-- | :-- | :-- | :-- |
-| Your chosen model | has to change | stays | stays |
-| An image pasted into the chat | visible if the model supports it | not reachable, their docs say save a file and report the path | handled directly |
-| What you get back | the model's own reading | usually a description | OCR text, layout regions, semantics, visual clues |
-| Where it cannot read | may invent | may invent | says so in `uncertainty` |
-| Cost | multimodal model pricing | usually per API call | agy's free quota, or a free Gemini key |
-| Setup | change config, change model | install a server, edit config | one CLI or one skill |
+No model swap, no prompt surgery, no local proxy. It starts with no key at all.
-The honest weaknesses: agy's free tier is a weekly quota and heavy use hits the wall (a free Gemini key sidesteps that). Session storage layouts are each harness's internals with no compatibility promise, so if recovery ever breaks, dragging the file in still works everywhere.
+![A text-only model hands an image to the vision engine through the modlens skill and gets structured JSON evidence back](https://raw.githubusercontent.com/liustack/modlens/main/assets/flow.en.png)
-## Quick start
+## Three steps
-**1. Install the skill.** Just tell your agent (Claude Code, Codex, OpenClaw, Cursor, ...):
+**1. Install the skill.** Tell your agent (Claude Code, Codex, OpenClaw, and Cursor all take this):
 ```text
 Install the skill from https://github.com/liustack/modlens
 ```
-or do it yourself:
+Or do it yourself: `npx -y skills add liustack/modlens`
-```bash
-npx -y skills add liustack/modlens
-```
-Harnesses look for skills in different places: Claude Code reads `~/.claude/skills/`, Codex reads `~/.codex/skills/`, Pi and OpenCode read `~/.agents/skills/`. Symlinks work in all of them, so linking the skill folder once keeps every agent on the latest version.
-**2. Wire up a vision engine.** Recommended: a free Gemini key from [aistudio.google.com](https://aistudio.google.com). Three minutes, no credit card, 5-10 seconds per image:
+**2. Give it a vision engine.** A free Gemini key from [aistudio.google.com](https://aistudio.google.com) is the fast answer: three minutes, no credit card, 5 to 10 seconds per image.
 ```bash
 modlens config set gemini-api.apiKey <key>
 modlens config set provider gemini-api
 ```
-Don't feel like typing those two lines? Tell your agent "set my Gemini key in modlens" and it runs them for you.
+Don't feel like typing those? Tell your agent "set my Gemini key in modlens" and it runs them. Skipping the sign-up is fine too: Antigravity CLI works with no key, it is just slower (15 to 40 seconds) and its free quota is tight.
-Skipping the sign-up is fine too: Antigravity CLI works with no key at all, it is just slower (15-40s) and its free quota is tight.
+**3. Use it.** Paste an image, or throw a path at it, and ask anything. The skill fires on its own.
-```bash
-curl -fsSL https://antigravity.google/cli/install.sh | bash
-agy    # opens browser sign-in, then exit
-```
+Requirements, in one line: Node 18+ (22.5+ for OpenCode paste recovery), macOS or Linux.
-**3. Use it.** Paste an image, or throw a path at it, and ask anything. The skill fires on its own.
+## Why pasting works here and nowhere else
+Pasting is handled end to end by the client. The moment an image lands in the chat box it is encoded and sent, and a vision MCP server never gets a chance to step in, which is why their docs can only tell you to save the file and report the path.
+ModLens takes the other route. Before those bytes are sent anywhere, the harness has already written them to local session storage. The skill goes there, pulls them back into a real file, and hands that to the vision engine. You do nothing, and the model gets the whole image instead of asking you for a path.
+All four harnesses are verified on real machines: Claude Code pinpoints the exact session from its injected session id, Pi stores sessions the same way, OpenCode swaps in SQLite, and Codex's pasted images already carry a temp path, so the path-tag route handles them. Before touching anything, `recover-paste` works out which harness it is running inside (process ancestry, then environment fingerprints) and reads only that harness's storage, so a neighbouring project's old sessions cannot impersonate it. Recovered files are written 0600.
+| | Swap in a multimodal model | Vision MCP servers | ModLens |
+| :-- | :-- | :-- | :-- |
+| Your chosen model | has to change | stays | stays |
+| An image pasted into the chat | visible if the model supports it | out of reach, save a file and report the path | handled directly |
+| What you get back | the model's own reading | usually a description | full transcription, layout regions, entities and relations, visual clues |
+| Where it cannot read | may invent | may invent | says so in `uncertainty` |
+| Cost | multimodal model pricing | usually per API call | agy's free quota, or a free Gemini key |
+| Setup | change config, change model | install a server, edit config | one CLI or one skill |
+The weaknesses sit here too: agy's free tier is a weekly quota and heavy use hits the wall (a free Gemini key sidesteps it). Session storage layouts are each harness's internals with no compatibility promise, so if recovery ever breaks, dragging the file in still works everywhere.
 ## See it work
@@ -213,7 +193,7 @@ Pointing OpenCode at DeepSeek takes two lines of setup: `opencode auth login`, p
 ## Why a bridge instead of a multimodal model?
 - **Keep your model.** You picked DeepSeek-V4-Flash (or gpt-oss, or whatever else) for its price and its reasoning, not its eyesight. ModLens adds sight without touching that choice.
-- **Evidence beats pixels.** Text models reason best over structured text, not raw pixels. ModLens hands them OCR plus layout plus semantics, already decoded, not a base64 blob.
+- **Evidence beats pixels.** Text models reason best over structured text, not raw pixels. ModLens hands them the transcribed words, the segmented layout, and the extracted meaning, not a base64 blob.
 - **Engines die, the bridge survives.** v1 ran on Gemini CLI's free tier until Google shut it down in June 2026. v2 moved to its successor, Antigravity CLI, behind the same provider interface, so the next engine swap costs one file, not a rewrite.
 ModSearch, ModLens's sibling project, plays the same trick for web search and page fetching: [liustack/modsearch](https://github.com/liustack/modsearch).

package/README.zh-CN.md CHANGED Viewed

@@ -9,75 +9,55 @@
   <p><a href="./README.md">English</a></p>
 </div>
-DeepSeek-V4-Flash 碗大又好吃，速度快，性能强，要说唯一的缺点就是没有多模态。不仅 DeepSeek-V4-Flash，只要是纯文本语言模型，跑在 Codex、Claude Code、Pi Agent、OpenClaw 中，都有这个问题。
+DeepSeek-V4-Flash 碗大又好吃，速度快、便宜、能打，唯独没有多模态。不止它，任何纯文本模型跑在 Codex、Claude Code、Pi、OpenCode 里，都撞同一堵墙：你甩过去一张报错截图，它看不见。
-ModLens 用最轻的方式解决它：不动你的配置，不装本地代理，就是一个视觉外挂，CLI 和 skill 两种用法。它产出的不是一句话描述，是结构化的视觉证据：文字、版面、区块、实体、关系、视觉线索。视觉引擎有五个可选，默认那个零 key 就能跑，最快的那个用免费 Gemini key，识图能力连 Fable 5 都吊打。原理如下：
+ModLens 给它装上视力，而且**你直接粘贴就行**。别的方案都要你先把图存成文件、再在对话里报一句路径，ModLens 直接从会话存储里把粘贴的图捞回来。回来的不是一句「这是一张报错截图」，是能引用的证据：图里的字一句不落地抄下来，版面按阅读顺序切成区块，画面里的实体和关系单列一份。
-![纯文本模型经 modlens skill 把图片交给视觉引擎，回来的是结构化 JSON 证据](https://raw.githubusercontent.com/liustack/modlens/main/assets/flow.zh.png)
-- **你直接粘贴就行。** 别的方案让你先存成文件再报路径，ModLens 从会话存储里把粘贴的图捞回来。
-- **给的是证据，不是印象。** OCR 全文、按阅读顺序排好的版面区块、语义实体、视觉线索，模型能引用具体内容。
-- **读不准就说读不准。** 拿不准的地方进 `uncertainty`，不编。像素坐标和置信度分数这两样模型最爱编的，v2 直接删了。
-- **不换模型，不改配置，不装代理。** 你选 DeepSeek 图的是价格和推理，不是视力，这个选择不用动。
-- **零 key 起步，想快就领个免费 key。** agy 不要 key，AI Studio 的免费 Gemini key 三分钟到手，识图 5-10 秒。
-- **装一次，四家 harness 通用。** Claude Code、Codex、Pi、OpenCode 都验证过。
-**环境要求**：Node 18+（OpenCode 的粘贴恢复需要 22.5+），macOS 或 Linux。 出问题看[故障排查](docs/troubleshooting.md)，里面按报错原文列了每一条的成因和解法。
-## 你可以直接粘贴图片
-别的方案让你先把图存成文件，再在对话里报一句路径。ModLens 让你直接粘贴。
+模型不用换，提示词不用改，本地不装代理。零 key 就能开跑。
-这不怪它们偷懒。粘贴这个动作从头到尾是客户端办的，图一进对话框就被转码发走，识图 MCP server 连插手的机会都没有，所以它们的文档只能教你存文件、报路径。ModLens 走的是另一条路：图片字节在发走之前，早被 harness 原样写进了本地会话存储，skill 直接去那里把它捞回来落成文件，再喂给视觉引擎。你什么都不用做，模型拿到的是完整图片，不是一句「麻烦告诉我路径」。
-四家 harness 真机验证过：Claude Code 按注入的会话 ID 精确定位，Pi 的存储路数和它同构，OpenCode 换成了 SQLite，Codex 的粘贴图本来就带临时路径，走路径标签就行。动手之前 `recover-paste` 会先认清自己跑在哪一家（查进程祖先链，核对环境变量指纹），只读那一家的存储，别家的旧会话冒充不了。
-放在一起看更清楚：
-| | 换个多模态模型 | 识图类 MCP server | ModLens |
-| :-- | :-- | :-- | :-- |
-| 你选的模型 | 得换掉 | 不用换 | 不用换 |
-| 粘贴进对话的图 | 模型支持就能看 | 接不住，文档让你先存文件报路径 | 直接接住 |
-| 拿到手的是什么 | 模型自己的理解 | 通常是一段描述 | OCR 全文、版面区块、语义、视觉线索 |
-| 读不准的地方 | 可能编 | 可能编 | 进 `uncertainty`，明说读不准 |
-| 花费 | 多模态模型的价格 | 多数按 API 计费 | agy 免费额度，或免费 Gemini key |
-| 上手 | 改配置换模型 | 装 server、改配置 | 一个 CLI 或一个 skill |
-诚实说短板：agy 的免费额度是周配额，重度用会撞墙（换成免费 Gemini key 就绕开了）。会话存储格式是各家 harness 的内部实现，没有兼容承诺，哪天捞不动了，拖文件永远是保底。
+![纯文本模型经 modlens skill 把图片交给视觉引擎，回来的是结构化 JSON 证据](https://raw.githubusercontent.com/liustack/modlens/main/assets/flow.zh.png)
-## 快速开始
+## 三步用起来
-**1. 装 skill。** 直接告诉你的 agent（Claude Code、Codex、OpenClaw、Cursor 等）：
+**一、装 skill。** 跟你的 agent 说一句就行（Claude Code、Codex、OpenClaw、Cursor 都吃这套）：
 ```text
 安装这个 skill https://github.com/liustack/modlens
 ```
-或者自己动手：
+自己动手也行：`npx -y skills add liustack/modlens`
-```bash
-npx -y skills add liustack/modlens
-```
-各家 harness 找 skill 的位置不一样：Claude Code 读 `~/.claude/skills/`，Codex 读 `~/.codex/skills/`，Pi 和 OpenCode 读 `~/.agents/skills/`。软链接在哪家都好使，把 skill 目录链一次，各家永远用最新版。
-**2. 接一个视觉引擎。** 推荐去 [aistudio.google.com](https://aistudio.google.com) 领个免费 Gemini key，三分钟，不要信用卡，出图 5-10 秒：
+**二、接一个视觉引擎。** 推荐去 [aistudio.google.com](https://aistudio.google.com) 领个免费 Gemini key，三分钟，不要信用卡，识图 5 到 10 秒：
 ```bash
 modlens config set gemini-api.apiKey <key>
 modlens config set provider gemini-api
 ```
-懒得敲这两行？跟 agent 说一句「帮我把 Gemini key 配进 modlens」，它自己会跑。
+懒得敲这两行？跟 agent 说「帮我把 Gemini key 配进 modlens」，它自己会跑。不想注册也行，装上 Antigravity CLI 零 key 开跑，代价是慢（15 到 40 秒），免费额度也紧。
-不想注册也行，装上 Antigravity CLI 就能零 key 开跑，代价是慢（15-40 秒），免费额度也紧：
+**三、直接用。** 粘一张图，或者甩个图片路径，随便问。skill 自己会触发。
-```bash
-curl -fsSL https://antigravity.google/cli/install.sh | bash
-agy    # 浏览器完成登录后退出
-```
+环境要求就一行：Node 18+（OpenCode 的粘贴恢复需要 22.5+），macOS 或 Linux。
+## 粘贴这件事，为什么只有它接得住
+粘贴从头到尾是客户端办的事。图一进对话框就被转码发走，识图类 MCP server 连插手的机会都没有，所以它们的文档只能教你：先存文件，再报路径。
+ModLens 走的是另一条路。图片字节在发给模型之前，早被 harness 原样写进了本地会话存储。skill 直接去那儿把它捞回来落成文件，喂给视觉引擎。你什么都不用做，模型拿到的是完整图片，而不是一句「麻烦告诉我路径」。
+四家 harness 都在真机上验证过：Claude Code 按注入的会话 ID 精确定位，Pi 的存储路数和它同构，OpenCode 换成了 SQLite，Codex 的粘贴图本来就带临时路径、走路径标签即可。动手之前 `recover-paste` 会先认清自己跑在哪一家（查进程祖先链，核对环境变量指纹），只读那一家的存储，隔壁项目的旧会话冒充不了，捞出来的文件权限是 0600。
+| | 换个多模态模型 | 识图类 MCP server | ModLens |
+| :-- | :-- | :-- | :-- |
+| 你选的模型 | 得换掉 | 不用换 | 不用换 |
+| 粘贴进对话的图 | 模型支持就能看 | 接不住，得先存文件报路径 | 直接接住 |
+| 拿到手的是什么 | 模型自己的理解 | 通常一段描述 | 全文转录、版面区块、实体关系、视觉线索 |
+| 读不准的地方 | 可能编 | 可能编 | 进 `uncertainty`，明说读不准 |
+| 花费 | 多模态模型的价格 | 多数按 API 计费 | agy 免费额度，或免费 Gemini key |
+| 上手 | 改配置换模型 | 装 server、改配置 | 一个 CLI 或一个 skill |
-**3. 用起来。** 粘贴一张图，或者甩个图片路径，随便问。skill 自己会触发。
+短板也摆这儿：agy 的免费额度是周配额，重度用会撞墙（换成免费 Gemini key 就绕开了）。会话存储格式是各家 harness 的内部实现，没有兼容承诺，哪天捞不动了，拖文件永远是保底。
 ## 看看效果
@@ -213,7 +193,7 @@ OpenCode 接 DeepSeek 只要两步：`opencode auth login` 选 DeepSeek 贴上 k
 ## 为什么外挂，而不是换多模态模型？
 - **模型不用换。** 你选 DeepSeek-V4-Flash（或 gpt-oss，或别的什么）图的是价格和推理能力，不是视力。ModLens 只加视力，不碰这个选择。
-- **证据强过像素。** 文本模型最会在结构化文本上推理，不是盯着原始像素。ModLens 递过去的是 OCR 加版面加语义，都是解好码的证据，不是一坨 base64。
+- **证据强过像素。** 文本模型最擅长在结构化文本上推理，不是盯着原始像素。ModLens 递过去的是转录好的文字、切好的版面、拆好的语义，不是一坨 base64。
 - **引擎会死，桥不会死。** v1 跑在 Gemini CLI 免费档上，2026 年 6 月被 Google 一刀切停掉。v2 换到继任者 Antigravity CLI，还是同一个 provider 接口，下次再换引擎，改一个文件就行，不用重写。
 姊妹项目 ModSearch 用同一招补上联网搜索和网页抓取：[liustack/modsearch](https://github.com/liustack/modsearch)。

package/dist/main.js CHANGED Viewed

@@ -1478,7 +1478,7 @@ function recoverPastedImages(options = {}) {
   return result;
 }
 const program = new Command();
-program.name("modlens").description("Plug-in vision for text-only LLMs: image in, structured JSON evidence out").version("2.7.9");
+program.name("modlens").description("Plug-in vision for text-only LLMs: image in, structured JSON evidence out").version("2.7.11");
 program.command("analyze", { isDefault: true }).description("Analyze an image into structured JSON evidence (default command)").requiredOption("-i, --input <path|url>", "Input image path or https URL").option("-o, --output <path>", "Write result JSON to a file").option("-m, --model <name>", "Provider model name").option("-p, --provider <name>", `Vision provider (${listProviders().join(", ")})`).option("--prompt <text>", "Extra focus for this image").option("--timeout <ms>", "Provider timeout in milliseconds", "180000").option("--provider-bin <path>", "Provider binary path (default: agy)").option("--workdir <path>", "Working directory for the provider").action(async (options) => {
   try {
     const timeoutMs = Number.parseInt(options.timeout, 10);

package/package.json CHANGED Viewed

@@ -1,6 +1,6 @@
 {
   "name": "@liustack/modlens",
-  "version": "2.7.9",
+  "version": "2.7.11",
   "description": "Plug-in vision for text-only LLMs, powered by the free Antigravity CLI",
   "type": "module",
   "bin": {

package/skills/modlens/SKILL.md CHANGED Viewed

@@ -1,6 +1,6 @@
 ---
 name: modlens
-description: "Plug-in vision for text-only models. Use whenever the user shares an image (local path, screenshot, photo, chart, document scan, or image URL) and the active model cannot see images or has no vision tool. Runs the modlens CLI to convert the image into structured JSON evidence: OCR text, layout, semantics, visual clues. Also use when the user asks how to install, configure, or switch modlens providers (Gemini API key, OpenAI-compatible endpoints, Claude API or Claude Code CLI)."
+description: "Plug-in vision for text-only models. Use whenever the user shares an image (local path, screenshot, photo, chart, document scan, or image URL) and the active model cannot see images or has no vision tool. Runs the modlens CLI to convert the image into structured JSON evidence: every word transcribed, layout regions, semantics, visual clues. Also use when the user asks how to install, configure, or switch modlens providers (Gemini API key, OpenAI-compatible endpoints, Claude API or Claude Code CLI)."
 allowed-tools:
   - Bash
 ---
@@ -11,7 +11,7 @@ Use this skill when:
 - The user provides an image path or image URL and asks anything about it
 - The active model has no native vision (text-only model in a coding agent)
-- You need OCR text, layout, or chart/document structure as evidence before reasoning
+- You need the text inside an image, its layout, or a chart's structure as evidence before reasoning
 - The user asks how to configure modlens, get an API key for it, or switch its provider: follow `references/configure.md` and run the commands for them
 Do not use this skill for:
@@ -91,7 +91,7 @@ Harnesses rarely hand you a clean path. First identify which harness you are in,
 Top level: `{ image, provider, result, meta }`. Inside `result`:
 - `summary`: one-paragraph description of the image
-- `ocr.full_text` + `ocr.lines[]`: transcribed text evidence
+- `ocr.full_text` + `ocr.lines[]`: every word in the image, transcribed (the field keeps the familiar `ocr` name, though a vision model does the reading, not an OCR engine)
 - `layout.regions[]`: typed blocks (`title`, `paragraph`, `table`, `chart`, `code`, ...) in reading order
 - `semantics`: scene, intent, entities, relations
 - `visual`: colors and style clues