@liustack/modlens 2.1.0 → 2.2.0

package/README.md

@@ -111,7 +111,7 @@ Reach for `-m gemini-3.1-pro-high` on dense screenshots or tricky documents. Out
 
 ## Providers and config
 
-ModLens ships four vision providers. `antigravity-cli` stays the default: zero keys, pure free quota.
+ModLens ships five vision providers. `antigravity-cli` stays the default: zero keys, pure free quota.
 
 | Provider | Needs | Typical speed | Notes |
 | :-- | :-- | :-- | :-- |
@@ -119,6 +119,7 @@ ModLens ships four vision providers. `antigravity-cli` stays the default: zero k
 | `gemini-api` | free AI Studio key | 5-10s | fastest free route, schema enforced server-side |
 | `openai` | baseUrl + apiKey + model | endpoint-dependent | any OpenAI-compatible multimodal endpoint (qwen-vl, GLM, ...) |
 | `anthropic` | `ANTHROPIC_API_KEY` | a few seconds | Claude Haiku by default, schema via forced tool call |
+| `claude-cli` | Claude Code signed in | 20-45s | no key, rides your Claude subscription, Read-only permissions |
 
 Config lives in `~/.modlens/config.json`. Environment variables override the file (`GEMINI_API_KEY`, `OPENAI_API_KEY`, `OPENAI_BASE_URL`, `ANTHROPIC_API_KEY`), and CLI flags override everything.
 

package/README.zh-CN.md

@@ -111,7 +111,7 @@ modlens -i <图片路径或 URL> [选项]
 
 ## Provider 与配置
 
-ModLens 内置四个视觉 provider，默认还是 `antigravity-cli`：零 key，纯免费额度。
+ModLens 内置五个视觉 provider，默认还是 `antigravity-cli`：零 key，纯免费额度。
 
 | Provider | 需要什么 | 速度 | 说明 |
 | :-- | :-- | :-- | :-- |
@@ -119,6 +119,7 @@ ModLens 内置四个视觉 provider，默认还是 `antigravity-cli`：零 key
 | `gemini-api` | 免费 AI Studio key | 5-10 秒 | 最快的免费路线，服务端强制 schema |
 | `openai` | baseUrl + apiKey + model | 看端点 | 任何 OpenAI 兼容的多模态端点（qwen-vl、GLM 等） |
 | `anthropic` | `ANTHROPIC_API_KEY` | 几秒 | 默认 Claude Haiku，强制工具调用保 schema |
+| `claude-cli` | Claude Code 已登录 | 20-45 秒 | 零 key，吃你的 Claude 订阅额度，只放行 Read 工具 |
 
 配置放在 `~/.modlens/config.json`，环境变量能盖过它（`GEMINI_API_KEY`、`OPENAI_API_KEY`、`OPENAI_BASE_URL`、`ANTHROPIC_API_KEY`），CLI 参数最大。
 

package/dist/main.js

@@ -79,7 +79,8 @@ const CONFIG_TEMPLATE = {
     "antigravity-cli": { model: "gemini-3.6-flash-low" },
     "gemini-api": { apiKey: "", model: "gemini-3.6-flash" },
     openai: { baseUrl: "", apiKey: "", model: "" },
-    anthropic: { apiKey: "", model: "claude-haiku-4-5-20251001" }
+    anthropic: { apiKey: "", model: "claude-haiku-4-5-20251001" },
+    "claude-cli": { model: "haiku" }
   }
 };
 function initConfigFile(configPath = CONFIG_PATH, force = false) {
@@ -269,11 +270,11 @@ function buildAntigravityInvocation(options) {
   };
 }
 function parseAntigravityOutput(stdout) {
-  const envelope = parseEnvelope(stdout);
+  const envelope = parseEnvelope$1(stdout);
   if (envelope.status && envelope.status !== "SUCCESS") {
     throw new Error(`Antigravity CLI reported status ${envelope.status}.`);
   }
-  const result = envelope.structured_output ?? (typeof envelope.response === "string" ? tryParseJson(envelope.response) : null);
+  const result = envelope.structured_output ?? (typeof envelope.response === "string" ? tryParseJson$1(envelope.response) : null);
   if (result === null || result === void 0) {
     throw new Error(
       "Antigravity CLI output contains no structured result. Check that the model finished the task (auth, quota, timeout)."
@@ -288,14 +289,14 @@ function parseAntigravityOutput(stdout) {
     }
   };
 }
-function parseEnvelope(stdout) {
+function parseEnvelope$1(stdout) {
   const trimmed = stdout.trim();
-  let parsed = tryParseJson(trimmed);
+  let parsed = tryParseJson$1(trimmed);
   if (parsed === null) {
     const firstBrace = trimmed.indexOf("{");
     const lastBrace = trimmed.lastIndexOf("}");
     if (firstBrace >= 0 && lastBrace > firstBrace) {
-      parsed = tryParseJson(trimmed.slice(firstBrace, lastBrace + 1));
+      parsed = tryParseJson$1(trimmed.slice(firstBrace, lastBrace + 1));
     }
   }
   if (!parsed || typeof parsed !== "object") {
@@ -303,7 +304,7 @@ function parseEnvelope(stdout) {
   }
   return parsed;
 }
-function tryParseJson(text) {
+function tryParseJson$1(text) {
   try {
     return JSON.parse(text);
   } catch {
@@ -432,7 +433,7 @@ Report your findings by calling the ${TOOL_NAME} tool.`;
   });
   if (!response.ok) {
     const body = await response.text();
-    throw new Error(`Anthropic API error ${response.status}: ${truncate$2(body)}`);
+    throw new Error(`Anthropic API error ${response.status}: ${truncate$3(body)}`);
   }
   const payload = await response.json();
   const toolUse = payload.content?.find((block) => block.type === "tool_use");
@@ -448,7 +449,7 @@ Report your findings by calling the ${TOOL_NAME} tool.`;
     }
   };
 }
-function truncate$2(text) {
+function truncate$3(text) {
   return text.length > 300 ? `${text.slice(0, 300)}...` : text;
 }
 const anthropicApiProvider = {
@@ -456,6 +457,92 @@ const anthropicApiProvider = {
   defaultModel: ANTHROPIC_DEFAULT_MODEL,
   execute: executeAnthropicApi
 };
+const CLAUDE_CLI_DEFAULT_MODEL = "haiku";
+function buildClaudeCliInvocation(options) {
+  if (options.imageKind === "remote") {
+    throw new Error(
+      "claude-cli provider reads local files only. Download the image first, or use -p gemini-api for remote URLs."
+    );
+  }
+  const prompt = buildVisionPrompt({
+    imageSource: options.imageSource,
+    imageKind: "local",
+    extraPrompt: options.extraPrompt
+  });
+  const args = [
+    "-p",
+    prompt,
+    "--output-format",
+    "json",
+    "--json-schema",
+    visionResultSchemaJson(),
+    "--allowedTools",
+    "Read",
+    "--model",
+    options.model || options.settings?.model || CLAUDE_CLI_DEFAULT_MODEL
+  ];
+  return {
+    command: options.providerBin || "claude",
+    args,
+    cwd: path.resolve(options.workdir || path.dirname(options.imageSource))
+  };
+}
+function parseClaudeCliOutput(stdout) {
+  const envelope = parseEnvelope(stdout);
+  if (envelope.is_error || envelope.subtype && envelope.subtype !== "success") {
+    throw new Error(
+      `Claude CLI reported ${envelope.subtype ?? "an error"}: ${truncate$2(envelope.result ?? "")}`
+    );
+  }
+  if (typeof envelope.result !== "string" || !envelope.result.trim()) {
+    throw new Error("Claude CLI output contains no result. Check login state (run: claude).");
+  }
+  let result;
+  try {
+    result = JSON.parse(envelope.result);
+  } catch {
+    throw new Error(`Claude CLI returned non-JSON result: ${truncate$2(envelope.result)}`);
+  }
+  return {
+    result,
+    meta: {
+      conversationId: envelope.session_id ?? null,
+      durationSeconds: typeof envelope.duration_ms === "number" ? envelope.duration_ms / 1e3 : null,
+      usage: envelope.usage ?? null
+    }
+  };
+}
+function parseEnvelope(stdout) {
+  const trimmed = stdout.trim();
+  let parsed = tryParseJson(trimmed);
+  if (parsed === null) {
+    const firstBrace = trimmed.indexOf("{");
+    const lastBrace = trimmed.lastIndexOf("}");
+    if (firstBrace >= 0 && lastBrace > firstBrace) {
+      parsed = tryParseJson(trimmed.slice(firstBrace, lastBrace + 1));
+    }
+  }
+  if (!parsed || typeof parsed !== "object") {
+    throw new Error("Failed to parse Claude CLI JSON output.");
+  }
+  return parsed;
+}
+function tryParseJson(text) {
+  try {
+    return JSON.parse(text);
+  } catch {
+    return null;
+  }
+}
+function truncate$2(text) {
+  return text.length > 300 ? `${text.slice(0, 300)}...` : text;
+}
+const claudeCliProvider = {
+  name: "claude-cli",
+  defaultModel: CLAUDE_CLI_DEFAULT_MODEL,
+  buildInvocation: buildClaudeCliInvocation,
+  parseOutput: parseClaudeCliOutput
+};
 const GEMINI_API_DEFAULT_MODEL = "gemini-3.6-flash";
 const DEFAULT_BASE_URL = "https://generativelanguage.googleapis.com";
 async function executeGeminiApi(options) {
@@ -615,7 +702,9 @@ const PROVIDERS = {
   openai: openaiCompatProvider,
   "openai-compat": openaiCompatProvider,
   anthropic: anthropicApiProvider,
-  claude: anthropicApiProvider
+  claude: anthropicApiProvider,
+  "claude-cli": claudeCliProvider,
+  "claude-code": claudeCliProvider
 };
 function resolveProvider(providerName = "antigravity-cli") {
   const normalized = providerName.trim().toLowerCase();
@@ -755,7 +844,7 @@ function runCommand(providerName, invocation, timeoutMs) {
   });
 }
 const program = new Command();
-program.name("modlens").description("Plug-in vision for text-only LLMs: image in, structured JSON evidence out").version("2.1.0");
+program.name("modlens").description("Plug-in vision for text-only LLMs: image in, structured JSON evidence out").version("2.2.0");
 program.command("analyze", { isDefault: true }).description("Analyze an image into structured JSON evidence (default command)").requiredOption("-i, --input <path|url>", "Input image path or https URL").option("-o, --output <path>", "Write result JSON to a file").option("-m, --model <name>", "Provider model name").option("-p, --provider <name>", `Vision provider (${listProviders().join(", ")})`).option("--prompt <text>", "Extra focus for this image").option("--timeout <ms>", "Provider timeout in milliseconds", "180000").option("--provider-bin <path>", "Provider binary path (default: agy)").option("--workdir <path>", "Working directory for the provider").action(async (options) => {
   try {
     const timeoutMs = Number.parseInt(options.timeout, 10);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@liustack/modlens",
-  "version": "2.1.0",
+  "version": "2.2.0",
   "description": "Plug-in vision for text-only LLMs, powered by the free Antigravity CLI",
   "type": "module",
   "bin": {

package/skills/modlens/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: modlens
-description: "Plug-in vision for text-only models. Use whenever the user shares an image (local path, screenshot, photo, chart, document scan, or image URL) and the active model cannot see images or has no vision tool. Runs the modlens CLI to convert the image into structured JSON evidence: OCR text, layout, semantics, visual clues."
+description: "Plug-in vision for text-only models. Use whenever the user shares an image (local path, screenshot, photo, chart, document scan, or image URL) and the active model cannot see images or has no vision tool. Runs the modlens CLI to convert the image into structured JSON evidence: OCR text, layout, semantics, visual clues. Also use when the user asks how to install, configure, or switch modlens providers (Gemini API key, OpenAI-compatible endpoints, Claude API or Claude Code CLI)."
 allowed-tools:
   - Bash
 ---
@@ -12,6 +12,7 @@ Use this skill when:
 - The user provides an image path or image URL and asks anything about it
 - The active model has no native vision (text-only model in a coding agent)
 - You need OCR text, layout, or chart/document structure as evidence before reasoning
+- The user asks how to configure modlens, get an API key for it, or switch its provider: follow `references/configure.md` and run the commands for them
 
 Do not use this skill for:
 
@@ -26,7 +27,7 @@ modlens --version
 
 If `modlens` is missing, run it via `npx @liustack/modlens` instead.
 
-ModLens supports four vision providers. Check what is configured:
+ModLens supports five vision providers. Check what is configured:
 
 ```bash
 modlens config show
@@ -36,8 +37,9 @@ modlens config show
 - **gemini-api**: needs `GEMINI_API_KEY` env or `modlens config set gemini-api.apiKey <key>` (free key from https://aistudio.google.com).
 - **openai**: any OpenAI-compatible multimodal endpoint; needs baseUrl + apiKey + model via env (`OPENAI_BASE_URL`, `OPENAI_API_KEY`) or `modlens config set openai.<field> <value>`.
 - **anthropic**: needs `ANTHROPIC_API_KEY` env or config; defaults to Claude Haiku.
+- **claude-cli**: rides an existing Claude Code login (`claude`), no key, Read-only tool permissions, local files only.
 
-`modlens config init` writes a starter config to `~/.modlens/config.json` when none exists.
+`modlens config init` writes a starter config to `~/.modlens/config.json` when none exists. Full setup recipes per provider: `references/configure.md`.
 
 ## Command
 
@@ -55,7 +57,7 @@ Optional flags:
 modlens -i <image> -o <output.json> -m <model> --prompt "<extra focus>" --timeout <ms>
 ```
 
-Speed expectations: `gemini-api` typically 5-10 seconds, `antigravity-cli` 15-40 seconds (full agent loop), `openai`/`anthropic` depend on the endpoint. For dense or hard images on antigravity-cli, try `-m gemini-3.1-pro-high`.
+Speed expectations: `gemini-api` typically 5-10 seconds, `antigravity-cli` 15-40 seconds and `claude-cli` 20-45 seconds (full agent loops), `openai`/`anthropic` depend on the endpoint. For dense or hard images on antigravity-cli, try `-m gemini-3.1-pro-high`.
 
 ## Finding the image path in the chat
 
@@ -85,7 +87,7 @@ Top level: `{ image, provider, result, meta }`. Inside `result`:
 - `visual`: colors and style clues
 - `uncertainty[]`: what the vision engine was unsure about
 
-Structure is enforced by schema on antigravity-cli (`--json-schema`), gemini-api (`responseJsonSchema`), and anthropic (forced tool call). The openai route uses a template prompt plus shape validation and fails loudly on mismatch.
+Structure is enforced by schema on antigravity-cli and claude-cli (`--json-schema`), gemini-api (`responseJsonSchema`), and anthropic (forced tool call). The openai route uses a template prompt plus shape validation and fails loudly on mismatch.
 
 ## Failure Handling
 

package/skills/modlens/references/configure.md

@@ -0,0 +1,85 @@
+# Configuring ModLens
+
+Read this when the user asks how to set up, configure, or switch ModLens providers. Prefer running the commands for the user over explaining them.
+
+## Where config lives
+
+`~/.modlens/config.json`, managed by the CLI. Precedence: CLI flags > environment variables > config file > built-in defaults. The default provider with zero config is `antigravity-cli`.
+
+```bash
+modlens config init                     # write a starter config (refuses to overwrite; --force to redo)
+modlens config show                     # effective file, API keys masked
+modlens config set provider <name>      # change the default provider
+modlens config set <provider>.<field> <value>   # fields: apiKey, baseUrl, model
+```
+
+`config set` writes the file with 0600 permissions.
+
+## Provider setup recipes
+
+### antigravity-cli (default, free, no key)
+
+Needs Antigravity CLI installed and signed in:
+
+```bash
+curl -fsSL https://antigravity.google/cli/install.sh | bash
+agy    # user must complete browser sign-in themselves, then exit
+```
+
+Any free Google account works; no Google AI Pro needed. Sign-in cannot be automated, ask the user to run `agy` once.
+
+### gemini-api (free key, fastest free route, 5-10s)
+
+1. The user creates a key at https://aistudio.google.com (three minutes, no credit card, free tier does not expire).
+2. Store it either way:
+
+```bash
+modlens config set gemini-api.apiKey <key>
+# or environment: export GEMINI_API_KEY=<key>
+```
+
+Default model `gemini-3.6-flash` has vision on the free tier (about 10-15 requests/min, 1500/day). Free-tier data may be used by Google to improve products; mention this if the user handles sensitive images.
+
+### openai (any OpenAI-compatible multimodal endpoint)
+
+Needs three values. Example for DashScope qwen:
+
+```bash
+modlens config set openai.baseUrl https://dashscope.aliyuncs.com/compatible-mode/v1
+modlens config set openai.apiKey <sk-key>
+modlens config set openai.model qwen3.6-27b
+```
+
+For official OpenAI: baseUrl `https://api.openai.com/v1`, a vision-capable model. Environment equivalents: `OPENAI_BASE_URL`, `OPENAI_API_KEY`. The model must be multimodal; text-only models will fail or hallucinate. This route has no server-side schema enforcement, so occasional shape failures are surfaced as explicit errors; retry or switch provider.
+
+### anthropic (Claude API key)
+
+```bash
+modlens config set anthropic.apiKey <sk-ant-key>
+# or: export ANTHROPIC_API_KEY=<key>
+```
+
+Default model is Claude Haiku (`claude-haiku-4-5-20251001`). Schema is enforced through a forced tool call.
+
+### claude-cli (Claude Code login, no key)
+
+Rides an existing `claude` sign-in, so it costs the user's Claude subscription quota, not a separate API bill. Requires Claude Code installed and logged in (`claude --version` to check). Runs with `--allowedTools Read` only. Local image files only; for remote URLs use gemini-api instead. Default model alias `haiku`.
+
+```bash
+modlens config set provider claude-cli   # make it the default if the user wants
+```
+
+## Choosing a provider for the user
+
+- Wants zero setup and free: `antigravity-cli` (needs agy sign-in, 15-40s per image).
+- Wants fast and free: `gemini-api` (three-minute key, 5-10s).
+- Already pays for Claude: `claude-cli` (no extra key) or `anthropic` (API billing).
+- Has a favorite multimodal endpoint (qwen, GLM, ...): `openai`.
+
+## Troubleshooting
+
+- Error names a missing env var or `config set` command: run exactly that.
+- `Provider CLI not found: agy`: install Antigravity CLI or switch provider.
+- `Claude CLI reported ...` or empty result: check `claude` login state.
+- openai route `does not match the vision schema`: retry once, then switch to gemini-api or anthropic.
+- `config init` refusing to run: the file exists; use `modlens config show` first, `--force` only if the user agrees to overwrite.