@liustack/modlens 3.6.0 → 3.7.0

package/CHANGELOG.md

@@ -1,5 +1,10 @@
 # Changelog
 
+## 3.7.0 - 2026-08-13
+
+- An image-extension path is now the skill's primary trigger. Transcript forensics on a Claude Code cli session explained why the skill loaded but never fired for a text-only model there: the harness silently swaps a pasted image for a usable `[Image: source: <path>]` line, so no failure ever pushes the model to consult its skill list, and a path in hand makes hand-rolled OCR the path of least resistance (the same model behind OpenCode, where the gateway error "model does not support image input" is loud, found and followed this skill exactly). The description now keys on what the model perceives first: any file path or URL ending in an image extension (.png, .jpg, .jpeg, .webp, .gif, .bmp, .heic) it cannot see behind is a hard trigger, with self-built OCR, PIL, and tesseract explicitly forbidden. Placeholder cues (`[Image #1]`, `[Unsupported Image]`) remain as the second tier.
+- First slice of auto mode (borrowing local harness vision, design in progress): a read-only discovery module probes the four supported harnesses without spending anything, and `doctor` grew an Auto section showing what it found. claude on PATH counts as borrowable (Anthropic's lineup is all multimodal), the codex model catalog is judged by its own `input_modalities`, pi's models-store is crossed with its stored credentials so only borrowable vision counts, and one `opencode models` listing is judged by a builtin vision-pattern table that harness metadata outranks. Results cache to `~/.modlens/auto-cache.json` (6h TTL, doctor always probes fresh). The `auto` config switch parses as a strict boolean and defaults to off; nothing routes through these results yet, so behavior with the switch off (or on) is unchanged from 3.6.0.
+
 ## 3.6.0 - 2026-08-13
 
 - The skill now triggers on what a text-only model can actually see. Field testing with DeepSeek behind an Anthropic-compatible gateway showed the old description asking the model to judge "can I see images", the exact self-assessment that fails when a gateway strips images silently, so the skill loaded but never fired. The description now keys on visible evidence: placeholders like `[Image #1]` and `[Unsupported Image]` trigger with a guard check as backup, and a `[Image: source: <path>]` line with no visible image content is a hard trigger, because that line means the harness stored the pasted image on disk and did not deliver it. Newer Claude Code builds write pastes to `~/.claude/image-cache/<session>/` and inject that line from the cli entrypoint (all models get it, vision models also get the real image and are immune to the trigger by the no-visible-content condition; the VSCode entrypoint injects nothing). The skill reads that path directly when it is alive, falls back to `recover-paste` when it is not, and never deletes Claude Code's own cache files.

package/dist/main.js

@@ -3,7 +3,7 @@ import { Command } from "commander";
 import * as fs from "fs";
 import * as path from "path";
 import * as childProcess from "child_process";
-import { spawn } from "child_process";
+import { spawn, execFileSync } from "child_process";
 import * as os from "os";
 import { fileURLToPath } from "url";
 import require$$0$1 from "node:assert";
@@ -28149,13 +28149,19 @@ function setConfigValue(dottedKey, value, configPath = CONFIG_PATH) {
   const config2 = loadConfigFile(configPath);
   if (dottedKey === "provider") {
     config2.provider = value;
+  } else if (dottedKey === "auto") {
+    const normalized = value.trim().toLowerCase();
+    if (normalized !== "true" && normalized !== "false") {
+      throw new Error("auto must be true or false.");
+    }
+    config2.auto = normalized === "true";
   } else if (dottedKey.startsWith("guards.")) {
     setGuardsValue(config2, dottedKey.slice("guards.".length), value);
   } else {
     const dot = dottedKey.indexOf(".");
     if (dot <= 0 || dot === dottedKey.length - 1) {
       throw new Error(
-        `Invalid config key: ${dottedKey}. Use "provider", "guards.<denyModels|allowModels|denyWhenUnknown>", or "<provider>.<apiKey|baseUrl|model|extraBody>".`
+        `Invalid config key: ${dottedKey}. Use "provider", "auto", "guards.<denyModels|allowModels|denyWhenUnknown>", or "<provider>.<apiKey|baseUrl|model|extraBody>".`
       );
     }
     const providerName = dottedKey.slice(0, dot);
@@ -28657,6 +28663,286 @@ function runCommand(providerName, invocation, timeoutMs, describeFailure) {
     child.on("close", (code) => settle(code));
   });
 }
+function denyPatterns(guards) {
+  return stringPatterns(guards?.denyModels);
+}
+function allowPatterns(guards) {
+  return stringPatterns(guards?.allowModels);
+}
+function stringPatterns(raw) {
+  if (!Array.isArray(raw)) {
+    return [];
+  }
+  return raw.filter((pattern) => typeof pattern === "string");
+}
+function globMatch(pattern, value) {
+  const regex = pattern.split(/([*?])/).map((part) => {
+    if (part === "*") {
+      return ".*";
+    }
+    if (part === "?") {
+      return ".";
+    }
+    return part.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
+  }).join("");
+  return new RegExp(`^${regex}$`, "i").test(value);
+}
+function evaluateGuard(guards, detection) {
+  const deny = denyPatterns(guards);
+  const allow = allowPatterns(guards);
+  if (!detection.model) {
+    if (guards?.denyWhenUnknown === true) {
+      return {
+        ...detection,
+        guard: "deny",
+        reason: "model unknown and denyWhenUnknown is set"
+      };
+    }
+    return {
+      ...detection,
+      guard: "allow",
+      reason: deny.length === 0 && allow.length === 0 ? "no deny rules configured" : "model unknown, failing open"
+    };
+  }
+  if (deny.length === 0 && allow.length === 0) {
+    return { ...detection, guard: "allow", reason: "no deny rules configured" };
+  }
+  const candidates = [detection.model];
+  if (detection.provider) {
+    candidates.push(`${detection.provider}/${detection.model}`);
+  }
+  const firstMatch = (patterns) => patterns.find((pattern) => candidates.some((candidate) => globMatch(pattern, candidate)));
+  const denied = firstMatch(deny);
+  if (denied) {
+    return {
+      ...detection,
+      guard: "deny",
+      matched: denied,
+      reason: "model has native vision per guards.denyModels"
+    };
+  }
+  if (allow.length > 0) {
+    const allowed = firstMatch(allow);
+    if (allowed) {
+      return {
+        ...detection,
+        guard: "allow",
+        matched: allowed,
+        reason: "model is on guards.allowModels"
+      };
+    }
+    return {
+      ...detection,
+      guard: "deny",
+      reason: "not on guards.allowModels: only listed models run the engine"
+    };
+  }
+  return { ...detection, guard: "allow", reason: "not on the deny list" };
+}
+const VISION_MODEL_PATTERNS = [
+  "claude-*",
+  "gpt-4o*",
+  "gpt-4.1*",
+  "gpt-5*",
+  "o3*",
+  "o4*",
+  "gemini-*",
+  "glm-*v*",
+  "qwen*-vl*",
+  "qwen3.5-plus*",
+  "qwen3.6-plus*",
+  "kimi-k2.5*",
+  "kimi-k2.6*",
+  "kimi-k2.7*",
+  "kimi-k3*",
+  "moonshot-v1-*vision*",
+  "minimax-vl*",
+  "minimax-m3*",
+  "deepseek-vl*",
+  "deepseek-ocr*",
+  "janus*",
+  "pixtral*",
+  "llama-4*",
+  "llama-3.2-*vision*",
+  "grok-4*",
+  "grok-2-vision*",
+  "internvl*"
+];
+function isVisionModel(modelId) {
+  const bare = modelId.includes("/") ? modelId.slice(modelId.lastIndexOf("/") + 1) : modelId;
+  return VISION_MODEL_PATTERNS.some((pattern) => globMatch(pattern, bare));
+}
+const DEFAULT_TTL_MS = 6 * 60 * 60 * 1e3;
+const CLI_TIMEOUT_MS = 1e4;
+function defaultRunCli(bin, args, timeoutMs) {
+  return execFileSync(bin, args, { encoding: "utf-8", timeout: timeoutMs, stdio: "pipe" });
+}
+function timed(run) {
+  const start = Date.now();
+  const probe = run();
+  return { ...probe, elapsedMs: Date.now() - start };
+}
+function readJson(filePath) {
+  return JSON.parse(fs.readFileSync(filePath, "utf-8"));
+}
+function probeClaude(env) {
+  return timed(() => {
+    const cliPath = findOnPath("claude", env);
+    if (!cliPath) {
+      return {
+        harness: "claude-code",
+        cliFound: false,
+        visionModels: [],
+        source: "none"
+      };
+    }
+    return {
+      harness: "claude-code",
+      cliFound: true,
+      cliPath,
+      visionModels: ["anthropic/* (all current models)"],
+      source: "builtin-table"
+    };
+  });
+}
+function probeCodex(env, home) {
+  return timed(() => {
+    const cliPath = findOnPath("codex", env);
+    const base = { harness: "codex", cliFound: cliPath !== null };
+    if (!cliPath) {
+      return { ...base, visionModels: [], source: "none" };
+    }
+    const codexHome = path.join(home, ".codex");
+    const loggedIn = fs.existsSync(path.join(codexHome, "auth.json"));
+    try {
+      const toml = fs.readFileSync(path.join(codexHome, "config.toml"), "utf-8");
+      const catalogPath = toml.match(/^model_catalog_json\s*=\s*"([^"]+)"/m)?.[1];
+      if (!catalogPath) {
+        return { ...base, cliPath, loggedIn, visionModels: [], source: "none" };
+      }
+      const catalog = readJson(catalogPath);
+      const vision = (catalog.models ?? []).filter((m) => m.slug && (m.input_modalities ?? []).includes("image")).map((m) => m.slug);
+      return {
+        ...base,
+        cliPath,
+        loggedIn,
+        visionModels: vision,
+        source: "metadata"
+      };
+    } catch (error) {
+      return {
+        ...base,
+        cliPath,
+        loggedIn,
+        visionModels: [],
+        source: "none",
+        error: error instanceof Error ? error.message : String(error)
+      };
+    }
+  });
+}
+function probePi(env, home) {
+  return timed(() => {
+    const cliPath = findOnPath("pi", env);
+    const base = { harness: "pi", cliFound: cliPath !== null };
+    if (!cliPath) {
+      return { ...base, visionModels: [], source: "none" };
+    }
+    const agentDir = path.join(home, ".pi", "agent");
+    try {
+      const auth = readJson(path.join(agentDir, "auth.json"));
+      const providersWithCreds = new Set(Object.keys(auth));
+      const store = readJson(path.join(agentDir, "models-store.json"));
+      const vision = [];
+      for (const entry of Object.values(store)) {
+        for (const model of entry?.models ?? []) {
+          if (model.id && (model.input ?? []).includes("image") && model.provider && providersWithCreds.has(model.provider)) {
+            vision.push(model.id);
+          }
+        }
+      }
+      return {
+        ...base,
+        cliPath,
+        loggedIn: providersWithCreds.size > 0,
+        visionModels: vision,
+        source: "metadata"
+      };
+    } catch (error) {
+      return {
+        ...base,
+        cliPath,
+        visionModels: [],
+        source: "none",
+        error: error instanceof Error ? error.message : String(error)
+      };
+    }
+  });
+}
+function probeOpencode(env, runCli) {
+  return timed(() => {
+    const cliPath = findOnPath("opencode", env);
+    const base = { harness: "opencode", cliFound: cliPath !== null };
+    if (!cliPath) {
+      return { ...base, visionModels: [], source: "none" };
+    }
+    try {
+      const listing = runCli(cliPath, ["models"], CLI_TIMEOUT_MS);
+      const vision = listing.split("\n").map((line) => line.trim()).filter((line) => line.length > 0 && isVisionModel(line));
+      return { ...base, cliPath, visionModels: vision, source: "builtin-table" };
+    } catch (error) {
+      return {
+        ...base,
+        cliPath,
+        visionModels: [],
+        source: "none",
+        error: error instanceof Error ? error.message : String(error)
+      };
+    }
+  });
+}
+function readCache(cachePath, ttlMs) {
+  try {
+    const cached = readJson(cachePath);
+    if (!cached.cachedAt || !Array.isArray(cached.probes)) {
+      return null;
+    }
+    if (Date.now() - Date.parse(cached.cachedAt) > ttlMs) {
+      return null;
+    }
+    return cached;
+  } catch {
+    return null;
+  }
+}
+function discoverAuto(options = {}) {
+  const env = options.env ?? process.env;
+  const home = options.home ?? os.homedir();
+  const cachePath = options.cachePath ?? path.join(home, ".modlens", "auto-cache.json");
+  const ttlMs = options.ttlMs ?? DEFAULT_TTL_MS;
+  if (!options.fresh) {
+    const cached = readCache(cachePath, ttlMs);
+    if (cached) {
+      return { probes: cached.probes, cachedAt: cached.cachedAt, fromCache: true };
+    }
+  }
+  const runCli = options.runCli ?? defaultRunCli;
+  const probes = [
+    probeClaude(env),
+    probeCodex(env, home),
+    probeOpencode(env, runCli),
+    probePi(env, home)
+  ];
+  const cachedAt = (/* @__PURE__ */ new Date()).toISOString();
+  try {
+    fs.mkdirSync(path.dirname(cachePath), { recursive: true });
+    fs.writeFileSync(cachePath, JSON.stringify({ cachedAt, probes }, null, 2), {
+      mode: 384
+    });
+  } catch {
+  }
+  return { probes, cachedAt, fromCache: false };
+}
 const HARNESS_BY_BASENAME = {
   claude: "claude-code",
   "claude-code": "claude-code",
@@ -29259,82 +29545,6 @@ function sniffModel(harness, cwd, env, roots = {}) {
     return null;
   }
 }
-function denyPatterns(guards) {
-  return stringPatterns(guards?.denyModels);
-}
-function allowPatterns(guards) {
-  return stringPatterns(guards?.allowModels);
-}
-function stringPatterns(raw) {
-  if (!Array.isArray(raw)) {
-    return [];
-  }
-  return raw.filter((pattern) => typeof pattern === "string");
-}
-function globMatch(pattern, value) {
-  const regex = pattern.split(/([*?])/).map((part) => {
-    if (part === "*") {
-      return ".*";
-    }
-    if (part === "?") {
-      return ".";
-    }
-    return part.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
-  }).join("");
-  return new RegExp(`^${regex}$`, "i").test(value);
-}
-function evaluateGuard(guards, detection) {
-  const deny = denyPatterns(guards);
-  const allow = allowPatterns(guards);
-  if (!detection.model) {
-    if (guards?.denyWhenUnknown === true) {
-      return {
-        ...detection,
-        guard: "deny",
-        reason: "model unknown and denyWhenUnknown is set"
-      };
-    }
-    return {
-      ...detection,
-      guard: "allow",
-      reason: deny.length === 0 && allow.length === 0 ? "no deny rules configured" : "model unknown, failing open"
-    };
-  }
-  if (deny.length === 0 && allow.length === 0) {
-    return { ...detection, guard: "allow", reason: "no deny rules configured" };
-  }
-  const candidates = [detection.model];
-  if (detection.provider) {
-    candidates.push(`${detection.provider}/${detection.model}`);
-  }
-  const firstMatch = (patterns) => patterns.find((pattern) => candidates.some((candidate) => globMatch(pattern, candidate)));
-  const denied = firstMatch(deny);
-  if (denied) {
-    return {
-      ...detection,
-      guard: "deny",
-      matched: denied,
-      reason: "model has native vision per guards.denyModels"
-    };
-  }
-  if (allow.length > 0) {
-    const allowed = firstMatch(allow);
-    if (allowed) {
-      return {
-        ...detection,
-        guard: "allow",
-        matched: allowed,
-        reason: "model is on guards.allowModels"
-      };
-    }
-    return {
-      ...detection,
-      guard: "deny",
-      reason: "not on guards.allowModels: only listed models run the engine"
-    };
-  }
-  return { ...detection, guard: "allow", reason: "not on the deny list" };
-}
 function detectActiveModel(options) {
   const env = options.env ?? process.env;
   const envModel = env.MODLENS_MODEL?.trim();
@@ -29513,7 +29723,13 @@ function buildDoctorReport(input) {
       matched: guardVerdict.matched,
       reason: guardVerdict.reason
     },
-    config: inspectConfigFile(configPath)
+    config: inspectConfigFile(configPath),
+    // doctor is the "what would auto find" view, so it always probes fresh
+    // (and rewrites the cache); regular runs will read the cache instead.
+    auto: {
+      enabled: input.config.auto === true,
+      probes: discoverAuto({ env, fresh: true, ...input.auto }).probes
+    }
   };
 }
 function mark(ok) {
@@ -29562,6 +29778,30 @@ function renderDoctorReport(report) {
     `  verdict: ${report.guard.verdict}${report.guard.matched ? ` (matched "${report.guard.matched}")` : ""}, ${report.guard.reason}`
   );
   lines.push("");
+  lines.push("Auto (borrowable local harness vision; off by default)");
+  lines.push(
+    `  enabled: ${report.auto.enabled}${report.auto.enabled ? "" : " (turn on: modlens config set auto true)"}`
+  );
+  for (const probe of report.auto.probes) {
+    if (!probe.cliFound) {
+      lines.push(`  ${probe.harness}: cli not found`);
+      continue;
+    }
+    const parts = [];
+    const shown = probe.visionModels.slice(0, 3).join(", ");
+    parts.push(
+      probe.visionModels.length === 0 ? "no vision models" : `${probe.visionModels.length} vision model(s): ${shown}${probe.visionModels.length > 3 ? ", ..." : ""}`
+    );
+    if (probe.loggedIn !== void 0) {
+      parts.push(probe.loggedIn ? "logged in" : "no credentials found");
+    }
+    parts.push(`via ${probe.source}, ${probe.elapsedMs}ms`);
+    if (probe.error) {
+      parts.push(`error: ${probe.error}`);
+    }
+    lines.push(`  ${probe.harness}: ${parts.join(", ")}`);
+  }
+  lines.push("");
   lines.push("Config file");
   lines.push(`  path: ${report.config.path}`);
   if (report.config.exists) {
@@ -29754,7 +29994,7 @@ function recoverPastedImages(options = {}) {
   return result;
 }
 const program = new Command();
-program.name("modlens").description("Plug-in vision for text-only LLMs: image in, structured JSON evidence out").version("3.6.0");
+program.name("modlens").description("Plug-in vision for text-only LLMs: image in, structured JSON evidence out").version("3.7.0");
 program.command("analyze", { isDefault: true }).description("Analyze an image into structured JSON evidence (default command)").requiredOption("-i, --input <path|url>", "Input image path or https URL").option("-o, --output <path>", "Write result JSON to a file").option("-m, --model <name>", "Provider model name").option("-p, --provider <name>", `Vision provider (${listProviders().join(", ")})`).option("--prompt <text>", "Extra focus for this image").option("--timeout <ms>", "Provider timeout in milliseconds", "180000").option("--provider-bin <path>", "Provider binary path (default: agy)").option("--workdir <path>", "Working directory for the provider").option(
   "--extra-body <json>",
   `JSON merged into the API request body, e.g. '{"thinking":{"type":"disabled"}}'`

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@liustack/modlens",
-  "version": "3.6.0",
+  "version": "3.7.0",
   "description": "Plug-in vision for text-only LLMs, powered by the free Antigravity CLI",
   "type": "module",
   "bin": {

package/skills/modlens/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: modlens
-description: "Plug-in vision for text-only models. Use whenever an image is in play and you cannot see its content: the user gives an image path, screenshot, photo, chart, document scan, or image URL, or a pasted image appears only as a placeholder such as `[Image #1]`, `[Unsupported Image]`, or an attachment you cannot view. Hard rule: a `[Image: source: <path>]` line with no visible image content means the harness stored the pasted image at that path and did not deliver it to you; run modlens on that path directly. If you can actually see the image, do not use this skill. When unsure, run `modlens guard` before the first read of a session: a deny verdict means the active model has native vision and must read the image itself. Runs the modlens CLI to convert the image into structured JSON evidence: every word transcribed, layout regions, semantics, visual clues. Also use when the user asks how to install, configure, or switch modlens providers (Gemini API key, OpenAI-compatible endpoints, Claude API or Claude Code CLI)."
+description: "Plug-in vision for text-only models. Hard rule: when a file path or URL with an image extension (.png, .jpg, .jpeg, .webp, .gif, .bmp, .heic) appears anywhere in the conversation (typed by the user, injected as a `[Image: source: <path>]` line, or inside a tag) and you cannot see that image's content, run this skill on it before any other approach: no self-built OCR, no PIL, no tesseract. Also triggers on pasted-image placeholders such as `[Image #1]` and `[Unsupported Image]`. If you can actually see the image, do not use this skill. When unsure, run `modlens guard` before the first read of a session: a deny verdict means the active model has native vision and must read the image itself. Runs the modlens CLI to convert the image into structured JSON evidence: every word transcribed, layout regions, semantics, visual clues. Also use when the user asks how to install, configure, or switch modlens providers (Gemini API key, OpenAI-compatible endpoints, Claude API or Claude Code CLI)."
 compatibility: Requires network access and one of node 22+/npx, bun/bunx, or a preinstalled modlens binary on PATH.
 allowed-tools: Bash
 ---
@@ -9,7 +9,7 @@ allowed-tools: Bash
 
 Use this skill when:
 
-- The user provides an image path or image URL and asks anything about it
+- A file path or URL ending in an image extension (`.png`, `.jpg`, `.jpeg`, `.webp`, `.gif`, `.bmp`, `.heic`) appears and you cannot see the image behind it. The path alone is the trigger: hand it to modlens first, do not Read the bytes yourself, do not build your own OCR out of PIL or tesseract
 - A pasted image reaches you only as a placeholder: `[Image #1]`, `[Unsupported Image]`, a `[Image: source: <path>]` line, or an attachment whose content you cannot see
 - The active model has no native vision (text-only model in a coding agent)
 - You need the text inside an image, its layout, or a chart's structure as evidence before reasoning
@@ -34,11 +34,11 @@ The launcher finds a working way to run modlens and forwards your arguments to i
 
 ### If you cannot run the launcher script
 
-Some harnesses forbid running scripts. Reason through the same order by hand and run the first line that works (the pinned version is 3.6.0):
+Some harnesses forbid running scripts. Reason through the same order by hand and run the first line that works (the pinned version is 3.7.0):
 
-1. A `modlens` on `PATH` whose major version is 3 and is at least 3.6.0: `modlens <args>`.
-2. Otherwise, if `npx` exists: `npx --yes --package @liustack/modlens@3.6.0 modlens <args>`.
-3. Otherwise, if `bunx` exists: `bunx --bun @liustack/modlens@3.6.0 <args>`.
+1. A `modlens` on `PATH` whose major version is 3 and is at least 3.7.0: `modlens <args>`.
+2. Otherwise, if `npx` exists: `npx --yes --package @liustack/modlens@3.7.0 modlens <args>`.
+3. Otherwise, if `bunx` exists: `bunx --bun @liustack/modlens@3.7.0 <args>`.
 4. Otherwise none of these runtimes is here. Tell the user no JavaScript runtime was found and that installing Node 22.13+ (https://nodejs.org) or Bun (https://bun.sh) is the next step. Do not claim modlens itself failed.
 
 `references/runtime.md` documents the version pin, the compatibility rule, and the diagnostic fields.

package/skills/modlens/references/runtime.md

@@ -8,7 +8,7 @@ shell syntax.
 
 ## Pinned version
 
-- Pinned CLI version: 3.6.0
+- Pinned CLI version: 3.7.0
 - npm package: `@liustack/modlens`
 - CLI binary name: `modlens`
 

package/skills/modlens/scripts/run.ps1

@@ -24,7 +24,7 @@ $ErrorActionPreference = 'Stop'
 # package.json version, and the release script rewrites it on every bump.
 $Package = '@liustack/modlens'
 $Bin = 'modlens'
-$Pinned = '3.6.0'
+$Pinned = '3.7.0'
 # -------------------------------------------------------------------------------
 
 $NativeNote = 'no native artifact is published for this tool yet; phase A ships npm launch paths only'

package/skills/modlens/scripts/run.sh

@@ -22,7 +22,7 @@ set -eu
 # package.json version, and the release script rewrites it on every bump.
 PKG="@liustack/modlens"
 BIN="modlens"
-PINNED="3.6.0"
+PINNED="3.7.0"
 # -------------------------------------------------------------------------------
 
 NATIVE_NOTE="no native artifact is published for this tool yet; phase A ships npm launch paths only"