@liustack/modlens 3.9.1 → 3.10.0

package/CHANGELOG.md

@@ -1,5 +1,11 @@
 # Changelog
 
+## 3.10.0 - 2026-08-14
+
+- **Pasting into the dsh Web UI now works end to end, thumbnails included.** The plugin registers a wrapper provider whose model list carries two entries, `DeepSeek-V4-Flash (modlens vision)` and `DeepSeek-V4-Pro (modlens vision)`; select one and pastes are admitted, because dsh's image admission asks the selected provider for input modalities and the wrapper declares image input. Conversion happens at request time on the wire messages only: the durable session log and the UI keep the native image blocks (your paste keeps its thumbnail), each pasted attachment is read once through the modlens CLI and cached for every later step, and the wrapped request is answered by the same DeepSeek route it always used. The wrap covers DeepSeek and GLM text models only; their own vision models (deepseek-vl/ocr, janus, glm v-series) are excluded by name pattern and by declared modality, so a model that can already see is never wrapped. The `agent/pre-step` auto-read from 3.9.0 becomes opt-in (`autoRead: true`) since request-time conversion owns the paste path and preserves the native UX. Registration also stopped failing silently: the adapter now supplies the base-class methods dsh's duck-typing expects (`providerInfo`, `providerRetryPolicy`), whose absence was exactly the invisible failure, and a registration error is logged to the harness console instead of swallowed.
+- **Paste auto-read read a field the attachment store never had ([#17](https://github.com/liustack/modlens/issues/17)).** dsh's `attachments.readImage` returns `{ ref, data }`, but the 3.9.1 plugin read `stored.bytes` and `stored.mediaType`, so every pasted image degraded to a `Buffer.from(undefined)` TypeError block. The plugin now reads the real shape (`data`, with the media type on `ref`), and a missing byte field fails with a named error pointing at the dsh attachment shape instead of a bare TypeError, so the next developer-preview drift stays legible. Thanks to @nico2656-ui for the rc.6 type archaeology and a verified patch.
+- The README leads its demo gallery with the real dsh paste session (recovered from the machine that ran it), states the wrap scope, and lists both variant names verbatim under a rather more visible first-plugin banner.
+
 ## 3.9.1 - 2026-08-13
 
 - dsh follow-ups that missed the 3.9.0 tag during a GitHub outage: the plugin row references the bare package name via a root export, so the dsh plugin list shows `modlens` instead of `modlens/dsh`; install lines use `npx -y @deepseek-ai/dsh` (the developer preview has no global binary); and both READMEs state the paste status honestly, since the dsh DeepSeek adapter declares text-only input and Web-UI image admission runs before any plugin hook, `read_image` is the working path today and paste auto-read stays wired for when images can enter.

package/README.md

@@ -6,6 +6,8 @@
 
 <p align="center"><b>Give a text-only model sight, and just paste the image.</b></p>
 
+<p align="center">🥇 <b>The FIRST vision plugin for DeepSeek Harness (dsh)</b> 🥇</p>
+
 <p align="center">
   <a href="./README.zh-CN.md">简体中文</a> ·
   <a href="docs/troubleshooting.md">Troubleshooting</a> ·
@@ -32,7 +34,7 @@ Issues are welcome any time: [open one](https://github.com/liustack/modlens/issu
 
 ## Highlights
 
-**🥇 The first plug-in vision plugin for DeepSeek Harness (dsh):** one command, `npx -y @deepseek-ai/dsh plugin --profile web add @liustack/modlens`, and the text-only DeepSeek model behind dsh reads images through a native `read_image` tool. (Pasted-image auto-read is wired but currently blocked upstream: the dsh DeepSeek adapter declares text-only, so the Web UI refuses pastes before any plugin runs; reference images by file path for now.)
+**🥇 The first vision plugin for DeepSeek Harness (dsh):** one command, `npx -y @deepseek-ai/dsh plugin --profile web add @liustack/modlens`, and the text-only DeepSeek model behind dsh reads images through a native `read_image` tool. For pasted images, switch the model selector to one of the two entries the plugin adds, **`DeepSeek-V4-Flash (modlens vision)`** or **`DeepSeek-V4-Pro (modlens vision)`**: pastes are admitted there, converted to evidence at request time (your message keeps its native thumbnail), and answered by the same DeepSeek route. The wrap covers DeepSeek and GLM text models only; their own vision models are excluded automatically.
 
 **Paste an image and it reads it.** No saving to a file and passing a path first.
 
@@ -65,6 +67,10 @@ Once installed, just chat. Paste an image or drop a path, ask anything, and the
 
 Unedited runs, all driving a text-only DeepSeek-V4-Flash.
 
+The newest one first: pasting a screenshot straight into DeepSeek Harness on the `DeepSeek-V4-Flash (modlens vision)` variant. The paste keeps its native thumbnail, the trajectory shows the image arriving "already transcribed by the modlens vision bridge", and the answer walks the UI element by element.
+
+![Pasting an image straight into DeepSeek Harness, read through the modlens vision plugin](https://raw.githubusercontent.com/liustack/modlens/main/assets/demo-dsh-paste.jpg)
+
 A tweet screenshot in the Codex desktop app. It reads the author, the caption, the photo itself (down to what both people are wearing), the timestamp, and every engagement number: 5.4M views, 1.6K replies, 5.7K reposts, 116K likes.
 
 ![Text-only DeepSeek reading a tweet screenshot in full detail via ModLens](https://raw.githubusercontent.com/liustack/modlens/main/assets/demo-codex-app.jpg)

package/README.zh-CN.md

@@ -6,6 +6,8 @@
 
 <p align="center"><b>为纯文本模型补上视觉能力，直接粘贴图片就能识别。</b></p>
 
+<p align="center">🥇 <b>全网第一个支持 DeepSeek Harness（dsh）的视觉插件</b> 🥇</p>
+
 <p align="center">
   <a href="./README.md">English</a> ·
   <a href="docs/troubleshooting.md">故障排查</a> ·
@@ -32,7 +34,7 @@ DeepSeek 和 GLM 没有视觉能力，无法进行图片识别。ModLens 借助
 
 ## 亮点
 
-**🥇 全网第一个支持 DeepSeek Harness（dsh）的外挂视觉识别插件：**一条命令 `npx -y @deepseek-ai/dsh plugin --profile web add @liustack/modlens`，dsh 背后的纯文本 DeepSeek 模型即可通过原生 `read_image` 工具读图。（粘贴自动识别已内置，但目前被上游拦截：dsh 的 DeepSeek 适配器声明纯文本，Web 界面在任何插件运行前就拒绝贴图，暂时请用文件路径指图。）
+**🥇 全网第一个支持 DeepSeek Harness（dsh）的外挂视觉识别插件：**一条命令 `npx -y @deepseek-ai/dsh plugin --profile web add @liustack/modlens`，dsh 背后的纯文本 DeepSeek 模型即可通过原生 `read_image` 工具读图。要粘贴识图，把模型选择器切到插件新增的两个条目之一：**`DeepSeek-V4-Flash (modlens vision)`** 或 **`DeepSeek-V4-Pro (modlens vision)`**，贴图放行、发请求时转成证据（你的消息保留原生缩略图）、仍由原 DeepSeek 路由回答。包装只覆盖 DeepSeek 与 GLM 的文本模型，两家自己的视觉型号自动排除。
 
 **直接粘贴图片识别** 无需先保存成文件再提供路径。
 
@@ -63,7 +65,7 @@ agy                                                           # 浏览器完成
 npx -y @deepseek-ai/dsh plugin --profile web add @liustack/modlens
 ```
 
-装完即有 `read_image` 工具（粘贴自动识别待上游放行图片进气后自动生效）。引擎配置同样在 `~/.modlens`，详见[宿主接入](docs/harness-setup.md)。
+装完即有 `read_image` 工具，选「(modlens vision)」模型变体即可直接粘贴识图。引擎配置同样在 `~/.modlens`，详见[宿主接入](docs/harness-setup.md)。
 
 ## 用法
 
@@ -73,6 +75,10 @@ npx -y @deepseek-ai/dsh plugin --profile web add @liustack/modlens
 
 以下均为原样记录，驱动的都是纯文本的 DeepSeek-V4-Flash。
 
+最新的一条放最前：在 DeepSeek Harness 里选 `DeepSeek-V4-Flash (modlens vision)` 变体直接粘贴截图。粘贴保留原生缩略图，轨迹里可见图片抵达时「已由 modlens 视觉桥转写」，回答逐个元素还原了界面。
+
+![在 DeepSeek Harness 中直接粘贴图片，经 modlens 视觉插件读取](https://raw.githubusercontent.com/liustack/modlens/main/assets/demo-dsh-paste.jpg)
+
 Codex 桌面 App 中识别一张推文截图。作者、配文、照片内容（连两人的穿着都在内）、发帖时间和全部互动数据（540 万浏览、1.6K 回复、5.7K 转发、11.6 万点赞）逐项读出。
 
 ![纯文本 DeepSeek 通过 ModLens 读出推文截图的全部细节](https://raw.githubusercontent.com/liustack/modlens/main/assets/demo-codex-app.jpg)

package/dist/main.js

@@ -30560,7 +30560,7 @@ function recoverPastedImages(options = {}) {
   return result;
 }
 const program = new Command();
-program.name("modlens").description("Plug-in vision for text-only LLMs: image in, structured JSON evidence out").version("3.9.1");
+program.name("modlens").description("Plug-in vision for text-only LLMs: image in, structured JSON evidence out").version("3.10.0");
 program.command("analyze", { isDefault: true }).description("Analyze an image into structured JSON evidence (default command)").requiredOption("-i, --input <path|url>", "Input image path or https URL").option("-o, --output <path>", "Write result JSON to a file").option("-m, --model <name>", "Provider model name").option("-p, --provider <name>", `Vision provider (${listProviders().join(", ")})`).option("--prompt <text>", "Extra focus for this image").option("--timeout <ms>", "Provider timeout in milliseconds", "180000").option("--provider-bin <path>", "Provider binary path (default: agy)").option("--workdir <path>", "Working directory for the provider").option(
   "--extra-body <json>",
   `JSON merged into the API request body, e.g. '{"thinking":{"type":"disabled"}}'`

package/dsh/index.js

@@ -22,7 +22,7 @@ const OUTPUT_SCHEMA = JSON.parse(
 const CLI_TIMEOUT_MS = 180_000
 
 export const name = 'modlens'
-export const inject = ['tools', 'agents', 'attachments']
+export const inject = ['tools', 'agents', 'attachments', 'llm']
 
 const MEDIA_EXT = {
   'image/png': '.png',
@@ -32,9 +32,15 @@ const MEDIA_EXT = {
 }
 
 export function apply(ctx, config = {}) {
-  if (config.autoRead !== false) {
+  // Off by default since the vision provider converts at request time and
+  // keeps the durable log (and the UI thumbnail) intact; turn it on only for
+  // setups where images enter through a provider this plugin does not wrap.
+  if (config.autoRead === true) {
     registerAutoRead(ctx)
   }
+  if (config.visionProvider !== false) {
+    registerVisionProvider(ctx, config)
+  }
   // Registered as a raw JSON-Schema tool definition (no dsh package imports:
   // the developer-preview registry accepts these and out-of-tree resolution
   // of @deepseek-ai/dsh-tools is not yet reliable), so this plugin owns its
@@ -100,6 +106,117 @@ export function apply(ctx, config = {}) {
   })
 }
 
+/**
+ * Phase 3: the paste unlock. dsh's image admission asks the selected
+ * provider's adapter for inputModalities, and the DeepSeek adapter hardcodes
+ * text-only, so pastes are refused before any plugin hook runs. This wrapper
+ * registers a NEW provider whose model metadata declares image input and
+ * whose stream() is a one-line delegation back to the real route. Pick the
+ * wrapped model in the model selector, paste, and the pre-step rewrite below
+ * turns the image into evidence text before the delegated request goes out;
+ * the upstream serializer's own image rejection stays as the fail-closed
+ * backstop. Guarded feature-detection: if the llm registration surface moved
+ * (developer preview), the plugin quietly stays a read_image-only tool.
+ */
+function registerVisionProvider(ctx, config) {
+  const upstream = config.upstream || 'deepseek-official'
+  const providerId = config.providerId || 'deepseek-modlens'
+  // Wrap only the text-only members of these families. Their own vision
+  // models (present or future: deepseek-vl/ocr/janus, glm-4.5v, glm-5v-...)
+  // need no bridge and are excluded by name and by declared modality.
+  const families = config.families || ['deepseek', 'glm']
+  const VISION_ID = /(deepseek-(vl|ocr)|janus|glm-[\d.]*v(\b|-))/i
+  const shouldWrap = (info) => {
+    const id = String(info?.id ?? '').toLowerCase()
+    if (!families.some((family) => id.startsWith(family))) return false
+    if (VISION_ID.test(id)) return false
+    if (Array.isArray(info?.inputModalities) && info.inputModalities.includes('image')) return false
+    return true
+  }
+  if (typeof ctx.llm?.registerAdapter !== 'function' || typeof ctx.llm?.stream !== 'function') {
+    return
+  }
+  const withVision = (info) => ({
+    ...info,
+    provider: providerId,
+    inputModalities: ['text', 'image'],
+  })
+  try {
+    ctx.llm.registerAdapter([providerId], {
+      // Duck-typing LlmAdapter: providerInfo/providerRetryPolicy are base-class
+      // defaults a plain object must supply itself (their absence is exactly
+      // the silent registration failure this catch used to swallow).
+      providerInfo(provider) {
+        return { id: provider, name: 'DeepSeek (modlens vision)' }
+      },
+      providerRetryPolicy() {
+        return undefined
+      },
+      async listModels(_provider, signal) {
+        try {
+          const models = await ctx.llm.listModels(upstream, signal)
+          return models.filter(shouldWrap).map((model) => ({
+            ...withVision(model),
+            name: `${model.name ?? model.id} (modlens vision)`,
+          }))
+        } catch {
+          return []
+        }
+      },
+      async resolveModel(_provider, model, signal) {
+        const info = await ctx.llm.resolveModelInfo(upstream, model, signal)
+        if (!shouldWrap(info)) {
+          throw new Error(`model "${model}" is outside the modlens vision wrap scope`)
+        }
+        return { ...withVision(info), id: model }
+      },
+      stream(options) {
+        // Convert at request time, not at log time: the durable session log
+        // keeps the real image blocks (so the UI shows the paste natively),
+        // and only the wire messages carry evidence text. Cached per
+        // attachment, since the same history rides every later step.
+        const self = this
+        return (async function* () {
+          const messages = await convertImagesToEvidence(ctx, options.messages, options.signal, self)
+          yield* ctx.llm.stream({ ...options, provider: upstream, messages })
+        })()
+      },
+      evidenceCache: new Map(),
+    })
+  } catch (error) {
+    // DUPLICATE_ADAPTER or a preview-era surface change: degrade to the
+    // read_image-only plugin, but say so in the harness log instead of
+    // vanishing (a swallowed TypeError here once hid a missing base method).
+    console.error(`[modlens] vision provider registration skipped: ${error}`)
+  }
+}
+
+async function convertImagesToEvidence(ctx, messages, signal, adapter) {
+  const out = []
+  for (const message of messages) {
+    if (!Array.isArray(message.content) || !message.content.some((b) => b?.type === 'image')) {
+      out.push(message)
+      continue
+    }
+    const content = []
+    for (const block of message.content) {
+      if (block?.type !== 'image') {
+        content.push(block)
+        continue
+      }
+      const key = JSON.stringify(block.attachment ?? block)
+      let text = adapter.evidenceCache.get(key)
+      if (text === undefined) {
+        text = (await readImageBlock(ctx, block, signal)).text
+        adapter.evidenceCache.set(key, text)
+      }
+      content.push({ type: 'text', text })
+    }
+    out.push({ ...message, content })
+  }
+  return out
+}
+
 /**
  * Phase 2: paste auto-route. When entered messages carry image blocks (the
  * Web UI's paste/drop intake) and the model behind dsh is text-only, rewrite
@@ -148,13 +265,20 @@ async function readImageBlock(ctx, block, signal) {
   const { join } = await import('node:path')
   let dir
   try {
+    // StoredImageAttachment carries { ref, data: Uint8Array }; the media type
+    // rides the reference (verified against dsh attachment/src/types.ts).
     const stored = await ctx.attachments.readImage(block.attachment, signal)
+    if (!stored?.data) {
+      // Named failure instead of Buffer.from(undefined)'s bare TypeError the
+      // next time a developer-preview release moves the field (issue #17).
+      throw new Error(
+        "attachments.readImage returned no 'data' bytes; the dsh attachment shape may have changed",
+      )
+    }
     dir = await mkdtemp(join(tmpdir(), 'modlens-dsh-'))
-    const file = join(
-      dir,
-      `paste${MEDIA_EXT[stored.mediaType ?? block.attachment?.mediaType] ?? '.png'}`,
-    )
-    await writeFile(file, Buffer.from(stored.bytes), { mode: 0o600 })
+    const mediaType = stored.ref?.mediaType ?? block.attachment?.mediaType
+    const file = join(dir, `paste${MEDIA_EXT[mediaType] ?? '.png'}`)
+    await writeFile(file, Buffer.from(stored.data), { mode: 0o600 })
     const cli = process.env.MODLENS_DSH_CLI || CLI_PATH
     const { stdout, stderr, code } = await run(
       process.execPath,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@liustack/modlens",
-  "version": "3.9.1",
+  "version": "3.10.0",
   "description": "Plug-in vision for text-only LLMs, powered by the free Antigravity CLI",
   "type": "module",
   "bin": {

package/skills/modlens/SKILL.md

@@ -20,11 +20,11 @@ powershell -ExecutionPolicy Bypass -File <skill-dir>\scripts\run.ps1 <args>
 
 It resolves a working runtime (PATH `modlens`, then `npx`, then `bunx`) and forwards your arguments unchanged. Exit 78 means no runtime: relay the `nextSteps` from its stderr JSON instead of retrying.
 
-If your harness forbids running scripts, reason through the same order by hand and run the first line that works (the pinned version is 3.9.1):
+If your harness forbids running scripts, reason through the same order by hand and run the first line that works (the pinned version is 3.10.0):
 
-1. A `modlens` on `PATH` whose major version is 3 and is at least 3.9.1: `modlens <args>`.
-2. Otherwise, if `npx` exists: `npx --yes --package @liustack/modlens@3.9.1 modlens <args>`.
-3. Otherwise, if `bunx` exists: `bunx --bun @liustack/modlens@3.9.1 <args>`.
+1. A `modlens` on `PATH` whose major version is 3 and is at least 3.10.0: `modlens <args>`.
+2. Otherwise, if `npx` exists: `npx --yes --package @liustack/modlens@3.10.0 modlens <args>`.
+3. Otherwise, if `bunx` exists: `bunx --bun @liustack/modlens@3.10.0 <args>`.
 4. Otherwise tell the user no JavaScript runtime was found and that installing Node 22.13+ (https://nodejs.org) or Bun (https://bun.sh) is the next step. Do not claim modlens itself failed.
 
 `references/runtime.md` documents the pin and the diagnostic fields.

package/skills/modlens/references/runtime.md

@@ -8,7 +8,7 @@ shell syntax.
 
 ## Pinned version
 
-- Pinned CLI version: 3.9.1
+- Pinned CLI version: 3.10.0
 - npm package: `@liustack/modlens`
 - CLI binary name: `modlens`
 

package/skills/modlens/scripts/run.ps1

@@ -24,7 +24,7 @@ $ErrorActionPreference = 'Stop'
 # package.json version, and the release script rewrites it on every bump.
 $Package = '@liustack/modlens'
 $Bin = 'modlens'
-$Pinned = '3.9.1'
+$Pinned = '3.10.0'
 # -------------------------------------------------------------------------------
 
 $NativeNote = 'no native artifact is published for this tool yet; phase A ships npm launch paths only'

package/skills/modlens/scripts/run.sh

@@ -22,7 +22,7 @@ set -eu
 # package.json version, and the release script rewrites it on every bump.
 PKG="@liustack/modlens"
 BIN="modlens"
-PINNED="3.9.1"
+PINNED="3.10.0"
 # -------------------------------------------------------------------------------
 
 NATIVE_NOTE="no native artifact is published for this tool yet; phase A ships npm launch paths only"