dsh-tool-vision 0.1.0 → 0.3.1

package/README.md

@@ -1,18 +1,40 @@
 # dsh-tool-vision
 
+[![中文文档](https://img.shields.io/badge/%E4%B8%AD%E6%96%87%E6%96%87%E6%A1%A3-blue)](README.zh.md)
+
+**GitHub**: [Scorp1o117/dsh-tool-vision](https://github.com/Scorp1o117/dsh-tool-vision) · **npm**: [dsh-tool-vision](https://www.npmjs.com/package/dsh-tool-vision)
+
 External vision model for [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness).
 
-DeepSeek's own models are text-only, and dsh-llm has no multimodal content
-block yet. This plugin bridges the gap with a model-facing tool that sends an
-image (local file, or http(s) URL) to **any OpenAI-compatible**
-`/chat/completions` endpoint that supports `image_url` content parts, and
-returns the vision model's textual answer into the agent loop.
+DeepSeek's own models are text-only, and the harness derives every model
+request strictly from the session log (`llm/stream` requests must equal the
+durable derivation — the agent-loop invariant). This plugin bridges the gap
+in two ways:
+
+1. **`inspect_image` tool** — sends an image (local file, or http(s) URL) to
+   **any OpenAI-compatible** `/chat/completions` endpoint that supports
+   `image_url` content parts, and returns the vision model's textual answer
+   into the agent loop.
+2. **Image bridge (v0.2.1)** — pasted images are turned into `inspect_image`
+   hints *before they enter the durable log*, on the `agent/pre-step`
+   waterfall (the one seam where the harness lets a plugin replace the
+   messages of a proposed step). Images already logged by an older version
+   are repaired lazily with a surface `replace` on the session's first
+   pre-step. Only models listed in `multimodalModels` receive image blocks
+   directly; a model's declared `inputModalities` are never consulted,
+   because profiles routinely declare `input: [text, image]` on text-only
+   models just to pass the harness's prompt-admission check.
 
 - Zero dependencies beyond the dsh SDK — works with any compatible endpoint:
   OpenAI GPT-4o, Qwen-VL (DashScope), GLM-4V (Zhipu), Moonshot, Gemini
   compatible endpoints, local Ollama, etc.
 - Registered on the **global tools layer**: every agent in the process can
   call `inspect_image`.
+- **Web UI settings section (v0.3.0)**: Settings → 视觉模型 edits the
+  `tool-vision` namespace (API endpoint, write-only key, model, bridge
+  options) in `settings.yaml`; changes hot-apply without a restart. The API
+  key lives in `settings.yaml`, not the profile patch. Mount by package name
+  (`name: 'dsh-tool-vision'`) so the web client bundle is discovered.
 
 ## Install
 
@@ -47,6 +69,42 @@ Or load it from a local path without npm:
 | `timeoutMs` | `60000` | Per-request timeout. |
 | `maxImageBytes` | `10MB` | Largest accepted local image. |
 | `description` | default | Tool description shown to the model. |
+| `bridgeTextOnly` | `true` | Bridge pasted images to text hints on models that cannot see images. |
+| `bridgeExportDir` | temp | Export dir for bridged images (`os.tmpdir()/dsh-vision-bridge`). |
+| `multimodalModels` | `[]` | Model ids that receive image blocks directly (e.g. `mimo-v2.5`). |
+
+## Image bridge setup
+
+1. In your model settings, declare image input on the models you paste
+   images onto, so the harness admits image messages (pi-ai style):
+   ```yaml
+   llm-pi-ai:
+     providers:
+       your-provider:
+         models:
+           - id: deepseek-v4-flash
+             input: [text, image]
+   ```
+2. List genuinely multimodal models in the plugin config so they receive
+   image blocks untouched:
+   ```yaml
+   - id: tool-vision
+     name: 'dsh-tool-vision'
+     config:
+       multimodalModels: ['mimo-v2.5', 'grok-4.5']
+   ```
+
+Then pasting an image while on a text-only model stores a hint like
+`[User sent an image, exported to: <path>. Inspect it with the inspect_image tool...]`
+in the transcript (the pasted image no longer renders as pixels in that
+message), and the agent inspects it through the configured vision endpoint.
+
+> Why not `llm/stream`? The harness freezes every request and the agent-loop
+> invariant fails any request whose messages diverge from the session-log
+> derivation (`log-reconstruction desync`), and this cordis waterfall's
+> `next()` cannot replace request arguments. The `agent/pre-step` waterfall is
+> the supported seam: its decision messages *become* the durable log, so the
+> invariant stays satisfied.
 
 Key resolution order: `config.apiKey` → `process.env[apiKeyEnv]` →
 `process.env.OPENAI_API_KEY`.
@@ -69,8 +127,9 @@ Example endpoints (`baseURL`):
 
 ## Limitations
 
-- The image enters the conversation as a text description (a transcript, not
-  pixels) — pixel-precise tasks may be inaccurate.
+- A bridged image enters the conversation as a text hint (a transcript, not
+  pixels) — pixel-precise in-context reasoning is not available to text-only
+  models; the vision model's description comes back through `inspect_image`.
 - Images are base64-transferred; mind privacy and size limits.
 - Independent of the dsh-llm routing/retry system; failures return clear
   errors to the agent.

package/README.zh.md

@@ -1,12 +1,38 @@
 # dsh-tool-vision
 
-给 DeepSeek Harness 外接**视觉模型**的工具插件。DeepSeek 自家模型是纯文本的，而 dsh-llm 目前还没有多模态内容块类型——本插件用一条 OpenAI 兼容接口补上这个缺口：
+**GitHub**: [Scorp1o117/dsh-tool-vision](https://github.com/Scorp1o117/dsh-tool-vision) · **npm**: [dsh-tool-vision](https://www.npmjs.com/package/dsh-tool-vision) · [English](README.md)
 
-Agent 调用工具 `inspect_image`，把本地图片（或 http(s) 图片 URL）发给任意兼容 `image_url` 内容块的 `/chat/completions` 端点，把视觉模型的文字回答带回对话。
+给 [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) 外接**视觉模型**的插件。
 
-- 注册在**全局工具层**：进程内所有 Agent 都能调用 `inspect_image`
-- 支持任意 OpenAI 兼容视觉端点：OpenAI GPT-4o / Qwen-VL (DashScope) / GLM-4V (智谱) / Moonshot / Gemini 兼容端点 / Ollama 本地等
-- 本地图片自动转 base64 data URL，检查格式与大小上限
+DeepSeek 自家模型是纯文本的，而且 harness 的每次模型请求都**严格从会话日志推导**（`llm/stream` 请求必须与持久化推导一致，否则 agent-loop invariant 会报 `log-reconstruction desync`）。本插件用两条路径补上缺口：
+
+1. **`inspect_image` 工具** —— 把图片（本地文件或 http(s) URL）发给任意支持 `image_url` 内容块的 OpenAI 兼容 `/chat/completions` 端点，把视觉模型的文字回答带回对话。
+2. **图片桥（v0.2.1）** —— 粘贴的图片在**进入持久化日志之前**就被转换成 `inspect_image` 指引文本，拦截点是 `agent/pre-step` waterfall（这是 harness 唯一允许插件替换"进入某一步的消息"的缝；替换后的消息会**成为**持久化的 `user/message` 日志，所以请求重建 invariant 天然满足）。旧版本已经写进日志的图片消息，会在该会话下一次 pre-step 时用 surface `replace` 惰性修复。只有 `multimodalModels` 白名单内的模型直收图片块；**不参考模型的 `inputModalities` 声明**——因为很多配置为了通过 prompt 准入检查，会给纯文本模型声明 `input: [text, image]`（那只是声明，不代表上游真的能吃 `image_url`）。
+
+- 除 dsh SDK 外零依赖 —— 兼容任意端点：OpenAI GPT-4o、Qwen-VL（DashScope）、GLM-4V（智谱）、Moonshot、Gemini 兼容端点、本地 Ollama 等。
+- 注册在**全局工具层**：进程内所有 Agent 都能调用 `inspect_image`。
+- **Web UI 设置栏（v0.3.0）**：设置 → 视觉模型 编辑 `tool-vision` 命名空间（API 地址、只写密钥、模型、桥接选项），写入 `settings.yaml`，**改动即时生效无需重启**。API 密钥存放在 `settings.yaml` 而非 profile patch；插件按包名挂载（`name: 'dsh-tool-vision'`）以便 web 端发现客户端 bundle。
+
+## 安装
+
+在 profile patch（`$DSH_HOME/profiles/<name>/cordis.patch.yml`）里挂载：
+
+```yaml
+- insert:
+    - id: tool-vision
+      name: 'dsh-tool-vision'     # 前置：在 profile 里 pnpm add dsh-tool-vision
+      config:
+        baseURL: 'https://api.openai.com/v1'
+        apiKeyEnv: 'VISION_API_KEY'
+        model: 'gpt-4o-mini'
+```
+
+不装 npm 包、直接加载本地路径：
+
+```yaml
+    - id: tool-vision
+      name: './plugins/dsh-tool-vision/index.js'
+```
 
 ## 配置
 
@@ -20,41 +46,59 @@ Agent 调用工具 `inspect_image`，把本地图片（或 http(s) 图片 URL）
 | `timeoutMs` | `60000` | 单次请求超时 |
 | `maxImageBytes` | `10MB` | 本地图片大小上限 |
 | `description` | 默认描述 | 工具描述（模型可见） |
+| `bridgeTextOnly` | `true` | 把粘贴图片转成文本指引（发给看不懂图片的模型时） |
+| `bridgeExportDir` | 临时目录 | 桥接图片导出目录（`os.tmpdir()/dsh-vision-bridge`） |
+| `multimodalModels` | `[]` | 直发图片块的模型 id（如 `mimo-v2.5`） |
 
-密钥解析顺序：`config.apiKey` → `process.env[apiKeyEnv]` → `process.env.OPENAI_API_KEY`。
+## 图片桥配置
 
-## 挂载
+1. 在模型设置里给要贴图的模型声明图片输入（pi-ai 风格），让 harness 放行图片消息：
+   ```yaml
+   llm-pi-ai:
+     providers:
+       your-provider:
+         models:
+           - id: deepseek-v4-flash
+             input: [text, image]
+   ```
+2. 在插件配置里列出真正多模态的模型，让它们直收图片块：
+   ```yaml
+   - id: tool-vision
+     name: 'dsh-tool-vision'
+     config:
+       multimodalModels: ['mimo-v2.5', 'grok-4.5']
+   ```
 
-在 profile 的 `cordis.patch.yml` 里 insert：
+之后在文本模型下贴图，转录里会留下一条指引：
+`[User sent an image, exported to: <path>. Inspect it with the inspect_image tool...]`
+（该消息不再以像素图形式渲染），Agent 会调用视觉端点查看并把结果带回对话。
 
-```yaml
-- insert:
-    - id: tool-vision
-      name: './plugins/dsh-tool-vision/index.js'
-      config:
-        baseURL: 'https://dashscope.aliyuncs.com/compatible-mode/v1'
-        apiKeyEnv: 'DASHSCOPE_API_KEY'
-        model: 'qwen-vl-plus'
-```
+> 为什么不用 `llm/stream`？harness 会冻结每个请求，且 agent-loop invariant 会拒绝任何与会话日志推导不一致的请求；这个 cordis 版本的 waterfall `next()` 也无法替换请求参数。`agent/pre-step` 才是受支持的缝：它的决策消息**会成为**持久化日志，invariant 天然成立。
+
+密钥解析顺序：`config.apiKey` → `process.env[apiKeyEnv]` → `process.env.OPENAI_API_KEY`。
 
 ## 工具：`inspect_image`
 
-| 参数 | 必填 | 说明 |
+| 参数 | 必填 | 含义 |
 |---|---|---|
-| `path` | ✅ | 图片路径（绝对路径，或相对当前 workspace）或 http(s) URL |
-| `question` | – | 针对图片的具体问题；缺省为"详细描述" |
-| `detail` | – | `auto` / `low` / `high`，图片分辨率提示 |
+| `path` | ✅ | 图片路径（绝对路径，或相对当前工作区）或 http(s) URL |
+| `question` | – | 可选的具体问题 |
+| `detail` | – | `auto` / `low` / `high` 分辨率提示 |
+
+示例端点（`baseURL`）：
+
+- **OpenAI**：`https://api.openai.com/v1` —— `gpt-4o`、`gpt-4o-mini`
+- **阿里云 DashScope（Qwen-VL）**：`https://dashscope.aliyuncs.com/compatible-mode/v1` —— `qwen-vl-plus`、`qwen-vl-max`
+- **智谱（GLM-4V）**：`https://open.bigmodel.cn/api/paas/v4` —— `glm-4v-flash`（免费档）、`glm-4v-plus`
+- **Moonshot（Kimi）**：`https://api.moonshot.cn/v1` —— `moonshot-v1-8k-vision-preview`
+- **Ollama 本地**：`http://localhost:11434/v1` —— `llama3.2-vision`（无需密钥）
 
-示例端点（`baseURL` 写法）：
+## 限制
 
-- **OpenAI**：`https://api.openai.com/v1`，模型 `gpt-4o` / `gpt-4o-mini`
-- **阿里云百炼 (Qwen-VL)**：`https://dashscope.aliyuncs.com/compatible-mode/v1`，模型 `qwen-vl-plus` / `qwen-vl-max`
-- **智谱 (GLM-4V)**：`https://open.bigmodel.cn/api/paas/v4`，模型 `glm-4v-flash`（免费档）/ `glm-4v-plus`
-- **Moonshot (Kimi)**：`https://api.moonshot.cn/v1`，模型 `moonshot-v1-8k-vision-preview`
-- **Ollama 本地**：`http://localhost:11434/v1`，模型 `llama3.2-vision`（无需 key）
+- 被桥接的图片以文本指引进入对话（转录而非像素）——文本模型无法做像素级上下文推理；视觉模型的描述通过 `inspect_image` 回传。
+- 图片以 base64 传输；注意隐私与大小限制。
+- 独立于 dsh-llm 的路由/重试体系；失败会向 Agent 返回明确错误。
 
-## 局限
+## License
 
-- 图片内容以文字描述进入对话，Agent 得到的是"转述"而非像素——精确到像素的任务（如测量）可能不够准
-- 图片会在请求中 base64 传输，注意隐私与大小
-- 不依赖 dsh-llm 服务，因此不走模型路由/重试体系；失败会以明确的错误信息返回给 Agent
+MIT

package/client.js

@@ -0,0 +1,274 @@
+/**
+ * dsh-tool-vision — browser half.
+ *
+ * A "视觉模型" section inside the Web UI settings page: edits the
+ * `tool-vision` settings namespace (API endpoint, key, model, bridge
+ * options) through the settings scope transport. Changes hot-apply via the
+ * host settings provider — no restart needed.
+ *
+ * Hand-written ModuleLoader bundle — no build step required.
+ */
+window.__ModuleLoader__.load({
+  id: "dsh-tool-vision",
+  factory: (require) => {
+    var module = { exports: {} };
+    var exports = module.exports;
+    Object.defineProperty(exports, Symbol.toStringTag, { value: "Module" });
+    var react = require("react");
+    var h = react.createElement;
+
+    // ── CSS (theme tokens) ────────────────────────────────────────────────
+    var CSS = ".__tv_root{max-width:640px;display:flex;flex-direction:column;gap:10px}" +
+      ".__tv_field{display:flex;flex-direction:column;gap:4px}" +
+      ".__tv_label{font-size:12px;font-weight:600;color:var(--dsw-alias-label-primary);display:flex;align-items:center;gap:6px}" +
+      ".__tv_override{font-size:10px;color:var(--dsw-alias-state-business-primary);border:1px solid var(--dsw-alias-border-l2);border-radius:4px;padding:0 4px}" +
+      ".__tv_hint{font-size:11px;color:var(--dsw-alias-label-tertiary)}" +
+      ".__tv_input{border:1px solid var(--dsw-alias-border-l2);background:var(--dsw-alias-bg-layer-3);font:inherit;color:var(--dsw-alias-label-primary);border-radius:8px;padding:6px 10px;font-size:13px;box-sizing:border-box;width:100%}" +
+      ".__tv_row{display:flex;align-items:center;gap:8px}" +
+      ".__tv_check{accent-color:var(--dsw-alias-state-business-primary)}" +
+      ".__tv_actions{display:flex;gap:8px;align-items:center;margin-top:4px}" +
+      ".__tv_btn{border:1px solid var(--dsw-alias-border-l2);background:var(--dsw-alias-bg-layer-3);color:var(--dsw-alias-label-primary);border-radius:8px;padding:6px 14px;font:inherit;font-size:13px;cursor:pointer}" +
+      ".__tv_btn:hover:not(:disabled){border-color:var(--dsw-alias-state-business-primary)}" +
+      ".__tv_btn:disabled{opacity:.5;cursor:default}" +
+      ".__tv_btnPrimary{border-color:var(--dsw-alias-state-business-primary);background:var(--dsw-alias-state-business-primary);color:var(--dsw-alias-label-on-accent)}" +
+      ".__tv_status{font-size:12px;color:var(--dsw-alias-label-tertiary)}" +
+      ".__tv_error{font-size:12px;color:var(--dsw-alias-state-error-primary)}" +
+      ".__tv_unavailable{font-size:13px;color:var(--dsw-alias-label-tertiary)}";
+    var tagId = "dsh-tool-vision/main.css";
+    if (typeof document !== "undefined" && document.querySelector("style[data-plugin-css=" + JSON.stringify(tagId) + "]") === null) {
+      var tag = document.createElement("style");
+      tag.dataset.plugin = "dsh-tool-vision";
+      tag.dataset.pluginCss = tagId;
+      tag.textContent = CSS;
+      document.head.appendChild(tag);
+    }
+
+    // ── locale ────────────────────────────────────────────────────────────
+    var NS = "toolVision";
+    var inject = ["slots", "locale", "settingsScope"];
+    var zh = {
+      nav: "视觉模型",
+      intro: "外置视觉模型配置：Agent 通过 inspect_image 工具把图片发给该端点分析。修改后即时生效（settings.yaml 热重载）。",
+      apiKeyHint: "留空保持当前密钥。密钥只写不读，不会回显。",
+      maxTokens: "最大输出 Tokens",
+      timeoutMs: "请求超时（毫秒）",
+      maxImageBytes: "本地图片大小上限（字节）",
+      bridgeTextOnly: "图片桥接（文本模型贴图自动转 inspect_image 指引）",
+      bridgeExportDir: "桥接图片导出目录（空 = 系统临时目录）",
+      multimodalModels: "多模态白名单（逗号分隔，这些模型直收图片块）",
+      save: "保存",
+      reset: "恢复默认",
+      saved: "已保存",
+      saving: "保存中…",
+      error: "保存失败",
+      unavailable: "设置命名空间不可用（服务端未注册 tool-vision 命名空间？）",
+      overridden: "已覆盖",
+      loading: "加载中…"
+    };
+    var en = {
+      nav: "Vision Model",
+      intro: "External vision model config: the agent sends images to this endpoint via the inspect_image tool. Changes apply immediately (settings.yaml hot-reload).",
+      apiKeyHint: "Leave blank to keep the current key. The key is write-only and never echoed.",
+      maxTokens: "Max output tokens",
+      timeoutMs: "Request timeout (ms)",
+      maxImageBytes: "Max local image size (bytes)",
+      bridgeTextOnly: "Image bridge (pasted images on text-only models become inspect_image hints)",
+      bridgeExportDir: "Bridge export dir (empty = system temp)",
+      multimodalModels: "Multimodal whitelist (comma-separated; these models receive image blocks directly)",
+      save: "Save",
+      reset: "Reset",
+      saved: "Saved",
+      saving: "Saving…",
+      error: "Save failed",
+      unavailable: "Settings namespace unavailable (tool-vision namespace not registered server-side?)",
+      overridden: "overridden",
+      loading: "Loading…"
+    };
+
+    // ── field spec ────────────────────────────────────────────────────────
+    var FIELDS = [
+      { key: "baseURL", label: "API Base URL", type: "text", placeholder: "https://api.openai.com/v1" },
+      { key: "apiKey", label: "API Key", type: "password", secret: true },
+      { key: "apiKeyEnv", label: "API Key 环境变量（apiKey 为空时读取）", type: "text" },
+      { key: "model", label: "视觉模型", type: "text", placeholder: "gpt-4o-mini" },
+      { key: "maxTokens", label: "最大输出 Tokens", type: "number" },
+      { key: "timeoutMs", label: "请求超时（毫秒）", type: "number" },
+      { key: "maxImageBytes", label: "图片大小上限（字节）", type: "number" },
+      { key: "bridgeTextOnly", label: "图片桥接开关", type: "checkbox" },
+      { key: "bridgeExportDir", label: "桥接导出目录", type: "text" },
+      { key: "multimodalModels", label: "多模态白名单（逗号分隔）", type: "csv" }
+    ];
+    var ZH_HINTS = {
+      apiKey: "apiKeyHint",
+      maxTokens: "maxTokens",
+      timeoutMs: "timeoutMs",
+      maxImageBytes: "maxImageBytes",
+      bridgeTextOnly: "bridgeTextOnly",
+      bridgeExportDir: "bridgeExportDir",
+      multimodalModels: "multimodalModels"
+    };
+
+    function labelOf(f) {
+      return f.label;
+    }
+
+    // ── component ─────────────────────────────────────────────────────────
+    function VisionSection(props) {
+      var t = props.t;
+      var scope = props.scope;
+      var snapshot = react.useSyncExternalStore(
+        function (cb) { return scope.subscribe(cb); },
+        function () { return scope.getSnapshot(); }
+      );
+      var ready = snapshot.status === "ready" && snapshot.value !== void 0;
+      var [draft, setDraft] = react.useState({});
+      var [busy, setBusy] = react.useState(false);
+      var [notice, setNotice] = react.useState(null);
+      var [error, setError] = react.useState(null);
+
+      react.useEffect(function () {
+        scope.load();
+        return function () { scope.dispose(); };
+      }, [scope]);
+      // Re-seed the draft when the snapshot value changes (after a save or a host change).
+      react.useEffect(function () {
+        if (ready) setDraft(function (prev) { return Object.assign({}, prev, valueToDraft(snapshot.value)); });
+        // eslint-disable-next-line react-hooks/exhaustive-deps
+      }, [ready, snapshot.value]);
+
+      if (snapshot.status === "unavailable") {
+        return h("p", { className: "__tv_unavailable" }, t("unavailable"));
+      }
+      if (!ready) return h("p", { className: "__tv_status" }, t("loading"));
+
+      var value = snapshot.value;
+      var user = snapshot.user || {};
+
+      function fieldDraft(f) {
+        if (f.type === "csv") return draft[f.key] !== void 0 ? draft[f.key] : draftToCsv(value[f.key]);
+        if (f.type === "checkbox") return draft[f.key] !== void 0 ? draft[f.key] : Boolean(value[f.key]);
+        return draft[f.key] !== void 0 ? draft[f.key] : String(value[f.key] ?? "");
+      }
+      function setField(f, v) {
+        setDraft(function (prev) {
+          var next = Object.assign({}, prev);
+          next[f.key] = v;
+          return next;
+        });
+        setNotice(null);
+        setError(null);
+      }
+
+      function onSave() {
+        setBusy(true); setNotice(null); setError(null);
+        var writes = FIELDS.map(function (f) {
+          var d = fieldDraft(f);
+          if (f.type === "csv") {
+            var arr = String(d).split(",").map(function (s) { return s.trim(); }).filter(Boolean);
+            var cur = value[f.key] || [];
+            if (arr.length === cur.length && arr.every(function (x, i) { return x === cur[i]; })) return Promise.resolve();
+            return scope.set(f.key, arr);
+          }
+          if (f.type === "checkbox") {
+            if (Boolean(d) === Boolean(value[f.key])) return Promise.resolve();
+            return Boolean(d) ? scope.set(f.key, true) : scope.unset(f.key);
+          }
+          if (f.type === "password") {
+            if (!d) return Promise.resolve(); // blank keeps the current key
+            if (d === String(value[f.key] ?? "")) return Promise.resolve();
+            return scope.set(f.key, d);
+          }
+          if (String(d) === String(value[f.key] ?? "")) return Promise.resolve();
+          if (String(d).trim() === "" && !(f.key in user)) return Promise.resolve();
+          return String(d).trim() === "" ? scope.unset(f.key) : scope.set(f.key, f.type === "number" ? Number(d) : d);
+        });
+        Promise.all(writes).then(function () {
+          setBusy(false); setNotice(t("saved"));
+        }).catch(function (e) {
+          setBusy(false); setError(t("error") + ": " + String(e && e.message || e));
+        });
+      }
+
+      function onReset() {
+        setBusy(true); setNotice(null); setError(null);
+        Promise.all(FIELDS.map(function (f) { return scope.unset(f.key); })).then(function () {
+          setBusy(false); setNotice(t("saved"));
+        }).catch(function (e) {
+          setBusy(false); setError(t("error") + ": " + String(e && e.message || e));
+        });
+      }
+
+      return h("div", { className: "__tv_root" },
+        h("p", { className: "__tv_hint", style: { margin: "0 0 4px" } }, t("intro")),
+        FIELDS.map(function (f) {
+          var overridden = f.key in user;
+          if (f.type === "checkbox") {
+            return h("label", { key: f.key, className: "__tv_field" },
+              h("span", { className: "__tv_row" },
+                h("input", { className: "__tv_check", type: "checkbox", checked: Boolean(fieldDraft(f)), onChange: function (e) { setField(f, e.target.checked); } }),
+                h("span", { className: "__tv_label" }, labelOf(f)),
+                overridden ? h("span", { className: "__tv_override" }, t("overridden")) : null
+              ),
+              f.key in ZH_HINTS ? h("span", { className: "__tv_hint" }, t(ZH_HINTS[f.key])) : null
+            );
+          }
+          return h("label", { key: f.key, className: "__tv_field" },
+            h("span", { className: "__tv_label" },
+              labelOf(f),
+              overridden ? h("span", { className: "__tv_override" }, t("overridden")) : null
+            ),
+            h("input", {
+              className: "__tv_input",
+              type: f.type === "password" ? "password" : f.type === "number" ? "number" : "text",
+              value: fieldDraft(f),
+              placeholder: f.type === "password" ? (overridden ? "••••••••" : t("apiKeyHint")) : (f.placeholder || ""),
+              onChange: function (e) { setField(f, e.target.value); }
+            }),
+            f.key in ZH_HINTS ? h("span", { className: "__tv_hint" }, t(ZH_HINTS[f.key])) : null
+          );
+        }),
+        h("div", { className: "__tv_actions" },
+          h("button", { type: "button", className: "__tv_btn __tv_btnPrimary", onClick: onSave, disabled: busy || !snapshot.writable }, t("save")),
+          h("button", { type: "button", className: "__tv_btn", onClick: onReset, disabled: busy || !snapshot.writable }, t("reset")),
+          notice ? h("span", { className: "__tv_status" }, notice) : null,
+          busy ? h("span", { className: "__tv_status" }, t("saving")) : null,
+          error ? h("span", { className: "__tv_error" }, error) : null
+        )
+      );
+    }
+
+    function valueToDraft(value) {
+      var out = {};
+      for (var i = 0; i < FIELDS.length; i += 1) {
+        var f = FIELDS[i];
+        out[f.key] = f.type === "csv" ? draftToCsv(value[f.key]) : f.type === "checkbox" ? Boolean(value[f.key]) : String(value[f.key] ?? "");
+      }
+      return out;
+    }
+    function draftToCsv(arr) {
+      return Array.isArray(arr) ? arr.join(", ") : String(arr ?? "");
+    }
+
+    // ── plugin ────────────────────────────────────────────────────────────
+    function apply(ctx) {
+      var t = ctx.locale.bind(NS);
+      ctx.effect(function () { return ctx.locale.register(NS, { zh: zh, en: en }); }, "dsh-tool-vision: dictionaries");
+      var scope = ctx.settingsScope.bind({ namespace: "tool-vision" });
+      ctx.slots.inject("settings.section", function () {
+        return ctx.slots.register({
+          name: "settings.section",
+          id: "tool-vision",
+          order: 25,
+          label: function () { return t("nav"); },
+          locale: NS
+        }, function (props) {
+          return h(VisionSection, Object.assign({}, props, { scope: scope }));
+        });
+      });
+    }
+
+    exports.apply = apply;
+    exports.inject = inject;
+    return module.exports;
+  }
+});

package/index.js

@@ -1,24 +1,41 @@
 /**
  * dsh-tool-vision — external vision model for DeepSeek Harness.
  *
- * DeepSeek's own models are text-only, and dsh-llm has no multimodal content
- * block type yet. This plugin bridges the gap with a model-facing tool that
- * sends an image (local file, or http(s) URL) to any OpenAI-compatible
- * chat/completions endpoint that supports `image_url` content parts, and
- * returns the vision model's textual answer into the agent loop.
+ * Two capabilities:
  *
- * Registered on the GLOBAL tools layer, so every agent in the process can
- * call `inspect_image`.
+ * 1. `inspect_image` tool — sends an image (local file, or http(s) URL) to
+ *    any OpenAI-compatible chat/completions endpoint that supports
+ *    `image_url` content parts, and returns the vision model's text answer.
+ *
+ * 2. Image bridge — pasted images are bridged to text hints before they
+ *    enter a text-only model's request:
+ *
+ *    - New images are bridged on the `agent/pre-step` waterfall (the only
+ *      seam where the harness lets a plugin replace the messages that enter
+ *      a step — they become the durable `user/message` log, so the
+ *      `llm/stream` request-reconstruction invariant stays satisfied).
+ *    - Images already logged before the plugin was installed (or before a
+ *      server restart) are repaired lazily with a surface `replace`, one
+ *      event at a time, on the first pre-step of the session.
+ *
+ *    The bridged hint points at an exported local copy of the image, which
+ *    the agent hands to `inspect_image`. Models listed in
+ *    `multimodalModels` (or whose resolved `inputModalities` include
+ *    "image") receive image blocks directly and are never bridged.
  */
-import { readFile, stat } from "node:fs/promises";
-import { extname, isAbsolute, resolve as resolvePath } from "node:path";
+import { mkdir, readFile, stat, writeFile } from "node:fs/promises";
+import { extname, isAbsolute, join, resolve as resolvePath } from "node:path";
+import os from "node:os";
 import z from "@deepseek-ai/schemastery";
 import { defineTool } from "@deepseek-ai/dsh-tools";
+import { installSettingsSection, settingsNamespace } from "@deepseek-ai/dsh-settings";
 
 /** Cordis plugin name. */
 const name = "tool-vision";
-/** The tool registry this row contributes to. */
-const inject = ["tools"];
+/** The tool registry, the llm seam (model capability lookup), and the attachment store. */
+const inject = ["tools", "llm", "attachments"];
+/** Settings namespace owned by this plugin (Web UI settings section). */
+const NS = settingsNamespace("tool-vision");
 
 const DEFAULT_DESCRIPTION =
   "Analyze an image using an external vision-capable model through an OpenAI-compatible API. " +
@@ -31,20 +48,26 @@ const DEFAULT_DESCRIPTION =
 const Config = z.object({
   /** Base URL of an OpenAI-compatible API, e.g. https://api.openai.com/v1 or https://dashscope.aliyuncs.com/compatible-mode/v1 */
   baseURL: z.string().default("https://api.openai.com/v1"),
-  /** API key; takes precedence over apiKeyEnv. */
-  apiKey: z.string().default(""),
+  /** API key; takes precedence over apiKeyEnv. Rendered as a write-only secret in the Web UI. */
+  apiKey: z.string().default("").role("secret"),
   /** Environment variable holding the API key. */
   apiKeyEnv: z.string().default("VISION_API_KEY"),
   /** Vision model id served by the endpoint. */
   model: z.string().default("gpt-4o-mini"),
   /** Max output tokens for the vision call. */
-  maxTokens: z.number().default(1024),
+  maxTokens: z.number().default(4096),
   /** Per-request timeout in milliseconds. */
   timeoutMs: z.number().default(60000),
   /** Largest local image accepted, in bytes. */
   maxImageBytes: z.number().default(10 * 1024 * 1024),
   /** Tool description shown to the model; overrides the default. */
   description: z.string().default(DEFAULT_DESCRIPTION),
+  /** Bridge pasted images to text hints on models that cannot see images. */
+  bridgeTextOnly: z.boolean().default(true),
+  /** Export directory for bridged images; empty = system temp. */
+  bridgeExportDir: z.string().default(""),
+  /** Model ids that receive image blocks directly (never bridged). */
+  multimodalModels: z.array(z.string()).default([]),
 });
 
 const MIME_BY_EXT = {
@@ -59,6 +82,179 @@ const MIME_BY_EXT = {
   ".ico": "image/x-icon",
 };
 
+const EXT_BY_MEDIA = {
+  "image/png": ".png",
+  "image/jpeg": ".jpg",
+  "image/webp": ".webp",
+  "image/gif": ".gif",
+};
+
+/** True when any message carries an image content block. */
+function hasImageBlock(messages) {
+  return (messages ?? []).some((m) =>
+    Array.isArray(m?.content) && m.content.some((b) => b?.type === "image"),
+  );
+}
+
+/** Deep-freeze an acyclic JSON-safe value in place (the harness freezes every durable message). */
+function deepFreeze(value) {
+  if (value === null || typeof value !== "object" || Object.isFrozen(value)) return value;
+  if (Array.isArray(value)) {
+    for (const item of value) deepFreeze(item);
+    return Object.freeze(value);
+  }
+  for (const key of Object.keys(value)) deepFreeze(value[key]);
+  return Object.freeze(value);
+}
+
+/** Export one attachment to disk; returns the file path (cached per process). */
+const exportedPaths = new Map();
+async function exportImage(attachment, ctx, dir) {
+  const cached = exportedPaths.get(attachment.attachmentId);
+  if (cached) return cached;
+  const { data } = await ctx.attachments.readImage(attachment);
+  const ext = EXT_BY_MEDIA[attachment.mediaType] ?? ".img";
+  const safeName = attachment.name
+    ? attachment.name
+        .replace(/\.[^.]+$/, "")
+        .replace(/[^\w\-]+/g, "_")
+        .replace(/^_+|_+$/g, "")
+        .slice(0, 40)
+    : "";
+  const base = (safeName ? `${safeName}_` : "") + attachment.attachmentId.slice(0, 12);
+  const path = join(dir, `${base}${ext}`);
+  await writeFile(path, data);
+  exportedPaths.set(attachment.attachmentId, path);
+  return path;
+}
+
+/**
+ * Replace image content blocks with text hints pointing at exported files.
+ * Non-image messages are returned as-is (same reference); bridged messages
+ * are fresh, deep-frozen objects with the original identity and source.
+ * Exported for unit testing; `ctx` only needs `attachments`.
+ */
+async function bridgeMessages(messages, ctx, dir) {
+  const next = [];
+  for (const message of messages) {
+    const content = message?.content;
+    if (!Array.isArray(content) || !content.some((b) => b?.type === "image")) {
+      next.push(message);
+      continue;
+    }
+    const blocks = [];
+    for (const block of content) {
+      if (block?.type !== "image") {
+        blocks.push(block);
+        continue;
+      }
+      const path = await exportImage(block.attachment, ctx, dir);
+      const name = block.attachment.name ? ` (${block.attachment.name})` : "";
+      blocks.push({
+        type: "text",
+        text:
+          `[User sent an image${name}, exported to: ${path}. ` +
+          `Inspect it with the inspect_image tool to see its content.]`,
+      });
+    }
+    next.push(deepFreeze({ ...message, content: blocks }));
+  }
+  return next;
+}
+
+/**
+ * Whether the session's current model may receive image blocks directly.
+ * Uses the last logged request header first, then the agent's own options,
+ * and consults ONLY the `multimodalModels` whitelist — never the model's
+ * declared `inputModalities`, because profiles routinely declare
+ * `input: [text, image]` on text-only models to pass the harness's prompt
+ * admission check (that declaration says nothing about whether the upstream
+ * endpoint really accepts `image_url` parts).
+ * Returns true when bridging is disabled (nothing would be bridged anyway).
+ */
+async function currentModelAcceptsImage(agent, config) {
+  if (!config.bridgeTextOnly) return true;
+  const header = agent?.session?.requestHeader?.();
+  const model = header?.config?.model ?? agent?.options?.model;
+  if (!model) return false;
+  return config.multimodalModels.includes(model);
+}
+
+/**
+ * Lazily bridge image blocks that are already part of the session log
+ * (pasted before the plugin was active, or before a restart). Each affected
+ * event is rewritten once with a surface `replace`, which swaps the durable
+ * derivation (and the transcript) to the text hint. Events that are no
+ * longer on the surface (already shadowed) are skipped and remembered.
+ * `repaired` tracks per-session state: a `Set` of handled seqs plus a
+ * monotonic scan cursor.
+ */
+async function repairLoggedImages(ctx, session, exportDir, repaired) {
+  const events = session.events;
+  for (let index = repaired.cursor; index < events.length; index += 1) {
+    const event = events[index];
+    if (event.type !== "user/message" || repaired.set.has(event.seq)) {
+      repaired.set.add(event.seq);
+      continue;
+    }
+    const content = event.data?.content;
+    if (!Array.isArray(content) || !content.some((b) => b?.type === "image")) {
+      repaired.set.add(event.seq);
+      continue;
+    }
+    const [bridged] = await bridgeMessages([event.data], ctx, exportDir);
+    try {
+      session.append("user/message", bridged, {
+        surfaceOp: { op: "replace", start: event.seq, end: event.seq },
+        sourceEventSeqs: [event.seq],
+      });
+      ctx.logger.info(`[tool-vision] bridged logged image at seq ${event.seq} (${session.id})`);
+    } catch (error) {
+      ctx.logger.debug(`[tool-vision] skip repair of seq ${event.seq}: ${String(error)}`);
+    }
+    repaired.set.add(event.seq);
+  }
+  repaired.cursor = events.length;
+}
+
+/**
+ * Install the pre-step bridge at the root level. Agent-scoped waterfalls
+ * admit untagged (root) listeners, so one listener serves every agent —
+ * including sessions resumed after a server restart, which never re-fire
+ * `session/created` for per-agent attachments. Runs before every proposed
+ * step: new pasted images are bridged into the durable log, and stuck
+ * logged images are repaired, before the model request is derived from it.
+ */
+function attachPreStepBridge(ctx, getConfig, exportDir) {
+  const repairedBySession = new Map();
+  ctx.on("agent/pre-step", async (payload, next) => {
+    const decision = await next();
+    if (!decision || decision.kind !== "enter") return decision;
+    const agent = payload?.agent;
+    if (!agent?.session) return decision;
+    try {
+      const acceptsImage = await currentModelAcceptsImage(agent, getConfig());
+      if (!acceptsImage) {
+        let repaired = repairedBySession.get(agent.session.id);
+        if (!repaired) {
+          repaired = { set: new Set(), cursor: 0 };
+          repairedBySession.set(agent.session.id, repaired);
+        }
+        await repairLoggedImages(ctx, agent.session, exportDir, repaired).catch((error) => {
+          ctx.logger.warn(`[tool-vision] logged-image repair failed: ${String(error)}`);
+        });
+      }
+      if (acceptsImage) return decision;
+      const messages = await bridgeMessages(decision.messages, ctx, exportDir);
+      if (messages.every((message, index) => message === decision.messages[index])) return decision;
+      return { ...decision, messages };
+    } catch (error) {
+      ctx.logger.warn(`[tool-vision] pre-step bridge failed: ${String(error)}`);
+      return decision;
+    }
+  });
+}
+
 function resolveApiKey(config) {
   if (config.apiKey) return config.apiKey;
   if (config.apiKeyEnv) {
@@ -134,7 +330,13 @@ async function callVision(config, imageUrl, question, detail, signal) {
         `vision endpoint returned ${response.status}: ${detailText} (endpoint ${endpoint})`,
       );
     }
-    const answer = body?.choices?.[0]?.message?.content;
+    // Reasoning models (mimo-v2.5, deepseek-r1, ...) spend the token budget on
+    // `reasoning_content` first; when the final `content` is empty or was cut
+    // off by max_tokens, fall back to the reasoning text so the answer is
+    // still useful.
+    const message = body?.choices?.[0]?.message;
+    let answer = message?.content ?? "";
+    if (!answer.trim()) answer = message?.reasoning_content ?? "";
     if (typeof answer !== "string" || !answer.trim()) {
       throw new Error("vision endpoint returned an empty response");
     }
@@ -146,9 +348,32 @@ async function callVision(config, imageUrl, question, detail, signal) {
 }
 
 function apply(ctx, config) {
+  // ── settings-backed configuration ─────────────────────────────────────────
+  // The composition entry stays the `base` layer; a registered `tool-vision`
+  // settings section (Web UI section, settings.yaml) overlays it live, so
+  // edits hot-apply without a restart. `current` is read at use time.
+  let current = config;
+  const getConfig = () => current;
+  installSettingsSection(ctx, NS, Config, config, {
+    setSource: (source) => {
+      current = source;
+    },
+    onChange: () => {},
+  });
+
+  // ── image bridge: pasted images become inspect_image hints on text-only models ──
+  if (getConfig().bridgeTextOnly) {
+    const exportDir = getConfig().bridgeExportDir || join(os.tmpdir(), "dsh-vision-bridge");
+    mkdir(exportDir, { recursive: true }).catch(() => {});
+    // Root-level listener: agent-scoped waterfalls admit untagged listeners,
+    // so one registration serves every agent (new and resumed alike) and the
+    // agent is read from the fused payload.
+    attachPreStepBridge(ctx, getConfig, exportDir);
+  }
+
   ctx.tools.register(defineTool({
     name: "inspect_image",
-    description: config.description,
+    description: getConfig().description,
     parameters: {
       path: {
         type: "string",
@@ -170,12 +395,27 @@ function apply(ctx, config) {
       render: (_args, value) => [{ type: "text", text: value }],
     },
     async execute(args, exec) {
+      const cfg = getConfig();
       const cwd = exec.agent?.session?.header?.cwd ?? process.cwd();
-      const { url, note } = await toImageUrl(args.path, cwd, config);
-      const answer = await callVision(config, url, args.question, args.detail, exec.signal);
+      const { url, note } = await toImageUrl(args.path, cwd, cfg);
+      const answer = await callVision(cfg, url, args.question, args.detail, exec.signal);
       return note === url ? answer : `${answer}\n\n(image: ${note})`;
     },
   }));
 }
 
-export { Config, DEFAULT_DESCRIPTION, apply, inject, name };
+export {
+  Config,
+  DEFAULT_DESCRIPTION,
+  EXT_BY_MEDIA,
+  apply,
+  attachPreStepBridge,
+  bridgeMessages,
+  currentModelAcceptsImage,
+  deepFreeze,
+  exportImage,
+  hasImageBlock,
+  inject,
+  name,
+  repairLoggedImages,
+};

package/package.json

@@ -1,18 +1,27 @@
-{
+{
   "name": "dsh-tool-vision",
-  "version": "0.1.0",
-  "description": "External vision model for DeepSeek Harness: inspect_image sends local images or http(s) image URLs to any OpenAI-compatible chat/completions endpoint and returns the vision model's text answer.",
+  "version": "0.3.1",
+  "description": "DeepSeek Harness 外置视觉模型插件：inspect_image 把本地图片或 http(s) 图片 URL 发给任意 OpenAI 兼容端点，视觉模型看图的文字回答直接带回对话；附 Web UI 设置栏。",
   "type": "module",
   "main": "index.js",
   "exports": {
-    ".": "./index.js"
+    ".": "./index.js",
+    "./client": "./client.js",
+    "./package.json": "./package.json"
+  },
+  "dsh": {
+    "client": {
+      "platform": "web"
+    }
   },
   "files": [
     "index.js",
+    "client.js",
     "README.md",
     "README.zh.md",
     "LICENSE"
   ],
+  "homepage": "https://github.com/Scorp1o117/dsh-tool-vision",
   "keywords": [
     "dsh",
     "dsh-plugin",
@@ -37,8 +46,11 @@
     "url": "https://github.com/Scorp1o117/dsh-tool-vision/issues"
   },
   "dependencies": {
+    "@deepseek-ai/dsh-settings": "^0.1.0-rc.6",
     "@deepseek-ai/dsh-tools": "^0.1.0-rc.6",
     "@deepseek-ai/schemastery": "^3.18.1"
+  },
+  "peerDependencies": {
+    "react": "^18.2.0"
   }
 }
-