dsh-ocr-local 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE ADDED
@@ -0,0 +1,31 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 dsh-ocr-local contributors
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
22
+
23
+ ---
24
+
25
+ Third-party components:
26
+
27
+ - PP-OCRv5 models (PP-OCRv5_mobile_det.onnx / PP-OCRv5_mobile_rec.onnx)
28
+ and ppocrv5_dict.txt are from PaddleOCR (https://github.com/PaddlePaddle/PaddleOCR),
29
+ Apache License 2.0. Downloaded at install time by ocr/download_models.py,
30
+ not bundled in this repository.
31
+ - ONNX Runtime is MIT licensed (https://github.com/microsoft/onnxruntime).
package/README.en.md ADDED
@@ -0,0 +1,218 @@
1
+ # dsh-ocr-local
2
+
3
+ [English](README.en.md) · [中文](README.md)
4
+
5
+ [![license](https://img.shields.io/npm/l/dsh-ocr-local?style=flat-square)](LICENSE) [![GitHub](https://img.shields.io/badge/GitHub-grelvan%2Fdsh--ocr--local-2f81f7?style=flat-square)](https://github.com/grelvan/dsh-ocr-local)
6
+
7
+ A **local OCR fallback** for DeepSeek Harness (Web): when the model a session is
8
+ routed to **cannot accept image input**, this plugin reads the text out of a
9
+ pasted image for it. When the model *can* see images, the plugin stays completely
10
+ silent.
11
+
12
+ The engine is PP-OCRv5 + ONNX Runtime — **CPU-only, fully offline**, and your
13
+ images never leave your machine.
14
+
15
+ ## When the plugin acts, and when it stays silent
16
+
17
+ This is the most important table here. The decision is based on the input
18
+ capabilities the session's **actually routed model** declares
19
+ (`inputModalities`) — it is not a guess:
20
+
21
+ | Model routed by the session | Plugin behaviour | What the model actually receives |
22
+ | --- | --- | --- |
23
+ | **Explicitly cannot accept images** (e.g. a text-only model) | ✅ **Acts**: the image is saved to a local cache and its path is injected, so the model calls `ocr_image` | Text. For these models the harness substitutes only `[image omitted because this model accepts text only; …]` — **no path at all**, so without this plugin the model cannot read the image |
24
+ | **Explicitly accepts images** (multimodal model) | 🔇 **Silent**: no cache write, no hint injected | The image itself. The harness also prefixes a **read-only copy path**, so the model can call `ocr_image` on that path when it needs verbatim characters |
25
+ | **Cannot be determined** (provider not registered / lookup failed / no modalities declared) | 🔇 **Silent** | Handled by the harness as usual. Better not to intrude than to add an OCR hint to a model that may well see images |
26
+
27
+ "Silent when undetermined" is deliberate. If you genuinely want unconditional
28
+ intervention, set `autoOcr: 'always'` (see Configuration below).
29
+
30
+ > On **unlisted model ids**: the DeepSeek adapter explicitly returns
31
+ > `inputModalities: ["text"]` for ids that are not in the model catalog, so those
32
+ > text-only routes are covered by row 1. Other providers that report no modality
33
+ > information for undeclared models fall through to row 3 (silent).
34
+
35
+ ## Relationship to vision models
36
+
37
+ The two paths do not interfere, because they operate at different layers:
38
+
39
+ - **This plugin only does local OCR**: it caches the image and asks the model to
40
+ call `ocr_image`.
41
+ - **Whether the image is sent to the model** is decided by model capability and
42
+ client configuration; the plugin does not interfere.
43
+
44
+ So with a multimodal model you do not need to do anything: the plugin withdraws
45
+ on its own. If you want the text read anyway, just tell the agent "read this
46
+ image with ocr_image".
47
+
48
+ ## Quick start
49
+
50
+ ### Step 1: Install the plugin
51
+
52
+ DSH profiles are isolated, so install the plugin into **the profile you use**
53
+ (the Web profile is usually called `web`):
54
+
55
+ ```sh
56
+ npx -y @deepseek-ai/dsh plugin --profile web add dsh-ocr-local
57
+ ```
58
+
59
+ **Or straight from GitHub** (to track the latest commits, or when npm is
60
+ unreachable):
61
+
62
+ ```sh
63
+ npx -y @deepseek-ai/dsh plugin --profile web add github:grelvan/dsh-ocr-local
64
+ ```
65
+
66
+ **Or from a local clone:**
67
+
68
+ ```sh
69
+ git clone https://github.com/grelvan/dsh-ocr-local.git
70
+ npx -y @deepseek-ai/dsh plugin --profile web add ./dsh-ocr-local
71
+ ```
72
+
73
+ **Restart dsh** after installing, or the plugin will not take effect.
74
+
75
+ ### Step 2: Prepare the recognition engine (once)
76
+
77
+ Send the agent any image and say:
78
+
79
+ > read the text in this image
80
+
81
+ If the engine is not ready yet, the tool tells you what is missing. Then say:
82
+
83
+ > install the OCR environment with the ocr_setup tool
84
+
85
+ The plugin will **create a virtualenv → install Python dependencies → download
86
+ the models** (about 20MB). After that every recognition runs locally in seconds.
87
+
88
+ > Manual install works too (replace `<profile>` with your profile name, e.g. `web`):
89
+ >
90
+ > ```sh
91
+ > python ~/.dsh/profiles/<profile>/node_modules/dsh-ocr-local/ocr/setup.py
92
+ > ```
93
+
94
+ ### Step 3: Use it
95
+
96
+ **Option A: paste a screenshot (most common)**
97
+
98
+ Press Ctrl+V / Cmd+V in the Web composer. The image enters the session through
99
+ the browser's native attachment flow, and then splits according to the table
100
+ above: text-only model → this plugin covers it; multimodal model → the model
101
+ looks at the image directly.
102
+
103
+ **Option B: give the agent a path**
104
+
105
+ Send the agent the absolute path of an image file and say "read this image".
106
+
107
+ ## What it handles / limits
108
+
109
+ | ✅ Good at | ⚠️ Mediocre |
110
+ | --- | --- |
111
+ | Screenshots, error dialogs, chat logs | Very small text (e.g. 4px) may have a few wrong characters |
112
+ | Mixed Chinese + English, long paragraphs | Complex backgrounds, stylized fonts, handwriting |
113
+ | Dark-theme screenshots (auto-inverted) | Blurry or heavily compressed images |
114
+
115
+ In the output, **lines that are too small or low-confidence are flagged ⚠**, so
116
+ you can tell which characters are not fully trustworthy.
117
+
118
+ ### Why local OCR is still worth it for verbatim fidelity
119
+
120
+ A multimodal model looking at an image and local OCR reading it do **not** get
121
+ the same amount of information:
122
+
123
+ | | Multimodal model looking directly | Local OCR |
124
+ | --- | --- | --- |
125
+ | Resolution reaching the model | Bounded by the harness per-image pixel budget (640,000 px by default). A 1920×1080 screenshot is downscaled to about 1066×600 | Detection runs at a 736px longest side, but **recognition crops from the original image**, enlarging small text first (glyph height floor 20px, up to 6×) |
126
+ | Output | A paraphrase by the model, with **no signal about what it is unsure of** | Per-line text plus confidence / glyph height / box coordinates, with low-confidence lines flagged ⚠ |
127
+ | Best for | Understanding what is happening in the screenshot | Copying code, checking error messages, verifying hashes, reading table numbers |
128
+
129
+ An honest boundary note: the detection stage's 736px longest side is **more
130
+ conservative than the multimodal path's ~1066px**, so very small text in a
131
+ full-screen 4K screenshot can be missed here too. The accurate claim is
132
+ **"lines it does detect are transcribed more precisely"**, not "unlimited
133
+ resolution".
134
+
135
+ ## Configuration (optional; defaults are fine)
136
+
137
+ Config file: `~/.dsh/profiles/web/cordis.patch.yml`
138
+
139
+ ```yaml
140
+ - insert:
141
+ - id: ocr
142
+ name: 'dsh-ocr-local'
143
+ config:
144
+ autoOcr: true # see the table below
145
+ pythonPath: ~/miniconda3/envs/ocr/bin/python # optional: pick a Python
146
+ modelDir: ~/.dsh-ocr/models # optional: model directory
147
+ maxCacheFiles: 300 # optional: cache file cap
148
+ maxCacheAgeDays: 30 # optional: cache retention
149
+ ```
150
+
151
+ `autoOcr` has three states:
152
+
153
+ | Value | Behaviour |
154
+ | --- | --- |
155
+ | `true` (default) | Automatic: acts **only when the model explicitly cannot accept images** |
156
+ | `false` | No automatic intervention at all. The model can still call `ocr_image` on request |
157
+ | `'always'` | Unconditional intervention (a hint is injected even for vision-capable models). The old 0.3.x behaviour; useful for troubleshooting |
158
+
159
+ Environment variables:
160
+
161
+ | Variable | Purpose |
162
+ | --- | --- |
163
+ | `DSH_OCR_MODELS_MIRROR` | Mirror prefix for model downloads (e.g. `https://ghproxy.com/`) |
164
+ | `DSH_OCR_PYTHON` | Which Python the OCR engine uses (auto-detected by default) |
165
+ | `DSH_OCR_MODELS` | Model directory (defaults to `~/.dsh-ocr/models`) |
166
+
167
+ ## FAQ
168
+
169
+ **Q: I pasted an image but the model says it cannot see it.**
170
+ First check whether the current model is text-only (the only case this plugin is
171
+ meant to cover). If it is, verify the engine is ready ("check the OCR
172
+ environment with ocr_setup"). With a multimodal model the model simply looks at
173
+ the image, and this plugin is silent by design.
174
+
175
+ **Q: "Environment not ready" / "missing dependencies"?**
176
+ Tell the agent "install the OCR environment with ocr_setup", or run
177
+ `python ~/.dsh/profiles/web/node_modules/dsh-ocr-local/ocr/setup.py` manually.
178
+
179
+ **Q: Model download is slow or fails?**
180
+ Set the mirror and retry (idempotent):
181
+ `DSH_OCR_MODELS_MIRROR=https://ghproxy.com/ python .../ocr/setup.py`
182
+
183
+ **Q: pip reports externally-managed-environment (PEP 668)?**
184
+ Do not add `--break-system-packages`. Use `ocr/setup.py` — it creates a
185
+ virtualenv automatically and sidesteps the system Python restriction.
186
+
187
+ **Q: The recognition has wrong characters.**
188
+ Check the ⚠ flags in the output. For very small text the engine does misread:
189
+ scale the original image up and retry, or ask the agent to double-check that line.
190
+
191
+ **Q: Why does the plugin no longer intercept my paste?**
192
+ As of 0.4.0 it does not. The Web composer already intakes pasted images through
193
+ its native attachment flow; intercepting only prevented vision-capable models
194
+ from seeing the image. The old `pasteToPath` config key is gone — you can delete it.
195
+
196
+ ## How it works (one sentence)
197
+
198
+ A pasted image enters the session as an attachment through the native flow →
199
+ the plugin listens for `user/message` events → it queries the routed model's
200
+ `inputModalities` → **only when the model explicitly cannot accept images** does
201
+ it save the image to `~/.dsh/ocr/cache` and inject its path → the model calls
202
+ `ocr_image` → local PP-OCRv5 models (ONNX Runtime, CPU-only) → text. Models are
203
+ downloaded to `~/.dsh-ocr/models` on first use and everything is offline
204
+ afterwards. More detail in [docs/usage.md](docs/usage.md).
205
+
206
+ ## Upgrading
207
+
208
+ ```sh
209
+ npx -y @deepseek-ai/dsh plugin --profile web update dsh-ocr-local
210
+ ```
211
+
212
+ If you installed from a local directory, `git pull` and re-run `add` in the
213
+ plugin directory.
214
+
215
+ ## License
216
+
217
+ MIT (code). The recognition models are Apache-2.0 (PaddleOCR) and are downloaded
218
+ during setup. See [LICENSE](LICENSE).
package/README.md ADDED
@@ -0,0 +1,195 @@
1
+ # dsh-ocr-local
2
+
3
+ [English](README.en.md) · [中文](README.md)
4
+
5
+ [![license](https://img.shields.io/npm/l/dsh-ocr-local?style=flat-square)](LICENSE) [![GitHub](https://img.shields.io/badge/GitHub-grelvan%2Fdsh--ocr-local-2f81f7?style=flat-square)](https://github.com/grelvan/dsh-ocr-local)
6
+
7
+ 给 DeepSeek Harness(Web 端)装一个**本地 OCR 兜底**:当会话路由到的模型**不支持图片输入**时,
8
+ 把图片里的文字读出来给模型;模型能看图时,插件完全静默、不插手。
9
+
10
+ 识别引擎是 PP-OCRv5 + ONNX Runtime,**纯 CPU、完全离线**,图片不会离开你的电脑。
11
+
12
+ ## 插件什么时候生效,什么时候静默
13
+
14
+ 这是理解本插件最重要的一张表。判定基于当前会话实际路由到的模型**声明的输入能力**
15
+ (`inputModalities`),而不是猜:
16
+
17
+ | 会话路由到的模型 | 插件行为 | 模型实际拿到什么 |
18
+ | --- | --- | --- |
19
+ | **明确声明不支持图片**(如纯文本模型) | ✅ **生效**:图片存到本地缓存,并注入路径提示 → 模型调 `ocr_image` 识别 | 文字。Harness 对这类模型只给一句 `[image omitted because this model accepts text only; …]`,**没有任何路径**,所以没有本插件模型就完全读不到图 |
20
+ | **明确声明支持图片**(多模态模型) | 🔇 **静默**:不存缓存、不注入提示 | 图片本身。Harness 还会在图片前附一条**只读副本路径**,模型想逐字核对时可以直接对那个路径调 `ocr_image` |
21
+ | **无法确定**(provider 未注册 / 查询失败 / 没声明模态) | 🔇 **静默** | 按 Harness 原样处理。宁可不打扰,也不在一个可能能看图的模型上多塞提示 |
22
+
23
+ "无法确定就静默"是刻意的。如果你确实需要无条件介入,把 `autoOcr` 设成 `'always'`(见下方配置)。
24
+
25
+ > 关于「未登记的 model id」:DeepSeek 适配器对没写进模型目录的 id 会**显式**返回
26
+ > `inputModalities: ["text"]`,所以这类纯文本路由会被上表第一行正确覆盖。
27
+ > 其它 provider 若对未声明模型返回「无模态信息」,则落到第三行(静默)。
28
+
29
+ ## 与视觉模型的关系
30
+
31
+ 两条链路互不干扰,因为它们在**不同的层**做事:
32
+
33
+ - **插件只管本地 OCR**:把图存到本地、提示模型去调 `ocr_image`。
34
+ - **图要不要发给模型**由模型能力 / 客户端配置决定,插件不干预。
35
+
36
+ 所以多模态模型下你不需要为本插件做任何事:它会自己退场。想手动让它读图里的字,
37
+ 随时可以直接对 agent 说「用 ocr_image 读这张图」。
38
+
39
+ ## 快速开始
40
+
41
+ ### 第 1 步:安装插件
42
+
43
+ DSH 的 profile 互相隔离,插件要装到**你要用的那个 profile**(Web 端通常叫 `web`):
44
+
45
+ ```sh
46
+ npx -y @deepseek-ai/dsh plugin --profile web add dsh-ocr-local
47
+ ```
48
+
49
+ **也可以直接从 GitHub 安装**(想跟最新提交、或 npm 不可达时):
50
+
51
+ ```sh
52
+ npx -y @deepseek-ai/dsh plugin --profile web add github:grelvan/dsh-ocr-local
53
+ ```
54
+
55
+ **或本地克隆后安装:**
56
+
57
+ ```sh
58
+ git clone https://github.com/grelvan/dsh-ocr-local.git
59
+ npx -y @deepseek-ai/dsh plugin --profile web add ./dsh-ocr-local
60
+ ```
61
+
62
+ 装完**重启 dsh**,插件才会生效。
63
+
64
+ ### 第 2 步:准备识别引擎(只需一次)
65
+
66
+ 把任意一张图片发给 agent,说:
67
+
68
+ > 识别这张图片
69
+
70
+ 如果引擎还没装好,工具会告诉你缺什么。这时再对 agent 说:
71
+
72
+ > 用 ocr_setup 工具安装 OCR 环境
73
+
74
+ 插件会自动完成三件事:**建虚拟环境 → 装 Python 依赖 → 下载识别模型**(约 20MB),
75
+ 之后每次识别都在本地秒级完成。
76
+
77
+ > 想手动装也可以(把 `<profile>` 换成你的 profile 名,如 `web`):
78
+ >
79
+ > ```sh
80
+ > python ~/.dsh/profiles/<profile>/node_modules/dsh-ocr-local/ocr/setup.py
81
+ > ```
82
+
83
+ ### 第 3 步:开始使用
84
+
85
+ **方式 A:粘贴截图(最常用)**
86
+
87
+ 在 Web 输入框里按 Ctrl+V / Cmd+V。图片走浏览器原生的附件流程进入会话,
88
+ 之后按上面的表自动分流:纯文本模型 → 本插件兜底;多模态模型 → 模型直接看图。
89
+
90
+ **方式 B:告诉 agent 图片路径**
91
+
92
+ 把图片文件的绝对路径发给 agent,说「识别这张图片」。
93
+
94
+ ## 能识别什么 / 有什么限制
95
+
96
+ | ✅ 擅长 | ⚠️ 效果一般 |
97
+ | --- | --- |
98
+ | 截图、报错弹窗、聊天记录 | 极小的字(如 4px)可能有个别错字 |
99
+ | 中文 + 英文混排、长段落 | 复杂背景、艺术字、手写体 |
100
+ | 暗色主题截图(自动反色处理) | 模糊或严重压缩的图片 |
101
+
102
+ 识别结果里,**字太小或置信度低的行会标注 ⚠**,方便你判断哪些字不能全信。
103
+
104
+ ### 为什么"逐字保真"这件事仍然值得用本地 OCR
105
+
106
+ 多模态模型看图和本地 OCR 读图,拿到的**信息量不一样**:
107
+
108
+ | | 多模态模型直接看图 | 本地 OCR |
109
+ | --- | --- | --- |
110
+ | 送进模型的分辨率 | 受限于 Harness 的每图像素预算(默认 640,000 px)。1920×1080 的截图会被压到约 1066×600 | 检测阶段最长边 736px,但**识别阶段是从原图裁块**,小字还会先放大(字高下限 20px,最多 6×) |
111
+ | 输出 | 模型的一段转述,**不告诉你它哪里不确定** | 逐行文本 + 每行置信度 / 字高 / 坐标框,低置信行标 ⚠ |
112
+ | 适合 | 看懂截图里发生了什么 | 抄代码、对报错信息、核 hash、读表格数字 |
113
+
114
+ 诚实的边界说明:检测阶段的最长边限制是 736px,**比多模态链路的约 1066px 更保守**,
115
+ 所以 4K 全屏截图里极小的字,本插件也可能漏检。准确的说法是
116
+ **"检得到的行,认得更准"**,而不是"分辨率无上限"。
117
+
118
+ ## 配置(可选,默认不用动)
119
+
120
+ 配置文件:`~/.dsh/profiles/web/cordis.patch.yml`
121
+
122
+ ```yaml
123
+ - insert:
124
+ - id: ocr
125
+ name: 'dsh-ocr-local'
126
+ config:
127
+ autoOcr: true # 见下表
128
+ pythonPath: ~/miniconda3/envs/ocr/bin/python # 可选:指定 Python
129
+ modelDir: ~/.dsh-ocr/models # 可选:模型目录
130
+ maxCacheFiles: 300 # 可选:图片缓存最多文件数
131
+ maxCacheAgeDays: 30 # 可选:图片缓存保留天数
132
+ ```
133
+
134
+ `autoOcr` 三态:
135
+
136
+ | 值 | 行为 |
137
+ | --- | --- |
138
+ | `true`(默认) | 自动判定:**只有模型明确不支持图片输入时**才介入 |
139
+ | `false` | 完全关闭自动介入。仍可手动让模型调 `ocr_image` |
140
+ | `'always'` | 无条件介入(即使模型能看图也注入提示)。旧版 0.3.x 的行为,排查问题时可用 |
141
+
142
+ 常用环境变量:
143
+
144
+ | 变量 | 作用 |
145
+ | --- | --- |
146
+ | `DSH_OCR_MODELS_MIRROR` | 模型下载镜像前缀(国内下载慢时设,如 `https://ghproxy.com/`) |
147
+ | `DSH_OCR_PYTHON` | 指定 OCR 用哪个 Python(默认自动找) |
148
+ | `DSH_OCR_MODELS` | 模型存放目录(默认 `~/.dsh-ocr/models`) |
149
+
150
+ ## 常见问题
151
+
152
+ **Q:粘贴了图片,但模型说看不到图?**
153
+ 先确认当前模型是不是纯文本模型(这是本插件唯一该生效的场景)。如果是,检查引擎是否就绪
154
+ (对 agent 说「用 ocr_setup 检查 OCR 环境」)。如果用的是多模态模型,模型直接看图即可,
155
+ 本插件按设计就是静默的。
156
+
157
+ **Q:提示「环境未就绪」/「缺少依赖」?**
158
+ 对 agent 说「用 ocr_setup 安装 OCR 环境」即可自动修复;或手动运行
159
+ `python ~/.dsh/profiles/web/node_modules/dsh-ocr-local/ocr/setup.py`。
160
+
161
+ **Q:模型下载很慢或失败?**
162
+ 设镜像后重试(幂等,可反复跑):
163
+ `DSH_OCR_MODELS_MIRROR=https://ghproxy.com/ python .../ocr/setup.py`
164
+
165
+ **Q:系统提示 pip externally-managed-environment(PEP 668)?**
166
+ 不要加 `--break-system-packages`。直接用 `ocr/setup.py`——它会自动创建虚拟环境,
167
+ 绕开系统 Python 的限制。
168
+
169
+ **Q:识别结果有错字?**
170
+ 看输出里的 ⚠ 标注。字太小时模型确实会看走眼:把原图放大一点再试,
171
+ 或让 agent 把对应行再确认一遍。
172
+
173
+ **Q:为什么插件不拦截我的粘贴了?**
174
+ 0.4.0 起不再拦截。Web 输入框原生就把粘贴的图片收进附件流程,拦截只会让
175
+ 本来能看图的模型反而看不到图。旧配置里的 `pasteToPath` 已失效,可以删掉。
176
+
177
+ ## 工作原理(一句话)
178
+
179
+ Web 端粘贴的图片按原生流程进入会话成为附件 → 插件监听 `user/message` 事件 →
180
+ 查询当前路由模型的 `inputModalities` → **只有明确不支持图片时**,把图片存到
181
+ `~/.dsh/ocr/cache` 并向模型注入路径提示 → 模型调 `ocr_image` → 本地 PP-OCRv5 模型
182
+ (ONNX Runtime,纯 CPU)→ 文字。模型第一次使用时下载到 `~/.dsh-ocr/models`,
183
+ 之后完全离线。更多细节见 [docs/usage.md](docs/usage.md)。
184
+
185
+ ## 升级
186
+
187
+ ```sh
188
+ npx -y @deepseek-ai/dsh plugin --profile web update dsh-ocr-local
189
+ ```
190
+
191
+ 从本地目录安装的话,`git pull` 后在插件目录重跑一次 `add` 即可。
192
+
193
+ ## 许可
194
+
195
+ MIT(代码)。识别模型 Apache-2.0(PaddleOCR),安装时自动下载。见 [LICENSE](LICENSE)。
@@ -0,0 +1,4 @@
1
+ # dsh bundle patch: inserts this plugin into a profile's layer stack.
2
+ - insert:
3
+ - id: ocr
4
+ name: 'dsh-ocr-local'
@@ -0,0 +1,82 @@
1
+ /**
2
+ * dsh-ocr-local — 能力判定(纯函数,零依赖,可单测)。
3
+ *
4
+ * 插件只在**明确接入不支持多模态的模型**时才介入:把进入会话的图片存到本地
5
+ * 缓存并向模型注入路径提示,让它调用 `ocr_image` 本地识别。
6
+ *
7
+ * 判定三态:
8
+ * - 模型明确声明支持 image(`inputModalities` 含 `image`)→ 静默,交给视觉链路
9
+ * - 模型明确声明只支持 text(含未登记的 model id,适配器按其显式返回处理)→ 介入
10
+ * - 无法确定(没有 llm 服务 / provider 未注册 / 查询失败 / 没返回模态)→ 静默
11
+ *
12
+ * 「无法确定时静默」是有意为之:宁可不打扰,也不要在一个其实能看图的模型上
13
+ * 多塞一条 OCR 提示。需要无条件介入时用 `autoOcr: 'always'`。
14
+ */
15
+
16
+ /** autoOcr 生效模式。 */
17
+ export const AUTO_OCR_MODES = ['off', 'auto', 'always']
18
+
19
+ /**
20
+ * 把插件配置解析成生效模式。
21
+ * @param config - 插件配置对象。
22
+ * @returns `off`(关闭)| `auto`(仅明确不支持多模态时介入,默认)| `always`(总是介入)。
23
+ */
24
+ export function autoOcrMode(config = {}) {
25
+ if (config.autoOcr === false) return 'off'
26
+ if (config.autoOcr === 'always') return 'always'
27
+ return 'auto'
28
+ }
29
+
30
+ /**
31
+ * 给定模型声明的输入模态,判断是否**明确**不支持图片输入。
32
+ * @param modalities - `inputModalities` 数组;undefined / 空数组表示无法确定。
33
+ * @returns 明确只支持文本时为 true。
34
+ */
35
+ export function isExplicitlyTextOnly(modalities) {
36
+ if (!Array.isArray(modalities) || modalities.length === 0) return false
37
+ return !modalities.includes('image')
38
+ }
39
+
40
+ /**
41
+ * 是否为一张刚进入会话的图片注入本地 OCR 路径提示。
42
+ * @param mode - {@link autoOcrMode} 的返回值。
43
+ * @param modalities - 解析到的 `inputModalities`;undefined 表示无法确定。
44
+ * @returns 应当注入时为 true。
45
+ */
46
+ export function shouldInjectOcrPath(mode, modalities) {
47
+ if (mode === 'off') return false
48
+ if (mode === 'always') return true
49
+ return isExplicitlyTextOnly(modalities)
50
+ }
51
+
52
+ /**
53
+ * 从候选来源里选出「下一条请求会用哪个 provider/model」。
54
+ * 按可信度排序:用户刚切换的待生效选择 → 上一次真实请求的请求头 → 会话创建时的路由。
55
+ * @param candidates - `{ pending, header, options }`,每项可为 undefined 或
56
+ * `{ provider, model }` 形状的对象。
57
+ * @returns `{ provider, model, source }`,都取不到时返回 undefined。
58
+ */
59
+ export function pickRoute(candidates = {}) {
60
+ const order = [
61
+ ['pending', candidates.pending],
62
+ ['header', candidates.header],
63
+ ['options', candidates.options],
64
+ ]
65
+ for (const [source, value] of order) {
66
+ if (!value) continue
67
+ const provider = typeof value.provider === 'string' ? value.provider.trim() : ''
68
+ const model = typeof value.model === 'string' ? value.model.trim() : ''
69
+ if (provider && model) return { provider, model, source }
70
+ }
71
+ return undefined
72
+ }
73
+
74
+ /**
75
+ * 跨会话共用的模态缓存键。
76
+ * @param provider - provider 路由 id。
77
+ * @param model - 模型 id。
78
+ * @returns 稳定的缓存键。
79
+ */
80
+ export function modalityCacheKey(provider, model) {
81
+ return `${provider}\u0000${model}`
82
+ }