deep-read-summarize 0.3.18 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -3,6 +3,65 @@
3
3
  All notable changes to this project will be documented in this file.
4
4
  The format is based on [Keep a Changelog](https://keepachangelog.com/).
5
5
 
6
+ ## [0.4.0] — 2026-10-25
7
+
8
+ 保留完整精读与独立原文核验,减少脚本搬运、并发空等和无效修订,并修复来源质量判断与视频分P等问题。
9
+
10
+ ### Added
11
+
12
+ - **参数直达工具 `deep_read_summarize`**:宿主具有 tools / workflowEngine 服务时,由插件直接提交工作流,模型只传 input/type/options,无需原样重发约 4.6 万字符脚本。转发调用代理和取消信号,完成或失败后释放 workflow run;旧宿主保留原 workflow + skill 入口。
13
+ - **阶段耗时 `timings`**:返回识别、获取、精读、成稿、质量核验和总耗时(毫秒),供同资料、模型与质量门槛下比较。测试调度模拟不代表真实 LLM 加速比例。
14
+ - **显式自定义解析器目录**:插件顶层 `config.customParsersDir` 可加载独立目录,各 profile 隔离;包内 `custom-parsers/` 纳入 npm 分发。
15
+
16
+ ### Fixed
17
+
18
+ - **视频来源质量**:只有简介或未确认来源的视频保持质量降级,不能因“简介的所有行都读过”而报告完整精读通过;来源类型进入 QC 上下文,并在笔记中标明限制。
19
+ - **B站多P**:保留短链最终 URL 与 p,按 `pages[p-1]` 选择 CID、标题和时长;字幕、音频使用同一分P,非法或越界参数不能默默回退第1P。
20
+ - **异常与重试**:类型识别和获取的可恢复调用错误返回对应 stage,真正 FATAL 继续终止;缺总行数、失败精读块、来源不完整等不能靠改写解决的问题不再消耗全文修订轮次。
21
+ - **格式与引用**:严格检查模板字符串,拒绝非法 YAML 裸标量和错误字段类型;区分 Markdown 引文的可见原话与强调、行号等展示信息,减少真实引用被误判。
22
+ - **文件名**:过滤控制字符、设备保留名与非法尾缀;对清洗或截短的标题增加稳定后缀,限制 UTF-8 文件名长度,避免 NUL、超长标题落盘失败与不同非法标题折叠到同名路径。
23
+ - **URL 指纹**:保留可选择不同资料的尾斜杠、空 query 与 fragment;新版键与旧缓存隔离,防止旧版碰撞继续跳过资料。
24
+ - **解析器更新**:注册后生成技能和工具调用使用最新注册表,不再引用模块加载时的旧脚本;插件构建阶段定位本版本转写脚本,避免每次递归搜索其他安装目录。
25
+ - **转写热启动**:健康环境直接执行一次 Python runner,移除单独 import 探测;仅明确缺依赖或版本不匹配时才要求 uv 并自举,已有可用环境不再被“找不到 uv”提前拒绝。
26
+ - **转写版本记录**:固定已验证的 `faster-whisper==1.2.1`,记录实际 Python / 依赖版本;纠正文档“uv / Python补丁 / 全部依赖完全锁定”的不实承诺。
27
+
28
+ ### Performance
29
+
30
+ - **固定三个并发槽位**:一个任务完成即领取下一块,取消每批三个都结束才启动下一批的屏障,输出仍按原分块顺序合并。
31
+ - **均衡分块工作量**:在既有代理数和 maxChunks 上限内按行数均分已知全文,超长逻辑段可拆分,保持每行恰好覆盖一次。
32
+ - **缩减 QC 上下文**:保留原文路径、行号、缺口与待核验引文,移除重复的完整 Map 笔记;独立核验仍须读取原文。
33
+
34
+ ### Compatibility
35
+
36
+ - 默认模型、篇幅、章节与引用门槛不降低;新工具是可选入口,原入口保留。新返回字段为补充元数据。
37
+ - 新版缓存指纹可能导致升级后的首次重处理;清洗过或过长标题的建议文件名会改变,正常标题保持不变。
38
+ - 原文真实性仍取决于核验代理实际读取原文。逐字稿内容缓存、常驻 ASR worker、分层 QC 和共享转写环境的冷启动互斥留待单独验证,未在本次发布中声称完成。
39
+
40
+ ## [0.3.19] — 2026-10-24
41
+
42
+ 修复 B04–B10,并让质量检查结合程序门禁和可回查的原文证据(O01)。
43
+
44
+ ### Fixed
45
+
46
+ - **成稿与修订保留(B04)**:拒绝空白和非字符串初稿;修订调用失败、返回空值或格式/篇幅退化时保留上一份有效笔记。质量检查失败不再导致正常初稿丢失。
47
+ - **自定义解析器(B05)**:使用 Acorn 解析 CommonJS 模块并打包本地依赖,保留方法简写、模块常量、helper、JSON 和循环依赖。无法在沙箱支持的 Node/外部依赖和动态加载在构建时明确报错;独立的程序化函数仍可直接注册。
48
+ - **视频定位(B06)**:`includeTimestamps` 贯穿获取和转写,按真实分段 start/end 输出毫秒时间范围;平台字幕也保留 cue 时间。关闭时间戳仍逐段换行,不再把长逐字稿压成一行。
49
+ - **配置边界(B07)**:workflow 与插件 config 共用同一份默认值和校验器。检查有限整数、区间、布尔、路径和枚举;直接调用非法参数 FATAL,profile config 逐项告警回退。重试次数限制为 0–3。
50
+ - **转写完成信号(B08)**:复验上一版的修复;仅在本次进程成功、非空产物校验并原子发布后输出 DONE,空结果和所有失败路径均不输出成功信号。
51
+ - **缓存中断恢复(B09)**:每条 JSONL 记录以独立前后换行包围,单次 O_APPEND 追加,不读改尾部或留下恢复锁。损坏尾行不再吞掉下一条成功记录;短写明确失败,元数据不能覆盖保留字段。
52
+ - **发布身份(B10)**:checkout 前验证完整 tag ref 和远端对象,checkout 后核对 commit/包版本;按 ref、SHA、模式、nonce 和 attempt 精确认领本次 run,不再认领第一个新 ID。版本文件与发布表格先在内存预演,避免坏表格先留下已 bump 的版本;发布时同步更新锁文件的根版本。
53
+
54
+ ### Changed
55
+
56
+ - **原文核验(O01)**:格式、必要章节和正文长度由程序检查;核验代理读取本次原文,返回全部分块摘录、关键引文原句及行号。缺证据、错行号、引文不匹配或已知精读缺口均不能通过,单独 `pass: true` 无效。新增 `qualityChecks`、`qualityEvidence` 返回字段。
57
+ - 中文/日文字符按字、其他文字按词计数;排除元数据、代码块、HTML 注释和链接地址。frontmatter 按提供的标量/列表模板校验,关键逐字引用集中在第 4 节。
58
+ - 新增一个锁定的生产依赖 Acorn,用于构建期 JavaScript 解析;CI 与发布工作流通过 `npm ci --ignore-scripts` 安装锁定依赖。
59
+
60
+ ### Compatibility
61
+
62
+ - 默认配置仍可使用。依赖宽松强转(如字符串布尔值、负数重试)或超出范围的参数现在会报错;自定义解析器须使用纯提示词构造逻辑和静态本地模块依赖。
63
+ - 原文核验比仅检查成稿多读取资料,质量状态会更严格;引用真实性仍取决于核验代理实际读取原文,返回的摘录和行号用于回查。
64
+
6
65
  ## [0.3.18] — 2026-10-23
7
66
 
8
67
  修复自动解析器分流、并发临时文件串稿和分块上限导致的全文遗漏。
package/README.en.md CHANGED
@@ -74,6 +74,10 @@ Three waves, normally N+3 sub-agent calls (fetch, N chunk reads, draft, and qual
74
74
 
75
75
  Chunk plans are sorted, duplicate ranges trimmed, and gaps filled. Plans above `maxChunks` merge adjacent ranges instead of discarding the document tail. The returned `coverageComplete` indicates coverage of all declared lines based on a valid `totalLines`; without a line count, known ranges are retained, `qualityPassed` is set to `false`, and the coverage uncertainty is reported.
76
76
 
77
+ Quality checks combine code and source review. Code validates the supplied scalar/list frontmatter template, required sections, and length (CJK characters plus other words, excluding metadata, code blocks, HTML comments, and link URLs). A separate reviewer reads the current source file and returns per-chunk excerpts and line-bound evidence for verbatim quotations in section 4. A bare `pass: true` cannot override these checks. `qualityChecks` exposes the gates, and `qualityEvidence` exposes the source path and evidence. Source authenticity still depends on the reviewer's actual source reading; uncertainty fails the gate. Empty, rejected, or structurally degraded repairs retain the preceding usable note.
78
+
79
+ Direct workflow options are strict: `minWords` is an integer in 1–100000, `maxChunks` is a positive integer (values above 12 are capped), and `maxRetries` is an integer in 0–3. Invalid values and unknown keys fail before work begins. Plugin profile configuration instead warns per invalid field and uses its default, preserving host startup.
80
+
77
81
  ```
78
82
  parsers/ Parsers for each input type, dispatched by type
79
83
  book.js Books: PDF/EPUB/MOBI text extraction, chapter chunking
@@ -93,7 +97,9 @@ workflow.js the workflow meta and assembly logic (parser inlining, schema i
93
97
 
94
98
  > The script body lives in `lib/workflow-script.txt` on purpose: this file was corrupted twice by "escaping on top of escaping". It is plain text now, so reading and editing it needs no encoding step.
95
99
 
96
- To change how an input type is handled, drop a parser with the same interface into `custom-parsers/`; a type with the same name overrides the built-in implementation, and you can also add a brand-new type name (e.g. `podcast`) - every registered parser is inlined into the workflow script, and both the `type` whitelist and the schema's `kind` enum follow the registry. The interface has only three fields: `name`, `types`, `buildPrompt(input, opts)`.
100
+ To override or add a parser, use the package's `custom-parsers/` directory (included in npm packaging), or set the plugin-level `config.customParsersDir` to your own directory. This is not a workflow option. Per-profile directories remain isolated, and rebuilding skill content uses the current registry, including programmatic registrations. Parsers expose `name`, `types`, and `buildPrompt(input, opts)`; registered types are inlined into the workflow and its schema.
101
+
102
+ File parsers bundle their CommonJS modules and literal relative `.js`, `.cjs`, and `.json` dependencies, preserving module constants, helpers, method shorthand, and cycles. Acorn parses JavaScript during script construction and is installed automatically by npm; it does not run in the workflow sandbox. Node built-ins, external packages, and dynamic `require`/`import` are unsupported in sandbox parsers and fail at build time. Programmatically registered functions must be self-contained; use a file parser for module closures.
97
103
 
98
104
  ---
99
105
 
@@ -119,7 +125,7 @@ Further notes:
119
125
  deep-read-summarize is a DSH plugin installable via npm or locally:
120
126
 
121
127
  ```bash
122
- # npm install (published to the registry, no third-party dependencies)
128
+ # npm install (includes Acorn for parser construction)
123
129
  npm install deep-read-summarize
124
130
 
125
131
  # or local install (dsh profile directory)
@@ -130,7 +136,8 @@ pnpm add ./deep-read-summarize-0.3.10.tgz
130
136
  ```
131
137
 
132
138
  After installation it registers:
133
- - the `deep-read-summarize` workflow (meta + script)
139
+ - the `deep_read_summarize` tool when the host provides tools and workflowEngine services; models send only input/type/options
140
+ - the `deep-read-summarize` workflow definition (meta + script), with the existing workflow-tool route retained for older hosts
134
141
  - the `deep-read-summarize` skill (`skills/deep-read-summarize/SKILL.md`)
135
142
  - four parsers (`parsers/`) and JSON Schemas (`schemas/`)
136
143
 
@@ -144,18 +151,18 @@ After installation it registers:
144
151
  npm install deep-read-summarize
145
152
 
146
153
  # Option B: develop from source
147
- git clone https://github.com/<your-org>/deep-read-summarize.git
154
+ git clone https://github.com/PensiveFei/deep-read-summarize.git
148
155
  cd deep-read-summarize
149
- npm install # no third-party dependencies, just initialization
156
+ npm ci # install locked dependencies
150
157
  ```
151
158
 
152
- ### 2. Verify the environment (offline, no API key needed)
159
+ ### 2. Verify installation (offline, no API key needed)
153
160
 
154
161
  ```bash
155
- npm test # 35 fixture tests, all run offline
162
+ node -e "console.log(require('deep-read-summarize').name)"
156
163
  ```
157
164
 
158
- Seeing `TOTAL: 25 passed, 0 failed` means the environment is ready.
165
+ Seeing `deep-read-summarize` confirms the package loads. Source checkouts can additionally run `npm test` and `npm run test:node`; tests are not included in the npm package.
159
166
 
160
167
  ### 3. Feed a demo file with one command
161
168
 
@@ -170,9 +177,9 @@ meta: deep-read-summarize
170
177
  parsers: book, paper, video, web
171
178
  ```
172
179
 
173
- ### 4. Real run (requires the DSH workflow tool)
180
+ ### 4. Real run (requires DSH)
174
181
 
175
- Pass the JSON below to DSH's workflow tool (see Usage):
182
+ Prefer passing the JSON below directly to `deep_read_summarize`. On older hosts, use the skill's meta + script with the existing `workflow` tool:
176
183
 
177
184
  ```jsonc
178
185
  { "input": "https://arxiv.org/abs/2307.09042", "type": "paper", "options": { "maxChunks": 4, "fastMode": true } }
@@ -185,7 +192,7 @@ The workflow returns a structured result: `{ ok, kind, title, filePath, qualityP
185
192
  ---
186
193
  ## Usage
187
194
 
188
- Pass this JSON to DSH's workflow tool:
195
+ The `deep_read_summarize` tool takes these parameters directly. On older hosts, use them as workflow args with the skill's meta + script:
189
196
 
190
197
  ```jsonc
191
198
  {
@@ -222,7 +229,7 @@ Notes are written to `options.outputDir` (default `./output`, can point at your
222
229
 
223
230
  ### Frontmatter profiles (`options.frontmatter`)
224
231
 
225
- The default is `obsidian`, byte-for-byte what earlier versions produced. Profiles exist because `type`, `aliases` and `url` are **reserved keys in Hugo** with different meanings -- dropping an Obsidian note into Hugo changes how the page renders, rather than adding a few harmless fields.
232
+ The default is `obsidian`, with the same keys as earlier versions and safely quoted scalar/list values. Profiles exist because `type`, `aliases` and `url` are **reserved keys in Hugo** with different meanings -- dropping an Obsidian note into Hugo changes how the page renders, rather than adding a few harmless fields.
226
233
 
227
234
  | Key | `obsidian` (default) | `generic` | `minimal` |
228
235
  |---|---|---|---|
@@ -259,7 +266,7 @@ Video text extraction is a **unified pipeline** (the old "three tiers" are gone)
259
266
 
260
267
  `options.transcribe`: **default `true`** (auto-transcribe when no subtitles); set `false` to skip transcription (use subtitles/desc or degrade).
261
268
 
262
- **Transcription toolchain (identical for you and the author)**: the script **bootstraps itself** — it uses `uv` to create a **Python 3.12** environment + **Tsinghua mirror** to install `faster-whisper`, downloads the model via `HF_ENDPOINT=https://hf-mirror.com` and **caches it for reuse**; faster-whisper bundles PyAV to decode audio, so **no separate ffmpeg** is needed; **versions are pinned** (Python 3.12 / faster-whisper / uv) so the author's environment matches yours. The script is written in **Node**, so **Windows / macOS / Linux share one implementation and one command** (`node scripts/transcribe.js --audio <file> --out <txt>`). Transcription runs on **CPU** by default (`options.device`); GPU needs a CUDA runtime (cuBLAS/cuDNN), so `options.device: "auto"` is opt-in and falls back to CPU automatically. Cache location per platform: Windows `%LOCALAPPDATA%\deep-read-summarize`, macOS `~/Library/Caches/deep-read-summarize`, Linux `$XDG_CACHE_HOME/deep-read-summarize`. Python provisioning uses a mirror for speed and **falls back to a direct GitHub download when the mirror fails** (a dead mirror must not make the feature uninstallable); override it with the `DRS_UV_PYTHON_MIRROR` environment variable (empty string = never use a mirror).
269
+ **Transcription toolchain**: `uv` provisions **Python 3.12** and installs **`faster-whisper==1.2.1`** from the Tsinghua mirror; models use `HF_ENDPOINT=https://hf-mirror.com` and the local cache. Healthy cached environments run directly without requiring uv or a separate Python import probe. Actual interpreter and dependency versions are recorded in the cache's `runtime.json`; Python patch releases, uv, and all transitive dependencies are not fully pinned. PyAV decodes audio without a separate ffmpeg. All three platforms use `node scripts/transcribe.js --audio <file> --out <txt>`. CPU is the default; `options.device: "auto"` can try a supplied CUDA runtime and falls back to CPU. Cache roots are `%LOCALAPPDATA%\deep-read-summarize` on Windows, `~/Library/Caches/deep-read-summarize` on macOS, and `$XDG_CACHE_HOME/deep-read-summarize` on Linux (falling back to `~/.cache`). Python provisioning tries a mirror and then direct GitHub; `DRS_UV_PYTHON_MIRROR` overrides it, with an empty value selecting direct downloads.
263
270
 
264
271
  > ⚠️ **The first transcription downloads the model first (small ≈ 484 MB, via hf-mirror, not GitHub) and may take a few minutes** — please be patient; **it downloads only once** on your machine, then every later transcription uses the cache and starts instantly. **Only subtitle-less videos trigger transcription** (subtitled videos use their subtitles directly — no trigger, no download).
265
272
 
@@ -311,9 +318,21 @@ Two classes:
311
318
 
312
319
  ---
313
320
 
321
+ ## Faster runs with the same quality gates
322
+
323
+ - Prefer `deep_read_summarize` on supported hosts: the model submits parameters instead of reproducing the entire workflow script. Older hosts retain the existing route.
324
+ - All known source lines remain covered. Up to three reading tasks run concurrently, and each free slot immediately starts another task. Line-balanced chunks avoid one oversized task holding up a batch.
325
+ - Independent source review remains enabled, with duplicate full Map notes removed from review contexts. Missing line counts, failed reads, and description-only video sources cannot be fixed by rewriting the draft, so those states no longer consume full repair loops.
326
+ - Video `textSource: "desc"` is explicitly degraded and cannot pass as a complete deep read. Bilibili `p` selects the matching CID, part title, and duration.
327
+ - Returned `timings` reports stage durations in milliseconds. Compare the same sources, model, output size, and quality gates; scheduling simulations are not real LLM speedup measurements.
328
+
329
+ This release does not lower model size, output length, or source coverage. Persistent ASR workers, transcript caching, and hierarchical review need separate quality and performance benchmarks.
330
+
314
331
  ## Cost
315
332
 
316
- A paper, 6 chunks, full mode: roughly 15–25k tokens. `fastMode` with a lower `maxChunks` saves about 40%.
333
+ Token use depends on document length, chunk count, and output size. Source review reads the document again, and each repair round adds a repair and a review call. `fastMode` reduces output sections; reducing `maxChunks` limits agent count while preserving the complete source.
334
+
335
+ Programmatic `lib/cache` callers: URL trailing slashes, explicit empty queries, and fragments are preserved because they may select different content. New fingerprints are isolated from old records, so an upgrade can cause one reprocessing pass. Callers must still verify source freshness and configuration before reusing results.
317
336
 
318
337
  ---
319
338
 
package/README.md CHANGED
@@ -74,6 +74,10 @@ Markdown 笔记
74
74
 
75
75
  分块计划会排序、裁掉重复区间并补齐缺口,超过 `maxChunks` 时合并相邻块以保留全文。返回的 `coverageComplete` 表示是否依据有效的 `totalLines` 覆盖全部行号;缺少总行数时保留已知范围,同时将 `qualityPassed` 置为 `false` 并说明覆盖无法确认。
76
76
 
77
+ 质量检查分两部分:程序按提供的标量/列表 frontmatter 模板检查字段、必要章节和篇幅(汉字按字、其他文字按词,排除 frontmatter、代码块、HTML 注释及链接地址);独立核验代理读取本次原文文件,逐块检查覆盖,并为第 4 节的逐字引用返回原句和行号。单独的 `pass: true` 无法通过检查。`qualityChecks` 返回检查项,`qualityEvidence` 返回原文路径、分块摘录和引用证据;原文真实性判断仍由读取原文的代理完成,无法核实就报告未通过。修订为空、调用失败或结构退化时保留上一份有效成稿。
78
+
79
+ 直接调用 workflow 的 options 必须类型正确:`minWords` 为 1–100000 的整数,`maxChunks` 为正整数(大于 12 仍夹到 12),`maxRetries` 为 0–3 的整数;非法值和未知键在启动时 FATAL。插件 profile 的 config 则逐项告警并回退默认值,避免阻止宿主启动。
80
+
77
81
  ```
78
82
  parsers/ 各输入类型的解析器,按类型分发
79
83
  book.js 书籍:PDF/EPUB/MOBI 文本提取、章节分块
@@ -92,7 +96,9 @@ workflow.js workflow 的 meta 与组装逻辑(解析器内联、schema
92
96
 
93
97
  > 脚本正文单独放 `lib/workflow-script.txt`:这个文件曾经因为「转义套转义」被改坏过两次,现在它是普通文本,读改都不需要编码步骤。
94
98
 
95
- 想换某个输入类型的处理方式,在 `custom-parsers/` 放一个同接口的解析器即可,同名类型会覆盖内置实现;用新类型名(如 `podcast`)也可以——注册表里的解析器会全部内联进 workflow 脚本,`type` 白名单与 schema 的 `kind` 枚举都跟着注册表走。接口只有三个字段:`name`、`types`、`buildPrompt(input, opts)`。
99
+ 想换某个输入类型的处理方式,在插件包的 `custom-parsers/` 放一个同接口的解析器即可,该目录会随 npm 包分发。也可在插件顶层 config 中指定 `customParsersDir`,使用自己的解析器目录;它不是 workflow options。相同类型会覆盖内置实现,也可新增 `podcast` 等类型。程序化注册后重新生成技能会使用最新注册表;各 profile 的专属目录相互隔离。接口为 `name`、`types`、`buildPrompt(input, opts)`。
100
+
101
+ 解析器的 CommonJS 模块会连同字面量相对路径的 `.js` / `.cjs` / `.json` 依赖一起打包,保留模块级常量、helper、方法简写和循环依赖。构建时使用 Acorn 解析 JavaScript;它由 npm 自动安装,不进入 workflow 沙箱。沙箱中的解析器只负责生成提示词,不支持 Node 内置模块、外部包或动态 `require` / `import`;这些依赖会在构建时明确报错。程序化注册的函数须自包含,模块闭包请使用文件解析器。
96
102
 
97
103
  ---
98
104
 
@@ -118,7 +124,7 @@ workflow.js workflow 的 meta 与组装逻辑(解析器内联、schema
118
124
  deep-read-summarize 是 DSH 插件,可通过 npm 或本地安装:
119
125
 
120
126
  ```bash
121
- # npm 安装(已发布到 registry,无第三方依赖)
127
+ # npm 安装(自动安装用于解析器构建的 Acorn)
122
128
  npm install deep-read-summarize
123
129
 
124
130
  # 或本地安装(dsh profile 目录)
@@ -129,7 +135,8 @@ pnpm add ./deep-read-summarize-0.3.10.tgz
129
135
  ```
130
136
 
131
137
  安装后自动注册:
132
- - `deep-read-summarize` workflow(meta + script)
138
+ - `deep_read_summarize` 工具(宿主提供 tools / workflowEngine 服务时);模型只传 input/type/options,插件直接提交已加载的工作流
139
+ - `deep-read-summarize` 工作流定义(meta + script;旧宿主保留通过 workflow 工具调用的方式)
133
140
  - `deep-read-summarize` 技能(`skills/deep-read-summarize/SKILL.md`)
134
141
  - 四种解析器(`parsers/`)与 JSON Schema(`schemas/`)
135
142
 
@@ -143,18 +150,18 @@ pnpm add ./deep-read-summarize-0.3.10.tgz
143
150
  npm install deep-read-summarize
144
151
 
145
152
  # 方式 B:从源码开发
146
- git clone https://github.com/<your-org>/deep-read-summarize.git
153
+ git clone https://github.com/PensiveFei/deep-read-summarize.git
147
154
  cd deep-read-summarize
148
- npm install # 无第三方依赖,仅初始化
155
+ npm ci # 安装锁定依赖
149
156
  ```
150
157
 
151
- ### 2. 验证环境(离线,不需要任何 API key)
158
+ ### 2. 验证安装(离线,不需要任何 API key)
152
159
 
153
160
  ```bash
154
- npm test # 35 项 fixture 测试,全部离线跑通
161
+ node -e "console.log(require('deep-read-summarize').name)"
155
162
  ```
156
163
 
157
- 看到 `TOTAL: 25 passed, 0 failed` 即环境就绪。
164
+ 看到 `deep-read-summarize` 即包可正常加载。从源码开发时再运行 `npm test` 和 `npm run test:node`;测试不随 npm 包分发。
158
165
 
159
166
  ### 3. 一行命令喂入 demo 文件
160
167
 
@@ -169,9 +176,9 @@ meta: deep-read-summarize
169
176
  parsers: book, paper, video, web
170
177
  ```
171
178
 
172
- ### 4. 真实运行(需要 DSH workflow 工具)
179
+ ### 4. 真实运行(需要 DSH)
173
180
 
174
- 把下面的 JSON 传给 DSH 的 workflow 工具(见下节「用法」):
181
+ 优先把下面的 JSON 直接传给 `deep_read_summarize` 工具。旧宿主使用技能提供的 meta + script,通过 `workflow` 工具执行:
175
182
 
176
183
  ```jsonc
177
184
  { "input": "https://arxiv.org/abs/2307.09042", "type": "paper", "options": { "maxChunks": 4, "fastMode": true } }
@@ -184,7 +191,7 @@ parsers: book, paper, video, web
184
191
  ---
185
192
  ## 用法
186
193
 
187
- 把下面这段 JSON 传给 DSH 的 workflow 工具:
194
+ `deep_read_summarize` 工具只需要以下参数;旧宿主将其作为 `workflow` 的 args,并使用技能中的 meta + script:
188
195
 
189
196
  ```jsonc
190
197
  {
@@ -221,7 +228,7 @@ parsers: book, paper, video, web
221
228
 
222
229
  ### frontmatter 三档(`options.frontmatter`)
223
230
 
224
- 默认 `obsidian`,与历史版本**逐字一致**。之所以要分档:`type`、`aliases`、`url` 在 Hugo 里是**保留键**,语义和 Obsidian 完全不同——直接丢进 Hugo 会改变渲染行为,而不是多几个无害字段。
231
+ 默认 `obsidian`,键集与历史版本一致,标量和列表值使用安全引号。之所以要分档:`type`、`aliases`、`url` 在 Hugo 里是**保留键**,语义和 Obsidian 完全不同——直接丢进 Hugo 会改变渲染行为,而不是多几个无害字段。
225
232
 
226
233
  | 键 | `obsidian`(默认) | `generic` | `minimal` |
227
234
  |---|---|---|---|
@@ -258,7 +265,7 @@ parsers: book, paper, video, web
258
265
 
259
266
  `options.transcribe`:**默认 `true`**(无字幕自动转写);设为 `false` 则跳过转写(只用字幕/desc 或降级)。
260
267
 
261
- **转写工具链(本机/用户一致)**:脚本**自举**——用 `uv` 建 **Python 3.12** 环境 + **清华镜像**装 `faster-whisper` + `HF_ENDPOINT=https://hf-mirror.com` 下模型并**缓存复用**;faster-whisper 内置 PyAV 解码音频,**无需单独 ffmpeg**;**版本已锁定**(Python 3.12 / faster-whisper / uv),保证本机与用户环境一致。脚本是 **Node** 写的,**Windows / macOS / Linux 同一份实现、同一条命令**(`node scripts/transcribe.js --audio <音频> --out <txt>`);转写默认走 **CPU**(`options.device` 默认 `cpu`),GPU 需自备 CUDA 运行库(cuBLAS/cuDNN),可设 `options.device: "auto"` 尝试,失败会自动回退 CPU。缓存目录按平台取:Windows `%LOCALAPPDATA%\deep-read-summarize`、macOS `~/Library/Caches/deep-read-summarize`、Linux `$XDG_CACHE_HOME/deep-read-summarize`。 装 Python 时默认走镜像加速,**镜像失败会自动改用直连 GitHub 重试**(镜像挂掉不该让用户彻底装不上);可用环境变量 `DRS_UV_PYTHON_MIRROR` 覆盖(设为空字符串 = 完全不用镜像)。
268
+ **转写工具链**:脚本用 `uv` 创建 **Python 3.12** 环境,从清华镜像安装 **`faster-whisper==1.2.1`**,通过 `HF_ENDPOINT=https://hf-mirror.com` 下载并缓存模型。健康的缓存环境直接运行,不要求 uv 仍在 PATH,也不再启动单独的 Python 导入探测。实际解释器和依赖版本记录在缓存目录的 `runtime.json`;Python 补丁版本、uv 和传递依赖未全部锁定,不能据此声称环境逐字一致。PyAV 可直接解码音频,无需单独 ffmpeg。三平台使用同一条命令 `node scripts/transcribe.js --audio <音频> --out <txt>`,默认 CPU;`options.device: "auto"` 可尝试自备的 CUDA,失败回退 CPU。缓存目录:Windows `%LOCALAPPDATA%\deep-read-summarize`、macOS `~/Library/Caches/deep-read-summarize`、Linux `$XDG_CACHE_HOME/deep-read-summarize`(未设置时用 `~/.cache`)。创建 Python 环境默认先用镜像,失败再直连 GitHub;`DRS_UV_PYTHON_MIRROR` 可覆盖镜像,空字符串表示直连。
262
269
 
263
270
  > ⚠️ **首次转写会先下载模型(small 约 484MB,走 hf-mirror,非 GitHub),可能需要几分钟**,请耐心等待;**本机只下这一次**,之后所有视频转写用缓存、秒开。**只有「无字幕」视频才会触发转写**(有字幕的视频直接用字幕,不触发、不下载)。
264
271
 
@@ -310,9 +317,21 @@ parsers: book, paper, video, web
310
317
 
311
318
  ---
312
319
 
320
+ ## 保持质量的提速
321
+
322
+ - 支持的新宿主优先使用 `deep_read_summarize`,模型只提交参数,省去复制整份工作流脚本。旧宿主自动保留原入口。
323
+ - 精读仍完整覆盖已知原文,最多三个任务并发;空闲槽位立即领取下一块。按行数均衡工作量,避免一个超大块拖住整批。
324
+ - QC 继续独立读取原文;不再在每轮上下文中重复塞入全部 Map 笔记。缺总行数、精读失败或只有视频简介等问题不能靠改写笔记解决,不再浪费全文修订轮次。
325
+ - 视频 `textSource: "desc"` 明确为降级来源,不能取得完整精读的质量通过状态。B站多 P 链接按 `p` 选择对应 CID、分 P 标题和时长。
326
+ - 返回 `timings` 提供各阶段毫秒耗时,用同一资料、模型、篇幅和质量门槛比较;调度测试不能当成真实 LLM 加速比例。
327
+
328
+ 本次未改小模型档位、减少字数门槛或抽样跳过原文。常驻 ASR 模型、逐字稿缓存和分层 QC 仍需单独做质量与性能基准。
329
+
313
330
  ## 成本
314
331
 
315
- 一篇论文、6 块、完整模式,大约 15-25k token。用 `fastMode` 并调低 `maxChunks` 能省四成左右。
332
+ Token 用量取决于原文长度、分块数和成稿篇幅。原文核验会再次读取资料,每轮修订还会增加一次修订和一次核验调用。`fastMode` 减少输出章节;调低 `maxChunks` 限制代理数,同时保留全文覆盖。
333
+
334
+ 使用 `lib/cache` 的程序化调用方请注意:URL 尾斜杠、空 query 与 fragment 可能选择不同资料,现在会保留;新版指纹自动与旧版缓存隔离,升级后可能需要重新处理一次。该工具不负责证明远程内容未更新,复用前仍须由调用方确认来源与配置。
316
335
 
317
336
  ---
318
337
 
package/cordis.patch.yml CHANGED
@@ -14,6 +14,7 @@
14
14
  # = warning + ignore, never a startup failure) and folds them into the default
15
15
  # args carried by the skill, so they become this profile's defaults.
16
16
  # config:
17
+ # customParsersDir: './custom-parsers' # optional plugin-level parser directory (not a workflow option)
17
18
  # outputDir: './output' # where notes are written (default: ./output)
18
19
  # tempDir: './.tmp' # temp files for chunked content (default: ./.tmp)
19
20
  # minWords: 2500 # minimum output length (default: 2500)
@@ -33,4 +34,4 @@
33
34
  # Idempotency is deliberately NOT a plugin config key: the workflow script runs
34
35
  # without a filesystem, so it cannot write a fingerprint record. Programmatic
35
36
  # callers can use `deep-read-summarize/lib/cache`
36
- # (fingerprint / hasProcessed / markProcessed).
37
+ # (fingerprint / hasProcessed / markProcessed).
package/docs/RELEASE.md CHANGED
@@ -138,7 +138,9 @@ npm view deep-read-summarize --prefer-online # 验证线
138
138
  | v0.3.15 | npm 已发布 | 工程卫生:锁定 LF、脚本正文独立成文、lint 扩检查、包瘦身、提示词改「标准 Markdown」 |
139
139
  | v0.3.16 | npm 已发布 | 新增 `options.frontmatter` 三档(obsidian / generic / minimal),解决 Hugo 保留键冲突;默认档不变 |
140
140
  | v0.3.17 | npm 已发布 | 修 `resolve()` 原型链回退与质量清单 required;新增发布脚本 `scripts/release.js` |
141
- | **v0.3.18** | **npm 已发布(当前)** | 修复自动解析器分流、并发临时文件串稿和分块上限导致的全文遗漏 |
141
+ | v0.3.18 | npm 已发布 | 修复自动解析器分流、并发临时文件串稿和分块上限导致的全文遗漏 |
142
+ | v0.3.19 | npm 已发布 | 修复 B04–B10,并让质量检查结合程序门禁和可回查的原文证据(O01)。 |
143
+ | **v0.4.0** | **npm 已发布(当前)** | 保留完整精读与独立原文核验,减少脚本搬运、并发空等和无效修订,并修复来源质量判断与视频分P等问题。 |
142
144
  | v1.0.0 | 计划 | API 稳定后发布 |
143
145
 
144
146
  > 早期迭代用 0.x 表达不稳定;API 稳定后再发 1.0.0。
package/index.js CHANGED
@@ -20,73 +20,37 @@ const path = require('node:path');
20
20
  const workflow = require('./workflow');
21
21
  const parsers = require('./parsers');
22
22
  const schemas = require('./schemas');
23
+ const { DEFAULT_OPTIONS, normalizeWorkflowOptions } = require('./lib/options');
23
24
 
24
25
  module.exports.name = 'deep-read-summarize';
25
26
  module.exports.inject = ['skills'];
26
27
 
27
- /** workflow 的 options 默认值(与 workflow 脚本内的默认值保持一致)。 */
28
- const DEFAULT_OPTIONS = {
29
- minWords: 2500,
30
- fastMode: false,
31
- maxChunks: 4,
32
- transcribe: true,
33
- requireCitations: true,
34
- includeTimestamps: false,
35
- outputDir: './output',
36
- tempDir: './.tmp',
37
- whisperModel: 'small',
38
- language: 'zh',
39
- device: 'cpu',
40
- maxRetries: 0,
41
- frontmatter: 'obsidian'
42
- };
43
-
44
- const OPTION_TYPES = {
45
- minWords: 'number',
46
- fastMode: 'boolean',
47
- maxChunks: 'number',
48
- transcribe: 'boolean',
49
- requireCitations: 'boolean',
50
- includeTimestamps: 'boolean',
51
- outputDir: 'string',
52
- tempDir: 'string',
53
- whisperModel: 'string',
54
- language: 'string',
55
- device: 'string',
56
- maxRetries: 'number',
57
- frontmatter: 'string'
58
- };
59
-
60
28
  /**
61
29
  * 归一化插件 config:只认白名单里的键,类型不符就告警并忽略(不抛错,
62
30
  * 免得一个拼错的配置项把整个 profile 拖死)。
63
31
  * 支持 `config: { outputDir: ... }` 与 `config: { options: { outputDir: ... } }` 两种写法。
64
32
  */
65
33
  function normalizeOptions(config) {
66
- const opts = { ...DEFAULT_OPTIONS };
67
- const raw = (config && config.options) || config || {};
68
- for (const key of Object.keys(DEFAULT_OPTIONS)) {
69
- const v = raw[key];
70
- if (v === undefined) continue;
71
- if (typeof v !== OPTION_TYPES[key]) {
72
- console.warn('[deep-read-summarize] ignoring config.' + key + ' — expected ' + OPTION_TYPES[key] + ', got ' + typeof v);
73
- continue;
74
- }
75
- opts[key] = v;
34
+ let raw = config && Object.prototype.hasOwnProperty.call(config, 'options') ? config.options : config;
35
+ if (raw && typeof raw === 'object' && !Array.isArray(raw) && raw === config) {
36
+ raw = Object.assign({}, raw);
37
+ delete raw.customParsersDir; // plugin-only executable extension directory
76
38
  }
77
- // 未知键此前是完全无声的:循环只遍历白名单,写错的键名既不生效也不告警。
78
- const unknown = Object.keys(raw).filter(function (k) {
79
- return !Object.prototype.hasOwnProperty.call(DEFAULT_OPTIONS, k);
80
- });
81
- if (unknown.length) {
82
- console.warn('[deep-read-summarize] ignoring unknown config key(s): ' + unknown.join(', '));
83
- console.warn('[deep-read-summarize] allowed keys: ' + Object.keys(DEFAULT_OPTIONS).join(', '));
39
+ return normalizeWorkflowOptions(raw, { mode: 'config', warn: function (message) { console.warn(message); } });
40
+ }
41
+
42
+ function configuredRegistry(config) {
43
+ if (!config || !Object.prototype.hasOwnProperty.call(config, 'customParsersDir')) return parsers.registry;
44
+ const dir = config.customParsersDir;
45
+ if (typeof dir !== 'string' || !dir.trim() || /[\u0000\r\n]/.test(dir)) {
46
+ console.warn('[deep-read-summarize] ignoring config.customParsersDir — expected a non-empty directory path');
47
+ return parsers.registry;
84
48
  }
85
- return opts;
49
+ return new parsers.ParserRegistry({ customParsersDir: dir });
86
50
  }
87
51
 
88
52
  /** 组装技能内容:SKILL.md + workflow meta/script/args 示例(script 为自包含版本)。 */
89
- function buildSkillContent(config = {}) {
53
+ function buildSkillContent(config = {}, runtime = {}) {
90
54
  const skillDir = path.join(__dirname, 'skills', 'deep-read-summarize');
91
55
  const body = readFileSync(path.join(skillDir, 'SKILL.md'), 'utf8');
92
56
  const argsExample = {
@@ -94,6 +58,12 @@ function buildSkillContent(config = {}) {
94
58
  type: 'auto',
95
59
  options: normalizeOptions(config)
96
60
  };
61
+ if (runtime.toolName) {
62
+ return [body.trim(), '', '## 调用精读工具',
63
+ '调用 `' + runtime.toolName + '`,只提交 input / type / options。工具返回完整 note 与建议 filePath;必须按上文执行同名文件询问、落盘和质量报告。',
64
+ 'args 示例(options 是本 profile 的默认值,用户另有要求时再覆盖):',
65
+ '```json', JSON.stringify(argsExample, null, 2), '```'].join('\n');
66
+ }
97
67
  return [
98
68
  body.trim(),
99
69
  '',
@@ -107,7 +77,7 @@ function buildSkillContent(config = {}) {
107
77
  '',
108
78
  'script(完整脚本,原样填入 script 参数;脚本已自包含解析器,无需宿主注入):',
109
79
  '```javascript',
110
- workflow.script,
80
+ workflow.buildSelfContainedScript(runtime.registry || configuredRegistry(config)),
111
81
  '```',
112
82
  '',
113
83
  'args 示例(options 是本 profile 的默认值,用户另有要求时再覆盖):',
@@ -117,21 +87,131 @@ function buildSkillContent(config = {}) {
117
87
  ].join('\n');
118
88
  }
119
89
 
120
- /** Cordis 插件入口:注册运行时技能,使模型可直接执行精读 workflow。 */
121
- function apply(ctx, config = {}) {
90
+ // This is the public DSH WorkflowEngine.start / ToolDefinition contract used
91
+ // by dsh-tool-workflow itself. Optional services keep older profiles working.
92
+ function optionalService(ctx, name) {
93
+ if (typeof ctx.get === 'function') return ctx.get(name);
94
+ return ctx[name];
95
+ }
96
+
97
+ function readingTool(engine, defaults, registry) {
98
+ return {
99
+ name: 'deep_read_summarize',
100
+ description: '深度精读书籍、论文、视频或网页,返回完整 Markdown 笔记、建议保存路径及原文质量核验结果。拿到结果后必须检查同名文件并保存 note;质量未通过或有内容缺口时必须报告。',
101
+ parameters: {
102
+ type: 'object', additionalProperties: false,
103
+ properties: {
104
+ input: { type: 'string', description: '内容链接或本地文件路径' },
105
+ type: { type: 'string', description: 'auto 或已注册的解析器类型,默认 auto' },
106
+ options: { type: 'object', additionalProperties: true, description: '覆盖当前 profile 的精读选项' }
107
+ }, required: ['input']
108
+ },
109
+ output: {
110
+ schema: { type: 'object', additionalProperties: true },
111
+ // Never clip the note: the main agent needs its complete value to save it.
112
+ render: function (args, value) { return [{ type: 'text', text: JSON.stringify(value) }]; }
113
+ },
114
+ async execute(args, exec) {
115
+ if (!args || typeof args !== 'object' || Array.isArray(args)) throw new Error('[FATAL] expected an input object');
116
+ if (Object.keys(args).some(function (key) { return ['input', 'type', 'options'].indexOf(key) < 0; })) throw new Error('[FATAL] unknown deep_read_summarize argument');
117
+ if (typeof args.input !== 'string' || !args.input.trim()) throw new Error('[FATAL] args.input 必须是非空字符串');
118
+ if (args.type !== undefined && typeof args.type !== 'string') throw new Error('[FATAL] args.type 必须是字符串');
119
+ if (args.options !== undefined && (!args.options || typeof args.options !== 'object' || Array.isArray(args.options))) throw new Error('[FATAL] options 必须是对象');
120
+ const options = normalizeWorkflowOptions(Object.assign({}, defaults, args.options));
121
+ if (!exec || !exec.agent) throw new Error('deep_read_summarize requires a calling agent');
122
+ if (exec.signal && exec.signal.aborted) throw new Error('deep_read_summarize was cancelled before start');
123
+ const run = engine.start({ script: workflow.buildSelfContainedScript(registry), meta: workflow.meta,
124
+ args: { input: args.input, type: args.type || 'auto', options: options }, parent: exec.agent, signal: exec.signal });
125
+ try {
126
+ const result = await run.result;
127
+ if (result.stopReason !== 'completed') throw new Error('deep_read_summarize ' + result.stopReason + ': ' + (result.error || 'workflow did not complete'));
128
+ if (!result.value || typeof result.value !== 'object' || Array.isArray(result.value)) throw new Error('deep_read_summarize returned an invalid workflow result');
129
+ return result.value;
130
+ } finally { await run.dispose(); }
131
+ }
132
+ };
133
+ }
134
+
135
+ function skillDefinition(config, runtime) {
122
136
  const skillDir = path.join(__dirname, 'skills', 'deep-read-summarize');
123
- return ctx.skills.register({
137
+ return {
124
138
  name: 'deep-read-summarize',
125
139
  description: workflow.meta.description,
126
140
  whenToUse: workflow.meta.whenToUse,
127
- content: buildSkillContent(config),
141
+ content: buildSkillContent(config, runtime),
128
142
  resourceBase: { kind: 'directory', path: skillDir },
129
- // source 是技能的「来源桶」(runtime / bundled / user-dsh …),不是自由描述:
130
- // 本技能随插件包分发,因此登记为 bundled。
131
143
  source: 'bundled'
144
+ };
145
+ }
146
+
147
+ // Cordis inject() owns a child fiber which activates/restarts as these optional
148
+ // services appear/disappear. Keep a fallback skill while that child is pending.
149
+ function applyWithServiceLifecycle(ctx, config, registry) {
150
+ let stopped = false;
151
+ let releaseSkill = ctx.skills.register(skillDefinition(config, { registry: registry }));
152
+ function replaceSkill(toolName) {
153
+ if (typeof releaseSkill === 'function') releaseSkill();
154
+ releaseSkill = undefined;
155
+ releaseSkill = ctx.skills.register(skillDefinition(config, { registry: registry, toolName: toolName }));
156
+ }
157
+ function parentCanRegister() {
158
+ // Public Cordis FiberState: LOADING=1, ACTIVE=2; never restore while unloading.
159
+ return !stopped && (!ctx.fiber || ctx.fiber.state === 1 || ctx.fiber.state === 2);
160
+ }
161
+ // Cordis treats callbacks with a prototype as constructors. An arrow keeps
162
+ // the returned cleanup function in its effect lifecycle.
163
+ const child = ctx.inject(['tools', 'workflowEngine'], (ready) => {
164
+ const tools = optionalService(ready, 'tools');
165
+ const engine = optionalService(ready, 'workflowEngine');
166
+ const releaseTool = tools.register(readingTool(engine, normalizeOptions(config), registry));
167
+ try { replaceSkill('deep_read_summarize'); }
168
+ catch (error) {
169
+ if (typeof releaseTool === 'function') releaseTool();
170
+ if (parentCanRegister()) replaceSkill();
171
+ throw error;
172
+ }
173
+ return function () {
174
+ try {
175
+ if (typeof releaseSkill === 'function') releaseSkill();
176
+ releaseSkill = undefined;
177
+ } finally {
178
+ if (typeof releaseTool === 'function') releaseTool();
179
+ if (parentCanRegister()) replaceSkill();
180
+ }
181
+ };
132
182
  });
183
+ return async function () {
184
+ stopped = true;
185
+ try { if (child && typeof child.dispose === 'function') await child.dispose(); }
186
+ finally {
187
+ if (typeof releaseSkill === 'function') releaseSkill();
188
+ releaseSkill = undefined;
189
+ }
190
+ };
133
191
  }
134
192
 
193
+ /** Cordis 插件入口:注册运行时技能,使模型可直接执行精读 workflow。 */
194
+ const apply = (ctx, config = {}) => {
195
+ const registry = configuredRegistry(config);
196
+ if (typeof ctx.inject === 'function') return applyWithServiceLifecycle(ctx, config, registry);
197
+ const tools = optionalService(ctx, 'tools');
198
+ const engine = optionalService(ctx, 'workflowEngine');
199
+ const canRunDirectly = tools && typeof tools.register === 'function' && engine && typeof engine.start === 'function';
200
+ const unregisterTool = canRunDirectly ? tools.register(readingTool(engine, normalizeOptions(config), registry)) : undefined;
201
+ let unregisterSkill;
202
+ try { unregisterSkill = ctx.skills.register(skillDefinition(config, {
203
+ registry: registry, toolName: canRunDirectly ? 'deep_read_summarize' : undefined
204
+ })); } catch (error) {
205
+ if (typeof unregisterTool === 'function') unregisterTool();
206
+ throw error;
207
+ }
208
+ if (!canRunDirectly) return unregisterSkill;
209
+ return function () {
210
+ try { if (typeof unregisterSkill === 'function') unregisterSkill(); }
211
+ finally { if (typeof unregisterTool === 'function') unregisterTool(); }
212
+ };
213
+ };
214
+
135
215
  // 兼容旧式加载(部分工具按 load() 读取插件定义)
136
216
  function load() {
137
217
  return { name: 'deep-read-summarize', workflow, parsers, schemas };
@@ -145,3 +225,4 @@ module.exports.schemas = schemas;
145
225
  module.exports.buildSkillContent = buildSkillContent;
146
226
  module.exports.DEFAULT_OPTIONS = DEFAULT_OPTIONS;
147
227
  module.exports.normalizeOptions = normalizeOptions;
228
+ module.exports.normalizeWorkflowOptions = normalizeWorkflowOptions;