deep-read-summarize 0.3.6 → 0.3.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -3,6 +3,32 @@
3
3
  All notable changes to this project will be documented in this file.
4
4
  The format is based on [Keep a Changelog](https://keepachangelog.com/).
5
5
 
6
+ ## [0.3.7] — 2026-09-09
7
+
8
+ 针对 **DSH 0.1.2-rc.1** 的一轮兼容性核查与打磨(对使用者无破坏性变更)。
9
+
10
+ ### Fixed
11
+
12
+ - **笔记落盘步骤缺失(最容易丢数据的一条)**:SKILL.md 写「默认写到 `./output/`」,但 DSH 的 workflow 契约规定脚本只能协调子代理——**沙箱内没有文件系统**,脚本实际只返回 `{ filePath, note }`。照原文档执行,模型会以为文件已经写好,笔记只存在于返回值里。现在 SKILL.md 新增明确的第 8 步「用 `write` 工具把 `note` 写入 `filePath`」,两版 README 同步说明。
13
+ - **`npm run test:node` 在 Node 24 上直接失败**:`node --test tests/` 在该版本会把 `tests/` 当模块解析(`MODULE_NOT_FOUND`)。改为 `node --test`(自动发现)。
14
+ - **CI 从不运行 `tests/index.test.js`**:里面正是 0.3.3 事故(入口缺 `apply()` → 宿主拒绝启动)的 Cordis 契约回归测试,却因为只在 `node --test` 下跑而从未被执行。CI 新增该步骤,矩阵补上 **Node 24**(宿主实际运行版本)。
15
+
16
+ ### Changed
17
+
18
+ - **npm 包不再携带 docs 截图**:`files` 由整个 `docs/` 收窄为两份 RELEASE 文档。0.3.6 发布之后仓库才提交的 4 张 showcase 图(合计约 1.14MB)会让下次发版的 tarball 从 99KB 涨到 1.1MB;现在回到约 100KB。图片仍随仓库分发,README 改用 GitHub 绝对地址引用(npm 页面照常显示),顺带把一直没有被任何文档引用的演示图补进 README。
19
+ - **技能 `source` 改为 `bundled`**:该字段是技能的「来源桶」(`runtime`/`bundled`/`user-dsh`…),此前填的是自由描述 `plugin:deep-read-summarize`。
20
+ - README/README.en.md 的「兼容性」节改写为:已验证宿主(DSH 0.1.2-rc.1)+ 依赖面表格 + 两条契约细节(沙箱无文件系统;违反 schema 子集是**终止**而非降级)。
21
+ - `.gitignore` 增加 `*.tgz`(本地 pack 产物不再出现在 `git status`)。
22
+
23
+ ### Added
24
+
25
+ - **workflow 契约回归测试(零依赖)**:schema 子集递归校验(含反向断言,确保校验器不会空放行)、`agent()` 选项白名单、沙箱禁用 API(`require`/`fs`/`process`/`fetch`/定时器)扫描、脚本四解析器自包含、技能内容包含 meta + script + args 与落盘步骤、插件 Cordis 契约 + 技能登记行为。测试 28 → **35**。
26
+
27
+ ### Compatibility
28
+
29
+ - 实测证据(0.1.2-rc.1):两个 `agent()` schema 用宿主真实的 `assertObjectJsonSchema` 校验**通过**,对照用例 `minLength` 被拒(证明校验器有效、不是空放行);脚本只使用 `agent/parallel/phase/log/args`;`agent()` 失败返回 `null` 已被脚本正确处理。
30
+ - 未改动 workflow 脚本本身与解析器行为;本次修复集中在「落盘契约、测试与 CI、打包体积、文档准确性」。
31
+
6
32
  ## [0.3.6] — 2026-08-28
7
33
 
8
34
  ### Added
package/README.en.md ADDED
@@ -0,0 +1,310 @@
1
+ # deep-read-summarize
2
+
3
+ [![MIT license](https://img.shields.io/github/license/PensiveFei/deep-read-summarize)](https://github.com/PensiveFei/deep-read-summarize/blob/main/LICENSE)
4
+ [![release](https://img.shields.io/github/v/release/PensiveFei/deep-read-summarize)](https://github.com/PensiveFei/deep-read-summarize/releases)
5
+ [![CI](https://img.shields.io/github/actions/workflow/status/PensiveFei/deep-read-summarize/ci.yml)](https://github.com/PensiveFei/deep-read-summarize/actions/workflows/ci.yml)
6
+ [![npm downloads](https://badgen.net/npm/dw/deep-read-summarize)](https://www.npmjs.com/package/deep-read-summarize)
7
+
8
+ > **Disclaimer**: This is an **unofficial third-party tool**. It is not affiliated with, endorsed by, or sponsored by DeepSeek or the DeepSeek Harness project. "DeepSeek" and "DeepSeek Harness" are trademarks of their respective owners. This project only orchestrates the workflow tool available in your DSH environment; it does not redistribute any DeepSeek software.
9
+
10
+ > **⚠️ Most importantly: this is an "assisted deep-reading" tool, not a substitute for your own learning.** It helps you quickly extract, summarize, and structure what you read, but **real understanding, thinking, and critique must still come from you**. Always treat the original source as authoritative — do not mistake the generated notes for the learning itself.
11
+
12
+ A deep-reading workflow for DSH (DeepSeek Harness): feed it a book, a paper, a video link, or a web page, and it produces a structured Obsidian note.
13
+
14
+ Content is split into chunks, read in parallel by multiple sub-agents, then merged into a Markdown file with YAML frontmatter. Key conclusions carry citations to the original source, and a quality check runs before the final draft.
15
+
16
+ > If you run into problems, or see how it could be better, **feel free to open a GitHub issue**.
17
+
18
+ > Note: DSH is currently a developer preview and its interfaces may change. This repo targets a specific version of the workflow-tool semantics — see [Compatibility](#compatibility).
19
+
20
+ ---
21
+
22
+ ## What it looks like
23
+
24
+ (Images use absolute GitHub URLs so they render on npm too; the PNGs ship with the repository, not inside the npm package.)
25
+
26
+ | Deep-read paper note | Video note metadata |
27
+ | --- | --- |
28
+ | ![Paper note](https://raw.githubusercontent.com/PensiveFei/deep-read-summarize/main/docs/showcase/showcase-paper-note-cot.png) | ![Video note metadata](https://raw.githubusercontent.com/PensiveFei/deep-read-summarize/main/docs/showcase/showcase-video-note-metadata.png) |
29
+
30
+ | Video note action items | In-chat digest |
31
+ | --- | --- |
32
+ | ![Video note action items](https://raw.githubusercontent.com/PensiveFei/deep-read-summarize/main/docs/showcase/showcase-video-note-actions.png) | ![In-chat digest](https://raw.githubusercontent.com/PensiveFei/deep-read-summarize/main/docs/showcase/showcase-in-chat-digest.png) |
33
+
34
+ ---
35
+
36
+ ## What it does
37
+
38
+ - Books (PDF/EPUB/MOBI), papers (arXiv/PDF/HTML), videos (**subtitles first, auto-transcription when there are none** — produces a full transcript for deep reading), web pages
39
+ - Long content is chunked and read in parallel by sub-agents, then merged
40
+ - Sub-task outputs are constrained by JSON Schema; invalid outputs are retried automatically
41
+ - Key citations must carry page/chapter/paragraph references to reduce fabrication
42
+ - Configuration errors abort immediately; a chunk that fails to parse is skipped and marked as a gap
43
+ - Output drops straight into Obsidian and works with Dataview
44
+
45
+ ---
46
+
47
+ ## Structure
48
+
49
+ ```
50
+ Input (link or file path)
51
+ │
52
+ ▼
53
+ Parser registry ── book / paper / video / web
54
+ │
55
+ ▼
56
+ Wave 1 Fetch content → write temp file → build chunk plan
57
+ │
58
+ ▼
59
+ Wave 2 N sub-agents deep-read chunks in parallel (Map)
60
+ │
61
+ ▼
62
+ Wave 3 Merge draft + quality check (Reduce)
63
+ │
64
+ ▼
65
+ Obsidian note
66
+ ```
67
+
68
+ Three waves, roughly N+2 sub-agents in total, where N is the number of chunks.
69
+
70
+ ```
71
+ parsers/ Parsers for each input type, dispatched by type
72
+ book.js Books: PDF/EPUB/MOBI text extraction, chapter chunking
73
+ paper.js Papers: arXiv/PDF/HTML structure detection
74
+ video.js Video: full transcript (subtitles first; auto-transcribe via scripts/transcribe.ps1 when none)
75
+ web.js Web pages: main-text extraction
76
+ index.js Registry: parser discovery and fallback
77
+ schemas/ JSON Schema for sub-task outputs
78
+ scripts/ lint, security checks
79
+ tests/ fixture tests and validation scripts
80
+ workflow.js The workflow script itself (meta + script)
81
+ ```
82
+
83
+ To change how an input type is handled, drop a parser with the same interface into `custom-parsers/`; a type with the same name overrides the built-in implementation. The interface has only three fields: `name`, `types`, `buildPrompt(input, opts)`.
84
+
85
+ ---
86
+
87
+ ## dsh.so ecosystem
88
+
89
+ This repo carries the `dsh-plugin` topic and the `dsh` field in `package.json`, so it can be indexed by the dsh.so registry.
90
+
91
+ dsh.so listing requirements: a public repo, the `dsh-plugin` topic (or the dsh field), README install instructions, and an SPDX license (MIT).
92
+ After submission it is marked **Declared** (self-declared compatibility); if you actually use it and verify compatibility, report your results in the official Discussions — that is the only way to be promoted to **Verified**.
93
+
94
+ ---
95
+ ## Installation (DSH plugin)
96
+
97
+ deep-read-summarize is a DSH plugin installable via npm, the dsh.so ecosystem, or locally:
98
+
99
+ ```bash
100
+ # npm install (published to the registry, no third-party dependencies)
101
+ npm install deep-read-summarize
102
+
103
+ # or local install (dsh profile directory)
104
+ pnpm add ./deep-read-summarize-0.3.6.tgz
105
+ # then append to the dsh config's dsh.profile.bundles:
106
+ # - deep-read-summarize
107
+ # restart dsh web (POST /dsh-market/restart)
108
+ ```
109
+
110
+ After installation it registers:
111
+ - the `deep-read-summarize` workflow (meta + script)
112
+ - the `deep-read-summarize` skill (`skills/deep-read-summarize/SKILL.md`)
113
+ - four parsers (`parsers/`) and JSON Schemas (`schemas/`)
114
+
115
+ ---
116
+ ## Quick start (~5 minutes)
117
+
118
+ ### 1. Install
119
+
120
+ ```bash
121
+ # Option A: npm install directly (no clone needed)
122
+ npm install deep-read-summarize
123
+
124
+ # Option B: develop from source
125
+ git clone https://github.com/<your-org>/deep-read-summarize.git
126
+ cd deep-read-summarize
127
+ npm install # no third-party dependencies, just initialization
128
+ ```
129
+
130
+ ### 2. Verify the environment (offline, no API key needed)
131
+
132
+ ```bash
133
+ npm test # 25 fixture tests, all run offline
134
+ ```
135
+
136
+ Seeing `TOTAL: 25 passed, 0 failed` means the environment is ready.
137
+
138
+ ### 3. Feed a demo file with one command
139
+
140
+ ```bash
141
+ node -e "const wf = require('./workflow.js'); console.log('meta:', wf.meta.name); console.log('parsers:', wf.parsers.list().map(p => p.name).join(', '));"
142
+ ```
143
+
144
+ Output looks like:
145
+
146
+ ```
147
+ meta: deep-read-summarize
148
+ parsers: book, paper, video, web
149
+ ```
150
+
151
+ ### 4. Real run (requires the DSH workflow tool)
152
+
153
+ Pass the JSON below to DSH's workflow tool (see Usage):
154
+
155
+ ```jsonc
156
+ { "input": "https://arxiv.org/abs/2307.09042", "type": "paper", "options": { "maxChunks": 4, "fastMode": true } }
157
+ ```
158
+
159
+ The workflow returns a structured result: `{ ok, kind, title, filePath, qualityPassed, note }`, where `note` is the final Markdown note.
160
+
161
+ > ⚠️ **The caller persists the note.** DSH's workflow contract gives the script no filesystem access — it only coordinates subagents — so the script returns the note body plus a suggested `filePath` and never writes the file itself. The bundled SKILL.md tells the model to write `note` to `filePath` with the `write` tool after the run; do the same when you invoke the workflow by hand, or the note exists only in the return value.
162
+
163
+ ---
164
+ ## Usage
165
+
166
+ Pass this JSON to DSH's workflow tool:
167
+
168
+ ```jsonc
169
+ {
170
+ "input": "https://arxiv.org/abs/2307.09042", // link or file path
171
+ "type": "auto", // auto | book | paper | video | web
172
+ "options": {
173
+ "minWords": 2500,
174
+ "fastMode": false, // true skips sections 5-7, faster
175
+ "maxChunks": 4, // chunk limit, 1-12 (default lowered to control sub-agent count and time)
176
+ "transcribe": true, // video: auto local transcription (faster-whisper) when no subtitles; false skips it — see Video
177
+ "requireCitations": true, // whether key conclusions must carry citations
178
+ "includeTimestamps": false,
179
+ "outputDir": "./output", // note output directory (can point at your Obsidian vault)
180
+ "tempDir": "./.tmp" // temp file directory
181
+ }
182
+ }
183
+ ```
184
+
185
+ Examples:
186
+
187
+ | Type | input |
188
+ |------|-------|
189
+ | Paper | `https://arxiv.org/abs/2307.09042` |
190
+ | Book | local path, e.g. `~/books/xxx.pdf` |
191
+ | Video | `https://youtube.com/watch?v=xxx` |
192
+ | Web page | `https://example.com/article` |
193
+
194
+ Notes are written to `options.outputDir` (default `./output`, can point at your Obsidian vault), with the filename taken from the content title.
195
+
196
+ ### Video: full transcript (subtitles first, auto-transcription when none)
197
+
198
+ Video text extraction is a **unified pipeline** (the old "three tiers" are gone): the goal is to get the **full transcript**, then deep-read it.
199
+
200
+ > **⚠️ To preserve deep-reading quality, processing may take noticeably longer (a deliberate trade-off, not a hang)**: a **subtitle-less** video is transcribed first (roughly 0.5–2× the video length on CPU) before deep reading, so such videos are **much slower than subtitled videos**; in exchange you get a **full transcript + real deep reading**. **Subtitled** videos remain seconds-fast. Please do not mistake the slower transcription for a hang.
201
+
202
+ 1. **Platform subtitles available** (Bilibili's official API AI subtitles / YouTube CC) → **use them directly** (= full text, fastest, zero dependencies).
203
+ 2. **No public subtitles** → use the bundled transcription `scripts/transcribe.ps1` (faster-whisper small / int8 / VAD / Chinese) to get the full text.
204
+ 3. **Neither works** → **degrade gracefully**: prompt the user to supply a transcript manually, or fall back to `desc` as background; **never block**.
205
+
206
+ `options.transcribe`: **default `true`** (auto-transcribe when no subtitles); set `false` to skip transcription (use subtitles/desc or degrade).
207
+
208
+ **Transcription toolchain (identical for you and the author)**: the script **bootstraps itself** — it uses `uv` to create a **Python 3.12** environment + **Tsinghua mirror** to install `faster-whisper`, downloads the model via `HF_ENDPOINT=https://hf-mirror.com` and **caches it for reuse**; faster-whisper bundles PyAV to decode audio, so **no separate ffmpeg** is needed; **versions are pinned** (Python 3.12 / faster-whisper / uv) so the author's environment matches yours.
209
+
210
+ > ⚠️ **The first transcription downloads the model first (small ≈ 484 MB, via hf-mirror, not GitHub) and may take a few minutes** — please be patient; **it downloads only once** on your machine, then every later transcription uses the cache and starts instantly. **Only subtitle-less videos trigger transcription** (subtitled videos use their subtitles directly — no trigger, no download).
211
+
212
+ **Quality/speed**: `small` + `int8` quantization + `VAD` silence filtering gives usable Chinese quality and is CPU-friendly. **Subtitled videos are not transcribed (fast)**; subtitle-less videos are transcribed at roughly 0.5–2× the video length on CPU.
213
+
214
+ **Failure handling**: install/download failures **report an error and degrade** (prompt for a manual transcript), never silently and never blocking for long. Use `options.transcribe: false` to skip transcription.
215
+
216
+ **How to know what source was used**: source info is **not written into the note** (to keep it clean); it surfaces in: ① the run log's `[source] <value> -> <label>` line; ② the `textSource` field in the returned result (`subtitle / transcription / desc / manual`).
217
+
218
+ ---
219
+
220
+ ## Variance and influencing factors (why length and content may differ)
221
+
222
+ **Output length and content vary noticeably with the model and the input; this is normal.**
223
+
224
+ ### Model
225
+ - **ASR model tier**: `small` (default, usable for Chinese) → `base` (faster, slightly lower quality) → `medium` (better, slower). Adjust with `options.whisperModel`; download size and speed grow with the model.
226
+ - **Language/accent**: Chinese/English/mixed language, dialect, accent, stress → **different transcription accuracy and time**; for pure English video you can set `language: "en"`.
227
+ - **Deep-reading LLM**: uses the host DSH model; different context/reasoning models → **different quality, length, and speed**.
228
+ - **Non-determinism**: both the LLM and ASR are stochastic, so **re-running the same input may produce slightly different results** (content/length).
229
+
230
+ ### Input
231
+ - **Video**:
232
+ - Whether **public subtitles** exist is the biggest differentiator — public subtitles = seconds-fast; login-only AI subtitles or no subtitles = triggers transcription, **noticeably slower**.
233
+ - **Length**: longer video = longer transcription (≈ 0.5–2× the video length on CPU).
234
+ - **Audio quality/noise/low volume/pure BGM/no voice**: transcription is error-prone or **empty**; supply a transcript manually in that case.
235
+ - **Paper/book/web page**:
236
+ - Format: PDF/EPUB/MOBI/HTML/arXiv; **scanned OCR, broken PDFs, dense formulas/figures** → different extraction quality and time.
237
+ - **Term density/length** → different chunk count and deep-reading depth.
238
+ - **Chunking structure**: different types chunk differently (chapter/topic/paragraph), affecting **output structure** and coverage.
239
+
240
+ ### Hardware
241
+ - **GPU vs CPU**: GPU transcription can be an order of magnitude faster than CPU (`device='auto'` picks automatically). Subtitle-less videos are ≈ 0.5–2× the length on CPU.
242
+
243
+ ### Other
244
+ - **First run vs reuse**: the first run downloads the model (≈ 484 MB); later runs reuse the cache.
245
+ - **Token/cost**: chunk count, length, and citations drive token usage; `fastMode`/`maxChunks` save tokens.
246
+ - **Citation/fabrication**: transcription or OCR errors may be quoted as "original text"; **re-check key citations**.
247
+ - **Boundaries**: pure BGM, distorted audio → empty/poor transcription; no subtitles and transcription unavailable → degrade to a prompt for a manual transcript.
248
+
249
+ ---
250
+
251
+ ## How failures are handled
252
+
253
+ Two classes:
254
+
255
+ - **Configuration errors** (missing input, invalid type, malformed options): throw immediately and abort — no half-baked output.
256
+ - **Content problems** (fetch failed, a chunk's deep-read failed): return `{ ok: false, stage, fatal: false }` and let the caller decide. A failed chunk is skipped and marked as a gap in the draft; the whole run is not aborted.
257
+
258
+ ---
259
+
260
+ ## Cost
261
+
262
+ A paper, 6 chunks, full mode: roughly 15–25k tokens. `fastMode` with a lower `maxChunks` saves about 40%.
263
+
264
+ ---
265
+
266
+ ## Security & copyright
267
+
268
+ - Fetching videos/web pages makes external requests, executed by sub-agents under DSH's sandbox and approval policy; don't let a model run arbitrary scripts unattended.
269
+ - Sub-agents need appropriate permissions to write files; if permissions are insufficient, the workflow returns the content and the main agent handles persistence.
270
+ - **Local transcription**: video transcription (faster-whisper) runs **locally**; audio/subtitles **do not leave your machine**; network requests to the corresponding platforms are made only when fetching video metadata/subtitles/audio.
271
+ - This repo contains only the pipeline itself: workflow definitions, parser code, prompt templates, schemas. **It contains no extracted copyrighted content.** Test fixtures are self-written public-domain fables. When you process copyrighted material with this workflow, responsibility for the output is yours.
272
+
273
+ ---
274
+
275
+ ## Development
276
+
277
+ ```bash
278
+ npm install
279
+ npm run lint # node --check on all JS
280
+ npm test # fixture tests (25, all offline)
281
+ npm run validate # pre-release validation (including security checks)
282
+ ```
283
+
284
+ `npm publish` automatically runs `prepublishOnly` (tests + lint + security checks) first; any failure blocks the publish.
285
+
286
+ See CHANGELOG.md for changes and CONTRIBUTING.md for contribution guidelines.
287
+
288
+ ---
289
+
290
+ ## Compatibility
291
+
292
+ **Verified host: DSH 0.1.2-rc.1** (Node 24). The dependency surface is exactly two things:
293
+
294
+ | Surface | Current contract | How this repo uses it |
295
+ | --- | --- | --- |
296
+ | workflow script hooks | `agent()` / `pipeline()` / `parallel()` / `phase()` / `log()` / `args` | only `agent/parallel/phase/log/args`; `agent()` options limited to `label/phase/schema` (the engine accepts only `label/phase/schema/provider/model` and treats anything else as fatal) |
297
+ | JSON Schema subset | `type / properties / required / additionalProperties / items / enum / const / oneOf` plus annotation keys | both `agent()` schemas verified against the host's real `assertObjectJsonSchema` |
298
+
299
+ Two contract details that bite:
300
+
301
+ - **The script sandbox has no filesystem, network, timers or `require`** — it coordinates subagents; the caller persists the note (see ⚠️ above).
302
+ - **Breaking the schema subset terminates the run** rather than degrading: the engine raises `UNSUPPORTED_SCHEMA` and the whole workflow fails. `npm test` carries a regression for this, including a reverse assertion so the checker cannot pass vacuously.
303
+
304
+ After upgrading DSH, run `npm test` and `npm run test:node`; if something breaks, cross-reference CHANGELOG.md.
305
+
306
+ ---
307
+
308
+ ## License
309
+
310
+ MIT — see [LICENSE](LICENSE).
package/README.md CHANGED
@@ -3,7 +3,7 @@
3
3
  [![MIT license](https://img.shields.io/github/license/PensiveFei/deep-read-summarize)](https://github.com/PensiveFei/deep-read-summarize/blob/main/LICENSE)
4
4
  [![release](https://img.shields.io/github/v/release/PensiveFei/deep-read-summarize)](https://github.com/PensiveFei/deep-read-summarize/releases)
5
5
  [![CI](https://img.shields.io/github/actions/workflow/status/PensiveFei/deep-read-summarize/ci.yml)](https://github.com/PensiveFei/deep-read-summarize/actions/workflows/ci.yml)
6
- [![npm downloads](https://img.shields.io/npm/dw/deep-read-summarize)](https://www.npmjs.com/package/deep-read-summarize)
6
+ [![npm downloads](https://badgen.net/npm/dw/deep-read-summarize)](https://www.npmjs.com/package/deep-read-summarize)
7
7
 
8
8
  > **Disclaimer**: This is an **unofficial third-party tool**. It is not affiliated with, endorsed by, or sponsored by DeepSeek or the DeepSeek Harness project. "DeepSeek" and "DeepSeek Harness" are trademarks of their respective owners. This project only orchestrates the workflow tool available in your DSH environment; it does not redistribute any DeepSeek software.
9
9
 
@@ -19,6 +19,20 @@
19
19
 
20
20
  ---
21
21
 
22
+ ## 实际效果
23
+
24
+ (图用 GitHub 绝对地址,npm 页面同样能显示;图片本身不打进 npm 包,只随仓库分发)
25
+
26
+ | 精读论文 | 视频笔记(元信息) |
27
+ | --- | --- |
28
+ | ![论文精读笔记](https://raw.githubusercontent.com/PensiveFei/deep-read-summarize/main/docs/showcase/showcase-paper-note-cot.png) | ![视频笔记元信息](https://raw.githubusercontent.com/PensiveFei/deep-read-summarize/main/docs/showcase/showcase-video-note-metadata.png) |
29
+
30
+ | 视频笔记(行动要点) | 会话内摘要 |
31
+ | --- | --- |
32
+ | ![视频笔记行动要点](https://raw.githubusercontent.com/PensiveFei/deep-read-summarize/main/docs/showcase/showcase-video-note-actions.png) | ![会话内摘要](https://raw.githubusercontent.com/PensiveFei/deep-read-summarize/main/docs/showcase/showcase-in-chat-digest.png) |
33
+
34
+ ---
35
+
22
36
  ## 能做什么
23
37
 
24
38
  - 书籍(PDF/EPUB/MOBI)、论文(arXiv/PDF/HTML)、视频(**字幕优先,无字幕自动转写**,出品完整逐字稿精读)、网页
@@ -144,6 +158,8 @@ parsers: book, paper, video, web
144
158
 
145
159
  工作流会返回结构化结果:`{ ok, kind, title, filePath, qualityPassed, note }`,`note` 即最终 Markdown 笔记。
146
160
 
161
+ > ⚠️ **笔记由主代理落盘**:DSH 的 workflow 契约规定脚本只能协调子代理(沙箱内没有文件系统),所以脚本只返回 `note` 正文与建议路径 `filePath`,**不会自己写文件**。插件自带的 SKILL.md 已明确要求模型在拿到结果后用 `write` 工具写入 `filePath`——自己拼 workflow 调用时也要记得这一步,否则笔记只存在于返回值里。
162
+
147
163
  ---
148
164
  ## 用法
149
165
 
@@ -273,12 +289,19 @@ npm publish 会自动先跑 prepublishOnly(测试 + lint + 安全检查),
273
289
 
274
290
  ## 兼容性
275
291
 
276
- DSH 还在快速迭代,有过破坏性变更。本仓库的依赖面是:
292
+ **已验证的宿主:DSH 0.1.2-rc.1**(Node 24)。本仓库的依赖面只有两处:
293
+
294
+ | 依赖面 | 现行契约 | 本仓库的用法 |
295
+ | --- | --- | --- |
296
+ | workflow 脚本钩子 | `agent()` / `pipeline()` / `parallel()` / `phase()` / `log()` / `args` | 只用了 `agent/parallel/phase/log/args`;`agent()` 选项只用 `label/phase/schema`(引擎只接受 `label/phase/schema/provider/model`,其它选项会被判为 fatal) |
297
+ | JSON Schema 子集 | `type / properties / required / additionalProperties / items / enum / const / oneOf` + 注解键 | 两个 `agent()` schema 已用宿主真实的 `assertObjectJsonSchema` 实测通过 |
298
+
299
+ 两点容易踩的契约细节:
277
300
 
278
- - workflow 工具的 `agent()`、`parallel()`、`phase()`、`log()`、`args`
279
- - JSON Schema 子集:`type / properties / required / additionalProperties / items / enum / const / oneOf`
301
+ - **脚本沙箱没有文件系统/网络/定时器/require**——只能协调子代理;笔记落盘由主代理完成(见上文 ⚠️)。
302
+ - **违反上述子集不是降级而是终止**:引擎抛 `UNSUPPORTED_SCHEMA`,整条 workflow 直接失败。仓库的 `npm test` 里有对应回归(含反向断言,确保校验器不是空放行)。
280
303
 
281
- 升级 DSH 后先跑一遍 `npm test`。如果坏了,对照 CHANGELOG.md 里的版本记录排查。
304
+ 升级 DSH 后先跑 `npm test` + `npm run test:node`;坏了对照 CHANGELOG.md 排查。
282
305
 
283
306
  ---
284
307
 
package/index.js CHANGED
@@ -1,7 +1,7 @@
1
1
  // ============================================================================
2
2
  // deep-read-summarize — DSH plugin entry
3
3
  // ============================================================================
4
- // 标准 Cordis 插件(DSH 0.1.x / rc7 契约):导出 name + inject + apply(ctx)。
4
+ // 标准 Cordis 插件(DSH 0.1.x 契约,已在 0.1.2-rc.1 实测):导出 name + inject + apply(ctx)。
5
5
  // apply() 将插件自带的 SKILL.md 注册为运行时技能,并把 workflow 的
6
6
  // meta + 自包含 script + args 示例嵌入技能内容,使模型可直接用 workflow
7
7
  // 工具执行精读流程。
@@ -47,7 +47,7 @@ function buildSkillContent() {
47
47
  ].join('\n');
48
48
  }
49
49
 
50
- /** rc7 Cordis 插件入口:注册运行时技能,使模型可直接执行精读 workflow。 */
50
+ /** Cordis 插件入口:注册运行时技能,使模型可直接执行精读 workflow。 */
51
51
  function apply(ctx, config = {}) {
52
52
  const skillDir = path.join(__dirname, 'skills', 'deep-read-summarize');
53
53
  return ctx.skills.register({
@@ -56,7 +56,9 @@ function apply(ctx, config = {}) {
56
56
  whenToUse: workflow.meta.whenToUse,
57
57
  content: buildSkillContent(),
58
58
  resourceBase: { kind: 'directory', path: skillDir },
59
- source: 'plugin:deep-read-summarize'
59
+ // source 是技能的「来源桶」(runtime / bundled / user-dsh …),不是自由描述:
60
+ // 本技能随插件包分发,因此登记为 bundled。
61
+ source: 'bundled'
60
62
  });
61
63
  }
62
64
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "deep-read-summarize",
3
- "version": "0.3.6",
3
+ "version": "0.3.7",
4
4
  "description": "Deep reading & summarization for DSH: books/papers/videos/web → structured Obsidian notes. Plugin parsers, MapReduce deep-read, JSON Schema output, idempotent cache.",
5
5
  "type": "commonjs",
6
6
  "main": "index.js",
@@ -20,7 +20,8 @@
20
20
  "skills",
21
21
  "scripts",
22
22
  "tests",
23
- "docs",
23
+ "docs/RELEASE.md",
24
+ "docs/RELEASE-v0.1.0.md",
24
25
  "cordis.patch.yml",
25
26
  "CHANGELOG.md",
26
27
  "README.md",
@@ -33,7 +34,7 @@
33
34
  "lint": "node scripts/lint.js",
34
35
  "validate": "node tests/quick_validate.js && node scripts/security-check.js",
35
36
  "security": "node scripts/security-check.js",
36
- "test:node": "node --test tests/",
37
+ "test:node": "node --test",
37
38
  "prepublishOnly": "node tests/run-tests.js && node scripts/lint.js && node scripts/security-check.js"
38
39
  },
39
40
  "dsh": {
@@ -20,7 +20,7 @@ whenToUse: 用户提供一本书、论文(arXiv/PDF)、视频链接(YouTub
20
20
  ## 输入
21
21
 
22
22
  用户提供:内容链接或本地文件路径。
23
- 可选参数:`type`(auto/book/paper/video/web)、`options`(minWords/fastMode/maxChunks/requireCitations/includeTimestamps/outputDir)。
23
+ 可选参数:`type`(auto/book/paper/video/web)、`options`(minWords / fastMode / maxChunks / maxRetries / requireCitations / includeTimestamps / transcribe / cache / outputDir / tempDir)。
24
24
 
25
25
  ## 执行步骤
26
26
 
@@ -31,10 +31,13 @@ whenToUse: 用户提供一本书、论文(arXiv/PDF)、视频链接(YouTub
31
31
  5. **波次2 并行精读**:每个分块一个子代理深度精读(Map)
32
32
  6. **波次3 合并成稿**:整合为完整笔记(Reduce),内嵌质量自检
33
33
  7. **质量校验**:可选重试(maxRetries),检查覆盖度/引用真实性/术语一致/格式完整/篇幅/语言
34
+ 8. **落盘(必须,别跳过)**:workflow 工具只返回 `note`(笔记正文)与 `filePath`(建议路径)——**脚本本身没有文件系统权限**。拿到结果后要用 `write` 工具把 `note` 原样写入 `filePath`(目录不存在就先创建);不写就等于笔记丢了,用户只会看到一段返回文本
34
35
 
35
36
  ## 输出
36
37
 
37
- Markdown 笔记(默认写到 `./output/`,可用 `options.outputDir` 指向 Obsidian 仓库)
38
+ workflow 返回 `{ ok, filePath, note, qualityPassed, qualityIssues, failedChunks, textSource }`。
39
+
40
+ > ⚠️ **`note` 就是完整笔记正文,`filePath` 只是建议路径**:脚本运行在无文件系统的沙箱里(DSH 的 workflow 契约:脚本只负责协调子代理),**它不会替你写文件**。必须由你(主代理)在拿到结果后用 `write` 工具把它写到 `filePath`——默认 `./output/`,可用 `options.outputDir` 指向 Obsidian 仓库。
38
41
 
39
42
  ```markdown
40
43
  ---
@@ -69,6 +72,7 @@ status: 已完成
69
72
  - **质量/速度**:small/int8 + VAD 静音过滤,中文质量可用且 CPU 友好;有字幕视频不转写(快)。装不上/失败则降级,不静默。
70
73
  - YouTube 用 yt-dlp 抓 CC;未装时**绝不下载 exe 二进制**(GitHub 直连易卡死),改用 winget/pip;仍不可用则转写或降级。
71
74
  - 运行 yt-dlp 加 `--socket-timeout 15 --retries 3` 防超时。
75
+ - **平台限制**:无字幕自动转写依赖 `scripts/transcribe.ps1`(PowerShell + `uv` + Python 3.12),**目前仅在 Windows 可用**;macOS/Linux 请让用户提供转写文本,或用平台字幕/desc 降级。
72
76
  - 默认不标时间戳(`options.includeTimestamps` 可开)
73
77
 
74
78
  ## 参考
@@ -79,6 +79,34 @@ test('fixture fable.txt exists and is public-domain self-authored', () => {
79
79
  assert.ok(!fx.includes('Copyright ©'));
80
80
  });
81
81
 
82
+ // ---------- Test: plugin entry contract (Cordis) ----------
83
+ // Regression for 0.3.3: the entry exported only { load, workflow, parsers,
84
+ // schemas } - no apply method - so dsh rejected the plugin at startup
85
+ // ("invalid plugin, expect function or object with an \"apply\" method"),
86
+ // the web server exited immediately, and the launcher just sat waiting on a
87
+ // dead port. The entry must satisfy the Cordis contract, always.
88
+ const plugin = require('../index');
89
+
90
+ test('plugin entry satisfies the Cordis contract (function or object with apply)', () => {
91
+ const ok = typeof plugin === 'function' ||
92
+ (plugin && typeof plugin === 'object' && typeof plugin.apply === 'function');
93
+ assert.ok(ok, 'entry must be a function or an object with an apply method');
94
+ assert.ok(typeof plugin.apply === 'function', 'apply must be callable');
95
+ });
96
+
97
+ test('plugin entry carries name + inject for the bundle row', () => {
98
+ assert.ok(typeof plugin.name === 'string' && plugin.name !== '', 'name must be a non-empty string');
99
+ assert.ok(Array.isArray(plugin.inject) && plugin.inject.every((s) => typeof s === 'string'),
100
+ 'inject must be an array of strings');
101
+ });
102
+
103
+ test('plugin entry keeps the legacy load/workflow/parsers exports', () => {
104
+ assert.ok(typeof plugin.load === 'function', 'legacy load() export missing');
105
+ assert.ok(plugin.workflow && plugin.workflow.meta, 'workflow export missing');
106
+ assert.ok(plugin.parsers && typeof plugin.parsers.resolve === 'function', 'parsers export missing');
107
+ assert.ok(plugin.schemas, 'schemas export missing');
108
+ });
109
+
82
110
  // ---------- Test: security scanner (negative/positive samples) ----------
83
111
  const { scanDir } = require('../scripts/security-check');
84
112
  test('security scanner flags fake secrets and skips clean/ignored files', () => {
@@ -98,6 +126,140 @@ test('security scanner flags fake secrets and skips clean/ignored files', () =>
98
126
  }
99
127
  });
100
128
 
129
+ // ---------- Test: workflow script contract(DSH workflow 工具)----------
130
+ // 现行契约(0.1.2-rc.1):agent/parallel/pipeline/phase/log/args 六个钩子;agent() 选项只允许
131
+ // label/phase/schema/provider/model(其它选项会被引擎判为 fatal);schema 只允许
132
+ // type/properties/required/additionalProperties/items/enum/const/oneOf + 注解键;
133
+ // 脚本沙箱没有文件系统/网络/定时器/require。任何一条违反都会在真机上直接终止整条 workflow。
134
+ const workflow = require('../workflow');
135
+ const wfScript = workflow.script;
136
+
137
+ const ALLOWED_SCHEMA_KEYWORDS = [
138
+ 'type', 'properties', 'required', 'additionalProperties', 'items', 'enum', 'const', 'oneOf',
139
+ 'description', 'title', 'default', 'examples',
140
+ ];
141
+ const ALLOWED_AGENT_OPTIONS = ['label', 'phase', 'schema', 'provider', 'model'];
142
+
143
+ /** 从脚本里取出 \`const XSchema = { ... }\` 的字面量并求值(脚本里的 schema 只是对象字面量)。 */
144
+ function extractSchemaLiterals(text) {
145
+ const out = [];
146
+ const re = /const (\w*[Ss]chema) = \{/g;
147
+ let m;
148
+ while ((m = re.exec(text)) !== null) {
149
+ const from = text.indexOf('{', m.index);
150
+ let depth = 0;
151
+ let i = from;
152
+ for (; i < text.length; i += 1) {
153
+ if (text[i] === '{') depth += 1;
154
+ else if (text[i] === '}') { depth -= 1; if (depth === 0) break; }
155
+ }
156
+ out.push({ name: m[1], value: eval('(' + text.slice(from, i + 1) + ')') }); // eslint-disable-line no-eval
157
+ }
158
+ return out;
159
+ }
160
+
161
+ /**
162
+ * 递归校验一个 JSON Schema 节点:只允许 DSH 子集的关键字。
163
+ * 只有 schema 位置的关键字受限——\`properties\` 下的键名是数据字段名,不参与校验。
164
+ * @returns {string[]} 违规关键字(含路径)
165
+ */
166
+ function unsupportedSchemaKeywords(node, path = '$') {
167
+ if (node === null || typeof node !== 'object' || Array.isArray(node)) return [];
168
+ const bad = [];
169
+ for (const key of Object.keys(node)) {
170
+ if (!ALLOWED_SCHEMA_KEYWORDS.includes(key)) bad.push(path + '.' + key);
171
+ }
172
+ if (node.properties && typeof node.properties === 'object') {
173
+ for (const [name, child] of Object.entries(node.properties)) {
174
+ bad.push(...unsupportedSchemaKeywords(child, path + '.properties.' + name));
175
+ }
176
+ }
177
+ if (node.items) bad.push(...unsupportedSchemaKeywords(node.items, path + '.items'));
178
+ if (Array.isArray(node.oneOf)) node.oneOf.forEach((child, i) => bad.push(...unsupportedSchemaKeywords(child, path + '.oneOf[' + i + ']')));
179
+ return bad;
180
+ }
181
+
182
+ test('workflow: agent() schemas stay inside the DSH-supported subset', () => {
183
+ // 违反子集 = 引擎抛 UNSUPPORTED_SCHEMA = 整条 workflow 直接终止(不是降级)。
184
+ const schemas = extractSchemaLiterals(wfScript);
185
+ assert.ok(schemas.length >= 2, 'expected the fetch/check schemas, found ' + schemas.length);
186
+ for (const s of schemas) {
187
+ assert.deepStrictEqual(unsupportedSchemaKeywords(s.value), [],
188
+ s.name + ' uses unsupported schema keyword(s)');
189
+ }
190
+ });
191
+
192
+ test('workflow: the schema-subset checker actually rejects an unsupported keyword', () => {
193
+ // 反向断言:确保上面的“通过”不是因为校验器写空了。
194
+ const bad = unsupportedSchemaKeywords({ type: 'object', properties: { a: { type: 'string', minLength: 3 } } });
195
+ assert.deepStrictEqual(bad, ['$.properties.a.minLength']);
196
+ });
197
+
198
+ test('workflow: agent() options stay inside the engine whitelist', () => {
199
+ const re = /agent\([^,]*,\s*\{([^}]*)\}/g;
200
+ let m;
201
+ let calls = 0;
202
+ while ((m = re.exec(wfScript)) !== null) {
203
+ calls += 1;
204
+ const keys = [];
205
+ const keyRe = /([A-Za-z_][A-Za-z0-9_]*)\s*:/g;
206
+ let k;
207
+ while ((k = keyRe.exec(m[1])) !== null) keys.push(k[1]);
208
+ for (const key of keys) {
209
+ assert.ok(ALLOWED_AGENT_OPTIONS.includes(key),
210
+ 'agent() option "' + key + '" is not supported by the engine (allowed: ' + ALLOWED_AGENT_OPTIONS.join(', ') + ')');
211
+ }
212
+ }
213
+ assert.ok(calls >= 5, 'expected the script agent() calls, found ' + calls);
214
+ });
215
+
216
+ test('workflow: the script never reaches for APIs the sandbox does not provide', () => {
217
+ const forbidden = [
218
+ [/\brequire\s*\(/, 'require()'],
219
+ [/\bprocess\s*\./, 'process'],
220
+ [/\bfs\s*\./, 'fs'],
221
+ [/\bfetch\s*\(/, 'fetch()'],
222
+ [/\bsetTimeout\s*\(/, 'setTimeout()'],
223
+ [/\bsetInterval\s*\(/, 'setInterval()'],
224
+ [/\b__dirname\b/, '__dirname'],
225
+ [/\bchild_process\b/, 'child_process'],
226
+ ];
227
+ for (const [re, label] of forbidden) {
228
+ assert.ok(!re.test(wfScript), 'the workflow sandbox provides no ' + label);
229
+ }
230
+ });
231
+
232
+ test('workflow: the script self-contains all four parsers', () => {
233
+ for (const name of ['book', 'paper', 'video', 'web']) {
234
+ assert.ok(wfScript.includes(name + ': { buildPrompt:'), 'parser "' + name + '" is not inlined');
235
+ }
236
+ assert.ok(wfScript.includes('const parsers = args._parsers || __parsers;'), 'the args._parsers fallback is missing');
237
+ });
238
+
239
+ // ---------- Test: skill content(复用本文件顶部已有的 plugin 引用)----------
240
+ test('skill content carries meta + script + args and the write-back step', () => {
241
+ const content = plugin.buildSkillContent();
242
+ assert.ok(content.includes(workflow.meta.name), 'meta missing from the skill body');
243
+ assert.ok(content.includes(wfScript), 'the workflow script must be embedded verbatim');
244
+ assert.ok(content.includes('"input"'), 'the args example is missing');
245
+ // 脚本没有文件系统权限:技能必须明确要求主代理把 note 落盘,否则笔记会丢。
246
+ assert.ok(/filePath/.test(content) && /write/.test(content), 'the skill must tell the model to persist note -> filePath');
247
+ });
248
+
249
+ test('plugin entry satisfies the Cordis contract and registers one bundled skill', () => {
250
+ assert.strictEqual(plugin.name, 'deep-read-summarize');
251
+ assert.deepStrictEqual(plugin.inject, ['skills']);
252
+ assert.strictEqual(typeof plugin.apply, 'function');
253
+ const registered = [];
254
+ const ctx = { skills: { register: (skill) => { registered.push(skill); return () => {}; } } };
255
+ const dispose = plugin.apply(ctx, {});
256
+ assert.strictEqual(registered.length, 1);
257
+ assert.strictEqual(registered[0].name, 'deep-read-summarize');
258
+ assert.strictEqual(registered[0].source, 'bundled');
259
+ assert.ok(registered[0].content.length > 1000);
260
+ assert.strictEqual(typeof dispose, 'function');
261
+ });
262
+
101
263
  // ---------- Run cache tests ----------
102
264
  console.log('\n=== cache tests ===');
103
265
  const cacheResult = require('./cache.test.js');