deep-read-summarize 0.3.7 → 0.3.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +71 -1
- package/README.en.md +13 -9
- package/README.md +14 -10
- package/cordis.patch.yml +14 -7
- package/docs/RELEASE.md +44 -7
- package/index.js +60 -6
- package/package.json +4 -3
- package/parsers/book.js +1 -1
- package/parsers/video.js +54 -18
- package/schemas/index.js +9 -2
- package/scripts/transcribe.js +306 -0
- package/skills/deep-read-summarize/SKILL.md +11 -9
- package/tests/index.test.js +319 -1
- package/tests/run-tests.js +7 -3
- package/workflow.js +89 -76
- package/scripts/transcribe.ps1 +0 -68
package/CHANGELOG.md
CHANGED
|
@@ -3,6 +3,76 @@
|
|
|
3
3
|
All notable changes to this project will be documented in this file.
|
|
4
4
|
The format is based on [Keep a Changelog](https://keepachangelog.com/).
|
|
5
5
|
|
|
6
|
+
## [0.3.9] — 2026-09-22
|
|
7
|
+
|
|
8
|
+
**macOS / Linux 支持**。这一版补上最后一块 Windows-only 的拼图:此前「无字幕视频自动转写」依赖 `scripts/transcribe.ps1`(PowerShell),SKILL.md 里明确写着「**目前仅在 Windows 可用**」;现在换成跨平台的 Node 脚本,三平台同一份实现、同一条命令。
|
|
9
|
+
|
|
10
|
+
### Added
|
|
11
|
+
|
|
12
|
+
- **`scripts/transcribe.js` 取代 `scripts/transcribe.ps1`**:跨平台转写自举脚本(Windows / macOS / Linux)。平台差异只集中在四个纯函数里(`cacheRoot` / `venvPython` / `uvInstallHint` / `findUv`),它们都接受注入的 `platform/env/home`——因此**任意 OS 上都能跑全平台矩阵单测**,不必依赖 macOS 机器。
|
|
13
|
+
- 缓存目录:Windows `%LOCALAPPDATA%\deep-read-summarize`、macOS `~/Library/Caches/deep-read-summarize`、Linux `$XDG_CACHE_HOME/deep-read-summarize`
|
|
14
|
+
- uv 探测:先扫 PATH,再回退 `~/.local/bin`、`/opt/homebrew/bin`、`/usr/local/bin`、`~/.cargo/bin`(macOS 从图形界面启动时不继承 shell PATH,这一步是必需的);找不到时按平台给出安装命令(winget / brew / curl 安装器)
|
|
15
|
+
- `--self-check`:只打印解析出的路径,不安装、不下载、不联网——CI 三平台用它做冒烟
|
|
16
|
+
- **`options.device` 选项**(默认 `cpu`,可选 `auto`/`cuda`):从 options → workflow 脚本 → 解析器 → 命令行全程打通(与 `whisperModel` / `language` 同一条路径),不是写死在提示词里。`auto`/`cuda` 需自备 CUDA 运行库,失败自动回退 CPU。
|
|
17
|
+
- **CI 新增 `cross-platform` 作业**:在 ubuntu / macos / windows 三个 runner 上跑 lint + `--self-check`,真正在 macOS 上执行一遍平台分支。
|
|
18
|
+
|
|
19
|
+
### Fixed
|
|
20
|
+
|
|
21
|
+
- **`device='auto'` 在「有 NVIDIA 显卡但没装 CUDA 运行库」的机器上必崩(真机复现)**:ctranslate2 只要探测到显卡(`get_cuda_device_count() == 1`)就会在 `auto` 下选 CUDA,随后在推理时抛 `Library cublas64_12.dll is not found or cannot be loaded`。这是 0.3.6 起就潜伏在 `transcribe.ps1` 里的问题(本机 RTX 3050 复现)。现在默认 `device=cpu`(与 README 承诺的「CPU 友好、零额外依赖」一致);设 `options.device: "auto"` 时,失败也会**自动回退 CPU 并打印原因**。
|
|
22
|
+
- **转写中途失败会留下半截 `transcript.txt`**:调用方(提示词第 ④ 步)是按「输出文件非空且稳定」判断转写完成的,半截文件会被**误判成成功**。现在先在内存里收集全部分段,成功后才落盘。
|
|
23
|
+
- **`parsers/video.js` 提示词里的 PowerShell 路径少了分隔符**:`$env:USERPROFILE\.dsh` 写在 JS 字符串里,`\.` 被当成未知转义退化成 `.`,实际发出去的提示词是 `$env:USERPROFILE.dsh`。改用正斜杠(PowerShell 同样接受)。
|
|
24
|
+
- **`parsers/book.js` 让代理用 `pwsh` 调 pdftotext**(第二轮全仓复查才发现,第一轮只盯着转写链路):macOS 默认没有 PowerShell,**书籍/PDF 这条路在 macOS 上会直接卡住**。现在改成平台中立的 pdftotext 说明,并给出三平台安装方式(macOS `brew install poppler` / Windows `winget install poppler` / Debian·Ubuntu `apt install poppler-utils`)。
|
|
25
|
+
- **`--out` 的父目录不存在时失败信息难以理解**:现在脚本自己先把父目录建好(`./.tmp` 这类调用方路径),不再让 python 抛一句难懂的 `No such file or directory`。
|
|
26
|
+
|
|
27
|
+
### Changed
|
|
28
|
+
|
|
29
|
+
- **`parsers/video.js` 提示词改为平台中立**:同时给出 Windows 与 macOS/Linux 的脚本定位命令(`Get-ChildItem … -Filter transcribe.js` / `find "$HOME/.dsh" -name transcribe.js`),yt-dlp 安装同时给出 `winget` 与 `brew`;转写命令三平台统一为 `node <脚本路径> --audio … --out … --device cpu --model … --language …`。
|
|
30
|
+
- 提示词**不能**按平台分支:它会被内联进 workflow 沙箱脚本,而沙箱不提供 `process`(`tests/run-tests.js` 有对应断言),所以两套写法都写出来、由执行代理按自己的系统选。
|
|
31
|
+
- 转写输出不再带结尾空格(改用 `' '.join(...)`)。
|
|
32
|
+
- 提示词里「转写执行」的 ①–⑥ 子步骤此前被模板编号成了一个独立的顶层步骤(`7. ① 先取音频…`,后续行反而没有编号),现在与标题合并为同一步,编号回到 `6. 转写执行(…):` + 未编号的 ①–⑥。
|
|
33
|
+
- `README.md` / `README.en.md` / `SKILL.md` 同步:删掉「仅 Windows 可用」,补上跨平台命令、平台缓存目录与 `options.device` 说明。
|
|
34
|
+
|
|
35
|
+
### Compatibility
|
|
36
|
+
|
|
37
|
+
- **Windows 用户零迁移成本**:缓存目录与 `transcribe.ps1` 逐字一致,**已下载的模型不会重下**(真机验证:复用既有 venv + 模型,未触发下载)。
|
|
38
|
+
- 唯一入口改名 `transcribe.ps1` → `transcribe.js`;提示词文本随之变化(脚本名、参数形式、`options.device`),依赖提示词逐字比对的 fork 需要同步。
|
|
39
|
+
- 转写默认从 `auto` 变为 `cpu`:原 `auto` 在无 CUDA 运行库的机器上本来就会崩,真正的 GPU 用户可显式传 `--device auto`。
|
|
40
|
+
|
|
41
|
+
## [0.3.8] — 2026-09-16
|
|
42
|
+
|
|
43
|
+
又一次「代码审查 + 真机复现」驱动的修复轮。这一轮的核心发现不是某一行写错了,而是**几处「文档承诺」与「代码实际行为」对不上**:选项算出来不用、schema 抄了两份各自漂移、配置文件里的键全是空转。所有 P0 都用 mock 掉 `agent/parallel/phase/log` 的方式**跑真实脚本复现过**,并各自补了回归测试。测试 35 → **62**(fixture 35 + node:test 27)。
|
|
44
|
+
|
|
45
|
+
### Fixed
|
|
46
|
+
|
|
47
|
+
- **frontmatter 注入:标题里带引号会产出坏 YAML(P0)**。`title: "《" + finalTitle + "》…"` 是裸拼,内容标题含 `"`(如 *The "Best" Method*)时 frontmatter 直接被截断,Obsidian / Dataview 全部解析失败。现在统一走 `yamlStr()`(JSON 字符串转义,是 YAML double-quoted 风格的子集),`title / aliases / author / year / type / url` 全部转义;标题里的换行/制表也一并折叠,不再污染 `.md` 文件名。
|
|
48
|
+
- **幂等命中分支会返回空笔记(P0)**。脚本里那段 `args._processedKeys` 早返回,命中时返回 `note: ""` + `filePath: ""`;而 SKILL.md 第 8 步要求主代理「把 `note` 写入 `filePath`」——命中即写出空文件。宿主实际上**从不注入 `args._processedKeys`**(0.1.x 全仓 0 命中),这段代码永远不会命中,一旦命中就是数据事故。整段删除。
|
|
49
|
+
- **分块计划不受 `maxChunks` 约束(P0)**。提示词里写着「块数不超过 N 块」,但脚本原样相信子代理返回的 `chunkPlan`:实测 `maxChunks: 4` 会起 **10 个子代理**——这正是 0.3.x 反复踩的「单次调用超时」的成因。现在按上限截断,并在收敛时记日志。
|
|
50
|
+
- **分块行号区间不校验(P0)**。倒置区间(`startLine: 900, endLine: 100`)会原样拼成 `limit=-799` 交给 `read`。现在丢弃倒置/越界的块,并按 `totalLines` 夹紧上界;全部无效时按可降级错误返回,不再空跑。
|
|
51
|
+
- **全非法字符的标题会写出隐藏文件**:`///` 之类清洗后为空 → `./output/.md`。现在兜底为 `未命名内容.md`。
|
|
52
|
+
- **`options.transcribe: false` 完全无效**。`parsers/video.js` 里 `wantTranscribe` 算完从未被使用,`true` 与 `false` 生成的提示词**逐字相同**,README/SKILL 却都承诺「false 则跳过转写」。现在提示词真的分流:禁用时明确禁止定位/运行 `transcribe.ps1`、禁止下载模型,直接走降级。
|
|
53
|
+
- **`options.whisperModel` / `options.language` 文档教用、代码没有**。README 写着「可用 `options.whisperModel` 调整」「纯英文视频可设 `language: "en"`」,但脚本既不读也不透传,转写命令里 `-Model small -Language zh` 是写死的。现在两个选项都从 options → 脚本 → 解析器 → 命令行全程打通,默认值保持 `small` / `zh`(对既有用户零行为变化)。
|
|
54
|
+
- **`kind` 会变成 `"auto"` 流进笔记**。`type: auto` 且子代理没回 `kind` 时,`frontmatter` 的 `type` 与 `tags` 会被写成 `auto`。现在 `auto` 只接受解析器表里真实存在的类型,否则退回实际使用的回退解析器。
|
|
55
|
+
|
|
56
|
+
### Changed
|
|
57
|
+
|
|
58
|
+
- **插件 config 不再是空转**。`apply(ctx, config)` 此前收了 config 从不读它,`cordis.patch.yml` 里列的 8 个键(`outputDir`/`tempDir`/`minWords`/…)全部无效。现在会做白名单校验(类型不符 → 告警并忽略,不会因为一个拼错的键把整个 profile 拖死),合并进技能里的 args 示例,成为该 profile 的默认值。`cordis.patch.yml` 的注释同步改写(含 `maxChunks` 6 → 4)。
|
|
59
|
+
- **schema 不再有两份真相**。脚本里那份手抄的 `fetchSchema` 已删除,改由 `workflow.js` 在构建自包含脚本时从 `schemas/index.js` **注入**;`kind` 的 `enum` 跟着解析器注册表走。此前手抄副本已经漂移(丢了 `enum`)。同时把 `chunkReadSchema` / `qualityChecklistSchema` **没有被 workflow 使用**这件事写进 `schemas/index.js` 头部——它们留给程序化调用方。
|
|
60
|
+
- **自定义解析器真正可用**。以前 `type` 白名单写死 `auto|book|paper|video|web`,且自包含脚本只内联这四个,`custom-parsers/` 里的**新类型**端到端不可达。现在内联的是注册表里的全部解析器,白名单与 schema 枚举都从注册表推导;同名覆盖照旧有效。
|
|
61
|
+
- **幂等从「脚本假装会做」改成「文档如实说不会做」**。脚本沙箱没有文件系统,写不了指纹记录——`lib/cache.js` 从 0.1.0 起就没被 workflow 调用过。现在 SKILL.md 第 2 步改为:由主代理在落盘前检查目标 `filePath` 是否已存在;程序化去重可用 `deep-read-summarize/lib/cache`(`package.json` 的 `exports` 补上了这个子路径,此前被 exports map 挡住根本 require 不到)。
|
|
62
|
+
- SKILL.md 新增第 9 步:`ok: false` 要说明卡在哪一步,`qualityPassed: false` / `failedChunks > 0` 必须在回复里点出来,不许拿半成品冒充成品。
|
|
63
|
+
- `prepublishOnly` 补跑 `node --test`(此前发布门禁只跑 fixture 测试,`tests/index.test.js` 不在门禁内)。
|
|
64
|
+
- npm 描述去掉 `idempotent cache`(工作流层确实没有这个能力)。
|
|
65
|
+
|
|
66
|
+
### Added
|
|
67
|
+
|
|
68
|
+
- **workflow 运行时回归测试(12 → 27 项 `node:test`)**:用 mock 掉 `agent/parallel/phase/log` 的方式跑**真实脚本**,逐条锁住本轮修的 P0/P1——分块收敛、区间夹紧、全无效计划降级、YAML 转义、隐藏文件名兜底、`kind` 兜底、幂等早返回已移除、`transcribe` 分流、`whisperModel`/`language` 透传、config 校验与透传、schema 单一真相、自定义类型端到端(含自定义解析器内联与枚举跟进)、未知类型仍然响亮报错、`lib/cache` 可达。
|
|
69
|
+
|
|
70
|
+
### Compatibility
|
|
71
|
+
|
|
72
|
+
- 对既有用户**无破坏性变更**:默认值全部保持不变(`maxChunks` 4、`transcribe` true、`whisperModel` small、`language` zh、`outputDir` `./output`)。
|
|
73
|
+
- 行为变化仅在于「以前被静默忽略的东西现在生效了」:超过 `maxChunks` 的计划会被截断(子代理数下降、耗时下降)、非法分块区间会被丢弃(此前会把负数 limit 传给 `read`)。
|
|
74
|
+
- 历史文档 `docs/RELEASE-v0.1.0.md` 里 v0.1.0 的「幂等缓存」描述保持原样(它是对当时意图的记录),实际能力以本节与 `schemas/index.js` 的说明为准。
|
|
75
|
+
|
|
6
76
|
## [0.3.7] — 2026-09-09
|
|
7
77
|
|
|
8
78
|
针对 **DSH 0.1.2-rc.1** 的一轮兼容性核查与打磨(对使用者无破坏性变更)。
|
|
@@ -167,4 +237,4 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/).
|
|
|
167
237
|
### Added
|
|
168
238
|
- Initial deep-reading pipeline (5 serial waves)
|
|
169
239
|
- Template prompts for books/papers/videos/web
|
|
170
|
-
- Obsidian note template with YAML frontmatter
|
|
240
|
+
- Obsidian note template with YAML frontmatter
|
package/README.en.md
CHANGED
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
[](https://github.com/PensiveFei/deep-read-summarize/blob/main/LICENSE)
|
|
4
4
|
[](https://github.com/PensiveFei/deep-read-summarize/releases)
|
|
5
5
|
[](https://github.com/PensiveFei/deep-read-summarize/actions/workflows/ci.yml)
|
|
6
|
-
[](https://www.npmjs.com/package/deep-read-summarize)
|
|
7
7
|
|
|
8
8
|
> **Disclaimer**: This is an **unofficial third-party tool**. It is not affiliated with, endorsed by, or sponsored by DeepSeek or the DeepSeek Harness project. "DeepSeek" and "DeepSeek Harness" are trademarks of their respective owners. This project only orchestrates the workflow tool available in your DSH environment; it does not redistribute any DeepSeek software.
|
|
9
9
|
|
|
@@ -37,7 +37,8 @@ Content is split into chunks, read in parallel by multiple sub-agents, then merg
|
|
|
37
37
|
|
|
38
38
|
- Books (PDF/EPUB/MOBI), papers (arXiv/PDF/HTML), videos (**subtitles first, auto-transcription when there are none** — produces a full transcript for deep reading), web pages
|
|
39
39
|
- Long content is chunked and read in parallel by sub-agents, then merged
|
|
40
|
-
-
|
|
40
|
+
- The fetch/chunk plan and the quality check are constrained by JSON Schema, so their output is stable and machine-readable
|
|
41
|
+
- Chunk count — and therefore sub-agent count — is enforced by `maxChunks` (a runaway plan blows the per-call time limit)
|
|
41
42
|
- Key citations must carry page/chapter/paragraph references to reduce fabrication
|
|
42
43
|
- Configuration errors abort immediately; a chunk that fails to parse is skipped and marked as a gap
|
|
43
44
|
- Output drops straight into Obsidian and works with Dataview
|
|
@@ -71,16 +72,16 @@ Three waves, roughly N+2 sub-agents in total, where N is the number of chunks.
|
|
|
71
72
|
parsers/ Parsers for each input type, dispatched by type
|
|
72
73
|
book.js Books: PDF/EPUB/MOBI text extraction, chapter chunking
|
|
73
74
|
paper.js Papers: arXiv/PDF/HTML structure detection
|
|
74
|
-
video.js Video: full transcript (subtitles first; auto-transcribe via scripts/transcribe.
|
|
75
|
+
video.js Video: full transcript (subtitles first; auto-transcribe via scripts/transcribe.js when none)
|
|
75
76
|
web.js Web pages: main-text extraction
|
|
76
77
|
index.js Registry: parser discovery and fallback
|
|
77
|
-
schemas/ JSON Schema for
|
|
78
|
-
scripts/ lint, security checks
|
|
78
|
+
schemas/ JSON Schema for the output contract (injected into the script by workflow.js - single source of truth)
|
|
79
|
+
scripts/ lint, security checks, cross-platform transcription bootstrap (transcribe.js)
|
|
79
80
|
tests/ fixture tests and validation scripts
|
|
80
81
|
workflow.js The workflow script itself (meta + script)
|
|
81
82
|
```
|
|
82
83
|
|
|
83
|
-
To change how an input type is handled, drop a parser with the same interface into `custom-parsers/`; a type with the same name overrides the built-in implementation. The interface has only three fields: `name`, `types`, `buildPrompt(input, opts)`.
|
|
84
|
+
To change how an input type is handled, drop a parser with the same interface into `custom-parsers/`; a type with the same name overrides the built-in implementation, and you can also add a brand-new type name (e.g. `podcast`) - every registered parser is inlined into the workflow script, and both the `type` whitelist and the schema's `kind` enum follow the registry. The interface has only three fields: `name`, `types`, `buildPrompt(input, opts)`.
|
|
84
85
|
|
|
85
86
|
---
|
|
86
87
|
|
|
@@ -101,7 +102,7 @@ deep-read-summarize is a DSH plugin installable via npm, the dsh.so ecosystem, o
|
|
|
101
102
|
npm install deep-read-summarize
|
|
102
103
|
|
|
103
104
|
# or local install (dsh profile directory)
|
|
104
|
-
pnpm add ./deep-read-summarize-0.3.
|
|
105
|
+
pnpm add ./deep-read-summarize-0.3.9.tgz
|
|
105
106
|
# then append to the dsh config's dsh.profile.bundles:
|
|
106
107
|
# - deep-read-summarize
|
|
107
108
|
# restart dsh web (POST /dsh-market/restart)
|
|
@@ -174,6 +175,9 @@ Pass this JSON to DSH's workflow tool:
|
|
|
174
175
|
"fastMode": false, // true skips sections 5-7, faster
|
|
175
176
|
"maxChunks": 4, // chunk limit, 1-12 (default lowered to control sub-agent count and time)
|
|
176
177
|
"transcribe": true, // video: auto local transcription (faster-whisper) when no subtitles; false skips it — see Video
|
|
178
|
+
"whisperModel": "small", // video ASR tier: small | base | medium
|
|
179
|
+
"language": "zh", // video ASR language: use "en" for English-only videos
|
|
180
|
+
"device": "cpu", // ASR device: cpu (default) | auto | cuda; auto/cuda needs a CUDA runtime and falls back to cpu
|
|
177
181
|
"requireCitations": true, // whether key conclusions must carry citations
|
|
178
182
|
"includeTimestamps": false,
|
|
179
183
|
"outputDir": "./output", // note output directory (can point at your Obsidian vault)
|
|
@@ -200,12 +204,12 @@ Video text extraction is a **unified pipeline** (the old "three tiers" are gone)
|
|
|
200
204
|
> **⚠️ To preserve deep-reading quality, processing may take noticeably longer (a deliberate trade-off, not a hang)**: a **subtitle-less** video is transcribed first (roughly 0.5–2× the video length on CPU) before deep reading, so such videos are **much slower than subtitled videos**; in exchange you get a **full transcript + real deep reading**. **Subtitled** videos remain seconds-fast. Please do not mistake the slower transcription for a hang.
|
|
201
205
|
|
|
202
206
|
1. **Platform subtitles available** (Bilibili's official API AI subtitles / YouTube CC) → **use them directly** (= full text, fastest, zero dependencies).
|
|
203
|
-
2. **No public subtitles** → use the bundled transcription `scripts/transcribe.
|
|
207
|
+
2. **No public subtitles** → use the bundled transcription `scripts/transcribe.js` (faster-whisper small / int8 / VAD / Chinese) to get the full text.
|
|
204
208
|
3. **Neither works** → **degrade gracefully**: prompt the user to supply a transcript manually, or fall back to `desc` as background; **never block**.
|
|
205
209
|
|
|
206
210
|
`options.transcribe`: **default `true`** (auto-transcribe when no subtitles); set `false` to skip transcription (use subtitles/desc or degrade).
|
|
207
211
|
|
|
208
|
-
**Transcription toolchain (identical for you and the author)**: the script **bootstraps itself** — it uses `uv` to create a **Python 3.12** environment + **Tsinghua mirror** to install `faster-whisper`, downloads the model via `HF_ENDPOINT=https://hf-mirror.com` and **caches it for reuse**; faster-whisper bundles PyAV to decode audio, so **no separate ffmpeg** is needed; **versions are pinned** (Python 3.12 / faster-whisper / uv) so the author's environment matches yours.
|
|
212
|
+
**Transcription toolchain (identical for you and the author)**: the script **bootstraps itself** — it uses `uv` to create a **Python 3.12** environment + **Tsinghua mirror** to install `faster-whisper`, downloads the model via `HF_ENDPOINT=https://hf-mirror.com` and **caches it for reuse**; faster-whisper bundles PyAV to decode audio, so **no separate ffmpeg** is needed; **versions are pinned** (Python 3.12 / faster-whisper / uv) so the author's environment matches yours. The script is written in **Node**, so **Windows / macOS / Linux share one implementation and one command** (`node scripts/transcribe.js --audio <file> --out <txt>`). Transcription runs on **CPU** by default (`options.device`); GPU needs a CUDA runtime (cuBLAS/cuDNN), so `options.device: "auto"` is opt-in and falls back to CPU automatically. Cache location per platform: Windows `%LOCALAPPDATA%\deep-read-summarize`, macOS `~/Library/Caches/deep-read-summarize`, Linux `$XDG_CACHE_HOME/deep-read-summarize`.
|
|
209
213
|
|
|
210
214
|
> ⚠️ **The first transcription downloads the model first (small ≈ 484 MB, via hf-mirror, not GitHub) and may take a few minutes** — please be patient; **it downloads only once** on your machine, then every later transcription uses the cache and starts instantly. **Only subtitle-less videos trigger transcription** (subtitled videos use their subtitles directly — no trigger, no download).
|
|
211
215
|
|
package/README.md
CHANGED
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
[](https://github.com/PensiveFei/deep-read-summarize/blob/main/LICENSE)
|
|
4
4
|
[](https://github.com/PensiveFei/deep-read-summarize/releases)
|
|
5
5
|
[](https://github.com/PensiveFei/deep-read-summarize/actions/workflows/ci.yml)
|
|
6
|
-
[](https://www.npmjs.com/package/deep-read-summarize)
|
|
7
7
|
|
|
8
8
|
> **Disclaimer**: This is an **unofficial third-party tool**. It is not affiliated with, endorsed by, or sponsored by DeepSeek or the DeepSeek Harness project. "DeepSeek" and "DeepSeek Harness" are trademarks of their respective owners. This project only orchestrates the workflow tool available in your DSH environment; it does not redistribute any DeepSeek software.
|
|
9
9
|
|
|
@@ -37,7 +37,8 @@
|
|
|
37
37
|
|
|
38
38
|
- 书籍(PDF/EPUB/MOBI)、论文(arXiv/PDF/HTML)、视频(**字幕优先,无字幕自动转写**,出品完整逐字稿精读)、网页
|
|
39
39
|
- 长内容分块后由并行子代理精读,再合并
|
|
40
|
-
-
|
|
40
|
+
- 获取/分块与质量校验的输出受 JSON Schema 约束,结构稳定、下游可解析
|
|
41
|
+
- 分块计划与子代理数量受 `maxChunks` 强制约束(计划失控会顶穿单次调用时限)
|
|
41
42
|
- 关键引用必须标注页码/章节/段落,降低编造风险
|
|
42
43
|
- 配置错误直接报错终止;某块内容解析失败则跳过并标记缺口
|
|
43
44
|
- 输出可直接放进 Obsidian,配合 Dataview 使用
|
|
@@ -71,16 +72,16 @@ Obsidian 笔记
|
|
|
71
72
|
parsers/ 各输入类型的解析器,按类型分发
|
|
72
73
|
book.js 书籍:PDF/EPUB/MOBI 文本提取、章节分块
|
|
73
74
|
paper.js 论文:arXiv/PDF/HTML 结构识别
|
|
74
|
-
video.js 视频:完整逐字稿(字幕优先,无字幕自动转写,走 scripts/transcribe.
|
|
75
|
+
video.js 视频:完整逐字稿(字幕优先,无字幕自动转写,走 scripts/transcribe.js)
|
|
75
76
|
web.js 网页:正文提取
|
|
76
77
|
index.js 注册表:解析器发现与回退
|
|
77
|
-
schemas/
|
|
78
|
-
scripts/ lint
|
|
78
|
+
schemas/ 输出约束的 JSON Schema(波次1 由 workflow.js 注入脚本,单一真相)
|
|
79
|
+
scripts/ lint、安全检查、跨平台转写引导(transcribe.js)
|
|
79
80
|
tests/ fixture 测试与验证脚本
|
|
80
81
|
workflow.js workflow 脚本本体(meta + script)
|
|
81
82
|
```
|
|
82
83
|
|
|
83
|
-
想换某个输入类型的处理方式,在 `custom-parsers/`
|
|
84
|
+
想换某个输入类型的处理方式,在 `custom-parsers/` 放一个同接口的解析器即可,同名类型会覆盖内置实现;用新类型名(如 `podcast`)也可以——注册表里的解析器会全部内联进 workflow 脚本,`type` 白名单与 schema 的 `kind` 枚举都跟着注册表走。接口只有三个字段:`name`、`types`、`buildPrompt(input, opts)`。
|
|
84
85
|
|
|
85
86
|
---
|
|
86
87
|
|
|
@@ -101,7 +102,7 @@ deep-read-summarize 是 DSH 插件,可通过 npm、dsh.so 生态或本地安
|
|
|
101
102
|
npm install deep-read-summarize
|
|
102
103
|
|
|
103
104
|
# 或本地安装(dsh profile 目录)
|
|
104
|
-
pnpm add ./deep-read-summarize-0.3.
|
|
105
|
+
pnpm add ./deep-read-summarize-0.3.9.tgz
|
|
105
106
|
# 然后在 dsh 配置的 dsh.profile.bundles 追加:
|
|
106
107
|
# - deep-read-summarize
|
|
107
108
|
# 重启 dsh web 即可(POST /dsh-market/restart)
|
|
@@ -174,6 +175,9 @@ parsers: book, paper, video, web
|
|
|
174
175
|
"fastMode": false, // true 时跳过 5-7 节,速度快一些
|
|
175
176
|
"maxChunks": 4, // 分块上限,1-12(默认调低,控制子代理数与耗时)
|
|
176
177
|
"transcribe": true, // 视频:无字幕时自动本地转写(faster-whisper);false 则跳过转写,见「视频」节
|
|
178
|
+
"whisperModel": "small", // 视频转写模型档位:small | base | medium
|
|
179
|
+
"language": "zh", // 视频转写语言:纯英文视频可设 "en"
|
|
180
|
+
"device": "cpu", // 转写设备:cpu(默认)| auto | cuda;auto/cuda 需自备 CUDA 运行库,失败自动回退 cpu
|
|
177
181
|
"requireCitations": true, // 关键结论是否必须标注出处
|
|
178
182
|
"includeTimestamps": false,
|
|
179
183
|
"outputDir": "./output", // 笔记输出目录(可指向 Obsidian 仓库)
|
|
@@ -200,12 +204,12 @@ parsers: book, paper, video, web
|
|
|
200
204
|
> **⚠️ 为保证精读质量,处理时间可能略长(刻意取舍,不是卡死)**:**无字幕**视频会先做一次转写(CPU 上约为视频时长的 0.5–2×)再精读,所以这类视频整体会**明显比有字幕视频慢**;换来的是**完整逐字稿 + 真正的精读**。**有字幕**视频仍是秒级。请勿把转写的较慢误判为卡死。
|
|
201
205
|
|
|
202
206
|
1. **有平台字幕**(B站官方 API 的 AI 字幕 / YouTube CC)→ **直接用字幕**(=全文,最快、零依赖)。
|
|
203
|
-
2. **无公开字幕** → 用插件自带转写 `scripts/transcribe.
|
|
207
|
+
2. **无公开字幕** → 用插件自带转写 `scripts/transcribe.js`(faster-whisper small / int8 / VAD / 中文)得到全文。
|
|
204
208
|
3. 都不行 → **降级**:提示手动提供转写文本,或退回 `desc` 作背景;**绝不阻塞**。
|
|
205
209
|
|
|
206
210
|
`options.transcribe`:**默认 `true`**(无字幕自动转写);设为 `false` 则跳过转写(只用字幕/desc 或降级)。
|
|
207
211
|
|
|
208
|
-
**转写工具链(本机/用户一致)**:脚本**自举**——用 `uv` 建 **Python 3.12** 环境 + **清华镜像**装 `faster-whisper` + `HF_ENDPOINT=https://hf-mirror.com` 下模型并**缓存复用**;faster-whisper 内置 PyAV 解码音频,**无需单独 ffmpeg**;**版本已锁定**(Python 3.12 / faster-whisper / uv
|
|
212
|
+
**转写工具链(本机/用户一致)**:脚本**自举**——用 `uv` 建 **Python 3.12** 环境 + **清华镜像**装 `faster-whisper` + `HF_ENDPOINT=https://hf-mirror.com` 下模型并**缓存复用**;faster-whisper 内置 PyAV 解码音频,**无需单独 ffmpeg**;**版本已锁定**(Python 3.12 / faster-whisper / uv),保证本机与用户环境一致。脚本是 **Node** 写的,**Windows / macOS / Linux 同一份实现、同一条命令**(`node scripts/transcribe.js --audio <音频> --out <txt>`);转写默认走 **CPU**(`options.device` 默认 `cpu`),GPU 需自备 CUDA 运行库(cuBLAS/cuDNN),可设 `options.device: "auto"` 尝试,失败会自动回退 CPU。缓存目录按平台取:Windows `%LOCALAPPDATA%\deep-read-summarize`、macOS `~/Library/Caches/deep-read-summarize`、Linux `$XDG_CACHE_HOME/deep-read-summarize`。
|
|
209
213
|
|
|
210
214
|
> ⚠️ **首次转写会先下载模型(small 约 484MB,走 hf-mirror,非 GitHub),可能需要几分钟**,请耐心等待;**本机只下这一次**,之后所有视频转写用缓存、秒开。**只有「无字幕」视频才会触发转写**(有字幕的视频直接用字幕,不触发、不下载)。
|
|
211
215
|
|
|
@@ -307,4 +311,4 @@ npm publish 会自动先跑 prepublishOnly(测试 + lint + 安全检查),
|
|
|
307
311
|
|
|
308
312
|
## License
|
|
309
313
|
|
|
310
|
-
MIT,见 [LICENSE](LICENSE)。
|
|
314
|
+
MIT,见 [LICENSE](LICENSE)。
|
package/cordis.patch.yml
CHANGED
|
@@ -10,13 +10,20 @@
|
|
|
10
10
|
- id: deep-read-summarize
|
|
11
11
|
name: 'deep-read-summarize'
|
|
12
12
|
|
|
13
|
-
# All plugin config keys are optional
|
|
13
|
+
# All plugin config keys are optional. The plugin validates each one (wrong type
|
|
14
|
+
# = warning + ignore, never a startup failure) and folds them into the default
|
|
15
|
+
# args carried by the skill, so they become this profile's defaults.
|
|
14
16
|
# config:
|
|
15
17
|
# outputDir: './output' # where notes are written (default: ./output)
|
|
16
18
|
# tempDir: './.tmp' # temp files for chunked content (default: ./.tmp)
|
|
17
|
-
# minWords: 2500 # minimum output length
|
|
18
|
-
# fastMode: false # skip sections 5-7 for speed
|
|
19
|
-
# maxChunks:
|
|
20
|
-
#
|
|
21
|
-
#
|
|
22
|
-
#
|
|
19
|
+
# minWords: 2500 # minimum output length (default: 2500)
|
|
20
|
+
# fastMode: false # skip sections 5-7 for speed (default: false)
|
|
21
|
+
# maxChunks: 4 # chunk cap; also caps sub-agent count (1-12, default: 4)
|
|
22
|
+
# transcribe: true # video: auto-transcribe when no subtitles (default: true)
|
|
23
|
+
# requireCitations: true # key conclusions must cite source location (default: true)
|
|
24
|
+
# includeTimestamps: false # video timestamps (default: false)
|
|
25
|
+
#
|
|
26
|
+
# Idempotency is deliberately NOT a plugin config key: the workflow script runs
|
|
27
|
+
# without a filesystem, so it cannot write a fingerprint record. Programmatic
|
|
28
|
+
# callers can use `deep-read-summarize/lib/cache`
|
|
29
|
+
# (fingerprint / hasProcessed / markProcessed).
|
package/docs/RELEASE.md
CHANGED
|
@@ -32,18 +32,55 @@
|
|
|
32
32
|
|
|
33
33
|
---
|
|
34
34
|
|
|
35
|
-
## npm
|
|
35
|
+
## npm 发布流程
|
|
36
|
+
|
|
37
|
+
### 首选:npm Trusted Publishing(OIDC,无 token、无 OTP)
|
|
38
|
+
|
|
39
|
+
```yaml
|
|
40
|
+
.github/workflows/publish.yml → 在 GitHub Release 被发布时自动发布
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
凭据由 GitHub OIDC 在运行时现场换取,仓库里不存任何 npm token,也不会触发 npm 的 2FA 提示。
|
|
44
|
+
|
|
45
|
+
```bash
|
|
46
|
+
# 1. 本地全绿
|
|
47
|
+
npm run lint && npm test && npm run test:node && npm run validate
|
|
48
|
+
|
|
49
|
+
# 2. 打 tag 并推送
|
|
50
|
+
git tag -a vX.Y.Z -m "vX.Y.Z" && git push origin vX.Y.Z
|
|
51
|
+
|
|
52
|
+
# 3. 建 Release —— 这一步触发 publish.yml
|
|
53
|
+
gh release create vX.Y.Z --title "vX.Y.Z" --notes-file <notes 文件>
|
|
54
|
+
|
|
55
|
+
# 4. 验证线上
|
|
56
|
+
npm view deep-read-summarize --prefer-online
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
**前置(每个包一次性,只能在网页上配)**:
|
|
60
|
+
`https://www.npmjs.com/package/<包名>/access` → Trusted Publishers → Add,填
|
|
61
|
+
Provider `GitHub Actions` / Organization or user `PensiveFei` / Repository `deep-read-summarize` /
|
|
62
|
+
Workflow filename `publish.yml` / Environment 留空。
|
|
63
|
+
|
|
64
|
+
**历史 tag 里没有这个 workflow 时**(workflow 定义从 main 取,包内容从 tag 取):
|
|
65
|
+
|
|
66
|
+
```bash
|
|
67
|
+
gh workflow run publish.yml --ref main -f publish_ref=refs/tags/vX.Y.Z
|
|
68
|
+
gh run watch
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
### 回退:本地发布(需要交互式 2FA)
|
|
72
|
+
|
|
73
|
+
> ⚠️ 2026-07 起 npm 已限制「绕过 2FA 的 granular access token」用于**直接发布**,
|
|
74
|
+
> 所以本地发布只能靠交互式 OTP —— 不再有「建个长期 token 一劳永逸」这条路。
|
|
36
75
|
|
|
37
76
|
```bash
|
|
38
77
|
cd <项目目录>
|
|
39
|
-
npm run lint && npm test && npm run validate # 先本地全绿
|
|
40
|
-
npm
|
|
41
|
-
npm
|
|
42
|
-
npm view deep-read-summarize --prefer-online # 验证线上(注意:发布后首次查询可能 404,索引有延迟)
|
|
78
|
+
npm run lint && npm test && npm run test:node && npm run validate # 先本地全绿
|
|
79
|
+
npm publish --otp=<6位码> # prepublishOnly 自动跑门禁
|
|
80
|
+
npm view deep-read-summarize --prefer-online # 验证线上(首次查询可能索引延迟)
|
|
43
81
|
```
|
|
44
82
|
|
|
45
|
-
-
|
|
46
|
-
- 本沙箱环境 npm 默认缓存目录可能被拒(EPERM),用 npm publish --cache <工作区内目录>
|
|
83
|
+
- 本沙箱环境 npm 默认缓存目录可能被拒(EPERM),用 `npm publish --cache <可写目录>`
|
|
47
84
|
- 版本号:功能增强 → minor(0.x → 0.y);修复 → patch;不兼容 → major
|
|
48
85
|
|
|
49
86
|
## 当前版本速览
|
package/index.js
CHANGED
|
@@ -1,11 +1,15 @@
|
|
|
1
1
|
// ============================================================================
|
|
2
2
|
// deep-read-summarize — DSH plugin entry
|
|
3
3
|
// ============================================================================
|
|
4
|
-
// 标准 Cordis 插件(DSH 0.1.x 契约,已在 0.1.2-rc.1 实测):导出 name + inject + apply(ctx)。
|
|
4
|
+
// 标准 Cordis 插件(DSH 0.1.x 契约,已在 0.1.2-rc.1 实测):导出 name + inject + apply(ctx, config)。
|
|
5
5
|
// apply() 将插件自带的 SKILL.md 注册为运行时技能,并把 workflow 的
|
|
6
6
|
// meta + 自包含 script + args 示例嵌入技能内容,使模型可直接用 workflow
|
|
7
7
|
// 工具执行精读流程。
|
|
8
8
|
//
|
|
9
|
+
// config:插件级默认 options(cordis.patch.yml 的 config 段)。此前 apply()
|
|
10
|
+
// 收了 config 却从不读它——配置文件里写的那些键全是空转。现在这些键会被
|
|
11
|
+
// 校验、合并进技能里的 args 示例,成为该 profile 的默认值。
|
|
12
|
+
//
|
|
9
13
|
// 同时保留 load / workflow / parsers / schemas 导出,兼容旧式加载读取。
|
|
10
14
|
// 安装方式(dsh profile 目录): pnpm add ./xxx.tgz → dsh.profile.bundles 追加
|
|
11
15
|
// ============================================================================
|
|
@@ -20,10 +24,58 @@ const schemas = require('./schemas');
|
|
|
20
24
|
module.exports.name = 'deep-read-summarize';
|
|
21
25
|
module.exports.inject = ['skills'];
|
|
22
26
|
|
|
27
|
+
/** workflow 的 options 默认值(与 workflow 脚本内的默认值保持一致)。 */
|
|
28
|
+
const DEFAULT_OPTIONS = {
|
|
29
|
+
minWords: 2500,
|
|
30
|
+
fastMode: false,
|
|
31
|
+
maxChunks: 4,
|
|
32
|
+
transcribe: true,
|
|
33
|
+
requireCitations: true,
|
|
34
|
+
includeTimestamps: false,
|
|
35
|
+
outputDir: './output',
|
|
36
|
+
tempDir: './.tmp'
|
|
37
|
+
};
|
|
38
|
+
|
|
39
|
+
const OPTION_TYPES = {
|
|
40
|
+
minWords: 'number',
|
|
41
|
+
fastMode: 'boolean',
|
|
42
|
+
maxChunks: 'number',
|
|
43
|
+
transcribe: 'boolean',
|
|
44
|
+
requireCitations: 'boolean',
|
|
45
|
+
includeTimestamps: 'boolean',
|
|
46
|
+
outputDir: 'string',
|
|
47
|
+
tempDir: 'string'
|
|
48
|
+
};
|
|
49
|
+
|
|
50
|
+
/**
|
|
51
|
+
* 归一化插件 config:只认白名单里的键,类型不符就告警并忽略(不抛错,
|
|
52
|
+
* 免得一个拼错的配置项把整个 profile 拖死)。
|
|
53
|
+
* 支持 `config: { outputDir: ... }` 与 `config: { options: { outputDir: ... } }` 两种写法。
|
|
54
|
+
*/
|
|
55
|
+
function normalizeOptions(config) {
|
|
56
|
+
const opts = { ...DEFAULT_OPTIONS };
|
|
57
|
+
const raw = (config && config.options) || config || {};
|
|
58
|
+
for (const key of Object.keys(DEFAULT_OPTIONS)) {
|
|
59
|
+
const v = raw[key];
|
|
60
|
+
if (v === undefined) continue;
|
|
61
|
+
if (typeof v !== OPTION_TYPES[key]) {
|
|
62
|
+
console.warn('[deep-read-summarize] ignoring config.' + key + ' — expected ' + OPTION_TYPES[key] + ', got ' + typeof v);
|
|
63
|
+
continue;
|
|
64
|
+
}
|
|
65
|
+
opts[key] = v;
|
|
66
|
+
}
|
|
67
|
+
return opts;
|
|
68
|
+
}
|
|
69
|
+
|
|
23
70
|
/** 组装技能内容:SKILL.md + workflow meta/script/args 示例(script 为自包含版本)。 */
|
|
24
|
-
function buildSkillContent() {
|
|
71
|
+
function buildSkillContent(config = {}) {
|
|
25
72
|
const skillDir = path.join(__dirname, 'skills', 'deep-read-summarize');
|
|
26
73
|
const body = readFileSync(path.join(skillDir, 'SKILL.md'), 'utf8');
|
|
74
|
+
const argsExample = {
|
|
75
|
+
input: '<链接或本地路径>',
|
|
76
|
+
type: 'auto',
|
|
77
|
+
options: normalizeOptions(config)
|
|
78
|
+
};
|
|
27
79
|
return [
|
|
28
80
|
body.trim(),
|
|
29
81
|
'',
|
|
@@ -35,14 +87,14 @@ function buildSkillContent() {
|
|
|
35
87
|
JSON.stringify(workflow.meta, null, 2),
|
|
36
88
|
'```',
|
|
37
89
|
'',
|
|
38
|
-
'script(完整脚本,原样填入 script
|
|
90
|
+
'script(完整脚本,原样填入 script 参数;脚本已自包含解析器,无需宿主注入):',
|
|
39
91
|
'```javascript',
|
|
40
92
|
workflow.script,
|
|
41
93
|
'```',
|
|
42
94
|
'',
|
|
43
|
-
'args
|
|
95
|
+
'args 示例(options 是本 profile 的默认值,用户另有要求时再覆盖):',
|
|
44
96
|
'```json',
|
|
45
|
-
|
|
97
|
+
JSON.stringify(argsExample, null, 2),
|
|
46
98
|
'```'
|
|
47
99
|
].join('\n');
|
|
48
100
|
}
|
|
@@ -54,7 +106,7 @@ function apply(ctx, config = {}) {
|
|
|
54
106
|
name: 'deep-read-summarize',
|
|
55
107
|
description: workflow.meta.description,
|
|
56
108
|
whenToUse: workflow.meta.whenToUse,
|
|
57
|
-
content: buildSkillContent(),
|
|
109
|
+
content: buildSkillContent(config),
|
|
58
110
|
resourceBase: { kind: 'directory', path: skillDir },
|
|
59
111
|
// source 是技能的「来源桶」(runtime / bundled / user-dsh …),不是自由描述:
|
|
60
112
|
// 本技能随插件包分发,因此登记为 bundled。
|
|
@@ -73,3 +125,5 @@ module.exports.workflow = workflow;
|
|
|
73
125
|
module.exports.parsers = parsers;
|
|
74
126
|
module.exports.schemas = schemas;
|
|
75
127
|
module.exports.buildSkillContent = buildSkillContent;
|
|
128
|
+
module.exports.DEFAULT_OPTIONS = DEFAULT_OPTIONS;
|
|
129
|
+
module.exports.normalizeOptions = normalizeOptions;
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "deep-read-summarize",
|
|
3
|
-
"version": "0.3.
|
|
4
|
-
"description": "Deep reading & summarization for DSH: books/papers/videos/web → structured Obsidian notes. Plugin parsers, MapReduce deep-read, JSON Schema
|
|
3
|
+
"version": "0.3.9",
|
|
4
|
+
"description": "Deep reading & summarization for DSH: books/papers/videos/web → structured Obsidian notes. Plugin parsers, MapReduce deep-read, JSON Schema-constrained sub-tasks.",
|
|
5
5
|
"type": "commonjs",
|
|
6
6
|
"main": "index.js",
|
|
7
7
|
"exports": {
|
|
@@ -9,6 +9,7 @@
|
|
|
9
9
|
"./workflow": "./workflow.js",
|
|
10
10
|
"./parsers": "./parsers/index.js",
|
|
11
11
|
"./schemas": "./schemas/index.js",
|
|
12
|
+
"./lib/cache": "./lib/cache.js",
|
|
12
13
|
"./package.json": "./package.json"
|
|
13
14
|
},
|
|
14
15
|
"files": [
|
|
@@ -35,7 +36,7 @@
|
|
|
35
36
|
"validate": "node tests/quick_validate.js && node scripts/security-check.js",
|
|
36
37
|
"security": "node scripts/security-check.js",
|
|
37
38
|
"test:node": "node --test",
|
|
38
|
-
"prepublishOnly": "node tests/run-tests.js && node scripts/lint.js && node scripts/security-check.js"
|
|
39
|
+
"prepublishOnly": "node tests/run-tests.js && node --test && node scripts/lint.js && node scripts/security-check.js"
|
|
39
40
|
},
|
|
40
41
|
"dsh": {
|
|
41
42
|
"bundle": {
|
package/parsers/book.js
CHANGED
|
@@ -20,7 +20,7 @@ module.exports = {
|
|
|
20
20
|
fetchTarget: "书籍全文",
|
|
21
21
|
kindLabel: "书籍(book)",
|
|
22
22
|
steps: [
|
|
23
|
-
"获取全文:PDF
|
|
23
|
+
"获取全文:PDF→优先用 read 读取;读不出正文时用 pdftotext(poppler 命令行工具,三平台通用;未装时按系统装:macOS `brew install poppler`、Windows `winget install poppler`、Debian/Ubuntu `apt install poppler-utils`);扫描版 PDF 需 OCR;EPUB/MOBI→解包或转文本;纯文本→直接读取。",
|
|
24
24
|
"全文写入目标文件(不截断、不加评注),用 read 确认总行数。",
|
|
25
25
|
"识别书籍结构:目录/序言/各章节标题及其行号范围。"
|
|
26
26
|
],
|
package/parsers/video.js
CHANGED
|
@@ -6,6 +6,12 @@
|
|
|
6
6
|
// ② 无公开字幕 → 用插件自带转写(faster-whisper small/int8/VAD,镜像+缓存)得到全文
|
|
7
7
|
// ③ 都不行 → 降级(提示人工提供转写文本,或退回 desc 作背景)。
|
|
8
8
|
// 本机/用户使用一致:只依赖环境变量 + 相对/标准路径,不写死本机绝对路径。
|
|
9
|
+
// 跨平台:Windows / macOS / Linux 同等支持。转写脚本是 Node 写的(scripts/transcribe.js),
|
|
10
|
+
// 三平台**命令行完全一致**;平台差异只在「脚本定位命令」与「yt-dlp 安装方式」上,
|
|
11
|
+
// 两套写法都在提示词里给出,由执行代理按自己的系统选择。
|
|
12
|
+
// ⚠️ 本提示词会被内联进 workflow 沙箱脚本执行,沙箱**不提供 process / require**,
|
|
13
|
+
// 所以这里不能按平台做分支(写了 process.platform 会直接被回归测试拦下),
|
|
14
|
+
// 只能把各平台的写法都列出来。详见 tests/run-tests.js 的沙箱 API 断言。
|
|
9
15
|
// 提示词骨架复用 parsers/_prompt.js,此处只提供差异化配置。
|
|
10
16
|
// ============================================================================
|
|
11
17
|
|
|
@@ -14,36 +20,66 @@ const prompt = require("./_prompt");
|
|
|
14
20
|
module.exports = {
|
|
15
21
|
name: "video",
|
|
16
22
|
types: ["video"],
|
|
17
|
-
description: "视频解析器(B站/抖音/YouTube →
|
|
23
|
+
description: "视频解析器(B站/抖音/YouTube → 完整逐字稿,字幕优先,无字幕自动转写;Windows/macOS/Linux)",
|
|
18
24
|
|
|
19
25
|
buildPrompt: function (input, opts) {
|
|
20
|
-
const
|
|
21
|
-
const
|
|
26
|
+
const o = opts || {};
|
|
27
|
+
const tempDir = o.tempDir || "./.tmp";
|
|
28
|
+
// 默认 true:无字幕时自动走本地转写。false 时必须真的改变提示词——
|
|
29
|
+
// 此前这里算出来的值从未被使用,transcribe:false 与 true 生成的提示词逐字相同。
|
|
30
|
+
const wantTranscribe = o.transcribe !== false;
|
|
31
|
+
const whisperModel = o.whisperModel || "small";
|
|
32
|
+
const language = o.language || "zh";
|
|
33
|
+
const device = o.device || "cpu";
|
|
22
34
|
const degradeMsg = "需要人工提供转写文本或视频文案";
|
|
35
|
+
|
|
36
|
+
const noSubtitleBranch = wantTranscribe
|
|
37
|
+
? "② 若**无公开字幕**,用插件自带 faster-whisper 转写得到全文;③ 仍不可行才降级(提示人工提供转写文本 或 退回 desc 作背景)。"
|
|
38
|
+
: "② **本次已禁用转写**(options.transcribe=false):无公开字幕时直接降级(提示人工提供转写文本 或 退回 desc 作背景),不要尝试转写。";
|
|
39
|
+
const biliNoSubtitle = wantTranscribe
|
|
40
|
+
? " 3) 无公开字幕时(textSource=transcription):按下方「转写执行」用 faster-whisper 转写整段音频,得到完整全文。"
|
|
41
|
+
: " 3) 无公开字幕时:转写已被禁用(options.transcribe=false),按降级处理,不要取音频、不要转写。";
|
|
42
|
+
const douyinTail = wantTranscribe
|
|
43
|
+
? "desc 较完整时可直接作为文本(textSource=desc);若 desc 太短,仍按下方「转写执行」转写(textSource=transcription)。"
|
|
44
|
+
: "desc 较完整时可直接作为文本(textSource=desc);若 desc 太短,直接降级提示用户补充文案(转写已禁用)。";
|
|
45
|
+
const ytTail = wantTranscribe
|
|
46
|
+
? "已安装时用 yt-dlp 抓字幕(CC)转文本;无 CC 则转写。"
|
|
47
|
+
: "已安装时用 yt-dlp 抓字幕(CC)转文本;无 CC 则降级(转写已禁用)。";
|
|
48
|
+
|
|
23
49
|
const steps = [
|
|
24
50
|
"记录文本来源(metadata.textSource):subtitle(平台字幕)/ transcription(本地转写)/ desc(仅简介,作为降级/背景)/ manual(用户提供)。",
|
|
25
|
-
"统一流程:① 先取视频字幕(B站 AI 字幕 / YouTube yt-dlp CC
|
|
26
|
-
"若输入是 B 站链接(bilibili.com / b23.tv,短链先用 curl -L 跟随重定向提取 BV 号):用 curl(Windows
|
|
51
|
+
"统一流程:① 先取视频字幕(B站 AI 字幕 / YouTube yt-dlp CC);" + noSubtitleBranch,
|
|
52
|
+
"若输入是 B 站链接(bilibili.com / b23.tv,短链先用 curl -L 跟随重定向提取 BV 号):用 curl(Windows 10+ 与 macOS/Linux 均自带,零安装)调官方公开 API:\n" +
|
|
27
53
|
" 1) 元数据+简介:curl 'https://api.bilibili.com/x/web-interface/view?bvid=<BV>' → 取 title / desc(简介仅作背景/补充,不满一篇精读)/ owner.name / duration / aid / cid\n" +
|
|
28
54
|
" 2) 字幕(**优先,最快**):curl 'https://api.bilibili.com/x/player/v2?aid=<aid>&cid=<cid>' 取 subtitle 列表;若有则下载转文本(textSource=subtitle)——这已是完整逐字稿,直接使用\n" +
|
|
29
|
-
|
|
55
|
+
biliNoSubtitle,
|
|
30
56
|
|
|
31
|
-
"若输入是抖音链接/分享口令(douyin.com / v.douyin.com):用户分享时复制的文字本身就是该视频的文案(desc)。
|
|
32
|
-
"若输入是 YouTube 等其他平台:先探测是否已装 yt-dlp
|
|
57
|
+
"若输入是抖音链接/分享口令(douyin.com / v.douyin.com):用户分享时复制的文字本身就是该视频的文案(desc)。" + douyinTail,
|
|
58
|
+
"若输入是 YouTube 等其他平台:先探测是否已装 yt-dlp(运行 `yt-dlp --version`)。若未安装:绝对不要去下载 yt-dlp 二进制(GitHub 直连在部分网络下极慢且易中断,会卡死整个流程);**按系统装**——Windows 先试 `winget install yt-dlp.yt-dlp`,macOS 先试 `brew install yt-dlp`,三平台都可再试 `pip install -U yt-dlp -i https://pypi.tuna.tsinghua.edu.cn/simple`;" +
|
|
59
|
+
(wantTranscribe ? "仍不可用则按下方「转写执行」直接转写或降级。" : "仍不可用则直接降级(转写已禁用)。") + ytTail
|
|
60
|
+
];
|
|
33
61
|
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
"
|
|
37
|
-
"
|
|
38
|
-
|
|
39
|
-
|
|
62
|
+
if (wantTranscribe) {
|
|
63
|
+
steps.push(
|
|
64
|
+
"转写执行(faster-whisper " + whisperModel + " / int8 / " + language + ",镜像+缓存,保证本机与用户一致):\n" +
|
|
65
|
+
" ① 先取音频:B站用 playurl(curl 'https://api.bilibili.com/x/player/playurl?avid=<aid>&cid=<cid>&fnval=16&fourk=1' 的 dash.audio[0].baseUrl)或 yt-dlp,下载存到 '" + tempDir + "/audio.m4a';\n" +
|
|
66
|
+
" ② 定位插件转写脚本(文件名固定 `transcribe.js`):Windows 用 `Get-ChildItem -Path $env:USERPROFILE/.dsh -Recurse -Filter transcribe.js`;macOS/Linux 用 `find \"$HOME/.dsh\" -name transcribe.js`(也可直接用 glob 搜 *transcribe.js);\n" +
|
|
67
|
+
" ③ 运行(**三平台命令行完全一致**,脚本是 Node 写的,与 pwsh / bash 无关):`node <脚本路径> --audio '" + tempDir + "/audio.m4a' --out '" + tempDir + "/transcript.txt' --device " + device + " --model " + whisperModel + " --language " + language + "`;\n" +
|
|
68
|
+
" ④ 转写可能较慢:用 run_in_background: true 启动后,轮询输出文件 '" + tempDir + "/transcript.txt' 是否写出内容(非空且稳定),或日志出现 DONE;设合理超时,超时/失败则降级;\n" +
|
|
69
|
+
" ⑤ 成功后读取 transcript.txt 作为全文(textSource=transcription)。脚本自举:uv 建 Python 3.12 环境 + 镜像装 faster-whisper + hf-mirror 下模型并缓存;Windows/macOS/Linux 是**同一份实现**。**⚠️ 首次转写会先下载模型(small 约 484MB,hf-mirror,非 GitHub)并较慢——先向用户说明这是正常的一次性下载,勿当成卡死;之后缓存复用、秒开。**\n" +
|
|
70
|
+
" ⑥ 若 `node` 不在 PATH(macOS 从图形界面启动时常见),先用 `command -v node` 确认;确认不可用就直接降级,不要为了跑转写脚本去安装 Node。"
|
|
71
|
+
);
|
|
72
|
+
} else {
|
|
73
|
+
steps.push(
|
|
74
|
+
"【转写已禁用】options.transcribe=false:只允许使用平台字幕或简介(desc)。不要定位或运行 scripts/transcribe.js,不要下载模型,不要安装任何转写工具;拿不到字幕就按降级处理。"
|
|
75
|
+
);
|
|
76
|
+
}
|
|
40
77
|
|
|
41
|
-
|
|
42
|
-
];
|
|
78
|
+
steps.push('若以上均不可用或抓取失败:输出 { "saved": false, "message": "' + degradeMsg + '" }。');
|
|
43
79
|
|
|
44
80
|
return prompt.buildFetchPrompt({
|
|
45
81
|
input: input,
|
|
46
|
-
opts:
|
|
82
|
+
opts: o,
|
|
47
83
|
fetchTarget: "视频完整逐字稿(字幕优先,无字幕自动转写)",
|
|
48
84
|
kindLabel: "视频(video)",
|
|
49
85
|
steps: steps,
|
|
@@ -51,4 +87,4 @@ module.exports = {
|
|
|
51
87
|
jsonExample: '{ "kind": "video", "saved": true, "totalLines": N, "metadata": { "title": "", "author": "", "year": "", "language": "zh|en", "textSource": "subtitle|transcription|desc|manual" }, "chunkPlan": [ { "id": 1, "topic": "主题段", "startLine": 1, "endLine": 200 } ] }'
|
|
52
88
|
});
|
|
53
89
|
}
|
|
54
|
-
};
|
|
90
|
+
};
|