dsh-tingdong 0.0.0-stage → 0.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +125 -2
- package/bin/dsh-tingdong.js +141 -0
- package/lib/asr-worker.js +111 -0
- package/lib/asr.js +116 -0
- package/lib/cli-llm.js +54 -0
- package/lib/drill.js +112 -0
- package/lib/edge.js +94 -0
- package/lib/export.js +92 -0
- package/lib/importer.js +324 -0
- package/lib/llm.js +66 -0
- package/lib/lookup.js +68 -0
- package/lib/models.js +93 -0
- package/lib/paths.js +44 -0
- package/lib/pcm.js +115 -0
- package/lib/server.js +299 -0
- package/lib/srs.js +71 -0
- package/lib/store.js +237 -0
- package/lib/today.js +102 -0
- package/lib/tracks.js +280 -0
- package/lib/translate.js +165 -0
- package/lib/tts-worker.js +178 -0
- package/lib/voices.js +142 -0
- package/lib/vps.js +173 -0
- package/package.json +65 -4
- package/plugin/client.js +143 -0
- package/plugin/cordis.patch.yml +6 -0
- package/plugin/index.js +140 -0
- package/plugin/manage.js +129 -0
- package/plugin/tools.js +132 -0
- package/web/core.js +283 -0
- package/web/dict.js +22 -0
- package/web/edit.js +108 -0
- package/web/listen.js +250 -0
- package/web/panel.css +310 -0
- package/web/panel.html +26 -0
- package/web/panel.js +773 -0
- package/web/study.js +345 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 AIcivilization
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,3 +1,126 @@
|
|
|
1
|
-
#
|
|
1
|
+
# 听懂 · dsh-tingdong
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
把英文有声读物(MP3)变成听力课,练到**听到英文就知道是什么意思**。
|
|
4
|
+
[DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness)(dsh)插件,也能脱离 DSH 单独用。
|
|
5
|
+
|
|
6
|
+
## 它做什么
|
|
7
|
+
|
|
8
|
+
1. **导入**:选一个 MP3(或整个文件夹),本机识别出每一句英文和它的起止时间。
|
|
9
|
+
识别用的是 DSH 自带语音输入的同一套模型(SenseVoice + Silero VAD),音频不上传;用过 DSH 语音输入就不用再下载。
|
|
10
|
+
2. **讲解**:用你在 DSH 里选的大模型逐句校对英文(修正识别错误,如 "resna tuning" → "resonant tuning")、
|
|
11
|
+
翻成口语化中文、挑出生词(带音标)、写一句「听音提示」(哪里连读、弱读、吞音),再写全篇概述和关键术语表。
|
|
12
|
+
3. **练习**(右侧栏「听懂」页签,或手机浏览器直接打开):
|
|
13
|
+
|
|
14
|
+
| 模式 | 做法 |
|
|
15
|
+
| --- | --- |
|
|
16
|
+
| 今日学习 | 首页一个按钮:所有课里该复习的句子 + 每天 10 句新句子串成一组;显示连续学习天数、今天学了几句、约几分钟、最近 7 天 |
|
|
17
|
+
| 精听 | 每句先**只听不看**猜意思 → 看英文找出没听出来的词(**点任意一个词看意思**)→ 看中文和生词 → 再听一遍 → 自评「没听懂 / 半懂 / 听懂了」 |
|
|
18
|
+
| 复习 | 间隔重复,三种考法:**自评**、**选意思**(只听声音,三选一)、**听写**(打出来,逐词标出没听出来的)。没听懂的一分钟后再来,听懂的隔 10 分钟、1 天、3 天、7 天…再考;7 天后还能听懂算掌握。可以只练「收藏」或「难句」 |
|
|
19
|
+
| 磨耳朵 | 不用动手:每句放 1–4 遍,停几秒让你想意思,再接下一句;字幕可以逐步显示(第 1 遍不看字 → 第 2 遍英文 → 想的时候中文)。做成一段真正的音频,**锁屏也能一直放**,耳机线控切句,也能下载 |
|
|
20
|
+
| 连续听 | 一句接一句不停地播,英文 / 中文字幕分别开关;最后关掉字幕整篇听 |
|
|
21
|
+
| 词汇 | 全篇关键术语、每句的生词、你查过的词,点一下听原句 |
|
|
22
|
+
|
|
23
|
+
**导出**(每课的「导出」页签):
|
|
24
|
+
|
|
25
|
+
- **中文配音(音频)**:可以下载 **MP3** 或无损 **FLAC**(44.1 kHz / 16 位,和合成出来的原始音频同源,不是 MP3 转的)。把英文换成中文配音,只有中文、没有背景音,每句中文从对应英文开始的时刻开始说,总长和原音频一样(也可以对齐「原声·去空白」)。比空档长的句子自动读快一点(最快 1.6 倍),不会挤到下一句。两种语音可选:
|
|
26
|
+
- **本地 Kokoro**:离线,55 个普通话女声、45 个普通话男声,每个都能先试听;第一次用时下载约 147 MB 模型
|
|
27
|
+
- **微软 Edge**:晓晓、云希、云扬等 14 种(含台湾、粤语、方言),要联网。插件调用开源工具 [edge-tts](https://github.com/rany2/edge-tts),点「安装 Edge 语音组件」会装进插件自己的 Python 虚拟环境(需要电脑上有 Python 3)。文字会发到微软的服务器;这不是微软公开的正式接口,可能失效
|
|
28
|
+
- **文字**:双语 / 英文 / 中文字幕(SRT)、双语歌词 LRC(Walkman 等播放器滚动显示)、**录音稿**(每句的开始时间、英文时长、可用时长和中英对照,照着自己录中文配音)、表格 CSV。时间可按原音频或「原声·去空白」
|
|
29
|
+
|
|
30
|
+
卡片右上角 ☆ 收藏,复习和磨耳朵里可以只练收藏的
|
|
31
|
+
- **改句子**:识别偶尔认错字、切错句,卡片右上角 ✎ 可以改英文 / 中文、和下一句合并、拆成两句、重新讲解、标为非讲话
|
|
32
|
+
- **换设备接着学**:学到第几句、选的哪种声音、复习进度都存在服务器上
|
|
33
|
+
- **手机**:按钮加大,左右滑动换句,评分按钮固定在底部,锁屏界面和耳机线控可以播放 / 暂停 / 换句
|
|
34
|
+
|
|
35
|
+
**声音**:同一课可以换声音听,在课程页上方切换,每种都能下载成 MP3 放到手机上:
|
|
36
|
+
|
|
37
|
+
| 声音 | 说明 |
|
|
38
|
+
| --- | --- |
|
|
39
|
+
| 原声(完整) | 原始音频,含背景音乐和长空白 |
|
|
40
|
+
| 原声·去空白 | 只留讲话,句间停 0.7 秒。34 分钟的引导冥想剩 11 分 40 秒 |
|
|
41
|
+
| 原声·压背景 | 同上,再滤掉低沉的双耳节拍和高频噪声 |
|
|
42
|
+
| 系统朗读 | macOS 自带语音读校对后的英文;在系统设置里下载「高级 / 增强」英文语音后会自动用上 |
|
|
43
|
+
| Kokoro 朗读 | 离线神经网络语音(11 种美式 / 英式男女声,可调语速),第一次用时下载约 103 MB 模型 |
|
|
44
|
+
|
|
45
|
+
新导入的课会自动生成两条原声轨,朗读的声音在「概览 → 声音」里点「生成」。
|
|
46
|
+
|
|
47
|
+
电脑快捷键:空格 重听 · 回车 下一步 · 1/2/3 评分 · ←/→ 上一句/下一句 · S 变速(1× / 0.85× / 0.7×)· L 单句循环
|
|
48
|
+
|
|
49
|
+
在 DSH 对话里也可以直接说:「把 /Volumes/…/CD4 导入听懂」「我这课哪些句子总听不懂?给我讲讲」——插件给 AI 注册了 `tingdong_import`、`tingdong_lessons`、`tingdong_sentences` 三个工具。
|
|
50
|
+
|
|
51
|
+
**更新 / 卸载**:首页标题旁显示版本号,打开时自动查 npm 上有没有新版本;有就出现「更新到 vX」,点一下由 DSH 的插件管理器装好(和插件市场同一套机制),重启 DSH 生效。「卸载」只移除插件,课程和学习进度留在 `~/.dsh/dsh-tingdong`。
|
|
52
|
+
|
|
53
|
+
## 安装
|
|
54
|
+
|
|
55
|
+
需要 [ffmpeg](https://ffmpeg.org)(解码 MP3):Mac 上 `brew install ffmpeg`。
|
|
56
|
+
|
|
57
|
+
装进 DSH(把 `desktop` 换成你的 profile 名;先完全退出 DSH):
|
|
58
|
+
|
|
59
|
+
```bash
|
|
60
|
+
dsh plugin --profile desktop add /path/to/dsh-go
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
重新打开 DSH,在对话页打开右侧栏,选「听懂 · 英文听力」。设置里的「听懂」分区会显示 ffmpeg、识别模型、翻译模型是否就绪。
|
|
64
|
+
|
|
65
|
+
### 可选配置
|
|
66
|
+
|
|
67
|
+
在 profile 的 `cordis.patch.yml` 里:
|
|
68
|
+
|
|
69
|
+
```yaml
|
|
70
|
+
- id: dsh-tingdong
|
|
71
|
+
config:
|
|
72
|
+
provider: dsh-model-workbuddy-ai # 翻译用的模型;不填就用 DSH 的默认模型
|
|
73
|
+
model: workbuddy-ai/deepseek-v4.1-flash-sg
|
|
74
|
+
ffmpeg: /opt/homebrew/bin/ffmpeg # 找不到 ffmpeg 时填
|
|
75
|
+
threads: 2 # 识别线程数
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
## 用手机学(VPS)
|
|
79
|
+
|
|
80
|
+
在电脑上做课(识别、翻译、生成声音都很吃 CPU),发到 VPS 上的 DSH,手机浏览器打开学习:
|
|
81
|
+
|
|
82
|
+
1. VPS 上装好 DSH(比如用 [dsh-vps](https://github.com/AIcivilization/dsh-vps)),也装上本插件:
|
|
83
|
+
`dsh plugin --profile web add dsh-tingdong`(以运行 DSH 的用户执行,再重启 DSH)。VPS 上不需要 ffmpeg。
|
|
84
|
+
2. 电脑上的「听懂」首页底部「手机学习(VPS)」:选 SSH 主机(`~/.ssh/config` 里的,要能用钥匙免密码登录)→「查找 VPS 上的 DSH」→ 选一个。
|
|
85
|
+
3. 点「全部发送到 VPS」,或在每课「管理」里单独发送。只发课程内容和声音,**不会覆盖 VPS 上的学习进度**。
|
|
86
|
+
4. 手机打开 `https://<你的域名>/api-dsh-tingdong/panel`(首页会显示这个地址),可以「添加到主屏幕」当 App 用。
|
|
87
|
+
|
|
88
|
+
发送走 SSH(tar 流解到 VPS 上 DSH 用户的 `~/.dsh/dsh-tingdong/lessons/`),不经过网页登录门。
|
|
89
|
+
|
|
90
|
+
## 不开 DSH 也能用
|
|
91
|
+
|
|
92
|
+
```bash
|
|
93
|
+
node bin/dsh-tingdong.js import "/Volumes/kuozhan/门罗/CD4" # 整个文件夹,每个音频一课
|
|
94
|
+
node bin/dsh-tingdong.js serve # 打开 http://127.0.0.1:4810
|
|
95
|
+
node bin/dsh-tingdong.js list
|
|
96
|
+
node bin/dsh-tingdong.js voices all --track kokoro --voice am_michael # 批量生成声音轨
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
命令行版的模型:`--base-url / --api-key / --model`,或环境变量 `TINGDONG_BASE_URL / TINGDONG_API_KEY / TINGDONG_MODEL`(任何 OpenAI 兼容接口);都没给就用本机的 dsh-model 网关。
|
|
100
|
+
|
|
101
|
+
## 数据
|
|
102
|
+
|
|
103
|
+
都在 `~/.dsh/dsh-tingdong/lessons/<课程>/`:`lesson.json`(句子、翻译、讲解、查过的词)、`progress.json`(复习进度、学到第几句、选的声音、收藏)、`audio.mp3`(压缩成 64 kbps 单声道,原文件所在的移动硬盘拔掉也能听)、`tracks/`(各种声音和它们的时间表)。
|
|
104
|
+
模型在 `~/.dsh/dsh-tingdong/models/`(Kokoro 英文、Kokoro 中文);Edge 语音组件在 `~/.dsh/dsh-tingdong/edge-tts/`;识别模型优先复用 DSH 语音输入下好的。
|
|
105
|
+
DSH 插件和命令行版共用这份数据。
|
|
106
|
+
|
|
107
|
+
## 实测
|
|
108
|
+
|
|
109
|
+
Hemi-Sync Gateway Wave IV「Adventure 1 (One Year Patterning)」,34 分钟 MP3(大段音乐 + 引导语):
|
|
110
|
+
识别 16 秒(M 系列 Mac,4 线程),切出 96 句,模型判出 6 处音乐里的杂音并跳过;DeepSeek 翻译讲解约 1 分钟。
|
|
111
|
+
|
|
112
|
+
## 开发
|
|
113
|
+
|
|
114
|
+
```bash
|
|
115
|
+
npm install
|
|
116
|
+
npm test
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
- `lib/asr-worker.js` 识别子进程(sherpa-onnx 是同步推理,放在 DSH 主进程里会卡界面)
|
|
120
|
+
- `lib/translate.js` 校对 + 翻译 + 生词 + 听音提示;缺句自动缩小批次重试,失败的留空,之后可「补全翻译」
|
|
121
|
+
- `lib/tracks.js` 声音轨:原声按句截取拼接;朗读交给 `lib/tts-worker.js` 子进程(Kokoro / macOS say),`lib/pcm.js` 统一拼接编码并出时间表
|
|
122
|
+
- `lib/srs.js` 间隔复习;`lib/store.js` 原子写入的 JSON 存储;`lib/today.js` 今日学习(按用户时区算日期)
|
|
123
|
+
- `lib/export.js` 文字导出;`lib/edge.js` Edge 语音组件(edge-tts)的安装和调用
|
|
124
|
+
- `lib/drill.js` 磨耳朵音频;`lib/lookup.js` 点词查意思、重新讲解;`lib/vps.js` 经 SSH 发送到 VPS
|
|
125
|
+
- `plugin/` DSH 宿主端(路由、AI 工具)和界面入口(右侧栏、设置);路由先过 DSH 的 `connection.requestRejection()` 登录校验
|
|
126
|
+
- `web/` 学习面板(原生 JS 模块,无构建):`panel.js` 路由和首页 · `study.js` 精听、复习、今日学习 · `listen.js` 连续听、磨耳朵、词汇 · `edit.js` 改句子、查词 · `core.js` 播放器和公共状态
|
|
@@ -0,0 +1,141 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
// bin/dsh-tingdong.js — 不开 DSH 也能用:导入音频、打开学习面板
|
|
3
|
+
//
|
|
4
|
+
// dsh-tingdong import <文件或文件夹> [--title 标题] 识别 + 翻译,做成课
|
|
5
|
+
// dsh-tingdong serve [--port 4810] 在浏览器里打开学习面板
|
|
6
|
+
// dsh-tingdong list 看有哪些课
|
|
7
|
+
// dsh-tingdong voices <课程id|all> [--track clean,cleannr,say,kokoro] [--voice 名字] [--speed 0.9]
|
|
8
|
+
// 生成声音轨(去空白原声 / 压背景 / 系统朗读 / Kokoro)
|
|
9
|
+
//
|
|
10
|
+
// 模型:--base-url / --api-key / --model,或环境变量 TINGDONG_BASE_URL / TINGDONG_API_KEY / TINGDONG_MODEL;
|
|
11
|
+
// 都没有就用本机的 dsh-model 网关。
|
|
12
|
+
|
|
13
|
+
import { createServer } from 'node:http'
|
|
14
|
+
import { fileURLToPath } from 'node:url'
|
|
15
|
+
import { cliLlm } from '../lib/cli-llm.js'
|
|
16
|
+
import { Importer } from '../lib/importer.js'
|
|
17
|
+
import { createHandler } from '../lib/server.js'
|
|
18
|
+
import { listLessons } from '../lib/store.js'
|
|
19
|
+
|
|
20
|
+
const PKG_ROOT = fileURLToPath(new URL('..', import.meta.url))
|
|
21
|
+
const PREFIX = '/api-dsh-tingdong'
|
|
22
|
+
|
|
23
|
+
function parse(argv) {
|
|
24
|
+
const flags = {}
|
|
25
|
+
const rest = []
|
|
26
|
+
for (let k = 0; k < argv.length; k++) {
|
|
27
|
+
const a = argv[k]
|
|
28
|
+
if (a.startsWith('--')) {
|
|
29
|
+
const [key, val] = a.slice(2).split('=', 2)
|
|
30
|
+
if (val !== undefined) flags[key] = val
|
|
31
|
+
else if (argv[k + 1] && !argv[k + 1].startsWith('--')) flags[key] = argv[++k]
|
|
32
|
+
else flags[key] = true
|
|
33
|
+
} else rest.push(a)
|
|
34
|
+
}
|
|
35
|
+
return { flags, rest }
|
|
36
|
+
}
|
|
37
|
+
|
|
38
|
+
const fmtTime = s => `${Math.floor(s / 60)}:${String(Math.round(s % 60)).padStart(2, '0')}`
|
|
39
|
+
|
|
40
|
+
async function main() {
|
|
41
|
+
const { flags, rest } = parse(process.argv.slice(2))
|
|
42
|
+
const cmd = rest.shift()
|
|
43
|
+
const options = () => ({ ffmpeg: flags.ffmpeg, threads: Number(flags.threads) || 4 })
|
|
44
|
+
|
|
45
|
+
if (cmd === 'list') {
|
|
46
|
+
for (const l of await listLessons()) {
|
|
47
|
+
const s = l.stats
|
|
48
|
+
console.log(`${l.id} ${l.titleZh || l.title} [${l.status}] ${fmtTime(l.duration)} ${l.sentences} 句 掌握 ${s.mastered}/${s.total} 待复习 ${s.due}`)
|
|
49
|
+
}
|
|
50
|
+
return
|
|
51
|
+
}
|
|
52
|
+
|
|
53
|
+
if (cmd === 'voices') {
|
|
54
|
+
const which = rest[0]
|
|
55
|
+
if (!which) throw new Error('用法:dsh-tingdong voices <课程id|all> [--track clean,cleannr,say,kokoro]')
|
|
56
|
+
const ids = which === 'all' ? (await listLessons()).filter(l => l.sentences).map(l => l.id) : [which]
|
|
57
|
+
const tracks = String(typeof flags.track === 'string' ? flags.track : 'clean,cleannr,say,kokoro').split(',').map(x => x.trim()).filter(Boolean)
|
|
58
|
+
const importer = new Importer({ llm: () => null, options, log: m => console.error(m) })
|
|
59
|
+
let last = ''
|
|
60
|
+
importer.onChange(() => {
|
|
61
|
+
for (const j of importer.list()) {
|
|
62
|
+
if (j.state !== 'running') continue
|
|
63
|
+
const key = `${j.id}|${j.stage}|${j.total ? Math.floor(j.done / j.total * 4) : 0}`
|
|
64
|
+
if (key !== last) { last = key; console.log(`${j.title}:${j.stage}${j.total ? ` ${j.done}/${j.total}` : ''}`) }
|
|
65
|
+
}
|
|
66
|
+
})
|
|
67
|
+
for (const id of ids) for (const t of tracks) {
|
|
68
|
+
await importer.makeTrack(id, t, { voice: typeof flags.voice === 'string' ? flags.voice : undefined, speed: flags.speed ? Number(flags.speed) : undefined })
|
|
69
|
+
}
|
|
70
|
+
await new Promise(resolve => {
|
|
71
|
+
const check = () => { if (!importer.running && !importer.jobs.some(j => j.state === 'queued')) resolve() }
|
|
72
|
+
importer.onChange(() => setImmediate(check))
|
|
73
|
+
check()
|
|
74
|
+
})
|
|
75
|
+
for (const j of importer.list()) console.log(`${j.state === 'done' ? '✓' : '✗'} ${j.title}${j.error ? ':' + j.error : ''}`)
|
|
76
|
+
return
|
|
77
|
+
}
|
|
78
|
+
|
|
79
|
+
if (cmd === 'import') {
|
|
80
|
+
if (!rest[0]) throw new Error('用法:dsh-tingdong import <文件或文件夹>')
|
|
81
|
+
const llm = await cliLlm(flags)
|
|
82
|
+
console.log(llm ? `翻译模型:${llm.label()}(${llm.via})` : '没有可用的大模型:只做英文识别,之后用 serve 打开面板点「补全翻译」')
|
|
83
|
+
const importer = new Importer({ llm: () => llm, options, log: m => console.error(m) })
|
|
84
|
+
// 只在换阶段、或进度每过 20% 时打一行
|
|
85
|
+
let last = ''
|
|
86
|
+
importer.onChange(() => {
|
|
87
|
+
for (const j of importer.list()) {
|
|
88
|
+
if (j.state !== 'running') continue
|
|
89
|
+
const step = j.total ? Math.floor(j.done / j.total * 5) : 0
|
|
90
|
+
const key = `${j.id}|${j.stage}|${step}`
|
|
91
|
+
if (key === last) continue
|
|
92
|
+
last = key
|
|
93
|
+
console.log(`${j.title}:${j.stage}${j.total ? ` ${j.done}/${j.total} ${j.message || ''}` : ''}`)
|
|
94
|
+
}
|
|
95
|
+
})
|
|
96
|
+
const items = await importer.importPath(rest[0], { title: typeof flags.title === 'string' ? flags.title : '' })
|
|
97
|
+
for (const it of items) if (it.existed) console.log(`已经导入过:${it.title}(${it.lessonId})`)
|
|
98
|
+
await new Promise(resolve => {
|
|
99
|
+
const check = () => { if (!importer.running && !importer.jobs.some(j => j.state === 'queued')) resolve() }
|
|
100
|
+
importer.onChange(() => setImmediate(check))
|
|
101
|
+
check()
|
|
102
|
+
})
|
|
103
|
+
for (const j of importer.list()) console.log(`${j.state === 'done' ? '✓' : '✗'} ${j.title}${j.error ? ':' + j.error : ''}`)
|
|
104
|
+
for (const l of await listLessons()) if (items.some(it => it.lessonId === l.id)) {
|
|
105
|
+
console.log(` ${l.titleZh || l.title}:${l.sentences} 句,已翻译 ${l.translated}${l.note ? ';' + l.note : ''}`)
|
|
106
|
+
}
|
|
107
|
+
return
|
|
108
|
+
}
|
|
109
|
+
|
|
110
|
+
if (cmd === 'serve') {
|
|
111
|
+
const llm = await cliLlm(flags)
|
|
112
|
+
const importer = new Importer({ llm: () => llm, options, log: m => console.error(m) })
|
|
113
|
+
const handle = createHandler({
|
|
114
|
+
prefix: PREFIX, pkgRoot: PKG_ROOT, importer, options,
|
|
115
|
+
status: () => ({ host: 'cli', llm: llm ? llm.label() : '' }),
|
|
116
|
+
})
|
|
117
|
+
const port = Number(flags.port) || 4810
|
|
118
|
+
const server = createServer((req, res) => {
|
|
119
|
+
if (req.url === '/' || req.url === '') { res.writeHead(302, { location: PREFIX + '/panel' }); return res.end() }
|
|
120
|
+
if (!String(req.url).startsWith(PREFIX)) { res.writeHead(404); return res.end() }
|
|
121
|
+
handle(req, res)
|
|
122
|
+
})
|
|
123
|
+
server.listen(port, '127.0.0.1', () => {
|
|
124
|
+
console.log(`学习面板:http://127.0.0.1:${port}${PREFIX}/panel`)
|
|
125
|
+
console.log(llm ? `翻译模型:${llm.label()}(${llm.via})` : '没有可用的大模型:导入后只有英文')
|
|
126
|
+
})
|
|
127
|
+
return
|
|
128
|
+
}
|
|
129
|
+
|
|
130
|
+
console.log(`听懂 dsh-tingdong — 把英文有声读物变成听力课
|
|
131
|
+
|
|
132
|
+
dsh-tingdong import <文件或文件夹> [--title 标题]
|
|
133
|
+
dsh-tingdong serve [--port 4810]
|
|
134
|
+
dsh-tingdong list
|
|
135
|
+
dsh-tingdong voices <课程id|all> [--track clean,cleannr,say,kokoro] [--voice 名字] [--speed 0.9]
|
|
136
|
+
|
|
137
|
+
模型:--base-url URL --api-key KEY --model ID(或 TINGDONG_BASE_URL / TINGDONG_API_KEY / TINGDONG_MODEL)
|
|
138
|
+
其他:--ffmpeg <ffmpeg 路径> --threads <识别线程数,默认 4>`)
|
|
139
|
+
}
|
|
140
|
+
|
|
141
|
+
main().catch(e => { console.error(e.message || e); process.exit(1) })
|
|
@@ -0,0 +1,111 @@
|
|
|
1
|
+
// lib/asr-worker.js — 识别子进程:ffmpeg 解码 → Silero VAD 切句 → SenseVoice 识别
|
|
2
|
+
//
|
|
3
|
+
// sherpa-onnx 的推理是同步的,放在 DSH 主进程里会卡住整个界面,所以单独起一个进程
|
|
4
|
+
// (和 DSH 自带语音输入的做法一样:process.execPath + ELECTRON_RUN_AS_NODE=1)。
|
|
5
|
+
// 音频边解码边处理,内存占用和音频长短无关。
|
|
6
|
+
//
|
|
7
|
+
// 输入:argv[2] 是 JSON 参数。输出:stdout 每行一个 JSON 事件:
|
|
8
|
+
// {type:'duration', seconds} {type:'progress', seconds} {type:'segment', start, end, text}
|
|
9
|
+
// {type:'done', segments, inferenceSeconds} {type:'error', message}
|
|
10
|
+
|
|
11
|
+
import { spawn } from 'node:child_process'
|
|
12
|
+
import { createRequire } from 'node:module'
|
|
13
|
+
|
|
14
|
+
const RATE = 16000
|
|
15
|
+
const WINDOW = 512
|
|
16
|
+
|
|
17
|
+
const emit = obj => process.stdout.write(JSON.stringify(obj) + '\n')
|
|
18
|
+
|
|
19
|
+
function main() {
|
|
20
|
+
const opt = JSON.parse(process.argv[2] || '{}')
|
|
21
|
+
const sherpa = createRequire(import.meta.url)('sherpa-onnx-node')
|
|
22
|
+
const threads = opt.threads || 2
|
|
23
|
+
|
|
24
|
+
const recognizer = new sherpa.OfflineRecognizer({
|
|
25
|
+
featConfig: { sampleRate: RATE, featureDim: 80 },
|
|
26
|
+
modelConfig: {
|
|
27
|
+
senseVoice: { model: opt.models.model, language: opt.language || 'en', useInverseTextNormalization: 1 },
|
|
28
|
+
tokens: opt.models.tokens, numThreads: threads, provider: 'cpu', debug: 0,
|
|
29
|
+
},
|
|
30
|
+
})
|
|
31
|
+
const maxSpeech = opt.maxSpeechSeconds || 20
|
|
32
|
+
const minSilence = opt.minSilenceSeconds || 0.6
|
|
33
|
+
const vad = new sherpa.Vad({
|
|
34
|
+
sileroVad: {
|
|
35
|
+
model: opt.models.vad, threshold: opt.vadThreshold || 0.5, minSilenceDuration: minSilence,
|
|
36
|
+
minSpeechDuration: 0.25, maxSpeechDuration: maxSpeech, windowSize: WINDOW,
|
|
37
|
+
},
|
|
38
|
+
sampleRate: RATE, numThreads: 1, provider: 'cpu', debug: 0,
|
|
39
|
+
}, maxSpeech + minSilence + 5)
|
|
40
|
+
|
|
41
|
+
let count = 0
|
|
42
|
+
let inference = 0
|
|
43
|
+
const drain = () => {
|
|
44
|
+
while (!vad.isEmpty()) {
|
|
45
|
+
const seg = vad.front(false)
|
|
46
|
+
const t0 = performance.now()
|
|
47
|
+
const stream = recognizer.createStream()
|
|
48
|
+
stream.acceptWaveform({ sampleRate: RATE, samples: seg.samples })
|
|
49
|
+
recognizer.decode(stream)
|
|
50
|
+
const text = recognizer.getResult(stream).text.trim()
|
|
51
|
+
inference += performance.now() - t0
|
|
52
|
+
if (text) {
|
|
53
|
+
count++
|
|
54
|
+
emit({ type: 'segment', start: seg.start / RATE, end: (seg.start + seg.samples.length) / RATE, text })
|
|
55
|
+
}
|
|
56
|
+
vad.pop()
|
|
57
|
+
}
|
|
58
|
+
}
|
|
59
|
+
|
|
60
|
+
const ff = spawn(opt.ffmpeg, ['-hide_banner', '-nostdin', '-i', opt.file, '-vn', '-ac', '1', '-ar', String(RATE), '-f', 'f32le', 'pipe:1'],
|
|
61
|
+
{ stdio: ['ignore', 'pipe', 'pipe'] })
|
|
62
|
+
let stderr = ''
|
|
63
|
+
ff.stderr.on('data', d => {
|
|
64
|
+
stderr = (stderr + d.toString('utf8')).slice(-8000)
|
|
65
|
+
const m = /Duration:\s*(\d+):(\d+):(\d+(?:\.\d+)?)/.exec(stderr)
|
|
66
|
+
if (m && !main.sentDuration) { main.sentDuration = true; emit({ type: 'duration', seconds: +m[1] * 3600 + +m[2] * 60 + +m[3] }) }
|
|
67
|
+
})
|
|
68
|
+
|
|
69
|
+
let carry = Buffer.alloc(0)
|
|
70
|
+
let fed = 0
|
|
71
|
+
let lastReport = 0
|
|
72
|
+
let pending = new Float32Array(0)
|
|
73
|
+
ff.stdout.on('data', chunk => {
|
|
74
|
+
const buf = carry.length ? Buffer.concat([carry, chunk]) : chunk
|
|
75
|
+
const usable = buf.length - (buf.length % 4)
|
|
76
|
+
carry = buf.subarray(usable)
|
|
77
|
+
// 复制一份:Electron 的 V8 内存笼不允许直接把 Node Buffer 交给原生代码
|
|
78
|
+
const fresh = new Float32Array(usable / 4)
|
|
79
|
+
new Uint8Array(fresh.buffer).set(buf.subarray(0, usable))
|
|
80
|
+
let samples = fresh
|
|
81
|
+
if (pending.length) { samples = new Float32Array(pending.length + fresh.length); samples.set(pending); samples.set(fresh, pending.length) }
|
|
82
|
+
let i = 0
|
|
83
|
+
for (; i + WINDOW <= samples.length; i += WINDOW) {
|
|
84
|
+
vad.acceptWaveform(samples.subarray(i, i + WINDOW))
|
|
85
|
+
drain()
|
|
86
|
+
}
|
|
87
|
+
pending = samples.slice(i)
|
|
88
|
+
fed += usable / 4
|
|
89
|
+
if (fed - lastReport > RATE * 10) { lastReport = fed; emit({ type: 'progress', seconds: fed / RATE }) }
|
|
90
|
+
})
|
|
91
|
+
ff.on('error', e => { emit({ type: 'error', message: `ffmpeg 启动失败:${e.message}` }); process.exit(1) })
|
|
92
|
+
ff.on('close', code => {
|
|
93
|
+
if (code !== 0) {
|
|
94
|
+
emit({ type: 'error', message: `ffmpeg 解码失败(${code}):${stderr.trim().split('\n').slice(-2).join(' ')}` })
|
|
95
|
+
process.exit(1)
|
|
96
|
+
}
|
|
97
|
+
if (pending.length) vad.acceptWaveform(pending)
|
|
98
|
+
vad.flush()
|
|
99
|
+
drain()
|
|
100
|
+
emit({ type: 'progress', seconds: fed / RATE })
|
|
101
|
+
emit({ type: 'done', segments: count, inferenceSeconds: inference / 1000 })
|
|
102
|
+
process.exit(0)
|
|
103
|
+
})
|
|
104
|
+
}
|
|
105
|
+
|
|
106
|
+
try {
|
|
107
|
+
main()
|
|
108
|
+
} catch (e) {
|
|
109
|
+
emit({ type: 'error', message: e && e.message ? e.message : String(e) })
|
|
110
|
+
process.exit(1)
|
|
111
|
+
}
|
package/lib/asr.js
ADDED
|
@@ -0,0 +1,116 @@
|
|
|
1
|
+
// lib/asr.js — 在子进程里识别一个音频文件,返回带时间的句子;顺手做压缩音频
|
|
2
|
+
//
|
|
3
|
+
// 识别结果先按 VAD 切出来的片段给,再用 mergeSegments() 拼成适合练听力的句子:
|
|
4
|
+
// 太碎的("First." "Next.")和后一句接上,太长的不拼。
|
|
5
|
+
|
|
6
|
+
import { spawn } from 'node:child_process'
|
|
7
|
+
import { fileURLToPath } from 'node:url'
|
|
8
|
+
import { findFfmpeg, FFMPEG_HINT } from './paths.js'
|
|
9
|
+
|
|
10
|
+
const WORKER = fileURLToPath(new URL('./asr-worker.js', import.meta.url))
|
|
11
|
+
|
|
12
|
+
/**
|
|
13
|
+
* @param {string} file 音频文件(mp3 / m4a / wav / flac……ffmpeg 能读的都行)
|
|
14
|
+
* @param {object} opt { models, ffmpeg?, threads?, language?, signal?, onEvent? }
|
|
15
|
+
* @returns {Promise<{duration:number, segments:{start:number,end:number,text:string}[]}>}
|
|
16
|
+
*/
|
|
17
|
+
export function transcribe(file, opt) {
|
|
18
|
+
const ffmpeg = findFfmpeg(opt.ffmpeg)
|
|
19
|
+
if (!ffmpeg) return Promise.reject(new Error(FFMPEG_HINT))
|
|
20
|
+
const args = {
|
|
21
|
+
file, ffmpeg, models: opt.models, language: opt.language || 'en', threads: opt.threads || 2,
|
|
22
|
+
vadThreshold: opt.vadThreshold, minSilenceSeconds: opt.minSilenceSeconds, maxSpeechSeconds: opt.maxSpeechSeconds,
|
|
23
|
+
}
|
|
24
|
+
return new Promise((resolve, reject) => {
|
|
25
|
+
// Electron 里 process.execPath 是 App 本体:ELECTRON_RUN_AS_NODE=1 让它当 Node 用
|
|
26
|
+
const child = spawn(process.execPath, [WORKER, JSON.stringify(args)], {
|
|
27
|
+
stdio: ['ignore', 'pipe', 'pipe'],
|
|
28
|
+
env: { ...process.env, ELECTRON_RUN_AS_NODE: '1' },
|
|
29
|
+
})
|
|
30
|
+
const segments = []
|
|
31
|
+
let duration = 0
|
|
32
|
+
let failure = null
|
|
33
|
+
let buf = ''
|
|
34
|
+
let stderr = ''
|
|
35
|
+
const onAbort = () => child.kill('SIGKILL')
|
|
36
|
+
if (opt.signal) {
|
|
37
|
+
if (opt.signal.aborted) onAbort()
|
|
38
|
+
else opt.signal.addEventListener('abort', onAbort, { once: true })
|
|
39
|
+
}
|
|
40
|
+
child.stdout.on('data', d => {
|
|
41
|
+
buf += d.toString('utf8')
|
|
42
|
+
let nl
|
|
43
|
+
while ((nl = buf.indexOf('\n')) >= 0) {
|
|
44
|
+
const line = buf.slice(0, nl)
|
|
45
|
+
buf = buf.slice(nl + 1)
|
|
46
|
+
let ev
|
|
47
|
+
try { ev = JSON.parse(line) } catch { continue }
|
|
48
|
+
if (ev.type === 'segment') segments.push({ start: round(ev.start), end: round(ev.end), text: ev.text })
|
|
49
|
+
else if (ev.type === 'duration') duration = ev.seconds
|
|
50
|
+
else if (ev.type === 'error') failure = ev.message
|
|
51
|
+
if (opt.onEvent) { try { opt.onEvent(ev) } catch { /* 进度回调出错不影响识别 */ } }
|
|
52
|
+
}
|
|
53
|
+
})
|
|
54
|
+
child.stderr.on('data', d => { stderr = (stderr + d).slice(-4000) })
|
|
55
|
+
child.on('error', reject)
|
|
56
|
+
child.on('close', code => {
|
|
57
|
+
if (opt.signal) opt.signal.removeEventListener('abort', onAbort)
|
|
58
|
+
if (opt.signal && opt.signal.aborted) return reject(new Error('已取消'))
|
|
59
|
+
if (code !== 0 || failure) return reject(new Error(failure || `识别进程退出(${code}):${stderr.trim().split('\n').slice(-3).join(' ')}`))
|
|
60
|
+
resolve({ duration, segments })
|
|
61
|
+
})
|
|
62
|
+
})
|
|
63
|
+
}
|
|
64
|
+
|
|
65
|
+
const round = n => Math.round(n * 100) / 100
|
|
66
|
+
|
|
67
|
+
const ENDS = /[.!?。!?…"”']$/
|
|
68
|
+
|
|
69
|
+
/**
|
|
70
|
+
* 把 VAD 片段拼成句子:前一段很短(<1.6s)或没说完(没有句末标点),且停顿 <1.2s、拼完不超过 14s,就接上
|
|
71
|
+
*/
|
|
72
|
+
export function mergeSegments(segs, { maxGap = 1.2, maxLen = 14, short = 1.6 } = {}) {
|
|
73
|
+
const out = []
|
|
74
|
+
for (const s of segs) {
|
|
75
|
+
const text = String(s.text || '').trim()
|
|
76
|
+
if (!text) continue
|
|
77
|
+
const prev = out[out.length - 1]
|
|
78
|
+
if (prev) {
|
|
79
|
+
const gap = s.start - prev.end
|
|
80
|
+
const prevShort = prev.end - prev.start < short
|
|
81
|
+
const unfinished = !ENDS.test(prev.text)
|
|
82
|
+
if (gap < maxGap && (prevShort || unfinished) && s.end - prev.start <= maxLen) {
|
|
83
|
+
prev.end = s.end
|
|
84
|
+
prev.text = joinText(prev.text, text)
|
|
85
|
+
continue
|
|
86
|
+
}
|
|
87
|
+
}
|
|
88
|
+
out.push({ start: s.start, end: s.end, text })
|
|
89
|
+
}
|
|
90
|
+
return out
|
|
91
|
+
}
|
|
92
|
+
|
|
93
|
+
/** "First." + "Your energy…" → "First, your energy…" 不改;只是用空格接上,交给翻译那一步去校对 */
|
|
94
|
+
function joinText(a, b) {
|
|
95
|
+
return `${a} ${b}`.replace(/\s+/g, ' ').trim()
|
|
96
|
+
}
|
|
97
|
+
|
|
98
|
+
/** 压缩成单声道 64 kbps MP3(固定码率,浏览器跳转到某一秒才准) */
|
|
99
|
+
export function compactAudio(src, dest, opt = {}) {
|
|
100
|
+
const ffmpeg = findFfmpeg(opt.ffmpeg)
|
|
101
|
+
if (!ffmpeg) return Promise.reject(new Error(FFMPEG_HINT))
|
|
102
|
+
return new Promise((resolve, reject) => {
|
|
103
|
+
const child = spawn(ffmpeg, ['-hide_banner', '-nostdin', '-y', '-i', src, '-vn', '-map_metadata', '-1', '-ac', '1', '-ar', '44100',
|
|
104
|
+
'-codec:a', 'libmp3lame', '-b:a', '64k', dest], { stdio: ['ignore', 'ignore', 'pipe'] })
|
|
105
|
+
let stderr = ''
|
|
106
|
+
child.stderr.on('data', d => { stderr = (stderr + d).slice(-4000) })
|
|
107
|
+
const onAbort = () => child.kill('SIGKILL')
|
|
108
|
+
if (opt.signal) opt.signal.addEventListener('abort', onAbort, { once: true })
|
|
109
|
+
child.on('error', reject)
|
|
110
|
+
child.on('close', code => {
|
|
111
|
+
if (opt.signal) opt.signal.removeEventListener('abort', onAbort)
|
|
112
|
+
if (code === 0) resolve(dest)
|
|
113
|
+
else reject(new Error(`音频压缩失败:${stderr.trim().split('\n').slice(-2).join(' ')}`))
|
|
114
|
+
})
|
|
115
|
+
})
|
|
116
|
+
}
|
package/lib/cli-llm.js
ADDED
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
// lib/cli-llm.js — 命令行版用哪个模型
|
|
2
|
+
//
|
|
3
|
+
// 顺序:命令行参数 > 环境变量(TINGDONG_BASE_URL / TINGDONG_API_KEY / TINGDONG_MODEL)
|
|
4
|
+
// > 本机的 dsh-model 网关(~/.dsh-model,同一生态的插件,地址和密钥都在本机)。
|
|
5
|
+
// 没指定模型时,在接口的模型列表里优先挑 deepseek。
|
|
6
|
+
|
|
7
|
+
import { readFile } from 'node:fs/promises'
|
|
8
|
+
import { homedir } from 'node:os'
|
|
9
|
+
import { join } from 'node:path'
|
|
10
|
+
import { openaiLlm } from './llm.js'
|
|
11
|
+
|
|
12
|
+
async function dshModelGateway() {
|
|
13
|
+
const root = join(homedir(), '.dsh-model')
|
|
14
|
+
try {
|
|
15
|
+
const cfg = JSON.parse(await readFile(join(root, 'config.json'), 'utf8'))
|
|
16
|
+
const yaml = await readFile(join(root, 'engine.yaml'), 'utf8')
|
|
17
|
+
const key = /api-keys:\s*\n\s*-\s*["']?([^"'\n]+)/.exec(yaml)
|
|
18
|
+
const port = Number(cfg.port) || 8317
|
|
19
|
+
return { baseURL: `http://127.0.0.1:${port}/v1`, apiKey: key ? key[1].trim() : '' }
|
|
20
|
+
} catch {
|
|
21
|
+
return null
|
|
22
|
+
}
|
|
23
|
+
}
|
|
24
|
+
|
|
25
|
+
async function pickModel(baseURL, apiKey) {
|
|
26
|
+
try {
|
|
27
|
+
const r = await fetch(baseURL.replace(/\/+$/, '') + '/models', {
|
|
28
|
+
headers: apiKey ? { authorization: `Bearer ${apiKey}` } : {},
|
|
29
|
+
signal: AbortSignal.timeout(6000),
|
|
30
|
+
})
|
|
31
|
+
const ids = ((await r.json()).data || []).map(m => m.id).filter(id => typeof id === 'string')
|
|
32
|
+
return ids.find(id => /deepseek/i.test(id) && !/embed|bge/i.test(id)) || ids.find(id => !/embed|bge/i.test(id)) || ''
|
|
33
|
+
} catch {
|
|
34
|
+
return ''
|
|
35
|
+
}
|
|
36
|
+
}
|
|
37
|
+
|
|
38
|
+
/** @returns {Promise<object|null>} complete() 接口;没有可用模型返回 null */
|
|
39
|
+
export async function cliLlm(flags = {}) {
|
|
40
|
+
let baseURL = flags['base-url'] || process.env.TINGDONG_BASE_URL || ''
|
|
41
|
+
let apiKey = flags['api-key'] || process.env.TINGDONG_API_KEY || ''
|
|
42
|
+
let model = flags.model || process.env.TINGDONG_MODEL || ''
|
|
43
|
+
let via = 'env'
|
|
44
|
+
if (!baseURL) {
|
|
45
|
+
const gw = await dshModelGateway()
|
|
46
|
+
if (gw) { baseURL = gw.baseURL; apiKey = apiKey || gw.apiKey; via = 'dsh-model' }
|
|
47
|
+
}
|
|
48
|
+
if (!baseURL) return null
|
|
49
|
+
if (!model) model = await pickModel(baseURL, apiKey)
|
|
50
|
+
if (!model) return null
|
|
51
|
+
const llm = openaiLlm({ baseURL, apiKey, model })
|
|
52
|
+
llm.via = via
|
|
53
|
+
return llm
|
|
54
|
+
}
|