dsh-voice 0.8.1 → 0.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +13 -0
- package/README.en.md +17 -15
- package/README.md +17 -15
- package/docs/USAGE.en.md +39 -16
- package/docs/USAGE.md +41 -18
- package/lib/client.js +5067 -0
- package/lib/config.d.ts +7 -0
- package/lib/config.js +21 -0
- package/lib/edge-tts.d.ts +4 -0
- package/lib/edge-tts.js +38 -1
- package/lib/host-config.js +7 -0
- package/lib/index.d.ts +3 -1
- package/lib/index.js +4 -1
- package/lib/openai-tts.d.ts +19 -6
- package/lib/openai-tts.js +163 -2
- package/lib/stt.d.ts +19 -0
- package/lib/stt.js +157 -9
- package/lib/tools.d.ts +4 -2
- package/lib/tools.js +380 -70
- package/lib/tts-text.d.ts +46 -3
- package/lib/tts-text.js +345 -16
- package/lib/web.d.ts +66 -0
- package/lib/web.js +677 -0
- package/locale/en.json +1 -1
- package/locale/zh.json +1 -1
- package/package.json +10 -5
- package/skills/voice-workflow/SKILL.md +51 -19
- package/lib/update-client.js +0 -745
package/locale/zh.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"meta": {
|
|
3
3
|
"title": "语音",
|
|
4
|
-
"description": "
|
|
4
|
+
"description": "5 套主题语音工作台(原生清透 / 米白纸感 / Claude 暖书卷 / 8-Bit 像素 / 晴空薄荷,支持在线试听 300+ 音色与 10 套人设、浏览器麦克风速记、自定义角色与发音词典管理):零密钥 Edge 与 OpenAI/SiliconFlow 语音合成(TTS)、精确 MP3 时长反馈、字幕时间轴反向卡点配音、播客 BGM 自动闪避与 -16 LUFS 响度均衡、SHA-256 缓存声音克隆、发音词典替读、以及带说话人区分的多引擎语音转文字(STT)。"
|
|
5
5
|
}
|
|
6
6
|
}
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "dsh-voice",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"description": "DeepSeek Harness
|
|
3
|
+
"version": "0.9.0",
|
|
4
|
+
"description": "DeepSeek Harness 语音工作台插件(支持 5 套工作台主题:原生清透 / 米白纸感 / Claude 暖书卷 / 8-Bit 像素 / 晴空薄荷):零密钥 Edge TTS 朗读、声音克隆、字幕卡点配音、播客 BGM 自动闪避、多角色与字幕导出、在线试听及多引擎 ASR 转写。欢迎提交 issues 和 PR。",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "lib/index.js",
|
|
7
7
|
"types": "lib/index.d.ts",
|
|
@@ -12,7 +12,7 @@
|
|
|
12
12
|
},
|
|
13
13
|
"./cordis.patch.yml": "./cordis.patch.yml",
|
|
14
14
|
"./package.json": "./package.json",
|
|
15
|
-
"./client": "./lib/
|
|
15
|
+
"./client": "./lib/client.js",
|
|
16
16
|
"./locale/*.json": "./locale/*.json",
|
|
17
17
|
"./icon": "./assets/plugin-icon.webp"
|
|
18
18
|
},
|
|
@@ -31,7 +31,7 @@
|
|
|
31
31
|
"skills"
|
|
32
32
|
],
|
|
33
33
|
"scripts": {
|
|
34
|
-
"build": "tsc -p tsconfig.json && node scripts/build-updates.mjs",
|
|
34
|
+
"build": "tsc -p tsconfig.json && node scripts/build-client.mjs && node scripts/build-updates.mjs",
|
|
35
35
|
"typecheck": "tsc -p tsconfig.json --noEmit",
|
|
36
36
|
"test": "npm run build && node --test \"test/*.test.mjs\"",
|
|
37
37
|
"test:integration": "npm run build && node --test \"integration/*.test.mjs\"",
|
|
@@ -43,7 +43,12 @@
|
|
|
43
43
|
"patch": "./cordis.patch.yml"
|
|
44
44
|
},
|
|
45
45
|
"client": {
|
|
46
|
-
"platform": "web"
|
|
46
|
+
"platform": "web",
|
|
47
|
+
"inject": [
|
|
48
|
+
"@deepseek-ai/dsh-client-ui-settings",
|
|
49
|
+
"@deepseek-ai/dsh-client-ui-layout",
|
|
50
|
+
"@deepseek-ai/dsh-client-ui-sidebar"
|
|
51
|
+
]
|
|
47
52
|
}
|
|
48
53
|
},
|
|
49
54
|
"keywords": [
|
|
@@ -1,14 +1,16 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: voice-workflow
|
|
3
|
-
description: Best-practice workflows for dsh-voice: 300+ online Edge voice discovery, 10 built-in
|
|
3
|
+
description: Best-practice workflows for dsh-voice: 5-theme Voice Studio sidebar workbench, 300+ online Edge voice discovery, 10 built-in personas, exact MP3 duration feedback (durationMs/durationSec), pronunciation lexicon (pronunciations), reverse .srt/.vtt video dubbing (subtitleInput), podcast BGM auto-ducking (bgmFile/bgmVolume/bgmDucking) with -16 LUFS mastering, emotion/acting instructions, SiliconFlow SHA-256 cached cloning & customVoices aliases, direct file TTS with Markdown cleanup, and speaker-diarized transcription (diarize). Load when turning documents/subtitles into speech, hitting exact duration targets, producing podcasts with BGM, cloning voices, or transcribing audio.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Voice Workflow (`dsh-voice`)
|
|
7
7
|
|
|
8
|
-
Use this guide when converting articles, notes, or
|
|
8
|
+
Use this guide when converting articles, notes, briefings, or subtitle timelines into audio/subtitles, auditioning voices/personas, cloning reference voices, mixing podcast BGM, or transcribing recordings inside the active session workspace.
|
|
9
9
|
|
|
10
|
-
## 1.
|
|
10
|
+
## 1. Native Voice Studio Workbench, Discovery & Auditions
|
|
11
11
|
|
|
12
|
+
- **5-theme Voice Studio sidebar workbench**:
|
|
13
|
+
- Users can open the native DSH sidebar workbench (5 themes: 原生清透 / 米白纸感 / Claude 暖书卷 / 8-Bit 像素 / 晴空薄荷 + interactive Coachmark onboarding tour) to audition 300+ voices and 10 personas in-panel, dictate with the browser microphone, and manage `customVoices` and `pronunciations` rules without editing YAML.
|
|
12
14
|
- **Online 300+ Edge voice catalog + 45+ curated built-ins**:
|
|
13
15
|
- Call `voice_list({ online: true, locale, gender, tag })` to fetch and cache (24h in-memory) Microsoft's live 300+ neural voice catalog. If offline or unreachable, it falls back automatically (`source: "builtin-fallback"`) to 45+ curated voices spanning child (`zh-CN-XiaoshuangNeural`, `zh-CN-XiaoyouNeural`), teen/anime (`zh-CN-YunxiaNeural`), mature/documentary (`zh-CN-YunzeNeural`, `zh-CN-XiaoqiuNeural`), elder (`zh-CN-XiaoruiNeural`), passionate (`zh-CN-YunhaoNeural`), and dialects (Northeast `zh-CN-liaoning-XiaobeiNeural`, Shaanxi `zh-CN-shaanxi-XiaoniNeural`, Sichuan `zh-CN-sichuan-YunxiNeural`, Henan `zh-CN-henan-YundengNeural`, Shandong `zh-CN-shandong-YunxiangNeural`, Cantonese `zh-HK-WanLungNeural` / `zh-HK-HiuMaanNeural`, Taiwanese `zh-TW-YunJheNeural` / `zh-TW-HsiaoChenNeural` / `zh-TW-HsiaoYuNeural`).
|
|
14
16
|
- `voice_list` also returns `personas` (unless `includePersonas: false`) and any `customVoices` aliases configured in DSH Settings.
|
|
@@ -18,30 +20,52 @@ Use this guide when converting articles, notes, or briefings into audio/subtitle
|
|
|
18
20
|
- **Interactive audition page**:
|
|
19
21
|
- Call `voice_preview({ voices: [...], text })` with raw voice IDs, built-in `persona` IDs/labels, or `customVoices` alias names. It writes MP3 samples and an interactive `voice_previews/index.html` browser audition page (with playback speed toggles and one-click ID copy).
|
|
20
22
|
|
|
21
|
-
## 2. Direct File Synthesis &
|
|
23
|
+
## 2. Direct File Synthesis, Pronunciation Lexicon & Exact Duration Targets
|
|
22
24
|
|
|
23
25
|
- **Do not copy large `.md` or `.txt` files into `text` arguments.** Pass `inputFile` directly to `voice_tts`:
|
|
24
26
|
```json
|
|
25
27
|
{
|
|
26
28
|
"inputFile": "docs/article.md",
|
|
27
29
|
"persona": "documentary-narrator",
|
|
28
|
-
"
|
|
30
|
+
"pronunciations": [{ "word": "Nginx", "alias": "Engine X" }],
|
|
31
|
+
"output": "article.mp3",
|
|
32
|
+
"subtitleOutput": "article.srt"
|
|
29
33
|
}
|
|
30
34
|
```
|
|
35
|
+
- **Pronunciation lexicon (`pronunciations`)**:
|
|
36
|
+
- Pass `pronunciations` as `[{ word, alias }]` (or `{ "Nginx": "Engine X" }`) on `voice_tts`, or configure persistent rules in DSH Settings `pronunciations` (per-call rules override same-word Settings rules).
|
|
37
|
+
- On Edge TTS, rules compile to SSML `<sub alias="...">` so exported `.srt`/`.vtt` subtitles preserve the original spelling (`Nginx`) while speech pronounces the alias (`Engine X`). On OpenAI/SiliconFlow/custom engines, spoken text applies non-recursive longest-first replacement while fallback subtitle cues retain the original text.
|
|
38
|
+
- **Exact MP3 duration feedback (`durationMs` / `durationSec`)**:
|
|
39
|
+
- Every `voice_tts` call inspects the final MP3 stream across CBR/VBR and multi-engine chunks (generating matched-sample-rate silence frames for pauses) and returns exact `durationMs` and `durationSec`.
|
|
40
|
+
- When targeting a strict duration (e.g. a 60s short-video voiceover), check `durationSec` and re-adjust `rate` or trim script length accordingly.
|
|
31
41
|
- `cleanMarkdown` defaults to `true` when `inputFile` ends with `.md` or `.markdown` (pass `cleanMarkdown: true` explicitly for inline Markdown in `text`). It strips fenced code blocks, URLs, Markdown links/images, headings, emphasis markers, and HTML tags while preserving punctuation and paragraph breaks.
|
|
32
42
|
- Texts longer than `5,000` characters (up to `100,000` characters) are automatically split on sentence/clause boundaries and merged into a single seamless MP3 in one `voice_tts` call.
|
|
33
43
|
|
|
34
|
-
## 3.
|
|
44
|
+
## 3. Reverse `.srt` / `.vtt` Video Dubbing (`subtitleInput`)
|
|
45
|
+
|
|
46
|
+
- When dubbing an existing video subtitle timeline into voiceover audio, pass `subtitleInput: "video.srt"` (or `.vtt`, up to 200 cues) to `voice_tts`:
|
|
47
|
+
```json
|
|
48
|
+
{
|
|
49
|
+
"subtitleInput": "video.srt",
|
|
50
|
+
"persona": "bilingual-tech-male",
|
|
51
|
+
"pronunciations": [{ "word": "SQL", "alias": "S Q L" }],
|
|
52
|
+
"output": "dubbed.mp3",
|
|
53
|
+
"subtitleOutput": "dubbed.srt"
|
|
54
|
+
}
|
|
55
|
+
```
|
|
56
|
+
- Each cue is synthesized and measured with `inspectMp3Buffer`; if a cue runs more than `8%` past its `[startMs, endMs]` window, `voice_tts` automatically scales its `rate` (clamped `-30%` to `+85%`) and re-synthesizes once, then inserts matched-sample-rate millisecond silence padding between cues so audio stays locked to the video timeline.
|
|
57
|
+
|
|
58
|
+
## 4. Oral Script Adaptation for Briefings
|
|
35
59
|
|
|
36
60
|
When the user asks to summarize notes, changelogs, or reports into a spoken briefing rather than reading a file verbatim:
|
|
37
61
|
|
|
38
62
|
- Rewrite bullet lists, Markdown tables, code identifiers, and bare URLs into natural spoken sentences before calling `voice_tts`.
|
|
39
|
-
- Spell out or paraphrase abbreviations and symbols (
|
|
40
|
-
- Keep target duration in mind (roughly 220–260 Chinese characters or 130–160 English words per minute at `+0%` rate)
|
|
63
|
+
- Spell out or paraphrase abbreviations and symbols (or register technical acronyms in `pronunciations`) so the narration sounds like a human host.
|
|
64
|
+
- Keep target duration in mind (roughly 220–260 Chinese characters or 130–160 English words per minute at `+0%` rate) and verify with the returned `durationSec`.
|
|
41
65
|
|
|
42
|
-
##
|
|
66
|
+
## 5. Multi-Speaker Podcasts, BGM Auto-Ducking & Loudness Mastering
|
|
43
67
|
|
|
44
|
-
Produce two-host or multi-role dialogues in a single MP3 using `voice_tts` `segments` (up to 50 segments per call),
|
|
68
|
+
Produce two-host or multi-role dialogues with background music in a single MP3 using `voice_tts` `segments` (up to 50 segments per call), `bgmFile`, `bgmVolume`, `bgmDucking`, and `-16 LUFS` broadcast loudness mastering:
|
|
45
69
|
|
|
46
70
|
```json
|
|
47
71
|
{
|
|
@@ -53,41 +77,49 @@ Produce two-host or multi-role dialogues in a single MP3 using `voice_tts` `segm
|
|
|
53
77
|
},
|
|
54
78
|
{
|
|
55
79
|
"persona": "podcast-host-male",
|
|
56
|
-
"text": "
|
|
80
|
+
"text": "没错,现在长文稿、BGM 自动闪避和响度均衡都可以一次搞定。",
|
|
57
81
|
"rate": "+5%",
|
|
58
82
|
"pauseMs": 300
|
|
59
83
|
}
|
|
60
84
|
],
|
|
85
|
+
"bgmFile": "assets/bgm.mp3",
|
|
86
|
+
"bgmVolume": 0.18,
|
|
87
|
+
"bgmDucking": true,
|
|
61
88
|
"normalizeLoudness": true,
|
|
62
89
|
"output": "podcast.mp3",
|
|
63
90
|
"subtitleOutput": "podcast.srt"
|
|
64
91
|
}
|
|
65
92
|
```
|
|
66
93
|
|
|
94
|
+
- **Podcast BGM Auto-Ducking (`bgmFile`, `bgmVolume`, `bgmDucking`)**:
|
|
95
|
+
- Loops workspace `bgmFile` under the speech track at `bgmVolume` (`0.02–1.0`, default `0.18`), applies FFmpeg `sidechaincompress` ducking when `bgmDucking !== false` so music automatically dips whenever hosts speak, adds a smooth fade-out at the end, and optionally chains `-16 LUFS` loudness normalization (`normalizeLoudness: true`). Requires local `ffmpeg`.
|
|
67
96
|
- **Emotion, acting & paralinguistics (`instructions`)**:
|
|
68
97
|
- Pass natural-language acting prompts via `instructions` on `voice_tts` or per segment in `segments[]`.
|
|
69
98
|
- On SiliconFlow `CosyVoice2-0.5B`, `instructions` is automatically prepended with `<|endofprompt|>` and supports inline `[laughter]` and `[breath]` tags.
|
|
70
99
|
- On OpenAI `gpt-4o-mini-tts`, `instructions` is passed natively as the top-level API parameter.
|
|
71
100
|
- On custom Kokoro endpoints, `voice` also accepts voice-blending expressions such as `zf_xiaobei*0.6+zf_xiaoni*0.4`.
|
|
72
|
-
- **
|
|
73
|
-
-
|
|
101
|
+
- **Standalone loudness mastering (`normalizeLoudness: true`)**:
|
|
102
|
+
- When `bgmFile` is omitted, runs local `ffmpeg` (`loudnorm=I=-16:TP=-1.5:LRA=11`) after concatenation so multi-speaker or cross-voice podcasts have consistent broadcast volume. (Requires `ffmpeg` installed locally.)
|
|
74
103
|
|
|
75
|
-
##
|
|
104
|
+
## 6. Zero-Shot Voice Cloning (`referenceAudio`) & `customVoices` Aliases
|
|
76
105
|
|
|
77
|
-
- **Zero-shot reference cloning**:
|
|
106
|
+
- **Zero-shot reference cloning with SHA-256 URI caching**:
|
|
78
107
|
- Pass a 5–10 second clean workspace speech clip (`≤10 MB`, `.wav`/`.mp3`/etc.) in `referenceAudio` alongside its transcript in `referenceText` on `voice_tts` or `segments[]`.
|
|
79
108
|
- Requires `ttsEngine` set to `siliconflow` (`FunAudioLLM/CosyVoice2-0.5B`) or `custom` (e.g., local GPT-SoVITS / IndexTTS). Using `referenceAudio` while `ttsEngine: "edge"` throws an actionable error guiding the user to switch engines.
|
|
109
|
+
- On SiliconFlow, `dsh-voice` automatically uploads the reference clip to `/uploads/audio/voice` once per SHA-256 hash and caches the returned `speech:...` URI across podcast segments instead of re-sending megabytes of base64 audio on every turn.
|
|
80
110
|
- **Reusable named aliases (`customVoices`)**:
|
|
81
|
-
- Users can save named characters (`name`, `voice`, `persona`, `rate`, `pitch`, `instructions`, `referenceAudio`, `referenceText`) in DSH Settings `customVoices` (live-reloaded without restart) and invoke them by `name` in `voice_tts`, `segments[]`, or `voice_preview`.
|
|
111
|
+
- Users can save named characters (`name`, `voice`, `persona`, `rate`, `pitch`, `instructions`, `referenceAudio`, `referenceText`) in DSH Settings `customVoices` or the Voice Studio sidebar (live-reloaded without restart) and invoke them by `name` in `voice_tts`, `segments[]`, or `voice_preview`.
|
|
82
112
|
|
|
83
|
-
##
|
|
113
|
+
## 7. Synchronized Subtitles for Video Pipelines (`subtitleOutput`)
|
|
84
114
|
|
|
85
|
-
- Pass `subtitleOutput: "out.srt"` or `"out.vtt"` to `voice_tts` (single-speaker or `
|
|
115
|
+
- Pass `subtitleOutput: "out.srt"` or `"out.vtt"` to `voice_tts` (single-speaker, `segments`, or `subtitleInput`) to export synchronized sentence-boundary subtitles alongside the MP3 at zero extra API cost.
|
|
86
116
|
- Pass `subtitleOutput: "transcript.srt"` (or `.vtt`) to `voice_stt` to automatically request word/segment timestamps and write subtitles alongside the transcript.
|
|
87
117
|
- Hand the resulting `.mp3` and `.srt`/`.vtt` files directly to `dsh-ffmpeg` for subtitle burn-in or `dsh-remotion` for timeline composition.
|
|
88
118
|
|
|
89
|
-
##
|
|
119
|
+
## 8. Speaker-Diarized Transcription (`diarize`) & Health Checks
|
|
90
120
|
|
|
121
|
+
- **Speaker diarization (`diarize: true` on `voice_stt`)**:
|
|
122
|
+
- Pass `diarize: true` to `voice_stt` to separate multi-speaker recordings: splits stereo channels via local `ffmpeg` when available (`说话人 A` / `说话人 B`), or reads/infers speaker turn switches in `verbose_json`, returning `speakers`, formatted `[说话人 A 00:01]` transcript paragraphs, and `[speaker]`-prefixed `.srt`/`.vtt` cues.
|
|
91
123
|
- **Domain accuracy**: Pass project names, acronyms, and speaker terms in `voice_stt({ audio, prompt: "DeepSeek Harness, Cordis, ...", output: "meeting.txt" })`.
|
|
92
124
|
- **Large recordings (>25 MB)**: Pass `autoCompress: true` to `voice_stt`. When local `ffmpeg` is available, `dsh-voice` automatically transcodes (16kHz mono) and splits recordings larger than 24 MB into segments, merges the transcript, and shifts subtitle timestamps accurately.
|
|
93
125
|
- **Local / keyless Whisper**: If `asrEngine` is `custom` pointing to a loopback URL (e.g. `http://127.0.0.1:8000/v1`), no `asrApiKey` is required.
|