dsh-voice 0.8.0 → 0.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/locale/zh.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "meta": {
3
3
  "title": "语音",
4
- "description": "零密钥 Edge(300+ 在线音色与全年龄/方言内置)及 OpenAI/SiliconFlow 语音合成(TTS)、10 大音色人设、情绪演技指令、零样本克隆与自定义别名、长文清理、多角色播客与响度均衡、SRT/VTT 字幕、网页试听及多引擎语音转文字(STT)。"
4
+ "description": "5 套主题语音工作台(原生清透 / 米白纸感 / Claude 暖书卷 / 8-Bit 像素 / 晴空薄荷,支持在线试听 300+ 音色与 10 套人设、浏览器麦克风速记、自定义角色与发音词典管理):零密钥 Edge 与 OpenAI/SiliconFlow 语音合成(TTS)、精确 MP3 时长反馈、字幕时间轴反向卡点配音、播客 BGM 自动闪避与 -16 LUFS 响度均衡、SHA-256 缓存声音克隆、发音词典替读、以及带说话人区分的多引擎语音转文字(STT)。"
5
5
  }
6
6
  }
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "dsh-voice",
3
- "version": "0.8.0",
4
- "description": "DeepSeek Harness 语音插件:零密钥 Edge TTS 朗读、多角色播客与字幕导出、音色试听、兼容 ASR 转写与工作区输出保护。欢迎提交 issues 和 PR。",
3
+ "version": "0.9.0",
4
+ "description": "DeepSeek Harness 语音工作台插件(支持 5 套工作台主题:原生清透 / 米白纸感 / Claude 暖书卷 / 8-Bit 像素 / 晴空薄荷):零密钥 Edge TTS 朗读、声音克隆、字幕卡点配音、播客 BGM 自动闪避、多角色与字幕导出、在线试听及多引擎 ASR 转写。欢迎提交 issues 和 PR。",
5
5
  "type": "module",
6
6
  "main": "lib/index.js",
7
7
  "types": "lib/index.d.ts",
@@ -12,7 +12,7 @@
12
12
  },
13
13
  "./cordis.patch.yml": "./cordis.patch.yml",
14
14
  "./package.json": "./package.json",
15
- "./client": "./lib/update-client.js",
15
+ "./client": "./lib/client.js",
16
16
  "./locale/*.json": "./locale/*.json",
17
17
  "./icon": "./assets/plugin-icon.webp"
18
18
  },
@@ -31,7 +31,7 @@
31
31
  "skills"
32
32
  ],
33
33
  "scripts": {
34
- "build": "tsc -p tsconfig.json && node scripts/build-updates.mjs",
34
+ "build": "tsc -p tsconfig.json && node scripts/build-client.mjs && node scripts/build-updates.mjs",
35
35
  "typecheck": "tsc -p tsconfig.json --noEmit",
36
36
  "test": "npm run build && node --test \"test/*.test.mjs\"",
37
37
  "test:integration": "npm run build && node --test \"integration/*.test.mjs\"",
@@ -43,7 +43,12 @@
43
43
  "patch": "./cordis.patch.yml"
44
44
  },
45
45
  "client": {
46
- "platform": "web"
46
+ "platform": "web",
47
+ "inject": [
48
+ "@deepseek-ai/dsh-client-ui-settings",
49
+ "@deepseek-ai/dsh-client-ui-layout",
50
+ "@deepseek-ai/dsh-client-ui-sidebar"
51
+ ]
47
52
  }
48
53
  },
49
54
  "keywords": [
@@ -70,7 +75,7 @@
70
75
  "https-proxy-agent": "^9.1.0",
71
76
  "undici": "^8.10.0",
72
77
  "ws": "^8.21.3",
73
- "@stardustlc/dsh-updates": "0.2.3"
78
+ "@stardustlc/dsh-updates": "0.2.4"
74
79
  },
75
80
  "devDependencies": {
76
81
  "@types/node": "^24.0.0",
@@ -1,14 +1,16 @@
1
1
  ---
2
2
  name: voice-workflow
3
- description: Best-practice workflows for dsh-voice: 300+ online Edge voice discovery, 10 built-in voice personas, emotion/acting instructions, zero-shot reference cloning & customVoices aliases, direct file TTS with Markdown cleanup, oral script adaptation, multi-speaker podcasts with -16 LUFS loudness mastering, SRT/VTT subtitle export, interactive voice auditions, and large-recording transcription. Load when turning documents/notes into speech, producing podcasts/subtitles, cloning voices, or transcribing audio.
3
+ description: Best-practice workflows for dsh-voice: 5-theme Voice Studio sidebar workbench, 300+ online Edge voice discovery, 10 built-in personas, exact MP3 duration feedback (durationMs/durationSec), pronunciation lexicon (pronunciations), reverse .srt/.vtt video dubbing (subtitleInput), podcast BGM auto-ducking (bgmFile/bgmVolume/bgmDucking) with -16 LUFS mastering, emotion/acting instructions, SiliconFlow SHA-256 cached cloning & customVoices aliases, direct file TTS with Markdown cleanup, and speaker-diarized transcription (diarize). Load when turning documents/subtitles into speech, hitting exact duration targets, producing podcasts with BGM, cloning voices, or transcribing audio.
4
4
  ---
5
5
 
6
6
  # Voice Workflow (`dsh-voice`)
7
7
 
8
- Use this guide when converting articles, notes, or briefings into audio/subtitles, auditioning voices/personas, cloning reference voices, or transcribing recordings inside the active session workspace.
8
+ Use this guide when converting articles, notes, briefings, or subtitle timelines into audio/subtitles, auditioning voices/personas, cloning reference voices, mixing podcast BGM, or transcribing recordings inside the active session workspace.
9
9
 
10
- ## 1. Discover Voices, Pick Personas & Audition Before Long Renders
10
+ ## 1. Native Voice Studio Workbench, Discovery & Auditions
11
11
 
12
+ - **5-theme Voice Studio sidebar workbench**:
13
+ - Users can open the native DSH sidebar workbench (5 themes: 原生清透 / 米白纸感 / Claude 暖书卷 / 8-Bit 像素 / 晴空薄荷 + interactive Coachmark onboarding tour) to audition 300+ voices and 10 personas in-panel, dictate with the browser microphone, and manage `customVoices` and `pronunciations` rules without editing YAML.
12
14
  - **Online 300+ Edge voice catalog + 45+ curated built-ins**:
13
15
  - Call `voice_list({ online: true, locale, gender, tag })` to fetch and cache (24h in-memory) Microsoft's live 300+ neural voice catalog. If offline or unreachable, it falls back automatically (`source: "builtin-fallback"`) to 45+ curated voices spanning child (`zh-CN-XiaoshuangNeural`, `zh-CN-XiaoyouNeural`), teen/anime (`zh-CN-YunxiaNeural`), mature/documentary (`zh-CN-YunzeNeural`, `zh-CN-XiaoqiuNeural`), elder (`zh-CN-XiaoruiNeural`), passionate (`zh-CN-YunhaoNeural`), and dialects (Northeast `zh-CN-liaoning-XiaobeiNeural`, Shaanxi `zh-CN-shaanxi-XiaoniNeural`, Sichuan `zh-CN-sichuan-YunxiNeural`, Henan `zh-CN-henan-YundengNeural`, Shandong `zh-CN-shandong-YunxiangNeural`, Cantonese `zh-HK-WanLungNeural` / `zh-HK-HiuMaanNeural`, Taiwanese `zh-TW-YunJheNeural` / `zh-TW-HsiaoChenNeural` / `zh-TW-HsiaoYuNeural`).
14
16
  - `voice_list` also returns `personas` (unless `includePersonas: false`) and any `customVoices` aliases configured in DSH Settings.
@@ -18,30 +20,52 @@ Use this guide when converting articles, notes, or briefings into audio/subtitle
18
20
  - **Interactive audition page**:
19
21
  - Call `voice_preview({ voices: [...], text })` with raw voice IDs, built-in `persona` IDs/labels, or `customVoices` alias names. It writes MP3 samples and an interactive `voice_previews/index.html` browser audition page (with playback speed toggles and one-click ID copy).
20
22
 
21
- ## 2. Direct File Synthesis & Markdown Cleanup (`inputFile`)
23
+ ## 2. Direct File Synthesis, Pronunciation Lexicon & Exact Duration Targets
22
24
 
23
25
  - **Do not copy large `.md` or `.txt` files into `text` arguments.** Pass `inputFile` directly to `voice_tts`:
24
26
  ```json
25
27
  {
26
28
  "inputFile": "docs/article.md",
27
29
  "persona": "documentary-narrator",
28
- "output": "article.mp3"
30
+ "pronunciations": [{ "word": "Nginx", "alias": "Engine X" }],
31
+ "output": "article.mp3",
32
+ "subtitleOutput": "article.srt"
29
33
  }
30
34
  ```
35
+ - **Pronunciation lexicon (`pronunciations`)**:
36
+ - Pass `pronunciations` as `[{ word, alias }]` (or `{ "Nginx": "Engine X" }`) on `voice_tts`, or configure persistent rules in DSH Settings `pronunciations` (per-call rules override same-word Settings rules).
37
+ - On Edge TTS, rules compile to SSML `<sub alias="...">` so exported `.srt`/`.vtt` subtitles preserve the original spelling (`Nginx`) while speech pronounces the alias (`Engine X`). On OpenAI/SiliconFlow/custom engines, spoken text applies non-recursive longest-first replacement while fallback subtitle cues retain the original text.
38
+ - **Exact MP3 duration feedback (`durationMs` / `durationSec`)**:
39
+ - Every `voice_tts` call inspects the final MP3 stream across CBR/VBR and multi-engine chunks (generating matched-sample-rate silence frames for pauses) and returns exact `durationMs` and `durationSec`.
40
+ - When targeting a strict duration (e.g. a 60s short-video voiceover), check `durationSec` and re-adjust `rate` or trim script length accordingly.
31
41
  - `cleanMarkdown` defaults to `true` when `inputFile` ends with `.md` or `.markdown` (pass `cleanMarkdown: true` explicitly for inline Markdown in `text`). It strips fenced code blocks, URLs, Markdown links/images, headings, emphasis markers, and HTML tags while preserving punctuation and paragraph breaks.
32
42
  - Texts longer than `5,000` characters (up to `100,000` characters) are automatically split on sentence/clause boundaries and merged into a single seamless MP3 in one `voice_tts` call.
33
43
 
34
- ## 3. Oral Script Adaptation for Briefings
44
+ ## 3. Reverse `.srt` / `.vtt` Video Dubbing (`subtitleInput`)
45
+
46
+ - When dubbing an existing video subtitle timeline into voiceover audio, pass `subtitleInput: "video.srt"` (or `.vtt`, up to 200 cues) to `voice_tts`:
47
+ ```json
48
+ {
49
+ "subtitleInput": "video.srt",
50
+ "persona": "bilingual-tech-male",
51
+ "pronunciations": [{ "word": "SQL", "alias": "S Q L" }],
52
+ "output": "dubbed.mp3",
53
+ "subtitleOutput": "dubbed.srt"
54
+ }
55
+ ```
56
+ - Each cue is synthesized and measured with `inspectMp3Buffer`; if a cue runs more than `8%` past its `[startMs, endMs]` window, `voice_tts` automatically scales its `rate` (clamped `-30%` to `+85%`) and re-synthesizes once, then inserts matched-sample-rate millisecond silence padding between cues so audio stays locked to the video timeline.
57
+
58
+ ## 4. Oral Script Adaptation for Briefings
35
59
 
36
60
  When the user asks to summarize notes, changelogs, or reports into a spoken briefing rather than reading a file verbatim:
37
61
 
38
62
  - Rewrite bullet lists, Markdown tables, code identifiers, and bare URLs into natural spoken sentences before calling `voice_tts`.
39
- - Spell out or paraphrase abbreviations and symbols (e.g., turn `->`, `>=`, or file paths into natural transitions) so the narration sounds like a human host.
40
- - Keep target duration in mind (roughly 220–260 Chinese characters or 130–160 English words per minute at `+0%` rate).
63
+ - Spell out or paraphrase abbreviations and symbols (or register technical acronyms in `pronunciations`) so the narration sounds like a human host.
64
+ - Keep target duration in mind (roughly 220–260 Chinese characters or 130–160 English words per minute at `+0%` rate) and verify with the returned `durationSec`.
41
65
 
42
- ## 4. Multi-Speaker Podcasts, Emotion Acting & Loudness Mastering
66
+ ## 5. Multi-Speaker Podcasts, BGM Auto-Ducking & Loudness Mastering
43
67
 
44
- Produce two-host or multi-role dialogues in a single MP3 using `voice_tts` `segments` (up to 50 segments per call), with optional per-segment `persona`, `instructions`, `referenceAudio`/`referenceText`, and `-16 LUFS` broadcast loudness mastering:
68
+ Produce two-host or multi-role dialogues with background music in a single MP3 using `voice_tts` `segments` (up to 50 segments per call), `bgmFile`, `bgmVolume`, `bgmDucking`, and `-16 LUFS` broadcast loudness mastering:
45
69
 
46
70
  ```json
47
71
  {
@@ -53,41 +77,49 @@ Produce two-host or multi-role dialogues in a single MP3 using `voice_tts` `segm
53
77
  },
54
78
  {
55
79
  "persona": "podcast-host-male",
56
- "text": "没错,现在长文稿、人设配音和响度均衡都可以一次搞定。",
80
+ "text": "没错,现在长文稿、BGM 自动闪避和响度均衡都可以一次搞定。",
57
81
  "rate": "+5%",
58
82
  "pauseMs": 300
59
83
  }
60
84
  ],
85
+ "bgmFile": "assets/bgm.mp3",
86
+ "bgmVolume": 0.18,
87
+ "bgmDucking": true,
61
88
  "normalizeLoudness": true,
62
89
  "output": "podcast.mp3",
63
90
  "subtitleOutput": "podcast.srt"
64
91
  }
65
92
  ```
66
93
 
94
+ - **Podcast BGM Auto-Ducking (`bgmFile`, `bgmVolume`, `bgmDucking`)**:
95
+ - Loops workspace `bgmFile` under the speech track at `bgmVolume` (`0.02–1.0`, default `0.18`), applies FFmpeg `sidechaincompress` ducking when `bgmDucking !== false` so music automatically dips whenever hosts speak, adds a smooth fade-out at the end, and optionally chains `-16 LUFS` loudness normalization (`normalizeLoudness: true`). Requires local `ffmpeg`.
67
96
  - **Emotion, acting & paralinguistics (`instructions`)**:
68
97
  - Pass natural-language acting prompts via `instructions` on `voice_tts` or per segment in `segments[]`.
69
98
  - On SiliconFlow `CosyVoice2-0.5B`, `instructions` is automatically prepended with `<|endofprompt|>` and supports inline `[laughter]` and `[breath]` tags.
70
99
  - On OpenAI `gpt-4o-mini-tts`, `instructions` is passed natively as the top-level API parameter.
71
100
  - On custom Kokoro endpoints, `voice` also accepts voice-blending expressions such as `zf_xiaobei*0.6+zf_xiaoni*0.4`.
72
- - **Podcast loudness mastering (`normalizeLoudness: true`)**:
73
- - Runs local `ffmpeg` (`loudnorm=I=-16:TP=-1.5:LRA=11`) after concatenation so multi-speaker or cross-voice podcasts have consistent broadcast volume. (Requires `ffmpeg` installed locally.)
101
+ - **Standalone loudness mastering (`normalizeLoudness: true`)**:
102
+ - When `bgmFile` is omitted, runs local `ffmpeg` (`loudnorm=I=-16:TP=-1.5:LRA=11`) after concatenation so multi-speaker or cross-voice podcasts have consistent broadcast volume. (Requires `ffmpeg` installed locally.)
74
103
 
75
- ## 5. Zero-Shot Voice Cloning (`referenceAudio`) & `customVoices` Aliases
104
+ ## 6. Zero-Shot Voice Cloning (`referenceAudio`) & `customVoices` Aliases
76
105
 
77
- - **Zero-shot reference cloning**:
106
+ - **Zero-shot reference cloning with SHA-256 URI caching**:
78
107
  - Pass a 5–10 second clean workspace speech clip (`≤10 MB`, `.wav`/`.mp3`/etc.) in `referenceAudio` alongside its transcript in `referenceText` on `voice_tts` or `segments[]`.
79
108
  - Requires `ttsEngine` set to `siliconflow` (`FunAudioLLM/CosyVoice2-0.5B`) or `custom` (e.g., local GPT-SoVITS / IndexTTS). Using `referenceAudio` while `ttsEngine: "edge"` throws an actionable error guiding the user to switch engines.
109
+ - On SiliconFlow, `dsh-voice` automatically uploads the reference clip to `/uploads/audio/voice` once per SHA-256 hash and caches the returned `speech:...` URI across podcast segments instead of re-sending megabytes of base64 audio on every turn.
80
110
  - **Reusable named aliases (`customVoices`)**:
81
- - Users can save named characters (`name`, `voice`, `persona`, `rate`, `pitch`, `instructions`, `referenceAudio`, `referenceText`) in DSH Settings `customVoices` (live-reloaded without restart) and invoke them by `name` in `voice_tts`, `segments[]`, or `voice_preview`.
111
+ - Users can save named characters (`name`, `voice`, `persona`, `rate`, `pitch`, `instructions`, `referenceAudio`, `referenceText`) in DSH Settings `customVoices` or the Voice Studio sidebar (live-reloaded without restart) and invoke them by `name` in `voice_tts`, `segments[]`, or `voice_preview`.
82
112
 
83
- ## 6. Synchronized Subtitles for Video Pipelines (`subtitleOutput`)
113
+ ## 7. Synchronized Subtitles for Video Pipelines (`subtitleOutput`)
84
114
 
85
- - Pass `subtitleOutput: "out.srt"` or `"out.vtt"` to `voice_tts` (single-speaker or `segments`) to export synchronized sentence-boundary subtitles alongside the MP3 at zero extra API cost.
115
+ - Pass `subtitleOutput: "out.srt"` or `"out.vtt"` to `voice_tts` (single-speaker, `segments`, or `subtitleInput`) to export synchronized sentence-boundary subtitles alongside the MP3 at zero extra API cost.
86
116
  - Pass `subtitleOutput: "transcript.srt"` (or `.vtt`) to `voice_stt` to automatically request word/segment timestamps and write subtitles alongside the transcript.
87
117
  - Hand the resulting `.mp3` and `.srt`/`.vtt` files directly to `dsh-ffmpeg` for subtitle burn-in or `dsh-remotion` for timeline composition.
88
118
 
89
- ## 7. Meeting Transcription & Health Checks
119
+ ## 8. Speaker-Diarized Transcription (`diarize`) & Health Checks
90
120
 
121
+ - **Speaker diarization (`diarize: true` on `voice_stt`)**:
122
+ - Pass `diarize: true` to `voice_stt` to separate multi-speaker recordings: splits stereo channels via local `ffmpeg` when available (`说话人 A` / `说话人 B`), or reads/infers speaker turn switches in `verbose_json`, returning `speakers`, formatted `[说话人 A 00:01]` transcript paragraphs, and `[speaker]`-prefixed `.srt`/`.vtt` cues.
91
123
  - **Domain accuracy**: Pass project names, acronyms, and speaker terms in `voice_stt({ audio, prompt: "DeepSeek Harness, Cordis, ...", output: "meeting.txt" })`.
92
124
  - **Large recordings (>25 MB)**: Pass `autoCompress: true` to `voice_stt`. When local `ffmpeg` is available, `dsh-voice` automatically transcodes (16kHz mono) and splits recordings larger than 24 MB into segments, merges the transcript, and shifts subtitle timestamps accurately.
93
125
  - **Local / keyless Whisper**: If `asrEngine` is `custom` pointing to a loopback URL (e.g. `http://127.0.0.1:8000/v1`), no `asrApiKey` is required.