dsh-voice 0.9.1 → 0.10.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,23 @@
1
1
  # 更新记录
2
2
 
3
+ ## 0.10.1 (2026-11-18)
4
+
5
+ - 升级共享更新组件至 `@stardustlc/dsh-updates@0.2.5`:独立安装插件详情页底栏去重(保留组合包组件委派与对话页热更新卡片),补齐 `bundle-in-use` 组合包占用提示说明,并在检查结果过期或远端版本变更时自动刷新元数据,彻底消除 `check-first` 卡顿。
6
+ - Upgrade `@stardustlc/dsh-updates` to `0.2.5`: deduplicate standalone native plugin detail footers while keeping Suite delegation and in-chat reminder cards, surface clear `bundle-in-use` guidance, and automatically refresh expired update checks without `check-first` stalls.
7
+
8
+ ## 0.10.0 (2026-11-18)
9
+
10
+ - 短视频逐词/短句字幕与 `.ass` 卡拉 OK 高亮导出:`voice_tts` 新增 `subtitleGranularity`(`sentence` / `clause` / `word`)与 `maxCharsPerCue`(默认 `16`),支持导出 `1080×1920` SubStation Alpha `.ass` 字幕(含 `\kf` 逐词卡拉 OK 时间轴),可直接配合 `dsh-ffmpeg` 烧录竖屏花字。
11
+ - Short-video clause/word subtitles & `.ass` karaoke export: add `subtitleGranularity` (`sentence` / `clause` / `word`) and `maxCharsPerCue` (default `16`) to `voice_tts`, with `1080×1920` SubStation Alpha `.ass` output emitting `\kf` word-level karaoke timing ready for `dsh-ffmpeg` burn-in.
12
+ - 内联停顿标签与英文词边界发音保护:台词内支持 `[pause:500ms]` / `[break:600ms]` / `[停顿0.6s]` 导演级气口标记(Edge 编译为 SSML `<break>`,OpenAI/SiliconFlow 拼接同采样率静音帧,导出字幕自动剔除标记);`pronunciations` 新增 ASCII 词边界保护,确保 `OS`、`UI`、`SQL` 等缩写绝不误伤 `POST`、`BUILD` 等单词子串。
13
+ - Inline speech pause tags & word-boundary-safe pronunciation lexicon: compile inline `[pause:500ms]` / `[break:600ms]` / `[停顿0.6s]` markers to SSML `<break>` on Edge and matched-sample-rate silence frames on OpenAI/SiliconFlow (automatically stripped from subtitles), and add ASCII word-boundary protection to `pronunciations` so short acronyms like `OS`, `UI`, and `SQL` never corrupt substrings inside `POST` or `BUILD`.
14
+ - 有声书/播客自动分章与原生 MP3 `CHAP`/`CTOC` 书签嵌入:多级标题 Markdown 文稿(`inputFile`)或带 `title` 的分段播客(`segments[]`)自动写入纯 TypeScript 生成的 ID3v2.3 `CHAP`/`CTOC` 章节书签,并支持 `chaptersOutput` 导出 `.md` 节目时间轴或 `.json` 章节清单。
15
+ - Audiobook/podcast auto-chapters & native MP3 `CHAP`/`CTOC` bookmarks: automatically embed pure-TS ID3v2.3 `CHAP`/`CTOC` chapter tags from multi-heading Markdown (`inputFile`) or titled `segments[]`, with optional `chaptersOutput` (`.md` show-notes timestamps or `.json`).
16
+ - 视频文件直传转写、零漂移 `atempo` 字幕配音、VBR 拼接修复与 Edge 热连接池:`voice_stt` 支持直接传入 `.mp4`/`.mkv`/`.mov`/`.avi` 视频容器自动调用本机 FFmpeg `-vn` 抽取 16kHz 单声道音频转写;`subtitleInput` 在超密台词加速后仍超窗时启用 FFmpeg `atempo` 保调精压实现零累积漂移;修复多段 VBR MP3 拼接残留首段 `Xing`/`Info`/`VBRI` 帧头锁死进度条的问题;Edge TTS 新增 WebSocket 保活连接池(`closeEdgeConnectionPool`)与瞬时断连自动重试。
17
+ - Direct video container STT, zero-drift `atempo` dubbing, VBR concat fix & Edge keep-alive WSS pool: auto-extract `-vn` 16kHz mono audio from `.mp4`/`.mkv`/`.mov`/`.avi` in `voice_stt`; add FFmpeg `atempo` pitch-preserving window fitting on dense `subtitleInput` cues; strip stale `Xing`/`Info`/`VBRI` metadata frames on multi-segment VBR MP3 concatenation; and add an Edge TTS keep-alive WebSocket pool (`closeEdgeConnectionPool`) with transient retry.
18
+ - Voice Studio 侧边栏「真·创作闭环」升级:新增「8 秒录音一键生成克隆角色」(`saveCloneVoice` 落盘 `voice_clones/` + 自动 ASR 转写参考台词 + 自动注册 `customVoices`)、可视化多角色对话合成器(`synthesizeDialogue` 面板内直接生成 MP3 与同步字幕)、第 5 标签页「工作区成品库」(`workspaceLibrary` / `loadWorkspaceMedia`)以及顶栏点击跳播的同步字幕跟读播放器与音频下载按钮。
19
+ - Voice Studio workbench "True Creative Loop": add 8-second 1-click mic voice clone creation (`saveCloneVoice` -> `voice_clones/` + auto-ASR + `customVoices`), visual multi-speaker dialogue synthesis (`synthesizeDialogue`), the 5th **Workspace Library** tab (`workspaceLibrary` / `loadWorkspaceMedia`), and an interactive click-to-seek synchronized subtitle follow-along player bar with direct audio download.
20
+
3
21
  ## 0.9.1 (2026-11-18)
4
22
 
5
23
  - 运行时技能自动挂载与设置回退 ID 对齐:`apply` 通过 `ctx.inject(['skills'])` 将随包 `voice-workflow` 技能自动注册进宿主运行时技能表(支持 `effect`/`dispose` 热卸载);Voice Studio 面板在缺少 `fiber` 元信息时的设置更新目标 ID 对齐 `cordis.patch.yml` 的 `'voice'`,并严格遵守宿主 `settings.writable` 只读状态。
package/README.en.md CHANGED
@@ -2,9 +2,9 @@
2
2
 
3
3
  # Voice
4
4
 
5
- **Turn text into natural speech and recordings into clean transcripts.**
5
+ **Turn text into natural speech and audio/video into clean transcripts.**
6
6
 
7
- Voice Studio workbench plugin for DeepSeek Harness (`>=0.2.2-rc.3`): includes a **native 5-theme sidebar Voice Studio workbench** (Native Airy / Warm Cream / Claude Warmth / Bright 8-Bit / Airy Mint with interactive Coachmark tour) alongside zero-key Edge neural TTS (300+ live online voices, 45+ all-age/dialect built-ins, 10 personas) and OpenAI / SiliconFlow / custom engines (emotion acting, Kokoro blending, SHA-256 cached zero-shot cloning). Supports **exact MP3 duration feedback (`durationMs` / `durationSec`)**, **pronunciation lexicon (`pronunciations`)**, **reverse `.srt`/`.vtt` video dubbing (`subtitleInput`)**, **podcast BGM auto-ducking & `-16 LUFS` mastering**, and **speaker-diarized (`diarize`)** multi-engine STT strictly confined to your session workspace.
7
+ Voice Studio workbench plugin for DeepSeek Harness (`>=0.2.2-rc.3`): includes a **native 5-theme Voice Studio "True Creative Loop" sidebar workbench** (Native Airy / Warm Cream / Claude Warmth / Bright 8-Bit / Airy Mint with Coachmark tour, 8-second browser mic direct-to-clone creator, visual multi-speaker dialogue synthesizer, workspace media library, and click-to-seek synchronized subtitle player) alongside zero-key Edge neural TTS (keep-alive WSS pool, 300+ online voices, 45+ built-ins, 10 personas) and OpenAI / SiliconFlow / custom engines (emotion acting, Kokoro blending, SHA-256 cached zero-shot cloning). Supports **short-video clause/word subtitles & karaoke `.ass` export (`\kf`)**, **inline `[pause:500ms]` speech tags & word-boundary-safe `pronunciations`**, **native MP3 ID3v2 `CHAP`/`CTOC` chapters (`chaptersOutput`)**, **zero-drift `atempo` `.srt`/`.vtt` dubbing & VBR-accurate `durationSec`**, **podcast BGM auto-ducking & `-16 LUFS` mastering**, and **direct video container (`.mp4`/`.mkv`/`.mov`/`.avi`) speaker-diarized (`diarize`) STT** strictly confined to your session workspace.
8
8
 
9
9
  [![npm](https://img.shields.io/npm/v/dsh-voice)](https://www.npmjs.com/package/dsh-voice) [![downloads](https://raw.githubusercontent.com/STARDUSTLC666/dsh-suite/npm-downloads/assets/dsh-voice-downloads.svg)](https://www.npmjs.com/package/dsh-voice) [![total downloads](https://raw.githubusercontent.com/STARDUSTLC666/dsh-suite/npm-downloads/assets/dsh-voice-total-downloads.svg)](https://www.npmjs.com/package/dsh-voice)
10
10
 
@@ -19,10 +19,10 @@ Voice Studio workbench plugin for DeepSeek Harness (`>=0.2.2-rc.3`): includes a
19
19
 
20
20
  ## Highlights
21
21
 
22
- - **Native 5-Theme Voice Studio Sidebar Workbench**: Switch live between 5 synced themes (**Native Airy / Warm Cream / Claude Warmth / Bright 8-Bit / Airy Mint**) with an interactive Coachmark onboarding tour. Audition 300+ Edge voices and 10 `persona` presets in-panel (`0.8x–1.5x` playback + duration badge), dictate with your browser microphone, and manage `customVoices` aliases and `pronunciations` rules visually.
23
- - **Exact MP3 Duration, Pronunciation Lexicon & Reverse `.srt`/`.vtt` Dubbing**: Inspects CBR/VBR and multi-engine MP3 streams to return exact `durationMs` / `durationSec` with matched-sample-rate silence frames (for hitting exact 60s short-video voiceover targets). `pronunciations` compiles to SSML `<sub alias>` on Edge (subtitles keep `Nginx` while speech says `Engine X`) and plain non-recursive replacement on OpenAI/SiliconFlow. `subtitleInput` scales per-cue speech rate and pads millisecond silences to stay locked to `.srt`/`.vtt` video timelines.
24
- - **Podcast BGM Auto-Ducking, Emotion Acting & SHA-256 Cached Cloning**: Pass `bgmFile`, `bgmVolume`, and `bgmDucking` alongside `segments` for FFmpeg `sidechaincompress` music ducking, smooth fade-out, and optional `-16 LUFS` loudness mastering. Use `instructions` for acting direction (`<|endofprompt|>` + `[laughter]`/`[breath]` on CosyVoice2, native on `gpt-4o-mini-tts`, Kokoro voice blending), while SiliconFlow caches uploaded `referenceAudio` URIs by SHA-256 hash instead of re-uploading megabytes per segment.
25
- - **Long-form File Synthesis, Browser Auditions, Speaker Diarization & Proxies**: `inputFile` reads `.md`/`.txt` with automatic Markdown cleanup (up to 100k chars); `voice_preview` writes `voice_previews/index.html`; `voice_stt` supports `groq`/`openai`/`siliconflow`/keyless loopback `custom`, `autoCompress: true` for >25 MB audio, and `diarize: true` (FFmpeg stereo channel split or `[Speaker A]` turn labeling); native `http://`, `https://`, `socks5://`, `socks5h://` proxy support.
22
+ - **Native 5-Theme Voice Studio "True Creative Loop" Workbench**: Switch live across **Native Airy / Warm Cream / Claude Warmth / Bright 8-Bit / Airy Mint** with an interactive Coachmark tour. Audition 300+ Edge voices & 10 `persona` presets (`0.8x–1.5x` speed), **record 8 seconds on your browser mic to create a cloned character in 1 click** (`voice_clones/` + auto-ASR + `customVoices` registration), **compose and synthesize multi-speaker dialogues visually in-panel**, and play back generated files in the **Workspace Library + click-to-seek synchronized subtitle player**.
23
+ - **Short-Video Clause/Word Subtitles, Karaoke `.ass` Export, Inline Pauses & Word-Boundary Lexicon**: `subtitleGranularity` (`"sentence"` | `"clause"` | `"word"`) + `maxCharsPerCue` (default `16`) produces punchy vertical-video captions, while `subtitleOutput: "clip.ass"` emits `1080x1920` SubStation Alpha cues with `{\kf}` word-by-word karaoke timing ready for `dsh-ffmpeg` burn-in. Inline `[pause:500ms]` / `[break:600ms]` / `[停顿0.6s]` tags compile to Edge SSML `<break>` or matched-sample-rate silence frames (automatically stripped from subtitles), and ASCII word-boundary protection on `pronunciations` ensures acronyms like `OS`, `UI`, and `SQL` never corrupt words like `POST` or `BUILD`.
24
+ - **Native MP3 Chapters (`CHAP`/`CTOC`), Zero-Drift `atempo` Dubbing, VBR Fix & BGM Ducking**: Multi-heading Markdown (`#`/`##`/`###`) in `inputFile` or `title` on `segments[]` automatically embeds clickable ID3v2 `CHAP`/`CTOC` MP3 bookmarks and writes show-notes timestamps or JSON via `chaptersOutput` (`"chapters.md"` / `"chapters.json"`). `subtitleInput` chains FFmpeg `atempo` pitch-preserving window fitting for zero cumulative drift; VBR `Xing`/`Info`/`VBRI` frame stripping fixes multi-segment player seekbars and `durationSec`; Edge TTS pools keep-alive WSS connections with transient retry; full support for `bgmFile` ducking, `-16 LUFS` mastering, acting `instructions`, and SHA-256 cached cloning.
25
+ - **Direct Video Container STT (`.mp4`/`.mkv`/`.mov`/`.avi`), Long-Form Files & Diarization**: Pass video files straight to `voice_stt` to automatically strip video tracks (`-vn`) into lightweight 16kHz mono audio via local `ffmpeg` before ASR upload, alongside `diarize: true` speaker separation, `autoCompress: true` (>25 MB), and keyless loopback Whisper. Includes 100k-char `inputFile` synthesis, `voice_preview` HTML audition pages, and native `http://`, `https://`, `socks5://`, `socks5h://` proxies.
26
26
 
27
27
  ## Install
28
28
 
@@ -32,38 +32,40 @@ Open **Plugins** in the DSH desktop app and install `dsh-voice`, or use the CLI:
32
32
  dsh plugin --profile desktop add dsh-voice
33
33
  ```
34
34
 
35
- For the web profile, replace `desktop` with `web`. Restart DSH once after installing. Requires DSH `>=0.2.2-rc.3`. The plugin bundles the `voice-workflow` skill so your assistant automatically follows best practices for voice discovery, personas, cloning, subtitle dubbing, BGM podcasts, and diarized transcription.
35
+ For the web profile, replace `desktop` with `web`. Restart DSH once after installing. Requires DSH `>=0.2.2-rc.3`. The plugin bundles the `voice-workflow` skill so your assistant automatically follows best practices for voice discovery, karaoke `.ass` subtitles, chaptered audiobooks/podcasts, cloning, zero-drift dubbing, BGM ducking, and audio/video transcription.
36
36
 
37
37
  <a id="start-using-it"></a>
38
38
  ## Get Started in 3 Steps
39
39
 
40
- 1. **Open the Voice Studio sidebar workbench (zero API keys for TTS)**: Follow the Coachmark tour in the 5-theme sidebar workbench to audition 300+ Edge voices and 10 built-in `persona` presets, manage custom characters and pronunciation rules, or have the assistant narrate Markdown files, produce podcasts, and dub `.srt` timelines immediately.
41
- 2. **Optional TTS engines (acting/cloning) or ASR keys (loopback Whisper is keyless)**: Open **DSH Settings → Plugins → Voice (`dsh-voice`)** — switch `ttsEngine` to `siliconflow`, `openai`, or `custom` for emotion `instructions` or `referenceAudio` cloning (live on save). For browser mic dictation or transcription, pick `asrEngine` (`groq`, `openai`, `siliconflow`, or `custom`) and add `asrApiKey` (loopback `127.0.0.1`/`localhost` needs no key).
42
- 3. **Preview, synthesize, or transcribe inside your workspace**: Keep scripts, `.srt`/`.vtt` timelines, BGM clips, reference clips, and recordings inside your session workspace; generated MP3s, `.txt` transcripts, `.srt`/`.vtt` subtitles, and preview pages stay safely inside the workspace and auto-number instead of overwriting existing files.
40
+ 1. **Open the Voice Studio sidebar workbench (zero API keys for TTS)**: Follow the Coachmark tour in the 5-theme workbench to audition 300+ Edge voices and 10 `persona` presets, compose and synthesize multi-speaker dialogues directly in-panel with synchronized subtitle follow-along, or ask the assistant to narrate chaptered Markdown files, export karaoke `.ass` captions, and dub `.srt` timelines.
41
+ 2. **Optional TTS engines (acting/cloning) or ASR keys (1-click 8s mic clone & keyless loopback Whisper)**: Open **DSH Settings → Plugins → Voice (`dsh-voice`)** — switch `ttsEngine` to `siliconflow`, `openai`, or `custom` for emotion `instructions` or `referenceAudio` cloning (and use the 8s browser mic recorder in Voice Studio to create cloned characters in one click). For audio/video transcription or mic dictation, pick `asrEngine` (`groq`, `openai`, `siliconflow`, or `custom`) and add `asrApiKey` (loopback `127.0.0.1`/`localhost` needs no key).
42
+ 3. **Synthesize, transcribe, and review in the Workspace Library**: Keep scripts, `.srt`/`.vtt` timelines, BGM tracks, reference clips, and audio/video files inside your session workspace; generated chaptered MP3s, `.ass`/`.srt`/`.vtt` subtitles, `chapters.md` show notes, and preview pages stay safely inside the workspace (auto-numbering without overwriting) and play right inside the **Workspace Library** tab with click-to-seek subtitles.
43
43
 
44
44
  ## Copy-Ready Prompts for Everyday Work
45
45
 
46
46
  - **Discover online voices & personas and audition in the browser**
47
47
  > Fetch online voices tagged `podcast` and generate previews in `voice_previews/index.html` comparing them with `late-night-radio` and `bilingual-tech-female` so I can pick one.
48
- - **Turn a Markdown report into a two-host podcast with BGM auto-ducking**
49
- > Read `weekly-report.md` in this workspace and turn it into a two-host dialogue using `podcast-host-female` and `podcast-host-male`, mix `assets/bgm.mp3` with `bgmDucking: true` and `normalizeLoudness: true`, and save `podcast.mp3` and `podcast.srt`.
50
- - **Reverse-dub a `.srt` timeline with custom pronunciation rules**
51
- > Dub `video.srt` in this workspace using `bilingual-tech-male`, pronouncing `Nginx` as `Engine X` while keeping `Nginx` in subtitles, save `dubbed.mp3` and `dubbed.srt`, and tell me the exact duration.
52
- - **Zero-shot clone or emotion-directed voiceover (SiliconFlow / custom engine)**
48
+ - **Turn a multi-heading Markdown report into a chaptered podcast/audiobook with BGM ducking**
49
+ > Read `weekly-report.md` in this workspace into a chaptered audio episode with `assets/bgm.mp3` (`bgmDucking: true`, `normalizeLoudness: true`), and save `podcast.mp3` (with embedded MP3 chapters), `podcast.srt`, and show notes `chapters.md`.
50
+ - **Export short-video clause captions & karaoke `.ass` with inline pauses**
51
+ > Read this short-video script with `energetic-anchor` (keep the `[pause:500ms]` beats, set `subtitleGranularity: "clause"` and `maxCharsPerCue: 14`), and export `clip.mp3` plus karaoke `clip.ass` for `dsh-ffmpeg` burn-in.
52
+ - **Zero-drift reverse-dub a `.srt` timeline with word-boundary pronunciation rules**
53
+ > Dub `video.srt` in this workspace using `bilingual-tech-male`, pronouncing `OS` as `Operating System` and `SQL` as `S Q L` without affecting words like `POST`, save `dubbed.mp3` and `dubbed.srt`, and report exact `durationSec`.
54
+ - **Zero-shot clone or emotion-directed voiceover (or 8s mic clone in Voice Studio)**
53
55
  > Clone the voice from `sample_ref.wav` (transcript: "Hello, I am Alex, nice to meet you") in this workspace and read this welcome script with a warm, cheerful tone, saving `welcome.mp3`.
54
- - **Transcribe a large meeting recording with speaker diarization & subtitles**
55
- > Transcribe `meeting.m4a` in this workspace with `diarize: true` (auto-compress if over 25 MB), save `meeting.txt` and `meeting.srt`, and list the action items.
56
+ - **Transcribe a `.mp4` video or large meeting recording with speaker diarization**
57
+ > Transcribe `lecture.mp4` directly in this workspace with `diarize: true`, save `lecture.txt` and `lecture.srt`, and list the action items.
56
58
 
57
59
  ## Quick Troubleshooting Checklist
58
60
 
59
61
  | Symptom | Likely Cause | Fix |
60
62
  | :--- | :--- | :--- |
61
- | Health check says `asrApiKey` is missing, but you only want TTS | ASR key is only needed for audio-to-text transcription | **Keep using TTS directly** — health check reports `ttsReady` / `sttReady` separately and supports `scope: "tts"` |
63
+ | Health check says `asrApiKey` is missing, but you only want TTS | ASR key is only needed for audio/video transcription | **Keep using TTS directly** — health check reports `ttsReady` / `sttReady` separately and supports `scope: "tts"` |
62
64
  | `referenceAudio` fails saying cloning engine is required | Default `edge` engine does not support zero-shot reference cloning | Switch `ttsEngine` to `siliconflow` or `custom` in **DSH Settings → Plugins → Voice** (applies live) |
63
- | `normalizeLoudness` or `bgmFile` fails saying `ffmpeg` is missing | `-16 LUFS` loudness mastering and BGM ducking use local `ffmpeg` | Install `ffmpeg` locally (or pre-process with `dsh-ffmpeg`), or omit `normalizeLoudness`/`bgmFile` |
64
- | "Must stay inside the session workspace" error | Script, subtitle, BGM, recording, reference clip, or output resolves outside workspace | Copy the file into the current DSH workspace folder first |
65
+ | Direct video STT, `bgmFile`, `normalizeLoudness`, or dense `atempo` fit says `ffmpeg` is missing | Video audio extraction (`-vn`), BGM ducking, `-16 LUFS` mastering, and `atempo` fitting require local `ffmpeg` | Install `ffmpeg` locally (or pre-process with `dsh-ffmpeg`), or pass audio directly / omit BGM & loudness flags |
66
+ | "Must stay inside the session workspace" error | Script, subtitle, BGM, video/audio, reference clip, or output resolves outside workspace | Copy the file into the current DSH workspace folder first |
65
67
  | Network / connection error | Corporate firewall or proxy required to reach Edge TTS / ASR | Set `proxyUrl` (`http://`, `https://`, `socks5://`, or `socks5h://`) in **DSH Settings → Plugins → Voice** (applies live) |
66
- | Audio exceeds 25 MB limit | Standard Whisper/ASR upload limit is 25 MB | Ask the assistant to enable `autoCompress: true` (uses local `ffmpeg`) or pre-process with `dsh-ffmpeg` |
68
+ | Audio exceeds 25 MB limit | Standard Whisper/ASR upload limit is 25 MB | Ask the assistant to enable `autoCompress: true` (videos auto-extract 16kHz mono via local `ffmpeg`) or pre-process with `dsh-ffmpeg` |
67
69
 
68
70
  ## Plugin updates
69
71
 
package/README.md CHANGED
@@ -2,9 +2,9 @@
2
2
 
3
3
  # Voice
4
4
 
5
- **把文字变成自然人声,把会议录音转成干净文字。**
5
+ **把文字变成自然人声,把音视频变成干净文字。**
6
6
 
7
- DeepSeek Harness(`>=0.2.2-rc.3`)语音工作台插件:原生提供 **5 套主题可切换的侧边栏语音工作台**(原生清透 / 米白纸感 / Claude 暖书卷 / 8-Bit 像素 / 晴空薄荷,附 Coachmark 新手引导),支持零密钥 Edge 神经网络语音合成(300+ 在线音色与 10 套人设)与 OpenAI / SiliconFlow / 自定义备用引擎(情绪演技、Kokoro 混音、SHA-256 缓存零样本克隆),具备 **精确 MP3 时长反馈(`durationMs` / `durationSec`)**、**发音词典替读(`pronunciations`)**、**`.srt` / `.vtt` 字幕反向卡点配音(`subtitleInput`)**、**播客 BGM 自动闪避与 `-16 LUFS` 响度均衡**,以及支持 **说话人区分(`diarize`)** 的多引擎语音转文字(STT),严格将读写限制在当前会话工作区内。
7
+ DeepSeek Harness(`>=0.2.2-rc.3`)语音工作台插件:原生提供 **5 套主题可切换的 Voice Studio 真·创作闭环侧边栏工作台**(原生清透 / 米白纸感 / Claude 暖书卷 / 8-Bit 像素 / 晴空薄荷,支持 8 秒麦克风一键建克隆角色、可视化多角色对话直出 MP3、工作区成品库与点击跳转字幕跟读播放器),内建零密钥 Edge 神经网络语音(连接池复用、300+ 在线音色与 10 套人设)及 OpenAI / SiliconFlow / 自定义引擎(情绪演技、Kokoro 混音、SHA-256 缓存零样本克隆),支持 **短视频词/短句粒度与卡拉 OK `.ass` 字幕(`\kf` 逐词高亮)**、**文稿内联停顿标记(`[pause:500ms]`)与词边界安全发音词典(`pronunciations`)**、**原生 MP3 章节书签(ID3v2 `CHAP`/`CTOC` + `chaptersOutput`)**、**零漂移 `.srt`/`.vtt` 字幕反向配音(`atempo`)与 VBR 多段精确时长(`durationSec`)**、**播客 BGM 自动闪避与 `-16 LUFS` 响度均衡**,以及支持 **视频文件直传抽音(`.mp4`/`.mkv`/`.mov`/`.avi`)与说话人区分(`diarize`)** 的多引擎 STT,严格限制在当前会话工作区内。
8
8
 
9
9
  [![npm](https://img.shields.io/npm/v/dsh-voice)](https://www.npmjs.com/package/dsh-voice) [![downloads](https://raw.githubusercontent.com/STARDUSTLC666/dsh-suite/npm-downloads/assets/dsh-voice-downloads.svg)](https://www.npmjs.com/package/dsh-voice) [![total downloads](https://raw.githubusercontent.com/STARDUSTLC666/dsh-suite/npm-downloads/assets/dsh-voice-total-downloads.svg)](https://www.npmjs.com/package/dsh-voice)
10
10
 
@@ -19,10 +19,10 @@ DeepSeek Harness(`>=0.2.2-rc.3`)语音工作台插件:原生提供 **5 套
19
19
 
20
20
  ## 核心亮点
21
21
 
22
- - **原生 5 套主题 Voice Studio 侧边栏工作台**:支持 **原生清透 / 米白纸感 / Claude 暖书卷 / 8-Bit 像素 / 晴空薄荷** 5 套实时同步主题与 Coachmark 新手引导;在侧边栏即可在线试听 300+ Edge 音色与 10 套内置 `persona`(支持 `0.8x–1.5x` 倍速与实时时长显示)、使用浏览器麦克风语音速记转写,并可视化管理 `customVoices` 自定义角色与 `pronunciations` 发音词典。
23
- - **精确 MP3 时长反馈、发音词典与 `.srt` / `.vtt` 字幕反向卡点配音**:跨 CBR/VBR 与多引擎流精确计算 `durationMs` / `durationSec` 并生成匹配采样率的静音帧(轻松卡准 60 秒短视频配音);`pronunciations` 在 Edge 自动编译为 SSML `<sub alias>`(字幕保留 `Nginx` 原拼写、语音读作 `Engine X`),在 OpenAI/SiliconFlow 自动做非递归替读;`subtitleInput` 支持按既有 `.srt` / `.vtt` 时间轴自动逐条测速加速并补齐毫秒静音间隙对齐画面。
24
- - **播客 BGM 自动闪避、情绪演技与 SHA-256 缓存零样本克隆**:`segments` 编排多主持人对话时可传入 `bgmFile` / `bgmVolume` / `bgmDucking`(FFmpeg `sidechaincompress` 人声闪避 + 结尾平滑淡出 + 可选 `-16 LUFS` 响度均衡);`instructions` 透传演技指令(支持 CosyVoice2 `<|endofprompt|>`、`[laughter]`/`[breath]` 与 Kokoro 混音);SiliconFlow 克隆按参考音频 SHA-256 哈希自动缓存 `speech:...` URI,多段播客无需重复上传 MB 级参考音频。
25
- - **长文直读、网页试听、说话人区分转写与代理**:`inputFile` 直读工作区 `.md` / `.txt` 并自动清理 Markdown(最高 10 万字自动按句分片);`voice_preview` 生成 `voice_previews/index.html` 交互式试听页;`voice_stt` 支持 `groq` / `openai` / `siliconflow` / `custom`(本地回环免密)、`autoCompress`(>25MB 自动压缩切片)与 `diarize: true`(双声道分离或说话人轮次标记 `[说话人 A]`);代理原生支持 `http://`、`https://`、`socks5://` 与 `socks5h://`。
22
+ - **Voice Studio 真·创作闭环侧边栏工作台(5 套主题)**:实时同步 **原生清透 / 米白纸感 / Claude 暖书卷 / 8-Bit 像素 / 晴空薄荷** 主题与 Coachmark 引导;支持在线试听 300+ Edge 音色与 10 套 `persona`、**浏览器麦克风 8 秒录音一键生成克隆角色**(自动保存至 `voice_clones/`、自动 ASR 提取参考台词并注册 `customVoices`)、**可视化多角色对话编排器面板内直出 MP3**,以及 **工作区成品库 + 点击字幕跳转跟读播放器**。
23
+ - **短视频短句/逐词字幕、卡拉 OK `.ass` 导出、内联停顿与词边界替读**:`subtitleGranularity`(`"sentence"` | `"clause"` | `"word"`)配合 `maxCharsPerCue`(默认 `16`)自动切出紧凑竖屏短句,`subtitleOutput: "clip.ass"` 直出带 `\kf` 逐词高亮时间轴的 `1080x1920` SubStation Alpha 字幕供 `dsh-ffmpeg` 烧录;文稿内嵌 `[pause:500ms]` / `[break:600ms]` / `[停顿0.6s]` 自动编译为 Edge `<break>` 或匹配采样率静音帧并在字幕中自动剥离;`pronunciations` 具备 ASCII 词边界保护(`OS`、`UI`、`SQL` 绝不误伤 `POST`、`BUILD`)。
24
+ - **有声书/播客原生 MP3 章节(`CHAP`/`CTOC`)、零漂移配音、VBR 修复与 BGM 闪避**:多级标题 Markdown(`#`/`##`/`###`)或带 `title` 的 `segments[]` 自动写入可点击跳转的 MP3 ID3v2 `CHAP`/`CTOC` 章节书签并可通过 `chaptersOutput` 导出 `chapters.md` / `chapters.json`;`subtitleInput` 叠加 FFmpeg `atempo` 保调精缩实现密集字幕零累积漂移;多段 VBR MP3 拼接自动剥离 `Xing`/`Info`/`VBRI` 帧头确保进度条与 `durationSec` 完整准确;Edge TTS 内置 Keep-Alive WebSocket 连接池与瞬断重试;完整支持 `bgmFile` 自动闪避、`-16 LUFS` 响度均衡、演技 `instructions` 与 SiliconFlow SHA-256 缓存克隆。
25
+ - **视频文件直传转写(`.mp4`/`.mkv`/`.mov`/`.avi`)、长文直读与说话人区分**:`voice_stt` 支持直接传入视频容器并通过本机 `ffmpeg` 自动剥离画面(`-vn`)提取 16kHz 单声道音频送审 ASR,同时支持 `diarize: true` 说话人区分、`autoCompress`(>25MB 自动切片)与本地回环免密 Whisper;`inputFile` 直读 10 万字长文,`voice_preview` 生成交互式网页试听页,代理原生支持 `http://`、`https://`、`socks5://` 与 `socks5h://`。
26
26
 
27
27
  ## 安装
28
28
 
@@ -32,38 +32,40 @@ DeepSeek Harness(`>=0.2.2-rc.3`)语音工作台插件:原生提供 **5 套
32
32
  dsh plugin --profile desktop add dsh-voice
33
33
  ```
34
34
 
35
- 网页版把 `desktop` 改为 `web`;安装后重启一次 DSH。宿主要求 `>=0.2.2-rc.3`。插件附带 `voice-workflow` 技能包,会在语音合成、人设配音、声音克隆、字幕卡点、播客混音与转写任务中自动指引助手采用最佳实践。
35
+ 网页版把 `desktop` 改为 `web`;安装后重启一次 DSH。宿主要求 `>=0.2.2-rc.3`。插件附带 `voice-workflow` 技能包,会在语音合成、短视频卡拉 OK 字幕、章节有声书、声音克隆、字幕卡点、播客混音与音视频转写任务中自动指引助手采用最佳实践。
36
36
 
37
37
  <a id="开始使用"></a>
38
38
  ## 最快 3 步用起来
39
39
 
40
- 1. **打开侧边栏 Voice Studio 工作台,零密钥即可开箱试听与配音**:装好插件后无需填任何密钥,即可在 5 主题侧边栏工作台跟随 Coachmark 引导试听 300+ Edge 音色与 10 种内置 `persona`、管理自定义角色与发音词典,并直接让助手直读 Markdown 长文、合成多角色播客或按 `.srt` 字幕卡点配音。
41
- 2. **按需配置备用 TTS(情绪/克隆)或 ASR 密钥(本地回环免密)**:打开 DSH「**设置 → 插件 → 语音(dsh-voice)**」或在工作台操作:如需情绪指令或零样本声音克隆,可切换 `ttsEngine` 为 `siliconflow` / `openai` / `custom`(保存即生效);如需使用浏览器麦克风速记或转写录音,选择 `asrEngine`(`groq` / `openai` / `siliconflow` / `custom`)并填入 `asrApiKey`(本地 `127.0.0.1` / `localhost` Whisper 免密)。
42
- 3. **试听选声 → 合成或转写交付**:把待转写录音、背景音乐、字幕时间轴、参考克隆音频或文稿放到当前会话工作区,生成的 MP3、`.txt` 纪要、`.srt` / `.vtt` 字幕与试听页面都会安全落在工作区,默认同名自动编号不误盖。
40
+ 1. **打开侧边栏 Voice Studio 工作台,零密钥即可试听、编排对话与合成**:装好插件后无需填任何密钥,即可在 5 主题侧边栏工作台试听 300+ Edge 音色与 10 种 `persona`、在面板内可视化编排双人对话直出 MP3 并跟读字幕,或让助手直读多章节 Markdown、导出 `.ass` 卡拉 OK 字幕及按 `.srt` 时间轴零漂移配音。
41
+ 2. **按需配置备用 TTS(情绪/克隆)或 ASR 密钥(支持 8 秒录音一键克隆与本地免密 Whisper)**:打开 DSH「**设置 → 插件 → 语音(dsh-voice)**」:如需情绪指令或零样本克隆,将 `ttsEngine` 切为 `siliconflow` / `openai` / `custom`(保存即生效,还可在工作台直接录 8 秒人声一键注册克隆角色);如需音视频转写或麦克风速记,选择 `asrEngine`(`groq` / `openai` / `siliconflow` / `custom`)并填入 `asrApiKey`(本地 `127.0.0.1` / `localhost` 免密)。
42
+ 3. **试听选声 → 合成或转写交付 → 成品库跟读验收**:把文稿、`.srt`/`.vtt` 字幕、BGM、参考克隆音频或音视频文件放进当前会话工作区,生成的带章节 MP3、`.ass`/`.srt`/`.vtt` 字幕、`chapters.md` 时间轴与试听页均安全落在工作区内(默认同名自动编号不误盖),并可在工作台「工作区成品库」随时点击字幕跳转回放。
43
43
 
44
44
  ## 平时怎么对助手说(可直接复制的自然语言示例)
45
45
 
46
46
  - **联网发现音色/人设并在浏览器试听页里对比**
47
47
  > 联网查一下带 `podcast` 标签的中文和英文音色,连同 `late-night-radio`、`bilingual-tech-female` 人设一起生成短试听,用 `voice_previews/index.html` 让我挑选。
48
- - **直接把 Markdown 长文变成带 BGM 自动闪避的双人播客**
49
- > 直接读取工作区里的 `weekly-report.md`,用 `podcast-host-female` 和 `podcast-host-male` 编排成双人播客,配上 `assets/bgm.mp3` 开启 BGM 自动闪避和 `-16 LUFS` 响度均衡,输出 `podcast.mp3` 和 `podcast.srt`。
50
- - **按现有 `.srt` 字幕反向卡点配音 + 发音词典替读**
51
- > 按工作区里的 `video.srt` 时间轴用 `bilingual-tech-male` 反向卡点配音(把 `Nginx` 替读成 `Engine X`,字幕仍保留 `Nginx`),导出 `dubbed.mp3` 和 `dubbed.srt`,并告诉我精确时长。
52
- - **用工作区 5–10 秒参考音频做零样本克隆或情绪配音(SiliconFlow / 自定义引擎)**
48
+ - **把多标题 Markdown 合成带原生 MP3 章节书签与 BGM 闪避的播客/有声书**
49
+ > 直接读取工作区里的 `weekly-report.md`,生成带原生 MP3 章节书签的有声节目(配上 `assets/bgm.mp3` 开启 BGM 自动闪避和 `-16 LUFS` 响度均衡),同时输出 `podcast.mp3`、`podcast.srt` 和节目单 `chapters.md`。
50
+ - **生成竖屏短视频短句卡拉 OK `.ass` 动态高亮字幕 + 内联停顿**
51
+ > 用 `energetic-anchor` 朗读这段口播稿(保留稿子里的 `[pause:500ms]` 停顿,设置 `subtitleGranularity: "clause"`、`maxCharsPerCue: 14`),导出 `clip.mp3` 和带逐词高亮的 `clip.ass` 方便后续烧录字幕。
52
+ - **按现有 `.srt` 字幕零漂移反向配音 + 词边界安全发音词典**
53
+ > 按工作区里的 `video.srt` 时间轴用 `bilingual-tech-male` 零漂移卡点配音(把 `OS` 替读成 `Operating System`、`SQL` 替读成 `S Q L`,不要影响 `POST`),导出 `dubbed.mp3` 和 `dubbed.srt` 并告诉我精确时长。
54
+ - **零样本克隆配音(或直接在工作台 8 秒麦克风录音建角色)**
53
55
  > 用工作区里的 `sample_ref.wav`(台词是「你好,我是小智,很高兴认识大家」)作为参考声音,用轻快带笑意的语气朗读这段欢迎词,导出 `welcome.mp3`。
54
- - **把大体积会议录音转成带说话人区分的纪要与字幕**
55
- > 把当前目录里的 `meeting.m4a` 转写成中文(如果超过 25MB 自动压缩分段,开启 `diarize: true` 区分说话人),同时输出 `meeting.txt` 和 `meeting.srt`,再提炼待办事项清单。
56
+ - **直接转写 `.mp4` 录屏/视频或大体积会议录音并区分说话人**
57
+ > 直接把工作区里的 `lecture.mp4` 抽音转写成中文(开启 `diarize: true` 区分说话人),同时输出 `lecture.txt` 和 `lecture.srt`,再提炼待办事项清单。
56
58
 
57
59
  ## 常见卡点自检
58
60
 
59
61
  | 遇到的现象 | 10 秒定位原因 | 怎么修 |
60
62
  | :--- | :--- | :--- |
61
- | 自检提示「缺 ASR 密钥」但只想做朗读配音 | ASR 密钥仅录音转文字需要,不影响朗读 | **直接正常合成语音即可**(自检可按 `tts` 范围单独查看 `ttsReady`) |
63
+ | 自检提示「缺 ASR 密钥」但只想做朗读配音 | ASR 密钥仅音视频转文字需要,不影响朗读 | **直接正常合成语音即可**(自检可按 `tts` 范围单独查看 `ttsReady`) |
62
64
  | 传入 `referenceAudio` 提示引擎不支持克隆 | 默认 `edge` 引擎不支持零样本参考音频克隆 | 在 DSH「设置 → 插件 → 语音」将 `ttsEngine` 切为 `siliconflow` 或 `custom`(保存即生效) |
63
- | 开启 `normalizeLoudness` 或 `bgmFile` 报错缺工具 | `-16 LUFS` 响度均衡与 BGM 自动闪避混音需要本机 `ffmpeg` | 本机安装 `ffmpeg`(或先用 `dsh-ffmpeg` 处理),未安装时先不开启该开关 |
64
- | 提示「必须位于会话工作区内」 | 文稿、字幕、BGM、录音、参考音频或输出路径指到了工作区外面 | 先把文件复制进当前会话工作区文件夹再让助手处理 |
65
+ | 视频直传转写、`bgmFile`、`normalizeLoudness` 或超密字幕 `atempo` 提示缺工具 | 视频抽音(`-vn`)、BGM 闪避、`-16 LUFS` 均衡与 `atempo` 精缩需本机 `ffmpeg` | 本机安装 `ffmpeg`(或配合 `dsh-ffmpeg` 处理),未安装时先传纯音频或关闭混音/均衡开关 |
66
+ | 提示「必须位于会话工作区内」 | 文稿、字幕、BGM、音视频、参考音频或输出路径指到了工作区外面 | 先把文件复制进当前会话工作区文件夹再让助手处理 |
65
67
  | 合成或转写报网络连接失败 | 受限网络无法直连在线语音 / ASR 服务 | 在 DSH「设置 → 插件 → 语音」填入本地代理 `proxyUrl`(支持 `http://` / `https://` / `socks5://` / `socks5h://`,保存即生效) |
66
- | 音频超过 25MB 无法转写 | 在线 ASR 接口单文件体积上限为 25MB | 让助手开启 `autoCompress: true`(需本机有 `ffmpeg`)自动压缩/切片转写,或先用 `dsh-ffmpeg` 处理 |
68
+ | 音频超过 25MB 无法转写 | 在线 ASR 接口单文件体积上限为 25MB | 让助手开启 `autoCompress: true`(视频文件会自动抽音压缩,需本机 `ffmpeg`)或先用 `dsh-ffmpeg` 处理 |
67
69
 
68
70
  ## 插件更新
69
71
 
package/docs/USAGE.en.md CHANGED
@@ -4,13 +4,13 @@
4
4
 
5
5
  ## Get Started in 3 Steps
6
6
 
7
- ### Step 1: Know What Works Out of the Box, 5-Theme Workbench & Optional Setup
7
+ ### Step 1: Know What Works Out of the Box, 5-Theme Creative-Loop Workbench & Optional Setup
8
8
 
9
- - **Native 5-theme Voice Studio sidebar workbench**: Open Voice Studio in the DSH sidebar with 5 synced themes (**Native Airy / Warm Cream / Claude Warmth / Bright 8-Bit / Airy Mint**) and an interactive Coachmark onboarding tour. Audition 300+ online voices and 10 personas (`0.8x–1.5x` speed + duration badge), dictate with your browser microphone (`MediaRecorder`), and manage `customVoices` and `pronunciations` rules without editing YAML.
10
- - **Text-to-speech (`voice_tts`, `voice_list`, `voice_preview`) works with zero config**: Uses Edge neural voices by default with zero API keys — discover 300+ live online Edge voices or 45+ all-age/dialect built-ins, apply 10 ready-made `persona` presets, reverse-dub `.srt`/`.vtt` timelines (`subtitleInput`), apply `pronunciations`, inspect exact `durationMs`/`durationSec`, generate an interactive browser audition page, narrate long workspace files, compose multi-speaker podcasts, and export `.srt`/`.vtt` subtitles. Switch `ttsEngine` to `siliconflow`, `openai`, or `custom` in Settings when you need emotion `instructions`, Kokoro voice blending, or zero-shot `referenceAudio` cloning.
11
- - **Speech-to-text (`voice_stt`) needs an ASR key for cloud endpoints (loopback Whisper is keyless)**:
12
- - **Recommended**: Open **DSH → Settings → Plugins → Voice (`dsh-voice`)**, pick `asrEngine` (`groq` by default, `openai`, `siliconflow`, or `custom`), and paste `asrApiKey` (masked and **applied live on save without restarting**; when `custom` points to `127.0.0.1` or `localhost`, no key is required);
13
- - **Custom voice aliases (`customVoices`) & pronunciation lexicon (`pronunciations`)**: Save reusable named characters and pronunciation overrides (e.g. `Nginx` → `Engine X`) in DSH Settings or Voice Studio (live-reloaded without restarting);
9
+ - **Native 5-theme Voice Studio "True Creative Loop" workbench**: Open Voice Studio in the DSH sidebar with 5 synced themes (**Native Airy / Warm Cream / Claude Warmth / Bright 8-Bit / Airy Mint**) and an interactive Coachmark tour. Audition 300+ online voices and 10 personas (`0.8x–1.5x` speed, 1-click MP3 download, and an **interactive click-to-seek synchronized subtitle player**), compose and synthesize multi-speaker dialogues visually right in the **Studio** tab, **record 8 seconds on your browser mic to create a cloned role in 1 click** (saves to `voice_clones/` + auto-ASR `referenceText` + registers in `customVoices`), browse generated files in the **Workspace Library** tab, and manage `pronunciations` rules without editing YAML.
10
+ - **Text-to-speech (`voice_tts`, `voice_list`, `voice_preview`) works with zero config**: Uses Edge neural voices by default (with a keep-alive WebSocket connection pool and transient retry) with zero API keys — discover 300+ live online Edge voices or 45+ all-age/dialect built-ins, apply 10 `persona` presets, export short-video clause/word captions & `.ass` karaoke subtitles (`subtitleGranularity`, `maxCharsPerCue`), use inline `[pause:500ms]` speech tags, apply word-boundary-safe `pronunciations`, embed native MP3 ID3v2 `CHAP`/`CTOC` chapters (`chaptersOutput`), zero-drift reverse-dub `.srt`/`.vtt` timelines (`subtitleInput`), inspect VBR-accurate `durationMs`/`durationSec`, and generate an interactive browser audition page. Switch `ttsEngine` to `siliconflow`, `openai`, or `custom` in Settings when you need emotion `instructions`, Kokoro blending, or zero-shot `referenceAudio` cloning.
11
+ - **Speech-to-text (`voice_stt`) supports direct video files & keyless loopback Whisper**:
12
+ - **Recommended**: Open **DSH → Settings → Plugins → Voice (`dsh-voice`)**, pick `asrEngine` (`groq` by default, `openai`, `siliconflow`, or `custom`), and paste `asrApiKey` (masked and **applied live on save without restarting**; when `custom` points to `127.0.0.1` or `localhost`, no key is required). Pass audio or `.mp4`/`.mkv`/`.mov`/`.avi` video files directly;
13
+ - **Custom voice aliases (`customVoices`) & word-boundary pronunciation lexicon (`pronunciations`)**: Save reusable named characters and pronunciation rules (e.g. `Nginx` → `Engine X`, `OS` → `Operating System` without corrupting `POST`) in DSH Settings or Voice Studio (live-reloaded without restarting);
14
14
  - **CLI / Environment variable**: Set `DSH_VOICE_ASR_KEY` (and optionally `DSH_VOICE_TTS_KEY`) or configure `cordis.patch.yml`:
15
15
 
16
16
  ```yaml
@@ -27,6 +27,8 @@
27
27
  # pronunciations:
28
28
  # - word: Nginx
29
29
  # alias: Engine X
30
+ # - word: OS # ASCII word-boundary safe: matches "Mac OS"/"OS系统", never "POST"
31
+ # alias: Operating System
30
32
  # - word: SQL
31
33
  # alias: S Q L
32
34
  # customVoices:
@@ -36,7 +38,7 @@
36
38
  # - name: kokoro-blend
37
39
  # voice: 'zf_xiaobei*0.6+zf_xiaoni*0.4'
38
40
  # - name: alex-clone
39
- # referenceAudio: voices/alex.wav
41
+ # referenceAudio: voice_clones/clone-alex.webm
40
42
  # referenceText: 'Hello, this is my reference voice sample.'
41
43
  asrEngine: groq # groq | openai | siliconflow | custom
42
44
  asrModel: whisper-large-v3-turbo # optional; defaults to engine preset
@@ -44,13 +46,13 @@
44
46
  # proxyUrl: socks5://127.0.0.1:1080 # supports http://, https://, socks5://, socks5h://
45
47
  ```
46
48
 
47
- ### Step 2: Keep Scripts, Subtitles, BGM, Reference Clips, Audio & Outputs Inside Your Session Workspace
49
+ ### Step 2: Keep Scripts, Subtitles, BGM, Reference Clips, Audio/Video & Outputs Inside Your Session Workspace
48
50
 
49
- All `voice_tts` input files (`inputFile`), reverse-dubbing subtitle files (`subtitleInput`, ≤200 cues), podcast background music (`bgmFile`), zero-shot cloning clips (`referenceAudio`, ≤10 MB), input audio (`audio`), and generated MP3 / `.txt` / `.srt` / `.vtt` / preview files must resolve under the active DSH session workspace (`session.header.cwd`) to protect host files. Copy outside files into the workspace first.
51
+ All `voice_tts` input files (`inputFile`), reverse-dubbing subtitle files (`subtitleInput`, ≤200 cues), podcast background music (`bgmFile`), zero-shot cloning clips (`referenceAudio`, ≤10 MB), input audio/video (`audio`), and generated MP3 / `.txt` / `.srt` / `.vtt` / `.ass` / `chaptersOutput` / `voice_clones/` / `voice_outputs/` / preview files must resolve under the active DSH session workspace (`session.header.cwd`) to protect host files. Copy outside files into the workspace first.
50
52
 
51
- ### Step 3: Discover Voices/Personas, Audition, Then Synthesize or Transcribe
53
+ ### Step 3: Discover Voices/Personas, Audition, Synthesize or Transcribe, and Review in Workspace Library
52
54
 
53
- Unsure which voice fits best? Call `voice_list({ online: true, locale, gender, tag })` to fetch Microsoft's live 300+ neural voices (cached 24h in memory with automatic fallback to 45+ curated built-ins, plus the 10 built-in `persona` recipes and configured `customVoices`), then call `voice_preview` to render short samples plus an interactive `voice_previews/index.html` page. The plugin ships `skills/voice-workflow/SKILL.md` so your assistant automatically uses `inputFile`, Markdown cleanup, `persona`, `segments`, `-16 LUFS` loudness normalization, and subtitle export.
55
+ Unsure which voice fits best? Call `voice_list({ online: true, locale, gender, tag })` to fetch Microsoft's live 300+ neural voices (cached 24h in memory with automatic fallback to 45+ curated built-ins, plus the 10 built-in `persona` recipes and configured `customVoices`), then call `voice_preview` to render short samples plus an interactive `voice_previews/index.html` page. The plugin ships `skills/voice-workflow/SKILL.md` so your assistant automatically pairs `inputFile`, native MP3 chapters, karaoke `.ass` subtitles, inline `[pause:500ms]` beats, `segments` BGM podcasts, and direct video STT.
54
56
 
55
57
  ---
56
58
 
@@ -58,16 +60,18 @@ Unsure which voice fits best? Call `voice_list({ online: true, locale, gender, t
58
60
 
59
61
  - **Discover online voices & personas and audition via the interactive HTML page**
60
62
  > Run `voice_list` with `online: true` for tag `podcast`, then preview a few alongside `late-night-radio` and `anime-boy` in `voice_previews/index.html` so I can pick my favorite.
61
- - **Narrate a Markdown file directly with a persona and synchronized subtitles**
62
- > Synthesize `guide.md` in this workspace using `documentary-narrator` into `guide.mp3` and export `guide.srt`.
63
- - **Create a loudness-balanced two-host podcast MP3 with BGM auto-ducking**
64
- > Turn this release note into a two-host podcast using `podcast-host-female` and `podcast-host-male`, mix `assets/bgm.mp3` with `bgmDucking: true` and `normalizeLoudness: true`, saving `episode.mp3` and `episode.srt`.
65
- - **Reverse-dub an existing `.srt`/`.vtt` timeline with custom pronunciation rules**
66
- > Dub `video.srt` in this workspace using `bilingual-tech-male`, pronouncing `Nginx` as `Engine X` (keep `Nginx` in subtitles), save `dubbed.mp3` and `dubbed.srt`, and report exact `durationSec`.
67
- - **Emotion-directed acting or 5–10s zero-shot voice cloning (SiliconFlow / OpenAI / custom engine)**
63
+ - **Narrate a multi-heading Markdown book/article with native MP3 chapters & show notes**
64
+ > Synthesize `guide.md` in this workspace using `documentary-narrator` into `guide.mp3` (embedding MP3 chapters), and export `guide.srt` plus chapter timestamps `guide-chapters.md`.
65
+ - **Export vertical short-video clause captions & karaoke `.ass` with inline pauses**
66
+ > Read this short-video script with `energetic-anchor` (keep `[pause:500ms]` pauses, set `subtitleGranularity: "clause"` and `maxCharsPerCue: 14`), and save `clip.mp3` plus karaoke `clip.ass` for `dsh-ffmpeg` burn-in.
67
+ - **Create a chaptered, loudness-balanced two-host podcast MP3 with BGM auto-ducking**
68
+ > Turn this release note into a chaptered two-host podcast (`title` on each segment) using `podcast-host-female` and `podcast-host-male`, mix `assets/bgm.mp3` with `bgmDucking: true` and `normalizeLoudness: true`, saving `episode.mp3`, `episode.srt`, and `episode-chapters.md`.
69
+ - **Zero-drift reverse-dub an existing `.srt`/`.vtt` timeline with word-boundary pronunciations**
70
+ > Dub `video.srt` in this workspace using `bilingual-tech-male`, pronouncing `OS` as `Operating System` and `Nginx` as `Engine X` (protecting words like `POST` and keeping original spelling in subtitles), save `dubbed.mp3` and `dubbed.srt`, and report exact `durationSec`.
71
+ - **Emotion-directed acting or 5–10s zero-shot voice cloning (or 8s mic clone in Voice Studio)**
68
72
  > Use `ref.wav` (transcript: "Hello, what a great day to start!") in this workspace for zero-shot cloning with cheerful acting `instructions` and inline `[laughter]`, saving `greeting.mp3`.
69
- - **Transcribe a large recording with speaker diarization, auto-compression, and subtitles**
70
- > Transcribe `interview.m4a` in this workspace with `diarize: true` (enable `autoCompress` if over 25 MB; prompt hint: DeepSeek Harness, Cordis), and save `interview.txt` and `interview.srt`.
73
+ - **Transcribe a `.mp4` video or large recording directly with speaker diarization & subtitles**
74
+ > Transcribe `lecture.mp4` directly in this workspace with `diarize: true` (prompt hint: DeepSeek Harness, Cordis), and save `lecture.txt` and `lecture.srt`.
71
75
  - **Scoped offline configuration self-check**
72
76
  > Run a voice health check to verify `ttsReady`, `sttReady`, voice/persona/alias status, prosody settings, proxy, and ASR key status.
73
77
 
@@ -81,11 +85,11 @@ Run `voice_health` first (offline check with zero network traffic; supports `sco
81
85
  | :--- | :--- | :--- |
82
86
  | Health check says `asrApiKey` is missing | `asrApiKey` and `DSH_VOICE_ASR_KEY` are both empty | Safe to ignore if you only need TTS (`ttsReady` is `true`), or pass `scope: "tts"`; for local Whisper, set `asrEngine: custom` with `127.0.0.1`/`localhost` (no key needed) |
83
87
  | `referenceAudio` fails saying cloning engine is required | `ttsEngine` is still `edge` (Edge does not support zero-shot reference cloning) | Switch `ttsEngine` to `siliconflow` (`CosyVoice2-0.5B`) or `custom` (e.g. local GPT-SoVITS / IndexTTS) in **DSH Settings → Plugins → Voice** |
84
- | `normalizeLoudness` or `bgmFile` fails saying `ffmpeg` is missing | `-16 LUFS` loudness mastering and `sidechaincompress` BGM ducking require local `ffmpeg` | Install `ffmpeg` locally (or pre-process with `dsh-ffmpeg`), or omit `normalizeLoudness`/`bgmFile` |
88
+ | Direct video STT, `normalizeLoudness`, `bgmFile`, or dense `atempo` fit says `ffmpeg` is missing | Video stream stripping (`-vn`), `-16 LUFS` loudness mastering, `sidechaincompress` BGM ducking, and `atempo` window fitting require local `ffmpeg` | Install `ffmpeg` locally (or pre-process with `dsh-ffmpeg`), or pass plain audio / omit BGM & loudness flags |
85
89
  | Invalid voice ID, rate, or pitch error | `ttsVoice` is not a valid Edge ID, built-in `persona`, or `customVoices` alias, or `ttsRate`/`ttsPitch` format is invalid | Run `voice_list` to pick a valid ID or persona, and fix `ttsVoice` / `ttsRate` / `ttsPitch` in Settings |
86
90
  | Network error on TTS or STT | Direct connection blocked by corporate network | Enter `proxyUrl` (`http://`, `https://`, `socks5://`, or `socks5h://`) in **DSH Settings → Plugins → Voice** (applies live) |
87
- | Path outside session workspace error | Script, subtitle, BGM, reference clip, audio, or output resolves outside workspace | Copy the file into the current session workspace folder first |
88
- | Audio exceeds 25 MB limit | Standard Whisper / ASR multipart cap is 25 MB | Pass `autoCompress: true` to `voice_stt` (uses local `ffmpeg` to compress/segment automatically) or pre-process with `dsh-ffmpeg` |
91
+ | Path outside session workspace error | Script, subtitle, BGM, reference clip, video/audio, or output resolves outside workspace | Copy the file into the current session workspace folder first |
92
+ | Audio exceeds 25 MB limit | Standard Whisper / ASR multipart cap is 25 MB | Pass `autoCompress: true` to `voice_stt` (video files auto-extract 16kHz mono via local `ffmpeg`; large audio compresses/segments automatically) or pre-process with `dsh-ffmpeg` |
89
93
  | ASR or TTS refuses a 3xx redirect | Redirects are blocked so audio/keys never leak to another host | Set `asrBaseUrl` / `ttsBaseUrl` directly to the final non-redirecting `/v1` endpoint |
90
94
  | Existing output file was not overwritten | `overwrite: false` (default) suffixes `_1`, `_2` safely | Use the exact path returned by the tool, or enable `overwrite` in Settings |
91
95
 
@@ -97,56 +101,53 @@ Run `voice_health` first (offline check with zero network traffic; supports `sco
97
101
 
98
102
  | Tool | Purpose | Key Parameters & Notes |
99
103
  | :--- | :--- | :--- |
100
- | `voice_tts` | Synthesize MP3 & subtitles from text, files, timelines, or dialogues | Supports `text`, `inputFile` (≤100k chars auto-chunked), `cleanMarkdown` (default `true` for `.md`), `subtitleInput` (`.srt`/`.vtt` reverse timeline dubbing, ≤200 cues), `segments` (≤50 turns, per-turn `voice`/`persona`/`rate`/`pitch`/`pauseMs`/`instructions`/`referenceAudio`/`referenceText`), `pronunciations` (`[{ word, alias }]` or `{ word: alias }`), `bgmFile` / `bgmVolume` (`0.02–1.0`, default `0.18`) / `bgmDucking` (default `true`), `normalizeLoudness` (`-16 LUFS`), `persona`, `instructions`, `referenceAudio` (≤10 MB), `referenceText`, `subtitleOutput` (`.srt`/`.vtt`), `voice`, `rate`, `pitch`, `output`; returns exact `durationMs` and `durationSec` |
101
- | `voice_stt` | Transcribe audio to text & subtitles (zero-copy upload) | `audio` required; supports `diarize` (`true` enables FFmpeg stereo channel split or `[Speaker]` turn diarization with `speakers` output), `autoCompress` (uses local `ffmpeg` for >25 MB), `timestamps`, `subtitleOutput` (`.srt`/`.vtt`), `model`, `language`, `prompt`, `output`, `engine` |
104
+ | `voice_tts` | Synthesize MP3, chapters & subtitles from text, files, timelines, or dialogues | Supports `text` (with inline `[pause:500ms]`/`[break:600ms]`/`[停顿0.6s]`), `inputFile` (≤100k chars auto-chunked; multi-heading `.md` embeds native MP3 ID3v2 `CHAP`/`CTOC` chapters), `cleanMarkdown` (default `true` for `.md`), `subtitleInput` (`.srt`/`.vtt` reverse dubbing, ≤200 cues, auto rate-scale + FFmpeg `atempo` zero-drift fit), `segments` (≤50 turns, per-turn `title`/`voice`/`persona`/`rate`/`pitch`/`pauseMs`/`instructions`/`referenceAudio`/`referenceText`), `pronunciations` (word-boundary safe), `subtitleOutput` (`.srt`/`.vtt`/`.ass` karaoke), `subtitleGranularity` (`sentence`/`clause`/`word`), `maxCharsPerCue` (`4–80`, default `16`), `chaptersOutput` (`.md`/`.txt` timestamps or `.json`), `bgmFile` / `bgmVolume` / `bgmDucking`, `normalizeLoudness` (`-16 LUFS`), `persona`, `instructions`, `referenceAudio` (≤10 MB), `referenceText`, `voice`, `rate`, `pitch`, `output`; returns VBR-accurate `durationMs`, `durationSec`, and optional `chapters` / `chaptersFile` |
105
+ | `voice_stt` | Transcribe audio & video containers to text & subtitles | `audio` required (accepts audio plus `.mp4`/`.mkv`/`.mov`/`.avi` video containers, automatically stripping `-vn` into 16kHz mono via local `ffmpeg`); supports `diarize` (`true` enables FFmpeg stereo split or `[Speaker]` turn diarization with `speakers`), `autoCompress` (uses local `ffmpeg` for >25 MB), `timestamps`, `subtitleOutput` (`.srt`/`.vtt`), `model`, `language`, `prompt`, `output`, `engine` (`groq`/`openai`/`siliconflow`/`custom`) |
102
106
  | `voice_list` | Discover online/built-in voices, personas & custom aliases | Optional `online` (`true` fetches & caches 300+ live Microsoft Edge voices for 24h, falling back to 45+ curated built-ins), `locale`, `gender` (`Female`/`Male`), `tag` (e.g. `podcast`, `multilingual`, `dialect`, `child`, `teen`, `mature`, `elder`, `anime`, `documentary`), `includePersonas` (default `true`) |
103
107
  | `voice_preview` | Bounded-concurrency (≤3) batch previews + HTML page | Optional `voices` (≤8; accepts raw voice IDs, built-in `persona` IDs/labels, or `customVoices` aliases), `text` (≤200 chars), `outputDir`, `htmlIndex` (default `true`, writes `voice_previews/index.html`) |
104
108
  | `voice_health` | Offline configuration self-check | Optional `scope` (`all`/`tts`/`stt`), reports `ttsReady` / `sttReady` across engines, voice/persona/`customVoices` aliases, prosody, proxy, and keys |
105
109
 
106
- ### `0.9.0` Voice Studio & Pro Audio Capabilities
107
-
108
- 1. **Native 5-Theme Voice Studio Sidebar Workbench**:
109
- - Features 5 live-synced themes (**Native Airy / Warm Cream / Claude Warmth / Bright 8-Bit / Airy Mint**) and an interactive spotlight Coachmark onboarding tour;
110
- - Includes 4 workbench tabs: **Voices & Personas** (live audition 300+ voices & 10 personas, `0.8x–1.5x` playback, set default), **Studio & Prompts** (live duration badge + 1-click prompt templates), **Mic & Custom Voices** (browser `MediaRecorder` dictation -> `transcribeMic`, plus `customVoices` editor), and **Lexicon & Health** (`pronunciations` rules + `ttsReady`/`sttReady` badges).
111
- 2. **Exact MP3 Duration Feedback (`durationMs` / `durationSec`) & Matched-Sample-Rate Silence Frames**:
112
- - Inspects CBR/VBR and multi-engine MP3 frame headers to compute exact output duration and generates matched-sample-rate silence frames for pauses and timeline padding so agents can reliably target exact durations (such as 60s short-video voiceovers).
113
- 3. **Pronunciation Lexicon (`pronunciations`)**:
114
- - Compiles to SSML `<sub alias>` on Edge (exported `.srt`/`.vtt` subtitles preserve `Nginx` while speech says `Engine X`), and applies longest-first non-recursive replacement on OpenAI/SiliconFlow/custom engines while keeping original spellings in fallback subtitle cues.
115
- 4. **Reverse `.srt` / `.vtt` Video Dubbing (`subtitleInput`)**:
116
- - Parses `.srt`/`.vtt` cues, measures synthesized cue duration, re-synthesizes cues exceeding their time window by >8% with scaled speech rate (clamped `-30%` to `+85%`), and inserts millisecond silence gaps to stay aligned with the video timeline.
117
- 5. **Podcast BGM Auto-Ducking (`bgmFile`, `bgmVolume`, `bgmDucking`)**:
118
- - Uses local `ffmpeg` to loop workspace BGM, duck music under speech via `sidechaincompress`, add a smooth fade-out at the end, and optionally apply `-16 LUFS` loudness normalization (`normalizeLoudness: true`).
119
- 6. **Speaker Diarization (`diarize: true`) & SiliconFlow SHA-256 Cached Cloning URI**:
120
- - `voice_stt({ diarize: true })` splits stereo channels via `ffmpeg` when available or parses/infers speaker turns from `verbose_json` to output `[说话人 A]` paragraphs, `speakers` list, and speaker-prefixed `.srt`/`.vtt` cues;
121
- - SiliconFlow zero-shot cloning uploads `referenceAudio` once per SHA-256 hash to `/uploads/audio/voice` and caches the returned `speech:...` URI across podcast segments.
122
-
123
- ### `0.8.0` 4-Layer Voice Diversity Overview
124
-
125
- 1. **300+ Live Edge Voice Discovery & 45+ All-Age/Dialect Built-ins**:
126
- - `voice_list({ online: true })` fetches and caches (24h) Microsoft's live 300+ neural voice catalog, with automatic offline fallback (`source: "builtin-fallback"`) to 45+ curated voices:
127
- - **Child / Cute**: `zh-CN-XiaoshuangNeural`, `zh-CN-XiaoyouNeural`
128
- - **Teen / Anime**: `zh-CN-YunxiaNeural`
129
- - **Mature / Documentary / Elder**: `zh-CN-YunzeNeural`, `zh-CN-XiaoqiuNeural`, `zh-CN-XiaoruiNeural`
130
- - **Passionate / Drama**: `zh-CN-YunhaoNeural`, `zh-CN-YunfengNeural`, `zh-CN-XiaohanNeural`, `zh-CN-XiaomoNeural`, `zh-CN-XiaoxuanNeural`
131
- - **Chinese Dialects & HK/TW**: Northeast (`zh-CN-liaoning-XiaobeiNeural`), Shaanxi (`zh-CN-shaanxi-XiaoniNeural`), Sichuan (`zh-CN-sichuan-YunxiNeural`), Henan (`zh-CN-henan-YundengNeural`), Shandong (`zh-CN-shandong-YunxiangNeural`), Cantonese M/F (`zh-HK-WanLungNeural` / `zh-HK-HiuMaanNeural`), Taiwanese M/F (`zh-TW-YunJheNeural` / `zh-TW-HsiaoChenNeural` / `zh-TW-HsiaoYuNeural`)
132
- 2. **10 Ready-Made Personas (`persona`)**:
133
- - `late-night-radio`, `podcast-host-female`, `podcast-host-male`, `documentary-narrator`, `bedtime-story`, `anime-boy`, `energetic-anchor`, `news-briefing`, `bilingual-tech-female`, `bilingual-tech-male`.
134
- 3. **Emotion, Acting & Kokoro Blending (`instructions`)**:
135
- - Sent natively as `instructions` on OpenAI `gpt-4o-mini-tts`;
136
- - Automatically formatted as `<|endofprompt|>` prefix on SiliconFlow `FunAudioLLM/CosyVoice2-0.5B` with inline `[laughter]` / `[breath]` paralinguistics;
137
- - Supports Kokoro voice-blending expressions (`zf_xiaobei*0.6+zf_xiaoni*0.4`) on custom endpoints.
138
- 4. **Zero-Shot Voice Cloning (`referenceAudio` + `referenceText`) & `customVoices` Aliases**:
139
- - Clone any voice from a 5–10s clean workspace clip (≤10 MB) on `ttsEngine: 'siliconflow'` or `'custom'`;
140
- - Save reusable named presets in DSH Settings `customVoices` (live-reloaded without restart) for `voice_tts`, `segments[]`, and `voice_preview`.
110
+ ### `0.10.0` True Creative Loop, Karaoke Subtitles, MP3 Chapters & Pro Audio Upgrades
111
+
112
+ 1. **Short-Video Word/Clause Subtitles & Karaoke `.ass` Export**:
113
+ - `voice_tts` adds `subtitleGranularity` (`"sentence"` default | `"clause"` compact short-video phrases | `"word"` per-word cues) and `maxCharsPerCue` (default `16`, counting CJK/fullwidth as `1` and ASCII as `0.5`, splitting at clause punctuation or `≥350ms` silence gaps);
114
+ - `subtitleOutput: "clip.ass"` outputs `1080x1920` SubStation Alpha (`v4.00+`) files with genuine `{\kf<cs>}` word-by-word karaoke highlight timing ready for `dsh-ffmpeg` burn-in.
115
+ 2. **Inline Speech Pause Tags & Word-Boundary-Safe Pronunciation Lexicon**:
116
+ - Embed `[pause:500ms]`, `[break:600ms]`, or `[停顿0.6s]` (`50–10000ms`) directly in scripts — compiled to native SSML `<break time="...ms"/>` on Edge TTS and matched-sample-rate MP3 silence frames on OpenAI/SiliconFlow/custom engines, and automatically stripped from `.srt`/`.vtt`/`.ass` subtitles;
117
+ - `pronunciations` enforces ASCII word boundaries on alphanumeric terms (`OS`, `UI`, `SQL`, `AI`) so `"Mac OS"` and `"OS系统"` match cleanly while substrings inside `"POST"`, `"HOST"`, or `"BUILD"` are never corrupted.
118
+ 3. **Audiobook & Podcast Chapters (Native MP3 ID3v2.3 `CHAP`/`CTOC` + `chaptersOutput`)**:
119
+ - Multi-heading Markdown (`#` / `##` / `###`, ignoring `#` inside fenced code blocks, with 350ms inter-chapter breathing room) in `inputFile` or `title` on `segments[]` embeds native clickable ID3v2.3 `CTOC` + `CHAP` bookmarks directly into the MP3 (preserving exact `inspectMp3Buffer` duration measurement);
120
+ - Optional `chaptersOutput` (`"chapters.md"` / `"chapters.txt"` or `"chapters.json"`) writes `MM:SS Title` show-notes timestamps or structured JSON alongside the audio.
121
+ 4. **Direct Video Container Transcription in `voice_stt`**:
122
+ - Passing `.mp4`, `.mkv`, `.mov`, or `.avi` (plus `.flv`/`.wmv`/`.m4v`/`.ts`/`.mts`) directly to `voice_stt` automatically strips the video stream (`-vn`) into lightweight 16kHz mono audio via local `ffmpeg` before uploading to ASR — even when the video file is under 25 MB (browser mic `.webm` clips still upload directly without requiring `ffmpeg`).
123
+ 5. **Zero-Drift `.srt` Dubbing (`atempo`), VBR Concatenation Fix & Edge Keep-Alive WSS Pool**:
124
+ - `subtitleInput` follows rate-scaled re-synthesis with pitch-preserving FFmpeg `atempo` window fitting whenever a dense cue still exceeds its window by `>3%`, eliminating cumulative timeline drift;
125
+ - `concatMp3Buffers` strips leading/trailing ID3 tags and per-segment `Xing`/`Info`/`VBRI` metadata frames on multi-segment VBR MP3 concatenation so player seekbars and `durationSec` reflect the full episode;
126
+ - Edge TTS reuses a proxy-keyed keep-alive WebSocket connection pool (with unref'd idle timers) and retries transient network drops automatically.
127
+ 6. **Voice Studio Workbench "True Creative Loop" (5 Tabs)**:
128
+ - **8-Second Mic Direct-to-Clone Creator** (**Mic & Custom Voices** tab): Record 5–10s in the browser to save under `voice_clones/`, auto-transcribe `referenceText` via ASR, and register in `customVoices` in 1 click;
129
+ - **Visual Multi-Speaker Dialogue Composer** (**Studio** tab): Add/edit speaker turns, optionally save `.mp3` + `.srt` to `voice_outputs/`, and synthesize & play multi-host dialogues directly inside the sidebar;
130
+ - **5th "Workspace Library" Tab & Click-to-Seek Subtitle Player**: Browse workspace audio + paired `.srt`/`.vtt` files, download MP3s in 1 click, and click any highlighted cue in the player bar to seek immediately.
131
+
132
+ ### `0.8.0`–`0.9.0` Voice Studio & Diversity Foundation
133
+
134
+ 1. **5 Synced Themes & Coachmark Tour**: Live theme switching across **Native Airy / Warm Cream / Claude Warmth / Bright 8-Bit / Airy Mint** with spotlight onboarding.
135
+ 2. **300+ Live Edge Voices, 45+ All-Age/Dialect Built-ins & 10 `persona` Presets**:
136
+ - `voice_list({ online: true })` caches Microsoft's 300+ neural voices for 24h with offline fallback (`source: "builtin-fallback"`) to 45+ curated voices across child (`XiaoshuangNeural`/`XiaoyouNeural`), teen/anime (`YunxiaNeural`), mature/documentary/elder (`YunzeNeural`/`XiaoqiuNeural`/`XiaoruiNeural`), passionate (`YunhaoNeural`), and Chinese dialects (Northeast, Shaanxi, Sichuan, Henan, Shandong, Cantonese, Taiwanese);
137
+ - 10 ready-made `persona` presets: `late-night-radio`, `podcast-host-female`, `podcast-host-male`, `documentary-narrator`, `bedtime-story`, `anime-boy`, `energetic-anchor`, `news-briefing`, `bilingual-tech-female`, `bilingual-tech-male`.
138
+ 3. **Podcast BGM Auto-Ducking (`bgmFile`), `-16 LUFS` Mastering, Emotion `instructions` & SHA-256 Cached Cloning**:
139
+ - `bgmFile` ducks music under speech via FFmpeg `sidechaincompress` with end fade-out and optional `-16 LUFS` loudness normalization (`normalizeLoudness: true`);
140
+ - `instructions` passes natively to `gpt-4o-mini-tts` or formats `<|endofprompt|>` on CosyVoice2 (with `[laughter]`/`[breath]`), plus Kokoro voice blending (`zf_xiaobei*0.6+zf_xiaoni*0.4`); SiliconFlow caches uploaded `referenceAudio` `speech:...` URIs by SHA-256 hash across segments.
141
+ 4. **Speaker Diarization (`diarize: true`)**: Splits stereo channels via `ffmpeg` or infers speaker turns from `verbose_json` to emit `[说话人 A]` paragraphs, `speakers` list, and speaker-prefixed `.srt`/`.vtt` cues.
141
142
 
142
143
  ### Engine Presets & Workspace Guardrails
143
144
 
144
145
  - **Engine presets**:
145
146
  - `asrEngine`: `groq` (`whisper-large-v3-turbo`), `openai` (`whisper-1`), `siliconflow` (`FunAudioLLM/SenseVoiceSmall`), `custom` (keyless when targeting loopback `127.0.0.1` / `localhost`).
146
147
  - `ttsEngine`: `edge` (zero-key default), `openai` (`gpt-4o-mini-tts`), `siliconflow` (`FunAudioLLM/CosyVoice2-0.5B`), `custom` (OpenAI `/audio/speech`-compatible endpoint).
147
- - **Workspace confinement & symlink protection**: All reads and writes (`inputFile`, `referenceAudio`, `audio`, and all outputs) stay under `session.header.cwd`, symlinks are never written through, new files use `O_EXCL`, and `.txt`/subtitle outputs use `0600` permissions on POSIX.
148
- - **Pinned endpoint & redirect block**: Requests only go to the configured endpoint, mismatched `engine` overrides are rejected, and `3xx` redirects abort immediately.
149
- - **Live timeouts & instant cancellation**: `timeoutMs` and `customVoices` update dynamically on config save; `voice_tts`, `voice_preview`, `voice_list`, and `voice_stt` propagate `exec.signal` to close WebSockets and abort compression/uploads immediately on cancel.
148
+ - **Workspace confinement & symlink protection**: All reads and writes (`inputFile`, `referenceAudio`, `audio`, `chaptersOutput`, Studio `voice_clones`/`voice_outputs`, and all outputs) stay under `session.header.cwd`, symlinks are never written through, new files use `O_EXCL`, and `.txt`/subtitle outputs use `0600` permissions on POSIX.
149
+ - **Pinned endpoint & redirect block**: Requests only go to the configured endpoint, mismatched `engine` overrides are rejected, `3xx` redirects abort immediately, and Studio HTTP endpoints enforce `X-DSH-VOICE: 1`, same-origin guards, and secret masking.
150
+ - **Live timeouts & instant cancellation**: `timeoutMs`, `pronunciations`, and `customVoices` update dynamically on config save; `voice_tts`, `voice_preview`, `voice_list`, and `voice_stt` propagate `exec.signal` to close WebSockets and abort extraction/compression/uploads immediately on cancel.
150
151
 
151
152
  ## Install, Uninstall & Development
152
153