dsh-voice 0.6.1 → 0.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,23 @@
1
1
  # 更新记录
2
2
 
3
+ ## 0.8.0 (2026-11-18)
4
+
5
+ - 全量音色发现与内置目录扩容(Layer 1–2):`voice_list` 支持 `online: true` 联网拉取并缓存微软 Edge 300+ 实时音色目录(24 小时内存缓存,断网自动回退);内置音色由 29 个扩充至 45 个,覆盖童声(晓双/晓悠)、少年动漫(云夏)、纪录片/成熟(云泽/晓秋)、老年(晓睿)、激昂解说(云皓)及粤语/台湾腔/河南话/山东话等方言,并附带 `tags` 标签筛选。
6
+ - Full live voice discovery & catalog expansion (Layers 1–2): `voice_list` adds `online: true` to fetch and cache Microsoft Edge's 300+ live neural voice catalog for 24h (with automatic offline fallback); built-in `VOICES` expands from 29 to 45 voices covering child, teen/anime, mature/documentary, elder, expressive commercial, and regional Chinese dialects with tag filtering.
7
+ - 10 组开箱即用角色人设(Layer 2–4):新增 `VOICE_PERSONAS` 与 `persona` 参数(如 `late-night-radio`、`documentary-narrator`、`anime-boy`、`bedtime-story`、`bilingual-tech-female` 等),可直接用于 `voice_tts`、`segments[]` 分段播客及 `voice_preview` 试听对比。
8
+ - 10 ready-made voice personas (Layers 2–4): introduce `VOICE_PERSONAS` and the `persona` parameter (e.g., `late-night-radio`, `documentary-narrator`, `anime-boy`, `bedtime-story`, `bilingual-tech-female`) across `voice_tts`, multi-speaker `segments[]`, and `voice_preview`.
9
+ - 情绪演技指令、零样本声音克隆与自定义别名(Layer 3–4):新增 `instructions` 自然语言语气控制(自动映射 SiliconFlow `CosyVoice2-0.5B` `<|endofprompt|>` 前缀及 `[laughter]`/`[breath]` 副语言标记,以及 OpenAI `gpt-4o-mini-tts` `instructions`);新增工作区边界受控的 `referenceAudio` + `referenceText` 零样本克隆与可热更新的 `customVoices` 自定义别名设置;新增 `normalizeLoudness: true` 调用本机 FFmpeg 执行 `-16 LUFS` 播客响度均衡。
10
+ - Emotion acting, zero-shot cloning & custom aliases (Layers 3–4): add natural-language `instructions` (mapped to `<|endofprompt|>` with `[laughter]`/`[breath]` on SiliconFlow `CosyVoice2-0.5B`, or `instructions` on `gpt-4o-mini-tts`), workspace-bounded `referenceAudio` + `referenceText` zero-shot voice cloning, live-reloaded `customVoices` aliases in DSH Settings, and optional `-16 LUFS` podcast loudness mastering via `normalizeLoudness: true`.
11
+
12
+ ## 0.7.0 (2026-11-18)
13
+
14
+ - 稳定性与协议加固(P0):严格校验 Edge TTS `Path:audio` 音频帧头与 `turn.end` 截断保护;HTTP 403 自动读取响应 `Date` 校准时钟偏差;剔除 XML 1.0 非法控制字符并转义 SSML 属性;扩展方言、多语种及 HD 音色校验与筛选;支持配置 `timeoutMs` 热更新读取与动态 `isConcurrencySafe`;`voice_health` 新增 `ttsReady`/`sttReady`、`scope` 影响面、韵律参数检查及本机免密钥 Whisper 探测。
15
+ - P0 hardening: validate Edge TTS `Path:audio` binary frame headers and enforce strict `turn.end` truncation guards; auto-calibrate clock skew from HTTP 403 `Date` headers; strip illegal XML 1.0 control chars and escape SSML attributes; expand dialect, multilingual, and HD voice validation/filtering; resolve `timeoutMs` dynamically with live `isConcurrencySafe`; and enrich `voice_health` with `ttsReady`/`sttReady`, `scope`, prosody validation, and keyless loopback Whisper probing.
16
+ - 核心能力升级(P1):`voice_tts` 新增 `inputFile` 文档直读、Markdown 清理与 10 万字符长文本自动分段合成,支持多角色 `segments` 播客合成及原生 MP3 静音帧拼接;`voice_tts` 与 `voice_stt` 均支持导出 `.srt`/`.vtt` 字幕(`subtitleOutput`);`voice_stt` 采用 `openAsBlob` 零拷贝上传,并支持 `autoCompress` 对超 25MB 音频调用 FFmpeg 转码或分段识别。
17
+ - P1 core upgrades: add `inputFile` reading, Markdown cleanup, and 100k-char auto-chunking to `voice_tts`, plus multi-speaker `segments` podcast synthesis with native MP3 silence frames; support `.srt`/`.vtt` `subtitleOutput` in both `voice_tts` and `voice_stt`; switch `voice_stt` to zero-copy `openAsBlob` uploads and add `autoCompress` FFmpeg transcode/segmentation for >25MB audio.
18
+ - 生态扩展与交互增强(P2):新增 `ttsEngine` 对 OpenAI / SiliconFlow / 自定义网关的支持与 `siliconflow` ASR/TTS 预设;内置零外部依赖的 `socks5://` 与 `socks5h://` 代理隧道;`voice_preview` 生成可视化 `index.html` 音色试听卡片;新增内置工作流技能 `skills/voice-workflow/SKILL.md`。
19
+ - P2 ecosystem & UX: add `ttsEngine` support for OpenAI, SiliconFlow, and custom endpoints with `siliconflow` ASR/TTS presets; add zero-dependency `socks5://`/`socks5h://` proxy tunneling; generate a visual `index.html` audition card in `voice_preview`; and bundle `skills/voice-workflow/SKILL.md`.
20
+
3
21
  ## 0.6.1 (2026-11-18)
4
22
 
5
23
  - 同步 `@stardustlc/dsh-updates` 0.2.3:侧边栏工作台平时默认隐藏底部更新栏(零高度占用),仅在检测到兼容新版本或执行更新时浮现单行精简提示条;完整更新设置与检查入口统一保留在 DSH「插件」详情页底部。
package/README.en.md CHANGED
@@ -4,7 +4,7 @@
4
4
 
5
5
  **Turn text into natural speech and recordings into clean transcripts.**
6
6
 
7
- Voice plugin for DeepSeek Harness (`>=0.2.2-rc.3`): provides zero-key Edge neural text-to-speech (TTS), batch voice previews, native live Settings forms with secret masking, and OpenAI/Groq-compatible speech-to-text (STT) strictly scoped to your session workspace.
7
+ Voice plugin for DeepSeek Harness (`>=0.2.2-rc.3`): provides zero-key Edge neural text-to-speech (with 300+ live online voices, 45+ all-age/dialect built-ins, and 10 ready-made personas) alongside OpenAI/SiliconFlow/custom engines (supporting emotion/acting instructions, Kokoro voice blending, and 5–10s zero-shot voice cloning), direct file synthesis with Markdown cleanup, multi-speaker podcasts with `-16 LUFS` loudness mastering, synchronized `.srt`/`.vtt` subtitles, interactive browser previews, and Groq/OpenAI/SiliconFlow/local Whisper speech-to-text (STT) strictly scoped to your session workspace.
8
8
 
9
9
  [![npm](https://img.shields.io/npm/v/dsh-voice)](https://www.npmjs.com/package/dsh-voice) [![downloads](https://raw.githubusercontent.com/STARDUSTLC666/dsh-suite/npm-downloads/assets/dsh-voice-downloads.svg)](https://www.npmjs.com/package/dsh-voice) [![total downloads](https://raw.githubusercontent.com/STARDUSTLC666/dsh-suite/npm-downloads/assets/dsh-voice-total-downloads.svg)](https://www.npmjs.com/package/dsh-voice)
10
10
 
@@ -17,6 +17,13 @@ Voice plugin for DeepSeek Harness (`>=0.2.2-rc.3`): provides zero-key Edge neura
17
17
  <img src="https://raw.githubusercontent.com/STARDUSTLC666/dsh-voice/master/docs/assets/intro/workflow-en-light.svg" alt="Turn text into speech and recordings into text. · workflow illustration" width="100%">
18
18
  </picture>
19
19
 
20
+ ## Highlights
21
+
22
+ - **300+ Live Edge Voices, All-Age/Dialect Built-ins & 10 Personas**: `voice_list({ online: true })` fetches and caches Microsoft's live 300+ neural voice catalog for 24h, falling back automatically offline to 45+ curated built-ins spanning child (`Xiaoshuang`/`Xiaoyou`), teen/anime (`Yunxia`), mature/documentary (`Yunze`/`Xiaoqiu`), elder (`Xiaorui`), passionate (`Yunhao`), and dialects (Northeast, Shaanxi, Sichuan, Henan, Shandong, Cantonese M/F, Taiwanese M/F). Ten built-in `persona` recipes (`late-night-radio`, `documentary-narrator`, `anime-boy`, etc.) work directly in `voice_tts`, `segments[]`, and `voice_preview`.
23
+ - **Emotion Acting, Voice Blending & Zero-Shot Cloning**: Pass natural-language acting prompts via `instructions` (mapped automatically to `<|endofprompt|>` with `[laughter]` / `[breath]` support on SiliconFlow `CosyVoice2-0.5B`, and sent natively to OpenAI `gpt-4o-mini-tts`), plus Kokoro voice-blending expressions (`zf_xiaobei*0.6+zf_xiaoni*0.4`) on custom endpoints. Clone any voice from a 5–10s workspace audio clip (`referenceAudio` + `referenceText` on `siliconflow`/`custom`) or save live-reloaded aliases in Settings `customVoices`.
24
+ - **Long-form File Synthesis, Podcasts, Loudness Mastering & Subtitles**: Read workspace `.md`/`.txt` files via `inputFile` with automatic Markdown cleanup and sentence-boundary chunking up to 100k chars; compose multi-speaker `segments` with `pauseMs` silences and optional `-16 LUFS` broadcast mastering (`normalizeLoudness: true` via local `ffmpeg`); export synchronized `.srt`/`.vtt` subtitles alongside TTS or STT outputs.
25
+ - **Interactive Auditions, Oversize Audio & Proxy Support**: `voice_preview` auditions raw voice IDs, `persona` names, and `customVoices` aliases into `voice_previews/index.html` with speed controls and one-click ID copy; `voice_stt` supports `groq`, `openai`, `siliconflow`, and keyless loopback `custom` Whisper with `autoCompress: true` for >25 MB audio; native `http://`, `https://`, `socks5://`, `socks5h://` proxy support applies live on save.
26
+
20
27
  ## Install
21
28
 
22
29
  Open **Plugins** in the DSH desktop app and install `dsh-voice`, or use the CLI:
@@ -25,32 +32,36 @@ Open **Plugins** in the DSH desktop app and install `dsh-voice`, or use the CLI:
25
32
  dsh plugin --profile desktop add dsh-voice
26
33
  ```
27
34
 
28
- For the web profile, replace `desktop` with `web`. Restart DSH once after installing. Requires DSH `>=0.2.2-rc.3`.
35
+ For the web profile, replace `desktop` with `web`. Restart DSH once after installing. Requires DSH `>=0.2.2-rc.3`. The plugin bundles the `voice-workflow` skill so your assistant automatically follows best practices for voice discovery, personas, cloning, podcasts, subtitles, and transcription.
29
36
 
30
37
  <a id="start-using-it"></a>
31
38
  ## Get Started in 3 Steps
32
39
 
33
- 1. **Read-aloud works out of the box (zero API keys)**: Open a DSH session in your target workspace and ask the assistant to preview voices or turn any script into an MP3 immediately.
34
- 2. **Add an ASR key when you need transcription (applies live)**: Open **DSH Settings → Plugins → Voice (`dsh-voice`)** and enter your Groq or OpenAI-compatible `asrApiKey` (masked and applied immediately without restarting; `DSH_VOICE_ASR_KEY` env var is also supported).
35
- 3. **Preview, synthesize, or transcribe inside your workspace**: Place recordings inside your session workspace; generated MP3s and `.txt` transcripts stay safely inside the workspace and auto-number instead of overwriting existing files.
40
+ 1. **TTS and voice personas work out of the box (zero API keys)**: Open a DSH session in your workspace to discover 300+ online Edge voices, use 10 built-in `persona` presets, audition samples, narrate long Markdown files, produce multi-speaker podcasts, and export `.srt`/`.vtt` subtitles immediately.
41
+ 2. **Optional TTS engines (acting/cloning) or ASR keys (loopback Whisper is keyless)**: Open **DSH Settings → Plugins → Voice (`dsh-voice`)** — switch `ttsEngine` to `siliconflow`, `openai`, or `custom` for emotion `instructions` or `referenceAudio` cloning (and optionally save reusable `customVoices` aliases, live on save). For transcription, pick `asrEngine` (`groq`, `openai`, `siliconflow`, or `custom`) and add `asrApiKey` (loopback `127.0.0.1`/`localhost` needs no key).
42
+ 3. **Preview, synthesize, or transcribe inside your workspace**: Keep scripts, reference clips, and recordings inside your session workspace; generated MP3s, `.txt` transcripts, `.srt`/`.vtt` subtitles, and preview pages stay safely inside the workspace and auto-number instead of overwriting existing files.
36
43
 
37
44
  ## Copy-Ready Prompts for Everyday Work
38
45
 
39
- - **Audition 3 voices first, then synthesize the full audio (zero config)**
40
- > Generate short voice preview samples for 3 natural narration voices so I can listen first, then synthesize the full MP3 once I pick one.
41
- - **Turn a written briefing into a ready-to-play MP3**
42
- > Condense today's notes into a 2-minute spoken script and generate `morning-brief.mp3` at `+10%` speed.
43
- - **Transcribe a meeting recording & extract action items**
44
- > Transcribe `meeting.m4a` in this workspace, save the full transcript to `meeting.txt`, and list the action items.
46
+ - **Discover online voices & personas and audition in the browser**
47
+ > Fetch online voices tagged `podcast` and generate previews in `voice_previews/index.html` comparing them with `late-night-radio` and `bilingual-tech-female` so I can pick one.
48
+ - **Turn a Markdown report into a balanced two-host podcast with subtitles**
49
+ > Read `weekly-report.md` in this workspace and turn it into a two-host dialogue using `podcast-host-female` and `podcast-host-male` with `normalizeLoudness` enabled, saving `podcast.mp3` and `podcast.srt`.
50
+ - **Zero-shot clone or emotion-directed voiceover (SiliconFlow / custom engine)**
51
+ > Clone the voice from `sample_ref.wav` (transcript: "Hello, I am Alex, nice to meet you") in this workspace and read this welcome script with a warm, cheerful tone, saving `welcome.mp3`.
52
+ - **Transcribe a large meeting recording with subtitles & action items**
53
+ > Transcribe `meeting.m4a` in this workspace (auto-compress if over 25 MB), save `meeting.txt` and `meeting.srt`, and list the action items.
45
54
 
46
55
  ## Quick Troubleshooting Checklist
47
56
 
48
57
  | Symptom | Likely Cause | Fix |
49
58
  | :--- | :--- | :--- |
50
- | Health check says `asrApiKey` is missing, but you only want TTS | ASR key is only needed for audio-to-text transcription | **Keep using TTS directly** — no key is required for speech synthesis or previews |
51
- | "Must stay inside the session workspace" error | Recording or output path points outside the workspace folder | Copy the audio file into the current DSH workspace folder first |
52
- | Network / connection error | Corporate firewall or proxy required to reach Edge TTS / ASR | Set `proxyUrl` (e.g. `http://127.0.0.1:7890`) in **DSH Settings → Plugins → Voice** (applies live) |
53
- | Audio exceeds 25 MB limit | Standard Whisper/ASR upload limit is 25 MB | Compress or split the audio with `dsh-ffmpeg` first, then transcribe |
59
+ | Health check says `asrApiKey` is missing, but you only want TTS | ASR key is only needed for audio-to-text transcription | **Keep using TTS directly** — health check reports `ttsReady` / `sttReady` separately and supports `scope: "tts"` |
60
+ | `referenceAudio` fails saying cloning engine is required | Default `edge` engine does not support zero-shot reference cloning | Switch `ttsEngine` to `siliconflow` or `custom` in **DSH Settings → Plugins → Voice** (applies live) |
61
+ | `normalizeLoudness` fails saying `ffmpeg` is missing | `-16 LUFS` broadcast loudness mastering uses local `ffmpeg` | Install `ffmpeg` locally (or pre-process with `dsh-ffmpeg`), or omit `normalizeLoudness` |
62
+ | "Must stay inside the session workspace" error | Script, recording, reference clip, or output resolves outside workspace | Copy the file into the current DSH workspace folder first |
63
+ | Network / connection error | Corporate firewall or proxy required to reach Edge TTS / ASR | Set `proxyUrl` (`http://`, `https://`, `socks5://`, or `socks5h://`) in **DSH Settings → Plugins → Voice** (applies live) |
64
+ | Audio exceeds 25 MB limit | Standard Whisper/ASR upload limit is 25 MB | Ask the assistant to enable `autoCompress: true` (uses local `ffmpeg`) or pre-process with `dsh-ffmpeg` |
54
65
 
55
66
  ## Plugin updates
56
67
 
package/README.md CHANGED
@@ -4,7 +4,7 @@
4
4
 
5
5
  **把文字变成自然人声,把会议录音转成干净文字。**
6
6
 
7
- DeepSeek Harness(`>=0.2.2-rc.3`)语音插件:内置零密钥 Edge 神经网络语音合成(TTS)与短试听音色挑选,支持原生设置表单与热生效、OpenAI / Groq 兼容语音转文字(STT),并严格将读写限制在当前会话工作区内。
7
+ DeepSeek Harness(`>=0.2.2-rc.3`)语音插件:内置零密钥 Edge 神经网络语音合成(支持 300+ 在线音色发现、45+ 全年龄与方言精选、10 大内置音色人设)与 OpenAI / SiliconFlow / 自定义备用引擎(支持情绪演技指令、Kokoro 混音与 5–10 秒零样本声音克隆),支持长文直读与 Markdown 清理、多角色播客拼接与 `-16 LUFS` 响度均衡、同步 `.srt` / `.vtt` 字幕导出、浏览器交互式试听页,以及 Groq / OpenAI / SiliconFlow / 本地 Whisper 语音转文字(STT),并严格将读写限制在当前会话工作区内。
8
8
 
9
9
  [![npm](https://img.shields.io/npm/v/dsh-voice)](https://www.npmjs.com/package/dsh-voice) [![downloads](https://raw.githubusercontent.com/STARDUSTLC666/dsh-suite/npm-downloads/assets/dsh-voice-downloads.svg)](https://www.npmjs.com/package/dsh-voice) [![total downloads](https://raw.githubusercontent.com/STARDUSTLC666/dsh-suite/npm-downloads/assets/dsh-voice-total-downloads.svg)](https://www.npmjs.com/package/dsh-voice)
10
10
 
@@ -17,6 +17,13 @@ DeepSeek Harness(`>=0.2.2-rc.3`)语音插件:内置零密钥 Edge 神经
17
17
  <img src="https://raw.githubusercontent.com/STARDUSTLC666/dsh-voice/master/docs/assets/intro/workflow-zh-light.svg" alt="把文字变成声音,把录音变成文字。 · 流程示意" width="100%">
18
18
  </picture>
19
19
 
20
+ ## 核心亮点
21
+
22
+ - **300+ 在线 Edge 音色、全年龄/方言与 10 大内置人设**:`voice_list({ online: true })` 动态拉取并 24 小时缓存微软全量 300+ 神经音色,断网自动回退至 45+ 精选内置音色(覆盖童声 `Xiaoshuang`/`Xiaoyou`、少年动漫 `Yunxia`、纪录片/成熟 `Yunze`/`Xiaoqiu`、老年 `Xiaorui`、激昂 `Yunhao` 及东北/陕西/四川/河南/山东/粤语男女/台湾腔男女);内置 `late-night-radio`、`documentary-narrator`、`anime-boy` 等 10 种开箱即用 `persona`,可在 `voice_tts`、`segments[]` 与 `voice_preview` 中直接调用。
23
+ - **情绪演技指令、Kokoro 混音与零样本克隆**:通过 `instructions` 传入自然语言情绪与表演提示(SiliconFlow `CosyVoice2-0.5B` 自动映射 `<|endofprompt|>` 并支持 `[laughter]` / `[breath]` 副语言标记,OpenAI `gpt-4o-mini-tts` 原生透传 `instructions`),自定义端点支持 Kokoro 混音表达式(`zf_xiaobei*0.6+zf_xiaoni*0.4`);搭配 `referenceAudio` + `referenceText`(`siliconflow` / `custom` 引擎)可用 5–10 秒工作区音频零样本克隆,或在 DSH 设置 `customVoices` 中保存热更新复用别名。
24
+ - **长文直读、多角色播客、响度均衡与零成本字幕**:`inputFile` 直读工作区 `.md` / `.txt` 并自动清理 Markdown(最高 10 万字自动按句分片拼接);`segments` 编排多主持人对话与停顿(`pauseMs`),支持 `normalizeLoudness: true`(本机 `ffmpeg` `-16 LUFS` 广电级响度均衡);零额外开销导出 `.srt` / `.vtt` 字幕(TTS 与 STT 均支持)。
25
+ - **浏览器试听页、大文件转写与代理**:`voice_preview` 支持按音色 ID、内置 `persona` 与 `customVoices` 别名批量试听并生成 `voice_previews/index.html`(倍速试听与一键复制);`voice_stt` 支持 `groq` / `openai` / `siliconflow` / `custom`(本地回环免密)及 `autoCompress`(>25MB 自动压缩切片);代理原生支持 `http://`、`https://`、`socks5://` 与 `socks5h://`(保存即生效)。
26
+
20
27
  ## 安装
21
28
 
22
29
  在 DSH 桌面版「插件」面板搜索安装 `dsh-voice`,或使用命令行:
@@ -25,32 +32,36 @@ DeepSeek Harness(`>=0.2.2-rc.3`)语音插件:内置零密钥 Edge 神经
25
32
  dsh plugin --profile desktop add dsh-voice
26
33
  ```
27
34
 
28
- 网页版把 `desktop` 改为 `web`;安装后重启一次 DSH。宿主要求 `>=0.2.2-rc.3`。
35
+ 网页版把 `desktop` 改为 `web`;安装后重启一次 DSH。宿主要求 `>=0.2.2-rc.3`。插件附带 `voice-workflow` 技能包,会在语音合成、人设配音、声音克隆、播客与转写任务中自动指引助手采用最佳实践。
29
36
 
30
37
  <a id="开始使用"></a>
31
38
  ## 最快 3 步用起来
32
39
 
33
- 1. **文字转语音开箱即用(零 API 密钥)**:装好插件后无需填任何密钥,直接在工作区新建会话,就能让助手试听多种音色并把文章、播客稿或早报合成 MP3。
34
- 2. **录音转文字按需填密钥(保存即生效)**:需要把录音转文字时,打开 DSH「**设置 → 插件 → 语音(dsh-voice)**」填入 Groq 或 OpenAI 兼容的 `asrApiKey`(自动脱敏、保存即生效;也支持环境变量 `DSH_VOICE_ASR_KEY`)。
35
- 3. **试听选声 → 合成或转写交付**:把待转写录音放到当前会话工作区,合成出的 MP3 或 `.txt` 纪要也会安全落在工作区且同名自动编号不误盖。
40
+ 1. **文字转语音与人设配音开箱即用(零 API 密钥)**:装好插件后无需填任何密钥,即可联网发现 300+ Edge 音色、使用 10 种内置 `persona`、在网页试听页对比、直读 Markdown 长文、合成多角色播客并导出 `.srt` / `.vtt` 字幕。
41
+ 2. **按需配置备用 TTS(情绪/克隆)或 ASR 密钥(本地回环免密)**:打开 DSH「**设置 → 插件 → 语音(dsh-voice)**」:如需情绪指令或零样本声音克隆,可切换 `ttsEngine` 为 `siliconflow` / `openai` / `custom`(也可在 `customVoices` 保存常用角色别名,保存即生效);如需转写录音,选择 `asrEngine`(`groq` / `openai` / `siliconflow` / `custom`)并填入 `asrApiKey`(本地 `127.0.0.1` / `localhost` Whisper 免密)。
42
+ 3. **试听选声 → 合成或转写交付**:把待转写录音、参考克隆音频或文稿放到当前会话工作区,生成的 MP3、`.txt` 纪要、`.srt` / `.vtt` 字幕与试听页面都会安全落在工作区,默认同名自动编号不误盖。
36
43
 
37
44
  ## 平时怎么对助手说(可直接复制的自然语言示例)
38
45
 
39
- - **先生成短试听挑音色,再合成完整配音(零配置)**
40
- > 挑 3 个适合播客的中英文音色给我生成短试听样例,等我选好后再把这篇早报稿合成完整 MP3。
41
- - **一步把总结变成可带走听的语音早报**
42
- > 把今天的重点文档浓缩成 300 字口播稿,用温和自然的中文男声(语速 +10%)生成 `morning-brief.mp3`。
43
- - **把工作区里的会议录音转成文字并提炼纪要**
44
- > 把当前目录里的 `meeting.m4a` 转写成中文,原稿存到 `meeting.txt`,再帮我整理一份待办事项清单。
46
+ - **联网发现音色/人设并在浏览器试听页里对比**
47
+ > 联网查一下带 `podcast` 标签的中文和英文音色,连同 `late-night-radio`、`bilingual-tech-female` 人设一起生成短试听,用 `voice_previews/index.html` 让我挑选。
48
+ - **直接把 Markdown 长文变成双人播客 + 响度均衡 + 字幕**
49
+ > 直接读取工作区里的 `weekly-report.md`,用 `podcast-host-female` 和 `podcast-host-male` 两个人设编排成双人对话播客,开启响度均衡,输出 `podcast.mp3` 和 `podcast.srt`。
50
+ - **用工作区 5–10 秒参考音频做零样本克隆或情绪配音(SiliconFlow / 自定义引擎)**
51
+ > 用工作区里的 `sample_ref.wav`(台词是「你好,我是小智,很高兴认识大家」)作为参考声音,用轻快带笑意的语气朗读这段欢迎词,导出 `welcome.mp3`。
52
+ - **把大体积会议录音转成文字、字幕与纪要**
53
+ > 把当前目录里的 `meeting.m4a` 转写成中文(如果超过 25MB 就自动压缩分段),同时输出 `meeting.txt` 和 `meeting.srt`,再提炼待办事项清单。
45
54
 
46
55
  ## 常见卡点自检
47
56
 
48
57
  | 遇到的现象 | 10 秒定位原因 | 怎么修 |
49
58
  | :--- | :--- | :--- |
50
- | 自检提示「缺 ASR 密钥」但只想做朗读配音 | ASR 密钥仅录音转文字需要,不影响朗读 | **直接正常合成语音即可**;日后需要转写录音再去「设置 → 插件 → 语音」补填 `asrApiKey` |
51
- | 提示「必须位于会话工作区内」 | 录音或输出路径指到了工作区文件夹外面 | 先把音频复制进当前会话工作区文件夹再让助手转写 |
52
- | 合成或转写报网络连接失败 | 受限网络无法直连在线语音 / ASR 服务 | 在 DSH「设置 → 插件 → 语音」填入本地 HTTP 代理 `proxyUrl`(如 `http://127.0.0.1:7890`,保存即生效) |
53
- | 音频超过 25MB 无法转写 | 在线 ASR 接口单文件体积上限为 25MB | 先让助手用 `dsh-ffmpeg` 压缩码率或按时间切段后再转写 |
59
+ | 自检提示「缺 ASR 密钥」但只想做朗读配音 | ASR 密钥仅录音转文字需要,不影响朗读 | **直接正常合成语音即可**(自检可按 `tts` 范围单独查看 `ttsReady`) |
60
+ | 传入 `referenceAudio` 提示引擎不支持克隆 | 默认 `edge` 引擎不支持零样本参考音频克隆 | 在 DSH「设置 → 插件 → 语音」将 `ttsEngine` 切为 `siliconflow` 或 `custom`(保存即生效) |
61
+ | 开启 `normalizeLoudness` 报错缺工具 | `-16 LUFS` 响度均衡需要本机 `ffmpeg` | 本机安装 `ffmpeg`(或先用 `dsh-ffmpeg` 处理),未安装时先不开启该开关 |
62
+ | 提示「必须位于会话工作区内」 | 文稿、录音、参考音频或输出路径指到了工作区外面 | 先把文件复制进当前会话工作区文件夹再让助手处理 |
63
+ | 合成或转写报网络连接失败 | 受限网络无法直连在线语音 / ASR 服务 | 在 DSH「设置 → 插件 → 语音」填入本地代理 `proxyUrl`(支持 `http://` / `https://` / `socks5://` / `socks5h://`,保存即生效) |
64
+ | 音频超过 25MB 无法转写 | 在线 ASR 接口单文件体积上限为 25MB | 让助手开启 `autoCompress: true`(需本机有 `ffmpeg`)自动压缩/切片转写,或先用 `dsh-ffmpeg` 处理 |
54
65
 
55
66
  ## 插件更新
56
67
 
package/docs/USAGE.en.md CHANGED
@@ -4,82 +4,126 @@
4
4
 
5
5
  ## Get Started in 3 Steps
6
6
 
7
- ### Step 1: Know What Works Out of the Box vs. Optional ASR Setup
7
+ ### Step 1: Know What Works Out of the Box vs. Optional TTS / ASR / Alias Setup
8
8
 
9
- - **Text-to-speech (`voice_tts`, `voice_list`, `voice_preview`) works with zero config**: No API keys needed — install the plugin and synthesize speech or audition voices right away.
10
- - **Speech-to-text (`voice_stt`) needs an ASR key when you want transcription**:
11
- - **Recommended**: Open **DSH → Settings → Plugins → Voice (`dsh-voice`)**, pick `asrEngine` (`groq` by default, `openai`, or `custom`), and paste `asrApiKey` (masked and **applied live on save without restarting**);
12
- - **CLI / Environment variable**: Set `DSH_VOICE_ASR_KEY` or configure `cordis.patch.yml`:
9
+ - **Text-to-speech (`voice_tts`, `voice_list`, `voice_preview`) works with zero config**: Uses Edge neural voices by default with zero API keys — discover 300+ live online Edge voices or 45+ all-age/dialect built-ins, apply 10 ready-made `persona` presets, generate an interactive browser audition page, narrate long workspace files, compose multi-speaker podcasts, and export `.srt`/`.vtt` subtitles. Switch `ttsEngine` to `siliconflow`, `openai`, or `custom` in Settings when you need emotion `instructions`, Kokoro voice blending, or zero-shot `referenceAudio` cloning.
10
+ - **Speech-to-text (`voice_stt`) needs an ASR key for cloud endpoints (loopback Whisper is keyless)**:
11
+ - **Recommended**: Open **DSH → Settings → Plugins → Voice (`dsh-voice`)**, pick `asrEngine` (`groq` by default, `openai`, `siliconflow`, or `custom`), and paste `asrApiKey` (masked and **applied live on save without restarting**; when `custom` points to `127.0.0.1` or `localhost`, no key is required);
12
+ - **Custom voice aliases (`customVoices`)**: Save reusable named characters (`name`, `voice`, `persona`, `rate`, `pitch`, `instructions`, `referenceAudio`, `referenceText`) in DSH Settings `customVoices` (live-reloaded without restarting) and reference them directly in `voice_tts`, `segments[]`, and `voice_preview`;
13
+ - **CLI / Environment variable**: Set `DSH_VOICE_ASR_KEY` (and optionally `DSH_VOICE_TTS_KEY`) or configure `cordis.patch.yml`:
13
14
 
14
15
  ```yaml
15
16
  - id: voice
16
17
  name: 'dsh-voice'
17
18
  config:
18
- ttsVoice: zh-CN-XiaoxiaoNeural # default voice (applies live)
19
+ ttsEngine: edge # edge | openai | siliconflow | custom
20
+ ttsVoice: zh-CN-XiaoxiaoNeural # default voice ID, built-in persona, or customVoices alias (applies live)
19
21
  ttsRate: '+0%' # default speech rate, e.g. +15% or -10%
20
22
  ttsPitch: '+0Hz' # default pitch, e.g. +2Hz or -1Hz
21
- asrEngine: groq # groq | openai | custom
23
+ # ttsBaseUrl: https://api.siliconflow.cn/v1 # optional for non-edge engines
24
+ # ttsModel: FunAudioLLM/CosyVoice2-0.5B
25
+ # ttsApiKey: sk-... # prefer DSH Settings or DSH_VOICE_TTS_KEY
26
+ # customVoices:
27
+ # - name: narrator-wang
28
+ # persona: documentary-narrator
29
+ # rate: '-5%'
30
+ # - name: kokoro-blend
31
+ # voice: 'zf_xiaobei*0.6+zf_xiaoni*0.4'
32
+ # - name: alex-clone
33
+ # referenceAudio: voices/alex.wav
34
+ # referenceText: 'Hello, this is my reference voice sample.'
35
+ asrEngine: groq # groq | openai | siliconflow | custom
22
36
  asrModel: whisper-large-v3-turbo # optional; defaults to engine preset
23
- # asrApiKey: gsk_... # prefer DSH Settings or env var DSH_VOICE_ASR_KEY
24
- # proxyUrl: http://127.0.0.1:7890 # optional HTTP(S) proxy URL
37
+ # asrApiKey: gsk_... # prefer DSH Settings or DSH_VOICE_ASR_KEY (optional for loopback custom)
38
+ # proxyUrl: socks5://127.0.0.1:1080 # supports http://, https://, socks5://, socks5h://
25
39
  ```
26
40
 
27
- ### Step 2: Keep Audio & Outputs Inside Your Session Workspace
41
+ ### Step 2: Keep Scripts, Reference Clips, Audio & Outputs Inside Your Session Workspace
28
42
 
29
- All input audio and output MP3 / `.txt` files must resolve under the active DSH session workspace (`session.header.cwd`) to protect host files. Copy outside recordings into the workspace first.
43
+ All `voice_tts` input files (`inputFile`), zero-shot cloning clips (`referenceAudio`, ≤10 MB), input audio (`audio`), and generated MP3 / `.txt` / `.srt` / `.vtt` / preview files must resolve under the active DSH session workspace (`session.header.cwd`) to protect host files. Copy outside files into the workspace first.
30
44
 
31
- ### Step 3: Audition Voices, Then Synthesize or Transcribe
45
+ ### Step 3: Discover Voices/Personas, Audition, Then Synthesize or Transcribe
32
46
 
33
- Unsure which voice fits best? Ask the assistant to generate short preview samples in parallel before rendering your full script.
47
+ Unsure which voice fits best? Call `voice_list({ online: true, locale, gender, tag })` to fetch Microsoft's live 300+ neural voices (cached 24h in memory with automatic fallback to 45+ curated built-ins, plus the 10 built-in `persona` recipes and configured `customVoices`), then call `voice_preview` to render short samples plus an interactive `voice_previews/index.html` page. The plugin ships `skills/voice-workflow/SKILL.md` so your assistant automatically uses `inputFile`, Markdown cleanup, `persona`, `segments`, `-16 LUFS` loudness normalization, and subtitle export.
34
48
 
35
49
  ---
36
50
 
37
51
  ## Copy-Ready Prompts for Everyday Work
38
52
 
39
- - **Audition voices before committing to a full render**
40
- > List the built-in voices and generate short preview MP3s for 4 distinct voices so I can choose my favorite.
41
- - **Render a polished voiceover MP3**
42
- > Synthesize this release summary into `release-note.mp3` using `en-US-AriaNeural` at `+10%` speed.
43
- - **Transcribe a recording with domain terminology hints**
44
- > Transcribe `interview.mp3` in this workspace (prompt hint: DeepSeek Harness, Cordis), save the full text to `interview.txt`, and list the 5 key takeaways.
45
- - **Offline configuration self-check**
46
- > Run a voice health check to verify my default TTS voice, proxy, and ASR key status.
53
+ - **Discover online voices & personas and audition via the interactive HTML page**
54
+ > Run `voice_list` with `online: true` for tag `podcast`, then preview a few alongside `late-night-radio` and `anime-boy` in `voice_previews/index.html` so I can pick my favorite.
55
+ - **Narrate a Markdown file directly with a persona and synchronized subtitles**
56
+ > Synthesize `guide.md` in this workspace using `documentary-narrator` into `guide.mp3` and export `guide.srt`.
57
+ - **Create a loudness-balanced two-host podcast MP3**
58
+ > Turn this release note into a two-host podcast using `podcast-host-female` and `podcast-host-male` with `normalizeLoudness: true`, saving `episode.mp3` and `episode.srt`.
59
+ - **Emotion-directed acting or 5–10s zero-shot voice cloning (SiliconFlow / OpenAI / custom engine)**
60
+ > Use `ref.wav` (transcript: "Hello, what a great day to start!") in this workspace for zero-shot cloning with cheerful acting `instructions` and inline `[laughter]`, saving `greeting.mp3`.
61
+ - **Transcribe a large recording with auto-compression and subtitles**
62
+ > Transcribe `interview.m4a` in this workspace (enable `autoCompress` if over 25 MB; prompt hint: DeepSeek Harness, Cordis), and save `interview.txt` and `interview.srt`.
63
+ - **Scoped offline configuration self-check**
64
+ > Run a voice health check to verify `ttsReady`, `sttReady`, voice/persona/alias status, prosody settings, proxy, and ASR key status.
47
65
 
48
66
  ---
49
67
 
50
68
  ## Quick Troubleshooting Checklist
51
69
 
52
- Run `voice_health` first (offline check with zero network traffic that clearly separates `voice_stt`-only issues from TTS):
70
+ Run `voice_health` first (offline check with zero network traffic; supports `scope: "tts" | "stt" | "all"` and reports `ttsReady` and `sttReady` separately):
53
71
 
54
72
  | Symptom | Usual Cause | Quick Fix |
55
73
  | :--- | :--- | :--- |
56
- | Health check says `asrApiKey` is missing | `asrApiKey` and `DSH_VOICE_ASR_KEY` are both empty | Safe to ignore if you only need TTS (`voice_tts` needs no key); for transcription, fill `asrApiKey` in **DSH Settings → Plugins → Voice** (applies live) |
57
- | Invalid voice ID error | `ttsVoice` does not match Edge neural voice format | Run `voice_list` to pick a valid ID (e.g. `zh-CN-XiaoxiaoNeural`) and update `ttsVoice` in Settings |
58
- | Network error on TTS or STT | Direct connection blocked by corporate network | Enter an HTTP(S) `proxyUrl` (e.g. `http://127.0.0.1:7890`) in **DSH Settings → Plugins → Voice** (applies live) |
59
- | Path outside session workspace error | Audio or output path resolves outside workspace | Copy the recording into the current session workspace folder first |
60
- | Audio exceeds 25 MB limit | Standard Whisper / ASR multipart cap is 25 MB | Compress or split the recording with `dsh-ffmpeg` before transcribing |
61
- | ASR refuses a 3xx redirect | Redirects are blocked so audio/keys never leak to another host | Set `asrBaseUrl` directly to the final non-redirecting `/v1` endpoint |
74
+ | Health check says `asrApiKey` is missing | `asrApiKey` and `DSH_VOICE_ASR_KEY` are both empty | Safe to ignore if you only need TTS (`ttsReady` is `true`), or pass `scope: "tts"`; for local Whisper, set `asrEngine: custom` with `127.0.0.1`/`localhost` (no key needed) |
75
+ | `referenceAudio` fails saying cloning engine is required | `ttsEngine` is still `edge` (Edge does not support zero-shot reference cloning) | Switch `ttsEngine` to `siliconflow` (`CosyVoice2-0.5B`) or `custom` (e.g. local GPT-SoVITS / IndexTTS) in **DSH Settings → Plugins → Voice** |
76
+ | `normalizeLoudness` fails saying `ffmpeg` is missing | `-16 LUFS` broadcast loudness mastering requires local `ffmpeg` | Install `ffmpeg` locally (or pre-process with `dsh-ffmpeg`), or omit `normalizeLoudness` |
77
+ | Invalid voice ID, rate, or pitch error | `ttsVoice` is not a valid Edge ID, built-in `persona`, or `customVoices` alias, or `ttsRate`/`ttsPitch` format is invalid | Run `voice_list` to pick a valid ID or persona, and fix `ttsVoice` / `ttsRate` / `ttsPitch` in Settings |
78
+ | Network error on TTS or STT | Direct connection blocked by corporate network | Enter `proxyUrl` (`http://`, `https://`, `socks5://`, or `socks5h://`) in **DSH Settings → Plugins → Voice** (applies live) |
79
+ | Path outside session workspace error | Script, reference clip, audio, or output resolves outside workspace | Copy the file into the current session workspace folder first |
80
+ | Audio exceeds 25 MB limit | Standard Whisper / ASR multipart cap is 25 MB | Pass `autoCompress: true` to `voice_stt` (uses local `ffmpeg` to compress/segment automatically) or pre-process with `dsh-ffmpeg` |
81
+ | ASR or TTS refuses a 3xx redirect | Redirects are blocked so audio/keys never leak to another host | Set `asrBaseUrl` / `ttsBaseUrl` directly to the final non-redirecting `/v1` endpoint |
62
82
  | Existing output file was not overwritten | `overwrite: false` (default) suffixes `_1`, `_2` safely | Use the exact path returned by the tool, or enable `overwrite` in Settings |
63
83
 
64
84
  ---
65
85
 
66
- ## Tool & Safety Reference
86
+ ## Tool & Voice Diversity Reference
67
87
 
68
88
  ### Registered Tools (All 5)
69
89
 
70
90
  | Tool | Purpose | Key Parameters & Notes |
71
91
  | :--- | :--- | :--- |
72
- | `voice_tts` | Synthesize MP3 from text (keyless Edge neural TTS) | `text` required (≤5000 chars); optional `voice`, `rate`, `pitch`, `output` |
73
- | `voice_stt` | Transcribe audio to text (Groq / OpenAI / custom) | `audio` required (≤25 MB); optional `model`, `language`, `prompt`, `output` |
74
- | `voice_list` | Browse curated multilingual & dialect voice catalog | No parameters |
75
- | `voice_preview` | Bounded-concurrency (≤3) batch voice preview MP3s | Optional `voices` (≤8), `text` (≤200 chars), `outputDir` |
76
- | `voice_health` | Offline configuration self-check | Checks TTS voice, proxy, and ASR engine/endpoint/key |
77
-
78
- ### Workspace & Credential Guardrails
79
-
80
- - **Workspace confinement & symlink protection**: All reads and writes stay under `session.header.cwd`, symlinks are never written through, new files use `O_EXCL`, and `.txt` transcripts use `0600` permissions on POSIX.
81
- - **Pinned endpoint & redirect block**: ASR requests only go to the configured `asrBaseUrl`, mismatched `engine` overrides are rejected, and `3xx` redirects abort immediately before sending audio.
82
- - **Instant cancellation**: `voice_tts`, `voice_preview`, and `voice_stt` propagate `exec.signal` to close WebSockets and abort uploads immediately on cancel.
92
+ | `voice_tts` | Synthesize MP3 & subtitles from text, files, or dialogues | Supports `text`, `inputFile` (≤100k chars auto-chunked), `cleanMarkdown` (default `true` for `.md`), `segments` (≤50 turns, per-turn `voice`/`persona`/`rate`/`pitch`/`pauseMs`/`instructions`/`referenceAudio`/`referenceText`), `persona`, `instructions`, `referenceAudio` (≤10 MB), `referenceText`, `normalizeLoudness` (`-16 LUFS`), `subtitleOutput` (`.srt`/`.vtt`), `voice`, `rate`, `pitch`, `output` |
93
+ | `voice_stt` | Transcribe audio to text & subtitles (zero-copy upload) | `audio` required; supports `autoCompress` (uses local `ffmpeg` for >25 MB), `timestamps`, `subtitleOutput` (`.srt`/`.vtt`), `model`, `language`, `prompt`, `output`, `engine` |
94
+ | `voice_list` | Discover online/built-in voices, personas & custom aliases | Optional `online` (`true` fetches & caches 300+ live Microsoft Edge voices for 24h, falling back to 45+ curated built-ins), `locale`, `gender` (`Female`/`Male`), `tag` (e.g. `podcast`, `multilingual`, `dialect`, `child`, `teen`, `mature`, `elder`, `anime`, `documentary`), `includePersonas` (default `true`) |
95
+ | `voice_preview` | Bounded-concurrency (≤3) batch previews + HTML page | Optional `voices` (≤8; accepts raw voice IDs, built-in `persona` IDs/labels, or `customVoices` aliases), `text` (≤200 chars), `outputDir`, `htmlIndex` (default `true`, writes `voice_previews/index.html`) |
96
+ | `voice_health` | Offline configuration self-check | Optional `scope` (`all`/`tts`/`stt`), reports `ttsReady` / `sttReady` across engines, voice/persona/`customVoices` aliases, prosody, proxy, and keys |
97
+
98
+ ### `0.8.0` 4-Layer Voice Diversity Overview
99
+
100
+ 1. **300+ Live Edge Voice Discovery & 45+ All-Age/Dialect Built-ins**:
101
+ - `voice_list({ online: true })` fetches and caches (24h) Microsoft's live 300+ neural voice catalog, with automatic offline fallback (`source: "builtin-fallback"`) to 45+ curated voices:
102
+ - **Child / Cute**: `zh-CN-XiaoshuangNeural`, `zh-CN-XiaoyouNeural`
103
+ - **Teen / Anime**: `zh-CN-YunxiaNeural`
104
+ - **Mature / Documentary / Elder**: `zh-CN-YunzeNeural`, `zh-CN-XiaoqiuNeural`, `zh-CN-XiaoruiNeural`
105
+ - **Passionate / Drama**: `zh-CN-YunhaoNeural`, `zh-CN-YunfengNeural`, `zh-CN-XiaohanNeural`, `zh-CN-XiaomoNeural`, `zh-CN-XiaoxuanNeural`
106
+ - **Chinese Dialects & HK/TW**: Northeast (`zh-CN-liaoning-XiaobeiNeural`), Shaanxi (`zh-CN-shaanxi-XiaoniNeural`), Sichuan (`zh-CN-sichuan-YunxiNeural`), Henan (`zh-CN-henan-YundengNeural`), Shandong (`zh-CN-shandong-YunxiangNeural`), Cantonese M/F (`zh-HK-WanLungNeural` / `zh-HK-HiuMaanNeural`), Taiwanese M/F (`zh-TW-YunJheNeural` / `zh-TW-HsiaoChenNeural` / `zh-TW-HsiaoYuNeural`)
107
+ 2. **10 Ready-Made Personas (`persona`)**:
108
+ - `late-night-radio`, `podcast-host-female`, `podcast-host-male`, `documentary-narrator`, `bedtime-story`, `anime-boy`, `energetic-anchor`, `news-briefing`, `bilingual-tech-female`, `bilingual-tech-male`.
109
+ 3. **Emotion, Acting & Kokoro Blending (`instructions`)**:
110
+ - Sent natively as `instructions` on OpenAI `gpt-4o-mini-tts`;
111
+ - Automatically formatted as `<|endofprompt|>` prefix on SiliconFlow `FunAudioLLM/CosyVoice2-0.5B` with inline `[laughter]` / `[breath]` paralinguistics;
112
+ - Supports Kokoro voice-blending expressions (`zf_xiaobei*0.6+zf_xiaoni*0.4`) on custom endpoints.
113
+ 4. **Zero-Shot Voice Cloning (`referenceAudio` + `referenceText`) & `customVoices` Aliases**:
114
+ - Clone any voice from a 5–10s clean workspace clip (≤10 MB) on `ttsEngine: 'siliconflow'` or `'custom'`;
115
+ - Save reusable named presets in DSH Settings `customVoices` (live-reloaded without restart) for `voice_tts`, `segments[]`, and `voice_preview`.
116
+ 5. **Podcast Loudness Mastering (`normalizeLoudness: true`)**:
117
+ - Runs local `ffmpeg` (`loudnorm=I=-16:TP=-1.5:LRA=11`) after synthesis/concatenation for even `-16 LUFS` broadcast loudness across voices.
118
+
119
+ ### Engine Presets & Workspace Guardrails
120
+
121
+ - **Engine presets**:
122
+ - `asrEngine`: `groq` (`whisper-large-v3-turbo`), `openai` (`whisper-1`), `siliconflow` (`FunAudioLLM/SenseVoiceSmall`), `custom` (keyless when targeting loopback `127.0.0.1` / `localhost`).
123
+ - `ttsEngine`: `edge` (zero-key default), `openai` (`gpt-4o-mini-tts`), `siliconflow` (`FunAudioLLM/CosyVoice2-0.5B`), `custom` (OpenAI `/audio/speech`-compatible endpoint).
124
+ - **Workspace confinement & symlink protection**: All reads and writes (`inputFile`, `referenceAudio`, `audio`, and all outputs) stay under `session.header.cwd`, symlinks are never written through, new files use `O_EXCL`, and `.txt`/subtitle outputs use `0600` permissions on POSIX.
125
+ - **Pinned endpoint & redirect block**: Requests only go to the configured endpoint, mismatched `engine` overrides are rejected, and `3xx` redirects abort immediately.
126
+ - **Live timeouts & instant cancellation**: `timeoutMs` and `customVoices` update dynamically on config save; `voice_tts`, `voice_preview`, `voice_list`, and `voice_stt` propagate `exec.signal` to close WebSockets and abort compression/uploads immediately on cancel.
83
127
 
84
128
  ## Install, Uninstall & Development
85
129
 
package/docs/USAGE.md CHANGED
@@ -4,82 +4,126 @@
4
4
 
5
5
  ## 最快 3 步用起来
6
6
 
7
- ### 第 1 步:了解零配置范围与可选 ASR 配置
7
+ ### 第 1 步:了解零配置范围与可选 TTS / ASR / 自定义别名配置
8
8
 
9
- - **文字转语音(`voice_tts` / `voice_list` / `voice_preview`)零配置即用**:无需申请任何 API 密钥,安装后联网即可试听音色与合成 MP3。
10
- - **录音转文字(`voice_stt`)按需填密钥**:
11
- - **推荐方式**:打开 **DSH → 设置 → 插件 → 语音(`dsh-voice`)**,选择 `asrEngine`(默认 `groq`,也可选 `openai` 或 `custom`),填入 `asrApiKey`(自动脱敏,**保存立即生效、无需重启**);
12
- - **命令行 / 环境变量方式**:设置环境变量 `DSH_VOICE_ASR_KEY`,或在 `cordis.patch.yml` 中配置:
9
+ - **文字转语音(`voice_tts` / `voice_list` / `voice_preview`)零配置即用**:默认使用 Edge 神经语音,无需任何 API 密钥即可联网发现 300+ 实时音色或使用 45+ 全年龄/方言内置音色、调用 10 大内置 `persona` 人设、生成网页试听页、直读长文并合成 MP3 与 `.srt`/`.vtt` 字幕;如需情绪指令(`instructions`)、Kokoro 混音或零样本参考克隆(`referenceAudio`),可在设置中切换 `ttsEngine` 为 `siliconflow`、`openai` 或 `custom`。
10
+ - **录音转文字(`voice_stt`)按需填密钥(本地回环免密)**:
11
+ - **推荐方式**:打开 **DSH → 设置 → 插件 → 语音(`dsh-voice`)**,选择 `asrEngine`(默认 `groq`,可选 `openai`、`siliconflow` 或 `custom`),填入 `asrApiKey`(自动脱敏,**保存立即生效、无需重启**;当 `custom` 指向 `127.0.0.1` / `localhost` 本地 Whisper 服务时无需密钥);
12
+ - **自定义音色别名(`customVoices`)**:可在 DSH 设置的 `customVoices` 列表中保存命名角色(支持绑定 `voice`、`persona`、`rate`、`pitch`、`instructions`、`referenceAudio`、`referenceText`),保存即热更新生效,可直接在 `voice_tts`、`segments[]` 与 `voice_preview` 中按别名调用;
13
+ - **命令行 / 环境变量方式**:设置环境变量 `DSH_VOICE_ASR_KEY`(及可选 `DSH_VOICE_TTS_KEY`),或在 `cordis.patch.yml` 中配置:
13
14
 
14
15
  ```yaml
15
16
  - id: voice
16
17
  name: 'dsh-voice'
17
18
  config:
18
- ttsVoice: zh-CN-XiaoxiaoNeural # 默认朗读音色(保存即生效)
19
+ ttsEngine: edge # edge | openai | siliconflow | custom
20
+ ttsVoice: zh-CN-XiaoxiaoNeural # 默认音色 ID、内置 persona 或 customVoices 别名(保存即生效)
19
21
  ttsRate: '+0%' # 默认语速,如 +15% 或 -10%
20
22
  ttsPitch: '+0Hz' # 默认音调,如 +2Hz 或 -1Hz
21
- asrEngine: groq # groq | openai | custom
23
+ # ttsBaseUrl: https://api.siliconflow.cn/v1 # 非 edge 引擎时可自定义
24
+ # ttsModel: FunAudioLLM/CosyVoice2-0.5B
25
+ # ttsApiKey: sk-... # 推荐在 DSH 设置页填写或用 DSH_VOICE_TTS_KEY
26
+ # customVoices:
27
+ # - name: 老王
28
+ # persona: documentary-narrator
29
+ # rate: '-5%'
30
+ # - name: 混音女声
31
+ # voice: 'zf_xiaobei*0.6+zf_xiaoni*0.4'
32
+ # - name: 小智克隆
33
+ # referenceAudio: voices/xiaozhi.wav
34
+ # referenceText: '你好,这是我的声音参考样本。'
35
+ asrEngine: groq # groq | openai | siliconflow | custom
22
36
  asrModel: whisper-large-v3-turbo # 可选;留空按引擎预设自动填入
23
- # asrApiKey: gsk_... # 推荐在 DSH 设置页填写或用环境变量 DSH_VOICE_ASR_KEY
24
- # proxyUrl: http://127.0.0.1:7890 # 网络需要 HTTP(S) 代理时填写
37
+ # asrApiKey: gsk_... # 推荐在 DSH 设置页填写或用 DSH_VOICE_ASR_KEY(本地回环 custom 可留空)
38
+ # proxyUrl: socks5://127.0.0.1:1080 # 支持 http://、https://、socks5://、socks5h://
25
39
  ```
26
40
 
27
- ### 第 2 步:在工作区内准备文稿或录音
41
+ ### 第 2 步:在工作区内准备文稿、参考音频或录音
28
42
 
29
- 为了防止路径越界读取隐私文件,所有待转写音频与生成的 MP3 / `.txt` 都严格限制在**当前会话工作区**内;外部录音请先复制进工作区目录。
43
+ 为了防止路径越界读取隐私文件,所有 `voice_tts` 直读文稿(`inputFile`)、零样本克隆参考音频(`referenceAudio`,≤10MB)、待转写音频(`audio`)以及生成的 MP3 / `.txt` / `.srt` / `.vtt` / 试听页都严格限制在**当前会话工作区**内;外部文件请先复制进工作区目录。
30
44
 
31
- ### 第 3 步:试听选定音色,合成配音或转写纪要
45
+ ### 第 3 步:发现音色/人设 → 试听对比 → 合成或转写
32
46
 
33
- 不确定哪个声音好听时,先让助手生成几段 3 秒短试听样例对比,再一键合成正式音频。
47
+ 不确定哪个声音合适时,先让助手调用 `voice_list({ online: true, locale, gender, tag })` 联网发现微软 300+ 神经音色(24h 缓存,断网自动回退 45+ 精选音色,并一并返回 10 大内置 `persona` 与 `customVoices` 别名),再用 `voice_preview` 生成短试听样例与交互式 `voice_previews/index.html` 浏览器试听页;插件内建 `skills/voice-workflow/SKILL.md` 工作流指南,助手会自动优先使用 `inputFile`、Markdown 清理、`persona`、`segments` 播客编排、`-16 LUFS` 响度均衡与字幕导出。
34
48
 
35
49
  ---
36
50
 
37
51
  ## 平时怎么对助手说(可直接复制的自然语言示例)
38
52
 
39
- - **批量短试听挑选音色(不盲猜 ID)**
40
- > 先帮我看看常用中英文音色清单,挑 4 个风格不同的音色生成一段短试听 MP3 让我选。
41
- - **把文章或报告合成可播放的语音文件**
42
- > 用 `zh-CN-YunxiNeural` 男声(语速 `+10%`)把这段产品更新说明合成到 `update-audio.mp3`。
43
- - **录音转写 + 专有名词纠偏 + 落盘存档**
44
- > 把工作区里的 `interview.mp3` 转写成中文(提示词加上专有名词:DeepSeek Harness、Cordis),完整转写存进 `interview.txt`,并总结 5 条核心结论。
45
- - **不确定代理或密钥是否就绪时先自检**
46
- > 跑一下语音插件自检,看看当前默认音色、代理和语音转文字密钥状态。
53
+ - **联网发现音色并连同内置人设生成网页试听页**
54
+ > 调用 `voice_list` 联网按 `podcast` 标签筛选中英文音色,挑几个连同 `late-night-radio` 和 `anime-boy` 人设一起生成短试听,打开 `voice_previews/index.html` 让我选。
55
+ - **直接朗读 Markdown 长文并导出同步字幕**
56
+ > 用 `documentary-narrator` 人设直接读取工作区 `guide.md` 合成 `guide.mp3`,同时导出 `guide.srt`。
57
+ - **制作响度均衡的双人对话播客 MP3**
58
+ > 把这篇更新整理成双人播客,女主持用 `podcast-host-female`,男主持用 `podcast-host-male`,开启 `normalizeLoudness` 响度均衡,导出 `episode.mp3` 和 `episode.srt`。
59
+ - **情绪演技配音或 5–10 秒零样本声音克隆(SiliconFlow / OpenAI / 自定义引擎)**
60
+ > 用工作区的 `ref.wav`(对应台词「今天天气真好,我们出发吧」)零样本克隆声音,加上 `[laughter]` 和轻快热情的 `instructions` 语气提示,合成 `greeting.mp3`。
61
+ - **大体积录音转写 + 专有名词纠偏 + 字幕导出**
62
+ > 把工作区里的 `interview.m4a` 转写成中文(如果超过 25MB 自动压缩分段,提示词加上:DeepSeek Harness、Cordis),存成 `interview.txt` 和 `interview.srt`。
63
+ - **按范围执行离线配置自检**
64
+ > 跑一下语音插件自检(分别看看 `ttsReady` 和 `sttReady`),检查默认音色/人设/自定义别名、语速音调格式、代理和 ASR 配置状态。
47
65
 
48
66
  ---
49
67
 
50
68
  ## 常见卡点自检
51
69
 
52
- 遇到问题先让助手执行 `voice_health`(纯本地检查,不发起网络请求,并明确区分哪些问题只影响 `voice_stt`):
70
+ 遇到问题先让助手执行 `voice_health`(纯本地检查,不发起网络请求;支持 `scope: "tts" | "stt" | "all"`,并分别返回 `ttsReady` 与 `sttReady`):
53
71
 
54
72
  | 现象 | 常见原因 | 一步修复 |
55
73
  | :--- | :--- | :--- |
56
- | 自检提示「缺 ASR 密钥」 | 尚未填写 `asrApiKey` 且未设 `DSH_VOICE_ASR_KEY` | 若只需朗读配音可忽略(`voice_tts` 不需要密钥);若需转写录音,在 **DSH「设置 → 插件 → 语音」**填入 `asrApiKey` 保存即生效 |
57
- | 提示音色 ID 不合法 | `ttsVoice` 拼写不符合 Edge 神经音色格式 | 让助手运行 `voice_list` 查看合法 ID(如 `zh-CN-XiaoxiaoNeural`),在设置页更新 `ttsVoice` |
58
- | `voice_tts` 或 `voice_stt` 报网络失败 | 当前网络无法直连 Edge 朗读服务或外网 ASR 接口 | 在 **DSH「设置 → 插件 → 语音」**填写 `proxyUrl`(如 `http://127.0.0.1:7890`,保存即生效) |
59
- | 提示文件必须位于会话工作区内 | 传入了工作区外部路径或越界符号链接 | 将录音复制到当前会话工作区文件夹内再执行 |
60
- | 音频超过 25MB 报超限 | OpenAI / Groq 兼容 ASR 单文件上限 25MB | 先用 `dsh-ffmpeg` 压低码率或分段裁剪后再转写 |
61
- | ASR 返回 3xx 重定向被拦截 | 为防音频与密钥被转发到第三方主机,插件拒绝自动跟随重定向 | 将 `asrBaseUrl` 直接改为服务商最终无跳转的 `/v1` 地址 |
74
+ | 自检提示「缺 ASR 密钥」 | 尚未填写 `asrApiKey` 且未设 `DSH_VOICE_ASR_KEY` | 若只需朗读配音可忽略(`ttsReady` 为 `true` 时 `voice_tts` 正常可用),或传 `scope: "tts"`;本地 Whisper 可设 `asrEngine: custom` 指向 `127.0.0.1` / `localhost` 免密运行 |
75
+ | 传入 `referenceAudio` 报错不支持克隆 | 当前 `ttsEngine` 仍为 `edge`(Edge 不支持参考音频克隆) | 在 **DSH「设置 → 插件 → 语音」**将 `ttsEngine` 切为 `siliconflow`(`CosyVoice2-0.5B`)或 `custom`(如本地 GPT-SoVITS / IndexTTS) |
76
+ | 开启 `normalizeLoudness` 报错缺 `ffmpeg` | `-16 LUFS` 广电响度均衡依赖本机 `ffmpeg` | 本机安装 `ffmpeg`(或先用 `dsh-ffmpeg` 处理),或暂时关闭 `normalizeLoudness` |
77
+ | 提示音色 ID、语速或音调不合法 | `ttsVoice` 既非合法 Edge ID 也不在内置 `persona`/`customVoices` 中,或 `ttsRate`/`ttsPitch` 格式不对 | 用 `voice_list` 查看合法 ID 与人设,在设置页修正 `ttsVoice`、`ttsRate`、`ttsPitch` |
78
+ | `voice_tts` 或 `voice_stt` 报网络失败 | 当前网络无法直连 Edge 朗读服务或外网 ASR 接口 | 在 **DSH「设置 → 插件 → 语音」**填写 `proxyUrl`(支持 `http://`、`https://`、`socks5://`、`socks5h://`,保存即生效) |
79
+ | 提示文件必须位于会话工作区内 | 传入了工作区外部路径或越界符号链接 | 将文稿、参考音频或录音复制到当前会话工作区文件夹内再执行 |
80
+ | 音频超过 25MB 报超限 | OpenAI / Groq 兼容 ASR 单文件上限 25MB | 调用 `voice_stt` 时传入 `autoCompress: true`(调用本机 `ffmpeg` 自动转码压缩/分段),或先用 `dsh-ffmpeg` 处理 |
81
+ | ASR 返回 3xx 重定向被拦截 | 为防音频与密钥被转发到第三方主机,插件拒绝自动跟随重定向 | 将 `asrBaseUrl` 或 `ttsBaseUrl` 直接改为服务商最终无跳转的 `/v1` 地址 |
62
82
  | 同名文件没有覆盖旧文件 | 默认 `overwrite: false` 防误删,自动追加 `_1`、`_2` 序号 | 查看工具返回的实际落盘路径,或在设置中显式开启 `overwrite` |
63
83
 
64
84
  ---
65
85
 
66
- ## 工具与安全边界参考
86
+ ## 工具与语音多样性参考
67
87
 
68
88
  ### 注册工具清单(共 5 个)
69
89
 
70
90
  | 工具 | 作用 | 关键参数与说明 |
71
91
  | :--- | :--- | :--- |
72
- | `voice_tts` | 文字合成 MP3(零密钥 Edge 在线神经语音) | `text` 必填(≤5000 字符);`voice` / `rate` / `pitch` / `output` 可选 |
73
- | `voice_stt` | 音频转文字(Groq / OpenAI / 自定义兼容端点) | `audio` 必填(≤25MB);`model` / `language` / `prompt` / `output` 可选 |
74
- | `voice_list` | 查看内置常用多语种与方言音色目录 | 无参数 |
75
- | `voice_preview` | 有界并发(最多 3 路)批量生成短试听 MP3 | `voices`(最多 8 个)/ `text`(≤200 字)/ `outputDir` 可选 |
76
- | `voice_health` | 本地配置自检(不触网) | 逐项检查默认音色、代理、ASR 引擎/地址/密钥与影响面 |
77
-
78
- ### 安全与文件保护细节
79
-
80
- - **工作区沙箱与符号链接防护**:所有读写路径解析后必须位于 `session.header.cwd` 内;拒绝跟随符号链接覆写,新建文件使用 `O_EXCL` 防止并发竞态覆盖,转写 `.txt` 在类 Unix 系统以 `0600` 权限写入。
81
- - **防凭据串台与防重定向泄露**:ASR 请求固定发往配置的 `asrBaseUrl`,拒绝模型调用时用 `engine` 临时切到其他厂商,且遇 `3xx` 重定向立即中止上传。
82
- - **随时可取消**:`voice_tts`、`voice_preview` 与 `voice_stt` 全程透传 `exec.signal`,用户点击停止即刻关闭 WebSocket 或中止上传。
92
+ | `voice_tts` | 文字/文稿/播客合成 MP3 与字幕 | 支持 `text`、`inputFile`(≤10 万字自动分片拼接)、`cleanMarkdown`(`.md` 默认开启)、`segments`(最多 50 段,每段独立支持 `voice`/`persona`/`rate`/`pitch`/`pauseMs`/`instructions`/`referenceAudio`/`referenceText`)、`persona`、`instructions`、`referenceAudio`(≤10MB)、`referenceText`、`normalizeLoudness`(`-16 LUFS`)、`subtitleOutput`(`.srt`/`.vtt`)、`voice`、`rate`、`pitch`、`output` |
93
+ | `voice_stt` | 音频转文字与字幕(零拷贝流式上传) | `audio` 必填;支持 `autoCompress`(>25MB 用本机 `ffmpeg` 自动压缩分段)、`timestamps`、`subtitleOutput`(`.srt`/`.vtt`)、`model`、`language`、`prompt`、`output`、`engine` |
94
+ | `voice_list` | 在线/离线发现音色、内置人设与自定义别名 | 可选 `online`(`true` 拉取并 24h 缓存微软 300+ 实时音色,断网自动回退 45+ 精选内置)、`locale`、`gender`(`Female`/`Male`)、`tag`(如 `podcast`/`multilingual`/`dialect`/`child`/`teen`/`mature`/`elder`/`anime`/`documentary`)、`includePersonas`(默认 `true`) |
95
+ | `voice_preview` | 有界并发(≤3)批量试听并生成网页试听页 | 可选 `voices`(≤8,支持原始音色 ID、10 大内置 `persona` ID/名称及 `customVoices` 别名)、`text`(≤200 字)、`outputDir`、`htmlIndex`(默认 `true`,生成 `voice_previews/index.html`) |
96
+ | `voice_health` | 本地配置自检(不触网) | 可选 `scope`(`all`/`tts`/`stt`),返回 `ttsReady` / `sttReady` 并校验引擎、音色/人设/`customVoices` 别名、语速音调、代理与密钥 |
97
+
98
+ ### `0.8.0` 语音多样性四层能力一览
99
+
100
+ 1. **全量发现与全年龄/方言内置(45+ 内置 / 300+ 在线)**:
101
+ - `voice_list({ online: true })` 联网拉取微软 Edge 300+ 全量神经音色并内存缓存 24 小时,离线自动回退至 45+ 精选内置音色:
102
+ - **童声/幼童**:`zh-CN-XiaoshuangNeural`、`zh-CN-XiaoyouNeural`
103
+ - **少年/动漫**:`zh-CN-YunxiaNeural`
104
+ - **成熟/纪录片/老年**:`zh-CN-YunzeNeural`、`zh-CN-XiaoqiuNeural`、`zh-CN-XiaoruiNeural`
105
+ - **激昂/情感/戏剧**:`zh-CN-YunhaoNeural`、`zh-CN-YunfengNeural`、`zh-CN-XiaohanNeural`、`zh-CN-XiaomoNeural`、`zh-CN-XiaoxuanNeural`
106
+ - **方言与港澳台**:东北(`zh-CN-liaoning-XiaobeiNeural`)、陕西(`zh-CN-shaanxi-XiaoniNeural`)、四川(`zh-CN-sichuan-YunxiNeural`)、河南(`zh-CN-henan-YundengNeural`)、山东(`zh-CN-shandong-YunxiangNeural`)、粤语男女(`zh-HK-WanLungNeural` / `zh-HK-HiuMaanNeural`)、台湾男女(`zh-TW-YunJheNeural` / `zh-TW-HsiaoChenNeural` / `zh-TW-HsiaoYuNeural`)
107
+ 2. **10 大开箱即用音色人设(`persona`)**:
108
+ - `late-night-radio`、`podcast-host-female`、`podcast-host-male`、`documentary-narrator`、`bedtime-story`、`anime-boy`、`energetic-anchor`、`news-briefing`、`bilingual-tech-female`、`bilingual-tech-male`。
109
+ 3. **情绪演技指令与 Kokoro 混音(`instructions`)**:
110
+ - `gpt-4o-mini-tts` 直接发送原生 `instructions` 字段;
111
+ - SiliconFlow `FunAudioLLM/CosyVoice2-0.5B` 自动将 `instructions` 拼接为 `<|endofprompt|>` 前缀,并支持文本内联 `[laughter]`、`[breath]` 等副语言标记;
112
+ - 自定义 Kokoro 端点支持如 `zf_xiaobei*0.6+zf_xiaoni*0.4` 的音色加权混合表达式。
113
+ 4. **零样本参考克隆(`referenceAudio` + `referenceText`)与 `customVoices` 别名**:
114
+ - 在 `siliconflow` 或 `custom` 引擎下传入工作区内 5–10 秒干净语音(≤10MB)及对应参考台词即可零样本克隆;
115
+ - 在 DSH 设置 `customVoices` 配置一次即可按中文或英文别名在 `voice_tts`、`segments[]` 与 `voice_preview` 中热更新复用。
116
+ 5. **播客广电级响度均衡(`normalizeLoudness: true`)**:
117
+ - 合成或拼接完成后调用本机 `ffmpeg` 执行 `-16 LUFS`(`loudnorm=I=-16:TP=-1.5:LRA=11`)标准化,消除多角色或跨引擎音色之间的音量落差。
118
+
119
+ ### 引擎预设与安全细节
120
+
121
+ - **引擎预设**:
122
+ - `asrEngine`:`groq`(`whisper-large-v3-turbo`)、`openai`(`whisper-1`)、`siliconflow`(`FunAudioLLM/SenseVoiceSmall`)、`custom`(支持本地回环地址免密钥)。
123
+ - `ttsEngine`:`edge`(零密钥默认)、`openai`(`gpt-4o-mini-tts`)、`siliconflow`(`FunAudioLLM/CosyVoice2-0.5B`)、`custom`(OpenAI `/audio/speech` 兼容端点)。
124
+ - **工作区沙箱与符号链接防护**:所有读写路径(含 `inputFile`、`referenceAudio`、`audio` 与各类输出)解析后必须位于 `session.header.cwd` 内;拒绝跟随符号链接覆写,新建文件使用 `O_EXCL` 防止并发竞态覆盖,转写 `.txt` 与字幕在类 Unix 系统以 `0600` 权限写入。
125
+ - **防凭据串台与防重定向泄露**:ASR 与 TTS 请求固定发往配置端点,拒绝模型调用时用 `engine` 临时切换厂商,遇 `3xx` 重定向立即中止请求。
126
+ - **动态超时与随时可取消**:`timeoutMs` 与 `customVoices` 随设置热更新即时生效;`voice_tts`、`voice_preview`、`voice_list` 与 `voice_stt` 全程透传 `exec.signal`,取消即关闭连接或中止压缩/上传。
83
127
 
84
128
  ## 安装、卸载与开发
85
129