dsh-voice-mode 0.1.0

package/LICENSE

@@ -0,0 +1,23 @@
+MIT License
+
+Copyright (c) 2026 dsh-voice-mode contributors
+Portions derived from dsh-voice (https://github.com/haoku123/dsh-voice):
+Copyright (c) 2026 haoku123
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.en.md

@@ -0,0 +1,267 @@
+# dsh-voice-mode
+
+[![npm version](https://img.shields.io/npm/v/dsh-voice-mode?style=flat-square)](https://www.npmjs.com/package/dsh-voice-mode)
+[![License](https://img.shields.io/npm/l/dsh-voice-mode?style=flat-square)](LICENSE)
+[![dsh-plugin](https://img.shields.io/badge/dsh-plugin-voice-%232ea043?style=flat-square)](https://github.com/topics/dsh-plugin)
+
+> Full-duplex voice conversation mode for DeepSeek Harness (dsh): speak, get a
+> spoken answer. Streamed zipformer2 ASR → editable draft → auto send → the
+> final reply is read out sentence-by-sentence via Edge TTS, and your voice
+> interrupts playback and the running turn. No API key.
+>
+> 中文说明见 [README.md](./README.md)。
+
+![demo](assets/demo.gif)
+
+## Features
+
+- **Voice mode**: toggle with the microphone button in the input toolbar or the global shortcut `Ctrl+Shift+V`; globally single-active (only one session is in voice mode at a time; switching sessions yields automatically)
+- **Two interaction modes (switchable in settings)**:
+  - `toggle` (default) continuous listening: RMS VAD segmentation → streaming zipformer2 ASR (words appear as you speak, live caption preview) → automatic sentence split and send after 2 s of silence; hold `Ctrl` to force an immediate send
+  - `hold` push-to-talk: short tap to enter/exit, **hold the mic button to talk, release to send** (swipe up to cancel, `Esc`/blur abandons the segment); hold `Ctrl` to record-by-keyboard, release to send
+- **Wake word (optional, off by default)**: after setting `wakeWord`, entering voice mode starts in standby, and recognition only begins once the wake word is spoken (e.g. `你好小D`), preventing accidental triggers
+- **Output pipeline**: only the final answer's `text-delta` is read (reasoning/tool calls are skipped), streamed sentence-by-sentence via Edge TTS with a live caption overlay at the bottom-right; tool calls trigger a beep; the full text is still written to the chat; in voice mode a spoken-format system prompt is injected (short natural sentences, no Markdown decoration), and the reader side strips markers as well for a smoother listening experience
+- **Barge-in**: three sensitivity levels of voice-onset detection → local mute + host synth queue invalidation (epoch) + running turn cancellation (the half-finished part is kept and naturally flows into your new message)
+- **Lazy model download with progress**: the zipformer2 Chinese streaming model (~160 MB, `.part` resumable) is downloaded on first use with live progress in the status bar; `npm run prefetch` can pre-download it
+- **Resilience**: mic-denied red hint, visible model-download failure, TTS unreachable status hint (auto retry), failed submit keeps the text in the draft, SSE auto-reconnect
+- **Settings**: Settings → Plugins → voice-mode, with voice / rate / interrupt sensitivity / silence pause / idle timeout / model mirror / auto send / interaction mode / wake word; **voices are previewable** (the "试听/Preview" button synthesizes and plays the current voice at the current rate instantly, no need to enter voice mode; custom ShortNames are previewable too)
+- **Idle exit**: auto-exit and mic release after 10 minutes of inactivity
+
+## Interaction gestures
+
+| Gesture | Behaviour |
+| --- | --- |
+| Click the mic button / `Ctrl+Shift+V` | Enter / exit voice mode |
+| Just speak, pause 2 s (toggle) | Auto sentence split and send |
+| Hold `Ctrl` (toggle, ≥250 ms speech) | Force-send the current segment immediately |
+| **Hold the mic button (hold)** | Hold to talk, release to send; swipe up / `Esc` / blur abandons the segment; <250 ms tap exits the mode |
+| Hold `Ctrl` (hold, ≥600 ms) | Keyboard push-to-talk, release to send |
+| Speak the wake word first (if configured) | Activate from standby into listening (then recognition and sending begin) |
+| Speak while AI is reading | Interrupt playback and cancel the running turn |
+| Type in the input box | Auto-exit voice mode (draft is kept) |
+
+## Installation
+
+**Requirements**: dsh web (Node ≥ 18), a modern browser (Chrome / Edge / Firefox, supporting `getUserMedia` and Web Audio).
+
+```sh
+# Option 1: from npm (recommended)
+dsh plugin --profile web add dsh-voice-mode
+# Equivalent via npx if the dsh CLI is not installed locally:
+npx -y @deepseek-ai/dsh plugin --profile web add dsh-voice-mode
+
+# Option 2: local tarball
+dsh plugin --profile web add ./dsh-voice-mode-0.1.0.tgz
+
+# Option 3: from source
+git clone https://github.com/qishuilalala/dsh-voice-mode.git
+cd dsh-voice-mode/plugin/dsh-voice-mode && pnpm install && pnpm build
+dsh plugin --profile web add .
+```
+
+**Bundle plugins require a dsh restart to take effect** (restart methods by platform):
+
+- **Linux (systemd)**: `systemctl restart dsh`
+- **Windows / macOS / manual hosting**: restart your dsh process (kill and run `dsh web` again, or restart it in your service manager)
+
+**Optional**: pre-download the ASR model to reduce the download wait on the first voice-mode entry:
+
+```sh
+npm run prefetch          # run inside the plugin dir; writes to the platform cache dir
+# or specify the cache location: node scripts/prefetch.mjs --cache-dir /where/ever/models
+```
+
+## Usage
+
+1. Click the mic button in the input toolbar (or press `Ctrl+Shift+V`) to enter voice mode; a status bar appears above the input box
+2. Choose how to speak: just talk and let the 2 s pause auto-send (toggle); or hold the mic button and release to send (hold)
+3. The AI answer is read sentence-by-sentence with a caption overlay at the bottom-right; click "Skip" or just start speaking to interrupt
+4. Click "Exit" in the status bar (or press `Ctrl+Shift+V` again) to leave voice mode
+
+On first entry the recognition model is downloaded; the status bar shows `正在加载模型… <file> <percent>%`.
+
+If a wake word is configured, you land in standby first (the status bar prompts `说『唤醒词』开始`), and recognizing starts after you speak the wake word.
+
+## Settings (Settings → Plugins → Plugins config → 语音模式)
+
+| Key | Default | Description |
+| --- | --- | --- |
+| `voice` | `zh-CN-XiaoxiaoNeural` | Edge TTS voice (see the common voices table below), **applies live**; the inline "试听" button previews it at the current rate (both listed voices and custom ShortNames are previewable; failures show a visible hint) |
+| `rate` | `1.0` | Reading speed multiplier (0.5 slow ～ 2.0 fast), **applies live** |
+| `interruptLevel` | `0` | Barge-in sensitivity: 0 high threshold / 1 medium / 2 low |
+| `silenceMs` | `2000` | Silence pause in ms that marks the end of a complete sentence |
+| `idleTimeoutMinutes` | `10` | Minutes of inactivity before auto-exiting voice mode |
+| `modelHost` | default | ASR model download host (use `https://hf-mirror.com` on mainland networks) |
+| `autoSend` | `true` | Auto-send after a finalized transcript; when off, text only goes to the draft (hold `Ctrl` / release in hold mode still sends) |
+| `mode` | `toggle` | Interaction mode: `toggle` continuous listening + 2 s silence split; `hold` push-to-talk, release to send (short tap exits) |
+| `wakeWord` | empty (off) | Wake word (e.g. `你好小D`): speak it after entering to activate, avoiding accidental triggers; empty = off |
+
+Effect timing: `voice`/`rate` take effect **immediately** (TTS hot-swap); the rest apply on the next voice-mode entry. Defaults come from the plugin config (`base` layer) — they follow the config unless explicitly changed.
+
+### Common voices (full list: `node scripts/list-voices.mjs`)
+
+| ShortName | Description |
+| --- | --- |
+| `zh-CN-XiaoxiaoNeural` | Xiaoxiao · Female (default) |
+| `zh-CN-XiaoyiNeural` | Xiaoyi · Female |
+| `zh-CN-YunxiNeural` | Yunxi · Male |
+| `zh-CN-YunjianNeural` | Yunjian · Male |
+| `zh-CN-YunyangNeural` | Yunyang · Male |
+| `zh-CN-YunxiaNeural` | Yunxia · Male |
+| `zh-CN-liaoning-XiaobeiNeural` | Xiaobei · Northeastern Mandarin · Female |
+| `zh-CN-shaanxi-XiaoniNeural` | Xiaoni · Shaanxi Mandarin · Female |
+| `zh-HK-HiuMaanNeural` | HiuMaan · Cantonese · Female |
+| `zh-HK-WanLungNeural` | WanLung · Cantonese · Male |
+| `zh-TW-HsiaoYuNeural` | HsiaoYu · Taiwanese Mandarin · Female |
+| `zh-TW-YunJheNeural` | YunJhe · Taiwanese Mandarin · Male |
+| `en-US-AriaNeural` | Aria · English · Female |
+| `en-US-GuyNeural` | Guy · English · Male |
+
+## Configuration (bundle patch / settings.yaml)
+
+You can also edit the `voice-mode:` section of `~/.dsh/settings.yaml` directly (the GUI card and RPC write to the same document layer):
+
+```yaml
+- id: voice-mode
+  name: dsh-voice-mode
+  config:
+    enabled: true                 # false = disables voice mode entirely (toggle rejects)
+    cacheDir: ~/.cache/dsh-voice-mode/models   # overridable; platform default otherwise
+    # Defaults seeded for the settings (the settings panel overrides; the panel is authoritative):
+    voice: zh-CN-XiaoxiaoNeural
+    rate: 1.0
+    interruptLevel: 0
+    silenceMs: 2000
+    idleTimeoutMinutes: 10
+    modelHost: https://huggingface.co
+```
+
+> Note: the effective values of `voice/rate/interruptLevel/silenceMs/idleTimeoutMinutes/modelHost/autoSend`
+> come from the **settings panel**; the bundle config only seeds defaults for those keys
+> (`enabled/cacheDir` remain bundle-config-only).
+> The plugin HTTP namespace is fixed to `/voice-mode` (matching the client bundle contract; not configurable).
+
+## API
+
+| Route | Description |
+| --- | --- |
+| `GET /voice-mode/stream` | SSE: `event: audio` (`{sessionId, seq, text, audio(base64 MP3)}`), `event: mode` (global single-active ownership), `event: tool` (beep), `event: asr-progress / asr-ready / asr-error / tts-error` |
+| `POST /voice-mode/toggle` | `{sessionId, on}` enter/exit voice mode (globally single-active) |
+| `POST /voice-mode/asr` | Raw f32 LE 16k PCM payload → `{text}` (streaming zipformer2); returns `202 {loading}` until the model is ready; `?reset=1` discards the in-flight segment (used on wake-word hit) |
+| `POST /voice-mode/cancel` | `{sessionId}` invalidates the TTS queue and drops the in-flight ASR segment |
+| `POST /voice-mode/preview` | `{voice, rate?}` one-shot synthesis preview → `audio/mpeg` (400 missing voice / voice too long; 502 synthesis failure, e.g. invalid ShortName; 403 when the plugin's `enabled=false`). Does not require voice mode to be active; uses an isolated synthesis connection and does not affect the reading queue |
+| `GET /voice-mode/config` | Client bootstrap parameters (silence threshold / sensitivity / voice and rate, etc.) |
+| `GET /voice-mode` | Health check `{ok, name, enabled, active}` |
+
+## Model & cache
+
+- Recognition model: `csukuangfj/sherpa-onnx-streaming-zipformer-zh-int8-2025-06-30` (encoder ≈154 MB / decoder / joiner / tokens, ~160 MB total), running host-side via sherpa-onnx (Node WASM, Apache-2.0, natively cross-platform)
+- Cache directory defaults by platform:
+  - **Windows**: `%LOCALAPPDATA%\dsh-voice-mode\models`
+  - **macOS / Linux**: `~/.cache/dsh-voice-mode/models`
+  - both overridable via `cacheDir`
+- Downloads use `.part` resume; `huggingface.co` falls back to `hf-mirror.com` on failure (configurable via `modelHost`)
+
+## How it works
+
+![architecture](assets/architecture.svg)
+
+```
+input:  mic ──RMS VAD (2s silence split)──▶ POST /voice-mode/asr (f32 PCM, 16k, incremental)
+                                            │ zipformer2 streaming ASR (host-side WASM)
+                                            ▼
+        composer draft ──autoSend──▶ model stream ──llm/stream tap (active voice session only)
+                                            │ text-delta filter → sentence segmentation
+                                            ▼
+        browser ◀── SSE /voice-mode/stream ◀── TtsQueue (msedge-tts sentence-by-sentence)
+```
+
+- Speech and reading only happen for the session pointed to by the global single-active pointer `activeVoiceSession`; other sessions pass through `llm/stream` with zero overhead (mode isolation)
+- The `llm/stream` tap is lossless: every chunk passes through unchanged; segmentation/synthesis only observe and never block the model stream
+- zipformer2 runs host-side (sherpa-onnx Node WASM); the browser only captures audio (`getUserMedia` 16k mono) and does endpoint detection
+- The TTS queue is per-session with an epoch version: old frames are all invalidated after a barge-in, so it is truly silent
+
+## Known limitations
+
+- Barge-in relies on browser echo cancellation (`echoCancellation`); loud speaker volume may leak into the mic (no JS-level AEC)
+- `Ctrl+Shift+V` overrides the browser's "paste as plain text" shortcut (normal `Ctrl+V` paste still works)
+- The recognition model prioritizes Simplified Chinese; recognition quality is affected by ambient noise
+- Browser autoplay policy: reading requires prior user interaction on the page (clicking the mic satisfies it); if the browser blocks playback and the status bar shows no hint, make sure the page is foregrounded and not muted
+- **The wake word is a lightweight implementation** (text matching on the streaming transcript, not a dedicated KWS engine): it may lag or misfire in noisy environments; the wake word itself never enters the chat (the buffer is dropped on hit)
+- In hold mode, switching windows/tabs while holding **abandons the segment** (prevents continuous recording); come back and hold again
+- The hero (new-session empty state) has no voice entry: voice mode is a session-level feature; enter a session first and use the mic button in the input toolbar
+- The preview request timeout uses `AbortSignal.timeout` (Chrome 103+ / Firefox 100+ / Safari 16+); on older browsers clicking preview immediately shows a failure hint — an expected degradation
+
+## Troubleshooting
+
+| Symptom | Fix |
+| --- | --- |
+| Mic click does nothing, red hint in the status bar | The browser denied mic permission: allow it in the address bar and retry |
+| Status bar stuck on `正在加载模型… x%` | Check the network; the model is large (160 MB) — `npm run prefetch` first; on mainland networks set `modelHost` to `https://hf-mirror.com` |
+| Status bar shows `语音模型下载失败` | Both mirrors are unreachable: check network/proxy and re-enter voice mode (resumable) |
+| Caption appears (overlay) but no sound | Check system volume/output; if autoplay is blocked, click anywhere on the page and retry |
+| Status bar shows `朗读连接失败：正在重试…` | Edge TTS unreachable (overseas service), auto-retries; if it persists, check network/proxy |
+| Poor recognition | Get closer to the mic, reduce ambient noise; if echo remains, raise the interrupt sensitivity by one step |
+| Hold mode has no effect | Make sure hold mode is active and you're in voice mode (button shows `按住说话`); the browser window must be in the foreground |
+| Preview button reports synthesis failure | Edge TTS unreachable (overseas) or the ShortName doesn't exist: verify the name (`node scripts/list-voices.mjs` lists all) and retry later |
+
+## Development
+
+### Dependency discipline (important)
+
+Three categories, each with its own home:
+
+- **Third-party runtime deps** (`msedge-tts` / `sherpa-onnx`) and **registry-resolvable framework packages**
+  (`@deepseek-ai/schemastery`) → `dependencies`. schemastery is a public npm package and the
+  dsh host platform does not shadow it internally, so installing it into a profile causes no
+  version conflicts.
+- **Host framework packages** (`@deepseek-ai/cordis` / `@deepseek-ai/dsh-web` / `react`) →
+  `peerDependencies`. Host packages are provided by the dsh runtime; putting them in
+  dependencies makes dshmarket treat it as "shadowing host versions" and blocks marketplace
+  upgrades. Peer versions must match the current dsh runtime
+  (locally: cordis `^4.0.1`, dsh-web `^0.1.0-rc.6 || ^0.1.1-rc.0`, react `^18.2.0`),
+  and be bumped when dsh is upgraded.
+- **Type-only references** (`@deepseek-ai/dsh-settings` / `dsh-host-webserver` / `dsh-llm`) →
+  no runtime imports between instances (`import type` + esbuild stripping), no declaration needed;
+  during development the types link via pnpm `file:` to the local dsh distribution's node_modules
+  (the registry's rc.1 type snapshot lags behind the distribution; the distribution types are the
+  runtime truth).
+
+`dependencies` must contain only true third-party runtime deps — never host-shared packages; after
+changing deps, run `npm pack --dry-run` and `pnpm test` as regression.
+
+### Build & test
+
+```sh
+pnpm install && pnpm build    # esbuild: lib/index.js (host) + lib/client.js (browser)
+pnpm test                     # segmenter/wakeword unit tests + pre-release self-check (no network)
+node test/hold-e2e.js         # hold-mode acceptance (standalone browser, /asr route interception)
+systemctl restart dsh         # Linux; restart the dsh process on other platforms
+```
+
+> Note: dsh installs the plugin as a pnpm `file:` link (directory copy); after `node build.mjs` you
+> must copy `lib/client.js` to `<profile>/node_modules/dsh-voice-mode/lib/` and restart dsh before
+> the browser picks up the new bundle.
+
+### Structure
+
+```
+src/index.ts      host: single-active pointer, llm/stream tap, SSE, settings registration
+src/asr-host.ts   host: zipformer2 streaming ASR + lazy model download (.part resume)
+src/tts-queue.ts  host: per-session TTS queue + epoch barge-in
+src/segmenter.ts  host: sentence segmentation (markdown stripping + terminating punctuation)
+src/client.tsx    client: mic button + status bar + reading overlay + barge-in
+src/asr.ts        client: getUserMedia + RMS VAD + partial polling
+scripts/prefetch.mjs  model pre-download (cross-platform cache dir + resume)
+test/segmenter.test.mjs  sentence segmentation unit tests
+test/wakeword.test.mjs    wake-word matching unit tests
+test/verify-client.mjs   pre-release self-check (bundle manifest/exports/shape)
+test/hold-e2e.js          hold-mode end-to-end acceptance (standalone browser)
+scripts/list-voices.mjs   print all Edge TTS voices (source of the voice table)
+```
+
+For release and awesome-dsh-plugin submission flow, see `BEST_PRACTICES.md` and `docs/publish/` in the repo root.
+
+## License
+
+MIT

package/README.md

@@ -0,0 +1,271 @@
+# dsh-voice-mode
+
+[![npm version](https://img.shields.io/npm/v/dsh-voice-mode?style=flat-square)](https://www.npmjs.com/package/dsh-voice-mode)
+[![License](https://img.shields.io/npm/l/dsh-voice-mode?style=flat-square)](LICENSE)
+[![dsh-plugin](https://img.shields.io/badge/dsh-plugin-voice-%232ea043?style=flat-square)](https://github.com/topics/dsh-plugin)
+
+> Full-duplex voice conversation mode for DeepSeek Harness (dsh): speak, get a
+> spoken answer. Streamed zipformer2 ASR → editable draft → auto send → the
+> final reply is read out sentence-by-sentence via Edge TTS, and your voice
+> interrupts playback and the running turn. No API key.
+
+DeepSeek Harness 语音双工对话模式：会话内一键进入 → 边说边出字的流式识别 → 停顿自动发送 → 最终答复按句流式朗读 + 实时字幕，开口即可打断（barge-in）。无需 API Key，模型在本地宿主端推理。
+
+[English](./README.en.md)
+
+![demo](assets/demo.gif)
+
+## 特性
+
+- **语音模式**：输入框工具排麦克风按钮或全局快捷键 `Ctrl+Shift+V` 进入/退出；全局单活（同一时刻仅一个会话处于语音模式，切换会话自动让出）
+- **两种交互模式（设置可切换）**：
+  - `toggle`（默认）持续聆听：RMS VAD 分段 → zipformer2 流式识别（边说边出字，实时字幕预览）→ 静音 2 秒自动断句进草稿并自动发送；按住 `Ctrl` 强制立即发送
+  - `hold` 按住说话：短按进入/退出，**按住麦克风按钮说话、松手即发**（滑出取消、`Esc`/失焦放弃本段）；`Ctrl` 按住即录、松开即发
+- **唤醒词（可选，默认关）**：设置 `wakeWord` 后，进入语音模式处于待机态，说出唤醒词才开始识别（如「你好小D」），避免误触
+- **输出链路**：只朗读最终答复的 `text-delta`（reasoning/工具调用不读），按句流式 Edge TTS 朗读 + 右下角实时字幕浮层；工具调用触发提示音；全文照常写入聊天记录；可选的口语化提示词（设置 `spokenFormat`，默认关）让语音会话回复为自然短句、不带 Markdown 排版符号，朗读侧再做一轮标记剥离，听感更顺
+- **开口打断（barge-in）**：三档灵敏度的发声前沿检测 → 本地静音 + host 合成队列作废（epoch）+ 正在运行的回合取消（保留半截并自然续入你的新消息）
+- **模型懒加载与进度**：首次使用自动下载 zipformer2 中文流式模型（约 160MB，`.part` 断点续传），状态条实时显示下载进度；可用 `npm run prefetch` 预下载
+- **容错**：麦克风被拒红点提示、模型下载失败可见提示、TTS 连接失败状态条提示（自动重试）、提交失败文字留在草稿、SSE 断线自动重连
+- **设置**：设置 → 插件配置 → voice-mode，可调音色 / 语速 / 打断灵敏度 / 静音停顿 / 空闲超时 / 模型镜像 / 自动发送 / 交互模式 / 唤醒词 / 口语化提示词；**音色可试听**（「试听」按钮按当前音色 + 当前语速即时合成预览，无需进入语音模式；自定义 ShortName 同样可试听）
+- **空闲退出**：10 分钟无活动自动退出并释放麦克风
+
+## 操作手势
+
+| 手势 | 行为 |
+| --- | --- |
+| 点按麦克风按钮 / `Ctrl+Shift+V` | 进入 / 退出语音模式 |
+| 直接说话，停顿 2 秒（toggle） | 自动断句并发送 |
+| 按住 `Ctrl`（toggle，≥250ms 语音） | 强制立即发送当前段 |
+| **按住麦克风按钮（hold）** | 按住说话，松手发送；向上滑出 / `Esc` / 失去焦点放弃本段；<250ms 短按退出模式 |
+| 按住 `Ctrl`（hold，≥600ms） | 键盘按住说话，松开即发 |
+| 先喊一声唤醒词（已配置） | 从待机激活为聆听（其后才识别与发送） |
+| AI 朗读时开口说话 | 打断朗读并取消当前回合 |
+| 在输入框打字 | 自动退出语音模式（草稿保留） |
+
+## 安装
+
+**要求**：dsh web（Node ≥ 18），现代浏览器（Chrome / Edge / Firefox，需支持 `getUserMedia` 与 Web Audio）。
+
+```sh
+# 方式一：从 npm 安装（推荐）
+dsh plugin --profile web add dsh-voice-mode
+# 等价形式（本机未装 dsh CLI 时由 npx 临时拉起）：
+npx -y @deepseek-ai/dsh plugin --profile web add dsh-voice-mode
+
+# 方式二：本地 tarball
+dsh plugin --profile web add ./dsh-voice-mode-0.1.0.tgz
+
+# 方式三：从源码安装
+git clone https://github.com/qishuilalala/dsh-voice-mode.git
+cd dsh-voice-mode/plugin/dsh-voice-mode && pnpm install && pnpm build
+dsh plugin --profile web add .
+```
+
+**bundle 插件需重启 dsh 生效**（不同平台的重启方式）：
+
+- **Linux（systemd）**：`systemctl restart dsh`
+- **Windows / macOS / 手动托管**：重启你的 dsh 进程（结束进程后重新 `dsh web`，或在其服务管理器中重启）
+
+**可选**：预下载 ASR 模型，减少首次进入语音模式的下载等待：
+
+```sh
+npm run prefetch          # 插件目录内执行；默认写到平台缓存目录
+# 或指定缓存位置：node scripts/prefetch.mjs --cache-dir /where/ever/models
+```
+
+## 使用
+
+1. 点击输入框工具排的麦克风按钮（或按 `Ctrl+Shift+V`）进入语音模式，输入框上方出现状态条
+2. 说话方式二选一：直接说、停顿 2 秒自动发送（toggle）；或按住麦克风按钮、松手发送（hold）
+3. AI 回复逐句朗读，右下角浮层显示字幕；点「跳过」或直接开口打断
+4. 点状态条「退出」（或再按 `Ctrl+Shift+V`）退出语音模式
+
+首次进入会下载识别模型，状态条显示 `正在加载模型… <文件> <百分比>%`。
+
+配置了唤醒词时，进入后会先处于待机态（状态条提示「说『唤醒词』开始」），说完唤醒词即激活。
+
+## 设置（设置 → Plugins → 插件配置 → 语音模式）
+
+| 键 | 默认 | 说明 |
+| --- | --- | --- |
+| `voice` | `zh-CN-XiaoxiaoNeural` | Edge TTS 音色（见下方常用音色表），**即时生效**；行内「试听」按钮按当前音色 + 当前语速合成预览（下拉常用音色与「自定义」ShortName 均可试听，失败有可见提示） |
+| `rate` | `1.0` | 朗读语速倍率（0.5 慢速 ～ 2.0 快速），**即时生效** |
+| `interruptLevel` | `0` | 发声打断灵敏度：0 高门槛 / 1 中 / 2 低 |
+| `silenceMs` | `2000` | 说完整一句的静音停顿毫秒数 |
+| `idleTimeoutMinutes` | `10` | 无活动自动退出语音模式的分钟数 |
+| `modelHost` | 默认源 | ASR 模型下载源（国内网络填 `https://hf-mirror.com`） |
+| `autoSend` | `true` | 识别定稿后自动发送；关闭则只进草稿（按住 `Ctrl` / hold 松手仍会发送） |
+| `mode` | `toggle` | 交互模式：`toggle` 持续聆听 + 2s 静音断句；`hold` 按住说话、松手发送（短按退出） |
+| `wakeWord` | 空（关） | 唤醒词（如「你好小D」）：进入后先说唤醒词激活，避免误触；空 = 关闭 |
+| `spokenFormat` | `false` | 语音会话注入口语化提示词：开启后**仅当前语音会话**的回复被注入「口语化短句、不用 Markdown 排版符号」提示词（朗读更顺），**即时生效**（关闭后对后续回复立即失效；默认关） |
+
+生效范围：`voice`/`rate`/`spokenFormat` 修改后**立即生效**（TTS 热切换 / 提示词组装时实时读取）；其余设置下次进入语音模式时生效。设置项默认值由插件配置（`base` 层）提供——未显式修改时跟随配置。
+
+### 常用音色（完整清单见 `node scripts/list-voices.mjs`）
+
+| ShortName | 说明 |
+| --- | --- |
+| `zh-CN-XiaoxiaoNeural` | 晓晓 · 女声（默认） |
+| `zh-CN-XiaoyiNeural` | 晓伊 · 女声 |
+| `zh-CN-YunxiNeural` | 云希 · 男声 |
+| `zh-CN-YunjianNeural` | 云健 · 男声 |
+| `zh-CN-YunyangNeural` | 云扬 · 男声 |
+| `zh-CN-YunxiaNeural` | 云夏 · 男声 |
+| `zh-CN-liaoning-XiaobeiNeural` | 小北 · 东北话 · 女声 |
+| `zh-CN-shaanxi-XiaoniNeural` | 小妮 · 陕西话 · 女声 |
+| `zh-HK-HiuMaanNeural` | 晓曼 · 粤语 · 女声 |
+| `zh-HK-WanLungNeural` | 云龙 · 粤语 · 男声 |
+| `zh-TW-HsiaoYuNeural` | 小雨 · 台湾腔 · 女声 |
+| `zh-TW-YunJheNeural` | 云哲 · 台湾腔 · 男声 |
+| `en-US-AriaNeural` | Aria · 英语 · 女声 |
+| `en-US-GuyNeural` | Guy · 英语 · 男声 |
+
+## 配置（bundle patch / settings.yaml）
+
+也可直接编辑 `~/.dsh/settings.yaml` 的 `voice-mode:` 段（GUI 卡片与 RPC 写入同一文档层）：
+
+```yaml
+- id: voice-mode
+  name: dsh-voice-mode
+  config:
+    enabled: true                 # false = 关闭整个语音模式（无法进入，toggle 拒绝）
+    cacheDir: ~/.cache/dsh-voice-mode/models   # 可覆盖；默认按平台
+    # 以下为设置项的默认播种值（设置面板可覆盖；最终生效值以设置面板为准）：
+    voice: zh-CN-XiaoxiaoNeural
+    rate: 1.0
+    interruptLevel: 0
+    silenceMs: 2000
+    idleTimeoutMinutes: 10
+    modelHost: https://huggingface.co
+```
+
+> 说明：`voice/rate/interruptLevel/silenceMs/idleTimeoutMinutes/modelHost/autoSend`
+> 的最终生效值以**设置面板**为准；bundle 配置仅为这些键提供默认播种值
+> （`enabled/cacheDir` 仍只由 bundle 配置控制）。
+> 插件 HTTP 命名空间固定为 `/voice-mode`（与客户端 bundle 契约一致，不可配置）。
+
+## API
+
+| 路由 | 说明 |
+| --- | --- |
+| `GET /voice-mode/stream` | SSE：`event: audio`（`{sessionId, seq, text, audio(base64 MP3)}`）、`event: mode`（全局单活归属）、`event: tool`（提示音）、`event: asr-progress / asr-ready / asr-error / tts-error` |
+| `POST /voice-mode/toggle` | `{sessionId, on}` 进入/退出语音模式（全局单活） |
+| `POST /voice-mode/asr` | 原始 f32 LE 16k PCM 载荷 → `{text}`（流式 zipformer2）；模型未就绪返回 `202 {loading}`；`?reset=1` 丢弃进行中识别段（唤醒词命中清场用） |
+| `POST /voice-mode/cancel` | `{sessionId}` 作废 TTS 队列并丢弃在途 ASR 段 |
+| `POST /voice-mode/preview` | `{voice, rate?}` 一次性合成试听 → `audio/mpeg`（400 缺 voice / voice 过长；502 合成失败，含非法 ShortName；插件 `enabled=false` 时 403）。不要求语音模式激活，使用独立合成连接，不影响朗读队列 |
+| `GET /voice-mode/config` | 客户端引导参数（静音阈值 / 灵敏度 / 音色语速等） |
+| `GET /voice-mode` | 健康检查 `{ok, name, enabled, active}` |
+
+## 模型与缓存
+
+- 识别模型：`csukuangfj/sherpa-onnx-streaming-zipformer-zh-int8-2025-06-30`（encoder ≈154MB / decoder / joiner / tokens，共约 160MB），宿主端 sherpa-onnx（Node WASM，Apache-2.0，天然跨平台）
+- 缓存目录默认值按平台：
+  - **Windows**：`%LOCALAPPDATA%\dsh-voice-mode\models`
+  - **macOS / Linux**：`~/.cache/dsh-voice-mode/models`
+  - 均可通过 `cacheDir` 配置覆盖
+- 下载走 `.part` 断点续传，`huggingface.co` 失败自动回退 `hf-mirror.com`（可配置 `modelHost`）
+
+## 工作原理
+
+![architecture](assets/architecture.svg)
+
+```
+input:  mic ──RMS VAD（2s 静音切句）──▶ POST /voice-mode/asr（f32 PCM，16k，增量解码）
+                                           │ zipformer2 流式识别（宿主端 WASM）
+                                           ▼
+        composer draft ──autoSend──▶ model stream ──llm/stream tap（仅活跃语音会话）
+                                           │ text-delta 过滤 → 句子切分
+                                           ▼
+        browser ◀── SSE /voice-mode/stream ◀── TtsQueue（msedge-tts 逐句合成）
+```
+
+- 语音与朗读只发生在全局单活指针 `activeVoiceSession` 指定的会话；普通会话 `llm/stream` 直达、零开销（模式隔离）
+- `llm/stream` tap 无损：每个 chunk 原样透传，切句/合成只旁观，不阻塞模型流
+- zipformer2 在宿主端推理（sherpa-onnx Node WASM），浏览器只负责采集（`getUserMedia` 16k 单声道）与端点检测
+- TTS 队列按会话隔离 + epoch 版本号：打断后旧帧全部作废，真正静音
+
+## 已知限制
+
+- 发声打断依赖浏览器回声消除（`echoCancellation`）；扬声器音量过大时可能漏声到麦克风（JS 层无法做 AEC）
+- `Ctrl+Shift+V` 会覆盖浏览器「粘贴纯文本」快捷键（普通粘贴仍可用 `Ctrl+V`）
+- 识别模型为简体中文优先；识别质量受环境噪声影响
+- 浏览器自动播放策略：朗读需要页面已有用户交互（点击麦克风即满足）；若浏览器拦截播放且状态条无提示，请确认网页处于前台且非静音状态
+- **唤醒词为轻量实现**（基于流式识别文本匹配，非专用 KWS 引擎）：嘈杂环境可能延迟或误激活；唤醒词本身不会进入聊天（命中即丢弃缓冲）
+- hold 模式按住时如果切换窗口/标签页会**放弃本段**（防持续收音），回来需重新按住
+- hero（新会话空态）没有语音入口：语音模式是会话级功能，请先进入会话使用输入框麦克风按钮
+- 「试听」的请求超时兜底使用 `AbortSignal.timeout`（Chrome 103+ / Firefox 100+ / Safari 16+）；更老的浏览器点击试听会立即显示失败提示，属预期降级
+- `spokenFormat` 提示词经官方 `system-prompt/assemble` 瀑布注入；若当前会话使用**完整提示词**配置（persona `complete: true` 的 agent preset），其提示词会整体替换系统提示词（官方 complete 契约），此时口语化提示词不注入
+
+## 故障排查
+
+| 现象 | 处理 |
+| --- | --- |
+| 点麦克风无反应，状态条提示红字 | 浏览器拒绝了麦克风权限：地址栏允许麦克风后重试 |
+| 状态条显示「正在加载模型… x%」卡住 | 检查网络；模型大（160MB）可先 `npm run prefetch`；国内网络把 `modelHost` 配成 `https://hf-mirror.com` |
+| 状态条显示「语音模型下载失败」 | 两镜像均不可达：检查网络/代理后重新进入语音模式（断点续传） |
+| 有字幕（浮层）但听不到声音 | 检查系统音量/输出设备；浏览器自动播放被拦时点击页面任意处后再试 |
+| 状态条显示「朗读连接失败：正在重试…」 | Edge TTS 服务不可达（境外服务），稍后自动重试；持续失败请检查网络/代理 |
+| 识别不准 | 靠近麦克风、降低环境噪声；还有回声时把「打断灵敏度」调高一档 |
+| hold 模式按住没反应 | 确认切换到了 hold 模式并处于语音模式中（按钮显示「按住说话」）；浏览器窗口需在前台 |
+| 「试听」按钮提示合成失败 | Edge TTS 服务不可达（境外服务）或音色名（ShortName）不存在：核对音色名（`node scripts/list-voices.mjs` 可查全部），稍后重试 |
+
+## 开发
+
+### 依赖纪律（重要）
+
+分三类，各有归属：
+
+- **第三方运行依赖**（`msedge-tts` / `sherpa-onnx`）与 **registry 可解析的框架包**
+  （`@deepseek-ai/schemastery`）→ `dependencies`。schemastery 是公开 npm 包且
+  dsh 宿主平台内部不遮蔽它，装进 profile 不会引发版本冲突。
+- **宿主框架包**（`@deepseek-ai/cordis` / `@deepseek-ai/dsh-web` / `react`）→
+  `peerDependencies`。宿主包由 dsh 运行时提供；若进 dependencies 会被 dshmarket
+  判定「遮蔽宿主版本」而拦截插件市场升级。peer 版本须与当前 dsh 运行时一致
+  （本机：cordis `^4.0.1`、dsh-web `^0.1.0-rc.6 || ^0.1.1-rc.0`、react `^18.2.0`），
+  升级 dsh 时同步更新。
+- **仅类型引用**（`@deepseek-ai/dsh-settings` / `dsh-host-webserver` / `dsh-llm` /
+  `dsh-system-prompt`）→ 实例之间没有任何运行时 import（`import type` + esbuild 剔除），
+  无需声明；开发期类型直接**实体化复制**自本机 dsh 发行版 node_modules（勿用指向宿主
+  node_modules 的绝对路径 **symlink**：其内部 `cordis` 解析到宿主实例，与插件 `.pnpm`
+  的 cordis 模块身份分裂，`declare module '@deepseek-ai/cordis'` 类型增强全部失效；
+  registry 的 rc.1 类型快照落后于发行版，发行版类型才是运行时真值）。
+
+`dependencies` 只保留真正的第三方运行依赖，禁止把宿主共享包写进去；改依赖后
+跑 `npm pack --dry-run` 与 `pnpm test` 回归。
+
+### 构建与测试
+
+```sh
+pnpm install && pnpm build    # esbuild：lib/index.js（host）+ lib/client.js（browser）
+pnpm test                     # segmenter/wakeword 单测 + 发布前自检（均无需网络）
+node test/hold-e2e.js         # hold 模式验收（独立浏览器，/asr 路由拦截）
+bash test/spoken-prompt-rpc.sh  # 口语化提示词验证（RPC 直发，无需浏览器；需在线 TTS）
+systemctl restart dsh         # Linux；其他平台重启 dsh 进程
+```
+
+> 注意：dsh 安装的是 pnpm `file:` 链接（目录拷贝），改完 `node build.mjs` 后需把
+> `lib/client.js` 同步到 `<profile>/node_modules/dsh-voice-mode/lib/` 再重启 dsh，
+> 浏览器才能拿到新 bundle。
+
+### 结构
+
+```
+src/index.ts      host：单活指针、llm/stream tap、SSE、settings 注册
+src/asr-host.ts   host：zipformer2 流式识别 + 模型懒下载（.part 断点续传）
+src/tts-queue.ts  host：逐会话 TTS 队列 + epoch 打断机制
+src/segmenter.ts  host：句子切分（markdown 剥离 + 终止标点）
+src/client.tsx    client：麦克风按钮 + 状态条 + 朗读浮层 + 打断
+src/asr.ts        client：getUserMedia + RMS VAD + partial 轮询
+scripts/prefetch.mjs  模型预下载（跨平台缓存目录 + 断点续传）
+test/segmenter.test.mjs 句子切分单元测试
+test/wakeword.test.mjs    唤醒词匹配单元测试
+test/verify-client.mjs   发布前自检（bundle 清单/导出/形状）
+test/hold-e2e.js          hold 模式端到端验收（独立浏览器）
+scripts/list-voices.mjs   打印 Edge TTS 全部音色（音色表来源）
+```
+
+发布与精选列表提交流程见仓库根 `BEST_PRACTICES.md` 与 `docs/publish/`。
+
+## 许可
+
+MIT