dsh-voice-mode 0.7.9 → 0.7.10

package/README.en.md

@@ -14,7 +14,7 @@ interrupts playback and the running turn. No API key.
 
 ![dsh-voice-mode full-duplex voice conversation](https://raw.githubusercontent.com/qishuilalala/dsh-voice-mode/HEAD/assets/hero-banner.png)
 
-> **Version note (v0.7.9, 2026-09-18)**: **Wake-word pipeline overhaul** (Issue #10 + real-machine retest) — a harness driving the real engine pinpointed and fixed five flow defects: ① TTS echo polluting the standby segment while the agent reads (could not wake); ② only above-threshold frames uploaded, so trailing characters never flushed (wake word truncated); ③ discarding the whole segment on a wake hit (saying "wake word + command" in one breath sent only the tail); ④ the command hanging when the hit arrived after you stopped speaking; ⑤ pausing after the wake word closed the command window (command lost). **The wake word may now be said together with your command — it is stripped and never sent**, standby live-shows what it heard, and "wake word … pause … command" works. Otherwise as v0.7.7: Edge cloud TTS by default, local TTS (VITS / Kokoro) optional; silence split defaults to 1500 ms.
+> **Version note (v0.7.10, 2026-09-18)**: **Silent-audio causes in the output pipeline fixed** — ① a sentence whose cloud-TTS synthesis fails after 3 retries is no longer dropped silently (the status bar now says one sentence failed and was skipped; previously this showed up as "the reply is occasionally not read" with no way to tell why); ② audio playback no longer goes silent after the browser suspends the AudioContext (background tab / long idle) — the context is resumed on every enqueue and on any click/keypress; ③ dropped incomplete sentences (SSE frame loss) now leave a diagnostic trace. Otherwise as v0.7.9: **Wake-word pipeline overhaul** (Issue #10 + real-machine retest) — a harness driving the real engine pinpointed and fixed five flow defects: ① TTS echo polluting the standby segment while the agent reads (could not wake); ② only above-threshold frames uploaded, so trailing characters never flushed (wake word truncated); ③ discarding the whole segment on a wake hit (saying "wake word + command" in one breath sent only the tail); ④ the command hanging when the hit arrived after you stopped speaking; ⑤ pausing after the wake word closed the command window (command lost). **The wake word may now be said together with your command — it is stripped and never sent**, standby live-shows what it heard, and "wake word … pause … command" works. Otherwise as v0.7.7: Edge cloud TTS by default, local TTS (VITS / Kokoro) optional; silence split defaults to 1500 ms.
 
 ---
 

package/README.md

@@ -13,7 +13,7 @@ DeepSeek Harness 语音双工对话模式：会话内一键进入 → 边说边
 
 ![语音模式：实时字幕与状态条](https://raw.githubusercontent.com/qishuilalala/dsh-voice-mode/HEAD/assets/screenshot-voice.png)
 
-> **版本说明（v0.7.9，2026-09-18）**：**唤醒词链路全面修复**（Issue #10 + 真机复测）——台架驱动真引擎逐项定位并修掉五处流程缺陷：① 朗读期 TTS 回声污染待机段（喊不醒）；② 只上传超门限帧导致尾字不 flush（唤醒词只剩半截）；③ 唤醒命中丢整段（连说「唤醒词+命令」只发出去尾部几个字）；④ 命中晚于停口时命令悬挂不定稿；⑤ 只喊唤醒词后停顿会关掉命令窗口（命令丢失）。**现在唤醒词可与命令连说、词头自动剥离不进消息**，待机态实时显示「它听到了什么」，支持「唤醒词…停顿…命令」。其余同 v0.7.7：朗读默认 Edge 云端，本地 TTS（VITS / Kokoro）可选；静音断句默认 1500 毫秒。
+> **版本说明（v0.7.10，2026-09-18）**：**输出链路静默丢音根治**——① 云端 TTS 单句重试耗尽后不再静默跳句（现在状态条提示「有一句朗读失败，已跳过」，此前表现为「AI 回复偶尔不朗读」且无从判断）；② 浏览器挂起 AudioContext 后的「无声播放」（UI 显示朗读中、字幕照走）现在每次入队 + 任意点击/按键自动恢复；③ SSE 丢帧导致的坏句丢弃留诊断痕迹。其余同 v0.7.9：**唤醒词链路全面修复**（Issue #10 + 真机复测）——台架驱动真引擎逐项定位并修掉五处流程缺陷：① 朗读期 TTS 回声污染待机段（喊不醒）；② 只上传超门限帧导致尾字不 flush（唤醒词只剩半截）；③ 唤醒命中丢整段（连说「唤醒词+命令」只发出去尾部几个字）；④ 命中晚于停口时命令悬挂不定稿；⑤ 只喊唤醒词后停顿会关掉命令窗口（命令丢失）。**现在唤醒词可与命令连说、词头自动剥离不进消息**，待机态实时显示「它听到了什么」，支持「唤醒词…停顿…命令」。其余同 v0.7.7：朗读默认 Edge 云端，本地 TTS（VITS / Kokoro）可选；静音断句默认 1500 毫秒。
 
 ---
 
@@ -221,6 +221,7 @@ flowchart LR
 | 点麦克风无反应，状态条红字 | 浏览器拒绝麦克风：地址栏（iOS 为 设置 → Safari → 麦克风）开启后重试 |
 | 状态条「正在加载模型… x%」卡住 | 检查网络；模型较大可先 `npm run prefetch`；国内网络 `modelHost` 配 `https://hf-mirror.com` |
 | 朗读无声音/无字幕 | 本地引擎首次合成需加载模型；若持续失败查看状态条提示（自动退避重试）；确认页面前台且未静音 |
+| AI 回复**偶尔不朗读**（某句没声/整条没声） | 已定位三类原因并修（v0.7.10）：① 云端 TTS 偶发超时——单句自动重试 3 次，仍失败会**跳过该句并在状态条提示**「有一句朗读失败，已跳过」（此前是静默丢句）；② 浏览器挂起 AudioContext（切到后台标签页/长时间静音后回来）——播放被调度但无声、UI 却显示朗读中；现在每次入队与**任意点击/按键**都会自动恢复；③ SSE 断线丢帧导致整句不完整——按设计丢弃该句（避免播坏音频），可开 `localStorage['dsh-voice-mode.telemetry']='1'` 看 `tts-drop-sentence` 诊断 |
 | 语音模式进不去 | 检查插件 `enabled`；多标签页时确认当前会话为活动会话 |
 | 识别到但不是我要说的 | 环境噪声或唤醒词误判：降低音量、提高 `interruptLevel`（高门槛）或启用 `wakeWord` |
 | 唤醒词唤不醒 | 先看待机态状态条的实时转写（它听到了什么）：同音字 1 字内已自动容错；停顿约 1.5 秒（= 静音断句时长）再说，清掉待机段残留语音、从段首重新匹配；换一个转写稳定的词（建议 3-4 字） |

package/lib/client.js

@@ -1373,6 +1373,7 @@ function estimateBulkDelay(mic, ref, opts = {}) {
 var zh = {
   stateVoiceMode: "\u8BED\u97F3\u6A21\u5F0F",
   ttsNoticeFail: "\u6717\u8BFB\u8FDE\u63A5\u5931\u8D25\uFF1A\u6B63\u5728\u91CD\u8BD5\u2026",
+  ttsSkipNotice: "\u6709\u4E00\u53E5\u6717\u8BFB\u5931\u8D25\uFF0C\u5DF2\u8DF3\u8FC7\uFF08\u4E91\u7AEF\u6717\u8BFB\u7F51\u7EDC\u6296\u52A8\uFF0C\u53EF\u91CD\u53D1\u8FD9\u6761\u6D88\u606F\uFF09",
   enterFail: "\u8FDB\u5165\u8BED\u97F3\u6A21\u5F0F\u5931\u8D25",
   disabled: "\u8BED\u97F3\u6A21\u5F0F\u5DF2\u7981\u7528\uFF08\u63D2\u4EF6 enabled=false\uFF09",
   sendFailKept: "\u53D1\u9001\u5931\u8D25\uFF0C\u5DF2\u4FDD\u7559\u5728\u8349\u7A3F",
@@ -1552,6 +1553,7 @@ var zh = {
 var en = {
   stateVoiceMode: "Voice Mode",
   ttsNoticeFail: "Read-aloud connection lost: retrying\u2026",
+  ttsSkipNotice: "One sentence failed to read and was skipped (cloud TTS network hiccup \u2014 resend the message to retry)",
   enterFail: "Failed to enter voice mode",
   disabled: "Voice mode disabled (plugin enabled=false)",
   sendFailKept: "Send failed; text kept in draft",
@@ -2823,7 +2825,7 @@ var TELEMETRY_VIEW = [
   { stage: "first-tts-chunk", key: "telFirstChunk" },
   { stage: "first-audio-played", key: "telFirstPlayed" }
 ];
-var BUILD_TAG = "3a98e88";
+var BUILD_TAG = "466b0f3";
 var TELEMETRY_FLAG = "dsh-voice-mode.telemetry";
 var telemetryEnabled = typeof localStorage !== "undefined" && localStorage.getItem(TELEMETRY_FLAG) === "1";
 console.log("[dsh-voice] build=" + BUILD_TAG);
@@ -2907,6 +2909,11 @@ function setLastVoiceSession(id) {
 }
 function apply(ctx) {
   const bus = createVoiceBus(void 0, ctx);
+  if (typeof document !== "undefined") {
+    const resumeAudio = () => bus.warmAudio();
+    document.addEventListener("pointerdown", resumeAudio, { passive: true });
+    document.addEventListener("keydown", resumeAudio, { passive: true });
+  }
   ctx.slots.inject(
     "conversation.input.right",
     () => ctx.slots.register(
@@ -3080,6 +3087,7 @@ function createAudioEngine(setUi, onPlayed, onPlaybackRef, onAllPlayed) {
   };
   return {
     push(frame) {
+      warm();
       if (fallback || !ctx) {
         pending.push(frame);
         if (fallbackAudio.paused) playFallback();
@@ -3393,6 +3401,16 @@ function createVoiceBus(basePath = BASE_PATH2, ctx) {
       } catch {
       }
     });
+    source.addEventListener("tts-skip", (e) => {
+      try {
+        const p = JSON.parse(e.data);
+        if (p.sessionId !== activeSessionId) return;
+        ui.ttsNotice = t("ttsSkipNotice");
+        notify();
+        debugLog("tts-skip", { text: (p.text ?? "").slice(0, 40) });
+      } catch {
+      }
+    });
     source.addEventListener("tts-error", (e) => {
       try {
         const p = JSON.parse(e.data);
@@ -3448,6 +3466,7 @@ function createVoiceBus(basePath = BASE_PATH2, ctx) {
     }
     if (frame.final) {
       if (frame.chunkId !== curChunkCount) {
+        debugLog("tts-drop-sentence", { got: frame.chunkId, have: curChunkCount, sentenceId: frame.sentenceId });
         curSentenceId = null;
         curChunks = [];
         curBytes = 0;

package/lib/index.js

@@ -1131,9 +1131,12 @@ var TtsQueue = class {
   engine;
   /** TTS 全体不可达通知（每会话去重，成功后复位）。 */
   onError;
+  /** 单句合成重试耗尽被跳过通知（每句一次；此前是静默丢句——真机「回复偶尔不朗读」根因之一）。 */
+  onSkip;
   constructor(options) {
     this.engine = options.engine;
     this.onError = options.onError;
+    this.onSkip = options.onSkip;
   }
   /** 当前引擎音频 MIME（/preview 的 Content-Type 也用它）。 */
   get mime() {
@@ -1233,7 +1236,10 @@ var TtsQueue = class {
           }
         }
         if (item.epoch !== q.epoch) continue;
-        if (buf === null) continue;
+        if (buf === null) {
+          this.onSkip?.(sessionId, item.text);
+          continue;
+        }
         q.errorNotified = false;
         q.backoff = 0;
         const sentenceId = q.seq++;
@@ -2104,7 +2110,9 @@ function apply(ctx, config) {
   let activeKokoroModel = vset.kokoroModel;
   const queue = new TtsQueue({
     engine: makeEngine(engineKind),
-    onError: (sessionId) => broadcast("tts-error", { sessionId })
+    onError: (sessionId) => broadcast("tts-error", { sessionId }),
+    // 单句重试耗尽被跳过：显式下行（客户端提示 + 诊断），不再静默丢句。
+    onSkip: (sessionId, text) => broadcast("tts-skip", { sessionId, text: text.slice(0, 80) })
   });
   queue.updateVoice(vset.voice, vset.rate);
   const unsubscribe = queue.subscribe((frame) => broadcast("audio", frame));

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "dsh-voice-mode",
   "description": "Full-duplex voice plugin for DeepSeek Harness: local zipformer2 streaming ASR (no API key) → editable draft; Edge TTS or local VITS / Kokoro read-aloud with live captions; true barge-in; hardened HTTP surface + model SHA256 pinning; compatible with all dsh releases since 0.1.1-rc.2 (verified 0.1.1 → 0.1.5-rc.2). · DSH 语音双工插件：流式识别入草稿、按句朗读+实时字幕、开口即打断；免 API Key、安全加固、全版本兼容（0.1.1-rc.2 起，含 0.1.5-rc.2 端到端验证）。",
-  "version": "0.7.9",
+  "version": "0.7.10",
   "type": "module",
   "main": "lib/index.js",
   "repository": {
@@ -42,7 +42,7 @@
   "scripts": {
     "build": "node build.mjs",
     "prepack": "node build.mjs",
-    "test": "node test/segmenter.test.mjs && node test/aec.test.mjs && node test/download.test.mjs && node test/endpoint.test.mjs && node test/endpoint-short.test.mjs && node test/resample.test.mjs && node test/sense-worker.test.mjs && node test/detect-route.test.mjs && node test/verify-client.mjs && node test/asr-host-rebuild.test.mjs && node test/barge-in-manual.test.mjs && node test/sense-lang.test.mjs && node test/caption-a11y.test.mjs && node test/emotion.test.mjs && node test/emotion-tts-local.test.mjs && node test/emotion-integration.test.mjs && node test/backchannel.test.mjs && node test/strings-coverage.test.mjs && node test/yield-ms-wiring.test.mjs && node test/matchBackchannel.test.mjs && node test/settings-load.test.mjs && node test/preview-error.test.mjs && node test/hold-clear.test.mjs && node test/barge-in-detect.test.mjs && node test/wakeword.test.mjs && node test/wake-standby.test.mjs && node test/wake-flow.test.mjs",
+    "test": "node test/segmenter.test.mjs && node test/aec.test.mjs && node test/download.test.mjs && node test/endpoint.test.mjs && node test/endpoint-short.test.mjs && node test/resample.test.mjs && node test/sense-worker.test.mjs && node test/detect-route.test.mjs && node test/verify-client.mjs && node test/asr-host-rebuild.test.mjs && node test/barge-in-manual.test.mjs && node test/sense-lang.test.mjs && node test/caption-a11y.test.mjs && node test/emotion.test.mjs && node test/emotion-tts-local.test.mjs && node test/emotion-integration.test.mjs && node test/backchannel.test.mjs && node test/strings-coverage.test.mjs && node test/yield-ms-wiring.test.mjs && node test/matchBackchannel.test.mjs && node test/settings-load.test.mjs && node test/preview-error.test.mjs && node test/hold-clear.test.mjs && node test/barge-in-detect.test.mjs && node test/wakeword.test.mjs && node test/wake-standby.test.mjs && node test/wake-flow.test.mjs && node test/tts-playback.test.mjs",
     "verify:client": "node test/verify-client.mjs",
     "prefetch": "node scripts/prefetch.mjs",
     "list-voices": "node scripts/list-voices.mjs",