@flowingspring/dsh-voco 0.2.0 → 0.2.2

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 QUAN
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,31 +1,56 @@
 # `@flowingspring/dsh-voco`
-
-English | [中文](README.zh.md)
-
-Browser PCM uses lightweight local silence detection and is uploaded as WAV to SiliconFlow after 1.5 seconds of silence.
-
-The `dsh-voco` patch-layer bundle attaches provider transport directly to the current source Session and restores recent conversation from durable `voice/utterance-end` events. A lightweight frontend model answers ordinary conversation directly and delegates only tool-requiring work. Delegations from one source Session continuously reuse one ordinary task Agent Session. `voice/agent-binding-state` persists the Voice conversation, background Agent, workspace, last task, last-used time, and status; restart recovery prefers that complete record while remaining compatible with `voice/task-session-bound`. Full results and progress remain in the task UI, while an independent model rewrites the final result and user-actionable questions or warnings against the original request. Progress updates stay in the folded task trace and do not create extra Voice-window messages or TTS requests. The background Agent cannot directly specify spoken text. Root-owned audio continues across navigation, and the browser history index exposes saved Voice Sessions in the sidebar.
-
-## Model Experience
-
-### Voice profile composition
-
-#### What the model sees
-
-Voice-initiated work reaches the source Session's fixed background task Agent only as an accepted `realtime_delegation` envelope and exact-id updates. That task Agent alone receives the scoped `send_voice_message` backend tool for structured `progress | result | warning | error | question` events; the bridge creates or resumes the exact target directly, so no project-listing tool is added. The local provider owns speech input and output while the bridge exposes only task orchestration tools.
-
-#### Token effect
-
-The lightweight frontend route consumes one model call per new utterance. Delegated work additionally consumes task-model tokens for execution, backend reporting, and auxiliary event rewrites; local VAD, ASR, and TTS add no per-minute voice API charge.
-
-#### KV Cache effect
-
-Only accepted commands extend the continuously reused task Agent history. Durable binding state restores that exact Agent Session after restart, while recent Voice Session utterances restore the local conversation.
-
-## Known Limitations and Deferred Work
-
-- The shipped provider is local CPU speech; the service seam keeps model details out of the assistant consumer.
-- Raw audio remains process-local; a fresh provider connection restores bounded context from durable completed utterance text.
-- An active task is persisted as `interrupted` when the service stops. Recovery reports its last spoken progress without automatically replaying the task or any side-effecting command.
-- The filtered Voice history index is browser-local; clearing site data does not delete the underlying Sessions.
-- The browser client surface targets the dsh Web UI: it is emitted by the copied dsh client tsdown preset and loads through the dsh web runtime's `window.__ModuleLoader__` contract. The server-side packages are transport-agnostic, but the microphone/playback UI is not a standalone browser plugin.
+
+[![npm version](https://img.shields.io/npm/v/@flowingspring/dsh-voco.svg)](https://www.npmjs.com/package/@flowingspring/dsh-voco)
+[![license](https://img.shields.io/npm/l/@flowingspring/dsh-voco.svg)](https://github.com/lgquan/dsh-voco/blob/master/LICENSE)
+
+English | [中文](README.zh.md)
+
+Persistent, interruptible voice conversations for the DSH Web UI. Speak naturally, get an immediate conversational response, and delegate workspace tasks to a durable background Agent Session without losing context.
+
+## Install
+
+```sh
+dsh plugin --profile web add @flowingspring/dsh-voco
+dsh web
+```
+
+Install the DSH CLI first if needed:
+
+```sh
+npm install -g @deepseek-ai/dsh
+```
+
+## Configure speech recognition
+
+Set a [SiliconFlow](https://siliconflow.cn/) API key in the DSH environment:
+
+```dotenv
+SILICONFLOW_API_KEY=your-key
+```
+
+The plugin uses `XingChenAGI/XingChenASR-V3.2-Ultra` for cloud speech recognition and Edge TTS with `zh-CN-XiaoxiaoNeural` for speech output. Browser audio is uploaded only after lightweight local speech and silence detection.
+
+## Highlights
+
+- Keeps one background Agent Session bound to each Voice Session, including after DSH restarts.
+- Answers ordinary conversation directly and delegates only work that needs tools.
+- Speaks a short contextual acknowledgement before starting delegated Agent work.
+- Keeps full task reports in the task UI while speaking a concise, purpose-written result.
+- Supports interruption, browser navigation, reconnects, and restored conversation history.
+- Ships the server and browser surfaces as one public npm package.
+
+## Configuration
+
+The default utterance boundary is 1.5 seconds of continuous silence. Advanced settings such as `silenceDurationMs`, `speechThreshold`, `minSpeechDurationMs`, and `maxUtteranceMs` are available in the plugin profile configuration.
+
+## Requirements and limitations
+
+- The microphone and playback surface targets the DSH Web UI and is not a framework-independent browser plugin.
+- Speech recognition requires network access and a SiliconFlow API key.
+- Voice responses currently use the Chinese Xiaoxiao Edge TTS voice by default.
+
+Source, development instructions, and issue tracking are available in the [GitHub repository](https://github.com/lgquan/dsh-voco).
+
+## License
+
+[MIT](LICENSE)

package/README.zh.md

@@ -1,31 +1,56 @@
 # `@flowingspring/dsh-voco`
-
-[English](README.md) | 中文
-
-浏览器 PCM 只在本地做轻量静音检测，连续静音三秒后以 WAV 上传硅基流动完成识别。
-
-`dsh-voco` 的 `voice` profile patch-layer bundle。它把 provider transport 直接挂到当前来源 Session，并从持久化的 `voice/utterance-end` 事件恢复最近对话。轻量前台模型直接回答普通对话，只委派需要工具的工作。同一来源 Session 的委派持续复用并持久绑定一个普通 Task Agent Session；`voice/agent-binding-state` 保存语音会话、后台 Agent、工作区、最后任务、最近使用时间和状态，服务重启后优先从该记录恢复，并兼容旧的 `voice/task-session-bound`。完整结果和进度保留在任务界面；独立模型结合用户原话重写最终结果以及需要用户处理的问题或警告。进度只显示在折叠任务详情中，不会额外创建语音消息或 TTS 请求。后台 Agent 不能直接指定播报文本。root 持有的音频在跳转期间持续运行，浏览器历史索引从侧栏展示已保存的 Voice Session。
-
-## 模型体验
-
-### 语音 profile 组合
-
-#### 模型看到什么
-
-语音发起的工作只以已接受的 `realtime_delegation` 信封与准确 id 更新到达来源 Session 所绑定的固定后台 Task Agent。只有该 Task Agent 收到作用域内的 `send_voice_message` 后台工具，用于发送结构化的 `progress | result | warning | error | question` 事件；桥接层直接创建或恢复准确目标，因此不增加 project 列举工具。本地 provider 负责语音输入输出，桥接层只暴露任务编排工具。
-
-#### Token 影响
-
-每个新的前台话语会消耗一次轻量模型路由调用。被委派的工作还会消耗任务执行、后台回报和辅助事件改写的模型 token；本地 VAD、ASR、TTS 不产生按分钟的语音 API 费用。
-
-#### KV Cache 影响
-
-只有已接受的 command 扩展持续复用的 Task Agent 历史。持久绑定状态会在重启后恢复同一个 Agent Session；Voice Session 的最近话语用于恢复本地语音对话。
-
-## 已知限制与后续工作
-
-- 随附的 provider 是本地 CPU 语音；service seam 保证 assistant consumer 不依赖模型细节。
-- 原始音频仍限于当前进程；新的 provider 连接会从已完成的持久 utterance 文本恢复有界上下文。
-- 服务停止时，活跃任务会持久标记为 `interrupted`。恢复时只告知最后一次已播报进度，不会自动重放任务或任何有副作用的命令。
-- 筛选后的语音历史索引只属于当前浏览器；清除站点数据不会删除底层 Session。
-- 浏览器客户端界面面向 dsh Web UI：它由复制而来的 dsh client tsdown 预设构建，并通过 dsh web 运行时的 `window.__ModuleLoader__` 契约加载。服务端包与传输无关，但麦克风／播放 UI 不是独立浏览器插件。
+
+[![npm version](https://img.shields.io/npm/v/@flowingspring/dsh-voco.svg)](https://www.npmjs.com/package/@flowingspring/dsh-voco)
+[![许可证](https://img.shields.io/npm/l/@flowingspring/dsh-voco.svg)](https://github.com/lgquan/dsh-voco/blob/master/LICENSE)
+
+[English](README.md) | 中文
+
+面向 DSH Web UI 的可恢复、可打断语音对话插件。你可以自然说出需求、立即获得口语回复，并把需要工具的工作委派给持续复用的后台 Agent Session，而不会丢失任务上下文。
+
+## 安装
+
+```sh
+dsh plugin --profile web add @flowingspring/dsh-voco
+dsh web
+```
+
+如果尚未安装 DSH 命令行：
+
+```sh
+npm install -g @deepseek-ai/dsh
+```
+
+## 配置语音识别
+
+在 DSH 的运行环境中设置[硅基流动](https://siliconflow.cn/) API Key：
+
+```dotenv
+SILICONFLOW_API_KEY=你的密钥
+```
+
+插件使用 `XingChenAGI/XingChenASR-V3.2-Ultra` 完成云端语音识别，并通过 Edge TTS 的 `zh-CN-XiaoxiaoNeural` 音色输出语音。浏览器只在本地做轻量起音和静音检测，确认一句话结束后才上传音频。
+
+## 主要功能
+
+- 每个 Voice Session 持续绑定一个后台 Agent Session，重启 DSH 后也能恢复。
+- 普通聊天直接回答，只有需要工具的工作才委派给后台 Agent。
+- 委派任务启动前立即播报一句贴合当前请求的简短确认语。
+- 完整任务报告保留在任务界面，语音只播报专门生成的简洁结果。
+- 支持语音打断、页面切换、断线重连以及历史对话恢复。
+- 服务端和浏览器界面统一通过一个公开 npm 包发行。
+
+## 配置项
+
+默认以连续静音 1.5 秒作为一句话的边界。`silenceDurationMs`、`speechThreshold`、`minSpeechDurationMs` 和 `maxUtteranceMs` 等高级参数可在插件 profile 配置中调整。
+
+## 要求与限制
+
+- 麦克风和播放界面面向 DSH Web UI，并不是框架无关的浏览器插件。
+- 云端语音识别需要网络连接及硅基流动 API Key。
+- 语音回复目前默认使用 Edge TTS 的中文晓晓音色。
+
+源码、开发说明和问题反馈请前往 [GitHub 仓库](https://github.com/lgquan/dsh-voco)。
+
+## 许可证
+
+[MIT](LICENSE)

package/lib/client.js

@@ -18,9 +18,9 @@ window.__ModuleLoader__.load({
 			document.head.appendChild(tag);
 		}
 		var VoiceControl_module_css_default = {
-			"active": "PapC8q_active",
 			"voice-pulse": "PapC8q_voice-pulse",
-			"button": "PapC8q_button"
+			"button": "PapC8q_button",
+			"active": "PapC8q_active"
 		};
 		//#endregion
 		//#region ../ui-voice/src/client/VoiceControl.tsx
@@ -89,17 +89,17 @@ window.__ModuleLoader__.load({
 			document.head.appendChild(tag);
 		}
 		var VoiceHistoryAction_module_css_default = {
-			"list": "hhHVeG_list",
-			"empty": "hhHVeG_empty",
+			"title": "hhHVeG_title",
 			"panel": "hhHVeG_panel",
-			"wave": "hhHVeG_wave",
-			"meta": "hhHVeG_meta",
 			"row": "hhHVeG_row",
-			"close": "hhHVeG_close",
-			"rowText": "hhHVeG_rowText",
-			"title": "hhHVeG_title",
+			"trigger": "hhHVeG_trigger",
+			"wave": "hhHVeG_wave",
 			"header": "hhHVeG_header",
-			"trigger": "hhHVeG_trigger"
+			"empty": "hhHVeG_empty",
+			"rowText": "hhHVeG_rowText",
+			"close": "hhHVeG_close",
+			"list": "hhHVeG_list",
+			"meta": "hhHVeG_meta"
 		};
 		//#endregion
 		//#region ../ui-voice/src/client/VoiceHistoryAction.tsx
@@ -211,23 +211,23 @@ window.__ModuleLoader__.load({
 			document.head.appendChild(tag);
 		}
 		var VoiceNodeViews_module_css_default = {
+			"taskTitle": "KSbXkW_taskTitle",
 			"bubble": "KSbXkW_bubble",
-			"taskCancel": "KSbXkW_taskCancel",
-			"meta": "KSbXkW_meta",
-			"voiceBadge": "KSbXkW_voiceBadge",
 			"taskInput": "KSbXkW_taskInput",
-			"taskSummary": "KSbXkW_taskSummary",
-			"taskDetails": "KSbXkW_taskDetails",
 			"taskStatus": "KSbXkW_taskStatus",
+			"taskDetails": "KSbXkW_taskDetails",
 			"taskDot": "KSbXkW_taskDot",
-			"taskTitle": "KSbXkW_taskTitle",
-			"taskChevron": "KSbXkW_taskChevron",
 			"taskActions": "KSbXkW_taskActions",
-			"miniWave": "KSbXkW_miniWave",
 			"taskCard": "KSbXkW_taskCard",
+			"taskSummary": "KSbXkW_taskSummary",
+			"miniWave": "KSbXkW_miniWave",
+			"utterance": "KSbXkW_utterance",
+			"meta": "KSbXkW_meta",
 			"taskLink": "KSbXkW_taskLink",
+			"taskCancel": "KSbXkW_taskCancel",
+			"voiceBadge": "KSbXkW_voiceBadge",
 			"taskUpdate": "KSbXkW_taskUpdate",
-			"utterance": "KSbXkW_utterance"
+			"taskChevron": "KSbXkW_taskChevron"
 		};
 		//#endregion
 		//#region ../ui-voice/src/client/VoiceNodeViews.tsx
@@ -355,12 +355,12 @@ window.__ModuleLoader__.load({
 			document.head.appendChild(tag);
 		}
 		var VoiceOverlay_module_css_default = {
-			"voice-wave": "AAuDqa_voice-wave",
-			"stop": "AAuDqa_stop",
-			"status": "AAuDqa_status",
 			"wave": "AAuDqa_wave",
 			"root": "AAuDqa_root",
-			"link": "AAuDqa_link"
+			"link": "AAuDqa_link",
+			"stop": "AAuDqa_stop",
+			"status": "AAuDqa_status",
+			"voice-wave": "AAuDqa_voice-wave"
 		};
 		//#endregion
 		//#region ../ui-voice/src/client/VoiceOverlay.tsx
@@ -431,6 +431,11 @@ window.__ModuleLoader__.load({
 			generation = 0;
 			frame;
 			pendingText = [];
+			onConversationStarted;
+			conversationStarted = false;
+			constructor(options = {}) {
+				this.onConversationStarted = options.onConversationStarted;
+			}
 			/** Return the identity-stable snapshot until a voice fact changes. */
 			getSnapshot = () => this.snapshot;
 			/** Subscribe to state and frame-coalesced transcript changes. */
@@ -447,6 +452,7 @@ window.__ModuleLoader__.load({
 			async start(sessionId) {
 				const generation = ++this.generation;
 				this.pendingText.splice(0);
+				this.conversationStarted = false;
 				const contexts = createAudioContexts();
 				await this.closeTransport();
 				if (generation !== this.generation) {
@@ -584,7 +590,9 @@ window.__ModuleLoader__.load({
 					return;
 				}
 				if (type === "transcription.completed") {
-					this.setText(stringOf(event.utteranceId), stringOf(event.text), "immediate");
+					const text = stringOf(event.text);
+					this.setText(stringOf(event.utteranceId), text, "immediate");
+					if (text.trim() !== "") this.markConversationStarted();
 					return;
 				}
 				if (type === "output_text.started") {
@@ -598,6 +606,7 @@ window.__ModuleLoader__.load({
 				if (type === "output_text.done") {
 					const text = stringOf(event.text);
 					if (text !== "") this.setText(stringOf(event.utteranceId), text, "immediate");
+					if (text.trim() !== "") this.markConversationStarted();
 					return;
 				}
 				if (type === "output_audio.started") {
@@ -610,6 +619,13 @@ window.__ModuleLoader__.load({
 				}
 				if (type === "error") this.handleUnexpectedClose();
 			}
+			markConversationStarted() {
+				if (this.conversationStarted) return;
+				const sessionId = this.snapshot.sessionId;
+				if (sessionId === void 0) return;
+				this.conversationStarted = true;
+				this.onConversationStarted?.(sessionId);
+			}
 			publish(snapshot) {
 				if (this.frame !== void 0) {
 					cancelAnimationFrame(this.frame);
@@ -743,7 +759,7 @@ window.__ModuleLoader__.load({
 					return;
 				}
 				const record = recordOf(event);
-				if (record?.type === "transcription.started" || record?.type === "response.interrupted") stopPlayback();
+				if (record?.type === "response.interrupted") stopPlayback();
 				if (record?.type === "output_audio.done") {
 					outputDone = true;
 					notifyPlaybackEnded();
@@ -1357,9 +1373,19 @@ registerProcessor('dsh-pcm-capture', DshPcmCaptureProcessor)
 			"sessions",
 			"locale"
 		];
+		/** Refresh the Host-backed Session list when the runtime exposes its baseline pull. */
+		function refreshSessions(ctx) {
+			const service = ctx.sessions;
+			if (typeof service.refresh !== "function") return;
+			service.refresh.call(ctx.sessions).catch((cause) => {
+				console.warn("voice session list refresh failed:", cause);
+			});
+		}
 		/** Mount root transport controls and durable Voice conversation renderers. @param ctx - browser context. */
 		function apply(ctx) {
-			const controller = new VoiceController();
+			const controller = new VoiceController({ onConversationStarted: () => {
+				refreshSessions(ctx);
+			} });
 			const history = new VoiceHistoryStore();
 			const textSubmit = new VoiceTextSubmitBridge(controller, (sessionId) => {
 				const scope = ctx.sessions.scope(sessionId);