@goodandready/dsh-voice 0.9.30 โ†’ 0.9.31

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,252 +1,252 @@
1
- # ๐Ÿ“ฆ @goodandready/dsh-voice
2
-
3
- <div align="center">
4
-
5
- <h3>Zero-Latency Streaming Dictation & Multi-Provider Voice Input for DeepSeek Harness</h3>
6
-
7
- <p align="center">
8
- <a href="https://www.npmjs.com/package/@goodandready/dsh-voice"><img src="https://img.shields.io/npm/v/@goodandready/dsh-voice.svg?style=for-the-badge&color=6366f1&labelColor=1e1b4b" alt="npm version"></a>
9
- <a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-10b981.svg?style=for-the-badge&color=10b981&labelColor=064e3b" alt="license"></a>
10
- <a href="https://github.com/topics/dsh-plugin"><img src="https://img.shields.io/badge/DSH-Plugin-8b5cf6.svg?style=for-the-badge&labelColor=2e1065" alt="DSH Plugin"></a>
11
- <a href="https://nodejs.org"><img src="https://img.shields.io/badge/Node-20%2B-f59e0b.svg?style=for-the-badge&labelColor=451a03" alt="Node version"></a>
12
- </p>
13
-
14
- <p align="center">
15
- <a href="https://goodandready.app/"><img src="https://img.shields.io/badge/All_Author_Projects-goodandready.app-ff4500.svg?style=for-the-badge&logo=rocket&logoColor=white&labelColor=1a1a2e" alt="All Author Projects"></a>
16
- </p>
17
-
18
- <p align="center">
19
- <a href="README.md"><b>๐Ÿ‡ฌ๐Ÿ‡ง English</b></a> โ€ข
20
- <a href="README.ru.md"><b>๐Ÿ‡ท๐Ÿ‡บ ะ ัƒััะบะธะน</b></a> โ€ข
21
- <a href="README.zh.md"><b>๐Ÿ‡จ๐Ÿ‡ณ ไธญๆ–‡่ฏดๆ˜Ž</b></a>
22
- </p>
23
-
24
- <table align="center">
25
- <tr>
26
- <td align="center">
27
- โญ <strong>If you like this plugin, please star it on GitHub</strong> โ€” it shows me that the plugin is useful to you and motivates me to keep developing it.
28
- <br><br>
29
- ๐Ÿ› <strong>If you find a bug or would like to request a feature</strong>, open a GitHub issue in any language โ€” I will review your proposal and implement useful suggestions in a future plugin version.
30
- </td>
31
- </tr>
32
- </table>
33
-
34
- </div>
35
-
36
- ---
37
-
38
- ## Requirements
39
- ### Browser Microphone & Secure Context\nModern browsers require a **Secure Context** (`https://` or `http://localhost` / `http://127.0.0.1`) for microphone capture (`navigator.mediaDevices.getUserMedia`). When accessing DSH over a local network (e.g., `http://192.168.1.x:3080` via `dsh-lanmode`), browsers will block microphone access by default. Serve DSH behind an HTTPS reverse proxy or configure browser security flags (`chrome://flags/#unsafely-treat-insecure-origin-as-secure`) for LAN IP access.\n
40
-
41
- ## โšก Overview
42
-
43
- **`dsh-voice`** brings voice superpowers to the **DeepSeek Harness** Web UI. Whether you need hands-free real-time streaming dictation segmented on natural breath pauses or crisp voice notes with keyboard/mouse Push-to-Talk gestures, `dsh-voice` ensures your audio is never lost thanks to **automatic multi-provider fallback chains**.
44
-
45
- ```mermaid
46
- graph LR
47
- subgraph Client [Browser Web UI]
48
- Mic[๐ŸŽ™๏ธ Dictation Mic] -->|VAD Cut on Pause| Stream[Audio Chunks]
49
- Wave[๐ŸŒŠ Voice Message] -->|Hold / Release| PTT[Push-to-Talk]
50
- end
51
-
52
- subgraph Host [DSH Host Backend]
53
- Stream --> FFMPEG[ffmpeg 16kHz Transcoder]
54
- PTT --> FFMPEG
55
- FFMPEG --> Chain{Fallback Chain}
56
-
57
- Chain -->|1st Priority| P1[Deepgram / Nova-2]
58
- Chain -.->|On Rate Limit / 429| P2[Groq / Whisper Turbo]
59
- Chain -.->|On Failure| P3[Local whisper.cpp / Offline]
60
- end
61
-
62
- subgraph Output [Target]
63
- P1 --> Composer[๐Ÿ’ฌ Web Composer / Chat]
64
- P2 --> Composer
65
- P3 --> Composer
66
- end
67
-
68
- style Client fill:#1e1e2e,stroke:#89b4fa,stroke-width:2px,color:#cdd6f4
69
- style Host fill:#181825,stroke:#cba6f7,stroke-width:2px,color:#cdd6f4
70
- style Output fill:#11111b,stroke:#a6e3a1,stroke-width:2px,color:#cdd6f4
71
- ```
72
-
73
- ---
74
-
75
- ## โœจ Key Features
76
-
77
- * ๐ŸŽ™๏ธ **Streaming Dictation with VAD**: Speech is automatically sliced at natural pauses (`vadSilenceMs`, default 700ms) and typed into the composer in real time.
78
- * ๐ŸŒŠ **Voice Notes with Cancel Window**: Record your thought and have it automatically dispatched to the agent after a safety countdown (`autoSendMs`, default 4000ms).
79
- * ๐ŸŽฎ **Tactile Push-to-Talk**:
80
- * **Mouse**: Hold the wave button โ€” releasing sends the message; dragging pointer away discards.
81
- * **Keyboard**: Hold <kbd>Ctrl</kbd> (or custom hotkey) for hands-free speaking; press <kbd>Esc</kbd> to cancel.
82
- * โšก **Live In-Browser Captions (`browser`)**: Web Speech API speech-to-text with real-time floating captions as you speak (Note: standard browser recognition typically streams audio to vendor servers like Google in Chrome; for true 100% offline privacy, use local `whisper` or `sensevoice`).
83
- * ๐Ÿ›ก๏ธ **Ironclad Multi-Provider Fallbacks**: If your primary cloud provider runs out of credits or hits a 429 rate limit, requests seamlessly fail over down the chain.
84
- * ๐Ÿง  **Context Glossary Injection**: Automatically extracts code variables and identifiers from your composer draft to steer STT model accuracy on technical jargon.
85
- * ๐ŸŽต **Embedded Audio Player**: Preview, scrubber, and playback of your recorded voice message directly in chat and the composer dock.
86
- * ๐Ÿ”‡ **Hardware Noise Suppression Toggle**: Configurable in settings to toggle browser-level noise suppression, echo cancellation, and auto gain control.
87
- * ๐Ÿ“Š **Provider Latency & Health Dashboard**: Live visual telemetry of provider latency (ms), success rates, and errors directly within the settings UI.
88
- * ๐Ÿ”’ **Zero API Key Leakage**: Keys are resolved on the host via `ctx.credentials` (`credentialRef`) and never transmitted to browser clients.
89
- * ๐Ÿ–ฅ๏ธ **Offline Local Whisper Server**: Automatically boots and manages [whisper.cpp](https://github.com/ggerganov/whisper.cpp) (`whisper-server`) with on-the-fly `ffmpeg` transcode.
90
- * โšก **SenseVoice-ONNX / Sherpa-ONNX** *(0.8.11, updated 0.8.32)*: Ultra-fast (~50โ€“100ms) non-autoregressive local STT engine with 1-click automatic model installation to `~/.dsh/models/sensevoice`.
91
- * ๐ŸŽ™๏ธ **Gated Turn-Taking & Echo Prevention** *(0.8.32)*: Automatically mutes the mic while the assistant is speaking (`dsh:tts:start`/`stop`) and supports speech-triggered Barge-In.
92
- * ๐Ÿ‘ป **Live Ghost Interim Preview** *(0.8.32)*: Visual real-time preview of spoken words before final chunk transcription.
93
- * โฑ๏ธ **SVG Silence Ring Timer** *(0.8.32)*: Circular animated countdown indicator in the composer dock during the pending message delay.
94
- * ๐Ÿ—ฃ๏ธ **Hands-free Voice Actions** *(0.8.32)*: Spoken commands ("send", "cancel", "clear", "new line") trigger UI actions directly.
95
- * ๐Ÿงฉ **Structured Prompt Voice Selection** *(0.8.32)*: Spoken options automatically select buttons in interactive assistant choice prompts.
96
- * ๐Ÿ“– **Developer Lexicon & IT Jargon Correction** *(0.8.32)*: Phonetic normalization for developer slang (GitHub, Docker, Kubernetes, pnpm) with customizable settings dictionary.
97
- * ๐ŸŒŠ **Liquid Wave & Dynamic Orb Visualizer** *(0.8.12)*: Smooth animated audio visualization in the recording pill with real-time mic volume reactivity. Switch between organic multi-layer liquid waves, pulsating radiant orb, classic bars, or off.
98
-
99
- ---
100
-
101
- ## ๐Ÿ“ What's New in 0.8.32
102
-
103
- 10-point feature epic & SenseVoice 1-Click Installer (Issue #129):
104
-
105
- | Area | Feature | Description |
106
- |------|---------|-------------|
107
- | Local ASR | **1-Click SenseVoice-Small** | Automated download and extraction of `model.int8.onnx` and `tokens.txt` directly to `~/.dsh/models/sensevoice` via host loopback endpoint. |
108
- | Turn-Taking | **Gated Mode** | Microphone input is muted when assistant audio starts (`dsh:tts:start`) and unmuted on `dsh:tts:stop`, eliminating acoustic feedback. |
109
- | Interruption | **Barge-In** | Speaking immediately signals assistant TTS to pause and abort current speech playback. |
110
- | Feedback | **Ghost Interim Preview** | Semi-transparent live text preview in composer pill during recognition before sentence finalizing. |
111
- | Countdown | **SVG Silence Ring** | Circular SVG progress ring visually ticking down pending auto-send delay. |
112
- | Control | **Hands-free Actions** | Spoken control words ("send", "cancel", "clear", "new line") trigger actions instead of becoming message text. |
113
- | Prompts | **Structured Prompts** | Spoken replies automatically match and submit options in active harness interactive prompts. |
114
- | Vocabulary | **IT Jargon Normalizer** | Auto-corrects spoken developer slang to canonical spelling (GitHub, Docker, Kubernetes, etc.) with custom dictionary in Settings. |
115
- | Configuration | **Individual Toggles** | Dedicated switches for every enhancement in Settings card with native `--dsw-alias-*` token styling. |
116
-
117
- ---
118
-
119
- ## ๐Ÿ“ What's New in 0.8.19
120
-
121
- Quality batch after the v0.8.18 review (Gitea #79โ€“#87, PR #88):
122
-
123
- | Area | Change |
124
- |------|--------|
125
- | Settings placeholders | Model/path hints resolve through locale at **render time** โ€” no frozen i18n keys (#79) |
126
- | Model field hints | `whisperModel` / `sensevoiceModel` show path-to-model copy, not binary-autostart hints (#82) |
127
- | Visualizer | Liquid wave / dynamic orb read **DSH theme tokens** instead of hardcoded hex (#80) |
128
- | Style isolation | Injected CSS uses `data-dsh-plugin="dsh-voice"` so neighbour HMR cleanup cannot strip styles (#81) |
129
- | Source language | Code comments, errors, and tests are English; EN edit-command phrases added. Spoken RU STT patterns remain for recognition (#86) |
130
- | Locales | **Changed:** the full inline `ru` dictionary was removed. English is the only bundled locale. Install the translation plugin for Russian UI (#85) |
131
- | Client source | Browser client is built from ordered `lib/client-src/*.js` fragments via `npm run build:client` (#87) |
132
- | Process docs | Added `docs/design/DESIGN.md`, `index.md`, project `AGENTS.md`, `docs/testing/unit.md` (#83) |
133
-
134
- > [!IMPORTANT]
135
- > **v0.8.19 locale behavior:** without a translation plugin the Web UI stays in English. Session/edit **spoken** command phrases still match Russian speech for STT where documented; interface labels do not ship a second dictionary.
136
-
137
- ---
138
-
139
- ## ๐ŸŽฎ Four Ways to Speak
140
-
141
- | Mode | Gesture / Trigger | Behavior |
142
- |---|---|---|
143
- | **Dictation** | Click <kbd>๐ŸŽ™๏ธ Mic</kbd> | Speech is sliced on pauses (`vadSilenceMs`) and typed live into composer |
144
- | **Voice Message** | Click <kbd>๐ŸŒŠ Wave</kbd> | Records until stopped, then sends after cancel window (`autoSendMs`) |
145
- | **Mouse PTT** | Hold <kbd>๐ŸŒŠ Wave</kbd> | Records while held; release sends message, drag off button to discard |
146
- | **Keyboard PTT** | Hold <kbd>Ctrl</kbd> | Hands-free recording; release sends message, press <kbd>Esc</kbd> to discard |
147
-
148
- > [!TIP]
149
- > You can customize the keyboard modifier in settings (`hotkey`: `Control`, `Alt`, `Shift`, or any `KeyboardEvent.code`).
150
-
151
- ---
152
-
153
- ## ๐Ÿ› ๏ธ Supported Providers Matrix
154
-
155
- | Provider Key | Service Backend | Default Model | Credential Ref | Features & Notes |
156
- |---|---|---|---|---|
157
- | `browser-webgpu` | In-Browser WebGPU/WASM Whisper | `onnx-community/whisper-tiny` | *None* | 100% private offline client-side GPU transcription; falls back to server chain |
158
- | `browser` | Web Speech API | Native Browser | *None* | Zero latency, floating live captions in Chrome (vendor server-based; not offline) |
159
- | `deepgram` | Deepgram API | `nova-2` | `DEEPGRAM_API_KEY` | Ultra-fast cloud transcription |
160
- | `groq` | Groq Whisper | `whisper-large-v3-turbo` | `GROQ_API_KEY` | Near-instant inference speed |
161
- | `hf` | HuggingFace Inference | `openai/whisper-large-v3` | `HF_TOKEN` | High-accuracy open Whisper |
162
- | `local-whisper` | Local whisper.cpp | Server defined | *None* | 100% private, offline, no internet needed |
163
- | `sensevoice` | SenseVoice-ONNX / Sherpa-ONNX | `SenseVoiceSmall` | *None* | Ultra-fast (~50ms) local STT; supports Rockchip RK3588 NPU (rknpu), OpenVINO, CUDA |
164
-
165
- ### ๐Ÿš€ Ready-Made Presets (Plug & Play)
166
-
167
- Just specify the name in your fallback chain and add the corresponding API key:
168
- * `openai` (`whisper-1`) โ†’ `OPENAI_API_KEY`
169
- * `siliconflow` (`SenseVoiceSmall`) โ†’ `SILICONFLOW_API_KEY`
170
- * `mistral` (`voxtral-mini-latest`) โ†’ `MISTRAL_API_KEY`
171
- * `openrouter` (`google/gemini-2.5-flash`) โ†’ `OPENROUTER_API_KEY`
172
- * `deepinfra` (`whisper-large-v3-turbo`) โ†’ `DEEPINFRA_API_KEY`
173
- * `fireworks` (`whisper-v3-turbo`) โ†’ `FIREWORKS_API_KEY`
174
-
175
- ---
176
-
177
- ## ๐Ÿ“ฆ Quick Installation
178
-
179
- ```bash
180
- dsh plugin --profile web add @goodandready/dsh-voice
181
- ```
182
-
183
- > [!IMPORTANT]
184
- > Restart DSH Web UI after installation (`systemctl --user restart dsh-web`) and refresh your browser tab.
185
-
186
- ---
187
-
188
- ## โš™๏ธ Configuration
189
-
190
- Open **Settings โ†’ Plugins โ†’ Plugin settings โ†’ Voice** in the Web UI:
191
-
192
- ```yaml
193
- - id: dsh-voice
194
- config:
195
- dictation:
196
- language: "" # auto-detect by default; or specify "en", "ru", "zh", etc.
197
- vadSilenceMs: 700
198
- chain:
199
- - provider: deepgram
200
- - provider: groq
201
- - provider: local-whisper
202
- message:
203
- language: "" # auto-detect by default; or specify "en", "ru", "zh", etc.
204
- autoSendMs: 4000
205
- chain:
206
- - provider: openai
207
- - provider: local-whisper
208
- hotkey: Control
209
- autoStart: true
210
- whisperModel: /models/ggml-medium-q8_0.bin
211
- ```
212
-
213
- > **Note on language handling**: Setting `language: ""` (default) allows providers to auto-detect the spoken language. Spoken numeral normalization (`wordsToDigits`) and standard Russian tech slang correction (`DEFAULT_JARGON_DICTIONARY`) activate conditionally only when Russian is selected or auto-detected in the transcript, ensuring non-Russian speech is never distorted.
214
-
215
- ---
216
-
217
- ## ๐Ÿค– Agent Tool & HTTP API
218
-
219
- ### Agent Tool (`transcribe_audio`)
220
- Registers `transcribe_audio(file_path, language?)` in `ctx.tools`, allowing agents to analyze audio files, interview recordings, and voice notes directly from disk. Path containment is strictly enforced against allowed directories (`allowedAudioDirs`, `~/.dsh`, `tmpdir`, `cwd`) with symlink traversal escape prevention and magic-byte audio format validation.
221
-
222
- ### Internal HTTP Endpoints
223
- * `POST /dsh-voice/transcribe` โ€” `{ dataBase64, mimeType, mode }` โ†’ `{ ok, text, provider, tookMs }`
224
- * `POST /dsh-voice/polish` โ€” `{ text }` โ†’ `{ ok, text }`
225
- * `GET /dsh-voice/status` โ€” Returns daemon status, active chains, SenseVoice and `effectiveSensevoiceProvider`.
226
- * `GET /dsh-voice/config` โ€” Returns live plugin configuration snapshot.
227
- * `PUT /dsh-voice/config` โ€” Updates and persists plugin configuration across network.
228
- * `GET/POST /dsh-voice/sensevoice-installer` โ€” SenseVoice 1-click model installation status and trigger (protected by `isTrustedCaller`, model paths redacted).
229
- * `POST /api/dsh-voice/update` โ€” Triggers plugin self-update to latest compatible npm version (protected by local caller verification and profile lock check).
230
- * `GET /api/dsh-voice/update` โ€” Returns update check status (current version, latest version, updateAvailable).
231
-
232
- ### ๐Ÿ”’ Profile Lock & Controlled Operator Recovery Policy
233
- When updating or installing plugins, DeepSeek Harness profiles coordinate concurrent operations using `<profile-dir>/package.json.lock` (for example, `~/.dsh/profiles/<profile>/package.json.lock`).
234
- * **Canonical Contender Policy**: The contender process never deletes or unlinks an existing lockfile. Automatic stale lock removal by contenders is prohibited to eliminate TOCTOU (Time-of-Check to Time-of-Use) race conditions and prevent unintended lockfile clobbering.
235
- * **Diagnostics**: If `<profile-dir>/package.json.lock` is present, update attempts return `409 Conflict` with clear diagnostics:
236
- - If held by an active process, the PID is reported.
237
- - If filesystem inspection encounters access errors (`EACCES`, `EIO`), the operation fails closed with the error code preserved. Filesystem errors require inspecting filesystem health, permissions, or mount options โ€” **do not** remove the lockfile in response to `EACCES`/`EIO`.
238
- * **Controlled Operator Recovery Workflow**:
239
- If a previous installation was forcefully interrupted (e.g. system crash, OOM kill) leaving a stale lockfile, recovery is strictly an operator action and must follow this controlled procedure:
240
- 1. **Establish Exclusive Maintenance**: Ensure no concurrent processes, CLI commands (`dsh plugin add`), web UI operations, or automated updaters are running or scheduled to start for the target profile.
241
- 2. **Verify Quiescence and Ownership**: Check for active processes (`pgrep`, `ps`, `fuser`, or `lsof`) to confirm that no live process is currently performing package operations in the target profile directory. Lock age alone or observing an unparsed PID is **not** sufficient justification to delete a lock while another installer could start. If quiescence or non-ownership cannot be verified, do not remove the lock.
242
- 3. **Remove Confirmed Orphan Lock**: Only after confirming the lock is an orphan and exclusive maintenance is established, remove the specific profile lock without recursive or force flags:
243
- ```bash
244
- rm ~/.dsh/profiles/<profile>/package.json.lock
245
- ```
246
- 4. **Resume Operations**: Re-enable installations and retry the plugin update.
247
-
248
- ---
249
-
250
- ## ๐Ÿ“„ License
251
-
252
- MIT ยฉ [GooDAnDReaDY](https://github.com/GooDAnDReaDY)
1
+ # ๐Ÿ“ฆ @goodandready/dsh-voice
2
+
3
+ <div align="center">
4
+
5
+ <h3>Zero-Latency Streaming Dictation & Multi-Provider Voice Input for DeepSeek Harness</h3>
6
+
7
+ <p align="center">
8
+ <a href="https://www.npmjs.com/package/@goodandready/dsh-voice"><img src="https://img.shields.io/npm/v/@goodandready/dsh-voice.svg?style=for-the-badge&color=6366f1&labelColor=1e1b4b" alt="npm version"></a>
9
+ <a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-10b981.svg?style=for-the-badge&color=10b981&labelColor=064e3b" alt="license"></a>
10
+ <a href="https://github.com/topics/dsh-plugin"><img src="https://img.shields.io/badge/DSH-Plugin-8b5cf6.svg?style=for-the-badge&labelColor=2e1065" alt="DSH Plugin"></a>
11
+ <a href="https://nodejs.org"><img src="https://img.shields.io/badge/Node-20%2B-f59e0b.svg?style=for-the-badge&labelColor=451a03" alt="Node version"></a>
12
+ </p>
13
+
14
+ <p align="center">
15
+ <a href="https://goodandready.app/"><img src="https://img.shields.io/badge/All_Author_Projects-goodandready.app-ff4500.svg?style=for-the-badge&logo=rocket&logoColor=white&labelColor=1a1a2e" alt="All Author Projects"></a>
16
+ </p>
17
+
18
+ <p align="center">
19
+ <a href="README.md"><b>๐Ÿ‡ฌ๐Ÿ‡ง English</b></a> โ€ข
20
+ <a href="README.ru.md"><b>๐Ÿ‡ท๐Ÿ‡บ ะ ัƒััะบะธะน</b></a> โ€ข
21
+ <a href="README.zh.md"><b>๐Ÿ‡จ๐Ÿ‡ณ ไธญๆ–‡่ฏดๆ˜Ž</b></a>
22
+ </p>
23
+
24
+ <table align="center">
25
+ <tr>
26
+ <td align="center">
27
+ โญ <strong>If you like this plugin, please star it on GitHub</strong> โ€” it shows me that the plugin is useful to you and motivates me to keep developing it.
28
+ <br><br>
29
+ ๐Ÿ› <strong>If you find a bug or would like to request a feature</strong>, open a GitHub issue in any language โ€” I will review your proposal and implement useful suggestions in a future plugin version.
30
+ </td>
31
+ </tr>
32
+ </table>
33
+
34
+ </div>
35
+
36
+ ---
37
+
38
+ ## Requirements
39
+ ### Browser Microphone & Secure Context\nModern browsers require a **Secure Context** (`https://` or `http://localhost` / `http://127.0.0.1`) for microphone capture (`navigator.mediaDevices.getUserMedia`). When accessing DSH over a local network (e.g., `http://192.168.1.x:3080` via `dsh-lanmode`), browsers will block microphone access by default. Serve DSH behind an HTTPS reverse proxy or configure browser security flags (`chrome://flags/#unsafely-treat-insecure-origin-as-secure`) for LAN IP access.\n
40
+
41
+ ## โšก Overview
42
+
43
+ **`dsh-voice`** brings voice superpowers to the **DeepSeek Harness** Web UI. Whether you need hands-free real-time streaming dictation segmented on natural breath pauses or crisp voice notes with keyboard/mouse Push-to-Talk gestures, `dsh-voice` ensures your audio is never lost thanks to **automatic multi-provider fallback chains**.
44
+
45
+ ```mermaid
46
+ graph LR
47
+ subgraph Client [Browser Web UI]
48
+ Mic[๐ŸŽ™๏ธ Dictation Mic] -->|VAD Cut on Pause| Stream[Audio Chunks]
49
+ Wave[๐ŸŒŠ Voice Message] -->|Hold / Release| PTT[Push-to-Talk]
50
+ end
51
+
52
+ subgraph Host [DSH Host Backend]
53
+ Stream --> FFMPEG[ffmpeg 16kHz Transcoder]
54
+ PTT --> FFMPEG
55
+ FFMPEG --> Chain{Fallback Chain}
56
+
57
+ Chain -->|1st Priority| P1[Deepgram / Nova-2]
58
+ Chain -.->|On Rate Limit / 429| P2[Groq / Whisper Turbo]
59
+ Chain -.->|On Failure| P3[Local whisper.cpp / Offline]
60
+ end
61
+
62
+ subgraph Output [Target]
63
+ P1 --> Composer[๐Ÿ’ฌ Web Composer / Chat]
64
+ P2 --> Composer
65
+ P3 --> Composer
66
+ end
67
+
68
+ style Client fill:#1e1e2e,stroke:#89b4fa,stroke-width:2px,color:#cdd6f4
69
+ style Host fill:#181825,stroke:#cba6f7,stroke-width:2px,color:#cdd6f4
70
+ style Output fill:#11111b,stroke:#a6e3a1,stroke-width:2px,color:#cdd6f4
71
+ ```
72
+
73
+ ---
74
+
75
+ ## โœจ Key Features
76
+
77
+ * ๐ŸŽ™๏ธ **Streaming Dictation with VAD**: Speech is automatically sliced at natural pauses (`vadSilenceMs`, default 700ms) and typed into the composer in real time.
78
+ * ๐ŸŒŠ **Voice Notes with Cancel Window**: Record your thought and have it automatically dispatched to the agent after a safety countdown (`autoSendMs`, default 4000ms).
79
+ * ๐ŸŽฎ **Tactile Push-to-Talk**:
80
+ * **Mouse**: Hold the wave button โ€” releasing sends the message; dragging pointer away discards.
81
+ * **Keyboard**: Hold <kbd>Ctrl</kbd> (or custom hotkey) for hands-free speaking; press <kbd>Esc</kbd> to cancel.
82
+ * โšก **Live In-Browser Captions (`browser`)**: Web Speech API speech-to-text with real-time floating captions as you speak (Note: standard browser recognition typically streams audio to vendor servers like Google in Chrome; for true 100% offline privacy, use local `whisper` or `sensevoice`).
83
+ * ๐Ÿ›ก๏ธ **Ironclad Multi-Provider Fallbacks**: If your primary cloud provider runs out of credits or hits a 429 rate limit, requests seamlessly fail over down the chain.
84
+ * ๐Ÿง  **Context Glossary Injection**: Automatically extracts code variables and identifiers from your composer draft to steer STT model accuracy on technical jargon.
85
+ * ๐ŸŽต **Embedded Audio Player**: Preview, scrubber, and playback of your recorded voice message directly in chat and the composer dock.
86
+ * ๐Ÿ”‡ **Hardware Noise Suppression Toggle**: Configurable in settings to toggle browser-level noise suppression, echo cancellation, and auto gain control.
87
+ * ๐Ÿ“Š **Provider Latency & Health Dashboard**: Live visual telemetry of provider latency (ms), success rates, and errors directly within the settings UI.
88
+ * ๐Ÿ”’ **Zero API Key Leakage**: Keys are resolved on the host via `ctx.credentials` (`credentialRef`) and never transmitted to browser clients.
89
+ * ๐Ÿ–ฅ๏ธ **Offline Local Whisper Server**: Automatically boots and manages [whisper.cpp](https://github.com/ggerganov/whisper.cpp) (`whisper-server`) with on-the-fly `ffmpeg` transcode.
90
+ * โšก **SenseVoice-ONNX / Sherpa-ONNX** *(0.8.11, updated 0.8.32)*: Ultra-fast (~50โ€“100ms) non-autoregressive local STT engine with 1-click automatic model installation to `~/.dsh/models/sensevoice`.
91
+ * ๐ŸŽ™๏ธ **Gated Turn-Taking & Echo Prevention** *(0.8.32)*: Automatically mutes the mic while the assistant is speaking (`dsh:tts:start`/`stop`) and supports speech-triggered Barge-In.
92
+ * ๐Ÿ‘ป **Live Ghost Interim Preview** *(0.8.32)*: Visual real-time preview of spoken words before final chunk transcription.
93
+ * โฑ๏ธ **SVG Silence Ring Timer** *(0.8.32)*: Circular animated countdown indicator in the composer dock during the pending message delay.
94
+ * ๐Ÿ—ฃ๏ธ **Hands-free Voice Actions** *(0.8.32)*: Spoken commands ("send", "cancel", "clear", "new line") trigger UI actions directly.
95
+ * ๐Ÿงฉ **Structured Prompt Voice Selection** *(0.8.32)*: Spoken options automatically select buttons in interactive assistant choice prompts.
96
+ * ๐Ÿ“– **Developer Lexicon & IT Jargon Correction** *(0.8.32)*: Phonetic normalization for developer slang (GitHub, Docker, Kubernetes, pnpm) with customizable settings dictionary.
97
+ * ๐ŸŒŠ **Liquid Wave & Dynamic Orb Visualizer** *(0.8.12)*: Smooth animated audio visualization in the recording pill with real-time mic volume reactivity. Switch between organic multi-layer liquid waves, pulsating radiant orb, classic bars, or off.
98
+
99
+ ---
100
+
101
+ ## ๐Ÿ“ What's New in 0.8.32
102
+
103
+ 10-point feature epic & SenseVoice 1-Click Installer (Issue #129):
104
+
105
+ | Area | Feature | Description |
106
+ |------|---------|-------------|
107
+ | Local ASR | **1-Click SenseVoice-Small** | Automated download and extraction of `model.int8.onnx` and `tokens.txt` directly to `~/.dsh/models/sensevoice` via host loopback endpoint. |
108
+ | Turn-Taking | **Gated Mode** | Microphone input is muted when assistant audio starts (`dsh:tts:start`) and unmuted on `dsh:tts:stop`, eliminating acoustic feedback. |
109
+ | Interruption | **Barge-In** | Speaking immediately signals assistant TTS to pause and abort current speech playback. |
110
+ | Feedback | **Ghost Interim Preview** | Semi-transparent live text preview in composer pill during recognition before sentence finalizing. |
111
+ | Countdown | **SVG Silence Ring** | Circular SVG progress ring visually ticking down pending auto-send delay. |
112
+ | Control | **Hands-free Actions** | Spoken control words ("send", "cancel", "clear", "new line") trigger actions instead of becoming message text. |
113
+ | Prompts | **Structured Prompts** | Spoken replies automatically match and submit options in active harness interactive prompts. |
114
+ | Vocabulary | **IT Jargon Normalizer** | Auto-corrects spoken developer slang to canonical spelling (GitHub, Docker, Kubernetes, etc.) with custom dictionary in Settings. |
115
+ | Configuration | **Individual Toggles** | Dedicated switches for every enhancement in Settings card with native `--dsw-alias-*` token styling. |
116
+
117
+ ---
118
+
119
+ ## ๐Ÿ“ What's New in 0.8.19
120
+
121
+ Quality batch after the v0.8.18 review (Gitea #79โ€“#87, PR #88):
122
+
123
+ | Area | Change |
124
+ |------|--------|
125
+ | Settings placeholders | Model/path hints resolve through locale at **render time** โ€” no frozen i18n keys (#79) |
126
+ | Model field hints | `whisperModel` / `sensevoiceModel` show path-to-model copy, not binary-autostart hints (#82) |
127
+ | Visualizer | Liquid wave / dynamic orb read **DSH theme tokens** instead of hardcoded hex (#80) |
128
+ | Style isolation | Injected CSS uses `data-dsh-plugin="dsh-voice"` so neighbour HMR cleanup cannot strip styles (#81) |
129
+ | Source language | Code comments, errors, and tests are English; EN edit-command phrases added. Spoken RU STT patterns remain for recognition (#86) |
130
+ | Locales | **Changed:** the full inline `ru` dictionary was removed. English is the only bundled locale. Install the translation plugin for Russian UI (#85) |
131
+ | Client source | Browser client is built from ordered `lib/client-src/*.js` fragments via `npm run build:client` (#87) |
132
+ | Process docs | Added `docs/design/DESIGN.md`, `index.md`, project `AGENTS.md`, `docs/testing/unit.md` (#83) |
133
+
134
+ > [!IMPORTANT]
135
+ > **v0.8.19 locale behavior:** without a translation plugin the Web UI stays in English. Session/edit **spoken** command phrases still match Russian speech for STT where documented; interface labels do not ship a second dictionary.
136
+
137
+ ---
138
+
139
+ ## ๐ŸŽฎ Four Ways to Speak
140
+
141
+ | Mode | Gesture / Trigger | Behavior |
142
+ |---|---|---|
143
+ | **Dictation** | Click <kbd>๐ŸŽ™๏ธ Mic</kbd> | Speech is sliced on pauses (`vadSilenceMs`) and typed live into composer |
144
+ | **Voice Message** | Click <kbd>๐ŸŒŠ Wave</kbd> | Records until stopped, then sends after cancel window (`autoSendMs`) |
145
+ | **Mouse PTT** | Hold <kbd>๐ŸŒŠ Wave</kbd> | Records while held; release sends message, drag off button to discard |
146
+ | **Keyboard PTT** | Hold <kbd>Ctrl</kbd> | Hands-free recording; release sends message, press <kbd>Esc</kbd> to discard |
147
+
148
+ > [!TIP]
149
+ > You can customize the keyboard modifier in settings (`hotkey`: `Control`, `Alt`, `Shift`, or any `KeyboardEvent.code`).
150
+
151
+ ---
152
+
153
+ ## ๐Ÿ› ๏ธ Supported Providers Matrix
154
+
155
+ | Provider Key | Service Backend | Default Model | Credential Ref | Features & Notes |
156
+ |---|---|---|---|---|
157
+ | `browser-webgpu` | In-Browser WebGPU/WASM Whisper | `onnx-community/whisper-tiny` | *None* | 100% private offline client-side GPU transcription; falls back to server chain |
158
+ | `browser` | Web Speech API | Native Browser | *None* | Zero latency, floating live captions in Chrome (vendor server-based; not offline) |
159
+ | `deepgram` | Deepgram API | `nova-2` | `DEEPGRAM_API_KEY` | Ultra-fast cloud transcription |
160
+ | `groq` | Groq Whisper | `whisper-large-v3-turbo` | `GROQ_API_KEY` | Near-instant inference speed |
161
+ | `hf` | HuggingFace Inference | `openai/whisper-large-v3` | `HF_TOKEN` | High-accuracy open Whisper |
162
+ | `local-whisper` | Local whisper.cpp | Server defined | *None* | 100% private, offline, no internet needed |
163
+ | `sensevoice` | SenseVoice-ONNX / Sherpa-ONNX | `SenseVoiceSmall` | *None* | Ultra-fast (~50ms) local STT; supports Rockchip RK3588 NPU (rknpu), OpenVINO, CUDA |
164
+
165
+ ### ๐Ÿš€ Ready-Made Presets (Plug & Play)
166
+
167
+ Just specify the name in your fallback chain and add the corresponding API key:
168
+ * `openai` (`whisper-1`) โ†’ `OPENAI_API_KEY`
169
+ * `siliconflow` (`SenseVoiceSmall`) โ†’ `SILICONFLOW_API_KEY`
170
+ * `mistral` (`voxtral-mini-latest`) โ†’ `MISTRAL_API_KEY`
171
+ * `openrouter` (`google/gemini-2.5-flash`) โ†’ `OPENROUTER_API_KEY`
172
+ * `deepinfra` (`whisper-large-v3-turbo`) โ†’ `DEEPINFRA_API_KEY`
173
+ * `fireworks` (`whisper-v3-turbo`) โ†’ `FIREWORKS_API_KEY`
174
+
175
+ ---
176
+
177
+ ## ๐Ÿ“ฆ Quick Installation
178
+
179
+ ```bash
180
+ dsh plugin --profile web add @goodandready/dsh-voice
181
+ ```
182
+
183
+ > [!IMPORTANT]
184
+ > Restart DSH Web UI after installation (`systemctl --user restart dsh-web`) and refresh your browser tab.
185
+
186
+ ---
187
+
188
+ ## โš™๏ธ Configuration
189
+
190
+ Open **Settings โ†’ Plugins โ†’ Plugin settings โ†’ Voice** in the Web UI:
191
+
192
+ ```yaml
193
+ - id: dsh-voice
194
+ config:
195
+ dictation:
196
+ language: "" # auto-detect by default; or specify "en", "ru", "zh", etc.
197
+ vadSilenceMs: 700
198
+ chain:
199
+ - provider: deepgram
200
+ - provider: groq
201
+ - provider: local-whisper
202
+ message:
203
+ language: "" # auto-detect by default; or specify "en", "ru", "zh", etc.
204
+ autoSendMs: 4000
205
+ chain:
206
+ - provider: openai
207
+ - provider: local-whisper
208
+ hotkey: Control
209
+ autoStart: true
210
+ whisperModel: /models/ggml-medium-q8_0.bin
211
+ ```
212
+
213
+ > **Note on language handling**: Setting `language: ""` (default) allows providers to auto-detect the spoken language. Spoken numeral normalization (`wordsToDigits`) and standard Russian tech slang correction (`DEFAULT_JARGON_DICTIONARY`) activate conditionally only when Russian is selected or auto-detected in the transcript, ensuring non-Russian speech is never distorted.
214
+
215
+ ---
216
+
217
+ ## ๐Ÿค– Agent Tool & HTTP API
218
+
219
+ ### Agent Tool (`transcribe_audio`)
220
+ Registers `transcribe_audio(file_path, language?)` in `ctx.tools`, allowing agents to analyze audio files, interview recordings, and voice notes directly from disk. Path containment is strictly enforced against allowed directories (`allowedAudioDirs`, `~/.dsh`, `tmpdir`, `cwd`) with symlink traversal escape prevention and magic-byte audio format validation.
221
+
222
+ ### Internal HTTP Endpoints
223
+ * `POST /dsh-voice/transcribe` โ€” `{ dataBase64, mimeType, mode }` โ†’ `{ ok, text, provider, tookMs }`
224
+ * `POST /dsh-voice/polish` โ€” `{ text }` โ†’ `{ ok, text }`
225
+ * `GET /dsh-voice/status` โ€” Returns daemon status, active chains, SenseVoice and `effectiveSensevoiceProvider`.
226
+ * `GET /dsh-voice/config` โ€” Returns live plugin configuration snapshot.
227
+ * `PUT /dsh-voice/config` โ€” Updates and persists plugin configuration across network.
228
+ * `GET/POST /dsh-voice/sensevoice-installer` โ€” SenseVoice 1-click model installation status and trigger (protected by `isTrustedCaller`, model paths redacted).
229
+ * `POST /api/dsh-voice/update` โ€” Triggers plugin self-update to latest compatible npm version (protected by local caller verification and profile lock check).
230
+ * `GET /api/dsh-voice/update` โ€” Returns update check status (current version, latest version, updateAvailable).
231
+
232
+ ### ๐Ÿ”’ Profile Lock & Controlled Operator Recovery Policy
233
+ When updating or installing plugins, DeepSeek Harness profiles coordinate concurrent operations using `<profile-dir>/package.json.lock` (for example, `~/.dsh/profiles/<profile>/package.json.lock`).
234
+ * **Canonical Contender Policy**: The contender process never deletes or unlinks an existing lockfile. Automatic stale lock removal by contenders is prohibited to eliminate TOCTOU (Time-of-Check to Time-of-Use) race conditions and prevent unintended lockfile clobbering.
235
+ * **Diagnostics**: If `<profile-dir>/package.json.lock` is present, update attempts return `409 Conflict` with clear diagnostics:
236
+ - If held by an active process, the PID is reported.
237
+ - If filesystem inspection encounters access errors (`EACCES`, `EIO`), the operation fails closed with the error code preserved. Filesystem errors require inspecting filesystem health, permissions, or mount options โ€” **do not** remove the lockfile in response to `EACCES`/`EIO`.
238
+ * **Controlled Operator Recovery Workflow**:
239
+ If a previous installation was forcefully interrupted (e.g. system crash, OOM kill) leaving a stale lockfile, recovery is strictly an operator action and must follow this controlled procedure:
240
+ 1. **Establish Exclusive Maintenance**: Ensure no concurrent processes, CLI commands (`dsh plugin add`), web UI operations, or automated updaters are running or scheduled to start for the target profile.
241
+ 2. **Verify Quiescence and Ownership**: Check for active processes (`pgrep`, `ps`, `fuser`, or `lsof`) to confirm that no live process is currently performing package operations in the target profile directory. Lock age alone or observing an unparsed PID is **not** sufficient justification to delete a lock while another installer could start. If quiescence or non-ownership cannot be verified, do not remove the lock.
242
+ 3. **Remove Confirmed Orphan Lock**: Only after confirming the lock is an orphan and exclusive maintenance is established, remove the specific profile lock without recursive or force flags:
243
+ ```bash
244
+ rm ~/.dsh/profiles/<profile>/package.json.lock
245
+ ```
246
+ 4. **Resume Operations**: Re-enable installations and retry the plugin update.
247
+
248
+ ---
249
+
250
+ ## ๐Ÿ“„ License
251
+
252
+ MIT ยฉ [GooDAnDReaDY](https://github.com/GooDAnDReaDY)