@goodandready/dsh-voice 0.8.9 → 0.8.11
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +1 -1
- package/README.md +137 -181
- package/README.ru.md +172 -0
- package/README.zh.md +116 -0
- package/lib/chain.js +18 -3
- package/lib/client.js +290 -8
- package/lib/index.js +304 -5
- package/lib/providers.js +62 -2
- package/lib/stats.js +72 -0
- package/package.json +2 -2
package/README.zh.md
ADDED
|
@@ -0,0 +1,116 @@
|
|
|
1
|
+
# 📦 @goodandready/dsh-voice
|
|
2
|
+
|
|
3
|
+
<div align="center">
|
|
4
|
+
|
|
5
|
+
<h3>DeepSeek Harness 零延迟流式语音听写与多服务商对讲输入插件</h3>
|
|
6
|
+
|
|
7
|
+
<p align="center">
|
|
8
|
+
<a href="https://www.npmjs.com/package/@goodandready/dsh-voice"><img src="https://img.shields.io/npm/v/@goodandready/dsh-voice.svg?style=for-the-badge&color=6366f1&labelColor=1e1b4b" alt="npm version"></a>
|
|
9
|
+
<a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-10b981.svg?style=for-the-badge&color=10b981&labelColor=064e3b" alt="license"></a>
|
|
10
|
+
<a href="https://github.com/topics/dsh-plugin"><img src="https://img.shields.io/badge/DSH-Plugin-8b5cf6.svg?style=for-the-badge&labelColor=2e1065" alt="DSH Plugin"></a>
|
|
11
|
+
<a href="https://nodejs.org"><img src="https://img.shields.io/badge/Node-20%2B-f59e0b.svg?style=for-the-badge&labelColor=451a03" alt="Node version"></a>
|
|
12
|
+
</p>
|
|
13
|
+
|
|
14
|
+
<p align="center">
|
|
15
|
+
<a href="https://goodandready.app/"><img src="https://img.shields.io/badge/作者全部项目-goodandready.app-ff4500.svg?style=for-the-badge&logo=rocket&logoColor=white&labelColor=1a1a2e" alt="作者全部项目"></a>
|
|
16
|
+
</p>
|
|
17
|
+
|
|
18
|
+
<p align="center">
|
|
19
|
+
<a href="README.md"><b>🇬🇧 English</b></a> •
|
|
20
|
+
<a href="README.ru.md"><b>🇷🇺 Русский</b></a> •
|
|
21
|
+
<a href="README.zh.md"><b>🇨🇳 中文说明</b></a>
|
|
22
|
+
</p>
|
|
23
|
+
|
|
24
|
+
</div>
|
|
25
|
+
|
|
26
|
+
---
|
|
27
|
+
|
|
28
|
+
## ⚡ 插件概览
|
|
29
|
+
|
|
30
|
+
**`dsh-voice`** 为 **DeepSeek Harness** Web 界面带来极速语音交互体验。无论是按自然停顿切分的流式听写，还是带撤回保护的语音消息以及键盘/鼠标 Push-to-Talk 对讲，`dsh-voice` 凭借**多服务商自动故障转移备用链**确保您的录音万无一失。
|
|
31
|
+
|
|
32
|
+
```mermaid
|
|
33
|
+
graph LR
|
|
34
|
+
subgraph Client [前端 Web 浏览器]
|
|
35
|
+
Mic[🎙️ 听写麦克风] -->|VAD 停顿切分| Stream[音频数据切片]
|
|
36
|
+
Wave[🌊 语音消息] -->|长按 / 松开| PTT[Push-to-Talk]
|
|
37
|
+
end
|
|
38
|
+
|
|
39
|
+
subgraph Host [DSH 服务端 Host]
|
|
40
|
+
Stream --> FFMPEG[ffmpeg 16kHz 转码器]
|
|
41
|
+
PTT --> FFMPEG
|
|
42
|
+
FFMPEG --> Chain{备用链轮询}
|
|
43
|
+
|
|
44
|
+
Chain -->|首选优先级| P1[Deepgram / Nova-2]
|
|
45
|
+
Chain -.->|遭遇限流 / 429| P2[Groq / Whisper Turbo]
|
|
46
|
+
Chain -.->|异常故障时| P3[本地 whisper.cpp / 离线]
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
subgraph Output [输出目标]
|
|
50
|
+
P1 --> Composer[💬 聊天输入框]
|
|
51
|
+
P2 --> Composer
|
|
52
|
+
P3 --> Composer
|
|
53
|
+
end
|
|
54
|
+
|
|
55
|
+
style Client fill:#1e1e2e,stroke:#89b4fa,stroke-width:2px,color:#cdd6f4
|
|
56
|
+
style Host fill:#181825,stroke:#cba6f7,stroke-width:2px,color:#cdd6f4
|
|
57
|
+
style Output fill:#11111b,stroke:#a6e3a1,stroke-width:2px,color:#cdd6f4
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
---
|
|
61
|
+
|
|
62
|
+
## ✨ 核心亮点
|
|
63
|
+
|
|
64
|
+
* 🎙️ **VAD 智能流式听写**：说话停顿自动切句（`vadSilenceMs`，默认 700ms），文字实时追加至输入框。
|
|
65
|
+
* 🌊 **带撤回窗口的语音消息**：录制完整语音，转写后在倒计时（`autoSendMs`，默认 4000ms）结束后自动发送。
|
|
66
|
+
* 🎮 **沉浸式 Push-to-Talk 对讲**：
|
|
67
|
+
* **鼠标操作**：按住声波按钮开始录音，松开发送，拖离按钮取消。
|
|
68
|
+
* **键盘操作**：按住 <kbd>Ctrl</kbd> 无需鼠标即刻说话，按 <kbd>Esc</kbd> 放弃本次录音。
|
|
69
|
+
* ⚡ **浏览器本地零延迟同声字幕 (`browser`)**：Chrome Web Speech API 本地离线解析，说话同时浮动显示实时字幕。
|
|
70
|
+
* 🛡️ **多服务商自动容灾切换**：首选 API 额度耗尽或遭遇 429 限流时，毫秒级顺位切换备用引擎。
|
|
71
|
+
* 🧠 **上下文术语注入 (Context Glossary)**：自动从输入草稿中提取代码变量名与专业术语，引导 STT 模型精准转写专业词汇。
|
|
72
|
+
* 🎵 **内嵌音频播放器**：在输入框与录音浮层中随时试听和回放刚刚录制的原始音频片段。
|
|
73
|
+
* 🔇 **硬件降噪切换开关**：在插件设置中自由开关浏览器级降噪、回声消除与自动增益控制。
|
|
74
|
+
* 📊 **服务商延迟与健康监控看板**：在设置界面实时掌握每个语音引擎的延迟（毫秒）、成功率与调用状态。
|
|
75
|
+
* 🔒 **API 密钥安全隔离**：密钥由服务端 `ctx.credentials` 统一解析，绝不向浏览器前端泄漏。
|
|
76
|
+
* 🖥️ **本地 whisper.cpp 服务端直连**：自动管理 `whisper-server` 进程，结合 `ffmpeg` 实现实时音频转码。
|
|
77
|
+
* ⚡ **SenseVoice-ONNX / Sherpa-ONNX** *(0.8.11)*：超快速（~50–100ms）非自回归本地语音识别引擎，自动清除情绪/事件标签。支持 Sherpa-ONNX HTTP 和 OpenAI 兼容端点。
|
|
78
|
+
* 🌐 **实时音频流** *(0.8.11)*：低延迟 WebSocket 桥接（`/dsh-voice/realtime`），支持 OpenAI Realtime API 或本地 Sherpa-ONNX 流式传输。API 密钥安全保留在服务端。
|
|
79
|
+
|
|
80
|
+
---
|
|
81
|
+
|
|
82
|
+
## 🎮 四种语音输入方式
|
|
83
|
+
|
|
84
|
+
| 交互模式 | 触发手势 | 行为效果 |
|
|
85
|
+
|---|---|---|
|
|
86
|
+
| **流式听写** | 单击 <kbd>🎙️ 麦克风</kbd> | 按停顿切分语音 (`vadSilenceMs`)，文本实时录入输入框 |
|
|
87
|
+
| **语音消息** | 单击 <kbd>🌊 声波</kbd> | 持续录音至手动停止，转写后进入撤回倒计时 (`autoSendMs`) |
|
|
88
|
+
| **鼠标对讲** | 长按 <kbd>🌊 声波</kbd> | 按住录音；松开发送，光标拖出按钮区域取消 |
|
|
89
|
+
| **键盘对讲** | 按住 <kbd>Ctrl</kbd> | 免鼠标快捷 Push-to-Talk 录音；按 <kbd>Esc</kbd> 取消 |
|
|
90
|
+
|
|
91
|
+
---
|
|
92
|
+
|
|
93
|
+
## 🛠️ 服务商矩阵
|
|
94
|
+
|
|
95
|
+
| 服务商标识 | 对应引擎 | 默认模型 | 环境变量凭据 | 特性说明 |
|
|
96
|
+
|---|---|---|---|---|
|
|
97
|
+
| `browser` | Web Speech API | 浏览器原生引擎 | *无需密钥* | 零延迟同声字幕输出 |
|
|
98
|
+
| `deepgram` | Deepgram API | `nova-2` | `DEEPGRAM_API_KEY` | 极速高精云端转写 |
|
|
99
|
+
| `groq` | Groq Whisper | `whisper-large-v3-turbo` | `GROQ_API_KEY` | 毫秒级极速推理 |
|
|
100
|
+
| `hf` | HuggingFace Inference | `openai/whisper-large-v3` | `HF_TOKEN` | 经典高精度开源模型 |
|
|
101
|
+
| `local-whisper` | 本地 whisper.cpp | 启动参数指定 | *无需密钥* | 100% 离线私密运行 |
|
|
102
|
+
| `sensevoice` | SenseVoice-ONNX / Sherpa-ONNX | `SenseVoiceSmall` | *无需密钥* | 超快速（~50ms）本地非自回归 STT |
|
|
103
|
+
|
|
104
|
+
---
|
|
105
|
+
|
|
106
|
+
## 📦 安装指南
|
|
107
|
+
|
|
108
|
+
```bash
|
|
109
|
+
dsh plugin --profile web add @goodandready/dsh-voice
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
---
|
|
113
|
+
|
|
114
|
+
## 📄 开源协议
|
|
115
|
+
|
|
116
|
+
MIT © [GooDAnDReaDY](https://github.com/GooDAnDReaDY)
|
package/lib/chain.js
CHANGED
|
@@ -9,21 +9,36 @@ function normalizeError(err) {
|
|
|
9
9
|
/**
|
|
10
10
|
* @param order {string[]} порядок ключей провайдеров
|
|
11
11
|
* @param providers {Record<string, () => Promise<{ok, provider?, text?, reason?}>>}
|
|
12
|
+
* @param onAttempt {(provider: string, res: {ok: boolean, tookMs: number, reason?: string}) => void} опциональный колбэк статистики
|
|
12
13
|
* @returns {Promise<{provider: string, text: string, tookMs: number}>}
|
|
13
14
|
*/
|
|
14
|
-
export async function runChain(order, providers) {
|
|
15
|
+
export async function runChain(order, providers, onAttempt) {
|
|
15
16
|
const t0 = Date.now()
|
|
16
17
|
const keys = (Array.isArray(order) ? order : []).filter((k) => typeof providers[k] === 'function')
|
|
17
18
|
const errors = []
|
|
18
19
|
for (const key of keys) {
|
|
20
|
+
const startT = Date.now()
|
|
19
21
|
try {
|
|
20
22
|
const out = await providers[key]()
|
|
23
|
+
const tookMs = Date.now() - startT
|
|
21
24
|
if (out && out.ok) {
|
|
25
|
+
if (typeof onAttempt === 'function') {
|
|
26
|
+
onAttempt(out.provider || key, { ok: true, tookMs })
|
|
27
|
+
}
|
|
22
28
|
return { provider: out.provider || key, text: out.text, tookMs: Date.now() - t0 }
|
|
23
29
|
}
|
|
24
|
-
|
|
30
|
+
const reason = (out && out.reason) || 'unknown'
|
|
31
|
+
if (typeof onAttempt === 'function') {
|
|
32
|
+
onAttempt((out && out.provider) || key, { ok: false, tookMs, reason })
|
|
33
|
+
}
|
|
34
|
+
errors.push(`${(out && out.provider) || key}: ${reason}`)
|
|
25
35
|
} catch (err) {
|
|
26
|
-
|
|
36
|
+
const tookMs = Date.now() - startT
|
|
37
|
+
const normErr = normalizeError(err)
|
|
38
|
+
if (typeof onAttempt === 'function') {
|
|
39
|
+
onAttempt(key, { ok: false, tookMs, reason: normErr })
|
|
40
|
+
}
|
|
41
|
+
errors.push(`${key}: ${normErr}`)
|
|
27
42
|
}
|
|
28
43
|
}
|
|
29
44
|
throw new Error(`all providers failed (${errors.join('; ')})`)
|
package/lib/client.js
CHANGED
|
@@ -130,6 +130,36 @@ window.__ModuleLoader__.load({
|
|
|
130
130
|
'save': 'Save',
|
|
131
131
|
'saved': 'Saved ✓',
|
|
132
132
|
'openrouterWarning': 'OpenRouter has no /audio/transcriptions \u2014 use the openai-chat-audio template there',
|
|
133
|
+
'noiseSuppression': 'Hardware noise suppression',
|
|
134
|
+
'noiseSuppressionHint': 'Enable browser noise suppression, echo cancellation, and auto gain control',
|
|
135
|
+
'contextGlossary': 'Context glossary injection',
|
|
136
|
+
'contextGlossaryHint': 'Auto-extract code identifiers and terms from composer to improve STT accuracy',
|
|
137
|
+
'providerDashboard': 'Provider Latency & Health',
|
|
138
|
+
'avgLatency': 'Avg latency',
|
|
139
|
+
'successRate': 'Success',
|
|
140
|
+
'fast': 'Fast',
|
|
141
|
+
'normal': 'Normal',
|
|
142
|
+
'slow': 'Slow',
|
|
143
|
+
'error': 'Error',
|
|
144
|
+
'idle': 'No calls',
|
|
145
|
+
'play': 'Play',
|
|
146
|
+
'pause': 'Pause',
|
|
147
|
+
'listenBack': 'Listen back',
|
|
148
|
+
'lastRecording': 'Last voice note',
|
|
149
|
+
'sensevoiceHint': 'SenseVoice-ONNX / Sherpa-ONNX · ultra-fast local STT (~50ms)',
|
|
150
|
+
'sensevoiceEndpoint': 'SenseVoice: endpoint',
|
|
151
|
+
'sensevoiceEndpointHint': 'POST endpoint of sherpa-onnx or compatible server',
|
|
152
|
+
'sensevoiceBin': 'SenseVoice: binary',
|
|
153
|
+
'sensevoiceBinHint': 'used when autostart is on',
|
|
154
|
+
'sensevoiceModel': 'SenseVoice: model path',
|
|
155
|
+
'sensevoiceAutostart': 'Autostart the SenseVoice server',
|
|
156
|
+
'realtimeStreaming': 'Realtime audio streaming',
|
|
157
|
+
'realtimeStreamingHint': 'Low-latency streaming via WebSocket (OpenAI Realtime API / sherpa-onnx)',
|
|
158
|
+
'realtimeProvider': 'Realtime provider',
|
|
159
|
+
'realtimeModel': 'Realtime model',
|
|
160
|
+
'realtimeActive': 'Streaming live…',
|
|
161
|
+
'realtimeConnect': 'Start streaming',
|
|
162
|
+
'realtimeDisconnect': 'Stop streaming',
|
|
133
163
|
}
|
|
134
164
|
const ru = {
|
|
135
165
|
'saveFailed': 'Часть полей не сохранилась —',
|
|
@@ -239,6 +269,36 @@ window.__ModuleLoader__.load({
|
|
|
239
269
|
'save': 'Сохранить',
|
|
240
270
|
'saved': 'Сохранено ✓',
|
|
241
271
|
'openrouterWarning': 'У OpenRouter нет /audio/transcriptions \u2014 там нужен шаблон openai-chat-audio',
|
|
272
|
+
'noiseSuppression': 'Аппаратное шумоподавление',
|
|
273
|
+
'noiseSuppressionHint': 'Включить шумоподавление, эхоподавление и АРУ микрофона в браузере',
|
|
274
|
+
'contextGlossary': 'Контекстный словарь терминов',
|
|
275
|
+
'contextGlossaryHint': 'Авто-извлечение кода и терминов из композера для повышения точности STT',
|
|
276
|
+
'providerDashboard': 'Задержка и здоровье провайдеров',
|
|
277
|
+
'avgLatency': 'Ср. задержка',
|
|
278
|
+
'successRate': 'Успешность',
|
|
279
|
+
'fast': 'Быстро',
|
|
280
|
+
'normal': 'Норма',
|
|
281
|
+
'slow': 'Медленно',
|
|
282
|
+
'error': 'Сбой',
|
|
283
|
+
'idle': 'Нет вызовов',
|
|
284
|
+
'play': 'Слушать',
|
|
285
|
+
'pause': 'Пауза',
|
|
286
|
+
'listenBack': 'Прослушать запись',
|
|
287
|
+
'lastRecording': 'Последняя запись',
|
|
288
|
+
'sensevoiceHint': 'SenseVoice-ONNX / Sherpa-ONNX · сверхбыстрый локальный STT (~50мс)',
|
|
289
|
+
'sensevoiceEndpoint': 'SenseVoice: endpoint',
|
|
290
|
+
'sensevoiceEndpointHint': 'POST endpoint сервера sherpa-onnx или совместимого',
|
|
291
|
+
'sensevoiceBin': 'SenseVoice: бинарь',
|
|
292
|
+
'sensevoiceBinHint': 'используется при автозапуске',
|
|
293
|
+
'sensevoiceModel': 'SenseVoice: путь к модели',
|
|
294
|
+
'sensevoiceAutostart': 'Автозапуск сервера SenseVoice',
|
|
295
|
+
'realtimeStreaming': 'Потоковое аудио в реальном времени',
|
|
296
|
+
'realtimeStreamingHint': 'Низколатентный стриминг через WebSocket (OpenAI Realtime API / sherpa-onnx)',
|
|
297
|
+
'realtimeProvider': 'Провайдер реалтайма',
|
|
298
|
+
'realtimeModel': 'Модель реалтайма',
|
|
299
|
+
'realtimeActive': 'Стриминг идёт…',
|
|
300
|
+
'realtimeConnect': 'Начать стриминг',
|
|
301
|
+
'realtimeDisconnect': 'Остановить стриминг',
|
|
242
302
|
}
|
|
243
303
|
|
|
244
304
|
// Строки нужны и вне компонентов — в обработчиках записи, в подписях
|
|
@@ -259,7 +319,23 @@ window.__ModuleLoader__.load({
|
|
|
259
319
|
'.dvo-err{color:var(--dsw-alias-state-error-primary)}' +
|
|
260
320
|
'.dvo-count{font-variant-numeric:tabular-nums;font-size:13px;color:var(--dsw-alias-label-secondary)}' +
|
|
261
321
|
'.dvo-spin{animation:dvo-spin 1s linear infinite}' +
|
|
262
|
-
'@keyframes dvo-spin{from{transform:rotate(0)}to{transform:rotate(360deg)}}'
|
|
322
|
+
'@keyframes dvo-spin{from{transform:rotate(0)}to{transform:rotate(360deg)}}' +
|
|
323
|
+
'.dvo-btn-active{color:var(--dsw-alias-label-primary);border-color:var(--dsw-alias-label-primary);background:var(--dsw-alias-bg-layer-2)}' +
|
|
324
|
+
'.dvo-audio-wrap{display:flex;align-items:center;gap:8px;padding:2px 10px;background:var(--dsw-alias-bg-layer-2);border-radius:14px;border:1px solid var(--dsw-alias-border-l1)}' +
|
|
325
|
+
'.dvo-audio-play{display:flex;align-items:center;justify-content:center;width:26px;height:26px;border-radius:50%;background:var(--dsw-alias-label-primary);color:var(--dsw-alias-bg-layer-1);border:0;cursor:pointer;padding:0}' +
|
|
326
|
+
'.dvo-audio-play:hover{opacity:0.9}' +
|
|
327
|
+
'.dvo-audio-time{font-size:12px;font-variant-numeric:tabular-nums;color:var(--dsw-alias-label-secondary)}' +
|
|
328
|
+
'.dvo-dash{display:flex;flex-direction:column;gap:8px;margin-top:8px;background:var(--dsw-alias-bg-layer-2);border:1px solid var(--dsw-alias-border-l2);border-radius:10px;padding:12px}' +
|
|
329
|
+
'.dvo-dash-grid{display:grid;grid-template-columns:repeat(auto-fill,minmax(130px,1fr));gap:8px;margin-top:4px}' +
|
|
330
|
+
'.dvo-dash-item{display:flex;flex-direction:column;gap:4px;padding:8px 10px;background:var(--dsw-alias-bg-layer-3);border:1px solid var(--dsw-alias-border-l1);border-radius:8px}' +
|
|
331
|
+
'.dvo-dash-name{font-size:12px;font-weight:600;color:var(--dsw-alias-label-primary)}' +
|
|
332
|
+
'.dvo-dash-row{display:flex;justify-content:space-between;align-items:center;font-size:11px;color:var(--dsw-alias-label-secondary)}' +
|
|
333
|
+
'.dvo-badge{display:inline-flex;align-items:center;font-size:11px;padding:1px 6px;border-radius:6px;font-weight:600}' +
|
|
334
|
+
'.dvo-badge-fast{background:rgba(16,185,129,0.15);color:#10b981}' +
|
|
335
|
+
'.dvo-badge-norm{background:rgba(245,158,11,0.15);color:#f59e0b}' +
|
|
336
|
+
'.dvo-badge-slow{background:rgba(249,115,22,0.15);color:#f97316}' +
|
|
337
|
+
'.dvo-badge-err{background:rgba(239,68,68,0.15);color:#ef4444}' +
|
|
338
|
+
'.dvo-badge-idle{background:var(--dsw-alias-bg-layer-2);color:var(--dsw-alias-label-tertiary)}'
|
|
263
339
|
const cssId = 'dsh-voice/client.module.css'
|
|
264
340
|
if (typeof document !== 'undefined' && !document.querySelector('style[data-plugin-css="' + cssId + '"]')) {
|
|
265
341
|
const tag = document.createElement('style')
|
|
@@ -277,9 +353,11 @@ window.__ModuleLoader__.load({
|
|
|
277
353
|
rec: null,
|
|
278
354
|
levels: [],
|
|
279
355
|
pending: null, // {text, leftMs} — окно отмены режима message
|
|
356
|
+
lastNote: null, // {blob, url, mime, text} — последняя запись
|
|
357
|
+
showPlayer: false,
|
|
280
358
|
inputActions: null,
|
|
281
359
|
input: null,
|
|
282
|
-
settings: { vadSilenceMs: 700, autoSendMs: 4000, stream: false, streamChunkMs: 1200, vadAdapt: 0 },
|
|
360
|
+
settings: { vadSilenceMs: 700, autoSendMs: 4000, stream: false, streamChunkMs: 1200, vadAdapt: 0, noiseSuppression: true, contextGlossary: true },
|
|
283
361
|
listeners: new Set(),
|
|
284
362
|
notify() { this.listeners.forEach((l) => l()) },
|
|
285
363
|
set(patch) { Object.assign(this, patch); this.notify() },
|
|
@@ -294,6 +372,11 @@ window.__ModuleLoader__.load({
|
|
|
294
372
|
|
|
295
373
|
// ---------------------------------------------------------------- icons
|
|
296
374
|
const ic = { width: 18, height: 18, viewBox: '0 0 24 24', fill: 'none', stroke: 'currentColor', strokeWidth: 2, strokeLinecap: 'round', strokeLinejoin: 'round' }
|
|
375
|
+
const playIcon = () => React.createElement('svg', Object.assign({}, ic, { viewBox: '0 0 24 24' }),
|
|
376
|
+
React.createElement('polygon', { points: '6 4 20 12 6 20 6 4', fill: 'currentColor', stroke: 'none' }))
|
|
377
|
+
const pauseIcon = () => React.createElement('svg', Object.assign({}, ic, { viewBox: '0 0 24 24' }),
|
|
378
|
+
React.createElement('rect', { x: 6, y: 4, width: 4, height: 16, fill: 'currentColor', stroke: 'none' }),
|
|
379
|
+
React.createElement('rect', { x: 14, y: 4, width: 4, height: 16, fill: 'currentColor', stroke: 'none' }))
|
|
297
380
|
const micIcon = () => React.createElement('svg', ic,
|
|
298
381
|
React.createElement('path', { d: 'M12 1a3 3 0 0 0-3 3v8a3 3 0 0 0 6 0V4a3 3 0 0 0-3-3z' }),
|
|
299
382
|
React.createElement('path', { d: 'M19 10v2a7 7 0 0 1-14 0v-2' }),
|
|
@@ -329,12 +412,34 @@ window.__ModuleLoader__.load({
|
|
|
329
412
|
})
|
|
330
413
|
}
|
|
331
414
|
|
|
415
|
+
function extractContextKeywords() {
|
|
416
|
+
if (!voice.settings || voice.settings.contextGlossary === false) return []
|
|
417
|
+
const text = (voice.input && typeof voice.input.draft === 'string') ? voice.input.draft : ''
|
|
418
|
+
if (!text || text.length < 3) return []
|
|
419
|
+
const matches = text.match(/\b[A-Za-z_][A-Za-z0-9_]{2,29}\b/g) || []
|
|
420
|
+
const stop = new Set(['the', 'and', 'for', 'are', 'but', 'not', 'you', 'all', 'any', 'can', 'her', 'was', 'one', 'our', 'out', 'day', 'get', 'has', 'him', 'his', 'how', 'man', 'new', 'now', 'old', 'see', 'two', 'way', 'who', 'boy', 'did', 'its', 'let', 'put', 'say', 'she', 'too', 'use'])
|
|
421
|
+
const words = []
|
|
422
|
+
const seen = new Set()
|
|
423
|
+
for (const m of matches) {
|
|
424
|
+
const lower = m.toLowerCase()
|
|
425
|
+
if (!stop.has(lower) && !seen.has(lower)) {
|
|
426
|
+
seen.add(lower)
|
|
427
|
+
words.push(m)
|
|
428
|
+
if (words.length >= 30) break
|
|
429
|
+
}
|
|
430
|
+
}
|
|
431
|
+
return words
|
|
432
|
+
}
|
|
433
|
+
|
|
332
434
|
async function sendAudio(blob, mime, mode) {
|
|
333
435
|
const dataBase64 = await blobToBase64(blob)
|
|
436
|
+
const payload = { dataBase64, mimeType: mime, mode }
|
|
437
|
+
const contextWords = extractContextKeywords()
|
|
438
|
+
if (contextWords && contextWords.length > 0) payload.contextWords = contextWords
|
|
334
439
|
const res = await fetch('/dsh-voice/transcribe', {
|
|
335
440
|
method: 'POST',
|
|
336
441
|
headers: { 'content-type': 'application/json' },
|
|
337
|
-
body: JSON.stringify(
|
|
442
|
+
body: JSON.stringify(payload),
|
|
338
443
|
})
|
|
339
444
|
let parsed = null
|
|
340
445
|
try { parsed = await res.json() } catch (e) { /* не json */ }
|
|
@@ -554,8 +659,13 @@ window.__ModuleLoader__.load({
|
|
|
554
659
|
if (typeof navigator === 'undefined' || !navigator.mediaDevices || !navigator.mediaDevices.getUserMedia) {
|
|
555
660
|
throw new Error(t('micUnavailable'))
|
|
556
661
|
}
|
|
557
|
-
|
|
558
|
-
const audio = {
|
|
662
|
+
const ns = voice.settings.noiseSuppression !== false
|
|
663
|
+
const audio = {
|
|
664
|
+
channelCount: 1,
|
|
665
|
+
echoCancellation: ns,
|
|
666
|
+
noiseSuppression: ns,
|
|
667
|
+
autoGainControl: ns,
|
|
668
|
+
}
|
|
559
669
|
if (voice.settings.micDeviceId) audio.deviceId = { exact: voice.settings.micDeviceId }
|
|
560
670
|
const stream = await navigator.mediaDevices.getUserMedia({ audio })
|
|
561
671
|
let mimeType = 'audio/webm;codecs=opus'
|
|
@@ -809,6 +919,12 @@ window.__ModuleLoader__.load({
|
|
|
809
919
|
teardown(rec)
|
|
810
920
|
voice.rec = null
|
|
811
921
|
if (blob.size < 1200) { voice.set({ phase: 'idle' }); return }
|
|
922
|
+
if (voice.lastNote && voice.lastNote.url) {
|
|
923
|
+
try { URL.revokeObjectURL(voice.lastNote.url) } catch (e) { /* игнор */ }
|
|
924
|
+
}
|
|
925
|
+
let noteUrl = ''
|
|
926
|
+
try { noteUrl = URL.createObjectURL(blob) } catch (e) { /* игнор */ }
|
|
927
|
+
voice.lastNote = { blob, url: noteUrl, mime: rec.mime, text: '' }
|
|
812
928
|
voice.set({ phase: 'processing' })
|
|
813
929
|
try {
|
|
814
930
|
const out = await sendAudio(blob, rec.mime, mode)
|
|
@@ -816,8 +932,9 @@ window.__ModuleLoader__.load({
|
|
|
816
932
|
const text = out.text || ''
|
|
817
933
|
if (!text) { voice.set({ phase: 'error', error: t('nothingHeard') }); return }
|
|
818
934
|
appendDraft(text)
|
|
935
|
+
if (voice.lastNote) voice.lastNote.text = text
|
|
819
936
|
if (mode === 'message') {
|
|
820
|
-
voice.set({ phase: 'pending', pending: { text: text, leftMs: voice.settings.autoSendMs } })
|
|
937
|
+
voice.set({ phase: 'pending', pending: { text: text, leftMs: voice.settings.autoSendMs, audioUrl: noteUrl } })
|
|
821
938
|
} else {
|
|
822
939
|
voice.set({ phase: 'idle' })
|
|
823
940
|
}
|
|
@@ -899,12 +1016,21 @@ window.__ModuleLoader__.load({
|
|
|
899
1016
|
onPointerUp: () => endHold(false),
|
|
900
1017
|
onPointerLeave: () => { if (hold.armed) endHold(true) },
|
|
901
1018
|
}, waveIcon()),
|
|
1019
|
+
voice.lastNote && voice.lastNote.url
|
|
1020
|
+
? React.createElement('button', {
|
|
1021
|
+
type: 'button', className: 'dvo-btn' + (voice.showPlayer ? ' dvo-btn-active' : ''),
|
|
1022
|
+
title: t('listenBack'),
|
|
1023
|
+
onClick: () => voice.set({ showPlayer: !voice.showPlayer }),
|
|
1024
|
+
}, playIcon())
|
|
1025
|
+
: null,
|
|
902
1026
|
)
|
|
903
1027
|
}
|
|
904
1028
|
|
|
905
1029
|
function RecordPill(props) {
|
|
906
1030
|
const v = useVoice()
|
|
907
1031
|
const canvasRef = React.useRef(null)
|
|
1032
|
+
const audioRef = React.useRef(null)
|
|
1033
|
+
const [isPlaying, setIsPlaying] = React.useState(false)
|
|
908
1034
|
voice.inputActions = props.inputActions
|
|
909
1035
|
voice.input = props.input
|
|
910
1036
|
const ctx = props.ctx
|
|
@@ -1002,7 +1128,39 @@ window.__ModuleLoader__.load({
|
|
|
1002
1128
|
return () => dispose()
|
|
1003
1129
|
}, [v.phase])
|
|
1004
1130
|
|
|
1005
|
-
if (v.phase === 'idle')
|
|
1131
|
+
if (v.phase === 'idle') {
|
|
1132
|
+
if (!voice.showPlayer || !voice.lastNote || !voice.lastNote.url) return null
|
|
1133
|
+
return React.createElement('div', { className: 'dvo-pill' },
|
|
1134
|
+
React.createElement('div', { className: 'dvo-audio-wrap' },
|
|
1135
|
+
React.createElement('button', {
|
|
1136
|
+
type: 'button', className: 'dvo-audio-play',
|
|
1137
|
+
title: isPlaying ? t('pause') : t('play'),
|
|
1138
|
+
onClick: () => {
|
|
1139
|
+
const el = audioRef.current
|
|
1140
|
+
if (!el) return
|
|
1141
|
+
if (el.paused) { el.play().catch(() => {}); setIsPlaying(true) }
|
|
1142
|
+
else { el.pause(); setIsPlaying(false) }
|
|
1143
|
+
},
|
|
1144
|
+
}, isPlaying ? pauseIcon() : playIcon()),
|
|
1145
|
+
React.createElement('audio', {
|
|
1146
|
+
ref: audioRef, src: voice.lastNote.url,
|
|
1147
|
+
onEnded: () => setIsPlaying(false),
|
|
1148
|
+
onPause: () => setIsPlaying(false),
|
|
1149
|
+
onPlay: () => setIsPlaying(true),
|
|
1150
|
+
}),
|
|
1151
|
+
React.createElement('span', { className: 'dvo-audio-time' }, t('lastRecording')),
|
|
1152
|
+
),
|
|
1153
|
+
React.createElement('span', { className: 'dvo-status' }, voice.lastNote.text || ''),
|
|
1154
|
+
React.createElement('button', {
|
|
1155
|
+
type: 'button', className: 'dvo-pbtn', title: t('hide'),
|
|
1156
|
+
onClick: () => {
|
|
1157
|
+
if (audioRef.current) audioRef.current.pause()
|
|
1158
|
+
setIsPlaying(false)
|
|
1159
|
+
voice.set({ showPlayer: false })
|
|
1160
|
+
},
|
|
1161
|
+
}, xIcon()),
|
|
1162
|
+
)
|
|
1163
|
+
}
|
|
1006
1164
|
|
|
1007
1165
|
if (v.phase === 'recording') {
|
|
1008
1166
|
const inBrowser = !!voice.browser
|
|
@@ -1049,6 +1207,27 @@ window.__ModuleLoader__.load({
|
|
|
1049
1207
|
)
|
|
1050
1208
|
}
|
|
1051
1209
|
return React.createElement('div', { className: 'dvo-pill' },
|
|
1210
|
+
voice.lastNote && voice.lastNote.url
|
|
1211
|
+
? React.createElement('div', { className: 'dvo-audio-wrap' },
|
|
1212
|
+
React.createElement('button', {
|
|
1213
|
+
type: 'button', className: 'dvo-audio-play',
|
|
1214
|
+
title: isPlaying ? t('pause') : t('play'),
|
|
1215
|
+
onClick: () => {
|
|
1216
|
+
const el = audioRef.current
|
|
1217
|
+
if (!el) return
|
|
1218
|
+
if (el.paused) { el.play().catch(() => {}); setIsPlaying(true) }
|
|
1219
|
+
else { el.pause(); setIsPlaying(false) }
|
|
1220
|
+
},
|
|
1221
|
+
}, isPlaying ? pauseIcon() : playIcon()),
|
|
1222
|
+
React.createElement('audio', {
|
|
1223
|
+
ref: audioRef, src: voice.lastNote.url,
|
|
1224
|
+
onEnded: () => setIsPlaying(false),
|
|
1225
|
+
onPause: () => setIsPlaying(false),
|
|
1226
|
+
onPlay: () => setIsPlaying(true),
|
|
1227
|
+
}),
|
|
1228
|
+
React.createElement('span', { className: 'dvo-audio-time' }, t('listenBack')),
|
|
1229
|
+
)
|
|
1230
|
+
: null,
|
|
1052
1231
|
React.createElement('span', { className: 'dvo-status' }, t('sendingIn')),
|
|
1053
1232
|
React.createElement('span', { className: 'dvo-count' }, left + t('secondsShort')),
|
|
1054
1233
|
React.createElement('button', { type: 'button', className: 'dvo-pbtn', title: t('keepPending'), onClick: keepPending }, xIcon()),
|
|
@@ -1092,6 +1271,8 @@ window.__ModuleLoader__.load({
|
|
|
1092
1271
|
bargeIn: !!(data && data.bargeIn),
|
|
1093
1272
|
polishSend: !!(data && data.modes && data.modes.message && data.modes.message.polishSend),
|
|
1094
1273
|
sessionCommands: !!(data && data.modes && data.modes.message && data.modes.message.sessionCommands),
|
|
1274
|
+
noiseSuppression: data && data.noiseSuppression !== false,
|
|
1275
|
+
contextGlossary: data && data.contextGlossary !== false,
|
|
1095
1276
|
})
|
|
1096
1277
|
})
|
|
1097
1278
|
.catch(() => { /* без подсказки хоста клавиши просто не будет */ })
|
|
@@ -1118,7 +1299,7 @@ window.__ModuleLoader__.load({
|
|
|
1118
1299
|
|
|
1119
1300
|
// ------------------------------------------------------- settings page
|
|
1120
1301
|
const BUILTIN = [
|
|
1121
|
-
'browser', 'deepgram', 'groq', 'hf', 'local-whisper',
|
|
1302
|
+
'browser', 'deepgram', 'groq', 'hf', 'local-whisper', 'sensevoice',
|
|
1122
1303
|
// Заготовки: адрес и модель уже прописаны на хосте, нужен только ключ.
|
|
1123
1304
|
'openai', 'siliconflow', 'deepinfra', 'fireworks', 'mistral', 'openrouter',
|
|
1124
1305
|
]
|
|
@@ -1133,6 +1314,7 @@ window.__ModuleLoader__.load({
|
|
|
1133
1314
|
openrouter: t('openrouterHint'),
|
|
1134
1315
|
deepgram: 'nova-2', groq: 'whisper-large-v3-turbo',
|
|
1135
1316
|
hf: 'openai/whisper-large-v3', 'local-whisper': t('localHint'),
|
|
1317
|
+
sensevoice: t('sensevoiceHint'),
|
|
1136
1318
|
}
|
|
1137
1319
|
const LANGS = ['auto', 'ru', 'en', 'uk', 'de']
|
|
1138
1320
|
|
|
@@ -1362,9 +1544,25 @@ window.__ModuleLoader__.load({
|
|
|
1362
1544
|
polishSend: !!(value && value.message && value.message.polishSend),
|
|
1363
1545
|
sessionCommands: !!(value && value.message && value.message.sessionCommands),
|
|
1364
1546
|
polishBaseUrl: String((snap && snap.polishBaseUrl) || ''),
|
|
1547
|
+
noiseSuppression: value.noiseSuppression !== false,
|
|
1548
|
+
contextGlossary: value.contextGlossary !== false,
|
|
1365
1549
|
})
|
|
1366
1550
|
}, [ready, value, snap])
|
|
1367
1551
|
|
|
1552
|
+
const [statsData, setStatsData] = React.useState({})
|
|
1553
|
+
React.useEffect(() => {
|
|
1554
|
+
let alive = true
|
|
1555
|
+
fetch('/dsh-voice/status', { cache: 'no-store' })
|
|
1556
|
+
.then((r) => r.json())
|
|
1557
|
+
.then((data) => {
|
|
1558
|
+
if (alive && data && data.providerStats) {
|
|
1559
|
+
setStatsData(data.providerStats)
|
|
1560
|
+
}
|
|
1561
|
+
})
|
|
1562
|
+
.catch(() => {})
|
|
1563
|
+
return () => { alive = false }
|
|
1564
|
+
}, [])
|
|
1565
|
+
|
|
1368
1566
|
if (!ready) {
|
|
1369
1567
|
const waiting = !snap || snap.status === 'loading'
|
|
1370
1568
|
return React.createElement('div', { className: 'dvs-wrap' },
|
|
@@ -1412,6 +1610,8 @@ window.__ModuleLoader__.load({
|
|
|
1412
1610
|
stream: !!(draft.dictation && draft.dictation.stream),
|
|
1413
1611
|
streamChunkMs: Number(draft.dictation && draft.dictation.streamChunkMs) || 1200,
|
|
1414
1612
|
vadAdapt: Number(draft.dictation && draft.dictation.vadAdapt) || 0,
|
|
1613
|
+
noiseSuppression: draft.noiseSuppression !== false,
|
|
1614
|
+
contextGlossary: draft.contextGlossary !== false,
|
|
1415
1615
|
})
|
|
1416
1616
|
// Композер держит обработчик клавиши: пусть перечитает настройку,
|
|
1417
1617
|
// иначе новая клавиша заработает только после перезагрузки страницы.
|
|
@@ -1582,6 +1782,16 @@ window.__ModuleLoader__.load({
|
|
|
1582
1782
|
onChange: (e) => setTop('voiceCommands', e.target.checked),
|
|
1583
1783
|
})),
|
|
1584
1784
|
micField(),
|
|
1785
|
+
React.createElement('label', { className: 'dvs-field', title: t('noiseSuppressionHint') }, t('noiseSuppression'),
|
|
1786
|
+
React.createElement('input', {
|
|
1787
|
+
type: 'checkbox', checked: draft ? draft.noiseSuppression !== false : true, disabled: !writable,
|
|
1788
|
+
onChange: (e) => setTop('noiseSuppression', e.target.checked),
|
|
1789
|
+
})),
|
|
1790
|
+
React.createElement('label', { className: 'dvs-field', title: t('contextGlossaryHint') }, t('contextGlossary'),
|
|
1791
|
+
React.createElement('input', {
|
|
1792
|
+
type: 'checkbox', checked: draft ? draft.contextGlossary !== false : true, disabled: !writable,
|
|
1793
|
+
onChange: (e) => setTop('contextGlossary', e.target.checked),
|
|
1794
|
+
})),
|
|
1585
1795
|
React.createElement('label', { className: 'dvs-field' }, t('vocabulary'),
|
|
1586
1796
|
React.createElement('textarea', {
|
|
1587
1797
|
rows: 3, disabled: !writable,
|
|
@@ -1604,6 +1814,78 @@ window.__ModuleLoader__.load({
|
|
|
1604
1814
|
onChange: (e) => setTop('polishKeyEnv', e.target.value),
|
|
1605
1815
|
})),
|
|
1606
1816
|
),
|
|
1817
|
+
React.createElement('div', { className: 'dvs-block' },
|
|
1818
|
+
React.createElement('div', { className: 'dvs-h' }, 'SenseVoice-ONNX / Sherpa-ONNX'),
|
|
1819
|
+
React.createElement('div', { className: 'dvs-sub' }, t('sensevoiceHint')),
|
|
1820
|
+
textField('sensevoiceUrl', t('sensevoiceEndpoint'), t('sensevoiceEndpointHint')),
|
|
1821
|
+
textField('sensevoiceBin', t('sensevoiceBin'), t('sensevoiceBinHint')),
|
|
1822
|
+
textField('sensevoiceModel', t('sensevoiceModel'), t('sensevoiceBinHint')),
|
|
1823
|
+
React.createElement('label', { className: 'dvs-field' }, t('sensevoiceAutostart'),
|
|
1824
|
+
React.createElement('input', {
|
|
1825
|
+
type: 'checkbox', checked: !!(draft && draft.sensevoiceAutostart), disabled: !writable,
|
|
1826
|
+
onChange: (e) => setTop('sensevoiceAutostart', e.target.checked),
|
|
1827
|
+
})),
|
|
1828
|
+
),
|
|
1829
|
+
React.createElement('div', { className: 'dvs-block' },
|
|
1830
|
+
React.createElement('div', { className: 'dvs-h' }, t('realtimeStreaming')),
|
|
1831
|
+
React.createElement('div', { className: 'dvs-sub' }, t('realtimeStreamingHint')),
|
|
1832
|
+
React.createElement('label', { className: 'dvs-field' }, t('realtimeStreaming'),
|
|
1833
|
+
React.createElement('input', {
|
|
1834
|
+
type: 'checkbox', checked: !!(draft && draft.realtimeStreaming), disabled: !writable,
|
|
1835
|
+
onChange: (e) => setTop('realtimeStreaming', e.target.checked),
|
|
1836
|
+
})),
|
|
1837
|
+
React.createElement('label', { className: 'dvs-field' }, t('realtimeProvider'),
|
|
1838
|
+
React.createElement('select', {
|
|
1839
|
+
value: String((draft && draft.realtimeProvider) || 'openai'), disabled: !writable,
|
|
1840
|
+
onChange: (e) => setTop('realtimeProvider', e.target.value),
|
|
1841
|
+
},
|
|
1842
|
+
React.createElement('option', { value: 'openai' }, 'OpenAI Realtime API'),
|
|
1843
|
+
React.createElement('option', { value: 'sherpa-onnx' }, 'Sherpa-ONNX (local)'),
|
|
1844
|
+
)),
|
|
1845
|
+
textField('realtimeModel', t('realtimeModel'), ''),
|
|
1846
|
+
),
|
|
1847
|
+
React.createElement('div', { className: 'dvs-block' },
|
|
1848
|
+
React.createElement('div', { className: 'dvs-h' }, t('providerDashboard')),
|
|
1849
|
+
React.createElement('div', { className: 'dvo-dash' },
|
|
1850
|
+
React.createElement('div', { className: 'dvo-dash-grid' },
|
|
1851
|
+
Object.keys(statsData).length === 0
|
|
1852
|
+
? React.createElement('div', { className: 'dvs-sub' }, t('idle'))
|
|
1853
|
+
: Object.entries(statsData).map(([key, item]) => {
|
|
1854
|
+
const avg = item.avgTookMs || 0
|
|
1855
|
+
const hasErrors = item.failures > 0
|
|
1856
|
+
let badgeClass = 'dvo-badge-idle'
|
|
1857
|
+
let badgeText = t('idle')
|
|
1858
|
+
if (item.attempts > 0) {
|
|
1859
|
+
if (hasErrors && item.successes === 0) {
|
|
1860
|
+
badgeClass = 'dvo-badge-err'
|
|
1861
|
+
badgeText = t('error')
|
|
1862
|
+
} else if (avg > 0 && avg < 400) {
|
|
1863
|
+
badgeClass = 'dvo-badge-fast'
|
|
1864
|
+
badgeText = avg + 'ms · ' + t('fast')
|
|
1865
|
+
} else if (avg >= 400 && avg <= 1500) {
|
|
1866
|
+
badgeClass = 'dvo-badge-norm'
|
|
1867
|
+
badgeText = avg + 'ms · ' + t('normal')
|
|
1868
|
+
} else {
|
|
1869
|
+
badgeClass = 'dvo-badge-slow'
|
|
1870
|
+
badgeText = avg + 'ms · ' + t('slow')
|
|
1871
|
+
}
|
|
1872
|
+
}
|
|
1873
|
+
const rate = item.attempts > 0 ? Math.round((item.successes / item.attempts) * 100) : 100
|
|
1874
|
+
return React.createElement('div', { key, className: 'dvo-dash-item', title: item.lastError ? `Error: ${item.lastError}` : '' },
|
|
1875
|
+
React.createElement('div', { className: 'dvo-dash-name' }, key),
|
|
1876
|
+
React.createElement('div', { className: 'dvo-dash-row' },
|
|
1877
|
+
React.createElement('span', { className: 'dvo-badge ' + badgeClass }, badgeText),
|
|
1878
|
+
React.createElement('span', null, rate + '%'),
|
|
1879
|
+
),
|
|
1880
|
+
React.createElement('div', { className: 'dvo-dash-row' },
|
|
1881
|
+
React.createElement('span', null, `${item.successes}/${item.attempts}`),
|
|
1882
|
+
React.createElement('span', null, t('successRate')),
|
|
1883
|
+
),
|
|
1884
|
+
)
|
|
1885
|
+
})
|
|
1886
|
+
)
|
|
1887
|
+
)
|
|
1888
|
+
),
|
|
1607
1889
|
React.createElement('div', { className: 'dvs-row' },
|
|
1608
1890
|
React.createElement('button', { type: 'button', className: 'dvs-save', disabled: !writable, onClick: save }, t('save')),
|
|
1609
1891
|
saved ? React.createElement('span', { className: 'dvs-ok' }, t('saved')) : null,
|