@goodandready/dsh-voice 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 dsh-voice contributors
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md ADDED
@@ -0,0 +1,110 @@
1
+ # dsh-voice
2
+
3
+ Voice input for the DeepSeek Harness Web GUI, in two modes, each with its own
4
+ provider fallback chain.
5
+
6
+ **Dictation** — press the mic, talk, and the text lands in the composer as you
7
+ go: speech is cut into phrases on silence and each phrase is transcribed on its
8
+ own. Press again to stop; sending stays in your hands.
9
+
10
+ **Voice message** — press the wave button, record, press again. The transcript
11
+ is inserted and sent to the agent after a short cancel window.
12
+
13
+ Both modes fall back across providers, so one outage or rate limit does not
14
+ lose your recording. API keys never reach the browser: audio is posted to the
15
+ plugin's own route and the host talks to the providers.
16
+
17
+ ## Install
18
+
19
+ ```bash
20
+ # From npm:
21
+ dsh plugin --profile web add @goodandready/dsh-voice
22
+
23
+ # Locally from a checkout:
24
+ dsh plugin --profile web add file:/path/to/dsh-voice
25
+ ```
26
+
27
+ Restart the Web UI afterwards, then hard-refresh the browser.
28
+
29
+ ## Providers
30
+
31
+ | Key | Service | Default model | Credential |
32
+ |---|---|---|---|
33
+ | `deepgram` | Deepgram | `nova-2` | `DEEPGRAM_API_KEY` |
34
+ | `groq` | Groq | `whisper-large-v3-turbo` | `GROQ_API_KEY` |
35
+ | `hf` | HuggingFace Inference | `openai/whisper-large-v3` | `HF_TOKEN` |
36
+ | `local-whisper` | local whisper.cpp server | model given at server start | none, fully offline |
37
+
38
+ Keys are read through the DSH credentials service (Settings → Credentials, or
39
+ `$DSH_HOME/.credentials.yaml`), falling back to the process environment. A
40
+ provider without a key is skipped, not fatal.
41
+
42
+ ## Configure (Web GUI)
43
+
44
+ Settings → **Голос** (Voice) has three blocks:
45
+
46
+ - **Dictation** — fallback chain (provider + optional model per row, order is
47
+ the order of attempts), language, and the silence threshold that ends a
48
+ phrase (`vadSilenceMs`, default 700 ms).
49
+ - **Voice message** — its own independent chain, language, and the cancel
50
+ window before the message is sent (`autoSendMs`, default 4000 ms).
51
+ - **General** — local whisper endpoint, binary, model, autostart.
52
+
53
+ Speed matters for dictation and accuracy for messages, which is why the chains
54
+ are separate: a sensible pair is Deepgram → Groq → local for dictation and
55
+ Groq → HuggingFace → local for messages.
56
+
57
+ ## Local whisper.cpp
58
+
59
+ The local provider needs a running [whisper.cpp](https://github.com/ggerganov/whisper.cpp)
60
+ server:
61
+
62
+ ```bash
63
+ whisper-server -m /path/to/ggml-medium-q8_0.bin --host 127.0.0.1 --port 8001
64
+ ```
65
+
66
+ Set `whisperModel` (and `whisperBin` if it is not in `PATH`) and the plugin
67
+ launches the server itself when `autoStart` is on. While `whisperModel` is
68
+ empty, autostart stays off.
69
+
70
+ **ffmpeg is required for this provider.** whisper.cpp accepts WAV only and
71
+ rejects the webm/opus the browser records, so the host converts each recording
72
+ to 16 kHz mono WAV before forwarding it. Point `ffmpegBin` at your binary if it
73
+ is not in `PATH`.
74
+
75
+ ## Tool
76
+
77
+ The plugin also registers `transcribe_audio(file_path, language?)` for the
78
+ agent, using the voice-message chain. Useful for recordings and interviews that
79
+ are already files on disk.
80
+
81
+ ## Routes
82
+
83
+ | Route | Purpose |
84
+ |---|---|
85
+ | `POST /dsh-voice/transcribe` | `{dataBase64, mimeType, mode}` → `{ok, text, provider, tookMs}` |
86
+ | `GET /dsh-voice/status` | whisper server state and the effective chains |
87
+
88
+ ## Structure
89
+
90
+ ```
91
+ lib/index.js host: config, routes, transcribe_audio, whisper autostart
92
+ lib/providers.js the four providers, pure functions (network injected)
93
+ lib/chain.js fallback walk over a chain
94
+ lib/wav.js webm/opus → WAV 16 kHz mono via ffmpeg
95
+ lib/client.js browser: composer buttons, recording, settings page
96
+ test/ node --test units for the chain and the providers
97
+ ```
98
+
99
+ Run the tests with `npm test` (no dependencies, Node's built-in runner).
100
+
101
+ ## Requirements
102
+
103
+ - DeepSeek Harness with the Web GUI
104
+ - Node 20+
105
+ - ffmpeg, for the local whisper provider
106
+ - a microphone reachable from the browser (HTTPS or localhost)
107
+
108
+ ## License
109
+
110
+ MIT
@@ -0,0 +1,19 @@
1
+ # dsh-voice bundle layer: applied automatically when the package is installed
2
+ # as a profile bundle (package.json declares dsh.bundle.patch).
3
+ #
4
+ # `name` must stay the full npm package name: the client-modules registry
5
+ # resolves the browser bundle by the loader entry name, so a shortened name
6
+ # leaves the UI half silently out of window.__DSH_BOOT__.
7
+ #
8
+ # Override any field in the profile's own cordis.patch.yml, e.g. to point the
9
+ # plugin at a local whisper.cpp build:
10
+ #
11
+ # - id: dsh-voice
12
+ # config:
13
+ # whisperBin: /opt/whisper.cpp/build/bin/whisper-server
14
+ # whisperModel: /opt/whisper.cpp/models/ggml-medium-q8_0.bin
15
+ #
16
+ - insert:
17
+ - id: dsh-voice
18
+ name: '@goodandready/dsh-voice'
19
+ config: {}
package/lib/chain.js ADDED
@@ -0,0 +1,30 @@
1
+ // Перебор цепочки провайдеров с фоллбеком. Без сети и без cordis — чистая
2
+ // функция, чтобы поведение при отказах можно было проверить юнит-тестом.
3
+
4
+ function normalizeError(err) {
5
+ const cause = err?.cause?.message || err?.message || String(err)
6
+ return cause.slice(0, 200)
7
+ }
8
+
9
+ /**
10
+ * @param order {string[]} порядок ключей провайдеров
11
+ * @param providers {Record<string, () => Promise<{ok, provider?, text?, reason?}>>}
12
+ * @returns {Promise<{provider: string, text: string, tookMs: number}>}
13
+ */
14
+ export async function runChain(order, providers) {
15
+ const t0 = Date.now()
16
+ const keys = (Array.isArray(order) ? order : []).filter((k) => typeof providers[k] === 'function')
17
+ const errors = []
18
+ for (const key of keys) {
19
+ try {
20
+ const out = await providers[key]()
21
+ if (out && out.ok) {
22
+ return { provider: out.provider || key, text: out.text, tookMs: Date.now() - t0 }
23
+ }
24
+ errors.push(`${(out && out.provider) || key}: ${(out && out.reason) || 'unknown'}`)
25
+ } catch (err) {
26
+ errors.push(`${key}: ${normalizeError(err)}`)
27
+ }
28
+ }
29
+ throw new Error(`all providers failed (${errors.join('; ')})`)
30
+ }
package/lib/client.js ADDED
@@ -0,0 +1,604 @@
1
+ // dsh-voice — клиентская половина (браузер).
2
+ //
3
+ // Две кнопки в conversation.input.right:
4
+ // микрофон — диктовка: речь режется по паузам, каждый кусок распознаётся и
5
+ // дописывается в строку ввода; отправка остаётся за пользователем;
6
+ // волна — голосовое: одна запись целиком, после распознавания текст
7
+ // уходит агенту по истечении окна отмены.
8
+ //
9
+ // Полоска записи живёт в conversation.input.dock, страница настроек — в
10
+ // settings.section.
11
+
12
+ window.__ModuleLoader__.load({
13
+ id: '@goodandready/dsh-voice',
14
+ factory: (require) => {
15
+ var module = { exports: {} }
16
+ var exports = module.exports
17
+ Object.defineProperty(exports, Symbol.toStringTag, { value: 'Module' })
18
+ let React = require('react')
19
+
20
+ const NS = 'dsh-voice'
21
+
22
+ // ------------------------------------------------------------------ css
23
+ const CSS =
24
+ '.dvo-btn{display:flex;align-items:center;justify-content:center;width:30px;height:30px;border-radius:50%;border:1px solid var(--dsw-alias-border-l1);background:transparent;color:var(--dsw-alias-label-secondary);cursor:pointer;padding:0;box-sizing:border-box}' +
25
+ '.dvo-btn:hover{color:var(--dsw-alias-label-primary);border-color:var(--dsw-alias-border-l2)}' +
26
+ '.dvo-btn[data-err="1"]{color:var(--dsw-alias-state-error-primary);border-color:var(--dsw-alias-state-error-primary)}' +
27
+ '.dvo-pill{display:flex;align-items:center;gap:10px;height:52px;border-radius:26px;background:var(--dsw-alias-bg-layer-1);border:1px solid var(--dsw-alias-border-l1);padding:0 14px;width:100%;max-width:720px;margin:0 auto;box-shadow:0 8px 24px rgba(0,0,0,.18);box-sizing:border-box}' +
28
+ '.dvo-pbtn{display:flex;align-items:center;justify-content:center;width:34px;height:34px;border-radius:50%;cursor:pointer;padding:0;flex:none;box-sizing:border-box;border:1px solid var(--dsw-alias-border-l2);background:transparent;color:var(--dsw-alias-label-primary)}' +
29
+ '.dvo-pbtn:hover{background:var(--dsw-alias-bg-layer-2)}' +
30
+ '.dvo-wave{flex:1;height:40px;width:100%;color:var(--dsw-alias-label-primary)}' +
31
+ '.dvo-status{display:flex;align-items:center;gap:8px;color:var(--dsw-alias-label-secondary);font-size:13px;flex:1;overflow:hidden;text-overflow:ellipsis;white-space:nowrap}' +
32
+ '.dvo-err{color:var(--dsw-alias-state-error-primary)}' +
33
+ '.dvo-count{font-variant-numeric:tabular-nums;font-size:13px;color:var(--dsw-alias-label-secondary)}' +
34
+ '.dvo-spin{animation:dvo-spin 1s linear infinite}' +
35
+ '@keyframes dvo-spin{from{transform:rotate(0)}to{transform:rotate(360deg)}}'
36
+ const cssId = 'dsh-voice/client.module.css'
37
+ if (typeof document !== 'undefined' && !document.querySelector('style[data-plugin-css="' + cssId + '"]')) {
38
+ const tag = document.createElement('style')
39
+ tag.textContent = CSS
40
+ tag.setAttribute('data-plugin', 'dsh-voice')
41
+ tag.dataset.pluginCss = cssId
42
+ document.head.appendChild(tag)
43
+ }
44
+
45
+ // ---------------------------------------------------------------- store
46
+ const voice = {
47
+ phase: 'idle', // idle | recording | processing | pending | error
48
+ mode: 'dictation', // dictation | message
49
+ error: '',
50
+ rec: null,
51
+ levels: [],
52
+ pending: null, // {text, leftMs} — окно отмены режима message
53
+ inputActions: null,
54
+ input: null,
55
+ settings: { vadSilenceMs: 700, autoSendMs: 4000 },
56
+ listeners: new Set(),
57
+ notify() { this.listeners.forEach((l) => l()) },
58
+ set(patch) { Object.assign(this, patch); this.notify() },
59
+ subscribe(l) { this.listeners.add(l); return () => this.listeners.delete(l) },
60
+ }
61
+
62
+ function useVoice() {
63
+ const [, force] = React.useReducer((x) => x + 1, 0)
64
+ React.useEffect(() => voice.subscribe(force), [])
65
+ return voice
66
+ }
67
+
68
+ // ---------------------------------------------------------------- icons
69
+ const ic = { width: 18, height: 18, viewBox: '0 0 24 24', fill: 'none', stroke: 'currentColor', strokeWidth: 2, strokeLinecap: 'round', strokeLinejoin: 'round' }
70
+ const micIcon = () => React.createElement('svg', ic,
71
+ React.createElement('path', { d: 'M12 1a3 3 0 0 0-3 3v8a3 3 0 0 0 6 0V4a3 3 0 0 0-3-3z' }),
72
+ React.createElement('path', { d: 'M19 10v2a7 7 0 0 1-14 0v-2' }),
73
+ React.createElement('line', { x1: 12, y1: 19, x2: 12, y2: 23 }))
74
+ const waveIcon = () => React.createElement('svg', ic,
75
+ React.createElement('line', { x1: 4, y1: 10, x2: 4, y2: 14 }),
76
+ React.createElement('line', { x1: 8, y1: 7, x2: 8, y2: 17 }),
77
+ React.createElement('line', { x1: 12, y1: 4, x2: 12, y2: 20 }),
78
+ React.createElement('line', { x1: 16, y1: 7, x2: 16, y2: 17 }),
79
+ React.createElement('line', { x1: 20, y1: 10, x2: 20, y2: 14 }))
80
+ const xIcon = () => React.createElement('svg', ic,
81
+ React.createElement('line', { x1: 18, y1: 6, x2: 6, y2: 18 }),
82
+ React.createElement('line', { x1: 6, y1: 6, x2: 18, y2: 18 }))
83
+ const stopIcon = () => React.createElement('svg', ic,
84
+ React.createElement('rect', { x: 7, y: 7, width: 10, height: 10, rx: 2.5, fill: 'currentColor', stroke: 'none' }))
85
+ const spinIcon = () => React.createElement('svg', Object.assign({}, ic, { className: 'dvo-spin' }),
86
+ React.createElement('path', { d: 'M21 12a9 9 0 1 1-6.219-8.56' }))
87
+ const warnIcon = () => React.createElement('svg', ic,
88
+ React.createElement('path', { d: 'M10.29 3.86L1.82 18a2 2 0 0 0 1.71 3h16.94a2 2 0 0 0 1.71-3L13.71 3.86a2 2 0 0 0-3.42 0z' }),
89
+ React.createElement('line', { x1: 12, y1: 9, x2: 12, y2: 13 }),
90
+ React.createElement('line', { x1: 12, y1: 17, x2: 12.01, y2: 17 }))
91
+
92
+ // ------------------------------------------------------------ transport
93
+ function blobToBase64(blob) {
94
+ return new Promise((resolve, reject) => {
95
+ if (typeof FileReader === 'undefined') { reject(new Error('FileReader not supported')); return }
96
+ const fr = new FileReader()
97
+ fr.onload = () => { const s = String(fr.result || ''); resolve(s.indexOf(',') >= 0 ? s.slice(s.indexOf(',') + 1) : s) }
98
+ fr.onerror = () => reject(new Error('failed to read audio'))
99
+ fr.readAsDataURL(blob)
100
+ })
101
+ }
102
+
103
+ async function sendAudio(blob, mime, mode) {
104
+ const dataBase64 = await blobToBase64(blob)
105
+ const res = await fetch('/dsh-voice/transcribe', {
106
+ method: 'POST',
107
+ headers: { 'content-type': 'application/json' },
108
+ body: JSON.stringify({ dataBase64, mimeType: mime, mode }),
109
+ })
110
+ let parsed = null
111
+ try { parsed = await res.json() } catch (e) { /* не json */ }
112
+ if (!res.ok || !parsed || !parsed.ok) {
113
+ throw new Error((parsed && parsed.error && parsed.error.message) || `HTTP ${res.status}`)
114
+ }
115
+ return String(parsed.text || '').trim()
116
+ }
117
+
118
+ function appendDraft(text) {
119
+ const actions = voice.inputActions
120
+ if (!actions || typeof actions.setDraft !== 'function') {
121
+ voice.set({ phase: 'error', error: 'Композер недоступен' })
122
+ return
123
+ }
124
+ const draft = voice.input && typeof voice.input.draft === 'string' ? voice.input.draft : ''
125
+ actions.setDraft(draft ? draft + ' ' + text : text)
126
+ }
127
+
128
+ // ------------------------------------------------------------ recording
129
+ function teardown(rec) {
130
+ if (!rec) return
131
+ try { rec.stream.getTracks().forEach((t) => t.stop()) } catch (e) { /* уже остановлен */ }
132
+ if (rec.audioCtx) { try { rec.audioCtx.close() } catch (e) { /* уже закрыт */ } }
133
+ }
134
+
135
+ function waitStop(recorder) {
136
+ return new Promise((resolve) => recorder.addEventListener('stop', resolve, { once: true }))
137
+ }
138
+
139
+ async function openMic(mode) {
140
+ if (typeof navigator === 'undefined' || !navigator.mediaDevices || !navigator.mediaDevices.getUserMedia) {
141
+ throw new Error('Микрофон недоступен: нужен HTTPS или localhost')
142
+ }
143
+ if (typeof MediaRecorder === 'undefined') throw new Error('MediaRecorder не поддерживается браузером')
144
+ const stream = await navigator.mediaDevices.getUserMedia({
145
+ audio: { channelCount: 1, echoCancellation: true, noiseSuppression: true },
146
+ })
147
+ let mimeType = 'audio/webm;codecs=opus'
148
+ if (!MediaRecorder.isTypeSupported(mimeType)) mimeType = ''
149
+ const recorder = mimeType ? new MediaRecorder(stream, { mimeType }) : new MediaRecorder(stream)
150
+ const rec = {
151
+ recorder, stream, mode,
152
+ chunks: [],
153
+ mime: mimeType || recorder.mimeType || 'audio/webm',
154
+ audioCtx: null, analyser: null,
155
+ cutting: false, closing: false,
156
+ silenceMs: 0, hadSpeech: false,
157
+ }
158
+ recorder.ondataavailable = (e) => { if (e.data && e.data.size > 0) rec.chunks.push(e.data) }
159
+ const AC = typeof AudioContext !== 'undefined' ? AudioContext
160
+ : (typeof webkitAudioContext !== 'undefined' ? webkitAudioContext : null)
161
+ if (AC) {
162
+ rec.audioCtx = new AC()
163
+ const src = rec.audioCtx.createMediaStreamSource(stream)
164
+ rec.analyser = rec.audioCtx.createAnalyser()
165
+ rec.analyser.fftSize = 128
166
+ src.connect(rec.analyser)
167
+ }
168
+ // Без timeslice: только тогда каждый stop() даёт самостоятельный webm-файл.
169
+ recorder.start()
170
+ return rec
171
+ }
172
+
173
+ function currentLevel(rec) {
174
+ if (!rec || !rec.analyser) return 0
175
+ const data = new Uint8Array(rec.analyser.frequencyBinCount)
176
+ rec.analyser.getByteFrequencyData(data)
177
+ let sum = 0
178
+ for (let i = 0; i < data.length; i++) sum += data[i]
179
+ return Math.min(1, (sum / data.length / 255) * 2.2)
180
+ }
181
+
182
+ // Режет текущую фразу: останавливает рекордер, отправляет готовый файл и
183
+ // немедленно начинает новую запись тем же рекордером.
184
+ function cutPhrase() {
185
+ const rec = voice.rec
186
+ if (!rec || rec.cutting || rec.closing) return
187
+ rec.cutting = true
188
+ const stopped = waitStop(rec.recorder)
189
+ try { rec.recorder.stop() } catch (e) { /* уже остановлен */ }
190
+ stopped.then(async () => {
191
+ const blob = new Blob(rec.chunks, { type: rec.mime })
192
+ rec.chunks = []
193
+ rec.silenceMs = 0
194
+ rec.hadSpeech = false
195
+ if (!rec.closing) {
196
+ try { rec.recorder.start() } catch (e) { /* поток закрылся */ }
197
+ }
198
+ rec.cutting = false
199
+ if (blob.size < 1200) return // слишком короткий кусок — это не речь
200
+ try {
201
+ const text = await sendAudio(blob, rec.mime, 'dictation')
202
+ if (text) appendDraft(text)
203
+ } catch (e) {
204
+ voice.set({ error: String(e && e.message ? e.message : e) })
205
+ }
206
+ })
207
+ }
208
+
209
+ function startRecording(mode) {
210
+ if (voice.phase !== 'idle' && voice.phase !== 'error') return
211
+ voice.set({ phase: 'recording', mode, error: '', levels: [] })
212
+ openMic(mode)
213
+ .then((rec) => { voice.rec = rec; voice.notify() })
214
+ .catch((err) => voice.set({ phase: 'error', error: String(err && err.message ? err.message : err), rec: null }))
215
+ }
216
+
217
+ function startDictation() { startRecording('dictation') }
218
+ function startMessage() { startRecording('message') }
219
+
220
+ function cancelCurrent() {
221
+ const rec = voice.rec
222
+ voice.pending = null
223
+ if (!rec) { voice.set({ phase: 'idle', error: '' }); return }
224
+ rec.closing = true
225
+ const stopped = waitStop(rec.recorder)
226
+ try { rec.recorder.stop() } catch (e) { /* уже остановлен */ }
227
+ stopped.then(() => { teardown(rec); voice.rec = null; voice.set({ phase: 'idle', error: '' }) })
228
+ }
229
+
230
+ // Останов по второму нажатию: диктовка досылает хвост, голосовое —
231
+ // отправляет всю запись и открывает окно отмены.
232
+ function stopCurrent() {
233
+ const rec = voice.rec
234
+ if (!rec || rec.closing) return
235
+ rec.closing = true
236
+ const mode = rec.mode
237
+ const stopped = waitStop(rec.recorder)
238
+ try { rec.recorder.stop() } catch (e) { /* уже остановлен */ }
239
+ stopped.then(async () => {
240
+ const blob = new Blob(rec.chunks, { type: rec.mime })
241
+ teardown(rec)
242
+ voice.rec = null
243
+ if (blob.size < 1200) { voice.set({ phase: 'idle' }); return }
244
+ voice.set({ phase: 'processing' })
245
+ try {
246
+ const text = await sendAudio(blob, rec.mime, mode)
247
+ if (!text) { voice.set({ phase: 'error', error: 'Речь не распознана' }); return }
248
+ appendDraft(text)
249
+ if (mode === 'message') {
250
+ voice.set({ phase: 'pending', pending: { text: text, leftMs: voice.settings.autoSendMs } })
251
+ } else {
252
+ voice.set({ phase: 'idle' })
253
+ }
254
+ } catch (e) {
255
+ voice.set({ phase: 'error', error: String(e && e.message ? e.message : e) })
256
+ }
257
+ })
258
+ }
259
+
260
+ function submitPending() {
261
+ voice.pending = null
262
+ voice.set({ phase: 'idle' })
263
+ const actions = voice.inputActions
264
+ if (actions && typeof actions.submit === 'function') {
265
+ setTimeout(() => { try { actions.submit() } catch (e) { /* композер занят */ } }, 0)
266
+ }
267
+ }
268
+
269
+ function keepPending() {
270
+ voice.pending = null
271
+ voice.set({ phase: 'idle' })
272
+ }
273
+
274
+ // ----------------------------------------------------------- components
275
+ function VoiceButtons(props) {
276
+ const v = useVoice()
277
+ voice.inputActions = props.inputActions
278
+ voice.input = props.input
279
+ if (v.phase !== 'idle' && v.phase !== 'error') return null
280
+ const err = v.phase === 'error'
281
+ return React.createElement(React.Fragment, null,
282
+ React.createElement('button', {
283
+ type: 'button', className: 'dvo-btn', 'data-err': err ? '1' : '0',
284
+ title: err ? v.error : 'Голосовой набор', onClick: startDictation,
285
+ }, micIcon()),
286
+ React.createElement('button', {
287
+ type: 'button', className: 'dvo-btn', 'data-err': err ? '1' : '0',
288
+ title: err ? v.error : 'Голосовое сообщение', onClick: startMessage,
289
+ }, waveIcon()),
290
+ )
291
+ }
292
+
293
+ function RecordPill(props) {
294
+ const v = useVoice()
295
+ const canvasRef = React.useRef(null)
296
+ voice.inputActions = props.inputActions
297
+ voice.input = props.input
298
+ const ctx = props.ctx
299
+
300
+ // VAD: копим тишину и режем фразу, когда пауза превысила порог.
301
+ React.useEffect(() => {
302
+ if (v.phase !== 'recording') return
303
+ const tick = 50
304
+ const dispose = ctx.interval(() => {
305
+ const rec = voice.rec
306
+ if (!rec) return
307
+ const level = currentLevel(rec)
308
+ voice.levels.push(level)
309
+ if (voice.levels.length > 150) voice.levels.shift()
310
+ if (level > 0.06) { rec.hadSpeech = true; rec.silenceMs = 0 }
311
+ else if (rec.hadSpeech) rec.silenceMs += tick
312
+ if (rec.mode === 'dictation' && rec.hadSpeech && rec.silenceMs >= voice.settings.vadSilenceMs) {
313
+ cutPhrase()
314
+ }
315
+ }, tick)
316
+ return () => dispose()
317
+ }, [v.phase])
318
+
319
+ // Осциллограмма.
320
+ React.useEffect(() => {
321
+ if (v.phase !== 'recording') return
322
+ const dispose = ctx.interval(() => {
323
+ const canvas = canvasRef.current
324
+ if (!canvas) return
325
+ const g = canvas.getContext('2d')
326
+ const w = canvas.width, h = canvas.height
327
+ g.clearRect(0, 0, w, h)
328
+ if (!voice.waveColor) {
329
+ try { voice.waveColor = getComputedStyle(canvas).color || '#fff' } catch (e) { voice.waveColor = '#fff' }
330
+ }
331
+ const levels = voice.levels
332
+ const midY = h / 2
333
+ for (let i = 0; i < levels.length && i * 7 < w; i++) {
334
+ const level = levels[levels.length - 1 - i]
335
+ const age = i / levels.length
336
+ const x = w - 10 - i * 7
337
+ const hh = Math.max(2.5, level * (h - 6) * 0.5 * (1 - age * 0.35))
338
+ g.globalAlpha = 1 - age * 0.75
339
+ g.fillStyle = voice.waveColor
340
+ g.fillRect(x, midY - hh, 3.5, hh * 2)
341
+ }
342
+ g.globalAlpha = 1
343
+ }, 50)
344
+ return () => dispose()
345
+ }, [v.phase])
346
+
347
+ // Окно отмены режима message.
348
+ React.useEffect(() => {
349
+ if (v.phase !== 'pending') return
350
+ const tick = 100
351
+ const dispose = ctx.interval(() => {
352
+ const p = voice.pending
353
+ if (!p) return
354
+ p.leftMs -= tick
355
+ if (p.leftMs <= 0) { submitPending(); return }
356
+ voice.notify()
357
+ }, tick)
358
+ return () => dispose()
359
+ }, [v.phase])
360
+
361
+ if (v.phase === 'idle') return null
362
+
363
+ if (v.phase === 'recording') {
364
+ const hint = v.mode === 'dictation' ? 'Диктовка — текст дописывается в строку' : 'Запись голосового'
365
+ return React.createElement('div', { className: 'dvo-pill' },
366
+ React.createElement('button', { type: 'button', className: 'dvo-pbtn', title: 'Отмена', onClick: cancelCurrent }, xIcon()),
367
+ React.createElement('canvas', { className: 'dvo-wave', ref: canvasRef, width: 720, height: 40 }),
368
+ React.createElement('span', { className: 'dvo-status' }, hint),
369
+ React.createElement('button', { type: 'button', className: 'dvo-pbtn', title: 'Стоп', onClick: stopCurrent }, stopIcon()),
370
+ )
371
+ }
372
+
373
+ if (v.phase === 'processing') {
374
+ return React.createElement('div', { className: 'dvo-pill' },
375
+ React.createElement('span', { className: 'dvo-status' }, spinIcon(), 'Распознаю…'))
376
+ }
377
+
378
+ if (v.phase === 'pending') {
379
+ const left = Math.max(0, Math.ceil((voice.pending ? voice.pending.leftMs : 0) / 1000))
380
+ return React.createElement('div', { className: 'dvo-pill' },
381
+ React.createElement('span', { className: 'dvo-status' }, 'Отправляю агенту через'),
382
+ React.createElement('span', { className: 'dvo-count' }, left + ' с'),
383
+ React.createElement('button', { type: 'button', className: 'dvo-pbtn', title: 'Отменить отправку', onClick: keepPending }, xIcon()),
384
+ )
385
+ }
386
+
387
+ return React.createElement('div', { className: 'dvo-pill' },
388
+ React.createElement('span', { className: 'dvo-status dvo-err' }, warnIcon(), v.error),
389
+ React.createElement('button', { type: 'button', className: 'dvo-pbtn', title: 'Скрыть', onClick: () => voice.set({ phase: 'idle', error: '' }) }, xIcon()),
390
+ )
391
+ }
392
+
393
+ // --------------------------------------------------------------- slots
394
+ function registerComposer(ctx) {
395
+ ctx.slots.inject('conversation.input.right', () => ctx.slots.register(
396
+ { name: 'conversation.input.right', id: '@goodandready/dsh-voice', order: 6, label: () => 'Голос' },
397
+ (props) => React.createElement(VoiceButtons, { input: props.input, inputActions: props.inputActions }),
398
+ ))
399
+ ctx.slots.inject('conversation.input.dock', () => ctx.slots.register(
400
+ { name: 'conversation.input.dock', id: 'dsh-voice-rec', order: 0, label: () => 'Запись голоса' },
401
+ (props) => React.createElement(RecordPill, { input: props.input, inputActions: props.inputActions, ctx: ctx }),
402
+ ))
403
+ }
404
+
405
+ // ------------------------------------------------------- settings page
406
+ const PROVIDERS = ['deepgram', 'groq', 'hf', 'local-whisper']
407
+ const MODEL_HINT = {
408
+ deepgram: 'nova-2', groq: 'whisper-large-v3-turbo',
409
+ hf: 'openai/whisper-large-v3', 'local-whisper': 'задаётся при запуске сервера',
410
+ }
411
+ const LANGS = ['auto', 'ru', 'en', 'uk', 'de']
412
+
413
+ const SET_CSS =
414
+ '.dvs-wrap{display:flex;flex-direction:column;gap:22px;padding:4px 0;max-width:720px}' +
415
+ '.dvs-block{display:flex;flex-direction:column;gap:10px}' +
416
+ '.dvs-h{font-size:14px;font-weight:600;color:var(--dsw-alias-label-primary)}' +
417
+ '.dvs-sub{font-size:12px;color:var(--dsw-alias-label-secondary)}' +
418
+ '.dvs-row{display:flex;gap:8px;align-items:center}' +
419
+ '.dvs-row select,.dvs-row input{background:var(--dsw-alias-bg-layer-1);border:1px solid var(--dsw-alias-border-l1);color:var(--dsw-alias-label-primary);border-radius:6px;padding:6px 8px;font-size:13px}' +
420
+ '.dvs-row .dvs-model{flex:1}' +
421
+ '.dvs-field{display:flex;flex-direction:column;gap:4px;font-size:12px;color:var(--dsw-alias-label-secondary)}' +
422
+ '.dvs-field input,.dvs-field select{background:var(--dsw-alias-bg-layer-1);border:1px solid var(--dsw-alias-border-l1);color:var(--dsw-alias-label-primary);border-radius:6px;padding:6px 8px;font-size:13px}' +
423
+ '.dvs-mini{border:1px solid var(--dsw-alias-border-l2);background:transparent;color:var(--dsw-alias-label-primary);border-radius:6px;width:28px;height:28px;cursor:pointer;flex:none}' +
424
+ '.dvs-save{background:var(--dsw-alias-brand-primary);color:#fff;border:none;border-radius:6px;padding:7px 14px;font-size:13px;cursor:pointer}' +
425
+ '.dvs-ok{font-size:12px;color:var(--dsw-alias-state-success-primary)}' +
426
+ '.dvs-bad{font-size:12px;color:var(--dsw-alias-state-error-primary)}'
427
+ const setCssId = 'dsh-voice/settings.module.css'
428
+ if (typeof document !== 'undefined' && !document.querySelector('style[data-plugin-css="' + setCssId + '"]')) {
429
+ const tag = document.createElement('style')
430
+ tag.textContent = SET_CSS
431
+ tag.setAttribute('data-plugin', 'dsh-voice')
432
+ tag.dataset.pluginCss = setCssId
433
+ document.head.appendChild(tag)
434
+ }
435
+
436
+ // Редактор одной цепочки: строки «провайдер + модель» с перестановкой.
437
+ function ChainEditor(props) {
438
+ const rows = Array.isArray(props.value) ? props.value : []
439
+ const change = (i, patch) => {
440
+ const next = rows.map((r, k) => (k === i ? Object.assign({}, r, patch) : r))
441
+ props.onChange(next)
442
+ }
443
+ const move = (i, delta) => {
444
+ const j = i + delta
445
+ if (j < 0 || j >= rows.length) return
446
+ const next = rows.slice()
447
+ const tmp = next[i]; next[i] = next[j]; next[j] = tmp
448
+ props.onChange(next)
449
+ }
450
+ const remove = (i) => props.onChange(rows.filter((_, k) => k !== i))
451
+ const add = () => props.onChange(rows.concat([{ provider: 'local-whisper', model: '' }]))
452
+
453
+ return React.createElement('div', { className: 'dvs-block' },
454
+ rows.map((row, i) => React.createElement('div', { className: 'dvs-row', key: i },
455
+ React.createElement('select', {
456
+ value: row.provider, disabled: !props.writable,
457
+ onChange: (e) => change(i, { provider: e.target.value }),
458
+ }, PROVIDERS.map((p) => React.createElement('option', { key: p, value: p }, p))),
459
+ React.createElement('input', {
460
+ className: 'dvs-model', value: row.model || '', disabled: !props.writable,
461
+ placeholder: MODEL_HINT[row.provider] || '', onChange: (e) => change(i, { model: e.target.value }),
462
+ }),
463
+ React.createElement('button', { type: 'button', className: 'dvs-mini', title: 'Выше', disabled: !props.writable, onClick: () => move(i, -1) }, '↑'),
464
+ React.createElement('button', { type: 'button', className: 'dvs-mini', title: 'Ниже', disabled: !props.writable, onClick: () => move(i, 1) }, '↓'),
465
+ React.createElement('button', { type: 'button', className: 'dvs-mini', title: 'Убрать', disabled: !props.writable, onClick: () => remove(i) }, '×'),
466
+ )),
467
+ React.createElement('div', { className: 'dvs-row' },
468
+ React.createElement('button', { type: 'button', className: 'dvs-mini', title: 'Добавить провайдера', disabled: !props.writable, onClick: add }, '+'),
469
+ React.createElement('span', { className: 'dvs-sub' }, 'Порядок сверху вниз — порядок попыток'),
470
+ ),
471
+ )
472
+ }
473
+
474
+ function VoiceSection(props) {
475
+ const ctx = props.ctx
476
+ const scope = ctx.settingsScope.bind({ namespace: NS })
477
+ const [snap, setSnap] = React.useState(null)
478
+ const [draft, setDraft] = React.useState(null)
479
+ const [saved, setSaved] = React.useState(false)
480
+ const [err, setErr] = React.useState('')
481
+
482
+ React.useEffect(() => {
483
+ let alive = true
484
+ const render = () => { if (alive) setSnap(scope.getSnapshot()) }
485
+ render()
486
+ const off = scope.subscribe(render)
487
+ return () => { alive = false; off() }
488
+ }, [])
489
+
490
+ const value = snap && snap.value ? snap.value : {}
491
+ const writable = snap ? snap.writable !== false : false
492
+ React.useEffect(() => { if (snap && draft === null) setDraft(JSON.parse(JSON.stringify(value))) }, [snap, draft, value])
493
+ // Клиент должен знать порог VAD и окно отмены — они живут в тех же настройках.
494
+ React.useEffect(() => {
495
+ if (!snap) return
496
+ voice.settings = {
497
+ vadSilenceMs: Number(value && value.dictation && value.dictation.vadSilenceMs) || 700,
498
+ autoSendMs: Number(value && value.message && value.message.autoSendMs) || 4000,
499
+ }
500
+ }, [snap])
501
+
502
+ if (!snap) return React.createElement('div', { className: 'dvs-wrap' }, 'Загрузка…')
503
+
504
+ const setIn = (mode, key, v) => setDraft((d) => {
505
+ const next = JSON.parse(JSON.stringify(d || {}))
506
+ next[mode] = next[mode] || {}
507
+ next[mode][key] = v
508
+ return next
509
+ })
510
+ const setTop = (key, v) => setDraft((d) => Object.assign({}, d || {}, { [key]: v }))
511
+
512
+ const save = async () => {
513
+ setErr(''); setSaved(false)
514
+ if (!draft) return
515
+ try {
516
+ for (const k of Object.keys(draft)) await scope.set(k, draft[k])
517
+ voice.settings = {
518
+ vadSilenceMs: Number(draft.dictation && draft.dictation.vadSilenceMs) || 700,
519
+ autoSendMs: Number(draft.message && draft.message.autoSendMs) || 4000,
520
+ }
521
+ setSaved(true); setTimeout(() => setSaved(false), 2000)
522
+ } catch (e) { setErr(String(e && e.message ? e.message : e)) }
523
+ }
524
+
525
+ const modeVal = (mode, key, fallback) => {
526
+ const m = draft && draft[mode]
527
+ return m && m[key] !== undefined ? m[key] : fallback
528
+ }
529
+
530
+ const langField = (mode) => React.createElement('label', { className: 'dvs-field' }, 'Язык',
531
+ React.createElement('select', {
532
+ value: modeVal(mode, 'language', 'ru'), disabled: !writable,
533
+ onChange: (e) => setIn(mode, 'language', e.target.value),
534
+ }, LANGS.map((o) => React.createElement('option', { key: o, value: o }, o))))
535
+
536
+ const numField = (mode, key, label, hint) => React.createElement('label', { className: 'dvs-field' }, label,
537
+ React.createElement('input', {
538
+ type: 'number', value: modeVal(mode, key, ''), disabled: !writable,
539
+ onChange: (e) => setIn(mode, key, Number(e.target.value)),
540
+ }),
541
+ React.createElement('span', { className: 'dvs-sub' }, hint))
542
+
543
+ const textField = (key, label, hint) => React.createElement('label', { className: 'dvs-field' }, label,
544
+ React.createElement('input', {
545
+ value: draft && draft[key] !== undefined ? draft[key] : '', disabled: !writable,
546
+ onChange: (e) => setTop(key, e.target.value),
547
+ }),
548
+ React.createElement('span', { className: 'dvs-sub' }, hint))
549
+
550
+ return React.createElement('div', { className: 'dvs-wrap' },
551
+ React.createElement('div', { className: 'dvs-block' },
552
+ React.createElement('div', { className: 'dvs-h' }, 'Голосовой набор'),
553
+ React.createElement('div', { className: 'dvs-sub' }, 'Речь режется по паузам, текст дописывается в строку ввода.'),
554
+ React.createElement(ChainEditor, {
555
+ value: draft && draft.dictation ? draft.dictation.chain : [], writable: writable,
556
+ onChange: (v) => setIn('dictation', 'chain', v),
557
+ }),
558
+ langField('dictation'),
559
+ numField('dictation', 'vadSilenceMs', 'Пауза до конца фразы, мс', 'Меньше — чаще куски и быстрее текст, но выше риск обрезать слово'),
560
+ ),
561
+ React.createElement('div', { className: 'dvs-block' },
562
+ React.createElement('div', { className: 'dvs-h' }, 'Голосовое сообщение'),
563
+ React.createElement('div', { className: 'dvs-sub' }, 'Одна запись целиком, после распознавания уходит агенту.'),
564
+ React.createElement(ChainEditor, {
565
+ value: draft && draft.message ? draft.message.chain : [], writable: writable,
566
+ onChange: (v) => setIn('message', 'chain', v),
567
+ }),
568
+ langField('message'),
569
+ numField('message', 'autoSendMs', 'Окно отмены, мс', 'Сколько времени можно отменить автоматическую отправку'),
570
+ ),
571
+ React.createElement('div', { className: 'dvs-block' },
572
+ React.createElement('div', { className: 'dvs-h' }, 'Общее'),
573
+ textField('whisperUrl', 'Локальный whisper: endpoint', 'POST /inference сервера whisper.cpp'),
574
+ textField('whisperBin', 'Локальный whisper: бинарь', 'используется при автозапуске'),
575
+ textField('whisperModel', 'Локальный whisper: модель', 'используется при автозапуске'),
576
+ React.createElement('label', { className: 'dvs-field' }, 'Автозапуск локального whisper',
577
+ React.createElement('input', {
578
+ type: 'checkbox', checked: !!(draft && draft.autoStart), disabled: !writable,
579
+ onChange: (e) => setTop('autoStart', e.target.checked),
580
+ })),
581
+ ),
582
+ React.createElement('div', { className: 'dvs-row' },
583
+ React.createElement('button', { type: 'button', className: 'dvs-save', disabled: !writable, onClick: save }, 'Сохранить'),
584
+ saved ? React.createElement('span', { className: 'dvs-ok' }, 'Сохранено ✓') : null,
585
+ err ? React.createElement('span', { className: 'dvs-bad' }, err) : null,
586
+ ),
587
+ )
588
+ }
589
+
590
+ function registerSettings(ctx) {
591
+ ctx.slots.inject('settings.section', () => ctx.slots.register(
592
+ { name: 'settings.section', id: '@goodandready/dsh-voice', order: 30, label: () => 'Голос', inject: () => ({ ctx: ctx }) },
593
+ VoiceSection,
594
+ ))
595
+ }
596
+
597
+ exports.inject = ['timer', 'slots', 'settingsScope']
598
+ exports.apply = function apply(ctx) {
599
+ registerComposer(ctx)
600
+ registerSettings(ctx)
601
+ }
602
+ return module.exports
603
+ },
604
+ })
package/lib/index.js ADDED
@@ -0,0 +1,266 @@
1
+ // dsh-voice — хост-половина.
2
+ //
3
+ // Два режима ввода голосом, у каждого своя цепочка провайдеров:
4
+ // dictation — браузер режет речь по паузам и шлёт куски, текст дописывается
5
+ // в строку ввода;
6
+ // message — одна запись целиком, текст уходит агенту после окна отмены.
7
+ //
8
+ // Роуты:
9
+ // POST /dsh-voice/transcribe {dataBase64, mimeType, mode} -> {ok, text, provider, tookMs}
10
+ // GET /dsh-voice/status -> {ok, whisperRunning, modes}
11
+ //
12
+ // Ключи провайдеров читаются на хосте через ctx.credentials и в браузер не
13
+ // попадают.
14
+
15
+ import z from '@deepseek-ai/schemastery'
16
+ import { defineTool } from '@deepseek-ai/dsh-tools'
17
+ import { credentialRef } from '@deepseek-ai/dsh-credentials'
18
+ import { readFile, stat } from 'node:fs/promises'
19
+ import path from 'node:path'
20
+ import { runChain } from './chain.js'
21
+ import { makeProviders, PROVIDER_KEYS, DEFAULT_MODELS } from './providers.js'
22
+ import { toWav16k } from './wav.js'
23
+
24
+ export const name = 'dsh-voice'
25
+ export const inject = ['tools', 'credentials', 'webServer', 'shell']
26
+
27
+ const ChainEntry = z.object({
28
+ provider: z.string().default('local-whisper')
29
+ .description(`Provider key. One of: ${PROVIDER_KEYS.join(', ')}.`),
30
+ model: z.string().default('')
31
+ .description('Model override. Empty means the provider default.'),
32
+ })
33
+
34
+ export const Config = z.object({
35
+ dictation: z.object({
36
+ chain: z.array(ChainEntry)
37
+ .default([{ provider: 'deepgram', model: '' }, { provider: 'groq', model: '' }, { provider: 'local-whisper', model: '' }])
38
+ .description('Fallback chain for dictation. Speed matters more than accuracy here.'),
39
+ language: z.string().default('ru'),
40
+ vadSilenceMs: z.number().default(700)
41
+ .description('Silence longer than this ends a phrase and sends the chunk.'),
42
+ }).default({}),
43
+ message: z.object({
44
+ chain: z.array(ChainEntry)
45
+ .default([{ provider: 'groq', model: '' }, { provider: 'hf', model: '' }, { provider: 'local-whisper', model: '' }])
46
+ .description('Fallback chain for voice messages. Accuracy matters more than speed.'),
47
+ language: z.string().default('ru'),
48
+ autoSendMs: z.number().default(4000)
49
+ .description('Cancel window before the recognized text is sent to the agent.'),
50
+ }).default({}),
51
+ deepgramKeyEnv: z.string().default('DEEPGRAM_API_KEY'),
52
+ groqKeyEnv: z.string().default('GROQ_API_KEY'),
53
+ hfTokenEnv: z.string().default('HF_TOKEN'),
54
+ whisperUrl: z.string().default('http://127.0.0.1:8001/inference'),
55
+ whisperBin: z.string().default('whisper-server')
56
+ .description('whisper.cpp server binary, looked up in PATH unless an absolute path is given.'),
57
+ whisperModel: z.string().default('')
58
+ .description('Absolute path to the ggml model. Autostart stays off while this is empty; '
59
+ + 'point it at your own model file to let the plugin launch whisper.cpp itself.'),
60
+ autoStart: z.boolean().default(true)
61
+ .description('Launch the local whisper.cpp server on activation if the port is free. '
62
+ + 'Requires whisperModel to be set.'),
63
+ ffmpegBin: z.string().default('ffmpeg')
64
+ .description('ffmpeg used to convert browser webm/opus into the WAV that whisper.cpp requires.'),
65
+ timeoutMs: z.number().default(120000),
66
+ maxFileBytes: z.number().default(25 * 1024 * 1024),
67
+ })
68
+
69
+ const MIME_BY_EXT = {
70
+ '.wav': 'audio/wav', '.mp3': 'audio/mpeg', '.m4a': 'audio/mp4', '.mp4': 'audio/mp4',
71
+ '.ogg': 'audio/ogg', '.oga': 'audio/ogg', '.flac': 'audio/flac', '.webm': 'audio/webm', '.aac': 'audio/aac',
72
+ }
73
+
74
+ function writeJson(res, code, body) {
75
+ try {
76
+ res.writeHead(code, { 'Content-Type': 'application/json', 'Cache-Control': 'no-store' })
77
+ res.end(JSON.stringify(body))
78
+ } catch { /* сокет мог закрыться */ }
79
+ }
80
+
81
+ function readBody(req, maxBytes) {
82
+ return new Promise((resolve, reject) => {
83
+ const chunks = []
84
+ let size = 0
85
+ req.on('data', (c) => {
86
+ size += c.length
87
+ if (size > maxBytes) { reject(new Error('body too large')); req.destroy(); return }
88
+ chunks.push(c)
89
+ })
90
+ req.on('end', () => resolve(Buffer.concat(chunks)))
91
+ req.on('error', reject)
92
+ })
93
+ }
94
+
95
+ export function apply(ctx, config) {
96
+ let child = null
97
+
98
+ async function resolveKey(ref) {
99
+ try {
100
+ const resolved = await ctx.credentials.resolve(credentialRef(ref))
101
+ if (resolved && resolved.value) return resolved.value
102
+ } catch { /* падаем в окружение */ }
103
+ return process.env[ref] || ''
104
+ }
105
+
106
+ async function whisperAlive() {
107
+ try {
108
+ const controller = new AbortController()
109
+ const t = setTimeout(() => controller.abort(), 2000)
110
+ const res = await fetch(config.whisperUrl.split('/inference')[0] + '/', { signal: controller.signal })
111
+ clearTimeout(t)
112
+ return res.ok
113
+ } catch { return false }
114
+ }
115
+
116
+ async function startWhisper() {
117
+ if (!config.autoStart) return false
118
+ // Без пути к модели запускать нечего: пакет не знает, где она лежит у
119
+ // конкретного пользователя, и молча стартовать чужой бинарь не должен.
120
+ if (!config.whisperModel) return false
121
+ if (await whisperAlive()) return true
122
+ try {
123
+ const spec = ctx.shell.resolve({
124
+ command: `${JSON.stringify(config.whisperBin)} -m ${JSON.stringify(config.whisperModel)}`
125
+ + ` --host 127.0.0.1 --port 8001 -t 8 -p 1 -l ${config.dictation.language}`,
126
+ timeoutMs: 0,
127
+ stdoutMaxBytes: 4 * 1024 * 1024,
128
+ })
129
+ child = ctx.shell.start(spec)
130
+ for (let i = 0; i < 20; i++) {
131
+ await new Promise((r) => setTimeout(r, 500))
132
+ if (await whisperAlive()) return true
133
+ }
134
+ return false
135
+ } catch { return false }
136
+ }
137
+
138
+ startWhisper().catch(() => {})
139
+
140
+ // Общий путь распознавания: собрать провайдеров по цепочке режима и пройти её.
141
+ async function transcribe(modeCfg, bytes, mime, signal) {
142
+ const models = {}
143
+ const order = []
144
+ for (const entry of Array.isArray(modeCfg.chain) ? modeCfg.chain : []) {
145
+ if (!PROVIDER_KEYS.includes(entry.provider)) continue
146
+ order.push(entry.provider)
147
+ models[entry.provider] = entry.model || DEFAULT_MODELS[entry.provider]
148
+ }
149
+ const providers = makeProviders(
150
+ { resolveKey, fetchImpl: fetch, cfg: config, toWav: (b) => toWav16k(b, config.ffmpegBin) },
151
+ { bytes, mime, lang: modeCfg.language, signal, models },
152
+ )
153
+ return runChain(order, providers)
154
+ }
155
+
156
+ ctx.effect(() => ctx.webServer.register({
157
+ kind: 'exact',
158
+ path: '/dsh-voice/status',
159
+ handler: async (req, res) => {
160
+ if (req.method !== 'GET') { writeJson(res, 405, { ok: false, error: { code: 'method', message: 'GET only' } }); return }
161
+ writeJson(res, 200, {
162
+ ok: true,
163
+ whisperRunning: await whisperAlive(),
164
+ modes: {
165
+ dictation: { chain: config.dictation.chain, language: config.dictation.language, vadSilenceMs: config.dictation.vadSilenceMs },
166
+ message: { chain: config.message.chain, language: config.message.language, autoSendMs: config.message.autoSendMs },
167
+ },
168
+ })
169
+ },
170
+ }), 'dsh-voice: /status route')
171
+
172
+ ctx.effect(() => ctx.webServer.register({
173
+ kind: 'exact',
174
+ path: '/dsh-voice/transcribe',
175
+ handler: async (req, res) => {
176
+ if (req.method !== 'POST') { writeJson(res, 405, { ok: false, error: { code: 'method', message: 'POST only' } }); return }
177
+ let raw
178
+ try {
179
+ raw = await readBody(req, config.maxFileBytes + 1024 * 1024)
180
+ } catch (e) {
181
+ writeJson(res, 400, { ok: false, error: { code: 'body', message: e.message } }); return
182
+ }
183
+ let payload
184
+ try { payload = JSON.parse(raw.toString('utf8') || '{}') } catch { payload = {} }
185
+
186
+ const dataBase64 = typeof payload.dataBase64 === 'string' ? payload.dataBase64 : ''
187
+ if (!dataBase64) { writeJson(res, 400, { ok: false, error: { code: 'no-audio', message: 'no audio data' } }); return }
188
+ const mime = typeof payload.mimeType === 'string' && payload.mimeType ? payload.mimeType : 'audio/webm'
189
+ const modeCfg = payload.mode === 'message' ? config.message : config.dictation
190
+
191
+ let bytes
192
+ try { bytes = Buffer.from(dataBase64, 'base64') } catch { bytes = null }
193
+ if (!bytes || bytes.length === 0) {
194
+ writeJson(res, 400, { ok: false, error: { code: 'decode', message: 'failed to decode audio' } }); return
195
+ }
196
+ if (bytes.length > config.maxFileBytes) {
197
+ writeJson(res, 413, { ok: false, error: { code: 'too-large', message: `audio is ${bytes.length} bytes, max ${config.maxFileBytes}` } }); return
198
+ }
199
+
200
+ // Локальный whisper в цепочке — поднимаем сервер заранее, иначе первый
201
+ // же чанк уйдёт в отказ, пока сервер стартует.
202
+ if ((modeCfg.chain || []).some((e) => e.provider === 'local-whisper') && !(await whisperAlive())) {
203
+ await startWhisper()
204
+ }
205
+
206
+ const controller = new AbortController()
207
+ const timer = setTimeout(() => controller.abort(), config.timeoutMs)
208
+ try {
209
+ const out = await transcribe(modeCfg, bytes, mime, controller.signal)
210
+ writeJson(res, 200, { ok: true, text: out.text, provider: out.provider, tookMs: out.tookMs })
211
+ } catch (e) {
212
+ writeJson(res, 502, { ok: false, error: { code: 'chain', message: String(e && e.message || e) } })
213
+ } finally {
214
+ clearTimeout(timer)
215
+ }
216
+ },
217
+ }), 'dsh-voice: /transcribe route')
218
+
219
+ ctx.tools.register(
220
+ defineTool({
221
+ name: 'transcribe_audio',
222
+ description:
223
+ 'Recognize speech in an audio file and return the transcript as text. '
224
+ + 'Uses the voice-message fallback chain from the dsh-voice settings, so a single '
225
+ + 'provider outage or rate limit does not fail the request. '
226
+ + 'Use for voice messages, recordings, interviews.',
227
+ parameters: {
228
+ file_path: { type: 'string', required: true, description: 'Absolute path to the audio file (wav, mp3, m4a, ogg, flac, webm).' },
229
+ language: { type: 'string', description: `Recognition language code. Default: ${config.message.language}.` },
230
+ },
231
+ output: {
232
+ schema: {
233
+ type: 'object',
234
+ additionalProperties: false,
235
+ properties: { provider: { type: 'string' }, text: { type: 'string' }, tookMs: { type: 'integer' } },
236
+ },
237
+ render(args, value) {
238
+ const body = value.text.length > 4000
239
+ ? `${value.text.slice(0, 4000)}\n…[truncated ${value.text.length} chars]`
240
+ : value.text
241
+ return [{ type: 'text', text: `transcribe_audio (${value.provider}, ${value.tookMs}ms):\n${body}` }]
242
+ },
243
+ },
244
+ isConcurrencySafe: () => false,
245
+ timeoutMs: config.timeoutMs * 3 + 5000,
246
+ async execute(args, exec) {
247
+ const filePath = String(args.file_path || '').trim()
248
+ if (!filePath) throw new Error('transcribe_audio: file_path is required')
249
+ const info = await stat(filePath).catch(() => null)
250
+ if (!info) throw new Error(`transcribe_audio: file not found: ${filePath}`)
251
+ if (info.size > config.maxFileBytes) {
252
+ throw new Error(`transcribe_audio: file too large (${info.size} bytes, max ${config.maxFileBytes})`)
253
+ }
254
+ if (info.size < 100) throw new Error('transcribe_audio: file is empty or too small')
255
+ const mime = MIME_BY_EXT[path.extname(filePath).toLowerCase()] || 'audio/wav'
256
+ const bytes = await readFile(filePath)
257
+ const modeCfg = { ...config.message, language: String(args.language || config.message.language) }
258
+ return transcribe(modeCfg, bytes, mime, exec.signal)
259
+ },
260
+ }),
261
+ )
262
+
263
+ ctx.effect(() => () => {
264
+ if (child) { try { child.kill && child.kill() } catch { /* уже мёртв */ } }
265
+ }, 'dsh-voice: stop whisper child')
266
+ }
@@ -0,0 +1,118 @@
1
+ // Четыре провайдера распознавания речи. Чистые функции: сеть приходит
2
+ // параметром (fetchImpl), ключи — через resolveKey, поэтому всё проверяется
3
+ // без реальных запросов.
4
+
5
+ export const PROVIDER_KEYS = ['deepgram', 'groq', 'hf', 'local-whisper']
6
+
7
+ export const DEFAULT_MODELS = {
8
+ deepgram: 'nova-2',
9
+ groq: 'whisper-large-v3-turbo',
10
+ hf: 'openai/whisper-large-v3',
11
+ 'local-whisper': '',
12
+ }
13
+
14
+ function pickModel(models, key) {
15
+ const chosen = models && typeof models[key] === 'string' ? models[key].trim() : ''
16
+ return chosen || DEFAULT_MODELS[key]
17
+ }
18
+
19
+ function fileName(mime) {
20
+ if (mime.includes('wav')) return 'audio.wav'
21
+ if (mime.includes('ogg')) return 'audio.ogg'
22
+ if (mime.includes('mp4')) return 'audio.m4a'
23
+ return 'audio.webm'
24
+ }
25
+
26
+ export function makeProviders(deps, req) {
27
+ const { resolveKey, fetchImpl, cfg } = deps
28
+ const { bytes, mime, lang, signal, models } = req
29
+
30
+ async function deepgram() {
31
+ const key = await resolveKey(cfg.deepgramKeyEnv)
32
+ if (!key) return { ok: false, provider: 'deepgram', reason: `no ${cfg.deepgramKeyEnv}` }
33
+ const model = pickModel(models, 'deepgram')
34
+ const url = `https://api.deepgram.com/v1/listen?model=${encodeURIComponent(model)}`
35
+ + `&language=${encodeURIComponent(lang)}&smart_format=true`
36
+ const res = await fetchImpl(url, {
37
+ method: 'POST',
38
+ headers: { authorization: `Token ${key}`, 'content-type': mime },
39
+ body: bytes,
40
+ signal,
41
+ })
42
+ if (!res.ok) throw new Error(`Deepgram HTTP ${res.status}`)
43
+ const data = await res.json()
44
+ const text = (data?.results?.channels?.[0]?.alternatives?.[0]?.transcript || '').trim()
45
+ return { ok: text.length > 0, provider: 'deepgram', text, reason: text ? '' : 'empty transcript' }
46
+ }
47
+
48
+ async function groq() {
49
+ const key = await resolveKey(cfg.groqKeyEnv)
50
+ if (!key) return { ok: false, provider: 'groq', reason: `no ${cfg.groqKeyEnv}` }
51
+ const form = new FormData()
52
+ form.append('file', new Blob([bytes], { type: mime }), fileName(mime))
53
+ form.append('model', pickModel(models, 'groq'))
54
+ form.append('language', lang)
55
+ form.append('response_format', 'json')
56
+ const res = await fetchImpl('https://api.groq.com/openai/v1/audio/transcriptions', {
57
+ method: 'POST',
58
+ headers: { authorization: `Bearer ${key}` },
59
+ body: form,
60
+ signal,
61
+ })
62
+ if (!res.ok) throw new Error(`Groq HTTP ${res.status}`)
63
+ const data = await res.json()
64
+ const text = (data?.text || '').trim()
65
+ return { ok: text.length > 0, provider: 'groq', text, reason: text ? '' : 'empty transcript' }
66
+ }
67
+
68
+ async function hf() {
69
+ const token = await resolveKey(cfg.hfTokenEnv)
70
+ if (!token) return { ok: false, provider: 'hf', reason: `no ${cfg.hfTokenEnv}` }
71
+ const model = pickModel(models, 'hf')
72
+ const res = await fetchImpl(`https://router.huggingface.co/hf-inference/models/${model}`, {
73
+ method: 'POST',
74
+ headers: { authorization: `Bearer ${token}`, 'content-type': mime, 'x-wait-for-model': 'true' },
75
+ body: bytes,
76
+ signal,
77
+ })
78
+ if (!res.ok) throw new Error(`HF HTTP ${res.status}`)
79
+ const data = await res.json()
80
+ const text = (data?.text || '').trim()
81
+ return { ok: text.length > 0, provider: 'hf', text, reason: text ? '' : 'empty transcript' }
82
+ }
83
+
84
+ // whisper.cpp принимает только WAV — на webm/opus из браузера его сервер
85
+ // отвечает "Invalid request". Перегоняем, если формат не WAV.
86
+ async function localWhisper() {
87
+ let sendBytes = bytes
88
+ let sendMime = mime
89
+ if (!mime.includes('wav')) {
90
+ if (typeof deps.toWav !== 'function') {
91
+ return { ok: false, provider: 'local-whisper', reason: 'local whisper needs WAV, no converter configured' }
92
+ }
93
+ try {
94
+ sendBytes = await deps.toWav(bytes)
95
+ sendMime = 'audio/wav'
96
+ } catch (e) {
97
+ return { ok: false, provider: 'local-whisper', reason: `local whisper: ${String(e && e.message || e)}` }
98
+ }
99
+ }
100
+ const form = new FormData()
101
+ form.append('file', new Blob([sendBytes], { type: sendMime }), fileName(sendMime))
102
+ form.append('language', lang)
103
+ form.append('response_format', 'json')
104
+ const res = await fetchImpl(cfg.whisperUrl, { method: 'POST', body: form, signal })
105
+ // whisper.cpp отвечает 400 с JSON-телом на внутренних сбоях (например, не
106
+ // смог декодировать аудио) — читаем причину, а не бросаем исключение.
107
+ if (!res.ok) {
108
+ let detail = `HTTP ${res.status}`
109
+ try { const e = await res.json(); if (e?.error) detail = e.error } catch { /* тело не json */ }
110
+ return { ok: false, provider: 'local-whisper', reason: `local whisper: ${detail}` }
111
+ }
112
+ const data = await res.json()
113
+ const text = (data?.text || '').trim()
114
+ return { ok: text.length > 0, provider: 'local-whisper', text, reason: text ? '' : 'empty transcript' }
115
+ }
116
+
117
+ return { deepgram, groq, hf, 'local-whisper': localWhisper }
118
+ }
package/lib/wav.js ADDED
@@ -0,0 +1,39 @@
1
+ // Перегон произвольного аудио в WAV 16 кГц моно.
2
+ //
3
+ // Локальный whisper.cpp принимает только WAV: на webm/opus, который пишет
4
+ // браузер, его сервер отвечает "Invalid request". API-провайдеры webm едят
5
+ // как есть, поэтому конвертация нужна ровно для локальной ноги цепочки.
6
+
7
+ import { spawn } from 'node:child_process'
8
+
9
+ /**
10
+ * @param bytes {Buffer|Uint8Array} исходное аудио в любом контейнере
11
+ * @param ffmpegBin {string} путь к ffmpeg
12
+ * @returns {Promise<Buffer>} WAV 16 кГц моно
13
+ */
14
+ export function toWav16k(bytes, ffmpegBin = 'ffmpeg') {
15
+ return new Promise((resolve, reject) => {
16
+ const proc = spawn(ffmpegBin, [
17
+ '-hide_banner', '-loglevel', 'error',
18
+ '-i', 'pipe:0',
19
+ '-ar', '16000', '-ac', '1',
20
+ '-f', 'wav', 'pipe:1',
21
+ ])
22
+ const out = []
23
+ const err = []
24
+ proc.stdout.on('data', (c) => out.push(c))
25
+ proc.stderr.on('data', (c) => err.push(c))
26
+ proc.on('error', (e) => reject(new Error(`ffmpeg unavailable: ${e.message}`)))
27
+ proc.on('close', (code) => {
28
+ if (code !== 0) {
29
+ reject(new Error(`ffmpeg exit ${code}: ${Buffer.concat(err).toString('utf8').slice(0, 200)}`))
30
+ return
31
+ }
32
+ const wav = Buffer.concat(out)
33
+ if (wav.length < 64) { reject(new Error('ffmpeg produced empty output')); return }
34
+ resolve(wav)
35
+ })
36
+ proc.stdin.on('error', () => { /* ffmpeg закрыл вход раньше — код возврата всё расскажет */ })
37
+ proc.stdin.end(Buffer.from(bytes))
38
+ })
39
+ }
package/package.json ADDED
@@ -0,0 +1,61 @@
1
+ {
2
+ "name": "@goodandready/dsh-voice",
3
+ "version": "0.3.0",
4
+ "description": "Voice input for DeepSeek Harness: dictation chunked by pauses and voice messages, each with its own provider fallback chain (Deepgram, Groq, HuggingFace, local whisper.cpp).",
5
+ "license": "MIT",
6
+ "type": "module",
7
+ "main": "./lib/index.js",
8
+ "exports": {
9
+ ".": "./lib/index.js",
10
+ "./client": "./lib/client.js",
11
+ "./package.json": "./package.json",
12
+ "./cordis.patch.yml": "./cordis.patch.yml"
13
+ },
14
+ "files": [
15
+ "lib/",
16
+ "cordis.patch.yml",
17
+ "README.md",
18
+ "LICENSE"
19
+ ],
20
+ "keywords": [
21
+ "dsh",
22
+ "dsh-plugin",
23
+ "deepseek-harness",
24
+ "voice",
25
+ "stt",
26
+ "dictation",
27
+ "whisper",
28
+ "deepgram",
29
+ "groq"
30
+ ],
31
+ "repository": {
32
+ "type": "git",
33
+ "url": "https://github.com/GooDAnDReaDY/dsh-voice.git"
34
+ },
35
+ "homepage": "https://github.com/GooDAnDReaDY/dsh-voice",
36
+ "bugs": {
37
+ "url": "https://github.com/GooDAnDReaDY/dsh-voice/issues"
38
+ },
39
+ "scripts": {
40
+ "test": "node --test test/*.test.mjs"
41
+ },
42
+ "dsh": {
43
+ "bundle": {
44
+ "patch": "./cordis.patch.yml"
45
+ },
46
+ "client": {
47
+ "platform": "web",
48
+ "inject": [
49
+ "@deepseek-ai/dsh-client-runtime",
50
+ "@deepseek-ai/dsh-client-ui-slots"
51
+ ]
52
+ }
53
+ },
54
+ "peerDependencies": {
55
+ "@deepseek-ai/cordis": "^4.0.1",
56
+ "@deepseek-ai/dsh-tools": "^0.1.0-rc.6",
57
+ "@deepseek-ai/dsh-credentials": "^0.1.0-rc.6",
58
+ "@deepseek-ai/dsh-host-webserver": "^0.1.0-rc.6",
59
+ "@deepseek-ai/schemastery": "^3.18.1"
60
+ }
61
+ }