@goodandready/dsh-tts 0.3.16 → 0.3.20
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +1 -1
- package/README.md +192 -87
- package/README.ru.md +195 -0
- package/README.zh.md +190 -0
- package/lib/client/worklet.js +68 -1
- package/lib/client.js +382 -8
- package/lib/engines/f5.js +43 -2
- package/lib/engines/kokoro.js +29 -2
- package/lib/engines/manager.js +235 -0
- package/lib/engines/types.js +47 -2
- package/lib/index.js +211 -13
- package/lib/text.js +31 -2
- package/package.json +4 -4
package/LICENSE
CHANGED
package/README.md
CHANGED
|
@@ -1,100 +1,205 @@
|
|
|
1
|
-
# dsh-tts
|
|
2
|
-
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
1
|
+
# 📦 @goodandready/dsh-tts
|
|
2
|
+
|
|
3
|
+
<div align="center">
|
|
4
|
+
|
|
5
|
+
<h3>Multi-Provider Text-to-Speech Voice Synthesis with Local Neural Engines, Sub-300ms Streaming, IT Dictionary & Messenger Integration for DeepSeek Harness</h3>
|
|
6
|
+
|
|
7
|
+
<p align="center">
|
|
8
|
+
<a href="https://www.npmjs.com/package/@goodandready/dsh-tts"><img src="https://img.shields.io/npm/v/@goodandready/dsh-tts.svg?style=for-the-badge&color=6366f1&labelColor=1e1b4b" alt="npm version"></a>
|
|
9
|
+
<a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-10b981.svg?style=for-the-badge&color=10b981&labelColor=064e3b" alt="license"></a>
|
|
10
|
+
<a href="https://github.com/topics/dsh-plugin"><img src="https://img.shields.io/badge/DSH-Plugin-8b5cf6.svg?style=for-the-badge&labelColor=2e1065" alt="DSH Plugin"></a>
|
|
11
|
+
<a href="https://nodejs.org"><img src="https://img.shields.io/badge/Node-20%2B-f59e0b.svg?style=for-the-badge&labelColor=451a03" alt="Node version"></a>
|
|
12
|
+
</p>
|
|
13
|
+
|
|
14
|
+
<p align="center">
|
|
15
|
+
<a href="https://goodandready.app/"><img src="https://img.shields.io/badge/All_Author_Projects-goodandready.app-ff4500.svg?style=for-the-badge&logo=rocket&logoColor=white&labelColor=1a1a2e" alt="All Author Projects"></a>
|
|
16
|
+
</p>
|
|
17
|
+
|
|
18
|
+
<p align="center">
|
|
19
|
+
<a href="README.md"><b>🇬🇧 English</b></a> •
|
|
20
|
+
<a href="README.ru.md"><b>🇷🇺 Русский</b></a> •
|
|
21
|
+
<a href="README.zh.md"><b>🇨🇳 中文说明</b></a>
|
|
22
|
+
</p>
|
|
23
|
+
|
|
24
|
+
</div>
|
|
25
|
+
|
|
26
|
+
---
|
|
27
|
+
|
|
28
|
+
## ⚡ Overview
|
|
29
|
+
|
|
30
|
+
**`dsh-tts`** provides robust, lifelike spoken voice synthesis for assistant replies in the **DeepSeek Harness** Web UI. When **Speak agent replies** is enabled, each finished assistant turn or real-time streaming chunk is synthesized on the host and streamed directly to the browser.
|
|
31
|
+
|
|
32
|
+
API keys never reach client browsers: synthesis is executed entirely on the host backend across **independent multi-provider fallback chains**, including completely offline neural models (Kokoro-82M and F5-TTS).
|
|
33
|
+
|
|
34
|
+
```mermaid
|
|
35
|
+
graph LR
|
|
36
|
+
subgraph Input [Assistant Message]
|
|
37
|
+
Reply[💬 Agent Reply Text] --> Scrub[Smart Text Scrubbing & IT Dictionary]
|
|
38
|
+
end
|
|
39
|
+
|
|
40
|
+
subgraph Stream [Low-Latency Streaming]
|
|
41
|
+
Scrub --> SSE[SSE /dsh-tts/stream]
|
|
42
|
+
SSE --> Worklet[AudioWorklet PCM Processor]
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
subgraph Cache [Performance Layer]
|
|
46
|
+
Scrub --> LRU{Disk LRU Cache}
|
|
47
|
+
LRU -->|Cache Hit| Play[Immediate Audio Playback]
|
|
48
|
+
end
|
|
49
|
+
|
|
50
|
+
subgraph Fallback [TTS Provider Fallback Chain]
|
|
51
|
+
LRU -->|Cache Miss| Chain{Active Chain}
|
|
52
|
+
Chain -->|Priority 1| P1[Kokoro / F5-TTS Local Offline]
|
|
53
|
+
Chain -.->|Cloud Neural| P2[ElevenLabs / OpenAI / CosyVoice]
|
|
54
|
+
Chain -.->|Free Cloud / Edge| P3[EdgeTTS / SiliconFlow]
|
|
55
|
+
Chain -.->|System Fallback| P4[Local Piper / eSpeak NG]
|
|
56
|
+
end
|
|
57
|
+
|
|
58
|
+
subgraph Output [Delivery & Integrations]
|
|
59
|
+
P1 --> Store[Save to Cache]
|
|
60
|
+
P2 --> Store
|
|
61
|
+
P3 --> Store
|
|
62
|
+
P4 --> Store
|
|
63
|
+
Store --> Play
|
|
64
|
+
Store --> Msg[Telegram / Discord via dsh-messenger-gateway]
|
|
65
|
+
end
|
|
66
|
+
|
|
67
|
+
style Input fill:#1e1e2e,stroke:#89b4fa,stroke-width:2px,color:#cdd6f4
|
|
68
|
+
style Stream fill:#181825,stroke:#89dceb,stroke-width:2px,color:#cdd6f4
|
|
69
|
+
style Cache fill:#181825,stroke:#cba6f7,stroke-width:2px,color:#cdd6f4
|
|
70
|
+
style Fallback fill:#11111b,stroke:#a6e3a1,stroke-width:2px,color:#cdd6f4
|
|
71
|
+
style Output fill:#181825,stroke:#f38ba8,stroke-width:2px,color:#cdd6f4
|
|
72
|
+
```
|
|
9
73
|
|
|
10
|
-
|
|
74
|
+
---
|
|
75
|
+
|
|
76
|
+
## 🚀 Key Features
|
|
77
|
+
|
|
78
|
+
### 1. 📴 Offline Local Neural Engines (Kokoro CPU & F5-TTS GPU)
|
|
79
|
+
* **Kokoro-82M (CPU)**: 82M-parameter lightweight neural model running locally on CPU via ONNX Runtime. High-speed synthesis with zero cloud dependencies.
|
|
80
|
+
* **F5-TTS (GPU)**: Zero-shot diffusion transformer voice synthesis running on NVIDIA GPUs via a local inference daemon.
|
|
81
|
+
* **ModelManager UI**: Direct manual installation in settings with real-time download progress bar, SHA-256 validation, and deletion. No silent or automatic multi-gigabyte downloads.
|
|
82
|
+
|
|
83
|
+
### 2. ⚡ Real-Time Streaming Audio (< 300 ms Latency)
|
|
84
|
+
* **AudioWorklet (`TTSWorklet`)**: High-performance Web Audio Worklet processor playing seamless Float32Array PCM chunks at 24 kHz without audible clicks or buffer underruns.
|
|
85
|
+
* **Server-Sent Events (SSE)**: Dedicated `/dsh-tts/stream` route delivering synthesized chunks to connected browsers instantly.
|
|
86
|
+
|
|
87
|
+
### 3. 🎙️ Voice Duplex & VAD Barge-In (with `@goodandready/dsh-voice`)
|
|
88
|
+
* **Full-Duplex Conversation**: Automatic voice reply synthesis upon completion of speech dictation.
|
|
89
|
+
* **VAD Barge-In**: Immediately mutes assistant speech playback when user voice activity is detected.
|
|
90
|
+
* **Installation Guard**: If `@goodandready/dsh-voice` is not present, settings controls are disabled with an explicit instruction banner (`dsh plugin --profile web add @goodandready/dsh-voice`).
|
|
91
|
+
|
|
92
|
+
### 4. 📚 Built-in IT Terminology Pronunciation Dictionary
|
|
93
|
+
* **Pre-configured Lexicon**: Correct phonetic pronunciation for common technical abbreviations and developer terms:
|
|
94
|
+
- `SQL` $\rightarrow$ "сиквел"
|
|
95
|
+
- `Nginx` $\rightarrow$ "энджинкс"
|
|
96
|
+
- `Kubernetes` / `K8s` $\rightarrow$ "кубернетис"
|
|
97
|
+
- `Docker` $\rightarrow$ "докер", `API` $\rightarrow$ "апи", `JSON` $\rightarrow$ "джейсон", `YAML` $\rightarrow$ "ямл"
|
|
98
|
+
- `GUI`, `CLI`, `CI/CD`, `PR`, `Regex`, `OAuth`, `HTTP`, `HTTPS`, `CPU`, `GPU`, `RAM`
|
|
99
|
+
* **Interactive UI Editor**: Edit rules, preview phonetic substitutions with the **▶ Listen** button, and populate standard IT terms with one click.
|
|
100
|
+
|
|
101
|
+
### 5. 👥 Multi-Agent Personas & Subagent Voice Overrides
|
|
102
|
+
* Assign distinct voices, providers, models, and audio chimes to individual subagents (e.g. `coder`, `reviewer`, `planner`, `tester`).
|
|
103
|
+
* Automatically matches incoming assistant turn events (`session.agent` or `message.agent`).
|
|
104
|
+
|
|
105
|
+
### 6. 💬 Messenger Voice Notes Integration (with `@goodandready/dsh-messenger-gateway`)
|
|
106
|
+
* Generates voice audio for Telegram and Discord bot replies via `POST /dsh-tts/speak`.
|
|
107
|
+
* Protective dependency check with installation hint when gateway plugin is missing.
|
|
108
|
+
|
|
109
|
+
---
|
|
110
|
+
|
|
111
|
+
## 🛠️ Complete Supported Providers Matrix (18 Backends)
|
|
112
|
+
|
|
113
|
+
| Provider Key | Service Backend | Default Model | Default Voice | Credential Ref | Features & Notes |
|
|
114
|
+
|---|---|---|---|---|---|
|
|
115
|
+
| `kokoro` | Local Kokoro-82M ONNX | `hexgrad/Kokoro-82M` | `af_bella` | *None* | **100% offline CPU neural synthesis** |
|
|
116
|
+
| `f5` | Local F5-TTS GPU Daemon | `F5-TTS` | Default | *None* | **High-fidelity GPU zero-shot voice synthesis** |
|
|
117
|
+
| `elevenlabs` | ElevenLabs API | `eleven_multilingual_v2` | `Rachel` | `ELEVENLABS_API_KEY` | Ultra-realistic, emotional nuance |
|
|
118
|
+
| `openai` | OpenAI Audio | `gpt-4o-mini-tts` / `tts-1` | `alloy` | `OPENAI_API_KEY` | High-quality industry standard |
|
|
119
|
+
| `edge` | Microsoft Edge Online | `ru-RU-SvetlanaNeural` | `ru-RU-SvetlanaNeural` | *None* | **Free, high-fidelity neural TTS without API keys** |
|
|
120
|
+
| `siliconflow` | SiliconFlow CosyVoice | `FunAudioLLM/CosyVoice2-0.5B` | Default | `SILICONFLOW_API_KEY` | State-of-the-art CosyVoice2 neural engine |
|
|
121
|
+
| `deepinfra` | DeepInfra Kokoro | `hexgrad/Kokoro-82M` | Default | `DEEPINFRA_API_KEY` | Fast open-weights Kokoro synthesis |
|
|
122
|
+
| `fireworks` | Fireworks AI | `kokoro` | Default | `FIREWORKS_API_KEY` | Ultra-low latency Kokoro inference |
|
|
123
|
+
| `minimax` | MiniMax Speech | `speech-01-turbo` | Default | `MINIMAX_API_KEY` | High-expressiveness neural voice |
|
|
124
|
+
| `mimo` | Xiaomi MiMo Audio | `mimo-v2.5-tts` | Default | `MIMO_API_KEY` | Low-latency streaming TTS |
|
|
125
|
+
| `google` | Google Cloud TTS | `gemini-2.5-flash-preview-tts` | Language default | `GEMINI_API_KEY` | Multilingual Google Gemini voice synthesis |
|
|
126
|
+
| `azure` | Azure Cognitive Speech | `en-US-JennyNeural` | Region default | `AZURE_SPEECH_KEY` | Enterprise neural synthesis |
|
|
127
|
+
| `deepgram` | Deepgram Aura | `aura-asteria-en` | `asteria` | `DEEPGRAM_API_KEY` | Ultra-low latency voice output |
|
|
128
|
+
| `groq` | Groq TTS | `playai-tts` | `default` | `GROQ_API_KEY` | Near-instant inference speed |
|
|
129
|
+
| `openrouter` | OpenRouter Audio | `openai/gpt-4o-mini-tts` | `alloy` | `OPENROUTER_API_KEY` | Unified router access |
|
|
130
|
+
| `custom` | Custom OpenAI-compatible | Configurable | Configurable | `CUSTOM_TTS_API_KEY` | Any `/v1/audio/speech` endpoint |
|
|
131
|
+
| `piper` | Local Piper ONNX | Local ONNX weights | Model default | *None* | 100% offline neural engine |
|
|
132
|
+
| `espeak` | Local eSpeak NG | System synth | `ru` / `en` | *None* | 100% offline lightweight fallback |
|
|
133
|
+
|
|
134
|
+
---
|
|
135
|
+
|
|
136
|
+
## 🧹 Smart Text Scrubbing & Formatting Engine
|
|
137
|
+
|
|
138
|
+
Before text reaches speech synthesizers, `dsh-tts` intelligently sanitizes and filters the message so the assistant doesn't read out syntax noise:
|
|
139
|
+
* **Fenced Code Blocks**: Spoken as *"code block, N lines"* / *"блок кода, N строк"*.
|
|
140
|
+
* **Markdown Tables**: Spoken as *"table, N rows"* / *"таблица, N строк"*.
|
|
141
|
+
* **Summary Intros**: Spoken as *"Summary of the reply"* / *"Пересказ ответа"*.
|
|
142
|
+
* **Narration Filters**: Skip asterisk actions (`*smiles*`), narrate quotes only, and apply custom regex removal.
|
|
143
|
+
|
|
144
|
+
---
|
|
145
|
+
|
|
146
|
+
## 📦 Quick Installation
|
|
11
147
|
|
|
12
148
|
```bash
|
|
13
|
-
# From npm (after a GitHub/npm release):
|
|
14
149
|
dsh plugin --profile web add @goodandready/dsh-tts
|
|
15
|
-
|
|
16
|
-
# From a local checkout:
|
|
17
|
-
dsh plugin --profile web add file:/path/to/dsh-tts
|
|
18
150
|
```
|
|
19
151
|
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
152
|
+
> [!IMPORTANT]
|
|
153
|
+
> Restart DSH Web UI after installation (`systemctl --user restart dsh-web`) and refresh your browser tab.
|
|
154
|
+
|
|
155
|
+
---
|
|
156
|
+
|
|
157
|
+
## ⚙️ Configuration Recipes (`settings.yaml`)
|
|
158
|
+
|
|
159
|
+
```yaml
|
|
160
|
+
dsh-tts:
|
|
161
|
+
speakReplies: true
|
|
162
|
+
enableLocalEngines: true
|
|
163
|
+
kokoroEnabled: true
|
|
164
|
+
streamingEnabled: true
|
|
165
|
+
enableItDictionary: true
|
|
166
|
+
voiceDuplexEnabled: true
|
|
167
|
+
vadBargeIn: true
|
|
168
|
+
messengerTtsEnabled: true
|
|
169
|
+
cache: true
|
|
170
|
+
cacheMaxMb: 150
|
|
171
|
+
autoDetect: true
|
|
172
|
+
chain:
|
|
173
|
+
- provider: kokoro
|
|
174
|
+
- provider: edge
|
|
175
|
+
voice: ru-RU-SvetlanaNeural
|
|
176
|
+
- provider: openai
|
|
177
|
+
model: tts-1
|
|
178
|
+
voice: alloy
|
|
179
|
+
roles:
|
|
180
|
+
coder:
|
|
181
|
+
provider: openai
|
|
182
|
+
voice: onyx
|
|
183
|
+
reviewer:
|
|
184
|
+
provider: edge
|
|
185
|
+
voice: ru-RU-DmitryNeural
|
|
46
186
|
```
|
|
47
187
|
|
|
48
|
-
|
|
49
|
-
a key-rotation plugin.
|
|
50
|
-
|
|
51
|
-
## Providers
|
|
52
|
-
|
|
53
|
-
| Key | Service | Default model / voice | Credential |
|
|
54
|
-
|---|---|---|---|
|
|
55
|
-
| `openai` | OpenAI Audio Speech | `gpt-4o-mini-tts` / `alloy` | `OPENAI_API_KEY` |
|
|
56
|
-
| `elevenlabs` | ElevenLabs | `eleven_multilingual_v2` | `ELEVENLABS_API_KEY` |
|
|
57
|
-
| `google` | Gemini TTS | `gemini-2.5-flash-preview-tts` / `Kore` | `GEMINI_API_KEY` |
|
|
58
|
-
| `azure` | Azure Speech | neural voice + region | `AZURE_SPEECH_KEY` + region |
|
|
59
|
-
| `groq` | Groq PlayAI | `playai-tts` | `GROQ_API_KEY` |
|
|
60
|
-
| `deepgram` | Deepgram Aura | `aura-asteria-en` | `DEEPGRAM_API_KEY` |
|
|
61
|
-
| `openrouter` | OpenRouter `/audio/speech` | `openai/gpt-4o-mini-tts-2025-12-15` | `OPENROUTER_API_KEY` |
|
|
62
|
-
| `edge` | Microsoft Edge Neural via `edge-tts` CLI | `ru-RU-SvetlanaNeural` | none |
|
|
63
|
-
| `piper` | local Piper | path to an ONNX model | none |
|
|
64
|
-
| `espeak` | eSpeak NG | voice `ru` | none |
|
|
65
|
-
|
|
66
|
-
Default chain: `edge` → `piper` → `espeak`. Deepgram Aura does not list Russian;
|
|
67
|
-
keep it below a Russian-capable voice if you speak Russian.
|
|
68
|
-
|
|
69
|
-
### Free local / CLI providers
|
|
70
|
-
|
|
71
|
-
```bash
|
|
72
|
-
pip install edge-tts # provides the `edge-tts` binary
|
|
73
|
-
# piper: install the Piper binary and point Settings at your .onnx model
|
|
74
|
-
sudo apt install espeak-ng # or the equivalent package on your OS
|
|
75
|
-
```
|
|
76
|
-
|
|
77
|
-
## Tool
|
|
78
|
-
|
|
79
|
-
`speak_text(text)` — synthesize with the same chain. The tool result is a short
|
|
80
|
-
status line; it does not inject audio into the model history.
|
|
81
|
-
|
|
82
|
-
## Routes
|
|
188
|
+
---
|
|
83
189
|
|
|
84
|
-
|
|
85
|
-
|---|---|
|
|
86
|
-
| `GET /dsh-tts/status` | chain and whether auto-speak is on |
|
|
87
|
-
| `GET` or `PUT /dsh-tts/config` | settings plus configured/writable/ref per cloud provider. PUT may include keys; values are stored as credentials and never echoed. |
|
|
88
|
-
| `PUT` or `DELETE /dsh-tts/credential` | writes or clears one host credential immediately |
|
|
89
|
-
| `GET /dsh-tts/pending?after=` | audio produced for finished replies |
|
|
90
|
-
| `POST /dsh-tts/speak` | `{text}` -> `{ok, provider, mime, audioBase64}` |
|
|
190
|
+
## 🤖 HTTP Endpoints Reference
|
|
91
191
|
|
|
92
|
-
|
|
192
|
+
* `GET /dsh-tts/stream` — Real-time Server-Sent Events (SSE) audio streaming.
|
|
193
|
+
* `POST /dsh-tts/speak` — `{ text, voice?, model? }` → Returns synthesized audio.
|
|
194
|
+
* `POST /dsh-tts/preview` — `{ provider, model, voice, text? }` → Test voice playback in UI.
|
|
195
|
+
* `GET /dsh-tts/models/status` — Reports local Kokoro and F5-TTS model installation states.
|
|
196
|
+
* `POST /dsh-tts/models/install` — `{ engine: 'kokoro' | 'f5' }` → Starts HuggingFace model download.
|
|
197
|
+
* `DELETE /dsh-tts/models/delete` — `{ engine: 'kokoro' | 'f5' }` → Removes local model files.
|
|
198
|
+
* `GET /dsh-tts/integrations` — Status of sibling plugins (`dsh-voice`, `dsh-messenger-gateway`).
|
|
199
|
+
* `GET /dsh-tts/status` — Returns active chain state, cache statistics, and engine readiness.
|
|
93
200
|
|
|
94
|
-
|
|
95
|
-
- Node 20+
|
|
96
|
-
- Optional: `edge-tts`, `piper`, `espeak-ng` for the free providers
|
|
201
|
+
---
|
|
97
202
|
|
|
98
|
-
## License
|
|
203
|
+
## 📄 License
|
|
99
204
|
|
|
100
|
-
MIT
|
|
205
|
+
MIT © [GooDAnDReaDY](https://github.com/GooDAnDReaDY)
|
package/README.ru.md
ADDED
|
@@ -0,0 +1,195 @@
|
|
|
1
|
+
# 📦 @goodandready/dsh-tts
|
|
2
|
+
|
|
3
|
+
<div align="center">
|
|
4
|
+
|
|
5
|
+
<h3>Многопровайдерная озвучка ответов агента с локальными нейродвижками, потоковым звуком (<300 мс), IT-словарём и интеграцией с мессенджерами для DeepSeek Harness</h3>
|
|
6
|
+
|
|
7
|
+
<p align="center">
|
|
8
|
+
<a href="https://www.npmjs.com/package/@goodandready/dsh-tts"><img src="https://img.shields.io/npm/v/@goodandready/dsh-tts.svg?style=for-the-badge&color=6366f1&labelColor=1e1b4b" alt="npm version"></a>
|
|
9
|
+
<a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-10b981.svg?style=for-the-badge&color=10b981&labelColor=064e3b" alt="license"></a>
|
|
10
|
+
<a href="https://github.com/topics/dsh-plugin"><img src="https://img.shields.io/badge/DSH-Plugin-8b5cf6.svg?style=for-the-badge&labelColor=2e1065" alt="DSH Plugin"></a>
|
|
11
|
+
<a href="https://nodejs.org"><img src="https://img.shields.io/badge/Node-20%2B-f59e0b.svg?style=for-the-badge&labelColor=451a03" alt="Node version"></a>
|
|
12
|
+
</p>
|
|
13
|
+
|
|
14
|
+
<p align="center">
|
|
15
|
+
<a href="https://goodandready.app/"><img src="https://img.shields.io/badge/Все_проекты_автора-goodandready.app-ff4500.svg?style=for-the-badge&logo=rocket&logoColor=white&labelColor=1a1a2e" alt="Все проекты автора"></a>
|
|
16
|
+
</p>
|
|
17
|
+
|
|
18
|
+
<p align="center">
|
|
19
|
+
<a href="README.md"><b>🇬🇧 English</b></a> •
|
|
20
|
+
<a href="README.ru.md"><b>🇷🇺 Русский</b></a> •
|
|
21
|
+
<a href="README.zh.md"><b>🇨🇳 中文说明</b></a>
|
|
22
|
+
</p>
|
|
23
|
+
|
|
24
|
+
</div>
|
|
25
|
+
|
|
26
|
+
---
|
|
27
|
+
|
|
28
|
+
## ⚡ Обзор
|
|
29
|
+
|
|
30
|
+
**`dsh-tts`** обеспечивает качественное голосовое озвучивание ответов ассистента в веб-интерфейсе **DeepSeek Harness**. При включённой опции **«Читать ответы агента»**, каждая готовая реплика или потоковый фрагмент синтезируется на хосте и мгновенно воспроизводится в браузере.
|
|
31
|
+
|
|
32
|
+
API-ключи никогда не передаются в браузер: синтез аудио выполняется на стороне хоста через **независимые цепочки отказоустойчивости (фолбеков)**, включая полностью локальные оффлайн-нейросети (Kokoro-82M и F5-TTS).
|
|
33
|
+
|
|
34
|
+
```mermaid
|
|
35
|
+
graph LR
|
|
36
|
+
subgraph Input [Ответ ассистента]
|
|
37
|
+
Reply[💬 Текст ответа агента] --> Scrub[Умная очистка и IT-словарь]
|
|
38
|
+
end
|
|
39
|
+
|
|
40
|
+
subgraph Stream [Потоковый звук < 300 мс]
|
|
41
|
+
Scrub --> SSE[SSE /dsh-tts/stream]
|
|
42
|
+
SSE --> Worklet[AudioWorklet PCM процессор]
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
subgraph Cache [Слой производительности]
|
|
46
|
+
Scrub --> LRU{Дисковый LRU кэш}
|
|
47
|
+
LRU -->|Попадание в кэш| Play[Мгновенное воспроизведение]
|
|
48
|
+
end
|
|
49
|
+
|
|
50
|
+
subgraph Fallback [Цепочка фолбеков TTS]
|
|
51
|
+
LRU -->|Промах кэша| Chain{Активная цепочка}
|
|
52
|
+
Chain -->|1-й приоритет| P1[Kokoro / F5-TTS Локально]
|
|
53
|
+
Chain -.->|Облачные нейросети| P2[ElevenLabs / OpenAI / CosyVoice]
|
|
54
|
+
Chain -.->|Бесплатные облака| P3[EdgeTTS / SiliconFlow]
|
|
55
|
+
Chain -.->|Системный фолбек| P4[Локальный Piper / eSpeak NG]
|
|
56
|
+
end
|
|
57
|
+
|
|
58
|
+
subgraph Output [Вывод и интеграции]
|
|
59
|
+
P1 --> Store[Сохранение в кэш]
|
|
60
|
+
P2 --> Store
|
|
61
|
+
P3 --> Store
|
|
62
|
+
P4 --> Store
|
|
63
|
+
Store --> Play
|
|
64
|
+
Store --> Msg[Telegram / Discord через dsh-messenger-gateway]
|
|
65
|
+
end
|
|
66
|
+
|
|
67
|
+
style Input fill:#1e1e2e,stroke:#89b4fa,stroke-width:2px,color:#cdd6f4
|
|
68
|
+
style Stream fill:#181825,stroke:#89dceb,stroke-width:2px,color:#cdd6f4
|
|
69
|
+
style Cache fill:#181825,stroke:#cba6f7,stroke-width:2px,color:#cdd6f4
|
|
70
|
+
style Fallback fill:#11111b,stroke:#a6e3a1,stroke-width:2px,color:#cdd6f4
|
|
71
|
+
style Output fill:#181825,stroke:#f38ba8,stroke-width:2px,color:#cdd6f4
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
---
|
|
75
|
+
|
|
76
|
+
## 🚀 Ключевые возможности
|
|
77
|
+
|
|
78
|
+
### 1. 📴 Полностью локальные оффлайн-движки (Kokoro CPU и F5-TTS GPU)
|
|
79
|
+
* **Kokoro-82M (CPU)**: Компактная нейросетевая модель (82 млн параметров), работающая локально на CPU через ONNX Runtime. Быстрый синтез без внешних API и интернета.
|
|
80
|
+
* **F5-TTS (GPU)**: Диффузионный трансформер нулевого выстрела (zero-shot) на видеокартах NVIDIA через выделенный локальный демон инференса.
|
|
81
|
+
* **ModelManager в UI**: Ручная загрузка моделей прямо из настроек плагина с отображением процентов и прогресс-бара, проверкой SHA-256 и кнопкой удаления. Никаких скрытых или автозагрузок гигабайтных весов.
|
|
82
|
+
|
|
83
|
+
### 2. ⚡ Потоковое воспроизведение звука с задержкой < 300 мс
|
|
84
|
+
* **AudioWorklet (`TTSWorklet`)**: Высокопроизводительный процессор веб-аудио с бесшовным кольцевым буфером для воспроизведения PCM-чанков (24 кГц) без щелчков и пауз.
|
|
85
|
+
* **Server-Sent Events (SSE)**: Роут `/dsh-tts/stream` передаёт синтезированные куски речи сразу в браузер, исключая задержки традиционного полинга.
|
|
86
|
+
|
|
87
|
+
### 3. 🎙️ Голосовой диалог и VAD Barge-In (в связке с `@goodandready/dsh-voice`)
|
|
88
|
+
* **Voice-to-Voice дуплекс**: Автоматический запуск озвучки ответа сразу после завершения голосовой диктовки.
|
|
89
|
+
* **VAD Barge-In**: Мгновенное глушение речи ассистента, как только пользователь начинает говорить в микрофон.
|
|
90
|
+
* **Защитная блокировка**: Если плагин `@goodandready/dsh-voice` не установлен, переключатели блокируются (`disabled`), а в интерфейсе отображается подсказка: `dsh plugin --profile web add @goodandready/dsh-voice`.
|
|
91
|
+
|
|
92
|
+
### 4. 📚 Предустановленный словарь IT-произношения
|
|
93
|
+
* **Корректная транскрипция терминов**: Автоматическая замена аббревиатур и названий:
|
|
94
|
+
- `SQL` $\rightarrow$ «сиквел»
|
|
95
|
+
- `Nginx` $\rightarrow$ «энджинкс»
|
|
96
|
+
- `Kubernetes` / `K8s` $\rightarrow$ «кубернетис»
|
|
97
|
+
- `Docker` $\rightarrow$ «докер», `API` $\rightarrow$ «апи», `JSON` $\rightarrow$ «джейсон», `YAML` $\rightarrow$ «ямл»
|
|
98
|
+
- `GUI`, `CLI`, `CI/CD`, `PR`, `Regex`, `OAuth`, `HTTP`, `HTTPS`, `CPU`, `GPU`, `RAM`
|
|
99
|
+
* **Интерактивный редактор в UI**: Правка правил, кнопка **«▶ Прослушать»** для каждого слова и кнопка **«Вставить IT-термины в таблицу»** в один клик.
|
|
100
|
+
|
|
101
|
+
### 5. 👥 Персонализация голосов субагентов (Multi-Agent Personas)
|
|
102
|
+
* Назначение индивидуального голоса, провайдера, модели и сигнала (chime) для каждого субагента (`coder`, `reviewer`, `planner`, `tester` и др.).
|
|
103
|
+
* Автоматический перехват имени агента из событий сессии (`session.agent` или `event.data.agent`).
|
|
104
|
+
|
|
105
|
+
### 6. 💬 Голосовые сообщения в мессенджеры (`@goodandready/dsh-messenger-gateway`)
|
|
106
|
+
* Озвучивание ответов и отправка войсов в Telegram и Discord через роут `POST /dsh-tts/speak`.
|
|
107
|
+
* Защитная проверка наличия плагина-шлюза с подсказкой по установке.
|
|
108
|
+
|
|
109
|
+
---
|
|
110
|
+
|
|
111
|
+
## 🛠️ Матрица поддерживаемых провайдеров (18 бэкендов)
|
|
112
|
+
|
|
113
|
+
| Ключ провайдера | Сервис | Модель по умолчанию | Голос по умолчанию | Имя ключа (credential) | Особенности |
|
|
114
|
+
|---|---|---|---|---|---|
|
|
115
|
+
| `kokoro` | Локальный Kokoro-82M ONNX | `hexgrad/Kokoro-82M` | `af_bella` | *Не требуется* | **100% локальный нейросинтез на CPU** |
|
|
116
|
+
| `f5` | Локальный F5-TTS GPU | `F5-TTS` | По умолчанию | *Не требуется* | **Качественный zero-shot синтез на NVIDIA GPU** |
|
|
117
|
+
| `elevenlabs` | ElevenLabs API | `eleven_multilingual_v2` | `Rachel` | `ELEVENLABS_API_KEY` | Реалистичный эмоциональный синтез |
|
|
118
|
+
| `openai` | OpenAI Audio | `gpt-4o-mini-tts` / `tts-1` | `alloy` | `OPENAI_API_KEY` | Эталонное качество индустрии |
|
|
119
|
+
| `edge` | Microsoft Edge Online | `ru-RU-SvetlanaNeural` | `ru-RU-SvetlanaNeural` | *Не требуется* | **Бесплатный нейросинтез без API-ключей** |
|
|
120
|
+
| `siliconflow` | SiliconFlow CosyVoice | `FunAudioLLM/CosyVoice2-0.5B` | По умолчанию | `SILICONFLOW_API_KEY` | Нейродвижок CosyVoice2 |
|
|
121
|
+
| `deepinfra` | DeepInfra Kokoro | `hexgrad/Kokoro-82M` | По умолчанию | `DEEPINFRA_API_KEY` | Быстрый Kokoro в облаке |
|
|
122
|
+
| `fireworks` | Fireworks AI | `kokoro` | По умолчанию | `FIREWORKS_API_KEY` | Минимальная задержка инференса |
|
|
123
|
+
| `minimax` | MiniMax Speech | `speech-01-turbo` | По умолчанию | `MINIMAX_API_KEY` | Выразительный эмоциональный голос |
|
|
124
|
+
| `mimo` | Xiaomi MiMo Audio | `mimo-v2.5-tts` | По умолчанию | `MIMO_API_KEY` | Потоковый синтез речи |
|
|
125
|
+
| `google` | Google Cloud TTS | `gemini-2.5-flash-preview-tts` | По умолчанию | `GEMINI_API_KEY` | Многоязычный Gemini TTS |
|
|
126
|
+
| `azure` | Azure Cognitive Speech | `en-US-JennyNeural` | По умолчанию | `AZURE_SPEECH_KEY` | Корпоративный нейросинтез |
|
|
127
|
+
| `deepgram` | Deepgram Aura | `aura-asteria-en` | `asteria` | `DEEPGRAM_API_KEY` | Сверхнизкая задержка ответа |
|
|
128
|
+
| `groq` | Groq TTS | `playai-tts` | `default` | `GROQ_API_KEY` | Мгновенная генерация |
|
|
129
|
+
| `openrouter` | OpenRouter Audio | `openai/gpt-4o-mini-tts` | `alloy` | `OPENROUTER_API_KEY` | Доступ через единый роутер |
|
|
130
|
+
| `custom` | OpenAI-совместимый API | Настраивается | Настраивается | `CUSTOM_TTS_API_KEY` | Любой эндпоинт `/v1/audio/speech` |
|
|
131
|
+
| `piper` | Локальный Piper ONNX | Локальные веса | По умолчанию | *Не требуется* | Полностью оффлайн движок |
|
|
132
|
+
| `espeak` | Локальный eSpeak NG | Системный | `ru` / `en` | *Не требуется* | Легковесный системный синтезатор |
|
|
133
|
+
|
|
134
|
+
---
|
|
135
|
+
|
|
136
|
+
## 📦 Быстрая установка
|
|
137
|
+
|
|
138
|
+
```bash
|
|
139
|
+
dsh plugin --profile web add @goodandready/dsh-tts
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
> [!IMPORTANT]
|
|
143
|
+
> После установки перезапустите Web UI (`systemctl --user restart dsh-web`) и обновите вкладку браузера.
|
|
144
|
+
|
|
145
|
+
---
|
|
146
|
+
|
|
147
|
+
## ⚙️ Пример конфигурации (`settings.yaml`)
|
|
148
|
+
|
|
149
|
+
```yaml
|
|
150
|
+
dsh-tts:
|
|
151
|
+
speakReplies: true
|
|
152
|
+
enableLocalEngines: true
|
|
153
|
+
kokoroEnabled: true
|
|
154
|
+
streamingEnabled: true
|
|
155
|
+
enableItDictionary: true
|
|
156
|
+
voiceDuplexEnabled: true
|
|
157
|
+
vadBargeIn: true
|
|
158
|
+
messengerTtsEnabled: true
|
|
159
|
+
cache: true
|
|
160
|
+
cacheMaxMb: 150
|
|
161
|
+
autoDetect: true
|
|
162
|
+
chain:
|
|
163
|
+
- provider: kokoro
|
|
164
|
+
- provider: edge
|
|
165
|
+
voice: ru-RU-SvetlanaNeural
|
|
166
|
+
- provider: openai
|
|
167
|
+
model: tts-1
|
|
168
|
+
voice: alloy
|
|
169
|
+
roles:
|
|
170
|
+
coder:
|
|
171
|
+
provider: openai
|
|
172
|
+
voice: onyx
|
|
173
|
+
reviewer:
|
|
174
|
+
provider: edge
|
|
175
|
+
voice: ru-RU-DmitryNeural
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
---
|
|
179
|
+
|
|
180
|
+
## 🤖 HTTP роуты API
|
|
181
|
+
|
|
182
|
+
* `GET /dsh-tts/stream` — Потоковая передача аудио по SSE в реальном времени.
|
|
183
|
+
* `POST /dsh-tts/speak` — `{ text, voice?, model? }` → Возвращает аудиофайл для воспроизведения.
|
|
184
|
+
* `POST /dsh-tts/preview` — `{ provider, model, voice, text? }` → Тестовое прослушивание голоса в UI.
|
|
185
|
+
* `GET /dsh-tts/models/status` — Статус установленных локальных моделей (Kokoro, F5-TTS).
|
|
186
|
+
* `POST /dsh-tts/models/install` — `{ engine: 'kokoro' | 'f5' }` → Запуск загрузки модели с HuggingFace.
|
|
187
|
+
* `DELETE /dsh-tts/models/delete` — `{ engine: 'kokoro' | 'f5' }` → Удаление локальной модели.
|
|
188
|
+
* `GET /dsh-tts/integrations` — Проверка статуса связанных плагинов (`dsh-voice`, `dsh-messenger-gateway`).
|
|
189
|
+
* `GET /dsh-tts/status` — Общий статус цепочки, статистика кэша и готовность движков.
|
|
190
|
+
|
|
191
|
+
---
|
|
192
|
+
|
|
193
|
+
## 📄 Лицензия
|
|
194
|
+
|
|
195
|
+
MIT © [GooDAnDReaDY](https://github.com/GooDAnDReaDY)
|