@drakulavich/kesha-voice-kit 1.4.2 → 1.4.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +25 -144
- package/package.json +4 -1
package/README.md
CHANGED
|
@@ -14,7 +14,7 @@
|
|
|
14
14
|
<p align="center"><b>Open-source voice toolkit.</b> Optimized for Apple Silicon (CoreML), works on any platform (ONNX fallback).<br>A collection of small, fast, open-source audio models — packaged as CLI tools and an <a href="https://github.com/openclaw/openclaw">OpenClaw</a> skill for LLM agents.</p>
|
|
15
15
|
|
|
16
16
|
- **Speech-to-text** — 25 languages, ~15x faster than Whisper on Apple Silicon, ~2.5x on CPU
|
|
17
|
-
- **
|
|
17
|
+
- **Text-to-speech** — Kokoro (EN) + Piper (RU) + macOS system voices, SSML preview
|
|
18
18
|
- **Rust engine** — single 20MB binary, no ffmpeg, no Python, no native Node addons
|
|
19
19
|
- **OpenClaw-ready** — plug into your LLM agent as a voice processing skill
|
|
20
20
|
|
|
@@ -30,61 +30,17 @@ kesha install # downloads engine + models
|
|
|
30
30
|
kesha audio.ogg # transcript to stdout
|
|
31
31
|
```
|
|
32
32
|
|
|
33
|
-
|
|
33
|
+
Air-gapped or behind a corporate mirror? See [docs/model-mirror.md](docs/model-mirror.md).
|
|
34
34
|
|
|
35
|
-
|
|
35
|
+
## Speech-to-text
|
|
36
36
|
|
|
37
37
|
```bash
|
|
38
|
-
export KESHA_MODEL_MIRROR=https://models.corp.internal/kesha
|
|
39
|
-
kesha install # ASR + lang-id + TTS models fetch from your mirror
|
|
40
|
-
kesha status # confirms the active Mirror URL
|
|
41
|
-
```
|
|
42
|
-
|
|
43
|
-
Unset / empty falls back to `huggingface.co` with no regression. The engine binary itself still comes from GitHub Releases — this env var only redirects model downloads.
|
|
44
|
-
|
|
45
|
-
## OpenClaw Integration
|
|
46
|
-
|
|
47
|
-
Kesha Voice Kit ships as a plugin for [OpenClaw](https://github.com/openclaw/openclaw) — give your LLM agent ears. No API keys, everything runs locally on your machine.
|
|
48
|
-
|
|
49
|
-
```bash
|
|
50
|
-
bun add -g @drakulavich/kesha-voice-kit && kesha install
|
|
51
|
-
openclaw plugins install @drakulavich/kesha-voice-kit
|
|
52
|
-
openclaw config set tools.media.audio.models \
|
|
53
|
-
'[{"type":"cli","command":"kesha","args":["--format","transcript","{{MediaPath}}"],"timeoutSeconds":15}]'
|
|
54
|
-
```
|
|
55
|
-
|
|
56
|
-
> If audio transcription is not already enabled: `openclaw config set tools.media.audio.enabled true`
|
|
57
|
-
|
|
58
|
-
Your agent receives a voice message in Telegram/WhatsApp/Slack, Kesha transcribes it locally, and the agent sees enriched context:
|
|
59
|
-
|
|
60
|
-
```
|
|
61
|
-
Таити, Таити! Не были мы ни в какой Таити! Нас и тут неплохо кормят.
|
|
62
|
-
[lang: ru, confidence: 1.00]
|
|
63
|
-
```
|
|
64
|
-
|
|
65
|
-
Manage the plugin with `openclaw plugins list`, `openclaw plugins disable kesha-voice-kit`, or `openclaw plugins uninstall kesha-voice-kit`.
|
|
66
|
-
|
|
67
|
-
## Raycast Extension (macOS)
|
|
68
|
-
|
|
69
|
-
Transcribe audio and speak clipboard text from Raycast's launcher without opening a terminal. Two commands:
|
|
70
|
-
|
|
71
|
-
- **Transcribe Selected Audio** — pick an audio file in Finder, hit the command, transcript lands on your clipboard.
|
|
72
|
-
- **Speak Clipboard** — synthesize whatever's on your clipboard and play it through the default output.
|
|
73
|
-
|
|
74
|
-
Source + install instructions: [`raycast/`](raycast/). Tracked in [#145](https://github.com/drakulavich/kesha-voice-kit/issues/145); not yet on the Raycast Store — install via `ray develop` from the subdirectory.
|
|
75
|
-
|
|
76
|
-
## CLI Tools
|
|
77
|
-
|
|
78
|
-
```bash
|
|
79
|
-
kesha install # download engine and models
|
|
80
38
|
kesha audio.ogg # transcribe (plain text)
|
|
81
39
|
kesha --format transcript audio.ogg # text + language/confidence
|
|
82
40
|
kesha --format json audio.ogg # full JSON with lang fields
|
|
83
|
-
kesha --
|
|
84
|
-
kesha --toon audio.ogg # compact LLM-friendly TOON (same data as --json)
|
|
41
|
+
kesha --toon audio.ogg # compact LLM-friendly TOON
|
|
85
42
|
kesha --verbose audio.ogg # show language detection details
|
|
86
43
|
kesha --lang en audio.ogg # warn if detected language differs
|
|
87
|
-
kesha --vad lecture.m4a # segment with Silero VAD first (long/silence-heavy audio)
|
|
88
44
|
kesha status # show installed backend info
|
|
89
45
|
```
|
|
90
46
|
|
|
@@ -101,136 +57,61 @@ $ kesha freedom.ogg tahiti.ogg
|
|
|
101
57
|
|
|
102
58
|
Stdout: transcript. Stderr: errors. Pipe-friendly. Also available as `parakeet` command (backward-compatible alias).
|
|
103
59
|
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
For meetings, lectures, and podcasts, enable Silero VAD so Parakeet only sees the speech bits. Segment boundaries land at natural speech starts/ends instead of arbitrary cuts, and long silences are skipped entirely.
|
|
60
|
+
For long / silence-heavy audio, use `--vad` (auto-on past 120 s). Details: [docs/vad.md](docs/vad.md).
|
|
107
61
|
|
|
108
|
-
|
|
109
|
-
kesha install --vad # one-time, ~2.3MB
|
|
110
|
-
kesha lecture.m4a # auto-on when audio ≥ 120s and VAD installed
|
|
111
|
-
kesha --vad short-clip.ogg # force VAD on any input
|
|
112
|
-
kesha --no-vad meeting.m4a # force VAD off even on long audio
|
|
113
|
-
```
|
|
114
|
-
|
|
115
|
-
Auto-triggers at 120 s so voice messages (< 30 s of near-pure speech) stay on the fast path. If you have long audio without VAD installed, Kesha prints a one-time stderr hint. Defaults: threshold 0.5, min-speech 250 ms, min-silence 100 ms, 30 ms edge padding. See issues #128 (base) and #187 (auto-trigger).
|
|
116
|
-
|
|
117
|
-
## Text-to-Speech
|
|
62
|
+
## Text-to-speech
|
|
118
63
|
|
|
119
|
-
Kesha speaks back via Kokoro-82M (English) and Piper (Russian)
|
|
64
|
+
Kesha speaks back via Kokoro-82M (English) and Piper (Russian) — voice auto-picks from the text's language:
|
|
120
65
|
|
|
121
66
|
```bash
|
|
122
|
-
kesha install --tts
|
|
67
|
+
kesha install --tts # ~490MB (Kokoro + Piper RU + ONNX G2P, opt-in)
|
|
123
68
|
kesha say "Hello, world" > hello.wav
|
|
124
|
-
kesha say "Привет, мир" > privet.wav
|
|
125
|
-
echo "long text" | kesha say > reply.wav
|
|
126
|
-
kesha say --out reply.wav "text"
|
|
127
|
-
kesha say --voice en-af_heart "text" # explicit voice overrides auto-routing
|
|
128
|
-
kesha say --list-voices
|
|
69
|
+
kesha say "Привет, мир" > privet.wav # auto-routes to ru-denis
|
|
129
70
|
```
|
|
130
71
|
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
**Supported voices:**
|
|
134
|
-
- English: `en-af_heart` (default), plus any Kokoro voice you download into `~/.cache/kesha/models/kokoro-82m/voices/`
|
|
135
|
-
- Russian: `ru-denis` (default). More speakers (dmitri, irina, ruslan) are ready to drop in once needed.
|
|
136
|
-
- macOS system voices: `macos-<identifier-or-language>` routes to `AVSpeechSynthesizer`. Zero install, any of the 180+ voices already on your Mac.
|
|
137
|
-
|
|
138
|
-
### macOS system voices
|
|
139
|
-
|
|
140
|
-
`kesha say --voice macos-*` routes through `AVSpeechSynthesizer` on macOS, so you get voice synthesis for free — no 490 MB TTS bundle. The sidecar binary ships alongside `kesha-engine` on darwin-arm64 releases (#141); `kesha install` places both in `~/.cache/kesha/bin/`.
|
|
72
|
+
macOS system voices, SSML, voice listing, and the full voice catalogue: [docs/tts.md](docs/tts.md).
|
|
141
73
|
|
|
142
|
-
|
|
143
|
-
kesha say --list-voices | grep ^macos- # discover installed voices
|
|
144
|
-
kesha say --voice macos-com.apple.voice.compact.en-US.Samantha "Hello" > out.wav
|
|
145
|
-
kesha say --voice macos-ru-RU "Привет, мир" > hello-ru.wav # language-code fallback
|
|
146
|
-
```
|
|
147
|
-
|
|
148
|
-
Voice id format: `macos-<id>` where `<id>` is either a full Apple identifier (`com.apple.voice.compact.en-US.Samantha`) or a language code (`en-US`, `ru-RU`) — the Swift helper tries the identifier first and falls back to the language. Output is mono float32 @ 22050 Hz, structurally identical to Piper.
|
|
149
|
-
|
|
150
|
-
Quality tradeoff is honest: macOS system voices are notification-grade. Use them when you want zero-install TTS on macOS; keep Kokoro/Piper for anything that needs to sound good.
|
|
151
|
-
|
|
152
|
-
### SSML (preview)
|
|
74
|
+
## Performance
|
|
153
75
|
|
|
154
|
-
|
|
76
|
+
> **~15x faster than Whisper** on Apple Silicon (M3 Pro), **~2.5x faster** on CPU
|
|
155
77
|
|
|
156
|
-
|
|
157
|
-
kesha say --ssml '<speak>Hello <break time="500ms"/> world.</speak>'
|
|
158
|
-
kesha say --ssml --voice ru-denis '<speak>Привет <break time="1s"/> мир.</speak>'
|
|
159
|
-
```
|
|
78
|
+
Compared against Whisper `large-v3-turbo` — all engines auto-detect language.
|
|
160
79
|
|
|
161
|
-
|
|
162
|
-
|---|---|
|
|
163
|
-
| `<speak>` | ✅ required root |
|
|
164
|
-
| `<break time="Nms"\|"Ns"\|default>` | ✅ inserts silence of the given duration |
|
|
165
|
-
| plain text inside `<speak>` | ✅ synthesized via the selected engine |
|
|
166
|
-
| `<emphasis>`, `<prosody>`, `<phoneme>`, `<say-as>` | ⚠️ stripped with a stderr warning (contained text still synthesized); tracked in [#122](https://github.com/drakulavich/kesha-voice-kit/issues/122) |
|
|
167
|
-
| `<!DOCTYPE>` | ❌ rejected (hardening against XXE) |
|
|
80
|
+
|
|
168
81
|
|
|
169
|
-
|
|
82
|
+
See [BENCHMARK.md](BENCHMARK.md) for the full per-file breakdown (Russian + English).
|
|
170
83
|
|
|
171
84
|
## What's Inside
|
|
172
85
|
|
|
173
|
-
Kesha Voice Kit bundles open-source models optimized for on-device inference:
|
|
174
|
-
|
|
175
86
|
| Model | Task | Size | Source |
|
|
176
87
|
|---|---|---|---|
|
|
177
88
|
| NVIDIA Parakeet TDT 0.6B v3 | Speech-to-text | ~2.5GB | [HuggingFace](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3) |
|
|
178
89
|
| SpeechBrain ECAPA-TDNN | Audio language detection | ~86MB | [HuggingFace](https://huggingface.co/speechbrain/lang-id-voxlingua107-ecapa) |
|
|
179
90
|
| Apple NLLanguageRecognizer | Text language detection | built-in | macOS system framework |
|
|
180
91
|
| Silero VAD v5 (opt-in) | Voice activity detection | ~2.3MB | [snakers4/silero-vad](https://github.com/snakers4/silero-vad) |
|
|
92
|
+
| Kokoro-82M / Piper (opt-in) | Text-to-speech | ~490MB | [Kokoro](https://huggingface.co/hexgrad/Kokoro-82M) · [Piper](https://github.com/rhasspy/piper) |
|
|
181
93
|
|
|
182
94
|
All models run through `kesha-engine` — a Rust binary using [FluidAudio](https://github.com/FluidInference/FluidAudio) (CoreML) on Apple Silicon and [ort](https://github.com/pykeio/ort) (ONNX Runtime) on other platforms.
|
|
183
95
|
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
> **~15x faster than Whisper** on Apple Silicon (M3 Pro), **~2.5x faster** on CPU
|
|
187
|
-
|
|
188
|
-
Compared against Whisper `large-v3-turbo` — all engines auto-detect language.
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
<details>
|
|
193
|
-
<summary>Full results with per-file breakdown</summary>
|
|
194
|
-
|
|
195
|
-
See [BENCHMARK.md](BENCHMARK.md) — includes Russian (real voice messages) and English transcription results with all four engines.
|
|
96
|
+
Audio decoding via [symphonia](https://github.com/pdeljanov/Symphonia) — WAV, MP3, OGG/Opus, FLAC, AAC, M4A. No ffmpeg.
|
|
196
97
|
|
|
197
|
-
|
|
98
|
+
## Languages
|
|
198
99
|
|
|
199
|
-
|
|
100
|
+
- **Speech-to-text (25):** Bulgarian, Croatian, Czech, Danish, Dutch, English, Estonian, Finnish, French, German, Greek, Hungarian, Italian, Latvian, Lithuanian, Maltese, Polish, Portuguese, Romanian, Russian, Slovak, Slovenian, Spanish, Swedish, Ukrainian.
|
|
101
|
+
- **Audio language detection (107):** [full list](https://huggingface.co/speechbrain/lang-id-voxlingua107-ecapa).
|
|
200
102
|
|
|
201
|
-
|
|
103
|
+
## Integrations
|
|
202
104
|
|
|
203
|
-
|
|
204
|
-
|
|
205
|
-
| WAV | `.wav` |
|
|
206
|
-
| MP3 | `.mp3` |
|
|
207
|
-
| OGG Vorbis/Opus | `.ogg`, `.opus` |
|
|
208
|
-
| FLAC | `.flac` |
|
|
209
|
-
| AAC / M4A | `.aac`, `.m4a` |
|
|
210
|
-
|
|
211
|
-
## Supported Languages
|
|
212
|
-
|
|
213
|
-
**Speech-to-text (25):** :bulgaria: Bulgarian, :croatia: Croatian, :czech_republic: Czech, :denmark: Danish, :netherlands: Dutch, :gb: English, :estonia: Estonian, :finland: Finnish, :fr: French, :de: German, :greece: Greek, :hungary: Hungarian, :it: Italian, :latvia: Latvian, :lithuania: Lithuanian, :malta: Maltese, :poland: Polish, :portugal: Portuguese, :romania: Romanian, :ru: Russian, :slovakia: Slovak, :slovenia: Slovenian, :es: Spanish, :sweden: Swedish, :ukraine: Ukrainian
|
|
214
|
-
|
|
215
|
-
**Audio language detection (107):** Full list at [speechbrain/lang-id-voxlingua107-ecapa](https://huggingface.co/speechbrain/lang-id-voxlingua107-ecapa)
|
|
216
|
-
|
|
217
|
-
## Architecture
|
|
218
|
-
|
|
219
|
-
```
|
|
220
|
-
kesha audio.ogg
|
|
221
|
-
→ kesha-engine (Rust binary)
|
|
222
|
-
├── Apple Silicon? → FluidAudio (CoreML / Neural Engine)
|
|
223
|
-
└── Other? → ort (ONNX Runtime / CPU)
|
|
224
|
-
→ transcript to stdout
|
|
225
|
-
```
|
|
105
|
+
- **OpenClaw** — give your LLM agent ears. Install & config: [docs/openclaw.md](docs/openclaw.md).
|
|
106
|
+
- **Raycast** (macOS) — transcribe selected audio & speak clipboard from the launcher. Source + install: [`raycast/`](raycast/).
|
|
226
107
|
|
|
227
108
|
## Programmatic API
|
|
228
109
|
|
|
229
110
|
```typescript
|
|
230
111
|
import { transcribe, downloadModel } from "@drakulavich/kesha-voice-kit/core";
|
|
231
112
|
|
|
232
|
-
await downloadModel();
|
|
233
|
-
const text = await transcribe("audio.ogg");
|
|
113
|
+
await downloadModel(); // install engine + models
|
|
114
|
+
const text = await transcribe("audio.ogg"); // transcribe
|
|
234
115
|
```
|
|
235
116
|
|
|
236
117
|
## Requirements
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@drakulavich/kesha-voice-kit",
|
|
3
|
-
"version": "1.4.
|
|
3
|
+
"version": "1.4.3",
|
|
4
4
|
"description": "Open-source voice toolkit. Speech-to-text (25 langs, ~19x faster than Whisper on Apple Silicon, ONNX fallback on CPU), text-to-speech (Kokoro + Piper + ~180 macOS system voices, SSML), voice-activity detection, language detection (107 langs). Rust engine, OpenClaw skill.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
|
@@ -36,6 +36,8 @@
|
|
|
36
36
|
"test": "bun test",
|
|
37
37
|
"test:unit": "bun test tests/unit/",
|
|
38
38
|
"test:integration": "bun test tests/integration/",
|
|
39
|
+
"lint": "bunx tsc --noEmit",
|
|
40
|
+
"lint:tsgo": "bunx tsgo --noEmit",
|
|
39
41
|
"postinstall": "node scripts/postinstall.cjs"
|
|
40
42
|
},
|
|
41
43
|
"keywords": [
|
|
@@ -65,6 +67,7 @@
|
|
|
65
67
|
},
|
|
66
68
|
"devDependencies": {
|
|
67
69
|
"@types/bun": "latest",
|
|
70
|
+
"@typescript/native-preview": "^7.0.0-dev.20260423.1",
|
|
68
71
|
"fast-xml-parser": "^5.5.10",
|
|
69
72
|
"typescript": "^6.0.2"
|
|
70
73
|
},
|