@sfgartland/better-pi-voice 0.0.0-stage → 0.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md ADDED
@@ -0,0 +1,67 @@
1
+ # Changelog
2
+
3
+ All notable changes to this project are documented here. The format follows
4
+ [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to
5
+ [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
6
+
7
+ ## [Unreleased]
8
+
9
+ ## [0.1.1] - 2026-10-16
10
+
11
+ ### Added
12
+
13
+ - A `media/demo.mp4` storyboard of the before/after transcription, wired into the
14
+ package manifest as `pi.video` so the [pi package gallery](https://pi.dev/packages)
15
+ can preview it.
16
+
17
+ ### Fixed
18
+
19
+ - **A skill file no longer becomes the keyterm `SKILL`.** Keyterms are mined from the
20
+ basenames of files the session touches, so reading `…/skills/<name>/SKILL.md` mined
21
+ the generic document stem `SKILL`. The local backend's correction pass then rewrote
22
+ ordinary words one edit away from it — `still`, `spill`, `skull` and `swill` all came
23
+ back as `SKILL`. Generic document names (`SKILL`, `README`, `INDEX`, `LICENSE`,
24
+ `CHANGELOG`, …) now mine their parent directory instead, which carries the name that
25
+ actually matters, and a mined term that collides with the seed jargon is no longer
26
+ eligible for fuzzy repair.
27
+
28
+ ## [0.1.0] - 2026-10-16
29
+
30
+ First release.
31
+
32
+ ### Added
33
+
34
+ - Streaming voice input for pi, one shortcut away (`ctrl+alt+z` by default), with
35
+ hold-to-talk and tap modes and a hard cap so a stuck key cannot leave the microphone
36
+ open.
37
+ - **Two interchangeable backends**: Deepgram `nova-3` streaming, or local Parakeet via
38
+ `transcribe-cpp`.
39
+ - **Repository-aware vocabulary biasing.** Keyterms are mined from the project name,
40
+ the git repository, file paths touched by the session, paths mentioned in messages,
41
+ a developer-jargon seed list and your own list, then sent to Deepgram as repeated
42
+ `keyterm` parameters (native biasing) and updatable live with a `Configure` message.
43
+ The local backend, which cannot be vocabulary-biased (`supports("vocabulary")` is
44
+ `false` for the Parakeet GGUFs), gets a deterministic correction pass instead:
45
+ re-joining identifiers the recogniser split, and repairing one misheard token within
46
+ a small edit distance of a keyterm.
47
+ - **A recovery layer that never loses a sentence**: one early retry before the first
48
+ transcript, replay of the retained audio when a stream dies unconfirmed, salvage of a
49
+ partial transcript when the socket dies late, promotion of an unreported interim, and
50
+ a circuit breaker after repeated fast failures. Audio is buffered in memory only and
51
+ never written to disk.
52
+ - **Local batch mode with pause-aware chunking**: audio is cut at real (silence-checked)
53
+ pauses while you speak and transcribed in the background, so only the tail is left at
54
+ finalize. The pause threshold is derived per recording, and a chunk is only cut where
55
+ the signal is genuinely quiet, so a chunk never starts mid-word.
56
+ - **Guided setup** (`/voice setup`, or the first shortcut press with no settings):
57
+ backend, key or model (with checksum-verified, resumable model downloads), run mode,
58
+ and shortcut, ending in an overview of the result.
59
+ - Commands: `/voice`, `/voice setup`, `/voice backend`, `/voice model`, `/voice key`,
60
+ `/voice set`, and `/voice-keyterms add|remove|list`.
61
+ - Diagnostics that name the cause rather than the symptom — a failed connection
62
+ distinguishes a rejected key, an unreachable host, and an HTTP error.
63
+ - A `media/screenshot.png` before/after card, exposed to the gallery as `pi.image`.
64
+
65
+ [Unreleased]: https://github.com/sfgartland/better-pi-voice/compare/v0.1.1...HEAD
66
+ [0.1.1]: https://github.com/sfgartland/better-pi-voice/releases/tag/v0.1.1
67
+ [0.1.0]: https://github.com/sfgartland/better-pi-voice/releases/tag/v0.1.0
package/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Severin Gartland
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md CHANGED
@@ -1,3 +1,406 @@
1
- # Temporary Holding Version
1
+ # @sfgartland/better-pi-voice
2
2
 
3
- This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
3
+ ![the same sentence transcribed without keyterms (sub agent, Worktree T's config) and with them (subagent, worktree tsconfig)](https://raw.githubusercontent.com/sfgartland/better-pi-voice/main/media/screenshot.png)
4
+
5
+ Streaming voice input for pi. Hold a key, talk, and the words land in the editor —
6
+ in the right spelling, including the jargon and filenames generic speech
7
+ recognition gets wrong. ([8-second demo](https://github.com/sfgartland/better-pi-voice/raw/main/media/demo.mp4))
8
+
9
+ Two interchangeable backends behind one pipeline:
10
+
11
+ | | Backend | Biasing | Needs |
12
+ |---|---|---|---|
13
+ | Cloud | **Deepgram `nova-3`** streaming | `keyterm` prompting (native) | a Deepgram API key |
14
+ | Local | **Parakeet** via `transcribe-cpp` | post-hoc correction pass only | a Parakeet GGUF |
15
+
16
+ ## Why this exists
17
+
18
+ Generic dictation is tuned for English prose, not for a terminal. It writes
19
+ "sub agent" and "work tree", guesses at filenames, turns `tsconfig` into "T's
20
+ config", and has no idea what your project is even called. That is not mostly a
21
+ model-quality problem — it is a missing-context problem, and it is fixable: tell
22
+ the recogniser the words the project actually uses. Cloud dictation is also a
23
+ black box: when the socket drops mid-sentence, most tools lose the sentence.
24
+ Local dictation usually trades accuracy for privacy and gives up on both.
25
+
26
+ `better-pi-voice` attacks all three: vocabulary biasing mined from the repo and
27
+ session, a recovery layer that will not drop a sentence, and a real local option
28
+ behind the same pipeline.
29
+
30
+ Claude Code's dictation is unusually good, and it isn't the model. Reverse-engineering
31
+ it (`claude-code@2.1.295`) turned up three things worth stealing:
32
+
33
+ 1. **Vocabulary biasing from the repository.** Before every session it pushes the
34
+ project name, the paths already in play, and a hardcoded list of developer jargon
35
+ (`subagent`, `worktree`, `OAuth`, `gRPC`…) to the recogniser. This is the single
36
+ biggest accuracy lever and it costs nothing.
37
+ 2. **Server-side endpointing** at 16 kHz mono, so finals land promptly instead of
38
+ waiting for you to press stop.
39
+ 3. **A recovery layer that never loses a sentence** — one retry before the first
40
+ transcript, replaying buffered audio when a stream silently drops, salvaging a
41
+ partial transcript when the socket dies, promoting an unreported interim to a
42
+ final, and a circuit breaker so a broken microphone fails loudly.
43
+
44
+ All three are reproduced in this extension. The design document with the full extraction notes is
45
+ at `docs/superpowers/specs/2026-10-15-pi-voice-input-design.md`.
46
+
47
+ ## What it actually buys you
48
+
49
+ A fixed sentence containing five terms, transcribed twice through the live Deepgram
50
+ API — once with no keyterms, once with the repository's terms:
51
+
52
+ ```
53
+ without keyterms … so the sub agent can use the Worktree T's config, then check
54
+ the Marginalia OAuth webhook.
55
+ with keyterms … so the subagent can use the worktree tsconfig, then check the
56
+ marginalia OAuth webhook.
57
+ ```
58
+
59
+ `tsconfig` → "T's config" is the failure mode in miniature: correct English, wrong
60
+ words. Two of the five terms came back wrong without biasing and all five were right
61
+ with it.
62
+
63
+ ## How it compares to the other pi voice extensions
64
+
65
+ Three voice extensions are in play, and they optimise for different things. This
66
+ one is for **accurate dictation of code vocabulary**, from the cloud or locally:
67
+
68
+ | | [`@earendil-works/pi-voice`](https://github.com/earendil-works/pi-voice) | [`pi-voice-stt`](https://github.com/cgarrot/pi-voice-stt) | **better-pi-voice** |
69
+ |---|---|---|---|
70
+ | Recognition | local only (`transcribe-cpp`) | several cloud providers + OpenAI-compatible | **Deepgram `nova-3` streaming** or local Parakeet |
71
+ | Repo-aware vocabulary biasing | no | no | **yes** — keyterms mined from the repo and session, native on Deepgram |
72
+ | Never-lose-a-sentence recovery | no | no | **yes** — retry, silent-drop replay, partial salvage, circuit breaker |
73
+ | Capture | bundled `pvrecorder` | `ffmpeg` (or a Mac bridge) | `arecord`, falling back to SoX `rec` |
74
+ | Transcribe an audio file | yes (`transcribe_file`) | yes (`transcribe_audio`) | no — this extension is for live dictation |
75
+ | Guided setup | `/voice-settings` onboarding, model download | config file / environment | `/voice setup` wizard, model download |
76
+ | Best for | a fully local, offline dictation tool | choosing among many cloud providers | dictating code accurately, with a private fallback |
77
+
78
+ All three are MIT and coexist as pi packages, so there is no need to pick
79
+ ideologically: use `pi-voice` if you want everything strictly local, `pi-voice-stt`
80
+ if you want a particular provider, and this one if the words that come back are the
81
+ thing you care about.
82
+
83
+ ## Install
84
+
85
+ ```bash
86
+ pi install /path/to/better-pi-voice # from a local checkout
87
+ pi install npm:@sfgartland/better-pi-voice # once published
88
+ ```
89
+
90
+ Then add a Deepgram key. The easiest way is `/voice key`:
91
+
92
+ - `/voice key <path-to-dotenv>` — use a key from an existing dotenv file (it checks one is
93
+ actually in there first), then switches to Deepgram.
94
+ - `/voice key` — paste the key at the prompt; it is written to `~/.pi/agent/better-pi-voice.key`
95
+ with `0600` permissions and referenced from the settings, so the secret never lands in
96
+ `better-pi-voice.json` (which is the file people sync into dotfiles repos).
97
+ - `/voice key --inline` — store the key in `better-pi-voice.json` itself. Explicit opt-in:
98
+ only do this if that file is not tracked by git.
99
+
100
+ You can also export it, which overrides everything the settings point at:
101
+
102
+ ```bash
103
+ DEEPGRAM_API_KEY=... pi
104
+ ```
105
+
106
+ or set `deepgram.apiKeyFile` to any dotenv by hand. Precedence is: environment variable,
107
+ then `deepgram.apiKey` (inline), then `deepgram.apiKeyFile`.
108
+
109
+ ## Guided setup
110
+
111
+ Run `/voice setup`, or just press the shortcut on a fresh install and it walks you
112
+ through it. The wizard asks, in order: how voice should run (Deepgram cloud or local
113
+ Parakeet), how to supply the key or which model to use, whether to hold or tap, and
114
+ which shortcut. Then it saves, and ends with an **overview** pane: the settings it
115
+ wrote, how to use them (hold or press the shortcut, where the text goes), and how to
116
+ change any of it later with `/voice setup`. Enter closes it; Esc goes back and
117
+ re-runs the wizard. There is no live microphone test to sit through — try the
118
+ shortcut yourself when you are ready.
119
+
120
+ For the key, if `DEEPGRAM_API_KEY` is already in your environment the wizard offers to
121
+ just use it; otherwise you can paste a key (kept `0600`, never in settings) or point at
122
+ a dotenv file. For the local backend, the model menu lists the known GGUFs best-first
123
+ with their download size — **selecting one downloads it** (checksum-verified, with a
124
+ progress line) — then falls back to any GGUF already on disk or a path you give.
125
+
126
+ If the model you chose can stream, the wizard then asks how it should run and spells out
127
+ the tradeoff: **Batch** transcribes in short slices while you talk (works with every
128
+ model, and faster on most machines) versus **Streaming**, which decodes continuously for
129
+ lower delay but needs a fast CPU to keep up in real time. A batch-only model skips the
130
+ question and says why. See [Batch mode](#batch-mode-the-default) below for the detail.
131
+
132
+ ## Use
133
+
134
+ The default binding is `ctrl+alt+z`; **hold it while you speak and release**. The
135
+ status line names the active backend and model (`● deepgram nova-3: listening`,
136
+ `transcribing with parakeet parakeet-unified-en-0.6b…`) and keeps showing it while
137
+ idle, so it is always obvious which one is listening. An audio-level meter appears
138
+ above the editor while recording.
139
+
140
+ | Mode | Start | Stop | Safety |
141
+ |---|---|---|---|
142
+ | `hold` (default) | press | release | falls back if the terminal never reports release |
143
+ | `tap` | press | press again | 15 s of silence, 120 s hard cap |
144
+
145
+ A hard cap applies in both modes so a stuck key cannot leave the microphone open.
146
+
147
+ Commands:
148
+
149
+ - `/voice` — backend, model, mode, shortcut, keyterm count, whether the key is visible, state
150
+ - `/voice setup` — guided setup (backend, key/model, mode, shortcut, then an overview of the result)
151
+ - `/voice backend` — pick a backend from a menu
152
+ - `/voice backend deepgram|parakeet` — switch directly, then reload
153
+ - `/voice model` — pick a local GGUF already in the Hugging Face cache, then reload
154
+ - `/voice model /path/to/model.gguf` — use a specific model file
155
+ - `/voice key <path-to-dotenv>` — use a key from a dotenv file, then switch to Deepgram
156
+ - `/voice key` — paste a key; stored `0600` in `~/.pi/agent/better-pi-voice.key`
157
+ - `/voice key --inline` — store the key in the settings file (explicit opt-in)
158
+ - `/voice set <key> <value>` — set `backend`, `model`, `mode`, `language`, `correction`,
159
+ `submitOnStop`, `chunkSeconds`, `silenceLevel`, `shortcut`, `apiKeyEnv` or `apiKeyFile`, then reload
160
+ - `/voice-keyterms` — list custom terms
161
+ - `/voice-keyterms add <term>` / `remove <term>`
162
+
163
+ The backend is **Deepgram by default**. Switching rewrites `better-pi-voice.json`
164
+ (fully expanded, so it stops looking like the minimal file you hand-wrote) and then
165
+ reloads the extension so the change takes effect immediately. You can also just edit
166
+ `"backend"` in the settings file and restart.
167
+
168
+ On a first run with no settings file, voice configures itself: Deepgram when an API
169
+ key is already reachable, otherwise the best local model already on disk. It then tells
170
+ you which backend it chose and which key to hold. If neither is available it writes
171
+ nothing and says what to set.
172
+
173
+ ## Keyterms
174
+
175
+ Terms are gathered from four places, most specific first:
176
+
177
+ 1. `keyterms.user` — your own list
178
+ 2. a built-in seed list of developer jargon
179
+ 3. the project directory name and the git repository name
180
+ 4. files touched by `read`/`write`/`edit` tool calls and paths mentioned in your messages
181
+
182
+ They are normalised (camelCase split, separators split, 3–20 characters, printable
183
+ ASCII, deduplicated) and capped well inside Deepgram's documented 100-term / 500-token
184
+ limit. A comma-joined keyterm list is accepted by Deepgram and silently boosts
185
+ *nothing*, so terms are sent as repeated query parameters — there is a test for that.
186
+
187
+ ## The local backend, honestly
188
+
189
+ **Parakeet cannot be vocabulary-biased.** `transcribe-cpp` 0.4.0 does have a
190
+ `vocabulary` input, but only for models that advertise `supports("vocabulary")`, and
191
+ the Parakeet GGUFs do not — loading `parakeet-tdt_ctc-110m`, the recommended model,
192
+ returns `false` for both `vocabulary` and `context_prompt`. So locally you get live
193
+ interims *or* native biasing, not both.
194
+
195
+ This extension takes interims, and makes up the difference with a deliberately dumb
196
+ deterministic repair pass that runs on finals only. It does two things:
197
+
198
+ 1. **Rejoin identifiers the recogniser split.** `tsconfig` comes back as "TS config".
199
+ Two adjacent tokens are joined only when their concatenation is *exactly* a keyterm,
200
+ they are separated by exactly one space, and the shorter one is an abbreviation
201
+ (uppercase, or at most two characters). "TS config" qualifies; "back end" and
202
+ "work tree" deliberately do not, because in the next sentence those are two words.
203
+ 2. **Repair a single misheard token**, within one edit of a keyterm for words up to 7
204
+ characters or two edits for longer ones, and never for tokens under 5 characters.
205
+ Distance is Damerau, so an adjacent swap counts once — that is how ASR mangles a
206
+ technical term most often (`subagent` → `subagnet`).
207
+
208
+ On the real output of the local backend for the test sentence, that turns
209
+ `the Worktree TS config` into `the Worktree tsconfig`. It does **not** rescue
210
+ `DRAM backend` from `deepgram backend` — too many edits, and below the length floor.
211
+
212
+ ```jsonc
213
+ "correction": "safe" // "off" | "safe" | "report"
214
+ ```
215
+
216
+ `"report"` changes nothing and logs what it *would* change — use it to find out
217
+ whether the pass is earning its keep before trusting it.
218
+
219
+ If you want true local biasing instead, a whisper-family model with `initialPrompt`
220
+ plus a batch re-pass at finalize is the route; this extension does not implement it.
221
+
222
+ **It is also the slow part.** How fast it runs depends on the machine; the numbers below
223
+ come from one reference machine — an 8-core x86_64 laptop using the bundled
224
+ `linux-x64-cpu-vulkan` build on the CPU path (its GPU was not usable there):
225
+
226
+ | | measured |
227
+ |---|---|
228
+ | first model load (698 MB, cold page cache) | ~2.7 s |
229
+ | warm reload (file already cached) | ~0.4 s |
230
+ | **streaming decode** | **~0.9× real-time — only just keeps up** |
231
+ | **batch** transcribe of the same audio | **3.5× real-time** |
232
+ | first interim out of the stream | ~0.6 s |
233
+
234
+ So live streaming works, but without headroom: the decoder consumes audio about as
235
+ fast as you speak it, and text drifts further behind over a long utterance. The batch
236
+ path is nearly four times faster, which is why a record-then-transcribe mode is worth
237
+ considering (see below). Treat the local backend as the private/offline option rather
238
+ than the fast one until it is running on usable compute.
239
+
240
+ ### Which local models can stream
241
+
242
+ `transcribe-cpp` ships 27 models; only these four can do live dictation at all:
243
+
244
+ | model | size | languages |
245
+ |---|---|---|
246
+ | `parakeet-unified-en-0.6b` | 731 MB | English |
247
+ | `multitalker-parakeet-streaming-0.6b-v1` | 734 MB | English |
248
+ | `nemotron-3.5-asr-streaming-0.6b` | 751 MB | 32 languages |
249
+ | `Voxtral-Mini-4B-Realtime-2602` | 4.7 GB | 13 languages |
250
+
251
+ There is **no streaming model lighter than parakeet-0.6b**. The genuinely small ones —
252
+ `medasr` (128 MB), `parakeet-tdt_ctc-110m` (135 MB), `canary-180m-flash` (218 MB),
253
+ `SenseVoiceSmall` (253 MB), `whisper-small` (270 MB) — are all batch-only, so using one
254
+ means giving up live text and transcribing after you release the key. With batch running
255
+ at 3.5× real-time that is a reasonable trade, and the pipeline already retains the whole
256
+ utterance, but it is not implemented yet.
257
+
258
+ For non-English dictation, `nemotron-3.5-asr-streaming-0.6b` is the only streaming option
259
+ that is not parakeet, at the same size.
260
+
261
+ ### Batch mode (the default)
262
+
263
+ `parakeet.mode` chooses how the local backend works, and **`batch` is the default**
264
+ because it is typically several times faster than streaming, and because it is the only
265
+ option for the small models, none of which can stream.
266
+
267
+ In batch mode the audio is transcribed in pieces while you are still speaking, so only
268
+ the tail is left when you release the key:
269
+
270
+ - `parakeet.chunkSeconds` (default 30) is a **target, not a limit**, and only its lower
271
+ 70% matters: the audio is cut at the **first real pause** at or after that point and
272
+ transcribed in the background. A pause arriving later than the target still triggers a
273
+ cut, and unbroken speech simply grows a longer chunk
274
+ - `parakeet.chunkSeconds: 0` disables chunking and transcribes once, at the end
275
+ - a chunk is only cut where the signal is quiet enough to be a pause. The threshold is
276
+ **derived from each recording** (`parakeet.silenceLevel: "auto"`): the noise floor is the
277
+ second-lowest level in the buffer, speech is the 75th percentile, and a boundary counts
278
+ as a pause when it falls in the lowest 30% of that range. The floor is a near-minimum
279
+ rather than a low percentile because pauses are brief — two silent chunks in forty leave
280
+ a percentile reporting pure speech, while the minimum still finds them. A fixed number
281
+ cannot span rooms — of two real recordings, one never dropped below 0.05
282
+ (music playing) and the other paused around 0.3. Pin a number to override.
283
+ - when the level barely varies there is no pause structure to find, so nothing is cut and
284
+ the whole utterance is transcribed in one piece. That is the safe answer for unbroken
285
+ speech and for steady background noise.
286
+ - **the default is calibrated for a model that tolerates a chunk starting mid-word**,
287
+ which the recommended `parakeet-tdt_ctc-110m` does. It matters: `canary-180m-flash`
288
+ returns an *empty* transcript for any clip that starts mid-utterance — in testing, at
289
+ offsets 4, 6, 8 and 10 s into a 24 s clip it produced 0 characters every time, while
290
+ parakeet-tdt_ctc-110m produced 292, 253, 215 and 192. If you use canary, set
291
+ `silenceLevel` to 0.05 so cuts only happen at true silence.
292
+
293
+ That last rule is not fussiness. Cutting mid-utterance loses words outright: canary
294
+ returns an **empty transcript** for any clip that begins mid-sentence, so a chunk cut at
295
+ merely the *quietest* moment silently discarded everything in it. Requiring real silence
296
+ costs tail latency on unbroken speech instead of costing speech.
297
+
298
+ ### Keeping the model loaded
299
+
300
+ The model is **released after every recording by default**, because a warm reload is only
301
+ about 0.4 s and holding ~700 MB between recordings is not a good trade on a laptop. Set
302
+ `parakeet.idleUnloadMs` to keep it resident instead — useful for larger models, where a
303
+ reload costs real time. When residency is on, the model is released by whichever of these
304
+ happens first:
305
+
306
+ - the idle window expires (`parakeet.idleUnloadMs`)
307
+ - the machine gets busy — one-minute load average per CPU reaching **0.8**
308
+ (`parakeet.loadShedRatio`), checked once a minute
309
+
310
+ A new recording cancels a pending release, and session shutdown releases immediately.
311
+ Note that the load-shedding half only matters while residency is on: at the default of 0
312
+ the memory is already back after each recording, so there is nothing to shed.
313
+
314
+ ## Settings
315
+
316
+ `~/.pi/agent/better-pi-voice.json`. Every field is optional; anything missing or
317
+ malformed falls back to a default and warns rather than breaking the microphone.
318
+
319
+ ```jsonc
320
+ {
321
+ "backend": "deepgram", // "deepgram" | "parakeet"
322
+ "mode": "hold", // "hold" | "tap"
323
+ "shortcut": "ctrl+alt+z",
324
+ "language": "en",
325
+ "deepgram": {
326
+ "apiKeyEnv": "DEEPGRAM_API_KEY",
327
+ "apiKey": null, // inline key, only written by /voice key --inline
328
+ "apiKeyFile": null, // dotenv file, used when the env var is unset
329
+ "model": "nova-3",
330
+ "smartFormat": true,
331
+ "endpointingMs": 300, // silence before an utterance is finalised
332
+ "utteranceEndMs": 1000
333
+ },
334
+ "parakeet": {
335
+ "modelPath": null, // null = read pi-voice's config if present
336
+ "mode": "batch", // "batch" (faster, chunked) | "streaming"
337
+ "chunkSeconds": 30, // cut at a pause near this length; 0 = only at the end
338
+ "silenceLevel": "auto", // "auto" derives the pause threshold per recording; a number pins it
339
+ "idleUnloadMs": 0, // 0 = release after each recording (see above)
340
+ "loadShedRatio": 0.8 // with residency on, release early once load per CPU reaches this
341
+ },
342
+ "keyterms": {
343
+ "enabled": true,
344
+ "seed": true,
345
+ "mineSessionFiles": true,
346
+ "maxTerms": 60,
347
+ "user": []
348
+ },
349
+ "correction": "safe",
350
+ "submitOnStop": false // false = leave the text in the editor
351
+ }
352
+ ```
353
+
354
+ ## When something goes wrong
355
+
356
+ Failures are reported with a cause rather than "check your network":
357
+
358
+ - **The microphone starts first.** Capture begins as soon as the key goes down, and
359
+ audio is buffered and then sent the instant the backend is ready — so the first
360
+ syllable is not lost while a model loads or a websocket connects. The microphone is
361
+ also probed at session start, so a missing device is reported then rather than when you
362
+ first hold the key.
363
+ - **Connection failures are diagnosed.** When the websocket fails, the extension makes
364
+ a small authenticated REST call to Deepgram and reports which it was: `Deepgram
365
+ rejected the API key`, `could not reach Deepgram (getaddrinfo ENOTFOUND ...)`, or
366
+ `Deepgram returned HTTP 503`. The websocket close code alone cannot distinguish those.
367
+ - **Errors carry their detail.** Whatever the socket's error event holds is included
368
+ in the message instead of a bare "connection error".
369
+ - **Speech is never lost.** One retry before the first transcript, replay of the
370
+ retained audio when a stream silently drops, partial transcripts salvaged, unreported
371
+ interims promoted to finals, and a circuit breaker after three fast failures.
372
+
373
+ ## Privacy
374
+
375
+ **Cloud mode sends your microphone audio to Deepgram.** It is billed per minute of
376
+ audio, it transcribes on their servers, and it is the default because it is the more
377
+ accurate of the two. Local mode sends nothing anywhere; use it for anything sensitive.
378
+ Buffered audio is held in memory only, is never written to disk, and is discarded when
379
+ the session ends.
380
+
381
+ ## Requirements
382
+
383
+ - Linux with `arecord` (alsa-utils), or SoX's `rec` as a fallback. macOS/Windows use
384
+ `rec` and are a degraded path — untested by us.
385
+ - Hold-to-talk needs Kitty keyboard protocol flag 2 for key *release*. It is verified
386
+ working on Kitty. Everywhere else it falls back to tap-to-toggle and says so once.
387
+ - The local backend needs `transcribe-cpp` and a Parakeet GGUF.
388
+
389
+ ## Development
390
+
391
+ ```bash
392
+ bun test # no network or microphone required
393
+ bunx tsc --noEmit
394
+ bunx biome check .
395
+
396
+ # live check against Deepgram, with and without keyterms
397
+ DEEPGRAM_API_KEY=... bun scripts/deepgram-smoke.ts speech.pcm
398
+ DEEPGRAM_API_KEY=... bun scripts/deepgram-smoke.ts speech.pcm subagent,worktree,tsconfig
399
+
400
+ # live check of the local backend (loads the GGUF, so it is slow)
401
+ bun scripts/parakeet-smoke.ts speech.pcm
402
+ ```
403
+
404
+ The pipeline is tested without a microphone, a network, or a model: capture, the
405
+ backend, and time are all injected seams, so every recovery rule has a test that runs
406
+ in milliseconds.