@agimon-ai/doompi-voice 0.0.1-alpha.21 → 0.0.1-alpha.23
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +78 -96
- package/package.json +11 -9
package/README.md
CHANGED
|
@@ -1,122 +1,104 @@
|
|
|
1
1
|
# @agimon-ai/doompi-voice
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Speech capture, local transcription engines, playback, and autonomous Voice mode for DoomPi on macOS.
|
|
4
4
|
|
|
5
|
-
|
|
6
|
-
capture on and keep talking to the same session. Audio stays on the machine; only bounded
|
|
7
|
-
state metadata and transcript candidates return from the private worker.
|
|
5
|
+
Part of the [DoomPi distribution](https://www.npmjs.com/package/@agimon-ai/doompi).
|
|
8
6
|
|
|
9
|
-
Voice
|
|
10
|
-
activity appears briefly in the Doom modeline, and the primary agent speaks its own
|
|
11
|
-
start, milestone, and final updates through a session-scoped `narrate` tool. Other
|
|
12
|
-
extensions can still contribute spoken actions through the narration bus.
|
|
7
|
+
Voice can transcribe one recording or keep an exact-active session listening. It also gives the primary agent a bounded `narrate` tool and accepts narration requests from other extensions.
|
|
13
8
|
|
|
14
|
-
|
|
15
|
-
it with the distribution.
|
|
9
|
+
> **Alpha:** Voice state, tool, and platform support may change between releases.
|
|
16
10
|
|
|
17
|
-
##
|
|
11
|
+
## Requirements
|
|
18
12
|
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
13
|
+
- Node.js 22.19.0 or newer
|
|
14
|
+
- Pi 0.84.2
|
|
15
|
+
- macOS recording and `say` playback support
|
|
16
|
+
- One supported local transcription engine: `whisper-cli`, `whisper`, or `mlx_whisper`
|
|
17
|
+
- A microphone permitted by macOS privacy settings
|
|
22
18
|
|
|
23
|
-
##
|
|
19
|
+
## Install
|
|
24
20
|
|
|
25
|
-
|
|
26
|
-
`.doom/modes.yaml`. The bare package name follows its Pi manifest and selects the Doom
|
|
27
|
-
adapter. Explicit `/extensions/pi` and `/extensions/doom` subpaths remain available.
|
|
21
|
+
DoomPi loads the Doom adapter as core. For standalone Pi:
|
|
28
22
|
|
|
29
|
-
|
|
23
|
+
```bash
|
|
24
|
+
pi install npm:@agimon-ai/doompi-voice
|
|
25
|
+
```
|
|
30
26
|
|
|
31
|
-
|
|
32
|
-
|
|
27
|
+
| Entry | Purpose |
|
|
28
|
+
| ----------------------------------------- | ------------------------------------------------------- |
|
|
29
|
+
| `@agimon-ai/doompi-voice/extensions/doom` | DoomPi mode, Leader, footer, and narration integration |
|
|
30
|
+
| `@agimon-ai/doompi-voice/extensions/pi` | Standalone Pi adapter |
|
|
31
|
+
| `@agimon-ai/doompi-voice/voice-tools` | Compatibility export for the shared Voice tool contract |
|
|
33
32
|
|
|
34
|
-
##
|
|
33
|
+
## Configure
|
|
35
34
|
|
|
36
|
-
|
|
35
|
+
A complete `~/.pi/.doom/config.yaml` example using Python Whisper is:
|
|
37
36
|
|
|
38
37
|
```yaml
|
|
39
38
|
voice:
|
|
39
|
+
engine: openai-whisper
|
|
40
|
+
language: auto
|
|
41
|
+
recorder:
|
|
42
|
+
device: default
|
|
43
|
+
adapters:
|
|
44
|
+
openai-whisper:
|
|
45
|
+
binary: whisper
|
|
46
|
+
model:
|
|
47
|
+
id: base
|
|
40
48
|
autoCapture:
|
|
41
49
|
model: provider/model-id
|
|
50
|
+
startPhrases: []
|
|
51
|
+
stopPhrases: []
|
|
42
52
|
utteranceIdleMs: 3000
|
|
53
|
+
transcriptionTimeoutMs: 120000
|
|
43
54
|
tts:
|
|
44
55
|
engine: macos-say
|
|
56
|
+
voice: Samantha
|
|
57
|
+
rate: 190
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
Replace `provider/model-id` with a model configured in Pi, or omit `autoCapture` when only manual transcription is required. `utteranceIdleMs` accepts 1500–10000 ms and defaults to 3000.
|
|
61
|
+
|
|
62
|
+
## Commands and tools
|
|
63
|
+
|
|
64
|
+
- `SPC v v` records/transcribes once; it does not enable autonomous Voice tools.
|
|
65
|
+
- `SPC v a` toggles autonomous capture.
|
|
66
|
+
- `describe_voice_tools` returns the exact-active session's spoken capability catalog.
|
|
67
|
+
- `use_voice_tools` executes a bounded ordered batch against that catalog.
|
|
68
|
+
- `narrate` speaks one primary-agent-authored utterance.
|
|
69
|
+
|
|
70
|
+
Each narration is limited to 4,096 characters, waits for playback, and returns `completed`, `interrupted`, `superseded`, or `failed`. Starting, draining, shutdown, reload, deactivation, or a stale session fails closed. Only the exact active TUI session receives Voice-owned tools.
|
|
71
|
+
|
|
72
|
+
If no `narrate` attempt is made before a final response, exact-active Voice can produce one bounded fallback utterance. Short finals use deterministic text; longer finals may use `autoCapture.model` for one bounded summary and degrade to a deterministic excerpt on failure. These optional model calls consume provider quota.
|
|
73
|
+
|
|
74
|
+
## Data flow and recovery
|
|
75
|
+
|
|
76
|
+
Capture, PCM validation, activity detection, spooling, WAV creation, normalization, and configured local Whisper execution run in a private supervised worker. Private spool directories use mode `0700` and files use `0600`. Unacknowledged turns can be rediscovered after a worker restart.
|
|
77
|
+
|
|
78
|
+
This is not a blanket “nothing leaves the machine” guarantee:
|
|
79
|
+
|
|
80
|
+
- PCM/audio remains in local worker storage and playback paths.
|
|
81
|
+
- Transcript candidates and bounded state strings cross the worker/process boundary.
|
|
82
|
+
- Pi receives transcript text as user input.
|
|
83
|
+
- Command correction and long-final fallback text can be sent to the configured model provider.
|
|
84
|
+
- Telemetry may contain bounded operational metadata; review its sink configuration.
|
|
85
|
+
|
|
86
|
+
Manual recordings and autonomous spool windows are bounded to five minutes. Deactivation or cancellation interrupts pending capture/playback. A manual reload does not silently reactivate the microphone.
|
|
87
|
+
|
|
88
|
+
## Public API
|
|
89
|
+
|
|
90
|
+
The root exports audio infrastructure, PCM/VAD/utterance services, narration and playback contracts, command correction, fallback narration, and Voice types. Host adapters should use the declared extension subpaths rather than generated paths.
|
|
91
|
+
|
|
92
|
+
## Development
|
|
93
|
+
|
|
94
|
+
```bash
|
|
95
|
+
pnpm build
|
|
96
|
+
pnpm typecheck
|
|
97
|
+
pnpm test
|
|
98
|
+
pnpm lint
|
|
45
99
|
```
|
|
46
100
|
|
|
47
|
-
|
|
48
|
-
complete. It accepts 1500 through 10000 milliseconds and defaults to 3000. The worker uses
|
|
49
|
-
acoustic activity only as an endpoint hint: it keeps the full turn PCM, including pauses
|
|
50
|
-
and quiet speech, and transcribes a new immutable full-turn revision for each decision.
|
|
51
|
-
`autoCapture.model` performs bounded, patch-only command correction. It is also reused by
|
|
52
|
-
the final-response safety net described below, but never rewrites a direct `narrate` call.
|
|
53
|
-
|
|
54
|
-
## Voice tools without a tool pile
|
|
55
|
-
|
|
56
|
-
When the canonical autonomous session reaches exact `active`, Pi sees three Voice-owned
|
|
57
|
-
tools while unrelated active tools keep their order:
|
|
58
|
-
|
|
59
|
-
- `describe_voice_tools` returns the session's spoken capabilities and an opaque catalog token.
|
|
60
|
-
- `use_voice_tools` executes a bounded batch against that token, in order.
|
|
61
|
-
- `narrate` speaks one complete primary-agent-authored utterance verbatim.
|
|
62
|
-
|
|
63
|
-
The first two names are façades. `narrate` is a separate mode-owned tool and never appears
|
|
64
|
-
inside their capability catalog. Each narration is limited to 4,096 characters, awaits
|
|
65
|
-
physical playback, and reports `completed`, `interrupted`, `superseded`, or `failed`.
|
|
66
|
-
While the tool is available, the primary agent must call it when starting work, after an
|
|
67
|
-
interesting or meaningful finding, before requesting user feedback or a decision, and
|
|
68
|
-
before ending the task with a user-facing final response. One concise call is enough for a
|
|
69
|
-
short conversation or clarification. Ordinary status, repetitive low-level progress, and
|
|
70
|
-
intermediate response text remains silent.
|
|
71
|
-
|
|
72
|
-
If a run reaches its final response without even attempting `narrate`, Voice provides one
|
|
73
|
-
turn-end safety-net utterance. Any attempted call suppresses the fallback regardless of its
|
|
74
|
-
playback outcome, so Voice never duplicates or retries agent-authored speech. A sanitized
|
|
75
|
-
final of at most 320 characters is spoken deterministically. Longer finals are sanitized
|
|
76
|
-
and bounded to 4,096 characters before `autoCapture.model` produces one concise JSON
|
|
77
|
-
utterance with no reasoning, retries, or cache retention and an eight-second deadline.
|
|
78
|
-
Malformed, failed, or timed-out generation degrades to a bounded deterministic excerpt
|
|
79
|
-
instead of silence. This is the only lifecycle-derived speech; Voice does not synthesize
|
|
80
|
-
automatic intent, plan, milestone, or tool-progress narration.
|
|
81
|
-
|
|
82
|
-
Direct and fallback narration are gated by exact active state, matching TUI session, and
|
|
83
|
-
the same activation generation. Starting, draining, shutdown, deactivation, reload, and
|
|
84
|
-
stale-session execution fail closed. A stop, tool cancellation, or mode transition
|
|
85
|
-
interrupts playback or pending fallback generation. Both paths enter the canonical
|
|
86
|
-
playback coordinator at `final` priority. Task, workflow, and user-feedback packages may
|
|
87
|
-
still publish higher-priority external narration through the shared narration contract.
|
|
88
|
-
|
|
89
|
-
Extensions register callbacks through
|
|
90
|
-
`@agimon-ai/doompi-extension-contracts/voice-tools`; those callbacks never become direct Pi
|
|
91
|
-
tools. `@agimon-ai/doompi-voice/voice-tools` remains as a compatibility import for the same
|
|
92
|
-
contract.
|
|
93
|
-
|
|
94
|
-
A committed Voice-originated domain switch is the only reload that resumes capture. It
|
|
95
|
-
starts fresh session, capture, and turn identities. A manual reload does not silently turn
|
|
96
|
-
the microphone back on.
|
|
97
|
-
|
|
98
|
-
## Local worker and privacy
|
|
99
|
-
|
|
100
|
-
Capture, PCM validation, voice activity detection, spooling, WAV creation, normalization,
|
|
101
|
-
and Whisper all run in a private supervised worker thread. Audio and filesystem paths do
|
|
102
|
-
not cross its control protocol; bounded state metadata and transcript candidates do.
|
|
103
|
-
|
|
104
|
-
Manual capture stops explicitly or after five minutes. Autonomous capture uses the same
|
|
105
|
-
five-minute bound for each spool. An idle window is replaced without disabling Voice; a
|
|
106
|
-
window with confirmed speech is finalized for transcription.
|
|
107
|
-
|
|
108
|
-
Unacknowledged turns remain in private `0700`/`0600` storage and are rediscovered after a
|
|
109
|
-
worker restart. Neural VAD is advertised only when verified pure-WASM runtime and model
|
|
110
|
-
assets exist; otherwise Voice uses its worker-side adaptive fallback.
|
|
111
|
-
|
|
112
|
-
## Entry points
|
|
113
|
-
|
|
114
|
-
| Import | Purpose |
|
|
115
|
-
| ----------------------------------------- | ----------------------------- |
|
|
116
|
-
| `@agimon-ai/doompi-voice` | Library API |
|
|
117
|
-
| `@agimon-ai/doompi-voice/extensions/pi` | Standalone Pi adapter |
|
|
118
|
-
| `@agimon-ai/doompi-voice/extensions/doom` | Doompi adapter |
|
|
119
|
-
| `@agimon-ai/doompi-voice/voice-tools` | Compatibility contract export |
|
|
101
|
+
Maintained by [Agimon](https://agimon.ai/about).
|
|
120
102
|
|
|
121
103
|
## License
|
|
122
104
|
|
package/package.json
CHANGED
|
@@ -1,13 +1,15 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@agimon-ai/doompi-voice",
|
|
3
|
-
"version": "0.0.1-alpha.
|
|
4
|
-
"description": "
|
|
3
|
+
"version": "0.0.1-alpha.23",
|
|
4
|
+
"description": "macOS voice capture, transcription, text-to-speech, and autonomous narration for Pi agents.",
|
|
5
5
|
"keywords": [
|
|
6
|
-
"ai",
|
|
7
6
|
"coding-agent",
|
|
8
|
-
"developer-tools",
|
|
9
7
|
"doompi",
|
|
10
|
-
"
|
|
8
|
+
"macos",
|
|
9
|
+
"pi-extension",
|
|
10
|
+
"speech-to-text",
|
|
11
|
+
"text-to-speech",
|
|
12
|
+
"voice"
|
|
11
13
|
],
|
|
12
14
|
"homepage": "https://agimon.ai",
|
|
13
15
|
"license": "MIT",
|
|
@@ -50,10 +52,10 @@
|
|
|
50
52
|
"inversify": "8.2.1",
|
|
51
53
|
"reflect-metadata": "0.2.2",
|
|
52
54
|
"xstate": "5.32.5",
|
|
53
|
-
"@agimon-ai/doompi-
|
|
54
|
-
"@agimon-ai/doompi-
|
|
55
|
-
"@agimon-ai/doompi-
|
|
56
|
-
"@agimon-ai/doompi-ui": "0.0.1-alpha.
|
|
55
|
+
"@agimon-ai/doompi-extension-contracts": "0.0.1-alpha.23",
|
|
56
|
+
"@agimon-ai/doompi-config": "0.0.1-alpha.23",
|
|
57
|
+
"@agimon-ai/doompi-telemetry": "0.0.1-alpha.23",
|
|
58
|
+
"@agimon-ai/doompi-ui": "0.0.1-alpha.23"
|
|
57
59
|
},
|
|
58
60
|
"devDependencies": {
|
|
59
61
|
"@earendil-works/pi-coding-agent": "0.84.2",
|