@agimon-ai/doompi-voice 0.0.1-alpha.21 → 0.0.1-alpha.22

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2) hide show
  1. package/README.md +74 -96
  2. package/package.json +5 -5
package/README.md CHANGED
@@ -1,122 +1,100 @@
1
1
  # @agimon-ai/doompi-voice
2
2
 
3
- **Speech in. Audio stays home.**
3
+ Speech capture, local transcription engines, playback, and autonomous Voice mode for DoomPi on macOS.
4
4
 
5
- This package adds local speech-to-text to Pi. Use it for one recording or leave autonomous
6
- capture on and keep talking to the same session. Audio stays on the machine; only bounded
7
- state metadata and transcript candidates return from the private worker.
5
+ Voice can transcribe one recording or keep an exact-active session listening. It also gives the primary agent a bounded `narrate` tool and accepts narration requests from other extensions.
8
6
 
9
- Voice is a minor mode. Autonomous capture reports itself beside `INSERT`, microphone
10
- activity appears briefly in the Doom modeline, and the primary agent speaks its own
11
- start, milestone, and final updates through a session-scoped `narrate` tool. Other
12
- extensions can still contribute spoken actions through the narration bus.
7
+ > **Alpha:** Voice state, tool, and platform support may change between releases.
13
8
 
14
- This is part of [Doompi](https://www.npmjs.com/package/@agimon-ai/doompi). Most users get
15
- it with the distribution.
9
+ ## Requirements
10
+
11
+ - Node.js 22.19.0 or newer
12
+ - Pi 0.84.2
13
+ - macOS recording and `say` playback support
14
+ - One supported local transcription engine: `whisper-cli`, `whisper`, or `mlx_whisper`
15
+ - A microphone permitted by macOS privacy settings
16
16
 
17
17
  ## Install
18
18
 
19
+ DoomPi loads the Doom adapter as core. For standalone Pi:
20
+
19
21
  ```bash
20
- npm install @agimon-ai/doompi-voice
22
+ pi install npm:@agimon-ai/doompi-voice
21
23
  ```
22
24
 
23
- ## How it loads
24
-
25
- Doompi loads Voice as core. It is not a major-mode layer and does not appear in
26
- `.doom/modes.yaml`. The bare package name follows its Pi manifest and selects the Doom
27
- adapter. Explicit `/extensions/pi` and `/extensions/doom` subpaths remain available.
28
-
29
- ## Leader
30
-
31
- - `SPC v v` records or transcribes once. It never enables the Voice minor mode or Voice-owned tools.
32
- - `SPC v a` toggles autonomous capture with primary-agent narration.
25
+ | Entry | Purpose |
26
+ | ----------------------------------------- | ------------------------------------------------------- |
27
+ | `@agimon-ai/doompi-voice/extensions/doom` | DoomPi mode, Leader, footer, and narration integration |
28
+ | `@agimon-ai/doompi-voice/extensions/pi` | Standalone Pi adapter |
29
+ | `@agimon-ai/doompi-voice/voice-tools` | Compatibility export for the shared Voice tool contract |
33
30
 
34
- ## Autonomous capture
31
+ ## Configure
35
32
 
36
- Configure autonomous capture in `~/.pi/.doom/config.yaml`:
33
+ A complete `~/.pi/.doom/config.yaml` example using Python Whisper is:
37
34
 
38
35
  ```yaml
39
36
  voice:
37
+ engine: openai-whisper
38
+ language: auto
39
+ recorder:
40
+ device: default
41
+ adapters:
42
+ openai-whisper:
43
+ binary: whisper
44
+ model:
45
+ id: base
40
46
  autoCapture:
41
47
  model: provider/model-id
48
+ startPhrases: []
49
+ stopPhrases: []
42
50
  utteranceIdleMs: 3000
51
+ transcriptionTimeoutMs: 120000
43
52
  tts:
44
53
  engine: macos-say
54
+ voice: Samantha
55
+ rate: 190
45
56
  ```
46
57
 
47
- `utteranceIdleMs` is the pause after the last voiced frame before Doompi treats the turn as
48
- complete. It accepts 1500 through 10000 milliseconds and defaults to 3000. The worker uses
49
- acoustic activity only as an endpoint hint: it keeps the full turn PCM, including pauses
50
- and quiet speech, and transcribes a new immutable full-turn revision for each decision.
51
- `autoCapture.model` performs bounded, patch-only command correction. It is also reused by
52
- the final-response safety net described below, but never rewrites a direct `narrate` call.
53
-
54
- ## Voice tools without a tool pile
55
-
56
- When the canonical autonomous session reaches exact `active`, Pi sees three Voice-owned
57
- tools while unrelated active tools keep their order:
58
-
59
- - `describe_voice_tools` returns the session's spoken capabilities and an opaque catalog token.
60
- - `use_voice_tools` executes a bounded batch against that token, in order.
61
- - `narrate` speaks one complete primary-agent-authored utterance verbatim.
62
-
63
- The first two names are façades. `narrate` is a separate mode-owned tool and never appears
64
- inside their capability catalog. Each narration is limited to 4,096 characters, awaits
65
- physical playback, and reports `completed`, `interrupted`, `superseded`, or `failed`.
66
- While the tool is available, the primary agent must call it when starting work, after an
67
- interesting or meaningful finding, before requesting user feedback or a decision, and
68
- before ending the task with a user-facing final response. One concise call is enough for a
69
- short conversation or clarification. Ordinary status, repetitive low-level progress, and
70
- intermediate response text remains silent.
71
-
72
- If a run reaches its final response without even attempting `narrate`, Voice provides one
73
- turn-end safety-net utterance. Any attempted call suppresses the fallback regardless of its
74
- playback outcome, so Voice never duplicates or retries agent-authored speech. A sanitized
75
- final of at most 320 characters is spoken deterministically. Longer finals are sanitized
76
- and bounded to 4,096 characters before `autoCapture.model` produces one concise JSON
77
- utterance with no reasoning, retries, or cache retention and an eight-second deadline.
78
- Malformed, failed, or timed-out generation degrades to a bounded deterministic excerpt
79
- instead of silence. This is the only lifecycle-derived speech; Voice does not synthesize
80
- automatic intent, plan, milestone, or tool-progress narration.
81
-
82
- Direct and fallback narration are gated by exact active state, matching TUI session, and
83
- the same activation generation. Starting, draining, shutdown, deactivation, reload, and
84
- stale-session execution fail closed. A stop, tool cancellation, or mode transition
85
- interrupts playback or pending fallback generation. Both paths enter the canonical
86
- playback coordinator at `final` priority. Task, workflow, and user-feedback packages may
87
- still publish higher-priority external narration through the shared narration contract.
88
-
89
- Extensions register callbacks through
90
- `@agimon-ai/doompi-extension-contracts/voice-tools`; those callbacks never become direct Pi
91
- tools. `@agimon-ai/doompi-voice/voice-tools` remains as a compatibility import for the same
92
- contract.
93
-
94
- A committed Voice-originated domain switch is the only reload that resumes capture. It
95
- starts fresh session, capture, and turn identities. A manual reload does not silently turn
96
- the microphone back on.
97
-
98
- ## Local worker and privacy
99
-
100
- Capture, PCM validation, voice activity detection, spooling, WAV creation, normalization,
101
- and Whisper all run in a private supervised worker thread. Audio and filesystem paths do
102
- not cross its control protocol; bounded state metadata and transcript candidates do.
103
-
104
- Manual capture stops explicitly or after five minutes. Autonomous capture uses the same
105
- five-minute bound for each spool. An idle window is replaced without disabling Voice; a
106
- window with confirmed speech is finalized for transcription.
107
-
108
- Unacknowledged turns remain in private `0700`/`0600` storage and are rediscovered after a
109
- worker restart. Neural VAD is advertised only when verified pure-WASM runtime and model
110
- assets exist; otherwise Voice uses its worker-side adaptive fallback.
111
-
112
- ## Entry points
113
-
114
- | Import | Purpose |
115
- | ----------------------------------------- | ----------------------------- |
116
- | `@agimon-ai/doompi-voice` | Library API |
117
- | `@agimon-ai/doompi-voice/extensions/pi` | Standalone Pi adapter |
118
- | `@agimon-ai/doompi-voice/extensions/doom` | Doompi adapter |
119
- | `@agimon-ai/doompi-voice/voice-tools` | Compatibility contract export |
58
+ Replace `provider/model-id` with a model configured in Pi, or omit `autoCapture` when only manual transcription is required. `utteranceIdleMs` accepts 1500–10000 ms and defaults to 3000.
59
+
60
+ ## Commands and tools
61
+
62
+ - `SPC v v` records/transcribes once; it does not enable autonomous Voice tools.
63
+ - `SPC v a` toggles autonomous capture.
64
+ - `describe_voice_tools` returns the exact-active session's spoken capability catalog.
65
+ - `use_voice_tools` executes a bounded ordered batch against that catalog.
66
+ - `narrate` speaks one primary-agent-authored utterance.
67
+
68
+ Each narration is limited to 4,096 characters, waits for playback, and returns `completed`, `interrupted`, `superseded`, or `failed`. Starting, draining, shutdown, reload, deactivation, or a stale session fails closed. Only the exact active TUI session receives Voice-owned tools.
69
+
70
+ If no `narrate` attempt is made before a final response, exact-active Voice can produce one bounded fallback utterance. Short finals use deterministic text; longer finals may use `autoCapture.model` for one bounded summary and degrade to a deterministic excerpt on failure. These optional model calls consume provider quota.
71
+
72
+ ## Data flow and recovery
73
+
74
+ Capture, PCM validation, activity detection, spooling, WAV creation, normalization, and configured local Whisper execution run in a private supervised worker. Private spool directories use mode `0700` and files use `0600`. Unacknowledged turns can be rediscovered after a worker restart.
75
+
76
+ This is not a blanket “nothing leaves the machine” guarantee:
77
+
78
+ - PCM/audio remains in local worker storage and playback paths.
79
+ - Transcript candidates and bounded state strings cross the worker/process boundary.
80
+ - Pi receives transcript text as user input.
81
+ - Command correction and long-final fallback text can be sent to the configured model provider.
82
+ - Telemetry may contain bounded operational metadata; review its sink configuration.
83
+
84
+ Manual recordings and autonomous spool windows are bounded to five minutes. Deactivation or cancellation interrupts pending capture/playback. A manual reload does not silently reactivate the microphone.
85
+
86
+ ## Public API
87
+
88
+ The root exports audio infrastructure, PCM/VAD/utterance services, narration and playback contracts, command correction, fallback narration, and Voice types. Host adapters should use the declared extension subpaths rather than generated paths.
89
+
90
+ ## Development
91
+
92
+ ```bash
93
+ pnpm build
94
+ pnpm typecheck
95
+ pnpm test
96
+ pnpm lint
97
+ ```
120
98
 
121
99
  ## License
122
100
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@agimon-ai/doompi-voice",
3
- "version": "0.0.1-alpha.21",
3
+ "version": "0.0.1-alpha.22",
4
4
  "description": "Local speech to text extension for Pi",
5
5
  "keywords": [
6
6
  "ai",
@@ -50,10 +50,10 @@
50
50
  "inversify": "8.2.1",
51
51
  "reflect-metadata": "0.2.2",
52
52
  "xstate": "5.32.5",
53
- "@agimon-ai/doompi-config": "0.0.1-alpha.21",
54
- "@agimon-ai/doompi-telemetry": "0.0.1-alpha.21",
55
- "@agimon-ai/doompi-extension-contracts": "0.0.1-alpha.21",
56
- "@agimon-ai/doompi-ui": "0.0.1-alpha.21"
53
+ "@agimon-ai/doompi-config": "0.0.1-alpha.22",
54
+ "@agimon-ai/doompi-extension-contracts": "0.0.1-alpha.22",
55
+ "@agimon-ai/doompi-telemetry": "0.0.1-alpha.22",
56
+ "@agimon-ai/doompi-ui": "0.0.1-alpha.22"
57
57
  },
58
58
  "devDependencies": {
59
59
  "@earendil-works/pi-coding-agent": "0.84.2",