osdy-pi 1.11.16 → 1.11.18
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +60 -6
- package/docs/kokoro.md +91 -0
- package/extensions/osdy-pi/control-center-speech.test.ts +280 -0
- package/extensions/osdy-pi/control-center-speech.ts +82 -0
- package/extensions/osdy-pi/control-center.test.ts +8 -8
- package/extensions/osdy-pi/control-center.ts +28 -7
- package/extensions/osdy-pi/kokoro-audio-player.test.ts +136 -0
- package/extensions/osdy-pi/kokoro-audio-player.ts +150 -0
- package/extensions/osdy-pi/kokoro-speech-adapter.test.ts +128 -0
- package/extensions/osdy-pi/kokoro-speech-adapter.ts +146 -0
- package/extensions/osdy-pi/response-speech.test.ts +369 -0
- package/extensions/osdy-pi/response-speech.ts +167 -0
- package/extensions/osdy-pi/runtime-speech.test.ts +99 -0
- package/extensions/osdy-pi/runtime.test.ts +15 -15
- package/extensions/osdy-pi/runtime.ts +35 -3
- package/extensions/osdy-pi/speech-adapter.test.ts +88 -0
- package/extensions/osdy-pi/speech-adapter.ts +106 -0
- package/extensions/osdy-pi/speech-service.test.ts +265 -0
- package/extensions/osdy-pi/speech-service.ts +130 -0
- package/extensions/osdy-pi/speech-settings.test.ts +31 -0
- package/extensions/osdy-pi/speech-settings.ts +68 -0
- package/extensions/osdy-pi/speech-text.test.ts +21 -0
- package/extensions/osdy-pi/speech-text.ts +41 -0
- package/extensions/osdy-pi/speech-types.ts +22 -0
- package/package.json +3 -2
package/README.md
CHANGED
|
@@ -8,7 +8,9 @@ Osdy Pi gives [Pi](https://github.com/earendil-works/pi) a themed, responsive te
|
|
|
8
8
|
|
|
9
9
|
**Audio notifications share a bundled 180 ms default sound on macOS and Windows**, with a persistent master mute in `/osdyConfig` → **Sounds**. Custom paths take precedence: startup flag → saved global path → bundled default. Muting preserves those paths and visual alerts; explicit **Test effective** still plays while muted. See [Audio notifications](#audio-notifications).
|
|
10
10
|
|
|
11
|
-
**
|
|
11
|
+
**Local response narration is opt-in on macOS**: `/osdyConfig` → **Speech** offers a one-line selector for installed macOS voices and optional local Kokoro voices, with explicit previews without saving or enabling narration. Stop from chat with **Ctrl+Option+S** or `/osdy-pi speech stop`. Speech stays independent of notification sounds and their mute. See [Response speech](#response-speech).
|
|
12
|
+
|
|
13
|
+
**The Control Center consolidates Osdy controls in `/osdyConfig`**: eleven categories cover appearance, Git, sounds, accounts, usage, TODO/Agents provider settings, and opt-in response speech. Account quota previews query a selected stored Codex profile without switching it. The header shows the installed Osdy package version when it fits; constrained layouts show only the centered mascot. See [Control Center in Pi](#control-center-in-pi) for confirmation and persistence boundaries.
|
|
12
14
|
|
|
13
15
|
**Codex banked resets show counts and per-reset local expiry** in `/usage` and the Control Center. Press **u** in `/usage` or select **Usage → Use or check banked reset** for Cancel-first, active-account use or pending-attempt checks; stored-profile previews stay read-only. See [Use or check a banked reset](#use-or-check-a-banked-reset).
|
|
14
16
|
|
|
@@ -16,7 +18,8 @@ Osdy Pi gives [Pi](https://github.com/earendil-works/pi) a themed, responsive te
|
|
|
16
18
|
|
|
17
19
|
| Area | What ships |
|
|
18
20
|
| --- | --- |
|
|
19
|
-
| Control Center | **Ctrl+Option+O** or `/osdyConfig` opens
|
|
21
|
+
| Control Center | **Ctrl+Option+O** or `/osdyConfig` opens eleven categories, including confirmed TODO/Agents provider selection, explicit per-profile Codex quota previews, and Speech. |
|
|
22
|
+
| Speech | Default-off local macOS narration, optional Spanish Kokoro voices, one saved voice, explicit draft previews, and chat-accessible Stop. |
|
|
20
23
|
| Audio | Shared bundled default, per-event custom paths, and persistent automatic-audio mute; [explicit tests remain available](#audio-notifications). |
|
|
21
24
|
| Local analytics (1.10.0) | `/osdy-usage` opens Summary, Profiles, Models, and History pages for recorded tokens and estimated cost—not billing or subscription quota. |
|
|
22
25
|
| Themes introduced in 1.11.0 | Spider-Man Classic, Miles Morales, and Spider-Verse add three distinct dark palettes. |
|
|
@@ -62,14 +65,15 @@ Osdy Pi's own extension and themes are bundled—do not install them separately.
|
|
|
62
65
|
|
|
63
66
|
### Control Center in Pi
|
|
64
67
|
|
|
65
|
-
Press **Ctrl+Option+O** or run **`/osdyConfig`** in interactive Pi to open the Osdy Control Center. The
|
|
68
|
+
Press **Ctrl+Option+O** or run **`/osdyConfig`** in interactive Pi to open the Osdy Control Center. The eleven categories—**Theme, Header, Mascot, Editor, Git, Sounds, Account, Usage, TODO, Agents, and Speech**—provide inline controls or data in the same modal. Existing commands remain available.
|
|
66
69
|
|
|
67
|
-
| Shortcut |
|
|
70
|
+
| Shortcut | Action | Command fallback |
|
|
68
71
|
| --- | --- | --- |
|
|
69
72
|
| **Ctrl+Option+O** | Osdy Control Center | `/osdyConfig` |
|
|
70
73
|
| **Ctrl+Option+U** | Active Codex subscription quota—not local token/cost analytics | `/usage` |
|
|
74
|
+
| **Ctrl+Option+S** | Stop speech from chat | `/osdy-pi speech stop` |
|
|
71
75
|
|
|
72
|
-
On macOS, **Option = Alt** in Pi's shortcut names: these bindings register as `ctrl+alt+o` and `ctrl+alt+
|
|
76
|
+
On macOS, **Option = Alt** in Pi's shortcut names: these bindings register as `ctrl+alt+o`, `ctrl+alt+u`, and `ctrl+alt+s`. On other platforms, use **Ctrl+Alt+O**, **Ctrl+Alt+U**, and **Ctrl+Alt+S**. Your terminal must send Option as **Alt/Meta** and support **Ctrl+Alt chords**; registration does not guarantee actual hardware delivery. Check **`/hotkeys`** for bindings, and use the command fallback if the terminal, OS, or another extension intercepts or cannot deliver the chord.
|
|
73
77
|
|
|
74
78
|
**Ctrl+Option+O capability limits:** Account **Switch**, **Set default**, and **Clear default** refuse changes while Pi is busy; retry when idle. The same actions through `/osdyConfig` retain idle waiting, as does switching through `/osdy-account`. Read-only account quota previews remain available while busy. Successful TODO/Agents selection from Ctrl+Option+O saves settings, closes/disposes the panel, releases its editor hold, then asks you to run **`/reload` manually** (or restart Pi). From `/osdyConfig`, the same sequence still reloads automatically once; restart Pi if reload fails.
|
|
75
79
|
|
|
@@ -89,6 +93,7 @@ Positive counts also trigger a best-effort read-only reset-details GET within th
|
|
|
89
93
|
- **Git** shows the current branch (or detached HEAD), changed-entry count/clean status, and the current widget position as a read-only session detail. Working-tree enablement applies and saves through the same owner as `/osdy-pi working-tree`; Git inspection never writes Git. `/osdy-pi diff` and position commands are unchanged.
|
|
90
94
|
- **Sounds** shows each event's saved and effective path/source. Configure opens a single-line path editor **inside this overlay**: enter a readable `.mp3`/`.wav` path, then **Enter** to validate and save immediately, or **Esc** to cancel editing. Relative paths resolve against the project directory and save as absolute paths. Clear removes only that event's saved path from the existing v1 audio store (`<agent-dir>/extensions/osdy-pi/audio-notifications.json`). Notifications read this store afresh; refresh failures are reported separately from save failures.
|
|
91
95
|
- **Sounds → Mute automatic audio / Enable automatic audio** is a persistent master switch, enabled by default (including existing v1 files). Muting suppresses all automatic event sounds, even startup-flag paths, without losing saved paths or changing visual notifications. It applies to the next notification without reload; already-started playback is not stopped. **Test effective** still plays while muted; opening categories and configuring paths never play. Startup `--osdy-pi-sound-<event>` flags override saved paths, so **clearing a saved path does not disable flag-configured playback**. Tests report unavailable files/platforms or playback failures; playback remains macOS/Windows only. Configure, Clear, and the legacy `/osdy-pi sound setup` wizard preserve the master state.
|
|
96
|
+
- **Speech** shows enabled state, saved voice, and rate, with **Enable/Disable**, **Read last response**, **Stop**, one horizontal **Voice** selector, and explicit **Preview voice**. On the Voice row, **←/→** browses a draft and **Enter** saves; **Tab** returns to categories. Preview plays a fixed sample of the draft without saving or enabling narration. Speech is independent of Sounds mute. See [Response speech](#response-speech).
|
|
92
97
|
- **Account** shows one row per validated profile, marked **Active** and **Default** where applicable. Select a profile, then use its visible context bar: **v View usage**, **s Switch** (inactive profiles), **d Set default**, and **c Clear default** (when a default exists). **v** or **Enter** explicitly queries that profile's stored Codex credentials without activating it; no second action list appears. Opening, navigating or highlighting never queries; stored profiles are never polled. A dedicated quota view reuses `/usage` dashboard content: Session/Weekly cards, additional bucket groups, remaining-capacity bars, resets, and the quotas/account summary with plan and credits when available. Identity belongs to the selected stored profile; unknown provider/model metadata is labeled honestly. Checked and updated times use local time; absent data is unknown, not zero. Refresh/Back controls and query feedback stay pinned while long dashboard content scrolls. **r Refresh** repeats the selected profile query; **s Switch**, when available, confirms activation of the currently viewed inactive profile. Cancelling that confirmation preserves the quota view and scroll position. **b Back** or **Esc** returns to the overview with the profile selected, without closing the overlay. In quota details, **↑/↓** selects Refresh/Back and **Enter** activates it; **Page Up/Down** and **Home/End** scroll long quota data. Switch, set default, and clear default require an **in-screen confirmation** with Cancel selected initially; **Esc** cancels confirmation without closing the overlay. Single blank separators group the confirmation title, complete effects, choices, and instructions; gaps yield on short viewports. Resize if the full confirmation cannot fit—acceptance stays hidden and blocked. Switching through `/osdyConfig` waits for Pi to become idle; Ctrl+Option+O refuses busy account changes. Successful switching activates through the existing backend and refreshes subscription usage. Login remains Pi-owned (`/login`); create/rename/remove remain available in `/osdy-account`. Category navigation never changes profiles or credentials.
|
|
93
98
|
- **Usage** shows the runtime's cached active Codex subscription quota (remaining-capacity bars and percentages, window, reset/fetch times) and the existing local history owner's recorded token/cache totals, cost gaps, and model detail. Select **Range: day/week/month** for the current local calendar period; toggle **Accounts: all/current** for local analytics (unmanaged when no valid current profile exists). Quota always belongs to the active account, not the analytics filter; local token/cost totals never become quota percentages. Select **Refresh quota and local history** explicitly; rendering and filters never fetch. Empty, loading, unavailable, and limited/incomplete coverage are reported honestly. Estimates are not billing or subscription quota. `/usage` remains the subscription dashboard; `/osdy-usage` remains local analytics.
|
|
94
99
|
- **Esc** closes without undoing live or saved changes, except when cancelling a confirmation/input or returning from Account quota details. Account read-only queries are cancelled on Back/Esc, category exit or close; late results are ignored even if cancellation is not honored. Other in-flight operations still finish and are not claimed cancelled. Late results cannot replace a closed overlay or another category. On narrow terminals, categories and details share the viewport; use Tab to switch.
|
|
@@ -438,6 +443,8 @@ The enabled state, editor mode, working-tree visibility preference, header, and
|
|
|
438
443
|
| Working tree | `/osdy-pi working-tree on\|off\|toggle\|status` |
|
|
439
444
|
| Working tree | `/osdy-pi working-tree position top\|bottom\|status` |
|
|
440
445
|
| Audio | `/osdy-pi sound setup` |
|
|
446
|
+
| Response speech | `/osdy-pi speech on\|off\|status\|stop\|last` |
|
|
447
|
+
| Response speech Stop | **Ctrl+Option+S** or `/osdy-pi speech stop` |
|
|
441
448
|
| Diff | `/osdy-pi diff` |
|
|
442
449
|
| Package | `/osdy-pi uninstall` |
|
|
443
450
|
| Codex subscription | **Ctrl+Option+U** or `/usage` |
|
|
@@ -567,6 +574,52 @@ Use `working-tree position top` or `bottom` to place the summary above or below
|
|
|
567
574
|
| Go back | `esc`, `backspace`, `left`, or `h` |
|
|
568
575
|
| Close | `q` or `ctrl+c` |
|
|
569
576
|
|
|
577
|
+
## Response speech
|
|
578
|
+
|
|
579
|
+
**Off by default.** Interactive Pi can read the settled final assistant response locally on macOS using the feminine **Paulina** voice (Spanish, Mexico), initially at **180 words/minute**. No cloud speech service is used. For npm installs, confirm the resolved release includes Speech; merging source alone does not prove publication.
|
|
580
|
+
|
|
581
|
+
| Command | Action |
|
|
582
|
+
| --- | --- |
|
|
583
|
+
| `/osdy-pi speech on` | Save automatic narration enabled. |
|
|
584
|
+
| `/osdy-pi speech off` | Save automatic narration disabled and stop pending/active narration. |
|
|
585
|
+
| `/osdy-pi speech status` | Show enabled state, voice, rate, and platform availability. |
|
|
586
|
+
| `/osdy-pi speech last` | Read the current session's last successful final prose, even while automatic narration is off; refuses while Pi is busy. |
|
|
587
|
+
| `/osdy-pi speech stop` | Cancel pending starts and active narration without changing the enabled preference. |
|
|
588
|
+
|
|
589
|
+
From chat, press **Ctrl+Option+S** on macOS (**Ctrl+Alt+S** elsewhere) for the same Stop action, including preview playback. It works while Pi is busy without aborting the agent, editing your input, or changing the saved enabled/voice/rate settings. Explicit Stop suppresses automatic speech for the rest of the current run, not later runs. Run `/reload` or restart Pi after updating Osdy; check `/hotkeys` and use `/osdy-pi speech stop` if your terminal cannot deliver the chord.
|
|
590
|
+
|
|
591
|
+
Open **`/osdyConfig` → Speech** (the last category) to enable/disable narration, read the last response, stop playback, or select a voice:
|
|
592
|
+
|
|
593
|
+
1. Use **↑/↓** to reach the single **Voice: ‹ … ›** row. **←/→** browses installed macOS voices and **Kokoro Dora / Kokoro Alex / Kokoro Santa**, wrapping at both ends. The row marks the draft **saved** or **unsaved**; the summary always identifies the saved voice. Browsing never saves or plays audio.
|
|
594
|
+
2. Select **Preview voice** and press **Enter** to hear a fixed Spanish sample for Kokoro or the existing fixed English sample for native voices, independent of the last response. Preview does **not** save the draft or enable narration, and works while narration is disabled. It may replace current playback. Leaving Speech or closing the panel cancels only its preview, not unrelated narration.
|
|
595
|
+
3. Return to **Voice** and press **Enter** to save the draft for subsequent narration. Preview, other controls, and failed saves preserve the draft. **Read last response** uses the saved voice, not an unsaved draft. Saving never plays audio, enables narration, or changes already-playing audio.
|
|
596
|
+
|
|
597
|
+
Opening Speech starts at the playback controls; **Home** returns there, **Tab** returns to categories, and arrows outside the Voice row retain normal pane focus behavior. The feminine Paulina voice remains the default until you save another voice.
|
|
598
|
+
|
|
599
|
+
The panel shows enabled/saved-voice/rate state. Native speech uses supported voices already installed on the Pi host and needs no server or model download. Optional **Kokoro** needs a separately started local MLX-Audio server and an explicitly downloaded model; listing or saving a voice does not download anything. See [local Kokoro setup](docs/kokoro.md) for prerequisites and validation status. The extension never installs dependencies or starts the server automatically. Empty/unavailable voice lists leave narration controls accessible. Rate tuning is not exposed; the existing wpm display maps to Kokoro speed as **rate / 180**, not a guarantee of identical spoken wpm. Selection saves globally in the independent `<agent-dir>/extensions/osdy-pi/speech.json` store and survives reload/restart; **Sounds mute and event notifications are unchanged and independent**.
|
|
600
|
+
|
|
601
|
+
### Optional Kokoro voices
|
|
602
|
+
|
|
603
|
+
On Apple Silicon Macs, the same selector includes three local Spanish voices:
|
|
604
|
+
|
|
605
|
+
| Voice | Preset |
|
|
606
|
+
| --- | --- |
|
|
607
|
+
| **Kokoro Dora** | Feminine, `ef_dora` |
|
|
608
|
+
| **Kokoro Alex** | Masculine, `em_alex` |
|
|
609
|
+
| **Kokoro Santa** | Masculine, `em_santa` |
|
|
610
|
+
|
|
611
|
+
Follow [the Kokoro setup and startup guide](docs/kokoro.md) to prepare the isolated Python environment, explicitly download the free model, and start MLX-Audio on **`http://127.0.0.1:8000`**. Keep that service running while using Kokoro: the extension does **not** start it automatically. A preview reporting **`Kokoro synthesis failed`** can mean the server is stopped; check the service and the optional `OSDY_PI_KOKORO_URL` loopback origin before reinstalling anything. Native voices do not need this service.
|
|
612
|
+
|
|
613
|
+
After starting it, browse to a Kokoro voice and use **Preview voice** to listen without changing your saved voice or enabling narration. Save only when you want subsequent narration to use that voice. Generation is local, but the complete WAV is produced before playback; this is not streamed narration. Initial model loading is slower than warm requests.
|
|
614
|
+
|
|
615
|
+
**Known limit:** long Spanish responses may be truncated by the current upstream phonemizer. Osdy normalizes newlines, so adding line breaks is not a workaround in this integration. Prefer an installed native voice for long responses until chunking/completeness is evaluated. The [guide](docs/kokoro.md#observed-local-synthesis) records tested versions and short-sample measurements; these are not quality or latency guarantees.
|
|
616
|
+
|
|
617
|
+
### Narration lifecycle and compatibility
|
|
618
|
+
|
|
619
|
+
Automatic narration waits for final settlement, including retries, compaction, and queued continuations—not every message or intermediate agent end. Only successful final text is read, omitting fenced code, tools, thinking, and Markdown noise. Cancelled, failed, empty, and tool-only newest responses never fall back to older text automatically. Explicit replay retains the last successful response only in memory for this session; switching sessions, reload, and shutdown clear it. New input/runs stop narration, and queued follow-ups remain eligible once their final response settles. Explicit **Stop** also cancels pending starts and suppresses automatic narration for the rest of that run; explicit replay remains available when idle. Cancelling post-run compaction discards that run's pending response. Speech launch never waits for the full spoken playback in an agent lifecycle handler.
|
|
620
|
+
|
|
621
|
+
Speech is **TUI only**: child RPC, JSON, and print runs never play it. Unsupported platforms or an unavailable selected voice report failure for explicit playback; no substitute voice is silently used. Automatic failures stay silent rather than interrupting the agent. Audio plays on the **Pi host**: over SSH, that means the remote host's audio output, not your local terminal speakers.
|
|
622
|
+
|
|
570
623
|
## Audio notifications
|
|
571
624
|
|
|
572
625
|
Osdy Pi plays one shared, short, subtle original bundled WAV by default for every unconfigured audio event on macOS and Windows. Automatic audio is enabled by default; mute it in **Sounds** to turn it off without changing visual alerts. Custom readable `.mp3` or `.wav` files can replace the default per event. Other platforms safely skip playback. macOS playback requires the system `afplay` command; Windows playback requires `powershell.exe` and the Windows Media Player COM component (`WMPlayer.OCX`).
|
|
@@ -650,10 +703,11 @@ This launches `pi` from `PATH`, loads this checkout with `-e`, and uses `.pi-dev
|
|
|
650
703
|
The explicit argument bypasses the launcher's default-account activation; Pi's `--no-session` disables session persistence, **not settings writes or provider access**.
|
|
651
704
|
An isolated agent directory does **not** make custom audio paths or real provider accounts harmless. Use intentional disposable test profiles/files for profile writes, switches, default confirmations, and audio tests; do not assume an isolated account store makes real credentials safe.
|
|
652
705
|
|
|
653
|
-
- Check `/hotkeys` lists `ctrl+alt+o` and `ctrl+alt+
|
|
706
|
+
- Check `/hotkeys` lists `ctrl+alt+o`, `ctrl+alt+u`, and `ctrl+alt+s`. On macOS, configure Option to send Alt/Meta and verify the terminal supports Ctrl+Alt chords; actual Ctrl+Option delivery still needs a hardware check. Open both **Ctrl+Option+O** and `/osdyConfig` in regular and fullscreen Pi (Ctrl+Alt+O on other platforms); use the slash command if the chord is intercepted or unsupported. Check arrows/Home/End/Page Up/Down, Tab or left/right, Enter, and Esc; shrink and restore the terminal, checking both panes remain reachable. With disposable TODO/Agents settings, verify Ctrl+Option+O asks for manual `/reload` only after closing, while `/osdyConfig` reloads automatically.
|
|
654
707
|
- In disposable settings, change a visual preference, wait for **Saved globally**, close and reopen, then reload/restart to check persistence. Esc does not undo changes; distinguish save/application errors from success.
|
|
655
708
|
- Account (manual): with disposable profiles only, check Cancel-first switch/set-default/clear-default confirmations, Esc cancellation, idle switching, and current/default labels. Verify Ctrl+Option+O refuses all three changes while busy, succeeds when idle, and `/osdyConfig` still waits for idle. Check one row per profile with **v/s/d/c** context actions and **Enter** usage fallback, without a second action list. From **v** quota details, check eligible **s Switch** opens the same Cancel-first confirmation, with blank separators between the title/effects, choices, and instructions; Cancel/Esc must retain the quota view and scroll position. Check gaps yield on short viewports without hiding safety content or enabling acceptance when the full confirmation cannot fit. Check remaining-capacity bars, **r Refresh**, **b Back/Esc** returning to the selected profile, long-view scrolling, no query on navigation, no stored-profile polling, and cancellation on exit. Create/rename/remove stay in `/osdy-account`, login in `/login`.
|
|
656
709
|
- Subscription quota (manual): compare **Ctrl+Option+U** and `/usage` (Ctrl+Alt+U on other platforms), including **r Refresh** and Esc/q closing. Verify actual terminal chord delivery; use `/usage` if intercepted or unsupported.
|
|
710
|
+
- Speech Stop (manual): during intentional narration or a voice preview, press **Ctrl+Option+S** in chat; confirm audio stops without aborting the agent or changing editor text/settings. Check actual terminal delivery, with `/osdy-pi speech stop` as fallback. Registration and automated callback tests do not establish a hardware pass.
|
|
657
711
|
- Usage (manual): check day/week/month and all/current filters, empty/unavailable/limited states, and explicit refresh. Authenticated quota and post-switch account correctness need an intentionally authorized live provider check; local estimates are not billing.
|
|
658
712
|
- Sounds (manual): use disposable readable `.mp3`/`.wav` files to check configure, cancel, clear, and startup-flag precedence. Only choose **Test effective** when playback is intended. The user reported successful live playback of the bundled default; exhaustive macOS/Windows coverage still needs a live check.
|
|
659
713
|
|
package/docs/kokoro.md
ADDED
|
@@ -0,0 +1,91 @@
|
|
|
1
|
+
# Optional local Spanish speech with Kokoro
|
|
2
|
+
|
|
3
|
+
Kokoro adds **Kokoro Dora**, **Kokoro Alex**, and **Kokoro Santa** to the existing Speech selector. Native macOS speech remains the default: **Paulina**, narration **off**, **180 wpm**. Native voices never need a server. Kokoro uses a free local model, not cloud speech or voice cloning; initial playback support is macOS only.
|
|
4
|
+
|
|
5
|
+
## Quick path
|
|
6
|
+
|
|
7
|
+
1. On an Apple Silicon Mac, explicitly prepare the isolated Python environment below and start the local MLX-Audio service in a separate terminal.
|
|
8
|
+
2. In Pi, open **`/osdyConfig` → Speech → Voice**, browse with **←/→**, then choose **Preview voice**. Kokoro previews use a fixed Spanish sample; native voices retain their fixed English sample. Preview uses the unsaved draft without saving it or enabling narration.
|
|
9
|
+
3. Return to **Voice** and press **Enter** to save the draft. **Read last response** uses the saved voice even while narration is disabled. Enable automatic narration only if wanted.
|
|
10
|
+
4. Use **Stop**, **Ctrl+Option+S** from chat, or **`/osdy-pi speech stop`** to cancel pending synthesis or playback. If the terminal cannot deliver the shortcut, use the command. Leaving Speech or closing its panel stops only its preview, not unrelated narration.
|
|
11
|
+
|
|
12
|
+
## Explicit environment setup
|
|
13
|
+
|
|
14
|
+
Prerequisites: an **Apple Silicon Mac** and **uv**. The example explicitly downloads a managed Python 3.11 interpreter inside the project rather than installing Python packages globally. No npm runtime dependency is added.
|
|
15
|
+
|
|
16
|
+
The isolated environment was prepared successfully on an **M3 Pro / 18 GiB RAM** with **Python 3.11.15**, **MLX-Audio 0.5.9**, **Misaki 0.9.4**, and **espeakng-loader 0.2.4**. A Spanish `EspeakG2P(language="es")` probe returned phonemes successfully. `espeakng-loader` supplies the native library used by Misaki in this setup; no separate global Homebrew install was needed. All three Spanish voices also produced valid 24 kHz mono WAV through the local HTTP server. Audible playback and subjective listening quality were not evaluated.
|
|
17
|
+
|
|
18
|
+
Run from the project root. These environment variables must also be set in every terminal that runs setup, model download, or the server:
|
|
19
|
+
|
|
20
|
+
```bash
|
|
21
|
+
export KOKORO_HOME="$PWD/.pi-dev/kokoro"
|
|
22
|
+
export UV_CACHE_DIR="$KOKORO_HOME/cache/uv"
|
|
23
|
+
export UV_PYTHON_INSTALL_DIR="$KOKORO_HOME/python"
|
|
24
|
+
export HF_HOME="$KOKORO_HOME/cache/huggingface"
|
|
25
|
+
export HF_HUB_CACHE="$HF_HOME/hub"
|
|
26
|
+
export HUGGINGFACE_HUB_CACHE="$HF_HUB_CACHE"
|
|
27
|
+
export XDG_CACHE_HOME="$KOKORO_HOME/cache/xdg"
|
|
28
|
+
export PIP_CACHE_DIR="$KOKORO_HOME/cache/pip"
|
|
29
|
+
export TORCH_HOME="$KOKORO_HOME/cache/torch"
|
|
30
|
+
export NUMBA_CACHE_DIR="$KOKORO_HOME/cache/numba"
|
|
31
|
+
export MPLCONFIGDIR="$KOKORO_HOME/cache/matplotlib"
|
|
32
|
+
export PYTHONPYCACHEPREFIX="$KOKORO_HOME/cache/pycache"
|
|
33
|
+
export HF_HUB_DISABLE_TELEMETRY=1
|
|
34
|
+
mkdir -p "$KOKORO_HOME"
|
|
35
|
+
uv --no-config venv --managed-python --python 3.11 "$KOKORO_HOME/venv"
|
|
36
|
+
uv --no-config pip install --python "$KOKORO_HOME/venv/bin/python" \
|
|
37
|
+
'mlx-audio[server,tts]==0.5.9' 'misaki[en]==0.9.4' \
|
|
38
|
+
'espeakng-loader==0.2.4' soundfile
|
|
39
|
+
# Explicitly download before playback: a first download can exceed the request timeout.
|
|
40
|
+
"$KOKORO_HOME/venv/bin/python" -c \
|
|
41
|
+
'from huggingface_hub import snapshot_download; snapshot_download("mlx-community/Kokoro-82M-bf16")'
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
The managed interpreter, virtualenv, package caches, and model cache are explicitly project-contained. If you prefer an existing Python 3.10+ interpreter, omit `--managed-python` and supply its path to `--python`. Do not silently fall back to a global package environment. Keep any additional dependency caches project-local too.
|
|
45
|
+
|
|
46
|
+
### Start the server separately
|
|
47
|
+
|
|
48
|
+
With the same environment variables, start the server. These bind flags were exercised successfully with MLX-Audio 0.5.9; inspect `--help` when changing versions:
|
|
49
|
+
|
|
50
|
+
```bash
|
|
51
|
+
"$KOKORO_HOME/venv/bin/python" -m mlx_audio.server --help
|
|
52
|
+
"$KOKORO_HOME/venv/bin/python" -m mlx_audio.server --host 127.0.0.1 --port 8000
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
Bind only to **127.0.0.1:8000**, never `0.0.0.0` or a LAN address. Leave the process running to keep the model warm. Model files are downloaded only during explicit model setup or the first explicit Kokoro request, not by catalog browsing, selection, or native speech. The extension never installs Python packages, downloads models on startup, or launches the server automatically. Dependency/model setup can require internet access; synthesis sends response text only to the local service. Use a trusted local service that does not forward or log sensitive text.
|
|
56
|
+
|
|
57
|
+
## Integration contract
|
|
58
|
+
|
|
59
|
+
| Setting | Fixed behavior |
|
|
60
|
+
| --- | --- |
|
|
61
|
+
| Default origin | `http://127.0.0.1:8000` |
|
|
62
|
+
| Optional override | `OSDY_PI_KOKORO_URL`, set in Pi's launch environment |
|
|
63
|
+
| Allowed origins | HTTP with literal `127.0.0.1` or `[::1]` only; optional port, no path beyond `/`, credentials, query, or fragment. Hostnames including `localhost` are not accepted. Redirects are never followed. |
|
|
64
|
+
| Model | `mlx-community/Kokoro-82M-bf16` |
|
|
65
|
+
| Voices | `Kokoro Dora` → `ef_dora`; `Kokoro Alex` → `em_alex`; `Kokoro Santa` → `em_santa` |
|
|
66
|
+
| Language / format | Spanish `lang_code: "e"`; complete WAV generated before playback |
|
|
67
|
+
| Rate | Existing wpm setting mapped to bounded Kokoro `speed = rate / 180`; 180 maps to 1.0, not a measured words-per-minute promise |
|
|
68
|
+
| Settings | Existing v1 voice-name setting; no new engine selector, command grammar, or migration |
|
|
69
|
+
|
|
70
|
+
This is **not audio streaming**. Automatic speech waits for the successful settled final assistant response; it does not speak unfinished responses, thinking, tools, or fenced code. The first cold request may include model download/loading and be much slower than warm synthesis. Long responses also wait for the full WAV before playback; no latency or quality target is established here.
|
|
71
|
+
|
|
72
|
+
## Failures and verification
|
|
73
|
+
|
|
74
|
+
An unavailable server, missing Spanish dependency, rejected origin, synthesis failure, or playback failure is an error—not permission to silently substitute a native voice. Explicit playback reports failure; automatic speech failures do not interrupt the agent. Choose an installed native voice explicitly if you want to continue without Kokoro. Speech remains independent of Sounds mute.
|
|
75
|
+
|
|
76
|
+
### Observed local synthesis
|
|
77
|
+
|
|
78
|
+
An isolated MLX-Audio 0.5.9 server on an M3 Pro / 18 GiB generated the same short Spanish sample through `/v1/audio/speech` with the fixed model/language/WAV contract:
|
|
79
|
+
|
|
80
|
+
| Request | Complete WAV generation | Audio duration |
|
|
81
|
+
| --- | --- | --- |
|
|
82
|
+
| Dora, first inference after download | 5.664 s | 4.350 s |
|
|
83
|
+
| Dora, same sample with warm model | 0.268 s | 4.350 s |
|
|
84
|
+
| Alex | 1.176 s | 4.325 s |
|
|
85
|
+
| Santa | 1.123 s | 4.325 s |
|
|
86
|
+
|
|
87
|
+
The one-time model download took about 291 seconds. These are short-sample measurements, not general latency guarantees or time-to-first-stream benchmarks. Generated WAVs were validated without playing sound; no subjective naturalness or accent ranking is established. Automated tests and independent public-interface probes passed for request cancellation, native independence, selector drafts and settings preservation. The independent check also exercised a real filesystem and a harmless child process: private directory/file modes were 0700/0600, Stop terminated the process, and WAV cleanup waited for child close. No audible player was invoked.
|
|
88
|
+
|
|
89
|
+
**Long Spanish responses are not yet validated.** MLX-Audio's Spanish phonemizer warns: `Chunking logic not yet implemented, so long texts may be truncated unless you split them with '\n'.` Osdy currently normalizes newlines before synthesis, so adding newlines to a response is not an app-level workaround. Prefer a native voice for long responses until Kokoro chunking/completeness is evaluated. Listening quality, audible-device playback, physical shortcut delivery, clean-machine installation and installed-package execution remain unverified.
|
|
90
|
+
|
|
91
|
+
Primary references: [MLX-Audio server and TTS guidance](https://github.com/Blaizzy/mlx-audio) and [Kokoro-82M model/language documentation](https://huggingface.co/hexgrad/Kokoro-82M).
|
|
@@ -0,0 +1,280 @@
|
|
|
1
|
+
import assert from "node:assert/strict";
|
|
2
|
+
import { existsSync } from "node:fs";
|
|
3
|
+
import { registerHooks } from "node:module";
|
|
4
|
+
import test from "node:test";
|
|
5
|
+
import { fileURLToPath } from "node:url";
|
|
6
|
+
import type { ExtensionContext } from "@earendil-works/pi-coding-agent";
|
|
7
|
+
import type { SpeechSettings } from "./speech-types.js";
|
|
8
|
+
import { visibleWidth } from "@earendil-works/pi-tui";
|
|
9
|
+
|
|
10
|
+
registerHooks({ resolve(specifier, context, nextResolve) {
|
|
11
|
+
if (specifier.startsWith(".") && specifier.endsWith(".js") && context.parentURL?.endsWith(".ts")) {
|
|
12
|
+
const url = new URL(`${specifier.slice(0, -3)}.ts`, context.parentURL);
|
|
13
|
+
if (existsSync(fileURLToPath(url))) return { shortCircuit: true, url: url.href };
|
|
14
|
+
}
|
|
15
|
+
return nextResolve(specifier, context);
|
|
16
|
+
} });
|
|
17
|
+
const { CONTROL_CENTER_CATEGORIES, ControlCenter } = await import("./control-center.js");
|
|
18
|
+
const { createControlCenterSpeech } = await import("./control-center-speech.js");
|
|
19
|
+
|
|
20
|
+
void test("Speech appends after all existing categories without changing their indices", () => {
|
|
21
|
+
assert.deepEqual([...CONTROL_CENTER_CATEGORIES], ["Theme", "Header", "Mascot", "Editor", "Git", "Sounds", "Account", "Usage", "TODO", "Agents", "Speech"]);
|
|
22
|
+
});
|
|
23
|
+
|
|
24
|
+
void test("Speech displays enabled/voice/rate and routes only Enable/Disable, Read last response and Stop", async () => {
|
|
25
|
+
let settings: SpeechSettings = { version: 1, enabled: false, voice: "Paulina", rate: 180 };
|
|
26
|
+
const actions: string[] = [];
|
|
27
|
+
const speech = createControlCenterSpeech({ settings: () => Promise.resolve(settings),
|
|
28
|
+
voices: () => Promise.resolve([]),
|
|
29
|
+
selectVoice: () => Promise.resolve({ failed: false, message: "Voice saved" }),
|
|
30
|
+
previewVoice: () => Promise.resolve({ failed: false, message: "Preview started" }), stopPreview: () => {},
|
|
31
|
+
command: args => {
|
|
32
|
+
actions.push(args.join(" "));
|
|
33
|
+
if (args[0] === "on" || args[0] === "off") settings = { ...settings, enabled: args[0] === "on" };
|
|
34
|
+
return Promise.resolve({ failed: args[0] === "last", message: args[0] === "last" ? "No successful final response" : "Speech applied" });
|
|
35
|
+
} }, {} as ExtensionContext);
|
|
36
|
+
const initial = await speech.read();
|
|
37
|
+
assert.match(initial.summary, /disabled.*Paulina.*180 wpm/);
|
|
38
|
+
assert.deepEqual(initial.rows.map(row => row.label), ["Enable", "Read last response", "Stop"]);
|
|
39
|
+
const panel = new ControlCenter({ speech,
|
|
40
|
+
theme: () => ({ name: "dark", appearance: "dark", fg: (_color, text) => text }),
|
|
41
|
+
readThemes: () => [], applyTheme: () => ({ success: true }), requestRender: () => {}, height: () => 30, close: () => {},
|
|
42
|
+
});
|
|
43
|
+
panel.handleInput("\x1b[F"); await settle();
|
|
44
|
+
assert.deepEqual(actions, [], "opening and rendering never play");
|
|
45
|
+
assert.match(panel.render(120).join("\n"), /Paulina.*180/);
|
|
46
|
+
panel.handleInput("\x1b[C"); panel.handleInput("\r"); await settle();
|
|
47
|
+
assert.deepEqual(actions, ["on"]); assert.match(panel.render(120).join("\n"), /Disable/);
|
|
48
|
+
panel.handleInput("\r"); await settle(); assert.deepEqual(actions, ["on", "off"]);
|
|
49
|
+
panel.handleInput("\x1b[B"); panel.handleInput("\r"); await settle();
|
|
50
|
+
assert.match(panel.render(120).join("\n"), /No successful final response/);
|
|
51
|
+
panel.handleInput("\x1b[B"); panel.handleInput("\r"); await settle();
|
|
52
|
+
assert.deepEqual(actions, ["on", "off", "last", "stop"]);
|
|
53
|
+
panel.dispose();
|
|
54
|
+
});
|
|
55
|
+
void test("Speech explains native and optional Kokoro previews without adding controls", async () => {
|
|
56
|
+
const previews: string[] = [];
|
|
57
|
+
const speech = createControlCenterSpeech({
|
|
58
|
+
settings: () => Promise.resolve({ version: 1, enabled: false, voice: "Paulina", rate: 180 }),
|
|
59
|
+
voices: () => Promise.resolve(["Paulina", "Kokoro Dora"]),
|
|
60
|
+
selectVoice: () => { throw new Error("must not save while browsing or previewing"); },
|
|
61
|
+
previewVoice: voice => { previews.push(voice); return Promise.resolve({ failed: false, message: "Preview started" }); },
|
|
62
|
+
stopPreview: () => {}, command: () => { throw new Error("must not enable narration"); },
|
|
63
|
+
}, {} as ExtensionContext);
|
|
64
|
+
const initial = await speech.read();
|
|
65
|
+
assert.match(initial.note, /installed macOS voices/);
|
|
66
|
+
assert.match(initial.note, /optional Kokoro.*local server/i);
|
|
67
|
+
assert.equal(initial.rows.length, 5);
|
|
68
|
+
const draft = speech.moveVoice(1)!;
|
|
69
|
+
assert.match(draft.summary, /disabled.*Paulina.*180 wpm/);
|
|
70
|
+
assert.deepEqual(draft.rows[3]?.action, { kind: "speech-voice", voice: "Kokoro Dora" });
|
|
71
|
+
assert.deepEqual(draft.rows[4]?.action, { kind: "speech-preview", voice: "Kokoro Dora" });
|
|
72
|
+
assert.match(draft.rows[4]?.details?.join(" ") ?? "", /Spanish.*Kokoro.*English.*native/);
|
|
73
|
+
await speech.apply({ kind: "speech-preview", voice: "Kokoro Dora" });
|
|
74
|
+
assert.deepEqual(previews, ["Kokoro Dora"]);
|
|
75
|
+
assert.match((await speech.read()).rows[3]!.label, /Kokoro Dora.*unsaved/);
|
|
76
|
+
});
|
|
77
|
+
void test("Stale Control Center speech actions cannot launch in a replaced session", async () => {
|
|
78
|
+
let calls = 0;
|
|
79
|
+
const speech = createControlCenterSpeech({
|
|
80
|
+
settings: () => Promise.resolve({ version: 1, enabled: false, voice: "Paulina", rate: 180 }),
|
|
81
|
+
voices: () => Promise.resolve([]),
|
|
82
|
+
previewVoice: () => { calls++; return Promise.resolve({ failed: false, message: "Preview started" }); }, stopPreview: () => {},
|
|
83
|
+
selectVoice: () => { calls++; return Promise.resolve({ failed: false, message: "saved" }); },
|
|
84
|
+
command: () => { calls++; return Promise.resolve({ failed: false, message: "started" }); },
|
|
85
|
+
}, {} as ExtensionContext, () => false);
|
|
86
|
+
for (const action of [{ kind: "speech-last" }, { kind: "speech-preview", voice: "Paulina" }, { kind: "speech-voice", voice: "Paulina" }] as const) {
|
|
87
|
+
const result = await speech.apply(action);
|
|
88
|
+
assert.equal(result.failed, true); assert.match(result.message, /session changed/);
|
|
89
|
+
}
|
|
90
|
+
assert.equal(calls, 0);
|
|
91
|
+
});
|
|
92
|
+
void test("One horizontal Voice selector wraps silently and Enter alone saves among many voices", async () => {
|
|
93
|
+
let settings: SpeechSettings = { version: 1, enabled: true, voice: "Voice 050", rate: 180 };
|
|
94
|
+
const names = Array.from({ length: 120 }, (_, i) => `Voice ${String(i).padStart(3, "0")}`);
|
|
95
|
+
const selected: string[] = [];
|
|
96
|
+
const playback: string[] = [];
|
|
97
|
+
const speech = createControlCenterSpeech({
|
|
98
|
+
settings: () => Promise.resolve(settings), voices: () => Promise.resolve(names),
|
|
99
|
+
previewVoice: voice => { playback.push(voice); return Promise.resolve({ failed: false, message: "Preview started" }); }, stopPreview: () => {},
|
|
100
|
+
selectVoice: (voice: string) => {
|
|
101
|
+
selected.push(voice); settings = { ...settings, voice };
|
|
102
|
+
return Promise.resolve({ failed: false, message: `Voice saved globally: ${voice}` });
|
|
103
|
+
},
|
|
104
|
+
command: args => { playback.push(args.join(" ")); return Promise.resolve({ failed: false, message: "Speech applied" }); },
|
|
105
|
+
}, {} as ExtensionContext);
|
|
106
|
+
const view = await speech.read();
|
|
107
|
+
assert.equal(view.rows.length, 5);
|
|
108
|
+
assert.match(view.rows[3]!.label, /Voice: ‹ Voice 050 ›/);
|
|
109
|
+
const panel = new ControlCenter({ speech,
|
|
110
|
+
theme: () => ({ name: "dark", appearance: "dark", fg: (_color, text) => text }),
|
|
111
|
+
readThemes: () => [], applyTheme: () => ({ success: true }), requestRender: () => {}, height: () => 18, close: () => {},
|
|
112
|
+
});
|
|
113
|
+
panel.handleInput("\x1b[F"); await settle(); panel.handleInput("\x1b[C");
|
|
114
|
+
assert.match(panel.render(100).join("\n"), /> Disable/);
|
|
115
|
+
for (let i = 0; i < 3; i++) panel.handleInput("\x1b[B");
|
|
116
|
+
for (let i = 0; i < 70; i++) panel.handleInput("\x1b[C");
|
|
117
|
+
assert.match(panel.render(100).join("\n"), /> Voice: ‹ Voice 000 ›.*unsaved/);
|
|
118
|
+
panel.handleInput("\x1b[D");
|
|
119
|
+
assert.match(panel.render(100).join("\n"), /> Voice: ‹ Voice 119 ›.*unsaved/);
|
|
120
|
+
assert.deepEqual(selected, []); assert.deepEqual(playback, []);
|
|
121
|
+
panel.handleInput("\r"); await settle();
|
|
122
|
+
assert.deepEqual(selected, ["Voice 119"]); assert.deepEqual(playback, []);
|
|
123
|
+
assert.match(panel.render(100).join("\n"), /Voice saved globally: Voice 119/);
|
|
124
|
+
assert.match((await speech.read()).rows[3]!.label, /Voice: ‹ Voice 119 ›.*saved/);
|
|
125
|
+
panel.handleInput("\x1b[H"); panel.handleInput("\x1b[B"); panel.handleInput("\r"); await settle();
|
|
126
|
+
assert.deepEqual(playback, ["last"]);
|
|
127
|
+
panel.dispose();
|
|
128
|
+
});
|
|
129
|
+
|
|
130
|
+
void test("Default voice aliases mark one installed entry; failed discovery keeps narration controls", async () => {
|
|
131
|
+
const owner = {
|
|
132
|
+
settings: () => Promise.resolve<SpeechSettings>({ version: 1, enabled: false, voice: "Paulina", rate: 180 }),
|
|
133
|
+
voices: () => Promise.resolve<readonly string[]>(["Paulina (Enhanced)", "Paulina (Español (México))"]),
|
|
134
|
+
previewVoice: () => Promise.resolve({ failed: false, message: "Preview started" }), stopPreview: () => {},
|
|
135
|
+
selectVoice: () => Promise.resolve({ failed: false, message: "saved" }),
|
|
136
|
+
command: () => Promise.resolve({ failed: false, message: "applied" }),
|
|
137
|
+
};
|
|
138
|
+
const speech = createControlCenterSpeech(owner, {} as ExtensionContext);
|
|
139
|
+
const initial = await speech.read();
|
|
140
|
+
assert.equal(initial.rows.filter(row => row.current).length, 1);
|
|
141
|
+
assert.deepEqual(initial.rows[3]?.action, { kind: "speech-voice", voice: "Paulina (Enhanced)" });
|
|
142
|
+
owner.voices = () => Promise.reject(new Error("voice catalog unavailable"));
|
|
143
|
+
const unavailable = await speech.read();
|
|
144
|
+
assert.deepEqual(unavailable.rows.map(row => row.label), ["Enable", "Read last response", "Stop"]);
|
|
145
|
+
assert.match(unavailable.note, /voice catalog unavailable/);
|
|
146
|
+
});
|
|
147
|
+
void test("Explicit preview uses the unsaved draft, preserves it through failed saves and cleans up only previews", async () => {
|
|
148
|
+
const previews: string[] = [];
|
|
149
|
+
const saves: string[] = [];
|
|
150
|
+
const commands: string[] = [];
|
|
151
|
+
let stops = 0;
|
|
152
|
+
let finishPreview: ((result: { failed: boolean; message: string }) => void) | undefined;
|
|
153
|
+
const speech = createControlCenterSpeech({
|
|
154
|
+
settings: () => Promise.resolve({ version: 1, enabled: false, voice: "Paulina", rate: 180 }),
|
|
155
|
+
voices: () => Promise.resolve(["Paulina", "Samantha"]),
|
|
156
|
+
selectVoice: voice => { saves.push(voice); return Promise.resolve({ failed: true, message: "Save refused" }); },
|
|
157
|
+
previewVoice: voice => { previews.push(voice); return new Promise(resolve => { finishPreview = resolve; }); },
|
|
158
|
+
stopPreview: () => { stops++; },
|
|
159
|
+
command: args => { commands.push(args.join(" ")); return Promise.resolve({ failed: false, message: "Applied" }); },
|
|
160
|
+
}, {} as ExtensionContext);
|
|
161
|
+
const panel = new ControlCenter({ speech,
|
|
162
|
+
theme: () => ({ name: "dark", appearance: "dark", fg: (_color, text) => text }),
|
|
163
|
+
readThemes: () => [], applyTheme: () => ({ success: true }), requestRender: () => {}, height: () => 30, close: () => {},
|
|
164
|
+
});
|
|
165
|
+
panel.handleInput("\x1b[F"); await settle(); panel.handleInput("\x1b[C");
|
|
166
|
+
for (let i = 0; i < 3; i++) panel.handleInput("\x1b[B");
|
|
167
|
+
panel.handleInput("\x1b[C");
|
|
168
|
+
assert.deepEqual(saves, []); assert.deepEqual(previews, []);
|
|
169
|
+
panel.handleInput("\x1b[B"); panel.handleInput("\r");
|
|
170
|
+
assert.deepEqual(previews, ["Samantha"]);
|
|
171
|
+
assert.doesNotMatch(panel.render(100).join("\n"), /Saving globally/);
|
|
172
|
+
finishPreview?.({ failed: false, message: "Preview started" }); await settle();
|
|
173
|
+
assert.match(panel.render(100).join("\n"), /Voice: ‹ Samantha ›.*unsaved/);
|
|
174
|
+
panel.handleInput("\x1b[A"); panel.handleInput("\r"); await settle();
|
|
175
|
+
assert.deepEqual(saves, ["Samantha"]);
|
|
176
|
+
assert.match(panel.render(100).join("\n"), /Voice: ‹ Samantha ›.*unsaved/);
|
|
177
|
+
panel.handleInput("\x1b[H"); panel.handleInput("\x1b[B"); panel.handleInput("\r"); await settle();
|
|
178
|
+
assert.deepEqual(commands, ["last"]);
|
|
179
|
+
assert.match(panel.render(100).join("\n"), /Voice: ‹ Samantha ›.*unsaved/);
|
|
180
|
+
panel.handleInput("\t"); panel.handleInput("\x1b[A"); await settle();
|
|
181
|
+
assert.equal(stops, 1, "leaving Speech stops only preview");
|
|
182
|
+
panel.dispose(); panel.dispose();
|
|
183
|
+
assert.equal(stops, 2, "disposal is idempotent");
|
|
184
|
+
assert.deepEqual(commands, ["last"], "cleanup never issues narration Stop");
|
|
185
|
+
panel.handleInput("\r"); assert.deepEqual(previews, ["Samantha"]);
|
|
186
|
+
});
|
|
187
|
+
void test("Empty and single catalogs retain controls; only the ready selector owns arrows and narrow rendering fits", async () => {
|
|
188
|
+
for (const names of [[], ["Paulina"]]) {
|
|
189
|
+
const actions: string[] = [];
|
|
190
|
+
let resolveVoices: ((voices: readonly string[]) => void) | undefined;
|
|
191
|
+
let loading = true;
|
|
192
|
+
const speech = createControlCenterSpeech({
|
|
193
|
+
settings: () => Promise.resolve({ version: 1, enabled: false, voice: "Paulina", rate: 180 }),
|
|
194
|
+
voices: () => loading ? new Promise(resolve => { resolveVoices = resolve; }) : Promise.resolve(names),
|
|
195
|
+
selectVoice: voice => { actions.push(`save:${voice}`); return Promise.resolve({ failed: false, message: "Saved" }); },
|
|
196
|
+
previewVoice: voice => { actions.push(`preview:${voice}`); return Promise.resolve({ failed: false, message: "Preview" }); },
|
|
197
|
+
stopPreview: () => {}, command: args => { actions.push(args.join(" ")); return Promise.resolve({ failed: false, message: "Applied" }); },
|
|
198
|
+
}, {} as ExtensionContext);
|
|
199
|
+
const panel = new ControlCenter({ speech,
|
|
200
|
+
theme: () => ({ name: "dark", appearance: "dark", fg: (_color, text) => text }),
|
|
201
|
+
readThemes: () => [], applyTheme: () => ({ success: true }), requestRender: () => {}, height: () => 18, close: () => {},
|
|
202
|
+
});
|
|
203
|
+
panel.handleInput("\x1b[F"); await settle();
|
|
204
|
+
panel.handleInput("\x1b[C"); panel.handleInput("\r"); assert.deepEqual(actions, [], "loading blocks activation");
|
|
205
|
+
panel.handleInput("\x1b[D");
|
|
206
|
+
assert.match(panel.render(100).join("\n"), /> Speech/, "loading arrows still change pane focus");
|
|
207
|
+
loading = false; resolveVoices?.(names); await settle();
|
|
208
|
+
panel.handleInput("\x1b[C");
|
|
209
|
+
assert.match(panel.render(100).join("\n"), /> Enable/);
|
|
210
|
+
panel.handleInput("\x1b[D");
|
|
211
|
+
assert.match(panel.render(100).join("\n"), /> Speech/, "controls do not capture arrows");
|
|
212
|
+
panel.handleInput("\x1b[C"); panel.handleInput("\x1b[F");
|
|
213
|
+
if (names.length) {
|
|
214
|
+
panel.handleInput("\x1b[A");
|
|
215
|
+
for (const key of ["\x1b[D", "\x1b[C"]) {
|
|
216
|
+
panel.handleInput(key);
|
|
217
|
+
assert.match(panel.render(100).join("\n"), /> Voice: ‹ Paulina ›.*saved/);
|
|
218
|
+
}
|
|
219
|
+
for (const width of [1, 12, 29, 57, 58, 100]) {
|
|
220
|
+
const lines = panel.render(width);
|
|
221
|
+
assert.ok(lines.length <= 18);
|
|
222
|
+
assert.ok(lines.every(line => visibleWidth(line) <= width));
|
|
223
|
+
}
|
|
224
|
+
panel.handleInput("\t"); assert.match(panel.render(100).join("\n"), /> Speech/);
|
|
225
|
+
} else {
|
|
226
|
+
assert.equal((await speech.read()).rows.length, 3);
|
|
227
|
+
assert.match(panel.render(100).join("\n"), /> Stop/);
|
|
228
|
+
}
|
|
229
|
+
assert.deepEqual(actions, []); panel.dispose();
|
|
230
|
+
}
|
|
231
|
+
});
|
|
232
|
+
|
|
233
|
+
void test("Closing a pending preview cancels its sample only and ignores all late completion and input", async () => {
|
|
234
|
+
let finish: ((result: { failed: boolean; message: string }) => void) | undefined;
|
|
235
|
+
let stops = 0;
|
|
236
|
+
let previews = 0;
|
|
237
|
+
let commands = 0;
|
|
238
|
+
let closed = 0;
|
|
239
|
+
const speech = createControlCenterSpeech({
|
|
240
|
+
settings: () => Promise.resolve({ version: 1, enabled: false, voice: "Paulina", rate: 180 }),
|
|
241
|
+
voices: () => Promise.resolve(["Paulina", "Samantha"]),
|
|
242
|
+
selectVoice: () => Promise.resolve({ failed: false, message: "Saved" }),
|
|
243
|
+
previewVoice: () => { previews++; return new Promise(resolve => { finish = resolve; }); },
|
|
244
|
+
stopPreview: () => { stops++; },
|
|
245
|
+
command: () => { commands++; return Promise.resolve({ failed: false, message: "Applied" }); },
|
|
246
|
+
}, {} as ExtensionContext);
|
|
247
|
+
const panel = new ControlCenter({ speech,
|
|
248
|
+
theme: () => ({ name: "dark", appearance: "dark", fg: (_color, text) => text }),
|
|
249
|
+
readThemes: () => [], applyTheme: () => ({ success: true }), requestRender: () => {}, height: () => 30, close: () => { closed++; },
|
|
250
|
+
});
|
|
251
|
+
panel.handleInput("\x1b[F"); await settle(); panel.handleInput("\x1b[C"); panel.handleInput("\x1b[F");
|
|
252
|
+
panel.handleInput("\r"); panel.handleInput("\r"); assert.equal(previews, 1, "busy blocks duplicates");
|
|
253
|
+
panel.handleInput("\x1b[A"); panel.handleInput("\x1b[C");
|
|
254
|
+
assert.match(panel.render(100).join("\n"), /Voice: ‹ Paulina ›.*saved/, "busy cannot browse draft");
|
|
255
|
+
panel.handleInput("\x1b"); panel.dispose();
|
|
256
|
+
assert.equal(stops, 1); assert.equal(closed, 1); assert.equal(commands, 0);
|
|
257
|
+
finish?.({ failed: false, message: "Late preview" }); await settle();
|
|
258
|
+
panel.handleInput("\r"); assert.equal(previews, 1); assert.deepEqual(panel.render(100), []);
|
|
259
|
+
});
|
|
260
|
+
|
|
261
|
+
void test("Clean drafts follow saved aliases but dirty drafts survive external changes and catalog errors", async () => {
|
|
262
|
+
let voice = "Paulina";
|
|
263
|
+
let unavailable = false;
|
|
264
|
+
const speech = createControlCenterSpeech({
|
|
265
|
+
settings: () => Promise.resolve({ version: 1, enabled: false, voice, rate: 180 }),
|
|
266
|
+
voices: () => unavailable ? Promise.reject(new Error("catalog failed")) : Promise.resolve(["Paulina (Enhanced)", "Samantha", "Victoria"]),
|
|
267
|
+
selectVoice: () => Promise.resolve({ failed: true, message: "Save failed" }),
|
|
268
|
+
previewVoice: () => Promise.resolve({ failed: false, message: "Preview" }), stopPreview: () => {},
|
|
269
|
+
command: () => Promise.resolve({ failed: false, message: "Applied" }),
|
|
270
|
+
}, {} as ExtensionContext);
|
|
271
|
+
assert.match((await speech.read()).rows[3]!.label, /Paulina \(Enhanced\).*saved/);
|
|
272
|
+
voice = "Samantha";
|
|
273
|
+
assert.match((await speech.read()).rows[3]!.label, /Samantha.*saved/);
|
|
274
|
+
assert.match(speech.moveVoice(1)!.rows[3]!.label, /Victoria.*unsaved/);
|
|
275
|
+
voice = "Paulina";
|
|
276
|
+
assert.match((await speech.read()).rows[3]!.label, /Victoria.*unsaved/);
|
|
277
|
+
unavailable = true; assert.equal((await speech.read()).rows.length, 3);
|
|
278
|
+
unavailable = false; assert.match((await speech.read()).rows[3]!.label, /Victoria.*unsaved/);
|
|
279
|
+
});
|
|
280
|
+
const settle = async () => { for (let i = 0; i < 12; i++) await Promise.resolve(); };
|
|
@@ -0,0 +1,82 @@
|
|
|
1
|
+
import type { ExtensionContext } from "@earendil-works/pi-coding-agent";
|
|
2
|
+
import type { ControlCenterDetail, ControlCenterService } from "./control-center.js";
|
|
3
|
+
import type { ResponseSpeech } from "./response-speech.js";
|
|
4
|
+
import type { SpeechSettings } from "./speech-types.js";
|
|
5
|
+
|
|
6
|
+
export type ControlCenterSpeechAction =
|
|
7
|
+
| { kind: "speech-enabled"; value: boolean }
|
|
8
|
+
| { kind: "speech-last" }
|
|
9
|
+
| { kind: "speech-stop" }
|
|
10
|
+
| { kind: "speech-voice"; voice: string }
|
|
11
|
+
| { kind: "speech-preview"; voice: string };
|
|
12
|
+
|
|
13
|
+
export interface ControlCenterSpeechService extends ControlCenterService<ControlCenterSpeechAction> {
|
|
14
|
+
moveVoice(direction: -1 | 1): ControlCenterDetail | undefined;
|
|
15
|
+
stopPreview(): void;
|
|
16
|
+
}
|
|
17
|
+
|
|
18
|
+
/** Presentation projection only; the lifecycle owner performs every speech action. */
|
|
19
|
+
export function createControlCenterSpeech(speech: ResponseSpeech, ctx: ExtensionContext,
|
|
20
|
+
isCurrent: () => boolean = () => true): ControlCenterSpeechService {
|
|
21
|
+
let settings: SpeechSettings | undefined;
|
|
22
|
+
let voices: readonly string[] = [];
|
|
23
|
+
let savedVoice: string | undefined;
|
|
24
|
+
let draft: string | undefined;
|
|
25
|
+
let voiceNote = "No voices available.";
|
|
26
|
+
const project = (): ControlCenterDetail => {
|
|
27
|
+
if (!settings) return { summary: "Speech unavailable", note: voiceNote, rows: [] };
|
|
28
|
+
return {
|
|
29
|
+
summary: `Speech: ${settings.enabled ? "enabled" : "disabled"} · Saved voice: ${settings.voice} · Rate: ${settings.rate} wpm`,
|
|
30
|
+
note: `Local speech: installed macOS voices or optional Kokoro (requires a separately started local server). Independent of Sounds mute. Read last response uses the saved voice while disabled. ${voiceNote}`,
|
|
31
|
+
rows: [
|
|
32
|
+
{ label: settings.enabled ? "Disable" : "Enable", current: false, action: { kind: "speech-enabled", value: !settings.enabled } },
|
|
33
|
+
{ label: "Read last response", current: false, action: { kind: "speech-last" } },
|
|
34
|
+
{ label: "Stop", current: false, action: { kind: "speech-stop" } },
|
|
35
|
+
...(!voices.length || draft === undefined ? [] : [
|
|
36
|
+
{ label: `Voice: ‹ ${draft} › (${draft === savedVoice ? "saved" : "unsaved"})`, current: draft === savedVoice,
|
|
37
|
+
action: { kind: "speech-voice" as const, voice: draft }, speechSelector: true,
|
|
38
|
+
details: ["←/→ browse draft (wraps); Enter saves. Browsing and saving never play audio.", `Saved voice: ${settings.voice}. Preview voice explicitly plays the draft without saving.`] },
|
|
39
|
+
{ label: "Preview voice", current: false, action: { kind: "speech-preview" as const, voice: draft },
|
|
40
|
+
details: ["Play a fixed sample: Spanish for Kokoro, English for native voices; does not save or enable narration."] },
|
|
41
|
+
]),
|
|
42
|
+
],
|
|
43
|
+
};
|
|
44
|
+
};
|
|
45
|
+
return {
|
|
46
|
+
async read() {
|
|
47
|
+
const nextSettings = await speech.settings();
|
|
48
|
+
let nextVoices: readonly string[] = [];
|
|
49
|
+
voiceNote = "No voices available.";
|
|
50
|
+
try {
|
|
51
|
+
nextVoices = await speech.voices(ctx);
|
|
52
|
+
if (nextVoices.length) voiceNote = "←/→ on Voice browses; Enter saves; Preview voice plays an explicit sample. Tab returns to categories.";
|
|
53
|
+
} catch (error) {
|
|
54
|
+
voiceNote = `Voices unavailable: ${error instanceof Error ? error.message : "Unknown error"}`;
|
|
55
|
+
}
|
|
56
|
+
const nextSaved = nextVoices.find(voice => voice === nextSettings.voice) ??
|
|
57
|
+
nextVoices.find(voice => voice.startsWith(`${nextSettings.voice} (`) && voice.endsWith(")"));
|
|
58
|
+
// Follow external saved changes only while the draft was clean. Never discard an unsaved choice on refresh.
|
|
59
|
+
if (nextVoices.length) {
|
|
60
|
+
if (draft === undefined || draft === savedVoice || !nextVoices.includes(draft)) draft = nextSaved ?? nextVoices[0];
|
|
61
|
+
savedVoice = nextSaved;
|
|
62
|
+
}
|
|
63
|
+
settings = nextSettings;
|
|
64
|
+
voices = nextVoices;
|
|
65
|
+
return project();
|
|
66
|
+
},
|
|
67
|
+
moveVoice(direction) {
|
|
68
|
+
if (!isCurrent() || !voices.length || draft === undefined) return undefined;
|
|
69
|
+
const index = voices.indexOf(draft);
|
|
70
|
+
draft = voices[(index + direction + voices.length) % voices.length];
|
|
71
|
+
return project();
|
|
72
|
+
},
|
|
73
|
+
stopPreview() { speech.stopPreview(ctx); },
|
|
74
|
+
apply(action) {
|
|
75
|
+
if (!isCurrent()) return Promise.resolve({ failed: true, message: "Speech session changed; reopen Control Center." });
|
|
76
|
+
if (action.kind === "speech-preview") return speech.previewVoice(action.voice, ctx);
|
|
77
|
+
if (action.kind === "speech-voice") return speech.selectVoice(action.voice, ctx);
|
|
78
|
+
if (action.kind === "speech-enabled") return speech.command([action.value ? "on" : "off"], ctx);
|
|
79
|
+
return speech.command([action.kind === "speech-last" ? "last" : "stop"], ctx);
|
|
80
|
+
},
|
|
81
|
+
};
|
|
82
|
+
}
|