dsh-live-voice 0.0.1-alpha.1

package/DEVELOPMENT.md

@@ -0,0 +1,54 @@
+# Development and local testing
+
+```sh
+npm ci
+npm run typecheck
+npm test
+npm run build
+```
+
+Implementation, test, and build-script source is TypeScript (`.ts`). `npm run typecheck` compiles the TypeScript project; `npm test` typechecks, builds the browser and host bundles, transpiles the TypeScript tests, then runs them. Tests exercise engines, coordination, transcript edits, RPC, configuration, and UI contracts, but do not record the microphone. `npm run dev` watches the browser and host bundles only; it is not a replacement DSH server.
+
+## Current local installation
+
+Install through the official CLI only:
+
+```sh
+dsh plugin add --profile web link:/Users/bizup/GitRepos/dsh-live-voice
+```
+
+The CLI registers the bundle, which loads this package’s own patch. Do not add a second `insert` for `dsh-live-voice` in the profile YAML: it causes a duplicate loader entry and prevents startup. The earlier manual activation workaround has been withdrawn. The plugin uses public `connection.fetch.register` routes below `/api/dsh-live-voice`, preserving the existing DSH authentication carrier. Only `connection` injection is needed. Adding `webServer` did not fix the custom-channel context error in the installed SDK; the custom channel was removed. Linked server source changes apply on the next normal DSH startup; a browser refresh alone does not reload server code.
+
+After rebuilding, refresh the existing authenticated DSH page. No automatic HMR guarantee is made. The existing URL remains http://127.0.0.1:3080. Unauthenticated HTTP requests return 401; never bypass this or extract credentials to test the plugin.
+
+## First use
+
+Open DSH Settings → Live Voice. Speech output, Speech recognition, and Conversation are separate collapsible cards; provider and Voice detection controls are nested collapsible sections. Preferences are stored in this browser; changing them stops active voice resources, saves immediately, and applies to the next operation. The Conversation card controls automatic announcement of new assistant messages during voice conversations; it is enabled by default, and playback waits while speech recognition reports user activity. Listening mode has a selection-specific explanation: Speakers/gated releases capture during playback, while Headphones/open microphone keeps capture available for interruption. Sending mode defaults to Manual. Automatic mode waits 4 seconds by default after a final recognized phrase (configurable from 2 to 10 seconds), displays a cancellable countdown in the voice bar, and uses the normal public DSH input submission action. New speech or an edited draft cancels the pending send. Separately, **Assistant response delay** requires continuous silence after detected user speech before queued automatic playback begins (default 3 seconds, configurable from 1 to 10); renewed speech restarts that wait. Manual per-message playback is not delayed. Sending another user message also leaves current assistant audio playing by default. Enable **Stop assistant speech when I send a message** to opt into interruption on a newly observed user or steering turn; loaded history does not trigger it. Choose browser speech (local voices only) or macOS say (plays through the host device). Choose Browser SpeechRecognition or Whisper HTTP, the recognition language, and speaker/headphone mode.
+
+- Microphone: dictate into the editable composer. Stop keeps the text; cancel removes only an unchanged interim hypothesis.
+- Conversation: continuous recognition plus spoken new assistant text. Review the composer and use the existing DSH Send control. There is no automatic send.
+- Speakers: recognition is gated during playback. Use Take microphone to interrupt.
+- Headphones: speech activity pauses playback. Resume is explicit, never triggered just by silence.
+- Global Stop speech cancels remaining speech. End conversation stops capture, recognition and playback, but not agent text generation.
+- Per-message speaker controls replay only the addressed visible assistant message.
+- Ctrl+Shift+Space works with the page focused when a single composer is active, not globally across the OS.
+
+## Privacy and compatibility
+
+Browser recognition can be configured for on-device processing or the browser recognition service. In local mode the user may allow the browser-native language-pack installation flow. The UI warns when browser-service processing may transmit microphone audio. The waveform analyser remains local.
+
+Whisper HTTP is host-side. In DSH Settings → Live Voice, choose **Whisper HTTP — DSH host** under Speech recognition. Provider-specific fields appear directly inside a single **Connection settings** subcard, without repeating the selected engine name or nesting an additional connection box. A nested **Silence detection** subcard appears inside Speech recognition only for engines whose audio is captured and segmented by this plugin. It exposes three simple pause profiles: Short (900 ms), Natural (1500 ms, default), and Long (2200 ms). Browser SpeechRecognition does not show this block because the browser controls its segmentation. Recognition language is sent to the active provider: Whisper HTTP adds **Automatic — detect language** and sends `auto` to whisper.cpp; Browser SpeechRecognition requires a concrete language and does not show Automatic. The provider panel exposes the inference endpoint URL, health URL/path, and request timeout, plus **Test connection**, **Save Whisper settings**, and **Reload saved settings**. Saved settings live only on the DSH host in `~/.dsh/dsh-live-voice-whisper.json` (owner-only permissions), never in browser storage; only unauthenticated loopback HTTP URLs are accepted. The default inference endpoint is `http://127.0.0.1:8080/inference`. The legacy `DSH_LIVE_VOICE_WHISPER_URL` remains the initial default before host settings are saved. The browser reuses the microphone stream, applies the selected pause profile (or a 20-second maximum), resamples to mono 16 kHz PCM16 WAV, and posts bounded utterances to DSH’s authenticated `/api/dsh-live-voice/whisper/transcribe` route. DSH validates the WAV and forwards it as multipart `file` to whisper.cpp. Current whisper.cpp HTTP accepts only complete files and returns one final transcript; it has no WebSocket, SSE, or true realtime input protocol, so the plugin does not falsely label this engine as streaming.
+
+macOS say uses a private temporary text file and remains active until the child closes and cleanup finishes. The host RPC carries no persistent transcript log. Browser output accepts only voices explicitly marked local. Browser and host may be different devices.
+
+## Observed browser capability
+
+An isolated headless instance of the installed Chrome 152 reported `processLocally` and the availability API, but `pt-BR` and `en-US` were `downloadable`, not `available`. No download or recording was performed. This isolated profile does not establish whether the maintainer’s normal browser profile already has those packs. Run `node --experimental-strip-types scripts/probe-browser.ts` to repeat this capability-only check using the installed macOS Chrome.
+
+`node --experimental-strip-types scripts/preview-ui.ts` produces an isolated component screenshot with a synthetic meter level, not a screenshot of the authenticated DSH GUI.
+
+## Manual acceptance still required
+
+In the authenticated DSH page, verify controls appear once, settings open/close, say play/pause/resume/stop, browser voice discovery, microphone permissions, local recognition availability, waveform, typing during dictation, cancellation, speaker/headphone interruption, streaming, session switching and narrow-screen layout. The maintainer removes older voice plugins.
+
+Automated tests use fake speech resources and never invoke audible system playback. Actual browser microphone recognition, host speech output, and speaker audibility require an explicit manual check and must not be claimed verified by automated tests.

package/HISTORY.md

@@ -0,0 +1,19 @@
+# The story behind DSH Live Voice
+
+## 2026-09-14 — Why I am starting this
+
+I want talking to DSH to feel more like talking to another person. Not perfectly human, and not magic—just a conversation where I can speak, listen, interrupt, and be heard without having to manage the tools all the time.
+
+For a long time, existing voice plugins solved what I needed. I used them, forked them, and made my own versions with changes that mattered to me. That worked for a while. But eventually, I kept coming back to the same problem: the part that listened and the part that spoke did not know enough about each other.
+
+I use speakers. Sometimes, while the assistant was reading an answer aloud, I would start recording something and my microphone would pick up its voice. What I was trying to say got mixed up with what it was saying. Other times, I wanted to speak, but the assistant was still talking and I had no easy way to make it stop and listen. The tools were useful, but using them together could get in the way of the conversation.
+
+That is the point where I started wanting one thing to look after both sides. If I am speaking, it should know to give me room. If it is speaking, I should be able to take my turn without fighting the playback. With speakers, I imagine pressing a shortcut to interrupt and start talking. With headphones, where the microphone is not picking up the assistant in the same way, I would like to just begin speaking and have it pause.
+
+And I really do mean pause, not always throw the answer away. Sometimes I only need a moment, and I want it to continue from where it stopped. Other times, what I say changes the conversation. I want there to be room for both. I also do not want to have to wait for the whole answer to finish appearing before I can say something.
+
+Running this on my own machine matters to me. Both understanding my voice and speaking back should be possible locally, with outside providers available as an option rather than a requirement. But I do not want this to be something that only works for me, on my computer, after all my personal adjustments.
+
+I want other people to be able to use it too. Someone should be able to see what works on their machine, get started without a long list of mandatory settings, and easily change the voice or recognition option when they want to. The focus should be on the person trying to have a conversation, not on making that person understand how every piece works.
+
+So I am starting DSH Live Voice from scratch. Not because the tools I used were not valuable—they helped me for a long time—but because I have reached a point where I need listening and speaking to be part of the same experience. For now, that is the beginning of the story: a problem I keep running into, and something I want to make better for myself and for other people.