dsh-live-voice 0.0.1-developing → 0.1.0

package/README.md

@@ -1,119 +1,111 @@
 # DSH Live Voice
 
-**Local-first voice conversations for DSH.**
+[![npm version](https://img.shields.io/npm/v/dsh-live-voice?logo=npm&label=npm&color=brightgreen)](https://www.npmjs.com/package/dsh-live-voice)
+[![dsh.pub registry status](https://dsh.pub/api/badges/victorwads/dsh-live-voice.svg)](https://dsh.pub/en/plugins/dsh-live-voice/)
+[![license](https://img.shields.io/badge/license-GPL--3.0--only-blue)](LICENSE)
 
-Designed to run speech recognition (speech-to-text, STT) and speech synthesis (text-to-speech, TTS) on your own machine, with optional external providers. The goal is a single plugin that coordinates listening and speaking, rather than two independent voice tools competing for the microphone and speakers.
+**Local-first speech recognition, voice output, and continuous voice conversations for DSH.**
 
-## Project status
+DSH Live Voice coordinates the microphone, composer, assistant messages, and speech output in one plugin — without letting listening and speaking compete with each other.
 
-**Early development — no working DSH plugin is available yet.**
+## Install
 
-Version `0.0.1-developing` is a documentation-only development placeholder. Everything described below as planned behavior is a product goal, not an implemented feature. There are no runtime dependencies, build pipeline, or executable entry points yet.
+Install the public repository through dsh.pub into your DSH web profile:
 
-## Why this project exists
-
-Separate STT and TTS integrations can interfere with one another: the microphone may transcribe the assistant’s own speaker output, and the user may have no convenient way to interrupt a long spoken answer.
-
-DSH Live Voice aims to own the conversational state of both directions. The experience should approach a natural conversation: listen, speak, yield, pause, and resume when appropriate. It does not promise human-level timing or perfect echo suppression.
-
-## Acknowledgments
-
-A heartfelt thank you to [GooDAnDReaDY](https://github.com/GooDAnDReaDY), creator of [dsh-voice](https://github.com/GooDAnDReaDY/dsh-voice), and [Alan2Z](https://github.com/Alan2Z), creator of [dsh-speak](https://github.com/Alan2Z/dsh-speak).
-
-Your projects solved my voice needs in DSH for a long time, and I am genuinely grateful for the work you shared. I forked both repositories, built my own versions, and adapted behaviors that mattered to my workflow. Those projects were useful in their own right and helped me understand what I needed next.
-
-Eventually, I reached a point where I needed one codebase to coordinate both listening and speaking. In my setup, the independent plugins could overlap or interfere with each other because they did not share a conversation coordinator. That is why I am starting DSH Live Voice from scratch—not because their work lacked value, but because my next step requires a unified approach.
-
-My goal is a user-focused voice experience: less mandatory configuration, sensible defaults, and a simple way to switch speech recognition and speech synthesis engines without having to understand the internals. These are goals for this new project, not claims about features already available.
-
-If you only need one side of the experience—voice input or spoken output—please explore and support their projects. They may be exactly what you need. Thank you both for making DSH more useful to me and for sharing your work with the community.
-
-## Local-first, provider-optional
-
-- Prioritize local speech recognition and local speech synthesis.
-- Keep external speech providers optional, with explicit configuration.
-- Explain which capabilities, models, dependencies, and permissions are required by each available option.
-- Make capture, recognition, and external audio transmission states visible.
-- Do not equate local voice processing with a fully offline assistant: the DSH language model may still use a remote service.
-
-Specific engines, supported operating systems, hardware requirements, and installation steps have not been selected or verified yet. Local execution is the product priority, not a claim that the current package runs everywhere.
-
-## Planned conversation experience
-
-### Speaker mode
-
-While the assistant is speaking, microphone audio should not enter speech recognition, to reduce feedback from speaker output. A user-triggered shortcut should stop or pause playback and let the user speak. Whether capture stays active while recognition is gated remains a technical decision; the UI must distinguish those states.
-
-### Headphone mode
-
-The microphone may remain available during assistant playback. Detecting user speech should pause the assistant. Voice activity detection can respond faster, while recognized words provide stronger evidence with additional latency. The exact interruption policy is not decided yet.
-
-Users should be able to choose the conversation mode explicitly. A detected output device alone is not reliable evidence of acoustic isolation.
-
-### Pause, resume, and cancel
-
-These are separate operations:
-
-- **Pause:** suspend playback while preserving the remaining content and playback position where supported.
-- **Resume:** continue preserved speech when it is still relevant.
-- **Cancel:** discard obsolete speech so it cannot play later.
-
-A brief interruption may mean “wait, then continue” or may introduce a new request. Silence alone must not force an old answer to resume. How users signal these intentions remains an open design question. Exact resume may require plugin-owned audio buffering and is not guaranteed by every TTS engine.
-
-### Speaking during response streaming
-
-The user should be able to speak while DSH is still streaming a response. Text generation, speech synthesis, audio playback, and microphone recognition must be coordinated but tracked separately. Pausing playback need not stop text generation. Sending a new request or stopping generation depends on the actual DSH integration APIs.
-
-A future speech queue should synthesize suitable text segments in order and prevent delayed audio from canceled answers from playing.
-
-## Environment and capabilities
-
-The planned interface should explain available options based on capabilities, not only an operating-system label. Relevant checks include microphone permission, audio devices, local engine availability, playback controls, and shortcut scope.
-
-The DSH host and the browser may run on different machines. Detection must distinguish the host environment from the machine with the microphone and speakers. Global shortcuts and browser-focused shortcuts must not be presented as equivalent.
+```sh
+npx dshpub add victorwads/dsh-live-voice --profile web
+```
 
-## Proposed architecture
+The installer resolves the public repository to an exact commit before adding the bundle. Open **DSH Settings → Live Voice** after the next normal DSH startup.
+
+## Why this project exists and Acknowledgments
+
+Listening and speaking should work together, so you can interrupt and be heard without the assistant’s voice getting in the way. [Read the story behind the project](HISTORY.md).
+
+A heartfelt thank you to [GooDAnDReaDY](https://github.com/GooDAnDReaDY) for [dsh-voice](https://github.com/GooDAnDReaDY/dsh-voice) and [Alan2Z](https://github.com/Alan2Z) for [dsh-speak](https://github.com/Alan2Z/dsh-speak). Your projects solved my voice needs in DSH for a while, and I am grateful for the work you shared. Eventually, I reached a point where I needed one codebase to coordinate both listening and speaking. [Read the full story](HISTORY.md).
+
+## Features
+
+| | Capability |
+|---|---|
+| 🎙️ | Voice typing directly into the DSH composer |
+| 💬 | Continuous voice conversations with automatic assistant speech |
+| 🧠 | Browser SpeechRecognition, local loopback whisper.cpp, or Qwen3-ASR on Apple MLX |
+| 🔊 | Browser speech synthesis, native macOS `say`, or Qwen3-TTS on Apple MLX |
+| ⏱️ | Manual or automatic sending after configurable silence |
+| 🫁 | Stable-silence delay prevents breathing pauses from starting assistant speech |
+| 🎧 | Open-microphone mode for headphones |
+| 🔒 | Gated microphone mode for speakers |
+| ✋ | Pause, resume, stop, and manual interruption controls |
+| 🔈 | Play individual assistant messages on demand |
+| 🏠 | Whisper audio reaches the local server only through the authenticated DSH host |
+
+## Conversation flow
+
+The assistant never starts automatic playback while you are speaking. If a response is already waiting — including another assistant message — it waits until you finish and the configured continuous-silence delay has passed.
+
+### Speakers — gated listening (default)
+
+Listening and playback take turns so the assistant does not hear its own voice.
+
+```mermaid
+sequenceDiagram
+    participant Interface
+    actor Você
+    actor Assistente
+
+    Note over Interface,Assistente: Aguardando você falar…
+    activate Você
+    Você->>Assistente: Começa a falar
+    Assistente-->>Você: Escuta enquanto você fala
+    Note over Interface,Assistente: Você parou de falar
+    opt Envio manual
+        Você->>Interface: Revisa a mensagem reconhecida
+        Interface-->>Você: Envia quando estiver pronto
+    end
+    opt Envio automático
+        Note over Você,Assistente: Envia após a contagem de silêncio
+    end
+    deactivate Você
+    Você->>Assistente: Entrega sua mensagem
+    Assistente-->>Você: Resposta pronta — aguarda silêncio contínuo
+    Assistente-->>Você: Para de escutar
+    activate Assistente
+    Assistente->>Você: Fala a resposta em voz alta
+    deactivate Assistente
+    Note over Interface,Assistente: Escutando novamente — aguardando você falar…
+```
 
-These are responsibility boundaries, not implemented modules or a fixed directory layout:
+### Headphones — open microphone
 
-| Area | Responsibility |
-| --- | --- |
-| Environment and capabilities | Detect available features and explain limitations. |
-| Conversation coordinator | Manage turns, interruptions, pause, resume, and cancellation. |
-| Voice input | Capture audio, detect speech, and integrate STT engines. |
-| Voice output | Integrate TTS engines and manage buffering, queues, and playback. |
-| DSH integration | Connect messages, response streaming, and session lifecycle. |
-| Interface and controls | Expose modes, devices, permissions, state, and shortcuts. |
+The microphone remains open during playback, allowing your voice to pause the assistant.
 
-Input, output, and generation need separate states because listening and speaking may overlap. One plugin may support multiple engines. Reusing existing DSH plugins versus integrating engines directly requires inspection of the actual extension APIs first.
 
-## Repository guide
+### Other conversation settings
 
-- [`package.json`](package.json): npm package metadata.
-- [`HISTORY.md`](HISTORY.md): English-language, append-only product history and decision record.
-- [`AGENTS.md`](AGENTS.md): instructions for AI agents working on this project.
-- [`LICENSE`](LICENSE): GNU GPL version 3 license.
+- **Sending mode:** review and send manually by default, or send automatically after a configurable silence countdown.
+- **Assistant response delay:** choose how long you must remain silent before automatic playback starts; speaking again restarts the wait.
+- **Automatic assistant speech:** turn automatic playback of new assistant messages on or off.
+- **Sent-message interruption:** sending another message does not stop current audio by default, but you can enable that behavior.
+- **Manual playback:** play any individual assistant message on demand without waiting for the automatic-playback delay.
 
-## Development and contributions
+## Local-first architecture
 
-Design discussion and contributions are welcome. Read the history and agent instructions before changing project direction. Keep proposals distinct from implemented capabilities, and document new decisions without rewriting previous history entries.
+- **Recognition:** Browser SpeechRecognition, loopback whisper.cpp HTTP, or Qwen3-ASR through a host-local Apple MLX server.
+- **Speech output:** browser/device audio, native macOS `say`, or host-local Qwen3-TTS with WAV playback in the browser.
+- **Whisper transport:** complete WAV utterances through authenticated same-origin DSH routes.
+- **Privacy:** raw audio and transcripts are not logged by default.
 
-There is no runnable development setup or automated test suite yet. To inspect the intended npm publication without publishing:
+Speech processing can run locally, but the DSH language model may still be remote.
 
-```sh
-npm publish --dry-run --tag developing
-```
+## Qwen3 HTTP engine
 
-A dry run does not reserve the npm name or verify publishing authorization. Actual publication requires an available name and an authorized npm account. Use an explicit `developing` tag for prereleases; do not publish, commit, or push without the maintainer’s authorization.
-
-## Keywords
-
-`dsh`, `local-first`, `local-voice`, `voice-conversation`, `speech-to-text`, `text-to-speech`, `speech-recognition`, `speech-synthesis`, `stt`, `tts`, `turn-taking`, `voice-interruption`
+When a compatible Qwen3 speech API is already running on the DSH host, choose **Qwen3 ASR — local MLX server** under Speech recognition and **Qwen3 TTS — local MLX server** under Speech output. Configure its base URL in **DSH Settings → Live Voice**. The Qwen server may use any HTTP or HTTPS base URL reachable from the DSH host. The plugin supports the OminiX-API contract and standard OpenAI-style speech endpoints at `GET /health`, `POST /v1/audio/transcriptions`, and `POST /v1/audio/speech`; it does not install, start, stop, or manage that external service or its model weights.
 
 ## License
 
-DSH Live Voice is licensed under the [GNU General Public License, version 3 only](LICENSE) (`GPL-3.0-only`).
+[GPL-3.0-only](LICENSE). Commercial use and redistribution are allowed subject to the GPL. Third-party speech engines and models may have separate licenses.
 
-Internal and commercial use are allowed, and copies may be sold. When distributing covered software, with or without modifications and whether free or paid, you must comply with the GPL, preserve recipients’ GPL rights, and provide the corresponding source as required by the license. Private modifications do not have to be published merely because they are used internally. Providing a network service without conveying copies does not, by itself, trigger GPL source-distribution obligations.
+## Keywords
 
-This is a summary; the license text governs. Third-party engines and model weights may have their own licenses.
+`dsh`, `dsh-plugin`, `deepseek-harness`, `local-first`, `local-voice`, `voice-assistant`, `voice-conversation`, `continuous-conversation`, `voice-dictation`, `speech-to-text`, `text-to-speech`, `speech-recognition`, `speech-synthesis`, `stt`, `tts`, `whisper`, `whisper-cpp`, `web-speech-api`, `macos-say`, `turn-taking`, `voice-interruption`, `silence-detection`

package/cordis.patch.yml

@@ -0,0 +1,3 @@
+- insert:
+    - id: dsh-live-voice
+      name: dsh-live-voice