dsh-live-voice 0.2.2 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +39 -1
- package/DEVELOPMENT.md +13 -0
- package/LICENSE +201 -674
- package/PLAN.md +28 -0
- package/README.md +65 -110
- package/docs/ARCHITECTURE.md +40 -0
- package/docs/CHOOSING-AN-ENGINE.md +120 -0
- package/docs/CONFIGURATION.md +149 -0
- package/docs/REVIEW.md +35 -0
- package/docs/VOICE-LIFECYCLE.md +59 -0
- package/lib/client.js +3652 -1576
- package/lib/server.js +641 -453
- package/package.json +17 -6
- package/scripts/build.ts +16 -4
- package/scripts/preview-ui.ts +1 -1
- package/src/{client/index.ts → app/client/apply.tsx} +152 -59
- package/src/app/client/i18n/DshLanguageBoundary.tsx +42 -0
- package/src/app/client/i18n/catalogs/base.ts +231 -0
- package/src/app/client/i18n/catalogs/en.ts +258 -0
- package/src/app/client/i18n/catalogs/es.ts +279 -0
- package/src/app/client/i18n/catalogs/fr.ts +281 -0
- package/src/app/client/i18n/catalogs/hi.ts +263 -0
- package/src/app/client/i18n/catalogs/index.ts +18 -0
- package/src/app/client/i18n/catalogs/pt-BR.ts +270 -0
- package/src/app/client/i18n/catalogs/zh.ts +248 -0
- package/src/app/client/i18n/index.ts +4 -0
- package/src/app/client/i18n/registerDshLocales.ts +43 -0
- package/src/app/client/i18n/runtime.tsx +58 -0
- package/src/app/client/index.ts +3 -0
- package/src/app/client/registerSlots.tsx +30 -0
- package/src/app/client/slotDefinitions.ts +32 -0
- package/src/app/server/apply.ts +479 -0
- package/src/app/server/index.ts +2 -0
- package/src/app/server/registerRoutes.ts +11 -0
- package/src/modules/conversation/components/ConversationControls.tsx +40 -0
- package/src/modules/conversation/components/ConversationStatusBar.tsx +155 -0
- package/src/modules/conversation/components/DeliveryModeButton.tsx +43 -0
- package/src/modules/conversation/components/MicrophoneButton.tsx +6 -0
- package/src/modules/conversation/components/PlaybackControls.tsx +41 -0
- package/src/modules/conversation/components/SpeakButton.tsx +30 -0
- package/src/modules/conversation/components/Waveform.tsx +68 -0
- package/src/modules/conversation/components/conversationStatus.ts +17 -0
- package/src/modules/conversation/components/createConversationComponents.tsx +505 -0
- package/src/modules/conversation/components/index.ts +8 -0
- package/src/modules/conversation/hooks/index.ts +2 -0
- package/src/modules/conversation/hooks/useConversationActions.ts +27 -0
- package/src/modules/conversation/hooks/useConversationController.ts +15 -0
- package/src/modules/conversation/index.ts +3 -0
- package/src/{client → modules/conversation/models}/chat.ts +2 -1
- package/src/modules/conversation/models/index.ts +1 -0
- package/src/{core → modules/core}/coordinator.ts +133 -7
- package/src/modules/core/index.ts +6 -0
- package/src/{engines/qwen-http-host.ts → modules/core/qwen/QwenHttpHost.ts} +2 -2
- package/src/modules/core/qwen/QwenSettings.tsx +148 -0
- package/src/{core → modules/core}/settings.ts +23 -0
- package/src/modules/recognition/components/RecognitionCapabilityStatus.tsx +19 -0
- package/src/{engines/recognition/qwen-http.ts → modules/recognition/engines/qwen/QwenRecognitionEngine.ts} +1 -1
- package/src/{engines/recognition/whisper-http.ts → modules/recognition/engines/whisper/WhisperRecognitionEngine.ts} +1 -1
- package/src/modules/recognition/engines/whisper/WhisperSettings.tsx +150 -0
- package/src/modules/recognition/index.ts +4 -0
- package/src/modules/recognition/qwen/QwenRecognitionSettings.tsx +4 -0
- package/src/modules/settings/components/LiveVoiceSettings.tsx +72 -0
- package/src/modules/settings/components/SettingsHeader.tsx +49 -0
- package/src/modules/settings/components/VersionBadges.tsx +39 -0
- package/src/modules/settings/components/createLiveVoiceSettings.tsx +8 -0
- package/src/modules/settings/components/index.ts +4 -0
- package/src/modules/settings/hooks/index.ts +3 -0
- package/src/modules/settings/hooks/useAudioDevices.ts +38 -0
- package/src/modules/settings/hooks/useLiveVoiceSettings.ts +4 -0
- package/src/modules/settings/hooks/useReleaseStatus.ts +17 -0
- package/src/modules/settings/index.ts +4 -0
- package/src/modules/settings/models/index.ts +1 -0
- package/src/modules/settings/models/settingsStorage.ts +14 -0
- package/src/modules/settings/sections/conversation/ConversationDelaySettings.tsx +24 -0
- package/src/modules/settings/sections/conversation/ConversationSettingsSection.tsx +23 -0
- package/src/modules/settings/sections/conversation/DeliverySettings.tsx +38 -0
- package/src/modules/settings/sections/conversation/HoldToTalkSettings.tsx +20 -0
- package/src/modules/settings/sections/conversation/VoiceModeSettings.tsx +31 -0
- package/src/modules/settings/sections/conversation/index.ts +5 -0
- package/src/modules/settings/sections/index.ts +3 -0
- package/src/modules/settings/sections/recognition/RecognitionEngineSettings.tsx +109 -0
- package/src/modules/settings/sections/recognition/RecognitionFilterSettings.tsx +33 -0
- package/src/modules/settings/sections/recognition/RecognitionSettingsSection.tsx +20 -0
- package/src/modules/settings/sections/recognition/RecognitionStatus.tsx +19 -0
- package/src/modules/settings/sections/recognition/SilenceDetectionSettings.tsx +58 -0
- package/src/modules/settings/sections/recognition/VoiceCommandSettings.tsx +41 -0
- package/src/modules/settings/sections/recognition/index.ts +5 -0
- package/src/modules/settings/sections/speak/OutputFilterSettings.tsx +39 -0
- package/src/modules/settings/sections/speak/PlaybackPolicySettings.tsx +26 -0
- package/src/modules/settings/sections/speak/SpeakSettingsSection.tsx +16 -0
- package/src/modules/settings/sections/speak/SpeechAdvancedSettings.tsx +81 -0
- package/src/modules/settings/sections/speak/SpeechEngineSettings.tsx +80 -0
- package/src/modules/settings/sections/speak/index.ts +4 -0
- package/src/modules/settings/services/releases.ts +117 -0
- package/src/modules/speak/engines/audio/HostAudioEngine.ts +167 -0
- package/src/modules/speak/engines/audio/M4aAacTranscoder.ts +58 -0
- package/src/modules/speak/engines/qwen/QwenSpeakingEngine.ts +14 -0
- package/src/{engines/speaking/say.ts → modules/speak/engines/say/SaySpeakingEngine.ts} +26 -4
- package/src/modules/speak/index.ts +5 -0
- package/src/modules/speak/qwen/QwenSpeakingSettings.tsx +4 -0
- package/src/modules/speak/services/speechQueue.ts +2 -0
- package/src/server.ts +9 -365
- package/src/shared/design-system/buttons/IconButton.tsx +33 -0
- package/src/shared/design-system/buttons/PillButton.tsx +11 -0
- package/src/shared/design-system/buttons/ToggleButton.tsx +6 -0
- package/src/shared/design-system/buttons/index.ts +3 -0
- package/src/shared/design-system/feedback/ErrorMessage.tsx +27 -0
- package/src/shared/design-system/feedback/StatusBadge.tsx +9 -0
- package/src/shared/design-system/feedback/StatusMessage.tsx +4 -0
- package/src/shared/design-system/feedback/index.ts +3 -0
- package/src/shared/design-system/forms/CheckboxField.tsx +20 -0
- package/src/shared/design-system/forms/NumberField.tsx +12 -0
- package/src/shared/design-system/forms/SelectField.tsx +20 -0
- package/src/shared/design-system/forms/TextAreaField.tsx +12 -0
- package/src/shared/design-system/forms/TextField.tsx +12 -0
- package/src/shared/design-system/forms/index.ts +5 -0
- package/src/shared/design-system/icons/Icon.tsx +20 -0
- package/src/shared/design-system/icons/icons.ts +16 -0
- package/src/shared/design-system/icons/index.ts +2 -0
- package/src/shared/design-system/index.ts +5 -0
- package/src/shared/design-system/layout/SettingsCard.tsx +9 -0
- package/src/shared/design-system/layout/SettingsSection.tsx +9 -0
- package/src/shared/design-system/layout/SettingsSubcard.tsx +14 -0
- package/src/shared/design-system/layout/SettingsTabs.tsx +54 -0
- package/src/shared/design-system/layout/index.ts +4 -0
- package/src/{client/styles.ts → styles/index.ts} +3 -1
- package/src/client/components.ts +0 -1080
- package/src/client/qwen-settings.ts +0 -144
- package/src/client/whisper-settings.ts +0 -147
- package/src/engines/speaking/qwen-http.ts +0 -124
- /package/src/{core → modules/core}/filters.ts +0 -0
- /package/src/{core → modules/core}/microphone.ts +0 -0
- /package/src/{core → modules/core}/ownership.ts +0 -0
- /package/src/{core → modules/core}/transcript.ts +0 -0
- /package/src/{engines/recognition/browser.ts → modules/recognition/engines/browser/BrowserRecognitionEngine.ts} +0 -0
- /package/src/{engines/recognition/whisper-http-host.ts → modules/recognition/engines/whisper/whisperRecognitionHost.ts} +0 -0
- /package/src/{engines/speaking/browser.ts → modules/speak/engines/browser/BrowserSpeakingEngine.ts} +0 -0
- /package/src/{engines/speaking/say-client.ts → modules/speak/engines/say/sayClient.ts} +0 -0
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
# Voice Lifecycle and Turn-Taking Contract
|
|
2
|
+
|
|
3
|
+
This is the natural-language behavioral contract for conversation lifecycle and turn management. Code and tests should be reviewed against these rules.
|
|
4
|
+
|
|
5
|
+
## State scopes
|
|
6
|
+
|
|
7
|
+
### Persisted preferences
|
|
8
|
+
|
|
9
|
+
Engine, language, input/output device, Speakers/Headphones mode, delivery mode, delays, filters, and customized voice-command phrases may survive conversations and reloads through `dsh-live-voice.settings`.
|
|
10
|
+
|
|
11
|
+
### Application continuity
|
|
12
|
+
|
|
13
|
+
An explicit choice to keep voice conversation mode active may follow the committed composer across a chat route change. This continuity does not authorize transient operational state to leak into the new composer.
|
|
14
|
+
|
|
15
|
+
### Per-session transient state
|
|
16
|
+
|
|
17
|
+
Soft mute, capture, recognition activity, partial transcript ownership, pending delivery countdowns, silence timers, queued playback, paused playback, temporary errors, abort controllers, and resource ownership belong to the current session. They must be reset or cancelled when that session ends or is replaced. In particular, a new voice session must not start soft-muted merely because the preceding session was muted.
|
|
18
|
+
|
|
19
|
+
### Per-operation state
|
|
20
|
+
|
|
21
|
+
Permission requests, engine starts, transcription requests, synthesis requests, and audio preparation each have their own cancellation generation. Results from an obsolete generation must not modify the current composer or playback queue.
|
|
22
|
+
|
|
23
|
+
## Independent states
|
|
24
|
+
|
|
25
|
+
Capture, recognition, user speech activity, message delivery, agent generation, synthesis, and playback are related but distinct. Mute is not stop. Pause is not cancel. Stopping speech must not silently stop recognition. Ending the conversation must release all voice resources.
|
|
26
|
+
|
|
27
|
+
## Turn-taking
|
|
28
|
+
|
|
29
|
+
- **Speakers:** playback gates recognition/capture so speaker output is not recognized. Manual interruption transfers ownership only after old playback teardown succeeds.
|
|
30
|
+
- **Headphones:** the microphone can remain open. Interruption requires qualifying transcript/activity rather than a single noise event.
|
|
31
|
+
- Automatic delivery starts only after final transcription and the configured quiet delay. New speech, edits, cancellation, or session replacement cancels a pending delivery.
|
|
32
|
+
- Automatic assistant playback waits for continuous silence and pending transcription/delivery work. Renewed speech restarts the delay. Manual per-message playback is not delayed.
|
|
33
|
+
- Sending a user message does not stop existing assistant playback unless the explicit interruption preference is enabled. Loaded history never counts as a new user turn.
|
|
34
|
+
|
|
35
|
+
## Lifecycle transitions
|
|
36
|
+
|
|
37
|
+
| Transition | Required result |
|
|
38
|
+
| --- | --- |
|
|
39
|
+
| Start input | Acquire current ownership; never inherit stale soft mute or transcript state |
|
|
40
|
+
| Soft mute | Keep recognition available for the resume command, but discard dictation |
|
|
41
|
+
| Resume | Clear only the current session's soft mute |
|
|
42
|
+
| Stop input | Cancel starts, capture, recognition, partial hypotheses, and delivery timers |
|
|
43
|
+
| End conversation | Stop input and playback, clear queues/timers/transient errors, release ownership |
|
|
44
|
+
| Switch chat with voice mode active | Retire the old composer, cancel its operations, create clean transient state for the new composer |
|
|
45
|
+
| Settings change | Stop affected resources before applying the normalized preference to the next operation |
|
|
46
|
+
| Unmount/dispose/pagehide | Invalidate pending work and release every owned resource |
|
|
47
|
+
| Late async result | Ignore it unless its session and operation generation are still current |
|
|
48
|
+
|
|
49
|
+
## Invariants
|
|
50
|
+
|
|
51
|
+
1. A new session never inherits soft mute.
|
|
52
|
+
2. Old recognition never writes into a new composer.
|
|
53
|
+
3. Cancelled or replaced playback never resumes itself.
|
|
54
|
+
4. Only the current owner may control shared hardware or host playback.
|
|
55
|
+
5. Failed teardown blocks unsafe replacement rather than overlapping resources.
|
|
56
|
+
6. End conversation leaves no capture, playback, countdown, queued transcript, or silence timer active.
|
|
57
|
+
7. Raw audio and transcript content are not logged by default.
|
|
58
|
+
|
|
59
|
+
These rules require regression tests where automation is possible and physical-device checks where browser permissions or hardware behavior cannot be simulated faithfully.
|