uneeq-js 3.12.1 → 3.13.0

package/README.md

@@ -64,6 +64,34 @@ Logs will only be displayed if they are at or above the log level specified when
 Log messages will be prefixed with the string 'UneeQ: ' so that uneeq-js logs can be differentiated from client logs easily.
 
 
+## Beta features
+
+### `audioUpstreamMode` (beta for non-default values)
+
+`audioUpstreamMode` on `UneeqConfig` controls where user microphone audio is delivered. **Omit the field or leave it `undefined` for the existing GA behaviour — no existing caller needs to change anything.**
+
+| Value | Status | What it does |
+| --- | --- | --- |
+| `'speech-recognition-service'` | **GA / default** | Mic is captured with browser AEC/AGC/NS, sent to UneeQ's STT WebSocket (Google or Deepgram). Transcripts flow to Renny over the data channel. This is the behaviour when the field is omitted. |
+| `'pixel-streaming'` | **Beta** | Mic is attached to the Pixel Streaming WebRTC peer connection as an upstream audio track (Epic's `UseMic` flag). No STT client is instantiated; `speechRecognitionProvider` is ignored. Use when Renny drives conversation in-renderer. |
+| `'both'` | **Beta** | Mic is delivered to BOTH the PS peer connection AND the STT WebSocket. Two independent `getUserMedia` captures run in parallel. Use when Renny needs raw audio AND we still want server-side transcripts (captions, compliance, transcript-driven LLM). |
+
+**Headphone caveat:** the Epic Pixel Streaming library captures the PS mic track without AEC/AGC/NS, so expect echo/feedback unless the user is wearing headphones. Not a concern in the default `'speech-recognition-service'` mode.
+
+**Shape and defaults may change while the two non-default modes are beta.** The library emits an `info`-level log line at session init whenever a beta mode is active so the current upstream routing is visible in support captures:
+
+```
+audioUpstreamMode='pixel-streaming' is beta; behaviour may change.
+```
+
+**New APIs on the `Uneeq` instance (beta):**
+
+- `muteUpstreamMic()` / `unmuteUpstreamMic()` — mute/unmute the PS upstream mic track only. No-op with a warning in `'speech-recognition-service'` mode (there is no PS track to control). In `'pixel-streaming'` / `'both'` mode these are preferred over `enableMicrophone()` if you only want to gate the PS leg (e.g. to stop feeding audio into the in-renderer agent without pausing STT transcripts).
+- `pauseSpeechRecognition()` / `resumeSpeechRecognition()` — unchanged in behaviour, but now explicitly warn and return `false` in `'pixel-streaming'` mode since there is no STT leg to pause.
+- `enableMicrophone(enabled)` — now dispatches to whichever leg(s) are active for the current mode: STT in `'speech-recognition-service'`, PS in `'pixel-streaming'`, both in `'both'`.
+
+See [`documentation/PS_AUDIO_UPSTREAM.md`](./documentation/PS_AUDIO_UPSTREAM.md) for the full design.
+
 ## Notes
 #### onnxruntime-web
 onnxruntime-web only works on version 1.15.1 at this time. Upgrading this package will likely break VAD funtionality.