@voicethere/client 0.16.1 → 0.17.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +82 -49
- package/README.md +21 -0
- package/dist/browser/browser-session.d.ts +5 -0
- package/dist/browser/browser-session.d.ts.map +1 -1
- package/dist/browser/browser-session.js +3 -0
- package/dist/browser/browser-session.js.map +1 -1
- package/dist/browser/browser-voice-session.d.ts +13 -0
- package/dist/browser/browser-voice-session.d.ts.map +1 -1
- package/dist/browser/browser-voice-session.js +11 -0
- package/dist/browser/browser-voice-session.js.map +1 -1
- package/dist/embed/index.d.ts +1 -1
- package/dist/embed/index.d.ts.map +1 -1
- package/dist/embed/index.js +28 -29
- package/dist/embed/index.js.map +1 -1
- package/dist/embed/spoken-chat-caption.d.ts +1 -12
- package/dist/embed/spoken-chat-caption.d.ts.map +1 -1
- package/dist/embed/spoken-chat-caption.js +1 -334
- package/dist/embed/spoken-chat-caption.js.map +1 -1
- package/dist/index.d.ts +1 -0
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +1 -0
- package/dist/index.js.map +1 -1
- package/dist/node/index.d.ts +1 -0
- package/dist/node/index.d.ts.map +1 -1
- package/dist/node/index.js +1 -0
- package/dist/node/index.js.map +1 -1
- package/dist/spoken-text-reveal.d.ts +54 -0
- package/dist/spoken-text-reveal.d.ts.map +1 -0
- package/dist/spoken-text-reveal.js +734 -0
- package/dist/spoken-text-reveal.js.map +1 -0
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,10 +1,32 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
-
## [
|
|
3
|
+
## [0.17.1] - 2026-10-30
|
|
4
|
+
|
|
5
|
+
### Fixed
|
|
6
|
+
|
|
7
|
+
- A spoken reply interrupted before its first word was revealed (a barge-in, often the listener's speakers echoing into the mic) no longer disappears. Once the reply's audio has started, `onAgentText` and the spoken caption show at least its first sentence when the turn is interrupted, replaced by the next turn, or when the session closes or reconnects. A reply whose audio never started still shows nothing. An `agent_speaking_start` that arrives just before a turn's first `assistant_delta` now counts for that turn and its text starts revealing right away, and closing the session while a completed reply is still being revealed reports its full text.
|
|
8
|
+
|
|
9
|
+
## [0.17.0] - 2026-10-30
|
|
10
|
+
|
|
11
|
+
### Added
|
|
12
|
+
|
|
13
|
+
- Spoken text reveal for voice sessions. New session options `onAgentText` and `spokenTextReveal` (`"playback"` default, or `"immediate"`) on `connectBrowserSession`, `connectBrowserVoiceSession` and the `./node` entry. Spoken `assistant_delta` / `assistant_done` messages (`spoken: true`, paired by `turnId`) are revealed word by word while the agent speaks, with `final` and `inReplyTo` on each update. Typed turns are reported at once and the `chat_reply` that follows is not reported twice. The reveal and its timers are disposed when the session closes.
|
|
14
|
+
- `createSpokenChatCaption` handles spoken `assistant_delta` / `assistant_done` natively, and `createAgentTextReveal` is exported from `.`, `./browser`, `./embed` and `./node`. The caption module moved to `src/spoken-text-reveal.ts`; `@voicethere/client/embed` keeps exporting it.
|
|
15
|
+
|
|
16
|
+
### Changed
|
|
17
|
+
|
|
18
|
+
- The embed widget now paces spoken agent replies by default (`streamSpokenText` unset). Spoken text waits for `agent_speaking_start` and is never shown before the audio, except that a completed reply whose audio never starts is shown in full after 5 s. Plain `chat_reply` text is still shown at once; `streamSpokenText: true` paces it too and `false` paces nothing.
|
|
19
|
+
|
|
20
|
+
## [0.16.1] - 2026-10-30
|
|
4
21
|
|
|
5
22
|
### Fixed
|
|
6
23
|
|
|
7
24
|
- `createSpokenChatCaption` can caption a streamed reply: send `chat_reply` with `partial: true` while text grows and a final `chat_reply` without it. A streamed caption pauses between speech chunks instead of jumping to the full text.
|
|
25
|
+
|
|
26
|
+
## [0.16.0] - 2026-10-30
|
|
27
|
+
|
|
28
|
+
### Fixed
|
|
29
|
+
|
|
8
30
|
- `createSpokenChatCaption` is now exported from `@voicethere/client/embed`. It shipped in the package but could not be imported. `VOICETHERE_CHAT_CHANNEL_LABEL` is now exported from `@voicethere/client/browser` as well. A new test packs the package and checks the public surface of every `exports` subpath.
|
|
9
31
|
|
|
10
32
|
## [0.15.0] - 2026-10-28
|
|
@@ -117,7 +139,7 @@
|
|
|
117
139
|
- Session error code `SESSION_ENDED_DURING_RECONNECT` when the gateway rejects the reconnect token with HTTP 401.
|
|
118
140
|
- `disconnect()` / `disconnectAsync()` send `{ "type": "client_hangup" }` (`CLIENT_HANGUP_MESSAGE_TYPE`) on the control channel before closing.
|
|
119
141
|
|
|
120
|
-
## 0.10.0
|
|
142
|
+
## [0.10.0] - 2026-10-04
|
|
121
143
|
|
|
122
144
|
### Changed
|
|
123
145
|
|
|
@@ -129,19 +151,19 @@
|
|
|
129
151
|
- `startSession` failures from a rejected `POST /sessions` now include `httpStatus`.
|
|
130
152
|
- Exported `widgetKeyMapUrl` and `widgetProjectBootstrapUrl` URL builders.
|
|
131
153
|
|
|
132
|
-
## 0.9.0
|
|
154
|
+
## [0.9.0] - 2026-09-26
|
|
133
155
|
|
|
134
156
|
### Added
|
|
135
157
|
|
|
136
158
|
- **Key-only embed bootstrap** — `createVoiceThereWidgetAsync({ clientKey })` loads `{cdn}/widgets/by-key/{sha256(clientKey)}/bootstrap.json` (no `Authorization`; `credentials: "omit"`), caches the document in `sessionStorage` for repeat navigations, and merges optional published `widget` appearance. Legacy `configUrl` still fetches `config.json` only. Default hosts: `https://sessions.voicethere.io/v1` and `https://cdn.voicethere.io` (staging sessions host pairs with `https://cdn.voicethere.dev`).
|
|
137
159
|
|
|
138
|
-
## 0.8.3
|
|
160
|
+
## [0.8.3] - 2026-09-21
|
|
139
161
|
|
|
140
162
|
### Added
|
|
141
163
|
|
|
142
164
|
- **Embed spoken chat captions** — optional `streamSpokenText` (default off) reveals incoming agent `chat_reply` text word-by-word during speech; snaps on `agent_speaking_end` and freezes on `barge_in`. Supports `chat_reply` fields `stream`, `utteranceId`, and `durationMs` on voice-control messages.
|
|
143
165
|
|
|
144
|
-
## 0.8.2
|
|
166
|
+
## [0.8.2] - 2026-09-19
|
|
145
167
|
|
|
146
168
|
### Added
|
|
147
169
|
|
|
@@ -155,13 +177,13 @@
|
|
|
155
177
|
- **Embed panel UX** — opening the chat hides the launcher pill; panel header includes a close control (`data-vt-open` on the root).
|
|
156
178
|
- **Embed transcript** — message bubbles replace the debug `<pre>` log; `onControlMessage` appends incoming agent/chat text.
|
|
157
179
|
|
|
158
|
-
## 0.8.1
|
|
180
|
+
## [0.8.1] - 2026-09-01
|
|
159
181
|
|
|
160
182
|
### Fixed
|
|
161
183
|
|
|
162
184
|
- **Mic-denied synthetic fallback** — When microphone permission is denied or unavailable, the synthetic outbound track now sends inaudible looping white noise instead of silence so WebRTC is not DTX-suppressed, the runner receives `ontrack`, and the voice conversation can start. The production embed API `createVoiceThereWidget({ apiBase, projectId, clientKey, mode })` (no `configUrl`) is unchanged.
|
|
163
185
|
|
|
164
|
-
## 0.8.0
|
|
186
|
+
## [0.8.0] - 2026-08-31
|
|
165
187
|
|
|
166
188
|
### Added
|
|
167
189
|
|
|
@@ -172,7 +194,7 @@
|
|
|
172
194
|
- **Embed widget CDN config** — `createVoiceThereWidgetAsync` still mounts the launcher from inline options when `configUrl` is missing, unreachable, or returns invalid JSON; warns in the console instead of rejecting the promise.
|
|
173
195
|
- **Audio visualizer idle frame** — `attachAudioVisualizer().stop()` and ended MediaStream tracks now clear the canvas and draw a flat midline instead of freezing on the last analyser snapshot.
|
|
174
196
|
|
|
175
|
-
## 0.7.29
|
|
197
|
+
## [0.7.29] - 2026-08-28
|
|
176
198
|
|
|
177
199
|
### Added
|
|
178
200
|
|
|
@@ -187,13 +209,13 @@
|
|
|
187
209
|
|
|
188
210
|
- **Reconnect callbacks** — `onIceRecovery`, `onReconnecting`, and `onReconnected` accept an optional 2nd arg `VoiceSessionReconnectInfo` (`reason`, effective `iceTransportPolicy`) for E2E/load-test diagnostics.
|
|
189
211
|
|
|
190
|
-
## 0.7.28
|
|
212
|
+
## [0.7.28] - 2026-08-15
|
|
191
213
|
|
|
192
214
|
### Fixed
|
|
193
215
|
|
|
194
216
|
- Node runtime (`createNodeWebRtcRuntime`) waits for ICE gathering (`gatheringComplete` when available) before sending the SDP answer so relay candidates are in the answer. Browser default remains trickle-immediate.
|
|
195
217
|
|
|
196
|
-
## 0.7.27
|
|
218
|
+
## [0.7.27] - 2026-08-12
|
|
197
219
|
|
|
198
220
|
### Added
|
|
199
221
|
|
|
@@ -202,77 +224,77 @@
|
|
|
202
224
|
- **`audioInputDeviceId`** option on connect (`deviceId: { ideal }` for initial GUM).
|
|
203
225
|
- **`client/src/browser/microphone.ts`** helpers exported from `@voicethere/client/browser`.
|
|
204
226
|
|
|
205
|
-
## 0.7.26
|
|
227
|
+
## [0.7.26] - 2026-08-05
|
|
206
228
|
|
|
207
229
|
### Added
|
|
208
230
|
|
|
209
231
|
- **ICE recovery (relay-leaning)** — before same-session auto-reconnect budget: on PeerConnection fail (or stuck ICE checking ≥12s with `nominated=0`), rebuild via signaling with `iceTransportPolicy=relay`. Uses `onIceRecovery` / `maxIceRecoveryAttempts` (default **1**) and does **not** increment `autoReconnectAttempts` or call `onReconnecting`. Set `maxIceRecoveryAttempts=0` for legacy behavior; `iceRecoveryStuckCheckingMs=0` disables proactive stuck-checking.
|
|
210
232
|
- **`webrtc ice_triage`** — compact diagnostics line (selected/nominated/succeeded/failed + host/srflx/relay counts) in `formatWebRtcDiagnosticsLines` / `formatIceTriageLine`.
|
|
211
233
|
|
|
212
|
-
## 0.7.25
|
|
234
|
+
## [0.7.25] - 2026-08-03
|
|
213
235
|
|
|
214
236
|
### Fixed
|
|
215
237
|
|
|
216
238
|
- **`session_reconnect_token`** — after updating join credentials, forward the control message to `onControlMessage` so E2E waiters and app hooks observe the token (previously returned early after internal update only).
|
|
217
239
|
|
|
218
|
-
## 0.7.24
|
|
240
|
+
## [0.7.24] - 2026-08-03
|
|
219
241
|
|
|
220
242
|
### Added
|
|
221
243
|
|
|
222
244
|
- **`onReconnected`** — callback after same-session reconnect reaches ready again (not on first connect). Enables load/e2e mid-turn phrase resend without treating reconnect as a fresh session.
|
|
223
245
|
|
|
224
|
-
## 0.7.23
|
|
246
|
+
## [0.7.23] - 2026-08-03
|
|
225
247
|
|
|
226
248
|
### Added
|
|
227
249
|
|
|
228
250
|
- **`session_reconnect_token`** — voice-control handler updates in-memory join credentials when the runner mints an opaque reconnect token; same-session `reconnect()` and auto-reconnect open signaling with the latest token.
|
|
229
251
|
- **`forceCloseSignalingForTests()`** — test/E2E hook on `BrowserVoiceSession` to force-close the signaling WebSocket and trigger `signaling_closed` auto-reconnect.
|
|
230
252
|
|
|
231
|
-
## 0.7.22
|
|
253
|
+
## [0.7.22] - 2026-07-31
|
|
232
254
|
|
|
233
255
|
### Fixed
|
|
234
256
|
|
|
235
257
|
- **POST `/sessions` 429 mapping** — monthly usage gate responses (`NWRTC_MONTHLY_USAGE_EXCEEDED`) now surface `MONTHLY_USAGE_EXCEEDED` instead of always `CAPACITY_EXCEEDED`.
|
|
236
258
|
|
|
237
|
-
## 0.7.21
|
|
259
|
+
## [0.7.21] - 2026-07-31
|
|
238
260
|
|
|
239
261
|
### Fixed
|
|
240
262
|
|
|
241
263
|
- **`getConnectionState()` after idle kick** — inbound `session_close` now sets internal `connectionState` to `closed` (matching connection status), so callers polling `getConnectionState()` observe the disconnect.
|
|
242
264
|
|
|
243
|
-
## 0.7.20
|
|
265
|
+
## [0.7.20] - 2026-07-31
|
|
244
266
|
|
|
245
267
|
### Changed
|
|
246
268
|
|
|
247
269
|
- **Deps** — `@node-webrtc-rust/{sdk,signaling}` **0.7.0** (Opus SDP omits `maxaveragebitrate` unless env set; encode default 400 kbps).
|
|
248
270
|
|
|
249
|
-
## 0.7.19
|
|
271
|
+
## [0.7.19] - 2026-07-30
|
|
250
272
|
|
|
251
273
|
### Fixed
|
|
252
274
|
|
|
253
275
|
- **Inbound `session_close`** — idle kick and other server-initiated closes now emit a non-recoverable `session_error` (`SESSION_IDLE_TIMEOUT` or `WEBRTC_CONNECTION_CLOSED`) and stop auto-reconnect so dashboard chat does not stay "Connected".
|
|
254
276
|
- **`auto_reconnect_exhausted`** — emits `WEBRTC_RECONNECT_EXHAUSTED`, marks the session terminal, and publishes a closed/not-ready connection snapshot.
|
|
255
277
|
|
|
256
|
-
## 0.7.18
|
|
278
|
+
## [0.7.18] - 2026-07-30
|
|
257
279
|
|
|
258
280
|
### Fixed
|
|
259
281
|
|
|
260
282
|
- **Browser ICE:** send the SDP answer immediately after `setLocalDescription` and trickle candidates via `onicecandidate`, instead of waiting for `iceGatheringState === "complete"`. Waiting for gather-complete delayed CreatePermission on the runner and caused Chromium sessions (especially TURN relay) to fail with `WEBRTC_CONNECTION_FAILED` while Node clients succeeded.
|
|
261
283
|
- **`waitForConnected`:** fail fast when `connectionState` / `iceConnectionState` enter `failed` or `disconnected`, instead of waiting for the full timeout (dashboard half-open recovery).
|
|
262
284
|
|
|
263
|
-
## 0.7.17
|
|
285
|
+
## [0.7.17] - 2026-07-28
|
|
264
286
|
|
|
265
287
|
### Changed
|
|
266
288
|
|
|
267
289
|
- Treat control-channel JSON `position` as high-frequency: log at debug (with `keepalive` / `state` / `tick`) so load/redis-sync smokes stay quiet unless `LOAD_TEST_CLIENT_DEBUG=1`.
|
|
268
290
|
|
|
269
|
-
## 0.7.16
|
|
291
|
+
## [0.7.16] - 2026-07-28
|
|
270
292
|
|
|
271
293
|
### Changed
|
|
272
294
|
|
|
273
295
|
- Stop logging `dc/control_not_open` / `dc/sync_not_open` on every failed send while channels are connecting. Open/close transitions remain single `dc/open` / `dc/close` events (avoids spam from readiness polls).
|
|
274
296
|
|
|
275
|
-
## 0.7.15
|
|
297
|
+
## [0.7.15] - 2026-07-28
|
|
276
298
|
|
|
277
299
|
### Added
|
|
278
300
|
|
|
@@ -283,7 +305,7 @@
|
|
|
283
305
|
|
|
284
306
|
- Consumers that require abortable `startSession` (canonical voice soak) must install this release (or newer). Older published `0.7.14` builds do not advertise the capability.
|
|
285
307
|
|
|
286
|
-
## 0.7.14
|
|
308
|
+
## [0.7.14] - 2026-07-27
|
|
287
309
|
|
|
288
310
|
### Added
|
|
289
311
|
|
|
@@ -297,31 +319,31 @@
|
|
|
297
319
|
- **Removed startup ICE PC reconstruction** — race-prone automatic peer rebuild on failed/disconnected startup was dropped. Existing same-session signaling reconnect is unchanged.
|
|
298
320
|
- Diagnostic redaction covers `http(s)`, `ws(s)`, `turn(s)`, `stun` URIs, join tokens, JSON credentials, and SDP `ice-pwd`; async diagnostic handler rejections are isolated.
|
|
299
321
|
|
|
300
|
-
## 0.7.13
|
|
322
|
+
## [0.7.13] - 2026-07-26
|
|
301
323
|
|
|
302
324
|
### Added
|
|
303
325
|
|
|
304
326
|
- **ICE candidate-pair diagnostics** — `getWebRtcDiagnostics()` / `summarizeRtcStatsReport` now resolve local/remote candidate types for every retained pair (order-independent), including failed non-selected pairs. Aggregate counts add `prflx` / unknown local candidates and total local candidate reports. Formatted lines stay redacted (no addresses, ports, URLs, or credentials).
|
|
305
327
|
|
|
306
|
-
## 0.7.12
|
|
328
|
+
## [0.7.12] - 2026-07-24
|
|
307
329
|
|
|
308
330
|
### Added
|
|
309
331
|
|
|
310
332
|
- **`fetchSessionApi`** — retries session-service `POST`/`GET` on Envoy-style 502–504 / `upstream connect error` / connection termination (and network `fetch failed`), with short backoff. Wired into `startSession` and status polling so rolling deploys do not fail one-shot polls.
|
|
311
333
|
|
|
312
|
-
## 0.7.11
|
|
334
|
+
## [0.7.11] - 2026-07-23
|
|
313
335
|
|
|
314
336
|
### Changed
|
|
315
337
|
|
|
316
338
|
- **Adaptive session provisioning polling** — replace fixed 1s status polls with backoff + jitter, and honor Session API `retry_after_ms` hints to cut control-plane load during wait queues.
|
|
317
339
|
|
|
318
|
-
## 0.7.10
|
|
340
|
+
## [0.7.10] - 2026-07-21
|
|
319
341
|
|
|
320
342
|
### Fixed
|
|
321
343
|
|
|
322
344
|
- **Same-session reconnect after `WEBRTC_SDP_NEGOTIATION_FAILED`** — malformed/empty remote offers (missing `a=ice-ufrag`) now trigger signaling reconnect under the default `same-session` policy so `waitForConnected()` can recover instead of hanging until timeout (scale-up burst flake class).
|
|
323
345
|
|
|
324
|
-
## 0.7.9
|
|
346
|
+
## [0.7.9] - 2026-07-17
|
|
325
347
|
|
|
326
348
|
### Added
|
|
327
349
|
|
|
@@ -332,19 +354,19 @@
|
|
|
332
354
|
|
|
333
355
|
- **`waitForConnected()` after early offer failure** — preserve non-retriable SDP handler errors via `pendingConnectFailure` so callers fail fast instead of waiting for the connect timeout.
|
|
334
356
|
|
|
335
|
-
## 0.7.8
|
|
357
|
+
## [0.7.8] - 2026-07-17
|
|
336
358
|
|
|
337
359
|
### Fixed
|
|
338
360
|
|
|
339
361
|
- **Terminal `session_error` vs transport failure** — when the server sends `session_error` with `recoverable: false` (e.g. `AGENT_HANDLER_FAILED`), mark the session as gracefully disconnected so auto-reconnect does not overwrite the error with `WEBRTC_CONNECTION_FAILED`.
|
|
340
362
|
|
|
341
|
-
## 0.7.7
|
|
363
|
+
## [0.7.7] - 2026-07-15
|
|
342
364
|
|
|
343
365
|
### Changed
|
|
344
366
|
|
|
345
367
|
- **Data-channel debug noise** — `keepalive`, `state`, and `tick` payloads log at `debug` instead of `info` (inbound and `sendToAgent` outbound). Staging E2E omits them unless `LOAD_TEST_CLIENT_DEBUG=1`.
|
|
346
368
|
|
|
347
|
-
## 0.7.6
|
|
369
|
+
## [0.7.6] - 2026-07-15
|
|
348
370
|
|
|
349
371
|
### Added
|
|
350
372
|
|
|
@@ -355,31 +377,31 @@
|
|
|
355
377
|
|
|
356
378
|
- **`waitForConnected()` during reconnect** — preserve the connect wait promise across signaling rejoin and ignore intentional peer-connection teardown when applying a replacement server offer.
|
|
357
379
|
|
|
358
|
-
## 0.7.5
|
|
380
|
+
## [0.7.5] - 2026-07-13
|
|
359
381
|
|
|
360
382
|
### Changed
|
|
361
383
|
|
|
362
384
|
- **`waitForConnected()` timeout** — error message includes connection phase, peer connection state, and data-channel flags for E2E/Loki triage without extra debug tooling.
|
|
363
385
|
|
|
364
|
-
## 0.7.4
|
|
386
|
+
## [0.7.4] - 2026-07-10
|
|
365
387
|
|
|
366
388
|
### Added
|
|
367
389
|
|
|
368
390
|
- **`getWebRtcDiagnostics()`** on `BrowserVoiceSession` — ICE transport state plus `getStats()` candidate-pair / relay summary for connect failure triage in staging E2E.
|
|
369
391
|
|
|
370
|
-
## 0.7.3
|
|
392
|
+
## [0.7.3] - 2026-07-07
|
|
371
393
|
|
|
372
394
|
### Fixed
|
|
373
395
|
|
|
374
396
|
- **`waitForIceGatheringComplete` on Node** — fall back to `onicegatheringstatechange` when `RTCPeerConnection.addEventListener` is unavailable (`@node-webrtc-rust/sdk`), fixing headless E2E load tests that use `@voicethere/client/node`.
|
|
375
397
|
|
|
376
|
-
## 0.7.2
|
|
398
|
+
## [0.7.2] - 2026-07-07
|
|
377
399
|
|
|
378
400
|
### Fixed
|
|
379
401
|
|
|
380
402
|
- **Browser answer ICE gathering** — wait for local ICE gathering to complete before sending the SDP answer (matches agent offer and `@node-webrtc-rust/signaling` auto-negotiate), reducing first-connect `WEBRTC_CONNECTION_FAILED` when TURN relay candidates are required.
|
|
381
403
|
|
|
382
|
-
## 0.7.1
|
|
404
|
+
## [0.7.1] - 2026-07-07
|
|
383
405
|
|
|
384
406
|
### Added
|
|
385
407
|
|
|
@@ -396,7 +418,7 @@
|
|
|
396
418
|
|
|
397
419
|
- **Initial signaling join** — first connect uses a dedicated join path instead of the same-session reconnect flow, avoiding spurious peer-connection resets and reconnect debug noise.
|
|
398
420
|
|
|
399
|
-
## 0.7.0
|
|
421
|
+
## [0.7.0] - 2026-07-03
|
|
400
422
|
|
|
401
423
|
### Added
|
|
402
424
|
|
|
@@ -410,7 +432,7 @@
|
|
|
410
432
|
|
|
411
433
|
- **`connectBrowserSession` mic request** — `requestMic` is derived from resolved mode (`voice` and `voice+data` request mic; data-only provisioning maps to `chat` without mic).
|
|
412
434
|
|
|
413
|
-
## 0.6.1
|
|
435
|
+
## [0.6.1] - 2026-06-28
|
|
414
436
|
|
|
415
437
|
### Fixed
|
|
416
438
|
|
|
@@ -421,7 +443,7 @@
|
|
|
421
443
|
- **Split binary receive callbacks** — browser voice sessions now support a dedicated `onSyncBinaryMessage` callback for `voicethere-sync`, while `onBinaryMessage` handles `voice-control` binary frames.
|
|
422
444
|
- **Regression coverage** — added tests to lock callback forwarding and channel-specific binary routing.
|
|
423
445
|
|
|
424
|
-
## 0.6.0
|
|
446
|
+
## [0.6.0] - 2026-06-28
|
|
425
447
|
|
|
426
448
|
### Changed
|
|
427
449
|
|
|
@@ -432,13 +454,13 @@
|
|
|
432
454
|
|
|
433
455
|
- **Boundary regression tests** — added tests that fail if browser-facing sources reintroduce `@node-webrtc-rust/sdk` imports.
|
|
434
456
|
|
|
435
|
-
## 0.5.7
|
|
457
|
+
## [0.5.7] - 2026-06-28
|
|
436
458
|
|
|
437
459
|
### Added
|
|
438
460
|
|
|
439
461
|
- **Root connection error handler surface** — session and signaling transport errors now bubble to the process-wide root connection error handler so apps can centralize telemetry/logging without wiring `.on("error")` on every connection.
|
|
440
462
|
|
|
441
|
-
## 0.5.6
|
|
463
|
+
## [0.5.6] - 2026-06-23
|
|
442
464
|
|
|
443
465
|
### Added
|
|
444
466
|
|
|
@@ -448,20 +470,20 @@
|
|
|
448
470
|
|
|
449
471
|
- **Dependencies** — `@node-webrtc-rust/signaling@0.6.2`; peer `@node-webrtc-rust/sdk` `>=0.6.2`.
|
|
450
472
|
|
|
451
|
-
## 0.5.5
|
|
473
|
+
## [0.5.5] - 2026-06-20
|
|
452
474
|
|
|
453
475
|
### Fixed
|
|
454
476
|
|
|
455
477
|
- **`waitForConnected` after transport failure** — clear the internal connect promise when the peer connection enters `failed` or `closed` so same-session auto-reconnect and a subsequent `waitForConnected()` can succeed instead of hanging on a stale rejected promise.
|
|
456
478
|
- **`micPump: "external"` Node connect** — send a short PCM kick on mic attach (before answer handling) so headless load tests emit RTP during ICE like the default silent pump.
|
|
457
479
|
|
|
458
|
-
## 0.5.4
|
|
480
|
+
## [0.5.4] - 2026-06-19
|
|
459
481
|
|
|
460
482
|
### Added
|
|
461
483
|
|
|
462
484
|
- **`micPump`** on `connectBrowserVoiceSession` — `'silent'` (default) or `'external'` so headless load tests can pump scripted PCM via `writeSample` without racing the silent mic loop
|
|
463
485
|
|
|
464
|
-
## 0.5.3
|
|
486
|
+
## [0.5.3] - 2026-06-19
|
|
465
487
|
|
|
466
488
|
### Added
|
|
467
489
|
|
|
@@ -475,7 +497,7 @@
|
|
|
475
497
|
|
|
476
498
|
- Recoverable WebRTC disconnects (`WEBRTC_CONNECTION_FAILED`, `WEBRTC_CONNECTION_CLOSED`) emit `session_error` with `recoverable: true` and trigger same-session retry when policy allows
|
|
477
499
|
|
|
478
|
-
## 0.5.2
|
|
500
|
+
## [0.5.2] - 2026-06-19
|
|
479
501
|
|
|
480
502
|
### Added
|
|
481
503
|
|
|
@@ -489,7 +511,18 @@
|
|
|
489
511
|
- **Node headless voice** — attach mic with `addTrack` **before** `setRemoteDescription` so the answer SDP includes `sendrecv` (required by `@node-webrtc-rust/sdk`; fixes runner mic-track timeout in staging E2E).
|
|
490
512
|
- **Mic RTP pump** — after `connected`, send kick + silent PCM frames via `writeSample` so the server receives `ontrack` and starts VoiceAgent.
|
|
491
513
|
|
|
492
|
-
## 0.5.
|
|
514
|
+
## [0.5.1] - 2026-06-18
|
|
515
|
+
|
|
516
|
+
### Fixed
|
|
517
|
+
|
|
518
|
+
- Node headless voice (`@voicethere/client/node`): the microphone track is now attached with `addTrack` before the remote description is set, so the answer SDP includes `sendrecv`.
|
|
519
|
+
- After the connection is up, the Node mic sends a short kick frame and then silent PCM frames through `writeSample`, so the server receives the audio track and starts the voice agent. The pump stops when the connection leaves `connected`.
|
|
520
|
+
|
|
521
|
+
### Changed
|
|
522
|
+
|
|
523
|
+
- Browser test harness README: documents the CORS requirements for calling the session API from the browser.
|
|
524
|
+
|
|
525
|
+
## [0.5.0] - 2026-06-17
|
|
493
526
|
|
|
494
527
|
### Added
|
|
495
528
|
|
|
@@ -497,7 +530,7 @@
|
|
|
497
530
|
- Injectable `WebRtcRuntime` on `connectBrowserVoiceSession` for Node test harnesses
|
|
498
531
|
- `waitForConnected`, `onControlMessage` on browser voice sessions
|
|
499
532
|
|
|
500
|
-
## 0.4.0
|
|
533
|
+
## [0.4.0] - 2026-06-16
|
|
501
534
|
|
|
502
535
|
### Added
|
|
503
536
|
|
|
@@ -505,7 +538,7 @@
|
|
|
505
538
|
- `onBinaryMessage` callback for inbound binary frames.
|
|
506
539
|
- Export `VOICE_SYNC_CHANNEL_LABEL` (matches helpers / agent sync channel).
|
|
507
540
|
|
|
508
|
-
## 0.3.0
|
|
541
|
+
## [0.3.0] - 2026-06-15
|
|
509
542
|
|
|
510
543
|
### Added
|
|
511
544
|
|
|
@@ -514,6 +547,6 @@
|
|
|
514
547
|
- `templates/` — React hook, embed HTML, debug page starters
|
|
515
548
|
- CI workflow and release tag publish (mirror `@voicethere/cli`)
|
|
516
549
|
|
|
517
|
-
## 0.1.0
|
|
550
|
+
## [0.1.0]
|
|
518
551
|
|
|
519
552
|
- Initial `connectVoiceSession` cloud/local signaling helper
|
package/README.md
CHANGED
|
@@ -217,6 +217,27 @@ await connectBrowserSession({
|
|
|
217
217
|
|
|
218
218
|
Agents send these events with `@voicethere/agent` 0.11+ on current runners. Older agents send none, so the callbacks simply never fire. `onControlMessage` still receives the raw payloads.
|
|
219
219
|
|
|
220
|
+
## Spoken text reveal
|
|
221
|
+
|
|
222
|
+
When the agent speaks, its text arrives as `assistant_delta` (one per sentence, `spoken: true`) and `assistant_done` messages, sent before the audio is heard. Pass `onAgentText` and the session reveals that text word by word while the agent talks, instead of showing every sentence the moment it is sent.
|
|
223
|
+
|
|
224
|
+
```ts
|
|
225
|
+
const session = await connectBrowserSession({
|
|
226
|
+
credentials,
|
|
227
|
+
onAgentText: ({ turnId, text, spoken, final, inReplyTo }) => {
|
|
228
|
+
// `text` is the whole turn so far. Replace the line for `turnId`.
|
|
229
|
+
renderAgentLine(turnId, text);
|
|
230
|
+
},
|
|
231
|
+
});
|
|
232
|
+
```
|
|
233
|
+
|
|
234
|
+
- `onAgentText` is called once per visible change. `text` is the full turn so far, keyed by `turnId`; `final` is `true` on the last update of a turn.
|
|
235
|
+
- `spokenTextReveal` is `"playback"` by default: the reveal follows `agent_speaking_start` / `agent_speaking_end`, pauses between spoken chunks, and stops at what was heard on `barge_in` or when `assistant_done` has `status` other than `completed`. Set `spokenTextReveal: "immediate"` to get each sentence at once.
|
|
236
|
+
- Typed turns (`assistant_delta` / `assistant_done` without `spoken`) are reported at once with `spoken: false`. The `chat_reply` that follows them repeats the same text and is not reported.
|
|
237
|
+
- `onControlMessage` still receives every raw message.
|
|
238
|
+
- A spoken turn is never shown before its audio: it waits for `agent_speaking_start`. If a completed reply never gets one, the full text is shown after 5 seconds.
|
|
239
|
+
- The embed widget option `streamSpokenText` has three values: unset (default) paces spoken replies and shows plain `chat_reply` text at once; `true` paces spoken replies and `chat_reply` text; `false` paces nothing and shows spoken replies at once.
|
|
240
|
+
|
|
220
241
|
## Embed widget customization
|
|
221
242
|
|
|
222
243
|
Import from `@voicethere/client/embed`. For production snippets, pass only a **client API key** (`vthc_…`) to `createVoiceThereWidgetAsync({ clientKey })`; the client fetches bootstrap JSON from the CDN (project id, session API base, and optional published appearance). Staging pages can pass `apiBase: "https://sessions.voicethere.dev/v1"` so the client uses the staging CDN. Legacy integrations may still pass `configUrl` (stable `config.json`) or inline `projectId` / `apiBase`.
|
|
@@ -13,6 +13,10 @@ export type ConnectBrowserSessionOptions = {
|
|
|
13
13
|
onSessionError?: import("../session-errors.js").SessionErrorHandler;
|
|
14
14
|
onControlMessage?: (payload: Record<string, unknown>) => void;
|
|
15
15
|
onAgentState?: BrowserVoiceSessionOptions["onAgentState"];
|
|
16
|
+
/** `playback` (default) paces spoken agent text with the audio; `immediate` reports it at once. */
|
|
17
|
+
spokenTextReveal?: BrowserVoiceSessionOptions["spokenTextReveal"];
|
|
18
|
+
/** The only callback that reports agent text (see {@link BrowserVoiceSessionOptions.onAgentText}). */
|
|
19
|
+
onAgentText?: BrowserVoiceSessionOptions["onAgentText"];
|
|
16
20
|
onAgentInterrupted?: BrowserVoiceSessionOptions["onAgentInterrupted"];
|
|
17
21
|
onBinaryMessage?: BrowserVoiceSessionOptions["onBinaryMessage"];
|
|
18
22
|
onSyncBinaryMessage?: BrowserVoiceSessionOptions["onSyncBinaryMessage"];
|
|
@@ -46,6 +50,7 @@ export { CustomerContextTooLargeError, MAX_CUSTOMER_CONTEXT_BYTES, startSession,
|
|
|
46
50
|
export { DEFAULT_SESSION_API_RETRY_DELAYS_MS, fetchSessionApi, isRetryableSessionApiBody, isRetryableSessionApiFailure, isRetryableSessionApiNetworkError, isRetryableSessionApiStatus, type FetchSessionApiOptions, type SessionApiRetryRuntime, } from "./session-api-retry.js";
|
|
47
51
|
export { createLocalSessionError, emitSessionError, isSessionErrorEvent, LOCAL_SESSION_ERROR_CODES, REMOTE_SESSION_ERROR_CODES, SESSION_ERROR_CODES, type SessionErrorCode, type SessionErrorEvent, type SessionErrorHandler, } from "../session-errors.js";
|
|
48
52
|
export { ConnectionError, createConnectionError, formatConnectionErrorSource, getRootConnectionErrorHandler, reportConnectionError, setRootConnectionErrorHandler, type ConnectionErrorSource, type RootConnectionErrorHandler, } from "../connection-errors.js";
|
|
53
|
+
export { createAgentTextReveal, createSpokenChatCaption, type AgentTextReveal, type AgentTextUpdate, type SpokenChatCaption, type SpokenChatCaptionOptions, type SpokenTurnUpsertInfo, type SpokenTextRevealMode, } from "../spoken-text-reveal.js";
|
|
49
54
|
export { attachAudioVisualizer } from "./audio-visualizer.js";
|
|
50
55
|
export { createDebugConsole } from "./debug-console.js";
|
|
51
56
|
export type { SessionCredentials, SessionStatusResponse, StartSessionOptions, StartSessionResult, } from "./session-provision.js";
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"browser-session.d.ts","sourceRoot":"","sources":["../../src/browser/browser-session.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,oBAAoB,CAAC;AACvD,OAAO,KAAK,EAAE,kBAAkB,EAAE,MAAM,wBAAwB,CAAC;AAEjE,OAAO,EAEL,KAAK,mBAAmB,EACxB,KAAK,0BAA0B,EAChC,MAAM,4BAA4B,CAAC;AACpC,OAAO,EAGL,KAAK,kBAAkB,EACxB,MAAM,oBAAoB,CAAC;AAC5B,OAAO,KAAK,EACV,sBAAsB,EAEvB,MAAM,+BAA+B,CAAC;AAEvC,MAAM,MAAM,4BAA4B,GAAG;IACzC,IAAI,CAAC,EAAE,kBAAkB,CAAC;IAC1B,WAAW,EAAE,kBAAkB,CAAC;IAChC,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,YAAY,CAAC,EAAE,gBAAgB,CAAC;IAChC,YAAY,CAAC,EAAE,YAAY,CAAC;IAC5B,eAAe,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;IAC1C,cAAc,CAAC,EAAE,OAAO,sBAAsB,EAAE,mBAAmB,CAAC;IACpE,gBAAgB,CAAC,EAAE,CAAC,OAAO,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,KAAK,IAAI,CAAC;IAC9D,YAAY,CAAC,EAAE,0BAA0B,CAAC,cAAc,CAAC,CAAC;IAC1D,kBAAkB,CAAC,EAAE,0BAA0B,CAAC,oBAAoB,CAAC,CAAC;IACtE,eAAe,CAAC,EAAE,0BAA0B,CAAC,iBAAiB,CAAC,CAAC;IAChE,mBAAmB,CAAC,EAAE,0BAA0B,CAAC,qBAAqB,CAAC,CAAC;IACxE,iBAAiB,CAAC,EAAE,0BAA0B,CAAC,mBAAmB,CAAC,CAAC;IACpE,eAAe,CAAC,EAAE,OAAO,4BAA4B,EAAE,eAAe,CAAC;IACvE,wBAAwB,CAAC,EAAE,MAAM,CAAC;IAClC,iBAAiB,CAAC,EAAE,MAAM,CAAC;IAC3B,sBAAsB,CAAC,EAAE,MAAM,CAAC;IAChC,aAAa,CAAC,EAAE,OAAO,4BAA4B,EAAE,0BAA0B,CAAC,eAAe,CAAC,CAAC;IACjG,cAAc,CAAC,EAAE,OAAO,4BAA4B,EAAE,0BAA0B,CAAC,gBAAgB,CAAC,CAAC;IACnG,aAAa,CAAC,EAAE,OAAO,4BAA4B,EAAE,0BAA0B,CAAC,eAAe,CAAC,CAAC;IACjG,kBAAkB,CAAC,EAAE,CAAC,MAAM,EAAE,sBAAsB,KAAK,IAAI,CAAC;IAC9D,kBAAkB,CAAC,EAAE,0BAA0B,CAAC,oBAAoB,CAAC,CAAC;IACtE,iBAAiB,CAAC,EAAE,0BAA0B,CAAC,mBAAmB,CAAC,CAAC;IACpE,oEAAoE;IACpE,kBAAkB,CAAC,EAAE,MAAM,CAAC;IAC5B,mFAAmF;IACnF,eAAe,CAAC,EAAE,0BAA0B,CAAC,iBAAiB,CAAC,CAAC;CACjE,CAAC;AAEF,MAAM,MAAM,cAAc,GAAG,mBAAmB,GAAG;IACjD,IAAI,EAAE,kBAAkB,CAAC;CAC1B,CAAC;AAEF,wBAAsB,qBAAqB,CACzC,OAAO,EAAE,4BAA4B,GACpC,OAAO,CAAC,cAAc,CAAC,
|
|
1
|
+
{"version":3,"file":"browser-session.d.ts","sourceRoot":"","sources":["../../src/browser/browser-session.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,oBAAoB,CAAC;AACvD,OAAO,KAAK,EAAE,kBAAkB,EAAE,MAAM,wBAAwB,CAAC;AAEjE,OAAO,EAEL,KAAK,mBAAmB,EACxB,KAAK,0BAA0B,EAChC,MAAM,4BAA4B,CAAC;AACpC,OAAO,EAGL,KAAK,kBAAkB,EACxB,MAAM,oBAAoB,CAAC;AAC5B,OAAO,KAAK,EACV,sBAAsB,EAEvB,MAAM,+BAA+B,CAAC;AAEvC,MAAM,MAAM,4BAA4B,GAAG;IACzC,IAAI,CAAC,EAAE,kBAAkB,CAAC;IAC1B,WAAW,EAAE,kBAAkB,CAAC;IAChC,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,YAAY,CAAC,EAAE,gBAAgB,CAAC;IAChC,YAAY,CAAC,EAAE,YAAY,CAAC;IAC5B,eAAe,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;IAC1C,cAAc,CAAC,EAAE,OAAO,sBAAsB,EAAE,mBAAmB,CAAC;IACpE,gBAAgB,CAAC,EAAE,CAAC,OAAO,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,KAAK,IAAI,CAAC;IAC9D,YAAY,CAAC,EAAE,0BAA0B,CAAC,cAAc,CAAC,CAAC;IAC1D,mGAAmG;IACnG,gBAAgB,CAAC,EAAE,0BAA0B,CAAC,kBAAkB,CAAC,CAAC;IAClE,sGAAsG;IACtG,WAAW,CAAC,EAAE,0BAA0B,CAAC,aAAa,CAAC,CAAC;IACxD,kBAAkB,CAAC,EAAE,0BAA0B,CAAC,oBAAoB,CAAC,CAAC;IACtE,eAAe,CAAC,EAAE,0BAA0B,CAAC,iBAAiB,CAAC,CAAC;IAChE,mBAAmB,CAAC,EAAE,0BAA0B,CAAC,qBAAqB,CAAC,CAAC;IACxE,iBAAiB,CAAC,EAAE,0BAA0B,CAAC,mBAAmB,CAAC,CAAC;IACpE,eAAe,CAAC,EAAE,OAAO,4BAA4B,EAAE,eAAe,CAAC;IACvE,wBAAwB,CAAC,EAAE,MAAM,CAAC;IAClC,iBAAiB,CAAC,EAAE,MAAM,CAAC;IAC3B,sBAAsB,CAAC,EAAE,MAAM,CAAC;IAChC,aAAa,CAAC,EAAE,OAAO,4BAA4B,EAAE,0BAA0B,CAAC,eAAe,CAAC,CAAC;IACjG,cAAc,CAAC,EAAE,OAAO,4BAA4B,EAAE,0BAA0B,CAAC,gBAAgB,CAAC,CAAC;IACnG,aAAa,CAAC,EAAE,OAAO,4BAA4B,EAAE,0BAA0B,CAAC,eAAe,CAAC,CAAC;IACjG,kBAAkB,CAAC,EAAE,CAAC,MAAM,EAAE,sBAAsB,KAAK,IAAI,CAAC;IAC9D,kBAAkB,CAAC,EAAE,0BAA0B,CAAC,oBAAoB,CAAC,CAAC;IACtE,iBAAiB,CAAC,EAAE,0BAA0B,CAAC,mBAAmB,CAAC,CAAC;IACpE,oEAAoE;IACpE,kBAAkB,CAAC,EAAE,MAAM,CAAC;IAC5B,mFAAmF;IACnF,eAAe,CAAC,EAAE,0BAA0B,CAAC,iBAAiB,CAAC,CAAC;CACjE,CAAC;AAEF,MAAM,MAAM,cAAc,GAAG,mBAAmB,GAAG;IACjD,IAAI,EAAE,kBAAkB,CAAC;CAC1B,CAAC;AAEF,wBAAsB,qBAAqB,CACzC,OAAO,EAAE,4BAA4B,GACpC,OAAO,CAAC,cAAc,CAAC,CA2DzB;AAED,wBAAsB,kBAAkB,CACtC,OAAO,EAAE,IAAI,CAAC,0BAA0B,EAAE,YAAY,CAAC,GACtD,OAAO,CAAC,mBAAmB,CAAC,CAE9B;AAED,YAAY,EACV,eAAe,EACf,yBAAyB,GAC1B,MAAM,4BAA4B,CAAC;AACpC,OAAO,EACL,yBAAyB,EACzB,6BAA6B,GAC9B,MAAM,2BAA2B,CAAC;AACnC,OAAO,EACL,0BAA0B,EAC1B,0BAA0B,EAC1B,wBAAwB,EACxB,2BAA2B,EAC3B,KAAK,oBAAoB,EACzB,KAAK,wBAAwB,EAC7B,KAAK,eAAe,EACpB,KAAK,mBAAmB,EACxB,KAAK,0BAA0B,GAChC,MAAM,4BAA4B,CAAC;AACpC,OAAO,EACL,0BAA0B,EAC1B,oBAAoB,GACrB,MAAM,4BAA4B,CAAC;AACpC,YAAY,EACV,qBAAqB,EACrB,eAAe,EACf,cAAc,GACf,MAAM,aAAa,CAAC;AACrB,OAAO,EACL,4BAA4B,EAC5B,0BAA0B,EAC1B,YAAY,GACb,MAAM,wBAAwB,CAAC;AAChC,OAAO,EACL,mCAAmC,EACnC,eAAe,EACf,yBAAyB,EACzB,4BAA4B,EAC5B,iCAAiC,EACjC,2BAA2B,EAC3B,KAAK,sBAAsB,EAC3B,KAAK,sBAAsB,GAC5B,MAAM,wBAAwB,CAAC;AAChC,OAAO,EACL,uBAAuB,EACvB,gBAAgB,EAChB,mBAAmB,EACnB,yBAAyB,EACzB,0BAA0B,EAC1B,mBAAmB,EACnB,KAAK,gBAAgB,EACrB,KAAK,iBAAiB,EACtB,KAAK,mBAAmB,GACzB,MAAM,sBAAsB,CAAC;AAC9B,OAAO,EACL,eAAe,EACf,qBAAqB,EACrB,2BAA2B,EAC3B,6BAA6B,EAC7B,qBAAqB,EACrB,6BAA6B,EAC7B,KAAK,qBAAqB,EAC1B,KAAK,0BAA0B,GAChC,MAAM,yBAAyB,CAAC;AACjC,OAAO,EACL,qBAAqB,EACrB,uBAAuB,EACvB,KAAK,eAAe,EACpB,KAAK,eAAe,EACpB,KAAK,iBAAiB,EACtB,KAAK,wBAAwB,EAC7B,KAAK,oBAAoB,EACzB,KAAK,oBAAoB,GAC1B,MAAM,0BAA0B,CAAC;AAClC,OAAO,EAAE,qBAAqB,EAAE,MAAM,uBAAuB,CAAC;AAC9D,OAAO,EAAE,kBAAkB,EAAE,MAAM,oBAAoB,CAAC;AAExD,YAAY,EACV,kBAAkB,EAClB,qBAAqB,EACrB,mBAAmB,EACnB,kBAAkB,GACnB,MAAM,wBAAwB,CAAC;AAChC,OAAO,EACL,sBAAsB,EACtB,yBAAyB,EACzB,KAAK,kBAAkB,EACvB,KAAK,qBAAqB,GAC3B,MAAM,oBAAoB,CAAC;AAC5B,YAAY,EACV,wBAAwB,EACxB,qBAAqB,EACrB,wBAAwB,EACxB,sBAAsB,EACtB,sBAAsB,GACvB,MAAM,+BAA+B,CAAC;AACvC,OAAO,EACL,8BAA8B,EAC9B,uBAAuB,EACvB,2BAA2B,EAC3B,gCAAgC,EAChC,2BAA2B,EAC3B,2BAA2B,EAC3B,uBAAuB,EACvB,uBAAuB,GACxB,MAAM,+BAA+B,CAAC;AACvC,OAAO,EACL,wBAAwB,EACxB,mBAAmB,EACnB,4BAA4B,EAC5B,uBAAuB,EACvB,KAAK,cAAc,EACnB,KAAK,iBAAiB,EACtB,KAAK,6BAA6B,EAClC,KAAK,kBAAkB,GACxB,MAAM,yBAAyB,CAAC;AAEjC,YAAY,EAAE,YAAY,EAAE,UAAU,EAAE,MAAM,oBAAoB,CAAC;AACnE,YAAY,EAAE,aAAa,EAAE,MAAM,qBAAqB,CAAC;AACzD,OAAO,EACL,wBAAwB,EACxB,gCAAgC,GACjC,MAAM,qBAAqB,CAAC;AAC7B,OAAO,EACL,4BAA4B,EAC5B,KAAK,gCAAgC,EACrC,KAAK,eAAe,EACpB,KAAK,eAAe,GACrB,MAAM,4BAA4B,CAAC;AACpC,YAAY,EACV,2BAA2B,EAC3B,6BAA6B,GAC9B,MAAM,gCAAgC,CAAC;AACxC,OAAO,EAAE,sBAAsB,EAAE,MAAM,gCAAgC,CAAC;AACxE,OAAO,EACL,mBAAmB,EACnB,KAAK,kBAAkB,GACxB,MAAM,0BAA0B,CAAC;AAClC,OAAO,EACL,iBAAiB,EACjB,wBAAwB,EACxB,qBAAqB,EACrB,aAAa,EACb,KAAK,gBAAgB,EACrB,KAAK,eAAe,GACrB,MAAM,iBAAiB,CAAC;AACzB,OAAO,EACL,wBAAwB,EACxB,mBAAmB,EACnB,KAAK,kBAAkB,GACxB,MAAM,qBAAqB,CAAC"}
|
|
@@ -31,6 +31,8 @@ export async function connectBrowserSession(options) {
|
|
|
31
31
|
onSessionError: options.onSessionError,
|
|
32
32
|
onControlMessage: options.onControlMessage,
|
|
33
33
|
onAgentState: options.onAgentState,
|
|
34
|
+
spokenTextReveal: options.spokenTextReveal,
|
|
35
|
+
onAgentText: options.onAgentText,
|
|
34
36
|
onAgentInterrupted: options.onAgentInterrupted,
|
|
35
37
|
onBinaryMessage: options.onBinaryMessage,
|
|
36
38
|
onSyncBinaryMessage: options.onSyncBinaryMessage,
|
|
@@ -60,6 +62,7 @@ export { CustomerContextTooLargeError, MAX_CUSTOMER_CONTEXT_BYTES, startSession,
|
|
|
60
62
|
export { DEFAULT_SESSION_API_RETRY_DELAYS_MS, fetchSessionApi, isRetryableSessionApiBody, isRetryableSessionApiFailure, isRetryableSessionApiNetworkError, isRetryableSessionApiStatus, } from "./session-api-retry.js";
|
|
61
63
|
export { createLocalSessionError, emitSessionError, isSessionErrorEvent, LOCAL_SESSION_ERROR_CODES, REMOTE_SESSION_ERROR_CODES, SESSION_ERROR_CODES, } from "../session-errors.js";
|
|
62
64
|
export { ConnectionError, createConnectionError, formatConnectionErrorSource, getRootConnectionErrorHandler, reportConnectionError, setRootConnectionErrorHandler, } from "../connection-errors.js";
|
|
65
|
+
export { createAgentTextReveal, createSpokenChatCaption, } from "../spoken-text-reveal.js";
|
|
63
66
|
export { attachAudioVisualizer } from "./audio-visualizer.js";
|
|
64
67
|
export { createDebugConsole } from "./debug-console.js";
|
|
65
68
|
export { BrowserSessionModeType, ProvisionedRunnerModeType, } from "./session-modes.js";
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"browser-session.js","sourceRoot":"","sources":["../../src/browser/browser-session.ts"],"names":[],"mappings":"AAGA,OAAO,EACL,0BAA0B,GAG3B,MAAM,4BAA4B,CAAC;AACpC,OAAO,EACL,sBAAsB,EACtB,yBAAyB,GAE1B,MAAM,oBAAoB,CAAC;
|
|
1
|
+
{"version":3,"file":"browser-session.js","sourceRoot":"","sources":["../../src/browser/browser-session.ts"],"names":[],"mappings":"AAGA,OAAO,EACL,0BAA0B,GAG3B,MAAM,4BAA4B,CAAC;AACpC,OAAO,EACL,sBAAsB,EACtB,yBAAyB,GAE1B,MAAM,oBAAoB,CAAC;AA4C5B,MAAM,CAAC,KAAK,UAAU,qBAAqB,CACzC,OAAqC;IAErC,MAAM,UAAU,GACd,OAAO,CAAC,WAAW,CAAC,IAAI,KAAK,yBAAyB,CAAC,IAAI;QACzD,CAAC,CAAC,sBAAsB,CAAC,IAAI;QAC7B,CAAC,CAAC,OAAO,CAAC,WAAW,CAAC,IAAI,CAAC;IAC/B,MAAM,YAAY,GAAG,OAAO,CAAC,IAAI,IAAI,UAAU,CAAC;IAChD,IACE,UAAU,KAAK,sBAAsB,CAAC,IAAI;QAC1C,YAAY,KAAK,sBAAsB,CAAC,IAAI,EAC5C,CAAC;QACD,MAAM,IAAI,KAAK,CAAC,qDAAqD,CAAC,CAAC;IACzE,CAAC;IACD,IACE,UAAU,KAAK,sBAAsB,CAAC,KAAK;QAC3C,YAAY,KAAK,sBAAsB,CAAC,KAAK,EAC7C,CAAC;QACD,MAAM,IAAI,KAAK,CAAC,sDAAsD,CAAC,CAAC;IAC1E,CAAC;IACD,MAAM,UAAU,GACd,YAAY,KAAK,sBAAsB,CAAC,KAAK;QAC7C,YAAY,KAAK,sBAAsB,CAAC,YAAY,CAAC;IACvD,MAAM,SAAS,GACb,YAAY,KAAK,sBAAsB,CAAC,YAAY;QAClD,CAAC,CAAC,gBAAgB;QAClB,CAAC,CAAC,YAAY,KAAK,sBAAsB,CAAC,KAAK;YAC7C,CAAC,CAAC,OAAO;YACT,CAAC,CAAC,MAAM,CAAC;IACf,MAAM,OAAO,GAAG,MAAM,0BAA0B,CAAC;QAC/C,WAAW,EAAE,OAAO,CAAC,WAAW;QAChC,MAAM,EAAE,OAAO,CAAC,MAAM;QACtB,UAAU;QACV,SAAS;QACT,YAAY,EAAE,OAAO,CAAC,YAAY;QAClC,YAAY,EAAE,OAAO,CAAC,YAAY;QAClC,eAAe,EAAE,OAAO,CAAC,eAAe;QACxC,cAAc,EAAE,OAAO,CAAC,cAAc;QACtC,gBAAgB,EAAE,OAAO,CAAC,gBAAgB;QAC1C,YAAY,EAAE,OAAO,CAAC,YAAY;QAClC,gBAAgB,EAAE,OAAO,CAAC,gBAAgB;QAC1C,WAAW,EAAE,OAAO,CAAC,WAAW;QAChC,kBAAkB,EAAE,OAAO,CAAC,kBAAkB;QAC9C,eAAe,EAAE,OAAO,CAAC,eAAe;QACxC,mBAAmB,EAAE,OAAO,CAAC,mBAAmB;QAChD,iBAAiB,EAAE,OAAO,CAAC,iBAAiB;QAC5C,eAAe,EAAE,OAAO,CAAC,eAAe;QACxC,wBAAwB,EAAE,OAAO,CAAC,wBAAwB;QAC1D,iBAAiB,EAAE,OAAO,CAAC,iBAAiB;QAC5C,sBAAsB,EAAE,OAAO,CAAC,sBAAsB;QACtD,aAAa,EAAE,OAAO,CAAC,aAAa;QACpC,cAAc,EAAE,OAAO,CAAC,cAAc;QACtC,aAAa,EAAE,OAAO,CAAC,aAAa;QACpC,kBAAkB,EAAE,OAAO,CAAC,kBAAkB;QAC9C,kBAAkB,EAAE,OAAO,CAAC,kBAAkB;QAC9C,iBAAiB,EAAE,OAAO,CAAC,iBAAiB;QAC5C,kBAAkB,EAAE,UAAU,CAAC,CAAC,CAAC,OAAO,CAAC,kBAAkB,CAAC,CAAC,CAAC,SAAS;QACvE,eAAe,EAAE,OAAO,CAAC,eAAe;KACzC,CAAC,CAAC;IAEH,OAAO,EAAE,GAAG,OAAO,EAAE,IAAI,EAAE,YAAY,EAAE,CAAC;AAC5C,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,kBAAkB,CACtC,OAAuD;IAEvD,OAAO,0BAA0B,CAAC,EAAE,GAAG,OAAO,EAAE,UAAU,EAAE,KAAK,EAAE,CAAC,CAAC;AACvE,CAAC;AAMD,OAAO,EACL,yBAAyB,EACzB,6BAA6B,GAC9B,MAAM,2BAA2B,CAAC;AACnC,OAAO,EACL,0BAA0B,EAC1B,0BAA0B,EAC1B,wBAAwB,EACxB,2BAA2B,GAM5B,MAAM,4BAA4B,CAAC;AACpC,OAAO,EACL,0BAA0B,EAC1B,oBAAoB,GACrB,MAAM,4BAA4B,CAAC;AAMpC,OAAO,EACL,4BAA4B,EAC5B,0BAA0B,EAC1B,YAAY,GACb,MAAM,wBAAwB,CAAC;AAChC,OAAO,EACL,mCAAmC,EACnC,eAAe,EACf,yBAAyB,EACzB,4BAA4B,EAC5B,iCAAiC,EACjC,2BAA2B,GAG5B,MAAM,wBAAwB,CAAC;AAChC,OAAO,EACL,uBAAuB,EACvB,gBAAgB,EAChB,mBAAmB,EACnB,yBAAyB,EACzB,0BAA0B,EAC1B,mBAAmB,GAIpB,MAAM,sBAAsB,CAAC;AAC9B,OAAO,EACL,eAAe,EACf,qBAAqB,EACrB,2BAA2B,EAC3B,6BAA6B,EAC7B,qBAAqB,EACrB,6BAA6B,GAG9B,MAAM,yBAAyB,CAAC;AACjC,OAAO,EACL,qBAAqB,EACrB,uBAAuB,GAOxB,MAAM,0BAA0B,CAAC;AAClC,OAAO,EAAE,qBAAqB,EAAE,MAAM,uBAAuB,CAAC;AAC9D,OAAO,EAAE,kBAAkB,EAAE,MAAM,oBAAoB,CAAC;AAQxD,OAAO,EACL,sBAAsB,EACtB,yBAAyB,GAG1B,MAAM,oBAAoB,CAAC;AAQ5B,OAAO,EACL,8BAA8B,EAC9B,uBAAuB,EACvB,2BAA2B,EAC3B,gCAAgC,EAChC,2BAA2B,EAC3B,2BAA2B,EAC3B,uBAAuB,EACvB,uBAAuB,GACxB,MAAM,+BAA+B,CAAC;AACvC,OAAO,EACL,wBAAwB,EACxB,mBAAmB,EACnB,4BAA4B,EAC5B,uBAAuB,GAKxB,MAAM,yBAAyB,CAAC;AAIjC,OAAO,EACL,wBAAwB,EACxB,gCAAgC,GACjC,MAAM,qBAAqB,CAAC;AAC7B,OAAO,EACL,4BAA4B,GAI7B,MAAM,4BAA4B,CAAC;AAKpC,OAAO,EAAE,sBAAsB,EAAE,MAAM,gCAAgC,CAAC;AACxE,OAAO,EACL,mBAAmB,GAEpB,MAAM,0BAA0B,CAAC;AAClC,OAAO,EACL,iBAAiB,EACjB,wBAAwB,EACxB,qBAAqB,EACrB,aAAa,GAGd,MAAM,iBAAiB,CAAC;AACzB,OAAO,EACL,wBAAwB,EACxB,mBAAmB,GAEpB,MAAM,qBAAqB,CAAC"}
|
|
@@ -1,3 +1,4 @@
|
|
|
1
|
+
import { type AgentTextUpdate, type SpokenTextRevealMode } from "../spoken-text-reveal.js";
|
|
1
2
|
import { type SessionErrorHandler } from "../session-errors.js";
|
|
2
3
|
import type { SessionCredentials } from "./session-provision.js";
|
|
3
4
|
import type { DebugConsole } from "./debug-console.js";
|
|
@@ -99,6 +100,18 @@ export type BrowserVoiceSessionOptions = {
|
|
|
99
100
|
onSessionError?: SessionErrorHandler;
|
|
100
101
|
/** Fired for JSON messages on voice-control (e.g. speech_event). */
|
|
101
102
|
onControlMessage?: (payload: Record<string, unknown>) => void;
|
|
103
|
+
/**
|
|
104
|
+
* How spoken agent text (`assistant_*` with `spoken: true`) is reported to {@link onAgentText}.
|
|
105
|
+
* `playback` (default) reveals it word by word while the agent speaks; `immediate` reports each
|
|
106
|
+
* sentence as soon as the agent hands it to speech.
|
|
107
|
+
*/
|
|
108
|
+
spokenTextReveal?: SpokenTextRevealMode;
|
|
109
|
+
/**
|
|
110
|
+
* The only callback that reports agent text. One call per visible change, keyed by `turnId`;
|
|
111
|
+
* `text` is the whole turn so far. The typed turn's trailing `chat_reply` is not reported.
|
|
112
|
+
* `onControlMessage` still receives every raw message.
|
|
113
|
+
*/
|
|
114
|
+
onAgentText?: (update: AgentTextUpdate) => void;
|
|
102
115
|
/** Fired when the agent reports what it is doing (`agent_state` on voice-control). Older agents send none. */
|
|
103
116
|
onAgentState?: (event: AgentStateEvent) => void;
|
|
104
117
|
/** Fired when the caller cut the agent off (`agent_interrupted` on voice-control). Older agents send none. */
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"browser-voice-session.d.ts","sourceRoot":"","sources":["../../src/browser/browser-voice-session.ts"],"names":[],"mappings":"AAMA,OAAO,EAML,KAAK,mBAAmB,EACzB,MAAM,sBAAsB,CAAC;AAC9B,OAAO,KAAK,EAAE,kBAAkB,EAAE,MAAM,wBAAwB,CAAC;AACjE,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,oBAAoB,CAAC;AAKvD,OAAO,KAAK,EAAE,qBAAqB,EAAE,eAAe,EAAE,MAAM,aAAa,CAAC;AAC1E,OAAO,EAYL,KAAK,sBAAsB,EAC3B,KAAK,sBAAsB,EAC5B,MAAM,+BAA+B,CAAC;AACvC,OAAO,EAEL,KAAK,iBAAiB,EACvB,MAAM,yBAAyB,CAAC;AACjC,OAAO,EAEL,KAAK,aAAa,EACnB,MAAM,qBAAqB,CAAC;AAC7B,OAAO,EAEL,KAAK,eAAe,EACrB,MAAM,4BAA4B,CAAC;AACpC,OAAO,EAGL,KAAK,6BAA6B,EACnC,MAAM,gCAAgC,CAAC;AAOxC,OAAO,EAGL,KAAK,eAAe,EACrB,MAAM,iBAAiB,CAAC;AACzB,OAAO,EAGL,KAAK,kBAAkB,EACxB,MAAM,qBAAqB,CAAC;AAE7B,YAAY,EAAE,kBAAkB,EAAE,MAAM,qBAAqB,CAAC;AAU9D;;;GAGG;AACH,wBAAgB,sBAAsB,CACpC,GAAG,EAAE,yBAAyB,GAAG,IAAI,GAAG,SAAS,GAChD,OAAO,CAGT;AAED,yFAAyF;AACzF,wBAAgB,wBAAwB,CACtC,IAAI,EAAE,qBAAqB,GAAG,yBAAyB,GAAG,IAAI,GAC7D,yBAAyB,GAAG,IAAI,CASlC;AA2BD;;;;GAIG;AACH,eAAO,MAAM,2BAA2B,QAAS,CAAC;AAGlD;;;;;GAKG;AACH,eAAO,MAAM,oCAAoC,OAAQ,CAAC;AAW1D;;;GAGG;AACH,eAAO,MAAM,2BAA2B,OAAQ,CAAC;AAcjD;;;GAGG;AACH,eAAO,MAAM,0BAA0B,kBAAkB,CAAC;AAI1D,eAAO,MAAM,0BAA0B,uBAAuB,CAAC;AAC/D,eAAO,MAAM,2BAA2B,kBAAkB,CAAC;AAC3D,kFAAkF;AAClF,eAAO,MAAM,wBAAwB,oBAAoB,CAAC;AAE1D,MAAM,MAAM,eAAe,GAAG,SAAS,GAAG,MAAM,CAAC;AAEjD,6DAA6D;AAC7D,MAAM,MAAM,oBAAoB,GAAG,CAAC,IAAI,EAAE,WAAW,KAAK,IAAI,CAAC;AAC/D,+DAA+D;AAC/D,MAAM,MAAM,wBAAwB,GAAG,CAAC,IAAI,EAAE,WAAW,KAAK,IAAI,CAAC;AAEnE,MAAM,MAAM,eAAe,GAAG,cAAc,GAAG,aAAa,CAAC;AAE7D,qFAAqF;AACrF,MAAM,MAAM,yBAAyB,GAAG;IACtC,MAAM,EAAE,MAAM,CAAC;IACf,6EAA6E;IAC7E,kBAAkB,CAAC,EAAE,KAAK,GAAG,OAAO,CAAC;CACtC,CAAC;AAEF,MAAM,MAAM,0BAA0B,GAAG;IACvC,WAAW,EAAE,kBAAkB,CAAC;IAChC;;;;;;;;OAQG;IACH,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,UAAU,CAAC,EAAE,OAAO,CAAC;IACrB;;;;OAIG;IACH,oBAAoB,CAAC,EAAE,OAAO,CAAC;IAC/B,gFAAgF;IAChF,kBAAkB,CAAC,EAAE,MAAM,CAAC;IAC5B;;;OAGG;IACH,OAAO,CAAC,EAAE,QAAQ,GAAG,UAAU,CAAC;IAChC,YAAY,CAAC,EAAE,gBAAgB,CAAC;IAChC,YAAY,CAAC,EAAE,YAAY,CAAC;IAC5B,4DAA4D;IAC5D,OAAO,CAAC,EAAE,aAAa,CAAC;IACxB;;;OAGG;IACH,kBAAkB,CAAC,EAAE,KAAK,GAAG,OAAO,CAAC;IACrC,yFAAyF;IACzF,iBAAiB,CAAC,EAAE,6BAA6B,CAAC;IAClD,iEAAiE;IACjE,eAAe,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;IAC1C,6EAA6E;IAC7E,cAAc,CAAC,EAAE,mBAAmB,CAAC;IACrC,oEAAoE;IACpE,gBAAgB,CAAC,EAAE,CAAC,OAAO,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,KAAK,IAAI,CAAC;IAC9D,8GAA8G;IAC9G,YAAY,CAAC,EAAE,CAAC,KAAK,EAAE,eAAe,KAAK,IAAI,CAAC;IAChD,8GAA8G;IAC9G,kBAAkB,CAAC,EAAE,CAAC,KAAK,EAAE,qBAAqB,KAAK,IAAI,CAAC;IAC5D,gDAAgD;IAChD,eAAe,CAAC,EAAE,oBAAoB,CAAC;IACvC,kDAAkD;IAClD,mBAAmB,CAAC,EAAE,wBAAwB,CAAC;IAC/C;;;OAGG;IACH,iBAAiB,CAAC,EAAE,CAAC,KAAK,EAAE,gBAAgB,KAAK,IAAI,CAAC;IACtD;;;;OAIG;IACH,eAAe,CAAC,EAAE,eAAe,CAAC;IAClC;;;;;OAKG;IACH,iBAAiB,CAAC,EAAE,MAAM,CAAC;IAC3B;;;;;OAKG;IACH,yBAAyB,CAAC,EAAE,MAAM,CAAC;IACnC;;;OAGG;IACH,wBAAwB,CAAC,EAAE,OAAO,CAAC;QACjC,WAAW,EAAE,MAAM,CAAC;QACpB,QAAQ,EAAE,MAAM,CAAC;QACjB,IAAI,EAAE,MAAM,CAAC;QACb,YAAY,EAAE,MAAM,CAAC;KACtB,CAAC,CAAC;IACH;;;;;;OAMG;IACH,wBAAwB,CAAC,EAAE,MAAM,CAAC;IAClC;;;;OAIG;IACH,sBAAsB,CAAC,EAAE,MAAM,CAAC;IAChC;;;;OAIG;IACH,0BAA0B,CAAC,EAAE,MAAM,CAAC;IACpC,gFAAgF;IAChF,aAAa,CAAC,EAAE,CAAC,OAAO,EAAE,MAAM,EAAE,IAAI,CAAC,EAAE,yBAAyB,KAAK,IAAI,CAAC;IAC5E,cAAc,CAAC,EAAE,CAAC,OAAO,EAAE,MAAM,EAAE,IAAI,CAAC,EAAE,yBAAyB,KAAK,IAAI,CAAC;IAC7E;;;OAGG;IACH,aAAa,CAAC,EAAE,CAAC,OAAO,EAAE,MAAM,EAAE,IAAI,CAAC,EAAE,yBAAyB,KAAK,IAAI,CAAC;IAC5E;;;;OAIG;IACH,SAAS,CAAC,EAAE,sBAAsB,CAAC;IACnC,wFAAwF;IACxF,kBAAkB,CAAC,EAAE,CAAC,MAAM,EAAE,sBAAsB,KAAK,IAAI,CAAC;IAC9D,mFAAmF;IACnF,eAAe,CAAC,EAAE,CAAC,KAAK,EAAE,kBAAkB,KAAK,IAAI,CAAC;CACvD,CAAC;AAEF,MAAM,MAAM,mBAAmB,GAAG;IAChC,MAAM,EAAE,MAAM,CAAC;IACf;;;;OAIG;IACH,UAAU,EAAE,MAAM,IAAI,CAAC;IACvB;;;OAGG;IACH,eAAe,EAAE,MAAM,OAAO,CAAC,eAAe,CAAC,CAAC;IAChD,wFAAwF;IACxF,eAAe,EAAE,CAAC,MAAM,CAAC,EAAE,MAAM,KAAK,IAAI,CAAC;IAC3C,SAAS,EAAE,CAAC,IAAI,EAAE,MAAM,KAAK,IAAI,CAAC;IAClC,QAAQ,EAAE,CAAC,IAAI,EAAE,MAAM,KAAK,IAAI,CAAC;IACjC,uEAAuE;IACvE,WAAW,EAAE,CAAC,OAAO,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,KAAK,IAAI,CAAC;IACxD,4CAA4C;IAC5C,UAAU,EAAE,CAAC,IAAI,EAAE,WAAW,GAAG,UAAU,KAAK,IAAI,CAAC;IACrD,2EAA2E;IAC3E,cAAc,EAAE,CAAC,IAAI,EAAE,WAAW,GAAG,UAAU,KAAK,IAAI,CAAC;IACzD,YAAY,EAAE,MAAM,WAAW,GAAG,IAAI,CAAC;IACvC,kFAAkF;IAClF,qBAAqB,EAAE,MAAM,OAAO,CAAC;QAAE,QAAQ,EAAE,MAAM,CAAC;QAAC,KAAK,EAAE,MAAM,CAAA;KAAE,EAAE,CAAC,CAAC;IAC5E,mFAAmF;IACnF,qBAAqB,EAAE,MAAM,MAAM,GAAG,IAAI,CAAC;IAC3C,kBAAkB,EAAE,MAAM,eAAe,CAAC;IAC1C;;;OAGG;IACH,mBAAmB,EAAE,CAAC,QAAQ,EAAE,MAAM,GAAG,IAAI,KAAK,OAAO,CAAC,IAAI,CAAC,CAAC;IAChE;;;OAGG;IACH,uBAAuB,EAAE,MAAM,OAAO,CAAC,OAAO,CAAC,CAAC;IAChD,oFAAoF;IACpF,qBAAqB,EAAE,MAAM,kBAAkB,CAAC;IAChD;;;OAGG;IACH,mBAAmB,EAAE,MAAM,OAAO,CAAC,OAAO,CAAC,CAAC;IAC5C;;;;;;OAMG;IACH,gBAAgB,EAAE,CAAC,SAAS,CAAC,EAAE,MAAM,KAAK,OAAO,CAAC,IAAI,CAAC,CAAC;IACxD,kBAAkB,EAAE,MAAM,sBAAsB,GAAG,KAAK,CAAC;IACzD,mBAAmB,EAAE,MAAM,sBAAsB,CAAC;IAClD,gEAAgE;IAChE,oBAAoB,EAAE,MAAM,OAAO,CAAC,iBAAiB,GAAG,IAAI,CAAC,CAAC;IAC9D,2FAA2F;IAC3F,SAAS,EAAE,MAAM,OAAO,CAAC,IAAI,CAAC,CAAC;IAC/B;;;OAGG;IACH,2BAA2B,EAAE,MAAM,IAAI,CAAC;CACzC,CAAC;AA8EF,wBAAsB,0BAA0B,CAC9C,OAAO,EAAE,0BAA0B,GAClC,OAAO,CAAC,mBAAmB,CAAC,
|
|
1
|
+
{"version":3,"file":"browser-voice-session.d.ts","sourceRoot":"","sources":["../../src/browser/browser-voice-session.ts"],"names":[],"mappings":"AAMA,OAAO,EAGL,KAAK,eAAe,EACpB,KAAK,oBAAoB,EAC1B,MAAM,0BAA0B,CAAC;AAClC,OAAO,EAML,KAAK,mBAAmB,EACzB,MAAM,sBAAsB,CAAC;AAC9B,OAAO,KAAK,EAAE,kBAAkB,EAAE,MAAM,wBAAwB,CAAC;AACjE,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,oBAAoB,CAAC;AAKvD,OAAO,KAAK,EAAE,qBAAqB,EAAE,eAAe,EAAE,MAAM,aAAa,CAAC;AAC1E,OAAO,EAYL,KAAK,sBAAsB,EAC3B,KAAK,sBAAsB,EAC5B,MAAM,+BAA+B,CAAC;AACvC,OAAO,EAEL,KAAK,iBAAiB,EACvB,MAAM,yBAAyB,CAAC;AACjC,OAAO,EAEL,KAAK,aAAa,EACnB,MAAM,qBAAqB,CAAC;AAC7B,OAAO,EAEL,KAAK,eAAe,EACrB,MAAM,4BAA4B,CAAC;AACpC,OAAO,EAGL,KAAK,6BAA6B,EACnC,MAAM,gCAAgC,CAAC;AAOxC,OAAO,EAGL,KAAK,eAAe,EACrB,MAAM,iBAAiB,CAAC;AACzB,OAAO,EAGL,KAAK,kBAAkB,EACxB,MAAM,qBAAqB,CAAC;AAE7B,YAAY,EAAE,kBAAkB,EAAE,MAAM,qBAAqB,CAAC;AAU9D;;;GAGG;AACH,wBAAgB,sBAAsB,CACpC,GAAG,EAAE,yBAAyB,GAAG,IAAI,GAAG,SAAS,GAChD,OAAO,CAGT;AAED,yFAAyF;AACzF,wBAAgB,wBAAwB,CACtC,IAAI,EAAE,qBAAqB,GAAG,yBAAyB,GAAG,IAAI,GAC7D,yBAAyB,GAAG,IAAI,CASlC;AA2BD;;;;GAIG;AACH,eAAO,MAAM,2BAA2B,QAAS,CAAC;AAGlD;;;;;GAKG;AACH,eAAO,MAAM,oCAAoC,OAAQ,CAAC;AAW1D;;;GAGG;AACH,eAAO,MAAM,2BAA2B,OAAQ,CAAC;AAcjD;;;GAGG;AACH,eAAO,MAAM,0BAA0B,kBAAkB,CAAC;AAI1D,eAAO,MAAM,0BAA0B,uBAAuB,CAAC;AAC/D,eAAO,MAAM,2BAA2B,kBAAkB,CAAC;AAC3D,kFAAkF;AAClF,eAAO,MAAM,wBAAwB,oBAAoB,CAAC;AAE1D,MAAM,MAAM,eAAe,GAAG,SAAS,GAAG,MAAM,CAAC;AAEjD,6DAA6D;AAC7D,MAAM,MAAM,oBAAoB,GAAG,CAAC,IAAI,EAAE,WAAW,KAAK,IAAI,CAAC;AAC/D,+DAA+D;AAC/D,MAAM,MAAM,wBAAwB,GAAG,CAAC,IAAI,EAAE,WAAW,KAAK,IAAI,CAAC;AAEnE,MAAM,MAAM,eAAe,GAAG,cAAc,GAAG,aAAa,CAAC;AAE7D,qFAAqF;AACrF,MAAM,MAAM,yBAAyB,GAAG;IACtC,MAAM,EAAE,MAAM,CAAC;IACf,6EAA6E;IAC7E,kBAAkB,CAAC,EAAE,KAAK,GAAG,OAAO,CAAC;CACtC,CAAC;AAEF,MAAM,MAAM,0BAA0B,GAAG;IACvC,WAAW,EAAE,kBAAkB,CAAC;IAChC;;;;;;;;OAQG;IACH,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,UAAU,CAAC,EAAE,OAAO,CAAC;IACrB;;;;OAIG;IACH,oBAAoB,CAAC,EAAE,OAAO,CAAC;IAC/B,gFAAgF;IAChF,kBAAkB,CAAC,EAAE,MAAM,CAAC;IAC5B;;;OAGG;IACH,OAAO,CAAC,EAAE,QAAQ,GAAG,UAAU,CAAC;IAChC,YAAY,CAAC,EAAE,gBAAgB,CAAC;IAChC,YAAY,CAAC,EAAE,YAAY,CAAC;IAC5B,4DAA4D;IAC5D,OAAO,CAAC,EAAE,aAAa,CAAC;IACxB;;;OAGG;IACH,kBAAkB,CAAC,EAAE,KAAK,GAAG,OAAO,CAAC;IACrC,yFAAyF;IACzF,iBAAiB,CAAC,EAAE,6BAA6B,CAAC;IAClD,iEAAiE;IACjE,eAAe,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;IAC1C,6EAA6E;IAC7E,cAAc,CAAC,EAAE,mBAAmB,CAAC;IACrC,oEAAoE;IACpE,gBAAgB,CAAC,EAAE,CAAC,OAAO,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,KAAK,IAAI,CAAC;IAC9D;;;;OAIG;IACH,gBAAgB,CAAC,EAAE,oBAAoB,CAAC;IACxC;;;;OAIG;IACH,WAAW,CAAC,EAAE,CAAC,MAAM,EAAE,eAAe,KAAK,IAAI,CAAC;IAChD,8GAA8G;IAC9G,YAAY,CAAC,EAAE,CAAC,KAAK,EAAE,eAAe,KAAK,IAAI,CAAC;IAChD,8GAA8G;IAC9G,kBAAkB,CAAC,EAAE,CAAC,KAAK,EAAE,qBAAqB,KAAK,IAAI,CAAC;IAC5D,gDAAgD;IAChD,eAAe,CAAC,EAAE,oBAAoB,CAAC;IACvC,kDAAkD;IAClD,mBAAmB,CAAC,EAAE,wBAAwB,CAAC;IAC/C;;;OAGG;IACH,iBAAiB,CAAC,EAAE,CAAC,KAAK,EAAE,gBAAgB,KAAK,IAAI,CAAC;IACtD;;;;OAIG;IACH,eAAe,CAAC,EAAE,eAAe,CAAC;IAClC;;;;;OAKG;IACH,iBAAiB,CAAC,EAAE,MAAM,CAAC;IAC3B;;;;;OAKG;IACH,yBAAyB,CAAC,EAAE,MAAM,CAAC;IACnC;;;OAGG;IACH,wBAAwB,CAAC,EAAE,OAAO,CAAC;QACjC,WAAW,EAAE,MAAM,CAAC;QACpB,QAAQ,EAAE,MAAM,CAAC;QACjB,IAAI,EAAE,MAAM,CAAC;QACb,YAAY,EAAE,MAAM,CAAC;KACtB,CAAC,CAAC;IACH;;;;;;OAMG;IACH,wBAAwB,CAAC,EAAE,MAAM,CAAC;IAClC;;;;OAIG;IACH,sBAAsB,CAAC,EAAE,MAAM,CAAC;IAChC;;;;OAIG;IACH,0BAA0B,CAAC,EAAE,MAAM,CAAC;IACpC,gFAAgF;IAChF,aAAa,CAAC,EAAE,CAAC,OAAO,EAAE,MAAM,EAAE,IAAI,CAAC,EAAE,yBAAyB,KAAK,IAAI,CAAC;IAC5E,cAAc,CAAC,EAAE,CAAC,OAAO,EAAE,MAAM,EAAE,IAAI,CAAC,EAAE,yBAAyB,KAAK,IAAI,CAAC;IAC7E;;;OAGG;IACH,aAAa,CAAC,EAAE,CAAC,OAAO,EAAE,MAAM,EAAE,IAAI,CAAC,EAAE,yBAAyB,KAAK,IAAI,CAAC;IAC5E;;;;OAIG;IACH,SAAS,CAAC,EAAE,sBAAsB,CAAC;IACnC,wFAAwF;IACxF,kBAAkB,CAAC,EAAE,CAAC,MAAM,EAAE,sBAAsB,KAAK,IAAI,CAAC;IAC9D,mFAAmF;IACnF,eAAe,CAAC,EAAE,CAAC,KAAK,EAAE,kBAAkB,KAAK,IAAI,CAAC;CACvD,CAAC;AAEF,MAAM,MAAM,mBAAmB,GAAG;IAChC,MAAM,EAAE,MAAM,CAAC;IACf;;;;OAIG;IACH,UAAU,EAAE,MAAM,IAAI,CAAC;IACvB;;;OAGG;IACH,eAAe,EAAE,MAAM,OAAO,CAAC,eAAe,CAAC,CAAC;IAChD,wFAAwF;IACxF,eAAe,EAAE,CAAC,MAAM,CAAC,EAAE,MAAM,KAAK,IAAI,CAAC;IAC3C,SAAS,EAAE,CAAC,IAAI,EAAE,MAAM,KAAK,IAAI,CAAC;IAClC,QAAQ,EAAE,CAAC,IAAI,EAAE,MAAM,KAAK,IAAI,CAAC;IACjC,uEAAuE;IACvE,WAAW,EAAE,CAAC,OAAO,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,KAAK,IAAI,CAAC;IACxD,4CAA4C;IAC5C,UAAU,EAAE,CAAC,IAAI,EAAE,WAAW,GAAG,UAAU,KAAK,IAAI,CAAC;IACrD,2EAA2E;IAC3E,cAAc,EAAE,CAAC,IAAI,EAAE,WAAW,GAAG,UAAU,KAAK,IAAI,CAAC;IACzD,YAAY,EAAE,MAAM,WAAW,GAAG,IAAI,CAAC;IACvC,kFAAkF;IAClF,qBAAqB,EAAE,MAAM,OAAO,CAAC;QAAE,QAAQ,EAAE,MAAM,CAAC;QAAC,KAAK,EAAE,MAAM,CAAA;KAAE,EAAE,CAAC,CAAC;IAC5E,mFAAmF;IACnF,qBAAqB,EAAE,MAAM,MAAM,GAAG,IAAI,CAAC;IAC3C,kBAAkB,EAAE,MAAM,eAAe,CAAC;IAC1C;;;OAGG;IACH,mBAAmB,EAAE,CAAC,QAAQ,EAAE,MAAM,GAAG,IAAI,KAAK,OAAO,CAAC,IAAI,CAAC,CAAC;IAChE;;;OAGG;IACH,uBAAuB,EAAE,MAAM,OAAO,CAAC,OAAO,CAAC,CAAC;IAChD,oFAAoF;IACpF,qBAAqB,EAAE,MAAM,kBAAkB,CAAC;IAChD;;;OAGG;IACH,mBAAmB,EAAE,MAAM,OAAO,CAAC,OAAO,CAAC,CAAC;IAC5C;;;;;;OAMG;IACH,gBAAgB,EAAE,CAAC,SAAS,CAAC,EAAE,MAAM,KAAK,OAAO,CAAC,IAAI,CAAC,CAAC;IACxD,kBAAkB,EAAE,MAAM,sBAAsB,GAAG,KAAK,CAAC;IACzD,mBAAmB,EAAE,MAAM,sBAAsB,CAAC;IAClD,gEAAgE;IAChE,oBAAoB,EAAE,MAAM,OAAO,CAAC,iBAAiB,GAAG,IAAI,CAAC,CAAC;IAC9D,2FAA2F;IAC3F,SAAS,EAAE,MAAM,OAAO,CAAC,IAAI,CAAC,CAAC;IAC/B;;;OAGG;IACH,2BAA2B,EAAE,MAAM,IAAI,CAAC;CACzC,CAAC;AA8EF,wBAAsB,0BAA0B,CAC9C,OAAO,EAAE,0BAA0B,GAClC,OAAO,CAAC,mBAAmB,CAAC,CAo7E9B"}
|
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
import { createConnectionError, dispatchConnectionError, } from "../connection-errors.js";
|
|
2
2
|
import { appendJoinToken } from "../resolve-connection.js";
|
|
3
|
+
import { createAgentTextReveal, } from "../spoken-text-reveal.js";
|
|
3
4
|
import { createLocalSessionError, emitSessionError, isSessionErrorEvent, parseLegacyAgentError, } from "../session-errors.js";
|
|
4
5
|
import { parseAgentInterruptedEvent, parseAgentStateEvent, } from "../agent-control-events.js";
|
|
5
6
|
import { buildWebRtcConnectionStatus, CONNECTION_INTERRUPTED_MESSAGE, CONNECTION_LOST_MESSAGE, CONNECTION_RESTORED_MESSAGE, CONNECTION_SESSION_ENDED_MESSAGE, isHalfOpenConnection, formatWebRtcConnectTimeoutMessage, isWebRtcConnectionReady, resolveHalfOpenFailFastMs, resolveReadinessProfile, } from "./webrtc-connection-status.js";
|
|
@@ -194,6 +195,12 @@ export async function connectBrowserVoiceSession(options) {
|
|
|
194
195
|
: null;
|
|
195
196
|
const playbackAudioElement = options.audioElement ?? ownedAudioElement;
|
|
196
197
|
let audioPlaybackState = "idle";
|
|
198
|
+
const agentTextReveal = options.onAgentText
|
|
199
|
+
? createAgentTextReveal({
|
|
200
|
+
mode: options.spokenTextReveal,
|
|
201
|
+
onAgentText: options.onAgentText,
|
|
202
|
+
})
|
|
203
|
+
: null;
|
|
197
204
|
const disposeOwnedAudioElement = () => {
|
|
198
205
|
if (!ownedAudioElement)
|
|
199
206
|
return;
|
|
@@ -552,6 +559,7 @@ export async function connectBrowserVoiceSession(options) {
|
|
|
552
559
|
occurred_at: new Date().toISOString(),
|
|
553
560
|
});
|
|
554
561
|
debug?.info("session", "remote_close", closeCode ?? closeReason ?? "unknown");
|
|
562
|
+
agentTextReveal?.dispose();
|
|
555
563
|
connectionState = "closed";
|
|
556
564
|
if (connectionSnapshot.peerConnectionState !== "closed") {
|
|
557
565
|
updateConnectionSnapshot({
|
|
@@ -563,6 +571,7 @@ export async function connectBrowserVoiceSession(options) {
|
|
|
563
571
|
return;
|
|
564
572
|
}
|
|
565
573
|
options.onControlMessage?.(message);
|
|
574
|
+
agentTextReveal?.handleControlMessage(message);
|
|
566
575
|
dispatchAgentEvents(message);
|
|
567
576
|
};
|
|
568
577
|
const stopDeadPairsProbe = () => {
|
|
@@ -2198,6 +2207,7 @@ export async function connectBrowserVoiceSession(options) {
|
|
|
2198
2207
|
pc = null;
|
|
2199
2208
|
}
|
|
2200
2209
|
ws = null;
|
|
2210
|
+
agentTextReveal?.dispose();
|
|
2201
2211
|
disposeCurrentMic();
|
|
2202
2212
|
disposeOwnedAudioElement();
|
|
2203
2213
|
micRtpSender = null;
|
|
@@ -2303,6 +2313,7 @@ export async function connectBrowserVoiceSession(options) {
|
|
|
2303
2313
|
syncChannel = null;
|
|
2304
2314
|
pc = null;
|
|
2305
2315
|
ws = null;
|
|
2316
|
+
agentTextReveal?.dispose();
|
|
2306
2317
|
disposeCurrentMic();
|
|
2307
2318
|
disposeOwnedAudioElement();
|
|
2308
2319
|
micRtpSender = null;
|