realtime-avatar 0.27.4 → 0.28.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -97,10 +97,11 @@ Recordings may finish processing after a call ends. Use `listRecordings({ sessio
97
97
  them, or refresh `getRecording(recordingId)` while processing. These methods and
98
98
  `getRecordingAccess` require a server key with `recordings:read`.
99
99
 
100
- Save `recordingId`, not a playback URL. Files are retained until `retainedUntil` (30 days by
101
- default); each URL expires at `expiresAt` (up to one hour, capped by retention). Obtain fresh
100
+ Save `recordingId`, not a playback URL. Files are retained until `retainedUntil` (the platform's
101
+ existing default is 36,500 days); each URL expires at `expiresAt` (up to one hour, capped by retention). Obtain fresh
102
102
  access before replaying or seeking after expiry. An expired URL does not delete the file.
103
103
  Treat the URL as private: anyone who has it can play that file until it expires.
104
+ This is the existing platform retention policy, not a new SDK setting or extension.
104
105
 
105
106
  For participant replay, pass every ready `{ recording, url }` asset to `RecordingPlayer`
106
107
  from `realtime-avatar/react`. It accepts combined participant MP4s and separate audio/video
@@ -197,6 +198,51 @@ Two facts follow from that picture and drive everything else:
197
198
 
198
199
  ---
199
200
 
201
+ ## Custom voice (webhook)
202
+
203
+ Speak a call in your own voice. Each sentence she is about to say is POSTed, signed, to an
204
+ endpoint your server hosts, and the audio it streams back is what she says, lip-synced, in
205
+ `avatar` and `voice` mode alike:
206
+
207
+ ```ts
208
+ const call = await rta.startCall({
209
+ avatarId,
210
+ voiceWebhook: { url: "https://api.example.com/voice", secret: process.env.VOICE_WEBHOOK_SECRET!, voiceId: "narrator" },
211
+ });
212
+ ```
213
+
214
+ ```ts
215
+ // POST https://api.example.com/voice
216
+ import { verifyVoiceWebhook } from "realtime-avatar";
217
+
218
+ const raw = Buffer.from(await request.arrayBuffer()); // the RAW bytes, as for transcripts
219
+ const { text, voice_id, output } = await verifyVoiceWebhook(raw, request.headers, process.env.VOICE_WEBHOOK_SECRET!);
220
+ return new Response(await myTts(text, voice_id), { headers: { "content-type": "audio/pcm;rate=24000" } });
221
+ ```
222
+
223
+ - `voiceWebhook` is `{ url, secret, voiceId?, language?, sampleRate?, timeoutMs? }`, derived from
224
+ the platform's `voice_webhook`. It carries a secret, so only the server entries send it; it
225
+ replaces `voice`, and a mint with both is refused with `422`. The URL must be https.
226
+ - Reply `200` with audio, streamed as soon as you have it. `timeoutMs` (default 5000) bounds the
227
+ first audio byte of each attempt and every later gap between chunks. `audio/pcm` (alias
228
+ `audio/raw`) is raw signed 16-bit little-endian mono at `output.sample_rate` (default 24000), or
229
+ name the rate with `audio/pcm;rate=…`; no other raw type is accepted. OpenAI's
230
+ `response_format: "pcm"` is exactly 24 kHz `audio/pcm`. A 2xx with no body (a `204`,
231
+ `Content-Length: 0`, or an empty first chunk) is `voice_webhook_empty_audio`.
232
+ - `verifyVoiceWebhook` throws `RealtimeAvatarWebhookError` with a `code` (`missing_signature`,
233
+ `bad_signature`, `stale_timestamp`, `invalid_body`).
234
+ - A `401` or `403` from your endpoint ends the call. A timeout, an unreachable endpoint, or a
235
+ `408`, `429` or 5xx is retried once, only before any audio has arrived, as a new POST with a new
236
+ `request_id`. Any other failure, or a retry that fails too, drops the rest of that reply: the
237
+ sentences after it are not POSTed, and the next reply tries again. Repeated failures end the
238
+ call with a fatal error carrying the last failure's code.
239
+ - In the client, `onVoiceError` on `useRealtimeSession` and `AvatarCall` fires with
240
+ `{ code, message, status?, fatal, requestId? }`, and `voiceError` holds the last one of this
241
+ call. `fatal` means the call is ending; `onEnded` follows with `failed`.
242
+
243
+ Working endpoints: `apps/quickstart/nextjs-app-router/app/api/voice/route.ts` and `POST /api/voice`
244
+ in `apps/quickstart/python-fastapi/main.py`.
245
+
200
246
  ## Input source on room messages
201
247
 
202
248
  On React and React Native, `session.sendTurn(text)` automatically observes text.
@@ -248,7 +294,7 @@ against its deployed OpenAPI, then run `npm run check`.** Do not hand-edit the g
248
294
 
249
295
  ```ts
250
296
  // calls
251
- rta.startCall({ avatarId, mode?, instructions?, context?, maxSeconds?, video?, recording?, connectionHistory?, contentPolicy?, transcript?, metadata?, llm? })
297
+ rta.startCall({ avatarId, mode?, instructions?, context?, maxSeconds?, video?, recording?, connectionHistory?, contentPolicy?, transcript?, metadata?, llm?, voiceWebhook? })
252
298
  rta.endCall(sessionId, { reason? }) // free an abandoned call's slot; idempotent, never throws
253
299
  rta.leaveQueue(queueTicketId, { reason? }) // give up a queued call's place in line; idempotent, never throws
254
300
 
@@ -286,8 +332,51 @@ rta.iterateSessions({ from, to }) // the same, paging handled
286
332
 
287
333
  // webhooks
288
334
  verifyTranscript(rawBytes, headers, secret)
335
+ verifyVoiceWebhook(rawBytes, headers, secret) // one sentence to speak, typed; see "Custom voice (webhook)"
336
+ ```
337
+
338
+ ### Session page validation
339
+
340
+ `listSessions` parses the complete response using the generated platform contract and
341
+ returns `{ sessions, nextCursor, from, to }`. Missing required fields, invalid statuses,
342
+ invalid billing numbers and unknown page or session fields throw
343
+ the public `RealtimeAvatarError` with a contract-mismatch message, without retaining the
344
+ rejected response or validation details, including when successful response bytes are not JSON.
345
+ They are not coerced into empty pages, failed sessions or null values. Every own metadata key,
346
+ including `__proto__`, is preserved without prototype mutation. Metadata is an open
347
+ record and keeps its contents, and optional contract fields such as `releaseIdentity` are
348
+ preserved when returned. `iterateSessions` applies the same validation on every page;
349
+ cursor forwarding and the served window remain unchanged.
350
+
351
+ ### Optional speech diagnostics
352
+
353
+ ```ts
354
+ for await (const session of rta.iterateSessions({ from, to, speechDiagnostics: true })) {
355
+ const diagnostics = session.speechDiagnostics;
356
+ if (diagnostics?.status === "available") {
357
+ const endToEndMs = diagnostics.latency_ms.e2e?.p50;
358
+ // An absent leg is unknown, not zero.
359
+ } else if (diagnostics?.status === "unavailable") {
360
+ console.log(diagnostics.reason);
361
+ }
362
+ }
289
363
  ```
290
364
 
365
+ `listSessions` accepts the same option. Only `true` sends `X-RTA-Speech-Diagnostics: 1`;
366
+ false and omission send no header. Iteration keeps it on every page without adding a query
367
+ parameter, request body or call metadata. There is no general request-header override.
368
+ The `available` branch carries whole-session dialogue per-leg `latency_ms` statistics
369
+ (`count`, `p50`, `p90`, `max`) and coverage (`first_at`, `last_at`, `lines`). Coverage timestamps
370
+ may be null or log timestamps without a timezone. Missing legs remain absent. These are
371
+ not pooled turn percentiles, query-date-window slices, or browser-audible latency. The
372
+ `unavailable` branch carries a canonical reason, never fabricated healthy zero. Older servers
373
+ may omit `speechDiagnostics` even when requested; preserve that absence. No raw logs or
374
+ warnings are returned. Malformed diagnostics throw the same public `RealtimeAvatarError`.
375
+
376
+ The API owns opt-in limits: at most 50 sessions per page, 65,536 bytes per summary,
377
+ a 5,000 ms total page deadline and concurrency 4. The SDK does not clamp `limit` locally or
378
+ reimplement server budgets; it follows the returned cursor and served window.
379
+
291
380
  ---
292
381
 
293
382
  ## Subpaths
@@ -299,7 +388,7 @@ Server — these hold your API key:
299
388
 
300
389
  | Import | What it is |
301
390
  | --- | --- |
302
- | `realtime-avatar` | `RealtimeAvatar`, `isQueued`, `verifyTranscript`, the two error classes |
391
+ | `realtime-avatar` | `RealtimeAvatar`, `isQueued`, `verifyTranscript`, `verifyVoiceWebhook`, the three error classes |
303
392
  | `realtime-avatar/server` | The same client with no route adapters — 18.8 KB |
304
393
  | `realtime-avatar/nextjs` | `createRealtimeAvatarRoute` — App Router `{ GET, POST }` |
305
394
  | `realtime-avatar/hono` | `realtimeAvatarHono` — Hono, Workers, Bun, Deno |
@@ -1,8 +1,11 @@
1
- import { sessionModeSchema, liveKitSessionReleaseReasonSchema, RealtimeAvatar, ROUTE_TIMEOUT_MS, isQueued, RealtimeAvatarHttpError, RealtimeAvatarError } from './chunk-VVQIYWKT.js';
1
+ import { sessionModeSchema, liveKitSessionReleaseReasonSchema, RealtimeAvatar, ROUTE_TIMEOUT_MS, isQueued, RealtimeAvatarHttpError, RealtimeAvatarError } from './chunk-V7A4NRKC.js';
2
2
  import { z } from 'zod';
3
3
 
4
4
  // ../http-client/src/generated/error-semantics.ts
5
5
  var ERROR_SEMANTICS = {
6
+ "capacity_full": { statuses: [429], message: "All session seats are busy. Keep your queue ticket and retry shortly.", retryable: true },
7
+ "capacity_no_workers": { statuses: [429], message: "No session workers are available. Keep your queue ticket and retry shortly.", retryable: true },
8
+ "capacity_draining": { statuses: [429], message: "Session workers are restarting or draining. Keep your queue ticket and retry shortly.", retryable: true },
6
9
  "bad_request": { statuses: [400], message: "Check the request and try again.", retryable: false },
7
10
  "unauthorized": { statuses: [401], message: "Authentication is required to use Realtime Avatar.", retryable: false },
8
11
  "payment_required": { statuses: [402], message: "Add credits to continue using realtime avatars.", retryable: false },
@@ -31,6 +34,7 @@ var ERROR_SEMANTICS = {
31
34
  "service_warming": { statuses: [503], message: "Realtime Avatar is warming up. Try again in a moment.", retryable: true },
32
35
  "avatar_not_ready": { statuses: [409], message: "This avatar is still being built \u2014 it is usually ready within a minute. Try again then.", retryable: true },
33
36
  "clip_library_not_ready": { statuses: [409], message: "This avatar's motion clips are still rendering. Try again in a minute.", retryable: true },
37
+ "avatar_not_callable": { statuses: [409], message: "This avatar only hosts a live stream and cannot take calls. Choose another avatar.", retryable: false },
34
38
  "recording_unsupported": { statuses: [501], message: "Session recording is not enabled for this deployment.", retryable: false },
35
39
  "recording_requires_new_room": { statuses: [422], message: "Recording requires a new session room. Start a fresh session to record it.", retryable: false },
36
40
  "recording_unavailable": { statuses: [503], message: "Session recording is temporarily unavailable. Try again in a moment.", retryable: true }
@@ -8,6 +8,14 @@ var RealtimeAvatarError = class extends Error {
8
8
  this.name = "RealtimeAvatarError";
9
9
  }
10
10
  };
11
+ var RealtimeAvatarWebhookError = class extends RealtimeAvatarError {
12
+ code;
13
+ constructor(code, message, options) {
14
+ super(message, options);
15
+ this.name = "RealtimeAvatarWebhookError";
16
+ this.code = code;
17
+ }
18
+ };
11
19
  function isRequestId(value) {
12
20
  return typeof value === "string" && /^[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}$/i.test(value);
13
21
  }
@@ -336,6 +344,20 @@ function transcriptionToWire(input) {
336
344
  }
337
345
  var schema03 = z.enum(["minor-sexual-content-v1", "safe-content-v1"]);
338
346
  var contentPolicySchema = schema03;
347
+ var schema04 = z.literal("voice_error");
348
+ var schema111 = z.enum(["voice_webhook_url_not_allowed", "voice_webhook_unauthorized", "voice_webhook_unreachable", "voice_webhook_timeout", "voice_webhook_http_error", "voice_webhook_unsupported_audio", "voice_webhook_empty_audio"]);
349
+ var schema28 = z.string();
350
+ var schema33 = z.int().min(-9007199254740991).max(9007199254740991);
351
+ var schema43 = z.boolean();
352
+ var schema52 = z.object({
353
+ "kind": schema04,
354
+ "code": schema111,
355
+ "message": schema28,
356
+ "status": schema33.optional(),
357
+ "fatal": schema43,
358
+ "request_id": schema28.optional()
359
+ });
360
+ var voiceErrorFrameSchema = schema52;
339
361
 
340
362
  // ../client/src/wire.ts
341
363
  var DEFAULT_AVATAR_ID = "maria";
@@ -627,13 +649,15 @@ var clipAckFrameSchema = z.object({
627
649
  accepted: z.boolean(),
628
650
  reason: z.string().max(64)
629
651
  }).strip();
652
+ var voiceErrorFrameSchema2 = voiceErrorFrameSchema.extend({ code: z.string().min(1) });
630
653
  z.discriminatedUnion("kind", [
631
654
  sessionClockFrameSchema,
632
655
  endingFrameSchema,
633
656
  closingTurnDoneFrameSchema,
634
657
  endedFrameSchema,
635
658
  behaviorStateFrameSchema,
636
- clipAckFrameSchema
659
+ clipAckFrameSchema,
660
+ voiceErrorFrameSchema2
637
661
  ]);
638
662
  var liveKitCapacitySnapshotSchema = z.object({
639
663
  // Placement identity + per-worker session ceiling. These are always present on the
@@ -672,10 +696,169 @@ var liveKitSessionReleaseReasonSchema = z.enum([
672
696
  "manual",
673
697
  "idle_timeout"
674
698
  ]);
699
+ var schema05 = z.url().regex(new RegExp("^https:\\/\\/")).max(500);
700
+ var schema112 = z.string().min(16).max(200);
701
+ var schema29 = z.string().min(1).max(200);
702
+ var schema34 = z.string().min(2).max(35);
703
+ var schema44 = z.literal([8e3, 16e3, 22050, 24e3, 44100, 48e3]);
704
+ var schema53 = z.int().min(1e3).max(15e3);
705
+ var schema62 = z.strictObject({
706
+ "url": schema05,
707
+ "secret": schema112,
708
+ "voice_id": schema29.optional(),
709
+ "language": schema34.optional(),
710
+ "sample_rate": schema44.optional(),
711
+ "timeout_ms": schema53.optional()
712
+ });
713
+ var schema72 = z.literal("voice.synthesize");
714
+ var schema82 = z.literal(1);
715
+ var schema92 = z.string().min(1).max(200);
716
+ var schema102 = z.string();
717
+ var schema113 = z.string().min(1).max(5e3);
718
+ var schema122 = z.string();
719
+ var schema132 = z.null();
720
+ var schema142 = z.union([schema122, schema132]);
721
+ var schema152 = z.string();
722
+ var schema162 = z.union([schema152, schema132]);
723
+ var schema172 = z.literal("pcm_s16le");
724
+ var schema182 = z.literal([8e3, 16e3, 22050, 24e3, 44100, 48e3]);
725
+ var schema192 = z.object({
726
+ "encoding": schema172,
727
+ "sample_rate": schema182,
728
+ "channels": schema82
729
+ });
730
+ var schema202 = z.string().min(1).max(64);
731
+ var schema212 = z.string().max(200);
732
+ var schema222 = z.record(schema202, schema212).refine((value) => Object.keys(value).length <= 16, { message: "Too many keys: at most 16" });
733
+ var schema232 = z.object({
734
+ "type": schema72,
735
+ "version": schema82,
736
+ "request_id": schema92,
737
+ "session_id": schema102,
738
+ "text": schema113,
739
+ "voice_id": schema142,
740
+ "language": schema162,
741
+ "output": schema192,
742
+ "client_metadata": schema222
743
+ });
744
+ var voiceWebhookWireSchema = schema62;
745
+ var voiceWebhookOptionsSchema = z.strictObject({
746
+ url: voiceWebhookWireSchema.shape.url,
747
+ secret: voiceWebhookWireSchema.shape.secret,
748
+ voiceId: voiceWebhookWireSchema.shape.voice_id,
749
+ language: voiceWebhookWireSchema.shape.language,
750
+ sampleRate: voiceWebhookWireSchema.shape.sample_rate,
751
+ timeoutMs: voiceWebhookWireSchema.shape.timeout_ms
752
+ });
753
+ function voiceWebhookToWire(input) {
754
+ const options = voiceWebhookOptionsSchema.parse(input);
755
+ return {
756
+ url: options.url,
757
+ secret: options.secret,
758
+ ...options.voiceId !== void 0 ? { voice_id: options.voiceId } : {},
759
+ ...options.language !== void 0 ? { language: options.language } : {},
760
+ ...options.sampleRate !== void 0 ? { sample_rate: options.sampleRate } : {},
761
+ ...options.timeoutMs !== void 0 ? { timeout_ms: options.timeoutMs } : {}
762
+ };
763
+ }
764
+ var voiceWebhookRequestSchema = schema232;
765
+ var schema06 = z.string().min(8).max(80);
766
+ var schema114 = z.enum(["eou", "llm", "tts", "decision", "hold", "e2e"]);
767
+ var schema210 = z.literal("interrupt_stop");
768
+ var schema35 = z.union([schema114, schema210]);
769
+ var schema45 = z.int().max(9007199254740991).gt(0);
770
+ var schema54 = z.number().min(0);
771
+ var schema63 = z.strictObject({
772
+ "count": schema45,
773
+ "max": schema54,
774
+ "p50": schema54,
775
+ "p90": schema54
776
+ });
777
+ var schema73 = z.partialRecord(schema35, schema63);
778
+ var schema83 = z.literal("available");
779
+ var schema93 = z.literal("dialogue");
780
+ var schema103 = z.string().regex(new RegExp("^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z|([+-](?:[01]\\d|2[0-3]):[0-5]\\d))|(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?)$"));
781
+ var schema115 = z.null();
782
+ var schema123 = z.union([schema103, schema115]);
783
+ var schema133 = z.int().min(0).max(9007199254740991);
784
+ var schema143 = z.strictObject({
785
+ "first_at": schema123,
786
+ "last_at": schema123,
787
+ "lines": schema133
788
+ });
789
+ var schema153 = z.strictObject({
790
+ "latency_ms": schema73,
791
+ "status": schema83,
792
+ "sourceRole": schema93,
793
+ "coverage": schema143
794
+ });
795
+ var schema163 = z.literal("unavailable");
796
+ var schema173 = z.enum(["not_closed", "unconfigured", "missing", "invalid", "oversized", "unscorable", "no_measured_latency", "timeout", "aborted", "storage_unavailable"]);
797
+ var schema183 = z.strictObject({
798
+ "status": schema163,
799
+ "reason": schema173
800
+ });
801
+ var schema193 = z.union([schema153, schema183]);
802
+ var schema203 = z.string().min(1).max(200);
803
+ var schema213 = z.union([schema203, schema115]);
804
+ var schema223 = z.string().regex(new RegExp("^[0-9a-f]{40}$"));
805
+ var schema233 = z.union([schema223, schema115]);
806
+ var schema242 = z.literal("cloudflare-worker-version");
807
+ var schema252 = z.literal("authenticated-worker-heartbeat");
808
+ var schema262 = z.union([schema252, schema115]);
809
+ var schema272 = z.string().min(1).max(160);
810
+ var schema282 = z.union([schema272, schema115]);
811
+ var schema292 = z.union([schema133, schema115]);
812
+ var schema30 = z.iso.datetime({ offset: true }).regex(new RegExp("^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z|([+-](?:[01]\\d|2[0-3]):[0-5]\\d)))$"));
813
+ var schema31 = z.union([schema30, schema115]);
814
+ var schema322 = z.strictObject({
815
+ "deploymentSource": schema242,
816
+ "workerSource": schema262,
817
+ "workerId": schema282,
818
+ "podStartedAtMs": schema292,
819
+ "workerObservedAt": schema31
820
+ });
821
+ var schema332 = z.strictObject({
822
+ "deploymentId": schema213,
823
+ "workerRevision": schema233,
824
+ "provenance": schema322
825
+ });
826
+ var schema342 = z.string();
827
+ var schema352 = z.union([schema342, schema115]);
828
+ var schema36 = z.enum(["reserved", "started", "released", "failed"]);
829
+ var schema37 = z.union([schema54, schema115]);
830
+ var schema38 = z.custom((value) => typeof value === "object" && value !== null && !Array.isArray(value));
831
+ var schema39 = z.strictObject({
832
+ "sessionId": schema06,
833
+ "speechDiagnostics": schema193.optional(),
834
+ "releaseIdentity": schema332.optional(),
835
+ "avatarId": schema352,
836
+ "avatarName": schema352,
837
+ "status": schema36,
838
+ "startedAt": schema352,
839
+ "endedAt": schema352,
840
+ "activeSeconds": schema37,
841
+ "billedCreditMicros": schema292,
842
+ "metadata": schema38,
843
+ "createdAt": schema342
844
+ });
845
+ var schema40 = z.array(schema39);
846
+ var schema41 = z.iso.datetime({ offset: true }).regex(new RegExp("^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$"));
847
+ var schema422 = z.strictObject({
848
+ "data": schema40,
849
+ "nextCursor": schema352,
850
+ "from": schema41,
851
+ "to": schema41
852
+ });
853
+ var listUsageSessionsResponseSchema = schema422;
854
+ function toSessionsPage(page) {
855
+ const { data, ...pagination } = page;
856
+ return { ...pagination, sessions: data };
857
+ }
675
858
 
676
859
  // ../http-client/src/client.ts
677
860
  var DEFAULT_BASE_URL = "https://realtimeavatar.ai/api/v1";
678
- var SDK_VERSION = "0.27.4";
861
+ var SDK_VERSION = "0.28.0";
679
862
  var RealtimeAvatar = class {
680
863
  #apiKey;
681
864
  #baseUrl;
@@ -723,6 +906,7 @@ var RealtimeAvatar = class {
723
906
  }
724
907
  if (options.maxSeconds !== void 0) body.max_session_seconds = Math.floor(options.maxSeconds);
725
908
  if (options.voice !== void 0) body.voice = options.voice;
909
+ if (options.voiceWebhook !== void 0) body.voice_webhook = voiceWebhookToWire(options.voiceWebhook);
726
910
  if (options.metadata !== void 0) body.client_metadata = options.metadata;
727
911
  if (options.recording !== void 0) body.recording = recordingModeSchema.parse(options.recording);
728
912
  if (options.connectionHistory !== void 0) body.connection_history = options.connectionHistory;
@@ -1081,6 +1265,10 @@ var RealtimeAvatar = class {
1081
1265
  *
1082
1266
  * Tagging is optional and nothing degrades without it — you just cannot attribute a
1083
1267
  * session to one of your users. Requires a key with the `usage:read` scope.
1268
+ *
1269
+ * `speechDiagnostics: true` opts into whole-session dialogue per-leg measurements, not
1270
+ * browser-audible latency. Unavailable evidence is explicit; old-server absence stays absent.
1271
+ * The API owns the opt-in page cap and budgets; the SDK forwards the requested limit.
1084
1272
  */
1085
1273
  async listSessions(options = {}) {
1086
1274
  const query = new URLSearchParams();
@@ -1090,8 +1278,22 @@ var RealtimeAvatar = class {
1090
1278
  if (options.cursor) query.set("cursor", options.cursor);
1091
1279
  if (options.endUserId) query.set("endUserId", options.endUserId);
1092
1280
  const suffix = query.size > 0 ? `?${query}` : "";
1093
- const page = await this.#json(await this.#request("GET", `/usage/sessions${suffix}`));
1094
- return toUsageSessionPage(page);
1281
+ const response = await this.#request("GET", `/usage/sessions${suffix}`, {
1282
+ headers: options.speechDiagnostics === true ? { "X-RTA-Speech-Diagnostics": "1" } : void 0
1283
+ });
1284
+ const mismatch = "Usage sessions response did not match the SDK contract";
1285
+ let raw;
1286
+ try {
1287
+ raw = await this.#json(response);
1288
+ } catch (error) {
1289
+ if (error instanceof SyntaxError) throw new RealtimeAvatarError(mismatch);
1290
+ throw error;
1291
+ }
1292
+ const page = listUsageSessionsResponseSchema.safeParse(raw);
1293
+ if (!page.success) {
1294
+ throw new RealtimeAvatarError(mismatch);
1295
+ }
1296
+ return toSessionsPage(page.data);
1095
1297
  }
1096
1298
  /**
1097
1299
  * Every session in a window, following the cursor for you.
@@ -1140,6 +1342,9 @@ var RealtimeAvatar = class {
1140
1342
  authorization: `Bearer ${this.#apiKey}`,
1141
1343
  "user-agent": this.#userAgent
1142
1344
  };
1345
+ if (init.headers?.["X-RTA-Speech-Diagnostics"] === "1") {
1346
+ headers["X-RTA-Speech-Diagnostics"] = "1";
1347
+ }
1143
1348
  let body = init.body;
1144
1349
  if (init.json !== void 0) {
1145
1350
  headers["content-type"] = "application/json";
@@ -1184,34 +1389,6 @@ var RealtimeAvatar = class {
1184
1389
  throw new RealtimeAvatarHttpError(response.status, code, text, response.headers.get("x-request-id") ?? void 0);
1185
1390
  }
1186
1391
  };
1187
- function toUsageSessionPage(raw) {
1188
- const body = isRecord(raw) ? raw : {};
1189
- const rows = Array.isArray(body.data) ? body.data : [];
1190
- return {
1191
- sessions: rows.filter(isRecord).map((row) => ({
1192
- sessionId: String(row.sessionId ?? ""),
1193
- avatarId: typeof row.avatarId === "string" ? row.avatarId : null,
1194
- // Added to the contract alongside avatarId so a usage row is readable without a
1195
- // second lookup. Defensive like its siblings: an older server that omits it
1196
- // yields null rather than the string "undefined".
1197
- avatarName: typeof row.avatarName === "string" ? row.avatarName : null,
1198
- status: usageStatus(row.status),
1199
- startedAt: typeof row.startedAt === "string" ? row.startedAt : null,
1200
- endedAt: typeof row.endedAt === "string" ? row.endedAt : null,
1201
- activeSeconds: typeof row.activeSeconds === "number" ? row.activeSeconds : null,
1202
- billedCreditMicros: typeof row.billedCreditMicros === "number" ? row.billedCreditMicros : null,
1203
- metadata: isRecord(row.metadata) ? row.metadata : {},
1204
- createdAt: String(row.createdAt ?? "")
1205
- })),
1206
- nextCursor: typeof body.nextCursor === "string" ? body.nextCursor : null,
1207
- from: String(body.from ?? ""),
1208
- to: String(body.to ?? "")
1209
- };
1210
- }
1211
- var USAGE_STATUSES = ["reserved", "started", "released", "failed"];
1212
- function usageStatus(value) {
1213
- return USAGE_STATUSES.find((s) => s === value) ?? "failed";
1214
- }
1215
1392
  function isRecord(value) {
1216
1393
  return typeof value === "object" && value !== null && !Array.isArray(value);
1217
1394
  }
@@ -1282,13 +1459,35 @@ function isStream(body) {
1282
1459
 
1283
1460
  // ../http-client/src/webhook.ts
1284
1461
  async function verifyTranscript(rawBody, headers, secret, options = {}) {
1462
+ const text = await verifiedBody("transcript webhook", rawBody, headers, secret, options);
1463
+ return JSON.parse(text);
1464
+ }
1465
+ async function verifyVoiceWebhook(rawBody, headers, secret, options = {}) {
1466
+ const text = await verifiedBody("voice webhook", rawBody, headers, secret, options);
1467
+ let decoded;
1468
+ try {
1469
+ decoded = JSON.parse(text);
1470
+ } catch (cause) {
1471
+ throw new RealtimeAvatarWebhookError("invalid_body", "voice webhook body is not JSON", { cause });
1472
+ }
1473
+ const parsed = voiceWebhookRequestSchema.safeParse(decoded);
1474
+ if (!parsed.success) {
1475
+ throw new RealtimeAvatarWebhookError("invalid_body", "voice webhook body does not match the voice.synthesize contract", {
1476
+ cause: parsed.error
1477
+ });
1478
+ }
1479
+ return parsed.data;
1480
+ }
1481
+ async function verifiedBody(label, rawBody, headers, secret, options) {
1285
1482
  const get = (name) => headers instanceof Headers ? headers.get(name) ?? void 0 : headers[name];
1286
1483
  const signature = get("x-rta-signature");
1287
1484
  const timestamp = get("x-rta-timestamp");
1288
- if (!signature || !timestamp) throw new Error("missing x-rta-signature / x-rta-timestamp");
1485
+ if (!signature || !timestamp) {
1486
+ throw new RealtimeAvatarWebhookError("missing_signature", "missing x-rta-signature / x-rta-timestamp");
1487
+ }
1289
1488
  const skew = Math.abs(Date.now() / 1e3 - Number(timestamp));
1290
1489
  if (!Number.isFinite(skew) || skew > (options.toleranceSeconds ?? 300)) {
1291
- throw new Error("transcript webhook timestamp is outside the replay window");
1490
+ throw new RealtimeAvatarWebhookError("stale_timestamp", `${label} timestamp is outside the replay window`);
1292
1491
  }
1293
1492
  const text = typeof rawBody === "string" ? rawBody : new TextDecoder().decode(rawBody);
1294
1493
  const key = await crypto.subtle.importKey(
@@ -1300,8 +1499,10 @@ async function verifyTranscript(rawBody, headers, secret, options = {}) {
1300
1499
  );
1301
1500
  const mac = await crypto.subtle.sign("HMAC", key, new TextEncoder().encode(`${timestamp}.${text}`));
1302
1501
  const expected = `v1=${[...new Uint8Array(mac)].map((b) => b.toString(16).padStart(2, "0")).join("")}`;
1303
- if (!timingSafeEqual(expected, signature)) throw new Error("transcript webhook signature mismatch");
1304
- return JSON.parse(text);
1502
+ if (!timingSafeEqual(expected, signature)) {
1503
+ throw new RealtimeAvatarWebhookError("bad_signature", `${label} signature mismatch`);
1504
+ }
1505
+ return text;
1305
1506
  }
1306
1507
  function timingSafeEqual(a, b) {
1307
1508
  if (a.length !== b.length) return false;
@@ -1315,4 +1516,4 @@ function isQueued(result) {
1315
1516
  return "queued" in result;
1316
1517
  }
1317
1518
 
1318
- export { ROUTE_TIMEOUT_MS, RealtimeAvatar, RealtimeAvatarError, RealtimeAvatarHttpError, clipLibraryDeclarationSchema, isQueued, listRecordingsResponseSchema, liveKitSessionReleaseReasonSchema, recordingAccessResponseSchema, recordingArtifactSchema, recordingModeSchema, sessionModeSchema, verifyTranscript };
1519
+ export { ROUTE_TIMEOUT_MS, RealtimeAvatar, RealtimeAvatarError, RealtimeAvatarHttpError, RealtimeAvatarWebhookError, clipLibraryDeclarationSchema, isQueued, listRecordingsResponseSchema, liveKitSessionReleaseReasonSchema, recordingAccessResponseSchema, recordingArtifactSchema, recordingModeSchema, sessionModeSchema, verifyTranscript, verifyVoiceWebhook };
package/dist/express.d.ts CHANGED
@@ -1,5 +1,5 @@
1
- import { P as ProxyConfig } from './types-dqrXXrhz.js';
2
- import './types-BKv6pxzd.js';
1
+ import { P as ProxyConfig } from './types-Bli1UeP0.js';
2
+ import './types-yiemDkm0.js';
3
3
  import 'zod';
4
4
 
5
5
  type Expressish = {
package/dist/express.js CHANGED
@@ -1,5 +1,5 @@
1
- import { createProxyHandler } from './chunk-NNX2ZSA3.js';
2
- import './chunk-VVQIYWKT.js';
1
+ import { createProxyHandler } from './chunk-K3VC6BF7.js';
2
+ import './chunk-V7A4NRKC.js';
3
3
  import './chunk-MCIXU6XP.js';
4
4
 
5
5
  // ../proxy/src/express.ts
package/dist/hono.d.ts CHANGED
@@ -1,5 +1,5 @@
1
- import { P as ProxyConfig } from './types-dqrXXrhz.js';
2
- import './types-BKv6pxzd.js';
1
+ import { P as ProxyConfig } from './types-Bli1UeP0.js';
2
+ import './types-yiemDkm0.js';
3
3
  import 'zod';
4
4
 
5
5
  /**
package/dist/hono.js CHANGED
@@ -1,5 +1,5 @@
1
- import { createProxyHandler } from './chunk-NNX2ZSA3.js';
2
- import './chunk-VVQIYWKT.js';
1
+ import { createProxyHandler } from './chunk-K3VC6BF7.js';
2
+ import './chunk-V7A4NRKC.js';
3
3
  import './chunk-MCIXU6XP.js';
4
4
 
5
5
  // ../proxy/src/hono.ts
package/dist/index.d.ts CHANGED
@@ -1,5 +1,5 @@
1
- import { a as CallPolicy, C as CallMode, S as StartCallResult, E as EndCallOptions, A as Avatar, b as AvatarUpdate, c as AvatarSourceSwap, d as ClipLibraryDeclaration, e as ClipLibraryUpdate, L as LoopRedirect, f as ClipLibrary, g as AssetKind, h as Asset, i as ListSessionsOptions, U as UsageSessionPage, j as UsageSession, k as ListRecordingsQuery, l as ListRecordingsResponse, R as RecordingArtifact, m as RecordingAccessResponse, n as CreditBalance, T as TranscriptPayload } from './types-BKv6pxzd.js';
2
- export { o as AvatarClip, p as CallConnection, q as CallQueued, r as ClipDeclaration, s as ClipLibraryPlan, t as ClipSource, u as ContextMessage, v as EndCallReason, w as ExternalLlm, x as RecordingArtifactStatus, y as RecordingMode, V as VideoPolicy, z as isQueued, B as listRecordingsResponseSchema, D as recordingAccessResponseSchema, F as recordingArtifactSchema, G as recordingModeSchema } from './types-BKv6pxzd.js';
1
+ import { a as CallPolicy, C as CallMode, S as StartCallResult, E as EndCallOptions, A as Avatar, b as AvatarUpdate, c as AvatarSourceSwap, d as ClipLibraryDeclaration, e as ClipLibraryUpdate, L as LoopRedirect, f as ClipLibrary, g as AssetKind, h as Asset, i as ListSessionsOptions, U as UsageSessionPage, j as UsageSession, k as ListRecordingsQuery, l as ListRecordingsResponse, R as RecordingArtifact, m as RecordingAccessResponse, n as CreditBalance, T as TranscriptPayload, V as VoiceWebhookRequest } from './types-yiemDkm0.js';
2
+ export { o as AvatarClip, p as CallConnection, q as CallQueued, r as ClipDeclaration, s as ClipLibraryPlan, t as ClipSource, u as ContextMessage, v as EndCallReason, w as ExternalLlm, x as RecordingArtifactStatus, y as RecordingMode, z as VideoPolicy, B as VoiceWebhook, D as VoiceWebhookErrorCode, F as isQueued, G as listRecordingsResponseSchema, H as recordingAccessResponseSchema, I as recordingArtifactSchema, J as recordingModeSchema } from './types-yiemDkm0.js';
3
3
  import { ConnectionHistoryResponse } from './connection-history.js';
4
4
  import { z } from 'zod';
5
5
 
@@ -278,6 +278,10 @@ declare class RealtimeAvatar {
278
278
  *
279
279
  * Tagging is optional and nothing degrades without it — you just cannot attribute a
280
280
  * session to one of your users. Requires a key with the `usage:read` scope.
281
+ *
282
+ * `speechDiagnostics: true` opts into whole-session dialogue per-leg measurements, not
283
+ * browser-audible latency. Unavailable evidence is explicit; old-server absence stays absent.
284
+ * The API owns the opt-in page cap and budgets; the SDK forwards the requested limit.
281
285
  */
282
286
  listSessions(options?: ListSessionsOptions): Promise<UsageSessionPage>;
283
287
  /**
@@ -304,6 +308,19 @@ declare class RealtimeAvatarError extends Error {
304
308
  cause?: unknown;
305
309
  });
306
310
  }
311
+ /** Why `verifyTranscript` or `verifyVoiceWebhook` refused a request. Answer each with a 4xx. */
312
+ type WebhookVerificationErrorCode = "missing_signature" | "bad_signature" | "stale_timestamp" | "invalid_body";
313
+ /**
314
+ * A webhook request that did not verify. `code` says why: the signature headers are absent, the
315
+ * signature does not match the raw body, the timestamp is outside the replay window, or a
316
+ * correctly signed body is not the contract.
317
+ */
318
+ declare class RealtimeAvatarWebhookError extends RealtimeAvatarError {
319
+ readonly code: WebhookVerificationErrorCode;
320
+ constructor(code: WebhookVerificationErrorCode, message: string, options?: {
321
+ cause?: unknown;
322
+ });
323
+ }
307
324
  /**
308
325
  * A non-2xx from the API.
309
326
  *
@@ -348,6 +365,10 @@ declare class RealtimeAvatarHttpError extends RealtimeAvatarError {
348
365
  get isBilling(): boolean;
349
366
  }
350
367
 
368
+ type WebhookHeaders = Headers | Record<string, string | undefined>;
369
+ type VerifyOptions = {
370
+ toleranceSeconds?: number;
371
+ };
351
372
  /**
352
373
  * Verify a transcript webhook.
353
374
  *
@@ -359,9 +380,22 @@ declare class RealtimeAvatarHttpError extends RealtimeAvatarError {
359
380
  * const transcript = await verifyTranscript(raw, request.headers, process.env.TRANSCRIPT_SECRET!);
360
381
  * ```
361
382
  */
362
- declare function verifyTranscript(rawBody: Uint8Array | string, headers: Headers | Record<string, string | undefined>, secret: string, options?: {
363
- toleranceSeconds?: number;
364
- }): Promise<TranscriptPayload>;
383
+ declare function verifyTranscript(rawBody: Uint8Array | string, headers: WebhookHeaders, secret: string, options?: VerifyOptions): Promise<TranscriptPayload>;
384
+ /**
385
+ * Verify one `voice.synthesize` request from a call minted with `voiceWebhook`, and return it
386
+ * typed. Same signature scheme and the same rule as {@link verifyTranscript}: pass the RAW bytes.
387
+ *
388
+ * Throws {@link RealtimeAvatarWebhookError}. Answer a signature or timestamp refusal with 401 and
389
+ * `invalid_body` with 400. The worker ends the call on a 401 or 403, because a secret that does not
390
+ * match will not match on the next sentence either.
391
+ *
392
+ * ```ts
393
+ * const raw = Buffer.from(await request.arrayBuffer());
394
+ * const { text, output } = await verifyVoiceWebhook(raw, request.headers, process.env.VOICE_SECRET!);
395
+ * // reply 200, Content-Type audio/pcm: s16le mono at output.sample_rate, streamed
396
+ * ```
397
+ */
398
+ declare function verifyVoiceWebhook(rawBody: Uint8Array | string, headers: WebhookHeaders, secret: string, options?: VerifyOptions): Promise<VoiceWebhookRequest>;
365
399
 
366
400
  declare const clipLibraryDeclarationSchema: z.ZodObject<{
367
401
  idle: z.ZodOptional<z.ZodObject<{
@@ -389,4 +423,4 @@ declare const clipLibraryDeclarationSchema: z.ZodObject<{
389
423
  }, z.core.$strict>>>>;
390
424
  }, z.core.$strict>;
391
425
 
392
- export { Asset, AssetKind, Avatar, AvatarSourceSwap, AvatarUpdate, CallMode, CallPolicy, ClipLibrary, ClipLibraryDeclaration, ClipLibraryUpdate, CreditBalance, EndCallOptions, ListRecordingsQuery, ListRecordingsResponse, ListSessionsOptions, LoopRedirect, RealtimeAvatar, RealtimeAvatarError, RealtimeAvatarHttpError, type RealtimeAvatarOptions, RecordingAccessResponse, RecordingArtifact, type StartCallOptions, StartCallResult, TranscriptPayload, UsageSession, UsageSessionPage, clipLibraryDeclarationSchema, verifyTranscript };
426
+ export { Asset, AssetKind, Avatar, AvatarSourceSwap, AvatarUpdate, CallMode, CallPolicy, ClipLibrary, ClipLibraryDeclaration, ClipLibraryUpdate, CreditBalance, EndCallOptions, ListRecordingsQuery, ListRecordingsResponse, ListSessionsOptions, LoopRedirect, RealtimeAvatar, RealtimeAvatarError, RealtimeAvatarHttpError, type RealtimeAvatarOptions, RealtimeAvatarWebhookError, RecordingAccessResponse, RecordingArtifact, type StartCallOptions, StartCallResult, TranscriptPayload, UsageSession, UsageSessionPage, VoiceWebhookRequest, type WebhookVerificationErrorCode, clipLibraryDeclarationSchema, verifyTranscript, verifyVoiceWebhook };
package/dist/index.js CHANGED
@@ -1,2 +1,2 @@
1
- export { RealtimeAvatar, RealtimeAvatarError, RealtimeAvatarHttpError, clipLibraryDeclarationSchema, isQueued, listRecordingsResponseSchema, recordingAccessResponseSchema, recordingArtifactSchema, recordingModeSchema, verifyTranscript } from './chunk-VVQIYWKT.js';
1
+ export { RealtimeAvatar, RealtimeAvatarError, RealtimeAvatarHttpError, RealtimeAvatarWebhookError, clipLibraryDeclarationSchema, isQueued, listRecordingsResponseSchema, recordingAccessResponseSchema, recordingArtifactSchema, recordingModeSchema, verifyTranscript, verifyVoiceWebhook } from './chunk-V7A4NRKC.js';
2
2
  import './chunk-MCIXU6XP.js';
package/dist/nextjs.d.ts CHANGED
@@ -1,5 +1,5 @@
1
- import { P as ProxyConfig } from './types-dqrXXrhz.js';
2
- import './types-BKv6pxzd.js';
1
+ import { P as ProxyConfig } from './types-Bli1UeP0.js';
2
+ import './types-yiemDkm0.js';
3
3
  import 'zod';
4
4
 
5
5
  /**