realtime-avatar 0.25.0 → 0.26.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -224,6 +224,7 @@ all), and the `video` policy types are deliberately not one-to-one with the wire
224
224
  // calls
225
225
  rta.startCall({ avatarId, mode?, instructions?, context?, maxSeconds?, video?, recording?, connectionHistory?, transcript?, metadata? })
226
226
  rta.endCall(sessionId, { reason? }) // free an abandoned call's slot; idempotent, never throws
227
+ rta.leaveQueue(queueTicketId, { reason? }) // give up a queued call's place in line; idempotent, never throws
227
228
 
228
229
  // optional recordings; server only, requires recordings:read
229
230
  rta.listRecordings({ sessionId, limit?, cursor? })
@@ -294,6 +295,61 @@ call. Policy — `instructions`, `maxSeconds`, `voice`, `video` — is decided i
294
295
  your server. A route that spreads the request body into `startCall` hands your caller your
295
296
  system prompt and your bill.
296
297
 
298
+ ### Styling
299
+
300
+ `AvatarCall` and `AvatarVideoSurface` lay themselves out with inline styles, so they need no
301
+ CSS build and no configuration: no stylesheet to import and no Tailwind `@source` for this
302
+ package. The surface fills its container; give that container a definite size (an aspect
303
+ ratio plus a height, say). `className` on the surface adds your own classes; to change a
304
+ property the surface sets itself, such as its 100% size, pass `style`.
305
+
306
+ `AvatarCall`'s `children` render in a layer that fills the call's box above the video and
307
+ the live badge, so an absolutely positioned control places itself against the box and stays
308
+ clickable while she is live.
309
+
310
+ ### Microphone and sound
311
+
312
+ The handle reports the two local failures a "live" call can hide. `call.microphone` says
313
+ whether she can hear the user (`off`, `pending`, `on`, `muted`, `blocked`, `unavailable`; the
314
+ last two carry `reason`, `message` and `hint`), and `onMicrophoneProblem` fires once per new
315
+ problem. A device that stops mid-call reads `pending` while LiveKit retries the default device,
316
+ and `unavailable` (`device-lost`) only once that fails. `call.retryMicrophone()` asks again after
317
+ the user fixes it, and `call.setMicrophoneEnabled(enabled)` mutes and unmutes. While the call is
318
+ waiting or connecting those only record the choice (nothing is captured before the room
319
+ connects), a mute made then is kept when the call goes live, and after `end()` they do nothing.
320
+
321
+ `call.audio` says whether the user can hear her (`unknown`, `allowed`, `blocked`; `unknown` again
322
+ once the call ended). While it is `blocked`, `AvatarCall` shows a "Tap to turn on sound" button,
323
+ top-centre in a polite live region, that calls `call.startAudio()`. Pass `audioUnlockPrompt` a
324
+ function to render your own (your words, your placement), or `false` for nothing.
325
+ `startAudio()` must run inside a click or tap handler; it never rejects and resolves whether
326
+ playback is allowed afterwards.
327
+
328
+ Every state is derived from LiveKit's public events, and LiveKit still owns the device: it
329
+ captures on connect and stops when the room leaves. A microphone error from an earlier call on
330
+ the same room is not reported for the next one. A publish that fails after the microphone was
331
+ captured reaches only the room's `onError` and stays `pending`: that callback also carries
332
+ unrelated failures, so it is not guessed into a microphone problem.
333
+
334
+ ### Ending a call
335
+
336
+ `call.end()` (and `useRealtimeSession().end()`) is terminal from every phase, including
337
+ `waiting` in the queue and `connecting`. It stops the queue retry and the reconnect ladder,
338
+ releases the held session or the queue ticket, leaves the room (which stops the microphone),
339
+ moves `status` to `"ended"` and fires `onEnded({ reason: "user_ended" })` once. Ending a call
340
+ that is already over does nothing. `sayAndEnd(text)` is the graceful path when she should speak
341
+ a last line first.
342
+
343
+ **After `end()`, nothing starts a call except you, explicitly.** Changing `avatarId`, `mode` or
344
+ `listen` on an ended `AvatarCall` does not redial; remount it, for example with a `key`:
345
+
346
+ ```tsx
347
+ <AvatarCall key={callId} client={client} avatarId={avatarId} /> // new callId → new call
348
+ ```
349
+
350
+ On the lower-level hooks, `reconnect()` redials (a double tap mints once), and so does setting
351
+ `active` to false and back. `reset()` does not undo an end.
352
+
297
353
  ### Optional connection details
298
354
 
299
355
  `AvatarCall` provides call status, actions and end reasons for your default UI. To show
@@ -390,6 +446,28 @@ room-wide to whatever you render inside it (`playoutDelaySeconds={false}` opts o
390
446
  `attachAvatarTools` runs your functions in the page. Nothing is executed on the platform, and
391
447
  a tool has **2.5 seconds** to answer before the call gives up on it and tells her it failed.
392
448
 
449
+ A failed registration throws `ToolRegistrationError`, whose `retryable` says whether trying again
450
+ can help: the agent not there or not armed by the deadline (a slow start, or a session minted
451
+ without `client_tools`; one attempt cannot tell which), or an RPC transport failure, is retryable;
452
+ an oversized manifest or a request LiveKit or the agent refused is not.
453
+
454
+ In React, `useCharacterTools(tools)` registers them while the room is connected and returns
455
+ their state: `idle`, `registering` (with `attempt`), `ready` (`registered`, `rejected`) or
456
+ `error`. A retryable failure is retried twice (after 1s, then 3s) while the room stays
457
+ connected; only then is it `error`. It re-registers when the manifest changes (names,
458
+ descriptions, parameters), not when the `tools` object does, so an inline object is fine and
459
+ each call reaches your latest `execute`. With `AvatarCall`, call it from a component you render
460
+ as a child, which runs inside the call's room, and show its state rather than ignore it:
461
+
462
+ ```tsx
463
+ function Tools() {
464
+ const tools = useCharacterTools(myTools);
465
+ return tools.status === "error" ? <p role="status">Tools are off for this call: {tools.error}</p> : null;
466
+ }
467
+
468
+ <AvatarCall client={client} avatarId="ava_…">{() => <Tools />}</AvatarCall>
469
+ ```
470
+
393
471
  ## What `/react` exports, and what it deliberately does not
394
472
 
395
473
  31 names, down from 82 on 2026-08-26. Two groups came out and are not coming back:
package/dist/browser.d.ts CHANGED
@@ -1,4 +1,4 @@
1
- export { D as DEFAULT_AVATAR_PLAYOUT_DELAY_SECONDS, P as PlayoutDelayTarget, a as applyAvatarPlayoutDelay, b as applyPlayoutDelay } from './playout-delay-DWOc8iCl.js';
1
+ export { D as DEFAULT_AVATAR_PLAYOUT_DELAY_SECONDS, E as EnableMicrophoneOptions, M as MicrophoneCapableRoom, a as MicrophoneFailureReason, b as MicrophoneProblem, c as MicrophoneProblemReason, d as MicrophoneResult, P as PlayoutDelayTarget, e as applyAvatarPlayoutDelay, f as applyPlayoutDelay, g as describeMicrophoneFailure, h as enableMicrophone } from './playout-delay-CtRZdtsj.js';
2
2
 
3
3
  type VideoSupport = {
4
4
  supported: true;
@@ -10,91 +10,6 @@ type VideoSupport = {
10
10
  /** Check before starting a billed video session; this does not prove negotiated playback. */
11
11
  declare function checkVideoSupport(codec?: string): VideoSupport;
12
12
 
13
- /**
14
- * Turning the microphone on, and saying what went wrong when it does not.
15
- *
16
- * WHY THIS EXISTS. `room.localParticipant.setMicrophoneEnabled(true)` is one line, so every
17
- * integration writes it inline and none of them handle its rejection — the reject path is
18
- * invisible until a real user is on a real machine. When it fails there is nothing to see:
19
- * the call is already connected, the page is mid-`await`, and the rejection becomes an
20
- * unhandled promise nobody is looking at. The reported symptom is always the same sentence —
21
- * "the mic won't start" — and it covers at least six unrelated causes with different fixes,
22
- * one of which is not in the browser at all.
23
- *
24
- * So this returns a RESULT rather than throwing: the cause becomes a value the caller has to
25
- * handle, and each one carries the sentence that names its actual fix. `message` is what the
26
- * browser said; `hint` is what the person should do.
27
- *
28
- * It is deliberately structural — it never imports `livekit-client`, so it pins no version
29
- * and adds no bytes. Anything with a `localParticipant.setMicrophoneEnabled` satisfies it.
30
- */
31
- /** The sliver of livekit-client's `Room` this needs. Structural on purpose — see above. */
32
- interface MicrophoneCapableRoom {
33
- localParticipant: {
34
- setMicrophoneEnabled(enabled: boolean): Promise<unknown>;
35
- };
36
- }
37
- /**
38
- * Why the microphone did not start. Every variant has a DIFFERENT fix, which is the whole
39
- * reason this is an enum and not a boolean.
40
- */
41
- type MicrophoneFailureReason =
42
- /** No `navigator.mediaDevices` — the page is not a secure context. */
43
- "insecure-origin"
44
- /** The permission prompt was never answered. getUserMedia may never settle; we time out. */
45
- | "no-answer"
46
- /** The BROWSER denied it — the site's own permission, resettable from the address bar. */
47
- | "denied-by-browser"
48
- /** The OS denied it — macOS System Settings, nothing the page or the site can change. */
49
- | "denied-by-os"
50
- /** No input device is connected, or none matches the constraints. */
51
- | "no-device"
52
- /** A device exists but something else holds it — another tab, app, or a stale call. */
53
- | "device-in-use"
54
- /** Anything unrecognised. `message` still carries what the browser said. */
55
- | "unknown";
56
- type MicrophoneResult = {
57
- ok: true;
58
- } | {
59
- ok: false;
60
- reason: MicrophoneFailureReason;
61
- /** What the browser reported, verbatim. Show it — it is often exact. */
62
- message: string;
63
- /** What the person should actually do about it. */
64
- hint: string;
65
- };
66
- interface EnableMicrophoneOptions {
67
- /**
68
- * How long to wait before calling it a no-answer. Default 15s.
69
- *
70
- * This is not belt-and-braces. Per the getUserMedia specification a prompt the user never
71
- * dismisses may leave the promise permanently pending — "neither resolve nor reject" — and
72
- * a page that awaits it in its connect path hangs on a state that has no error and no end.
73
- * A deadline turns that into a reportable outcome.
74
- */
75
- timeoutMs?: number;
76
- }
77
- /**
78
- * Publish the caller's microphone, and report WHY if it does not start.
79
- *
80
- * ```ts
81
- * const mic = await enableMicrophone(room);
82
- * if (!mic.ok) {
83
- * status.textContent = `Microphone: ${mic.message}`;
84
- * help.textContent = mic.hint; // the sentence that names the fix
85
- * }
86
- * ```
87
- *
88
- * Never throws. A call that cannot start the microphone is an ordinary outcome of asking a
89
- * person for a device, not a programmer error, so it belongs in the return type.
90
- *
91
- * Note it does NOT hang up on failure — the call is still live and still billing, and only
92
- * the caller knows whether a text-only session is useful to them. If it is not, disconnect
93
- * in the `!ok` branch; leaving a room connected against a device the user never granted is
94
- * how the retry ends up contending with the call that is still holding it.
95
- */
96
- declare function enableMicrophone(room: MicrophoneCapableRoom, options?: EnableMicrophoneOptions): Promise<MicrophoneResult>;
97
-
98
13
  /**
99
14
  * Making the character AUDIBLE — the half of "audio works" that has nothing to do with the
100
15
  * microphone.
@@ -251,4 +166,4 @@ type BoomerangPlayback = {
251
166
  */
252
167
  declare function attachBoomerangPlayback(video: HTMLVideoElement, options?: BoomerangPlaybackOptions): BoomerangPlayback;
253
168
 
254
- export { type AttachRemoteAudioOptions, type AttachableTrack, type AudioCapableRoom, type BoomerangPlayback, type BoomerangPlaybackOptions, type EnableMicrophoneOptions, type MicrophoneCapableRoom, type MicrophoneFailureReason, type MicrophoneResult, type PreparableParticipant, type PreparableRoom, type PrepareAvatarRoomOptions, type PreparedAvatarRoom, type RemoteAudioAttachment, type VideoSupport, attachBoomerangPlayback, attachRemoteAudio, checkVideoSupport, enableMicrophone, prepareAvatarRoom };
169
+ export { type AttachRemoteAudioOptions, type AttachableTrack, type AudioCapableRoom, type BoomerangPlayback, type BoomerangPlaybackOptions, type PreparableParticipant, type PreparableRoom, type PrepareAvatarRoomOptions, type PreparedAvatarRoom, type RemoteAudioAttachment, type VideoSupport, attachBoomerangPlayback, attachRemoteAudio, checkVideoSupport, prepareAvatarRoom };
package/dist/browser.js CHANGED
@@ -18,6 +18,7 @@ var HINTS = {
18
18
  "denied-by-os": "The operating system is blocking the browser itself, so no site can record. On macOS: System Settings > Privacy & Security > Microphone, enable your browser, then RESTART it \u2014 the change does not apply to a running browser.",
19
19
  "no-device": "No microphone is available. Connect one, and if you are on Bluetooth headphones check they are in a mode that has a microphone.",
20
20
  "device-in-use": "Another application or browser tab is holding the microphone. Close it \u2014 including any call from this page you did not hang up \u2014 and try again.",
21
+ "device-lost": "The microphone stopped during the call: it was unplugged, or access to it was turned off. Reconnect it or allow access again, then turn the microphone back on.",
21
22
  unknown: "Try again, and if it persists include the message above in your report."
22
23
  };
23
24
  function isOperatingSystemDenial(message) {
@@ -54,6 +55,10 @@ function classify(error) {
54
55
  function fail(reason, message) {
55
56
  return { ok: false, reason, message, hint: HINTS[reason] };
56
57
  }
58
+ function describeMicrophoneFailure(error) {
59
+ const { reason, message } = classify(error);
60
+ return { reason, message, hint: HINTS[reason] };
61
+ }
57
62
  async function enableMicrophone(room, options = {}) {
58
63
  const timeoutMs = options.timeoutMs ?? 15e3;
59
64
  if (typeof navigator === "undefined" || !navigator.mediaDevices) {
@@ -253,4 +258,4 @@ function now() {
253
258
  return typeof performance !== "undefined" ? performance.now() : Date.now();
254
259
  }
255
260
 
256
- export { DEFAULT_AVATAR_PLAYOUT_DELAY_SECONDS, applyAvatarPlayoutDelay, applyPlayoutDelay, attachBoomerangPlayback, attachRemoteAudio, checkVideoSupport, enableMicrophone, prepareAvatarRoom };
261
+ export { DEFAULT_AVATAR_PLAYOUT_DELAY_SECONDS, applyAvatarPlayoutDelay, applyPlayoutDelay, attachBoomerangPlayback, attachRemoteAudio, checkVideoSupport, describeMicrophoneFailure, enableMicrophone, prepareAvatarRoom };
@@ -654,7 +654,7 @@ z.object({
654
654
  queue_position: z.number().int().positive().optional(),
655
655
  recommended_retry_ms: z.number().int().nonnegative()
656
656
  }).strict();
657
- z.enum([
657
+ var liveKitSessionReleaseReasonSchema = z.enum([
658
658
  "page_hide",
659
659
  "disconnected",
660
660
  "superseded",
@@ -665,7 +665,7 @@ z.enum([
665
665
 
666
666
  // ../http-client/src/client.ts
667
667
  var DEFAULT_BASE_URL = "https://realtimeavatar.ai/api/v1";
668
- var SDK_VERSION = "0.25.0";
668
+ var SDK_VERSION = "0.26.0";
669
669
  var RealtimeAvatar = class {
670
670
  #apiKey;
671
671
  #baseUrl;
@@ -777,7 +777,25 @@ var RealtimeAvatar = class {
777
777
  */
778
778
  async endCall(sessionId, options = {}) {
779
779
  if (!sessionId) return false;
780
- const body = { session_id: sessionId };
780
+ return this.#release({ session_id: sessionId }, options);
781
+ }
782
+ /**
783
+ * Give up a QUEUED call's place in line, by the `queueTicketId` that `startCall` returned.
784
+ *
785
+ * A queued call has no session yet, so {@link endCall} cannot reach it: the release contract
786
+ * carries the ticket in its own field, and a ticket sent as a session id names nothing and is
787
+ * acknowledged as an idempotent no-op. The place is then held until its TTL, at the front of
788
+ * the queue, in front of everyone behind it.
789
+ *
790
+ * Same contract as `endCall`: best-effort, `true` only when the platform acknowledged it,
791
+ * never a throw, and only for tickets YOUR SERVER was handed.
792
+ */
793
+ async leaveQueue(queueTicketId, options = {}) {
794
+ if (!queueTicketId) return false;
795
+ return this.#release({ queue_ticket_id: queueTicketId }, options);
796
+ }
797
+ async #release(target, options) {
798
+ const body = { ...target };
781
799
  if (options.reason !== void 0) body.reason = options.reason;
782
800
  if (options.capacityPool !== void 0) body.capacity_pool = options.capacityPool;
783
801
  try {
@@ -1281,4 +1299,4 @@ function isQueued(result) {
1281
1299
  return "queued" in result;
1282
1300
  }
1283
1301
 
1284
- export { ROUTE_TIMEOUT_MS, RealtimeAvatar, RealtimeAvatarError, RealtimeAvatarHttpError, clipLibraryDeclarationSchema, isQueued, listRecordingsResponseSchema, recordingAccessResponseSchema, recordingArtifactSchema, recordingModeSchema, verifyTranscript };
1302
+ export { ROUTE_TIMEOUT_MS, RealtimeAvatar, RealtimeAvatarError, RealtimeAvatarHttpError, clipLibraryDeclarationSchema, isQueued, listRecordingsResponseSchema, liveKitSessionReleaseReasonSchema, recordingAccessResponseSchema, recordingArtifactSchema, recordingModeSchema, sessionModeSchema, verifyTranscript };
@@ -1,4 +1,5 @@
1
- import { RealtimeAvatar, ROUTE_TIMEOUT_MS, isQueued, RealtimeAvatarHttpError, RealtimeAvatarError } from './chunk-GMYGHDY2.js';
1
+ import { sessionModeSchema, liveKitSessionReleaseReasonSchema, RealtimeAvatar, ROUTE_TIMEOUT_MS, isQueued, RealtimeAvatarHttpError, RealtimeAvatarError } from './chunk-HMESE54B.js';
2
+ import { z } from 'zod';
2
3
 
3
4
  // ../http-client/src/generated/error-semantics.ts
4
5
  var ERROR_SEMANTICS = {
@@ -38,6 +39,17 @@ var MESSAGE_SHAPE_PATTERNS = { "machine": { "source": "[_=<>{}\\[\\]|\\\\$`]|\\w
38
39
  Object.values(MESSAGE_SHAPE_PATTERNS).map(
39
40
  (pattern) => new RegExp(pattern.source, pattern.flags)
40
41
  );
42
+ var proxyConnectRequestSchema = z.object({
43
+ avatarId: z.string().min(1),
44
+ mode: sessionModeSchema.optional()
45
+ });
46
+ var proxyEndRequestSchema = z.object({
47
+ session_id: z.string().min(1).optional(),
48
+ queue_ticket_id: z.string().min(1).optional(),
49
+ reason: liveKitSessionReleaseReasonSchema.optional().catch(void 0)
50
+ }).strict().refine((body) => body.session_id !== void 0 || body.queue_ticket_id !== void 0, {
51
+ message: "session_id or queue_ticket_id is required"
52
+ });
41
53
 
42
54
  // ../proxy/src/config.ts
43
55
  var MISCONFIGURED = "internal_error";
@@ -97,19 +109,25 @@ function createProxyHandler(config) {
97
109
  try {
98
110
  if (operation === "avatars") return json({ data: await rta.listAvatars() });
99
111
  if (operation === "credits") return json(await rta.creditBalance());
112
+ const raw = await request.json().catch(() => void 0);
100
113
  if (operation === "end") {
101
- const ended = await request.json().catch(() => ({}));
102
- const sessionId = ended.session_id ?? ended.queue_ticket_id;
103
- if (!sessionId) return json({ error: "session_id or queue_ticket_id is required" }, 422);
104
- const owns = config.ownsSession ? await config.ownsSession({ request, sessionId }) : minted.delete(sessionId);
105
- if (!owns) return new Response(null, { status: 204 });
114
+ const parsed2 = proxyEndRequestSchema.safeParse(raw);
115
+ if (!parsed2.success) return json({ error: "session_id or queue_ticket_id is required" }, 422);
116
+ const ended = parsed2.data;
106
117
  const reason = ended.reason === "page_hide" || ended.reason === "unmount" ? ended.reason : "manual";
107
- await rta.endCall(sessionId, { reason });
118
+ const owned = async (id) => config.ownsSession ? await config.ownsSession({ request, sessionId: id }) : minted.delete(id);
119
+ if (ended.session_id !== void 0 && await owned(ended.session_id)) {
120
+ await rta.endCall(ended.session_id, { reason });
121
+ }
122
+ if (ended.queue_ticket_id !== void 0 && await owned(ended.queue_ticket_id)) {
123
+ await rta.leaveQueue(ended.queue_ticket_id, { reason });
124
+ }
108
125
  return new Response(null, { status: 204 });
109
126
  }
110
- const body = await request.json().catch(() => ({}));
111
- if (!body.avatarId) return json({ error: "avatarId is required" }, 422);
112
- const mode = body.mode === "voice" ? "voice" : "avatar";
127
+ const parsed = proxyConnectRequestSchema.safeParse(raw);
128
+ if (!parsed.success) return json({ error: "avatarId is required" }, 422);
129
+ const body = parsed.data;
130
+ const mode = body.mode ?? "avatar";
113
131
  const decided = await config.session?.({ request, avatarId: body.avatarId, mode });
114
132
  if (decided instanceof Response) return decided;
115
133
  const call = await rta.startCall({ avatarId: body.avatarId, mode, ...decided ?? {} });
package/dist/express.js CHANGED
@@ -1,5 +1,5 @@
1
- import { createProxyHandler } from './chunk-DNHHQ5GZ.js';
2
- import './chunk-GMYGHDY2.js';
1
+ import { createProxyHandler } from './chunk-XJPGQMEW.js';
2
+ import './chunk-HMESE54B.js';
3
3
  import './chunk-MCIXU6XP.js';
4
4
 
5
5
  // ../proxy/src/express.ts
package/dist/hono.js CHANGED
@@ -1,5 +1,5 @@
1
- import { createProxyHandler } from './chunk-DNHHQ5GZ.js';
2
- import './chunk-GMYGHDY2.js';
1
+ import { createProxyHandler } from './chunk-XJPGQMEW.js';
2
+ import './chunk-HMESE54B.js';
3
3
  import './chunk-MCIXU6XP.js';
4
4
 
5
5
  // ../proxy/src/hono.ts
package/dist/index.d.ts CHANGED
@@ -82,6 +82,18 @@ declare class RealtimeAvatar {
82
82
  * request body lets any visitor hang up any call on your account.
83
83
  */
84
84
  endCall(sessionId: string, options?: EndCallOptions): Promise<boolean>;
85
+ /**
86
+ * Give up a QUEUED call's place in line, by the `queueTicketId` that `startCall` returned.
87
+ *
88
+ * A queued call has no session yet, so {@link endCall} cannot reach it: the release contract
89
+ * carries the ticket in its own field, and a ticket sent as a session id names nothing and is
90
+ * acknowledged as an idempotent no-op. The place is then held until its TTL, at the front of
91
+ * the queue, in front of everyone behind it.
92
+ *
93
+ * Same contract as `endCall`: best-effort, `true` only when the platform acknowledged it,
94
+ * never a throw, and only for tickets YOUR SERVER was handed.
95
+ */
96
+ leaveQueue(queueTicketId: string, options?: Pick<EndCallOptions, "reason">): Promise<boolean>;
85
97
  /**
86
98
  * Register a character from a looping clip you host.
87
99
  *
package/dist/index.js CHANGED
@@ -1,2 +1,2 @@
1
- export { RealtimeAvatar, RealtimeAvatarError, RealtimeAvatarHttpError, clipLibraryDeclarationSchema, isQueued, listRecordingsResponseSchema, recordingAccessResponseSchema, recordingArtifactSchema, recordingModeSchema, verifyTranscript } from './chunk-GMYGHDY2.js';
1
+ export { RealtimeAvatar, RealtimeAvatarError, RealtimeAvatarHttpError, clipLibraryDeclarationSchema, isQueued, listRecordingsResponseSchema, recordingAccessResponseSchema, recordingArtifactSchema, recordingModeSchema, verifyTranscript } from './chunk-HMESE54B.js';
2
2
  import './chunk-MCIXU6XP.js';
package/dist/nextjs.js CHANGED
@@ -1,5 +1,5 @@
1
- import { createProxyHandler } from './chunk-DNHHQ5GZ.js';
2
- import './chunk-GMYGHDY2.js';
1
+ import { createProxyHandler } from './chunk-XJPGQMEW.js';
2
+ import './chunk-HMESE54B.js';
3
3
  import './chunk-MCIXU6XP.js';
4
4
 
5
5
  // ../proxy/src/nextjs.ts
@@ -0,0 +1,168 @@
1
+ /**
2
+ * Turning the microphone on, and saying what went wrong when it does not.
3
+ *
4
+ * WHY THIS EXISTS. `room.localParticipant.setMicrophoneEnabled(true)` is one line, so every
5
+ * integration writes it inline and none of them handle its rejection — the reject path is
6
+ * invisible until a real user is on a real machine. When it fails there is nothing to see:
7
+ * the call is already connected, the page is mid-`await`, and the rejection becomes an
8
+ * unhandled promise nobody is looking at. The reported symptom is always the same sentence —
9
+ * "the mic won't start" — and it covers at least six unrelated causes with different fixes,
10
+ * one of which is not in the browser at all.
11
+ *
12
+ * So this returns a RESULT rather than throwing: the cause becomes a value the caller has to
13
+ * handle, and each one carries the sentence that names its actual fix. `message` is what the
14
+ * browser said; `hint` is what the person should do.
15
+ *
16
+ * It is deliberately structural — it never imports `livekit-client`, so it pins no version
17
+ * and adds no bytes. Anything with a `localParticipant.setMicrophoneEnabled` satisfies it.
18
+ */
19
+ /** The sliver of livekit-client's `Room` this needs. Structural on purpose — see above. */
20
+ interface MicrophoneCapableRoom {
21
+ localParticipant: {
22
+ setMicrophoneEnabled(enabled: boolean): Promise<unknown>;
23
+ };
24
+ }
25
+ /**
26
+ * Why the microphone did not start. Every variant has a DIFFERENT fix, which is the whole
27
+ * reason this is an enum and not a boolean.
28
+ */
29
+ type MicrophoneFailureReason =
30
+ /** No `navigator.mediaDevices` — the page is not a secure context. */
31
+ "insecure-origin"
32
+ /** The permission prompt was never answered. getUserMedia may never settle; we time out. */
33
+ | "no-answer"
34
+ /** The BROWSER denied it — the site's own permission, resettable from the address bar. */
35
+ | "denied-by-browser"
36
+ /** The OS denied it — macOS System Settings, nothing the page or the site can change. */
37
+ | "denied-by-os"
38
+ /** No input device is connected, or none matches the constraints. */
39
+ | "no-device"
40
+ /** A device exists but something else holds it — another tab, app, or a stale call. */
41
+ | "device-in-use"
42
+ /** Anything unrecognised. `message` still carries what the browser said. */
43
+ | "unknown";
44
+ type MicrophoneResult = {
45
+ ok: true;
46
+ } | {
47
+ ok: false;
48
+ reason: MicrophoneFailureReason;
49
+ /** What the browser reported, verbatim. Show it — it is often exact. */
50
+ message: string;
51
+ /** What the person should actually do about it. */
52
+ hint: string;
53
+ };
54
+ interface EnableMicrophoneOptions {
55
+ /**
56
+ * How long to wait before calling it a no-answer. Default 15s.
57
+ *
58
+ * This is not belt-and-braces. Per the getUserMedia specification a prompt the user never
59
+ * dismisses may leave the promise permanently pending — "neither resolve nor reject" — and
60
+ * a page that awaits it in its connect path hangs on a state that has no error and no end.
61
+ * A deadline turns that into a reportable outcome.
62
+ */
63
+ timeoutMs?: number;
64
+ }
65
+ /**
66
+ * Why a microphone that was WORKING stopped: the device was unplugged, or access was revoked
67
+ * mid-call. LiveKit tries the default device first and mutes the track when that fails too.
68
+ * Only a live call can report this; `enableMicrophone` never does.
69
+ */
70
+ type MicrophoneProblemReason = MicrophoneFailureReason | "device-lost";
71
+ /** A microphone problem as a value: the cause, what the browser said, and what to do. */
72
+ type MicrophoneProblem = {
73
+ reason: MicrophoneProblemReason;
74
+ message: string;
75
+ hint: string;
76
+ };
77
+ /**
78
+ * Describe a microphone failure, whoever caught it. The one classifier behind both
79
+ * {@link enableMicrophone} and the React call handle's `microphone` state, so the two can never
80
+ * give the same failure different advice.
81
+ */
82
+ declare function describeMicrophoneFailure(error: unknown): MicrophoneProblem;
83
+ /**
84
+ * Publish the caller's microphone, and report WHY if it does not start.
85
+ *
86
+ * ```ts
87
+ * const mic = await enableMicrophone(room);
88
+ * if (!mic.ok) {
89
+ * status.textContent = `Microphone: ${mic.message}`;
90
+ * help.textContent = mic.hint; // the sentence that names the fix
91
+ * }
92
+ * ```
93
+ *
94
+ * Never throws. A call that cannot start the microphone is an ordinary outcome of asking a
95
+ * person for a device, not a programmer error, so it belongs in the return type.
96
+ *
97
+ * Note it does NOT hang up on failure — the call is still live and still billing, and only
98
+ * the caller knows whether a text-only session is useful to them. If it is not, disconnect
99
+ * in the `!ok` branch; leaving a room connected against a device the user never granted is
100
+ * how the retry ends up contending with the call that is still holding it.
101
+ */
102
+ declare function enableMicrophone(room: MicrophoneCapableRoom, options?: EnableMicrophoneOptions): Promise<MicrophoneResult>;
103
+
104
+ /**
105
+ * Receiver-side playout delay (SECONDS) for the avatar media tracks.
106
+ *
107
+ * The avatar is published from a GPU datacenter and reaches viewers over the
108
+ * public internet, where a few percent of packet loss is normal. A shallow
109
+ * receiver buffer stalls (freezes) on every loss while it waits for a
110
+ * retransmit; a deeper buffer recovers the lost packets BEFORE playout, so the
111
+ * viewer sees smooth 25fps instead of freezing. Measured at ~5% loss this took
112
+ * the stream from ~11fps with multi-second freezes to a steady 25fps with zero
113
+ * freezes.
114
+ *
115
+ * 0.5s is applied via the native `RemoteTrack.setPlayoutDelay`, which sets the
116
+ * receiver's `playoutDelayHint` — the same Chromium jitter-buffer knob the old
117
+ * hand-rolled `jitterBufferTarget` reached into (the spec renamed
118
+ * playoutDelayHint→jitterBufferTarget; both influence the same buffer depth),
119
+ * but typed and SDK-owned so there is no cast or feature-probe. The hint is NOT
120
+ * free at the open: the receiver holds the FIRST video frame toward the target
121
+ * (~250-450ms of measured TTFF), a deliberate cost paid once for the freeze-free
122
+ * steady state above. The open-small-then-ramp variant that tried to dodge that
123
+ * cost was shipped, failed twice (the rVFC ramp trigger is throttled on the
124
+ * opacity:0-mounted element, stranding the shallow buffer), and was deliberately
125
+ * removed in 2262e4c7b — do not re-propose it. No-op on browsers that don't
126
+ * support the hint (the SDK warns and moves on).
127
+ *
128
+ * This lives in the vanilla browser package, not the React one, because the
129
+ * React surface applies it for you and a plain-DOM page has to be given it.
130
+ * Until this file existed the helper was only reachable through
131
+ * `realtime-avatar/react`, so every plain-DOM adopter — every one of this repo's
132
+ * own demos included — ran with the browser's default shallow buffer and froze on
133
+ * every lost packet. Nobody should have to know this knob exists: `prepareAvatarRoom`
134
+ * (and `attachRemoteAudio`, which calls it) applies it to every track the room
135
+ * subscribes, and `RealtimeAvatarLiveKitRoom` does the same for React. This file is
136
+ * the one place the number and its reasoning live.
137
+ * Structural like its siblings: no `livekit-client` import, no version pin.
138
+ */
139
+ declare const DEFAULT_AVATAR_PLAYOUT_DELAY_SECONDS = 0.5;
140
+ /**
141
+ * A track that can take a native playout-delay hint — livekit-client's subscribed
142
+ * `RemoteTrack`. Duck-typed on the method rather than `instanceof`, so a
143
+ * placeholder or local track is skipped and the helper is testable with a fake.
144
+ */
145
+ interface PlayoutDelayTarget {
146
+ setPlayoutDelay(delayInSeconds: number): void;
147
+ }
148
+ /**
149
+ * Apply the receiver playout delay to ONE subscribed track.
150
+ *
151
+ * Call it for every track `RoomEvent.TrackSubscribed` hands you, audio and video
152
+ * alike, before or after `attach()` — WebRTC syncs the pair to the LARGER hint,
153
+ * so applying it to one stream and not the other lets the lips lag the voice.
154
+ * Returns `true` when the hint was applied and `false` when the object cannot take
155
+ * one (a local track, a placeholder, a not-yet-subscribed publication), so a
156
+ * caller can log the miss without a try/catch. The delay is floored at 0 so a
157
+ * negative input can never throw.
158
+ */
159
+ declare function applyPlayoutDelay(track: unknown, delaySeconds?: number): boolean;
160
+ /**
161
+ * Apply the SAME native playout delay to the avatar's audio + video tracks
162
+ * (kept equal so the a/v sync stays lip-locked). Pure and exported so the clamp
163
+ * and the both-tracks-equal invariant are unit-testable without a DOM; the React
164
+ * hook `useAvatarPlayoutDelay` is a thin effect wrapper around this.
165
+ */
166
+ declare function applyAvatarPlayoutDelay(videoTrack: unknown, audioTrack: unknown, delaySeconds?: number): void;
167
+
168
+ export { DEFAULT_AVATAR_PLAYOUT_DELAY_SECONDS as D, type EnableMicrophoneOptions as E, type MicrophoneCapableRoom as M, type PlayoutDelayTarget as P, type MicrophoneFailureReason as a, type MicrophoneProblem as b, type MicrophoneProblemReason as c, type MicrophoneResult as d, applyAvatarPlayoutDelay as e, applyPlayoutDelay as f, describeMicrophoneFailure as g, enableMicrophone as h };