realtime-voice-agents 2.0.1 → 2.2.0

package/README.md

@@ -104,6 +104,26 @@ geminiLive({ model: 'gemini-2.5-flash-native-audio-preview-12-2025', voice: 'Aoe
 
 One `SessionOptions` surface configures all three; where a provider can't honor a knob, the fallback is documented and pinned by the parity test suite.
 
+## Provider fallbacks
+
+One bad API key, an exhausted quota, or a provider outage should not send your calls to dead air. Give the bridge backup providers and it tries them in order while the call is being established:
+
+```ts
+const bridge = new TwilioRealtimeBridge({
+  agent,
+  provider: openaiRealtime(), // primary
+  fallbacks: [xaiRealtime(), geminiLive()], // tried in order if it fails to come up
+});
+```
+
+- **Covers the real failure modes.** A missing API key (the factories defer their credential check to call time precisely so the chain can absorb it), an expired/revoked key (HTTP 401), exhausted credits/quota (403/429), a provider internal error (5xx or a dropped socket), and a hung endpoint (connect timeout) all walk the chain — anything that keeps a provider from coming up.
+- **Connect-time only.** A dead provider is dropped and the next one is tried immediately — no backoff between attempts. Once a provider answers, the call stays with it: mid-call reconnects reuse the same provider (per the `session.reconnect` policy), and a mid-call death past that budget fails the call rather than switching voices mid-conversation.
+- **Observable.** Each advance emits `provider.fallback` (`{ from, to, error }`) on the session — count these to alarm on a degraded primary.
+- **Voices don't cross vendors.** Configure the voice per factory (`openaiRealtime({ voice: 'marin' })`, `xaiRealtime({ voice: 'eve' })`) rather than on the `Agent` — an OpenAI voice name would fail the xAI/Gemini connect and the chain would skip past a healthy provider.
+- **Latency.** Each dead provider costs up to its `connectTimeoutMs` (default 10s) before the next is tried — set a tighter one on the primary if its endpoint tends to hang rather than refuse. A [pre-synthesized greeting](#pre-synthesized-greeting-15s-to-first-word) bursts onto the line before any handshake, so the caller hears a voice while the chain walks.
+
+Testing it: `FakeOpenAIServer.start({ refuseConnections: true })` gives you a provider that is "down", and `{ rejectUpgrade: { status: 401, body: 'invalid_api_key' } }` one that rejects like a real auth/quota failure (both flippable at runtime to script recoveries) — see `src/bridge/fallback.test.ts` for ready-made scenarios.
+
 ## Tools: Zod schemas + execution strategies
 
 ```ts
@@ -193,6 +213,41 @@ session: {
 
 Blocked attempts emit `interruption.blocked` with a cause (`guard` | `rate_limit` | `tool_running` | …). Honored ones flush Twilio, truncate the model's context to the heard milliseconds, and emit `playback.interrupted` with exactly how much the caller heard.
 
+## Noise-adaptive VAD (opt-in)
+
+Server VAD tuned for quiet rooms misfires on noisy lines — street, car, speakerphone — as phantom barge-ins and chopped replies. `noiseAdaptiveVad` measures the line itself: a per-frame μ-law meter estimates the caller's noise floor (a low percentile over a sliding window, so speech doesn't read as noise), and when it stays high, the bridge escalates turn detection mid-call.
+
+```ts
+session: {
+  vad: { type: 'server', threshold: 0.5 },
+  noiseAdaptiveVad: {},                  // {} = defaults below
+  // mode: 'auto',                       // 'suggest' = events only, you apply
+  // noiseFloorDb: -45,                  // trigger floor, dBFS
+  // windowMs: 5000, sustainMs: 3000,    // how much/how long analyzed audio
+  // cooldownMs: 15_000, maxSteps: 1,    // escalation pacing (per call)
+  // thresholdStep: 0.1, maxThreshold: 0.9,
+}
+```
+
+How a step lands, per provider:
+
+| Provider | Escalation |
+| --- | --- |
+| OpenAI | `threshold` +0.1/step, auto-applied via ack-gated `session.update`; baseline = your `vad.threshold`, else OpenAI's documented 0.5 |
+| xAI | same, from xAI's documented **0.85** default, clamped to its 0.1–0.9 range |
+| Gemini | `vad.suggestion` event only (`startSensitivity: 'low'` analog) — the Live API has no mid-session config updates |
+| semantic VAD | `vad.suggestion` event only: `eagerness: 'low'` trades **end-of-turn latency** (waits up to ~8s) for stability, so that call is yours |
+
+The baseline comes from the provider's ACKnowledged effective config plus its declared `capabilities.vadTuning` profile — never guessed (assuming 0.5 on xAI would *lower* its 0.85 default). Escalation is one-way per call: steps up, never back down; `maxSteps` and `cooldownMs` bound the blast radius.
+
+Every decision emits `vad.suggestion` (recommended config + `noiseFloorDb`/`analyzedMs`/`elapsedMs` metrics); an applied-and-acknowledged step also emits `vad.adjusted`. In `mode: 'suggest'` nothing is applied automatically — accept with `session.updateVad(info.suggested)`, which persists across reconnects and rebases future escalation on top of it. `updateVad(null)` disables turn detection AND suspends adaptation (it never re-enables VAD by itself).
+
+Fine print:
+
+- Caller speech, agent playback (speakerphone bleed), and the pre-synthesized greeting are excluded from the floor estimate — a long monologue in a quiet room never escalates. `windowMs` counts **analyzed** idle-line audio, so warmup can span 30–60s of real conversation; the metrics exist to tune this from field data.
+- Enabling the feature also serializes ALL mid-call session updates for that session: one in flight, acknowledged before the next; an ack timeout reconnects into known-good state. Disabled = the legacy fire-and-forget behavior, untouched.
+- Complementary knobs: OpenAI's native `audio.input.noise_reduction` (reachable via the provider's `sessionOptions`) runs before VAD and may fix much of the problem upstream; `interruptions.rateLimit` reacts to barge-in churn after the fact, while this reacts to the audio itself. All three coexist.
+
 ## Pre-synthesized greeting (~1.5s to first word)
 
 The slowest part of answering is the provider handshake. Pre-record the greeting once, and the bridge burst-writes it onto the call **while the session is still connecting** — then keeps the model from greeting twice (instruction reinforcement + assistant-turn seeding + suppressed auto-greet) and gates caller audio until Twilio's mark confirms playout.
@@ -217,7 +272,7 @@ Bundled presets (all synthesized, license-free, seamless loops): `elevator-jazz`
 
 ## Events (session)
 
-`call.started/ended/failed` · `provider.connected/reconnecting/reconnected/closed` · `agent.speech.started/ended` (generation) · **`playback.started/finished/interrupted`** (what the caller heard, mark-confirmed) · `user.speech.started/ended` · `transcript.user/agent` · `tool.started/completed/failed` · `tool.approval.required` · `agent.handoff` · `interruption` / `interruption.blocked` · `background_audio.started/stopped` · `dtmf` · `usage.updated` · `error`.
+`call.started/ended/failed` · `provider.connected/fallback/reconnecting/reconnected/closed` · `agent.speech.started/ended` (generation) · **`playback.started/finished/interrupted`** (what the caller heard, mark-confirmed) · `user.speech.started/ended` · `transcript.user/agent` · `tool.started/completed/failed` · `tool.approval.required` · `agent.handoff` · `interruption` / `interruption.blocked` · `vad.suggestion` / `vad.adjusted` (noise-adaptive VAD) · `background_audio.started/stopped` · `dtmf` · `usage.updated` · `error`.
 
 ```ts
 bridge.on('session.started', (session) => {
@@ -244,6 +299,7 @@ session: {
   reconnect: { maxAttempts: 5, initialDelayMs: 250, maxDelayMs: 8000, jitter: true },
   hangup: { markTimeoutMs: 7000 },              // goodbye watchdog
   vad: undefined,                               // normalized VAD, mapped per provider
+  noiseAdaptiveVad: undefined,                  // opt-in noise → VAD escalation ({} enables; see its section)
   toolResultDelivery: 'afterPlayback',          // or 'immediate'
   toolBackgroundAudio: undefined,               // default hold audio for tools
   handoffVoicePolicy: 'keep',                   // or 'reconnect' to switch voices
@@ -260,7 +316,7 @@ Outbound calls: the greeting waits for a human — feed your status callback int
 `realtime-voice-agents/testing` ships the harness this package is tested with:
 
 - **`FakeTwilioMediaStream`** — a scripted caller with an exact playout simulation: marks echo only after the media before them "plays"; `clear` discards buffered audio and echoes pending marks, like real Twilio.
-- **`FakeOpenAIServer`** — a real-WebSocket GA-protocol server you script (`sendAudioResponse`, `sendToolCall`, `sendSpeechStarted`, drops).
+- **`FakeOpenAIServer`** — a real-WebSocket GA-protocol server you script (`sendAudioResponse`, `sendToolCall`, `sendSpeechStarted`, drops, `refuseConnections` for down-provider/fallback scenarios).
 - **`FakeGeminiLive`** — a scripted `@google/genai` seam for the Gemini provider.
 
 ```ts

package/dist/{BaseRealtimeProvider-BehPNT1r.d.cts → BaseRealtimeProvider-BL75_HHh.d.cts}

@@ -17,6 +17,21 @@ declare const noopLogger: Logger;
 declare function consoleLogger(context?: Record<string, unknown>): Logger;
 //#endregion
 //#region src/providers/base/capabilities.d.ts
+/**
+ * The provider's server-VAD tuning envelope. Declared by provider factories
+ * (never by the generic OpenAI-compatible base — an arbitrary compatible
+ * service must not silently inherit OpenAI's defaults). Noise-adaptive VAD
+ * refuses to invent a threshold baseline: absent both this profile and an
+ * explicit configured threshold, the numeric ladder stays unavailable.
+ */
+interface VadTuningProfile {
+  /** The provider's effective server-VAD threshold when none is configured. */
+  defaultServerThreshold?: number;
+  /** Lowest threshold the provider accepts (clamp target). */
+  minServerThreshold?: number;
+  /** Highest threshold the provider accepts (clamp target). */
+  maxServerThreshold?: number;
+}
 /**
  * Capability flags per provider. The engine branches on these — never on a
  * provider's name — so new providers slot in without touching session logic.
@@ -43,6 +58,11 @@ interface ProviderCapabilities {
    * Optional so existing third-party providers keep compiling (absent = false).
    */
   vadInterruptControl?: boolean;
+  /**
+   * Server-VAD threshold envelope for noise-adaptive tuning. Optional; absent
+   * means the provider declared none (see VadTuningProfile).
+   */
+  vadTuning?: VadTuningProfile;
 }
 //#endregion
 //#region src/providers/base/events.d.ts
@@ -182,6 +202,13 @@ interface ProviderSessionInit {
    * (documented fallback: guard protects buffered audio only).
    */
   bridgeOwnsInterruptions?: boolean;
+  /**
+   * Serialize mid-session updates: one session.update in flight at a time,
+   * each acknowledged (or timed out) before the next is sent. Set by the
+   * bridge when noise-adaptive VAD is configured; absent/false keeps the
+   * legacy fire-and-forget update behavior exactly as before.
+   */
+  serializedSessionUpdates?: boolean;
   /** Provider-native session options, deep-merged last (escape hatch). */
   providerOptions?: Record<string, unknown>;
 }
@@ -194,6 +221,15 @@ interface SendTextOptions {
 interface SendToolResultOptions {
   triggerResponse?: boolean;
 }
+interface SessionUpdateOptions {
+  /**
+   * Resolve with the provider's acknowledgement of THIS update (`true`) rather
+   * than at send time. Meaningful only with serialized session updates (see
+   * `ProviderSessionInit.serializedSessionUpdates`); providers without ack
+   * support resolve `false`/`void`, which callers treat as not-acknowledged.
+   */
+  awaitAck?: boolean;
+}
 /**
  * The provider contract: μ-law in, μ-law out, normalized events.
  * Transcoding (when the provider speaks PCM) lives inside the provider, so
@@ -217,8 +253,22 @@ declare abstract class BaseRealtimeProvider extends TypedEmitter<ProviderEvents>
   abstract createResponse(options?: {
     instructions?: string;
   }): void;
-  /** Mid-session config change (instructions/tools/voice) where supported. */
-  abstract updateSession(patch: Partial<ProviderSessionInit>): Promise<void>;
+  /**
+   * Mid-session config change (instructions/tools/voice/vad) where supported.
+   * Return `true` to signal the provider ACKNOWLEDGED the update — the
+   * noise-adaptive VAD auto-commit requires it. The `boolean | void` return
+   * keeps existing third-party `Promise<void>` implementations compiling;
+   * anything other than `true` is treated as not-acknowledged.
+   */
+  abstract updateSession(patch: Partial<ProviderSessionInit>, options?: SessionUpdateOptions): Promise<boolean | void>;
+  /**
+   * The last ACKNOWLEDGED turn-detection config on the live session — what was
+   * actually sent and confirmed, never desired/pending state. Undefined until
+   * the provider reports one (third-party providers may never set it).
+   */
+  getEffectiveVad(): VadConfig | null | undefined;
+  /** Set by subclasses when a config carrying `vad` is acknowledged. */
+  protected effectiveVadValue: VadConfig | null | undefined;
   /** Interrupt in-flight generation (where the wire protocol supports it). */
   cancelResponse(): void;
   /** Trim the last assistant item to what the caller actually heard. */
@@ -236,4 +286,4 @@ interface ProviderFactoryContext {
 /** A fresh provider per call; connection config is captured in the factory closure. */
 type ProviderFactory = (context: ProviderFactoryContext) => BaseRealtimeProvider;
 //#endregion
-export { noopLogger as _, ProviderToolSchema as a, VadConfig as c, ProviderEvents as d, ProviderToolCall as f, consoleLogger as g, Logger as h, ProviderSessionInit as i, ProviderAudioDelta as l, ProviderCapabilities as m, ProviderFactory as n, SendTextOptions as o, ProviderUsage as p, ProviderFactoryContext as r, SendToolResultOptions as s, BaseRealtimeProvider as t, ProviderCloseInfo as u };
+export { Logger as _, ProviderToolSchema as a, SessionUpdateOptions as c, ProviderCloseInfo as d, ProviderEvents as f, VadTuningProfile as g, ProviderCapabilities as h, ProviderSessionInit as i, VadConfig as l, ProviderUsage as m, ProviderFactory as n, SendTextOptions as o, ProviderToolCall as p, ProviderFactoryContext as r, SendToolResultOptions as s, BaseRealtimeProvider as t, ProviderAudioDelta as u, consoleLogger as v, noopLogger as y };

package/dist/{BaseRealtimeProvider-DI4pKtOb.cjs → BaseRealtimeProvider-C2mRn1V_.cjs}

@@ -46,6 +46,16 @@ function deepMerge(base, patch) {
 * instance — the session layer owns reconnect policy and context re-injection.
 */
 var BaseRealtimeProvider = class extends require_events.TypedEmitter {
+	/**
+	* The last ACKNOWLEDGED turn-detection config on the live session — what was
+	* actually sent and confirmed, never desired/pending state. Undefined until
+	* the provider reports one (third-party providers may never set it).
+	*/
+	getEffectiveVad() {
+		return this.effectiveVadValue;
+	}
+	/** Set by subclasses when a config carrying `vad` is acknowledged. */
+	effectiveVadValue = void 0;
 	/** Interrupt in-flight generation (where the wire protocol supports it). */
 	cancelResponse() {}
 	/** Trim the last assistant item to what the caller actually heard. */

package/dist/{BaseRealtimeProvider-BQigr5mB.mjs → BaseRealtimeProvider-C9C3s8jx.mjs}

@@ -46,6 +46,16 @@ function deepMerge(base, patch) {
 * instance — the session layer owns reconnect policy and context re-injection.
 */
 var BaseRealtimeProvider = class extends TypedEmitter {
+	/**
+	* The last ACKNOWLEDGED turn-detection config on the live session — what was
+	* actually sent and confirmed, never desired/pending state. Undefined until
+	* the provider reports one (third-party providers may never set it).
+	*/
+	getEffectiveVad() {
+		return this.effectiveVadValue;
+	}
+	/** Set by subclasses when a config carrying `vad` is acknowledged. */
+	effectiveVadValue = void 0;
 	/** Interrupt in-flight generation (where the wire protocol supports it). */
 	cancelResponse() {}
 	/** Trim the last assistant item to what the caller actually heard. */

package/dist/{BaseRealtimeProvider-ClP8Wx1X.d.mts → BaseRealtimeProvider-CWJ81HIt.d.mts}

@@ -17,6 +17,21 @@ declare const noopLogger: Logger;
 declare function consoleLogger(context?: Record<string, unknown>): Logger;
 //#endregion
 //#region src/providers/base/capabilities.d.ts
+/**
+ * The provider's server-VAD tuning envelope. Declared by provider factories
+ * (never by the generic OpenAI-compatible base — an arbitrary compatible
+ * service must not silently inherit OpenAI's defaults). Noise-adaptive VAD
+ * refuses to invent a threshold baseline: absent both this profile and an
+ * explicit configured threshold, the numeric ladder stays unavailable.
+ */
+interface VadTuningProfile {
+  /** The provider's effective server-VAD threshold when none is configured. */
+  defaultServerThreshold?: number;
+  /** Lowest threshold the provider accepts (clamp target). */
+  minServerThreshold?: number;
+  /** Highest threshold the provider accepts (clamp target). */
+  maxServerThreshold?: number;
+}
 /**
  * Capability flags per provider. The engine branches on these — never on a
  * provider's name — so new providers slot in without touching session logic.
@@ -43,6 +58,11 @@ interface ProviderCapabilities {
    * Optional so existing third-party providers keep compiling (absent = false).
    */
   vadInterruptControl?: boolean;
+  /**
+   * Server-VAD threshold envelope for noise-adaptive tuning. Optional; absent
+   * means the provider declared none (see VadTuningProfile).
+   */
+  vadTuning?: VadTuningProfile;
 }
 //#endregion
 //#region src/providers/base/events.d.ts
@@ -182,6 +202,13 @@ interface ProviderSessionInit {
    * (documented fallback: guard protects buffered audio only).
    */
   bridgeOwnsInterruptions?: boolean;
+  /**
+   * Serialize mid-session updates: one session.update in flight at a time,
+   * each acknowledged (or timed out) before the next is sent. Set by the
+   * bridge when noise-adaptive VAD is configured; absent/false keeps the
+   * legacy fire-and-forget update behavior exactly as before.
+   */
+  serializedSessionUpdates?: boolean;
   /** Provider-native session options, deep-merged last (escape hatch). */
   providerOptions?: Record<string, unknown>;
 }
@@ -194,6 +221,15 @@ interface SendTextOptions {
 interface SendToolResultOptions {
   triggerResponse?: boolean;
 }
+interface SessionUpdateOptions {
+  /**
+   * Resolve with the provider's acknowledgement of THIS update (`true`) rather
+   * than at send time. Meaningful only with serialized session updates (see
+   * `ProviderSessionInit.serializedSessionUpdates`); providers without ack
+   * support resolve `false`/`void`, which callers treat as not-acknowledged.
+   */
+  awaitAck?: boolean;
+}
 /**
  * The provider contract: μ-law in, μ-law out, normalized events.
  * Transcoding (when the provider speaks PCM) lives inside the provider, so
@@ -217,8 +253,22 @@ declare abstract class BaseRealtimeProvider extends TypedEmitter<ProviderEvents>
   abstract createResponse(options?: {
     instructions?: string;
   }): void;
-  /** Mid-session config change (instructions/tools/voice) where supported. */
-  abstract updateSession(patch: Partial<ProviderSessionInit>): Promise<void>;
+  /**
+   * Mid-session config change (instructions/tools/voice/vad) where supported.
+   * Return `true` to signal the provider ACKNOWLEDGED the update — the
+   * noise-adaptive VAD auto-commit requires it. The `boolean | void` return
+   * keeps existing third-party `Promise<void>` implementations compiling;
+   * anything other than `true` is treated as not-acknowledged.
+   */
+  abstract updateSession(patch: Partial<ProviderSessionInit>, options?: SessionUpdateOptions): Promise<boolean | void>;
+  /**
+   * The last ACKNOWLEDGED turn-detection config on the live session — what was
+   * actually sent and confirmed, never desired/pending state. Undefined until
+   * the provider reports one (third-party providers may never set it).
+   */
+  getEffectiveVad(): VadConfig | null | undefined;
+  /** Set by subclasses when a config carrying `vad` is acknowledged. */
+  protected effectiveVadValue: VadConfig | null | undefined;
   /** Interrupt in-flight generation (where the wire protocol supports it). */
   cancelResponse(): void;
   /** Trim the last assistant item to what the caller actually heard. */
@@ -236,4 +286,4 @@ interface ProviderFactoryContext {
 /** A fresh provider per call; connection config is captured in the factory closure. */
 type ProviderFactory = (context: ProviderFactoryContext) => BaseRealtimeProvider;
 //#endregion
-export { noopLogger as _, ProviderToolSchema as a, VadConfig as c, ProviderEvents as d, ProviderToolCall as f, consoleLogger as g, Logger as h, ProviderSessionInit as i, ProviderAudioDelta as l, ProviderCapabilities as m, ProviderFactory as n, SendTextOptions as o, ProviderUsage as p, ProviderFactoryContext as r, SendToolResultOptions as s, BaseRealtimeProvider as t, ProviderCloseInfo as u };
+export { Logger as _, ProviderToolSchema as a, SessionUpdateOptions as c, ProviderCloseInfo as d, ProviderEvents as f, VadTuningProfile as g, ProviderCapabilities as h, ProviderSessionInit as i, VadConfig as l, ProviderUsage as m, ProviderFactory as n, SendTextOptions as o, ProviderToolCall as p, ProviderFactoryContext as r, SendToolResultOptions as s, BaseRealtimeProvider as t, ProviderAudioDelta as u, consoleLogger as v, noopLogger as y };

package/dist/{GeminiLiveProvider-DvkTgzjG.d.cts → GeminiLiveProvider-Cc1dkxW6.d.cts}

@@ -1,4 +1,4 @@
-import { c as VadConfig, h as Logger, i as ProviderSessionInit, m as ProviderCapabilities, o as SendTextOptions, s as SendToolResultOptions, t as BaseRealtimeProvider } from "./BaseRealtimeProvider-BehPNT1r.cjs";
+import { _ as Logger, h as ProviderCapabilities, i as ProviderSessionInit, l as VadConfig, o as SendTextOptions, s as SendToolResultOptions, t as BaseRealtimeProvider } from "./BaseRealtimeProvider-BL75_HHh.cjs";
 //#region src/providers/gemini/GeminiLiveProvider.d.ts
 /** Structural slice of @google/genai's live session (also what fakes implement). */
 interface GeminiLiveSessionLike {

package/dist/{GeminiLiveProvider-x2nyx5aO.d.mts → GeminiLiveProvider-TvY_cQQZ.d.mts}

@@ -1,4 +1,4 @@
-import { c as VadConfig, h as Logger, i as ProviderSessionInit, m as ProviderCapabilities, o as SendTextOptions, s as SendToolResultOptions, t as BaseRealtimeProvider } from "./BaseRealtimeProvider-ClP8Wx1X.mjs";
+import { _ as Logger, h as ProviderCapabilities, i as ProviderSessionInit, l as VadConfig, o as SendTextOptions, s as SendToolResultOptions, t as BaseRealtimeProvider } from "./BaseRealtimeProvider-CWJ81HIt.mjs";
 //#region src/providers/gemini/GeminiLiveProvider.d.ts
 /** Structural slice of @google/genai's live session (also what fakes implement). */
 interface GeminiLiveSessionLike {

package/dist/{OpenAICompatibleProvider-Bdtl-UXH.mjs → OpenAICompatibleProvider-D-2OOVBU.mjs}

@@ -1,5 +1,5 @@
-import { a as noopLogger, t as BaseRealtimeProvider } from "./BaseRealtimeProvider-BQigr5mB.mjs";
-import { t as buildSessionUpdate } from "./session-config-BVLl7-ha.mjs";
+import { a as noopLogger, t as BaseRealtimeProvider } from "./BaseRealtimeProvider-C9C3s8jx.mjs";
+import { t as buildSessionUpdate } from "./session-config-CbifLlkV.mjs";
 import WebSocket$1 from "ws";
 //#region src/providers/openai-compatible/OpenAICompatibleProvider.ts
 /**
@@ -41,6 +41,10 @@ var OpenAICompatibleProvider = class extends BaseRealtimeProvider {
 	responseActive = false;
 	pendingCreate = null;
 	lastCreateSent = null;
+	/** Serialized-update pipeline (enabled via init.serializedSessionUpdates). */
+	serializedUpdates = false;
+	updateQueue = [];
+	updateInFlight = null;
 	constructor(config, logger = noopLogger) {
 		super();
 		this.config = config;
@@ -62,14 +66,11 @@ var OpenAICompatibleProvider = class extends BaseRealtimeProvider {
 	}
 	async connect(init) {
 		if (this.ws) await this.close();
-		let vad = init.vad !== void 0 ? init.vad : this.config.defaultVad;
-		if (init.bridgeOwnsInterruptions && this.capabilities.vadInterruptControl && vad !== null) vad = {
-			interruptResponse: false,
-			...vad ?? { type: "server" }
-		};
+		this.failPendingUpdates();
+		this.serializedUpdates = init.serializedSessionUpdates === true;
 		this.sessionInit = {
 			...init,
-			vad,
+			vad: this.resolveVad(init.vad, init.bridgeOwnsInterruptions),
 			transcription: init.transcription !== void 0 ? init.transcription : this.config.defaultTranscription
 		};
 		this.intentionalClose = false;
@@ -148,6 +149,7 @@ var OpenAICompatibleProvider = class extends BaseRealtimeProvider {
 			ws.on("close", (code, reasonBuf) => {
 				const reason = reasonBuf?.toString();
 				this.ready = false;
+				this.failPendingUpdates();
 				if (!settled) {
 					fail(/* @__PURE__ */ new Error(`${this.name} socket closed during setup (${code} ${reason ?? ""})`));
 					return;
@@ -159,6 +161,7 @@ var OpenAICompatibleProvider = class extends BaseRealtimeProvider {
 				});
 			});
 		});
+		this.effectiveVadValue = this.sessionInit.vad;
 	}
 	async close() {
 		this.intentionalClose = true;
@@ -239,13 +242,87 @@ var OpenAICompatibleProvider = class extends BaseRealtimeProvider {
 			...create.instructions ? { response: { instructions: create.instructions } } : {}
 		});
 	}
-	async updateSession(patch) {
+	async updateSession(patch, options = {}) {
 		if (!this.sessionInit) throw new Error("updateSession before connect");
+		let resolved = patch;
+		if ("vad" in patch) resolved = {
+			...patch,
+			vad: this.resolveVad(patch.vad, this.sessionInit.bridgeOwnsInterruptions)
+		};
 		this.sessionInit = {
 			...this.sessionInit,
-			...patch
+			...resolved
+		};
+		if (!this.serializedUpdates) {
+			const open = this.ws?.readyState === WebSocket$1.OPEN;
+			this.send(this.buildSessionPayload());
+			return open;
+		}
+		let resolveSent;
+		let resolveAck;
+		const sent = new Promise((resolve) => resolveSent = resolve);
+		const acked = new Promise((resolve) => resolveAck = resolve);
+		this.updateQueue.push({
+			payload: this.buildSessionPayload(),
+			touchesVad: "vad" in resolved,
+			resolvedVad: resolved.vad,
+			resolveSent,
+			resolveAck,
+			ackTimer: null
+		});
+		this.pumpUpdateQueue();
+		return options.awaitAck ? acked : sent;
+	}
+	/**
+	* Bridge-owned barge-in: the server must NOT auto-cancel the active
+	* response on speech onset, or a guard-blocked interruption still kills
+	* the sentence mid-air. An explicit vad.interruptResponse wins (spread
+	* order). Applied on connect AND on every vad patch.
+	*/
+	resolveVad(vad, bridgeOwnsInterruptions) {
+		let resolved = vad !== void 0 ? vad : this.config.defaultVad;
+		if (bridgeOwnsInterruptions && this.capabilities.vadInterruptControl && resolved !== null) resolved = {
+			interruptResponse: false,
+			...resolved ?? { type: "server" }
 		};
-		this.send(this.buildSessionPayload());
+		return resolved;
+	}
+	/** Send the next queued update once nothing is awaiting its ack. */
+	pumpUpdateQueue() {
+		if (this.updateInFlight || this.updateQueue.length === 0) return;
+		if (this.ws?.readyState !== WebSocket$1.OPEN) {
+			this.failPendingUpdates();
+			return;
+		}
+		const entry = this.updateQueue.shift();
+		this.updateInFlight = entry;
+		this.send(entry.payload);
+		entry.resolveSent(true);
+		const timeoutMs = this.config.sessionUpdateAckTimeoutMs ?? 3e3;
+		entry.ackTimer = setTimeout(() => {
+			entry.ackTimer = null;
+			if (this.updateInFlight !== entry) return;
+			this.updateInFlight = null;
+			entry.resolveAck(false);
+			this.logger.warn("session.update not acknowledged in time — desyncing the connection", { timeoutMs });
+			this.ws?.terminate();
+		}, timeoutMs);
+		entry.ackTimer.unref?.();
+	}
+	/** Settle every queued/in-flight update as not-acked (socket gone/replaced). */
+	failPendingUpdates() {
+		const inFlight = this.updateInFlight;
+		this.updateInFlight = null;
+		if (inFlight) {
+			if (inFlight.ackTimer) clearTimeout(inFlight.ackTimer);
+			inFlight.resolveAck(false);
+		}
+		const queued = this.updateQueue;
+		this.updateQueue = [];
+		for (const entry of queued) {
+			entry.resolveSent(false);
+			entry.resolveAck(false);
+		}
 	}
 	buildSessionPayload() {
 		return (this.config.buildSession ?? buildSessionUpdate)(this.sessionInit, {
@@ -267,6 +344,16 @@ var OpenAICompatibleProvider = class extends BaseRealtimeProvider {
 	}
 	handleEvent(event) {
 		switch (event.type) {
+			case "session.updated": {
+				const entry = this.updateInFlight;
+				if (!entry) break;
+				this.updateInFlight = null;
+				if (entry.ackTimer) clearTimeout(entry.ackTimer);
+				if (entry.touchesVad) this.effectiveVadValue = entry.resolvedVad;
+				entry.resolveAck(true);
+				this.pumpUpdateQueue();
+				break;
+			}
 			case "response.created": {
 				const responseId = event.response?.id ?? `resp_${Date.now()}`;
 				this.currentResponseId = responseId;

package/dist/{OpenAICompatibleProvider-NS4cKVQj.cjs → OpenAICompatibleProvider-Mp0Mefbh.cjs}

@@ -1,6 +1,6 @@
 const require_rolldown_runtime = require("./rolldown-runtime-VH7oDXx4.cjs");
-const require_BaseRealtimeProvider = require("./BaseRealtimeProvider-DI4pKtOb.cjs");
-const require_session_config = require("./session-config-c8sOw1XL.cjs");
+const require_BaseRealtimeProvider = require("./BaseRealtimeProvider-C2mRn1V_.cjs");
+const require_session_config = require("./session-config-CqJm2Kxz.cjs");
 let ws = require("ws");
 ws = require_rolldown_runtime.__toESM(ws, 1);
 //#region src/providers/openai-compatible/OpenAICompatibleProvider.ts
@@ -43,6 +43,10 @@ var OpenAICompatibleProvider = class extends require_BaseRealtimeProvider.BaseRe
 	responseActive = false;
 	pendingCreate = null;
 	lastCreateSent = null;
+	/** Serialized-update pipeline (enabled via init.serializedSessionUpdates). */
+	serializedUpdates = false;
+	updateQueue = [];
+	updateInFlight = null;
 	constructor(config, logger = require_BaseRealtimeProvider.noopLogger) {
 		super();
 		this.config = config;
@@ -64,14 +68,11 @@ var OpenAICompatibleProvider = class extends require_BaseRealtimeProvider.BaseRe
 	}
 	async connect(init) {
 		if (this.ws) await this.close();
-		let vad = init.vad !== void 0 ? init.vad : this.config.defaultVad;
-		if (init.bridgeOwnsInterruptions && this.capabilities.vadInterruptControl && vad !== null) vad = {
-			interruptResponse: false,
-			...vad ?? { type: "server" }
-		};
+		this.failPendingUpdates();
+		this.serializedUpdates = init.serializedSessionUpdates === true;
 		this.sessionInit = {
 			...init,
-			vad,
+			vad: this.resolveVad(init.vad, init.bridgeOwnsInterruptions),
 			transcription: init.transcription !== void 0 ? init.transcription : this.config.defaultTranscription
 		};
 		this.intentionalClose = false;
@@ -150,6 +151,7 @@ var OpenAICompatibleProvider = class extends require_BaseRealtimeProvider.BaseRe
 			ws$1.on("close", (code, reasonBuf) => {
 				const reason = reasonBuf?.toString();
 				this.ready = false;
+				this.failPendingUpdates();
 				if (!settled) {
 					fail(/* @__PURE__ */ new Error(`${this.name} socket closed during setup (${code} ${reason ?? ""})`));
 					return;
@@ -161,6 +163,7 @@ var OpenAICompatibleProvider = class extends require_BaseRealtimeProvider.BaseRe
 				});
 			});
 		});
+		this.effectiveVadValue = this.sessionInit.vad;
 	}
 	async close() {
 		this.intentionalClose = true;
@@ -241,13 +244,87 @@ var OpenAICompatibleProvider = class extends require_BaseRealtimeProvider.BaseRe
 			...create.instructions ? { response: { instructions: create.instructions } } : {}
 		});
 	}
-	async updateSession(patch) {
+	async updateSession(patch, options = {}) {
 		if (!this.sessionInit) throw new Error("updateSession before connect");
+		let resolved = patch;
+		if ("vad" in patch) resolved = {
+			...patch,
+			vad: this.resolveVad(patch.vad, this.sessionInit.bridgeOwnsInterruptions)
+		};
 		this.sessionInit = {
 			...this.sessionInit,
-			...patch
+			...resolved
+		};
+		if (!this.serializedUpdates) {
+			const open = this.ws?.readyState === ws.default.OPEN;
+			this.send(this.buildSessionPayload());
+			return open;
+		}
+		let resolveSent;
+		let resolveAck;
+		const sent = new Promise((resolve) => resolveSent = resolve);
+		const acked = new Promise((resolve) => resolveAck = resolve);
+		this.updateQueue.push({
+			payload: this.buildSessionPayload(),
+			touchesVad: "vad" in resolved,
+			resolvedVad: resolved.vad,
+			resolveSent,
+			resolveAck,
+			ackTimer: null
+		});
+		this.pumpUpdateQueue();
+		return options.awaitAck ? acked : sent;
+	}
+	/**
+	* Bridge-owned barge-in: the server must NOT auto-cancel the active
+	* response on speech onset, or a guard-blocked interruption still kills
+	* the sentence mid-air. An explicit vad.interruptResponse wins (spread
+	* order). Applied on connect AND on every vad patch.
+	*/
+	resolveVad(vad, bridgeOwnsInterruptions) {
+		let resolved = vad !== void 0 ? vad : this.config.defaultVad;
+		if (bridgeOwnsInterruptions && this.capabilities.vadInterruptControl && resolved !== null) resolved = {
+			interruptResponse: false,
+			...resolved ?? { type: "server" }
 		};
-		this.send(this.buildSessionPayload());
+		return resolved;
+	}
+	/** Send the next queued update once nothing is awaiting its ack. */
+	pumpUpdateQueue() {
+		if (this.updateInFlight || this.updateQueue.length === 0) return;
+		if (this.ws?.readyState !== ws.default.OPEN) {
+			this.failPendingUpdates();
+			return;
+		}
+		const entry = this.updateQueue.shift();
+		this.updateInFlight = entry;
+		this.send(entry.payload);
+		entry.resolveSent(true);
+		const timeoutMs = this.config.sessionUpdateAckTimeoutMs ?? 3e3;
+		entry.ackTimer = setTimeout(() => {
+			entry.ackTimer = null;
+			if (this.updateInFlight !== entry) return;
+			this.updateInFlight = null;
+			entry.resolveAck(false);
+			this.logger.warn("session.update not acknowledged in time — desyncing the connection", { timeoutMs });
+			this.ws?.terminate();
+		}, timeoutMs);
+		entry.ackTimer.unref?.();
+	}
+	/** Settle every queued/in-flight update as not-acked (socket gone/replaced). */
+	failPendingUpdates() {
+		const inFlight = this.updateInFlight;
+		this.updateInFlight = null;
+		if (inFlight) {
+			if (inFlight.ackTimer) clearTimeout(inFlight.ackTimer);
+			inFlight.resolveAck(false);
+		}
+		const queued = this.updateQueue;
+		this.updateQueue = [];
+		for (const entry of queued) {
+			entry.resolveSent(false);
+			entry.resolveAck(false);
+		}
 	}
 	buildSessionPayload() {
 		return (this.config.buildSession ?? require_session_config.buildSessionUpdate)(this.sessionInit, {
@@ -269,6 +346,16 @@ var OpenAICompatibleProvider = class extends require_BaseRealtimeProvider.BaseRe
 	}
 	handleEvent(event) {
 		switch (event.type) {
+			case "session.updated": {
+				const entry = this.updateInFlight;
+				if (!entry) break;
+				this.updateInFlight = null;
+				if (entry.ackTimer) clearTimeout(entry.ackTimer);
+				if (entry.touchesVad) this.effectiveVadValue = entry.resolvedVad;
+				entry.resolveAck(true);
+				this.pumpUpdateQueue();
+				break;
+			}
 			case "response.created": {
 				const responseId = event.response?.id ?? `resp_${Date.now()}`;
 				this.currentResponseId = responseId;