@breeze.blue/sdk 0.5.0 → 0.6.0

package/CHANGELOG.md

@@ -1,5 +1,51 @@
 # Changelog
 
+## 0.6.0
+
+- Added opt-in `textToSpeech.realtime.connectManaged(...)` for long logical
+  conversations. It derives session keepalive from `session.ready`, rotates
+  bounded physical WebSockets at turn boundaries, obtains fresh session
+  credentials for every epoch, and performs bounded jittered reconnects while
+  idle without replaying active turns. Replacement creation and retry delays
+  are cancellable,
+  and planned rotations briefly drain outstanding `usage.committed` events
+  from the old epoch without blocking the next turn.
+- Managed heartbeat now requires a timely server response, so a half-open idle
+  WebSocket is replaced with fresh credentials instead of accepting the next
+  turn. Active turns still fail as `TURN_INTERRUPTED` and are never replayed.
+  Physical epochs also become eligible for turn-boundary rotation after 10
+  minutes by default, independently of a longer server-advertised deadline.
+  An active turn can delay the switch; both safety windows are configurable.
+- Aligned managed realtime error handling with protocol semantics: terminal
+  auth, billing, and policy failures are not retried; recoverable turn errors
+  stay on the same socket; rejected concurrency admission clears the local
+  pending turn; transient 408, 425, 429, and 5xx replacement-session failures
+  use the bounded reconnect budget; a new turn on the healthy old epoch wins
+  even when an in-flight planned candidate fails; and active-turn transport
+  failures remain non-replayed. Shared upstream
+  `GENERATION_CAPACITY_EXCEEDED` waits for the corresponding
+  `turn.cancelled` and keeps the logical session usable.
+- Added typed `session.expiring` and realtime expiry fields. Managed browser
+  callers can provide a `sessionFactory` that fetches a fresh client secret
+  from their own backend for every physical connection.
+- Fixed the realtime `audio()` iterator so server errors and abnormal closes
+  throw `BreezeBlueRealtimeError` instead of ending silently, and preserved
+  close-code semantics when a WebSocket `error` event arrives just before its
+  `close` event.
+
+## 0.5.1
+
+- Added official voice publishing metadata to `voices.savePreview(...)` and
+  `voices.edit(...)`, including `primaryCategoryCode`, `visibility`, and the
+  typed `SavedVoice` response.
+- Aligned voice, history, and generation metadata with the public API by using
+  `origin` instead of the legacy category fields; voice search now accepts an
+  `origin` filter.
+- Made `voices.edit(...)` support metadata-only updates without requiring a
+  voice name.
+- Improved the realtime TTS example to start consuming before sending text and
+  continue until `turn.done`, so early audio is not missed.
+
 ## 0.5.0
 
 - Fixed `textToSpeech.realtime.connect(...)` so a failed WebSocket handshake

package/README.md

@@ -112,29 +112,35 @@ Realtime needs a `WebSocket` implementation. Browsers, edge runtimes, and
 Node 22+ provide a global one; on Node 20, pass a constructor (for example the
 `ws` package's `WebSocket`) via `new BreezeBlueClient({ webSocket })`.
 
+Start consuming before appending text. Audio can arrive after `flush()` and may
+continue after `endTurn()`; keep the single consumer running until `turn.done`.
+
 ```ts
 const connection = await client.textToSpeech.realtime.connect("voc_...", {
-  modelId: "bluebell-v1",
+  modelId: "breeze-tts-2",
 });
 
+const pcmChunks: Uint8Array[] = [];
+const consumer = (async () => {
+  for await (const message of connection) {
+    if (message.type === "audio") {
+      // Forward this chunk to your playback or transport layer immediately.
+      pcmChunks.push(message.audio);
+    } else if (message.type === "error") {
+      throw new Error(`Realtime TTS failed: ${message.code}: ${message.message}`);
+    } else if (message.type === "turn.done") {
+      return;
+    }
+  }
+  throw new Error("Realtime session closed before turn.done");
+})();
+
 connection.startTurn("turn_1");
 connection.appendText("Hello from Breeze.");
 connection.flush();
 connection.endTurn();
 
-const pcmChunks: Uint8Array[] = [];
-for await (const message of connection) {
-  if (message.type === "audio") {
-    pcmChunks.push(message.audio);
-    continue;
-  }
-  if (message.type === "error") {
-    throw new Error(`Realtime TTS failed: ${message.code}: ${message.message}`);
-  }
-  if (message.type === "turn.done") {
-    break; // all audio for this turn has been delivered
-  }
-}
+await consumer;
 connection.close();
 ```
 
@@ -142,33 +148,98 @@ Always consume the connection (or its `audio()` / `events()` iterators) while a
 turn is active, and call `close()` when you are done. When the SDK detects a
 connection failure — an invalid server frame, or more than 16 MiB of messages
 piling up unconsumed — it closes the connection and the iterator rejects with
-a `BreezeBlueRealtimeError`.
+a `BreezeBlueRealtimeError`. The `audio()` helper also throws
+`BreezeBlueRealtimeError` for server `error` events and abnormal WebSocket
+closes instead of silently ending.
 
 Events are a typed discriminated union (`session.ready`, `turn.started`,
 `audio.started`, `turn.done`, `turn.cancelled`, `usage.committed`,
-`session.closed`, `error`, `pong`), with camelCase fields such as `turnId`,
-`historyItemId`, and `ttfaMs`. Treat the union as non-exhaustive: the server
-may add event types, and the SDK delivers unknown JSON events unchanged —
-ignore event types you do not recognize instead of switching exhaustively.
+`session.expiring`, `session.closed`, `error`, `pong`), with camelCase fields
+such as `turnId`, `historyItemId`, `expiresAt`, and `ttfaMs`. Treat the union as
+non-exhaustive: the server may add event types, and the SDK delivers unknown
+JSON events unchanged — ignore event types you do not recognize instead of
+switching exhaustively.
 
-For connections that sit idle between turns, send a keepalive ping inside the
-session's `inactivityTimeoutSeconds` window; the server answers with a `pong`
-event:
+Send a keepalive ping inside the session's `inactivityTimeoutSeconds` window;
+the server answers with a `pong` event. Keep it running during active turns too:
+a long TTFA or upstream stall with no audio does not pause the idle deadline.
 
 ```ts
-const keepalive = setInterval(() => connection.ping(), 30_000);
+const keepalive = setInterval(() => connection.ping(), 10_000);
 // ... run turns ...
 clearInterval(keepalive);
 connection.close();
 ```
 
+For a long logical conversation, use the opt-in managed connection. It derives
+a safe keepalive interval from `session.ready` and requires a server response
+after each ping. A missing acknowledgement replaces an idle half-open socket;
+if a turn is active, it reports `TURN_INTERRUPTED` and never replays commands.
+The manager also marks a physical WebSocket for turn-boundary rotation after
+10 minutes by default (or earlier when the server deadline requires it). An
+active turn can delay the switch beyond that threshold; the server-advertised
+hard lifetime still applies. The manager
+performs up to three unexpected idle reconnect attempts per interruption with
+fresh sessions and jittered exponential backoff:
+
+```ts
+const connection = await client.textToSpeech.realtime.connectManaged("voc_...", {
+  modelId: "breeze-tts-2",
+});
+
+const consumer = (async () => {
+  for await (const message of connection) {
+    if (message.type === "audio") {
+      // Play or forward this PCM chunk immediately.
+    } else if (message.type === "session.ready") {
+      // A new physical epoch is ready; the logical connection remains the same.
+    } else if (message.type === "turn.done") {
+      return;
+    }
+  }
+})();
+
+await connection.startTurnWhenReady("turn_1");
+connection.appendText("Hello from a long-running conversation.");
+connection.endTurn();
+await consumer;
+connection.close();
+```
+
+Use `heartbeatTimeoutMs` to tune the acknowledgement deadline and
+`maxPhysicalSessionMs` to tune the client-side epoch cap. Set
+`maxPhysicalSessionMs: 0` only when you intentionally want to rely solely on
+the server-advertised deadline.
+
+When an idle reconnect is handled successfully, the logical iterator stays
+open, suppresses the replaced physical socket's terminal event, and emits the
+new epoch's `session.ready`. The replacement sends its first heartbeat
+immediately and resumes the derived cadence after an inbound acknowledgement.
+Reconnect attempts are bounded per interruption. `startTurnWhenReady(...)`
+waits only when one of these idle replacements is already in progress and
+`turn.start` has not been sent; await it before sending any other turn command.
+After a planned rotation, an old epoch with outstanding `usage.committed`
+events drains in parallel for up to five seconds and closes as soon as all
+known completed turns settle. This event is best-effort; use history and usage
+APIs as the durable source of truth.
+
+The managed connection never buffers turn content or replays a command. The
+bounded `startTurnWhenReady(...)` wait happens before its first WebSocket
+write. If a WebSocket is interrupted while a turn is active, its iterator rejects with
+`BreezeBlueRealtimeError` and `error.code === "TURN_INTERRUPTED"`; decide from
+your application conversation state whether and how to start a new turn.
+Treat the manager as an active realtime call, not a presence channel. Always
+call `connection.close()` when the call ends, the user leaves, or the page
+enters a long-lived background state; otherwise its heartbeat intentionally
+keeps a server WebSocket slot occupied.
+
 To cut time to first audio, create the session ahead of time (for example while
 your app is still preparing the turn) and connect with its `clientSecret` when
 the first text is ready — only the WebSocket handshake remains:
 
 ```ts
 const session = await client.textToSpeech.realtime.createSession("voc_...", {
-  modelId: "bluebell-v1",
+  modelId: "breeze-tts-2",
 });
 
 // Later, when the first text is ready:
@@ -189,16 +260,57 @@ const connection = await browserClient.textToSpeech.realtime.connect("voc_...",
 });
 ```
 
+For a managed browser connection, provide a `sessionFactory`. The SDK invokes it
+for every physical epoch, so the callback must fetch a newly minted secret from
+your backend rather than cache the first response. Session configuration belongs
+in that backend request when a custom factory is used:
+
+```ts
+import { BreezeBlueRealtimeError } from "@breeze.blue/sdk";
+
+const connection = await browserClient.textToSpeech.realtime.connectManaged("voc_...", {
+  sessionFactory: async ({ signal }) => {
+    const response = await fetch("/api/breeze-realtime-session", {
+      method: "POST",
+      signal,
+    });
+    if (!response.ok) {
+      const reconnect =
+        [408, 425, 429].includes(response.status) || response.status >= 500;
+      throw new BreezeBlueRealtimeError("Could not create realtime session", {
+        code: "SESSION_FACTORY_ERROR",
+        reconnect,
+      });
+    }
+    return response.json(); // { clientSecret, websocketUrl? }
+  },
+});
+```
+
+Honor the factory `signal`: the SDK aborts it when the logical connection closes
+or the physical-epoch timeout expires. A custom callback that ignores the
+signal must still enforce its own bounded request timeout.
+
 Session parameters (`modelId`, `languageCode`, `instructions`, `voiceSettings`,
 `inactivityTimeoutSeconds`, `enableLogging`) are fixed when the session is
 created. `connect` ignores them when `clientSecret` or `websocketUrl` is
 provided and logs a warning.
 
 If a realtime WebSocket is interrupted by a network change, service deployment,
-or upstream realtime worker restart, handle `error.meta.reconnect === true` or a
-`session.closed` event with `reconnect === true` by creating a new connection and
-starting a new turn from your own conversation state. Active turns are not
-resumed in place.
+or upstream realtime worker restart, `connectManaged(...)` handles bounded
+reconnects only while there is no active turn. With the lower-level
+`connect(...)`, handle `error.meta.reconnect === true` or a `session.closed`
+event with `reconnect === true` by creating a new session and starting a new
+turn from your own conversation state. Active turns are never resumed in place.
+During a managed reconnect, transient session-creation responses (408, 425, 429,
+and 5xx) share the same bounded reconnect budget; other 4xx responses fail the
+logical connection immediately. A custom `sessionFactory` should preserve that
+distinction with `new BreezeBlueRealtimeError(message, { reconnect })`; a plain
+`Error` cannot communicate a terminal policy response and is treated as
+transient inside the same bounded budget. `GENERATION_CAPACITY_EXCEEDED` is
+turn-scoped: wait for the following `turn.cancelled`, back off using
+`meta.retryAfterSeconds`, and start a new turn on the same managed logical
+connection without replaying text.
 
 The API uses the default text-to-speech model when `modelId` is omitted. If
 you need to select a model explicitly, call `client.models.list()` and pass one

package/dist/client.d.ts

@@ -1,5 +1,5 @@
 import { AudioResponse } from "./audio.js";
-import type { AudioRequestOptions, BreezeBlueWebSocket, AsyncTextToSpeechJob, Balance, BreezeBlueClientOptions, GenericStatus, GenerationJob, HistoryItem, HistoryList, HistoryListParams, Model, RequestOptions, RealtimeTextToSpeechConnectOptions, RealtimeTextToSpeechEvent, RealtimeTextToSpeechMessage, RealtimeTextToSpeechSession, RealtimeTextToSpeechSessionRequest, SavedVoice, SaveVoiceRequest, StreamTextToSpeechOptions, TextToSpeechRequest, TtsEnhanceRequest, TtsEnhanceResponse, Usage, UsageParams, Voice, VoiceClonePreview, VoiceClonePreviewRequest, VoiceDesignRequest, VoiceDesignResponse, VoiceEditRequest, VoiceList, VoiceSearchParams, VoiceSettings } from "./types.js";
+import type { AudioRequestOptions, BreezeBlueWebSocket, AsyncTextToSpeechJob, Balance, BreezeBlueClientOptions, GenericStatus, GenerationJob, HistoryItem, HistoryList, HistoryListParams, Model, RequestOptions, RealtimeTextToSpeechConnectOptions, RealtimeTextToSpeechEvent, RealtimeTextToSpeechManagedConnectOptions, RealtimeTextToSpeechMessage, RealtimeTextToSpeechSession, RealtimeTextToSpeechSessionRequest, SavedVoice, SaveVoiceRequest, StreamTextToSpeechOptions, TextToSpeechRequest, TtsEnhanceRequest, TtsEnhanceResponse, Usage, UsageParams, Voice, VoiceClonePreview, VoiceClonePreviewRequest, VoiceDesignRequest, VoiceDesignResponse, VoiceEditRequest, VoiceList, VoiceSearchParams, VoiceSettings } from "./types.js";
 export declare class BreezeBlueClient {
     readonly apiKey: string | undefined;
     readonly baseUrl: string;
@@ -33,6 +33,13 @@ declare class RealtimeTextToSpeechResource {
     constructor(client: BreezeBlueClient);
     createSession(voiceId: string, request?: RealtimeTextToSpeechSessionRequest, options?: RequestOptions): Promise<RealtimeTextToSpeechSession>;
     connect(voiceId: string, options?: RealtimeTextToSpeechConnectOptions): Promise<RealtimeTextToSpeechConnection>;
+    /**
+     * Open an SDK-managed logical connection backed by bounded physical
+     * WebSocket sessions. The manager keeps idle sessions alive, rotates at
+     * turn boundaries, and performs bounded idle reconnects with fresh
+     * credentials. It never replays an interrupted active turn.
+     */
+    connectManaged(voiceId: string, options?: RealtimeTextToSpeechManagedConnectOptions): Promise<ManagedRealtimeTextToSpeechConnection>;
 }
 export declare class RealtimeTextToSpeechConnection implements AsyncIterable<RealtimeTextToSpeechMessage> {
     private readonly socket;
@@ -41,9 +48,11 @@ export declare class RealtimeTextToSpeechConnection implements AsyncIterable<Rea
     private closed;
     private queuedBytes;
     private failure;
+    private closeEventGraceTimer;
     private constructor();
     static open(socket: BreezeBlueWebSocket, options?: {
         timeout?: number;
+        signal?: AbortSignal;
     }): Promise<RealtimeTextToSpeechConnection>;
     startTurn(turnId?: string): void;
     appendText(text: string): void;
@@ -59,9 +68,101 @@ export declare class RealtimeTextToSpeechConnection implements AsyncIterable<Rea
     private sendJson;
     private handleMessage;
     private handleClose;
+    private handleSocketError;
+    private push;
+    private fail;
+    private finish;
+    private clearCloseEventGraceTimer;
+}
+type RealtimeConnectionFactory = (signal: AbortSignal) => Promise<RealtimeTextToSpeechConnection>;
+/**
+ * A logical realtime TTS connection backed by bounded physical WebSocket
+ * epochs. Use `textToSpeech.realtime.connectManaged(...)` to create one.
+ *
+ * The manager automatically keeps an idle session alive, rotates before the
+ * server session limit at turn boundaries, and makes bounded reconnect
+ * attempts while idle. It does not buffer turn content or replay commands;
+ * startTurnWhenReady() may wait before its first turn.start write. If the
+ * transport is interrupted during a turn, iteration rejects with
+ * `BreezeBlueRealtimeError`.
+ */
+export declare class ManagedRealtimeTextToSpeechConnection implements AsyncIterable<RealtimeTextToSpeechMessage> {
+    private readonly createConnection;
+    private readonly queue;
+    private readonly drainingEpochs;
+    private readonly openAttempts;
+    private readonly waiters;
+    private readonly heartbeatIntervalMs;
+    private readonly heartbeatTimeoutMs;
+    private readonly readyTimeoutMs;
+    private readonly rotationMarginMs;
+    private readonly maxPhysicalSessionMs;
+    private readonly maxReconnectAttempts;
+    private readonly reconnectBaseDelayMs;
+    private current;
+    private heartbeatTimer;
+    private heartbeatAckTimer;
+    private rotationTimer;
+    private retryDelay;
+    private transitionPromise;
+    private transitionKind;
+    private epochSequence;
+    private queuedBytes;
+    private activeTurn;
+    private rotationPending;
+    private closed;
+    private failure;
+    private constructor();
+    static open(createConnection: RealtimeConnectionFactory, options?: RealtimeTextToSpeechManagedConnectOptions): Promise<ManagedRealtimeTextToSpeechConnection>;
+    startTurn(turnId?: string): void;
+    /**
+     * Start a turn on the current physical epoch, waiting only when an idle
+     * unexpected reconnect is already replacing a missing epoch.
+     *
+     * No other turn command is buffered. Await this method before calling
+     * appendText(), flush(), endTurn(), or cancelTurn().
+     */
+    startTurnWhenReady(turnId?: string, options?: {
+        signal?: AbortSignal;
+    }): Promise<void>;
+    appendText(text: string): void;
+    flush(): void;
+    endTurn(): void;
+    cancelTurn(): void;
+    /** Send an immediate ping in addition to the managed keepalive schedule. */
+    ping(): void;
+    close(): void;
+    events(): AsyncIterable<RealtimeTextToSpeechEvent>;
+    audio(): AsyncIterable<Uint8Array>;
+    [Symbol.asyncIterator](): AsyncIterator<RealtimeTextToSpeechMessage>;
+    private openEpoch;
+    private installEpoch;
+    private pumpEpoch;
+    private handleEpochMessage;
+    private handleEpochEnd;
+    private requestUnexpectedReconnect;
+    private beginTransition;
+    private replaceEpoch;
+    private deferPlannedTransitionForActiveTurn;
+    private scheduleTimers;
+    private scheduleNextHeartbeat;
+    private acknowledgeHeartbeat;
+    private requireCurrentConnection;
+    private waitForStartTurnTransition;
+    private throwIfStartTurnAborted;
     private push;
     private fail;
     private finish;
+    private clearTimers;
+    private waitForRetryDelay;
+    private cancelRetryDelay;
+    private completeOpenAttempt;
+    private cancelOpenAttempt;
+    private cancelOpenAttempts;
+    private beginSettlementDrain;
+    private handleSettlementDrainMessage;
+    private finishSettlementDrain;
+    private closeSettlementDrains;
 }
 declare class GenerationJobsResource {
     private readonly client;