@furious.luke/argus-js 0.5.4 → 0.5.5

package/README.md

@@ -158,7 +158,10 @@ const publisher = new Publisher({
       void audio.play();
     },
     onAssistantText({ utteranceId, text }) {
-      renderCaption(utteranceId, text);
+      appendCaption(utteranceId, text); // APPEND to the bubble keyed by utteranceId
+    },
+    onAssistantTextFinished({ utteranceId }) {
+      finalizeCaption(utteranceId); // this utterance's caption is complete
     },
     onUserTextResult(result) {
       console.log(result.messageId, result.accepted);
@@ -175,16 +178,26 @@ publisher.sendUserText(crypto.randomUUID(), "Stop and explain that again");
 attached but silent between utterances. `sendUserText` is admitted only while
 the customer server owns a live control-token notify subscription.
 
+One assistant reply arrives as **many** `onAssistantText` chunks paced to the
+audio, all sharing the **same `utteranceId`**. Group by that id and **append**
+each chunk to the one caption element — creating a new element per callback is
+what scatters a single reply across the UI. Chunks travel a reliable, ordered
+data channel, so appending in arrival order reconstructs the text (there is no
+sequence number). Treat `onAssistantTextFinished` (same `utteranceId`, emitted on
+that same ordered channel after the last chunk) as the completion boundary —
+not the `utterance_finished` lifecycle event, which can overtake the last chunk.
+
 ### `new Publisher(options)`
 
 | Option | Type | Description |
 | --- | --- | --- |
-| `gatewayURLs` | `string[]` | **Required.** The `gateway_urls` from the join response. All are raced simultaneously; the first to return `ready` wins. |
+| `gatewayURLs` | `string[]` | **Required.** The `gateway_urls` from the join response. All are opened at once; the region whose acknowledgement (`accepted`) returns first is selected on network path, and only that region places the stream. The rest are held as standbys to fail over to. |
 | `token` | `string` | **Required.** The short-lived join token from the join response. |
 | `iceServers` | `RTCIceServer[]` | Optional extra ICE servers (e.g. your own STUN). TURN is supplied automatically by the winning gateway. |
 | `iceTransportPolicy` | `RTCIceTransportPolicy` | Passed to the underlying `RTCPeerConnection`. Defaults to `"all"`. Set `"relay"` to force media through TURN only (verifies the relay path end to end). |
 | `turnTransportPolicy` | `"all" \| "udp" \| "tls"` | Restricts gateway TURN URLs. Defaults to `"all"`; use `"tls"` with relay-only ICE to verify TURN over TLS. Startup fails if the required transport was not advertised. |
-| `gatewayHandshakeTimeoutMs` | `number` | Overall deadline for the initial gateway race and `accepted` → `ready` handshake. Unaccepted sockets are replaced after 3 seconds so a blackholed TCP flow cannot consume the full deadline. Defaults to 20 seconds. |
+| `gatewayHandshakeTimeoutMs` | `number` | Overall deadline for the whole gateway race — selection, placement, and any failovers. Unaccepted sockets are replaced after 3 seconds so a blackholed TCP flow cannot consume the full deadline. Defaults to 20 seconds. |
+| `gatewayFailoverTimeoutMs` | `number` | How long the selected region has to return `ready` before the publisher fails over to the next-fastest standby. A socket that closes or errors fails over immediately regardless; this is the backstop for a region that goes silent. Kept generous so a slow-but-good region is not abandoned for one that is merely closer to the control plane. Defaults to 8 seconds, capped at 20. |
 | `peerConnectionTimeoutMs` | `number` | Deadline after the initial offer for WebRTC to reach `connected`. Defaults to 30 seconds. |
 | `signalingReconnectTimeoutMs` | `number` | How long to retry a dropped signaling socket against the selected regional gateway. Defaults to 20 seconds. |
 | `preferredVideoCodecs` | `string[]` | Preferred video codecs, most-preferred first, as RTP MIME types (e.g. `"video/VP9"`, `"video/H264"`). Each published video track offers these ahead of the rest, so the browser sends the first one the media server also accepts. Defaults to `["video/VP9"]`. Pass `[]` to leave the browser's native order untouched. Codecs the browser lacks (or `setCodecPreferences` support, e.g. older Safari) are ignored — negotiation always falls back cleanly. |
@@ -219,15 +232,17 @@ the customer server owns a live control-token notify subscription.
 | `onRecoveryStateChange(event)` | Argus detected stalled media and the publisher started, escalated, completed, or failed automatic recovery. |
 | `onRecoveryRequired(event)` | Automatic recovery could not restore media, or capture ended and the host must ask the user for a new screen share. |
 | `onSpeechTrack(track, streams)` | The inbound `speech` track arrived after `enableSpeech()` — attach it to an `<audio>` element to play text-to-speech. |
-| `onAssistantText({ utteranceId, text })` | Assistant text arrived; it is paced with synthesized speech when speech is enabled and delivered immediately in text-only mode. |
+| `onAssistantText({ utteranceId, text })` | One caption chunk arrived; paced with synthesized speech when speech is enabled, immediate in text-only mode. An utterance emits many chunks sharing one `utteranceId` — append them to the bubble keyed by that id rather than rendering each separately. |
+| `onAssistantTextFinished({ utteranceId })` | The utterance's caption stream is complete. Emitted on the same ordered channel after its last `onAssistantText` chunk; finalize the visible caption here rather than on `utterance_finished`. |
 | `onUserTextResult({ messageId, accepted, reason })` | The server accepted or rejected a `sendUserText` message. |
 | `onError(error)` | A fatal error occurred (signaling error, WebRTC connection failure/timeout, or signaling resume timed out). |
 
 ## How `start()` works
 
-1. **Gateway race.** Every URL in `gatewayURLs` is opened at once with the token in the query string. The first to complete the two-phase handshake (`accepted` → `proceed` → `ready`) wins; the rest are closed. This picks the lowest-latency region without a separate probe.
-2. **TURN + read token.** The winning gateway's `ready` message carries per-session TURN credentials (merged into the ICE configuration) and the read token exposed as `frameReadToken`.
-3. **WebRTC.** A peer connection and the text data channel are created. Any
+1. **Gateway race.** Every URL in `gatewayURLs` is opened at once with the token in the query string. Selection and placement are two separate steps: the region whose `accepted` returns *first* is selected — purely on network path, without waiting on any placement work — and the browser sends `proceed` to that one only, holding the rest open as standbys. This picks the lowest-latency region without a separate probe, and keeps a region's distance to the control plane out of the choice.
+2. **Placement, with failover.** The selected region does its placement work and returns `ready`. If its socket closes or errors, the browser fails over to the next-fastest standby immediately; if it just goes silent, it fails over after `gatewayFailoverTimeoutMs`. A region that answers `unavailable` (transiently unable to serve) is retried after a backoff rather than counted out. A region that reports the stream is already bound elsewhere sends a `placement_redirect`, and the browser reconnects to the region that holds it — so a mistimed failover self-heals.
+3. **TURN + read token.** The selected gateway's `ready` message carries per-session TURN credentials (merged into the ICE configuration) and the read token exposed as `frameReadToken`.
+4. **WebRTC.** A peer connection and the text data channel are created. Any
    initial media track is added and labelled, then an offer is sent. Remote ICE
    candidates that arrive before the SDP answer are buffered and flushed once
    the answer is applied. Locally gathered candidates remain queued until their

package/dist/index.cjs

@@ -98,6 +98,12 @@ var defaultSignalingReconnectTimeoutMs = 2e4;
 var defaultGatewayHandshakeTimeoutMs = 2e4;
 var defaultPeerConnectionTimeoutMs = 3e4;
 var initialGatewayAttemptTimeoutMs = 3e3;
+var defaultGatewayFailoverTimeoutMs = 8e3;
+var maxGatewayFailoverTimeoutMs = 2e4;
+var defaultGatewayRetryBackoffMs = 3e3;
+var minGatewayRetryBackoffMs = 250;
+var maxGatewayRetryBackoffMs = 5e3;
+var maxPlacementRedirects = 2;
 var signalingResumeAttemptTimeoutMs = 3e3;
 var signalingResumeMaxBackoffMs = 3e3;
 var senderRestartPauseMs = 100;
@@ -551,6 +557,16 @@ var Publisher = class {
   // -------------------------------------------------------------------------
   // Private helpers
   // -------------------------------------------------------------------------
+  // raceGateways opens every candidate gateway at once, then decides in two
+  // separate moments. SELECTION: the first socket to deliver `accepted` (a cheap,
+  // control-plane-free acknowledgement) is chosen on network path; the browser
+  // sends `proceed` on that one only and keeps the rest as standbys. PLACEMENT:
+  // the selected region does its control-plane work and returns `ready`. If the
+  // selection dies (socket close/error → immediately) or stalls past the failover
+  // deadline (a hung-but-open socket), the browser abandons it — closing the
+  // socket cancels that region's placement server-side — and selects the
+  // next-fastest standby. A `placement_redirect` points the browser at the region
+  // that already holds the stream so a mistimed failover self-heals.
   raceGateways(signal) {
     return new Promise((resolve, reject) => {
       const { gatewayURLs, token } = this.opts;
@@ -560,43 +576,149 @@ var Publisher = class {
       }
       const sockets = [];
       const attemptTimers = /* @__PURE__ */ new Map();
+      const reopenTimers = /* @__PURE__ */ new Set();
+      const standbys = [];
+      let selected = null;
+      let failoverTimer = null;
+      let redirects = 0;
       let settled = false;
       let timeoutTimer = null;
+      const failoverMs = Math.min(
+        maxGatewayFailoverTimeoutMs,
+        Math.max(0, this.opts.gatewayFailoverTimeoutMs ?? defaultGatewayFailoverTimeoutMs)
+      );
       const clearTimeoutTimer = () => {
         if (timeoutTimer !== null) clearTimeout(timeoutTimer);
         timeoutTimer = null;
       };
+      const clearFailoverTimer = () => {
+        if (failoverTimer !== null) clearTimeout(failoverTimer);
+        failoverTimer = null;
+      };
+      const clearReopenTimers = () => {
+        for (const timer of reopenTimers) clearTimeout(timer);
+        reopenTimers.clear();
+      };
       const clearAttemptTimer = (socket) => {
         const timer = attemptTimers.get(socket);
         if (timer !== void 0) clearTimeout(timer);
         attemptTimers.delete(socket);
       };
+      const detach = (socket) => {
+        clearAttemptTimer(socket);
+        socket.onmessage = null;
+        socket.onerror = null;
+        socket.onclose = null;
+      };
+      const dropStandby = (socket) => {
+        const i = standbys.indexOf(socket);
+        if (i !== -1) standbys.splice(i, 1);
+      };
       const closeAll = (except) => {
         for (const s of sockets) {
-          clearAttemptTimer(s);
           if (s !== except) {
-            s.onmessage = null;
-            s.onerror = null;
-            s.onclose = null;
+            detach(s);
             s.close();
           }
         }
       };
-      const checkAllFailed = () => {
-        if (settled) return;
+      const win = (ws, readyInfo, gatewayURL) => {
+        settled = true;
+        clearTimeoutTimer();
+        clearFailoverTimer();
+        clearReopenTimers();
+        signal.removeEventListener("abort", abort);
+        closeAll(ws);
+        resolve({ ws, readyInfo, gatewayURL });
+      };
+      const fail = (err) => {
+        settled = true;
+        clearTimeoutTimer();
+        clearFailoverTimer();
+        clearReopenTimers();
+        signal.removeEventListener("abort", abort);
+        closeAll();
+        reject(err);
+      };
+      const checkExhausted = () => {
+        if (settled || selected !== null || standbys.length > 0 || reopenTimers.size > 0) return;
         if (sockets.every((s) => s.readyState === WebSocket.CLOSED || s.readyState === WebSocket.CLOSING)) {
-          settled = true;
-          clearTimeoutTimer();
-          signal.removeEventListener("abort", abort);
-          reject(new Error("all gateways failed to connect"));
+          fail(new Error("all gateways failed to connect"));
         }
       };
-      const abort = () => {
+      const select = (ws) => {
+        selected = ws;
+        dropStandby(ws);
+        try {
+          ws.send(JSON.stringify({ type: "proceed" }));
+        } catch {
+          socketDown(ws);
+          return;
+        }
+        clearFailoverTimer();
+        failoverTimer = setTimeout(() => failover(ws), failoverMs);
+      };
+      const failover = (deadSocket) => {
+        if (settled || deadSocket !== selected) return;
+        clearFailoverTimer();
+        detach(deadSocket);
+        deadSocket.close();
+        selected = null;
+        const next = standbys.shift();
+        if (next) {
+          select(next);
+        } else {
+          checkExhausted();
+        }
+      };
+      const redirect = (gatewayURL) => {
         if (settled) return;
-        settled = true;
-        clearTimeoutTimer();
+        if (redirects >= maxPlacementRedirects) {
+          fail(new Error("too many placement redirects"));
+          return;
+        }
+        redirects++;
+        clearFailoverTimer();
         closeAll();
-        reject(new PublisherStoppedError("publisher stopped"));
+        standbys.length = 0;
+        selected = null;
+        try {
+          openGateway(gatewayURL);
+        } catch (err) {
+          fail(err instanceof Error ? err : new Error(String(err)));
+        }
+      };
+      const retryUnavailable = (ws, gatewayURL, retryAfterMs) => {
+        detach(ws);
+        ws.close();
+        const delay = Math.min(
+          maxGatewayRetryBackoffMs,
+          Math.max(minGatewayRetryBackoffMs, retryAfterMs ?? defaultGatewayRetryBackoffMs)
+        );
+        const timer = setTimeout(() => {
+          reopenTimers.delete(timer);
+          if (settled) return;
+          try {
+            openGateway(gatewayURL);
+          } catch (err) {
+            fail(err instanceof Error ? err : new Error(String(err)));
+          }
+        }, delay);
+        reopenTimers.add(timer);
+      };
+      const socketDown = (ws) => {
+        if (settled) return;
+        clearAttemptTimer(ws);
+        dropStandby(ws);
+        if (ws === selected) {
+          failover(ws);
+        } else {
+          checkExhausted();
+        }
+      };
+      const abort = () => {
+        if (settled) return;
+        fail(new PublisherStoppedError("publisher stopped"));
       };
       if (signal.aborted) {
         abort();
@@ -609,10 +731,7 @@ var Publisher = class {
       );
       timeoutTimer = setTimeout(() => {
         if (settled) return;
-        settled = true;
-        signal.removeEventListener("abort", abort);
-        closeAll();
-        reject(new Error(`gateway handshake timed out after ${timeoutMs}ms`));
+        fail(new Error(`gateway handshake timed out after ${timeoutMs}ms`));
       }, timeoutMs);
       const openGateway = (gatewayURL) => {
         if (settled) return;
@@ -624,58 +743,51 @@ var Publisher = class {
         const attemptTimer = setTimeout(() => {
           attemptTimers.delete(ws);
           if (settled || accepted) return;
-          ws.onmessage = null;
-          ws.onerror = null;
-          ws.onclose = null;
+          detach(ws);
           ws.close();
           try {
             openGateway(gatewayURL);
           } catch (err) {
-            settled = true;
-            clearTimeoutTimer();
-            signal.removeEventListener("abort", abort);
-            closeAll();
-            reject(err);
+            fail(err instanceof Error ? err : new Error(String(err)));
           }
         }, initialGatewayAttemptTimeoutMs);
         attemptTimers.set(ws, attemptTimer);
         ws.onmessage = (ev) => {
           if (settled) return;
+          let msg;
           try {
-            const msg = JSON.parse(ev.data);
-            if (!accepted && msg.type === "accepted") {
-              accepted = true;
-              clearAttemptTimer(ws);
-              ws.send(JSON.stringify({ type: "proceed" }));
-            } else if (accepted && msg.type === "ready") {
-              settled = true;
-              clearTimeoutTimer();
-              signal.removeEventListener("abort", abort);
-              closeAll(ws);
-              resolve({ ws, readyInfo: msg, gatewayURL });
-            }
+            msg = JSON.parse(ev.data);
           } catch {
+            return;
+          }
+          if (!accepted) {
+            if (msg.type === "unavailable") {
+              retryUnavailable(ws, gatewayURL, msg.retry_after_ms);
+              return;
+            }
+            if (msg.type !== "accepted") return;
+            accepted = true;
+            clearAttemptTimer(ws);
+            if (selected === null) select(ws);
+            else standbys.push(ws);
+            return;
+          }
+          if (ws !== selected) return;
+          if (msg.type === "ready") {
+            win(ws, msg, gatewayURL);
+          } else if (msg.type === "placement_redirect" && msg.gateway_url) {
+            redirect(msg.gateway_url);
           }
         };
-        ws.onerror = () => {
-          clearAttemptTimer(ws);
-          checkAllFailed();
-        };
-        ws.onclose = () => {
-          clearAttemptTimer(ws);
-          checkAllFailed();
-        };
+        ws.onerror = () => socketDown(ws);
+        ws.onclose = () => socketDown(ws);
       };
       try {
         for (const gatewayURL of gatewayURLs) {
           openGateway(gatewayURL);
         }
       } catch (err) {
-        settled = true;
-        clearTimeoutTimer();
-        signal.removeEventListener("abort", abort);
-        closeAll();
-        reject(err);
+        fail(err instanceof Error ? err : new Error(String(err)));
       }
     });
   }