@furious.luke/argus-js 0.5.4 → 0.5.6

package/README.md

@@ -66,6 +66,7 @@ const publisher = new Publisher({
       });
     },
     onConnectionStateChange: (state) => console.log("state:", state),
+    onConnectionQualityChange: ({ level }) => console.log("connection quality:", level),
     onRecoveryStateChange: (event) => console.log("media recovery:", event),
     // Show a button or prompt. From its click handler, call captureScreen()
     // and then publisher.publish(newStream, "screen").
@@ -158,7 +159,10 @@ const publisher = new Publisher({
       void audio.play();
     },
     onAssistantText({ utteranceId, text }) {
-      renderCaption(utteranceId, text);
+      appendCaption(utteranceId, text); // APPEND to the bubble keyed by utteranceId
+    },
+    onAssistantTextFinished({ utteranceId }) {
+      finalizeCaption(utteranceId); // this utterance's caption is complete
     },
     onUserTextResult(result) {
       console.log(result.messageId, result.accepted);
@@ -175,19 +179,32 @@ publisher.sendUserText(crypto.randomUUID(), "Stop and explain that again");
 attached but silent between utterances. `sendUserText` is admitted only while
 the customer server owns a live control-token notify subscription.
 
+One assistant reply arrives as **many** `onAssistantText` chunks paced to the
+audio, all sharing the **same `utteranceId`**. Group by that id and **append**
+each chunk to the one caption element — creating a new element per callback is
+what scatters a single reply across the UI. Chunks travel a reliable, ordered
+data channel, so appending in arrival order reconstructs the text (there is no
+sequence number). Treat `onAssistantTextFinished` (same `utteranceId`, emitted on
+that same ordered channel after the last chunk) as the completion boundary —
+not the `utterance_finished` lifecycle event, which can overtake the last chunk.
+
 ### `new Publisher(options)`
 
 | Option | Type | Description |
 | --- | --- | --- |
-| `gatewayURLs` | `string[]` | **Required.** The `gateway_urls` from the join response. All are raced simultaneously; the first to return `ready` wins. |
+| `gatewayURLs` | `string[]` | **Required.** The `gateway_urls` from the join response. All are opened at once; the region whose acknowledgement (`accepted`) returns first is selected on network path, and only that region places the stream. The rest are held as standbys to fail over to. |
 | `token` | `string` | **Required.** The short-lived join token from the join response. |
 | `iceServers` | `RTCIceServer[]` | Optional extra ICE servers (e.g. your own STUN). TURN is supplied automatically by the winning gateway. |
 | `iceTransportPolicy` | `RTCIceTransportPolicy` | Passed to the underlying `RTCPeerConnection`. Defaults to `"all"`. Set `"relay"` to force media through TURN only (verifies the relay path end to end). |
 | `turnTransportPolicy` | `"all" \| "udp" \| "tls"` | Restricts gateway TURN URLs. Defaults to `"all"`; use `"tls"` with relay-only ICE to verify TURN over TLS. Startup fails if the required transport was not advertised. |
-| `gatewayHandshakeTimeoutMs` | `number` | Overall deadline for the initial gateway race and `accepted` → `ready` handshake. Unaccepted sockets are replaced after 3 seconds so a blackholed TCP flow cannot consume the full deadline. Defaults to 20 seconds. |
+| `gatewayHandshakeTimeoutMs` | `number` | Overall deadline for the whole gateway race — selection, placement, and any failovers. Unaccepted sockets are replaced after 3 seconds so a blackholed TCP flow cannot consume the full deadline. Defaults to 20 seconds. |
+| `gatewayFailoverTimeoutMs` | `number` | How long the selected region has to return `ready` before the publisher fails over to the next-fastest standby. A socket that closes or errors fails over immediately regardless; this is the backstop for a region that goes silent. Kept generous so a slow-but-good region is not abandoned for one that is merely closer to the control plane. Defaults to 8 seconds, capped at 20. |
 | `peerConnectionTimeoutMs` | `number` | Deadline after the initial offer for WebRTC to reach `connected`. Defaults to 30 seconds. |
 | `signalingReconnectTimeoutMs` | `number` | How long to retry a dropped signaling socket against the selected regional gateway. Defaults to 20 seconds. |
 | `preferredVideoCodecs` | `string[]` | Preferred video codecs, most-preferred first, as RTP MIME types (e.g. `"video/VP9"`, `"video/H264"`). Each published video track offers these ahead of the rest, so the browser sends the first one the media server also accepts. Defaults to `["video/VP9"]`. Pass `[]` to leave the browser's native order untouched. Codecs the browser lacks (or `setCodecPreferences` support, e.g. older Safari) are ignored — negotiation always falls back cleanly. |
+| `connectionStatsIntervalMs` | `number` | How often to poll `RTCPeerConnection.getStats()` for connection-quality assessment while connected. Defaults to 2000. Set to `0` to disable stats polling entirely (`onConnectionStats`/`onConnectionQualityChange` will not fire). |
+| `connectionQualityThresholds` | `Partial<ConnectionQualityThresholds>` | Overrides for the quality classification thresholds. Packet loss is the primary axis; RTT and jitter can only push the level worse, never better. Any omitted field keeps its default (see below). |
+| `connectionQualityDebounceSamples` | `number` | Consecutive worse-than-current samples required before a downgrade is committed, damping transient blips. Improvements are reported on the first better sample. Defaults to 2. |
 | `callbacks` | `PublisherCallbacks` | Optional lifecycle callbacks (see below). |
 
 ### Methods & properties
@@ -219,15 +236,19 @@ the customer server owns a live control-token notify subscription.
 | `onRecoveryStateChange(event)` | Argus detected stalled media and the publisher started, escalated, completed, or failed automatic recovery. |
 | `onRecoveryRequired(event)` | Automatic recovery could not restore media, or capture ended and the host must ask the user for a new screen share. |
 | `onSpeechTrack(track, streams)` | The inbound `speech` track arrived after `enableSpeech()` — attach it to an `<audio>` element to play text-to-speech. |
-| `onAssistantText({ utteranceId, text })` | Assistant text arrived; it is paced with synthesized speech when speech is enabled and delivered immediately in text-only mode. |
+| `onAssistantText({ utteranceId, text })` | One caption chunk arrived; paced with synthesized speech when speech is enabled, immediate in text-only mode. An utterance emits many chunks sharing one `utteranceId` — append them to the bubble keyed by that id rather than rendering each separately. |
+| `onAssistantTextFinished({ utteranceId })` | The utterance's caption stream is complete. Emitted on the same ordered channel after its last `onAssistantText` chunk; finalize the visible caption here rather than on `utterance_finished`. |
 | `onUserTextResult({ messageId, accepted, reason })` | The server accepted or rejected a `sendUserText` message. |
+| `onConnectionStats(sample)` | A periodic `ConnectionStatsSample` (loss ratio, RTT, jitter, send/available bitrate, encoder limitation) — the raw feed behind `onConnectionQualityChange`. Fires every `connectionStatsIntervalMs` while connected. |
+| `onConnectionQualityChange({ level, sample })` | The derived connection-quality level (`"good" \| "fair" \| "poor" \| "critical"`) changed, including the first assessment after connecting. Downgrades are debounced; upgrades fire immediately. React to a degrading uplink here — warn the user, or drop a secondary track. |
 | `onError(error)` | A fatal error occurred (signaling error, WebRTC connection failure/timeout, or signaling resume timed out). |
 
 ## How `start()` works
 
-1. **Gateway race.** Every URL in `gatewayURLs` is opened at once with the token in the query string. The first to complete the two-phase handshake (`accepted` → `proceed` → `ready`) wins; the rest are closed. This picks the lowest-latency region without a separate probe.
-2. **TURN + read token.** The winning gateway's `ready` message carries per-session TURN credentials (merged into the ICE configuration) and the read token exposed as `frameReadToken`.
-3. **WebRTC.** A peer connection and the text data channel are created. Any
+1. **Gateway race.** Every URL in `gatewayURLs` is opened at once with the token in the query string. Selection and placement are two separate steps: the region whose `accepted` returns *first* is selected — purely on network path, without waiting on any placement work — and the browser sends `proceed` to that one only, holding the rest open as standbys. This picks the lowest-latency region without a separate probe, and keeps a region's distance to the control plane out of the choice.
+2. **Placement, with failover.** The selected region does its placement work and returns `ready`. If its socket closes or errors, the browser fails over to the next-fastest standby immediately; if it just goes silent, it fails over after `gatewayFailoverTimeoutMs`. A region that answers `unavailable` (transiently unable to serve) is retried after a backoff rather than counted out. A region that reports the stream is already bound elsewhere sends a `placement_redirect`, and the browser reconnects to the region that holds it — so a mistimed failover self-heals.
+3. **TURN + read token.** The selected gateway's `ready` message carries per-session TURN credentials (merged into the ICE configuration) and the read token exposed as `frameReadToken`.
+4. **WebRTC.** A peer connection and the text data channel are created. Any
    initial media track is added and labelled, then an offer is sent. Remote ICE
    candidates that arrive before the SDP answer are buffered and flushed once
    the answer is applied. Locally gathered candidates remain queued until their
@@ -272,6 +293,53 @@ pass it to `publish` (or `replaceStream`) for that track type. A browser cannot
 silently reacquire a screen share after the user or operating system ends it, so
 `capture_ended` always requires host UI and a fresh `captureScreen()` call.
 
+## Connection quality
+
+While connected, the publisher polls the peer connection's WebRTC stats every
+`connectionStatsIntervalMs` (default 2s) and derives a coarse quality level so
+you can react to a degrading uplink without parsing `getStats()` yourself. Each
+poll delivers a raw `ConnectionStatsSample` to `onConnectionStats`; whenever the
+derived level changes it delivers a `ConnectionQuality` to
+`onConnectionQualityChange`.
+
+```ts
+const pub = new Publisher({
+  gatewayURLs,
+  token,
+  callbacks: {
+    onConnectionQualityChange: ({ level, sample }) => {
+      if (level === "poor" || level === "critical") {
+        showBanner(`Weak connection — ${Math.round(sample.lossRatio * 100)}% packet loss`);
+      } else {
+        hideBanner();
+      }
+    },
+  },
+});
+```
+
+Packet loss is the primary signal; round-trip time, jitter, and a
+bandwidth-limited encoder can only push the level worse, never better. To avoid
+flapping on a momentary blip, **downgrades** are debounced — they require
+`connectionQualityDebounceSamples` (default 2) consecutive worse samples before
+committing — while **upgrades** are reported on the first improved sample so
+recovery is reflected promptly. The first sample after connecting always emits
+the baseline level.
+
+The default `ConnectionQualityThresholds` (all overridable via
+`connectionQualityThresholds`):
+
+| Level | Packet loss | RTT | Jitter |
+| --- | --- | --- | --- |
+| `fair` | ≥ 2% | ≥ 300 ms | ≥ 50 ms |
+| `poor` | ≥ 5% | ≥ 600 ms | ≥ 150 ms |
+| `critical` | ≥ 12% | ≥ 1000 ms | — |
+
+Loss ratio and send bitrate are windowed over each interval; RTT, jitter, and
+available bitrate are point-in-time. A sample field is `null` when the browser
+did not report the underlying stat. This is a detection surface only — the
+publisher does not itself lower bitrate or resolution in response.
+
 ## Browser support
 
 Requires a browser with WebRTC (`RTCPeerConnection`) and `WebSocket` — all current evergreen browsers. There is no Node.js runtime support; this is a browser-only package.