@furious.luke/argus-js 0.5.5 → 0.5.7

package/README.md

@@ -66,6 +66,7 @@ const publisher = new Publisher({
       });
     },
     onConnectionStateChange: (state) => console.log("state:", state),
+    onConnectionQualityChange: ({ level }) => console.log("connection quality:", level),
     onRecoveryStateChange: (event) => console.log("media recovery:", event),
     // Show a button or prompt. From its click handler, call captureScreen()
     // and then publisher.publish(newStream, "screen").
@@ -201,6 +202,9 @@ not the `utterance_finished` lifecycle event, which can overtake the last chunk.
 | `peerConnectionTimeoutMs` | `number` | Deadline after the initial offer for WebRTC to reach `connected`. Defaults to 30 seconds. |
 | `signalingReconnectTimeoutMs` | `number` | How long to retry a dropped signaling socket against the selected regional gateway. Defaults to 20 seconds. |
 | `preferredVideoCodecs` | `string[]` | Preferred video codecs, most-preferred first, as RTP MIME types (e.g. `"video/VP9"`, `"video/H264"`). Each published video track offers these ahead of the rest, so the browser sends the first one the media server also accepts. Defaults to `["video/VP9"]`. Pass `[]` to leave the browser's native order untouched. Codecs the browser lacks (or `setCodecPreferences` support, e.g. older Safari) are ignored — negotiation always falls back cleanly. |
+| `connectionStatsIntervalMs` | `number` | How often to poll `RTCPeerConnection.getStats()` for connection-quality assessment while connected. Defaults to 2000. Set to `0` to disable stats polling entirely (`onConnectionStats`/`onConnectionQualityChange` will not fire). |
+| `connectionQualityThresholds` | `Partial<ConnectionQualityThresholds>` | Overrides for the quality classification thresholds. Packet loss is the primary axis; RTT and jitter can only push the level worse, never better. Any omitted field keeps its default (see below). |
+| `connectionQualityDebounceSamples` | `number` | Consecutive worse-than-current samples required before a downgrade is committed, damping transient blips. Improvements are reported on the first better sample. Defaults to 2. |
 | `callbacks` | `PublisherCallbacks` | Optional lifecycle callbacks (see below). |
 
 ### Methods & properties
@@ -235,6 +239,9 @@ not the `utterance_finished` lifecycle event, which can overtake the last chunk.
 | `onAssistantText({ utteranceId, text })` | One caption chunk arrived; paced with synthesized speech when speech is enabled, immediate in text-only mode. An utterance emits many chunks sharing one `utteranceId` — append them to the bubble keyed by that id rather than rendering each separately. |
 | `onAssistantTextFinished({ utteranceId })` | The utterance's caption stream is complete. Emitted on the same ordered channel after its last `onAssistantText` chunk; finalize the visible caption here rather than on `utterance_finished`. |
 | `onUserTextResult({ messageId, accepted, reason })` | The server accepted or rejected a `sendUserText` message. |
+| `onConnectionStats(sample)` | A periodic `ConnectionStatsSample` (loss ratio, RTT, jitter, send/available bitrate, encoder limitation) — the raw feed behind `onConnectionQualityChange`. Fires every `connectionStatsIntervalMs` while connected. |
+| `onConnectionQualityChange({ level, sample })` | The derived connection-quality level (`"good" \| "fair" \| "poor" \| "critical"`) changed, including the first assessment after connecting. Downgrades are debounced; upgrades fire immediately. React to a degrading uplink here — warn the user, or drop a secondary track. |
+| `onSpeechQualityChange({ degraded, realtimeFactor })` | Server-side text-to-speech generation crossed the realtime boundary: `degraded: true` when the TTS provider dropped below realtime (`realtimeFactor` < 1.0), synthesizing slower than it plays and starving playout, `degraded: false` once it climbs comfortably back above realtime. The degraded transition is raised **live during synthesis** (a long, slow utterance is reported while it happens, not at its end). Edge-triggered (no flapping at the boundary). A **distinct axis from `onConnectionQualityChange`** (that is network health) — react by warning the user or offering a text fallback. |
 | `onError(error)` | A fatal error occurred (signaling error, WebRTC connection failure/timeout, or signaling resume timed out). |
 
 ## How `start()` works
@@ -287,6 +294,53 @@ pass it to `publish` (or `replaceStream`) for that track type. A browser cannot
 silently reacquire a screen share after the user or operating system ends it, so
 `capture_ended` always requires host UI and a fresh `captureScreen()` call.
 
+## Connection quality
+
+While connected, the publisher polls the peer connection's WebRTC stats every
+`connectionStatsIntervalMs` (default 2s) and derives a coarse quality level so
+you can react to a degrading uplink without parsing `getStats()` yourself. Each
+poll delivers a raw `ConnectionStatsSample` to `onConnectionStats`; whenever the
+derived level changes it delivers a `ConnectionQuality` to
+`onConnectionQualityChange`.
+
+```ts
+const pub = new Publisher({
+  gatewayURLs,
+  token,
+  callbacks: {
+    onConnectionQualityChange: ({ level, sample }) => {
+      if (level === "poor" || level === "critical") {
+        showBanner(`Weak connection — ${Math.round(sample.lossRatio * 100)}% packet loss`);
+      } else {
+        hideBanner();
+      }
+    },
+  },
+});
+```
+
+Packet loss is the primary signal; round-trip time, jitter, and a
+bandwidth-limited encoder can only push the level worse, never better. To avoid
+flapping on a momentary blip, **downgrades** are debounced — they require
+`connectionQualityDebounceSamples` (default 2) consecutive worse samples before
+committing — while **upgrades** are reported on the first improved sample so
+recovery is reflected promptly. The first sample after connecting always emits
+the baseline level.
+
+The default `ConnectionQualityThresholds` (all overridable via
+`connectionQualityThresholds`):
+
+| Level | Packet loss | RTT | Jitter |
+| --- | --- | --- | --- |
+| `fair` | ≥ 2% | ≥ 300 ms | ≥ 50 ms |
+| `poor` | ≥ 5% | ≥ 600 ms | ≥ 150 ms |
+| `critical` | ≥ 12% | ≥ 1000 ms | — |
+
+Loss ratio and send bitrate are windowed over each interval; RTT, jitter, and
+available bitrate are point-in-time. A sample field is `null` when the browser
+did not report the underlying stat. This is a detection surface only — the
+publisher does not itself lower bitrate or resolution in response.
+
 ## Browser support
 
 Requires a browser with WebRTC (`RTCPeerConnection`) and `WebSocket` — all current evergreen browsers. There is no Node.js runtime support; this is a browser-only package.

package/dist/index.cjs

@@ -94,6 +94,28 @@ function selectGatewayTURNURLs(advertised, policy = "all") {
   }
   return selected;
 }
+var qualityLevels = ["good", "fair", "poor", "critical"];
+function qualitySeverity(level) {
+  return qualityLevels.indexOf(level);
+}
+function nowMs() {
+  return typeof performance !== "undefined" && typeof performance.now === "function" ? performance.now() : Date.now();
+}
+var limitationSeverity = { none: 0, other: 1, cpu: 2, bandwidth: 3 };
+function worseLimitation(current, next) {
+  if (current === null) return next;
+  return (limitationSeverity[next] ?? 1) > (limitationSeverity[current] ?? 1) ? next : current;
+}
+function findNominatedCandidatePair(stats) {
+  let found;
+  stats.forEach((report) => {
+    const value = report;
+    if (!found && value.type === "candidate-pair" && value.state === "succeeded" && value.nominated === true) {
+      found = value;
+    }
+  });
+  return found;
+}
 var defaultSignalingReconnectTimeoutMs = 2e4;
 var defaultGatewayHandshakeTimeoutMs = 2e4;
 var defaultPeerConnectionTimeoutMs = 3e4;
@@ -114,6 +136,18 @@ var negotiationReconnectGraceMs = 5e3;
 var minimumIntentionalTrackEndRetentionMs = 35e3;
 var maxUserTextBytes = 4 * 1024;
 var maxRetainedICECandidates = 64;
+var defaultConnectionStatsIntervalMs = 2e3;
+var defaultConnectionQualityDebounceSamples = 2;
+var defaultConnectionQualityThresholds = {
+  fairLossRatio: 0.02,
+  poorLossRatio: 0.05,
+  criticalLossRatio: 0.12,
+  fairRttMs: 300,
+  poorRttMs: 600,
+  criticalRttMs: 1e3,
+  fairJitterMs: 50,
+  poorJitterMs: 150
+};
 var ReportedPublisherError = class extends Error {
   constructor(message, fatal = false) {
     super(message);
@@ -156,6 +190,17 @@ var Publisher = class {
   gatewayURL = null;
   lastReportedICEPath = null;
   watchedICETransports = /* @__PURE__ */ new WeakSet();
+  // Periodic getStats() polling for connection-quality assessment. The timer runs
+  // only while connected; lastStatsSample anchors the windowed loss/bitrate deltas
+  // (loss is tracked per outbound SSRC so it can be paired to the remote's report),
+  // and currentQualityLevel/qualityDowngradeStreak drive the debounced classifier.
+  connectionStatsTimer = null;
+  // Guards against overlapping getStats() calls: if one outlives the interval,
+  // ticks are skipped until it resolves so samples never complete out of order.
+  statsSampleInFlight = false;
+  lastStatsSample = null;
+  currentQualityLevel = null;
+  qualityDowngradeStreak = 0;
   stopped = true;
   // Every start/stop boundary advances lifecycleGeneration. Async work captures
   // the generation it belongs to and may never mutate or terminate a later run.
@@ -368,10 +413,13 @@ var Publisher = class {
         if (state === "connected") {
           this.clearPeerConnectionTimeout();
           void this.reportSelectedICEPath(pc);
+          this.startConnectionStatsLoop(generation, pc);
           this.opts.callbacks?.onConnected?.();
         } else if (state === "failed") {
           this.clearPeerConnectionTimeout();
           this.terminateWithError(new Error("WebRTC connection failed"), true, generation);
+        } else {
+          this.stopConnectionStatsLoop();
         }
       };
       if (initialTrack) {
@@ -506,6 +554,7 @@ var Publisher = class {
     this.runAbort?.abort();
     this.runAbort = null;
     this.clearPeerConnectionTimeout();
+    this.stopConnectionStatsLoop();
     this.stopped = true;
     this.cancelAllMediaRecovery();
     this.reconnectGeneration++;
@@ -994,6 +1043,7 @@ var Publisher = class {
     this.runAbort?.abort();
     this.runAbort = null;
     this.clearPeerConnectionTimeout();
+    this.stopConnectionStatsLoop();
     this.stopped = true;
     this.cancelAllMediaRecovery();
     this.reconnectGeneration++;
@@ -1071,6 +1121,17 @@ var Publisher = class {
         this.completeMediaRecovery(msg.track);
         break;
       }
+      case "speech_slow": {
+        this.opts.callbacks?.onSpeechQualityChange?.({
+          degraded: true,
+          realtimeFactor: msg.realtime_factor
+        });
+        break;
+      }
+      case "speech_recovered": {
+        this.opts.callbacks?.onSpeechQualityChange?.({ degraded: false });
+        break;
+      }
       case "error": {
         const err = new ReportedPublisherError(msg.error, msg.fatal === true);
         const pending = this.pendingAnswer;
@@ -1464,6 +1525,249 @@ var Publisher = class {
     } catch {
     }
   }
+  // startConnectionStatsLoop begins periodic getStats() sampling once the peer
+  // connection is connected. It is a no-op when polling is disabled or no
+  // consumer is listening, and it re-baselines on each call so a reconnect after
+  // an ICE restart starts a fresh quality assessment.
+  startConnectionStatsLoop(generation, pc) {
+    const intervalMs = Math.max(
+      0,
+      this.opts.connectionStatsIntervalMs ?? defaultConnectionStatsIntervalMs
+    );
+    const listening = !!this.opts.callbacks?.onConnectionStats || !!this.opts.callbacks?.onConnectionQualityChange;
+    if (intervalMs === 0 || !listening) return;
+    this.stopConnectionStatsLoop();
+    const timer = setInterval(() => {
+      if (this.connectionStatsTimer !== timer) return;
+      if (!this.isActiveRun(generation, pc)) {
+        this.stopConnectionStatsLoop();
+        return;
+      }
+      if (this.statsSampleInFlight) return;
+      this.statsSampleInFlight = true;
+      void this.sampleConnectionQuality(generation, pc, timer);
+    }, intervalMs);
+    this.connectionStatsTimer = timer;
+  }
+  stopConnectionStatsLoop() {
+    if (this.connectionStatsTimer !== null) clearInterval(this.connectionStatsTimer);
+    this.connectionStatsTimer = null;
+    this.statsSampleInFlight = false;
+    this.lastStatsSample = null;
+    this.currentQualityLevel = null;
+    this.qualityDowngradeStreak = 0;
+  }
+  async sampleConnectionQuality(generation, pc, timer) {
+    let stats = null;
+    try {
+      stats = await pc.getStats();
+    } catch {
+    }
+    if (this.connectionStatsTimer !== timer) return;
+    this.statsSampleInFlight = false;
+    if (!stats || !this.isActiveRun(generation, pc) || pc.connectionState !== "connected") {
+      return;
+    }
+    const sample = this.buildStatsSample(stats);
+    this.opts.callbacks?.onConnectionStats?.(sample);
+    if (this.connectionStatsTimer !== timer || !this.isActiveRun(generation, pc) || pc.connectionState !== "connected") {
+      return;
+    }
+    this.updateConnectionQuality(sample);
+  }
+  // buildStatsSample derives one sample from a getStats() report. Loss is a mean of
+  // per-stream loss fractions weighted by each stream's packets sent this window, so
+  // every stream carrying traffic contributes. A stream uses its remote fractionLost
+  // when present — the remote's loss ratio over its RR interval, self-aligned and
+  // excluding retransmissions (which ride a separate SSRC); dividing the remote's
+  // Δ(packetsLost) by the local Δ(packetsSent) would instead misalign an RTCP-timed
+  // numerator with a continuously-updated denominator — and otherwise falls back to
+  // its own windowed Δ(packetsLost) / Δ(distinct media packets sent), a denominator
+  // that excludes retransmissions (packetsSent - retransmittedPacketsSent). Mixing the
+  // two per stream keeps a lossy stream from being dropped when a sibling has fractionLost.
+  //
+  // Deltas are taken only over outbound stats-object ids present in BOTH this and the
+  // previous sample. A track replace/unpublish (or a recycled SSRC) deletes the old
+  // stats object and creates a new one with a new id; counting a vanished stream's
+  // missing tail, or a fresh object's cumulative total as an interval delta, would
+  // corrupt loss. Excluding the symmetric difference baselines new objects (they count
+  // from their next sample) and drops departed ones, so continuous streams still measure
+  // loss even when an SSRC number is reused across distinct stats objects.
+  buildStatsSample(stats) {
+    const timestamp = nowMs();
+    let nackCount = 0;
+    let pliCount = 0;
+    let haveOutbound = false;
+    let haveOutboundVideo = false;
+    let rttSeconds = null;
+    let jitterSeconds = null;
+    let limitationReason = null;
+    let selectedPairID;
+    let transportBytes = null;
+    const streams = /* @__PURE__ */ new Map();
+    const ssrcToId = /* @__PURE__ */ new Map();
+    stats.forEach((report) => {
+      const value = report;
+      switch (value.type) {
+        case "outbound-rtp": {
+          haveOutbound = true;
+          if (value.kind === "video") haveOutboundVideo = true;
+          const ssrc = typeof value.ssrc === "number" ? value.ssrc : NaN;
+          const sent = typeof value.packetsSent === "number" ? value.packetsSent : 0;
+          const retransmitted = typeof value.retransmittedPacketsSent === "number" ? value.retransmittedPacketsSent : 0;
+          const bytesSent = typeof value.bytesSent === "number" ? value.bytesSent : 0;
+          if (typeof value.nackCount === "number") nackCount += value.nackCount;
+          if (typeof value.pliCount === "number") pliCount += value.pliCount;
+          if (typeof value.qualityLimitationReason === "string") {
+            limitationReason = worseLimitation(limitationReason, value.qualityLimitationReason);
+          }
+          if (typeof value.id === "string") {
+            const expected = Math.max(0, sent - retransmitted);
+            streams.set(value.id, { expected, lost: null, fractionLost: null, bytesSent });
+            if (!Number.isNaN(ssrc)) ssrcToId.set(ssrc, value.id);
+          }
+          break;
+        }
+        case "remote-inbound-rtp": {
+          if (typeof value.roundTripTime === "number") {
+            rttSeconds = Math.max(rttSeconds ?? 0, value.roundTripTime);
+          }
+          if (typeof value.jitter === "number") {
+            jitterSeconds = Math.max(jitterSeconds ?? 0, value.jitter);
+          }
+          break;
+        }
+        case "transport": {
+          if (typeof value.selectedCandidatePairId === "string") {
+            selectedPairID = value.selectedCandidatePairId;
+          }
+          if (typeof value.bytesSent === "number") {
+            transportBytes = (transportBytes ?? 0) + value.bytesSent;
+          }
+          break;
+        }
+        default:
+          break;
+      }
+    });
+    stats.forEach((report) => {
+      const value = report;
+      if (value.type !== "remote-inbound-rtp") return;
+      const outboundId = typeof value.localId === "string" && streams.has(value.localId) ? value.localId : typeof value.ssrc === "number" ? ssrcToId.get(value.ssrc) : void 0;
+      if (outboundId === void 0) return;
+      const stream = streams.get(outboundId);
+      if (!stream) return;
+      const lost = typeof value.packetsLost === "number" ? value.packetsLost : 0;
+      stream.lost = (stream.lost ?? 0) + lost;
+      if (typeof value.fractionLost === "number" && Number.isFinite(value.fractionLost)) {
+        stream.fractionLost = Math.min(1, Math.max(0, value.fractionLost));
+      }
+    });
+    const pair = (selectedPairID ? stats.get(selectedPairID) : void 0) ?? findNominatedCandidatePair(stats);
+    const availableOutgoingBitrate = pair && typeof pair.availableOutgoingBitrate === "number" ? pair.availableOutgoingBitrate : null;
+    if (rttSeconds === null && pair && typeof pair.currentRoundTripTime === "number") {
+      rttSeconds = pair.currentRoundTripTime;
+    }
+    const prev = this.lastStatsSample;
+    let weightedFractionSum = 0;
+    let weightSum = 0;
+    let intersectionBytesDelta = 0;
+    let hadStreamOverlap = false;
+    streams.forEach((current, id) => {
+      const before = prev?.streams.get(id);
+      if (!before) return;
+      hadStreamOverlap = true;
+      intersectionBytesDelta += Math.max(0, current.bytesSent - before.bytesSent);
+      const deltaExpected = Math.max(0, current.expected - before.expected);
+      if (deltaExpected <= 0) return;
+      let fraction = null;
+      if (current.fractionLost !== null) {
+        fraction = current.fractionLost;
+      } else if (current.lost !== null && before.lost !== null) {
+        fraction = Math.min(1, Math.max(0, current.lost - before.lost) / deltaExpected);
+      }
+      if (fraction === null) return;
+      weightedFractionSum += fraction * deltaExpected;
+      weightSum += deltaExpected;
+    });
+    const deltaSeconds = prev ? (timestamp - prev.timestamp) / 1e3 : 0;
+    this.lastStatsSample = { timestamp, transportBytes, streams };
+    const lossRatio = weightSum > 0 ? Math.min(1, weightedFractionSum / weightSum) : 0;
+    let sendBitrate = null;
+    if (prev && deltaSeconds > 0) {
+      if (transportBytes !== null && prev.transportBytes !== null) {
+        sendBitrate = Math.max(0, transportBytes - prev.transportBytes) * 8 / deltaSeconds;
+      } else if (hadStreamOverlap) {
+        sendBitrate = intersectionBytesDelta * 8 / deltaSeconds;
+      }
+    }
+    return {
+      timestamp,
+      lossRatio,
+      rttMs: rttSeconds !== null ? rttSeconds * 1e3 : null,
+      jitterMs: jitterSeconds !== null ? jitterSeconds * 1e3 : null,
+      availableOutgoingBitrate,
+      sendBitrate,
+      qualityLimitationReason: haveOutboundVideo ? limitationReason ?? "none" : null,
+      nackCount: haveOutbound ? nackCount : null,
+      pliCount: haveOutbound ? pliCount : null
+    };
+  }
+  resolveQualityThresholds() {
+    return { ...defaultConnectionQualityThresholds, ...this.opts.connectionQualityThresholds ?? {} };
+  }
+  // classifyQuality maps a sample to a level. Loss is the primary axis; RTT,
+  // jitter, and a bandwidth-limited encoder can only raise severity.
+  classifyQuality(sample) {
+    const t = this.resolveQualityThresholds();
+    let severity = 0;
+    if (sample.lossRatio >= t.criticalLossRatio) severity = Math.max(severity, 3);
+    else if (sample.lossRatio >= t.poorLossRatio) severity = Math.max(severity, 2);
+    else if (sample.lossRatio >= t.fairLossRatio) severity = Math.max(severity, 1);
+    if (sample.rttMs !== null) {
+      if (sample.rttMs >= t.criticalRttMs) severity = Math.max(severity, 3);
+      else if (sample.rttMs >= t.poorRttMs) severity = Math.max(severity, 2);
+      else if (sample.rttMs >= t.fairRttMs) severity = Math.max(severity, 1);
+    }
+    if (sample.jitterMs !== null) {
+      if (sample.jitterMs >= t.poorJitterMs) severity = Math.max(severity, 2);
+      else if (sample.jitterMs >= t.fairJitterMs) severity = Math.max(severity, 1);
+    }
+    if (sample.qualityLimitationReason === "bandwidth") severity = Math.max(severity, 1);
+    return qualityLevels[severity];
+  }
+  // updateConnectionQuality commits level transitions with hysteresis: an
+  // improvement is reported on the first better sample, while a degradation must
+  // persist for connectionQualityDebounceSamples consecutive samples to commit,
+  // so a single blip does not flap the reported level.
+  updateConnectionQuality(sample) {
+    const candidate = this.classifyQuality(sample);
+    const current = this.currentQualityLevel;
+    if (current === null || candidate === current) {
+      this.qualityDowngradeStreak = 0;
+      if (candidate !== current) {
+        this.currentQualityLevel = candidate;
+        this.opts.callbacks?.onConnectionQualityChange?.({ level: candidate, sample });
+      }
+      return;
+    }
+    if (qualitySeverity(candidate) < qualitySeverity(current)) {
+      this.qualityDowngradeStreak = 0;
+      this.currentQualityLevel = candidate;
+      this.opts.callbacks?.onConnectionQualityChange?.({ level: candidate, sample });
+      return;
+    }
+    const needed = Math.max(
+      1,
+      this.opts.connectionQualityDebounceSamples ?? defaultConnectionQualityDebounceSamples
+    );
+    this.qualityDowngradeStreak += 1;
+    if (this.qualityDowngradeStreak >= needed) {
+      this.qualityDowngradeStreak = 0;
+      this.currentQualityLevel = candidate;
+      this.opts.callbacks?.onConnectionQualityChange?.({ level: candidate, sample });
+    }
+  }
   async reportSelectedICEPath(pc) {
     if (this.pc !== pc || this.stopped) return;
     let stats;