replicas-computer 0.3.5 → 0.3.7

package/README.md

@@ -24,7 +24,7 @@ ffmpeg captures the display and encodes 1920×1080 H.264 (and Opus from PulseAud
 
 WebRTC is the default transport, with automatic WebSocket fallback. A viewer can choose `?transport=websocket`; `REPLICAS_DESKTOP_STREAM_TRANSPORT=websocket` changes the server default for viewers that do not choose. Dashboard URLs carry the selected transport explicitly, and saved transport preferences are preserved. The viewer's WebSocket to `/stream` is the control plane either way: it carries the server's SDP offer, the viewer's answer, trickled ICE candidates, keepalives and telemetry. Pointer moves ride an unreliable data channel. When the server advertises `inputRace`, clicks, keys, wheel events and held-input release race a direct copy against ordered WebSocket delivery. The input bridge accepts only the next action ID, ignoring duplicates and out-of-order direct copies; the socket fills gaps without SCTP retransmission delays or a reorder queue. Older peers keep their existing input path. The watchdog tracks decoded frames so CSS-hidden viewers do not fall back merely because the browser stops presenting video. An explicit server-side WebRTC failure, a network that cannot pass UDP, or six seconds without decoded video switches the viewer to the socket, where the server reassembles frames and WebCodecs decodes them in the browser.
 
-The server makes the offer, so payload types and SSRCs are fixed and ffmpeg emits them directly. Four H.264 payload types are offered (High 4:4:4 Predictive, High, Main, Constrained Baseline); the encoder runs the best profile every connected viewer accepts, so one viewer without 4:4:4 support moves everyone to 4:2:0, and socket viewers or caps below 8 Mbps do the same. Two profiles select the encoder tuning: `balanced` (CRF 14, default chroma quantisation) and `text` (CRF 10, chroma QP offset −6); `?profile=` picks one and the most recent viewer's choice applies to the shared encoder. Quality adapts on a six-rung bitrate ladder that keeps resolution and frame rate and adds two CRF per rung under the 20 Mbps cap; `?adapt=0` pins the top rung.
+The server makes the offer, so payload types and SSRCs are fixed and ffmpeg emits them directly. Four H.264 payload types are offered (High 4:4:4 Predictive, High, Main, Constrained Baseline); the encoder runs the best profile every connected viewer accepts, so one viewer without 4:4:4 support moves everyone to 4:2:0, and socket viewers or caps below 8 Mbps do the same. Two profiles select the encoder tuning: `balanced` (CRF 14, default chroma quantisation) and `text` (CRF 10, chroma QP offset −6); `?profile=` picks one and the most recent viewer's choice applies to the shared encoder. Quality adapts on a bitrate ladder down to 750 kbps that keeps resolution and frame rate and adds two CRF per rung under the default 20 Mbps cap; `?adapt=0` pins the top rung.
 
 Every video byte for a WebRTC viewer passes through `PacedSender`. libdatachannel's own pacer and NACK responder are not used because repairs bypassed the pacer and neither accounted for duplicates, so recovery traffic could exceed the link while fresh frames queued for seconds. The budget is 1.25× the rung's encoder cap and covers media, repairs and protective copies together: repairs take at most 30% of it over a rolling second, copies are sent only from idle budget while loss is observed and never exceed the original media bytes, and a queue that grows past 150 ms is cut back to whole frames a decoder can still follow (unsent delta frames go, the in-flight frame completes, a keyframe is requested and further deltas are dropped until it arrives). Sequence numbers are assigned as packets leave, so dropped frames never read as loss. Receiver PLI, NACK and REMB are parsed from RTCP. RTT growth (the lowest probe of the last second against the session's baseline, so a single late probe is not a queue), loss, jitter-buffer growth and stale-queue cuts drive the ladder down, and clean seconds drive it up. REMB is not a congestion vote: the receiver's estimate is bounded by what recently arrived and climbs by tens of kbps per second, so it trails every burst of motion on a quiet screen for a minute or more, and its delay detector reads the sender's pacing bursts on a quiet screen as overuse. It only bounds how far a severe descent (heavy loss or a starved decoder) steps, and only while it has stopped climbing below the sent rate. `scripts/probe-viewer.ts` drives the viewer page in a Chrome with remote debugging and prints what it measures and what the server decides each second; `scripts/probe-rtt-jitter.ts` feeds the controller a synthetic WebRTC viewer with jittery RTT probes.
 
@@ -39,7 +39,7 @@ The server records every decision that changes the picture (viewer connects and
 | `REPLICAS_DESKTOP_STREAM_TRANSPORT` | `webrtc` | Transport for viewers whose URL does not choose one |
 | `REPLICAS_DESKTOP_STREAM_PROFILE` | `balanced` | Encoder profile for viewers whose URL does not choose one |
 | `REPLICAS_DESKTOP_STREAM_ADAPTIVE` | `1` | `0` pins the top quality rung by default |
-| `REPLICAS_DESKTOP_STREAM_ICE_SERVERS` | `stun:stun.l.google.com:19302` | Comma-separated STUN/TURN servers; `turn:user:secret@host:port?transport=tcp` adds a relay. Re-read from the workspace runtime env on every viewer, so the monolith's `DESKTOP_STREAM_ICE_SERVERS` reaches running workspaces |
+| `REPLICAS_DESKTOP_STREAM_ICE_SERVERS` | `stun:stun.l.google.com:19302` | Comma-separated STUN/TURN servers; `turn:user:secret@host:port?transport=tcp` adds a relay. Percent-encode the username and password; the browser receives their decoded values. Re-read from the workspace runtime env on every viewer, so the monolith's `DESKTOP_STREAM_ICE_SERVERS` reaches running workspaces |
 | `REPLICAS_DESKTOP_STREAM_ICE_PORTS` | ephemeral | `begin-end` UDP port range for the server's ICE candidates |
 | `REPLICAS_DESKTOP_STREAM_WEBRTC` | `1` | `0` serves the WebSocket transport only |
 | `REPLICAS_DESKTOP_STREAM_COLOR` | `bt709` | `legacy` disables the explicit BT.709 conversion and signalling |
@@ -55,4 +55,4 @@ replicas-computer daemon
 
 `setup --check` reports missing OS dependencies, including the native WebRTC module, without changing the machine. `setup` installs package-owned assets and the `replicas-computer.service` systemd unit. Operational commands use the daemon over a user-only Unix socket.
 
-The sandbox image builds node-datachannel from pinned libdatachannel source with `scripts/build-webrtc.sh`, which applies `patches/libdatachannel-dtls-bio.patch` (upstream DTLS BIO synchronisation fix, [2059d2a](https://github.com/paullouisageneau/libdatachannel/commit/2059d2a3bb81dc3dc24686fdc92c90b6167ec4fd)) and `patches/libdatachannel-nack-duplicates.patch`. It needs CMake, Ninja, a C++ compiler, Git and OpenSSL development libraries. Remove each patch when the pinned addon includes its fix.
+The sandbox image builds node-datachannel from pinned libdatachannel source with `scripts/build-webrtc.sh`, which applies `patches/libdatachannel-dtls-bio.patch` (upstream DTLS BIO synchronisation fix, [2059d2a](https://github.com/paullouisageneau/libdatachannel/commit/2059d2a3bb81dc3dc24686fdc92c90b6167ec4fd)) and `patches/libdatachannel-nack-duplicates.patch`. It also applies `patches/libjuice-ice-role-presence.patch` so valid zero-valued ICE role attributes do not break Chrome TURN/TCP connectivity. It needs CMake, Ninja, a C++ compiler, Git and OpenSSL development libraries. Remove each patch when the pinned addon includes its fix.

package/dist/index.js

@@ -2937,7 +2937,7 @@ async function computerStatusCommand() {
 }
 
 // src/version.ts
-var COMPUTER_USE_VERSION = "0.3.5";
+var COMPUTER_USE_VERSION = "0.3.7";
 var COMPUTER_USE_PROTOCOL_VERSION = 1;
 
 // src/program.ts
@@ -3755,6 +3755,62 @@ var InputBridge = class {
   }
 };
 
+// ../shared/src/desktop-stream.ts
+var DEFAULT_STREAM_REQUEST = {
+  transport: "webrtc",
+  profile: "balanced",
+  adaptive: true,
+  iceTransportPolicy: "all"
+};
+function parseStreamRequest(query, defaults = DEFAULT_STREAM_REQUEST) {
+  const transport = query.get("transport");
+  const profile = query.get("profile");
+  const adapt = query.get("adapt");
+  const ice = query.get("ice");
+  return {
+    transport: transport === "webrtc" || transport === "websocket" ? transport : defaults.transport,
+    profile: profile === "text" || profile === "balanced" ? profile : defaults.profile,
+    adaptive: adapt === "0" ? false : adapt === "1" ? true : defaults.adaptive,
+    iceTransportPolicy: ice === "relay" || ice === "all" ? ice : defaults.iceTransportPolicy
+  };
+}
+var H264_PROFILES = {
+  high444: { payloadType: 99, profileLevelId: "f4001f", codec: "avc1.F4002A", label: "High 4:4:4" },
+  high: { payloadType: 96, profileLevelId: "64001f", codec: "avc1.64002A", label: "High" },
+  main: { payloadType: 97, profileLevelId: "4d001f", codec: "avc1.4D002A", label: "Main" },
+  baseline: { payloadType: 98, profileLevelId: "42e01f", codec: "avc1.42E02A", label: "Baseline" }
+};
+var PROFILE_ORDER = ["high444", "high", "main", "baseline"];
+var DESKTOP_WEBRTC_FAILURE_REASONS = {
+  server_unavailable: "server has no WebRTC",
+  browser_unavailable: "browser has no WebRTC",
+  connection_failed: "WebRTC connection failed",
+  server_connection_failed: "Server WebRTC connection failed",
+  codec_unsupported: "browser cannot receive H.264 over WebRTC",
+  negotiation_failed: "negotiation failed",
+  video_timeout: "WebRTC did not decode video in time",
+  video_stalled: "WebRTC video stopped arriving"
+};
+function isDesktopWebRtcFailure(value) {
+  if (!isRecord(value) || typeof value.reasonCode !== "string" || !Object.hasOwn(DESKTOP_WEBRTC_FAILURE_REASONS, value.reasonCode) || typeof value.turnConfigured !== "boolean") return false;
+  return [
+    "id",
+    "connectionState",
+    "iceConnectionState",
+    "iceGatheringState",
+    "signalingState",
+    "iceTransportPolicy",
+    "serverVersion",
+    "serverUnavailableReason",
+    "localCandidateTypes",
+    "remoteCandidateTypes",
+    "iceErrorCodes",
+    "errorName",
+    "path",
+    "codec"
+  ].every((key) => typeof value[key] === "string" && value[key].length <= 200) && ["elapsedMs", "candidateErrors", "framesDecoded", "packetsReceived", "packetsLost", "bytesReceived", "lastFrameAgeMs", "encoderFps"].every((key) => typeof value[key] === "number" && Number.isFinite(value[key]));
+}
+
 // src/input-validation.ts
 var finite = (value) => typeof value === "number" && Number.isFinite(value);
 var optionalFinite = (value) => value === void 0 || finite(value);
@@ -3780,7 +3836,7 @@ function validInput(value) {
     case "candidate":
       return typeof value.candidate === "string" && value.candidate.length < 1024 && typeof value.mid === "string" && value.mid.length < 16;
     case "fallback":
-      return typeof value.reason === "string" && value.reason.length < 200;
+      return typeof value.reason === "string" && value.reason.length < 200 && (value.failure === void 0 || isDesktopWebRtcFailure(value.failure));
     case "pointer":
       return finite(value.x) && finite(value.y);
     case "button":
@@ -3801,33 +3857,6 @@ function parseInput(raw) {
   return validInput(value) ? value : void 0;
 }
 
-// ../shared/src/desktop-stream.ts
-var DEFAULT_STREAM_REQUEST = {
-  transport: "webrtc",
-  profile: "balanced",
-  adaptive: true,
-  iceTransportPolicy: "all"
-};
-function parseStreamRequest(query, defaults = DEFAULT_STREAM_REQUEST) {
-  const transport = query.get("transport");
-  const profile = query.get("profile");
-  const adapt = query.get("adapt");
-  const ice = query.get("ice");
-  return {
-    transport: transport === "webrtc" || transport === "websocket" ? transport : defaults.transport,
-    profile: profile === "text" || profile === "balanced" ? profile : defaults.profile,
-    adaptive: adapt === "0" ? false : adapt === "1" ? true : defaults.adaptive,
-    iceTransportPolicy: ice === "relay" || ice === "all" ? ice : defaults.iceTransportPolicy
-  };
-}
-var H264_PROFILES = {
-  high444: { payloadType: 99, profileLevelId: "f4001f", codec: "avc1.F4002A", label: "High 4:4:4" },
-  high: { payloadType: 96, profileLevelId: "64001f", codec: "avc1.64002A", label: "High" },
-  main: { payloadType: 97, profileLevelId: "4d001f", codec: "avc1.4D002A", label: "Main" },
-  baseline: { payloadType: 98, profileLevelId: "42e01f", codec: "avc1.42E02A", label: "Baseline" }
-};
-var PROFILE_ORDER = ["high444", "high", "main", "baseline"];
-
 // src/media-protocol.ts
 var STREAM_PROTOCOL_VERSION = 3;
 var MEDIA_PACKET_HEADER_BYTES = 12;
@@ -4097,7 +4126,15 @@ function parseIceServers(spec) {
   const native = (spec ?? "stun:stun.l.google.com:19302").split(",").map((server) => server.trim()).filter(Boolean);
   const browser = native.map((url) => {
     const match = url.match(/^(turns?):([^:@]+):(.+)@([^@]+)$/);
-    return match ? { urls: `${match[1]}:${match[4]}`, username: match[2], credential: match[3] } : { urls: url };
+    if (!match) return { urls: url };
+    const [username, credential] = match.slice(2, 4).map((value) => {
+      try {
+        return decodeURIComponent(value);
+      } catch {
+        return value;
+      }
+    });
+    return { urls: `${match[1]}:${match[4]}`, username, credential };
   });
   return { native, browser };
 }
@@ -4255,7 +4292,7 @@ var profiles = {
   balanced: { crf: 14, chromaQpOffset: 0 },
   text: { crf: 10, chromaQpOffset: -6 }
 };
-var rungCaps = [configuredTopKbps, 12e3, 8e3, 5e3, 3e3, 1500].map((kbps) => Math.min(kbps, configuredTopKbps));
+var rungCaps = [configuredTopKbps, 12e3, 8e3, 5e3, 3e3, 1500, 750].map((kbps) => Math.min(kbps, configuredTopKbps));
 var qualityLadder = (profile) => rungCaps.map((kbps, index) => ({ kbps, crf: profiles[profile].crf + 2 * index, keyframeMs: index >= 4 ? 2e3 : 1e3 }));
 var fullChromaMinKbps = 8e3;
 var fallbackKeyframeMs = 250;
@@ -4371,6 +4408,7 @@ async function runMediaServer() {
   let videoBytes = 0;
   let audioBytes = 0;
   const events = [];
+  const webrtcFailures = [];
   const samples = [];
   const record = (event, detail) => {
     if (events.push({ at: Date.now(), event, detail }) > eventHistory) events.shift();
@@ -4497,6 +4535,7 @@ async function runMediaServer() {
         colorMode === "bt709" ? `scale=out_color_matrix=bt709:out_range=tv,format=${pixelFormat}` : `format=${pixelFormat}`
       ];
       const x264 = [
+        `fps=${fps}/1`,
         "repeat-headers=1",
         "sliced-threads=1",
         "slices=4",
@@ -4534,6 +4573,8 @@ async function runMediaServer() {
         "1",
         "-i",
         `${display}.0`,
+        "-filter_threads",
+        "1",
         "-vf",
         filters.join(","),
         ...warmupFrames ? ["-force_key_frames", `expr:eq(n,${warmupFrames})`, "-forced-idr", "1"] : [],
@@ -4904,7 +4945,14 @@ async function runMediaServer() {
     record("keyframe", reason);
     void videoEncoder.replace();
   };
-  const enterFallback = (viewer, reason) => {
+  const enterFallback = (viewer, reason, failure) => {
+    if (failure && !viewer.failureReported && viewer.request.transport === "webrtc") {
+      viewer.failureReported = true;
+      const entry = { ...failure, at: Date.now() };
+      if (webrtcFailures.push(entry) > 50) webrtcFailures.shift();
+      process.stderr.write(`[desktop-stream] webrtc fallback diagnostics: ${JSON.stringify(entry)}
+`);
+    }
     if (viewer.fallback) return;
     viewer.fallback = true;
     viewer.waitingForKeyframe = true;
@@ -5001,7 +5049,7 @@ async function runMediaServer() {
         viewer.rtc?.candidate(message.candidate, message.mid);
         return;
       case "fallback":
-        enterFallback(viewer, message.reason);
+        enterFallback(viewer, message.reason, message.failure);
         return;
       case "keyframe":
         if (!viewer.fallback) requestKeyframe("viewer watchdog", void 0, viewer);
@@ -5016,6 +5064,8 @@ async function runMediaServer() {
   const config = {
     type: "config",
     protocolVersion: STREAM_PROTOCOL_VERSION,
+    serverVersion: COMPUTER_USE_VERSION,
+    ...!webrtc ? { webrtcUnavailableReason: webrtcEnabled ? "native_module_unavailable" : "disabled" } : {},
     inputRace: true,
     width,
     height,
@@ -5070,7 +5120,7 @@ async function runMediaServer() {
         width,
         height,
         fps,
-        ...url.pathname === "/telemetry" ? { events, samples } : {}
+        ...url.pathname === "/telemetry" ? { events, samples, webrtcFailures } : {}
       }));
       return;
     }
@@ -5168,9 +5218,10 @@ async function runMediaServer() {
         });
       } catch (error) {
         record("webrtc error", `could not create a peer connection: ${error instanceof Error ? error.message : String(error)}`);
+        reply({ type: "webrtc-failed" });
       }
     }
-    if (!viewer.rtc) enterFallback(viewer, offerWebRtc ? "webrtc unavailable" : "websocket requested");
+    if (!viewer.rtc) enterFallback(viewer, request.transport === "webrtc" ? "webrtc unavailable" : "websocket requested");
     ensureEncoders();
     socket.on("message", (raw, binary) => {
       if (binary) return;

package/dist/stream-viewer.js

@@ -3,39 +3,6 @@ function isRecord(value) {
   return typeof value === "object" && value !== null && !Array.isArray(value);
 }
 
-// src/input-validation.ts
-var finite = (value) => typeof value === "number" && Number.isFinite(value);
-var iceServer = (value) => {
-  if (!isRecord(value)) return false;
-  return typeof value.urls === "string" && (value.username === void 0 || typeof value.username === "string") && (value.credential === void 0 || typeof value.credential === "string");
-};
-function validServerMessage(value) {
-  if (!isRecord(value) || typeof value.type !== "string") return false;
-  switch (value.type) {
-    case "ping":
-    case "pong":
-      return finite(value.sentAt);
-    case "webrtc-failed":
-      return true;
-    case "offer":
-      return typeof value.sdp === "string" && value.sdp.length < 65536;
-    case "candidate":
-      return typeof value.candidate === "string" && value.candidate.length < 1024 && typeof value.mid === "string" && value.mid.length < 16;
-    case "config":
-      return finite(value.protocolVersion) && (value.inputRace === void 0 || typeof value.inputRace === "boolean") && finite(value.width) && finite(value.height) && finite(value.fps) && typeof value.videoCodec === "string" && typeof value.audioCodec === "string" && finite(value.audioSampleRate) && finite(value.audioChannels) && isRecord(value.request) && (value.iceServers === void 0 || Array.isArray(value.iceServers) && value.iceServers.every(iceServer));
-    case "telemetry":
-      return isRecord(value.encoder) && isRecord(value.cpu) && isRecord(value.viewer) && Array.isArray(value.events);
-    default:
-      return false;
-  }
-}
-
-// src/wheel.ts
-function wheelDeltaPixels(event, pageHeight) {
-  const scale = event.deltaMode === WheelEvent.DOM_DELTA_LINE ? 32 : event.deltaMode === WheelEvent.DOM_DELTA_PAGE ? pageHeight : 1;
-  return { deltaX: event.deltaX * scale, deltaY: event.deltaY * scale };
-}
-
 // ../shared/src/desktop-stream.ts
 var DEFAULT_STREAM_REQUEST = {
   transport: "webrtc",
@@ -67,6 +34,49 @@ function h264ProfileLabel(sdpFmtpLine) {
   return match?.label ?? id ?? "";
 }
 var DESKTOP_TRANSPORT_MESSAGE = "replicas-desktop-transport";
+var DESKTOP_WEBRTC_FAILURE_REASONS = {
+  server_unavailable: "server has no WebRTC",
+  browser_unavailable: "browser has no WebRTC",
+  connection_failed: "WebRTC connection failed",
+  server_connection_failed: "Server WebRTC connection failed",
+  codec_unsupported: "browser cannot receive H.264 over WebRTC",
+  negotiation_failed: "negotiation failed",
+  video_timeout: "WebRTC did not decode video in time",
+  video_stalled: "WebRTC video stopped arriving"
+};
+
+// src/input-validation.ts
+var finite = (value) => typeof value === "number" && Number.isFinite(value);
+var iceServer = (value) => {
+  if (!isRecord(value)) return false;
+  return typeof value.urls === "string" && (value.username === void 0 || typeof value.username === "string") && (value.credential === void 0 || typeof value.credential === "string");
+};
+function validServerMessage(value) {
+  if (!isRecord(value) || typeof value.type !== "string") return false;
+  switch (value.type) {
+    case "ping":
+    case "pong":
+      return finite(value.sentAt);
+    case "webrtc-failed":
+      return true;
+    case "offer":
+      return typeof value.sdp === "string" && value.sdp.length < 65536;
+    case "candidate":
+      return typeof value.candidate === "string" && value.candidate.length < 1024 && typeof value.mid === "string" && value.mid.length < 16;
+    case "config":
+      return finite(value.protocolVersion) && (value.serverVersion === void 0 || typeof value.serverVersion === "string") && (value.webrtcUnavailableReason === void 0 || typeof value.webrtcUnavailableReason === "string") && (value.inputRace === void 0 || typeof value.inputRace === "boolean") && finite(value.width) && finite(value.height) && finite(value.fps) && typeof value.videoCodec === "string" && typeof value.audioCodec === "string" && finite(value.audioSampleRate) && finite(value.audioChannels) && isRecord(value.request) && (value.iceServers === void 0 || Array.isArray(value.iceServers) && value.iceServers.every(iceServer));
+    case "telemetry":
+      return isRecord(value.encoder) && isRecord(value.cpu) && isRecord(value.viewer) && Array.isArray(value.events);
+    default:
+      return false;
+  }
+}
+
+// src/wheel.ts
+function wheelDeltaPixels(event, pageHeight) {
+  const scale = event.deltaMode === WheelEvent.DOM_DELTA_LINE ? 32 : event.deltaMode === WheelEvent.DOM_DELTA_PAGE ? pageHeight : 1;
+  return { deltaX: event.deltaX * scale, deltaY: event.deltaY * scale };
+}
 
 // src/media-protocol.ts
 var STREAM_PROTOCOL_VERSION = 3;
@@ -138,6 +148,13 @@ globalThis.__replicasDesktop = state;
 var viewerEvents = [];
 state.events = viewerEvents;
 var decoderResets = 0;
+var connectionStartedAt = performance.now();
+var failure;
+var localCandidateTypes = /* @__PURE__ */ new Set();
+var remoteCandidateTypes = /* @__PURE__ */ new Set();
+var iceErrorCodes = /* @__PURE__ */ new Set();
+var candidateErrors = 0;
+var negotiationErrorName = "";
 var note = (event, detail) => {
   if (viewerEvents.push({ at: Date.now(), event, detail }) > 50) viewerEvents.shift();
 };
@@ -208,7 +225,7 @@ var sendInput = (message, unreliable = false) => {
 };
 var announceTransport = () => {
   if (window.parent === window) return;
-  window.parent.postMessage({ type: DESKTOP_TRANSPORT_MESSAGE, transport: transport ?? "connecting", reason: transportReason, path: route.path, codec: route.codec }, "*");
+  window.parent.postMessage({ type: DESKTOP_TRANSPORT_MESSAGE, transport: transport ?? "connecting", reason: transportReason, path: route.path, codec: route.codec, failure }, "*");
 };
 var closePeer = (preserveDeadline = false) => {
   if (!preserveDeadline) {
@@ -345,7 +362,7 @@ var handleBinary = (data) => {
   }
   if (!videoDecoder || !activeVideoConfig || videoDecoder.state !== "configured") return;
   const key = kind === 1 /* VideoKey */;
-  if (videoDecoder.decodeQueueSize > 3) {
+  if (videoDecoder.decodeQueueSize > 12) {
     videoDecoder.reset();
     videoDecoder.configure(activeVideoConfig);
     waitingForKeyframe = true;
@@ -379,8 +396,38 @@ var unlockAudio = () => {
     });
   });
 };
-var useFallback = (reason, failed = false) => {
+var useFallback = (reasonCode, failed = false) => {
   if (transport === "websocket" || !config) return;
+  const reason = reasonCode === "websocket_selected" ? "WebSocket transport selected" : reasonCode === "recent_failure" ? "recent WebRTC failure" : DESKTOP_WEBRTC_FAILURE_REASONS[reasonCode];
+  if (request.transport === "webrtc" && reasonCode !== "websocket_selected" && reasonCode !== "recent_failure") {
+    failure = {
+      id: crypto.randomUUID?.() ?? `${Date.now()}-${Math.random().toString(36).slice(2)}`,
+      reasonCode,
+      elapsedMs: Math.round(performance.now() - connectionStartedAt),
+      connectionState: peer?.connectionState ?? "unavailable",
+      iceConnectionState: peer?.iceConnectionState ?? "unavailable",
+      iceGatheringState: peer?.iceGatheringState ?? "unavailable",
+      signalingState: peer?.signalingState ?? "unavailable",
+      iceTransportPolicy: request.iceTransportPolicy,
+      turnConfigured: config.iceServers?.some((server) => /^turns?:/i.test(server.urls)) ?? false,
+      serverVersion: config.serverVersion ?? "unknown",
+      serverUnavailableReason: config.webrtcUnavailableReason ?? "",
+      localCandidateTypes: [...localCandidateTypes].join(","),
+      remoteCandidateTypes: [...remoteCandidateTypes].join(","),
+      iceErrorCodes: [...iceErrorCodes].join(","),
+      candidateErrors,
+      errorName: negotiationErrorName,
+      path: route.path,
+      codec: route.codec,
+      framesDecoded: inbound?.framesDecoded ?? 0,
+      packetsReceived: inbound?.packetsReceived ?? 0,
+      packetsLost: inbound?.packetsLost ?? 0,
+      bytesReceived: inbound?.bytesReceived ?? 0,
+      lastFrameAgeMs: lastFrameAt ? Math.round(performance.now() - lastFrameAt) : -1,
+      encoderFps: telemetry?.encoder.fps ?? -1
+    };
+    state.lastWebRtcFailure = failure;
+  }
   closePeer();
   if (failed) webrtcSkips = webrtcSkipsAfterFailure;
   note("fallback to socket", reason);
@@ -388,11 +435,11 @@ var useFallback = (reason, failed = false) => {
   transportReason = reason;
   route = { ...route, path: "proxy", codec: config.videoCodec };
   announceTransport();
+  sendSocket({ type: "fallback", reason, failure });
   if (!TrackGenerator) {
     showStatus("This browser cannot render the desktop stream");
     return;
   }
-  sendSocket({ type: "fallback", reason });
   resetVideoTrack();
   void configureVideo(config).catch((error) => showStatus(describe(error)));
   void screen.play().catch(() => showStatus("Click the stream to begin playback"));
@@ -427,8 +474,14 @@ var negotiate = async (sdp) => {
   };
   pc.onicecandidate = (event) => {
     if (peer !== pc) return;
+    if (event.candidate?.type) localCandidateTypes.add(event.candidate.type);
     if (event.candidate?.candidate) sendSocket({ type: "candidate", candidate: event.candidate.candidate, mid: event.candidate.sdpMid ?? "0" });
   };
+  pc.onicecandidateerror = (event) => {
+    if (peer !== pc) return;
+    iceErrorCodes.add(event.errorCode);
+    note("ice candidate error", String(event.errorCode));
+  };
   pc.oniceconnectionstatechange = () => {
     if (peer === pc) note("ice", pc.iceConnectionState);
   };
@@ -444,7 +497,7 @@ var negotiate = async (sdp) => {
       unlockAudio();
     } else if (pc.connectionState === "failed") {
       if (transport === "webrtc") socket?.close();
-      else useFallback("WebRTC connection failed", true);
+      else useFallback("connection_failed", true);
     }
   };
   try {
@@ -456,7 +509,7 @@ var negotiate = async (sdp) => {
     if (peer !== pc || !answer.sdp) return;
     const video = answer.sdp.split(/^m=/m).find((section) => section.startsWith("video")) ?? "";
     if (video.startsWith("video 0 ") || !/^a=rtpmap:\d+ H264\/90000/m.test(video)) {
-      useFallback("browser cannot receive H.264 over WebRTC", true);
+      useFallback("codec_unsupported", true);
       return;
     }
     sendSocket({ type: "answer", sdp: answer.sdp });
@@ -484,15 +537,22 @@ var handleServerMessage = (message, viaSocket) => {
     return;
   }
   if (message.type === "webrtc-failed") {
-    useFallback("Server WebRTC connection failed", true);
+    useFallback("server_connection_failed", true);
     return;
   }
   if (message.type === "offer") {
-    void negotiate(message.sdp).catch((error) => useFallback(`negotiation failed: ${describe(error)}`, true));
+    void negotiate(message.sdp).catch((error) => {
+      negotiationErrorName = error instanceof Error ? error.name : "unknown";
+      useFallback("negotiation_failed", true);
+    });
     return;
   }
   if (message.type === "candidate") {
-    void peer?.addIceCandidate({ candidate: message.candidate, sdpMid: message.mid }).catch(() => {
+    const candidateType = message.candidate.match(/\btyp (host|srflx|prflx|relay)\b/)?.[1];
+    if (candidateType) remoteCandidateTypes.add(candidateType);
+    const pc = peer;
+    void pc?.addIceCandidate({ candidate: message.candidate, sdpMid: message.mid }).catch(() => {
+      if (peer === pc) candidateErrors += 1;
     });
     return;
   }
@@ -504,15 +564,15 @@ var handleServerMessage = (message, viaSocket) => {
   config = message;
   request = message.request;
   fitScreen();
-  if (request.transport !== "webrtc") useFallback("WebSocket transport selected");
-  else if (!message.iceServers) useFallback("server has no WebRTC");
-  else if (typeof RTCPeerConnection !== "function") useFallback("browser has no WebRTC");
+  if (request.transport !== "webrtc") useFallback("websocket_selected");
+  else if (!message.iceServers) useFallback("server_unavailable");
+  else if (typeof RTCPeerConnection !== "function") useFallback("browser_unavailable");
   else if (webrtcSkips > 0) {
     webrtcSkips -= 1;
-    useFallback("recent WebRTC failure");
+    useFallback("recent_failure");
   } else if (!transport) {
     showStatus("Establishing connection\u2026");
-    webrtcDeadline ??= setTimeout(() => useFallback("WebRTC did not decode video in time", true), webrtcFrameTimeoutMs);
+    webrtcDeadline ??= setTimeout(() => useFallback("video_timeout", true), webrtcFrameTimeoutMs);
   }
 };
 var handleServerText = (data, viaSocket) => {
@@ -529,6 +589,19 @@ var connect = () => {
   clearTimeout(reconnectTimer);
   closePeer();
   transport = void 0;
+  transportReason = "";
+  failure = void 0;
+  config = void 0;
+  telemetry = void 0;
+  connectionStartedAt = performance.now();
+  localCandidateTypes.clear();
+  remoteCandidateTypes.clear();
+  iceErrorCodes.clear();
+  candidateErrors = 0;
+  negotiationErrorName = "";
+  lastFrameAt = 0;
+  route = { path: "", codec: "", decoded: "", kbps: 0, displayedFps: 0 };
+  announceTransport();
   socket?.close();
   showStatus(reconnectAttempt ? "reconnecting\u2026" : "connecting\u2026");
   const query = new URLSearchParams([...params].filter(([key]) => ["transport", "profile", "adapt", "ice"].includes(key)));
@@ -754,7 +827,7 @@ setInterval(() => {
   if (screen.paused && screen.srcObject instanceof MediaStream && screen.srcObject.getVideoTracks().some((track) => track.readyState === "live")) return;
   const now = performance.now();
   if (now - lastFrameAt >= webrtcFrameTimeoutMs) {
-    useFallback("WebRTC video stopped arriving", true);
+    useFallback("video_stalled", true);
     return;
   }
   if (inputChannel?.readyState !== "open") return;
@@ -778,7 +851,7 @@ setInterval(() => {
     displayedWindow = { at: now, frames: presentedFrames };
   }
   const elapsedSeconds = (now - renderedWindowStartedAt) / 1e3;
-  if (transport !== "webrtc" && elapsedSeconds < 1) return;
+  if (!peer && transport !== "webrtc" && elapsedSeconds < 1) return;
   const stats = {
     sequence: ++statsSequence,
     videoFps: renderedFrames / elapsedSeconds,
@@ -789,7 +862,7 @@ setInterval(() => {
   renderedFrames = 0;
   decodedAudioPackets = 0;
   renderedWindowStartedAt = now;
-  if (transport === "webrtc" && peer) void reportWebRtcStats(peer, stats).catch(() => {
+  if (peer) void reportWebRtcStats(peer, stats).catch(() => {
   });
   else if (transport === "websocket") {
     sendInput({ type: "stats", ...stats });

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "replicas-computer",
-  "version": "0.3.5",
+  "version": "0.3.7",
   "description": "Replicas workspace computer-use runtime",
   "type": "module",
   "main": "dist/index.js",
@@ -12,7 +12,8 @@
     "assets",
     "scripts/build-webrtc.sh",
     "patches/libdatachannel-dtls-bio.patch",
-    "patches/libdatachannel-nack-duplicates.patch"
+    "patches/libdatachannel-nack-duplicates.patch",
+    "patches/libjuice-ice-role-presence.patch"
   ],
   "scripts": {
     "build": "bun run typecheck && tsup",

package/patches/libjuice-ice-role-presence.patch

@@ -0,0 +1,96 @@
+diff --git a/src/agent.c b/src/agent.c
+--- a/src/agent.c
++++ b/src/agent.c
+@@ -1437,7 +1437,7 @@
+ 			return -1;
+ 
+ 		ice_candidate_pair_t *pair = entry->pair;
+-		if (msg->ice_controlling == msg->ice_controlled) {
++		if (msg->has_ice_controlling == msg->has_ice_controlled) {
+ 			JLOG_WARN("Controlling and controlled attributes mismatch in request");
+ 			agent_send_stun_binding(agent, entry, STUN_CLASS_RESP_ERROR, 400, msg->transaction_id,
+ 			                        NULL);
+@@ -1451,7 +1451,7 @@
+ 		//  ERROR-CODE attribute with a value of 487 (Role Conflict) but retains its role.
+ 		//  * If the agent's tiebreaker value is less than the contents of the ICE-CONTROLLING
+ 		//  attribute, the agent switches to the controlled role.
+-		if (agent->mode == AGENT_MODE_CONTROLLING && msg->ice_controlling) {
++		if (agent->mode == AGENT_MODE_CONTROLLING && msg->has_ice_controlling) {
+ 			JLOG_WARN("ICE role conflict (both controlling)");
+ 			if (agent->ice_tiebreaker >= msg->ice_controlling) {
+ 				JLOG_DEBUG("Asking remote peer to switch roles");
+@@ -1471,7 +1471,7 @@
+ 		//  * If the agent's tiebreaker value is less than the contents of the ICE-CONTROLLED
+ 		//  attribute, the agent generates a Binding error response and includes an ERROR-CODE
+ 		//  attribute with a value of 487 (Role Conflict) but retains its role.
+-		if (agent->mode == AGENT_MODE_CONTROLLED && msg->ice_controlled) {
++		if (agent->mode == AGENT_MODE_CONTROLLED && msg->has_ice_controlled) {
+ 			JLOG_WARN("ICE role conflict (both controlled)");
+ 			if (agent->ice_tiebreaker >= msg->ice_controlling) {
+ 				JLOG_DEBUG("Switching to controlling role");
+@@ -1485,7 +1485,7 @@
+ 			break;
+ 		}
+ 		if (msg->use_candidate) {
+-			if (!msg->ice_controlling) {
++			if (!msg->has_ice_controlling) {
+ 				JLOG_WARN("STUN message use_candidate missing ice_controlling attribute");
+ 				agent_send_stun_binding(agent, entry, STUN_CLASS_RESP_ERROR, 400,
+ 				                        msg->transaction_id, NULL);
+@@ -1715,6 +1715,8 @@
+ 			password = agent->remote.ice_pwd;
+ 			msg.ice_controlling = agent->mode == AGENT_MODE_CONTROLLING ? agent->ice_tiebreaker : 0;
+ 			msg.ice_controlled = agent->mode == AGENT_MODE_CONTROLLED ? agent->ice_tiebreaker : 0;
++			msg.has_ice_controlling = agent->mode == AGENT_MODE_CONTROLLING;
++			msg.has_ice_controlled = agent->mode == AGENT_MODE_CONTROLLED;
+ 
+ 			// RFC 8445 7.1.1. PRIORITY
+ 			// The PRIORITY attribute MUST be included in a Binding request and be set to the value
+diff --git a/src/stun.c b/src/stun.c
+--- a/src/stun.c
++++ b/src/stun.c
+@@ -157,14 +157,14 @@
+ 			goto overflow;
+ 		pos += len;
+ 	}
+-	if (msg->ice_controlling) {
++	if (msg->has_ice_controlling || msg->ice_controlling) {
+ 		uint64_t ice_controlling = htonll(msg->ice_controlling);
+ 		len = stun_write_attr(pos, end - pos, STUN_ATTR_ICE_CONTROLLING, &ice_controlling, 8);
+ 		if (len <= 0)
+ 			goto overflow;
+ 		pos += len;
+ 	}
+-	if (msg->ice_controlled) {
++	if (msg->has_ice_controlled || msg->ice_controlled) {
+ 		uint64_t ice_controlled = htonll(msg->ice_controlled);
+ 		len = stun_write_attr(pos, end - pos, STUN_ATTR_ICE_CONTROLLED, &ice_controlled, 8);
+ 		if (len <= 0)
+@@ -916,6 +916,7 @@
+ 		}
+ 		uint32_t *value32 = (uint32_t *)attr->value;
+ 		msg->ice_controlling = ((uint64_t)ntohl(value32[0]) << 32) | ntohl(value32[1]);
++		msg->has_ice_controlling = true;
+ 		break;
+ 	}
+ 	case STUN_ATTR_ICE_CONTROLLED: {
+@@ -926,6 +927,7 @@
+ 		}
+ 		uint32_t *value32 = (uint32_t *)attr->value;
+ 		msg->ice_controlled = ((uint64_t)ntohl(value32[0]) << 32) | ntohl(value32[1]);
++		msg->has_ice_controlled = true;
+ 		break;
+ 	}
+ 	case STUN_ATTR_CHANNEL_NUMBER: {
+diff --git a/src/stun.h b/src/stun.h
+--- a/src/stun.h
++++ b/src/stun.h
+@@ -319,6 +319,8 @@
+ 	uint32_t priority;
+ 	uint64_t ice_controlling;
+ 	uint64_t ice_controlled;
++	bool has_ice_controlling;
++	bool has_ice_controlled;
+ 	bool use_candidate;
+ 	addr_record_t mapped;
+ 

package/scripts/build-webrtc.sh

@@ -19,6 +19,7 @@ git -C "$source_dir" checkout -q --detach FETCH_HEAD
 git -C "$source_dir" submodule update --init --recursive --depth 1
 git -C "$source_dir" apply "$script_dir/../patches/libdatachannel-dtls-bio.patch"
 git -C "$source_dir" apply "$script_dir/../patches/libdatachannel-nack-duplicates.patch"
+git -C "$source_dir/deps/libjuice" apply "$script_dir/../patches/libjuice-ice-role-presence.patch"
 
 cmake -S "$build_root" -B "$build_root/build" -G Ninja \
   -DCMAKE_BUILD_TYPE=Release \