@torrent-tv/proxy 2.9.48 → 2.9.50

package/CHANGELOG.md

@@ -1,3 +1,11 @@
+## 2.9.50
+
+- **Fix**: Seeking could get stuck in an infinite restart loop on some containers (observed: AVI with VBR MP3 audio), producing nothing for ~90 s until the whole WebRTC session died — the on-screen symptom of "seeking does nothing." Root cause, two parts: (1) `-accurate_seek -ss X` before `-i` trusts the container's own on-the-fly seek/index to land near X; for this AVI it pointed at a position with no valid frame boundary at all, so ffmpeg failed outright ("Seek failed" / "Header missing") — not just imprecisely — and every retry re-tried the SAME bad container-computed position. (2) `#ensureEncodingFor`/`#fireSettledSeek` never checked for a `"failed"` session state, and `#startEncodeRun` unconditionally resets state back to `"starting"` on every call — so a failed run's next client poll silently re-armed and re-ran the identical failing seek, forever. Fixed both: the video-keyframe probe (previously only used for the copy path's segment boundaries) now also feeds a two-step seek — jump to the nearest REAL keyframe (a position ffmpeg has already proven it can decode, read directly from the packet list, not the container's live index) before `-i`, then trim the short residual precisely after `-i` (always frame-accurate, no reliance on `-accurate_seek`'s trust in the container). A circuit breaker caps consecutive fast failures (exits within 2 s — never did real work) at the SAME target to 3 before the session is left in its terminal `failed` state instead of looping — a different seek target still gets a fresh attempt budget. Verified: the keyframe-snap helper against synthetic data, and the breaker's state machine (3 attempts at one target → blocked, a different target → fresh budget, only 4 real ffmpeg spawns instead of an unbounded loop).
+
+## 2.9.49
+
+- **New**: `getSessionProgress` (the transcode-session progress endpoint) now also reports `outputMbps` — the observed produced bitrate from recently completed segment files (already computed internally for the viewer-link budget check, `#checkLinkBudget`), so the browser can turn its OWN measured link throughput into a delivery-speed multiplier for the unified download/transcode/delivery playback-start ETA, the same way the transcode's own `speed` already is one.
+
 ## 2.9.48
 
 - **Fix**: The "bytes still needed to resume" figure shown while buffering could jump UP mid-poll even though nothing regressed, which read as confusing/broken. Root cause: the resume-window progress (`resumeNeededBytes`/`resumeDownloadedBytes`) was always computed against the LIVE read position, which slides forward as the file is read/transcoded further — so when the window moved past an already-downloaded piece into a fresh, never-touched one, "bytes needed" jumped up (a moving reference frame, not a real setback). `getFileStats` now accepts an optional `resumeAnchorByteStart` and always returns the byte offset the window was computed against; the browser client captures that offset on the FIRST poll of a buffering episode and sends it back on every subsequent poll of the SAME episode, so the window stays pinned to a fixed target and the figure only ever decreases as real download progress happens. Verified: with the anchor pinned, repeated polls report the same "needed" while the live read position moves with no new data, and a real download of a piece inside the frozen window correctly decreases it.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@torrent-tv/proxy",
-  "version": "2.9.48",
+  "version": "2.9.50",
   "description": "Torrent proxy client that exposes webseed-like HTTP stream endpoint.",
   "license": "GPL-3.0-or-later",
   "publishConfig": {

package/routes/api/transcode-sessions/progress/get.js

@@ -14,7 +14,7 @@ export async function handleApiTranscodeSessionsProgressGet(req, reply, { hlsSes
     return reply.code(400).send({ error: "sessionId is required." });
   }
 
-  const progress = hlsSessionManager.getSessionProgress(sessionId);
+  const progress = await hlsSessionManager.getSessionProgress(sessionId);
   if (!progress) {
     return reply.code(404).send({ error: "Transcode session was not found." });
   }

package/services/hls-session-manager.js

@@ -72,6 +72,19 @@ const SEEK_SETTLE_MAX_MS = 2_500;
 // escalation step: SIGTERM, then SIGKILL) before spawning its replacement into
 // the same session directory. See #startEncodeRun.
 const ENCODE_RUN_TERMINATE_GRACE_MS = 2_000;
+// A seek-restart run that exits this fast never did real work — it failed at
+// the seek/open step itself (container demux error, bad audio frame boundary,
+// etc.), not mid-stream. Used to tell a genuine seek failure apart from a
+// later, unrelated crash so the circuit breaker below only counts the former.
+const SEEK_FAST_FAIL_MS = 2_000;
+// Circuit breaker: consecutive fast failures AT THE SAME target before we stop
+// auto-retrying and leave the session in its terminal "failed" state (surfaced
+// to the client as a clean, retryable error) instead of looping forever. The
+// keyframe-snap seek (see #startEncodeRun) already fixes the dominant failure
+// mode (an unreliable container-computed seek position); this is a safety net
+// for whatever residual case still fails — not a second competing "fix" that
+// blindly retries the identical command hoping for a different result.
+const MAX_SEEK_FAILURES = 3;
 // Idle TTL: a session is disposed this long after the last segment/playlist
 // access. Long enough that a viewer who pauses, backgrounds the tab, or briefly
 // turns the phone off can resume WITHOUT a cold ffmpeg restart (the warm session
@@ -575,6 +588,27 @@ function computeSegmentBoundaries({ transcodeVideo, durationSeconds, segDur, key
   return boundaries.length >= 2 ? boundaries : uniform();
 }
 
+/**
+ * The largest keyframe time that does not exceed `target`, from a SORTED
+ * (ascending) array of keyframe times such as {@link probeVideoKeyframeTimes}
+ * returns. Null when `target` is before the first keyframe or the array is
+ * empty — the caller then falls back to its unsnapped target.
+ *
+ * @param {number[]} keyframeTimes - Sorted ascending.
+ * @param {number} target
+ * @returns {number | null}
+ */
+function nearestKeyframeAtOrBefore(keyframeTimes, target) {
+  let result = null;
+  for (const time of keyframeTimes) {
+    if (time > target) {
+      break;
+    }
+    result = time;
+  }
+  return result;
+}
+
 function isWarmupTimeoutError(error) {
   if (!(error instanceof Error)) {
     return false;
@@ -621,6 +655,12 @@ function normalizeLogFileName(fileName, fileIndex) {
  * @property {number}  encodeRunGeneration - Bumped on every #startEncodeRun call;
  *   lets a call that awaited the previous ffmpeg's exit detect it was superseded
  *   by a newer restart request and abort instead of spawning a second process.
+ * @property {number[] | null} keyframeTimes - Real source keyframe times
+ *   (sorted seconds), or null when the probe failed/timed out. Used to snap a
+ *   source seek onto a known-valid position (see #startEncodeRun).
+ * @property {number}  seekFailureTarget - Segment index of the last fast seek
+ *   failure, for the consecutive-failure circuit breaker (see MAX_SEEK_FAILURES).
+ * @property {number}  seekFailureCount  - Consecutive fast failures at seekFailureTarget.
  */
 
 /**
@@ -828,12 +868,22 @@ export class HlsSessionManager {
 
     // For the video-copy path we cannot insert keyframes, so the playlist's
     // segment boundaries must match the source's real keyframes (otherwise the
-    // player sees gaps on seek). Probe them; on failure we fall back to a
-    // uniform grid (current behaviour). Re-encoded video uses a uniform grid
-    // (its fixed GOP makes the cuts land there).
+    // player sees gaps on seek). Re-encoded video uses a uniform grid for
+    // segment boundaries instead (its fixed GOP makes the cuts land there —
+    // computeSegmentBoundaries ignores keyframeTimes when transcodeVideo).
+    //
+    // But the probe is ALSO used for something both branches need: choosing a
+    // SOURCE seek position ffmpeg can actually land on. `-ss` before `-i` trusts
+    // the container's own on-the-fly seek/index, which for some containers
+    // (observed: AVI with VBR MP3 audio) can point at a position with no valid
+    // frame boundary at all — ffmpeg then fails outright ("Seek failed" /
+    // "Header missing"), not just imprecisely. Snapping the seek to the nearest
+    // KNOWN real keyframe (see #startEncodeRun) avoids that. So probe for both
+    // branches; on failure both fall back to their current behaviour (uniform
+    // grid for boundaries, raw target for seeking) — no regression.
     let keyframeTimes = null;
     let keyframeMs = -1; // -1 = not run (skipped)
-    if (hasDuration && !transcodeVideo) {
+    if (hasDuration) {
       // Short timeout: mp4 keyframes come from the moov index (fast); containers
       // that force a full packet scan time out and fall back to a uniform grid,
       // so this never adds more than ~6 s to session start.
@@ -842,8 +892,8 @@ export class HlsSessionManager {
       keyframeMs = Date.now() - keyframeStartMs;
       if (!keyframeTimes) {
         logger.warn(
-          `transcode ${sessionId}: keyframe probe unavailable; using uniform grid ` +
-            `for copied video "${logName}" (seek precision may be reduced)`
+          `transcode ${sessionId}: keyframe probe unavailable; using uniform grid / raw seek targets ` +
+            `for "${logName}" (seek precision may be reduced)`
         );
       }
     }
@@ -953,6 +1003,11 @@ export class HlsSessionManager {
       // keyframe positions for copied video. Drives the playlist and seeking.
       segmentBoundaries,
       segmentCount,
+      // Real source keyframe times (sorted seconds), or null when the probe
+      // failed/timed out. Used by #startEncodeRun to snap a source seek onto a
+      // KNOWN valid position instead of trusting the container's own on-the-fly
+      // seek at an arbitrary target — see the probe call above for why.
+      keyframeTimes,
       playlistText: hasDuration ? this.#buildVodPlaylist(segmentBoundaries) : "",
       // Segment index the current ffmpeg run started producing from.
       encodeStartIndex: 0,
@@ -966,6 +1021,12 @@ export class HlsSessionManager {
       seekSettleTimer: null,
       seekTarget: null,
       seekFirstFarAt: 0,
+      // Circuit breaker: consecutive FAST failures (see SEEK_FAST_FAIL_MS) at
+      // seekFailureTarget. Reset whenever a run starts at a DIFFERENT target or
+      // survives past the fast-fail window. See the exit handler in
+      // #wireEncodeProcess and MAX_SEEK_FAILURES.
+      seekFailureTarget: -1,
+      seekFailureCount: 0,
       progress: {
         state: "starting",
         processedSeconds: 0,
@@ -1512,12 +1573,40 @@ export class HlsSessionManager {
     // (startSeconds is already a real-keyframe offset from 0, so add back the
     // container start time); for re-encode startSeconds is a plain grid offset.
     const seekSeconds = session.transcodeVideo ? startSeconds : startSeconds + sourceStartTime;
-    if (seekSeconds > 0) {
-      // Accurate seek before -i (decodes from the preceding keyframe and trims
-      // to the exact point), so the first output frame is exactly at the target.
-      args.push("-accurate_seek", "-ss", String(seekSeconds));
+    // Two-step seek when we have a real keyframe map: jump to a KNOWN-valid
+    // keyframe (coarse, before -i — safe because WE sourced it from ffprobe,
+    // not the container's own on-the-fly seek/index) and trim the short
+    // residual (bounded by the keyframe interval) precisely AFTER -i, which is
+    // always frame-accurate regardless of -accurate_seek.
+    //
+    // Root cause this works around: `-accurate_seek -ss X` before -i trusts the
+    // CONTAINER's own seek to land near X. For some containers (observed: AVI
+    // with VBR MP3 audio) that on-the-fly seek can point at a position with no
+    // valid frame boundary at all — ffmpeg fails outright ("Seek failed" /
+    // "Header missing"), not just imprecisely, and repeatedly so since every
+    // retry re-tries the SAME bad container-computed position. A keyframe we
+    // read directly from the packet list is a position ffmpeg has already
+    // proven it can decode.
+    const snappedKeyframe = Array.isArray(session.keyframeTimes) && session.keyframeTimes.length > 0
+      ? nearestKeyframeAtOrBefore(session.keyframeTimes, seekSeconds)
+      : null;
+    if (snappedKeyframe !== null) {
+      const residualSeconds = Math.max(0, seekSeconds - snappedKeyframe);
+      if (snappedKeyframe > 0) {
+        args.push("-ss", String(snappedKeyframe));
+      }
+      args.push("-i", session.inputUrl);
+      if (residualSeconds > 0) {
+        args.push("-ss", String(residualSeconds));
+      }
+    } else {
+      if (seekSeconds > 0) {
+        // No keyframe map (probe failed/timed out) — fall back to the previous
+        // behaviour: trust the container's own accurate seek.
+        args.push("-accurate_seek", "-ss", String(seekSeconds));
+      }
+      args.push("-i", session.inputUrl);
     }
-    args.push("-i", session.inputUrl);
     if (session.transcodeVideo) {
       // Branch A (re-encode): fixed GOP makes keyframes land exactly on the
       // segment grid; relabel output onto the original timeline so segment N
@@ -1709,6 +1798,30 @@ export class HlsSessionManager {
         void this.#startEncodeRun(session, session.encodeStartIndex);
         return;
       }
+      // Circuit-breaker bookkeeping: a seek-restart run that exits THIS fast
+      // never did real work — it failed at the seek/open step itself, not
+      // mid-stream (see SEEK_FAST_FAIL_MS). Track consecutive fast failures at
+      // the SAME target so #ensureEncodingFor/#fireSettledSeek (which check
+      // this below) can stop retrying instead of looping forever on a position
+      // that keeps failing even with the keyframe-snapped seek.
+      const elapsedMs = Date.now() - session.lastRestartAt;
+      if (elapsedMs < SEEK_FAST_FAIL_MS && session.encodeStartIndex > 0) {
+        if (session.seekFailureTarget === session.encodeStartIndex) {
+          session.seekFailureCount += 1;
+        } else {
+          session.seekFailureTarget = session.encodeStartIndex;
+          session.seekFailureCount = 1;
+        }
+        logger.warn(
+          `transcode ${session.id} fast failure at segment #${session.encodeStartIndex} ` +
+            `(${elapsedMs}ms) — ${session.seekFailureCount}/${MAX_SEEK_FAILURES} consecutive`
+        );
+      } else {
+        // Real progress was made (or this was the very first run) — not a
+        // repeating seek failure. Reset the breaker.
+        session.seekFailureTarget = -1;
+        session.seekFailureCount = 0;
+      }
       session.state = "failed";
       session.progress.state = "failed";
       session.progress.updatedAt = Date.now();
@@ -1743,6 +1856,15 @@ export class HlsSessionManager {
     if (withinWindow) {
       return;
     }
+    // Circuit breaker: this exact target has already failed MAX_SEEK_FAILURES
+    // times in a row (fast failures — see #wireEncodeProcess's exit handler).
+    // Stop auto-retrying it; session.state stays "failed" so getFileStream
+    // reports a clean, retryable error instead of looping forever. A DIFFERENT
+    // target (the viewer seeking elsewhere) is unaffected — it gets its own
+    // fresh attempt budget.
+    if (index === session.seekFailureTarget && session.seekFailureCount >= MAX_SEEK_FAILURES) {
+      return;
+    }
     // Far request = a server-side seek. Do NOT restart on the first one:
     // debounce a burst of scattered requests into a single restart at the
     // position the player ended on. Record the latest target and (re)arm the
@@ -1776,6 +1898,14 @@ export class HlsSessionManager {
       session.seekFirstFarAt = 0;
       return;
     }
+    // Circuit breaker (defense in depth): a timer armed before the cap was hit
+    // could still be pending when it was reached — do not fire the restart it
+    // was going to make. See the matching check in #ensureEncodingFor.
+    if (target === session.seekFailureTarget && session.seekFailureCount >= MAX_SEEK_FAILURES) {
+      session.seekTarget = null;
+      session.seekFirstFarAt = 0;
+      return;
+    }
     // Minimum gap between actual restarts (the settle already collapses bursts;
     // this only guards back-to-back seeks). If still cooling down, re-arm once
     // for the remaining cooldown instead of restarting now.
@@ -1970,9 +2100,9 @@ export class HlsSessionManager {
    * Also refreshes `lastAccessedAt` to prevent the session from expiring.
    *
    * @param {string} sessionId
-   * @returns {object | null}
+   * @returns {Promise<object | null>}
    */
-  getSessionProgress(sessionId) {
+  async getSessionProgress(sessionId) {
     if (!isSafeSessionId(sessionId)) {
       return null;
     }
@@ -1990,6 +2120,13 @@ export class HlsSessionManager {
     const warmupRemainingSeconds = isWarmupPhase
       ? Math.max(0, warmupTotalSeconds - warmupElapsedSeconds)
       : null;
+    // Observed OUTPUT bitrate (Mbit/s) from recently completed segment sizes —
+    // already computed for the viewer-link budget check (#checkLinkBudget); also
+    // exposed here so the browser can turn its OWN measured link throughput into
+    // a "content-seconds delivered per wall-clock second" rate for the unified
+    // three-stage ETA (download / transcode / delivery), the same way the
+    // transcode's own `speed` already is one. Null when not enough segments yet.
+    const outputMbps = await this.#observedStreamMbps(session);
     return {
       sessionId: session.id,
       state: session.progress.state,
@@ -2005,6 +2142,7 @@ export class HlsSessionManager {
       // of a percentage of the whole-file transcode.
       segmentDurationSec: this.segmentDurationSec,
       speed: session.progress.speed,
+      outputMbps,
       updatedAt: session.progress.updatedAt,
       error: session.state === "failed" ? session.lastError : ""
     };