@torrent-tv/proxy 2.9.90 → 2.9.91

package/CHANGELOG.md

@@ -1,3 +1,8 @@
+## 2.9.91
+
+- **Fix**: The encoder no longer runs away from the viewer. Nothing bounded how far ahead it produced: measured 2026-08-04, three minutes after a film was opened the encode had reached 00:39:24 of a 01:26:51 source at 12.8x while the viewer was still at the start, and the torrent had pulled 80% of 4.7 GB to feed it — the pool owner's bandwidth and disk spent on a viewer who may watch two minutes, the pieces being read evicted from memory by pieces forty minutes ahead, and the swarm busy with anything but the segment being waited for. An encoder more than two minutes of content ahead of the last segment its viewer asked for is now **suspended**, and released once the viewer is within a minute of it — or at once when a segment is requested. Suspended rather than killed on purpose: restarting costs about nine seconds on this hardware, so a viewer reaching the end of the produced range would stall every time, while suspending keeps the process, its input and its position. POSIX only; where `SIGSTOP` does not exist the attempt fails once, is logged, and that session keeps the old behaviour. Every path that terminates an encoder now releases it first — a suspended process does not act on `SIGTERM` until it is continued, which would have hung the wait a seek performs before starting its replacement.
+- **New**: A reader reports what it waited for. When a read blocks a second or more on a piece, the log names the piece, its position in the read, and the offset the read started at. The first segment after a seek-restart costs 9.2-9.4 s and there was no way to tell whether that is the swarm, the piece picker or ffmpeg; now there is.
+
 ## 2.9.90
 
 - **New**: The output container is chosen per session, by the viewer, instead of once per proxy. `POST /api/transcode-sessions` accepts `segmentFormat`; `--segment-format` remains the default for a client that expresses no preference, and an unrecognised value falls back to it rather than to the library default. The browser is the only party that knows what its media stack will accept for the tracks it asked to be copied: a copied MP3 track cannot be appended from fMP4 at all (`audio/mp4; codecs="mp4a.69"` is refused by MediaSource) but works from MPEG-TS, which hls.js demuxes itself and hands to a plain `audio/mpeg` buffer — the same file, the same browser, silent loop one way and normal playback the other. Sessions are keyed by container too, so two viewers wanting different ones do not share an encoder. Nothing branches on the format outside `services/segment-formats/`; the manager now reads it off the session.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@torrent-tv/proxy",
-  "version": "2.9.90",
+  "version": "2.9.91",
   "description": "Torrent proxy client that exposes webseed-like HTTP stream endpoint.",
   "license": "GPL-3.0-or-later",
   "publishConfig": {

package/services/hls-session-manager.js

@@ -63,6 +63,23 @@ const RUN_FIRST_SEGMENT_GRACE_MS = 30_000;
 // and restart ffmpeg at the demanded segment. Conservative — a slow-but-moving
 // encode keeps advancing `updatedAt`, so this only fires on a true freeze.
 const ENCODER_STALL_MS = 12_000;
+// How far ahead of the viewer the encoder may run before it is stopped, and how
+// far it must fall back to before it is let go again.
+//
+// Nothing used to bound this. Measured 2026-08-04 on the copy path: three
+// minutes after a film was opened the encode had reached 00:39:24 of a 01:26:51
+// source at 12.8x while the viewer was still at the start, and the torrent had
+// pulled 80% of 4.7 GB to feed it. That costs the pool owner's bandwidth and
+// disk for a viewer who may watch two minutes, evicts from memory the pieces
+// the viewer is actually reading, and competes for the swarm with the segment
+// being waited on.
+//
+// In seconds of content rather than segments, because a segment is 4 s of
+// re-encoded video but a whole keyframe interval on the copy path. Generous
+// enough that ordinary watching never touches it: the encoder fills two minutes
+// ahead, stops, and is released as soon as the viewer has spent a minute of it.
+const LOOKAHEAD_PAUSE_SECONDS = 120;
+const LOOKAHEAD_RESUME_SECONDS = 60;
 // Seek debounce. A far (out-of-window) segment request is a server-side seek.
 // Rather than restart ffmpeg on the first one, wait a short quiet period:
 // further far requests re-arm it and update the target to the latest index, so
@@ -787,6 +804,7 @@ export class HlsSessionManager {
     // benchmark (the only path that can pick/step resolution). Cheap no-op scan
     // otherwise.
     this.budgetTimer = setInterval(() => {
+      this.#enforceLookAhead();
       void this.#enforceRealtimeBudget();
     }, BUDGET_CHECK_INTERVAL_MS);
     this.budgetTimer.unref();
@@ -1160,6 +1178,12 @@ export class HlsSessionManager {
       // Bumped by every viewer seek; a held segment request that started under
       // an older value gives up at once. See requestSeek.
       waitEpoch: 0,
+      // Highest segment the viewer has actually asked for, and whether the
+      // encoder is currently suspended for running too far past it.
+      // See #enforceLookAhead.
+      lastRequestedSegment: null,
+      encoderPaused: false,
+      encoderPauseUnsupported: false,
       seekFirstFarAt: 0,
       // Circuit breaker: consecutive FAST failures (see SEEK_FAST_FAIL_MS) at
       // seekFailureTarget. Reset whenever a run starts at a DIFFERENT target or
@@ -1549,6 +1573,94 @@ export class HlsSessionManager {
     return true;
   }
 
+  /**
+   * Stop encoders that have run too far ahead of their viewer, and release
+   * those the viewer has caught up with.
+   *
+   * The encoder is SUSPENDED, not killed. Killing would be simpler, but
+   * restarting it costs about nine seconds on this hardware — the torrent has
+   * to serve a fresh position and ffmpeg has to reach its first keyframe — so a
+   * viewer reaching the end of the produced range would stall every time.
+   * Suspending keeps the process, its open input and its position, and costs
+   * nothing to undo.
+   *
+   * POSIX only. `SIGSTOP` does not exist on Windows, where `process.kill`
+   * throws; the attempt is made once per session and, if it fails, that session
+   * simply keeps its old unbounded behaviour rather than breaking.
+   *
+   * @returns {void}
+   */
+  #enforceLookAhead() {
+    for (const session of this.sessionsById.values()) {
+      if (!session || session.state === "disposed" || !session.ffmpeg) {
+        continue;
+      }
+      const encodedTo = Number(session.progress?.processedSeconds);
+      if (!Number.isFinite(encodedTo)) {
+        continue;
+      }
+      // Where the viewer is. Before the first segment request, the position the
+      // run started at — so a session nobody has read from yet is bounded too.
+      const viewerAt = Number.isInteger(session.lastRequestedSegment)
+        ? this.#segmentStartTime(session, session.lastRequestedSegment)
+        : this.#segmentStartTime(session, session.encodeStartIndex ?? 0);
+      const ahead = encodedTo - viewerAt;
+      if (!session.encoderPaused && ahead > LOOKAHEAD_PAUSE_SECONDS) {
+        this.#pauseEncoder(session, `${Math.round(ahead)}s ahead of the viewer`);
+      } else if (session.encoderPaused && ahead <= LOOKAHEAD_RESUME_SECONDS) {
+        this.#resumeEncoder(session, `${Math.round(ahead)}s ahead of the viewer`);
+      }
+    }
+  }
+
+  /**
+   * Suspend a session's encoder. No-op when already paused or unsupported here.
+   *
+   * @param {HlsSession} session
+   * @param {string} reason
+   * @returns {void}
+   */
+  #pauseEncoder(session, reason) {
+    if (session.encoderPaused || session.encoderPauseUnsupported || !session.ffmpeg?.pid) {
+      return;
+    }
+    try {
+      process.kill(session.ffmpeg.pid, "SIGSTOP");
+    } catch (error) {
+      session.encoderPauseUnsupported = true;
+      logger.info(
+        `transcode ${session.id} cannot suspend the encoder on this platform ` +
+          `(${error instanceof Error ? error.message : String(error)}); look-ahead stays unbounded`
+      );
+      return;
+    }
+    session.encoderPaused = true;
+    logger.info(
+      `transcode ${session.id} encoder suspended — ${reason} ` +
+        `"${session.fileName}"`
+    );
+  }
+
+  /**
+   * Let a suspended encoder run again.
+   *
+   * @param {HlsSession} session
+   * @param {string} reason
+   * @returns {void}
+   */
+  #resumeEncoder(session, reason) {
+    if (!session.encoderPaused || !session.ffmpeg?.pid) {
+      return;
+    }
+    try {
+      process.kill(session.ffmpeg.pid, "SIGCONT");
+    } catch {
+      // The process is gone; the exit handler will deal with it.
+    }
+    session.encoderPaused = false;
+    logger.info(`transcode ${session.id} encoder resumed — ${reason} "${session.fileName}"`);
+  }
+
   async #enforceRealtimeBudget() {
     if (this.videoEncoder?.kind !== "software") {
       return;
@@ -1728,6 +1840,9 @@ export class HlsSessionManager {
   async #startEncodeRun(session, startIndex) {
     const generation = ++session.encodeRunGeneration;
     const previousFfmpeg = session.ffmpeg;
+    // A suspended process does not act on SIGTERM until it is continued, so the
+    // wait below would never end. Let it run before asking it to stop.
+    this.#resumeEncoder(session, "terminating for a new run");
     if (previousFfmpeg && !hasChildExited(previousFfmpeg)) {
       try {
         previousFfmpeg.kill("SIGTERM");
@@ -1760,6 +1875,7 @@ export class HlsSessionManager {
     // Terminate any existing encode process before starting a new one.  The
     // old process's exit handler no-ops because session.ffmpeg is reassigned
     // below (it checks identity).
+    this.#resumeEncoder(session, "terminating");
     if (session.ffmpeg && !session.ffmpeg.killed) {
       try {
         session.ffmpeg.kill("SIGTERM");
@@ -1933,6 +2049,7 @@ export class HlsSessionManager {
     // judged finished — see getFileStream.
     session.usesExplicitCuts = Boolean(cutTimes && cutTimes.length > 0);
     session.encodeStartIndex = safeIndex;
+    session.encoderPaused = false;
     session.pendingRestartIndex = -1;
     session.lastRestartAt = Date.now();
     session.state = session.state === "disposed" ? "disposed" : "starting";
@@ -2585,6 +2702,17 @@ export class HlsSessionManager {
 
     const filePath = path.join(session.dirPath, fileName);
     const isPlaylist = fileName === PLAYLIST_FILE_NAME;
+    if (!isPlaylist) {
+      // Where the viewer actually is. Recorded for every segment request,
+      // served or not, because it is what bounds how far ahead the encoder is
+      // allowed to run — see #enforceLookAhead.
+      const requested = session.segmentFormat.segmentIndexFromName(fileName);
+      if (requested >= 0) {
+        session.lastRequestedSegment = requested;
+        // A viewer who has caught up must not wait out the monitor's interval.
+        this.#resumeEncoder(session, "a segment was requested");
+      }
+    }
     try {
       await access(filePath);
 
@@ -2793,6 +2921,7 @@ export class HlsSessionManager {
       session.seekSettleTimer = null;
     }
 
+    this.#resumeEncoder(session, "session disposed");
     if (session.ffmpeg && !session.ffmpeg.killed) {
       session.ffmpeg.kill("SIGTERM");
       await waitForChildExit(session.ffmpeg);

package/services/torrent-worker/piece-reader.js

@@ -21,6 +21,10 @@
  */
 
 import { findSharedStore } from "../piece-store/shared-piece-store.js";
+import { logger } from "../../utils/logger.js";
+
+/** Only waits at least this long are reported; sequential reading stays silent. */
+const PIECE_WAIT_LOG_MS = 1_000;
 
 /**
  * How far ahead of the read head pieces are asked for.
@@ -318,7 +322,21 @@ export async function* readFragments({
         criticalMark = markCritical(torrent, pieceIndex, Math.min(lastPiece, pieceIndex + criticalRun), criticalMark);
       }
 
+      const waitStartedAt = Date.now();
       await whenPieceReady(torrent, pieceIndex, cancellation);
+      // What a reader spent waiting for data, attributed to the exact piece. A
+      // seek's cost is dominated by the first segment after the encoder
+      // restarts (measured 9.2-9.4 s), and without this there is no way to say
+      // whether that is the swarm, the picker, or ffmpeg. Logged only when the
+      // wait is long enough to matter, so ordinary sequential reading is silent.
+      const waitedMs = Date.now() - waitStartedAt;
+      if (waitedMs >= PIECE_WAIT_LOG_MS) {
+        logger.info(
+          `piece-reader: waited ${waitedMs}ms for piece ${pieceIndex} ` +
+            `(${pieceIndex - firstPiece + 1} of ${lastPiece - firstPiece + 1} in a read from ` +
+            `${(start / 1024 / 1024).toFixed(0)}MB of "${file.name}")`
+        );
+      }
 
       // Pinned BEFORE it is located, and before any await that could let an
       // eviction run: the offset is only meaningful while the piece is held.