@torrent-tv/proxy 2.9.99 → 2.9.101

package/CHANGELOG.md

@@ -1,3 +1,13 @@
+## 2.9.101
+
+- **New**: The playback plan reports what this host takes to produce a session's first segment — the median of its last eight, measured from session-create to a servable segment (782-1518 ms on the field host). The browser needs it for the gap between "the file is downloaded" and "a segment exists", where until now it assumed the pipeline merely keeps up with realtime and therefore showed 15 s where 3.8 s were left. It is per-host, so a weak box and a fast one each answer for themselves.
+
+## 2.9.100
+
+- **New**: A long wait for a piece now says who was working on it. The open question about a seek is that a single 8 MiB piece takes 3.0-4.6 s while the swarm as a whole moves 4-6 MB/s, so only about 2 MB/s reaches the piece being waited for — and whether that is because few peers hold it, few are being asked, or each is slow could not be told apart from outside. The line now carries the rate achieved on that piece and, sampled at its peak while waiting, how many connected peers had it, how many were asked, and how many blocks were in flight.
+- **New**: The keyframe index is read alongside the codec probe instead of after it. Both wait for the same tail of the file; measured 2026-08-04, a probe of 722-1206 ms was followed by an index read of 311-430 ms, all of it before the first segment could be produced. Started together the second is free. Fire and forget, sharing the cache a session would fill itself.
+- **Chore**: Every encoder run is numbered in the log. A burst of seeks starts several runs within a second and every line about them carries the session id, which is the same for all of them — so the command that failed could not be told from the ones that succeeded around it. That is the state the unexplained `Cannot write moov atom before AC3 packets` was found in.
+
 ## 2.9.99
 
 - **New**: A source can be told to start before anyone asks to play it — `POST /api/sources/:sourceKey/warm`. Everything a cold torrent must do first takes seconds and none of it depends on which file is wanted: announce to the trackers, connect to peers, be unchoked by them. Given a file index it also fetches the two pieces at that file's edges, which is what the codec probe reads and what took **6.7 s of the 10.3 s** before playback in the session measured 2026-08-04. All of it used to begin only once a file had been chosen, because it was buried inside the playback plan. The route returns as soon as the work is under way and reports a refusal rather than an error — nothing is broken if a warm-up does not happen, since the ordinary path still does all of it.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@torrent-tv/proxy",
-  "version": "2.9.99",
+  "version": "2.9.101",
   "description": "Torrent proxy client that exposes webseed-like HTTP stream endpoint.",
   "license": "GPL-3.0-or-later",
   "publishConfig": {

package/server.js

@@ -163,7 +163,9 @@ export async function startProxyServer({ host, port, transcodeAudio, ffmpegBin,
     transcodeAudioEnabled: transcodeAudio,
     localBaseUrl: hlsSessionManager.localBaseUrl,
     sourceRegistry,
-    torrentPool
+    torrentPool,
+    warmKeyframeIndex: (params) => hlsSessionManager.warmKeyframeIndex(params),
+    expectedFirstSegmentMs: () => hlsSessionManager.expectedFirstSegmentMs()
   });
 
   app.get("/health", async (req, reply) => handleHealthGet(req, reply, { version }));

package/services/hls-session-manager.js

@@ -800,6 +800,14 @@ function normalizeLogFileName(fileName, fileIndex) {
  * expired after {@link HlsSessionManagerOptions.sessionTtlMs} of idle time.
  */
 export class HlsSessionManager {
+  /**
+   * Recent times from session-create to a servable first segment, in ms.
+   * See #rememberFirstSegmentLatency.
+   *
+   * @type {number[]}
+   */
+  #firstSegmentLatencies = [];
+
   /**
    * @param {HlsSessionManagerOptions} options
    */
@@ -1379,6 +1387,29 @@ export class HlsSessionManager {
     return null;
   }
 
+  /**
+   * Read the file's keyframe index into the cache before a session needs it.
+   *
+   * The index lives at the END of a Matroska file, which is also where the
+   * codec probe reads — both wait for the same piece to arrive, and they used
+   * to do it one after the other: measured 2026-08-04, a probe of 722-1206 ms
+   * followed by an index read of 311-430 ms, all of it before the first
+   * segment. Started together, the second costs nothing.
+   *
+   * Never rejects and is never awaited by the caller: a session that finds
+   * nothing cached simply reads it itself, as before.
+   *
+   * @param {{ sourceKey: string, fileIndex: number, inputUrl: URL, logName: string }} params
+   * @returns {Promise<void>}
+   */
+  async warmKeyframeIndex({ sourceKey, fileIndex, inputUrl, logName }) {
+    try {
+      await this.#readContainerKeyframes({ sourceKey, fileIndex, inputUrl, logName });
+    } catch {
+      // Best effort by construction.
+    }
+  }
+
   async #readContainerKeyframes({ sourceKey, fileIndex, inputUrl, logName }) {
     const cacheKey = `${sourceKey}:${fileIndex}`;
     if (this.keyframeIndexCache.has(cacheKey)) {
@@ -2165,7 +2196,15 @@ export class HlsSessionManager {
     // were verified to handle a copied AC-3 track on this very host, so the
     // arguments that run actually received are the missing evidence. One line
     // per run, and a run happens at most every few seconds.
-    logger.info(`transcode ${session.id} ffmpeg ${describeFfmpegArgs(args)}`);
+    // Numbered, because a burst of seeks starts several runs in one second and
+    // every line about them carries the SESSION id, which is the same for all.
+    // Without a run number the command that failed cannot be told from the two
+    // that succeeded around it — which is exactly the state the unexplained
+    // `Cannot write moov atom before AC3 packets` was found in.
+    session.runCounter = (session.runCounter ?? 0) + 1;
+    const runLabel = `run#${session.runCounter}`;
+    session.runLabel = runLabel;
+    logger.info(`transcode ${session.id} ${runLabel} ffmpeg ${describeFfmpegArgs(args)}`);
 
     const ffmpeg = spawn(this.ffmpegBin, args, {
       cwd: session.dirPath,
@@ -2191,7 +2230,7 @@ export class HlsSessionManager {
     session.budgetSlowSince = 0;
 
     logger.info(
-      `transcode ${session.id} encode-run from segment #${safeIndex} ` +
+      `transcode ${session.id} ${session.runLabel} encode-run from segment #${safeIndex} ` +
         `(${formatSeconds(startSeconds)}) "${session.fileName}"`
     );
 
@@ -2379,7 +2418,9 @@ export class HlsSessionManager {
       session.state = "failed";
       session.progress.state = "failed";
       session.progress.updatedAt = Date.now();
-      logger.error(`transcode ${session.id} encode-run failed: ${session.lastError}`);
+      logger.error(
+        `transcode ${session.id} ${session.runLabel ?? "run#?"} encode-run failed: ${session.lastError}`
+      );
     });
   }
 
@@ -2694,6 +2735,45 @@ export class HlsSessionManager {
     return session.requestSeqCounter;
   }
 
+  /**
+   * Remember how long this host took to make a session's first segment.
+   *
+   * The browser has to answer "how long until playback" during the gap between
+   * the file being downloaded and the first segment existing, and until now it
+   * assumed the pipeline merely keeps up with realtime — which on the measured
+   * session meant showing 15 s where 3.8 s were left, and showing it as a jump
+   * UP from 5.5 s. This host knows the real figure because it has just done it
+   * several times: 782 ms, 1052 ms, 1387 ms, 1518 ms on the sessions measured
+   * 2026-08-04/05. A median of recent runs is a measurement, not an assumption,
+   * and it is per-host, so a weak box and a fast one each get their own.
+   *
+   * @param {number} latencyMs
+   * @returns {void}
+   */
+  #rememberFirstSegmentLatency(latencyMs) {
+    if (!Number.isFinite(latencyMs) || latencyMs <= 0) {
+      return;
+    }
+    this.#firstSegmentLatencies.push(latencyMs);
+    if (this.#firstSegmentLatencies.length > FIRST_SEGMENT_SAMPLES) {
+      this.#firstSegmentLatencies.shift();
+    }
+  }
+
+  /**
+   * What this host typically takes to produce a session's first segment, in
+   * milliseconds — the median of recent runs, or null before any has finished.
+   *
+   * @returns {number | null}
+   */
+  expectedFirstSegmentMs() {
+    if (this.#firstSegmentLatencies.length === 0) {
+      return null;
+    }
+    const sorted = [...this.#firstSegmentLatencies].sort((left, right) => left - right);
+    return sorted[Math.floor(sorted.length / 2)];
+  }
+
   /**
    * How many times the viewer has moved since this session started.
    *
@@ -2848,6 +2928,7 @@ export class HlsSessionManager {
       // — the time from session-create entry to a playable first segment.
       if (!isPlaylist && !session.firstSegmentLogged) {
         session.firstSegmentLogged = true;
+        this.#rememberFirstSegmentLatency(Date.now() - session.createEntryMs);
         logger.info(
           `cold-start ${sessionId.slice(0, 8)}: first-segment ready +${Date.now() - session.createEntryMs}ms`
         );

package/services/playback-planner.js

@@ -263,7 +263,17 @@ export function createPlaybackPlanner({
   transcodeAudioEnabled,
   localBaseUrl,
   sourceRegistry,
-  torrentPool
+  torrentPool,
+  // Optional. Reports what this host typically takes to produce a session's
+  // first segment. The browser needs it for the gap between "the file is
+  // downloaded" and "a segment exists": until now it assumed the pipeline
+  // merely keeps up with realtime, and showed 15 s where 3.8 s were left.
+  expectedFirstSegmentMs,
+  // Optional. Called once the file's edges are downloaded, so the keyframe
+  // index — which reads the same tail of the file — is fetched alongside the
+  // codec probe instead of after it. Late-bound to the HLS session manager,
+  // which owns the cache both of them share.
+  warmKeyframeIndex
 }) {
   /** @type {Map<string, PlaybackPlan>} */
   const cache = new Map();
@@ -365,6 +375,17 @@ export function createPlaybackPlanner({
       // file, and an unsupported codec like xvid gets copied → black video.
       await torrentPool.prefetchFileEdges(torrent, fileIndex);
       edgesReadyMs = Date.now() - planEntryMs;
+      // The keyframe index reads the tail of the file, which the probe has just
+      // waited for as well. Started here it overlaps the probe instead of
+      // following the whole plan — worth 311-430 ms of the time before the
+      // first segment. Fire and forget: the session reads it itself if this has
+      // not finished, and both share one cache entry.
+      warmKeyframeIndex?.({
+        sourceKey,
+        fileIndex,
+        inputUrl: new URL(directUrl),
+        logName: file.name
+      });
       let probe = await probeStreamCodecs({ ffmpegBin, inputUrl: directUrl, userAgent });
       const probeDeadline = Date.now() + Math.max(0, maxWaitMs);
       let attempt = 0;
@@ -380,6 +401,7 @@ export function createPlaybackPlanner({
       }
       const { audioCodec, videoCodec, container, durationSeconds, videoWidth, videoHeight, audioTracks, subtitleTracks } = probe;
       const codecsDetected = audioCodec.length > 0 || videoCodec.length > 0;
+      const firstSegmentMs = expectedFirstSegmentMs?.() ?? null;
       logger.info(
         `plan ${sourceKey.slice(0, 8)}:${fileIndex} torrent-ready=${torrentReadyMs}ms ` +
           `file-edges=${edgesReadyMs - torrentReadyMs}ms probe=${Date.now() - planEntryMs - edgesReadyMs}ms ` +
@@ -405,7 +427,10 @@ export function createPlaybackPlanner({
         videoHeight,
         // Full track inventory for the browser's audio/subtitle menus.
         audioTracks: audioTracks ?? [],
-        subtitleTracks: subtitleTracks ?? []
+        subtitleTracks: subtitleTracks ?? [],
+        // What this host has recently taken to make a session's first segment.
+        // Null until one has finished since startup.
+        expectedFirstSegmentMs: firstSegmentMs
       };
       // Only cache a plan whose codecs were actually detected. An empty probe is
       // a "header not downloaded yet" signal, not a valid result — caching it

package/services/torrent-worker/piece-reader.js

@@ -154,6 +154,41 @@ function clearCritical(torrent, { from, to }) {
   }
 }
 
+/**
+ * Who is working on the piece a reader is blocked on, right now.
+ *
+ * The open question about a seek: a single 8 MiB piece takes 3.0-4.6 s to
+ * arrive while the swarm as a whole is moving 4-6 MB/s, so roughly 2 MB/s is
+ * reaching the piece that is actually being waited for. Whether that is because
+ * few peers hold it, few are being asked, or each is slow cannot be told apart
+ * from the outside — these three counts tell them apart.
+ *
+ * `wire.requests` is what has been asked of that peer and not yet answered; a
+ * block is 16 KB, so `blocks x 16 KB` is the work in flight on this piece.
+ *
+ * @param {import("webtorrent").Torrent} torrent
+ * @param {number} pieceIndex
+ * @returns {{ peers: number, holders: number, askedOf: number, blocks: number }}
+ */
+export function pieceSupply(torrent, pieceIndex) {
+  const wires = Array.isArray(torrent?.wires) ? torrent.wires : [];
+  let holders = 0;
+  let askedOf = 0;
+  let blocks = 0;
+  for (const wire of wires) {
+    if (wire?.peerPieces?.get?.(pieceIndex)) {
+      holders += 1;
+    }
+    const requests = Array.isArray(wire?.requests) ? wire.requests : [];
+    const forThisPiece = requests.filter((request) => request?.piece === pieceIndex).length;
+    if (forThisPiece > 0) {
+      askedOf += 1;
+      blocks += forThisPiece;
+    }
+  }
+  return { peers: wires.length, holders, askedOf, blocks };
+}
+
 /**
  * Wait until a piece has been downloaded and verified.
  *
@@ -421,7 +456,20 @@ export async function* readFragments({
       }
 
       const waitStartedAt = Date.now();
-      await whenPieceReady(torrent, pieceIndex, cancellation);
+      // Sampled while waiting rather than after: once the piece lands, nothing
+      // is outstanding on it any more and every count reads zero.
+      let supply = null;
+      const supplyProbe = setInterval(() => {
+        const sample = pieceSupply(torrent, pieceIndex);
+        if (!supply || sample.blocks > supply.blocks) {
+          supply = sample;
+        }
+      }, 500);
+      try {
+        await whenPieceReady(torrent, pieceIndex, cancellation);
+      } finally {
+        clearInterval(supplyProbe);
+      }
       // What a reader spent waiting for data, attributed to the exact piece. A
       // seek's cost is dominated by the first segment after the encoder
       // restarts (measured 9.2-9.4 s), and without this there is no way to say
@@ -429,10 +477,16 @@ export async function* readFragments({
       // wait is long enough to matter, so ordinary sequential reading is silent.
       const waitedMs = Date.now() - waitStartedAt;
       if (waitedMs >= PIECE_WAIT_LOG_MS) {
+        const rateKbps = Math.round(pieceLength / 1024 / (waitedMs / 1000));
         logger.info(
           `piece-reader: waited ${waitedMs}ms for piece ${pieceIndex} ` +
             `(${pieceIndex - firstPiece + 1} of ${lastPiece - firstPiece + 1} in a read from ` +
-            `${(start / 1024 / 1024).toFixed(0)}MB of "${file.name}")`
+            `${(start / 1024 / 1024).toFixed(0)}MB of "${file.name}") ` +
+            `— ${rateKbps}KB/s on this piece; ` +
+            (supply
+              ? `${supply.holders}/${supply.peers} peers had it, ${supply.askedOf} were asked, ` +
+                `${supply.blocks} blocks (${Math.round((supply.blocks * 16384) / 1024)}KB) in flight at peak`
+              : "no sample taken")
         );
       }