@torrent-tv/proxy 2.9.99 → 2.9.100

package/CHANGELOG.md

@@ -1,3 +1,9 @@
+## 2.9.100
+
+- **New**: A long wait for a piece now says who was working on it. The open question about a seek is that a single 8 MiB piece takes 3.0-4.6 s while the swarm as a whole moves 4-6 MB/s, so only about 2 MB/s reaches the piece being waited for — and whether that is because few peers hold it, few are being asked, or each is slow could not be told apart from outside. The line now carries the rate achieved on that piece and, sampled at its peak while waiting, how many connected peers had it, how many were asked, and how many blocks were in flight.
+- **New**: The keyframe index is read alongside the codec probe instead of after it. Both wait for the same tail of the file; measured 2026-08-04, a probe of 722-1206 ms was followed by an index read of 311-430 ms, all of it before the first segment could be produced. Started together the second is free. Fire and forget, sharing the cache a session would fill itself.
+- **Chore**: Every encoder run is numbered in the log. A burst of seeks starts several runs within a second and every line about them carries the session id, which is the same for all of them — so the command that failed could not be told from the ones that succeeded around it. That is the state the unexplained `Cannot write moov atom before AC3 packets` was found in.
+
 ## 2.9.99
 
 - **New**: A source can be told to start before anyone asks to play it — `POST /api/sources/:sourceKey/warm`. Everything a cold torrent must do first takes seconds and none of it depends on which file is wanted: announce to the trackers, connect to peers, be unchoked by them. Given a file index it also fetches the two pieces at that file's edges, which is what the codec probe reads and what took **6.7 s of the 10.3 s** before playback in the session measured 2026-08-04. All of it used to begin only once a file had been chosen, because it was buried inside the playback plan. The route returns as soon as the work is under way and reports a refusal rather than an error — nothing is broken if a warm-up does not happen, since the ordinary path still does all of it.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@torrent-tv/proxy",
-  "version": "2.9.99",
+  "version": "2.9.100",
   "description": "Torrent proxy client that exposes webseed-like HTTP stream endpoint.",
   "license": "GPL-3.0-or-later",
   "publishConfig": {

package/server.js

@@ -163,7 +163,8 @@ export async function startProxyServer({ host, port, transcodeAudio, ffmpegBin,
     transcodeAudioEnabled: transcodeAudio,
     localBaseUrl: hlsSessionManager.localBaseUrl,
     sourceRegistry,
-    torrentPool
+    torrentPool,
+    warmKeyframeIndex: (params) => hlsSessionManager.warmKeyframeIndex(params)
   });
 
   app.get("/health", async (req, reply) => handleHealthGet(req, reply, { version }));

package/services/hls-session-manager.js

@@ -1379,6 +1379,29 @@ export class HlsSessionManager {
     return null;
   }
 
+  /**
+   * Read the file's keyframe index into the cache before a session needs it.
+   *
+   * The index lives at the END of a Matroska file, which is also where the
+   * codec probe reads — both wait for the same piece to arrive, and they used
+   * to do it one after the other: measured 2026-08-04, a probe of 722-1206 ms
+   * followed by an index read of 311-430 ms, all of it before the first
+   * segment. Started together, the second costs nothing.
+   *
+   * Never rejects and is never awaited by the caller: a session that finds
+   * nothing cached simply reads it itself, as before.
+   *
+   * @param {{ sourceKey: string, fileIndex: number, inputUrl: URL, logName: string }} params
+   * @returns {Promise<void>}
+   */
+  async warmKeyframeIndex({ sourceKey, fileIndex, inputUrl, logName }) {
+    try {
+      await this.#readContainerKeyframes({ sourceKey, fileIndex, inputUrl, logName });
+    } catch {
+      // Best effort by construction.
+    }
+  }
+
   async #readContainerKeyframes({ sourceKey, fileIndex, inputUrl, logName }) {
     const cacheKey = `${sourceKey}:${fileIndex}`;
     if (this.keyframeIndexCache.has(cacheKey)) {
@@ -2165,7 +2188,15 @@ export class HlsSessionManager {
     // were verified to handle a copied AC-3 track on this very host, so the
     // arguments that run actually received are the missing evidence. One line
     // per run, and a run happens at most every few seconds.
-    logger.info(`transcode ${session.id} ffmpeg ${describeFfmpegArgs(args)}`);
+    // Numbered, because a burst of seeks starts several runs in one second and
+    // every line about them carries the SESSION id, which is the same for all.
+    // Without a run number the command that failed cannot be told from the two
+    // that succeeded around it — which is exactly the state the unexplained
+    // `Cannot write moov atom before AC3 packets` was found in.
+    session.runCounter = (session.runCounter ?? 0) + 1;
+    const runLabel = `run#${session.runCounter}`;
+    session.runLabel = runLabel;
+    logger.info(`transcode ${session.id} ${runLabel} ffmpeg ${describeFfmpegArgs(args)}`);
 
     const ffmpeg = spawn(this.ffmpegBin, args, {
       cwd: session.dirPath,
@@ -2191,7 +2222,7 @@ export class HlsSessionManager {
     session.budgetSlowSince = 0;
 
     logger.info(
-      `transcode ${session.id} encode-run from segment #${safeIndex} ` +
+      `transcode ${session.id} ${session.runLabel} encode-run from segment #${safeIndex} ` +
         `(${formatSeconds(startSeconds)}) "${session.fileName}"`
     );
 
@@ -2379,7 +2410,9 @@ export class HlsSessionManager {
       session.state = "failed";
       session.progress.state = "failed";
       session.progress.updatedAt = Date.now();
-      logger.error(`transcode ${session.id} encode-run failed: ${session.lastError}`);
+      logger.error(
+        `transcode ${session.id} ${session.runLabel ?? "run#?"} encode-run failed: ${session.lastError}`
+      );
     });
   }
 

package/services/playback-planner.js

@@ -263,7 +263,12 @@ export function createPlaybackPlanner({
   transcodeAudioEnabled,
   localBaseUrl,
   sourceRegistry,
-  torrentPool
+  torrentPool,
+  // Optional. Called once the file's edges are downloaded, so the keyframe
+  // index — which reads the same tail of the file — is fetched alongside the
+  // codec probe instead of after it. Late-bound to the HLS session manager,
+  // which owns the cache both of them share.
+  warmKeyframeIndex
 }) {
   /** @type {Map<string, PlaybackPlan>} */
   const cache = new Map();
@@ -365,6 +370,17 @@ export function createPlaybackPlanner({
       // file, and an unsupported codec like xvid gets copied → black video.
       await torrentPool.prefetchFileEdges(torrent, fileIndex);
       edgesReadyMs = Date.now() - planEntryMs;
+      // The keyframe index reads the tail of the file, which the probe has just
+      // waited for as well. Started here it overlaps the probe instead of
+      // following the whole plan — worth 311-430 ms of the time before the
+      // first segment. Fire and forget: the session reads it itself if this has
+      // not finished, and both share one cache entry.
+      warmKeyframeIndex?.({
+        sourceKey,
+        fileIndex,
+        inputUrl: new URL(directUrl),
+        logName: file.name
+      });
       let probe = await probeStreamCodecs({ ffmpegBin, inputUrl: directUrl, userAgent });
       const probeDeadline = Date.now() + Math.max(0, maxWaitMs);
       let attempt = 0;

package/services/torrent-worker/piece-reader.js

@@ -154,6 +154,41 @@ function clearCritical(torrent, { from, to }) {
   }
 }
 
+/**
+ * Who is working on the piece a reader is blocked on, right now.
+ *
+ * The open question about a seek: a single 8 MiB piece takes 3.0-4.6 s to
+ * arrive while the swarm as a whole is moving 4-6 MB/s, so roughly 2 MB/s is
+ * reaching the piece that is actually being waited for. Whether that is because
+ * few peers hold it, few are being asked, or each is slow cannot be told apart
+ * from the outside — these three counts tell them apart.
+ *
+ * `wire.requests` is what has been asked of that peer and not yet answered; a
+ * block is 16 KB, so `blocks x 16 KB` is the work in flight on this piece.
+ *
+ * @param {import("webtorrent").Torrent} torrent
+ * @param {number} pieceIndex
+ * @returns {{ peers: number, holders: number, askedOf: number, blocks: number }}
+ */
+export function pieceSupply(torrent, pieceIndex) {
+  const wires = Array.isArray(torrent?.wires) ? torrent.wires : [];
+  let holders = 0;
+  let askedOf = 0;
+  let blocks = 0;
+  for (const wire of wires) {
+    if (wire?.peerPieces?.get?.(pieceIndex)) {
+      holders += 1;
+    }
+    const requests = Array.isArray(wire?.requests) ? wire.requests : [];
+    const forThisPiece = requests.filter((request) => request?.piece === pieceIndex).length;
+    if (forThisPiece > 0) {
+      askedOf += 1;
+      blocks += forThisPiece;
+    }
+  }
+  return { peers: wires.length, holders, askedOf, blocks };
+}
+
 /**
  * Wait until a piece has been downloaded and verified.
  *
@@ -421,7 +456,20 @@ export async function* readFragments({
       }
 
       const waitStartedAt = Date.now();
-      await whenPieceReady(torrent, pieceIndex, cancellation);
+      // Sampled while waiting rather than after: once the piece lands, nothing
+      // is outstanding on it any more and every count reads zero.
+      let supply = null;
+      const supplyProbe = setInterval(() => {
+        const sample = pieceSupply(torrent, pieceIndex);
+        if (!supply || sample.blocks > supply.blocks) {
+          supply = sample;
+        }
+      }, 500);
+      try {
+        await whenPieceReady(torrent, pieceIndex, cancellation);
+      } finally {
+        clearInterval(supplyProbe);
+      }
       // What a reader spent waiting for data, attributed to the exact piece. A
       // seek's cost is dominated by the first segment after the encoder
       // restarts (measured 9.2-9.4 s), and without this there is no way to say
@@ -429,10 +477,16 @@ export async function* readFragments({
       // wait is long enough to matter, so ordinary sequential reading is silent.
       const waitedMs = Date.now() - waitStartedAt;
       if (waitedMs >= PIECE_WAIT_LOG_MS) {
+        const rateKbps = Math.round(pieceLength / 1024 / (waitedMs / 1000));
         logger.info(
           `piece-reader: waited ${waitedMs}ms for piece ${pieceIndex} ` +
             `(${pieceIndex - firstPiece + 1} of ${lastPiece - firstPiece + 1} in a read from ` +
-            `${(start / 1024 / 1024).toFixed(0)}MB of "${file.name}")`
+            `${(start / 1024 / 1024).toFixed(0)}MB of "${file.name}") ` +
+            `— ${rateKbps}KB/s on this piece; ` +
+            (supply
+              ? `${supply.holders}/${supply.peers} peers had it, ${supply.askedOf} were asked, ` +
+                `${supply.blocks} blocks (${Math.round((supply.blocks * 16384) / 1024)}KB) in flight at peak`
+              : "no sample taken")
         );
       }