@torrent-tv/proxy 2.9.97 → 2.9.99

package/CHANGELOG.md

@@ -1,3 +1,14 @@
+## 2.9.99
+
+- **New**: A source can be told to start before anyone asks to play it — `POST /api/sources/:sourceKey/warm`. Everything a cold torrent must do first takes seconds and none of it depends on which file is wanted: announce to the trackers, connect to peers, be unchoked by them. Given a file index it also fetches the two pieces at that file's edges, which is what the codec probe reads and what took **6.7 s of the 10.3 s** before playback in the session measured 2026-08-04. All of it used to begin only once a file had been chosen, because it was buried inside the playback plan. The route returns as soon as the work is under way and reports a refusal rather than an error — nothing is broken if a warm-up does not happen, since the ordinary path still does all of it.
+- **Chore**: Two callers asking for the same file's edges at once now share one prefetch instead of opening a second pair of readers, each claiming a window and holding pieces. That happens by design on a single-video torrent, where the warm-up and the playback plan both want them.
+
+## 2.9.98
+
+- **Fix**: The upload is no longer raised at moments when nobody wants a byte. A torrent with no reader was counted as starving whenever its download read low — which it always does while the encoder is held back for running ahead of the viewer. Measured: four cycles of 512 KB/s and back in three minutes, each reported as `earn unchoke … down=0KB/s`. Starvation now requires somebody to be waiting.
+- **Fix**: A session start no longer looks like a burst of seeks. The codec probe and the keyframe index read through the same route as the encoder and visit the first bytes and the last ones, which from byte offsets alone is indistinguishable from a viewer dragging the slider — two spurious "the viewer moved" per start, and more on every encoder restart. The encoder's input URL now says that it is the read that follows the viewer, and only that read counts.
+- **Chore**: The per-run ffmpeg log line abbreviates the list of cut times to its count and its two ends. There is one cut per segment — 830 on a two-hour film, about 7 KB of log per run — and the list is only ever consulted for whether cutting was explicit, where it starts and how far it reaches.
+
 ## 2.9.97
 
 - **Fix**: The generous upload of 2.9.96 did not actually reach the moment it was written for. Only torrents with a registered reader were shown to the upload policy, and the first thing done with a new torrent — fetching the file's head and tail for the codec probe — reads through `createReadStream` without registering one. So for the whole of that wait, 8.36 s of the 11.46 s before playback in the measured session, the torrent looked unused and the upload stayed at the near-silent idle floor, during the exact seconds peers decide whether to serve us. A torrent in a hurry now counts whether or not anything is reading it. The selection is a named function of its own so it can be tested without a live swarm — the fault was in which torrents were considered, not in what was decided about them.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@torrent-tv/proxy",
-  "version": "2.9.97",
+  "version": "2.9.99",
   "description": "Torrent proxy client that exposes webseed-like HTTP stream endpoint.",
   "license": "GPL-3.0-or-later",
   "publishConfig": {

package/routes/api/sources/warm/post.js

@@ -0,0 +1,83 @@
+import { logger } from "../../../../utils/logger.js";
+
+/**
+ * Start fetching a source before anyone asks to play it.
+ *
+ * POST /api/sources/:sourceKey/warm   { fileIndex?: number }
+ *
+ * Everything a torrent must do before the first byte of video can be served
+ * takes seconds and none of it depends on the viewer: announce to the
+ * trackers, connect to peers, be unchoked by them, and fetch the two pieces at
+ * the file's edges that the codec probe reads. Measured 2026-08-04 on a cold
+ * 7.4 GB torrent: 6.7 s of the 10.3 s before playback was those two pieces
+ * arriving, with the swarm ramping from nothing.
+ *
+ * That work used to begin only when a file had been chosen, because it was
+ * buried inside the playback plan. It can begin as soon as the viewer has
+ * picked a TORRENT — while they are still reading the list of episodes — and
+ * then most or all of it has happened by the time they choose.
+ *
+ * Returns as soon as the work is under way. The caller is not waiting for a
+ * result; it is only saying "you may start". Failures are logged and answered
+ * as `started: false` rather than as an error, because nothing is broken if a
+ * warm-up does not happen — the ordinary path still does all of it.
+ *
+ * @param {import("fastify").FastifyRequest} req
+ * @param {import("fastify").FastifyReply} reply
+ * @param {{
+ *   sourceRegistry: ReturnType<import("../../../../store/source-registry.js").createSourceRegistry>,
+ *   torrentPool: import("../../../../services/torrent-pool.js").TorrentPool
+ * }} deps
+ * @returns {Promise<void>}
+ */
+export async function handleApiSourceWarmPost(req, reply, { sourceRegistry, torrentPool }) {
+  const sourceKey = typeof req.params.sourceKey === "string" ? req.params.sourceKey.trim() : "";
+  if (!sourceKey) {
+    return reply.code(400).send({ error: "sourceKey is required." });
+  }
+
+  const sourceRecord = sourceRegistry.get(sourceKey);
+  if (!sourceRecord) {
+    return reply.code(404).send({ error: "Source key was not found." });
+  }
+
+  const body = req.body && typeof req.body === "object" && !Array.isArray(req.body) ? req.body : {};
+  const requestedIndex = Number(body.fileIndex);
+  const fileIndex = Number.isInteger(requestedIndex) && requestedIndex >= 0 ? requestedIndex : null;
+
+  // Adding the torrent is what announces to the trackers and starts connecting
+  // to peers, and it is also what a magnet needs in order to fetch its
+  // metadata. It is awaited because everything else needs the torrent object,
+  // and because until it resolves there is nothing to report.
+  let torrent;
+  try {
+    torrent = await torrentPool.getTorrent(sourceRecord.sourceType, sourceRecord.source);
+  } catch (error) {
+    const message = error instanceof Error ? error.message : String(error);
+    logger.warn(`warm ${sourceKey.slice(0, 8)}: could not add the torrent: ${message}`);
+    return reply.send({ started: false, swarm: false, edges: false });
+  }
+
+  // The edges are only worth fetching once it is known WHICH file will be
+  // played: on a season pack, warming twenty episodes' worth would spend the
+  // pool owner's bandwidth on nineteen files nobody opened. The caller passes
+  // an index when the torrent holds a single video, and again later if it
+  // wants to.
+  let edges = false;
+  if (fileIndex !== null && torrent.files?.[fileIndex]) {
+    edges = true;
+    // Deliberately not awaited: this is the multi-second part, and the point of
+    // the whole route is that the viewer goes on choosing while it happens.
+    Promise.resolve(torrentPool.prefetchFileEdges(torrent, fileIndex)).catch((error) => {
+      const message = error instanceof Error ? error.message : String(error);
+      logger.warn(`warm ${sourceKey.slice(0, 8)}: file edges failed: ${message}`);
+    });
+  }
+
+  logger.info(
+    `warm ${sourceKey.slice(0, 8)}: swarm started for "${torrent.name}"` +
+      (edges ? `, fetching the edges of file ${fileIndex}` : ", file not chosen yet")
+  );
+
+  return reply.send({ started: true, swarm: true, edges });
+}

package/routes/stream/get.js

@@ -144,8 +144,14 @@ export async function handleStreamGet(req, reply, { sourceRegistry, torrentPool
   // byte offset) downloads first instead of waiting behind the sequential
   // backlog — this is what caused ~15-18 s stalls when seeking into an
   // undownloaded region.
+  // Only the encoder's own input read tracks where the viewer is. Everything
+  // else that comes through here — the codec probe, the keyframe index, a
+  // subtitle fetch — reads the file's edges, and treating those as a viewer
+  // position made every session start and every encoder restart look like a
+  // burst of seeks (measured: two spurious "the viewer moved" per start).
   torrentPool.prioritizeByteRange(torrent, fileIndex, range ? range.start : 0, undefined, {
-    wholeFileRead: range === null
+    wholeFileRead: range === null,
+    isPlaybackRead: req.query.reader === "playback"
   });
 
   const start = range ? range.start : 0;

package/server.js

@@ -19,6 +19,7 @@ import { handleHealthzGet } from "./routes/healthz/get.js";
 import { handleApiSourcesPost } from "./routes/api/sources/post.js";
 import { handleApiSourceStatsGet } from "./routes/api/sources/stats/get.js";
 import { handleApiSourceFilesGet } from "./routes/api/sources/files/get.js";
+import { handleApiSourceWarmPost } from "./routes/api/sources/warm/post.js";
 import { handleApiPlaybackPlanPost } from "./routes/api/playback-plan/post.js";
 import { handleApiSubtitlesGet } from "./routes/api/subtitles/get.js";
 import { handleApiTranscodeSessionsPost } from "./routes/api/transcode-sessions/post.js";
@@ -176,6 +177,9 @@ export async function startProxyServer({ host, port, transcodeAudio, ffmpegBin,
   app.get("/api/sources/:sourceKey/files", async (req, reply) =>
     handleApiSourceFilesGet(req, reply, { sourceRegistry, torrentPool })
   );
+  app.post("/api/sources/:sourceKey/warm", async (req, reply) =>
+    handleApiSourceWarmPost(req, reply, { sourceRegistry, torrentPool })
+  );
   app.post("/api/playback-plan", async (req, reply) =>
     handleApiPlaybackPlanPost(req, reply, { playbackPlanner })
   );

package/services/hls-session-manager.js

@@ -672,6 +672,33 @@ function computeSegmentBoundaries({ transcodeVideo, durationSeconds, segDur, key
  * @returns {number[] | null} Times relative to the run start, or null when the
  *   boundaries cannot serve (missing, or the index is outside them).
  */
+/**
+ * The ffmpeg command as one readable line.
+ *
+ * Everything is shown as passed except the list of cut times, which is one
+ * value per segment — 830 of them on a two-hour film, about 7 KB of log for a
+ * single run, repeated on every restart. The count and the two ends say
+ * everything the list is ever consulted for: whether cutting was explicit at
+ * all, how far it reaches, and where it starts.
+ *
+ * @param {string[]} args
+ * @returns {string}
+ */
+export function describeFfmpegArgs(args) {
+  const parts = [];
+  for (let index = 0; index < args.length; index += 1) {
+    const value = args[index];
+    if (value === "-segment_times" && typeof args[index + 1] === "string") {
+      const times = args[index + 1].split(",");
+      parts.push(value, `<${times.length} cuts ${times[0]}..${times[times.length - 1]}>`);
+      index += 1;
+      continue;
+    }
+    parts.push(value);
+  }
+  return parts.join(" ");
+}
+
 export function segmentCutTimesFrom(boundaries, startIndex) {
   if (!Array.isArray(boundaries) || boundaries.length < 2) {
     return null;
@@ -999,6 +1026,12 @@ export class HlsSessionManager {
     const readWindowBytes = await this.#readWindowBytesFor(sourceKey, fileIndex, durationSeconds);
     if (readWindowBytes > 0) {
       inputUrl.searchParams.set("windowBytes", String(readWindowBytes));
+      // This read, and only this read, follows the viewer. The codec probe and
+      // the keyframe index also go through `/stream`, and they jump between the
+      // first bytes and the last ones — which is indistinguishable, from byte
+      // offsets alone, from someone dragging the slider. Saying so here is
+      // cheaper and more truthful than guessing from the offsets.
+      inputUrl.searchParams.set("reader", "playback");
     }
     if (!hasDuration) {
       logger.warn(
@@ -2132,7 +2165,7 @@ export class HlsSessionManager {
     // were verified to handle a copied AC-3 track on this very host, so the
     // arguments that run actually received are the missing evidence. One line
     // per run, and a run happens at most every few seconds.
-    logger.info(`transcode ${session.id} ffmpeg ${args.join(" ")}`);
+    logger.info(`transcode ${session.id} ffmpeg ${describeFfmpegArgs(args)}`);
 
     const ffmpeg = spawn(this.ffmpegBin, args, {
       cwd: session.dirPath,

package/services/torrent-pool.js

@@ -136,7 +136,11 @@ export function torrentsForUploadPolicy(torrents, usageByTorrent, now) {
   const chosen = [];
   for (const torrent of torrents) {
     const usage = usageByTorrent?.get?.(torrent);
-    if ((usage && usage.size > 0) || (torrent?.hurryUntil ?? 0) > now) {
+    const hasReader = Boolean(usage && usage.size > 0);
+    if (hasReader || (torrent?.hurryUntil ?? 0) > now) {
+      // Recorded so the policy can tell "nothing is arriving and somebody is
+      // waiting" from "nothing is arriving because nobody asked".
+      torrent.hasActiveReader = hasReader;
       chosen.push(torrent);
     }
   }
@@ -176,7 +180,13 @@ export function decideUploadLimit(activeTorrents, opts = {}) {
     // iterate the piece array and throw on webtorrent 3.x when a piece is null
     // (deselected / mid-verify), which would crash this timer every cycle.
     const notDone = torrent?.done !== true;
-    const starving = notDone && downloadSpeed < starvingSpeed;
+    // A torrent nobody is reading is not starving, however still its download
+    // looks. The encoder is held back once it is far enough ahead of the
+    // viewer, and while it is held nothing is requested — measured
+    // 2026-08-04: four cycles of 512 -> 50 KB/s in three minutes, each
+    // reported as `earn unchoke ... down=0KB/s`, all of them raising the
+    // upload at moments when no byte was wanted by anyone.
+    const starving = notDone && torrent?.hasActiveReader !== false && downloadSpeed < starvingSpeed;
     if (starving && chokedInterested >= chokedThreshold) {
       const name = typeof torrent?.name === "string" ? torrent.name : "?";
       return {
@@ -373,6 +383,14 @@ export class TorrentPool {
    */
   #readPositionByTorrent = new Map();
 
+  /**
+   * Edge prefetches currently running, keyed by infoHash and file index, so two
+   * callers asking at the same time share one.
+   *
+   * @type {Map<string, Promise<void>>}
+   */
+  #edgePrefetches = new Map();
+
   /** Global disk cap in bytes (0 = disabled). */
   #maxDiskBytes = 0;
 
@@ -1024,6 +1042,37 @@ export class TorrentPool {
     fileIndex,
     { headBytes = 256 * 1024, tailBytes = 2 * 1024 * 1024, timeoutMs = 300_000 } = {}
   ) {
+    if (!torrent || !Array.isArray(torrent.files)) {
+      return;
+    }
+    // Two callers can ask for the same edges at once: the warm-up that starts
+    // when a torrent is picked, and the playback plan a moment later. Reading
+    // the same two pieces twice costs nothing in bandwidth — the torrent
+    // fetches each piece once — but it does open a second pair of readers, each
+    // claiming a window and holding pieces. One is enough.
+    const inFlightKey = `${torrent.infoHash}:${fileIndex}`;
+    const running = this.#edgePrefetches.get(inFlightKey);
+    if (running) {
+      return running;
+    }
+    const prefetch = this.#prefetchFileEdgesOnce(torrent, fileIndex, { headBytes, tailBytes, timeoutMs });
+    this.#edgePrefetches.set(inFlightKey, prefetch);
+    try {
+      return await prefetch;
+    } finally {
+      this.#edgePrefetches.delete(inFlightKey);
+    }
+  }
+
+  /**
+   * The body of {@link prefetchFileEdges}, without the de-duplication.
+   *
+   * @param {import("webtorrent").Torrent} torrent
+   * @param {number} fileIndex
+   * @param {{ headBytes: number, tailBytes: number, timeoutMs: number }} options
+   * @returns {Promise<void>}
+   */
+  async #prefetchFileEdgesOnce(torrent, fileIndex, { headBytes, tailBytes, timeoutMs }) {
     if (!torrent || !Array.isArray(torrent.files)) {
       return;
     }
@@ -1125,9 +1174,11 @@ export class TorrentPool {
    * @param {number} fileIndex
    * @param {number} byteStart - Start offset within the file.
    * @param {number} [windowBytes] - Unused; kept so callers need not change.
-   * @param {{ wholeFileRead?: boolean }} [options] - `wholeFileRead` marks a
-   *   request that carried no byte range, i.e. one that merely opens the file at
-   *   0 rather than asking to read from there. See the guard below.
+   * @param {{ wholeFileRead?: boolean, isPlaybackRead?: boolean }} [options] -
+   *   `wholeFileRead` marks a request that carried no byte range, i.e. one that
+   *   merely opens the file at 0 rather than asking to read from there.
+   *   `isPlaybackRead` marks the encoder's input read, the only one that
+   *   follows the viewer. See the guards below.
    * @returns {void}
    */
   prioritizeByteRange(
@@ -1182,10 +1233,15 @@ export class TorrentPool {
     const isJump =
       previousStart === undefined || Math.abs(safeStart - previousStart) > PRIORITY_WINDOW_BYTES;
     if (isJump) {
-      // A jump is a seek. Whatever the swarm was giving us was for somewhere
-      // else, and the pieces at the new position have to be earned from peers
-      // that are choking us — the same standing start as a fresh torrent.
-      this.#markHurry(torrent, "the viewer moved");
+      // A jump in the read that follows the viewer is a seek. Whatever the
+      // swarm was giving us was for somewhere else, and the pieces at the new
+      // position have to be earned from peers that are choking us — the same
+      // standing start as a fresh torrent. A jump in any OTHER read is the
+      // codec probe or the keyframe index visiting the ends of the file, and
+      // nobody is waiting on those the way a viewer waits on a seek.
+      if (options.isPlaybackRead) {
+        this.#markHurry(torrent, "the viewer moved");
+      }
       const percent = ((safeStart / fileLength) * 100).toFixed(1);
       logger.info(
         `torrent-pool: [${String(torrent.infoHash).slice(0, 8)}] read position -> ` +

package/test/read-window.test.js

@@ -219,7 +219,8 @@ test("criticality marks the window being waited for, not the whole range", async
   try {
     const iterator = readFragments({
       torrent, fileIndex: 0, start: 0, end: 8000 * PIECE - 1,
-      cancellation: { isCancelled: () => false }
+      cancellation: { isCancelled: () => false },
+      windowBytes: WINDOW_PIECES * PIECE
     });
     const pending = iterator.next();
     await new Promise((resolve) => setImmediate(resolve));

package/test/upload-hurry.test.js

@@ -94,3 +94,28 @@ test("a torrent with no reader still reaches the policy while it is in a hurry",
     "a torrent being read still counts, hurry or not"
   );
 });
+
+test("a torrent nobody is reading is not treated as starving", () => {
+  // The encoder is suspended once it is far enough ahead, and while it is
+  // suspended nothing is requested — so the download reads zero without anyone
+  // waiting. Measured before this guard: four cycles of 512 -> 50 KB/s in three
+  // minutes, every one of them reported as `earn unchoke ... down=0KB/s`.
+  const idleButChoked = {
+    name: "film.mkv",
+    hasActiveReader: false,
+    wires: [
+      { amInterested: true, peerChoking: true },
+      { amInterested: true, peerChoking: true }
+    ],
+    downloadSpeed: 0,
+    done: false
+  };
+  assert.equal(decideUploadLimit([idleButChoked], { now: NOW }).bytesPerSec, 50 * 1024);
+
+  const waiting = { ...idleButChoked, hasActiveReader: true };
+  assert.equal(
+    decideUploadLimit([waiting], { now: NOW }).bytesPerSec,
+    512 * 1024,
+    "a reader that IS waiting must still earn unchoke slots"
+  );
+});