@torrent-tv/proxy 2.9.86 → 2.9.88

package/CHANGELOG.md

@@ -1,3 +1,11 @@
+## 2.9.88
+
+- **Fix**: A seek no longer makes the swarm walk the file to get there. Two faults, both confirmed by running WebTorrent's own selection code on the numbers of a measured session (588 pieces, download at 38.4%, seek to 89.1%). First: a selection carries an `offset` — how many pieces from its start are already downloaded — and the picker scans from `from + offset`; `deselect` subtracts an interval and copies that offset into what survives, so demoting the pieces behind the playhead left `{523-587, offset 226}`, a selection whose scan begins at piece 749 of 587. The seek target ended up wanted by nobody. The range is now re-selected right after the demotion, which replaces the dead entry with a fresh one starting at the playhead. Second: a request with no byte range was reported as an ordinary read at offset 0, and ffmpeg opens its input with exactly such a request and abandons it as soon as it seeks — as do the keyframe index and the codec probe, four of them around every encoder restart. Each one re-selected the whole file from piece zero, undoing the seek; the picker then skipped what was on disk and downloaded forward from the first hole. Measured cost of the pair: a seek to 89.1% of a 4.7 GB film fetched **2.47 GB over 93 s** where one 8 MiB piece was needed. A range-less read now sets the read position only when nothing else has.
+
+## 2.9.87
+
+- **Fix**: fMP4 playback no longer stops after the first segment. A segment's position was being written into **every** fragment it contains, and the explicit-cut muxer puts several in one segment — `frag_keyframe` opens a fragment at each keyframe while a cut point comes only every few keyframes. Measured: a 6 s piece carries three fragments per track, at 0, 2 and 4 s of its own clock; all three were stamped with the segment's start, so they claimed the same decode time and the player rejected the segment. In the field (2.9.86, this session) that showed as segments 1 and 2 requested in an endless alternation, each served in tens of milliseconds with the transcode healthy at 12x, while the picture froze a few seconds in. The position is now applied as a shift: each track's first fragment sets the base and the rest keep their distance from it. With one fragment per track — what the `hls` muxer produces — a shift and a write are the same thing, so the other path is unchanged. Verified end to end on the addon host: four pieces cut, split, stamped and reassembled the way a player does, then probed — 600 frames over 24 s, decode timestamps rising by exactly 0.04 s across every segment join, no duplicates, clean decode.
+
 ## 2.9.86
 
 - **Fix**: fMP4 playback starts again. The real reason ffmpeg exited before writing anything was the audio, not the file names: the MP4 muxer derives a copied AC-3 track's `dac3` box from the bitstream, so it cannot write `moov` until the first audio packet arrives, while our `empty_moov` demands it at header time — `Cannot write moov atom before AC3 packets. Set the delay_moov flag to fix this.`, captured in the field on a copied AC-3 source. `delay_moov` is now passed alongside it. The `hls` muxer sets that flag itself, which is why the fault appeared only once the muxing moved to the `segment` muxer in 2.9.84; MPEG-TS has no `moov` and was never affected. Verified in the addon container on an AC-3 source: without the flag the exact command the proxy runs fails, with it the segments are written, and the piece layout is unchanged (`ftyp moov moof mdat … mfra`), so the init split added in 2.9.84 still cuts in the same places — headers of consecutive pieces differ in four bytes, all inside `elst`, which the `tfdt` rewriting already overrides.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@torrent-tv/proxy",
-  "version": "2.9.86",
+  "version": "2.9.88",
   "description": "Torrent proxy client that exposes webseed-like HTTP stream endpoint.",
   "license": "GPL-3.0-or-later",
   "publishConfig": {

package/routes/stream/get.js

@@ -94,7 +94,9 @@ export async function handleStreamGet(req, reply, { sourceRegistry, torrentPool
   // byte offset) downloads first instead of waiting behind the sequential
   // backlog — this is what caused ~15-18 s stalls when seeking into an
   // undownloaded region.
-  torrentPool.prioritizeByteRange(torrent, fileIndex, range ? range.start : 0);
+  torrentPool.prioritizeByteRange(torrent, fileIndex, range ? range.start : 0, undefined, {
+    wholeFileRead: range === null
+  });
 
   const start = range ? range.start : 0;
   const end = range ? range.end : file.length - 1;

package/services/segment-formats/mp4-boxes.js

@@ -113,6 +113,19 @@ export function readTrackTimescales(initSegment) {
  * independently-addressable segment anyway: it carries its own position, so it
  * is valid against any init for the same tracks.
  *
+ * A SEGMENT MAY HOLD SEVERAL FRAGMENTS PER TRACK, so the segment's start is
+ * applied as a SHIFT, not as a value written into every `tfdt`. The muxer that
+ * takes explicit cut times uses `frag_keyframe`, which opens a fragment at each
+ * keyframe, while a cut point is only every few keyframes — measured on the
+ * field host: a 6 s piece carried three fragments per track, at 0, 2 and 4 s of
+ * its own clock. Writing the segment's start into all three made them claim the
+ * same decode time; the player rejected the segment and re-fetched it forever
+ * (field 2026-08-04: segments 1 and 2 alternating for minutes, each served in
+ * tens of milliseconds, transcode healthy at 12x). Each track's first fragment
+ * therefore defines the base and the rest keep their distance from it. With one
+ * fragment per track — what the `hls` muxer produces — a shift and a write are
+ * the same thing, so both paths are served by this.
+ *
  * Mutates a copy; the caller's buffer is untouched.
  *
  * @param {Buffer} segment
@@ -128,6 +141,8 @@ export function stampSegmentStartTime(segment, startSeconds, trackTimescales) {
   // `tfhd` carries the track id and always precedes the `tfdt` inside the same
   // `traf`, so an ordered pass pairs each `tfdt` with its track's timescale.
   let currentTrackId = null;
+  /** @type {Map<number, number>} trackId → decode time of that track's first fragment. */
+  const fragmentBase = new Map();
   walkBoxes(stamped, (type, bodyStart, bodyEnd) => {
     if (type === "tfhd") {
       if (bodyStart + 8 <= bodyEnd) {
@@ -143,17 +158,27 @@ export function stampSegmentStartTime(segment, startSeconds, trackTimescales) {
       return;
     }
     const version = stamped[bodyStart];
-    const value = Math.round(startSeconds * timescale);
+    if (version === 1 ? bodyStart + 12 > bodyEnd : bodyStart + 8 > bodyEnd) {
+      return;
+    }
+    const existing =
+      version === 1
+        ? Number(stamped.readBigUInt64BE(bodyStart + 4))
+        : stamped.readUInt32BE(bodyStart + 4);
+    if (!fragmentBase.has(currentTrackId)) {
+      fragmentBase.set(currentTrackId, existing);
+    }
+    // Distance from the track's first fragment in this segment. Never negative:
+    // decode times only move forward, and a malformed one must not drag a later
+    // fragment behind the segment's start.
+    const withinSegment = Math.max(0, existing - (fragmentBase.get(currentTrackId) ?? 0));
+    const value = Math.round(startSeconds * timescale) + withinSegment;
     if (version === 1) {
-      if (bodyStart + 12 <= bodyEnd) {
-        stamped.writeBigUInt64BE(BigInt(value), bodyStart + 4);
-      }
-    } else if (bodyStart + 8 <= bodyEnd) {
+      stamped.writeBigUInt64BE(BigInt(value), bodyStart + 4);
+    } else if (value <= 0xffffffff) {
       // A 32-bit field cannot express beyond ~2^32 ticks; leave it rather than
       // write a wrapped value (the player would land somewhere arbitrary).
-      if (value <= 0xffffffff) {
-        stamped.writeUInt32BE(value, bodyStart + 4);
-      }
+      stamped.writeUInt32BE(value, bodyStart + 4);
     }
   });
   return stamped;

package/services/torrent-pool.js

@@ -1048,9 +1048,18 @@ export class TorrentPool {
    * @param {number} fileIndex
    * @param {number} byteStart - Start offset within the file.
    * @param {number} [windowBytes] - Bytes ahead of `byteStart` to mark critical.
+   * @param {{ wholeFileRead?: boolean }} [options] - `wholeFileRead` marks a
+   *   request that carried no byte range, i.e. one that merely opens the file at
+   *   0 rather than asking to read from there. See the guard below.
    * @returns {void}
    */
-  prioritizeByteRange(torrent, fileIndex, byteStart, windowBytes = PRIORITY_WINDOW_BYTES) {
+  prioritizeByteRange(
+    torrent,
+    fileIndex,
+    byteStart,
+    windowBytes = PRIORITY_WINDOW_BYTES,
+    options = {}
+  ) {
     if (!torrent || typeof torrent.critical !== "function" || !Array.isArray(torrent.files)) {
       return;
     }
@@ -1080,6 +1089,20 @@ export class TorrentPool {
       this.#readPositionByTorrent.set(torrent, readPositions);
     }
     const previousStart = readPositions.get(fileIndex);
+
+    // A request with no byte range says nothing about where the viewer is. ffmpeg
+    // opens its input with a plain GET and abandons it the moment it seeks, and
+    // the keyframe index and the codec probe do the same — four such reads around
+    // every encoder restart, each one arriving here as "position 0". Acting on
+    // them undoes the seek that just happened: the whole file is re-selected from
+    // piece 0, the picker skips the pieces already on disk and walks the swarm
+    // forward from the first hole. Measured on a 4.7 GB film: a seek to 89.1%
+    // downloaded 2.47 GB over 93 s before the segment could be served. So a
+    // whole-file read only sets the position when nothing else has.
+    if (options.wholeFileRead && previousStart !== undefined) {
+      return;
+    }
+
     readPositions.set(fileIndex, safeStart);
 
     // Log jumps only. Sequential reading calls this on every range request and
@@ -1133,6 +1156,16 @@ export class TorrentPool {
     if (playheadPiece > fileStartPiece && typeof torrent.deselect === "function") {
       try {
         torrent.deselect(fileStartPiece, playheadPiece - 1);
+        // `deselect` subtracts the interval and copies the selection's `offset`
+        // — how many pieces from its start are already downloaded — into what
+        // remains. The picker scans from `from + offset`, so the surviving
+        // selection starts scanning far past its own end and can never yield a
+        // piece: measured with the library's own `Selections`, deselecting
+        // 0-522 from `{0-587, offset 226}` leaves `{523-587, offset 226}`,
+        // i.e. a scan starting at piece 749 of 587. The seek target ends up
+        // wanted by nobody. Re-selecting the same range replaces that dead
+        // entry with a fresh one whose offset is 0.
+        torrent.select(playheadPiece, fileEndPiece, 1);
         this.#selectedFromPiece.get(torrent)?.set(fileIndex, playheadPiece);
       } catch {
         // Best effort — never break streaming because demotion failed.

package/services/torrent-worker/client.js

@@ -198,11 +198,17 @@ export class TorrentWorkerClient {
   /**
    * Reorder piece selection around a read position (seek prioritisation).
    *
-   * @param {{ sourceKey: string, fileIndex: number, byteStart: number, windowBytes?: number }} params
+   * @param {{ sourceKey: string, fileIndex: number, byteStart: number, windowBytes?: number, wholeFileRead?: boolean }} params
    * @returns {Promise<void>}
    */
-  async prioritizeByteRange({ sourceKey, fileIndex, byteStart, windowBytes }) {
-    await this.#caller.call(Command.PRIORITIZE, { sourceKey, fileIndex, byteStart, windowBytes });
+  async prioritizeByteRange({ sourceKey, fileIndex, byteStart, windowBytes, wholeFileRead }) {
+    await this.#caller.call(Command.PRIORITIZE, {
+      sourceKey,
+      fileIndex,
+      byteStart,
+      windowBytes,
+      wholeFileRead
+    });
   }
 
   /**

package/services/torrent-worker/pool-adapter.js

@@ -141,15 +141,22 @@ export class WorkerTorrentPool {
    * @param {number} fileIndex
    * @param {number} byteStart
    * @param {number} [windowBytes]
+   * @param {{ wholeFileRead?: boolean }} [options]
    * @returns {void}
    */
-  prioritizeByteRange(torrent, fileIndex, byteStart, windowBytes) {
+  prioritizeByteRange(torrent, fileIndex, byteStart, windowBytes, options) {
     const sourceKey = torrent?.sourceKey;
     if (!sourceKey) {
       return;
     }
     void this.#client
-      .prioritizeByteRange({ sourceKey, fileIndex, byteStart, windowBytes })
+      .prioritizeByteRange({
+        sourceKey,
+        fileIndex,
+        byteStart,
+        windowBytes,
+        wholeFileRead: options?.wholeFileRead === true
+      })
       .catch(() => undefined);
   }
 

package/services/torrent-worker/worker.js

@@ -307,7 +307,9 @@ async function runCommand(command, params, id) {
 
     case Command.PRIORITIZE: {
       const torrent = await requireTorrent(params.sourceKey);
-      pool.prioritizeByteRange(torrent, params.fileIndex, params.byteStart, params.windowBytes);
+      pool.prioritizeByteRange(torrent, params.fileIndex, params.byteStart, params.windowBytes, {
+        wholeFileRead: params.wholeFileRead === true
+      });
       return true;
     }
 

package/test/fmp4-timestamps.test.js

@@ -0,0 +1,140 @@
+/**
+ * Stamping a segment's position onto its fragments.
+ *
+ * A segment produced by the explicit-cut muxer holds SEVERAL fragments per
+ * track — `frag_keyframe` opens one at every keyframe, while a cut point comes
+ * only every few keyframes. Writing the segment's start into each of them made
+ * them all claim the same decode time, and the player re-fetched the segment
+ * forever. The positions inside a segment must therefore survive.
+ */
+
+import test from "node:test";
+import assert from "node:assert/strict";
+import { stampSegmentStartTime } from "../services/segment-formats/mp4-boxes.js";
+
+/**
+ * @param {string} type
+ * @param {Buffer} body
+ * @returns {Buffer}
+ */
+function box(type, body) {
+  const header = Buffer.alloc(8);
+  header.writeUInt32BE(8 + body.length, 0);
+  header.write(type, 4, "latin1");
+  return Buffer.concat([header, body]);
+}
+
+/**
+ * One fragment: `moof > traf > (tfhd, tfdt)`, plus the payload it describes.
+ *
+ * @param {{ trackId: number, decodeTime: number, version?: 0 | 1 }} fragment
+ * @returns {Buffer}
+ */
+function makeFragment({ trackId, decodeTime, version = 1 }) {
+  const tfhdBody = Buffer.alloc(8);
+  tfhdBody.writeUInt32BE(trackId, 4);
+  const tfdtBody = Buffer.alloc(version === 1 ? 12 : 8);
+  tfdtBody[0] = version;
+  if (version === 1) {
+    tfdtBody.writeBigUInt64BE(BigInt(decodeTime), 4);
+  } else {
+    tfdtBody.writeUInt32BE(decodeTime, 4);
+  }
+  const traf = box("traf", Buffer.concat([box("tfhd", tfhdBody), box("tfdt", tfdtBody)]));
+  return Buffer.concat([box("moof", traf), box("mdat", Buffer.alloc(16, 0x5a))]);
+}
+
+/**
+ * Every `tfdt` in order, as `[trackId, decodeTime]`.
+ *
+ * @param {Buffer} segment
+ * @returns {Array<[number, number]>}
+ */
+function readDecodeTimes(segment) {
+  const found = [];
+  let offset = 0;
+  let trackId = 0;
+  while (offset + 8 <= segment.length) {
+    const size = segment.readUInt32BE(offset);
+    const type = segment.toString("latin1", offset + 4, offset + 8);
+    if (type === "tfhd") {
+      trackId = segment.readUInt32BE(offset + 12);
+    } else if (type === "tfdt") {
+      const version = segment[offset + 8];
+      found.push([
+        trackId,
+        version === 1 ? Number(segment.readBigUInt64BE(offset + 12)) : segment.readUInt32BE(offset + 12)
+      ]);
+    }
+    // Descend into the containers on the way to `tfdt`; skip anything else
+    // whole, so `mdat` is never walked into.
+    offset += type === "moof" || type === "traf" ? 8 : size;
+  }
+  return found;
+}
+
+test("fragments keep their distance from the start of the segment", () => {
+  // A 6 s segment holding three fragments at 0, 2 and 4 s of its own clock —
+  // the shape measured on the field host.
+  const timescale = 16_000;
+  const segment = Buffer.concat([
+    makeFragment({ trackId: 1, decodeTime: 0 }),
+    makeFragment({ trackId: 1, decodeTime: 2 * timescale }),
+    makeFragment({ trackId: 1, decodeTime: 4 * timescale })
+  ]);
+
+  const stamped = stampSegmentStartTime(segment, 6, new Map([[1, timescale]]));
+
+  assert.deepEqual(
+    readDecodeTimes(stamped).map(([, time]) => time / timescale),
+    [6, 8, 10],
+    "each fragment must land at the segment's start plus its own offset"
+  );
+});
+
+test("each track is shifted by its own base", () => {
+  const videoScale = 16_000;
+  const audioScale = 44_100;
+  // Audio does not start at zero: its frames do not align with the video's.
+  const segment = Buffer.concat([
+    makeFragment({ trackId: 1, decodeTime: 0 }),
+    makeFragment({ trackId: 2, decodeTime: 1024 }),
+    makeFragment({ trackId: 1, decodeTime: 2 * videoScale }),
+    makeFragment({ trackId: 2, decodeTime: 1024 + 2 * audioScale })
+  ]);
+
+  const stamped = stampSegmentStartTime(
+    segment,
+    6,
+    new Map([
+      [1, videoScale],
+      [2, audioScale]
+    ])
+  );
+
+  assert.deepEqual(readDecodeTimes(stamped), [
+    [1, 6 * videoScale],
+    [2, 6 * audioScale],
+    [1, 8 * videoScale],
+    [2, 8 * audioScale]
+  ]);
+});
+
+test("a single fragment is written outright, as before", () => {
+  const stamped = stampSegmentStartTime(
+    makeFragment({ trackId: 1, decodeTime: 1280 }),
+    12,
+    new Map([[1, 16_000]])
+  );
+  assert.deepEqual(readDecodeTimes(stamped), [[1, 12 * 16_000]]);
+});
+
+test("a 32-bit field too small for the position is left alone", () => {
+  const segment = makeFragment({ trackId: 1, decodeTime: 0, version: 0 });
+  const stamped = stampSegmentStartTime(segment, 1_000_000, new Map([[1, 90_000]]));
+  assert.deepEqual(
+    readDecodeTimes(stamped),
+    [[1, 0]],
+    "a wrapped value would send the player somewhere arbitrary; leaving it is the lesser harm"
+  );
+});

package/test/piece-selection-offset.test.js

@@ -0,0 +1,85 @@
+/**
+ * @file What WebTorrent's own selection bookkeeping does when a seek demotes
+ * the pieces behind the playhead.
+ *
+ * This is the library's behaviour, not ours, and `prioritizeByteRange` depends
+ * on it: a selection carries an `offset` — how many pieces from its start are
+ * already downloaded — and the picker scans from `from + offset`
+ * (`torrent.js`, `for (piece = next.from + next.offset; piece <= next.to; …)`).
+ * `deselect` subtracts an interval and copies that offset into what survives,
+ * so the remaining selection can end up scanning past its own end and yield
+ * nothing at all.
+ *
+ * Measured consequence before the fix: a seek to 89.1% of a 4.7 GB film left
+ * the seek target wanted by nobody, a later range-less read re-selected the
+ * whole file, and the swarm walked it from the first missing piece — 2.47 GB
+ * over 93 s before the segment could be served.
+ *
+ * If a WebTorrent upgrade changes any of this, these tests fail rather than the
+ * behaviour silently regressing.
+ */
+
+import test from "node:test";
+import assert from "node:assert/strict";
+import { Selections } from "webtorrent/lib/selections.js";
+
+// The numbers are the measured session: 588 pieces of 8 MiB, sequential
+// download had reached 38.4% (piece 226), the viewer seeked to 89.1% (piece
+// 523).
+const LAST_PIECE = 587;
+const DOWNLOADED_TO = 226;
+const PLAYHEAD = 523;
+
+/** Where the picker would start scanning this selection. */
+const scanStart = (selection) => selection.from + selection.offset;
+
+test("deselecting the gap behind the playhead leaves a selection that yields nothing", () => {
+  const selections = new Selections();
+  selections.insert({ from: 0, to: LAST_PIECE, offset: 0, priority: 1 });
+  // What `_gcSelections` does as pieces arrive.
+  selections.get(0).offset = DOWNLOADED_TO;
+
+  selections.remove({ from: 0, to: PLAYHEAD - 1, isStreamSelection: false });
+
+  assert.equal(selections.length, 1);
+  const survivor = selections.get(0);
+  assert.equal(survivor.from, PLAYHEAD, "the surviving selection starts at the playhead");
+  assert.equal(survivor.offset, DOWNLOADED_TO, "and it kept the offset of the range it came from");
+  assert.ok(
+    scanStart(survivor) > survivor.to,
+    `scan would start at piece ${scanStart(survivor)} of ${survivor.to} — nothing is downloadable`
+  );
+});
+
+test("re-selecting the same range restores a scan that starts at the playhead", () => {
+  const selections = new Selections();
+  selections.insert({ from: 0, to: LAST_PIECE, offset: 0, priority: 1 });
+  selections.get(0).offset = DOWNLOADED_TO;
+
+  // Exactly what prioritizeByteRange does on a forward seek.
+  selections.remove({ from: 0, to: PLAYHEAD - 1, isStreamSelection: false });
+  selections.insert({ from: PLAYHEAD, to: LAST_PIECE, offset: 0, priority: 1 });
+
+  assert.equal(selections.length, 1, "the dead selection was replaced, not added to");
+  const selection = selections.get(0);
+  assert.equal(scanStart(selection), PLAYHEAD, "the picker now starts at the seek target");
+  assert.equal(selection.to, LAST_PIECE);
+});
+
+test("selecting the whole file again undoes the demotion", () => {
+  const selections = new Selections();
+  selections.insert({ from: 0, to: LAST_PIECE, offset: 0, priority: 1 });
+  selections.get(0).offset = DOWNLOADED_TO;
+  selections.remove({ from: 0, to: PLAYHEAD - 1, isStreamSelection: false });
+  selections.insert({ from: PLAYHEAD, to: LAST_PIECE, offset: 0, priority: 1 });
+
+  // A range-less read reporting position 0 used to land here.
+  selections.insert({ from: 0, to: LAST_PIECE, offset: 0, priority: 1 });
+
+  assert.equal(selections.length, 1);
+  assert.equal(
+    scanStart(selections.get(0)),
+    0,
+    "the whole file is selected again, so the picker falls back to the first missing piece"
+  );
+});

package/test/stream-route.test.js

@@ -20,7 +20,7 @@ import { handleStreamGet } from "../routes/stream/get.js";
  */
 function harness({ method, range }) {
   const opened = [];
-  const state = { claims: 0 };
+  const state = { claims: 0, prioritized: [] };
 
   const sent = { code: 200, headers: {}, body: undefined, called: false };
   const reply = {
@@ -72,7 +72,9 @@ function harness({ method, range }) {
       state.claims += 1;
       return () => undefined;
     },
-    prioritizeByteRange() {}
+    prioritizeByteRange(_torrent, fileIndex, byteStart, _windowBytes, options) {
+      state.prioritized.push({ byteStart, wholeFileRead: options?.wholeFileRead === true });
+    }
   };
 
   const req = {
@@ -119,3 +121,25 @@ test("GET with a range streams only that range", async () => {
   assert.equal(sent.headers["content-range"], "bytes 100-199/5869669065");
   assert.equal(sent.headers["content-length"], "100");
 });
+
+// A request with no byte range says nothing about where the viewer is: ffmpeg
+// opens its input with a plain GET and abandons it the moment it seeks, and the
+// keyframe index and the codec probe do the same — four such reads around every
+// encoder restart. Reported as ordinary reads at offset 0, they undid the seek
+// that had just happened and sent the swarm walking the file from its first
+// missing piece; a seek to 89.1% of a 4.7 GB film downloaded 2.47 GB that way.
+test("a range-less GET is reported as a whole-file read", async () => {
+  const { req, reply, state, deps } = harness({ method: "GET" });
+
+  await handleStreamGet(req, reply, deps);
+
+  assert.deepEqual(state.prioritized, [{ byteStart: 0, wholeFileRead: true }]);
+});
+
+test("a ranged GET is reported as a real read position", async () => {
+  const { req, reply, state, deps } = harness({ method: "GET", range: "bytes=4390000000-" });
+
+  await handleStreamGet(req, reply, deps);
+
+  assert.deepEqual(state.prioritized, [{ byteStart: 4_390_000_000, wholeFileRead: false }]);
+});