@torrent-tv/proxy 2.38.0 → 2.38.1

package/CHANGELOG.md

@@ -1,3 +1,7 @@
+## 2.38.1
+
+- **New**: When a blocked piece cannot be steered anywhere, the wait line says what is holding it. The steering added in 2.29.0 often places nothing — `steered onto 0 of 9 asks (8 peers held it)`, measured 2026-08-18 while eight peers had the piece — because every block is already reserved and WebTorrent will not hand out a second request for the same block (`Piece.reserve()` answers -1; the only mention of an endgame in the library is a commented-out line). Duplicating those blocks is the standard remedy and costs a block's traffic each time, so this measures the tail before anything is built on it: `tail 2/512 blocks missing, held by 1@12KB/s 1@900KB/s`, slowest wire first, and `held by nobody` when the piece has not been asked for at all. Sampled at the instant an attempt placed nothing rather than once at the start, so the numbers and the reason they are printed describe the same moment. If the missing blocks turn out to sit on one slow wire, duplication is aimed at the right thing; if they are spread across fast ones, the wait has another cause and that work should not be done.
+
 ## 2.38.0
 
 - **Fix**: An MP4's keyframe times are read as composition times, on the track the handler names. Two faults, both measured on real releases over the swarm (`research/mp4-composition-times-2026-08-19.md`). (1) The reader took sample times from `stts`, which is DECODE order, and used neither `ctts` nor `elst`: ISO/IEC 14496-12 says `CT(n) = DT(n) + CTTS(n)` (§8.6.1.3) and the edit list then shifts that (§8.6.6.3). Every LostFilm MP4 measured carries a composition offset AND an edit list cancelling it exactly, which is why decode times had been right on them; `Firefly.S01E03.720p.mp4` carries the same 2002-tick offset with NO edit list, and its times were **62.1 ms early on all 34 keyframes** compared against ffmpeg's own `pts_time` — a constant that closes to four decimals as offset (0.08342 s) minus the container start (0.02133 s). After the fix that file matches ffmpeg to the container start, which `computeSegmentBoundaries` already subtracts, and `Superman.720p` — where the terms cancel — is unchanged and exact to 0.0000 s. Version 1 offsets are read as SIGNED, which is what that version exists for; an empty edit (`media_time = -1`) is skipped rather than treated as a shift. (2) The video track was "the first one carrying sync samples", and the handler was never read. That worked only because all seven measured files put video first; the standard identifies a track by `hdlr`, and a file whose audio track carries sync samples, or one leading with a cover-art video track, would have been read from the wrong place — the same defect fixed in the Matroska reader the day before, arrived at from the other side.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@torrent-tv/proxy",
-  "version": "2.38.0",
+  "version": "2.38.1",
   "description": "Torrent proxy client that exposes webseed-like HTTP stream endpoint.",
   "license": "GPL-3.0-or-later",
   "publishConfig": {

package/services/torrent-worker/fastest-wires.js

@@ -126,3 +126,74 @@ export function askFastestWiresFor(torrent, pieceIndex, limit = 3) {
     fastestBytesPerSecond: candidates.length > 0 ? speedOf(candidates[0]) : 0
   };
 }
+
+/**
+ * What is actually holding up a piece the reader is blocked on.
+ *
+ * The steering above can only move a block that the library will hand over, and
+ * the field says it often hands over nothing: `steered onto 0 of 9 asks (8
+ * peers held it)`, recorded 2026-08-18 while eight peers had the piece. When
+ * every block of a piece is already reserved, `Piece.reserve()` answers -1 and
+ * there is no request left to place — the read then ends when the SLOWEST
+ * holder delivers its block, however fast the rest of the swarm is.
+ *
+ * Duplicating those last blocks onto faster wires is the standard remedy, and
+ * it is not free: every duplicate is a block's worth of traffic paid twice. So
+ * this describes the tail before anything is built with it — how many blocks
+ * are still missing, and on which wires they sit, with each wire's speed. If
+ * the missing blocks turn out to sit on one slow wire, duplication is aimed at
+ * exactly the right thing; if they are spread across fast ones, the wait has
+ * another cause and this work should not be done at all.
+ *
+ * Reads only: nothing here changes a reservation or places a request.
+ *
+ * @param {import("webtorrent").Torrent} torrent
+ * @param {number} pieceIndex
+ * @returns {{ chunks: number, missing: number,
+ *   outstanding: Array<{ blocks: number, bytesPerSecond: number, choking: boolean }> } | null}
+ *   Null only when the piece object is gone — it completed and was cleared.
+ *   A piece nobody has reserved a block of yet is NOT null: `torrent-piece`
+ *   creates its buffer lazily on the first reserve, so a piece the picker has
+ *   not reached reads as every block missing and nothing outstanding, which is
+ *   the most informative answer this can give — the wait is not on a slow
+ *   holder, it is on nobody having been asked.
+ */
+export function describePieceTail(torrent, pieceIndex) {
+  const piece = torrent?.pieces?.[pieceIndex];
+  if (!piece) {
+    return null;
+  }
+  const buffer = Array.isArray(piece._buffer) ? piece._buffer : null;
+  const chunks = Number.isFinite(piece._chunks)
+    ? piece._chunks
+    : (buffer ? buffer.length : 0);
+  // No buffer means no block of this piece has been reserved yet, so all of it
+  // is missing. Reading that as "no tail" hid the case worth seeing most.
+  let missing = chunks;
+  if (buffer) {
+    missing = 0;
+    for (let index = 0; index < chunks; index += 1) {
+      if (!buffer[index]) {
+        missing += 1;
+      }
+    }
+  }
+
+  const wires = Array.isArray(torrent?.wires) ? torrent.wires : [];
+  const outstanding = [];
+  for (const wire of wires) {
+    const requests = Array.isArray(wire?.requests) ? wire.requests : [];
+    const blocks = requests.filter((request) => request?.piece === pieceIndex).length;
+    if (blocks > 0) {
+      outstanding.push({
+        blocks,
+        bytesPerSecond: speedOf(wire),
+        choking: wire?.peerChoking === true
+      });
+    }
+  }
+  // Slowest first: that is the wire the read is waiting on, and the one a
+  // duplicate would be aimed past.
+  outstanding.sort((left, right) => left.bytesPerSecond - right.bytesPerSecond);
+  return { chunks, missing, outstanding };
+}

package/services/torrent-worker/piece-reader.js

@@ -22,7 +22,7 @@
 
 import { findSharedStore } from "../piece-store/shared-piece-store.js";
 import { logger } from "../../utils/logger.js";
-import { askFastestWiresFor, canPlaceRequests } from "./fastest-wires.js";
+import { askFastestWiresFor, canPlaceRequests, describePieceTail } from "./fastest-wires.js";
 import { minimumBufferFrom, requiredSpeedFrom } from "../supply-margin.js";
 
 /** Only waits at least this long are reported; sequential reading stays silent. */
@@ -566,9 +566,18 @@ export async function* readFragments({
       // of bandwidth and the reader still waited 1.0-4.5 s, 47 times in two
       // minutes, on pieces five peers already had.
       let pushed = { asked: 0, attempted: 0, considered: 0, fastestBytesPerSecond: 0 };
+      // The tail as it stood at an attempt that placed NOTHING — the state the
+      // duplication work has to answer, and the only one worth a line. Sampled
+      // at that instant rather than once up front, because the steering runs
+      // again every half second and the piece changes under it; the last such
+      // reading is kept, so the line describes the most recent failure.
+      let tailWhenNothingPlaced = null;
       const pushToFastest = () => {
         try {
           const result = askFastestWiresFor(torrent, pieceIndex);
+          if (result.asked === 0) {
+            tailWhenNothingPlaced = describePieceTail(torrent, pieceIndex);
+          }
           pushed = {
             asked: pushed.asked + result.asked,
             // Summed like the successes, so the line compares two totals over
@@ -586,6 +595,9 @@ export async function* readFragments({
       if (canPlaceRequests(torrent)) {
         pushToFastest();
       } else {
+        // Nothing can be placed at all on this build, so the tail is the whole
+        // of the answer.
+        tailWhenNothingPlaced = describePieceTail(torrent, pieceIndex);
         logger.warn(
           "piece-reader: this webtorrent build offers no way to place a request; " +
             "the blocked piece cannot be steered onto a faster peer"
@@ -657,6 +669,19 @@ export async function* readFragments({
             `; steered onto ${pushed.asked} of ${pushed.attempted} asks (${pushed.considered} peers held it)` +
             (pushed.fastestBytesPerSecond > 0
               ? `, fastest ${Math.round(pushed.fastestBytesPerSecond / 1024)}KB/s`
+              : "") +
+            // Only when the steering placed nothing, which is the case that
+            // decides whether duplicating the tail is worth building: it says
+            // how much of the piece is still missing and which wires are
+            // holding it, slowest first.
+            (tailWhenNothingPlaced
+              ? `; tail ${tailWhenNothingPlaced.missing}/${tailWhenNothingPlaced.chunks} blocks missing, held by ` +
+                (tailWhenNothingPlaced.outstanding.length > 0
+                  ? tailWhenNothingPlaced.outstanding
+                    .map((wire) => `${wire.blocks}@${Math.round(wire.bytesPerSecond / 1024)}KB/s` +
+                      (wire.choking ? " (choking)" : ""))
+                    .join(" ")
+                  : "nobody")
               : "")
         );
       }

package/test/piece-tail.test.js

@@ -0,0 +1,97 @@
+/**
+ * @file What a blocked piece's tail looks like, before anything is built on it.
+ *
+ * The steering shipped in 2.29.0 often places nothing — `steered onto 0 of 9
+ * asks (8 peers held it)`, measured 2026-08-18 — because every block of the
+ * piece is already reserved and the library will not hand out a second request
+ * for the same block. Duplicating those blocks is the standard remedy and costs
+ * traffic, so the tail is described first: how much is missing, and on which
+ * wires it sits.
+ */
+
+import test from "node:test";
+import assert from "node:assert/strict";
+import { describePieceTail } from "../services/torrent-worker/fastest-wires.js";
+
+/** A torrent whose piece has some blocks in hand and some in flight. */
+function torrentWith({ buffer, wires }) {
+  return {
+    pieces: [{ _buffer: buffer, _chunks: buffer.length }],
+    wires
+  };
+}
+
+function wire({ has = true, blocks = 0, speed = 0, choking = false }) {
+  return {
+    peerPieces: { get: () => has },
+    requests: Array.from({ length: blocks }, () => ({ piece: 0 })),
+    downloadSpeed: () => speed,
+    peerChoking: choking
+  };
+}
+
+test("the tail names how much is missing and who is holding it", () => {
+  const tail = describePieceTail(
+    torrentWith({
+      // Four blocks, two already in hand.
+      buffer: [new Uint8Array(1), new Uint8Array(1), null, null],
+      wires: [
+        wire({ blocks: 1, speed: 900_000 }),
+        wire({ blocks: 1, speed: 12_000 }),
+        wire({ has: true, blocks: 0, speed: 5_000_000 })
+      ]
+    }),
+    0
+  );
+
+  assert.equal(tail.missing, 2);
+  assert.equal(tail.chunks, 4);
+  assert.equal(tail.outstanding.length, 2, "only the wires actually holding a block of it");
+  assert.equal(
+    tail.outstanding[0].bytesPerSecond,
+    12_000,
+    "slowest first — that is the wire the read is waiting on"
+  );
+});
+
+test("a piece nobody is fetching reads as entirely missing, held by nobody", () => {
+  const tail = describePieceTail(
+    torrentWith({ buffer: [null, null], wires: [wire({ has: false, blocks: 0 })] }),
+    0
+  );
+
+  assert.equal(tail.outstanding.length, 0);
+  assert.equal(tail.missing, 2, "and the missing count still says the piece is untouched");
+});
+
+test("a piece no block of which has been reserved is described, not skipped", () => {
+  // `torrent-piece` builds its buffer lazily on the first reserve, so a piece
+  // the picker has not reached has none. Reading that as "no tail" hid the
+  // clearest answer there is: the wait is on nobody having been asked.
+  const tail = describePieceTail(
+    { pieces: [{ _buffer: null, _chunks: 512 }], wires: [wire({ has: true, blocks: 0, speed: 900_000 })] },
+    0
+  );
+
+  assert.equal(tail.missing, 512, "every block of it is missing");
+  assert.equal(tail.chunks, 512);
+  assert.equal(tail.outstanding.length, 0, "and not one of them has been asked for");
+});
+
+test("a completed piece is not described at all", () => {
+  // WebTorrent nulls `pieces[index]` once the piece is verified and stored.
+  assert.equal(describePieceTail({ pieces: [null], wires: [] }, 0), null);
+  assert.equal(describePieceTail({ pieces: [], wires: [] }, 0), null);
+});
+
+test("a wire that is choking us is named as such", () => {
+  const tail = describePieceTail(
+    torrentWith({
+      buffer: [null],
+      wires: [wire({ blocks: 1, speed: 100, choking: true })]
+    }),
+    0
+  );
+
+  assert.equal(tail.outstanding[0].choking, true, "a block reserved by a choking wire is going nowhere");
+});