@torrent-tv/proxy 2.55.0 → 2.55.1

package/CHANGELOG.md

@@ -1,3 +1,8 @@
+## 2.55.1
+
+- **Fix**: A source is now keyed by its own infohash, not by a hash of the request bytes. A magnet URI and a `.torrent` file for the same content are different bytes, so the old key (`sha1` of the source) named the same film as two unrelated sources — measured 2026-08-19, `a518ff46…` and `7ab2fb5d…` for one infohash `11f09299…` — sharing neither the swarm, nor a cache, nor any work already downloaded, and surfacing as `WebTorrent client error: Cannot add duplicate torrent`. The new key (`services/torrent-source-key.js`) reads the infohash straight out of the magnet's `btih` or the `.torrent`'s own `info` dictionary via `parse-torrent`, synchronously, with no network round trip — so both forms of the same torrent now share one entry from the first request, on both the pool's own map (`torrent-pool.js`) and the worker-thread boundary (`torrent-worker/pool-adapter.js`), instead of relying on WebTorrent's own duplicate-add error to reconcile them after the fact.
+- **Chore**: The decode-cost reading now logs how many MB/s the pipe was actually fed against how many MB/s the measured speed implies were needed (item 4(d2)) — a divergence between the two is worth a second look before trusting the reading. A `write()`-return-value signal was tried first, to say outright whether the pipe or the decoder was the limit, and measured false on every reading taken while building it — including clips this host decodes at 15-80x with slack to spare — so it does not discriminate and was not shipped; only the byte count, which is real, is kept.
+
 ## 2.55.0
 
 - **Fix**: The decode calibration was measuring the loop rather than the decode. It looped each clip with `-stream_loop -1`, and a loop is not free: measured 2026-08-22, a restart costs **0.03 s on the 480p clip and 0.12 s on the 1080p one** — it scales with the picture, so it is the decoder tearing down and re-allocating its frame buffers rather than anything about reading the file. A five-second clip decoded at 55x restarts eleven times a second, and that cost dominated the reading: the same clips measured 53.7x looped against 80.3x in one continuous pass, and 11.8x against 15.8x. Worse, the bias depends on BOTH the clip's own resolution and the host's speed — the two axes the fit exists to separate — so it did not cancel out, it tilted the fit. That is the fast-host failure recorded on 2026-08-20, where a desktop read 1080p at 9.35 Mbit/s as cheaper than 720p at 9.94, an ordering no decoder produces, and the H.264 fit refused to solve at all. A host that could not fit H.264 got no decode figure whatsoever, which is exactly the host most able to serve.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@torrent-tv/proxy",
-  "version": "2.55.0",
+  "version": "2.55.1",
   "description": "Torrent proxy client that exposes webseed-like HTTP stream endpoint.",
   "license": "GPL-3.0-or-later",
   "publishConfig": {
@@ -35,6 +35,7 @@
     "franc": "^6.2.0",
     "get-port": "^7.1.0",
     "node-datachannel": "^0.32.0",
+    "parse-torrent": "^11.0.23",
     "uint8-util": "2.2.6",
     "webtorrent": "2.8.5",
     "werift": "^0.24.2",

package/research/preset-benchmark-size-scaling-2026-08-22.md

@@ -0,0 +1,55 @@
+# Does the preset benchmark's pixels/sec figure hold at a different size? (2026-08-22)
+
+Roadmap item 4(e): the software-preset benchmark (`benchmarkSoftwarePresets`,
+`services/hwaccel.js`) measures encoder throughput at one fixed reference size —
+640×360 — and every later use of that figure (deciding which quality rungs this
+host can sustain) scales it by pixel count, assuming cost per pixel is constant
+regardless of frame size. This was never checked against a second size.
+
+## Measurement
+
+Same clip (`cal-h264-1080-hi.mp4`), same `libx264` presets, same measurement
+method as the shipped benchmark (slope between two `-progress` reports,
+excluding startup) — run once at the existing 640×360 reference and once at
+1280×720 (4× the pixel count), on the developer's desktop:
+
+| preset    | 640×360        | 1280×720       | ratio (should be ~1.0) |
+|-----------|----------------|----------------|-------------------------|
+| ultrafast | 1322.8 Mpx/s   | 588.6 Mpx/s    | 0.445                   |
+| fast      | 176.6 Mpx/s    | 128.1 Mpx/s    | 0.725                   |
+
+Script: ad-hoc, not committed (reused the benchmark's own ffmpeg arguments at a
+second `-vf scale=`/`-s` size; available on request if this needs re-running).
+
+## Reading
+
+The assumption does not hold. Cost per pixel RISES with frame size on this
+host — by 2.25× for `ultrafast`, 1.38× for `fast`, at 4× the pixel count — and
+the effect is preset-dependent, not a single constant. A quality ladder priced
+from the 640×360 figure alone over-states what this host can sustain at 1080p,
+worse for the faster presets that the ladder falls back to under load.
+
+## Why this does not become a code change yet
+
+One host, one clip, two sizes. [[feedback_pool_diversity]] — a correction fitted
+to this desktop and applied to every proxy in the pool would be exactly the
+mistake that memory exists to prevent: the pool spans ARM boards (HA Yellow /
+CM4) with different cache sizes and memory bandwidth, where a size penalty of
+this shape could be smaller, larger, or absent. [[feedback_no_fudged_numbers]] —
+publishing 0.445/0.725 as universal constants is fabricating a number for hosts
+that were never measured.
+
+The benchmark already measures per-host (item 4(f), 4(h)); a size-scaling
+correction belongs there too — the benchmark itself gains a second, cheap
+measurement point (one preset, one second size) and derives its own exponent
+from that host's two readings, the same way `fitDecodeCost` derives its terms
+from clips rather than from a stated formula. That is the remaining work under
+item 4(e); this note is the measurement that justifies doing it, not the fix
+itself.
+
+## What is verified
+
+The measurement method matches the shipped benchmark exactly (real footage,
+raw frames decoded once, slope between two progress reports) — the only
+variable changed was the target size, so this is not comparing two different
+methodologies.

package/services/hwaccel.js

@@ -865,7 +865,7 @@ export async function benchmarkContention({ ffmpegBin, logger, clipsDir = CALIBR
     log.warn("hwaccel: contention could not be measured; costs will be added as though jobs were independent");
     return null;
   }
-  const alone = await decodePipedStream(ffmpegBin, stream);
+  const alone = await decodePipedStream(ffmpegBin, stream, log);
   if (!alone?.speed) {
     log.warn("hwaccel: contention could not be measured; costs will be added as though jobs were independent");
     return null;
@@ -892,7 +892,7 @@ export async function benchmarkContention({ ffmpegBin, logger, clipsDir = CALIBR
       await new Promise((resolve) => {
         setTimeout(resolve, 2_000);
       });
-      const withCompany = await decodePipedStream(ffmpegBin, stream);
+      const withCompany = await decodePipedStream(ffmpegBin, stream, log);
       if (withCompany?.speed) {
         beside.push({ others, speed: withCompany.speed });
       }
@@ -985,7 +985,7 @@ async function fitOneFamily({ ffmpegBin, log, clipsDir, family, clips }) {
   }
   for (const [index, clip] of clips.entries()) {
     const stream = streams[index];
-    const measured = stream ? await decodePipedStream(ffmpegBin, stream) : null;
+    const measured = stream ? await decodePipedStream(ffmpegBin, stream, log) : null;
     if (!measured?.speed) {
       log.warn(
         `hwaccel: decode benchmark "${clip}" said nothing; ${family} not measured` +
@@ -1276,7 +1276,7 @@ export async function measureDecodeSlope(ffmpegBin, clipPath, family = "h264") {
  * @param {{ bytes: Buffer, demuxer: string, megapixelsPerSecond: number, megabitsPerSecond: number, fps: number }} stream
  * @returns {Promise<{ speed: number, windowSec: number, megapixelsPerSecond: number, megabitsPerSecond: number } | null>}
  */
-function decodePipedStream(ffmpegBin, stream) {
+function decodePipedStream(ffmpegBin, stream, log = { info: () => {}, warn: () => {} }) {
   return new Promise((resolve) => {
     const args = [
       "-hide_banner", "-loglevel", "error", "-nostats",
@@ -1298,6 +1298,18 @@ function decodePipedStream(ffmpegBin, stream) {
     let settled = false;
     let child;
     const startedAt = Date.now();
+    // Whether the FEED, not the decoder, could be what this reading measures
+    // (item 4(d2)). `child.stdin.write()` returning false was tried as the
+    // signal — never once true would mean this process was never ahead of the
+    // pipe — and measured false on every reading taken while writing this,
+    // including clips this same host decodes at 15-80x with room to spare, so
+    // it does not discriminate: `write()`'s return value tracks Node's own
+    // internal watermark against the size of what was just handed to it, not
+    // real drain state, and answered "no slack" identically whether the pipe
+    // or the decoder was the true limit. Rather than publish a verdict that
+    // reads the same in both cases, only the byte count is kept, for the
+    // MB/s figure below — a number to read, not a boolean to trust.
+    let bytesWritten = 0;
     const finish = () => {
       if (settled) {
         return;
@@ -1329,11 +1341,25 @@ function decodePipedStream(ffmpegBin, stream) {
         resolve({ error: lastErrorLine(stderr) || `the window was ${windowSec.toFixed(2)}s of ${producedSec.toFixed(2)}s produced` });
         return;
       }
+      const speed = producedSec / windowSec;
+      // Diagnostic only — logged, not acted on. The required rate is what
+      // THIS reading implies was needed to keep the decoder fed; the achieved
+      // rate is what this process actually pushed over its whole run (a wider
+      // span than `windowSec`, since the feed starts before the first kept
+      // sample — a looser figure is enough for a sanity comparison).
+      const elapsedSec = (Date.now() - startedAt) / 1000;
+      const achievedMBps = elapsedSec > 0 ? bytesWritten / elapsedSec / 1e6 : 0;
+      const requiredMBps = ((stream.megabitsPerSecond * 1e6) / 8) * speed / 1e6;
+      log.info(
+        `hwaccel: decode pipe fed ${achievedMBps.toFixed(1)} MB/s, ${requiredMBps.toFixed(1)} MB/s ` +
+        `needed for ${speed.toFixed(2)}x — a reading where these are far apart is worth a second look`
+      );
       resolve({
-        speed: producedSec / windowSec,
+        speed,
         windowSec,
         megapixelsPerSecond: stream.megapixelsPerSecond,
-        megabitsPerSecond: stream.megabitsPerSecond
+        megabitsPerSecond: stream.megabitsPerSecond,
+        pipeThroughputMBps: achievedMBps
       });
     };
     const timer = setTimeout(finish, DECODE_WINDOW_MAX_MS);
@@ -1351,8 +1377,13 @@ function decodePipedStream(ffmpegBin, stream) {
     // Keep the decoder fed. `write` returning false means the pipe is full, and
     // the next copy goes on the `drain` — so the decoder is never starved and
     // this process never buffers more than the pipe holds.
+    const writeOnce = () => {
+      const accepted = child.stdin.write(stream.bytes);
+      bytesWritten += stream.bytes.length;
+      return accepted;
+    };
     const feed = () => {
-      while (!settled && child.stdin.writable && child.stdin.write(stream.bytes)) {
+      while (!settled && child.stdin.writable && writeOnce()) {
         // Written straight through; go round again.
       }
     };

package/services/torrent-pool.js

@@ -2,11 +2,12 @@
  * @file WebTorrent client pool.
  *
  * Manages a shared WebTorrent client instance and a map of active torrents
- * keyed by a hash of their source. Tracks file-level usage so that only
- * the pieces needed by active streams are selected for download.
+ * keyed by their own infohash (see `torrent-source-key.js`), so a magnet and
+ * a `.torrent` file for the same content share one entry. Tracks file-level
+ * usage so that only the pieces needed by active streams are selected for
+ * download.
  */
 
-import crypto from "node:crypto";
 import dns from "node:dns/promises";
 import os from "node:os";
 import path from "node:path";
@@ -14,6 +15,7 @@ import { rmSync, statfsSync } from "node:fs";
 import WebTorrent from "webtorrent";
 import { logger } from "../utils/logger.js";
 import { SharedPieceStore, findSharedStore } from "./piece-store/shared-piece-store.js";
+import { deriveSourceKey } from "./torrent-source-key.js";
 
 // The DHT's entry points. Two of the three the library ships answer nothing —
 // measured 2026-08-21 from the addon host: `router.bittorrent.com` and
@@ -1338,7 +1340,9 @@ export class TorrentPool {
    * @returns {Promise<import("webtorrent").Torrent>}
    */
   async getTorrent(sourceType, source) {
-    const key = `${sourceType}:${crypto.createHash("sha1").update(source).digest("hex")}`;
+    // Keyed by the torrent's own infohash, not the source bytes — a magnet
+    // and a `.torrent` file for the same content must name the same swarm.
+    const key = await deriveSourceKey(sourceType, source);
 
     // Already resolved — return immediately.
     const existing = this.torrents.get(key);

package/services/torrent-source-key.js

@@ -0,0 +1,38 @@
+/**
+ * @file Canonical identity for a torrent source: its own infohash.
+ *
+ * A magnet URI and a `.torrent` file for the same content are different
+ * bytes, so naming a source by a hash of those bytes gave the same film two
+ * unrelated keys — one from a magnet, another from a `.torrent` — sharing
+ * neither the swarm, nor a cache, nor any work already done. Measured
+ * 2026-08-19: `a518ff46…` and `7ab2fb5d…` for one infohash `11f09299…`.
+ *
+ * The infohash is the one thing both forms carry for the same content, and
+ * both carry it synchronously — a magnet's `btih` needs no network, and a
+ * `.torrent` file's SHA-1 of its own `info` dictionary needs no metadata
+ * exchange either. `parse-torrent` (already in the dependency tree via
+ * `webtorrent`) reads either form without touching the network.
+ */
+
+import parseTorrent from "parse-torrent";
+
+/**
+ * @param {"magnet" | "torrent"} sourceType
+ * @param {string} source - Magnet URI, or base64-encoded `.torrent` bytes.
+ * @returns {Promise<string>} `torrent:<infohash>` — identical for a magnet
+ *   and a `.torrent` file describing the same content.
+ */
+export async function deriveSourceKey(sourceType, source) {
+  const torrentId = sourceType === "torrent" ? Buffer.from(source, "base64") : source;
+  let parsed;
+  try {
+    parsed = await parseTorrent(torrentId);
+  } catch (error) {
+    const message = error instanceof Error ? error.message : String(error);
+    throw new Error(`Could not read an infohash from this ${sourceType}: ${message}`);
+  }
+  if (!parsed?.infoHash) {
+    throw new Error(`Could not read an infohash from this ${sourceType}.`);
+  }
+  return `torrent:${parsed.infoHash}`;
+}

package/services/torrent-worker/pool-adapter.js

@@ -22,22 +22,8 @@
  *    across calls for the same torrent.
  */
 
-import crypto from "node:crypto";
 import { TorrentWorkerClient } from "./client.js";
-
-/**
- * Stable key for a source, matching how the worker keys its torrents.
- *
- * Derived from the source itself rather than handed out per request, so two
- * routes asking for the same torrent name the same thing on the worker side.
- *
- * @param {"magnet" | "torrent"} sourceType
- * @param {string} source
- * @returns {string}
- */
-function deriveSourceKey(sourceType, source) {
-  return `${sourceType}:${crypto.createHash("sha1").update(source).digest("hex")}`;
-}
+import { deriveSourceKey } from "../torrent-source-key.js";
 
 /**
  * A torrent pool whose work happens on another thread.
@@ -65,7 +51,7 @@ export class WorkerTorrentPool {
    * @returns {Promise<object>}
    */
   async getTorrent(sourceType, source) {
-    const sourceKey = deriveSourceKey(sourceType, source);
+    const sourceKey = await deriveSourceKey(sourceType, source);
     const existing = this.#torrents.get(sourceKey);
     if (existing) {
       return existing;

package/test/decode-measurement.test.js

@@ -71,3 +71,13 @@ test("the clip's own characteristics come back with the reading", async () => {
   assert.ok(measured.megabitsPerSecond > 5 && measured.megabitsPerSecond < 15);
   assert.ok(measured.windowSec >= 0.5, "the slope is taken over a window wide enough to divide by");
 });
+
+test("a reading carries what the pipe was actually fed, for item 4(d2)'s sanity check", async () => {
+  const measured = await measureDecodeSlope(ffmpegStatic, clip("cal-h264-480-lo.mp4"));
+
+  assert.ok(measured);
+  assert.ok(
+    measured.pipeThroughputMBps > 0,
+    "the write path pushed a nonzero amount of the clip through, or nothing was measured at all"
+  );
+});