@torrent-tv/proxy 2.9.42 → 2.9.44

package/CHANGELOG.md

@@ -1,3 +1,12 @@
+## 2.9.44
+
+- **Fix**: Roll back to WebTorrent **2.8.5** (pinned) — 3.x introduced two regressions that broke downloading. (1) `torrent.downloaded`/`file.downloaded`/`file.progress` throw on a `deselect`-ed null piece (worked around in 2.9.43). (2) Worse: the internal piece picker itself throws `Cannot read properties of null (reading 'reserve'/'missing')` when it tries to request a block from a piece our seek prioritization (`prioritizeByteRange` `deselect`) removed — download freezes dead after a seek (field-observed: file stuck at ~51%, `down=0`, picker crashing every second). 2.8.5 is the known-good version: `select`/`deselect`/`critical` and the byte getters all work (verified — add, multi-file download, and the full deselect+critical seek pattern run with zero crashes on 2.8.5). Also pinned **`uint8-util` 2.2.6**: 2.8.5's own range is `^2.2.5`, which *allows* the incompatible 2.3.x that a fresh global install pulled (the original `arr2hex` crash), so the transitive version must be forced back — webtorrent dedupes to 2.2.6 while sub-deps that need 2.3.x keep their own nested copy. The 2.9.43 null-safe getter helpers are now redundant (2.8.5 getters never throw) but left in as harmless defensive code.
+
+## 2.9.43
+
+- **Fix**: Torrents stalled at the metadata/download stage on WebTorrent 3.x — the adaptive upload throttle dropped the client-wide limit to `0` whenever no file had an active reader (e.g. the window before the first read is acquired). In WebTorrent 3.x `throttleUpload(0)` blocks the ENTIRE swarm exchange client-wide — even peer connections and DOWNLOAD — not just seeding (verified: `throttleUpload(0)` → 0 peers, 0 download; `throttleUpload(8KB/s)` → peers connect, multi-MB/s download). The idle branch now returns a minimal keep-alive floor (`UPLOAD_IDLE_FLOOR_BYTES` = 8 KB/s) instead of 0; still effectively no seeding, but the swarm stays alive.
+- **Fix**: Spurious `uncaughtException: Cannot read properties of null (reading 'length')` every few seconds during playback. WebTorrent 3.x nulls `pieces[index]` for pieces we removed from the download set via `deselect` (file selection, seek-behind-playhead demotion), and its own `torrent.downloaded` / `file.downloaded` / `file.progress` getters do not guard that null — they threw in our disk-cap sweep and stats builder. Added null-safe `torrentDownloadedBytes` / `fileDownloadedBytes` helpers (a deselected piece = 0 downloaded, the correct value, while still counting every other piece) and use them in `#currentDiskBytes`/`#enforceDiskCap` and `getFileStats`. Verified byte-for-byte identical to WebTorrent's own getters when no piece is null. (The underlying WebTorrent getter bug is filed upstream; it is non-fatal — download survives it — but the throws were noisy and risky.)
+
 ## 2.9.42
 
 - **Fix**: Torrents failed to load with a proxy crash — the REAL root cause (2.9.41 misdiagnosed it). WebTorrent 2.8.5's `Torrent._onTorrentId` does `arr2hex(parsedTorrent.infoHash)`, but `parse-torrent` returns `infoHash` as a hex **string**. `uint8-util` **2.3.x** rewrote `arr2hex` to require a TypedArray (`Buffer.from(data.buffer …)`); a string's `.buffer` is `undefined` → `Buffer.from(undefined)` → `ERR_INVALID_ARG_TYPE` thrown in a detached microtask. `uint8-util` 2.2.x iterated the argument and tolerated a string, so it only broke once the addon's unpinned global `npm install` pulled 2.3.x. It hit **every** torrent (v1/v2/hybrid alike — `arr2hex` is always called). Diagnosed by reproducing `client.add` inside the addon container and isolating `arr2hex('<hex>')` throwing on 2.3.2 but not 2.2.6. Fix: update **WebTorrent 2.8.5 → 3.x**, where the maintainer replaced that line with `parsedTorrent.infoHash?.substring(0, 7)` (no `arr2hex` on the string) — a proper dependency-forward fix, not a version pin, so `uint8-util`/`parse-torrent` stay current. Verified: the exact broken combo (webtorrent 3.0.16 + uint8-util 2.3.2 + parse-torrent 11.0.23) now adds cleanly, and the full API the proxy uses (`select`/`deselect`/`critical`/`_critical`/`wires`/`throttleUpload`/`createReadStream`/`destroy({destroyStore})`) is unchanged in 3.x.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@torrent-tv/proxy",
-  "version": "2.9.42",
+  "version": "2.9.44",
   "description": "Torrent proxy client that exposes webseed-like HTTP stream endpoint.",
   "license": "GPL-3.0-or-later",
   "publishConfig": {
@@ -29,7 +29,8 @@
     "franc": "^6.2.0",
     "get-port": "^7.1.0",
     "node-datachannel": "^0.32.0",
-    "webtorrent": "^3.0.16",
+    "uint8-util": "2.2.6",
+    "webtorrent": "2.8.5",
     "ws": "^8.18.2"
   }
 }

package/services/torrent-pool.js

@@ -52,13 +52,19 @@ const DISK_CAP_SWEEP_INTERVAL_MS = 30_000;
 
 // Adaptive upload. Seeding to the BitTorrent swarm does not help our viewer (we
 // deliver over our own WebRTC/HTTPS channel) — it is pure uplink cost and the
-// riskiest legal act (active distribution). So the default is minimal: no
-// seeding when nothing is being watched, and only a token upload while actively
-// downloading. BUT zero upload can get us choked by tit-for-tat (peers re-rank
+// riskiest legal act (active distribution). So the default is minimal: a
+// near-zero keep-alive when nothing is being watched (NOT 0 — that blocks the
+// swarm in wt3.x), and only a token upload while actively downloading. BUT zero
+// upload can get us choked by tit-for-tat (peers re-rank
 // and stop sending) → slower download → the exact starvation we fight. So the
 // limit is ADAPTIVE: raised only when download is starving AND the wires show
 // reciprocity is the cause (many peers we want data from are choking us).
 const UPLOAD_FLOOR_BYTES = 50 * 1024;            // token upload while a reader is active
+// Minimal keep-alive upload when NOTHING is being watched. It must NOT be 0:
+// in webtorrent 3.x `throttleUpload(0)` blocks ALL swarm exchange client-wide
+// (even peer connections and DOWNLOAD), so a torrent still fetching metadata /
+// pieces would stall. A few KB/s is negligible seeding but keeps the swarm alive.
+const UPLOAD_IDLE_FLOOR_BYTES = 8 * 1024;
 const UPLOAD_BOOST_BYTES = 512 * 1024;           // raised to earn tit-for-tat unchoke slots
 const UPLOAD_STARVING_SPEED_BYTES = 200 * 1024;  // download below this (with demand) = starving
 const UPLOAD_CHOKED_WIRE_THRESHOLD = 2;          // interested-but-choked wires implying reciprocity
@@ -69,24 +75,27 @@ const UPLOAD_ADJUST_INTERVAL_MS = 5_000;
  * currently have an active reader. Pure function so the policy is unit-testable
  * without a live swarm.
  *
- * - No active readers → 0 (stop seeding entirely; nothing is being watched).
+ * - No active readers → a MINIMAL keep-alive floor (NOT 0: `throttleUpload(0)`
+ *   blocks the whole swarm client-wide in webtorrent 3.x, stalling any torrent
+ *   still downloading). Effectively no seeding, just enough to keep peers.
  * - Any active torrent starving (still wants data, download barely trickling)
  *   AND showing reciprocity choke (>= threshold wires we are interested in that
  *   are choking us) → boost, to earn unchoke slots.
  * - Otherwise → floor (token upload, avoids an immediate choke without seeding).
  *
- * @param {Array<{ wires?: Array<{ amInterested?: boolean, peerChoking?: boolean }>, downloadSpeed?: number, progress?: number, name?: string }>} activeTorrents
- * @param {{ floor?: number, boost?: number, starvingSpeed?: number, chokedThreshold?: number }} [opts]
+ * @param {Array<{ wires?: Array<{ amInterested?: boolean, peerChoking?: boolean }>, downloadSpeed?: number, done?: boolean, name?: string }>} activeTorrents
+ * @param {{ floor?: number, idleFloor?: number, boost?: number, starvingSpeed?: number, chokedThreshold?: number }} [opts]
  * @returns {{ bytesPerSec: number, reason: string }}
  */
 export function decideUploadLimit(activeTorrents, opts = {}) {
   const floor = opts.floor ?? UPLOAD_FLOOR_BYTES;
+  const idleFloor = opts.idleFloor ?? UPLOAD_IDLE_FLOOR_BYTES;
   const boost = opts.boost ?? UPLOAD_BOOST_BYTES;
   const starvingSpeed = opts.starvingSpeed ?? UPLOAD_STARVING_SPEED_BYTES;
   const chokedThreshold = opts.chokedThreshold ?? UPLOAD_CHOKED_WIRE_THRESHOLD;
 
   if (!Array.isArray(activeTorrents) || activeTorrents.length === 0) {
-    return { bytesPerSec: 0, reason: "idle: no active readers — stop seeding" };
+    return { bytesPerSec: idleFloor, reason: "idle: minimal keep-alive (0 blocks the swarm in wt3.x)" };
   }
 
   for (const torrent of activeTorrents) {
@@ -95,8 +104,11 @@ export function decideUploadLimit(activeTorrents, opts = {}) {
       (wire) => wire && wire.amInterested === true && wire.peerChoking === true
     ).length;
     const downloadSpeed = typeof torrent?.downloadSpeed === "number" ? torrent.downloadSpeed : 0;
-    const progress = typeof torrent?.progress === "number" ? torrent.progress : 0;
-    const starving = progress < 1 && downloadSpeed < starvingSpeed;
+    // Use `done` (a plain boolean) — NOT `progress`/`downloaded`, whose getters
+    // iterate the piece array and throw on webtorrent 3.x when a piece is null
+    // (deselected / mid-verify), which would crash this timer every cycle.
+    const notDone = torrent?.done !== true;
+    const starving = notDone && downloadSpeed < starvingSpeed;
     if (starving && chokedInterested >= chokedThreshold) {
       const name = typeof torrent?.name === "string" ? torrent.name : "?";
       return {
@@ -168,6 +180,87 @@ function decodeTorrentSource(sourceType, source) {
   throw new Error("Unsupported sourceType. Expected magnet or torrent.");
 }
 
+/**
+ * Downloaded bytes of a single piece, treating a null (deselected) piece as 0.
+ *
+ * webtorrent 3.x sets `pieces[index] = null` for pieces we removed from the
+ * download set via `deselect` (file selection, seek-behind-playhead demotion).
+ * Its own `get downloaded` / `get progress` do NOT guard that null and throw
+ * `Cannot read properties of null (reading 'length')`, which crashed our
+ * disk-cap and stats timers every cycle. A deselected piece has 0 downloaded
+ * bytes, so treating null as 0 is the correct value — and keeps the OTHER
+ * pieces counted (a blanket try/catch that returned 0 for the whole torrent
+ * would under-report disk usage and freeze the progress bar at 0%).
+ *
+ * @param {import("webtorrent").Torrent} torrent
+ * @param {number} index
+ * @returns {number}
+ */
+function pieceDownloadedBytes(torrent, index) {
+  const len = index === torrent.pieces.length - 1 ? torrent.lastPieceLength : torrent.pieceLength;
+  if (torrent.bitfield.get(index)) {
+    return len; // verified
+  }
+  const piece = torrent.pieces[index];
+  return piece ? len - piece.missing : 0; // in-progress, or null (deselected) → 0
+}
+
+/**
+ * Total downloaded bytes across ALL pieces, null-safe (mirrors webtorrent's
+ * `torrent.downloaded` but never throws on a deselected null piece).
+ *
+ * @param {import("webtorrent").Torrent} torrent
+ * @returns {number}
+ */
+export function torrentDownloadedBytes(torrent) {
+  if (!torrent?.bitfield || !Array.isArray(torrent.pieces)) {
+    return 0;
+  }
+  let downloaded = 0;
+  for (let index = 0; index < torrent.pieces.length; index += 1) {
+    downloaded += pieceDownloadedBytes(torrent, index);
+  }
+  return downloaded;
+}
+
+/**
+ * Downloaded bytes of a single file, null-safe (mirrors webtorrent's
+ * `file.downloaded`, including the first/last-piece offset trims, but never
+ * throws on a deselected null piece).
+ *
+ * @param {import("webtorrent").Torrent} torrent
+ * @param {import("webtorrent").TorrentFile & { _startPiece?: number, _endPiece?: number, offset?: number, length?: number }} file
+ * @returns {number}
+ */
+export function fileDownloadedBytes(torrent, file) {
+  if (!torrent?.bitfield || !Array.isArray(torrent.pieces) || !file) {
+    return 0;
+  }
+  const start = file._startPiece;
+  const end = file._endPiece;
+  const pieceLength = torrent.pieceLength;
+  if (!Number.isInteger(start) || !Number.isInteger(end) || !(pieceLength > 0)) {
+    return 0;
+  }
+  let downloaded = 0;
+  for (let index = start; index <= end; index += 1) {
+    const pieceDownloaded = pieceDownloadedBytes(torrent, index);
+    downloaded += pieceDownloaded;
+    if (index === start) {
+      // First piece may carry irrelevant bytes from the previous file.
+      const irrelevant = file.offset % pieceLength;
+      downloaded -= Math.min(irrelevant, pieceDownloaded);
+    }
+    if (index === end) {
+      // Last piece may carry irrelevant bytes from the next file.
+      const lastLen = index === torrent.pieces.length - 1 ? torrent.lastPieceLength : pieceLength;
+      const irrelevant = lastLen - ((file.offset + file.length) % pieceLength);
+      downloaded -= Math.min(irrelevant, pieceDownloaded);
+    }
+  }
+  return downloaded;
+}
+
 /**
  * Shared WebTorrent pool.
  *
@@ -312,8 +405,7 @@ export class TorrentPool {
   #currentDiskBytes() {
     let total = 0;
     for (const torrent of this.torrents.values()) {
-      const downloaded = typeof torrent?.downloaded === "number" ? torrent.downloaded : 0;
-      total += Math.max(0, downloaded);
+      total += Math.max(0, torrentDownloadedBytes(torrent));
     }
     return total;
   }
@@ -345,7 +437,7 @@ export class TorrentPool {
       if (used <= this.#maxDiskBytes) {
         break;
       }
-      const freed = typeof torrent?.downloaded === "number" ? Math.max(0, torrent.downloaded) : 0;
+      const freed = Math.max(0, torrentDownloadedBytes(torrent));
       const name = typeof torrent?.name === "string" ? torrent.name : "(unknown)";
       const gb = (this.#maxDiskBytes / (1024 * 1024 * 1024)).toFixed(1);
       logger.info(
@@ -651,11 +743,16 @@ export class TorrentPool {
 
     const header = this.#getHeaderRangeProgress(torrent, file);
 
+    // Null-safe downloaded/progress (webtorrent's own getters throw on a
+    // deselected null piece — see fileDownloadedBytes).
+    const fileLength = typeof file.length === "number" ? file.length : 0;
+    const fileDownloaded = fileDownloadedBytes(torrent, file);
+
     return {
       ...base,
-      fileProgress: typeof file.progress === "number" ? file.progress : 0,
-      fileDownloaded: typeof file.downloaded === "number" ? file.downloaded : 0,
-      fileLength: typeof file.length === "number" ? file.length : 0,
+      fileProgress: fileLength > 0 ? Math.max(0, Math.min(1, fileDownloaded / fileLength)) : 0,
+      fileDownloaded,
+      fileLength,
       // Phase-1 progress: how much of the header/index region (the bytes the
       // codec probe needs before transcoding can start) is downloaded. Counted
       // by whole pieces from the torrent bitfield, so it advances coarsely