@torrent-tv/proxy 2.9.39 → 2.9.40

package/CHANGELOG.md

@@ -1,3 +1,8 @@
+## 2.9.40
+
+- **New**: Adaptive upload throttle. Seeding to the BitTorrent swarm does not help our viewer (we deliver over our own channel) — it is pure uplink cost and the riskiest legal act — so the client-wide upload limit now defaults to **off** (`throttleUpload(0)`, was WebTorrent's unlimited default) and is raised only when needed. A 5 s adjuster sets: **0** when no file has an active reader (stop seeding entirely once nothing is being watched); a low **floor** (50 KB/s) while a reader is active (a token upload so tit-for-tat does not choke us to zero); and a **boost** (512 KB/s) only when a torrent is starving (download barely trickling while it still needs data) AND its wires show reciprocity choke (≥2 peers we want data from are choking us) — earning unchoke slots to un-starve the download. The policy is a pure function (`decideUploadLimit`, unit-tested); each change is logged for field tuning. Client-wide limit (one active torrent is the norm today).
+- **Fix**: Seek-aware piece prioritization now actually makes a far seek download the seek target first. On every `/stream` range request the proxy deselects the pieces BEHIND the read position, so WebTorrent's picker — which scans each selection sequentially from its first undownloaded piece — starts at the playhead instead of fetching the undownloaded gap behind it. Previously only a `critical()` window was marked, but `critical` does not reorder the scan (it only enables hotswap: re-requesting a block from a faster peer), so a seek into a large undownloaded region still waited behind the sequential backlog. Behind-playhead pieces are only dropped from the download set (stop fetching), never deleted — a backward seek re-selects them via the same call, and the whole file is re-selected on the next reader acquire; the pinned head/tail (codec probe) is unaffected. The critical read-ahead window (now 16 MB) is reset each call so it stays a moving window rather than accumulating over the whole file across seeks. Single-active-reader scope (the multi-viewer union window is roadmap item 23).
+
 ## 2.9.39
 
 - **Chore**: Log the stack (first frames) of WebTorrent `warning` events, not just the message. Field diagnosis: a playback froze mid-file with repeated `torrent-pool: … warning: Connection error: Cannot read properties of null (reading 'type')` (a WebTorrent µTP null-peer NPE, webtorrent#1932/#1940) while the swarm had seeders — peer connections were failing and the download starved. The old handler logged only the terse message, hiding which library path threw; the stack pinpoints it before we mitigate (next: prefer HTTP/DHT over the timing-out UDP trackers, then consider disabling µTP).

package/docs/socks5-egress-research.md

@@ -0,0 +1,114 @@
+# SOCKS5 torrent egress — feasibility research (NOT implemented)
+
+Research only (2026-07-10), no code. Question: can the proxy route its
+**torrent** traffic (peers, trackers, DHT) through an owner-configured SOCKS5
+endpoint (e.g. a VPN provider's) to hide the owner's home IP from the swarm,
+while WebRTC to the viewer stays direct? Roadmap item 8 in the root
+`CLAUDE.md`.
+
+Stack: WebTorrent 2.8.5, Node. Verified against
+`node_modules/webtorrent/lib/*.js` and `index.js`.
+
+## What "torrent traffic" actually is (and which parts can be SOCKS5'd)
+
+SOCKS5 forwards a connection through a relay so the destination sees the
+relay's IP, not ours. It supports TCP (CONNECT) and, in theory, UDP (UDP
+ASSOCIATE) — but UDP-ASSOCIATE is rarely usable from Node libraries that bind
+their own UDP sockets. So per traffic type:
+
+| Traffic | Transport | Where in WebTorrent | SOCKS5? |
+|---|---|---|---|
+| Peer wire (outgoing) | TCP | `torrent.js` `net.connect(opts)` (~L2112) | **Yes** (TCP CONNECT) |
+| Peer wire (outgoing) | µTP (UDP) | `torrent.js` `utp.connect()` (~L2110) | No (UDP, own socket) → **disable** |
+| HTTP/HTTPS trackers | TCP | `bittorrent-tracker` (HTTP agent) | **Yes** (agent) |
+| WSS trackers | TCP (WebSocket) | `bittorrent-tracker` (ws agent) | **Yes** (agent) |
+| UDP trackers (`udp://`) | UDP | `bittorrent-tracker` | No → **drop** (filter announce list) |
+| DHT | UDP | `bittorrent-dht` | No → **disable** (`dht:false`) |
+| LSD (local peer discovery) | UDP multicast | conn-pool | LAN-only leak → **disable** (`lsd:false`) |
+| Incoming peers | TCP/µTP listener | conn-pool | N/A (inbound; see note) |
+
+Conclusion: a clean, leak-free config is **TCP-only through the SOCKS5 relay**
+with every UDP path turned OFF. This is the standard "torrent proxy" model and
+its standard caveat: you lose µTP peers, UDP trackers, and DHT peer discovery —
+peer counts drop, but HTTP/WSS-tracker swarms (like the rutracker trackers our
+test torrents use) still work, and TCP peers are plentiful.
+
+WebTorrent already exposes the needed off-switches (verified in `index.js`):
+`utp: false` (L82), `dht: false` (L127), `lsd: false` (L76), `webSeeds`
+(L161). Announce-list filtering to `http(s)://`/`wss://` is ours to do.
+
+## The hard part: no dialer hook for TCP peers
+
+WebTorrent has **no public option** to supply a custom socket/dialer for
+outgoing peer connections — it calls `net.connect(opts)` directly inside
+`torrent.js`. Three ways around it:
+
+### Approach A — userland, scoped monkeypatch of `net.connect` (+ disable UDP)
+Wrap `net.connect` so that for a **peer destination** it returns a socket
+tunneled through SOCKS5 (via the already-present `socks` package —
+`SocksClient.createConnection({ proxy, command:'connect', destination })`),
+and for everything else (our loopback HTTP server, the signalling tunnel,
+ffmpeg's `127.0.0.1` fetch, DNS) it calls the real `net.connect`.
+- The `socks` dep is ALREADY in `node_modules` (currently transitive) — would
+  become a direct dependency.
+- Shim detail: `net.connect` returns a socket synchronously and connects
+  async; `SocksClient.createConnection` is Promise-based. Need a small adapter
+  that returns a `net.Socket`-like duplex immediately and attaches the real
+  tunneled socket once the SOCKS handshake resolves (or emits `error`). The
+  `socks` package's event API supports this.
+- Scoping rule: SOCKS only for non-local IPv4/IPv6 destinations; NEVER for
+  `127.0.0.1`/`::1`/the tunnel host — otherwise we'd route our own control
+  plane through the VPN.
+- Trackers/webseeds (HTTP/WSS): pass a `socks-proxy-agent` as the HTTP/ws
+  agent to `bittorrent-tracker` / webseed fetches (need to confirm WebTorrent
+  threads an `agent` option through; may need a small patch).
+- Pros: no fork, no build-chain changes, deployment-agnostic. Cons: global
+  monkeypatch is delicate; must be airtight on the scoping rule.
+
+### Approach B — patch/fork WebTorrent to add a `createConnection` hook
+Add an option so `torrent.js` uses `opts.createConnection ?? net.connect`.
+Cleaner and explicit, but a fork to maintain (or an upstream PR — issue
+webtorrent#807 "SOCKS Proxy Support" is open and unresolved, so upstream is
+unlikely soon). More correct long-term if A proves too fragile.
+
+### Approach C — network-level (VPN container / namespace), NOT in-proxy
+Run the whole proxy process behind a VPN with a kill-switch (e.g. a `gluetun`
+sidecar container, or a Linux network namespace bound to the VPN). Catches
+ALL traffic — TCP, µTP, DHT, UDP trackers — leak-proof, zero proxy code.
+Trade-off: not per-app (the whole proxy egresses via VPN, including the
+signalling tunnel unless split), heavier ops, and awkward for the HA addon
+(`host_network: true`). This is the ROBUST alternative to document alongside;
+for a bare-Docker/npm operator it may actually be the better answer than
+in-proxy SOCKS5.
+
+## Kill-switch (fail-closed) — mandatory for the feature to mean anything
+If the SOCKS5 relay is down, torrent traffic must NOT silently fall back to a
+direct connection (that leaks the exact IP we set out to hide). With Approach
+A this is natural: a failed SOCKS handshake fails the peer connection with no
+direct retry, AND all UDP paths are already disabled, so nothing bypasses the
+relay. Must add a startup connectivity self-test + a clear log line
+(BitPlay's "test proxy" pattern), and surface "proxy down → torrent paused"
+rather than leaking.
+
+## Recommended direction (when we build it)
+1. Primary: **Approach A** — `socks` for TCP peers via a scoped `net.connect`
+   shim + `socks-proxy-agent` for HTTP/WSS trackers + webseeds, with
+   `utp:false`, `dht:false`, `lsd:false`, and announce-list filtered to
+   TCP-based trackers. Fail-closed by construction.
+2. Document **Approach C** (VPN container) in the addon/deployment docs as the
+   leak-proof alternative for operators who prefer it.
+3. Approach B only if A's monkeypatch proves too fragile in practice.
+
+## Open questions to resolve at build time (not now)
+- Does WebTorrent 2.8.x thread an `agent`/proxy option to `bittorrent-tracker`
+  and webseed fetches, or is a small patch needed there too?
+- `socks` socket-shim: cleanest way to present a pre-connect `net.Socket`
+  (custom Duplex vs a paused real socket) without confusing
+  `bittorrent-protocol`'s wire setup.
+- Incoming peers: with UDP off and outbound via SOCKS, do we still accept
+  inbound TCP peers directly (that inbound listener exposes the home IP to
+  connecting peers)? For full hiding, inbound may need to be disabled too
+  (accept that peer discovery becomes outbound-only) — decide per privacy goal.
+- Interaction with the UPnP port mapping / WebRTC UDP port (those are for the
+  viewer transport, NOT torrent egress — must stay direct; confirm the shim's
+  scoping never touches them).

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@torrent-tv/proxy",
-  "version": "2.9.39",
+  "version": "2.9.40",
   "description": "Torrent proxy client that exposes webseed-like HTTP stream endpoint.",
   "license": "GPL-3.0-or-later",
   "publishConfig": {

package/services/torrent-pool.js

@@ -24,10 +24,15 @@ const WEBTORRENT_STORE_ROOT = path.join(os.tmpdir(), "webtorrent");
 // idle (viewer gone) frees the disk. Re-requesting re-adds (re-downloads) it.
 const TORRENT_IDLE_TTL_MS = 300_000;
 
-// Bytes ahead of a read position to mark CRITICAL (download-first) on each
-// range request. Big enough to unstick a seek into an undownloaded region,
-// small enough not to make "everything critical" (which defeats prioritization).
-const PRIORITY_WINDOW_BYTES = 8 * 1024 * 1024;
+// Bytes ahead of a read position to mark CRITICAL on each range request. In
+// WebTorrent, `critical` does NOT reorder the sequential piece scan — it enables
+// HOTSWAP (re-request a block from a faster peer when a slower one already
+// reserved it), so this is the near read-ahead cushion where stealing from slow
+// peers pays off. The actual "download the seek target first" effect comes from
+// deselecting the gap BEHIND the playhead (see prioritizeByteRange). Kept a
+// moving window (reset each call) so criticality never accumulates over the
+// whole file across seeks, which would make hotswap thrash.
+const PRIORITY_WINDOW_BYTES = 16 * 1024 * 1024;
 
 // The file's header/index region the codec probe needs (phase 1). Must match
 // the ranges prefetchFileEdges fetches: leading bytes + trailing bytes.
@@ -43,6 +48,67 @@ const HEADER_TAIL_BYTES = 2 * 1024 * 1024;
 const DISK_CAP_ABSOLUTE_MAX_BYTES = 10 * 1024 * 1024 * 1024; // 10 GB
 const DISK_CAP_SWEEP_INTERVAL_MS = 30_000;
 
+// Adaptive upload. Seeding to the BitTorrent swarm does not help our viewer (we
+// deliver over our own WebRTC/HTTPS channel) — it is pure uplink cost and the
+// riskiest legal act (active distribution). So the default is minimal: no
+// seeding when nothing is being watched, and only a token upload while actively
+// downloading. BUT zero upload can get us choked by tit-for-tat (peers re-rank
+// and stop sending) → slower download → the exact starvation we fight. So the
+// limit is ADAPTIVE: raised only when download is starving AND the wires show
+// reciprocity is the cause (many peers we want data from are choking us).
+const UPLOAD_FLOOR_BYTES = 50 * 1024;            // token upload while a reader is active
+const UPLOAD_BOOST_BYTES = 512 * 1024;           // raised to earn tit-for-tat unchoke slots
+const UPLOAD_STARVING_SPEED_BYTES = 200 * 1024;  // download below this (with demand) = starving
+const UPLOAD_CHOKED_WIRE_THRESHOLD = 2;          // interested-but-choked wires implying reciprocity
+const UPLOAD_ADJUST_INTERVAL_MS = 5_000;
+
+/**
+ * Decide the client-wide upload limit (bytes/sec) from the torrents that
+ * currently have an active reader. Pure function so the policy is unit-testable
+ * without a live swarm.
+ *
+ * - No active readers → 0 (stop seeding entirely; nothing is being watched).
+ * - Any active torrent starving (still wants data, download barely trickling)
+ *   AND showing reciprocity choke (>= threshold wires we are interested in that
+ *   are choking us) → boost, to earn unchoke slots.
+ * - Otherwise → floor (token upload, avoids an immediate choke without seeding).
+ *
+ * @param {Array<{ wires?: Array<{ amInterested?: boolean, peerChoking?: boolean }>, downloadSpeed?: number, progress?: number, name?: string }>} activeTorrents
+ * @param {{ floor?: number, boost?: number, starvingSpeed?: number, chokedThreshold?: number }} [opts]
+ * @returns {{ bytesPerSec: number, reason: string }}
+ */
+export function decideUploadLimit(activeTorrents, opts = {}) {
+  const floor = opts.floor ?? UPLOAD_FLOOR_BYTES;
+  const boost = opts.boost ?? UPLOAD_BOOST_BYTES;
+  const starvingSpeed = opts.starvingSpeed ?? UPLOAD_STARVING_SPEED_BYTES;
+  const chokedThreshold = opts.chokedThreshold ?? UPLOAD_CHOKED_WIRE_THRESHOLD;
+
+  if (!Array.isArray(activeTorrents) || activeTorrents.length === 0) {
+    return { bytesPerSec: 0, reason: "idle: no active readers — stop seeding" };
+  }
+
+  for (const torrent of activeTorrents) {
+    const wires = Array.isArray(torrent?.wires) ? torrent.wires : [];
+    const chokedInterested = wires.filter(
+      (wire) => wire && wire.amInterested === true && wire.peerChoking === true
+    ).length;
+    const downloadSpeed = typeof torrent?.downloadSpeed === "number" ? torrent.downloadSpeed : 0;
+    const progress = typeof torrent?.progress === "number" ? torrent.progress : 0;
+    const starving = progress < 1 && downloadSpeed < starvingSpeed;
+    if (starving && chokedInterested >= chokedThreshold) {
+      const name = typeof torrent?.name === "string" ? torrent.name : "?";
+      return {
+        bytesPerSec: boost,
+        reason:
+          `earn unchoke — "${name}" choked=${chokedInterested}/${wires.length} ` +
+          `down=${Math.round(downloadSpeed / 1024)}KB/s`
+      };
+    }
+  }
+
+  return { bytesPerSec: floor, reason: "active readers, not choke-starved" };
+}
+
 /**
  * Compute the default disk cap: the smaller of a fixed 10 GB and half of the
  * currently free space on the store's filesystem (so a tiny host is never
@@ -140,6 +206,12 @@ export class TorrentPool {
   /** Periodic disk-cap enforcement timer. */
   #diskSweepTimer = null;
 
+  /** Current client-wide upload limit in bytes/sec (adaptive). -1 = not yet set. */
+  #uploadLimit = -1;
+
+  /** Periodic adaptive-upload adjustment timer. */
+  #uploadAdjustTimer = null;
+
   /**
    * @param {{ maxDiskBytes?: number }} [options]
    *   `maxDiskBytes` caps total downloaded torrent data; when omitted a
@@ -192,6 +264,41 @@ export class TorrentPool {
       this.#diskSweepTimer = setInterval(() => this.#enforceDiskCap(), DISK_CAP_SWEEP_INTERVAL_MS);
       this.#diskSweepTimer.unref?.();
     }
+
+    // Adaptive upload: start with seeding OFF (nothing is being watched yet),
+    // then let the periodic adjuster raise it to the floor while a reader is
+    // active and to the boost when download is choke-starved. WebTorrent's
+    // default is unlimited upload, which we explicitly do NOT want.
+    if (typeof this.client.throttleUpload === "function") {
+      this.client.throttleUpload(0);
+      this.#uploadLimit = 0;
+    }
+    this.#uploadAdjustTimer = setInterval(() => this.#adjustUploadLimit(), UPLOAD_ADJUST_INTERVAL_MS);
+    this.#uploadAdjustTimer.unref?.();
+  }
+
+  /**
+   * Re-evaluate and apply the client-wide upload limit from current swarm state
+   * (see {@link decideUploadLimit}). Runs on a timer; only calls into WebTorrent
+   * when the target changes, and logs each change for field tuning.
+   *
+   * @returns {void}
+   */
+  #adjustUploadLimit() {
+    if (!this.client || this.client.destroyed || typeof this.client.throttleUpload !== "function") {
+      return;
+    }
+    const active = [...this.torrents.values()].filter((torrent) => {
+      const usage = this.fileUsageByTorrent.get(torrent);
+      return usage && usage.size > 0;
+    });
+    const { bytesPerSec, reason } = decideUploadLimit(active);
+    if (bytesPerSec === this.#uploadLimit) {
+      return;
+    }
+    this.#uploadLimit = bytesPerSec;
+    this.client.throttleUpload(bytesPerSec);
+    logger.info(`torrent-pool: upload limit -> ${Math.round(bytesPerSec / 1024)} KB/s (${reason})`);
   }
 
   /**
@@ -701,16 +808,42 @@ export class TorrentPool {
   }
 
   /**
-   * Mark the torrent pieces covering a byte window of a file as CRITICAL, so
-   * WebTorrent downloads them before the rest of the selected file. Called on
-   * every range request: after a seek, the new read position jumps the download
-   * queue instead of waiting behind the sequential backlog (which caused
-   * ~15-18 s stalls when seeking into an undownloaded region).
+   * Bias the torrent's download toward the current read position, so a seek
+   * downloads the seek target first instead of waiting behind the sequential
+   * backlog (which caused ~15-18 s stalls when seeking into an undownloaded
+   * region). Called on every range request.
+   *
+   * Two levers, matched to how WebTorrent's picker actually works:
+   *
+   * 1. **Demote the gap BEHIND the playhead** — `deselect(fileStart, playhead-1)`.
+   *    The picker scans each selection sequentially from its first UNdownloaded
+   *    piece; with the whole file selected, a far forward seek would make it
+   *    fetch the undownloaded gap behind the new position first. Removing that
+   *    gap from the selection makes the scan START at the playhead, so all peer
+   *    capacity goes to the pieces the player needs next. This only STOPS
+   *    fetching the gap; already-downloaded pieces stay on disk (deleting them
+   *    is Disk hygiene Level 2), and a later backward seek re-selects the region
+   *    via this same call. The whole file is re-selected by `file.select()` on
+   *    the next `acquireFile`, so nothing is permanently dropped.
+   *
+   * 2. **Critical read-ahead window** — `critical(playhead, playhead+window)`.
+   *    `critical` does not reorder the scan; it enables HOTSWAP (re-request a
+   *    block from a faster peer when a slow one reserved it) over the near
+   *    window. Reset first so criticality stays a moving window rather than
+   *    accumulating over the whole file across seeks.
+   *
+   * Scope: single active reader per file (≈100% today). The multi-viewer union
+   * window — demote only where behind for ALL sessions — is deferred (roadmap
+   * item 23); here the latest read position wins.
+   *
+   * The pinned head/tail (prefetchFileEdges, codec probe) is downloaded up front
+   * and lives forward of the playhead (tail) or is already on disk (head), so
+   * demotion never costs the probe its data.
    *
    * @param {import("webtorrent").Torrent} torrent
    * @param {number} fileIndex
    * @param {number} byteStart - Start offset within the file.
-   * @param {number} [windowBytes] - Bytes ahead of `byteStart` to prioritize.
+   * @param {number} [windowBytes] - Bytes ahead of `byteStart` to mark critical.
    * @returns {void}
    */
   prioritizeByteRange(torrent, fileIndex, byteStart, windowBytes = PRIORITY_WINDOW_BYTES) {
@@ -726,18 +859,43 @@ export class TorrentPool {
       return;
     }
     const fileOffset = Number.isFinite(file.offset) ? file.offset : 0;
+    const fileLength = Number(file.length);
+    if (!Number.isFinite(fileLength) || fileLength <= 0) {
+      return;
+    }
+    const fileStartPiece = Math.floor(fileOffset / pieceLength);
+    const fileEndPiece = Math.floor((fileOffset + fileLength - 1) / pieceLength);
+
     const safeStart = Math.max(0, Number(byteStart) || 0);
     const absStart = fileOffset + safeStart;
-    const absEnd = Math.min(fileOffset + file.length - 1, absStart + Math.max(1, windowBytes) - 1);
-    const startPiece = Math.floor(absStart / pieceLength);
-    const endPiece = Math.floor(absEnd / pieceLength);
-    if (endPiece < startPiece) {
-      return;
+    const playheadPiece = Math.floor(absStart / pieceLength);
+    const absWindowEnd = Math.min(
+      fileOffset + fileLength - 1,
+      absStart + Math.max(1, windowBytes) - 1
+    );
+    const windowEndPiece = Math.floor(absWindowEnd / pieceLength);
+
+    // (1) Demote the gap behind the playhead so the picker scans forward from
+    //     the read position. Only when there IS a gap (not at the file start).
+    if (playheadPiece > fileStartPiece && typeof torrent.deselect === "function") {
+      try {
+        torrent.deselect(fileStartPiece, playheadPiece - 1);
+      } catch {
+        // Best effort — never break streaming because demotion failed.
+      }
     }
-    try {
-      torrent.critical(startPiece, endPiece);
-    } catch {
-      // Best effort — never break streaming because prioritization failed.
+
+    // (2) Reset criticality to a moving read-ahead window (hotswap over the near
+    //     pieces), so it does not accumulate over the whole file across seeks.
+    if (Array.isArray(torrent._critical)) {
+      torrent._critical.length = 0;
+    }
+    if (windowEndPiece >= playheadPiece) {
+      try {
+        torrent.critical(playheadPiece, windowEndPiece);
+      } catch {
+        // Best effort.
+      }
     }
   }
 
@@ -760,6 +918,11 @@ export class TorrentPool {
       clearInterval(this.#diskSweepTimer);
       this.#diskSweepTimer = null;
     }
+    // Stop periodic adaptive-upload adjustment.
+    if (this.#uploadAdjustTimer) {
+      clearInterval(this.#uploadAdjustTimer);
+      this.#uploadAdjustTimer = null;
+    }
     // Cancel any pending idle-removal timers — destroyAll handles teardown.
     for (const timer of this.#idleTimers.values()) {
       clearTimeout(timer);