@torrent-tv/proxy 2.9.76 → 2.9.77

package/CHANGELOG.md

@@ -1,3 +1,10 @@
+## 2.9.77
+
+- **Fix**: Anything naming a source while that source was still being added got `Unknown source` — which is false, because the source exists and is merely not ready. Adding a magnet takes as long as its metadata does, seconds to tens of seconds, and the browser polls stats and asks for a playback plan throughout that window. The worker registered the torrent only once the add had **finished**; it now registers the pending add itself, so callers wait for it. Reproduced with a magnet nobody seeds: stats, the file listing and a read all failed instantly while the add was in flight, and all three now wait. A source that was never added is still an error, and a failed add is forgotten rather than replayed to every later caller.
+- **Fix**: File claims are held per reader instead of per file. The proxy reads one file from several places at once — ffmpeg's input, the keyframe index, the codec probe, a second viewer — and claims keyed by `sourceKey:fileIndex` were therefore shared: the first reader to finish released the hold while the others were still reading, leaving the data free to be evicted under them. Each acquire now returns its own claim identity and a release names exactly that claim, so a duplicate or late release matches nothing, is logged, and harms no one. A counter would have restored the arithmetic but kept the ambiguity.
+- **Fix**: `HEAD /stream` no longer starts a read of the whole file. Fastify serves HEAD from the GET handler, so a HEAD opened a full-file read whose body Node discarded while the read itself ran on, the response never completed, and the next request on that keep-alive connection waited behind it — measured in the field as headers in 23 ms followed by a 15 s stall, which is where a 73 s transcode-session create came from. It also has to report the real size: the keyframe index asks for it with this very request and treats zero as "no index", silently falling back to an invented segment grid, so the response is written to the raw socket rather than through `reply.send()`, which substitutes `content-length: 0` for an empty payload.
+- **Chore**: `prefetchFileEdges` takes an options object at every layer, matching `TorrentPool`. The worker adapter declared positional parameters instead, so the planner's options object arrived as `headBytes` and only worked because it was passed along far enough to be destructured at the far end; anyone calling it as documented silently got the defaults.
+
 ## 2.9.76
 
 - **Fix**: A read that failed inside the torrent thread left the reader waiting forever, and a read that failed part-way looked like a file that had simply ended. Two halves of one hole, both present since the thread split: the worker sent the end-of-read marker from its `finally` even when the read had thrown, and the main thread had no handler for a read error at all — so the report was dropped as unknown. That is why the 2.9.71 defect took three releases to find: every symptom said "empty file", never "this read failed, here is why". Now the marker is sent only on success and the failure fails the caller's stream. Covered end to end by a test that hung before the fix.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@torrent-tv/proxy",
-  "version": "2.9.76",
+  "version": "2.9.77",
   "description": "Torrent proxy client that exposes webseed-like HTTP stream endpoint.",
   "license": "GPL-3.0-or-later",
   "publishConfig": {

package/routes/stream/get.js

@@ -1,116 +1,141 @@
-/**
- * @file Byte-range aware torrent file streaming endpoint.
- *
- * Accepts either a `sourceKey` (registered via POST /api/sources) or a raw
- * `sourceType` + `source` pair.  Responds with HTTP 206 for range requests
- * and HTTP 200 for full-file requests.
- */
-
-import { parseRange } from "../../utils/parse-range.js";
-
-/**
- * Resolve source parameters from the query string.
- * Prefers a registered `sourceKey`; falls back to inline `sourceType`+`source`.
- *
- * @param {import("fastify").FastifyRequest["query"]} query
- * @param {ReturnType<import("../../store/source-registry.js").createSourceRegistry>} sourceRegistry
- * @returns {{ sourceType: string, source: string }}
- */
-function getSourceParams(query, sourceRegistry) {
-  const sourceKey = typeof query.sourceKey === "string" ? query.sourceKey : "";
-  const sourceTypeFromQuery = typeof query.sourceType === "string" ? query.sourceType : "";
-  const sourceFromQuery = typeof query.source === "string" ? query.source : "";
-
-  const sourceRecord = sourceKey ? sourceRegistry.get(sourceKey) : null;
-  const sourceType = sourceRecord?.sourceType ?? sourceTypeFromQuery;
-  const source = sourceRecord?.source ?? sourceFromQuery;
-  return { sourceType, source };
-}
-
-/**
- * Stream a torrent file over HTTP with byte-range support.
- *
- * GET /stream
- *
- * @param {import("fastify").FastifyRequest} req
- * @param {import("fastify").FastifyReply} reply
- * @param {{ sourceRegistry: ReturnType<import("../../store/source-registry.js").createSourceRegistry>, torrentPool: import("../../services/torrent-pool.js").TorrentPool }} deps
- * @returns {Promise<void>}
- */
-export async function handleStreamGet(req, reply, { sourceRegistry, torrentPool }) {
-  const fileIndexRaw = typeof req.query.fileIndex === "string" ? req.query.fileIndex : "";
-  const fileIndex = Number(fileIndexRaw);
-  const { sourceType, source } = getSourceParams(req.query, sourceRegistry);
-
-  if (!sourceType || !source || !Number.isInteger(fileIndex) || fileIndex < 0) {
-    return reply
-      .code(400)
-      .send({ error: "sourceKey or sourceType+source with fileIndex are required." });
-  }
-
-  let torrent;
-  try {
-    torrent = await torrentPool.getTorrent(sourceType, source);
-  } catch (error) {
-    const message = error instanceof Error ? error.message : String(error);
-    return reply.code(500).send({ error: `Failed to load torrent source: ${message}` });
-  }
-
-  const file = torrent.files[fileIndex];
-  if (!file) {
-    return reply.code(404).send({ error: "File index was not found in torrent." });
-  }
-
-  const releaseFile = torrentPool.acquireFile(torrent, fileIndex);
-
-  const range = parseRange(req.headers.range, file.length);
-  // Prioritize the pieces at this read position so a seek (a request at a new
-  // byte offset) downloads first instead of waiting behind the sequential
-  // backlog — this is what caused ~15-18 s stalls when seeking into an
-  // undownloaded region.
-  torrentPool.prioritizeByteRange(torrent, fileIndex, range ? range.start : 0);
-  reply.header("Accept-Ranges", "bytes");
-  reply.header("Content-Type", "application/octet-stream");
-  reply.header("Content-Disposition", `inline; filename*=UTF-8''${encodeURIComponent(file.name)}`);
-
-  if (!range) {
-    reply.header("Content-Length", String(file.length));
-    const stream = file.createReadStream();
-    bindRelease(stream, reply, releaseFile);
-    return reply.send(stream);
-  }
-
-  const contentLength = range.end - range.start + 1;
-  reply.code(206);
-  reply.header("Content-Length", String(contentLength));
-  reply.header("Content-Range", `bytes ${range.start}-${range.end}/${file.length}`);
-  const stream = file.createReadStream({ start: range.start, end: range.end });
-  bindRelease(stream, reply, releaseFile);
-  return reply.send(stream);
-}
-
-/**
- * Attach event listeners that release the file reference exactly once when
- * the stream or the underlying HTTP connection closes.
- *
- * @param {import("node:stream").Readable} stream
- * @param {import("fastify").FastifyReply} reply
- * @param {() => void} release
- * @returns {void}
- */
-function bindRelease(stream, reply, release) {
-  let released = false;
-  const releaseOnce = () => {
-    if (released) {
-      return;
-    }
-    released = true;
-    release();
-  };
-
-  stream.on("close", releaseOnce);
-  stream.on("end", releaseOnce);
-  stream.on("error", releaseOnce);
-  reply.raw.once("close", releaseOnce);
-  reply.raw.once("finish", releaseOnce);
-}
+/**
+ * @file Byte-range aware torrent file streaming endpoint.
+ *
+ * Accepts either a `sourceKey` (registered via POST /api/sources) or a raw
+ * `sourceType` + `source` pair.  Responds with HTTP 206 for range requests
+ * and HTTP 200 for full-file requests.
+ */
+
+import { parseRange } from "../../utils/parse-range.js";
+
+/**
+ * Resolve source parameters from the query string.
+ * Prefers a registered `sourceKey`; falls back to inline `sourceType`+`source`.
+ *
+ * @param {import("fastify").FastifyRequest["query"]} query
+ * @param {ReturnType<import("../../store/source-registry.js").createSourceRegistry>} sourceRegistry
+ * @returns {{ sourceType: string, source: string }}
+ */
+function getSourceParams(query, sourceRegistry) {
+  const sourceKey = typeof query.sourceKey === "string" ? query.sourceKey : "";
+  const sourceTypeFromQuery = typeof query.sourceType === "string" ? query.sourceType : "";
+  const sourceFromQuery = typeof query.source === "string" ? query.source : "";
+
+  const sourceRecord = sourceKey ? sourceRegistry.get(sourceKey) : null;
+  const sourceType = sourceRecord?.sourceType ?? sourceTypeFromQuery;
+  const source = sourceRecord?.source ?? sourceFromQuery;
+  return { sourceType, source };
+}
+
+/**
+ * Stream a torrent file over HTTP with byte-range support.
+ *
+ * GET /stream
+ *
+ * @param {import("fastify").FastifyRequest} req
+ * @param {import("fastify").FastifyReply} reply
+ * @param {{ sourceRegistry: ReturnType<import("../../store/source-registry.js").createSourceRegistry>, torrentPool: import("../../services/torrent-pool.js").TorrentPool }} deps
+ * @returns {Promise<void>}
+ */
+export async function handleStreamGet(req, reply, { sourceRegistry, torrentPool }) {
+  const fileIndexRaw = typeof req.query.fileIndex === "string" ? req.query.fileIndex : "";
+  const fileIndex = Number(fileIndexRaw);
+  const { sourceType, source } = getSourceParams(req.query, sourceRegistry);
+
+  if (!sourceType || !source || !Number.isInteger(fileIndex) || fileIndex < 0) {
+    return reply
+      .code(400)
+      .send({ error: "sourceKey or sourceType+source with fileIndex are required." });
+  }
+
+  let torrent;
+  try {
+    torrent = await torrentPool.getTorrent(sourceType, source);
+  } catch (error) {
+    const message = error instanceof Error ? error.message : String(error);
+    return reply.code(500).send({ error: `Failed to load torrent source: ${message}` });
+  }
+
+  const file = torrent.files[fileIndex];
+  if (!file) {
+    return reply.code(404).send({ error: "File index was not found in torrent." });
+  }
+
+  // HEAD asks what a GET would return, not for the bytes. Fastify serves HEAD
+  // from this same handler, which used to mean a HEAD started a read of the
+  // WHOLE file: the body was discarded by Node, but the read ran on, the
+  // response never finished, and the next request on that keep-alive connection
+  // waited behind it. Measured on the field host: the keyframe-index HEAD
+  // returned headers in 23 ms and then held the connection until its 15 s
+  // timeout, which is where the 73 s transcode-session create went.
+  if (req.method === "HEAD") {
+    // Written to the raw response on purpose. Answering through `reply.send()`
+    // with no payload makes Fastify set `content-length: 0`, which is worse
+    // than useless here: the keyframe index asks for the file size with this
+    // very request and treats 0 as "no index available", silently falling back
+    // to an invented segment grid. Hijacking leaves the response to us, and
+    // Node omits the body for HEAD by itself.
+    reply.hijack();
+    reply.raw.writeHead(200, {
+      "Accept-Ranges": "bytes",
+      "Content-Type": "application/octet-stream",
+      "Content-Length": String(file.length),
+      "Content-Disposition": `inline; filename*=UTF-8''${encodeURIComponent(file.name)}`
+    });
+    reply.raw.end();
+    return;
+  }
+
+  const releaseFile = torrentPool.acquireFile(torrent, fileIndex);
+
+  const range = parseRange(req.headers.range, file.length);
+  // Prioritize the pieces at this read position so a seek (a request at a new
+  // byte offset) downloads first instead of waiting behind the sequential
+  // backlog — this is what caused ~15-18 s stalls when seeking into an
+  // undownloaded region.
+  torrentPool.prioritizeByteRange(torrent, fileIndex, range ? range.start : 0);
+  reply.header("Accept-Ranges", "bytes");
+  reply.header("Content-Type", "application/octet-stream");
+  reply.header("Content-Disposition", `inline; filename*=UTF-8''${encodeURIComponent(file.name)}`);
+
+  if (!range) {
+    reply.header("Content-Length", String(file.length));
+    const stream = file.createReadStream();
+    bindRelease(stream, reply, releaseFile);
+    return reply.send(stream);
+  }
+
+  const contentLength = range.end - range.start + 1;
+  reply.code(206);
+  reply.header("Content-Length", String(contentLength));
+  reply.header("Content-Range", `bytes ${range.start}-${range.end}/${file.length}`);
+  const stream = file.createReadStream({ start: range.start, end: range.end });
+  bindRelease(stream, reply, releaseFile);
+  return reply.send(stream);
+}
+
+/**
+ * Attach event listeners that release the file reference exactly once when
+ * the stream or the underlying HTTP connection closes.
+ *
+ * @param {import("node:stream").Readable} stream
+ * @param {import("fastify").FastifyReply} reply
+ * @param {() => void} release
+ * @returns {void}
+ */
+function bindRelease(stream, reply, release) {
+  let released = false;
+  const releaseOnce = () => {
+    if (released) {
+      return;
+    }
+    released = true;
+    release();
+  };
+
+  stream.on("close", releaseOnce);
+  stream.on("end", releaseOnce);
+  stream.on("error", releaseOnce);
+  reply.raw.once("close", releaseOnce);
+  reply.raw.once("finish", releaseOnce);
+}