@torrent-tv/proxy 2.83.5 → 2.83.6

package/CHANGELOG.md

@@ -1,3 +1,11 @@
+## 2.83.6
+- **Fix**: A line forwarded from the torrent thread is no longer judged twice. That thread holds the same repeat rule and applies it before forwarding, and the main thread ran it again on a different history. It does not lose lines today — a re-printed line carries its held-back count, which makes the text unique — but that is an accident of the wording, against a promise this module states outright: a line cannot reach the console and miss the file. It also filled the main thread's bounded map with keys that can never repeat.
+- **Fix**: The LEVEL travels with a forwarded line. It was dropped at the send, so every warning and every error the torrent thread has ever written arrived on the other side as information, and was coloured and recorded as such.
+- **Fix**: The wait after a lost input is cleared only by a run that PRODUCED something, and the field documenting it said "cleared by any other ending" — the rule it replaced, two screens above the code that contradicted it.
+- **New**: `torrent-pool: N piece claim(s) withdrawn` is printed. The pool counted them and nothing read the count, which is the state this project's own rule calls indistinguishable from having noticed once. The gap between it and the stores' `withdrawn=` is the reading: announcements far above withdrawals mean the stores are mostly dropping pieces that were never completed, and a withdrawal count stuck at zero while announcements climb means the mechanism is not reaching the torrent at all.
+- **Chore**: What the plan remembers about an output's input is forgotten when the priority map says nobody is coming there. Three maps keyed by address would otherwise keep an entry for the life of the process, and the wait they describe would greet whoever opened that output next.
+- **Chore**: The suffix saying how many repeats were held back was built in two branches with the same expression; one function now. The quiet check returned milliseconds that its only reader compared against zero; a boolean now.
+
 ## 2.83.5
 - **Fix**: ONE OWNER OF THE FACT "THIS PROXY HAS PIECE N", AND IT IS THE STORE. The disk tier drops a piece once every reader is past it — correctly, and that is what bounds the spill — while the library kept a second copy of the same fact in its completion bitfield and nothing reconciled them. A read then concluded the piece was had, asked for it, was told it was absent, and failed; nor was it ever fetched again, because the library does not download what it believes it owns. Field 2026-09-12: a film played 80 seconds (1920 frames, and the picture never moved again); the encoder ran on to 725 s, so some 565 spilled pieces fell behind every read head and were dropped, piece 0 among them; the encoder lost its input and restarted, which re-opens the input at byte 0; and `/stream` answered `0 of 2363497962 bytes: Piece 0 is verified but absent from the store` to every read for the next 92 minutes while the browser retried one segment against a healthy transport at 4 ms.
 - **Fix**: So the store announces when it can no longer produce a piece AT ALL — not resident, not on disk, not inside a file held whole — and the pool withdraws the library's claim. The eviction's own bargain, written in its comment as "a seek back re-downloads it", is true for the first time. The three cases are one rule rather than a list of call sites: the disk tier says only that IT has lost a piece, and the store decides whether that is a loss at all, so a piece dropped as a duplicate of an assembled file says nothing and a piece still resident says nothing.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@torrent-tv/proxy",
-  "version": "2.83.5",
+  "version": "2.83.6",
   "description": "Torrent proxy client that exposes webseed-like HTTP stream endpoint.",
   "license": "GPL-3.0-or-later",
   "publishConfig": {

package/services/orchestrators/EncodeOrchestrator.js

@@ -63,7 +63,8 @@ export class EncodeOrchestrator {
 
   /**
    * How many times in a row an output's run has died because its input was not
-   * there. Cleared by any other ending, which is the input answering.
+   * there. Cleared only by a run that PRODUCED something, which is the only
+   * proof that the input can be read — see `#noteInputAvailability`.
    *
    * @type {Map<string, number>}
    */
@@ -307,6 +308,14 @@ export class EncodeOrchestrator {
    */
   notePriorityMap(address, zones) {
     this.demand.state(address, zones);
+    // NOBODY IS COMING HERE, so what was remembered about this output's input
+    // is about nobody. Kept, those three entries would stay for the life of the
+    // process, and the wait they describe would greet whoever opens this output
+    // next — a viewer arriving is new information, and one attempt for them is
+    // right whatever the last one met.
+    if (!Array.isArray(zones) || zones.length === 0) {
+      this.#forgetInputState(address);
+    }
   }
 
   /**
@@ -408,8 +417,7 @@ export class EncodeOrchestrator {
     // AND ONLY WHILE NOTHING IS PRODUCING THERE. A run still alive on this
     // output is proof the input can be read, whatever a run beside it met, so
     // the wait must not silence an output that is working.
-    const quietMs = this.#quietFor(address);
-    if (quietMs > 0 && this.runsOn(address).every((run) => !run.isAlive)) {
+    if (this.#isQuiet(address) && this.runsOn(address).every((run) => !run.isAlive)) {
       return;
     }
     // ONE MAP, NOT ONE WINDOW PER VIEWER PER ZONE.
@@ -812,23 +820,45 @@ export class EncodeOrchestrator {
   }
 
   /**
-   * How long this output still has to wait before anything is placed on it, in
-   * milliseconds; zero when it may be planned now.
+   * Whether this output is still waiting before anything may be placed on it.
+   *
+   * A boolean rather than the milliseconds left: the figure had one reader and
+   * that reader only asked whether it was above zero, so it was a number
+   * computed and thrown away. How long the wait is was said when it began.
    *
    * @param {string} address
-   * @returns {number}
+   * @returns {boolean}
    */
-  #quietFor(address) {
+  #isQuiet(address) {
     const until = this.#quietUntil.get(address);
     if (!Number.isFinite(until)) {
-      return 0;
+      return false;
     }
-    const left = until - this.now();
-    if (left <= 0) {
+    if (until - this.now() <= 0) {
       this.#quietUntil.delete(address);
-      return 0;
+      return false;
+    }
+    return true;
+  }
+
+  /**
+   * Forget everything remembered about an output nobody is producing for.
+   *
+   * Three maps are keyed by address, and an output that goes away leaves an
+   * entry in each for the life of the process. Small, and exactly the shape of
+   * accumulation this layer was built to remove from the session.
+   *
+   * @param {string} address
+   * @returns {void}
+   */
+  #forgetInputState(address) {
+    this.#inputLostAttempts.delete(address);
+    this.#quietUntil.delete(address);
+    const timer = this.#quietTimers.get(address);
+    if (timer) {
+      clearTimeout(timer);
+      this.#quietTimers.delete(address);
     }
-    return left;
   }
 
   /**

package/services/torrent-pool.js

@@ -1067,6 +1067,9 @@ export class TorrentPool {
    */
   #wholeSources = new Set();
 
+  /** How many claims this pool has withdrawn, over its whole life. */
+  #claimsWithdrawn = 0;
+
   /**
    * Say that every file of a source is held whole.
    *
@@ -1165,9 +1168,6 @@ export class TorrentPool {
     }
   }
 
-  /** How many claims this pool has withdrawn, over its whole life. */
-  #claimsWithdrawn = 0;
-
   get claimsWithdrawn() {
     return this.#claimsWithdrawn;
   }

package/services/torrent-worker/client.js

@@ -17,7 +17,7 @@
 import { Worker } from "node:worker_threads";
 import { Readable } from "node:stream";
 import { fileURLToPath } from "node:url";
-import { logger } from "../../utils/logger.js";
+import { logger, writeAlreadyDecided } from "../../utils/logger.js";
 import { createCaller, createReceiveStream } from "./channel.js";
 import { Command, Event } from "./protocol.js";
 
@@ -218,7 +218,10 @@ export class TorrentWorkerClient {
           this.#fragmentReaders.delete(message.id);
           break;
         case Event.LOG:
-          logger.info(`torrent-worker: ${message.message}`);
+          // Written as it is: the worker holds the same repeat rule and has
+          // already applied it, and deciding again here would be one decision
+          // taken twice on two different histories.
+          writeAlreadyDecided(message.level ?? "info", `torrent-worker: ${message.message}`);
           break;
         case Event.FILE_COMPLETE:
           // Recorded here so the stream route can answer from the file without

package/services/torrent-worker/worker.js

@@ -56,8 +56,12 @@ import { forwardLogsTo, logger } from "../../utils/logger.js";
  * copy of the logger that had no file, and every one of their lines was lost:
  * measured over a whole 49 938-line file, not one of them was in it.
  */
-forwardLogsTo((_level, message) => {
-  parentPort.postMessage({ type: Event.LOG, message });
+forwardLogsTo((level, message) => {
+  // THE LEVEL TRAVELS TOO. It was dropped here, so every line this thread wrote
+  // — a warning about a spill that failed, an error about a torrent that went
+  // away — arrived on the other side as information and was coloured and
+  // recorded as such.
+  parentPort.postMessage({ type: Event.LOG, level, message });
 });
 
 // Imported dynamically, and that is load-bearing: static imports are RESOLVED
@@ -658,6 +662,7 @@ startMemoryReport({
 const STORE_REPORT_INTERVAL_MS = 60_000;
 
 /** Last reported reserve, so an unchanged one stays silent. */
+let lastClaimsWithdrawn = 0;
 let lastReserveBytes = 0;
 
 /** Last reported figures per store, so unchanged ones stay silent. */
@@ -695,6 +700,20 @@ setInterval(() => {
       );
     }
   }
+  // WHAT THE ANNOUNCEMENTS ACTUALLY DID. The stores count every piece they
+  // stopped being able to produce; this counts the ones where the library did
+  // still hold a claim and it was taken back. The GAP between the two is the
+  // reading: announcements far above withdrawals mean the stores are mostly
+  // dropping pieces that were never completed, and a withdrawal count stuck at
+  // zero while announcements climb means the mechanism is not reaching the
+  // torrent at all.
+  if (pool.claimsWithdrawn !== lastClaimsWithdrawn) {
+    lastClaimsWithdrawn = pool.claimsWithdrawn;
+    log(
+      `torrent-pool: ${pool.claimsWithdrawn} piece claim(s) withdrawn — pieces this proxy ` +
+      "had dropped and has now told the swarm it needs again"
+    );
+  }
   const reserveNow = machineReserveBytes();
   if (reserveNow !== reserveBefore || reserveNow !== lastReserveBytes) {
     lastReserveBytes = reserveNow;

package/test/logger-repeats.test.js

@@ -14,7 +14,7 @@
 
 import test from "node:test";
 import assert from "node:assert/strict";
-import { logger } from "../utils/logger.js";
+import { logger, writeAlreadyDecided } from "../utils/logger.js";
 
 /** What reached the console while `body` ran. */
 function captured(body) {
@@ -109,6 +109,20 @@ test("when it is said again, it says how many were held back", async () => {
   assert.ok(Number(said[2]) > 0, `the span it covers must be stated: ${later[0]}`);
 });
 
+test("a line already decided on another thread is written as it is", () => {
+  const message = unique("forwarded");
+  const lines = captured(() => {
+    writeAlreadyDecided("info", message);
+    writeAlreadyDecided("info", message);
+    writeAlreadyDecided("warn", message);
+  });
+  // The worker holds the same rule and applies it before forwarding. Deciding
+  // again here would be one decision taken twice on two different histories,
+  // and the file's own promise is that a line cannot reach the console and miss
+  // the file.
+  assert.equal(lines.length, 3, "a forwarded line is not judged a second time");
+});
+
 test("every level goes through the same rule", () => {
   const message = unique("levels");
   const lines = captured(() => {

package/utils/logger.js

@@ -166,6 +166,30 @@ export const logger = {
   error: (message) => write("error", message, chalk.red, console.error)
 };
 
+/**
+ * Write a line that has ALREADY been through the repeat rule on another thread.
+ *
+ * The worker holds the same rule and applies it before forwarding, so running
+ * it again here would be one decision taken twice, on two different histories.
+ * It does not lose lines today — a re-printed line carries its held-back count,
+ * which makes the text unique — but that is an accident of the wording, and the
+ * file's own promise is that a line cannot reach the console and miss the file.
+ * It also filled this thread's bounded map with keys that can never repeat.
+ *
+ * @param {string} level
+ * @param {string} message - Already decided; written as it is.
+ * @returns {void}
+ */
+export function writeAlreadyDecided(level, message) {
+  const colour = level === "error"
+    ? chalk.red
+    : level === "warn" ? chalk.yellow : level === "success" ? chalk.green : chalk.cyan;
+  const toConsole = level === "error" ? console.error : level === "warn" ? console.warn : console.log;
+  const line = `${PREFIX} [${ts()}] ${message}`;
+  toConsole(colour(line));
+  toFile(line);
+}
+
 /**
  * An established fact is said once, then with decreasing frequency.
  *
@@ -195,6 +219,22 @@ const REPEAT_KEYS = 512;
 /** @type {Map<string, { suppressed: number, printedAt: number, interval: number }>} */
 const recent = new Map();
 
+/**
+ * What to append when a line is said again after repeats were held back.
+ *
+ * One function because it was written twice, in the two branches that decide to
+ * speak — which is how two statements of one rule drift apart.
+ *
+ * @param {number} heldBack
+ * @param {number} overMs
+ * @returns {string}
+ */
+function heldBackSuffix(heldBack, overMs) {
+  return heldBack > 0
+    ? ` [said ${heldBack} more time(s) in the last ${(overMs / 1000).toFixed(1)}s]`
+    : "";
+}
+
 /**
  * Whether this line is a repeat to hold back, and what to say if it is not.
  *
@@ -223,12 +263,7 @@ function repeatCheck(message) {
       }
     }
     recent.set(message, { suppressed: 0, printedAt: now, interval: REPEAT_FIRST_MS });
-    return {
-      hold: false,
-      suffix: heldBack > 0
-        ? ` [said ${heldBack} more time(s) in the last ${(overMs / 1000).toFixed(1)}s]`
-        : ""
-    };
+    return { hold: false, suffix: heldBackSuffix(heldBack, overMs) };
   }
   if (now - seen.printedAt < seen.interval) {
     seen.suppressed += 1;
@@ -242,14 +277,9 @@ function repeatCheck(message) {
     printedAt: now,
     interval: Math.min(REPEAT_MAX_MS, seen.interval * 2)
   });
-  return {
-    hold: false,
-    // SAID, not merely hidden: the rate is the fact here, and a log that quietly
-    // drops repeats reports a healthy proxy where a loop was spinning.
-    suffix: heldBack > 0
-      ? ` [said ${heldBack} more time(s) in the last ${(overMs / 1000).toFixed(1)}s]`
-      : ""
-  };
+  // SAID, not merely hidden: the rate is the fact here, and a log that quietly
+  // drops repeats reports a healthy proxy where a loop was spinning.
+  return { hold: false, suffix: heldBackSuffix(heldBack, overMs) };
 }
 
 /**