@torrent-tv/proxy 2.15.3 → 2.17.0

package/CHANGELOG.md

@@ -1,3 +1,15 @@
+## 2.17.0
+
+- **New**: The encoder is benchmarked on real footage instead of a generated test pattern, and measured by ffmpeg's own progress rather than by the clock around the process. The pattern has flat areas and no grain and encodes **1.23x** cheaper than film on the same machine and preset — an error that always points at offering a rung the host cannot hold. Timing whole runs was the second error: process startup is ~0.4 s, which put `fast` and `ultrafast` within 1.24x of each other when they differ by three times. The clip is decoded once to raw frames in a temp file (feeding them through a pipe measured the pipe: the fastest presets want hundreds of megabytes a second), each preset is read from the slope between two progress reports, and the run is stopped as soon as a second of it has been covered.
+- **Fix**: A preset that ends before its window is covered is still measured, but never over a window of no width — two reports a millisecond apart would have called a host twenty times faster than it is, and one such reading is what every ladder decision is then taken from. A position ffmpeg reports as the smallest signed 64-bit integer (some builds print that instead of `N/A` before the first packet) is discarded, and a slope above a thousand times realtime is treated as a fault rather than as a fast machine.
+- **Fix**: Which rungs may be offered is decided from the CHEAPEST preset's throughput, not from the largest reading in the array. Measurements scatter on a busy machine — `faster` read below `fast` twice on 2026-08-15 — and taking the maximum let one noisy reading of an expensive preset raise the bar that decides what is offered. Choosing a preset still scans every entry rather than stopping at the first miss, because there the direction of that error costs picture quality, not playback.
+- **Fix**: A host with nothing measured says so in those words — `the quality ladder is UNFILTERED on this host` — because that is what an empty benchmark means, and the previous wording said only that presets were unmeasured. The benchmark also can no longer stop the proxy from starting: a missing or read-only temp directory, or a locked file after a kill, is a host left unmeasured, not a process that fails to listen.
+- **Fix**: The host-load line counts CPU per PROCESS across readings, and only for processes present in both. A seek kills ffmpeg and starts another whose counter begins at zero, so subtracting one total from another printed shares like `-598%`; and on a host without `/proc` the sum of no readings was reported as a confident `0%` beside honest `n/a`s.
+
+## 2.16.0
+
+- **New**: While an encoder runs, one line every five seconds says what the MACHINE is doing: the share of it ffmpeg is getting, the share everything else is taking, the share spent waiting on a disk, the CPU's current clock and its temperature. The budget predicts a rung from benchmarks taken at startup on an idle box, and on 2026-08-15 it predicted 1.83x for a rung that then ran at 0.90-0.999x with nothing else encoding — and no log anywhere could say which of the candidate reasons it was. Now the reading exists: an encoder starved of cores, a machine that has dropped its clock or grown hot, and work around the encode that nobody counted all look different in this line. Linux-only and best effort — a host without `/proc` writes nothing and nothing else changes.
+
 ## 2.15.3
 
 - **Fix**: A magnet whose swarm never answered no longer poisons the film for good. It leaves a torrent with the right infohash and no file list, and WebTorrent then refuses the same film opened from a `.torrent` as a duplicate — so the answer to every later attempt came from the entry that knows nothing: `Proxy playback plan request failed (404): File index was not found in torrent`, reproduced in a browser 2026-08-15, and no reload could clear it because the useless entry outlives them all. A source that carries the metadata now replaces one that lacks it.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@torrent-tv/proxy",
-  "version": "2.15.3",
+  "version": "2.17.0",
   "description": "Torrent proxy client that exposes webseed-like HTTP stream endpoint.",
   "license": "GPL-3.0-or-later",
   "publishConfig": {

package/server.js

@@ -124,21 +124,15 @@ export async function startProxyServer({ host, port, transcodeAudio, ffmpegBin,
   // For software libx264, benchmark preset throughput once at startup so the
   // session manager can pick the highest-quality preset that still encodes each
   // stream faster than realtime. Hardware encoders use their own fixed preset.
-  const softwarePresetBenchmark = videoEncoder?.kind === "software"
-    ? await benchmarkSoftwarePresets({ ffmpegBin, logger })
-    : null;
-  // A re-encode pays for decoding as well, and the preset benchmark measures
-  // only the encoder — which is how a rung this host runs at 0.39x came to be
-  // offered as if it cleared realtime 2.5× over (measured 2026-08-14). Solve
-  // the decode cost from the bundled calibration clips once at startup; every
-  // source is then priced from figures the probe already has.
-  // Only the software path can read it: the budget and the ladder both bail
-  // on a missing preset benchmark, and that is only produced for libx264. A
-  // host with a hardware encoder would pay three decodes at every start for a
-  // figure nothing would ever ask for.
+  // The decode model first, and the preset benchmark second — they are
+  // independent now (the presets are timed on raw frames), but the order costs
+  // nothing and keeps the two figures side by side in the log.
   const decodeCostModel = videoEncoder?.kind === "software"
     ? await benchmarkDecodeCost({ ffmpegBin, logger })
     : null;
+  const softwarePresetBenchmark = videoEncoder?.kind === "software"
+    ? await benchmarkSoftwarePresets({ ffmpegBin, logger })
+    : null;
   // Whether this ffmpeg build can tone-map HDR→SDR (zscale + tonemap filters).
   // Detected once; the session manager applies the tonemap chain only for HDR
   // sources on the software path when available.

package/services/hls-session-manager.js

@@ -18,6 +18,7 @@ import { spawn } from "node:child_process";
 import { createRequire } from "node:module";
 import { logger } from "../utils/logger.js";
 import { readKeyframeIndex } from "./container-index/index.js";
+import { readMachineState, readProcessCpuSeconds, readSystemCpu, shareOfMachine } from "./host-load.js";
 
 /** Own package version, stamped onto session-start log lines. */
 const PROXY_VERSION = createRequire(import.meta.url)("../package.json").version;
@@ -1193,6 +1194,8 @@ export class HlsSessionManager {
    * @type {Map<string, { costSec: number, version: number }>}
    */
   #observedDecodeCost = new Map();
+  /** The previous reading of the machine, to compare the next one against. */
+  #hostLoadSample = null;
 
   /**
    * @param {HlsSessionManagerOptions} options
@@ -2641,7 +2644,95 @@ export class HlsSessionManager {
     logger.info(`transcode ${session.id} encoder resumed — ${reason} "${session.fileName}"`);
   }
 
+  /**
+   * One line per interval about the MACHINE, while an encoder is running on it.
+   *
+   * The budget predicts a rung from benchmarks taken at startup on an idle box,
+   * and on 2026-08-15 it predicted 1.83x for a rung that ran at 0.90-0.999x
+   * with nothing else encoding. Every candidate explanation is measurable — the
+   * encoder not getting the cores, the machine having dropped its clock or
+   * grown hot, the work around the encode costing more than anyone counted —
+   * and none of them was being measured, so the gap could only be argued about.
+   *
+   * Written only while something is encoding, and only when a reading is
+   * available: on a host without `/proc` this says nothing at all.
+   */
+  async #reportHostLoad() {
+    const encoding = [...this.sessionsById.values()].filter(
+      (session) => session?.ffmpeg != null && !hasChildExited(session.ffmpeg) && session.state !== "disposed"
+    );
+    if (encoding.length === 0) {
+      this.#hostLoadSample = null;
+      return;
+    }
+    // EVERY encoder, added up. One of them is meaningless on a host that runs a
+    // picture and an audio track at once, and picking the first would have
+    // reported whichever the map happened to hold.
+    // Kept per PROCESS, not as one total. The set changes between readings —
+    // a seek kills ffmpeg and starts another with a new pid whose counter
+    // begins at zero, a session ends, a rendition begins — and subtracting one
+    // total from another across a changed set produces nonsense: a restart
+    // alone would print something like `ffmpeg=-598%`. Only pids present in
+    // BOTH readings are counted, so a process that came or went contributes
+    // nothing rather than a lie.
+    const pids = encoding.map((session) => session.ffmpeg?.pid ?? null).filter((pid) => pid !== null);
+    const [system, ...cpuReadings] = await Promise.all([
+      readSystemCpu(),
+      ...pids.map((pid) => readProcessCpuSeconds(pid))
+    ]);
+    /** @type {Map<number, number>} */
+    const byPid = new Map();
+    pids.forEach((pid, index) => {
+      const seconds = cpuReadings[index];
+      if (seconds !== null) {
+        byPid.set(pid, seconds);
+      }
+    });
+    const sample = { takenAt: Date.now(), byPid, system };
+    const previous = this.#hostLoadSample;
+    this.#hostLoadSample = sample;
+    if (previous === null) {
+      return; // the first reading is only something to compare against
+    }
+    // Summed over the pids both readings hold, so nothing is measured against a
+    // process that was not there before. Unknown stays unknown: on a host with
+    // no `/proc` there are no readings at all, and the share is null rather
+    // than a confident zero.
+    let encoderDelta = null;
+    for (const [pid, seconds] of sample.byPid) {
+      const before = previous.byPid?.get(pid);
+      if (before !== undefined && seconds >= before) {
+        encoderDelta = (encoderDelta ?? 0) + (seconds - before);
+      }
+    }
+    const share = shareOfMachine(
+      { takenAt: previous.takenAt, processCpuSeconds: encoderDelta === null ? null : 0, system: previous.system },
+      { takenAt: sample.takenAt, processCpuSeconds: encoderDelta, system: sample.system }
+    );
+    if (share === null) {
+      return;
+    }
+    // How many of them are stopped by the look-ahead cap. Without this a zero
+    // share reads as an encoder being starved of the machine, when it is an
+    // encoder deliberately not running — which is what the first readings on
+    // the addon host actually were (2026-08-15: `ffmpeg=0% system=24%`, both
+    // encoders suspended, and the speed beside it a stale figure from before
+    // they stopped).
+    const suspended = encoding.filter((session) => session.encoderPaused === true).length;
+    const running = encoding.length - suspended;
+    const machine = await readMachineState();
+    const asPercent = (value) => (value === null ? "n/a" : `${Math.round(value * 100)}%`);
+    logger.info(
+      `host-load: ffmpeg=${asPercent(share.processShare)} system=${asPercent(share.systemShare)} ` +
+      `iowait=${asPercent(share.iowaitShare)} cpu=${machine.megahertz === null ? "n/a" : `${machine.megahertz}MHz`} ` +
+      `temp=${machine.celsius === null ? "n/a" : `${machine.celsius}C`} ` +
+      `encoders=${running} running` + (suspended > 0 ? ` +${suspended} suspended` : "") +
+      ` over=${share.elapsedSec.toFixed(1)}s`
+    );
+  }
+
   async #enforceRealtimeBudget() {
+    void this.#reportHostLoad();
     if (this.videoEncoder?.kind !== "software") {
       return;
     }

package/services/host-load.js

@@ -0,0 +1,154 @@
+/**
+ * @file What the machine is doing while an encoder runs on it.
+ *
+ * The budget predicts a rung from two benchmarks taken at startup on an idle
+ * machine. On 2026-08-15 it predicted 1.83x for a rung that then ran at
+ * 0.90-0.999x with nothing else encoding, and nothing in any log said why. The
+ * candidates are all measurable and none of them was being measured:
+ *
+ *   - the encoder is not getting the cores (something else is taking them, or
+ *     it is waiting on input);
+ *   - the machine is no longer the machine that was benchmarked, because it has
+ *     dropped its clock or grown hot;
+ *   - the work around the encode — hashing pieces, serving segments — costs
+ *     more than anyone counted.
+ *
+ * Everything here is Linux-specific and best effort: a host without these files
+ * reports nulls and nothing above it changes. The proxy stays
+ * deployment-agnostic (`../CLAUDE.md`).
+ */
+
+import { readFile } from "node:fs/promises";
+import os from "node:os";
+
+/**
+ * Clock ticks per second, the unit `/proc/<pid>/stat` counts CPU time in.
+ * `getconf CLK_TCK` is 100 on every Linux this runs on; spawning a process to
+ * ask would cost more than the measurement.
+ */
+const CLOCK_TICKS_PER_SECOND = 100;
+
+/**
+ * @typedef {object} CpuTotals
+ * @property {number} busySeconds - Everything but idle.
+ * @property {number} idleSeconds
+ * @property {number} iowaitSeconds - Idle because it is waiting for a disk.
+ */
+
+/**
+ * The system's CPU time since boot, from `/proc/stat`.
+ *
+ * @returns {Promise<CpuTotals | null>}
+ */
+export async function readSystemCpu() {
+  try {
+    const text = await readFile("/proc/stat", "utf8");
+    const line = text.split("\n", 1)[0];
+    if (!line.startsWith("cpu ")) {
+      return null;
+    }
+    const fields = line.trim().split(/\s+/).slice(1).map(Number);
+    if (fields.length < 5 || fields.some((value) => !Number.isFinite(value))) {
+      return null;
+    }
+    const [user, nice, system, idle, iowait] = fields;
+    const busyTicks = user + nice + system + fields.slice(5).reduce((sum, value) => sum + value, 0);
+    return {
+      busySeconds: busyTicks / CLOCK_TICKS_PER_SECOND,
+      idleSeconds: idle / CLOCK_TICKS_PER_SECOND,
+      iowaitSeconds: iowait / CLOCK_TICKS_PER_SECOND
+    };
+  } catch {
+    return null; // not Linux, or /proc is not mounted
+  }
+}
+
+/**
+ * The CPU time one process has used, from `/proc/<pid>/stat`.
+ *
+ * The comm field can itself contain spaces and brackets, so the fields are
+ * counted from the LAST `)` rather than by splitting the whole line — the usual
+ * trap with this file.
+ *
+ * @param {number} pid
+ * @returns {Promise<number | null>} Seconds of CPU, or null.
+ */
+export async function readProcessCpuSeconds(pid) {
+  if (!Number.isInteger(pid) || pid <= 0) {
+    return null;
+  }
+  try {
+    const text = await readFile(`/proc/${pid}/stat`, "utf8");
+    const afterComm = text.slice(text.lastIndexOf(")") + 2).trim().split(/\s+/);
+    // After the comm and state fields, utime is index 11 and stime index 12.
+    const utime = Number(afterComm[11]);
+    const stime = Number(afterComm[12]);
+    if (!Number.isFinite(utime) || !Number.isFinite(stime)) {
+      return null;
+    }
+    return (utime + stime) / CLOCK_TICKS_PER_SECOND;
+  } catch {
+    return null; // the process is gone, or this is not Linux
+  }
+}
+
+/**
+ * Is this still the machine the benchmarks were taken on: its clock and its
+ * temperature.
+ *
+ * @returns {Promise<{ megahertz: number | null, celsius: number | null }>}
+ */
+export async function readMachineState() {
+  const [frequency, temperature] = await Promise.all([
+    readFile("/sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq", "utf8").catch(() => null),
+    readFile("/sys/class/thermal/thermal_zone0/temp", "utf8").catch(() => null)
+  ]);
+  const kilohertz = frequency === null ? Number.NaN : Number(frequency.trim());
+  const milliCelsius = temperature === null ? Number.NaN : Number(temperature.trim());
+  return {
+    megahertz: Number.isFinite(kilohertz) ? Math.round(kilohertz / 1000) : null,
+    celsius: Number.isFinite(milliCelsius) ? Math.round(milliCelsius / 100) / 10 : null
+  };
+}
+
+/**
+ * Turn two readings into the shares of ONE second of wall clock that each part
+ * of the machine spent working. A share is per whole machine: 1.0 means every
+ * core was busy for the whole interval.
+ *
+ * @param {{ takenAt: number, processCpuSeconds: number | null, system: CpuTotals | null }} before
+ * @param {{ takenAt: number, processCpuSeconds: number | null, system: CpuTotals | null }} after
+ * @param {number} [cores=os.cpus().length]
+ * @returns {{ elapsedSec: number, processShare: number | null, systemShare: number | null, iowaitShare: number | null } | null}
+ */
+export function shareOfMachine(before, after, cores = os.cpus().length) {
+  const elapsedSec = (after.takenAt - before.takenAt) / 1000;
+  if (!(elapsedSec > 0) || !(cores > 0)) {
+    return null;
+  }
+  const machineSeconds = elapsedSec * cores;
+  const processShare = before.processCpuSeconds !== null && after.processCpuSeconds !== null
+    ? (after.processCpuSeconds - before.processCpuSeconds) / machineSeconds
+    : null;
+  const systemShare = before.system !== null && after.system !== null
+    ? (after.system.busySeconds - before.system.busySeconds) / machineSeconds
+    : null;
+  const iowaitShare = before.system !== null && after.system !== null
+    ? (after.system.iowaitSeconds - before.system.iowaitSeconds) / machineSeconds
+    : null;
+  return { elapsedSec, processShare, systemShare, iowaitShare };
+}
+
+/**
+ * One reading of everything above, to be compared against the next.
+ *
+ * @param {number | null} pid - The encoder's process, when one is running.
+ * @returns {Promise<{ takenAt: number, processCpuSeconds: number | null, system: CpuTotals | null }>}
+ */
+export async function sampleHost(pid) {
+  const [processCpuSeconds, system] = await Promise.all([
+    pid === null ? Promise.resolve(null) : readProcessCpuSeconds(pid),
+    readSystemCpu()
+  ]);
+  return { takenAt: Date.now(), processCpuSeconds, system };
+}