@torrent-tv/proxy 2.16.0 → 2.18.0

package/CHANGELOG.md

@@ -1,3 +1,17 @@
+## 2.18.0
+
+- **New**: Copying the picture is no longer priced at nothing. It demuxes, re-encodes the audio and writes segments, and it is what runs BESIDE every rung warmed for a quality change — the field measured it at 7.92-8.02x, about an eighth of a second of work per second of video. The figure is not a constant: a session that is copying reports its own speed, and the reciprocal of that IS the cost, learned per file as the decode cost already is (median of recent readings, only from a run past its own start, never from a suspended one).
+- **New**: A rung is judged against the machine it will actually have. The cost of what the family is already committed to is added to the rung's own before the check, so the arithmetic of 2026-08-15 comes out as it did in the field: 0.125 for the copy plus about 1.05 for the 240p rung is more than the one second of work per second the machine has. Unmeasured means zero, so a host that has observed nothing is exactly as permissive as before.
+- **Fix**: A copy reading taken while the torrent is short is discarded. A re-encode near realtime may be the host's limit; a copy near realtime is a copy waiting for data, since copying runs at eight times realtime — and filing that as the price of copying would refuse rungs on the download's account. An audio rendition is excluded from this learning too: it carries no picture, and its speed is the price of a soundtrack, not of a copy.
+
+## 2.17.0
+
+- **New**: The encoder is benchmarked on real footage instead of a generated test pattern, and measured by ffmpeg's own progress rather than by the clock around the process. The pattern has flat areas and no grain and encodes **1.23x** cheaper than film on the same machine and preset — an error that always points at offering a rung the host cannot hold. Timing whole runs was the second error: process startup is ~0.4 s, which put `fast` and `ultrafast` within 1.24x of each other when they differ by three times. The clip is decoded once to raw frames in a temp file (feeding them through a pipe measured the pipe: the fastest presets want hundreds of megabytes a second), each preset is read from the slope between two progress reports, and the run is stopped as soon as a second of it has been covered.
+- **Fix**: A preset that ends before its window is covered is still measured, but never over a window of no width — two reports a millisecond apart would have called a host twenty times faster than it is, and one such reading is what every ladder decision is then taken from. A position ffmpeg reports as the smallest signed 64-bit integer (some builds print that instead of `N/A` before the first packet) is discarded, and a slope above a thousand times realtime is treated as a fault rather than as a fast machine.
+- **Fix**: Which rungs may be offered is decided from the CHEAPEST preset's throughput, not from the largest reading in the array. Measurements scatter on a busy machine — `faster` read below `fast` twice on 2026-08-15 — and taking the maximum let one noisy reading of an expensive preset raise the bar that decides what is offered. Choosing a preset still scans every entry rather than stopping at the first miss, because there the direction of that error costs picture quality, not playback.
+- **Fix**: A host with nothing measured says so in those words — `the quality ladder is UNFILTERED on this host` — because that is what an empty benchmark means, and the previous wording said only that presets were unmeasured. The benchmark also can no longer stop the proxy from starting: a missing or read-only temp directory, or a locked file after a kill, is a host left unmeasured, not a process that fails to listen.
+- **Fix**: The host-load line counts CPU per PROCESS across readings, and only for processes present in both. A seek kills ffmpeg and starts another whose counter begins at zero, so subtracting one total from another printed shares like `-598%`; and on a host without `/proc` the sum of no readings was reported as a confident `0%` beside honest `n/a`s.
+
 ## 2.16.0
 
 - **New**: While an encoder runs, one line every five seconds says what the MACHINE is doing: the share of it ffmpeg is getting, the share everything else is taking, the share spent waiting on a disk, the CPU's current clock and its temperature. The budget predicts a rung from benchmarks taken at startup on an idle box, and on 2026-08-15 it predicted 1.83x for a rung that then ran at 0.90-0.999x with nothing else encoding — and no log anywhere could say which of the candidate reasons it was. Now the reading exists: an encoder starved of cores, a machine that has dropped its clock or grown hot, and work around the encode that nobody counted all look different in this line. Linux-only and best effort — a host without `/proc` writes nothing and nothing else changes.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@torrent-tv/proxy",
-  "version": "2.16.0",
+  "version": "2.18.0",
   "description": "Torrent proxy client that exposes webseed-like HTTP stream endpoint.",
   "license": "GPL-3.0-or-later",
   "publishConfig": {

package/server.js

@@ -124,21 +124,15 @@ export async function startProxyServer({ host, port, transcodeAudio, ffmpegBin,
   // For software libx264, benchmark preset throughput once at startup so the
   // session manager can pick the highest-quality preset that still encodes each
   // stream faster than realtime. Hardware encoders use their own fixed preset.
-  const softwarePresetBenchmark = videoEncoder?.kind === "software"
-    ? await benchmarkSoftwarePresets({ ffmpegBin, logger })
-    : null;
-  // A re-encode pays for decoding as well, and the preset benchmark measures
-  // only the encoder — which is how a rung this host runs at 0.39x came to be
-  // offered as if it cleared realtime 2.5× over (measured 2026-08-14). Solve
-  // the decode cost from the bundled calibration clips once at startup; every
-  // source is then priced from figures the probe already has.
-  // Only the software path can read it: the budget and the ladder both bail
-  // on a missing preset benchmark, and that is only produced for libx264. A
-  // host with a hardware encoder would pay three decodes at every start for a
-  // figure nothing would ever ask for.
+  // The decode model first, and the preset benchmark second — they are
+  // independent now (the presets are timed on raw frames), but the order costs
+  // nothing and keeps the two figures side by side in the log.
   const decodeCostModel = videoEncoder?.kind === "software"
     ? await benchmarkDecodeCost({ ffmpegBin, logger })
     : null;
+  const softwarePresetBenchmark = videoEncoder?.kind === "software"
+    ? await benchmarkSoftwarePresets({ ffmpegBin, logger })
+    : null;
   // Whether this ffmpeg build can tone-map HDR→SDR (zscale + tonemap filters).
   // Detected once; the session manager applies the tonemap chain only for HDR
   // sources on the software path when available.

package/services/hls-session-manager.js

@@ -18,7 +18,7 @@ import { spawn } from "node:child_process";
 import { createRequire } from "node:module";
 import { logger } from "../utils/logger.js";
 import { readKeyframeIndex } from "./container-index/index.js";
-import { readMachineState, sampleHost, shareOfMachine } from "./host-load.js";
+import { readMachineState, readProcessCpuSeconds, readSystemCpu, shareOfMachine } from "./host-load.js";
 
 /** Own package version, stamped onto session-start log lines. */
 const PROXY_VERSION = createRequire(import.meta.url)("../package.json").version;
@@ -1194,6 +1194,15 @@ export class HlsSessionManager {
    * @type {Map<string, { costSec: number, version: number }>}
    */
   #observedDecodeCost = new Map();
+  /**
+   * What copying costs, per source file, learned the same way: `key ->
+   * { costSec, readings, version }`. A copy is what runs BESIDE a rung being
+   * warmed, and pricing it at nothing is what let a host be told it had a whole
+   * machine for the rung.
+   *
+   * @type {Map<string, { costSec: number, readings: number[], version: number }>}
+   */
+  #observedCopyCost = new Map();
   /** The previous reading of the machine, to compare the next one against. */
   #hostLoadSample = null;
 
@@ -2665,27 +2674,69 @@ export class HlsSessionManager {
       this.#hostLoadSample = null;
       return;
     }
-    // The busiest single encoder is the one worth naming; the system share
-    // below covers everything else the box is doing.
-    const watched = encoding[0];
-    const sample = await sampleHost(watched.ffmpeg?.pid ?? null);
+    // EVERY encoder, added up. One of them is meaningless on a host that runs a
+    // picture and an audio track at once, and picking the first would have
+    // reported whichever the map happened to hold.
+    // Kept per PROCESS, not as one total. The set changes between readings —
+    // a seek kills ffmpeg and starts another with a new pid whose counter
+    // begins at zero, a session ends, a rendition begins — and subtracting one
+    // total from another across a changed set produces nonsense: a restart
+    // alone would print something like `ffmpeg=-598%`. Only pids present in
+    // BOTH readings are counted, so a process that came or went contributes
+    // nothing rather than a lie.
+    const pids = encoding.map((session) => session.ffmpeg?.pid ?? null).filter((pid) => pid !== null);
+    const [system, ...cpuReadings] = await Promise.all([
+      readSystemCpu(),
+      ...pids.map((pid) => readProcessCpuSeconds(pid))
+    ]);
+    /** @type {Map<number, number>} */
+    const byPid = new Map();
+    pids.forEach((pid, index) => {
+      const seconds = cpuReadings[index];
+      if (seconds !== null) {
+        byPid.set(pid, seconds);
+      }
+    });
+    const sample = { takenAt: Date.now(), byPid, system };
     const previous = this.#hostLoadSample;
     this.#hostLoadSample = sample;
     if (previous === null) {
       return; // the first reading is only something to compare against
     }
-    const share = shareOfMachine(previous, sample);
+    // Summed over the pids both readings hold, so nothing is measured against a
+    // process that was not there before. Unknown stays unknown: on a host with
+    // no `/proc` there are no readings at all, and the share is null rather
+    // than a confident zero.
+    let encoderDelta = null;
+    for (const [pid, seconds] of sample.byPid) {
+      const before = previous.byPid?.get(pid);
+      if (before !== undefined && seconds >= before) {
+        encoderDelta = (encoderDelta ?? 0) + (seconds - before);
+      }
+    }
+    const share = shareOfMachine(
+      { takenAt: previous.takenAt, processCpuSeconds: encoderDelta === null ? null : 0, system: previous.system },
+      { takenAt: sample.takenAt, processCpuSeconds: encoderDelta, system: sample.system }
+    );
     if (share === null) {
       return;
     }
+    // How many of them are stopped by the look-ahead cap. Without this a zero
+    // share reads as an encoder being starved of the machine, when it is an
+    // encoder deliberately not running — which is what the first readings on
+    // the addon host actually were (2026-08-15: `ffmpeg=0% system=24%`, both
+    // encoders suspended, and the speed beside it a stale figure from before
+    // they stopped).
+    const suspended = encoding.filter((session) => session.encoderPaused === true).length;
+    const running = encoding.length - suspended;
     const machine = await readMachineState();
     const asPercent = (value) => (value === null ? "n/a" : `${Math.round(value * 100)}%`);
     logger.info(
       `host-load: ffmpeg=${asPercent(share.processShare)} system=${asPercent(share.systemShare)} ` +
       `iowait=${asPercent(share.iowaitShare)} cpu=${machine.megahertz === null ? "n/a" : `${machine.megahertz}MHz`} ` +
       `temp=${machine.celsius === null ? "n/a" : `${machine.celsius}C`} ` +
-      `encoders=${encoding.length} speed=${watched.progress?.speed ?? "n/a"} ` +
-      `over=${share.elapsedSec.toFixed(1)}s`
+      `encoders=${running} running` + (suspended > 0 ? ` +${suspended} suspended` : "") +
+      ` over=${share.elapsedSec.toFixed(1)}s`
     );
   }
 
@@ -4507,6 +4558,10 @@ export class HlsSessionManager {
       heights: ordered,
       ownHeight: own,
       playingHeight: playing,
+      // What the family is already spending while a rung is considered. The
+      // picture being COPIED is the common case and used to be priced at
+      // nothing; measured, it is about an eighth of the machine.
+      concurrentCostSec: this.#committedCostOf(owner),
       sourceWidth: Number(owner.sourceWidth) || 0,
       sourceHeight: Math.round(Number(owner.sourceHeight) || 0),
       fps: Number(owner.outputFps) || TRANSCODE_FPS,
@@ -4606,9 +4661,76 @@ export class HlsSessionManager {
     this.#learnDecodeCost(session, speed);
   }
 
+  /**
+   * What COPYING this file costs on this host, learned from a session that is
+   * doing it: seconds of work per second of video.
+   *
+   * A copy is not free. It demuxes, it re-encodes the audio, it writes
+   * segments, and it runs BESIDE every rung warmed for a quality change — so a
+   * budget that prices it at nothing predicts a machine that does not exist.
+   * The figure is the reciprocal of the speed the session reports, which is the
+   * measurement itself rather than a model of it.
+   *
+   * Median of recent readings, taken only from a run past its own start, for
+   * the same reasons as the decode cost beside it.
+   *
+   * @param {HlsSession} session
+   * @param {number} speed
+   */
+  async #learnCopyCost(session, speed) {
+    const runStartedAt = Number(session.encodeRunStartedAt);
+    if (!Number.isFinite(runStartedAt) || Date.now() - runStartedAt < DECODE_LEARNING_SETTLE_MS) {
+      return;
+    }
+    if (session.encoderPaused === true) {
+      return; // a suspended run reports a cumulative figure that is decaying
+    }
+    // Always asked, not only below realtime. A re-encode near 1x may be the
+    // host; a COPY near 1x is a copy waiting for the torrent, because copying
+    // is what a machine does at eight times realtime — and a starved reading
+    // filed as the price of copying would refuse rungs on the download's
+    // account.
+    if (await this.#classifyTranscodeBound(session) === "download") {
+      return;
+    }
+    const costSec = 1 / speed;
+    if (!(costSec > 0) || !Number.isFinite(costSec)) {
+      return;
+    }
+    const key = `${session.sourceKey}:${session.fileIndex}`;
+    const known = this.#observedCopyCost.get(key);
+    const readings = [...(known?.readings ?? []), costSec].slice(-DECODE_LEARNING_READINGS);
+    const sorted = [...readings].sort((left, right) => left - right);
+    const median = sorted[Math.floor(sorted.length / 2)];
+    if (known && Math.abs(median - known.costSec) / known.costSec < DECODE_LEARNING_CHANGE) {
+      this.#observedCopyCost.set(key, { ...known, readings });
+      return;
+    }
+    this.#observedCopyCost.set(key, { costSec: median, readings, version: (known?.version ?? 0) + 1 });
+    logger.info(
+      `transcode: ${session.fileName} copies at ${(1 / median).toFixed(2)}x on this host ` +
+        `(median of ${readings.length}, latest ${speed.toFixed(2)}x)`
+    );
+  }
+
   #learnDecodeCost(session, speed) {
-    if (session.transcodeVideo !== true || !(speed > 0)) {
-      return; // a copied video decodes nothing, so it says nothing about decoding
+    if (!(speed > 0)) {
+      return;
+    }
+    if (session.transcodeVideo !== true) {
+      // A copied video decodes nothing, so it says nothing about DECODING —
+      // but it says exactly what COPYING costs, which the budget has been
+      // treating as free. Field 2026-08-15: a copy ran at 7.92-8.02x, i.e.
+      // about an eighth of a second of work per second of video, while a rung
+      // warmed beside it needed the rest of the machine.
+      //
+      // An audio rendition also carries no video, and its speed is the cost of
+      // encoding a soundtrack — a different quantity that must not be filed
+      // under what copying the picture costs.
+      if (session.audioOnly !== true) {
+        void this.#learnCopyCost(session, speed);
+      }
+      return;
     }
     const runStartedAt = Number(session.encodeRunStartedAt);
     if (!Number.isFinite(runStartedAt) || Date.now() - runStartedAt < DECODE_LEARNING_SETTLE_MS) {
@@ -4746,6 +4868,28 @@ export class HlsSessionManager {
    * @param {{ heights: number[], ownHeight: number, sourceWidth: number, sourceHeight: number, fps: number, source: { megapixelsPerSecond: number, megabitsPerSecond: number } | null, transcodeVideo: boolean }} params
    * @returns {number[]}
    */
+  /**
+   * Seconds of work per second of video this family is ALREADY committed to,
+   * beside any rung being considered.
+   *
+   * Today that is the copy of the picture, where the video is copied and its
+   * cost has been observed. An encoded picture is not added: a viewer changing
+   * quality leaves the rung they are on, so the two do not overlap for long,
+   * and the warm-up that does overlap is bounded by the switch. An audio
+   * rendition is not added either, until its cost is measured the same way —
+   * counting it at a guess would refuse rungs on arithmetic nobody took.
+   *
+   * @param {HlsSession} session
+   * @returns {number}
+   */
+  #committedCostOf(session) {
+    if (session.transcodeVideo === true) {
+      return 0;
+    }
+    const observed = this.#observedCopyCost.get(`${session.sourceKey}:${session.fileIndex}`);
+    return observed && observed.costSec > 0 ? observed.costSec : 0;
+  }
+
   #sustainableHeights({
     heights,
     ownHeight,
@@ -4755,7 +4899,8 @@ export class HlsSessionManager {
     fps,
     source,
     transcodeVideo,
-    observedDecodeCostSec = null
+    observedDecodeCostSec = null,
+    concurrentCostSec = 0
   }) {
     const benchmark = this.softwarePresetBenchmark;
     if (!Array.isArray(benchmark) || benchmark.length === 0 || sourceHeight <= 0 || sourceWidth <= 0) {
@@ -4793,7 +4938,8 @@ export class HlsSessionManager {
         decodeModel: this.decodeCostModel,
         source,
         outputPixelsPerSec: width * height * fps,
-        observedDecodeCostSec
+        observedDecodeCostSec,
+        concurrentCostSec
       });
       if (sustainable) {
         kept.push(height);