@torrent-tv/proxy 2.32.0 → 2.33.0

package/CHANGELOG.md

@@ -1,3 +1,8 @@
+## 2.33.0
+
+- **New**: A quality step is judged on the machine it will actually run on. The encoder benchmark measures a QUIET host — one ffmpeg and nothing else — while the addon host was measured 99 % busy, and a step predicted at 1.83x ran at 1.01-1.12x (2026-08-17). The offer now multiplies each prediction by the share of the machine that is free, taken from the same `host-load` reading that is already printed every five seconds. What is subtracted is ONLY the work nobody has been charged for — the kernel, the container, whatever else the owner runs — because our own encoders are already priced by the concurrency arithmetic and the proxy's own work per megabyte moved. Charging those here as well is what shipped in 2.21.0 and emptied the quality menu down to a single copied height. On the field reading the correction is about 0.77, and the "not offering" line now says what the machine had to spare when it decided.
+- **New**: Each step reports what its prediction was worth. When a step runs with the machine to itself, the log states the speed it was predicted at, the speed it measured, and the ratio — so the error that REMAINS after the availability correction is a number in the field rather than an argument. It is written when it moves by more than a tenth, so a steady step says it once. On the field case that correction takes 1.83x to 1.41x against 1.01-1.12x measured: part of the gap, not all of it, and this line is how the rest gets found.
+
 ## 2.32.0
 
 - **New**: The decode cost is fitted from a clip set that can be checked, and a term the measurements do not determine is refused instead of published as a zero. The set that shipped until now was three clips for three unknowns — an EXACT system, with two of the clips at the same pixel rate — and such a system cannot fail visibly: it returns whatever satisfies its equations. On 2026-08-17 it returned `0.007542 × Mpx/s + 0.000000 × Mbit/s + 0.0000 s/s`, so a film's own bitrate never entered its price, and the prediction built on it was 1.8-2.2x optimistic against the same file measured while playing. The new set is six clips — three sizes × two bitrates, the axes varied INDEPENDENTLY — cut from the same Netflix Open Content "Meridian" footage (CC BY 4.0, `assets/calibration/NOTICE.md`), 7.7 MB against 8.8 MB before. Three spare measurements give the fit a residual, and with it two questions it could not ask before: whether a term's whole effect across the measured range exceeds the scatter, and whether the coefficient exceeds its own standard error. A term that fails either is dropped, the rest are fitted again, and the log names it — a zero now means "not measured" only when it says so. A NEGATIVE coefficient is dropped too rather than clamped to zero: more pixels cannot cost less work, so a negative fit is noise beating an effect, not a discovery about the host. Measured on the developer's machine, the new set determines all three terms (`0.000520 × Mpx/s + 0.002086 × Mbit/s + 0.0033 s/s`, typical disagreement 0.0012 s/s), and the bitrate term it recovers matches the difference between the two 1080p clips to 15 %. The arithmetic is a pure module with the degenerate case as a test (`services/decode-cost-fit.js`).

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@torrent-tv/proxy",
-  "version": "2.32.0",
+  "version": "2.33.0",
   "description": "Torrent proxy client that exposes webseed-like HTTP stream endpoint.",
   "license": "GPL-3.0-or-later",
   "publishConfig": {

package/services/available-share.js

@@ -0,0 +1,89 @@
+/**
+ * @file How much of the machine a new encoder can actually have.
+ *
+ * The encoder benchmark measures a QUIET host: one ffmpeg, nothing else. A real
+ * encode runs on a machine that is also downloading, hashing and serving, and
+ * on the addon host that machine was measured 99 % busy — `ffmpeg=52-60%
+ * proxy=17-24% system=99%` — while a step predicted at 1.83x ran at 1.01-1.12x
+ * (2026-08-17).
+ *
+ * What this corrects is ONLY the part nobody has been charged for. That
+ * distinction is the whole of the file, because the alternative has already
+ * shipped once and broke the product: in 2.21.0 the budget ADDED what else was
+ * running while the per-file costs were being LEARNED from runs that already
+ * contained that other work, so every cost was counted twice, every re-encoded
+ * step was refused, and the quality menu emptied itself down to the one copied
+ * height (fixed in 2.21.1).
+ *
+ * So: our own encoders are priced by the concurrency arithmetic, and the
+ * proxy's own work — the torrent, the hashing, the delivery — is priced per
+ * megabyte moved. Both are already in the budget. What is NOT in it is
+ * everything else the machine does: the kernel, the container runtime, whatever
+ * else the owner runs on their box. That is what is subtracted here, and
+ * nothing more.
+ */
+
+/**
+ * The share of the machine available to a new encoder.
+ *
+ * @param {object} reading - Fractions of the WHOLE machine (all cores), as
+ *   `shareOfMachine` reports them.
+ * @param {number | null} reading.systemBusy - Everything the machine is doing.
+ * @param {number | null} reading.encoderShare - Our own ffmpeg processes.
+ * @param {number | null} reading.proxyShare - The proxy process itself.
+ * @returns {{ share: number, unattributed: number, known: boolean }}
+ *   `known` is false when the host does not report its own load — then the
+ *   share is 1 and the caller must say it is uncorrected rather than pretend.
+ */
+export function availableShareFrom(reading = {}) {
+  const systemBusy = finite(reading.systemBusy);
+  if (systemBusy === null) {
+    // No reading at all: not every host has /proc. An uncorrected prediction is
+    // the honest answer, and the caller says so.
+    return { share: 1, unattributed: 0, known: false };
+  }
+  const ours = (finite(reading.encoderShare) ?? 0) + (finite(reading.proxyShare) ?? 0);
+  // Rounding, and the two readings being taken microseconds apart, can put our
+  // own share fractionally above the system total. Below zero is not a
+  // measurement of anything.
+  const unattributed = clamp(systemBusy - ours, 0, 1);
+  return { share: clamp(1 - unattributed, 0, 1), unattributed, known: true };
+}
+
+/**
+ * Apply the correction to a predicted speed.
+ *
+ * A speed is work per unit time, so a machine that can give a new encoder only
+ * `share` of itself produces `share ×` the speed the benchmark measured alone.
+ *
+ * @param {number} predictedSpeed - From the quiet-host benchmark.
+ * @param {{ share: number, known: boolean }} availability
+ * @returns {number}
+ */
+export function correctForAvailability(predictedSpeed, availability) {
+  if (!Number.isFinite(predictedSpeed) || predictedSpeed <= 0) {
+    return predictedSpeed;
+  }
+  if (!availability?.known) {
+    return predictedSpeed;
+  }
+  return predictedSpeed * availability.share;
+}
+
+/**
+ * @param {unknown} value
+ * @returns {number | null}
+ */
+function finite(value) {
+  return Number.isFinite(value) ? Number(value) : null;
+}
+
+/**
+ * @param {number} value
+ * @param {number} low
+ * @param {number} high
+ * @returns {number}
+ */
+function clamp(value, low, high) {
+  return Math.min(high, Math.max(low, value));
+}

package/services/hls-session-manager.js

@@ -20,6 +20,7 @@ import { logger } from "../utils/logger.js";
 import { readKeyframeIndex } from "./container-index/index.js";
 import { readMachineState, readProcessCpuSeconds, readProxyCpuSeconds, readSystemCpu, shareOfMachine } from "./host-load.js";
 import { speedFromReadings } from "./encoder-readings.js";
+import { availableShareFrom, correctForAvailability } from "./available-share.js";
 import {
   ENCODE_RUN_EVENT,
   ENCODE_RUN_STATE,
@@ -1902,6 +1903,12 @@ export class HlsSessionManager {
       // software hosts, else the client target). 0 = keep source.
       encodeWidth,
       encodeHeight,
+      // What the offer predicted this height would do on this machine, so the
+      // field can say what the prediction was worth once the step runs. Null
+      // when the step was never judged — a copied stream needs no encoder and
+      // is never predicted.
+      predictedSpeedWhenOffered: this.lastPredictedByHeight?.get(encodeHeight) ?? null,
+      lastPredictionRatio: null,
       // The NAME of this rung, fixed at the height that was asked for. It is
       // deliberately not the height being encoded: a viewer who picked 480p on
       // a host that then starts them at 360p, or steps down to it later, goes
@@ -3217,13 +3224,25 @@ export class HlsSessionManager {
     const proxyShare = Number.isFinite(previous.proxyCpuSeconds)
       ? (sample.proxyCpuSeconds - previous.proxyCpuSeconds) / (share.elapsedSec * cores)
       : null;
+    // Kept for the quality offer, which predicts from a benchmark taken on a
+    // QUIET host: the same reading that is printed here says how much of the
+    // machine a new encoder could actually have. Only what nobody has been
+    // charged for is subtracted — see `available-share.js`.
+    this.hostAvailability = availableShareFrom({
+      systemBusy: share.systemShare,
+      encoderShare: share.processShare,
+      proxyShare
+    });
     logger.info(
       `host-load: ffmpeg=${asPercent(share.processShare)} proxy=${asPercent(proxyShare)} ` +
       `system=${asPercent(share.systemShare)} ` +
       `iowait=${asPercent(share.iowaitShare)} cpu=${machine.megahertz === null ? "n/a" : `${machine.megahertz}MHz`} ` +
       `temp=${machine.celsius === null ? "n/a" : `${machine.celsius}C`} ` +
       `encoders=${running} running` + (suspended > 0 ? ` +${suspended} suspended` : "") +
-      ` over=${share.elapsedSec.toFixed(1)}s`
+      ` over=${share.elapsedSec.toFixed(1)}s` +
+      // What the offer will multiply a prediction by, in the same line as the
+      // readings it comes from.
+      ` available=${asPercent(this.hostAvailability.share)}`
     );
   }
 
@@ -3391,6 +3410,10 @@ export class HlsSessionManager {
     // although nothing is producing that step any more. The next reading of the
     // new encode replaces it.
     session.lastAloneSpeed = null;
+    // And so is the prediction it was compared against: it described the step
+    // this session has just left.
+    session.predictedSpeedWhenOffered = this.lastPredictedByHeight?.get(rung.height) ?? null;
+    session.lastPredictionRatio = null;
     // Priced the same way the offer and the starting rung are. Choosing the
     // preset on the encoder alone treats decoding as free, which is how the
     // check and the encode came to disagree in the first place — and here it
@@ -5444,6 +5467,25 @@ export class HlsSessionManager {
     // 0.3x reads as 3.33 s of work per second of video, more than the machine
     // has — and every other quality step was refused on the download's account.
     session.lastAloneSpeed = speed;
+    // What the offer predicted for this very step, against what it then did
+    // with the machine to itself. The prediction is corrected for the share of
+    // the machine that was free at the time, so this ratio is the error that
+    // remains AFTER that correction — which is the only way to tell whether a
+    // stage of roadmap item 3 moved anything. Written when it changes by more
+    // than a tenth, so a steady step says it once rather than every five
+    // seconds.
+    if (Number.isFinite(session.predictedSpeedWhenOffered) && session.predictedSpeedWhenOffered > 0) {
+      const ratio = speed / session.predictedSpeedWhenOffered;
+      const lastSaid = session.lastPredictionRatio;
+      if (!Number.isFinite(lastSaid) || Math.abs(ratio - lastSaid) > 0.1) {
+        session.lastPredictionRatio = ratio;
+        logger.info(
+          `prediction ${session.id.slice(0, 8)} ${session.encodeHeight || "source"}p: ` +
+          `predicted ${session.predictedSpeedWhenOffered.toFixed(2)}x, measured ${speed.toFixed(2)}x ` +
+          `(ratio ${ratio.toFixed(2)}; 1.00 would mean the arithmetic describes this machine)`
+        );
+      }
+    }
     if (kind === "audio") {
       // What is left after the work that was already accounted for. `null` when
       // the subtraction leaves nothing positive, which means the reading says
@@ -5962,6 +6004,12 @@ export class HlsSessionManager {
     const kept = [];
     /** @type {string[]} */
     const dropped = [];
+    // What each height was predicted to do on THIS machine, kept so a session
+    // started at that height can be compared against it once it runs. The
+    // manager holds the last answer, because the offer is computed on the path
+    // that serves every request while a session is created elsewhere.
+    /** @type {Map<number, number | null>} */
+    const predictedByHeight = new Map();
     for (const height of heights) {
       // The height an encoder is ALREADY producing, and the source's own height
       // when the FAMILY serves it by copy — neither has to be predicted,
@@ -6004,7 +6052,7 @@ export class HlsSessionManager {
         0,
         concurrentCostSec - (runningCostByHeight?.get(height) ?? 0)
       );
-      const { speed, sustainable } = canSustainOutput({
+      const { speed } = canSustainOutput({
         benchmark,
         decodeModel: this.decodeCostModel,
         source,
@@ -6012,11 +6060,24 @@ export class HlsSessionManager {
         observedDecodeCostSec,
         concurrentCostSec: concurrentBesideThis
       });
-      if (sustainable) {
+      // The benchmark behind that figure was taken on a QUIET host — one
+      // ffmpeg and nothing else. The machine a step will actually run on is
+      // also running the kernel, the container and whatever else its owner
+      // does, and on the addon host that was measured at 99 % busy with a
+      // quarter of it unattributed. Only the unattributed part is charged
+      // here: our own encoders are already in `concurrentBesideThis` and the
+      // proxy's own work is already priced per megabyte moved.
+      const onThisMachine = correctForAvailability(speed, this.hostAvailability);
+      // Kept against the step's own session, so that when it runs the field
+      // says what the prediction was worth. Without this the only comparison
+      // available is between two figures written minutes apart in different
+      // lines of the log.
+      predictedByHeight.set(height, onThisMachine);
+      if (onThisMachine !== null && onThisMachine >= REALTIME_SPEED_MARGIN) {
         kept.push(height);
         continue;
       }
-      dropped.push(`${height}p=${speed === null ? "n/a" : `${speed.toFixed(2)}x`}`);
+      dropped.push(`${height}p=${onThisMachine === null ? "n/a" : `${onThisMachine.toFixed(2)}x`}`);
     }
     // Written when the ANSWER changes, not when the answer is recomputed. This
     // is asked on the path that serves every playlist, init and segment, and
@@ -6026,12 +6087,19 @@ export class HlsSessionManager {
     if (dropped.length > 0) {
       const line =
         `transcode: not offering ${dropped.join(" ")} — below realtime × ${REALTIME_SPEED_MARGIN} ` +
+        // Said with the figures, because a step refused on a busy machine and
+        // one refused on an idle machine are different facts about the host.
+        (this.hostAvailability?.known
+          ? `on a machine with ${Math.round(this.hostAvailability.share * 100)}% to spare `
+          : "") +
         `(offering ${kept.map((height) => `${height}p`).join(" ")})`;
       if (line !== this.#lastOfferLine) {
         this.#lastOfferLine = line;
         logger.info(line);
       }
+      this.lastPredictedByHeight = predictedByHeight;
     } else {
+      this.lastPredictedByHeight = predictedByHeight;
       this.#lastOfferLine = "";
     }
     return kept;

package/test/available-share.test.js

@@ -0,0 +1,76 @@
+/**
+ * @file What the machine can spare for a new encoder — and what must not be
+ * charged twice.
+ *
+ * The numbers are the addon host's own, measured 2026-08-17 while a 240p step
+ * ran at 1.01-1.12x against a prediction of 1.83x.
+ */
+
+import assert from "node:assert/strict";
+import test from "node:test";
+
+import { availableShareFrom, correctForAvailability } from "../services/available-share.js";
+
+test("only the work nobody has been charged for is subtracted", () => {
+  // The field reading: the box is saturated, our encoder has more than half of
+  // it, and the proxy's own work a fifth.
+  const availability = availableShareFrom({ systemBusy: 0.99, encoderShare: 0.56, proxyShare: 0.2 });
+
+  assert.equal(availability.known, true);
+  // 0.99 − 0.76 = 0.23 belongs to the kernel, the container and whatever else
+  // the owner runs. Our own encoders are priced by the concurrency arithmetic
+  // and the proxy's work per megabyte moved; charging them here as well is the
+  // double counting that emptied the quality menu in 2.21.0.
+  assert.ok(Math.abs(availability.unattributed - 0.23) < 1e-9, `${availability.unattributed}`);
+  assert.ok(Math.abs(availability.share - 0.77) < 1e-9, `${availability.share}`);
+});
+
+test("a quiet machine is not corrected at all", () => {
+  const availability = availableShareFrom({ systemBusy: 0.05, encoderShare: 0.04, proxyShare: 0.01 });
+  assert.equal(availability.unattributed, 0);
+  assert.equal(availability.share, 1);
+  assert.equal(correctForAvailability(4, availability), 4);
+});
+
+test("a host that cannot say says so, and nothing is corrected", () => {
+  // Not every host has /proc. An uncorrected figure is honest; a figure
+  // corrected by an invented share is not.
+  const availability = availableShareFrom({ systemBusy: null, encoderShare: null, proxyShare: null });
+  assert.equal(availability.known, false);
+  assert.equal(availability.share, 1);
+  assert.equal(correctForAvailability(2.5, availability), 2.5, "an unknown machine leaves the prediction alone");
+  assert.equal(availableShareFrom().known, false);
+});
+
+test("readings that overlap by a rounding do not produce a negative machine", () => {
+  // The two samples are taken microseconds apart, so our own share can come out
+  // fractionally above the system total.
+  const availability = availableShareFrom({ systemBusy: 0.60, encoderShare: 0.58, proxyShare: 0.05 });
+  assert.equal(availability.unattributed, 0);
+  assert.equal(availability.share, 1);
+});
+
+test("a machine given over entirely to other work leaves nothing", () => {
+  const availability = availableShareFrom({ systemBusy: 1, encoderShare: 0, proxyShare: 0 });
+  assert.equal(availability.share, 0);
+  assert.equal(correctForAvailability(4, availability), 0, "a step cannot be offered on a machine with nothing left");
+});
+
+test("the correction is proportional, and only ever downwards", () => {
+  const availability = availableShareFrom({ systemBusy: 0.99, encoderShare: 0.56, proxyShare: 0.2 });
+  // The field case: a step predicted at 1.83x on a quiet box.
+  const corrected = correctForAvailability(1.83, availability);
+  assert.ok(Math.abs(corrected - 1.4091) < 0.001, `${corrected}`);
+  // And the honest note this test exists to record: 1.41x is still above the
+  // 1.01-1.12x that step actually ran at. This correction closes part of the
+  // gap, not all of it — which is why the per-step field comparison ships with
+  // it rather than after it.
+  assert.ok(corrected > 1.12, "the remaining difference is what the field check is for");
+});
+
+test("nothing is corrected without a prediction to correct", () => {
+  const availability = availableShareFrom({ systemBusy: 0.99, encoderShare: 0.5, proxyShare: 0.2 });
+  assert.equal(correctForAvailability(0, availability), 0);
+  assert.equal(correctForAvailability(Number.NaN, availability), Number.NaN);
+  assert.equal(correctForAvailability(-1, availability), -1);
+});