@torrent-tv/proxy 2.89.3 → 2.89.4

package/CHANGELOG.md

@@ -1,3 +1,15 @@
+## 2.89.4
+
+- **Fix**: An encoder's speed is read over its own working time. The stream route says when a run's input starts and stops waiting for bytes, each input read carries the number of the run it feeds, and a run's clock (`RunClock`) takes out that waiting and the time it was stopped. Field 2026-10-01: a copy waited 32.93 s and 43.91 s for two pieces and read 0.21x, which the start forecast then carried over the whole remaining film while it charged the same download separately. The forecast's processing rate and the learned price of copying, decoding and soundtracks now measure the machine.
+- **Fix**: Every quality step is judged on the viewer's report, with every term measured. Down: on its trend their buffer runs dry before another output could have the piece they need, and either their link carries less than the stream or this machine makes the picture slower than realtime over its own working time; a shortfall of the swarm is not answered with a smaller picture, which reads the same input. Up: the machine makes the picture at realtime or faster, their buffer is not draining and holds the time another output takes to be ready, and their link carries the next height. Gone: the 0.95x threshold and fifteen-second window of slowness, the sixty-second window before a step back up, the 80 % share of a measured link (`LINK_SAFETY`), and the download-bound test built on `fileProgress >= 0.999` and a download rate against the file's byte rate. While a viewer waits for playback to start nothing steps down for the machine: their buffer is not draining.
+- **Fix**: An encoder's price and the forecast's processing rate are learned each time it closes a piece, on every host. The five-second timer only samples the host's load and the torrents' download rates; hardware hosts used to learn nothing, because the timer stopped before its learning part there.
+- **Fix**: The stream an output sends is measured over every finished piece, each over its own length on the cut table, from the sizes the segment store already holds. It used to be the last five pieces but the newest, timed as if every piece were one length — wrong for a copy, which is cut at the source's keyframes — and read from the disk on every report.
+- **Fix**: A request to move a viewer's quality stands for as long as the conditions it was asked under hold, not for a chosen 45 s. Each judgement of the viewer's report asks for what it needs again; a request it does not repeat — their buffer stopped draining, the room for a step up went, the smaller picture is no longer smaller — is let go and said once.
+- **Fix**: The speed a file's supply asks for is no longer cut down at a lost share of 0.99. The share is strictly below one wherever a figure is given — at least one gap of delivery lies inside the span — so the speed is finite; a supply that delivered for a thousandth of the time now asks a thousand times realtime instead of a hundred, which no re-encode meets and which leaves the copy.
+- **Fix**: An encoder's speed has one owner, the run. It measures itself between two closed pieces over its own working time and announces each reading once; the plan, the quality budget, the playback forecast and the learned price all read that one figure. The plan and the orchestrator used ffmpeg's cumulative `speed=` instead, which divides by every second the encoder was stopped or its input starved, so a slow swarm read as a slow machine and the plan bought encoders that read the same input. Gone with it: the orchestrator's per-output copy of the fastest speed (which nothing filled any more), the quality state's copy of the last reading, and the `speed` field of a progress report, which no page reads.
+- **Chore**: Whether a closed piece holds the whole of its stretch is decided by the encoding (`encode/piece-completeness.js`), which knows the cut table: `EncodeRuns` judges a closed file before it is published and a stored piece left by an earlier process when the forecast reads it. The segment store keeps the bytes and the media intervals read from them, and takes a piece off the disk only at the encoding's word. Behaviour is unchanged. The store used to decide it, which made it read the cut table, a fact of the encoding.
+- **Chore**: The three-second shortest window a speed was read over is gone: a speed is the difference of two samples taken at the encoder's own progress reports. So is the check that kept a starved reading out of the learned price, which judged starvation by two chosen numbers (`fileProgress >= 0.999` and a download rate against the file's byte rate); the waiting is now subtracted instead of guessed at.
+
 ## 2.89.3
 
 - **Chore**: The same release as 2.89.2, which the registry accepted as a staged version and never published (`409 Cannot publish over previously staged version` with no staged version listed).

package/docs/encode-architecture.md

@@ -295,20 +295,31 @@ The page has no quality control: the quality is always automatic. What moves a
 viewer is decided here, per viewer, and each part of it is a measurement.
 
 1. **Judged on the viewer's report, not on a timer.** `QualityController.noteViewerReported`
-   runs from the net-report route. The chosen fifteen-second window of a slow
-   link, the ten-second buffer threshold and the thirty-second wait after every
-   action are gone.
-2. **Down when the buffer would run dry first.** Their link does not carry the
-   stream they are given AND, on the buffer's trend, it ends sooner than another
-   output could close the piece they need, counting the time until their next
-   report (`quality/drain-threat.js`). The trend is a least-squares fit over the
-   shortest run of their reports spanning one segment
+   runs from the net-report route, for the step down, the picture they see and
+   the step up alike. The chosen fifteen-second window of a slow link or a slow
+   machine, the 0.95x threshold of slowness, the sixty-second window before a
+   step back up, the ten-second buffer threshold, the thirty-second wait after
+   every action and the 80 % share of the link are gone. The five-second timer
+   only samples the host's load and the torrents' download rates; an encoder's
+   price is learned when it closes a piece.
+2. **Down when the buffer would run dry first, for a reason a smaller output
+   removes.** On the buffer's trend it ends sooner than another output could
+   close the piece they need, counting the time until their next report
+   (`quality/drain-threat.js`), AND either their link carries less than the
+   stream they are given, or this machine makes the picture slower than
+   realtime over its run's own working time (`encode/RunClock.js`: the time its
+   input waited for the swarm and the time it was stopped are taken out). A
+   threat neither explains — the swarm is short — is not answered with a
+   smaller picture, which reads the same input. The trend is a least-squares fit
+   over the shortest run of their reports spanning one segment
    (`viewer/buffer-trend.js`), because the buffer rises a segment at a time and
-   two readings catch only the rise or the fall. The time to readiness is this
-   host's measured first-segment time. The levers: a lower limit of the same
-   height, then a lower height asked of the player as URGENT — their page
-   switches as soon as the rung is ready, without waiting for a cushion. Where
-   neither can be prepared they stay on what they are given, with no message.
+   two readings catch only the rise or the fall. The time to readiness is the
+   observed preparation time of another output of this mode, else this host's
+   measured first-segment time. The levers: for the link, a lower limit of the
+   same height, then a lower height; for the machine, a lower height. A lower
+   height is asked of the player as URGENT — their page switches as soon as the
+   rung is ready, without waiting for a cushion. Where nothing can be prepared
+   they stay on what they are given, with no message.
 3. **The picture the viewer sees bounds a re-encode.** The page sends the
    frame it would show without enlarging, in physical pixels; it is kept on
    `Viewer.visiblePicture`. The bound is the smallest rung of the source's
@@ -319,12 +330,21 @@ viewer is decided here, per viewer, and each part of it is a measurement.
 4. **A smaller picture moves the viewer only onto a READY rung**
    (`Renditions.heightReadyFor`: the piece they ask for next closed on it);
    with none ready, the next judgement of their link or of the machine applies
-   the bound. **A larger picture** asks one rung up when the machine and their
-   link have room.
-5. **A step up is let go when its conditions go back**: a draining buffer, a
-   link that no longer carries the stream, or no room for the output. The page
-   drops the move it was preparing when the proxy stops asking for it.
-6. **The page switches a variant when the rung is ready**, and for a request
+   the bound.
+5. **Up one rung when there is room, every term measured**: the picture on
+   their screen is made at least at realtime over its run's own working time (a
+   copy is not limited by an encoder), their buffer is not draining and holds
+   at least the time another output takes to be ready, their link carries the
+   next height, and the picture they see is not already served. A rung this
+   host has been measured failing at is not offered, so the step back up cannot
+   return to it; no window has to pass.
+6. **A request stands for as long as its conditions hold**: each judgement
+   of the viewer's report asks for what it needs again, and a request it does
+   not repeat is let go — a draining buffer for a step up, a buffer that no
+   longer drains for a step down, no room for the output, a picture that is no
+   longer smaller. No chosen time ends a request. The page drops the move it
+   was preparing when the proxy stops asking for it.
+7. **The page switches a variant when the rung is ready**, and for a request
    that is not urgent only once it holds `minimumBufferSeconds`.
 
 ## A place on the machine
@@ -620,6 +640,22 @@ different figures, none of which reproduced the answer the plan had given.
 A zero in `firstByte`, `kill` or `refetch` is a measurement nobody has taken, not
 a free operation. It is printed so that reading it as free is a choice.
 
+**`speed` has one owner: the run.** `EncodeRun.speedX` is the film a run made
+between two closed pieces over its own working time (`encode/RunClock.js`), so
+the time its input waited for the swarm and the time it was stopped are not in
+it. The run announces each reading once (`onSpeedMeasured`); `EncodeCost` learns
+the price of the output from it, the quality budget and the playback forecast
+read the newest reading of the output's runs, and the plan reads the runs
+themselves. Before a run has closed two pieces, and after a restart, the plan
+uses `startingSpeedFor`: what this output was last measured doing alone, or
+what the startup measurements predict. Nothing keeps a copy across restarts,
+and ffmpeg's cumulative `speed=` is read by nothing: it divides by every second
+the encoder was stopped or starved.
+
+What the swarm delivers is not in this figure, by construction. It is the
+separate supply term `refetch`, and while that is unmeasured (printed as zero)
+the plan's arrivals assume the input keeps up.
+
 It was missing, and its absence cost three wrong diagnoses of one field session.
 The line printed the windows, the budget and where the live runs stood; the
 intervals the actions carried were the one thing it did not print. What that

package/docs/playback-start-forecast.md

@@ -14,7 +14,11 @@ Each output's remaining processing work subtracts progress only within its
 actual live run intervals. Initial video rates use existing codec/size/encoder
 calibration. Initial audio rates use short packaged per-codec measurements and
 sample work (encoding) or encoded bytes (copying). Actual output progress
-replaces these initial rates. Startup is charged once, separately from processing
+replaces these initial rates. That rate is the run's processing speed: film
+made over the run's own working time, with the time its input waited for the
+swarm and the time it was stopped taken out (`RunClock`, fed by the stream
+route, which marks each wait of a run's input read). The download is charged
+once, by the source service, and not a second time inside the processing rate. Startup is charged once, separately from processing
 throughput, and disappears once actual run progress establishes processing.
 
 Input arrival and processing overlap. With arrival time D(m), processing rate
@@ -84,7 +88,11 @@ Seconds appear again only for rates, schedules and the answer.
 A closed non-final fragment is reusable only if the end of its last frame, on
 every track, reaches its next declared cut less `SEGMENT_CUT_TIME_DELTA_SECONDS`,
 the delta the muxer is configured with. That is a fact of production, so no
-browser rule enters it. Interrupted fragments are excluded on publication and
-when reading an existing cache, so the production schedule treats their media
-as unfinished work. The final fragment uses its actual media end rather than
-the container's approximate duration.
+browser rule enters it, and the encoding decides it
+(`encode/piece-completeness.js`): `EncodeRuns` judges a closed file before it is
+published, and judges a stored piece left by an earlier process when the
+forecast asks for it, taking a short one off the disk. The segment store keeps
+the bytes and the media intervals read from them and judges nothing. Interrupted
+fragments are therefore never served or counted, so the production schedule
+treats their media as unfinished work. The final fragment uses its actual media
+end rather than the container's approximate duration.

package/docs/storage-architecture.md

@@ -167,6 +167,12 @@ finished pieces inside the ENCODING layer was the encoder owning its own disk.
 It did not move into the room's own directory either: it is a CLAIMANT, and a
 claimant inside the owner of the resource is the same fault one floor down.
 
+The store keeps a produced piece's bytes and the media intervals read from them;
+it does not judge whether a piece is whole. That is a fact of production, read
+against the output's cut table, and the encoding decides it
+(`encode/piece-completeness.js`): before a piece is published, and for a stored
+piece left by an earlier process, which it asks the store to `remove`.
+
 **Four directories, deliberately.** They share one property — bytes on a medium
 with a limit — and that is what makes them one layer. What they are is three
 different things: a torrent piece is born when it is downloaded and addressed by

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@torrent-tv/proxy",
-  "version": "2.89.3",
+  "version": "2.89.4",
   "description": "Torrent proxy client that exposes webseed-like HTTP stream endpoint.",
   "license": "GPL-3.0-or-later",
   "publishConfig": {

package/routes/stream/get.js

@@ -10,6 +10,37 @@ import { open } from "node:fs/promises";
 import { parseRange } from "../../utils/parse-range.js";
 import { logger } from "../../utils/logger.js";
 
+/**
+ * The same fragments, with every wait for the next one marked: `mark(true)`
+ * before it is asked for and `mark(false)` when it has come. The time between
+ * is time the encoder reading this response waits for its input.
+ *
+ * @template T
+ * @param {AsyncIterable<T>} source
+ * @param {(waiting: boolean) => void} mark
+ * @returns {AsyncGenerator<T>}
+ */
+async function* markingWaits(source, mark) {
+  const iterator = source[Symbol.asyncIterator]();
+  try {
+    while (true) {
+      mark(true);
+      let step;
+      try {
+        step = await iterator.next();
+      } finally {
+        mark(false);
+      }
+      if (step.done) {
+        return;
+      }
+      yield step.value;
+    }
+  } finally {
+    await iterator.return?.();
+  }
+}
+
 /**
  * Resolve source parameters from the query string.
  * Prefers a registered `sourceKey`; falls back to inline `sourceType`+`source`.
@@ -138,7 +169,7 @@ function serveWholeFile(req, reply, file, handle) {
   return reply.send(handle.createReadStream({ start, end }));
 }
 
-export async function handleStreamGet(req, reply, { sourceRegistry, torrentPool, noteInputBytes = null }) {
+export async function handleStreamGet(req, reply, { sourceRegistry, torrentPool, noteInputBytes = null, noteInputWaiting = null }) {
   const fileIndexRaw = typeof req.query.fileIndex === "string" ? req.query.fileIndex : "";
   const fileIndex = Number(fileIndexRaw);
   const { sourceType, source, sourceKey } = getSourceParams(req.query, sourceRegistry);
@@ -265,6 +296,11 @@ export async function handleStreamGet(req, reply, { sourceRegistry, torrentPool,
   // file — and two sessions can read one file, so the file cannot stand in for
   // the session.
   const sessionId = typeof req.query.session === "string" ? req.query.session : "";
+  // Which run of that output, so the time its input waits is that run's.
+  const runToken = Number(req.query.run);
+  const markWaits = noteInputWaiting && Number.isInteger(runToken)
+    ? (waiting) => noteInputWaiting(runToken, waiting)
+    : null;
 
   const fragments = typeof file.createFragmentReader === "function"
     ? file.createFragmentReader({ start, end, windowBytes })
@@ -286,7 +322,7 @@ export async function handleStreamGet(req, reply, { sourceRegistry, torrentPool,
 
     let sent = 0;
     try {
-      for await (const fragment of fragments) {
+      for await (const fragment of markWaits ? markingWaits(fragments, markWaits) : fragments) {
         if (reply.raw.writableEnded || reply.raw.destroyed) {
           fragment.release();
           break;