@torrent-tv/proxy 2.80.12 → 2.80.14

package/CHANGELOG.md

@@ -1,3 +1,22 @@
+## 2.80.14
+
+- **Fix**: What the master playlist declares a variant carries is MEASURED. It was `height * height * 3.2` — for 1080 exactly the 3 732 480 the field declared for a file carrying 18.4 Mbit/s, five times more — and the comment justifying it said no measurement existed before encoding starts. Both do: the average is the file's own length over its duration, exact at session creation, and the peak is the biggest piece produced over the span it covers, equal to the average until one exists. A re-encoded height is declared at the cap we impose; a smaller height at the pixel share of the source's rate. `BANDWIDTH` and `AVERAGE-BANDWIDTH` are both emitted, as the specification names them. This is not cosmetic: the browser sizes its cushion in BYTES from `BANDWIDTH`, so 120 s asked bought 56 MB — 26 s of that film — and the deepest it ever held was 17.1 s. One expression, no case analysis: absent measurements and absent caps are written as the identity of the operation that consumes them (`services/output/rates.js`, eight checks).
+- **Fix**: How much faster than realtime a step must run is computed from the share of the reading's time that was lost, over whole cycles. The model is unchanged and was always right; the two quantities fed into it were not — the WORST single interruption over the MEDIAN gap between interruptions, a maximum over a median, from populations that need not be the same events. Field 2026-09-08: 0.79 s over 0.01 s gave **158.60x** on a file already downloaded whole, and the quality budget refused every step against it forty times in one session. The same measurements as a share of time lost give 1.00x. The field case the old formula was written for is still caught: 3.50x refuses the 1.5 step that ran at 1.05x and stalled.
+- **Fix**: Only what a read is stopped on reaches MEMORY. `AHEAD` used to, and it is exactly the speculative lead the swarm is told about: the priority map states one claimant per zone, so a film with seven zones arrived at the piece store as five holders — one `NEAR` and four `AHEAD` — on a session with two reads. Field: `5 reader(s) want 24 piece(s) of 25 the store may hold`, so every admission had to evict a piece somebody had declared and 100 of 1395 did, beside 6565 spills and 7575 revivals in 44 minutes at a median 0.0 s on disk. Raising the allowance does not touch it — a lead stated over the rest of the film grows to fill whatever it is given.
+- **New**: The piece store NAMES its holders instead of counting them. Five read as five encoders on a session that had two, and choosing between narrowing the windows and raising the allowance was guesswork without the names.
+- **New**: Whether the page is on screen, and whether the picture was pulled out of it into picture-in-picture, travel with every viewer report and live on the viewer beside where it stands. A hidden tab has its timers throttled — 800 ms of event-loop lag measured — so it asks for nothing and looked exactly like a viewer holding a full cushion; delivery stood still for the last six minutes of the field session and nothing said the tab had gone. Two facts and not one, because picture-in-picture is watching WITH the tab hidden. A viewer who is not on screen consumes nothing, which is what a paused one already meant: no deadline, and their place in the priority map kept.
+- **Chore**: `hls-session-manager.js` is 5 lines shorter: the shape of the film the master declares — which heights splice, how big the source is, how it is packaged, what it carries — moved to `LiveOutputs.masterFactsOf`, which is the layer that holds it.
+- **Chore**: Both architecture documents carry the new arithmetic and how to read it.
+
+## 2.80.13
+
+- **New**: Whether the priority map is being served IN ITS OWN ORDER is measured, for both things that read it. Until now the map's ranks were built, published, and then thrown away by everyone who acted on them — a map read backwards would have looked identical in every line this proxy writes. The measure is one wait recorded against the rank the map gave the thing waited for at the moment it was asked for, on one scale for both layers (`services/priority/WaitLedger.js`, `SegmentDemand.rankOf`, eight checks). The encoding's own state line carries `served[now N wait(s) median Xms worst Yms, soon …, later …]` and each `[hold]` line names the band it belonged to; a band that never waited prints `none` rather than a zero, because silence is not a good result. Read: long waits at `now` mean the urgent zone is not being served first, long waits lower down with none at `now` mean the zones are the wrong width.
+- **New**: What the swarm is ACTUALLY TOLD is said out loud, on change, per level: `the swarm is told, for "film.mkv": NEAR 1 zone(s) 42MB, AHEAD 4 zone(s) 310MB, TAIL 1 zone(s) 1900MB (7 band(s) of the map, over 1218s of film)`. `applyPriorityMap` was silent — that the map had been built was visible in the encoding's line, that the download had received it was visible nowhere, and a whole field session carried not one line about it. Per level rather than per zone, because the register has five levels against the map's however-many bands and the fit between them is the one thing there that can be wrong.
+- **Fix**: A read that ends SHORTER than the `Content-Length` already sent says so, loudly. The header is committed before the first byte and the reader's `close()` ends the iteration with no accounting, so only the `fail()` path was ever logged: a client could receive a truncated body with no way to know it, and ffmpeg's mp4 demuxer — which holds the sample table and asks for samples past what arrived — then parses at wrong offsets. That is exactly the `Invalid NAL unit size (negative)` and `missing picture in access unit` seen 2138 times in one field session on a COPIED picture, where no encoder touches a frame. A single clean read of the same file through the same route produced none, and four concurrent ones produced none; what the field session also had was a piece store whose readers wanted every piece it could hold and 100 of 1395 evictions taking a piece a reader had declared.
+- **Fix**: The reads that feed ffmpeg are failed when the torrent thread dies. Both death handlers walked the ordinary reads and left `#fragmentReaders` untouched — which is what `/stream` uses for an encoder — so an encoder's input neither ended nor errored, it went quiet. Field 2026-08-31, three times: the thread died, both ffmpeg runs stayed alive and stopped producing (167 `holding segment-00085.mp4 … encoder alive` lines), and the viewer was left on a spinner that never ended. Recorded as open since then.
+- **Chore**: `hls-session-manager.js` is 2 lines shorter: the zone walk it did by hand is one call to the map that owns ranks, and the explanation of what an empty map means moved to `SegmentDemand.rankOf`, which is the thing it is about.
+- **Chore**: Both architecture documents gain the measurement and how to read it.
+
 ## 2.80.12
 
 - **Chore**: The predicate that decides whether an output is the one somebody is consuming asks each layer only for what it owns. Written first as `LiveOutputs.watchedBy(session, viewer)`, it had the output layer reading a viewer's fields — a coupling in the wrong direction, however plain the values. It is `supersededBy(session, stepOnScreen)` now, taking a plain id: which step is on somebody's screen belongs to the person, which output a step supersedes belongs to the film's shape, and the one place that legitimately holds both puts them together.

package/docs/download-architecture.md

@@ -72,6 +72,71 @@ wanted, and two thirds of reads came back from disk.
 **The background fill** states one file at a time — a soundtrack pulled whole
 once the cushion is full, and no further.
 
+## Whether the map reaches the swarm, and in what shape
+
+Applied in silence until 2026-09-08: that the map had been BUILT was visible in
+the encoding's own line, that the download had received it was visible nowhere,
+and a whole field session carried not one line about it. Said now on change, per
+LEVEL rather than per zone — the register has five levels and the map has as many
+bands as the film needs, and the fit between the two is the one thing here that
+could be wrong:
+
+```
+torrent-pool: the swarm is told, for "film.mkv": NEAR 1 zone(s) 42MB,
+AHEAD 4 zone(s) 310MB, TAIL 1 zone(s) 1900MB (7 band(s) of the map, over 1218s of film)
+```
+
+In megabytes, because that is what a swarm delivers. `NEAR` absent while
+somebody is watching means the map is not arriving; every band landing in one
+level means the fit has collapsed.
+
+Beside it, where the waits fell: the reader records each wait against the level
+the map put its piece in, and the `supply` line carries the table. Long waits at
+`blocked` mean the urgent zone is too narrow; long waits further out mean the
+lead is.
+
+## What reaches MEMORY is not what reaches the swarm
+
+`BLOCKED` and `NEAR` only. Memory holds what will be READ soon; the swarm is
+told what will be DOWNLOADED soon, and the map states the second over the whole
+rest of the film.
+
+`AHEAD` used to reach memory too, and it is exactly the speculative lead. The
+map states one claimant per zone, so a film with seven zones arrived at the piece
+store as five separate holders — one `NEAR` and four `AHEAD` — each covering tens
+of megabytes. Field 2026-09-08: `5 reader(s) want 24 piece(s) of 25 the store may
+hold (widest window 17)`, on a session with exactly two reads. The union of what
+was declared equalled the whole capacity, so every admission had to evict a piece
+somebody had declared, and 100 of 1395 evictions did; beside that, 6565 spills
+and 7575 revivals in 44 minutes with a median 0.0 s on disk.
+
+Raising the allowance does not touch it: a lead stated over the rest of the film
+grows to fill whatever memory it is given, and the ratio is unchanged.
+
+The store's own line now NAMES its holders (`[priority-map:22:0 read:…]`) rather
+than counting them, because the count read as five encoders on a session that
+had two, and choosing between "narrow the windows" and "raise the allowance" was
+guesswork without the names.
+
+## How much faster than realtime a step must run
+
+`1 / (1 - the share of the reading's time that was lost to waiting)`, over whole
+cycles: from the first interruption's start to the last one's, which holds
+exactly one running stretch per interruption in it.
+
+The model is unchanged and was always right — if a fraction `f` of the time
+delivers nothing, producing one second of film takes `1/(1 - f)` seconds. What
+was wrong were the two quantities fed into it: the WORST single interruption
+divided by the MEDIAN gap between interruptions, a maximum over a median, from
+populations that need not be the same events. It asks what would happen if the
+worst interruption recurred at the typical rate — a compound case that never
+occurs — and it divides by a gap that goes to zero whenever interruptions arrive
+in a burst.
+
+Field 2026-09-08: 0.79 s over 0.01 s gave **158.60x** on a file already
+downloaded whole, and the quality budget refused every step against it forty
+times in one session. The same measurements as a share of time lost give 1.00x.
+
 ## The five levels
 
 | level | what it is | stated |

package/docs/encode-architecture.md

@@ -136,6 +136,58 @@ one-segment backoff are gone — a second debounce on a signal the browser had
 already debounced, and every millisecond of it was dead time in front of the
 viewer.
 
+## Whether the map is being served in its own order
+
+The zones say what matters most. What the viewer actually waited for is measured
+where a viewer measurably waits — the one place in the proxy that holds a
+request for a named segment — and recorded against the rank the map gave that
+segment **at the moment it was asked for**. It reaches the `encode:` state line
+as `served[...]`:
+
+```
+served[now 42 wait(s) median 180ms worst 1900ms, soon 6 wait(s) median 90ms worst 240ms, later none]
+```
+
+Read it like this:
+
+| what it says | what is wrong |
+|---|---|
+| long waits at `now` | the urgent zone is not being served first — a fault in whoever acts on the map |
+| long waits at `soon`/`later`, none at `now` | the zones are the wrong width: the urgent one too narrow, so the viewer reaches material only ranked "soon" |
+| `now none` while the viewer is watching | nothing was ever urgent — the map is not reaching this output |
+| a band reading `none` | silence, not a zero, and it is printed as `none` so it cannot be read as "no waits, all good" |
+
+Ranks are collapsed into three bands because the map's own scale is as long as
+the film needs — a hundred ranks on a long file — and a hundred-row table says
+nothing a reader can hold. The width of `soon` is a tenth of the top rank, which
+is the map's own shape (its zones widen geometrically) rather than a threshold
+chosen for the table.
+
+The download half is measured the same way and on the same scale, so the two are
+comparable: `download-architecture.md`.
+
+## What the master playlist declares, and why it is not cosmetic
+
+`BANDWIDTH` and `AVERAGE-BANDWIDTH` per variant, both measured
+(`services/output/rates.js`):
+
+- the average is the file's own length over its duration, exact and known when
+  the session is created;
+- the peak is the biggest piece produced over the span it covers, and equals the
+  average until a piece exists — a ratio invented meanwhile would be the
+  fabrication this replaced;
+- a re-encoded height is declared at the cap we impose, which is exact;
+- a smaller height at the pixel share of the source's rate, which errs high, and
+  high is the safe direction.
+
+It used to be `height * height * 3.2`. **The browser sizes its cushion in BYTES
+from `BANDWIDTH`**, so a figure five times low makes the cushion five times
+shallow: field 2026-09-08, 3.73 Mbit/s declared for a file carrying 18.4, 120 s
+asked bought 56 MB — 26 s of that film — and the deepest the browser ever held
+was 17.1 s. Inflating it is not the answer either: hls.js compares it against
+its own estimate of the link to decide a level is unplayable, and its recovery
+then moves level by itself, which does not honour our pinning.
+
 ## What is checked
 
 `test/one-authority.test.js` holds the shape: one caller of `#startEncodeRun`,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@torrent-tv/proxy",
-  "version": "2.80.12",
+  "version": "2.80.14",
   "description": "Torrent proxy client that exposes webseed-like HTTP stream endpoint.",
   "license": "GPL-3.0-or-later",
   "publishConfig": {

package/routes/api/transcode-sessions/net-report/post.js

@@ -1,53 +1,64 @@
-/**
- * Accept a viewer link report for a transcode session (adaptive bitrate).
- * The browser measures its own data-channel throughput per segment fetch and
- * posts a rolling median + its buffered seconds; the session manager's budget
- * loop uses the latest report as the link-deficit downshift trigger.
- *
- * POST /api/transcode-sessions/:sessionId/net-report
- * Body: { linkMbps: number, bufferedAheadSec: number,
- *         consumerId?: string, positionSeconds?: number }
- *
- * `consumerId` and `positionSeconds` say WHO is reporting and WHERE they are.
- * A copied picture is one session shared by every viewer of it, so without them
- * the proxy could only act on whichever viewer reported last. Both are
- * optional: a browser that sends neither is treated exactly as before.
- *
- * Best-effort telemetry: invalid body → 400, unknown session → 404, ok → 204.
- *
- * @param {import("fastify").FastifyRequest} req
- * @param {import("fastify").FastifyReply} reply
- * @param {{ hlsSessionManager: import("../../../../services/hls-session-manager.js").HlsSessionManager }} deps
- * @returns {Promise<void>}
- */
-export async function handleApiTranscodeSessionNetReportPost(req, reply, { hlsSessionManager }) {
-  const sessionId = typeof req.params.sessionId === "string" ? req.params.sessionId : "";
-  const body = req.body && typeof req.body === "object" && !Array.isArray(req.body) ? req.body : {};
-  const linkMbps = Number(body.linkMbps);
-  const bufferedAheadSec = Number(body.bufferedAheadSec);
-  if (!sessionId || !Number.isFinite(linkMbps) || linkMbps <= 0 || !Number.isFinite(bufferedAheadSec) || bufferedAheadSec < 0) {
-    return reply.code(400).send({ error: "linkMbps (>0) and bufferedAheadSec (>=0) are required." });
-  }
-
-  // Neither is required, and neither can make a report invalid: they are what
-  // the proxy uses to tell the viewers of one session apart, and a report
-  // without them is still a truthful reading of somebody's link.
-  const consumerId = typeof body.consumerId === "string" ? body.consumerId.trim() : "";
-  // Whether the picture is moving. Absent from a page that does not say, and
-  // then the viewer counts as playing, which is what every page meant before it
-  // could say otherwise.
-  const playing = typeof body.playing === "boolean" ? body.playing : undefined;
-  const positionSeconds = Number(body.positionSeconds);
-  const recorded = hlsSessionManager.recordNetReport(sessionId, {
-    linkMbps,
-    bufferedAheadSec,
-    consumerId,
-    playing,
-    positionSeconds:
-      Number.isFinite(positionSeconds) && positionSeconds >= 0 ? positionSeconds : undefined
-  });
-  if (!recorded) {
-    return reply.code(404).send({ error: "Transcode session was not found." });
-  }
-  return reply.code(204).send();
-}
+/**
+ * Accept a viewer link report for a transcode session (adaptive bitrate).
+ * The browser measures its own data-channel throughput per segment fetch and
+ * posts a rolling median + its buffered seconds; the session manager's budget
+ * loop uses the latest report as the link-deficit downshift trigger.
+ *
+ * POST /api/transcode-sessions/:sessionId/net-report
+ * Body: { linkMbps: number, bufferedAheadSec: number,
+ *         consumerId?: string, positionSeconds?: number }
+ *
+ * `consumerId` and `positionSeconds` say WHO is reporting and WHERE they are.
+ * A copied picture is one session shared by every viewer of it, so without them
+ * the proxy could only act on whichever viewer reported last. Both are
+ * optional: a browser that sends neither is treated exactly as before.
+ *
+ * Best-effort telemetry: invalid body → 400, unknown session → 404, ok → 204.
+ *
+ * @param {import("fastify").FastifyRequest} req
+ * @param {import("fastify").FastifyReply} reply
+ * @param {{ hlsSessionManager: import("../../../../services/hls-session-manager.js").HlsSessionManager }} deps
+ * @returns {Promise<void>}
+ */
+export async function handleApiTranscodeSessionNetReportPost(req, reply, { hlsSessionManager }) {
+  const sessionId = typeof req.params.sessionId === "string" ? req.params.sessionId : "";
+  const body = req.body && typeof req.body === "object" && !Array.isArray(req.body) ? req.body : {};
+  const linkMbps = Number(body.linkMbps);
+  const bufferedAheadSec = Number(body.bufferedAheadSec);
+  if (!sessionId || !Number.isFinite(linkMbps) || linkMbps <= 0 || !Number.isFinite(bufferedAheadSec) || bufferedAheadSec < 0) {
+    return reply.code(400).send({ error: "linkMbps (>0) and bufferedAheadSec (>=0) are required." });
+  }
+
+  // Neither is required, and neither can make a report invalid: they are what
+  // the proxy uses to tell the viewers of one session apart, and a report
+  // without them is still a truthful reading of somebody's link.
+  const consumerId = typeof body.consumerId === "string" ? body.consumerId.trim() : "";
+  // Whether the picture is moving. Absent from a page that does not say, and
+  // then the viewer counts as playing, which is what every page meant before it
+  // could say otherwise.
+  const playing = typeof body.playing === "boolean" ? body.playing : undefined;
+  // Whether the page is on screen, and whether the picture was pulled out of
+  // it. A hidden tab has its timers throttled, so it asks for nothing and looks
+  // exactly like a viewer holding a full cushion; picture-in-picture is the case
+  // that makes the distinction necessary, because there the tab is hidden and
+  // the viewer is watching. Absent from a page that does not say, and then the
+  // viewer is on screen, as every page meant before it could say otherwise.
+  const onScreen = typeof body.onScreen === "boolean" ? body.onScreen : undefined;
+  const inPictureInPicture =
+    typeof body.inPictureInPicture === "boolean" ? body.inPictureInPicture : undefined;
+  const positionSeconds = Number(body.positionSeconds);
+  const recorded = hlsSessionManager.recordNetReport(sessionId, {
+    linkMbps,
+    bufferedAheadSec,
+    consumerId,
+    playing,
+    onScreen,
+    inPictureInPicture,
+    positionSeconds:
+      Number.isFinite(positionSeconds) && positionSeconds >= 0 ? positionSeconds : undefined
+  });
+  if (!recorded) {
+    return reply.code(404).send({ error: "Transcode session was not found." });
+  }
+  return reply.code(204).send();
+}