@torrent-tv/proxy 2.87.0 → 2.87.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +4 -0
- package/CLAUDE.md +8 -4
- package/docs/browser-proxy-contract.md +94 -0
- package/package.json +1 -1
- package/routes/api/transcode-sessions/post.js +1 -1
- package/routes/transcode/audio-file/get.js +1 -1
- package/routes/transcode/audio-warm/get.js +1 -1
- package/routes/transcode/session-file/get.js +6 -383
- package/routes/transcode/variant-file/get.js +2 -3
- package/routes/transcode/variant-warm/get.js +2 -1
- package/services/encode/EncodeOrchestrator.js +23 -13
- package/services/encode/EncodeRuns.js +15 -5
- package/services/encode/OutputOpening.js +16 -6
- package/services/encode/quality/QualityController.js +6 -2
- package/services/server/http-responses.js +60 -0
- package/services/server/transcode-session-files.js +325 -0
- package/services/server/wire-outputs.js +29 -4
- package/test/coverage-follows-the-disk.test.js +6 -1
- package/test/encoded-output.test.js +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,7 @@
|
|
|
1
|
+
## 2.87.1
|
|
2
|
+
|
|
3
|
+
- **Chore**: Keep HTTP route modules independent by moving shared response and transcode-file handling into server services. Pass encoding components narrow segment-storage interfaces and document the browser-facing HTTP contract.
|
|
4
|
+
|
|
1
5
|
## 2.87.0
|
|
2
6
|
|
|
3
7
|
- **New**: Each proxy qualifies its selectable software and detected hardware encoder modes at startup with production arguments and independent segment decoding (`encode/calibration.js`). The configuration key includes the encoder build and machine or device details; speed is measured at five frame sizes, interpolated only within that range, with the measured error taken off. Software remains qualified for runtime fallback, and a device with no qualified mode uses software.
|
package/CLAUDE.md
CHANGED
|
@@ -14,12 +14,16 @@ Linux-only host (e.g. POSIX-only signals must degrade elsewhere).
|
|
|
14
14
|
|
|
15
15
|
## Layout
|
|
16
16
|
|
|
17
|
+
Browser request and response fields, compatibility defaults, and status meanings
|
|
18
|
+
are recorded in `docs/browser-proxy-contract.md`.
|
|
19
|
+
|
|
17
20
|
- `bin/cli.js` — CLI entry; resolves `ffmpegBin` (uses `--ffmpeg-bin` if given,
|
|
18
21
|
else bundled ffmpeg-static, else PATH `ffmpeg`).
|
|
19
|
-
- `server.js` —
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
22
|
+
- `server.js` — the composition root: creates Fastify, initializes process-wide
|
|
23
|
+
resources, calls `services/server/wire-outputs.js` for the output and encode
|
|
24
|
+
service graph, registers each route explicitly, and passes its dependencies.
|
|
25
|
+
`wire-outputs.js` is a subordinate assembler for that graph; it does not
|
|
26
|
+
create the HTTP server or register routes.
|
|
23
27
|
- `routes/<path>/<method>.js` — same convention as the server repo.
|
|
24
28
|
- `routes/stream/get.js` — byte-range torrent file streaming (HTTP 206).
|
|
25
29
|
- `routes/api/playback-plan/post.js` — codec/container/duration probe result.
|
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
# Browser–proxy playback contract
|
|
2
|
+
|
|
3
|
+
This document records the HTTP contract used by the browser playback code and
|
|
4
|
+
implemented by this proxy. The same paths and payloads pass through either the
|
|
5
|
+
direct HTTP transport or the WebRTC data channel. It is a description of the
|
|
6
|
+
current code, not a generated schema or a version negotiation protocol.
|
|
7
|
+
|
|
8
|
+
## Source and file discovery
|
|
9
|
+
|
|
10
|
+
| Request | Required input | Successful answer |
|
|
11
|
+
|---|---|---|
|
|
12
|
+
| `POST /api/sources` | `{ sourceType, source }` | `{ sourceKey }` identifies the registered source. |
|
|
13
|
+
| `GET /api/sources/:sourceKey/files?maxWaitMs=N` | Registered `sourceKey`; optional wait budget | While magnet metadata is unavailable: `{ pending: true }`. When ready: `{ name, infoHash, files, items }`. `files` carries `fileIndex`, `name`, `relativePath`, `length`, and `kind`; `items` maps each video `fileIndex` to audio, subtitle, and image file indices. |
|
|
14
|
+
| `GET /api/sources/:sourceKey/stats?fileIndex=N` | Registered source; optional file index | Current peer, transfer, header, and file progress readings. A reading may not contain every measurement; absent or `null` measurements are unknown. |
|
|
15
|
+
| `POST /api/sources/:sourceKey/warm` | Optional `{ fileIndex, positionSeconds }`; an empty object warms source metadata and the first video | `{ started, swarm, edges, sidecars }` reports which preparation started. Work continues in the background. |
|
|
16
|
+
| `GET /api/link-probe?bytes=N` | Positive byte count | A no-store binary body of zero bytes, capped at 2 MiB. The browser measures completed transfer time; an unavailable route leaves link speed unknown. |
|
|
17
|
+
|
|
18
|
+
`pending: true` means the requested metadata or plan is not ready yet. The
|
|
19
|
+
browser polls again. It does not mean that the torrent has no video or that the
|
|
20
|
+
request failed. A non-pending file-list answer uses the proxy's file indices;
|
|
21
|
+
the browser must keep those indices when opening a file.
|
|
22
|
+
|
|
23
|
+
## Playback plan and transcode session
|
|
24
|
+
|
|
25
|
+
`POST /api/playback-plan` takes `{ sourceKey, fileIndex, userAgent }`. Its answer
|
|
26
|
+
contains `mode`, `directUrl`, codec and container names, duration and source
|
|
27
|
+
dimensions, track inventories, paired sidecar files, `offeredHeights`, and
|
|
28
|
+
`pending`. The browser uses the codec fields to make the direct-versus-transcode
|
|
29
|
+
decision itself; `mode` is advisory.
|
|
30
|
+
|
|
31
|
+
`sidecarSubtitles` lists subtitle files the proxy paired with this video. Each
|
|
32
|
+
entry carries the torrent `fileIndex` and the proxy's filename-derived language
|
|
33
|
+
and release metadata. The browser uses these entries to fetch subtitle files
|
|
34
|
+
and to identify a remembered choice. `sidecarImages` is returned beside it but
|
|
35
|
+
is not needed for playback. Older proxies and plans made with transcoding
|
|
36
|
+
disabled may omit these fields; the browser treats a missing or non-array list
|
|
37
|
+
as empty.
|
|
38
|
+
|
|
39
|
+
The browser maps a missing `mode` to `direct`, uses its registered `/stream`
|
|
40
|
+
URL when `directUrl` is absent, maps missing codec/container names to empty
|
|
41
|
+
strings, maps missing duration or dimensions to `0`, and maps missing audio or
|
|
42
|
+
subtitle track arrays to `[]`. It maps missing or non-object `offeredHeights` to
|
|
43
|
+
`null`; for an object value, missing or invalid `copy` and `transcode` lists
|
|
44
|
+
become empty arrays. `null` means no offer was supplied, while empty lists
|
|
45
|
+
explicitly offer no heights for their respective playback branch. Only
|
|
46
|
+
`pending: true` asks the browser to poll again.
|
|
47
|
+
|
|
48
|
+
`POST /api/transcode-sessions` requires `sourceKey`, `fileIndex`, and
|
|
49
|
+
`consumerId`. The browser also sends the requested audio/video transcode flags,
|
|
50
|
+
the visible picture size, whether audio is requested as separate renditions,
|
|
51
|
+
the start position, selected audio track, and segment format when known. The
|
|
52
|
+
answer contains `sessionId`, `playlistPath`, `offeredHeights`, declared `tracks`,
|
|
53
|
+
and `lookaheadSeconds`. `masterPath` and `variantHeight` are present only when
|
|
54
|
+
the session has quality variants.
|
|
55
|
+
|
|
56
|
+
The browser interprets `409` with `outcome: "output-unavailable"` as a refusal
|
|
57
|
+
for this viewer's link and `outcome: "no-capacity"` as a proxy capacity
|
|
58
|
+
refusal. Other non-success statuses remain errors; an error body only adds
|
|
59
|
+
display detail and is not required for status handling.
|
|
60
|
+
|
|
61
|
+
## Session control and media files
|
|
62
|
+
|
|
63
|
+
| Request | Meaning |
|
|
64
|
+
|---|---|
|
|
65
|
+
| `POST /api/transcode-sessions/:id/seek` | `{ positionSeconds, consumerId, generation }` reports the viewer's seek. A successful answer is `204`; the proxy handles the seek server side. |
|
|
66
|
+
| `GET /api/transcode-sessions/:id/progress?consumer=:id` | Reads current session progress and also keeps the viewer's session alive. The browser consumes available progress fields; an unreadable or failed answer does not imply that playback has ended. |
|
|
67
|
+
| `POST /api/transcode-sessions/:id/net-report` | `{ consumerId, bufferedAheadSec, ... }` reports the viewer's buffer and optional state/link measurements. `consumerId` and non-negative `bufferedAheadSec` are required; missing `linkMbps` does not reject the report and leaves the last link reading intact. Missing `playing` becomes `false`; missing `waiting` is derived from the playback and buffer state. Missing `positionSeconds` leaves the position unchanged, missing `qualityMode` leaves the prior mode, `onScreen` defaults to `true`, and `inPictureInPicture` defaults to `false`. Success is `204`. |
|
|
68
|
+
| `POST /api/transcode-sessions/:id/fragment-far` | Diagnostic fragment and buffer positions; does not change encoding. Best effort, success `204`. |
|
|
69
|
+
| `POST /api/transcode-sessions/:id/release` | `{ consumerId, reason }` releases this viewer's session assignment. |
|
|
70
|
+
| `GET /transcode/:id/:fileName` | HLS playlist, init, or media segment. A file that is still being produced is answered with retryable `503`, not `202`. |
|
|
71
|
+
| `GET /transcode/:id/v/:height/warm` and `/transcode/:id/a/:track/warm` | Prepare the requested video height or audio track at a position. `204` means ready. Audio warm-up `404` means the proxy does not support this operation for that session. |
|
|
72
|
+
| `GET /stream?sourceKey=…&fileIndex=N` | Direct file bytes with HTTP range support. |
|
|
73
|
+
|
|
74
|
+
## Subtitles
|
|
75
|
+
|
|
76
|
+
`GET /api/subtitles` requires `sourceKey` and `fileIndex`. Without `trackIndex`
|
|
77
|
+
it reads a paired sidecar file. With `trackIndex`, it reads cues from an
|
|
78
|
+
embedded text track and may take `consumerId` to subscribe that viewer to
|
|
79
|
+
subsequent cue pushes. `since` asks for cues after a previously received
|
|
80
|
+
cursor.
|
|
81
|
+
|
|
82
|
+
`200` returns WebVTT. Embedded cue answers may include
|
|
83
|
+
`X-Subtitle-Cursor`, `X-Subtitle-Covered-Clusters`, and
|
|
84
|
+
`X-Subtitle-Indexed-Clusters`; language detection may include
|
|
85
|
+
`X-Subtitle-Language` and `X-Subtitle-Language-Name`. While an embedded track
|
|
86
|
+
is being extracted, the response is `202` with `{ pending: true }`; the browser
|
|
87
|
+
waits and repeats the request. A missing cursor or language header means that
|
|
88
|
+
the browser has no value to preserve or display yet. A failed sidecar or
|
|
89
|
+
embedded-track request skips that track without failing video playback.
|
|
90
|
+
|
|
91
|
+
Subtitle cue pushes are delivered through the established proxy connection,
|
|
92
|
+
not by a separate browser polling route. After reconnect, the browser repeats
|
|
93
|
+
the embedded-track request with `since` to restore the subscription and receive
|
|
94
|
+
missed cues.
|
package/package.json
CHANGED
|
@@ -11,7 +11,7 @@
|
|
|
11
11
|
|
|
12
12
|
import { logger } from "../../../utils/logger.js";
|
|
13
13
|
import { OUTPUT_NO_CAPACITY, OUTPUT_UNAVAILABLE } from "../../../services/encode/output/index.js";
|
|
14
|
-
import { replyNoCapacity, replyOutputUnavailable } from "
|
|
14
|
+
import { replyNoCapacity, replyOutputUnavailable } from "../../../services/server/http-responses.js";
|
|
15
15
|
|
|
16
16
|
/**
|
|
17
17
|
* Extract a plain object from the request body, guarding against
|
|
@@ -12,7 +12,7 @@
|
|
|
12
12
|
* instead of it: both encoders run, one for the rung and one for the audio.
|
|
13
13
|
*/
|
|
14
14
|
|
|
15
|
-
import { consumerOf, refusedAsStale, serveSessionFile } from "
|
|
15
|
+
import { consumerOf, refusedAsStale, serveSessionFile } from "../../../services/server/transcode-session-files.js";
|
|
16
16
|
|
|
17
17
|
/**
|
|
18
18
|
* @param {import("fastify").FastifyRequest} req
|
|
@@ -12,7 +12,7 @@
|
|
|
12
12
|
* switch into bytes that already exist; 503 while it is still being made.
|
|
13
13
|
*/
|
|
14
14
|
|
|
15
|
-
import { waitForSessionFile } from "
|
|
15
|
+
import { waitForSessionFile } from "../../../services/server/transcode-session-files.js";
|
|
16
16
|
|
|
17
17
|
/** How long to hold the request before telling the caller to retry. */
|
|
18
18
|
const WARM_WAIT_MS = 12_000;
|
|
@@ -1,5 +1,9 @@
|
|
|
1
|
-
import {
|
|
2
|
-
|
|
1
|
+
import {
|
|
2
|
+
consumerOf,
|
|
3
|
+
refusedAsStale,
|
|
4
|
+
serveSessionFile,
|
|
5
|
+
statedGenerationOf
|
|
6
|
+
} from "../../../services/server/transcode-session-files.js";
|
|
3
7
|
|
|
4
8
|
/**
|
|
5
9
|
* How long a request for a file not yet produced is held: AS LONG AS THE
|
|
@@ -13,7 +17,6 @@ import { bandOf, waits } from "../../../services/viewer/WaitLedger.js";
|
|
|
13
17
|
* this route answers by then. A requester that states nothing is held until it
|
|
14
18
|
* closes the connection, which every HTTP client does when it stops waiting.
|
|
15
19
|
*/
|
|
16
|
-
const HOLD_HEADER = "x-hold-ms";
|
|
17
20
|
|
|
18
21
|
/**
|
|
19
22
|
* Serve HLS playlist and segment files from an active transcode session.
|
|
@@ -47,383 +50,3 @@ export async function handleTranscodeSessionFileGet(req, reply, { serving, viewe
|
|
|
47
50
|
);
|
|
48
51
|
return serveSessionFile(req, reply, { serving, viewerRequests, sessionId, fileName });
|
|
49
52
|
}
|
|
50
|
-
|
|
51
|
-
/**
|
|
52
|
-
* The answer given when no output suits this viewer: nothing this proxy holds
|
|
53
|
-
* or could produce is admitted by their link. Machine-readable, because the
|
|
54
|
-
* page has to explain it to the viewer and offer what they can do; the
|
|
55
|
-
* figures are the ones the decision was made on.
|
|
56
|
-
*
|
|
57
|
-
* 409 and not a 5xx: nothing is wrong with the proxy, and repeating the same
|
|
58
|
-
* request against the same link gives the same answer.
|
|
59
|
-
*
|
|
60
|
-
* @param {import("fastify").FastifyReply} reply
|
|
61
|
-
* @param {{ reason: string, figures: object | null, wantedKey?: string }} details
|
|
62
|
-
* @returns {import("fastify").FastifyReply}
|
|
63
|
-
*/
|
|
64
|
-
export function replyOutputUnavailable(reply, details) {
|
|
65
|
-
return reply.code(409).send({
|
|
66
|
-
error: `No output suits this viewer: ${details?.reason ?? "unknown"}.`,
|
|
67
|
-
outcome: "output-unavailable",
|
|
68
|
-
reason: details?.reason ?? "",
|
|
69
|
-
figures: details?.figures ?? null
|
|
70
|
-
});
|
|
71
|
-
}
|
|
72
|
-
|
|
73
|
-
/**
|
|
74
|
-
* The answer given when this machine cannot take the output asked for: nothing
|
|
75
|
-
* it has shown it can encode fits, or it has no place for one more encoder.
|
|
76
|
-
* The page answers it by asking the rest of the pool before anything plays.
|
|
77
|
-
*
|
|
78
|
-
* @param {import("fastify").FastifyReply} reply
|
|
79
|
-
* @param {{ reason: string, figures?: object | null }} details
|
|
80
|
-
* @returns {import("fastify").FastifyReply}
|
|
81
|
-
*/
|
|
82
|
-
export function replyNoCapacity(reply, details) {
|
|
83
|
-
return reply.code(409).send({
|
|
84
|
-
error: `This proxy cannot take this video now: ${details?.reason ?? "unknown"}.`,
|
|
85
|
-
outcome: "no-capacity",
|
|
86
|
-
reason: details?.reason ?? "",
|
|
87
|
-
figures: details?.figures ?? null
|
|
88
|
-
});
|
|
89
|
-
}
|
|
90
|
-
|
|
91
|
-
/**
|
|
92
|
-
* The answer given when an address this viewer was already given cannot be
|
|
93
|
-
* given again without risking a piece under a header it may not match. The
|
|
94
|
-
* page answers it by starting a new viewing where the picture is — which is
|
|
95
|
-
* why it is not a "retry": the same request would be answered the same way.
|
|
96
|
-
*
|
|
97
|
-
* @param {import("fastify").FastifyReply} reply
|
|
98
|
-
* @param {{ reason: string }} details
|
|
99
|
-
* @returns {import("fastify").FastifyReply}
|
|
100
|
-
*/
|
|
101
|
-
export function replyAssignmentLost(reply, details) {
|
|
102
|
-
return reply.code(409).send({
|
|
103
|
-
error: `This part was answered by an output that has gone: ${details?.reason ?? "unknown"}.`,
|
|
104
|
-
outcome: "assignment-lost",
|
|
105
|
-
reason: details?.reason ?? ""
|
|
106
|
-
});
|
|
107
|
-
}
|
|
108
|
-
|
|
109
|
-
/**
|
|
110
|
-
* Which viewer is asking, or an empty string where the transport names nobody.
|
|
111
|
-
*
|
|
112
|
-
* @param {import("fastify").FastifyRequest} req
|
|
113
|
-
* @returns {string}
|
|
114
|
-
*/
|
|
115
|
-
export function consumerOf(req) {
|
|
116
|
-
return typeof req.query?.consumer === "string" ? req.query.consumer : "";
|
|
117
|
-
}
|
|
118
|
-
|
|
119
|
-
/**
|
|
120
|
-
* The generation the request says it was made in, or NaN when it says none.
|
|
121
|
-
*
|
|
122
|
-
* STAMPED BY THE PAGE when the request was SENT (`webrtc-hls-loader.js`), not
|
|
123
|
-
* when it arrived: a request made before a seek and delivered after it still
|
|
124
|
-
* says it belongs to the viewing that was left. Only a stated non-negative
|
|
125
|
-
* integer counts; anything else is "nothing stated".
|
|
126
|
-
*
|
|
127
|
-
* @param {import("fastify").FastifyRequest} req
|
|
128
|
-
* @returns {number}
|
|
129
|
-
*/
|
|
130
|
-
export function statedGenerationOf(req) {
|
|
131
|
-
const raw = req.query?.generation;
|
|
132
|
-
if (typeof raw !== "string" || !/^\d+$/.test(raw)) {
|
|
133
|
-
return Number.NaN;
|
|
134
|
-
}
|
|
135
|
-
return Number(raw);
|
|
136
|
-
}
|
|
137
|
-
|
|
138
|
-
/**
|
|
139
|
-
* Answer at once, and do nothing else, when the request belongs to a viewing
|
|
140
|
-
* its viewer has left and whose window for new requests has passed.
|
|
141
|
-
*
|
|
142
|
-
* BEFORE ANY OTHER STEP OF THE ROUTE. Resolving a quality step or a soundtrack
|
|
143
|
-
* can create an output and registers the viewer on it; a stale request that got
|
|
144
|
-
* that far would leave both behind and be refused only afterwards. The answer
|
|
145
|
-
* is the same one a request made pointless by a seek already gets — 503 with
|
|
146
|
-
* `Retry-After: 0` — because it is the same situation seen from the other end.
|
|
147
|
-
*
|
|
148
|
-
* @param {import("fastify").FastifyRequest} req
|
|
149
|
-
* @param {import("fastify").FastifyReply} reply
|
|
150
|
-
* @param {{ acceptsRequest: Function }} viewerRequests
|
|
151
|
-
* @param {string} fileName - For the log line.
|
|
152
|
-
* @returns {boolean} Whether the request was answered here.
|
|
153
|
-
*/
|
|
154
|
-
export function refusedAsStale(req, reply, viewerRequests, fileName) {
|
|
155
|
-
const consumerId = consumerOf(req);
|
|
156
|
-
const stated = statedGenerationOf(req);
|
|
157
|
-
if (viewerRequests.acceptsRequest(consumerId, stated)) {
|
|
158
|
-
return false;
|
|
159
|
-
}
|
|
160
|
-
logger.info(
|
|
161
|
-
`[hold] ${fileName} refused: made in generation ${stated} of ${consumerId}, ` +
|
|
162
|
-
`a viewing they have left and whose window for new requests has passed`
|
|
163
|
-
);
|
|
164
|
-
reply.header("Retry-After", "0");
|
|
165
|
-
reply.code(503).send({ error: "Made for a viewing that has been left." });
|
|
166
|
-
return true;
|
|
167
|
-
}
|
|
168
|
-
|
|
169
|
-
/**
|
|
170
|
-
* Serve one playlist or segment from a named session.
|
|
171
|
-
*
|
|
172
|
-
* Split from the route above because the variant route
|
|
173
|
-
* (`/transcode/:sessionId/v/:height/:fileName`) serves the same files from
|
|
174
|
-
* another session of the same family, and must hold, log and answer them
|
|
175
|
-
* identically — a switch of quality must not go through a different code path
|
|
176
|
-
* from the stream it switches away from.
|
|
177
|
-
*
|
|
178
|
-
* @param {import("fastify").FastifyRequest} req
|
|
179
|
-
* @param {import("fastify").FastifyReply} reply
|
|
180
|
-
* @param {{ serving: object, viewerRequests: object, sessionId: string, fileName: string }} params
|
|
181
|
-
* @returns {Promise<void>}
|
|
182
|
-
*/
|
|
183
|
-
export async function serveSessionFile(req, reply, { serving, viewerRequests, sessionId, fileName }) {
|
|
184
|
-
// Which viewer is asking. One session serves everyone watching a copied
|
|
185
|
-
// picture, so without it their positions collapse into one and a seek by the
|
|
186
|
-
// viewer in front releases the requests held for the viewer behind. Absent on
|
|
187
|
-
// a plain HTTP transport, where no loader of ours builds the URL, and then
|
|
188
|
-
// the single shared position decides as it always did.
|
|
189
|
-
const consumerId = consumerOf(req);
|
|
190
|
-
// Hold the request only briefly, then answer "retry" instead of waiting for
|
|
191
|
-
// the segment. iOS's native HLS player (AVPlayer) enforces a hard ~3.5 s
|
|
192
|
-
// deadline on RESPONSE HEADERS and raises -12889 ("No response for media
|
|
193
|
-
// file") when it passes — it then cancels in-flight requests, probes
|
|
194
|
-
// neighbouring positions and can restart the stream from the beginning. That
|
|
195
|
-
// is exactly the post-seek "player thrashing" seen in the field, because a
|
|
196
|
-
// seek restarts ffmpeg and the first segment then takes far longer than 3.5 s
|
|
197
|
-
// to appear. Holding the connection for 30 s (as this did) guaranteed the
|
|
198
|
-
// timeout on every seek. A short hold keeps the fast path intact (a ready or
|
|
199
|
-
// nearly-ready segment is still served on the first request) while a slow one
|
|
200
|
-
// gets a prompt retryable answer, which resets the player's own deadline.
|
|
201
|
-
// hls.js is unaffected: it consumes the 503 through its retry policy, whose
|
|
202
|
-
// budget the client widens to match (see hls-player.js fragLoadPolicy).
|
|
203
|
-
// Instrumented wait. `clientAborted` flips when the player drops the
|
|
204
|
-
// connection while we are still holding it — the single most informative
|
|
205
|
-
// signal about its real patience, and observable only from this side.
|
|
206
|
-
const holdStartedAt = Date.now();
|
|
207
|
-
let clientAborted = false;
|
|
208
|
-
let onClientAbort = () => {};
|
|
209
|
-
const clientGone = new Promise((resolve) => {
|
|
210
|
-
onClientAbort = () => {
|
|
211
|
-
clientAborted = true;
|
|
212
|
-
resolve();
|
|
213
|
-
};
|
|
214
|
-
});
|
|
215
|
-
req.raw.on("close", onClientAbort);
|
|
216
|
-
const statedHoldMs = Number(req.headers?.[HOLD_HEADER]);
|
|
217
|
-
|
|
218
|
-
const result = await waitForSessionFile(serving, sessionId, fileName, {
|
|
219
|
-
holdMs: Number.isFinite(statedHoldMs) && statedHoldMs > 0 ? statedHoldMs : Number.POSITIVE_INFINITY,
|
|
220
|
-
until: clientGone,
|
|
221
|
-
consumerId
|
|
222
|
-
});
|
|
223
|
-
|
|
224
|
-
req.raw.off("close", onClientAbort);
|
|
225
|
-
const heldMs = Date.now() - holdStartedAt;
|
|
226
|
-
if (result.isPlaylist !== true) {
|
|
227
|
-
const outcome = clientAborted
|
|
228
|
-
? "client-aborted"
|
|
229
|
-
: result.kind === "ok" ? "served" : result.kind;
|
|
230
|
-
// AGAINST THE RANK THE MAP GAVE IT. A wait is the only thing that says
|
|
231
|
-
// whether the prioritisation is being used well, and this is the one place
|
|
232
|
-
// in the proxy where a viewer is measurably waiting for a named segment.
|
|
233
|
-
// Recorded even when the segment was served at once: a run of short waits
|
|
234
|
-
// at the top rank is what "the urgent zone is being served first" looks
|
|
235
|
-
// like, and without them the table would hold only the failures.
|
|
236
|
-
const ranked = result.ranked ?? null;
|
|
237
|
-
if (ranked) {
|
|
238
|
-
waits.note(ranked.address, heldMs, ranked.rank, ranked.topRank);
|
|
239
|
-
}
|
|
240
|
-
logger.info(
|
|
241
|
-
`[hold] ${fileName} ${outcome} after ${heldMs}ms` +
|
|
242
|
-
(ranked ? ` (the map wants it ${bandOf(ranked.rank, ranked.topRank)}, rank ${ranked.rank} of ${ranked.topRank})` : "")
|
|
243
|
-
);
|
|
244
|
-
}
|
|
245
|
-
|
|
246
|
-
if (result.kind === "not-found") {
|
|
247
|
-
return reply.code(404).send({ error: "Transcode session file was not found." });
|
|
248
|
-
}
|
|
249
|
-
if (result.kind === "superseded") {
|
|
250
|
-
// The viewer moved while this was being held, and this segment is not the
|
|
251
|
-
// one they moved TO — that case is kept and waited out, see
|
|
252
|
-
// `waitForSessionFile`. Answer at once so the player can ask for where it
|
|
253
|
-
// is now; `Retry-After: 0` because there is nothing to wait for.
|
|
254
|
-
//
|
|
255
|
-
// Named in the log, because a refusal that says only "superseded" is what
|
|
256
|
-
// made the 2026-08-18 freeze take a day to explain: the player was refused
|
|
257
|
-
// the segment at its own seek target and nothing recorded which segment or
|
|
258
|
-
// where the viewer was.
|
|
259
|
-
logger.info(
|
|
260
|
-
`[hold] ${fileName} refused: the viewer is at ` +
|
|
261
|
-
`${viewerRequests.viewerPositionOf(sessionId, consumerId).toFixed(1)}s and this is not the segment there`
|
|
262
|
-
);
|
|
263
|
-
reply.header("Retry-After", "0");
|
|
264
|
-
return reply.code(503).send({ error: "Superseded by a seek." });
|
|
265
|
-
}
|
|
266
|
-
if (result.kind === "warming-up") {
|
|
267
|
-
// The segment is still being produced (e.g. just after a seek-restart).
|
|
268
|
-
// Return a retryable 503 — never 202, which hls.js cannot consume as a
|
|
269
|
-
// media segment — so the player retries the fetch shortly.
|
|
270
|
-
reply.header("Retry-After", "1");
|
|
271
|
-
// `Retry-After` tells the player to re-request THIS segment after a short
|
|
272
|
-
// pause. Without it a bare 503 reads as "nothing here", and the player goes
|
|
273
|
-
// looking elsewhere: because our synthetic VOD playlist lists every segment
|
|
274
|
-
// of the file, it believes they all exist and SCANS them (field log: one
|
|
275
|
-
// user seek produced probes at #617, #717, #732…). That scan is what used
|
|
276
|
-
// to steer the encoder off the real target. Whether iOS's native player
|
|
277
|
-
// honours the hint is not guaranteed — its behaviour is closed — but this
|
|
278
|
-
// is the standard, correct way to say "wait, don't look elsewhere", and
|
|
279
|
-
// hls.js already retries the same fragment regardless.
|
|
280
|
-
reply.header("Retry-After", "1");
|
|
281
|
-
// SAY WHY, to the viewer and not only to the log. The same refusal is
|
|
282
|
-
// written into `proxy.log` with the rank the priority map gave this
|
|
283
|
-
// segment; the page had no access to that and told the viewer it did not
|
|
284
|
-
// know why. Field 2026-09-14: `rank 1 of 100` — nothing was making it —
|
|
285
|
-
// printed sixty seconds before the page gave up saying the opposite.
|
|
286
|
-
return reply.code(503).send({
|
|
287
|
-
error: "Transcode segment is still being produced.",
|
|
288
|
-
reason: warmingReason(result.ranked ?? null)
|
|
289
|
-
});
|
|
290
|
-
}
|
|
291
|
-
if (result.kind === "failed") {
|
|
292
|
-
return reply.code(500).send({ error: result.message });
|
|
293
|
-
}
|
|
294
|
-
|
|
295
|
-
if (result.isPlaylist) {
|
|
296
|
-
reply.header("Cache-Control", "no-store");
|
|
297
|
-
} else {
|
|
298
|
-
reply.header("Cache-Control", "public, max-age=60");
|
|
299
|
-
holdWhileSending(reply, result.stream, viewerRequests.holdResponse(sessionId, consumerId));
|
|
300
|
-
}
|
|
301
|
-
reply.header("Content-Type", result.contentType);
|
|
302
|
-
return reply.send(result.stream);
|
|
303
|
-
}
|
|
304
|
-
|
|
305
|
-
/**
|
|
306
|
-
* Keep the output this response comes from held until the response is over.
|
|
307
|
-
*
|
|
308
|
-
* Over means any of `finish`, `close` or `error` of the response, or `error`
|
|
309
|
-
* of the stream being sent — whichever comes first, and usually more than one
|
|
310
|
-
* of them comes: a response that finishes also closes. `release` is safe to
|
|
311
|
-
* call any number of times, which is what makes listening to all of them
|
|
312
|
-
* correct rather than a leak or a double release.
|
|
313
|
-
*
|
|
314
|
-
* A response already over before it began (the requester went during the
|
|
315
|
-
* hold) is released at once: none of its events will fire again.
|
|
316
|
-
*
|
|
317
|
-
* @param {import("fastify").FastifyReply} reply
|
|
318
|
-
* @param {import("node:stream").Readable | undefined} stream
|
|
319
|
-
* @param {() => void} release
|
|
320
|
-
* @returns {void}
|
|
321
|
-
*/
|
|
322
|
-
function holdWhileSending(reply, stream, release) {
|
|
323
|
-
const response = reply.raw;
|
|
324
|
-
if (!response || response.destroyed || response.writableEnded) {
|
|
325
|
-
release();
|
|
326
|
-
return;
|
|
327
|
-
}
|
|
328
|
-
response.once("finish", release);
|
|
329
|
-
response.once("close", release);
|
|
330
|
-
response.once("error", release);
|
|
331
|
-
stream?.once?.("error", release);
|
|
332
|
-
}
|
|
333
|
-
|
|
334
|
-
/**
|
|
335
|
-
* Ask `serving.getFileStream()` again each time the file may have become
|
|
336
|
-
* available — a segment's publication, or a waited-on invalidation — until it
|
|
337
|
-
* is, the session fails, the requester's stated wait runs out, or the
|
|
338
|
-
* requester goes.
|
|
339
|
-
*
|
|
340
|
-
* @param {object} serving - `services/server/SegmentServing.js`
|
|
341
|
-
* @param {string} sessionId
|
|
342
|
-
* @param {string} fileName
|
|
343
|
-
* @param {object} hold
|
|
344
|
-
* @param {number} hold.holdMs - How long the requester will wait; may be
|
|
345
|
-
* infinite when it stated nothing.
|
|
346
|
-
* @param {Promise<void>} [hold.until] - Settles when the requester has gone.
|
|
347
|
-
* @param {string} [hold.consumerId] - Which viewer is asking, where the
|
|
348
|
-
* transport carries it. Their own position is what decides whether a held
|
|
349
|
-
* request has been made pointless by a seek — a session can have several
|
|
350
|
-
* viewers, and the seek epoch belongs to all of them.
|
|
351
|
-
* @returns {Promise<Awaited<ReturnType<import("../../../services/server/SegmentServing.js").SegmentServing["getFileStream"]>>>}
|
|
352
|
-
*/
|
|
353
|
-
export async function waitForSessionFile(serving, sessionId, fileName, { holdMs, until = null, consumerId = "" }) {
|
|
354
|
-
const startedAt = Date.now();
|
|
355
|
-
let gone = false;
|
|
356
|
-
const requesterGone = until ? until.then(() => { gone = true; }) : new Promise(() => {});
|
|
357
|
-
// The viewer's position when this request was made. A seek makes every held
|
|
358
|
-
// request stale — it asks for a segment nobody is going to watch — and hls.js
|
|
359
|
-
// keeps only ONE fragment load outstanding, so holding on blocks the request
|
|
360
|
-
// the player actually needs now. Measured: 57 s of a 58 s backward seek was
|
|
361
|
-
// this wait, and the segment the viewer wanted took 15 ms once it was asked
|
|
362
|
-
// for.
|
|
363
|
-
let seekEpoch = serving.seekEpoch(sessionId);
|
|
364
|
-
/** @type {{ address: string, rank: number, topRank: number } | null} */
|
|
365
|
-
let lastRanked = null;
|
|
366
|
-
while (!gone && Date.now() - startedAt < holdMs) {
|
|
367
|
-
const result = await serving.getFileStream(sessionId, fileName, { consumerId });
|
|
368
|
-
if (result.kind !== "warming-up") {
|
|
369
|
-
return result;
|
|
370
|
-
}
|
|
371
|
-
if (serving.seekEpoch(sessionId) !== seekEpoch) {
|
|
372
|
-
// A seek moved the epoch. Whether THIS request is stale depends on which
|
|
373
|
-
// segment it asks for: the one the viewer has just landed on races the
|
|
374
|
-
// seek notification and would otherwise be refused at the exact moment it
|
|
375
|
-
// is needed — measured 2026-08-18, two 503s within 80 ms on the segment
|
|
376
|
-
// at the seek target, after which the player never asked for it again.
|
|
377
|
-
if (!serving.requestStillWanted(sessionId, fileName, consumerId)) {
|
|
378
|
-
return { kind: "superseded" };
|
|
379
|
-
}
|
|
380
|
-
logger.info(
|
|
381
|
-
`[hold] ${fileName} kept across a seek: it is the segment the viewer now needs`
|
|
382
|
-
);
|
|
383
|
-
seekEpoch = serving.seekEpoch(sessionId);
|
|
384
|
-
}
|
|
385
|
-
// The rank the map last gave this segment, so a request that runs out of
|
|
386
|
-
// patience is still counted against what it was promised. Kept across the
|
|
387
|
-
// polls because the timeout path has no result of its own.
|
|
388
|
-
lastRanked = result.ranked ?? lastRanked;
|
|
389
|
-
const remaining = Math.max(0, holdMs - (Date.now() - startedAt));
|
|
390
|
-
const waitedForSegment = await serving.waitForSegment(sessionId, fileName, remaining, until);
|
|
391
|
-
if (!waitedForSegment && !gone) {
|
|
392
|
-
// Playlists and init files are not segment publications.
|
|
393
|
-
await Promise.race([delay(Math.min(300, remaining)), requesterGone]);
|
|
394
|
-
}
|
|
395
|
-
}
|
|
396
|
-
return { kind: "warming-up", ranked: lastRanked };
|
|
397
|
-
}
|
|
398
|
-
|
|
399
|
-
/**
|
|
400
|
-
* Resolve after a given number of milliseconds.
|
|
401
|
-
*
|
|
402
|
-
* @param {number} ms
|
|
403
|
-
* @returns {Promise<void>}
|
|
404
|
-
*/
|
|
405
|
-
function delay(ms) {
|
|
406
|
-
return new Promise((resolve) => {
|
|
407
|
-
setTimeout(resolve, ms);
|
|
408
|
-
});
|
|
409
|
-
}
|
|
410
|
-
|
|
411
|
-
/**
|
|
412
|
-
* What to tell the VIEWER when a segment is not there yet.
|
|
413
|
-
*
|
|
414
|
-
* In their terms — the film, and what is being done about it — never in ours: a
|
|
415
|
-
* viewer cannot read a log, does not know what a rank is, and is not helped by
|
|
416
|
-
* being told which of our parts is waiting for which other.
|
|
417
|
-
*
|
|
418
|
-
* @param {{ rank: number, topRank: number } | null} ranked - What the priority
|
|
419
|
-
* map thought of this segment while the request was held.
|
|
420
|
-
* @returns {string}
|
|
421
|
-
*/
|
|
422
|
-
function warmingReason(ranked) {
|
|
423
|
-
if (ranked && ranked.rank < ranked.topRank) {
|
|
424
|
-
// The map is working somewhere else in the film. That is the answer the
|
|
425
|
-
// failure of 2026-09-14 needed and nobody was given.
|
|
426
|
-
return "This part of the film is not being prepared yet — the proxy is working further along.";
|
|
427
|
-
}
|
|
428
|
-
return "This part of the film is still being prepared.";
|
|
429
|
-
}
|
|
@@ -13,14 +13,13 @@
|
|
|
13
13
|
* no variant is ever asked for unless the viewer picked it.
|
|
14
14
|
*/
|
|
15
15
|
|
|
16
|
+
import { replyAssignmentLost, replyOutputUnavailable } from "../../../services/server/http-responses.js";
|
|
16
17
|
import {
|
|
17
18
|
consumerOf,
|
|
18
19
|
refusedAsStale,
|
|
19
|
-
replyAssignmentLost,
|
|
20
|
-
replyOutputUnavailable,
|
|
21
20
|
serveSessionFile,
|
|
22
21
|
statedGenerationOf
|
|
23
|
-
} from "
|
|
22
|
+
} from "../../../services/server/transcode-session-files.js";
|
|
24
23
|
|
|
25
24
|
/**
|
|
26
25
|
* @param {import("fastify").FastifyRequest} req
|
|
@@ -13,7 +13,8 @@
|
|
|
13
13
|
* knowing there is something to fetch.
|
|
14
14
|
*/
|
|
15
15
|
|
|
16
|
-
import { replyOutputUnavailable
|
|
16
|
+
import { replyOutputUnavailable } from "../../../services/server/http-responses.js";
|
|
17
|
+
import { waitForSessionFile } from "../../../services/server/transcode-session-files.js";
|
|
17
18
|
import { OUTPUT_UNAVAILABLE } from "../../../services/encode/output/index.js";
|
|
18
19
|
|
|
19
20
|
/** How long to hold the warm-up request before telling the caller to retry. */
|