@torrent-tv/proxy 2.11.0 → 2.12.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +16 -0
- package/package.json +1 -1
- package/routes/transcode/session-file/get.js +1 -1
- package/routes/transcode/variant-warm/get.js +71 -0
- package/server.js +7 -0
- package/services/hls-session-manager.js +308 -28
- package/test/keyframe-index-accuracy.test.js +61 -0
- package/test/quality-variants.test.js +83 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,19 @@
|
|
|
1
|
+
## 2.12.1
|
|
2
|
+
|
|
3
|
+
- **Fix**: The grid a copied stream is cut on now describes the FILE, not the container's index. A copy can only be cut where a keyframe already is, and nothing cheaper than the index can say where that is before a byte is encoded — but an index can be wrong. Reproduced 2026-08-12 against one file, both ways: with an honest index every produced segment started exactly where declared; with the index moved 1.8 s, every segment started 1.8 s early and matched no boundary at all. The field showed the second shape, so the mechanism was never at fault and the data was. The truth arrives anyway, one segment at a time — a produced piece states where it really begins — and it is now written back into the grid, which the whole family shares. That is what lets a re-encoded rung be cut to match a copied one: it is forced onto times the copy really uses. A correction that would cross its neighbours is refused, since that is a reading from a run that began somewhere else.
|
|
4
|
+
- **Fix**: A warm-up is no longer cancelled by the stream that is still playing. The cancellation stood before the check for whether the active rung had actually changed, and the rung on screen asks for its own segments every few seconds — so the rung being prepared was stopped 117 ms and 1.5 s after two warm-ups began (measured 2026-08-12), and the viewer then waited out the full thirty-second warm-up for a segment nobody was making, and waited again for the switch. One switch took 43.6 s.
|
|
5
|
+
- **Fix**: Warming the height the base session itself serves repositions it. It was skipped because it "is the base", but the base is parked wherever the viewer left it with its encoder stopped: warming 400p found it still at `run from #0`, so the switch had nothing to fetch.
|
|
6
|
+
- **Fix**: Repositioning inside this class names the session it means. `requestSeek` forwards to the rung on screen, which is right for the browser — it knows only the base id — and wrong for everything internal: warming a rung moved the rung already playing instead. Split into the public forwarding call and an internal literal one.
|
|
7
|
+
|
|
8
|
+
## 2.12.0
|
|
9
|
+
|
|
10
|
+
- **Fix**: A rung warmed for a switch the viewer did not make is stopped. Only becoming active stopped the rung being left, so trying two rungs in a row left the first encoding for nobody — three encoders at once on a host sized for one, which is the opposite of what warming is for.
|
|
11
|
+
- **Fix**: The warm-up closes the handle it opened. It answers without sending the bytes, and on formats whose segments are served straight off disk that left a file descriptor behind on every quality pick; enough of them and every read fails, segments included.
|
|
12
|
+
|
|
13
|
+
- **New**: A quality rung is prepared before the player is told to switch to it — `GET /transcode/:id/v/:height/warm?position=<seconds>`. A rung is an encoder that does not exist until it is asked for, so switching first and waiting second put the whole of its cold start on screen as a spinner: measured 2026-08-11, the first segment of a 240p rung producing at 1.2x took 15 988 ms, and the viewer watched all of it. The rung on screen deliberately keeps its own encoder until the player actually moves, so the wait happens behind a picture that is still playing. Both encoders run for the length of the warm-up, which is what the switch costs to be invisible.
|
|
14
|
+
- **Fix**: A viewer who names a resolution keeps the ladder beneath it. Forcing a rung disabled the realtime budget outright, so on 2026-08-11 a viewer picked 480p on a host that encodes it at 0.27-0.78x and the stream simply never caught up — nothing could step in, because the one thing that steps in had been switched off. The encode now STARTS at the size asked for and may still be stepped down under it. The rung's height is its name and does not move with a downshift, so the player goes on addressing it by the height it chose; what changes is the picture, and a smaller picture that plays beats a correct label that freezes.
|
|
15
|
+
- **New**: When a produced segment starts somewhere other than the playlist says, the line now names which boundary it DOES fall on. The two possible faults need opposite fixes and the numbers alone do not separate them: matching boundary #N-1 means this proxy's own numbering is shifted, matching none means the container's index describes times the file does not have. Measured 2026-08-11 on a 1080p Matroska, three samples out by 3.5-4.6 s, all matching #N-1.
|
|
16
|
+
|
|
1
17
|
## 2.11.0
|
|
2
18
|
|
|
3
19
|
- **Fix**: A quality change places the new rung where the PLAYER asked for it, not where the rung being left had read to. After a level switch hls.js discards what it had buffered ahead and fetches from the picture's own position, so its first request for the new rung IS that position; the read head is a whole buffer further on. Measured 2026-08-11 on a switch back up to 400p: a 240p rung encoding at 5-6x had read 56 s past the picture, the run was placed at 3084 s, the player needed 3028 s, and nothing it asked for was ever produced.
|
package/package.json
CHANGED
|
@@ -140,7 +140,7 @@ export async function serveSessionFile(req, reply, { hlsSessionManager, sessionI
|
|
|
140
140
|
* @param {number} timeoutMs
|
|
141
141
|
* @returns {Promise<Awaited<ReturnType<import("../../../services/hls-session-manager.js").HlsSessionManager["getFileStream"]>>>}
|
|
142
142
|
*/
|
|
143
|
-
async function waitForSessionFile(hlsSessionManager, sessionId, fileName, timeoutMs) {
|
|
143
|
+
export async function waitForSessionFile(hlsSessionManager, sessionId, fileName, timeoutMs) {
|
|
144
144
|
const startedAt = Date.now();
|
|
145
145
|
// One sequence number for THIS request, reused by every poll below, so the
|
|
146
146
|
// session can tell a newly-arrived request apart from an old one polling
|
|
@@ -0,0 +1,71 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @file GET /transcode/:sessionId/v/:height/warm?position=<seconds> — prepare a
|
|
3
|
+
* quality rung before the player is told to switch to it.
|
|
4
|
+
*
|
|
5
|
+
* A rung is an encoder that does not exist until someone asks for it, so a
|
|
6
|
+
* switch made first and waited for second shows the viewer a spinner for as
|
|
7
|
+
* long as the first segment takes to produce — 15 988 ms, measured 2026-08-11
|
|
8
|
+
* on a rung producing at 1.2x. Asking first and switching second moves that
|
|
9
|
+
* wait to where it cannot be seen: the rung on screen goes on playing, and it
|
|
10
|
+
* keeps its own encoder until the player actually moves.
|
|
11
|
+
*
|
|
12
|
+
* Answers when the segment at that position is ready, so the caller can switch
|
|
13
|
+
* knowing there is something to fetch.
|
|
14
|
+
*/
|
|
15
|
+
|
|
16
|
+
import { waitForSessionFile } from "../session-file/get.js";
|
|
17
|
+
|
|
18
|
+
/** How long to hold the warm-up request before telling the caller to retry. */
|
|
19
|
+
const WARM_WAIT_MS = 30_000;
|
|
20
|
+
|
|
21
|
+
/**
|
|
22
|
+
* @param {import("fastify").FastifyRequest} req
|
|
23
|
+
* @param {import("fastify").FastifyReply} reply
|
|
24
|
+
* @param {{ hlsSessionManager: import("../../../services/hls-session-manager.js").HlsSessionManager }} deps
|
|
25
|
+
* @returns {Promise<void>}
|
|
26
|
+
*/
|
|
27
|
+
export async function handleTranscodeVariantWarmGet(req, reply, { hlsSessionManager }) {
|
|
28
|
+
const baseSessionId = typeof req.params.sessionId === "string" ? req.params.sessionId : "";
|
|
29
|
+
const height = Number(req.params.height);
|
|
30
|
+
const positionSeconds = Number(req.query?.position);
|
|
31
|
+
|
|
32
|
+
if (!Number.isInteger(height) || height <= 0 || !Number.isFinite(positionSeconds) || positionSeconds < 0) {
|
|
33
|
+
return reply.code(400).send({ error: "A height and a non-negative position are required." });
|
|
34
|
+
}
|
|
35
|
+
|
|
36
|
+
let prepared;
|
|
37
|
+
try {
|
|
38
|
+
prepared = await hlsSessionManager.prepareVariant(baseSessionId, height, positionSeconds);
|
|
39
|
+
} catch (error) {
|
|
40
|
+
const message = error instanceof Error ? error.message : String(error);
|
|
41
|
+
reply.header("Retry-After", "1");
|
|
42
|
+
return reply.code(503).send({ error: `Could not prepare the quality variant: ${message}` });
|
|
43
|
+
}
|
|
44
|
+
if (!prepared) {
|
|
45
|
+
return reply.code(404).send({ error: "No such quality variant for this transcode session." });
|
|
46
|
+
}
|
|
47
|
+
|
|
48
|
+
const result = await waitForSessionFile(
|
|
49
|
+
hlsSessionManager,
|
|
50
|
+
prepared.sessionId,
|
|
51
|
+
prepared.fileName,
|
|
52
|
+
WARM_WAIT_MS
|
|
53
|
+
);
|
|
54
|
+
if (result.kind === "file") {
|
|
55
|
+
// The bytes are not sent — the player fetches them itself the moment it
|
|
56
|
+
// switches, and by then they are on disk — but the handle opened to reach
|
|
57
|
+
// them is ours to close. Some formats answer with a real file descriptor
|
|
58
|
+
// rather than bytes already in memory, and one left behind per quality pick
|
|
59
|
+
// walks a long-running proxy to EMFILE, where every read fails, segments
|
|
60
|
+
// included.
|
|
61
|
+
result.stream?.destroy?.();
|
|
62
|
+
return reply.code(204).send();
|
|
63
|
+
}
|
|
64
|
+
if (result.kind === "failed") {
|
|
65
|
+
return reply.code(500).send({ error: result.message });
|
|
66
|
+
}
|
|
67
|
+
// Still being produced. The caller may switch anyway — it will simply wait
|
|
68
|
+
// where it would have waited before — or ask again.
|
|
69
|
+
reply.header("Retry-After", "1");
|
|
70
|
+
return reply.code(503).send({ error: "The quality variant is still warming up." });
|
|
71
|
+
}
|
package/server.js
CHANGED
|
@@ -30,6 +30,7 @@ import { handleApiTranscodeSessionSeekPost } from "./routes/api/transcode-sessio
|
|
|
30
30
|
import { handleStreamGet } from "./routes/stream/get.js";
|
|
31
31
|
import { handleTranscodeSessionFileGet } from "./routes/transcode/session-file/get.js";
|
|
32
32
|
import { handleTranscodeVariantFileGet } from "./routes/transcode/variant-file/get.js";
|
|
33
|
+
import { handleTranscodeVariantWarmGet } from "./routes/transcode/variant-warm/get.js";
|
|
33
34
|
import { createSourceRegistry } from "./store/source-registry.js";
|
|
34
35
|
import { WorkerTorrentPool } from "./services/torrent-worker/pool-adapter.js";
|
|
35
36
|
import { HlsSessionManager } from "./services/hls-session-manager.js";
|
|
@@ -219,6 +220,12 @@ export async function startProxyServer({ host, port, transcodeAudio, ffmpegBin,
|
|
|
219
220
|
// A quality variant's files. Registered before the static handler for the
|
|
220
221
|
// same reason as the line above, and kept a separate route rather than a
|
|
221
222
|
// wildcard so the height stays a parsed parameter.
|
|
223
|
+
// Registered BEFORE the variant file route: `warm` is not a file name, and
|
|
224
|
+
// Fastify matches a static segment ahead of a parameter either way — stated
|
|
225
|
+
// here so the order is not "tidied" into a bug.
|
|
226
|
+
app.get("/transcode/:sessionId/v/:height/warm", async (req, reply) =>
|
|
227
|
+
handleTranscodeVariantWarmGet(req, reply, { hlsSessionManager })
|
|
228
|
+
);
|
|
222
229
|
app.get("/transcode/:sessionId/v/:height/:fileName", async (req, reply) =>
|
|
223
230
|
handleTranscodeVariantFileGet(req, reply, { hlsSessionManager })
|
|
224
231
|
);
|
|
@@ -155,6 +155,34 @@ export function noteIndexDeviation(check, index, deviationSec) {
|
|
|
155
155
|
}
|
|
156
156
|
}
|
|
157
157
|
|
|
158
|
+
/**
|
|
159
|
+
* The same budget, starting at the top of its own ladder.
|
|
160
|
+
*
|
|
161
|
+
* The automatic choice takes the highest rung this host can encode faster than
|
|
162
|
+
* realtime. A viewer who names a resolution has already made that choice, so
|
|
163
|
+
* the encode starts where they said — and the ladder stays, because a host that
|
|
164
|
+
* turns out unable to keep up must still have somewhere to go.
|
|
165
|
+
*
|
|
166
|
+
* @param {{ ladder: { width: number, height: number }[], rungIndex: number } | null} budget
|
|
167
|
+
* @param {number} outputFps
|
|
168
|
+
* @param {unknown} benchmark
|
|
169
|
+
* @returns {object | null}
|
|
170
|
+
*/
|
|
171
|
+
function startAtLadderTop(budget, outputFps, benchmark) {
|
|
172
|
+
const top = budget?.ladder?.[0];
|
|
173
|
+
if (!top) {
|
|
174
|
+
return null;
|
|
175
|
+
}
|
|
176
|
+
const fps = Number.isInteger(outputFps) && outputFps > 0 ? outputFps : TRANSCODE_FPS;
|
|
177
|
+
return {
|
|
178
|
+
...budget,
|
|
179
|
+
width: top.width,
|
|
180
|
+
height: top.height,
|
|
181
|
+
preset: pickSoftwarePreset(benchmark, top.width * top.height * fps),
|
|
182
|
+
rungIndex: 0
|
|
183
|
+
};
|
|
184
|
+
}
|
|
185
|
+
|
|
158
186
|
/**
|
|
159
187
|
* The consumer a base session registers on its variants.
|
|
160
188
|
*
|
|
@@ -1391,15 +1419,23 @@ export class HlsSessionManager {
|
|
|
1391
1419
|
Array.isArray(keyframeTimes) &&
|
|
1392
1420
|
keyframeTimes.length > 0 &&
|
|
1393
1421
|
(!transcodeVideo || inheritedGrid != null);
|
|
1394
|
-
|
|
1395
|
-
|
|
1396
|
-
|
|
1397
|
-
|
|
1398
|
-
|
|
1399
|
-
|
|
1400
|
-
|
|
1401
|
-
|
|
1402
|
-
:
|
|
1422
|
+
// A rung takes the grid it was handed, rather than working one out again
|
|
1423
|
+
// from the same index. The two are not the same table: the one it is handed
|
|
1424
|
+
// has been CORRECTED wherever a produced segment showed the index to be
|
|
1425
|
+
// wrong, and it is those corrected times the copy actually cuts at. Building
|
|
1426
|
+
// it afresh here would put the rung back on the index's fiction and undo the
|
|
1427
|
+
// alignment it exists for.
|
|
1428
|
+
const segmentBoundaries = Array.isArray(inheritedGrid?.boundaries) && inheritedGrid.boundaries.length > 1
|
|
1429
|
+
? [...inheritedGrid.boundaries]
|
|
1430
|
+
: (hasDuration
|
|
1431
|
+
? computeSegmentBoundaries({
|
|
1432
|
+
useKeyframeGrid,
|
|
1433
|
+
durationSeconds,
|
|
1434
|
+
segDur: this.segmentDurationSec,
|
|
1435
|
+
keyframeTimes,
|
|
1436
|
+
startTime: sourceStartTime
|
|
1437
|
+
})
|
|
1438
|
+
: []);
|
|
1403
1439
|
const usingKeyframeBoundaries = useKeyframeGrid;
|
|
1404
1440
|
const segmentCount = segmentBoundaries.length > 1 ? segmentBoundaries.length - 1 : 0;
|
|
1405
1441
|
|
|
@@ -1414,16 +1450,23 @@ export class HlsSessionManager {
|
|
|
1414
1450
|
// resolution, so encode exactly that box (capped to source by the scale
|
|
1415
1451
|
// filter) with the default preset, and the runtime downswitch is skipped
|
|
1416
1452
|
// for the session (budgetLadder stays null).
|
|
1417
|
-
const
|
|
1418
|
-
|
|
1419
|
-
:
|
|
1420
|
-
|
|
1421
|
-
|
|
1422
|
-
|
|
1423
|
-
|
|
1424
|
-
|
|
1425
|
-
|
|
1426
|
-
|
|
1453
|
+
const chosenBudget = this.#chooseEncodeBudget({
|
|
1454
|
+
transcodeVideo,
|
|
1455
|
+
targetWidth: normalizedTargetWidth,
|
|
1456
|
+
targetHeight: normalizedTargetHeight,
|
|
1457
|
+
sourceWidth,
|
|
1458
|
+
sourceHeight,
|
|
1459
|
+
outputFps
|
|
1460
|
+
});
|
|
1461
|
+
// A forced resolution starts at exactly that size — the viewer asked for it
|
|
1462
|
+
// — but KEEPS the ladder beneath it. Discarding the ladder is what left a
|
|
1463
|
+
// viewer with no picture at all on 2026-08-11: they picked 480p on a host
|
|
1464
|
+
// that encodes it at 0.27-0.78x, and with the runtime downshift disabled
|
|
1465
|
+
// nothing could step in, so the stream simply never caught up. A smaller
|
|
1466
|
+
// picture that plays beats a correct label that freezes. The rung's NAME is
|
|
1467
|
+
// settled separately and does not move with a downshift, so the player goes
|
|
1468
|
+
// on addressing it by the height it chose.
|
|
1469
|
+
const encodeBudget = forceManualQuality ? startAtLadderTop(chosenBudget, outputFps, this.softwarePresetBenchmark) : chosenBudget;
|
|
1427
1470
|
const softwarePreset = encodeBudget?.preset ?? null;
|
|
1428
1471
|
// Effective encode box: the budget's downscaled resolution when applied,
|
|
1429
1472
|
// otherwise the client target (0 = keep source, handled by buildVideoArgs).
|
|
@@ -1607,8 +1650,12 @@ export class HlsSessionManager {
|
|
|
1607
1650
|
`${sourceWidth && sourceHeight ? `src=${sourceWidth}x${sourceHeight} ` : ""}` +
|
|
1608
1651
|
// Effective encode resolution: budget-on (auto downscale from the
|
|
1609
1652
|
// ceiling), manual (user-forced, budget off), or unset (keep source).
|
|
1610
|
-
`${transcodeVideo && encodeBudget
|
|
1611
|
-
|
|
1653
|
+
`${transcodeVideo && encodeBudget
|
|
1654
|
+
? `enc=${encodeWidth}x${encodeHeight}@${outputFps} ` +
|
|
1655
|
+
`quality=${forceManualQuality ? "manual" : "auto"} ` +
|
|
1656
|
+
`budget=${encodeBudget.ladder ? `rung ${encodeBudget.rungIndex + 1}/${encodeBudget.ladder.length}` : "off"} `
|
|
1657
|
+
: ""}` +
|
|
1658
|
+
`${transcodeVideo && !encodeBudget && forceManualQuality ? `enc=${encodeWidth || "src"}x${encodeHeight || "src"}@${outputFps} quality=manual budget=off ` : ""}` +
|
|
1612
1659
|
// HDR source and whether the tone-map chain was applied (vs washed-out
|
|
1613
1660
|
// fallback when the filters are missing or on a hardware encoder).
|
|
1614
1661
|
`${transcodeVideo && mediaInfo.isHdr ? `hdr=1 tonemap=${applyTonemap ? "on" : "off"} ` : ""}` +
|
|
@@ -3315,7 +3362,27 @@ export class HlsSessionManager {
|
|
|
3315
3362
|
// variants, so a seek it reports means the stream on screen.
|
|
3316
3363
|
named.viewerPositionSeconds = positionSeconds;
|
|
3317
3364
|
named.lastAccessedAt = Date.now();
|
|
3318
|
-
|
|
3365
|
+
return this.#seekSession(this.#activeVariant(named), positionSeconds);
|
|
3366
|
+
}
|
|
3367
|
+
|
|
3368
|
+
/**
|
|
3369
|
+
* Reposition THIS session, with no forwarding.
|
|
3370
|
+
*
|
|
3371
|
+
* {@link requestSeek} exists for the browser, which names the base session and
|
|
3372
|
+
* means the rung on screen. Everything inside this class means the session it
|
|
3373
|
+
* is holding: warming a rung has to move THAT rung, and forwarding sent the
|
|
3374
|
+
* seek to the one already playing instead — measured 2026-08-12, warming the
|
|
3375
|
+
* base's own height moved the 540p rung and left the base parked at the start,
|
|
3376
|
+
* so the switch had nothing to fetch.
|
|
3377
|
+
*
|
|
3378
|
+
* @param {HlsSession} session
|
|
3379
|
+
* @param {number} positionSeconds
|
|
3380
|
+
* @returns {boolean}
|
|
3381
|
+
*/
|
|
3382
|
+
#seekSession(session, positionSeconds) {
|
|
3383
|
+
if (!session || session.state === "disposed") {
|
|
3384
|
+
return false;
|
|
3385
|
+
}
|
|
3319
3386
|
session.viewerPositionSeconds = positionSeconds;
|
|
3320
3387
|
session.lastAccessedAt = Date.now();
|
|
3321
3388
|
// Every segment request being held right now was made for the position the
|
|
@@ -3735,9 +3802,18 @@ export class HlsSessionManager {
|
|
|
3735
3802
|
session.indexCheck ??= newIndexCheck();
|
|
3736
3803
|
noteIndexDeviation(session.indexCheck, index, deviation);
|
|
3737
3804
|
if (deviation > SEGMENT_START_DISAGREEMENT_SEC) {
|
|
3805
|
+
// Which boundary the true start DOES match, if any. This is what tells
|
|
3806
|
+
// the two possible faults apart, and they need opposite fixes: matching
|
|
3807
|
+
// boundary #N-1 means our numbering is shifted by one — a fault in this
|
|
3808
|
+
// code, where the run begins — while matching nothing means the container
|
|
3809
|
+
// index describes times the file does not have. Measured 2026-08-11,
|
|
3810
|
+
// three samples all matched N-1, which is why the line now says so
|
|
3811
|
+
// instead of leaving it to be inferred from the numbers.
|
|
3812
|
+
const at = this.#boundaryIndexAt(session, trueStart);
|
|
3738
3813
|
logger.warn(
|
|
3739
3814
|
`transcode ${session.id} segment #${index} really starts at ` +
|
|
3740
|
-
`${trueStart.toFixed(3)}s
|
|
3815
|
+
`${trueStart.toFixed(3)}s (boundary ${at === null ? "none" : `#${at}`}), ` +
|
|
3816
|
+
`the playlist says ${declaredStart.toFixed(3)}s — ` +
|
|
3741
3817
|
(session.transcodeVideo
|
|
3742
3818
|
// A re-encode was TOLD to put a keyframe here and did not, so this
|
|
3743
3819
|
// rung's segments no longer stand where the stream it accompanies
|
|
@@ -3746,6 +3822,115 @@ export class HlsSessionManager {
|
|
|
3746
3822
|
: "the container's keyframe index disagrees with the file; using the file")
|
|
3747
3823
|
);
|
|
3748
3824
|
}
|
|
3825
|
+
this.correctBoundaryFromSegment(session, index, trueStart);
|
|
3826
|
+
}
|
|
3827
|
+
|
|
3828
|
+
/**
|
|
3829
|
+
* Replace a boundary the index got wrong with the time the file actually has.
|
|
3830
|
+
*
|
|
3831
|
+
* The grid of a copied stream comes from the container's keyframe index,
|
|
3832
|
+
* because a copy can only be cut where a keyframe already is and nothing
|
|
3833
|
+
* cheaper than the index can say where that is before a single byte is
|
|
3834
|
+
* encoded. An index can be wrong — proven 2026-08-12 by reproducing both
|
|
3835
|
+
* cases against the same file: with an honest index every produced segment
|
|
3836
|
+
* started exactly where declared, and with one moved 1.8 s the segments
|
|
3837
|
+
* started 1.8 s early, matching no boundary at all. The field showed the
|
|
3838
|
+
* second shape.
|
|
3839
|
+
*
|
|
3840
|
+
* The truth arrives anyway, one segment at a time: a produced piece states
|
|
3841
|
+
* where it really begins. Writing it back makes the grid describe the file
|
|
3842
|
+
* instead of the index — and it is what lets a re-encoded rung be cut to
|
|
3843
|
+
* match a copied one, because the rung is then forced onto times the copy
|
|
3844
|
+
* really uses. The alternative considered and rejected was to stop offering
|
|
3845
|
+
* quality on files with a bad index, which is not a fix but a withdrawal.
|
|
3846
|
+
*
|
|
3847
|
+
* The whole family shares one grid, so a correction reaches all of it: a rung
|
|
3848
|
+
* created afterwards inherits a table that is true wherever anyone has looked.
|
|
3849
|
+
*
|
|
3850
|
+
* @param {HlsSession} session
|
|
3851
|
+
* @param {number} index
|
|
3852
|
+
* @param {number} trueStart
|
|
3853
|
+
* @returns {void}
|
|
3854
|
+
*/
|
|
3855
|
+
correctBoundaryFromSegment(session, index, trueStart) {
|
|
3856
|
+
const boundaries = session.segmentBoundaries;
|
|
3857
|
+
if (!Array.isArray(boundaries) || index <= 0 || index >= boundaries.length - 1) {
|
|
3858
|
+
// Index 0 is the start of the file and the last entry is its end; neither
|
|
3859
|
+
// is a cut, and neither can be learned from a segment.
|
|
3860
|
+
return;
|
|
3861
|
+
}
|
|
3862
|
+
if (Math.abs(boundaries[index] - trueStart) <= SEGMENT_START_DISAGREEMENT_SEC) {
|
|
3863
|
+
return;
|
|
3864
|
+
}
|
|
3865
|
+
// A correction that would put this boundary at or past its neighbours is not
|
|
3866
|
+
// a correction — it is a reading from a run that started somewhere else, and
|
|
3867
|
+
// applying it would make the table describe nothing at all.
|
|
3868
|
+
if (trueStart <= boundaries[index - 1] || trueStart >= boundaries[index + 1]) {
|
|
3869
|
+
return;
|
|
3870
|
+
}
|
|
3871
|
+
const wasAt = boundaries[index];
|
|
3872
|
+
for (const member of this.#familyOf(session)) {
|
|
3873
|
+
if (Array.isArray(member.segmentBoundaries) && member.segmentBoundaries.length === boundaries.length) {
|
|
3874
|
+
member.segmentBoundaries[index] = trueStart;
|
|
3875
|
+
}
|
|
3876
|
+
}
|
|
3877
|
+
logger.info(
|
|
3878
|
+
`transcode ${session.id} boundary #${index} corrected ${wasAt.toFixed(3)}s → ` +
|
|
3879
|
+
`${trueStart.toFixed(3)}s from the file itself`
|
|
3880
|
+
);
|
|
3881
|
+
}
|
|
3882
|
+
|
|
3883
|
+
/**
|
|
3884
|
+
* Every session cut on one grid: a base and its quality rungs.
|
|
3885
|
+
*
|
|
3886
|
+
* @param {HlsSession} session
|
|
3887
|
+
* @returns {HlsSession[]}
|
|
3888
|
+
*/
|
|
3889
|
+
#familyOf(session) {
|
|
3890
|
+
const bases = session.variantBases instanceof Set
|
|
3891
|
+
? [...session.variantBases]
|
|
3892
|
+
: [];
|
|
3893
|
+
const roots = bases.length > 0 ? bases : [session.id];
|
|
3894
|
+
const family = new Set([session]);
|
|
3895
|
+
for (const rootId of roots) {
|
|
3896
|
+
const root = this.sessionsById.get(rootId);
|
|
3897
|
+
if (!root) {
|
|
3898
|
+
continue;
|
|
3899
|
+
}
|
|
3900
|
+
family.add(root);
|
|
3901
|
+
if (root.variants instanceof Map) {
|
|
3902
|
+
for (const variantId of root.variants.values()) {
|
|
3903
|
+
const variant = this.sessionsById.get(variantId);
|
|
3904
|
+
if (variant) {
|
|
3905
|
+
family.add(variant);
|
|
3906
|
+
}
|
|
3907
|
+
}
|
|
3908
|
+
}
|
|
3909
|
+
}
|
|
3910
|
+
return [...family];
|
|
3911
|
+
}
|
|
3912
|
+
|
|
3913
|
+
/**
|
|
3914
|
+
* The boundary a time falls on, or null when it falls on none of them.
|
|
3915
|
+
*
|
|
3916
|
+
* Within the same tolerance a disagreement is judged by, so "matches boundary
|
|
3917
|
+
* #N-1" and "matches nothing" mean what they say.
|
|
3918
|
+
*
|
|
3919
|
+
* @param {HlsSession} session
|
|
3920
|
+
* @param {number} seconds
|
|
3921
|
+
* @returns {number | null}
|
|
3922
|
+
*/
|
|
3923
|
+
#boundaryIndexAt(session, seconds) {
|
|
3924
|
+
const boundaries = session.segmentBoundaries;
|
|
3925
|
+
if (!Array.isArray(boundaries)) {
|
|
3926
|
+
return null;
|
|
3927
|
+
}
|
|
3928
|
+
for (let index = 0; index < boundaries.length; index += 1) {
|
|
3929
|
+
if (Math.abs(boundaries[index] - seconds) <= SEGMENT_START_DISAGREEMENT_SEC) {
|
|
3930
|
+
return index;
|
|
3931
|
+
}
|
|
3932
|
+
}
|
|
3933
|
+
return null;
|
|
3749
3934
|
}
|
|
3750
3935
|
|
|
3751
3936
|
/**
|
|
@@ -4032,7 +4217,13 @@ export class HlsSessionManager {
|
|
|
4032
4217
|
// be interchangeable with it. A base on the uniform grid needs nothing
|
|
4033
4218
|
// passed: the variant computes the same even grid from the same duration.
|
|
4034
4219
|
inheritedGrid: base.cutGrid === "keyframe"
|
|
4035
|
-
? {
|
|
4220
|
+
? {
|
|
4221
|
+
// The table as it stands NOW, corrections included — not the index
|
|
4222
|
+
// it was first built from.
|
|
4223
|
+
boundaries: base.segmentBoundaries,
|
|
4224
|
+
keyframeTimes: base.keyframeTimes,
|
|
4225
|
+
containerFormat: base.containerFormat
|
|
4226
|
+
}
|
|
4036
4227
|
: null,
|
|
4037
4228
|
acquireSource: base.acquireSource
|
|
4038
4229
|
})
|
|
@@ -4133,6 +4324,74 @@ export class HlsSessionManager {
|
|
|
4133
4324
|
return { sessionId: variant.id };
|
|
4134
4325
|
}
|
|
4135
4326
|
|
|
4327
|
+
/**
|
|
4328
|
+
* Prepare a rung the viewer is about to switch to, without switching to it.
|
|
4329
|
+
*
|
|
4330
|
+
* The rung does not exist until it is asked for, so the moment the player is
|
|
4331
|
+
* told to switch it has nothing to fetch and the viewer watches a spinner
|
|
4332
|
+
* while an encoder starts from nothing — measured 2026-08-11 at 15 988 ms for
|
|
4333
|
+
* the first segment of a rung producing at 1.2x. Nothing can make that
|
|
4334
|
+
* production instant; what CAN be done is to have it happen while the rung
|
|
4335
|
+
* the viewer is on is still playing.
|
|
4336
|
+
*
|
|
4337
|
+
* So this creates and positions the variant and says which segment to wait
|
|
4338
|
+
* for, and deliberately does NOT mark it active: the rung on screen keeps its
|
|
4339
|
+
* encoder until the player actually moves. Both encoders run for the length
|
|
4340
|
+
* of the warm-up, which is the price of the switch not being visible.
|
|
4341
|
+
*
|
|
4342
|
+
* @param {string} baseSessionId
|
|
4343
|
+
* @param {number} height
|
|
4344
|
+
* @param {number} positionSeconds - Where the switch will happen.
|
|
4345
|
+
* @returns {Promise<{ sessionId: string, fileName: string } | null>}
|
|
4346
|
+
*/
|
|
4347
|
+
async prepareVariant(baseSessionId, height, positionSeconds) {
|
|
4348
|
+
if (!isSafeSessionId(baseSessionId)) {
|
|
4349
|
+
return null;
|
|
4350
|
+
}
|
|
4351
|
+
const base = this.sessionsById.get(baseSessionId);
|
|
4352
|
+
if (!base || base.state === "disposed") {
|
|
4353
|
+
return null;
|
|
4354
|
+
}
|
|
4355
|
+
if (!this.#variantHeights(base).includes(height)) {
|
|
4356
|
+
return null;
|
|
4357
|
+
}
|
|
4358
|
+
const index = this.#segmentIndexForTime(base, positionSeconds);
|
|
4359
|
+
const variant = await this.resolveVariantSession(baseSessionId, height, index);
|
|
4360
|
+
if (!variant) {
|
|
4361
|
+
return null;
|
|
4362
|
+
}
|
|
4363
|
+
// A rung warmed for a switch that was never made. Nothing else would ever
|
|
4364
|
+
// stop it: only becoming active stops the rung being left, so a viewer
|
|
4365
|
+
// trying two rungs in a row would leave the first encoding for nobody until
|
|
4366
|
+
// the look-ahead cap suspended it — three encoders at once on a host sized
|
|
4367
|
+
// for one, which is the opposite of what warming is for.
|
|
4368
|
+
const stillWarming = base.warmingVariantId;
|
|
4369
|
+
if (stillWarming && stillWarming !== variant.id) {
|
|
4370
|
+
const abandoned = this.sessionsById.get(stillWarming);
|
|
4371
|
+
if (abandoned && abandoned.id !== this.#activeVariant(base).id) {
|
|
4372
|
+
this.#stopEncodeRun(abandoned, "warmed for a switch the viewer did not make");
|
|
4373
|
+
}
|
|
4374
|
+
}
|
|
4375
|
+
base.warmingVariantId = variant.id === base.id ? null : variant.id;
|
|
4376
|
+
// An existing rung may be parked wherever it was left, so it is pointed at
|
|
4377
|
+
// the switch position exactly as an activation would — the difference is
|
|
4378
|
+
// only that the rung on screen keeps its own encoder meanwhile.
|
|
4379
|
+
variant.lastAccessedAt = Date.now();
|
|
4380
|
+
// Anything that is not the rung on screen has to be pointed at the switch
|
|
4381
|
+
// position — INCLUDING the base. Skipping it because it is the base was a
|
|
4382
|
+
// defect: the base is parked wherever it was when the viewer left it, and
|
|
4383
|
+
// its encoder was stopped then. Measured 2026-08-12, warming 400p at
|
|
4384
|
+
// 6506.5s found the base still at `run from #0`, so the segment the switch
|
|
4385
|
+
// needed was never produced and the viewer got nothing at all.
|
|
4386
|
+
if (variant.id !== this.#activeVariant(base).id) {
|
|
4387
|
+
this.#seekSession(variant, this.#segmentStartTime(base, index));
|
|
4388
|
+
}
|
|
4389
|
+
logger.info(
|
|
4390
|
+
`transcode ${base.id} warming ${height}p at ${positionSeconds.toFixed(1)}s (segment #${index})`
|
|
4391
|
+
);
|
|
4392
|
+
return { sessionId: variant.id, fileName: variant.segmentFormat.segmentFileName(index) };
|
|
4393
|
+
}
|
|
4394
|
+
|
|
4136
4395
|
/**
|
|
4137
4396
|
* Record which variant the viewer is watching, and give it the encoder.
|
|
4138
4397
|
*
|
|
@@ -4149,8 +4408,27 @@ export class HlsSessionManager {
|
|
|
4149
4408
|
#noteVariantActive(base, variant, wantedIndex = -1) {
|
|
4150
4409
|
const previous = this.#activeVariant(base);
|
|
4151
4410
|
if (previous.id === variant.id) {
|
|
4411
|
+
// The rung on screen asking for more of itself, which it does every few
|
|
4412
|
+
// seconds. Nothing is being decided here — and deciding anything was the
|
|
4413
|
+
// defect: the warm-up was cancelled by the next segment the CURRENT rung
|
|
4414
|
+
// fetched, measured 2026-08-12 at 117 ms and 1.5 s after two warm-ups
|
|
4415
|
+
// began, so the rung being prepared was stopped before it had encoded
|
|
4416
|
+
// anything and the viewer waited out the full thirty-second warm-up for a
|
|
4417
|
+
// segment nobody was making, then waited again for the switch itself.
|
|
4152
4418
|
return;
|
|
4153
4419
|
}
|
|
4420
|
+
// A rung is being left, so whatever was warmed is decided: either it is the
|
|
4421
|
+
// rung now being switched to, or the viewer went somewhere else and it must
|
|
4422
|
+
// stop like any other rung nobody is watching. Nothing else would ever stop
|
|
4423
|
+
// it — only the rung being LEFT is stopped below.
|
|
4424
|
+
const warmed = base.warmingVariantId;
|
|
4425
|
+
base.warmingVariantId = null;
|
|
4426
|
+
if (warmed && warmed !== variant.id && warmed !== previous.id) {
|
|
4427
|
+
const abandoned = this.sessionsById.get(warmed);
|
|
4428
|
+
if (abandoned) {
|
|
4429
|
+
this.#stopEncodeRun(abandoned, "warmed for a switch the viewer did not make");
|
|
4430
|
+
}
|
|
4431
|
+
}
|
|
4154
4432
|
const position = this.#variantStartSeconds(base, wantedIndex);
|
|
4155
4433
|
base.activeVariantId = variant.id;
|
|
4156
4434
|
logger.info(
|
|
@@ -4165,10 +4443,12 @@ export class HlsSessionManager {
|
|
|
4165
4443
|
this.#stopEncodeRun(previous, `the viewer moved to ${this.variantHeightOf(variant)}p`);
|
|
4166
4444
|
if (position > 0) {
|
|
4167
4445
|
variant.viewerPositionSeconds = position;
|
|
4168
|
-
//
|
|
4169
|
-
//
|
|
4170
|
-
//
|
|
4171
|
-
|
|
4446
|
+
// The rung being switched TO, named literally: a warm-up may have left
|
|
4447
|
+
// the family pointing elsewhere, and forwarding would move that one
|
|
4448
|
+
// instead. A rung just created already starts here and is told so rather
|
|
4449
|
+
// than restarted; one that existed before is parked where it was left,
|
|
4450
|
+
// and this is what brings it to the viewer.
|
|
4451
|
+
this.#seekSession(variant, position);
|
|
4172
4452
|
}
|
|
4173
4453
|
}
|
|
4174
4454
|
|
|
@@ -56,6 +56,67 @@ test("a deviation within tolerance is not a disagreement, but still shows in the
|
|
|
56
56
|
assert.equal(check.maxDeviationSec, 0.2, "and it is still worth knowing how close to the line it ran");
|
|
57
57
|
});
|
|
58
58
|
|
|
59
|
+
test("a boundary the index got wrong is replaced by the time the file really has", async (t) => {
|
|
60
|
+
const { HlsSessionManager } = await import("../services/hls-session-manager.js");
|
|
61
|
+
const manager = new HlsSessionManager({
|
|
62
|
+
enabled: true,
|
|
63
|
+
ffmpegBin: "ffmpeg",
|
|
64
|
+
localBindHost: "127.0.0.1",
|
|
65
|
+
localPort: 9090
|
|
66
|
+
});
|
|
67
|
+
t.after(() => manager.disposeAll());
|
|
68
|
+
const base = {
|
|
69
|
+
id: "aaaaaaaa-1111-2222-3333-444444444444",
|
|
70
|
+
fileName: "film.mkv",
|
|
71
|
+
state: "ready",
|
|
72
|
+
transcodeVideo: false,
|
|
73
|
+
segmentBoundaries: [0, 10, 20, 30, 40],
|
|
74
|
+
indexCheck: newIndexCheck(),
|
|
75
|
+
variants: new Map(),
|
|
76
|
+
segmentFormat: { segmentFileName: (index) => `segment-${index}.mp4` }
|
|
77
|
+
};
|
|
78
|
+
const rung = {
|
|
79
|
+
id: "bbbbbbbb-1111-2222-3333-444444444444",
|
|
80
|
+
fileName: "film.mkv",
|
|
81
|
+
state: "ready",
|
|
82
|
+
transcodeVideo: true,
|
|
83
|
+
segmentBoundaries: [0, 10, 20, 30, 40],
|
|
84
|
+
indexCheck: newIndexCheck(),
|
|
85
|
+
variantBases: new Set([base.id])
|
|
86
|
+
};
|
|
87
|
+
base.variants.set(540, rung.id);
|
|
88
|
+
manager.sessionsById.set(base.id, base);
|
|
89
|
+
manager.sessionsById.set(rung.id, rung);
|
|
90
|
+
|
|
91
|
+
// The copy produced segment #2, and it really begins at 17.4 s — the index
|
|
92
|
+
// said 20. This is the shape reproduced from the field on 2026-08-12.
|
|
93
|
+
manager.correctBoundaryFromSegment(base, 2, 17.4);
|
|
94
|
+
|
|
95
|
+
assert.equal(
|
|
96
|
+
base.segmentBoundaries[2],
|
|
97
|
+
17.4,
|
|
98
|
+
"the grid must describe the file, not the index — a rung forced onto 20 s would not join the copy"
|
|
99
|
+
);
|
|
100
|
+
assert.equal(
|
|
101
|
+
rung.segmentBoundaries[2],
|
|
102
|
+
17.4,
|
|
103
|
+
"the family shares one grid, so a correction reaches the rungs cut against it"
|
|
104
|
+
);
|
|
105
|
+
assert.deepEqual(
|
|
106
|
+
base.segmentBoundaries,
|
|
107
|
+
[0, 10, 17.4, 30, 40],
|
|
108
|
+
"only the boundary that was shown to be wrong moves"
|
|
109
|
+
);
|
|
110
|
+
|
|
111
|
+
// A reading that cannot be a boundary is not evidence about one. It comes
|
|
112
|
+
// from a run that started somewhere else, and applying it would leave the
|
|
113
|
+
// table describing nothing.
|
|
114
|
+
manager.correctBoundaryFromSegment(base, 2, 35);
|
|
115
|
+
manager.correctBoundaryFromSegment(base, 2, 5);
|
|
116
|
+
manager.correctBoundaryFromSegment(base, 0, 3);
|
|
117
|
+
assert.deepEqual(base.segmentBoundaries, [0, 10, 17.4, 30, 40], "out-of-order readings are refused");
|
|
118
|
+
});
|
|
119
|
+
|
|
59
120
|
test("a segment requested again is not new evidence", () => {
|
|
60
121
|
const check = newIndexCheck();
|
|
61
122
|
|
|
@@ -283,6 +283,89 @@ test("a rung is placed where the player asked it for, not where the other rung h
|
|
|
283
283
|
);
|
|
284
284
|
});
|
|
285
285
|
|
|
286
|
+
test("warming a rung prepares it without taking the encoder from the one on screen", async (t) => {
|
|
287
|
+
const { manager, base, dirPath } = await managerWithBase();
|
|
288
|
+
t.after(async () => {
|
|
289
|
+
await manager.disposeAll();
|
|
290
|
+
await rm(dirPath, { recursive: true, force: true });
|
|
291
|
+
});
|
|
292
|
+
const variant = fakeSession({ id: VARIANT_ID, encodeHeight: 540, dirPath });
|
|
293
|
+
variant.variantHeight = 540;
|
|
294
|
+
variant.variantBases = new Set([BASE_ID]);
|
|
295
|
+
manager.sessionsById.set(VARIANT_ID, variant);
|
|
296
|
+
base.variants = new Map([[540, VARIANT_ID]]);
|
|
297
|
+
const encoder = fakeEncoder();
|
|
298
|
+
base.ffmpeg = encoder;
|
|
299
|
+
|
|
300
|
+
const prepared = await manager.prepareVariant(BASE_ID, 540, 240);
|
|
301
|
+
|
|
302
|
+
assert.deepEqual(
|
|
303
|
+
prepared,
|
|
304
|
+
{ sessionId: VARIANT_ID, fileName: "segment-00060.mp4" },
|
|
305
|
+
"the caller is told which segment to wait for — 240 s on a four-second grid"
|
|
306
|
+
);
|
|
307
|
+
assert.equal(variant.seekTarget, 59, "the rung is pointed at the switch position, one back for the keyframe");
|
|
308
|
+
assert.equal(base.activeVariantId, undefined, "nothing has switched yet");
|
|
309
|
+
assert.equal(base.ffmpeg, encoder, "the picture on screen keeps its encoder until the player actually moves");
|
|
310
|
+
assert.deepEqual(encoder.signals, [], "stopping it here is what would put the spinner back");
|
|
311
|
+
});
|
|
312
|
+
|
|
313
|
+
test("the rung on screen fetching its own segments does not cancel a warm-up", async (t) => {
|
|
314
|
+
const { manager, base, dirPath } = await managerWithBase();
|
|
315
|
+
t.after(async () => {
|
|
316
|
+
await manager.disposeAll();
|
|
317
|
+
await rm(dirPath, { recursive: true, force: true });
|
|
318
|
+
});
|
|
319
|
+
const variant = fakeSession({ id: VARIANT_ID, encodeHeight: 540, dirPath });
|
|
320
|
+
variant.variantHeight = 540;
|
|
321
|
+
variant.variantBases = new Set([BASE_ID]);
|
|
322
|
+
manager.sessionsById.set(VARIANT_ID, variant);
|
|
323
|
+
base.variants = new Map([[540, VARIANT_ID]]);
|
|
324
|
+
const warmedEncoder = fakeEncoder();
|
|
325
|
+
variant.ffmpeg = warmedEncoder;
|
|
326
|
+
base.ffmpeg = fakeEncoder();
|
|
327
|
+
await manager.prepareVariant(BASE_ID, 540, 100);
|
|
328
|
+
|
|
329
|
+
// The viewer has not moved: the rung they are watching goes on asking for its
|
|
330
|
+
// own segments, every few seconds, for as long as they watch.
|
|
331
|
+
await manager.resolveVariantFile(BASE_ID, 812, "segment-00026.mp4");
|
|
332
|
+
await manager.resolveVariantFile(BASE_ID, 812, "segment-00027.mp4");
|
|
333
|
+
|
|
334
|
+
assert.equal(base.warmingVariantId, VARIANT_ID, "the rung being prepared is still being prepared");
|
|
335
|
+
assert.deepEqual(
|
|
336
|
+
warmedEncoder.signals,
|
|
337
|
+
[],
|
|
338
|
+
"cancelling it here left the viewer waiting out the whole warm-up for a segment nobody was making"
|
|
339
|
+
);
|
|
340
|
+
});
|
|
341
|
+
|
|
342
|
+
test("warming the height the base itself serves still points it at the switch", async (t) => {
|
|
343
|
+
const { manager, base, dirPath } = await managerWithBase();
|
|
344
|
+
t.after(async () => {
|
|
345
|
+
await manager.disposeAll();
|
|
346
|
+
await rm(dirPath, { recursive: true, force: true });
|
|
347
|
+
});
|
|
348
|
+
// The viewer is on another rung; the base is parked where they left it, with
|
|
349
|
+
// its encoder stopped. Warming its height must bring it back.
|
|
350
|
+
const variant = fakeSession({ id: VARIANT_ID, encodeHeight: 540, dirPath });
|
|
351
|
+
variant.variantHeight = 540;
|
|
352
|
+
manager.sessionsById.set(VARIANT_ID, variant);
|
|
353
|
+
base.variants = new Map([[540, VARIANT_ID]]);
|
|
354
|
+
base.activeVariantId = VARIANT_ID;
|
|
355
|
+
base.ffmpeg = null;
|
|
356
|
+
base.encodeStartIndex = 0;
|
|
357
|
+
|
|
358
|
+
await manager.prepareVariant(BASE_ID, 812, 400);
|
|
359
|
+
|
|
360
|
+
// 400 s falls on the boundary between #99 and #100, and a run starts one
|
|
361
|
+
// segment back so the player has the preceding keyframe.
|
|
362
|
+
assert.equal(
|
|
363
|
+
base.seekTarget,
|
|
364
|
+
98,
|
|
365
|
+
"the base is parked at the start, so warming its height must reposition it like any other rung"
|
|
366
|
+
);
|
|
367
|
+
});
|
|
368
|
+
|
|
286
369
|
test("the viewer's position is kept current by the segments they ask for", async (t) => {
|
|
287
370
|
const { manager, base, dirPath } = await managerWithBase();
|
|
288
371
|
t.after(async () => {
|