@torrent-tv/proxy 2.9.63 → 2.9.65
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,12 @@
|
|
|
1
|
+
## 2.9.65
|
|
2
|
+
|
|
3
|
+
- **Fix**: Segment boundaries on the video-COPY path are now the source's **real** keyframe positions, read from the container's own index (`services/container-index/`), instead of an invented 4 s grid. ffmpeg can only cut a copied stream at existing keyframes, so the declared grid was simply false — measured on a field file, the true keyframe spacing is **10.43 s**, meaning roughly two of every three declared boundaries could not exist. Players punish this in two ways, both seen in the field 2026-08-02: on a long file the player stops trusting the playlist and walks it from segment #1 to locate a seek (a 1:30 seek produced requests #1, #2, #45, #86 … #1187 and never arrived), and on a short one it presents **audio with no picture**, because a segment beginning without a keyframe has nothing to decode from.
|
|
4
|
+
- **New**: `services/container-index/` — reads a file's keyframe table directly from the container (Matroska Cues today; MP4/AVI to follow) via two point reads: the head, to learn where the table lives, then the table itself. Measured against a 5.5 GB torrent-backed file: **570 keyframes in 0.8 s from 16 KB**, versus a full packet scan that found 77 in 45 s and never finished. Transport-agnostic by construction — it takes a byte-range function and knows nothing of torrents, HTTP or sessions — and cached per (source, file), so re-opens and seeks reuse the first read. Files with no readable index (live captures, interrupted writes, damaged uploads, MPEG-TS) return nothing and keep the previous fallback.
|
|
5
|
+
|
|
6
|
+
## 2.9.64
|
|
7
|
+
|
|
8
|
+
- **Fix**: The 2.9.63 pull-to-lowest-awaited-segment dragged the encoder to the start of the file. A seek to #1354 restarted at **#123** — the position of the *previous* watch — because requests outstanding from before the seek still counted toward `lowestAwaitedIndex`. Two fixes: the awaited floor is cleared the moment a new seek arrives (earlier requests describe where the player used to be, not where it is going), and the pull is bounded by `SEEK_PULL_LIMIT_SEGMENTS` (120) below the target — anything deeper is a leftover, not the preceding keyframe.
|
|
9
|
+
|
|
1
10
|
## 2.9.63
|
|
2
11
|
|
|
3
12
|
- **Fix**: A seek landed the encoder on exactly the requested segment, which is the one position the player never asks for — so it produced files nobody was waiting for and playback hung. Per Apple HLS authoring guidance, a player given a position locates the nearest keyframe **preceding** it, decodes from there, and only then presents from the requested point; it therefore always fetches segments **below** the target. Measured on iOS: a seek to #1082 fetched from #1074 (8 back), one to #1358 fetched from #1301 (57 back) and asked for **nothing at or above** the target. The encoder now starts `SEEK_BACKOFF_SEGMENTS` (12) before the requested segment, and — since the needed depth varies and no fixed number covers it — is pulled down further to the lowest segment the player is actually waiting on, which its own requests report exactly (`lowestAwaitedIndex`). Only ever moves the start earlier, never later. Costs a few seconds of extra encoding per seek.
|
package/package.json
CHANGED
|
@@ -0,0 +1,129 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @file Minimal EBML reader — the encoding Matroska/WebM are built from.
|
|
3
|
+
*
|
|
4
|
+
* Only what a keyframe index needs: walk elements, read unsigned integers,
|
|
5
|
+
* descend into containers. Deliberately not a general EBML implementation.
|
|
6
|
+
*
|
|
7
|
+
* Why not the `ebml` npm package (MIT, otherwise fine): its decoder is a
|
|
8
|
+
* *stream* decoder — it must be fed the file from byte zero. We read two small
|
|
9
|
+
* ranges out of a multi-gigabyte torrent-backed file and never the whole thing,
|
|
10
|
+
* so a parser that starts mid-file is exactly what we need and exactly what it
|
|
11
|
+
* cannot do (verified 2026-08-02: feeding it the Cues range throws
|
|
12
|
+
* "Unrepresentable length"). The element encoding itself is small enough to
|
|
13
|
+
* implement directly, so we do — the shape follows the Matroska specification
|
|
14
|
+
* and the same logic those libraries implement.
|
|
15
|
+
*/
|
|
16
|
+
|
|
17
|
+
/**
|
|
18
|
+
* A variable-length integer, as EBML encodes both element ids and sizes.
|
|
19
|
+
*
|
|
20
|
+
* The first set bit of the leading byte marks the width: `1xxxxxxx` is one
|
|
21
|
+
* byte, `01xxxxxx` two, and so on up to eight. For a SIZE the marker bit is
|
|
22
|
+
* removed and the remainder is the value; for an ID the bytes are kept intact,
|
|
23
|
+
* because the id *is* those bytes (that is how `0x1C53BB6B` identifies Cues).
|
|
24
|
+
*
|
|
25
|
+
* @param {Buffer} buffer
|
|
26
|
+
* @param {number} offset
|
|
27
|
+
* @param {boolean} keepMarker - True for ids, false for sizes.
|
|
28
|
+
* @returns {{ value: number, length: number } | null} Null when truncated or malformed.
|
|
29
|
+
*/
|
|
30
|
+
export function readVint(buffer, offset, keepMarker) {
|
|
31
|
+
if (offset >= buffer.length) {
|
|
32
|
+
return null;
|
|
33
|
+
}
|
|
34
|
+
const first = buffer[offset];
|
|
35
|
+
if (first === 0) {
|
|
36
|
+
return null; // No marker bit in the first byte: not a valid vint start.
|
|
37
|
+
}
|
|
38
|
+
let length = 1;
|
|
39
|
+
let mask = 0x80;
|
|
40
|
+
while (length <= 8 && (first & mask) === 0) {
|
|
41
|
+
length += 1;
|
|
42
|
+
mask >>= 1;
|
|
43
|
+
}
|
|
44
|
+
if (length > 8 || offset + length > buffer.length) {
|
|
45
|
+
return null;
|
|
46
|
+
}
|
|
47
|
+
let value = keepMarker ? first : first & (mask - 1);
|
|
48
|
+
for (let index = 1; index < length; index += 1) {
|
|
49
|
+
// Values beyond 2^53 cannot be represented exactly; sizes and positions in
|
|
50
|
+
// real files stay far below that, so plain arithmetic is safe here.
|
|
51
|
+
value = value * 256 + buffer[offset + index];
|
|
52
|
+
}
|
|
53
|
+
return { value, length };
|
|
54
|
+
}
|
|
55
|
+
|
|
56
|
+
/**
|
|
57
|
+
* Iterate the elements directly inside `buffer`, without descending.
|
|
58
|
+
*
|
|
59
|
+
* Yields each element's id, the offset of its payload and its size. An element
|
|
60
|
+
* whose payload runs past the end of the buffer is still yielded (its header is
|
|
61
|
+
* intact and the caller may only need the id), but iteration stops after it.
|
|
62
|
+
*
|
|
63
|
+
* @param {Buffer} buffer
|
|
64
|
+
* @param {number} [start=0]
|
|
65
|
+
* @param {number} [end=buffer.length]
|
|
66
|
+
* @yields {{ id: number, dataOffset: number, size: number }}
|
|
67
|
+
*/
|
|
68
|
+
export function* iterateElements(buffer, start = 0, end = buffer.length) {
|
|
69
|
+
let offset = start;
|
|
70
|
+
while (offset < end) {
|
|
71
|
+
const id = readVint(buffer, offset, true);
|
|
72
|
+
if (!id) {
|
|
73
|
+
return;
|
|
74
|
+
}
|
|
75
|
+
const size = readVint(buffer, offset + id.length, false);
|
|
76
|
+
if (!size) {
|
|
77
|
+
return;
|
|
78
|
+
}
|
|
79
|
+
const dataOffset = offset + id.length + size.length;
|
|
80
|
+
yield { id: id.value, dataOffset, size: size.value };
|
|
81
|
+
if (dataOffset + size.value > end) {
|
|
82
|
+
return; // Truncated payload — nothing dependable follows it.
|
|
83
|
+
}
|
|
84
|
+
offset = dataOffset + size.value;
|
|
85
|
+
}
|
|
86
|
+
}
|
|
87
|
+
|
|
88
|
+
/**
|
|
89
|
+
* Read an EBML unsigned integer payload (big-endian, variable width).
|
|
90
|
+
*
|
|
91
|
+
* @param {Buffer} buffer
|
|
92
|
+
* @param {number} offset
|
|
93
|
+
* @param {number} size
|
|
94
|
+
* @returns {number}
|
|
95
|
+
*/
|
|
96
|
+
export function readUint(buffer, offset, size) {
|
|
97
|
+
let value = 0;
|
|
98
|
+
for (let index = 0; index < size; index += 1) {
|
|
99
|
+
value = value * 256 + buffer[offset + index];
|
|
100
|
+
}
|
|
101
|
+
return value;
|
|
102
|
+
}
|
|
103
|
+
|
|
104
|
+
/**
|
|
105
|
+
* Depth-first search for the first element with `id`, descending only into the
|
|
106
|
+
* container ids listed in `descendInto`.
|
|
107
|
+
*
|
|
108
|
+
* @param {Buffer} buffer
|
|
109
|
+
* @param {number} id - Element id to find.
|
|
110
|
+
* @param {number[]} descendInto - Container ids worth entering.
|
|
111
|
+
* @param {number} [start=0]
|
|
112
|
+
* @param {number} [end=buffer.length]
|
|
113
|
+
* @returns {{ dataOffset: number, size: number } | null}
|
|
114
|
+
*/
|
|
115
|
+
export function findElement(buffer, id, descendInto, start = 0, end = buffer.length) {
|
|
116
|
+
for (const element of iterateElements(buffer, start, end)) {
|
|
117
|
+
if (element.id === id) {
|
|
118
|
+
return { dataOffset: element.dataOffset, size: element.size };
|
|
119
|
+
}
|
|
120
|
+
if (descendInto.includes(element.id)) {
|
|
121
|
+
const limit = Math.min(end, element.dataOffset + element.size);
|
|
122
|
+
const found = findElement(buffer, id, descendInto, element.dataOffset, limit);
|
|
123
|
+
if (found) {
|
|
124
|
+
return found;
|
|
125
|
+
}
|
|
126
|
+
}
|
|
127
|
+
}
|
|
128
|
+
return null;
|
|
129
|
+
}
|
|
@@ -0,0 +1,76 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @file Container keyframe index — where a file's real keyframes are, read
|
|
3
|
+
* from the container's own tables rather than by scanning the media.
|
|
4
|
+
*
|
|
5
|
+
* The problem it solves: on the video-COPY path ffmpeg can only cut segments at
|
|
6
|
+
* the source's existing keyframes. If our playlist declares an even grid
|
|
7
|
+
* instead, the declared times do not match the media, and players respond by
|
|
8
|
+
* either walking the whole file to rebuild the timeline or showing audio with
|
|
9
|
+
* no picture (both seen in the field 2026-08-02). Getting those positions by
|
|
10
|
+
* decoding is not an option here — the file is served from a torrent, and a
|
|
11
|
+
* full packet scan of 5.5 GB found 77 keyframes in 45 s without finishing.
|
|
12
|
+
*
|
|
13
|
+
* Containers already store this. The reader takes a byte-range function and
|
|
14
|
+
* does a couple of point reads, so it works over any transport and knows
|
|
15
|
+
* nothing about torrents, HTTP or our session model.
|
|
16
|
+
*
|
|
17
|
+
* Format support is deliberately partial: a container we cannot index returns
|
|
18
|
+
* null, and the caller falls back (re-encode with forced keyframes, which is
|
|
19
|
+
* what Jellyfin, hls-media-server and hls-vod-too all do unconditionally).
|
|
20
|
+
*/
|
|
21
|
+
|
|
22
|
+
import { logger } from "../../utils/logger.js";
|
|
23
|
+
import { isMatroska, readMatroskaKeyframeTimes } from "./matroska.js";
|
|
24
|
+
|
|
25
|
+
/**
|
|
26
|
+
* @callback ReadRange
|
|
27
|
+
* @param {number} start - First byte, inclusive.
|
|
28
|
+
* @param {number} end - Last byte, inclusive.
|
|
29
|
+
* @returns {Promise<Buffer | null>} The bytes, or null when unavailable.
|
|
30
|
+
*/
|
|
31
|
+
|
|
32
|
+
// Enough to identify any supported container from its magic bytes.
|
|
33
|
+
const SNIFF_BYTES = 16;
|
|
34
|
+
|
|
35
|
+
/**
|
|
36
|
+
* Read a file's keyframe times from its container index.
|
|
37
|
+
*
|
|
38
|
+
* @param {object} params
|
|
39
|
+
* @param {ReadRange} params.readRange
|
|
40
|
+
* @param {number} params.fileSize
|
|
41
|
+
* @param {string} [params.label] - For logging only.
|
|
42
|
+
* @returns {Promise<number[] | null>} Ascending seconds, or null when this file
|
|
43
|
+
* has no readable index — the caller must then not claim to know the grid.
|
|
44
|
+
*/
|
|
45
|
+
export async function readKeyframeIndex({ readRange, fileSize, label = "" }) {
|
|
46
|
+
if (typeof readRange !== "function" || !Number.isFinite(fileSize) || fileSize <= 0) {
|
|
47
|
+
return null;
|
|
48
|
+
}
|
|
49
|
+
|
|
50
|
+
const startedAt = Date.now();
|
|
51
|
+
let times = null;
|
|
52
|
+
try {
|
|
53
|
+
const sniff = await readRange(0, Math.min(SNIFF_BYTES - 1, fileSize - 1));
|
|
54
|
+
if (!sniff) {
|
|
55
|
+
return null;
|
|
56
|
+
}
|
|
57
|
+
if (isMatroska(sniff)) {
|
|
58
|
+
times = await readMatroskaKeyframeTimes(readRange, fileSize);
|
|
59
|
+
}
|
|
60
|
+
// Other containers fall through as null until their readers land; MP4's
|
|
61
|
+
// sync-sample table and AVI's index are the next candidates.
|
|
62
|
+
} catch (error) {
|
|
63
|
+
// A malformed or partially-downloaded index must never take playback down —
|
|
64
|
+
// it only means we do not know the grid, which the caller handles.
|
|
65
|
+
logger.warn(`container-index: failed to read index for "${label}": ${error?.message ?? error}`);
|
|
66
|
+
return null;
|
|
67
|
+
}
|
|
68
|
+
|
|
69
|
+
const elapsedMs = Date.now() - startedAt;
|
|
70
|
+
if (times) {
|
|
71
|
+
logger.info(`container-index: ${times.length} keyframes from the container index in ${elapsedMs}ms for "${label}"`);
|
|
72
|
+
} else {
|
|
73
|
+
logger.info(`container-index: no usable index for "${label}" (${elapsedMs}ms)`);
|
|
74
|
+
}
|
|
75
|
+
return times;
|
|
76
|
+
}
|
|
@@ -0,0 +1,200 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @file Keyframe index for Matroska (MKV/WebM), read without downloading the file.
|
|
3
|
+
*
|
|
4
|
+
* Matroska stores a Cues element — a table of "at time T, a keyframe starts at
|
|
5
|
+
* byte P" — and a SeekHead near the start listing where each top-level element
|
|
6
|
+
* lives. So two point reads suffice: the head, to learn where Cues is, then
|
|
7
|
+
* Cues itself. Measured on a 5.5 GB torrent-backed file: 4 KB + 12 KB, both
|
|
8
|
+
* effectively instant, versus a full packet scan that found 77 keyframes in
|
|
9
|
+
* 45 s and never finished.
|
|
10
|
+
*
|
|
11
|
+
* This matters because on the video-COPY path the encoder can only cut on the
|
|
12
|
+
* source's own keyframes. A playlist declaring an even grid is then a lie, and
|
|
13
|
+
* players react badly to it — either walking the whole file to rebuild the
|
|
14
|
+
* timeline, or presenting audio with no picture because a segment begins with
|
|
15
|
+
* no keyframe to decode from (both observed in the field 2026-08-02).
|
|
16
|
+
*/
|
|
17
|
+
|
|
18
|
+
import { findElement, iterateElements, readUint } from "./ebml-reader.js";
|
|
19
|
+
|
|
20
|
+
// Element ids, from the Matroska specification.
|
|
21
|
+
const ID_SEGMENT = 0x18538067;
|
|
22
|
+
const ID_SEEK_HEAD = 0x114d9b74;
|
|
23
|
+
const ID_SEEK = 0x4dbb;
|
|
24
|
+
const ID_SEEK_ID = 0x53ab;
|
|
25
|
+
const ID_SEEK_POSITION = 0x53ac;
|
|
26
|
+
const ID_INFO = 0x1549a966;
|
|
27
|
+
const ID_TIMESTAMP_SCALE = 0x2ad7b1;
|
|
28
|
+
const ID_CUES = 0x1c53bb6b;
|
|
29
|
+
const ID_CUE_POINT = 0xbb;
|
|
30
|
+
const ID_CUE_TIME = 0xb3;
|
|
31
|
+
|
|
32
|
+
// How much of the file start to read. Must cover the EBML header, the SeekHead
|
|
33
|
+
// and Info; 64 KB is generous for every real muxer (the file measured needed
|
|
34
|
+
// under 4 KB) while still trivial to fetch.
|
|
35
|
+
const HEAD_BYTES = 64 * 1024;
|
|
36
|
+
// Cap on the Cues read. A two-hour film indexes to tens of KB; anything beyond
|
|
37
|
+
// this is not a normal index and not worth pulling over a torrent.
|
|
38
|
+
const MAX_CUES_BYTES = 8 * 1024 * 1024;
|
|
39
|
+
// Matroska's default timestamp scale (nanoseconds per tick) when Info omits it.
|
|
40
|
+
const DEFAULT_TIMESTAMP_SCALE = 1_000_000;
|
|
41
|
+
|
|
42
|
+
/**
|
|
43
|
+
* Whether this looks like a Matroska file (the EBML magic `0x1A45DFA3`).
|
|
44
|
+
*
|
|
45
|
+
* @param {Buffer} head
|
|
46
|
+
* @returns {boolean}
|
|
47
|
+
*/
|
|
48
|
+
export function isMatroska(head) {
|
|
49
|
+
return head.length >= 4 && head.readUInt32BE(0) === 0x1a45dfa3;
|
|
50
|
+
}
|
|
51
|
+
|
|
52
|
+
/**
|
|
53
|
+
* Locate the Segment element and the SeekHead entries inside it.
|
|
54
|
+
*
|
|
55
|
+
* Seek positions are relative to the start of Segment's payload, not to the
|
|
56
|
+
* file, so that base has to come back with them.
|
|
57
|
+
*
|
|
58
|
+
* @param {Buffer} head
|
|
59
|
+
* @returns {{ segmentDataOffset: number, entries: Map<number, number> } | null}
|
|
60
|
+
*/
|
|
61
|
+
function readSeekHead(head) {
|
|
62
|
+
let segmentDataOffset = -1;
|
|
63
|
+
for (const element of iterateElements(head)) {
|
|
64
|
+
if (element.id === ID_SEGMENT) {
|
|
65
|
+
segmentDataOffset = element.dataOffset;
|
|
66
|
+
break;
|
|
67
|
+
}
|
|
68
|
+
}
|
|
69
|
+
if (segmentDataOffset < 0) {
|
|
70
|
+
return null;
|
|
71
|
+
}
|
|
72
|
+
|
|
73
|
+
const seekHead = findElement(head, ID_SEEK_HEAD, [], segmentDataOffset);
|
|
74
|
+
if (!seekHead) {
|
|
75
|
+
return null;
|
|
76
|
+
}
|
|
77
|
+
|
|
78
|
+
const entries = new Map();
|
|
79
|
+
const seekHeadEnd = Math.min(head.length, seekHead.dataOffset + seekHead.size);
|
|
80
|
+
for (const seek of iterateElements(head, seekHead.dataOffset, seekHeadEnd)) {
|
|
81
|
+
if (seek.id !== ID_SEEK) {
|
|
82
|
+
continue;
|
|
83
|
+
}
|
|
84
|
+
const seekEnd = Math.min(seekHeadEnd, seek.dataOffset + seek.size);
|
|
85
|
+
let targetId = null;
|
|
86
|
+
let position = null;
|
|
87
|
+
for (const field of iterateElements(head, seek.dataOffset, seekEnd)) {
|
|
88
|
+
if (field.id === ID_SEEK_ID) {
|
|
89
|
+
targetId = readUint(head, field.dataOffset, field.size);
|
|
90
|
+
} else if (field.id === ID_SEEK_POSITION) {
|
|
91
|
+
position = readUint(head, field.dataOffset, field.size);
|
|
92
|
+
}
|
|
93
|
+
}
|
|
94
|
+
if (targetId !== null && position !== null) {
|
|
95
|
+
entries.set(targetId, position);
|
|
96
|
+
}
|
|
97
|
+
}
|
|
98
|
+
return { segmentDataOffset, entries };
|
|
99
|
+
}
|
|
100
|
+
|
|
101
|
+
/**
|
|
102
|
+
* Timestamp scale (nanoseconds per tick) declared in Info, or the default.
|
|
103
|
+
*
|
|
104
|
+
* @param {Buffer} head
|
|
105
|
+
* @param {number} segmentDataOffset
|
|
106
|
+
* @returns {number}
|
|
107
|
+
*/
|
|
108
|
+
function readTimestampScale(head, segmentDataOffset) {
|
|
109
|
+
const info = findElement(head, ID_INFO, [], segmentDataOffset);
|
|
110
|
+
if (!info) {
|
|
111
|
+
return DEFAULT_TIMESTAMP_SCALE;
|
|
112
|
+
}
|
|
113
|
+
const infoEnd = Math.min(head.length, info.dataOffset + info.size);
|
|
114
|
+
for (const field of iterateElements(head, info.dataOffset, infoEnd)) {
|
|
115
|
+
if (field.id === ID_TIMESTAMP_SCALE) {
|
|
116
|
+
const scale = readUint(head, field.dataOffset, field.size);
|
|
117
|
+
return scale > 0 ? scale : DEFAULT_TIMESTAMP_SCALE;
|
|
118
|
+
}
|
|
119
|
+
}
|
|
120
|
+
return DEFAULT_TIMESTAMP_SCALE;
|
|
121
|
+
}
|
|
122
|
+
|
|
123
|
+
/**
|
|
124
|
+
* Cue times (seconds, ascending) from a Cues payload.
|
|
125
|
+
*
|
|
126
|
+
* @param {Buffer} cues
|
|
127
|
+
* @param {number} timestampScale - Nanoseconds per tick.
|
|
128
|
+
* @returns {number[]}
|
|
129
|
+
*/
|
|
130
|
+
function readCueTimes(cues, timestampScale) {
|
|
131
|
+
const times = [];
|
|
132
|
+
const secondsPerTick = timestampScale / 1e9;
|
|
133
|
+
for (const point of iterateElements(cues)) {
|
|
134
|
+
if (point.id !== ID_CUE_POINT) {
|
|
135
|
+
continue;
|
|
136
|
+
}
|
|
137
|
+
const pointEnd = Math.min(cues.length, point.dataOffset + point.size);
|
|
138
|
+
for (const field of iterateElements(cues, point.dataOffset, pointEnd)) {
|
|
139
|
+
if (field.id === ID_CUE_TIME) {
|
|
140
|
+
times.push(readUint(cues, field.dataOffset, field.size) * secondsPerTick);
|
|
141
|
+
break;
|
|
142
|
+
}
|
|
143
|
+
}
|
|
144
|
+
}
|
|
145
|
+
times.sort((left, right) => left - right);
|
|
146
|
+
return times;
|
|
147
|
+
}
|
|
148
|
+
|
|
149
|
+
/**
|
|
150
|
+
* Read the keyframe times of a Matroska file using only two point reads.
|
|
151
|
+
*
|
|
152
|
+
* @param {(start: number, end: number) => Promise<Buffer | null>} readRange
|
|
153
|
+
* Inclusive byte range reader; returns null when the range is unavailable.
|
|
154
|
+
* @param {number} fileSize
|
|
155
|
+
* @returns {Promise<number[] | null>} Ascending seconds, or null when the file
|
|
156
|
+
* carries no usable index (see the module doc for when that happens).
|
|
157
|
+
*/
|
|
158
|
+
export async function readMatroskaKeyframeTimes(readRange, fileSize) {
|
|
159
|
+
const head = await readRange(0, Math.min(HEAD_BYTES, Math.max(0, fileSize - 1)));
|
|
160
|
+
if (!head || !isMatroska(head)) {
|
|
161
|
+
return null;
|
|
162
|
+
}
|
|
163
|
+
|
|
164
|
+
const seekHead = readSeekHead(head);
|
|
165
|
+
if (!seekHead) {
|
|
166
|
+
return null; // No SeekHead — a streamed or truncated mux.
|
|
167
|
+
}
|
|
168
|
+
|
|
169
|
+
const cuesRelative = seekHead.entries.get(ID_CUES);
|
|
170
|
+
if (cuesRelative === undefined) {
|
|
171
|
+
return null; // Indexless file: live capture, interrupted write, damaged upload.
|
|
172
|
+
}
|
|
173
|
+
|
|
174
|
+
// SeekHead positions are relative to Segment's payload.
|
|
175
|
+
const cuesOffset = seekHead.segmentDataOffset + cuesRelative;
|
|
176
|
+
if (!Number.isFinite(cuesOffset) || cuesOffset <= 0 || cuesOffset >= fileSize) {
|
|
177
|
+
return null;
|
|
178
|
+
}
|
|
179
|
+
|
|
180
|
+
// The element header states the payload size, but reading it costs a round
|
|
181
|
+
// trip; fetch a bounded window instead and let the parser stop at the end of
|
|
182
|
+
// what it got. Cues sits near the file end, so the window is clamped there.
|
|
183
|
+
const cuesEnd = Math.min(fileSize - 1, cuesOffset + MAX_CUES_BYTES);
|
|
184
|
+
const cuesChunk = await readRange(cuesOffset, cuesEnd);
|
|
185
|
+
if (!cuesChunk || cuesChunk.length === 0) {
|
|
186
|
+
return null;
|
|
187
|
+
}
|
|
188
|
+
|
|
189
|
+
// The window starts exactly at the Cues element, so its own header comes
|
|
190
|
+
// first; step over it to reach the CuePoints.
|
|
191
|
+
const cuesElement = [...iterateElements(cuesChunk, 0, cuesChunk.length)][0];
|
|
192
|
+
if (!cuesElement || cuesElement.id !== ID_CUES) {
|
|
193
|
+
return null;
|
|
194
|
+
}
|
|
195
|
+
const payloadEnd = Math.min(cuesChunk.length, cuesElement.dataOffset + cuesElement.size);
|
|
196
|
+
const payload = cuesChunk.subarray(cuesElement.dataOffset, payloadEnd);
|
|
197
|
+
|
|
198
|
+
const times = readCueTimes(payload, readTimestampScale(head, seekHead.segmentDataOffset));
|
|
199
|
+
return times.length > 0 ? times : null;
|
|
200
|
+
}
|
|
@@ -16,6 +16,7 @@ import { randomUUID } from "node:crypto";
|
|
|
16
16
|
import { spawn } from "node:child_process";
|
|
17
17
|
import { createRequire } from "node:module";
|
|
18
18
|
import { logger } from "../utils/logger.js";
|
|
19
|
+
import { readKeyframeIndex } from "./container-index/index.js";
|
|
19
20
|
|
|
20
21
|
/** Own package version, stamped onto session-start log lines. */
|
|
21
22
|
const PROXY_VERSION = createRequire(import.meta.url)("../package.json").version;
|
|
@@ -85,6 +86,13 @@ const ENCODER_STALL_MS = 12_000;
|
|
|
85
86
|
// player is actually waiting on when that is lower still. Costs a few seconds
|
|
86
87
|
// of extra encoding per seek.
|
|
87
88
|
const SEEK_BACKOFF_SEGMENTS = 12;
|
|
89
|
+
// Hard limit on how far below the requested segment the start may be pulled by
|
|
90
|
+
// `lowestAwaitedIndex`. Without it a stale request from earlier playback drags
|
|
91
|
+
// the encoder across the whole file: field 2026-08-02, a seek to #1354 was
|
|
92
|
+
// pulled to #123 — the start of the previous watch — because requests from
|
|
93
|
+
// before the seek were still counted. Anything deeper than this is not the
|
|
94
|
+
// preceding keyframe, it is a leftover.
|
|
95
|
+
const SEEK_PULL_LIMIT_SEGMENTS = 120;
|
|
88
96
|
const SEEK_SETTLE_MS = 1_200;
|
|
89
97
|
// Hard cap on the total settle wait, measured from the first far request of a
|
|
90
98
|
// burst, so a still-moving scrubber cannot delay a genuine seek forever.
|
|
@@ -485,6 +493,17 @@ async function probeVideoKeyframeTimes(ffmpegBin, inputUrl, timeoutMs = 25_000)
|
|
|
485
493
|
ffprobeBinFor(ffmpegBin),
|
|
486
494
|
[
|
|
487
495
|
"-v", "error",
|
|
496
|
+
// `-skip_frame nokey` makes the decoder discard non-keyframes, so the
|
|
497
|
+
// probe reads only what it needs. Without it a full packet scan of a
|
|
498
|
+
// ~5 GB MKV cannot finish inside any sane budget over a torrent-backed
|
|
499
|
+
// input, the probe returns nothing, and the playlist falls back to a
|
|
500
|
+
// uniform grid — which on the COPY path is a lie: cuts land on the
|
|
501
|
+
// source's real keyframes, not on a 4 s ruler. The player then finds
|
|
502
|
+
// the declared times do not match the media, stops trusting the
|
|
503
|
+
// playlist and walks the file from segment #1 to locate the seek
|
|
504
|
+
// position by hand (field 2026-08-02: a seek to 1:30 produced requests
|
|
505
|
+
// #1, #2, #45, #86, #123 … #1187, taking minutes and never arriving).
|
|
506
|
+
"-skip_frame", "nokey",
|
|
488
507
|
"-select_streams", "v:0",
|
|
489
508
|
"-show_entries", "packet=pts_time,flags",
|
|
490
509
|
"-of", "csv=p=0",
|
|
@@ -728,6 +747,11 @@ export class HlsSessionManager {
|
|
|
728
747
|
this.startupWaitMs = startupWaitMs;
|
|
729
748
|
this.localBaseUrl = buildHttpBaseUrl(localBindHost, localPort);
|
|
730
749
|
this.sessionsById = new Map();
|
|
750
|
+
// Container keyframe index per (source, file). Immutable per file, so one
|
|
751
|
+
// read serves every session, re-open and seek. Null means "this file has no
|
|
752
|
+
// readable index" and is cached too — no point retrying a scan that cannot
|
|
753
|
+
// succeed.
|
|
754
|
+
this.keyframeIndexCache = new Map();
|
|
731
755
|
this.sessionIdBySource = new Map();
|
|
732
756
|
this.cleanupTimer = setInterval(() => {
|
|
733
757
|
void this.cleanupExpired();
|
|
@@ -906,12 +930,22 @@ export class HlsSessionManager {
|
|
|
906
930
|
// packet scan time out and fall back to a uniform grid, so this never adds
|
|
907
931
|
// more than ~6 s to session start.
|
|
908
932
|
const keyframeStartMs = Date.now();
|
|
909
|
-
|
|
933
|
+
// Read the container's OWN keyframe table (Cues/stss) rather than
|
|
934
|
+
// scanning the media. On the copy path ffmpeg can only cut at the
|
|
935
|
+
// source's existing keyframes, so these times ARE the segment
|
|
936
|
+
// boundaries — declaring an even grid instead is a falsehood the player
|
|
937
|
+
// punishes: it walks the whole file to rebuild the timeline, or presents
|
|
938
|
+
// audio with no picture because a segment starts with nothing decodable
|
|
939
|
+
// (both field-observed 2026-08-02). Scanning cannot supply them here —
|
|
940
|
+
// the file comes off a torrent, and a full packet scan of 5.5 GB found 77
|
|
941
|
+
// keyframes in 45 s without finishing, while the container index yields
|
|
942
|
+
// all 570 in 0.8 s from two point reads (16 KB).
|
|
943
|
+
keyframeTimes = await this.#readContainerKeyframes({ sourceKey, fileIndex, inputUrl, logName });
|
|
910
944
|
keyframeMs = Date.now() - keyframeStartMs;
|
|
911
945
|
if (!keyframeTimes) {
|
|
912
946
|
logger.warn(
|
|
913
|
-
`transcode ${sessionId}:
|
|
914
|
-
`
|
|
947
|
+
`transcode ${sessionId}: no container keyframe index for "${logName}"; ` +
|
|
948
|
+
`falling back to a uniform grid — segment boundaries will not match the media`
|
|
915
949
|
);
|
|
916
950
|
}
|
|
917
951
|
} else if (hasDuration && transcodeVideo) {
|
|
@@ -1150,6 +1184,56 @@ export class HlsSessionManager {
|
|
|
1150
1184
|
* spans `[boundaries[i], boundaries[i+1])`.
|
|
1151
1185
|
* @returns {string}
|
|
1152
1186
|
*/
|
|
1187
|
+
/**
|
|
1188
|
+
* Keyframe times for a source file, from the container's own index.
|
|
1189
|
+
*
|
|
1190
|
+
* Cached per (source, file) because the answer never changes for a given
|
|
1191
|
+
* file: a second session, a re-open or a seek all reuse the first read
|
|
1192
|
+
* instead of repeating it.
|
|
1193
|
+
*
|
|
1194
|
+
* Reads byte ranges through the proxy's own /stream route, so it goes through
|
|
1195
|
+
* the same torrent piece prioritisation as everything else and needs no
|
|
1196
|
+
* separate access path.
|
|
1197
|
+
*
|
|
1198
|
+
* @param {{ sourceKey: string, fileIndex: number, inputUrl: URL, logName: string }} params
|
|
1199
|
+
* @returns {Promise<number[] | null>} Ascending seconds, or null when this
|
|
1200
|
+
* file carries no readable index.
|
|
1201
|
+
*/
|
|
1202
|
+
async #readContainerKeyframes({ sourceKey, fileIndex, inputUrl, logName }) {
|
|
1203
|
+
const cacheKey = `${sourceKey}:${fileIndex}`;
|
|
1204
|
+
if (this.keyframeIndexCache.has(cacheKey)) {
|
|
1205
|
+
return this.keyframeIndexCache.get(cacheKey);
|
|
1206
|
+
}
|
|
1207
|
+
|
|
1208
|
+
const url = inputUrl.toString();
|
|
1209
|
+
let fileSize = 0;
|
|
1210
|
+
try {
|
|
1211
|
+
const head = await fetch(url, { method: "HEAD" });
|
|
1212
|
+
fileSize = Number(head.headers.get("content-length")) || 0;
|
|
1213
|
+
} catch {
|
|
1214
|
+
return null;
|
|
1215
|
+
}
|
|
1216
|
+
if (fileSize <= 0) {
|
|
1217
|
+
return null;
|
|
1218
|
+
}
|
|
1219
|
+
|
|
1220
|
+
const readRange = async (start, end) => {
|
|
1221
|
+
try {
|
|
1222
|
+
const response = await fetch(url, { headers: { Range: `bytes=${start}-${end}` } });
|
|
1223
|
+
if (!response.ok && response.status !== 206) {
|
|
1224
|
+
return null;
|
|
1225
|
+
}
|
|
1226
|
+
return Buffer.from(await response.arrayBuffer());
|
|
1227
|
+
} catch {
|
|
1228
|
+
return null;
|
|
1229
|
+
}
|
|
1230
|
+
};
|
|
1231
|
+
|
|
1232
|
+
const times = await readKeyframeIndex({ readRange, fileSize, label: logName });
|
|
1233
|
+
this.keyframeIndexCache.set(cacheKey, times);
|
|
1234
|
+
return times;
|
|
1235
|
+
}
|
|
1236
|
+
|
|
1153
1237
|
#buildVodPlaylist(boundaries) {
|
|
1154
1238
|
const count = Math.max(0, boundaries.length - 1);
|
|
1155
1239
|
let maxDuration = 0;
|
|
@@ -2063,6 +2147,10 @@ export class HlsSessionManager {
|
|
|
2063
2147
|
// player needs a segment containing the preceding keyframe, so one that
|
|
2064
2148
|
// begins exactly at the target is useless to it.
|
|
2065
2149
|
const startIndex = Math.max(0, index - SEEK_BACKOFF_SEGMENTS);
|
|
2150
|
+
// A new seek invalidates everything the player was waiting for before it:
|
|
2151
|
+
// those requests describe where it USED to be. Clearing here is what keeps
|
|
2152
|
+
// the pull below anchored to this seek.
|
|
2153
|
+
session.lowestAwaitedIndex = -1;
|
|
2066
2154
|
logger.info(
|
|
2067
2155
|
`transcode ${session.id} viewer seek to ${positionSeconds.toFixed(1)}s → segment #${index}, ` +
|
|
2068
2156
|
`starting at #${startIndex} (${SEEK_BACKOFF_SEGMENTS} back for the preceding keyframe)`
|
|
@@ -2168,7 +2256,8 @@ export class HlsSessionManager {
|
|
|
2168
2256
|
// requests say exactly how far back it needs the keyframe, so honour that
|
|
2169
2257
|
// rather than a guess. Only ever pulls the start EARLIER, never later.
|
|
2170
2258
|
const awaited = session.lowestAwaitedIndex;
|
|
2171
|
-
const
|
|
2259
|
+
const pullFloor = Math.max(0, target - SEEK_PULL_LIMIT_SEGMENTS);
|
|
2260
|
+
const effectiveTarget = awaited >= pullFloor && awaited < target ? awaited : target;
|
|
2172
2261
|
if (effectiveTarget !== target) {
|
|
2173
2262
|
logger.info(
|
|
2174
2263
|
`transcode ${session.id} pulling encode start #${target} → #${effectiveTarget} ` +
|