@torrent-tv/proxy 2.72.1 → 2.73.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +17 -0
- package/docs/container-architecture.md +66 -0
- package/package.json +1 -1
- package/routes/api/sources/warm/post.js +19 -0
- package/routes/stream/get.js +18 -1
- package/server.js +25 -1
- package/services/container/AviContainer.js +36 -0
- package/services/container/Container.js +41 -0
- package/services/container/MatroskaContainer.js +186 -1
- package/services/container/Mp4Container.js +158 -29
- package/services/container-index/ebml-reader.js +30 -0
- package/services/hls-session-manager.js +167 -37
- package/services/orchestrators/ContainerOrchestrator.js +22 -0
- package/services/piece-store/shared-piece-store.js +1512 -1502
- package/services/torrent-pool.js +46 -0
- package/services/torrent-worker/client.js +21 -0
- package/services/torrent-worker/container-tracks.js +134 -0
- package/services/torrent-worker/fastest-wires.js +29 -6
- package/services/torrent-worker/piece-reader.js +11 -4
- package/services/torrent-worker/pool-adapter.js +35 -0
- package/services/torrent-worker/protocol.js +12 -0
- package/services/torrent-worker/worker.js +45 -1
- package/test/container-media-info.test.js +228 -0
- package/test/piece-store-reservations.test.js +31 -8
- package/test/resume-warm.test.js +39 -0
- package/test/tail-duplication.test.js +48 -4
package/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,20 @@
|
|
|
1
|
+
## 2.73.0
|
|
2
|
+
|
|
3
|
+
- **New**: `Container.readMediaInfo()` — what a file declares about ITSELF: its format, its duration, and where its own timeline begins. Matroska reads Duration and TimestampScale from `Info` and takes the start from the first Cluster's timestamp (RFC 9559 states no start-time element, so that IS the start); MP4 reads `mvhd` and the empty edit of `elst` (ISO/IEC 14496-12 §8.6.6); AVI multiplies microseconds per frame by the frame count. `null` means the container does not declare the field, which is a final answer rather than "ask someone else".
|
|
4
|
+
- **Fix**: Session creation no longer spends eight seconds learning where a soundtrack shipped as its own file begins. It spawned an ffmpeg against this proxy's own `/stream` and waited out the probe's whole 8 s budget — measured in the field on 2026-09-03 at 8078 ms, 8112 ms and 8121 ms, three cold starts out of three. The container layer had read the same header of the same file **in 8 ms, in the same second**: two readers of one header, a thousandfold apart. Worse, the eight seconds bought nothing — the probe's early exit is gated on a parsed DURATION, and a partially downloaded file prints `Duration: N/A` with the start time on that very line, so the answer was in hand from the first second. The value now comes from the container reading, and creating a session does not wait for it at all: what is known is used, the reading runs behind, and the encode run takes the freshest value at spawn — the same shape `session.keyframeTimes` already had. Where the file has not said, the two timelines are taken to AGREE rather than assumed to differ, because asserting an offset nobody has read is inventing a number.
|
|
5
|
+
- **Chore**: Checked against ffmpeg by hand rather than assumed. A Matroska file offset by 0.130435 s: ffmpeg reported `Duration: 00:00:02.13, start: 0.130000`, this reader `durationSeconds 2.131, startTimeSeconds 0.13`. The same content as MP4: `start: 0.000000` against 0. Six checks in `test/container-media-info.test.js` build their fixtures byte by byte instead of running an encoder, so the suite measures the parsing and not the machine.
|
|
6
|
+
- **New**: `/progress` carries `inputBytes` — the bytes the swarm has delivered to THIS session's own input read. It is the only sign of life a session can give before its first frame exists, because `processedSeconds` cannot move until the decoder has one. Field 2026-09-03: a cold retry blocked 46.3 s on a single piece with `processedSeconds` frozen at the start position throughout, and the browser declared the proxy dead 0.4 s before the piece landed. Counted per SESSION and never per torrent — in that same episode the torrent received 55.9 MB while the picture's own reads received 4.5 MB of it, so a torrent-wide figure would have called a starved session healthy.
|
|
7
|
+
- **New**: `POST /api/sources/:key/warm` accepts `positionSeconds` and fetches the region under it. The warm-up fetched a file's two EDGES, because that is what the codec probe reads; the region the viewer will actually resume at was asked for by nobody until the encoder opened its input — 53 s after the Retry button in the field case, and it then took another 46 s. Where that region lies is worked out from the file's own length and duration, which is the first thing `readMediaInfo` is used for beyond its own purpose. Started after the edges and never awaited by them: the edges gate the playback plan.
|
|
8
|
+
- **Fix**: A blocked reader's tail is duplicated on every attempt, not only on the attempts where no ordinary request could be placed. A read ends when its LAST block arrives, and that block is reserved to one wire whether or not other blocks are still free to ask for. Field 2026-09-03: 46.3 s on one piece, ordinary requests placed on 54 of 87 attempts, a tail of 3 blocks of 512 held by wires at 51-99 KB/s to the very end, and duplication — which ran only on the attempts that placed nothing — managed 5 blocks in the whole wait. Bounded by the tail's own length: 16 blocks (256 KB) against a 4-16 MB piece, above which the piece is still arriving normally and a second copy would spend the shared link on bytes already on their way.
|
|
9
|
+
- **Fix**: One wire refusing a duplicate no longer ends the pass. Pipelines are per wire; stopping at the first refusal rested on the stated reasoning that the remaining wires were "no emptier", which is an assumption about other peers' queues that nothing here measures.
|
|
10
|
+
- **Chore**: `docs/container-architecture.md` states the boundary outright — a fact the container declares is read from the container, a fact only the media has is measured from the media — with the three jobs that stay ffmpeg's and the measurements behind each. It also records why byte access does not belong on a track, and names the one thing that is genuinely split (a subtitle track carries byte positions while the reading of them lives in the worker) together with the obstacle to closing it: the track table fetches what is missing while the cue walk deliberately reads only what is downloaded, and one container instance holds one reader.
|
|
11
|
+
|
|
12
|
+
## 2.72.2
|
|
13
|
+
|
|
14
|
+
- **Fix**: A claim for room in the piece store could never end. Two waits inside one function shared a single field for when the wait began — the wait for the disk, and the wait for a piece that may be evicted — and each of them zeroed that field on giving up, which restarted the other's clock. Measured 2026-09-03: the claim cycled for ever, five seconds per side, `grewWaitingForDisk` and `waitedForPins` each climbing once every five seconds while `blockedByPins` stayed at 0, so the refusal the store is supposed to end with was unreachable. A claim's patience is now held by the claim itself and not by the store, which is also correct with several claimants: one caller giving up used to reset the wait of every other. A claim that cannot be met now fails after 5 s — measured 5025 ms — with `Every resident piece is pinned and nothing moved`.
|
|
15
|
+
- **Chore**: The whole suite can finish again. `test/piece-store-reservations.test.js` did not fail on the defect above, it HUNG, so `node --test` never completed at all and no release since 2.72.0 had been covered by a full run. The three checks that drive a held disk write now carry a 30 s bound, so a return of this defect is a failure in seconds rather than a stoppage.
|
|
16
|
+
- **Chore**: Two further checks in that file were failing and nobody could see it, because the file stopped before reaching them. Measured on the unmodified code, both fail there identically, so neither is from this release. One asked whether the claim was waiting by reading `waitedForPins` alone, while the store was demonstrably waiting for the disk and counting `waitedForDisk`; it now accepts either. The other gave the store three blocks and then required a piece to be admitted while a block was in flight, which is a full store by the store's own accounting — it has four now, so what the check measures is what it was written for: a completing spill must not put the stale copy back on disk.
|
|
17
|
+
|
|
1
18
|
## 2.72.1
|
|
2
19
|
|
|
3
20
|
- **Fix**: Subtitles on an embedded ASS track were shown to the viewer as the whole dialogue row — `21,0,Default,,0000,0000,0000,,I am the powerful Demon King of the Sixth Heaven.` — because one function both unwrapped the container's framing and stripped the codec's markup, and decided which framing it held by counting commas. Matroska writes eight fields before the text and takes the two timing fields out into the block's own timestamp (`matroska.org/technical/subtitles.html`); a row in a `.ass` FILE has nine and states its own column order in `[Events]`. The function expected the file's shape, found nine fields where it wanted more than nine, and returned the row untouched. The test that covered this passed because its fixture carried the file's two timestamps — a shape that never occurs on that path.
|
|
@@ -28,6 +28,72 @@ read from another file, and a raw `.srt` needs no `Container` subclass because
|
|
|
28
28
|
it has no track table and no index to read: the whole file is the payload, and
|
|
29
29
|
`SubtitleController` already reads it as such.
|
|
30
30
|
|
|
31
|
+
## Who answers what — the rule
|
|
32
|
+
|
|
33
|
+
A fact the container DECLARES is read from the container. A fact only the media
|
|
34
|
+
itself has is measured from the media, which means ffmpeg.
|
|
35
|
+
|
|
36
|
+
That is the whole boundary, and it is not about speed. Speed is a consequence:
|
|
37
|
+
this layer asks for the smallest region that holds the answer — 64 KB of header,
|
|
38
|
+
the Cues block, a sample table — while ffmpeg cannot be asked for a bounded
|
|
39
|
+
region at all. Its input analysis pulls megabytes before it will say anything,
|
|
40
|
+
and over a torrent those megabytes may not exist yet. Measured 2026-09-03 on one
|
|
41
|
+
`.mka`: this layer read its header in **8 ms**, and an ffmpeg reading the same
|
|
42
|
+
header of the same file through the proxy's own `/stream`, in the same second,
|
|
43
|
+
took **8121 ms** — and spent all of it waiting for a DURATION its caller did not
|
|
44
|
+
want, because its early exit is gated on one.
|
|
45
|
+
|
|
46
|
+
So ffmpeg keeps exactly three jobs, and nothing else:
|
|
47
|
+
|
|
48
|
+
1. producing media — the encode run;
|
|
49
|
+
2. measuring THIS MACHINE — encoder detection and its strict test, decode
|
|
50
|
+
calibration, the contention penalty;
|
|
51
|
+
3. answering what the container does not declare — above all keyframe positions
|
|
52
|
+
in a container with no index, where a packet scan is the only source. Even
|
|
53
|
+
there the container is asked FIRST: measured, the container index gave 570
|
|
54
|
+
keyframes in 0.8 s from two point reads of 16 KB, while a scan of the same
|
|
55
|
+
file found 77 in 45 s and did not finish.
|
|
56
|
+
|
|
57
|
+
`null` in `ContainerMediaInfo` means the container does not declare the field.
|
|
58
|
+
That is a final answer about the container, and the point at which a caller may
|
|
59
|
+
go to the media — not "unknown, ask again".
|
|
60
|
+
|
|
61
|
+
## Where byte access lives, and why not on a track
|
|
62
|
+
|
|
63
|
+
A `ContainerTrack` is a DECLARATION. It carries no `readRange` and no file
|
|
64
|
+
identity, and it should not: byte access is the `Container`'s, injected as
|
|
65
|
+
`readRange(start, end)` and bound to one file.
|
|
66
|
+
|
|
67
|
+
The obvious-looking improvement — hand the track a reader so it can fetch its own
|
|
68
|
+
bytes — was reviewed on 2026-09-03 and is **not** the right shape:
|
|
69
|
+
|
|
70
|
+
- for video and audio, this layer never reads the payload at all. It goes to
|
|
71
|
+
ffmpeg by URL, where seeking and gigabytes belong. A `readRange` on a
|
|
72
|
+
`VideoTrack` would be a capability with no consumer, inviting reads of a size
|
|
73
|
+
this path is not built for;
|
|
74
|
+
- tracks cross the worker boundary as PLAIN OBJECTS (`plainTrack()`), because a
|
|
75
|
+
class instance does not survive it as a class. A back-reference to a container
|
|
76
|
+
cannot cross either, so such a track would be able to read its own bytes only
|
|
77
|
+
on the thread where the container is already at hand.
|
|
78
|
+
|
|
79
|
+
What IS split, and is worth closing: `SubtitleTrack` carries `clusterPositions`
|
|
80
|
+
and `samples` — byte POSITIONS — while the reading of those positions lives in
|
|
81
|
+
`torrent-worker/subtitle-cues.js`, which builds a `readRange` of its own. Two
|
|
82
|
+
halves of one action in two layers.
|
|
83
|
+
|
|
84
|
+
The shape that closes it is `Container.readCuesOf(track)` — the container already
|
|
85
|
+
holds `readRange`, and the series `readTracks` / `readKeyframeIndex` /
|
|
86
|
+
`readMediaInfo` / `cueTextOf` is exactly where "ask the container" belongs.
|
|
87
|
+
|
|
88
|
+
**It is not done yet, and the obstacle is real rather than effort.** The two
|
|
89
|
+
readers want different read POLICIES over the same file: the track table fetches
|
|
90
|
+
what is missing from the swarm (`readFetching`), while the cue walk deliberately
|
|
91
|
+
reads only what is already downloaded (`readHeld`) so that turning subtitles on
|
|
92
|
+
never pulls bytes the viewer is not waiting for. One container instance per file
|
|
93
|
+
holds one `readRange`, so as things stand it cannot serve both. Resolving that —
|
|
94
|
+
a read policy per call, or something else — is the design question to answer
|
|
95
|
+
before the move, and answering it in passing would settle it by accident.
|
|
96
|
+
|
|
31
97
|
## Layers
|
|
32
98
|
|
|
33
99
|
```mermaid
|
package/package.json
CHANGED
|
@@ -49,6 +49,13 @@ export async function handleApiSourceWarmPost(req, reply, { sourceRegistry, torr
|
|
|
49
49
|
const body = req.body && typeof req.body === "object" && !Array.isArray(req.body) ? req.body : {};
|
|
50
50
|
const requestedIndex = Number(body.fileIndex);
|
|
51
51
|
const fileIndex = Number.isInteger(requestedIndex) && requestedIndex >= 0 ? requestedIndex : null;
|
|
52
|
+
// Where the viewer is about to resume, if they are resuming. The edges below
|
|
53
|
+
// are what the codec probe reads; this is what the VIEWER will read, and until
|
|
54
|
+
// now nothing asked for it before the encoder did.
|
|
55
|
+
const requestedPosition = Number(body.positionSeconds);
|
|
56
|
+
const positionSeconds = Number.isFinite(requestedPosition) && requestedPosition > 0
|
|
57
|
+
? requestedPosition
|
|
58
|
+
: 0;
|
|
52
59
|
|
|
53
60
|
// Adding the torrent is what announces to the trackers and starts connecting
|
|
54
61
|
// to peers, and it is also what a magnet needs in order to fetch its
|
|
@@ -77,6 +84,18 @@ export async function handleApiSourceWarmPost(req, reply, { sourceRegistry, torr
|
|
|
77
84
|
const message = error instanceof Error ? error.message : String(error);
|
|
78
85
|
logger.warn(`warm ${sourceKey.slice(0, 8)}: file edges failed: ${message}`);
|
|
79
86
|
});
|
|
87
|
+
// Started AFTER the edges, and not awaited by them either: the edges gate
|
|
88
|
+
// the playback plan, so they must not queue behind a region nobody is
|
|
89
|
+
// reading yet. This one only has to arrive before the encoder does, and the
|
|
90
|
+
// encoder is a plan and a session away.
|
|
91
|
+
if (positionSeconds > 0 && typeof torrentPool.warmResumePosition === "function") {
|
|
92
|
+
Promise.resolve(torrentPool.warmResumePosition(torrent, fileIndex, positionSeconds)).catch(
|
|
93
|
+
(error) => {
|
|
94
|
+
const message = error instanceof Error ? error.message : String(error);
|
|
95
|
+
logger.warn(`warm ${sourceKey.slice(0, 8)}: the viewer's position failed: ${message}`);
|
|
96
|
+
}
|
|
97
|
+
);
|
|
98
|
+
}
|
|
80
99
|
}
|
|
81
100
|
|
|
82
101
|
// The files that carry this episode's OTHER soundtracks and its subtitles.
|
package/routes/stream/get.js
CHANGED
|
@@ -78,7 +78,7 @@ async function waitForTorrent(torrentPool, sourceType, source) {
|
|
|
78
78
|
* @param {{ sourceRegistry: ReturnType<import("../../store/source-registry.js").createSourceRegistry>, torrentPool: import("../../services/torrent-pool.js").TorrentPool }} deps
|
|
79
79
|
* @returns {Promise<void>}
|
|
80
80
|
*/
|
|
81
|
-
export async function handleStreamGet(req, reply, { sourceRegistry, torrentPool }) {
|
|
81
|
+
export async function handleStreamGet(req, reply, { sourceRegistry, torrentPool, noteInputBytes = null }) {
|
|
82
82
|
const fileIndexRaw = typeof req.query.fileIndex === "string" ? req.query.fileIndex : "";
|
|
83
83
|
const fileIndex = Number(fileIndexRaw);
|
|
84
84
|
const { sourceType, source } = getSourceParams(req.query, sourceRegistry);
|
|
@@ -172,6 +172,12 @@ export async function handleStreamGet(req, reply, { sourceRegistry, torrentPool
|
|
|
172
172
|
const windowBytes =
|
|
173
173
|
Number.isFinite(windowBytesRaw) && windowBytesRaw > 0 ? windowBytesRaw : undefined;
|
|
174
174
|
|
|
175
|
+
// Which transcode session this read feeds, when it feeds one. Put on the URL
|
|
176
|
+
// by the session that builds it, because this route otherwise knows only a
|
|
177
|
+
// file — and two sessions can read one file, so the file cannot stand in for
|
|
178
|
+
// the session.
|
|
179
|
+
const sessionId = typeof req.query.session === "string" ? req.query.session : "";
|
|
180
|
+
|
|
175
181
|
const fragments = typeof file.createFragmentReader === "function"
|
|
176
182
|
? file.createFragmentReader({ start, end, windowBytes })
|
|
177
183
|
: null;
|
|
@@ -201,6 +207,17 @@ export async function handleStreamGet(req, reply, { sourceRegistry, torrentPool
|
|
|
201
207
|
reply.raw.write(fragment.bytes, (error) => (error ? reject(error) : resolve()));
|
|
202
208
|
});
|
|
203
209
|
sent += fragment.bytes.length;
|
|
210
|
+
// Say so, if this read belongs to a transcode session. It is the only
|
|
211
|
+
// proof that a session which has produced nothing yet is nevertheless
|
|
212
|
+
// being fed: the encoder's own progress cannot move until its first
|
|
213
|
+
// frame is decoded, and a viewer waiting for that first frame was being
|
|
214
|
+
// told the proxy had died while the swarm was delivering to it. Field
|
|
215
|
+
// 2026-09-03: 46.3 s on one piece, `processedSeconds` frozen at the
|
|
216
|
+
// start position throughout, and the browser gave up 0.4 s before the
|
|
217
|
+
// piece landed.
|
|
218
|
+
if (noteInputBytes) {
|
|
219
|
+
noteInputBytes(sessionId, fragment.bytes.length);
|
|
220
|
+
}
|
|
204
221
|
// Only now are these bytes gone: the piece can be unpinned, and the
|
|
205
222
|
// slot it occupies reused. Releasing before this point corrupts the
|
|
206
223
|
// response silently.
|
package/server.js
CHANGED
|
@@ -193,6 +193,23 @@ export async function startProxyServer({
|
|
|
193
193
|
// The file's audio tracks, for the master playlist's rendition group. Already
|
|
194
194
|
// probed for the browser's audio menu; read from there rather than probed again.
|
|
195
195
|
getCachedAudioTracks: (params) => playbackPlanner.getCachedAudioTracks(params),
|
|
196
|
+
// What a file declares about itself, read by the container layer from the
|
|
197
|
+
// same header its track table comes from. This is how the session learns
|
|
198
|
+
// where a soundtrack shipped as its own file begins — it used to spawn an
|
|
199
|
+
// ffmpeg over this proxy's own HTTP to ask the same question of the same
|
|
200
|
+
// bytes, and that read cost 8.1 s of every cold start (field 2026-09-03).
|
|
201
|
+
getContainerMediaInfo: async ({ sourceKey, fileIndex }) => {
|
|
202
|
+
const record = sourceRegistry.get(sourceKey);
|
|
203
|
+
if (!record || typeof torrentPool.getContainerMediaInfo !== "function") {
|
|
204
|
+
return null;
|
|
205
|
+
}
|
|
206
|
+
try {
|
|
207
|
+
const torrent = await torrentPool.getTorrent(record.sourceType, record.source);
|
|
208
|
+
return await torrentPool.getContainerMediaInfo(torrent, fileIndex);
|
|
209
|
+
} catch {
|
|
210
|
+
return null;
|
|
211
|
+
}
|
|
212
|
+
},
|
|
196
213
|
// Pull one whole file onto the disk. Used for a soundtrack that ships beside
|
|
197
214
|
// the picture, once the encoder is as far ahead of the viewer as it is
|
|
198
215
|
// allowed to get — the one moment the swarm's capacity is demonstrably
|
|
@@ -260,7 +277,14 @@ export async function startProxyServer({
|
|
|
260
277
|
})
|
|
261
278
|
);
|
|
262
279
|
app.get("/stream", async (req, reply) =>
|
|
263
|
-
handleStreamGet(req, reply, {
|
|
280
|
+
handleStreamGet(req, reply, {
|
|
281
|
+
sourceRegistry,
|
|
282
|
+
torrentPool,
|
|
283
|
+
// So a session that has produced nothing yet can still show it is being
|
|
284
|
+
// fed. The route knows only files; the session id rides on the URL the
|
|
285
|
+
// session itself built.
|
|
286
|
+
noteInputBytes: (sessionId, bytes) => hlsSessionManager.noteInputBytes(sessionId, bytes)
|
|
287
|
+
})
|
|
264
288
|
);
|
|
265
289
|
app.post("/api/transcode-sessions", async (req, reply) =>
|
|
266
290
|
handleApiTranscodeSessionsPost(req, reply, { hlsSessionManager, sourceRegistry, torrentPool })
|
|
@@ -36,6 +36,42 @@ export class AviContainer extends Container {
|
|
|
36
36
|
})];
|
|
37
37
|
}
|
|
38
38
|
|
|
39
|
+
/**
|
|
40
|
+
* Duration from the main AVI header, per the RIFF AVI specification: the
|
|
41
|
+
* header states microseconds per frame and the total number of frames, and
|
|
42
|
+
* their product is the length.
|
|
43
|
+
*
|
|
44
|
+
* An AVI has no edit list and no timeline offset of any kind, so its start is
|
|
45
|
+
* zero — a declaration of the format itself, not an absence.
|
|
46
|
+
*
|
|
47
|
+
* @returns {Promise<import("./Container.js").ContainerMediaInfo>}
|
|
48
|
+
*/
|
|
49
|
+
async readMediaInfo() {
|
|
50
|
+
if (this.mediaInfo) {
|
|
51
|
+
return this.mediaInfo;
|
|
52
|
+
}
|
|
53
|
+
/** @type {import("./Container.js").ContainerMediaInfo} */
|
|
54
|
+
const info = { format: this.formatName, durationSeconds: null, startTimeSeconds: 0 };
|
|
55
|
+
this.mediaInfo = info;
|
|
56
|
+
const head = await this.readRange(0, Math.min(4095, this.fileSize - 1));
|
|
57
|
+
if (!head || !isAvi(head)) {
|
|
58
|
+
return info;
|
|
59
|
+
}
|
|
60
|
+
// RIFF("AVI ") -> LIST("hdrl") -> avih. The avih chunk's payload begins with
|
|
61
|
+
// dwMicroSecPerFrame and its fifth field is dwTotalFrames.
|
|
62
|
+
const at = head.indexOf("avih", 0, "latin1");
|
|
63
|
+
if (at < 0 || at + 8 + 20 > head.length) {
|
|
64
|
+
return info;
|
|
65
|
+
}
|
|
66
|
+
const payload = at + 8;
|
|
67
|
+
const microsecondsPerFrame = head.readUInt32LE(payload);
|
|
68
|
+
const totalFrames = head.readUInt32LE(payload + 16);
|
|
69
|
+
if (microsecondsPerFrame > 0 && totalFrames > 0) {
|
|
70
|
+
info.durationSeconds = (microsecondsPerFrame * totalFrames) / 1e6;
|
|
71
|
+
}
|
|
72
|
+
return info;
|
|
73
|
+
}
|
|
74
|
+
|
|
39
75
|
async readKeyframeIndex() {
|
|
40
76
|
const r = await readAviKeyframeTimes(this.readRange, this.fileSize);
|
|
41
77
|
if (!r) return null;
|
|
@@ -12,6 +12,19 @@
|
|
|
12
12
|
* - AVI RIFF §: LIST hdrl, idx1
|
|
13
13
|
*/
|
|
14
14
|
|
|
15
|
+
/**
|
|
16
|
+
* What one file declares about itself. Every field is either a value the
|
|
17
|
+
* container states or `null`, which means the container does not state it —
|
|
18
|
+
* a defined absence, not "nobody has looked yet".
|
|
19
|
+
*
|
|
20
|
+
* @typedef {object} ContainerMediaInfo
|
|
21
|
+
* @property {string} format - "matroska" | "mp4" | "avi" | "unknown".
|
|
22
|
+
* @property {number | null} durationSeconds
|
|
23
|
+
* @property {number | null} startTimeSeconds - Where this file's own timeline
|
|
24
|
+
* begins. Two files of one release need not agree on it, and the difference
|
|
25
|
+
* is what keeps a soundtrack shipped separately aligned with its picture.
|
|
26
|
+
*/
|
|
27
|
+
|
|
15
28
|
export class Container {
|
|
16
29
|
/**
|
|
17
30
|
* @param {object} params
|
|
@@ -52,6 +65,34 @@ export class Container {
|
|
|
52
65
|
return null;
|
|
53
66
|
}
|
|
54
67
|
|
|
68
|
+
/**
|
|
69
|
+
* What this file DECLARES about itself as a whole, as distinct from what its
|
|
70
|
+
* individual tracks declare.
|
|
71
|
+
*
|
|
72
|
+
* The rule this method exists to hold: a fact the container declares is read
|
|
73
|
+
* from the container; a fact only the media itself has is measured from the
|
|
74
|
+
* media. Both halves used to be asked of ffmpeg, so the same header was read
|
|
75
|
+
* twice — measured 2026-09-03, this layer read one `.mka` header in 8 ms while
|
|
76
|
+
* a second ffmpeg read the same header over HTTP for 8121 ms, in the same
|
|
77
|
+
* second, on the same file.
|
|
78
|
+
*
|
|
79
|
+
* `null` is not "unknown, ask someone else". It means the container does not
|
|
80
|
+
* declare the field, which is a final answer about the container and the point
|
|
81
|
+
* at which a caller may go to the media — see `docs/container-architecture.md`.
|
|
82
|
+
*
|
|
83
|
+
* @returns {Promise<import("./Container.js").ContainerMediaInfo>}
|
|
84
|
+
*/
|
|
85
|
+
async readMediaInfo() {
|
|
86
|
+
if (!this.mediaInfo) {
|
|
87
|
+
this.mediaInfo = {
|
|
88
|
+
format: this.formatName,
|
|
89
|
+
durationSeconds: null,
|
|
90
|
+
startTimeSeconds: null
|
|
91
|
+
};
|
|
92
|
+
}
|
|
93
|
+
return this.mediaInfo;
|
|
94
|
+
}
|
|
95
|
+
|
|
55
96
|
/**
|
|
56
97
|
* Subtitle-specific: where cues live (Matroska cluster positions or MP4 sample ranges).
|
|
57
98
|
* Returned via track objects' clusterPositions/samples, so base has no extra method — tracks carry it.
|
|
@@ -22,9 +22,29 @@ import { AudioTrack } from "../tracks/AudioTrack.js";
|
|
|
22
22
|
import { TextSubtitleTrack, TEXT_CODECS_MATROSKA } from "../tracks/TextSubtitleTrack.js";
|
|
23
23
|
import { ImageSubtitleTrack } from "../tracks/ImageSubtitleTrack.js";
|
|
24
24
|
import { ContainerTrack } from "../tracks/ContainerTrack.js";
|
|
25
|
-
import { findElement, iterateElements, readUint } from "../container-index/ebml-reader.js";
|
|
25
|
+
import { findElement, iterateElements, readFloat, readUint } from "../container-index/ebml-reader.js";
|
|
26
26
|
|
|
27
27
|
const HEAD_BYTES = 64 * 1024;
|
|
28
|
+
const ID_SEGMENT = 0x18538067;
|
|
29
|
+
const ID_SEEK_HEAD = 0x114d9b74;
|
|
30
|
+
const ID_SEEK = 0x4dbb;
|
|
31
|
+
const ID_SEEK_ID = 0x53ab;
|
|
32
|
+
const ID_SEEK_POSITION = 0x53ac;
|
|
33
|
+
const ID_INFO = 0x1549a966;
|
|
34
|
+
const ID_TIMESTAMP_SCALE = 0x2ad7b1;
|
|
35
|
+
const ID_DURATION = 0x4489;
|
|
36
|
+
const ID_CLUSTER = 0x1f43b675;
|
|
37
|
+
const ID_TIMESTAMP = 0xe7;
|
|
38
|
+
const ID_SIMPLE_BLOCK = 0xa3;
|
|
39
|
+
const ID_BLOCK_GROUP = 0xa0;
|
|
40
|
+
/** RFC 9559 §5.1.2.1: nanoseconds per tick when Info omits TimestampScale. */
|
|
41
|
+
const DEFAULT_TIMESTAMP_SCALE = 1_000_000;
|
|
42
|
+
/**
|
|
43
|
+
* How much to read at a cluster whose position came from the SeekHead. A
|
|
44
|
+
* cluster's Timestamp is the first child every muxer writes, so this only has
|
|
45
|
+
* to cover the element header and that one field.
|
|
46
|
+
*/
|
|
47
|
+
const CLUSTER_PROBE_BYTES = 4 * 1024;
|
|
28
48
|
const ID_TRACKS = 0x1654ae6b;
|
|
29
49
|
const ID_TRACK_ENTRY = 0xae;
|
|
30
50
|
const ID_TRACK_NUMBER = 0xd7;
|
|
@@ -70,6 +90,171 @@ export class MatroskaContainer extends Container {
|
|
|
70
90
|
return isMatroska(head);
|
|
71
91
|
}
|
|
72
92
|
|
|
93
|
+
/**
|
|
94
|
+
* Duration and the start of this file's own timeline, per RFC 9559 §5.1.2.
|
|
95
|
+
*
|
|
96
|
+
* Duration is stated in `Info` as a FLOAT in ticks, so it needs the file's
|
|
97
|
+
* `TimestampScale` to become seconds. The start of the timeline is not stated
|
|
98
|
+
* anywhere — Matroska has no such element — so it is the timestamp of the
|
|
99
|
+
* first Cluster, which is what the first frame is placed against.
|
|
100
|
+
*
|
|
101
|
+
* @returns {Promise<import("./Container.js").ContainerMediaInfo>}
|
|
102
|
+
*/
|
|
103
|
+
async readMediaInfo() {
|
|
104
|
+
if (this.mediaInfo) {
|
|
105
|
+
return this.mediaInfo;
|
|
106
|
+
}
|
|
107
|
+
/** @type {import("./Container.js").ContainerMediaInfo} */
|
|
108
|
+
const info = { format: this.formatName, durationSeconds: null, startTimeSeconds: null };
|
|
109
|
+
this.mediaInfo = info;
|
|
110
|
+
const head = await this.readRange(0, Math.min(HEAD_BYTES - 1, this.fileSize - 1));
|
|
111
|
+
if (!head || !isMatroska(head)) {
|
|
112
|
+
return info;
|
|
113
|
+
}
|
|
114
|
+
const segment = findElement(head, ID_SEGMENT, []);
|
|
115
|
+
if (!segment) {
|
|
116
|
+
return info;
|
|
117
|
+
}
|
|
118
|
+
const scale = MatroskaContainer.#timestampScaleOf(head, segment.dataOffset);
|
|
119
|
+
const infoElement = findElement(head, ID_INFO, [], segment.dataOffset);
|
|
120
|
+
if (infoElement) {
|
|
121
|
+
const infoEnd = Math.min(head.length, infoElement.dataOffset + infoElement.size);
|
|
122
|
+
for (const field of iterateElements(head, infoElement.dataOffset, infoEnd)) {
|
|
123
|
+
if (field.id !== ID_DURATION) {
|
|
124
|
+
continue;
|
|
125
|
+
}
|
|
126
|
+
const ticks = readFloat(head, field.dataOffset, field.size);
|
|
127
|
+
if (ticks !== null && ticks > 0) {
|
|
128
|
+
info.durationSeconds = (ticks * scale) / 1e9;
|
|
129
|
+
}
|
|
130
|
+
break;
|
|
131
|
+
}
|
|
132
|
+
}
|
|
133
|
+
info.startTimeSeconds = await this.#firstClusterSeconds(head, segment.dataOffset, scale);
|
|
134
|
+
return info;
|
|
135
|
+
}
|
|
136
|
+
|
|
137
|
+
/**
|
|
138
|
+
* `TimestampScale` from Info, or the specification's default.
|
|
139
|
+
*
|
|
140
|
+
* @param {Buffer} head
|
|
141
|
+
* @param {number} segmentDataOffset
|
|
142
|
+
* @returns {number} Nanoseconds per tick.
|
|
143
|
+
*/
|
|
144
|
+
static #timestampScaleOf(head, segmentDataOffset) {
|
|
145
|
+
const infoElement = findElement(head, ID_INFO, [], segmentDataOffset);
|
|
146
|
+
if (!infoElement) {
|
|
147
|
+
return DEFAULT_TIMESTAMP_SCALE;
|
|
148
|
+
}
|
|
149
|
+
const infoEnd = Math.min(head.length, infoElement.dataOffset + infoElement.size);
|
|
150
|
+
for (const field of iterateElements(head, infoElement.dataOffset, infoEnd)) {
|
|
151
|
+
if (field.id === ID_TIMESTAMP_SCALE) {
|
|
152
|
+
const scale = readUint(head, field.dataOffset, field.size);
|
|
153
|
+
return scale > 0 ? scale : DEFAULT_TIMESTAMP_SCALE;
|
|
154
|
+
}
|
|
155
|
+
}
|
|
156
|
+
return DEFAULT_TIMESTAMP_SCALE;
|
|
157
|
+
}
|
|
158
|
+
|
|
159
|
+
/**
|
|
160
|
+
* The timestamp of the first Cluster, in seconds.
|
|
161
|
+
*
|
|
162
|
+
* Tried in the head window first, because a muxer writes the first cluster
|
|
163
|
+
* straight after Tracks and both usually fit; a file whose Tracks element is
|
|
164
|
+
* large enough to push it out is answered from the SeekHead instead, with one
|
|
165
|
+
* short read at the position it names.
|
|
166
|
+
*
|
|
167
|
+
* @param {Buffer} head
|
|
168
|
+
* @param {number} segmentDataOffset
|
|
169
|
+
* @param {number} scale - Nanoseconds per tick.
|
|
170
|
+
* @returns {Promise<number | null>} Null when no cluster could be read.
|
|
171
|
+
*/
|
|
172
|
+
async #firstClusterSeconds(head, segmentDataOffset, scale) {
|
|
173
|
+
/**
|
|
174
|
+
* @param {Buffer} buffer
|
|
175
|
+
* @param {number} dataOffset
|
|
176
|
+
* @param {number} end
|
|
177
|
+
* @returns {number | null}
|
|
178
|
+
*/
|
|
179
|
+
const timestampIn = (buffer, dataOffset, end) => {
|
|
180
|
+
for (const field of iterateElements(buffer, dataOffset, end)) {
|
|
181
|
+
if (field.id === ID_TIMESTAMP) {
|
|
182
|
+
const ticks = readUint(buffer, field.dataOffset, field.size);
|
|
183
|
+
return Number.isFinite(ticks) ? (ticks * scale) / 1e9 : null;
|
|
184
|
+
}
|
|
185
|
+
// Timestamp is written before any frame. Stopping at the first one keeps
|
|
186
|
+
// this from walking a cluster's whole payload, which is megabytes and
|
|
187
|
+
// usually not in the buffer at all.
|
|
188
|
+
if (field.id === ID_SIMPLE_BLOCK || field.id === ID_BLOCK_GROUP) {
|
|
189
|
+
return null;
|
|
190
|
+
}
|
|
191
|
+
}
|
|
192
|
+
return null;
|
|
193
|
+
};
|
|
194
|
+
|
|
195
|
+
for (const element of iterateElements(head, segmentDataOffset, head.length)) {
|
|
196
|
+
if (element.id !== ID_CLUSTER) {
|
|
197
|
+
continue;
|
|
198
|
+
}
|
|
199
|
+
return timestampIn(head, element.dataOffset, Math.min(head.length, element.dataOffset + element.size));
|
|
200
|
+
}
|
|
201
|
+
|
|
202
|
+
const position = MatroskaContainer.#seekPositionOf(head, segmentDataOffset, ID_CLUSTER);
|
|
203
|
+
if (position === null) {
|
|
204
|
+
return null;
|
|
205
|
+
}
|
|
206
|
+
const at = segmentDataOffset + position;
|
|
207
|
+
if (at >= this.fileSize) {
|
|
208
|
+
return null;
|
|
209
|
+
}
|
|
210
|
+
const chunk = await this.readRange(at, Math.min(this.fileSize - 1, at + CLUSTER_PROBE_BYTES - 1));
|
|
211
|
+
if (!chunk) {
|
|
212
|
+
return null;
|
|
213
|
+
}
|
|
214
|
+
for (const element of iterateElements(chunk, 0, chunk.length)) {
|
|
215
|
+
if (element.id !== ID_CLUSTER) {
|
|
216
|
+
continue;
|
|
217
|
+
}
|
|
218
|
+
return timestampIn(chunk, element.dataOffset, Math.min(chunk.length, element.dataOffset + element.size));
|
|
219
|
+
}
|
|
220
|
+
return null;
|
|
221
|
+
}
|
|
222
|
+
|
|
223
|
+
/**
|
|
224
|
+
* Where the SeekHead says an element lives, relative to the Segment's payload.
|
|
225
|
+
*
|
|
226
|
+
* @param {Buffer} head
|
|
227
|
+
* @param {number} segmentDataOffset
|
|
228
|
+
* @param {number} wantedId
|
|
229
|
+
* @returns {number | null}
|
|
230
|
+
*/
|
|
231
|
+
static #seekPositionOf(head, segmentDataOffset, wantedId) {
|
|
232
|
+
const seekHead = findElement(head, ID_SEEK_HEAD, [], segmentDataOffset);
|
|
233
|
+
if (!seekHead) {
|
|
234
|
+
return null;
|
|
235
|
+
}
|
|
236
|
+
const seekHeadEnd = Math.min(head.length, seekHead.dataOffset + seekHead.size);
|
|
237
|
+
for (const seek of iterateElements(head, seekHead.dataOffset, seekHeadEnd)) {
|
|
238
|
+
if (seek.id !== ID_SEEK) {
|
|
239
|
+
continue;
|
|
240
|
+
}
|
|
241
|
+
const seekEnd = Math.min(seekHeadEnd, seek.dataOffset + seek.size);
|
|
242
|
+
let targetId = null;
|
|
243
|
+
let position = null;
|
|
244
|
+
for (const field of iterateElements(head, seek.dataOffset, seekEnd)) {
|
|
245
|
+
if (field.id === ID_SEEK_ID) {
|
|
246
|
+
targetId = readUint(head, field.dataOffset, field.size);
|
|
247
|
+
} else if (field.id === ID_SEEK_POSITION) {
|
|
248
|
+
position = readUint(head, field.dataOffset, field.size);
|
|
249
|
+
}
|
|
250
|
+
}
|
|
251
|
+
if (targetId === wantedId && position !== null) {
|
|
252
|
+
return position;
|
|
253
|
+
}
|
|
254
|
+
}
|
|
255
|
+
return null;
|
|
256
|
+
}
|
|
257
|
+
|
|
73
258
|
async readTracks() {
|
|
74
259
|
const head = await this.readRange(0, Math.min(HEAD_BYTES - 1, this.fileSize - 1));
|
|
75
260
|
if (!head || !isMatroska(head)) return [];
|