@torrent-tv/proxy 2.41.0 → 2.43.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +10 -0
- package/package.json +1 -1
- package/routes/api/subtitles/get.js +133 -0
- package/services/container-index/matroska-blocks.js +202 -0
- package/services/container-index/matroska-subtitles.js +262 -0
- package/services/container-index/mp4-subtitles.js +380 -0
- package/services/torrent-worker/client.js +20 -0
- package/services/torrent-worker/pool-adapter.js +32 -0
- package/services/torrent-worker/protocol.js +4 -0
- package/services/torrent-worker/subtitle-cues.js +291 -0
- package/services/torrent-worker/worker.js +19 -0
- package/test/matroska-blocks.test.js +0 -0
- package/test/mp4-subtitles.test.js +127 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,13 @@
|
|
|
1
|
+
## 2.43.0
|
|
2
|
+
|
|
3
|
+
- **New**: An MP4's text subtitles are read the same way, and more cheaply than Matroska's. Where a Matroska cue costs whatever cluster holds it — the picture around it included — an MP4 states every sample's own byte range in its sample table (ISO/IEC 14496-12 §8.6.1.2, §8.7.3-8.7.5), so a cue costs its own few dozen bytes and nothing else. The tables are read out of the `moov` the keyframe reader already fetches: `stts` for when each cue starts and how long it lasts, `stsz` for its length, `stsc` with `stco`/`co64` for where its bytes are. `tx3g` (3GPP timed text) and `wvtt` (WebVTT in MP4) are decoded; `stpp` (TTML) is XML and is deliberately left out rather than half-shown. An empty sample is the format's way of saying nothing is on screen and is not turned into a blank cue. Same rule as before: only samples whose bytes are already downloaded are read, so a cue never costs a request.
|
|
4
|
+
- **Chore**: The MP4 reader has its own tests over a file built byte by byte — the sample table walked into times and offsets, the gap sample dropped, a `wvtt` payload decoded, and a file with no text track answering with nothing.
|
|
5
|
+
|
|
6
|
+
## 2.42.0
|
|
7
|
+
|
|
8
|
+
- **New**: Embedded text subtitles are read out of the clusters the film is already downloading, and no longer extracted with ffmpeg. Measured 2026-08-19 on `Minions.and.Monsters.1080p.mkv`: the browser asked for a track, gave up at its own 60 s limit, and the proxy answered **752 seconds later** with 3040 bytes — because a subtitle stream is sparse and ffmpeg walks the whole container whatever range is asked of it. Measured twice more to be sure: `-ss 1200 -t 4` read to the end of the file and pulled the download from 2.7 % to 81 % of 6.5 GB, and `-copyts -ss 600 -to 604` took 154 s on a copy already 81 % local and still emitted the whole track. A subtitle block sits in the same cluster as the picture around it, so those clusters are in hand anyway: the cue points of the subtitle track name them, the blocks are read where every piece covering them is already downloaded, and nothing is requested from the swarm. **Cost: zero extra bytes**, and the cues for the part being watched are ready before the viewer reaches it — which is the rule this was held to, subtitles arriving like the picture or not at all. On the field file the plan reads in 3.8 s over the swarm and names all four tracks with their languages, and the cues come out with their real times (`118.41s → 125.71s «МАГИЯ ГОЛЛИВУДА»`). A file this cannot be read from falls back to the old extraction, unchanged. `S_TEXT/UTF8` needs no conversion; `S_TEXT/ASS` and `S_TEXT/SSA` have their dialogue fields stripped; image subtitles (PGS, VobSub) are deliberately not offered, since this path cannot show them.
|
|
9
|
+
- **Chore**: The Matroska block reader is its own module with its own tests (`services/container-index/matroska-blocks.js`): cluster time plus the block's own offset, the duration out of the block group, other tracks skipped, negative offsets placed correctly, and lacing stepped over rather than read as text.
|
|
10
|
+
|
|
1
11
|
## 2.41.0
|
|
2
12
|
|
|
3
13
|
- **Fix**: An embedded subtitle track is prepared in the background and kept, instead of being extracted afresh inside a request the browser cannot hold open. Extracting one makes ffmpeg read the WHOLE film, because subtitles are interleaved through it — measured 2026-08-19 on a release with three tracks: track 0 produced **3040 bytes over 752 seconds**, track 1 76 KB over 193 s, track 2 68 KB over 55 s, with the data channel idle throughout (`maxBuffered=0`, the time all in reading the body). The browser gives up at sixty seconds, and every retry started the same twelve-minute scan again, so the first track never arrived at all. The route now starts the work once per `(source, file, track)`, answers `202 { pending: true }` while it runs, and serves the kept result the moment it exists. The scan still takes what it takes; what changes is that it happens once and its result is not thrown away.
|
package/package.json
CHANGED
|
@@ -96,6 +96,27 @@ export async function handleApiSubtitlesGet(req, reply, { sourceRegistry, torren
|
|
|
96
96
|
return reply.code(400).send({ error: "trackIndex must be a non-negative integer." });
|
|
97
97
|
}
|
|
98
98
|
|
|
99
|
+
// From the clusters the viewer has already downloaded, if this file can be
|
|
100
|
+
// read that way. Costs no network at all and answers with the part of the
|
|
101
|
+
// film they are watching; the rest arrives as they watch it. Only when the
|
|
102
|
+
// container cannot be read this way does the old extraction run.
|
|
103
|
+
const held = await cuesFromDownloadedClusters(torrentPool, torrent, fileIndex, trackIndex);
|
|
104
|
+
if (held !== null) {
|
|
105
|
+
setLanguageHeaders(reply, held.language);
|
|
106
|
+
reply.header("content-type", "text/vtt; charset=utf-8");
|
|
107
|
+
reply.header("cache-control", "no-store");
|
|
108
|
+
reply.header("access-control-allow-origin", "*");
|
|
109
|
+
// How much of the film these cues cover, so the browser knows to ask again
|
|
110
|
+
// as playback moves into clusters that were not downloaded yet.
|
|
111
|
+
reply.header("X-Subtitle-Covered-Clusters", String(held.coveredClusters));
|
|
112
|
+
reply.header("X-Subtitle-Indexed-Clusters", String(held.indexedClusters));
|
|
113
|
+
reply.raw.setHeader(
|
|
114
|
+
"Access-Control-Expose-Headers",
|
|
115
|
+
"X-Subtitle-Language, X-Subtitle-Language-Name, X-Subtitle-Covered-Clusters, X-Subtitle-Indexed-Clusters"
|
|
116
|
+
);
|
|
117
|
+
return reply.send(held.vtt);
|
|
118
|
+
}
|
|
119
|
+
|
|
99
120
|
const inputUrl = new URL("/stream", `${localBaseUrl}/`);
|
|
100
121
|
inputUrl.searchParams.set("sourceKey", sourceKey);
|
|
101
122
|
inputUrl.searchParams.set("fileIndex", String(fileIndex));
|
|
@@ -215,3 +236,115 @@ function readFileFully(file, maxBytes) {
|
|
|
215
236
|
stream.on("error", reject);
|
|
216
237
|
});
|
|
217
238
|
}
|
|
239
|
+
|
|
240
|
+
/**
|
|
241
|
+
* The cues of a track from clusters already downloaded, as WebVTT.
|
|
242
|
+
*
|
|
243
|
+
* `trackIndex` is the browser's number for the subtitle stream — its position
|
|
244
|
+
* among the subtitle streams, as ffprobe lists them — while Matroska blocks
|
|
245
|
+
* carry the file's own track number. The plan lists the text tracks in file
|
|
246
|
+
* order, so the browser's Nth subtitle stream is the Nth entry.
|
|
247
|
+
*
|
|
248
|
+
* @param {object} torrentPool
|
|
249
|
+
* @param {object} torrent
|
|
250
|
+
* @param {number} fileIndex
|
|
251
|
+
* @param {number} trackIndex
|
|
252
|
+
* @returns {Promise<{ vtt: string, language: object | null, coveredClusters: number, indexedClusters: number } | null>}
|
|
253
|
+
* Null when this file cannot be read this way, and then the caller falls back.
|
|
254
|
+
*/
|
|
255
|
+
async function cuesFromDownloadedClusters(torrentPool, torrent, fileIndex, trackIndex) {
|
|
256
|
+
if (typeof torrentPool?.getSubtitleTracks !== "function") {
|
|
257
|
+
return null;
|
|
258
|
+
}
|
|
259
|
+
let tracks;
|
|
260
|
+
try {
|
|
261
|
+
tracks = await torrentPool.getSubtitleTracks(torrent, fileIndex);
|
|
262
|
+
} catch {
|
|
263
|
+
return null;
|
|
264
|
+
}
|
|
265
|
+
const track = Array.isArray(tracks) ? tracks[trackIndex] : null;
|
|
266
|
+
if (!track) {
|
|
267
|
+
return null;
|
|
268
|
+
}
|
|
269
|
+
let held;
|
|
270
|
+
try {
|
|
271
|
+
held = await torrentPool.getSubtitleCues(torrent, fileIndex, track.trackNumber);
|
|
272
|
+
} catch {
|
|
273
|
+
return null;
|
|
274
|
+
}
|
|
275
|
+
if (!held || !Array.isArray(held.cues)) {
|
|
276
|
+
return null;
|
|
277
|
+
}
|
|
278
|
+
const vtt = cuesToVtt(held.cues, held.codecId);
|
|
279
|
+
const language = held.cues.length > 0
|
|
280
|
+
? detectLanguage(held.cues.map((cue) => cue.text).join("\n"))
|
|
281
|
+
: null;
|
|
282
|
+
return {
|
|
283
|
+
vtt,
|
|
284
|
+
language,
|
|
285
|
+
coveredClusters: held.coveredClusters ?? 0,
|
|
286
|
+
indexedClusters: held.indexedClusters ?? 0
|
|
287
|
+
};
|
|
288
|
+
}
|
|
289
|
+
|
|
290
|
+
/**
|
|
291
|
+
* WebVTT from cues read out of the container.
|
|
292
|
+
*
|
|
293
|
+
* A cue with no duration — a SimpleBlock, which subtitles rarely use — is given
|
|
294
|
+
* the time until the next one, and the last such cue a few seconds. That is not
|
|
295
|
+
* an invention about the film: it is what a player does with an open-ended cue,
|
|
296
|
+
* made explicit here so the file is valid.
|
|
297
|
+
*
|
|
298
|
+
* @param {{ startSeconds: number, endSeconds: number | null, text: string }[]} cues
|
|
299
|
+
* @param {string} codecId
|
|
300
|
+
* @returns {string}
|
|
301
|
+
*/
|
|
302
|
+
function cuesToVtt(cues, codecId) {
|
|
303
|
+
const lines = ["WEBVTT", ""];
|
|
304
|
+
const isAss = codecId === "S_TEXT/ASS" || codecId === "S_TEXT/SSA";
|
|
305
|
+
cues.forEach((cue, index) => {
|
|
306
|
+
const next = cues[index + 1];
|
|
307
|
+
const end = cue.endSeconds ?? (next ? next.startSeconds : cue.startSeconds + 4);
|
|
308
|
+
const text = isAss ? assDialogueToText(cue.text) : cue.text.trim();
|
|
309
|
+
if (!text) {
|
|
310
|
+
return;
|
|
311
|
+
}
|
|
312
|
+
lines.push(`${vttTime(cue.startSeconds)} --> ${vttTime(end)}`);
|
|
313
|
+
lines.push(text);
|
|
314
|
+
lines.push("");
|
|
315
|
+
});
|
|
316
|
+
return lines.join("\n");
|
|
317
|
+
}
|
|
318
|
+
|
|
319
|
+
/**
|
|
320
|
+
* The visible text of an ASS dialogue row.
|
|
321
|
+
*
|
|
322
|
+
* A block carries the fields after `Dialogue:` without their header — nine of
|
|
323
|
+
* them, then the text, which itself holds override groups in braces.
|
|
324
|
+
*
|
|
325
|
+
* @param {string} raw
|
|
326
|
+
* @returns {string}
|
|
327
|
+
*/
|
|
328
|
+
function assDialogueToText(raw) {
|
|
329
|
+
const fields = raw.split(",");
|
|
330
|
+
const text = fields.length > 9 ? fields.slice(9).join(",") : raw;
|
|
331
|
+
return text
|
|
332
|
+
.replace(/\{[^}]*\}/g, "")
|
|
333
|
+
.replace(/\\N/gi, "\n")
|
|
334
|
+
.trim();
|
|
335
|
+
}
|
|
336
|
+
|
|
337
|
+
/**
|
|
338
|
+
* A time in the form WebVTT requires.
|
|
339
|
+
*
|
|
340
|
+
* @param {number} seconds
|
|
341
|
+
* @returns {string}
|
|
342
|
+
*/
|
|
343
|
+
function vttTime(seconds) {
|
|
344
|
+
const safe = Math.max(0, seconds);
|
|
345
|
+
const hours = Math.floor(safe / 3600);
|
|
346
|
+
const minutes = Math.floor((safe % 3600) / 60);
|
|
347
|
+
const rest = safe % 60;
|
|
348
|
+
return `${String(hours).padStart(2, "0")}:${String(minutes).padStart(2, "0")}:` +
|
|
349
|
+
`${rest.toFixed(3).padStart(6, "0")}`;
|
|
350
|
+
}
|
|
@@ -0,0 +1,202 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @file Subtitle blocks out of Matroska clusters, from bytes already in hand.
|
|
3
|
+
*
|
|
4
|
+
* A subtitle track is sparse — a few kilobytes spread across a whole film — and
|
|
5
|
+
* ffmpeg cannot extract a time range of one without walking the container to
|
|
6
|
+
* the end. Measured 2026-08-19 on `Minions.and.Monsters.1080p.mkv`: asking for
|
|
7
|
+
* four seconds at minute twenty read the file through and pulled the download
|
|
8
|
+
* from 2.7 % to 81 %; asking with `-copyts -ss -to` took 154 s on a copy that
|
|
9
|
+
* was already 81 % local and still emitted the whole track. So the subtitles
|
|
10
|
+
* are not asked of ffmpeg.
|
|
11
|
+
*
|
|
12
|
+
* They do not have to be. A subtitle block sits in the same cluster as the
|
|
13
|
+
* picture around it, and those clusters are being downloaded anyway for the
|
|
14
|
+
* viewer to watch. Reading them as they arrive costs no network at all, and it
|
|
15
|
+
* puts the cues ahead of the playhead by construction — which is the whole
|
|
16
|
+
* requirement: subtitles arrive like the picture does, or they are not offered.
|
|
17
|
+
*
|
|
18
|
+
* This module is the byte-level half: given a cluster's bytes, it returns the
|
|
19
|
+
* blocks of one track with their times. It reads element headers and skips
|
|
20
|
+
* payloads by their declared length; nothing is decoded.
|
|
21
|
+
*
|
|
22
|
+
* Structure, from RFC 9559 §5.1.3 and §5.1.4:
|
|
23
|
+
*
|
|
24
|
+
* Cluster (0x1F43B675)
|
|
25
|
+
* Timestamp (0xE7) — the cluster's own time, in ticks
|
|
26
|
+
* SimpleBlock (0xA3) — a block with no duration of its own
|
|
27
|
+
* BlockGroup (0xA0)
|
|
28
|
+
* Block (0xA1) — the same header, and where subtitles live
|
|
29
|
+
* BlockDuration (0x9B) — how long the cue stays on screen
|
|
30
|
+
*
|
|
31
|
+
* A block's header is: the track number as a variable-length integer, a signed
|
|
32
|
+
* 16-bit timestamp relative to the cluster, and one byte of flags. Subtitles
|
|
33
|
+
* are normally in a BlockGroup, because a cue without a duration has no end.
|
|
34
|
+
*/
|
|
35
|
+
|
|
36
|
+
import { iterateElements, readUint, readVint } from "./ebml-reader.js";
|
|
37
|
+
|
|
38
|
+
const ID_TIMESTAMP = 0xe7;
|
|
39
|
+
const ID_SIMPLE_BLOCK = 0xa3;
|
|
40
|
+
const ID_BLOCK_GROUP = 0xa0;
|
|
41
|
+
const ID_BLOCK = 0xa1;
|
|
42
|
+
const ID_BLOCK_DURATION = 0x9b;
|
|
43
|
+
|
|
44
|
+
/** Bits 1-2 of the flags byte say how a block is laced, or that it is not. */
|
|
45
|
+
const LACING_MASK = 0x06;
|
|
46
|
+
const LACING_NONE = 0x00;
|
|
47
|
+
const LACING_XIPH = 0x02;
|
|
48
|
+
const LACING_FIXED = 0x04;
|
|
49
|
+
const LACING_EBML = 0x06;
|
|
50
|
+
|
|
51
|
+
/**
|
|
52
|
+
* @typedef {object} SubtitleBlock
|
|
53
|
+
* @property {number} startSeconds - When the cue appears.
|
|
54
|
+
* @property {number | null} durationSeconds - How long it stays, or null when
|
|
55
|
+
* the block carried no duration (a SimpleBlock; the caller decides).
|
|
56
|
+
* @property {Buffer} payload - The block's own bytes, still in the codec's form.
|
|
57
|
+
*/
|
|
58
|
+
|
|
59
|
+
/**
|
|
60
|
+
* Read one block's header.
|
|
61
|
+
*
|
|
62
|
+
* @param {Buffer} buffer
|
|
63
|
+
* @param {number} start - First byte of the block's payload.
|
|
64
|
+
* @param {number} end - One past its last byte.
|
|
65
|
+
* @returns {{ trackNumber: number, relativeTicks: number, flags: number, dataOffset: number } | null}
|
|
66
|
+
*/
|
|
67
|
+
function readBlockHeader(buffer, start, end) {
|
|
68
|
+
const track = readVint(buffer, start, false);
|
|
69
|
+
if (!track || track.value === null) {
|
|
70
|
+
return null;
|
|
71
|
+
}
|
|
72
|
+
const timestampAt = start + track.length;
|
|
73
|
+
// Signed, and it can be negative: a block may belong slightly before the
|
|
74
|
+
// cluster it is stored in.
|
|
75
|
+
if (timestampAt + 3 > end) {
|
|
76
|
+
return null;
|
|
77
|
+
}
|
|
78
|
+
return {
|
|
79
|
+
trackNumber: Number(track.value),
|
|
80
|
+
relativeTicks: buffer.readInt16BE(timestampAt),
|
|
81
|
+
flags: buffer[timestampAt + 2],
|
|
82
|
+
dataOffset: timestampAt + 3
|
|
83
|
+
};
|
|
84
|
+
}
|
|
85
|
+
|
|
86
|
+
/**
|
|
87
|
+
* Where a laced block's first frame begins.
|
|
88
|
+
*
|
|
89
|
+
* Subtitles are rarely laced, but a block that IS laced starts with a frame
|
|
90
|
+
* count and a table of sizes, and reading the payload without stepping over
|
|
91
|
+
* them yields the table as though it were text.
|
|
92
|
+
*
|
|
93
|
+
* @param {Buffer} buffer
|
|
94
|
+
* @param {number} dataOffset - First byte after the block header.
|
|
95
|
+
* @param {number} end
|
|
96
|
+
* @param {number} flags
|
|
97
|
+
* @returns {number | null} The offset of the first frame, or null when the
|
|
98
|
+
* lacing cannot be read.
|
|
99
|
+
*/
|
|
100
|
+
function firstFrameOffset(buffer, dataOffset, end, flags) {
|
|
101
|
+
const lacing = flags & LACING_MASK;
|
|
102
|
+
if (lacing === LACING_NONE) {
|
|
103
|
+
return dataOffset;
|
|
104
|
+
}
|
|
105
|
+
if (dataOffset >= end) {
|
|
106
|
+
return null;
|
|
107
|
+
}
|
|
108
|
+
const frames = buffer[dataOffset] + 1;
|
|
109
|
+
let at = dataOffset + 1;
|
|
110
|
+
if (lacing === LACING_FIXED) {
|
|
111
|
+
return at;
|
|
112
|
+
}
|
|
113
|
+
if (lacing === LACING_XIPH) {
|
|
114
|
+
// Each size but the last is a run of 0xFF bytes ending in a smaller one.
|
|
115
|
+
for (let frame = 0; frame < frames - 1; frame += 1) {
|
|
116
|
+
while (at < end && buffer[at] === 0xff) {
|
|
117
|
+
at += 1;
|
|
118
|
+
}
|
|
119
|
+
at += 1;
|
|
120
|
+
}
|
|
121
|
+
return at <= end ? at : null;
|
|
122
|
+
}
|
|
123
|
+
if (lacing === LACING_EBML) {
|
|
124
|
+
// The first size is a plain variable-length integer, the rest are signed
|
|
125
|
+
// differences from it; either way each is one such integer to step over.
|
|
126
|
+
for (let frame = 0; frame < frames - 1; frame += 1) {
|
|
127
|
+
const size = readVint(buffer, at, false);
|
|
128
|
+
if (!size) {
|
|
129
|
+
return null;
|
|
130
|
+
}
|
|
131
|
+
at += size.length;
|
|
132
|
+
}
|
|
133
|
+
return at <= end ? at : null;
|
|
134
|
+
}
|
|
135
|
+
return null;
|
|
136
|
+
}
|
|
137
|
+
|
|
138
|
+
/**
|
|
139
|
+
* Every block of one track inside one cluster.
|
|
140
|
+
*
|
|
141
|
+
* @param {Buffer} buffer - Bytes holding the cluster's payload.
|
|
142
|
+
* @param {{ dataOffset: number, size: number }} cluster - Where that payload is.
|
|
143
|
+
* @param {number} trackNumber - The track to keep.
|
|
144
|
+
* @param {number} secondsPerTick - From the segment's timestamp scale.
|
|
145
|
+
* @returns {SubtitleBlock[]}
|
|
146
|
+
*/
|
|
147
|
+
export function blocksOfTrack(buffer, cluster, trackNumber, secondsPerTick) {
|
|
148
|
+
const end = Math.min(buffer.length, cluster.dataOffset + cluster.size);
|
|
149
|
+
/** @type {SubtitleBlock[]} */
|
|
150
|
+
const blocks = [];
|
|
151
|
+
let clusterTicks = null;
|
|
152
|
+
|
|
153
|
+
const take = (blockStart, blockEnd, durationTicks) => {
|
|
154
|
+
const header = readBlockHeader(buffer, blockStart, blockEnd);
|
|
155
|
+
if (!header || header.trackNumber !== trackNumber || clusterTicks === null) {
|
|
156
|
+
return;
|
|
157
|
+
}
|
|
158
|
+
const payloadAt = firstFrameOffset(buffer, header.dataOffset, blockEnd, header.flags);
|
|
159
|
+
if (payloadAt === null || payloadAt >= blockEnd) {
|
|
160
|
+
return;
|
|
161
|
+
}
|
|
162
|
+
blocks.push({
|
|
163
|
+
startSeconds: (clusterTicks + header.relativeTicks) * secondsPerTick,
|
|
164
|
+
durationSeconds: durationTicks === null ? null : durationTicks * secondsPerTick,
|
|
165
|
+
payload: buffer.subarray(payloadAt, blockEnd)
|
|
166
|
+
});
|
|
167
|
+
};
|
|
168
|
+
|
|
169
|
+
for (const element of iterateElements(buffer, cluster.dataOffset, end)) {
|
|
170
|
+
const elementEnd = Math.min(end, element.dataOffset + element.size);
|
|
171
|
+
if (element.id === ID_TIMESTAMP) {
|
|
172
|
+
clusterTicks = readUint(buffer, element.dataOffset, element.size);
|
|
173
|
+
continue;
|
|
174
|
+
}
|
|
175
|
+
if (element.id === ID_SIMPLE_BLOCK) {
|
|
176
|
+
take(element.dataOffset, elementEnd, null);
|
|
177
|
+
continue;
|
|
178
|
+
}
|
|
179
|
+
if (element.id !== ID_BLOCK_GROUP) {
|
|
180
|
+
continue;
|
|
181
|
+
}
|
|
182
|
+
// A group holds the block and, for a subtitle, the duration that says when
|
|
183
|
+
// the cue leaves the screen. Both are read before either is used, because
|
|
184
|
+
// the duration may be written after the block.
|
|
185
|
+
let blockStart = null;
|
|
186
|
+
let blockEnd = null;
|
|
187
|
+
let durationTicks = null;
|
|
188
|
+
for (const field of iterateElements(buffer, element.dataOffset, elementEnd)) {
|
|
189
|
+
const fieldEnd = Math.min(elementEnd, field.dataOffset + field.size);
|
|
190
|
+
if (field.id === ID_BLOCK) {
|
|
191
|
+
blockStart = field.dataOffset;
|
|
192
|
+
blockEnd = fieldEnd;
|
|
193
|
+
} else if (field.id === ID_BLOCK_DURATION) {
|
|
194
|
+
durationTicks = readUint(buffer, field.dataOffset, field.size);
|
|
195
|
+
}
|
|
196
|
+
}
|
|
197
|
+
if (blockStart !== null) {
|
|
198
|
+
take(blockStart, blockEnd, durationTicks);
|
|
199
|
+
}
|
|
200
|
+
}
|
|
201
|
+
return blocks;
|
|
202
|
+
}
|
|
@@ -0,0 +1,262 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @file The subtitle tracks of a Matroska file, and where their cues live.
|
|
3
|
+
*
|
|
4
|
+
* Two halves, both reading only what is asked for by byte range:
|
|
5
|
+
*
|
|
6
|
+
* - {@link readSubtitlePlan} — once per file: which tracks are text
|
|
7
|
+
* subtitles, in what codec, and the cluster positions their cue points
|
|
8
|
+
* name. Two short reads, the head and the Cues element, exactly as the
|
|
9
|
+
* keyframe reader already does.
|
|
10
|
+
* - {@link harvestCluster} — per cluster, over bytes already downloaded:
|
|
11
|
+
* the cues inside it, ready to be shown.
|
|
12
|
+
*
|
|
13
|
+
* Kept apart from `matroska.js` because that file answers one question (where
|
|
14
|
+
* are the keyframes) and is read on the path that starts playback; this one is
|
|
15
|
+
* only ever consulted for a viewer who asked for subtitles.
|
|
16
|
+
*/
|
|
17
|
+
|
|
18
|
+
import { findElement, iterateElements, readUint } from "./ebml-reader.js";
|
|
19
|
+
import { blocksOfTrack } from "./matroska-blocks.js";
|
|
20
|
+
|
|
21
|
+
const ID_SEGMENT = 0x18538067;
|
|
22
|
+
const ID_SEEK_HEAD = 0x114d9b74;
|
|
23
|
+
const ID_SEEK = 0x4dbb;
|
|
24
|
+
const ID_SEEK_ID = 0x53ab;
|
|
25
|
+
const ID_SEEK_POSITION = 0x53ac;
|
|
26
|
+
const ID_INFO = 0x1549a966;
|
|
27
|
+
const ID_TIMESTAMP_SCALE = 0x2ad7b1;
|
|
28
|
+
const ID_TRACKS = 0x1654ae6b;
|
|
29
|
+
const ID_TRACK_ENTRY = 0xae;
|
|
30
|
+
const ID_TRACK_NUMBER = 0xd7;
|
|
31
|
+
const ID_TRACK_TYPE = 0x83;
|
|
32
|
+
const ID_CODEC_ID = 0x86;
|
|
33
|
+
const ID_CODEC_PRIVATE = 0x63a2;
|
|
34
|
+
const ID_LANGUAGE = 0x22b59c;
|
|
35
|
+
const ID_NAME = 0x536e;
|
|
36
|
+
const ID_FLAG_DEFAULT = 0x88;
|
|
37
|
+
const ID_CUES = 0x1c53bb6b;
|
|
38
|
+
const ID_CUE_POINT = 0xbb;
|
|
39
|
+
const ID_CUE_TRACK_POSITIONS = 0xb7;
|
|
40
|
+
const ID_CUE_TRACK = 0xf7;
|
|
41
|
+
const ID_CUE_CLUSTER_POSITION = 0xf1;
|
|
42
|
+
|
|
43
|
+
/** TrackType 17 is subtitles; 1 is video and 2 audio. */
|
|
44
|
+
const TRACK_TYPE_SUBTITLE = 17;
|
|
45
|
+
/** How much of the file start to read: the same window the keyframe reader uses. */
|
|
46
|
+
const HEAD_BYTES = 64 * 1024;
|
|
47
|
+
/** Cap on the Cues read; a long film indexes to tens of KB. */
|
|
48
|
+
const MAX_CUES_BYTES = 8 * 1024 * 1024;
|
|
49
|
+
const DEFAULT_TIMESTAMP_SCALE = 1_000_000;
|
|
50
|
+
|
|
51
|
+
/**
|
|
52
|
+
* The codecs whose blocks are text this proxy can turn into WebVTT.
|
|
53
|
+
*
|
|
54
|
+
* `S_TEXT/UTF8` is a plain line of text and needs nothing. `S_TEXT/ASS` and
|
|
55
|
+
* `S_TEXT/SSA` carry a dialogue row whose fields have to be stripped, and their
|
|
56
|
+
* header lives in CodecPrivate — supported, with the stripping done where the
|
|
57
|
+
* cue is turned into WebVTT. `S_HDMV/PGS` and `S_VOBSUB` are pictures, not
|
|
58
|
+
* text, and are deliberately absent: offering them would promise something this
|
|
59
|
+
* path cannot deliver.
|
|
60
|
+
*/
|
|
61
|
+
const TEXT_CODECS = new Set(["S_TEXT/UTF8", "S_TEXT/ASS", "S_TEXT/SSA"]);
|
|
62
|
+
|
|
63
|
+
/**
|
|
64
|
+
* @typedef {object} SubtitleTrackPlan
|
|
65
|
+
* @property {number} trackNumber - As the blocks name it.
|
|
66
|
+
* @property {string} codecId
|
|
67
|
+
* @property {string} language - The three-letter code the file declares.
|
|
68
|
+
* @property {string} name - What the file calls the track, if anything.
|
|
69
|
+
* @property {boolean} isDefault
|
|
70
|
+
* @property {string} codecPrivate - The ASS/SSA header, base64, or "".
|
|
71
|
+
* @property {number[]} clusterPositions - File offsets of clusters whose cue
|
|
72
|
+
* points name this track, ascending. Empty when the file indexes only its
|
|
73
|
+
* picture, and then the caller has to walk clusters as they arrive instead.
|
|
74
|
+
*/
|
|
75
|
+
|
|
76
|
+
/**
|
|
77
|
+
* Read a string element, trimming the padding some muxers leave.
|
|
78
|
+
*
|
|
79
|
+
* @param {Buffer} buffer
|
|
80
|
+
* @param {{ dataOffset: number, size: number }} element
|
|
81
|
+
* @returns {string}
|
|
82
|
+
*/
|
|
83
|
+
function readString(buffer, element) {
|
|
84
|
+
return buffer
|
|
85
|
+
.toString("utf8", element.dataOffset, element.dataOffset + element.size)
|
|
86
|
+
.replace(/\0+$/, "");
|
|
87
|
+
}
|
|
88
|
+
|
|
89
|
+
/**
|
|
90
|
+
* Everything about a file's text subtitle tracks that can be learned without
|
|
91
|
+
* reading the film.
|
|
92
|
+
*
|
|
93
|
+
* @param {(start: number, end: number) => Promise<Buffer | null>} readRange
|
|
94
|
+
* @param {number} fileSize
|
|
95
|
+
* @returns {Promise<{ tracks: SubtitleTrackPlan[], secondsPerTick: number, segmentDataOffset: number } | null>}
|
|
96
|
+
*/
|
|
97
|
+
export async function readSubtitlePlan(readRange, fileSize) {
|
|
98
|
+
const head = await readRange(0, Math.min(HEAD_BYTES, Math.max(0, fileSize - 1)));
|
|
99
|
+
if (!head || head.length < 4 || head.readUInt32BE(0) !== 0x1a45dfa3) {
|
|
100
|
+
return null;
|
|
101
|
+
}
|
|
102
|
+
const segment = findElement(head, ID_SEGMENT, []);
|
|
103
|
+
if (!segment) {
|
|
104
|
+
return null;
|
|
105
|
+
}
|
|
106
|
+
const base = segment.dataOffset;
|
|
107
|
+
|
|
108
|
+
const info = findElement(head, ID_INFO, [], base);
|
|
109
|
+
let scale = DEFAULT_TIMESTAMP_SCALE;
|
|
110
|
+
if (info) {
|
|
111
|
+
const declared = findElement(head, ID_TIMESTAMP_SCALE, [], info.dataOffset, info.dataOffset + info.size);
|
|
112
|
+
if (declared) {
|
|
113
|
+
const value = readUint(head, declared.dataOffset, declared.size);
|
|
114
|
+
if (value > 0) {
|
|
115
|
+
scale = value;
|
|
116
|
+
}
|
|
117
|
+
}
|
|
118
|
+
}
|
|
119
|
+
|
|
120
|
+
const tracksElement = findElement(head, ID_TRACKS, [], base);
|
|
121
|
+
if (!tracksElement) {
|
|
122
|
+
return null;
|
|
123
|
+
}
|
|
124
|
+
const tracksEnd = Math.min(head.length, tracksElement.dataOffset + tracksElement.size);
|
|
125
|
+
/** @type {SubtitleTrackPlan[]} */
|
|
126
|
+
const tracks = [];
|
|
127
|
+
for (const entry of iterateElements(head, tracksElement.dataOffset, tracksEnd)) {
|
|
128
|
+
if (entry.id !== ID_TRACK_ENTRY) {
|
|
129
|
+
continue;
|
|
130
|
+
}
|
|
131
|
+
const entryEnd = Math.min(tracksEnd, entry.dataOffset + entry.size);
|
|
132
|
+
let trackNumber = null;
|
|
133
|
+
let type = null;
|
|
134
|
+
let codecId = "";
|
|
135
|
+
let language = "";
|
|
136
|
+
let name = "";
|
|
137
|
+
let codecPrivate = "";
|
|
138
|
+
let isDefault = true;
|
|
139
|
+
for (const field of iterateElements(head, entry.dataOffset, entryEnd)) {
|
|
140
|
+
if (field.id === ID_TRACK_NUMBER) {
|
|
141
|
+
trackNumber = readUint(head, field.dataOffset, field.size);
|
|
142
|
+
} else if (field.id === ID_TRACK_TYPE) {
|
|
143
|
+
type = readUint(head, field.dataOffset, field.size);
|
|
144
|
+
} else if (field.id === ID_CODEC_ID) {
|
|
145
|
+
codecId = readString(head, field);
|
|
146
|
+
} else if (field.id === ID_LANGUAGE) {
|
|
147
|
+
language = readString(head, field);
|
|
148
|
+
} else if (field.id === ID_NAME) {
|
|
149
|
+
name = readString(head, field);
|
|
150
|
+
} else if (field.id === ID_FLAG_DEFAULT) {
|
|
151
|
+
isDefault = readUint(head, field.dataOffset, field.size) === 1;
|
|
152
|
+
} else if (field.id === ID_CODEC_PRIVATE) {
|
|
153
|
+
codecPrivate = head.toString("base64", field.dataOffset, field.dataOffset + field.size);
|
|
154
|
+
}
|
|
155
|
+
}
|
|
156
|
+
if (type !== TRACK_TYPE_SUBTITLE || trackNumber === null || !TEXT_CODECS.has(codecId)) {
|
|
157
|
+
continue;
|
|
158
|
+
}
|
|
159
|
+
tracks.push({
|
|
160
|
+
trackNumber,
|
|
161
|
+
codecId,
|
|
162
|
+
language,
|
|
163
|
+
name,
|
|
164
|
+
isDefault,
|
|
165
|
+
codecPrivate,
|
|
166
|
+
clusterPositions: []
|
|
167
|
+
});
|
|
168
|
+
}
|
|
169
|
+
if (tracks.length === 0) {
|
|
170
|
+
return { tracks, secondsPerTick: scale / 1e9, segmentDataOffset: base };
|
|
171
|
+
}
|
|
172
|
+
|
|
173
|
+
// Where the clusters holding those tracks are. A file that indexes only its
|
|
174
|
+
// picture leaves these empty, which is not a failure: the caller then reads
|
|
175
|
+
// the clusters the viewer's own playback brings in.
|
|
176
|
+
const seekHead = findElement(head, ID_SEEK_HEAD, [], base);
|
|
177
|
+
let cuesRelative;
|
|
178
|
+
if (seekHead) {
|
|
179
|
+
const seekEnd = Math.min(head.length, seekHead.dataOffset + seekHead.size);
|
|
180
|
+
for (const seek of iterateElements(head, seekHead.dataOffset, seekEnd)) {
|
|
181
|
+
if (seek.id !== ID_SEEK) {
|
|
182
|
+
continue;
|
|
183
|
+
}
|
|
184
|
+
let target = null;
|
|
185
|
+
let position = null;
|
|
186
|
+
for (const field of iterateElements(head, seek.dataOffset, Math.min(seekEnd, seek.dataOffset + seek.size))) {
|
|
187
|
+
if (field.id === ID_SEEK_ID) {
|
|
188
|
+
target = readUint(head, field.dataOffset, field.size);
|
|
189
|
+
} else if (field.id === ID_SEEK_POSITION) {
|
|
190
|
+
position = readUint(head, field.dataOffset, field.size);
|
|
191
|
+
}
|
|
192
|
+
}
|
|
193
|
+
if (target === ID_CUES && position !== null) {
|
|
194
|
+
cuesRelative = position;
|
|
195
|
+
}
|
|
196
|
+
}
|
|
197
|
+
}
|
|
198
|
+
if (cuesRelative !== undefined) {
|
|
199
|
+
const cuesAt = base + cuesRelative;
|
|
200
|
+
if (cuesAt > 0 && cuesAt < fileSize) {
|
|
201
|
+
const chunk = await readRange(cuesAt, Math.min(fileSize - 1, cuesAt + MAX_CUES_BYTES));
|
|
202
|
+
const element = chunk && [...iterateElements(chunk, 0, chunk.length)][0];
|
|
203
|
+
if (element && element.id === ID_CUES) {
|
|
204
|
+
const body = chunk.subarray(element.dataOffset, Math.min(chunk.length, element.dataOffset + element.size));
|
|
205
|
+
const byTrack = new Map(tracks.map((track) => [track.trackNumber, new Set()]));
|
|
206
|
+
for (const point of iterateElements(body, 0, body.length)) {
|
|
207
|
+
if (point.id !== ID_CUE_POINT) {
|
|
208
|
+
continue;
|
|
209
|
+
}
|
|
210
|
+
const pointEnd = Math.min(body.length, point.dataOffset + point.size);
|
|
211
|
+
for (const field of iterateElements(body, point.dataOffset, pointEnd)) {
|
|
212
|
+
if (field.id !== ID_CUE_TRACK_POSITIONS) {
|
|
213
|
+
continue;
|
|
214
|
+
}
|
|
215
|
+
let cueTrack = null;
|
|
216
|
+
let position = null;
|
|
217
|
+
for (const inner of iterateElements(body, field.dataOffset, Math.min(pointEnd, field.dataOffset + field.size))) {
|
|
218
|
+
if (inner.id === ID_CUE_TRACK) {
|
|
219
|
+
cueTrack = readUint(body, inner.dataOffset, inner.size);
|
|
220
|
+
} else if (inner.id === ID_CUE_CLUSTER_POSITION) {
|
|
221
|
+
position = readUint(body, inner.dataOffset, inner.size);
|
|
222
|
+
}
|
|
223
|
+
}
|
|
224
|
+
if (position !== null && byTrack.has(cueTrack)) {
|
|
225
|
+
byTrack.get(cueTrack).add(base + position);
|
|
226
|
+
}
|
|
227
|
+
}
|
|
228
|
+
}
|
|
229
|
+
for (const track of tracks) {
|
|
230
|
+
track.clusterPositions = [...byTrack.get(track.trackNumber)].sort((left, right) => left - right);
|
|
231
|
+
}
|
|
232
|
+
}
|
|
233
|
+
}
|
|
234
|
+
}
|
|
235
|
+
return { tracks, secondsPerTick: scale / 1e9, segmentDataOffset: base };
|
|
236
|
+
}
|
|
237
|
+
|
|
238
|
+
/**
|
|
239
|
+
* The cues of one track inside one cluster.
|
|
240
|
+
*
|
|
241
|
+
* @param {Buffer} bytes - The cluster, from its own element header onward.
|
|
242
|
+
* @param {number} trackNumber
|
|
243
|
+
* @param {number} secondsPerTick
|
|
244
|
+
* @returns {{ startSeconds: number, endSeconds: number | null, text: string }[]}
|
|
245
|
+
*/
|
|
246
|
+
export function harvestCluster(bytes, trackNumber, secondsPerTick) {
|
|
247
|
+
const header = [...iterateElements(bytes, 0, bytes.length)][0];
|
|
248
|
+
if (!header) {
|
|
249
|
+
return [];
|
|
250
|
+
}
|
|
251
|
+
const blocks = blocksOfTrack(
|
|
252
|
+
bytes,
|
|
253
|
+
{ dataOffset: header.dataOffset, size: header.size },
|
|
254
|
+
trackNumber,
|
|
255
|
+
secondsPerTick
|
|
256
|
+
);
|
|
257
|
+
return blocks.map((block) => ({
|
|
258
|
+
startSeconds: block.startSeconds,
|
|
259
|
+
endSeconds: block.durationSeconds === null ? null : block.startSeconds + block.durationSeconds,
|
|
260
|
+
text: block.payload.toString("utf8")
|
|
261
|
+
}));
|
|
262
|
+
}
|