@torrent-tv/proxy 2.55.14 → 2.56.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +8 -0
- package/package.json +1 -1
- package/routes/api/subtitles/get.js +13 -4
- package/services/container-index/matroska-subtitles.js +6 -0
- package/services/container-index/mp4-subtitles.js +25 -1
- package/services/data-channel-handler.js +4 -4
- package/services/torrent-worker/client.js +2 -1
- package/services/torrent-worker/subtitle-cues.js +182 -21
- package/services/torrent-worker/worker.js +28 -2
- package/test/subtitle-track-numbering.test.js +262 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,11 @@
|
|
|
1
|
+
## 2.56.0
|
|
2
|
+
|
|
3
|
+
- **Fix**: A subtitle track is asked for by ffmpeg's own number, `0:s:N`, which counts EVERY subtitle stream the file carries — and the container plan counted only the ones it can turn into text, dropping PGS and VobSub before numbering. On a release whose picture-based track comes first the two numberings ran a place apart, with two consequences, both silent: a pushed cue named a track the browser does not know and was dropped, and the browser's own request found no track at all and fell through to the ffmpeg extraction, which reads the whole film for a few kilobytes of text (752 s measured on one file, 2026-08-19). Every text track now carries `declaredIndex` — its position among all the file's subtitle tracks, counted by what a track IS and not by what this code can read — and both the push and `/api/subtitles?trackIndex=` use it. In an MP4 that count includes the `subp` and `clcp` handlers, which ffmpeg also calls subtitle streams.
|
|
4
|
+
- **Fix**: One walk of a file at a time. The walk marks a cluster as read only after fetching and parsing it — two suspension points — while it is started both on every verified piece and on a 3 s timer, so on a fast download several passes read and parsed the same cluster and could push one line twice under different found-order numbers. Each of those reads is a WebTorrent file stream, which selects and deselects its pieces, so the repetition reached the piece picker as well. Walks are now serialized per file, the plan is read once even when two callers arrive together, a warmup triggered while the previous one is still walking is dropped rather than queued, and forgetting a file waits for its walk instead of leaving one running beside a fresh copy of the state.
|
|
5
|
+
- **Fix**: A read of already-downloaded bytes that never ends is given up after 30 s, with a line saying so. It had no bound, and with walks now serialized one such read would have held that file's queue — the browser's own request for its subtitles included — for the rest of the session.
|
|
6
|
+
- **New**: A subtitle push says what it is ABOUT: the film-time span its new cues cover, how many of the file's indexed clusters have been walked, and the found-order cursor. Read against the position being played — which the browser now logs beside it — that separates cues that arrived late from cues that arrived early for a stretch nobody is watching. Report 2026-08-26: embedded subtitles appear "after some time", and no line on either side could say whether the cues held covered the playhead (`research/subtitle-delay-2026-08-26.md`).
|
|
7
|
+
- **New**: The push carries a `cursor`, so a browser that loses the subscription — which a reconnect does, since the subscription belongs to the channel — can ask for exactly what it missed instead of the whole track.
|
|
8
|
+
|
|
1
9
|
## 2.55.14
|
|
2
10
|
|
|
3
11
|
- **Fix**: `utp-native` moves to 2.5.3-ttv.5, which carries ten defects found by reading the whole binding after the seventh crash of this family named its frame. Three callbacks read the connection pointer without checking it exists — and the socket carries none unless it was accepted or dialled, while its destructor announces itself regardless; the ttv.4 patch created one such path itself. The read callback copied at an accumulated offset without ever consulting the buffer length it maintains, so a peer sending more between two hand-offs wrote past the end — heap corruption does not fault where it happens, which is what six deaths inside libuv bookkeeping look like. A connection could be destroyed twice, deleting already-deleted napi references, and the socket pointer was never cleared. The callback macro checked none of the three napi results it then used. Plus IPv6, which the module never had at all, and the resolver that fed it addresses it could not use. Tests on the target: 77 checks, no failures — two of them were failing or hanging before.
|
package/package.json
CHANGED
|
@@ -268,9 +268,10 @@ function readFileFully(file, maxBytes) {
|
|
|
268
268
|
* The cues of a track from clusters already downloaded, as WebVTT.
|
|
269
269
|
*
|
|
270
270
|
* `trackIndex` is the browser's number for the subtitle stream — its position
|
|
271
|
-
* among the subtitle streams, as
|
|
272
|
-
* carry the file's own track number. The plan lists the
|
|
273
|
-
*
|
|
271
|
+
* among ALL the subtitle streams, as ffmpeg lists them — while Matroska blocks
|
|
272
|
+
* carry the file's own track number. The plan lists only the tracks that can
|
|
273
|
+
* become text, so it is matched on `declaredIndex`, which each track carries
|
|
274
|
+
* for exactly this: its place in the file's full list of subtitle tracks.
|
|
274
275
|
*
|
|
275
276
|
* @param {object} torrentPool
|
|
276
277
|
* @param {object} torrent
|
|
@@ -289,7 +290,15 @@ async function cuesFromDownloadedClusters(torrentPool, torrent, fileIndex, track
|
|
|
289
290
|
} catch {
|
|
290
291
|
return null;
|
|
291
292
|
}
|
|
292
|
-
|
|
293
|
+
// BY the browser's number, not by position in this list. The list holds only
|
|
294
|
+
// the tracks that can become WebVTT, while the browser counts every subtitle
|
|
295
|
+
// stream ffmpeg lists — so on a file carrying a PGS or VobSub track the two
|
|
296
|
+
// ran one apart, and the request either found the wrong track or found none
|
|
297
|
+
// and fell through to the ffmpeg extraction below, which reads the whole film
|
|
298
|
+
// (752 s measured, 2026-08-19) for cues already in hand.
|
|
299
|
+
const track = Array.isArray(tracks)
|
|
300
|
+
? tracks.find((candidate) => candidate.declaredIndex === trackIndex) ?? null
|
|
301
|
+
: null;
|
|
293
302
|
if (!track) {
|
|
294
303
|
return null;
|
|
295
304
|
}
|
|
@@ -63,6 +63,11 @@ const TEXT_CODECS = new Set(["S_TEXT/UTF8", "S_TEXT/ASS", "S_TEXT/SSA"]);
|
|
|
63
63
|
/**
|
|
64
64
|
* @typedef {object} SubtitleTrackPlan
|
|
65
65
|
* @property {number} trackNumber - As the blocks name it.
|
|
66
|
+
* @property {number} declaredIndex - Its position among ALL of the file's
|
|
67
|
+
* subtitle tracks, picture-based ones included — which is the number ffmpeg
|
|
68
|
+
* gives the same stream in `0:s:N`, and therefore the only number the browser
|
|
69
|
+
* ever names. Text tracks alone are not a numbering: a file whose PGS track
|
|
70
|
+
* comes first would have every text track one lower here than in the browser.
|
|
66
71
|
* @property {string} codecId
|
|
67
72
|
* @property {string} language - The three-letter code the file declares.
|
|
68
73
|
* @property {string} name - What the file calls the track, if anything.
|
|
@@ -178,6 +183,7 @@ export async function readSubtitlePlan(readRange, fileSize) {
|
|
|
178
183
|
}
|
|
179
184
|
tracks.push({
|
|
180
185
|
trackNumber,
|
|
186
|
+
declaredIndex: declared.length - 1,
|
|
181
187
|
codecId,
|
|
182
188
|
language,
|
|
183
189
|
name,
|
|
@@ -29,6 +29,15 @@ const MAX_MOOV_BYTES = 32 * 1024 * 1024;
|
|
|
29
29
|
|
|
30
30
|
/** Handlers that mean "this track is text on screen". */
|
|
31
31
|
const TEXT_HANDLERS = new Set(["text", "sbtl", "subt"]);
|
|
32
|
+
/**
|
|
33
|
+
* Every handler ffmpeg's mov demuxer turns into a SUBTITLE stream, whether or
|
|
34
|
+
* not this file can read it — `subp` is a DVD subpicture and `clcp` closed
|
|
35
|
+
* captions, both pictures or caption data rather than text. They are counted
|
|
36
|
+
* because `declaredIndex` has to equal ffmpeg's `0:s:N`, and a track left out
|
|
37
|
+
* of the count shifts every text track after it, which is the very defect
|
|
38
|
+
* `declaredIndex` exists to remove.
|
|
39
|
+
*/
|
|
40
|
+
const SUBTITLE_HANDLERS = new Set([...TEXT_HANDLERS, "subp", "clcp"]);
|
|
32
41
|
/** Sample formats this can turn into cues. `stpp` (TTML) is XML and is not one. */
|
|
33
42
|
const TEXT_FORMATS = new Set(["tx3g", "text", "wvtt"]);
|
|
34
43
|
|
|
@@ -218,6 +227,11 @@ function sampleOffsets(moov, stsc, chunkOffsets, sizes) {
|
|
|
218
227
|
/**
|
|
219
228
|
* @typedef {object} Mp4SubtitleTrack
|
|
220
229
|
* @property {number} trackId
|
|
230
|
+
* @property {number} declaredIndex - Its position among ALL of the file's
|
|
231
|
+
* subtitle tracks, including the ones whose sample format this cannot turn
|
|
232
|
+
* into cues (`stpp` TTML). That is the number ffmpeg gives the same stream in
|
|
233
|
+
* `0:s:N`, which is the number the browser names; counting only the readable
|
|
234
|
+
* ones would shift every track after a TTML one.
|
|
221
235
|
* @property {string} format - `tx3g`, `text` or `wvtt`.
|
|
222
236
|
* @property {string} language - Three letters, as the file declares them.
|
|
223
237
|
* @property {Mp4SubtitleSample[]} samples - In time order.
|
|
@@ -246,6 +260,9 @@ export async function readMp4SubtitlePlan(readRange, fileSize) {
|
|
|
246
260
|
|
|
247
261
|
/** @type {Mp4SubtitleTrack[]} */
|
|
248
262
|
const tracks = [];
|
|
263
|
+
// Counts every subtitle track the file has, whether or not this can read it,
|
|
264
|
+
// so the number handed out matches ffmpeg's `0:s:N`. See `declaredIndex`.
|
|
265
|
+
let declaredIndex = -1;
|
|
249
266
|
for (const trak of childrenOf(moov, moovBox.dataOffset, moov.length, "trak")) {
|
|
250
267
|
const mdia = childOf(moov, trak.dataOffset, trak.end, "mdia");
|
|
251
268
|
if (!mdia) {
|
|
@@ -253,6 +270,13 @@ export async function readMp4SubtitlePlan(readRange, fileSize) {
|
|
|
253
270
|
}
|
|
254
271
|
const hdlr = childOf(moov, mdia.dataOffset, mdia.end, "hdlr");
|
|
255
272
|
const handler = hdlr ? moov.toString("latin1", hdlr.dataOffset + 8, hdlr.dataOffset + 12) : "";
|
|
273
|
+
if (!SUBTITLE_HANDLERS.has(handler)) {
|
|
274
|
+
continue;
|
|
275
|
+
}
|
|
276
|
+
// Counted before the readability checks below, and before the handler is
|
|
277
|
+
// narrowed to the text ones: this number is the track's place in the file,
|
|
278
|
+
// not its place among the tracks this code can turn into cues.
|
|
279
|
+
declaredIndex += 1;
|
|
256
280
|
if (!TEXT_HANDLERS.has(handler)) {
|
|
257
281
|
continue;
|
|
258
282
|
}
|
|
@@ -337,7 +361,7 @@ export async function readMp4SubtitlePlan(readRange, fileSize) {
|
|
|
337
361
|
});
|
|
338
362
|
}
|
|
339
363
|
}
|
|
340
|
-
tracks.push({ trackId, format, language, samples });
|
|
364
|
+
tracks.push({ trackId, declaredIndex, format, language, samples });
|
|
341
365
|
}
|
|
342
366
|
return { tracks };
|
|
343
367
|
}
|
|
@@ -21,7 +21,7 @@
|
|
|
21
21
|
* { type: "response-start", requestId, status, headers } (JSON string)
|
|
22
22
|
* { type: "response-error", requestId, error: string } (JSON string)
|
|
23
23
|
* { type: "pong", id } (JSON string)
|
|
24
|
-
* { type: "subtitle-cues", fileIndex, trackIndex, cues, language } (JSON string)
|
|
24
|
+
* { type: "subtitle-cues", fileIndex, trackIndex, cues, language, cursor } (JSON string)
|
|
25
25
|
* ```
|
|
26
26
|
* The last one is unsolicited — sent the moment new cues are read from a
|
|
27
27
|
* file's already-downloaded pieces, to whichever channel last asked for that
|
|
@@ -385,10 +385,10 @@ export function createDataChannelHandler({ proxyPort, onLog, getTransportSnapsho
|
|
|
385
385
|
* are tiny (kilobytes at most for a whole track), so this is one message,
|
|
386
386
|
* not a stream.
|
|
387
387
|
*
|
|
388
|
-
* @param {{ sourceKey: string, fileIndex: number, trackIndex: number, cues: object[], language: string }} event
|
|
388
|
+
* @param {{ sourceKey: string, fileIndex: number, trackIndex: number, cues: object[], language: string, cursor: number }} event
|
|
389
389
|
* @returns {void}
|
|
390
390
|
*/
|
|
391
|
-
function publishSubtitleCues({ sourceKey, fileIndex, trackIndex, cues, language }) {
|
|
391
|
+
function publishSubtitleCues({ sourceKey, fileIndex, trackIndex, cues, language, cursor }) {
|
|
392
392
|
const set = subtitleSubscribers.get(`${sourceKey}:${fileIndex}`);
|
|
393
393
|
if (!set || set.size === 0) {
|
|
394
394
|
log(
|
|
@@ -397,7 +397,7 @@ export function createDataChannelHandler({ proxyPort, onLog, getTransportSnapsho
|
|
|
397
397
|
);
|
|
398
398
|
return;
|
|
399
399
|
}
|
|
400
|
-
const message = { type: "subtitle-cues", fileIndex, trackIndex, cues, language };
|
|
400
|
+
const message = { type: "subtitle-cues", fileIndex, trackIndex, cues, language, cursor };
|
|
401
401
|
const total = set.size;
|
|
402
402
|
let sent = 0;
|
|
403
403
|
for (const channel of set) {
|
|
@@ -194,7 +194,8 @@ export class TorrentWorkerClient {
|
|
|
194
194
|
fileIndex: message.fileIndex,
|
|
195
195
|
trackIndex: message.trackIndex,
|
|
196
196
|
cues: message.cues,
|
|
197
|
-
language: message.language
|
|
197
|
+
language: message.language,
|
|
198
|
+
cursor: message.cursor
|
|
198
199
|
});
|
|
199
200
|
break;
|
|
200
201
|
default:
|
|
@@ -30,6 +30,8 @@ const CLUSTER_HEADER_PROBE = 64;
|
|
|
30
30
|
* reading it would be a large read for nothing.
|
|
31
31
|
*/
|
|
32
32
|
const MAX_CLUSTER_BYTES = 32 * 1024 * 1024;
|
|
33
|
+
/** How long a read of already-held bytes may take before it is given up. */
|
|
34
|
+
const READ_ABANDON_MS = 30_000;
|
|
33
35
|
|
|
34
36
|
/** @type {Map<string, { plan: object | null, harvested: Map<number, Set<number>>, cues: Map<number, object[]> }>} */
|
|
35
37
|
const byFile = new Map();
|
|
@@ -78,9 +80,38 @@ function readHeld(file, start, end) {
|
|
|
78
80
|
resolve(null);
|
|
79
81
|
return;
|
|
80
82
|
}
|
|
83
|
+
let settled = false;
|
|
84
|
+
/** @type {ReturnType<typeof setTimeout> | null} */
|
|
85
|
+
let abandon = null;
|
|
86
|
+
const settle = (value) => {
|
|
87
|
+
if (settled) {
|
|
88
|
+
return;
|
|
89
|
+
}
|
|
90
|
+
settled = true;
|
|
91
|
+
if (abandon !== null) {
|
|
92
|
+
clearTimeout(abandon);
|
|
93
|
+
}
|
|
94
|
+
if (value === null) {
|
|
95
|
+
stream.destroy?.();
|
|
96
|
+
}
|
|
97
|
+
resolve(value);
|
|
98
|
+
};
|
|
99
|
+
// A read of bytes the torrent already holds either answers or it does not.
|
|
100
|
+
// This is not a measurement of anything and no figure is derived from it:
|
|
101
|
+
// it is the point past which such a read is presumed lost, so that one
|
|
102
|
+
// stream which never ends cannot hold this file's walk — and with it the
|
|
103
|
+
// browser's own request for its subtitles — for the rest of the session.
|
|
104
|
+
abandon = setTimeout(() => {
|
|
105
|
+
logger.info(
|
|
106
|
+
`subtitles: a read of ${start}-${end} in "${String(file.name).slice(0, 40)}" ` +
|
|
107
|
+
`did not finish in ${READ_ABANDON_MS / 1000}s and was given up`
|
|
108
|
+
);
|
|
109
|
+
settle(null);
|
|
110
|
+
}, READ_ABANDON_MS);
|
|
111
|
+
abandon.unref?.();
|
|
81
112
|
stream.on("data", (chunk) => chunks.push(chunk));
|
|
82
|
-
stream.on("end", () =>
|
|
83
|
-
stream.on("error", () =>
|
|
113
|
+
stream.on("end", () => settle(Buffer.concat(chunks)));
|
|
114
|
+
stream.on("error", () => settle(null));
|
|
84
115
|
});
|
|
85
116
|
}
|
|
86
117
|
|
|
@@ -97,10 +128,41 @@ function readHeld(file, start, end) {
|
|
|
97
128
|
* @returns {Promise<object | null>}
|
|
98
129
|
*/
|
|
99
130
|
async function planFor(torrent, fileIndex, key) {
|
|
131
|
+
const state = stateFor(key);
|
|
132
|
+
if (state.plan !== null) {
|
|
133
|
+
return state.plan;
|
|
134
|
+
}
|
|
135
|
+
// The head and the Cues table are two reads that DO wait on the swarm, so two
|
|
136
|
+
// callers arriving together would both make them. One promise, awaited by
|
|
137
|
+
// whoever asks while it is in flight.
|
|
138
|
+
if (!state.planPromise) {
|
|
139
|
+
state.planPromise = readPlan(torrent, fileIndex, state).finally(() => {
|
|
140
|
+
state.planPromise = null;
|
|
141
|
+
});
|
|
142
|
+
}
|
|
143
|
+
return state.planPromise;
|
|
144
|
+
}
|
|
145
|
+
|
|
146
|
+
/**
|
|
147
|
+
* The state kept for one file, created on first use.
|
|
148
|
+
*
|
|
149
|
+
* @param {string} key - `sourceKey:fileIndex`.
|
|
150
|
+
* @returns {object}
|
|
151
|
+
*/
|
|
152
|
+
function stateFor(key) {
|
|
100
153
|
let state = byFile.get(key);
|
|
154
|
+
// A state that has been forgotten is not handed out again, even in the moment
|
|
155
|
+
// between the call and the walk that was still running finishing.
|
|
156
|
+
if (state?.forgotten === true) {
|
|
157
|
+
state = undefined;
|
|
158
|
+
}
|
|
101
159
|
if (!state) {
|
|
102
160
|
state = {
|
|
103
161
|
plan: null,
|
|
162
|
+
planPromise: null,
|
|
163
|
+
forgotten: false,
|
|
164
|
+
// One walk of a file at a time — see `serialize`.
|
|
165
|
+
chain: Promise.resolve(),
|
|
104
166
|
harvested: new Map(),
|
|
105
167
|
cues: new Map(),
|
|
106
168
|
seq: new Map(),
|
|
@@ -112,9 +174,45 @@ async function planFor(torrent, fileIndex, key) {
|
|
|
112
174
|
};
|
|
113
175
|
byFile.set(key, state);
|
|
114
176
|
}
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
177
|
+
return state;
|
|
178
|
+
}
|
|
179
|
+
|
|
180
|
+
/**
|
|
181
|
+
* Run `work` after every walk of this file already started, and before any
|
|
182
|
+
* started after it.
|
|
183
|
+
*
|
|
184
|
+
* Both entry points here — a browser's own pull and the warmup that runs ahead
|
|
185
|
+
* of it — mark a cluster as walked only AFTER reading and parsing it, which is
|
|
186
|
+
* two suspension points later. Until 2.56.0 nothing stopped a second call
|
|
187
|
+
* arriving in between: `warmActiveFiles` runs on every verified piece AND on a
|
|
188
|
+
* 3 s timer, so on a fast download the same cluster was read and parsed several
|
|
189
|
+
* times over and the same line could be pushed twice under different `seq`
|
|
190
|
+
* numbers. Each of those reads is a WebTorrent file stream, which selects and
|
|
191
|
+
* deselects its pieces, so the repetition reached the piece picker as well.
|
|
192
|
+
*
|
|
193
|
+
* @template T
|
|
194
|
+
* @param {object} state
|
|
195
|
+
* @param {() => Promise<T>} work
|
|
196
|
+
* @returns {Promise<T>}
|
|
197
|
+
*/
|
|
198
|
+
function serialize(state, work) {
|
|
199
|
+
const run = state.chain.then(work, work);
|
|
200
|
+
// The queue must survive a failed walk, so what is chained is the settled
|
|
201
|
+
// form; the caller still sees the rejection.
|
|
202
|
+
state.chain = run.then(() => undefined, () => undefined);
|
|
203
|
+
return run;
|
|
204
|
+
}
|
|
205
|
+
|
|
206
|
+
/**
|
|
207
|
+
* Read one file's subtitle plan — the tracks it declares and where the clusters
|
|
208
|
+
* holding them are. Called once per file; see `planFor`.
|
|
209
|
+
*
|
|
210
|
+
* @param {object} torrent
|
|
211
|
+
* @param {number} fileIndex
|
|
212
|
+
* @param {object} state
|
|
213
|
+
* @returns {Promise<object>}
|
|
214
|
+
*/
|
|
215
|
+
async function readPlan(torrent, fileIndex, state) {
|
|
118
216
|
const file = torrent?.files?.[fileIndex];
|
|
119
217
|
// `declared` is what the container itself says about its subtitle tracks, in
|
|
120
218
|
// its own order. Empty means the container said nothing — which is a real
|
|
@@ -139,6 +237,7 @@ async function planFor(torrent, fileIndex, key) {
|
|
|
139
237
|
...empty,
|
|
140
238
|
tracks: mp4.tracks.map((track, order) => ({
|
|
141
239
|
trackNumber: track.trackId,
|
|
240
|
+
declaredIndex: Number.isInteger(track.declaredIndex) ? track.declaredIndex : order,
|
|
142
241
|
codecId: track.format,
|
|
143
242
|
language: track.language,
|
|
144
243
|
name: "",
|
|
@@ -159,10 +258,13 @@ async function planFor(torrent, fileIndex, key) {
|
|
|
159
258
|
state.plan = plan ?? empty;
|
|
160
259
|
if (state.plan.tracks.length > 0) {
|
|
161
260
|
logger.info(
|
|
162
|
-
`subtitles: "${String(file.name).slice(0, 40)}" has ${state.plan.tracks.length} text track(s)
|
|
261
|
+
`subtitles: "${String(file.name).slice(0, 40)}" has ${state.plan.tracks.length} text track(s) ` +
|
|
262
|
+
`of ${state.plan.declared.length} declared — ` +
|
|
163
263
|
state.plan.tracks
|
|
164
|
-
|
|
165
|
-
|
|
264
|
+
// `s:N` is the number the browser names (ffmpeg's own), and it differs
|
|
265
|
+
// from the file's track number whenever a picture track sits among them.
|
|
266
|
+
.map((track) => `s:${track.declaredIndex}=${track.trackNumber}:${track.language || "?"}` +
|
|
267
|
+
`${track.name ? `/${track.name}` : ""}(${track.clusterPositions.length} indexed)`)
|
|
166
268
|
.join(" ")
|
|
167
269
|
);
|
|
168
270
|
}
|
|
@@ -206,11 +308,27 @@ function nextSeq(state, trackNumber) {
|
|
|
206
308
|
export async function cuesHeldFor(torrent, fileIndex, sourceKey, trackNumber) {
|
|
207
309
|
const key = `${sourceKey}:${fileIndex}`;
|
|
208
310
|
const plan = await planFor(torrent, fileIndex, key);
|
|
209
|
-
const state =
|
|
311
|
+
const state = stateFor(key);
|
|
210
312
|
const track = plan?.tracks?.find((candidate) => candidate.trackNumber === trackNumber) ?? null;
|
|
211
313
|
if (!track) {
|
|
212
314
|
return { cues: [], coveredClusters: 0, indexedClusters: 0, track: null };
|
|
213
315
|
}
|
|
316
|
+
return serialize(state, () => walkFor(torrent, fileIndex, state, plan, track, trackNumber));
|
|
317
|
+
}
|
|
318
|
+
|
|
319
|
+
/**
|
|
320
|
+
* The walk itself. Only ever entered through `cuesHeldFor`, which is what keeps
|
|
321
|
+
* one file to one walk at a time.
|
|
322
|
+
*
|
|
323
|
+
* @param {object} torrent
|
|
324
|
+
* @param {number} fileIndex
|
|
325
|
+
* @param {object} state
|
|
326
|
+
* @param {object} plan
|
|
327
|
+
* @param {object} track
|
|
328
|
+
* @param {number} trackNumber
|
|
329
|
+
* @returns {Promise<{ cues: object[], coveredClusters: number, indexedClusters: number, track: object | null }>}
|
|
330
|
+
*/
|
|
331
|
+
async function walkFor(torrent, fileIndex, state, plan, track, trackNumber) {
|
|
214
332
|
const file = torrent.files[fileIndex];
|
|
215
333
|
let harvested = state.harvested.get(trackNumber);
|
|
216
334
|
if (!harvested) {
|
|
@@ -343,18 +461,20 @@ export async function cuesHeldFor(torrent, fileIndex, sourceKey, trackNumber) {
|
|
|
343
461
|
* @param {string} sourceKey
|
|
344
462
|
* @returns {Promise<{ trackIndex: number, cues: object[], language: string }[]>}
|
|
345
463
|
* One entry per track that gained at least one cue since the last call.
|
|
346
|
-
* `trackIndex` is the track's position among
|
|
347
|
-
*
|
|
348
|
-
* container's own track number,
|
|
464
|
+
* `trackIndex` is `declaredIndex` — the track's position among ALL the file's
|
|
465
|
+
* subtitle tracks, which is ffmpeg's `0:s:N` and the only number the browser
|
|
466
|
+
* knows. NOT the container's own track number, and not the position among the
|
|
467
|
+
* readable tracks either: counting those alone puts every text track after a
|
|
468
|
+
* picture-based one in the wrong place.
|
|
349
469
|
*/
|
|
350
470
|
export async function warmSubtitleCues(torrent, fileIndex, sourceKey) {
|
|
351
471
|
const key = `${sourceKey}:${fileIndex}`;
|
|
352
472
|
const plan = await planFor(torrent, fileIndex, key);
|
|
353
|
-
const state =
|
|
473
|
+
const state = stateFor(key);
|
|
354
474
|
const fresh = [];
|
|
355
475
|
const tracks = plan?.tracks ?? [];
|
|
356
|
-
for (let
|
|
357
|
-
const track = tracks[
|
|
476
|
+
for (let order = 0; order < tracks.length; order += 1) {
|
|
477
|
+
const track = tracks[order];
|
|
358
478
|
const held = await cuesHeldFor(torrent, fileIndex, sourceKey, track.trackNumber);
|
|
359
479
|
const since = state.pushed.get(track.trackNumber) ?? 0;
|
|
360
480
|
const newCues = held.cues.filter((cue) => (Number(cue.seq) || 0) > since);
|
|
@@ -363,10 +483,21 @@ export async function warmSubtitleCues(torrent, fileIndex, sourceKey) {
|
|
|
363
483
|
}
|
|
364
484
|
const highest = newCues.reduce((max, cue) => Math.max(max, Number(cue.seq) || 0), since);
|
|
365
485
|
state.pushed.set(track.trackNumber, highest);
|
|
486
|
+
const cues = finalizeCues(newCues, held.track?.codecId ?? track.codecId);
|
|
366
487
|
fresh.push({
|
|
367
|
-
|
|
368
|
-
|
|
369
|
-
|
|
488
|
+
// ffmpeg's own numbering, which is the only one the browser knows.
|
|
489
|
+
trackIndex: Number.isInteger(track.declaredIndex) ? track.declaredIndex : order,
|
|
490
|
+
cues,
|
|
491
|
+
language: held.track?.language ?? "",
|
|
492
|
+
// Where the browser should resume from if it has to ask again — after a
|
|
493
|
+
// reconnect, which loses the subscription these pushes ride on.
|
|
494
|
+
cursor: highest,
|
|
495
|
+
// What this batch is ABOUT, in film time, so a log can be read against
|
|
496
|
+
// the position being played.
|
|
497
|
+
spanStartSeconds: cues.length > 0 ? cues[0].startSeconds : null,
|
|
498
|
+
spanEndSeconds: cues.length > 0 ? cues[cues.length - 1].endSeconds : null,
|
|
499
|
+
walkedClusters: held.coveredClusters ?? 0,
|
|
500
|
+
indexedClusters: held.indexedClusters ?? 0
|
|
370
501
|
});
|
|
371
502
|
}
|
|
372
503
|
return fresh;
|
|
@@ -430,8 +561,9 @@ function assDialogueToText(raw) {
|
|
|
430
561
|
*/
|
|
431
562
|
export async function subtitleTracksOf(torrent, fileIndex, sourceKey) {
|
|
432
563
|
const plan = await planFor(torrent, fileIndex, `${sourceKey}:${fileIndex}`);
|
|
433
|
-
return (plan?.tracks ?? []).map((track) => ({
|
|
564
|
+
return (plan?.tracks ?? []).map((track, order) => ({
|
|
434
565
|
trackNumber: track.trackNumber,
|
|
566
|
+
declaredIndex: Number.isInteger(track.declaredIndex) ? track.declaredIndex : order,
|
|
435
567
|
codecId: track.codecId,
|
|
436
568
|
language: track.language,
|
|
437
569
|
name: track.name,
|
|
@@ -471,10 +603,39 @@ export function forgetSubtitles(sourceKey, fileIndex) {
|
|
|
471
603
|
if (fileIndex === undefined) {
|
|
472
604
|
for (const key of [...byFile.keys()]) {
|
|
473
605
|
if (key.startsWith(`${sourceKey}:`)) {
|
|
474
|
-
|
|
606
|
+
forgetOne(key);
|
|
475
607
|
}
|
|
476
608
|
}
|
|
477
609
|
return;
|
|
478
610
|
}
|
|
479
|
-
|
|
611
|
+
forgetOne(`${sourceKey}:${fileIndex}`);
|
|
612
|
+
}
|
|
613
|
+
|
|
614
|
+
/**
|
|
615
|
+
* Drop one file's state, but not while a walk of it is still running: the
|
|
616
|
+
* record of which clusters have been read lives in that state, and a walk left
|
|
617
|
+
* writing into a discarded copy while a new one starts beside it is the one
|
|
618
|
+
* path that defeats the serialization above.
|
|
619
|
+
*
|
|
620
|
+
* @param {string} key
|
|
621
|
+
* @returns {void}
|
|
622
|
+
*/
|
|
623
|
+
function forgetOne(key) {
|
|
624
|
+
const state = byFile.get(key);
|
|
625
|
+
if (!state) {
|
|
626
|
+
return;
|
|
627
|
+
}
|
|
628
|
+
// Held, so that a walk started before this call is not left orphaned; the
|
|
629
|
+
// entry is dropped the moment the queue empties, and nothing is handed this
|
|
630
|
+
// state in the meantime.
|
|
631
|
+
state.forgotten = true;
|
|
632
|
+
void state.chain.then(() => {
|
|
633
|
+
if (byFile.get(key) === state) {
|
|
634
|
+
byFile.delete(key);
|
|
635
|
+
}
|
|
636
|
+
}, () => {
|
|
637
|
+
if (byFile.get(key) === state) {
|
|
638
|
+
byFile.delete(key);
|
|
639
|
+
}
|
|
640
|
+
});
|
|
480
641
|
}
|
|
@@ -522,12 +522,27 @@ function warmActiveFiles(sourceKey, torrent) {
|
|
|
522
522
|
return;
|
|
523
523
|
}
|
|
524
524
|
for (const fileIndex of usage.keys()) {
|
|
525
|
+
const key = `${sourceKey}:${fileIndex}`;
|
|
526
|
+
// A trigger that arrives while the previous pass is still walking is
|
|
527
|
+
// dropped, not queued. `verified` fires per piece, so on a fast download
|
|
528
|
+
// these arrive many times a second; the walk is serialized per file anyway,
|
|
529
|
+
// and a queue of identical passes would only postpone the one that has
|
|
530
|
+
// something new to find.
|
|
531
|
+
if (warmupInFlight.has(key)) {
|
|
532
|
+
continue;
|
|
533
|
+
}
|
|
534
|
+
warmupInFlight.add(key);
|
|
525
535
|
warmSubtitleCues(torrent, fileIndex, sourceKey)
|
|
526
536
|
.then((fresh) => {
|
|
527
537
|
for (const entry of fresh) {
|
|
538
|
+
const span = entry.spanStartSeconds === null
|
|
539
|
+
? "empty"
|
|
540
|
+
: `${entry.spanStartSeconds.toFixed(1)}-${entry.spanEndSeconds.toFixed(1)}s`;
|
|
528
541
|
log(
|
|
529
542
|
`subtitle push ${sourceKey.slice(0, 8)}:${fileIndex} track ${entry.trackIndex}: ` +
|
|
530
|
-
`${entry.cues.length} new cue(s)
|
|
543
|
+
`${entry.cues.length} new cue(s) covering ${span}, ` +
|
|
544
|
+
`clusters walked ${entry.walkedClusters}/${entry.indexedClusters}, cursor ${entry.cursor}, ` +
|
|
545
|
+
"posting to main thread"
|
|
531
546
|
);
|
|
532
547
|
parentPort.postMessage({
|
|
533
548
|
type: Event.SUBTITLE_CUES_READY,
|
|
@@ -535,16 +550,27 @@ function warmActiveFiles(sourceKey, torrent) {
|
|
|
535
550
|
fileIndex,
|
|
536
551
|
trackIndex: entry.trackIndex,
|
|
537
552
|
cues: entry.cues,
|
|
538
|
-
language: entry.language
|
|
553
|
+
language: entry.language,
|
|
554
|
+
cursor: entry.cursor
|
|
539
555
|
});
|
|
540
556
|
}
|
|
541
557
|
})
|
|
542
558
|
.catch((error) => {
|
|
543
559
|
log(`subtitle warmup ${sourceKey}:${fileIndex} failed: ${error instanceof Error ? error.message : error}`);
|
|
560
|
+
})
|
|
561
|
+
.finally(() => {
|
|
562
|
+
warmupInFlight.delete(key);
|
|
544
563
|
});
|
|
545
564
|
}
|
|
546
565
|
}
|
|
547
566
|
|
|
567
|
+
/**
|
|
568
|
+
* Files whose warmup pass has not finished yet, by `sourceKey:fileIndex`.
|
|
569
|
+
*
|
|
570
|
+
* @type {Set<string>}
|
|
571
|
+
*/
|
|
572
|
+
const warmupInFlight = new Set();
|
|
573
|
+
|
|
548
574
|
/**
|
|
549
575
|
* Torrents already wired to warm their subtitle cues the moment a piece
|
|
550
576
|
* verifies, so the same torrent is not listened to twice.
|
|
@@ -0,0 +1,262 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @file The number a subtitle track is asked for by, and one walk per file.
|
|
3
|
+
*
|
|
4
|
+
* Two rules, both found by reading on 2026-08-26 after the report that embedded
|
|
5
|
+
* subtitles appear late (`research/subtitle-delay-2026-08-26.md`):
|
|
6
|
+
*
|
|
7
|
+
* 1. The browser names a track by ffmpeg's `0:s:N`, which counts EVERY subtitle
|
|
8
|
+
* stream. The container plan drops the picture-based ones — PGS, VobSub —
|
|
9
|
+
* because they cannot become WebVTT, so counting the kept ones is a
|
|
10
|
+
* different numbering as soon as a file carries one of each. What that cost:
|
|
11
|
+
* the push landed on a track the browser does not know, and the browser's
|
|
12
|
+
* own request found no track at all and fell through to the ffmpeg
|
|
13
|
+
* extraction, which reads the whole film (752 s measured, 2026-08-19).
|
|
14
|
+
* 2. A file is walked once at a time. The walk marks a cluster as read only
|
|
15
|
+
* after two suspension points, and it is started both on every verified
|
|
16
|
+
* piece and on a 3 s timer, so two passes could read and parse the same
|
|
17
|
+
* cluster and push the same line twice.
|
|
18
|
+
*/
|
|
19
|
+
|
|
20
|
+
import test from "node:test";
|
|
21
|
+
import assert from "node:assert/strict";
|
|
22
|
+
import { Readable } from "node:stream";
|
|
23
|
+
import { readSubtitlePlan } from "../services/container-index/matroska-subtitles.js";
|
|
24
|
+
import { cuesHeldFor, forgetSubtitles } from "../services/torrent-worker/subtitle-cues.js";
|
|
25
|
+
|
|
26
|
+
const ID_EBML = 0x1a45dfa3;
|
|
27
|
+
const ID_SEGMENT = 0x18538067;
|
|
28
|
+
const ID_SEEK_HEAD = 0x114d9b74;
|
|
29
|
+
const ID_SEEK = 0x4dbb;
|
|
30
|
+
const ID_SEEK_ID = 0x53ab;
|
|
31
|
+
const ID_SEEK_POSITION = 0x53ac;
|
|
32
|
+
const ID_INFO = 0x1549a966;
|
|
33
|
+
const ID_TIMESTAMP_SCALE = 0x2ad7b1;
|
|
34
|
+
const ID_TRACKS = 0x1654ae6b;
|
|
35
|
+
const ID_TRACK_ENTRY = 0xae;
|
|
36
|
+
const ID_TRACK_NUMBER = 0xd7;
|
|
37
|
+
const ID_TRACK_TYPE = 0x83;
|
|
38
|
+
const ID_CODEC_ID = 0x86;
|
|
39
|
+
const ID_LANGUAGE = 0x22b59c;
|
|
40
|
+
const ID_CUES = 0x1c53bb6b;
|
|
41
|
+
const ID_CUE_POINT = 0xbb;
|
|
42
|
+
const ID_CUE_TIME = 0xb3;
|
|
43
|
+
const ID_CUE_TRACK_POSITIONS = 0xb7;
|
|
44
|
+
const ID_CUE_TRACK = 0xf7;
|
|
45
|
+
const ID_CUE_CLUSTER_POSITION = 0xf1;
|
|
46
|
+
const ID_CLUSTER = 0x1f43b675;
|
|
47
|
+
const ID_TIMESTAMP = 0xe7;
|
|
48
|
+
|
|
49
|
+
/** An element id, as the bytes the specification gives it. */
|
|
50
|
+
function idBytes(id) {
|
|
51
|
+
const bytes = [];
|
|
52
|
+
let rest = id;
|
|
53
|
+
while (rest > 0) {
|
|
54
|
+
bytes.unshift(rest & 0xff);
|
|
55
|
+
rest = Math.floor(rest / 256);
|
|
56
|
+
}
|
|
57
|
+
return Buffer.from(bytes);
|
|
58
|
+
}
|
|
59
|
+
|
|
60
|
+
/** A size, as a four-byte EBML variable-length integer. */
|
|
61
|
+
function sizeBytes(size) {
|
|
62
|
+
const buffer = Buffer.alloc(4);
|
|
63
|
+
buffer.writeUInt32BE(size, 0);
|
|
64
|
+
buffer[0] |= 0x10;
|
|
65
|
+
return buffer;
|
|
66
|
+
}
|
|
67
|
+
|
|
68
|
+
function element(id, payload) {
|
|
69
|
+
return Buffer.concat([idBytes(id), sizeBytes(payload.length), payload]);
|
|
70
|
+
}
|
|
71
|
+
|
|
72
|
+
function uintElement(id, value) {
|
|
73
|
+
const bytes = [];
|
|
74
|
+
let rest = value;
|
|
75
|
+
do {
|
|
76
|
+
bytes.unshift(rest & 0xff);
|
|
77
|
+
rest = Math.floor(rest / 256);
|
|
78
|
+
} while (rest > 0);
|
|
79
|
+
return element(id, Buffer.from(bytes));
|
|
80
|
+
}
|
|
81
|
+
|
|
82
|
+
function stringElement(id, value) {
|
|
83
|
+
return element(id, Buffer.from(value, "utf8"));
|
|
84
|
+
}
|
|
85
|
+
|
|
86
|
+
/**
|
|
87
|
+
* An unsigned value at a FIXED four bytes. A cue's cluster position has to be
|
|
88
|
+
* written twice — once to measure the table, once with the position that
|
|
89
|
+
* measurement produced — and a value-sized element would make the second table
|
|
90
|
+
* a different length from the first, moving the very cluster it names.
|
|
91
|
+
*/
|
|
92
|
+
function uint32Element(id, value) {
|
|
93
|
+
const payload = Buffer.alloc(4);
|
|
94
|
+
payload.writeUInt32BE(value, 0);
|
|
95
|
+
return element(id, payload);
|
|
96
|
+
}
|
|
97
|
+
|
|
98
|
+
function trackEntry({ number, type, codecId, language }) {
|
|
99
|
+
return element(ID_TRACK_ENTRY, Buffer.concat([
|
|
100
|
+
uintElement(ID_TRACK_NUMBER, number),
|
|
101
|
+
uintElement(ID_TRACK_TYPE, type),
|
|
102
|
+
stringElement(ID_CODEC_ID, codecId),
|
|
103
|
+
stringElement(ID_LANGUAGE, language)
|
|
104
|
+
]));
|
|
105
|
+
}
|
|
106
|
+
|
|
107
|
+
/**
|
|
108
|
+
* A file whose subtitle tracks are, in the container's own order: a picture
|
|
109
|
+
* one, then two text ones. ffmpeg numbers those `0:s:0`, `0:s:1`, `0:s:2`;
|
|
110
|
+
* the plan can only read the last two.
|
|
111
|
+
*
|
|
112
|
+
* The Cues table points both text tracks at one cluster, which is written after
|
|
113
|
+
* the table so its position can be stated.
|
|
114
|
+
*
|
|
115
|
+
* @returns {{ file: Buffer, clusterAt: number }}
|
|
116
|
+
*/
|
|
117
|
+
function buildFile() {
|
|
118
|
+
const info = element(ID_INFO, uintElement(ID_TIMESTAMP_SCALE, 1_000_000));
|
|
119
|
+
const tracks = element(ID_TRACKS, Buffer.concat([
|
|
120
|
+
trackEntry({ number: 1, type: 1, codecId: "V_MPEG4/ISO/AVC", language: "und" }),
|
|
121
|
+
trackEntry({ number: 2, type: 17, codecId: "S_HDMV/PGS", language: "eng" }),
|
|
122
|
+
trackEntry({ number: 3, type: 17, codecId: "S_TEXT/UTF8", language: "rus" }),
|
|
123
|
+
trackEntry({ number: 4, type: 17, codecId: "S_TEXT/ASS", language: "eng" })
|
|
124
|
+
]));
|
|
125
|
+
|
|
126
|
+
// Built twice: the cue points state where the cluster is, and that position
|
|
127
|
+
// is only known once everything before it has its final length. Every size
|
|
128
|
+
// and position here is written at a fixed width, so the draft and the final
|
|
129
|
+
// table are the same length.
|
|
130
|
+
const cuesWith = (clusterAt) => element(ID_CUES, Buffer.concat([
|
|
131
|
+
element(ID_CUE_POINT, Buffer.concat([
|
|
132
|
+
uintElement(ID_CUE_TIME, 1000),
|
|
133
|
+
element(ID_CUE_TRACK_POSITIONS, Buffer.concat([
|
|
134
|
+
uintElement(ID_CUE_TRACK, 3),
|
|
135
|
+
uint32Element(ID_CUE_CLUSTER_POSITION, clusterAt)
|
|
136
|
+
])),
|
|
137
|
+
element(ID_CUE_TRACK_POSITIONS, Buffer.concat([
|
|
138
|
+
uintElement(ID_CUE_TRACK, 4),
|
|
139
|
+
uint32Element(ID_CUE_CLUSTER_POSITION, clusterAt)
|
|
140
|
+
]))
|
|
141
|
+
]))
|
|
142
|
+
]));
|
|
143
|
+
|
|
144
|
+
const seekEntry = (targetId, position) => element(ID_SEEK, Buffer.concat([
|
|
145
|
+
element(ID_SEEK_ID, idBytes(targetId)),
|
|
146
|
+
element(ID_SEEK_POSITION, (() => {
|
|
147
|
+
const buffer = Buffer.alloc(4);
|
|
148
|
+
buffer.writeUInt32BE(position, 0);
|
|
149
|
+
return buffer;
|
|
150
|
+
})())
|
|
151
|
+
]));
|
|
152
|
+
const seekHeadWith = (infoAt, tracksAt, cuesAt) => element(ID_SEEK_HEAD, Buffer.concat([
|
|
153
|
+
seekEntry(ID_INFO, infoAt),
|
|
154
|
+
seekEntry(ID_TRACKS, tracksAt),
|
|
155
|
+
seekEntry(ID_CUES, cuesAt)
|
|
156
|
+
]));
|
|
157
|
+
|
|
158
|
+
const headLength = seekHeadWith(0, 0, 0).length;
|
|
159
|
+
const infoAt = headLength;
|
|
160
|
+
const tracksAt = infoAt + info.length;
|
|
161
|
+
const cuesAt = tracksAt + tracks.length;
|
|
162
|
+
// A position in the Cues table is measured from the Segment's payload, and so
|
|
163
|
+
// is the one the reader turns it into.
|
|
164
|
+
const clusterRelative = cuesAt + cuesWith(0).length;
|
|
165
|
+
|
|
166
|
+
// Enough of a cluster to be read and recognised: its own header and a
|
|
167
|
+
// timestamp. No blocks, so it yields no cues — what the walk test counts is
|
|
168
|
+
// that its bytes are fetched once, and that does not depend on their content.
|
|
169
|
+
const cluster = element(ID_CLUSTER, uintElement(ID_TIMESTAMP, 1000));
|
|
170
|
+
|
|
171
|
+
const segmentPayload = Buffer.concat([
|
|
172
|
+
seekHeadWith(infoAt, tracksAt, cuesAt),
|
|
173
|
+
info,
|
|
174
|
+
tracks,
|
|
175
|
+
cuesWith(clusterRelative),
|
|
176
|
+
cluster
|
|
177
|
+
]);
|
|
178
|
+
const ebml = element(ID_EBML, Buffer.from([0x42, 0x86, 0x81, 0x01]));
|
|
179
|
+
const segment = element(ID_SEGMENT, segmentPayload);
|
|
180
|
+
const segmentDataOffset = ebml.length + segment.length - segmentPayload.length;
|
|
181
|
+
return {
|
|
182
|
+
file: Buffer.concat([ebml, segment]),
|
|
183
|
+
clusterAt: segmentDataOffset + clusterRelative
|
|
184
|
+
};
|
|
185
|
+
}
|
|
186
|
+
|
|
187
|
+
function readerOver(file) {
|
|
188
|
+
return async (start, end) => {
|
|
189
|
+
const last = Math.min(end, file.length - 1);
|
|
190
|
+
return start > last ? null : file.subarray(start, last + 1);
|
|
191
|
+
};
|
|
192
|
+
}
|
|
193
|
+
|
|
194
|
+
test("a text track is numbered as ffmpeg numbers it, past the picture ones", async () => {
|
|
195
|
+
const { file } = buildFile();
|
|
196
|
+
|
|
197
|
+
const plan = await readSubtitlePlan(readerOver(file), file.length);
|
|
198
|
+
|
|
199
|
+
assert.equal(plan.declared.length, 3, "all three subtitle tracks are declared");
|
|
200
|
+
assert.deepEqual(plan.tracks.map((track) => track.trackNumber), [3, 4], "only the text ones are readable");
|
|
201
|
+
assert.deepEqual(
|
|
202
|
+
plan.tracks.map((track) => track.declaredIndex),
|
|
203
|
+
[1, 2],
|
|
204
|
+
"the PGS track is 0:s:0, so the text tracks are 0:s:1 and 0:s:2 — not 0 and 1"
|
|
205
|
+
);
|
|
206
|
+
});
|
|
207
|
+
|
|
208
|
+
/**
|
|
209
|
+
* A torrent holding one file entirely, counting the byte ranges read from it.
|
|
210
|
+
*
|
|
211
|
+
* @param {Buffer} bytes
|
|
212
|
+
* @returns {{ torrent: object, reads: Array<{ start: number, end: number }> }}
|
|
213
|
+
*/
|
|
214
|
+
function torrentOver(bytes) {
|
|
215
|
+
const reads = [];
|
|
216
|
+
const file = {
|
|
217
|
+
name: "film.mkv",
|
|
218
|
+
length: bytes.length,
|
|
219
|
+
offset: 0,
|
|
220
|
+
createReadStream({ start = 0, end = bytes.length - 1 } = {}) {
|
|
221
|
+
reads.push({ start, end });
|
|
222
|
+
// Asynchronous on purpose: a read that resolves in the same tick would
|
|
223
|
+
// hide exactly the interleaving this test is about.
|
|
224
|
+
return Readable.from((async function* chunks() {
|
|
225
|
+
await new Promise((resolve) => setImmediate(resolve));
|
|
226
|
+
yield bytes.subarray(start, end + 1);
|
|
227
|
+
})());
|
|
228
|
+
}
|
|
229
|
+
};
|
|
230
|
+
return {
|
|
231
|
+
reads,
|
|
232
|
+
torrent: {
|
|
233
|
+
pieceLength: 1024,
|
|
234
|
+
bitfield: { get: () => true },
|
|
235
|
+
files: [file]
|
|
236
|
+
}
|
|
237
|
+
};
|
|
238
|
+
}
|
|
239
|
+
|
|
240
|
+
test("two walks of one file at the same time read each cluster once", async () => {
|
|
241
|
+
const { file, clusterAt } = buildFile();
|
|
242
|
+
const { torrent, reads } = torrentOver(file);
|
|
243
|
+
const sourceKey = "torrent:numbering-test";
|
|
244
|
+
forgetSubtitles(sourceKey);
|
|
245
|
+
|
|
246
|
+
// Both text tracks at once, which is what the warmup does on every verified
|
|
247
|
+
// piece and every three seconds.
|
|
248
|
+
const [first, second] = await Promise.all([
|
|
249
|
+
cuesHeldFor(torrent, 0, sourceKey, 3),
|
|
250
|
+
cuesHeldFor(torrent, 0, sourceKey, 4)
|
|
251
|
+
]);
|
|
252
|
+
|
|
253
|
+
assert.equal(first.coveredClusters, 1, "the cluster the table names was walked");
|
|
254
|
+
assert.equal(second.coveredClusters, 1, "and the second track sees the same walk, not its own");
|
|
255
|
+
const clusterReads = reads.filter((range) => range.start === clusterAt);
|
|
256
|
+
assert.equal(
|
|
257
|
+
clusterReads.length,
|
|
258
|
+
2,
|
|
259
|
+
"one probe of the cluster's header and one read of its body — not two of each"
|
|
260
|
+
);
|
|
261
|
+
forgetSubtitles(sourceKey);
|
|
262
|
+
});
|