@torrent-tv/proxy 2.55.14 → 2.56.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,3 +1,11 @@
1
+ ## 2.56.0
2
+
3
+ - **Fix**: A subtitle track is asked for by ffmpeg's own number, `0:s:N`, which counts EVERY subtitle stream the file carries — and the container plan counted only the ones it can turn into text, dropping PGS and VobSub before numbering. On a release whose picture-based track comes first the two numberings ran a place apart, with two consequences, both silent: a pushed cue named a track the browser does not know and was dropped, and the browser's own request found no track at all and fell through to the ffmpeg extraction, which reads the whole film for a few kilobytes of text (752 s measured on one file, 2026-08-19). Every text track now carries `declaredIndex` — its position among all the file's subtitle tracks, counted by what a track IS and not by what this code can read — and both the push and `/api/subtitles?trackIndex=` use it. In an MP4 that count includes the `subp` and `clcp` handlers, which ffmpeg also calls subtitle streams.
4
+ - **Fix**: One walk of a file at a time. The walk marks a cluster as read only after fetching and parsing it — two suspension points — while it is started both on every verified piece and on a 3 s timer, so on a fast download several passes read and parsed the same cluster and could push one line twice under different found-order numbers. Each of those reads is a WebTorrent file stream, which selects and deselects its pieces, so the repetition reached the piece picker as well. Walks are now serialized per file, the plan is read once even when two callers arrive together, a warmup triggered while the previous one is still walking is dropped rather than queued, and forgetting a file waits for its walk instead of leaving one running beside a fresh copy of the state.
5
+ - **Fix**: A read of already-downloaded bytes that never ends is given up after 30 s, with a line saying so. It had no bound, and with walks now serialized one such read would have held that file's queue — the browser's own request for its subtitles included — for the rest of the session.
6
+ - **New**: A subtitle push says what it is ABOUT: the film-time span its new cues cover, how many of the file's indexed clusters have been walked, and the found-order cursor. Read against the position being played — which the browser now logs beside it — that separates cues that arrived late from cues that arrived early for a stretch nobody is watching. Report 2026-08-26: embedded subtitles appear "after some time", and no line on either side could say whether the cues held covered the playhead (`research/subtitle-delay-2026-08-26.md`).
7
+ - **New**: The push carries a `cursor`, so a browser that loses the subscription — which a reconnect does, since the subscription belongs to the channel — can ask for exactly what it missed instead of the whole track.
8
+
1
9
  ## 2.55.14
2
10
 
3
11
  - **Fix**: `utp-native` moves to 2.5.3-ttv.5, which carries ten defects found by reading the whole binding after the seventh crash of this family named its frame. Three callbacks read the connection pointer without checking it exists — and the socket carries none unless it was accepted or dialled, while its destructor announces itself regardless; the ttv.4 patch created one such path itself. The read callback copied at an accumulated offset without ever consulting the buffer length it maintains, so a peer sending more between two hand-offs wrote past the end — heap corruption does not fault where it happens, which is what six deaths inside libuv bookkeeping look like. A connection could be destroyed twice, deleting already-deleted napi references, and the socket pointer was never cleared. The callback macro checked none of the three napi results it then used. Plus IPv6, which the module never had at all, and the resolver that fed it addresses it could not use. Tests on the target: 77 checks, no failures — two of them were failing or hanging before.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@torrent-tv/proxy",
3
- "version": "2.55.14",
3
+ "version": "2.56.0",
4
4
  "description": "Torrent proxy client that exposes webseed-like HTTP stream endpoint.",
5
5
  "license": "GPL-3.0-or-later",
6
6
  "publishConfig": {
@@ -268,9 +268,10 @@ function readFileFully(file, maxBytes) {
268
268
  * The cues of a track from clusters already downloaded, as WebVTT.
269
269
  *
270
270
  * `trackIndex` is the browser's number for the subtitle stream — its position
271
- * among the subtitle streams, as ffprobe lists them — while Matroska blocks
272
- * carry the file's own track number. The plan lists the text tracks in file
273
- * order, so the browser's Nth subtitle stream is the Nth entry.
271
+ * among ALL the subtitle streams, as ffmpeg lists them — while Matroska blocks
272
+ * carry the file's own track number. The plan lists only the tracks that can
273
+ * become text, so it is matched on `declaredIndex`, which each track carries
274
+ * for exactly this: its place in the file's full list of subtitle tracks.
274
275
  *
275
276
  * @param {object} torrentPool
276
277
  * @param {object} torrent
@@ -289,7 +290,15 @@ async function cuesFromDownloadedClusters(torrentPool, torrent, fileIndex, track
289
290
  } catch {
290
291
  return null;
291
292
  }
292
- const track = Array.isArray(tracks) ? tracks[trackIndex] : null;
293
+ // BY the browser's number, not by position in this list. The list holds only
294
+ // the tracks that can become WebVTT, while the browser counts every subtitle
295
+ // stream ffmpeg lists — so on a file carrying a PGS or VobSub track the two
296
+ // ran one apart, and the request either found the wrong track or found none
297
+ // and fell through to the ffmpeg extraction below, which reads the whole film
298
+ // (752 s measured, 2026-08-19) for cues already in hand.
299
+ const track = Array.isArray(tracks)
300
+ ? tracks.find((candidate) => candidate.declaredIndex === trackIndex) ?? null
301
+ : null;
293
302
  if (!track) {
294
303
  return null;
295
304
  }
@@ -63,6 +63,11 @@ const TEXT_CODECS = new Set(["S_TEXT/UTF8", "S_TEXT/ASS", "S_TEXT/SSA"]);
63
63
  /**
64
64
  * @typedef {object} SubtitleTrackPlan
65
65
  * @property {number} trackNumber - As the blocks name it.
66
+ * @property {number} declaredIndex - Its position among ALL of the file's
67
+ * subtitle tracks, picture-based ones included — which is the number ffmpeg
68
+ * gives the same stream in `0:s:N`, and therefore the only number the browser
69
+ * ever names. Text tracks alone are not a numbering: a file whose PGS track
70
+ * comes first would have every text track one lower here than in the browser.
66
71
  * @property {string} codecId
67
72
  * @property {string} language - The three-letter code the file declares.
68
73
  * @property {string} name - What the file calls the track, if anything.
@@ -178,6 +183,7 @@ export async function readSubtitlePlan(readRange, fileSize) {
178
183
  }
179
184
  tracks.push({
180
185
  trackNumber,
186
+ declaredIndex: declared.length - 1,
181
187
  codecId,
182
188
  language,
183
189
  name,
@@ -29,6 +29,15 @@ const MAX_MOOV_BYTES = 32 * 1024 * 1024;
29
29
 
30
30
  /** Handlers that mean "this track is text on screen". */
31
31
  const TEXT_HANDLERS = new Set(["text", "sbtl", "subt"]);
32
+ /**
33
+ * Every handler ffmpeg's mov demuxer turns into a SUBTITLE stream, whether or
34
+ * not this file can read it — `subp` is a DVD subpicture and `clcp` closed
35
+ * captions, both pictures or caption data rather than text. They are counted
36
+ * because `declaredIndex` has to equal ffmpeg's `0:s:N`, and a track left out
37
+ * of the count shifts every text track after it, which is the very defect
38
+ * `declaredIndex` exists to remove.
39
+ */
40
+ const SUBTITLE_HANDLERS = new Set([...TEXT_HANDLERS, "subp", "clcp"]);
32
41
  /** Sample formats this can turn into cues. `stpp` (TTML) is XML and is not one. */
33
42
  const TEXT_FORMATS = new Set(["tx3g", "text", "wvtt"]);
34
43
 
@@ -218,6 +227,11 @@ function sampleOffsets(moov, stsc, chunkOffsets, sizes) {
218
227
  /**
219
228
  * @typedef {object} Mp4SubtitleTrack
220
229
  * @property {number} trackId
230
+ * @property {number} declaredIndex - Its position among ALL of the file's
231
+ * subtitle tracks, including the ones whose sample format this cannot turn
232
+ * into cues (`stpp` TTML). That is the number ffmpeg gives the same stream in
233
+ * `0:s:N`, which is the number the browser names; counting only the readable
234
+ * ones would shift every track after a TTML one.
221
235
  * @property {string} format - `tx3g`, `text` or `wvtt`.
222
236
  * @property {string} language - Three letters, as the file declares them.
223
237
  * @property {Mp4SubtitleSample[]} samples - In time order.
@@ -246,6 +260,9 @@ export async function readMp4SubtitlePlan(readRange, fileSize) {
246
260
 
247
261
  /** @type {Mp4SubtitleTrack[]} */
248
262
  const tracks = [];
263
+ // Counts every subtitle track the file has, whether or not this can read it,
264
+ // so the number handed out matches ffmpeg's `0:s:N`. See `declaredIndex`.
265
+ let declaredIndex = -1;
249
266
  for (const trak of childrenOf(moov, moovBox.dataOffset, moov.length, "trak")) {
250
267
  const mdia = childOf(moov, trak.dataOffset, trak.end, "mdia");
251
268
  if (!mdia) {
@@ -253,6 +270,13 @@ export async function readMp4SubtitlePlan(readRange, fileSize) {
253
270
  }
254
271
  const hdlr = childOf(moov, mdia.dataOffset, mdia.end, "hdlr");
255
272
  const handler = hdlr ? moov.toString("latin1", hdlr.dataOffset + 8, hdlr.dataOffset + 12) : "";
273
+ if (!SUBTITLE_HANDLERS.has(handler)) {
274
+ continue;
275
+ }
276
+ // Counted before the readability checks below, and before the handler is
277
+ // narrowed to the text ones: this number is the track's place in the file,
278
+ // not its place among the tracks this code can turn into cues.
279
+ declaredIndex += 1;
256
280
  if (!TEXT_HANDLERS.has(handler)) {
257
281
  continue;
258
282
  }
@@ -337,7 +361,7 @@ export async function readMp4SubtitlePlan(readRange, fileSize) {
337
361
  });
338
362
  }
339
363
  }
340
- tracks.push({ trackId, format, language, samples });
364
+ tracks.push({ trackId, declaredIndex, format, language, samples });
341
365
  }
342
366
  return { tracks };
343
367
  }
@@ -21,7 +21,7 @@
21
21
  * { type: "response-start", requestId, status, headers } (JSON string)
22
22
  * { type: "response-error", requestId, error: string } (JSON string)
23
23
  * { type: "pong", id } (JSON string)
24
- * { type: "subtitle-cues", fileIndex, trackIndex, cues, language } (JSON string)
24
+ * { type: "subtitle-cues", fileIndex, trackIndex, cues, language, cursor } (JSON string)
25
25
  * ```
26
26
  * The last one is unsolicited — sent the moment new cues are read from a
27
27
  * file's already-downloaded pieces, to whichever channel last asked for that
@@ -385,10 +385,10 @@ export function createDataChannelHandler({ proxyPort, onLog, getTransportSnapsho
385
385
  * are tiny (kilobytes at most for a whole track), so this is one message,
386
386
  * not a stream.
387
387
  *
388
- * @param {{ sourceKey: string, fileIndex: number, trackIndex: number, cues: object[], language: string }} event
388
+ * @param {{ sourceKey: string, fileIndex: number, trackIndex: number, cues: object[], language: string, cursor: number }} event
389
389
  * @returns {void}
390
390
  */
391
- function publishSubtitleCues({ sourceKey, fileIndex, trackIndex, cues, language }) {
391
+ function publishSubtitleCues({ sourceKey, fileIndex, trackIndex, cues, language, cursor }) {
392
392
  const set = subtitleSubscribers.get(`${sourceKey}:${fileIndex}`);
393
393
  if (!set || set.size === 0) {
394
394
  log(
@@ -397,7 +397,7 @@ export function createDataChannelHandler({ proxyPort, onLog, getTransportSnapsho
397
397
  );
398
398
  return;
399
399
  }
400
- const message = { type: "subtitle-cues", fileIndex, trackIndex, cues, language };
400
+ const message = { type: "subtitle-cues", fileIndex, trackIndex, cues, language, cursor };
401
401
  const total = set.size;
402
402
  let sent = 0;
403
403
  for (const channel of set) {
@@ -194,7 +194,8 @@ export class TorrentWorkerClient {
194
194
  fileIndex: message.fileIndex,
195
195
  trackIndex: message.trackIndex,
196
196
  cues: message.cues,
197
- language: message.language
197
+ language: message.language,
198
+ cursor: message.cursor
198
199
  });
199
200
  break;
200
201
  default:
@@ -30,6 +30,8 @@ const CLUSTER_HEADER_PROBE = 64;
30
30
  * reading it would be a large read for nothing.
31
31
  */
32
32
  const MAX_CLUSTER_BYTES = 32 * 1024 * 1024;
33
+ /** How long a read of already-held bytes may take before it is given up. */
34
+ const READ_ABANDON_MS = 30_000;
33
35
 
34
36
  /** @type {Map<string, { plan: object | null, harvested: Map<number, Set<number>>, cues: Map<number, object[]> }>} */
35
37
  const byFile = new Map();
@@ -78,9 +80,38 @@ function readHeld(file, start, end) {
78
80
  resolve(null);
79
81
  return;
80
82
  }
83
+ let settled = false;
84
+ /** @type {ReturnType<typeof setTimeout> | null} */
85
+ let abandon = null;
86
+ const settle = (value) => {
87
+ if (settled) {
88
+ return;
89
+ }
90
+ settled = true;
91
+ if (abandon !== null) {
92
+ clearTimeout(abandon);
93
+ }
94
+ if (value === null) {
95
+ stream.destroy?.();
96
+ }
97
+ resolve(value);
98
+ };
99
+ // A read of bytes the torrent already holds either answers or it does not.
100
+ // This is not a measurement of anything and no figure is derived from it:
101
+ // it is the point past which such a read is presumed lost, so that one
102
+ // stream which never ends cannot hold this file's walk — and with it the
103
+ // browser's own request for its subtitles — for the rest of the session.
104
+ abandon = setTimeout(() => {
105
+ logger.info(
106
+ `subtitles: a read of ${start}-${end} in "${String(file.name).slice(0, 40)}" ` +
107
+ `did not finish in ${READ_ABANDON_MS / 1000}s and was given up`
108
+ );
109
+ settle(null);
110
+ }, READ_ABANDON_MS);
111
+ abandon.unref?.();
81
112
  stream.on("data", (chunk) => chunks.push(chunk));
82
- stream.on("end", () => resolve(Buffer.concat(chunks)));
83
- stream.on("error", () => resolve(null));
113
+ stream.on("end", () => settle(Buffer.concat(chunks)));
114
+ stream.on("error", () => settle(null));
84
115
  });
85
116
  }
86
117
 
@@ -97,10 +128,41 @@ function readHeld(file, start, end) {
97
128
  * @returns {Promise<object | null>}
98
129
  */
99
130
  async function planFor(torrent, fileIndex, key) {
131
+ const state = stateFor(key);
132
+ if (state.plan !== null) {
133
+ return state.plan;
134
+ }
135
+ // The head and the Cues table are two reads that DO wait on the swarm, so two
136
+ // callers arriving together would both make them. One promise, awaited by
137
+ // whoever asks while it is in flight.
138
+ if (!state.planPromise) {
139
+ state.planPromise = readPlan(torrent, fileIndex, state).finally(() => {
140
+ state.planPromise = null;
141
+ });
142
+ }
143
+ return state.planPromise;
144
+ }
145
+
146
+ /**
147
+ * The state kept for one file, created on first use.
148
+ *
149
+ * @param {string} key - `sourceKey:fileIndex`.
150
+ * @returns {object}
151
+ */
152
+ function stateFor(key) {
100
153
  let state = byFile.get(key);
154
+ // A state that has been forgotten is not handed out again, even in the moment
155
+ // between the call and the walk that was still running finishing.
156
+ if (state?.forgotten === true) {
157
+ state = undefined;
158
+ }
101
159
  if (!state) {
102
160
  state = {
103
161
  plan: null,
162
+ planPromise: null,
163
+ forgotten: false,
164
+ // One walk of a file at a time — see `serialize`.
165
+ chain: Promise.resolve(),
104
166
  harvested: new Map(),
105
167
  cues: new Map(),
106
168
  seq: new Map(),
@@ -112,9 +174,45 @@ async function planFor(torrent, fileIndex, key) {
112
174
  };
113
175
  byFile.set(key, state);
114
176
  }
115
- if (state.plan !== null) {
116
- return state.plan;
117
- }
177
+ return state;
178
+ }
179
+
180
+ /**
181
+ * Run `work` after every walk of this file already started, and before any
182
+ * started after it.
183
+ *
184
+ * Both entry points here — a browser's own pull and the warmup that runs ahead
185
+ * of it — mark a cluster as walked only AFTER reading and parsing it, which is
186
+ * two suspension points later. Until 2.56.0 nothing stopped a second call
187
+ * arriving in between: `warmActiveFiles` runs on every verified piece AND on a
188
+ * 3 s timer, so on a fast download the same cluster was read and parsed several
189
+ * times over and the same line could be pushed twice under different `seq`
190
+ * numbers. Each of those reads is a WebTorrent file stream, which selects and
191
+ * deselects its pieces, so the repetition reached the piece picker as well.
192
+ *
193
+ * @template T
194
+ * @param {object} state
195
+ * @param {() => Promise<T>} work
196
+ * @returns {Promise<T>}
197
+ */
198
+ function serialize(state, work) {
199
+ const run = state.chain.then(work, work);
200
+ // The queue must survive a failed walk, so what is chained is the settled
201
+ // form; the caller still sees the rejection.
202
+ state.chain = run.then(() => undefined, () => undefined);
203
+ return run;
204
+ }
205
+
206
+ /**
207
+ * Read one file's subtitle plan — the tracks it declares and where the clusters
208
+ * holding them are. Called once per file; see `planFor`.
209
+ *
210
+ * @param {object} torrent
211
+ * @param {number} fileIndex
212
+ * @param {object} state
213
+ * @returns {Promise<object>}
214
+ */
215
+ async function readPlan(torrent, fileIndex, state) {
118
216
  const file = torrent?.files?.[fileIndex];
119
217
  // `declared` is what the container itself says about its subtitle tracks, in
120
218
  // its own order. Empty means the container said nothing — which is a real
@@ -139,6 +237,7 @@ async function planFor(torrent, fileIndex, key) {
139
237
  ...empty,
140
238
  tracks: mp4.tracks.map((track, order) => ({
141
239
  trackNumber: track.trackId,
240
+ declaredIndex: Number.isInteger(track.declaredIndex) ? track.declaredIndex : order,
142
241
  codecId: track.format,
143
242
  language: track.language,
144
243
  name: "",
@@ -159,10 +258,13 @@ async function planFor(torrent, fileIndex, key) {
159
258
  state.plan = plan ?? empty;
160
259
  if (state.plan.tracks.length > 0) {
161
260
  logger.info(
162
- `subtitles: "${String(file.name).slice(0, 40)}" has ${state.plan.tracks.length} text track(s) ` +
261
+ `subtitles: "${String(file.name).slice(0, 40)}" has ${state.plan.tracks.length} text track(s) ` +
262
+ `of ${state.plan.declared.length} declared — ` +
163
263
  state.plan.tracks
164
- .map((track) => `${track.trackNumber}:${track.language || "?"}${track.name ? `/${track.name}` : ""}` +
165
- `(${track.clusterPositions.length} indexed)`)
264
+ // `s:N` is the number the browser names (ffmpeg's own), and it differs
265
+ // from the file's track number whenever a picture track sits among them.
266
+ .map((track) => `s:${track.declaredIndex}=${track.trackNumber}:${track.language || "?"}` +
267
+ `${track.name ? `/${track.name}` : ""}(${track.clusterPositions.length} indexed)`)
166
268
  .join(" ")
167
269
  );
168
270
  }
@@ -206,11 +308,27 @@ function nextSeq(state, trackNumber) {
206
308
  export async function cuesHeldFor(torrent, fileIndex, sourceKey, trackNumber) {
207
309
  const key = `${sourceKey}:${fileIndex}`;
208
310
  const plan = await planFor(torrent, fileIndex, key);
209
- const state = byFile.get(key);
311
+ const state = stateFor(key);
210
312
  const track = plan?.tracks?.find((candidate) => candidate.trackNumber === trackNumber) ?? null;
211
313
  if (!track) {
212
314
  return { cues: [], coveredClusters: 0, indexedClusters: 0, track: null };
213
315
  }
316
+ return serialize(state, () => walkFor(torrent, fileIndex, state, plan, track, trackNumber));
317
+ }
318
+
319
+ /**
320
+ * The walk itself. Only ever entered through `cuesHeldFor`, which is what keeps
321
+ * one file to one walk at a time.
322
+ *
323
+ * @param {object} torrent
324
+ * @param {number} fileIndex
325
+ * @param {object} state
326
+ * @param {object} plan
327
+ * @param {object} track
328
+ * @param {number} trackNumber
329
+ * @returns {Promise<{ cues: object[], coveredClusters: number, indexedClusters: number, track: object | null }>}
330
+ */
331
+ async function walkFor(torrent, fileIndex, state, plan, track, trackNumber) {
214
332
  const file = torrent.files[fileIndex];
215
333
  let harvested = state.harvested.get(trackNumber);
216
334
  if (!harvested) {
@@ -343,18 +461,20 @@ export async function cuesHeldFor(torrent, fileIndex, sourceKey, trackNumber) {
343
461
  * @param {string} sourceKey
344
462
  * @returns {Promise<{ trackIndex: number, cues: object[], language: string }[]>}
345
463
  * One entry per track that gained at least one cue since the last call.
346
- * `trackIndex` is the track's position among `plan.tracks` the same
347
- * indexing `/api/subtitles?trackIndex=` and the browser's menu use, NOT the
348
- * container's own track number, which the browser never sees.
464
+ * `trackIndex` is `declaredIndex` — the track's position among ALL the file's
465
+ * subtitle tracks, which is ffmpeg's `0:s:N` and the only number the browser
466
+ * knows. NOT the container's own track number, and not the position among the
467
+ * readable tracks either: counting those alone puts every text track after a
468
+ * picture-based one in the wrong place.
349
469
  */
350
470
  export async function warmSubtitleCues(torrent, fileIndex, sourceKey) {
351
471
  const key = `${sourceKey}:${fileIndex}`;
352
472
  const plan = await planFor(torrent, fileIndex, key);
353
- const state = byFile.get(key);
473
+ const state = stateFor(key);
354
474
  const fresh = [];
355
475
  const tracks = plan?.tracks ?? [];
356
- for (let trackIndex = 0; trackIndex < tracks.length; trackIndex += 1) {
357
- const track = tracks[trackIndex];
476
+ for (let order = 0; order < tracks.length; order += 1) {
477
+ const track = tracks[order];
358
478
  const held = await cuesHeldFor(torrent, fileIndex, sourceKey, track.trackNumber);
359
479
  const since = state.pushed.get(track.trackNumber) ?? 0;
360
480
  const newCues = held.cues.filter((cue) => (Number(cue.seq) || 0) > since);
@@ -363,10 +483,21 @@ export async function warmSubtitleCues(torrent, fileIndex, sourceKey) {
363
483
  }
364
484
  const highest = newCues.reduce((max, cue) => Math.max(max, Number(cue.seq) || 0), since);
365
485
  state.pushed.set(track.trackNumber, highest);
486
+ const cues = finalizeCues(newCues, held.track?.codecId ?? track.codecId);
366
487
  fresh.push({
367
- trackIndex,
368
- cues: finalizeCues(newCues, held.track?.codecId ?? track.codecId),
369
- language: held.track?.language ?? ""
488
+ // ffmpeg's own numbering, which is the only one the browser knows.
489
+ trackIndex: Number.isInteger(track.declaredIndex) ? track.declaredIndex : order,
490
+ cues,
491
+ language: held.track?.language ?? "",
492
+ // Where the browser should resume from if it has to ask again — after a
493
+ // reconnect, which loses the subscription these pushes ride on.
494
+ cursor: highest,
495
+ // What this batch is ABOUT, in film time, so a log can be read against
496
+ // the position being played.
497
+ spanStartSeconds: cues.length > 0 ? cues[0].startSeconds : null,
498
+ spanEndSeconds: cues.length > 0 ? cues[cues.length - 1].endSeconds : null,
499
+ walkedClusters: held.coveredClusters ?? 0,
500
+ indexedClusters: held.indexedClusters ?? 0
370
501
  });
371
502
  }
372
503
  return fresh;
@@ -430,8 +561,9 @@ function assDialogueToText(raw) {
430
561
  */
431
562
  export async function subtitleTracksOf(torrent, fileIndex, sourceKey) {
432
563
  const plan = await planFor(torrent, fileIndex, `${sourceKey}:${fileIndex}`);
433
- return (plan?.tracks ?? []).map((track) => ({
564
+ return (plan?.tracks ?? []).map((track, order) => ({
434
565
  trackNumber: track.trackNumber,
566
+ declaredIndex: Number.isInteger(track.declaredIndex) ? track.declaredIndex : order,
435
567
  codecId: track.codecId,
436
568
  language: track.language,
437
569
  name: track.name,
@@ -471,10 +603,39 @@ export function forgetSubtitles(sourceKey, fileIndex) {
471
603
  if (fileIndex === undefined) {
472
604
  for (const key of [...byFile.keys()]) {
473
605
  if (key.startsWith(`${sourceKey}:`)) {
474
- byFile.delete(key);
606
+ forgetOne(key);
475
607
  }
476
608
  }
477
609
  return;
478
610
  }
479
- byFile.delete(`${sourceKey}:${fileIndex}`);
611
+ forgetOne(`${sourceKey}:${fileIndex}`);
612
+ }
613
+
614
+ /**
615
+ * Drop one file's state, but not while a walk of it is still running: the
616
+ * record of which clusters have been read lives in that state, and a walk left
617
+ * writing into a discarded copy while a new one starts beside it is the one
618
+ * path that defeats the serialization above.
619
+ *
620
+ * @param {string} key
621
+ * @returns {void}
622
+ */
623
+ function forgetOne(key) {
624
+ const state = byFile.get(key);
625
+ if (!state) {
626
+ return;
627
+ }
628
+ // Held, so that a walk started before this call is not left orphaned; the
629
+ // entry is dropped the moment the queue empties, and nothing is handed this
630
+ // state in the meantime.
631
+ state.forgotten = true;
632
+ void state.chain.then(() => {
633
+ if (byFile.get(key) === state) {
634
+ byFile.delete(key);
635
+ }
636
+ }, () => {
637
+ if (byFile.get(key) === state) {
638
+ byFile.delete(key);
639
+ }
640
+ });
480
641
  }
@@ -522,12 +522,27 @@ function warmActiveFiles(sourceKey, torrent) {
522
522
  return;
523
523
  }
524
524
  for (const fileIndex of usage.keys()) {
525
+ const key = `${sourceKey}:${fileIndex}`;
526
+ // A trigger that arrives while the previous pass is still walking is
527
+ // dropped, not queued. `verified` fires per piece, so on a fast download
528
+ // these arrive many times a second; the walk is serialized per file anyway,
529
+ // and a queue of identical passes would only postpone the one that has
530
+ // something new to find.
531
+ if (warmupInFlight.has(key)) {
532
+ continue;
533
+ }
534
+ warmupInFlight.add(key);
525
535
  warmSubtitleCues(torrent, fileIndex, sourceKey)
526
536
  .then((fresh) => {
527
537
  for (const entry of fresh) {
538
+ const span = entry.spanStartSeconds === null
539
+ ? "empty"
540
+ : `${entry.spanStartSeconds.toFixed(1)}-${entry.spanEndSeconds.toFixed(1)}s`;
528
541
  log(
529
542
  `subtitle push ${sourceKey.slice(0, 8)}:${fileIndex} track ${entry.trackIndex}: ` +
530
- `${entry.cues.length} new cue(s) found, posting to main thread`
543
+ `${entry.cues.length} new cue(s) covering ${span}, ` +
544
+ `clusters walked ${entry.walkedClusters}/${entry.indexedClusters}, cursor ${entry.cursor}, ` +
545
+ "posting to main thread"
531
546
  );
532
547
  parentPort.postMessage({
533
548
  type: Event.SUBTITLE_CUES_READY,
@@ -535,16 +550,27 @@ function warmActiveFiles(sourceKey, torrent) {
535
550
  fileIndex,
536
551
  trackIndex: entry.trackIndex,
537
552
  cues: entry.cues,
538
- language: entry.language
553
+ language: entry.language,
554
+ cursor: entry.cursor
539
555
  });
540
556
  }
541
557
  })
542
558
  .catch((error) => {
543
559
  log(`subtitle warmup ${sourceKey}:${fileIndex} failed: ${error instanceof Error ? error.message : error}`);
560
+ })
561
+ .finally(() => {
562
+ warmupInFlight.delete(key);
544
563
  });
545
564
  }
546
565
  }
547
566
 
567
+ /**
568
+ * Files whose warmup pass has not finished yet, by `sourceKey:fileIndex`.
569
+ *
570
+ * @type {Set<string>}
571
+ */
572
+ const warmupInFlight = new Set();
573
+
548
574
  /**
549
575
  * Torrents already wired to warm their subtitle cues the moment a piece
550
576
  * verifies, so the same torrent is not listened to twice.
@@ -0,0 +1,262 @@
1
+ /**
2
+ * @file The number a subtitle track is asked for by, and one walk per file.
3
+ *
4
+ * Two rules, both found by reading on 2026-08-26 after the report that embedded
5
+ * subtitles appear late (`research/subtitle-delay-2026-08-26.md`):
6
+ *
7
+ * 1. The browser names a track by ffmpeg's `0:s:N`, which counts EVERY subtitle
8
+ * stream. The container plan drops the picture-based ones — PGS, VobSub —
9
+ * because they cannot become WebVTT, so counting the kept ones is a
10
+ * different numbering as soon as a file carries one of each. What that cost:
11
+ * the push landed on a track the browser does not know, and the browser's
12
+ * own request found no track at all and fell through to the ffmpeg
13
+ * extraction, which reads the whole film (752 s measured, 2026-08-19).
14
+ * 2. A file is walked once at a time. The walk marks a cluster as read only
15
+ * after two suspension points, and it is started both on every verified
16
+ * piece and on a 3 s timer, so two passes could read and parse the same
17
+ * cluster and push the same line twice.
18
+ */
19
+
20
+ import test from "node:test";
21
+ import assert from "node:assert/strict";
22
+ import { Readable } from "node:stream";
23
+ import { readSubtitlePlan } from "../services/container-index/matroska-subtitles.js";
24
+ import { cuesHeldFor, forgetSubtitles } from "../services/torrent-worker/subtitle-cues.js";
25
+
26
+ const ID_EBML = 0x1a45dfa3;
27
+ const ID_SEGMENT = 0x18538067;
28
+ const ID_SEEK_HEAD = 0x114d9b74;
29
+ const ID_SEEK = 0x4dbb;
30
+ const ID_SEEK_ID = 0x53ab;
31
+ const ID_SEEK_POSITION = 0x53ac;
32
+ const ID_INFO = 0x1549a966;
33
+ const ID_TIMESTAMP_SCALE = 0x2ad7b1;
34
+ const ID_TRACKS = 0x1654ae6b;
35
+ const ID_TRACK_ENTRY = 0xae;
36
+ const ID_TRACK_NUMBER = 0xd7;
37
+ const ID_TRACK_TYPE = 0x83;
38
+ const ID_CODEC_ID = 0x86;
39
+ const ID_LANGUAGE = 0x22b59c;
40
+ const ID_CUES = 0x1c53bb6b;
41
+ const ID_CUE_POINT = 0xbb;
42
+ const ID_CUE_TIME = 0xb3;
43
+ const ID_CUE_TRACK_POSITIONS = 0xb7;
44
+ const ID_CUE_TRACK = 0xf7;
45
+ const ID_CUE_CLUSTER_POSITION = 0xf1;
46
+ const ID_CLUSTER = 0x1f43b675;
47
+ const ID_TIMESTAMP = 0xe7;
48
+
49
+ /** An element id, as the bytes the specification gives it. */
50
+ function idBytes(id) {
51
+ const bytes = [];
52
+ let rest = id;
53
+ while (rest > 0) {
54
+ bytes.unshift(rest & 0xff);
55
+ rest = Math.floor(rest / 256);
56
+ }
57
+ return Buffer.from(bytes);
58
+ }
59
+
60
+ /** A size, as a four-byte EBML variable-length integer. */
61
+ function sizeBytes(size) {
62
+ const buffer = Buffer.alloc(4);
63
+ buffer.writeUInt32BE(size, 0);
64
+ buffer[0] |= 0x10;
65
+ return buffer;
66
+ }
67
+
68
+ function element(id, payload) {
69
+ return Buffer.concat([idBytes(id), sizeBytes(payload.length), payload]);
70
+ }
71
+
72
+ function uintElement(id, value) {
73
+ const bytes = [];
74
+ let rest = value;
75
+ do {
76
+ bytes.unshift(rest & 0xff);
77
+ rest = Math.floor(rest / 256);
78
+ } while (rest > 0);
79
+ return element(id, Buffer.from(bytes));
80
+ }
81
+
82
+ function stringElement(id, value) {
83
+ return element(id, Buffer.from(value, "utf8"));
84
+ }
85
+
86
+ /**
87
+ * An unsigned value at a FIXED four bytes. A cue's cluster position has to be
88
+ * written twice — once to measure the table, once with the position that
89
+ * measurement produced — and a value-sized element would make the second table
90
+ * a different length from the first, moving the very cluster it names.
91
+ */
92
+ function uint32Element(id, value) {
93
+ const payload = Buffer.alloc(4);
94
+ payload.writeUInt32BE(value, 0);
95
+ return element(id, payload);
96
+ }
97
+
98
+ function trackEntry({ number, type, codecId, language }) {
99
+ return element(ID_TRACK_ENTRY, Buffer.concat([
100
+ uintElement(ID_TRACK_NUMBER, number),
101
+ uintElement(ID_TRACK_TYPE, type),
102
+ stringElement(ID_CODEC_ID, codecId),
103
+ stringElement(ID_LANGUAGE, language)
104
+ ]));
105
+ }
106
+
107
+ /**
108
+ * A file whose subtitle tracks are, in the container's own order: a picture
109
+ * one, then two text ones. ffmpeg numbers those `0:s:0`, `0:s:1`, `0:s:2`;
110
+ * the plan can only read the last two.
111
+ *
112
+ * The Cues table points both text tracks at one cluster, which is written after
113
+ * the table so its position can be stated.
114
+ *
115
+ * @returns {{ file: Buffer, clusterAt: number }}
116
+ */
117
+ function buildFile() {
118
+ const info = element(ID_INFO, uintElement(ID_TIMESTAMP_SCALE, 1_000_000));
119
+ const tracks = element(ID_TRACKS, Buffer.concat([
120
+ trackEntry({ number: 1, type: 1, codecId: "V_MPEG4/ISO/AVC", language: "und" }),
121
+ trackEntry({ number: 2, type: 17, codecId: "S_HDMV/PGS", language: "eng" }),
122
+ trackEntry({ number: 3, type: 17, codecId: "S_TEXT/UTF8", language: "rus" }),
123
+ trackEntry({ number: 4, type: 17, codecId: "S_TEXT/ASS", language: "eng" })
124
+ ]));
125
+
126
+ // Built twice: the cue points state where the cluster is, and that position
127
+ // is only known once everything before it has its final length. Every size
128
+ // and position here is written at a fixed width, so the draft and the final
129
+ // table are the same length.
130
+ const cuesWith = (clusterAt) => element(ID_CUES, Buffer.concat([
131
+ element(ID_CUE_POINT, Buffer.concat([
132
+ uintElement(ID_CUE_TIME, 1000),
133
+ element(ID_CUE_TRACK_POSITIONS, Buffer.concat([
134
+ uintElement(ID_CUE_TRACK, 3),
135
+ uint32Element(ID_CUE_CLUSTER_POSITION, clusterAt)
136
+ ])),
137
+ element(ID_CUE_TRACK_POSITIONS, Buffer.concat([
138
+ uintElement(ID_CUE_TRACK, 4),
139
+ uint32Element(ID_CUE_CLUSTER_POSITION, clusterAt)
140
+ ]))
141
+ ]))
142
+ ]));
143
+
144
+ const seekEntry = (targetId, position) => element(ID_SEEK, Buffer.concat([
145
+ element(ID_SEEK_ID, idBytes(targetId)),
146
+ element(ID_SEEK_POSITION, (() => {
147
+ const buffer = Buffer.alloc(4);
148
+ buffer.writeUInt32BE(position, 0);
149
+ return buffer;
150
+ })())
151
+ ]));
152
+ const seekHeadWith = (infoAt, tracksAt, cuesAt) => element(ID_SEEK_HEAD, Buffer.concat([
153
+ seekEntry(ID_INFO, infoAt),
154
+ seekEntry(ID_TRACKS, tracksAt),
155
+ seekEntry(ID_CUES, cuesAt)
156
+ ]));
157
+
158
+ const headLength = seekHeadWith(0, 0, 0).length;
159
+ const infoAt = headLength;
160
+ const tracksAt = infoAt + info.length;
161
+ const cuesAt = tracksAt + tracks.length;
162
+ // A position in the Cues table is measured from the Segment's payload, and so
163
+ // is the one the reader turns it into.
164
+ const clusterRelative = cuesAt + cuesWith(0).length;
165
+
166
+ // Enough of a cluster to be read and recognised: its own header and a
167
+ // timestamp. No blocks, so it yields no cues — what the walk test counts is
168
+ // that its bytes are fetched once, and that does not depend on their content.
169
+ const cluster = element(ID_CLUSTER, uintElement(ID_TIMESTAMP, 1000));
170
+
171
+ const segmentPayload = Buffer.concat([
172
+ seekHeadWith(infoAt, tracksAt, cuesAt),
173
+ info,
174
+ tracks,
175
+ cuesWith(clusterRelative),
176
+ cluster
177
+ ]);
178
+ const ebml = element(ID_EBML, Buffer.from([0x42, 0x86, 0x81, 0x01]));
179
+ const segment = element(ID_SEGMENT, segmentPayload);
180
+ const segmentDataOffset = ebml.length + segment.length - segmentPayload.length;
181
+ return {
182
+ file: Buffer.concat([ebml, segment]),
183
+ clusterAt: segmentDataOffset + clusterRelative
184
+ };
185
+ }
186
+
187
+ function readerOver(file) {
188
+ return async (start, end) => {
189
+ const last = Math.min(end, file.length - 1);
190
+ return start > last ? null : file.subarray(start, last + 1);
191
+ };
192
+ }
193
+
194
+ test("a text track is numbered as ffmpeg numbers it, past the picture ones", async () => {
195
+ const { file } = buildFile();
196
+
197
+ const plan = await readSubtitlePlan(readerOver(file), file.length);
198
+
199
+ assert.equal(plan.declared.length, 3, "all three subtitle tracks are declared");
200
+ assert.deepEqual(plan.tracks.map((track) => track.trackNumber), [3, 4], "only the text ones are readable");
201
+ assert.deepEqual(
202
+ plan.tracks.map((track) => track.declaredIndex),
203
+ [1, 2],
204
+ "the PGS track is 0:s:0, so the text tracks are 0:s:1 and 0:s:2 — not 0 and 1"
205
+ );
206
+ });
207
+
208
+ /**
209
+ * A torrent holding one file entirely, counting the byte ranges read from it.
210
+ *
211
+ * @param {Buffer} bytes
212
+ * @returns {{ torrent: object, reads: Array<{ start: number, end: number }> }}
213
+ */
214
+ function torrentOver(bytes) {
215
+ const reads = [];
216
+ const file = {
217
+ name: "film.mkv",
218
+ length: bytes.length,
219
+ offset: 0,
220
+ createReadStream({ start = 0, end = bytes.length - 1 } = {}) {
221
+ reads.push({ start, end });
222
+ // Asynchronous on purpose: a read that resolves in the same tick would
223
+ // hide exactly the interleaving this test is about.
224
+ return Readable.from((async function* chunks() {
225
+ await new Promise((resolve) => setImmediate(resolve));
226
+ yield bytes.subarray(start, end + 1);
227
+ })());
228
+ }
229
+ };
230
+ return {
231
+ reads,
232
+ torrent: {
233
+ pieceLength: 1024,
234
+ bitfield: { get: () => true },
235
+ files: [file]
236
+ }
237
+ };
238
+ }
239
+
240
+ test("two walks of one file at the same time read each cluster once", async () => {
241
+ const { file, clusterAt } = buildFile();
242
+ const { torrent, reads } = torrentOver(file);
243
+ const sourceKey = "torrent:numbering-test";
244
+ forgetSubtitles(sourceKey);
245
+
246
+ // Both text tracks at once, which is what the warmup does on every verified
247
+ // piece and every three seconds.
248
+ const [first, second] = await Promise.all([
249
+ cuesHeldFor(torrent, 0, sourceKey, 3),
250
+ cuesHeldFor(torrent, 0, sourceKey, 4)
251
+ ]);
252
+
253
+ assert.equal(first.coveredClusters, 1, "the cluster the table names was walked");
254
+ assert.equal(second.coveredClusters, 1, "and the second track sees the same walk, not its own");
255
+ const clusterReads = reads.filter((range) => range.start === clusterAt);
256
+ assert.equal(
257
+ clusterReads.length,
258
+ 2,
259
+ "one probe of the cluster's header and one read of its body — not two of each"
260
+ );
261
+ forgetSubtitles(sourceKey);
262
+ });