@torrent-tv/proxy 2.43.2 → 2.45.0

package/CHANGELOG.md

@@ -1,3 +1,14 @@
+## 2.45.0
+
+- **Fix**: A run is cut where the PLAYER was told the cuts are. There are two boundary tables — the live one, corrected as produced segments reveal where the file's cuts truly are, and the one the playlist text was written from — and a player places every fragment by the text it holds, which never changes. The cut list handed to ffmpeg came from the live table, so every correction moved the run away from the timeline the player is reading: measured 2026-08-20 on `Minions.and.Monsters.1080p.mkv`, 119 of 125 produced segments arrived a uniform 2.002 s before the times their playlist named, against the 0.5 s hls.js bridges, and a fragment that does not land is fetched again — on 2026-08-17 two of them 1908 times each. A seek is resolved on the same table for the same reason: the time being resolved came out of that playlist. The corrections keep their purpose, which is to describe the FILE — a variant created later inherits the corrected table and publishes it, so its own playlist and its own cuts agree from the start — but they may no longer move the cuts of a session already being read.
+- **New**: `FlagDefault` is read from the container itself rather than from ffmpeg's description of it. Matroska's flag DEFAULTS TO 1 and ffmpeg has applied that default by the time it prints `(default)`, so a file whose muxer marked no subtitle track is indistinguishable in the banner from one that marked every track — and the difference is the whole question, since one means "show this" and the other means "the file has no opinion". The EBML reader already walks the Tracks element for subtitle extraction and now also records whether the element was WRITTEN. Lining the two readings up is the awkward part and it is checked rather than assumed: ffmpeg numbers `0:s:N` over every subtitle stream in container order, so position is the correspondence, but each pair must agree on language or on title — one that agrees on neither, or a differing count, and the container reading is not used at all, leaving the probe's own flags with nothing claimed for them. `mergeContainerSubtitleFlags` is pure and tested.
+- **Chore**: The line reporting a segment that began away from its grid said "the playlist says", while the figure it prints comes from the live table. Reading that log on 2026-08-20 cost a wrong diagnosis; it now says "the grid says".
+
+## 2.44.0
+
+- **Fix**: The cues a browser is missing are found by the order they were READ, not by where they sit in the film. Cues come out of whichever clusters are downloaded, and a torrent does not arrive in film order, so the set of known cues grows in the MIDDLE as well as at the end — and the cursor shipped in 2.43.1 was a time. Measured 2026-08-20: a viewer at 272 s was answered with cues out to 1176 s, and from that moment every cue between the two was filtered away for the rest of the session, with 59 of 276 clusters read. The subtitles the viewer was about to need had become unreachable, while cues fifteen minutes ahead kept arriving. Each cue now carries the order it was found in, `?since=<n>` selects by that, and the answer states the next cursor in `X-Subtitle-Cursor`. `?after=<seconds>` still works, for a browser that has not been reloaded.
+- **Fix**: One walk over the container fills EVERY subtitle track, instead of one walk per track. A Matroska cluster carries the blocks of every track that has anything to say over its span, so the bytes that answer one track answer them all — but the set of clusters already read was kept per track, so the same bytes were fetched and parsed as many times as the film has subtitle tracks. On the field film that was five reads of everything, each costing 0.2-5.2 s, for cues that together weigh a few kilobytes. The union of the tracks' cluster lists is walked once and every track is filled from it, which is also why offering all of them costs no more than offering one.
+
 ## 2.43.2
 
 - **Fix**: The proxy no longer dies without a word in the middle of a film. It was a segmentation fault in the uTP native library — `on_uv_read` parsed a sender address that a FAILED read never produced, and libuv passes null there — so a read error dereferenced a null pointer on the thread that owns the torrent client. Three core dumps in two days, each about three and a half hours into an otherwise healthy run, each with the same top frame; the last one on a swarm of 63-75 peers delivering 13 MB/s, one segment after a successful 158 Mbit/s send. Fixed in our build (`@torrent-tv/utp-native@2.5.3-ttv.3`, which this now depends on) and absent from upstream master. Detail: `research/utp-native-null-addr-2026-08-20.md`.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@torrent-tv/proxy",
-  "version": "2.43.2",
+  "version": "2.45.0",
   "description": "Torrent proxy client that exposes webseed-like HTTP stream endpoint.",
   "license": "GPL-3.0-or-later",
   "publishConfig": {

package/routes/api/subtitles/get.js

@@ -100,18 +100,30 @@ export async function handleApiSubtitlesGet(req, reply, { sourceRegistry, torren
   // read that way. Costs no network at all and answers with the part of the
   // film they are watching; the rest arrives as they watch it. Only when the
   // container cannot be read this way does the old extraction run.
-  // Where the browser's copy of this track ends. It already holds every cue
-  // before this second, so sending them again is bytes for nothing: measured
+  // How many cues this browser has already been sent, counted in the order
+  // they were FOUND. Sending them again is bytes for nothing: measured
   // 2026-08-19, one track is 76 KB and the browser asked for it every few
   // seconds while the film downloaded. Absent or unparsable means "send
   // everything", which is what a browser asking for the first time wants.
+  //
+  // Found-order, not time. A cue's time cannot serve as a cursor here: the
+  // cues are read out of whichever clusters are downloaded, and those are not
+  // contiguous, so the set grows in the middle as well as at the end. The old
+  // `?after=<seconds>` therefore threw away every cue that turned up BEHIND
+  // the furthest one already sent — which is the stretch the viewer is about
+  // to watch. Measured 2026-08-20: a viewer at 272 s was sent cues out to
+  // 1176 s, and from that moment nothing between the two could ever reach
+  // them, with 59 of 276 clusters read. `after` is still honoured so an older
+  // browser keeps working.
+  const since = Number.parseInt(String(req.query?.since ?? ""), 10);
   const after = Number.parseFloat(String(req.query?.after ?? ""));
   const held = await cuesFromDownloadedClusters(
     torrentPool,
     torrent,
     fileIndex,
     trackIndex,
-    Number.isFinite(after) ? after : null
+    Number.isFinite(after) ? after : null,
+    Number.isInteger(since) ? since : null
   );
   if (held !== null) {
     setLanguageHeaders(reply, held.language);
@@ -122,9 +134,11 @@ export async function handleApiSubtitlesGet(req, reply, { sourceRegistry, torren
     // as playback moves into clusters that were not downloaded yet.
     reply.header("X-Subtitle-Covered-Clusters", String(held.coveredClusters));
     reply.header("X-Subtitle-Indexed-Clusters", String(held.indexedClusters));
+    // What to send back as `?since=` next time.
+    reply.header("X-Subtitle-Cursor", String(held.cursor));
     reply.raw.setHeader(
       "Access-Control-Expose-Headers",
-      "X-Subtitle-Language, X-Subtitle-Language-Name, X-Subtitle-Covered-Clusters, X-Subtitle-Indexed-Clusters"
+      "X-Subtitle-Language, X-Subtitle-Language-Name, X-Subtitle-Covered-Clusters, X-Subtitle-Indexed-Clusters, X-Subtitle-Cursor"
     );
     return reply.send(held.vtt);
   }
@@ -264,7 +278,7 @@ function readFileFully(file, maxBytes) {
  * @returns {Promise<{ vtt: string, language: object | null, coveredClusters: number, indexedClusters: number } | null>}
  *   Null when this file cannot be read this way, and then the caller falls back.
  */
-async function cuesFromDownloadedClusters(torrentPool, torrent, fileIndex, trackIndex, after = null) {
+async function cuesFromDownloadedClusters(torrentPool, torrent, fileIndex, trackIndex, after = null, since = null) {
   if (typeof torrentPool?.getSubtitleTracks !== "function") {
     return null;
   }
@@ -290,9 +304,12 @@ async function cuesFromDownloadedClusters(torrentPool, torrent, fileIndex, track
   // Only what the browser does not have. The language is still detected from
   // EVERYTHING held, because three new lines say much less about a language
   // than the whole track does.
-  const fresh = Number.isFinite(after)
-    ? held.cues.filter((cue) => cue.startSeconds > after)
-    : held.cues;
+  const cursor = held.cues.reduce((highest, cue) => Math.max(highest, Number(cue.seq) || 0), 0);
+  const fresh = Number.isInteger(since)
+    ? held.cues.filter((cue) => (Number(cue.seq) || 0) > since)
+    : Number.isFinite(after)
+      ? held.cues.filter((cue) => cue.startSeconds > after)
+      : held.cues;
   const vtt = cuesToVtt(fresh, held.codecId);
   const language = held.cues.length > 0
     ? detectLanguage(held.cues.map((cue) => cue.text).join("\n"))
@@ -300,6 +317,7 @@ async function cuesFromDownloadedClusters(torrentPool, torrent, fileIndex, track
   return {
     vtt,
     language,
+    cursor,
     coveredClusters: held.coveredClusters ?? 0,
     indexedClusters: held.indexedClusters ?? 0
   };

package/services/container-index/matroska-subtitles.js

@@ -92,7 +92,7 @@ function readString(buffer, element) {
  *
  * @param {(start: number, end: number) => Promise<Buffer | null>} readRange
  * @param {number} fileSize
- * @returns {Promise<{ tracks: SubtitleTrackPlan[], secondsPerTick: number, segmentDataOffset: number } | null>}
+ * @returns {Promise<{ tracks: SubtitleTrackPlan[], declared: object[], secondsPerTick: number, segmentDataOffset: number } | null>}
  */
 export async function readSubtitlePlan(readRange, fileSize) {
   const head = await readRange(0, Math.min(HEAD_BYTES, Math.max(0, fileSize - 1)));
@@ -124,6 +124,15 @@ export async function readSubtitlePlan(readRange, fileSize) {
   const tracksEnd = Math.min(head.length, tracksElement.dataOffset + tracksElement.size);
   /** @type {SubtitleTrackPlan[]} */
   const tracks = [];
+  /**
+   * Every subtitle track the file declares, in the order the Tracks element
+   * names them, text or picture. This is not for extraction — `tracks` is —
+   * but for lining ffmpeg's `0:s:N` numbering up against the container, which
+   * only holds while nothing is missing from the middle of the list.
+   *
+   * @type {Array<{ trackNumber: number, codecId: string, language: string, name: string, isDefault: boolean, declaresDefault: boolean }>}
+   */
+  const declared = [];
   for (const entry of iterateElements(head, tracksElement.dataOffset, tracksEnd)) {
     if (entry.id !== ID_TRACK_ENTRY) {
       continue;
@@ -135,7 +144,13 @@ export async function readSubtitlePlan(readRange, fileSize) {
     let language = "";
     let name = "";
     let codecPrivate = "";
+    // Matroska's `FlagDefault` DEFAULTS TO 1, so a file whose muxer wrote it on
+    // no track is indistinguishable, once the default has been applied, from
+    // one that wrote it on every track — which is how ffmpeg's banner prints it
+    // and why the banner cannot answer this. Both are kept: what the flag
+    // amounts to, and whether the file said anything at all.
     let isDefault = true;
+    let declaresDefault = false;
     for (const field of iterateElements(head, entry.dataOffset, entryEnd)) {
       if (field.id === ID_TRACK_NUMBER) {
         trackNumber = readUint(head, field.dataOffset, field.size);
@@ -149,11 +164,16 @@ export async function readSubtitlePlan(readRange, fileSize) {
         name = readString(head, field);
       } else if (field.id === ID_FLAG_DEFAULT) {
         isDefault = readUint(head, field.dataOffset, field.size) === 1;
+        declaresDefault = true;
       } else if (field.id === ID_CODEC_PRIVATE) {
         codecPrivate = head.toString("base64", field.dataOffset, field.dataOffset + field.size);
       }
     }
-    if (type !== TRACK_TYPE_SUBTITLE || trackNumber === null || !TEXT_CODECS.has(codecId)) {
+    if (type !== TRACK_TYPE_SUBTITLE || trackNumber === null) {
+      continue;
+    }
+    declared.push({ trackNumber, codecId, language, name, isDefault, declaresDefault });
+    if (!TEXT_CODECS.has(codecId)) {
       continue;
     }
     tracks.push({
@@ -167,7 +187,7 @@ export async function readSubtitlePlan(readRange, fileSize) {
     });
   }
   if (tracks.length === 0) {
-    return { tracks, secondsPerTick: scale / 1e9, segmentDataOffset: base };
+    return { tracks, declared, secondsPerTick: scale / 1e9, segmentDataOffset: base };
   }
 
   // Where the clusters holding those tracks are. A file that indexes only its
@@ -232,7 +252,7 @@ export async function readSubtitlePlan(readRange, fileSize) {
       }
     }
   }
-  return { tracks, secondsPerTick: scale / 1e9, segmentDataOffset: base };
+  return { tracks, declared, secondsPerTick: scale / 1e9, segmentDataOffset: base };
 }
 
 /**

package/services/hls-session-manager.js

@@ -2434,6 +2434,21 @@ export class HlsSessionManager {
    * @param {number} index
    * @returns {number}
    */
+  /**
+   * The boundary table the player is working from: the one its playlist was
+   * written from, falling back to the live table when no playlist was built
+   * from a table at all (no duration, so no synthetic playlist — and then
+   * nothing the player holds contradicts it).
+   *
+   * @param {HlsSession} session
+   * @returns {number[]}
+   */
+  publishedGridFor(session) {
+    return Array.isArray(session.publishedBoundaries) && session.publishedBoundaries.length > 0
+      ? session.publishedBoundaries
+      : (session.segmentBoundaries ?? []);
+  }
+
   #publishedStartTime(session, index) {
     const boundaries = Array.isArray(session.publishedBoundaries) && session.publishedBoundaries.length > 0
       ? session.publishedBoundaries
@@ -2486,7 +2501,10 @@ export class HlsSessionManager {
    * @returns {number}
    */
   #segmentIndexForTime(session, t) {
-    const boundaries = Array.isArray(session.segmentBoundaries) ? session.segmentBoundaries : [];
+    // The player's grid, for the same reason the cut list uses it: the time
+    // being resolved came from the playlist the player holds, so the index it
+    // means is the index that playlist gives it.
+    const boundaries = this.publishedGridFor(session);
     if (boundaries.length < 2) {
       return Math.max(0, Math.floor(t / this.segmentDurationSec));
     }
@@ -3797,8 +3815,22 @@ export class HlsSessionManager {
     // writes no self-contained pieces, so nothing could read a true start and
     // segments were stamped with times the file does not have — the 4.17 s
     // speech-against-subtitles drift, back again.
+    //
+    // Cut on the grid the PLAYER WAS GIVEN, not on the corrected one. A player
+    // places a fragment by the playlist it holds, and that text was written
+    // once and never changes; the live table keeps moving as produced segments
+    // reveal where the file's cuts really are. Cutting on the moved table makes
+    // every run faithful to a timeline nobody sent the player — measured
+    // 2026-08-20, the picture's segments arriving a uniform 2.002 s before the
+    // times the playlist named for them, which is four times what hls.js will
+    // bridge, so the fragment does not land and is asked for again.
+    //
+    // The corrections keep their purpose: they describe the file, and a variant
+    // created later inherits the corrected table and PUBLISHES it, so its own
+    // playlist and its own cuts agree from the start. What they may not do is
+    // move the cuts of a session whose playlist is already being read.
     const gridCutTimes = explicitTimes && (!session.transcodeVideo || session.cutGrid === "keyframe")
-      ? segmentCutTimesFrom(session.segmentBoundaries, safeIndex)
+      ? segmentCutTimesFrom(this.publishedGridFor(session), safeIndex)
       : null;
     // Cut times are stated on the grid, for both branches.
     //
@@ -5117,7 +5149,7 @@ export class HlsSessionManager {
         logger.warn(
         `transcode ${session.id} segment #${index} really starts at ` +
         `${trueStart.toFixed(3)}s (boundary ${at === null ? "none" : `#${at}`}), ` +
-        `the playlist says ${declaredStart.toFixed(3)}s — ` +
+        `the grid says ${declaredStart.toFixed(3)}s — ` +
         (session.audioOnly === true
           // A soundtrack is cut exactly where it was asked to be, so a
           // disagreement here is not a reading about the file at all: it is the