@torrent-tv/proxy 2.43.1 → 2.44.0

package/CHANGELOG.md

@@ -1,3 +1,16 @@
+## 2.44.0
+
+- **Fix**: The cues a browser is missing are found by the order they were READ, not by where they sit in the film. Cues come out of whichever clusters are downloaded, and a torrent does not arrive in film order, so the set of known cues grows in the MIDDLE as well as at the end — and the cursor shipped in 2.43.1 was a time. Measured 2026-08-20: a viewer at 272 s was answered with cues out to 1176 s, and from that moment every cue between the two was filtered away for the rest of the session, with 59 of 276 clusters read. The subtitles the viewer was about to need had become unreachable, while cues fifteen minutes ahead kept arriving. Each cue now carries the order it was found in, `?since=<n>` selects by that, and the answer states the next cursor in `X-Subtitle-Cursor`. `?after=<seconds>` still works, for a browser that has not been reloaded.
+- **Fix**: One walk over the container fills EVERY subtitle track, instead of one walk per track. A Matroska cluster carries the blocks of every track that has anything to say over its span, so the bytes that answer one track answer them all — but the set of clusters already read was kept per track, so the same bytes were fetched and parsed as many times as the film has subtitle tracks. On the field film that was five reads of everything, each costing 0.2-5.2 s, for cues that together weigh a few kilobytes. The union of the tracks' cluster lists is walked once and every track is filled from it, which is also why offering all of them costs no more than offering one.
+
+## 2.43.2
+
+- **Fix**: The proxy no longer dies without a word in the middle of a film. It was a segmentation fault in the uTP native library — `on_uv_read` parsed a sender address that a FAILED read never produced, and libuv passes null there — so a read error dereferenced a null pointer on the thread that owns the torrent client. Three core dumps in two days, each about three and a half hours into an otherwise healthy run, each with the same top frame; the last one on a swarm of 63-75 peers delivering 13 MB/s, one segment after a successful 158 Mbit/s send. Fixed in our build (`@torrent-tv/utp-native@2.5.3-ttv.3`, which this now depends on) and absent from upstream master. Detail: `research/utp-native-null-addr-2026-08-20.md`.
+- **Fix**: A soundtrack no longer moves the grid the picture is cut on. The boundary table is the picture's cut list — it is built from the container's keyframe index, and a copied stream can be cut nowhere else — but a produced sound segment was writing its own start into it too, and the two readings are of different things: a soundtrack has no keyframes and is cut exactly where `-segment_times` asks, to within one audio frame, while the picture's cuts are the file's real keyframes. Measured 2026-08-20 on `Minions.and.Monsters.1080p.mkv`, boundary #521 was corrected 2086.084s → 2084.082s by the picture and 2084.082s → 2086.033s by the sound 1.6 s later — **1.951 s apart**, against the 0.25 s that stops a correction and the 0.5 s a player bridges. Each reading contradicted the table the other had just written, so it never converged and the correction repeated for as long as the film ran. Only a session carrying picture may correct the grid now.
+- **Chore**: The line that reports a produced segment starting away from the playlist says what it measures. On the picture that is the container's keyframe index being wrong, which is what it always claimed; on a soundtrack there is no keyframe involved at all, and what it measures is how far the grid has moved since that run was launched. The per-boundary warning and the periodic summary both name the two apart now (`keyframe-index` against `sound-vs-grid`), and the summary no longer ends a soundtrack's figures with a count of keyframes read.
+- **Chore**: The per-boundary warning is limited to once per segment per five seconds, like the playlist-disagreement line beside it. A run keeps cutting on the `-segment_times` list it was launched with, so once the picture has moved the grid under a soundtrack every one of that run's segments deviates — and the same segment is produced and served again and again while a player refuses it. A line each time buries the first one, which is the one somebody is reading the log for. The soundtrack summary also keeps the tolerance its count was made against; only the count of keyframes read is dropped, since a soundtrack has none.
+- **Note, so the next field session is not read as a regression**: this closes the oscillation, not the gap. A run already producing keeps cutting on the `-segment_times` list it was launched with, so after a correction its segments still begin at pre-correction times until it is restarted — only a member whose run BEGINS at the corrected boundary is moved. Making the published grid agree with where runs really begin is the separate piece of work the code has been carrying a note about since 2026-08-17.
+
 ## 2.43.1
 
 - **New**: A subtitle request can say where the browser's copy ends (`?after=<seconds>`) and gets back only the cues past it. A track read out of downloaded clusters grows as the film does, and the browser was being sent all of it every few seconds — 76 KB a time on the field file — for the few lines at its end. The language is still detected from every cue held rather than from the handful being sent, because three lines say much less about a language than a whole track does.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@torrent-tv/proxy",
-  "version": "2.43.1",
+  "version": "2.44.0",
   "description": "Torrent proxy client that exposes webseed-like HTTP stream endpoint.",
   "license": "GPL-3.0-or-later",
   "publishConfig": {
@@ -44,6 +44,6 @@
     "@biomejs/biome": "^2.5.7"
   },
   "overrides": {
-    "utp-native": "npm:@torrent-tv/utp-native@2.5.3-ttv.2"
+    "utp-native": "npm:@torrent-tv/utp-native@2.5.3-ttv.3"
   }
 }

package/routes/api/subtitles/get.js

@@ -100,18 +100,30 @@ export async function handleApiSubtitlesGet(req, reply, { sourceRegistry, torren
   // read that way. Costs no network at all and answers with the part of the
   // film they are watching; the rest arrives as they watch it. Only when the
   // container cannot be read this way does the old extraction run.
-  // Where the browser's copy of this track ends. It already holds every cue
-  // before this second, so sending them again is bytes for nothing: measured
+  // How many cues this browser has already been sent, counted in the order
+  // they were FOUND. Sending them again is bytes for nothing: measured
   // 2026-08-19, one track is 76 KB and the browser asked for it every few
   // seconds while the film downloaded. Absent or unparsable means "send
   // everything", which is what a browser asking for the first time wants.
+  //
+  // Found-order, not time. A cue's time cannot serve as a cursor here: the
+  // cues are read out of whichever clusters are downloaded, and those are not
+  // contiguous, so the set grows in the middle as well as at the end. The old
+  // `?after=<seconds>` therefore threw away every cue that turned up BEHIND
+  // the furthest one already sent — which is the stretch the viewer is about
+  // to watch. Measured 2026-08-20: a viewer at 272 s was sent cues out to
+  // 1176 s, and from that moment nothing between the two could ever reach
+  // them, with 59 of 276 clusters read. `after` is still honoured so an older
+  // browser keeps working.
+  const since = Number.parseInt(String(req.query?.since ?? ""), 10);
   const after = Number.parseFloat(String(req.query?.after ?? ""));
   const held = await cuesFromDownloadedClusters(
     torrentPool,
     torrent,
     fileIndex,
     trackIndex,
-    Number.isFinite(after) ? after : null
+    Number.isFinite(after) ? after : null,
+    Number.isInteger(since) ? since : null
   );
   if (held !== null) {
     setLanguageHeaders(reply, held.language);
@@ -122,9 +134,11 @@ export async function handleApiSubtitlesGet(req, reply, { sourceRegistry, torren
     // as playback moves into clusters that were not downloaded yet.
     reply.header("X-Subtitle-Covered-Clusters", String(held.coveredClusters));
     reply.header("X-Subtitle-Indexed-Clusters", String(held.indexedClusters));
+    // What to send back as `?since=` next time.
+    reply.header("X-Subtitle-Cursor", String(held.cursor));
     reply.raw.setHeader(
       "Access-Control-Expose-Headers",
-      "X-Subtitle-Language, X-Subtitle-Language-Name, X-Subtitle-Covered-Clusters, X-Subtitle-Indexed-Clusters"
+      "X-Subtitle-Language, X-Subtitle-Language-Name, X-Subtitle-Covered-Clusters, X-Subtitle-Indexed-Clusters, X-Subtitle-Cursor"
     );
     return reply.send(held.vtt);
   }
@@ -264,7 +278,7 @@ function readFileFully(file, maxBytes) {
  * @returns {Promise<{ vtt: string, language: object | null, coveredClusters: number, indexedClusters: number } | null>}
  *   Null when this file cannot be read this way, and then the caller falls back.
  */
-async function cuesFromDownloadedClusters(torrentPool, torrent, fileIndex, trackIndex, after = null) {
+async function cuesFromDownloadedClusters(torrentPool, torrent, fileIndex, trackIndex, after = null, since = null) {
   if (typeof torrentPool?.getSubtitleTracks !== "function") {
     return null;
   }
@@ -290,9 +304,12 @@ async function cuesFromDownloadedClusters(torrentPool, torrent, fileIndex, track
   // Only what the browser does not have. The language is still detected from
   // EVERYTHING held, because three new lines say much less about a language
   // than the whole track does.
-  const fresh = Number.isFinite(after)
-    ? held.cues.filter((cue) => cue.startSeconds > after)
-    : held.cues;
+  const cursor = held.cues.reduce((highest, cue) => Math.max(highest, Number(cue.seq) || 0), 0);
+  const fresh = Number.isInteger(since)
+    ? held.cues.filter((cue) => (Number(cue.seq) || 0) > since)
+    : Number.isFinite(after)
+      ? held.cues.filter((cue) => cue.startSeconds > after)
+      : held.cues;
   const vtt = cuesToVtt(fresh, held.codecId);
   const language = held.cues.length > 0
     ? detectLanguage(held.cues.map((cue) => cue.text).join("\n"))
@@ -300,6 +317,7 @@ async function cuesFromDownloadedClusters(torrentPool, torrent, fileIndex, track
   return {
     vtt,
     language,
+    cursor,
     coveredClusters: held.coveredClusters ?? 0,
     indexedClusters: held.indexedClusters ?? 0
   };