@torrent-tv/proxy 2.24.1 → 2.25.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,3 +1,13 @@
1
+ ## 2.25.1
2
+
3
+ - **Fix**: Picture and sound are back in step. Two releases in a row moved a segment's stamp toward the playlist — 2.24.1 per session, 2.25.0 by one offset for the whole family — and both desynced playback in the field the same day. The reason is what the first segment of a run is: it is not CUT at all, it begins where ffmpeg's seek landed, and the picture must land on a keyframe while the sound needs none, so after every restart the two runs genuinely begin at different real times and the whole run carries that difference (measured: the sound's #292 began at 1587.892 s and #293 at 1592.692 s, one segment apart, the run shifted 2.5 s from the grid). Labelling each track with its own true time is what keeps them together in real time; a segment is stamped with its own start again, as it was for weeks before 2.24.1. What stays from those releases is the part that was right: one published timeline per family, and a warning when a piece lands further from the playlist than a player will bridge.
4
+ - **Chore**: The run's state now answers the questions its process handle used to be asked. Ten sites that re-derived "is this run alive" from a child-process handle, and every read of "is it suspended", now read the state machine shipped in 2.23.0; the `encoderPaused` field is gone. The two places that ask about a NAMED process — the predecessor a restart is replacing, and a deliberate stop — still ask the OS, which remains the authority on whether a pid exists.
5
+
6
+ ## 2.25.0
7
+
8
+ - **Fix**: Picture and sound drifted apart after a seek, by exactly the amount the grid had been corrected. 2.24.1 made every segment stamp itself against the playlist its own session published — but each session froze that playlist at its own creation, and a soundtrack or a quality step is created later than the picture it accompanies, so it froze a table that had since been corrected. Two members of one family then stated the same moment differently, and the corrections measured on the field file are 0.6-2.9 s. A family now publishes ONE timeline: a session created inside a family takes its base's published table verbatim and writes its own playlist from it, while the live table goes on being corrected for cutting, which is what keeps a re-encoded step aligned with the copy it joins.
9
+ - **New**: The read window grows into a lead instead of staying a fixed length. Every wait that cost time widens it by a piece; every piece already in hand gives one back, down to the size the caller sized from the file's own byte rate. The ceiling is this reader's share of the store's memory, so widening can never ask for more than the store can hold. Measured 2026-08-17, the swarm delivered 5.1-5.9 MB/s against a film consumed at about 1 MB/s while the reader still blocked 47 times in two minutes — a fivefold surplus that never became distance ahead of the head.
10
+
1
11
  ## 2.24.1
2
12
 
3
13
  - **Fix**: Seeking could leave a film dead. After a seek the encoder restarts at the segment before the target, and every segment it then produces states its own position, read out of the piece. On a file whose container index is wrong those positions disagree with the playlist the player is holding — measured 2026-08-17, a seek to 1590.4 s produced audio segments #292 and #293 carrying 1587.892 s and 1592.692 s against a playlist saying 1585.376 s and 1590.585 s. A fragment landing further from where the playlist put it than a player will bridge (hls.js bridges `maxBufferHole`, 0.5 s by default) is not recognised as buffered, so the browser asks for it again: those two segments were fetched **1908 times each over ten minutes**, every one served in 4 ms, with the picture frozen and nothing in either log saying why. A segment is now stamped where the playlist the player holds says it begins, whenever the piece's own figure is further away than that; within it the piece's own figure is kept, which is what keeps speech and subtitles together on a file whose index is slightly out. The boundary table goes on being corrected from produced segments — that is what lets a re-encoded step be cut like the copy it joins — but the correction no longer moves segments under a player holding the original playlist: the published table is frozen when the playlist text is written from it. Pinned by `test/published-timeline.test.js` with the field figures.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@torrent-tv/proxy",
3
- "version": "2.24.1",
3
+ "version": "2.25.1",
4
4
  "description": "Torrent proxy client that exposes webseed-like HTTP stream endpoint.",
5
5
  "license": "GPL-3.0-or-later",
6
6
  "publishConfig": {
@@ -20,7 +20,13 @@ import { logger } from "../utils/logger.js";
20
20
  import { readKeyframeIndex } from "./container-index/index.js";
21
21
  import { readMachineState, readProcessCpuSeconds, readProxyCpuSeconds, readSystemCpu, shareOfMachine } from "./host-load.js";
22
22
  import { speedFromReadings } from "./encoder-readings.js";
23
- import { ENCODE_RUN_EVENT, ENCODE_RUN_STATE, INITIAL_RUN_STATE, nextState } from "./encode-run-state.js";
23
+ import {
24
+ ENCODE_RUN_EVENT,
25
+ ENCODE_RUN_STATE,
26
+ INITIAL_RUN_STATE,
27
+ nextState,
28
+ processCanBeSignalled
29
+ } from "./encode-run-state.js";
24
30
  import { ENCODE_EXIT, classifyEncodeExit } from "./encode-exit.js";
25
31
 
26
32
  /** Own package version, stamped onto session-start log lines. */
@@ -1741,6 +1747,14 @@ export class HlsSessionManager {
1741
1747
  })
1742
1748
  : []);
1743
1749
  const usingKeyframeBoundaries = useKeyframeGrid;
1750
+ // What this session will PUBLISH. A member of a family takes its base's
1751
+ // published table verbatim; a session with no base publishes what it cuts
1752
+ // at. The two differ exactly by the corrections made since the family's
1753
+ // first playlist was written, and that difference is what must never reach
1754
+ // the player as two different timelines.
1755
+ const publishedGrid = Array.isArray(inheritedGrid?.published) && inheritedGrid.published.length > 1
1756
+ ? inheritedGrid.published
1757
+ : (hasDuration ? [...segmentBoundaries] : null);
1744
1758
  const segmentCount = segmentBoundaries.length > 1 ? segmentBoundaries.length - 1 : 0;
1745
1759
 
1746
1760
  // Realtime budget (software encoder): pick the output resolution + libx264
@@ -1809,7 +1823,7 @@ export class HlsSessionManager {
1809
1823
  // `encode-run-state.js`. Written at every event today and read by nothing
1810
1824
  // yet: a refused pair in the log is the model disagreeing with reality,
1811
1825
  // and that disagreement is the measurement this release exists to take.
1812
- // The fields it will replace — `state`, `progress.state`, `encoderPaused`
1826
+ // The fields it replaces — `state`, `progress.state`
1813
1827
  // and the repeated liveness checks — keep their current writes meanwhile.
1814
1828
  runState: INITIAL_RUN_STATE,
1815
1829
  lastError: "",
@@ -1908,13 +1922,18 @@ export class HlsSessionManager {
1908
1922
  // spliced into the copy (roadmap item 28).
1909
1923
  containerFormat,
1910
1924
  indexCheck: newIndexCheck(),
1911
- playlistText: hasDuration ? this.#buildVodPlaylist(segmentBoundaries, segmentFormat) : "",
1912
- // The table AS PUBLISHED, frozen the moment the playlist text is written
1913
- // from it. `segmentBoundaries` keeps being corrected from produced
1914
- // segments that is what makes a re-encoded rung cut like the copy it
1915
- // joins — but the player's own copy of the timeline never changes, and a
1916
- // segment must be stamped against the copy the player has.
1917
- publishedBoundaries: hasDuration ? [...segmentBoundaries] : null,
1925
+ playlistText: hasDuration ? this.#buildVodPlaylist(publishedGrid, segmentFormat) : "",
1926
+ // The table AS PUBLISHED what every playlist of this family states, and
1927
+ // what every segment of it is stamped against. Inherited whole from the
1928
+ // base when there is one, so a rung or a soundtrack created later
1929
+ // publishes the same timeline as the picture it plays with; only a family
1930
+ // with no base freezes a copy of its own.
1931
+ //
1932
+ // `segmentBoundaries` keeps being corrected from produced segments — that
1933
+ // is what makes a re-encoded rung cut like the copy it joins — and those
1934
+ // corrections deliberately do NOT reach this copy: the player's timeline
1935
+ // was sent once and cannot be revised.
1936
+ publishedBoundaries: publishedGrid,
1918
1937
  // Segment index the current ffmpeg run started producing from.
1919
1938
  encodeStartIndex: 0,
1920
1939
  // Guards against repeatedly restarting to the same seek position.
@@ -1939,7 +1958,6 @@ export class HlsSessionManager {
1939
1958
  // encoder is currently suspended for running too far past it.
1940
1959
  // See #enforceLookAhead.
1941
1960
  lastRequestedSegment: null,
1942
- encoderPaused: false,
1943
1961
  encoderPauseUnsupported: false,
1944
1962
  seekFirstFarAt: 0,
1945
1963
  // Circuit breaker: consecutive FAST failures (see SEEK_FAST_FAIL_MS) at
@@ -2731,7 +2749,7 @@ export class HlsSessionManager {
2731
2749
  if (aheadSeconds === null) {
2732
2750
  // The segment the viewer needs does not exist. Whatever else is on disk,
2733
2751
  // this encoder has work to do right now.
2734
- if (session.encoderPaused && this.#resumeEncoder(session, "the viewer needs a segment nobody has made")) {
2752
+ if (session.runState === ENCODE_RUN_STATE.SUSPENDED && this.#resumeEncoder(session, "the viewer needs a segment nobody has made")) {
2735
2753
  this.#transitionRun(session, ENCODE_RUN_EVENT.RESUME_ORDERED);
2736
2754
  }
2737
2755
  return;
@@ -2759,7 +2777,7 @@ export class HlsSessionManager {
2759
2777
  );
2760
2778
  }
2761
2779
 
2762
- if (!session.encoderPaused && aheadSeconds > LOOKAHEAD_PAUSE_SECONDS) {
2780
+ if (session.runState !== ENCODE_RUN_STATE.SUSPENDED && aheadSeconds > LOOKAHEAD_PAUSE_SECONDS) {
2763
2781
  // The decision names what it was taken on. Suspending the encoder stops
2764
2782
  // the only thing that reads the input, so a wrong reading here stops the
2765
2783
  // download too — measured 2026-08-06: the log said "135s ahead" while
@@ -2773,7 +2791,7 @@ export class HlsSessionManager {
2773
2791
  `(viewer at #${viewerSegment}, unbroken through #${reading.lastCovered}, ` +
2774
2792
  `${reading.total} segment file(s) present)`
2775
2793
  );
2776
- } else if (session.encoderPaused && aheadSeconds <= LOOKAHEAD_RESUME_SECONDS) {
2794
+ } else if (session.runState === ENCODE_RUN_STATE.SUSPENDED && aheadSeconds <= LOOKAHEAD_RESUME_SECONDS) {
2777
2795
  if (this.#resumeEncoder(session, `${Math.round(aheadSeconds)}s ahead of the viewer`)) {
2778
2796
  this.#transitionRun(session, ENCODE_RUN_EVENT.RESUME_ORDERED);
2779
2797
  }
@@ -2916,7 +2934,7 @@ export class HlsSessionManager {
2916
2934
  #pauseEncoder(session, reason) {
2917
2935
  // Any pair spanning this would count a stopped encoder as slow.
2918
2936
  session.learnSample = null;
2919
- if (session.encoderPaused || session.encoderPauseUnsupported || !session.ffmpeg?.pid) {
2937
+ if (session.runState === ENCODE_RUN_STATE.SUSPENDED || session.encoderPauseUnsupported || !session.ffmpeg?.pid) {
2920
2938
  return;
2921
2939
  }
2922
2940
  try {
@@ -2929,7 +2947,6 @@ export class HlsSessionManager {
2929
2947
  );
2930
2948
  return;
2931
2949
  }
2932
- session.encoderPaused = true;
2933
2950
  this.#transitionRun(session, ENCODE_RUN_EVENT.SUSPEND_ORDERED);
2934
2951
  logger.info(
2935
2952
  `transcode ${session.id} encoder suspended — ${reason} ` +
@@ -2952,7 +2969,7 @@ export class HlsSessionManager {
2952
2969
  #resumeEncoder(session, reason) {
2953
2970
  // Any pair spanning this would count a stopped encoder as slow.
2954
2971
  session.learnSample = null;
2955
- if (!session.encoderPaused || !session.ffmpeg?.pid) {
2972
+ if (session.runState !== ENCODE_RUN_STATE.SUSPENDED || !session.ffmpeg?.pid) {
2956
2973
  return false;
2957
2974
  }
2958
2975
  let continued = true;
@@ -2964,7 +2981,6 @@ export class HlsSessionManager {
2964
2981
  // stops a dead run being reported as producing again.
2965
2982
  continued = false;
2966
2983
  }
2967
- session.encoderPaused = false;
2968
2984
  // Two records of one moment must not contradict each other: a line saying
2969
2985
  // the encoder resumed, beside a return value saying nothing was resumed, is
2970
2986
  // the sort of pair that costs an hour of reading a field log.
@@ -3067,9 +3083,9 @@ export class HlsSessionManager {
3067
3083
 
3068
3084
  async #reportHostLoad() {
3069
3085
  const encoding = [...this.sessionsById.values()].filter(
3070
- (session) => session?.ffmpeg != null && !hasChildExited(session.ffmpeg) && session.state !== "disposed"
3086
+ (session) => processCanBeSignalled(session?.runState) && session.state !== "disposed"
3071
3087
  );
3072
- const runningNow = encoding.filter((session) => session.encoderPaused !== true);
3088
+ const runningNow = encoding.filter((session) => session.runState !== ENCODE_RUN_STATE.SUSPENDED);
3073
3089
  if (runningNow.length === 0) {
3074
3090
  // No encoder is RUNNING. A suspended one costs nothing, and counting it
3075
3091
  // as work meant this was never reached: measured 2026-08-15, four minutes
@@ -3143,7 +3159,7 @@ export class HlsSessionManager {
3143
3159
  // the addon host actually were (2026-08-15: `ffmpeg=0% system=24%`, both
3144
3160
  // encoders suspended, and the speed beside it a stale figure from before
3145
3161
  // they stopped).
3146
- const suspended = encoding.filter((session) => session.encoderPaused === true).length;
3162
+ const suspended = encoding.filter((session) => session.runState === ENCODE_RUN_STATE.SUSPENDED).length;
3147
3163
  const running = encoding.length - suspended;
3148
3164
  const machine = await readMachineState();
3149
3165
  const asPercent = (value) => (value === null ? "n/a" : `${Math.round(value * 100)}%`);
@@ -3740,7 +3756,6 @@ export class HlsSessionManager {
3740
3756
  // judged finished — see getFileStream.
3741
3757
  session.usesExplicitCuts = Boolean(cutTimes && cutTimes.length > 0);
3742
3758
  session.encodeStartIndex = safeIndex;
3743
- session.encoderPaused = false;
3744
3759
  session.pendingRestartIndex = -1;
3745
3760
  session.lastRestartAt = Date.now();
3746
3761
  session.state = session.state === "disposed" ? "disposed" : "starting";
@@ -4178,7 +4193,7 @@ export class HlsSessionManager {
4178
4193
  }
4179
4194
  // Nothing is encoding: a rung the viewer has switched away from is left
4180
4195
  // exactly so, and its held requests must not bring its encoder back.
4181
- if (session.ffmpeg == null || hasChildExited(session.ffmpeg)) {
4196
+ if (!processCanBeSignalled(session.runState)) {
4182
4197
  return;
4183
4198
  }
4184
4199
  // A seek already settling is about to move the encoder to where the VIEWER
@@ -4362,7 +4377,7 @@ export class HlsSessionManager {
4362
4377
  // "already covered", so nothing could ever restart it. Measured 2026-08-04:
4363
4378
  // one ffmpeg failure turned into a session that answered 500 to every
4364
4379
  // segment for as long as the viewer kept trying.
4365
- const runIsAlive = session.ffmpeg != null && !hasChildExited(session.ffmpeg);
4380
+ const runIsAlive = processCanBeSignalled(session.runState);
4366
4381
  if (runIsAlive && index >= head && index <= currentSeg + MAX_LOOKAHEAD_SEGMENTS) {
4367
4382
  logger.info(
4368
4383
  `transcode ${session.id} seek to ${positionSeconds.toFixed(1)}s (#${index}) ` +
@@ -4416,7 +4431,7 @@ export class HlsSessionManager {
4416
4431
  // ten seconds, each killing a run that was encoding #865. The guard below
4417
4432
  // did not catch it — it only decides whether to let the current run finish,
4418
4433
  // not whether a new run is needed at all.
4419
- if (target === session.encodeStartIndex && session.ffmpeg != null && !hasChildExited(session.ffmpeg)) {
4434
+ if (target === session.encodeStartIndex && processCanBeSignalled(session.runState)) {
4420
4435
  logger.info(
4421
4436
  `transcode ${session.id} seek #${target} ignored — the current run already starts there`
4422
4437
  );
@@ -4444,7 +4459,7 @@ export class HlsSessionManager {
4444
4459
  // Holding it was also expensive in the other direction: a genuine second
4445
4460
  // seek could be delayed by the whole grace.
4446
4461
  const producedThisRun = this.#producedSecondsThisRun(session);
4447
- const runIsAlive = session.ffmpeg != null && !hasChildExited(session.ffmpeg);
4462
+ const runIsAlive = processCanBeSignalled(session.runState);
4448
4463
  const allowedBecause = !runIsAlive
4449
4464
  ? "run is dead"
4450
4465
  : `viewer moved; run had produced ${producedThisRun.toFixed(1)}s`;
@@ -5046,7 +5061,7 @@ export class HlsSessionManager {
5046
5061
  .map((member) => this.#observedAudioCost.get(this.#audioCostKey(member))?.version ?? 0)
5047
5062
  .reduce((total, one) => total + one, 0);
5048
5063
  const running = [...this.#familyOf(owner)]
5049
- .filter((member) => member.ffmpeg != null && !hasChildExited(member.ffmpeg)).length;
5064
+ .filter((member) => processCanBeSignalled(member.runState)).length;
5050
5065
  // What each running encode was last seen doing, which is BOTH an input to
5051
5066
  // the answer twice over — it withdraws a step measured below realtime, and
5052
5067
  // it prices every running picture in the committed total — and a figure
@@ -5204,7 +5219,7 @@ export class HlsSessionManager {
5204
5219
  for (const session of this.sessionsById.values()) {
5205
5220
  if (session?.ffmpeg != null &&
5206
5221
  !hasChildExited(session.ffmpeg) &&
5207
- session.encoderPaused !== true &&
5222
+ session.runState !== ENCODE_RUN_STATE.SUSPENDED &&
5208
5223
  session.state !== "disposed") {
5209
5224
  running += 1;
5210
5225
  }
@@ -5218,7 +5233,7 @@ export class HlsSessionManager {
5218
5233
  session.state === "disposed" ||
5219
5234
  session.state === "failed" ||
5220
5235
  !session.ffmpeg ||
5221
- session.encoderPaused === true
5236
+ session.runState === ENCODE_RUN_STATE.SUSPENDED
5222
5237
  ) {
5223
5238
  session.learnSample = null;
5224
5239
  return;
@@ -5323,7 +5338,7 @@ export class HlsSessionManager {
5323
5338
  if (!Number.isFinite(runStartedAt) || Date.now() - runStartedAt < DECODE_LEARNING_SETTLE_MS) {
5324
5339
  return;
5325
5340
  }
5326
- if (session.encoderPaused === true) {
5341
+ if (session.runState === ENCODE_RUN_STATE.SUSPENDED) {
5327
5342
  return; // a suspended run reports a cumulative figure that is decaying
5328
5343
  }
5329
5344
  // Always asked, not only below realtime. A re-encode near 1x may be the
@@ -5374,7 +5389,7 @@ export class HlsSessionManager {
5374
5389
  if (!Number.isFinite(runStartedAt) || Date.now() - runStartedAt < DECODE_LEARNING_SETTLE_MS) {
5375
5390
  return;
5376
5391
  }
5377
- if (session.encoderPaused === true) {
5392
+ if (session.runState === ENCODE_RUN_STATE.SUSPENDED) {
5378
5393
  return;
5379
5394
  }
5380
5395
  if (await this.#classifyTranscodeBound(session) === "download") {
@@ -5617,7 +5632,7 @@ export class HlsSessionManager {
5617
5632
  #pricedConcurrentCost(session) {
5618
5633
  let cost = 0;
5619
5634
  for (const member of this.#familyOf(session)) {
5620
- if (member === session || member.ffmpeg == null || hasChildExited(member.ffmpeg)) {
5635
+ if (member === session || !processCanBeSignalled(member.runState)) {
5621
5636
  continue;
5622
5637
  }
5623
5638
  if (member.audioOnly === true) {
@@ -5646,7 +5661,7 @@ export class HlsSessionManager {
5646
5661
  // price to look up for another film's session — so a reading taken while
5647
5662
  // one is running cannot be attributed either.
5648
5663
  return this.#runningEncoders() > this.#familyOf(session).filter(
5649
- (member) => member.ffmpeg != null && !hasChildExited(member.ffmpeg)
5664
+ (member) => processCanBeSignalled(member.runState)
5650
5665
  ).length
5651
5666
  ? null
5652
5667
  : cost;
@@ -5670,7 +5685,7 @@ export class HlsSessionManager {
5670
5685
  if (member.audioOnly === true || member.transcodeVideo !== true) {
5671
5686
  continue;
5672
5687
  }
5673
- if (member.ffmpeg == null || hasChildExited(member.ffmpeg)) {
5688
+ if (!processCanBeSignalled(member.runState)) {
5674
5689
  continue;
5675
5690
  }
5676
5691
  const height = this.variantHeightOf(member);
@@ -5714,7 +5729,7 @@ export class HlsSessionManager {
5714
5729
  // that cost is spread, not a discount on it — and pricing a parked
5715
5730
  // encoder at zero would offer a step on the strength of a pause that ends
5716
5731
  // the moment the viewer catches up.
5717
- if (member.ffmpeg == null || hasChildExited(member.ffmpeg)) {
5732
+ if (!processCanBeSignalled(member.runState)) {
5718
5733
  continue;
5719
5734
  }
5720
5735
  if (member.audioOnly === true) {
@@ -6134,8 +6149,14 @@ export class HlsSessionManager {
6134
6149
  inheritedGrid: base.cutGrid === "keyframe"
6135
6150
  ? {
6136
6151
  // The table as it stands NOW, corrections included — not the index
6137
- // it was first built from.
6152
+ // it was first built from. This is what the new session CUTS at.
6138
6153
  boundaries: base.segmentBoundaries,
6154
+ // And this is what it must SAY, which is not the same thing: every
6155
+ // member of a family has to publish one timeline, or two sessions
6156
+ // stamp the same moment differently and the picture and the sound
6157
+ // drift apart by exactly the corrections made between their two
6158
+ // creations (field 2026-08-17, corrections of 0.6-2.9 s).
6159
+ published: base.publishedBoundaries,
6139
6160
  keyframeTimes: base.keyframeTimes,
6140
6161
  containerFormat: base.containerFormat
6141
6162
  }
@@ -6670,6 +6691,7 @@ export class HlsSessionManager {
6670
6691
  inheritedGrid: base.cutGrid === "keyframe"
6671
6692
  ? {
6672
6693
  boundaries: base.segmentBoundaries,
6694
+ published: base.publishedBoundaries,
6673
6695
  keyframeTimes: base.keyframeTimes,
6674
6696
  containerFormat: base.containerFormat
6675
6697
  }
@@ -7059,9 +7081,30 @@ export class HlsSessionManager {
7059
7081
  // is kept, because it is the honest one and it is what keeps speech and
7060
7082
  // subtitles together on a file whose index is slightly out (2026-08-06,
7061
7083
  // 4.17 s of drift on a Matroska index that lied).
7062
- let stampStart = trueStart ?? publishedStart;
7084
+ // STAMPED WITH ITS OWN TRUE START, always. Two attempts at moving it
7085
+ // toward the playlist both made things worse, and the reason is in what
7086
+ // the first segment of a run is: it is not CUT at all — it begins where
7087
+ // ffmpeg's seek landed. The picture must land on a keyframe; the sound
7088
+ // needs none and starts at the instant asked for. So after every
7089
+ // restart the two runs genuinely begin at different real times, and the
7090
+ // whole run carries that difference (field 2026-08-17: the sound's
7091
+ // #292 began at 1587.892 s and #293 at 1592.692 s — exactly one segment
7092
+ // apart, the whole run shifted 2.5 s from the grid).
7093
+ //
7094
+ // Labelling each track with its own true time is therefore what keeps
7095
+ // picture and sound together in real time. Moving them onto the
7096
+ // published grid — separately (2.24.1) or by one family offset (2.25.0)
7097
+ // — closes a gap that is real and opens one that is not: it desynced
7098
+ // playback in the field within the hour, twice.
7099
+ //
7100
+ // What that leaves unsolved is the reason those attempts were made: a
7101
+ // playlist that disagrees with the media by more than a player bridges
7102
+ // makes hls.js refetch the same fragment for ever (1908 times each for
7103
+ // two segments, measured). The answer to THAT is to make the published
7104
+ // grid agree with where the runs really begin — not to relabel the
7105
+ // media. Recorded as its own roadmap item rather than guessed at here.
7106
+ const stampStart = trueStart ?? publishedStart;
7063
7107
  if (trueStart !== null && Math.abs(trueStart - publishedStart) > PLAYER_BUFFER_HOLE_SEC) {
7064
- stampStart = publishedStart;
7065
7108
  this.#notePlaylistDisagreement(session, index, trueStart, publishedStart);
7066
7109
  }
7067
7110
  const prepared = session.segmentFormat.prepareSegmentBytes(bytes, {
@@ -60,6 +60,37 @@ export function readWindowFor({ pieceIndex, lastPiece, windowPieces }) {
60
60
  return { from: pieceIndex, to: Math.min(lastPiece, pieceIndex + span - 1) };
61
61
  }
62
62
 
63
+ /**
64
+ * How wide the window should be after a piece that made the reader wait — or
65
+ * did not.
66
+ *
67
+ * The swarm's surplus is what pays for this. Measured 2026-08-17 on the field
68
+ * torrent: 5.1-5.9 MB/s delivered against a film consumed at about 1 MB/s, and
69
+ * the reader still blocked 47 times in two minutes, median 1.5 s, worst 4.5 s.
70
+ * A fivefold surplus never became distance ahead of the head, because the
71
+ * window is a fixed number of seconds of playback and everything past it is
72
+ * ordinary background fill at no priority.
73
+ *
74
+ * So the window follows the evidence: every wait that mattered widens it by a
75
+ * piece, every piece that was already there narrows it back toward the size the
76
+ * caller asked for. Nothing here is chosen — the wait is measured, the
77
+ * threshold is the one that already defines "a wait worth recording", and the
78
+ * ceiling is this reader's share of the store's memory, so widening can never
79
+ * cost more than the store can hold.
80
+ *
81
+ * @param {{ current: number, base: number, ceiling: number, waitedMs: number, waitThresholdMs: number }} params
82
+ * @returns {number}
83
+ */
84
+ export function nextWindowPieces({ current, base, ceiling, waitedMs, waitThresholdMs }) {
85
+ const floor = Math.max(1, Math.floor(base));
86
+ const top = Math.max(floor, Math.floor(ceiling));
87
+ const now = Math.min(top, Math.max(floor, Math.floor(current)));
88
+ if (waitedMs >= waitThresholdMs) {
89
+ return Math.min(top, now + 1);
90
+ }
91
+ return Math.max(floor, now - 1);
92
+ }
93
+
63
94
  /**
64
95
  * Add this reader's window to the download set as a stream selection.
65
96
  *
@@ -310,7 +341,26 @@ export async function* readFragments({
310
341
  // critical, and never deselected anything — so ffmpeg's opening
311
342
  // `bytes 0-<EOF>` left a permanent selection over the entire file, and no
312
343
  // later prioritisation could outrank it.
313
- const windowPieces = Math.max(1, Math.ceil(Math.max(1, windowBytes) / pieceLength));
344
+ const basePieces = Math.max(1, Math.ceil(Math.max(1, windowBytes) / pieceLength));
345
+ // What the window is RIGHT NOW. It starts at what the caller sized in seconds
346
+ // of playback and grows while the reader keeps being made to wait — see
347
+ // `nextWindowPieces`.
348
+ let windowPieces = basePieces;
349
+ /**
350
+ * The widest this reader may go: its share of what the store can hold in
351
+ * memory. Measured rather than chosen — the capacity is the store's own, and
352
+ * the number of readers is how many windows are declared on it right now.
353
+ *
354
+ * @returns {number}
355
+ */
356
+ const ceilingPieces = () => {
357
+ const capacity = Number(store?.capacity);
358
+ if (!Number.isFinite(capacity) || capacity <= 0) {
359
+ return basePieces;
360
+ }
361
+ const readers = Math.max(1, store.protectedRanges?.().length ?? 1);
362
+ return Math.max(basePieces, Math.floor(capacity / readers));
363
+ };
314
364
  /** @type {{ from: number, to: number } | null} */
315
365
  let window = null;
316
366
  /** @type {{ from: number, to: number } | null} */
@@ -408,6 +458,20 @@ export async function* readFragments({
408
458
  // whether that is the swarm, the picker, or ffmpeg. Logged only when the
409
459
  // wait is long enough to matter, so ordinary sequential reading is silent.
410
460
  const waitedMs = Date.now() - waitStartedAt;
461
+ // The window answers to what just happened: a wait means the lead was too
462
+ // short, an immediate hit means it is longer than it needs to be. Applied
463
+ // before the logging below so the line reports the window the next piece
464
+ // will actually use.
465
+ const widened = nextWindowPieces({
466
+ current: windowPieces,
467
+ base: basePieces,
468
+ ceiling: ceilingPieces(),
469
+ waitedMs,
470
+ waitThresholdMs: PIECE_WAIT_LOG_MS
471
+ });
472
+ if (widened !== windowPieces) {
473
+ windowPieces = widened;
474
+ }
411
475
  if (waitedMs >= PIECE_WAIT_LOG_MS) {
412
476
  const rateKbps = Math.round(pieceLength / 1024 / (waitedMs / 1000));
413
477
  logger.info(
@@ -63,6 +63,7 @@ async function managerWithRunAhead() {
63
63
  seekTarget: null,
64
64
  waitEpoch: 0,
65
65
  firstWantedAt: new Map(),
66
+ runState: "PRODUCING",
66
67
  ffmpeg: { pid: 4321, exitCode: null, signalCode: null, kill() {}, once(event, handler) { if (event === "exit") handler(); } },
67
68
  progress: { state: "running", processedSeconds: RUN_STARTS_AT * SEGMENT_SECONDS + 400, startPositionSeconds: RUN_STARTS_AT * SEGMENT_SECONDS }
68
69
  };
@@ -286,6 +286,7 @@ test("a segment request hands the encoder to the variant the viewer moved to", a
286
286
  base.lastRequestedSegment = 25;
287
287
  const encoder = fakeEncoder();
288
288
  base.ffmpeg = encoder;
289
+ base.runState = "PRODUCING";
289
290
 
290
291
  const served = await manager.resolveVariantFile(BASE_ID, 540, "segment-00025.mp4");
291
292
 
@@ -317,6 +318,7 @@ test("a rung is placed where the player asked it for, not where the other rung h
317
318
  manager.sessionsById.set(VARIANT_ID, variant);
318
319
  base.variants = new Map([[540, VARIANT_ID]]);
319
320
  base.ffmpeg = fakeEncoder();
321
+ base.runState = "PRODUCING";
320
322
  // The rung being left had read fourteen segments further than the picture had
321
323
  // played — an encoder running at several times realtime fills the buffer far
322
324
  // ahead. Measured 2026-08-11: 56 s of gap, and using the read head placed the
@@ -347,6 +349,7 @@ test("warming a rung prepares it without taking the encoder from the one on scre
347
349
  base.variants = new Map([[540, VARIANT_ID]]);
348
350
  const encoder = fakeEncoder();
349
351
  base.ffmpeg = encoder;
352
+ base.runState = "PRODUCING";
350
353
 
351
354
  const prepared = await manager.prepareVariant(BASE_ID, 540, 240);
352
355
 
@@ -373,6 +376,7 @@ test("a rung warmed at the playhead survives the switch that lands just ahead of
373
376
  manager.sessionsById.set(VARIANT_ID, variant);
374
377
  base.variants = new Map([[540, VARIANT_ID]]);
375
378
  base.ffmpeg = fakeEncoder();
379
+ base.runState = "PRODUCING";
376
380
 
377
381
  // Warmed AT THE PLAYHEAD (240 s = segment #60), which is what the browser
378
382
  // sends from server 0.10.0 onwards, and the run is alive and has produced a
@@ -380,6 +384,7 @@ test("a rung warmed at the playhead survives the switch that lands just ahead of
380
384
  await manager.prepareVariant(BASE_ID, 540, 240);
381
385
  variant.encodeStartIndex = 59;
382
386
  variant.ffmpeg = fakeEncoder();
387
+ variant.runState = "PRODUCING";
383
388
  variant.progress = { ...variant.progress, processedSeconds: 268 };
384
389
  variant.seekTarget = null;
385
390
  variant.seekSettleTimer = null;
@@ -413,6 +418,7 @@ test("a rung warmed PAST the switch is repositioned, which is what warming late
413
418
  manager.sessionsById.set(VARIANT_ID, variant);
414
419
  base.variants = new Map([[540, VARIANT_ID]]);
415
420
  base.ffmpeg = fakeEncoder();
421
+ base.runState = "PRODUCING";
416
422
 
417
423
  // The same session, warmed where the BUFFER ended rather than where the
418
424
  // picture was — 60 s further on, which is an ordinary cushion. This is what
@@ -420,6 +426,7 @@ test("a rung warmed PAST the switch is repositioned, which is what warming late
420
426
  await manager.prepareVariant(BASE_ID, 540, 300);
421
427
  variant.encodeStartIndex = 74;
422
428
  variant.ffmpeg = fakeEncoder();
429
+ variant.runState = "PRODUCING";
423
430
  variant.progress = { ...variant.progress, processedSeconds: 310 };
424
431
  variant.seekTarget = null;
425
432
  variant.seekSettleTimer = null;
@@ -447,6 +454,7 @@ test("the rung on screen fetching its own segments does not cancel a warm-up", a
447
454
  const warmedEncoder = fakeEncoder();
448
455
  variant.ffmpeg = warmedEncoder;
449
456
  base.ffmpeg = fakeEncoder();
457
+ base.runState = "PRODUCING";
450
458
  await manager.prepareVariant(BASE_ID, 540, 100);
451
459
 
452
460
  // The viewer has not moved: the rung they are watching goes on asking for its
@@ -521,6 +529,7 @@ test("a playlist or an init segment does not move the encoder", async (t) => {
521
529
  manager.sessionsById.set(VARIANT_ID, variant);
522
530
  base.variants = new Map([[540, VARIANT_ID]]);
523
531
  base.ffmpeg = fakeEncoder();
532
+ base.runState = "PRODUCING";
524
533
 
525
534
  base.variants = new Map([[540, VARIANT_ID]]);
526
535
  await manager.resolveVariantFile(BASE_ID, 540, "index.m3u8");
@@ -719,6 +728,7 @@ test("an audio track is prepared at the position the switch will land on", async
719
728
  rendition.audioOnly = true;
720
729
  rendition.audioTrackIndex = 1;
721
730
  rendition.ffmpeg = fakeEncoder();
731
+ rendition.runState = "PRODUCING";
722
732
  rendition.encodeStartIndex = 0;
723
733
  manager.sessionsById.set(VARIANT_ID, rendition);
724
734
  base.audioRenditionSessions = new Map([[1, VARIANT_ID]]);
@@ -790,6 +800,7 @@ test("a quality step being warmed is not refused by its own cost", async (t) =>
790
800
  // Running, and running well: it says of itself that it holds twice realtime,
791
801
  // i.e. half a second of work per second of video.
792
802
  warming.ffmpeg = fakeEncoder();
803
+ warming.runState = "PRODUCING";
793
804
  warming.lastAloneSpeed = 2;
794
805
  manager.sessionsById.set(BASE_ID, base);
795
806
  manager.sessionsById.set(VARIANT_ID, warming);
@@ -19,7 +19,11 @@ import { EventEmitter } from "node:events";
19
19
  import os from "node:os";
20
20
  import path from "node:path";
21
21
  import fs from "node:fs/promises";
22
- import { readFragments, readWindowFor } from "../services/torrent-worker/piece-reader.js";
22
+ import {
23
+ nextWindowPieces,
24
+ readFragments,
25
+ readWindowFor
26
+ } from "../services/torrent-worker/piece-reader.js";
23
27
  import { SharedPieceStore } from "../services/piece-store/shared-piece-store.js";
24
28
 
25
29
  const PIECE = 1024;
@@ -276,3 +280,50 @@ test("a reader that is abandoned mid-fragment does not keep the piece pinned", a
276
280
  await fs.rm(directory, { recursive: true, force: true });
277
281
  }
278
282
  });
283
+
284
+ // ------------------------------------------ the window that grows into a lead
285
+
286
+ test("a wait that mattered widens the window by a piece", () => {
287
+ // Field 2026-08-17: 5.1-5.9 MB/s delivered against ~1 MB/s consumed, and the
288
+ // reader still blocked 47 times in two minutes. The surplus never became
289
+ // distance ahead of the head.
290
+ assert.equal(
291
+ nextWindowPieces({ current: 4, base: 4, ceiling: 12, waitedMs: 1457, waitThresholdMs: 1000 }),
292
+ 5
293
+ );
294
+ });
295
+
296
+ test("a piece that was already there gives a piece back", () => {
297
+ assert.equal(
298
+ nextWindowPieces({ current: 7, base: 4, ceiling: 12, waitedMs: 0, waitThresholdMs: 1000 }),
299
+ 6
300
+ );
301
+ });
302
+
303
+ test("it never shrinks below what the caller asked for", () => {
304
+ assert.equal(
305
+ nextWindowPieces({ current: 4, base: 4, ceiling: 12, waitedMs: 0, waitThresholdMs: 1000 }),
306
+ 4
307
+ );
308
+ });
309
+
310
+ test("it never grows past this reader's share of the store", () => {
311
+ assert.equal(
312
+ nextWindowPieces({ current: 12, base: 4, ceiling: 12, waitedMs: 4453, waitThresholdMs: 1000 }),
313
+ 12
314
+ );
315
+ // A ceiling below the base cannot pull the window under it: the caller sized
316
+ // the base from the file's own byte rate, and a store too small to hold it is
317
+ // an argument about memory, not about what the reader needs next.
318
+ assert.equal(
319
+ nextWindowPieces({ current: 4, base: 4, ceiling: 1, waitedMs: 2000, waitThresholdMs: 1000 }),
320
+ 4
321
+ );
322
+ });
323
+
324
+ test("a wait exactly at the threshold counts as a wait", () => {
325
+ assert.equal(
326
+ nextWindowPieces({ current: 4, base: 4, ceiling: 9, waitedMs: 1000, waitThresholdMs: 1000 }),
327
+ 5
328
+ );
329
+ });
@@ -1,338 +1,334 @@
1
- /**
2
- * @file A finished segment on disk must reach the viewer as bytes.
3
- *
4
- * The module tests cover each piece of the fMP4 path on its own, and every one
5
- * of them passed while playback was dead: 2.9.124 called
6
- * `readSelfContainedStartSeconds` from `fmp4.js` without importing it, and the
7
- * unit test imports that function straight from `mp4-boxes.js`, so the gap
8
- * between a module and its CALLER was invisible. This test asks the session
9
- * manager for a segment that exists and insists on getting it.
10
- *
11
- * The second half pins the reason a one-word slip cost a whole release: the
12
- * failure was reported as "still being produced". Measured 2026-08-08 — segment
13
- * #0 was held for 45 281 ms with twelve finished segments in the directory, and
14
- * the log said nothing at all. Anything that goes wrong while preparing a file
15
- * that EXISTS must be named and answered, never turned into an endless wait.
16
- */
17
-
18
- import test from "node:test";
19
- import assert from "node:assert/strict";
20
- import { mkdtemp, rm, writeFile } from "node:fs/promises";
21
- import os from "node:os";
22
- import path from "node:path";
23
- import { HlsSessionManager } from "../services/hls-session-manager.js";
24
- import { ENCODE_RUN_STATE } from "../services/encode-run-state.js";
25
- import { fmp4Format } from "../services/segment-formats/fmp4.js";
26
-
27
- const MOVIE_TIMESCALE = 1000;
28
- const VIDEO_TIMESCALE = 90_000;
29
- const AUDIO_TIMESCALE = 48_000;
30
- const SEGMENT_START_SECONDS = 12.5;
31
- const SESSION_ID = "11111111-2222-3333-4444-555555555555";
32
-
33
- /**
34
- * @param {string} type
35
- * @param {Buffer} body
36
- * @returns {Buffer}
37
- */
38
- function box(type, body) {
39
- const head = Buffer.alloc(8);
40
- head.writeUInt32BE(8 + body.length, 0);
41
- head.write(type, 4, "latin1");
42
- return Buffer.concat([head, body]);
43
- }
44
-
45
- /**
46
- * `elst` holding one empty edit — how the `segment` muxer records where the
47
- * piece sits on the source timeline.
48
- *
49
- * @param {number} offsetSeconds
50
- * @returns {Buffer}
51
- */
52
- function emptyEdit(offsetSeconds) {
53
- const body = Buffer.alloc(16);
54
- body.writeUInt32BE(1, 4); // entry count
55
- body.writeUInt32BE(Math.round(offsetSeconds * MOVIE_TIMESCALE), 8); // duration
56
- body.writeInt32BE(-1, 12); // media_time
57
- return box("elst", body);
58
- }
59
-
60
- /**
61
- * @param {number} trackId
62
- * @param {number} timescale
63
- * @param {number} offsetSeconds
64
- * @returns {Buffer}
65
- */
66
- function trak(trackId, timescale, offsetSeconds) {
67
- const tkhdBody = Buffer.alloc(84);
68
- tkhdBody.writeUInt32BE(trackId, 12);
69
- const mdhdBody = Buffer.alloc(20);
70
- mdhdBody.writeUInt32BE(timescale, 12);
71
- return box("trak", Buffer.concat([
72
- box("tkhd", tkhdBody),
73
- box("edts", emptyEdit(offsetSeconds)),
74
- box("mdia", box("mdhd", mdhdBody))
75
- ]));
76
- }
77
-
78
- /**
79
- * @param {number} trackId
80
- * @returns {Buffer}
81
- */
82
- function traf(trackId) {
83
- const tfhdBody = Buffer.alloc(8);
84
- tfhdBody.writeUInt32BE(trackId, 4);
85
- const tfdtBody = Buffer.alloc(12);
86
- tfdtBody.writeUInt8(1, 0); // version 1 — 64-bit
87
- tfdtBody.writeBigUInt64BE(0n, 4); // what ffmpeg writes: zero
88
- return box("traf", Buffer.concat([box("tfhd", tfhdBody), box("tfdt", tfdtBody)]));
89
- }
90
-
91
- /**
92
- * A piece shaped like the `segment` muxer's output: header, two fragments and a
93
- * trailing random-access index, all in one file.
94
- *
95
- * @param {number} offsetSeconds
96
- * @returns {Buffer}
97
- */
98
- function selfContainedPiece(offsetSeconds) {
99
- const mvhdBody = Buffer.alloc(100);
100
- mvhdBody.writeUInt32BE(MOVIE_TIMESCALE, 12);
101
- const moov = box("moov", Buffer.concat([
102
- box("mvhd", mvhdBody),
103
- trak(1, VIDEO_TIMESCALE, offsetSeconds),
104
- trak(2, AUDIO_TIMESCALE, offsetSeconds)
105
- ]));
106
- const moof = box("moof", Buffer.concat([traf(1), traf(2)]));
107
- const mdat = box("mdat", Buffer.alloc(64, 0x5a));
108
- const mfra = box("mfra", Buffer.alloc(24, 0));
109
- return Buffer.concat([box("ftyp", Buffer.alloc(16, 0)), moov, moof, mdat, mfra]);
110
- }
111
-
112
- /**
113
- * A manager holding one session whose segments are already on disk, cut at
114
- * explicit times — the ordinary keyframe-cut path.
115
- *
116
- * @param {{ segmentFormat?: object }} [overrides]
117
- * @returns {Promise<{ manager: HlsSessionManager, session: object, dirPath: string }>}
118
- */
119
- async function managerWithReadySegment(overrides = {}) {
120
- const dirPath = await mkdtemp(path.join(os.tmpdir(), "segment-serve-"));
121
- const piece = selfContainedPiece(SEGMENT_START_SECONDS);
122
- // Two segments, because a piece is only finished once the next one exists.
123
- await writeFile(path.join(dirPath, "segment-00000.mp4"), piece);
124
- await writeFile(path.join(dirPath, "segment-00001.mp4"), piece);
125
-
126
- const manager = new HlsSessionManager({
127
- enabled: true,
128
- ffmpegBin: "ffmpeg",
129
- localBindHost: "127.0.0.1",
130
- localPort: 9090
131
- });
132
- const session = {
133
- id: SESSION_ID,
134
- dirPath,
135
- state: "ready",
136
- fileName: "video.mkv",
137
- startedAt: Date.now(),
138
- createEntryMs: Date.now(),
139
- lastAccessedAt: Date.now(),
140
- ffmpeg: null,
141
- lastError: "",
142
- consumers: new Set(),
143
- segmentFormat: overrides.segmentFormat ?? fmp4Format,
144
- usesExplicitCuts: true,
145
- useSyntheticPlaylist: true,
146
- playlistText: "#EXTM3U\n",
147
- segmentBoundaries: [0, SEGMENT_START_SECONDS, 25],
148
- initBytes: fmp4Format.extractInit(piece),
149
- encodeStartIndex: 0,
150
- firstSegmentLogged: false,
151
- waitEpoch: 0
152
- };
153
- manager.sessionsById.set(SESSION_ID, session);
154
- return { manager, session, dirPath };
155
- }
156
-
157
- test("serving a segment records what its real start says about the container's index", async (t) => {
158
- const { manager, session, dirPath } = await managerWithReadySegment();
159
- t.after(async () => {
160
- await manager.disposeAll();
161
- await rm(dirPath, { recursive: true, force: true });
162
- });
163
- // The tally is counted in the module tests; what this pins is that serving a
164
- // segment reaches it at all. A counter nothing increments reports a clean
165
- // index for every file forever, which is worse than no measurement.
166
- session.indexCheck = { checked: 0, disagreed: 0, maxDeviationSec: 0, firstDisagreementIndex: -1, seen: new Set() };
167
-
168
- await manager.getFileStream(SESSION_ID, "segment-00000.mp4", { requestSeq: 1 });
169
-
170
- assert.equal(session.indexCheck.checked, 1, "the boundary that was just produced must have been examined");
171
- });
172
-
173
- test("a segment that exists is served, not reported as still being produced", async (t) => {
174
- const { manager, dirPath, session } = await managerWithReadySegment();
175
- t.after(async () => {
176
- await manager.disposeAll();
177
- await rm(dirPath, { recursive: true, force: true });
178
- });
179
-
180
- const result = await manager.getFileStream(SESSION_ID, "segment-00000.mp4", { requestSeq: 1 });
181
-
182
- assert.equal(result.kind, "file", "a finished segment on disk must come back as bytes");
183
- assert.equal(result.contentType, fmp4Format.segmentContentType);
184
-
185
- const chunks = [];
186
- for await (const chunk of result.stream) {
187
- chunks.push(chunk);
188
- }
189
- const served = Buffer.concat(chunks);
190
- assert.equal(served.toString("latin1", 4, 8), "moof", "the init header must be stripped off a media segment");
191
-
192
- // Where the PLAYLIST the player holds puts this segment which is what the
193
- // fragment must be stamped with whenever the piece's own position is further
194
- // away than a player will bridge. This fixture's piece says 12.5 s while the
195
- // playlist says 0, and stamping the piece's figure is what killed a film on
196
- // 2026-08-17: the browser asked for two segments 1908 times each over ten
197
- // minutes, each served in 4 ms, because a fragment landing 2.5 s from where
198
- // it was expected is not recognised as buffered. Reading the piece's own
199
- // position still happens — it is the step that threw in 2.9.124, it feeds the
200
- // index tally and it corrects the grid for rungs made later — it just no
201
- // longer contradicts the timeline the player was sent. See
202
- // test/published-timeline.test.js for the rule itself.
203
- assert.equal(
204
- Number(served.readBigUInt64BE(served.indexOf("tfdt") + 8)),
205
- 0,
206
- "the segment must be stamped where the playlist the player holds says it begins"
207
- );
208
- assert.equal(
209
- session.indexCheck.checked,
210
- 1,
211
- "and the piece's own position must still have been read, or nothing measures the index"
212
- );
213
- });
214
-
215
- test("a fault while preparing an existing segment is named, not turned into a wait", async (t) => {
216
- const broken = {
217
- ...fmp4Format,
218
- readSegmentStartSeconds() {
219
- throw new ReferenceError("readSelfContainedStartSeconds is not defined");
220
- }
221
- };
222
- const { manager, dirPath } = await managerWithReadySegment({ segmentFormat: broken });
223
- t.after(async () => {
224
- await manager.disposeAll();
225
- await rm(dirPath, { recursive: true, force: true });
226
- });
227
-
228
- const result = await manager.getFileStream(SESSION_ID, "segment-00000.mp4", { requestSeq: 1 });
229
-
230
- assert.equal(
231
- result.kind,
232
- "failed",
233
- "answering 'warming-up' hides the fault and holds every request until the viewer gives up"
234
- );
235
- assert.match(result.message, /readSelfContainedStartSeconds is not defined/);
236
- });
237
-
238
- test("a run's FIRST segment is served once the encoder has passed it, without waiting for a next one", async (t) => {
239
- const { manager, session, dirPath } = await managerWithReadySegment();
240
- t.after(async () => {
241
- await manager.disposeAll();
242
- await rm(dirPath, { recursive: true, force: true });
243
- });
244
- // The shape a resume takes: a run begun mid-file, so its first segment has no
245
- // successor and nothing is producing one. Waiting for that successor is what
246
- // held #317 for 46 s and then answered 404 to a browser that had given up.
247
- await rm(path.join(dirPath, "segment-00001.mp4"));
248
- session.encodeStartIndex = 0;
249
- session.ffmpeg = { killed: true, kill() {} };
250
- session.progress = { processedSeconds: SEGMENT_START_SECONDS + 10 };
251
-
252
- const result = await manager.getFileStream(SESSION_ID, "segment-00000.mp4", { requestSeq: 1 });
253
-
254
- assert.equal(
255
- result.kind,
256
- "file",
257
- "the encoder is past this segment's end, so it is finished the absence of a next one says nothing"
258
- );
259
- });
260
-
261
- test("a segment is found in the run directory that produced it, newest run first", async (t) => {
262
- const { manager, session, dirPath } = await managerWithReadySegment();
263
- t.after(async () => {
264
- await manager.disposeAll();
265
- await rm(dirPath, { recursive: true, force: true });
266
- });
267
- // Runs write into a directory each — that is what lets a restart begin
268
- // without waiting for its predecessor to die, which measured 0.7-1.3 s of
269
- // every seek. A later run's answer supersedes an earlier one's, because the
270
- // older file may be the truncated output of a run that was killed mid-write.
271
- const { mkdir } = await import("node:fs/promises");
272
- const piece = selfContainedPiece(SEGMENT_START_SECONDS);
273
- await mkdir(path.join(dirPath, "run-1"), { recursive: true });
274
- await mkdir(path.join(dirPath, "run-2"), { recursive: true });
275
- await writeFile(path.join(dirPath, "run-1", "segment-00000.mp4"), Buffer.alloc(8));
276
- await writeFile(path.join(dirPath, "run-2", "segment-00000.mp4"), piece);
277
- await writeFile(path.join(dirPath, "run-2", "segment-00001.mp4"), piece);
278
- await rm(path.join(dirPath, "segment-00000.mp4"));
279
- await rm(path.join(dirPath, "segment-00001.mp4"));
280
- session.encodeStartIndex = 0;
281
-
282
- const result = await manager.getFileStream(SESSION_ID, "segment-00000.mp4", { requestSeq: 1 });
283
-
284
- assert.equal(result.kind, "file", "a segment produced by a run must be found in that run's directory");
285
- const chunks = [];
286
- for await (const chunk of result.stream) {
287
- chunks.push(chunk);
288
- }
289
- assert.ok(
290
- Buffer.concat(chunks).length > 8,
291
- "the newest run's output must win the older file here is the 8-byte stub a killed run leaves"
292
- );
293
- });
294
-
295
- test("serving a run's own segment moves the run out of STARTING", async (t) => {
296
- const { manager, session, dirPath } = await managerWithReadySegment();
297
- t.after(async () => {
298
- await manager.disposeAll();
299
- await rm(dirPath, { recursive: true, force: true });
300
- });
301
- // The table lives in `encode-run-state.js` and is tested there as a graph.
302
- // What this pins is that a real serve REACHES it: a state nothing writes
303
- // describes every run as still starting, for ever, and the log built on it
304
- // would say so too.
305
- session.runState = ENCODE_RUN_STATE.STARTING;
306
- // Where the run in force is writing. The fixture's segments live directly in
307
- // the session directory, which is exactly what a single run's directory is
308
- // here.
309
- session.runDirPath = dirPath;
310
-
311
- await manager.getFileStream(SESSION_ID, "segment-00000.mp4", { requestSeq: 1 });
312
-
313
- assert.equal(session.runState, ENCODE_RUN_STATE.PRODUCING);
314
- });
315
-
316
- test("a segment left by an earlier run does not claim the new run has produced", async (t) => {
317
- const { manager, session, dirPath } = await managerWithReadySegment();
318
- t.after(async () => {
319
- await manager.disposeAll();
320
- await rm(dirPath, { recursive: true, force: true });
321
- });
322
- // A seek places the new run in a directory of its own; the previous run's
323
- // segments stay servable and are served from theirs. They say nothing about
324
- // what the run now starting has done — and a run believed to be producing is
325
- // one the look-ahead may suspend and the seek path may wave through as
326
- // "already covered by the running encode".
327
- //
328
- // Deliberately a segment ABOVE the new run's start index, because that is the
329
- // case an index comparison gets wrong: after a backward seek the old run's
330
- // output sits ahead of the new run's beginning.
331
- session.runState = ENCODE_RUN_STATE.STARTING;
332
- session.encodeStartIndex = 0;
333
- session.runDirPath = path.join(dirPath, "run-7");
334
-
335
- await manager.getFileStream(SESSION_ID, "segment-00000.mp4", { requestSeq: 1 });
336
-
337
- assert.equal(session.runState, ENCODE_RUN_STATE.STARTING);
338
- });
1
+ /**
2
+ * @file A finished segment on disk must reach the viewer as bytes.
3
+ *
4
+ * The module tests cover each piece of the fMP4 path on its own, and every one
5
+ * of them passed while playback was dead: 2.9.124 called
6
+ * `readSelfContainedStartSeconds` from `fmp4.js` without importing it, and the
7
+ * unit test imports that function straight from `mp4-boxes.js`, so the gap
8
+ * between a module and its CALLER was invisible. This test asks the session
9
+ * manager for a segment that exists and insists on getting it.
10
+ *
11
+ * The second half pins the reason a one-word slip cost a whole release: the
12
+ * failure was reported as "still being produced". Measured 2026-08-08 — segment
13
+ * #0 was held for 45 281 ms with twelve finished segments in the directory, and
14
+ * the log said nothing at all. Anything that goes wrong while preparing a file
15
+ * that EXISTS must be named and answered, never turned into an endless wait.
16
+ */
17
+
18
+ import test from "node:test";
19
+ import assert from "node:assert/strict";
20
+ import { mkdtemp, rm, writeFile } from "node:fs/promises";
21
+ import os from "node:os";
22
+ import path from "node:path";
23
+ import { HlsSessionManager } from "../services/hls-session-manager.js";
24
+ import { ENCODE_RUN_STATE } from "../services/encode-run-state.js";
25
+ import { fmp4Format } from "../services/segment-formats/fmp4.js";
26
+
27
+ const MOVIE_TIMESCALE = 1000;
28
+ const VIDEO_TIMESCALE = 90_000;
29
+ const AUDIO_TIMESCALE = 48_000;
30
+ const SEGMENT_START_SECONDS = 12.5;
31
+ const SESSION_ID = "11111111-2222-3333-4444-555555555555";
32
+
33
+ /**
34
+ * @param {string} type
35
+ * @param {Buffer} body
36
+ * @returns {Buffer}
37
+ */
38
+ function box(type, body) {
39
+ const head = Buffer.alloc(8);
40
+ head.writeUInt32BE(8 + body.length, 0);
41
+ head.write(type, 4, "latin1");
42
+ return Buffer.concat([head, body]);
43
+ }
44
+
45
+ /**
46
+ * `elst` holding one empty edit — how the `segment` muxer records where the
47
+ * piece sits on the source timeline.
48
+ *
49
+ * @param {number} offsetSeconds
50
+ * @returns {Buffer}
51
+ */
52
+ function emptyEdit(offsetSeconds) {
53
+ const body = Buffer.alloc(16);
54
+ body.writeUInt32BE(1, 4); // entry count
55
+ body.writeUInt32BE(Math.round(offsetSeconds * MOVIE_TIMESCALE), 8); // duration
56
+ body.writeInt32BE(-1, 12); // media_time
57
+ return box("elst", body);
58
+ }
59
+
60
+ /**
61
+ * @param {number} trackId
62
+ * @param {number} timescale
63
+ * @param {number} offsetSeconds
64
+ * @returns {Buffer}
65
+ */
66
+ function trak(trackId, timescale, offsetSeconds) {
67
+ const tkhdBody = Buffer.alloc(84);
68
+ tkhdBody.writeUInt32BE(trackId, 12);
69
+ const mdhdBody = Buffer.alloc(20);
70
+ mdhdBody.writeUInt32BE(timescale, 12);
71
+ return box("trak", Buffer.concat([
72
+ box("tkhd", tkhdBody),
73
+ box("edts", emptyEdit(offsetSeconds)),
74
+ box("mdia", box("mdhd", mdhdBody))
75
+ ]));
76
+ }
77
+
78
+ /**
79
+ * @param {number} trackId
80
+ * @returns {Buffer}
81
+ */
82
+ function traf(trackId) {
83
+ const tfhdBody = Buffer.alloc(8);
84
+ tfhdBody.writeUInt32BE(trackId, 4);
85
+ const tfdtBody = Buffer.alloc(12);
86
+ tfdtBody.writeUInt8(1, 0); // version 1 — 64-bit
87
+ tfdtBody.writeBigUInt64BE(0n, 4); // what ffmpeg writes: zero
88
+ return box("traf", Buffer.concat([box("tfhd", tfhdBody), box("tfdt", tfdtBody)]));
89
+ }
90
+
91
+ /**
92
+ * A piece shaped like the `segment` muxer's output: header, two fragments and a
93
+ * trailing random-access index, all in one file.
94
+ *
95
+ * @param {number} offsetSeconds
96
+ * @returns {Buffer}
97
+ */
98
+ function selfContainedPiece(offsetSeconds) {
99
+ const mvhdBody = Buffer.alloc(100);
100
+ mvhdBody.writeUInt32BE(MOVIE_TIMESCALE, 12);
101
+ const moov = box("moov", Buffer.concat([
102
+ box("mvhd", mvhdBody),
103
+ trak(1, VIDEO_TIMESCALE, offsetSeconds),
104
+ trak(2, AUDIO_TIMESCALE, offsetSeconds)
105
+ ]));
106
+ const moof = box("moof", Buffer.concat([traf(1), traf(2)]));
107
+ const mdat = box("mdat", Buffer.alloc(64, 0x5a));
108
+ const mfra = box("mfra", Buffer.alloc(24, 0));
109
+ return Buffer.concat([box("ftyp", Buffer.alloc(16, 0)), moov, moof, mdat, mfra]);
110
+ }
111
+
112
+ /**
113
+ * A manager holding one session whose segments are already on disk, cut at
114
+ * explicit times — the ordinary keyframe-cut path.
115
+ *
116
+ * @param {{ segmentFormat?: object }} [overrides]
117
+ * @returns {Promise<{ manager: HlsSessionManager, session: object, dirPath: string }>}
118
+ */
119
+ async function managerWithReadySegment(overrides = {}) {
120
+ const dirPath = await mkdtemp(path.join(os.tmpdir(), "segment-serve-"));
121
+ const piece = selfContainedPiece(SEGMENT_START_SECONDS);
122
+ // Two segments, because a piece is only finished once the next one exists.
123
+ await writeFile(path.join(dirPath, "segment-00000.mp4"), piece);
124
+ await writeFile(path.join(dirPath, "segment-00001.mp4"), piece);
125
+
126
+ const manager = new HlsSessionManager({
127
+ enabled: true,
128
+ ffmpegBin: "ffmpeg",
129
+ localBindHost: "127.0.0.1",
130
+ localPort: 9090
131
+ });
132
+ const session = {
133
+ id: SESSION_ID,
134
+ dirPath,
135
+ state: "ready",
136
+ fileName: "video.mkv",
137
+ startedAt: Date.now(),
138
+ createEntryMs: Date.now(),
139
+ lastAccessedAt: Date.now(),
140
+ ffmpeg: null,
141
+ lastError: "",
142
+ consumers: new Set(),
143
+ segmentFormat: overrides.segmentFormat ?? fmp4Format,
144
+ usesExplicitCuts: true,
145
+ useSyntheticPlaylist: true,
146
+ playlistText: "#EXTM3U\n",
147
+ segmentBoundaries: [0, SEGMENT_START_SECONDS, 25],
148
+ initBytes: fmp4Format.extractInit(piece),
149
+ encodeStartIndex: 0,
150
+ firstSegmentLogged: false,
151
+ waitEpoch: 0
152
+ };
153
+ manager.sessionsById.set(SESSION_ID, session);
154
+ return { manager, session, dirPath };
155
+ }
156
+
157
+ test("serving a segment records what its real start says about the container's index", async (t) => {
158
+ const { manager, session, dirPath } = await managerWithReadySegment();
159
+ t.after(async () => {
160
+ await manager.disposeAll();
161
+ await rm(dirPath, { recursive: true, force: true });
162
+ });
163
+ // The tally is counted in the module tests; what this pins is that serving a
164
+ // segment reaches it at all. A counter nothing increments reports a clean
165
+ // index for every file forever, which is worse than no measurement.
166
+ session.indexCheck = { checked: 0, disagreed: 0, maxDeviationSec: 0, firstDisagreementIndex: -1, seen: new Set() };
167
+
168
+ await manager.getFileStream(SESSION_ID, "segment-00000.mp4", { requestSeq: 1 });
169
+
170
+ assert.equal(session.indexCheck.checked, 1, "the boundary that was just produced must have been examined");
171
+ });
172
+
173
+ test("a segment that exists is served, not reported as still being produced", async (t) => {
174
+ const { manager, dirPath, session } = await managerWithReadySegment();
175
+ t.after(async () => {
176
+ await manager.disposeAll();
177
+ await rm(dirPath, { recursive: true, force: true });
178
+ });
179
+
180
+ const result = await manager.getFileStream(SESSION_ID, "segment-00000.mp4", { requestSeq: 1 });
181
+
182
+ assert.equal(result.kind, "file", "a finished segment on disk must come back as bytes");
183
+ assert.equal(result.contentType, fmp4Format.segmentContentType);
184
+
185
+ const chunks = [];
186
+ for await (const chunk of result.stream) {
187
+ chunks.push(chunk);
188
+ }
189
+ const served = Buffer.concat(chunks);
190
+ assert.equal(served.toString("latin1", 4, 8), "moof", "the init header must be stripped off a media segment");
191
+
192
+ // The piece's OWN start, carried into the fragment it belongs to. Two
193
+ // releases tried moving it toward the playlist instead (2.24.1 per session,
194
+ // 2.25.0 by one family offset) and both desynced picture from sound in the
195
+ // field the same day: the first segment of a run is not cut, it begins where
196
+ // the seek landed, and the picture must land on a keyframe while the sound
197
+ // need not. Reading the piece's position is also the step that threw in
198
+ // 2.9.124, and it still feeds the index tally and the grid correction.
199
+ assert.equal(
200
+ Number(served.readBigUInt64BE(served.indexOf("tfdt") + 8)),
201
+ Math.round(SEGMENT_START_SECONDS * VIDEO_TIMESCALE),
202
+ "the segment must be stamped with where it really begins"
203
+ );
204
+ assert.equal(
205
+ session.indexCheck.checked,
206
+ 1,
207
+ "and the piece's own position must still have been read, or nothing measures the index"
208
+ );
209
+ });
210
+
211
+ test("a fault while preparing an existing segment is named, not turned into a wait", async (t) => {
212
+ const broken = {
213
+ ...fmp4Format,
214
+ readSegmentStartSeconds() {
215
+ throw new ReferenceError("readSelfContainedStartSeconds is not defined");
216
+ }
217
+ };
218
+ const { manager, dirPath } = await managerWithReadySegment({ segmentFormat: broken });
219
+ t.after(async () => {
220
+ await manager.disposeAll();
221
+ await rm(dirPath, { recursive: true, force: true });
222
+ });
223
+
224
+ const result = await manager.getFileStream(SESSION_ID, "segment-00000.mp4", { requestSeq: 1 });
225
+
226
+ assert.equal(
227
+ result.kind,
228
+ "failed",
229
+ "answering 'warming-up' hides the fault and holds every request until the viewer gives up"
230
+ );
231
+ assert.match(result.message, /readSelfContainedStartSeconds is not defined/);
232
+ });
233
+
234
+ test("a run's FIRST segment is served once the encoder has passed it, without waiting for a next one", async (t) => {
235
+ const { manager, session, dirPath } = await managerWithReadySegment();
236
+ t.after(async () => {
237
+ await manager.disposeAll();
238
+ await rm(dirPath, { recursive: true, force: true });
239
+ });
240
+ // The shape a resume takes: a run begun mid-file, so its first segment has no
241
+ // successor and nothing is producing one. Waiting for that successor is what
242
+ // held #317 for 46 s and then answered 404 to a browser that had given up.
243
+ await rm(path.join(dirPath, "segment-00001.mp4"));
244
+ session.encodeStartIndex = 0;
245
+ session.ffmpeg = { killed: true, kill() {} };
246
+ session.progress = { processedSeconds: SEGMENT_START_SECONDS + 10 };
247
+
248
+ const result = await manager.getFileStream(SESSION_ID, "segment-00000.mp4", { requestSeq: 1 });
249
+
250
+ assert.equal(
251
+ result.kind,
252
+ "file",
253
+ "the encoder is past this segment's end, so it is finished — the absence of a next one says nothing"
254
+ );
255
+ });
256
+
257
+ test("a segment is found in the run directory that produced it, newest run first", async (t) => {
258
+ const { manager, session, dirPath } = await managerWithReadySegment();
259
+ t.after(async () => {
260
+ await manager.disposeAll();
261
+ await rm(dirPath, { recursive: true, force: true });
262
+ });
263
+ // Runs write into a directory each — that is what lets a restart begin
264
+ // without waiting for its predecessor to die, which measured 0.7-1.3 s of
265
+ // every seek. A later run's answer supersedes an earlier one's, because the
266
+ // older file may be the truncated output of a run that was killed mid-write.
267
+ const { mkdir } = await import("node:fs/promises");
268
+ const piece = selfContainedPiece(SEGMENT_START_SECONDS);
269
+ await mkdir(path.join(dirPath, "run-1"), { recursive: true });
270
+ await mkdir(path.join(dirPath, "run-2"), { recursive: true });
271
+ await writeFile(path.join(dirPath, "run-1", "segment-00000.mp4"), Buffer.alloc(8));
272
+ await writeFile(path.join(dirPath, "run-2", "segment-00000.mp4"), piece);
273
+ await writeFile(path.join(dirPath, "run-2", "segment-00001.mp4"), piece);
274
+ await rm(path.join(dirPath, "segment-00000.mp4"));
275
+ await rm(path.join(dirPath, "segment-00001.mp4"));
276
+ session.encodeStartIndex = 0;
277
+
278
+ const result = await manager.getFileStream(SESSION_ID, "segment-00000.mp4", { requestSeq: 1 });
279
+
280
+ assert.equal(result.kind, "file", "a segment produced by a run must be found in that run's directory");
281
+ const chunks = [];
282
+ for await (const chunk of result.stream) {
283
+ chunks.push(chunk);
284
+ }
285
+ assert.ok(
286
+ Buffer.concat(chunks).length > 8,
287
+ "the newest run's output must win — the older file here is the 8-byte stub a killed run leaves"
288
+ );
289
+ });
290
+
291
+ test("serving a run's own segment moves the run out of STARTING", async (t) => {
292
+ const { manager, session, dirPath } = await managerWithReadySegment();
293
+ t.after(async () => {
294
+ await manager.disposeAll();
295
+ await rm(dirPath, { recursive: true, force: true });
296
+ });
297
+ // The table lives in `encode-run-state.js` and is tested there as a graph.
298
+ // What this pins is that a real serve REACHES it: a state nothing writes
299
+ // describes every run as still starting, for ever, and the log built on it
300
+ // would say so too.
301
+ session.runState = ENCODE_RUN_STATE.STARTING;
302
+ // Where the run in force is writing. The fixture's segments live directly in
303
+ // the session directory, which is exactly what a single run's directory is
304
+ // here.
305
+ session.runDirPath = dirPath;
306
+
307
+ await manager.getFileStream(SESSION_ID, "segment-00000.mp4", { requestSeq: 1 });
308
+
309
+ assert.equal(session.runState, ENCODE_RUN_STATE.PRODUCING);
310
+ });
311
+
312
+ test("a segment left by an earlier run does not claim the new run has produced", async (t) => {
313
+ const { manager, session, dirPath } = await managerWithReadySegment();
314
+ t.after(async () => {
315
+ await manager.disposeAll();
316
+ await rm(dirPath, { recursive: true, force: true });
317
+ });
318
+ // A seek places the new run in a directory of its own; the previous run's
319
+ // segments stay servable and are served from theirs. They say nothing about
320
+ // what the run now starting has done — and a run believed to be producing is
321
+ // one the look-ahead may suspend and the seek path may wave through as
322
+ // "already covered by the running encode".
323
+ //
324
+ // Deliberately a segment ABOVE the new run's start index, because that is the
325
+ // case an index comparison gets wrong: after a backward seek the old run's
326
+ // output sits ahead of the new run's beginning.
327
+ session.runState = ENCODE_RUN_STATE.STARTING;
328
+ session.encodeStartIndex = 0;
329
+ session.runDirPath = path.join(dirPath, "run-7");
330
+
331
+ await manager.getFileStream(SESSION_ID, "segment-00000.mp4", { requestSeq: 1 });
332
+
333
+ assert.equal(session.runState, ENCODE_RUN_STATE.STARTING);
334
+ });