@torrent-tv/proxy 2.88.6 → 2.89.1

package/assets/calibration/audio/NOTICE.md

@@ -1,19 +1,19 @@
-# Audio calibration samples
-
-These files contain deterministic synthetic pink noise, encoded at build time
-by `scripts/build-audio-samples.mjs`. They contain no recorded third-party audio.
-The manifest records the actual duration, channels, sample rate and byte count
-reported by ffprobe. They are distributed under the package's GPL-3.0-or-later
-license.
-
-Startup reads each packaged sample once for packet copying and once for AAC
-processing. It neither generates input files nor loops them. The operation
-timeout is the existing encoder benchmark timeout; it is not an ETA coefficient.
-
-Process startup is measured separately from FFmpeg's reported processing time;
-its printed clock precision bounds runtime uncertainty. Input reading is
-included in processing. For encoding, the
-reference work scales by input audio samples (channels multiplied by sample
-rate); for copying, a declared source bitrate scales encoded-byte work. These
-are approximate initial rates, replaced by actual per-output progress. Unknown
-dimensions retain the measured reference rather than fabricated metadata.
+# Audio calibration samples
+
+These files contain deterministic synthetic pink noise, encoded at build time
+by `scripts/build-audio-samples.mjs`. They contain no recorded third-party audio.
+The manifest records the actual duration, channels, sample rate and byte count
+reported by ffprobe. They are distributed under the package's GPL-3.0-or-later
+license.
+
+Startup reads each packaged sample once for packet copying and once for AAC
+processing. It neither generates input files nor loops them. The operation
+timeout is the existing encoder benchmark timeout; it is not an ETA coefficient.
+
+Process startup is measured separately from FFmpeg's reported processing time;
+its printed clock precision bounds runtime uncertainty. Input reading is
+included in processing. For encoding, the
+reference work scales by input audio samples (channels multiplied by sample
+rate); for copying, a declared source bitrate scales encoded-byte work. These
+are approximate initial rates, replaced by actual per-output progress. Unknown
+dimensions retain the measured reference rather than fabricated metadata.

package/assets/calibration/audio/manifest.json

@@ -1,98 +1,98 @@
-[
-  {
-    "codec": "aac",
-    "file": "aac.m4a",
-    "durationSeconds": 4,
-    "channels": 2,
-    "samplingFrequency": 48000,
-    "bytes": 66060
-  },
-  {
-    "codec": "ac3",
-    "file": "ac3.ac3",
-    "durationSeconds": 4,
-    "channels": 2,
-    "samplingFrequency": 48000,
-    "bytes": 96000
-  },
-  {
-    "codec": "eac3",
-    "file": "eac3.eac3",
-    "durationSeconds": 4,
-    "channels": 2,
-    "samplingFrequency": 48000,
-    "bytes": 96000
-  },
-  {
-    "codec": "dts",
-    "file": "dts.dts",
-    "durationSeconds": 4.0051,
-    "channels": 2,
-    "samplingFrequency": 48000,
-    "bytes": 706500
-  },
-  {
-    "codec": "flac",
-    "file": "flac.flac",
-    "durationSeconds": 4,
-    "channels": 2,
-    "samplingFrequency": 48000,
-    "bytes": 723158
-  },
-  {
-    "codec": "opus",
-    "file": "opus.opus",
-    "durationSeconds": 4.0065,
-    "channels": 2,
-    "samplingFrequency": 48000,
-    "bytes": 32457
-  },
-  {
-    "codec": "vorbis",
-    "file": "vorbis.ogg",
-    "durationSeconds": 4,
-    "channels": 2,
-    "samplingFrequency": 48000,
-    "bytes": 42108
-  },
-  {
-    "codec": "truehd",
-    "file": "truehd.thd",
-    "durationSeconds": null,
-    "channels": 2,
-    "samplingFrequency": 48000,
-    "bytes": 589492
-  },
-  {
-    "codec": "mp3",
-    "file": "mp3.mp3",
-    "durationSeconds": 4,
-    "channels": 2,
-    "samplingFrequency": 48000,
-    "bytes": 64748
-  },
-  {
-    "codec": "mp2",
-    "file": "mp2.mp2",
-    "durationSeconds": 4.008,
-    "channels": 2,
-    "samplingFrequency": 48000,
-    "bytes": 192384
-  },
-  {
-    "codec": "alac",
-    "file": "alac.m4a",
-    "durationSeconds": 4,
-    "channels": 2,
-    "samplingFrequency": 48000,
-    "bytes": 712871
-  },
-  {
-    "codec": "pcm",
-    "file": "pcm.wav",
-    "durationSeconds": 4,
-    "channels": 2,
-    "samplingFrequency": 48000,
-    "bytes": 768078
-  }
-]
+[
+  {
+    "codec": "aac",
+    "file": "aac.m4a",
+    "durationSeconds": 4,
+    "channels": 2,
+    "samplingFrequency": 48000,
+    "bytes": 66060
+  },
+  {
+    "codec": "ac3",
+    "file": "ac3.ac3",
+    "durationSeconds": 4,
+    "channels": 2,
+    "samplingFrequency": 48000,
+    "bytes": 96000
+  },
+  {
+    "codec": "eac3",
+    "file": "eac3.eac3",
+    "durationSeconds": 4,
+    "channels": 2,
+    "samplingFrequency": 48000,
+    "bytes": 96000
+  },
+  {
+    "codec": "dts",
+    "file": "dts.dts",
+    "durationSeconds": 4.0051,
+    "channels": 2,
+    "samplingFrequency": 48000,
+    "bytes": 706500
+  },
+  {
+    "codec": "flac",
+    "file": "flac.flac",
+    "durationSeconds": 4,
+    "channels": 2,
+    "samplingFrequency": 48000,
+    "bytes": 723158
+  },
+  {
+    "codec": "opus",
+    "file": "opus.opus",
+    "durationSeconds": 4.0065,
+    "channels": 2,
+    "samplingFrequency": 48000,
+    "bytes": 32457
+  },
+  {
+    "codec": "vorbis",
+    "file": "vorbis.ogg",
+    "durationSeconds": 4,
+    "channels": 2,
+    "samplingFrequency": 48000,
+    "bytes": 42108
+  },
+  {
+    "codec": "truehd",
+    "file": "truehd.thd",
+    "durationSeconds": null,
+    "channels": 2,
+    "samplingFrequency": 48000,
+    "bytes": 589492
+  },
+  {
+    "codec": "mp3",
+    "file": "mp3.mp3",
+    "durationSeconds": 4,
+    "channels": 2,
+    "samplingFrequency": 48000,
+    "bytes": 64748
+  },
+  {
+    "codec": "mp2",
+    "file": "mp2.mp2",
+    "durationSeconds": 4.008,
+    "channels": 2,
+    "samplingFrequency": 48000,
+    "bytes": 192384
+  },
+  {
+    "codec": "alac",
+    "file": "alac.m4a",
+    "durationSeconds": 4,
+    "channels": 2,
+    "samplingFrequency": 48000,
+    "bytes": 712871
+  },
+  {
+    "codec": "pcm",
+    "file": "pcm.wav",
+    "durationSeconds": 4,
+    "channels": 2,
+    "samplingFrequency": 48000,
+    "bytes": 768078
+  }
+]

package/docs/container-architecture.md

@@ -58,6 +58,35 @@ So ffmpeg keeps exactly three jobs, and nothing else:
 That is a final answer about the container, and the point at which a caller may
 go to the media — not "unknown, ask again".
 
+## A read that has not arrived is not an answer
+
+Every read a container makes goes through `strictReader` (`container/unavailable.js`):
+it answers with every byte asked for, or throws `BytesUnavailable`. A reading has
+therefore four outcomes — the value; a proven absence ("this file has no Cues");
+a refusal of ours, stated with its reason and size (an MP4 `moov` larger than
+this program reads whole: `moovRefused`); and "not here yet". Only the first
+three are facts about the file, and only they are kept — by the container, by
+`ContainerOrchestrator`, by `KeyframeTables` and by `SubtitleCues`.
+
+Until 2026-10-01 "not here yet" came back as `null`, the value the readers
+return for an absent element. A subtitle plan read while its Cues table was
+downloading was kept as "no clusters", and an embedded track showed nothing for
+a whole session; the keyframe table, the container choice, the track table,
+MP4's `moov` and AVI's `idx1` had the same shape
+(`research/subtitles-never-appear-2026-10-01.md`).
+
+## Elements are read where they lie
+
+Matroska's top-level elements are found by walking their HEADERS from the
+Segment's start to the first cluster, each costing only its header, and from the
+SeekHead for what lies after the clusters — so a Cues element stored before the
+clusters with no SeekHead entry (RFC 9559 §6.4) is found. Elements are then read
+by their own size through `container/ebml-stream.js`, one portion at a time; the
+portion is the torrent's piece length, handed in, never chosen here. Data that
+is not needed is skipped without being assembled; a needed element larger than
+one portion is refused by name. There is no fixed window for the Cues table,
+the Tracks element or a cluster any more.
+
 ## Where byte access lives, and why not on a track
 
 A `ContainerTrack` is a DECLARATION. It carries no `readRange` and no file
@@ -77,26 +106,52 @@ bytes — was reviewed on 2026-09-03 and is **not** the right shape:
   on the thread where the container is already at hand.
 
 **That split is closed.** `SubtitleTrack` carries `clusterPositions` and
-`samples` — byte POSITIONS — and the reading of those positions is now the
-container's: `MatroskaContainer.walkHeldClusters(plan, walked)` and
-`Mp4Container.readHeldSamples(track, harvested)`, beside `readTracks`,
-`readKeyframeIndex`, `readMediaInfo` and `cueTextOf`.
-
-The obstacle was real and this is how it was answered. The two readers want
-different read POLICIES over the same file: the track table fetches what is
-missing from the swarm, while the cue walk deliberately reads only what is
-already downloaded, so that turning subtitles on never pulls bytes the viewer is
-not waiting for. A container is now built with BOTH — `readRange` that fetches,
-`readHeld` that does not, and `isHeld` that says whether a range can be read
-without fetching — so one instance per file serves both readers and the per-file
-cache is kept.
-
-What the container is NOT given is the torrent. Those three functions are the
-whole of what the caller knows and the container does not, and reducing the
-torrent to them is what lets the reading live where the format is specified.
-`torrent-worker/subtitle-cues.js` supplies them (`containerOver`) and keeps only
-what is genuinely its own: the found-order cursor a browser follows, the
-per-file state, and one walk of a file at a time.
+`samples` — byte POSITIONS — and the reading of those positions is the
+container's: `MatroskaContainer.readHeldCues` (over `container/matroska-clusters.js`)
+and `Mp4Container.readHeldSamples`, beside `readTracks`, `readKeyframeIndex`,
+`readMediaInfo` and `cueTextOf`.
+
+The two readers want different read POLICIES over the same file: the track
+table fetches what is missing from the swarm, while the cue walk reads only
+what is already downloaded, so that turning subtitles on never pulls bytes the
+viewer is not waiting for. **One container per file serves both**: it is built
+with the fetching `readRange`, and the walk's reader of downloaded bytes — which
+ranges are whole, a strict read of them, the portion, where viewers stand — is
+handed to `readHeldCues` on every pass (`HeldReader`), because what is
+downloaded changes between passes. The walk used to build a second container of
+its own over the same file and read the head and the Cues table again, under a
+different rule for a read that had not arrived.
+
+What the container is NOT given is the torrent. `media/SubtitleCues.js` is
+handed a `HeldFile` of plain functions by `server.js` and keeps only what is
+genuinely its own: the found-order cursor a browser follows, the per-file
+state, one walk of a file at a time, and taking back the cues of a cluster the
+container withdraws.
+
+## Where a cluster is, when the Cues table does not say
+
+RFC 9559 §22.1 only recommends that each subtitle frame be referenced by the
+Cues table, and a file may have none. `container/matroska-clusters.js` finds
+clusters from the downloaded bytes: positions the file establishes (the first
+cluster, every `CueClusterPosition` of every track), a chain of top-level
+elements from each established one, and — only where the file has no Cues — a
+search for the Cluster id. What the search finds is a candidate; it is used only
+once every child is one §5.1.3 allows, exactly one Timestamp is present, the
+children fill it exactly, every block names a declared track, and any CRC-32 or
+PrevSize matches. A candidate that an established cluster turns out to contain
+is withdrawn with its cues, and the browser removes them by number. The cluster
+a viewer stands in is read first, then the one before it, and pushed at once.
+
+The walk reads a cluster's STRUCTURE — every child's header and the first bytes
+of each block, which name its track — and a subtitle block's data by its own
+size. It never moves a picture's frames into the main thread: measured on the
+addon host 2026-10-01, reading clusters a portion at a time moved every frame
+of the film across the thread boundary, and the collector then took three times
+as long as the walk. Each read also hands the loop back before the walk goes on,
+because the torrent thread replies faster than the message port empties. The
+search decides each candidate as it finds it; one accepted is followed by its
+chain, and the search resumes where the chain ends, so a file whose clusters
+follow one another is searched only up to the first cluster after each gap.
 
 ## Layers
 
@@ -166,10 +221,10 @@ flowchart TB
 - `ContainerFactory.create({readRange,fileSize})` — sniffs 16 bytes, returns precise `Container` subclass. No torrent knowledge.
 - `ContainerOrchestrator` — per-file cache (`sourceKey:fileIndex`), `getTracks()` / `getMediaInfo()` / `getKeyframeIndex()`. Transport-agnostic.
 - **The keyframe table is read once per file, and the container is not what remembers it.** `Container.readKeyframeIndex` parses and returns; each format implements `parseKeyframeIndex`. It used to keep the answer, and that made this the one fact stored in two places — here and in the file's `KeyframeTable` on the main thread. That copy had no reader, and since the parse itself moved to the main thread there is no second thread for a second copy to sit in.
-- **`KeyframeTable` / `KeyframeTables` — the answer, and the policy around getting one.** The table is one object per file (`media/container/KeyframeTable.js`), held by everyone who reads that file rather than copied, so a read that lands after a session was made still reaches it. `media/KeyframeTables.js` reads it once per file whoever asks, joins the second asker to the read already running, bounds how long any one caller waits (`KEYFRAME_TABLE_BUDGET_MS`, measured), and never records a read that threw as an answer. Its reader is one function handed in at construction, so it knows nothing of torrents or HTTP. It is not `ContainerOrchestrator` because of the THREAD: that one needs byte ranges and lives in the worker, this one lives beside the sessions and reaches it across the channel.
+- **`KeyframeTable` / `KeyframeTables` — the answer, and the policy around getting one.** The table is one object per file (`media/container/KeyframeTable.js`), held by everyone who reads that file rather than copied, so a read that lands after a session was made still reaches it. `media/KeyframeTables.js` reads it once per file whoever asks, joins the second asker to the read already running, bounds how long any one caller waits (`KEYFRAME_TABLE_BUDGET_MS`, measured), and never records a read that threw as an answer. Its reader is one function handed in at construction, so it knows nothing of torrents or HTTP. It is not `ContainerOrchestrator`: both live on the main thread, but that one keeps the file's container — which keeps its own Cues reading — and this keeps the answer the sessions hold and the wait around it. A read that has not arrived rejects with `BytesUnavailable` and is read again when pieces of the file arrive (`readAgainIfUnanswered`).
 - **`answered` and `readable` are different questions.** `answered` says a reader came back; `readable` says it came back with times. A file that answered nothing must be re-encoded for ever (MPEG-TS: 669 real keyframes, no index of any kind); a file that has not answered is a shortage of bytes off the swarm. Held as loose fields in a bag of probe results nothing told them apart, and a passing shortage was written onto the file as a property of the bytes.
 - **The second reader is the packet probe** (`media/keyframe-probe.js`): it finds keyframes by decoding, for containers that state no index. It is never waited for, and `KeyframeTable.learn` keeps the fuller answer whichever arrives second — measured 2026-08-02, a scan found 77 keyframes in 45 s without finishing against all 570 in 0.8 s from the index.
-- `SubtitleOrchestrator` — wraps `torrent-worker/subtitle-cues.js` (`planFor`, `cuesHeldFor`, `warmSubtitleCues`) behind the `ContainerTrack` abstraction. Routes depend on this, not on the worker directly. The reading itself is the containers' — that module supplies the torrent's read policy and keeps the cursor.
+- `SubtitleOrchestrator` — wraps `media/SubtitleCues.js` (`cuesHeldFor`, `warmSubtitleCues`, `subtitleTracksOf`) behind the `ContainerTrack` abstraction, handed in by `server.js`. Routes depend on this. The reading itself is the file's one container's; `SubtitleCues` keeps the cursor and the per-file state.
 - `PlaybackController` / `SubtitleController` — thin interface adapters; `routes/api/*` delegate to them, handle HTTP headers (`X-Subtitle-Language`, `X-Subtitle-Cursor`) only.
 
 ## Legacy

package/docs/playback-start-forecast.md

@@ -1,56 +1,56 @@
-# Playback start forecast
-
-For resource r, measured service is S_r(t) = rate_r * t. New observations
-replace the conditional future rate; polling cannot invent acceleration,
-exponential recovery or a new physical measurement.
-
-Source work uses the file's measured byte density, missing local byte ranges
-and the observed torrent download rate. Shared input ranges are downloaded
-once. Files on the same torrent share a queue; distinct torrents use distinct
-observed services. Local input reading is included in effective processing
-speed, rather than charged again as a separate disk or memory delay.
-
-Each output's remaining processing work subtracts progress only within its
-actual live run intervals. Initial video rates use existing codec/size/encoder
-calibration. Initial audio rates use short packaged per-codec measurements and
-sample work (encoding) or encoded bytes (copying). Actual output progress
-replaces these initial rates. Startup is charged once, separately from processing
-throughput, and disappears once actual run progress establishes processing.
-
-Input arrival and processing overlap. With arrival time D(m), processing rate
-e and previous completion C, completion for an interval [a,b] is
-
-    max(C + (b-a)/e, sup_m(D(m) + (b-m)/e)).
-
-Between service boundaries these functions are linear, so only endpoints are
-needed. Client delivery follows production and shares the observed link queue.
-Measured client ranges and prepared media establish each track's deadlines.
-
-The minimum start delay for this conditional trajectory is
-
-    max(0, max_j(completion_j - presentation_deadline_j)).
-
-This covers the remaining finite movie, including rates below real time. Slow
-positive service gives a finite preproduction delay; it is not rejected by a
-browser-capacity threshold or a capped search. Observed supply-interruption
-coverage is held in prepared proxy media and client media, without requiring
-the browser to hold it all.
-
-Critical review: future rates can change, average byte density is an initial
-mapping rather than an exact packet index, initial audio sample-work scaling
-does not model every decoder's nonlinear cost, and client receive/append time
-is not separately measured. These are approximation limits, not hidden weights.
-Actual progress, segment sizes and link observations refine the prediction.
-The client countdown uses elapsed local monotonic time; expiration cannot
-authorize playback. Only the proxy's readiness and real track coverage can.
-
-Zero service, missing media facts and a genuine timestamp gap remain explicit
-noncomputable states. The UI reports their causes instead of an estimating
-placeholder or a fabricated duration. Small coded-frame joins follow measured
-frame durations and Chromium's continuous-track rule, not a hand-picked margin.
-
-A closed non-final fragment is reusable only if its measured coded-frame
-coverage reaches its next declared cut. Interrupted one-frame fragments are
-excluded on publication and when reading an existing cache, so the production
-schedule treats their media as unfinished work. The final fragment uses its
-actual media end rather than the container's approximate duration.
+# Playback start forecast
+
+For resource r, measured service is S_r(t) = rate_r * t. New observations
+replace the conditional future rate; polling cannot invent acceleration,
+exponential recovery or a new physical measurement.
+
+Source work uses the file's measured byte density, missing local byte ranges
+and the observed torrent download rate. Shared input ranges are downloaded
+once. Files on the same torrent share a queue; distinct torrents use distinct
+observed services. Local input reading is included in effective processing
+speed, rather than charged again as a separate disk or memory delay.
+
+Each output's remaining processing work subtracts progress only within its
+actual live run intervals. Initial video rates use existing codec/size/encoder
+calibration. Initial audio rates use short packaged per-codec measurements and
+sample work (encoding) or encoded bytes (copying). Actual output progress
+replaces these initial rates. Startup is charged once, separately from processing
+throughput, and disappears once actual run progress establishes processing.
+
+Input arrival and processing overlap. With arrival time D(m), processing rate
+e and previous completion C, completion for an interval [a,b] is
+
+    max(C + (b-a)/e, sup_m(D(m) + (b-m)/e)).
+
+Between service boundaries these functions are linear, so only endpoints are
+needed. Client delivery follows production and shares the observed link queue.
+Measured client ranges and prepared media establish each track's deadlines.
+
+The minimum start delay for this conditional trajectory is
+
+    max(0, max_j(completion_j - presentation_deadline_j)).
+
+This covers the remaining finite movie, including rates below real time. Slow
+positive service gives a finite preproduction delay; it is not rejected by a
+browser-capacity threshold or a capped search. Observed supply-interruption
+coverage is held in prepared proxy media and client media, without requiring
+the browser to hold it all.
+
+Critical review: future rates can change, average byte density is an initial
+mapping rather than an exact packet index, initial audio sample-work scaling
+does not model every decoder's nonlinear cost, and client receive/append time
+is not separately measured. These are approximation limits, not hidden weights.
+Actual progress, segment sizes and link observations refine the prediction.
+The client countdown uses elapsed local monotonic time; expiration cannot
+authorize playback. Only the proxy's readiness and real track coverage can.
+
+Zero service, missing media facts and a genuine timestamp gap remain explicit
+noncomputable states. The UI reports their causes instead of an estimating
+placeholder or a fabricated duration. Small coded-frame joins follow measured
+frame durations and Chromium's continuous-track rule, not a hand-picked margin.
+
+A closed non-final fragment is reusable only if its measured coded-frame
+coverage reaches its next declared cut. Interrupted one-frame fragments are
+excluded on publication and when reading an existing cache, so the production
+schedule treats their media as unfinished work. The final fragment uses its
+actual media end rather than the container's approximate duration.