@torrent-tv/proxy 2.54.0 → 2.55.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,3 +1,19 @@
1
+ ## 2.55.1
2
+
3
+ - **Fix**: A source is now keyed by its own infohash, not by a hash of the request bytes. A magnet URI and a `.torrent` file for the same content are different bytes, so the old key (`sha1` of the source) named the same film as two unrelated sources — measured 2026-08-19, `a518ff46…` and `7ab2fb5d…` for one infohash `11f09299…` — sharing neither the swarm, nor a cache, nor any work already downloaded, and surfacing as `WebTorrent client error: Cannot add duplicate torrent`. The new key (`services/torrent-source-key.js`) reads the infohash straight out of the magnet's `btih` or the `.torrent`'s own `info` dictionary via `parse-torrent`, synchronously, with no network round trip — so both forms of the same torrent now share one entry from the first request, on both the pool's own map (`torrent-pool.js`) and the worker-thread boundary (`torrent-worker/pool-adapter.js`), instead of relying on WebTorrent's own duplicate-add error to reconcile them after the fact.
4
+ - **Chore**: The decode-cost reading now logs how many MB/s the pipe was actually fed against how many MB/s the measured speed implies were needed (item 4(d2)) — a divergence between the two is worth a second look before trusting the reading. A `write()`-return-value signal was tried first, to say outright whether the pipe or the decoder was the limit, and measured false on every reading taken while building it — including clips this host decodes at 15-80x with slack to spare — so it does not discriminate and was not shipped; only the byte count, which is real, is kept.
5
+
6
+ ## 2.55.0
7
+
8
+ - **Fix**: The decode calibration was measuring the loop rather than the decode. It looped each clip with `-stream_loop -1`, and a loop is not free: measured 2026-08-22, a restart costs **0.03 s on the 480p clip and 0.12 s on the 1080p one** — it scales with the picture, so it is the decoder tearing down and re-allocating its frame buffers rather than anything about reading the file. A five-second clip decoded at 55x restarts eleven times a second, and that cost dominated the reading: the same clips measured 53.7x looped against 80.3x in one continuous pass, and 11.8x against 15.8x. Worse, the bias depends on BOTH the clip's own resolution and the host's speed — the two axes the fit exists to separate — so it did not cancel out, it tilted the fit. That is the fast-host failure recorded on 2026-08-20, where a desktop read 1080p at 9.35 Mbit/s as cheaper than 720p at 9.94, an ordering no decoder produces, and the H.264 fit refused to solve at all. A host that could not fit H.264 got no decode figure whatsoever, which is exactly the host most able to serve.
9
+ - **New**: The clip is fed to the decoder as ONE Annex-B elementary stream, written to its stdin over and over. Parameter sets are inline in Annex-B and it can be joined by plain byte concatenation — that is what a broadcast is — so more bytes are simply more stream: nothing re-opens, nothing re-initialises, and there is no restart inside the measured window. The lift out of the container is a copy, not a re-encode, and it goes straight to a pipe: no temporary media is written at any point. Error against the continuous-pass truth is now −0.2 % and −5.5 %, with the readings spread 2-6 %, against −25 % and −33 % for the loop. Every reading on the developer's desktop moved, by up to 66 %, and they are monotonic in both axes for the first time.
10
+ - **New**: The measured window is half a second instead of one. What used to make a long window necessary was the clip restarting inside it; with the stream continuous, the only thing left to average over is the timing jitter of two progress lines, which is milliseconds. Measured at half a second: −3.0 % and +0.6 %.
11
+ - **Fix**: The contention penalty was wrong for the same reason. It compares a decode alone against the same decode beside an encoder, and both readings carried the loop, but not equally — the machine's speed differs between them. Decoding alone now reads 77.5x where it read about 53x.
12
+ - **Chore**: A codec family's clips are lifted out of their containers in ONE ffmpeg run, and the startup is shorter than before rather than longer. The lift is a copy, so its cost is almost entirely the process: one per clip added 11 s here, and running them concurrently did not help — six at once took 4.75 s against 0.89 s for one, so the machine serialises them. One run with many inputs and many outputs costs one process. Measured end to end on the same desktop, alternating old and new: **35.3 s before, 31.6 s after**, with the readings corrected. The contention benchmark lifts its clip once and decodes the same bytes three times instead of lifting it again for each reading. Done before the measurements and never beside them: a remux running next to a decode is a second job on the machine, and this benchmark exists to find out what one job costs.
13
+ - **Fix**: The lift has a time bound and is killed on it. It was the only ffmpeg run in this file without one, and it is awaited before the proxy's tunnel opens — so a remux that never exited was a startup that never finished, with nothing said. Its failures, and the decode's, now carry ffmpeg's own last line instead of "said nothing".
14
+ - **Fix**: A codec family with no Annex-B mapping fails by name instead of being lifted with H.264's filter. This matters for what comes next: AV1 has no Annex-B form at all — its packaging is OBU — and MPEG-2 and VC-1 have no `*_mp4toannexb` filter, so all three of the families the roadmap plans need another route through here, and finding that out as "the clip failed" would send the reader after the clip.
15
+ - **Chore**: `test/decode-measurement.test.js` states the orderings the readings must have — a bigger picture costs more than a smaller one at the same bitrate, a thicker stream more than a thin one at the same size, HEVC more than H.264 — rather than any number, since the numbers belong to whatever machine runs them. Those are the properties the loop inverted, and nothing was checking them.
16
+
1
17
  ## 2.54.0
2
18
 
3
19
  - **Fix**: The automatic quality step no longer changes the SIZE of the picture underneath a session the browser is already decoding. The fMP4 init segment is fetched once — a player reads `#EXT-X-MAP` and never asks again — and `avc1` keeps SPS and PPS in it rather than in the fragments, so every fragment produced after a size change was decoded against parameter sets describing a picture that was no longer being made. Measured 2026-08-21 across five viewing attempts: both re-encoded sessions of the five were destroyed by it. On `LXH-12.TS` the encoder left 1280x720 for 960x540 at 13:30:06 and the browser went on reporting `decode … size=1280x720` for the next three and a half minutes — 67 readings, not one of them 960x540 — while the viewer watched a band of macroblock garbage over a smeared field. On `c0930.com_chijyo0073.wmv` the same act at 13:35:36 made the element error on the first mismatched fragment, close the MediaSource, throw `bufferAppendError InvalidStateError` on both tracks and sit at `size=0x0 readyState=0` for four and a half minutes. Which of the two happens is the decoder's choice, not ours, and no layer reported an error either time. A change of resolution is a change of VARIANT, as the standard has it: every height is already published in the master with its own init, so the proxy now ASKS the browser to move — the same act the manual menu performs, which has never had this fault.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@torrent-tv/proxy",
3
- "version": "2.54.0",
3
+ "version": "2.55.1",
4
4
  "description": "Torrent proxy client that exposes webseed-like HTTP stream endpoint.",
5
5
  "license": "GPL-3.0-or-later",
6
6
  "publishConfig": {
@@ -35,6 +35,7 @@
35
35
  "franc": "^6.2.0",
36
36
  "get-port": "^7.1.0",
37
37
  "node-datachannel": "^0.32.0",
38
+ "parse-torrent": "^11.0.23",
38
39
  "uint8-util": "2.2.6",
39
40
  "webtorrent": "2.8.5",
40
41
  "werift": "^0.24.2",
@@ -0,0 +1,55 @@
1
+ # Does the preset benchmark's pixels/sec figure hold at a different size? (2026-08-22)
2
+
3
+ Roadmap item 4(e): the software-preset benchmark (`benchmarkSoftwarePresets`,
4
+ `services/hwaccel.js`) measures encoder throughput at one fixed reference size —
5
+ 640×360 — and every later use of that figure (deciding which quality rungs this
6
+ host can sustain) scales it by pixel count, assuming cost per pixel is constant
7
+ regardless of frame size. This was never checked against a second size.
8
+
9
+ ## Measurement
10
+
11
+ Same clip (`cal-h264-1080-hi.mp4`), same `libx264` presets, same measurement
12
+ method as the shipped benchmark (slope between two `-progress` reports,
13
+ excluding startup) — run once at the existing 640×360 reference and once at
14
+ 1280×720 (4× the pixel count), on the developer's desktop:
15
+
16
+ | preset | 640×360 | 1280×720 | ratio (should be ~1.0) |
17
+ |-----------|----------------|----------------|-------------------------|
18
+ | ultrafast | 1322.8 Mpx/s | 588.6 Mpx/s | 0.445 |
19
+ | fast | 176.6 Mpx/s | 128.1 Mpx/s | 0.725 |
20
+
21
+ Script: ad-hoc, not committed (reused the benchmark's own ffmpeg arguments at a
22
+ second `-vf scale=`/`-s` size; available on request if this needs re-running).
23
+
24
+ ## Reading
25
+
26
+ The assumption does not hold. Cost per pixel RISES with frame size on this
27
+ host — by 2.25× for `ultrafast`, 1.38× for `fast`, at 4× the pixel count — and
28
+ the effect is preset-dependent, not a single constant. A quality ladder priced
29
+ from the 640×360 figure alone over-states what this host can sustain at 1080p,
30
+ worse for the faster presets that the ladder falls back to under load.
31
+
32
+ ## Why this does not become a code change yet
33
+
34
+ One host, one clip, two sizes. [[feedback_pool_diversity]] — a correction fitted
35
+ to this desktop and applied to every proxy in the pool would be exactly the
36
+ mistake that memory exists to prevent: the pool spans ARM boards (HA Yellow /
37
+ CM4) with different cache sizes and memory bandwidth, where a size penalty of
38
+ this shape could be smaller, larger, or absent. [[feedback_no_fudged_numbers]] —
39
+ publishing 0.445/0.725 as universal constants is fabricating a number for hosts
40
+ that were never measured.
41
+
42
+ The benchmark already measures per-host (item 4(f), 4(h)); a size-scaling
43
+ correction belongs there too — the benchmark itself gains a second, cheap
44
+ measurement point (one preset, one second size) and derives its own exponent
45
+ from that host's two readings, the same way `fitDecodeCost` derives its terms
46
+ from clips rather than from a stated formula. That is the remaining work under
47
+ item 4(e); this note is the measurement that justifies doing it, not the fix
48
+ itself.
49
+
50
+ ## What is verified
51
+
52
+ The measurement method matches the shipped benchmark exactly (real footage,
53
+ raw frames decoded once, slope between two progress reports) — the only
54
+ variable changed was the target size, so this is not comparing two different
55
+ methodologies.
@@ -22,6 +22,7 @@
22
22
 
23
23
  import { spawn } from "node:child_process";
24
24
  import { mkdtempSync, readdirSync, rmSync, statSync } from "node:fs";
25
+ import { mkdtemp, readFile, rm } from "node:fs/promises";
25
26
  import os from "node:os";
26
27
  import path from "node:path";
27
28
  import { fitDecodeCost } from "./decode-cost-fit.js";
@@ -763,10 +764,20 @@ const CALIBRATION_SETS = {
763
764
  const CALIBRATION_CLIPS = CALIBRATION_SETS.h264;
764
765
  const CALIBRATION_DIR = path.join(path.dirname(fileURLToPath(import.meta.url)), "..", "assets", "calibration");
765
766
  // How wide the measured window must be before the slope is trusted, and how
766
- // long to wait for it at most. A second of decoding is thousands of frames on a
767
- // quick host and dozens on a weak one; both give a slope, and neither costs the
768
- // startup more than a second per clip.
769
- const DECODE_WINDOW_MIN_SEC = 1;
767
+ // long to wait for it at most.
768
+ //
769
+ // Half a second, and it is the TIMING noise that sets it rather than the amount
770
+ // of video: the slope is output time against wall time, both read from the same
771
+ // two progress lines, and the jitter in stamping one is milliseconds — so half
772
+ // a second of window is a fraction of a percent of error on any host. What used
773
+ // to make a longer window necessary was the clip restarting inside it, and that
774
+ // is gone: the stream is continuous now. Measured 2026-08-22 against the
775
+ // continuous-pass truth on a desktop: -3.0 % and +0.6 % at half a second, with
776
+ // the readings spread 2-6 %, against -25 % and -33 % for the loop it replaces.
777
+ // Half a second also costs the startup about 0.6 s per clip less, which matters
778
+ // because every clip of every codec family is paid for before any viewer
779
+ // exists.
780
+ const DECODE_WINDOW_MIN_SEC = 0.5;
770
781
  const DECODE_WINDOW_MAX_MS = 8000;
771
782
 
772
783
  /**
@@ -844,8 +855,18 @@ export async function benchmarkContention({ ffmpegBin, logger, clipsDir = CALIBR
844
855
  // company, not the clip's own cost, so the smallest one says it soonest.
845
856
  const clip = path.join(clipsDir, "cal-h264-480-lo.mp4");
846
857
  const startedAt = Date.now();
847
- const alone = await measureDecodeSlope(ffmpegBin, clip);
848
- if (!alone) {
858
+ // Lifted once and decoded three times from the same bytes. Going through
859
+ // `measureDecodeSlope` lifted it again for every reading — three process
860
+ // starts on a path that is awaited before the proxy's tunnel opens, for a
861
+ // remux whose result had not changed.
862
+ const streams = await extractFamilyStreams(ffmpegBin, [clip], "h264");
863
+ const stream = streams?.[0];
864
+ if (!stream) {
865
+ log.warn("hwaccel: contention could not be measured; costs will be added as though jobs were independent");
866
+ return null;
867
+ }
868
+ const alone = await decodePipedStream(ffmpegBin, stream, log);
869
+ if (!alone?.speed) {
849
870
  log.warn("hwaccel: contention could not be measured; costs will be added as though jobs were independent");
850
871
  return null;
851
872
  }
@@ -871,8 +892,8 @@ export async function benchmarkContention({ ffmpegBin, logger, clipsDir = CALIBR
871
892
  await new Promise((resolve) => {
872
893
  setTimeout(resolve, 2_000);
873
894
  });
874
- const withCompany = await measureDecodeSlope(ffmpegBin, clip);
875
- if (withCompany) {
895
+ const withCompany = await decodePipedStream(ffmpegBin, stream, log);
896
+ if (withCompany?.speed) {
876
897
  beside.push({ others, speed: withCompany.speed });
877
898
  }
878
899
  }
@@ -947,10 +968,29 @@ async function fitOneFamily({ ffmpegBin, log, clipsDir, family, clips }) {
947
968
  const startedAllAt = Date.now();
948
969
  /** @type {Array<{ megapixelsPerSecond: number, megabitsPerSecond: number, costSecondsPerSecond: number }>} */
949
970
  const samples = [];
950
- for (const clip of clips) {
951
- const measured = await measureDecodeSlope(ffmpegBin, path.join(clipsDir, clip));
952
- if (!measured) {
953
- log.warn(`hwaccel: decode benchmark "${clip}" failed or said nothing; ${family} not measured`);
971
+ // Every clip of the family is lifted out of its container FIRST, in one
972
+ // ffmpeg run. See `extractFamilyStreams` for why one run rather than one per
973
+ // clip, and why before the measurements rather than beside them.
974
+ const streams = await extractFamilyStreams(
975
+ ffmpegBin,
976
+ clips.map((clip) => path.join(clipsDir, clip)),
977
+ family
978
+ );
979
+ if (!streams) {
980
+ log.warn(
981
+ `hwaccel: ${family} cannot be lifted out of its container — no Annex-B filter is mapped for it, ` +
982
+ `so its clips were never measured`
983
+ );
984
+ return null;
985
+ }
986
+ for (const [index, clip] of clips.entries()) {
987
+ const stream = streams[index];
988
+ const measured = stream ? await decodePipedStream(ffmpegBin, stream, log) : null;
989
+ if (!measured?.speed) {
990
+ log.warn(
991
+ `hwaccel: decode benchmark "${clip}" said nothing; ${family} not measured` +
992
+ (measured?.error ? ` — ${measured.error}` : " — the clip could not be lifted out of its container")
993
+ );
954
994
  return null;
955
995
  }
956
996
  const cost = 1 / measured.speed;
@@ -984,88 +1024,372 @@ async function fitOneFamily({ ffmpegBin, log, clipsDir, family, clips }) {
984
1024
  }
985
1025
 
986
1026
  /**
987
- * Measure how fast this host DECODES a clip, from ffmpeg’s own report of how
1027
+ * The bitstream filter and demuxer that turn a clip's video track into a
1028
+ * continuous elementary stream, by codec family.
1029
+ *
1030
+ * H.264 and HEVC in MP4 keep their parameter sets in the container's `avcC` /
1031
+ * `hvcC` and their access units length-prefixed; Annex-B carries them inline,
1032
+ * with start codes, which is what makes plain byte concatenation a valid
1033
+ * stream. That is the property this whole measurement rests on.
1034
+ */
1035
+ const ANNEX_B_BY_FAMILY = {
1036
+ h264: { filter: "h264_mp4toannexb", demuxer: "h264" },
1037
+ hevc: { filter: "hevc_mp4toannexb", demuxer: "hevc" },
1038
+ hevc10: { filter: "hevc_mp4toannexb", demuxer: "hevc" }
1039
+ };
1040
+
1041
+ /**
1042
+ * The last complaint in an ffmpeg stderr, for a line that has to say why.
1043
+ *
1044
+ * @param {string} stderr
1045
+ * @returns {string}
1046
+ */
1047
+ function lastErrorLine(stderr) {
1048
+ const lines = String(stderr ?? "")
1049
+ .split(/\r?\n/)
1050
+ .map((line) => line.trim())
1051
+ .filter((line) => line.length > 0);
1052
+ return lines[lines.length - 1] ?? "";
1053
+ }
1054
+
1055
+ /**
1056
+ * How long the lift may take before it is abandoned. It is a remux of a few
1057
+ * megabytes, so this is not a budget — it is the difference between a startup
1058
+ * that reports a failure and one that never finishes. Every other ffmpeg run in
1059
+ * this file has such a bound; this one did not, and it is awaited before the
1060
+ * proxy's tunnel opens.
1061
+ */
1062
+ const EXTRACT_TIMEOUT_MS = 20_000;
1063
+
1064
+ /**
1065
+ * Lift a whole family's clips out of their containers, as Annex-B elementary
1066
+ * streams, in ONE ffmpeg run.
1067
+ *
1068
+ * No re-encoding — the frames are copied — so the work itself is trivial and
1069
+ * the cost is almost entirely the process. Doing one process per clip added
1070
+ * 11 s to the startup here (fourteen clips at about 0.83 s each), and running
1071
+ * them concurrently did not help: six at once took 4.75 s against 0.89 s for
1072
+ * one, so the machine serialises them. One run with many inputs and many
1073
+ * outputs costs one process.
1074
+ *
1075
+ * The outputs go to temporary files because several outputs cannot share one
1076
+ * pipe; they are read into memory and deleted immediately, and nothing about
1077
+ * this measurement is kept between runs.
1078
+ *
1079
+ * Before the measurements, never beside them: a remux running next to a decode
1080
+ * is a second job on the machine, and this benchmark exists to find out what
1081
+ * ONE job costs here.
1082
+ *
1083
+ * @param {string} ffmpegBin
1084
+ * @param {string[]} clipPaths
1085
+ * @param {string} family
1086
+ * @returns {Promise<Array<{ bytes: Buffer, demuxer: string, megapixelsPerSecond: number, megabitsPerSecond: number, fps: number } | null> | null>}
1087
+ * One entry per clip, in order; null when the family cannot be lifted at all.
1088
+ */
1089
+ async function extractFamilyStreams(ffmpegBin, clipPaths, family) {
1090
+ const shape = ANNEX_B_BY_FAMILY[family];
1091
+ // A family with no mapping is a hard failure, not a silent fallback to
1092
+ // H.264's filter. AV1 has no Annex-B form at all (its packaging is OBU), and
1093
+ // MPEG-2 and VC-1 have no `*_mp4toannexb` filter — so the three families the
1094
+ // roadmap plans next cannot come through here, and finding that out as
1095
+ // "the clip failed" would send the reader after the clip.
1096
+ if (!shape) {
1097
+ return null;
1098
+ }
1099
+ const workDir = await mkdtemp(path.join(os.tmpdir(), "ttv-calibration-"));
1100
+ const outputs = clipPaths.map((_, index) => path.join(workDir, `stream-${index}.${shape.demuxer}`));
1101
+ /** @type {string[]} */
1102
+ const args = ["-hide_banner", "-loglevel", "info", "-nostats", "-y"];
1103
+ for (const clipPath of clipPaths) {
1104
+ args.push("-i", clipPath);
1105
+ }
1106
+ for (const [index, output] of outputs.entries()) {
1107
+ args.push("-map", `${index}:v:0`, "-c:v", "copy", "-bsf:v", shape.filter, "-f", shape.demuxer, output);
1108
+ }
1109
+ const stderr = await runCapturingStderr(ffmpegBin, args, EXTRACT_TIMEOUT_MS);
1110
+ try {
1111
+ if (stderr === null) {
1112
+ return null;
1113
+ }
1114
+ // One banner block per input, in the order they were given. Read rather
1115
+ // than declared, so replacing a clip cannot silently invalidate the fit
1116
+ // that rests on it.
1117
+ const blocks = splitInputBlocks(stderr, clipPaths.length);
1118
+ return await Promise.all(clipPaths.map(async (_, index) => {
1119
+ const block = blocks[index];
1120
+ if (!block) {
1121
+ return null;
1122
+ }
1123
+ const clipInfo = parseClipCharacteristics(block);
1124
+ const fps = parseFfmpegVideoFps(block);
1125
+ if (!clipInfo || !(fps > 0)) {
1126
+ return null;
1127
+ }
1128
+ let bytes;
1129
+ try {
1130
+ bytes = await readFile(outputs[index]);
1131
+ } catch {
1132
+ return null;
1133
+ }
1134
+ if (bytes.length === 0) {
1135
+ return null;
1136
+ }
1137
+ return {
1138
+ bytes,
1139
+ demuxer: shape.demuxer,
1140
+ megapixelsPerSecond: clipInfo.megapixelsPerSecond,
1141
+ megabitsPerSecond: clipInfo.megabitsPerSecond,
1142
+ fps
1143
+ };
1144
+ }));
1145
+ } finally {
1146
+ await rm(workDir, { recursive: true, force: true }).catch(() => {});
1147
+ }
1148
+ }
1149
+
1150
+ /**
1151
+ * The part of an ffmpeg banner describing each input, in order.
1152
+ *
1153
+ * ffmpeg prints one `Input #N, …` block per input and then the stream mapping;
1154
+ * the parsers here read a single input's facts, so they are given a single
1155
+ * input's text rather than the whole banner.
1156
+ *
1157
+ * @param {string} stderr
1158
+ * @param {number} count
1159
+ * @returns {string[]}
1160
+ */
1161
+ function splitInputBlocks(stderr, count) {
1162
+ /** @type {string[]} */
1163
+ const blocks = [];
1164
+ for (let index = 0; index < count; index += 1) {
1165
+ const from = stderr.indexOf(`Input #${index},`);
1166
+ if (from < 0) {
1167
+ blocks.push("");
1168
+ continue;
1169
+ }
1170
+ const nextInput = stderr.indexOf(`Input #${index + 1},`, from);
1171
+ const mapping = stderr.indexOf("Stream mapping:", from);
1172
+ const ends = [nextInput, mapping].filter((at) => at > from);
1173
+ blocks.push(stderr.slice(from, ends.length > 0 ? Math.min(...ends) : stderr.length));
1174
+ }
1175
+ return blocks;
1176
+ }
1177
+
1178
+ /**
1179
+ * Run ffmpeg to completion and return its stderr, or null when it failed or
1180
+ * outlasted its bound.
1181
+ *
1182
+ * @param {string} ffmpegBin
1183
+ * @param {string[]} args
1184
+ * @param {number} timeoutMs
1185
+ * @returns {Promise<string | null>}
1186
+ */
1187
+ function runCapturingStderr(ffmpegBin, args, timeoutMs) {
1188
+ return new Promise((resolve) => {
1189
+ let stderr = "";
1190
+ let settled = false;
1191
+ let child;
1192
+ const settle = (value) => {
1193
+ if (settled) {
1194
+ return;
1195
+ }
1196
+ settled = true;
1197
+ clearTimeout(timer);
1198
+ try {
1199
+ child?.kill("SIGKILL");
1200
+ } catch {
1201
+ // already gone
1202
+ }
1203
+ resolve(value);
1204
+ };
1205
+ const timer = setTimeout(() => settle(null), timeoutMs);
1206
+ try {
1207
+ child = spawn(ffmpegBin, args, { stdio: ["ignore", "ignore", "pipe"], windowsHide: true });
1208
+ } catch {
1209
+ settle(null);
1210
+ return;
1211
+ }
1212
+ child.stderr.on("data", (chunk) => {
1213
+ stderr += String(chunk);
1214
+ });
1215
+ child.on("error", () => settle(null));
1216
+ child.on("close", (code) => settle(code === 0 ? stderr : null));
1217
+ });
1218
+ }
1219
+
1220
+ /**
1221
+ * Measure how fast this host DECODES a clip, from ffmpeg's own report of how
988
1222
  * much video it has processed.
989
1223
  *
990
- * Wall-clock around the process cannot answer this: starting ffmpeg costs about
991
- * a second, and on a quick machine a five-second clip decodes in a tenth of
992
- * that, so the measurement would be of the program starting. Progress lines
993
- * arrive twice a second AFTER it has started, and the slope between two of them
994
- * video processed against time taken contains no part of the startup by
995
- * construction.
1224
+ * Two things are deliberately outside the measurement.
1225
+ *
1226
+ * **The process starting.** Wall-clock around the process cannot answer this:
1227
+ * starting ffmpeg costs about a second, and on a quick machine a five-second
1228
+ * clip decodes in a tenth of that, so the measurement would be of the program
1229
+ * starting. Progress lines arrive AFTER it has started, and the slope between
1230
+ * two of them — video processed against time taken — contains no part of the
1231
+ * startup by construction.
1232
+ *
1233
+ * **The clip restarting.** This used to loop the clip with `-stream_loop -1`,
1234
+ * and a loop is not free: measured 2026-08-22 on a desktop, a restart costs
1235
+ * 0.03 s on the 480p clip and 0.12 s on the 1080p one — the decoder tearing
1236
+ * down and re-allocating its frame buffers, which is why the price rises with
1237
+ * the picture. A five-second clip decoded at 55x restarts eleven times a
1238
+ * second, so that cost DOMINATED the reading: the same clips measured 53.7x
1239
+ * looped against 80.3x in one continuous pass, and 11.8x against 15.8x. Worse,
1240
+ * the bias is not shared — it depends on the clip's own resolution and on how
1241
+ * fast the host is — so it does not cancel out of the fit, it tilts it. That is
1242
+ * the fast-host failure recorded on 2026-08-20, where 1080p read cheaper than
1243
+ * 720p, which is not a thing a decoder does.
996
1244
  *
997
- * The clip is looped forever and the process killed as soon as the window is
998
- * wide enough, so the cost is bounded by the clock rather than by the clip:
999
- * roughly a second of measurement on any host, quick or slow.
1245
+ * So the clip is fed to the decoder as ONE stream instead. An Annex-B
1246
+ * elementary stream carries its parameter sets inline, so writing the same
1247
+ * bytes again is simply more stream the decoder never re-initialises, and
1248
+ * there is no restart inside the window to measure. Verified against the
1249
+ * continuous-pass truth on the same host: -0.2 % and -5.5 %, against -25 % and
1250
+ * -33 % for the loop. Nothing is written to disk and the process is killed as
1251
+ * soon as the window is wide enough.
1252
+ *
1253
+ * Exported because the property that broke here is checkable and was not being
1254
+ * checked: a bigger picture must cost more than a smaller one of the same
1255
+ * bitrate, and under the loop it did not.
1000
1256
  *
1001
1257
  * @param {string} ffmpegBin
1002
1258
  * @param {string} clipPath
1259
+ * @param {string} [family="h264"]
1260
+ * @returns {Promise<{ speed: number, windowSec: number, megapixelsPerSecond: number, megabitsPerSecond: number } | null>}
1261
+ */
1262
+ export async function measureDecodeSlope(ffmpegBin, clipPath, family = "h264") {
1263
+ const streams = await extractFamilyStreams(ffmpegBin, [clipPath], family);
1264
+ const stream = streams?.[0];
1265
+ if (!stream) {
1266
+ return null;
1267
+ }
1268
+ const measured = await decodePipedStream(ffmpegBin, stream);
1269
+ return measured?.speed ? measured : null;
1270
+ }
1271
+
1272
+ /**
1273
+ * Decode an elementary stream fed from memory, and report the slope.
1274
+ *
1275
+ * @param {string} ffmpegBin
1276
+ * @param {{ bytes: Buffer, demuxer: string, megapixelsPerSecond: number, megabitsPerSecond: number, fps: number }} stream
1003
1277
  * @returns {Promise<{ speed: number, windowSec: number, megapixelsPerSecond: number, megabitsPerSecond: number } | null>}
1004
1278
  */
1005
- function measureDecodeSlope(ffmpegBin, clipPath) {
1279
+ function decodePipedStream(ffmpegBin, stream, log = { info: () => {}, warn: () => {} }) {
1006
1280
  return new Promise((resolve) => {
1007
1281
  const args = [
1008
- "-hide_banner", "-loglevel", "info", "-nostats",
1009
- "-stream_loop", "-1",
1010
- "-i", clipPath,
1282
+ "-hide_banner", "-loglevel", "error", "-nostats",
1283
+ // A raw stream states no frame rate, so the one the container declared is
1284
+ // given back to it. It decides how output time advances, and therefore
1285
+ // what "seconds of video per second of clock" means.
1286
+ "-f", stream.demuxer, "-framerate", String(stream.fps), "-i", "pipe:0",
1011
1287
  "-an", "-f", "null", "-",
1012
1288
  "-progress", "pipe:1"
1013
1289
  ];
1014
1290
  /** @type {Array<{ wallSec: number, outSec: number }>} */
1015
1291
  const samples = [];
1016
- let stderr = "";
1017
1292
  let stdout = "";
1293
+ // Kept because this path depends on three things the old one did not: the
1294
+ // raw demuxer accepting the frame rate, the bitstream filter having
1295
+ // produced something parsable, and the fed concatenation being decodable.
1296
+ // Without it the only trace of any of those failing is "said nothing".
1297
+ let stderr = "";
1018
1298
  let settled = false;
1019
1299
  let child;
1020
1300
  const startedAt = Date.now();
1301
+ // Whether the FEED, not the decoder, could be what this reading measures
1302
+ // (item 4(d2)). `child.stdin.write()` returning false was tried as the
1303
+ // signal — never once true would mean this process was never ahead of the
1304
+ // pipe — and measured false on every reading taken while writing this,
1305
+ // including clips this same host decodes at 15-80x with room to spare, so
1306
+ // it does not discriminate: `write()`'s return value tracks Node's own
1307
+ // internal watermark against the size of what was just handed to it, not
1308
+ // real drain state, and answered "no slack" identically whether the pipe
1309
+ // or the decoder was the true limit. Rather than publish a verdict that
1310
+ // reads the same in both cases, only the byte count is kept, for the
1311
+ // MB/s figure below — a number to read, not a boolean to trust.
1312
+ let bytesWritten = 0;
1021
1313
  const finish = () => {
1022
1314
  if (settled) {
1023
1315
  return;
1024
1316
  }
1025
1317
  settled = true;
1026
1318
  clearTimeout(timer);
1319
+ try {
1320
+ child?.stdin?.destroy();
1321
+ } catch {
1322
+ // already gone
1323
+ }
1027
1324
  try {
1028
1325
  child?.kill("SIGKILL");
1029
1326
  } catch {
1030
1327
  // already gone
1031
1328
  }
1032
- // The first sample is the one that still carries the startup — it reports
1033
- // whatever was processed while the process was coming up. Everything is
1034
- // measured from the second onwards.
1329
+ // The first sample still carries the startup — it reports whatever was
1330
+ // processed while the process was coming up. Everything is measured from
1331
+ // the second onwards.
1035
1332
  const first = samples[1];
1036
1333
  const last = samples[samples.length - 1];
1037
- const clipInfo = parseClipCharacteristics(stderr);
1038
- if (!first || !last || !clipInfo) {
1039
- resolve(null);
1334
+ if (!first || !last) {
1335
+ resolve({ error: lastErrorLine(stderr) || "the decoder reported no progress" });
1040
1336
  return;
1041
1337
  }
1042
1338
  const windowSec = last.wallSec - first.wallSec;
1043
1339
  const producedSec = last.outSec - first.outSec;
1044
1340
  if (!(windowSec >= DECODE_WINDOW_MIN_SEC) || !(producedSec > 0)) {
1045
- resolve(null);
1341
+ resolve({ error: lastErrorLine(stderr) || `the window was ${windowSec.toFixed(2)}s of ${producedSec.toFixed(2)}s produced` });
1046
1342
  return;
1047
1343
  }
1344
+ const speed = producedSec / windowSec;
1345
+ // Diagnostic only — logged, not acted on. The required rate is what
1346
+ // THIS reading implies was needed to keep the decoder fed; the achieved
1347
+ // rate is what this process actually pushed over its whole run (a wider
1348
+ // span than `windowSec`, since the feed starts before the first kept
1349
+ // sample — a looser figure is enough for a sanity comparison).
1350
+ const elapsedSec = (Date.now() - startedAt) / 1000;
1351
+ const achievedMBps = elapsedSec > 0 ? bytesWritten / elapsedSec / 1e6 : 0;
1352
+ const requiredMBps = ((stream.megabitsPerSecond * 1e6) / 8) * speed / 1e6;
1353
+ log.info(
1354
+ `hwaccel: decode pipe fed ${achievedMBps.toFixed(1)} MB/s, ${requiredMBps.toFixed(1)} MB/s ` +
1355
+ `needed for ${speed.toFixed(2)}x — a reading where these are far apart is worth a second look`
1356
+ );
1048
1357
  resolve({
1049
- speed: producedSec / windowSec,
1358
+ speed,
1050
1359
  windowSec,
1051
- megapixelsPerSecond: clipInfo.megapixelsPerSecond,
1052
- megabitsPerSecond: clipInfo.megabitsPerSecond
1360
+ megapixelsPerSecond: stream.megapixelsPerSecond,
1361
+ megabitsPerSecond: stream.megabitsPerSecond,
1362
+ pipeThroughputMBps: achievedMBps
1053
1363
  });
1054
1364
  };
1055
1365
  const timer = setTimeout(finish, DECODE_WINDOW_MAX_MS);
1056
1366
  try {
1057
- child = spawn(ffmpegBin, args, { stdio: ["ignore", "pipe", "pipe"], windowsHide: true });
1058
- } catch {
1059
- // The timer would otherwise hold the event loop for its full wait and
1060
- // then run against a child that was never created.
1367
+ child = spawn(ffmpegBin, args, { stdio: ["pipe", "pipe", "pipe"], windowsHide: true });
1368
+ } catch (error) {
1061
1369
  clearTimeout(timer);
1062
1370
  settled = true;
1063
- resolve(null);
1371
+ resolve({ error: error instanceof Error ? error.message : String(error) });
1064
1372
  return;
1065
1373
  }
1066
1374
  child.stderr.on("data", (chunk) => {
1067
1375
  stderr += String(chunk);
1068
1376
  });
1377
+ // Keep the decoder fed. `write` returning false means the pipe is full, and
1378
+ // the next copy goes on the `drain` — so the decoder is never starved and
1379
+ // this process never buffers more than the pipe holds.
1380
+ const writeOnce = () => {
1381
+ const accepted = child.stdin.write(stream.bytes);
1382
+ bytesWritten += stream.bytes.length;
1383
+ return accepted;
1384
+ };
1385
+ const feed = () => {
1386
+ while (!settled && child.stdin.writable && writeOnce()) {
1387
+ // Written straight through; go round again.
1388
+ }
1389
+ };
1390
+ child.stdin.on("drain", feed);
1391
+ // The kill closes the pipe under the writer; that is the intended end.
1392
+ child.stdin.on("error", () => {});
1069
1393
  child.stdout.on("data", (chunk) => {
1070
1394
  stdout += String(chunk);
1071
1395
  let newline = stdout.indexOf("\n");
@@ -1084,15 +1408,26 @@ function measureDecodeSlope(ffmpegBin, clipPath) {
1084
1408
  finish();
1085
1409
  }
1086
1410
  });
1087
- child.on("error", () => {
1411
+ child.on("error", (error) => {
1088
1412
  if (settled) {
1089
1413
  return;
1090
1414
  }
1091
1415
  clearTimeout(timer);
1092
1416
  settled = true;
1093
- resolve(null);
1417
+ try {
1418
+ child?.stdin?.destroy();
1419
+ } catch {
1420
+ // already gone
1421
+ }
1422
+ try {
1423
+ child?.kill("SIGKILL");
1424
+ } catch {
1425
+ // already gone
1426
+ }
1427
+ resolve({ error: error instanceof Error ? error.message : String(error) });
1094
1428
  });
1095
1429
  child.on("close", finish);
1430
+ feed();
1096
1431
  });
1097
1432
  }
1098
1433
 
@@ -2,11 +2,12 @@
2
2
  * @file WebTorrent client pool.
3
3
  *
4
4
  * Manages a shared WebTorrent client instance and a map of active torrents
5
- * keyed by a hash of their source. Tracks file-level usage so that only
6
- * the pieces needed by active streams are selected for download.
5
+ * keyed by their own infohash (see `torrent-source-key.js`), so a magnet and
6
+ * a `.torrent` file for the same content share one entry. Tracks file-level
7
+ * usage so that only the pieces needed by active streams are selected for
8
+ * download.
7
9
  */
8
10
 
9
- import crypto from "node:crypto";
10
11
  import dns from "node:dns/promises";
11
12
  import os from "node:os";
12
13
  import path from "node:path";
@@ -14,6 +15,7 @@ import { rmSync, statfsSync } from "node:fs";
14
15
  import WebTorrent from "webtorrent";
15
16
  import { logger } from "../utils/logger.js";
16
17
  import { SharedPieceStore, findSharedStore } from "./piece-store/shared-piece-store.js";
18
+ import { deriveSourceKey } from "./torrent-source-key.js";
17
19
 
18
20
  // The DHT's entry points. Two of the three the library ships answer nothing —
19
21
  // measured 2026-08-21 from the addon host: `router.bittorrent.com` and
@@ -1338,7 +1340,9 @@ export class TorrentPool {
1338
1340
  * @returns {Promise<import("webtorrent").Torrent>}
1339
1341
  */
1340
1342
  async getTorrent(sourceType, source) {
1341
- const key = `${sourceType}:${crypto.createHash("sha1").update(source).digest("hex")}`;
1343
+ // Keyed by the torrent's own infohash, not the source bytes — a magnet
1344
+ // and a `.torrent` file for the same content must name the same swarm.
1345
+ const key = await deriveSourceKey(sourceType, source);
1342
1346
 
1343
1347
  // Already resolved — return immediately.
1344
1348
  const existing = this.torrents.get(key);
@@ -0,0 +1,38 @@
1
+ /**
2
+ * @file Canonical identity for a torrent source: its own infohash.
3
+ *
4
+ * A magnet URI and a `.torrent` file for the same content are different
5
+ * bytes, so naming a source by a hash of those bytes gave the same film two
6
+ * unrelated keys — one from a magnet, another from a `.torrent` — sharing
7
+ * neither the swarm, nor a cache, nor any work already done. Measured
8
+ * 2026-08-19: `a518ff46…` and `7ab2fb5d…` for one infohash `11f09299…`.
9
+ *
10
+ * The infohash is the one thing both forms carry for the same content, and
11
+ * both carry it synchronously — a magnet's `btih` needs no network, and a
12
+ * `.torrent` file's SHA-1 of its own `info` dictionary needs no metadata
13
+ * exchange either. `parse-torrent` (already in the dependency tree via
14
+ * `webtorrent`) reads either form without touching the network.
15
+ */
16
+
17
+ import parseTorrent from "parse-torrent";
18
+
19
+ /**
20
+ * @param {"magnet" | "torrent"} sourceType
21
+ * @param {string} source - Magnet URI, or base64-encoded `.torrent` bytes.
22
+ * @returns {Promise<string>} `torrent:<infohash>` — identical for a magnet
23
+ * and a `.torrent` file describing the same content.
24
+ */
25
+ export async function deriveSourceKey(sourceType, source) {
26
+ const torrentId = sourceType === "torrent" ? Buffer.from(source, "base64") : source;
27
+ let parsed;
28
+ try {
29
+ parsed = await parseTorrent(torrentId);
30
+ } catch (error) {
31
+ const message = error instanceof Error ? error.message : String(error);
32
+ throw new Error(`Could not read an infohash from this ${sourceType}: ${message}`);
33
+ }
34
+ if (!parsed?.infoHash) {
35
+ throw new Error(`Could not read an infohash from this ${sourceType}.`);
36
+ }
37
+ return `torrent:${parsed.infoHash}`;
38
+ }
@@ -22,22 +22,8 @@
22
22
  * across calls for the same torrent.
23
23
  */
24
24
 
25
- import crypto from "node:crypto";
26
25
  import { TorrentWorkerClient } from "./client.js";
27
-
28
- /**
29
- * Stable key for a source, matching how the worker keys its torrents.
30
- *
31
- * Derived from the source itself rather than handed out per request, so two
32
- * routes asking for the same torrent name the same thing on the worker side.
33
- *
34
- * @param {"magnet" | "torrent"} sourceType
35
- * @param {string} source
36
- * @returns {string}
37
- */
38
- function deriveSourceKey(sourceType, source) {
39
- return `${sourceType}:${crypto.createHash("sha1").update(source).digest("hex")}`;
40
- }
26
+ import { deriveSourceKey } from "../torrent-source-key.js";
41
27
 
42
28
  /**
43
29
  * A torrent pool whose work happens on another thread.
@@ -65,7 +51,7 @@ export class WorkerTorrentPool {
65
51
  * @returns {Promise<object>}
66
52
  */
67
53
  async getTorrent(sourceType, source) {
68
- const sourceKey = deriveSourceKey(sourceType, source);
54
+ const sourceKey = await deriveSourceKey(sourceType, source);
69
55
  const existing = this.#torrents.get(sourceKey);
70
56
  if (existing) {
71
57
  return existing;
@@ -0,0 +1,83 @@
1
+ /**
2
+ * @file What the decode measurement must never get wrong.
3
+ *
4
+ * The quality menu is predicted from these readings, so a reading that is
5
+ * merely PRECISE is worth nothing — it has to be ordered the way decoding is.
6
+ * Under `-stream_loop -1` it was not: the loop restart costs 0.03 s on a 480p
7
+ * clip and 0.12 s on a 1080p one (the decoder tearing down and re-allocating
8
+ * its frame buffers), a five-second clip at 55x restarts eleven times a second,
9
+ * and the resulting bias depends on the clip's own resolution and on how fast
10
+ * the host is. On a fast desktop, 2026-08-20, that made 1080p read cheaper than
11
+ * 720p — which is not a thing a decoder does, and the fit refused to solve.
12
+ *
13
+ * These tests state the orderings instead of the numbers: the numbers are
14
+ * properties of whatever machine runs them.
15
+ */
16
+
17
+ import test from "node:test";
18
+ import assert from "node:assert/strict";
19
+ import path from "node:path";
20
+ import { fileURLToPath } from "node:url";
21
+ import ffmpegStatic from "ffmpeg-static";
22
+ import { measureDecodeSlope } from "../services/hwaccel.js";
23
+
24
+ const CLIPS = path.join(path.dirname(fileURLToPath(import.meta.url)), "..", "assets", "calibration");
25
+ const clip = (name) => path.join(CLIPS, name);
26
+
27
+ test("a bigger picture costs more than a smaller one of the same bitrate", async () => {
28
+ const big = await measureDecodeSlope(ffmpegStatic, clip("cal-h264-1080-hi.mp4"));
29
+ const small = await measureDecodeSlope(ffmpegStatic, clip("cal-h264-480-hi.mp4"));
30
+
31
+ assert.ok(big, "the 1080p clip was measured");
32
+ assert.ok(small, "the 480p clip was measured");
33
+ assert.ok(
34
+ small.speed > big.speed,
35
+ `480p decoded at ${small.speed.toFixed(1)}x and 1080p at ${big.speed.toFixed(1)}x — ` +
36
+ `an ordering no decoder produces, which is what the loop used to invert`
37
+ );
38
+ });
39
+
40
+ test("a thicker stream costs more than a thin one of the same size", async () => {
41
+ const thick = await measureDecodeSlope(ffmpegStatic, clip("cal-h264-480-hi.mp4"));
42
+ const thin = await measureDecodeSlope(ffmpegStatic, clip("cal-h264-480-lo.mp4"));
43
+
44
+ assert.ok(thick && thin);
45
+ assert.ok(
46
+ thin.speed > thick.speed,
47
+ `the same picture at ${thin.megabitsPerSecond.toFixed(2)} Mbit/s decoded at ${thin.speed.toFixed(1)}x ` +
48
+ `and at ${thick.megabitsPerSecond.toFixed(2)} Mbit/s at ${thick.speed.toFixed(1)}x`
49
+ );
50
+ });
51
+
52
+ test("HEVC costs more than H.264 for the same picture on the same machine", async () => {
53
+ const h264 = await measureDecodeSlope(ffmpegStatic, clip("cal-h264-480-lo.mp4"), "h264");
54
+ const hevc = await measureDecodeSlope(ffmpegStatic, clip("cal-hevc-480-lo.mp4"), "hevc");
55
+
56
+ assert.ok(h264 && hevc, "both families are lifted out of their containers correctly");
57
+ assert.ok(
58
+ h264.speed > hevc.speed,
59
+ `H.264 ${h264.speed.toFixed(1)}x against HEVC ${hevc.speed.toFixed(1)}x — the reason the model ` +
60
+ `is fitted per codec family at all`
61
+ );
62
+ });
63
+
64
+ test("the clip's own characteristics come back with the reading", async () => {
65
+ const measured = await measureDecodeSlope(ffmpegStatic, clip("cal-h264-1080-hi.mp4"));
66
+
67
+ assert.ok(measured);
68
+ // Read from the container rather than declared anywhere, so replacing a clip
69
+ // cannot silently invalidate the fit that rests on it.
70
+ assert.ok(measured.megapixelsPerSecond > 40 && measured.megapixelsPerSecond < 60);
71
+ assert.ok(measured.megabitsPerSecond > 5 && measured.megabitsPerSecond < 15);
72
+ assert.ok(measured.windowSec >= 0.5, "the slope is taken over a window wide enough to divide by");
73
+ });
74
+
75
+ test("a reading carries what the pipe was actually fed, for item 4(d2)'s sanity check", async () => {
76
+ const measured = await measureDecodeSlope(ffmpegStatic, clip("cal-h264-480-lo.mp4"));
77
+
78
+ assert.ok(measured);
79
+ assert.ok(
80
+ measured.pipeThroughputMBps > 0,
81
+ "the write path pushed a nonzero amount of the clip through, or nothing was measured at all"
82
+ );
83
+ });