@torrent-tv/proxy 2.54.0 → 2.55.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +16 -0
- package/package.json +2 -1
- package/research/preset-benchmark-size-scaling-2026-08-22.md +55 -0
- package/services/hwaccel.js +379 -44
- package/services/torrent-pool.js +8 -4
- package/services/torrent-source-key.js +38 -0
- package/services/torrent-worker/pool-adapter.js +2 -16
- package/test/decode-measurement.test.js +83 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,19 @@
|
|
|
1
|
+
## 2.55.1
|
|
2
|
+
|
|
3
|
+
- **Fix**: A source is now keyed by its own infohash, not by a hash of the request bytes. A magnet URI and a `.torrent` file for the same content are different bytes, so the old key (`sha1` of the source) named the same film as two unrelated sources — measured 2026-08-19, `a518ff46…` and `7ab2fb5d…` for one infohash `11f09299…` — sharing neither the swarm, nor a cache, nor any work already downloaded, and surfacing as `WebTorrent client error: Cannot add duplicate torrent`. The new key (`services/torrent-source-key.js`) reads the infohash straight out of the magnet's `btih` or the `.torrent`'s own `info` dictionary via `parse-torrent`, synchronously, with no network round trip — so both forms of the same torrent now share one entry from the first request, on both the pool's own map (`torrent-pool.js`) and the worker-thread boundary (`torrent-worker/pool-adapter.js`), instead of relying on WebTorrent's own duplicate-add error to reconcile them after the fact.
|
|
4
|
+
- **Chore**: The decode-cost reading now logs how many MB/s the pipe was actually fed against how many MB/s the measured speed implies were needed (item 4(d2)) — a divergence between the two is worth a second look before trusting the reading. A `write()`-return-value signal was tried first, to say outright whether the pipe or the decoder was the limit, and measured false on every reading taken while building it — including clips this host decodes at 15-80x with slack to spare — so it does not discriminate and was not shipped; only the byte count, which is real, is kept.
|
|
5
|
+
|
|
6
|
+
## 2.55.0
|
|
7
|
+
|
|
8
|
+
- **Fix**: The decode calibration was measuring the loop rather than the decode. It looped each clip with `-stream_loop -1`, and a loop is not free: measured 2026-08-22, a restart costs **0.03 s on the 480p clip and 0.12 s on the 1080p one** — it scales with the picture, so it is the decoder tearing down and re-allocating its frame buffers rather than anything about reading the file. A five-second clip decoded at 55x restarts eleven times a second, and that cost dominated the reading: the same clips measured 53.7x looped against 80.3x in one continuous pass, and 11.8x against 15.8x. Worse, the bias depends on BOTH the clip's own resolution and the host's speed — the two axes the fit exists to separate — so it did not cancel out, it tilted the fit. That is the fast-host failure recorded on 2026-08-20, where a desktop read 1080p at 9.35 Mbit/s as cheaper than 720p at 9.94, an ordering no decoder produces, and the H.264 fit refused to solve at all. A host that could not fit H.264 got no decode figure whatsoever, which is exactly the host most able to serve.
|
|
9
|
+
- **New**: The clip is fed to the decoder as ONE Annex-B elementary stream, written to its stdin over and over. Parameter sets are inline in Annex-B and it can be joined by plain byte concatenation — that is what a broadcast is — so more bytes are simply more stream: nothing re-opens, nothing re-initialises, and there is no restart inside the measured window. The lift out of the container is a copy, not a re-encode, and it goes straight to a pipe: no temporary media is written at any point. Error against the continuous-pass truth is now −0.2 % and −5.5 %, with the readings spread 2-6 %, against −25 % and −33 % for the loop. Every reading on the developer's desktop moved, by up to 66 %, and they are monotonic in both axes for the first time.
|
|
10
|
+
- **New**: The measured window is half a second instead of one. What used to make a long window necessary was the clip restarting inside it; with the stream continuous, the only thing left to average over is the timing jitter of two progress lines, which is milliseconds. Measured at half a second: −3.0 % and +0.6 %.
|
|
11
|
+
- **Fix**: The contention penalty was wrong for the same reason. It compares a decode alone against the same decode beside an encoder, and both readings carried the loop, but not equally — the machine's speed differs between them. Decoding alone now reads 77.5x where it read about 53x.
|
|
12
|
+
- **Chore**: A codec family's clips are lifted out of their containers in ONE ffmpeg run, and the startup is shorter than before rather than longer. The lift is a copy, so its cost is almost entirely the process: one per clip added 11 s here, and running them concurrently did not help — six at once took 4.75 s against 0.89 s for one, so the machine serialises them. One run with many inputs and many outputs costs one process. Measured end to end on the same desktop, alternating old and new: **35.3 s before, 31.6 s after**, with the readings corrected. The contention benchmark lifts its clip once and decodes the same bytes three times instead of lifting it again for each reading. Done before the measurements and never beside them: a remux running next to a decode is a second job on the machine, and this benchmark exists to find out what one job costs.
|
|
13
|
+
- **Fix**: The lift has a time bound and is killed on it. It was the only ffmpeg run in this file without one, and it is awaited before the proxy's tunnel opens — so a remux that never exited was a startup that never finished, with nothing said. Its failures, and the decode's, now carry ffmpeg's own last line instead of "said nothing".
|
|
14
|
+
- **Fix**: A codec family with no Annex-B mapping fails by name instead of being lifted with H.264's filter. This matters for what comes next: AV1 has no Annex-B form at all — its packaging is OBU — and MPEG-2 and VC-1 have no `*_mp4toannexb` filter, so all three of the families the roadmap plans need another route through here, and finding that out as "the clip failed" would send the reader after the clip.
|
|
15
|
+
- **Chore**: `test/decode-measurement.test.js` states the orderings the readings must have — a bigger picture costs more than a smaller one at the same bitrate, a thicker stream more than a thin one at the same size, HEVC more than H.264 — rather than any number, since the numbers belong to whatever machine runs them. Those are the properties the loop inverted, and nothing was checking them.
|
|
16
|
+
|
|
1
17
|
## 2.54.0
|
|
2
18
|
|
|
3
19
|
- **Fix**: The automatic quality step no longer changes the SIZE of the picture underneath a session the browser is already decoding. The fMP4 init segment is fetched once — a player reads `#EXT-X-MAP` and never asks again — and `avc1` keeps SPS and PPS in it rather than in the fragments, so every fragment produced after a size change was decoded against parameter sets describing a picture that was no longer being made. Measured 2026-08-21 across five viewing attempts: both re-encoded sessions of the five were destroyed by it. On `LXH-12.TS` the encoder left 1280x720 for 960x540 at 13:30:06 and the browser went on reporting `decode … size=1280x720` for the next three and a half minutes — 67 readings, not one of them 960x540 — while the viewer watched a band of macroblock garbage over a smeared field. On `c0930.com_chijyo0073.wmv` the same act at 13:35:36 made the element error on the first mismatched fragment, close the MediaSource, throw `bufferAppendError InvalidStateError` on both tracks and sit at `size=0x0 readyState=0` for four and a half minutes. Which of the two happens is the decoder's choice, not ours, and no layer reported an error either time. A change of resolution is a change of VARIANT, as the standard has it: every height is already published in the master with its own init, so the proxy now ASKS the browser to move — the same act the manual menu performs, which has never had this fault.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@torrent-tv/proxy",
|
|
3
|
-
"version": "2.
|
|
3
|
+
"version": "2.55.1",
|
|
4
4
|
"description": "Torrent proxy client that exposes webseed-like HTTP stream endpoint.",
|
|
5
5
|
"license": "GPL-3.0-or-later",
|
|
6
6
|
"publishConfig": {
|
|
@@ -35,6 +35,7 @@
|
|
|
35
35
|
"franc": "^6.2.0",
|
|
36
36
|
"get-port": "^7.1.0",
|
|
37
37
|
"node-datachannel": "^0.32.0",
|
|
38
|
+
"parse-torrent": "^11.0.23",
|
|
38
39
|
"uint8-util": "2.2.6",
|
|
39
40
|
"webtorrent": "2.8.5",
|
|
40
41
|
"werift": "^0.24.2",
|
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
# Does the preset benchmark's pixels/sec figure hold at a different size? (2026-08-22)
|
|
2
|
+
|
|
3
|
+
Roadmap item 4(e): the software-preset benchmark (`benchmarkSoftwarePresets`,
|
|
4
|
+
`services/hwaccel.js`) measures encoder throughput at one fixed reference size —
|
|
5
|
+
640×360 — and every later use of that figure (deciding which quality rungs this
|
|
6
|
+
host can sustain) scales it by pixel count, assuming cost per pixel is constant
|
|
7
|
+
regardless of frame size. This was never checked against a second size.
|
|
8
|
+
|
|
9
|
+
## Measurement
|
|
10
|
+
|
|
11
|
+
Same clip (`cal-h264-1080-hi.mp4`), same `libx264` presets, same measurement
|
|
12
|
+
method as the shipped benchmark (slope between two `-progress` reports,
|
|
13
|
+
excluding startup) — run once at the existing 640×360 reference and once at
|
|
14
|
+
1280×720 (4× the pixel count), on the developer's desktop:
|
|
15
|
+
|
|
16
|
+
| preset | 640×360 | 1280×720 | ratio (should be ~1.0) |
|
|
17
|
+
|-----------|----------------|----------------|-------------------------|
|
|
18
|
+
| ultrafast | 1322.8 Mpx/s | 588.6 Mpx/s | 0.445 |
|
|
19
|
+
| fast | 176.6 Mpx/s | 128.1 Mpx/s | 0.725 |
|
|
20
|
+
|
|
21
|
+
Script: ad-hoc, not committed (reused the benchmark's own ffmpeg arguments at a
|
|
22
|
+
second `-vf scale=`/`-s` size; available on request if this needs re-running).
|
|
23
|
+
|
|
24
|
+
## Reading
|
|
25
|
+
|
|
26
|
+
The assumption does not hold. Cost per pixel RISES with frame size on this
|
|
27
|
+
host — by 2.25× for `ultrafast`, 1.38× for `fast`, at 4× the pixel count — and
|
|
28
|
+
the effect is preset-dependent, not a single constant. A quality ladder priced
|
|
29
|
+
from the 640×360 figure alone over-states what this host can sustain at 1080p,
|
|
30
|
+
worse for the faster presets that the ladder falls back to under load.
|
|
31
|
+
|
|
32
|
+
## Why this does not become a code change yet
|
|
33
|
+
|
|
34
|
+
One host, one clip, two sizes. [[feedback_pool_diversity]] — a correction fitted
|
|
35
|
+
to this desktop and applied to every proxy in the pool would be exactly the
|
|
36
|
+
mistake that memory exists to prevent: the pool spans ARM boards (HA Yellow /
|
|
37
|
+
CM4) with different cache sizes and memory bandwidth, where a size penalty of
|
|
38
|
+
this shape could be smaller, larger, or absent. [[feedback_no_fudged_numbers]] —
|
|
39
|
+
publishing 0.445/0.725 as universal constants is fabricating a number for hosts
|
|
40
|
+
that were never measured.
|
|
41
|
+
|
|
42
|
+
The benchmark already measures per-host (item 4(f), 4(h)); a size-scaling
|
|
43
|
+
correction belongs there too — the benchmark itself gains a second, cheap
|
|
44
|
+
measurement point (one preset, one second size) and derives its own exponent
|
|
45
|
+
from that host's two readings, the same way `fitDecodeCost` derives its terms
|
|
46
|
+
from clips rather than from a stated formula. That is the remaining work under
|
|
47
|
+
item 4(e); this note is the measurement that justifies doing it, not the fix
|
|
48
|
+
itself.
|
|
49
|
+
|
|
50
|
+
## What is verified
|
|
51
|
+
|
|
52
|
+
The measurement method matches the shipped benchmark exactly (real footage,
|
|
53
|
+
raw frames decoded once, slope between two progress reports) — the only
|
|
54
|
+
variable changed was the target size, so this is not comparing two different
|
|
55
|
+
methodologies.
|
package/services/hwaccel.js
CHANGED
|
@@ -22,6 +22,7 @@
|
|
|
22
22
|
|
|
23
23
|
import { spawn } from "node:child_process";
|
|
24
24
|
import { mkdtempSync, readdirSync, rmSync, statSync } from "node:fs";
|
|
25
|
+
import { mkdtemp, readFile, rm } from "node:fs/promises";
|
|
25
26
|
import os from "node:os";
|
|
26
27
|
import path from "node:path";
|
|
27
28
|
import { fitDecodeCost } from "./decode-cost-fit.js";
|
|
@@ -763,10 +764,20 @@ const CALIBRATION_SETS = {
|
|
|
763
764
|
const CALIBRATION_CLIPS = CALIBRATION_SETS.h264;
|
|
764
765
|
const CALIBRATION_DIR = path.join(path.dirname(fileURLToPath(import.meta.url)), "..", "assets", "calibration");
|
|
765
766
|
// How wide the measured window must be before the slope is trusted, and how
|
|
766
|
-
// long to wait for it at most.
|
|
767
|
-
//
|
|
768
|
-
//
|
|
769
|
-
|
|
767
|
+
// long to wait for it at most.
|
|
768
|
+
//
|
|
769
|
+
// Half a second, and it is the TIMING noise that sets it rather than the amount
|
|
770
|
+
// of video: the slope is output time against wall time, both read from the same
|
|
771
|
+
// two progress lines, and the jitter in stamping one is milliseconds — so half
|
|
772
|
+
// a second of window is a fraction of a percent of error on any host. What used
|
|
773
|
+
// to make a longer window necessary was the clip restarting inside it, and that
|
|
774
|
+
// is gone: the stream is continuous now. Measured 2026-08-22 against the
|
|
775
|
+
// continuous-pass truth on a desktop: -3.0 % and +0.6 % at half a second, with
|
|
776
|
+
// the readings spread 2-6 %, against -25 % and -33 % for the loop it replaces.
|
|
777
|
+
// Half a second also costs the startup about 0.6 s per clip less, which matters
|
|
778
|
+
// because every clip of every codec family is paid for before any viewer
|
|
779
|
+
// exists.
|
|
780
|
+
const DECODE_WINDOW_MIN_SEC = 0.5;
|
|
770
781
|
const DECODE_WINDOW_MAX_MS = 8000;
|
|
771
782
|
|
|
772
783
|
/**
|
|
@@ -844,8 +855,18 @@ export async function benchmarkContention({ ffmpegBin, logger, clipsDir = CALIBR
|
|
|
844
855
|
// company, not the clip's own cost, so the smallest one says it soonest.
|
|
845
856
|
const clip = path.join(clipsDir, "cal-h264-480-lo.mp4");
|
|
846
857
|
const startedAt = Date.now();
|
|
847
|
-
|
|
848
|
-
|
|
858
|
+
// Lifted once and decoded three times from the same bytes. Going through
|
|
859
|
+
// `measureDecodeSlope` lifted it again for every reading — three process
|
|
860
|
+
// starts on a path that is awaited before the proxy's tunnel opens, for a
|
|
861
|
+
// remux whose result had not changed.
|
|
862
|
+
const streams = await extractFamilyStreams(ffmpegBin, [clip], "h264");
|
|
863
|
+
const stream = streams?.[0];
|
|
864
|
+
if (!stream) {
|
|
865
|
+
log.warn("hwaccel: contention could not be measured; costs will be added as though jobs were independent");
|
|
866
|
+
return null;
|
|
867
|
+
}
|
|
868
|
+
const alone = await decodePipedStream(ffmpegBin, stream, log);
|
|
869
|
+
if (!alone?.speed) {
|
|
849
870
|
log.warn("hwaccel: contention could not be measured; costs will be added as though jobs were independent");
|
|
850
871
|
return null;
|
|
851
872
|
}
|
|
@@ -871,8 +892,8 @@ export async function benchmarkContention({ ffmpegBin, logger, clipsDir = CALIBR
|
|
|
871
892
|
await new Promise((resolve) => {
|
|
872
893
|
setTimeout(resolve, 2_000);
|
|
873
894
|
});
|
|
874
|
-
const withCompany = await
|
|
875
|
-
if (withCompany) {
|
|
895
|
+
const withCompany = await decodePipedStream(ffmpegBin, stream, log);
|
|
896
|
+
if (withCompany?.speed) {
|
|
876
897
|
beside.push({ others, speed: withCompany.speed });
|
|
877
898
|
}
|
|
878
899
|
}
|
|
@@ -947,10 +968,29 @@ async function fitOneFamily({ ffmpegBin, log, clipsDir, family, clips }) {
|
|
|
947
968
|
const startedAllAt = Date.now();
|
|
948
969
|
/** @type {Array<{ megapixelsPerSecond: number, megabitsPerSecond: number, costSecondsPerSecond: number }>} */
|
|
949
970
|
const samples = [];
|
|
950
|
-
|
|
951
|
-
|
|
952
|
-
|
|
953
|
-
|
|
971
|
+
// Every clip of the family is lifted out of its container FIRST, in one
|
|
972
|
+
// ffmpeg run. See `extractFamilyStreams` for why one run rather than one per
|
|
973
|
+
// clip, and why before the measurements rather than beside them.
|
|
974
|
+
const streams = await extractFamilyStreams(
|
|
975
|
+
ffmpegBin,
|
|
976
|
+
clips.map((clip) => path.join(clipsDir, clip)),
|
|
977
|
+
family
|
|
978
|
+
);
|
|
979
|
+
if (!streams) {
|
|
980
|
+
log.warn(
|
|
981
|
+
`hwaccel: ${family} cannot be lifted out of its container — no Annex-B filter is mapped for it, ` +
|
|
982
|
+
`so its clips were never measured`
|
|
983
|
+
);
|
|
984
|
+
return null;
|
|
985
|
+
}
|
|
986
|
+
for (const [index, clip] of clips.entries()) {
|
|
987
|
+
const stream = streams[index];
|
|
988
|
+
const measured = stream ? await decodePipedStream(ffmpegBin, stream, log) : null;
|
|
989
|
+
if (!measured?.speed) {
|
|
990
|
+
log.warn(
|
|
991
|
+
`hwaccel: decode benchmark "${clip}" said nothing; ${family} not measured` +
|
|
992
|
+
(measured?.error ? ` — ${measured.error}` : " — the clip could not be lifted out of its container")
|
|
993
|
+
);
|
|
954
994
|
return null;
|
|
955
995
|
}
|
|
956
996
|
const cost = 1 / measured.speed;
|
|
@@ -984,88 +1024,372 @@ async function fitOneFamily({ ffmpegBin, log, clipsDir, family, clips }) {
|
|
|
984
1024
|
}
|
|
985
1025
|
|
|
986
1026
|
/**
|
|
987
|
-
*
|
|
1027
|
+
* The bitstream filter and demuxer that turn a clip's video track into a
|
|
1028
|
+
* continuous elementary stream, by codec family.
|
|
1029
|
+
*
|
|
1030
|
+
* H.264 and HEVC in MP4 keep their parameter sets in the container's `avcC` /
|
|
1031
|
+
* `hvcC` and their access units length-prefixed; Annex-B carries them inline,
|
|
1032
|
+
* with start codes, which is what makes plain byte concatenation a valid
|
|
1033
|
+
* stream. That is the property this whole measurement rests on.
|
|
1034
|
+
*/
|
|
1035
|
+
const ANNEX_B_BY_FAMILY = {
|
|
1036
|
+
h264: { filter: "h264_mp4toannexb", demuxer: "h264" },
|
|
1037
|
+
hevc: { filter: "hevc_mp4toannexb", demuxer: "hevc" },
|
|
1038
|
+
hevc10: { filter: "hevc_mp4toannexb", demuxer: "hevc" }
|
|
1039
|
+
};
|
|
1040
|
+
|
|
1041
|
+
/**
|
|
1042
|
+
* The last complaint in an ffmpeg stderr, for a line that has to say why.
|
|
1043
|
+
*
|
|
1044
|
+
* @param {string} stderr
|
|
1045
|
+
* @returns {string}
|
|
1046
|
+
*/
|
|
1047
|
+
function lastErrorLine(stderr) {
|
|
1048
|
+
const lines = String(stderr ?? "")
|
|
1049
|
+
.split(/\r?\n/)
|
|
1050
|
+
.map((line) => line.trim())
|
|
1051
|
+
.filter((line) => line.length > 0);
|
|
1052
|
+
return lines[lines.length - 1] ?? "";
|
|
1053
|
+
}
|
|
1054
|
+
|
|
1055
|
+
/**
|
|
1056
|
+
* How long the lift may take before it is abandoned. It is a remux of a few
|
|
1057
|
+
* megabytes, so this is not a budget — it is the difference between a startup
|
|
1058
|
+
* that reports a failure and one that never finishes. Every other ffmpeg run in
|
|
1059
|
+
* this file has such a bound; this one did not, and it is awaited before the
|
|
1060
|
+
* proxy's tunnel opens.
|
|
1061
|
+
*/
|
|
1062
|
+
const EXTRACT_TIMEOUT_MS = 20_000;
|
|
1063
|
+
|
|
1064
|
+
/**
|
|
1065
|
+
* Lift a whole family's clips out of their containers, as Annex-B elementary
|
|
1066
|
+
* streams, in ONE ffmpeg run.
|
|
1067
|
+
*
|
|
1068
|
+
* No re-encoding — the frames are copied — so the work itself is trivial and
|
|
1069
|
+
* the cost is almost entirely the process. Doing one process per clip added
|
|
1070
|
+
* 11 s to the startup here (fourteen clips at about 0.83 s each), and running
|
|
1071
|
+
* them concurrently did not help: six at once took 4.75 s against 0.89 s for
|
|
1072
|
+
* one, so the machine serialises them. One run with many inputs and many
|
|
1073
|
+
* outputs costs one process.
|
|
1074
|
+
*
|
|
1075
|
+
* The outputs go to temporary files because several outputs cannot share one
|
|
1076
|
+
* pipe; they are read into memory and deleted immediately, and nothing about
|
|
1077
|
+
* this measurement is kept between runs.
|
|
1078
|
+
*
|
|
1079
|
+
* Before the measurements, never beside them: a remux running next to a decode
|
|
1080
|
+
* is a second job on the machine, and this benchmark exists to find out what
|
|
1081
|
+
* ONE job costs here.
|
|
1082
|
+
*
|
|
1083
|
+
* @param {string} ffmpegBin
|
|
1084
|
+
* @param {string[]} clipPaths
|
|
1085
|
+
* @param {string} family
|
|
1086
|
+
* @returns {Promise<Array<{ bytes: Buffer, demuxer: string, megapixelsPerSecond: number, megabitsPerSecond: number, fps: number } | null> | null>}
|
|
1087
|
+
* One entry per clip, in order; null when the family cannot be lifted at all.
|
|
1088
|
+
*/
|
|
1089
|
+
async function extractFamilyStreams(ffmpegBin, clipPaths, family) {
|
|
1090
|
+
const shape = ANNEX_B_BY_FAMILY[family];
|
|
1091
|
+
// A family with no mapping is a hard failure, not a silent fallback to
|
|
1092
|
+
// H.264's filter. AV1 has no Annex-B form at all (its packaging is OBU), and
|
|
1093
|
+
// MPEG-2 and VC-1 have no `*_mp4toannexb` filter — so the three families the
|
|
1094
|
+
// roadmap plans next cannot come through here, and finding that out as
|
|
1095
|
+
// "the clip failed" would send the reader after the clip.
|
|
1096
|
+
if (!shape) {
|
|
1097
|
+
return null;
|
|
1098
|
+
}
|
|
1099
|
+
const workDir = await mkdtemp(path.join(os.tmpdir(), "ttv-calibration-"));
|
|
1100
|
+
const outputs = clipPaths.map((_, index) => path.join(workDir, `stream-${index}.${shape.demuxer}`));
|
|
1101
|
+
/** @type {string[]} */
|
|
1102
|
+
const args = ["-hide_banner", "-loglevel", "info", "-nostats", "-y"];
|
|
1103
|
+
for (const clipPath of clipPaths) {
|
|
1104
|
+
args.push("-i", clipPath);
|
|
1105
|
+
}
|
|
1106
|
+
for (const [index, output] of outputs.entries()) {
|
|
1107
|
+
args.push("-map", `${index}:v:0`, "-c:v", "copy", "-bsf:v", shape.filter, "-f", shape.demuxer, output);
|
|
1108
|
+
}
|
|
1109
|
+
const stderr = await runCapturingStderr(ffmpegBin, args, EXTRACT_TIMEOUT_MS);
|
|
1110
|
+
try {
|
|
1111
|
+
if (stderr === null) {
|
|
1112
|
+
return null;
|
|
1113
|
+
}
|
|
1114
|
+
// One banner block per input, in the order they were given. Read rather
|
|
1115
|
+
// than declared, so replacing a clip cannot silently invalidate the fit
|
|
1116
|
+
// that rests on it.
|
|
1117
|
+
const blocks = splitInputBlocks(stderr, clipPaths.length);
|
|
1118
|
+
return await Promise.all(clipPaths.map(async (_, index) => {
|
|
1119
|
+
const block = blocks[index];
|
|
1120
|
+
if (!block) {
|
|
1121
|
+
return null;
|
|
1122
|
+
}
|
|
1123
|
+
const clipInfo = parseClipCharacteristics(block);
|
|
1124
|
+
const fps = parseFfmpegVideoFps(block);
|
|
1125
|
+
if (!clipInfo || !(fps > 0)) {
|
|
1126
|
+
return null;
|
|
1127
|
+
}
|
|
1128
|
+
let bytes;
|
|
1129
|
+
try {
|
|
1130
|
+
bytes = await readFile(outputs[index]);
|
|
1131
|
+
} catch {
|
|
1132
|
+
return null;
|
|
1133
|
+
}
|
|
1134
|
+
if (bytes.length === 0) {
|
|
1135
|
+
return null;
|
|
1136
|
+
}
|
|
1137
|
+
return {
|
|
1138
|
+
bytes,
|
|
1139
|
+
demuxer: shape.demuxer,
|
|
1140
|
+
megapixelsPerSecond: clipInfo.megapixelsPerSecond,
|
|
1141
|
+
megabitsPerSecond: clipInfo.megabitsPerSecond,
|
|
1142
|
+
fps
|
|
1143
|
+
};
|
|
1144
|
+
}));
|
|
1145
|
+
} finally {
|
|
1146
|
+
await rm(workDir, { recursive: true, force: true }).catch(() => {});
|
|
1147
|
+
}
|
|
1148
|
+
}
|
|
1149
|
+
|
|
1150
|
+
/**
|
|
1151
|
+
* The part of an ffmpeg banner describing each input, in order.
|
|
1152
|
+
*
|
|
1153
|
+
* ffmpeg prints one `Input #N, …` block per input and then the stream mapping;
|
|
1154
|
+
* the parsers here read a single input's facts, so they are given a single
|
|
1155
|
+
* input's text rather than the whole banner.
|
|
1156
|
+
*
|
|
1157
|
+
* @param {string} stderr
|
|
1158
|
+
* @param {number} count
|
|
1159
|
+
* @returns {string[]}
|
|
1160
|
+
*/
|
|
1161
|
+
function splitInputBlocks(stderr, count) {
|
|
1162
|
+
/** @type {string[]} */
|
|
1163
|
+
const blocks = [];
|
|
1164
|
+
for (let index = 0; index < count; index += 1) {
|
|
1165
|
+
const from = stderr.indexOf(`Input #${index},`);
|
|
1166
|
+
if (from < 0) {
|
|
1167
|
+
blocks.push("");
|
|
1168
|
+
continue;
|
|
1169
|
+
}
|
|
1170
|
+
const nextInput = stderr.indexOf(`Input #${index + 1},`, from);
|
|
1171
|
+
const mapping = stderr.indexOf("Stream mapping:", from);
|
|
1172
|
+
const ends = [nextInput, mapping].filter((at) => at > from);
|
|
1173
|
+
blocks.push(stderr.slice(from, ends.length > 0 ? Math.min(...ends) : stderr.length));
|
|
1174
|
+
}
|
|
1175
|
+
return blocks;
|
|
1176
|
+
}
|
|
1177
|
+
|
|
1178
|
+
/**
|
|
1179
|
+
* Run ffmpeg to completion and return its stderr, or null when it failed or
|
|
1180
|
+
* outlasted its bound.
|
|
1181
|
+
*
|
|
1182
|
+
* @param {string} ffmpegBin
|
|
1183
|
+
* @param {string[]} args
|
|
1184
|
+
* @param {number} timeoutMs
|
|
1185
|
+
* @returns {Promise<string | null>}
|
|
1186
|
+
*/
|
|
1187
|
+
function runCapturingStderr(ffmpegBin, args, timeoutMs) {
|
|
1188
|
+
return new Promise((resolve) => {
|
|
1189
|
+
let stderr = "";
|
|
1190
|
+
let settled = false;
|
|
1191
|
+
let child;
|
|
1192
|
+
const settle = (value) => {
|
|
1193
|
+
if (settled) {
|
|
1194
|
+
return;
|
|
1195
|
+
}
|
|
1196
|
+
settled = true;
|
|
1197
|
+
clearTimeout(timer);
|
|
1198
|
+
try {
|
|
1199
|
+
child?.kill("SIGKILL");
|
|
1200
|
+
} catch {
|
|
1201
|
+
// already gone
|
|
1202
|
+
}
|
|
1203
|
+
resolve(value);
|
|
1204
|
+
};
|
|
1205
|
+
const timer = setTimeout(() => settle(null), timeoutMs);
|
|
1206
|
+
try {
|
|
1207
|
+
child = spawn(ffmpegBin, args, { stdio: ["ignore", "ignore", "pipe"], windowsHide: true });
|
|
1208
|
+
} catch {
|
|
1209
|
+
settle(null);
|
|
1210
|
+
return;
|
|
1211
|
+
}
|
|
1212
|
+
child.stderr.on("data", (chunk) => {
|
|
1213
|
+
stderr += String(chunk);
|
|
1214
|
+
});
|
|
1215
|
+
child.on("error", () => settle(null));
|
|
1216
|
+
child.on("close", (code) => settle(code === 0 ? stderr : null));
|
|
1217
|
+
});
|
|
1218
|
+
}
|
|
1219
|
+
|
|
1220
|
+
/**
|
|
1221
|
+
* Measure how fast this host DECODES a clip, from ffmpeg's own report of how
|
|
988
1222
|
* much video it has processed.
|
|
989
1223
|
*
|
|
990
|
-
*
|
|
991
|
-
*
|
|
992
|
-
*
|
|
993
|
-
*
|
|
994
|
-
*
|
|
995
|
-
*
|
|
1224
|
+
* Two things are deliberately outside the measurement.
|
|
1225
|
+
*
|
|
1226
|
+
* **The process starting.** Wall-clock around the process cannot answer this:
|
|
1227
|
+
* starting ffmpeg costs about a second, and on a quick machine a five-second
|
|
1228
|
+
* clip decodes in a tenth of that, so the measurement would be of the program
|
|
1229
|
+
* starting. Progress lines arrive AFTER it has started, and the slope between
|
|
1230
|
+
* two of them — video processed against time taken — contains no part of the
|
|
1231
|
+
* startup by construction.
|
|
1232
|
+
*
|
|
1233
|
+
* **The clip restarting.** This used to loop the clip with `-stream_loop -1`,
|
|
1234
|
+
* and a loop is not free: measured 2026-08-22 on a desktop, a restart costs
|
|
1235
|
+
* 0.03 s on the 480p clip and 0.12 s on the 1080p one — the decoder tearing
|
|
1236
|
+
* down and re-allocating its frame buffers, which is why the price rises with
|
|
1237
|
+
* the picture. A five-second clip decoded at 55x restarts eleven times a
|
|
1238
|
+
* second, so that cost DOMINATED the reading: the same clips measured 53.7x
|
|
1239
|
+
* looped against 80.3x in one continuous pass, and 11.8x against 15.8x. Worse,
|
|
1240
|
+
* the bias is not shared — it depends on the clip's own resolution and on how
|
|
1241
|
+
* fast the host is — so it does not cancel out of the fit, it tilts it. That is
|
|
1242
|
+
* the fast-host failure recorded on 2026-08-20, where 1080p read cheaper than
|
|
1243
|
+
* 720p, which is not a thing a decoder does.
|
|
996
1244
|
*
|
|
997
|
-
*
|
|
998
|
-
*
|
|
999
|
-
*
|
|
1245
|
+
* So the clip is fed to the decoder as ONE stream instead. An Annex-B
|
|
1246
|
+
* elementary stream carries its parameter sets inline, so writing the same
|
|
1247
|
+
* bytes again is simply more stream — the decoder never re-initialises, and
|
|
1248
|
+
* there is no restart inside the window to measure. Verified against the
|
|
1249
|
+
* continuous-pass truth on the same host: -0.2 % and -5.5 %, against -25 % and
|
|
1250
|
+
* -33 % for the loop. Nothing is written to disk and the process is killed as
|
|
1251
|
+
* soon as the window is wide enough.
|
|
1252
|
+
*
|
|
1253
|
+
* Exported because the property that broke here is checkable and was not being
|
|
1254
|
+
* checked: a bigger picture must cost more than a smaller one of the same
|
|
1255
|
+
* bitrate, and under the loop it did not.
|
|
1000
1256
|
*
|
|
1001
1257
|
* @param {string} ffmpegBin
|
|
1002
1258
|
* @param {string} clipPath
|
|
1259
|
+
* @param {string} [family="h264"]
|
|
1260
|
+
* @returns {Promise<{ speed: number, windowSec: number, megapixelsPerSecond: number, megabitsPerSecond: number } | null>}
|
|
1261
|
+
*/
|
|
1262
|
+
export async function measureDecodeSlope(ffmpegBin, clipPath, family = "h264") {
|
|
1263
|
+
const streams = await extractFamilyStreams(ffmpegBin, [clipPath], family);
|
|
1264
|
+
const stream = streams?.[0];
|
|
1265
|
+
if (!stream) {
|
|
1266
|
+
return null;
|
|
1267
|
+
}
|
|
1268
|
+
const measured = await decodePipedStream(ffmpegBin, stream);
|
|
1269
|
+
return measured?.speed ? measured : null;
|
|
1270
|
+
}
|
|
1271
|
+
|
|
1272
|
+
/**
|
|
1273
|
+
* Decode an elementary stream fed from memory, and report the slope.
|
|
1274
|
+
*
|
|
1275
|
+
* @param {string} ffmpegBin
|
|
1276
|
+
* @param {{ bytes: Buffer, demuxer: string, megapixelsPerSecond: number, megabitsPerSecond: number, fps: number }} stream
|
|
1003
1277
|
* @returns {Promise<{ speed: number, windowSec: number, megapixelsPerSecond: number, megabitsPerSecond: number } | null>}
|
|
1004
1278
|
*/
|
|
1005
|
-
function
|
|
1279
|
+
function decodePipedStream(ffmpegBin, stream, log = { info: () => {}, warn: () => {} }) {
|
|
1006
1280
|
return new Promise((resolve) => {
|
|
1007
1281
|
const args = [
|
|
1008
|
-
"-hide_banner", "-loglevel", "
|
|
1009
|
-
|
|
1010
|
-
|
|
1282
|
+
"-hide_banner", "-loglevel", "error", "-nostats",
|
|
1283
|
+
// A raw stream states no frame rate, so the one the container declared is
|
|
1284
|
+
// given back to it. It decides how output time advances, and therefore
|
|
1285
|
+
// what "seconds of video per second of clock" means.
|
|
1286
|
+
"-f", stream.demuxer, "-framerate", String(stream.fps), "-i", "pipe:0",
|
|
1011
1287
|
"-an", "-f", "null", "-",
|
|
1012
1288
|
"-progress", "pipe:1"
|
|
1013
1289
|
];
|
|
1014
1290
|
/** @type {Array<{ wallSec: number, outSec: number }>} */
|
|
1015
1291
|
const samples = [];
|
|
1016
|
-
let stderr = "";
|
|
1017
1292
|
let stdout = "";
|
|
1293
|
+
// Kept because this path depends on three things the old one did not: the
|
|
1294
|
+
// raw demuxer accepting the frame rate, the bitstream filter having
|
|
1295
|
+
// produced something parsable, and the fed concatenation being decodable.
|
|
1296
|
+
// Without it the only trace of any of those failing is "said nothing".
|
|
1297
|
+
let stderr = "";
|
|
1018
1298
|
let settled = false;
|
|
1019
1299
|
let child;
|
|
1020
1300
|
const startedAt = Date.now();
|
|
1301
|
+
// Whether the FEED, not the decoder, could be what this reading measures
|
|
1302
|
+
// (item 4(d2)). `child.stdin.write()` returning false was tried as the
|
|
1303
|
+
// signal — never once true would mean this process was never ahead of the
|
|
1304
|
+
// pipe — and measured false on every reading taken while writing this,
|
|
1305
|
+
// including clips this same host decodes at 15-80x with room to spare, so
|
|
1306
|
+
// it does not discriminate: `write()`'s return value tracks Node's own
|
|
1307
|
+
// internal watermark against the size of what was just handed to it, not
|
|
1308
|
+
// real drain state, and answered "no slack" identically whether the pipe
|
|
1309
|
+
// or the decoder was the true limit. Rather than publish a verdict that
|
|
1310
|
+
// reads the same in both cases, only the byte count is kept, for the
|
|
1311
|
+
// MB/s figure below — a number to read, not a boolean to trust.
|
|
1312
|
+
let bytesWritten = 0;
|
|
1021
1313
|
const finish = () => {
|
|
1022
1314
|
if (settled) {
|
|
1023
1315
|
return;
|
|
1024
1316
|
}
|
|
1025
1317
|
settled = true;
|
|
1026
1318
|
clearTimeout(timer);
|
|
1319
|
+
try {
|
|
1320
|
+
child?.stdin?.destroy();
|
|
1321
|
+
} catch {
|
|
1322
|
+
// already gone
|
|
1323
|
+
}
|
|
1027
1324
|
try {
|
|
1028
1325
|
child?.kill("SIGKILL");
|
|
1029
1326
|
} catch {
|
|
1030
1327
|
// already gone
|
|
1031
1328
|
}
|
|
1032
|
-
// The first sample
|
|
1033
|
-
//
|
|
1034
|
-
//
|
|
1329
|
+
// The first sample still carries the startup — it reports whatever was
|
|
1330
|
+
// processed while the process was coming up. Everything is measured from
|
|
1331
|
+
// the second onwards.
|
|
1035
1332
|
const first = samples[1];
|
|
1036
1333
|
const last = samples[samples.length - 1];
|
|
1037
|
-
|
|
1038
|
-
|
|
1039
|
-
resolve(null);
|
|
1334
|
+
if (!first || !last) {
|
|
1335
|
+
resolve({ error: lastErrorLine(stderr) || "the decoder reported no progress" });
|
|
1040
1336
|
return;
|
|
1041
1337
|
}
|
|
1042
1338
|
const windowSec = last.wallSec - first.wallSec;
|
|
1043
1339
|
const producedSec = last.outSec - first.outSec;
|
|
1044
1340
|
if (!(windowSec >= DECODE_WINDOW_MIN_SEC) || !(producedSec > 0)) {
|
|
1045
|
-
resolve(
|
|
1341
|
+
resolve({ error: lastErrorLine(stderr) || `the window was ${windowSec.toFixed(2)}s of ${producedSec.toFixed(2)}s produced` });
|
|
1046
1342
|
return;
|
|
1047
1343
|
}
|
|
1344
|
+
const speed = producedSec / windowSec;
|
|
1345
|
+
// Diagnostic only — logged, not acted on. The required rate is what
|
|
1346
|
+
// THIS reading implies was needed to keep the decoder fed; the achieved
|
|
1347
|
+
// rate is what this process actually pushed over its whole run (a wider
|
|
1348
|
+
// span than `windowSec`, since the feed starts before the first kept
|
|
1349
|
+
// sample — a looser figure is enough for a sanity comparison).
|
|
1350
|
+
const elapsedSec = (Date.now() - startedAt) / 1000;
|
|
1351
|
+
const achievedMBps = elapsedSec > 0 ? bytesWritten / elapsedSec / 1e6 : 0;
|
|
1352
|
+
const requiredMBps = ((stream.megabitsPerSecond * 1e6) / 8) * speed / 1e6;
|
|
1353
|
+
log.info(
|
|
1354
|
+
`hwaccel: decode pipe fed ${achievedMBps.toFixed(1)} MB/s, ${requiredMBps.toFixed(1)} MB/s ` +
|
|
1355
|
+
`needed for ${speed.toFixed(2)}x — a reading where these are far apart is worth a second look`
|
|
1356
|
+
);
|
|
1048
1357
|
resolve({
|
|
1049
|
-
speed
|
|
1358
|
+
speed,
|
|
1050
1359
|
windowSec,
|
|
1051
|
-
megapixelsPerSecond:
|
|
1052
|
-
megabitsPerSecond:
|
|
1360
|
+
megapixelsPerSecond: stream.megapixelsPerSecond,
|
|
1361
|
+
megabitsPerSecond: stream.megabitsPerSecond,
|
|
1362
|
+
pipeThroughputMBps: achievedMBps
|
|
1053
1363
|
});
|
|
1054
1364
|
};
|
|
1055
1365
|
const timer = setTimeout(finish, DECODE_WINDOW_MAX_MS);
|
|
1056
1366
|
try {
|
|
1057
|
-
child = spawn(ffmpegBin, args, { stdio: ["
|
|
1058
|
-
} catch {
|
|
1059
|
-
// The timer would otherwise hold the event loop for its full wait and
|
|
1060
|
-
// then run against a child that was never created.
|
|
1367
|
+
child = spawn(ffmpegBin, args, { stdio: ["pipe", "pipe", "pipe"], windowsHide: true });
|
|
1368
|
+
} catch (error) {
|
|
1061
1369
|
clearTimeout(timer);
|
|
1062
1370
|
settled = true;
|
|
1063
|
-
resolve(
|
|
1371
|
+
resolve({ error: error instanceof Error ? error.message : String(error) });
|
|
1064
1372
|
return;
|
|
1065
1373
|
}
|
|
1066
1374
|
child.stderr.on("data", (chunk) => {
|
|
1067
1375
|
stderr += String(chunk);
|
|
1068
1376
|
});
|
|
1377
|
+
// Keep the decoder fed. `write` returning false means the pipe is full, and
|
|
1378
|
+
// the next copy goes on the `drain` — so the decoder is never starved and
|
|
1379
|
+
// this process never buffers more than the pipe holds.
|
|
1380
|
+
const writeOnce = () => {
|
|
1381
|
+
const accepted = child.stdin.write(stream.bytes);
|
|
1382
|
+
bytesWritten += stream.bytes.length;
|
|
1383
|
+
return accepted;
|
|
1384
|
+
};
|
|
1385
|
+
const feed = () => {
|
|
1386
|
+
while (!settled && child.stdin.writable && writeOnce()) {
|
|
1387
|
+
// Written straight through; go round again.
|
|
1388
|
+
}
|
|
1389
|
+
};
|
|
1390
|
+
child.stdin.on("drain", feed);
|
|
1391
|
+
// The kill closes the pipe under the writer; that is the intended end.
|
|
1392
|
+
child.stdin.on("error", () => {});
|
|
1069
1393
|
child.stdout.on("data", (chunk) => {
|
|
1070
1394
|
stdout += String(chunk);
|
|
1071
1395
|
let newline = stdout.indexOf("\n");
|
|
@@ -1084,15 +1408,26 @@ function measureDecodeSlope(ffmpegBin, clipPath) {
|
|
|
1084
1408
|
finish();
|
|
1085
1409
|
}
|
|
1086
1410
|
});
|
|
1087
|
-
child.on("error", () => {
|
|
1411
|
+
child.on("error", (error) => {
|
|
1088
1412
|
if (settled) {
|
|
1089
1413
|
return;
|
|
1090
1414
|
}
|
|
1091
1415
|
clearTimeout(timer);
|
|
1092
1416
|
settled = true;
|
|
1093
|
-
|
|
1417
|
+
try {
|
|
1418
|
+
child?.stdin?.destroy();
|
|
1419
|
+
} catch {
|
|
1420
|
+
// already gone
|
|
1421
|
+
}
|
|
1422
|
+
try {
|
|
1423
|
+
child?.kill("SIGKILL");
|
|
1424
|
+
} catch {
|
|
1425
|
+
// already gone
|
|
1426
|
+
}
|
|
1427
|
+
resolve({ error: error instanceof Error ? error.message : String(error) });
|
|
1094
1428
|
});
|
|
1095
1429
|
child.on("close", finish);
|
|
1430
|
+
feed();
|
|
1096
1431
|
});
|
|
1097
1432
|
}
|
|
1098
1433
|
|
package/services/torrent-pool.js
CHANGED
|
@@ -2,11 +2,12 @@
|
|
|
2
2
|
* @file WebTorrent client pool.
|
|
3
3
|
*
|
|
4
4
|
* Manages a shared WebTorrent client instance and a map of active torrents
|
|
5
|
-
* keyed by
|
|
6
|
-
*
|
|
5
|
+
* keyed by their own infohash (see `torrent-source-key.js`), so a magnet and
|
|
6
|
+
* a `.torrent` file for the same content share one entry. Tracks file-level
|
|
7
|
+
* usage so that only the pieces needed by active streams are selected for
|
|
8
|
+
* download.
|
|
7
9
|
*/
|
|
8
10
|
|
|
9
|
-
import crypto from "node:crypto";
|
|
10
11
|
import dns from "node:dns/promises";
|
|
11
12
|
import os from "node:os";
|
|
12
13
|
import path from "node:path";
|
|
@@ -14,6 +15,7 @@ import { rmSync, statfsSync } from "node:fs";
|
|
|
14
15
|
import WebTorrent from "webtorrent";
|
|
15
16
|
import { logger } from "../utils/logger.js";
|
|
16
17
|
import { SharedPieceStore, findSharedStore } from "./piece-store/shared-piece-store.js";
|
|
18
|
+
import { deriveSourceKey } from "./torrent-source-key.js";
|
|
17
19
|
|
|
18
20
|
// The DHT's entry points. Two of the three the library ships answer nothing —
|
|
19
21
|
// measured 2026-08-21 from the addon host: `router.bittorrent.com` and
|
|
@@ -1338,7 +1340,9 @@ export class TorrentPool {
|
|
|
1338
1340
|
* @returns {Promise<import("webtorrent").Torrent>}
|
|
1339
1341
|
*/
|
|
1340
1342
|
async getTorrent(sourceType, source) {
|
|
1341
|
-
|
|
1343
|
+
// Keyed by the torrent's own infohash, not the source bytes — a magnet
|
|
1344
|
+
// and a `.torrent` file for the same content must name the same swarm.
|
|
1345
|
+
const key = await deriveSourceKey(sourceType, source);
|
|
1342
1346
|
|
|
1343
1347
|
// Already resolved — return immediately.
|
|
1344
1348
|
const existing = this.torrents.get(key);
|
|
@@ -0,0 +1,38 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @file Canonical identity for a torrent source: its own infohash.
|
|
3
|
+
*
|
|
4
|
+
* A magnet URI and a `.torrent` file for the same content are different
|
|
5
|
+
* bytes, so naming a source by a hash of those bytes gave the same film two
|
|
6
|
+
* unrelated keys — one from a magnet, another from a `.torrent` — sharing
|
|
7
|
+
* neither the swarm, nor a cache, nor any work already done. Measured
|
|
8
|
+
* 2026-08-19: `a518ff46…` and `7ab2fb5d…` for one infohash `11f09299…`.
|
|
9
|
+
*
|
|
10
|
+
* The infohash is the one thing both forms carry for the same content, and
|
|
11
|
+
* both carry it synchronously — a magnet's `btih` needs no network, and a
|
|
12
|
+
* `.torrent` file's SHA-1 of its own `info` dictionary needs no metadata
|
|
13
|
+
* exchange either. `parse-torrent` (already in the dependency tree via
|
|
14
|
+
* `webtorrent`) reads either form without touching the network.
|
|
15
|
+
*/
|
|
16
|
+
|
|
17
|
+
import parseTorrent from "parse-torrent";
|
|
18
|
+
|
|
19
|
+
/**
|
|
20
|
+
* @param {"magnet" | "torrent"} sourceType
|
|
21
|
+
* @param {string} source - Magnet URI, or base64-encoded `.torrent` bytes.
|
|
22
|
+
* @returns {Promise<string>} `torrent:<infohash>` — identical for a magnet
|
|
23
|
+
* and a `.torrent` file describing the same content.
|
|
24
|
+
*/
|
|
25
|
+
export async function deriveSourceKey(sourceType, source) {
|
|
26
|
+
const torrentId = sourceType === "torrent" ? Buffer.from(source, "base64") : source;
|
|
27
|
+
let parsed;
|
|
28
|
+
try {
|
|
29
|
+
parsed = await parseTorrent(torrentId);
|
|
30
|
+
} catch (error) {
|
|
31
|
+
const message = error instanceof Error ? error.message : String(error);
|
|
32
|
+
throw new Error(`Could not read an infohash from this ${sourceType}: ${message}`);
|
|
33
|
+
}
|
|
34
|
+
if (!parsed?.infoHash) {
|
|
35
|
+
throw new Error(`Could not read an infohash from this ${sourceType}.`);
|
|
36
|
+
}
|
|
37
|
+
return `torrent:${parsed.infoHash}`;
|
|
38
|
+
}
|
|
@@ -22,22 +22,8 @@
|
|
|
22
22
|
* across calls for the same torrent.
|
|
23
23
|
*/
|
|
24
24
|
|
|
25
|
-
import crypto from "node:crypto";
|
|
26
25
|
import { TorrentWorkerClient } from "./client.js";
|
|
27
|
-
|
|
28
|
-
/**
|
|
29
|
-
* Stable key for a source, matching how the worker keys its torrents.
|
|
30
|
-
*
|
|
31
|
-
* Derived from the source itself rather than handed out per request, so two
|
|
32
|
-
* routes asking for the same torrent name the same thing on the worker side.
|
|
33
|
-
*
|
|
34
|
-
* @param {"magnet" | "torrent"} sourceType
|
|
35
|
-
* @param {string} source
|
|
36
|
-
* @returns {string}
|
|
37
|
-
*/
|
|
38
|
-
function deriveSourceKey(sourceType, source) {
|
|
39
|
-
return `${sourceType}:${crypto.createHash("sha1").update(source).digest("hex")}`;
|
|
40
|
-
}
|
|
26
|
+
import { deriveSourceKey } from "../torrent-source-key.js";
|
|
41
27
|
|
|
42
28
|
/**
|
|
43
29
|
* A torrent pool whose work happens on another thread.
|
|
@@ -65,7 +51,7 @@ export class WorkerTorrentPool {
|
|
|
65
51
|
* @returns {Promise<object>}
|
|
66
52
|
*/
|
|
67
53
|
async getTorrent(sourceType, source) {
|
|
68
|
-
const sourceKey = deriveSourceKey(sourceType, source);
|
|
54
|
+
const sourceKey = await deriveSourceKey(sourceType, source);
|
|
69
55
|
const existing = this.#torrents.get(sourceKey);
|
|
70
56
|
if (existing) {
|
|
71
57
|
return existing;
|
|
@@ -0,0 +1,83 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @file What the decode measurement must never get wrong.
|
|
3
|
+
*
|
|
4
|
+
* The quality menu is predicted from these readings, so a reading that is
|
|
5
|
+
* merely PRECISE is worth nothing — it has to be ordered the way decoding is.
|
|
6
|
+
* Under `-stream_loop -1` it was not: the loop restart costs 0.03 s on a 480p
|
|
7
|
+
* clip and 0.12 s on a 1080p one (the decoder tearing down and re-allocating
|
|
8
|
+
* its frame buffers), a five-second clip at 55x restarts eleven times a second,
|
|
9
|
+
* and the resulting bias depends on the clip's own resolution and on how fast
|
|
10
|
+
* the host is. On a fast desktop, 2026-08-20, that made 1080p read cheaper than
|
|
11
|
+
* 720p — which is not a thing a decoder does, and the fit refused to solve.
|
|
12
|
+
*
|
|
13
|
+
* These tests state the orderings instead of the numbers: the numbers are
|
|
14
|
+
* properties of whatever machine runs them.
|
|
15
|
+
*/
|
|
16
|
+
|
|
17
|
+
import test from "node:test";
|
|
18
|
+
import assert from "node:assert/strict";
|
|
19
|
+
import path from "node:path";
|
|
20
|
+
import { fileURLToPath } from "node:url";
|
|
21
|
+
import ffmpegStatic from "ffmpeg-static";
|
|
22
|
+
import { measureDecodeSlope } from "../services/hwaccel.js";
|
|
23
|
+
|
|
24
|
+
const CLIPS = path.join(path.dirname(fileURLToPath(import.meta.url)), "..", "assets", "calibration");
|
|
25
|
+
const clip = (name) => path.join(CLIPS, name);
|
|
26
|
+
|
|
27
|
+
test("a bigger picture costs more than a smaller one of the same bitrate", async () => {
|
|
28
|
+
const big = await measureDecodeSlope(ffmpegStatic, clip("cal-h264-1080-hi.mp4"));
|
|
29
|
+
const small = await measureDecodeSlope(ffmpegStatic, clip("cal-h264-480-hi.mp4"));
|
|
30
|
+
|
|
31
|
+
assert.ok(big, "the 1080p clip was measured");
|
|
32
|
+
assert.ok(small, "the 480p clip was measured");
|
|
33
|
+
assert.ok(
|
|
34
|
+
small.speed > big.speed,
|
|
35
|
+
`480p decoded at ${small.speed.toFixed(1)}x and 1080p at ${big.speed.toFixed(1)}x — ` +
|
|
36
|
+
`an ordering no decoder produces, which is what the loop used to invert`
|
|
37
|
+
);
|
|
38
|
+
});
|
|
39
|
+
|
|
40
|
+
test("a thicker stream costs more than a thin one of the same size", async () => {
|
|
41
|
+
const thick = await measureDecodeSlope(ffmpegStatic, clip("cal-h264-480-hi.mp4"));
|
|
42
|
+
const thin = await measureDecodeSlope(ffmpegStatic, clip("cal-h264-480-lo.mp4"));
|
|
43
|
+
|
|
44
|
+
assert.ok(thick && thin);
|
|
45
|
+
assert.ok(
|
|
46
|
+
thin.speed > thick.speed,
|
|
47
|
+
`the same picture at ${thin.megabitsPerSecond.toFixed(2)} Mbit/s decoded at ${thin.speed.toFixed(1)}x ` +
|
|
48
|
+
`and at ${thick.megabitsPerSecond.toFixed(2)} Mbit/s at ${thick.speed.toFixed(1)}x`
|
|
49
|
+
);
|
|
50
|
+
});
|
|
51
|
+
|
|
52
|
+
test("HEVC costs more than H.264 for the same picture on the same machine", async () => {
|
|
53
|
+
const h264 = await measureDecodeSlope(ffmpegStatic, clip("cal-h264-480-lo.mp4"), "h264");
|
|
54
|
+
const hevc = await measureDecodeSlope(ffmpegStatic, clip("cal-hevc-480-lo.mp4"), "hevc");
|
|
55
|
+
|
|
56
|
+
assert.ok(h264 && hevc, "both families are lifted out of their containers correctly");
|
|
57
|
+
assert.ok(
|
|
58
|
+
h264.speed > hevc.speed,
|
|
59
|
+
`H.264 ${h264.speed.toFixed(1)}x against HEVC ${hevc.speed.toFixed(1)}x — the reason the model ` +
|
|
60
|
+
`is fitted per codec family at all`
|
|
61
|
+
);
|
|
62
|
+
});
|
|
63
|
+
|
|
64
|
+
test("the clip's own characteristics come back with the reading", async () => {
|
|
65
|
+
const measured = await measureDecodeSlope(ffmpegStatic, clip("cal-h264-1080-hi.mp4"));
|
|
66
|
+
|
|
67
|
+
assert.ok(measured);
|
|
68
|
+
// Read from the container rather than declared anywhere, so replacing a clip
|
|
69
|
+
// cannot silently invalidate the fit that rests on it.
|
|
70
|
+
assert.ok(measured.megapixelsPerSecond > 40 && measured.megapixelsPerSecond < 60);
|
|
71
|
+
assert.ok(measured.megabitsPerSecond > 5 && measured.megabitsPerSecond < 15);
|
|
72
|
+
assert.ok(measured.windowSec >= 0.5, "the slope is taken over a window wide enough to divide by");
|
|
73
|
+
});
|
|
74
|
+
|
|
75
|
+
test("a reading carries what the pipe was actually fed, for item 4(d2)'s sanity check", async () => {
|
|
76
|
+
const measured = await measureDecodeSlope(ffmpegStatic, clip("cal-h264-480-lo.mp4"));
|
|
77
|
+
|
|
78
|
+
assert.ok(measured);
|
|
79
|
+
assert.ok(
|
|
80
|
+
measured.pipeThroughputMBps > 0,
|
|
81
|
+
"the write path pushed a nonzero amount of the clip through, or nothing was measured at all"
|
|
82
|
+
);
|
|
83
|
+
});
|