@torrent-tv/proxy 2.50.0 → 2.51.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +12 -0
- package/package.json +1 -1
- package/routes/api/transcode-sessions/fragment-far/post.js +60 -0
- package/server.js +4 -0
- package/services/hls-session-manager.js +247 -15
- package/services/torrent-pool.js +165 -2
- package/services/torrent-worker/worker.js +9 -2
- package/test/dht-bootstrap.test.js +72 -0
- package/test/run-position-follows-published-grid.test.js +87 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,15 @@
|
|
|
1
|
+
## 2.51.0
|
|
2
|
+
|
|
3
|
+
- **Fix**: A run is POSITIONED where the player was told the segment begins, on the same table its cuts are stated on. There are two boundary tables — the one the playlist text was written from, which never changes, and the live one, corrected as produced segments reveal where the file's cuts truly are. 2.45.0 moved the CUT LIST onto the published table and left the position on the live one, and that is one fault rather than two: `-segment_times` are measured from wherever the run really began, so any distance between the two carries into EVERY cut the run makes. The corrections run backwards, so each restart began a little earlier than the grid its cuts were stated on, and because the corrections accumulate, so did the distance. Measured 2026-08-21 on `JUFD665.mp4` — an MP4 whose index was read cleanly, 1765 keyframes, served by copy: after one seek restart a produced segment held the boundary **two** places before its own number (16.684 s, exactly 2.0000 segments), after the next restart **four** (33.5 s). The player's buffer then stops extending at all, because every fragment's content lands before the time its playlist entry names: `bufferEnd` stood still at 4571.1 s through four `frag-far` warnings until hls.js gave up and jumped the viewer 16.8 s forward. Four of those jumps in one window is what the viewer reported as sticking on every seek.
|
|
4
|
+
- **Fix**: The line about a segment that began away from its grid follows the numbers instead of the branch it is printed from. On the copy path it always read "the container's keyframe index disagrees with the file", including in the case this code's own neighbouring comment defines as a fault in this code — a segment holding another boundary **of the same list**. It now says which of the two it is, because they need opposite fixes and that sentence is what sent the reading of the session above after the file instead of after the arithmetic. The soundtrack's variant of the sentence stops claiming anything about a keyframe index: a soundtrack has no keyframes and is cut exactly where it was asked to be.
|
|
5
|
+
- **New**: A run says where it was positioned — the time, the boundary it came from, that boundary's time in the PUBLISHED table and in the LIVE one, the distance between them, and the number it starts counting from. Four numbers whose disagreement was invisible everywhere else.
|
|
6
|
+
- **New**: A correction to the live boundary table states the total drift it is part of. Corrections are applied one boundary at a time and each is small enough to look harmless; nobody was watching the sum, which is the quantity that actually moves the cuts.
|
|
7
|
+
- **Fix**: The realignment that starts the sound where a copied picture truly begins is given that instant EXPLICITLY. It used to be smuggled through the live boundary table — the correction wrote the measured time there and the restarted run read its position from the same place — which stops working the moment a run positions itself on the table the player holds, as it now must. Smuggled, the restart would land exactly where it already was: picture and sound stay apart AND a healthy soundtrack run is discarded for nothing, which is the shape the field already showed on 2026-08-21 — eleven audio restarts in under four minutes, eight of them dying with `run had produced 0.0s`.
|
|
8
|
+
- **Fix**: What a run was asked for is taken from the run rather than looked up again in a table. Two lines did the second — the one reporting where a run landed, and the one explaining a held segment — and once the position moved to the published grid they could disagree with the run by the distance between the two tables: a perfect landing reported as a drift, a real drift of the same size cancelled to zero, and a held segment printing a negative "produced" that sends the reader after the torrent when the encoder is the subject.
|
|
9
|
+
- **New**: The player's own "this fragment is nowhere near my buffer" reading is answered instead of being left in its console. `POST /api/transcode-sessions/:id/fragment-far` takes the report and the proxy replies in the log with the one fact only this side holds: which boundary the segment of that number really begins at, and whether that is the one its number claims. The report says which stream it is about, and a report about the soundtrack is answered from the soundtrack's own session: picture and sound are produced by two sessions positioned by two runs, which is how they come apart, so answering one from the other's records would state something confident about the wrong stream. Diagnostic only — nothing is repositioned on the strength of a browser's reading, because a wrong answer there would restart an encoder a viewer is waiting on.
|
|
10
|
+
- **Fix**: The DHT is given entry points that answer. Measured 2026-08-21 from the addon host: of the three bootstrap nodes the library ships, `router.bittorrent.com` and `router.utorrent.com` replied to nothing while a control datagram to a DNS server came back in 20 ms, and the third, `dht.transmissionbt.com`, is alive — it answered `find_node` with eight nodes — but on a host with global IPv6 its name resolves to an IPv6 address first while the DHT's socket is IPv4, so by name it was never reached. The list now carries a live node, and the names are resolved to IPv4 here rather than left to whichever family the host prefers. Measured on that host: 0 nodes after 21 s by name, 22 nodes in 5 s by address. The resolution is capped at two seconds per name, because it is awaited before the torrent client exists and a host whose resolver black-holes would otherwise hold that thread for c-ares' own four tries with nothing said. What it cost: on `JUFD665.mp4` the tracker answered `seeders=5` at 13:40:30 and the first peer connected at 13:44:47 — 4 min 17 s of a viewer watching an unexplained wait with an empty routing table beside them. A private torrent still ignores the DHT, by specification.
|
|
11
|
+
- **New**: The size of the DHT's routing table is said a minute after start, and an empty one is said as a warning. A bootstrap list rots — that is exactly what had happened here — and nothing reported it for as long as it was wrong, so the next list to die would be found the same way this one was: by hand, after a viewer waited four minutes.
|
|
12
|
+
|
|
1
13
|
## 2.50.0
|
|
2
14
|
|
|
3
15
|
- **Fix**: An AVI seek is asked for late enough to survive the container's own arithmetic. AVI names a keyframe by its frame NUMBER, and `services/container-index/avi.js` turned that into a time by multiplying by the frame duration the header declares — which lands 10-44 ms from the presentation time the demuxer computes, always under one frame (measured 2026-08-21 against the files themselves: 1196 index entries against 1196 real keyframes and 901 against 901, the frames exactly right and only their names off). A name sitting just BELOW its real keyframe seeks to before it and lands on the one before that, which is the same fault the landing offset already exists for. The reader now declares how far its times may be, and the request carries that on top. Matroska and MP4 declare nothing, because they state instants outright — nine files and 11 665 keyframes with not one disagreement.
|
package/package.json
CHANGED
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Accept the player's report that a delivered fragment sits far from the edge
|
|
3
|
+
* of what it has buffered.
|
|
4
|
+
*
|
|
5
|
+
* This is the earliest statement that exists of a stream coming apart, and
|
|
6
|
+
* until now it lived only in the browser's console. Measured 2026-08-21 on
|
|
7
|
+
* `JUFD665.mp4`: four of these across half a minute, each naming a gap of
|
|
8
|
+
* ~33.5 s, while the buffer stood still at 4571.1 s — and then hls.js gave up
|
|
9
|
+
* and jumped the viewer 16.8 s forward. The proxy is the only side that can say
|
|
10
|
+
* what the gap MEANS, because only it knows which boundary the segment it
|
|
11
|
+
* produced actually holds, and whether that is the one its number claims.
|
|
12
|
+
*
|
|
13
|
+
* POST /api/transcode-sessions/:sessionId/fragment-far
|
|
14
|
+
* Body: { sn: number, track?: string, fragStartSec: number, bufferEndSec: number, currentTimeSec: number }
|
|
15
|
+
*
|
|
16
|
+
* `track` is hls.js's own name for the stream the fragment belongs to ("main"
|
|
17
|
+
* or "audio"). It matters because picture and sound are produced by two
|
|
18
|
+
* different sessions positioned by two different runs — which is how they come
|
|
19
|
+
* apart in the first place — so a report answered against the wrong one is a
|
|
20
|
+
* confident statement about a stream nobody asked about.
|
|
21
|
+
*
|
|
22
|
+
* Diagnostic only — it changes nothing about the encode. Invalid body → 400,
|
|
23
|
+
* unknown session → 404, ok → 204.
|
|
24
|
+
*
|
|
25
|
+
* @param {import("fastify").FastifyRequest} req
|
|
26
|
+
* @param {import("fastify").FastifyReply} reply
|
|
27
|
+
* @param {{ hlsSessionManager: import("../../../../services/hls-session-manager.js").HlsSessionManager }} deps
|
|
28
|
+
* @returns {Promise<void>}
|
|
29
|
+
*/
|
|
30
|
+
export async function handleApiTranscodeSessionFragmentFarPost(req, reply, { hlsSessionManager }) {
|
|
31
|
+
const sessionId = typeof req.params.sessionId === "string" ? req.params.sessionId : "";
|
|
32
|
+
const body = req.body && typeof req.body === "object" && !Array.isArray(req.body) ? req.body : {};
|
|
33
|
+
const sn = Number(body.sn);
|
|
34
|
+
const track = typeof body.track === "string" ? body.track : "";
|
|
35
|
+
const fragStartSec = Number(body.fragStartSec);
|
|
36
|
+
const bufferEndSec = Number(body.bufferEndSec);
|
|
37
|
+
const currentTimeSec = Number(body.currentTimeSec);
|
|
38
|
+
if (
|
|
39
|
+
!sessionId ||
|
|
40
|
+
!Number.isInteger(sn) ||
|
|
41
|
+
sn < 0 ||
|
|
42
|
+
!Number.isFinite(fragStartSec) ||
|
|
43
|
+
!Number.isFinite(bufferEndSec) ||
|
|
44
|
+
!Number.isFinite(currentTimeSec)
|
|
45
|
+
) {
|
|
46
|
+
return reply.code(400).send({ error: "sn (integer >=0), fragStartSec, bufferEndSec and currentTimeSec are required." });
|
|
47
|
+
}
|
|
48
|
+
|
|
49
|
+
const recorded = hlsSessionManager.recordFragmentFar(sessionId, {
|
|
50
|
+
sn,
|
|
51
|
+
track,
|
|
52
|
+
fragStartSec,
|
|
53
|
+
bufferEndSec,
|
|
54
|
+
currentTimeSec
|
|
55
|
+
});
|
|
56
|
+
if (!recorded) {
|
|
57
|
+
return reply.code(404).send({ error: "Transcode session was not found." });
|
|
58
|
+
}
|
|
59
|
+
return reply.code(204).send();
|
|
60
|
+
}
|
package/server.js
CHANGED
|
@@ -26,6 +26,7 @@ import { handleApiTranscodeSessionsPost } from "./routes/api/transcode-sessions/
|
|
|
26
26
|
import { handleApiTranscodeSessionsProgressGet } from "./routes/api/transcode-sessions/progress/get.js";
|
|
27
27
|
import { handleApiTranscodeSessionReleasePost } from "./routes/api/transcode-sessions/release/post.js";
|
|
28
28
|
import { handleApiTranscodeSessionNetReportPost } from "./routes/api/transcode-sessions/net-report/post.js";
|
|
29
|
+
import { handleApiTranscodeSessionFragmentFarPost } from "./routes/api/transcode-sessions/fragment-far/post.js";
|
|
29
30
|
import { handleApiTranscodeSessionSeekPost } from "./routes/api/transcode-sessions/seek/post.js";
|
|
30
31
|
import { handleStreamGet } from "./routes/stream/get.js";
|
|
31
32
|
import { handleTranscodeSessionFileGet } from "./routes/transcode/session-file/get.js";
|
|
@@ -244,6 +245,9 @@ export async function startProxyServer({ host, port, transcodeAudio, ffmpegBin,
|
|
|
244
245
|
app.post("/api/transcode-sessions/:sessionId/net-report", async (req, reply) =>
|
|
245
246
|
handleApiTranscodeSessionNetReportPost(req, reply, { hlsSessionManager })
|
|
246
247
|
);
|
|
248
|
+
app.post("/api/transcode-sessions/:sessionId/fragment-far", async (req, reply) =>
|
|
249
|
+
handleApiTranscodeSessionFragmentFarPost(req, reply, { hlsSessionManager })
|
|
250
|
+
);
|
|
247
251
|
app.post("/api/transcode-sessions/:sessionId/seek", async (req, reply) =>
|
|
248
252
|
handleApiTranscodeSessionSeekPost(req, reply, { hlsSessionManager })
|
|
249
253
|
);
|
|
@@ -422,6 +422,12 @@ const LOOKAHEAD_RESUME_SECONDS = 60;
|
|
|
422
422
|
// encoding 125 s of content before reaching the viewer's position. Field
|
|
423
423
|
// 2026-08-02: a seek took 56 s, of which ~50 s was this backoff.
|
|
424
424
|
const SEEK_BACKOFF_SEGMENTS = 1;
|
|
425
|
+
|
|
426
|
+
// How many produced segments' true start times to remember, so a player's
|
|
427
|
+
// report about one of them can be answered. Two hundred is about twenty
|
|
428
|
+
// minutes of playback at these segment lengths — far more than the recent past
|
|
429
|
+
// a stall report can be about, and small enough to be free.
|
|
430
|
+
const TRUE_START_MEMORY = 200;
|
|
425
431
|
// How long to wait for a scrub to stop moving before acting on it. Small,
|
|
426
432
|
// because the browser already collapses a drag into ONE report
|
|
427
433
|
// (`SEEK_REPORT_DEBOUNCE_MS`, 300 ms) and only reports where it settled — this
|
|
@@ -1176,6 +1182,54 @@ export function segmentCutTimesFrom(boundaries, startIndex) {
|
|
|
1176
1182
|
return times;
|
|
1177
1183
|
}
|
|
1178
1184
|
|
|
1185
|
+
/**
|
|
1186
|
+
* How far the live boundary table has moved from the one the player holds, said
|
|
1187
|
+
* in words.
|
|
1188
|
+
*
|
|
1189
|
+
* The corrections are applied one boundary at a time and each is small enough
|
|
1190
|
+
* to look harmless; what nobody was watching is the total. It matters because a
|
|
1191
|
+
* run positioned on one table and cut on the other carries their difference into
|
|
1192
|
+
* every cut it makes — the fault of 2026-08-21, where the distance reached two
|
|
1193
|
+
* whole segments after one seek and four after the next. Printed beside each
|
|
1194
|
+
* correction so the total is visible while it is still small.
|
|
1195
|
+
*
|
|
1196
|
+
* @param {number[]} published - The table the playlist text was written from.
|
|
1197
|
+
* @param {number[]} live - The table corrected from produced segments.
|
|
1198
|
+
* @returns {string} A phrase, always readable, never throwing on odd input.
|
|
1199
|
+
*/
|
|
1200
|
+
export function describeGridDrift(published, live) {
|
|
1201
|
+
if (!Array.isArray(published) || !Array.isArray(live) || published.length === 0) {
|
|
1202
|
+
return "not comparable";
|
|
1203
|
+
}
|
|
1204
|
+
if (published.length !== live.length) {
|
|
1205
|
+
return `a different length (${published.length} against ${live.length})`;
|
|
1206
|
+
}
|
|
1207
|
+
let apart = 0;
|
|
1208
|
+
let worst = 0;
|
|
1209
|
+
let worstAt = -1;
|
|
1210
|
+
for (let index = 0; index < published.length; index += 1) {
|
|
1211
|
+
// Rounded to the millisecond BEFORE comparing, not only before printing.
|
|
1212
|
+
// Two boundaries moved by the same amount differ in the last bits of a
|
|
1213
|
+
// double, so an unrounded comparison picks between them by an accident
|
|
1214
|
+
// invisible in the printed figure — and the line would name a boundary the
|
|
1215
|
+
// reader cannot tell apart from the one before it. Rounded, ties keep the
|
|
1216
|
+
// earliest, which is also the one worth looking at first.
|
|
1217
|
+
const distance = Math.round(Math.abs(live[index] - published[index]) * 1000) / 1000;
|
|
1218
|
+
if (distance <= 0.001) {
|
|
1219
|
+
continue;
|
|
1220
|
+
}
|
|
1221
|
+
apart += 1;
|
|
1222
|
+
if (distance > worst) {
|
|
1223
|
+
worst = distance;
|
|
1224
|
+
worstAt = index;
|
|
1225
|
+
}
|
|
1226
|
+
}
|
|
1227
|
+
if (apart === 0) {
|
|
1228
|
+
return "identical";
|
|
1229
|
+
}
|
|
1230
|
+
return `${apart} of ${published.length} boundaries apart, worst ${worst.toFixed(3)}s at #${worstAt}`;
|
|
1231
|
+
}
|
|
1232
|
+
|
|
1179
1233
|
/**
|
|
1180
1234
|
* How much later than a keyframe to ASK, so that ffmpeg lands on that keyframe.
|
|
1181
1235
|
*
|
|
@@ -2550,6 +2604,22 @@ export class HlsSessionManager {
|
|
|
2550
2604
|
: (session.segmentBoundaries ?? []);
|
|
2551
2605
|
}
|
|
2552
2606
|
|
|
2607
|
+
/**
|
|
2608
|
+
* Where a run beginning at `index` must be positioned: the time the PLAYER
|
|
2609
|
+
* was told that segment starts at.
|
|
2610
|
+
*
|
|
2611
|
+
* Public because it is the invariant this class has broken twice, and a
|
|
2612
|
+
* private one cannot be pinned by a test. It must always be the table the cut
|
|
2613
|
+
* list is taken from — see the comment where a run is started.
|
|
2614
|
+
*
|
|
2615
|
+
* @param {HlsSession} session
|
|
2616
|
+
* @param {number} index
|
|
2617
|
+
* @returns {number}
|
|
2618
|
+
*/
|
|
2619
|
+
runStartTimeFor(session, index) {
|
|
2620
|
+
return this.#publishedStartTime(session, index);
|
|
2621
|
+
}
|
|
2622
|
+
|
|
2553
2623
|
#publishedStartTime(session, index) {
|
|
2554
2624
|
const boundaries = Array.isArray(session.publishedBoundaries) && session.publishedBoundaries.length > 0
|
|
2555
2625
|
? session.publishedBoundaries
|
|
@@ -2709,6 +2779,69 @@ export class HlsSessionManager {
|
|
|
2709
2779
|
return true;
|
|
2710
2780
|
}
|
|
2711
2781
|
|
|
2782
|
+
/**
|
|
2783
|
+
* Answer the player's report that a delivered fragment sits far from the edge
|
|
2784
|
+
* of its buffer, with the one fact only this side holds: which boundary the
|
|
2785
|
+
* segment of that number really begins at.
|
|
2786
|
+
*
|
|
2787
|
+
* The player can say the gap; it cannot say whether the cause is its own
|
|
2788
|
+
* loading or a run whose output no longer matches its numbering. Here both
|
|
2789
|
+
* are in hand — the time the playlist gave that segment, and, when the
|
|
2790
|
+
* segment has been served, the time it truly began at — so the line either
|
|
2791
|
+
* names a shifted run or clears this side of it.
|
|
2792
|
+
*
|
|
2793
|
+
* Diagnostic only: nothing is repositioned on the strength of a browser's
|
|
2794
|
+
* reading, deliberately, because a wrong answer here would restart an encoder
|
|
2795
|
+
* the viewer is waiting on.
|
|
2796
|
+
*
|
|
2797
|
+
* @param {string} sessionId
|
|
2798
|
+
* @param {{ sn: number, track?: string, fragStartSec: number, bufferEndSec: number, currentTimeSec: number }} report
|
|
2799
|
+
* @returns {boolean} False when no such session exists.
|
|
2800
|
+
*/
|
|
2801
|
+
recordFragmentFar(sessionId, { sn, track, fragStartSec, bufferEndSec, currentTimeSec }) {
|
|
2802
|
+
const named = this.sessionsById.get(sessionId);
|
|
2803
|
+
if (!named || named.state === "disposed") {
|
|
2804
|
+
return false;
|
|
2805
|
+
}
|
|
2806
|
+
// Which of the two streams the report is about. The browser addresses
|
|
2807
|
+
// everything to the video session's id — the soundtrack is served under
|
|
2808
|
+
// `/a/<n>/` on that same id — but it is a session of its own, with its own
|
|
2809
|
+
// run and its own position, and that is exactly the pair this report exists
|
|
2810
|
+
// to tell apart. Answering an audio report from the picture's records would
|
|
2811
|
+
// state, confidently, something about the wrong stream.
|
|
2812
|
+
const onScreen = this.#activeVariant(named);
|
|
2813
|
+
const session = track === "audio"
|
|
2814
|
+
? ([...this.#familyOf(onScreen)].find((member) => member.audioOnly === true) ?? onScreen)
|
|
2815
|
+
: onScreen;
|
|
2816
|
+
const gap = fragStartSec - bufferEndSec;
|
|
2817
|
+
const declared = this.#publishedStartTime(session, sn);
|
|
2818
|
+
const trueStart = session.trueStartByIndex instanceof Map ? session.trueStartByIndex.get(sn) : undefined;
|
|
2819
|
+
const verdict = trueStart === undefined
|
|
2820
|
+
// Where a segment truly began is only ever read off one that was cut on
|
|
2821
|
+
// an explicit list — a uniform grid has nothing to read back — so this is
|
|
2822
|
+
// "not recorded", which is not the same as "not produced", and the line
|
|
2823
|
+
// must not claim the second.
|
|
2824
|
+
? "where that segment began is not recorded on this side, so the gap cannot be attributed here"
|
|
2825
|
+
: (() => {
|
|
2826
|
+
const at = this.#boundaryIndexAt(session, trueStart, this.publishedGridFor(session));
|
|
2827
|
+
if (at === null) {
|
|
2828
|
+
return `it really began at ${trueStart.toFixed(3)}s, which is no boundary of this grid`;
|
|
2829
|
+
}
|
|
2830
|
+
if (at === sn) {
|
|
2831
|
+
return `it really began at boundary #${sn}, where it should — the gap is not this run's`;
|
|
2832
|
+
}
|
|
2833
|
+
return `it really began at boundary #${at}, ${sn - at} place(s) before its own number — ` +
|
|
2834
|
+
"this run's output does not match its numbering";
|
|
2835
|
+
})();
|
|
2836
|
+
logger.warn(
|
|
2837
|
+
`transcode ${session.id} the player is stuck: ${session.audioOnly === true ? "sound" : "picture"} ` +
|
|
2838
|
+
`fragment #${sn} starts ${gap.toFixed(1)}s past ` +
|
|
2839
|
+
`the end of its buffer (${bufferEndSec.toFixed(1)}s, viewer at ${currentTimeSec.toFixed(1)}s, ` +
|
|
2840
|
+
`the playlist puts it at ${declared.toFixed(3)}s) — ${verdict}`
|
|
2841
|
+
);
|
|
2842
|
+
return true;
|
|
2843
|
+
}
|
|
2844
|
+
|
|
2712
2845
|
/**
|
|
2713
2846
|
* Observed produced bitrate (Mbit/s) averaged over the last few COMPLETED
|
|
2714
2847
|
* segment files (the newest file may still be being written and is
|
|
@@ -3756,7 +3889,7 @@ export class HlsSessionManager {
|
|
|
3756
3889
|
const head = session.encodeStartIndex;
|
|
3757
3890
|
const processed = Number.isFinite(session.progress?.processedSeconds)
|
|
3758
3891
|
? session.progress.processedSeconds
|
|
3759
|
-
: this
|
|
3892
|
+
: this.runStartTimeFor(session, head);
|
|
3760
3893
|
const currentSeg = Math.max(head, this.#segmentIndexForTime(session, processed));
|
|
3761
3894
|
const boundLabel =
|
|
3762
3895
|
bound === "link" ? "viewer-link-bound" : bound === "unknown" ? "assuming CPU-bound" : "CPU-bound";
|
|
@@ -3798,9 +3931,18 @@ export class HlsSessionManager {
|
|
|
3798
3931
|
*
|
|
3799
3932
|
* @param {HlsSession} session
|
|
3800
3933
|
* @param {number} startIndex
|
|
3934
|
+
* @param {number} [positionSecondsOverride] - Begin the run at this instant
|
|
3935
|
+
* instead of at the time the playlist gives `startIndex`, keeping the
|
|
3936
|
+
* numbering and the cut list on the published grid. The one caller is the
|
|
3937
|
+
* realignment below: a COPIED picture cannot cut anywhere but at the
|
|
3938
|
+
* source's own keyframes, so its segment #N begins where the file says and
|
|
3939
|
+
* not where the grid does — and the sound that plays with it has to begin
|
|
3940
|
+
* at that same instant, or the two are apart by the difference. The output
|
|
3941
|
+
* is still labelled from the source clock (`-copyts`) and stamped on serve,
|
|
3942
|
+
* so the player sees both at the time the playlist names.
|
|
3801
3943
|
* @returns {Promise<void>}
|
|
3802
3944
|
*/
|
|
3803
|
-
async #startEncodeRun(session, startIndex) {
|
|
3945
|
+
async #startEncodeRun(session, startIndex, positionSecondsOverride) {
|
|
3804
3946
|
// A new run starts its own reckoning: a pair spanning the restart would
|
|
3805
3947
|
// count the gap between two runs as slow encoding.
|
|
3806
3948
|
session.learnSample = null;
|
|
@@ -3899,8 +4041,30 @@ export class HlsSessionManager {
|
|
|
3899
4041
|
}
|
|
3900
4042
|
|
|
3901
4043
|
const safeIndex = Number.isInteger(startIndex) && startIndex > 0 ? startIndex : 0;
|
|
3902
|
-
// 0-based output time of this segment, from the
|
|
3903
|
-
|
|
4044
|
+
// 0-based output time of this segment, from the table the PLAYER holds —
|
|
4045
|
+
// the same one the cut list below is taken from.
|
|
4046
|
+
//
|
|
4047
|
+
// These two were read from different tables until 2026-08-21, and that is
|
|
4048
|
+
// one fault, not two: `-segment_times` are measured from wherever the run
|
|
4049
|
+
// really began, so any distance between the position and the cut list moves
|
|
4050
|
+
// EVERY cut of that run by it. The live table keeps being corrected as
|
|
4051
|
+
// produced segments reveal where the file's cuts truly are, and those
|
|
4052
|
+
// corrections run backwards, so each restart began a little earlier than
|
|
4053
|
+
// the grid the cuts were stated on — and since the corrections accumulate,
|
|
4054
|
+
// so did the distance. Measured on `JUFD665.mp4`: after one seek restart a
|
|
4055
|
+
// produced segment held the boundary two places before its own number
|
|
4056
|
+
// (16.684 s, exactly 2.0000 segments), after the next it held the one four
|
|
4057
|
+
// places before (33.5 s). The player's buffer then stops extending at all,
|
|
4058
|
+
// because the content of every fragment lands before the time its playlist
|
|
4059
|
+
// entry names: `bufferEnd` stood still at 4571.1 s through four `frag-far`
|
|
4060
|
+
// warnings until hls.js gave up and jumped the viewer 16.8 s forward.
|
|
4061
|
+
//
|
|
4062
|
+
// 2.45.0 moved the CUT LIST onto the published table for this same reason
|
|
4063
|
+
// and left the position on the live one. Both belong on the published
|
|
4064
|
+
// table: a run must begin where the player was told the segment begins.
|
|
4065
|
+
const startSeconds = Number.isFinite(positionSecondsOverride)
|
|
4066
|
+
? positionSecondsOverride
|
|
4067
|
+
: this.runStartTimeFor(session, safeIndex);
|
|
3904
4068
|
const sourceStartTime = Number.isFinite(session.sourceStartTime) ? session.sourceStartTime : 0;
|
|
3905
4069
|
// Cut where this session's grid says, whoever is producing the frames. The
|
|
3906
4070
|
// times are measured from the start of THIS run; the same list serves as
|
|
@@ -4192,6 +4356,17 @@ export class HlsSessionManager {
|
|
|
4192
4356
|
`(+${Date.now() - restartEnteredAt}ms since the restart was asked for) ` +
|
|
4193
4357
|
`(${formatSeconds(startSeconds)}) "${session.fileName}"`
|
|
4194
4358
|
);
|
|
4359
|
+
// The four numbers a run is positioned by, said once, because their
|
|
4360
|
+
// disagreement is invisible everywhere else. The two tables are printed
|
|
4361
|
+
// side by side: while they differ, every cut of this run is off by the
|
|
4362
|
+
// difference, and nothing downstream can tell that from a bad index.
|
|
4363
|
+
const liveStart = this.#segmentStartTime(session, safeIndex);
|
|
4364
|
+
logger.info(
|
|
4365
|
+
`transcode ${session.id} ${session.runLabel} positioned at ${startSeconds.toFixed(3)}s ` +
|
|
4366
|
+
`for boundary #${safeIndex} (published ${startSeconds.toFixed(3)}s, ` +
|
|
4367
|
+
`live ${liveStart.toFixed(3)}s, apart ${(liveStart - startSeconds).toFixed(3)}s), ` +
|
|
4368
|
+
`numbering from #${safeIndex}`
|
|
4369
|
+
);
|
|
4195
4370
|
|
|
4196
4371
|
this.#wireEncodeProcess(session, ffmpeg);
|
|
4197
4372
|
}
|
|
@@ -4511,7 +4686,7 @@ export class HlsSessionManager {
|
|
|
4511
4686
|
// request just ahead of the live edge.
|
|
4512
4687
|
const processed = Number.isFinite(session.progress?.processedSeconds)
|
|
4513
4688
|
? session.progress.processedSeconds
|
|
4514
|
-
: this
|
|
4689
|
+
: this.runStartTimeFor(session, head);
|
|
4515
4690
|
const currentSeg = Math.max(head, this.#segmentIndexForTime(session, processed));
|
|
4516
4691
|
const withinWindow = index >= head && index <= currentSeg + MAX_LOOKAHEAD_SEGMENTS;
|
|
4517
4692
|
if (withinWindow) {
|
|
@@ -4804,7 +4979,7 @@ export class HlsSessionManager {
|
|
|
4804
4979
|
const head = session.encodeStartIndex;
|
|
4805
4980
|
const processed = Number.isFinite(session.progress?.processedSeconds)
|
|
4806
4981
|
? session.progress.processedSeconds
|
|
4807
|
-
: this
|
|
4982
|
+
: this.runStartTimeFor(session, head);
|
|
4808
4983
|
const currentSeg = Math.max(head, this.#segmentIndexForTime(session, processed));
|
|
4809
4984
|
// Already covered by the running encode — the data is on its way, so
|
|
4810
4985
|
// restarting would only destroy work the viewer is waiting for. The run has
|
|
@@ -5240,7 +5415,17 @@ export class HlsSessionManager {
|
|
|
5240
5415
|
return;
|
|
5241
5416
|
}
|
|
5242
5417
|
session.landingReportedForRun = index;
|
|
5243
|
-
|
|
5418
|
+
// What the run was ASKED for, taken from the run itself rather than looked
|
|
5419
|
+
// up again in a table. The two used to be the same lookup; they stopped
|
|
5420
|
+
// being so when a run began positioning on the published grid while this
|
|
5421
|
+
// read the live one, which made a perfect landing report a drift equal to
|
|
5422
|
+
// the distance between the tables — and cancelled a real landing error of
|
|
5423
|
+
// the same size to zero. A run also has one legitimate position that is in
|
|
5424
|
+
// no table at all: the realignment that starts the sound where the copied
|
|
5425
|
+
// picture truly begins.
|
|
5426
|
+
const asked = Number.isFinite(session.progress?.startPositionSeconds)
|
|
5427
|
+
? session.progress.startPositionSeconds
|
|
5428
|
+
: this.runStartTimeFor(session, index);
|
|
5244
5429
|
const drift = trueStart - asked;
|
|
5245
5430
|
if (!Number.isFinite(drift) || Math.abs(drift) <= PLAYER_BUFFER_HOLE_SEC) {
|
|
5246
5431
|
return;
|
|
@@ -5255,6 +5440,18 @@ export class HlsSessionManager {
|
|
|
5255
5440
|
#noteIndexAccuracy(session, index, trueStart, declaredStart) {
|
|
5256
5441
|
const deviation = Math.abs(trueStart - declaredStart);
|
|
5257
5442
|
session.indexCheck ??= newIndexCheck();
|
|
5443
|
+
// Where each produced segment truly began, kept so that a player reporting
|
|
5444
|
+
// a stall can be ANSWERED rather than merely believed. Bounded: only the
|
|
5445
|
+
// recent past can be the subject of such a report, and an unbounded map on
|
|
5446
|
+
// a two-hour film is a leak.
|
|
5447
|
+
session.trueStartByIndex ??= new Map();
|
|
5448
|
+
session.trueStartByIndex.set(index, trueStart);
|
|
5449
|
+
if (session.trueStartByIndex.size > TRUE_START_MEMORY) {
|
|
5450
|
+
const oldest = session.trueStartByIndex.keys().next();
|
|
5451
|
+
if (!oldest.done) {
|
|
5452
|
+
session.trueStartByIndex.delete(oldest.value);
|
|
5453
|
+
}
|
|
5454
|
+
}
|
|
5258
5455
|
// Did this segment begin at ANOTHER keyframe from the same list? Half an
|
|
5259
5456
|
// audio frame is the tolerance — anything the list names is exact, so a
|
|
5260
5457
|
// match is a match. `keyframeTimes` is the list the grid was built from, so
|
|
@@ -5299,7 +5496,18 @@ export class HlsSessionManager {
|
|
|
5299
5496
|
// rung's segments no longer stand where the stream it accompanies
|
|
5300
5497
|
// would have put them. That is a broken splice, not a wrong index.
|
|
5301
5498
|
? "this rung did not cut where its grid says; a switch to it will not join cleanly"
|
|
5302
|
-
|
|
5499
|
+
// A copy. The two possible faults need opposite fixes and the
|
|
5500
|
+
// numbers already tell them apart, so the sentence follows THEM
|
|
5501
|
+
// rather than the branch it is printed from. Until 2026-08-21 it
|
|
5502
|
+
// blamed the index either way — including through a session whose
|
|
5503
|
+
// segments each held the boundary two, then four places before
|
|
5504
|
+
// their own number, which is this code's own definition of a fault
|
|
5505
|
+
// in this code. That sentence is what sent the reading of that
|
|
5506
|
+
// session after the file instead of after the arithmetic.
|
|
5507
|
+
: at === null
|
|
5508
|
+
? "the container's keyframe index disagrees with the file; using the file"
|
|
5509
|
+
: `this began at another boundary of the same list, ${index - at} place(s) before ` +
|
|
5510
|
+
"its own number — the numbering of this run is shifted, not the index")
|
|
5303
5511
|
);
|
|
5304
5512
|
}
|
|
5305
5513
|
}
|
|
@@ -5382,7 +5590,9 @@ export class HlsSessionManager {
|
|
|
5382
5590
|
}
|
|
5383
5591
|
logger.info(
|
|
5384
5592
|
`transcode ${session.id} boundary #${index} corrected ${wasAt.toFixed(3)}s → ` +
|
|
5385
|
-
`${trueStart.toFixed(3)}s from the file itself`
|
|
5593
|
+
`${trueStart.toFixed(3)}s from the file itself` +
|
|
5594
|
+
`, and the live table is now ${describeGridDrift(this.publishedGridFor(session), boundaries)}` +
|
|
5595
|
+
" from the one the player holds"
|
|
5386
5596
|
);
|
|
5387
5597
|
// And every OTHER member whose run begins at this very boundary is moved
|
|
5388
5598
|
// to the same instant.
|
|
@@ -5418,9 +5628,18 @@ export class HlsSessionManager {
|
|
|
5418
5628
|
// time: a seek decides by index, finds this run already begins at #index,
|
|
5419
5629
|
// and answers "already within the running encode" — which is true about
|
|
5420
5630
|
// the index and false about the instant, and it is why the first version
|
|
5421
|
-
// of this fix moved nothing at all.
|
|
5422
|
-
//
|
|
5423
|
-
|
|
5631
|
+
// of this fix moved nothing at all.
|
|
5632
|
+
//
|
|
5633
|
+
// The instant is passed EXPLICITLY. It used to be smuggled through the
|
|
5634
|
+
// live boundary table — this function had just written `trueStart` into
|
|
5635
|
+
// it, and the run read its position from there — which stopped working
|
|
5636
|
+
// the moment a run began positioning itself on the table the player
|
|
5637
|
+
// holds, as it now must. Smuggled, the restart would land exactly where
|
|
5638
|
+
// it already was: picture and sound would stay apart and a healthy audio
|
|
5639
|
+
// run would be discarded for nothing, which is the shape the field
|
|
5640
|
+
// already showed (eleven restarts in four minutes, eight of them dying
|
|
5641
|
+
// with `run had produced 0.0s`).
|
|
5642
|
+
void this.#startEncodeRun(member, index, trueStart).catch(() => {});
|
|
5424
5643
|
}
|
|
5425
5644
|
}
|
|
5426
5645
|
|
|
@@ -5503,8 +5722,14 @@ export class HlsSessionManager {
|
|
|
5503
5722
|
* @param {number} seconds
|
|
5504
5723
|
* @returns {number | null}
|
|
5505
5724
|
*/
|
|
5506
|
-
#boundaryIndexAt(session, seconds) {
|
|
5507
|
-
|
|
5725
|
+
#boundaryIndexAt(session, seconds, table) {
|
|
5726
|
+
// The table is nameable because the two answer different questions. The
|
|
5727
|
+
// LIVE one says "is this a cut this file actually has", which is what a
|
|
5728
|
+
// reading taken off a produced segment is about. The PUBLISHED one says "is
|
|
5729
|
+
// this a cut the player believes in", which is what a report from the player
|
|
5730
|
+
// is about. Answering one with the other prints an index from one grid
|
|
5731
|
+
// beside a time from the other.
|
|
5732
|
+
const boundaries = Array.isArray(table) ? table : session.segmentBoundaries;
|
|
5508
5733
|
if (!Array.isArray(boundaries)) {
|
|
5509
5734
|
return null;
|
|
5510
5735
|
}
|
|
@@ -7985,7 +8210,14 @@ export class HlsSessionManager {
|
|
|
7985
8210
|
// causes indistinguishable: an encoder waiting for torrent pieces looks
|
|
7986
8211
|
// exactly like one that is encoding and simply has not finished. The
|
|
7987
8212
|
// difference is whether its position has moved at all.
|
|
7988
|
-
|
|
8213
|
+
// Where this run began, from the run — the same reckoning
|
|
8214
|
+
// `processedSeconds` is counted in. A table lookup here can disagree with
|
|
8215
|
+
// it by the distance between the two grids, which is enough to print a
|
|
8216
|
+
// negative "produced" and send the reader after the torrent when the
|
|
8217
|
+
// encoder is the subject.
|
|
8218
|
+
const runStartSeconds = Number.isFinite(session.progress?.startPositionSeconds)
|
|
8219
|
+
? session.progress.startPositionSeconds
|
|
8220
|
+
: this.runStartTimeFor(session, session.encodeStartIndex ?? 0);
|
|
7989
8221
|
const position = Number(session.progress?.processedSeconds);
|
|
7990
8222
|
const produced = Number.isFinite(position) ? position - runStartSeconds : null;
|
|
7991
8223
|
const speed = session.progress?.speed ?? "n/a";
|
package/services/torrent-pool.js
CHANGED
|
@@ -7,6 +7,7 @@
|
|
|
7
7
|
*/
|
|
8
8
|
|
|
9
9
|
import crypto from "node:crypto";
|
|
10
|
+
import dns from "node:dns/promises";
|
|
10
11
|
import os from "node:os";
|
|
11
12
|
import path from "node:path";
|
|
12
13
|
import { rmSync, statfsSync } from "node:fs";
|
|
@@ -14,6 +15,39 @@ import WebTorrent from "webtorrent";
|
|
|
14
15
|
import { logger } from "../utils/logger.js";
|
|
15
16
|
import { SharedPieceStore, findSharedStore } from "./piece-store/shared-piece-store.js";
|
|
16
17
|
|
|
18
|
+
// The DHT's entry points. Two of the three the library ships answer nothing —
|
|
19
|
+
// measured 2026-08-21 from the addon host: `router.bittorrent.com` and
|
|
20
|
+
// `router.utorrent.com` did not reply to a hand-written `ping` at all, while a
|
|
21
|
+
// control datagram to a DNS server came back in 20 ms, so the silence is
|
|
22
|
+
// theirs. They stay in the list because they cost nothing and may come back;
|
|
23
|
+
// what the list needed was entries that answer today.
|
|
24
|
+
//
|
|
25
|
+
// The names are RESOLVED HERE, to IPv4, and the addresses are what the library
|
|
26
|
+
// is given. That is the second half of the fault: `dht.transmissionbt.com` is
|
|
27
|
+
// alive — it answered `find_node` with eight nodes — but on a host with global
|
|
28
|
+
// IPv6 its name resolves to an IPv6 address first, and the DHT's socket is
|
|
29
|
+
// IPv4, so by name the one live entry was never reached. Measured the same day:
|
|
30
|
+
// by name, 0 nodes after 21 s, every run; by address, 22 nodes in 5 s.
|
|
31
|
+
//
|
|
32
|
+
// Addresses are not written down. They rot exactly as the old list rotted.
|
|
33
|
+
const DHT_BOOTSTRAP_NAMES = [
|
|
34
|
+
"dht.transmissionbt.com:6881",
|
|
35
|
+
"dht.libtorrent.org:25401",
|
|
36
|
+
"router.bittorrent.com:6881",
|
|
37
|
+
"router.utorrent.com:6881"
|
|
38
|
+
];
|
|
39
|
+
|
|
40
|
+
// How long after start a still-empty routing table is worth saying out loud.
|
|
41
|
+
// Bootstrapping takes seconds; a table empty after this is a list that has died
|
|
42
|
+
// and nobody has noticed, which is the state this host was found in.
|
|
43
|
+
const DHT_EMPTY_REPORT_MS = 60 * 1000;
|
|
44
|
+
|
|
45
|
+
// How long one bootstrap name may take to resolve. This is awaited before the
|
|
46
|
+
// torrent client is built, so it is time during which the thread answers
|
|
47
|
+
// nothing — and the DHT is best-effort, so a name that is slow to resolve is
|
|
48
|
+
// worth less than the delay of waiting for it.
|
|
49
|
+
const DHT_RESOLVE_TIMEOUT_MS = 2000;
|
|
50
|
+
|
|
17
51
|
// WebTorrent's default download root (see webtorrent lib/torrent.js: TMP =
|
|
18
52
|
// path.join(os.tmpdir(), 'webtorrent')). We use the default store, so all
|
|
19
53
|
// torrent data lives under here.
|
|
@@ -449,6 +483,103 @@ function isMagnetSource(torrentId) {
|
|
|
449
483
|
return typeof torrentId === "string" && /^magnet:\?/i.test(torrentId.trim());
|
|
450
484
|
}
|
|
451
485
|
|
|
486
|
+
/**
|
|
487
|
+
* Split a `host` or `host:port` bootstrap entry.
|
|
488
|
+
*
|
|
489
|
+
* @param {unknown} entry
|
|
490
|
+
* @returns {{ host: string, port: number } | null} Null for anything that is
|
|
491
|
+
* not a usable entry, so a typo drops one node instead of failing the client.
|
|
492
|
+
*/
|
|
493
|
+
export function parseBootstrapEntry(entry) {
|
|
494
|
+
if (typeof entry !== "string") {
|
|
495
|
+
return null;
|
|
496
|
+
}
|
|
497
|
+
const trimmed = entry.trim();
|
|
498
|
+
if (trimmed.length === 0) {
|
|
499
|
+
return null;
|
|
500
|
+
}
|
|
501
|
+
const colon = trimmed.lastIndexOf(":");
|
|
502
|
+
if (colon < 0) {
|
|
503
|
+
return { host: trimmed, port: 6881 };
|
|
504
|
+
}
|
|
505
|
+
const host = trimmed.slice(0, colon);
|
|
506
|
+
const port = Number(trimmed.slice(colon + 1));
|
|
507
|
+
if (host.length === 0 || !Number.isInteger(port) || port <= 0 || port > 65535) {
|
|
508
|
+
return null;
|
|
509
|
+
}
|
|
510
|
+
return { host, port };
|
|
511
|
+
}
|
|
512
|
+
|
|
513
|
+
/**
|
|
514
|
+
* Turn bootstrap names into `address:port` entries, keeping only IPv4.
|
|
515
|
+
*
|
|
516
|
+
* Why the resolution happens here rather than being left to the library: the
|
|
517
|
+
* DHT's socket is IPv4, and a name that resolves to IPv6 first is silently
|
|
518
|
+
* unreachable through it — which is how a live bootstrap node came to look
|
|
519
|
+
* dead on this host. See {@link DHT_BOOTSTRAP_NAMES}.
|
|
520
|
+
*
|
|
521
|
+
* Best-effort by construction. A name that does not resolve is dropped and
|
|
522
|
+
* said; if none resolve the caller gets an empty list and the library keeps its
|
|
523
|
+
* own defaults, which is no worse than before.
|
|
524
|
+
*
|
|
525
|
+
* @param {string[]} [names]
|
|
526
|
+
* @param {number} [timeoutMs] - How long one name may take before it is
|
|
527
|
+
* treated as unresolvable.
|
|
528
|
+
* @returns {Promise<string[]>}
|
|
529
|
+
*/
|
|
530
|
+
export async function resolveDhtBootstrap(names = DHT_BOOTSTRAP_NAMES, timeoutMs = DHT_RESOLVE_TIMEOUT_MS) {
|
|
531
|
+
const entries = names.map(parseBootstrapEntry).filter((entry) => entry !== null);
|
|
532
|
+
const resolved = [];
|
|
533
|
+
const failed = [];
|
|
534
|
+
await Promise.all(
|
|
535
|
+
entries.map(async ({ host, port }) => {
|
|
536
|
+
try {
|
|
537
|
+
// Capped, because this is awaited before the torrent client exists and
|
|
538
|
+
// therefore before this thread will answer anything. `dns.resolve4`
|
|
539
|
+
// talks to the host's resolver with c-ares' own defaults — about five
|
|
540
|
+
// seconds times four tries — so a black-holing resolver would hold the
|
|
541
|
+
// whole worker for twenty seconds with nothing said. A name that does
|
|
542
|
+
// not answer inside the cap is treated exactly like one that fails:
|
|
543
|
+
// dropped, named, and the rest of the list stands.
|
|
544
|
+
const addresses = await Promise.race([
|
|
545
|
+
dns.resolve4(host),
|
|
546
|
+
new Promise((_resolve, reject) => {
|
|
547
|
+
const timer = setTimeout(() => reject(new Error("timed out")), timeoutMs);
|
|
548
|
+
timer.unref?.();
|
|
549
|
+
})
|
|
550
|
+
]);
|
|
551
|
+
for (const address of addresses) {
|
|
552
|
+
resolved.push(`${address}:${port}`);
|
|
553
|
+
}
|
|
554
|
+
} catch {
|
|
555
|
+
failed.push(`${host}:${port}`);
|
|
556
|
+
}
|
|
557
|
+
})
|
|
558
|
+
);
|
|
559
|
+
if (failed.length > 0) {
|
|
560
|
+
logger.warn(`torrent-pool: DHT bootstrap names that do not resolve to IPv4: ${failed.join(", ")}`);
|
|
561
|
+
}
|
|
562
|
+
return resolved;
|
|
563
|
+
}
|
|
564
|
+
|
|
565
|
+
/**
|
|
566
|
+
* How many nodes the client's DHT knows, or null when it has no DHT at all.
|
|
567
|
+
*
|
|
568
|
+
* @param {{ dht?: { nodes?: { toArray?: () => unknown[] } } }} client
|
|
569
|
+
* @returns {number | null}
|
|
570
|
+
*/
|
|
571
|
+
export function dhtNodeCount(client) {
|
|
572
|
+
const nodes = client?.dht?.nodes;
|
|
573
|
+
if (!nodes || typeof nodes.toArray !== "function") {
|
|
574
|
+
return null;
|
|
575
|
+
}
|
|
576
|
+
try {
|
|
577
|
+
return nodes.toArray().length;
|
|
578
|
+
} catch {
|
|
579
|
+
return null;
|
|
580
|
+
}
|
|
581
|
+
}
|
|
582
|
+
|
|
452
583
|
export class TorrentPool {
|
|
453
584
|
/**
|
|
454
585
|
* In-flight `client.add()` promises keyed by the same key as `torrents`.
|
|
@@ -521,13 +652,16 @@ export class TorrentPool {
|
|
|
521
652
|
/** Periodic adaptive-upload adjustment timer. */
|
|
522
653
|
#uploadAdjustTimer = null;
|
|
523
654
|
|
|
655
|
+
/** One-shot timer that reports the size of the DHT's routing table. */
|
|
656
|
+
#dhtReportTimer = null;
|
|
657
|
+
|
|
524
658
|
/**
|
|
525
659
|
* @param {{ maxDiskBytes?: number }} [options]
|
|
526
660
|
* `maxDiskBytes` caps total downloaded torrent data; when omitted a
|
|
527
661
|
* default is computed from free disk (min(10 GB, half free)). Pass 0 to
|
|
528
662
|
* disable the cap.
|
|
529
663
|
*/
|
|
530
|
-
constructor({ maxDiskBytes, memoryBytes } = {}) {
|
|
664
|
+
constructor({ maxDiskBytes, memoryBytes, dhtBootstrap } = {}) {
|
|
531
665
|
this.#memoryBytes = Number.isFinite(memoryBytes) && memoryBytes > 0 ? memoryBytes : undefined;
|
|
532
666
|
|
|
533
667
|
// Sweep orphaned torrent data left by a previous hard kill (no graceful
|
|
@@ -541,8 +675,33 @@ export class TorrentPool {
|
|
|
541
675
|
logger.warn(`torrent-pool: could not sweep orphaned store at startup: ${message}`);
|
|
542
676
|
}
|
|
543
677
|
|
|
678
|
+
const bootstrap = Array.isArray(dhtBootstrap) ? dhtBootstrap.filter(Boolean) : [];
|
|
679
|
+
if (bootstrap.length > 0) {
|
|
680
|
+
logger.info(`torrent-pool: DHT bootstrap nodes: ${bootstrap.join(", ")}`);
|
|
681
|
+
} else {
|
|
682
|
+
logger.warn("torrent-pool: no DHT bootstrap nodes resolved; the library's own list is all there is");
|
|
683
|
+
}
|
|
544
684
|
/** @type {import("webtorrent").WebTorrent} */
|
|
545
|
-
this.client = new WebTorrent();
|
|
685
|
+
this.client = new WebTorrent(bootstrap.length > 0 ? { dht: { bootstrap } } : undefined);
|
|
686
|
+
// A bootstrap list rots, and it rots silently: the one the library ships
|
|
687
|
+
// had two dead entries and a third that could not be reached by name, and
|
|
688
|
+
// nothing said so for as long as that was true. Say it once, late enough
|
|
689
|
+
// that a slow bootstrap is not mistaken for a dead one.
|
|
690
|
+
this.#dhtReportTimer = setTimeout(() => {
|
|
691
|
+
const nodes = dhtNodeCount(this.client);
|
|
692
|
+
if (nodes === null) {
|
|
693
|
+
return;
|
|
694
|
+
}
|
|
695
|
+
if (nodes === 0) {
|
|
696
|
+
logger.warn(
|
|
697
|
+
"torrent-pool: the DHT knows no nodes a minute after start — its bootstrap list is not " +
|
|
698
|
+
"answering, so a torrent has only its trackers to find peers with"
|
|
699
|
+
);
|
|
700
|
+
return;
|
|
701
|
+
}
|
|
702
|
+
logger.info(`torrent-pool: the DHT knows ${nodes} nodes`);
|
|
703
|
+
}, DHT_EMPTY_REPORT_MS);
|
|
704
|
+
this.#dhtReportTimer.unref?.();
|
|
546
705
|
|
|
547
706
|
/**
|
|
548
707
|
* Active torrents keyed by `"${sourceType}:${sha1(source)}"`.
|
|
@@ -1681,6 +1840,10 @@ export class TorrentPool {
|
|
|
1681
1840
|
clearInterval(this.#uploadAdjustTimer);
|
|
1682
1841
|
this.#uploadAdjustTimer = null;
|
|
1683
1842
|
}
|
|
1843
|
+
if (this.#dhtReportTimer) {
|
|
1844
|
+
clearTimeout(this.#dhtReportTimer);
|
|
1845
|
+
this.#dhtReportTimer = null;
|
|
1846
|
+
}
|
|
1684
1847
|
// Cancel any pending idle-removal timers — destroyAll handles teardown.
|
|
1685
1848
|
for (const timer of this.#idleTimers.values()) {
|
|
1686
1849
|
clearTimeout(timer);
|
|
@@ -35,12 +35,19 @@ import { Command, Event } from "./protocol.js";
|
|
|
35
35
|
// would drag in WebTorrent — and with it the real `webrtc-polyfill` — before
|
|
36
36
|
// the hook above had a chance to register. Verified the hard way: with a static
|
|
37
37
|
// import the process still aborted, and the stack named the genuine polyfill.
|
|
38
|
-
const { TorrentPool } = await import("../torrent-pool.js");
|
|
38
|
+
const { TorrentPool, resolveDhtBootstrap } = await import("../torrent-pool.js");
|
|
39
39
|
const { collectStoreStats, findSharedStore } = await import("../piece-store/shared-piece-store.js");
|
|
40
40
|
|
|
41
|
+
// Resolved before the client exists, because the client builds its DHT in its
|
|
42
|
+
// own constructor and the addresses have to be in hand by then. Awaiting here
|
|
43
|
+
// costs the few milliseconds of a DNS answer, once, on a thread that has not
|
|
44
|
+
// been asked for anything yet.
|
|
45
|
+
const dhtBootstrap = await resolveDhtBootstrap();
|
|
46
|
+
|
|
41
47
|
const pool = new TorrentPool({
|
|
42
48
|
maxDiskBytes: workerData?.maxDiskBytes,
|
|
43
|
-
memoryBytes: workerData?.memoryBytes
|
|
49
|
+
memoryBytes: workerData?.memoryBytes,
|
|
50
|
+
dhtBootstrap
|
|
44
51
|
});
|
|
45
52
|
|
|
46
53
|
/** Torrents by sourceKey — the main thread names them, this thread owns them. */
|
|
@@ -0,0 +1,72 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @file The DHT's entry points are given as addresses, not names.
|
|
3
|
+
*
|
|
4
|
+
* Measured 2026-08-21 on the addon host. Two of the three bootstrap nodes the
|
|
5
|
+
* library ships answer nothing: `router.bittorrent.com` and `router.utorrent.com`
|
|
6
|
+
* did not reply to a hand-written `ping` at all, while a control datagram to a
|
|
7
|
+
* DNS server came back in 20 ms. The third, `dht.transmissionbt.com`, is alive —
|
|
8
|
+
* it answered `find_node` with eight nodes — but on a host with global IPv6 its
|
|
9
|
+
* name resolves to an IPv6 address first, and the DHT's socket is IPv4, so by
|
|
10
|
+
* name it was never reached. By name: 0 nodes after 21 s, every run. By address:
|
|
11
|
+
* 22 nodes in 5 s.
|
|
12
|
+
*/
|
|
13
|
+
|
|
14
|
+
import assert from "node:assert/strict";
|
|
15
|
+
import test from "node:test";
|
|
16
|
+
|
|
17
|
+
import { dhtNodeCount, parseBootstrapEntry, resolveDhtBootstrap } from "../services/torrent-pool.js";
|
|
18
|
+
|
|
19
|
+
test("a bootstrap entry is split into host and port", () => {
|
|
20
|
+
assert.deepEqual(parseBootstrapEntry("dht.libtorrent.org:25401"), {
|
|
21
|
+
host: "dht.libtorrent.org",
|
|
22
|
+
port: 25401
|
|
23
|
+
});
|
|
24
|
+
});
|
|
25
|
+
|
|
26
|
+
test("an entry with no port gets the DHT's own", () => {
|
|
27
|
+
assert.deepEqual(parseBootstrapEntry("router.bittorrent.com"), {
|
|
28
|
+
host: "router.bittorrent.com",
|
|
29
|
+
port: 6881
|
|
30
|
+
});
|
|
31
|
+
});
|
|
32
|
+
|
|
33
|
+
test("an unusable entry is dropped rather than breaking the client", () => {
|
|
34
|
+
// One bad line in the list must cost that node and nothing else — the DHT is
|
|
35
|
+
// best-effort by nature and a typo here would otherwise take the whole client
|
|
36
|
+
// down at construction.
|
|
37
|
+
assert.equal(parseBootstrapEntry(""), null);
|
|
38
|
+
assert.equal(parseBootstrapEntry(" "), null);
|
|
39
|
+
assert.equal(parseBootstrapEntry("host:0"), null);
|
|
40
|
+
assert.equal(parseBootstrapEntry("host:70000"), null);
|
|
41
|
+
assert.equal(parseBootstrapEntry("host:not-a-port"), null);
|
|
42
|
+
assert.equal(parseBootstrapEntry(":6881"), null);
|
|
43
|
+
assert.equal(parseBootstrapEntry(null), null);
|
|
44
|
+
assert.equal(parseBootstrapEntry(42), null);
|
|
45
|
+
});
|
|
46
|
+
|
|
47
|
+
test("the routing table's size is readable, and its absence is not an error", () => {
|
|
48
|
+
assert.equal(dhtNodeCount({ dht: { nodes: { toArray: () => [1, 2, 3] } } }), 3);
|
|
49
|
+
assert.equal(dhtNodeCount({ dht: { nodes: { toArray: () => [] } } }), 0);
|
|
50
|
+
// A client built without a DHT, or one whose internals moved: the difference
|
|
51
|
+
// between "no DHT" and "an empty DHT" is the whole point of the report, so
|
|
52
|
+
// the first must not be printed as the second.
|
|
53
|
+
assert.equal(dhtNodeCount({}), null);
|
|
54
|
+
assert.equal(dhtNodeCount({ dht: {} }), null);
|
|
55
|
+
assert.equal(dhtNodeCount(null), null);
|
|
56
|
+
assert.equal(dhtNodeCount({ dht: { nodes: { toArray: () => { throw new Error("gone"); } } } }), null);
|
|
57
|
+
});
|
|
58
|
+
|
|
59
|
+
test("a name that will not resolve inside the cap is dropped, not waited on", async () => {
|
|
60
|
+
// The whole call is awaited before the torrent client exists, so a resolver
|
|
61
|
+
// that black-holes must cost the cap and not c-ares' own four tries.
|
|
62
|
+
const started = Date.now();
|
|
63
|
+
// `.invalid` is reserved by RFC 2606 and never resolves; the cap is what
|
|
64
|
+
// bounds the wait when a resolver answers slowly rather than quickly.
|
|
65
|
+
const resolved = await resolveDhtBootstrap(["nothing.invalid:6881"], 200);
|
|
66
|
+
assert.deepEqual(resolved, []);
|
|
67
|
+
assert.ok(Date.now() - started < 2000, "the cap, not the resolver, decided when to give up");
|
|
68
|
+
});
|
|
69
|
+
|
|
70
|
+
test("an empty list is an answer, not a failure", async () => {
|
|
71
|
+
assert.deepEqual(await resolveDhtBootstrap([], 200), []);
|
|
72
|
+
});
|
|
@@ -0,0 +1,87 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @file A run is POSITIONED where the player was told the segment begins.
|
|
3
|
+
*
|
|
4
|
+
* The companion of `cuts-follow-published-grid`, and the half that was missing.
|
|
5
|
+
* 2.45.0 moved the cut list onto the published table and left the position on
|
|
6
|
+
* the live one — but `-segment_times` are measured from wherever the run really
|
|
7
|
+
* began, so any distance between the two moves EVERY cut of that run by it. The
|
|
8
|
+
* live table keeps being corrected, and the corrections run backwards, so each
|
|
9
|
+
* restart began a little earlier than the grid its cuts were stated on and the
|
|
10
|
+
* distance accumulated across restarts.
|
|
11
|
+
*
|
|
12
|
+
* Field 2026-08-21, `JUFD665.mp4` (MP4, copy path, index read cleanly): after
|
|
13
|
+
* one seek restart a produced segment held the boundary two places before its
|
|
14
|
+
* own number — 16.684 s, which is 2.0000 segments — and after the next restart,
|
|
15
|
+
* four places, 33.5 s. The player's buffer then stopped extending at all,
|
|
16
|
+
* because every fragment's content landed before the time its playlist entry
|
|
17
|
+
* named: `bufferEnd` stood still at 4571.1 s through four `frag-far` warnings
|
|
18
|
+
* until hls.js gave up and jumped the viewer 16.8 s forward.
|
|
19
|
+
*/
|
|
20
|
+
|
|
21
|
+
import assert from "node:assert/strict";
|
|
22
|
+
import test from "node:test";
|
|
23
|
+
|
|
24
|
+
import { HlsSessionManager, describeGridDrift, segmentCutTimesFrom } from "../services/hls-session-manager.js";
|
|
25
|
+
|
|
26
|
+
/** What the playlist in the player's hands says. */
|
|
27
|
+
const PUBLISHED = [0, 8.342, 16.684, 25.026, 33.368, 41.71];
|
|
28
|
+
/** The same grid after produced segments moved two of its cuts backwards. */
|
|
29
|
+
const CORRECTED = [0, 8.342, 14.682, 25.026, 31.366, 41.71];
|
|
30
|
+
|
|
31
|
+
/**
|
|
32
|
+
* @returns {{ manager: HlsSessionManager, session: object }}
|
|
33
|
+
*/
|
|
34
|
+
function sessionWithDriftedGrid() {
|
|
35
|
+
const manager = new HlsSessionManager({
|
|
36
|
+
enabled: true,
|
|
37
|
+
ffmpegBin: "ffmpeg",
|
|
38
|
+
localBindHost: "127.0.0.1",
|
|
39
|
+
localPort: 9090
|
|
40
|
+
});
|
|
41
|
+
const session = {
|
|
42
|
+
id: "picture",
|
|
43
|
+
segmentBoundaries: [...CORRECTED],
|
|
44
|
+
publishedBoundaries: [...PUBLISHED]
|
|
45
|
+
};
|
|
46
|
+
return { manager, session };
|
|
47
|
+
}
|
|
48
|
+
|
|
49
|
+
test("a run starts at the time the player was told, not at the corrected one", () => {
|
|
50
|
+
const { manager, session } = sessionWithDriftedGrid();
|
|
51
|
+
assert.equal(manager.runStartTimeFor(session, 2), 16.684);
|
|
52
|
+
assert.notEqual(manager.runStartTimeFor(session, 2), session.segmentBoundaries[2]);
|
|
53
|
+
});
|
|
54
|
+
|
|
55
|
+
test("position and cut list come from the same table", () => {
|
|
56
|
+
const { manager, session } = sessionWithDriftedGrid();
|
|
57
|
+
const grid = manager.publishedGridFor(session);
|
|
58
|
+
const start = manager.runStartTimeFor(session, 2);
|
|
59
|
+
// The cut list is stated as offsets from where the run begins. Adding the
|
|
60
|
+
// position back must land on the published boundaries exactly — which is the
|
|
61
|
+
// property that was false while the two came from different tables.
|
|
62
|
+
const absolute = segmentCutTimesFrom(grid, 2).map((offset) => Number((start + offset).toFixed(3)));
|
|
63
|
+
assert.deepEqual(absolute, [25.026, 33.368]);
|
|
64
|
+
});
|
|
65
|
+
|
|
66
|
+
test("a session that published no grid positions on the live one", () => {
|
|
67
|
+
const { manager } = sessionWithDriftedGrid();
|
|
68
|
+
const session = { id: "no-playlist", segmentBoundaries: [...CORRECTED], publishedBoundaries: [] };
|
|
69
|
+
assert.equal(manager.runStartTimeFor(session, 2), CORRECTED[2]);
|
|
70
|
+
});
|
|
71
|
+
|
|
72
|
+
test("an index beyond the table is clamped rather than returning nothing", () => {
|
|
73
|
+
const { manager, session } = sessionWithDriftedGrid();
|
|
74
|
+
assert.equal(manager.runStartTimeFor(session, 9999), PUBLISHED[PUBLISHED.length - 1]);
|
|
75
|
+
assert.equal(manager.runStartTimeFor(session, -3), PUBLISHED[0]);
|
|
76
|
+
});
|
|
77
|
+
|
|
78
|
+
test("the drift between the two tables is stated in full", () => {
|
|
79
|
+
assert.equal(describeGridDrift(PUBLISHED, PUBLISHED), "identical");
|
|
80
|
+
assert.equal(
|
|
81
|
+
describeGridDrift(PUBLISHED, CORRECTED),
|
|
82
|
+
"2 of 6 boundaries apart, worst 2.002s at #2"
|
|
83
|
+
);
|
|
84
|
+
assert.equal(describeGridDrift(PUBLISHED, [0, 1]), "a different length (6 against 2)");
|
|
85
|
+
assert.equal(describeGridDrift(null, CORRECTED), "not comparable");
|
|
86
|
+
assert.equal(describeGridDrift([], []), "not comparable");
|
|
87
|
+
});
|