@torrent-tv/proxy 2.47.0 → 2.49.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +12 -0
- package/bin/cli.js +6 -0
- package/package.json +1 -1
- package/services/core-dumps.js +106 -0
- package/services/hls-session-manager.js +95 -1
- package/services/torrent-worker/client.js +63 -2
- package/test/core-dumps.test.js +57 -0
- package/test/seek-landing.test.js +50 -0
- package/test/tunnel-renewal.test.js +13 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,15 @@
|
|
|
1
|
+
## 2.49.0
|
|
2
|
+
|
|
3
|
+
- **Fix**: The torrent worker is allowed to END rather than being torn down under itself. A core dump read on 2026-08-21 named the fault the proxy has been dying of: `SIGSEGV` in `v8::Value::IsArrayBufferView` reached through `napi_get_buffer_info` from utp-native's `on_utp_accept`, called from its UDP read — all of it inside `node::Environment::CleanupHandles`, under `FreeEnvironment`, on `Worker::Run`. That is a teardown race, not a data fault, which is why neither patch our forked library already carries touched it: a datagram arriving while the environment is being freed walks into an isolate that no longer exists. `destroyAll` called `Worker.terminate()` immediately after destroying the client inside, and `terminate()` frees the environment with libuv's handle callbacks still queued. It now waits for the thread to exit by itself — once the client is destroyed nothing holds its loop open — with `terminate()` kept as a five-second fallback, because a shutdown that hangs is worse than one that is forced.
|
|
4
|
+
- **New**: A worker thread that ends is noticed. Only `message` and `error` were listened for, so when the thread went away the proxy simply stopped and the log ended mid-sentence — five times in three days with not one line to say so, and no way to tell our own shutdown from the thread dying. An `exit` handler now says which of the two it was, and fails everything waiting rather than leaving it hanging.
|
|
5
|
+
- **New**: Core dumps are capped at the newest two at startup. Each is the worker's whole address space — 4.18 GB on the field host — and four of them had nearly filled a 235 GB disk. The newest stay because they are the evidence for the fault still open. `dumpsToRemove` is pure and tested.
|
|
6
|
+
|
|
7
|
+
## 2.48.0
|
|
8
|
+
|
|
9
|
+
- **Fix**: A copied picture now begins where it was asked to, so its cuts land on the times its playlist names. ffmpeg's own CLI moves an input seek back by `3*AV_TIME_BASE / 23` — **130.435 ms** — whenever the container does not declare `AVFMT_SEEK_TO_PTS` (Matroska does not) and a stream carries B-frames, which is sound in itself: such containers seek in decode order while the caller asks in presentation order. The consequence for a copy is that asking for a keyframe lands on the one BEFORE it, deterministically; and since `-segment_times` is measured from where the run really began while this code computed those offsets from the time it asked for, every cut of the run inherited one whole keyframe interval. Field 2026-08-20: 119 of 125 segments arriving a uniform 2.002 s early against the 0.5 s hls.js bridges, so every fragment was refused and re-fetched — on 2026-08-17 two of them 1908 times each. The request is now made that much later, bounded by half the distance to the next keyframe. Measured 2026-08-21 on Matroska with keyframes every 2 s: `-ss 10` produced a first segment starting at 8.000, `-ss 10.130435` one starting at 10.000; on MP4, where the heuristic does not fire, 10, 10.130435 and 10.2 all produced 10.000 — right in one case and harmless in the other. Not applied when the picture is re-encoded: a re-encode discards frames up to the requested time and already begins exactly there (`-ss 11` copied starts at 10.000, re-encoded at 11.000).
|
|
10
|
+
- **New**: A run that did not begin where it was asked to says so. The first piece a run produces is the only statement of its real origin that exists, and nothing compared the two — which is why the fault above stayed silent through two releases that touched the same grid. Said once per run, and only past what a player bridges.
|
|
11
|
+
- **Chore**: The tunnel-renewal test shuts its stand-in registry down deterministically. `WebSocketServer.close` waits for every connection to end and a renewal can leave one still closing, so a full suite run could hang for nine minutes on it.
|
|
12
|
+
|
|
1
13
|
## 2.47.0
|
|
2
14
|
|
|
3
15
|
- **Fix**: The tunnel is replaced before anything upstream ends it, so a viewer no longer arrives to find no proxy. Something between the proxy and the server closes the socket after exactly **100 min 15 s** — measured across a day of logs 2026-08-20, three intervals of 100:15 wherever a restart did not reset the clock, `code=1006` each time, and with the 30 s keepalive running throughout, so it is a lifetime cap and not an idle timeout. Reconnecting afterwards takes five seconds during which this proxy does not exist as far as the registry is concerned. The connection is now replaced at ninety minutes and the replacement takes over FIRST: the new socket registers itself, the server atomically supersedes the old one, and only then does the old one close — so there is no instant with nothing registered. A socket that finds itself superseded says so rather than reporting the tunnel as down, and an abrupt close nobody asked for still reconnects as before. Pinned by a test against a real WebSocket server.
|
package/bin/cli.js
CHANGED
|
@@ -23,6 +23,7 @@ import { registerClient } from "../services/registry-api.js";
|
|
|
23
23
|
import { createTunnelClient } from "../services/tunnel-client.js";
|
|
24
24
|
import { createWebRtcManager } from "../services/webrtc-manager.js";
|
|
25
25
|
import { createDataChannelHandler } from "../services/data-channel-handler.js";
|
|
26
|
+
import { pruneCoreDumps } from "../services/core-dumps.js";
|
|
26
27
|
import { collectHealthMetrics } from "../services/health-collector.js";
|
|
27
28
|
import { createPortMapper } from "../services/port-mapper.js";
|
|
28
29
|
import { classifyNat } from "../services/nat-classifier.js";
|
|
@@ -294,6 +295,11 @@ try {
|
|
|
294
295
|
actualPort = started.port;
|
|
295
296
|
const directBaseUrl = explicitBaseUrl || `http://${bindHost}:${actualPort}`;
|
|
296
297
|
|
|
298
|
+
// A native fault writes the whole address space out — 4.18 GB each on the
|
|
299
|
+
// field host, and four of them nearly filled a 235 GB disk. Keep the newest
|
|
300
|
+
// two, which are the evidence for the fault still open, and drop the rest.
|
|
301
|
+
void pruneCoreDumps(options.stateDir);
|
|
302
|
+
|
|
297
303
|
logger.info(`Starting @torrent-tv/proxy v${PROXY_VERSION}`);
|
|
298
304
|
logger.info(`Local stream endpoint: http://${bindHost}:${actualPort}/stream`);
|
|
299
305
|
logger.info(`Advertised direct URL: ${directBaseUrl}`);
|
package/package.json
CHANGED
|
@@ -0,0 +1,106 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @file Keep the last few core dumps and no more.
|
|
3
|
+
*
|
|
4
|
+
* A native fault on the torrent worker writes the whole address space out: on
|
|
5
|
+
* the field host that is **4.18 GB each**, and four of them had nearly filled a
|
|
6
|
+
* 235 GB disk by 2026-08-21. The dumps are worth having — the one read that day
|
|
7
|
+
* named a fault three days of reasoning had not — but only the recent ones are,
|
|
8
|
+
* and a full disk costs more than an old dump is worth.
|
|
9
|
+
*
|
|
10
|
+
* Deliberately not "delete them all": the newest are the evidence for the fault
|
|
11
|
+
* that is still open (roadmap item 7).
|
|
12
|
+
*/
|
|
13
|
+
|
|
14
|
+
import { readdir, rm, stat } from "node:fs/promises";
|
|
15
|
+
import path from "node:path";
|
|
16
|
+
|
|
17
|
+
import { logger } from "../utils/logger.js";
|
|
18
|
+
|
|
19
|
+
/** How many to keep, newest first. */
|
|
20
|
+
export const CORE_DUMPS_KEPT = 2;
|
|
21
|
+
|
|
22
|
+
/**
|
|
23
|
+
* Which dumps to remove, given what is there.
|
|
24
|
+
*
|
|
25
|
+
* Pure, so the rule can be tested without a filesystem: newest first by the
|
|
26
|
+
* time they were written, keep `keep`, name the rest.
|
|
27
|
+
*
|
|
28
|
+
* @param {Array<{ name: string, writtenAt: number }>} dumps
|
|
29
|
+
* @param {number} [keep]
|
|
30
|
+
* @returns {string[]} Names to delete, oldest first.
|
|
31
|
+
*/
|
|
32
|
+
export function dumpsToRemove(dumps, keep = CORE_DUMPS_KEPT) {
|
|
33
|
+
const sorted = [...(Array.isArray(dumps) ? dumps : [])]
|
|
34
|
+
.filter((dump) => typeof dump?.name === "string" && Number.isFinite(dump?.writtenAt))
|
|
35
|
+
.sort((left, right) => right.writtenAt - left.writtenAt);
|
|
36
|
+
return sorted.slice(Math.max(0, keep)).map((dump) => dump.name).reverse();
|
|
37
|
+
}
|
|
38
|
+
|
|
39
|
+
/**
|
|
40
|
+
* Whether a file name is a core dump this host wrote.
|
|
41
|
+
*
|
|
42
|
+
* The kernel's pattern on the addon host produces `core.<thread>.<pid>.<epoch>`
|
|
43
|
+
* — every one seen so far is `core.WorkerThread.81.…`, the thread the torrent
|
|
44
|
+
* client runs on. Matched loosely on the `core.` prefix so a differently
|
|
45
|
+
* configured host is still swept.
|
|
46
|
+
*
|
|
47
|
+
* @param {string} name
|
|
48
|
+
* @returns {boolean}
|
|
49
|
+
*/
|
|
50
|
+
export function isCoreDump(name) {
|
|
51
|
+
return typeof name === "string" && /^core\.[^/\\]+$/.test(name);
|
|
52
|
+
}
|
|
53
|
+
|
|
54
|
+
/**
|
|
55
|
+
* Delete all but the newest few core dumps in `dir`.
|
|
56
|
+
*
|
|
57
|
+
* Best-effort and never fatal: a proxy that cannot tidy its dumps still has to
|
|
58
|
+
* serve video.
|
|
59
|
+
*
|
|
60
|
+
* @param {string} dir
|
|
61
|
+
* @param {number} [keep]
|
|
62
|
+
* @returns {Promise<void>}
|
|
63
|
+
*/
|
|
64
|
+
export async function pruneCoreDumps(dir, keep = CORE_DUMPS_KEPT) {
|
|
65
|
+
if (typeof dir !== "string" || dir.length === 0) {
|
|
66
|
+
return;
|
|
67
|
+
}
|
|
68
|
+
/** @type {Array<{ name: string, writtenAt: number, bytes: number }>} */
|
|
69
|
+
const dumps = [];
|
|
70
|
+
try {
|
|
71
|
+
for (const name of await readdir(dir)) {
|
|
72
|
+
if (!isCoreDump(name)) {
|
|
73
|
+
continue;
|
|
74
|
+
}
|
|
75
|
+
try {
|
|
76
|
+
const info = await stat(path.join(dir, name));
|
|
77
|
+
if (info.isFile()) {
|
|
78
|
+
dumps.push({ name, writtenAt: info.mtimeMs, bytes: info.size });
|
|
79
|
+
}
|
|
80
|
+
} catch {
|
|
81
|
+
// silent-ok: it went away between listing and reading, which is the
|
|
82
|
+
// outcome this function wanted anyway.
|
|
83
|
+
}
|
|
84
|
+
}
|
|
85
|
+
} catch {
|
|
86
|
+
return; // No such directory, or unreadable. Nothing to tidy.
|
|
87
|
+
}
|
|
88
|
+
if (dumps.length === 0) {
|
|
89
|
+
return;
|
|
90
|
+
}
|
|
91
|
+
const doomed = dumpsToRemove(dumps, keep);
|
|
92
|
+
const freed = dumps
|
|
93
|
+
.filter((dump) => doomed.includes(dump.name))
|
|
94
|
+
.reduce((total, dump) => total + dump.bytes, 0);
|
|
95
|
+
for (const name of doomed) {
|
|
96
|
+
try {
|
|
97
|
+
await rm(path.join(dir, name), { force: true });
|
|
98
|
+
} catch {
|
|
99
|
+
// silent-ok: best effort, and the next start tries again.
|
|
100
|
+
}
|
|
101
|
+
}
|
|
102
|
+
logger.info(
|
|
103
|
+
`core dumps: ${dumps.length} present, keeping the newest ${Math.min(keep, dumps.length)}` +
|
|
104
|
+
(doomed.length > 0 ? `, removed ${doomed.length} (${(freed / 1e9).toFixed(2)} GB)` : "")
|
|
105
|
+
);
|
|
106
|
+
}
|
|
@@ -568,6 +568,36 @@ const SEGMENT_READ_HIGH_WATER_MARK = 4 * 1024 * 1024;
|
|
|
568
568
|
// second is below any drift a viewer could notice, so anything above it is the
|
|
569
569
|
// index being wrong about where a keyframe is rather than rounding.
|
|
570
570
|
const SEGMENT_START_DISAGREEMENT_SEC = 0.25;
|
|
571
|
+
/**
|
|
572
|
+
* What ffmpeg's own CLI subtracts from an input seek, and therefore what has to
|
|
573
|
+
* be added back to land where we asked.
|
|
574
|
+
*
|
|
575
|
+
* `fftools/ffmpeg_demux.c`, in `ifile_open`: when the container does not
|
|
576
|
+
* declare `AVFMT_SEEK_TO_PTS` — Matroska does not — and any stream carries
|
|
577
|
+
* B-frames, the seek target is moved back by `3*AV_TIME_BASE / 23` before
|
|
578
|
+
* `avformat_seek_file` is called. Its purpose is sound: such containers seek in
|
|
579
|
+
* decode order while the caller asks in presentation order, and with B-frames
|
|
580
|
+
* the two differ, so it backs off far enough to be sure of reaching the frame
|
|
581
|
+
* asked for.
|
|
582
|
+
*
|
|
583
|
+
* The consequence for a COPY is that asking for a keyframe lands on the one
|
|
584
|
+
* BEFORE it — deterministically, every time. Measured 2026-08-21 on a Matroska
|
|
585
|
+
* file with keyframes every 2 s: `-ss 10` produced a first segment starting at
|
|
586
|
+
* 8.000; `-ss 10.130435` produced one starting at 10.000. On MP4, where the
|
|
587
|
+
* heuristic does not fire, all of 10, 10.130435 and 10.2 produced 10.000 — so
|
|
588
|
+
* adding this is right in one case and harmless in the other.
|
|
589
|
+
*
|
|
590
|
+
* That landing is what `-segment_times` is measured from, while this code
|
|
591
|
+
* computes those offsets from the time it ASKED for. One keyframe interval
|
|
592
|
+
* apart, inherited by every cut of the run: 119 of 125 segments arriving a
|
|
593
|
+
* uniform 2.002 s early in the field, four times what a player bridges.
|
|
594
|
+
*
|
|
595
|
+
* Not applied when the picture is re-encoded: a re-encode decodes from the
|
|
596
|
+
* keyframe and discards frames up to the requested time, so its output already
|
|
597
|
+
* begins exactly where asked (measured the same day: `-ss 11` copied starts at
|
|
598
|
+
* 10.000, re-encoded at 11.000).
|
|
599
|
+
*/
|
|
600
|
+
const SEEK_LANDING_OFFSET_SEC = 3 / 23;
|
|
571
601
|
|
|
572
602
|
/**
|
|
573
603
|
* How far a fragment may land from where the playlist put it before the player
|
|
@@ -1146,6 +1176,34 @@ export function segmentCutTimesFrom(boundaries, startIndex) {
|
|
|
1146
1176
|
return times;
|
|
1147
1177
|
}
|
|
1148
1178
|
|
|
1179
|
+
/**
|
|
1180
|
+
* How much later than a keyframe to ASK, so that ffmpeg lands on that keyframe.
|
|
1181
|
+
*
|
|
1182
|
+
* Bounded by half the distance to the next keyframe, which matters only where
|
|
1183
|
+
* keyframes stand closer together than twice the offset. There no single value
|
|
1184
|
+
* can satisfy both worlds — asking too little lands a keyframe early when the
|
|
1185
|
+
* heuristic fires, asking too much lands a keyframe late when it does not — and
|
|
1186
|
+
* the bound picks the smaller error, which is then under one keyframe interval
|
|
1187
|
+
* and therefore under what a player bridges.
|
|
1188
|
+
*
|
|
1189
|
+
* @param {HlsSession} session
|
|
1190
|
+
* @param {number} keyframe - A real keyframe time the run is to begin at.
|
|
1191
|
+
* @returns {number} Seconds to add to the request.
|
|
1192
|
+
*/
|
|
1193
|
+
export function seekLandingOffsetFor(session, keyframe) {
|
|
1194
|
+
// A re-encode trims to the requested time itself, so it needs no help and
|
|
1195
|
+
// must not be pushed past what it was asked for.
|
|
1196
|
+
if (session?.transcodeVideo === true) {
|
|
1197
|
+
return 0;
|
|
1198
|
+
}
|
|
1199
|
+
const times = Array.isArray(session?.keyframeTimes) ? session.keyframeTimes : [];
|
|
1200
|
+
const next = times.find((time) => time > keyframe + 0.001);
|
|
1201
|
+
if (next === undefined) {
|
|
1202
|
+
return SEEK_LANDING_OFFSET_SEC;
|
|
1203
|
+
}
|
|
1204
|
+
return Math.min(SEEK_LANDING_OFFSET_SEC, (next - keyframe) / 2);
|
|
1205
|
+
}
|
|
1206
|
+
|
|
1149
1207
|
/**
|
|
1150
1208
|
* The largest keyframe time that does not exceed `target`, from a SORTED
|
|
1151
1209
|
* (ascending) array of keyframe times such as {@link probeVideoKeyframeTimes}
|
|
@@ -3932,7 +3990,7 @@ export class HlsSessionManager {
|
|
|
3932
3990
|
if (snappedKeyframe !== null) {
|
|
3933
3991
|
const residualSeconds = Math.max(0, seekSeconds - snappedKeyframe);
|
|
3934
3992
|
if (snappedKeyframe > 0) {
|
|
3935
|
-
args.push("-ss", ffmpegSeconds(snappedKeyframe));
|
|
3993
|
+
args.push("-ss", ffmpegSeconds(snappedKeyframe + seekLandingOffsetFor(session, snappedKeyframe)));
|
|
3936
3994
|
}
|
|
3937
3995
|
args.push("-i", session.inputUrl);
|
|
3938
3996
|
if (residualSeconds > 0) {
|
|
@@ -5124,6 +5182,41 @@ export class HlsSessionManager {
|
|
|
5124
5182
|
* @param {number} declaredStart - Seconds, from the playlist.
|
|
5125
5183
|
* @returns {void}
|
|
5126
5184
|
*/
|
|
5185
|
+
/**
|
|
5186
|
+
* Where the run REALLY began, against where it was asked to begin.
|
|
5187
|
+
*
|
|
5188
|
+
* The first piece a run produces is the only statement of this that exists,
|
|
5189
|
+
* and until now nothing compared the two. They disagree whenever the seek
|
|
5190
|
+
* lands somewhere other than the time asked for — which, before the landing
|
|
5191
|
+
* offset, was every run on a Matroska source with B-frames, by exactly one
|
|
5192
|
+
* keyframe interval. Every cut of the run then inherits it, because
|
|
5193
|
+
* `-segment_times` is measured from the landing.
|
|
5194
|
+
*
|
|
5195
|
+
* Said once per run, and only when it matters: within what a player bridges
|
|
5196
|
+
* there is nothing to report.
|
|
5197
|
+
*
|
|
5198
|
+
* @param {HlsSession} session
|
|
5199
|
+
* @param {number} index
|
|
5200
|
+
* @param {number} trueStart
|
|
5201
|
+
* @returns {void}
|
|
5202
|
+
*/
|
|
5203
|
+
#noteRunLanding(session, index, trueStart) {
|
|
5204
|
+
if (session.encodeStartIndex !== index || session.landingReportedForRun === index) {
|
|
5205
|
+
return;
|
|
5206
|
+
}
|
|
5207
|
+
session.landingReportedForRun = index;
|
|
5208
|
+
const asked = this.#segmentStartTime(session, index);
|
|
5209
|
+
const drift = trueStart - asked;
|
|
5210
|
+
if (!Number.isFinite(drift) || Math.abs(drift) <= PLAYER_BUFFER_HOLE_SEC) {
|
|
5211
|
+
return;
|
|
5212
|
+
}
|
|
5213
|
+
logger.warn(
|
|
5214
|
+
`transcode ${session.id} run began at ${trueStart.toFixed(3)}s but was asked for ` +
|
|
5215
|
+
`${asked.toFixed(3)}s — ${drift > 0 ? "+" : ""}${drift.toFixed(3)}s, and every cut of this ` +
|
|
5216
|
+
"run is measured from where it began, so the whole run is that far from its playlist"
|
|
5217
|
+
);
|
|
5218
|
+
}
|
|
5219
|
+
|
|
5127
5220
|
#noteIndexAccuracy(session, index, trueStart, declaredStart) {
|
|
5128
5221
|
const deviation = Math.abs(trueStart - declaredStart);
|
|
5129
5222
|
session.indexCheck ??= newIndexCheck();
|
|
@@ -7719,6 +7812,7 @@ export class HlsSessionManager {
|
|
|
7719
7812
|
: null;
|
|
7720
7813
|
const declaredStart = this.#segmentStartTime(session, index);
|
|
7721
7814
|
if (trueStart !== null) {
|
|
7815
|
+
this.#noteRunLanding(session, index, trueStart);
|
|
7722
7816
|
this.#noteIndexAccuracy(session, index, trueStart, declaredStart);
|
|
7723
7817
|
}
|
|
7724
7818
|
// WHERE THE PLAYER WAS TOLD THIS SEGMENT BEGINS, which is the playlist
|
|
@@ -22,6 +22,15 @@ import { createCaller, createReceiveStream } from "./channel.js";
|
|
|
22
22
|
import { Command, Event } from "./protocol.js";
|
|
23
23
|
|
|
24
24
|
const WORKER_URL = new URL("./worker.js", import.meta.url);
|
|
25
|
+
/**
|
|
26
|
+
* How long a shutdown waits for the thread to end by itself before forcing it.
|
|
27
|
+
*
|
|
28
|
+
* Ending by itself is what lets libuv drain the handle callbacks it is holding;
|
|
29
|
+
* forcing it is what ran one of those against a freed isolate. Five seconds is
|
|
30
|
+
* long enough for a destroyed torrent client to release its sockets and short
|
|
31
|
+
* enough that a shutdown never appears to hang.
|
|
32
|
+
*/
|
|
33
|
+
const WORKER_EXIT_GRACE_MS = 5_000;
|
|
25
34
|
|
|
26
35
|
/**
|
|
27
36
|
* Runs the torrent client on its own thread and exposes it to the main thread.
|
|
@@ -34,6 +43,12 @@ const WORKER_URL = new URL("./worker.js", import.meta.url);
|
|
|
34
43
|
*/
|
|
35
44
|
export class TorrentWorkerClient {
|
|
36
45
|
#worker;
|
|
46
|
+
/**
|
|
47
|
+
* Whether this shutdown was asked for, so the thread ending can be told from
|
|
48
|
+
* the thread dying. Without it both look identical from outside, which is how
|
|
49
|
+
* five crashes produced no line in the log.
|
|
50
|
+
*/
|
|
51
|
+
#stopping = false;
|
|
37
52
|
#caller;
|
|
38
53
|
/** Receive-side handles for in-flight reads, keyed by request id. */
|
|
39
54
|
#reads = new Map();
|
|
@@ -186,6 +201,30 @@ export class TorrentWorkerClient {
|
|
|
186
201
|
}
|
|
187
202
|
this.#reads.clear();
|
|
188
203
|
});
|
|
204
|
+
|
|
205
|
+
// A worker that ENDS was, until now, not noticed at all: only `message` and
|
|
206
|
+
// `error` were listened for. So when the thread went away the proxy simply
|
|
207
|
+
// stopped, the log ended mid-sentence, and nothing said whether we had
|
|
208
|
+
// asked for it — which is precisely the reading that was missing on
|
|
209
|
+
// 2026-08-21, when a core dump showed the thread faulting inside
|
|
210
|
+
// `Environment::CleanupHandles` and there was no way to tell our own
|
|
211
|
+
// shutdown from the thread ending on its own.
|
|
212
|
+
this.#worker.on("exit", (code) => {
|
|
213
|
+
if (this.#stopping) {
|
|
214
|
+
logger.info(`torrent-worker: thread ended as asked (code ${code})`);
|
|
215
|
+
return;
|
|
216
|
+
}
|
|
217
|
+
logger.error(
|
|
218
|
+
`torrent-worker: thread ended on its own with code ${code} — nobody asked it to. ` +
|
|
219
|
+
"Everything waiting on it is failed; the proxy has no torrent client until it is rebuilt."
|
|
220
|
+
);
|
|
221
|
+
const reason = new Error("Torrent worker ended unexpectedly.");
|
|
222
|
+
this.#caller.rejectAll(reason);
|
|
223
|
+
for (const [, read] of this.#reads) {
|
|
224
|
+
read.fail(reason);
|
|
225
|
+
}
|
|
226
|
+
this.#reads.clear();
|
|
227
|
+
});
|
|
189
228
|
}
|
|
190
229
|
|
|
191
230
|
/**
|
|
@@ -520,11 +559,33 @@ export class TorrentWorkerClient {
|
|
|
520
559
|
* @returns {Promise<void>}
|
|
521
560
|
*/
|
|
522
561
|
async destroyAll() {
|
|
562
|
+
this.#stopping = true;
|
|
523
563
|
try {
|
|
524
564
|
await this.#caller.call(Command.DESTROY_ALL, {});
|
|
525
565
|
} catch {
|
|
526
|
-
// Already gone —
|
|
566
|
+
// Already gone — the wait below settles either way.
|
|
527
567
|
}
|
|
528
|
-
|
|
568
|
+
// Let the thread END rather than tearing it down under itself.
|
|
569
|
+
//
|
|
570
|
+
// `terminate()` frees the environment immediately, with the handle
|
|
571
|
+
// callbacks libuv still holds queued. One of those is utp-native's UDP
|
|
572
|
+
// read, and running it against a freed isolate is what the core dump of
|
|
573
|
+
// 2026-08-21 caught: `on_utp_accept` → `napi_get_buffer_info` →
|
|
574
|
+
// `v8::Value::IsArrayBufferView` inside `Environment::CleanupHandles`.
|
|
575
|
+
// Once the client inside is destroyed the thread has nothing left holding
|
|
576
|
+
// its loop open, so it exits by itself and the callbacks drain first.
|
|
577
|
+
//
|
|
578
|
+
// `terminate()` stays as the bounded fallback, because a shutdown that
|
|
579
|
+
// hangs is worse than one that is forced.
|
|
580
|
+
await new Promise((resolve) => {
|
|
581
|
+
const timer = setTimeout(() => {
|
|
582
|
+
logger.warn("torrent-worker: thread did not end in 5s; terminating it");
|
|
583
|
+
void this.#worker.terminate().finally(resolve);
|
|
584
|
+
}, WORKER_EXIT_GRACE_MS);
|
|
585
|
+
this.#worker.once("exit", () => {
|
|
586
|
+
clearTimeout(timer);
|
|
587
|
+
resolve();
|
|
588
|
+
});
|
|
589
|
+
});
|
|
529
590
|
}
|
|
530
591
|
}
|
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @file Keep the last few core dumps and no more.
|
|
3
|
+
*
|
|
4
|
+
* Each is the worker thread's whole address space — 4.18 GB on the field host —
|
|
5
|
+
* and four of them had nearly filled a 235 GB disk by 2026-08-21. The newest
|
|
6
|
+
* are evidence for a fault that is still open, so they stay; the rest go.
|
|
7
|
+
*/
|
|
8
|
+
|
|
9
|
+
import assert from "node:assert/strict";
|
|
10
|
+
import test from "node:test";
|
|
11
|
+
|
|
12
|
+
import { dumpsToRemove, isCoreDump } from "../services/core-dumps.js";
|
|
13
|
+
|
|
14
|
+
const dumps = [
|
|
15
|
+
{ name: "core.WorkerThread.81.1787176646", writtenAt: 1787176646000 },
|
|
16
|
+
{ name: "core.WorkerThread.81.1787224562", writtenAt: 1787224562000 },
|
|
17
|
+
{ name: "core.WorkerThread.81.1787237468", writtenAt: 1787237468000 },
|
|
18
|
+
{ name: "core.WorkerThread.81.1787243278", writtenAt: 1787243278000 },
|
|
19
|
+
{ name: "core.WorkerThread.81.1787292750", writtenAt: 1787292750000 }
|
|
20
|
+
];
|
|
21
|
+
|
|
22
|
+
test("the newest two stay and the rest go, oldest first", () => {
|
|
23
|
+
assert.deepEqual(dumpsToRemove(dumps), [
|
|
24
|
+
"core.WorkerThread.81.1787176646",
|
|
25
|
+
"core.WorkerThread.81.1787224562",
|
|
26
|
+
"core.WorkerThread.81.1787237468"
|
|
27
|
+
]);
|
|
28
|
+
});
|
|
29
|
+
|
|
30
|
+
test("fewer than the limit leaves everything alone", () => {
|
|
31
|
+
assert.deepEqual(dumpsToRemove(dumps.slice(0, 2)), []);
|
|
32
|
+
assert.deepEqual(dumpsToRemove([]), []);
|
|
33
|
+
});
|
|
34
|
+
|
|
35
|
+
test("order on disk does not decide; the time written does", () => {
|
|
36
|
+
const shuffled = [dumps[3], dumps[0], dumps[4], dumps[2], dumps[1]];
|
|
37
|
+
assert.deepEqual(dumpsToRemove(shuffled, 1), [
|
|
38
|
+
"core.WorkerThread.81.1787176646",
|
|
39
|
+
"core.WorkerThread.81.1787224562",
|
|
40
|
+
"core.WorkerThread.81.1787237468",
|
|
41
|
+
"core.WorkerThread.81.1787243278"
|
|
42
|
+
]);
|
|
43
|
+
});
|
|
44
|
+
|
|
45
|
+
test("only core dumps are considered", () => {
|
|
46
|
+
assert.equal(isCoreDump("core.WorkerThread.81.1787292750"), true);
|
|
47
|
+
assert.equal(isCoreDump("proxy.log"), false);
|
|
48
|
+
assert.equal(isCoreDump("host-timings.json"), false);
|
|
49
|
+
// Not a path, and not a directory called core.
|
|
50
|
+
assert.equal(isCoreDump("core.foo/bar"), false);
|
|
51
|
+
assert.equal(isCoreDump(""), false);
|
|
52
|
+
});
|
|
53
|
+
|
|
54
|
+
test("nothing readable is not a reason to delete anything", () => {
|
|
55
|
+
assert.deepEqual(dumpsToRemove(null), []);
|
|
56
|
+
assert.deepEqual(dumpsToRemove([{ name: 5, writtenAt: "x" }]), []);
|
|
57
|
+
});
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @file Ask ffmpeg late enough that it lands where we meant.
|
|
3
|
+
*
|
|
4
|
+
* `fftools/ffmpeg_demux.c` moves an input seek back by `3*AV_TIME_BASE / 23` —
|
|
5
|
+
* 130.435 ms — whenever the container does not declare `AVFMT_SEEK_TO_PTS` and
|
|
6
|
+
* a stream carries B-frames. So asking for a keyframe lands on the one before
|
|
7
|
+
* it, and since `-segment_times` is measured from where the run really began,
|
|
8
|
+
* every cut of that run inherits the shift.
|
|
9
|
+
*
|
|
10
|
+
* Measured 2026-08-21 on Matroska with keyframes every 2 s: `-ss 10` produced a
|
|
11
|
+
* first segment starting at 8.000, `-ss 10.130435` one starting at 10.000. On
|
|
12
|
+
* MP4, where the heuristic does not fire, 10, 10.130435 and 10.2 all produced
|
|
13
|
+
* 10.000 — right in one case, harmless in the other.
|
|
14
|
+
*/
|
|
15
|
+
|
|
16
|
+
import assert from "node:assert/strict";
|
|
17
|
+
import test from "node:test";
|
|
18
|
+
|
|
19
|
+
import { seekLandingOffsetFor } from "../services/hls-session-manager.js";
|
|
20
|
+
|
|
21
|
+
const OFFSET = 3 / 23;
|
|
22
|
+
|
|
23
|
+
test("a copied picture is asked for one heuristic later than the keyframe", () => {
|
|
24
|
+
const session = { transcodeVideo: false, keyframeTimes: [0, 2.002, 4.004, 6.006] };
|
|
25
|
+
assert.equal(seekLandingOffsetFor(session, 2.002), OFFSET);
|
|
26
|
+
});
|
|
27
|
+
|
|
28
|
+
test("a re-encode is asked for exactly what it should produce", () => {
|
|
29
|
+
// It decodes from the keyframe and discards frames up to the requested time,
|
|
30
|
+
// so pushing the request later would start its output late.
|
|
31
|
+
const session = { transcodeVideo: true, keyframeTimes: [0, 2.002, 4.004] };
|
|
32
|
+
assert.equal(seekLandingOffsetFor(session, 2.002), 0);
|
|
33
|
+
});
|
|
34
|
+
|
|
35
|
+
test("the offset never reaches the next keyframe", () => {
|
|
36
|
+
// Keyframes 0.1 s apart: half of that is the most that can be added without
|
|
37
|
+
// risking a landing on the NEXT one where the heuristic does not fire.
|
|
38
|
+
const session = { transcodeVideo: false, keyframeTimes: [0, 0.1, 0.2, 0.3] };
|
|
39
|
+
assert.equal(seekLandingOffsetFor(session, 0.1), 0.05);
|
|
40
|
+
});
|
|
41
|
+
|
|
42
|
+
test("the last keyframe has nothing after it to collide with", () => {
|
|
43
|
+
const session = { transcodeVideo: false, keyframeTimes: [0, 2.002, 4.004] };
|
|
44
|
+
assert.equal(seekLandingOffsetFor(session, 4.004), OFFSET);
|
|
45
|
+
});
|
|
46
|
+
|
|
47
|
+
test("no keyframe list is still answered", () => {
|
|
48
|
+
assert.equal(seekLandingOffsetFor({ transcodeVideo: false }, 5), OFFSET);
|
|
49
|
+
assert.equal(seekLandingOffsetFor(null, 5), OFFSET);
|
|
50
|
+
});
|
|
@@ -33,8 +33,12 @@ async function startRegistry() {
|
|
|
33
33
|
let current = null;
|
|
34
34
|
let opened = 0;
|
|
35
35
|
let everEmpty = false;
|
|
36
|
+
/** Every socket ever accepted, so the server can be shut without waiting. */
|
|
37
|
+
const accepted = new Set();
|
|
36
38
|
server.on("connection", (socket) => {
|
|
37
39
|
opened += 1;
|
|
40
|
+
accepted.add(socket);
|
|
41
|
+
socket.on("close", () => { accepted.delete(socket); });
|
|
38
42
|
const previous = current;
|
|
39
43
|
current = socket;
|
|
40
44
|
// The replacement is registered BEFORE the old one is closed, so a reader
|
|
@@ -52,7 +56,15 @@ async function startRegistry() {
|
|
|
52
56
|
const { port } = server.address();
|
|
53
57
|
return {
|
|
54
58
|
url: `http://127.0.0.1:${port}`,
|
|
55
|
-
close
|
|
59
|
+
// `close` waits for every connection to end, and a renewal can leave one
|
|
60
|
+
// still closing, so they are ended here rather than waited on.
|
|
61
|
+
close: () => new Promise((resolve) => {
|
|
62
|
+
for (const socket of accepted) {
|
|
63
|
+
socket.terminate();
|
|
64
|
+
}
|
|
65
|
+
accepted.clear();
|
|
66
|
+
server.close(() => resolve());
|
|
67
|
+
}),
|
|
56
68
|
registered: () => (current && current.readyState === 1 ? 1 : 0),
|
|
57
69
|
killCurrent: () => { current?.terminate(); },
|
|
58
70
|
opened: () => opened,
|