@torrent-tv/proxy 2.78.0 → 2.80.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +23 -0
- package/bin/cli.js +20 -1
- package/biome.json +1 -0
- package/package.json +1 -1
- package/routes/api/transcode-sessions/net-report/post.js +5 -0
- package/services/encode/EncodePlan.js +38 -12
- package/services/encode/EncodeRun.js +70 -0
- package/services/encode/SegmentStore.js +59 -0
- package/services/encode/open-piece.js +88 -0
- package/services/encode/run-budget.js +89 -0
- package/services/encode/run-command.js +15 -0
- package/services/encode/run-costs.js +94 -0
- package/services/hls-session-manager.js +99 -344
- package/services/orchestrators/EncodeOrchestrator.js +134 -17
- package/services/{encode/DemandMap.js → priority/PriorityMap.js} +99 -34
- package/services/usrsctp-state.js +35 -6
- package/services/viewer/Viewer.js +83 -0
- package/test/encode-orchestrator.test.js +36 -8
- package/test/encode-plan.test.js +67 -5
- package/test/one-authority.test.js +105 -0
- package/test/{demand-map.test.js → priority-map.test.js} +49 -12
- package/test/usrsctp-state.test.js +18 -2
package/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,26 @@
|
|
|
1
|
+
## 2.80.0
|
|
2
|
+
|
|
3
|
+
- **Fix**: The reading of usrsctp's own state is OFF unless `--usrsctp-state` is given, and its deadline is counted by a separate process. On 2026-09-05 a delivery probe declared a wedge that lasted half a second and cleared itself; the reading it triggered attached gdb to this process, which stops every one of its eighty threads for as long as it is attached. The log ended mid-second, `/healthz` stopped answering, and the viewer waited a minute and was told the proxy had sent no video. The fifteen-second guard could not fire — it was a timer inside the process gdb had stopped — and killing gdb left the main thread deadlocked for good, so only restarting the addon recovered it. A means of diagnosis may not stop the product: this one attaches to a live process and is triggered by a verdict with a known history of false positives, so it is switched on deliberately while somebody is watching, and `timeout` counts its seconds from outside.
|
|
4
|
+
|
|
5
|
+
## 2.79.0
|
|
6
|
+
|
|
7
|
+
- **Fix**: Starting an encoder no longer stops another one. The line that did it looked for a live run whose own start was not below the new one's and killed it — a rule left from when a session held exactly one run and "the previous one" meant "the only one". Once a session could hold several, it began killing runs the plan had decided to keep: 294 stops for that reason in eight minutes of field 2026-09-05, against four starts a viewer had asked for. Who is stopped is decided in one place now, and starting is not it.
|
|
8
|
+
- **Fix**: The stretch an encoder is given reaches it. The plan computed where a run may work to, passed it, and the receiving parameter list did not name it — so it was dropped and the end was computed a second time somewhere else. That was the last remaining second answer to "how far may this run work".
|
|
9
|
+
- **Fix**: An encoder is built and handed back in the same call, so the stretch it is making is held from that instant. It used to answer with nothing and start the encoder behind the answer, leaving the stretch free in the map for as long as that took — and every pass in between started another encoder for it. Measured in the field: 684 starts in 482 seconds, of which 973 answers said the encoder was not there yet, which is every start without exception. Nothing is waited for on that path any more; the one thing it used to wait for was the death of the run it was replacing.
|
|
10
|
+
- **Fix**: Whether to move a running encoder is a comparison with both sides counted, and the terms are measured on this host rather than written into the code. Driving through material that exists costs this run's own encode time for it and costs the swarm the same bytes a second time; moving costs the death of the run, the start of another and the wait for its first output. A run whose speed nothing has measured is now KEPT: moving costs a known amount for an unknown gain, and a run that has produced nothing has nothing worth taking away. It used to answer the other way, and with the whole film ahead already made, every just-started encoder was moved before it produced anything.
|
|
11
|
+
- **New**: What a stop and a start cost is measured (`encode/run-costs.js`). A run times its own death from the moment it is told to stop, and the appearance of its first output from the moment it started. Both were absent: the start was one reading taken once on one machine and written in as a constant, and the other two were not counted at all, so the comparison priced one side of itself.
|
|
12
|
+
- **New**: The encoding orchestrator says what it believes — what is ready, which zones were stated and in what order, which encoders are running and at what speed, and the lowest number a viewer is waiting for. The line was written and called from nowhere, so on 2026-09-05 the question "why did the plan not see the gap the viewer was stopped at" had to be answered by inference and was not answered at all. Printed when it changes rather than on a timer.
|
|
13
|
+
- **Chore**: The piece a run had open when it ended is thrown away by the layer that owns the directories (`encode/open-piece.js`, `SegmentStore.discardOpenPieceOf`), on every ending but the one normal one. It used to be done by the single place that killed a run, and that place is gone.
|
|
14
|
+
|
|
15
|
+
- **Fix**: A seek does one thing — it puts the viewer where they now are. It used to do eleven, and wrote that position into five places: two fields on the picture's session, two on the soundtrack's, and the viewer. It also asked whether the jump would drag another viewer back, started an encoder itself, cancelled outstanding requests, backed off a segment to the preceding keyframe and set a timer to restart ffmpeg — so it was a third authority over the encoders, and not one of its branches ever asked what had already been made. A viewer jumping into a stretch that was finished and on disk got a fresh encoder for it.
|
|
16
|
+
- **Fix**: A viewer's position is held in SECONDS of film and nowhere else. It used to carry a segment number too, and the picture and the soundtrack of one film are cut independently and into different numbers of pieces — 454 against 401 on the field file of 2026-09-05 — so piece 48 of one is not the same moment as piece 48 of the other. Whoever holds a cut grid turns the seconds into their own numbers.
|
|
17
|
+
- **New**: The page says whether the picture is moving, and says it the moment it changes rather than at the next ten-second report. A viewer who has stopped consumes nothing, so nothing in front of them ever falls due and the work goes to whoever is watching — which follows from one boolean without any rule about pauses anywhere. Inferring it from a position that has not moved takes two reports and is wrong whenever a browser holding a full cushion goes quiet between segments, which it does.
|
|
18
|
+
- **Fix**: Below realtime the priority map says how many encoders are needed, and the zones grow rather than being equal. An encoder starting at `q` stays ahead of a viewer at `p` for `(q - p) x s / (1 - s)`, which grows with its distance from them — so equal shares are the wrong division, and by a wide margin: on the addon host with 2400 s of film left, 120 s held and 0.5x, equal shares need twenty encoders and growing ones need five (120, 240, 480, 960, 1920). Each zone is exactly as long as its bound allows, which makes the count the smallest that holds the viewer.
|
|
19
|
+
- **Fix**: How many encoders may run is the smallest of three measured limits, not one. The processor was the only one asked. Every encoder reads the same torrent, so together they cannot consume faster than it is delivered — on the field file a second of film weighs 842 KB against 2119-3347 KB/s from the swarm, so the speeds together may reach 2.5-4x, and a copy the processor would run at 8x runs at four. Encoders placed apart hold reading windows that do not overlap, so the piece store must hold their sum; on the addon host it was already full on the readers of one viewer. Which limit decided is said in the line.
|
|
20
|
+
- **New**: A piece is proven finished by the encoder's own word for it: ffmpeg names each file on a channel of its own the moment it closes it, and that name is kept. What stood there was the existence of the NEXT file, with two exceptions bolted on because it is not true — it is never true of the last piece of a run, so the first segment of every run was held from the viewer. Measured 2026-08-09: #807 held while it lay on disk, and in August the same shape held #317 for 46 seconds and then answered 404 to a browser that had given up.
|
|
21
|
+
- **Chore**: The priority map is a layer of its own (`services/priority/`), below both orchestrators and depending on nothing. It knows only priorities: what is downloaded is the download orchestrator's own knowledge, what is encoded is the encoding orchestrator's, and neither is visible from the map.
|
|
22
|
+
- **Chore**: A check on the shape rather than the behaviour (`test/one-authority.test.js`): one place stops an encoder for scheduling reasons, one place builds one, a start stops nothing, a seek starts nothing, and the stretch computed by the plan is the one handed over. Every rule in it is one that was broken on 2026-09-05.
|
|
23
|
+
|
|
1
24
|
## 2.78.0
|
|
2
25
|
|
|
3
26
|
- **Fix**: One authority over the encoders, where there were three. A viewer watching an episode on 2026-09-05 had their picture stop for 125 seconds while the proxy spawned and killed an encoder every 350-700ms, producing nothing. Two decisions were contradicting each other on every pass: the plan commanded a start inside the viewer's window, at #46; `planRunInterval` in the session manager moved that start to #78, because it counted every live run as claiming up to its head plus the look-ahead; and the plan then saw a run at #78 against a window of [27, 57], found no overlap and killed it as "nothing it was given is wanted". Neither coverage nor demand changed between two such passes, so the same start was commanded again, for as long as anybody watched. `planRunInterval` is deleted: where a run starts is the plan's decision and nothing moves it, and how far it may work is read off the one coverage map the plan itself uses.
|
package/bin/cli.js
CHANGED
|
@@ -85,6 +85,10 @@ program
|
|
|
85
85
|
.option("--no-transcode-audio", "Disable optional HLS AAC audio transcoding")
|
|
86
86
|
.option("--no-port-mapping", "Disable automatic UPnP/NAT-PMP port mapping")
|
|
87
87
|
.option("--delivery-sink", "Serve /api/delivery-sink, a torrent-free byte stream for delivery testing")
|
|
88
|
+
.option(
|
|
89
|
+
"--usrsctp-state",
|
|
90
|
+
"Read usrsctp's association state with gdb when a wedge is declared. OFF by default: gdb attaches to THIS process and stops every thread of it while it works."
|
|
91
|
+
)
|
|
88
92
|
.option("--max-disk-bytes <bytes>", "Cap total downloaded torrent data (0 = disabled; default min(10GB, half free disk))")
|
|
89
93
|
.option("--memory-bytes <bytes>", "Per-torrent budget for pieces kept in memory before spilling to disk (default 512MB)")
|
|
90
94
|
.option("--ffmpeg-bin <path>", "Path to ffmpeg binary")
|
|
@@ -345,7 +349,22 @@ try {
|
|
|
345
349
|
// declared (roadmap item 11) — no source rebuild, the module ships
|
|
346
350
|
// unstripped. A host without gdb just never gets a reading, the same way a
|
|
347
351
|
// host without tcpdump never gets a packet capture.
|
|
348
|
-
|
|
352
|
+
// OFF UNLESS ASKED FOR, and the reason is a field incident rather than
|
|
353
|
+
// caution. On 2026-09-05 a delivery probe declared a wedge that lasted half a
|
|
354
|
+
// second and cleared itself; the reading it triggered attached gdb to this
|
|
355
|
+
// process, which stopped all eighty of its threads. The log ended mid-second,
|
|
356
|
+
// /healthz stopped answering, and the viewer waited a minute and was told the
|
|
357
|
+
// proxy had sent no video. The fifteen-second guard could not fire — it is a
|
|
358
|
+
// timer inside the process gdb had stopped — and killing gdb left the main
|
|
359
|
+
// thread deadlocked for good, so only restarting the addon recovered it.
|
|
360
|
+
//
|
|
361
|
+
// A means of diagnosis may not stop the product. This one attaches to a live
|
|
362
|
+
// process and is triggered by a verdict with a known history of false
|
|
363
|
+
// positives, so it is a thing to switch on deliberately while watching, not
|
|
364
|
+
// something that arms itself.
|
|
365
|
+
usrsctpStateReader = options.usrsctpState === true
|
|
366
|
+
? createUsrsctpStateReader({ log: (message) => logger.info(message) })
|
|
367
|
+
: null;
|
|
349
368
|
|
|
350
369
|
// What this process holds, once a minute. The kernel killed the proxy on
|
|
351
370
|
// 2026-08-28 at 2.4 GB resident and the log had never said a word about
|
package/biome.json
CHANGED
package/package.json
CHANGED
|
@@ -33,11 +33,16 @@ export async function handleApiTranscodeSessionNetReportPost(req, reply, { hlsSe
|
|
|
33
33
|
// the proxy uses to tell the viewers of one session apart, and a report
|
|
34
34
|
// without them is still a truthful reading of somebody's link.
|
|
35
35
|
const consumerId = typeof body.consumerId === "string" ? body.consumerId.trim() : "";
|
|
36
|
+
// Whether the picture is moving. Absent from a page that does not say, and
|
|
37
|
+
// then the viewer counts as playing, which is what every page meant before it
|
|
38
|
+
// could say otherwise.
|
|
39
|
+
const playing = typeof body.playing === "boolean" ? body.playing : undefined;
|
|
36
40
|
const positionSeconds = Number(body.positionSeconds);
|
|
37
41
|
const recorded = hlsSessionManager.recordNetReport(sessionId, {
|
|
38
42
|
linkMbps,
|
|
39
43
|
bufferedAheadSec,
|
|
40
44
|
consumerId,
|
|
45
|
+
playing,
|
|
41
46
|
positionSeconds:
|
|
42
47
|
Number.isFinite(positionSeconds) && positionSeconds >= 0 ? positionSeconds : undefined
|
|
43
48
|
});
|
|
@@ -110,7 +110,10 @@ export function planEncoders({
|
|
|
110
110
|
runs,
|
|
111
111
|
maxRuns,
|
|
112
112
|
segmentSeconds,
|
|
113
|
-
restartCostSec
|
|
113
|
+
restartCostSec,
|
|
114
|
+
killCostSec = 0,
|
|
115
|
+
firstByteWaitSec = 0,
|
|
116
|
+
refetchSecPerFilmSecond = 0
|
|
114
117
|
}) {
|
|
115
118
|
/** @type {PlanAction[]} */
|
|
116
119
|
const stops = [];
|
|
@@ -174,14 +177,35 @@ export function planEncoders({
|
|
|
174
177
|
continue;
|
|
175
178
|
}
|
|
176
179
|
|
|
177
|
-
//
|
|
178
|
-
// Moving costs one restart. Both are measured; neither is chosen here.
|
|
180
|
+
// WHICH IS CHEAPER, AND BOTH SIDES COUNTED WHOLE.
|
|
179
181
|
//
|
|
180
|
-
//
|
|
181
|
-
//
|
|
182
|
-
//
|
|
183
|
-
|
|
184
|
-
|
|
182
|
+
// Driving through material that exists costs this run's own encode time for
|
|
183
|
+
// it, and costs the swarm the same bytes a second time — the film has to be
|
|
184
|
+
// fetched again to be encoded again.
|
|
185
|
+
//
|
|
186
|
+
// Moving costs the death of this run, the start of another, and the wait
|
|
187
|
+
// for the first bytes at the new position. The last of those is the largest
|
|
188
|
+
// in the field and the one nothing measures yet; while it is unmeasured it
|
|
189
|
+
// counts as zero, which makes moving look cheaper than it is.
|
|
190
|
+
//
|
|
191
|
+
// A run whose speed nothing has measured yet cannot be compared at all, and
|
|
192
|
+
// then it is KEPT. Moving costs a known amount for an unknown gain, and a
|
|
193
|
+
// fresh run has produced nothing, so taking its work away is certainly a
|
|
194
|
+
// loss. This used to answer the other way, and every just-started run was
|
|
195
|
+
// moved the moment anything ahead of it was covered — which, once the whole
|
|
196
|
+
// film ahead had been made, was always.
|
|
197
|
+
// Both sides have to be known for the comparison to mean anything. The
|
|
198
|
+
// encoder's own speed is one; what the swarm charges to fetch the same
|
|
199
|
+
// bytes again is the other, and where nothing has measured it the sum is
|
|
200
|
+
// not a cost but a fragment of one. Answering from a fragment biases the
|
|
201
|
+
// decision one way — towards moving, since the missing term is on the
|
|
202
|
+
// driving side — so an unknown term is a reason to keep, exactly as an
|
|
203
|
+
// unmeasured speed is.
|
|
204
|
+
const known = run.speedX > 0 && refetchSecPerFilmSecond > 0;
|
|
205
|
+
const refetchSec = coveredAhead * segmentSeconds * refetchSecPerFilmSecond;
|
|
206
|
+
const driveSec = known ? (coveredAhead * segmentSeconds) / run.speedX + refetchSec : null;
|
|
207
|
+
const moveSec = restartCostSec + killCostSec + firstByteWaitSec;
|
|
208
|
+
if (driveSec === null || driveSec <= moveSec) {
|
|
185
209
|
surviving.add(run);
|
|
186
210
|
keeps.push({ type: "keep", run, from: run.head, to: run.to });
|
|
187
211
|
continue;
|
|
@@ -194,10 +218,12 @@ export function planEncoders({
|
|
|
194
218
|
run,
|
|
195
219
|
from: gap,
|
|
196
220
|
to: endOfStretch(gap, free),
|
|
197
|
-
because:
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
221
|
+
because:
|
|
222
|
+
`driving through ${coveredAhead} covered segment(s) costs ${driveSec.toFixed(2)}s ` +
|
|
223
|
+
`(encode ${((coveredAhead * segmentSeconds) / run.speedX).toFixed(2)}s + ` +
|
|
224
|
+
`refetch ${refetchSec.toFixed(2)}s) against ${moveSec.toFixed(2)}s to move ` +
|
|
225
|
+
`(kill ${killCostSec.toFixed(2)}s + start ${restartCostSec.toFixed(2)}s + ` +
|
|
226
|
+
`first bytes ${firstByteWaitSec.toFixed(2)}s)`
|
|
201
227
|
});
|
|
202
228
|
}
|
|
203
229
|
|
|
@@ -96,6 +96,28 @@ export class EncodeRun {
|
|
|
96
96
|
/** @type {number} */
|
|
97
97
|
#startedAt = 0;
|
|
98
98
|
|
|
99
|
+
/** Half a name left over from the last chunk of the encoder's own channel. */
|
|
100
|
+
#closedTail = "";
|
|
101
|
+
|
|
102
|
+
/**
|
|
103
|
+
* When this run was told to stop, so the death itself can be priced.
|
|
104
|
+
*
|
|
105
|
+
* The plan weighs moving an encoder against letting it drive on, and dying is
|
|
106
|
+
* one of the terms. It was measured in the field at 430-729 ms — larger than
|
|
107
|
+
* the start it is added to — by a log line that lived in the one place that
|
|
108
|
+
* killed a run. That place is gone, so the run times its own death.
|
|
109
|
+
*/
|
|
110
|
+
#stopOrderedAt = 0;
|
|
111
|
+
|
|
112
|
+
/**
|
|
113
|
+
* When the first thing this run ever produced appeared.
|
|
114
|
+
*
|
|
115
|
+
* The other term the plan needs: a run started where nothing is downloaded
|
|
116
|
+
* waits for the swarm before it can encode a frame, and that wait is the
|
|
117
|
+
* largest part of what a restart costs. Nothing measured it before.
|
|
118
|
+
*/
|
|
119
|
+
#firstOutputAt = 0;
|
|
120
|
+
|
|
99
121
|
/** @type {number} */
|
|
100
122
|
#speedX = 0;
|
|
101
123
|
|
|
@@ -132,6 +154,9 @@ export class EncodeRun {
|
|
|
132
154
|
* @param {{ info: (line: string) => void, warn: (line: string) => void, error?: (line: string) => void }} params.logger
|
|
133
155
|
* @param {() => number} [params.now]
|
|
134
156
|
* @param {(ended: RunEnded) => void} [params.onEnded]
|
|
157
|
+
* @param {(name: string) => void} [params.onClosed] - Called with the file name
|
|
158
|
+
* of every piece the encoder has FINISHED writing, as the encoder itself
|
|
159
|
+
* names it on its own channel.
|
|
135
160
|
* @param {(progress: { processedSeconds: number | null, speed: string | null }) => void} [params.onProgress]
|
|
136
161
|
* Called for every `-progress` report. Seconds count from the START OF THIS
|
|
137
162
|
* RUN on both branches — neither `-output_ts_offset` nor `-copyts` changes
|
|
@@ -158,6 +183,7 @@ export class EncodeRun {
|
|
|
158
183
|
logger,
|
|
159
184
|
now,
|
|
160
185
|
onEnded,
|
|
186
|
+
onClosed,
|
|
161
187
|
onProgress,
|
|
162
188
|
lastSegmentIndex,
|
|
163
189
|
inputUnavailable,
|
|
@@ -174,6 +200,9 @@ export class EncodeRun {
|
|
|
174
200
|
this.now = typeof now === "function" ? now : Date.now;
|
|
175
201
|
this.onEnded = typeof onEnded === "function" ? onEnded : () => {};
|
|
176
202
|
this.onProgress = typeof onProgress === "function" ? onProgress : () => {};
|
|
203
|
+
// Told the NAME of every piece the encoder has closed. The name is the
|
|
204
|
+
// proof it is whole; nothing else here can prove that.
|
|
205
|
+
this.onClosed = typeof onClosed === "function" ? onClosed : () => {};
|
|
177
206
|
this.lastSegmentIndex = typeof lastSegmentIndex === "function" ? lastSegmentIndex : () => null;
|
|
178
207
|
this.inputUnavailable = typeof inputUnavailable === "function" ? inputUnavailable : () => false;
|
|
179
208
|
this.argsDescribed = argsDescribed;
|
|
@@ -304,6 +333,16 @@ export class EncodeRun {
|
|
|
304
333
|
*/
|
|
305
334
|
#wire(process) {
|
|
306
335
|
process.stdout?.on("data", (chunk) => this.#readProgress(String(chunk)));
|
|
336
|
+
// THE CHANNEL THE ENCODER NAMES ITS FINISHED PIECES ON.
|
|
337
|
+
//
|
|
338
|
+
// A name arrives here when ffmpeg CLOSES the file, so the name is proof the
|
|
339
|
+
// piece is whole — measured on the addon host 2026-09-05. Nothing else can
|
|
340
|
+
// prove it: a file on disk may still be being written, and the only other
|
|
341
|
+
// evidence available was the existence of the NEXT file, which never comes
|
|
342
|
+
// for the last piece of a run.
|
|
343
|
+
//
|
|
344
|
+
// Lines can arrive split, so what is left over is kept for the next chunk.
|
|
345
|
+
process.stdio?.[3]?.on("data", (chunk) => this.#readClosedPieces(String(chunk)));
|
|
307
346
|
process.stderr?.on("data", (chunk) => {
|
|
308
347
|
const line = String(chunk).trim();
|
|
309
348
|
if (line.length > 0) {
|
|
@@ -358,6 +397,9 @@ export class EncodeRun {
|
|
|
358
397
|
noteProduced(index) {
|
|
359
398
|
if (Number.isInteger(index) && index >= this.from) {
|
|
360
399
|
this.#produced.add(index);
|
|
400
|
+
if (this.#firstOutputAt === 0) {
|
|
401
|
+
this.#firstOutputAt = this.now();
|
|
402
|
+
}
|
|
361
403
|
if (this.#state === ENCODE_RUN_STATE.STARTING) {
|
|
362
404
|
this.#transition(ENCODE_RUN_EVENT.FIRST_SEGMENT);
|
|
363
405
|
}
|
|
@@ -369,6 +411,25 @@ export class EncodeRun {
|
|
|
369
411
|
*
|
|
370
412
|
* @param {number} speedX - Times realtime.
|
|
371
413
|
*/
|
|
414
|
+
/**
|
|
415
|
+
* Names of finished pieces, as the encoder writes them.
|
|
416
|
+
*
|
|
417
|
+
* @param {string} text
|
|
418
|
+
*/
|
|
419
|
+
#readClosedPieces(text) {
|
|
420
|
+
this.#closedTail += text;
|
|
421
|
+
const lines = this.#closedTail.split(/\r?\n/);
|
|
422
|
+
// The last piece of the chunk may be half a name; it waits for the rest.
|
|
423
|
+
this.#closedTail = lines.pop() ?? "";
|
|
424
|
+
for (const line of lines) {
|
|
425
|
+
const name = line.trim();
|
|
426
|
+
if (name.length === 0) {
|
|
427
|
+
continue;
|
|
428
|
+
}
|
|
429
|
+
this.onClosed(name);
|
|
430
|
+
}
|
|
431
|
+
}
|
|
432
|
+
|
|
372
433
|
noteSpeed(speedX) {
|
|
373
434
|
if (Number.isFinite(speedX) && speedX > 0) {
|
|
374
435
|
this.#speedX = speedX;
|
|
@@ -389,6 +450,7 @@ export class EncodeRun {
|
|
|
389
450
|
}
|
|
390
451
|
this.#stopping = true;
|
|
391
452
|
this.#stopReason = because;
|
|
453
|
+
this.#stopOrderedAt = this.now();
|
|
392
454
|
this.#transition(ENCODE_RUN_EVENT.STOP_ORDERED);
|
|
393
455
|
// A suspended process does not act on SIGTERM until it is continued, so the
|
|
394
456
|
// wait for its exit would never end. Let it run before asking it to stop.
|
|
@@ -554,6 +616,14 @@ export class EncodeRun {
|
|
|
554
616
|
to: this.to,
|
|
555
617
|
reached: this.reached,
|
|
556
618
|
livedMs,
|
|
619
|
+
// How long dying took, and how long the first output took to appear.
|
|
620
|
+
// Null where the run was never told to stop, or never produced anything:
|
|
621
|
+
// an absent measurement says so rather than reading as zero.
|
|
622
|
+
dyingMs: this.#stopOrderedAt > 0 ? this.now() - this.#stopOrderedAt : null,
|
|
623
|
+
firstOutputMs:
|
|
624
|
+
this.#firstOutputAt > 0 && this.#startedAt > 0
|
|
625
|
+
? this.#firstOutputAt - this.#startedAt
|
|
626
|
+
: null,
|
|
557
627
|
normal: ending === ENCODE_EXIT.COMPLETE,
|
|
558
628
|
lastError: this.lastError
|
|
559
629
|
};
|
|
@@ -39,6 +39,8 @@ import { rmSync } from "node:fs";
|
|
|
39
39
|
import os from "node:os";
|
|
40
40
|
import path from "node:path";
|
|
41
41
|
|
|
42
|
+
import { discardOpenPiece } from "./open-piece.js";
|
|
43
|
+
|
|
42
44
|
/** Where every output's segments live. One root for the process. */
|
|
43
45
|
export const DEFAULT_STORE_ROOT = path.join(os.tmpdir(), "torrent-tv-hls");
|
|
44
46
|
|
|
@@ -256,6 +258,30 @@ export class SegmentStore {
|
|
|
256
258
|
return proven.sort((left, right) => left - right);
|
|
257
259
|
}
|
|
258
260
|
|
|
261
|
+
/**
|
|
262
|
+
* Whether this piece is finished, and may therefore be served.
|
|
263
|
+
*
|
|
264
|
+
* Two proofs, and the first is the good one:
|
|
265
|
+
*
|
|
266
|
+
* 1. **the encoder said so** — it names each file on a channel of its own the
|
|
267
|
+
* moment it closes it, so the name is the writer's own statement that the
|
|
268
|
+
* piece is whole;
|
|
269
|
+
* 2. **the next file exists** — which only proves it for pieces this process
|
|
270
|
+
* did not watch being written, left by an earlier life of it. It is not
|
|
271
|
+
* true of the last piece of any run, and that is what used to hold the
|
|
272
|
+
* first segment of every run from the viewer.
|
|
273
|
+
*
|
|
274
|
+
* @param {string} key
|
|
275
|
+
* @param {number} index
|
|
276
|
+
* @returns {boolean}
|
|
277
|
+
*/
|
|
278
|
+
isClosed(key, index) {
|
|
279
|
+
if (this.#closed.get(key)?.has(index)) {
|
|
280
|
+
return true;
|
|
281
|
+
}
|
|
282
|
+
return this.refresh(key).byNumber.has(index + 1);
|
|
283
|
+
}
|
|
284
|
+
|
|
259
285
|
/**
|
|
260
286
|
* Say that a segment is closed for a reason the disk cannot show.
|
|
261
287
|
*
|
|
@@ -277,6 +303,39 @@ export class SegmentStore {
|
|
|
277
303
|
known.add(index);
|
|
278
304
|
}
|
|
279
305
|
|
|
306
|
+
/**
|
|
307
|
+
* Throw away the piece a run had open when it ended, if it is unusable.
|
|
308
|
+
*
|
|
309
|
+
* The store owns this output's directory and knows how its files are named,
|
|
310
|
+
* so it is the one place that can answer which file a run left open. The
|
|
311
|
+
* judging of a NON-EMPTY file — does it carry every track it should — needs
|
|
312
|
+
* the output's init bytes and belongs to whoever holds them; passed in, and
|
|
313
|
+
* absent it only an empty file is removed, which is the case that caused this
|
|
314
|
+
* to be written (a run stopped 548 ms after starting left a zero-byte file
|
|
315
|
+
* whose name then read as a segment made).
|
|
316
|
+
*
|
|
317
|
+
* @param {string} key
|
|
318
|
+
* @param {{ from: number, to: number } | null} within - The run's own
|
|
319
|
+
* numbers: several runs write into one directory, so the piece to discard
|
|
320
|
+
* has to be looked for inside the stretch the ended run was given.
|
|
321
|
+
* @param {((raw: Buffer) => boolean) | null} [judgeUsable]
|
|
322
|
+
* @returns {Promise<number | null>} The segment number removed, or null.
|
|
323
|
+
*/
|
|
324
|
+
async discardOpenPieceOf(key, within, judgeUsable = null) {
|
|
325
|
+
const format = this.#formats.get(key);
|
|
326
|
+
if (!format) {
|
|
327
|
+
return null;
|
|
328
|
+
}
|
|
329
|
+
const removed = await discardOpenPiece(this.directoryFor(key), format, within, judgeUsable);
|
|
330
|
+
if (removed !== null) {
|
|
331
|
+
this.#held.delete(key);
|
|
332
|
+
this.#logger?.info?.(
|
|
333
|
+
`segment store: discarded the open piece #${removed} of ${key.slice(0, 60)}`
|
|
334
|
+
);
|
|
335
|
+
}
|
|
336
|
+
return removed;
|
|
337
|
+
}
|
|
338
|
+
|
|
280
339
|
/**
|
|
281
340
|
* The one number in this output whose closure nothing on disk proves.
|
|
282
341
|
*
|
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @file The piece a run had open when it ended.
|
|
3
|
+
*
|
|
4
|
+
* A fact of a run's output directory, and therefore of the encoding layer. It
|
|
5
|
+
* lived in the eleven-thousand-line file that is being taken apart, where it was
|
|
6
|
+
* called by the one place that killed a run; stopping is decided in one place
|
|
7
|
+
* now and carried out in another, so the cleanup belongs to the layer that owns
|
|
8
|
+
* the directories rather than to whoever happened to do the killing.
|
|
9
|
+
*/
|
|
10
|
+
|
|
11
|
+
import { readdir, readFile, stat, unlink } from "node:fs/promises";
|
|
12
|
+
import path from "node:path";
|
|
13
|
+
|
|
14
|
+
/**
|
|
15
|
+
* Remove the piece a run had open when it ended, if that piece is unusable.
|
|
16
|
+
*
|
|
17
|
+
* The `segment` muxer creates its output file when it OPENS it and writes into
|
|
18
|
+
* it until the next cut, so at any instant exactly one file in a run's
|
|
19
|
+
* directory is unfinished: the highest-numbered one. A run that reaches the end
|
|
20
|
+
* of its work closes that file and it is a good piece; a run killed for a seek
|
|
21
|
+
* does not — measured 2026-09-03, ffmpeg exited 19 ms after SIGTERM and left
|
|
22
|
+
* `segment-00025.mp4` at zero bytes, which then closed the only hole in the
|
|
23
|
+
* numbering and convinced the look-ahead to keep the encoder stopped for having
|
|
24
|
+
* "produced" it.
|
|
25
|
+
*
|
|
26
|
+
* Only an unusable piece goes. A run stopped between two cuts leaves a finished
|
|
27
|
+
* file behind, and deleting good output would mean making it a second time.
|
|
28
|
+
*
|
|
29
|
+
* @param {string | null | undefined} runDirPath
|
|
30
|
+
* @param {{ isSegmentFileName: (name: string) => boolean, segmentIndexFromName: (name: string) => number }} segmentFormat
|
|
31
|
+
* @param {((raw: Buffer) => boolean) | null} judgeUsable - Whether a non-empty
|
|
32
|
+
* piece carries what it should. Null where nothing can say, and then only an
|
|
33
|
+
* empty file is removed.
|
|
34
|
+
* @returns {Promise<number | null>} The segment number removed, or null.
|
|
35
|
+
*/
|
|
36
|
+
export async function discardOpenPiece(runDirPath, segmentFormat, within, judgeUsable) {
|
|
37
|
+
if (!runDirPath || typeof segmentFormat?.isSegmentFileName !== "function") {
|
|
38
|
+
return null;
|
|
39
|
+
}
|
|
40
|
+
// Only inside the stretch the ended run was given. Every run of an output
|
|
41
|
+
// writes into one directory now — they are kept apart by their intervals
|
|
42
|
+
// rather than by a directory each — so the highest-numbered file in there may
|
|
43
|
+
// belong to a run that is still going, and removing it would take away a
|
|
44
|
+
// piece somebody is producing.
|
|
45
|
+
const from = Number.isInteger(within?.from) ? within.from : 0;
|
|
46
|
+
const to = Number.isInteger(within?.to) && within.to >= from ? within.to : Number.MAX_SAFE_INTEGER;
|
|
47
|
+
let highest = null;
|
|
48
|
+
try {
|
|
49
|
+
for (const name of await readdir(runDirPath)) {
|
|
50
|
+
if (!segmentFormat.isSegmentFileName(name)) {
|
|
51
|
+
continue;
|
|
52
|
+
}
|
|
53
|
+
const index = segmentFormat.segmentIndexFromName(name);
|
|
54
|
+
if (index < from || index > to) {
|
|
55
|
+
continue;
|
|
56
|
+
}
|
|
57
|
+
if (index >= 0 && (highest === null || index > highest.index)) {
|
|
58
|
+
highest = { index, name };
|
|
59
|
+
}
|
|
60
|
+
}
|
|
61
|
+
} catch {
|
|
62
|
+
return null; // The run wrote nothing, or its directory is already gone.
|
|
63
|
+
}
|
|
64
|
+
if (highest === null) {
|
|
65
|
+
return null;
|
|
66
|
+
}
|
|
67
|
+
const filePath = path.join(runDirPath, highest.name);
|
|
68
|
+
let unusable = false;
|
|
69
|
+
try {
|
|
70
|
+
const info = await stat(filePath);
|
|
71
|
+
if (info.size === 0) {
|
|
72
|
+
unusable = true;
|
|
73
|
+
} else if (typeof judgeUsable === "function") {
|
|
74
|
+
unusable = !judgeUsable(await readFile(filePath));
|
|
75
|
+
}
|
|
76
|
+
} catch {
|
|
77
|
+
return null; // Gone between the listing and the question.
|
|
78
|
+
}
|
|
79
|
+
if (!unusable) {
|
|
80
|
+
return null;
|
|
81
|
+
}
|
|
82
|
+
try {
|
|
83
|
+
await unlink(filePath);
|
|
84
|
+
return highest.index;
|
|
85
|
+
} catch {
|
|
86
|
+
return null; // Already removed.
|
|
87
|
+
}
|
|
88
|
+
}
|
|
@@ -0,0 +1,89 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @file How many encoders this machine can afford on one output.
|
|
3
|
+
*
|
|
4
|
+
* Not how many are wanted — that is the demand map's answer — but how many can
|
|
5
|
+
* run at once without making things worse. It is the SMALLEST of several
|
|
6
|
+
* limits, and every one of them is measured:
|
|
7
|
+
*
|
|
8
|
+
* 1. **the processor.** A second encoder slows the first: measured on the addon
|
|
9
|
+
* host at 1.70x for two at 854x480 and 1.98x at 1920x1080. The ladder is
|
|
10
|
+
* walked while the measured penalty still leaves every encoder above
|
|
11
|
+
* realtime, and it stops where the measurements stop rather than continuing
|
|
12
|
+
* a curve two points cannot describe;
|
|
13
|
+
* 2. **the swarm.** Every encoder reads the same torrent, so together they
|
|
14
|
+
* cannot consume faster than it is delivered. On the field file of
|
|
15
|
+
* 2026-09-05 a second of film weighs 842 KB and the swarm gave 2119-3347
|
|
16
|
+
* KB/s, so the encoders' speeds together may reach 2.5-4x — which is why a
|
|
17
|
+
* copy the processor would run at 8x runs at four, and why a third encoder
|
|
18
|
+
* makes all three slower rather than adding anything;
|
|
19
|
+
* 3. **memory for the torrent's pieces.** Encoders placed far apart hold
|
|
20
|
+
* windows that do not overlap, so the store must hold their SUM. On the
|
|
21
|
+
* addon host the store was already full on the readers of one viewer — "6
|
|
22
|
+
* readers want 14 pieces of 14 the store may hold" — so this is the limit
|
|
23
|
+
* that binds first there, not the processor.
|
|
24
|
+
*
|
|
25
|
+
* Which of them bound the answer is returned beside it, because "why is there
|
|
26
|
+
* only one encoder" is otherwise a question no log can answer.
|
|
27
|
+
*/
|
|
28
|
+
|
|
29
|
+
/**
|
|
30
|
+
* @typedef {object} RunBudget
|
|
31
|
+
* @property {number} runs - How many encoders may run on this output.
|
|
32
|
+
* @property {string} because - Which limit decided it.
|
|
33
|
+
*/
|
|
34
|
+
|
|
35
|
+
/**
|
|
36
|
+
* @param {object} params
|
|
37
|
+
* @param {number} params.byProcessor - What the processor allows, from the
|
|
38
|
+
* measured speed and the measured penalty for concurrency. At least one.
|
|
39
|
+
* @param {number} [params.speedX] - The measured speed of one encoder here.
|
|
40
|
+
* @param {number} [params.refetchSecPerFilmSecond] - Seconds of swarm time per
|
|
41
|
+
* second of film: the film's own byte rate over what the swarm delivers. One
|
|
42
|
+
* encoder at speed `s` therefore consumes `s * refetch` of the swarm, and
|
|
43
|
+
* what they may consume together is all of it. Absent where neither rate has
|
|
44
|
+
* been measured, and then the swarm does not bound the answer.
|
|
45
|
+
* @param {number} [params.storeBytes] - What the piece store may hold.
|
|
46
|
+
* @param {number} [params.readerWindowBytes] - What one encoder's reader keeps.
|
|
47
|
+
* @returns {RunBudget}
|
|
48
|
+
*/
|
|
49
|
+
export function affordableRuns({
|
|
50
|
+
byProcessor,
|
|
51
|
+
speedX,
|
|
52
|
+
refetchSecPerFilmSecond,
|
|
53
|
+
storeBytes,
|
|
54
|
+
readerWindowBytes
|
|
55
|
+
}) {
|
|
56
|
+
let runs = Number.isFinite(byProcessor) && byProcessor > 0 ? Math.floor(byProcessor) : 1;
|
|
57
|
+
let because = "the processor";
|
|
58
|
+
|
|
59
|
+
// The swarm. One encoder at speed `s` takes `s * refetch` of what is
|
|
60
|
+
// delivered, and everything running takes the sum, which cannot pass one.
|
|
61
|
+
if (
|
|
62
|
+
Number.isFinite(refetchSecPerFilmSecond) &&
|
|
63
|
+
refetchSecPerFilmSecond > 0 &&
|
|
64
|
+
Number.isFinite(speedX) &&
|
|
65
|
+
speedX > 0
|
|
66
|
+
) {
|
|
67
|
+
const bySwarm = Math.max(1, Math.floor(1 / (speedX * refetchSecPerFilmSecond)));
|
|
68
|
+
if (bySwarm < runs) {
|
|
69
|
+
runs = bySwarm;
|
|
70
|
+
because = "what the swarm delivers";
|
|
71
|
+
}
|
|
72
|
+
}
|
|
73
|
+
|
|
74
|
+
// Memory: encoders far apart hold windows that do not overlap.
|
|
75
|
+
if (
|
|
76
|
+
Number.isFinite(storeBytes) &&
|
|
77
|
+
storeBytes > 0 &&
|
|
78
|
+
Number.isFinite(readerWindowBytes) &&
|
|
79
|
+
readerWindowBytes > 0
|
|
80
|
+
) {
|
|
81
|
+
const byMemory = Math.max(1, Math.floor(storeBytes / readerWindowBytes));
|
|
82
|
+
if (byMemory < runs) {
|
|
83
|
+
runs = byMemory;
|
|
84
|
+
because = "memory for the torrent's pieces";
|
|
85
|
+
}
|
|
86
|
+
}
|
|
87
|
+
|
|
88
|
+
return { runs: Math.max(1, runs), because };
|
|
89
|
+
}
|
|
@@ -620,6 +620,21 @@ export function buildRunCommand({
|
|
|
620
620
|
"0.05",
|
|
621
621
|
"-segment_start_number",
|
|
622
622
|
String(safeIndex),
|
|
623
|
+
// THE ENCODER SAYS WHEN A PIECE IS FINISHED, on a channel of its own.
|
|
624
|
+
//
|
|
625
|
+
// Measured on the addon host 2026-09-05: a name appears in this list when
|
|
626
|
+
// the file is CLOSED, not when it is created — at the third sample
|
|
627
|
+
// `seg-000.mp4` was on disk and absent from the list, and it appeared at
|
|
628
|
+
// the fourth, in the same moment `seg-001.mp4` came into being. So a name
|
|
629
|
+
// here is the writer's own statement that the piece is whole.
|
|
630
|
+
//
|
|
631
|
+
// Without it, a finished file is indistinguishable from one still being
|
|
632
|
+
// written, and the only proof available was the existence of the NEXT
|
|
633
|
+
// one — which never comes for the last piece of every run.
|
|
634
|
+
"-segment_list",
|
|
635
|
+
"pipe:3",
|
|
636
|
+
"-segment_list_flags",
|
|
637
|
+
"+live",
|
|
623
638
|
...explicitTimes,
|
|
624
639
|
segmentFormat.segmentFileNameTemplate()
|
|
625
640
|
);
|