@torrent-tv/proxy 2.76.5 → 2.76.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +5 -0
- package/bin/cli.js +14 -2
- package/package.json +1 -1
- package/server.js +5 -1
- package/services/torrent-worker/client.js +13 -0
- package/services/torrent-worker/pool-adapter.js +384 -370
- package/services/torrent-worker/protocol.js +2 -0
- package/services/torrent-worker/worker.js +23 -0
- package/services/tunnel-client.js +18 -2
package/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,8 @@
|
|
|
1
|
+
## 2.76.6
|
|
2
|
+
|
|
3
|
+
- **New**: The proxy says which films it is already downloading, in the same health poll the browser makes before it picks one. A viewer sent to the proxy that has their film costs it the encode and nothing else — the torrent is there, its pieces are there, and where the output is identical they share that too — while the same viewer sent anywhere else starts the download from nothing. Answered from the torrent thread, which is the only truthful record: the main thread keeps stand-ins that are cleared on shutdown alone, so they name films this proxy let go of hours ago. A proxy that cannot answer is scored on its machine as before.
|
|
4
|
+
- **Chore**: Two doc comments in the torrent pool's interface described methods that had been moved or renamed out from under them, leaving the methods below without one. Each is back over its own.
|
|
5
|
+
|
|
1
6
|
## 2.76.5
|
|
2
7
|
|
|
3
8
|
- **Chore**: What encoding costs this machine is a class of its own (`services/quality/EncodeCost.js`), the third part taken out of the session manager. One subject — seconds of work per second of video — asked about a picture being re-encoded, a soundtrack, a copy, everything running beside the rung being judged, and turned into the list of heights the machine can hold. It holds what an encoder taught it, since it is the only thing that reads those figures, and it is given what it cannot work out for itself: which sessions belong to one file, the host's own readings, how many encoders are running, and what the file costs merely by being fetched. The host is asked at the moment of the question rather than copied when the object is made — the share of the machine that is free is re-read every few seconds, and a copy would price every later rung against a machine that has gone. The manager keeps the three methods that LEARN those costs; moving them is the next step.
|
package/bin/cli.js
CHANGED
|
@@ -473,8 +473,20 @@ try {
|
|
|
473
473
|
onSignal(sessionId, signal) {
|
|
474
474
|
webRtcManager?.handleSignal(sessionId, signal);
|
|
475
475
|
},
|
|
476
|
-
onHealthRequest() {
|
|
477
|
-
|
|
476
|
+
async onHealthRequest() {
|
|
477
|
+
// Which films this proxy holds travels with the health poll the browser
|
|
478
|
+
// already makes before it picks one: a viewer sent to a proxy that is
|
|
479
|
+
// downloading their film costs it the encode and nothing else, while the
|
|
480
|
+
// same viewer sent anywhere else starts the download from nothing. Asked
|
|
481
|
+
// of the worker, so a film this proxy let go of is not claimed.
|
|
482
|
+
let holds = [];
|
|
483
|
+
try {
|
|
484
|
+
holds = (await started?.torrentPool?.heldTorrents?.()) ?? [];
|
|
485
|
+
} catch {
|
|
486
|
+
// silent-ok: a proxy that cannot say what it holds is scored on its
|
|
487
|
+
// machine alone, which is what every proxy was scored on until now.
|
|
488
|
+
}
|
|
489
|
+
return { metrics: collectHealthMetrics(), holds };
|
|
478
490
|
},
|
|
479
491
|
// Whether this host could sustain a file it has only been told about. The
|
|
480
492
|
// same arithmetic the first offer uses, against this host's own startup
|
package/package.json
CHANGED
package/server.js
CHANGED
|
@@ -378,6 +378,10 @@ export async function startProxyServer({
|
|
|
378
378
|
// recorded from a browser request and published from the pool's side, so
|
|
379
379
|
// resolving one into the other is what lets the two ends agree on what
|
|
380
380
|
// they are both calling "sourceKey".
|
|
381
|
-
sourceRegistry
|
|
381
|
+
sourceRegistry,
|
|
382
|
+
// Which films this host holds, for the health poll the browser makes before
|
|
383
|
+
// it picks a proxy. The pool is on the worker thread and this is the way to
|
|
384
|
+
// it from the process that answers that poll.
|
|
385
|
+
torrentPool
|
|
382
386
|
};
|
|
383
387
|
}
|
|
@@ -403,6 +403,19 @@ export class TorrentWorkerClient {
|
|
|
403
403
|
return this.#caller.call(Command.TORRENT_TOTALS, {});
|
|
404
404
|
}
|
|
405
405
|
|
|
406
|
+
/**
|
|
407
|
+
* Which films this proxy holds right now, and how much of each.
|
|
408
|
+
*
|
|
409
|
+
* Asked of the worker rather than of the main thread's map of stand-ins: that
|
|
410
|
+
* map is only cleared on shutdown, so it names films this proxy let go of
|
|
411
|
+
* hours ago.
|
|
412
|
+
*
|
|
413
|
+
* @returns {Promise<{ held: { infoHash: string, progress: number, bytes: number }[] }>}
|
|
414
|
+
*/
|
|
415
|
+
async heldTorrents() {
|
|
416
|
+
return this.#caller.call(Command.HELD_TORRENTS, {});
|
|
417
|
+
}
|
|
418
|
+
|
|
406
419
|
/**
|
|
407
420
|
* Reorder piece selection around a read position (seek prioritisation).
|
|
408
421
|
*
|
|
@@ -1,370 +1,384 @@
|
|
|
1
|
-
/**
|
|
2
|
-
* @file `TorrentPool`'s interface, served from the worker thread.
|
|
3
|
-
*
|
|
4
|
-
* The routes, the planner, the health report and the session manager all reach
|
|
5
|
-
* for a torrent pool and use it the same handful of ways. Rather than rewrite
|
|
6
|
-
* every one of them to thread a `sourceKey` through and await what used to be
|
|
7
|
-
* immediate, this presents the shape they already expect and does the thread
|
|
8
|
-
* hop behind it. Swapping the implementation is then a one-line change at
|
|
9
|
-
* construction, and the call sites are untouched — which is what keeps a change
|
|
10
|
-
* of this size reviewable.
|
|
11
|
-
*
|
|
12
|
-
* Two accommodations are needed, and both are deliberate:
|
|
13
|
-
*
|
|
14
|
-
* - **`acquireFile` and `prioritizeByteRange` stay synchronous.** They return
|
|
15
|
-
* nothing the caller inspects, so the command is dispatched and not awaited.
|
|
16
|
-
* `acquireFile` hands back a release function exactly as before, which sends
|
|
17
|
-
* its own command when called. Awaiting them would mean touching every call
|
|
18
|
-
* site for no observable gain.
|
|
19
|
-
* - **`getTorrent` needs a `sourceKey`.** Torrent objects cannot cross a
|
|
20
|
-
* thread, so the worker keys them. Callers that have one pass it; the rest
|
|
21
|
-
* get one derived from the source itself, so the identity stays stable
|
|
22
|
-
* across calls for the same torrent.
|
|
23
|
-
*/
|
|
24
|
-
|
|
25
|
-
import { TorrentWorkerClient } from "./client.js";
|
|
26
|
-
import { deriveSourceKey } from "../torrent-source-key.js";
|
|
27
|
-
|
|
28
|
-
/**
|
|
29
|
-
* A torrent pool whose work happens on another thread.
|
|
30
|
-
*
|
|
31
|
-
* See `protocol.js` for why: the torrent was taking ~85% of the main thread and
|
|
32
|
-
* everything owed to a viewer queued behind it.
|
|
33
|
-
*/
|
|
34
|
-
export class WorkerTorrentPool {
|
|
35
|
-
#client;
|
|
36
|
-
/** Stand-ins by source key, so repeat calls return the same object. */
|
|
37
|
-
#torrents = new Map();
|
|
38
|
-
|
|
39
|
-
/**
|
|
40
|
-
* @param {{ maxDiskBytes?: number, memoryBytes?: number, stateDir?: string }} [options]
|
|
41
|
-
*/
|
|
42
|
-
constructor(options = {}) {
|
|
43
|
-
this.#client = new TorrentWorkerClient(options);
|
|
44
|
-
}
|
|
45
|
-
|
|
46
|
-
/**
|
|
47
|
-
* Load (or join) a torrent and return a stand-in for it.
|
|
48
|
-
*
|
|
49
|
-
* @param {"magnet" | "torrent"} sourceType
|
|
50
|
-
* @param {string} source
|
|
51
|
-
* @returns {Promise<object>}
|
|
52
|
-
*/
|
|
53
|
-
async getTorrent(sourceType, source) {
|
|
54
|
-
const sourceKey = await deriveSourceKey(sourceType, source);
|
|
55
|
-
const existing = this.#torrents.get(sourceKey);
|
|
56
|
-
if (existing) {
|
|
57
|
-
return existing;
|
|
58
|
-
}
|
|
59
|
-
const torrent = await this.#client.getTorrent({ sourceKey, sourceType, source });
|
|
60
|
-
this.#torrents.set(sourceKey, torrent);
|
|
61
|
-
return torrent;
|
|
62
|
-
}
|
|
63
|
-
|
|
64
|
-
/**
|
|
65
|
-
* Claim a file for reading; the returned function releases it.
|
|
66
|
-
*
|
|
67
|
-
* Synchronous by design — see the file header.
|
|
68
|
-
*
|
|
69
|
-
* @param {object} torrent - A stand-in from {@link getTorrent}.
|
|
70
|
-
* @param {number} fileIndex
|
|
71
|
-
* @returns {() => void}
|
|
72
|
-
*/
|
|
73
|
-
acquireFile(torrent, fileIndex) {
|
|
74
|
-
const sourceKey = torrent?.sourceKey;
|
|
75
|
-
if (!sourceKey) {
|
|
76
|
-
return () => undefined;
|
|
77
|
-
}
|
|
78
|
-
// Dispatched, not awaited — callers use the result immediately and inspect
|
|
79
|
-
// nothing. But the release MUST NOT overtake it: both are ordinary messages
|
|
80
|
-
// to the worker, and if release arrives first the reader count drops to zero
|
|
81
|
-
// while a read is still running. The idle sweep then removes the torrent AND
|
|
82
|
-
// its downloaded data out from under the encoder — field 2026-08-02:
|
|
83
|
-
// "removed idle torrent ... and its store" mid-playback, after which every
|
|
84
|
-
// read hung and ffmpeg saw an empty input ("Stream ends prematurely at 0").
|
|
85
|
-
// Chaining the release onto the acquire keeps them in order.
|
|
86
|
-
const acquired = this.#client.acquireFile(sourceKey, fileIndex).catch(() => null);
|
|
87
|
-
let released = false;
|
|
88
|
-
return () => {
|
|
89
|
-
if (released) {
|
|
90
|
-
return;
|
|
91
|
-
}
|
|
92
|
-
released = true;
|
|
93
|
-
// Release the claim this call opened, not "the file" — waiting for the
|
|
94
|
-
// acquire is also what tells us which claim that is.
|
|
95
|
-
void acquired
|
|
96
|
-
.then((claimId) => (claimId ? this.#client.releaseFile(claimId) : undefined))
|
|
97
|
-
.catch(() => undefined);
|
|
98
|
-
};
|
|
99
|
-
}
|
|
100
|
-
|
|
101
|
-
/**
|
|
102
|
-
*
|
|
103
|
-
*
|
|
104
|
-
* @
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
*
|
|
112
|
-
*
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
async getFileStats(torrent, fileIndex = null, options = {}) {
|
|
119
|
-
const sourceKey = torrent?.sourceKey;
|
|
120
|
-
if (!sourceKey) {
|
|
121
|
-
return null;
|
|
122
|
-
}
|
|
123
|
-
return this.#client.getFileStats({
|
|
124
|
-
sourceKey,
|
|
125
|
-
fileIndex,
|
|
126
|
-
resumeAnchorByteStart: options?.resumeAnchorByteStart ?? null
|
|
127
|
-
});
|
|
128
|
-
}
|
|
129
|
-
|
|
130
|
-
/**
|
|
131
|
-
*
|
|
132
|
-
*
|
|
133
|
-
*
|
|
134
|
-
*
|
|
135
|
-
*
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
*
|
|
139
|
-
*
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
async getSubtitleTracks(torrent, fileIndex) {
|
|
158
|
-
const sourceKey = torrent?.sourceKey;
|
|
159
|
-
if (!sourceKey) {
|
|
160
|
-
return [];
|
|
161
|
-
}
|
|
162
|
-
const answer = await this.#client.getSubtitleTracks({ sourceKey, fileIndex });
|
|
163
|
-
return Array.isArray(answer?.tracks) ? answer.tracks : [];
|
|
164
|
-
}
|
|
165
|
-
|
|
166
|
-
/**
|
|
167
|
-
* What the container itself declares about its subtitle tracks, in its own
|
|
168
|
-
* order and including the picture-based ones — for lining up against
|
|
169
|
-
* ffmpeg's own numbering.
|
|
170
|
-
*
|
|
171
|
-
* @param {object} torrent
|
|
172
|
-
* @param {number} fileIndex
|
|
173
|
-
* @returns {Promise<object[]>}
|
|
174
|
-
*/
|
|
175
|
-
async getDeclaredSubtitleTracks(torrent, fileIndex) {
|
|
176
|
-
const sourceKey = torrent?.sourceKey;
|
|
177
|
-
if (!sourceKey) {
|
|
178
|
-
return [];
|
|
179
|
-
}
|
|
180
|
-
const answer = await this.#client.getSubtitleTracks({ sourceKey, fileIndex });
|
|
181
|
-
return Array.isArray(answer?.declared) ? answer.declared : [];
|
|
182
|
-
}
|
|
183
|
-
|
|
184
|
-
/**
|
|
185
|
-
* Every track one file declares, read from its own header by the container
|
|
186
|
-
* layer.
|
|
187
|
-
*
|
|
188
|
-
* The audio menu is built from ffmpeg's `-i` banner, which carries neither
|
|
189
|
-
* `FlagOriginal`, `FlagCommentary`, `FlagVisualImpaired`, `FlagEnabled` nor
|
|
190
|
-
* `LanguageBCP47` — so without this a director's commentary and the film
|
|
191
|
-
* itself are indistinguishable in it. Also how a soundtrack shipped as its own
|
|
192
|
-
* file is read: a `.mka` is Matroska and the same reader serves it.
|
|
193
|
-
*
|
|
194
|
-
* @param {object} torrent
|
|
195
|
-
* @param {number} fileIndex
|
|
196
|
-
* @returns {Promise<object[]>}
|
|
197
|
-
*/
|
|
198
|
-
async getContainerTracks(torrent, fileIndex) {
|
|
199
|
-
const sourceKey = torrent?.sourceKey;
|
|
200
|
-
if (!sourceKey) {
|
|
201
|
-
return [];
|
|
202
|
-
}
|
|
203
|
-
const answer = await this.#client.getContainerTracks({ sourceKey, fileIndex });
|
|
204
|
-
return Array.isArray(answer?.tracks) ? answer.tracks : [];
|
|
205
|
-
}
|
|
206
|
-
|
|
207
|
-
/**
|
|
208
|
-
* Start fetching the region a viewer is about to resume at. Named in seconds
|
|
209
|
-
* here; the worker turns it into bytes, where the file's duration is readable.
|
|
210
|
-
*
|
|
211
|
-
* @param {object} torrent
|
|
212
|
-
* @param {number} fileIndex
|
|
213
|
-
* @param {number} positionSeconds
|
|
214
|
-
* @returns {Promise<boolean>}
|
|
215
|
-
*/
|
|
216
|
-
async warmResumePosition(torrent, fileIndex, positionSeconds) {
|
|
217
|
-
const sourceKey = torrent?.sourceKey;
|
|
218
|
-
if (!sourceKey) {
|
|
219
|
-
return false;
|
|
220
|
-
}
|
|
221
|
-
const answer = await this.#client.warmResumePosition({ sourceKey, fileIndex, positionSeconds });
|
|
222
|
-
return answer?.started === true;
|
|
223
|
-
}
|
|
224
|
-
|
|
225
|
-
/**
|
|
226
|
-
* What one file declares about itself: format, duration, and where its own
|
|
227
|
-
* timeline begins.
|
|
228
|
-
*
|
|
229
|
-
* @param {object} torrent
|
|
230
|
-
* @param {number} fileIndex
|
|
231
|
-
* @returns {Promise<import("../container/Container.js").ContainerMediaInfo | null>}
|
|
232
|
-
*/
|
|
233
|
-
async getContainerMediaInfo(torrent, fileIndex) {
|
|
234
|
-
const sourceKey = torrent?.sourceKey;
|
|
235
|
-
if (!sourceKey) {
|
|
236
|
-
return null;
|
|
237
|
-
}
|
|
238
|
-
const answer = await this.#client.getContainerMediaInfo({ sourceKey, fileIndex });
|
|
239
|
-
return answer?.info ?? null;
|
|
240
|
-
}
|
|
241
|
-
|
|
242
|
-
/**
|
|
243
|
-
* Where one file's keyframes are, from the container's own table. Null when
|
|
244
|
-
* this file has no readable index — which is a final answer about the file,
|
|
245
|
-
* and the reason a copy of it has to be re-encoded instead.
|
|
246
|
-
*
|
|
247
|
-
* @param {object} torrent
|
|
248
|
-
* @param {number} fileIndex
|
|
249
|
-
* @returns {Promise<{ times: number[], tolerance: number } | null>}
|
|
250
|
-
*/
|
|
251
|
-
async getContainerKeyframes(torrent, fileIndex) {
|
|
252
|
-
const sourceKey = torrent?.sourceKey;
|
|
253
|
-
if (!sourceKey) {
|
|
254
|
-
return null;
|
|
255
|
-
}
|
|
256
|
-
const answer = await this.#client.getContainerKeyframes({ sourceKey, fileIndex });
|
|
257
|
-
return answer?.index ?? null;
|
|
258
|
-
}
|
|
259
|
-
|
|
260
|
-
/**
|
|
261
|
-
* The audio tracks one file declares, in the order ffmpeg numbers them
|
|
262
|
-
* `0:a:N`.
|
|
263
|
-
*
|
|
264
|
-
* @param {object} torrent
|
|
265
|
-
* @param {number} fileIndex
|
|
266
|
-
* @returns {Promise<object[]>}
|
|
267
|
-
*/
|
|
268
|
-
async getDeclaredAudioTracks(torrent, fileIndex) {
|
|
269
|
-
const tracks = await this.getContainerTracks(torrent, fileIndex);
|
|
270
|
-
return tracks
|
|
271
|
-
.filter((track) => track?.type === "audio")
|
|
272
|
-
.sort((left, right) => (left.declaredIndex ?? 0) - (right.declaredIndex ?? 0));
|
|
273
|
-
}
|
|
274
|
-
|
|
275
|
-
/**
|
|
276
|
-
* The video track one file declares, or null where it declares none.
|
|
277
|
-
*
|
|
278
|
-
* One, because ffmpeg's `0:v:0` is what everything downstream is built on and
|
|
279
|
-
* a second video stream is a cover image far more often than a second film.
|
|
280
|
-
*
|
|
281
|
-
* @param {object} torrent
|
|
282
|
-
* @param {number} fileIndex
|
|
283
|
-
* @returns {Promise<object | null>}
|
|
284
|
-
*/
|
|
285
|
-
async getDeclaredVideoTrack(torrent, fileIndex) {
|
|
286
|
-
const tracks = await this.getContainerTracks(torrent, fileIndex);
|
|
287
|
-
return (
|
|
288
|
-
tracks
|
|
289
|
-
.filter((track) => track?.type === "video")
|
|
290
|
-
.sort((left, right) => (left.declaredIndex ?? 0) - (right.declaredIndex ?? 0))[0] ?? null
|
|
291
|
-
);
|
|
292
|
-
}
|
|
293
|
-
|
|
294
|
-
/**
|
|
295
|
-
* The cues of one subtitle track that the downloaded clusters already carry.
|
|
296
|
-
*
|
|
297
|
-
* @param {object} torrent
|
|
298
|
-
* @param {number} fileIndex
|
|
299
|
-
* @param {number} trackNumber
|
|
300
|
-
* @returns {Promise<object | null>}
|
|
301
|
-
*/
|
|
302
|
-
async getSubtitleCues(torrent, fileIndex, trackNumber) {
|
|
303
|
-
const sourceKey = torrent?.sourceKey;
|
|
304
|
-
if (!sourceKey) {
|
|
305
|
-
return null;
|
|
306
|
-
}
|
|
307
|
-
return this.#client.getSubtitleCues({ sourceKey, fileIndex, trackNumber });
|
|
308
|
-
}
|
|
309
|
-
|
|
310
|
-
/**
|
|
311
|
-
* Reorder piece selection around a read position.
|
|
312
|
-
*
|
|
313
|
-
* Synchronous by design — see the file header.
|
|
314
|
-
*
|
|
315
|
-
* @param {object} torrent
|
|
316
|
-
* @param {number} fileIndex
|
|
317
|
-
* @param {number} byteStart
|
|
318
|
-
* @param {number} [windowBytes]
|
|
319
|
-
* @param {{ wholeFileRead?: boolean }} [options]
|
|
320
|
-
* @returns {void}
|
|
321
|
-
*/
|
|
322
|
-
prioritizeByteRange(torrent, fileIndex, byteStart, windowBytes, options) {
|
|
323
|
-
const sourceKey = torrent?.sourceKey;
|
|
324
|
-
if (!sourceKey) {
|
|
325
|
-
return;
|
|
326
|
-
}
|
|
327
|
-
void this.#client
|
|
328
|
-
.prioritizeByteRange({
|
|
329
|
-
sourceKey,
|
|
330
|
-
fileIndex,
|
|
331
|
-
byteStart,
|
|
332
|
-
windowBytes,
|
|
333
|
-
wholeFileRead: options?.wholeFileRead === true
|
|
334
|
-
})
|
|
335
|
-
.catch(() => undefined);
|
|
336
|
-
}
|
|
337
|
-
|
|
338
|
-
/**
|
|
339
|
-
* Pre-fetch the head and tail the codec probe needs.
|
|
340
|
-
*
|
|
341
|
-
* Takes an options object, matching `TorrentPool.prefetchFileEdges` — this
|
|
342
|
-
* adapter exists to present that same interface. It previously declared
|
|
343
|
-
* positional parameters instead, so the planner's options object arrived as
|
|
344
|
-
* `headBytes` and only worked because it was passed along far enough to be
|
|
345
|
-
* destructured at the far end. Anyone calling it as documented got the
|
|
346
|
-
* defaults instead of the sizes they asked for.
|
|
347
|
-
*
|
|
348
|
-
* @param {object} torrent
|
|
349
|
-
* @param {number} fileIndex
|
|
350
|
-
* @param {{ headBytes?: number, tailBytes?: number, timeoutMs?: number }} [options]
|
|
351
|
-
* @returns {Promise<unknown>}
|
|
352
|
-
*/
|
|
353
|
-
async prefetchFileEdges(torrent, fileIndex, options = {}) {
|
|
354
|
-
const sourceKey = torrent?.sourceKey;
|
|
355
|
-
if (!sourceKey) {
|
|
356
|
-
return null;
|
|
357
|
-
}
|
|
358
|
-
return this.#client.prefetchFileEdges({ sourceKey, fileIndex, options });
|
|
359
|
-
}
|
|
360
|
-
|
|
361
|
-
/**
|
|
362
|
-
*
|
|
363
|
-
*
|
|
364
|
-
*
|
|
365
|
-
|
|
366
|
-
|
|
367
|
-
|
|
368
|
-
|
|
369
|
-
|
|
370
|
-
|
|
1
|
+
/**
|
|
2
|
+
* @file `TorrentPool`'s interface, served from the worker thread.
|
|
3
|
+
*
|
|
4
|
+
* The routes, the planner, the health report and the session manager all reach
|
|
5
|
+
* for a torrent pool and use it the same handful of ways. Rather than rewrite
|
|
6
|
+
* every one of them to thread a `sourceKey` through and await what used to be
|
|
7
|
+
* immediate, this presents the shape they already expect and does the thread
|
|
8
|
+
* hop behind it. Swapping the implementation is then a one-line change at
|
|
9
|
+
* construction, and the call sites are untouched — which is what keeps a change
|
|
10
|
+
* of this size reviewable.
|
|
11
|
+
*
|
|
12
|
+
* Two accommodations are needed, and both are deliberate:
|
|
13
|
+
*
|
|
14
|
+
* - **`acquireFile` and `prioritizeByteRange` stay synchronous.** They return
|
|
15
|
+
* nothing the caller inspects, so the command is dispatched and not awaited.
|
|
16
|
+
* `acquireFile` hands back a release function exactly as before, which sends
|
|
17
|
+
* its own command when called. Awaiting them would mean touching every call
|
|
18
|
+
* site for no observable gain.
|
|
19
|
+
* - **`getTorrent` needs a `sourceKey`.** Torrent objects cannot cross a
|
|
20
|
+
* thread, so the worker keys them. Callers that have one pass it; the rest
|
|
21
|
+
* get one derived from the source itself, so the identity stays stable
|
|
22
|
+
* across calls for the same torrent.
|
|
23
|
+
*/
|
|
24
|
+
|
|
25
|
+
import { TorrentWorkerClient } from "./client.js";
|
|
26
|
+
import { deriveSourceKey } from "../torrent-source-key.js";
|
|
27
|
+
|
|
28
|
+
/**
|
|
29
|
+
* A torrent pool whose work happens on another thread.
|
|
30
|
+
*
|
|
31
|
+
* See `protocol.js` for why: the torrent was taking ~85% of the main thread and
|
|
32
|
+
* everything owed to a viewer queued behind it.
|
|
33
|
+
*/
|
|
34
|
+
export class WorkerTorrentPool {
|
|
35
|
+
#client;
|
|
36
|
+
/** Stand-ins by source key, so repeat calls return the same object. */
|
|
37
|
+
#torrents = new Map();
|
|
38
|
+
|
|
39
|
+
/**
|
|
40
|
+
* @param {{ maxDiskBytes?: number, memoryBytes?: number, stateDir?: string }} [options]
|
|
41
|
+
*/
|
|
42
|
+
constructor(options = {}) {
|
|
43
|
+
this.#client = new TorrentWorkerClient(options);
|
|
44
|
+
}
|
|
45
|
+
|
|
46
|
+
/**
|
|
47
|
+
* Load (or join) a torrent and return a stand-in for it.
|
|
48
|
+
*
|
|
49
|
+
* @param {"magnet" | "torrent"} sourceType
|
|
50
|
+
* @param {string} source
|
|
51
|
+
* @returns {Promise<object>}
|
|
52
|
+
*/
|
|
53
|
+
async getTorrent(sourceType, source) {
|
|
54
|
+
const sourceKey = await deriveSourceKey(sourceType, source);
|
|
55
|
+
const existing = this.#torrents.get(sourceKey);
|
|
56
|
+
if (existing) {
|
|
57
|
+
return existing;
|
|
58
|
+
}
|
|
59
|
+
const torrent = await this.#client.getTorrent({ sourceKey, sourceType, source });
|
|
60
|
+
this.#torrents.set(sourceKey, torrent);
|
|
61
|
+
return torrent;
|
|
62
|
+
}
|
|
63
|
+
|
|
64
|
+
/**
|
|
65
|
+
* Claim a file for reading; the returned function releases it.
|
|
66
|
+
*
|
|
67
|
+
* Synchronous by design — see the file header.
|
|
68
|
+
*
|
|
69
|
+
* @param {object} torrent - A stand-in from {@link getTorrent}.
|
|
70
|
+
* @param {number} fileIndex
|
|
71
|
+
* @returns {() => void}
|
|
72
|
+
*/
|
|
73
|
+
acquireFile(torrent, fileIndex) {
|
|
74
|
+
const sourceKey = torrent?.sourceKey;
|
|
75
|
+
if (!sourceKey) {
|
|
76
|
+
return () => undefined;
|
|
77
|
+
}
|
|
78
|
+
// Dispatched, not awaited — callers use the result immediately and inspect
|
|
79
|
+
// nothing. But the release MUST NOT overtake it: both are ordinary messages
|
|
80
|
+
// to the worker, and if release arrives first the reader count drops to zero
|
|
81
|
+
// while a read is still running. The idle sweep then removes the torrent AND
|
|
82
|
+
// its downloaded data out from under the encoder — field 2026-08-02:
|
|
83
|
+
// "removed idle torrent ... and its store" mid-playback, after which every
|
|
84
|
+
// read hung and ffmpeg saw an empty input ("Stream ends prematurely at 0").
|
|
85
|
+
// Chaining the release onto the acquire keeps them in order.
|
|
86
|
+
const acquired = this.#client.acquireFile(sourceKey, fileIndex).catch(() => null);
|
|
87
|
+
let released = false;
|
|
88
|
+
return () => {
|
|
89
|
+
if (released) {
|
|
90
|
+
return;
|
|
91
|
+
}
|
|
92
|
+
released = true;
|
|
93
|
+
// Release the claim this call opened, not "the file" — waiting for the
|
|
94
|
+
// acquire is also what tells us which claim that is.
|
|
95
|
+
void acquired
|
|
96
|
+
.then((claimId) => (claimId ? this.#client.releaseFile(claimId) : undefined))
|
|
97
|
+
.catch(() => undefined);
|
|
98
|
+
};
|
|
99
|
+
}
|
|
100
|
+
|
|
101
|
+
/**
|
|
102
|
+
* Bytes every torrent here has moved.
|
|
103
|
+
*
|
|
104
|
+
* @returns {Promise<{ downloaded: number, uploaded: number }>}
|
|
105
|
+
*/
|
|
106
|
+
async getTorrentTotals() {
|
|
107
|
+
return this.#client.getTorrentTotals();
|
|
108
|
+
}
|
|
109
|
+
|
|
110
|
+
/**
|
|
111
|
+
* Live download figures for the progress display.
|
|
112
|
+
*
|
|
113
|
+
* @param {object} torrent
|
|
114
|
+
* @param {number | null} [fileIndex]
|
|
115
|
+
* @param {{ resumeAnchorByteStart?: number | null }} [options]
|
|
116
|
+
* @returns {Promise<object | null>}
|
|
117
|
+
*/
|
|
118
|
+
async getFileStats(torrent, fileIndex = null, options = {}) {
|
|
119
|
+
const sourceKey = torrent?.sourceKey;
|
|
120
|
+
if (!sourceKey) {
|
|
121
|
+
return null;
|
|
122
|
+
}
|
|
123
|
+
return this.#client.getFileStats({
|
|
124
|
+
sourceKey,
|
|
125
|
+
fileIndex,
|
|
126
|
+
resumeAnchorByteStart: options?.resumeAnchorByteStart ?? null
|
|
127
|
+
});
|
|
128
|
+
}
|
|
129
|
+
|
|
130
|
+
/**
|
|
131
|
+
* Fetch one whole file using only the room the viewer's own reading leaves.
|
|
132
|
+
*
|
|
133
|
+
* For a soundtrack or subtitle file shipped beside the picture: small next to
|
|
134
|
+
* the film, and having it on disk is what turns a later switch into a local
|
|
135
|
+
* read instead of a wait on the swarm.
|
|
136
|
+
*
|
|
137
|
+
* @param {object} torrent
|
|
138
|
+
* @param {number} fileIndex
|
|
139
|
+
* @returns {Promise<boolean>} Whether a fill was started by this call.
|
|
140
|
+
*/
|
|
141
|
+
async fillFileInBackground(torrent, fileIndex) {
|
|
142
|
+
const sourceKey = torrent?.sourceKey;
|
|
143
|
+
if (!sourceKey) {
|
|
144
|
+
return false;
|
|
145
|
+
}
|
|
146
|
+
const answer = await this.#client.fillFile({ sourceKey, fileIndex });
|
|
147
|
+
return answer?.started === true;
|
|
148
|
+
}
|
|
149
|
+
|
|
150
|
+
/**
|
|
151
|
+
* The text subtitle tracks a file carries, read from its own header.
|
|
152
|
+
*
|
|
153
|
+
* @param {object} torrent
|
|
154
|
+
* @param {number} fileIndex
|
|
155
|
+
* @returns {Promise<object[]>}
|
|
156
|
+
*/
|
|
157
|
+
async getSubtitleTracks(torrent, fileIndex) {
|
|
158
|
+
const sourceKey = torrent?.sourceKey;
|
|
159
|
+
if (!sourceKey) {
|
|
160
|
+
return [];
|
|
161
|
+
}
|
|
162
|
+
const answer = await this.#client.getSubtitleTracks({ sourceKey, fileIndex });
|
|
163
|
+
return Array.isArray(answer?.tracks) ? answer.tracks : [];
|
|
164
|
+
}
|
|
165
|
+
|
|
166
|
+
/**
|
|
167
|
+
* What the container itself declares about its subtitle tracks, in its own
|
|
168
|
+
* order and including the picture-based ones — for lining up against
|
|
169
|
+
* ffmpeg's own numbering.
|
|
170
|
+
*
|
|
171
|
+
* @param {object} torrent
|
|
172
|
+
* @param {number} fileIndex
|
|
173
|
+
* @returns {Promise<object[]>}
|
|
174
|
+
*/
|
|
175
|
+
async getDeclaredSubtitleTracks(torrent, fileIndex) {
|
|
176
|
+
const sourceKey = torrent?.sourceKey;
|
|
177
|
+
if (!sourceKey) {
|
|
178
|
+
return [];
|
|
179
|
+
}
|
|
180
|
+
const answer = await this.#client.getSubtitleTracks({ sourceKey, fileIndex });
|
|
181
|
+
return Array.isArray(answer?.declared) ? answer.declared : [];
|
|
182
|
+
}
|
|
183
|
+
|
|
184
|
+
/**
|
|
185
|
+
* Every track one file declares, read from its own header by the container
|
|
186
|
+
* layer.
|
|
187
|
+
*
|
|
188
|
+
* The audio menu is built from ffmpeg's `-i` banner, which carries neither
|
|
189
|
+
* `FlagOriginal`, `FlagCommentary`, `FlagVisualImpaired`, `FlagEnabled` nor
|
|
190
|
+
* `LanguageBCP47` — so without this a director's commentary and the film
|
|
191
|
+
* itself are indistinguishable in it. Also how a soundtrack shipped as its own
|
|
192
|
+
* file is read: a `.mka` is Matroska and the same reader serves it.
|
|
193
|
+
*
|
|
194
|
+
* @param {object} torrent
|
|
195
|
+
* @param {number} fileIndex
|
|
196
|
+
* @returns {Promise<object[]>}
|
|
197
|
+
*/
|
|
198
|
+
async getContainerTracks(torrent, fileIndex) {
|
|
199
|
+
const sourceKey = torrent?.sourceKey;
|
|
200
|
+
if (!sourceKey) {
|
|
201
|
+
return [];
|
|
202
|
+
}
|
|
203
|
+
const answer = await this.#client.getContainerTracks({ sourceKey, fileIndex });
|
|
204
|
+
return Array.isArray(answer?.tracks) ? answer.tracks : [];
|
|
205
|
+
}
|
|
206
|
+
|
|
207
|
+
/**
|
|
208
|
+
* Start fetching the region a viewer is about to resume at. Named in seconds
|
|
209
|
+
* here; the worker turns it into bytes, where the file's duration is readable.
|
|
210
|
+
*
|
|
211
|
+
* @param {object} torrent
|
|
212
|
+
* @param {number} fileIndex
|
|
213
|
+
* @param {number} positionSeconds
|
|
214
|
+
* @returns {Promise<boolean>}
|
|
215
|
+
*/
|
|
216
|
+
async warmResumePosition(torrent, fileIndex, positionSeconds) {
|
|
217
|
+
const sourceKey = torrent?.sourceKey;
|
|
218
|
+
if (!sourceKey) {
|
|
219
|
+
return false;
|
|
220
|
+
}
|
|
221
|
+
const answer = await this.#client.warmResumePosition({ sourceKey, fileIndex, positionSeconds });
|
|
222
|
+
return answer?.started === true;
|
|
223
|
+
}
|
|
224
|
+
|
|
225
|
+
/**
|
|
226
|
+
* What one file declares about itself: format, duration, and where its own
|
|
227
|
+
* timeline begins.
|
|
228
|
+
*
|
|
229
|
+
* @param {object} torrent
|
|
230
|
+
* @param {number} fileIndex
|
|
231
|
+
* @returns {Promise<import("../container/Container.js").ContainerMediaInfo | null>}
|
|
232
|
+
*/
|
|
233
|
+
async getContainerMediaInfo(torrent, fileIndex) {
|
|
234
|
+
const sourceKey = torrent?.sourceKey;
|
|
235
|
+
if (!sourceKey) {
|
|
236
|
+
return null;
|
|
237
|
+
}
|
|
238
|
+
const answer = await this.#client.getContainerMediaInfo({ sourceKey, fileIndex });
|
|
239
|
+
return answer?.info ?? null;
|
|
240
|
+
}
|
|
241
|
+
|
|
242
|
+
/**
|
|
243
|
+
* Where one file's keyframes are, from the container's own table. Null when
|
|
244
|
+
* this file has no readable index — which is a final answer about the file,
|
|
245
|
+
* and the reason a copy of it has to be re-encoded instead.
|
|
246
|
+
*
|
|
247
|
+
* @param {object} torrent
|
|
248
|
+
* @param {number} fileIndex
|
|
249
|
+
* @returns {Promise<{ times: number[], tolerance: number } | null>}
|
|
250
|
+
*/
|
|
251
|
+
async getContainerKeyframes(torrent, fileIndex) {
|
|
252
|
+
const sourceKey = torrent?.sourceKey;
|
|
253
|
+
if (!sourceKey) {
|
|
254
|
+
return null;
|
|
255
|
+
}
|
|
256
|
+
const answer = await this.#client.getContainerKeyframes({ sourceKey, fileIndex });
|
|
257
|
+
return answer?.index ?? null;
|
|
258
|
+
}
|
|
259
|
+
|
|
260
|
+
/**
|
|
261
|
+
* The audio tracks one file declares, in the order ffmpeg numbers them
|
|
262
|
+
* `0:a:N`.
|
|
263
|
+
*
|
|
264
|
+
* @param {object} torrent
|
|
265
|
+
* @param {number} fileIndex
|
|
266
|
+
* @returns {Promise<object[]>}
|
|
267
|
+
*/
|
|
268
|
+
async getDeclaredAudioTracks(torrent, fileIndex) {
|
|
269
|
+
const tracks = await this.getContainerTracks(torrent, fileIndex);
|
|
270
|
+
return tracks
|
|
271
|
+
.filter((track) => track?.type === "audio")
|
|
272
|
+
.sort((left, right) => (left.declaredIndex ?? 0) - (right.declaredIndex ?? 0));
|
|
273
|
+
}
|
|
274
|
+
|
|
275
|
+
/**
|
|
276
|
+
* The video track one file declares, or null where it declares none.
|
|
277
|
+
*
|
|
278
|
+
* One, because ffmpeg's `0:v:0` is what everything downstream is built on and
|
|
279
|
+
* a second video stream is a cover image far more often than a second film.
|
|
280
|
+
*
|
|
281
|
+
* @param {object} torrent
|
|
282
|
+
* @param {number} fileIndex
|
|
283
|
+
* @returns {Promise<object | null>}
|
|
284
|
+
*/
|
|
285
|
+
async getDeclaredVideoTrack(torrent, fileIndex) {
|
|
286
|
+
const tracks = await this.getContainerTracks(torrent, fileIndex);
|
|
287
|
+
return (
|
|
288
|
+
tracks
|
|
289
|
+
.filter((track) => track?.type === "video")
|
|
290
|
+
.sort((left, right) => (left.declaredIndex ?? 0) - (right.declaredIndex ?? 0))[0] ?? null
|
|
291
|
+
);
|
|
292
|
+
}
|
|
293
|
+
|
|
294
|
+
/**
|
|
295
|
+
* The cues of one subtitle track that the downloaded clusters already carry.
|
|
296
|
+
*
|
|
297
|
+
* @param {object} torrent
|
|
298
|
+
* @param {number} fileIndex
|
|
299
|
+
* @param {number} trackNumber
|
|
300
|
+
* @returns {Promise<object | null>}
|
|
301
|
+
*/
|
|
302
|
+
async getSubtitleCues(torrent, fileIndex, trackNumber) {
|
|
303
|
+
const sourceKey = torrent?.sourceKey;
|
|
304
|
+
if (!sourceKey) {
|
|
305
|
+
return null;
|
|
306
|
+
}
|
|
307
|
+
return this.#client.getSubtitleCues({ sourceKey, fileIndex, trackNumber });
|
|
308
|
+
}
|
|
309
|
+
|
|
310
|
+
/**
|
|
311
|
+
* Reorder piece selection around a read position.
|
|
312
|
+
*
|
|
313
|
+
* Synchronous by design — see the file header.
|
|
314
|
+
*
|
|
315
|
+
* @param {object} torrent
|
|
316
|
+
* @param {number} fileIndex
|
|
317
|
+
* @param {number} byteStart
|
|
318
|
+
* @param {number} [windowBytes]
|
|
319
|
+
* @param {{ wholeFileRead?: boolean }} [options]
|
|
320
|
+
* @returns {void}
|
|
321
|
+
*/
|
|
322
|
+
prioritizeByteRange(torrent, fileIndex, byteStart, windowBytes, options) {
|
|
323
|
+
const sourceKey = torrent?.sourceKey;
|
|
324
|
+
if (!sourceKey) {
|
|
325
|
+
return;
|
|
326
|
+
}
|
|
327
|
+
void this.#client
|
|
328
|
+
.prioritizeByteRange({
|
|
329
|
+
sourceKey,
|
|
330
|
+
fileIndex,
|
|
331
|
+
byteStart,
|
|
332
|
+
windowBytes,
|
|
333
|
+
wholeFileRead: options?.wholeFileRead === true
|
|
334
|
+
})
|
|
335
|
+
.catch(() => undefined);
|
|
336
|
+
}
|
|
337
|
+
|
|
338
|
+
/**
|
|
339
|
+
* Pre-fetch the head and tail the codec probe needs.
|
|
340
|
+
*
|
|
341
|
+
* Takes an options object, matching `TorrentPool.prefetchFileEdges` — this
|
|
342
|
+
* adapter exists to present that same interface. It previously declared
|
|
343
|
+
* positional parameters instead, so the planner's options object arrived as
|
|
344
|
+
* `headBytes` and only worked because it was passed along far enough to be
|
|
345
|
+
* destructured at the far end. Anyone calling it as documented got the
|
|
346
|
+
* defaults instead of the sizes they asked for.
|
|
347
|
+
*
|
|
348
|
+
* @param {object} torrent
|
|
349
|
+
* @param {number} fileIndex
|
|
350
|
+
* @param {{ headBytes?: number, tailBytes?: number, timeoutMs?: number }} [options]
|
|
351
|
+
* @returns {Promise<unknown>}
|
|
352
|
+
*/
|
|
353
|
+
async prefetchFileEdges(torrent, fileIndex, options = {}) {
|
|
354
|
+
const sourceKey = torrent?.sourceKey;
|
|
355
|
+
if (!sourceKey) {
|
|
356
|
+
return null;
|
|
357
|
+
}
|
|
358
|
+
return this.#client.prefetchFileEdges({ sourceKey, fileIndex, options });
|
|
359
|
+
}
|
|
360
|
+
|
|
361
|
+
/**
|
|
362
|
+
* Which films this proxy holds right now, and how much of each.
|
|
363
|
+
*
|
|
364
|
+
* What content affinity is decided from: a viewer of a film somebody here is
|
|
365
|
+
* already downloading costs this proxy the encode and nothing else, while the
|
|
366
|
+
* same viewer sent anywhere else starts the download from nothing.
|
|
367
|
+
*
|
|
368
|
+
* @returns {Promise<{ infoHash: string, progress: number, bytes: number }[]>}
|
|
369
|
+
*/
|
|
370
|
+
async heldTorrents() {
|
|
371
|
+
const answer = await this.#client.heldTorrents();
|
|
372
|
+
return Array.isArray(answer?.held) ? answer.held : [];
|
|
373
|
+
}
|
|
374
|
+
|
|
375
|
+
/**
|
|
376
|
+
* Shut the torrent client down and stop the thread.
|
|
377
|
+
*
|
|
378
|
+
* @returns {Promise<void>}
|
|
379
|
+
*/
|
|
380
|
+
async destroyAll() {
|
|
381
|
+
this.#torrents.clear();
|
|
382
|
+
await this.#client.destroyAll();
|
|
383
|
+
}
|
|
384
|
+
}
|
|
@@ -57,6 +57,8 @@ export const Command = {
|
|
|
57
57
|
FILE_STATS: "file-stats",
|
|
58
58
|
/** Bytes every torrent here has moved, for pricing the torrent's own cost. */
|
|
59
59
|
TORRENT_TOTALS: "torrent-totals",
|
|
60
|
+
/** Which films this proxy holds right now, for content affinity. */
|
|
61
|
+
HELD_TORRENTS: "held-torrents",
|
|
60
62
|
/** Reorder piece selection around a read position (seek prioritisation). */
|
|
61
63
|
PRIORITIZE: "prioritize",
|
|
62
64
|
/** Read a byte range; the body arrives as CHUNK messages. */
|
|
@@ -375,6 +375,29 @@ async function runCommand(command, params, id) {
|
|
|
375
375
|
return released;
|
|
376
376
|
}
|
|
377
377
|
|
|
378
|
+
case Command.HELD_TORRENTS: {
|
|
379
|
+
// Which films this proxy has right now, and how much of each. Answered
|
|
380
|
+
// from the live client rather than from the main thread's map of
|
|
381
|
+
// stand-ins, which is only cleared on shutdown and would name films this
|
|
382
|
+
// proxy let go of hours ago.
|
|
383
|
+
const held = [];
|
|
384
|
+
for (const torrent of pool.client?.torrents ?? []) {
|
|
385
|
+
const infoHash = String(torrent?.infoHash ?? "");
|
|
386
|
+
if (!infoHash) {
|
|
387
|
+
continue;
|
|
388
|
+
}
|
|
389
|
+
held.push({
|
|
390
|
+
infoHash,
|
|
391
|
+
// A viewer sent here for a film nobody has downloaded any of gains
|
|
392
|
+
// nothing, so the share is reported and the decision is made where
|
|
393
|
+
// the viewer is.
|
|
394
|
+
progress: Number.isFinite(torrent?.progress) ? torrent.progress : 0,
|
|
395
|
+
bytes: Number.isFinite(torrent?.downloaded) ? torrent.downloaded : 0
|
|
396
|
+
});
|
|
397
|
+
}
|
|
398
|
+
return { held };
|
|
399
|
+
}
|
|
400
|
+
|
|
378
401
|
case Command.TORRENT_TOTALS: {
|
|
379
402
|
// Downloaded and uploaded are counted apart: hashing every downloaded
|
|
380
403
|
// byte is work of a different order from sending one back to the swarm,
|
|
@@ -250,8 +250,24 @@ export function createTunnelClient({
|
|
|
250
250
|
|
|
251
251
|
// Health check: server requests current metrics for proxy scoring.
|
|
252
252
|
if (message.type === "health-request") {
|
|
253
|
-
|
|
254
|
-
|
|
253
|
+
// Awaited: the answer now carries which films this proxy holds, and the
|
|
254
|
+
// truthful list of those lives on the torrent thread.
|
|
255
|
+
void (async () => {
|
|
256
|
+
let answer = {};
|
|
257
|
+
try {
|
|
258
|
+
answer = typeof onHealthRequest === "function" ? await onHealthRequest() : {};
|
|
259
|
+
} catch {
|
|
260
|
+
// silent-ok: a proxy that cannot describe itself is scored on
|
|
261
|
+
// nothing rather than not answered at all, which would drop it out
|
|
262
|
+
// of every selection until the next poll.
|
|
263
|
+
}
|
|
264
|
+
send({
|
|
265
|
+
type: "health-response",
|
|
266
|
+
requestId: message.requestId,
|
|
267
|
+
metrics: answer?.metrics ?? answer ?? {},
|
|
268
|
+
holds: Array.isArray(answer?.holds) ? answer.holds : []
|
|
269
|
+
});
|
|
270
|
+
})();
|
|
255
271
|
return;
|
|
256
272
|
}
|
|
257
273
|
});
|