@torrent-tv/proxy 2.77.0 → 2.78.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +10 -0
- package/package.json +1 -1
- package/services/encode/DemandMap.js +187 -0
- package/services/encode/EncodePlan.js +283 -272
- package/services/encode/SegmentDemand.js +0 -0
- package/services/hls-session-manager.js +10902 -10846
- package/services/orchestrators/EncodeOrchestrator.js +13 -3
- package/test/demand-map.test.js +102 -0
- package/test/encode-plan.test.js +44 -2
- package/test/run-intervals.test.js +67 -303
package/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,13 @@
|
|
|
1
|
+
## 2.78.0
|
|
2
|
+
|
|
3
|
+
- **Fix**: One authority over the encoders, where there were three. A viewer watching an episode on 2026-09-05 had their picture stop for 125 seconds while the proxy spawned and killed an encoder every 350-700ms, producing nothing. Two decisions were contradicting each other on every pass: the plan commanded a start inside the viewer's window, at #46; `planRunInterval` in the session manager moved that start to #78, because it counted every live run as claiming up to its head plus the look-ahead; and the plan then saw a run at #78 against a window of [27, 57], found no overlap and killed it as "nothing it was given is wanted". Neither coverage nor demand changed between two such passes, so the same start was commanded again, for as long as anybody watched. `planRunInterval` is deleted: where a run starts is the plan's decision and nothing moves it, and how far it may work is read off the one coverage map the plan itself uses.
|
|
4
|
+
- **Fix**: A run is never stopped for standing outside a viewer's window. While a file is being encoded it is encoded whole; a viewer decides the ORDER the work is taken in and, through the machine's budget, how many processes take it — never whether a run may go on living. The stop this replaced was the other half of the contradiction above, and with it went the last two uses of "does this run overlap a window" in the file.
|
|
5
|
+
- **New**: What a viewer needs is a map, and the maps of several viewers merge into one (`services/encode/DemandMap.js`). Three zones per viewer, in seconds of film: what must be ready before they set off so that they never stop, what the machine reaches while they watch that, and the rest of the track. Merging takes the highest priority per second, so film two people want is as urgent as the more urgent of them and is made once. Which viewer asked never reaches the encoders.
|
|
6
|
+
- **Fix**: The size of that first zone is measured rather than chosen. Below realtime an encoder loses `1 - speed` of a second of film for every second played, so over the film in front of a viewer the shortfall is `remaining × (1 - speed)`, and that much has to exist before they start or they meet a stall partway through — at 0.25x on 400 seconds of film, 300 seconds. Above realtime only the allowance this file's own recent interruptions have shown to be necessary is needed. The 120 seconds that used to size this window were `LOOKAHEAD_PAUSE_SECONDS`, the threshold of a suspended encoder, one chosen number answering seven unrelated questions.
|
|
7
|
+
- **Fix**: The look-ahead limiter is gone, with both of its chosen numbers. It suspended a run 120 seconds in front of the viewer and woke it at 60 — deliberately pushing a run past the window the plan was asking about, which is what the plan then killed it for. What remains of it is a reading: how much film is ready in front of the earliest viewer, which commands nothing.
|
|
8
|
+
- **Fix**: A zone's priority reaches the register instead of being dropped on the way. `want()` took only the two ends, so the order the map states was computed and then discarded. It is a property of the window now, and the claimant's name stays the person's, so a viewer who leaves takes all three of their zones with them.
|
|
9
|
+
- **Fix**: The measured encode speed is read from the field that exists. `progress.speedX` appears nowhere in this codebase, so the speed was always zero and the middle zone was never built at all; the reading the budget already trusts is `recentSpeed.speed`.
|
|
10
|
+
|
|
1
11
|
## 2.77.0
|
|
2
12
|
|
|
3
13
|
- **Fix**: A viewer who has just arrived is watching. Presence and position were one field — the segment they last asked for — so a viewer who had asked for nothing counted as absent, and an output all of whose viewers count as absent has every encoder on it stopped. Field 2026-09-05: a soundtrack's encoder was stopped 1.25 s after it started, with nothing produced, so its `init.mp4` was never made; the picture could not be played without it, the browser waited sixty seconds and was told the proxy had sent no video. The viewer could not have rescued themselves — the segment request that would have marked them present needs the `init.mp4` the stopped encoder was going to make. They are now two facts: position is set the moment they arrive, from the position their own request names, and presence is a fact of their connection.
|
package/package.json
CHANGED
|
@@ -0,0 +1,187 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @file What one viewer needs, what all of them need together, and in what
|
|
3
|
+
* order the work should be taken.
|
|
4
|
+
*
|
|
5
|
+
* The shape, stated by the user 2026-09-05:
|
|
6
|
+
*
|
|
7
|
+
* > Usually you make a map for each viewer, then merge the maps, then decide
|
|
8
|
+
* > the best way of filling it given the encoders available, where they are now
|
|
9
|
+
* > and how many there are.
|
|
10
|
+
*
|
|
11
|
+
* Three questions, and this file answers the first two. The third — the filling
|
|
12
|
+
* — belongs to whoever holds the encoders, and it is handed the merged map
|
|
13
|
+
* instead of a list of windows.
|
|
14
|
+
*
|
|
15
|
+
* **ONE PRIORITISATION, TWO CONSUMERS.** Downloading and encoding keep
|
|
16
|
+
* different STATES — made / being made / free for one; downloaded / arriving,
|
|
17
|
+
* and which peers hold it at what speed, for the other — but they must agree
|
|
18
|
+
* about what matters first, or the swarm fetches what the encoder will not
|
|
19
|
+
* reach for another twenty minutes. That agreement is this map.
|
|
20
|
+
*
|
|
21
|
+
* **THE UNIT IS SECONDS OF FILM.** A map is a set of stretches with sizes and a
|
|
22
|
+
* length of its own, so it needs a unit, and seconds are the only one every
|
|
23
|
+
* term of the arithmetic is already stated in: encode speed is a ratio of
|
|
24
|
+
* seconds to seconds, the measured allowance below which an interruption
|
|
25
|
+
* reaches a viewer is seconds, the viewer's position is seconds, the film's
|
|
26
|
+
* length is seconds. Bytes cannot serve — how many a second costs is not known
|
|
27
|
+
* when a file is opened and is not constant across it, and a soundtrack in a
|
|
28
|
+
* file of its own has bytes of its own. Segment numbers cannot serve either:
|
|
29
|
+
* they exist only once a cut grid is read, and two outputs of one film number
|
|
30
|
+
* differently.
|
|
31
|
+
*
|
|
32
|
+
* **What this file must NOT know**, and the boundary is the point: nothing about
|
|
33
|
+
* containers, cut grids, pieces or bytes. Turning a stretch of seconds into the
|
|
34
|
+
* bytes of one track is the container's and the track's business, by whatever
|
|
35
|
+
* means suit that file — a Cues table, a sample table, or a walk when the file
|
|
36
|
+
* carries neither, which is the same answer they already give in order to play
|
|
37
|
+
* it at all. Getting those bytes is the downloader's business; making segments
|
|
38
|
+
* out of them is the encoder's.
|
|
39
|
+
*/
|
|
40
|
+
|
|
41
|
+
/**
|
|
42
|
+
* How urgently a stretch of film is wanted. Higher is sooner.
|
|
43
|
+
*
|
|
44
|
+
* @typedef {object} DemandZone
|
|
45
|
+
* @property {number} from - First second of film, inclusive.
|
|
46
|
+
* @property {number} to - Last second of film, inclusive.
|
|
47
|
+
* @property {number} priority - Higher is more urgent. Only the ORDER between
|
|
48
|
+
* zones is meaningful; the numbers themselves are not a scale.
|
|
49
|
+
*/
|
|
50
|
+
|
|
51
|
+
/** Where the viewer stands. Nothing outranks it. */
|
|
52
|
+
const AT_THE_VIEWER = 3;
|
|
53
|
+
|
|
54
|
+
/** In front of them, within reach while they watch what is already made. */
|
|
55
|
+
const IN_FRONT = 2;
|
|
56
|
+
|
|
57
|
+
/** The rest of the track: wanted, because the file is encoded whole. */
|
|
58
|
+
const THE_REST = 1;
|
|
59
|
+
|
|
60
|
+
/**
|
|
61
|
+
* One viewer's map.
|
|
62
|
+
*
|
|
63
|
+
* Three zones, and the two boundaries between them are measured rather than
|
|
64
|
+
* chosen:
|
|
65
|
+
*
|
|
66
|
+
* 1. **what must be ready before they set off, so that they never stop.** While
|
|
67
|
+
* they watch, film is consumed at one second per second and produced at
|
|
68
|
+
* `encodeSpeedX`. Above realtime the encoder gains on them, and all that is
|
|
69
|
+
* needed in front is the measured allowance for unevenness in the swarm and
|
|
70
|
+
* in production. Below realtime it LOSES `1 - speed` of a second for every
|
|
71
|
+
* second played, so over the film in front of them the shortfall is
|
|
72
|
+
* `remaining × (1 - speed)` — at 0.5x on twenty minutes ahead, ten minutes
|
|
73
|
+
* must exist before they start, or they meet a stall partway through;
|
|
74
|
+
* 2. **what the machine reaches while they watch zone 1** — in front of a
|
|
75
|
+
* moving viewer, so ahead of the rest, and nobody is waiting for it yet, so
|
|
76
|
+
* behind zone 1;
|
|
77
|
+
* 3. **the rest of the track.**
|
|
78
|
+
*
|
|
79
|
+
* @param {object} params
|
|
80
|
+
* @param {number} params.atSeconds - Where they are watching from.
|
|
81
|
+
* @param {number} params.durationSeconds - How long the film is.
|
|
82
|
+
* @param {number} params.allowanceSeconds - The measured depth below which an
|
|
83
|
+
* interruption reaches this viewer (`minimumBufferSeconds`).
|
|
84
|
+
* @param {number} params.encodeSpeedX - Measured encode speed against realtime
|
|
85
|
+
* for this track on this machine. Zero or less means nothing has measured it
|
|
86
|
+
* yet, and then zone 2 is left out rather than invented.
|
|
87
|
+
* @returns {DemandZone[]} Ascending, without gaps or overlaps, covering
|
|
88
|
+
* everything from where they are to the end of the film.
|
|
89
|
+
*/
|
|
90
|
+
export function mapForViewer({ atSeconds, durationSeconds, allowanceSeconds, encodeSpeedX }) {
|
|
91
|
+
const from = Number.isFinite(atSeconds) && atSeconds > 0 ? atSeconds : 0;
|
|
92
|
+
const end = Number.isFinite(durationSeconds) ? durationSeconds : 0;
|
|
93
|
+
if (!(end > from)) {
|
|
94
|
+
return [];
|
|
95
|
+
}
|
|
96
|
+
const remaining = end - from;
|
|
97
|
+
const allowance = Number.isFinite(allowanceSeconds) && allowanceSeconds > 0 ? allowanceSeconds : 0;
|
|
98
|
+
const speed = Number.isFinite(encodeSpeedX) && encodeSpeedX > 0 ? encodeSpeedX : 0;
|
|
99
|
+
const shortfall = speed > 0 && speed < 1 ? remaining * (1 - speed) : 0;
|
|
100
|
+
|
|
101
|
+
/** @type {DemandZone[]} */
|
|
102
|
+
const zones = [];
|
|
103
|
+
const readyBy = Math.min(end, from + allowance + shortfall);
|
|
104
|
+
zones.push({ from, to: readyBy, priority: AT_THE_VIEWER });
|
|
105
|
+
|
|
106
|
+
if (readyBy < end && speed > 0) {
|
|
107
|
+
// While they watch what zone 1 holds, the encoder makes `speed` times that
|
|
108
|
+
// much. Beyond it nobody is waiting yet.
|
|
109
|
+
const reach = Math.min(end, readyBy + (readyBy - from) * speed);
|
|
110
|
+
if (reach > readyBy) {
|
|
111
|
+
zones.push({ from: readyBy, to: reach, priority: IN_FRONT });
|
|
112
|
+
}
|
|
113
|
+
}
|
|
114
|
+
|
|
115
|
+
const covered = zones[zones.length - 1].to;
|
|
116
|
+
if (covered < end) {
|
|
117
|
+
zones.push({ from: covered, to: end, priority: THE_REST });
|
|
118
|
+
}
|
|
119
|
+
return zones;
|
|
120
|
+
}
|
|
121
|
+
|
|
122
|
+
/**
|
|
123
|
+
* Every viewer's map as one.
|
|
124
|
+
*
|
|
125
|
+
* The highest priority per second wins: film two people want is as urgent as
|
|
126
|
+
* the more urgent of them, and making it once serves both. What comes back has
|
|
127
|
+
* no overlaps, so the filling can walk it without asking about any individual
|
|
128
|
+
* viewer — which is the rule this layer exists to keep, that which viewer asked
|
|
129
|
+
* never reaches the encoders.
|
|
130
|
+
*
|
|
131
|
+
* @param {DemandZone[][]} maps
|
|
132
|
+
* @returns {DemandZone[]} Ascending by position.
|
|
133
|
+
*/
|
|
134
|
+
export function mergeMaps(maps) {
|
|
135
|
+
/** @type {DemandZone[]} */
|
|
136
|
+
const all = [];
|
|
137
|
+
for (const map of maps ?? []) {
|
|
138
|
+
for (const zone of map ?? []) {
|
|
139
|
+
if (Number.isFinite(zone?.from) && Number.isFinite(zone?.to) && zone.to > zone.from) {
|
|
140
|
+
all.push(zone);
|
|
141
|
+
}
|
|
142
|
+
}
|
|
143
|
+
}
|
|
144
|
+
if (all.length === 0) {
|
|
145
|
+
return [];
|
|
146
|
+
}
|
|
147
|
+
// Walked by BOUNDARIES rather than by second: a film is thousands of them and
|
|
148
|
+
// this is asked again on every change.
|
|
149
|
+
const points = [...new Set(all.flatMap((zone) => [zone.from, zone.to]))].sort(
|
|
150
|
+
(left, right) => left - right
|
|
151
|
+
);
|
|
152
|
+
/** @type {DemandZone[]} */
|
|
153
|
+
const merged = [];
|
|
154
|
+
for (let index = 0; index < points.length - 1; index += 1) {
|
|
155
|
+
const from = points[index];
|
|
156
|
+
const to = points[index + 1];
|
|
157
|
+
let priority = 0;
|
|
158
|
+
for (const zone of all) {
|
|
159
|
+
if (zone.from <= from && to <= zone.to && zone.priority > priority) {
|
|
160
|
+
priority = zone.priority;
|
|
161
|
+
}
|
|
162
|
+
}
|
|
163
|
+
if (priority <= 0) {
|
|
164
|
+
continue;
|
|
165
|
+
}
|
|
166
|
+
const previous = merged[merged.length - 1];
|
|
167
|
+
if (previous && previous.priority === priority && previous.to === from) {
|
|
168
|
+
previous.to = to;
|
|
169
|
+
continue;
|
|
170
|
+
}
|
|
171
|
+
merged.push({ from, to, priority });
|
|
172
|
+
}
|
|
173
|
+
return merged;
|
|
174
|
+
}
|
|
175
|
+
|
|
176
|
+
/**
|
|
177
|
+
* The merged map in the order the work is taken: most urgent first, and within
|
|
178
|
+
* one priority the earliest film first — that is where somebody is stopped.
|
|
179
|
+
*
|
|
180
|
+
* @param {DemandZone[]} merged
|
|
181
|
+
* @returns {DemandZone[]}
|
|
182
|
+
*/
|
|
183
|
+
export function inWorkingOrder(merged) {
|
|
184
|
+
return [...(merged ?? [])].sort(
|
|
185
|
+
(left, right) => right.priority - left.priority || left.from - right.from
|
|
186
|
+
);
|
|
187
|
+
}
|