@torrent-tv/proxy 2.83.4 → 2.83.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +13 -0
- package/package.json +1 -1
- package/research/piece-withdrawn-but-still-claimed-2026-09-12.md +175 -0
- package/research/priority-map-is-the-truth-2026-09-12.md +35 -17
- package/services/download/withdraw-claim.js +80 -0
- package/services/hls-session-manager.js +14 -42
- package/services/orchestrators/EncodeOrchestrator.js +133 -0
- package/services/piece-store/piece-disk-store.js +24 -1
- package/services/piece-store/shared-piece-store.js +71 -1
- package/services/torrent-pool.js +76 -2
- package/services/torrent-worker/piece-reader.js +30 -1
- package/services/torrent-worker/worker.js +8 -1
- package/test/input-lost-quiets-the-plan.test.js +261 -0
- package/test/logger-repeats.test.js +120 -0
- package/test/read-survives-withdrawal.test.js +164 -0
- package/test/withdraw-piece-claim.test.js +212 -0
- package/utils/logger.js +105 -7
package/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,16 @@
|
|
|
1
|
+
## 2.83.5
|
|
2
|
+
- **Fix**: ONE OWNER OF THE FACT "THIS PROXY HAS PIECE N", AND IT IS THE STORE. The disk tier drops a piece once every reader is past it — correctly, and that is what bounds the spill — while the library kept a second copy of the same fact in its completion bitfield and nothing reconciled them. A read then concluded the piece was had, asked for it, was told it was absent, and failed; nor was it ever fetched again, because the library does not download what it believes it owns. Field 2026-09-12: a film played 80 seconds (1920 frames, and the picture never moved again); the encoder ran on to 725 s, so some 565 spilled pieces fell behind every read head and were dropped, piece 0 among them; the encoder lost its input and restarted, which re-opens the input at byte 0; and `/stream` answered `0 of 2363497962 bytes: Piece 0 is verified but absent from the store` to every read for the next 92 minutes while the browser retried one segment against a healthy transport at 4 ms.
|
|
3
|
+
- **Fix**: So the store announces when it can no longer produce a piece AT ALL — not resident, not on disk, not inside a file held whole — and the pool withdraws the library's claim. The eviction's own bargain, written in its comment as "a seek back re-downloads it", is true for the first time. The three cases are one rule rather than a list of call sites: the disk tier says only that IT has lost a piece, and the store decides whether that is a loss at all, so a piece dropped as a duplicate of an assembled file says nothing and a piece still resident says nothing.
|
|
4
|
+
- **Chore**: The withdrawal is a plain function over plain values (`services/download/withdraw-claim.js`) and holds no pool, no store and no client, so it can be exercised with an object literal. It deliberately does not re-select the piece: every torrent is added with `deselect: true`, which makes the library skip that, and the download set keeps its one owner.
|
|
5
|
+
- **Chore**: `withdrawn=` joins the piece store's own line, so the bargain is checkable by number instead of by belief.
|
|
6
|
+
- **Fix**: A READ NOW SURVIVES A PIECE WITHDRAWN UNDER IT. An empty answer is a wait, not a failure: the claim has just gone, so the piece is being fetched again, and the read goes back one step and takes the whole ordinary path for it — declare it, steer it onto the fastest holders, wait. Once per piece; a second emptiness says the bytes are not coming and is reported as that rather than as "verified but absent", which was never true.
|
|
7
|
+
- **Fix**: THE DELAY AFTER A LOST INPUT NOW BINDS. It existed — doubling from 2 s to 15 s — and it was timed against the DEAD RUN, which the plan never consults: a run whose input has gone is not alive, so the plan read the stretch it held as free and placed another there at once, which died the same way. Field: 2432 ffmpeg starts in 23 minutes, one every 0.57 s, for 61 minutes, against a delay that had reached its ceiling long before. The wait lives beside the decision it governs now, per output, and the plan places nothing there until it is over — unless something on that output is still producing, which is proof the input can be read.
|
|
8
|
+
- **Fix**: The wait is lifted only by proof that the input was there, which is a segment having come out of it. Any other ending nearly did it, and that is wrong: at the moment of failure several runs end at once, and one ending `gone` with nothing produced would have lifted the wait the one beside it had just set.
|
|
9
|
+
- **Chore**: The two figures and the timer leave `hls-session-manager.js` for the encoding layer; the file is 28 lines shorter.
|
|
10
|
+
- **Fix**: AN ESTABLISHED FACT IS SAID ONCE, THEN WITH DECREASING FREQUENCY. One absent piece produced about 55 log lines a second: 235 000 lines in 92 minutes, of which 68.8 % were byte-identical repeats — measured, not estimated. Repeats are matched verbatim, the whole line, which catches nearly all of that and cannot merge two different statements; normalising numbers would catch a little more and would also merge the memory series, which exists precisely to catch a runaway. A held-back line is said again at doubling intervals up to a minute, and says how many were held back and over what span, because the rate is the fact.
|
|
11
|
+
- **Fix**: The log file may now grow to 1 GB before it turns over, from 32 MB. At the old size that one failure turned the file over twice inside the session it belonged to — 159 000 lines covering 38 minutes, then 76 385 covering 23 — so 61 minutes of 92 survived and the second rotation overwrote the turn holding the onset. The disk it was bounded for had 91.4 GB free.
|
|
12
|
+
- **Chore**: KNOWN CONSEQUENCE, stated rather than discovered later: a torrent whose pieces have been dropped is no longer `done`, so a film watched to the end is not assembled into a whole file. That is the honest state — we do not have those bytes — and the previous behaviour was worse than it looked: the assembly reads the file through the store, so it would have waited for pieces the library wrongly believed it had.
|
|
13
|
+
|
|
1
14
|
## 2.83.4
|
|
2
15
|
- **Chore**: WHAT IS IN A TORRENT IS NOW A THING RATHER THAN A RECOMPUTATION. `services/torrent/Contents.js` holds one torrent's list of files, its pictures with the sound, subtitles and images that belong to each, the order a person reads them in, and what belongs to nothing. The pairing rules were already there as functions; what did not exist was anywhere to keep their answer, so the warm-up on one path and the playback plan on another worked out the same grouping over the same list several times per opened film. Checked against the survey collection: 134 torrents, 44 of them with more than one picture, 1357 pictures and 421 files paired to one, not a single file paired to two — and all 1357 groupings agree with the per-picture call they replace, so nothing about the pairing changes.
|
|
3
16
|
- **Chore**: It holds only what the torrent says about itself. Nothing about a FILM: a poster or a title is an answer from a third party about an identity read from a file's own bytes, keyed by the picture's file, and it must never delay playback. No reference to another layer either — not a container, not a priority map, not a viewer; what leaves is plain indices into the torrent's own list.
|
package/package.json
CHANGED
|
@@ -0,0 +1,175 @@
|
|
|
1
|
+
# A film played 80 seconds and then stood still for 92 minutes — 2026-09-12
|
|
2
|
+
|
|
3
|
+
Proxy 2.83.4, addon 0.74.1, one viewer, `Reacher.S04E07.1080p.rus.LostFilm.TV.mkv`
|
|
4
|
+
(2 363 497 962 bytes). Reported by the user as "the film only managed to play
|
|
5
|
+
one minute nineteen seconds".
|
|
6
|
+
|
|
7
|
+
Every claim below is tagged by where it comes from: **[log]** read out of a
|
|
8
|
+
captured log, **[code]** read out of the source, **[measured]** computed from
|
|
9
|
+
the captured logs, **[derived]** reasoned from two of the above.
|
|
10
|
+
|
|
11
|
+
## 1. What the viewer saw
|
|
12
|
+
|
|
13
|
+
**[log, browser]** Every one of the 1112 `decode` readings in the captured
|
|
14
|
+
window is identical: `t=80.0s size=1920x1080 frames=1920 dropped=0
|
|
15
|
+
readyState=2`. 1920 frames at 24 fps is 80.0 s, so the picture advanced 80
|
|
16
|
+
seconds and then never moved again. The page stayed open until 18:52.
|
|
17
|
+
|
|
18
|
+
**[log, browser]** From then on the player asked for one segment and was
|
|
19
|
+
refused, about once a minute for 92 minutes:
|
|
20
|
+
|
|
21
|
+
```
|
|
22
|
+
18:30:47 non-fatal: fragLoadError currentTime=79.97 msReadyState=open
|
|
23
|
+
frag=segment-00008.mp4 sn=8 action=5 error=Error: HTTP Error 503 …
|
|
24
|
+
18:41:15 fatal: fragLoadError … sn=8 … HTTP Error 503
|
|
25
|
+
18:41:17 recovered fatal networkError, resuming at 80.0s
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
`msReadyState=open` throughout, so this is **not** the closed-MediaSource
|
|
29
|
+
failure of item 86. **[log]** The transport was healthy the whole time —
|
|
30
|
+
`rtt=4ms`, `state=connected ice=connected`, every counter advancing — so it is
|
|
31
|
+
not item 10 either.
|
|
32
|
+
|
|
33
|
+
## 2. What the proxy answered, and why
|
|
34
|
+
|
|
35
|
+
**[code]** 503 in `routes/transcode/session-file/get.js:143` is the
|
|
36
|
+
`warming-up` branch: the segment is still being produced. It was never produced.
|
|
37
|
+
|
|
38
|
+
**[log]** 239 times:
|
|
39
|
+
|
|
40
|
+
```
|
|
41
|
+
transcode 17ec5953 holding segment-00008.mp4: the file is not on disk
|
|
42
|
+
(runs from #8, viewer at #8, encoder alive, index #8, produced 0.0s at 2.23x
|
|
43
|
+
— the encoder has not made it)
|
|
44
|
+
[hold] segment-00008.mp4 warming-up after 60400ms (the map wants it now, rank 69 of 69)
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
**[log]** The encoder could not open its input, 5712 times:
|
|
48
|
+
|
|
49
|
+
```
|
|
50
|
+
encode-run #8..#247 lost its input ([http @ …] Error reading HTTP response: End of file)
|
|
51
|
+
Error opening input file http://127.0.0.1:9090/stream?sourceKey=94dda59f… (8514×)
|
|
52
|
+
Error opening input files: End of file (5712×)
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
**[log]** And `/stream` said exactly what was wrong, 2884 times:
|
|
56
|
+
|
|
57
|
+
```
|
|
58
|
+
stream: read of "Reacher.S04E07.1080p.rus.LostFilm.TV.mkv" bytes 0-2363497961
|
|
59
|
+
ended after 0 of 2363497962 bytes: Piece 0 is verified but absent from the store.
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
Zero bytes of 2.36 GB. ffmpeg reads an empty body as the end of the file.
|
|
63
|
+
|
|
64
|
+
## 3. The cause
|
|
65
|
+
|
|
66
|
+
**[log]** The torrent believed it had the whole file:
|
|
67
|
+
|
|
68
|
+
```
|
|
69
|
+
[stats] 94dda59f 5d276be5 peers=54 connected … file=100.0% header=8388608/8388608B
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
**[log]** The store did not:
|
|
73
|
+
|
|
74
|
+
```
|
|
75
|
+
piece-store "Reacher.S04E07…": resident=27/42 (108MB of 168MB allowed)
|
|
76
|
+
committed=108MB blocks=27 on-disk=26MB pinned=0 spilled=7
|
|
77
|
+
reads=5932 (73.8% from memory) spills=571 revivals=41
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
**[measured]** 571 spills at a 4 MB piece is about 2284 MB written to disk;
|
|
81
|
+
26 MB remained. So roughly 565 pieces had been removed. **[log]** It was not
|
|
82
|
+
the disk cap: `disk: 93634MB free; spilled pieces 26MB of 46656MB` — the
|
|
83
|
+
ceiling never came near binding, and 93 GB were free.
|
|
84
|
+
|
|
85
|
+
**[code]** That leaves exactly one remover: `PieceDiskStore.forgetBehind`,
|
|
86
|
+
called from `SharedPieceStore.reviseSpillCeiling`, which drops every piece
|
|
87
|
+
lying behind the earliest read head. **[log]** The encoder had reached
|
|
88
|
+
`proxyProcessed=725.643`, twelve minutes of film, so piece 0 — the start of the
|
|
89
|
+
file and its Matroska header — was far behind every reader and was dropped.
|
|
90
|
+
That is the rule working as designed.
|
|
91
|
+
|
|
92
|
+
**[code]** `piece-reader.js:151` then asks `torrent.bitfield.get(index)`. The
|
|
93
|
+
library still answered yes, because it had downloaded and hashed that piece
|
|
94
|
+
once. **Nothing ever told it otherwise:** `bitfield.set` does not occur
|
|
95
|
+
anywhere in `services/piece-store/`, `services/torrent-pool.js` or
|
|
96
|
+
`services/torrent-worker/`. So the read concluded the piece was had, asked the
|
|
97
|
+
store, was told it was absent, and threw.
|
|
98
|
+
|
|
99
|
+
**[derived]** And it was never fetched again, because the library does not
|
|
100
|
+
download a piece it believes it owns. The comment on `forgetBehind` states the
|
|
101
|
+
bargain in as many words — "a seek back re-downloads it, which is the same
|
|
102
|
+
bargain this tier makes whenever it drops a piece for room" — and that bargain
|
|
103
|
+
had never once been honoured.
|
|
104
|
+
|
|
105
|
+
**[derived]** The trigger is ordinary: an encoder restart re-opens its input at
|
|
106
|
+
byte 0 (`bytes 0-2363497961` in the log), which is precisely the region
|
|
107
|
+
`forgetBehind` removes first.
|
|
108
|
+
|
|
109
|
+
This is the "un-have problem" the parent `CLAUDE.md` names in the Level 2 disk
|
|
110
|
+
design — "mark the piece incomplete in the completion store → it re-downloads on
|
|
111
|
+
the next read". The eviction shipped; the un-have did not.
|
|
112
|
+
|
|
113
|
+
## 4. Two amplifiers, each a defect of its own
|
|
114
|
+
|
|
115
|
+
### The delay after a lost input did not bind
|
|
116
|
+
|
|
117
|
+
**[code]** `hls-session-manager.js` timed a retry against the DEAD RUN, doubling
|
|
118
|
+
2 s → 15 s. **[code]** `EncodeRun.isAlive` is false in `RETRY_WAIT`, and the
|
|
119
|
+
orchestrator plans from live runs only — so the plan saw the stretch as free and
|
|
120
|
+
placed a new run at once, on every event.
|
|
121
|
+
|
|
122
|
+
**[measured]** 2432 `restart for` lines in 23 minutes: one every 0.57 s, for 61
|
|
123
|
+
minutes, while the delay printed `retrying in 15s`. The comment beside that
|
|
124
|
+
timer predicted this exactly and the code did not prevent it.
|
|
125
|
+
|
|
126
|
+
### The log destroyed the record of its own failure
|
|
127
|
+
|
|
128
|
+
**[measured]** 76 385 lines in 23 minutes and 159 000 in the previous 38 — about
|
|
129
|
+
55 lines a second, of which **52 567 of 76 385 (68.8 %) are byte-identical
|
|
130
|
+
repeats**. **[code]** `MAX_FILE_BYTES` was 32 MiB with one previous turn kept, so
|
|
131
|
+
the file turned over twice inside the session: 61 minutes of 92 survived, and the
|
|
132
|
+
onset — everything before 17:51 — was overwritten by the second rotation and is
|
|
133
|
+
gone. **[log]** `/data` had 91.4 GB free.
|
|
134
|
+
|
|
135
|
+
## 5. What was built (proxy 2.83.5)
|
|
136
|
+
|
|
137
|
+
1. the store announces when it can no longer produce a piece at all — not
|
|
138
|
+
resident, not on disk, not in a file held whole — and the pool withdraws the
|
|
139
|
+
library's claim with `_markUnverified`. The disk tier states only its own
|
|
140
|
+
loss; the store decides whether that is a loss;
|
|
141
|
+
2. a read whose piece is withdrawn under it goes back one step and takes the
|
|
142
|
+
whole ordinary path again instead of throwing. Once per piece;
|
|
143
|
+
3. the delay lives in the orchestrator, per output, and the plan places nothing
|
|
144
|
+
there until it is over — unless something on that output is still producing.
|
|
145
|
+
It is lifted only by a run that actually produced;
|
|
146
|
+
4. an established fact is logged once, then at doubling intervals to a minute,
|
|
147
|
+
saying how many were held back. Matched verbatim; digits are deliberately not
|
|
148
|
+
normalised, because that would merge the memory series;
|
|
149
|
+
5. the log file may reach 1 GB before turning over.
|
|
150
|
+
|
|
151
|
+
## 6. What this does NOT establish
|
|
152
|
+
|
|
153
|
+
1. **Nothing here has been seen in the field.** The next session is the proof.
|
|
154
|
+
What should appear: `withdrawn=` climbing on the piece-store line while reads
|
|
155
|
+
keep succeeding, `its input was not there (attempt N) — placing nothing for
|
|
156
|
+
Xs`, and `[said N more time(s) …]` instead of a flood. What should disappear:
|
|
157
|
+
`Piece N is verified but absent from the store`.
|
|
158
|
+
2. **The onset was not read.** Both log turns begin with the failure already
|
|
159
|
+
established, so which piece failed FIRST, and whether the encoder's first
|
|
160
|
+
input loss had the same cause, is not known. The mechanism is established;
|
|
161
|
+
its first instance is not.
|
|
162
|
+
3. **A consequence, stated rather than discovered later:** a torrent whose
|
|
163
|
+
pieces have been withdrawn is no longer `done`, so a film watched to the end
|
|
164
|
+
is not assembled into a whole file. That is the honest state — those bytes
|
|
165
|
+
are not here — and the previous behaviour was worse than it looked, since the
|
|
166
|
+
assembly reads the file through the store and would have waited for pieces
|
|
167
|
+
the library wrongly believed it had. What is NOT measured is how often a film
|
|
168
|
+
now fails to be assembled, and whether keeping the pieces of a nearly-finished
|
|
169
|
+
file would be worth the disk.
|
|
170
|
+
4. **The re-download cost is not measured.** A seek back past the read heads now
|
|
171
|
+
waits for pieces to be fetched again instead of failing. How long that takes
|
|
172
|
+
on a real swarm, and whether the viewer notices, needs a session.
|
|
173
|
+
5. **`peers=54 connected of -4599 known (…, -1128 queued)`** — the negative
|
|
174
|
+
figures come from WebTorrent's own `_peersLength`/`_numQueued`, read for the
|
|
175
|
+
log only. Not investigated, not touched.
|
|
@@ -188,20 +188,38 @@ both fixed with a check apiece.
|
|
|
188
188
|
the moment of the departure and then never re-examined: in its swarm, with no
|
|
189
189
|
idle clock running. The pass acts on what it just said now.
|
|
190
190
|
|
|
191
|
-
##
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
|
|
205
|
-
|
|
206
|
-
|
|
207
|
-
|
|
191
|
+
## Where the shared statement lives — DECIDED, then REVERSED the same day
|
|
192
|
+
|
|
193
|
+
It was first published as a package, `@torrent-tv/torrent-contents`, so that
|
|
194
|
+
both runtimes could read the same three files: the proxy states what to fetch
|
|
195
|
+
from them, and the browser showed the list of episodes the instant a `.torrent`
|
|
196
|
+
was dropped, before a proxy had been chosen.
|
|
197
|
+
|
|
198
|
+
**Withdrawn within the hour, on the user's question: is the package better than
|
|
199
|
+
moving it all to the proxy, when the torrent has to go there after the choice
|
|
200
|
+
anyway.** It is not. The package bought one property — the instant list — with
|
|
201
|
+
three permanent costs: a third artifact to version and publish, two copies at
|
|
202
|
+
runtime that drift by version (a test was written for that drift, which is an
|
|
203
|
+
admission the risk is real), and three wiring points that break the page at load
|
|
204
|
+
with no error anybody sees (a second test was written for those).
|
|
205
|
+
|
|
206
|
+
The answer is the route that already lists a source's files. For a magnet
|
|
207
|
+
nothing changes at all — that is the path it has always taken — and the dropped
|
|
208
|
+
`.torrent` now takes the same one. What leaves the browser with it is not only
|
|
209
|
+
the classification but the whole notion of deciding anything from a name; what
|
|
210
|
+
stays is what only the browser can see (the trackers and the web seeds in the
|
|
211
|
+
file it holds) and how a name is SHOWN.
|
|
212
|
+
|
|
213
|
+
**The cost is answered by connecting earlier, which is the user's own point.**
|
|
214
|
+
Everything a viewer does needs a proxy, and choosing one and connecting to it
|
|
215
|
+
used to begin only after a file had been picked. It begins when the page loads
|
|
216
|
+
now, silently and with its failure swallowed, and `#acquireTransport` is
|
|
217
|
+
joinable — so whatever the viewer does next joins that attempt instead of
|
|
218
|
+
starting a second. Verified in a browser 2026-09-12: the page opens, the connect
|
|
219
|
+
runs on its own (`public-only connect failed …; trying local path` on a dev
|
|
220
|
+
server with no proxies registered), nothing is shown for it, and the page's own
|
|
221
|
+
adapter turns the proxy's answer into the three lists with the order preserved,
|
|
222
|
+
the torrent's own numbers kept and the release's repeated part taken off the
|
|
223
|
+
names.
|
|
224
|
+
|
|
225
|
+
`@torrent-tv/torrent-contents@1.0.0` is deprecated on npm with the reason.
|
|
@@ -0,0 +1,80 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @file Withdrawing the claim that this proxy has a piece.
|
|
3
|
+
*
|
|
4
|
+
* **The fact has one owner and it is the store.** The store holds the bytes, so
|
|
5
|
+
* it says what this proxy has. The library keeps a second copy of that fact in
|
|
6
|
+
* its completion bitfield, and until 2026-09-12 nothing reconciled the two: the
|
|
7
|
+
* disk tier drops a piece once every reader is past it — correctly, and that is
|
|
8
|
+
* what bounds the spill — while the bitfield went on saying the piece was
|
|
9
|
+
* verified. A read then concluded the piece was had, asked for it, was told it
|
|
10
|
+
* was absent, and failed. Nor was it ever fetched again, because the library
|
|
11
|
+
* does not download what it believes it already owns.
|
|
12
|
+
*
|
|
13
|
+
* Field 2026-09-12: a film played 80 seconds; the encoder ran on to 725 s, so
|
|
14
|
+
* some 565 spilled pieces fell behind every read head and were dropped,
|
|
15
|
+
* including piece 0; the encoder then lost its input and restarted, which
|
|
16
|
+
* re-opens the input at byte 0; and `/stream` answered `0 of 2363497962 bytes:
|
|
17
|
+
* Piece 0 is verified but absent from the store` to every read for the next 92
|
|
18
|
+
* minutes while the browser retried one segment and the picture stood still.
|
|
19
|
+
*
|
|
20
|
+
* So the eviction's own bargain — "a seek back re-downloads it" — is made true
|
|
21
|
+
* here, and the bitfield becomes a projection of what the store holds.
|
|
22
|
+
*
|
|
23
|
+
* **Why it is a plain function.** It takes the values it needs and holds
|
|
24
|
+
* nothing: no pool, no store, no client. That is what lets it be exercised with
|
|
25
|
+
* an object literal, and it is the rule the layers are checked against — a layer
|
|
26
|
+
* must be usable with plain values alone, with no process, no disk and no clock.
|
|
27
|
+
*/
|
|
28
|
+
|
|
29
|
+
/**
|
|
30
|
+
* Tell the library it no longer has a piece, so the next read of it waits for a
|
|
31
|
+
* download instead of failing.
|
|
32
|
+
*
|
|
33
|
+
* **What it deliberately does not do.** The library's `_markUnverified` would
|
|
34
|
+
* also re-select the piece; it does not here, because every torrent is added
|
|
35
|
+
* with `deselect: true`, which sets the library's own `_startAsDeselected` and
|
|
36
|
+
* makes it skip that call. The download set has exactly one owner —
|
|
37
|
+
* `SwarmSelection`, from the priority map — and a withdrawn piece is fetched
|
|
38
|
+
* again when a read states it, which is the same statement every other piece
|
|
39
|
+
* waits on.
|
|
40
|
+
*
|
|
41
|
+
* @param {object} what
|
|
42
|
+
* @param {number} what.index - The piece the store can no longer produce.
|
|
43
|
+
* @param {object[]} [what.files] - The store's own files; the torrent that owns
|
|
44
|
+
* them is found through one of them, because the store has no idea what a
|
|
45
|
+
* torrent is.
|
|
46
|
+
* @param {object} [what.torrent] - Given directly instead of through `files`.
|
|
47
|
+
* @param {(line: string) => void} [what.warn] - Said when the library refuses.
|
|
48
|
+
* @returns {"withdrawn" | "nothing-to-withdraw" | "no-torrent" | "refused"}
|
|
49
|
+
*/
|
|
50
|
+
export function withdrawClaim({ index, files, torrent, warn = () => undefined }) {
|
|
51
|
+
if (!Number.isInteger(index) || index < 0) {
|
|
52
|
+
return "nothing-to-withdraw";
|
|
53
|
+
}
|
|
54
|
+
const owner = torrent ?? files?.[0]?._torrent ?? null;
|
|
55
|
+
// A torrent being destroyed is the ordinary case at the end of a session, and
|
|
56
|
+
// it has no claim left to withdraw: the store is going with it.
|
|
57
|
+
if (!owner || owner.destroyed || typeof owner._markUnverified !== "function") {
|
|
58
|
+
return "no-torrent";
|
|
59
|
+
}
|
|
60
|
+
// NOTHING TO WITHDRAW, which is most of the time: the store also drops pieces
|
|
61
|
+
// it never completed, and marking one unverified that the library already
|
|
62
|
+
// knows is missing would re-create the piece and discard whatever blocks are
|
|
63
|
+
// in flight for it.
|
|
64
|
+
if (!owner.bitfield?.get?.(index)) {
|
|
65
|
+
return "nothing-to-withdraw";
|
|
66
|
+
}
|
|
67
|
+
try {
|
|
68
|
+
owner._markUnverified(index);
|
|
69
|
+
} catch (error) {
|
|
70
|
+
// This reaches into the library's own bookkeeping. If a later version
|
|
71
|
+
// changes it, the honest result is a line saying so rather than an eviction
|
|
72
|
+
// that fails.
|
|
73
|
+
warn(
|
|
74
|
+
`could not withdraw the claim on piece ${index} of ${owner.name ?? "?"}: ` +
|
|
75
|
+
`${error?.message ?? error}`
|
|
76
|
+
);
|
|
77
|
+
return "refused";
|
|
78
|
+
}
|
|
79
|
+
return "withdrawn";
|
|
80
|
+
}
|
|
@@ -441,13 +441,6 @@ const START_FAST_FAIL_MS = 2_000;
|
|
|
441
441
|
// for whatever residual case still fails — not a second competing "fix" that
|
|
442
442
|
// blindly retries the identical command hoping for a different result.
|
|
443
443
|
const MAX_FAILED_STARTS = 3;
|
|
444
|
-
// A run that lost its INPUT is retried rather than condemned: the torrent can
|
|
445
|
-
// be added again and the pieces downloaded again, so the data being gone is a
|
|
446
|
-
// wait, not a verdict. Backed off so a source that is truly unavailable costs a
|
|
447
|
-
// process every few seconds rather than continuously, and never given up on —
|
|
448
|
-
// the session's own idle TTL is what ends it if the viewer leaves.
|
|
449
|
-
const INPUT_RETRY_BASE_MS = 2_000;
|
|
450
|
-
const INPUT_RETRY_MAX_MS = 15_000;
|
|
451
444
|
// Idle TTL: a session is disposed this long after the last segment/playlist
|
|
452
445
|
// access. Long enough that a viewer who pauses, backgrounds the tab, or briefly
|
|
453
446
|
// turns the phone off can resume WITHOUT a cold ffmpeg restart (the warm session
|
|
@@ -1577,6 +1570,11 @@ export class HlsSessionManager {
|
|
|
1577
1570
|
contentionPenalties: this.contentionPenalties,
|
|
1578
1571
|
startingSpeedFor: (address) => this.encodeCost.speedForOutput(address),
|
|
1579
1572
|
segmentStore: this.segmentStore,
|
|
1573
|
+
// HOW IT ASKS TO DECIDE AGAIN. A plan that refuses to place anything
|
|
1574
|
+
// because an output's input is away needs something to bring it back:
|
|
1575
|
+
// nothing about the state changes while the data is missing, so no event
|
|
1576
|
+
// arrives on its own.
|
|
1577
|
+
planSoon: () => this.planEncodersSoon(),
|
|
1580
1578
|
logger
|
|
1581
1579
|
});
|
|
1582
1580
|
// What a start and a stop were measured to cost here, before any viewer
|
|
@@ -5709,33 +5707,19 @@ export class HlsSessionManager {
|
|
|
5709
5707
|
// condemn a session whose data merely went away.
|
|
5710
5708
|
if (ended.ending === ENCODE_EXIT.INPUT_LOST) {
|
|
5711
5709
|
session.inputRetryCount = (session.inputRetryCount ?? 0) + 1;
|
|
5712
|
-
const delayMs = Math.min(
|
|
5713
|
-
INPUT_RETRY_MAX_MS,
|
|
5714
|
-
INPUT_RETRY_BASE_MS * 2 ** Math.min(session.inputRetryCount - 1, 6)
|
|
5715
|
-
);
|
|
5716
5710
|
logger.warn(
|
|
5717
5711
|
`transcode ${session.id} encode-run #${ended.from}..#${ended.to} lost its input ` +
|
|
5718
|
-
`(${session.lastError})
|
|
5719
|
-
`(attempt ${session.inputRetryCount})`
|
|
5712
|
+
`(${session.lastError}) (attempt ${session.inputRetryCount})`
|
|
5720
5713
|
);
|
|
5721
|
-
|
|
5722
|
-
|
|
5723
|
-
|
|
5724
|
-
|
|
5725
|
-
|
|
5714
|
+
// HOW LONG TO WAIT IS THE PLAN'S, and this says only what happened. The
|
|
5715
|
+
// delay used to be timed here, against the dead run, which the plan never
|
|
5716
|
+
// consults — so it placed a fresh run at the same spot as fast as ffmpeg
|
|
5717
|
+
// could fail there: 2432 starts in 23 minutes in the field 2026-09-12,
|
|
5718
|
+
// against a delay that had reached its 15 s ceiling long before. The
|
|
5719
|
+
// orchestrator holds it now, beside the decision it governs.
|
|
5720
|
+
if (run.state === ENCODE_RUN_STATE.RETRY_WAIT) {
|
|
5726
5721
|
run.retryDue();
|
|
5727
|
-
|
|
5728
|
-
// decision. Where to start is the plan's, from where the viewers are;
|
|
5729
|
-
// this used to start one at the segment last requested, which is the
|
|
5730
|
-
// player's read head rather than anybody's position, and is a number
|
|
5731
|
-
// requests are explicitly not allowed to steer an encoder by.
|
|
5732
|
-
//
|
|
5733
|
-
// What the delay is for stays: the plan is a function of the state, and
|
|
5734
|
-
// nothing about the state changes while the torrent is away, so it would
|
|
5735
|
-
// command the same start as fast as ffmpeg could fail.
|
|
5736
|
-
this.planEncodersSoon();
|
|
5737
|
-
}, delayMs);
|
|
5738
|
-
session.inputRetryTimer.unref?.();
|
|
5722
|
+
}
|
|
5739
5723
|
return;
|
|
5740
5724
|
}
|
|
5741
5725
|
// A run that exits THIS fast never did real work: it failed at the start
|
|
@@ -7647,14 +7631,6 @@ export class HlsSessionManager {
|
|
|
7647
7631
|
* @returns {void}
|
|
7648
7632
|
*/
|
|
7649
7633
|
#stopEncodeRun(session, reason) {
|
|
7650
|
-
// Everything armed to re-decide on this session's behalf. A stop that
|
|
7651
|
-
// leaves the input-retry timer running is not a stop: it fires seconds
|
|
7652
|
-
// later and has the plan asked again for a session nobody is watching, and
|
|
7653
|
-
// torrent starvation, which is what arms it, is routine here.
|
|
7654
|
-
if (session.inputRetryTimer) {
|
|
7655
|
-
clearTimeout(session.inputRetryTimer);
|
|
7656
|
-
session.inputRetryTimer = null;
|
|
7657
|
-
}
|
|
7658
7634
|
const running = liveRunsOf(session);
|
|
7659
7635
|
if (running.length === 0) {
|
|
7660
7636
|
return;
|
|
@@ -9969,10 +9945,6 @@ export class HlsSessionManager {
|
|
|
9969
9945
|
}
|
|
9970
9946
|
}
|
|
9971
9947
|
|
|
9972
|
-
if (session.inputRetryTimer) {
|
|
9973
|
-
clearTimeout(session.inputRetryTimer);
|
|
9974
|
-
session.inputRetryTimer = null;
|
|
9975
|
-
}
|
|
9976
9948
|
|
|
9977
9949
|
// Whether the process is still RUNNING, not whether anyone has called kill
|
|
9978
9950
|
// on it: `.killed` means only that a signal was sent, and a run that ended
|
|
@@ -33,6 +33,20 @@ import { contentionPenalty } from "../encode/contention.js";
|
|
|
33
33
|
import { waits } from "../priority/WaitLedger.js";
|
|
34
34
|
import { SegmentDemand } from "../encode/SegmentDemand.js";
|
|
35
35
|
|
|
36
|
+
/**
|
|
37
|
+
* How long nothing is placed on an output whose input has just gone, and the
|
|
38
|
+
* ceiling that doubling reaches.
|
|
39
|
+
*
|
|
40
|
+
* Both are chosen, and are named here as chosen rather than dressed as
|
|
41
|
+
* measurements: what they bound is not how long the data takes to come back —
|
|
42
|
+
* that is the swarm's business and nobody here can know it — but how often it
|
|
43
|
+
* is worth asking. They replace the same two figures, with the same values,
|
|
44
|
+
* that lived in the session manager and governed only the dead run's own retry
|
|
45
|
+
* while the plan placed fresh runs beside it every half second.
|
|
46
|
+
*/
|
|
47
|
+
const INPUT_QUIET_BASE_MS = 2_000;
|
|
48
|
+
const INPUT_QUIET_MAX_MS = 15_000;
|
|
49
|
+
|
|
36
50
|
export class EncodeOrchestrator {
|
|
37
51
|
/** Output address to what has been made of it. @type {Map<string, CoverageMap>} */
|
|
38
52
|
#coverage = new Map();
|
|
@@ -47,6 +61,20 @@ export class EncodeOrchestrator {
|
|
|
47
61
|
/** How runs have ended, by cause. @type {Map<string, number>} */
|
|
48
62
|
#endings = new Map();
|
|
49
63
|
|
|
64
|
+
/**
|
|
65
|
+
* How many times in a row an output's run has died because its input was not
|
|
66
|
+
* there. Cleared by any other ending, which is the input answering.
|
|
67
|
+
*
|
|
68
|
+
* @type {Map<string, number>}
|
|
69
|
+
*/
|
|
70
|
+
#inputLostAttempts = new Map();
|
|
71
|
+
|
|
72
|
+
/** Output address to the time before which nothing is placed on it. @type {Map<string, number>} */
|
|
73
|
+
#quietUntil = new Map();
|
|
74
|
+
|
|
75
|
+
/** The wake-up per output, so a quiet output is reconsidered. @type {Map<string, NodeJS.Timeout>} */
|
|
76
|
+
#quietTimers = new Map();
|
|
77
|
+
|
|
50
78
|
/** The last state said out loud, so an unchanged state is not repeated. */
|
|
51
79
|
#lastDescribed = "";
|
|
52
80
|
|
|
@@ -97,6 +125,7 @@ export class EncodeOrchestrator {
|
|
|
97
125
|
refetchSecPerFilmSecond = () => 0,
|
|
98
126
|
startingSpeedFor = () => 0,
|
|
99
127
|
segmentStore = null,
|
|
128
|
+
planSoon = null,
|
|
100
129
|
logger,
|
|
101
130
|
now
|
|
102
131
|
}) {
|
|
@@ -124,6 +153,9 @@ export class EncodeOrchestrator {
|
|
|
124
153
|
this.segmentSeconds = segmentSeconds;
|
|
125
154
|
this.logger = logger;
|
|
126
155
|
this.now = typeof now === "function" ? now : Date.now;
|
|
156
|
+
// Asked for, not commanded: this class decides, and something else owns the
|
|
157
|
+
// loop that calls it. Absent in a test, where the clock is the test's own.
|
|
158
|
+
this.planSoon = typeof planSoon === "function" ? planSoon : () => undefined;
|
|
127
159
|
}
|
|
128
160
|
|
|
129
161
|
/**
|
|
@@ -366,6 +398,20 @@ export class EncodeOrchestrator {
|
|
|
366
398
|
});
|
|
367
399
|
}
|
|
368
400
|
}
|
|
401
|
+
// ITS INPUT WAS NOT THERE A MOMENT AGO, so nothing is placed yet. The sweep
|
|
402
|
+
// above still runs — a claim left by a dead run must be released whatever
|
|
403
|
+
// the reason — and only the placing waits. This is the one thing that makes
|
|
404
|
+
// the delay bind: it used to be timed against the dead run, which the plan
|
|
405
|
+
// does not consult, so a fresh run went to the same place as fast as ffmpeg
|
|
406
|
+
// could fail there.
|
|
407
|
+
//
|
|
408
|
+
// AND ONLY WHILE NOTHING IS PRODUCING THERE. A run still alive on this
|
|
409
|
+
// output is proof the input can be read, whatever a run beside it met, so
|
|
410
|
+
// the wait must not silence an output that is working.
|
|
411
|
+
const quietMs = this.#quietFor(address);
|
|
412
|
+
if (quietMs > 0 && this.runsOn(address).every((run) => !run.isAlive)) {
|
|
413
|
+
return;
|
|
414
|
+
}
|
|
369
415
|
// ONE MAP, NOT ONE WINDOW PER VIEWER PER ZONE.
|
|
370
416
|
//
|
|
371
417
|
// Two viewers a few seconds apart state stretches that overlap, and the plan
|
|
@@ -696,6 +742,93 @@ export class EncodeOrchestrator {
|
|
|
696
742
|
this.#runs.set(ended.address, remaining);
|
|
697
743
|
}
|
|
698
744
|
this.#endings.set(ended.ending, (this.#endings.get(ended.ending) ?? 0) + 1);
|
|
745
|
+
this.#noteInputAvailability(ended);
|
|
746
|
+
}
|
|
747
|
+
|
|
748
|
+
/**
|
|
749
|
+
* Remember, per output, that its input was not there — and until when nothing
|
|
750
|
+
* is to be placed on it.
|
|
751
|
+
*
|
|
752
|
+
* **Why the plan has to hold this.** A run whose input has gone is not alive,
|
|
753
|
+
* so the plan sees the stretch it held as free and places another run there at
|
|
754
|
+
* once — and the next one dies the same way, because nothing about the state
|
|
755
|
+
* has changed. There WAS a delay for exactly this, doubling from 2 s to 15 s,
|
|
756
|
+
* and it governed only the dead run's own retry while the plan went on placing
|
|
757
|
+
* fresh ones beside it. The comment beside that timer predicted this in as
|
|
758
|
+
* many words and the code did not prevent it. Field 2026-09-12: 2432 ffmpeg
|
|
759
|
+
* starts in 23 minutes, one every 0.57 s, for 61 minutes, against a delay that
|
|
760
|
+
* had long since reached its 15 s cap.
|
|
761
|
+
*
|
|
762
|
+
* So the delay lives where the decision is taken. It is not a cure for the
|
|
763
|
+
* input being away — that is the store's claim being withdrawn and the piece
|
|
764
|
+
* being fetched again — it is what stops one absent input from costing a
|
|
765
|
+
* thousand processes and a quarter of a million log lines while it is away.
|
|
766
|
+
*
|
|
767
|
+
* @param {{ address: string, ending: string }} ended
|
|
768
|
+
* @returns {void}
|
|
769
|
+
*/
|
|
770
|
+
#noteInputAvailability(ended) {
|
|
771
|
+
if (ended.ending !== ENCODE_EXIT.INPUT_LOST) {
|
|
772
|
+
// CLEARED ONLY BY PROOF THAT THE INPUT WAS THERE, which is a segment
|
|
773
|
+
// having come out of it. Any other ending was nearly the rule here and is
|
|
774
|
+
// wrong: at the moment of failure several runs end at once, and one of
|
|
775
|
+
// them ending `gone` or `stopped` without producing a thing says nothing
|
|
776
|
+
// about the input — it would have lifted the wait the one beside it had
|
|
777
|
+
// just set, which is the storm again with an extra step.
|
|
778
|
+
const produced =
|
|
779
|
+
ended.firstOutputMs !== null && ended.firstOutputMs !== undefined
|
|
780
|
+
? true
|
|
781
|
+
: Number.isFinite(ended.reached) && Number.isFinite(ended.from) && ended.reached >= ended.from;
|
|
782
|
+
if (produced) {
|
|
783
|
+
this.#inputLostAttempts.delete(ended.address);
|
|
784
|
+
this.#quietUntil.delete(ended.address);
|
|
785
|
+
}
|
|
786
|
+
return;
|
|
787
|
+
}
|
|
788
|
+
const attempts = (this.#inputLostAttempts.get(ended.address) ?? 0) + 1;
|
|
789
|
+
this.#inputLostAttempts.set(ended.address, attempts);
|
|
790
|
+
const delayMs = Math.min(
|
|
791
|
+
INPUT_QUIET_MAX_MS,
|
|
792
|
+
INPUT_QUIET_BASE_MS * 2 ** Math.min(attempts - 1, 6)
|
|
793
|
+
);
|
|
794
|
+
this.#quietUntil.set(ended.address, this.now() + delayMs);
|
|
795
|
+
this.logger.info(
|
|
796
|
+
`encode-plan on ${ended.address}: its input was not there (attempt ${attempts}) — ` +
|
|
797
|
+
`placing nothing for ${Math.round(delayMs / 1000)}s`
|
|
798
|
+
);
|
|
799
|
+
// THE WAKE-UP, because a plan that refuses to act needs something to ask it
|
|
800
|
+
// again: nothing about the state changes while the data is away, so no event
|
|
801
|
+
// would arrive to reconsider it.
|
|
802
|
+
const timer = setTimeout(() => {
|
|
803
|
+
this.#quietTimers.delete(ended.address);
|
|
804
|
+
this.planSoon();
|
|
805
|
+
}, delayMs);
|
|
806
|
+
timer.unref?.();
|
|
807
|
+
const previous = this.#quietTimers.get(ended.address);
|
|
808
|
+
if (previous) {
|
|
809
|
+
clearTimeout(previous);
|
|
810
|
+
}
|
|
811
|
+
this.#quietTimers.set(ended.address, timer);
|
|
812
|
+
}
|
|
813
|
+
|
|
814
|
+
/**
|
|
815
|
+
* How long this output still has to wait before anything is placed on it, in
|
|
816
|
+
* milliseconds; zero when it may be planned now.
|
|
817
|
+
*
|
|
818
|
+
* @param {string} address
|
|
819
|
+
* @returns {number}
|
|
820
|
+
*/
|
|
821
|
+
#quietFor(address) {
|
|
822
|
+
const until = this.#quietUntil.get(address);
|
|
823
|
+
if (!Number.isFinite(until)) {
|
|
824
|
+
return 0;
|
|
825
|
+
}
|
|
826
|
+
const left = until - this.now();
|
|
827
|
+
if (left <= 0) {
|
|
828
|
+
this.#quietUntil.delete(address);
|
|
829
|
+
return 0;
|
|
830
|
+
}
|
|
831
|
+
return left;
|
|
699
832
|
}
|
|
700
833
|
|
|
701
834
|
/**
|