@torrent-tv/proxy 2.83.4 → 2.83.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +21 -0
- package/package.json +1 -1
- package/research/piece-withdrawn-but-still-claimed-2026-09-12.md +175 -0
- package/research/priority-map-is-the-truth-2026-09-12.md +35 -17
- package/services/download/withdraw-claim.js +80 -0
- package/services/hls-session-manager.js +14 -42
- package/services/orchestrators/EncodeOrchestrator.js +163 -0
- package/services/piece-store/piece-disk-store.js +24 -1
- package/services/piece-store/shared-piece-store.js +71 -1
- package/services/torrent-pool.js +76 -2
- package/services/torrent-worker/client.js +5 -2
- package/services/torrent-worker/piece-reader.js +30 -1
- package/services/torrent-worker/worker.js +29 -3
- package/test/input-lost-quiets-the-plan.test.js +261 -0
- package/test/logger-repeats.test.js +134 -0
- package/test/read-survives-withdrawal.test.js +164 -0
- package/test/withdraw-piece-claim.test.js +212 -0
- package/utils/logger.js +135 -7
package/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,24 @@
|
|
|
1
|
+
## 2.83.6
|
|
2
|
+
- **Fix**: A line forwarded from the torrent thread is no longer judged twice. That thread holds the same repeat rule and applies it before forwarding, and the main thread ran it again on a different history. It does not lose lines today — a re-printed line carries its held-back count, which makes the text unique — but that is an accident of the wording, against a promise this module states outright: a line cannot reach the console and miss the file. It also filled the main thread's bounded map with keys that can never repeat.
|
|
3
|
+
- **Fix**: The LEVEL travels with a forwarded line. It was dropped at the send, so every warning and every error the torrent thread has ever written arrived on the other side as information, and was coloured and recorded as such.
|
|
4
|
+
- **Fix**: The wait after a lost input is cleared only by a run that PRODUCED something, and the field documenting it said "cleared by any other ending" — the rule it replaced, two screens above the code that contradicted it.
|
|
5
|
+
- **New**: `torrent-pool: N piece claim(s) withdrawn` is printed. The pool counted them and nothing read the count, which is the state this project's own rule calls indistinguishable from having noticed once. The gap between it and the stores' `withdrawn=` is the reading: announcements far above withdrawals mean the stores are mostly dropping pieces that were never completed, and a withdrawal count stuck at zero while announcements climb means the mechanism is not reaching the torrent at all.
|
|
6
|
+
- **Chore**: What the plan remembers about an output's input is forgotten when the priority map says nobody is coming there. Three maps keyed by address would otherwise keep an entry for the life of the process, and the wait they describe would greet whoever opened that output next.
|
|
7
|
+
- **Chore**: The suffix saying how many repeats were held back was built in two branches with the same expression; one function now. The quiet check returned milliseconds that its only reader compared against zero; a boolean now.
|
|
8
|
+
|
|
9
|
+
## 2.83.5
|
|
10
|
+
- **Fix**: ONE OWNER OF THE FACT "THIS PROXY HAS PIECE N", AND IT IS THE STORE. The disk tier drops a piece once every reader is past it — correctly, and that is what bounds the spill — while the library kept a second copy of the same fact in its completion bitfield and nothing reconciled them. A read then concluded the piece was had, asked for it, was told it was absent, and failed; nor was it ever fetched again, because the library does not download what it believes it owns. Field 2026-09-12: a film played 80 seconds (1920 frames, and the picture never moved again); the encoder ran on to 725 s, so some 565 spilled pieces fell behind every read head and were dropped, piece 0 among them; the encoder lost its input and restarted, which re-opens the input at byte 0; and `/stream` answered `0 of 2363497962 bytes: Piece 0 is verified but absent from the store` to every read for the next 92 minutes while the browser retried one segment against a healthy transport at 4 ms.
|
|
11
|
+
- **Fix**: So the store announces when it can no longer produce a piece AT ALL — not resident, not on disk, not inside a file held whole — and the pool withdraws the library's claim. The eviction's own bargain, written in its comment as "a seek back re-downloads it", is true for the first time. The three cases are one rule rather than a list of call sites: the disk tier says only that IT has lost a piece, and the store decides whether that is a loss at all, so a piece dropped as a duplicate of an assembled file says nothing and a piece still resident says nothing.
|
|
12
|
+
- **Chore**: The withdrawal is a plain function over plain values (`services/download/withdraw-claim.js`) and holds no pool, no store and no client, so it can be exercised with an object literal. It deliberately does not re-select the piece: every torrent is added with `deselect: true`, which makes the library skip that, and the download set keeps its one owner.
|
|
13
|
+
- **Chore**: `withdrawn=` joins the piece store's own line, so the bargain is checkable by number instead of by belief.
|
|
14
|
+
- **Fix**: A READ NOW SURVIVES A PIECE WITHDRAWN UNDER IT. An empty answer is a wait, not a failure: the claim has just gone, so the piece is being fetched again, and the read goes back one step and takes the whole ordinary path for it — declare it, steer it onto the fastest holders, wait. Once per piece; a second emptiness says the bytes are not coming and is reported as that rather than as "verified but absent", which was never true.
|
|
15
|
+
- **Fix**: THE DELAY AFTER A LOST INPUT NOW BINDS. It existed — doubling from 2 s to 15 s — and it was timed against the DEAD RUN, which the plan never consults: a run whose input has gone is not alive, so the plan read the stretch it held as free and placed another there at once, which died the same way. Field: 2432 ffmpeg starts in 23 minutes, one every 0.57 s, for 61 minutes, against a delay that had reached its ceiling long before. The wait lives beside the decision it governs now, per output, and the plan places nothing there until it is over — unless something on that output is still producing, which is proof the input can be read.
|
|
16
|
+
- **Fix**: The wait is lifted only by proof that the input was there, which is a segment having come out of it. Any other ending nearly did it, and that is wrong: at the moment of failure several runs end at once, and one ending `gone` with nothing produced would have lifted the wait the one beside it had just set.
|
|
17
|
+
- **Chore**: The two figures and the timer leave `hls-session-manager.js` for the encoding layer; the file is 28 lines shorter.
|
|
18
|
+
- **Fix**: AN ESTABLISHED FACT IS SAID ONCE, THEN WITH DECREASING FREQUENCY. One absent piece produced about 55 log lines a second: 235 000 lines in 92 minutes, of which 68.8 % were byte-identical repeats — measured, not estimated. Repeats are matched verbatim, the whole line, which catches nearly all of that and cannot merge two different statements; normalising numbers would catch a little more and would also merge the memory series, which exists precisely to catch a runaway. A held-back line is said again at doubling intervals up to a minute, and says how many were held back and over what span, because the rate is the fact.
|
|
19
|
+
- **Fix**: The log file may now grow to 1 GB before it turns over, from 32 MB. At the old size that one failure turned the file over twice inside the session it belonged to — 159 000 lines covering 38 minutes, then 76 385 covering 23 — so 61 minutes of 92 survived and the second rotation overwrote the turn holding the onset. The disk it was bounded for had 91.4 GB free.
|
|
20
|
+
- **Chore**: KNOWN CONSEQUENCE, stated rather than discovered later: a torrent whose pieces have been dropped is no longer `done`, so a film watched to the end is not assembled into a whole file. That is the honest state — we do not have those bytes — and the previous behaviour was worse than it looked: the assembly reads the file through the store, so it would have waited for pieces the library wrongly believed it had.
|
|
21
|
+
|
|
1
22
|
## 2.83.4
|
|
2
23
|
- **Chore**: WHAT IS IN A TORRENT IS NOW A THING RATHER THAN A RECOMPUTATION. `services/torrent/Contents.js` holds one torrent's list of files, its pictures with the sound, subtitles and images that belong to each, the order a person reads them in, and what belongs to nothing. The pairing rules were already there as functions; what did not exist was anywhere to keep their answer, so the warm-up on one path and the playback plan on another worked out the same grouping over the same list several times per opened film. Checked against the survey collection: 134 torrents, 44 of them with more than one picture, 1357 pictures and 421 files paired to one, not a single file paired to two — and all 1357 groupings agree with the per-picture call they replace, so nothing about the pairing changes.
|
|
3
24
|
- **Chore**: It holds only what the torrent says about itself. Nothing about a FILM: a poster or a title is an answer from a third party about an identity read from a file's own bytes, keyed by the picture's file, and it must never delay playback. No reference to another layer either — not a container, not a priority map, not a viewer; what leaves is plain indices into the torrent's own list.
|
package/package.json
CHANGED
|
@@ -0,0 +1,175 @@
|
|
|
1
|
+
# A film played 80 seconds and then stood still for 92 minutes — 2026-09-12
|
|
2
|
+
|
|
3
|
+
Proxy 2.83.4, addon 0.74.1, one viewer, `Reacher.S04E07.1080p.rus.LostFilm.TV.mkv`
|
|
4
|
+
(2 363 497 962 bytes). Reported by the user as "the film only managed to play
|
|
5
|
+
one minute nineteen seconds".
|
|
6
|
+
|
|
7
|
+
Every claim below is tagged by where it comes from: **[log]** read out of a
|
|
8
|
+
captured log, **[code]** read out of the source, **[measured]** computed from
|
|
9
|
+
the captured logs, **[derived]** reasoned from two of the above.
|
|
10
|
+
|
|
11
|
+
## 1. What the viewer saw
|
|
12
|
+
|
|
13
|
+
**[log, browser]** Every one of the 1112 `decode` readings in the captured
|
|
14
|
+
window is identical: `t=80.0s size=1920x1080 frames=1920 dropped=0
|
|
15
|
+
readyState=2`. 1920 frames at 24 fps is 80.0 s, so the picture advanced 80
|
|
16
|
+
seconds and then never moved again. The page stayed open until 18:52.
|
|
17
|
+
|
|
18
|
+
**[log, browser]** From then on the player asked for one segment and was
|
|
19
|
+
refused, about once a minute for 92 minutes:
|
|
20
|
+
|
|
21
|
+
```
|
|
22
|
+
18:30:47 non-fatal: fragLoadError currentTime=79.97 msReadyState=open
|
|
23
|
+
frag=segment-00008.mp4 sn=8 action=5 error=Error: HTTP Error 503 …
|
|
24
|
+
18:41:15 fatal: fragLoadError … sn=8 … HTTP Error 503
|
|
25
|
+
18:41:17 recovered fatal networkError, resuming at 80.0s
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
`msReadyState=open` throughout, so this is **not** the closed-MediaSource
|
|
29
|
+
failure of item 86. **[log]** The transport was healthy the whole time —
|
|
30
|
+
`rtt=4ms`, `state=connected ice=connected`, every counter advancing — so it is
|
|
31
|
+
not item 10 either.
|
|
32
|
+
|
|
33
|
+
## 2. What the proxy answered, and why
|
|
34
|
+
|
|
35
|
+
**[code]** 503 in `routes/transcode/session-file/get.js:143` is the
|
|
36
|
+
`warming-up` branch: the segment is still being produced. It was never produced.
|
|
37
|
+
|
|
38
|
+
**[log]** 239 times:
|
|
39
|
+
|
|
40
|
+
```
|
|
41
|
+
transcode 17ec5953 holding segment-00008.mp4: the file is not on disk
|
|
42
|
+
(runs from #8, viewer at #8, encoder alive, index #8, produced 0.0s at 2.23x
|
|
43
|
+
— the encoder has not made it)
|
|
44
|
+
[hold] segment-00008.mp4 warming-up after 60400ms (the map wants it now, rank 69 of 69)
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
**[log]** The encoder could not open its input, 5712 times:
|
|
48
|
+
|
|
49
|
+
```
|
|
50
|
+
encode-run #8..#247 lost its input ([http @ …] Error reading HTTP response: End of file)
|
|
51
|
+
Error opening input file http://127.0.0.1:9090/stream?sourceKey=94dda59f… (8514×)
|
|
52
|
+
Error opening input files: End of file (5712×)
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
**[log]** And `/stream` said exactly what was wrong, 2884 times:
|
|
56
|
+
|
|
57
|
+
```
|
|
58
|
+
stream: read of "Reacher.S04E07.1080p.rus.LostFilm.TV.mkv" bytes 0-2363497961
|
|
59
|
+
ended after 0 of 2363497962 bytes: Piece 0 is verified but absent from the store.
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
Zero bytes of 2.36 GB. ffmpeg reads an empty body as the end of the file.
|
|
63
|
+
|
|
64
|
+
## 3. The cause
|
|
65
|
+
|
|
66
|
+
**[log]** The torrent believed it had the whole file:
|
|
67
|
+
|
|
68
|
+
```
|
|
69
|
+
[stats] 94dda59f 5d276be5 peers=54 connected … file=100.0% header=8388608/8388608B
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
**[log]** The store did not:
|
|
73
|
+
|
|
74
|
+
```
|
|
75
|
+
piece-store "Reacher.S04E07…": resident=27/42 (108MB of 168MB allowed)
|
|
76
|
+
committed=108MB blocks=27 on-disk=26MB pinned=0 spilled=7
|
|
77
|
+
reads=5932 (73.8% from memory) spills=571 revivals=41
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
**[measured]** 571 spills at a 4 MB piece is about 2284 MB written to disk;
|
|
81
|
+
26 MB remained. So roughly 565 pieces had been removed. **[log]** It was not
|
|
82
|
+
the disk cap: `disk: 93634MB free; spilled pieces 26MB of 46656MB` — the
|
|
83
|
+
ceiling never came near binding, and 93 GB were free.
|
|
84
|
+
|
|
85
|
+
**[code]** That leaves exactly one remover: `PieceDiskStore.forgetBehind`,
|
|
86
|
+
called from `SharedPieceStore.reviseSpillCeiling`, which drops every piece
|
|
87
|
+
lying behind the earliest read head. **[log]** The encoder had reached
|
|
88
|
+
`proxyProcessed=725.643`, twelve minutes of film, so piece 0 — the start of the
|
|
89
|
+
file and its Matroska header — was far behind every reader and was dropped.
|
|
90
|
+
That is the rule working as designed.
|
|
91
|
+
|
|
92
|
+
**[code]** `piece-reader.js:151` then asks `torrent.bitfield.get(index)`. The
|
|
93
|
+
library still answered yes, because it had downloaded and hashed that piece
|
|
94
|
+
once. **Nothing ever told it otherwise:** `bitfield.set` does not occur
|
|
95
|
+
anywhere in `services/piece-store/`, `services/torrent-pool.js` or
|
|
96
|
+
`services/torrent-worker/`. So the read concluded the piece was had, asked the
|
|
97
|
+
store, was told it was absent, and threw.
|
|
98
|
+
|
|
99
|
+
**[derived]** And it was never fetched again, because the library does not
|
|
100
|
+
download a piece it believes it owns. The comment on `forgetBehind` states the
|
|
101
|
+
bargain in as many words — "a seek back re-downloads it, which is the same
|
|
102
|
+
bargain this tier makes whenever it drops a piece for room" — and that bargain
|
|
103
|
+
had never once been honoured.
|
|
104
|
+
|
|
105
|
+
**[derived]** The trigger is ordinary: an encoder restart re-opens its input at
|
|
106
|
+
byte 0 (`bytes 0-2363497961` in the log), which is precisely the region
|
|
107
|
+
`forgetBehind` removes first.
|
|
108
|
+
|
|
109
|
+
This is the "un-have problem" the parent `CLAUDE.md` names in the Level 2 disk
|
|
110
|
+
design — "mark the piece incomplete in the completion store → it re-downloads on
|
|
111
|
+
the next read". The eviction shipped; the un-have did not.
|
|
112
|
+
|
|
113
|
+
## 4. Two amplifiers, each a defect of its own
|
|
114
|
+
|
|
115
|
+
### The delay after a lost input did not bind
|
|
116
|
+
|
|
117
|
+
**[code]** `hls-session-manager.js` timed a retry against the DEAD RUN, doubling
|
|
118
|
+
2 s → 15 s. **[code]** `EncodeRun.isAlive` is false in `RETRY_WAIT`, and the
|
|
119
|
+
orchestrator plans from live runs only — so the plan saw the stretch as free and
|
|
120
|
+
placed a new run at once, on every event.
|
|
121
|
+
|
|
122
|
+
**[measured]** 2432 `restart for` lines in 23 minutes: one every 0.57 s, for 61
|
|
123
|
+
minutes, while the delay printed `retrying in 15s`. The comment beside that
|
|
124
|
+
timer predicted this exactly and the code did not prevent it.
|
|
125
|
+
|
|
126
|
+
### The log destroyed the record of its own failure
|
|
127
|
+
|
|
128
|
+
**[measured]** 76 385 lines in 23 minutes and 159 000 in the previous 38 — about
|
|
129
|
+
55 lines a second, of which **52 567 of 76 385 (68.8 %) are byte-identical
|
|
130
|
+
repeats**. **[code]** `MAX_FILE_BYTES` was 32 MiB with one previous turn kept, so
|
|
131
|
+
the file turned over twice inside the session: 61 minutes of 92 survived, and the
|
|
132
|
+
onset — everything before 17:51 — was overwritten by the second rotation and is
|
|
133
|
+
gone. **[log]** `/data` had 91.4 GB free.
|
|
134
|
+
|
|
135
|
+
## 5. What was built (proxy 2.83.5)
|
|
136
|
+
|
|
137
|
+
1. the store announces when it can no longer produce a piece at all — not
|
|
138
|
+
resident, not on disk, not in a file held whole — and the pool withdraws the
|
|
139
|
+
library's claim with `_markUnverified`. The disk tier states only its own
|
|
140
|
+
loss; the store decides whether that is a loss;
|
|
141
|
+
2. a read whose piece is withdrawn under it goes back one step and takes the
|
|
142
|
+
whole ordinary path again instead of throwing. Once per piece;
|
|
143
|
+
3. the delay lives in the orchestrator, per output, and the plan places nothing
|
|
144
|
+
there until it is over — unless something on that output is still producing.
|
|
145
|
+
It is lifted only by a run that actually produced;
|
|
146
|
+
4. an established fact is logged once, then at doubling intervals to a minute,
|
|
147
|
+
saying how many were held back. Matched verbatim; digits are deliberately not
|
|
148
|
+
normalised, because that would merge the memory series;
|
|
149
|
+
5. the log file may reach 1 GB before turning over.
|
|
150
|
+
|
|
151
|
+
## 6. What this does NOT establish
|
|
152
|
+
|
|
153
|
+
1. **Nothing here has been seen in the field.** The next session is the proof.
|
|
154
|
+
What should appear: `withdrawn=` climbing on the piece-store line while reads
|
|
155
|
+
keep succeeding, `its input was not there (attempt N) — placing nothing for
|
|
156
|
+
Xs`, and `[said N more time(s) …]` instead of a flood. What should disappear:
|
|
157
|
+
`Piece N is verified but absent from the store`.
|
|
158
|
+
2. **The onset was not read.** Both log turns begin with the failure already
|
|
159
|
+
established, so which piece failed FIRST, and whether the encoder's first
|
|
160
|
+
input loss had the same cause, is not known. The mechanism is established;
|
|
161
|
+
its first instance is not.
|
|
162
|
+
3. **A consequence, stated rather than discovered later:** a torrent whose
|
|
163
|
+
pieces have been withdrawn is no longer `done`, so a film watched to the end
|
|
164
|
+
is not assembled into a whole file. That is the honest state — those bytes
|
|
165
|
+
are not here — and the previous behaviour was worse than it looked, since the
|
|
166
|
+
assembly reads the file through the store and would have waited for pieces
|
|
167
|
+
the library wrongly believed it had. What is NOT measured is how often a film
|
|
168
|
+
now fails to be assembled, and whether keeping the pieces of a nearly-finished
|
|
169
|
+
file would be worth the disk.
|
|
170
|
+
4. **The re-download cost is not measured.** A seek back past the read heads now
|
|
171
|
+
waits for pieces to be fetched again instead of failing. How long that takes
|
|
172
|
+
on a real swarm, and whether the viewer notices, needs a session.
|
|
173
|
+
5. **`peers=54 connected of -4599 known (…, -1128 queued)`** — the negative
|
|
174
|
+
figures come from WebTorrent's own `_peersLength`/`_numQueued`, read for the
|
|
175
|
+
log only. Not investigated, not touched.
|
|
@@ -188,20 +188,38 @@ both fixed with a check apiece.
|
|
|
188
188
|
the moment of the departure and then never re-examined: in its swarm, with no
|
|
189
189
|
idle clock running. The pass acts on what it just said now.
|
|
190
190
|
|
|
191
|
-
##
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
|
|
205
|
-
|
|
206
|
-
|
|
207
|
-
|
|
191
|
+
## Where the shared statement lives — DECIDED, then REVERSED the same day
|
|
192
|
+
|
|
193
|
+
It was first published as a package, `@torrent-tv/torrent-contents`, so that
|
|
194
|
+
both runtimes could read the same three files: the proxy states what to fetch
|
|
195
|
+
from them, and the browser showed the list of episodes the instant a `.torrent`
|
|
196
|
+
was dropped, before a proxy had been chosen.
|
|
197
|
+
|
|
198
|
+
**Withdrawn within the hour, on the user's question: is the package better than
|
|
199
|
+
moving it all to the proxy, when the torrent has to go there after the choice
|
|
200
|
+
anyway.** It is not. The package bought one property — the instant list — with
|
|
201
|
+
three permanent costs: a third artifact to version and publish, two copies at
|
|
202
|
+
runtime that drift by version (a test was written for that drift, which is an
|
|
203
|
+
admission the risk is real), and three wiring points that break the page at load
|
|
204
|
+
with no error anybody sees (a second test was written for those).
|
|
205
|
+
|
|
206
|
+
The answer is the route that already lists a source's files. For a magnet
|
|
207
|
+
nothing changes at all — that is the path it has always taken — and the dropped
|
|
208
|
+
`.torrent` now takes the same one. What leaves the browser with it is not only
|
|
209
|
+
the classification but the whole notion of deciding anything from a name; what
|
|
210
|
+
stays is what only the browser can see (the trackers and the web seeds in the
|
|
211
|
+
file it holds) and how a name is SHOWN.
|
|
212
|
+
|
|
213
|
+
**The cost is answered by connecting earlier, which is the user's own point.**
|
|
214
|
+
Everything a viewer does needs a proxy, and choosing one and connecting to it
|
|
215
|
+
used to begin only after a file had been picked. It begins when the page loads
|
|
216
|
+
now, silently and with its failure swallowed, and `#acquireTransport` is
|
|
217
|
+
joinable — so whatever the viewer does next joins that attempt instead of
|
|
218
|
+
starting a second. Verified in a browser 2026-09-12: the page opens, the connect
|
|
219
|
+
runs on its own (`public-only connect failed …; trying local path` on a dev
|
|
220
|
+
server with no proxies registered), nothing is shown for it, and the page's own
|
|
221
|
+
adapter turns the proxy's answer into the three lists with the order preserved,
|
|
222
|
+
the torrent's own numbers kept and the release's repeated part taken off the
|
|
223
|
+
names.
|
|
224
|
+
|
|
225
|
+
`@torrent-tv/torrent-contents@1.0.0` is deprecated on npm with the reason.
|
|
@@ -0,0 +1,80 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @file Withdrawing the claim that this proxy has a piece.
|
|
3
|
+
*
|
|
4
|
+
* **The fact has one owner and it is the store.** The store holds the bytes, so
|
|
5
|
+
* it says what this proxy has. The library keeps a second copy of that fact in
|
|
6
|
+
* its completion bitfield, and until 2026-09-12 nothing reconciled the two: the
|
|
7
|
+
* disk tier drops a piece once every reader is past it — correctly, and that is
|
|
8
|
+
* what bounds the spill — while the bitfield went on saying the piece was
|
|
9
|
+
* verified. A read then concluded the piece was had, asked for it, was told it
|
|
10
|
+
* was absent, and failed. Nor was it ever fetched again, because the library
|
|
11
|
+
* does not download what it believes it already owns.
|
|
12
|
+
*
|
|
13
|
+
* Field 2026-09-12: a film played 80 seconds; the encoder ran on to 725 s, so
|
|
14
|
+
* some 565 spilled pieces fell behind every read head and were dropped,
|
|
15
|
+
* including piece 0; the encoder then lost its input and restarted, which
|
|
16
|
+
* re-opens the input at byte 0; and `/stream` answered `0 of 2363497962 bytes:
|
|
17
|
+
* Piece 0 is verified but absent from the store` to every read for the next 92
|
|
18
|
+
* minutes while the browser retried one segment and the picture stood still.
|
|
19
|
+
*
|
|
20
|
+
* So the eviction's own bargain — "a seek back re-downloads it" — is made true
|
|
21
|
+
* here, and the bitfield becomes a projection of what the store holds.
|
|
22
|
+
*
|
|
23
|
+
* **Why it is a plain function.** It takes the values it needs and holds
|
|
24
|
+
* nothing: no pool, no store, no client. That is what lets it be exercised with
|
|
25
|
+
* an object literal, and it is the rule the layers are checked against — a layer
|
|
26
|
+
* must be usable with plain values alone, with no process, no disk and no clock.
|
|
27
|
+
*/
|
|
28
|
+
|
|
29
|
+
/**
|
|
30
|
+
* Tell the library it no longer has a piece, so the next read of it waits for a
|
|
31
|
+
* download instead of failing.
|
|
32
|
+
*
|
|
33
|
+
* **What it deliberately does not do.** The library's `_markUnverified` would
|
|
34
|
+
* also re-select the piece; it does not here, because every torrent is added
|
|
35
|
+
* with `deselect: true`, which sets the library's own `_startAsDeselected` and
|
|
36
|
+
* makes it skip that call. The download set has exactly one owner —
|
|
37
|
+
* `SwarmSelection`, from the priority map — and a withdrawn piece is fetched
|
|
38
|
+
* again when a read states it, which is the same statement every other piece
|
|
39
|
+
* waits on.
|
|
40
|
+
*
|
|
41
|
+
* @param {object} what
|
|
42
|
+
* @param {number} what.index - The piece the store can no longer produce.
|
|
43
|
+
* @param {object[]} [what.files] - The store's own files; the torrent that owns
|
|
44
|
+
* them is found through one of them, because the store has no idea what a
|
|
45
|
+
* torrent is.
|
|
46
|
+
* @param {object} [what.torrent] - Given directly instead of through `files`.
|
|
47
|
+
* @param {(line: string) => void} [what.warn] - Said when the library refuses.
|
|
48
|
+
* @returns {"withdrawn" | "nothing-to-withdraw" | "no-torrent" | "refused"}
|
|
49
|
+
*/
|
|
50
|
+
export function withdrawClaim({ index, files, torrent, warn = () => undefined }) {
|
|
51
|
+
if (!Number.isInteger(index) || index < 0) {
|
|
52
|
+
return "nothing-to-withdraw";
|
|
53
|
+
}
|
|
54
|
+
const owner = torrent ?? files?.[0]?._torrent ?? null;
|
|
55
|
+
// A torrent being destroyed is the ordinary case at the end of a session, and
|
|
56
|
+
// it has no claim left to withdraw: the store is going with it.
|
|
57
|
+
if (!owner || owner.destroyed || typeof owner._markUnverified !== "function") {
|
|
58
|
+
return "no-torrent";
|
|
59
|
+
}
|
|
60
|
+
// NOTHING TO WITHDRAW, which is most of the time: the store also drops pieces
|
|
61
|
+
// it never completed, and marking one unverified that the library already
|
|
62
|
+
// knows is missing would re-create the piece and discard whatever blocks are
|
|
63
|
+
// in flight for it.
|
|
64
|
+
if (!owner.bitfield?.get?.(index)) {
|
|
65
|
+
return "nothing-to-withdraw";
|
|
66
|
+
}
|
|
67
|
+
try {
|
|
68
|
+
owner._markUnverified(index);
|
|
69
|
+
} catch (error) {
|
|
70
|
+
// This reaches into the library's own bookkeeping. If a later version
|
|
71
|
+
// changes it, the honest result is a line saying so rather than an eviction
|
|
72
|
+
// that fails.
|
|
73
|
+
warn(
|
|
74
|
+
`could not withdraw the claim on piece ${index} of ${owner.name ?? "?"}: ` +
|
|
75
|
+
`${error?.message ?? error}`
|
|
76
|
+
);
|
|
77
|
+
return "refused";
|
|
78
|
+
}
|
|
79
|
+
return "withdrawn";
|
|
80
|
+
}
|
|
@@ -441,13 +441,6 @@ const START_FAST_FAIL_MS = 2_000;
|
|
|
441
441
|
// for whatever residual case still fails — not a second competing "fix" that
|
|
442
442
|
// blindly retries the identical command hoping for a different result.
|
|
443
443
|
const MAX_FAILED_STARTS = 3;
|
|
444
|
-
// A run that lost its INPUT is retried rather than condemned: the torrent can
|
|
445
|
-
// be added again and the pieces downloaded again, so the data being gone is a
|
|
446
|
-
// wait, not a verdict. Backed off so a source that is truly unavailable costs a
|
|
447
|
-
// process every few seconds rather than continuously, and never given up on —
|
|
448
|
-
// the session's own idle TTL is what ends it if the viewer leaves.
|
|
449
|
-
const INPUT_RETRY_BASE_MS = 2_000;
|
|
450
|
-
const INPUT_RETRY_MAX_MS = 15_000;
|
|
451
444
|
// Idle TTL: a session is disposed this long after the last segment/playlist
|
|
452
445
|
// access. Long enough that a viewer who pauses, backgrounds the tab, or briefly
|
|
453
446
|
// turns the phone off can resume WITHOUT a cold ffmpeg restart (the warm session
|
|
@@ -1577,6 +1570,11 @@ export class HlsSessionManager {
|
|
|
1577
1570
|
contentionPenalties: this.contentionPenalties,
|
|
1578
1571
|
startingSpeedFor: (address) => this.encodeCost.speedForOutput(address),
|
|
1579
1572
|
segmentStore: this.segmentStore,
|
|
1573
|
+
// HOW IT ASKS TO DECIDE AGAIN. A plan that refuses to place anything
|
|
1574
|
+
// because an output's input is away needs something to bring it back:
|
|
1575
|
+
// nothing about the state changes while the data is missing, so no event
|
|
1576
|
+
// arrives on its own.
|
|
1577
|
+
planSoon: () => this.planEncodersSoon(),
|
|
1580
1578
|
logger
|
|
1581
1579
|
});
|
|
1582
1580
|
// What a start and a stop were measured to cost here, before any viewer
|
|
@@ -5709,33 +5707,19 @@ export class HlsSessionManager {
|
|
|
5709
5707
|
// condemn a session whose data merely went away.
|
|
5710
5708
|
if (ended.ending === ENCODE_EXIT.INPUT_LOST) {
|
|
5711
5709
|
session.inputRetryCount = (session.inputRetryCount ?? 0) + 1;
|
|
5712
|
-
const delayMs = Math.min(
|
|
5713
|
-
INPUT_RETRY_MAX_MS,
|
|
5714
|
-
INPUT_RETRY_BASE_MS * 2 ** Math.min(session.inputRetryCount - 1, 6)
|
|
5715
|
-
);
|
|
5716
5710
|
logger.warn(
|
|
5717
5711
|
`transcode ${session.id} encode-run #${ended.from}..#${ended.to} lost its input ` +
|
|
5718
|
-
`(${session.lastError})
|
|
5719
|
-
`(attempt ${session.inputRetryCount})`
|
|
5712
|
+
`(${session.lastError}) (attempt ${session.inputRetryCount})`
|
|
5720
5713
|
);
|
|
5721
|
-
|
|
5722
|
-
|
|
5723
|
-
|
|
5724
|
-
|
|
5725
|
-
|
|
5714
|
+
// HOW LONG TO WAIT IS THE PLAN'S, and this says only what happened. The
|
|
5715
|
+
// delay used to be timed here, against the dead run, which the plan never
|
|
5716
|
+
// consults — so it placed a fresh run at the same spot as fast as ffmpeg
|
|
5717
|
+
// could fail there: 2432 starts in 23 minutes in the field 2026-09-12,
|
|
5718
|
+
// against a delay that had reached its 15 s ceiling long before. The
|
|
5719
|
+
// orchestrator holds it now, beside the decision it governs.
|
|
5720
|
+
if (run.state === ENCODE_RUN_STATE.RETRY_WAIT) {
|
|
5726
5721
|
run.retryDue();
|
|
5727
|
-
|
|
5728
|
-
// decision. Where to start is the plan's, from where the viewers are;
|
|
5729
|
-
// this used to start one at the segment last requested, which is the
|
|
5730
|
-
// player's read head rather than anybody's position, and is a number
|
|
5731
|
-
// requests are explicitly not allowed to steer an encoder by.
|
|
5732
|
-
//
|
|
5733
|
-
// What the delay is for stays: the plan is a function of the state, and
|
|
5734
|
-
// nothing about the state changes while the torrent is away, so it would
|
|
5735
|
-
// command the same start as fast as ffmpeg could fail.
|
|
5736
|
-
this.planEncodersSoon();
|
|
5737
|
-
}, delayMs);
|
|
5738
|
-
session.inputRetryTimer.unref?.();
|
|
5722
|
+
}
|
|
5739
5723
|
return;
|
|
5740
5724
|
}
|
|
5741
5725
|
// A run that exits THIS fast never did real work: it failed at the start
|
|
@@ -7647,14 +7631,6 @@ export class HlsSessionManager {
|
|
|
7647
7631
|
* @returns {void}
|
|
7648
7632
|
*/
|
|
7649
7633
|
#stopEncodeRun(session, reason) {
|
|
7650
|
-
// Everything armed to re-decide on this session's behalf. A stop that
|
|
7651
|
-
// leaves the input-retry timer running is not a stop: it fires seconds
|
|
7652
|
-
// later and has the plan asked again for a session nobody is watching, and
|
|
7653
|
-
// torrent starvation, which is what arms it, is routine here.
|
|
7654
|
-
if (session.inputRetryTimer) {
|
|
7655
|
-
clearTimeout(session.inputRetryTimer);
|
|
7656
|
-
session.inputRetryTimer = null;
|
|
7657
|
-
}
|
|
7658
7634
|
const running = liveRunsOf(session);
|
|
7659
7635
|
if (running.length === 0) {
|
|
7660
7636
|
return;
|
|
@@ -9969,10 +9945,6 @@ export class HlsSessionManager {
|
|
|
9969
9945
|
}
|
|
9970
9946
|
}
|
|
9971
9947
|
|
|
9972
|
-
if (session.inputRetryTimer) {
|
|
9973
|
-
clearTimeout(session.inputRetryTimer);
|
|
9974
|
-
session.inputRetryTimer = null;
|
|
9975
|
-
}
|
|
9976
9948
|
|
|
9977
9949
|
// Whether the process is still RUNNING, not whether anyone has called kill
|
|
9978
9950
|
// on it: `.killed` means only that a signal was sent, and a run that ended
|