@torrent-tv/proxy 2.70.0 → 2.71.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +20 -0
- package/CLAUDE.md +12 -0
- package/docs/download-architecture.md +175 -0
- package/docs/logs.md +8 -0
- package/package.json +1 -1
- package/services/demand/DemandRegister.js +182 -0
- package/services/demand/Urgency.js +137 -0
- package/services/demand/Window.js +118 -0
- package/services/demand/index.js +11 -0
- package/services/demand/pieces.js +140 -0
- package/services/download/SwarmSelection.js +367 -0
- package/services/download/index.js +8 -0
- package/services/download/registry.js +96 -0
- package/services/piece-store/shared-piece-store.js +16 -0
- package/services/playback-planner.js +15 -1
- package/services/torrent-pool.js +64 -197
- package/services/torrent-worker/fastest-wires.js +319 -279
- package/services/torrent-worker/piece-reader.js +149 -183
- package/services/torrent-worker/worker.js +23 -3
- package/test/demand-register.test.js +195 -0
- package/test/fastest-wires.test.js +23 -1
- package/test/read-bands.test.js +17 -10
- package/test/read-window.test.js +20 -8
- package/test/swarm-selection.test.js +220 -0
- package/utils/logger.js +62 -20
package/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,23 @@
|
|
|
1
|
+
## 2.71.1
|
|
2
|
+
|
|
3
|
+
- **Fix**: A reader stated the same thing twice — `protectRange` to the piece store for memory, and a window to the torrent for download — and the two were separate lists that could drift. There is one statement now: `SwarmSelection.reconcile` derives both views from the register, so the swarm and the store are told what to do from the same words. Only the urgent levels reach memory: it holds what will be READ soon, and protecting the speculative tail would push out a piece the decoder is about to want.
|
|
4
|
+
- **New**: A file this machine cannot sustain at ANY height is refused rather than served badly. Both offered lists empty means not even copying the picture — which costs no encoder at all — can keep up, so a session made there produces a slideshow and takes the swarm and the processor from whoever is already watching. Field 2026-08-28: five sessions on one file put every rung at 0.04x of realtime. The plan now carries `cannotServe` with the reason, which is a different thing from a spinner that never ends.
|
|
5
|
+
- **Chore**: `SwarmSelection` takes its store lookup as a parameter, so the memory projection is driven by a test without constructing a real piece store.
|
|
6
|
+
|
|
7
|
+
## 2.71.0
|
|
8
|
+
|
|
9
|
+
- **Fix**: The torrent thread's log lines never reached the log file. A worker thread loads its own instance of every module, so the logger's file handle — set once, on the main thread — was null there for the life of the process. Measured over a whole 49 938-line file: zero lines from the piece reader and zero from the torrent pool, against 52 and 36 of them in the container's output, which every release destroys. That is why the comparison of the two claim strategies could never be read: it was being printed into a place we wipe ourselves. The worker now sends its lines to the main thread, which is the only writer — two threads appending to one file would race on the rotation and could interleave mid-line.
|
|
10
|
+
- **New**: `services/demand/` — what anybody wants, stated once and in BYTES. `Window` (claimant, file, byte range, urgency), `Urgency`, `DemandRegister` (live windows by claimant), and `pieces.js`, the one place bytes become piece numbers. Nothing in it knows about WebTorrent, the piece store or pieces. The rounding it removes is where a real failure lived: with 16 MiB pieces a 64 MB allowance is four places while two readers asking for 96 MB each want six, and both figures had already been floored before anything could notice.
|
|
11
|
+
- **New**: `services/download/SwarmSelection.js` is the only thing in the proxy that calls `select`, `deselect` or `critical`. There were four callers before and their comments record them fighting: a whole-file read undid a seek that had just happened and the swarm walked forward from the first hole — on a 4.7 GB film, 2.47 GB over 93 s before the segment could be served.
|
|
12
|
+
- **Fix**: Urgency is no longer a number handed to the library, because the library does not keep it. Measured against the vendored 2.8.5: selections are sorted by priority only when one is inserted, and `shufflePriority` then moves the selection just served to the back of the whole non-zero group — so distinct numbers give an order once and a round robin thereafter. Five levels are kept here instead, and a level is stated only while every level above it is satisfied. The library is given the one distinction it honours: non-zero for what is wanted now, zero for the speculative tail.
|
|
13
|
+
- **Fix**: The speculative levels are withdrawn rather than lowered. A peer that cannot help with anything urgent falls through the selection list to whatever is below it, and with a permanently low priority would spend the shared link on pieces nobody is waiting for — about a second of its own throughput at a time. A withdrawn window is not in the download set at all. The condition is global across torrents: two films on one proxy share the link.
|
|
14
|
+
- **Fix**: Nothing is fetched until somebody states a need — the torrent is added with `deselect: true`. WebTorrent's own default is to select the whole torrent, and this proxy used to undo that afterwards by deselecting the files nobody had opened, so on a season pack every episode was fetched for as long as the viewer took to choose one.
|
|
15
|
+
- **Fix**: The background fill is stated per FILE, from the furthest window in that file to that file's end. It used to take the furthest window across all files and the last piece across all files and claim everything between: with two viewers on two episodes of one release, that claimed every episode lying between them.
|
|
16
|
+
- **Chore**: The two claim strategies are one. Each read was assigned at random to a single band or four, and the waits sorted by which; the split halved the sample and the ten-wait threshold was never reached in either arm — nine reads against three on 2026-08-28, and forty waits against one on 2026-08-29, the only day both arms printed. Waits are recorded by the LEVEL the reader was stopped in now, which says whether a band is too narrow rather than whether banding is the wrong idea. `TORRENT_TV_READ_MODE` is gone.
|
|
17
|
+
- **Chore**: `setActiveFile` deleted — no caller anywhere in the repository. `#reassertReaderWindows`, `#updateBackgroundFill`, `#tailAfterWindows` and `#syncSelections` deleted with the mechanisms they patched over; `claimWindow`, `releaseWindow`, `markCritical` and `clearCritical` deleted from the reader.
|
|
18
|
+
- **New**: `askFastestWiresFor` counts the requests refused because every block of the piece was already reserved and displacement did not happen. WebTorrent's displacement thresholds are constants, not settings — the asker must be above 16 KB/s, the holder below 48 KB/s and twice as slow — so a holder at 50 KB/s is never displaced however long the piece has been waited for. That counter is what would justify replacing the rule; a zero says the thresholds are not what we are short of.
|
|
19
|
+
- **Chore**: `docs/download-architecture.md`, in the shape of `docs/container-architecture.md`: the two axes, what is not a third, the layers as a diagram, and what was deleted with the reason. 19 checks in `test/demand-register.test.js` and `test/swarm-selection.test.js`.
|
|
20
|
+
|
|
1
21
|
## 2.70.0
|
|
2
22
|
|
|
3
23
|
- **New**: The piece store keeps a pool of memory blocks instead of allocating one per piece. A block is one piece's worth of memory; it is taken from the free list, and put back there when its piece is written out. Field 2026-09-02: 7575 allocations of 4 MiB in 44 minutes, each released only when the collector reached it, which is why the process held 1.86 GB while the store's own accounting said 352 MB. A block goes back for re-use only AFTER the spill write has finished, because that write reads out of it; a block whose piece is re-put while a reader holds it is given up rather than recycled, and a counter says if one ever is not.
|
package/CLAUDE.md
CHANGED
|
@@ -25,6 +25,14 @@ Linux-only host (e.g. POSIX-only signals must degrade elsewhere).
|
|
|
25
25
|
long-polls while a segment is being produced, returns retryable 503 (never
|
|
26
26
|
202 — hls.js can't consume it).
|
|
27
27
|
- `services/`:
|
|
28
|
+
- `demand/` — what anybody wants, in BYTES: `Window` (claimant, file, byte
|
|
29
|
+
range, urgency), `Urgency` (BLOCKED / NEAR / AHEAD / TAIL / BEHIND),
|
|
30
|
+
`DemandRegister` (live windows by claimant), `pieces.js` (the one place
|
|
31
|
+
bytes become piece numbers). No WebTorrent, no piece store, no pieces.
|
|
32
|
+
- `download/` — `SwarmSelection`, the ONLY thing that calls `select`,
|
|
33
|
+
`deselect` or `critical`, and `registry.js`, which holds one per torrent and
|
|
34
|
+
owns the cross-torrent rule that withholds the speculative levels while
|
|
35
|
+
anything urgent is missing anywhere. See `docs/download-architecture.md`.
|
|
28
36
|
- `container/` — domain layer: `Container` (abstract, RFC 9559 / ISO 14496-12),
|
|
29
37
|
`MatroskaContainer` / `Mp4Container` / `AviContainer`, `ContainerFactory`
|
|
30
38
|
(sniff 16 bytes → precise subclass). See `docs/container-architecture.md`.
|
|
@@ -42,6 +50,10 @@ Linux-only host (e.g. POSIX-only signals must degrade elsewhere).
|
|
|
42
50
|
`routes/*` are now thin HTTP translators.
|
|
43
51
|
- `container-index/` — legacy readers (ebml-reader, matroska/mp4/avi keyframe and
|
|
44
52
|
subtitle tables) — used internally by `container/*`, deprecated as direct import.
|
|
53
|
+
- `docs/download-architecture.md` — the two axes of downloading: what is
|
|
54
|
+
wanted (`demand/`) against what the swarm is told (`download/`), why urgency
|
|
55
|
+
is not a number given to the library, and why the speculative levels are
|
|
56
|
+
withdrawn rather than lowered.
|
|
45
57
|
- `docs/logs.md` — where to find logs (HA `docker logs` + `/data/proxy.log`, DO
|
|
46
58
|
`infra-server-1` with forwarded frontend `POST /api/client-logs`). Browser
|
|
47
59
|
console not needed.
|
|
@@ -0,0 +1,175 @@
|
|
|
1
|
+
# Download architecture — what is wanted, and what the swarm is told
|
|
2
|
+
|
|
3
|
+
Two questions, kept apart, and neither of them is "which file".
|
|
4
|
+
|
|
5
|
+
## Two axes, and what is NOT a third
|
|
6
|
+
|
|
7
|
+
**What somebody needs** — a claimant, a file, a byte range, a level of urgency.
|
|
8
|
+
Stated in `services/demand/`. Knows nothing about WebTorrent, nothing about
|
|
9
|
+
pieces, nothing about the piece store: it is numbers and rules, and it is
|
|
10
|
+
testable without a torrent.
|
|
11
|
+
|
|
12
|
+
**What the swarm is told** — one class, `services/download/SwarmSelection.js`,
|
|
13
|
+
which reads the register and calls `select`, `deselect` and `critical`. It is
|
|
14
|
+
the only thing in this proxy that calls them.
|
|
15
|
+
|
|
16
|
+
"Which file" is not a third axis. A file nobody has stated a need for is simply
|
|
17
|
+
absent from the register, and a torrent is added with `deselect: true` so
|
|
18
|
+
nothing is fetched until something is stated. Before 2026-09-02 the library's
|
|
19
|
+
own default selected the whole torrent and this proxy undid that afterwards by
|
|
20
|
+
deselecting the files nobody had opened — so on a season pack every episode was
|
|
21
|
+
being fetched for as long as the viewer took to choose one.
|
|
22
|
+
|
|
23
|
+
## Layers
|
|
24
|
+
|
|
25
|
+
```mermaid
|
|
26
|
+
flowchart TB
|
|
27
|
+
subgraph Statements["services/demand — what is wanted"]
|
|
28
|
+
W[Window<br/>claimant, file, bytes, urgency]
|
|
29
|
+
U[Urgency<br/>BLOCKED NEAR AHEAD TAIL BEHIND]
|
|
30
|
+
R[DemandRegister<br/>live windows by claimant]
|
|
31
|
+
P[pieces.js<br/>the one bytes to pieces conversion]
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
subgraph Swarm["services/download — what the swarm is told"]
|
|
35
|
+
S[SwarmSelection<br/>select / deselect / critical]
|
|
36
|
+
G[registry<br/>one per torrent + the cross-torrent rule]
|
|
37
|
+
end
|
|
38
|
+
|
|
39
|
+
subgraph Claimants["who states needs"]
|
|
40
|
+
PR[piece-reader<br/>four bands per read]
|
|
41
|
+
BF[torrent-pool<br/>background fill, per file]
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
PR -->|state / withdraw| R
|
|
45
|
+
BF -->|state / withdraw| R
|
|
46
|
+
W --> R
|
|
47
|
+
U --> R
|
|
48
|
+
R --> S
|
|
49
|
+
P --> S
|
|
50
|
+
G --> S
|
|
51
|
+
S -->|the only caller| WT[(WebTorrent)]
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
## The five levels
|
|
55
|
+
|
|
56
|
+
| level | what it is | stated |
|
|
57
|
+
|---|---|---|
|
|
58
|
+
| `BLOCKED` | the bytes a reader is stopped on | always; may take a block from a slow peer |
|
|
59
|
+
| `NEAR` | the rest of that reader's window | always |
|
|
60
|
+
| `AHEAD` | the lead the encoder will reach | always |
|
|
61
|
+
| `TAIL` | to the end of the file | only while nothing urgent is missing, anywhere |
|
|
62
|
+
| `BEHIND` | the gap left by a forward seek | only while nothing urgent is missing, anywhere |
|
|
63
|
+
|
|
64
|
+
## Why urgency is not a number handed to the library
|
|
65
|
+
|
|
66
|
+
Measured against the vendored WebTorrent 2.8.5. Selections are sorted by
|
|
67
|
+
priority **only when one is inserted**:
|
|
68
|
+
|
|
69
|
+
```js
|
|
70
|
+
this._selections.sort((a, b) => b.priority - a.priority)
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
and after a wire's pipeline is filled from a selection, that selection is moved
|
|
74
|
+
to the back of the whole non-zero group:
|
|
75
|
+
|
|
76
|
+
```js
|
|
77
|
+
function shufflePriority (i) {
|
|
78
|
+
let last = i
|
|
79
|
+
for (let j = i; j < self._selections.length && self._selections.get(j).priority; j++) last = j
|
|
80
|
+
self._selections.swap(i, last)
|
|
81
|
+
}
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
So distinct non-zero numbers give an order once and a round robin thereafter.
|
|
85
|
+
Two things hold: non-zero rotates fairly, zero is always last.
|
|
86
|
+
|
|
87
|
+
The ordering is therefore kept in `DemandRegister.levelsToState`, by choosing
|
|
88
|
+
what to state at all, and the library is given only the distinction it honours —
|
|
89
|
+
`1` for anything wanted now, `0` for the speculative tail.
|
|
90
|
+
|
|
91
|
+
The rotation is wanted, not merely tolerated: with two viewers of one film both
|
|
92
|
+
stopped, both needs sit at `BLOCKED` and the swarm alternates between them
|
|
93
|
+
instead of always serving whoever asked first.
|
|
94
|
+
|
|
95
|
+
## Why the speculative levels are withdrawn rather than lowered
|
|
96
|
+
|
|
97
|
+
A peer that cannot help with anything urgent — it lacks those pieces, or every
|
|
98
|
+
block of them is reserved by somebody else — falls through the selection list to
|
|
99
|
+
whatever is below. With a permanently low priority it would then spend the
|
|
100
|
+
shared link on pieces nobody is waiting for, about a second of its own
|
|
101
|
+
throughput at a time (`PIPELINE_MAX_DURATION = 1`).
|
|
102
|
+
|
|
103
|
+
A withdrawn window is not in the download set at all, so there is nothing to
|
|
104
|
+
fall through to.
|
|
105
|
+
|
|
106
|
+
The condition is **global**, in `services/download/registry.js`, and not per
|
|
107
|
+
torrent: two films on one proxy share the link, so filling the tail of one while
|
|
108
|
+
a viewer of the other has a still picture spends the same bandwidth twice over.
|
|
109
|
+
|
|
110
|
+
## `select` against `critical` — two different things
|
|
111
|
+
|
|
112
|
+
`select(from, to, priority)` decides **what is asked for next**: it inserts into
|
|
113
|
+
the sorted selection list the picker walks.
|
|
114
|
+
|
|
115
|
+
`critical(from, to)` decides **whom it is asked of**. Every block of a piece is
|
|
116
|
+
reserved to exactly one wire; `piece.reserve()` returns `-1` once they all are,
|
|
117
|
+
and a fast idle peer walks past. The flag lets `_hotswap` take a block from the
|
|
118
|
+
slowest holder and give it to the asker:
|
|
119
|
+
|
|
120
|
+
```js
|
|
121
|
+
if (reservation === -1 && hotswap && self._hotswap(wire, index)) {
|
|
122
|
+
reservation = piece.reserve()
|
|
123
|
+
}
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
Its thresholds are constants in the library, not settings: the asker must be
|
|
127
|
+
above 16 KB/s, the holder below 48 KB/s and at least twice as slow. So a holder
|
|
128
|
+
at 50 KB/s is never displaced, however long the piece has been waited for.
|
|
129
|
+
Whether that costs us anything is measured rather than assumed — `askFastestWiresFor`
|
|
130
|
+
counts the requests refused while every block was reserved, and the wait line
|
|
131
|
+
prints it. A number there would justify replacing `_hotswap` on the torrent
|
|
132
|
+
object; a zero says the thresholds are not what we are short of.
|
|
133
|
+
|
|
134
|
+
## Several viewers
|
|
135
|
+
|
|
136
|
+
- **Same file.** Each reader is its own claimant. Two stopped viewers put two
|
|
137
|
+
disjoint ranges at `BLOCKED`, and the library's rotation alternates between
|
|
138
|
+
them. Two readers wanting the same pieces are merged into one instruction.
|
|
139
|
+
- **Same torrent, different files.** The background fill is stated **per file**,
|
|
140
|
+
from the furthest window in that file to that file's end. It used to take the
|
|
141
|
+
furthest window across all files and the last piece across all files and claim
|
|
142
|
+
everything between — with two viewers on two episodes of one release, that
|
|
143
|
+
claimed every episode lying between them.
|
|
144
|
+
- **Different torrents.** One register and one selection each, and the
|
|
145
|
+
speculative condition spans all of them.
|
|
146
|
+
|
|
147
|
+
## What was deleted, and why
|
|
148
|
+
|
|
149
|
+
- `claimWindow`, `releaseWindow`, `markCritical`, `clearCritical` in
|
|
150
|
+
`piece-reader.js` — the reader no longer speaks to the library.
|
|
151
|
+
- `#reassertReaderWindows` in `torrent-pool.js` — it read the piece store's
|
|
152
|
+
MEMORY claims and rebuilt download claims from them, because WebTorrent
|
|
153
|
+
deletes a selection once satisfied. `SwarmSelection.reconcile` does that from
|
|
154
|
+
the register, which is where the statement lives.
|
|
155
|
+
- `#updateBackgroundFill` and `#tailAfterWindows` — replaced by a stated need at
|
|
156
|
+
the `TAIL` level, per file.
|
|
157
|
+
- `#syncSelections` — obsolete once nothing is selected by default.
|
|
158
|
+
- `setActiveFile` — dead: no caller anywhere in the repository.
|
|
159
|
+
- The two claim strategies and the environment variable that chose between them
|
|
160
|
+
(`TORRENT_TV_READ_MODE`). Each read was assigned at random to one of them and
|
|
161
|
+
the waits were sorted by which, and the comparison never decided anything: the
|
|
162
|
+
split halved the sample, so on 2026-08-28 there were nine reads in one arm and
|
|
163
|
+
three in the other against a threshold of ten, and on 2026-08-29 the two arms
|
|
164
|
+
printed together for the first and only time as forty waits against one. Waits
|
|
165
|
+
are now recorded by the LEVEL the reader was stopped in, which says whether a
|
|
166
|
+
band is too narrow rather than whether banding is the wrong idea.
|
|
167
|
+
|
|
168
|
+
## What this does NOT do
|
|
169
|
+
|
|
170
|
+
The piece store keeps its own list of protected ranges for memory
|
|
171
|
+
(`protectRange` / `protectedRanges`). It is fed by the same readers with the
|
|
172
|
+
same windows, but it is a second list, and one of the two could still drift from
|
|
173
|
+
the other. Making the store read this register instead is the remaining half of
|
|
174
|
+
the deduplication; it was left out of the first release because the memory path
|
|
175
|
+
had just been rewritten and had not yet been seen in the field.
|
package/docs/logs.md
CHANGED
|
@@ -7,6 +7,14 @@ All logs are visible via container `docker logs`, no browser console copy-paste
|
|
|
7
7
|
Image `b34a1737/aarch64-addon-torrent_tv_proxy:<version>` = `@torrent-tv/proxy` `<version>` (see `ha-addon/torrent_tv_proxy/config.yaml`).
|
|
8
8
|
|
|
9
9
|
- Console + file are the same: the proxy logs to both `stdout` and `/data/proxy.log` on start (`logging to /data/proxy.log as well as the console`).
|
|
10
|
+
- **Since 2.71.0 that is true of the torrent thread too, and it was not before.**
|
|
11
|
+
A worker thread loads its own instance of every module, so the logger's file
|
|
12
|
+
handle — set once, on the main thread — was null in the worker for the life of
|
|
13
|
+
the process. Measured over a whole 49 938-line file: zero lines from the piece
|
|
14
|
+
reader (`read "…"`, `supply "…"`, every wait and its cause) and zero from the
|
|
15
|
+
torrent pool, against 52 and 36 of them in `docker logs`, which every release
|
|
16
|
+
destroys. The worker now sends its lines to the main thread, which is the only
|
|
17
|
+
writer; a line from there reads `torrent-worker: …`.
|
|
10
18
|
- Via container:
|
|
11
19
|
```bash
|
|
12
20
|
ssh ha "sudo docker logs app_b34a1737_torrent_tv_proxy --tail 200"
|
package/package.json
CHANGED
|
@@ -0,0 +1,182 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @file What is wanted from one torrent right now, and by whom.
|
|
3
|
+
*
|
|
4
|
+
* **One statement, two consumers.** Until 2026-09-02 the same intent was
|
|
5
|
+
* written down twice and reconciled by a third piece of code. A reader told the
|
|
6
|
+
* piece store `protectRange(readerId, from, to)` so its bytes would not be
|
|
7
|
+
* evicted from memory; it separately told the torrent `select(from, to, 1)` so
|
|
8
|
+
* they would be fetched; and the pool then read the store's memory claims every
|
|
9
|
+
* few seconds to rebuild the download claims WebTorrent had dropped once
|
|
10
|
+
* satisfied. One want, three places, and a sweeper between two of them.
|
|
11
|
+
*
|
|
12
|
+
* Here it is stated once. Memory reads this to decide what to keep resident;
|
|
13
|
+
* the swarm layer reads it to decide what to ask peers for. Neither owns it and
|
|
14
|
+
* neither can drift from the other.
|
|
15
|
+
*
|
|
16
|
+
* Knows nothing about WebTorrent, nothing about the piece store, and nothing
|
|
17
|
+
* about pieces — so it is testable with numbers alone.
|
|
18
|
+
*/
|
|
19
|
+
|
|
20
|
+
import { isConditional, Urgency, URGENCY_ORDER } from "./Urgency.js";
|
|
21
|
+
import { unionOf, Window } from "./Window.js";
|
|
22
|
+
|
|
23
|
+
export class DemandRegister {
|
|
24
|
+
/** Claimant → the one window it currently states. */
|
|
25
|
+
#windows = new Map();
|
|
26
|
+
|
|
27
|
+
/**
|
|
28
|
+
* State a need, replacing whatever that claimant said before.
|
|
29
|
+
*
|
|
30
|
+
* Replacing rather than adding is the point: a reader walking a film restates
|
|
31
|
+
* a moving window many times a second, and if those accumulated the download
|
|
32
|
+
* set would grow to the whole file within a minute.
|
|
33
|
+
*
|
|
34
|
+
* @param {object} params
|
|
35
|
+
* @param {string} params.claimant
|
|
36
|
+
* @param {number} params.fileIndex
|
|
37
|
+
* @param {number} params.byteStart
|
|
38
|
+
* @param {number} params.byteEnd
|
|
39
|
+
* @param {number} params.urgency
|
|
40
|
+
* @returns {Window}
|
|
41
|
+
*/
|
|
42
|
+
state({ claimant, fileIndex, byteStart, byteEnd, urgency }) {
|
|
43
|
+
const window = new Window({ claimant, fileIndex, byteStart, byteEnd, urgency });
|
|
44
|
+
this.#windows.set(claimant, window);
|
|
45
|
+
return window;
|
|
46
|
+
}
|
|
47
|
+
|
|
48
|
+
/**
|
|
49
|
+
* Withdraw what a claimant said.
|
|
50
|
+
*
|
|
51
|
+
* Named by claimant, so a release that arrives twice, or after the claimant
|
|
52
|
+
* has gone, withdraws nothing rather than somebody else's window. Must be
|
|
53
|
+
* called from a `finally`: a reader that ends without withdrawing keeps the
|
|
54
|
+
* swarm fetching for somebody who is no longer there.
|
|
55
|
+
*
|
|
56
|
+
* @param {string} claimant
|
|
57
|
+
* @returns {boolean} Whether there was anything to withdraw.
|
|
58
|
+
*/
|
|
59
|
+
withdraw(claimant) {
|
|
60
|
+
return this.#windows.delete(claimant);
|
|
61
|
+
}
|
|
62
|
+
|
|
63
|
+
/** How many claimants are stating something. */
|
|
64
|
+
get size() {
|
|
65
|
+
return this.#windows.size;
|
|
66
|
+
}
|
|
67
|
+
|
|
68
|
+
/**
|
|
69
|
+
* Every stated window, most urgent first.
|
|
70
|
+
*
|
|
71
|
+
* @returns {Window[]}
|
|
72
|
+
*/
|
|
73
|
+
windows() {
|
|
74
|
+
return [...this.#windows.values()].sort((left, right) => left.urgency - right.urgency);
|
|
75
|
+
}
|
|
76
|
+
|
|
77
|
+
/**
|
|
78
|
+
* The windows at one level of urgency.
|
|
79
|
+
*
|
|
80
|
+
* @param {number} urgency
|
|
81
|
+
* @returns {Window[]}
|
|
82
|
+
*/
|
|
83
|
+
at(urgency) {
|
|
84
|
+
return this.windows().filter((window) => window.urgency === urgency);
|
|
85
|
+
}
|
|
86
|
+
|
|
87
|
+
/**
|
|
88
|
+
* The union of what is wanted of one file at one level, in byte ranges.
|
|
89
|
+
*
|
|
90
|
+
* @param {number} fileIndex
|
|
91
|
+
* @param {number} urgency
|
|
92
|
+
* @returns {Array<{ byteStart: number, byteEnd: number }>}
|
|
93
|
+
*/
|
|
94
|
+
unionFor(fileIndex, urgency) {
|
|
95
|
+
return unionOf(
|
|
96
|
+
this.windows().filter((window) => window.fileIndex === fileIndex && window.urgency === urgency)
|
|
97
|
+
);
|
|
98
|
+
}
|
|
99
|
+
|
|
100
|
+
/**
|
|
101
|
+
* The union of everything wanted of one file, whatever the urgency.
|
|
102
|
+
*
|
|
103
|
+
* This is what memory is sized against: a piece inside any stated window will
|
|
104
|
+
* be read, and evicting it means fetching it back from disk moments later.
|
|
105
|
+
*
|
|
106
|
+
* @param {number} [fileIndex] - Every file when omitted.
|
|
107
|
+
* @returns {Array<{ byteStart: number, byteEnd: number }>}
|
|
108
|
+
*/
|
|
109
|
+
union(fileIndex) {
|
|
110
|
+
const wanted = fileIndex === undefined
|
|
111
|
+
? this.windows()
|
|
112
|
+
: this.windows().filter((window) => window.fileIndex === fileIndex);
|
|
113
|
+
// Ranges of different files must not be merged: byte 100 of file 0 and byte
|
|
114
|
+
// 100 of file 1 are different bytes.
|
|
115
|
+
const byFile = new Map();
|
|
116
|
+
for (const window of wanted) {
|
|
117
|
+
const list = byFile.get(window.fileIndex) ?? [];
|
|
118
|
+
list.push(window);
|
|
119
|
+
byFile.set(window.fileIndex, list);
|
|
120
|
+
}
|
|
121
|
+
return [...byFile.values()].flatMap((list) => unionOf(list));
|
|
122
|
+
}
|
|
123
|
+
|
|
124
|
+
/** Which files anybody is stating a need for. */
|
|
125
|
+
files() {
|
|
126
|
+
return [...new Set(this.windows().map((window) => window.fileIndex))].sort((a, b) => a - b);
|
|
127
|
+
}
|
|
128
|
+
|
|
129
|
+
/**
|
|
130
|
+
* The most urgent level anybody is stating, or null when nothing is stated.
|
|
131
|
+
*
|
|
132
|
+
* @returns {number | null}
|
|
133
|
+
*/
|
|
134
|
+
mostUrgent() {
|
|
135
|
+
const windows = this.windows();
|
|
136
|
+
return windows.length === 0 ? null : windows[0].urgency;
|
|
137
|
+
}
|
|
138
|
+
|
|
139
|
+
/**
|
|
140
|
+
* Which levels should be stated to the swarm, given what is still missing.
|
|
141
|
+
*
|
|
142
|
+
* The three urgent levels always. The two speculative ones only while
|
|
143
|
+
* everything above them is complete — and withdrawn whole the moment one is
|
|
144
|
+
* not, because a withdrawn window is not in the download set at all and a
|
|
145
|
+
* peer with nothing urgent to give cannot fall through to it. That is the
|
|
146
|
+
* difference from a permanently low priority, which is exactly what lets a
|
|
147
|
+
* peer fall through and spend the shared link on a piece nobody is waiting
|
|
148
|
+
* for.
|
|
149
|
+
*
|
|
150
|
+
* @param {(window: Window) => boolean} isSatisfied - Whether a window has
|
|
151
|
+
* everything it asked for. Given from outside because only the torrent
|
|
152
|
+
* knows what has arrived.
|
|
153
|
+
* @param {boolean} [speculativeAllowed] - Whether anything ELSEWHERE is still
|
|
154
|
+
* waiting for something urgent. False holds the speculative levels back
|
|
155
|
+
* even when this torrent has everything it needs: two films on one proxy
|
|
156
|
+
* share the link, so filling the tail of one while a viewer of the other
|
|
157
|
+
* has a still picture spends the same bandwidth twice over.
|
|
158
|
+
* @returns {number[]} Levels to state, most urgent first.
|
|
159
|
+
*/
|
|
160
|
+
levelsToState(isSatisfied, speculativeAllowed = true) {
|
|
161
|
+
const stating = [];
|
|
162
|
+
let everythingAboveIsSatisfied = speculativeAllowed;
|
|
163
|
+
for (const urgency of URGENCY_ORDER) {
|
|
164
|
+
if (isConditional(urgency) && !everythingAboveIsSatisfied) {
|
|
165
|
+
break;
|
|
166
|
+
}
|
|
167
|
+
stating.push(urgency);
|
|
168
|
+
const level = this.at(urgency);
|
|
169
|
+
if (level.some((window) => !isSatisfied(window))) {
|
|
170
|
+
everythingAboveIsSatisfied = false;
|
|
171
|
+
}
|
|
172
|
+
}
|
|
173
|
+
return stating;
|
|
174
|
+
}
|
|
175
|
+
|
|
176
|
+
/** Forget everything. The torrent is going. */
|
|
177
|
+
clear() {
|
|
178
|
+
this.#windows.clear();
|
|
179
|
+
}
|
|
180
|
+
}
|
|
181
|
+
|
|
182
|
+
export { Urgency };
|
|
@@ -0,0 +1,137 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @file How urgent a stated need is, and what follows from that.
|
|
3
|
+
*
|
|
4
|
+
* Five levels, ordered. The ordering is not a set of numbers handed to
|
|
5
|
+
* WebTorrent — measured 2026-09-02, the library does not keep them. Its picker
|
|
6
|
+
* sorts selections by priority only at the moment one is inserted, and then, on
|
|
7
|
+
* every wire it fills, moves the selection it just served to the BACK of the
|
|
8
|
+
* whole group with a non-zero priority:
|
|
9
|
+
*
|
|
10
|
+
* ```js
|
|
11
|
+
* function shufflePriority (i) {
|
|
12
|
+
* let last = i
|
|
13
|
+
* for (let j = i; j < self._selections.length && self._selections.get(j).priority; j++) last = j
|
|
14
|
+
* self._selections.swap(i, last)
|
|
15
|
+
* }
|
|
16
|
+
* ```
|
|
17
|
+
*
|
|
18
|
+
* So distinct non-zero numbers give an order once and a round robin thereafter.
|
|
19
|
+
* Only two things hold: non-zero rotates fairly, zero is always last.
|
|
20
|
+
*
|
|
21
|
+
* Therefore urgency here is expressed by WHETHER a need is stated to the swarm
|
|
22
|
+
* at all. A level is stated only while every level above it is satisfied, and
|
|
23
|
+
* withdrawn the moment one is not — a withdrawn need is not in the download set
|
|
24
|
+
* at all, so a peer with nothing urgent to give cannot fall through to it and
|
|
25
|
+
* spend the shared link on it. A permanently low priority does exactly that,
|
|
26
|
+
* which is why it is not what this does.
|
|
27
|
+
*
|
|
28
|
+
* The rotation is not merely tolerated, it is wanted: with two viewers of one
|
|
29
|
+
* film both stopped, both their needs sit at {@link Urgency.BLOCKED} and the
|
|
30
|
+
* swarm alternates between them instead of always serving whoever asked first.
|
|
31
|
+
*/
|
|
32
|
+
|
|
33
|
+
/**
|
|
34
|
+
* The levels, most urgent first. The value is the position, so `<` compares
|
|
35
|
+
* urgency and nothing else may be read into it.
|
|
36
|
+
*
|
|
37
|
+
* @enum {number}
|
|
38
|
+
*/
|
|
39
|
+
export const Urgency = {
|
|
40
|
+
/**
|
|
41
|
+
* The bytes a reader is stopped on. The picture is not moving until they
|
|
42
|
+
* arrive.
|
|
43
|
+
*/
|
|
44
|
+
BLOCKED: 0,
|
|
45
|
+
/**
|
|
46
|
+
* The rest of that reader's own window — the cushion being built. Late here
|
|
47
|
+
* and the picture stops in a few seconds.
|
|
48
|
+
*/
|
|
49
|
+
NEAR: 1,
|
|
50
|
+
/**
|
|
51
|
+
* Further along the file, as far as the encoder will reach. Late here costs
|
|
52
|
+
* nothing yet.
|
|
53
|
+
*/
|
|
54
|
+
AHEAD: 2,
|
|
55
|
+
/**
|
|
56
|
+
* To the end of the file. Wanted for certain if the viewer watches on, and
|
|
57
|
+
* not before anything above is satisfied.
|
|
58
|
+
*/
|
|
59
|
+
TAIL: 3,
|
|
60
|
+
/**
|
|
61
|
+
* The gap behind the playhead, left by a forward seek. Wanted only if the
|
|
62
|
+
* viewer seeks back, so it is last — but it is real: a backward seek into a
|
|
63
|
+
* hole is the longest wait this proxy produces (field: 93 s).
|
|
64
|
+
*/
|
|
65
|
+
BEHIND: 4
|
|
66
|
+
};
|
|
67
|
+
|
|
68
|
+
/** Every level, most urgent first. */
|
|
69
|
+
export const URGENCY_ORDER = [
|
|
70
|
+
Urgency.BLOCKED,
|
|
71
|
+
Urgency.NEAR,
|
|
72
|
+
Urgency.AHEAD,
|
|
73
|
+
Urgency.TAIL,
|
|
74
|
+
Urgency.BEHIND
|
|
75
|
+
];
|
|
76
|
+
|
|
77
|
+
/**
|
|
78
|
+
* Whether a level may take a block away from a slow peer and give it to a fast
|
|
79
|
+
* one — WebTorrent's `critical`.
|
|
80
|
+
*
|
|
81
|
+
* Only the level that is being waited on. Displacement throws away the part of
|
|
82
|
+
* the block the slow peer had already fetched, so it buys time exactly where
|
|
83
|
+
* time is what is short, and wastes bandwidth everywhere else.
|
|
84
|
+
*
|
|
85
|
+
* @param {number} urgency
|
|
86
|
+
* @returns {boolean}
|
|
87
|
+
*/
|
|
88
|
+
export function mayDisplaceSlowPeer(urgency) {
|
|
89
|
+
return urgency === Urgency.BLOCKED;
|
|
90
|
+
}
|
|
91
|
+
|
|
92
|
+
/**
|
|
93
|
+
* Whether a level is stated to the swarm only while the levels above it want
|
|
94
|
+
* nothing.
|
|
95
|
+
*
|
|
96
|
+
* The three urgent levels are always stated: they are what the viewer is
|
|
97
|
+
* waiting for. The two speculative ones are stated only in the room the urgent
|
|
98
|
+
* ones leave.
|
|
99
|
+
*
|
|
100
|
+
* @param {number} urgency
|
|
101
|
+
* @returns {boolean}
|
|
102
|
+
*/
|
|
103
|
+
export function isConditional(urgency) {
|
|
104
|
+
return urgency >= Urgency.TAIL;
|
|
105
|
+
}
|
|
106
|
+
|
|
107
|
+
/**
|
|
108
|
+
* The number handed to `torrent.select`.
|
|
109
|
+
*
|
|
110
|
+
* Two values only, because two is all the library keeps: everything urgent
|
|
111
|
+
* shares one non-zero priority and rotates fairly among itself; the
|
|
112
|
+
* speculative levels take zero, which the library always places last and never
|
|
113
|
+
* rotates into the group above.
|
|
114
|
+
*
|
|
115
|
+
* @param {number} urgency
|
|
116
|
+
* @returns {number}
|
|
117
|
+
*/
|
|
118
|
+
export function selectionPriority(urgency) {
|
|
119
|
+
return isConditional(urgency) ? 0 : 1;
|
|
120
|
+
}
|
|
121
|
+
|
|
122
|
+
/**
|
|
123
|
+
* A level's name, for the log.
|
|
124
|
+
*
|
|
125
|
+
* @param {number} urgency
|
|
126
|
+
* @returns {string}
|
|
127
|
+
*/
|
|
128
|
+
export function urgencyName(urgency) {
|
|
129
|
+
switch (urgency) {
|
|
130
|
+
case Urgency.BLOCKED: return "blocked";
|
|
131
|
+
case Urgency.NEAR: return "near";
|
|
132
|
+
case Urgency.AHEAD: return "ahead";
|
|
133
|
+
case Urgency.TAIL: return "tail";
|
|
134
|
+
case Urgency.BEHIND: return "behind";
|
|
135
|
+
default: return `urgency-${urgency}`;
|
|
136
|
+
}
|
|
137
|
+
}
|