@torrent-tv/proxy 2.83.3 → 2.83.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +19 -0
- package/package.json +1 -1
- package/research/handover-reader-claims-removal-2026-09-12.md +166 -0
- package/research/priority-map-is-the-truth-2026-09-12.md +207 -0
- package/routes/api/sources/files/get.js +23 -9
- package/routes/api/sources/warm/post.js +26 -22
- package/routes/api/transcode-sessions/post.js +1 -49
- package/routes/stream/get.js +2 -36
- package/services/controllers/SubtitleController.js +0 -3
- package/services/hls-session-manager.js +3 -68
- package/services/playback-planner.js +12 -13
- package/services/priority/PriorityOrchestrator.js +40 -6
- package/services/torrent/Contents.js +324 -0
- package/services/torrent/files.js +8 -1
- package/services/torrent-pool.js +302 -100
- package/services/torrent-worker/client.js +0 -24
- package/services/torrent-worker/pool-adapter.js +3 -33
- package/services/torrent-worker/protocol.js +0 -4
- package/services/torrent-worker/worker.js +52 -59
- package/test/file-edges.test.js +191 -0
- package/test/priority-map-emptied.test.js +207 -0
- package/test/source-files-route.test.js +103 -0
- package/test/stream-route.test.js +4 -8
- package/test/swarm-follows-readers.test.js +96 -14
- package/test/torrent-contents.test.js +235 -0
- package/test/upload-hurry.test.js +66 -28
- package/services/torrent-worker/file-claims.js +0 -91
- package/test/file-claims.test.js +0 -64
package/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,22 @@
|
|
|
1
|
+
## 2.83.4
|
|
2
|
+
- **Chore**: WHAT IS IN A TORRENT IS NOW A THING RATHER THAN A RECOMPUTATION. `services/torrent/Contents.js` holds one torrent's list of files, its pictures with the sound, subtitles and images that belong to each, the order a person reads them in, and what belongs to nothing. The pairing rules were already there as functions; what did not exist was anywhere to keep their answer, so the warm-up on one path and the playback plan on another worked out the same grouping over the same list several times per opened film. Checked against the survey collection: 134 torrents, 44 of them with more than one picture, 1357 pictures and 421 files paired to one, not a single file paired to two — and all 1357 groupings agree with the per-picture call they replace, so nothing about the pairing changes.
|
|
3
|
+
- **Chore**: It holds only what the torrent says about itself. Nothing about a FILM: a poster or a title is an answer from a third party about an identity read from a file's own bytes, keyed by the picture's file, and it must never delay playback. No reference to another layer either — not a container, not a priority map, not a viewer; what leaves is plain indices into the torrent's own list.
|
|
4
|
+
- **Chore**: Telling a real episode from an extra was measured and deliberately not built. Over those 134 torrents the words that would say so (`sample`, `trailer`, `extra`, `bonus`, `preview`, `making`) matched four files and all four were ordinary titles; the ratio of a video to the median of its own torrent runs continuously from 0.00 to 1.00 with no gap, because a collection of short clips is made of short clips. There is nothing to derive a rule from, and an invented one would decide the order in which a pool owner's bandwidth is spent.
|
|
5
|
+
- **Fix**: THE MAP NOW SAYS WHEN NOBODY WANTS A FILE ANY MORE. It could make only one of its two statements: a file whose viewers had all gone was deleted from the orchestrator's own memory and published nowhere, so the bands it had stated on the swarm's behalf stood until the torrent itself was removed. Read from the demand register — which is what decides what is fetched — a film nobody had watched for an hour was indistinguishable from one being watched now. A file with nobody on it is now published with nothing in it, and so is a file whose session has gone, once.
|
|
6
|
+
- **Fix**: A map with nothing in it is answered before the file's length, its duration or the torrent's list are looked at. All three are what the ordinary path needs to turn seconds into bytes, and all three are on their way out at exactly the moment a departure is said, so waiting on them would make the departure depend on facts that are already gone.
|
|
7
|
+
- **Fix**: An empty map cannot bring a torrent back. It is said when the last viewer of a file leaves, which is also when the torrent may be going, and the worker's ordinary lookup rebuilds a dead handle from its recipe — so asking that way would have re-added a torrent in order to be told that nothing is wanted of it.
|
|
8
|
+
- **Fix**: THE TWO ENDS OF A FILE ARE THE FILE'S, NOT A READ'S. A container keeps its directory at one end or the other — `ftyp` and an EBML header at the front, the `moov` at the very back of an MP4 that was not written for streaming — and nothing can be read of such a file until they arrive. They were wanted by nobody: the prefetch that fetches them is an ordinary read, and a read gives up what it stated the moment it finishes, so the ends of an open film were held by nothing at all once the codec probe was done. They are stated by the file now, urgent while the playback plan is waiting for them and kept afterwards at the level of something nobody is waiting for, and they are given up when the map says nobody wants that file any more.
|
|
9
|
+
- **Chore**: The size of that claim is not chosen and cannot be: one byte at each end, and since the piece is what a swarm delivers, one byte at each end IS one piece at each end. The 256 KB and 2 MB the prefetch reads are its own affair, and a directory bigger than the piece it starts in is asked for by the read that needs it.
|
|
10
|
+
- **New**: A torrent whose episode nobody has chosen yet warms its FIRST item — that picture and the sound and subtitles beside it — instead of nothing. One item, not twenty: a pack's worth of edges would spend the pool owner's bandwidth on nineteen files nobody opened, while one item is two pieces and is also the likeliest pick. It is stated as something nobody is waiting for, so on a proxy serving somebody else it costs nothing until their own film has everything it needs.
|
|
11
|
+
- **Fix**: NOTHING COUNTS READERS ANY MORE. The pool asked how many claims a torrent's files had, and that count decided whether it kept its swarm, whether it was removed, whether it was evicted under the disk cap, whether it could be seeded for and whether a complete file could be written out. It was a second bookkeeping of a fact the demand register already holds, and it was wrong in both directions: a torrent being opened has no readers, and a viewer who has left can leave one behind. One question is asked instead — is anything wanted of this torrent — and it is answered by what has been stated.
|
|
12
|
+
- **Fix**: A swarm is let go on a DEPARTURE and never on a beginning. A torrent nothing has ever been wanted of is one being opened, its metadata just landed and its file list on its way to the person choosing, and taking its swarm away there is the 2.83.1 failure down to the 741 connections. The rule is two facts and a table, separated from the doing so it can be exercised without a live client.
|
|
13
|
+
- **Fix**: A torrent is on the idle clock from the moment it exists. The only thing that ever started that clock was a reader letting go, so a file list fetched and never played was held for the life of the process.
|
|
14
|
+
- **Chore**: `acquireFile`, its release, the two worker commands behind it and the module that kept the claims are gone, with all seven call sites: the stream route, the transcode-session route, the subtitle controller, the read inside the worker, and the session manager's hold on its source. The session manager is 65 lines shorter for it.
|
|
15
|
+
- **New**: WHAT IS IN A TORRENT IS ANSWERED HERE, AND NOWHERE ELSE. `GET /api/sources/:key/files` returns the files in the order a person reads them — by folder, then by name, with runs of digits compared as numbers — each saying what it is, together with the pictures and what belongs to each of them by index. The browser used to decide all of that itself, twice: a list of video extensions in its torrent parser and a second, shorter pair inside its picker. The three had already diverged — measured 2026-09-12, `.dat` was offered there as video and not counted here, which also decides whether a sidecar whose name matches nothing can belong to the only video present. That one disagreement is settled by union: `.dat` is a Video CD's MPEG-1 stream, and a file offered and found unplayable costs one probe, while a film never offered cannot be reached at all.
|
|
16
|
+
- **Chore**: The paths leave here already relative to the torrent's root, so the rule for stripping the torrent's own name lives in one place instead of two that can disagree.
|
|
17
|
+
- **Fix**: A warm-up can no longer take the urgency away from a playback plan that somebody is waiting for. Both ask for the same two ends of a file, and the warm-up fires off the files beside the picture without awaiting them, so it can arrive second and restate the same claimants at its own lower level — putting a person on a loading screen behind a film somebody else is watching. The statement is raise-only; the caller entitled to put them back down is the read that has just finished, and it says so.
|
|
18
|
+
- **Fix**: A departure could be noticed by nobody. The background fill states a claim per file and withdraws it once nothing else wants that file, but the withdrawal happened inside a pass whose result nothing acted on — so a torrent everybody had left could be held "wanted" by the fill's own claim at the moment of the departure and then never re-examined: in its swarm, with no idle clock running.
|
|
19
|
+
|
|
1
20
|
## 2.83.3
|
|
2
21
|
|
|
3
22
|
- **Fix**: LEAVING A SWARM IS NOW A CONSEQUENCE OF A DEPARTURE, not something a pass over every torrent decides. 2.83.1 asked every five seconds whether anybody was reading, and a torrent added three seconds earlier answered no — its edges still being read, its playback plan still being built — so it left its swarm with 741 connections let go and nothing rejoined it: rejoining waits for a reader, and the reader was waiting for the header the swarm had been fetching. 2.83.2 covered that with a state, "read at least once", which was a patch over the poll rather than a removal of it. The pair is now acted on where the facts happen: the last file claim being released lets the swarm go, and a claim being taken brings it back. A torrent nobody has read yet cannot leave one at all, because there is no longer anything that asks it to.
|
package/package.json
CHANGED
|
@@ -0,0 +1,166 @@
|
|
|
1
|
+
# Handover: the priority map is the only thing that says a file is wanted
|
|
2
|
+
|
|
3
|
+
Written 2026-09-12, after the instruction that reversed the work in the tree.
|
|
4
|
+
|
|
5
|
+
## The instruction, in the user's own terms
|
|
6
|
+
|
|
7
|
+
The priority map answers ONE question: how much is this stretch of this file
|
|
8
|
+
wanted. Nothing anywhere asks WHO is reading a torrent. There are no reader
|
|
9
|
+
counters. Everything built around counting readers comes out.
|
|
10
|
+
|
|
11
|
+
The map is rebuilt when viewers come and go, so it already shows the current
|
|
12
|
+
state of affairs. A second bookkeeping of the same thing is what has to go.
|
|
13
|
+
|
|
14
|
+
## What is in the working tree right now, and why it is all wrong
|
|
15
|
+
|
|
16
|
+
HEAD is `79e802a`. On top of it, UNCOMMITTED:
|
|
17
|
+
|
|
18
|
+
| file | what was done to it |
|
|
19
|
+
|---|---|
|
|
20
|
+
| `services/torrent/ReaderClaims.js` | NEW, untracked. The whole class is the mistake. |
|
|
21
|
+
| `services/torrent-pool.js` | `fileUsageByTorrent` replaced by `this.readers = new ReaderClaims()`; a listener on it calls `rejoinSwarm`, `leaveSwarm` and `#scheduleIdleRemoval`; `torrentsForUploadPolicy` changed from taking a usage map to taking an `isRead` predicate. |
|
|
22
|
+
| `services/torrent-worker/worker.js` | `warmActiveFiles` (`:842`) and `keepWholeFiles` (`:1024`) read `pool.readers.of(torrent)`. |
|
|
23
|
+
| `test/swarm-follows-readers.test.js` | the announcement test rewritten around `ReaderClaims`. |
|
|
24
|
+
| `test/upload-hurry.test.js` | rewritten around the `isRead` predicate. |
|
|
25
|
+
|
|
26
|
+
All of it counts readers. All of it comes out. Two further facts about this
|
|
27
|
+
tree, both of which the next agent needs:
|
|
28
|
+
|
|
29
|
+
1. A bulk text replacement earlier in that session DESTROYED three methods of
|
|
30
|
+
`TorrentPool` — `#scheduleIdleRemoval`, `#cancelIdleRemoval` and
|
|
31
|
+
`#forgetDeadTorrent` — which were then called but not defined. They were
|
|
32
|
+
restored BY HAND from HEAD and adapted, not diffed line by line against the
|
|
33
|
+
original. One log line was changed deliberately: the idle-timer message no
|
|
34
|
+
longer prints a reader count. **Diff the restored bodies against
|
|
35
|
+
`git show HEAD:services/torrent-pool.js` before trusting them**, or discard
|
|
36
|
+
the tree and start from HEAD.
|
|
37
|
+
2. Discarding the uncommitted changes is probably the cheapest start, since
|
|
38
|
+
every one of them is being reversed. That is a destructive action and is the
|
|
39
|
+
USER'S CALL — ask before running anything that throws work away.
|
|
40
|
+
|
|
41
|
+
## What already exists and answers the question without counting anybody
|
|
42
|
+
|
|
43
|
+
Read these before designing. Line numbers are of the working tree.
|
|
44
|
+
|
|
45
|
+
**The map is built from viewers and published.** `services/priority/PriorityOrchestrator.js`
|
|
46
|
+
builds one map per film+file, and a second per output, from where the viewers
|
|
47
|
+
are. `services/priority/PriorityMap.js` holds the scale and the words for
|
|
48
|
+
reading it — `isAtAWatchingViewer`, `isBehindEverybody`, `isNobodyComingNow`.
|
|
49
|
+
|
|
50
|
+
**The map reaches the torrent thread as a command.** `Command.PRIORITY_MAP` in
|
|
51
|
+
`services/torrent-worker/worker.js:539` calls `pool.applyPriorityMap(torrent,
|
|
52
|
+
fileIndex, zones, durationSeconds)`.
|
|
53
|
+
|
|
54
|
+
**`applyPriorityMap` (`services/torrent-pool.js:1191`) is where seconds become
|
|
55
|
+
bytes**, and it writes the result into the demand register:
|
|
56
|
+
`demandFor(torrent).register.state({ claimant, fileIndex, byteStart, byteEnd,
|
|
57
|
+
urgency })`, with the claimant named `${MAP_CLAIMANT}:${fileIndex}:${index}`
|
|
58
|
+
(`:1257`). It WITHDRAWS its own stale claimants for that file afterwards
|
|
59
|
+
(`:1266-1273`), so a map that shrinks does shrink the register. **Whether a map
|
|
60
|
+
that wants NOTHING leaves the register empty for that file is not established —
|
|
61
|
+
check it, do not assume it.**
|
|
62
|
+
|
|
63
|
+
**The register is therefore the existing statement of what is wanted.**
|
|
64
|
+
`services/demand/DemandRegister.js` offers `size`, `files()`, `windows()`,
|
|
65
|
+
`at(urgency)`, `union(fileIndex)`, `mostUrgent()`, `withdraw(claimant)`,
|
|
66
|
+
`clear()`. `services/download/registry.js` offers `demandFor(torrent)`,
|
|
67
|
+
`hasUnmetDemand(torrent)`, `forgetTorrent(torrent)`, `reconcileAll()`.
|
|
68
|
+
|
|
69
|
+
## THE TRAP — read this before writing a line
|
|
70
|
+
|
|
71
|
+
**Not every claimant in the register is a viewer, and the ones that are not are
|
|
72
|
+
exactly what a naive rule kills.**
|
|
73
|
+
|
|
74
|
+
Measured claimants other than the map:
|
|
75
|
+
|
|
76
|
+
- `services/torrent-worker/piece-reader.js:670` — a read that is STOPPED on a
|
|
77
|
+
piece right now, stated at the BLOCKED level. Only a read can say this; the
|
|
78
|
+
map deliberately never states BLOCKED (see the comment at
|
|
79
|
+
`torrent-pool.js:1204`).
|
|
80
|
+
- `services/torrent-pool.js:1147`, `:1159`, `:1162` — the background fill and
|
|
81
|
+
the tail.
|
|
82
|
+
|
|
83
|
+
And several reads exist that no viewer has yet asked for: the codec probe
|
|
84
|
+
reading the file's head and tail, the keyframe table, the subtitle cluster
|
|
85
|
+
walk, a sidecar soundtrack being fetched whole.
|
|
86
|
+
|
|
87
|
+
**This is not hypothetical.** On 2026-09-11 a rule of exactly this shape shipped
|
|
88
|
+
as proxy 2.83.1: a torrent nobody had stated anything about left its swarm. A
|
|
89
|
+
torrent that has just been added has stated nothing yet, because its edges are
|
|
90
|
+
still being read and its playback plan is still being built. Field record:
|
|
91
|
+
added 20:57:17, first peer 20:57:18, out of the swarm 20:57:21 with 741
|
|
92
|
+
connections let go, and nothing ever rejoined it — rejoining waited for a
|
|
93
|
+
reader, and the reader was waiting for the header the swarm had been fetching.
|
|
94
|
+
`peers=0 connected of 0 known (94 queued, 5 tracker(s) offered up to 683
|
|
95
|
+
seeders)` for as long as the viewer would wait. Playback never started.
|
|
96
|
+
|
|
97
|
+
So "the map wants nothing, therefore let the swarm go" is WRONG as written. The
|
|
98
|
+
honest question is whether ANYTHING in the register wants bytes of this torrent
|
|
99
|
+
— the map's zones and every other claimant together — and that question has to
|
|
100
|
+
be asked of the register, which already knows, rather than of a counter.
|
|
101
|
+
|
|
102
|
+
## What has to change, and what each site should ask instead
|
|
103
|
+
|
|
104
|
+
Each of these currently asks "who is reading". Work out, per site, what it
|
|
105
|
+
actually needs from the register.
|
|
106
|
+
|
|
107
|
+
1. **`TorrentPool.acquireFile`** (`torrent-pool.js:1770`) and its release
|
|
108
|
+
function. Callers: `routes/stream/get.js:211`,
|
|
109
|
+
`routes/api/transcode-sessions/post.js:47`,
|
|
110
|
+
`services/controllers/SubtitleController.js:54`,
|
|
111
|
+
`services/torrent-worker/worker.js:251` and `:366`,
|
|
112
|
+
`services/torrent-worker/pool-adapter.js:88`,
|
|
113
|
+
`services/torrent-worker/client.js:327`. Note it also does two things that
|
|
114
|
+
are NOT counting: it cancels a pending idle removal and stamps
|
|
115
|
+
`#lastAccess`, which the disk-cap LRU reads. Decide where those two belong
|
|
116
|
+
before deleting the method.
|
|
117
|
+
2. **Leaving and rejoining the swarm.** `leaveSwarm` and `rejoinSwarm` are
|
|
118
|
+
exported from `torrent-pool.js:465` and `:497`. Today the trigger is the
|
|
119
|
+
first claim taken and the last released. It becomes a fact of the register.
|
|
120
|
+
3. **Idle removal.** `#scheduleIdleRemoval` fires on the last reader leaving and
|
|
121
|
+
re-checks for readers when the timer fires.
|
|
122
|
+
4. **The upload policy.** `torrentsForUploadPolicy` (`torrent-pool.js:349`) sets
|
|
123
|
+
`torrent.hasActiveReader` and `torrent.hasUnmetDemand`. The second already
|
|
124
|
+
comes from the register (`hasUnmetDemand`); the first is the counter.
|
|
125
|
+
5. **`#reportStalledDownloads`** (`:1311`) and **`#adjustUploadLimit`** (`:1354`)
|
|
126
|
+
skip a torrent with no readers.
|
|
127
|
+
6. **The disk-cap LRU** (`:1411`) picks torrents with no readers.
|
|
128
|
+
7. **`#removeTorrent`** (`:1886`) prints a refcount for diagnostics.
|
|
129
|
+
8. **`warmActiveFiles`** (`worker.js:842`) walks "the files being read" to keep
|
|
130
|
+
subtitle cues current. `register.files()` says which files anything wants.
|
|
131
|
+
9. **`keepWholeFiles`** (`worker.js:1024`) refuses to write a completed file out
|
|
132
|
+
while it is being read, so that the write does not take the disk from a
|
|
133
|
+
viewer. This one genuinely needs "is anybody reading THIS file right now" and
|
|
134
|
+
is the site most likely to need a different answer from the register than a
|
|
135
|
+
plain "is it wanted".
|
|
136
|
+
|
|
137
|
+
## Rules that hold whatever the design turns out to be
|
|
138
|
+
|
|
139
|
+
- **Plan first, approval before any edit.** No exceptions, including when told
|
|
140
|
+
to get on with it.
|
|
141
|
+
- **Never add a line to `services/hls-session-manager.js`.** It is being taken
|
|
142
|
+
apart. Deletion only; `git diff --numstat` must show more removed than added.
|
|
143
|
+
- **All code, comments and documents in the repositories are English.**
|
|
144
|
+
- **Do not run the whole test suite.** Run the tests of the files changed.
|
|
145
|
+
Relevant here: `test/swarm-follows-readers.test.js`, `test/upload-hurry.test.js`,
|
|
146
|
+
`test/file-claims.test.js`, `test/background-fill.test.js`,
|
|
147
|
+
`test/priority-map-download.test.js`, `test/read-window.test.js`,
|
|
148
|
+
`test/piece-reader.test.js`, `test/dead-torrent-handle.test.js`.
|
|
149
|
+
- **A CHANGELOG entry is part of the change**, written at the version
|
|
150
|
+
`npm run patch` will produce — the current `package.json` version plus one
|
|
151
|
+
patch. Never edit the version in `package.json` by hand.
|
|
152
|
+
- **Bumping the proxy means bumping the addon**, `ha-addon/torrent_tv_proxy/config.yaml`,
|
|
153
|
+
with its own CHANGELOG entry, and the proxy is published FIRST.
|
|
154
|
+
- **Publishing is pre-authorised.** Destructive actions are not.
|
|
155
|
+
- Refer to work by what it is, never by a list number.
|
|
156
|
+
|
|
157
|
+
## State of the tree as measured, 2026-09-12
|
|
158
|
+
|
|
159
|
+
- `node --check` passes on the three touched source files.
|
|
160
|
+
- Biome is clean over `services/`, `test/` and `routes/` — 282 files.
|
|
161
|
+
- Tests of the changed files: 131 of 131 pass.
|
|
162
|
+
- Nothing has been verified in the field; the released proxy is 2.83.3 with
|
|
163
|
+
addon 0.74.0 on the host.
|
|
164
|
+
|
|
165
|
+
None of that argues for keeping the work. It is recorded so that a failure
|
|
166
|
+
after the removal is not mistaken for a failure that was already there.
|
|
@@ -0,0 +1,207 @@
|
|
|
1
|
+
# The priority map is the only statement of what is wanted
|
|
2
|
+
|
|
3
|
+
Measurements and decisions behind the work listed in the roadmap as "the
|
|
4
|
+
priority map is the only source of truth about what is wanted". Written
|
|
5
|
+
2026-09-12, extended as the steps land.
|
|
6
|
+
|
|
7
|
+
Everything below is tagged by where it comes from: READ (from the source),
|
|
8
|
+
MEASURED (a number produced here), DECIDED (agreed with the user), or REFUSED
|
|
9
|
+
(considered and not built, with the reason).
|
|
10
|
+
|
|
11
|
+
## Why 2.83.1 stopped playback — READ
|
|
12
|
+
|
|
13
|
+
The map is built by `PriorityOrchestrator.publishFor`, and its only input is the
|
|
14
|
+
live SESSIONS (`hls-session-manager.js` hands it `sessionGroups`). A session
|
|
15
|
+
exists once a file has been chosen and its playback plan built. The torrent,
|
|
16
|
+
however, is added on `POST /api/sources/:key/warm`, while the person is still
|
|
17
|
+
reading the list of episodes.
|
|
18
|
+
|
|
19
|
+
So there is an interval in which the map cannot say anything at all, and in it
|
|
20
|
+
"the map wants nothing" is indistinguishable from "there is nobody to ask yet".
|
|
21
|
+
The field record of 2026-09-11 lies entirely inside that interval: added
|
|
22
|
+
20:57:17, first peer 20:57:18, out of the swarm 20:57:21 with 741 connections
|
|
23
|
+
let go.
|
|
24
|
+
|
|
25
|
+
## The same fault with the opposite sign — READ
|
|
26
|
+
|
|
27
|
+
When the last viewer of a file leaves, `publishFor` deletes that file's map from
|
|
28
|
+
its own memory and publishes NOTHING:
|
|
29
|
+
|
|
30
|
+
```js
|
|
31
|
+
for (const key of [...this.#maps.keys()]) {
|
|
32
|
+
if (!byFile.has(key)) { this.#maps.delete(key); this.#last.delete(key); }
|
|
33
|
+
}
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
`applyPriorityMap` withdraws a file's claimants only when it is CALLED, so the
|
|
37
|
+
demand register keeps that file's zones for the life of the torrent — nothing
|
|
38
|
+
else removes them but `forgetTorrent`, which runs when the torrent is removed
|
|
39
|
+
altogether.
|
|
40
|
+
|
|
41
|
+
Both ends are therefore broken: today the map can say neither "wanted" nor "not
|
|
42
|
+
wanted". That is what has to be fixed before the register can be believed by
|
|
43
|
+
anybody.
|
|
44
|
+
|
|
45
|
+
## The register alone cannot be asked — READ
|
|
46
|
+
|
|
47
|
+
A read states a claim only while it is STOPPED on a missing piece
|
|
48
|
+
(`piece-reader.js:765`, `Urgency.BLOCKED`) and withdraws it when the read ends
|
|
49
|
+
(`:1052`). So an empty register routinely means "at this instant everybody has
|
|
50
|
+
what they need", not "nobody wants this". A swarm rule built on the register
|
|
51
|
+
alone would release and rejoin repeatedly. The stable statement is the map,
|
|
52
|
+
which holds while people are there.
|
|
53
|
+
|
|
54
|
+
## No viewer-to-torrent link is needed — DECIDED
|
|
55
|
+
|
|
56
|
+
A session carries `sourceKey`, and the map is addressed by source and file. So
|
|
57
|
+
"is anybody here for this source" is already answered by any file of it having a
|
|
58
|
+
non-empty map. A second holder of that same relation would be the two-owners
|
|
59
|
+
fault this work removes, in a new place.
|
|
60
|
+
|
|
61
|
+
## What is in a torrent, as one statement — MEASURED
|
|
62
|
+
|
|
63
|
+
`services/torrent/Contents.js`, validated against the survey collection in
|
|
64
|
+
`Dropbox/trn` on 2026-09-12 (137 files, 134 parsed):
|
|
65
|
+
|
|
66
|
+
| | |
|
|
67
|
+
|---|---|
|
|
68
|
+
| torrents parsed | 134 |
|
|
69
|
+
| of them with more than one picture | 44 |
|
|
70
|
+
| items (pictures with what belongs to them) | 1357 |
|
|
71
|
+
| files paired to a picture | 421 |
|
|
72
|
+
| files paired to more than one picture | 0 |
|
|
73
|
+
| largest torrent | 265 pictures |
|
|
74
|
+
| items whose grouping differs from the per-picture call it replaces | 0 |
|
|
75
|
+
|
|
76
|
+
The last row is what says this is a move rather than a change: every one of the
|
|
77
|
+
1357 groupings agrees with `matchSidecarFiles` asked the old way, one picture at
|
|
78
|
+
a time.
|
|
79
|
+
|
|
80
|
+
The largest figure matters to the step that follows: a pack of 265 pictures is
|
|
81
|
+
why the edges are claimed one item at a time rather than for the whole list —
|
|
82
|
+
two pieces outstanding at a time, and the order is what bounds the cost.
|
|
83
|
+
|
|
84
|
+
## Telling an episode from an extra — REFUSED
|
|
85
|
+
|
|
86
|
+
Proposed, measured, and not built.
|
|
87
|
+
|
|
88
|
+
**Words.** `sample|trailer|extras?|bonus|preview|teaser|making|interview|
|
|
89
|
+
deleted|featurette` as whole tokens against every video path of the 134
|
|
90
|
+
torrents: **four matches, all four false**. Three are the word "making" inside
|
|
91
|
+
ordinary episode titles ("Making cash with her pussy", "Perfect pleasure making
|
|
92
|
+
pussy") and their sizes are 0.84 to 2.35 of their torrent's median, i.e. they
|
|
93
|
+
are the content. Not one real `Sample/` file exists in the collection, so the
|
|
94
|
+
rule cannot even be validated here — that is a property of this collection, not
|
|
95
|
+
a claim that extras do not exist.
|
|
96
|
+
|
|
97
|
+
**Size.** The ratio of each video to the median of the videos in its own torrent
|
|
98
|
+
runs continuously from 0.0000 to 1.0 with no gap anywhere: 53 videos sit under a
|
|
99
|
+
quarter of their median, and they are short clips that ARE what the torrent is
|
|
100
|
+
for (`REX1080_<name> - 720P_4000K_….mp4`, 6-23 MB, in collections of them). Two
|
|
101
|
+
shapes are genuinely not content — `preview1..5.mp4` in one torrent, and
|
|
102
|
+
`VIDEO_TS/VIDEO_TS.VOB` plus `VTS_01_0.VOB`, which are a DVD's menu and a
|
|
103
|
+
structure question rather than a size one.
|
|
104
|
+
|
|
105
|
+
So there is nothing here to derive a threshold from, and an invented one would
|
|
106
|
+
decide the order in which a stranger's bandwidth is spent. Every picture is an
|
|
107
|
+
item. The survey script is `scratchpad`-only and is reproduced by walking the
|
|
108
|
+
collection with `parse-torrent` and the two functions named above.
|
|
109
|
+
|
|
110
|
+
## Two classifications of one torrent, and they have already diverged — MEASURED
|
|
111
|
+
|
|
112
|
+
The browser classifies the file list itself (`public/domain/torrent-parser.js`,
|
|
113
|
+
`VIDEO_EXTENSIONS`, `classifyMediaFiles`, `orderForDisplay`) and the proxy does
|
|
114
|
+
it again (`services/torrent/files.js`). Compared 2026-09-12: the browser offers
|
|
115
|
+
`.dat` as video and the proxy does not count it, every other extension agreeing.
|
|
116
|
+
On such a torrent the person is offered a file the proxy does not consider a
|
|
117
|
+
picture, and `countVideoFiles` answers 0 — which also changes the one-picture
|
|
118
|
+
relaxation in the pairing.
|
|
119
|
+
|
|
120
|
+
One fact, one owner: the proxy owns the composition and the file-list route
|
|
121
|
+
returns it. The cost is one round trip for a `.torrent` the browser parsed
|
|
122
|
+
locally; the proxy holds those same bytes from registration, so it can answer
|
|
123
|
+
without the swarm.
|
|
124
|
+
|
|
125
|
+
## What the pool asks now, and the two things it no longer counts — DECIDED
|
|
126
|
+
|
|
127
|
+
One question: **is anything wanted of this torrent**, answered from the register
|
|
128
|
+
(`isWanted`). It is true from the moment a torrent is opened, because a torrent
|
|
129
|
+
with no file list yet cannot be stated about and is being fetched precisely
|
|
130
|
+
because somebody asked for it; and it becomes false exactly when the last
|
|
131
|
+
viewer leaves, because that is when the map is published with nothing in it.
|
|
132
|
+
|
|
133
|
+
Two facts decide the swarm, and they are separated from the doing
|
|
134
|
+
(`swarmDecisionFor`) because this is the rule that has failed twice in the
|
|
135
|
+
field:
|
|
136
|
+
|
|
137
|
+
| wanted | ever wanted | swarm | idle clock |
|
|
138
|
+
|---|---|---|---|
|
|
139
|
+
| yes | — | take | stopped |
|
|
140
|
+
| no | yes | let go | running |
|
|
141
|
+
| no | no | leave alone | running |
|
|
142
|
+
|
|
143
|
+
The last row is the 2.83.1 failure as a rule rather than as a mechanism: a
|
|
144
|
+
torrent nobody has ever wanted is one being OPENED, and taking its swarm away
|
|
145
|
+
there destroys the connections it will need a minute later.
|
|
146
|
+
|
|
147
|
+
**Leaving on `done` was considered and NOT built.** A complete torrent can want
|
|
148
|
+
nothing from a swarm, and the field case of 596 connections was exactly that.
|
|
149
|
+
But a piece can be lost after the fact — the disk tier discards pieces under
|
|
150
|
+
its own cap — and rejoining would then need a trigger nothing states today, so
|
|
151
|
+
the failure mode is a read that waits for ever. The complete-file case is
|
|
152
|
+
already answered from two directions: the upload policy refuses to seed when
|
|
153
|
+
nothing anybody asked for is missing, and `keepWholeFiles` destroys a torrent
|
|
154
|
+
whose every file has been written out whole.
|
|
155
|
+
|
|
156
|
+
**What is accepted as narrowed, and stated so it is not discovered as a
|
|
157
|
+
surprise.** A read that is flowing states nothing — it is reading bytes that
|
|
158
|
+
are present — so a torrent is no longer held by the mere existence of a read.
|
|
159
|
+
Three cases were checked: a `/stream` read belongs to a session whose map is
|
|
160
|
+
live; the subtitle walk reads only what is downloaded; the background fill of a
|
|
161
|
+
sidecar is covered by that file's ends. What is left is the disk cap evicting a
|
|
162
|
+
torrent whose read is in flight during a moment when nothing is stated for it.
|
|
163
|
+
The eviction order is the mitigation rather than a guard: candidates are sorted
|
|
164
|
+
by when they were last WANTED, so a torrent an encoder is reading sorts last
|
|
165
|
+
among them.
|
|
166
|
+
|
|
167
|
+
**A torrent is on the idle clock from the moment it exists.** Until now the only
|
|
168
|
+
thing that ever started that clock was a reader letting go, so a file list
|
|
169
|
+
fetched and never played was held for the life of the process.
|
|
170
|
+
|
|
171
|
+
## What the review of this work caught — READ
|
|
172
|
+
|
|
173
|
+
Two defects, both found by reading the finished code rather than by a test, and
|
|
174
|
+
both fixed with a check apiece.
|
|
175
|
+
|
|
176
|
+
1. **A warm-up could take the urgency away from a plan somebody was waiting
|
|
177
|
+
for.** The two ends of a file are stated by whoever asks for them, and there
|
|
178
|
+
are two such callers: the playback plan, which a person is watching a loading
|
|
179
|
+
screen for, and the warm-up, which by its whole purpose nobody is waiting
|
|
180
|
+
for. The warm-up fires off the files beside the picture without awaiting
|
|
181
|
+
them, so it can arrive second — and it restated the same claimants at its own
|
|
182
|
+
lower level. The statement is raise-only now; the one caller entitled to put
|
|
183
|
+
them back down is the read that has just finished, which says so.
|
|
184
|
+
2. **A departure could be noticed by nobody.** The background fill states a
|
|
185
|
+
claim of its own per file and withdraws it once nothing else wants that file
|
|
186
|
+
— but the withdrawal happened inside a pass that nothing acted on, so a
|
|
187
|
+
torrent everybody had left could be held "wanted" by the fill's own claim at
|
|
188
|
+
the moment of the departure and then never re-examined: in its swarm, with no
|
|
189
|
+
idle clock running. The pass acts on what it just said now.
|
|
190
|
+
|
|
191
|
+
## The shape of the shared statement — DECIDED
|
|
192
|
+
|
|
193
|
+
The classification is a PACKAGE rather than an answer the proxy returns, and the
|
|
194
|
+
reason is timing: the browser shows the list of episodes the instant a
|
|
195
|
+
`.torrent` is opened, before a proxy has been chosen, let alone connected.
|
|
196
|
+
Making it wait for an answer would trade one defect for a slower first screen.
|
|
197
|
+
|
|
198
|
+
`@torrent-tv/torrent-contents` is published from the proxy's own
|
|
199
|
+
`services/torrent`, and the browser imports it by the same bare name: Node
|
|
200
|
+
resolves it out of `node_modules`, and an import map in the page points that
|
|
201
|
+
name at `/vendor/torrent-contents/`, which the server serves straight from the
|
|
202
|
+
installed package. Nothing is bundled and nothing is copied.
|
|
203
|
+
|
|
204
|
+
Verified in a browser 2026-09-12: the page fetched `Contents.js`, `files.js` and
|
|
205
|
+
`naming.js` from that prefix with 200 each, `classifyMediaFiles` ran through the
|
|
206
|
+
package, and the episodes came back in reading order — `ep2` before `ep10` — with
|
|
207
|
+
a paired subtitle on its own episode and an unpaired soundtrack still offered.
|
|
@@ -1,3 +1,5 @@
|
|
|
1
|
+
import { contentsOf } from "../../../../services/torrent/Contents.js";
|
|
2
|
+
|
|
1
3
|
/**
|
|
2
4
|
* List the files of a registered source (torrent file OR magnet).
|
|
3
5
|
*
|
|
@@ -70,17 +72,29 @@ export async function handleApiSourceFilesGet(req, reply, { sourceRegistry, torr
|
|
|
70
72
|
}
|
|
71
73
|
|
|
72
74
|
const torrent = result;
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
75
|
+
// WHAT IS IN THIS TORRENT, decided here and nowhere else. The browser used to
|
|
76
|
+
// decide it again — a list of video extensions in its parser and a second,
|
|
77
|
+
// shorter pair inside its picker — and the three answers had already diverged
|
|
78
|
+
// (measured 2026-09-12: `.dat` was offered there and not counted here, which
|
|
79
|
+
// also decides whether a sidecar whose name matches nothing can belong to the
|
|
80
|
+
// only video present). It ships the paths already relative to the torrent
|
|
81
|
+
// root, so there is no stripping rule on the other side either.
|
|
82
|
+
const contents = contentsOf(torrent);
|
|
81
83
|
return reply.send({
|
|
82
84
|
name: torrent.name ?? "",
|
|
83
85
|
infoHash: torrent.infoHash ?? "",
|
|
84
|
-
|
|
86
|
+
// In the order a person reads them — by folder, then by name, with runs of
|
|
87
|
+
// digits compared as numbers. A torrent's own order is whatever the tool
|
|
88
|
+
// that made it chose, and it is routinely by size.
|
|
89
|
+
files: contents.files(),
|
|
90
|
+
// The pictures, each with what belongs to it. By index, because the files
|
|
91
|
+
// themselves are in the list above and saying them twice is how two copies
|
|
92
|
+
// of one fact start.
|
|
93
|
+
items: contents.items.map((item) => ({
|
|
94
|
+
fileIndex: item.fileIndex,
|
|
95
|
+
audio: item.audio.map((part) => part.fileIndex),
|
|
96
|
+
subtitles: item.subtitles.map((part) => part.fileIndex),
|
|
97
|
+
images: item.images.map((part) => part.fileIndex)
|
|
98
|
+
}))
|
|
85
99
|
});
|
|
86
100
|
}
|
|
@@ -1,9 +1,6 @@
|
|
|
1
1
|
import { logger } from "../../../../utils/logger.js";
|
|
2
|
-
import {
|
|
3
|
-
|
|
4
|
-
matchSidecarFiles,
|
|
5
|
-
TEXT_SUBTITLE_SIDECAR_EXTENSIONS
|
|
6
|
-
} from "../../../../services/torrent/files.js";
|
|
2
|
+
import { TEXT_SUBTITLE_SIDECAR_EXTENSIONS } from "../../../../services/torrent/files.js";
|
|
3
|
+
import { contentsOf } from "../../../../services/torrent/Contents.js";
|
|
7
4
|
|
|
8
5
|
/**
|
|
9
6
|
* Start fetching a source before anyone asks to play it.
|
|
@@ -70,17 +67,26 @@ export async function handleApiSourceWarmPost(req, reply, { sourceRegistry, torr
|
|
|
70
67
|
return reply.send({ started: false, swarm: false, edges: false });
|
|
71
68
|
}
|
|
72
69
|
|
|
73
|
-
//
|
|
74
|
-
//
|
|
75
|
-
//
|
|
76
|
-
//
|
|
77
|
-
//
|
|
70
|
+
// WHICH FILE THIS IS ABOUT. The caller names one when it knows — a torrent
|
|
71
|
+
// with a single video, or an episode the viewer has settled on — and on a
|
|
72
|
+
// season pack it names none, because nobody has chosen yet.
|
|
73
|
+
//
|
|
74
|
+
// Then the first item of the torrent stands in for the choice. One item, not
|
|
75
|
+
// twenty: warming a whole pack's worth of edges would spend the pool owner's
|
|
76
|
+
// bandwidth on nineteen files nobody opened, while one item is two pieces and
|
|
77
|
+
// is also the likeliest pick. It is stated as something nobody is waiting
|
|
78
|
+
// for, so on a proxy serving somebody else it costs nothing at all until
|
|
79
|
+
// their own film has everything it needs.
|
|
80
|
+
const contents = contentsOf(torrent);
|
|
81
|
+
const named = fileIndex !== null && torrent.files?.[fileIndex] ? fileIndex : null;
|
|
82
|
+
const candidate = named ?? contents.items[0]?.fileIndex ?? null;
|
|
83
|
+
|
|
78
84
|
let edges = false;
|
|
79
|
-
if (
|
|
85
|
+
if (candidate !== null) {
|
|
80
86
|
edges = true;
|
|
81
87
|
// Deliberately not awaited: this is the multi-second part, and the point of
|
|
82
88
|
// the whole route is that the viewer goes on choosing while it happens.
|
|
83
|
-
Promise.resolve(torrentPool.prefetchFileEdges(torrent,
|
|
89
|
+
Promise.resolve(torrentPool.prefetchFileEdges(torrent, candidate)).catch((error) => {
|
|
84
90
|
const message = error instanceof Error ? error.message : String(error);
|
|
85
91
|
logger.warn(`warm ${sourceKey.slice(0, 8)}: file edges failed: ${message}`);
|
|
86
92
|
});
|
|
@@ -88,8 +94,8 @@ export async function handleApiSourceWarmPost(req, reply, { sourceRegistry, torr
|
|
|
88
94
|
// the playback plan, so they must not queue behind a region nobody is
|
|
89
95
|
// reading yet. This one only has to arrive before the encoder does, and the
|
|
90
96
|
// encoder is a plan and a session away.
|
|
91
|
-
if (positionSeconds > 0 && typeof torrentPool.warmResumePosition === "function") {
|
|
92
|
-
Promise.resolve(torrentPool.warmResumePosition(torrent,
|
|
97
|
+
if (named !== null && positionSeconds > 0 && typeof torrentPool.warmResumePosition === "function") {
|
|
98
|
+
Promise.resolve(torrentPool.warmResumePosition(torrent, named, positionSeconds)).catch(
|
|
93
99
|
(error) => {
|
|
94
100
|
const message = error instanceof Error ? error.message : String(error);
|
|
95
101
|
logger.warn(`warm ${sourceKey.slice(0, 8)}: the viewer's position failed: ${message}`);
|
|
@@ -116,13 +122,8 @@ export async function handleApiSourceWarmPost(req, reply, { sourceRegistry, torr
|
|
|
116
122
|
// does. The rest of it is fetched when it is played, and nothing here spends
|
|
117
123
|
// the pool owner's bandwidth on a track nobody chose.
|
|
118
124
|
let sidecars = 0;
|
|
119
|
-
if (
|
|
120
|
-
const matched =
|
|
121
|
-
files: torrent.files,
|
|
122
|
-
videoIndex: fileIndex,
|
|
123
|
-
torrentName: typeof torrent.name === "string" ? torrent.name : "",
|
|
124
|
-
videoCount: countVideoFiles(torrent.files)
|
|
125
|
-
});
|
|
125
|
+
if (candidate !== null && Array.isArray(torrent.files)) {
|
|
126
|
+
const matched = contents.sidecarsOf(candidate);
|
|
126
127
|
const warmOne = (file, options) => {
|
|
127
128
|
sidecars += 1;
|
|
128
129
|
// Not awaited, like the picture's own edges above: the point of this route
|
|
@@ -162,7 +163,10 @@ export async function handleApiSourceWarmPost(req, reply, { sourceRegistry, torr
|
|
|
162
163
|
|
|
163
164
|
logger.info(
|
|
164
165
|
`warm ${sourceKey.slice(0, 8)}: swarm started for "${torrent.name}"` +
|
|
165
|
-
(edges
|
|
166
|
+
(edges
|
|
167
|
+
? `, fetching the edges of file ${candidate}` +
|
|
168
|
+
(named === null ? " — the first item, since nothing is chosen yet" : "")
|
|
169
|
+
: ", and it holds nothing to fetch the edges of") +
|
|
166
170
|
(sidecars > 0 ? ` and of ${sidecars} file(s) beside it` : "")
|
|
167
171
|
);
|
|
168
172
|
|
|
@@ -25,39 +25,6 @@ function getPayload(body) {
|
|
|
25
25
|
return {};
|
|
26
26
|
}
|
|
27
27
|
|
|
28
|
-
/**
|
|
29
|
-
* Claim the source's file so the pool cannot clean it up under a live session.
|
|
30
|
-
*
|
|
31
|
-
* Asynchronous underneath — the torrent lives on another thread — but the
|
|
32
|
-
* caller needs a release function immediately, so the claim is chased and the
|
|
33
|
-
* release waits for it.
|
|
34
|
-
*
|
|
35
|
-
* @param {{ sourceRegistry: object, torrentPool: object, sourceKey: string, fileIndex: number }} params
|
|
36
|
-
* @returns {() => void}
|
|
37
|
-
*/
|
|
38
|
-
function holdSource({ sourceRegistry, torrentPool, sourceKey, fileIndex }) {
|
|
39
|
-
let release = null;
|
|
40
|
-
let releasedEarly = false;
|
|
41
|
-
const record = sourceRegistry?.get?.(sourceKey);
|
|
42
|
-
if (!record) {
|
|
43
|
-
return () => {};
|
|
44
|
-
}
|
|
45
|
-
void Promise.resolve(torrentPool.getTorrent(record.sourceType, record.source))
|
|
46
|
-
.then((torrent) => {
|
|
47
|
-
release = torrentPool.acquireFile(torrent, fileIndex);
|
|
48
|
-
if (releasedEarly) {
|
|
49
|
-
release();
|
|
50
|
-
}
|
|
51
|
-
})
|
|
52
|
-
.catch(() => {});
|
|
53
|
-
return () => {
|
|
54
|
-
releasedEarly = true;
|
|
55
|
-
if (typeof release === "function") {
|
|
56
|
-
release();
|
|
57
|
-
release = null;
|
|
58
|
-
}
|
|
59
|
-
};
|
|
60
|
-
}
|
|
61
28
|
|
|
62
29
|
export async function handleApiTranscodeSessionsPost(req, reply, { hlsSessionManager, sourceRegistry, torrentPool }) {
|
|
63
30
|
const payload = getPayload(req.body);
|
|
@@ -108,22 +75,7 @@ export async function handleApiTranscodeSessionsPost(req, reply, { hlsSessionMan
|
|
|
108
75
|
: 0,
|
|
109
76
|
audioTrackIndex:
|
|
110
77
|
Number.isInteger(audioTrackIndex) && audioTrackIndex > 0 ? audioTrackIndex : 0,
|
|
111
|
-
segmentFormatId
|
|
112
|
-
// Hold the torrent for as long as this session lives. Reads take a claim
|
|
113
|
-
// only while they run, and a seek leaves a gap with no read at all — the
|
|
114
|
-
// disk sweep caught that gap on 2026-08-06 and deleted the film being
|
|
115
|
-
// watched.
|
|
116
|
-
// Takes the file to hold, because a session does not always read the file
|
|
117
|
-
// it was created for: a release whose dub ships as its own file gives that
|
|
118
|
-
// soundtrack a session of its own, reading a different index of the same
|
|
119
|
-
// torrent. Defaults to the picture, which is every other case.
|
|
120
|
-
acquireSource: (heldFileIndex = fileIndex) =>
|
|
121
|
-
holdSource({
|
|
122
|
-
sourceRegistry,
|
|
123
|
-
torrentPool,
|
|
124
|
-
sourceKey,
|
|
125
|
-
fileIndex: Number.isInteger(heldFileIndex) ? heldFileIndex : fileIndex
|
|
126
|
-
})
|
|
78
|
+
segmentFormatId
|
|
127
79
|
});
|
|
128
80
|
// The index of quality variants, when this session has more than one to
|
|
129
81
|
// offer. Its presence is what tells the browser it can change quality
|