@torrent-tv/proxy 2.89.8 → 2.89.10
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +9 -4
- package/CHANGELOG.md +9 -0
- package/docs/browser-proxy-contract.md +1 -1
- package/package.json +1 -1
- package/routes/transcode/audio-warm/get.js +7 -1
- package/services/transport/data-channel-handler.js +68 -35
- package/services/transport/delivery-probe.js +50 -94
- package/test/audio-warm-recorded.test.js +63 -0
- package/test/data-channel-viewer-lifecycle.test.js +70 -0
- package/test/delivery-probe-field.test.js +173 -0
- package/test/delivery-probe.test.js +20 -31
package/AGENTS.md
CHANGED
|
@@ -344,15 +344,20 @@ job in the `production` environment:
|
|
|
344
344
|
|
|
345
345
|
1. writes the version into `package.json`/`package-lock.json` and renames
|
|
346
346
|
`## Unreleased` to it, commits `chore(release): <version>` and tags it;
|
|
347
|
-
2. publishes `@torrent-tv/proxy` to npm with provenance
|
|
347
|
+
2. publishes `@torrent-tv/proxy` to npm with provenance, through npm trusted
|
|
348
|
+
publishing (the workflow's OIDC token; no npm token is stored);
|
|
348
349
|
3. pushes the tag and the commit and creates the GitHub release;
|
|
349
|
-
4.
|
|
350
|
+
4. waits until npm serves the version — npm accepts a publication before it
|
|
351
|
+
serves it, 6.5 minutes for 2.89.8 — then pushes
|
|
352
|
+
`fix(proxy)`/`feat(proxy): install proxy <version>` to
|
|
350
353
|
`torrent-tv/ha-addon` (`PROXY_VERSION` and the add-on changelog), whose own
|
|
351
354
|
workflow builds the add-on image and releases the add-on.
|
|
352
355
|
|
|
353
356
|
So the order proxy → add-on is kept by construction. A release can also be
|
|
354
|
-
started by hand from the Actions tab with an explicit `patch` or `minor` step
|
|
355
|
-
|
|
357
|
+
started by hand from the Actions tab with an explicit `patch` or `minor` step;
|
|
358
|
+
it still needs an entry under `## Unreleased`. Re-running a failed release job
|
|
359
|
+
is safe: once the tag exists, publication is skipped and only what is missing
|
|
360
|
+
is done. Updating the add-on on the Home Assistant host is still done there.
|
|
356
361
|
|
|
357
362
|
`.github/workflows/dependencies.yml` updates dependencies daily within the
|
|
358
363
|
ranges, runs the same checks and pushes to `main`.
|
package/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,12 @@
|
|
|
1
|
+
## 2.89.10
|
|
2
|
+
|
|
3
|
+
- **Fix**: An audio track warm-up that is still being made answers `503` with `warming: true`. That answer means the track and how it is produced are recorded for the viewer; a `503` without it is a preparation that failed and recorded nothing. Until now the two were the same status, so a page restating its soundtrack after a reconnect could not tell whether the proxy had kept it.
|
|
4
|
+
|
|
5
|
+
## 2.89.9
|
|
6
|
+
|
|
7
|
+
- **Fix**: The delivery probe judges a connection by how old the newest probe the browser has seen is, not by how long that probe took to arrive. The time it took is fixed once it has arrived, so on the wedge of 2026-09-28 it stayed 9 ms for a whole minute while the browser saw nothing after probe 1464, and every line read `flowing`; on idle healthy connections its 3 ms allowance left out the browser's own handling of the message, and `association-stopped` was printed for connections that delivered every probe. The age's allowance now also counts the probe interval, which `probeWedgeIsCertain` already counted. A real wedge is now named within about a second and reaches the packet witness.
|
|
8
|
+
- **Fix**: A viewer named on two connections at once stays until the last of them closes. The page raises a second connection before it lets go of one that stopped delivering, and may then close the second instead; the proxy kept only the newest connection per viewer, so closing that trial connection released a viewer still watching through the old one, and their subtitle pushes went to a closed channel. Pushes now go to the newest connection that still carries the viewer.
|
|
9
|
+
|
|
1
10
|
## 2.89.8
|
|
2
11
|
|
|
3
12
|
- **Chore**: Update dependencies: @fastify/cors 11.2.0 → 11.3.0, @fastify/helmet 13.0.2 → 13.1.1, @fastify/static 10.1.2 → 10.1.5, parse-torrent 11.0.23 → 11.0.24, werift 0.24.2 → 0.24.4, ws 8.21.1 → 8.22.0, @biomejs/biome 2.5.7 → 2.5.15; 78 package(s) changed in the lock file, 15 removed; 1 known vulnerability resolved.
|
|
@@ -94,7 +94,7 @@ soundtracks the viewer can choose from.
|
|
|
94
94
|
| `POST /api/transcode-sessions/:id/fragment-far` | Diagnostic fragment and buffer positions; does not change encoding. Best effort, success `204`. |
|
|
95
95
|
| `POST /api/transcode-sessions/:id/release` | `{ consumerId, reason }` releases this viewer's session assignment. |
|
|
96
96
|
| `GET /transcode/:id/:fileName` | HLS playlist, init, or media segment. A file that is still being produced is answered with retryable `503`, not `202`. |
|
|
97
|
-
| `GET /transcode/:id/v/:height/warm` and `/transcode/:id/a/:track/warm` | Prepare the requested video height or audio track at a position. `204` means ready. Audio warm-up `404` means the proxy does not support this operation for that session. Audio warm-up takes an optional `transcode=0\|1`: whether the browser cannot play THAT track's codec as it is. The proxy decides the track's mode by the rule used when a file is opened; without the parameter it assumes the browser needs what it needed for its current track. |
|
|
97
|
+
| `GET /transcode/:id/v/:height/warm` and `/transcode/:id/a/:track/warm` | Prepare the requested video height or audio track at a position. `204` means ready. Audio warm-up `404` means the proxy does not support this operation for that session. Audio warm-up takes an optional `transcode=0\|1`: whether the browser cannot play THAT track's codec as it is. The proxy decides the track's mode by the rule used when a file is opened; without the parameter it assumes the browser needs what it needed for its current track. Audio warm-up `503` with `{ warming: true }` means the track and its mode are recorded for this viewer and its segment is still being made; any other `503`, and `500`, recorded nothing. |
|
|
98
98
|
| `GET /stream?sourceKey=…&fileIndex=N` | Direct file bytes with HTTP range support. |
|
|
99
99
|
|
|
100
100
|
### Playback readiness forecast
|
package/package.json
CHANGED
|
@@ -90,6 +90,12 @@ export async function handleTranscodeAudioWarmGet(req, reply, { renditions, serv
|
|
|
90
90
|
}
|
|
91
91
|
// Still being produced. The caller may switch anyway — it will wait where it
|
|
92
92
|
// would have waited before — or ask again.
|
|
93
|
+
//
|
|
94
|
+
// `warming: true` says the track and how it is produced ARE recorded for this
|
|
95
|
+
// viewer — `prepareAudioTrack` records them before the wait — which the
|
|
96
|
+
// status alone cannot say: the 503 above is a preparation that recorded
|
|
97
|
+
// nothing. A page restating its soundtrack after a reconnect needs to know
|
|
98
|
+
// which of the two it got.
|
|
93
99
|
reply.header("Retry-After", "1");
|
|
94
|
-
return reply.code(503).send({ error: "The audio track is still warming up." });
|
|
100
|
+
return reply.code(503).send({ error: "The audio track is still warming up.", warming: true });
|
|
95
101
|
}
|
|
@@ -688,30 +688,71 @@ export function createDataChannelHandler({
|
|
|
688
688
|
sendChunkBytes = 0
|
|
689
689
|
}) {
|
|
690
690
|
/**
|
|
691
|
-
* Who is on each connection, by WebRTC session id
|
|
691
|
+
* Who is on each connection, by WebRTC session id, and the channel each of
|
|
692
|
+
* them named themselves on — null once that channel has closed.
|
|
692
693
|
*
|
|
693
694
|
* One connection carries one page, and a page holds one name per film it has
|
|
694
695
|
* open — so this is normally one id, and more than one only where a page has
|
|
695
696
|
* opened more than one.
|
|
696
697
|
*
|
|
697
|
-
*
|
|
698
|
+
* ONE PERSON MAY BE ON TWO CONNECTIONS AT ONCE, and this is the only record of
|
|
699
|
+
* it. The page raises a second connection before it lets go of the first
|
|
700
|
+
* when the first stops delivering, and it may then keep the first and close
|
|
701
|
+
* the second (it delivered again) or the reverse. The relation used to be
|
|
702
|
+
* held twice — this map and a second one naming each viewer's NEWEST
|
|
703
|
+
* connection — and the second could only point at one: a viewer named on a
|
|
704
|
+
* trial connection that was then closed was released while still watching
|
|
705
|
+
* through the old one. Both questions below are asked of this map instead.
|
|
706
|
+
*
|
|
707
|
+
* @type {Map<string, Map<string, DataChannel | null>>}
|
|
698
708
|
*/
|
|
699
709
|
const viewersOnConnection = new Map();
|
|
710
|
+
|
|
700
711
|
/**
|
|
701
|
-
* The
|
|
712
|
+
* The connections a viewer is still named on, oldest first.
|
|
702
713
|
*
|
|
703
|
-
*
|
|
704
|
-
*
|
|
705
|
-
|
|
714
|
+
* @param {string} consumerId
|
|
715
|
+
* @returns {string[]}
|
|
716
|
+
*/
|
|
717
|
+
function connectionsCarrying(consumerId) {
|
|
718
|
+
const carrying = [];
|
|
719
|
+
for (const [sessionId, viewers] of viewersOnConnection) {
|
|
720
|
+
if (viewers.has(consumerId)) {
|
|
721
|
+
carrying.push(sessionId);
|
|
722
|
+
}
|
|
723
|
+
}
|
|
724
|
+
return carrying;
|
|
725
|
+
}
|
|
726
|
+
|
|
727
|
+
/**
|
|
728
|
+
* The channel a viewer can be reached on right now: the one they named
|
|
729
|
+
* themselves on, on the newest connection that still carries them with that
|
|
730
|
+
* channel open.
|
|
731
|
+
*
|
|
732
|
+
* The transport knows a NAME and nothing else about a viewer: what it means,
|
|
733
|
+
* who holds it and what they are watching are somebody else's facts. Keeping
|
|
734
|
+
* only "this name is reachable here" is what lets a subscription made once
|
|
735
|
+
* outlive the channel it was made on — a reconnect, and a rotation done on
|
|
736
|
+
* purpose, both just add a connection to the map.
|
|
706
737
|
*
|
|
707
|
-
* @
|
|
738
|
+
* @param {string} consumerId
|
|
739
|
+
* @returns {DataChannel | null}
|
|
708
740
|
*/
|
|
709
|
-
|
|
741
|
+
function channelOfViewer(consumerId) {
|
|
742
|
+
let reachable = null;
|
|
743
|
+
for (const viewers of viewersOnConnection.values()) {
|
|
744
|
+
const channel = viewers.get(consumerId);
|
|
745
|
+
if (channel) {
|
|
746
|
+
reachable = channel;
|
|
747
|
+
}
|
|
748
|
+
}
|
|
749
|
+
return reachable;
|
|
750
|
+
}
|
|
710
751
|
|
|
711
752
|
/**
|
|
712
|
-
*
|
|
753
|
+
* This connection has gone, and with it everyone it was the last to carry.
|
|
713
754
|
*
|
|
714
|
-
* Said once per connection: a second call after the
|
|
755
|
+
* Said once per connection: a second call after the entry is removed must not
|
|
715
756
|
* announce departures a second time, and the two detectors below — the close
|
|
716
757
|
* event and the transport's own counters — can both fire for one death.
|
|
717
758
|
*
|
|
@@ -725,11 +766,15 @@ export function createDataChannelHandler({
|
|
|
725
766
|
if (!viewers || viewers.size === 0) {
|
|
726
767
|
return;
|
|
727
768
|
}
|
|
728
|
-
for (const consumerId of viewers) {
|
|
729
|
-
|
|
769
|
+
for (const consumerId of viewers.keys()) {
|
|
770
|
+
const still = connectionsCarrying(consumerId);
|
|
771
|
+
if (still.length > 0) {
|
|
772
|
+
onLog?.(
|
|
773
|
+
`[dc] Session ${sessionId.slice(0, 8)}: viewer ${consumerId} stays — still on ` +
|
|
774
|
+
`${still.map((id) => id.slice(0, 8)).join(", ")} (${because})`
|
|
775
|
+
);
|
|
730
776
|
continue;
|
|
731
777
|
}
|
|
732
|
-
connectionOfViewer.delete(consumerId);
|
|
733
778
|
onLog?.(`[dc] Session ${sessionId.slice(0, 8)}: viewer ${consumerId} left — ${because}`);
|
|
734
779
|
try {
|
|
735
780
|
onViewerGone(consumerId, because);
|
|
@@ -738,30 +783,20 @@ export function createDataChannelHandler({
|
|
|
738
783
|
}
|
|
739
784
|
}
|
|
740
785
|
}
|
|
741
|
-
/**
|
|
742
|
-
* The channel each viewer is reachable on right now, by the name the far end
|
|
743
|
-
* presented when it opened.
|
|
744
|
-
*
|
|
745
|
-
* The transport knows a NAME and nothing else about a viewer: what it means,
|
|
746
|
-
* who holds it and what they are watching are somebody else's facts. Keeping
|
|
747
|
-
* only "this name is reachable here" is what lets a subscription made once
|
|
748
|
-
* outlive the channel it was made on — a reconnect, and a rotation done on
|
|
749
|
-
* purpose, both just rewrite this entry.
|
|
750
|
-
*
|
|
751
|
-
* @type {Map<string, DataChannel>}
|
|
752
|
-
*/
|
|
753
|
-
const channelOfViewer = new Map();
|
|
754
786
|
|
|
755
787
|
/**
|
|
756
|
-
*
|
|
788
|
+
* This channel has closed: nobody can be reached on it any more. Who is on
|
|
789
|
+
* the connection does not change — that ends with the connection.
|
|
757
790
|
*
|
|
758
791
|
* @param {DataChannel} channel
|
|
759
792
|
* @returns {void}
|
|
760
793
|
*/
|
|
761
794
|
function forgetChannel(channel) {
|
|
762
|
-
for (const
|
|
763
|
-
|
|
764
|
-
|
|
795
|
+
for (const viewers of viewersOnConnection.values()) {
|
|
796
|
+
for (const [consumerId, on] of viewers) {
|
|
797
|
+
if (on === channel) {
|
|
798
|
+
viewers.set(consumerId, null);
|
|
799
|
+
}
|
|
765
800
|
}
|
|
766
801
|
}
|
|
767
802
|
}
|
|
@@ -795,7 +830,7 @@ export function createDataChannelHandler({
|
|
|
795
830
|
});
|
|
796
831
|
let sent = 0;
|
|
797
832
|
for (const consumerId of names) {
|
|
798
|
-
const channel = channelOfViewer
|
|
833
|
+
const channel = channelOfViewer(consumerId);
|
|
799
834
|
if (!channel) {
|
|
800
835
|
// Subscribed but not reachable this instant — between connections, or
|
|
801
836
|
// rotating. Nothing is dropped: they are still subscribed, and the walk
|
|
@@ -1011,12 +1046,10 @@ export function createDataChannelHandler({
|
|
|
1011
1046
|
// one person two the first time that happened, each with their own
|
|
1012
1047
|
// position.
|
|
1013
1048
|
if (message.type === "viewer" && typeof message.consumerId === "string" && message.consumerId) {
|
|
1014
|
-
const known = viewersOnConnection.get(sessionId) ?? new
|
|
1049
|
+
const known = viewersOnConnection.get(sessionId) ?? new Map();
|
|
1015
1050
|
const isNew = !known.has(message.consumerId);
|
|
1016
|
-
known.
|
|
1051
|
+
known.set(message.consumerId, channel);
|
|
1017
1052
|
viewersOnConnection.set(sessionId, known);
|
|
1018
|
-
connectionOfViewer.set(message.consumerId, sessionId);
|
|
1019
|
-
channelOfViewer.set(message.consumerId, channel);
|
|
1020
1053
|
if (isNew) {
|
|
1021
1054
|
log(`[dc] Session ${tag}: viewer ${message.consumerId} is on this connection`);
|
|
1022
1055
|
}
|
|
@@ -84,6 +84,8 @@ export const PROBE_INTERVAL_MS = 500;
|
|
|
84
84
|
* 15 instead of 11, and the gap of 12 is then what it was — a browser whose
|
|
85
85
|
* timers are frozen, not an association that stopped.
|
|
86
86
|
*
|
|
87
|
+
* The fifth term is the probe interval itself — see {@link allowedWaitMs}.
|
|
88
|
+
*
|
|
87
89
|
* @param {{ queuedBytes: number, bytesPerSecond: number, rttMs: number, echoIntervalMs?: number, peerLoopLagMs?: number, intervalMs?: number }} state
|
|
88
90
|
* @returns {number | null} Probes that may legitimately be outstanding, or null
|
|
89
91
|
* when no rate has been measured yet and nothing can be said.
|
|
@@ -100,11 +102,15 @@ export function allowedGap({
|
|
|
100
102
|
return null;
|
|
101
103
|
}
|
|
102
104
|
// At least one: a probe sent and not yet echoed is the ordinary state.
|
|
103
|
-
return Math.max(
|
|
105
|
+
return Math.max(
|
|
106
|
+
1,
|
|
107
|
+
Math.ceil(allowedWaitMs({ queuedBytes, bytesPerSecond, rttMs, echoIntervalMs, peerLoopLagMs, intervalMs }) / intervalMs)
|
|
108
|
+
);
|
|
104
109
|
}
|
|
105
110
|
|
|
106
111
|
/**
|
|
107
|
-
* How
|
|
112
|
+
* How old the newest probe the peer has reported may legitimately be, in
|
|
113
|
+
* milliseconds.
|
|
108
114
|
*
|
|
109
115
|
* THE QUANTITY {@link allowedGap} COMPUTES AND THEN THROWS AWAY by dividing it
|
|
110
116
|
* into probes. Probes are the wrong unit and always were: the same probe goes
|
|
@@ -114,20 +120,38 @@ export function allowedGap({
|
|
|
114
120
|
* and the honest measure of that is the age of the newest probe the peer has
|
|
115
121
|
* seen, which this is compared against.
|
|
116
122
|
*
|
|
117
|
-
*
|
|
118
|
-
* connection is achieving, the crossing, the peer's
|
|
119
|
-
* late the peer's event loop is running
|
|
123
|
+
* Five terms, every one measured or stated: the queue's own drain time at the
|
|
124
|
+
* rate this connection is achieving, the crossing both ways, the peer's
|
|
125
|
+
* reporting cadence, how late the peer's event loop is running, and the probe
|
|
126
|
+
* interval. The last is there because probes are discrete: a report composed
|
|
127
|
+
* just before probe N+1 arrives names N, which by then is one interval older
|
|
128
|
+
* than the newest probe sent. {@link probeWedgeIsCertain} has always counted
|
|
129
|
+
* it; this allowance left it out, so on an idle connection whose crossing is a
|
|
130
|
+
* few milliseconds the age crossed the allowance on ordinary ticks.
|
|
120
131
|
*
|
|
121
132
|
* @param {{ queuedBytes: number, bytesPerSecond: number, rttMs: number,
|
|
122
|
-
* echoIntervalMs?: number, peerLoopLagMs?: number }} state
|
|
133
|
+
* echoIntervalMs?: number, peerLoopLagMs?: number, intervalMs?: number }} state
|
|
123
134
|
* @returns {number}
|
|
124
135
|
*/
|
|
125
|
-
export function allowedWaitMs({
|
|
136
|
+
export function allowedWaitMs({
|
|
137
|
+
queuedBytes,
|
|
138
|
+
bytesPerSecond,
|
|
139
|
+
rttMs,
|
|
140
|
+
echoIntervalMs = 0,
|
|
141
|
+
peerLoopLagMs = 0,
|
|
142
|
+
intervalMs = PROBE_INTERVAL_MS
|
|
143
|
+
}) {
|
|
126
144
|
if (!(bytesPerSecond > 0)) {
|
|
127
145
|
return 0;
|
|
128
146
|
}
|
|
129
147
|
const drainMs = (Math.max(queuedBytes, 0) / bytesPerSecond) * 1000;
|
|
130
|
-
return
|
|
148
|
+
return (
|
|
149
|
+
drainMs +
|
|
150
|
+
Math.max(rttMs, 0) +
|
|
151
|
+
Math.max(echoIntervalMs, 0) +
|
|
152
|
+
Math.max(peerLoopLagMs, 0) +
|
|
153
|
+
Math.max(intervalMs, 0)
|
|
154
|
+
);
|
|
131
155
|
}
|
|
132
156
|
|
|
133
157
|
/**
|
|
@@ -293,23 +317,24 @@ export function readProbeState(state) {
|
|
|
293
317
|
const gap = Number.isInteger(seen) ? state.seq - Number(seen) : null;
|
|
294
318
|
const allowance = allowedOf(label);
|
|
295
319
|
// BY TIME WHERE IT IS KNOWN. The count is what the line prints, because it
|
|
296
|
-
// is what a reader recognises; what decides is a time
|
|
320
|
+
// is what a reader recognises; what decides is a time: how old the newest
|
|
321
|
+
// probe the peer has reported is.
|
|
297
322
|
//
|
|
298
|
-
//
|
|
299
|
-
//
|
|
300
|
-
//
|
|
301
|
-
// the
|
|
302
|
-
//
|
|
303
|
-
//
|
|
304
|
-
|
|
305
|
-
|
|
306
|
-
|
|
307
|
-
|
|
308
|
-
|
|
309
|
-
|
|
323
|
+
// NOT the one-way delay of that probe, which decided here from 2026-09-11
|
|
324
|
+
// to 2026-10-04. That delay is fixed the moment the probe arrives, so it
|
|
325
|
+
// describes a probe that WAS delivered and says nothing about one that was
|
|
326
|
+
// not: on the wedge of 2026-09-28 it stayed 9 ms for a whole minute while
|
|
327
|
+
// the peer saw nothing after probe 1464, and every line read `flowing`. Its
|
|
328
|
+
// allowance — half a round trip on an idle channel — also left out the
|
|
329
|
+
// peer's own handling of the message, so healthy connections read
|
|
330
|
+
// `association-stopped` (meta research/stalled-channel-recovery-2026-10-04.md).
|
|
331
|
+
// The age grows for as long as nothing is delivered, which is what this
|
|
332
|
+
// verdict exists to see.
|
|
333
|
+
const lagMs = timeOf(state.behindMs, label);
|
|
334
|
+
const mayWaitMs = timeOf(state.allowedWaitMs, label);
|
|
310
335
|
const lagText = lagMs === null || mayWaitMs === null
|
|
311
336
|
? ""
|
|
312
|
-
: ` ${Math.round(lagMs)}ms of ${Math.round(mayWaitMs)}ms
|
|
337
|
+
: ` ${Math.round(lagMs)}ms of ${Math.round(mayWaitMs)}ms`;
|
|
313
338
|
parts.push(`${label}=${seen ?? "?"}(gap ${gap ?? "?"} of ${allowance ?? "?"}${lagText})`);
|
|
314
339
|
if (allowance === null) {
|
|
315
340
|
continue;
|
|
@@ -442,10 +467,8 @@ export function createDeliveryProbe({
|
|
|
442
467
|
const rttMs = Number(delivery?.rttMs) || 0;
|
|
443
468
|
/** @type {Map<string, number | null>} */
|
|
444
469
|
const allowed = new Map();
|
|
445
|
-
/** How
|
|
470
|
+
/** How old each channel's newest reported probe may legitimately be. */
|
|
446
471
|
const allowedWait = new Map();
|
|
447
|
-
/** The same, for the measured one-way time where the clocks are reconciled. */
|
|
448
|
-
const allowedOneWay = new Map();
|
|
449
472
|
for (const [channel, label] of connection.channels) {
|
|
450
473
|
let queuedBytes = 0;
|
|
451
474
|
try {
|
|
@@ -466,21 +489,13 @@ export function createDeliveryProbe({
|
|
|
466
489
|
bytesPerSecond,
|
|
467
490
|
rttMs,
|
|
468
491
|
echoIntervalMs: connection.echoIntervalMs,
|
|
469
|
-
peerLoopLagMs: connection.peerLoopLagMs ?? 0
|
|
492
|
+
peerLoopLagMs: connection.peerLoopLagMs ?? 0,
|
|
493
|
+
intervalMs
|
|
470
494
|
});
|
|
471
495
|
const heldWait = allowedWait.get(label);
|
|
472
496
|
if (!Number.isFinite(heldWait) || waitMs > heldWait) {
|
|
473
497
|
allowedWait.set(label, waitMs);
|
|
474
498
|
}
|
|
475
|
-
// What a probe may take ONE WAY: the queue's own drain time and half the
|
|
476
|
-
// crossing. Neither the peer's reporting cadence nor its loop delay
|
|
477
|
-
// belongs here — with the clocks reconciled they are no longer in the
|
|
478
|
-
// measurement they would have to be allowed for.
|
|
479
|
-
const oneWayAllowed = allowedWaitMs({ queuedBytes, bytesPerSecond, rttMs: rttMs / 2 });
|
|
480
|
-
const heldOneWay = allowedOneWay.get(label);
|
|
481
|
-
if (!Number.isFinite(heldOneWay) || oneWayAllowed > heldOneWay) {
|
|
482
|
-
allowedOneWay.set(label, oneWayAllowed);
|
|
483
|
-
}
|
|
484
499
|
// Several channels can carry one label only in malformed cases; the
|
|
485
500
|
// larger allowance is the safer of the two.
|
|
486
501
|
const held = allowed.get(label);
|
|
@@ -536,11 +551,6 @@ export function createDeliveryProbe({
|
|
|
536
551
|
})
|
|
537
552
|
),
|
|
538
553
|
allowedWaitMs: Object.fromEntries(allowedWait.entries()),
|
|
539
|
-
// The measured forward delay, where the two clocks have been reconciled.
|
|
540
|
-
// Preferred over the age above because it is the thing itself: the age
|
|
541
|
-
// also carries the peer's reporting cadence and the way back.
|
|
542
|
-
oneWayMs: Object.fromEntries(connection.oneWayMs.entries()),
|
|
543
|
-
allowedOneWayMs: Object.fromEntries(allowedOneWay.entries()),
|
|
544
554
|
// Same arithmetic for the echo's own age: the peer cannot answer sooner
|
|
545
555
|
// than its own cadence allows, nor sooner than its own event loop runs.
|
|
546
556
|
echoStaleMs:
|
|
@@ -656,23 +666,6 @@ export function createDeliveryProbe({
|
|
|
656
666
|
lastSeenAdvanceAt: 0,
|
|
657
667
|
longestHealthySeenGapMs: 0,
|
|
658
668
|
sentAtBySeq: new Map(),
|
|
659
|
-
// THE TWO CLOCKS, separated by the exchange rather than assumed to
|
|
660
|
-
// agree. Every report carries when each channel's newest probe
|
|
661
|
-
// arrived at the peer and when the report itself left; with the time
|
|
662
|
-
// this proxy stamped into that probe, and the time the report
|
|
663
|
-
// arrives, that is the four timestamps an offset is computed from.
|
|
664
|
-
//
|
|
665
|
-
// The estimate is kept from the report with the SMALLEST round trip
|
|
666
|
-
// seen, because that is the one that queued the least — and the two
|
|
667
|
-
// directions here are as unequal as they get, film one way and almost
|
|
668
|
-
// nothing the other, which is exactly where an offset taken from a
|
|
669
|
-
// busy moment is wrong by half the difference.
|
|
670
|
-
/** @type {number | null} */
|
|
671
|
-
clockOffsetMs: null,
|
|
672
|
-
/** @type {number} */
|
|
673
|
-
offsetFromRoundTripMs: Number.POSITIVE_INFINITY,
|
|
674
|
-
/** How long the newest probe took to reach the peer, by channel. @type {Map<string, number>} */
|
|
675
|
-
oneWayMs: new Map(),
|
|
676
669
|
probeCaptureStarted: false,
|
|
677
670
|
timer: null
|
|
678
671
|
};
|
|
@@ -741,43 +734,6 @@ export function createDeliveryProbe({
|
|
|
741
734
|
connection.lastSeenAdvanceAt = now;
|
|
742
735
|
}
|
|
743
736
|
}
|
|
744
|
-
// THE FOUR TIMESTAMPS. `t1` is when this proxy sent the probe the peer
|
|
745
|
-
// is reporting, `t2` when the peer received it, `t3` when the peer sent
|
|
746
|
-
// this report, `t4` is now. From them the round trip is
|
|
747
|
-
// `(t4 - t1) - (t3 - t2)` — the peer's own thinking time removed — and
|
|
748
|
-
// the difference between the two clocks is `((t2 - t1) + (t3 - t4)) / 2`.
|
|
749
|
-
//
|
|
750
|
-
// The offset is kept from the report whose round trip was SMALLEST: that
|
|
751
|
-
// arithmetic assumes the two directions are equally quick, which here
|
|
752
|
-
// they are not — film one way, almost nothing the other — and the least
|
|
753
|
-
// queued sample is the one where the assumption costs least. A fresh
|
|
754
|
-
// minimum replaces the estimate, so a clock that jumps is followed within
|
|
755
|
-
// a few reports rather than believed for ever.
|
|
756
|
-
const peerSentAt = Number(echo?.sentAt);
|
|
757
|
-
const seenAt = echo?.seenAt;
|
|
758
|
-
if (Number.isFinite(peerSentAt) && seenAt && typeof seenAt === "object") {
|
|
759
|
-
for (const [label, peerSawAtRaw] of Object.entries(seenAt)) {
|
|
760
|
-
const peerSawAt = Number(peerSawAtRaw);
|
|
761
|
-
const newestSeenNumber = connection.seen.get(label);
|
|
762
|
-
const weSentAt = Number.isInteger(newestSeenNumber)
|
|
763
|
-
? connection.sentAtBySeq.get(newestSeenNumber)
|
|
764
|
-
: undefined;
|
|
765
|
-
if (!Number.isFinite(peerSawAt) || !Number.isFinite(weSentAt)) {
|
|
766
|
-
continue;
|
|
767
|
-
}
|
|
768
|
-
const roundTrip = (now - weSentAt) - (peerSentAt - peerSawAt);
|
|
769
|
-
if (roundTrip >= 0 && roundTrip < connection.offsetFromRoundTripMs) {
|
|
770
|
-
connection.offsetFromRoundTripMs = roundTrip;
|
|
771
|
-
connection.clockOffsetMs = ((peerSawAt - weSentAt) + (peerSentAt - now)) / 2;
|
|
772
|
-
}
|
|
773
|
-
if (connection.clockOffsetMs !== null) {
|
|
774
|
-
// What the probe itself took, one way, with the clocks reconciled.
|
|
775
|
-
// Negative only if the offset is stale, and then it says so rather
|
|
776
|
-
// than being hidden.
|
|
777
|
-
connection.oneWayMs.set(label, peerSawAt - weSentAt - connection.clockOffsetMs);
|
|
778
|
-
}
|
|
779
|
-
}
|
|
780
|
-
}
|
|
781
737
|
// What the far end says it has received at the transport level. It is the
|
|
782
738
|
// one figure that separates a backlog from a stopped association, and it
|
|
783
739
|
// arrives on the direction that goes on working through a freeze.
|
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
import test from "node:test";
|
|
2
|
+
import assert from "node:assert/strict";
|
|
3
|
+
|
|
4
|
+
import { handleTranscodeAudioWarmGet } from "../routes/transcode/audio-warm/get.js";
|
|
5
|
+
|
|
6
|
+
// A page restating its soundtrack after a reconnect has to know whether the
|
|
7
|
+
// proxy RECORDED it. Both answers below are 503; only one of them recorded.
|
|
8
|
+
|
|
9
|
+
function reply() {
|
|
10
|
+
return {
|
|
11
|
+
statusCode: 0,
|
|
12
|
+
payload: undefined,
|
|
13
|
+
header() {
|
|
14
|
+
return this;
|
|
15
|
+
},
|
|
16
|
+
code(status) {
|
|
17
|
+
this.statusCode = status;
|
|
18
|
+
return this;
|
|
19
|
+
},
|
|
20
|
+
send(payload) {
|
|
21
|
+
this.payload = payload;
|
|
22
|
+
return this;
|
|
23
|
+
}
|
|
24
|
+
};
|
|
25
|
+
}
|
|
26
|
+
|
|
27
|
+
const request = {
|
|
28
|
+
params: { sessionId: "base", track: "0" },
|
|
29
|
+
query: { position: "1641.2", consumer: "viewer-1", transcode: "1" }
|
|
30
|
+
};
|
|
31
|
+
|
|
32
|
+
test("a track still being made says it was recorded", async () => {
|
|
33
|
+
let preparedFor = null;
|
|
34
|
+
const answer = await handleTranscodeAudioWarmGet(request, reply(), {
|
|
35
|
+
renditions: {
|
|
36
|
+
prepareAudioTrack: async (...args) => {
|
|
37
|
+
preparedFor = args;
|
|
38
|
+
return { sessionId: "rendition", fileName: "segment-00268.mp4" };
|
|
39
|
+
}
|
|
40
|
+
},
|
|
41
|
+
serving: {
|
|
42
|
+
seekEpoch: () => 0,
|
|
43
|
+
// Anything but a file or a failure is "not made yet".
|
|
44
|
+
getFileStream: async () => ({ kind: "superseded" })
|
|
45
|
+
}
|
|
46
|
+
});
|
|
47
|
+
assert.deepEqual(preparedFor, ["base", 0, 1641.2, "viewer-1", false]);
|
|
48
|
+
assert.equal(answer.statusCode, 503);
|
|
49
|
+
assert.equal(answer.payload.warming, true);
|
|
50
|
+
});
|
|
51
|
+
|
|
52
|
+
test("a preparation that failed says nothing was recorded", async () => {
|
|
53
|
+
const answer = await handleTranscodeAudioWarmGet(request, reply(), {
|
|
54
|
+
renditions: {
|
|
55
|
+
prepareAudioTrack: async () => {
|
|
56
|
+
throw new Error("no encoder");
|
|
57
|
+
}
|
|
58
|
+
},
|
|
59
|
+
serving: {}
|
|
60
|
+
});
|
|
61
|
+
assert.equal(answer.statusCode, 503);
|
|
62
|
+
assert.equal(answer.payload.warming, undefined);
|
|
63
|
+
});
|
|
@@ -90,3 +90,73 @@ test("a closing connection does not release a viewer already named on its replac
|
|
|
90
90
|
{ consumerId: "viewer-1", reason: "the connection closed" }
|
|
91
91
|
]);
|
|
92
92
|
});
|
|
93
|
+
|
|
94
|
+
/**
|
|
95
|
+
* A page on an old connection that raises a trial connection beside it, the
|
|
96
|
+
* way it does when the old one stops delivering.
|
|
97
|
+
*/
|
|
98
|
+
function trialBeside(gone) {
|
|
99
|
+
const handler = createDataChannelHandler({
|
|
100
|
+
proxyPort: 9090,
|
|
101
|
+
getTransportSnapshot: () => ({ bytesSent: 0, bytesReceived: 0 }),
|
|
102
|
+
viewersWantingCues: () => ["viewer-1"],
|
|
103
|
+
onViewerGone: (consumerId, reason) => gone.push({ consumerId, reason })
|
|
104
|
+
});
|
|
105
|
+
const old = { control: new FakeDataChannel("proxy-control"), media: new FakeDataChannel("proxy") };
|
|
106
|
+
const trial = { control: new FakeDataChannel("proxy-control"), media: new FakeDataChannel("proxy") };
|
|
107
|
+
handler.handleChannel("old-peer", old.control);
|
|
108
|
+
handler.handleChannel("old-peer", old.media);
|
|
109
|
+
old.control.message(JSON.stringify({ type: "viewer", consumerId: "viewer-1" }));
|
|
110
|
+
handler.handleChannel("trial-peer", trial.control);
|
|
111
|
+
handler.handleChannel("trial-peer", trial.media);
|
|
112
|
+
trial.control.message(JSON.stringify({ type: "viewer", consumerId: "viewer-1" }));
|
|
113
|
+
const cuesOn = (channel) => channel.messages.filter((message) => message.includes("subtitle-cues")).length;
|
|
114
|
+
const push = () =>
|
|
115
|
+
handler.publishSubtitleCues({
|
|
116
|
+
sourceKey: "torrent:abc",
|
|
117
|
+
fileIndex: 0,
|
|
118
|
+
trackIndex: 0,
|
|
119
|
+
cues: [{ start: 1, end: 2, text: "line" }],
|
|
120
|
+
language: "en",
|
|
121
|
+
detectedLanguage: null,
|
|
122
|
+
cursor: 1
|
|
123
|
+
});
|
|
124
|
+
return { old, trial, cuesOn, push };
|
|
125
|
+
}
|
|
126
|
+
|
|
127
|
+
test("closing a trial connection keeps the viewer and their cues on the old one", () => {
|
|
128
|
+
// The old connection delivered again before the trial was adopted, so the
|
|
129
|
+
// page closes the trial. The viewer never left.
|
|
130
|
+
const gone = [];
|
|
131
|
+
const { old, trial, cuesOn, push } = trialBeside(gone);
|
|
132
|
+
trial.control.close();
|
|
133
|
+
trial.media.close();
|
|
134
|
+
assert.deepEqual(gone, [], "the viewer is still on the old connection");
|
|
135
|
+
push();
|
|
136
|
+
assert.equal(cuesOn(old.control), 1, "cues reach the channel the viewer named themselves on");
|
|
137
|
+
old.control.close();
|
|
138
|
+
old.media.close();
|
|
139
|
+
assert.deepEqual(gone, [{ consumerId: "viewer-1", reason: "the connection closed" }]);
|
|
140
|
+
});
|
|
141
|
+
|
|
142
|
+
test("after a trial connection is adopted, cues go to it and the old one's close releases nobody", () => {
|
|
143
|
+
const gone = [];
|
|
144
|
+
const { old, trial, cuesOn, push } = trialBeside(gone);
|
|
145
|
+
push();
|
|
146
|
+
assert.equal(cuesOn(trial.control), 1, "the newest connection carrying the viewer is used");
|
|
147
|
+
assert.equal(cuesOn(old.control), 0);
|
|
148
|
+
old.control.close();
|
|
149
|
+
old.media.close();
|
|
150
|
+
assert.deepEqual(gone, []);
|
|
151
|
+
push();
|
|
152
|
+
assert.equal(cuesOn(trial.control), 2);
|
|
153
|
+
});
|
|
154
|
+
|
|
155
|
+
test("a viewer whose newest channel closed is reached on the older connection", () => {
|
|
156
|
+
const gone = [];
|
|
157
|
+
const { old, trial, cuesOn, push } = trialBeside(gone);
|
|
158
|
+
trial.control.close();
|
|
159
|
+
push();
|
|
160
|
+
assert.equal(cuesOn(old.control), 1);
|
|
161
|
+
assert.deepEqual(gone, [], "the trial connection still has its media channel");
|
|
162
|
+
});
|
|
@@ -0,0 +1,173 @@
|
|
|
1
|
+
import test from "node:test";
|
|
2
|
+
import assert from "node:assert/strict";
|
|
3
|
+
|
|
4
|
+
import { createDeliveryProbe, PROBE_INTERVAL_MS } from "../services/transport/delivery-probe.js";
|
|
5
|
+
|
|
6
|
+
// The probe service run end to end, against channels that only record what is
|
|
7
|
+
// sent and a peer whose echoes this file writes. The shapes are the field's of
|
|
8
|
+
// 2026-09-28 (research/stalled-channel-recovery-2026-10-04.md in meta): two
|
|
9
|
+
// connections on one proxy, one healthy and idle, one wedged.
|
|
10
|
+
|
|
11
|
+
const LABELS = ["proxy", "proxy-control", "proxy-fast"];
|
|
12
|
+
|
|
13
|
+
/**
|
|
14
|
+
* A connection with three channels and a peer that echoes on its own cadence.
|
|
15
|
+
*
|
|
16
|
+
* `peer.delivering` decides whether the probes reach the far end; while it is
|
|
17
|
+
* false the far end goes on echoing — the reverse direction keeps working in a
|
|
18
|
+
* wedge — but the newest number it has seen stays where it stopped.
|
|
19
|
+
*
|
|
20
|
+
* @param {import("node:test").TestContext} t
|
|
21
|
+
* @param {{ rttMs?: number }} [options]
|
|
22
|
+
*/
|
|
23
|
+
function connection(t, { rttMs = 6 } = {}) {
|
|
24
|
+
t.mock.timers.enable({ apis: ["setInterval", "Date"], now: 0 });
|
|
25
|
+
const lines = [];
|
|
26
|
+
const captures = [];
|
|
27
|
+
const probe = createDeliveryProbe({
|
|
28
|
+
log: (line) => lines.push(line),
|
|
29
|
+
readDelivery: () => ({ bytesPerSecond: 3_000_000, rttMs }),
|
|
30
|
+
getTransportSnapshot: () => ({ remote: "peer" }),
|
|
31
|
+
witness: {
|
|
32
|
+
maybeCapture: (trigger) => {
|
|
33
|
+
captures.push({ at: Date.now(), trigger });
|
|
34
|
+
return true;
|
|
35
|
+
}
|
|
36
|
+
}
|
|
37
|
+
});
|
|
38
|
+
// The far end's own clock and how long a probe takes to become a fact there:
|
|
39
|
+
// half the crossing, and the page handling the message. The page reports
|
|
40
|
+
// when it saw each probe and when it sent the report, on its own clock — the
|
|
41
|
+
// timestamps the verdict used to turn into a one-way delay.
|
|
42
|
+
const PEER_CLOCK_OFFSET_MS = 37;
|
|
43
|
+
const HANDLING_MS = 4;
|
|
44
|
+
const peer = { delivering: true, seen: 0, seenAt: 0, channelBytes: 0, loopLagMs: 0, visibility: "visible" };
|
|
45
|
+
const channels = LABELS.map((label) => ({
|
|
46
|
+
label,
|
|
47
|
+
sendMessage(message) {
|
|
48
|
+
const { seq, sentAt } = JSON.parse(message);
|
|
49
|
+
if (peer.delivering && seq > peer.seen) {
|
|
50
|
+
peer.seen = seq;
|
|
51
|
+
peer.seenAt = sentAt + rttMs / 2 + HANDLING_MS + PEER_CLOCK_OFFSET_MS;
|
|
52
|
+
}
|
|
53
|
+
},
|
|
54
|
+
bufferedAmount: () => 0
|
|
55
|
+
}));
|
|
56
|
+
for (const channel of channels) {
|
|
57
|
+
probe.attach("s1", "s1", channel.label, channel);
|
|
58
|
+
}
|
|
59
|
+
const echo = () =>
|
|
60
|
+
probe.noteEcho("s1", {
|
|
61
|
+
type: "probe-echo",
|
|
62
|
+
seen: Object.fromEntries(LABELS.map((label) => [label, peer.seen])),
|
|
63
|
+
seenAt: Object.fromEntries(LABELS.map((label) => [label, peer.seenAt])),
|
|
64
|
+
sentAt: Date.now() + PEER_CLOCK_OFFSET_MS,
|
|
65
|
+
report: {
|
|
66
|
+
visibility: peer.visibility,
|
|
67
|
+
loopLagMs: peer.loopLagMs,
|
|
68
|
+
transportBytesReceived: peer.channelBytes,
|
|
69
|
+
channels: Object.fromEntries(LABELS.map((label) => [label, { messages: 0, bytes: peer.channelBytes }])),
|
|
70
|
+
pending: 0
|
|
71
|
+
}
|
|
72
|
+
});
|
|
73
|
+
t.after(() => probe.dispose());
|
|
74
|
+
return {
|
|
75
|
+
peer,
|
|
76
|
+
lines,
|
|
77
|
+
captures,
|
|
78
|
+
/**
|
|
79
|
+
* Run the clock, echoing every `echoEveryMs` — composed just before the
|
|
80
|
+
* probe of that instant arrives, which is the latest a healthy report can
|
|
81
|
+
* be.
|
|
82
|
+
*/
|
|
83
|
+
run(ms, echoEveryMs = PROBE_INTERVAL_MS) {
|
|
84
|
+
for (let elapsed = 0; elapsed < ms; elapsed += echoEveryMs) {
|
|
85
|
+
t.mock.timers.tick(echoEveryMs - 1);
|
|
86
|
+
echo();
|
|
87
|
+
t.mock.timers.tick(1);
|
|
88
|
+
}
|
|
89
|
+
},
|
|
90
|
+
verdicts(fromMs = 0) {
|
|
91
|
+
return lines
|
|
92
|
+
.filter((line) => line.startsWith("[dc-probe]"))
|
|
93
|
+
.map((line) => ({ verdict: line.split(" ")[2], at: Date.parse(line.match(/at=(\S+)/)[1]) }))
|
|
94
|
+
.filter((entry) => entry.at >= fromMs);
|
|
95
|
+
}
|
|
96
|
+
};
|
|
97
|
+
}
|
|
98
|
+
|
|
99
|
+
test("an idle healthy connection is not called stopped (2026-09-28, 35cff221)", (t) => {
|
|
100
|
+
// rtt 6 ms, every queue empty, nothing but probes crossing. The verdict that
|
|
101
|
+
// compared a one-way delay of 6-54 ms with an allowance of 3 ms printed
|
|
102
|
+
// `association-stopped` on this connection every five seconds.
|
|
103
|
+
const link = connection(t);
|
|
104
|
+
link.run(60_000);
|
|
105
|
+
// From the first echo on: before it there is nothing to judge.
|
|
106
|
+
const named = new Set(link.verdicts(PROBE_INTERVAL_MS).map((entry) => entry.verdict));
|
|
107
|
+
assert.deepEqual([...named], ["flowing"], `every probe was delivered: ${link.lines.at(-1)}`);
|
|
108
|
+
assert.equal(link.captures.length, 0);
|
|
109
|
+
});
|
|
110
|
+
|
|
111
|
+
test("a wedge is called stopped within seconds and reaches the witness (2026-09-28, 10b53a7e)", (t) => {
|
|
112
|
+
// Probe 1464 was the last the peer saw; echoes went on arriving for the
|
|
113
|
+
// whole minute. The old verdict read `flowing` on every line.
|
|
114
|
+
const link = connection(t, { rttMs: 26 });
|
|
115
|
+
link.run(30_000);
|
|
116
|
+
const wedgedAt = Date.now();
|
|
117
|
+
link.peer.delivering = false;
|
|
118
|
+
link.run(30_000);
|
|
119
|
+
const after = link.verdicts(wedgedAt);
|
|
120
|
+
const firstStop = after.find((entry) => entry.verdict === "association-stopped");
|
|
121
|
+
assert.ok(firstStop, `the wedge must be named: ${link.lines.at(-1)}`);
|
|
122
|
+
assert.ok(firstStop.at - wedgedAt <= 2_000, `named ${firstStop.at - wedgedAt} ms after the last delivery`);
|
|
123
|
+
assert.equal(after.at(-1).verdict, "association-stopped", "and it stays named while nothing arrives");
|
|
124
|
+
assert.equal(link.captures.length, 1, "one capture per wedge");
|
|
125
|
+
assert.ok(link.captures[0].at - wedgedAt <= 3_000, `captured ${link.captures[0].at - wedgedAt} ms in`);
|
|
126
|
+
});
|
|
127
|
+
|
|
128
|
+
test("delivery resuming clears the verdict and arms the next capture", (t) => {
|
|
129
|
+
const link = connection(t);
|
|
130
|
+
link.run(10_000);
|
|
131
|
+
link.peer.delivering = false;
|
|
132
|
+
link.run(10_000);
|
|
133
|
+
assert.equal(link.captures.length, 1);
|
|
134
|
+
const resumedAt = Date.now();
|
|
135
|
+
link.peer.delivering = true;
|
|
136
|
+
link.run(10_000);
|
|
137
|
+
assert.equal(link.verdicts(resumedAt).at(-1).verdict, "flowing");
|
|
138
|
+
link.peer.delivering = false;
|
|
139
|
+
link.run(10_000);
|
|
140
|
+
assert.equal(link.captures.length, 2, "a second wedge is a second capture");
|
|
141
|
+
});
|
|
142
|
+
|
|
143
|
+
test("a hidden tab answering late is read as flowing (field 2026-09-03 loop delays)", (t) => {
|
|
144
|
+
// A hidden tab echoes about once a second and reports its own event loop as
|
|
145
|
+
// late — 681 → 1881 ms in the field. Every probe still reaches it.
|
|
146
|
+
const link = connection(t);
|
|
147
|
+
link.run(10_000);
|
|
148
|
+
link.peer.visibility = "hidden";
|
|
149
|
+
link.peer.loopLagMs = 681;
|
|
150
|
+
link.run(5_000, 1_000);
|
|
151
|
+
link.peer.loopLagMs = 1881;
|
|
152
|
+
const hiddenAt = Date.now();
|
|
153
|
+
link.run(30_000, 2_000);
|
|
154
|
+
const named = new Set(link.verdicts(hiddenAt).map((entry) => entry.verdict));
|
|
155
|
+
assert.deepEqual([...named], ["flowing"], `every probe is delivered: ${link.lines.at(-1)}`);
|
|
156
|
+
});
|
|
157
|
+
|
|
158
|
+
test("late reports while film is still arriving are read as flowing", (t) => {
|
|
159
|
+
// A hidden tab reports every three seconds and says beforehand that its loop
|
|
160
|
+
// is that late; the far end's channel byte counter says film is crossing.
|
|
161
|
+
const link = connection(t);
|
|
162
|
+
link.run(10_000);
|
|
163
|
+
link.peer.visibility = "hidden";
|
|
164
|
+
link.peer.loopLagMs = 3_000;
|
|
165
|
+
link.run(1_000);
|
|
166
|
+
const lateAt = Date.now();
|
|
167
|
+
for (let index = 0; index < 20; index += 1) {
|
|
168
|
+
link.peer.channelBytes += 4_000_000;
|
|
169
|
+
link.run(3_000, 3_000);
|
|
170
|
+
}
|
|
171
|
+
const named = new Set(link.verdicts(lateAt).map((entry) => entry.verdict));
|
|
172
|
+
assert.deepEqual([...named], ["flowing"], `film is arriving: ${link.lines.at(-1)}`);
|
|
173
|
+
});
|
|
@@ -90,10 +90,11 @@ test("a channel that has never reported counts as behind, not as unknown", () =>
|
|
|
90
90
|
|
|
91
91
|
test("the allowance is the queue's own drain time, not a chosen number", () => {
|
|
92
92
|
// 8 MB queued at 8 MB/s is one second of draining; probes go twice a second,
|
|
93
|
-
// so two of them may legitimately be outstanding, plus the
|
|
93
|
+
// so two of them may legitimately be outstanding, plus the one sent after
|
|
94
|
+
// the peer composed its report, plus the round trip.
|
|
94
95
|
assert.equal(
|
|
95
96
|
allowedGap({ queuedBytes: 8 * 1024 * 1024, bytesPerSecond: 8 * 1024 * 1024, rttMs: 0 }),
|
|
96
|
-
Math.ceil(1000 / PROBE_INTERVAL_MS)
|
|
97
|
+
Math.ceil(1000 / PROBE_INTERVAL_MS) + 1
|
|
97
98
|
);
|
|
98
99
|
// An empty queue still allows the one probe that is always in flight.
|
|
99
100
|
assert.equal(allowedGap({ queuedBytes: 0, bytesPerSecond: 8 * 1024 * 1024, rttMs: 0 }), 1);
|
|
@@ -132,13 +133,14 @@ test("a burst big enough to explain the lag is not called a stopped association"
|
|
|
132
133
|
test("the peer's own answering cadence counts toward the allowance", () => {
|
|
133
134
|
// Field case 2026-08-27: queues empty, 3.4 MB/s crossing, tab hidden so the
|
|
134
135
|
// browser echoed about once a second. Without the peer's cadence the
|
|
135
|
-
// allowance is
|
|
136
|
+
// allowance is the probe interval and the crossing — two probes — and every
|
|
137
|
+
// other line read `association-stopped`.
|
|
136
138
|
const withoutCadence = allowedGap({
|
|
137
139
|
queuedBytes: 0,
|
|
138
140
|
bytesPerSecond: 3.4 * 1024 * 1024,
|
|
139
141
|
rttMs: 9
|
|
140
142
|
});
|
|
141
|
-
assert.equal(withoutCadence,
|
|
143
|
+
assert.equal(withoutCadence, 2);
|
|
142
144
|
const withCadence = allowedGap({
|
|
143
145
|
queuedBytes: 0,
|
|
144
146
|
bytesPerSecond: 3.4 * 1024 * 1024,
|
|
@@ -227,7 +229,8 @@ test("a frozen tab's own event-loop delay counts toward the allowance", () => {
|
|
|
227
229
|
// `flowing`. Nothing had stopped: the browser simply could not run the timer
|
|
228
230
|
// that answers a probe.
|
|
229
231
|
// The line printed `gap 12 of 11`; the queue was empty and the round trip
|
|
230
|
-
// 162 ms, so the cadence term is whatever
|
|
232
|
+
// 162 ms, so the cadence term is whatever made the allowance 11. The probe
|
|
233
|
+
// interval, added to the allowance on 2026-10-04, makes the same figures 12.
|
|
231
234
|
const measured = {
|
|
232
235
|
queuedBytes: 0,
|
|
233
236
|
bytesPerSecond: 300,
|
|
@@ -236,7 +239,7 @@ test("a frozen tab's own event-loop delay counts toward the allowance", () => {
|
|
|
236
239
|
};
|
|
237
240
|
const withoutLag = allowedGap(measured);
|
|
238
241
|
const withLag = allowedGap({ ...measured, peerLoopLagMs: 1881 });
|
|
239
|
-
assert.equal(withoutLag,
|
|
242
|
+
assert.equal(withoutLag, 12, `11 in the field plus the probe interval, not ${withoutLag}`);
|
|
240
243
|
assert.ok(withLag > 12, `a gap of 12 must fit inside ${withLag}`);
|
|
241
244
|
const allowed = Object.fromEntries(ALL.map((label) => [label, withLag]));
|
|
242
245
|
const { verdict, detail } = readProbeState(
|
|
@@ -358,34 +361,20 @@ test("with no send time recorded the count still decides", () => {
|
|
|
358
361
|
assert.equal(verdict, "association-stopped");
|
|
359
362
|
});
|
|
360
363
|
|
|
361
|
-
test("
|
|
362
|
-
//
|
|
363
|
-
//
|
|
364
|
-
//
|
|
365
|
-
// known, the measurement wins and its allowance carries neither.
|
|
364
|
+
test("a one-way delay of a probe that arrived says nothing about one that did not", () => {
|
|
365
|
+
// Field 2026-09-28, 10b53a7e: probe 1464 had taken 9 ms, and for the whole
|
|
366
|
+
// wedge it stayed the newest the peer had seen. A one-way figure in the
|
|
367
|
+
// state is not read: the age of that probe decides.
|
|
366
368
|
const { verdict } = readProbeState(
|
|
367
369
|
state(
|
|
368
|
-
{ proxy:
|
|
369
|
-
{
|
|
370
|
-
// The age says far behind against its allowance...
|
|
371
|
-
behindMs: { proxy: 9000, "proxy-control": 9000, "proxy-fast": 9000 },
|
|
372
|
-
allowedWaitMs: { proxy: 2000, "proxy-control": 2000, "proxy-fast": 2000 },
|
|
373
|
-
// ...while the probe itself took 300 ms of the 900 its queue may take.
|
|
374
|
-
oneWayMs: { proxy: 300, "proxy-control": 300, "proxy-fast": 300 },
|
|
375
|
-
allowedOneWayMs: { proxy: 900, "proxy-control": 900, "proxy-fast": 900 }
|
|
376
|
-
}
|
|
377
|
-
)
|
|
378
|
-
);
|
|
379
|
-
assert.equal(verdict, "flowing");
|
|
380
|
-
});
|
|
381
|
-
|
|
382
|
-
test("a one-way time past what the queue can account for is the association", () => {
|
|
383
|
-
const { verdict } = readProbeState(
|
|
384
|
-
state(
|
|
385
|
-
{ proxy: 40, "proxy-control": 41, "proxy-fast": 42 },
|
|
370
|
+
{ proxy: 1464, "proxy-control": 1464, "proxy-fast": 1464 },
|
|
386
371
|
{
|
|
387
|
-
|
|
388
|
-
|
|
372
|
+
seq: 1526,
|
|
373
|
+
behindMs: { proxy: 30_600, "proxy-control": 30_600, "proxy-fast": 30_600 },
|
|
374
|
+
allowedWaitMs: { proxy: 1029, "proxy-control": 1029, "proxy-fast": 1029 },
|
|
375
|
+
oneWayMs: { proxy: 9, "proxy-control": 9, "proxy-fast": 9 },
|
|
376
|
+
allowedOneWayMs: { proxy: 13, "proxy-control": 13, "proxy-fast": 13 },
|
|
377
|
+
peerBytesAdvancing: false
|
|
389
378
|
}
|
|
390
379
|
)
|
|
391
380
|
);
|