kikoeru 0.1.1 → 0.1.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +57 -44
- package/bin/{koed.js → kikoeru.js} +1 -1
- package/package.json +1 -1
- package/src/server.js +100 -24
- package/test/cache.test.js +125 -10
- package/test/chunking.test.js +41 -1
- package/test/config.test.js +42 -5
- package/test/helpers.js +7 -4
- package/test/upstream.test.js +7 -7
package/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
|
-
[](https://badge.fury.io/js/kikoeru)
|
|
2
2
|
|
|
3
|
-
#
|
|
3
|
+
# kikoeru
|
|
4
4
|
|
|
5
5
|
A small server that relays YouTube audio. It fetches the sound in
|
|
6
6
|
small pieces and passes the pieces to the listener right away, so
|
|
@@ -22,23 +22,23 @@ problem:
|
|
|
22
22
|
- Asking for small parts, one after another, stays fast to the end.
|
|
23
23
|
In our tests, a 16.6 MB file arrived in about 1.7 seconds this way.
|
|
24
24
|
|
|
25
|
-
A phone browser cannot be told to fetch in small parts. So
|
|
26
|
-
in the middle: the phone asks
|
|
25
|
+
A phone browser cannot be told to fetch in small parts. So kikoeru sits
|
|
26
|
+
in the middle: the phone asks kikoeru, and kikoeru asks YouTube in small
|
|
27
27
|
parts.
|
|
28
28
|
|
|
29
|
-
## What
|
|
29
|
+
## What kikoeru does
|
|
30
30
|
|
|
31
31
|
- It listens for requests of one form: `GET /<token>/<videoId>`.
|
|
32
32
|
- It looks up the video's sound address in a cache file (see below).
|
|
33
33
|
- It asks YouTube for the file in small parts of at most 4 MiB
|
|
34
34
|
(4,194,304 bytes) and sends each part to the listener as soon as
|
|
35
35
|
it arrives.
|
|
36
|
-
- It saves nothing. No sound file is ever written to disk.
|
|
36
|
+
- It saves nothing. No sound file is ever written to disk. kikoeru is a
|
|
37
37
|
relay, not a file host.
|
|
38
38
|
|
|
39
|
-
## What
|
|
39
|
+
## What kikoeru does not do
|
|
40
40
|
|
|
41
|
-
Finding the sound address is **not**
|
|
41
|
+
Finding the sound address is **not** kikoeru's job. A shell pipeline
|
|
42
42
|
(`yt-audio` and `yt-audio-batch`, which run `yt-dlp`) finds the
|
|
43
43
|
addresses and writes them to a cache file:
|
|
44
44
|
|
|
@@ -46,7 +46,7 @@ addresses and writes them to a cache file:
|
|
|
46
46
|
~/.yt-audio/cache.json
|
|
47
47
|
```
|
|
48
48
|
|
|
49
|
-
|
|
49
|
+
kikoeru only reads this file. For each video, the file holds the sound
|
|
50
50
|
address and the time it was found:
|
|
51
51
|
|
|
52
52
|
```json
|
|
@@ -60,20 +60,25 @@ address and the time it was found:
|
|
|
60
60
|
|
|
61
61
|
The rules are simple:
|
|
62
62
|
|
|
63
|
-
- Entry exists and
|
|
64
|
-
|
|
63
|
+
- Entry exists and its sound address has not expired yet → kikoeru
|
|
64
|
+
relays the sound. Sound addresses carry their own expiry time
|
|
65
|
+
(an `expire` value in the address, about 6 hours after the address
|
|
66
|
+
was found), and kikoeru serves the address until just before that
|
|
67
|
+
time. An address with no readable expiry is used until 6 hours
|
|
68
|
+
after it was found.
|
|
69
|
+
- Entry is missing or its address has expired → kikoeru answers 404.
|
|
65
70
|
(The shell pipeline will find a fresh address on its next run.)
|
|
66
71
|
|
|
67
72
|
Sound addresses stop working after some hours. If YouTube answers
|
|
68
|
-
403 ("forbidden") while
|
|
73
|
+
403 ("forbidden") while kikoeru is working, kikoeru reads the cache file
|
|
69
74
|
one more time — the shell pipeline may have put a fresh address in
|
|
70
|
-
it — and tries once more. If the address still does not work,
|
|
75
|
+
it — and tries once more. If the address still does not work, kikoeru
|
|
71
76
|
answers 404.
|
|
72
77
|
|
|
73
|
-
If YouTube fails while the sound is being sent,
|
|
74
|
-
the listener waiting. When no sound has been sent yet,
|
|
78
|
+
If YouTube fails while the sound is being sent, kikoeru never leaves
|
|
79
|
+
the listener waiting. When no sound has been sent yet, kikoeru answers
|
|
75
80
|
502 ("bad gateway") at once. When some sound has already been sent,
|
|
76
|
-
|
|
81
|
+
kikoeru ends the connection at once, so the player can show an error
|
|
77
82
|
instead of waiting for ever.
|
|
78
83
|
|
|
79
84
|
Built for the [reader](https://github.com/freddiefujiwara/reader)
|
|
@@ -82,32 +87,32 @@ web RSS reader, which plays YouTube videos like podcasts.
|
|
|
82
87
|
## Requirements
|
|
83
88
|
|
|
84
89
|
- Node.js 18 or later
|
|
85
|
-
- No extra packages.
|
|
90
|
+
- No extra packages. kikoeru uses only the Node.js standard library.
|
|
86
91
|
|
|
87
92
|
## Installation
|
|
88
93
|
|
|
89
94
|
From npm:
|
|
90
95
|
|
|
91
96
|
```bash
|
|
92
|
-
npm i -g
|
|
97
|
+
npm i -g kikoeru
|
|
93
98
|
```
|
|
94
99
|
|
|
95
100
|
Or from a tarball built from this repository:
|
|
96
101
|
|
|
97
102
|
```bash
|
|
98
|
-
npm pack # makes
|
|
99
|
-
npm i -g ./
|
|
103
|
+
npm pack # makes kikoeru-0.1.2.tgz
|
|
104
|
+
npm i -g ./kikoeru-0.1.2.tgz
|
|
100
105
|
```
|
|
101
106
|
|
|
102
107
|
With mise's Node on our home server, the global install goes to the
|
|
103
108
|
Node directory itself, so the module lands in
|
|
104
|
-
`~/.local/share/mise/installs/node/<version>/lib/node_modules/
|
|
105
|
-
and the `
|
|
109
|
+
`~/.local/share/mise/installs/node/<version>/lib/node_modules/kikoeru`
|
|
110
|
+
and the `kikoeru` command in the `bin/` folder next to it.
|
|
106
111
|
|
|
107
112
|
## Usage
|
|
108
113
|
|
|
109
114
|
```bash
|
|
110
|
-
|
|
115
|
+
kikoeru
|
|
111
116
|
```
|
|
112
117
|
|
|
113
118
|
This starts the server on `127.0.0.1:8791`. It only listens on the
|
|
@@ -139,11 +144,11 @@ Range: bytes=1000000-
|
|
|
139
144
|
Range: bytes=-1000
|
|
140
145
|
```
|
|
141
146
|
|
|
142
|
-
|
|
147
|
+
kikoeru answers `206` with exactly those bytes and a `Content-Range`
|
|
143
148
|
header. The parts are still fetched from YouTube in pieces of at
|
|
144
149
|
most 4 MiB. If the part cannot be given (for example, it starts
|
|
145
|
-
after the end of the file),
|
|
146
|
-
that
|
|
150
|
+
after the end of the file), kikoeru answers `416`. A `Range` header
|
|
151
|
+
that kikoeru cannot read is ignored, and the whole file is sent.
|
|
147
152
|
`If-Range` is ignored.
|
|
148
153
|
|
|
149
154
|
## The token
|
|
@@ -151,14 +156,14 @@ that koed cannot read is ignored, and the whole file is sent.
|
|
|
151
156
|
The token lives in a file:
|
|
152
157
|
|
|
153
158
|
```text
|
|
154
|
-
~/.config/
|
|
159
|
+
~/.config/kikoeru/token
|
|
155
160
|
```
|
|
156
161
|
|
|
157
|
-
- If the file exists,
|
|
158
|
-
- If the file does not exist,
|
|
162
|
+
- If the file exists, kikoeru reads the token from it.
|
|
163
|
+
- If the file does not exist, kikoeru makes a random token (32 hex
|
|
159
164
|
digits) on the first start, saves it to the file with permission
|
|
160
165
|
`600` (only the owner can read it), and uses it from then on.
|
|
161
|
-
- You can also set the token with the `
|
|
166
|
+
- You can also set the token with the `KIKOERU_TOKEN` environment
|
|
162
167
|
variable (see below). Then the file is not used.
|
|
163
168
|
|
|
164
169
|
Anyone who knows the token and a video id can listen through your
|
|
@@ -171,19 +176,27 @@ environment variable.
|
|
|
171
176
|
|
|
172
177
|
| Variable | Default | Meaning |
|
|
173
178
|
| --- | --- | --- |
|
|
174
|
-
| `
|
|
175
|
-
| `
|
|
176
|
-
| `
|
|
177
|
-
| `
|
|
179
|
+
| `KIKOERU_HOST` | `127.0.0.1` | Address to listen on |
|
|
180
|
+
| `KIKOERU_PORT` | `8791` | Port to listen on |
|
|
181
|
+
| `KIKOERU_TOKEN` | (contents of the token file) | The shared secret |
|
|
182
|
+
| `KIKOERU_TOKEN_FILE` | `~/.config/kikoeru/token` | Where the token is kept |
|
|
178
183
|
| `YT_AUDIO_CACHE` | `~/.yt-audio/cache.json` | The shell pipeline's cache file |
|
|
184
|
+
| `KIKOERU_CHUNK_SIZE` | `4194304` | Size of one upstream piece, in bytes |
|
|
185
|
+
| `KIKOERU_FALLBACK_FRESH_SEC` | `21600` | How long an address with no readable expiry is used, in seconds |
|
|
186
|
+
| `KIKOERU_EXPIRY_MARGIN_SEC` | `60` | Stop using an address this many seconds before it expires |
|
|
187
|
+
| `KIKOERU_META_TTL_MS` | `3000000` | How long a resolved size and type are remembered, in milliseconds |
|
|
188
|
+
|
|
189
|
+
The four numeric settings must be positive numbers. A value that is
|
|
190
|
+
not a number, or is zero or negative, is ignored and the default is
|
|
191
|
+
used instead.
|
|
179
192
|
|
|
180
193
|
## Example
|
|
181
194
|
|
|
182
195
|
Start the server, read the token, and ask for a video:
|
|
183
196
|
|
|
184
197
|
```bash
|
|
185
|
-
|
|
186
|
-
TOKEN=$(cat ~/.config/
|
|
198
|
+
kikoeru &
|
|
199
|
+
TOKEN=$(cat ~/.config/kikoeru/token)
|
|
187
200
|
curl -o sound.m4a "http://127.0.0.1:8791/$TOKEN/htzMcSM6Ft0"
|
|
188
201
|
```
|
|
189
202
|
|
|
@@ -203,17 +216,17 @@ curl -i "http://127.0.0.1:8791/wrong-token/htzMcSM6Ft0"
|
|
|
203
216
|
|
|
204
217
|
## Run it as a service (systemd)
|
|
205
218
|
|
|
206
|
-
On our home server,
|
|
207
|
-
file is `~/.config/systemd/user/
|
|
219
|
+
On our home server, kikoeru runs as a systemd user service. The unit
|
|
220
|
+
file is `~/.config/systemd/user/kikoeru.service`:
|
|
208
221
|
|
|
209
222
|
```ini
|
|
210
223
|
[Unit]
|
|
211
|
-
Description=
|
|
224
|
+
Description=kikoeru (YouTube audio streaming relay)
|
|
212
225
|
After=network-online.target
|
|
213
226
|
Wants=network-online.target
|
|
214
227
|
|
|
215
228
|
[Service]
|
|
216
|
-
ExecStart=/home/fumikazu/.local/share/mise/installs/node/24.14.1/bin/node /home/fumikazu/.local/share/mise/installs/node/24.14.1/lib/node_modules/
|
|
229
|
+
ExecStart=/home/fumikazu/.local/share/mise/installs/node/24.14.1/bin/node /home/fumikazu/.local/share/mise/installs/node/24.14.1/lib/node_modules/kikoeru/bin/kikoeru.js
|
|
217
230
|
Restart=on-failure
|
|
218
231
|
RestartSec=3
|
|
219
232
|
|
|
@@ -225,7 +238,7 @@ Change the Node version in the paths when Node is upgraded. Then:
|
|
|
225
238
|
|
|
226
239
|
```bash
|
|
227
240
|
systemctl --user daemon-reload
|
|
228
|
-
systemctl --user enable --now
|
|
241
|
+
systemctl --user enable --now kikoeru
|
|
229
242
|
```
|
|
230
243
|
|
|
231
244
|
If user lingering is on (`loginctl enable-linger`), the service
|
|
@@ -233,9 +246,9 @@ starts at boot and keeps running after logout.
|
|
|
233
246
|
|
|
234
247
|
## Reverse proxy
|
|
235
248
|
|
|
236
|
-
|
|
249
|
+
kikoeru only listens on the local machine, so we publish it with nginx
|
|
237
250
|
on its own host name. In this example the host is `y.ze.gs` and the
|
|
238
|
-
whole site is passed to
|
|
251
|
+
whole site is passed to kikoeru:
|
|
239
252
|
|
|
240
253
|
```nginx
|
|
241
254
|
server {
|
|
@@ -290,4 +303,4 @@ server on the local machine, so no network is needed. They cover:
|
|
|
290
303
|
|
|
291
304
|
## Contributing
|
|
292
305
|
|
|
293
|
-
Bug reports and pull requests are welcome on GitHub at https://github.com/freddiefujiwara/
|
|
306
|
+
Bug reports and pull requests are welcome on GitHub at https://github.com/freddiefujiwara/kikoeru
|
|
@@ -7,6 +7,6 @@ const cfg = loadConfig();
|
|
|
7
7
|
const server = createServer(cfg);
|
|
8
8
|
server.listen(cfg.port, cfg.host, () => {
|
|
9
9
|
console.log(
|
|
10
|
-
`[
|
|
10
|
+
`[kikoeru] listening on http://${cfg.host}:${cfg.port}/<token>/<videoId>`,
|
|
11
11
|
);
|
|
12
12
|
});
|
package/package.json
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"bin":{"
|
|
1
|
+
{"bin":{"kikoeru":"bin/kikoeru.js"},"description":"Streaming relay for YouTube audio: fetches googlevideo streams with small ranged requests and forwards them to the client without storing any files.","engines":{"node":">=18"},"license":"MIT","main":"src/server.js","name":"kikoeru","scripts":{"start":"node bin/kikoeru.js","test":"node --test \"test/**/*.test.js\""},"version":"0.1.3"}
|
package/src/server.js
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
'use strict';
|
|
2
2
|
|
|
3
3
|
/**
|
|
4
|
-
*
|
|
4
|
+
* kikoeru: a streaming relay for YouTube audio.
|
|
5
5
|
*
|
|
6
6
|
* Clients (e.g. a phone browser) that play a googlevideo audio URL directly
|
|
7
7
|
* are throttled to roughly the playback bitrate when they fetch the file
|
|
@@ -17,8 +17,10 @@
|
|
|
17
17
|
* Stream URL resolution is NOT this server's job. Extracting stream URLs
|
|
18
18
|
* with yt-dlp belongs to the shell pipeline (yt-audio / yt-audio-batch);
|
|
19
19
|
* this server only reads its cache (~/.yt-audio/cache.json) and relays:
|
|
20
|
-
* - entry present and
|
|
21
|
-
*
|
|
20
|
+
* - entry present and its URL not yet expired -> relay that URL.
|
|
21
|
+
* (The URL's own `expire` parameter decides; a URL without one is
|
|
22
|
+
* used until 6 h after its extraction time.)
|
|
23
|
+
* - entry missing or expired -> 404 (the shell refreshes it on its cycle).
|
|
22
24
|
* If the upstream answers 403 (URL expired server-side), the batch cache
|
|
23
25
|
* is re-read once (the shell may have refreshed it in the meantime) and
|
|
24
26
|
* the request is retried; if that still fails, the answer is 404.
|
|
@@ -36,22 +38,55 @@ const os = require('node:os');
|
|
|
36
38
|
const crypto = require('node:crypto');
|
|
37
39
|
|
|
38
40
|
const CHUNK_SIZE = 4 * 1024 * 1024; // upstream sub-range size
|
|
39
|
-
const
|
|
41
|
+
const FALLBACK_FRESH_SEC = 6 * 3600; // no expire param: ts + 6 h decides
|
|
42
|
+
const EXPIRY_MARGIN_SEC = 60; // stop serving this long before a URL expires
|
|
40
43
|
const META_TTL_MS = 50 * 60 * 1000; // resolved size/type/final-URL cache
|
|
41
44
|
const VIDEO_ID_RE = /^[A-Za-z0-9_-]{11}$/;
|
|
42
45
|
const ROUTE_RE = /^\/([^/]+)\/([^/]+)$/;
|
|
43
46
|
|
|
47
|
+
function positiveNumber(raw, fallback) {
|
|
48
|
+
const n = Number(raw);
|
|
49
|
+
return Number.isFinite(n) && n > 0 ? n : fallback;
|
|
50
|
+
}
|
|
51
|
+
|
|
52
|
+
function positiveInt(raw, fallback) {
|
|
53
|
+
return Math.max(1, Math.floor(positiveNumber(raw, fallback)));
|
|
54
|
+
}
|
|
55
|
+
|
|
56
|
+
/**
|
|
57
|
+
* Fill the numeric knobs of a config object with validated values.
|
|
58
|
+
* loadConfig feeds it the raw environment strings; createServer and
|
|
59
|
+
* resolveStreamUrl run it too, so a hand-built config object gets
|
|
60
|
+
* the same treatment. Anything that is not a positive number falls
|
|
61
|
+
* back to the built-in default, so a typo can never change the
|
|
62
|
+
* behaviour silently.
|
|
63
|
+
*/
|
|
64
|
+
function normalizeConfig(cfg) {
|
|
65
|
+
const c = cfg || {};
|
|
66
|
+
return {
|
|
67
|
+
...c,
|
|
68
|
+
chunkSize: positiveInt(c.chunkSize, CHUNK_SIZE),
|
|
69
|
+
fallbackFreshSec: positiveNumber(c.fallbackFreshSec, FALLBACK_FRESH_SEC),
|
|
70
|
+
expiryMarginSec: positiveNumber(c.expiryMarginSec, EXPIRY_MARGIN_SEC),
|
|
71
|
+
metaTtlMs: positiveNumber(c.metaTtlMs, META_TTL_MS),
|
|
72
|
+
};
|
|
73
|
+
}
|
|
74
|
+
|
|
44
75
|
function loadConfig(env = process.env) {
|
|
45
76
|
const home = os.homedir();
|
|
46
|
-
return {
|
|
47
|
-
host: env.
|
|
48
|
-
port: Number(env.
|
|
49
|
-
token: env.
|
|
77
|
+
return normalizeConfig({
|
|
78
|
+
host: env.KIKOERU_HOST || '127.0.0.1',
|
|
79
|
+
port: Number(env.KIKOERU_PORT || 8791),
|
|
80
|
+
token: env.KIKOERU_TOKEN || null,
|
|
50
81
|
tokenFile:
|
|
51
|
-
env.
|
|
82
|
+
env.KIKOERU_TOKEN_FILE || path.join(home, '.config/kikoeru/token'),
|
|
52
83
|
batchCacheFile:
|
|
53
84
|
env.YT_AUDIO_CACHE || path.join(home, '.yt-audio/cache.json'),
|
|
54
|
-
|
|
85
|
+
chunkSize: env.KIKOERU_CHUNK_SIZE,
|
|
86
|
+
fallbackFreshSec: env.KIKOERU_FALLBACK_FRESH_SEC,
|
|
87
|
+
expiryMarginSec: env.KIKOERU_EXPIRY_MARGIN_SEC,
|
|
88
|
+
metaTtlMs: env.KIKOERU_META_TTL_MS,
|
|
89
|
+
});
|
|
55
90
|
}
|
|
56
91
|
|
|
57
92
|
function ensureToken(cfg) {
|
|
@@ -77,16 +112,45 @@ function readJsonFile(file) {
|
|
|
77
112
|
}
|
|
78
113
|
|
|
79
114
|
/**
|
|
80
|
-
*
|
|
115
|
+
* The expiry (epoch seconds) a googlevideo stream URL carries in its
|
|
116
|
+
* `expire` query parameter, or null when the URL has none that parses.
|
|
117
|
+
*/
|
|
118
|
+
function urlExpirySec(url) {
|
|
119
|
+
let parsed;
|
|
120
|
+
try {
|
|
121
|
+
parsed = new URL(url);
|
|
122
|
+
} catch {
|
|
123
|
+
return null;
|
|
124
|
+
}
|
|
125
|
+
const raw = parsed.searchParams.get('expire');
|
|
126
|
+
if (!raw) return null;
|
|
127
|
+
const n = Number(raw);
|
|
128
|
+
return Number.isFinite(n) && n > 0 ? Math.floor(n) : null;
|
|
129
|
+
}
|
|
130
|
+
|
|
131
|
+
/**
|
|
132
|
+
* Resolve a still-usable stream URL for the video from the shell
|
|
81
133
|
* pipeline's batch cache, or null. This server never extracts URLs
|
|
82
134
|
* itself (no yt-dlp, no cache of its own).
|
|
135
|
+
*
|
|
136
|
+
* Usability follows the URL itself: googlevideo URLs carry an
|
|
137
|
+
* `expire` parameter (epoch seconds, roughly 6 h after extraction)
|
|
138
|
+
* and are served until just before that moment. URLs without a
|
|
139
|
+
* parseable `expire` fall back to the extraction timestamp plus
|
|
140
|
+
* FALLBACK_FRESH_SEC.
|
|
83
141
|
*/
|
|
84
142
|
function resolveStreamUrl(cfg, videoId) {
|
|
85
|
-
const
|
|
143
|
+
const c = normalizeConfig(cfg);
|
|
144
|
+
const batch = readJsonFile(c.batchCacheFile);
|
|
86
145
|
const entry = batch && batch[videoId];
|
|
87
146
|
if (!entry || !entry.url) return null;
|
|
88
147
|
const now = Math.floor(Date.now() / 1000);
|
|
89
|
-
|
|
148
|
+
const exp = urlExpirySec(entry.url);
|
|
149
|
+
if (exp !== null) {
|
|
150
|
+
return now < exp - c.expiryMarginSec ? entry.url : null;
|
|
151
|
+
}
|
|
152
|
+
const ts = entry.ts || 0;
|
|
153
|
+
if (!ts || now - ts >= c.fallbackFreshSec) return null;
|
|
90
154
|
return entry.url;
|
|
91
155
|
}
|
|
92
156
|
|
|
@@ -144,7 +208,7 @@ async function probeMeta(startUrl) {
|
|
|
144
208
|
async function getMeta(cfg, state, videoId, force = false) {
|
|
145
209
|
if (!force) {
|
|
146
210
|
const hit = state.meta.get(videoId);
|
|
147
|
-
if (hit && Date.now() - hit.at <
|
|
211
|
+
if (hit && Date.now() - hit.at < cfg.metaTtlMs) return hit;
|
|
148
212
|
}
|
|
149
213
|
const url = resolveStreamUrl(cfg, videoId);
|
|
150
214
|
if (!url) return null;
|
|
@@ -212,7 +276,7 @@ async function handle(cfg, state, token, req, res) {
|
|
|
212
276
|
try {
|
|
213
277
|
meta = await getMeta(cfg, state, videoId, retried);
|
|
214
278
|
} catch (e) {
|
|
215
|
-
console.error(`[
|
|
279
|
+
console.error(`[kikoeru] meta failed for ${videoId}: ${e.message}`);
|
|
216
280
|
meta = null;
|
|
217
281
|
}
|
|
218
282
|
if (!meta) {
|
|
@@ -249,7 +313,7 @@ async function handle(cfg, state, token, req, res) {
|
|
|
249
313
|
// re-read batch cache.
|
|
250
314
|
const tracker = { committed: false, wrote: false };
|
|
251
315
|
try {
|
|
252
|
-
const firstEnd = Math.min(start +
|
|
316
|
+
const firstEnd = Math.min(start + cfg.chunkSize - 1, end);
|
|
253
317
|
const up = await openUpstream(meta.finalUrl, {
|
|
254
318
|
Range: `bytes=${start}-${firstEnd}`,
|
|
255
319
|
});
|
|
@@ -275,7 +339,14 @@ async function handle(cfg, state, token, req, res) {
|
|
|
275
339
|
}
|
|
276
340
|
}
|
|
277
341
|
if (firstEnd < end) {
|
|
278
|
-
await streamRest(
|
|
342
|
+
await streamRest(
|
|
343
|
+
meta.finalUrl,
|
|
344
|
+
firstEnd + 1,
|
|
345
|
+
end,
|
|
346
|
+
res,
|
|
347
|
+
tracker,
|
|
348
|
+
cfg.chunkSize,
|
|
349
|
+
);
|
|
279
350
|
}
|
|
280
351
|
res.end();
|
|
281
352
|
return;
|
|
@@ -286,7 +357,7 @@ async function handle(cfg, state, token, req, res) {
|
|
|
286
357
|
continue;
|
|
287
358
|
}
|
|
288
359
|
if (e.code !== 'UPSTREAM_EXPIRED') {
|
|
289
|
-
console.error(`[
|
|
360
|
+
console.error(`[kikoeru] stream failed for ${videoId}: ${e.message}`);
|
|
290
361
|
}
|
|
291
362
|
if (!tracker.committed && !res.headersSent && !res.writableEnded) {
|
|
292
363
|
// Nothing has been sent to the client yet, so the failure can
|
|
@@ -305,11 +376,11 @@ async function handle(cfg, state, token, req, res) {
|
|
|
305
376
|
}
|
|
306
377
|
}
|
|
307
378
|
|
|
308
|
-
async function streamRest(finalUrl, start, end, res, tracker) {
|
|
379
|
+
async function streamRest(finalUrl, start, end, res, tracker, chunkSize) {
|
|
309
380
|
let pos = start;
|
|
310
381
|
while (pos <= end) {
|
|
311
382
|
if (res.destroyed || res.writableEnded) return;
|
|
312
|
-
const chunkEnd = Math.min(pos +
|
|
383
|
+
const chunkEnd = Math.min(pos + chunkSize - 1, end);
|
|
313
384
|
const up = await openUpstream(finalUrl, {
|
|
314
385
|
Range: `bytes=${pos}-${chunkEnd}`,
|
|
315
386
|
});
|
|
@@ -337,11 +408,12 @@ async function streamRest(finalUrl, start, end, res, tracker) {
|
|
|
337
408
|
}
|
|
338
409
|
|
|
339
410
|
function createServer(cfg) {
|
|
340
|
-
const
|
|
411
|
+
const full = normalizeConfig(cfg);
|
|
412
|
+
const token = ensureToken(full);
|
|
341
413
|
const state = { meta: new Map() };
|
|
342
414
|
const server = http.createServer((req, res) => {
|
|
343
|
-
handle(
|
|
344
|
-
console.error(`[
|
|
415
|
+
handle(full, state, token, req, res).catch((e) => {
|
|
416
|
+
console.error(`[kikoeru] handler error: ${e.message}`);
|
|
345
417
|
if (!res.headersSent) {
|
|
346
418
|
res.writeHead(500, { 'Content-Type': 'text/plain' });
|
|
347
419
|
}
|
|
@@ -354,9 +426,13 @@ function createServer(cfg) {
|
|
|
354
426
|
module.exports = {
|
|
355
427
|
createServer,
|
|
356
428
|
loadConfig,
|
|
429
|
+
normalizeConfig,
|
|
357
430
|
ensureToken,
|
|
358
431
|
parseRange,
|
|
359
432
|
resolveStreamUrl,
|
|
433
|
+
urlExpirySec,
|
|
360
434
|
CHUNK_SIZE,
|
|
361
|
-
|
|
435
|
+
FALLBACK_FRESH_SEC,
|
|
436
|
+
EXPIRY_MARGIN_SEC,
|
|
437
|
+
META_TTL_MS,
|
|
362
438
|
};
|
package/test/cache.test.js
CHANGED
|
@@ -1,16 +1,23 @@
|
|
|
1
1
|
'use strict';
|
|
2
2
|
|
|
3
3
|
// The batch cache is the only source of stream URLs. These tests pin
|
|
4
|
-
// down how the server judges it:
|
|
5
|
-
//
|
|
6
|
-
//
|
|
4
|
+
// down how the server judges it: an entry is relayed while its URL is
|
|
5
|
+
// unexpired (the URL's own `expire` parameter decides; without one,
|
|
6
|
+
// the extraction time plus 6 hours does), missing / expired / broken
|
|
7
|
+
// caches are a plain 404, and the server never falls over on a bad
|
|
8
|
+
// file. resolveStreamUrl is also unit-tested directly.
|
|
7
9
|
|
|
8
10
|
const { test, before, after } = require('node:test');
|
|
9
11
|
const assert = require('node:assert');
|
|
10
12
|
const fs = require('node:fs');
|
|
11
13
|
const path = require('node:path');
|
|
12
14
|
|
|
13
|
-
const {
|
|
15
|
+
const {
|
|
16
|
+
createServer,
|
|
17
|
+
resolveStreamUrl,
|
|
18
|
+
FALLBACK_FRESH_SEC,
|
|
19
|
+
EXPIRY_MARGIN_SEC,
|
|
20
|
+
} = require('../src/server');
|
|
14
21
|
const {
|
|
15
22
|
listen,
|
|
16
23
|
close,
|
|
@@ -43,7 +50,7 @@ after(async () => {
|
|
|
43
50
|
fs.rmSync(tmp, { recursive: true, force: true });
|
|
44
51
|
});
|
|
45
52
|
|
|
46
|
-
// Start a throwaway
|
|
53
|
+
// Start a throwaway kikoeru whose batch cache is `cacheFile`, run `fn`
|
|
47
54
|
// against its port, and always close it.
|
|
48
55
|
async function withServer(cacheFile, fn) {
|
|
49
56
|
const p = createServer({
|
|
@@ -74,6 +81,13 @@ function goodEntry(ts) {
|
|
|
74
81
|
return { url: `http://127.0.0.1:${upPort}/media`, ts };
|
|
75
82
|
}
|
|
76
83
|
|
|
84
|
+
// An entry whose URL carries an `expire` parameter, like a real
|
|
85
|
+
// googlevideo URL. The fake upstream serves /media regardless of the
|
|
86
|
+
// query string; only kikoeru's freshness judgment reads the value.
|
|
87
|
+
function expiringEntry(ts, expire) {
|
|
88
|
+
return { url: `http://127.0.0.1:${upPort}/media?expire=${expire}`, ts };
|
|
89
|
+
}
|
|
90
|
+
|
|
77
91
|
test('fresh entry (just written) is relayed', async () => {
|
|
78
92
|
await withServer(cacheWith(goodEntry(nowSec())), async (port) => {
|
|
79
93
|
const r = await get(port, `/${TOKEN}/${VIDEO_ID}`, {
|
|
@@ -84,8 +98,8 @@ test('fresh entry (just written) is relayed', async () => {
|
|
|
84
98
|
});
|
|
85
99
|
});
|
|
86
100
|
|
|
87
|
-
test('entry just inside the
|
|
88
|
-
const ts = nowSec() -
|
|
101
|
+
test('entry just inside the fallback freshness window is relayed', async () => {
|
|
102
|
+
const ts = nowSec() - FALLBACK_FRESH_SEC + 120; // 2 minutes of margin
|
|
89
103
|
await withServer(cacheWith(goodEntry(ts)), async (port) => {
|
|
90
104
|
const r = await get(port, `/${TOKEN}/${VIDEO_ID}`, {
|
|
91
105
|
Range: 'bytes=0-9',
|
|
@@ -94,8 +108,8 @@ test('entry just inside the 5 hour freshness window is relayed', async () => {
|
|
|
94
108
|
});
|
|
95
109
|
});
|
|
96
110
|
|
|
97
|
-
test('entry just past the
|
|
98
|
-
const ts = nowSec() -
|
|
111
|
+
test('entry just past the fallback freshness window gets 404', async () => {
|
|
112
|
+
const ts = nowSec() - FALLBACK_FRESH_SEC - 120; // 2 minutes of margin
|
|
99
113
|
await withServer(cacheWith(goodEntry(ts)), async (port) => {
|
|
100
114
|
const r = await get(port, `/${TOKEN}/${VIDEO_ID}`);
|
|
101
115
|
assert.equal(r.status, 404);
|
|
@@ -103,7 +117,7 @@ test('entry just past the 5 hour freshness window gets 404', async () => {
|
|
|
103
117
|
});
|
|
104
118
|
|
|
105
119
|
test('stale batch cache entry gets 404', async () => {
|
|
106
|
-
const ts = nowSec() -
|
|
120
|
+
const ts = nowSec() - FALLBACK_FRESH_SEC - 120; // older than 6 h
|
|
107
121
|
await withServer(cacheWith(goodEntry(ts)), async (port) => {
|
|
108
122
|
const r = await get(port, `/${TOKEN}/${VIDEO_ID}`, {
|
|
109
123
|
Range: 'bytes=0-99',
|
|
@@ -112,6 +126,71 @@ test('stale batch cache entry gets 404', async () => {
|
|
|
112
126
|
});
|
|
113
127
|
});
|
|
114
128
|
|
|
129
|
+
// --- expiry-based judgment (the URL's own `expire` parameter) --------
|
|
130
|
+
|
|
131
|
+
test('old entry is still relayed while its URL is unexpired', async () => {
|
|
132
|
+
// Extracted more than 5 hours ago (past the old fixed window), but
|
|
133
|
+
// the URL itself stays valid for another 30 minutes.
|
|
134
|
+
const entry = expiringEntry(nowSec() - 5 * 3600 - 600, nowSec() + 1800);
|
|
135
|
+
await withServer(cacheWith(entry), async (port) => {
|
|
136
|
+
const r = await get(port, `/${TOKEN}/${VIDEO_ID}`, {
|
|
137
|
+
Range: 'bytes=0-9',
|
|
138
|
+
});
|
|
139
|
+
assert.equal(r.status, 206);
|
|
140
|
+
assert.deepEqual(r.body, data.subarray(0, 10));
|
|
141
|
+
});
|
|
142
|
+
});
|
|
143
|
+
|
|
144
|
+
test('fresh entry gets 404 once its URL has expired', async () => {
|
|
145
|
+
const entry = expiringEntry(nowSec(), nowSec() - 60);
|
|
146
|
+
await withServer(cacheWith(entry), async (port) => {
|
|
147
|
+
const r = await get(port, `/${TOKEN}/${VIDEO_ID}`);
|
|
148
|
+
assert.equal(r.status, 404);
|
|
149
|
+
});
|
|
150
|
+
});
|
|
151
|
+
|
|
152
|
+
test('URL expiring within the safety margin gets 404', async () => {
|
|
153
|
+
const entry = expiringEntry(nowSec(), nowSec() + EXPIRY_MARGIN_SEC - 30);
|
|
154
|
+
await withServer(cacheWith(entry), async (port) => {
|
|
155
|
+
const r = await get(port, `/${TOKEN}/${VIDEO_ID}`);
|
|
156
|
+
assert.equal(r.status, 404);
|
|
157
|
+
});
|
|
158
|
+
});
|
|
159
|
+
|
|
160
|
+
test('URL expiring just beyond the safety margin is relayed', async () => {
|
|
161
|
+
const entry = expiringEntry(nowSec(), nowSec() + EXPIRY_MARGIN_SEC + 120);
|
|
162
|
+
await withServer(cacheWith(entry), async (port) => {
|
|
163
|
+
const r = await get(port, `/${TOKEN}/${VIDEO_ID}`, {
|
|
164
|
+
Range: 'bytes=0-9',
|
|
165
|
+
});
|
|
166
|
+
assert.equal(r.status, 206);
|
|
167
|
+
});
|
|
168
|
+
});
|
|
169
|
+
|
|
170
|
+
test('unparseable expire falls back to the timestamp window (fresh)', async () => {
|
|
171
|
+
const entry = {
|
|
172
|
+
url: `http://127.0.0.1:${upPort}/media?expire=abc`,
|
|
173
|
+
ts: nowSec(),
|
|
174
|
+
};
|
|
175
|
+
await withServer(cacheWith(entry), async (port) => {
|
|
176
|
+
const r = await get(port, `/${TOKEN}/${VIDEO_ID}`, {
|
|
177
|
+
Range: 'bytes=0-9',
|
|
178
|
+
});
|
|
179
|
+
assert.equal(r.status, 206);
|
|
180
|
+
});
|
|
181
|
+
});
|
|
182
|
+
|
|
183
|
+
test('unparseable expire falls back to the timestamp window (stale)', async () => {
|
|
184
|
+
const entry = {
|
|
185
|
+
url: `http://127.0.0.1:${upPort}/media?expire=abc`,
|
|
186
|
+
ts: nowSec() - FALLBACK_FRESH_SEC - 120,
|
|
187
|
+
};
|
|
188
|
+
await withServer(cacheWith(entry), async (port) => {
|
|
189
|
+
const r = await get(port, `/${TOKEN}/${VIDEO_ID}`);
|
|
190
|
+
assert.equal(r.status, 404);
|
|
191
|
+
});
|
|
192
|
+
});
|
|
193
|
+
|
|
115
194
|
test('empty batch cache gets 404 for every video', async () => {
|
|
116
195
|
const file = path.join(tmp, 'cache-empty.json');
|
|
117
196
|
writeCache(file, {});
|
|
@@ -193,3 +272,39 @@ test('resolveStreamUrl returns null when the file cannot be read', () => {
|
|
|
193
272
|
const file = path.join(tmp, 'no-such-cache.json');
|
|
194
273
|
assert.equal(resolveStreamUrl({ batchCacheFile: file }, VIDEO_ID), null);
|
|
195
274
|
});
|
|
275
|
+
|
|
276
|
+
test('resolveStreamUrl serves an old entry whose URL is unexpired', () => {
|
|
277
|
+
const entry = expiringEntry(nowSec() - 5 * 3600 - 600, nowSec() + 1800);
|
|
278
|
+
const file = cacheWith(entry);
|
|
279
|
+
assert.equal(resolveStreamUrl({ batchCacheFile: file }, VIDEO_ID), entry.url);
|
|
280
|
+
});
|
|
281
|
+
|
|
282
|
+
test('resolveStreamUrl returns null for a fresh entry whose URL expired', () => {
|
|
283
|
+
const entry = expiringEntry(nowSec(), nowSec() - 60);
|
|
284
|
+
const file = cacheWith(entry);
|
|
285
|
+
assert.equal(resolveStreamUrl({ batchCacheFile: file }, VIDEO_ID), null);
|
|
286
|
+
});
|
|
287
|
+
|
|
288
|
+
test('resolveStreamUrl honours an expiryMarginSec override', () => {
|
|
289
|
+
// The URL stays valid for another 30 minutes: fine with the
|
|
290
|
+
// default 60-second margin, unusable with a 1-hour margin.
|
|
291
|
+
const entry = expiringEntry(nowSec(), nowSec() + 1800);
|
|
292
|
+
const file = cacheWith(entry);
|
|
293
|
+
assert.equal(resolveStreamUrl({ batchCacheFile: file }, VIDEO_ID), entry.url);
|
|
294
|
+
assert.equal(
|
|
295
|
+
resolveStreamUrl({ batchCacheFile: file, expiryMarginSec: 3600 }, VIDEO_ID),
|
|
296
|
+
null,
|
|
297
|
+
);
|
|
298
|
+
});
|
|
299
|
+
|
|
300
|
+
test('resolveStreamUrl honours a fallbackFreshSec override', () => {
|
|
301
|
+
// Extracted 1 hour ago, no expire parameter: fine with the
|
|
302
|
+
// default 6-hour window, stale with a 30-minute window.
|
|
303
|
+
const entry = goodEntry(nowSec() - 3600);
|
|
304
|
+
const file = cacheWith(entry);
|
|
305
|
+
assert.equal(resolveStreamUrl({ batchCacheFile: file }, VIDEO_ID), entry.url);
|
|
306
|
+
assert.equal(
|
|
307
|
+
resolveStreamUrl({ batchCacheFile: file, fallbackFreshSec: 1800 }, VIDEO_ID),
|
|
308
|
+
null,
|
|
309
|
+
);
|
|
310
|
+
});
|
package/test/chunking.test.js
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
'use strict';
|
|
2
2
|
|
|
3
|
-
// The whole point of
|
|
3
|
+
// The whole point of kikoeru: the upstream is only ever asked for small
|
|
4
4
|
// ranged pieces (<= 4 MiB), never for a plain GET and never for one
|
|
5
5
|
// big range. These tests record every Range header the fake upstream
|
|
6
6
|
// receives for a > 10 MiB payload and check the pieces exactly: size,
|
|
@@ -140,3 +140,43 @@ test('a client range inside one chunk causes exactly one data request', async ()
|
|
|
140
140
|
const dataRequests = splitProbe(recordedRanges());
|
|
141
141
|
assert.deepEqual(dataRequests, [[1000, 1999]]);
|
|
142
142
|
});
|
|
143
|
+
|
|
144
|
+
test('a chunkSize override shrinks the upstream sub-ranges', async () => {
|
|
145
|
+
const small = makeData(5000);
|
|
146
|
+
const up = makeMediaUpstream(small);
|
|
147
|
+
const upPort2 = await listen(up.server);
|
|
148
|
+
const dir = tmpDir();
|
|
149
|
+
const cache = path.join(dir, 'cache.json');
|
|
150
|
+
writeCache(cache, {
|
|
151
|
+
[VIDEO_ID]: { url: `http://127.0.0.1:${upPort2}/media`, ts: nowSec() },
|
|
152
|
+
});
|
|
153
|
+
const p = createServer({
|
|
154
|
+
host: '127.0.0.1',
|
|
155
|
+
port: 0,
|
|
156
|
+
token: TOKEN,
|
|
157
|
+
tokenFile: path.join(dir, 'token'),
|
|
158
|
+
batchCacheFile: cache,
|
|
159
|
+
chunkSize: 1024,
|
|
160
|
+
});
|
|
161
|
+
const port = await listen(p);
|
|
162
|
+
try {
|
|
163
|
+
const r = await get(port, `/${TOKEN}/${VIDEO_ID}`);
|
|
164
|
+
assert.equal(r.status, 200);
|
|
165
|
+
assert.deepEqual(r.body, small);
|
|
166
|
+
const chunks = up.state.requests
|
|
167
|
+
.filter((q) => q.url === '/media')
|
|
168
|
+
.map((q) => parseRangeHeader(q.range))
|
|
169
|
+
.filter(([s, e]) => !(s === 0 && e === 0));
|
|
170
|
+
assert.deepEqual(chunks, [
|
|
171
|
+
[0, 1023],
|
|
172
|
+
[1024, 2047],
|
|
173
|
+
[2048, 3071],
|
|
174
|
+
[3072, 4095],
|
|
175
|
+
[4096, 4999],
|
|
176
|
+
]);
|
|
177
|
+
} finally {
|
|
178
|
+
await close(p);
|
|
179
|
+
await close(up.server);
|
|
180
|
+
fs.rmSync(dir, { recursive: true, force: true });
|
|
181
|
+
}
|
|
182
|
+
});
|
package/test/config.test.js
CHANGED
|
@@ -51,7 +51,7 @@ test('loadConfig uses the documented defaults', () => {
|
|
|
51
51
|
assert.equal(cfg.token, null);
|
|
52
52
|
assert.equal(
|
|
53
53
|
cfg.tokenFile,
|
|
54
|
-
path.join(os.homedir(), '.config/
|
|
54
|
+
path.join(os.homedir(), '.config/kikoeru/token'),
|
|
55
55
|
);
|
|
56
56
|
assert.equal(
|
|
57
57
|
cfg.batchCacheFile,
|
|
@@ -61,10 +61,10 @@ test('loadConfig uses the documented defaults', () => {
|
|
|
61
61
|
|
|
62
62
|
test('loadConfig honours every environment override', () => {
|
|
63
63
|
const cfg = loadConfig({
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
64
|
+
KIKOERU_HOST: '0.0.0.0',
|
|
65
|
+
KIKOERU_PORT: '9999',
|
|
66
|
+
KIKOERU_TOKEN: 'from-env',
|
|
67
|
+
KIKOERU_TOKEN_FILE: '/tmp/somewhere/token',
|
|
68
68
|
YT_AUDIO_CACHE: '/tmp/somewhere/cache.json',
|
|
69
69
|
});
|
|
70
70
|
assert.equal(cfg.host, '0.0.0.0');
|
|
@@ -74,6 +74,43 @@ test('loadConfig honours every environment override', () => {
|
|
|
74
74
|
assert.equal(cfg.batchCacheFile, '/tmp/somewhere/cache.json');
|
|
75
75
|
});
|
|
76
76
|
|
|
77
|
+
test('loadConfig defaults the numeric knobs to the built-in values', () => {
|
|
78
|
+
const cfg = loadConfig({});
|
|
79
|
+
assert.equal(cfg.chunkSize, 4 * 1024 * 1024);
|
|
80
|
+
assert.equal(cfg.fallbackFreshSec, 6 * 3600);
|
|
81
|
+
assert.equal(cfg.expiryMarginSec, 60);
|
|
82
|
+
assert.equal(cfg.metaTtlMs, 50 * 60 * 1000);
|
|
83
|
+
});
|
|
84
|
+
|
|
85
|
+
test('loadConfig honours the numeric knob overrides', () => {
|
|
86
|
+
const cfg = loadConfig({
|
|
87
|
+
KIKOERU_CHUNK_SIZE: '1048576',
|
|
88
|
+
KIKOERU_FALLBACK_FRESH_SEC: '7200',
|
|
89
|
+
KIKOERU_EXPIRY_MARGIN_SEC: '300',
|
|
90
|
+
KIKOERU_META_TTL_MS: '60000',
|
|
91
|
+
});
|
|
92
|
+
assert.equal(cfg.chunkSize, 1048576);
|
|
93
|
+
assert.equal(cfg.fallbackFreshSec, 7200);
|
|
94
|
+
assert.equal(cfg.expiryMarginSec, 300);
|
|
95
|
+
assert.equal(cfg.metaTtlMs, 60000);
|
|
96
|
+
});
|
|
97
|
+
|
|
98
|
+
test('loadConfig falls back to the defaults for invalid knob values', () => {
|
|
99
|
+
for (const bad of ['abc', '0', '-5', '']) {
|
|
100
|
+
const cfg = loadConfig({
|
|
101
|
+
KIKOERU_CHUNK_SIZE: bad,
|
|
102
|
+
KIKOERU_FALLBACK_FRESH_SEC: bad,
|
|
103
|
+
KIKOERU_EXPIRY_MARGIN_SEC: bad,
|
|
104
|
+
KIKOERU_META_TTL_MS: bad,
|
|
105
|
+
});
|
|
106
|
+
const label = JSON.stringify(bad);
|
|
107
|
+
assert.equal(cfg.chunkSize, 4 * 1024 * 1024, `chunkSize for ${label}`);
|
|
108
|
+
assert.equal(cfg.fallbackFreshSec, 6 * 3600, `fallbackFreshSec for ${label}`);
|
|
109
|
+
assert.equal(cfg.expiryMarginSec, 60, `expiryMarginSec for ${label}`);
|
|
110
|
+
assert.equal(cfg.metaTtlMs, 50 * 60 * 1000, `metaTtlMs for ${label}`);
|
|
111
|
+
}
|
|
112
|
+
});
|
|
113
|
+
|
|
77
114
|
// --- ensureToken ----------------------------------------------------------------
|
|
78
115
|
|
|
79
116
|
test('ensureToken returns the configured token without touching the file', () => {
|
package/test/helpers.js
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
'use strict';
|
|
2
2
|
|
|
3
|
-
// Shared helpers for the
|
|
3
|
+
// Shared helpers for the kikoeru test suites. This file is not a test
|
|
4
4
|
// itself (it does not match the *.test.js glob); it only builds fake
|
|
5
5
|
// upstreams, temp dirs and small HTTP clients so every suite can stay
|
|
6
6
|
// off the real network.
|
|
@@ -81,7 +81,7 @@ function makeData(size) {
|
|
|
81
81
|
}
|
|
82
82
|
|
|
83
83
|
function tmpDir() {
|
|
84
|
-
return fs.mkdtempSync(path.join(os.tmpdir(), '
|
|
84
|
+
return fs.mkdtempSync(path.join(os.tmpdir(), 'kikoeru-test-'));
|
|
85
85
|
}
|
|
86
86
|
|
|
87
87
|
function writeCache(file, obj) {
|
|
@@ -104,12 +104,15 @@ function makeMediaUpstream(data, contentType = 'audio/mp4') {
|
|
|
104
104
|
const range = req.headers.range || null;
|
|
105
105
|
state.requests.push({ url: req.url, range });
|
|
106
106
|
if (!range) state.plainGets++;
|
|
107
|
-
|
|
107
|
+
// Route on the pathname only: stream URLs may carry a query
|
|
108
|
+
// string (e.g. ?expire=...) and must still reach /media.
|
|
109
|
+
const pathname = (req.url || '').split('?')[0];
|
|
110
|
+
if (pathname === '/redir') {
|
|
108
111
|
res.writeHead(302, { Location: '/media' });
|
|
109
112
|
res.end();
|
|
110
113
|
return;
|
|
111
114
|
}
|
|
112
|
-
if (
|
|
115
|
+
if (pathname !== '/media') {
|
|
113
116
|
res.writeHead(404);
|
|
114
117
|
res.end();
|
|
115
118
|
return;
|
package/test/upstream.test.js
CHANGED
|
@@ -34,7 +34,7 @@ let upstream;
|
|
|
34
34
|
let upPort;
|
|
35
35
|
let tmp;
|
|
36
36
|
let hits; // per-path request counts
|
|
37
|
-
let cacheFile; // the batch cache the running
|
|
37
|
+
let cacheFile; // the batch cache the running kikoeru reads
|
|
38
38
|
let onFlakyChunk; // hook fired when /flaky gets a data (non-probe) request
|
|
39
39
|
|
|
40
40
|
function serveRange(req, res) {
|
|
@@ -121,7 +121,7 @@ after(async () => {
|
|
|
121
121
|
fs.rmSync(tmp, { recursive: true, force: true });
|
|
122
122
|
});
|
|
123
123
|
|
|
124
|
-
// Start an
|
|
124
|
+
// Start an kikoeru whose cache points VIDEO_ID at `upstreamPath`.
|
|
125
125
|
async function withServerAt(upstreamPath, fn) {
|
|
126
126
|
writeCache(cacheFile, {
|
|
127
127
|
[VIDEO_ID]: {
|
|
@@ -188,14 +188,14 @@ test('403 on the first data chunk retries with the refreshed cache URL', async (
|
|
|
188
188
|
|
|
189
189
|
test('403 on a data chunk with no cache refresh gives a prompt 502', async () => {
|
|
190
190
|
// The retry re-reads the same dead URL and fails again. Nothing
|
|
191
|
-
// has been sent to the client at that point, so
|
|
191
|
+
// has been sent to the client at that point, so kikoeru must answer
|
|
192
192
|
// promptly with 502 instead of leaving the request hanging.
|
|
193
193
|
hits = {};
|
|
194
194
|
onFlakyChunk = () => {};
|
|
195
195
|
try {
|
|
196
196
|
await withServerAt('/flaky', async (port) => {
|
|
197
197
|
const r = await getExpectTimeout(port, `/${TOKEN}/${VIDEO_ID}`);
|
|
198
|
-
assert.equal(r.timedOut, false, '
|
|
198
|
+
assert.equal(r.timedOut, false, 'kikoeru must answer, not hang');
|
|
199
199
|
assert.equal(r.status, 502);
|
|
200
200
|
});
|
|
201
201
|
} finally {
|
|
@@ -215,7 +215,7 @@ test('500 on a data chunk gives a prompt 502', async () => {
|
|
|
215
215
|
// is sent to the client, so it is answered with a prompt 502.
|
|
216
216
|
await withServerAt('/chunk500', async (port) => {
|
|
217
217
|
const r = await getExpectTimeout(port, `/${TOKEN}/${VIDEO_ID}`);
|
|
218
|
-
assert.equal(r.timedOut, false, '
|
|
218
|
+
assert.equal(r.timedOut, false, 'kikoeru must answer, not hang');
|
|
219
219
|
assert.equal(r.status, 502);
|
|
220
220
|
});
|
|
221
221
|
});
|
|
@@ -259,7 +259,7 @@ test('500 on a later chunk cuts the response short instead of hanging', async ()
|
|
|
259
259
|
// The payload is one chunk plus a little more, so the first chunk
|
|
260
260
|
// is streamed to the client and the failure only happens in
|
|
261
261
|
// streamRest, after the 200 status and the first bytes are out.
|
|
262
|
-
// No error status can be sent then;
|
|
262
|
+
// No error status can be sent then; kikoeru must cut the response so
|
|
263
263
|
// the client sees it end (incomplete) right away, not hang.
|
|
264
264
|
const bigSize = CHUNK_SIZE + 1024;
|
|
265
265
|
const big = makeData(bigSize);
|
|
@@ -298,7 +298,7 @@ test('500 on a later chunk cuts the response short instead of hanging', async ()
|
|
|
298
298
|
const port = await listen(p);
|
|
299
299
|
try {
|
|
300
300
|
const r = await getObserveEnd(port, `/${TOKEN}/${VIDEO_ID}`);
|
|
301
|
-
assert.equal(r.timedOut, false, '
|
|
301
|
+
assert.equal(r.timedOut, false, 'kikoeru must end the response, not hang');
|
|
302
302
|
assert.equal(r.status, 200);
|
|
303
303
|
assert.equal(r.bytes, CHUNK_SIZE, 'only the first chunk was delivered');
|
|
304
304
|
assert.equal(r.complete, false, 'the body ends incomplete');
|