hdm-cli 0.0.0-stage → 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +147 -2
- package/dist/cli.d.ts +2 -0
- package/dist/cli.js +118 -0
- package/dist/index.d.ts +36 -0
- package/dist/index.js +277 -0
- package/dist/progress.d.ts +9 -0
- package/dist/progress.js +67 -0
- package/package.json +37 -4
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Max
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,3 +1,148 @@
|
|
|
1
|
-
#
|
|
1
|
+
# hdm
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
hdm(Hyper Download Manager)是一个小巧的 Node.js 多连接文件下载器,既可以当命令行工具用,也可以当库调用。npm 包名是 `hdm-cli`,安装后的命令是 `hdm`。
|
|
4
|
+
|
|
5
|
+
- 通过 HTTP Range 请求把文件拆给多个连接并行下载
|
|
6
|
+
- 下载中断后可以从断点继续
|
|
7
|
+
- 连接掉线、卡住或请求失败时自动重试
|
|
8
|
+
- 显示进度条、下载速度和剩余时间
|
|
9
|
+
- 没有运行时依赖,发布包约 11 kB
|
|
10
|
+
|
|
11
|
+
需要 Node.js 20 或更高版本。
|
|
12
|
+
|
|
13
|
+
## 安装
|
|
14
|
+
|
|
15
|
+
```bash
|
|
16
|
+
npm install -g hdm-cli
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
也可以不安装,直接运行一次:
|
|
20
|
+
|
|
21
|
+
```bash
|
|
22
|
+
npx hdm-cli https://example.com/file.zip
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
## 命令行
|
|
26
|
+
|
|
27
|
+
```
|
|
28
|
+
hdm [选项] <url>
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
| 选项 | 说明 |
|
|
32
|
+
| --- | --- |
|
|
33
|
+
| `-o, --output <path>` | 输出文件,或一个已存在的目录。默认用远程文件名保存到当前目录。 |
|
|
34
|
+
| `-c, --connections <n>` | 并行连接数,1–32,默认 8。 |
|
|
35
|
+
| `-H, --header <line>` | 额外的请求头,例如 `-H "Cookie: a=b"`。可以重复使用。 |
|
|
36
|
+
| `-f, --force` | 输出文件已存在时覆盖它。 |
|
|
37
|
+
| `-h, --help` | 显示帮助。 |
|
|
38
|
+
| `-v, --version` | 显示版本号。 |
|
|
39
|
+
|
|
40
|
+
```bash
|
|
41
|
+
# 保存到某个目录,沿用远程文件名
|
|
42
|
+
hdm -o ~/Downloads https://example.com/file.zip
|
|
43
|
+
|
|
44
|
+
# 使用 16 个连接,并带上请求头
|
|
45
|
+
hdm -c 16 -H "Authorization: Bearer TOKEN" https://example.com/file.zip
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
```
|
|
49
|
+
file.zip(24.0 MB,8 个连接)
|
|
50
|
+
██████████████████████████████░░░░░░░░░░ 75.0% 18.0 MB / 24.0 MB 8.66 MB/s ETA 0:01
|
|
51
|
+
已保存 file.zip(24.0 MB,用时 0:03,平均 8.40 MB/s)
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
进度条只在输出到终端时绘制。在管道或 CI 日志里,只打印开头和结尾两行。所有信息都输出到标准错误。
|
|
55
|
+
|
|
56
|
+
### 断点续传
|
|
57
|
+
|
|
58
|
+
按 Ctrl-C 停止下载,再次运行同一条命令即可继续。进度保存在输出文件旁边的两个文件里:`<文件名>.part` 和 `<文件名>.part.json`,下载完成后会自动删除。
|
|
59
|
+
|
|
60
|
+
如果远程文件发生了变化,下载会从头开始,而不是续传。判断依据是文件大小、`ETag` 和 `Last-Modified` 响应头。
|
|
61
|
+
|
|
62
|
+
### 退出码
|
|
63
|
+
|
|
64
|
+
| 退出码 | 含义 |
|
|
65
|
+
| --- | --- |
|
|
66
|
+
| 0 | 下载成功。 |
|
|
67
|
+
| 1 | 下载失败。 |
|
|
68
|
+
| 2 | 参数无效。 |
|
|
69
|
+
| 130 | 被 Ctrl-C 中断。 |
|
|
70
|
+
|
|
71
|
+
## 作为库使用
|
|
72
|
+
|
|
73
|
+
```js
|
|
74
|
+
import { download } from 'hdm-cli';
|
|
75
|
+
|
|
76
|
+
const { path, size } = await download('https://example.com/file.zip', {
|
|
77
|
+
output: 'downloads/',
|
|
78
|
+
connections: 8,
|
|
79
|
+
onProgress: ({ downloaded, total }) => {
|
|
80
|
+
console.log(`${downloaded} / ${total}`);
|
|
81
|
+
},
|
|
82
|
+
});
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
### `download(url, options?)`
|
|
86
|
+
|
|
87
|
+
返回一个 Promise,结果为 `{ path, size }`:保存后文件的绝对路径和字节数。
|
|
88
|
+
|
|
89
|
+
| 选项 | 类型 | 默认值 | 说明 |
|
|
90
|
+
| --- | --- | --- | --- |
|
|
91
|
+
| `output` | `string` | 远程文件名 | 输出文件,或一个已存在的目录。 |
|
|
92
|
+
| `connections` | `number` | `8` | 最大并行连接数。 |
|
|
93
|
+
| `headers` | `Record<string, string>` | | 额外的请求头。 |
|
|
94
|
+
| `retries` | `number` | `5` | 每个连接允许的连续失败次数,超过后放弃。 |
|
|
95
|
+
| `timeout` | `number` | `30000` | 连接在这么多毫秒内没有收到数据就会被重启。 |
|
|
96
|
+
| `force` | `boolean` | `false` | 输出文件已存在时覆盖它。 |
|
|
97
|
+
| `signal` | `AbortSignal` | | 用于取消下载。已下载的部分会保留,供续传使用。 |
|
|
98
|
+
| `onStart` | `(info) => void` | | 在写入任何数据之前调用一次。 |
|
|
99
|
+
| `onProgress` | `(progress) => void` | | 每次写入数据后调用。 |
|
|
100
|
+
|
|
101
|
+
`onStart` 收到的参数:
|
|
102
|
+
|
|
103
|
+
| 字段 | 类型 | 说明 |
|
|
104
|
+
| --- | --- | --- |
|
|
105
|
+
| `path` | `string` | 文件将保存到的路径。 |
|
|
106
|
+
| `total` | `number \| null` | 文件字节数;服务器没有给出时为 `null`。 |
|
|
107
|
+
| `resumed` | `number` | 上一次运行已经下载到磁盘的字节数。 |
|
|
108
|
+
| `connections` | `number` | 本次下载使用的连接数。 |
|
|
109
|
+
| `resumable` | `boolean` | 中断后能否续传。 |
|
|
110
|
+
|
|
111
|
+
`onProgress` 收到 `{ downloaded, total }`,单位都是字节。`downloaded` 包含续传前已有的字节。
|
|
112
|
+
|
|
113
|
+
### 取消下载
|
|
114
|
+
|
|
115
|
+
```js
|
|
116
|
+
const controller = new AbortController();
|
|
117
|
+
setTimeout(() => controller.abort(), 5000);
|
|
118
|
+
|
|
119
|
+
try {
|
|
120
|
+
await download(url, { signal: controller.signal });
|
|
121
|
+
} catch (err) {
|
|
122
|
+
if (err.name !== 'AbortError') throw err;
|
|
123
|
+
// 用相同的参数再次调用 download() 即可续传。
|
|
124
|
+
}
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
## 行为说明
|
|
128
|
+
|
|
129
|
+
- **连接数。** 每个连接至少负责 1 MiB,所以小文件实际使用的连接数会少于设定值。
|
|
130
|
+
- **不支持 Range 的服务器。** 改用单个连接下载,中断后只能从头开始。
|
|
131
|
+
- **重试。** 遇到网络错误、连接卡住,或 HTTP 408、429、5xx 响应时会重试,连续失败的次数越多,等待越久。其他 HTTP 错误会让下载立即失败。
|
|
132
|
+
- **文件名。** 没有指定输出文件时,文件名先取自 `Content-Disposition` 响应头,其次取自 URL。可能让文件落到目标目录之外的字符会被替换掉。
|
|
133
|
+
- **已存在的文件。** 除非设置了 `force`,否则拒绝下载。
|
|
134
|
+
- **代理。** 没有代理选项。在支持的 Node.js 版本上,设置 `NODE_USE_ENV_PROXY=1` 后,Node.js 自身会读取 `HTTP_PROXY`、`HTTPS_PROXY` 和 `NO_PROXY`。
|
|
135
|
+
- **协议。** 只支持 HTTP 和 HTTPS。
|
|
136
|
+
|
|
137
|
+
## 开发
|
|
138
|
+
|
|
139
|
+
```bash
|
|
140
|
+
npm install
|
|
141
|
+
npm test
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
`npm test` 会编译 TypeScript 源码,并针对一个本地 HTTP 服务器运行测试。
|
|
145
|
+
|
|
146
|
+
## 许可证
|
|
147
|
+
|
|
148
|
+
[MIT](LICENSE)
|
package/dist/cli.d.ts
ADDED
package/dist/cli.js
ADDED
|
@@ -0,0 +1,118 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
import { readFile } from 'node:fs/promises';
|
|
3
|
+
import { relative } from 'node:path';
|
|
4
|
+
import { parseArgs } from 'node:util';
|
|
5
|
+
import { download } from './index.js';
|
|
6
|
+
import { createProgressBar, formatBytes, formatDuration } from './progress.js';
|
|
7
|
+
const MAX_CONNECTIONS = 32;
|
|
8
|
+
const pkg = JSON.parse(await readFile(new URL('../package.json', import.meta.url), 'utf8'));
|
|
9
|
+
// The command name shown in messages is the one package.json installs.
|
|
10
|
+
const command = Object.keys(pkg.bin)[0];
|
|
11
|
+
const usage = `用法:${command} [选项] <url>
|
|
12
|
+
|
|
13
|
+
选项:
|
|
14
|
+
-o, --output <path> 输出文件,或要保存到的目录
|
|
15
|
+
-c, --connections <n> 并行连接数,1-${MAX_CONNECTIONS}(默认 8)
|
|
16
|
+
-H, --header <line> 额外的请求头,例如 -H "Cookie: a=b"(可重复)
|
|
17
|
+
-f, --force 输出文件已存在时覆盖
|
|
18
|
+
-h, --help 显示帮助
|
|
19
|
+
-v, --version 显示版本号
|
|
20
|
+
`;
|
|
21
|
+
function fail(message, code = 1) {
|
|
22
|
+
console.error(`${command}: ${message}`);
|
|
23
|
+
process.exit(code);
|
|
24
|
+
}
|
|
25
|
+
function parse() {
|
|
26
|
+
try {
|
|
27
|
+
return parseArgs({
|
|
28
|
+
allowPositionals: true,
|
|
29
|
+
options: {
|
|
30
|
+
output: { type: 'string', short: 'o' },
|
|
31
|
+
connections: { type: 'string', short: 'c', default: '8' },
|
|
32
|
+
header: { type: 'string', short: 'H', multiple: true, default: [] },
|
|
33
|
+
force: { type: 'boolean', short: 'f', default: false },
|
|
34
|
+
help: { type: 'boolean', short: 'h', default: false },
|
|
35
|
+
version: { type: 'boolean', short: 'v', default: false },
|
|
36
|
+
},
|
|
37
|
+
});
|
|
38
|
+
}
|
|
39
|
+
catch (err) {
|
|
40
|
+
// parseArgs reports in English; the option it complains about is the quoted part of the message.
|
|
41
|
+
const { code, message } = err;
|
|
42
|
+
const option = /'([^']+)'/.exec(message)?.[1] ?? '';
|
|
43
|
+
const reason = code === 'ERR_PARSE_ARGS_UNKNOWN_OPTION'
|
|
44
|
+
? `未知选项 ${option}`
|
|
45
|
+
: code === 'ERR_PARSE_ARGS_INVALID_OPTION_VALUE'
|
|
46
|
+
? `选项 ${option} 的参数缺失或无效`
|
|
47
|
+
: message;
|
|
48
|
+
fail(`${reason}\n\n${usage}`, 2);
|
|
49
|
+
}
|
|
50
|
+
}
|
|
51
|
+
const { values, positionals } = parse();
|
|
52
|
+
if (values.help) {
|
|
53
|
+
console.log(usage);
|
|
54
|
+
process.exit(0);
|
|
55
|
+
}
|
|
56
|
+
if (values.version) {
|
|
57
|
+
console.log(pkg.version);
|
|
58
|
+
process.exit(0);
|
|
59
|
+
}
|
|
60
|
+
if (positionals.length !== 1)
|
|
61
|
+
fail(`需要且只能提供一个 URL\n\n${usage}`, 2);
|
|
62
|
+
const url = positionals[0];
|
|
63
|
+
if (!URL.canParse(url) || !/^https?:$/.test(new URL(url).protocol)) {
|
|
64
|
+
fail(`不是 http(s) 地址:${url}`, 2);
|
|
65
|
+
}
|
|
66
|
+
const connections = Number(values.connections);
|
|
67
|
+
if (!Number.isInteger(connections) || connections < 1 || connections > MAX_CONNECTIONS) {
|
|
68
|
+
fail(`--connections 必须是 1 到 ${MAX_CONNECTIONS} 之间的整数`, 2);
|
|
69
|
+
}
|
|
70
|
+
const headers = {};
|
|
71
|
+
for (const line of values.header) {
|
|
72
|
+
const colon = line.indexOf(':');
|
|
73
|
+
if (colon < 1)
|
|
74
|
+
fail(`请求头格式无效(应为 "名称: 值"):${line}`, 2);
|
|
75
|
+
headers[line.slice(0, colon).trim()] = line.slice(colon + 1).trim();
|
|
76
|
+
}
|
|
77
|
+
const ctl = new AbortController();
|
|
78
|
+
// Only the first Ctrl-C is handled, so a second one still kills the process outright.
|
|
79
|
+
process.once('SIGINT', () => ctl.abort());
|
|
80
|
+
const started = Date.now();
|
|
81
|
+
let info;
|
|
82
|
+
let bar;
|
|
83
|
+
try {
|
|
84
|
+
const { path, size } = await download(url, {
|
|
85
|
+
output: values.output,
|
|
86
|
+
connections,
|
|
87
|
+
headers,
|
|
88
|
+
force: values.force,
|
|
89
|
+
signal: ctl.signal,
|
|
90
|
+
onStart(start) {
|
|
91
|
+
info = start;
|
|
92
|
+
const size = start.total === null ? '大小未知' : formatBytes(start.total);
|
|
93
|
+
const resumed = start.resumed ? `,从 ${formatBytes(start.resumed)} 处续传` : '';
|
|
94
|
+
console.error(`${shorten(start.path)}(${size},${start.connections} 个连接${resumed})`);
|
|
95
|
+
bar = createProgressBar(start.total, start.resumed);
|
|
96
|
+
},
|
|
97
|
+
onProgress: (progress) => bar?.update(progress.downloaded),
|
|
98
|
+
});
|
|
99
|
+
bar?.stop();
|
|
100
|
+
const seconds = (Date.now() - started) / 1000;
|
|
101
|
+
const speed = formatBytes((size - (info?.resumed ?? 0)) / Math.max(seconds, 0.001));
|
|
102
|
+
console.error(`已保存 ${shorten(path)}(${formatBytes(size)},用时 ${formatDuration(seconds)},平均 ${speed}/s)`);
|
|
103
|
+
}
|
|
104
|
+
catch (err) {
|
|
105
|
+
bar?.stop();
|
|
106
|
+
if (ctl.signal.aborted) {
|
|
107
|
+
console.error(info?.resumable ? '已中断。再次运行同一条命令即可续传。' : '已中断。');
|
|
108
|
+
process.exit(130);
|
|
109
|
+
}
|
|
110
|
+
const { message, cause } = err;
|
|
111
|
+
// fetch reports network failures as a bare "fetch failed" with the real reason in `cause`.
|
|
112
|
+
fail(cause instanceof Error ? `请求失败:${cause.message}` : message);
|
|
113
|
+
}
|
|
114
|
+
/** Shows a path relative to the working directory when that is the shorter form. */
|
|
115
|
+
function shorten(path) {
|
|
116
|
+
const rel = relative('', path);
|
|
117
|
+
return rel && !rel.startsWith('..') ? rel : path;
|
|
118
|
+
}
|
package/dist/index.d.ts
ADDED
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
export interface DownloadOptions {
|
|
2
|
+
/** Output file, or an existing directory to save into. Defaults to the remote file name in the current directory. */
|
|
3
|
+
output?: string;
|
|
4
|
+
/** Maximum parallel connections. Default 8. */
|
|
5
|
+
connections?: number;
|
|
6
|
+
/** Extra request headers. */
|
|
7
|
+
headers?: Record<string, string>;
|
|
8
|
+
/** Consecutive failures tolerated per connection before giving up. Default 5. */
|
|
9
|
+
retries?: number;
|
|
10
|
+
/** Milliseconds a connection may stay silent before it is restarted. Default 30000. */
|
|
11
|
+
timeout?: number;
|
|
12
|
+
/** Overwrite the output file if it already exists. */
|
|
13
|
+
force?: boolean;
|
|
14
|
+
signal?: AbortSignal;
|
|
15
|
+
onStart?: (info: StartInfo) => void;
|
|
16
|
+
onProgress?: (progress: Progress) => void;
|
|
17
|
+
}
|
|
18
|
+
export interface StartInfo {
|
|
19
|
+
path: string;
|
|
20
|
+
/** Size in bytes, or null when the server does not report one. */
|
|
21
|
+
total: number | null;
|
|
22
|
+
/** Bytes already on disk from an earlier, interrupted run. */
|
|
23
|
+
resumed: number;
|
|
24
|
+
connections: number;
|
|
25
|
+
/** Whether an interrupted download can be continued by calling `download` again. */
|
|
26
|
+
resumable: boolean;
|
|
27
|
+
}
|
|
28
|
+
export interface Progress {
|
|
29
|
+
downloaded: number;
|
|
30
|
+
total: number | null;
|
|
31
|
+
}
|
|
32
|
+
export interface DownloadResult {
|
|
33
|
+
path: string;
|
|
34
|
+
size: number;
|
|
35
|
+
}
|
|
36
|
+
export declare function download(url: string, options?: DownloadOptions): Promise<DownloadResult>;
|
package/dist/index.js
ADDED
|
@@ -0,0 +1,277 @@
|
|
|
1
|
+
import { open, readFile, rename, rm, stat, writeFile } from 'node:fs/promises';
|
|
2
|
+
import { resolve } from 'node:path';
|
|
3
|
+
import { setTimeout as sleep } from 'node:timers/promises';
|
|
4
|
+
/** An error that retrying the same request cannot fix. */
|
|
5
|
+
class FatalError extends Error {
|
|
6
|
+
}
|
|
7
|
+
const MIN_SEGMENT = 1024 * 1024;
|
|
8
|
+
const SAVE_INTERVAL = 1000;
|
|
9
|
+
export async function download(url, options = {}) {
|
|
10
|
+
const { connections = 8, timeout = 30_000 } = options;
|
|
11
|
+
if (!Number.isInteger(connections) || connections < 1) {
|
|
12
|
+
throw new RangeError('connections 必须是正整数');
|
|
13
|
+
}
|
|
14
|
+
// Ranges address the bytes on the wire, so compressed transfer would break the offsets.
|
|
15
|
+
const headers = new Headers(options.headers);
|
|
16
|
+
headers.set('accept-encoding', 'identity');
|
|
17
|
+
const ctl = new AbortController();
|
|
18
|
+
const stop = () => ctl.abort(options.signal?.reason);
|
|
19
|
+
options.signal?.addEventListener('abort', stop, { once: true });
|
|
20
|
+
if (options.signal?.aborted)
|
|
21
|
+
stop();
|
|
22
|
+
let req;
|
|
23
|
+
try {
|
|
24
|
+
// Ask for the first byte only: a 206 reply proves ranges work and reports the full size.
|
|
25
|
+
req = await request(url, withRange(headers, 0, 0), ctl.signal, timeout);
|
|
26
|
+
const total = req.res.status === 206 ? contentRange(req.res)?.total : undefined;
|
|
27
|
+
if (total === undefined && (req.res.status === 206 || req.res.status === 416)) {
|
|
28
|
+
// Ranges are unusable (unknown size, or an empty file): start over with a plain request.
|
|
29
|
+
req.close();
|
|
30
|
+
req = await request(url, headers, ctl.signal, timeout);
|
|
31
|
+
}
|
|
32
|
+
if (!req.res.ok)
|
|
33
|
+
throw new Error(httpStatus(req.res));
|
|
34
|
+
const path = await resolveOutput(options.output, req.res);
|
|
35
|
+
if (!options.force && (await exists(path))) {
|
|
36
|
+
throw new Error(`文件已存在:${path}`);
|
|
37
|
+
}
|
|
38
|
+
if (total === undefined)
|
|
39
|
+
return await downloadStream(req, path, options);
|
|
40
|
+
req.close();
|
|
41
|
+
return await downloadRanges(url, headers, path, total, req.res, ctl, options);
|
|
42
|
+
}
|
|
43
|
+
finally {
|
|
44
|
+
req?.close();
|
|
45
|
+
options.signal?.removeEventListener('abort', stop);
|
|
46
|
+
}
|
|
47
|
+
}
|
|
48
|
+
/** Fallback for servers without range support: one connection, no resume. */
|
|
49
|
+
async function downloadStream(req, path, options) {
|
|
50
|
+
const { res } = req;
|
|
51
|
+
const length = res.headers.get('content-length');
|
|
52
|
+
// A compressed body is decoded by fetch, so its Content-Length says nothing about our byte count.
|
|
53
|
+
const total = length && !res.headers.has('content-encoding') ? Number(length) : null;
|
|
54
|
+
options.onStart?.({ path, total, resumed: 0, connections: 1, resumable: false });
|
|
55
|
+
const part = `${path}.part`;
|
|
56
|
+
const file = await open(part, 'w');
|
|
57
|
+
let downloaded = 0;
|
|
58
|
+
try {
|
|
59
|
+
for await (const chunk of res.body ?? []) {
|
|
60
|
+
req.alive();
|
|
61
|
+
await writeAll(file, chunk, downloaded);
|
|
62
|
+
downloaded += chunk.length;
|
|
63
|
+
options.onProgress?.({ downloaded, total });
|
|
64
|
+
}
|
|
65
|
+
}
|
|
66
|
+
finally {
|
|
67
|
+
await file.close();
|
|
68
|
+
}
|
|
69
|
+
if (total !== null && downloaded !== total) {
|
|
70
|
+
throw new Error(`连接提前关闭(已收到 ${downloaded} / ${total} 字节)`);
|
|
71
|
+
}
|
|
72
|
+
await rename(part, path);
|
|
73
|
+
return { path, size: downloaded };
|
|
74
|
+
}
|
|
75
|
+
async function downloadRanges(url, headers, path, total, probe, ctl, options) {
|
|
76
|
+
const { connections = 8, retries = 5, timeout = 30_000, onProgress } = options;
|
|
77
|
+
const { signal } = ctl;
|
|
78
|
+
const part = `${path}.part`;
|
|
79
|
+
const statePath = `${part}.json`;
|
|
80
|
+
const fresh = {
|
|
81
|
+
url,
|
|
82
|
+
total,
|
|
83
|
+
etag: probe.headers.get('etag'),
|
|
84
|
+
lastModified: probe.headers.get('last-modified'),
|
|
85
|
+
segments: split(total, connections),
|
|
86
|
+
};
|
|
87
|
+
const state = (await loadState(statePath, part, fresh)) ?? fresh;
|
|
88
|
+
if (state === fresh) {
|
|
89
|
+
const file = await open(part, 'w');
|
|
90
|
+
try {
|
|
91
|
+
await file.truncate(total);
|
|
92
|
+
}
|
|
93
|
+
finally {
|
|
94
|
+
await file.close();
|
|
95
|
+
}
|
|
96
|
+
}
|
|
97
|
+
const pending = state.segments.filter((seg) => seg.done < seg.end - seg.start + 1);
|
|
98
|
+
let downloaded = state.segments.reduce((sum, seg) => sum + seg.done, 0);
|
|
99
|
+
options.onStart?.({ path, total, resumed: downloaded, connections: pending.length, resumable: true });
|
|
100
|
+
const fetchRange = async (seg, file) => {
|
|
101
|
+
const from = seg.start + seg.done;
|
|
102
|
+
const req = await request(url, withRange(headers, from, seg.end), signal, timeout);
|
|
103
|
+
try {
|
|
104
|
+
const { res } = req;
|
|
105
|
+
if (res.status === 200)
|
|
106
|
+
throw new FatalError('服务器不再支持分片请求');
|
|
107
|
+
if (res.status !== 206) {
|
|
108
|
+
const retryable = res.status === 408 || res.status === 429 || res.status >= 500;
|
|
109
|
+
throw new (retryable ? Error : FatalError)(httpStatus(res));
|
|
110
|
+
}
|
|
111
|
+
const range = contentRange(res);
|
|
112
|
+
if (range?.start !== from || range.total !== total) {
|
|
113
|
+
throw new FatalError('下载过程中远程文件发生了变化');
|
|
114
|
+
}
|
|
115
|
+
for await (let chunk of res.body ?? []) {
|
|
116
|
+
req.alive();
|
|
117
|
+
const left = seg.end - seg.start + 1 - seg.done;
|
|
118
|
+
if (left === 0)
|
|
119
|
+
break;
|
|
120
|
+
if (chunk.length > left)
|
|
121
|
+
chunk = chunk.subarray(0, left);
|
|
122
|
+
await writeAll(file, chunk, seg.start + seg.done);
|
|
123
|
+
seg.done += chunk.length;
|
|
124
|
+
downloaded += chunk.length;
|
|
125
|
+
onProgress?.({ downloaded, total });
|
|
126
|
+
}
|
|
127
|
+
}
|
|
128
|
+
finally {
|
|
129
|
+
req.close();
|
|
130
|
+
}
|
|
131
|
+
};
|
|
132
|
+
const fetchSegment = async (seg) => {
|
|
133
|
+
const size = seg.end - seg.start + 1;
|
|
134
|
+
const file = await open(part, 'r+');
|
|
135
|
+
try {
|
|
136
|
+
for (let failures = 0; seg.done < size;) {
|
|
137
|
+
const before = seg.done;
|
|
138
|
+
let error = new Error('服务器未发送数据就关闭了连接');
|
|
139
|
+
try {
|
|
140
|
+
await fetchRange(seg, file);
|
|
141
|
+
}
|
|
142
|
+
catch (err) {
|
|
143
|
+
if (signal.aborted || err instanceof FatalError)
|
|
144
|
+
throw err;
|
|
145
|
+
error = err;
|
|
146
|
+
}
|
|
147
|
+
if (seg.done > before)
|
|
148
|
+
failures = 0;
|
|
149
|
+
else if (++failures > retries)
|
|
150
|
+
throw error;
|
|
151
|
+
if (seg.done < size)
|
|
152
|
+
await sleep(Math.min(250 * 2 ** failures, 15_000), undefined, { signal });
|
|
153
|
+
}
|
|
154
|
+
}
|
|
155
|
+
finally {
|
|
156
|
+
await file.close();
|
|
157
|
+
}
|
|
158
|
+
};
|
|
159
|
+
// Saves are chained so two never write the temp file at once. A failed periodic save is
|
|
160
|
+
// ignored: it only costs resume progress, and a full disk fails the data writes anyway.
|
|
161
|
+
let saving = Promise.resolve();
|
|
162
|
+
const save = () => (saving = saving.then(() => saveState(statePath, state)).catch(() => { }));
|
|
163
|
+
const timer = setInterval(save, SAVE_INTERVAL);
|
|
164
|
+
try {
|
|
165
|
+
// The first failure aborts the sibling connections; waiting for all keeps the file handles tidy.
|
|
166
|
+
await Promise.all(pending.map((seg) => fetchSegment(seg).catch((err) => ctl.abort(err))));
|
|
167
|
+
}
|
|
168
|
+
finally {
|
|
169
|
+
clearInterval(timer);
|
|
170
|
+
}
|
|
171
|
+
if (signal.aborted) {
|
|
172
|
+
await save();
|
|
173
|
+
throw signal.reason;
|
|
174
|
+
}
|
|
175
|
+
await saving;
|
|
176
|
+
await rename(part, path);
|
|
177
|
+
await rm(statePath, { force: true });
|
|
178
|
+
return { path, size: total };
|
|
179
|
+
}
|
|
180
|
+
/** Starts a request that is cancelled by `signal`, or when no data arrives for `timeout` ms. */
|
|
181
|
+
async function request(url, headers, signal, timeout) {
|
|
182
|
+
const ctl = new AbortController();
|
|
183
|
+
const abort = () => ctl.abort(signal.reason);
|
|
184
|
+
signal.addEventListener('abort', abort, { once: true });
|
|
185
|
+
if (signal.aborted)
|
|
186
|
+
abort();
|
|
187
|
+
const timer = setTimeout(() => ctl.abort(new Error(`${timeout / 1000} 秒内未收到数据`)), timeout);
|
|
188
|
+
const close = () => {
|
|
189
|
+
clearTimeout(timer);
|
|
190
|
+
signal.removeEventListener('abort', abort);
|
|
191
|
+
ctl.abort();
|
|
192
|
+
};
|
|
193
|
+
try {
|
|
194
|
+
const res = await fetch(url, { headers, signal: ctl.signal });
|
|
195
|
+
return { res, alive: () => timer.refresh(), close };
|
|
196
|
+
}
|
|
197
|
+
catch (err) {
|
|
198
|
+
close();
|
|
199
|
+
throw err;
|
|
200
|
+
}
|
|
201
|
+
}
|
|
202
|
+
function withRange(headers, start, end) {
|
|
203
|
+
const copy = new Headers(headers);
|
|
204
|
+
copy.set('range', `bytes=${start}-${end}`);
|
|
205
|
+
return copy;
|
|
206
|
+
}
|
|
207
|
+
function contentRange(res) {
|
|
208
|
+
const match = /^bytes (\d+)-\d+\/(\d+)$/.exec(res.headers.get('content-range') ?? '');
|
|
209
|
+
return match ? { start: Number(match[1]), total: Number(match[2]) } : undefined;
|
|
210
|
+
}
|
|
211
|
+
function httpStatus(res) {
|
|
212
|
+
return `HTTP ${res.status} ${res.statusText}`.trim();
|
|
213
|
+
}
|
|
214
|
+
function split(total, connections) {
|
|
215
|
+
const count = Math.max(1, Math.min(connections, Math.floor(total / MIN_SEGMENT)));
|
|
216
|
+
const size = Math.ceil(total / count);
|
|
217
|
+
return Array.from({ length: count }, (_, i) => ({
|
|
218
|
+
start: i * size,
|
|
219
|
+
end: Math.min(total, (i + 1) * size) - 1,
|
|
220
|
+
done: 0,
|
|
221
|
+
}));
|
|
222
|
+
}
|
|
223
|
+
/** Returns the saved state when it describes the same remote file and the partial file is intact. */
|
|
224
|
+
async function loadState(statePath, part, fresh) {
|
|
225
|
+
try {
|
|
226
|
+
const saved = JSON.parse(await readFile(statePath, 'utf8'));
|
|
227
|
+
const same = saved.url === fresh.url &&
|
|
228
|
+
saved.total === fresh.total &&
|
|
229
|
+
saved.etag === fresh.etag &&
|
|
230
|
+
saved.lastModified === fresh.lastModified;
|
|
231
|
+
if (same && Array.isArray(saved.segments) && (await stat(part)).size === fresh.total)
|
|
232
|
+
return saved;
|
|
233
|
+
}
|
|
234
|
+
catch {
|
|
235
|
+
// No state, or an unreadable one: start from scratch.
|
|
236
|
+
}
|
|
237
|
+
return null;
|
|
238
|
+
}
|
|
239
|
+
async function saveState(statePath, state) {
|
|
240
|
+
// Write-then-rename, so an interruption never leaves a truncated state file behind.
|
|
241
|
+
await writeFile(`${statePath}.tmp`, JSON.stringify(state));
|
|
242
|
+
await rename(`${statePath}.tmp`, statePath);
|
|
243
|
+
}
|
|
244
|
+
async function writeAll(file, data, position) {
|
|
245
|
+
for (let offset = 0; offset < data.length;) {
|
|
246
|
+
const { bytesWritten } = await file.write(data, offset, data.length - offset, position + offset);
|
|
247
|
+
offset += bytesWritten;
|
|
248
|
+
}
|
|
249
|
+
}
|
|
250
|
+
async function resolveOutput(output, res) {
|
|
251
|
+
if (output !== undefined && !(await isDirectory(output)))
|
|
252
|
+
return resolve(output);
|
|
253
|
+
return resolve(output ?? '.', remoteName(res));
|
|
254
|
+
}
|
|
255
|
+
function remoteName(res) {
|
|
256
|
+
const disposition = res.headers.get('content-disposition') ?? '';
|
|
257
|
+
let name;
|
|
258
|
+
try {
|
|
259
|
+
const encoded = /filename\*\s*=\s*[^']*'[^']*'([^;]+)/i.exec(disposition);
|
|
260
|
+
const plain = /filename\s*=\s*(?:"([^"]*)"|([^;]+))/i.exec(disposition);
|
|
261
|
+
name = encoded
|
|
262
|
+
? decodeURIComponent(encoded[1].trim())
|
|
263
|
+
: (plain?.[1] ?? plain?.[2]?.trim() ?? decodeURIComponent(new URL(res.url).pathname.split('/').pop()));
|
|
264
|
+
}
|
|
265
|
+
catch {
|
|
266
|
+
// Malformed percent-encoding or URL: fall through to the default name.
|
|
267
|
+
}
|
|
268
|
+
// The name comes from the server, so strip anything that could escape the target directory.
|
|
269
|
+
name = name?.replace(/[<>:"/\\|?*\x00-\x1f]/g, '_');
|
|
270
|
+
return !name || /^\.*$/.test(name) ? 'download' : name;
|
|
271
|
+
}
|
|
272
|
+
async function isDirectory(path) {
|
|
273
|
+
return stat(path).then((s) => s.isDirectory(), () => false);
|
|
274
|
+
}
|
|
275
|
+
async function exists(path) {
|
|
276
|
+
return stat(path).then(() => true, () => false);
|
|
277
|
+
}
|
|
@@ -0,0 +1,9 @@
|
|
|
1
|
+
export interface ProgressBar {
|
|
2
|
+
update(downloaded: number): void;
|
|
3
|
+
/** Draws the final state and moves to the next line. */
|
|
4
|
+
stop(): void;
|
|
5
|
+
}
|
|
6
|
+
/** A single-line progress bar. Draws nothing when `out` is not a terminal. */
|
|
7
|
+
export declare function createProgressBar(total: number | null, initial?: number, out?: NodeJS.WriteStream): ProgressBar;
|
|
8
|
+
export declare function formatBytes(bytes: number): string;
|
|
9
|
+
export declare function formatDuration(seconds: number): string;
|
package/dist/progress.js
ADDED
|
@@ -0,0 +1,67 @@
|
|
|
1
|
+
const REFRESH_MS = 100;
|
|
2
|
+
/** Speed is averaged over this window so the reading and the ETA do not jitter. */
|
|
3
|
+
const SPEED_WINDOW_MS = 3000;
|
|
4
|
+
const MAX_BAR_WIDTH = 40;
|
|
5
|
+
const MIN_BAR_WIDTH = 10;
|
|
6
|
+
/** A single-line progress bar. Draws nothing when `out` is not a terminal. */
|
|
7
|
+
export function createProgressBar(total, initial = 0, out = process.stderr) {
|
|
8
|
+
if (!out.isTTY)
|
|
9
|
+
return { update() { }, stop() { } };
|
|
10
|
+
let downloaded = initial;
|
|
11
|
+
const samples = [[Date.now(), initial]];
|
|
12
|
+
const render = () => {
|
|
13
|
+
const now = Date.now();
|
|
14
|
+
samples.push([now, downloaded]);
|
|
15
|
+
while (samples.length > 2 && now - samples[0][0] > SPEED_WINDOW_MS)
|
|
16
|
+
samples.shift();
|
|
17
|
+
const [since, base] = samples[0];
|
|
18
|
+
const speed = now > since ? ((downloaded - base) * 1000) / (now - since) : 0;
|
|
19
|
+
// Some pseudo-terminals report a width of 0.
|
|
20
|
+
const columns = out.columns || 80;
|
|
21
|
+
let line = `${formatBytes(downloaded)} ${formatBytes(speed)}/s`;
|
|
22
|
+
if (total) {
|
|
23
|
+
const ratio = Math.min(1, downloaded / total);
|
|
24
|
+
// Floored, so the display only reads 100% once every byte has arrived.
|
|
25
|
+
const percent = (Math.floor(ratio * 1000) / 10).toFixed(1).padStart(5);
|
|
26
|
+
const eta = speed > 0 ? formatDuration((total - downloaded) / speed) : '--:--';
|
|
27
|
+
line = `${percent}% ${formatBytes(downloaded)} / ${formatBytes(total)} ${formatBytes(speed)}/s ETA ${eta}`;
|
|
28
|
+
const width = Math.min(MAX_BAR_WIDTH, columns - line.length - 2);
|
|
29
|
+
if (width >= MIN_BAR_WIDTH) {
|
|
30
|
+
const filled = Math.floor(ratio * width);
|
|
31
|
+
line = `${'█'.repeat(filled)}${'░'.repeat(width - filled)} ${line}`;
|
|
32
|
+
}
|
|
33
|
+
}
|
|
34
|
+
// Stay one column short of the edge so the terminal never wraps the line.
|
|
35
|
+
out.write(`\r${line.slice(0, columns - 1)}\x1b[K`);
|
|
36
|
+
};
|
|
37
|
+
const timer = setInterval(render, REFRESH_MS);
|
|
38
|
+
render();
|
|
39
|
+
return {
|
|
40
|
+
update(value) {
|
|
41
|
+
downloaded = value;
|
|
42
|
+
},
|
|
43
|
+
stop() {
|
|
44
|
+
clearInterval(timer);
|
|
45
|
+
render();
|
|
46
|
+
out.write('\n');
|
|
47
|
+
},
|
|
48
|
+
};
|
|
49
|
+
}
|
|
50
|
+
export function formatBytes(bytes) {
|
|
51
|
+
const units = ['B', 'KB', 'MB', 'GB', 'TB'];
|
|
52
|
+
let value = bytes;
|
|
53
|
+
let unit = 0;
|
|
54
|
+
// Switching units at 1000 keeps the number to three digits before the decimal point.
|
|
55
|
+
while (value >= 1000 && unit < units.length - 1) {
|
|
56
|
+
value /= 1024;
|
|
57
|
+
unit++;
|
|
58
|
+
}
|
|
59
|
+
return `${unit === 0 ? Math.round(value) : value.toFixed(value < 10 ? 2 : 1)} ${units[unit]}`;
|
|
60
|
+
}
|
|
61
|
+
export function formatDuration(seconds) {
|
|
62
|
+
const s = Math.ceil(seconds);
|
|
63
|
+
const pad = (n) => String(n).padStart(2, '0');
|
|
64
|
+
const hours = Math.floor(s / 3600);
|
|
65
|
+
const minutes = Math.floor((s % 3600) / 60);
|
|
66
|
+
return hours ? `${hours}:${pad(minutes)}:${pad(s % 60)}` : `${minutes}:${pad(s % 60)}`;
|
|
67
|
+
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,39 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "hdm-cli",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"
|
|
5
|
-
"
|
|
6
|
-
|
|
3
|
+
"version": "0.1.0",
|
|
4
|
+
"description": "Hyper Download Manager: a lightweight multi-connection file downloader with resume and a progress bar. Zero runtime dependencies.",
|
|
5
|
+
"keywords": [
|
|
6
|
+
"download",
|
|
7
|
+
"downloader",
|
|
8
|
+
"download-manager",
|
|
9
|
+
"multi-connection",
|
|
10
|
+
"resume",
|
|
11
|
+
"cli"
|
|
12
|
+
],
|
|
13
|
+
"license": "MIT",
|
|
14
|
+
"type": "module",
|
|
15
|
+
"bin": {
|
|
16
|
+
"hdm": "dist/cli.js"
|
|
17
|
+
},
|
|
18
|
+
"exports": {
|
|
19
|
+
".": {
|
|
20
|
+
"types": "./dist/index.d.ts",
|
|
21
|
+
"default": "./dist/index.js"
|
|
22
|
+
}
|
|
23
|
+
},
|
|
24
|
+
"files": [
|
|
25
|
+
"dist"
|
|
26
|
+
],
|
|
27
|
+
"engines": {
|
|
28
|
+
"node": ">=20"
|
|
29
|
+
},
|
|
30
|
+
"scripts": {
|
|
31
|
+
"build": "tsc",
|
|
32
|
+
"test": "tsc && node --test test/*.test.js",
|
|
33
|
+
"prepublishOnly": "npm test"
|
|
34
|
+
},
|
|
35
|
+
"devDependencies": {
|
|
36
|
+
"@types/node": "^26.6.6",
|
|
37
|
+
"typescript": "^7.0.2"
|
|
38
|
+
}
|
|
39
|
+
}
|