justget 0.1.2 → 0.1.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.en.md +267 -0
- package/README.md +22 -1
- package/package.json +23 -5
package/README.en.md
ADDED
|
@@ -0,0 +1,267 @@
|
|
|
1
|
+
# JustGet
|
|
2
|
+
|
|
3
|
+
<div align="center">
|
|
4
|
+
|
|
5
|
+
**Multi-source racing chunked download accelerator**
|
|
6
|
+
|
|
7
|
+
> A multi-source, mirror-based download accelerator for GitHub releases & CDN files — parallel, chunked, resumable, China-network friendly.
|
|
8
|
+
|
|
9
|
+
[](https://opensource.org/licenses/MIT)
|
|
10
|
+
[](https://nodejs.org)
|
|
11
|
+
|
|
12
|
+
</div>
|
|
13
|
+
|
|
14
|
+
**English** · [简体中文](./README.md)
|
|
15
|
+
|
|
16
|
+
---
|
|
17
|
+
|
|
18
|
+
## ✨ Features
|
|
19
|
+
|
|
20
|
+
- 🔀 **Multi-source racing** — primary + multiple mirrors download in parallel, automatically pick the fastest source
|
|
21
|
+
- 🧩 **Chunked download** — large files split into chunks downloaded in parallel to maximize bandwidth
|
|
22
|
+
- 📊 **Intelligent scheduling** — dynamically adjust strategy based on speed and file size
|
|
23
|
+
- 🛡️ **Fault tolerance** — a single failed source does not break the whole download
|
|
24
|
+
- 🧮 **EWMA speed** — exponentially weighted moving average for stable speed estimation
|
|
25
|
+
- ✅ **Checksum validation** — md5 / sha1 / sha256 / sha512
|
|
26
|
+
|
|
27
|
+
## 📌 Status
|
|
28
|
+
|
|
29
|
+
| Module | Status |
|
|
30
|
+
|---|---|
|
|
31
|
+
| Types & config management (defaults / validation / merge) | ✅ Done |
|
|
32
|
+
| Chunk strategy (chunk size / ranges) | ✅ Done |
|
|
33
|
+
| Speed monitoring (EWMA / instant / average / peak) | ✅ Done |
|
|
34
|
+
| Temp-file naming & file validation (checksum) | ✅ Done |
|
|
35
|
+
| **Core downloader `download()`** (probe → chunks → warmup → decide → race → replace → merge → verify) | ✅ Done (verified with real downloads) |
|
|
36
|
+
| CLI wired to `download()` | ✅ Done |
|
|
37
|
+
|
|
38
|
+
Verified scenarios:
|
|
39
|
+
- Chunked download + merge + temp-file cleanup ✅
|
|
40
|
+
- Multi-source parallel racing (mirror startup) ✅
|
|
41
|
+
- Primary immediate failure (4xx / connection error) → mirror takes over automatically ✅
|
|
42
|
+
- Checksum validation (sha256) ✅
|
|
43
|
+
- Resume (existing output / merged temp-file reuse) ✅
|
|
44
|
+
- CLI end-to-end ✅
|
|
45
|
+
|
|
46
|
+
---
|
|
47
|
+
|
|
48
|
+
## 📦 Install
|
|
49
|
+
|
|
50
|
+
```bash
|
|
51
|
+
npm install justget
|
|
52
|
+
# or
|
|
53
|
+
yarn add justget
|
|
54
|
+
# or
|
|
55
|
+
pnpm add justget
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
Requires Node.js ≥ 18.
|
|
59
|
+
|
|
60
|
+
---
|
|
61
|
+
|
|
62
|
+
## 🚀 Quick Start
|
|
63
|
+
|
|
64
|
+
```typescript
|
|
65
|
+
import { download } from 'justget';
|
|
66
|
+
|
|
67
|
+
await download({
|
|
68
|
+
url: 'https://nodejs.org/dist/v20.10.0/node-v20.10.0-linux-x64.tar.xz',
|
|
69
|
+
output: './node.tar.xz',
|
|
70
|
+
});
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
With mirrors and progress:
|
|
74
|
+
|
|
75
|
+
```typescript
|
|
76
|
+
await download({
|
|
77
|
+
url: 'https://nodejs.org/dist/v20.10.0/node-v20.10.0-linux-x64.tar.xz',
|
|
78
|
+
output: './node.tar.xz',
|
|
79
|
+
mirrors: [
|
|
80
|
+
'https://npmmirror.com/mirrors/node/v20.10.0/node-v20.10.0-linux-x64.tar.xz',
|
|
81
|
+
'https://mirrors.tuna.tsinghua.edu.cn/nodejs-release/v20.10.0/node-v20.10.0-linux-x64.tar.xz',
|
|
82
|
+
],
|
|
83
|
+
options: {
|
|
84
|
+
onProgress: (p) => {
|
|
85
|
+
console.log(`${p.percentage.toFixed(1)}% | ${(p.speed / 1024).toFixed(0)} KB/s`);
|
|
86
|
+
},
|
|
87
|
+
},
|
|
88
|
+
});
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
More examples in [`examples/`](./examples/).
|
|
92
|
+
|
|
93
|
+
---
|
|
94
|
+
|
|
95
|
+
## 🖥️ Public API
|
|
96
|
+
|
|
97
|
+
### 1. `download(options: DownloadOptions): Promise<DownloadResult>`
|
|
98
|
+
|
|
99
|
+
Main entry point. Resolves with the download result; rejects on failure (including checksum mismatch).
|
|
100
|
+
|
|
101
|
+
**`DownloadOptions` parameters** (the `options` field is `DownloadConfig`; all optional, all with defaults):
|
|
102
|
+
|
|
103
|
+
| Field | Type | Default | Description |
|
|
104
|
+
|---|---|---|---|
|
|
105
|
+
| `url` | `string` | — | Primary URL (required) |
|
|
106
|
+
| `output` | `string` | — | Output file path (required) |
|
|
107
|
+
| `mirrors` | `string[]` | `[]` | Mirror URLs |
|
|
108
|
+
| `options.primaryWarmupTime` | `number` | `5000` | Primary warmup in ms (observe only, no mirrors during this period) |
|
|
109
|
+
| `options.speedThreshold` | `number` | `512000` | Speed threshold B/s; below this, consider starting mirrors |
|
|
110
|
+
| `options.fastPrimaryThreshold` | `number` | `2048000` | Fast-primary threshold B/s; above this, skip mirrors |
|
|
111
|
+
| `options.sizeThreshold` | `number` | `52428800` | File size threshold B; below this, skip mirrors |
|
|
112
|
+
| `options.minReplaceTime` | `number` | `30000` | Minimum time before replacing a slow source (ms) |
|
|
113
|
+
| `options.replaceCheckInterval` | `number` | `10000` | Source replacement check interval (ms) |
|
|
114
|
+
| `options.chunks` | `number \| 'auto'` | `'auto'` | Chunk count |
|
|
115
|
+
| `options.chunkSize` | `number \| 'auto'` | `'auto'` | Chunk size in bytes |
|
|
116
|
+
| `options.minChunkSize` | `number` | `1048576` | Minimum chunk size in bytes |
|
|
117
|
+
| `options.retries` | `number` | `3` | Retry count per source |
|
|
118
|
+
| `options.retryDelay` | `number` | `2000` | Retry delay (ms) |
|
|
119
|
+
| `options.timeout` | `number` | `300000` | Download timeout (ms) |
|
|
120
|
+
| `options.connectTimeout` | `number` | `10000` | Connection timeout (ms) |
|
|
121
|
+
| `options.checksum` | `string` | — | Expected checksum (validated after download) |
|
|
122
|
+
| `options.checksumAlgorithm` | `'md5'\|'sha1'\|'sha256'\|'sha512'` | `'sha256'` | Checksum algorithm |
|
|
123
|
+
| `options.onProgress` | `(p: ProgressInfo) => void` | — | Progress callback |
|
|
124
|
+
| `options.onComplete` | `(r: DownloadResult) => void` | — | Completion callback |
|
|
125
|
+
| `options.onError` | `(e: Error) => void` | — | Error callback |
|
|
126
|
+
|
|
127
|
+
### 2. Callbacks & Types
|
|
128
|
+
|
|
129
|
+
**`ProgressInfo`** (argument of `onProgress`):
|
|
130
|
+
|
|
131
|
+
```typescript
|
|
132
|
+
interface ProgressInfo {
|
|
133
|
+
downloadedBytes: number; // bytes downloaded
|
|
134
|
+
totalBytes: number; // total bytes
|
|
135
|
+
percentage: number; // 0-100
|
|
136
|
+
speed: number; // current speed B/s
|
|
137
|
+
eta: number; // estimated time remaining (ms)
|
|
138
|
+
sources: SourceProgress[]; // per-source progress
|
|
139
|
+
}
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
**`DownloadResult`** (return value of `download()`):
|
|
143
|
+
|
|
144
|
+
```typescript
|
|
145
|
+
interface DownloadResult {
|
|
146
|
+
path: string; // output path
|
|
147
|
+
bytes: number; // total bytes downloaded
|
|
148
|
+
duration: number; // duration (ms)
|
|
149
|
+
averageSpeed: number; // average speed B/s
|
|
150
|
+
sources: SourceResult[]; // per-source results
|
|
151
|
+
}
|
|
152
|
+
```
|
|
153
|
+
|
|
154
|
+
### 3. Other exports
|
|
155
|
+
|
|
156
|
+
Besides `download()`, the library also exports common utilities (for advanced use):
|
|
157
|
+
|
|
158
|
+
- Config: `DEFAULT_CONFIG`, `mergeConfig(user)`
|
|
159
|
+
- Chunk strategy: `calculateChunkStrategy`, `calculateChunkSize`, `calculateChunkCount`, `createChunkRanges`
|
|
160
|
+
- Speed: `SpeedMonitor`, `EWMA`, `getSpeedStats`
|
|
161
|
+
- Utils: `calculateChecksum`, `validateFile`, `getTempFileName`, `generateShortHash`, etc.
|
|
162
|
+
- Types: `DownloadOptions`, `DownloadConfig`, `ProgressInfo`, `DownloadResult`, etc.
|
|
163
|
+
|
|
164
|
+
---
|
|
165
|
+
|
|
166
|
+
## 🖥️ CLI
|
|
167
|
+
|
|
168
|
+
```bash
|
|
169
|
+
# after global install:
|
|
170
|
+
npm install -g justget
|
|
171
|
+
|
|
172
|
+
justget <url> [options]
|
|
173
|
+
```
|
|
174
|
+
|
|
175
|
+
### Options
|
|
176
|
+
|
|
177
|
+
| Option | Type | Default | Description |
|
|
178
|
+
|---|---|---|---|
|
|
179
|
+
| `<url>` | string | — | URL to download (required) |
|
|
180
|
+
| `-o, --output <path>` | string | filename from URL | Output path |
|
|
181
|
+
| `-m, --mirror <url>` | string[] | — | Mirror URL (repeatable) |
|
|
182
|
+
| `--warmup <ms>` | number | `5000` | Primary warmup |
|
|
183
|
+
| `--min-speed <bytes/s>` | number | `512000` | Speed threshold; below this, start mirrors |
|
|
184
|
+
| `--fast-primary-threshold <bytes/s>` | number | `2048000` | Above this primary speed, skip mirrors |
|
|
185
|
+
| `--min-size <bytes>` | number | `52428800` | Below this file size, skip mirrors |
|
|
186
|
+
| `--min-replace-time <ms>` | number | `30000` | Min time before replacing a slow source |
|
|
187
|
+
| `--replace-check-interval <ms>` | number | `10000` | Replacement check interval |
|
|
188
|
+
| `--chunks <n>` | number\|auto | `auto` | Chunk count |
|
|
189
|
+
| `--chunk-size <bytes>` | number\|auto | `auto` | Chunk size |
|
|
190
|
+
| `--min-chunk-size <bytes>` | number | `1048576` | Min chunk size |
|
|
191
|
+
| `--retries <n>` | number | `3` | Retries per source |
|
|
192
|
+
| `--retry-delay <ms>` | number | `2000` | Retry delay |
|
|
193
|
+
| `--timeout <ms>` | number | `300000` | Download timeout |
|
|
194
|
+
| `--connect-timeout <ms>` | number | `10000` | Connection timeout |
|
|
195
|
+
| `--checksum <hash>` | string | — | Expected checksum |
|
|
196
|
+
| `--checksum-algo <algo>` | string | `sha256` | Checksum algorithm |
|
|
197
|
+
| `-V, --version` | — | — | Version |
|
|
198
|
+
| `-h, --help` | — | — | Help |
|
|
199
|
+
|
|
200
|
+
### Examples
|
|
201
|
+
|
|
202
|
+
```bash
|
|
203
|
+
# basic
|
|
204
|
+
justget https://example.com/file.zip -o file.zip
|
|
205
|
+
|
|
206
|
+
# racing with mirrors
|
|
207
|
+
justget https://example.com/file.zip -o file.zip \
|
|
208
|
+
--mirror https://mirror1.com/file.zip \
|
|
209
|
+
--mirror https://mirror2.com/file.zip
|
|
210
|
+
|
|
211
|
+
# small-file fast mode (skip mirrors)
|
|
212
|
+
justget https://example.com/file.zip -o file.zip --min-size 0 --fast-primary-threshold 0
|
|
213
|
+
|
|
214
|
+
# checksum
|
|
215
|
+
justget https://example.com/file.zip -o file.zip \
|
|
216
|
+
--checksum a1b2c3... --checksum-algo sha256
|
|
217
|
+
|
|
218
|
+
# help
|
|
219
|
+
justget --help
|
|
220
|
+
```
|
|
221
|
+
|
|
222
|
+
---
|
|
223
|
+
|
|
224
|
+
## 🧠 How It Works
|
|
225
|
+
|
|
226
|
+
```
|
|
227
|
+
Phase 1 warmup → Phase 2 decide → Phase 3 race → Phase 4 optimize → Phase 5 finish
|
|
228
|
+
Primary Decide whether Race primary Replace slow Merge chunks,
|
|
229
|
+
warmup to start mirrors + mirrors sources verify, clean
|
|
230
|
+
(default 5s) (every 10s)
|
|
231
|
+
```
|
|
232
|
+
|
|
233
|
+
- **When mirrors start**: primary speed below `speedThreshold`, file larger than `sizeThreshold`, and primary speed below `fastPrimaryThreshold`; on immediate primary failure (4xx/5xx or network error), mirrors start unconditionally and early.
|
|
234
|
+
- **Chunk allocation**: the primary source takes chunks from the head of the pool, mirrors from the tail (primary keeps the leading part); merge concatenates by index order.
|
|
235
|
+
- **Slow-source replacement**: after `minReplaceTime`, every `replaceCheckInterval` the sources are evaluated; a source consistently below 1/3 of the fastest speed for `slowChecks` consecutive checks is aborted and its unfinished chunks are returned to the pool.
|
|
236
|
+
- **No Range support**: primary and mirrors download the whole file concurrently; first to finish wins.
|
|
237
|
+
- **Validation**: when `checksum` is set, the merged file is verified.
|
|
238
|
+
|
|
239
|
+
---
|
|
240
|
+
|
|
241
|
+
## 🗂️ Temporary Files
|
|
242
|
+
|
|
243
|
+
Naming: `.justget-{basename}-{hash}-{source}-{chunk}.{ext}`
|
|
244
|
+
|
|
245
|
+
- Cleaned automatically after completion
|
|
246
|
+
- Resume supported after an interrupted download (≥90% complete)
|
|
247
|
+
|
|
248
|
+
---
|
|
249
|
+
|
|
250
|
+
## 🛠️ Development
|
|
251
|
+
|
|
252
|
+
```bash
|
|
253
|
+
npm install # install dependencies (incl. devDependencies)
|
|
254
|
+
npm run build # tsc compile to dist/
|
|
255
|
+
npm run dev # watch mode
|
|
256
|
+
npm test # vitest unit tests
|
|
257
|
+
npm run lint # eslint
|
|
258
|
+
npm run format # prettier
|
|
259
|
+
```
|
|
260
|
+
|
|
261
|
+
## 📄 License
|
|
262
|
+
|
|
263
|
+
MIT License — © Wind Li
|
|
264
|
+
|
|
265
|
+
---
|
|
266
|
+
|
|
267
|
+
*Made with ❤️ by Wind Li*
|
package/README.md
CHANGED
|
@@ -2,13 +2,34 @@
|
|
|
2
2
|
|
|
3
3
|
<div align="center">
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
**高性能多源竞速分块下载加速器** / **Multi-source racing chunked download accelerator**
|
|
6
6
|
|
|
7
7
|
[](https://opensource.org/licenses/MIT)
|
|
8
8
|
[](https://nodejs.org)
|
|
9
9
|
|
|
10
10
|
</div>
|
|
11
11
|
|
|
12
|
+
**简体中文** · [English](./README.en.md)
|
|
13
|
+
|
|
14
|
+
---
|
|
15
|
+
|
|
16
|
+
## 🎯 为什么需要它 / Why this tool
|
|
17
|
+
|
|
18
|
+
> 在国内网络环境下,直接访问 GitHub 等国外仓库经常很慢,甚至超时失败。
|
|
19
|
+
> 本工具的思路很简单:**主站 + 多个镜像同时下载,谁快用谁**。
|
|
20
|
+
> 官方源慢或挂了,会自动切到可用的镜像,不用傻等超时。
|
|
21
|
+
|
|
22
|
+
```bash
|
|
23
|
+
# 示例:GitHub 官方源 + 国内镜像一起抢
|
|
24
|
+
justget https://github.com/xxx/releases/download/v1.0/file.zip \
|
|
25
|
+
-o file.zip \
|
|
26
|
+
-m https://gh-proxy.com/https://github.com/xxx/releases/download/v1.0/file.zip \
|
|
27
|
+
-m https://npmmirror.com/mirrors/xxx/file.zip
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
- 官方源快 → 直接用它,不折腾
|
|
31
|
+
- 官方源慢/失败 → 自动切镜像,一般几秒内就有结果
|
|
32
|
+
|
|
12
33
|
---
|
|
13
34
|
|
|
14
35
|
## ✨ 特性 / Features
|
package/package.json
CHANGED
|
@@ -1,14 +1,15 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "justget",
|
|
3
|
-
"version": "0.1.
|
|
4
|
-
"description": "Multi-source racing chunked downloader
|
|
3
|
+
"version": "0.1.3",
|
|
4
|
+
"description": "Multi-source racing chunked downloader / download accelerator for GitHub & mirror sources, China-network friendly. 多源竞速分块下载加速器(GitHub 镜像加速)",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "dist/index.js",
|
|
7
7
|
"types": "dist/index.d.ts",
|
|
8
8
|
"files": [
|
|
9
9
|
"dist/",
|
|
10
10
|
"LICENSE",
|
|
11
|
-
"README.md"
|
|
11
|
+
"README.md",
|
|
12
|
+
"README.en.md"
|
|
12
13
|
],
|
|
13
14
|
"bin": {
|
|
14
15
|
"justget": "dist/cli.js"
|
|
@@ -32,11 +33,28 @@
|
|
|
32
33
|
"download",
|
|
33
34
|
"downloader",
|
|
34
35
|
"multi-source",
|
|
36
|
+
"mirror",
|
|
35
37
|
"chunked",
|
|
36
38
|
"parallel",
|
|
37
|
-
"
|
|
39
|
+
"concurrent",
|
|
40
|
+
"resume",
|
|
41
|
+
"accelerator",
|
|
42
|
+
"segment",
|
|
43
|
+
"github",
|
|
44
|
+
"github-proxy",
|
|
45
|
+
"gh-proxy",
|
|
46
|
+
"cdn",
|
|
38
47
|
"http",
|
|
39
|
-
"
|
|
48
|
+
"cli",
|
|
49
|
+
"china-network",
|
|
50
|
+
"github-download",
|
|
51
|
+
"download-accelerator",
|
|
52
|
+
"下载加速",
|
|
53
|
+
"多源下载",
|
|
54
|
+
"镜像下载",
|
|
55
|
+
"分块下载",
|
|
56
|
+
"github加速",
|
|
57
|
+
"断点续传"
|
|
40
58
|
],
|
|
41
59
|
"author": "Wind Li",
|
|
42
60
|
"license": "MIT",
|