sc-dlp 1.0.0 → 1.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (3) hide show
  1. package/README.md +116 -40
  2. package/lib/cli.cjs +1 -1
  3. package/package.json +3 -3
package/README.md CHANGED
@@ -1,44 +1,45 @@
1
1
  # sc-dlp
2
2
 
3
- A high-performance command-line tool for downloading Scratch and TurboWarp projects. Supports projects from Scratch 1.4 (sb), 2.0 (sb2), 3.0 (sb3), and TurboWarp Share (sha256).
3
+ [![npm version](https://img.shields.io/npm/v/sc-dlp.svg)](https://www.npmjs.com/package/sc-dlp)
4
+ [![license](https://img.shields.io/npm/l/sc-dlp.svg)](https://github.com/rbstxt/sc-dlp/blob/main/LICENSE)
5
+
6
+ A high-performance CLI and Node.js library for downloading Scratch and TurboWarp projects. Supports projects from Scratch 1.4 (`sb`), 2.0 (`sb2`), 3.0 (`sb3`), and TurboWarp Share (`sha256`).
4
7
 
5
8
  `sc-dlp` is a fork of [forkphorus/sb-downloader](https://github.com/forkphorus/sb-downloader), redesigned for robust project archival, analysis, and research.
6
9
 
7
10
  ## Features
8
11
 
9
- - **TurboWarp Support**: Comprehensive support for `share.turbowarp.org` projects, including SHA-256 asset mapping and internal metadata extraction.
10
- - **Authentication**: Login with Scratch account to access projects in "My Stuff" or recently viewed projects.
11
- - **Advanced Collection Support**: Download projects from Loves, Favorites, Studios, Search results, Featured projects, Explore categories, and "My Stuff".
12
+ - **TurboWarp Support**: Full support for `share.turbowarp.org` projects, including SHA-256 asset mapping and internal metadata extraction.
13
+ - **Authentication**: Log in with your Scratch account to access projects in "My Stuff" or recently viewed projects.
14
+ - **Advanced Collection Support**: Download projects from Loves, Favorites, Studios, Search results, Featured projects, Explore categories, Remixes, and "My Stuff".
12
15
  - **Automatic Pagination**: Automatically fetches all projects from multi-page collections.
13
16
  - **Shorthand Aliases**: Quick access to any collection using simple patterns.
14
- - **High Performance**: Parallel project downloads, asset fetching, and smart hash verification.
15
- - **Rich CLI**: Professional logging with distinct prefixes (`[assets]`, `[mapping]`, `[compress]`), interactive prompts, and real-time progress bars.
17
+ - **High Performance**: Parallel project downloads (`-c`), asset caching/retries, and automatic MD5 hash verification.
18
+ - **Rich CLI & Library**: Real-time worker progress bars for terminals, clean logs for pipes/CI, and an ESM/CJS/TypeScript Node.js API.
16
19
 
17
20
  ## Installation
18
21
 
19
- ### From npm
22
+ Requires **Node.js 18.17.0 or later**.
23
+
24
+ ### Global CLI
20
25
 
21
26
  ```bash
22
27
  npm install -g sc-dlp
23
28
  ```
24
29
 
25
- Or run directly with `npx`:
30
+ ### Run without installing
26
31
 
27
32
  ```bash
28
33
  npx sc-dlp 60917032
29
34
  ```
30
35
 
31
- ### From source
36
+ ### As a Node.js library
32
37
 
33
38
  ```bash
34
- git clone https://github.com/rbstxt/sc-dlp.git
35
- cd sc-dlp
36
- npm install
37
- npm run build
38
- npm link
39
+ npm install sc-dlp
39
40
  ```
40
41
 
41
- ## Usage
42
+ ## CLI Usage
42
43
 
43
44
  Use project IDs, URLs, or shorthand aliases to download projects.
44
45
 
@@ -53,14 +54,9 @@ sc-dlp turbowarp/4cd76e33-0e7a-468f-85c9-17799748a864
53
54
  sc-dlp https://share.turbowarp.org/projects/4cd76e33-0e7a-468f-85c9-17799748a864
54
55
 
55
56
  # User collections
57
+ sc-dlp @scratchcat
56
58
  sc-dlp @scratchcat/loves
57
59
  sc-dlp @scratchcat/favorites
58
- sc-dlp @scratchcat
59
-
60
- # Authentication
61
- sc-dlp login
62
- sc-dlp status
63
- sc-dlp logout
64
60
 
65
61
  # Specialized collections
66
62
  sc-dlp mystuff/shared
@@ -70,6 +66,13 @@ sc-dlp recently
70
66
  sc-dlp 10123/remixes
71
67
  sc-dlp studio/1234
72
68
  sc-dlp search/platformer
69
+
70
+ # Authentication & configuration
71
+ sc-dlp login
72
+ sc-dlp status
73
+ sc-dlp logout
74
+ sc-dlp config --list
75
+ sc-dlp config --set conflictAction=skip
73
76
  ```
74
77
 
75
78
  ### Authentication
@@ -80,6 +83,18 @@ sc-dlp search/platformer
80
83
  - **Session ID**: Full session access using a `scratchsessionsid` cookie value.
81
84
  - **X-Token**: Limited permissions access. Suitable for `recently` and public data, but cannot access `mystuff`.
82
85
 
86
+ ### Configuration
87
+
88
+ Manage persistent defaults (language, conflict action, proxy, custom API URLs, TurboWarp mirrors) with `sc-dlp config`:
89
+
90
+ ```bash
91
+ sc-dlp config --list
92
+ sc-dlp config --get proxy
93
+ sc-dlp config --set proxy=http://127.0.0.1:8080
94
+ sc-dlp config --set language=ja
95
+ sc-dlp config --reset
96
+ ```
97
+
83
98
  ### Collection Aliases
84
99
 
85
100
  | Collection | Alias Pattern | Note |
@@ -97,42 +112,103 @@ sc-dlp search/platformer
97
112
 
98
113
  ### Options
99
114
 
100
- - `--help`: Shows the help screen.
101
- - `-c`, `--concurrency NUMBER`: Number of projects to download in parallel (default: 1).
102
- - `-m`, `--mtime`: Set file modification time to project last shared/modified date.
103
- - `-t`, `--write-thumbnail`: Download project thumbnail.
104
- - `--thumbnail-size WIDTHxHEIGHT`: Thumbnail size (default: 9999x9999).
105
- - `--playlist-start NUMBER`: Playlist item to start at.
115
+ - `login` / `status` / `logout`: Manage Scratch authentication.
116
+ - `config [options...]`: Manage saved configuration (`sc-dlp config --help` for details).
117
+ - `deleteaccount`: Download projects listed on the Scratch account deletion confirmation page.
118
+ - `-h`, `--help`: Show the help screen.
119
+ - `-c`, `--concurrency NUMBER`: Number of projects to download in parallel (default: `1`).
120
+ - `-l`, `--limit NUMBER`: Maximum number of projects to fetch from a collection.
121
+ - `--page NUMBER`: Fetch a specific page from a collection (e.g., users, favorites, loves, studios, search, mystuff).
122
+ - `--playlist-start NUMBER`: Playlist item to start at (default: `1`).
106
123
  - `--playlist-end NUMBER`: Playlist item to end at.
107
- - `-I`, `--playlist-items ITEMS`: Specific indices (e.g., 1,2,5-10).
108
- - `--page NUMBER`: Page number to fetch for collections (e.g., users, favorites, loves, studios, search, mystuff).
109
- - `--format EXT`: Output extension (sb3, sb2, sb, json).
110
- - `-j`, `--json-only`: Download only the `project.json` file.
111
- - `-a`, `--no-assets`: Download code only (no costumes/sounds).
124
+ - `-I`, `--playlist-items ITEMS`: Specific playlist indices (e.g., `1,2,5-10`).
125
+ - `--language CODE`: Override language for search results (default: `en`).
126
+ - `--project-token TOKEN`: Token to access private/unshared projects.
127
+ - `--format EXT`: Output file extension (`sb3`, `sb2`, `sb`, `json`; default: auto-detected).
128
+ - `-j`, `--json-only`: Download only the `project.json` file (implies `--format json`).
129
+ - `-a`, `--no-assets`: Download project code only (skip costumes and sounds).
112
130
  - `--conflict ACTION`: Conflict action: `overwrite`, `skip`, or `keep-both`.
113
- - `--retries NUMBER`: Number of times to retry failed asset downloads (default: 3).
114
- - `-v`, `--verbose`: Show detailed information and stack traces on error.
131
+ - `--retries NUMBER`: Number of times to retry failed asset downloads (default: `5`).
132
+ - `-m`, `--mtime`: Set file modification time to the project's last modified/shared date.
133
+ - `-t`, `--write-thumbnail`: Download the project thumbnail alongside the project file.
134
+ - `--thumbnail-size WIDTHxHEIGHT`: Thumbnail size (default: `9999x9999`).
135
+ - `-v`, `--verbose`: Show detailed logs and stack traces on error.
115
136
 
116
137
  ### File Conflict Resolution
117
138
 
118
- If a project with the same name and ID already exists in the target directory, `sc-dlp` will prompt you for action. By default, sc-dlp includes the project ID in the filename (e.g., `Title (12345).sb3`) to avoid conflicts between different projects with the same title.
139
+ If a project with the same name and ID already exists in the target directory, `sc-dlp` prompts you for an action (unless `--conflict` or `config --set conflictAction=...` is configured). By default, filenames include the project ID (e.g., `Title (12345).sb3`) to avoid collisions between projects with the same title.
119
140
 
120
141
  - **Overwrite**: Replace the existing file with the new download.
121
- - **Skip**: Abort the download for this specific project.
142
+ - **Skip**: Skip downloading this project.
122
143
  - **Keep both**: Automatically rename the new file by appending a number (e.g., `Project 2.sb3`).
123
144
 
124
- Once you make a choice, it will be automatically applied to all remaining conflicts within the same download session (e.g., when downloading a studio or user's projects) to ensure a smooth, unattended experience.
145
+ Once selected in a session, the choice is applied automatically to remaining conflicts in the same run.
146
+
147
+ ## Programmatic Usage (Node.js API)
148
+
149
+ `sc-dlp` ships ESM, CommonJS, and TypeScript declarations.
150
+
151
+ ```ts
152
+ import fs from "node:fs/promises";
153
+ import { downloadProjectFromID, getProjectsFromPlaylistURL } from "sc-dlp";
154
+
155
+ const project = await downloadProjectFromID("60917032", {
156
+ onProgress: (type, loaded, total) => {
157
+ console.log(`${type}: ${loaded}/${total}`);
158
+ },
159
+ });
160
+
161
+ await fs.writeFile(
162
+ `${project.title}.${project.type}`,
163
+ Buffer.from(project.arrayBuffer)
164
+ );
165
+
166
+ const studioProjects = await getProjectsFromPlaylistURL("studio/1234", {
167
+ limit: 20,
168
+ });
169
+ console.log(`Found ${studioProjects?.length ?? 0} projects`);
170
+ ```
171
+
172
+ CommonJS is also supported:
173
+
174
+ ```js
175
+ const { downloadProjectFromID } = require("sc-dlp");
176
+ ```
177
+
178
+ Main exported functions:
179
+
180
+ - `downloadProjectFromID(id, options?)`
181
+ - `downloadProjectFromURL(url, options?)`
182
+ - `downloadTurboWarpProject(id, options?)`
183
+ - `downloadProjectFromJSON(json, options?)`
184
+ - `downloadProjectFromBuffer(buffer, options?)`
185
+ - `getProjectMetadata(id, options?)`
186
+ - `getTurboWarpMetadata(id, options?)`
187
+ - `getProjectsFromPlaylistURL(input, options?)`
188
+ - `clearAssetCache()`
125
189
 
126
190
  ## Advanced
127
191
 
128
192
  ### Asset Integrity
129
193
 
130
- By default, `sc-dlp` calculates the MD5 hash of every downloaded asset. If the calculated hash differs from the hash provided in the project metadata, the tool automatically renames the file and updates the `project.json` to ensure the project remains loadable and consistent.
194
+ By default, `sc-dlp` calculates the MD5 hash of every downloaded asset. If the calculated hash differs from the hash referenced in `project.json`, the asset is automatically renamed and `project.json` is updated so the resulting project remains valid and loadable.
131
195
 
132
196
  ### TurboWarp Assets
133
197
 
134
- TurboWarp Share projects use SHA-256 for asset storage. `sc-dlp` automatically extracts the `md5extsToSha256` mapping from the project's page HTML to correctly fetch and rename these assets for Scratch compatibility.
198
+ TurboWarp Share projects use SHA-256 for asset storage. `sc-dlp` automatically extracts the `md5extsToSha256` mapping from the project page HTML to fetch and remap assets for Scratch compatibility.
135
199
 
136
200
  ## Privacy
137
201
 
138
- By default, `sc-dlp` communicates directly with Scratch APIs (`api.scratch.mit.edu`, `projects.scratch.mit.edu`, `assets.scratch.mit.edu`) and TurboWarp APIs (`share.turbowarp.org`). Session data is stored locally in `~/.sc-dlp/config.json`.
202
+ By default, `sc-dlp` communicates directly with Scratch APIs (`api.scratch.mit.edu`, `projects.scratch.mit.edu`, `assets.scratch.mit.edu`) and TurboWarp APIs (`share.turbowarp.org`). Configuration and session data are stored locally in `~/.sc-dlp/config.json` with `0600` permissions.
203
+
204
+ ## Development
205
+
206
+ To build and test from source:
207
+
208
+ ```bash
209
+ git clone https://github.com/rbstxt/sc-dlp.git
210
+ cd sc-dlp
211
+ npm install
212
+ npm run build
213
+ npm test
214
+ ```
package/lib/cli.cjs CHANGED
@@ -1,6 +1,6 @@
1
1
  #!/usr/bin/env node
2
2
  /*!
3
- * sc-dlp v1.0.0 <https://github.com/rbstxt/sc-dlp>
3
+ * sc-dlp v1.0.1 <https://github.com/rbstxt/sc-dlp>
4
4
  *
5
5
  * MIT License
6
6
  *
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "sc-dlp",
3
- "version": "1.0.0",
4
- "description": "A downloader for Scratch projects.",
3
+ "version": "1.0.1",
4
+ "description": "A high-performance CLI and Node.js library for downloading Scratch and TurboWarp projects.",
5
5
  "type": "module",
6
6
  "keywords": [
7
7
  "scratch",
@@ -80,4 +80,4 @@
80
80
  "typescript": "^5.9.2",
81
81
  "vitest": "^3.2.4"
82
82
  }
83
- }
83
+ }