replicate-predictions-downloader 2.0.1 → 2.0.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +24 -76
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -17,19 +17,31 @@ A Node.js script to download and organize all your Replicate predictions, includ
|
|
|
17
17
|
|
|
18
18
|
## Features
|
|
19
19
|
|
|
20
|
-
-
|
|
21
|
-
-
|
|
22
|
-
-
|
|
23
|
-
-
|
|
24
|
-
-
|
|
25
|
-
-
|
|
26
|
-
-
|
|
27
|
-
-
|
|
20
|
+
- Downloads all predictions from your Replicate account
|
|
21
|
+
- Organizes files by model and date
|
|
22
|
+
- Date filtering to avoid duplicate downloads
|
|
23
|
+
- Incremental downloads since last successful run
|
|
24
|
+
- Creates ZIP archives for each model (optional)
|
|
25
|
+
- Saves enhanced metadata for all predictions
|
|
26
|
+
- Shows detailed download statistics
|
|
27
|
+
- Preserves predictions before they expire
|
|
28
|
+
|
|
29
|
+
## Quick Start
|
|
30
|
+
|
|
31
|
+
```bash
|
|
32
|
+
npx replicate-predictions-downloader --last-run
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
You'll need your API token from [replicate.com/account/api-tokens](https://replicate.com/account/api-tokens) in the environment as `REPLICATE_API_TOKEN`.
|
|
28
36
|
|
|
29
37
|
## Why Use This Tool?
|
|
30
38
|
|
|
31
39
|
Replicate only stores your predictions temporarily. This tool helps you preserve your valuable work before it disappears, with intelligent organization to make your outputs easy to find later.
|
|
32
40
|
|
|
41
|
+
## Background
|
|
42
|
+
|
|
43
|
+
Replicate deletes predictions after 30 days. If you've been experimenting with models and want to keep your outputs - for portfolios, research, or just because you like them - you need to download them yourself. This tool does that, with some organization thrown in so you can actually find things later.
|
|
44
|
+
|
|
33
45
|
## Setup
|
|
34
46
|
|
|
35
47
|
### Prerequisites
|
|
@@ -88,6 +100,10 @@ node index.js
|
|
|
88
100
|
replicate-downloader
|
|
89
101
|
```
|
|
90
102
|
|
|
103
|
+
## What You'll See
|
|
104
|
+
|
|
105
|
+
When you run the tool, you'll see real-time progress as it fetches and downloads your predictions. The output includes details about what's being processed, download progress, and final statistics. Your files end up organized by model and date. There's also a metadata JSON if you're into that sort of thing.
|
|
106
|
+
|
|
91
107
|
### Date Filtering Options
|
|
92
108
|
|
|
93
109
|
Avoid duplicate downloads with intelligent date filtering:
|
|
@@ -202,74 +218,6 @@ The included .gitignore will help prevent this.
|
|
|
202
218
|
|
|
203
219
|
This project is licensed under the MIT License - see the [LICENSE.md](LICENSE.md) file for details.
|
|
204
220
|
|
|
205
|
-
## Background
|
|
206
|
-
|
|
207
|
-
Replicate is a powerful platform for running AI models, but predictions don't last forever - they expire after a period of time. This tool was originally created while working on LLM behavior/personality research when I realized I was losing valuable generated content.
|
|
208
|
-
|
|
209
|
-
After generating hundreds of images with Stable Diffusion, video clips, and other AI outputs, I needed a way to systematically download and organize everything before it disappeared. The tool has since evolved into a comprehensive solution for anyone who wants to preserve their Replicate predictions with intelligent organization and duplicate prevention.
|
|
210
|
-
|
|
211
|
-
## Quick Start
|
|
212
|
-
|
|
213
|
-
Get up and running in 2 minutes:
|
|
214
|
-
|
|
215
|
-
1. **Get your API token**: Visit [replicate.com/account/api-tokens](https://replicate.com/account/api-tokens)
|
|
216
|
-
|
|
217
|
-
2. **Set your token**:
|
|
218
|
-
```bash
|
|
219
|
-
export REPLICATE_API_TOKEN=your_token_here
|
|
220
|
-
```
|
|
221
|
-
|
|
222
|
-
3. **Install and run**:
|
|
223
|
-
```bash
|
|
224
|
-
# Clone and install
|
|
225
|
-
git clone https://github.com/closestfriend/replicate-predictions-downloader.git
|
|
226
|
-
cd replicate-predictions-downloader
|
|
227
|
-
npm install
|
|
228
|
-
|
|
229
|
-
# Download all your predictions
|
|
230
|
-
npm start
|
|
231
|
-
|
|
232
|
-
# Or just new ones since last run (recommended for regular use)
|
|
233
|
-
node index.js --last-run
|
|
234
|
-
```
|
|
235
|
-
|
|
236
|
-
That's it! Your predictions will be downloaded and organized by model and date.
|
|
237
|
-
|
|
238
|
-
## What You'll See
|
|
239
|
-
|
|
240
|
-
When you run the downloader, here's what happens:
|
|
241
|
-
|
|
242
|
-
```
|
|
243
|
-
Starting Replicate Predictions Downloader v2.0.0
|
|
244
|
-
Using date filter: since 2024-01-01
|
|
245
|
-
Fetching predictions from Replicate API...
|
|
246
|
-
Found 47 predictions to process
|
|
247
|
-
|
|
248
|
-
PROCESSING STATISTICS:
|
|
249
|
-
├── Total predictions: 47
|
|
250
|
-
├── Successful: 43 (91.5%)
|
|
251
|
-
├── Failed: 4 (8.5%)
|
|
252
|
-
└── Models found: 8
|
|
253
|
-
|
|
254
|
-
DOWNLOADING OUTPUTS:
|
|
255
|
-
├── stable-diffusion-xl: 25 images
|
|
256
|
-
├── midjourney-v6: 12 images
|
|
257
|
-
├── whisper-large-v3: 8 audio files
|
|
258
|
-
└── llama-2-70b-chat: 6 text files
|
|
259
|
-
|
|
260
|
-
SAVING TO: replicate_outputs_2024-01-15/
|
|
261
|
-
├── by-model/
|
|
262
|
-
│ ├── stable-diffusion-xl/
|
|
263
|
-
│ ├── midjourney-v6/
|
|
264
|
-
│ └── ...
|
|
265
|
-
├── stable-diffusion-xl.zip
|
|
266
|
-
├── midjourney-v6.zip
|
|
267
|
-
└── replicate_metadata_2024-01-15.json
|
|
268
|
-
|
|
269
|
-
Complete! Downloaded 51 files (127.3 MB)
|
|
270
|
-
Elapsed time: 2m 34s
|
|
271
|
-
Next time, use --last-run to download only new predictions
|
|
272
|
-
```
|
|
273
221
|
|
|
274
222
|
## Common Issues
|
|
275
223
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "replicate-predictions-downloader",
|
|
3
|
-
"version": "2.0.
|
|
3
|
+
"version": "2.0.2",
|
|
4
4
|
"description": "A Node.js script to download and organize all your Replicate predictions, including images, metadata, and other outputs",
|
|
5
5
|
"main": "index.js",
|
|
6
6
|
"type": "module",
|