replicate-predictions-downloader 2.0.0 → 2.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +97 -3
- package/package.json +9 -2
package/README.md
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
|
-
# Replicate Predictions Downloader
|
|
1
|
+
# Replicate Predictions Downloader
|
|
2
2
|
|
|
3
3
|
A Node.js script to download and organize all your Replicate predictions, including images, metadata, and other outputs before they expire.
|
|
4
4
|
|
|
5
|
-

|
|
6
6
|
|
|
7
7
|
<div align="center">
|
|
8
8
|
|
|
@@ -204,4 +204,98 @@ This project is licensed under the MIT License - see the [LICENSE.md](LICENSE.md
|
|
|
204
204
|
|
|
205
205
|
## Background
|
|
206
206
|
|
|
207
|
-
This tool was created while working on LLM behavior/personality research
|
|
207
|
+
Replicate is a powerful platform for running AI models, but predictions don't last forever - they expire after a period of time. This tool was originally created while working on LLM behavior/personality research when I realized I was losing valuable generated content.
|
|
208
|
+
|
|
209
|
+
After generating hundreds of images with Stable Diffusion, video clips, and other AI outputs, I needed a way to systematically download and organize everything before it disappeared. The tool has since evolved into a comprehensive solution for anyone who wants to preserve their Replicate predictions with intelligent organization and duplicate prevention.
|
|
210
|
+
|
|
211
|
+
## Quick Start
|
|
212
|
+
|
|
213
|
+
Get up and running in 2 minutes:
|
|
214
|
+
|
|
215
|
+
1. **Get your API token**: Visit [replicate.com/account/api-tokens](https://replicate.com/account/api-tokens)
|
|
216
|
+
|
|
217
|
+
2. **Set your token**:
|
|
218
|
+
```bash
|
|
219
|
+
export REPLICATE_API_TOKEN=your_token_here
|
|
220
|
+
```
|
|
221
|
+
|
|
222
|
+
3. **Install and run**:
|
|
223
|
+
```bash
|
|
224
|
+
# Clone and install
|
|
225
|
+
git clone https://github.com/closestfriend/replicate-predictions-downloader.git
|
|
226
|
+
cd replicate-predictions-downloader
|
|
227
|
+
npm install
|
|
228
|
+
|
|
229
|
+
# Download all your predictions
|
|
230
|
+
npm start
|
|
231
|
+
|
|
232
|
+
# Or just new ones since last run (recommended for regular use)
|
|
233
|
+
node index.js --last-run
|
|
234
|
+
```
|
|
235
|
+
|
|
236
|
+
That's it! Your predictions will be downloaded and organized by model and date.
|
|
237
|
+
|
|
238
|
+
## What You'll See
|
|
239
|
+
|
|
240
|
+
When you run the downloader, here's what happens:
|
|
241
|
+
|
|
242
|
+
```
|
|
243
|
+
Starting Replicate Predictions Downloader v2.0.0
|
|
244
|
+
Using date filter: since 2024-01-01
|
|
245
|
+
Fetching predictions from Replicate API...
|
|
246
|
+
Found 47 predictions to process
|
|
247
|
+
|
|
248
|
+
PROCESSING STATISTICS:
|
|
249
|
+
├── Total predictions: 47
|
|
250
|
+
├── Successful: 43 (91.5%)
|
|
251
|
+
├── Failed: 4 (8.5%)
|
|
252
|
+
└── Models found: 8
|
|
253
|
+
|
|
254
|
+
DOWNLOADING OUTPUTS:
|
|
255
|
+
├── stable-diffusion-xl: 25 images
|
|
256
|
+
├── midjourney-v6: 12 images
|
|
257
|
+
├── whisper-large-v3: 8 audio files
|
|
258
|
+
└── llama-2-70b-chat: 6 text files
|
|
259
|
+
|
|
260
|
+
SAVING TO: replicate_outputs_2024-01-15/
|
|
261
|
+
├── by-model/
|
|
262
|
+
│ ├── stable-diffusion-xl/
|
|
263
|
+
│ ├── midjourney-v6/
|
|
264
|
+
│ └── ...
|
|
265
|
+
├── stable-diffusion-xl.zip
|
|
266
|
+
├── midjourney-v6.zip
|
|
267
|
+
└── replicate_metadata_2024-01-15.json
|
|
268
|
+
|
|
269
|
+
Complete! Downloaded 51 files (127.3 MB)
|
|
270
|
+
Elapsed time: 2m 34s
|
|
271
|
+
Next time, use --last-run to download only new predictions
|
|
272
|
+
```
|
|
273
|
+
|
|
274
|
+
## Common Issues
|
|
275
|
+
|
|
276
|
+
**"Invalid API token" or "Authentication failed"**
|
|
277
|
+
- Double-check your token at [replicate.com/account/api-tokens](https://replicate.com/account/api-tokens)
|
|
278
|
+
- Make sure you've set `REPLICATE_API_TOKEN` in your environment or `.env` file
|
|
279
|
+
- Try: `echo $REPLICATE_API_TOKEN` to verify it's set
|
|
280
|
+
|
|
281
|
+
**"Network timeout" or connection errors**
|
|
282
|
+
- Check your internet connection
|
|
283
|
+
- The tool automatically retries failed downloads
|
|
284
|
+
- For persistent issues, try running with fewer concurrent requests by editing the CONFIG object
|
|
285
|
+
|
|
286
|
+
**"Permission denied" or file errors**
|
|
287
|
+
- Ensure you have write permissions in the current directory
|
|
288
|
+
- On Windows, try running as administrator if needed
|
|
289
|
+
- Make sure you have enough disk space for your downloads
|
|
290
|
+
|
|
291
|
+
**"No new predictions found"**
|
|
292
|
+
- If using `--last-run`, this means you're up to date!
|
|
293
|
+
- Try `--since "1 week ago"` to see recent predictions
|
|
294
|
+
- Use `--all` to redownload everything (may create duplicates)
|
|
295
|
+
|
|
296
|
+
**Files have weird names or special characters**
|
|
297
|
+
- The tool automatically sanitizes filenames
|
|
298
|
+
- Prompts longer than the limit (default: 50 chars) are truncated
|
|
299
|
+
- Adjust `maxPromptLength` in CONFIG if needed
|
|
300
|
+
|
|
301
|
+
**Need help?** Check existing [GitHub issues](https://github.com/closestfriend/replicate-predictions-downloader/issues) or create a new one.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "replicate-predictions-downloader",
|
|
3
|
-
"version": "2.0.
|
|
3
|
+
"version": "2.0.1",
|
|
4
4
|
"description": "A Node.js script to download and organize all your Replicate predictions, including images, metadata, and other outputs",
|
|
5
5
|
"main": "index.js",
|
|
6
6
|
"type": "module",
|
|
@@ -17,7 +17,14 @@
|
|
|
17
17
|
"predictions",
|
|
18
18
|
"organization",
|
|
19
19
|
"replicate-api",
|
|
20
|
-
"model-outputs"
|
|
20
|
+
"model-outputs",
|
|
21
|
+
"stable-diffusion",
|
|
22
|
+
"midjourney-alternative",
|
|
23
|
+
"ai-art",
|
|
24
|
+
"batch-download",
|
|
25
|
+
"backup",
|
|
26
|
+
"archive",
|
|
27
|
+
"cli"
|
|
21
28
|
],
|
|
22
29
|
"author": "closestfriend",
|
|
23
30
|
"license": "MIT",
|