pixivflow 2.0.41 → 2.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (50) hide show
  1. package/LICENSE +21 -674
  2. package/README.md +67 -574
  3. package/README_EN.md +71 -1523
  4. package/dist/commands/SchedulerCommand.d.ts.map +1 -1
  5. package/dist/commands/SchedulerCommand.js +5 -1
  6. package/dist/commands/SchedulerCommand.js.map +1 -1
  7. package/dist/download/DownloadManager.d.ts +9 -0
  8. package/dist/download/DownloadManager.d.ts.map +1 -1
  9. package/dist/download/DownloadManager.js +27 -0
  10. package/dist/download/DownloadManager.js.map +1 -1
  11. package/dist/download/pipeline/DownloadPipeline.d.ts +6 -0
  12. package/dist/download/pipeline/DownloadPipeline.d.ts.map +1 -1
  13. package/dist/download/pipeline/DownloadPipeline.js +4 -2
  14. package/dist/download/pipeline/DownloadPipeline.js.map +1 -1
  15. package/dist/package.json +1 -1
  16. package/dist/scheduler/Scheduler.d.ts +17 -4
  17. package/dist/scheduler/Scheduler.d.ts.map +1 -1
  18. package/dist/scheduler/Scheduler.js +58 -20
  19. package/dist/scheduler/Scheduler.js.map +1 -1
  20. package/dist/utils/errors.d.ts +9 -0
  21. package/dist/utils/errors.d.ts.map +1 -1
  22. package/dist/utils/errors.js +17 -1
  23. package/dist/utils/errors.js.map +1 -1
  24. package/dist/webui/package.json +1 -1
  25. package/dist/webui/routes/handlers/download-basic-handlers.d.ts.map +1 -1
  26. package/dist/webui/routes/handlers/download-basic-handlers.js +6 -20
  27. package/dist/webui/routes/handlers/download-basic-handlers.js.map +1 -1
  28. package/dist/webui/routes/handlers/download-url-handlers.d.ts +3 -6
  29. package/dist/webui/routes/handlers/download-url-handlers.d.ts.map +1 -1
  30. package/dist/webui/routes/handlers/download-url-handlers.js +72 -138
  31. package/dist/webui/routes/handlers/download-url-handlers.js.map +1 -1
  32. package/dist/webui/server/server-routes.d.ts.map +1 -1
  33. package/dist/webui/server/server-routes.js +6 -2
  34. package/dist/webui/server/server-routes.js.map +1 -1
  35. package/dist/webui/server/server.d.ts.map +1 -1
  36. package/dist/webui/server/server.js +25 -3
  37. package/dist/webui/server/server.js.map +1 -1
  38. package/dist/webui/services/DownloadTaskManager.d.ts +12 -0
  39. package/dist/webui/services/DownloadTaskManager.d.ts.map +1 -1
  40. package/dist/webui/services/DownloadTaskManager.js +119 -51
  41. package/dist/webui/services/DownloadTaskManager.js.map +1 -1
  42. package/dist/webui/utils/task-serialize.d.ts +31 -0
  43. package/dist/webui/utils/task-serialize.d.ts.map +1 -0
  44. package/dist/webui/utils/task-serialize.js +40 -0
  45. package/dist/webui/utils/task-serialize.js.map +1 -0
  46. package/dist/webui/websocket/DownloadStatus.d.ts +19 -0
  47. package/dist/webui/websocket/DownloadStatus.d.ts.map +1 -0
  48. package/dist/webui/websocket/DownloadStatus.js +61 -0
  49. package/dist/webui/websocket/DownloadStatus.js.map +1 -0
  50. package/package.json +11 -10
package/README_EN.md CHANGED
@@ -1,1571 +1,119 @@
1
- # ðŸŽĻ PixivFlow
1
+ # PixivFlow
2
2
 
3
- <div align="center">
3
+ **English** | [äļ­æ–‡](https://github.com/redtidev1918/PixivFlow/blob/master/README.md)
4
4
 
5
- **Intelligent Pixiv Automation Downloader | æ™ščƒ―įš„ Pixiv 臩åŠĻ化äļ‹č――å·Ĩ具**
5
+ Pixiv batch download and scheduled collection tool. Bulk-download
6
+ illustrations and novels, search by tag, filter by bookmarks and dates,
7
+ and run recurring collection jobs on a schedule. CLI and WebUI included.
8
+ Built with TypeScript and Node.js; runs on Windows, macOS, Linux and Docker.
6
9
 
7
- Make Pixiv artwork collection elegant and efficient | čŪĐ Pixiv ä―œå“æ”ķ集变åū—äž˜é›…č€ŒéŦ˜æ•ˆ
10
+ [![Version](https://img.shields.io/npm/v/pixivflow?style=flat-square)](https://www.npmjs.com/package/pixivflow)
11
+ [![Node](https://img.shields.io/badge/Node.js-18%2B_LTS-green.svg?style=flat-square&logo=node.js)](https://nodejs.org/)
12
+ [![License](https://img.shields.io/badge/License-MIT-blue.svg?style=flat-square)](LICENSE)
13
+ [![Documentation](https://img.shields.io/badge/Docs-redtidev1918.github.io-6366f1?style=flat-square)](https://redtidev1918.github.io/PixivFlow/)
8
14
 
9
- [![GitHub stars](https://img.shields.io/github/stars/zoidberg-xgd/pixivflow?style=for-the-badge&logo=github)](https://github.com/zoidberg-xgd/pixivflow/stargazers)
10
- [![GitHub forks](https://img.shields.io/github/forks/zoidberg-xgd/pixivflow?style=for-the-badge&logo=github)](https://github.com/zoidberg-xgd/pixivflow/network/members)
11
- [![GitHub issues](https://img.shields.io/github/issues/zoidberg-xgd/pixivflow?style=for-the-badge&logo=github)](https://github.com/zoidberg-xgd/pixivflow/issues)
12
- [![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg?style=for-the-badge)](https://www.gnu.org/licenses/gpl-3.0)
15
+ ## Install
13
16
 
14
- [![TypeScript](https://img.shields.io/badge/TypeScript-5.6+-blue.svg?style=flat-square&logo=typescript)](https://www.typescriptlang.org/)
15
- [![Node.js](https://img.shields.io/badge/Node.js-18%2B%20LTS-green.svg?style=flat-square&logo=node.js)](https://nodejs.org/)
16
- [![Platform](https://img.shields.io/badge/Platform-Windows%20%7C%20macOS%20%7C%20Linux-lightgrey.svg?style=flat-square)](https://github.com/zoidberg-xgd/pixivflow)
17
- [![Maintenance](https://img.shields.io/badge/Maintained-yes-green.svg?style=flat-square)](https://github.com/zoidberg-xgd/pixivflow/graphs/commit-activity)
18
-
19
- [Features](#features) â€Ē [Quick Start](#quick-start) â€Ē [CLI Commands](#cli-commands) â€Ē [Scripts](#script-tools) â€Ē [Use Cases](#use-cases)
20
-
21
- [📖 Full Documentation](https://zoidberg-xgd.github.io/PixivFlow/) | [English](README_EN.md) | [äļ­æ–‡](README.md)
22
-
23
- </div>
24
-
25
- ---
26
-
27
- <a id="table-of-contents"></a>
28
- ## Table of Contents
29
-
30
- <details>
31
- <summary><b>Click to expand full table of contents</b></summary>
32
-
33
- <br>
34
-
35
- **Getting Started**
36
- - [ðŸ’Ą What is PixivFlow?](#what-is-pixivflow)
37
- - [🌟 Why Choose PixivFlow?](#why-choose-pixivflow)
38
- - [ðŸŽŊ Core Philosophy](#core-philosophy)
39
- - [âœĻ Features](#features)
40
- - [🚀 Core Features](#core-features)
41
- - [🎁 Additional Advantages](#additional-advantages)
42
- - [🚀 Quick Start](#quick-start)
43
- - [📋 Requirements](#requirements)
44
- - [🎎 Quick Start (Recommended)](#quick-start-recommended)
45
- - [ðŸŽŊ Manual Setup](#manual-setup)
46
- - [🌐 API Server (Optional)](#api-server-optional)
47
- - [ðŸģ Using Docker (Recommended)](#using-docker-recommended)
48
-
49
- **Tools & Documentation**
50
- - [ðŸŽĻ CLI Commands](#cli-commands)
51
- - [🚀 Core Commands](#core-commands)
52
- - [⚙ïļ Configuration Management](#configuration-management)
53
- - [📊 Monitoring and Maintenance](#monitoring-and-maintenance)
54
- - [🛠ïļ Script Tools](#script-tools)
55
- - [ðŸŽŊ Main Control Script (Most Used)](#main-control-script-most-used)
56
- - [🔐 Login Management](#login-management)
57
- - [ðŸģ Docker Management](#docker-management)
58
- - [📚 Documentation](#documentation)
59
- - [🌟 Must Read for Beginners](#must-read-for-beginners)
60
- - [📘 Advanced Documentation](#advanced-documentation)
61
- - [ðŸģ Docker Documentation](#docker-documentation)
62
- - [📄 Project Documentation](#project-documentation)
63
-
64
- **Usage & Configuration**
65
- - [ðŸŽŊ Use Cases](#use-cases)
66
- - [Scenario 1: Daily Auto Collection](#scenario-1-daily-auto-collection-of-inspiration-materials)
67
- - [Scenario 2: Server Scheduled Collection](#scenario-2-server-scheduled-collection-of-specific-tags)
68
- - [Scenario 3: Quick Experience](#scenario-3-quick-experience---random-download)
69
- - [Scenario 4: One-Time Batch Download](#scenario-4-onetime-batch-download)
70
- - [📁 Project Structure](#project-structure)
71
- - [⚙ïļ Core Configuration](#core-configuration)
72
- - [Authentication Configuration](#authentication-configuration)
73
- - [Download Targets](#download-targets)
74
- - [Scheduled Tasks](#scheduled-tasks)
75
- - [Storage Configuration](#storage-configuration)
76
-
77
- **Troubleshooting & Advanced**
78
- - [🐛 FAQ](docs/USAGE.md)
79
- - [🔒 Security Tips](docs/USAGE.md)
80
- - [📊 Download Record Management](docs/USAGE.md)
81
- - [🚀 Advanced Usage](docs/USAGE.md)
82
- - [Deploy on Server](docs/DOCKER.md)
83
- - [Configure Multiple Download Tasks](docs/CONFIG.md)
84
- - [Using Proxy](docs/USAGE.md)
85
-
86
- **Project Information**
87
- - [📄 Open Source License](#open-source-license)
88
- - [🙏 Acknowledgments](#acknowledgments)
89
- - [ðŸ“Ū Get Help](#get-help)
90
- - [📈 Project Statistics](#project-statistics)
91
- - [ðŸĪ Contributing](#contributing)
92
- - [📝 Changelog](#changelog)
93
- - [Support the Project](#support-the-project)
94
-
95
- </details>
96
-
97
- ---
98
-
99
- <a id="what-is-pixivflow"></a>
100
- ## What is PixivFlow?
101
-
102
- **PixivFlow** is a **fully standalone** Pixiv artwork batch downloader designed for automation. No browser extension required, can run automatically in command line or on servers, supports scheduled tasks, intelligent deduplication, resume download, and more.
103
-
104
- As a professional **Pixiv downloader**, PixivFlow supports the following core features:
105
- - ðŸ“Ĩ **Batch Download Pixiv Illustrations and Novels**: Supports batch downloading artworks by tags, users, bookmarks, and other conditions
106
- - 🔗 **Direct URL Download**: Supports direct input of Pixiv URLs for download without modifying configuration files
107
- - ðŸŽŊ **Single Artwork Download**: Supports downloading single illustrations or novels by artwork ID
108
- - ⏰ **Scheduled Automatic Download**: Uses Cron expressions to configure scheduled tasks for automated downloads
109
-
110
- <a id="why-choose-pixivflow"></a>
111
- ### Why Choose PixivFlow?
112
-
113
- Compared to other Pixiv downloaders, PixivFlow focuses on **automation** and **server deployment** scenarios:
114
-
115
- | Advantage | Description |
116
- |-----------|-------------|
117
- | 🚀 **Fully Standalone** | No browser extension required, pure CLI tool that can run in any environment (servers, Docker, CI/CD) |
118
- | ðŸĪ– **True Automation** | Set once, run forever. Supports Cron scheduled tasks, no manual intervention needed |
119
- | ðŸ–Ĩïļ **Server Friendly** | Designed for servers, supports background running, process management, log rotation |
120
- | 🔐 **Secure & Reliable** | Uses OAuth 2.0 PKCE standard flow to ensure account security, avoids password leakage risks |
121
- | ðŸ“Ķ **Lightweight Deployment** | Low resource usage, no additional services needed (like databases, Redis), SQLite is enough |
122
- | 🛠ïļ **Out of the Box** | Rich script tools and setup wizard, get started in 3 steps |
123
-
124
- <a id="core-philosophy"></a>
125
- ### Core Philosophy
126
-
127
- - **Automation First**: Set once, run automatically, no manual intervention
128
- - **Intelligent Management**: Auto deduplication, resume download, error retry
129
- - **Simple & Easy**: 3 steps to get started, setup wizard guides you through
130
- - **Out of the Box**: Rich script tools, no need to remember complex commands
131
-
132
- ---
133
-
134
- <a id="features"></a>
135
- ## Features
136
-
137
- <a id="core-features"></a>
138
- ### Core Features
139
-
140
- | Feature | Description |
141
- |---------|-------------|
142
- | **ðŸ“Ĩ Batch Download** | Supports batch download of illustrations and novels, configurable download count and filters |
143
- | **🔗 Direct URL Download** | Supports direct input of Pixiv URLs for download without modifying configuration files |
144
- | **ðŸŽŊ Single Artwork Download** | Supports downloading single illustrations or novels by artwork ID |
145
- | **🏷ïļ Tag Search** | Search artworks by tags, supports exact match, partial match, and other modes |
146
- | **ðŸŽē Random Download** | One-click download random popular tag artworks for quick experience |
147
- | **⏰ Scheduled Tasks** | Cron expression configuration, supports daily, weekly, monthly automatic downloads |
148
- | **🔍 Smart Filtering** | Multi-dimensional filtering by bookmarks, date range, artwork type, etc. |
149
- | **ðŸ’ū Auto Deduplication** | SQLite database records history, automatically skips already downloaded artworks |
150
- | **🔄 Resume Download** | Automatically resumes after interruption, no need to restart |
151
- | **ðŸ›Ąïļ Error Handling** | Auto retry, error recovery, smart skip deleted/private artworks |
152
- | **ðŸ“Ą RESTful API** | Complete REST API with authentication, configuration, download, and statistics |
153
- | **🔌 WebSocket** | Real-time log streaming and download status updates |
154
- | **📊 Statistics Reports** | Detailed run logs and download statistics |
155
-
156
- <a id="additional-advantages"></a>
157
- ### Additional Advantages
158
-
159
- - ✅ **Fully Standalone**: No browser required, pure CLI tool
160
- - ✅ **API Server**: Provides RESTful API and WebSocket, can integrate with any frontend
161
- - ✅ **npm Package**: Can be installed as npm package, supports global and local installation
162
- - ✅ **Lightweight**: Low resource usage, suitable for long-term server running
163
- - ✅ **Open Source**: GPL-3.0 license, free to customize and distribute
164
- - ✅ **Type Safe**: Written in TypeScript with complete type hints
165
- - ✅ **Well Documented**: Detailed documentation and tutorials
166
-
167
- ---
168
-
169
- <a id="quick-start"></a>
170
- ## Quick Start
171
-
172
- <a id="requirements"></a>
173
- ### Requirements
174
-
175
- - **Node.js 18+** and **npm 9+** (Recommended: LTS versions 18.x, 20.x, 22.x, or 24.x)
176
- - **Pixiv account**
177
- - **Windows users**: Recommended to use WSL (`wsl --install`) or Git Bash
178
-
179
- > ⚠ïļ **Node.js Version Note**:
180
- > - Recommended to use **LTS (Long Term Support) versions**: 18.x, 20.x, 22.x, or 24.x
181
- > - Avoid using odd-numbered versions (e.g., 19.x, 21.x, 23.x), as these may not be supported by all dependencies
182
- > - If you see `EBADENGINE` warnings, consider switching to an LTS version
183
- >
184
- > ðŸ’Ą **Login Note**: The project uses Node.js library for login by default, **Python is not required**. Python gppt is only used as a fallback option (optional).
185
- > 📖 **Detailed Guide**: See [Quick Start Guide](docs/QUICKSTART.md)
186
-
187
- <a id="quick-start-recommended"></a>
188
- ### Quick Installation (Recommended ⭐)
189
-
190
- #### Method 1: Install from npm (Easiest)
17
+ Requires Node.js 18 or later (LTS).
191
18
 
192
19
  ```bash
193
- # Install globally from npm
194
20
  npm install -g pixivflow
195
-
196
- # Verify installation
197
21
  pixivflow --help
198
-
199
- # Login to account
200
- pixivflow login
201
-
202
- # Start downloading
203
- pixivflow download
204
- ```
205
-
206
- #### Method 2: Install from Source
207
-
208
- ```bash
209
- # 1. Clone the repository
210
- git clone https://github.com/zoidberg-xgd/pixivflow.git
211
- cd pixivflow
212
-
213
- # 2. Install dependencies
214
- npm install
215
-
216
- # 3. Login to account
217
- npm run login
218
-
219
- # 4. Start downloading
220
- npm run download
221
- ```
222
-
223
- **Or use one-click script** (automatically completes all setup):
224
-
225
- ```bash
226
- ./scripts/quick-start.sh
227
- ```
228
-
229
- ---
230
-
231
- ### 🌐 Global Installation (Optional)
232
-
233
- If you want to use `pixivflow` command from any directory, you can install it globally:
234
-
235
- #### Method 1: Install from npm (Recommended ⭐)
236
-
237
- ```bash
238
- # Install globally from npm
239
- npm install -g pixivflow
240
22
  ```
241
23
 
242
- #### Method 2: Install from Local Directory
24
+ For servers, prefer the Docker Compose setup described in
25
+ [DOCKER.md](docs/DOCKER.md). To build from source:
243
26
 
244
27
  ```bash
245
- # 1. Clone the repository
246
- git clone https://github.com/zoidberg-xgd/pixivflow.git
247
- cd pixivflow
248
-
249
- # 2. Install dependencies
28
+ git clone https://github.com/redtidev1918/PixivFlow.git
29
+ cd PixivFlow
250
30
  npm install
251
-
252
- # 3. Build the project
253
31
  npm run build
254
-
255
- # 4. Install globally
256
- npm install -g .
257
- ```
258
-
259
- #### Method 3: Install Directly from GitHub Repository
260
-
261
- ```bash
262
- # Install globally from GitHub repository
263
- npm install -g git+https://github.com/zoidberg-xgd/pixivflow.git
264
-
265
- # Note: After installation, you may need to build
266
- # Find the global installation directory (usually in npm root -g's parent directory/bin)
267
- # Or reinstall with build script specified
268
32
  ```
269
33
 
270
- #### Verify Global Installation
271
-
272
- After installation, you can use the `pixivflow` command from any directory:
273
-
274
- ```bash
275
- # View help
276
- pixivflow --help
277
-
278
- # Check installation location
279
- which pixivflow
280
-
281
- # Login to account
282
- pixivflow login
283
-
284
- # Execute download
285
- pixivflow download
286
-
287
- # Start scheduled task
288
- pixivflow scheduler
289
-
290
- # Normalize and reorganize downloaded files
291
- pixivflow normalize
292
- pixivflow normalize --dry-run # Preview changes without applying
293
-
294
- # Migrate configuration paths (convert absolute to relative)
295
- pixivflow migrate-config
296
- pixivflow migrate-config --dry-run # Preview migration changes
297
- ```
34
+ Termux / Android: see [TERMUX_INSTALL.md](docs/TERMUX_INSTALL.md).
298
35
 
299
- #### Configuration After Global Installation
36
+ ## Quick start
300
37
 
301
- After global installation, the configuration file location:
302
- - Configuration file: `~/.pixivflow/config/standalone.config.json` (if using default path)
303
- - Or use `--config` parameter to specify configuration file path
38
+ Sign in to your Pixiv account (creates OAuth credentials, one time):
304
39
 
305
40
  ```bash
306
- # Use custom configuration file
307
- pixivflow download --config /path/to/config.json
308
-
309
- # Login (will automatically create configuration file)
310
- pixivflow login
41
+ pixivflow login # desktop with a browser
42
+ pixivflow login-headless # headless server
311
43
  ```
312
44
 
313
- #### Update Global Installation
314
-
315
- If you need to update the globally installed version:
45
+ Download something — paste any Pixiv link (artwork, novel, series, or a user
46
+ profile are all recognized):
316
47
 
317
48
  ```bash
318
- # Method 1: Reinstall (from local directory)
319
- cd /path/to/pixivflow
320
- npm run build
321
- npm install -g .
322
-
323
- # Method 2: Update from GitHub
324
- npm install -g git+https://github.com/zoidberg-xgd/pixivflow.git
49
+ pixivflow download --url https://www.pixiv.net/artworks/123456789
325
50
  ```
326
51
 
327
- > **ðŸ’Ą Tips**:
328
- > - After global installation, you can use the `pixivflow` command directly from any directory
329
- > - First use requires running `pixivflow login` to login
330
- > - Configuration file will be automatically created in user home directory or project directory
331
- > - npm package: https://www.npmjs.com/package/pixivflow
332
-
333
- ---
334
-
335
- ### ðŸŽŊ Manual Setup
336
-
337
- If you want to manually control each step:
338
-
339
- #### 1ïļâƒĢ Install Dependencies
52
+ Bulk-download from your configured targets and start the scheduler:
340
53
 
341
54
  ```bash
342
- npm install
343
- ```
344
-
345
- #### 2ïļâƒĢ Login to Pixiv Account
346
-
347
- ```bash
348
- # Simplest way: Enter username and password in terminal (recommended)
349
- npm run login
350
-
351
- # Or use setup wizard (pure terminal login, interactive config)
352
- ./scripts/easy-setup.sh
353
- # or
354
- npm run setup
355
- ```
356
-
357
- **Login Notes**:
358
- - ✅ **Default login mode**: Uses Node.js library for login (no Python required)
359
- - ✅ **Python fallback option**: Python gppt can be used as a fallback option (optional)
360
- - ✅ Auto update config: Automatically updates refresh token in config after successful login
361
- - ✅ Setup wizard: Use `npm run setup` for interactive configuration
362
-
363
- #### 3ïļâƒĢ Configure Download Options (Optional)
364
-
365
- If using `npm run login` to login, you can configure download options later:
366
-
367
- ```bash
368
- # Run setup wizard
369
- ./scripts/easy-setup.sh
370
- # or
371
- npm run setup
372
- ```
373
-
374
- The setup wizard will automatically complete all settings, including:
375
- - ⚙ïļ Configure download options (tags, quantity, filters, etc.)
376
- - ⏰ Scheduled task settings
377
-
378
- #### 4ïļâƒĢ Start Downloading
379
-
380
- ```bash
381
- # Test download (recommended for first use)
382
- ./scripts/pixiv.sh test
383
-
384
- # Execute one download
385
- ./scripts/pixiv.sh once
386
-
387
- # Start scheduled task
388
- ./scripts/pixiv.sh run
389
- ```
390
-
391
- That's it! 🎉
392
-
393
- > **ðŸ’Ą Tip**: For first use, it's recommended to run `test` to download 1-2 artworks first, confirm the configuration is correct before formal use.
394
- >
395
-
396
- ---
397
-
398
- <a id="api-server-optional"></a>
399
- ### WebUI Backend API (Optional)
400
-
401
- PixivFlow provides a WebUI backend API service that supports management through RESTful API and WebSocket:
402
-
403
- ```bash
404
- # Start WebUI backend API service
405
- pixivflow webui # Default port 3000
406
-
407
- # Or specify port via environment variable
408
- PORT=8080 pixivflow webui
409
- ```
410
-
411
- **Main Features**:
412
- - **Authentication API**: Login, logout, Token management
413
- - **Configuration Management API**: View, edit, backup, restore configuration
414
- - **Download Management API**: Start/stop downloads, view task status
415
- - **Statistics API**: Download statistics, file list
416
- - **Logs API**: View running logs (supports WebSocket real-time push)
417
- - **File Management API**: File list, preview, operations
418
-
419
- > **Note**:
420
- > - PixivFlow only provides backend API, does not include frontend interface
421
- > - **Frontend Project**: Modern React frontend interface, see [pixivflow-webui](https://github.com/zoidberg-xgd/pixivflow-webui)
422
- > - Can be called directly via API, or connect to other frontend projects
423
- > - **API Documentation**: See [Usage Guide](docs/USAGE.md) for detailed API usage
424
-
425
-
426
- ---
427
-
428
- <a id="using-docker-recommended"></a>
429
- ### Using Docker (Recommended)
430
-
431
- PixivFlow supports Docker deployment, no need to install Node.js environment:
432
-
433
- #### Quick Start
434
-
435
- ```bash
436
- # 1. Prepare configuration file
437
- cp config/standalone.config.example.json config/standalone.config.json
438
-
439
- # 2. Login to Pixiv account (on host)
440
- npm run login
441
-
442
- # 3. Start scheduled task service
443
- docker-compose up -d pixivflow
444
-
445
- # Or start WebUI service
446
- docker-compose up -d pixivflow-webui
447
-
448
- # Or start both services
449
- docker-compose up -d
450
- ```
451
-
452
- #### Using Script Tools
453
-
454
- ```bash
455
- # 1. Initialize Docker environment
456
- ./scripts/pixiv.sh docker setup
457
-
458
- # 2. Login to Pixiv account
459
- ./scripts/pixiv.sh docker login
460
-
461
- # 3. Build and deploy
462
- ./scripts/pixiv.sh docker deploy
463
-
464
- # 4. Check status
465
- ./scripts/pixiv.sh docker status
466
-
467
- # 5. View logs
468
- ./scripts/pixiv.sh docker logs -f
469
- ```
470
-
471
- #### Docker Services
472
-
473
- `docker-compose.yml` provides two services:
474
-
475
- 1. **pixivflow** - Scheduled task service (default)
476
- - Automatically executes scheduled download tasks
477
- - Runs continuously in background
478
-
479
- 2. **pixivflow-webui** - WebUI management interface (optional)
480
- - Provides modern web management interface
481
- - Access address: http://localhost:3000
482
- - Supports file browsing, statistics viewing, task management, etc.
483
-
484
- #### Docker Common Commands
485
-
486
- ```bash
487
- # Start scheduled task service
488
- docker-compose up -d pixivflow
489
-
490
- # Start WebUI service
491
- docker-compose up -d pixivflow-webui
492
-
493
- # Start both services
494
- docker-compose up -d
495
-
496
- # View logs
497
- docker-compose logs -f pixivflow
498
- docker-compose logs -f pixivflow-webui
499
-
500
- # Stop services
501
- docker-compose stop
502
-
503
- # Stop and remove containers
504
- docker-compose down
505
-
506
- # Rebuild images
507
- docker-compose build
508
- ```
509
-
510
- #### Docker Script Commands
511
-
512
- - `docker setup` - Initialize Docker environment
513
- - `docker build` - Build Docker image
514
- - `docker deploy` - Deploy service (build + start)
515
- - `docker up` - Start service
516
- - `docker down` - Stop service
517
- - `docker status` - View service status
518
- - `docker logs` - View logs
519
- - `docker login` - Login to account in container
520
- - `docker random` - Random download artwork (for testing)
521
-
522
- > 📖 **Detailed Instructions**: See [Docker Usage Guide](docs/DOCKER.md)
523
-
524
- ---
525
-
526
- <a id="cli-commands"></a>
527
- ## CLI Commands
528
-
529
- > ðŸ’Ą **Recommended**: After global installation, you can directly use the `pixivflow` command without relying on scripts in the project directory.
530
-
531
- <a id="core-commands"></a>
532
- ### Core Commands
533
-
534
- ```bash
535
- # Use after global installation
536
- pixivflow login # Login to Pixiv account
537
- pixivflow download # Execute download
538
- pixivflow download --url <url> # Download directly from URL (supports illustration/novel/series) ⭐
539
- pixivflow random # Random download
540
- pixivflow scheduler # Start scheduled task
541
- pixivflow normalize # Normalize files
542
- pixivflow migrate-config # Migrate config
543
- pixivflow health # Health check (recommended ⭐)
544
- pixivflow status # View download statistics and recent records
545
- pixivflow logs # View run logs
546
- pixivflow setup # Interactive setup wizard (first time use) ⭐
547
- pixivflow dirs # View directory information (where files are saved) ⭐
548
- ```
549
-
550
- **URL Download Examples**:
551
- ```bash
552
- # Download illustration
553
- pixivflow download --url "https://www.pixiv.net/artworks/12345678"
554
-
555
- # Download novel
556
- pixivflow download --url "https://www.pixiv.net/novel/show.php?id=26132156"
557
-
558
- # Download novel series
559
- pixivflow download --url "https://www.pixiv.net/novel/series/14690617"
560
-
561
- # Download all works from a user
562
- pixivflow download --url "https://www.pixiv.net/users/123456"
563
- ```
564
-
565
- <a id="configuration-management"></a>
566
- ### Configuration Management
567
-
568
- ```bash
569
- pixivflow config # Config management (view/edit/backup/restore) ⭐
570
- pixivflow config show # View config
571
- pixivflow config set <key> <value> # Set config value (e.g., storage.downloadDirectory) ⭐
572
- pixivflow config backup # Backup config
573
- pixivflow config restore # Restore config
574
- pixivflow config validate # Validate config
575
- pixivflow config edit # Edit config
576
- ```
577
-
578
- **Config Set Examples**:
579
- ```bash
580
- # Set download directory
581
- pixivflow config set storage.downloadDirectory ./my-downloads
582
-
583
- # Set illustration directory
584
- pixivflow config set storage.illustrationDirectory ./my-illustrations
585
-
586
- # Set novel directory
587
- pixivflow config set storage.novelDirectory ./my-novels
588
- ```
589
-
590
- <a id="monitoring-and-maintenance"></a>
591
- ### Monitoring and Maintenance
592
-
593
- ```bash
594
- pixivflow monitor # Real-time monitoring of process status and performance metrics ⭐
595
- pixivflow maintain # Auto maintenance (clean logs, optimize database, etc.) ⭐
596
- pixivflow backup # Auto backup config and data ⭐
597
- ```
598
-
599
- > 📖 **Detailed Instructions**: See [Script Usage Guide](docs/SCRIPTS.md)
600
-
601
- ---
602
-
603
- <a id="script-tools"></a>
604
- ## Script Tools
605
-
606
- PixivFlow provides rich script tools so you don't need to remember complex npm commands. All scripts directly call built-in CLI functions for better performance and faster response.
607
-
608
- <a id="main-control-script-most-used"></a>
609
- ### Main Control Script (Most Used)
610
-
611
- ```bash
612
- ./scripts/pixiv.sh <command>
613
- ```
614
-
615
- | Command | Description |
616
- |---------|-------------|
617
- | `setup` | Interactive setup wizard (must run first time) |
618
- | `login` | Login to Pixiv account (interactive, directly calls built-in CLI) |
619
- | `test` | Test download (download small amount to verify config) |
620
- | `once` | Execute one download |
621
- | `random` | Random download one popular tag artwork (supports `--novel` for novels) |
622
- | `run` | Start scheduled task (runs continuously in background) |
623
- | `stop` | Stop running scheduled task |
624
- | `status` | View current running status |
625
- | `check` | Environment check (supports `--fix` for auto-fix) ⭐ New |
626
- | `update` | One-click update and fix (update code, dependencies, fix errors) ⭐ New |
627
- | `health` | Health check (check config, network, etc.) |
628
- | `logs` | View run logs |
629
-
630
- **ðŸ’Ą Tip**:
631
- - All commands directly call built-in CLI, no need to go through npm scripts, faster response.
632
- - **New Features**: `check --fix` and `update` commands support automatic fixing of common issues.
633
-
634
- <a id="login-management"></a>
635
- ### Login Management
636
-
637
- ```bash
638
- # Method 1: Use main control script (recommended)
639
- ./scripts/pixiv.sh login
640
-
641
- # Method 2: Use login script (supports more options)
642
- ./scripts/login.sh
643
-
644
- # Method 3: Use npm command
645
- npm run login
646
-
647
- # Headless login (provide username and password via parameters)
648
- ./scripts/pixiv.sh login -u your_username -p your_password
649
- ```
650
-
651
- ### ⚙ïļ Configuration Management
652
-
653
- ```bash
654
- # Interactive setup wizard
655
- ./scripts/easy-setup.sh
656
-
657
- # Configuration path migration (auto-fix absolute paths, for project migration)
658
- pixivflow migrate-config # Execute migration
659
- pixivflow migrate-config --dry-run # Preview changes
660
- pixivflow migrate-config --json # JSON format output
661
- # Or use npm command
662
- npm run start migrate-config # Execute migration
663
- npm run start migrate-config --dry-run # Preview changes
664
- ```
665
-
666
- ### 🔧 Environment Check and Auto-Fix ⭐ New Feature
667
-
668
- ```bash
669
- # Basic environment check
670
- ./scripts/pixiv.sh check
671
-
672
- # Auto-fix environment issues (recommended ⭐)
673
- ./scripts/pixiv.sh check --fix
674
-
675
- # One-click update and fix (update code, dependencies, fix errors)
676
- ./scripts/pixiv.sh update
677
-
678
- # Or use alias
679
- ./scripts/pixiv.sh fix
55
+ pixivflow download
56
+ pixivflow scheduler # long-running cron collection
680
57
  ```
681
58
 
682
- **New Feature Description**:
683
- - ✅ `check --fix`: Automatically install missing dependencies, create config, compile code
684
- - ✅ `update`: One-click update code, dependencies, and fix common errors
685
- - ✅ Smart detection: Automatically detect if build artifacts are outdated
686
- - ✅ Unified error handling: Provide clear error messages and fix suggestions
59
+ ## Download targets
687
60
 
688
- ### Deployment and Backup
61
+ Define what to collect in the `targets` section of your config. Conditions
62
+ combine:
689
63
 
690
- ```bash
691
- # Auto deploy to server
692
- ./scripts/auto-deploy.sh
693
- ```
64
+ | Field | Meaning | Example |
65
+ | --- | --- | --- |
66
+ | `type` | `illustration` or `novel` | `illustration` |
67
+ | `tag` | Search tag(s); multiple tags are OR'ed | `"éĒĻæ™Ŋ"` / `["watercolor","impasto"]` |
68
+ | `limit` | Max items per run | `20` |
69
+ | `minBookmarks` | Minimum bookmark count | `500` |
70
+ | `startDate` / `endDate` | Publish date range | `"2025-01-01"` |
694
71
 
695
- <a id="docker-management"></a>
696
- ### Docker Management
72
+ Downloaded items are tracked in a SQLite database and skipped automatically;
73
+ files that exist without a database record are reconciled, so the two never
74
+ conflict.
697
75
 
698
- ```bash
699
- # Use main control script
700
- ./scripts/pixiv.sh docker <command>
76
+ Interactive configuration wizard: `pixivflow setup`.
701
77
 
702
- # Or directly use Docker script
703
- ./scripts/docker.sh <command>
704
- ```
78
+ ## Common commands
705
79
 
706
- **Common Commands**:
707
- - `docker setup` - Initialize Docker environment
708
- - `docker build` - Build image
709
- - `docker deploy` - Deploy service
710
- - `docker up` - Start service
711
- - `docker down` - Stop service
712
- - `docker status` - View status
713
- - `docker logs` - View logs
714
- - `docker login` - Login to account
715
- - `docker test` - Test download
80
+ | Command | Purpose |
81
+ | --- | --- |
82
+ | `pixivflow download` | Run downloads per config |
83
+ | `pixivflow download --url <url>` | Direct download via URL |
84
+ | `pixivflow random` | Random popular artwork |
85
+ | `pixivflow scheduler` | Start scheduled jobs |
86
+ | `pixivflow config` | Manage config (view / edit / backup / restore) |
87
+ | `pixivflow status` | Download stats and recent records |
88
+ | `pixivflow health` | Health check: config, directories, connectivity |
716
89
 
717
- > 📖 **Detailed Instructions**: See [Docker Usage Guide](docs/DOCKER.md)
90
+ More commands in [CLI_MIGRATION_SUMMARY.md](docs/CLI_MIGRATION_SUMMARY.md).
718
91
 
719
- ---
720
-
721
- <a id="documentation"></a>
722
92
  ## Documentation
723
93
 
724
- > 📖 **Complete Documentation Index**: See [Documentation Navigation](docs/README.md) for a complete list and classification of all documents
725
-
726
- <a id="must-read-for-beginners"></a>
727
- ### Must Read for Beginners (Read in Order)
94
+ Full tutorial site: <https://redtidev1918.github.io/PixivFlow/>
728
95
 
729
96
  | Document | Description |
730
- |----------|-------------|
731
- | [⚡ QUICKSTART](docs/QUICKSTART.md) | **3-minute quick start** - Fastest way to get started |
732
- | [🔐 LOGIN](docs/LOGIN.md) | **Login process details** - Login problem solutions |
733
- | [📖 USAGE](docs/USAGE.md) | **Usage guide** - Feature usage instructions |
734
-
735
- <a id="feature-guides"></a>
736
- ### Feature Guides
737
-
738
- | Document | Description |
739
- |----------|-------------|
740
- | [📋 CONFIG](docs/CONFIG.md) | **Configuration file usage guide** - All configuration options explained |
741
- | [🛠ïļ SCRIPTS](docs/SCRIPTS.md) | **Script usage guide** - All scripts detailed instructions |
742
-
743
- <a id="deployment-and-environment"></a>
744
- ### Deployment and Environment
745
-
746
- | Document | Description |
747
- |----------|-------------|
748
- | [ðŸģ DOCKER](docs/DOCKER.md) | **Docker usage guide** - Docker deployment and usage (includes solutions for common issues) |
749
-
750
- <a id="project-documentation"></a>
751
- ### Project Documentation
752
-
753
- | Document | Description |
754
- |----------|-------------|
755
- | [🏗ïļ ARCHITECTURE](docs/ARCHITECTURE.md) | **Architecture documentation** - Project architecture and technical implementation details |
756
- | [🔄 CLI_MIGRATION](docs/CLI_MIGRATION_SUMMARY.md) | **CLI command migration summary** - CLI command migration and feature comparison |
757
- | [📝 CHANGELOG](docs/project/CHANGELOG.md) | Version changelog |
758
- | [ðŸĪ CONTRIBUTING](docs/project/CONTRIBUTING.md) | Contributing guide |
759
-
760
- ---
761
-
762
- <a id="use-cases"></a>
763
- ## Use Cases
764
-
765
- <a id="scenario-1-daily-auto-collection-of-inspiration-materials"></a>
766
- ### Scenario 1: Daily Auto Collection of Inspiration Materials
767
-
768
- **Requirement**: Automatically download high-quality landscape and illustration artworks daily as design materials
769
-
770
- **Configuration Example**:
771
-
772
- ```json
773
- {
774
- "targets": [
775
- {
776
- "type": "illustration",
777
- "tag": "éĒĻæ™Ŋ",
778
- "limit": 50,
779
- "minBookmarks": 1000
780
- },
781
- {
782
- "type": "illustration",
783
- "tag": "ã‚ĪãƒĐã‚đト",
784
- "limit": 30,
785
- "minBookmarks": 5000
786
- }
787
- ],
788
- "scheduler": {
789
- "enabled": true,
790
- "cron": "0 2 * * *"
791
- }
792
- }
793
- ```
794
-
795
- > Tip: `tagRelation: "or"` is supported. When set to `or`, the `tag` string is split by spaces, tags are queried sequentially, results are merged and deduplicated by item `id`, then sorted by `sort` and truncated by `limit`. It's recommended to increase `download.requestDelay` (e.g., 1500–3000ms) in this mode to reduce rate limiting.
796
-
797
- **Example 1b: Multi-tag OR search (sequential per-tag queries and merge)**
798
- ```json
799
- {
800
- "targets": [
801
- {
802
- "type": "novel",
803
- "tag": "éĒĻæ™Ŋ ã‚ĪãƒĐã‚đト りナã‚ļナãƒŦ",
804
- "tagRelation": "or",
805
- "limit": 10,
806
- "mode": "search",
807
- "searchTarget": "partial_match_for_tags",
808
- "sort": "popular_desc"
809
- }
810
- ]
811
- }
812
- ```
813
-
814
- **Run Method**:
815
-
816
- ```bash
817
- ./scripts/pixiv.sh run
818
- ```
819
-
820
- ---
821
-
822
- <a id="scenario-2-server-scheduled-collection-of-specific-tags"></a>
823
- ### Scenario 2: Server Scheduled Collection of Specific Tags
824
-
825
- **Requirement**: Weekly collect popular artworks of specific tags on server
826
-
827
- **Configuration Example**:
828
-
829
- ```json
830
- {
831
- "targets": [
832
- {
833
- "type": "illustration",
834
- "tag": "原įĨž",
835
- "limit": 100,
836
- "searchTarget": "partial_match_for_tags"
837
- }
838
- ],
839
- "scheduler": {
840
- "enabled": true,
841
- "cron": "0 0 * * 0",
842
- "timezone": "Asia/Shanghai"
843
- }
844
- }
845
- ```
846
-
847
- **Deployment Method**:
848
-
849
- ```bash
850
- # Use auto deploy script
851
- ./scripts/auto-deploy.sh
852
-
853
- # Or use PM2 to manage process
854
- pm2 start "npm run scheduler" --name pixivflow
855
- ```
856
-
857
- ---
858
-
859
- <a id="scenario-3-quick-experience---random-download"></a>
860
- ### Scenario 3: Quick Experience - Random Download
861
-
862
- **Requirement**: Quick experience tool, download one random artwork
863
-
864
- **Run Method**:
865
-
866
- ```bash
867
- # Random download illustration (default)
868
- npm run random
869
-
870
- # Random download novel
871
- npm run random -- --novel
872
- # or
873
- npm run random -- -n
874
-
875
- # Explicitly specify download illustration
876
- npm run random -- --illustration
877
- # or
878
- npm run random -- -i
879
-
880
- # Or use main program (if pixivflow is globally installed)
881
- pixivflow random
882
- pixivflow random --novel
883
- ```
884
-
885
- **Feature Description**:
886
- - ðŸŽē **Random Tag Selection**: Randomly select from popular tags (illustrations: éĒĻæ™Ŋ, ã‚ĪãƒĐã‚đト, りナã‚ļナãƒŦ, etc.; novels: å°čŠŽ, りナã‚ļナãƒŦ, ホãƒĐマ, etc.)
887
- - 🔍 **Random Artwork Selection**: Randomly select one artwork from search results
888
- - 🔐 **Auto Login**: If not logged in, will automatically guide login
889
- - ðŸ“Ĩ **Quick Experience**: Download 1 random artwork, quickly understand tool features
890
- - 📚 **Type Support**: Supports random download of both illustrations and novels
891
-
892
- ---
893
-
894
- <a id="scenario-4-onetime-batch-download"></a>
895
- ### Scenario 4: One-Time Batch Download
896
-
897
- **Requirement**: One-time download artworks of specified tags
898
-
899
- **Configuration Example**:
900
-
901
- ```json
902
- {
903
- "targets": [
904
- {
905
- "type": "illustration",
906
- "tag": "art",
907
- "limit": 500
908
- }
909
- ]
910
- }
911
- ```
912
-
913
- **Run Method**:
914
-
915
- ```bash
916
- # Use convenient script (recommended)
917
- ./scripts/pixiv.sh once
918
-
919
- # Or use npm command
920
- npm run download
921
-
922
- # Or use main program (if pixivflow is globally installed)
923
- pixivflow download
924
- ```
925
-
926
- > ðŸ’Ą **Tip**: All download tasks are implemented through configuration files, no need to modify source code. See [Configuration File Usage Guide](docs/CONFIG.md) for details.
927
-
928
- ---
929
-
930
- <a id="project-structure"></a>
931
- ## Project Structure
932
-
933
- ```
934
- pixivflow/
935
- ├── 📄 Configuration Files
936
- │ ├── config/
937
- │ │ ├── standalone.config.json # Main config (create yourself)
938
- │ │ └── standalone.config.example.json # Config template
939
- │
940
- ├── ðŸ’ŧ Source Code
941
- │ ├── src/
942
- │ │ ├── index.ts # Main program entry
943
- │ │ ├── setup-wizard.ts # Setup wizard
944
- │ │ ├── config.ts # Config management
945
- │ │ ├── logger.ts # Logging system
946
- │ │ ├── pixiv/ # Pixiv API
947
- │ │ │ ├── AuthClient.ts # Auth client
948
- │ │ │ └── PixivClient.ts # API client
949
- │ │ ├── download/ # Download module
950
- │ │ │ ├── DownloadManager.ts # Download manager
951
- │ │ │ └── FileService.ts # File service
952
- │ │ ├── storage/ # Data persistence
953
- │ │ │ └── Database.ts # SQLite database
954
- │ │ └── scheduler/ # Scheduled tasks
955
- │ │ └── Scheduler.ts # Task scheduler
956
- │
957
- ├── 🛠ïļ Script Tools
958
- │ ├── scripts/
959
- │ │ ├── pixiv.sh # Main control script (recommended)
960
- │ │ ├── easy-setup.sh # Setup wizard (backup)
961
- │ │ └── auto-deploy.sh # Auto deploy
962
- │ │
963
- │ │ ⚠ïļ Note: The following features have been migrated to CLI commands (use after global install):
964
- │ │ - Config management: `pixivflow config`
965
- │ │ - Health check: `pixivflow health`
966
- │ │ - Auto monitor: `pixivflow monitor`
967
- │ │ - Auto maintenance: `pixivflow maintain`
968
- │ │ - Auto backup: `pixivflow backup`
969
- │
970
- ├── ðŸ“Ķ Output Directories (auto created)
971
- │ ├── dist/ # Compiled output
972
- │ ├── downloads/ # Download directory
973
- │ │ ├── illustrations/ # Illustrations
974
- │ │ └── novels/ # Novels
975
- │ └── data/ # Data directory
976
- │ ├── pixiv-downloader.db # SQLite database
977
- │ ├── pixiv-downloader.log # Run logs
978
- │ └── metadata/ # Metadata directory (auto created)
979
- │ └── *.json # Artwork metadata JSON files
980
- │
981
- └── 📚 Documentation
982
- └── docs/ # Documentation directory
983
- ├── README.md # Documentation index
984
- ├── QUICKSTART.md # Quick start guide
985
- ├── LOGIN.md # Login guide
986
- ├── CONFIG.md # Configuration guide
987
- ├── USAGE.md # Usage guide
988
- ├── SCRIPTS.md # Script guide
989
- ├── ARCHITECTURE.md # Architecture documentation
990
- ├── CLI_MIGRATION_SUMMARY.md # CLI migration summary
991
- ├── DOCKER.md # Docker guide
992
- └── project/ # Project documentation
993
- ├── CHANGELOG.md # Changelog
994
- └── CONTRIBUTING.md # Contributing guide
995
- ```
996
-
997
- ---
998
-
999
- ## ⚙ïļ Core Configuration
1000
-
1001
- Configuration file is located at `config/standalone.config.json`. Below are key configuration items:
1002
-
1003
- ### Authentication Configuration
1004
-
1005
- ```json
1006
- {
1007
- "pixiv": {
1008
- "refreshToken": "your_refresh_token_here",
1009
- "clientId": "MOBrBDS8blbauoSck0ZfDbtuzpyT",
1010
- "clientSecret": "lsACyCD94FhDUtGTXi3QzcFE2uU1hqtDaKeqrdwj",
1011
- "userAgent": "PixivAndroidApp/5.0.234 (Android 11; Pixel 6)"
1012
- }
1013
- }
1014
- ```
1015
-
1016
- > ⚠ïļ `refreshToken` is automatically obtained through setup wizard, no need to manually fill
1017
-
1018
- ### Download Targets
1019
-
1020
- ```json
1021
- {
1022
- "targets": [
1023
- {
1024
- "type": "illustration", // Type: illustration or novel
1025
- "tag": "éĒĻæ™Ŋ", // Search tag
1026
- "limit": 20, // Download limit
1027
- "searchTarget": "partial_match_for_tags",
1028
- "minBookmarks": 500, // Minimum bookmarks
1029
- "startDate": "2024-01-01", // Start date (optional)
1030
- "endDate": "2024-12-31" // End date (optional)
1031
- }
1032
- ]
1033
- }
1034
- ```
1035
-
1036
- ### Scheduled Tasks
1037
-
1038
- ```json
1039
- {
1040
- "scheduler": {
1041
- "enabled": true, // Whether enabled
1042
- "cron": "0 3 * * *", // Cron expression
1043
- "timezone": "Asia/Shanghai" // Timezone
1044
- }
1045
- }
1046
- ```
1047
-
1048
- #### Cron Expression Quick Reference
1049
-
1050
- | Expression | Description |
1051
- |------------|-------------|
1052
- | `0 * * * *` | Every hour |
1053
- | `0 */6 * * *` | Every 6 hours |
1054
- | `0 2 * * *` | Daily at 2:00 |
1055
- | `0 0 * * 0` | Weekly on Sunday at 0:00 |
1056
- | `0 0 1 * *` | Monthly on 1st at 0:00 |
1057
-
1058
- ### Storage Configuration
1059
-
1060
- ```json
1061
- {
1062
- "storage": {
1063
- "databasePath": "./data/pixiv-downloader.db",
1064
- "downloadDirectory": "./downloads",
1065
- "illustrationDirectory": "./downloads/illustrations",
1066
- "novelDirectory": "./downloads/novels",
1067
- "illustrationOrganization": "byAuthorAndTag",
1068
- "novelOrganization": "byDateAndAuthor"
1069
- }
1070
- }
1071
- ```
1072
-
1073
- #### Directory Organization Modes
1074
-
1075
- | Mode | Description | Directory Structure Example |
1076
- |------|-------------|----------------------------|
1077
- | `flat` | Flat structure (default) | `illustrations/123456_Title_1.jpg` |
1078
- | `byAuthor` | Organize by author | `illustrations/AuthorName/123456_Title_1.jpg` |
1079
- | `byTag` | Organize by tag | `illustrations/TagName/123456_Title_1.jpg` |
1080
- | `byDate` | Organize by date | `illustrations/2024-12/123456_Title_1.jpg` |
1081
- | `byAuthorAndTag` | By author and tag | `illustrations/AuthorName/TagName/123456_Title_1.jpg` |
1082
- | `byDateAndAuthor` | By date and author | `illustrations/2024-12/AuthorName/123456_Title_1.jpg` |
1083
-
1084
- > ðŸ’Ą **Tip**: Using organization modes can make downloaded files more organized, easier to manage and find.
1085
-
1086
- > ðŸ’Ą **Tip**: Paths support auto-fix, the project will automatically detect and fix path issues on startup. Use `pixivflow migrate-config` to manually migrate configuration paths.
1087
- > 📚 **Complete Configuration Guide**: See [Configuration Guide](docs/CONFIG.md)
1088
-
1089
- ---
1090
-
1091
- ## 🐛 FAQ
1092
-
1093
- ### ❓ Setup Wizard Login Failed?
1094
-
1095
- **Symptoms**: Login fails after running `npm run setup`
1096
-
1097
- **Solutions**:
1098
- 1. Confirm you correctly entered Pixiv username and password in terminal
1099
- 2. Check network connection and proxy settings
1100
- 3. Re-run setup wizard: `npm run setup`
1101
-
1102
- ---
1103
-
1104
- ### ❓ Authentication Failed or Token Expired?
1105
-
1106
- **Symptoms**: "Authentication failed" or "401 Unauthorized" when downloading
1107
-
1108
- **Solutions**:
1109
-
1110
- ```bash
1111
- # Method 1: Use login script (recommended, simplest)
1112
- npm run login
1113
-
1114
- # Method 2: Re-run setup wizard
1115
- ./scripts/easy-setup.sh
1116
- # or
1117
- npm run setup
1118
- ```
1119
-
1120
- ---
1121
-
1122
- ### ❓ No Matching Artworks Found?
1123
-
1124
- **Symptoms**: Search results empty or download count is 0
1125
-
1126
- **Possible Causes**:
1127
- - Tag spelling error or doesn't exist
1128
- - Filter conditions too strict
1129
- - Network connection issues
1130
-
1131
- **Solutions**:
1132
- 1. Try common tags: `ã‚ĪãƒĐã‚đト`, `éĒĻæ™Ŋ`, `art`
1133
- 2. Lower `minBookmarks` value
1134
- 3. Check network connection and firewall settings
1135
- 4. Search on Pixiv website to confirm tag exists
1136
-
1137
- ---
1138
-
1139
- ### ❓ Scheduled Task Not Running?
1140
-
1141
- **Symptoms**: Scheduled task set but not auto downloading
1142
-
1143
- **Solutions**:
1144
-
1145
- ```bash
1146
- # 1. Environment check and auto-fix (recommended ⭐)
1147
- ./scripts/pixiv.sh check --fix
1148
-
1149
- # 2. One-click update and fix
1150
- ./scripts/pixiv.sh update
1151
-
1152
- # 3. Check configuration
1153
- pixivflow config validate
1154
-
1155
- # 4. View running status
1156
- ./scripts/pixiv.sh status
1157
-
1158
- # 5. Check logs
1159
- ./scripts/pixiv.sh logs
1160
-
1161
- # 6. Ensure program keeps running
1162
- # Use PM2 to manage process
1163
- pm2 start "npm run scheduler" --name pixivflow
1164
- pm2 save
1165
- pm2 startup
1166
- ```
1167
-
1168
- ---
1169
-
1170
- ### ❓ Slow Download Speed or Frequent Failures?
1171
-
1172
- **Possible Causes**: Unstable network connection or Pixiv server rate limiting
1173
-
1174
- **Solutions**:
1175
- 1. Check network connection
1176
- 2. Reduce concurrent download count
1177
- 3. Increase retry count and timeout
1178
- 4. Use proxy server (if needed)
1179
-
1180
- ---
1181
-
1182
- ### ❓ Encountered Deleted or Private Artworks?
1183
-
1184
- **Symptoms**: Some artworks cannot be downloaded during download process
1185
-
1186
- **Description**:
1187
- PixivFlow has built-in comprehensive error handling that automatically handles the following:
1188
-
1189
- - ✅ **Auto Skip Deleted Artworks**: If artwork is deleted by author, will auto skip and continue downloading others
1190
- - ✅ **Auto Skip Private Artworks**: If artwork is set to private or requires special permissions, will auto skip
1191
- - ✅ **Auto Skip Inaccessible Artworks**: If artwork cannot be accessed for other reasons (e.g., 404 error), will auto skip
1192
- - ✅ **Record Skip Count**: Will display skipped artwork count after download completes
1193
- - ✅ **Won't Interrupt Flow**: Single artwork download failure won't affect entire download task
1194
-
1195
- **Log Example**:
1196
-
1197
- ```
1198
- [INFO] Skipped 3 novel(s) (deleted, private, or inaccessible)
1199
- [INFO] Illustration tag éĒĻæ™Ŋ completed, { downloaded: 47 }
1200
- ```
1201
-
1202
- **Description**:
1203
- - 404 errors use `debug` level logs (silently skipped)
1204
- - Other errors use `warn` level logs (recorded but continue)
1205
- - All skipped artwork counts will be displayed at task end
1206
-
1207
- ---
1208
-
1209
- ### 🔍 View Detailed Logs
1210
-
1211
- ```bash
1212
- # View run logs
1213
- ./scripts/pixiv.sh logs
1214
-
1215
- # Or directly view log file
1216
- tail -f data/pixiv-downloader.log
1217
- ```
1218
-
1219
- ---
1220
-
1221
- ## 🔒 Security Tips
1222
-
1223
- > ⚠ïļ **Important**: Configuration files contain sensitive information, please pay attention to security
1224
-
1225
- ### ðŸ›Ąïļ Security Recommendations
1226
-
1227
- | Recommendation | Description |
1228
- |----------------|-------------|
1229
- | ✅ **Don't Share Config Files** | `config/standalone.config.json` contains sensitive authentication info |
1230
- | ✅ **Don't Commit to Git** | Ensure config file is in `.gitignore` (excluded by default) |
1231
- | ✅ **Regular Backups** | Use `pixivflow backup` to backup config and data |
1232
- | ✅ **Use Strong Passwords** | Protect your Pixiv account |
1233
- | ✅ **HTTPS Encryption** | All API requests use HTTPS secure transmission |
1234
- | ✅ **Regular Token Updates** | Regularly re-run setup wizard to update authentication info |
1235
-
1236
- ### 🔐 About refresh_token
1237
-
1238
- `refresh_token` is equivalent to your account password, having it allows access to your Pixiv account.
1239
-
1240
- **If refresh_token is leaked**:
1241
- 1. Immediately revoke authorization in Pixiv account settings
1242
- 2. Change Pixiv account password
1243
- 3. Re-run setup wizard to get new token
1244
-
1245
- ---
1246
-
1247
- ## 📊 Download Record Management
1248
-
1249
- All download records are saved in SQLite database (`data/pixiv-downloader.db`), including:
1250
-
1251
- - Artwork ID, title, author info
1252
- - Download time, file path
1253
- - Artwork statistics (views, bookmarks, etc.)
1254
-
1255
- ### View Download Records
1256
-
1257
- ```bash
1258
- # Use SQLite command line tool
1259
- sqlite3 data/pixiv-downloader.db "SELECT * FROM downloaded_artworks LIMIT 10;"
1260
-
1261
- # Or use GUI tools
1262
- # - DB Browser for SQLite
1263
- # - SQLiteStudio
1264
- ```
1265
-
1266
- ### Clean Download Records
1267
-
1268
- ```bash
1269
- # Use maintenance command (recommended)
1270
- pixivflow maintain
1271
-
1272
- # Or manually delete database (will re-download all artworks)
1273
- rm data/pixiv-downloader.db
1274
- ```
1275
-
1276
- ---
1277
-
1278
- ## 🚀 Advanced Usage
1279
-
1280
- ### Deploy on Server
1281
-
1282
- #### Method 1: Use Docker (Recommended ⭐)
1283
-
1284
- ```bash
1285
- # Docker mode deployment
1286
- ./scripts/auto-deploy.sh production docker
1287
-
1288
- # Or use Docker management script
1289
- ./scripts/pixiv.sh docker deploy
1290
- ```
1291
-
1292
- #### Method 2: Use Auto Deploy Script (Native Mode)
1293
-
1294
- ```bash
1295
- ./scripts/auto-deploy.sh
1296
- ```
1297
-
1298
- #### Method 3: Use PM2 Management
1299
-
1300
- ```bash
1301
- # Install PM2
1302
- npm install -g pm2
1303
-
1304
- # Start scheduled task
1305
- pm2 start "npm run scheduler" --name pixivflow
1306
-
1307
- # Save PM2 config
1308
- pm2 save
1309
-
1310
- # Set auto start on boot
1311
- pm2 startup
1312
- ```
1313
-
1314
- #### Method 4: Use systemd
1315
-
1316
- Create service file `/etc/systemd/system/pixivflow.service`:
1317
-
1318
- ```ini
1319
- [Unit]
1320
- Description=PixivFlow Automation Downloader
1321
- After=network.target
1322
-
1323
- [Service]
1324
- Type=simple
1325
- User=your-username
1326
- WorkingDirectory=/path/to/pixivflow
1327
- ExecStart=/usr/bin/node dist/index.js scheduler
1328
- Restart=on-failure
1329
- RestartSec=10
1330
-
1331
- [Install]
1332
- WantedBy=multi-user.target
1333
- ```
1334
-
1335
- Start service:
1336
-
1337
- ```bash
1338
- sudo systemctl enable pixivflow
1339
- sudo systemctl start pixivflow
1340
- sudo systemctl status pixivflow
1341
- ```
1342
-
1343
- ---
1344
-
1345
- ### Configure Multiple Download Tasks
1346
-
1347
- You can add multiple targets in the `targets` array:
1348
-
1349
- ```json
1350
- {
1351
- "targets": [
1352
- {
1353
- "type": "illustration",
1354
- "tag": "éĒĻæ™Ŋ",
1355
- "limit": 50
1356
- },
1357
- {
1358
- "type": "illustration",
1359
- "tag": "ã‚ĪãƒĐã‚đト",
1360
- "limit": 30,
1361
- "minBookmarks": 1000
1362
- },
1363
- {
1364
- "type": "novel",
1365
- "tag": "å°čŠŽ",
1366
- "limit": 10
1367
- }
1368
- ]
1369
- }
1370
- ```
1371
-
1372
- ---
1373
-
1374
- ### Using Proxy
1375
-
1376
- If you need to access Pixiv through proxy, there are two methods:
1377
-
1378
- #### Method 1: Use Environment Variables (Recommended ⭐)
1379
-
1380
- Program automatically reads proxy config from environment variables, no need to modify config file:
1381
-
1382
- ```bash
1383
- # Set proxy environment variables (priority: all_proxy > https_proxy > http_proxy)
1384
- export all_proxy=socks5://127.0.0.1:6153
1385
- # or
1386
- export https_proxy=http://127.0.0.1:6152
1387
- # or
1388
- export http_proxy=http://127.0.0.1:6152
1389
-
1390
- # Then run program
1391
- npm run download
1392
- ```
1393
-
1394
- **Supported Proxy Protocols**:
1395
- - `http://` - HTTP proxy
1396
- - `https://` - HTTPS proxy
1397
- - `socks5://` - SOCKS5 proxy
1398
- - `socks4://` - SOCKS4 proxy
1399
-
1400
- **Environment Variable Priority**:
1401
- 1. `all_proxy` or `ALL_PROXY` (highest priority)
1402
- 2. `https_proxy` or `HTTPS_PROXY`
1403
- 3. `http_proxy` or `HTTP_PROXY`
1404
-
1405
- #### Method 2: Config File Settings
1406
-
1407
- ```json
1408
- {
1409
- "network": {
1410
- "proxy": {
1411
- "enabled": true,
1412
- "host": "127.0.0.1",
1413
- "port": 7890,
1414
- "protocol": "http"
1415
- }
1416
- }
1417
- }
1418
- ```
1419
-
1420
- **Note**: If proxy is enabled in config file, environment variables won't override config file settings.
1421
-
1422
- ---
1423
-
1424
- ## 📄 Open Source License
1425
-
1426
- This project is open source under [GPL-3.0-or-later](LICENSE) license.
1427
-
1428
- **This means**:
1429
- - ✅ Free to use, modify and distribute
1430
- - ✅ Modified code must also be open source
1431
- - ✅ Must retain original author info and license notice
1432
-
1433
- ---
1434
-
1435
- ## 🙏 Acknowledgments
1436
-
1437
- ### Inspiration Sources
1438
-
1439
- - [PixivBatchDownloader](https://github.com/xuejianxianzun/PixivBatchDownloader) - Browser extension version
1440
- - [get-pixivpy-token](https://github.com/eggplants/get-pixivpy-token) - OAuth authentication implementation reference
1441
-
1442
- ### Thanks to All Contributors 🎉
1443
-
1444
- ---
1445
-
1446
- ## ðŸ“Ū Get Help
1447
-
1448
- Having issues? Here are multiple ways to get help:
1449
-
1450
- | Type | Channel | Description |
1451
- |------|---------|-------------|
1452
- | 🐛 **Bug Report** | [GitHub Issues](https://github.com/zoidberg-xgd/pixivflow/issues) | Report issues and bugs |
1453
- | ðŸ’Ą **Feature Suggestion** | [GitHub Discussions](https://github.com/zoidberg-xgd/pixivflow/discussions) | Propose new feature ideas |
1454
- | 📖 **Usage Questions** | [View Documentation](docs/README.md) | Read complete documentation |
1455
- | 🔧 **Auto-Fix Issues** | `./scripts/pixiv.sh check --fix` | Auto-fix environment issues ⭐ New |
1456
- | 🔄 **Update & Fix** | `./scripts/pixiv.sh update` | One-click update and fix ⭐ New |
1457
- | ✅ **Environment Check** | `./scripts/pixiv.sh health` | Run health check |
1458
- | 💎 **Community Discussion** | [Discussions](https://github.com/zoidberg-xgd/pixivflow/discussions) | Discuss with other users |
1459
- | 🔍 **FAQ** | [FAQ](#-faq) | View FAQ |
1460
-
1461
- ### Search Keywords
1462
-
1463
- If you're looking for similar tools, these keywords might help:
1464
-
1465
- - `pixiv downloader` - Pixiv downloader
1466
- - `pixiv batch download` - Pixiv batch download
1467
- - `pixiv automation` - Pixiv automation
1468
- - `pixiv cli` - Pixiv CLI tool
1469
- - `pixiv api` - Pixiv API client
1470
- - `pixiv scheduler` - Pixiv scheduled tasks
1471
- - `pixiv artwork downloader` - Pixiv artwork downloader
1472
- - `pixiv novel downloader` - Pixiv novel downloader
1473
-
1474
- ### Before Asking:
1475
-
1476
- 1. 🔍 Check [FAQ](#-faq) section
1477
- 2. 📖 Read relevant documentation
1478
- 3. ✅ Run health check `pixivflow health`
1479
- 4. 📋 View run logs `./scripts/pixiv.sh logs`
1480
-
1481
- ---
1482
-
1483
- ## 📈 Project Statistics
1484
-
1485
- <div align="center">
1486
-
1487
- ### Project Data
1488
-
1489
- ![GitHub repo size](https://img.shields.io/github/repo-size/zoidberg-xgd/pixivflow?style=flat-square)
1490
- ![GitHub language count](https://img.shields.io/github/languages/count/zoidberg-xgd/pixivflow?style=flat-square)
1491
- ![GitHub top language](https://img.shields.io/github/languages/top/zoidberg-xgd/pixivflow?style=flat-square)
1492
- ![GitHub last commit](https://img.shields.io/github/last-commit/zoidberg-xgd/pixivflow?style=flat-square)
1493
-
1494
- </div>
1495
-
1496
- ### Performance Metrics
1497
-
1498
- - ⚡ **Startup Speed**: < 2 seconds
1499
- - ðŸ“Ķ **Package Size**: < 5 MB (excluding dependencies)
1500
- - ðŸ’ū **Memory Usage**: < 100 MB (runtime)
1501
- - 🔄 **Download Speed**: Supports concurrent downloads, auto rate limiting
1502
- - 📊 **Database**: SQLite, lightweight, no additional service needed
1503
-
1504
- ---
1505
-
1506
- ## ðŸĪ Contributing
1507
-
1508
- We welcome all forms of contributions! Whether it's reporting bugs, proposing features, or submitting code, we're very grateful.
1509
-
1510
- ### How to Contribute
1511
-
1512
- 1. **Fork the Project**
1513
- 2. **Create Feature Branch** (`git checkout -b feature/AmazingFeature`)
1514
- 3. **Commit Changes** (`git commit -m 'Add some AmazingFeature'`)
1515
- 4. **Push to Branch** (`git push origin feature/AmazingFeature`)
1516
- 5. **Open Pull Request**
1517
-
1518
- ### Contributing Guide
1519
-
1520
- See [CONTRIBUTING.md](docs/project/CONTRIBUTING.md) for detailed contributing guide, including:
1521
- - Code of Conduct
1522
- - Development environment setup
1523
- - Code standards
1524
- - Commit standards
1525
- - Pull Request process
1526
-
1527
- ---
1528
-
1529
- ## 📝 Changelog
1530
-
1531
- See [CHANGELOG.md](docs/project/CHANGELOG.md) for detailed version update records.
1532
-
1533
- ---
1534
-
1535
- ## Support the Project
1536
-
1537
- If this project helps you, please consider:
1538
-
1539
- - ⭐ **Give the Project a Star** - Let more people discover this project
1540
- - ðŸī **Fork the Project** - Create your own version
1541
- - 🐛 **Report Bugs** - Help us improve
1542
- - ðŸ’Ą **Propose Suggestions** - Share your ideas
1543
- - ðŸ“Ē **Share with More People** - Let more people benefit
1544
- - ðŸ’ŧ **Contribute Code** - Participate in project development
1545
-
1546
- <div align="center">
1547
-
1548
- ### ⭐ Star This Project
1549
-
1550
- **[⭐ Star on GitHub](https://github.com/zoidberg-xgd/pixivflow)** - Let more people discover PixivFlow!
1551
-
1552
- ---
1553
-
1554
- **Related Links**:
1555
- - [npm Package](https://www.npmjs.com/package/pixivflow)
1556
- - [GitHub Repository](https://github.com/zoidberg-xgd/PixivFlow)
1557
- - [Frontend Project (pixivflow-webui)](https://github.com/zoidberg-xgd/pixivflow-webui)
1558
- - [Full Documentation](docs/README.md)
1559
- - [Issue Tracker](https://github.com/zoidberg-xgd/PixivFlow/issues)
1560
- - [Discussions](https://github.com/zoidberg-xgd/PixivFlow/discussions)
1561
-
1562
- ---
97
+ | --- | --- |
98
+ | [QUICKSTART](docs/QUICKSTART.md) | Get running in three minutes |
99
+ | [CONFIG](docs/CONFIG.md) | All configuration options |
100
+ | [USAGE](docs/USAGE.md) | Feature reference |
101
+ | [LOGIN](docs/LOGIN.md) | Account sign-in details |
102
+ | [DOCKER](docs/DOCKER.md) | Container deployment |
103
+ | [ARCHITECTURE](docs/ARCHITECTURE.md) | Architecture notes |
104
+ | [RELEASING](docs/RELEASING.md) | npm release workflow |
105
+ | [CHANGELOG](docs/project/CHANGELOG.md) | Version history |
1563
106
 
1564
- Made with âĪïļ by [zoidberg-xgd](https://github.com/zoidberg-xgd)
107
+ Chinese version: [README.md](README.md).
1565
108
 
1566
- **PixivFlow** - Make Pixiv artwork collection elegant and efficient
109
+ ## Feedback
1567
110
 
1568
- [⮆ Back to Top](#-pixivflow)
111
+ Bugs and feature requests go to
112
+ [Issues](https://github.com/redtidev1918/PixivFlow/issues); please run
113
+ `pixivflow health` first and include its output (strip tokens and other
114
+ secrets before sharing). Security issues are handled privately — see
115
+ [SECURITY.md](SECURITY.md).
1569
116
 
1570
- </div>
117
+ ## License
1571
118
 
119
+ [MIT](LICENSE)