pixivflow 2.20.4 → 2.21.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (33) hide show
  1. package/README.en.md +180 -53
  2. package/README.md +113 -59
  3. package/dist/commands/SchedulerCommand.js +66 -0
  4. package/dist/commands/SchedulerIdleLifecycle.d.ts +17 -0
  5. package/dist/commands/SchedulerIdleLifecycle.js +3 -1
  6. package/dist/commands/scheduler-runtime.d.ts +8 -0
  7. package/dist/commands/scheduler-runtime.js +121 -94
  8. package/dist/config/types.d.ts +26 -0
  9. package/dist/config/validation.js +21 -0
  10. package/dist/delivery/types.d.ts +15 -0
  11. package/dist/notification/NotificationPolicy.d.ts +2 -0
  12. package/dist/notification/NotificationPolicy.js +18 -1
  13. package/dist/package.json +1 -1
  14. package/dist/scheduler/MultiScheduleManager.d.ts +12 -1
  15. package/dist/scheduler/MultiScheduleManager.js +46 -50
  16. package/dist/scheduler/RecoveryPolicy.d.ts +45 -0
  17. package/dist/scheduler/RecoveryPolicy.js +63 -0
  18. package/dist/scheduler/ResourceAdmission.d.ts +68 -0
  19. package/dist/scheduler/ResourceAdmission.js +83 -0
  20. package/dist/scheduler/ScheduleTriggerServer.d.ts +16 -0
  21. package/dist/scheduler/ScheduleTriggerServer.js +55 -0
  22. package/dist/scheduler/Scheduler.d.ts +19 -2
  23. package/dist/scheduler/Scheduler.js +12 -3
  24. package/dist/scheduler/SlotCoordinator.d.ts +20 -0
  25. package/dist/scheduler/SlotCoordinator.js +44 -2
  26. package/dist/scheduler/TargetOutcome.d.ts +31 -0
  27. package/dist/scheduler/TargetOutcome.js +114 -0
  28. package/dist/storage/DatabaseMigration.js +17 -0
  29. package/dist/storage/repositories/SlotRepository.d.ts +29 -0
  30. package/dist/storage/repositories/SlotRepository.js +28 -2
  31. package/dist/version.js +1 -1
  32. package/dist/webui/package.json +1 -1
  33. package/package.json +2 -2
package/README.en.md CHANGED
@@ -2,39 +2,51 @@
2
2
 
3
3
  **Language / 语言:** [中文](README.md) · English
4
4
 
5
- Pixiv batch download and scheduled collection tool. Bulk-download
6
- illustrations and novels, search by tag, filter by bookmarks and dates,
7
- and run recurring collection jobs on a schedule. CLI and WebUI included.
8
- Built with TypeScript and Node.js; runs on Windows, macOS, Linux and Docker.
5
+ **Pixiv downloader, filter and automatic collection tool.**
6
+
7
+ Download a single Pixiv artwork (illustration, novel, ugoira) directly, or batch-collect
8
+ by tag, ranking, publish date and bookmark count, and let the scheduler keep collecting
9
+ on a cron. Results stay on your disk, or get delivered reliably over HTTP to another
10
+ service — downstreams are optional, PixivFlow alone covers
11
+ discover → filter → download → save.
9
12
 
10
13
  [![Version](https://img.shields.io/npm/v/pixivflow?style=flat-square)](https://www.npmjs.com/package/pixivflow)
11
14
  [![Node](https://img.shields.io/badge/Node.js-22.13%2B_LTS-green.svg?style=flat-square&logo=node.js)](https://nodejs.org/)
12
15
  [![License](https://img.shields.io/badge/License-MIT-blue.svg?style=flat-square)](LICENSE)
13
16
  [![Documentation](https://img.shields.io/badge/Docs-redtidev1918.github.io-6366f1?style=flat-square)](https://redtidev1918.github.io/PixivFlow/)
14
17
 
15
- ## Install
18
+ ## Typical scenarios
16
19
 
17
- Requires Node.js 22.13 or later; use a currently supported LTS release in production.
20
+ **1. Download one link.** Paste any Pixiv link artwork, novel, series, or user
21
+ profile are all recognized:
18
22
 
19
23
  ```bash
20
- npm install -g pixivflow
21
- pixivflow --help
24
+ pixivflow download --url https://www.pixiv.net/artworks/123456789
22
25
  ```
23
26
 
24
- For servers, prefer the Docker Compose setup described in
25
- [DOCKER.md](docs/DOCKER.md). To build from source:
27
+ **2. Batch download by conditions.** Define what to collect in your config (tag,
28
+ ranking, date range, minimum bookmarks) and run it in one pass. Downloaded items are
29
+ tracked in SQLite and skipped, so re-running never re-downloads. See
30
+ [Filtering and targets](#filtering-and-targets).
26
31
 
27
- ```bash
28
- git clone https://github.com/redtidev1918/PixivFlow.git
29
- cd PixivFlow
30
- npm install
31
- npm run build
32
- ```
32
+ **3. Collect on a schedule and deliver.** Run it under cron: discover, download, and
33
+ optionally hand files to another service — the local copy is deleted only after the
34
+ receiver confirms.
33
35
 
34
- Termux / Android: see [TERMUX_INSTALL.md](docs/TERMUX_INSTALL.md).
36
+ ```text
37
+ Pixiv ──► PixivFlow ──┬──► local, permanent (persistent)
38
+ └──► HTTP delivery (cache) ──► TelePost / any compatible service
39
+ ```
35
40
 
36
41
  ## Quick start
37
42
 
43
+ Requires Node.js 22.13 or later; use a currently supported LTS release in production.
44
+
45
+ ```bash
46
+ npm install -g pixivflow
47
+ pixivflow --help
48
+ ```
49
+
38
50
  Sign in to your Pixiv account (creates OAuth credentials, one time):
39
51
 
40
52
  ```bash
@@ -56,22 +68,27 @@ pixivflow download
56
68
  pixivflow scheduler # long-running cron collection
57
69
  ```
58
70
 
59
- ### Multiple schedules and atomic hot reload
71
+ Prefer not to hand-write config? The interactive wizard `pixivflow setup` generates it.
72
+ For a GUI, run `pixivflow webui` (frontend:
73
+ [pixivflow-webui](https://github.com/redtidev1918/pixivflow-webui)).
60
74
 
61
- `schedules[]` hosts independently timed target groups in one Node process.
62
- Plans share Pixiv authentication, SQLite, and file services, while a bounded
63
- serial queue prevents overlapping downloads from producing memory spikes.
64
- The active config is watched by default: replace it over SSH and PixivFlow
65
- fully validates the new snapshot before swapping the entire cron table. An
66
- invalid edit leaves the previous schedules running. In-flight work finishes
67
- on its old snapshot; the next run sees the new one.
75
+ Downloading Pixiv ugoira additionally needs `python3` and `ffmpeg`: frames are
76
+ composited into a looping GIF by per-frame delay, ready to be delivered as an animation.
77
+ The official Docker image includes both see
78
+ [configuration notes](docs/CONFIG.md#pixiv-动图ugoira).
68
79
 
69
- Legacy single-`scheduler` configs remain supported. `schedules`, `targets`,
70
- `delivery`, and `download` are hot-reloadable; changes to `pixiv`, `network`,
71
- or `storage` require a process restart. See the ready-to-edit
72
- [`config/fly-two-bots.example.json`](config/fly-two-bots.example.json) template.
80
+ Build from source:
81
+
82
+ ```bash
83
+ git clone https://github.com/redtidev1918/PixivFlow.git
84
+ cd PixivFlow
85
+ npm install
86
+ npm run build
87
+ ```
73
88
 
74
- ## Download targets
89
+ Termux / Android: see [TERMUX_INSTALL.md](docs/TERMUX_INSTALL.md).
90
+
91
+ ## Filtering and targets
75
92
 
76
93
  Define what to collect in the `targets` section of your config. Conditions
77
94
  combine:
@@ -86,30 +103,44 @@ combine:
86
103
 
87
104
  Downloaded items are tracked in a SQLite database and skipped automatically;
88
105
  files that exist without a database record are reconciled, so the two never
89
- conflict.
106
+ conflict. Illustration tasks with `mode: "topic"` keep a bounded hotness candidate
107
+ pool: if the top item is already downloaded, the next undownloaded one is promoted by
108
+ hotness instead of the run coming back empty.
109
+
110
+ ## Persistent and cache delivery modes
90
111
 
91
- ### Persistent and cache delivery modes
112
+ Each target (one tag / schedule) can use one of two storage modes:
92
113
 
93
- Each target can use `storageMode: "persistent"` (the default, keep files) or
94
- `storageMode: "cache"` (send files to a named delivery target and delete them
95
- only after success). The delivery layer is service-independent; this example
96
- merely translates an HTTP multipart submission API into configuration:
114
+ - **`persistent`** (the default): keep files forever.
115
+ - **`cache`**: send files to a named delivery target and delete the local copy only
116
+ after the receiver confirms saves disk.
117
+
118
+ A "delivery target" is just configuration: which URL to POST to and which fields to
119
+ send. It is not bound to any service and can point at any compatible HTTP endpoint;
120
+ [TelePost](https://github.com/redtidev1918/TelePost) and
121
+ [telepress](https://github.com/redtidev1918/telepress) are example downstreams. Example:
97
122
 
98
123
  ```json
99
124
  {
100
125
  "delivery": {
126
+ "outboxRetryBaseMs": 300000,
127
+ "outboxRetryMaxMs": 21600000,
101
128
  "targets": {
102
129
  "sharing-api": {
103
130
  "type": "httpMultipart",
104
- "url": "https://example.test/submissions",
105
- "readinessUrl": "https://example.test/ready",
106
- "notificationUrl": "https://example.test/notifications",
131
+ "url": "https://your-domain.example/api/bot1/v1/submissions",
132
+ "readinessUrl": "https://your-domain.example/ready",
133
+ "notificationUrl": "https://your-domain.example/api/bot1/v1/notifications",
107
134
  "headers": { "Authorization": "Bearer ${SHARING_TOKEN}" },
108
135
  "fileField": "files",
109
136
  "fields": { "title": "{{title}}" },
110
- "success": { "statuses": [201], "jsonPath": "ok", "equals": true }
137
+ "success": { "statuses": [201], "jsonPath": "ok", "equals": true },
138
+ "arrayFormat": "comma",
139
+ "maxAttempts": 3,
140
+ "retryDelayMs": 2000
111
141
  }
112
- }
142
+ },
143
+ "deleteAfterDelivery": true
113
144
  },
114
145
  "targets": [
115
146
  { "type": "illustration", "tag": "archive", "storageMode": "persistent" },
@@ -126,20 +157,89 @@ merely translates an HTTP multipart submission API into configuration:
126
157
  }
127
158
  ```
128
159
 
129
- Headers and URLs accept arbitrary `${ENV_NAME}` interpolation. Cache-mode
130
- illustration delivery also sends an optional, one-to-one Pixiv preview in the
131
- `previews` multipart field while retaining the original as the authoritative
132
- artifact. Failed delivery keeps both files in the SQLite outbox; the worker
133
- retries the same intent after restart. Final no-match
134
- notifications use the same durable outbox, so a restart or a temporary review
135
- endpoint outage does not silently lose the alert.
160
+ Headers and URLs accept `${ENV_NAME}` interpolation (never hard-code tokens). Cache-mode
161
+ illustration delivery also sends an optional, one-to-one Pixiv preview in the `previews`
162
+ multipart field while retaining the original as the authoritative artifact; a generic
163
+ receiver may ignore that field.
164
+
165
+ Operational notifications can point `notificationUrl` at an
166
+ [Apprise API](docs/APPRISE.md) instance, which fans out to Email, Telegram, Discord,
167
+ ntfy and more; PixivFlow does not implement those notification protocols itself.
168
+
169
+ - The `url` above is any compatible HTTP submission endpoint; the example uses
170
+ TelePost's `/api/botN/v1/submissions` (put the `tp_...` from `/gen_token` into
171
+ `SHARING_TOKEN` — that is the example service's own auth scheme).
172
+ - The same target can point at [telepress](https://github.com/redtidev1918/telepress)'s
173
+ `/publish/gallery` to publish illustrations as a Telegra.ph gallery, see the
174
+ "Telegraph (telegra.ph) gallery upload" section of [CONFIG.md](docs/CONFIG.md).
175
+
176
+ ## Automation and reliability
177
+
178
+ ### Multiple schedules and atomic hot reload
179
+
180
+ `schedules[]` hosts independently timed target groups in one Node process.
181
+ Plans share Pixiv authentication, SQLite, and file services, while a bounded
182
+ serial queue prevents overlapping downloads from producing memory spikes. This runs
183
+ comfortably on small machines — measured with `topic` discovery, selection and
184
+ download inside a 256 MB cgroup: peak RSS ≈ 106 MB, heapUsed ≈ 33 MB, no OOM
185
+ (see [DOCKER.md](docs/DOCKER.md)).
186
+ The active config is watched by default: replace it over SSH and PixivFlow
187
+ fully validates the new snapshot before swapping the entire cron table. An
188
+ invalid edit leaves the previous schedules running. In-flight work finishes
189
+ on its old snapshot; the next run sees the new one.
190
+
191
+ ```json
192
+ {
193
+ "scheduler": { "enabled": false, "cron": "0 3 * * *" },
194
+ "schedules": [
195
+ { "id": "bot1", "enabled": true, "cron": "10 5 * * *", "targetIds": ["bot1-art", "bot1-novel"] },
196
+ { "id": "bot2", "enabled": true, "cron": "30 5 * * *", "targetIds": ["bot2-art", "bot2-novel"] }
197
+ ],
198
+ "targets": [
199
+ { "id": "bot1-art", "type": "illustration", "mode": "ranking", "rankingDate": "YESTERDAY" },
200
+ { "id": "bot1-novel", "type": "novel", "mode": "ranking", "rankingDate": "YESTERDAY" }
201
+ ]
202
+ }
203
+ ```
204
+
205
+ Legacy single-`scheduler` configs remain supported. `schedules`, `targets`,
206
+ `delivery`, and `download` are hot-reloadable; changes to `pixiv`, `network`,
207
+ or `storage` require a process restart. See the ready-to-edit
208
+ [`config/fly-two-bots.example.json`](config/fly-two-bots.example.json) template.
136
209
 
137
- When `readinessUrl` is configured, a non-2xx readiness response defers the row
138
- without incrementing its attempt count. Use `pixivflow outbox list`,
139
- `inspect <id>`, `retry <id>` / `retry --dead`, and `cancel <id>` for recovery;
140
- `run-once` is not an outbox replay command.
210
+ ### Durable delivery outbox
211
+
212
+ Delivery is transactional (a SQLite outbox): at-least-once execution with
213
+ effectively-once visible effects. Every external side effect (one content delivery,
214
+ one notification) becomes a row with an idempotency key and a row-level lease; an
215
+ independent worker pumps it right away and retries with exponential backoff (5 minutes
216
+ up to 6 hours by default), and rows that exceed `maxAttempts` go dead and are recorded
217
+ as `failed` in the delivery ledger. After a crash or machine hang, expired `processing`
218
+ leases are taken over by the next process, which retries the same idempotent intent — a
219
+ downstream converges via `idempotent_replay` (same key, lost ACK) or
220
+ `duplicate_existing` (historical duplicate), so the channel still shows exactly one
221
+ message. Legacy file-based `delivery-outbox/*.json` manifests are migrated into SQLite
222
+ once at startup, idempotently. "Nothing to publish today" notifications share that table
223
+ but are pumped independently, so a temporarily unavailable review endpoint cannot block
224
+ content delivery.
225
+
226
+ When `readinessUrl` is configured, a non-2xx readiness response returns the row to
227
+ pending without incrementing its attempt count — strictly separate from `/live`'s
228
+ "process is alive" meaning. Dead letters are recovered through first-class CLI:
141
229
 
142
- Interactive configuration wizard: `pixivflow setup`.
230
+ ```bash
231
+ pixivflow outbox list --status dead
232
+ pixivflow outbox inspect <id>
233
+ pixivflow outbox retry <id> # only accepts dead rows, keeps the idempotency key
234
+ pixivflow outbox retry --dead
235
+ pixivflow outbox cancel <id> # cancels rows that have not run yet
236
+ ```
237
+
238
+ `run-once` re-runs the download plan; it is not an outbox replay, and you should not
239
+ hand-edit SQLite's `next_attempt_at`. For long-running hosts, use `pixivflow doctor`
240
+ (stuck slot/outbox leases, pending deliveries, dead rows; `--repair` converges) and
241
+ `pixivflow reconcile` (record downstream-confirmed historical duplicates in the delivery
242
+ ledger; dry-run by default).
143
243
 
144
244
  ## Common commands
145
245
 
@@ -149,16 +249,31 @@ Interactive configuration wizard: `pixivflow setup`.
149
249
  | `pixivflow download --url <url>` | Direct download via URL |
150
250
  | `pixivflow random` | Random popular artwork |
151
251
  | `pixivflow scheduler` | Start scheduled jobs |
252
+ | `pixivflow webui` | Start the WebUI |
152
253
  | `pixivflow config` | Manage config (view / edit / backup / restore) |
153
254
  | `pixivflow status` | Download stats and recent records |
154
255
  | `pixivflow health` | Health check: config, directories, connectivity |
256
+ | `pixivflow doctor` | Reliability check: stuck slot/outbox leases, pending deliveries, dead rows; `--repair` converges |
257
+ | `pixivflow reconcile` | Record downstream-confirmed historical duplicates (dry-run by default, `--repair` writes) |
258
+ | `pixivflow outbox` | List, inspect, replay dead letters, or cancel durable intents that have not run |
155
259
  | `pixivflow tags discover <seed>` | Discover related tags (Pixiv autocomplete + tag co-occurrence); lists candidates only |
156
260
  | `pixivflow tags apply <manifest> --target <id> --select <tag1,tag2>` | Atomically write chosen tags into config after manual confirmation, then hot-reload |
261
+ | `pixivflow topic resolve <topic>` | Inspect the derived tag space for a topic (`--type illustration\|novel`, `--refresh`) |
262
+ | `pixivflow topic test <topic> --date YESTERDAY` | Dry-run the candidates and Top N for one day, without downloading |
157
263
 
158
264
  `tags discover` calls the Pixiv autocomplete endpoint and samples recent illustrations/novels to count co-occurring tags, caching results for 7 days; it **never** changes active plans. After reviewing candidates, run `tags apply` to explicitly select tags: it validates the whole config, writes a backup and atomically replaces the file so a running scheduler hot-reloads it.
159
265
 
160
266
  More commands in [USAGE.md](docs/USAGE.md); see the [migration guide](docs/MIGRATION.md) for upgrading from v1 to v2.
161
267
 
268
+ ## Deployment
269
+
270
+ - **Docker / long-running server**: see [DOCKER.md](docs/DOCKER.md).
271
+ - **Android / Termux**: see [TERMUX_INSTALL.md](docs/TERMUX_INSTALL.md).
272
+ - **Composing with TelePost**: PixivFlow and TelePost are both usable on their own.
273
+ Only if you want to deploy the two together as one workflow do you need the
274
+ [pixivflow-telepost-deploy](https://github.com/redtidev1918/pixivflow-telepost-deploy)
275
+ deployment and operations kit.
276
+
162
277
  ## Documentation
163
278
 
164
279
  Full tutorial site: <https://redtidev1918.github.io/PixivFlow/>
@@ -179,6 +294,18 @@ Full tutorial site: <https://redtidev1918.github.io/PixivFlow/>
179
294
 
180
295
  Chinese version: [README.md](README.md).
181
296
 
297
+ ## Related projects
298
+
299
+ PixivFlow is fully standalone. These are the related projects in the same author's
300
+ ecosystem, and what each one owns:
301
+
302
+ | Project | What it is | When you need it |
303
+ | --- | --- | --- |
304
+ | [TelePost](https://github.com/redtidev1918/TelePost) | Telegram channel submission, moderation and automated publishing platform | When you want downloads to land in a Telegram channel for human review before publishing — configure it as a delivery downstream. This is an optional composition; PixivFlow does not depend on it |
305
+ | [pixivflow-telepost-deploy](https://github.com/redtidev1918/pixivflow-telepost-deploy) | Deployment and operations kit for PixivFlow + TelePost (Docker / VPS / cloud) | When you want to deploy and operate both projects together. Running PixivFlow alone does not need it |
306
+ | [pixivflow-webui](https://github.com/redtidev1918/pixivflow-webui) | WebUI frontend for PixivFlow | When you want a GUI to manage downloads and schedules |
307
+ | [pixiv-token-getter](https://github.com/redtidev1918/pixiv-token-getter) | PKCE OAuth login library and CLI (`ptg`) | PixivFlow's login dependency; also usable on its own to obtain Pixiv tokens |
308
+
182
309
  ## Feedback
183
310
 
184
311
  Bugs and feature requests go to
package/README.md CHANGED
@@ -2,42 +2,46 @@
2
2
 
3
3
  **语言 / Language:** 中文 · [English](README.en.md)
4
4
 
5
- Pixiv 批量下载与定时收集工具。支持插画和小说的批量下载、标签搜索、
6
- 多维度筛选和 Cron 定时任务,提供命令行与 WebUI 两种使用方式。
7
- 基于 TypeScript 和 Node.js,可在 Windows、macOS、Linux 及 Docker 中运行。
5
+ **Pixiv 下载、筛选与自动收集工具。**
6
+
7
+ 可以直接下载单个 Pixiv 作品(插画、小说、动图),也可以按标签、热度、日期和收藏数
8
+ 批量筛选,并通过 scheduler 定时自动收集。结果既能永久保存在本地,也能按需通过 HTTP
9
+ 可靠交付给其他服务——下游是可选的,PixivFlow 自己就能跑完「发现 → 筛选 → 下载 → 保存」
10
+ 的完整链路。
8
11
 
9
12
  [![Version](https://img.shields.io/npm/v/pixivflow?style=flat-square)](https://www.npmjs.com/package/pixivflow)
10
13
  [![Node](https://img.shields.io/badge/Node.js-22.13%2B_LTS-green.svg?style=flat-square&logo=node.js)](https://nodejs.org/)
11
14
  [![License](https://img.shields.io/badge/License-MIT-blue.svg?style=flat-square)](LICENSE)
12
15
  [![Documentation](https://img.shields.io/badge/Docs-redtidev1918.github.io-6366f1?style=flat-square)](https://redtidev1918.github.io/PixivFlow/)
13
16
 
14
- ## 安装
17
+ ## 典型场景
15
18
 
16
- 需要 Node.js 22.13 或更高版本;生产环境请使用仍受支持的 LTS。
17
- 下载 Pixiv 动图(ugoira)还需要 `python3` 和 `ffmpeg`:程序按逐帧延迟合成循环 GIF,
18
- 可直接交给 TelePost 作为动画发送。官方 Docker 镜像已包含两者,详见 [配置说明](docs/CONFIG.md#pixiv-动图ugoira)。
19
+ **1. 下载一个链接。** 直接粘贴任意 Pixiv 链接——插画、小说、系列、用户主页都能识别:
19
20
 
20
21
  ```bash
21
- npm install -g pixivflow
22
- pixivflow --help
22
+ pixivflow download --url https://www.pixiv.net/artworks/123456789
23
23
  ```
24
24
 
25
- 服务器部署推荐 Docker Compose,见 [DOCKER.md](docs/DOCKER.md);
26
- PixivFlow + TelePost 联合部署套件(含可选代理):
27
- [redtidev1918/pixivflow-telepost-deploy](https://github.com/redtidev1918/pixivflow-telepost-deploy)
28
- —— 一套配置,支持国内/海外、有/无公网 IP、VPS/Fly.io 任意场景。
29
- 从源码构建:
25
+ **2. 按条件批量下载。** 在配置里定义要收集什么(标签、榜单、发布日期、收藏数下限),
26
+ 一次跑完;已下载的作品由 SQLite 记录并自动跳过,重复运行不会重复拉取。
27
+ [筛选与下载目标](#筛选与下载目标)
30
28
 
31
- ```bash
32
- git clone https://github.com/redtidev1918/PixivFlow.git
33
- cd PixivFlow
34
- npm install
35
- npm run build
29
+ **3. 定时自动收集并交付。** 用 cron 长期挂机:定时发现、下载,再按需把内容投递给
30
+ 其他服务——对方确认收到后才删除本地副本。
31
+
32
+ ```text
33
+ Pixiv ──► PixivFlow ──┬──► 本地永久保存(persistent)
34
+ └──► HTTP 交付(cache)──► TelePost / 其他兼容服务
36
35
  ```
37
36
 
38
- Termux / Android 环境见 [TERMUX_INSTALL.md](docs/TERMUX_INSTALL.md)。
37
+ ## 快速开始
39
38
 
40
- ## 快速上手
39
+ 需要 Node.js 22.13 或更高版本;生产环境请使用仍受支持的 LTS。
40
+
41
+ ```bash
42
+ npm install -g pixivflow
43
+ pixivflow --help
44
+ ```
41
45
 
42
46
  登录 Pixiv 账号(生成 OAuth 凭据,只需一次):
43
47
 
@@ -59,35 +63,25 @@ pixivflow download
59
63
  pixivflow scheduler # 按 cron 配置长期挂机自动收集
60
64
  ```
61
65
 
62
- ### 单进程多计划与配置热重载
66
+ 不想手写配置?运行交互式向导 `pixivflow setup` 一步步生成。图形界面用
67
+ `pixivflow webui`(前端见 [pixivflow-webui](https://github.com/redtidev1918/pixivflow-webui))。
63
68
 
64
- `schedules[]` 可以为不同 target 组设置各自的 Cron。所有计划由一个 Node
65
- 进程托管,共享 Pixiv 客户端、SQLite 与文件服务;执行阶段使用有界串行队列,
66
- 适合 512 MiB 小内存机器(实测:`topic` 发现/采集/下载全程在 256 MB cgroup 限制下
67
- 稳定运行,峰值 RSS ≈ 106 MB、heapUsed ≈ 33 MB,无 OOM,见 [DOCKER.md](docs/DOCKER.md))。配置文件默认被监听,SSH/同步工具替换文件后会先完整
68
- 校验,再一次性替换全部调度项;无效 JSON、错误 Cron 或未知 target id 不会破坏
69
- 当前运行中的计划。正在执行的任务继续使用旧快照,下一次任务使用新快照。
69
+ 下载 Pixiv 动图(ugoira)还需要 `python3` `ffmpeg`:程序按逐帧延迟合成循环 GIF,
70
+ 可直接作为动画交付给下游。官方 Docker 镜像已包含两者,详见
71
+ [配置说明](docs/CONFIG.md#pixiv-动图ugoira)。
70
72
 
71
- ```json
72
- {
73
- "scheduler": { "enabled": false, "cron": "0 3 * * *" },
74
- "schedules": [
75
- { "id": "bot1", "enabled": true, "cron": "10 5 * * *", "targetIds": ["bot1-art", "bot1-novel"] },
76
- { "id": "bot2", "enabled": true, "cron": "30 5 * * *", "targetIds": ["bot2-art", "bot2-novel"] }
77
- ],
78
- "targets": [
79
- { "id": "bot1-art", "type": "illustration", "mode": "ranking", "rankingDate": "YESTERDAY" },
80
- { "id": "bot1-novel", "type": "novel", "mode": "ranking", "rankingDate": "YESTERDAY" }
81
- ]
82
- }
73
+ 从源码构建:
74
+
75
+ ```bash
76
+ git clone https://github.com/redtidev1918/PixivFlow.git
77
+ cd PixivFlow
78
+ npm install
79
+ npm run build
83
80
  ```
84
81
 
85
- 旧的单 `scheduler` 配置继续兼容。`pixiv`、`network`、`storage` 涉及长生命周期
86
- 连接或路径,修改后需要重启;`schedules`、`targets`、`delivery`、`download` 可以
87
- 热重载。完整双 Bot 缓存投递模板见
88
- [`config/fly-two-bots.example.json`](config/fly-two-bots.example.json)。
82
+ Termux / Android 环境见 [TERMUX_INSTALL.md](docs/TERMUX_INSTALL.md)。
89
83
 
90
- ## 下载目标
84
+ ## 筛选与下载目标
91
85
 
92
86
  在配置文件的 `targets` 中定义要收集的内容,多个条件可以组合:
93
87
 
@@ -103,7 +97,7 @@ pixivflow scheduler # 按 cron 配置长期挂机自动收集
103
97
  两者互不冲突。`mode: "topic"` 的插画任务会保留一个有界热度候选池:同一发布日期
104
98
  重复执行时若第一名已经下载,会按热度自动递补下一部未下载作品,而不是空跑。
105
99
 
106
- ### 本地留存与缓存交付
100
+ ## 本地留存与缓存交付
107
101
 
108
102
  每个 target(一个 tag / 计划)有两种保存方式:
109
103
 
@@ -111,7 +105,9 @@ pixivflow scheduler # 按 cron 配置长期挂机自动收集
111
105
  - **`cache`**:下载后投给一个「交付目标」(比如投稿机器人),对方确认收到后才删本地文件,省磁盘。
112
106
 
113
107
  「交付目标」就是一段配置:告诉 PixivFlow 把文件 POST 到哪个地址、带哪些字段。
114
- 它不绑定具体服务,可指向任何 HTTP 投稿接口(TelePost、telepress 等)。示例:
108
+ 它不绑定具体服务,可指向任意兼容的 HTTP 接口;
109
+ [TelePost](https://github.com/redtidev1918/TelePost) 与
110
+ [telepress](https://github.com/redtidev1918/telepress) 只是示例下游。示例:
115
111
 
116
112
  ```json
117
113
  {
@@ -119,12 +115,12 @@ pixivflow scheduler # 按 cron 配置长期挂机自动收集
119
115
  "outboxRetryBaseMs": 300000,
120
116
  "outboxRetryMaxMs": 21600000,
121
117
  "targets": {
122
- "tg-example": {
118
+ "sharing-api": {
123
119
  "type": "httpMultipart",
124
120
  "url": "https://your-domain.example/api/bot1/v1/submissions",
125
121
  "readinessUrl": "https://your-domain.example/ready",
126
122
  "notificationUrl": "https://your-domain.example/api/bot1/v1/notifications",
127
- "headers": { "Authorization": "Bearer ${TG_SUBMIT_TOKEN}" },
123
+ "headers": { "Authorization": "Bearer ${SHARING_TOKEN}" },
128
124
  "fileField": "files",
129
125
  "fields": { "title": "{{title}}" },
130
126
  "success": { "statuses": [201], "jsonPath": "ok", "equals": true },
@@ -142,7 +138,7 @@ pixivflow scheduler # 按 cron 配置长期挂机自动收集
142
138
  "tag": "更新",
143
139
  "storageMode": "cache",
144
140
  "delivery": {
145
- "target": "tg-example",
141
+ "target": "sharing-api",
146
142
  "fields": { "tags": ["公告", "更新"], "anonymous": false }
147
143
  }
148
144
  }
@@ -154,6 +150,49 @@ pixivflow scheduler # 按 cron 配置长期挂机自动收集
154
150
  插画 cache 投递会在 multipart 的 `previews` 字段携带 Pixiv 的低分辨率预览(与 `files`
155
151
  一一对应),原图仍是权威素材;通用接收端可以忽略该可选字段。
156
152
 
153
+ 运维通知可直接把 `notificationUrl` 指向 [Apprise API](docs/APPRISE.md),由 Apprise 统一发送
154
+ Email、Telegram、Discord、ntfy 等渠道;PixivFlow 不实现这些通知协议。
155
+
156
+ - 上面的 `url` 指向任意兼容的 HTTP 投稿接口;示例里用的是 TelePost 的
157
+ `/api/botN/v1/submissions`(把 `/gen_token` 得到的 `tp_...` 放进 `SHARING_TOKEN` 即可,
158
+ 这是示例服务自己的鉴权方式)。
159
+ - 同一目标也可指向 [telepress](https://github.com/redtidev1918/telepress) 的 `/publish/gallery`,
160
+ 把插画自动发布成 Telegra.ph 相册,见 [CONFIG.md](docs/CONFIG.md) 的
161
+ 「Telegraph(telegra.ph)相册上传」。
162
+
163
+ ## 自动化与可靠性
164
+
165
+ ### 单进程多计划与配置热重载
166
+
167
+ `schedules[]` 可以为不同 target 组设置各自的 Cron。所有计划由一个 Node
168
+ 进程托管,共享 Pixiv 客户端、SQLite 与文件服务;执行阶段使用有界串行队列,
169
+ 适合 512 MiB 小内存机器(实测:`topic` 发现/采集/下载全程在 256 MB cgroup 限制下
170
+ 稳定运行,峰值 RSS ≈ 106 MB、heapUsed ≈ 33 MB,无 OOM,见 [DOCKER.md](docs/DOCKER.md))。
171
+ 配置文件默认被监听,SSH/同步工具替换文件后会先完整校验,再一次性替换全部调度项;
172
+ 无效 JSON、错误 Cron 或未知 target id 不会破坏当前运行中的计划。正在执行的任务
173
+ 继续使用旧快照,下一次任务使用新快照。
174
+
175
+ ```json
176
+ {
177
+ "scheduler": { "enabled": false, "cron": "0 3 * * *" },
178
+ "schedules": [
179
+ { "id": "bot1", "enabled": true, "cron": "10 5 * * *", "targetIds": ["bot1-art", "bot1-novel"] },
180
+ { "id": "bot2", "enabled": true, "cron": "30 5 * * *", "targetIds": ["bot2-art", "bot2-novel"] }
181
+ ],
182
+ "targets": [
183
+ { "id": "bot1-art", "type": "illustration", "mode": "ranking", "rankingDate": "YESTERDAY" },
184
+ { "id": "bot1-novel", "type": "novel", "mode": "ranking", "rankingDate": "YESTERDAY" }
185
+ ]
186
+ }
187
+ ```
188
+
189
+ 旧的单 `scheduler` 配置继续兼容。`pixiv`、`network`、`storage` 涉及长生命周期
190
+ 连接或路径,修改后需要重启;`schedules`、`targets`、`delivery`、`download` 可以
191
+ 热重载。完整双 Bot 缓存投递模板见
192
+ [`config/fly-two-bots.example.json`](config/fly-two-bots.example.json)。
193
+
194
+ ### 投递事务发件箱
195
+
157
196
  投递是**事务发件箱(SQLite outbox)**式的,at-least-once 执行、effectively-once
158
197
  可见效果:每个外部副作用(一次内容投递、一条通知)在 `outbox` 表落一行,带幂等键和
159
198
  行级租约,独立 worker 立即泵送、指数退避重试(默认 5 分钟起步、最长 6 小时),超过
@@ -176,15 +215,9 @@ pixivflow outbox cancel <id> # 只取消尚未执行的 row
176
215
  ```
177
216
 
178
217
  `run-once` 会重新执行下载计划,不等价于 outbox replay;不要手改 SQLite 的
179
- `next_attempt_at`。
180
-
181
- 运维通知可直接把 `notificationUrl` 指向 [Apprise API](docs/APPRISE.md),由 Apprise 统一发送
182
- Email、Telegram、Discord、ntfy 等渠道;PixivFlow 不实现这些通知协议。
183
-
184
- - 上面的示例是投稿给一个 HTTP 接口:把 `/gen_token` 得到的 `tp_...` 放进 `TG_SUBMIT_TOKEN` 即可(这是示例服务自己的鉴权方式)。
185
- - 同一目标也可指向 [telepress](https://github.com/redtidev1918/telepress) 的 `/publish/gallery`,把插画自动发布成 Telegra.ph 相册,见 [CONFIG.md](docs/CONFIG.md) 的「Telegraph(telegra.ph)相册上传」。
186
-
187
- 不想手写配置?运行交互式向导 `pixivflow setup` 一步步生成。
218
+ `next_attempt_at`。长期运行环境的体检与收敛用 `pixivflow doctor`(卡住的 slot/outbox
219
+ 租约、pending 投递、dead 行,`--repair` 收敛)与 `pixivflow reconcile`(把下游已确认的
220
+ 历史重复登记进投递账本,默认 dry-run)。
188
221
 
189
222
  ## 常用命令
190
223
 
@@ -194,6 +227,7 @@ Email、Telegram、Discord、ntfy 等渠道;PixivFlow 不实现这些通知协
194
227
  | `pixivflow download --url <url>` | 通过 URL 直接下载 |
195
228
  | `pixivflow random` | 随机下载热门作品 |
196
229
  | `pixivflow scheduler` | 启动定时任务 |
230
+ | `pixivflow webui` | 启动 WebUI |
197
231
  | `pixivflow config` | 配置管理(查看 / 编辑 / 备份 / 恢复) |
198
232
  | `pixivflow status` | 下载统计与最近记录 |
199
233
  | `pixivflow health` | 健康检查:配置、目录可写性、连通性 |
@@ -209,6 +243,15 @@ Email、Telegram、Discord、ntfy 等渠道;PixivFlow 不实现这些通知协
209
243
 
210
244
  其他用法见 [USAGE.md](docs/USAGE.md);从 v1 升级到 v2 见 [迁移指南](docs/MIGRATION.md)。
211
245
 
246
+ ## 部署
247
+
248
+ - **Docker / 服务器长期挂机**:见 [DOCKER.md](docs/DOCKER.md)。
249
+ - **Android / Termux**:见 [TERMUX_INSTALL.md](docs/TERMUX_INSTALL.md)。
250
+ - **与 TelePost 组合部署**:PixivFlow 与 TelePost 都可以独立使用;只有当你希望把两者
251
+ 组合成一套完整工作流时,才需要
252
+ [pixivflow-telepost-deploy](https://github.com/redtidev1918/pixivflow-telepost-deploy)
253
+ 这个部署与运维套件。
254
+
212
255
  ## 文档
213
256
 
214
257
  完整教程站点:<https://redtidev1918.github.io/PixivFlow/>
@@ -229,6 +272,17 @@ Email、Telegram、Discord、ntfy 等渠道;PixivFlow 不实现这些通知协
229
272
 
230
273
  English version: [README.en.md](README.en.md).
231
274
 
275
+ ## 相关项目
276
+
277
+ PixivFlow 可以完全独立使用。下面是同一作者生态里与它相关的项目,以及各自负责什么:
278
+
279
+ | 项目 | 是什么 | 什么时候需要 |
280
+ | --- | --- | --- |
281
+ | [TelePost](https://github.com/redtidev1918/TelePost) | Telegram 频道投稿、审核与自动化发布平台 | 想把下载结果投进 Telegram 频道、先人工审核再发布时,把它配成 delivery 下游即可。这只是可选组合,PixivFlow 不依赖它 |
282
+ | [pixivflow-telepost-deploy](https://github.com/redtidev1918/pixivflow-telepost-deploy) | PixivFlow + TelePost 的部署与运维套件(Docker / VPS / 云平台) | 想一次性把上面两个项目部署并运维起来时。只跑 PixivFlow 不需要它 |
283
+ | [pixivflow-webui](https://github.com/redtidev1918/pixivflow-webui) | PixivFlow 的 WebUI 前端 | 想用图形界面管理下载与计划 |
284
+ | [pixiv-token-getter](https://github.com/redtidev1918/pixiv-token-getter) | PKCE OAuth 登录库与 CLI(`ptg`) | PixivFlow 的登录依赖;也可以单独用于获取 Pixiv token |
285
+
232
286
  ## 问题反馈
233
287
 
234
288
  Bug 与功能建议请提交到