pixivflow 2.20.5 → 2.21.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (32) hide show
  1. package/README.en.md +179 -56
  2. package/README.md +110 -57
  3. package/dist/commands/SchedulerCommand.js +63 -0
  4. package/dist/commands/SchedulerIdleLifecycle.d.ts +8 -0
  5. package/dist/commands/SchedulerIdleLifecycle.js +2 -1
  6. package/dist/commands/scheduler-runtime.js +17 -0
  7. package/dist/config/types.d.ts +26 -0
  8. package/dist/config/validation.js +21 -0
  9. package/dist/delivery/types.d.ts +15 -0
  10. package/dist/notification/NotificationPolicy.d.ts +2 -0
  11. package/dist/notification/NotificationPolicy.js +18 -1
  12. package/dist/package.json +1 -1
  13. package/dist/scheduler/MultiScheduleManager.d.ts +12 -1
  14. package/dist/scheduler/MultiScheduleManager.js +46 -50
  15. package/dist/scheduler/RecoveryPolicy.d.ts +45 -0
  16. package/dist/scheduler/RecoveryPolicy.js +63 -0
  17. package/dist/scheduler/ResourceAdmission.d.ts +68 -0
  18. package/dist/scheduler/ResourceAdmission.js +83 -0
  19. package/dist/scheduler/ScheduleTriggerServer.d.ts +16 -0
  20. package/dist/scheduler/ScheduleTriggerServer.js +55 -0
  21. package/dist/scheduler/Scheduler.d.ts +19 -2
  22. package/dist/scheduler/Scheduler.js +12 -3
  23. package/dist/scheduler/SlotCoordinator.d.ts +20 -0
  24. package/dist/scheduler/SlotCoordinator.js +44 -2
  25. package/dist/scheduler/TargetOutcome.d.ts +31 -0
  26. package/dist/scheduler/TargetOutcome.js +114 -0
  27. package/dist/storage/DatabaseMigration.js +17 -0
  28. package/dist/storage/repositories/SlotRepository.d.ts +29 -0
  29. package/dist/storage/repositories/SlotRepository.js +28 -2
  30. package/dist/version.js +1 -1
  31. package/dist/webui/package.json +1 -1
  32. package/package.json +2 -2
package/README.en.md CHANGED
@@ -2,41 +2,51 @@
2
2
 
3
3
  **Language / 语言:** [中文](README.md) · English
4
4
 
5
- Pixiv batch download and scheduled collection tool. Bulk-download
6
- illustrations and novels, search by tag, filter by bookmarks and dates,
7
- and run recurring collection jobs on a schedule. CLI and WebUI included.
8
- Built with TypeScript and Node.js; runs on Windows, macOS, Linux and Docker.
5
+ **Pixiv downloader, filter and automatic collection tool.**
6
+
7
+ Download a single Pixiv artwork (illustration, novel, ugoira) directly, or batch-collect
8
+ by tag, ranking, publish date and bookmark count, and let the scheduler keep collecting
9
+ on a cron. Results stay on your disk, or get delivered reliably over HTTP to another
10
+ service — downstreams are optional, PixivFlow alone covers
11
+ discover → filter → download → save.
9
12
 
10
13
  [![Version](https://img.shields.io/npm/v/pixivflow?style=flat-square)](https://www.npmjs.com/package/pixivflow)
11
14
  [![Node](https://img.shields.io/badge/Node.js-22.13%2B_LTS-green.svg?style=flat-square&logo=node.js)](https://nodejs.org/)
12
15
  [![License](https://img.shields.io/badge/License-MIT-blue.svg?style=flat-square)](LICENSE)
13
16
  [![Documentation](https://img.shields.io/badge/Docs-redtidev1918.github.io-6366f1?style=flat-square)](https://redtidev1918.github.io/PixivFlow/)
14
17
 
15
- ## Install
18
+ ## Typical scenarios
16
19
 
17
- Requires Node.js 22.13 or later; use a currently supported LTS release in production.
20
+ **1. Download one link.** Paste any Pixiv link artwork, novel, series, or user
21
+ profile are all recognized:
18
22
 
19
23
  ```bash
20
- npm install -g pixivflow
21
- pixivflow --help
24
+ pixivflow download --url https://www.pixiv.net/artworks/123456789
22
25
  ```
23
26
 
24
- For servers, prefer the Docker Compose setup described in
25
- [DOCKER.md](docs/DOCKER.md). To compose PixivFlow with TelePost, use the optional
26
- [pixivflow-telepost-deploy](https://github.com/redtidev1918/pixivflow-telepost-deploy)
27
- deployment and operations kit. To build from source:
27
+ **2. Batch download by conditions.** Define what to collect in your config (tag,
28
+ ranking, date range, minimum bookmarks) and run it in one pass. Downloaded items are
29
+ tracked in SQLite and skipped, so re-running never re-downloads. See
30
+ [Filtering and targets](#filtering-and-targets).
28
31
 
29
- ```bash
30
- git clone https://github.com/redtidev1918/PixivFlow.git
31
- cd PixivFlow
32
- npm install
33
- npm run build
34
- ```
32
+ **3. Collect on a schedule and deliver.** Run it under cron: discover, download, and
33
+ optionally hand files to another service — the local copy is deleted only after the
34
+ receiver confirms.
35
35
 
36
- Termux / Android: see [TERMUX_INSTALL.md](docs/TERMUX_INSTALL.md).
36
+ ```text
37
+ Pixiv ──► PixivFlow ──┬──► local, permanent (persistent)
38
+ └──► HTTP delivery (cache) ──► TelePost / any compatible service
39
+ ```
37
40
 
38
41
  ## Quick start
39
42
 
43
+ Requires Node.js 22.13 or later; use a currently supported LTS release in production.
44
+
45
+ ```bash
46
+ npm install -g pixivflow
47
+ pixivflow --help
48
+ ```
49
+
40
50
  Sign in to your Pixiv account (creates OAuth credentials, one time):
41
51
 
42
52
  ```bash
@@ -58,22 +68,27 @@ pixivflow download
58
68
  pixivflow scheduler # long-running cron collection
59
69
  ```
60
70
 
61
- ### Multiple schedules and atomic hot reload
71
+ Prefer not to hand-write config? The interactive wizard `pixivflow setup` generates it.
72
+ For a GUI, run `pixivflow webui` (frontend:
73
+ [pixivflow-webui](https://github.com/redtidev1918/pixivflow-webui)).
62
74
 
63
- `schedules[]` hosts independently timed target groups in one Node process.
64
- Plans share Pixiv authentication, SQLite, and file services, while a bounded
65
- serial queue prevents overlapping downloads from producing memory spikes.
66
- The active config is watched by default: replace it over SSH and PixivFlow
67
- fully validates the new snapshot before swapping the entire cron table. An
68
- invalid edit leaves the previous schedules running. In-flight work finishes
69
- on its old snapshot; the next run sees the new one.
75
+ Downloading Pixiv ugoira additionally needs `python3` and `ffmpeg`: frames are
76
+ composited into a looping GIF by per-frame delay, ready to be delivered as an animation.
77
+ The official Docker image includes both see
78
+ [configuration notes](docs/CONFIG.md#pixiv-动图ugoira).
70
79
 
71
- Legacy single-`scheduler` configs remain supported. `schedules`, `targets`,
72
- `delivery`, and `download` are hot-reloadable; changes to `pixiv`, `network`,
73
- or `storage` require a process restart. See the ready-to-edit
74
- [`config/fly-two-bots.example.json`](config/fly-two-bots.example.json) template.
80
+ Build from source:
81
+
82
+ ```bash
83
+ git clone https://github.com/redtidev1918/PixivFlow.git
84
+ cd PixivFlow
85
+ npm install
86
+ npm run build
87
+ ```
75
88
 
76
- ## Download targets
89
+ Termux / Android: see [TERMUX_INSTALL.md](docs/TERMUX_INSTALL.md).
90
+
91
+ ## Filtering and targets
77
92
 
78
93
  Define what to collect in the `targets` section of your config. Conditions
79
94
  combine:
@@ -88,32 +103,44 @@ combine:
88
103
 
89
104
  Downloaded items are tracked in a SQLite database and skipped automatically;
90
105
  files that exist without a database record are reconciled, so the two never
91
- conflict.
106
+ conflict. Illustration tasks with `mode: "topic"` keep a bounded hotness candidate
107
+ pool: if the top item is already downloaded, the next undownloaded one is promoted by
108
+ hotness instead of the run coming back empty.
109
+
110
+ ## Persistent and cache delivery modes
92
111
 
93
- ### Persistent and cache delivery modes
112
+ Each target (one tag / schedule) can use one of two storage modes:
94
113
 
95
- Each target can use `storageMode: "persistent"` (the default, keep files) or
96
- `storageMode: "cache"` (send files to a named delivery target and delete them
97
- only after success). The delivery layer is service-independent;
114
+ - **`persistent`** (the default): keep files forever.
115
+ - **`cache`**: send files to a named delivery target and delete the local copy only
116
+ after the receiver confirms saves disk.
117
+
118
+ A "delivery target" is just configuration: which URL to POST to and which fields to
119
+ send. It is not bound to any service and can point at any compatible HTTP endpoint;
98
120
  [TelePost](https://github.com/redtidev1918/TelePost) and
99
- [telepress](https://github.com/redtidev1918/telepress) are example downstreams,
100
- not dependencies. This example merely translates an HTTP multipart submission API into configuration:
121
+ [telepress](https://github.com/redtidev1918/telepress) are example downstreams. Example:
101
122
 
102
123
  ```json
103
124
  {
104
125
  "delivery": {
126
+ "outboxRetryBaseMs": 300000,
127
+ "outboxRetryMaxMs": 21600000,
105
128
  "targets": {
106
129
  "sharing-api": {
107
130
  "type": "httpMultipart",
108
- "url": "https://example.test/submissions",
109
- "readinessUrl": "https://example.test/ready",
110
- "notificationUrl": "https://example.test/notifications",
131
+ "url": "https://your-domain.example/api/bot1/v1/submissions",
132
+ "readinessUrl": "https://your-domain.example/ready",
133
+ "notificationUrl": "https://your-domain.example/api/bot1/v1/notifications",
111
134
  "headers": { "Authorization": "Bearer ${SHARING_TOKEN}" },
112
135
  "fileField": "files",
113
136
  "fields": { "title": "{{title}}" },
114
- "success": { "statuses": [201], "jsonPath": "ok", "equals": true }
137
+ "success": { "statuses": [201], "jsonPath": "ok", "equals": true },
138
+ "arrayFormat": "comma",
139
+ "maxAttempts": 3,
140
+ "retryDelayMs": 2000
115
141
  }
116
- }
142
+ },
143
+ "deleteAfterDelivery": true
117
144
  },
118
145
  "targets": [
119
146
  { "type": "illustration", "tag": "archive", "storageMode": "persistent" },
@@ -130,20 +157,89 @@ not dependencies. This example merely translates an HTTP multipart submission AP
130
157
  }
131
158
  ```
132
159
 
133
- Headers and URLs accept arbitrary `${ENV_NAME}` interpolation. Cache-mode
134
- illustration delivery also sends an optional, one-to-one Pixiv preview in the
135
- `previews` multipart field while retaining the original as the authoritative
136
- artifact. Failed delivery keeps both files in the SQLite outbox; the worker
137
- retries the same intent after restart. Final no-match
138
- notifications use the same durable outbox, so a restart or a temporary review
139
- endpoint outage does not silently lose the alert.
160
+ Headers and URLs accept `${ENV_NAME}` interpolation (never hard-code tokens). Cache-mode
161
+ illustration delivery also sends an optional, one-to-one Pixiv preview in the `previews`
162
+ multipart field while retaining the original as the authoritative artifact; a generic
163
+ receiver may ignore that field.
164
+
165
+ Operational notifications can point `notificationUrl` at an
166
+ [Apprise API](docs/APPRISE.md) instance, which fans out to Email, Telegram, Discord,
167
+ ntfy and more; PixivFlow does not implement those notification protocols itself.
168
+
169
+ - The `url` above is any compatible HTTP submission endpoint; the example uses
170
+ TelePost's `/api/botN/v1/submissions` (put the `tp_...` from `/gen_token` into
171
+ `SHARING_TOKEN` — that is the example service's own auth scheme).
172
+ - The same target can point at [telepress](https://github.com/redtidev1918/telepress)'s
173
+ `/publish/gallery` to publish illustrations as a Telegra.ph gallery, see the
174
+ "Telegraph (telegra.ph) gallery upload" section of [CONFIG.md](docs/CONFIG.md).
175
+
176
+ ## Automation and reliability
177
+
178
+ ### Multiple schedules and atomic hot reload
179
+
180
+ `schedules[]` hosts independently timed target groups in one Node process.
181
+ Plans share Pixiv authentication, SQLite, and file services, while a bounded
182
+ serial queue prevents overlapping downloads from producing memory spikes. This runs
183
+ comfortably on small machines — measured with `topic` discovery, selection and
184
+ download inside a 256 MB cgroup: peak RSS ≈ 106 MB, heapUsed ≈ 33 MB, no OOM
185
+ (see [DOCKER.md](docs/DOCKER.md)).
186
+ The active config is watched by default: replace it over SSH and PixivFlow
187
+ fully validates the new snapshot before swapping the entire cron table. An
188
+ invalid edit leaves the previous schedules running. In-flight work finishes
189
+ on its old snapshot; the next run sees the new one.
190
+
191
+ ```json
192
+ {
193
+ "scheduler": { "enabled": false, "cron": "0 3 * * *" },
194
+ "schedules": [
195
+ { "id": "bot1", "enabled": true, "cron": "10 5 * * *", "targetIds": ["bot1-art", "bot1-novel"] },
196
+ { "id": "bot2", "enabled": true, "cron": "30 5 * * *", "targetIds": ["bot2-art", "bot2-novel"] }
197
+ ],
198
+ "targets": [
199
+ { "id": "bot1-art", "type": "illustration", "mode": "ranking", "rankingDate": "YESTERDAY" },
200
+ { "id": "bot1-novel", "type": "novel", "mode": "ranking", "rankingDate": "YESTERDAY" }
201
+ ]
202
+ }
203
+ ```
204
+
205
+ Legacy single-`scheduler` configs remain supported. `schedules`, `targets`,
206
+ `delivery`, and `download` are hot-reloadable; changes to `pixiv`, `network`,
207
+ or `storage` require a process restart. See the ready-to-edit
208
+ [`config/fly-two-bots.example.json`](config/fly-two-bots.example.json) template.
140
209
 
141
- When `readinessUrl` is configured, a non-2xx readiness response defers the row
142
- without incrementing its attempt count. Use `pixivflow outbox list`,
143
- `inspect <id>`, `retry <id>` / `retry --dead`, and `cancel <id>` for recovery;
144
- `run-once` is not an outbox replay command.
210
+ ### Durable delivery outbox
211
+
212
+ Delivery is transactional (a SQLite outbox): at-least-once execution with
213
+ effectively-once visible effects. Every external side effect (one content delivery,
214
+ one notification) becomes a row with an idempotency key and a row-level lease; an
215
+ independent worker pumps it right away and retries with exponential backoff (5 minutes
216
+ up to 6 hours by default), and rows that exceed `maxAttempts` go dead and are recorded
217
+ as `failed` in the delivery ledger. After a crash or machine hang, expired `processing`
218
+ leases are taken over by the next process, which retries the same idempotent intent — a
219
+ downstream converges via `idempotent_replay` (same key, lost ACK) or
220
+ `duplicate_existing` (historical duplicate), so the channel still shows exactly one
221
+ message. Legacy file-based `delivery-outbox/*.json` manifests are migrated into SQLite
222
+ once at startup, idempotently. "Nothing to publish today" notifications share that table
223
+ but are pumped independently, so a temporarily unavailable review endpoint cannot block
224
+ content delivery.
225
+
226
+ When `readinessUrl` is configured, a non-2xx readiness response returns the row to
227
+ pending without incrementing its attempt count — strictly separate from `/live`'s
228
+ "process is alive" meaning. Dead letters are recovered through first-class CLI:
145
229
 
146
- Interactive configuration wizard: `pixivflow setup`.
230
+ ```bash
231
+ pixivflow outbox list --status dead
232
+ pixivflow outbox inspect <id>
233
+ pixivflow outbox retry <id> # only accepts dead rows, keeps the idempotency key
234
+ pixivflow outbox retry --dead
235
+ pixivflow outbox cancel <id> # cancels rows that have not run yet
236
+ ```
237
+
238
+ `run-once` re-runs the download plan; it is not an outbox replay, and you should not
239
+ hand-edit SQLite's `next_attempt_at`. For long-running hosts, use `pixivflow doctor`
240
+ (stuck slot/outbox leases, pending deliveries, dead rows; `--repair` converges) and
241
+ `pixivflow reconcile` (record downstream-confirmed historical duplicates in the delivery
242
+ ledger; dry-run by default).
147
243
 
148
244
  ## Common commands
149
245
 
@@ -153,16 +249,31 @@ Interactive configuration wizard: `pixivflow setup`.
153
249
  | `pixivflow download --url <url>` | Direct download via URL |
154
250
  | `pixivflow random` | Random popular artwork |
155
251
  | `pixivflow scheduler` | Start scheduled jobs |
252
+ | `pixivflow webui` | Start the WebUI |
156
253
  | `pixivflow config` | Manage config (view / edit / backup / restore) |
157
254
  | `pixivflow status` | Download stats and recent records |
158
255
  | `pixivflow health` | Health check: config, directories, connectivity |
256
+ | `pixivflow doctor` | Reliability check: stuck slot/outbox leases, pending deliveries, dead rows; `--repair` converges |
257
+ | `pixivflow reconcile` | Record downstream-confirmed historical duplicates (dry-run by default, `--repair` writes) |
258
+ | `pixivflow outbox` | List, inspect, replay dead letters, or cancel durable intents that have not run |
159
259
  | `pixivflow tags discover <seed>` | Discover related tags (Pixiv autocomplete + tag co-occurrence); lists candidates only |
160
260
  | `pixivflow tags apply <manifest> --target <id> --select <tag1,tag2>` | Atomically write chosen tags into config after manual confirmation, then hot-reload |
261
+ | `pixivflow topic resolve <topic>` | Inspect the derived tag space for a topic (`--type illustration\|novel`, `--refresh`) |
262
+ | `pixivflow topic test <topic> --date YESTERDAY` | Dry-run the candidates and Top N for one day, without downloading |
161
263
 
162
264
  `tags discover` calls the Pixiv autocomplete endpoint and samples recent illustrations/novels to count co-occurring tags, caching results for 7 days; it **never** changes active plans. After reviewing candidates, run `tags apply` to explicitly select tags: it validates the whole config, writes a backup and atomically replaces the file so a running scheduler hot-reloads it.
163
265
 
164
266
  More commands in [USAGE.md](docs/USAGE.md); see the [migration guide](docs/MIGRATION.md) for upgrading from v1 to v2.
165
267
 
268
+ ## Deployment
269
+
270
+ - **Docker / long-running server**: see [DOCKER.md](docs/DOCKER.md).
271
+ - **Android / Termux**: see [TERMUX_INSTALL.md](docs/TERMUX_INSTALL.md).
272
+ - **Composing with TelePost**: PixivFlow and TelePost are both usable on their own.
273
+ Only if you want to deploy the two together as one workflow do you need the
274
+ [pixivflow-telepost-deploy](https://github.com/redtidev1918/pixivflow-telepost-deploy)
275
+ deployment and operations kit.
276
+
166
277
  ## Documentation
167
278
 
168
279
  Full tutorial site: <https://redtidev1918.github.io/PixivFlow/>
@@ -183,6 +294,18 @@ Full tutorial site: <https://redtidev1918.github.io/PixivFlow/>
183
294
 
184
295
  Chinese version: [README.md](README.md).
185
296
 
297
+ ## Related projects
298
+
299
+ PixivFlow is fully standalone. These are the related projects in the same author's
300
+ ecosystem, and what each one owns:
301
+
302
+ | Project | What it is | When you need it |
303
+ | --- | --- | --- |
304
+ | [TelePost](https://github.com/redtidev1918/TelePost) | Telegram channel submission, moderation and automated publishing platform | When you want downloads to land in a Telegram channel for human review before publishing — configure it as a delivery downstream. This is an optional composition; PixivFlow does not depend on it |
305
+ | [pixivflow-telepost-deploy](https://github.com/redtidev1918/pixivflow-telepost-deploy) | Deployment and operations kit for PixivFlow + TelePost (Docker / VPS / cloud) | When you want to deploy and operate both projects together. Running PixivFlow alone does not need it |
306
+ | [pixivflow-webui](https://github.com/redtidev1918/pixivflow-webui) | WebUI frontend for PixivFlow | When you want a GUI to manage downloads and schedules |
307
+ | [pixiv-token-getter](https://github.com/redtidev1918/pixiv-token-getter) | PKCE OAuth login library and CLI (`ptg`) | PixivFlow's login dependency; also usable on its own to obtain Pixiv tokens |
308
+
186
309
  ## Feedback
187
310
 
188
311
  Bugs and feature requests go to
package/README.md CHANGED
@@ -2,41 +2,46 @@
2
2
 
3
3
  **语言 / Language:** 中文 · [English](README.en.md)
4
4
 
5
- Pixiv 批量下载与定时收集工具。支持插画和小说的批量下载、标签搜索、
6
- 多维度筛选和 Cron 定时任务,提供命令行与 WebUI 两种使用方式。
7
- 基于 TypeScript 和 Node.js,可在 Windows、macOS、Linux 及 Docker 中运行。
5
+ **Pixiv 下载、筛选与自动收集工具。**
6
+
7
+ 可以直接下载单个 Pixiv 作品(插画、小说、动图),也可以按标签、热度、日期和收藏数
8
+ 批量筛选,并通过 scheduler 定时自动收集。结果既能永久保存在本地,也能按需通过 HTTP
9
+ 可靠交付给其他服务——下游是可选的,PixivFlow 自己就能跑完「发现 → 筛选 → 下载 → 保存」
10
+ 的完整链路。
8
11
 
9
12
  [![Version](https://img.shields.io/npm/v/pixivflow?style=flat-square)](https://www.npmjs.com/package/pixivflow)
10
13
  [![Node](https://img.shields.io/badge/Node.js-22.13%2B_LTS-green.svg?style=flat-square&logo=node.js)](https://nodejs.org/)
11
14
  [![License](https://img.shields.io/badge/License-MIT-blue.svg?style=flat-square)](LICENSE)
12
15
  [![Documentation](https://img.shields.io/badge/Docs-redtidev1918.github.io-6366f1?style=flat-square)](https://redtidev1918.github.io/PixivFlow/)
13
16
 
14
- ## 安装
17
+ ## 典型场景
15
18
 
16
- 需要 Node.js 22.13 或更高版本;生产环境请使用仍受支持的 LTS。
17
- 下载 Pixiv 动图(ugoira)还需要 `python3` 和 `ffmpeg`:程序按逐帧延迟合成循环 GIF,
18
- 可直接交给 TelePost 作为动画发送。官方 Docker 镜像已包含两者,详见 [配置说明](docs/CONFIG.md#pixiv-动图ugoira)。
19
+ **1. 下载一个链接。** 直接粘贴任意 Pixiv 链接——插画、小说、系列、用户主页都能识别:
19
20
 
20
21
  ```bash
21
- npm install -g pixivflow
22
- pixivflow --help
22
+ pixivflow download --url https://www.pixiv.net/artworks/123456789
23
23
  ```
24
24
 
25
- 服务器部署见 [DOCKER.md](docs/DOCKER.md)。需要把 PixivFlow 与 TelePost 组合运行时,再使用
26
- [pixivflow-telepost-deploy](https://github.com/redtidev1918/pixivflow-telepost-deploy);
27
- 它是可选的部署与运维套件。
28
- 从源码构建:
25
+ **2. 按条件批量下载。** 在配置里定义要收集什么(标签、榜单、发布日期、收藏数下限),
26
+ 一次跑完;已下载的作品由 SQLite 记录并自动跳过,重复运行不会重复拉取。
27
+ 见[筛选与下载目标](#筛选与下载目标)。
29
28
 
30
- ```bash
31
- git clone https://github.com/redtidev1918/PixivFlow.git
32
- cd PixivFlow
33
- npm install
34
- npm run build
29
+ **3. 定时自动收集并交付。** 用 cron 长期挂机:定时发现、下载,再按需把内容投递给
30
+ 其他服务——对方确认收到后才删除本地副本。
31
+
32
+ ```text
33
+ Pixiv ──► PixivFlow ──┬──► 本地永久保存(persistent)
34
+ └──► HTTP 交付(cache)──► TelePost / 其他兼容服务
35
35
  ```
36
36
 
37
- Termux / Android 环境见 [TERMUX_INSTALL.md](docs/TERMUX_INSTALL.md)。
37
+ ## 快速开始
38
38
 
39
- ## 快速上手
39
+ 需要 Node.js 22.13 或更高版本;生产环境请使用仍受支持的 LTS。
40
+
41
+ ```bash
42
+ npm install -g pixivflow
43
+ pixivflow --help
44
+ ```
40
45
 
41
46
  登录 Pixiv 账号(生成 OAuth 凭据,只需一次):
42
47
 
@@ -58,35 +63,25 @@ pixivflow download
58
63
  pixivflow scheduler # 按 cron 配置长期挂机自动收集
59
64
  ```
60
65
 
61
- ### 单进程多计划与配置热重载
66
+ 不想手写配置?运行交互式向导 `pixivflow setup` 一步步生成。图形界面用
67
+ `pixivflow webui`(前端见 [pixivflow-webui](https://github.com/redtidev1918/pixivflow-webui))。
62
68
 
63
- `schedules[]` 可以为不同 target 组设置各自的 Cron。所有计划由一个 Node
64
- 进程托管,共享 Pixiv 客户端、SQLite 与文件服务;执行阶段使用有界串行队列,
65
- 适合 512 MiB 小内存机器(实测:`topic` 发现/采集/下载全程在 256 MB cgroup 限制下
66
- 稳定运行,峰值 RSS ≈ 106 MB、heapUsed ≈ 33 MB,无 OOM,见 [DOCKER.md](docs/DOCKER.md))。配置文件默认被监听,SSH/同步工具替换文件后会先完整
67
- 校验,再一次性替换全部调度项;无效 JSON、错误 Cron 或未知 target id 不会破坏
68
- 当前运行中的计划。正在执行的任务继续使用旧快照,下一次任务使用新快照。
69
+ 下载 Pixiv 动图(ugoira)还需要 `python3` `ffmpeg`:程序按逐帧延迟合成循环 GIF,
70
+ 可直接作为动画交付给下游。官方 Docker 镜像已包含两者,详见
71
+ [配置说明](docs/CONFIG.md#pixiv-动图ugoira)。
69
72
 
70
- ```json
71
- {
72
- "scheduler": { "enabled": false, "cron": "0 3 * * *" },
73
- "schedules": [
74
- { "id": "bot1", "enabled": true, "cron": "10 5 * * *", "targetIds": ["bot1-art", "bot1-novel"] },
75
- { "id": "bot2", "enabled": true, "cron": "30 5 * * *", "targetIds": ["bot2-art", "bot2-novel"] }
76
- ],
77
- "targets": [
78
- { "id": "bot1-art", "type": "illustration", "mode": "ranking", "rankingDate": "YESTERDAY" },
79
- { "id": "bot1-novel", "type": "novel", "mode": "ranking", "rankingDate": "YESTERDAY" }
80
- ]
81
- }
73
+ 从源码构建:
74
+
75
+ ```bash
76
+ git clone https://github.com/redtidev1918/PixivFlow.git
77
+ cd PixivFlow
78
+ npm install
79
+ npm run build
82
80
  ```
83
81
 
84
- 旧的单 `scheduler` 配置继续兼容。`pixiv`、`network`、`storage` 涉及长生命周期
85
- 连接或路径,修改后需要重启;`schedules`、`targets`、`delivery`、`download` 可以
86
- 热重载。完整双 Bot 缓存投递模板见
87
- [`config/fly-two-bots.example.json`](config/fly-two-bots.example.json)。
82
+ Termux / Android 环境见 [TERMUX_INSTALL.md](docs/TERMUX_INSTALL.md)。
88
83
 
89
- ## 下载目标
84
+ ## 筛选与下载目标
90
85
 
91
86
  在配置文件的 `targets` 中定义要收集的内容,多个条件可以组合:
92
87
 
@@ -102,7 +97,7 @@ pixivflow scheduler # 按 cron 配置长期挂机自动收集
102
97
  两者互不冲突。`mode: "topic"` 的插画任务会保留一个有界热度候选池:同一发布日期
103
98
  重复执行时若第一名已经下载,会按热度自动递补下一部未下载作品,而不是空跑。
104
99
 
105
- ### 本地留存与缓存交付
100
+ ## 本地留存与缓存交付
106
101
 
107
102
  每个 target(一个 tag / 计划)有两种保存方式:
108
103
 
@@ -120,12 +115,12 @@ pixivflow scheduler # 按 cron 配置长期挂机自动收集
120
115
  "outboxRetryBaseMs": 300000,
121
116
  "outboxRetryMaxMs": 21600000,
122
117
  "targets": {
123
- "tg-example": {
118
+ "sharing-api": {
124
119
  "type": "httpMultipart",
125
120
  "url": "https://your-domain.example/api/bot1/v1/submissions",
126
121
  "readinessUrl": "https://your-domain.example/ready",
127
122
  "notificationUrl": "https://your-domain.example/api/bot1/v1/notifications",
128
- "headers": { "Authorization": "Bearer ${TG_SUBMIT_TOKEN}" },
123
+ "headers": { "Authorization": "Bearer ${SHARING_TOKEN}" },
129
124
  "fileField": "files",
130
125
  "fields": { "title": "{{title}}" },
131
126
  "success": { "statuses": [201], "jsonPath": "ok", "equals": true },
@@ -143,7 +138,7 @@ pixivflow scheduler # 按 cron 配置长期挂机自动收集
143
138
  "tag": "更新",
144
139
  "storageMode": "cache",
145
140
  "delivery": {
146
- "target": "tg-example",
141
+ "target": "sharing-api",
147
142
  "fields": { "tags": ["公告", "更新"], "anonymous": false }
148
143
  }
149
144
  }
@@ -155,6 +150,49 @@ pixivflow scheduler # 按 cron 配置长期挂机自动收集
155
150
  插画 cache 投递会在 multipart 的 `previews` 字段携带 Pixiv 的低分辨率预览(与 `files`
156
151
  一一对应),原图仍是权威素材;通用接收端可以忽略该可选字段。
157
152
 
153
+ 运维通知可直接把 `notificationUrl` 指向 [Apprise API](docs/APPRISE.md),由 Apprise 统一发送
154
+ Email、Telegram、Discord、ntfy 等渠道;PixivFlow 不实现这些通知协议。
155
+
156
+ - 上面的 `url` 指向任意兼容的 HTTP 投稿接口;示例里用的是 TelePost 的
157
+ `/api/botN/v1/submissions`(把 `/gen_token` 得到的 `tp_...` 放进 `SHARING_TOKEN` 即可,
158
+ 这是示例服务自己的鉴权方式)。
159
+ - 同一目标也可指向 [telepress](https://github.com/redtidev1918/telepress) 的 `/publish/gallery`,
160
+ 把插画自动发布成 Telegra.ph 相册,见 [CONFIG.md](docs/CONFIG.md) 的
161
+ 「Telegraph(telegra.ph)相册上传」。
162
+
163
+ ## 自动化与可靠性
164
+
165
+ ### 单进程多计划与配置热重载
166
+
167
+ `schedules[]` 可以为不同 target 组设置各自的 Cron。所有计划由一个 Node
168
+ 进程托管,共享 Pixiv 客户端、SQLite 与文件服务;执行阶段使用有界串行队列,
169
+ 适合 512 MiB 小内存机器(实测:`topic` 发现/采集/下载全程在 256 MB cgroup 限制下
170
+ 稳定运行,峰值 RSS ≈ 106 MB、heapUsed ≈ 33 MB,无 OOM,见 [DOCKER.md](docs/DOCKER.md))。
171
+ 配置文件默认被监听,SSH/同步工具替换文件后会先完整校验,再一次性替换全部调度项;
172
+ 无效 JSON、错误 Cron 或未知 target id 不会破坏当前运行中的计划。正在执行的任务
173
+ 继续使用旧快照,下一次任务使用新快照。
174
+
175
+ ```json
176
+ {
177
+ "scheduler": { "enabled": false, "cron": "0 3 * * *" },
178
+ "schedules": [
179
+ { "id": "bot1", "enabled": true, "cron": "10 5 * * *", "targetIds": ["bot1-art", "bot1-novel"] },
180
+ { "id": "bot2", "enabled": true, "cron": "30 5 * * *", "targetIds": ["bot2-art", "bot2-novel"] }
181
+ ],
182
+ "targets": [
183
+ { "id": "bot1-art", "type": "illustration", "mode": "ranking", "rankingDate": "YESTERDAY" },
184
+ { "id": "bot1-novel", "type": "novel", "mode": "ranking", "rankingDate": "YESTERDAY" }
185
+ ]
186
+ }
187
+ ```
188
+
189
+ 旧的单 `scheduler` 配置继续兼容。`pixiv`、`network`、`storage` 涉及长生命周期
190
+ 连接或路径,修改后需要重启;`schedules`、`targets`、`delivery`、`download` 可以
191
+ 热重载。完整双 Bot 缓存投递模板见
192
+ [`config/fly-two-bots.example.json`](config/fly-two-bots.example.json)。
193
+
194
+ ### 投递事务发件箱
195
+
158
196
  投递是**事务发件箱(SQLite outbox)**式的,at-least-once 执行、effectively-once
159
197
  可见效果:每个外部副作用(一次内容投递、一条通知)在 `outbox` 表落一行,带幂等键和
160
198
  行级租约,独立 worker 立即泵送、指数退避重试(默认 5 分钟起步、最长 6 小时),超过
@@ -177,15 +215,9 @@ pixivflow outbox cancel <id> # 只取消尚未执行的 row
177
215
  ```
178
216
 
179
217
  `run-once` 会重新执行下载计划,不等价于 outbox replay;不要手改 SQLite 的
180
- `next_attempt_at`。
181
-
182
- 运维通知可直接把 `notificationUrl` 指向 [Apprise API](docs/APPRISE.md),由 Apprise 统一发送
183
- Email、Telegram、Discord、ntfy 等渠道;PixivFlow 不实现这些通知协议。
184
-
185
- - 上面的示例是投稿给一个 HTTP 接口:把 `/gen_token` 得到的 `tp_...` 放进 `TG_SUBMIT_TOKEN` 即可(这是示例服务自己的鉴权方式)。
186
- - 同一目标也可指向 [telepress](https://github.com/redtidev1918/telepress) 的 `/publish/gallery`,把插画自动发布成 Telegra.ph 相册,见 [CONFIG.md](docs/CONFIG.md) 的「Telegraph(telegra.ph)相册上传」。
187
-
188
- 不想手写配置?运行交互式向导 `pixivflow setup` 一步步生成。
218
+ `next_attempt_at`。长期运行环境的体检与收敛用 `pixivflow doctor`(卡住的 slot/outbox
219
+ 租约、pending 投递、dead 行,`--repair` 收敛)与 `pixivflow reconcile`(把下游已确认的
220
+ 历史重复登记进投递账本,默认 dry-run)。
189
221
 
190
222
  ## 常用命令
191
223
 
@@ -195,6 +227,7 @@ Email、Telegram、Discord、ntfy 等渠道;PixivFlow 不实现这些通知协
195
227
  | `pixivflow download --url <url>` | 通过 URL 直接下载 |
196
228
  | `pixivflow random` | 随机下载热门作品 |
197
229
  | `pixivflow scheduler` | 启动定时任务 |
230
+ | `pixivflow webui` | 启动 WebUI |
198
231
  | `pixivflow config` | 配置管理(查看 / 编辑 / 备份 / 恢复) |
199
232
  | `pixivflow status` | 下载统计与最近记录 |
200
233
  | `pixivflow health` | 健康检查:配置、目录可写性、连通性 |
@@ -210,6 +243,15 @@ Email、Telegram、Discord、ntfy 等渠道;PixivFlow 不实现这些通知协
210
243
 
211
244
  其他用法见 [USAGE.md](docs/USAGE.md);从 v1 升级到 v2 见 [迁移指南](docs/MIGRATION.md)。
212
245
 
246
+ ## 部署
247
+
248
+ - **Docker / 服务器长期挂机**:见 [DOCKER.md](docs/DOCKER.md)。
249
+ - **Android / Termux**:见 [TERMUX_INSTALL.md](docs/TERMUX_INSTALL.md)。
250
+ - **与 TelePost 组合部署**:PixivFlow 与 TelePost 都可以独立使用;只有当你希望把两者
251
+ 组合成一套完整工作流时,才需要
252
+ [pixivflow-telepost-deploy](https://github.com/redtidev1918/pixivflow-telepost-deploy)
253
+ 这个部署与运维套件。
254
+
213
255
  ## 文档
214
256
 
215
257
  完整教程站点:<https://redtidev1918.github.io/PixivFlow/>
@@ -230,6 +272,17 @@ Email、Telegram、Discord、ntfy 等渠道;PixivFlow 不实现这些通知协
230
272
 
231
273
  English version: [README.en.md](README.en.md).
232
274
 
275
+ ## 相关项目
276
+
277
+ PixivFlow 可以完全独立使用。下面是同一作者生态里与它相关的项目,以及各自负责什么:
278
+
279
+ | 项目 | 是什么 | 什么时候需要 |
280
+ | --- | --- | --- |
281
+ | [TelePost](https://github.com/redtidev1918/TelePost) | Telegram 频道投稿、审核与自动化发布平台 | 想把下载结果投进 Telegram 频道、先人工审核再发布时,把它配成 delivery 下游即可。这只是可选组合,PixivFlow 不依赖它 |
282
+ | [pixivflow-telepost-deploy](https://github.com/redtidev1918/pixivflow-telepost-deploy) | PixivFlow + TelePost 的部署与运维套件(Docker / VPS / 云平台) | 想一次性把上面两个项目部署并运维起来时。只跑 PixivFlow 不需要它 |
283
+ | [pixivflow-webui](https://github.com/redtidev1918/pixivflow-webui) | PixivFlow 的 WebUI 前端 | 想用图形界面管理下载与计划 |
284
+ | [pixiv-token-getter](https://github.com/redtidev1918/pixiv-token-getter) | PKCE OAuth 登录库与 CLI(`ptg`) | PixivFlow 的登录依赖;也可以单独用于获取 Pixiv token |
285
+
233
286
  ## 问题反馈
234
287
 
235
288
  Bug 与功能建议请提交到