pixivflow 3.2.0 → 3.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +26 -1
- package/dist/commands/SchedulerCommand.js +31 -51
- package/dist/commands/scheduler-runtime.js +47 -7
- package/dist/config/defaults.d.ts +2 -0
- package/dist/config/defaults.js +6 -0
- package/dist/config/types.d.ts +20 -0
- package/dist/config/validation.js +11 -0
- package/dist/delivery/DeliveryDispatcher.d.ts +14 -0
- package/dist/delivery/DeliveryDispatcher.js +14 -0
- package/dist/delivery/EventCallbackDelivery.d.ts +54 -0
- package/dist/delivery/EventCallbackDelivery.js +89 -0
- package/dist/delivery/OutboxWorker.d.ts +11 -0
- package/dist/delivery/OutboxWorker.js +51 -1
- package/dist/package.json +1 -1
- package/dist/scheduler/CandidateSearchParams.d.ts +76 -0
- package/dist/scheduler/CandidateSearchParams.js +94 -0
- package/dist/scheduler/JobCancellation.d.ts +36 -0
- package/dist/scheduler/JobCancellation.js +73 -0
- package/dist/scheduler/JobEventStream.d.ts +94 -0
- package/dist/scheduler/JobEventStream.js +251 -0
- package/dist/scheduler/JobFacade.d.ts +298 -0
- package/dist/scheduler/JobFacade.js +622 -0
- package/dist/scheduler/JobProjection.d.ts +67 -0
- package/dist/scheduler/JobProjection.js +32 -0
- package/dist/scheduler/JobView.d.ts +34 -0
- package/dist/scheduler/JobView.js +82 -0
- package/dist/scheduler/ManualJobAdmission.d.ts +126 -0
- package/dist/scheduler/ManualJobAdmission.js +310 -0
- package/dist/scheduler/ManualJobService.d.ts +57 -0
- package/dist/scheduler/ManualJobService.js +91 -0
- package/dist/scheduler/ManualRefetchAdapter.d.ts +26 -0
- package/dist/scheduler/ManualRefetchAdapter.js +41 -0
- package/dist/scheduler/MultiScheduleManager.d.ts +17 -0
- package/dist/scheduler/MultiScheduleManager.js +68 -6
- package/dist/scheduler/ProtocolErrors.d.ts +68 -0
- package/dist/scheduler/ProtocolErrors.js +94 -0
- package/dist/scheduler/ScheduleTriggerServer.d.ts +56 -7
- package/dist/scheduler/ScheduleTriggerServer.js +166 -1
- package/dist/scheduler/SlotBusinessStatus.d.ts +5 -1
- package/dist/scheduler/SlotBusinessStatus.js +6 -0
- package/dist/scheduler/SlotCoordinator.d.ts +49 -5
- package/dist/scheduler/SlotCoordinator.js +89 -22
- package/dist/scheduler/StallSweep.d.ts +65 -0
- package/dist/scheduler/StallSweep.js +105 -0
- package/dist/scheduler/TargetOutcome.d.ts +39 -1
- package/dist/scheduler/TargetOutcome.js +51 -1
- package/dist/scheduler/ledger-time.d.ts +23 -0
- package/dist/scheduler/ledger-time.js +37 -0
- package/dist/storage/DatabaseMigration.js +48 -0
- package/dist/storage/repositories/DeliveryRepository.d.ts +6 -0
- package/dist/storage/repositories/DeliveryRepository.js +13 -0
- package/dist/storage/repositories/OutboxRepository.d.ts +73 -1
- package/dist/storage/repositories/OutboxRepository.js +142 -0
- package/dist/storage/repositories/SlotRepository.d.ts +34 -0
- package/dist/storage/repositories/SlotRepository.js +61 -2
- package/dist/version.js +1 -1
- package/dist/webui/package.json +1 -1
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -257,6 +257,31 @@ pixivflow outbox cancel <id> # 只取消尚未执行的 row
|
|
|
257
257
|
租约、pending 投递、dead 行,`--repair` 收敛)与 `pixivflow reconcile`
|
|
258
258
|
(把下游已确认的历史重复登记进投递账本,默认 dry-run)。
|
|
259
259
|
|
|
260
|
+
### 面向编排方的 Job API(Workflow Protocol v1)
|
|
261
|
+
|
|
262
|
+
PixivFlow 只做「内容采集与处理引擎」:发现、筛选、下载、元数据与媒体处理、候选生成。
|
|
263
|
+
**审核 / 替换 / 队列 / 发布等业务语义不在这里实现**,由编排方(如 TelePost)负责。
|
|
264
|
+
两者之间通过一套稳定的 **Workflow Protocol v1** 用通用 Job 面交互,编排方不互调
|
|
265
|
+
内部接口、也不往请求里塞业务字段。scheduler 对外暴露:
|
|
266
|
+
|
|
267
|
+
| 端点 | 作用 |
|
|
268
|
+
| --- | --- |
|
|
269
|
+
| `GET /capabilities` | 能力发现:如实声明 `protocol_version` 与 `features`(`idempotency` / `cancel` / `events` / ...),能力声明必须诚实,没声明就是没有 |
|
|
270
|
+
| `POST /jobs` | 提交一个采集任务(如 `candidate_search`):`idempotency_key` 幂等,携带消费者声明的 `callback_url` 时事件会推送到该地址 |
|
|
271
|
+
| `GET /jobs` · `GET /jobs/{job_id}` | 按幂等键或按 id 查作业状态(含 `status`、`created_at`、`heartbeat_at`、`deadline_at`、`result`/`error`) |
|
|
272
|
+
| `GET /jobs/{job_id}/events` · `POST /jobs/{job_id}/events/ack` | 事件流与对账:`?after=` 续拉、`unacked` 计数、单调游标 ack;回调只是加速通道,漏送也能从这里补齐 |
|
|
273
|
+
|
|
274
|
+
**作业生命周期可信、可查,不再永久静默。** 每个作业都是持久化的、带完整状态机
|
|
275
|
+
(`queued → running → succeeded|failed|expired|cancelled`)、30 秒心跳、活性预算、
|
|
276
|
+
启动恢复与终态事件;终态必有 `result` 或 `error`。编排方可以随时查询作业、订阅事件、
|
|
277
|
+
对账补拉——再也回不到「点了重抓之后什么反应都没有」的那种状态。写入方与读取方都受
|
|
278
|
+
[外包层](docs/architecture/delivery-runtime.md) 与仓库内 `protocol/v1/`(schema、夹具
|
|
279
|
+
与验收清单)约束;规范与跨仓部署见
|
|
280
|
+
[pixivflow-telepost-deploy](https://github.com/redtidev1918/pixivflow-telepost-deploy)
|
|
281
|
+
的 [Workflow Protocol 文档](https://github.com/redtidev1918/pixivflow-telepost-deploy/blob/main/docs/architecture/workflow-protocol.md)。
|
|
282
|
+
旧的 `/internal/targets/:targetId/refetch` 仍作为字节兼容 shim 可用,与 Job 面共享同一
|
|
283
|
+
身份空间,但新集成请走 Job 面 / `POST /jobs`。
|
|
284
|
+
|
|
260
285
|
## 常用命令
|
|
261
286
|
|
|
262
287
|
| 命令 | 说明 |
|
|
@@ -316,7 +341,7 @@ PixivFlow 可以完全独立使用。下面是同一作者生态里与它相关
|
|
|
316
341
|
|
|
317
342
|
| 项目 | 是什么 | 什么时候需要 |
|
|
318
343
|
| --- | --- | --- |
|
|
319
|
-
| [TelePost](https://github.com/redtidev1918/TelePost) | Telegram 频道投稿、审核与自动化发布平台 |
|
|
344
|
+
| [TelePost](https://github.com/redtidev1918/TelePost) | Telegram 频道投稿、审核与自动化发布平台 | 作为投递下游接收下载结果,或作为**编排方**通过 [Workflow Protocol v1](#面向编排方的-job-apiworkflow-protocol-v1) 驱动 PixivFlow 提交候选查找作业(审核重抓即走此路)。这只是可选组合,PixivFlow 不依赖它 |
|
|
320
345
|
| [pixivflow-telepost-deploy](https://github.com/redtidev1918/pixivflow-telepost-deploy) | PixivFlow + TelePost 的部署与运维套件(Docker / VPS / 云平台) | 想一次性把上面两个项目部署并运维起来时。只跑 PixivFlow 不需要它 |
|
|
321
346
|
| [pixivflow-webui](https://github.com/redtidev1918/pixivflow-webui) | PixivFlow 的 WebUI 前端 | 想用图形界面管理下载与计划 |
|
|
322
347
|
| [pixivflow-desktop](https://github.com/redtidev1918/pixivflow-desktop) | PixivFlow 的官方桌面客户端(macOS / Windows / Linux),内置本仓库运行时与 WebUI | 想要双击即用的原生应用、不想自己装 Node 或起服务时。业务逻辑仍在本仓库,桌面端只负责启动、守护与打包 |
|
|
@@ -10,6 +10,9 @@ const config_1 = require("../config");
|
|
|
10
10
|
const MultiScheduleManager_1 = require("../scheduler/MultiScheduleManager");
|
|
11
11
|
const ScheduleTriggerServer_1 = require("../scheduler/ScheduleTriggerServer");
|
|
12
12
|
const SlotCoordinator_1 = require("../scheduler/SlotCoordinator");
|
|
13
|
+
const ManualJobAdmission_1 = require("../scheduler/ManualJobAdmission");
|
|
14
|
+
const ManualJobService_1 = require("../scheduler/ManualJobService");
|
|
15
|
+
const ManualRefetchAdapter_1 = require("../scheduler/ManualRefetchAdapter");
|
|
13
16
|
const schedules_1 = require("../scheduler/schedules");
|
|
14
17
|
const targetRoutes_1 = require("../delivery/targetRoutes");
|
|
15
18
|
const scheduler_runtime_1 = require("./scheduler-runtime");
|
|
@@ -82,6 +85,22 @@ class SchedulerCommand extends Command_1.BaseCommand {
|
|
|
82
85
|
const coordinator = new SlotCoordinator_1.SlotCoordinator(runtime.database);
|
|
83
86
|
const resolveConfig = () => manager['activeConfig'] ?? runtime.config;
|
|
84
87
|
const findPlan = (cfg, scheduleId) => cfg.schedules?.find((s) => s.id === scheduleId && s.enabled !== false);
|
|
88
|
+
// The ONE admission path for consumer-initiated candidate search. Both
|
|
89
|
+
// the legacy refetch endpoint (below) and the generic protocol job
|
|
90
|
+
// surface (`jobs`, below) translate their request into this call, so
|
|
91
|
+
// they share one identity space, one target resolution and one durable
|
|
92
|
+
// work item per idempotency key.
|
|
93
|
+
const admission = new ManualJobAdmission_1.ManualJobAdmission({
|
|
94
|
+
database: runtime.database,
|
|
95
|
+
coordinator,
|
|
96
|
+
config: resolveConfig,
|
|
97
|
+
admit: (planId, slot, targetId) => manager.triggerSchedule(planId, { slot, onlyTarget: targetId, triggerSource: 'manual' }),
|
|
98
|
+
});
|
|
99
|
+
const jobService = new ManualJobService_1.ManualJobService({
|
|
100
|
+
database: runtime.database,
|
|
101
|
+
config: resolveConfig,
|
|
102
|
+
admission,
|
|
103
|
+
});
|
|
85
104
|
// Durable dispatch. The trigger endpoint records the occurrence FIRST,
|
|
86
105
|
// then hands it to the shared scheduler and answers immediately. It must
|
|
87
106
|
// never await the download itself: a 10-40 minute run outlives the
|
|
@@ -159,57 +178,18 @@ class SchedulerCommand extends Command_1.BaseCommand {
|
|
|
159
178
|
};
|
|
160
179
|
},
|
|
161
180
|
drainOutbox: () => runtime.drainOutbox(),
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
throw new Error('refetch outcome endpoint not configured');
|
|
175
|
-
}
|
|
176
|
-
if ((target.delivery?.fields?.refetch_request_id ?? delivery.fields?.refetch_request_id) !== '{{refetchRequestId}}') {
|
|
177
|
-
throw new Error('refetch_request_id delivery field not configured');
|
|
178
|
-
}
|
|
179
|
-
const now = new Date();
|
|
180
|
-
const date = new Intl.DateTimeFormat('en-CA', {
|
|
181
|
-
timeZone: plan.timezone ?? 'UTC', year: 'numeric', month: '2-digit', day: '2-digit',
|
|
182
|
-
}).format(now);
|
|
183
|
-
const slot = {
|
|
184
|
-
slotId: `${plan.id}@manual-${requestId.toLowerCase()}`,
|
|
185
|
-
scheduleId: plan.id,
|
|
186
|
-
occurrenceAt: now.getTime(),
|
|
187
|
-
occurrenceDate: date,
|
|
188
|
-
occurrenceLabel: 'manual',
|
|
189
|
-
timezone: plan.timezone ?? 'UTC',
|
|
190
|
-
triggerSource: 'manual',
|
|
191
|
-
slotName: '审核群重抓',
|
|
192
|
-
slotDate: date,
|
|
193
|
-
manualRequestId: requestId,
|
|
194
|
-
correlationId: correlationId || undefined,
|
|
195
|
-
};
|
|
196
|
-
const existing = runtime.database.slots.getSlot(slot.slotId);
|
|
197
|
-
if (existing && (existing.scheduleId !== plan.id || existing.targetIds.length !== 1 || existing.targetIds[0] !== targetId)) {
|
|
198
|
-
throw new Error('ambiguous target');
|
|
199
|
-
}
|
|
200
|
-
const prepared = coordinator.prepare(slot, plan, [target]);
|
|
201
|
-
if (prepared.alreadyCompleted)
|
|
202
|
-
return { slotId: slot.slotId, disposition: 'already_completed' };
|
|
203
|
-
const started = manager.triggerSchedule(plan.id, { slot, onlyTarget: targetId, triggerSource: 'manual' });
|
|
204
|
-
return { slotId: slot.slotId, disposition: started ? 'accepted' : 'queued' };
|
|
205
|
-
},
|
|
206
|
-
refetchStatus: (targetId, requestId) => {
|
|
207
|
-
const slot = runtime.database.slots.findManualSlot(requestId, targetId);
|
|
208
|
-
const cell = slot && runtime.database.slots.getCell(slot.id, targetId);
|
|
209
|
-
return slot && cell
|
|
210
|
-
? { requestId, slotId: slot.id, state: cell.status, slotStatus: slot.status }
|
|
211
|
-
: null;
|
|
212
|
-
},
|
|
181
|
+
/**
|
|
182
|
+
* The generic Workflow Protocol v1 job surface (`/capabilities`,
|
|
183
|
+
* `/jobs`). It answers from the same ledger as the legacy endpoints
|
|
184
|
+
* and submits through the same admission path, so a job submitted
|
|
185
|
+
* here IS the work item a legacy refetch would have created.
|
|
186
|
+
*/
|
|
187
|
+
jobs: jobService,
|
|
188
|
+
// Legacy adapters (protocol §3.1): the historic refetch shape over
|
|
189
|
+
// the same shared admission, so `requestId` and `idempotency_key`
|
|
190
|
+
// are one identity space.
|
|
191
|
+
refetch: (0, ManualRefetchAdapter_1.legacyRefetchSubmit)(admission),
|
|
192
|
+
refetchStatus: (0, ManualRefetchAdapter_1.legacyRefetchStatus)(runtime.database),
|
|
213
193
|
/**
|
|
214
194
|
* Manual recovery of a FAILED target (§manual-recovery). Distinct
|
|
215
195
|
* from refetch: there is no review chain to replace and no
|
|
@@ -61,12 +61,14 @@ const token_maintenance_1 = require("../utils/token-maintenance");
|
|
|
61
61
|
const schedules_1 = require("../scheduler/schedules");
|
|
62
62
|
const SlotCoordinator_1 = require("../scheduler/SlotCoordinator");
|
|
63
63
|
const RecoveryPolicy_1 = require("../scheduler/RecoveryPolicy");
|
|
64
|
+
const CandidateSearchParams_1 = require("../scheduler/CandidateSearchParams");
|
|
64
65
|
const targetRoutes_1 = require("../delivery/targetRoutes");
|
|
65
66
|
const DeliveryLedgerPort_1 = require("../delivery/DeliveryLedgerPort");
|
|
66
67
|
const OutboxWorker_1 = require("../delivery/OutboxWorker");
|
|
67
68
|
const settleDeliveryTerminal_1 = require("../delivery/settleDeliveryTerminal");
|
|
68
69
|
const LegacyOutboxMigration_1 = require("../delivery/LegacyOutboxMigration");
|
|
69
70
|
const NotificationPolicy_1 = require("../notification/NotificationPolicy");
|
|
71
|
+
const JobEventStream_1 = require("../scheduler/JobEventStream");
|
|
70
72
|
const node_crypto_1 = require("node:crypto");
|
|
71
73
|
const placeholders_1 = require("../config/placeholders");
|
|
72
74
|
const logger_1 = require("../logger");
|
|
@@ -293,8 +295,20 @@ async function createSchedulerRuntime(configPathArg) {
|
|
|
293
295
|
// the long-running scheduler daemon; run-once drains explicitly before exit.
|
|
294
296
|
const deliveryDispatcher = new DeliveryDispatcher_1.DeliveryDispatcher(config.delivery, buildProxyUrl(config.network));
|
|
295
297
|
const notificationPolicy = new NotificationPolicy_1.NotificationPolicy(database, config);
|
|
298
|
+
// Consumer-facing job events (§events). The stream is a projection of the same
|
|
299
|
+
// `delivery_events` log; this sweep repairs a declared `callback_url` whose
|
|
300
|
+
// terminal event was lost with the process, exactly like
|
|
301
|
+
// reconcileScheduleSummaries repairs a terminal slot that was never notified.
|
|
302
|
+
const jobEvents = new JobEventStream_1.JobEventStream({
|
|
303
|
+
database,
|
|
304
|
+
config: () => config,
|
|
305
|
+
now: () => Date.now(),
|
|
306
|
+
});
|
|
296
307
|
const outboxWorker = new OutboxWorker_1.OutboxWorker(database, deliveryDispatcher, {
|
|
297
|
-
beforeDrain: () =>
|
|
308
|
+
beforeDrain: () => {
|
|
309
|
+
notificationPolicy.reconcileScheduleSummaries();
|
|
310
|
+
jobEvents.reconcileOutstanding();
|
|
311
|
+
},
|
|
298
312
|
retryBaseMs: config.delivery?.outboxRetryBaseMs,
|
|
299
313
|
retryMaxMs: config.delivery?.outboxRetryMaxMs,
|
|
300
314
|
// A confirmed ACK settles the owning Slot cell (submitted / duplicate /
|
|
@@ -342,6 +356,30 @@ async function createSchedulerRuntime(configPathArg) {
|
|
|
342
356
|
targets: targets.map((t) => t.id),
|
|
343
357
|
});
|
|
344
358
|
}
|
|
359
|
+
// Occurrence-scoped RETRIEVAL view of a generic `candidate_search` job (§6).
|
|
360
|
+
// Same pattern as the recovery policy above: a pure map over THIS
|
|
361
|
+
// occurrence's target snapshot, never a write to the global config. A slot
|
|
362
|
+
// with no stored view (every scheduled occurrence, and every slot written
|
|
363
|
+
// before the column existed) keeps the configured behaviour untouched.
|
|
364
|
+
const requestedSearch = (0, CandidateSearchParams_1.parseCandidateSearchParamsJson)(providedSlot?.paramsJson);
|
|
365
|
+
if (requestedSearch) {
|
|
366
|
+
targets = targets.map((target) => (0, CandidateSearchParams_1.applyCandidateSearchParams)(target, requestedSearch));
|
|
367
|
+
logger_1.logger.info('Generic job retrieval view applied to this occurrence', {
|
|
368
|
+
scheduleId: schedule.id,
|
|
369
|
+
slot: providedSlot?.slotId,
|
|
370
|
+
targets: targets.map((t) => t.id),
|
|
371
|
+
});
|
|
372
|
+
}
|
|
373
|
+
// `constraints.exclude` is a run-level duplicate filter, so it joins the
|
|
374
|
+
// history the runner already refuses to select from instead of inventing a
|
|
375
|
+
// second exclusion mechanism.
|
|
376
|
+
const requestedExclusions = requestedSearch ? (0, CandidateSearchParams_1.excludedWorkIdsFromParams)(requestedSearch) : null;
|
|
377
|
+
const excludedWorkIds = requestedExclusions
|
|
378
|
+
? {
|
|
379
|
+
illustration: [...(options.excludedWorkIds?.illustration ?? []), ...requestedExclusions.illustration],
|
|
380
|
+
novel: [...(options.excludedWorkIds?.novel ?? []), ...requestedExclusions.novel],
|
|
381
|
+
}
|
|
382
|
+
: options.excludedWorkIds;
|
|
345
383
|
if (targets.length === 0) {
|
|
346
384
|
logger_1.logger.warn('Scheduled plan has no selected targets; skipping', {
|
|
347
385
|
scheduleId: schedule.id,
|
|
@@ -362,6 +400,9 @@ async function createSchedulerRuntime(configPathArg) {
|
|
|
362
400
|
* overwrite the terminal `failed` record the abandon path wrote).
|
|
363
401
|
*/
|
|
364
402
|
let slotAbandoned = false;
|
|
403
|
+
// This run's lease identity. It is passed to `finish` so a rollup can tell
|
|
404
|
+
// "my own live lease" from "another worker is on this slot right now".
|
|
405
|
+
const runOwner = `run-${process.pid}-${(0, node_crypto_1.randomUUID)().slice(0, 8)}`;
|
|
365
406
|
if (!adhoc) {
|
|
366
407
|
if (providedSlot) {
|
|
367
408
|
slotCtx = providedSlot;
|
|
@@ -380,7 +421,6 @@ async function createSchedulerRuntime(configPathArg) {
|
|
|
380
421
|
slotCtx = resolved.context;
|
|
381
422
|
}
|
|
382
423
|
const activeSlot = slotCtx;
|
|
383
|
-
const runOwner = `run-${process.pid}-${(0, node_crypto_1.randomUUID)().slice(0, 8)}`;
|
|
384
424
|
const prepared = coordinator.prepare(slotCtx, schedule, targets);
|
|
385
425
|
if (prepared.alreadyCompleted && !onlyTarget) {
|
|
386
426
|
logger_1.logger.info('Slot already terminal; nothing to do', {
|
|
@@ -474,7 +514,7 @@ async function createSchedulerRuntime(configPathArg) {
|
|
|
474
514
|
let runTargets = selected.map((p) => p.target);
|
|
475
515
|
if (slotCtx && runTargets.length === 0) {
|
|
476
516
|
logger_1.logger.info('All slot cells already complete', { slot: slotCtx.slotId });
|
|
477
|
-
coordinator.finish(slotCtx, schedule, targets);
|
|
517
|
+
coordinator.finish(slotCtx, schedule, targets, { leaseOwner: runOwner });
|
|
478
518
|
for (const target of targets)
|
|
479
519
|
if (target.id)
|
|
480
520
|
notificationPolicy.noteTerminalRefetchCell(slotCtx.slotId, target.id);
|
|
@@ -540,8 +580,8 @@ async function createSchedulerRuntime(configPathArg) {
|
|
|
540
580
|
const manager = new DownloadManager_1.DownloadManager(scoped, pixivClient, database, fileService);
|
|
541
581
|
if (targetExecutionContexts)
|
|
542
582
|
manager.setTargetExecutionContexts(targetExecutionContexts);
|
|
543
|
-
if (
|
|
544
|
-
manager.setProcessedWorkIds(
|
|
583
|
+
if (excludedWorkIds)
|
|
584
|
+
manager.setProcessedWorkIds(excludedWorkIds);
|
|
545
585
|
manager.setTargetOutcomeHook(outcomeHook);
|
|
546
586
|
if (scheduleSlot) {
|
|
547
587
|
manager.slotContext = {
|
|
@@ -670,7 +710,7 @@ async function createSchedulerRuntime(configPathArg) {
|
|
|
670
710
|
// - the process is going away (`shutdown`). Deliberately leave the Slot
|
|
671
711
|
// non-terminal so recovery resumes the same occurrence after restart.
|
|
672
712
|
if (slotCtx && shouldTerminaliseAbortedSlot(activeAbortOrigin, slotAbandoned)) {
|
|
673
|
-
coordinator.finish(slotCtx, schedule, targets);
|
|
713
|
+
coordinator.finish(slotCtx, schedule, targets, { leaseOwner: runOwner });
|
|
674
714
|
for (const target of targets)
|
|
675
715
|
if (target.id)
|
|
676
716
|
notificationPolicy.noteTerminalRefetchCell(slotCtx.slotId, target.id);
|
|
@@ -691,7 +731,7 @@ async function createSchedulerRuntime(configPathArg) {
|
|
|
691
731
|
}
|
|
692
732
|
const duration = Math.round((Date.now() - startTime) / 1000);
|
|
693
733
|
if (slotCtx && !slotAbandoned) {
|
|
694
|
-
const summary = coordinator.finish(slotCtx, schedule, targets);
|
|
734
|
+
const summary = coordinator.finish(slotCtx, schedule, targets, { leaseOwner: runOwner });
|
|
695
735
|
for (const target of targets)
|
|
696
736
|
if (target.id)
|
|
697
737
|
notificationPolicy.noteTerminalRefetchCell(slotCtx.slotId, target.id);
|
|
@@ -24,6 +24,8 @@ export declare const DEFAULT_CONFIG: {
|
|
|
24
24
|
readonly watchConfig: true;
|
|
25
25
|
readonly reloadDebounceMs: 500;
|
|
26
26
|
readonly queueLimit: 8;
|
|
27
|
+
readonly queuedTimeoutMs: number;
|
|
28
|
+
readonly stallTimeoutMs: number;
|
|
27
29
|
readonly trigger: {
|
|
28
30
|
readonly port: 8090;
|
|
29
31
|
readonly host: "0.0.0.0";
|
package/dist/config/defaults.js
CHANGED
|
@@ -27,6 +27,12 @@ exports.DEFAULT_CONFIG = {
|
|
|
27
27
|
watchConfig: true,
|
|
28
28
|
reloadDebounceMs: 500,
|
|
29
29
|
queueLimit: 8,
|
|
30
|
+
// Liveness budgets (§liveness). Generous on purpose: production runs take
|
|
31
|
+
// 10-40 minutes and are sometimes queued hours behind a busy account, so a
|
|
32
|
+
// slot is only terminalised when its LEASE is dead and it stopped
|
|
33
|
+
// progressing — never merely because it is slow.
|
|
34
|
+
queuedTimeoutMs: 30 * 60 * 1000,
|
|
35
|
+
stallTimeoutMs: 15 * 60 * 1000,
|
|
30
36
|
trigger: {
|
|
31
37
|
port: 8090,
|
|
32
38
|
host: '0.0.0.0',
|
package/dist/config/types.d.ts
CHANGED
|
@@ -551,6 +551,26 @@ export interface SchedulerRuntimeConfig {
|
|
|
551
551
|
* One pending run per plan is retained; extra ticks are skipped.
|
|
552
552
|
*/
|
|
553
553
|
queueLimit?: number;
|
|
554
|
+
/**
|
|
555
|
+
* Liveness budget for admitted work that never started (§liveness). A Slot
|
|
556
|
+
* that is still `pending` with no live lease after this long is terminalised
|
|
557
|
+
* as `queued_too_long` (its cells fail with that reason) instead of being
|
|
558
|
+
* re-dispatched forever and reporting an eternal "unfinished" job. Values
|
|
559
|
+
* below one minute are ignored (the sweep then uses its default) and warned
|
|
560
|
+
* about at config validation; nothing here is fatal.
|
|
561
|
+
* Default: 1800000ms (30 minutes).
|
|
562
|
+
*/
|
|
563
|
+
queuedTimeoutMs?: number;
|
|
564
|
+
/**
|
|
565
|
+
* Liveness budget for a claimed run that stopped progressing (§liveness). A
|
|
566
|
+
* `running` Slot whose lease expired AND whose last heartbeat is older than
|
|
567
|
+
* this is terminalised as `stalled_no_heartbeat`. A LIVE lease is never
|
|
568
|
+
* eligible, so a genuinely long search is never killed. Values below one
|
|
569
|
+
* minute are ignored (the sweep then uses its default) and warned about at
|
|
570
|
+
* config validation; nothing here is fatal.
|
|
571
|
+
* Default: 900000ms (15 minutes).
|
|
572
|
+
*/
|
|
573
|
+
stallTimeoutMs?: number;
|
|
554
574
|
/**
|
|
555
575
|
* External-mode run-to-completion lifecycle. When true, the daemon exits the
|
|
556
576
|
* process once its OWN durable ledger says there is nothing left to do, so a
|
|
@@ -639,6 +639,17 @@ function validateConfig(config, location, databasePath) {
|
|
|
639
639
|
if (queueLimit !== undefined && (!Number.isInteger(queueLimit) || queueLimit < 0 || queueLimit > 100)) {
|
|
640
640
|
errors.push('schedulerRuntime.queueLimit: Must be an integer between 0 and 100');
|
|
641
641
|
}
|
|
642
|
+
// Liveness budgets (§liveness): a WARNING, never an error. A bad value must
|
|
643
|
+
// not stop the daemon from starting; the sweep clamps to its own default so
|
|
644
|
+
// the behaviour stays safe either way.
|
|
645
|
+
for (const key of ['queuedTimeoutMs', 'stallTimeoutMs']) {
|
|
646
|
+
const value = rt[key];
|
|
647
|
+
if (value === undefined)
|
|
648
|
+
continue;
|
|
649
|
+
if (typeof value !== 'number' || !Number.isInteger(value) || value < 60000) {
|
|
650
|
+
warnings.push(`schedulerRuntime.${key}: Should be an integer of at least 60000 (milliseconds); the liveness sweep uses its default instead`);
|
|
651
|
+
}
|
|
652
|
+
}
|
|
642
653
|
if (rt.exitWhenIdle !== undefined && typeof rt.exitWhenIdle !== 'boolean') {
|
|
643
654
|
errors.push('schedulerRuntime.exitWhenIdle: Must be a boolean');
|
|
644
655
|
}
|
|
@@ -1,10 +1,12 @@
|
|
|
1
1
|
import { DeliveryConfig } from '../config';
|
|
2
|
+
import { EventCallbackPayload } from './EventCallbackDelivery';
|
|
2
3
|
import { ReadinessProbeResult } from './HttpMultipartDelivery';
|
|
3
4
|
import { DeliveryNotificationRequest, DeliveryRequest, DeliveryResult } from './types';
|
|
4
5
|
/** Resolves named delivery targets without coupling the outbox to a provider. */
|
|
5
6
|
export declare class DeliveryDispatcher {
|
|
6
7
|
private readonly config;
|
|
7
8
|
private readonly proxyUrl?;
|
|
9
|
+
private readonly eventCallbacks;
|
|
8
10
|
constructor(config: DeliveryConfig | undefined, proxyUrl?: string | undefined);
|
|
9
11
|
hasTarget(name: string): boolean;
|
|
10
12
|
isReady(name: string): Promise<boolean>;
|
|
@@ -12,5 +14,17 @@ export declare class DeliveryDispatcher {
|
|
|
12
14
|
readinessProbe(name: string): Promise<ReadinessProbeResult>;
|
|
13
15
|
deliver(name: string, request: DeliveryRequest): Promise<DeliveryResult>;
|
|
14
16
|
notify(name: string, request: DeliveryNotificationRequest): Promise<DeliveryResult>;
|
|
17
|
+
/**
|
|
18
|
+
* Deliver one `$defs/Event` to a Task's declared `callback_url`.
|
|
19
|
+
*
|
|
20
|
+
* This does NOT go through a named delivery target: the endpoint is chosen by
|
|
21
|
+
* the consumer per Task, not by our config. It is still a delivery in every
|
|
22
|
+
* other sense — the caller is the outbox worker, so retry, backoff,
|
|
23
|
+
* dead-lettering and the idempotency index all apply unchanged.
|
|
24
|
+
*/
|
|
25
|
+
deliverEventCallback(url: string, request: {
|
|
26
|
+
payload: EventCallbackPayload;
|
|
27
|
+
idempotencyKey: string;
|
|
28
|
+
}): Promise<number>;
|
|
15
29
|
}
|
|
16
30
|
//# sourceMappingURL=DeliveryDispatcher.d.ts.map
|
|
@@ -2,6 +2,7 @@
|
|
|
2
2
|
Object.defineProperty(exports, "__esModule", { value: true });
|
|
3
3
|
exports.DeliveryDispatcher = void 0;
|
|
4
4
|
const errors_1 = require("../utils/errors");
|
|
5
|
+
const EventCallbackDelivery_1 = require("./EventCallbackDelivery");
|
|
5
6
|
const HttpMultipartDelivery_1 = require("./HttpMultipartDelivery");
|
|
6
7
|
const TelegramReviewDelivery_1 = require("./TelegramReviewDelivery");
|
|
7
8
|
const WebhookDelivery_1 = require("./WebhookDelivery");
|
|
@@ -9,9 +10,11 @@ const WebhookDelivery_1 = require("./WebhookDelivery");
|
|
|
9
10
|
class DeliveryDispatcher {
|
|
10
11
|
config;
|
|
11
12
|
proxyUrl;
|
|
13
|
+
eventCallbacks;
|
|
12
14
|
constructor(config, proxyUrl) {
|
|
13
15
|
this.config = config;
|
|
14
16
|
this.proxyUrl = proxyUrl;
|
|
17
|
+
this.eventCallbacks = new EventCallbackDelivery_1.EventCallbackDelivery(proxyUrl);
|
|
15
18
|
}
|
|
16
19
|
hasTarget(name) {
|
|
17
20
|
return Boolean(this.config?.targets?.[name]);
|
|
@@ -67,6 +70,17 @@ class DeliveryDispatcher {
|
|
|
67
70
|
}
|
|
68
71
|
return new HttpMultipartDelivery_1.HttpMultipartDelivery(target, this.proxyUrl).notifyOnce(request);
|
|
69
72
|
}
|
|
73
|
+
/**
|
|
74
|
+
* Deliver one `$defs/Event` to a Task's declared `callback_url`.
|
|
75
|
+
*
|
|
76
|
+
* This does NOT go through a named delivery target: the endpoint is chosen by
|
|
77
|
+
* the consumer per Task, not by our config. It is still a delivery in every
|
|
78
|
+
* other sense — the caller is the outbox worker, so retry, backoff,
|
|
79
|
+
* dead-lettering and the idempotency index all apply unchanged.
|
|
80
|
+
*/
|
|
81
|
+
async deliverEventCallback(url, request) {
|
|
82
|
+
return this.eventCallbacks.deliver({ url, payload: request.payload, idempotencyKey: request.idempotencyKey });
|
|
83
|
+
}
|
|
70
84
|
}
|
|
71
85
|
exports.DeliveryDispatcher = DeliveryDispatcher;
|
|
72
86
|
//# sourceMappingURL=DeliveryDispatcher.js.map
|
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Deliver one `$defs/Event` to the `callback_url` an accepted `$defs/Task`
|
|
3
|
+
* declared.
|
|
4
|
+
*
|
|
5
|
+
* At-least-once, exactly like every other delivery in this repo: the caller is
|
|
6
|
+
* the EXISTING `OutboxWorker`, which owns retry, backoff and dead-lettering. So
|
|
7
|
+
* this module performs exactly ONE HTTP POST and reports the outcome as a thrown
|
|
8
|
+
* error whose `status` lets `classifyError` decide whether the mistake was
|
|
9
|
+
* retryable (5xx / 429 -> retry, 4xx -> dead-letter). Transport errors are
|
|
10
|
+
* rethrown untouched, which keeps them retryable by construction.
|
|
11
|
+
*
|
|
12
|
+
* A separate delivery path is deliberately NOT introduced: the callback rides
|
|
13
|
+
* the same outbox, the same `idx_outbox_key` idempotency index and the same
|
|
14
|
+
* `delivery_events` audit log as outcome delivery.
|
|
15
|
+
*/
|
|
16
|
+
export interface EventCallbackPayload {
|
|
17
|
+
/** The `$defs/Event`. The POST body is exactly this object. */
|
|
18
|
+
event: unknown;
|
|
19
|
+
job_id: string;
|
|
20
|
+
/**
|
|
21
|
+
* Correlation hint read by `OutboxWorker.contextCorrelation` so the outbox
|
|
22
|
+
* audit rows for this callback are linked to the job's slot.
|
|
23
|
+
*/
|
|
24
|
+
context?: {
|
|
25
|
+
slotId?: string;
|
|
26
|
+
};
|
|
27
|
+
}
|
|
28
|
+
export declare const EVENT_CALLBACK_TIMEOUT_MS = 30000;
|
|
29
|
+
/**
|
|
30
|
+
* A callback endpoint answered with a non-2xx status. `status` is carried as an
|
|
31
|
+
* own enumerable property because `Error.message` is not a machine-readable
|
|
32
|
+
* contract — the outbox hands it to `classifyError(error, status)`.
|
|
33
|
+
*/
|
|
34
|
+
export declare class EventCallbackError extends Error {
|
|
35
|
+
readonly status: number;
|
|
36
|
+
constructor(message: string, status: number);
|
|
37
|
+
}
|
|
38
|
+
export declare class EventCallbackDelivery {
|
|
39
|
+
private readonly proxyUrl?;
|
|
40
|
+
private readonly timeoutMs;
|
|
41
|
+
private readonly dispatcher?;
|
|
42
|
+
constructor(proxyUrl?: string | undefined, timeoutMs?: number);
|
|
43
|
+
/**
|
|
44
|
+
* One POST of one event. Resolves with the HTTP status on success; throws
|
|
45
|
+
* `EventCallbackError` on a non-2xx status and the raw transport error when
|
|
46
|
+
* the request never completed.
|
|
47
|
+
*/
|
|
48
|
+
deliver(request: {
|
|
49
|
+
url: string;
|
|
50
|
+
payload: EventCallbackPayload;
|
|
51
|
+
idempotencyKey: string;
|
|
52
|
+
}): Promise<number>;
|
|
53
|
+
}
|
|
54
|
+
//# sourceMappingURL=EventCallbackDelivery.d.ts.map
|
|
@@ -0,0 +1,89 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
/**
|
|
3
|
+
* Deliver one `$defs/Event` to the `callback_url` an accepted `$defs/Task`
|
|
4
|
+
* declared.
|
|
5
|
+
*
|
|
6
|
+
* At-least-once, exactly like every other delivery in this repo: the caller is
|
|
7
|
+
* the EXISTING `OutboxWorker`, which owns retry, backoff and dead-lettering. So
|
|
8
|
+
* this module performs exactly ONE HTTP POST and reports the outcome as a thrown
|
|
9
|
+
* error whose `status` lets `classifyError` decide whether the mistake was
|
|
10
|
+
* retryable (5xx / 429 -> retry, 4xx -> dead-letter). Transport errors are
|
|
11
|
+
* rethrown untouched, which keeps them retryable by construction.
|
|
12
|
+
*
|
|
13
|
+
* A separate delivery path is deliberately NOT introduced: the callback rides
|
|
14
|
+
* the same outbox, the same `idx_outbox_key` idempotency index and the same
|
|
15
|
+
* `delivery_events` audit log as outcome delivery.
|
|
16
|
+
*/
|
|
17
|
+
Object.defineProperty(exports, "__esModule", { value: true });
|
|
18
|
+
exports.EventCallbackDelivery = exports.EventCallbackError = exports.EVENT_CALLBACK_TIMEOUT_MS = void 0;
|
|
19
|
+
exports.EVENT_CALLBACK_TIMEOUT_MS = 30_000;
|
|
20
|
+
/**
|
|
21
|
+
* A callback endpoint answered with a non-2xx status. `status` is carried as an
|
|
22
|
+
* own enumerable property because `Error.message` is not a machine-readable
|
|
23
|
+
* contract — the outbox hands it to `classifyError(error, status)`.
|
|
24
|
+
*/
|
|
25
|
+
class EventCallbackError extends Error {
|
|
26
|
+
status;
|
|
27
|
+
constructor(message, status) {
|
|
28
|
+
super(message);
|
|
29
|
+
this.name = 'EventCallbackError';
|
|
30
|
+
this.status = status;
|
|
31
|
+
}
|
|
32
|
+
}
|
|
33
|
+
exports.EventCallbackError = EventCallbackError;
|
|
34
|
+
class EventCallbackDelivery {
|
|
35
|
+
proxyUrl;
|
|
36
|
+
timeoutMs;
|
|
37
|
+
dispatcher;
|
|
38
|
+
constructor(proxyUrl, timeoutMs = exports.EVENT_CALLBACK_TIMEOUT_MS) {
|
|
39
|
+
this.proxyUrl = proxyUrl;
|
|
40
|
+
this.timeoutMs = timeoutMs;
|
|
41
|
+
if (proxyUrl) {
|
|
42
|
+
// Same egress path (and same undici ProxyAgent wiring) as every other
|
|
43
|
+
// outbound delivery.
|
|
44
|
+
const { ProxyAgent } = require('undici');
|
|
45
|
+
this.dispatcher = new ProxyAgent(proxyUrl);
|
|
46
|
+
}
|
|
47
|
+
}
|
|
48
|
+
/**
|
|
49
|
+
* One POST of one event. Resolves with the HTTP status on success; throws
|
|
50
|
+
* `EventCallbackError` on a non-2xx status and the raw transport error when
|
|
51
|
+
* the request never completed.
|
|
52
|
+
*/
|
|
53
|
+
async deliver(request) {
|
|
54
|
+
const controller = new AbortController();
|
|
55
|
+
const timer = setTimeout(() => controller.abort(), this.timeoutMs);
|
|
56
|
+
timer.unref?.();
|
|
57
|
+
const options = {
|
|
58
|
+
method: 'POST',
|
|
59
|
+
headers: {
|
|
60
|
+
'Content-Type': 'application/json',
|
|
61
|
+
'X-PixivFlow-Delivery': 'job-event',
|
|
62
|
+
// The consumer deduplicates on the event_id; the outbox deduplicates on
|
|
63
|
+
// its own key. Sending both is deliberate: either side can converge.
|
|
64
|
+
'X-Idempotency-Key': request.idempotencyKey,
|
|
65
|
+
},
|
|
66
|
+
body: JSON.stringify(request.payload.event),
|
|
67
|
+
signal: controller.signal,
|
|
68
|
+
};
|
|
69
|
+
if (this.dispatcher)
|
|
70
|
+
options.dispatcher = this.dispatcher;
|
|
71
|
+
try {
|
|
72
|
+
const response = await fetch(request.url, options);
|
|
73
|
+
// Drain (and bound) the body so the socket can be reused/closed.
|
|
74
|
+
await response.text().catch(() => '');
|
|
75
|
+
if (response.status >= 200 && response.status < 300)
|
|
76
|
+
return response.status;
|
|
77
|
+
// 409 means the consumer already recorded this event_id — idempotent
|
|
78
|
+
// convergence, not a failure.
|
|
79
|
+
if (response.status === 409)
|
|
80
|
+
return response.status;
|
|
81
|
+
throw new EventCallbackError(`job event callback answered HTTP ${response.status}`, response.status);
|
|
82
|
+
}
|
|
83
|
+
finally {
|
|
84
|
+
clearTimeout(timer);
|
|
85
|
+
}
|
|
86
|
+
}
|
|
87
|
+
}
|
|
88
|
+
exports.EventCallbackDelivery = EventCallbackDelivery;
|
|
89
|
+
//# sourceMappingURL=EventCallbackDelivery.js.map
|
|
@@ -81,6 +81,17 @@ export declare class OutboxWorker {
|
|
|
81
81
|
private pump;
|
|
82
82
|
/** Probe via structured readinessProbe when available; boolean fallback stays compatible. */
|
|
83
83
|
private checkReadiness;
|
|
84
|
+
/**
|
|
85
|
+
* Honour a consumer cancel.
|
|
86
|
+
*
|
|
87
|
+
* A job cancelled while a delivery attempt was in flight (or waiting to
|
|
88
|
+
* retry) must not be delivered afterwards: the ledger's cell already carries
|
|
89
|
+
* the terminal `cancelled_by_consumer` verdict, so the intent behind this row
|
|
90
|
+
* is void. The row is dead-lettered rather than retried — no amount of
|
|
91
|
+
* retrying can make a cancelled delivery correct — and the skip is logged so
|
|
92
|
+
* the operator can see what did NOT happen.
|
|
93
|
+
*/
|
|
94
|
+
private stopCancelledWork;
|
|
84
95
|
/** Record at most one deferral per outbox row per 60s (cold-start poll guard). */
|
|
85
96
|
private recordDeferred;
|
|
86
97
|
/** Pull correlation fields out of payload.context without trusting its shape. */
|