@bymax-one/nest-queue 1.2.9 → 1.2.10
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +55 -1
- package/README.md +7 -0
- package/dist/server/index.cjs +81 -24
- package/dist/server/index.d.cts +15 -4
- package/dist/server/index.d.ts +15 -4
- package/dist/server/index.mjs +82 -25
- package/package.json +2 -1
package/CHANGELOG.md
CHANGED
|
@@ -11,6 +11,59 @@ versioning follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
|
11
11
|
|
|
12
12
|
---
|
|
13
13
|
|
|
14
|
+
## [1.2.10] — 2026-11-01
|
|
15
|
+
|
|
16
|
+
Action required only if code relies on the status of a failed schedule: `upsertJobScheduler` no
|
|
17
|
+
longer reports every failure as `INVALID_REPEAT_OPTIONS` (400), see the Apply note. Healthy use is
|
|
18
|
+
unchanged. The public API, the types (the declaration files differ only in JSDoc comments) and the
|
|
19
|
+
peer ranges are unchanged. Measured with `diff -r` of a fresh build against the unpacked 1.2.9 tarball,
|
|
20
|
+
the runtime bundles (`index.mjs`, `index.cjs`) differ in code.
|
|
21
|
+
|
|
22
|
+
### Fixed
|
|
23
|
+
|
|
24
|
+
- **A failed schedule write was reported as an invalid cron.** `upsertJobScheduler` turned every
|
|
25
|
+
error without a string `code` into `INVALID_REPEAT_OPTIONS` (400, "pattern must be a valid cron
|
|
26
|
+
expression"). A Redis reply error carries no `code`, so a replica refusing the write (`READONLY`),
|
|
27
|
+
an out-of-memory server (`OOM`) or a missing login (`NOAUTH`) was reported as the caller's mistake,
|
|
28
|
+
and the caller was told not to retry. The pattern is now checked first with BullMQ's own repeat
|
|
29
|
+
strategy, a pure function with no Redis call, and only a pattern it refuses is
|
|
30
|
+
`INVALID_REPEAT_OPTIONS`; every failure of the write itself reaches the caller as it is. A
|
|
31
|
+
consumer-supplied `settings.repeatStrategy` replaces the parser, so the pattern is not checked.
|
|
32
|
+
A malformed pattern on a schedule whose iteration `limit` is already reached is now rejected as
|
|
33
|
+
well; BullMQ's own upsert returns before parsing such a schedule and silently accepts it.
|
|
34
|
+
- **A failing `@OnWorkerEvent` or `@OnQueueEvent` handler could end the process.** The listener was
|
|
35
|
+
bound straight onto the emitter, which neither awaits it nor catches a throw: an async handler
|
|
36
|
+
that rejected became an unhandled rejection, which ends a Node process by default, and a
|
|
37
|
+
synchronous throw was re-emitted as `error`. The failure is now logged with the handler's name
|
|
38
|
+
and the code `queue.event_handler_failed`, and the process carries on.
|
|
39
|
+
- **`drainOnShutdown` failed silently.** A queue whose drain failed kept its waiting jobs with no
|
|
40
|
+
trace in the log; it now logs a warning naming the queue and the cause.
|
|
41
|
+
- **A worker whose close failed was logged as a drain timeout.** Any rejection of the graceful
|
|
42
|
+
close was reported as "exceeded the drain budget"; a real error, such as a lost connection, now
|
|
43
|
+
has its own message with the cause. The worker is force-closed either way.
|
|
44
|
+
|
|
45
|
+
### Documentation
|
|
46
|
+
|
|
47
|
+
- **A duplicate does not add a job.** The README and the JSDoc of `enqueue`, `enqueueBulk` and the
|
|
48
|
+
flow methods now say that when a custom `jobId` already exists, or a `deduplication` rule keeps
|
|
49
|
+
the job already there, BullMQ answers with the id of the existing job together with the data of
|
|
50
|
+
this call, so the returned object does not describe what is stored; `getJob` returns it.
|
|
51
|
+
- **The scheduler and event-handler behaviour above is described in the README.**
|
|
52
|
+
- **Correction to 1.2.9.** On 1.2.8 only the `FlowProducer` listener could end the process; on
|
|
53
|
+
`Queue`, `Worker` and `QueueEvents` BullMQ's wrapper re-emitted the `TypeError` and the error was
|
|
54
|
+
reported with a misleading log line. 1.2.9 fixes both.
|
|
55
|
+
|
|
56
|
+
#### Apply to a derived backend
|
|
57
|
+
|
|
58
|
+
- If you branch on `INVALID_REPEAT_OPTIONS` to detect any failure of `upsertJobScheduler`, change
|
|
59
|
+
it: a valid pattern whose write fails now throws the original error (a Redis reply error, a
|
|
60
|
+
connection error), not a 400 `QueueException`. An unparseable pattern is still
|
|
61
|
+
`INVALID_REPEAT_OPTIONS`, now thrown before Redis is asked.
|
|
62
|
+
- A listener that fails is now logged instead of ending the process. If something relied on the
|
|
63
|
+
crash to stop a faulty deploy, watch for the log code `queue.event_handler_failed`.
|
|
64
|
+
|
|
65
|
+
---
|
|
66
|
+
|
|
14
67
|
## [1.2.9] — 2026-11-01
|
|
15
68
|
|
|
16
69
|
No action required: behaviour for an `Error` with a string `message` is unchanged, and the public API, the types
|
|
@@ -768,7 +821,8 @@ v6 peer range.
|
|
|
768
821
|
|
|
769
822
|
---
|
|
770
823
|
|
|
771
|
-
[Unreleased]: https://github.com/bymaxone/nest-queue/compare/v1.2.
|
|
824
|
+
[Unreleased]: https://github.com/bymaxone/nest-queue/compare/v1.2.10...HEAD
|
|
825
|
+
[1.2.10]: https://github.com/bymaxone/nest-queue/compare/v1.2.9...v1.2.10
|
|
772
826
|
[1.2.9]: https://github.com/bymaxone/nest-queue/compare/v1.2.8...v1.2.9
|
|
773
827
|
[1.2.8]: https://github.com/bymaxone/nest-queue/compare/v1.2.7...v1.2.8
|
|
774
828
|
[1.2.7]: https://github.com/bymaxone/nest-queue/compare/v1.2.6...v1.2.7
|
package/README.md
CHANGED
|
@@ -557,6 +557,8 @@ onFailed(job: Job | undefined, err: Error): void {
|
|
|
557
557
|
}
|
|
558
558
|
```
|
|
559
559
|
|
|
560
|
+
A listener that throws, or returns a promise that rejects, is logged with the handler's name and the code `queue.event_handler_failed` and the process carries on; it is never an unhandled rejection. Handle the error inside the listener when the failure matters.
|
|
561
|
+
|
|
560
562
|
### `@OnQueueEvent(eventName)`
|
|
561
563
|
|
|
562
564
|
Listens to BullMQ `QueueEvents` (global, cross-process events). The library opens one shared `QueueEvents` connection per queue when it discovers the first listener for it; there is no option to enable.
|
|
@@ -602,6 +604,7 @@ const schedulers = await this.queue.getJobSchedulers('notifications')
|
|
|
602
604
|
|
|
603
605
|
Cron-with-seconds (6-field patterns) is supported. Patterns are validated by BullMQ's
|
|
604
606
|
own `cron-parser`, not a hand-rolled regex.
|
|
607
|
+
A pattern the parser rejects is `INVALID_REPEAT_OPTIONS` (400) before Redis is asked. Every other failure of the upsert, such as a read-only replica or a lost connection, reaches you as it is.
|
|
605
608
|
|
|
606
609
|
---
|
|
607
610
|
|
|
@@ -643,6 +646,10 @@ await this.queue.enqueue(
|
|
|
643
646
|
)
|
|
644
647
|
```
|
|
645
648
|
|
|
649
|
+
### A duplicate does not add a job
|
|
650
|
+
|
|
651
|
+
When a custom `jobId` already exists, or a `deduplication` rule keeps the job that is already there (a rule with `replace` swaps it for the new one instead), `enqueue`, `enqueueBulk` and the flow methods add nothing: BullMQ answers with the id of the **existing** job together with the data and options of **this** call, so the returned object does not describe what is stored. Call `QueueService.getJob` when you need the stored job.
|
|
652
|
+
|
|
646
653
|
### OpenTelemetry
|
|
647
654
|
|
|
648
655
|
Pass a `bullmq-otel` telemetry instance (optional peer dep) to trace spans from `enqueue` through the handler:
|
package/dist/server/index.cjs
CHANGED
|
@@ -389,8 +389,15 @@ function attachDefaultErrorListener(emitter, logger, kind, queueName) {
|
|
|
389
389
|
|
|
390
390
|
// src/server/services/queue.service.ts
|
|
391
391
|
var MAX_BULK_SIZE = 1e3;
|
|
392
|
-
function
|
|
393
|
-
|
|
392
|
+
function assertValidCron(repeat, queueOptions) {
|
|
393
|
+
if (queueOptions.settings?.repeatStrategy !== void 0) return;
|
|
394
|
+
try {
|
|
395
|
+
bullmq.defaultRepeatStrategy(Date.now(), repeat);
|
|
396
|
+
} catch {
|
|
397
|
+
throw new QueueException(QUEUE_ERROR_CODES.INVALID_REPEAT_OPTIONS, common.HttpStatus.BAD_REQUEST, {
|
|
398
|
+
reason: "pattern must be a valid cron expression (5- or 6-field)"
|
|
399
|
+
});
|
|
400
|
+
}
|
|
394
401
|
}
|
|
395
402
|
var _queues;
|
|
396
403
|
exports.QueueService = class QueueService {
|
|
@@ -468,7 +475,11 @@ exports.QueueService = class QueueService {
|
|
|
468
475
|
* @param jobName - Job identifier used by `@Process` to dispatch.
|
|
469
476
|
* @param data - Typed job payload.
|
|
470
477
|
* @param options - Per-job BullMQ options (priority, delay, `jobId`, `deduplication`, ...).
|
|
471
|
-
* @returns The
|
|
478
|
+
* @returns The job BullMQ returned. When a custom `jobId` already exists, or a `deduplication`
|
|
479
|
+
* rule keeps the job already there (one with `replace` swaps it for the new job), nothing new
|
|
480
|
+
* is added: BullMQ answers with the id of the EXISTING job and the data and options of THIS
|
|
481
|
+
* call, so the returned object does not describe what is stored. Call
|
|
482
|
+
* {@link QueueService.getJob} when you need the stored job.
|
|
472
483
|
* @example Throttle: at most one reindex per term every 5 seconds. Both variable
|
|
473
484
|
* fields are percent-encoded, so no other tenant/term pair can build this key.
|
|
474
485
|
* const dedupId = `${encodeURIComponent(tenantId)}:reindex:${encodeURIComponent(term)}`
|
|
@@ -488,6 +499,9 @@ exports.QueueService = class QueueService {
|
|
|
488
499
|
*
|
|
489
500
|
* @param queueName - Target queue.
|
|
490
501
|
* @param jobs - Job descriptors (length must not exceed the bulk size cap).
|
|
502
|
+
* @returns The jobs BullMQ returned, in input order. An entry whose custom `jobId` already
|
|
503
|
+
* exists is not added again: it carries the EXISTING id with the data of this call, as for
|
|
504
|
+
* {@link QueueService.enqueue}.
|
|
491
505
|
*/
|
|
492
506
|
async enqueueBulk(queueName, jobs) {
|
|
493
507
|
if (jobs.length > MAX_BULK_SIZE) {
|
|
@@ -568,9 +582,9 @@ exports.QueueService = class QueueService {
|
|
|
568
582
|
* never duplicates it. This supersedes the repeatable-jobs surface removed in
|
|
569
583
|
* BullMQ v6.
|
|
570
584
|
*
|
|
571
|
-
* The schedule is
|
|
572
|
-
*
|
|
573
|
-
*
|
|
585
|
+
* The schedule is validated first, and a cron pattern BullMQ's parser rejects
|
|
586
|
+
* is `INVALID_REPEAT_OPTIONS` (400) before Redis is asked; any other failure
|
|
587
|
+
* propagates as it is.
|
|
574
588
|
*
|
|
575
589
|
* @param queueName - Target queue.
|
|
576
590
|
* @param schedulerId - Stable, unique scheduler identifier (the upsert key).
|
|
@@ -587,17 +601,7 @@ exports.QueueService = class QueueService {
|
|
|
587
601
|
data: template?.data ?? {},
|
|
588
602
|
...template?.opts === void 0 ? {} : { opts: template.opts }
|
|
589
603
|
};
|
|
590
|
-
if ("pattern" in repeat)
|
|
591
|
-
try {
|
|
592
|
-
const job2 = await queue.upsertJobScheduler(schedulerId, repeat, jobTemplate);
|
|
593
|
-
return job2;
|
|
594
|
-
} catch (err) {
|
|
595
|
-
if (isOperationalError(err)) throw err;
|
|
596
|
-
throw new QueueException(QUEUE_ERROR_CODES.INVALID_REPEAT_OPTIONS, common.HttpStatus.BAD_REQUEST, {
|
|
597
|
-
reason: "pattern must be a valid cron expression (5- or 6-field)"
|
|
598
|
-
});
|
|
599
|
-
}
|
|
600
|
-
}
|
|
604
|
+
if ("pattern" in repeat) assertValidCron(repeat, this.options.queueOptions);
|
|
601
605
|
const job = await queue.upsertJobScheduler(schedulerId, repeat, jobTemplate);
|
|
602
606
|
return job;
|
|
603
607
|
}
|
|
@@ -699,6 +703,9 @@ exports.FlowService = class FlowService {
|
|
|
699
703
|
*
|
|
700
704
|
* @param flow - Flow tree definition.
|
|
701
705
|
* @returns The root `JobNode` containing the created job and its children.
|
|
706
|
+
* A job whose custom `jobId` already exists is not added again: its node carries the EXISTING
|
|
707
|
+
* id with the data of this call, so the tree does not describe what is stored; call
|
|
708
|
+
* `QueueService.getJob` when you need it.
|
|
702
709
|
* @throws {QueueException} `FLOW_DISABLED` (503) when flows are not enabled.
|
|
703
710
|
*/
|
|
704
711
|
async add(flow) {
|
|
@@ -709,6 +716,7 @@ exports.FlowService = class FlowService {
|
|
|
709
716
|
*
|
|
710
717
|
* @param flows - Flow tree definitions.
|
|
711
718
|
* @returns The root `JobNode` for each flow, in input order.
|
|
719
|
+
* A job whose custom `jobId` already exists is not added again; see {@link FlowService.add}.
|
|
712
720
|
* @throws {QueueException} `FLOW_DISABLED` (503) when flows are not enabled.
|
|
713
721
|
*/
|
|
714
722
|
async addBulk(flows) {
|
|
@@ -1207,6 +1215,7 @@ var WORKER_EVENT_LISTENERS_METADATA_KEY = /* @__PURE__ */ Symbol("bymax_queue:wo
|
|
|
1207
1215
|
var QUEUE_EVENT_LISTENERS_METADATA_KEY = /* @__PURE__ */ Symbol("bymax_queue:queue_event_listeners");
|
|
1208
1216
|
|
|
1209
1217
|
// src/server/services/processor-discovery.service.ts
|
|
1218
|
+
var EVENT_HANDLER_FAILED = "queue.event_handler_failed";
|
|
1210
1219
|
var ProcessorDiscoveryService = class {
|
|
1211
1220
|
constructor(discovery, workers, events) {
|
|
1212
1221
|
this.discovery = discovery;
|
|
@@ -1301,7 +1310,14 @@ var ProcessorDiscoveryService = class {
|
|
|
1301
1310
|
for (const { eventName, methodKey } of workerListeners) {
|
|
1302
1311
|
const method = instance[methodKey];
|
|
1303
1312
|
if (typeof method === "function") {
|
|
1304
|
-
workerEmitter.on(
|
|
1313
|
+
workerEmitter.on(
|
|
1314
|
+
eventName,
|
|
1315
|
+
this.guardListener(instance, method, {
|
|
1316
|
+
decorator: "@OnWorkerEvent",
|
|
1317
|
+
eventName,
|
|
1318
|
+
methodKey
|
|
1319
|
+
})
|
|
1320
|
+
);
|
|
1305
1321
|
}
|
|
1306
1322
|
}
|
|
1307
1323
|
const queueListeners = Reflect.getOwnMetadata(QUEUE_EVENT_LISTENERS_METADATA_KEY, ctor) ?? [];
|
|
@@ -1311,11 +1327,46 @@ var ProcessorDiscoveryService = class {
|
|
|
1311
1327
|
for (const { eventName, methodKey } of queueListeners) {
|
|
1312
1328
|
const method = instance[methodKey];
|
|
1313
1329
|
if (typeof method === "function") {
|
|
1314
|
-
qeEmitter.on(
|
|
1330
|
+
qeEmitter.on(
|
|
1331
|
+
eventName,
|
|
1332
|
+
this.guardListener(instance, method, {
|
|
1333
|
+
decorator: "@OnQueueEvent",
|
|
1334
|
+
eventName,
|
|
1335
|
+
methodKey
|
|
1336
|
+
})
|
|
1337
|
+
);
|
|
1315
1338
|
}
|
|
1316
1339
|
}
|
|
1317
1340
|
}
|
|
1318
1341
|
}
|
|
1342
|
+
/**
|
|
1343
|
+
* Bind an event listener so a failing handler is reported instead of escaping.
|
|
1344
|
+
*
|
|
1345
|
+
* An `EventEmitter` neither awaits a listener nor catches its throw: a rejected promise becomes
|
|
1346
|
+
* an `unhandledRejection`, which ends the process by default, and BullMQ re-emits a synchronous
|
|
1347
|
+
* throw as `error`. A failing handler is the consumer's defect, never a reason to
|
|
1348
|
+
* stop the application, but it must still be visible, so the failure is logged with the handler
|
|
1349
|
+
* and a stable code and the process carries on.
|
|
1350
|
+
*
|
|
1351
|
+
* @param instance - The provider the handler belongs to (its `this`).
|
|
1352
|
+
* @param method - The handler method.
|
|
1353
|
+
* @param origin - Which decorator, event and method name the listener comes from.
|
|
1354
|
+
* @returns The listener to register on the emitter.
|
|
1355
|
+
*/
|
|
1356
|
+
guardListener(instance, method, origin) {
|
|
1357
|
+
const report = (cause) => {
|
|
1358
|
+
this.logger.error(
|
|
1359
|
+
`${origin.decorator}("${origin.eventName}") handler ${instance.constructor.name}.${String(origin.methodKey)} failed (${EVENT_HANDLER_FAILED}): ${describeError(cause)}`
|
|
1360
|
+
);
|
|
1361
|
+
};
|
|
1362
|
+
return (...args) => {
|
|
1363
|
+
try {
|
|
1364
|
+
void Promise.resolve(method.apply(instance, args)).catch(report);
|
|
1365
|
+
} catch (cause) {
|
|
1366
|
+
report(cause);
|
|
1367
|
+
}
|
|
1368
|
+
};
|
|
1369
|
+
}
|
|
1319
1370
|
};
|
|
1320
1371
|
ProcessorDiscoveryService = __decorateClass([
|
|
1321
1372
|
common.Injectable(),
|
|
@@ -1323,6 +1374,8 @@ ProcessorDiscoveryService = __decorateClass([
|
|
|
1323
1374
|
__decorateParam(1, common.Inject(exports.WorkerRegistry)),
|
|
1324
1375
|
__decorateParam(2, common.Inject(exports.QueueEventsRegistry))
|
|
1325
1376
|
], ProcessorDiscoveryService);
|
|
1377
|
+
var DrainTimeoutError = class extends Error {
|
|
1378
|
+
};
|
|
1326
1379
|
var QueueLifecycle = class {
|
|
1327
1380
|
constructor(workers, events, queues, flow, connection, resolved) {
|
|
1328
1381
|
this.workers = workers;
|
|
@@ -1368,10 +1421,10 @@ var QueueLifecycle = class {
|
|
|
1368
1421
|
for (const [name, worker] of this.workers.getAll()) {
|
|
1369
1422
|
try {
|
|
1370
1423
|
await this.closeWorkerWithTimeout(worker, drainTimeoutMs);
|
|
1371
|
-
} catch {
|
|
1424
|
+
} catch (cause) {
|
|
1372
1425
|
forced++;
|
|
1373
1426
|
this.logger.warn(
|
|
1374
|
-
`Worker "${name}" exceeded the ${String(drainTimeoutMs)}ms drain budget \u2014 forcing close (${QUEUE_ERROR_CODES.SHUTDOWN_TIMEOUT_EXCEEDED}); the in-flight job becomes stalled and is retried`
|
|
1427
|
+
cause instanceof DrainTimeoutError ? `Worker "${name}" exceeded the ${String(drainTimeoutMs)}ms drain budget \u2014 forcing close (${QUEUE_ERROR_CODES.SHUTDOWN_TIMEOUT_EXCEEDED}); the in-flight job becomes stalled and is retried` : `Worker "${name}" failed to close gracefully (${describeError(cause)}) \u2014 forcing close; the in-flight job becomes stalled and is retried`
|
|
1375
1428
|
);
|
|
1376
1429
|
void worker.close(true).catch(() => void 0);
|
|
1377
1430
|
}
|
|
@@ -1395,7 +1448,7 @@ var QueueLifecycle = class {
|
|
|
1395
1448
|
closed,
|
|
1396
1449
|
new Promise((_resolve, reject) => {
|
|
1397
1450
|
timer = setTimeout(() => {
|
|
1398
|
-
reject(new
|
|
1451
|
+
reject(new DrainTimeoutError());
|
|
1399
1452
|
}, timeoutMs);
|
|
1400
1453
|
})
|
|
1401
1454
|
]);
|
|
@@ -1434,8 +1487,12 @@ var QueueLifecycle = class {
|
|
|
1434
1487
|
*/
|
|
1435
1488
|
async drainQueuesIfRequested() {
|
|
1436
1489
|
if (!this.resolved.shutdown.drainOnShutdown) return;
|
|
1437
|
-
for (const [, queue] of this.queues.getCachedQueues()) {
|
|
1438
|
-
await queue.drain().catch(() =>
|
|
1490
|
+
for (const [name, queue] of this.queues.getCachedQueues()) {
|
|
1491
|
+
await queue.drain().catch((cause) => {
|
|
1492
|
+
this.logger.warn(
|
|
1493
|
+
`Queue "${name}" could not be drained on shutdown (drainOnShutdown): ${describeError(cause)}; its waiting jobs remain`
|
|
1494
|
+
);
|
|
1495
|
+
});
|
|
1439
1496
|
}
|
|
1440
1497
|
}
|
|
1441
1498
|
/** Close the flow producer (idempotent), swallowing any failure. */
|
package/dist/server/index.d.cts
CHANGED
|
@@ -453,7 +453,11 @@ declare class QueueService {
|
|
|
453
453
|
* @param jobName - Job identifier used by `@Process` to dispatch.
|
|
454
454
|
* @param data - Typed job payload.
|
|
455
455
|
* @param options - Per-job BullMQ options (priority, delay, `jobId`, `deduplication`, ...).
|
|
456
|
-
* @returns The
|
|
456
|
+
* @returns The job BullMQ returned. When a custom `jobId` already exists, or a `deduplication`
|
|
457
|
+
* rule keeps the job already there (one with `replace` swaps it for the new job), nothing new
|
|
458
|
+
* is added: BullMQ answers with the id of the EXISTING job and the data and options of THIS
|
|
459
|
+
* call, so the returned object does not describe what is stored. Call
|
|
460
|
+
* {@link QueueService.getJob} when you need the stored job.
|
|
457
461
|
* @example Throttle: at most one reindex per term every 5 seconds. Both variable
|
|
458
462
|
* fields are percent-encoded, so no other tenant/term pair can build this key.
|
|
459
463
|
* const dedupId = `${encodeURIComponent(tenantId)}:reindex:${encodeURIComponent(term)}`
|
|
@@ -470,6 +474,9 @@ declare class QueueService {
|
|
|
470
474
|
*
|
|
471
475
|
* @param queueName - Target queue.
|
|
472
476
|
* @param jobs - Job descriptors (length must not exceed the bulk size cap).
|
|
477
|
+
* @returns The jobs BullMQ returned, in input order. An entry whose custom `jobId` already
|
|
478
|
+
* exists is not added again: it carries the EXISTING id with the data of this call, as for
|
|
479
|
+
* {@link QueueService.enqueue}.
|
|
473
480
|
*/
|
|
474
481
|
enqueueBulk<TData = unknown, TResult = unknown>(queueName: string, jobs: readonly BulkJob<TData>[]): Promise<TypedJob<TData, TResult>[]>;
|
|
475
482
|
/**
|
|
@@ -501,9 +508,9 @@ declare class QueueService {
|
|
|
501
508
|
* never duplicates it. This supersedes the repeatable-jobs surface removed in
|
|
502
509
|
* BullMQ v6.
|
|
503
510
|
*
|
|
504
|
-
* The schedule is
|
|
505
|
-
*
|
|
506
|
-
*
|
|
511
|
+
* The schedule is validated first, and a cron pattern BullMQ's parser rejects
|
|
512
|
+
* is `INVALID_REPEAT_OPTIONS` (400) before Redis is asked; any other failure
|
|
513
|
+
* propagates as it is.
|
|
507
514
|
*
|
|
508
515
|
* @param queueName - Target queue.
|
|
509
516
|
* @param schedulerId - Stable, unique scheduler identifier (the upsert key).
|
|
@@ -607,6 +614,9 @@ declare class FlowService {
|
|
|
607
614
|
*
|
|
608
615
|
* @param flow - Flow tree definition.
|
|
609
616
|
* @returns The root `JobNode` containing the created job and its children.
|
|
617
|
+
* A job whose custom `jobId` already exists is not added again: its node carries the EXISTING
|
|
618
|
+
* id with the data of this call, so the tree does not describe what is stored; call
|
|
619
|
+
* `QueueService.getJob` when you need it.
|
|
610
620
|
* @throws {QueueException} `FLOW_DISABLED` (503) when flows are not enabled.
|
|
611
621
|
*/
|
|
612
622
|
add(flow: FlowJob): Promise<JobNode>;
|
|
@@ -615,6 +625,7 @@ declare class FlowService {
|
|
|
615
625
|
*
|
|
616
626
|
* @param flows - Flow tree definitions.
|
|
617
627
|
* @returns The root `JobNode` for each flow, in input order.
|
|
628
|
+
* A job whose custom `jobId` already exists is not added again; see {@link FlowService.add}.
|
|
618
629
|
* @throws {QueueException} `FLOW_DISABLED` (503) when flows are not enabled.
|
|
619
630
|
*/
|
|
620
631
|
addBulk(flows: readonly FlowJob[]): Promise<JobNode[]>;
|
package/dist/server/index.d.ts
CHANGED
|
@@ -453,7 +453,11 @@ declare class QueueService {
|
|
|
453
453
|
* @param jobName - Job identifier used by `@Process` to dispatch.
|
|
454
454
|
* @param data - Typed job payload.
|
|
455
455
|
* @param options - Per-job BullMQ options (priority, delay, `jobId`, `deduplication`, ...).
|
|
456
|
-
* @returns The
|
|
456
|
+
* @returns The job BullMQ returned. When a custom `jobId` already exists, or a `deduplication`
|
|
457
|
+
* rule keeps the job already there (one with `replace` swaps it for the new job), nothing new
|
|
458
|
+
* is added: BullMQ answers with the id of the EXISTING job and the data and options of THIS
|
|
459
|
+
* call, so the returned object does not describe what is stored. Call
|
|
460
|
+
* {@link QueueService.getJob} when you need the stored job.
|
|
457
461
|
* @example Throttle: at most one reindex per term every 5 seconds. Both variable
|
|
458
462
|
* fields are percent-encoded, so no other tenant/term pair can build this key.
|
|
459
463
|
* const dedupId = `${encodeURIComponent(tenantId)}:reindex:${encodeURIComponent(term)}`
|
|
@@ -470,6 +474,9 @@ declare class QueueService {
|
|
|
470
474
|
*
|
|
471
475
|
* @param queueName - Target queue.
|
|
472
476
|
* @param jobs - Job descriptors (length must not exceed the bulk size cap).
|
|
477
|
+
* @returns The jobs BullMQ returned, in input order. An entry whose custom `jobId` already
|
|
478
|
+
* exists is not added again: it carries the EXISTING id with the data of this call, as for
|
|
479
|
+
* {@link QueueService.enqueue}.
|
|
473
480
|
*/
|
|
474
481
|
enqueueBulk<TData = unknown, TResult = unknown>(queueName: string, jobs: readonly BulkJob<TData>[]): Promise<TypedJob<TData, TResult>[]>;
|
|
475
482
|
/**
|
|
@@ -501,9 +508,9 @@ declare class QueueService {
|
|
|
501
508
|
* never duplicates it. This supersedes the repeatable-jobs surface removed in
|
|
502
509
|
* BullMQ v6.
|
|
503
510
|
*
|
|
504
|
-
* The schedule is
|
|
505
|
-
*
|
|
506
|
-
*
|
|
511
|
+
* The schedule is validated first, and a cron pattern BullMQ's parser rejects
|
|
512
|
+
* is `INVALID_REPEAT_OPTIONS` (400) before Redis is asked; any other failure
|
|
513
|
+
* propagates as it is.
|
|
507
514
|
*
|
|
508
515
|
* @param queueName - Target queue.
|
|
509
516
|
* @param schedulerId - Stable, unique scheduler identifier (the upsert key).
|
|
@@ -607,6 +614,9 @@ declare class FlowService {
|
|
|
607
614
|
*
|
|
608
615
|
* @param flow - Flow tree definition.
|
|
609
616
|
* @returns The root `JobNode` containing the created job and its children.
|
|
617
|
+
* A job whose custom `jobId` already exists is not added again: its node carries the EXISTING
|
|
618
|
+
* id with the data of this call, so the tree does not describe what is stored; call
|
|
619
|
+
* `QueueService.getJob` when you need it.
|
|
610
620
|
* @throws {QueueException} `FLOW_DISABLED` (503) when flows are not enabled.
|
|
611
621
|
*/
|
|
612
622
|
add(flow: FlowJob): Promise<JobNode>;
|
|
@@ -615,6 +625,7 @@ declare class FlowService {
|
|
|
615
625
|
*
|
|
616
626
|
* @param flows - Flow tree definitions.
|
|
617
627
|
* @returns The root `JobNode` for each flow, in input order.
|
|
628
|
+
* A job whose custom `jobId` already exists is not added again; see {@link FlowService.add}.
|
|
618
629
|
* @throws {QueueException} `FLOW_DISABLED` (503) when flows are not enabled.
|
|
619
630
|
*/
|
|
620
631
|
addBulk(flows: readonly FlowJob[]): Promise<JobNode[]>;
|
package/dist/server/index.mjs
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
import { Injectable, Inject, ConfigurableModuleBuilder, Module, HttpStatus, Logger, HttpException } from '@nestjs/common';
|
|
2
2
|
import { DiscoveryService, DiscoveryModule } from '@nestjs/core';
|
|
3
3
|
import { Redis } from 'ioredis';
|
|
4
|
-
import { Worker, QueueEvents, Queue, FlowProducer } from 'bullmq';
|
|
4
|
+
import { Worker, QueueEvents, Queue, FlowProducer, defaultRepeatStrategy } from 'bullmq';
|
|
5
5
|
import { isAbsolute, extname } from 'path';
|
|
6
6
|
|
|
7
7
|
var __defProp = Object.defineProperty;
|
|
@@ -387,8 +387,15 @@ function attachDefaultErrorListener(emitter, logger, kind, queueName) {
|
|
|
387
387
|
|
|
388
388
|
// src/server/services/queue.service.ts
|
|
389
389
|
var MAX_BULK_SIZE = 1e3;
|
|
390
|
-
function
|
|
391
|
-
|
|
390
|
+
function assertValidCron(repeat, queueOptions) {
|
|
391
|
+
if (queueOptions.settings?.repeatStrategy !== void 0) return;
|
|
392
|
+
try {
|
|
393
|
+
defaultRepeatStrategy(Date.now(), repeat);
|
|
394
|
+
} catch {
|
|
395
|
+
throw new QueueException(QUEUE_ERROR_CODES.INVALID_REPEAT_OPTIONS, HttpStatus.BAD_REQUEST, {
|
|
396
|
+
reason: "pattern must be a valid cron expression (5- or 6-field)"
|
|
397
|
+
});
|
|
398
|
+
}
|
|
392
399
|
}
|
|
393
400
|
var _queues;
|
|
394
401
|
var QueueService = class {
|
|
@@ -466,7 +473,11 @@ var QueueService = class {
|
|
|
466
473
|
* @param jobName - Job identifier used by `@Process` to dispatch.
|
|
467
474
|
* @param data - Typed job payload.
|
|
468
475
|
* @param options - Per-job BullMQ options (priority, delay, `jobId`, `deduplication`, ...).
|
|
469
|
-
* @returns The
|
|
476
|
+
* @returns The job BullMQ returned. When a custom `jobId` already exists, or a `deduplication`
|
|
477
|
+
* rule keeps the job already there (one with `replace` swaps it for the new job), nothing new
|
|
478
|
+
* is added: BullMQ answers with the id of the EXISTING job and the data and options of THIS
|
|
479
|
+
* call, so the returned object does not describe what is stored. Call
|
|
480
|
+
* {@link QueueService.getJob} when you need the stored job.
|
|
470
481
|
* @example Throttle: at most one reindex per term every 5 seconds. Both variable
|
|
471
482
|
* fields are percent-encoded, so no other tenant/term pair can build this key.
|
|
472
483
|
* const dedupId = `${encodeURIComponent(tenantId)}:reindex:${encodeURIComponent(term)}`
|
|
@@ -486,6 +497,9 @@ var QueueService = class {
|
|
|
486
497
|
*
|
|
487
498
|
* @param queueName - Target queue.
|
|
488
499
|
* @param jobs - Job descriptors (length must not exceed the bulk size cap).
|
|
500
|
+
* @returns The jobs BullMQ returned, in input order. An entry whose custom `jobId` already
|
|
501
|
+
* exists is not added again: it carries the EXISTING id with the data of this call, as for
|
|
502
|
+
* {@link QueueService.enqueue}.
|
|
489
503
|
*/
|
|
490
504
|
async enqueueBulk(queueName, jobs) {
|
|
491
505
|
if (jobs.length > MAX_BULK_SIZE) {
|
|
@@ -566,9 +580,9 @@ var QueueService = class {
|
|
|
566
580
|
* never duplicates it. This supersedes the repeatable-jobs surface removed in
|
|
567
581
|
* BullMQ v6.
|
|
568
582
|
*
|
|
569
|
-
* The schedule is
|
|
570
|
-
*
|
|
571
|
-
*
|
|
583
|
+
* The schedule is validated first, and a cron pattern BullMQ's parser rejects
|
|
584
|
+
* is `INVALID_REPEAT_OPTIONS` (400) before Redis is asked; any other failure
|
|
585
|
+
* propagates as it is.
|
|
572
586
|
*
|
|
573
587
|
* @param queueName - Target queue.
|
|
574
588
|
* @param schedulerId - Stable, unique scheduler identifier (the upsert key).
|
|
@@ -585,17 +599,7 @@ var QueueService = class {
|
|
|
585
599
|
data: template?.data ?? {},
|
|
586
600
|
...template?.opts === void 0 ? {} : { opts: template.opts }
|
|
587
601
|
};
|
|
588
|
-
if ("pattern" in repeat)
|
|
589
|
-
try {
|
|
590
|
-
const job2 = await queue.upsertJobScheduler(schedulerId, repeat, jobTemplate);
|
|
591
|
-
return job2;
|
|
592
|
-
} catch (err) {
|
|
593
|
-
if (isOperationalError(err)) throw err;
|
|
594
|
-
throw new QueueException(QUEUE_ERROR_CODES.INVALID_REPEAT_OPTIONS, HttpStatus.BAD_REQUEST, {
|
|
595
|
-
reason: "pattern must be a valid cron expression (5- or 6-field)"
|
|
596
|
-
});
|
|
597
|
-
}
|
|
598
|
-
}
|
|
602
|
+
if ("pattern" in repeat) assertValidCron(repeat, this.options.queueOptions);
|
|
599
603
|
const job = await queue.upsertJobScheduler(schedulerId, repeat, jobTemplate);
|
|
600
604
|
return job;
|
|
601
605
|
}
|
|
@@ -697,6 +701,9 @@ var FlowService = class {
|
|
|
697
701
|
*
|
|
698
702
|
* @param flow - Flow tree definition.
|
|
699
703
|
* @returns The root `JobNode` containing the created job and its children.
|
|
704
|
+
* A job whose custom `jobId` already exists is not added again: its node carries the EXISTING
|
|
705
|
+
* id with the data of this call, so the tree does not describe what is stored; call
|
|
706
|
+
* `QueueService.getJob` when you need it.
|
|
700
707
|
* @throws {QueueException} `FLOW_DISABLED` (503) when flows are not enabled.
|
|
701
708
|
*/
|
|
702
709
|
async add(flow) {
|
|
@@ -707,6 +714,7 @@ var FlowService = class {
|
|
|
707
714
|
*
|
|
708
715
|
* @param flows - Flow tree definitions.
|
|
709
716
|
* @returns The root `JobNode` for each flow, in input order.
|
|
717
|
+
* A job whose custom `jobId` already exists is not added again; see {@link FlowService.add}.
|
|
710
718
|
* @throws {QueueException} `FLOW_DISABLED` (503) when flows are not enabled.
|
|
711
719
|
*/
|
|
712
720
|
async addBulk(flows) {
|
|
@@ -1205,6 +1213,7 @@ var WORKER_EVENT_LISTENERS_METADATA_KEY = /* @__PURE__ */ Symbol("bymax_queue:wo
|
|
|
1205
1213
|
var QUEUE_EVENT_LISTENERS_METADATA_KEY = /* @__PURE__ */ Symbol("bymax_queue:queue_event_listeners");
|
|
1206
1214
|
|
|
1207
1215
|
// src/server/services/processor-discovery.service.ts
|
|
1216
|
+
var EVENT_HANDLER_FAILED = "queue.event_handler_failed";
|
|
1208
1217
|
var ProcessorDiscoveryService = class {
|
|
1209
1218
|
constructor(discovery, workers, events) {
|
|
1210
1219
|
this.discovery = discovery;
|
|
@@ -1299,7 +1308,14 @@ var ProcessorDiscoveryService = class {
|
|
|
1299
1308
|
for (const { eventName, methodKey } of workerListeners) {
|
|
1300
1309
|
const method = instance[methodKey];
|
|
1301
1310
|
if (typeof method === "function") {
|
|
1302
|
-
workerEmitter.on(
|
|
1311
|
+
workerEmitter.on(
|
|
1312
|
+
eventName,
|
|
1313
|
+
this.guardListener(instance, method, {
|
|
1314
|
+
decorator: "@OnWorkerEvent",
|
|
1315
|
+
eventName,
|
|
1316
|
+
methodKey
|
|
1317
|
+
})
|
|
1318
|
+
);
|
|
1303
1319
|
}
|
|
1304
1320
|
}
|
|
1305
1321
|
const queueListeners = Reflect.getOwnMetadata(QUEUE_EVENT_LISTENERS_METADATA_KEY, ctor) ?? [];
|
|
@@ -1309,11 +1325,46 @@ var ProcessorDiscoveryService = class {
|
|
|
1309
1325
|
for (const { eventName, methodKey } of queueListeners) {
|
|
1310
1326
|
const method = instance[methodKey];
|
|
1311
1327
|
if (typeof method === "function") {
|
|
1312
|
-
qeEmitter.on(
|
|
1328
|
+
qeEmitter.on(
|
|
1329
|
+
eventName,
|
|
1330
|
+
this.guardListener(instance, method, {
|
|
1331
|
+
decorator: "@OnQueueEvent",
|
|
1332
|
+
eventName,
|
|
1333
|
+
methodKey
|
|
1334
|
+
})
|
|
1335
|
+
);
|
|
1313
1336
|
}
|
|
1314
1337
|
}
|
|
1315
1338
|
}
|
|
1316
1339
|
}
|
|
1340
|
+
/**
|
|
1341
|
+
* Bind an event listener so a failing handler is reported instead of escaping.
|
|
1342
|
+
*
|
|
1343
|
+
* An `EventEmitter` neither awaits a listener nor catches its throw: a rejected promise becomes
|
|
1344
|
+
* an `unhandledRejection`, which ends the process by default, and BullMQ re-emits a synchronous
|
|
1345
|
+
* throw as `error`. A failing handler is the consumer's defect, never a reason to
|
|
1346
|
+
* stop the application, but it must still be visible, so the failure is logged with the handler
|
|
1347
|
+
* and a stable code and the process carries on.
|
|
1348
|
+
*
|
|
1349
|
+
* @param instance - The provider the handler belongs to (its `this`).
|
|
1350
|
+
* @param method - The handler method.
|
|
1351
|
+
* @param origin - Which decorator, event and method name the listener comes from.
|
|
1352
|
+
* @returns The listener to register on the emitter.
|
|
1353
|
+
*/
|
|
1354
|
+
guardListener(instance, method, origin) {
|
|
1355
|
+
const report = (cause) => {
|
|
1356
|
+
this.logger.error(
|
|
1357
|
+
`${origin.decorator}("${origin.eventName}") handler ${instance.constructor.name}.${String(origin.methodKey)} failed (${EVENT_HANDLER_FAILED}): ${describeError(cause)}`
|
|
1358
|
+
);
|
|
1359
|
+
};
|
|
1360
|
+
return (...args) => {
|
|
1361
|
+
try {
|
|
1362
|
+
void Promise.resolve(method.apply(instance, args)).catch(report);
|
|
1363
|
+
} catch (cause) {
|
|
1364
|
+
report(cause);
|
|
1365
|
+
}
|
|
1366
|
+
};
|
|
1367
|
+
}
|
|
1317
1368
|
};
|
|
1318
1369
|
ProcessorDiscoveryService = __decorateClass([
|
|
1319
1370
|
Injectable(),
|
|
@@ -1321,6 +1372,8 @@ ProcessorDiscoveryService = __decorateClass([
|
|
|
1321
1372
|
__decorateParam(1, Inject(WorkerRegistry)),
|
|
1322
1373
|
__decorateParam(2, Inject(QueueEventsRegistry))
|
|
1323
1374
|
], ProcessorDiscoveryService);
|
|
1375
|
+
var DrainTimeoutError = class extends Error {
|
|
1376
|
+
};
|
|
1324
1377
|
var QueueLifecycle = class {
|
|
1325
1378
|
constructor(workers, events, queues, flow, connection, resolved) {
|
|
1326
1379
|
this.workers = workers;
|
|
@@ -1366,10 +1419,10 @@ var QueueLifecycle = class {
|
|
|
1366
1419
|
for (const [name, worker] of this.workers.getAll()) {
|
|
1367
1420
|
try {
|
|
1368
1421
|
await this.closeWorkerWithTimeout(worker, drainTimeoutMs);
|
|
1369
|
-
} catch {
|
|
1422
|
+
} catch (cause) {
|
|
1370
1423
|
forced++;
|
|
1371
1424
|
this.logger.warn(
|
|
1372
|
-
`Worker "${name}" exceeded the ${String(drainTimeoutMs)}ms drain budget \u2014 forcing close (${QUEUE_ERROR_CODES.SHUTDOWN_TIMEOUT_EXCEEDED}); the in-flight job becomes stalled and is retried`
|
|
1425
|
+
cause instanceof DrainTimeoutError ? `Worker "${name}" exceeded the ${String(drainTimeoutMs)}ms drain budget \u2014 forcing close (${QUEUE_ERROR_CODES.SHUTDOWN_TIMEOUT_EXCEEDED}); the in-flight job becomes stalled and is retried` : `Worker "${name}" failed to close gracefully (${describeError(cause)}) \u2014 forcing close; the in-flight job becomes stalled and is retried`
|
|
1373
1426
|
);
|
|
1374
1427
|
void worker.close(true).catch(() => void 0);
|
|
1375
1428
|
}
|
|
@@ -1393,7 +1446,7 @@ var QueueLifecycle = class {
|
|
|
1393
1446
|
closed,
|
|
1394
1447
|
new Promise((_resolve, reject) => {
|
|
1395
1448
|
timer = setTimeout(() => {
|
|
1396
|
-
reject(new
|
|
1449
|
+
reject(new DrainTimeoutError());
|
|
1397
1450
|
}, timeoutMs);
|
|
1398
1451
|
})
|
|
1399
1452
|
]);
|
|
@@ -1432,8 +1485,12 @@ var QueueLifecycle = class {
|
|
|
1432
1485
|
*/
|
|
1433
1486
|
async drainQueuesIfRequested() {
|
|
1434
1487
|
if (!this.resolved.shutdown.drainOnShutdown) return;
|
|
1435
|
-
for (const [, queue] of this.queues.getCachedQueues()) {
|
|
1436
|
-
await queue.drain().catch(() =>
|
|
1488
|
+
for (const [name, queue] of this.queues.getCachedQueues()) {
|
|
1489
|
+
await queue.drain().catch((cause) => {
|
|
1490
|
+
this.logger.warn(
|
|
1491
|
+
`Queue "${name}" could not be drained on shutdown (drainOnShutdown): ${describeError(cause)}; its waiting jobs remain`
|
|
1492
|
+
);
|
|
1493
|
+
});
|
|
1437
1494
|
}
|
|
1438
1495
|
}
|
|
1439
1496
|
/** Close the flow producer (idempotent), swallowing any failure. */
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@bymax-one/nest-queue",
|
|
3
|
-
"version": "1.2.
|
|
3
|
+
"version": "1.2.10",
|
|
4
4
|
"description": "NestJS dynamic module wrapping BullMQ — typed jobs, flows, job schedulers, deduplication, OpenTelemetry, graceful shutdown",
|
|
5
5
|
"author": "Bymax One <support@bymax.one>",
|
|
6
6
|
"license": "MIT",
|
|
@@ -55,6 +55,7 @@
|
|
|
55
55
|
},
|
|
56
56
|
"scripts": {
|
|
57
57
|
"build": "pnpm clean && tsup",
|
|
58
|
+
"check:api": "node scripts/check-api-surface.mjs",
|
|
58
59
|
"check:exports": "attw --pack .",
|
|
59
60
|
"check:mutants": "node scripts/check-mutation-directives.mjs",
|
|
60
61
|
"check:published": "node scripts/check-published-surface.mjs",
|