@bymax-one/nest-queue 1.2.14 → 1.2.16
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +50 -1
- package/README.md +77 -68
- package/dist/server/index.cjs +219 -107
- package/dist/server/index.d.cts +57 -23
- package/dist/server/index.d.ts +57 -23
- package/dist/server/index.mjs +219 -107
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -11,6 +11,53 @@ versioning follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
|
11
11
|
|
|
12
12
|
---
|
|
13
13
|
|
|
14
|
+
## [1.2.16] — 2026-11-14
|
|
15
|
+
|
|
16
|
+
Against 1.2.15 the built declarations gain a fifth optional `FlowService` constructor parameter, `enqueueTimeoutMs`; nothing else in the public API changes.
|
|
17
|
+
|
|
18
|
+
### Fixed
|
|
19
|
+
|
|
20
|
+
- **Timer-valued options that Node cannot honour are refused at startup.** `shutdown.drainTimeoutMs` and `connectionReadyTimeoutMs` accepted `NaN`, fractions and values above 2 147 483 647 ms; Node reads `NaN` and anything above that limit as a 1 ms timer, so the drain or the startup wait ended at once. Both must now be whole numbers from 1 to 2 147 483 647, like `commandTimeoutMs` and `enqueueTimeoutMs`.
|
|
21
|
+
- **Worker options are checked before BullMQ sees them.** `concurrency`, `limiter.max` and `limiter.duration` must be whole numbers (`NaN` passed the range check). `lockDuration` and `stalledInterval` must be whole numbers from 1000 to 2 147 483 647: BullMQ renews a lock at half of `lockDuration`, so under one second the renewal falls inside ordinary event-loop delay and a busy process loses locks it still holds.
|
|
22
|
+
- **`getJobSchedulers` has the same window bound as `getJobs`.** BullMQ reads an end index of `-1` as "every scheduler"; the window is now at most 1000 entries and a malformed one is refused with `INVALID_OPTIONS`.
|
|
23
|
+
- **`FlowService.add` and `addBulk` are bounded.** Each is held to `enqueueTimeoutMs` and rejects with `CONNECTION_TIMEOUT` (a flow that timed out may still have been added), and `addBulk` refuses more than 1000 flows with `BULK_ENQUEUE_FAILED`, as `enqueueBulk` does.
|
|
24
|
+
- **A call waiting for the connection is released when the client ends.** `ConnectionResolver.awaitReady()` kept its callers waiting until their deadline after ioredis had given up reconnecting; they are now released at once and the call reports the real failure.
|
|
25
|
+
|
|
26
|
+
### Documentation
|
|
27
|
+
|
|
28
|
+
- A handler that outlives `jobTimeoutMs` no longer counts against `concurrency`; the README says so.
|
|
29
|
+
|
|
30
|
+
#### Apply to a derived backend
|
|
31
|
+
|
|
32
|
+
- If you pass `shutdown.drainTimeoutMs` or `connectionReadyTimeoutMs` as a fraction or above 2 147 483 647, or a worker `lockDuration` or `stalledInterval` below 1000, startup or registration now throws; use a whole number in range.
|
|
33
|
+
- If you call `getJobSchedulers` with an open or oversized window, request at most 1000 entries per call.
|
|
34
|
+
- If you construct `FlowService` yourself, pass `enqueueTimeoutMs` as the fifth argument; it defaults to 10 000.
|
|
35
|
+
|
|
36
|
+
---
|
|
37
|
+
|
|
38
|
+
## [1.2.15] — 2026-11-14
|
|
39
|
+
|
|
40
|
+
Against 1.2.14 the built declarations gain `ConnectionResolver.awaitReady()`; nothing else in the public API changes.
|
|
41
|
+
|
|
42
|
+
### Fixed
|
|
43
|
+
|
|
44
|
+
- **A call made while the connection is down now reports the documented error.** With the owned connection's offline queue off, a call on a queue already in use during a refused-connection outage rejected at once with ioredis's plain `Stream isn't writeable and enableOfflineQueue options is false`, so a host retrying on `CONNECTION_TIMEOUT` as documented never retried. It is now `CONNECTION_TIMEOUT` with `details.operation` and `details.reason: 'offline'`, and the ioredis error as `cause`.
|
|
45
|
+
- **The first call on a queue no longer waits for Redis without a deadline.** `getMetrics`, `getJob`, `getJobs`, the scheduler calls, `pauseQueue`, `resumeQueue` and `cleanQueue` on a queue name not yet used waited in BullMQ for the client to be ready, with no bound, for as long as the outage lasted. Every `QueueService` call is now bounded by `enqueueTimeoutMs` and rejects with `CONNECTION_TIMEOUT`.
|
|
46
|
+
- **Shutdown no longer leaves a worker running after its drain budget.** A handler that never settled kept the worker's lock-renewal timer alive after `app.close()`: the process did not exit by itself and logged `Connection is closed.` repeatedly. BullMQ ignores a forced close once a graceful one has started, so the worker is now paused first and closed with `close(true)` as its first close when the budget runs out. What the abandoned handler produces after the budget, a result or a failure, is dropped, so BullMQ cannot resume the stopped worker and reopen its connection. The handler's own promise still keeps running, since user code cannot be cancelled. A sandboxed worker is the exception: its job runs in a child process the registry cannot wrap, and in a measured run the host process did not exit by itself after the budget ended with such a job still running.
|
|
47
|
+
|
|
48
|
+
### Documentation
|
|
49
|
+
|
|
50
|
+
- `enqueueTimeoutMs` is documented as the per-call deadline of every `QueueService` call, and `queue.connection_timeout` as covering a connection that is down.
|
|
51
|
+
|
|
52
|
+
#### Apply to a derived backend
|
|
53
|
+
|
|
54
|
+
- If you match the ioredis error `Stream isn't writeable and enableOfflineQueue options is false` on a call, match `CONNECTION_TIMEOUT` instead; `details.reason` is `'offline'` and the original error is its `cause`.
|
|
55
|
+
- If a `getMetrics`, `getJob`, `getJobs`, scheduler, pause, resume or clean call relied on waiting out an outage on a queue's first use, it now rejects with `CONNECTION_TIMEOUT` after `enqueueTimeoutMs`; raise the option to wait longer.
|
|
56
|
+
- If you replace `ConnectionResolver` with your own implementation, add `awaitReady(): Promise<void>`: `QueueService` calls it before the first call on each queue name.
|
|
57
|
+
- If you parse the shutdown log lines, the wording changed: `forcing close` is gone and the summary reads `N worker(s) stopped without draining`.
|
|
58
|
+
|
|
59
|
+
---
|
|
60
|
+
|
|
14
61
|
## [1.2.14] — 2026-11-14
|
|
15
62
|
|
|
16
63
|
Several defaults change in this release, so read the Apply note below. A boot that succeeds, an
|
|
@@ -1013,7 +1060,9 @@ v6 peer range.
|
|
|
1013
1060
|
|
|
1014
1061
|
---
|
|
1015
1062
|
|
|
1016
|
-
[Unreleased]: https://github.com/bymaxone/nest-queue/compare/v1.2.
|
|
1063
|
+
[Unreleased]: https://github.com/bymaxone/nest-queue/compare/v1.2.16...HEAD
|
|
1064
|
+
[1.2.16]: https://github.com/bymaxone/nest-queue/compare/v1.2.15...v1.2.16
|
|
1065
|
+
[1.2.15]: https://github.com/bymaxone/nest-queue/compare/v1.2.14...v1.2.15
|
|
1017
1066
|
[1.2.14]: https://github.com/bymaxone/nest-queue/compare/v1.2.13...v1.2.14
|
|
1018
1067
|
[1.2.13]: https://github.com/bymaxone/nest-queue/compare/v1.2.12...v1.2.13
|
|
1019
1068
|
[1.2.12]: https://github.com/bymaxone/nest-queue/compare/v1.2.11...v1.2.12
|
package/README.md
CHANGED
|
@@ -66,7 +66,7 @@ peer dependencies, so you control exact versions and the supply-chain surface st
|
|
|
66
66
|
- ✅ **Dual-mode connection** — bring your own `ioredis` client, or let the library open its own from a URL or an options object
|
|
67
67
|
- ✅ **Per-role retry policy** — `maxRetriesPerRequest: null` reaches only the duplicated Worker and QueueEvents connections; the producer keeps a finite retry limit, a 5 s command deadline and no offline queue on the connection the library opens, and `enqueue` has its own 10 s deadline, so a Redis outage rejects a call in seconds instead of holding the request; a shared client with `maxRetriesPerRequest: null` is the exception, and the library warns about it at boot
|
|
68
68
|
- ✅ **One namespace, four objects** — the configured `prefix` reaches `Queue`, `Worker`, `QueueEvents` **and** `FlowProducer`, so producers and consumers can never end up on separate keyspaces
|
|
69
|
-
- ✅ **Bounded graceful shutdown** — a single ordered sequence drains every worker under
|
|
69
|
+
- ✅ **Bounded graceful shutdown** — a single ordered sequence drains every worker under one deadline and then stops any that are still busy, whose jobs stall and are retried; delivery is at-least-once by design, so handlers must be idempotent
|
|
70
70
|
- ✅ **No unlistened emitter** — every `Queue`, `Worker`, `QueueEvents` and `FlowProducer` the library creates carries a fallback `error` listener, so a transient Redis fault cannot become an uncaught exception in your process; your own `@OnWorkerEvent('error')` or `@OnQueueEvent('error')` still runs alongside it
|
|
71
71
|
|
|
72
72
|
### 🧩 Developer Experience
|
|
@@ -229,22 +229,22 @@ export class EmailModule {}
|
|
|
229
229
|
|
|
230
230
|
Every option passed to `forRoot` / `forRootAsync`. Only `connection` is required.
|
|
231
231
|
|
|
232
|
-
| Option | Type | Default | Description
|
|
233
|
-
| -------------------------- | ------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
234
|
-
| `connection` | `{ client } \| { url, options? } \| { options }` | — | **Required.** Mode A supplies an existing `ioredis` client; Mode B lets the library open its own. The arms are mutually exclusive.
|
|
235
|
-
| `prefix` | `string` | `'bull'` | Redis key namespace. Applied to the `Queue`, `Worker`, `QueueEvents` **and** `FlowProducer` — see [Architecture](#-architecture).
|
|
236
|
-
| `defaultJobOptions` | `JobsOptions` | `attempts: 3` · exponential backoff `2000` ms with 50% jitter · `removeOnComplete: { age: 24 h, count: 1000 }` · `removeOnFail: { age: 7 d, count: 5000 }` | Merged into every job enqueued through the service; per-job options win. The bounded `removeOn*` caps are what keep Redis from growing without limit under a failure storm.
|
|
237
|
-
| `queueOptions` | `Partial<QueueOptions>` | `{}` | Extra BullMQ `Queue` options. `connection`, `defaultJobOptions` and `prefix` are owned by the library and cannot be set here.
|
|
238
|
-
| `flows.enabled` | `boolean` | `false` | `FlowService` is always provided; while disabled, every method except `close()` throws `FLOW_DISABLED`.
|
|
239
|
-
| `metrics.enabled` | `boolean` | `false` | `MetricsService` is always provided; while disabled, every method throws `METRICS_DISABLED`.
|
|
240
|
-
| `metrics.cacheTtlMs` | `number` | `5_000` | TTL of the metrics snapshot cache. `0` disables caching. Must be `>= 0`.
|
|
241
|
-
| `telemetry` | `Telemetry` | — | A BullMQ `Telemetry` instance (typically `new BullMQOtel(...)` from the optional `bullmq-otel` peer), attached to every `Queue` and `Worker`.
|
|
242
|
-
| `shutdown.drainTimeoutMs` | `number` | `30_000` | Upper bound on the graceful drain before workers are force-closed.
|
|
243
|
-
| `shutdown.drainOnShutdown` | `boolean` | `false` | Development and test only: once the HTTP server has stopped, call `drain()` on every cached queue, which **removes** its waiting jobs (delayed and active jobs are left alone). Leave it `false` in production.
|
|
244
|
-
| `connectionReadyTimeoutMs` | `number` | `10_000` | Mode B only: the budget for Redis to reach `ready
|
|
245
|
-
| `commandTimeoutMs` | `number` | `5_000` | Mode B only: the deadline for one Redis command on the connection the library opened (ioredis `commandTimeout`); an `options.commandTimeout` wins. A call that hits it rejects with `CONNECTION_TIMEOUT` (`details` carries `operation` and `timeoutMs`), as when `enqueueTimeoutMs` passes. Worker and QueueEvents connections have none.
|
|
246
|
-
| `enqueueTimeoutMs` | `number` | `10_000` | The deadline for one `
|
|
247
|
-
| `isGlobal` | `boolean` | `true` | Register the module globally, so `QueueService` is injectable without re-importing.
|
|
232
|
+
| Option | Type | Default | Description |
|
|
233
|
+
| -------------------------- | ------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
234
|
+
| `connection` | `{ client } \| { url, options? } \| { options }` | — | **Required.** Mode A supplies an existing `ioredis` client; Mode B lets the library open its own. The arms are mutually exclusive. |
|
|
235
|
+
| `prefix` | `string` | `'bull'` | Redis key namespace. Applied to the `Queue`, `Worker`, `QueueEvents` **and** `FlowProducer` — see [Architecture](#-architecture). |
|
|
236
|
+
| `defaultJobOptions` | `JobsOptions` | `attempts: 3` · exponential backoff `2000` ms with 50% jitter · `removeOnComplete: { age: 24 h, count: 1000 }` · `removeOnFail: { age: 7 d, count: 5000 }` | Merged into every job enqueued through the service; per-job options win. The bounded `removeOn*` caps are what keep Redis from growing without limit under a failure storm. |
|
|
237
|
+
| `queueOptions` | `Partial<QueueOptions>` | `{}` | Extra BullMQ `Queue` options. `connection`, `defaultJobOptions` and `prefix` are owned by the library and cannot be set here. |
|
|
238
|
+
| `flows.enabled` | `boolean` | `false` | `FlowService` is always provided; while disabled, every method except `close()` throws `FLOW_DISABLED`. |
|
|
239
|
+
| `metrics.enabled` | `boolean` | `false` | `MetricsService` is always provided; while disabled, every method throws `METRICS_DISABLED`. |
|
|
240
|
+
| `metrics.cacheTtlMs` | `number` | `5_000` | TTL of the metrics snapshot cache. `0` disables caching. Must be `>= 0`. |
|
|
241
|
+
| `telemetry` | `Telemetry` | — | A BullMQ `Telemetry` instance (typically `new BullMQOtel(...)` from the optional `bullmq-otel` peer), attached to every `Queue` and `Worker`. |
|
|
242
|
+
| `shutdown.drainTimeoutMs` | `number` | `30_000` | Upper bound on the graceful drain before workers are force-closed. A whole number from 1 to 2 147 483 647. |
|
|
243
|
+
| `shutdown.drainOnShutdown` | `boolean` | `false` | Development and test only: once the HTTP server has stopped, call `drain()` on every cached queue, which **removes** its waiting jobs (delayed and active jobs are left alone). Leave it `false` in production. |
|
|
244
|
+
| `connectionReadyTimeoutMs` | `number` | `10_000` | Mode B only: the budget for Redis to reach `ready`, a whole number from 1 to 2 147 483 647. A connection error while the client is still reconnecting does not end the wait; when the budget ends, startup fails with `CONNECTION_TIMEOUT` carrying `lastErrorCode` (a code such as `ECONNREFUSED`, never the message) and the client the library opened is closed. |
|
|
245
|
+
| `commandTimeoutMs` | `number` | `5_000` | Mode B only: the deadline for one Redis command on the connection the library opened (ioredis `commandTimeout`); an `options.commandTimeout` wins. A call that hits it rejects with `CONNECTION_TIMEOUT` (`details` carries `operation` and `timeoutMs`), as when `enqueueTimeoutMs` passes. Worker and QueueEvents connections have none. |
|
|
246
|
+
| `enqueueTimeoutMs` | `number` | `10_000` | The deadline for one `QueueService` call that reaches Redis, in either mode, including the wait for the client to be ready on a queue's first call; it rejects with `CONNECTION_TIMEOUT`, and an `enqueue` may still complete in Redis, so retry with a stable `jobId` or `deduplication`. |
|
|
247
|
+
| `isGlobal` | `boolean` | `true` | Register the module globally, so `QueueService` is injectable without re-importing. |
|
|
248
248
|
|
|
249
249
|
> [!WARNING]
|
|
250
250
|
> **`shutdown.drainOnShutdown` deletes jobs.** When it is `true`, shutdown calls `drain()` on every
|
|
@@ -440,7 +440,7 @@ When a custom `jobId` already exists, or a `deduplication` rule keeps the job th
|
|
|
440
440
|
|
|
441
441
|
### A timed-out `enqueue` may still have been added
|
|
442
442
|
|
|
443
|
-
When Redis does not answer in time, whether within `commandTimeoutMs` on the connection the library opened or within `enqueueTimeoutMs`, `enqueue` and `enqueueBulk` reject with `CONNECTION_TIMEOUT`, but the library cannot cancel the command: if Redis applies it late, the job exists anyway. Retrying a call that timed out can therefore add the job twice. For anything that must not run twice, pass a deterministic `jobId` (for example one derived from the business key): BullMQ ignores a second add with the same `jobId` for as long as the first job is still stored, so a retry inside that window is safe. A completed job is removed by `removeOnComplete`, and from then on the same `jobId` adds a new job, so keep the retry inside that window or keep the handler idempotent. A `deduplication` rule gives the same guarantee for work that is allowed to repeat after a window.
|
|
443
|
+
When Redis does not answer in time, whether within `commandTimeoutMs` on the connection the library opened or within `enqueueTimeoutMs`, or the connection is down, `enqueue` and `enqueueBulk` reject with `CONNECTION_TIMEOUT`, but the library cannot cancel the command: if Redis applies it late, the job exists anyway. Retrying a call that timed out can therefore add the job twice. For anything that must not run twice, pass a deterministic `jobId` (for example one derived from the business key): BullMQ ignores a second add with the same `jobId` for as long as the first job is still stored, so a retry inside that window is safe. A completed job is removed by `removeOnComplete`, and from then on the same `jobId` adds a new job, so keep the retry inside that window or keep the handler idempotent. A `deduplication` rule gives the same guarantee for work that is allowed to repeat after a window.
|
|
444
444
|
|
|
445
445
|
### OpenTelemetry
|
|
446
446
|
|
|
@@ -465,10 +465,15 @@ It implements a **bounded drain protocol** with at-least-once semantics:
|
|
|
465
465
|
1. All workers are closed **in parallel**, within one budget (`drainTimeoutMs`, default 30 s) that
|
|
466
466
|
covers steps 1 to 4 together: however many workers are stuck, consumption stops after one budget,
|
|
467
467
|
not one per worker.
|
|
468
|
-
2.
|
|
469
|
-
first.
|
|
470
|
-
3. On timeout, the worker is
|
|
471
|
-
|
|
468
|
+
2. Each worker is paused (it stops fetching and waits for its active jobs) and then closed, or the
|
|
469
|
+
budget runs out — whichever comes first.
|
|
470
|
+
3. On timeout, the worker is stopped with `worker.close(true)` as its first close, so it does not
|
|
471
|
+
wait for the job and its lock renewal ends; the lock expires and the job stalls and is retried
|
|
472
|
+
by the next instance. What the abandoned handler produces afterwards, a result or a failure, is
|
|
473
|
+
dropped, so the stopped worker cannot restart its connection. The handler's own promise keeps
|
|
474
|
+
running, because user code cannot be cancelled. A sandboxed (file-based) worker is the
|
|
475
|
+
exception: its job runs in a child process the registry cannot wrap, and in a measured run the
|
|
476
|
+
host process did not exit by itself after the budget ended with such a job still running.
|
|
472
477
|
4. Queue-events and the connections the library opened for workers and queue-events are closed,
|
|
473
478
|
each within what is left of the budget; a connection that has not quit by then is disconnected.
|
|
474
479
|
5. Once NestJS has stopped the HTTP server (`onApplicationShutdown`), the flow producer, the
|
|
@@ -478,7 +483,7 @@ It implements a **bounded drain protocol** with at-least-once semantics:
|
|
|
478
483
|
|
|
479
484
|
**Handlers must be idempotent.** Use `jobId`, `deduplication`, or application-level
|
|
480
485
|
idempotency keys to guard against reprocessing a job that was partially handled before
|
|
481
|
-
a crash or
|
|
486
|
+
a crash or a shutdown that outlasts the drain budget.
|
|
482
487
|
|
|
483
488
|
**DLQ pattern.** When a job exhausts its `attempts`, move it to a dead-letter queue:
|
|
484
489
|
|
|
@@ -506,14 +511,14 @@ BymaxQueueModule.forRoot({
|
|
|
506
511
|
|
|
507
512
|
## 🤔 Why over `@nestjs/bullmq`
|
|
508
513
|
|
|
509
|
-
| Concern | `@nestjs/bullmq` (official) | `@bymax-one/nest-queue`
|
|
510
|
-
| ------------- | ---------------------------------------------------- |
|
|
511
|
-
| Module setup | `BullModule.forRoot()` + `registerQueue()` per queue | Single `forRoot`/`forRootAsync`; queues created on demand by name
|
|
512
|
-
| Connection | Manual ioredis + `maxRetriesPerRequest` wiring | Dual mode (BYO / lib-owned); correct per-role retry policy applied automatically
|
|
513
|
-
| Job defaults | Per `registerQueue` | Centralized opinionated defaults with per-queue/per-job overrides
|
|
514
|
-
| Producer API | `@InjectQueue` + raw `queue.add` | Typed `enqueue<TData, TResult>()`, `enqueueBulk`, `upsertJobScheduler`
|
|
515
|
-
| Shutdown | Framework closes workers | Explicit bounded-drain protocol;
|
|
516
|
-
| Observability | Manual | `telemetry` passthrough (OpenTelemetry) + `MetricsService`
|
|
514
|
+
| Concern | `@nestjs/bullmq` (official) | `@bymax-one/nest-queue` |
|
|
515
|
+
| ------------- | ---------------------------------------------------- | ---------------------------------------------------------------------------------------- |
|
|
516
|
+
| Module setup | `BullModule.forRoot()` + `registerQueue()` per queue | Single `forRoot`/`forRootAsync`; queues created on demand by name |
|
|
517
|
+
| Connection | Manual ioredis + `maxRetriesPerRequest` wiring | Dual mode (BYO / lib-owned); correct per-role retry policy applied automatically |
|
|
518
|
+
| Job defaults | Per `registerQueue` | Centralized opinionated defaults with per-queue/per-job overrides |
|
|
519
|
+
| Producer API | `@InjectQueue` + raw `queue.add` | Typed `enqueue<TData, TResult>()`, `enqueueBulk`, `upsertJobScheduler` |
|
|
520
|
+
| Shutdown | Framework closes workers | Explicit bounded-drain protocol; stops a busy worker at the deadline; stalled accounting |
|
|
521
|
+
| Observability | Manual | `telemetry` passthrough (OpenTelemetry) + `MetricsService` |
|
|
517
522
|
|
|
518
523
|
**Choose this library** when you want a single opinionated setup, the `@bymax-one/nest-cache`
|
|
519
524
|
connection-sharing story, enforced defaults, and a tested bounded-shutdown protocol.
|
|
@@ -552,7 +557,7 @@ wire connection and defaults yourself.
|
|
|
552
557
|
(@Processor / @Process / @OnWorkerEvent)
|
|
553
558
|
│
|
|
554
559
|
QueueLifecycle
|
|
555
|
-
(bounded drain →
|
|
560
|
+
(bounded drain → stop → disconnect)
|
|
556
561
|
```
|
|
557
562
|
|
|
558
563
|
### Design Principles
|
|
@@ -732,7 +737,7 @@ import type {
|
|
|
732
737
|
| `cleanQueue` | `(queue, gracePeriodMs, limit, status?) → Promise<string[]>` | Remove up to `limit` jobs older than `gracePeriodMs` (`0` = no limit); returns removed ids |
|
|
733
738
|
| `upsertJobScheduler` | `<TData, TResult>(queue, id, repeat, template?) → Promise<Job<TData, TResult> \| undefined>` | Create or update a recurring Job Scheduler; returns the first scheduled job |
|
|
734
739
|
| `removeJobScheduler` | `(queue, id) → Promise<boolean>` | Remove a Job Scheduler by ID |
|
|
735
|
-
| `getJobSchedulers` | `(queue, start?, end?, asc?) → Promise<JobSchedulerJson[]>` | Page the schedulers registered on a queue
|
|
740
|
+
| `getJobSchedulers` | `(queue, start?, end?, asc?) → Promise<JobSchedulerJson[]>` | Page the schedulers registered on a queue (at most 1 000 per call) |
|
|
736
741
|
| `getOrCreateQueue` | `<TData, TResult>(queue, overrides?) → Queue<TData, TResult>` | Return the cached raw `Queue` instance |
|
|
737
742
|
| `getCachedQueues` | `() → ReadonlyMap<string, Queue>` | Read-only view of every queue created so far |
|
|
738
743
|
|
|
@@ -804,22 +809,24 @@ const metrics: QueueMetrics = await this.metricsService.get('email')
|
|
|
804
809
|
const all: readonly QueueMetrics[] = await this.metricsService.getAll()
|
|
805
810
|
```
|
|
806
811
|
|
|
812
|
+
`FlowService.add` and `addBulk` are held to `enqueueTimeoutMs` like every `QueueService` call and reject with `CONNECTION_TIMEOUT` when Redis does not answer in time. A flow that timed out may still have been added, so use a custom `jobId` when you retry.
|
|
813
|
+
|
|
807
814
|
### Other exports
|
|
808
815
|
|
|
809
|
-
| Export | What it is
|
|
810
|
-
| ------------------------------------------------------------------------------------------------------------------------------------- |
|
|
811
|
-
| `WorkerRegistry.unregister(queue)`, `list()` | Stop and remove a worker and its connection; list the queue names that have a worker
|
|
812
|
-
| `FlowService.addBulk(flows)`, `getProducer()`, `close()` | Add
|
|
813
|
-
| `MetricsService.invalidate(queue?)` | Drop the cached snapshot of one queue, or of all of them
|
|
814
|
-
| `QueueEventsRegistry` | The shared `QueueEvents` per queue (`getOrCreate`, `list`), used by `@OnQueueEvent`
|
|
815
|
-
| `BYMAX_QUEUE_OPTIONS`, `BYMAX_QUEUE_RESOLVED_OPTIONS` | Injection tokens for the options as given and with the defaults applied
|
|
816
|
-
| `BYMAX_QUEUE_REDIS_CLIENT`, `BYMAX_QUEUE_CONNECTION_MODE` | Injection tokens for the Queue-role ioredis client and for `'mode-a-byo'` or `'mode-b-owned'`
|
|
817
|
-
| `DEFAULT_WORKER_CONCURRENCY`, `MAX_WORKER_CONCURRENCY` | The worker concurrency used when none is given, and its ceiling
|
|
818
|
-
| `DEFAULT_JOB_OPTIONS`, `DEFAULT_METRICS_CACHE_TTL_MS`, `DEFAULT_SHUTDOWN_DRAIN_TIMEOUT_MS` | The defaults of the Configuration table
|
|
819
|
-
| `QUEUE_ERROR_MESSAGES` | The default message of each error code
|
|
820
|
-
| `BymaxQueueModuleOptions`, `BymaxQueueModuleAsyncOptions`, `BymaxQueueOptionsFactory`, `QueueConnectionConfig`, `QueueConnectionMode` | The option and connection types behind `forRoot` and `forRootAsync`
|
|
821
|
-
| `ProcessorMetadata`, `ProcessHandlerMetadata`, `QueueEventListenerMetadata` | The reflection metadata the decorators write
|
|
822
|
-
| `JobStatus`, `JOB_STATUS`, `BulkJob`, `SandboxedJob`, `WorkerEventName`, `QueueEventName`, `WorkerOptions` | Constants and types for the APIs above
|
|
816
|
+
| Export | What it is |
|
|
817
|
+
| ------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
|
|
818
|
+
| `WorkerRegistry.unregister(queue)`, `list()` | Stop and remove a worker and its connection; list the queue names that have a worker |
|
|
819
|
+
| `FlowService.addBulk(flows)`, `getProducer()`, `close()` | Add up to 1 000 flows at once; the underlying BullMQ `FlowProducer`; close it (a no-op while flows are disabled) |
|
|
820
|
+
| `MetricsService.invalidate(queue?)` | Drop the cached snapshot of one queue, or of all of them |
|
|
821
|
+
| `QueueEventsRegistry` | The shared `QueueEvents` per queue (`getOrCreate`, `list`), used by `@OnQueueEvent` |
|
|
822
|
+
| `BYMAX_QUEUE_OPTIONS`, `BYMAX_QUEUE_RESOLVED_OPTIONS` | Injection tokens for the options as given and with the defaults applied |
|
|
823
|
+
| `BYMAX_QUEUE_REDIS_CLIENT`, `BYMAX_QUEUE_CONNECTION_MODE` | Injection tokens for the Queue-role ioredis client and for `'mode-a-byo'` or `'mode-b-owned'` |
|
|
824
|
+
| `DEFAULT_WORKER_CONCURRENCY`, `MAX_WORKER_CONCURRENCY` | The worker concurrency used when none is given, and its ceiling |
|
|
825
|
+
| `DEFAULT_JOB_OPTIONS`, `DEFAULT_METRICS_CACHE_TTL_MS`, `DEFAULT_SHUTDOWN_DRAIN_TIMEOUT_MS` | The defaults of the Configuration table |
|
|
826
|
+
| `QUEUE_ERROR_MESSAGES` | The default message of each error code |
|
|
827
|
+
| `BymaxQueueModuleOptions`, `BymaxQueueModuleAsyncOptions`, `BymaxQueueOptionsFactory`, `QueueConnectionConfig`, `QueueConnectionMode` | The option and connection types behind `forRoot` and `forRootAsync` |
|
|
828
|
+
| `ProcessorMetadata`, `ProcessHandlerMetadata`, `QueueEventListenerMetadata` | The reflection metadata the decorators write |
|
|
829
|
+
| `JobStatus`, `JOB_STATUS`, `BulkJob`, `SandboxedJob`, `WorkerEventName`, `QueueEventName`, `WorkerOptions` | Constants and types for the APIs above |
|
|
823
830
|
|
|
824
831
|
---
|
|
825
832
|
|
|
@@ -850,18 +857,18 @@ try {
|
|
|
850
857
|
}
|
|
851
858
|
```
|
|
852
859
|
|
|
853
|
-
| Code | HTTP | When thrown
|
|
854
|
-
| ---------------------------------------- | :--: |
|
|
855
|
-
| `queue.invalid_options` | 500 | Module options failed bootstrap validation — missing `connection`, mutually exclusive connection arms,
|
|
856
|
-
| `queue.connection_invalid` | 500 | The resolver has no usable client: not yet initialized, Mode A's injected client is in an unusable `status`, or a Mode B client was refused by the server (`WRONGPASS`, `NOAUTH`) or stopped reconnecting during startup
|
|
857
|
-
| `queue.connection_requires_null_retries` | 500 | A consumer connection could not be coerced to `maxRetriesPerRequest: null`
|
|
858
|
-
| `queue.connection_timeout` | 500 | Mode B: Redis did not reach `ready` within `connectionReadyTimeoutMs` (`details` carries `timeoutMs` and the last error's `lastErrorCode`), or a `QueueService`/`FlowService` call got no answer
|
|
859
|
-
| `queue.invalid_repeat_options` | 400 | A Job Scheduler schedule was rejected — invalid cron pattern, or neither `pattern` nor `every`
|
|
860
|
-
| `queue.duplicate_processor` | 500 | Two `@Processor` classes claim the same queue name
|
|
861
|
-
| `queue.flow_disabled` | 503 | `FlowService` used without `flows.enabled`
|
|
862
|
-
| `queue.metrics_disabled` | 503 | `MetricsService` used without `metrics.enabled`
|
|
863
|
-
| `queue.bulk_enqueue_failed` | 500 | `enqueueBulk` exceeded the 1 000
|
|
864
|
-
| `queue.worker_registration_failed` | 500 | A worker could not be constructed, started, or closed
|
|
860
|
+
| Code | HTTP | When thrown |
|
|
861
|
+
| ---------------------------------------- | :--: | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
862
|
+
| `queue.invalid_options` | 500 | Module options failed bootstrap validation — missing `connection`, mutually exclusive connection arms, a `shutdown.drainTimeoutMs`, `connectionReadyTimeoutMs`, `commandTimeoutMs` or `enqueueTimeoutMs` that is not a whole number from 1 to 2 147 483 647, or negative `metrics.cacheTtlMs` |
|
|
863
|
+
| `queue.connection_invalid` | 500 | The resolver has no usable client: not yet initialized, Mode A's injected client is in an unusable `status`, or a Mode B client was refused by the server (`WRONGPASS`, `NOAUTH`) or stopped reconnecting during startup |
|
|
864
|
+
| `queue.connection_requires_null_retries` | 500 | A consumer connection could not be coerced to `maxRetriesPerRequest: null` |
|
|
865
|
+
| `queue.connection_timeout` | 500 | Mode B: Redis did not reach `ready` within `connectionReadyTimeoutMs` (`details` carries `timeoutMs` and the last error's `lastErrorCode`), or a `QueueService`/`FlowService` call got no answer from Redis: it timed out (`commandTimeoutMs` on the owned connection, or `enqueueTimeoutMs` for a `QueueService` call; `details` carries `operation` and `timeoutMs`) or the connection is down (`details` carries `operation` and `reason: 'offline'`); an enqueue may still have been added |
|
|
866
|
+
| `queue.invalid_repeat_options` | 400 | A Job Scheduler schedule was rejected — invalid cron pattern, or neither `pattern` nor `every` |
|
|
867
|
+
| `queue.duplicate_processor` | 500 | Two `@Processor` classes claim the same queue name |
|
|
868
|
+
| `queue.flow_disabled` | 503 | `FlowService` used without `flows.enabled` |
|
|
869
|
+
| `queue.metrics_disabled` | 503 | `MetricsService` used without `metrics.enabled` |
|
|
870
|
+
| `queue.bulk_enqueue_failed` | 500 | `enqueueBulk` or `FlowService.addBulk` exceeded the 1 000 cap, or the underlying `addBulk` failed |
|
|
871
|
+
| `queue.worker_registration_failed` | 500 | A worker could not be constructed, started, or closed |
|
|
865
872
|
|
|
866
873
|
### Reserved codes
|
|
867
874
|
|
|
@@ -869,12 +876,12 @@ These are part of the published catalog and stable, but nothing currently throws
|
|
|
869
876
|
They are listed so a consumer writing an exhaustive `switch` over `QueueErrorCode`
|
|
870
877
|
compiles today and keeps compiling when a future release starts emitting them.
|
|
871
878
|
|
|
872
|
-
| Code | Why it is not thrown today
|
|
873
|
-
| --------------------------------- |
|
|
874
|
-
| `queue.queue_not_found` | Queues are created on demand by name, so a lookup never misses
|
|
875
|
-
| `queue.job_not_found` | `getJob` resolves to `null` rather than throwing
|
|
876
|
-
| `queue.invalid_job_data` | Payloads are opaque; the library does not validate them
|
|
877
|
-
| `queue.shutdown_timeout_exceeded` | Exceeding the drain deadline
|
|
879
|
+
| Code | Why it is not thrown today |
|
|
880
|
+
| --------------------------------- | ----------------------------------------------------------------------------- |
|
|
881
|
+
| `queue.queue_not_found` | Queues are created on demand by name, so a lookup never misses |
|
|
882
|
+
| `queue.job_not_found` | `getJob` resolves to `null` rather than throwing |
|
|
883
|
+
| `queue.invalid_job_data` | Payloads are opaque; the library does not validate them |
|
|
884
|
+
| `queue.shutdown_timeout_exceeded` | Exceeding the drain deadline stops the workers still busy rather than raising |
|
|
878
885
|
|
|
879
886
|
The same catalog is exported from `@bymax-one/nest-queue/shared` with no dependencies,
|
|
880
887
|
so a frontend can map codes to messages without importing the server surface.
|
|
@@ -908,11 +915,11 @@ If you see it, the client you passed is not a plain ioredis client (a wrapper or
|
|
|
908
915
|
double whose `duplicate()` ignores its options). Pass the ioredis client itself, or let the
|
|
909
916
|
library open its own connection with Mode B. The producer keeps the client's own retry
|
|
910
917
|
policy; with a finite `maxRetriesPerRequest` that is what lets it fail fast during an outage, and
|
|
911
|
-
`
|
|
918
|
+
every `QueueService` call is bounded by `enqueueTimeoutMs` whatever the client does.
|
|
912
919
|
|
|
913
920
|
### A handler never settles (the queue stops moving)
|
|
914
921
|
|
|
915
|
-
A handler whose promise never settles holds its concurrency slot for as long as the process lives, and BullMQ keeps renewing its lock, so the stalled-job check never reclaims it: with every slot held, the rest of the queue waits. Set `jobTimeoutMs` in the worker `options` to bound it. When the time passes, the job fails with a `queue.job_timeout` error and retries under its `attempts` and `backoff`, and the slot is free again. The handler receives the job's `AbortSignal` as its second argument; it aborts at that moment, or when BullMQ cancels the job (`worker.cancelJob()`, `worker.cancelAllJobs()`), and closing the worker does not abort it; pass it to whatever the handler awaits so the work stops with the job. A handler that ignores the signal keeps running in the background, and its retry can overlap it, so it must stay idempotent. There is no default, and sandboxed workers do not accept the option.
|
|
922
|
+
A handler whose promise never settles holds its concurrency slot for as long as the process lives, and BullMQ keeps renewing its lock, so the stalled-job check never reclaims it: with every slot held, the rest of the queue waits. Set `jobTimeoutMs` in the worker `options` to bound it. When the time passes, the job fails with a `queue.job_timeout` error and retries under its `attempts` and `backoff`, and the slot is free again. The handler receives the job's `AbortSignal` as its second argument; it aborts at that moment, or when BullMQ cancels the job (`worker.cancelJob()`, `worker.cancelAllJobs()`), and closing the worker does not abort it; pass it to whatever the handler awaits so the work stops with the job. A handler that ignores the signal keeps running in the background, and its retry can overlap it, so it must stay idempotent. There is no default, and sandboxed workers do not accept the option. Once `jobTimeoutMs` has fired, `concurrency` no longer bounds how many handlers are alive: the slot is free for the next job while the timed-out handler may still be running, so a handler that ignores the signal can overlap its successors.
|
|
916
923
|
|
|
917
924
|
```typescript
|
|
918
925
|
@Processor('reports', { jobTimeoutMs: 60_000 })
|
|
@@ -930,7 +937,9 @@ Stalled jobs are re-queued automatically by BullMQ's stalled-job check. If worke
|
|
|
930
937
|
do not pick them up:
|
|
931
938
|
|
|
932
939
|
1. Verify `drainTimeoutMs` is larger than the worst-case job runtime.
|
|
933
|
-
2. Verify `lockDuration` (in the `options` you pass to `@Processor` or `register
|
|
940
|
+
2. Verify `lockDuration` (in the `options` you pass to `@Processor` or `register`; a whole
|
|
941
|
+
number from 1000 to 2 147 483 647, as is `stalledInterval`; `concurrency` and the
|
|
942
|
+
limiter values must be whole numbers too) covers
|
|
934
943
|
the handler execution time — a worker that runs longer than `lockDuration` will have its
|
|
935
944
|
lock stolen, marking the job as stalled.
|
|
936
945
|
3. BullMQ re-queues a stalled job up to its `maxStalledCount` times (default `1`) and then
|