@bymax-one/nest-queue 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md ADDED
@@ -0,0 +1,858 @@
1
+ <p align="center">
2
+ <img src="https://img.shields.io/badge/%40bymax--one-nest--queue-000000?style=for-the-badge&logo=nestjs&logoColor=E0234E" alt="@bymax-one/nest-queue" />
3
+ </p>
4
+
5
+ <h1 align="center">@bymax-one/nest-queue</h1>
6
+
7
+ <p align="center">
8
+ <strong>Typed BullMQ queues for NestJS</strong><br />
9
+ <sub>BullMQ 5 · Workers · Flows · Job Schedulers · Deduplication · OpenTelemetry · Graceful Shutdown · Zero Runtime Dependencies</sub>
10
+ </p>
11
+
12
+ <p align="center">
13
+ <a href="https://www.npmjs.com/package/@bymax-one/nest-queue"><img src="https://img.shields.io/npm/v/@bymax-one/nest-queue?style=flat-square&colorA=000000&colorB=000000" alt="npm version" /></a>
14
+ <a href="https://www.npmjs.com/package/@bymax-one/nest-queue"><img src="https://img.shields.io/npm/dm/@bymax-one/nest-queue?style=flat-square&colorA=000000&colorB=000000" alt="npm downloads" /></a>
15
+ <a href="https://github.com/bymaxone/nest-queue/actions/workflows/ci.yml"><img src="https://img.shields.io/github/actions/workflow/status/bymaxone/nest-queue/ci.yml?branch=main&style=flat-square&colorA=000000&label=CI" alt="CI status" /></a>
16
+ <a href="https://github.com/bymaxone/nest-queue/actions/workflows/ci.yml"><img src="https://img.shields.io/badge/coverage-100%25-brightgreen?style=flat-square&colorA=000000" alt="coverage" /></a>
17
+ <a href="https://github.com/bymaxone/nest-queue/blob/main/docs/mutation_testing_results.md"><img src="https://img.shields.io/badge/mutation-99.06%25-brightgreen?style=flat-square&colorA=000000" alt="mutation score" /></a>
18
+ <a href="https://scorecard.dev/viewer/?uri=github.com/bymaxone/nest-queue"><img src="https://api.scorecard.dev/projects/github.com/bymaxone/nest-queue/badge?style=flat-square" alt="OpenSSF Scorecard" /></a>
19
+ <a href="https://github.com/bymaxone/nest-queue/blob/main/LICENSE"><img src="https://img.shields.io/github/license/bymaxone/nest-queue?style=flat-square&colorA=000000&colorB=000000" alt="license" /></a>
20
+ <a href="https://www.typescriptlang.org/"><img src="https://img.shields.io/badge/TypeScript-strict-3178C6?style=flat-square&logo=typescript&logoColor=white" alt="TypeScript" /></a>
21
+ <a href="https://nodejs.org/"><img src="https://img.shields.io/badge/Node.js-24%2B-339933?style=flat-square&logo=node.js&logoColor=white" alt="Node.js" /></a>
22
+ </p>
23
+
24
+ <p align="center">
25
+ <a href="https://github.com/bymaxone/nest-queue">GitHub</a> ·
26
+ <a href="https://github.com/bymaxone/nest-queue/issues">Issues</a> ·
27
+ <a href="#-quick-start">Quick Start</a> ·
28
+ <a href="#-api-reference">API Reference</a> ·
29
+ <a href="https://github.com/bymaxone/nest-queue/tree/main/examples/nest-queue-example">Example App</a>
30
+ </p>
31
+
32
+ ---
33
+
34
+ ## ✨ Overview
35
+
36
+ `@bymax-one/nest-queue` is a NestJS dynamic module that wraps **BullMQ 5** behind a typed,
37
+ opinionated surface. Instead of wiring `ioredis` connections, per-queue registration, and
38
+ `maxRetriesPerRequest` policy by hand, you install one library and get typed producers,
39
+ decorator-based workers, flows, recurring **Job Schedulers**, native deduplication,
40
+ OpenTelemetry passthrough, and a tested bounded graceful-shutdown protocol.
41
+
42
+ The library has **zero direct dependencies** — `bullmq`, `ioredis`, and `@nestjs/*` arrive as
43
+ peer dependencies, so you control exact versions and the supply-chain surface stays minimal.
44
+
45
+ ### Why nest-queue?
46
+
47
+ - **One setup, every queue.** A single `forRoot`/`forRootAsync`; queues are created on demand by
48
+ name — no `registerQueue()` per queue.
49
+ - **Correct connections by default.** Dual-mode (bring-your-own client or lib-owned) with the
50
+ per-role `maxRetriesPerRequest` policy applied automatically — the Queue connection keeps
51
+ default retries (fail fast), only the duplicated Worker/QueueEvents connections force `null`.
52
+ - **Current BullMQ API only.** Recurring jobs use Job Schedulers (`upsertJobScheduler`), never the
53
+ removed `addRepeatable`. Cron is validated by BullMQ's own parser, never a hand-rolled regex.
54
+ - **Safe shutdown.** A bounded-drain protocol with at-least-once semantics, force-close, and
55
+ stalled-job accounting — verified against a real Redis (Testcontainers) E2E suite.
56
+
57
+ ---
58
+
59
+ ## 🔥 Features
60
+
61
+ ### 📤 Producing
62
+
63
+ - ✅ **Typed producer API** — `enqueue<TData, TResult>()` threads both generics through to the returned `Job`, so `job.data` and `job.returnvalue` need no cast
64
+ - ✅ **Bounded bulk insert** — `enqueueBulk` adds up to 1 000 jobs in a single Redis roundtrip and rejects a larger batch outright, rather than letting one call inflate the heap
65
+ - ✅ **Native deduplication** — simple, throttle, debounce and keep-last-if-active, passed straight through to BullMQ; the library writes no deduplication logic of its own
66
+ - ✅ **Job Schedulers** — cron (5-field, or 6-field with seconds) and fixed intervals through `upsertJobScheduler`; the `addRepeatable` API removed in BullMQ v6 is never called
67
+ - ✅ **Flows** — opt-in `FlowService` over BullMQ's `FlowProducer`, with fail-parent semantics preserved
68
+
69
+ ### ⚙️ Consuming
70
+
71
+ - ✅ **Decorator workers** — `@Processor`, `@Process`, `@OnWorkerEvent` and `@OnQueueEvent`, discovered automatically from your providers
72
+ - ✅ **Programmatic registry** — `WorkerRegistry.register()` for workers built at runtime, and `registerSandboxed()` for file-based out-of-process processors
73
+ - ✅ **Progress reporting** — `job.updateProgress()` with the matching worker event wired through to your handler
74
+
75
+ ### 🔌 Connections & Shutdown
76
+
77
+ - ✅ **Dual-mode connection** — bring your own `ioredis` client, or let the library open its own from a URL or an options object
78
+ - ✅ **Per-role retry policy** — `maxRetriesPerRequest: null` reaches only the duplicated Worker and QueueEvents connections; the producer keeps ioredis' defaults, so a Redis outage fails fast instead of hanging a request
79
+ - ✅ **One namespace, four objects** — the configured `prefix` reaches `Queue`, `Worker`, `QueueEvents` **and** `FlowProducer`, so producers and consumers can never end up on separate keyspaces
80
+ - ✅ **Bounded graceful shutdown** — a single ordered sequence drains every worker under a deadline and then force-closes; delivery is at-least-once by design, so handlers must be idempotent
81
+ - ✅ **No unlistened emitter** — every `Queue`, `Worker`, `QueueEvents` and `FlowProducer` the library creates carries a fallback `error` listener, so a transient Redis fault cannot become an uncaught exception in your process; your own `@OnWorkerEvent('error')` or `@OnQueueEvent('error')` still runs alongside it
82
+
83
+ ### 🧩 Developer Experience
84
+
85
+ - ✅ **Zero runtime dependencies** — BullMQ, ioredis and `@nestjs/*` all arrive as peer dependencies, so you pin the versions and the supply-chain surface stays yours
86
+ - ✅ **Two subpaths** — the server surface, and `./shared` carrying types and constants that pull in nothing at all
87
+ - ✅ **Stable error catalog** — namespaced codes exported from both subpaths, so a frontend maps a `code` without importing the server
88
+ - ✅ **Dual-format output** — ESM + CJS with declarations for each format, verified against the packed tarball on every PR
89
+ - ✅ **Opt-in observability** — OpenTelemetry passthrough via `bullmq-otel`, plus a `MetricsService` with a configurable TTL cache
90
+ - ✅ **Typed end to end** — TypeScript `strict` with `exactOptionalPropertyTypes` and `noUncheckedIndexedAccess`; zero `any`, no suppression comments
91
+
92
+ ---
93
+
94
+ ## 📦 Subpath Exports
95
+
96
+ `./shared` exists so a browser or edge bundle can import the job-status union and the
97
+ error-code catalog without pulling in `bullmq`, `ioredis`, or `@nestjs/*`. Nothing under
98
+ that subpath imports a runtime dependency.
99
+
100
+ | Subpath | Import | Purpose | Dependencies |
101
+ | -------------- | ------------------------------ | --------------------------------------------------------- | :-----------------------: |
102
+ | **`.`** | `@bymax-one/nest-queue` | Full server surface: module, services, decorators, types | NestJS · BullMQ · ioredis |
103
+ | **`./shared`** | `@bymax-one/nest-queue/shared` | Job-status union, error codes, metric and scheduler types | none |
104
+
105
+ ```
106
+ @bymax-one/nest-queue (server — NestJS + BullMQ + ioredis)
107
+ │
108
+ └── re-exports ──▶ @bymax-one/nest-queue/shared (zero dependencies)
109
+ ```
110
+
111
+ Both subpaths ship ESM **and** CommonJS with declarations for each format, so a
112
+ `require()` consumer receives CommonJS declarations rather than ESM ones. The
113
+ `pnpm check:exports` gate verifies this against the packed tarball.
114
+
115
+ ### Peer dependency matrix
116
+
117
+ | Subpath | Required peers |
118
+ | ------------ | ----------------------------------------------------------------------------------------------- |
119
+ | `.` (server) | `@nestjs/common ^11`, `@nestjs/core ^11`, `bullmq ^5.16`, `ioredis ^5`, `reflect-metadata ^0.2` |
120
+ | `./shared` | None |
121
+ | Optional | `bullmq-otel ^1` (OpenTelemetry) |
122
+
123
+ ---
124
+
125
+ > [!TIP]
126
+ > A runnable reference application lives in
127
+ > [`examples/nest-queue-example`](https://github.com/bymaxone/nest-queue/tree/main/examples/nest-queue-example) —
128
+ > a NestJS app that consumes this library and exercises producers, decorated
129
+ > processors, and a health endpoint against a real Redis.
130
+
131
+ ---
132
+
133
+ ## 🚀 Quick Start
134
+
135
+ ```bash
136
+ pnpm add @bymax-one/nest-queue bullmq ioredis
137
+ ```
138
+
139
+ Wire the module in your root `AppModule`:
140
+
141
+ ```typescript
142
+ import { Module } from '@nestjs/common'
143
+ import { BymaxQueueModule } from '@bymax-one/nest-queue'
144
+
145
+ @Module({
146
+ imports: [
147
+ BymaxQueueModule.forRoot({
148
+ connection: { url: process.env.REDIS_URL },
149
+ isGlobal: true,
150
+ }),
151
+ ],
152
+ })
153
+ export class AppModule {}
154
+ ```
155
+
156
+ Enqueue a job from any service:
157
+
158
+ ```typescript
159
+ import { Injectable } from '@nestjs/common'
160
+ import { QueueService } from '@bymax-one/nest-queue'
161
+
162
+ @Injectable()
163
+ export class EmailService {
164
+ constructor(private readonly queue: QueueService) {}
165
+
166
+ async sendWelcome(userId: string): Promise<void> {
167
+ await this.queue.enqueue<{ userId: string }, void>('email', 'welcome', { userId })
168
+ }
169
+ }
170
+ ```
171
+
172
+ Process jobs with a decorated handler:
173
+
174
+ ```typescript
175
+ import { Injectable } from '@nestjs/common'
176
+ import { Processor, Process } from '@bymax-one/nest-queue'
177
+ import type { Job } from '@bymax-one/nest-queue'
178
+
179
+ @Injectable()
180
+ @Processor('email')
181
+ export class EmailProcessor {
182
+ @Process('welcome')
183
+ async handleWelcome(job: Job<{ userId: string }, void>): Promise<void> {
184
+ // handle job.data.userId
185
+ }
186
+ }
187
+ ```
188
+
189
+ Register the processor in your module alongside `BymaxQueueModule`:
190
+
191
+ ```typescript
192
+ @Module({
193
+ providers: [EmailProcessor],
194
+ })
195
+ export class EmailModule {}
196
+ ```
197
+
198
+ ---
199
+
200
+ ## ⚙️ Configuration
201
+
202
+ Every option passed to `forRoot` / `forRootAsync`. Only `connection` is required.
203
+
204
+ | Option | Type | Default | Description |
205
+ | -------------------------- | ------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
206
+ | `connection` | `{ client } \| { url, options? } \| { options }` | — | **Required.** Mode A supplies an existing `ioredis` client; Mode B lets the library open its own. The three arms are mutually exclusive. |
207
+ | `prefix` | `string` | `'bull'` | Redis key namespace. Applied to the `Queue`, `Worker`, `QueueEvents` **and** `FlowProducer` — see [Architecture](#-architecture). |
208
+ | `defaultJobOptions` | `JobsOptions` | `attempts: 3` · exponential backoff `2000` ms · `removeOnComplete: { age: 24 h, count: 1000 }` · `removeOnFail: { age: 7 d, count: 5000 }` | Merged into every job enqueued through the service; per-job options win. The bounded `removeOn*` caps are what keep Redis from growing without limit under a failure storm. |
209
+ | `queueOptions` | `Partial<QueueOptions>` | `{}` | Extra BullMQ `Queue` options. `connection`, `defaultJobOptions` and `prefix` are owned by the library and cannot be set here. |
210
+ | `flows.enabled` | `boolean` | `false` | Registers `FlowService`. Injecting it while disabled throws `FLOW_DISABLED`. |
211
+ | `metrics.enabled` | `boolean` | `false` | Registers `MetricsService`. Injecting it while disabled throws `METRICS_DISABLED`. |
212
+ | `metrics.cacheTtlMs` | `number` | `5_000` | TTL of the metrics snapshot cache. `0` disables caching. Must be `>= 0`. |
213
+ | `telemetry` | `Telemetry` | — | A BullMQ `Telemetry` instance (typically `new BullMQOtel(...)` from the optional `bullmq-otel` peer), attached to every `Queue` and `Worker`. |
214
+ | `shutdown.drainTimeoutMs` | `number` | `30_000` | Upper bound on the graceful drain before workers are force-closed. Must be `> 0`. |
215
+ | `shutdown.drainOnShutdown` | `boolean` | `false` | Wait for the queue to drain, not just for in-flight jobs to finish. |
216
+ | `connectionReadyTimeoutMs` | `number` | `10_000` | Mode B only: how long to wait for Redis `ready` before failing with `CONNECTION_TIMEOUT`. |
217
+ | `isGlobal` | `boolean` | `true` | Register the module globally, so `QueueService` is injectable without re-importing. |
218
+
219
+ Invalid options are rejected at bootstrap with `QueueException(INVALID_OPTIONS)` — never
220
+ silently corrected. A misconfigured queue fails when the application starts, not when the
221
+ first job is enqueued.
222
+
223
+ ---
224
+
225
+ ## 🔌 Mode A — Bring Your Own Connection (recommended with `@bymax-one/nest-cache`)
226
+
227
+ Mode A accepts an existing `ioredis` client injected from your cache or connection layer.
228
+ Worker and QueueEvents connections are duplicated from this client with
229
+ `maxRetriesPerRequest: null` applied automatically; the Queue connection uses ioredis defaults.
230
+
231
+ ```typescript
232
+ import { BymaxQueueModule } from '@bymax-one/nest-queue'
233
+ import { CACHE_REDIS_CLIENT } from '@bymax-one/nest-cache'
234
+ import type { Redis } from 'ioredis'
235
+
236
+ BymaxQueueModule.forRootAsync({
237
+ inject: [CACHE_REDIS_CLIENT],
238
+ useFactory: (queueRedis: Redis) => ({
239
+ connection: { client: queueRedis },
240
+ isGlobal: true,
241
+ }),
242
+ })
243
+ ```
244
+
245
+ Mode A is the recommended setup when you already have `@bymax-one/nest-cache` in the app.
246
+ One client is shared; the lib duplicates it per role and applies the correct retry policy
247
+ without touching your cache connection.
248
+
249
+ ---
250
+
251
+ ## 🔌 Mode B — Lib-Owned Connection
252
+
253
+ Mode B lets the library own the Redis connections. Pass either a URL or raw ioredis options:
254
+
255
+ ```typescript
256
+ // URL form
257
+ BymaxQueueModule.forRoot({
258
+ connection: { url: process.env.REDIS_URL },
259
+ isGlobal: true,
260
+ })
261
+
262
+ // Options form
263
+ BymaxQueueModule.forRoot({
264
+ connection: {
265
+ options: {
266
+ host: process.env.REDIS_HOST,
267
+ port: Number(process.env.REDIS_PORT),
268
+ password: process.env.REDIS_PASSWORD,
269
+ tls: {},
270
+ },
271
+ },
272
+ isGlobal: true,
273
+ })
274
+ ```
275
+
276
+ The library opens one connection for queues and one per worker/QueueEvents listener, each
277
+ with `maxRetriesPerRequest: null` where BullMQ requires it.
278
+
279
+ ---
280
+
281
+ ## 📖 API Reference
282
+
283
+ ### `QueueService`
284
+
285
+ The central interaction point with BullMQ queues.
286
+
287
+ ```typescript
288
+ import { QueueService } from '@bymax-one/nest-queue'
289
+ import type {
290
+ Job,
291
+ JobsOptions,
292
+ JobSchedulerRepeatOptions,
293
+ QueueMetrics,
294
+ } from '@bymax-one/nest-queue'
295
+ ```
296
+
297
+ | Method | Signature | Description |
298
+ | -------------------- | -------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------ |
299
+ | `enqueue` | `<TData, TResult>(queue, name, data, opts?) → Promise<Job<TData, TResult>>` | Add a single job |
300
+ | `enqueueBulk` | `<TData, TResult>(queue, jobs) → Promise<Job<TData, TResult>[]>` | Add up to 1 000 jobs in one Redis roundtrip |
301
+ | `getJob` | `<TData, TResult>(queue, jobId) → Promise<Job<TData, TResult> \| null>` | Get job by ID; `null` when it does not exist |
302
+ | `getJobs` | `<TData, TResult>(queue, status, start?, end?) → Promise<Job<TData, TResult>[]>` | Page jobs in one status (default `0`–`50`) |
303
+ | `getMetrics` | `(queue) → Promise<QueueMetrics>` | Uncached snapshot of job counts per status, with a collection timestamp |
304
+ | `pauseQueue` | `(queue) → Promise<void>` | Pause a queue |
305
+ | `resumeQueue` | `(queue) → Promise<void>` | Resume a paused queue |
306
+ | `cleanQueue` | `(queue, graceMs, limit, status?) → Promise<string[]>` | Remove up to `limit` jobs older than `graceMs` (`0` = no limit); returns removed ids |
307
+ | `upsertJobScheduler` | `<TData, TResult>(queue, id, repeat, template?) → Promise<Job<TData, TResult> \| undefined>` | Create or update a recurring Job Scheduler; returns the first scheduled job |
308
+ | `removeJobScheduler` | `(queue, id) → Promise<boolean>` | Remove a Job Scheduler by ID |
309
+ | `getJobSchedulers` | `(queue, start?, end?, asc?) → Promise<JobSchedulerJson[]>` | Page the schedulers registered on a queue |
310
+ | `getOrCreateQueue` | `<TData, TResult>(queue, overrides?) → Queue<TData, TResult>` | Return the cached raw `Queue` instance |
311
+ | `getCachedQueues` | `() → ReadonlyMap<string, Queue>` | Read-only view of every queue created so far |
312
+
313
+ ### `WorkerRegistry`
314
+
315
+ Programmatic API to create and manage BullMQ workers at runtime.
316
+
317
+ ```typescript
318
+ import { WorkerRegistry } from '@bymax-one/nest-queue'
319
+ import type { ProgrammaticWorkerConfig, SandboxedWorkerConfig } from '@bymax-one/nest-queue'
320
+
321
+ // Register an in-process worker
322
+ registry.register({
323
+ queueName: 'email',
324
+ processor: async (job) => {
325
+ /* … */
326
+ },
327
+ options: { concurrency: 4 },
328
+ })
329
+
330
+ // Register a sandboxed (out-of-process) processor
331
+ registry.registerSandboxed({
332
+ queueName: 'heavy-compute',
333
+ processorFile: path.resolve(__dirname, 'compute.processor.js'),
334
+ options: { concurrency: 2 },
335
+ })
336
+ ```
337
+
338
+ ### `FlowService` (opt-in)
339
+
340
+ Enable flows in the module options and inject the service:
341
+
342
+ ```typescript
343
+ BymaxQueueModule.forRoot({
344
+ connection: { url: process.env.REDIS_URL },
345
+ flows: { enabled: true },
346
+ })
347
+
348
+ // Then inject:
349
+ import { FlowService } from '@bymax-one/nest-queue'
350
+ import type { FlowJob, JobNode } from '@bymax-one/nest-queue'
351
+
352
+ const tree: JobNode = await this.flows.add({
353
+ name: 'process-order',
354
+ queueName: 'orders',
355
+ data: { orderId: '123' },
356
+ children: [
357
+ { name: 'charge-payment', queueName: 'payments', data: {} },
358
+ { name: 'send-confirmation', queueName: 'email', data: {} },
359
+ ],
360
+ })
361
+ ```
362
+
363
+ ### `MetricsService` (opt-in)
364
+
365
+ Provide a per-queue snapshot with a configurable TTL cache:
366
+
367
+ ```typescript
368
+ BymaxQueueModule.forRoot({
369
+ connection: { url: process.env.REDIS_URL },
370
+ metrics: { enabled: true, cacheTtlMs: 5_000 },
371
+ })
372
+
373
+ import { MetricsService } from '@bymax-one/nest-queue'
374
+ import type { QueueMetrics } from '@bymax-one/nest-queue'
375
+
376
+ const metrics: QueueMetrics = await this.metricsService.getMetrics('email')
377
+ const all = await this.metricsService.getAll(['email', 'notifications'])
378
+ ```
379
+
380
+ ---
381
+
382
+ ## 🚨 Error Code Catalog
383
+
384
+ Whatever **this library rejects** is a `QueueException` carrying a stable,
385
+ transport-independent `code`. Branch on the code, never on the HTTP status or the
386
+ message text — the code is the contract.
387
+
388
+ Whatever **fails underneath it** is not wrapped at all. A dropped connection, a
389
+ timeout or any other operational fault reaches you as the original error, with its
390
+ own type and message, rather than being relabelled as invalid input. So the catalog
391
+ below is a complete list of the library's own rejections, and deliberately not a
392
+ complete list of everything a call can throw.
393
+
394
+ ```typescript
395
+ import { QueueException, QUEUE_ERROR_CODES } from '@bymax-one/nest-queue'
396
+
397
+ try {
398
+ await queue.enqueue('email', 'welcome', { userId })
399
+ } catch (err) {
400
+ if (err instanceof QueueException) {
401
+ const { code } = (err.getResponse() as { error: { code: string } }).error
402
+ if (code === QUEUE_ERROR_CODES.BULK_ENQUEUE_FAILED) {
403
+ /* … */
404
+ }
405
+ }
406
+ }
407
+ ```
408
+
409
+ | Code | HTTP | When thrown |
410
+ | ---------------------------------------- | :--: | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
411
+ | `queue.invalid_options` | 500 | Module options failed bootstrap validation — missing `connection`, mutually exclusive connection arms, non-positive `shutdown.drainTimeoutMs`, or negative `metrics.cacheTtlMs` |
412
+ | `queue.connection_invalid` | 500 | The resolver has no usable client: not yet initialized, or Mode A's injected client is in an unusable `status` |
413
+ | `queue.connection_requires_null_retries` | 500 | A consumer connection could not be coerced to `maxRetriesPerRequest: null` |
414
+ | `queue.connection_timeout` | 500 | Mode B: Redis did not reach `ready` within `connectionReadyTimeoutMs` |
415
+ | `queue.invalid_repeat_options` | 400 | A Job Scheduler schedule was rejected — invalid cron pattern, or neither `pattern` nor `every` |
416
+ | `queue.duplicate_processor` | 500 | Two `@Processor` classes claim the same queue name |
417
+ | `queue.flow_disabled` | 503 | `FlowService` used without `flows.enabled` |
418
+ | `queue.metrics_disabled` | 503 | `MetricsService` used without `metrics.enabled` |
419
+ | `queue.bulk_enqueue_failed` | 500 | `enqueueBulk` exceeded the 1 000-job cap, or the underlying `addBulk` failed |
420
+ | `queue.worker_registration_failed` | 500 | A worker could not be constructed, started, or closed |
421
+
422
+ ### Reserved codes
423
+
424
+ These are part of the published catalog and stable, but nothing currently throws them.
425
+ They are listed so a consumer writing an exhaustive `switch` over `QueueErrorCode`
426
+ compiles today and keeps compiling when a future release starts emitting them.
427
+
428
+ | Code | Why it is not thrown today |
429
+ | --------------------------------- | --------------------------------------------------------------------- |
430
+ | `queue.queue_not_found` | Queues are created on demand by name, so a lookup never misses |
431
+ | `queue.job_not_found` | `getJob` resolves to `null` rather than throwing |
432
+ | `queue.invalid_job_data` | Payloads are opaque; the library does not validate them |
433
+ | `queue.shutdown_timeout_exceeded` | Exceeding the drain deadline force-closes workers rather than raising |
434
+
435
+ The same catalog is exported from `@bymax-one/nest-queue/shared` with no dependencies,
436
+ so a frontend can map codes to messages without importing the server surface.
437
+
438
+ ---
439
+
440
+ ## 🎀 Decorators
441
+
442
+ ### `@Processor(queueName)`
443
+
444
+ Marks a class as a worker bound to `queueName`. The class must be an `@Injectable()` registered as a NestJS provider.
445
+
446
+ ### `@Process(jobName?)`
447
+
448
+ Method decorator for the handler. If `jobName` is omitted, the method handles all jobs on the queue. If provided, only jobs with that `name` are dispatched here.
449
+
450
+ ```typescript
451
+ @Process()
452
+ async handleAll(job: Job): Promise<void> { /* … */ }
453
+
454
+ @Process('send-email')
455
+ async handleSendEmail(job: Job<EmailPayload>): Promise<void> { /* … */ }
456
+ ```
457
+
458
+ ### `@OnWorkerEvent(eventName)`
459
+
460
+ Listens to BullMQ worker-local events (`'completed'`, `'failed'`, `'progress'`, `'active'`, `'stalled'`, …).
461
+
462
+ ```typescript
463
+ @OnWorkerEvent('failed')
464
+ onFailed(job: Job | undefined, err: Error): void {
465
+ console.error(`Job ${job?.id} failed:`, err.message)
466
+ }
467
+ ```
468
+
469
+ ### `@OnQueueEvent(eventName)`
470
+
471
+ Listens to BullMQ `QueueEvents` (global, cross-process events). Requires `queueEvents: { enabled: true }` in module options.
472
+
473
+ ```typescript
474
+ @OnQueueEvent('completed')
475
+ onCompleted(jobId: string, returnValue: string): void {
476
+ console.log(`Job ${jobId} completed`)
477
+ }
478
+ ```
479
+
480
+ ---
481
+
482
+ ## ⏰ Job Schedulers
483
+
484
+ Job Schedulers are the current recurring-jobs API in BullMQ (superseding the removed `addRepeatable` API). Use `upsertJobScheduler` to create or update a scheduler:
485
+
486
+ ```typescript
487
+ // Cron-based (every day at 9 AM)
488
+ await this.queue.upsertJobScheduler(
489
+ 'notifications',
490
+ 'daily-digest',
491
+ { pattern: '0 9 * * *' },
492
+ { name: 'send-digest', data: { type: 'daily' } },
493
+ )
494
+
495
+ // Interval-based (every 5 minutes)
496
+ await this.queue.upsertJobScheduler(
497
+ 'health-check',
498
+ 'ping',
499
+ { every: 5 * 60 * 1000 },
500
+ { name: 'ping', data: {} },
501
+ )
502
+
503
+ // Remove
504
+ await this.queue.removeJobScheduler('notifications', 'daily-digest')
505
+
506
+ // List
507
+ const schedulers = await this.queue.getJobSchedulers('notifications')
508
+ ```
509
+
510
+ Cron-with-seconds (6-field patterns) is supported. Patterns are validated by BullMQ's
511
+ own `cron-parser`, not a hand-rolled regex.
512
+
513
+ ---
514
+
515
+ ## 🔁 Deduplication & Telemetry
516
+
517
+ ### Deduplication
518
+
519
+ Pass the `deduplication` option directly on `enqueue` — the library delegates the behavior entirely to BullMQ:
520
+
521
+ ```typescript
522
+ // Simple: collapse duplicates until the first in-flight job completes
523
+ await this.queue.enqueue(
524
+ 'email',
525
+ 'welcome',
526
+ { userId },
527
+ {
528
+ deduplication: { id: `welcome:${userId}` },
529
+ },
530
+ )
531
+
532
+ // Throttle: ignore duplicates for 30 seconds
533
+ await this.queue.enqueue(
534
+ 'email',
535
+ 'welcome',
536
+ { userId },
537
+ {
538
+ deduplication: { id: `welcome:${userId}`, ttl: 30_000 },
539
+ },
540
+ )
541
+
542
+ // Debounce: keep only the latest; each duplicate resets the TTL
543
+ await this.queue.enqueue(
544
+ 'email',
545
+ 'welcome',
546
+ { userId },
547
+ {
548
+ deduplication: { id: `welcome:${userId}`, ttl: 5_000, extend: true, replace: true },
549
+ },
550
+ )
551
+ ```
552
+
553
+ ### OpenTelemetry
554
+
555
+ Pass a `bullmq-otel` telemetry instance (optional peer dep) to trace spans from `enqueue` through the handler:
556
+
557
+ ```typescript
558
+ import { BullMQOtel } from 'bullmq-otel'
559
+
560
+ BymaxQueueModule.forRoot({
561
+ connection: { url: process.env.REDIS_URL },
562
+ telemetry: new BullMQOtel('my-service', tracer),
563
+ })
564
+ ```
565
+
566
+ ---
567
+
568
+ ## 🛑 Graceful Shutdown
569
+
570
+ `QueueLifecycle` is registered and invoked automatically via NestJS `onApplicationShutdown`.
571
+ It implements a **bounded drain protocol** with at-least-once semantics:
572
+
573
+ 1. All workers are closed with a grace period (`drainTimeoutMs`, default 30 s).
574
+ 2. A `Promise.race` resolves either when the worker drains normally or when the
575
+ timeout fires — whichever comes first.
576
+ 3. On timeout, the worker is force-closed (`worker.close(true)`), preserving stalled
577
+ jobs for reprocessing by the next instance.
578
+ 4. In Mode B, Redis connections are disconnected after all workers are closed.
579
+
580
+ **Handlers must be idempotent.** Use `jobId`, `deduplication`, or application-level
581
+ idempotency keys to guard against reprocessing a job that was partially handled before
582
+ a crash or force-close.
583
+
584
+ **DLQ pattern.** When a job exhausts its `attempts`, move it to a dead-letter queue:
585
+
586
+ ```typescript
587
+ @OnWorkerEvent('failed')
588
+ async onFailed(job: Job | undefined, err: Error): Promise<void> {
589
+ if (job && job.attemptsMade >= (job.opts.attempts ?? 1)) {
590
+ await this.queue.enqueue(`${job.queueName}-dlq`, job.name, job.data, {
591
+ jobId: `dlq:${job.id}`,
592
+ })
593
+ }
594
+ }
595
+ ```
596
+
597
+ Tune `drainTimeoutMs` when jobs may run longer than 30 s:
598
+
599
+ ```typescript
600
+ BymaxQueueModule.forRoot({
601
+ connection: { url: process.env.REDIS_URL },
602
+ shutdown: { drainTimeoutMs: 60_000 },
603
+ })
604
+ ```
605
+
606
+ ---
607
+
608
+ ## 🤔 Why over `@nestjs/bullmq`
609
+
610
+ | Concern | `@nestjs/bullmq` (official) | `@bymax-one/nest-queue` |
611
+ | ------------- | ---------------------------------------------------- | -------------------------------------------------------------------------------- |
612
+ | Module setup | `BullModule.forRoot()` + `registerQueue()` per queue | Single `forRoot`/`forRootAsync`; queues created on demand by name |
613
+ | Connection | Manual ioredis + `maxRetriesPerRequest` wiring | Dual mode (BYO / lib-owned); correct per-role retry policy applied automatically |
614
+ | Job defaults | Per `registerQueue` | Centralized opinionated defaults with per-queue/per-job overrides |
615
+ | Producer API | `@InjectQueue` + raw `queue.add` | Typed `enqueue<TData, TResult>()`, `enqueueBulk`, `upsertJobScheduler` |
616
+ | Shutdown | Framework closes workers | Explicit bounded-drain protocol; force-close + stalled accounting |
617
+ | Observability | Manual | `telemetry` passthrough (OpenTelemetry) + `MetricsService` |
618
+
619
+ **Choose this library** when you want a single opinionated setup, the `@bymax-one/nest-cache`
620
+ connection-sharing story, enforced defaults, and a tested bounded-shutdown protocol.
621
+
622
+ **Choose `@nestjs/bullmq`** when you want the thinnest official binding and prefer to
623
+ wire connection and defaults yourself.
624
+
625
+ ---
626
+
627
+ ## 🏗️ Architecture
628
+
629
+ ```
630
+ BymaxQueueModule.forRoot / forRootAsync
631
+ │
632
+ validateOptions (fail fast)
633
+ │
634
+ applyDefaults (frozen)
635
+ │
636
+ ┌─────────────┴─────────────┐
637
+ │ ConnectionResolver │
638
+ │ Mode A: your ioredis │
639
+ │ Mode B: lib-owned │
640
+ └─────────────┬─────────────┘
641
+ │
642
+ ┌──────────────┬─────────────────┼─────────────────┬──────────────┐
643
+ │ │ │ │ │
644
+ QueueService WorkerRegistry QueueEventsRegistry FlowService MetricsService
645
+ (Queue) (Worker) (QueueEvents) (FlowProducer) (opt-in)
646
+ │ │ │ │
647
+ default duplicate() duplicate() default
648
+ retries maxRetries: null maxRetries: null retries
649
+ │ │ │ │
650
+ └──────────────┴────── same `prefix` ──────────────┘
651
+ │
652
+ ProcessorDiscoveryService
653
+ (@Processor / @Process / @OnWorkerEvent)
654
+ │
655
+ QueueLifecycle
656
+ (bounded drain → force close → disconnect)
657
+ ```
658
+
659
+ ### Design Principles
660
+
661
+ | Principle | Description |
662
+ | ----------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
663
+ | 🔌 **Connections are role-shaped** | BullMQ blocks a connection for consumers. Only the duplicated `Worker`/`QueueEvents` connections get `maxRetriesPerRequest: null`; the `Queue` and `FlowProducer` connection keeps ioredis' default retries, so a producer fails fast during a Redis outage instead of hanging a request. |
664
+ | 🏷️ **One namespace, four objects** | The configured `prefix` reaches `Queue`, `Worker`, `QueueEvents` and `FlowProducer`. BullMQ defaults an omitted prefix to `bull`, so a partial application splits producers from consumers silently. |
665
+ | 💥 **Fail at boot, not at runtime** | Options are validated before any connection is opened, and every rejection carries a distinct reason. |
666
+ | 🧬 **Explicit DI tokens** | Tokens are `Symbol()` and every injectable constructor parameter carries an explicit `@Inject`. The package ships bundled, where design-time metadata is not emitted — implicit resolution would break inside your build, not ours. |
667
+ | 📅 **Current BullMQ API only** | Recurring jobs use Job Schedulers; the `addRepeatable` API removed in BullMQ v6 is never called. Cron is validated by BullMQ's own parser. |
668
+ | 🧊 **Zero runtime dependencies** | `dependencies` is `{}`. You choose the exact `bullmq` and `ioredis` versions, and the supply-chain surface stays yours. |
669
+
670
+ ---
671
+
672
+ ## 🔐 Security Model
673
+
674
+ This library moves your application's data through Redis and runs your code against it.
675
+ Its security contract is about what it does with credentials, what it puts in a payload,
676
+ and which keyspace a job lands in.
677
+
678
+ ### Credentials never leave the configuration
679
+
680
+ Connection details arrive through injected options — the library never reads
681
+ `process.env` itself, so credentials live wherever your configuration layer keeps them.
682
+ No connection string, password, or TLS material is written to a log line, an exception
683
+ message, or an exception payload. `QueueException.details` carries only scalar
684
+ configuration values (`{ actualValue, expectedValue, limit, received }`), never a
685
+ connection object and never `job.data`.
686
+
687
+ ### Job data is opaque
688
+
689
+ The library treats `job.data` as a payload to transport, never to inspect. It is never
690
+ deep-merged (which would expose a prototype-pollution path from a queued payload), never
691
+ logged, and never copied into an error. Whatever you enqueue is what your handler
692
+ receives — so a payload holding a secret is a decision you make, and Redis is where it
693
+ will sit until the job is removed.
694
+
695
+ ### The prefix is a namespace, not a security boundary
696
+
697
+ `prefix` isolates one tenant's or one environment's keys from another's inside a single
698
+ Redis database. It is enforced consistently — `Queue`, `Worker`, `QueueEvents` and
699
+ `FlowProducer` all receive it — so jobs cannot be produced into one namespace and
700
+ consumed from another. It is **not** an access control: anything that can reach the Redis
701
+ instance can read every namespace on it. Cross-tenant isolation that must survive a
702
+ compromised client belongs in Redis ACLs or separate instances.
703
+
704
+ ### Shutdown is bounded and at-least-once
705
+
706
+ `QueueLifecycle` drains in-flight work under a deadline and then force-closes. A job still
707
+ running when the deadline expires is **not** acknowledged, so BullMQ's stalled-job check
708
+ re-queues it and it runs again. Delivery is at-least-once by design: **every handler must
709
+ be idempotent.** A handler that charges a card or sends an email without an idempotency
710
+ key will do it twice at some point.
711
+
712
+ ### Security Checklist
713
+
714
+ - [ ] Redis reachable only from the application network — never exposed publicly.
715
+ - [ ] Redis requires authentication (`requirepass` or ACL user) and TLS in transit.
716
+ - [ ] A distinct `prefix` per tenant or environment, and a distinct Redis ACL user or
717
+ instance where the isolation must hold against a compromised client.
718
+ - [ ] Every `@Process` handler is idempotent — retries and stalled-job recovery both
719
+ replay work.
720
+ - [ ] Job payloads carry identifiers, not secrets. If a payload must reference a secret,
721
+ store a reference and resolve it inside the handler.
722
+ - [ ] `removeOnComplete` / `removeOnFail` are bounded, so payloads do not accumulate in
723
+ Redis indefinitely.
724
+ - [ ] `shutdown.drainTimeoutMs` exceeds your worst-case job runtime, so a normal deploy
725
+ does not force-close work that was about to finish.
726
+
727
+ ---
728
+
729
+ ## 🛡️ Security Table
730
+
731
+ | Layer | Implementation |
732
+ | ----------------- | ---------------------------------------------------------------------------------------------------------- |
733
+ | Credentials | Injected options only; never read from `process.env`, never logged, never placed in an exception |
734
+ | Error payloads | `QueueException.details` restricted to scalar configuration values — no `job.data`, no connection object |
735
+ | Job payloads | Treated as opaque; never deep-merged (prototype-pollution guard), never logged |
736
+ | Namespacing | `prefix` applied uniformly to `Queue`, `Worker`, `QueueEvents` and `FlowProducer` |
737
+ | Connection policy | `maxRetriesPerRequest: null` confined to duplicated consumer connections; producers keep fail-fast retries |
738
+ | Delivery | At-least-once with a bounded drain; unacknowledged work is re-queued by BullMQ's stalled check |
739
+ | Supply chain | `dependencies: {}`; SHA-pinned Actions, OSV-Scanner, TruffleHog, OpenSSF Scorecard; npm publish over OIDC |
740
+ | Input validation | Options validated at bootstrap; cron patterns validated by BullMQ's own parser, never a hand-rolled regex |
741
+
742
+ > [!IMPORTANT]
743
+ > **Delivery is at-least-once, and `prefix` is not an access boundary.** Handlers must be
744
+ > idempotent, and anything with credentials for the Redis instance can read every
745
+ > namespace on it.
746
+
747
+ ---
748
+
749
+ ## 🧱 Tech Stack
750
+
751
+ - **Runtime:** Node.js 24+
752
+ - **Framework:** NestJS 11 (`ConfigurableModuleBuilder` + `setExtras`)
753
+ - **Queue engine:** BullMQ `^5.16` (peer) over `ioredis ^5` (peer)
754
+ - **Telemetry:** `bullmq-otel ^1` (optional peer)
755
+ - **Build:** tsup — ESM + CJS per subpath, with `.d.ts` _and_ `.d.cts` declarations
756
+ - **Tests:** Jest + `@testcontainers/redis` (E2E) + Stryker (mutation)
757
+ - **TypeScript:** 5.x strict (`noUncheckedIndexedAccess`, `exactOptionalPropertyTypes`), zero `any`
758
+
759
+ ---
760
+
761
+ ## 🧪 Testing & Quality
762
+
763
+ - **100% line/branch coverage** on every implemented file (hard CI gate).
764
+ - **Mutation score 99.06%** (Stryker `break 95`) — see [`docs/mutation_testing_results.md`](docs/mutation_testing_results.md).
765
+ - **Real-Redis E2E** via Testcontainers (workers, flows, schedulers, deduplication, retries,
766
+ DLQ, graceful shutdown).
767
+ - **Type-level API tests** (`pnpm test:types`) pin the published signatures, so a refactor
768
+ that widens or loosens one fails the build instead of reaching consumers.
769
+ - **Published-shape gates**: `pnpm check:exports` resolves every entrypoint the way each
770
+ module-resolution mode would (all four `attw` quadrants green, `node10` included), and
771
+ `pnpm smoke` resolves the packed tarball from a real consumer in ESM _and_ CommonJS.
772
+ - **Bundle budgets** enforced (`pnpm size`): server ≤ 18 KiB brotli, shared ≤ 2.5 KiB.
773
+ - **Supply chain:** SHA-pinned GitHub Actions, OSV-Scanner, TruffleHog, OpenSSF Scorecard,
774
+ npm publish with provenance (OIDC).
775
+
776
+ ```bash
777
+ pnpm typecheck && pnpm test:types && pnpm lint && pnpm test:cov:all && \
778
+ pnpm build && pnpm size && pnpm check:exports && pnpm smoke
779
+ ```
780
+
781
+ ---
782
+
783
+ ## 🩺 Troubleshooting
784
+
785
+ ### `CONNECTION_REQUIRES_NULL_RETRIES`
786
+
787
+ This error fires in Mode A when the injected client was not configured with
788
+ `maxRetriesPerRequest: null`. BullMQ workers require this setting to block on
789
+ Redis operations rather than giving up after a few retries.
790
+
791
+ If you use `@bymax-one/nest-cache`, pass a queue-dedicated client with `maxRetriesPerRequest: null`:
792
+
793
+ ```typescript
794
+ // In your cache module, create a second ioredis client for the queue
795
+ const queueRedis = new Redis({ ..., maxRetriesPerRequest: null })
796
+ ```
797
+
798
+ The library duplicates the client you pass and applies the correct setting; ensure
799
+ the **original** client passed to Mode A is usable as a duplication source (ioredis
800
+ allows duplicate on any state).
801
+
802
+ ### Jobs get stuck (not processed after restart)
803
+
804
+ Stalled jobs are re-queued automatically by BullMQ's stalled-job check. If workers
805
+ do not pick them up:
806
+
807
+ 1. Verify `drainTimeoutMs` is larger than the worst-case job runtime.
808
+ 2. Verify `lockDuration` (in `workerOptions`) covers the handler execution time —
809
+ a worker that runs longer than `lockDuration` will have its lock stolen, marking
810
+ the job as stalled.
811
+ 3. Check `worker.maxStalledCount` is `> 0` (default `1`) so stalled jobs re-queue.
812
+
813
+ ---
814
+
815
+ ## 🚫 What This Library Does NOT Do
816
+
817
+ - **NestJS only** — does not work with plain Express, standalone Fastify, or other frameworks.
818
+ - **Node.js only** — Deno and Bun are not supported.
819
+ - **BullMQ `^5.16` (1.x floor)** — v6 promotion is planned once the E2E suite is
820
+ green on the v5 + v6 matrix; see the BullMQ version policy in
821
+ [`CHANGELOG.md`](CHANGELOG.md).
822
+ - **No Redis Cluster sharding** — BullMQ's cluster support is not exercised by this library.
823
+ - **No built-in DLQ helper** — use the `@OnWorkerEvent('failed')` pattern above.
824
+ - **No Prometheus metrics export** — `getMetrics()` returns snapshots; publish them
825
+ via `prom-client` in your own health endpoint.
826
+
827
+ For the full list of limitations and their alternatives, see
828
+ [`docs/technical_specification.md § 16`](docs/technical_specification.md).
829
+
830
+ ---
831
+
832
+ ## 🤝 Contributing
833
+
834
+ Pull requests are welcome. Please open an issue first for significant changes.
835
+
836
+ - Read [`docs/technical_specification.md`](docs/technical_specification.md) for architecture decisions.
837
+ - Run the full gate listed in [`CONTRIBUTING.md`](CONTRIBUTING.md) before opening a PR.
838
+ - Conventional Commits are enforced by `commitlint.config.cjs`.
839
+
840
+ ---
841
+
842
+ ## 🔒 Security Policy
843
+
844
+ If you discover a security vulnerability, please **do not** open a public issue. Instead, email us
845
+ at **support@bymax.one** with details. We take security seriously and will respond promptly. See
846
+ [`SECURITY.md`](SECURITY.md) for the full policy.
847
+
848
+ ---
849
+
850
+ ## 📄 License
851
+
852
+ [MIT](./LICENSE) © [Bymax One](https://github.com/bymaxone)
853
+
854
+ ---
855
+
856
+ <p align="center">
857
+ <sub>Built with ❤️ by <a href="https://github.com/bymaxone">Bymax One</a></sub>
858
+ </p>