@fluojs/queue 1.0.1 → 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.ko.md +70 -8
- package/README.md +70 -8
- package/dist/dead-letter-manager.d.ts +10 -1
- package/dist/dead-letter-manager.d.ts.map +1 -1
- package/dist/dead-letter-manager.js +62 -0
- package/dist/helpers.d.ts +3 -1
- package/dist/helpers.d.ts.map +1 -1
- package/dist/helpers.js +6 -1
- package/dist/index.d.ts +2 -2
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +1 -1
- package/dist/module.d.ts +2 -2
- package/dist/module.d.ts.map +1 -1
- package/dist/module.js +88 -14
- package/dist/service.d.ts +50 -5
- package/dist/service.d.ts.map +1 -1
- package/dist/service.js +182 -44
- package/dist/status.d.ts +3 -1
- package/dist/status.d.ts.map +1 -1
- package/dist/status.js +33 -0
- package/dist/tokens.d.ts +55 -1
- package/dist/tokens.d.ts.map +1 -1
- package/dist/tokens.js +111 -1
- package/dist/types.d.ts +33 -0
- package/dist/types.d.ts.map +1 -1
- package/dist/worker-discovery.d.ts +2 -1
- package/dist/worker-discovery.d.ts.map +1 -1
- package/dist/worker-discovery.js +2 -2
- package/package.json +5 -5
package/README.ko.md
CHANGED
|
@@ -9,6 +9,7 @@ fluo를 위한 Redis 기반 분산 작업 처리 패키지입니다. 데코레
|
|
|
9
9
|
- [설치](#설치)
|
|
10
10
|
- [사용 시점](#사용-시점)
|
|
11
11
|
- [빠른 시작](#빠른-시작)
|
|
12
|
+
- [NestJS Queue Worker에서 마이그레이션](#nestjs-queue-worker에서-마이그레이션)
|
|
12
13
|
- [일반적인 패턴](#일반적인-패턴)
|
|
13
14
|
- [공개 API 개요](#공개-api-개요)
|
|
14
15
|
- [관련 패키지](#관련-패키지)
|
|
@@ -20,6 +21,8 @@ fluo를 위한 Redis 기반 분산 작업 처리 패키지입니다. 데코레
|
|
|
20
21
|
npm install @fluojs/queue @fluojs/redis
|
|
21
22
|
```
|
|
22
23
|
|
|
24
|
+
`@fluojs/queue`는 package manifest의 `engines.node` 선언에 따라 Node.js `>=20.0.0`이 필요합니다. fluo 애플리케이션의 나머지 부분이 runtime-portable API를 사용하더라도 이 패키지 수준 요구사항은 그대로 적용됩니다.
|
|
25
|
+
|
|
23
26
|
## 사용 시점
|
|
24
27
|
|
|
25
28
|
- 실행 시간이 길거나 리소스를 많이 사용하는 작업을 백그라운드에서 처리해야 할 때.
|
|
@@ -78,6 +81,20 @@ export class OrderService {
|
|
|
78
81
|
export class AppModule {}
|
|
79
82
|
```
|
|
80
83
|
|
|
84
|
+
## NestJS Queue Worker에서 마이그레이션
|
|
85
|
+
|
|
86
|
+
NestJS queue integration에서 이동하는 consumer는 metadata 기반 processor discovery를 fluo의 명시적 module 및 worker 계약으로 바꿔야 합니다. 이는 compatibility mode가 아니라 source migration입니다.
|
|
87
|
+
|
|
88
|
+
1. Backing Redis client를 `RedisModule.forRoot(...)`로 등록한 뒤 queue를 소유하는 module graph에서 `QueueModule.forRoot(...)`를 import합니다. NestJS async-module shape를 복사하거나 Queue가 environment configuration을 암묵적으로 읽을 것이라고 기대하지 마세요.
|
|
89
|
+
2. `@Processor(...)`, `@Process(...)` 또는 그 밖의 NestJS/Bull provider metadata를 TC39 표준 class decorator인 `@QueueWorker(JobClass, options?)`로 바꿉니다. 각 worker는 호출 가능한 `handle(job)` 메서드를 노출해야 합니다.
|
|
90
|
+
3. Decorated worker class를 singleton으로 `@Module({ providers: [...] })`에 추가합니다. Queue는 compiled provider/controller registration을 scan하며, `@Injectable()` metadata, emit된 constructor type, 임의로 import된 class는 scan하지 않습니다. Constructor dependency는 `@Inject(...)`로 명시적으로 선언합니다.
|
|
91
|
+
4. Worker가 queue registration에서 도달 가능하도록 유지합니다. 기본 global `QueueModule.forRoot()`는 compiled application graph 전체의 singleton worker를 discovery할 수 있습니다. `global: false`에서는 authored imports/exports를 통해 해당 registration에 도달할 수 있는 module로 discovery가 제한되며, 일치하는 Redis provider도 같은 module tree에서 도달 가능해야 합니다.
|
|
92
|
+
5. Queue lifecycle ownership과 중복되는 worker 소유 start/stop hook을 제거합니다. Queue는 application bootstrap 중 resource를 만들고 application bootstrap-ready handoff 이후에만 BullMQ processor를 시작하며, shutdown이 시작된 뒤에는 새 enqueue를 거부하고 active processor shutdown을 `workerShutdownTimeoutMs`로 제한한 다음 force-close를 요청합니다.
|
|
93
|
+
|
|
94
|
+
Cutover 전에는 persistence identity 차이를 반영하세요. NestJS Bull/BullMQ는 하나의 `queueName` 아래 여러 named job 값을 영속화할 수 있습니다. 반면 fluo는 job type마다 queue/worker pair 하나를 만들면서 worker의 `jobName`을 BullMQ queue name과 named job 양쪽에 사용합니다. 따라서 `jobName`만 설정해서는 여러 named job이 하나의 `queueName`을 공유하는 legacy topology를 보존할 수 없고, `@fluojs/queue`는 NestJS decorator metadata를 해석하거나 기존 serialized payload를 자동 변환하지 않습니다.
|
|
95
|
+
|
|
96
|
+
애플리케이션이 persisted-job cutover 방식을 선택해야 합니다. Producer를 전환하기 전에 기존 worker로 legacy queue를 drain하거나, 호환 payload를 변환해 fluo의 job별 queue로 다시 enqueue하거나, legacy worker가 이전 작업을 drain하는 동안 fluo에 별도 queue name을 사용하세요. 어떤 경로든 payload class shape, retry/backoff 설정, shutdown budget을 검증한 뒤 producer와 singleton `@QueueWorker(JobClass)` provider를 같은 `QueueModule.forRoot(...)` graph에 배포해야 합니다. `global: false`에서는 worker와 Redis reachability를 보존하고, 처리는 bootstrap-ready handoff 이후에만 시작하며 shutdown은 `workerShutdownTimeoutMs`로 제한된다는 점을 기억하세요.
|
|
97
|
+
|
|
81
98
|
## 일반적인 패턴
|
|
82
99
|
|
|
83
100
|
### 이름 있는 Redis 클라이언트
|
|
@@ -90,11 +107,38 @@ QueueModule.forRoot({ clientName: 'jobs' })
|
|
|
90
107
|
|
|
91
108
|
`@fluojs/queue`는 애플리케이션 부트스트랩 중 해당 Redis 클라이언트를 조회한 뒤 BullMQ용으로 큐가 소유하는 duplicate 연결을 만듭니다. 공유 `@fluojs/redis` 클라이언트의 소유권은 `RedisModule`에 남아 있으며, Queue는 자신이 만든 BullMQ duplicate 연결만 닫습니다. 이 duplicate 연결은 BullMQ Worker가 요구하는 `maxRetriesPerRequest: null` 설정으로 구성되어 시작 동작이 BullMQ의 실제 런타임 제약과 일치합니다.
|
|
92
109
|
|
|
110
|
+
`QueueModule.forRoot({ global: false })`를 사용하면 각 queue 등록은 해당 `QueueModule.forRoot(...)` 호출을 가져온 동일한 module tree에서 도달할 수 있는 worker만 탐색합니다. 서로 다른 scoped queue feature module은 서로 분리된 상태를 유지하며, Redis client provider도 같은 module tree 안에서 도달 가능해야 합니다.
|
|
111
|
+
|
|
112
|
+
### 범위가 지정된 Queue 등록
|
|
113
|
+
|
|
114
|
+
애플리케이션이 non-global queue 등록을 둘 이상 가져오면 명시적인 `scope`를 사용하세요. Scope 이름은 trim되며, 비어 있으면 안 되고, 컴파일된 module graph 안에서 고유해야 합니다. `QueueModule.forRoot({ global: false })`를 두 번 가져오는 duplicate default scoped registration이나 `QueueModule.forRoot({ global: false, scope: 'jobs' })`를 두 번 가져오는 duplicate explicit scope는 bootstrap 중 결정적인 오류로 실패합니다.
|
|
115
|
+
|
|
116
|
+
```typescript
|
|
117
|
+
import { Inject, Module } from '@fluojs/core';
|
|
118
|
+
import { getQueueLifecycleServiceToken, getQueueToken, QueueModule, type Queue } from '@fluojs/queue';
|
|
119
|
+
|
|
120
|
+
const EMAIL_QUEUE = getQueueToken('email');
|
|
121
|
+
const EMAIL_QUEUE_LIFECYCLE = getQueueLifecycleServiceToken('email');
|
|
122
|
+
|
|
123
|
+
@Inject(EMAIL_QUEUE)
|
|
124
|
+
export class EmailPublisher {
|
|
125
|
+
constructor(private readonly queue: Queue) {}
|
|
126
|
+
}
|
|
127
|
+
|
|
128
|
+
@Module({
|
|
129
|
+
imports: [QueueModule.forRoot({ global: false, scope: 'email' })],
|
|
130
|
+
providers: [EmailPublisher, EmailWorker],
|
|
131
|
+
})
|
|
132
|
+
export class EmailQueueModule {}
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
애플리케이션에 기본 queue 등록이 하나뿐이고 compatibility `QUEUE` 토큰이나 `QueueLifecycleService` 클래스를 직접 주입할 때만 `scope`를 생략하세요. Scoped registration에서는 각 feature module이 기본 compatibility token 대신 자신의 queue instance를 resolve하도록 `getQueueToken(scope)` 또는 `getQueueLifecycleServiceToken(scope)`를 주입하세요.
|
|
136
|
+
|
|
93
137
|
### 부트스트랩 및 종료 수명 주기
|
|
94
138
|
|
|
95
|
-
Queue는 애플리케이션 부트스트랩 중 worker를 탐색하고 Queue가 소유하는 BullMQ 리소스를 만들지만, BullMQ worker processor는 runtime이 전체 애플리케이션 bootstrap/readiness sequence 완료를 표시한 뒤에만 시작합니다. 다른 `onApplicationBootstrap()` hook에서 enqueue한 job은 Queue 서비스가 초기화된 뒤에는 받을 수 있으며, processor는 뒤에 실행되는 async bootstrap hook이나 애플리케이션 readiness보다 앞서 실행되지 않고 bootstrap-ready handoff 이후 실행됩니다.
|
|
139
|
+
Queue는 애플리케이션 부트스트랩 중 worker를 탐색하고 Queue가 소유하는 BullMQ 리소스를 만들지만, BullMQ worker processor는 runtime이 전체 애플리케이션 bootstrap/readiness sequence 완료를 표시한 뒤에만 시작합니다. 다른 `onApplicationBootstrap()` hook에서 enqueue한 job은 Queue 서비스가 초기화된 뒤에는 받을 수 있으며, processor는 뒤에 실행되는 async bootstrap hook이나 애플리케이션 readiness보다 앞서 실행되지 않고 bootstrap-ready handoff 이후 실행됩니다. Queue status는 해당 BullMQ processor가 실제로 시작될 때까지 degraded readiness를 보고합니다. Processor 시작에 실패하면 lifecycle이 `failed`로 이동하고, status snapshot은 worker를 ready로 숨기지 않고 실패를 노출합니다.
|
|
96
140
|
|
|
97
|
-
애플리케이션 종료가 시작되면 Queue는 상태를 `stopping`으로 바꾸고 새 enqueue를 거부한 다음 Queue 소유 worker/queue/connection을 닫고 pending dead-letter write
|
|
141
|
+
애플리케이션 종료가 시작되면 Queue는 상태를 `stopping`으로 바꾸고 새 enqueue를 거부한 다음 Queue 소유 worker/queue/connection을 닫고 pending dead-letter write의 drain을 시도합니다. Queue가 각 pending dead-letter write를 기다리는 시간은 최대 `5_000ms`입니다. 이 대기가 timeout되면 Queue는 timeout을 기록하고 해당 write를 pending count에서 제외한 뒤, record가 Redis에 도달했다는 보장 없이 종료를 계속합니다. Worker 종료에는 별도로 `workerShutdownTimeoutMs` bounded wait가 적용되므로 끝나지 않는 active processor가 애플리케이션 종료를 무기한 막을 수 없습니다. 이 timeout이 지나면 Queue는 로그를 남기고 BullMQ worker에 force-close를 요청한 뒤 나머지 리소스 정리를 계속합니다.
|
|
98
142
|
|
|
99
143
|
### 분산 재시도 (Distributed Retries)
|
|
100
144
|
|
|
@@ -113,6 +157,18 @@ Queue는 애플리케이션 부트스트랩 중 worker를 탐색하고 Queue가
|
|
|
113
157
|
|
|
114
158
|
`QueueModule.forRoot()`는 기본적으로 작업별 최근 데드 레터 엔트리 `1_000`개만 유지합니다. 무제한 보관이 꼭 필요하면 `defaultDeadLetterMaxEntries: false`로 opt-out 하고, 더 엄격한 운영 예산이 필요하면 더 작은 양의 정수를 지정하세요.
|
|
115
159
|
|
|
160
|
+
Queue의 Redis key를 직접 읽지 않고 record를 확인하려면 `QueueLifecycleService.inspectDeadLetters(jobName, { limit })` 또는 주입한 `Queue` facade의 같은 메서드를 사용하세요.
|
|
161
|
+
|
|
162
|
+
```typescript
|
|
163
|
+
const inspection = await queue.inspectDeadLetters('ProcessOrderJob', { limit: 25 });
|
|
164
|
+
|
|
165
|
+
for (const record of inspection.records) {
|
|
166
|
+
console.log(record.jobId, record.failedAt, record.errorMessage);
|
|
167
|
+
}
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
Inspection은 read-only이며 유효한 record를 최신순으로 반환합니다. Redis read를 worker lifecycle state로 gate하지 않으므로 inspection이 worker를 시작하지 않으며, backing Redis client에 접근 가능한 동안에는 Queue가 `idle`이거나 worker startup이 `failed`에 도달한 뒤에도 사용할 수 있습니다. Queue는 shared Redis client를 소유하지 않습니다. `RedisModule`이 해당 client를 종료한 뒤에는 post-shutdown availability를 보장하지 않고 backing Redis operation error를 그대로 전달합니다. Limit은 기본적으로 저장된 entry `100`개이며 최대 `1_000`개로 제한되고, 잘못된 limit은 기본값으로 대체됩니다. Malformed stored value는 결과에서 제외되고 해당 inspection window의 `malformedRecordCount`에 집계됩니다. `payload`는 `unknown`으로 유지되므로 애플리케이션 코드가 자신의 job data를 직접 narrow해야 합니다. Inspection은 job이나 dead-letter record를 삭제, replay 또는 mutate하지 않습니다.
|
|
171
|
+
|
|
116
172
|
Job은 JSON으로 직렬화 가능한 plain object여야 합니다. Queue는 enqueue 전에 job payload를 직렬화하고, worker 측에서 job prototype을 다시 입힙니다.
|
|
117
173
|
|
|
118
174
|
저수준 provider 조합을 루트 barrel API의 일부가 아니라 내부 구현 세부사항으로 취급해야 합니다. 저수준 provider helper는 문서화된 루트 barrel 계약에 포함되지 않습니다.
|
|
@@ -122,33 +178,39 @@ Job은 JSON으로 직렬화 가능한 plain object여야 합니다. Queue는 enq
|
|
|
122
178
|
### 핵심 구성 요소
|
|
123
179
|
- `QueueModule`: 큐 기능을 위한 기본 모듈입니다.
|
|
124
180
|
- `QueueModule.forRoot(options)`: 애플리케이션 수준 큐 등록을 구성합니다.
|
|
125
|
-
- `QueueLifecycleService`:
|
|
181
|
+
- `QueueLifecycleService`: 작업 enqueue, read-only dead-letter inspection, lifecycle/status snapshot 생성(`enqueue(job)`, `inspectDeadLetters(jobName, options?)`, `createPlatformStatusSnapshot()`)을 위한 기본 서비스입니다.
|
|
126
182
|
- `@QueueWorker(JobClass, options?)`: 특정 작업을 처리할 핸들러를 지정하는 데코레이터입니다.
|
|
127
183
|
- `QUEUE`: queue facade를 위한 호환성 주입 토큰입니다.
|
|
184
|
+
- `getQueueToken(scope?)`: Queue facade token helper입니다. `scope`를 생략하면 기본 `QUEUE` token을 반환하고, 비어 있지 않은 scope는 해당 scoped registration의 facade token을 반환합니다.
|
|
185
|
+
- `getQueueLifecycleServiceToken(scope?)`: Scoped queue registration을 위한 lifecycle service token helper입니다.
|
|
128
186
|
- `createQueuePlatformStatusSnapshot(...)`: lifecycle/readiness diagnostics를 위한 status snapshot helper입니다.
|
|
129
187
|
|
|
130
188
|
|
|
131
189
|
### 타입
|
|
132
|
-
- `Queue`: 애플리케이션 코드와 `QUEUE` 토큰에서 사용하는 `enqueue(job)`
|
|
190
|
+
- `Queue`: 애플리케이션 코드와 `QUEUE` 토큰에서 사용하는 `enqueue(job)` 및 read-only `inspectDeadLetters(jobName, options?)` facade입니다.
|
|
191
|
+
- `QueueDeadLetterInspectionOptions`: Bounded dead-letter inspection 설정(`limit`) 타입입니다.
|
|
192
|
+
- `QueueDeadLetterInspectionResult`: 최신순의 유효 record와 inspection window의 `malformedRecordCount`를 제공하는 결과 타입입니다.
|
|
193
|
+
- `QueueDeadLetterRecord`: `unknown` 애플리케이션 payload를 포함하는 typed dead-letter metadata입니다.
|
|
133
194
|
- `QueueJobType`: job payload class를 식별하고 rehydrate하는 데 사용하는 constructor 타입입니다.
|
|
134
195
|
- `QueueModuleOptions`: 전역 큐 설정(`global`, clientName, 기본 시도 횟수, `defaultBackoff`, 동시성, 전송률 제한, dead-letter retention 등)을 위한 타입입니다.
|
|
135
196
|
- `QueueWorkerOptions`: 개별 작업 설정(시도 횟수, 백오프, 동시성, jobName, 전송률 제한 등)을 위한 타입입니다.
|
|
136
197
|
- `QueueBackoffType`: 지원되는 retry backoff strategy 이름(`fixed`, `exponential`)입니다.
|
|
137
198
|
- `QueueBackoffOptions`: 재시도 백오프 설정(`type`, `delayMs`)을 위한 타입입니다.
|
|
138
199
|
- `QueueRateLimiterOptions`: worker 수준 distributed rate limiter 설정(`max`, `duration`)을 위한 타입입니다.
|
|
139
|
-
- `QueueLifecycleState`: Queue status adapter가 보고하는 lifecycle state(`idle`, `starting`, `started`, `stopping`, `stopped`)입니다.
|
|
140
|
-
- `QueueStatusAdapterInput`: `createQueuePlatformStatusSnapshot(...)`에 전달하는 normalized queue metrics 타입입니다.
|
|
141
|
-
- `QueuePlatformStatusSnapshot`: status helper
|
|
200
|
+
- `QueueLifecycleState`: Queue status adapter가 보고하는 lifecycle state(`idle`, `starting`, `started`, `stopping`, `stopped`, `failed`)입니다.
|
|
201
|
+
- `QueueStatusAdapterInput`: `createQueuePlatformStatusSnapshot(...)`에 전달하는 normalized queue metrics와 worker-start diagnostics 타입입니다.
|
|
202
|
+
- `QueuePlatformStatusSnapshot`: status helper와 `QueueLifecycleService.createPlatformStatusSnapshot()`이 반환하는 Queue 전용 readiness, health, ownership, detail snapshot 타입입니다.
|
|
142
203
|
|
|
143
204
|
`QueueModuleOptions`에는 `workerShutdownTimeoutMs`, `defaultDeadLetterMaxEntries` 같은 lifecycle 및 dead-letter retention 설정도 포함됩니다.
|
|
144
205
|
|
|
145
206
|
`QueueModuleOptions` 수명 주기/status 설정:
|
|
146
207
|
|
|
147
208
|
- `global`: queue module 등록을 global로 만들지 여부입니다. 기본값은 `true`이며, queue provider를 importing module graph 안에만 scope하고 싶으면 `false`를 지정합니다.
|
|
209
|
+
- `scope`: 고유한 non-empty queue registration scope입니다. 하나의 앱에 non-global queue registration이 여러 개 있으면 필요합니다.
|
|
148
210
|
- `workerShutdownTimeoutMs`: 종료 중 active worker processor를 기다리는 최대 시간입니다. 시간이 지나면 BullMQ worker를 force-close합니다. 기본값은 `30_000`입니다.
|
|
149
211
|
- `defaultDeadLetterMaxEntries`: job별로 유지할 dead-letter record의 최대 개수이며, trimming을 끄려면 `false`를 지정합니다. 기본값은 `1_000`입니다.
|
|
150
212
|
|
|
151
|
-
`createQueuePlatformStatusSnapshot(...)
|
|
213
|
+
`QueueLifecycleService.createPlatformStatusSnapshot()`은 `createQueuePlatformStatusSnapshot(...)`과 같은 공개 snapshot 계약을 사용합니다. Queue가 `started`에 도달하고 탐색된 모든 BullMQ worker processor가 시작된 뒤에만 readiness를 `ready`로 보고합니다. 이 조건이 유지되는 동안 pending dead-letter write가 있어도 readiness는 `ready`를 유지하지만, pending count가 0으로 돌아올 때까지 health는 degraded입니다. Processor가 아직 pending인 `started` resource와 `starting`은 degraded readiness, `stopping`은 not-ready/degraded, `stopped`는 not-ready/unhealthy, worker-start failure는 `workerStartFailures`와 `lastWorkerStartFailure` details를 포함해 not-ready/unhealthy로 보고합니다. Snapshot details에는 Redis dependency id, lifecycle state, ready/discovered worker 수, pending dead-letter write 수, `5_000ms` dead-letter drain timeout, `workerShutdownTimeoutMs`가 포함됩니다.
|
|
152
214
|
|
|
153
215
|
singleton `@QueueWorker()` provider/controller만 등록됩니다. request/transient worker는 discovery 중 건너뜁니다.
|
|
154
216
|
|
package/README.md
CHANGED
|
@@ -9,6 +9,7 @@ Redis-backed distributed job processing for fluo. It features decorator-based wo
|
|
|
9
9
|
- [Installation](#installation)
|
|
10
10
|
- [When to use](#when-to-use)
|
|
11
11
|
- [Quick Start](#quick-start)
|
|
12
|
+
- [Migrating from NestJS Queue Workers](#migrating-from-nestjs-queue-workers)
|
|
12
13
|
- [Common Patterns](#common-patterns)
|
|
13
14
|
- [Public API](#public-api)
|
|
14
15
|
- [Related Packages](#related-packages)
|
|
@@ -20,6 +21,8 @@ Redis-backed distributed job processing for fluo. It features decorator-based wo
|
|
|
20
21
|
npm install @fluojs/queue @fluojs/redis
|
|
21
22
|
```
|
|
22
23
|
|
|
24
|
+
`@fluojs/queue` requires Node.js `>=20.0.0`, as declared by `engines.node` in the package manifest. This package-level requirement still applies when the rest of a fluo application uses runtime-portable APIs.
|
|
25
|
+
|
|
23
26
|
## When to Use
|
|
24
27
|
|
|
25
28
|
- When you need to process long-running or resource-intensive tasks in the background.
|
|
@@ -78,6 +81,20 @@ export class OrderService {
|
|
|
78
81
|
export class AppModule {}
|
|
79
82
|
```
|
|
80
83
|
|
|
84
|
+
## Migrating from NestJS Queue Workers
|
|
85
|
+
|
|
86
|
+
Consumers moving from NestJS queue integrations must replace metadata-driven processor discovery with fluo's explicit module and worker contract. This is a source migration, not a compatibility mode:
|
|
87
|
+
|
|
88
|
+
1. Register the backing Redis client with `RedisModule.forRoot(...)`, then import `QueueModule.forRoot(...)` from the module graph that owns the queue. Do not copy NestJS async-module shapes or expect Queue to read environment configuration implicitly.
|
|
89
|
+
2. Replace `@Processor(...)`, `@Process(...)`, or other NestJS/Bull provider metadata with the TC39 standard class decorator `@QueueWorker(JobClass, options?)`. Each worker must expose a callable `handle(job)` method.
|
|
90
|
+
3. Add the decorated worker class to `@Module({ providers: [...] })` as a singleton. Queue scans compiled provider/controller registrations; it does not scan `@Injectable()` metadata, emitted constructor types, or arbitrary imported classes. Declare constructor dependencies explicitly with `@Inject(...)`.
|
|
91
|
+
4. Keep the worker reachable from the queue registration. The default global `QueueModule.forRoot()` can discover singleton workers across the compiled application graph. With `global: false`, discovery is limited to modules that can reach that specific registration through their authored imports/exports, and the matching Redis provider must be reachable from the same module tree.
|
|
92
|
+
5. Remove worker-owned start/stop hooks that duplicate Queue lifecycle ownership. Queue creates resources during application bootstrap, starts BullMQ processors only after the application bootstrap-ready handoff, rejects new enqueue calls after shutdown starts, and bounds active processor shutdown with `workerShutdownTimeoutMs` before requesting a force-close.
|
|
93
|
+
|
|
94
|
+
Before cutover, account for the persistence identity mismatch. NestJS Bull/BullMQ can persist multiple named job values under one `queueName`. fluo instead uses the worker's `jobName` as both the BullMQ queue name and the named job when it creates one queue/worker pair for each job type. Setting `jobName` alone therefore cannot preserve a legacy topology in which multiple named jobs share one `queueName`, and `@fluojs/queue` does not interpret NestJS decorator metadata or transform an existing serialized payload.
|
|
95
|
+
|
|
96
|
+
Choose an application-owned persisted-job cutover: drain the legacy queue with the old workers before switching producers; transform and re-enqueue compatible payloads into fluo's per-job queues; or use separate queue names for fluo while legacy workers drain old work. In every path, verify the payload class shape, retry/backoff settings, and shutdown budget, then deploy producers and singleton `@QueueWorker(JobClass)` providers through the same `QueueModule.forRoot(...)` graph. For `global: false`, preserve worker and Redis reachability, and remember that processing starts only after the bootstrap-ready handoff and shutdown is bounded by `workerShutdownTimeoutMs`.
|
|
97
|
+
|
|
81
98
|
## Common Patterns
|
|
82
99
|
|
|
83
100
|
### Named Redis Client
|
|
@@ -90,11 +107,38 @@ QueueModule.forRoot({ clientName: 'jobs' })
|
|
|
90
107
|
|
|
91
108
|
`@fluojs/queue` resolves that Redis client during application bootstrap, then creates queue-owned duplicate connections for BullMQ. The shared `@fluojs/redis` client remains owned by `RedisModule`; Queue closes only the duplicate BullMQ connections it creates. Those duplicate connections are configured with BullMQ's required `maxRetriesPerRequest: null` worker setting so startup behavior matches BullMQ's runtime constraints.
|
|
92
109
|
|
|
110
|
+
When `QueueModule.forRoot({ global: false })` is used, each queue registration only discovers workers that are reachable from the same module tree that imported that specific `QueueModule.forRoot(...)` call. Separate scoped queue feature modules stay isolated from one another, and the Redis client provider must be reachable from that same module tree.
|
|
111
|
+
|
|
112
|
+
### Scoped Queue Registrations
|
|
113
|
+
|
|
114
|
+
Use an explicit `scope` when an application imports more than one non-global queue registration. Scope names are trimmed, must be non-empty, and must be unique per compiled module graph. Duplicate default scoped registrations such as two `QueueModule.forRoot({ global: false })` imports, or duplicate explicit scopes such as two `QueueModule.forRoot({ global: false, scope: 'jobs' })` imports, fail deterministically during bootstrap.
|
|
115
|
+
|
|
116
|
+
```typescript
|
|
117
|
+
import { Inject, Module } from '@fluojs/core';
|
|
118
|
+
import { getQueueLifecycleServiceToken, getQueueToken, QueueModule, type Queue } from '@fluojs/queue';
|
|
119
|
+
|
|
120
|
+
const EMAIL_QUEUE = getQueueToken('email');
|
|
121
|
+
const EMAIL_QUEUE_LIFECYCLE = getQueueLifecycleServiceToken('email');
|
|
122
|
+
|
|
123
|
+
@Inject(EMAIL_QUEUE)
|
|
124
|
+
export class EmailPublisher {
|
|
125
|
+
constructor(private readonly queue: Queue) {}
|
|
126
|
+
}
|
|
127
|
+
|
|
128
|
+
@Module({
|
|
129
|
+
imports: [QueueModule.forRoot({ global: false, scope: 'email' })],
|
|
130
|
+
providers: [EmailPublisher, EmailWorker],
|
|
131
|
+
})
|
|
132
|
+
export class EmailQueueModule {}
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
Omit `scope` only when the application has a single default queue registration and injects the compatibility `QUEUE` token or `QueueLifecycleService` class directly. For scoped registrations, inject `getQueueToken(scope)` or `getQueueLifecycleServiceToken(scope)` so each feature module resolves its own queue instance instead of the default compatibility token.
|
|
136
|
+
|
|
93
137
|
### Bootstrap and Shutdown Lifecycle
|
|
94
138
|
|
|
95
|
-
Queue discovers workers and creates queue-owned BullMQ resources during application bootstrap, but BullMQ worker processors are started only after the runtime marks the full application bootstrap/readiness sequence complete. Jobs enqueued by other `onApplicationBootstrap()` hooks can be accepted once the Queue service is initialized, and their processors run after the bootstrap-ready handoff instead of racing ahead of later async bootstrap hooks or application readiness.
|
|
139
|
+
Queue discovers workers and creates queue-owned BullMQ resources during application bootstrap, but BullMQ worker processors are started only after the runtime marks the full application bootstrap/readiness sequence complete. Jobs enqueued by other `onApplicationBootstrap()` hooks can be accepted once the Queue service is initialized, and their processors run after the bootstrap-ready handoff instead of racing ahead of later async bootstrap hooks or application readiness. Queue status reports degraded readiness until those BullMQ processors have actually started; if a processor fails to start, the lifecycle moves to `failed` and status snapshots expose the failure instead of reporting the workers as ready.
|
|
96
140
|
|
|
97
|
-
Application shutdown marks Queue as `stopping`, rejects new enqueue attempts, closes queue-owned workers/queues/connections, and
|
|
141
|
+
Application shutdown marks Queue as `stopping`, rejects new enqueue attempts, closes queue-owned workers/queues/connections, and then attempts to drain pending dead-letter writes. Queue waits at most `5_000ms` for each pending dead-letter write. If that wait times out, Queue logs the timeout, stops counting the write as pending, and continues shutdown without guaranteeing that the record reached Redis. Worker shutdown is separately bounded by `workerShutdownTimeoutMs` so an active processor that never settles cannot block application shutdown indefinitely. When that timeout elapses, Queue logs the timeout and asks BullMQ to force-close the worker before continuing resource cleanup.
|
|
98
142
|
|
|
99
143
|
### Distributed Retries
|
|
100
144
|
|
|
@@ -113,6 +157,18 @@ When a worker exhausts its retry attempts, Queue appends a dead-letter record to
|
|
|
113
157
|
|
|
114
158
|
`QueueModule.forRoot()` keeps the most recent `1_000` dead-letter entries per job by default. Set `defaultDeadLetterMaxEntries: false` to opt out, or provide a smaller positive number when operators need a tighter retention budget.
|
|
115
159
|
|
|
160
|
+
Use `QueueLifecycleService.inspectDeadLetters(jobName, { limit })` or the same method on an injected `Queue` facade to inspect records without reading Queue's Redis keys directly:
|
|
161
|
+
|
|
162
|
+
```typescript
|
|
163
|
+
const inspection = await queue.inspectDeadLetters('ProcessOrderJob', { limit: 25 });
|
|
164
|
+
|
|
165
|
+
for (const record of inspection.records) {
|
|
166
|
+
console.log(record.jobId, record.failedAt, record.errorMessage);
|
|
167
|
+
}
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
Inspection is read-only and returns valid records in newest-first order. It reads Redis without lifecycle-gating the operation, so inspection does not start workers and remains usable while Queue is `idle` or after worker startup reaches `failed`, as long as the backing Redis client is reachable. Queue does not own the shared Redis client; after `RedisModule` shuts that client down, inspection propagates the backing Redis operation error instead of promising post-shutdown availability. The limit defaults to `100` stored entries and is capped at `1_000`; invalid limits fall back to the default. Malformed stored values are omitted and counted in `malformedRecordCount` for the inspected window, and `payload` remains `unknown` so application code must narrow its own job data. Inspection does not delete, replay, or mutate jobs or dead-letter records.
|
|
171
|
+
|
|
116
172
|
Jobs must be JSON-serializable plain objects. Queue serializes the job payload before enqueueing and rehydrates the job prototype on the worker side.
|
|
117
173
|
|
|
118
174
|
Treat low-level provider assembly as an internal implementation detail: low-level provider helpers are not part of the documented root-barrel contract.
|
|
@@ -122,33 +178,39 @@ Treat low-level provider assembly as an internal implementation detail: low-leve
|
|
|
122
178
|
### Core
|
|
123
179
|
- `QueueModule`: Main entry point for queue registration.
|
|
124
180
|
- `QueueModule.forRoot(options)`: Registers queue support for an application module.
|
|
125
|
-
- `QueueLifecycleService`: Primary service for enqueuing jobs (`enqueue(job)`).
|
|
181
|
+
- `QueueLifecycleService`: Primary service for enqueuing jobs, read-only dead-letter inspection, and lifecycle/status snapshots (`enqueue(job)`, `inspectDeadLetters(jobName, options?)`, `createPlatformStatusSnapshot()`).
|
|
126
182
|
- `@QueueWorker(JobClass, options?)`: Decorator to mark a class as a job handler.
|
|
127
183
|
- `QUEUE`: Compatibility injection token for the queue facade.
|
|
184
|
+
- `getQueueToken(scope?)`: Queue facade token helper. Omitting `scope` returns the default `QUEUE` token; a non-empty scope returns that scoped registration's facade token.
|
|
185
|
+
- `getQueueLifecycleServiceToken(scope?)`: Lifecycle service token helper for scoped queue registrations.
|
|
128
186
|
- `createQueuePlatformStatusSnapshot(...)`: Status snapshot helper for lifecycle/readiness diagnostics.
|
|
129
187
|
|
|
130
188
|
|
|
131
189
|
### Types
|
|
132
|
-
- `Queue`:
|
|
190
|
+
- `Queue`: Application facade with `enqueue(job)` and read-only `inspectDeadLetters(jobName, options?)` for application code and the `QUEUE` token.
|
|
191
|
+
- `QueueDeadLetterInspectionOptions`: Bounded dead-letter inspection settings (`limit`).
|
|
192
|
+
- `QueueDeadLetterInspectionResult`: Newest-first valid records plus `malformedRecordCount` for the inspected window.
|
|
193
|
+
- `QueueDeadLetterRecord`: Typed dead-letter metadata with an `unknown` application payload.
|
|
133
194
|
- `QueueJobType`: Constructor type used to identify and rehydrate a job payload class.
|
|
134
195
|
- `QueueModuleOptions`: Global queue settings (`global`, clientName, default attempts, `defaultBackoff`, concurrency, rate limiting, dead-letter retention).
|
|
135
196
|
- `QueueWorkerOptions`: Per-job settings (attempts, backoff, concurrency, jobName, rate limiting).
|
|
136
197
|
- `QueueBackoffType`: Supported retry backoff strategy names (`fixed`, `exponential`).
|
|
137
198
|
- `QueueBackoffOptions`: Retry backoff settings (`type`, `delayMs`).
|
|
138
199
|
- `QueueRateLimiterOptions`: Worker-level distributed rate limiter settings (`max`, `duration`).
|
|
139
|
-
- `QueueLifecycleState`: Lifecycle states reported by Queue status adapters (`idle`, `starting`, `started`, `stopping`, `stopped`).
|
|
140
|
-
- `QueueStatusAdapterInput`: Normalized queue metrics passed to `createQueuePlatformStatusSnapshot(...)`.
|
|
141
|
-
- `QueuePlatformStatusSnapshot`: Queue-specific readiness, health, ownership, and detail snapshot returned by the status helper.
|
|
200
|
+
- `QueueLifecycleState`: Lifecycle states reported by Queue status adapters (`idle`, `starting`, `started`, `stopping`, `stopped`, `failed`).
|
|
201
|
+
- `QueueStatusAdapterInput`: Normalized queue metrics and worker-start diagnostics passed to `createQueuePlatformStatusSnapshot(...)`.
|
|
202
|
+
- `QueuePlatformStatusSnapshot`: Queue-specific readiness, health, ownership, and detail snapshot returned by the status helper and `QueueLifecycleService.createPlatformStatusSnapshot()`.
|
|
142
203
|
|
|
143
204
|
`QueueModuleOptions` also includes lifecycle and dead-letter retention controls such as `workerShutdownTimeoutMs` and `defaultDeadLetterMaxEntries`.
|
|
144
205
|
|
|
145
206
|
`QueueModuleOptions` lifecycle/status controls:
|
|
146
207
|
|
|
147
208
|
- `global`: whether the queue module registration is global. Defaults to `true`; set `false` when queue providers should stay scoped to the importing module graph.
|
|
209
|
+
- `scope`: unique non-empty queue registration scope. Required when multiple non-global queue registrations exist in one app.
|
|
148
210
|
- `workerShutdownTimeoutMs`: maximum time to wait for active worker processors during shutdown before force-closing the BullMQ worker. Defaults to `30_000`.
|
|
149
211
|
- `defaultDeadLetterMaxEntries`: maximum retained dead-letter records per job, or `false` to disable trimming. Defaults to `1_000`.
|
|
150
212
|
|
|
151
|
-
`createQueuePlatformStatusSnapshot(...)
|
|
213
|
+
`QueueLifecycleService.createPlatformStatusSnapshot()` uses the same public snapshot contract as `createQueuePlatformStatusSnapshot(...)`. It reports readiness as `ready` only after Queue reaches `started` and every discovered BullMQ worker processor has started. While those conditions remain true, pending dead-letter writes keep readiness `ready` but degrade health until the pending count returns to zero. `started` resources with pending processors report degraded readiness, `starting` reports degraded readiness, `stopping` reports not-ready/degraded, `stopped` reports not-ready/unhealthy, and worker-start failures report not-ready/unhealthy with `workerStartFailures` and `lastWorkerStartFailure` details. Snapshot details include the Redis dependency id, lifecycle state, ready/discovered worker counts, pending dead-letter writes, the `5_000ms` dead-letter drain timeout, and `workerShutdownTimeoutMs`.
|
|
152
214
|
|
|
153
215
|
Only singleton `@QueueWorker()` providers/controllers are registered. Request/transient workers are skipped during discovery.
|
|
154
216
|
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
import type { ApplicationLogger } from '@fluojs/runtime';
|
|
2
|
-
import type { NormalizedQueueModuleOptions, QueueWorkerDescriptor } from './types.js';
|
|
2
|
+
import type { NormalizedQueueModuleOptions, QueueDeadLetterInspectionOptions, QueueDeadLetterInspectionResult, QueueWorkerDescriptor } from './types.js';
|
|
3
3
|
/**
|
|
4
4
|
* Describes the queue dead letter job contract.
|
|
5
5
|
*/
|
|
@@ -16,6 +16,7 @@ export interface QueueDeadLetterJob {
|
|
|
16
16
|
* Describes the queue redis dead letter client contract.
|
|
17
17
|
*/
|
|
18
18
|
export interface QueueRedisDeadLetterClient {
|
|
19
|
+
lrange(key: string, start: number, stop: number): Promise<string[]>;
|
|
19
20
|
ltrim(key: string, start: number, stop: number): Promise<unknown>;
|
|
20
21
|
rpush(key: string, value: string): Promise<unknown>;
|
|
21
22
|
}
|
|
@@ -29,6 +30,14 @@ export declare class QueueDeadLetterManager {
|
|
|
29
30
|
private readonly pendingWrites;
|
|
30
31
|
constructor(options: NormalizedQueueModuleOptions, logger: ApplicationLogger, getRedisClient: () => QueueRedisDeadLetterClient);
|
|
31
32
|
get pendingWriteCount(): number;
|
|
33
|
+
/**
|
|
34
|
+
* Reads and parses a bounded dead-letter snapshot without mutating Redis state.
|
|
35
|
+
*
|
|
36
|
+
* @param jobName Queue worker job name whose dead letters should be inspected.
|
|
37
|
+
* @param options Optional inspection limit, capped at `1_000` stored entries.
|
|
38
|
+
* @returns Valid records in newest-first order and the number of malformed entries omitted.
|
|
39
|
+
*/
|
|
40
|
+
inspect(jobName: string, options?: QueueDeadLetterInspectionOptions): Promise<QueueDeadLetterInspectionResult>;
|
|
32
41
|
trackTerminalFailure(descriptor: QueueWorkerDescriptor, job: QueueDeadLetterJob | undefined, error: Error): void;
|
|
33
42
|
drainPendingWrites(): Promise<void>;
|
|
34
43
|
private appendDeadLetterRecord;
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"dead-letter-manager.d.ts","sourceRoot":"","sources":["../src/dead-letter-manager.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAE,iBAAiB,EAAE,MAAM,iBAAiB,CAAC;AAGzD,OAAO,KAAK,
|
|
1
|
+
{"version":3,"file":"dead-letter-manager.d.ts","sourceRoot":"","sources":["../src/dead-letter-manager.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAE,iBAAiB,EAAE,MAAM,iBAAiB,CAAC;AAGzD,OAAO,KAAK,EACV,4BAA4B,EAC5B,gCAAgC,EAChC,+BAA+B,EAE/B,qBAAqB,EACtB,MAAM,YAAY,CAAC;AAQpB;;GAEG;AACH,MAAM,WAAW,kBAAkB;IACjC,YAAY,EAAE,MAAM,CAAC;IACrB,IAAI,EAAE,OAAO,CAAC;IACd,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,EAAE,CAAC,EAAE,MAAM,CAAC;IACZ,IAAI,EAAE;QACJ,QAAQ,CAAC,EAAE,MAAM,CAAC;KACnB,CAAC;CACH;AAED;;GAEG;AACH,MAAM,WAAW,0BAA0B;IACzC,MAAM,CAAC,GAAG,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,IAAI,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,EAAE,CAAC,CAAC;IACpE,KAAK,CAAC,GAAG,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,IAAI,EAAE,MAAM,GAAG,OAAO,CAAC,OAAO,CAAC,CAAC;IAClE,KAAK,CAAC,GAAG,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,GAAG,OAAO,CAAC,OAAO,CAAC,CAAC;CACrD;AAED;;GAEG;AACH,qBAAa,sBAAsB;IAI/B,OAAO,CAAC,QAAQ,CAAC,OAAO;IACxB,OAAO,CAAC,QAAQ,CAAC,MAAM;IACvB,OAAO,CAAC,QAAQ,CAAC,cAAc;IALjC,OAAO,CAAC,QAAQ,CAAC,aAAa,CAA4B;gBAGvC,OAAO,EAAE,4BAA4B,EACrC,MAAM,EAAE,iBAAiB,EACzB,cAAc,EAAE,MAAM,0BAA0B;IAGnE,IAAI,iBAAiB,IAAI,MAAM,CAE9B;IAED;;;;;;OAMG;IACG,OAAO,CACX,OAAO,EAAE,MAAM,EACf,OAAO,GAAE,gCAAqC,GAC7C,OAAO,CAAC,+BAA+B,CAAC;IAoB3C,oBAAoB,CAAC,UAAU,EAAE,qBAAqB,EAAE,GAAG,EAAE,kBAAkB,GAAG,SAAS,EAAE,KAAK,EAAE,KAAK,GAAG,IAAI;IAY1G,kBAAkB,IAAI,OAAO,CAAC,IAAI,CAAC;YAmB3B,sBAAsB;IA+BpC,OAAO,CAAC,iBAAiB;CAQ1B"}
|
|
@@ -1,6 +1,8 @@
|
|
|
1
1
|
import { cloneWithFallback } from '@fluojs/core/internal';
|
|
2
2
|
import { normalizePositiveInteger, withTimeout } from './helpers.js';
|
|
3
3
|
const DEAD_LETTER_DRAIN_TIMEOUT_MS = 5_000;
|
|
4
|
+
const DEFAULT_DEAD_LETTER_INSPECTION_LIMIT = 100;
|
|
5
|
+
const MAX_DEAD_LETTER_INSPECTION_LIMIT = 1_000;
|
|
4
6
|
|
|
5
7
|
/**
|
|
6
8
|
* Describes the queue dead letter job contract.
|
|
@@ -23,6 +25,33 @@ export class QueueDeadLetterManager {
|
|
|
23
25
|
get pendingWriteCount() {
|
|
24
26
|
return this.pendingWrites.size;
|
|
25
27
|
}
|
|
28
|
+
|
|
29
|
+
/**
|
|
30
|
+
* Reads and parses a bounded dead-letter snapshot without mutating Redis state.
|
|
31
|
+
*
|
|
32
|
+
* @param jobName Queue worker job name whose dead letters should be inspected.
|
|
33
|
+
* @param options Optional inspection limit, capped at `1_000` stored entries.
|
|
34
|
+
* @returns Valid records in newest-first order and the number of malformed entries omitted.
|
|
35
|
+
*/
|
|
36
|
+
async inspect(jobName, options = {}) {
|
|
37
|
+
const requestedLimit = normalizePositiveInteger(options.limit, DEFAULT_DEAD_LETTER_INSPECTION_LIMIT);
|
|
38
|
+
const limit = Math.min(requestedLimit, MAX_DEAD_LETTER_INSPECTION_LIMIT);
|
|
39
|
+
const serializedRecords = await this.getRedisClient().lrange(deadLetterKey(jobName), -limit, -1);
|
|
40
|
+
const records = [];
|
|
41
|
+
let malformedRecordCount = 0;
|
|
42
|
+
for (const serializedRecord of serializedRecords.reverse()) {
|
|
43
|
+
const record = parseDeadLetterRecord(serializedRecord, jobName);
|
|
44
|
+
if (record) {
|
|
45
|
+
records.push(record);
|
|
46
|
+
} else {
|
|
47
|
+
malformedRecordCount += 1;
|
|
48
|
+
}
|
|
49
|
+
}
|
|
50
|
+
return {
|
|
51
|
+
malformedRecordCount,
|
|
52
|
+
records
|
|
53
|
+
};
|
|
54
|
+
}
|
|
26
55
|
trackTerminalFailure(descriptor, job, error) {
|
|
27
56
|
if (!job || !this.isTerminalFailure(job, descriptor.attempts)) {
|
|
28
57
|
return;
|
|
@@ -75,4 +104,37 @@ function deadLetterKey(jobName) {
|
|
|
75
104
|
}
|
|
76
105
|
function isQueuePayload(value) {
|
|
77
106
|
return typeof value === 'object' && value !== null && !Array.isArray(value);
|
|
107
|
+
}
|
|
108
|
+
function parseDeadLetterRecord(serializedRecord, expectedJobName) {
|
|
109
|
+
let value;
|
|
110
|
+
try {
|
|
111
|
+
value = JSON.parse(serializedRecord);
|
|
112
|
+
} catch (error) {
|
|
113
|
+
if (error instanceof SyntaxError) {
|
|
114
|
+
return undefined;
|
|
115
|
+
}
|
|
116
|
+
throw error;
|
|
117
|
+
}
|
|
118
|
+
if (!isQueuePayload(value) || !Object.hasOwn(value, 'payload')) {
|
|
119
|
+
return undefined;
|
|
120
|
+
}
|
|
121
|
+
const {
|
|
122
|
+
attemptsMade,
|
|
123
|
+
errorMessage,
|
|
124
|
+
failedAt,
|
|
125
|
+
jobId,
|
|
126
|
+
jobName,
|
|
127
|
+
payload
|
|
128
|
+
} = value;
|
|
129
|
+
if (typeof attemptsMade !== 'number' || !Number.isInteger(attemptsMade) || attemptsMade < 0 || typeof errorMessage !== 'string' || typeof failedAt !== 'string' || typeof jobId !== 'string' || jobName !== expectedJobName) {
|
|
130
|
+
return undefined;
|
|
131
|
+
}
|
|
132
|
+
return {
|
|
133
|
+
attemptsMade,
|
|
134
|
+
errorMessage,
|
|
135
|
+
failedAt,
|
|
136
|
+
jobId,
|
|
137
|
+
jobName,
|
|
138
|
+
payload
|
|
139
|
+
};
|
|
78
140
|
}
|
package/dist/helpers.d.ts
CHANGED
|
@@ -15,6 +15,8 @@ export interface DiscoveryCandidate {
|
|
|
15
15
|
targetType: Function;
|
|
16
16
|
token: Token;
|
|
17
17
|
}
|
|
18
|
+
/** Selects whether one compiled module participates in worker discovery. */
|
|
19
|
+
export type DiscoveryModuleFilter = (compiledModule: CompiledModule) => boolean;
|
|
18
20
|
/**
|
|
19
21
|
* Scope from provider.
|
|
20
22
|
*
|
|
@@ -38,7 +40,7 @@ export declare function isClassProvider(provider: Provider): provider is Extract
|
|
|
38
40
|
* @param compiledModules The compiled modules.
|
|
39
41
|
* @returns The collect discovery candidates result.
|
|
40
42
|
*/
|
|
41
|
-
export declare function collectDiscoveryCandidates(compiledModules: readonly CompiledModule[]): DiscoveryCandidate[];
|
|
43
|
+
export declare function collectDiscoveryCandidates(compiledModules: readonly CompiledModule[], moduleFilter?: DiscoveryModuleFilter): DiscoveryCandidate[];
|
|
42
44
|
/**
|
|
43
45
|
* Normalize positive integer.
|
|
44
46
|
*
|
package/dist/helpers.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"helpers.d.ts","sourceRoot":"","sources":["../src/helpers.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,KAAK,KAAK,EAAE,MAAM,cAAc,CAAC;AAE1C,OAAO,KAAK,EAAE,QAAQ,EAAE,MAAM,YAAY,CAAC;AAC3C,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,iBAAiB,CAAC;AAEtD,OAAO,KAAK,EAAE,uBAAuB,EAAE,MAAM,YAAY,CAAC;AAE1D;;GAEG;AACH,MAAM,MAAM,KAAK,GAAG,SAAS,GAAG,WAAW,GAAG,WAAW,CAAC;AAE1D;;GAEG;AACH,MAAM,WAAW,kBAAkB;IACjC,UAAU,EAAE,MAAM,CAAC;IACnB,KAAK,EAAE,KAAK,CAAC;IACb,UAAU,EAAE,QAAQ,CAAC;IACrB,KAAK,EAAE,KAAK,CAAC;CACd;AAED;;;;;GAKG;AACH,wBAAgB,iBAAiB,CAAC,QAAQ,EAAE,QAAQ,GAAG,KAAK,CAU3D;AAED;;;;;GAKG;AACH,wBAAgB,eAAe,CAAC,QAAQ,EAAE,QAAQ,GAAG,QAAQ,IAAI,OAAO,CAAC,QAAQ,EAAE;IAAE,OAAO,EAAE,KAAK,CAAC;IAAC,QAAQ,EAAE,QAAQ,CAAA;CAAE,CAAC,CAEzH;AAED;;;;;GAKG;AACH,wBAAgB,0BAA0B,
|
|
1
|
+
{"version":3,"file":"helpers.d.ts","sourceRoot":"","sources":["../src/helpers.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,KAAK,KAAK,EAAE,MAAM,cAAc,CAAC;AAE1C,OAAO,KAAK,EAAE,QAAQ,EAAE,MAAM,YAAY,CAAC;AAC3C,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,iBAAiB,CAAC;AAEtD,OAAO,KAAK,EAAE,uBAAuB,EAAE,MAAM,YAAY,CAAC;AAE1D;;GAEG;AACH,MAAM,MAAM,KAAK,GAAG,SAAS,GAAG,WAAW,GAAG,WAAW,CAAC;AAE1D;;GAEG;AACH,MAAM,WAAW,kBAAkB;IACjC,UAAU,EAAE,MAAM,CAAC;IACnB,KAAK,EAAE,KAAK,CAAC;IACb,UAAU,EAAE,QAAQ,CAAC;IACrB,KAAK,EAAE,KAAK,CAAC;CACd;AAED,4EAA4E;AAC5E,MAAM,MAAM,qBAAqB,GAAG,CAAC,cAAc,EAAE,cAAc,KAAK,OAAO,CAAC;AAEhF;;;;;GAKG;AACH,wBAAgB,iBAAiB,CAAC,QAAQ,EAAE,QAAQ,GAAG,KAAK,CAU3D;AAED;;;;;GAKG;AACH,wBAAgB,eAAe,CAAC,QAAQ,EAAE,QAAQ,GAAG,QAAQ,IAAI,OAAO,CAAC,QAAQ,EAAE;IAAE,OAAO,EAAE,KAAK,CAAC;IAAC,QAAQ,EAAE,QAAQ,CAAA;CAAE,CAAC,CAEzH;AAED;;;;;GAKG;AACH,wBAAgB,0BAA0B,CACxC,eAAe,EAAE,SAAS,cAAc,EAAE,EAC1C,YAAY,GAAE,qBAAkC,GAC/C,kBAAkB,EAAE,CAwCtB;AAED;;;;;;GAMG;AACH,wBAAgB,wBAAwB,CAAC,KAAK,EAAE,MAAM,GAAG,SAAS,EAAE,QAAQ,EAAE,MAAM,GAAG,MAAM,CAY5F;AAED;;;;;;GAMG;AACH,wBAAgB,+BAA+B,CAC7C,KAAK,EAAE,MAAM,GAAG,KAAK,GAAG,SAAS,EACjC,QAAQ,EAAE,MAAM,GAAG,KAAK,GACvB,MAAM,GAAG,KAAK,CAgBhB;AAED;;;;;GAKG;AACH,wBAAgB,oBAAoB,CAAC,WAAW,EAAE,uBAAuB,GAAG,SAAS,GAAG,uBAAuB,GAAG,SAAS,CAS1H;AAED;;;;;;;GAOG;AACH,wBAAsB,WAAW,CAAC,CAAC,EACjC,OAAO,EAAE,OAAO,CAAC,CAAC,CAAC,EACnB,SAAS,EAAE,MAAM,EACjB,mBAAmB,EAAE,MAAM,KAAK,GAC/B,OAAO,CAAC,CAAC,CAAC,CAeZ"}
|
package/dist/helpers.js
CHANGED
|
@@ -8,6 +8,8 @@ import { getClassDiMetadata } from '@fluojs/core/internal';
|
|
|
8
8
|
* Describes the discovery candidate contract.
|
|
9
9
|
*/
|
|
10
10
|
|
|
11
|
+
/** Selects whether one compiled module participates in worker discovery. */
|
|
12
|
+
|
|
11
13
|
/**
|
|
12
14
|
* Scope from provider.
|
|
13
15
|
*
|
|
@@ -40,9 +42,12 @@ export function isClassProvider(provider) {
|
|
|
40
42
|
* @param compiledModules The compiled modules.
|
|
41
43
|
* @returns The collect discovery candidates result.
|
|
42
44
|
*/
|
|
43
|
-
export function collectDiscoveryCandidates(compiledModules) {
|
|
45
|
+
export function collectDiscoveryCandidates(compiledModules, moduleFilter = () => true) {
|
|
44
46
|
const candidates = [];
|
|
45
47
|
for (const compiledModule of compiledModules) {
|
|
48
|
+
if (!moduleFilter(compiledModule)) {
|
|
49
|
+
continue;
|
|
50
|
+
}
|
|
46
51
|
for (const provider of compiledModule.definition.providers ?? []) {
|
|
47
52
|
if (typeof provider === 'function') {
|
|
48
53
|
candidates.push({
|
package/dist/index.d.ts
CHANGED
|
@@ -2,6 +2,6 @@ export { QueueWorker } from './decorators.js';
|
|
|
2
2
|
export { QueueModule } from './module.js';
|
|
3
3
|
export { QueueLifecycleService } from './service.js';
|
|
4
4
|
export * from './status.js';
|
|
5
|
-
export { QUEUE } from './tokens.js';
|
|
6
|
-
export type { Queue, QueueBackoffOptions, QueueBackoffType, QueueJobType, QueueModuleOptions, QueueRateLimiterOptions, QueueWorkerOptions, } from './types.js';
|
|
5
|
+
export { getQueueLifecycleServiceToken, getQueueToken, QUEUE } from './tokens.js';
|
|
6
|
+
export type { Queue, QueueBackoffOptions, QueueBackoffType, QueueDeadLetterInspectionOptions, QueueDeadLetterInspectionResult, QueueDeadLetterRecord, QueueJobType, QueueModuleOptions, QueueRateLimiterOptions, QueueWorkerOptions, } from './types.js';
|
|
7
7
|
//# sourceMappingURL=index.d.ts.map
|
package/dist/index.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,WAAW,EAAE,MAAM,iBAAiB,CAAC;AAC9C,OAAO,EAAE,WAAW,EAAE,MAAM,aAAa,CAAC;AAC1C,OAAO,EAAE,qBAAqB,EAAE,MAAM,cAAc,CAAC;AACrD,cAAc,aAAa,CAAC;AAC5B,OAAO,EAAE,KAAK,EAAE,MAAM,aAAa,CAAC;
|
|
1
|
+
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,WAAW,EAAE,MAAM,iBAAiB,CAAC;AAC9C,OAAO,EAAE,WAAW,EAAE,MAAM,aAAa,CAAC;AAC1C,OAAO,EAAE,qBAAqB,EAAE,MAAM,cAAc,CAAC;AACrD,cAAc,aAAa,CAAC;AAC5B,OAAO,EAAE,6BAA6B,EAAE,aAAa,EAAE,KAAK,EAAE,MAAM,aAAa,CAAC;AAClF,YAAY,EACV,KAAK,EACL,mBAAmB,EACnB,gBAAgB,EAChB,gCAAgC,EAChC,+BAA+B,EAC/B,qBAAqB,EACrB,YAAY,EACZ,kBAAkB,EAClB,uBAAuB,EACvB,kBAAkB,GACnB,MAAM,YAAY,CAAC"}
|
package/dist/index.js
CHANGED
|
@@ -2,4 +2,4 @@ export { QueueWorker } from './decorators.js';
|
|
|
2
2
|
export { QueueModule } from './module.js';
|
|
3
3
|
export { QueueLifecycleService } from './service.js';
|
|
4
4
|
export * from './status.js';
|
|
5
|
-
export { QUEUE } from './tokens.js';
|
|
5
|
+
export { getQueueLifecycleServiceToken, getQueueToken, QUEUE } from './tokens.js';
|
package/dist/module.d.ts
CHANGED
|
@@ -5,10 +5,10 @@ import type { QueueModuleOptions } from './types.js';
|
|
|
5
5
|
*/
|
|
6
6
|
export declare class QueueModule {
|
|
7
7
|
/**
|
|
8
|
-
* Registers queue providers
|
|
8
|
+
* Registers queue providers using canonical `forRoot(...)` semantics.
|
|
9
9
|
*
|
|
10
10
|
* @param options Queue runtime defaults used by discovered workers and enqueued jobs.
|
|
11
|
-
* @returns A module definition that exports `
|
|
11
|
+
* @returns A module definition that exports default queue tokens when `scope` is omitted, or scoped queue tokens when `scope` is set.
|
|
12
12
|
*
|
|
13
13
|
* @example
|
|
14
14
|
* ```ts
|
package/dist/module.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"module.d.ts","sourceRoot":"","sources":["../src/module.ts"],"names":[],"mappings":"
|
|
1
|
+
{"version":3,"file":"module.d.ts","sourceRoot":"","sources":["../src/module.ts"],"names":[],"mappings":"AAIA,OAAO,EAA6D,KAAK,UAAU,EAAE,MAAM,iBAAiB,CAAC;AAqB7G,OAAO,KAAK,EAGV,kBAAkB,EACnB,MAAM,YAAY,CAAC;AAyLpB;;GAEG;AACH,qBAAa,WAAW;IACtB;;;;;;;;;;;;;;;;;;;;OAoBG;IACH,MAAM,CAAC,OAAO,CAAC,OAAO,GAAE,kBAAuB,GAAG,UAAU;CAa7D"}
|