@igniter-js/jobs 0.1.1 → 0.1.12
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +1118 -96
- package/CHANGELOG.md +8 -0
- package/README.md +2146 -93
- package/dist/{adapter-PiDCQWQd.d.mts → adapter-CXZxomI9.d.mts} +2 -2
- package/dist/{adapter-PiDCQWQd.d.ts → adapter-CXZxomI9.d.ts} +2 -2
- package/dist/adapters/bullmq.adapter.d.mts +2 -2
- package/dist/adapters/bullmq.adapter.d.ts +2 -2
- package/dist/adapters/bullmq.adapter.js +2 -2
- package/dist/adapters/bullmq.adapter.js.map +1 -1
- package/dist/adapters/bullmq.adapter.mjs +1 -1
- package/dist/adapters/bullmq.adapter.mjs.map +1 -1
- package/dist/adapters/index.d.mts +140 -2
- package/dist/adapters/index.d.ts +140 -2
- package/dist/adapters/index.js +864 -31
- package/dist/adapters/index.js.map +1 -1
- package/dist/adapters/index.mjs +863 -31
- package/dist/adapters/index.mjs.map +1 -1
- package/dist/adapters/memory.adapter.d.mts +2 -2
- package/dist/adapters/memory.adapter.d.ts +2 -2
- package/dist/adapters/memory.adapter.js +122 -30
- package/dist/adapters/memory.adapter.js.map +1 -1
- package/dist/adapters/memory.adapter.mjs +121 -29
- package/dist/adapters/memory.adapter.mjs.map +1 -1
- package/dist/index.d.mts +452 -342
- package/dist/index.d.ts +452 -342
- package/dist/index.js +1923 -1002
- package/dist/index.js.map +1 -1
- package/dist/index.mjs +1921 -1001
- package/dist/index.mjs.map +1 -1
- package/dist/shim.d.mts +36 -0
- package/dist/shim.d.ts +36 -0
- package/dist/shim.js +75 -0
- package/dist/shim.js.map +1 -0
- package/dist/shim.mjs +67 -0
- package/dist/shim.mjs.map +1 -0
- package/dist/telemetry/index.d.mts +281 -0
- package/dist/telemetry/index.d.ts +281 -0
- package/dist/telemetry/index.js +97 -0
- package/dist/telemetry/index.js.map +1 -0
- package/dist/telemetry/index.mjs +95 -0
- package/dist/telemetry/index.mjs.map +1 -0
- package/package.json +44 -11
package/AGENTS.md
CHANGED
|
@@ -1,131 +1,1153 @@
|
|
|
1
1
|
# AGENTS.md - @igniter-js/jobs
|
|
2
2
|
|
|
3
|
-
> **Last Updated:**
|
|
4
|
-
> **Version:** 0.0.
|
|
3
|
+
> **Last Updated:** 2026-01-29
|
|
4
|
+
> **Version:** 1.0.0-alpha.0
|
|
5
|
+
> **Goal:** This document serves as the complete operational manual for Code Agents maintaining and consuming the @igniter-js/jobs package.
|
|
5
6
|
|
|
6
|
-
|
|
7
|
+
---
|
|
7
8
|
|
|
8
|
-
## 1.
|
|
9
|
+
## 1. Package Vision & Context
|
|
9
10
|
|
|
10
|
-
`@igniter-js/jobs`
|
|
11
|
+
`@igniter-js/jobs` is the definitive background processing and task scheduling solution for the Igniter.js ecosystem. It transforms the often-unpredictable world of distributed queues into a type-safe, observable, and highly structured domain that feels like a native extension of the application's business logic.
|
|
11
12
|
|
|
12
|
-
|
|
13
|
+
### Why This Package Exists
|
|
13
14
|
|
|
15
|
+
In modern distributed systems, background processing is usually the first point of failure and the last point of visibility. Traditional queue libraries often suffer from:
|
|
16
|
+
|
|
17
|
+
- **Type Erasure:** Job payloads are treated as `any` or raw JSON, leading to "poison pill" jobs that crash workers after being enqueued from a different version of the app.
|
|
18
|
+
- **Context Isolation:** Accessing core resources like database connections, mailers, or configurations inside a worker usually requires brittle global variables or complex dependency injection hacks.
|
|
19
|
+
- **Observability Gaps:** Tracking a job from the moment it's enqueued in an API request to its final completion on a remote worker often requires manual, inconsistent instrumentation.
|
|
20
|
+
- **Multi-Tenant Leakage:** In SaaS environments, it's dangerously easy for a job intended for Organization A to leak context or data into Organization B's processing loop.
|
|
21
|
+
|
|
22
|
+
### The Igniter.js Solution
|
|
23
|
+
|
|
24
|
+
`@igniter-js/jobs` solves these problems by enforcing a strict **Infrastructure-as-Code** approach to background tasks:
|
|
25
|
+
|
|
26
|
+
1. **End-to-End Type Safety:** By using TypeScript's powerful inference engine, the package ensures that the `input` passed to `.dispatch()` is exactly what the `handler` expects.
|
|
27
|
+
2. **Resource-Aware Context:** The `withContext()` factory ensures that every job execution starts with a fresh, validated set of resources, exactly like a standard API request.
|
|
28
|
+
3. **Multi-Platform Support (Adapters):** Whether using **BullMQ** for massive production scale or an **In-Memory** adapter for unit tests, the business logic remains identical.
|
|
29
|
+
4. **Native Observability:** Telemetry and internal event publishing are "baked in," providing immediate visibility into enqueuing, execution times, failures, and retries without writing a single line of logging code.
|
|
30
|
+
|
|
31
|
+
---
|
|
32
|
+
|
|
33
|
+
## I. MAINTAINER GUIDE (Internal Architecture)
|
|
34
|
+
|
|
35
|
+
### 2. FileSystem Topology (Maintenance)
|
|
36
|
+
|
|
37
|
+
Maintainers must respect the following directory structure and responsibilities:
|
|
38
|
+
|
|
39
|
+
- **`src/builders/`**: The configuration factory.
|
|
40
|
+
- `main.builder.ts`: `IgniterJobsBuilder`. Manages immutable state accumulation and generic type narrowing for context and queues.
|
|
41
|
+
- `queue.builder.ts`: `IgniterQueueBuilder`. Fluent API for defining jobs and crons within a queue.
|
|
42
|
+
- `worker.builder.ts`: `IgniterWorkerBuilder`. Configures worker concurrency and lifecycle handlers.
|
|
43
|
+
- **`src/core/`**: The runtime heart.
|
|
44
|
+
- `manager.ts`: `IgniterJobsManager`. Implements the proxy-based API and the handler-wrapping logic.
|
|
45
|
+
- `queue.ts`: `IgniterQueue` facade for the queue builder.
|
|
46
|
+
- **`src/adapters/`**: The infrastructure boundary.
|
|
47
|
+
- `memory.adapter.ts`: Production-grade in-memory implementation for tests (non-persistent).
|
|
48
|
+
- `sqlite.adapter.ts`: SQLite-based persistent adapter for local/desktop/CLI environments.
|
|
49
|
+
- `bullmq.adapter.ts`: Reference to external `@igniter-js/adapter-bullmq` for production Redis-based queues.
|
|
50
|
+
- **`src/errors/`**: Resiliency definitions.
|
|
51
|
+
- `jobs.error.ts`: `IgniterJobsError`. Extends `IgniterError` with metadata-rich payloads and authoritative error codes.
|
|
52
|
+
- **`src/telemetry/`**: Observability registry.
|
|
53
|
+
- `index.ts`: defines Zod-validated telemetry events for the package.
|
|
54
|
+
- **`src/types/`**: Pure contract definitions.
|
|
55
|
+
- `adapter.ts`: `IgniterJobsAdapter` interface.
|
|
56
|
+
- `runtime.ts`: Accessor interfaces for the proxy API.
|
|
57
|
+
- `job.ts`: authoritative types for job definitions, contexts, and hooks.
|
|
58
|
+
- **`src/utils/`**: Static helper library.
|
|
59
|
+
- `events.utils.ts`: Pub/sub event helpers and channel composition.
|
|
60
|
+
- `id-generator.ts`: Deterministic ID utilities for adapters.
|
|
61
|
+
- `prefix.ts`: Consistency logic for queue names and event channels.
|
|
62
|
+
- `scope.ts`: Scope merging and extraction utilities.
|
|
63
|
+
- `telemetry.ts`: Telemetry helper utilities.
|
|
64
|
+
- `validation.ts`: Runtime schema validation utilities.
|
|
65
|
+
|
|
66
|
+
### 3. Architecture Deep-Dive
|
|
67
|
+
|
|
68
|
+
#### 3.1 The Handler-Wrapper Pattern
|
|
69
|
+
|
|
70
|
+
The core reliability of the package comes from the fact that user handlers are never called directly by the adapter. Instead, `core/manager.ts` wraps every handler in a sophisticated pipeline:
|
|
71
|
+
|
|
72
|
+
1. **Telemetry Injection:** A "Started" event is emitted before the handler runs.
|
|
73
|
+
2. **Context Resolution:** The `contextFactory` provided in the builder is invoked to create the execution environment.
|
|
74
|
+
3. **Metadata Unpacking:** The scope (tenant ID) is extracted from the job metadata and injected into the context.
|
|
75
|
+
4. **Schema Enforcement:** If an `input` schema is defined, the payload is validated _inside the worker_ before being passed to the handler.
|
|
76
|
+
5. **Lifecycle Management:** The wrapper catches errors, determines if it was the final attempt, and emits the corresponding "Success" or "Failure" telemetry.
|
|
77
|
+
|
|
78
|
+
#### 3.2 Proxy-Based Accessors
|
|
79
|
+
|
|
80
|
+
The `IgniterJobsRuntime` uses a dynamic property accessor pattern. When you call `jobs.email.sendWelcome`, you aren't accessing a hardcoded property. Instead, the runtime maps the `email` queue and `sendWelcome` job from its internal configuration. This allows for a fluent, "discoverable" API that is 100% type-safe without needing code generation.
|
|
81
|
+
|
|
82
|
+
### 4. Operational Flow Mapping (Pipelines)
|
|
83
|
+
|
|
84
|
+
#### 4.1 Method: `IgniterJobsBuilder.build()`
|
|
85
|
+
|
|
86
|
+
1. **Argument Validation:** Adapter, Service, Environment, and Context Factory must be present.
|
|
87
|
+
2. **Internal Logic:** Builds `IgniterJobsConfig` from builder state.
|
|
88
|
+
3. **Runtime Creation:** Instantiates `IgniterJobsManager` and calls `.toRuntime()`.
|
|
89
|
+
4. **Adapter Registration:** `IgniterJobsManager.ensureRegistered()` registers jobs and crons on the adapter.
|
|
90
|
+
5. **Result Formatting:** Returns a proxy runtime with queue/job accessors.
|
|
91
|
+
|
|
92
|
+
#### 4.2 Method: `job.dispatch(params)`
|
|
93
|
+
|
|
94
|
+
1. **Argument Validation:** Validates `input` against job schema.
|
|
95
|
+
2. **Telemetry (Enqueued):** Emits `igniter.jobs.job.enqueued` when telemetry is configured.
|
|
96
|
+
3. **Internal Logic:** Calls `IgniterJobsScopeUtils.mergeMetadataWithScope`.
|
|
97
|
+
4. **Adapter Call:** Calls `adapter.dispatch()`.
|
|
98
|
+
5. **Telemetry (Success):** Emits success event with `jobId`.
|
|
99
|
+
|
|
100
|
+
#### 4.3 Method: `worker.start()`
|
|
101
|
+
|
|
102
|
+
1. **Internal Logic:** Consolidates queues, handlers, and concurrency.
|
|
103
|
+
2. **Adapter Call:** Calls `adapter.createWorker()`.
|
|
104
|
+
3. **Result Formatting:** Returns a worker handle for control.
|
|
105
|
+
|
|
106
|
+
#### 4.4 Method: `queue.pause()`
|
|
107
|
+
|
|
108
|
+
1. **Internal Logic:** Resolves queue name.
|
|
109
|
+
2. **Adapter Call:** Calls `adapter.pauseQueue()`.
|
|
110
|
+
|
|
111
|
+
#### 4.5 Method: `job.get(id).retrieve()`
|
|
112
|
+
|
|
113
|
+
1. **Internal Logic:** Resolves queue and job name.
|
|
114
|
+
2. **Adapter Call:** Calls `adapter.getJob()`.
|
|
115
|
+
3. **Result Formatting:** Maps adapter result to `JobSearchResult`.
|
|
116
|
+
|
|
117
|
+
#### 4.6 Method: `job.get(id).retry()`
|
|
118
|
+
|
|
119
|
+
1. **Adapter Call:** Calls `adapter.retryJob()`.
|
|
120
|
+
2. **Result:** Adapter handles retry; telemetry emission depends on adapter implementation.
|
|
121
|
+
|
|
122
|
+
### 5. Dependency & Type Graph
|
|
123
|
+
|
|
124
|
+
The package is designed for maximum portability with minimal dependency creep.
|
|
125
|
+
|
|
126
|
+
- **`@igniter-js/common`**: For base `IgniterError`, `IgniterLogger`, and the `StandardSchemaV1` interface.
|
|
127
|
+
- **`@igniter-js/telemetry`**: Optional peer dependency for operational monitoring.
|
|
128
|
+
|
|
129
|
+
Type Flow:
|
|
130
|
+
`IgniterJobsBuilder` -> `IgniterJobsConfig` -> `IgniterJobsRuntime` -> `QueueAccessor` -> `JobAccessor`
|
|
131
|
+
|
|
132
|
+
### 6. Maintenance Checklist
|
|
133
|
+
|
|
134
|
+
1. **Parity Check:** If you add a new feature to the `IgniterJobsAdapter` interface, ensure it is implemented in `memory.adapter.ts`.
|
|
135
|
+
2. **Telemetry Audit:** Job lifecycle emits telemetry in `core/manager.ts`. Queue/worker telemetry depends on adapter behavior.
|
|
136
|
+
3. **Error Propagation:** Ensure that errors thrown in user hooks (`onSuccess`, `onFailure`) do not crash the worker loop.
|
|
137
|
+
4. **Inference Test:** Run `npm run typecheck` and verify that the `main.builder.spec.ts` can still autocomplete queue names.
|
|
138
|
+
|
|
139
|
+
### 7. Maintainer Troubleshooting
|
|
140
|
+
|
|
141
|
+
#### Issue: Context factory is called too many times
|
|
142
|
+
|
|
143
|
+
- **Check:** Is the adapter caching the context factory result per job execution?
|
|
144
|
+
- **Fix:** Move factory invocation to the wrapper logic in `core/manager.ts`.
|
|
145
|
+
|
|
146
|
+
#### Issue: Telemetry attributes are missing
|
|
147
|
+
|
|
148
|
+
- **Check:** Are the attributes prefixed with `ctx.job.` or `ctx.worker.`?
|
|
149
|
+
- **Fix:** Follow the schema in `telemetry/index.ts`.
|
|
150
|
+
|
|
151
|
+
---
|
|
152
|
+
|
|
153
|
+
## II. CONSUMER GUIDE (Developer Manual)
|
|
154
|
+
|
|
155
|
+
### 8. Distribution Anatomy (Consumption)
|
|
156
|
+
|
|
157
|
+
The package provides organized subpath exports for optimized bundling:
|
|
158
|
+
|
|
159
|
+
- **`@igniter-js/jobs`**: The main entry point. Use for `IgniterJobs` and `IgniterQueue`.
|
|
160
|
+
- **`@igniter-js/jobs/adapters`**: Built-in adapters (Memory, SQLite, BullMQ wrapper).
|
|
161
|
+
- **`@igniter-js/jobs/telemetry`**: Telemetry registry for registration.
|
|
162
|
+
|
|
163
|
+
### 9. Quick Start & Common Patterns
|
|
164
|
+
|
|
165
|
+
#### Pattern: The Singleton Instance
|
|
166
|
+
|
|
167
|
+
Setup your jobs in a central `src/services/jobs.ts` file:
|
|
168
|
+
|
|
169
|
+
```typescript
|
|
170
|
+
import { IgniterJobs } from "@igniter-js/jobs";
|
|
171
|
+
import { IgniterJobsBullMQAdapter } from "@igniter-js/jobs/adapters";
|
|
172
|
+
|
|
173
|
+
export const jobs = IgniterJobs.create()
|
|
174
|
+
.withAdapter(IgniterJobsBullMQAdapter.create({ redis }))
|
|
175
|
+
.withService("api")
|
|
176
|
+
.withEnvironment("production")
|
|
177
|
+
.withContext(async () => ({ db }))
|
|
178
|
+
.addQueue(myQueue)
|
|
179
|
+
.build();
|
|
180
|
+
```
|
|
181
|
+
|
|
182
|
+
#### Pattern: Scheduled Retries
|
|
183
|
+
|
|
184
|
+
Prevent permanent failures in external API calls:
|
|
185
|
+
|
|
186
|
+
```typescript
|
|
187
|
+
.addJob("sync", {
|
|
188
|
+
attempts: 5,
|
|
189
|
+
delay: 5000,
|
|
190
|
+
handler: async () => { /* ... */ }
|
|
191
|
+
})
|
|
192
|
+
```
|
|
193
|
+
|
|
194
|
+
### 10. Real-World Use Case Library
|
|
195
|
+
|
|
196
|
+
#### Case 1: E-commerce Order Expiry
|
|
197
|
+
|
|
198
|
+
- **Scenario:** Cancel unpaid orders after 1 hour.
|
|
199
|
+
- **Implementation:** `jobs.orders.cancel.schedule({ input: { id }, delay: 3600000 })`.
|
|
200
|
+
|
|
201
|
+
#### Case 2: Fintech Nightly Reconciliation
|
|
202
|
+
|
|
203
|
+
- **Scenario:** Verify 1M transactions against bank API.
|
|
204
|
+
- **Implementation:** Cron job at 3 AM. Dispatches batch jobs with 100 transactions each.
|
|
205
|
+
|
|
206
|
+
#### Case 3: Social Media Media Transcoding
|
|
207
|
+
|
|
208
|
+
- **Scenario:** Generate 5 video qualities after upload.
|
|
209
|
+
- **Implementation:** High-priority queue for 360p, low-priority for 4k.
|
|
210
|
+
|
|
211
|
+
#### Case 4: Healthcare Appointment Reminders
|
|
212
|
+
|
|
213
|
+
- **Scenario:** Send SMS 24h before appointment.
|
|
214
|
+
- **Implementation:** `.schedule({ at: reminderDate })`.
|
|
215
|
+
|
|
216
|
+
#### Case 5: SaaS Multi-Tenant CSV Import
|
|
217
|
+
|
|
218
|
+
- **Scenario:** Process 100k row CSV.
|
|
219
|
+
- **Implementation:** `job.progress(percent)` every 100 rows for real-time progress bar.
|
|
220
|
+
|
|
221
|
+
### 11. Domain-Specific Guidance
|
|
222
|
+
|
|
223
|
+
- **Financial Systems:** Always use `attempts` and `backoff`. Implement `onFailure` to alert engineering.
|
|
224
|
+
- **High-Volume Caching:** Use `delay` to batch invalidation events.
|
|
225
|
+
- **AI Processing:** Set concurrency to match your GPU/API limits.
|
|
226
|
+
|
|
227
|
+
### 12. Best Practices & Anti-Patterns
|
|
228
|
+
|
|
229
|
+
| Practice | Why? | Example |
|
|
230
|
+
| :------------------- | :------------------- | :-------------------------------------- |
|
|
231
|
+
| ✅ Small Payloads | Reduces overhead. | `input: { id: '123' }` |
|
|
232
|
+
| ✅ Context Injection | Keeps handlers pure. | `withContext(() => ({ db }))` |
|
|
233
|
+
| ✅ Idempotency | Jobs WILL retry. | Check status before taking action. |
|
|
234
|
+
| ❌ Sync I/O | Kills throughput. | `readFileSync(...)` |
|
|
235
|
+
| ❌ Global DB | Untestable. | `import { db } from './db'` in handler. |
|
|
236
|
+
|
|
237
|
+
---
|
|
238
|
+
|
|
239
|
+
## III. TECHNICAL REFERENCE & RESILIENCE
|
|
240
|
+
|
|
241
|
+
### 13. Exhaustive API Reference
|
|
242
|
+
|
|
243
|
+
| Class/Method | Parameters | Returns | Description |
|
|
244
|
+
| :--------------------- | :--------------- | :---------------- | :--------------------- |
|
|
245
|
+
| `IgniterJobs.create()` | - | `Builder` | Starts the fluent API. |
|
|
246
|
+
| `dispatch()` | `params` | `Promise<string>` | Enqueues a job. |
|
|
247
|
+
| `schedule()` | `params` | `Promise<string>` | Schedules a job. |
|
|
248
|
+
| `search()` | `target, filter` | `Promise<T[]>` | Management API. |
|
|
249
|
+
| `worker.create()` | - | `WorkerBuilder` | Configures a worker. |
|
|
250
|
+
|
|
251
|
+
### 14. Telemetry & Observability Registry
|
|
252
|
+
|
|
253
|
+
| Event Name | Group | Meaning |
|
|
254
|
+
| :--------------- | :----- | :---------------- |
|
|
255
|
+
| `job.enqueued` | job | Task accepted. |
|
|
256
|
+
| `job.started` | job | Task started. |
|
|
257
|
+
| `job.completed` | job | Task finished. |
|
|
258
|
+
| `job.failed` | job | Task threw error. |
|
|
259
|
+
| `worker.started` | worker | Worker is online. |
|
|
260
|
+
|
|
261
|
+
### 15. Troubleshooting & Error Code Library
|
|
262
|
+
|
|
263
|
+
#### JOBS_ADAPTER_REQUIRED
|
|
264
|
+
|
|
265
|
+
- **Context:** `build()`.
|
|
266
|
+
- **Cause:** No adapter provided.
|
|
267
|
+
- **Solution:** Call `.withAdapter()`.
|
|
268
|
+
|
|
269
|
+
#### JOBS_SERVICE_REQUIRED
|
|
270
|
+
|
|
271
|
+
- **Context:** `build()`.
|
|
272
|
+
- **Cause:** No service name.
|
|
273
|
+
- **Solution:** Call `.withService()`.
|
|
274
|
+
|
|
275
|
+
#### JOBS_CONTEXT_REQUIRED
|
|
276
|
+
|
|
277
|
+
- **Context:** `build()`.
|
|
278
|
+
- **Cause:** No context factory.
|
|
279
|
+
- **Solution:** Call `.withContext()`.
|
|
280
|
+
|
|
281
|
+
#### JOBS_CONFIGURATION_INVALID
|
|
282
|
+
|
|
283
|
+
- **Context:** Various.
|
|
284
|
+
- **Cause:** Missing environment or invalid scope.
|
|
285
|
+
- **Solution:** Follow builder hints.
|
|
286
|
+
|
|
287
|
+
#### JOBS_QUEUE_NOT_FOUND
|
|
288
|
+
|
|
289
|
+
- **Context:** Worker creation.
|
|
290
|
+
- **Cause:** Wrong queue name.
|
|
291
|
+
- **Solution:** Check `addQueue()` vs `worker.addQueue()`.
|
|
292
|
+
|
|
293
|
+
#### JOBS_QUEUE_DUPLICATE
|
|
294
|
+
|
|
295
|
+
- **Context:** Initialization.
|
|
296
|
+
- **Cause:** Same name twice.
|
|
297
|
+
- **Solution:** Use unique names.
|
|
298
|
+
|
|
299
|
+
#### JOBS_INVALID_DEFINITION
|
|
300
|
+
|
|
301
|
+
- **Context:** Job registration.
|
|
302
|
+
- **Cause:** Malformed job object.
|
|
303
|
+
- **Solution:** Ensure `handler` exists.
|
|
304
|
+
|
|
305
|
+
#### JOBS_HANDLER_REQUIRED
|
|
306
|
+
|
|
307
|
+
- **Context:** Job registration.
|
|
308
|
+
- **Cause:** Handler is null.
|
|
309
|
+
- **Solution:** Provide async function.
|
|
310
|
+
|
|
311
|
+
#### JOBS_DUPLICATE_JOB
|
|
312
|
+
|
|
313
|
+
- **Context:** Queue building.
|
|
314
|
+
- **Cause:** Two jobs with same name.
|
|
315
|
+
- **Solution:** Rename one.
|
|
316
|
+
|
|
317
|
+
#### JOBS_NOT_FOUND
|
|
318
|
+
|
|
319
|
+
- **Context:** `get(id)`.
|
|
320
|
+
- **Cause:** Invalid ID.
|
|
321
|
+
- **Solution:** Verify ID or check retention.
|
|
322
|
+
|
|
323
|
+
#### JOBS_NOT_REGISTERED
|
|
324
|
+
|
|
325
|
+
- **Context:** Worker execution.
|
|
326
|
+
- **Cause:** Worker doesn't know job.
|
|
327
|
+
- **Solution:** Check runtime config parity.
|
|
328
|
+
|
|
329
|
+
#### JOBS_EXECUTION_FAILED
|
|
330
|
+
|
|
331
|
+
- **Context:** Worker.
|
|
332
|
+
- **Cause:** Handler threw error.
|
|
333
|
+
- **Solution:** Check business logic.
|
|
334
|
+
|
|
335
|
+
#### JOBS_TIMEOUT
|
|
336
|
+
|
|
337
|
+
- **Context:** Worker.
|
|
338
|
+
- **Cause:** Job took too long.
|
|
339
|
+
- **Solution:** Increase `timeout` in job opts.
|
|
340
|
+
|
|
341
|
+
#### JOBS_CONTEXT_FACTORY_FAILED
|
|
342
|
+
|
|
343
|
+
- **Context:** Worker.
|
|
344
|
+
- **Cause:** Factory threw error.
|
|
345
|
+
- **Solution:** Check DB/resource health.
|
|
346
|
+
|
|
347
|
+
#### JOBS_VALIDATION_FAILED
|
|
348
|
+
|
|
349
|
+
- **Context:** `dispatch`.
|
|
350
|
+
- **Cause:** Schema mismatch.
|
|
351
|
+
- **Solution:** Correct the input.
|
|
352
|
+
|
|
353
|
+
#### JOBS_INVALID_INPUT
|
|
354
|
+
|
|
355
|
+
- **Context:** Runtime.
|
|
356
|
+
- **Cause:** Malformed data.
|
|
357
|
+
- **Solution:** Validate early.
|
|
358
|
+
|
|
359
|
+
#### JOBS_INVALID_CRON
|
|
360
|
+
|
|
361
|
+
- **Context:** Registration.
|
|
362
|
+
- **Cause:** Bad syntax.
|
|
363
|
+
- **Solution:** Check cron expression.
|
|
364
|
+
|
|
365
|
+
#### JOBS_INVALID_SCHEDULE
|
|
366
|
+
|
|
367
|
+
- **Context:** `schedule`.
|
|
368
|
+
- **Cause:** Past date.
|
|
369
|
+
- **Solution:** Use future date.
|
|
370
|
+
|
|
371
|
+
#### JOBS_SCOPE_ALREADY_DEFINED
|
|
372
|
+
|
|
373
|
+
- **Context:** Builder.
|
|
374
|
+
- **Cause:** Two scopes added.
|
|
375
|
+
- **Solution:** Only one allowed.
|
|
376
|
+
|
|
377
|
+
#### JOBS_WORKER_FAILED
|
|
378
|
+
|
|
379
|
+
- **Context:** Worker.
|
|
380
|
+
- **Cause:** Connection failure.
|
|
381
|
+
- **Solution:** Check Redis health.
|
|
382
|
+
|
|
383
|
+
#### JOBS_ADAPTER_ERROR
|
|
384
|
+
|
|
385
|
+
- **Context:** Generic.
|
|
386
|
+
- **Cause:** Backend failure.
|
|
387
|
+
- **Solution:** Check Redis/Adapter logs.
|
|
388
|
+
|
|
389
|
+
#### JOBS_ADAPTER_CONNECTION_FAILED
|
|
390
|
+
|
|
391
|
+
- **Context:** Generic.
|
|
392
|
+
- **Cause:** Redis down.
|
|
393
|
+
- **Solution:** Restore connectivity.
|
|
394
|
+
|
|
395
|
+
#### JOBS_SUBSCRIBE_FAILED
|
|
396
|
+
|
|
397
|
+
- **Context:** `subscribe`.
|
|
398
|
+
- **Cause:** Pub/Sub failure.
|
|
399
|
+
- **Solution:** Check Redis ACLs.
|
|
400
|
+
|
|
401
|
+
---
|
|
402
|
+
|
|
403
|
+
## IV. ADAPTERS REFERENCE
|
|
404
|
+
|
|
405
|
+
The `@igniter-js/jobs` package supports multiple adapters for different use cases and environments. This section provides comprehensive documentation for each adapter.
|
|
406
|
+
|
|
407
|
+
### 16. Adapter Architecture Overview
|
|
408
|
+
|
|
409
|
+
All adapters implement the `IgniterJobsAdapter` interface defined in `src/types/adapter.ts`. This interface ensures consistent behavior across different storage backends.
|
|
410
|
+
|
|
411
|
+
```
|
|
412
|
+
┌─────────────────────────────────────────────────────────────────┐
|
|
413
|
+
│ IgniterJobsAdapter Interface │
|
|
414
|
+
├─────────────────────────────────────────────────────────────────┤
|
|
415
|
+
│ Job Management │
|
|
416
|
+
│ ├── registerJob(queue, name, definition) │
|
|
417
|
+
│ ├── registerCron(queue, name, definition) │
|
|
418
|
+
│ ├── dispatch(params) → jobId │
|
|
419
|
+
│ ├── schedule(params) → jobId │
|
|
420
|
+
│ ├── getJob(id) → job | null │
|
|
421
|
+
│ ├── getJobState(id) → status | null │
|
|
422
|
+
│ ├── getJobLogs(id) → logs[] │
|
|
423
|
+
│ ├── getJobProgress(id) → number │
|
|
424
|
+
│ ├── removeJob(id) │
|
|
425
|
+
│ ├── retryJob(id) │
|
|
426
|
+
│ ├── promoteJob(id) │
|
|
427
|
+
│ └── moveJobToFailed(id, error) │
|
|
428
|
+
├─────────────────────────────────────────────────────────────────┤
|
|
429
|
+
│ Queue Management │
|
|
430
|
+
│ ├── listQueues() → queue[] │
|
|
431
|
+
│ ├── getQueueInfo(name) → info | null │
|
|
432
|
+
│ ├── getQueueJobCounts(name) → counts │
|
|
433
|
+
│ ├── pauseQueue(name) │
|
|
434
|
+
│ ├── resumeQueue(name) │
|
|
435
|
+
│ ├── drainQueue(name) → removed │
|
|
436
|
+
│ ├── cleanQueue(name, options) → cleaned │
|
|
437
|
+
│ ├── obliterateQueue(name, options) │
|
|
438
|
+
│ └── retryAllInQueue(name) → retried │
|
|
439
|
+
├─────────────────────────────────────────────────────────────────┤
|
|
440
|
+
│ Worker Management │
|
|
441
|
+
│ ├── createWorker(config) → WorkerHandle │
|
|
442
|
+
│ ├── getWorkers() → Map<id, worker> │
|
|
443
|
+
│ └── searchWorkers(filter) → workers[] │
|
|
444
|
+
├─────────────────────────────────────────────────────────────────┤
|
|
445
|
+
│ Search & Discovery │
|
|
446
|
+
│ ├── searchJobs(filter) → jobs[] │
|
|
447
|
+
│ └── searchQueues(filter) → queues[] │
|
|
448
|
+
├─────────────────────────────────────────────────────────────────┤
|
|
449
|
+
│ Pub/Sub │
|
|
450
|
+
│ ├── publishEvent(channel, data) │
|
|
451
|
+
│ └── subscribeEvent(channel, handler) → unsubscribe │
|
|
452
|
+
├─────────────────────────────────────────────────────────────────┤
|
|
453
|
+
│ Lifecycle │
|
|
454
|
+
│ └── shutdown() │
|
|
455
|
+
└─────────────────────────────────────────────────────────────────┘
|
|
456
|
+
```
|
|
457
|
+
|
|
458
|
+
### 17. Adapter Comparison Matrix
|
|
459
|
+
|
|
460
|
+
| Feature | Memory | SQLite | BullMQ |
|
|
461
|
+
|---------|--------|--------|--------|
|
|
462
|
+
| **Persistence** | ❌ In-memory only | ✅ File-based | ✅ Redis |
|
|
463
|
+
| **Multi-process** | ❌ Single process | ⚠️ Single process (WAL) | ✅ Distributed |
|
|
464
|
+
| **Installation** | Built-in | `better-sqlite3` | `bullmq` + Redis |
|
|
465
|
+
| **Use Case** | Tests, demos | Desktop, CLI, MCP | Production, scale |
|
|
466
|
+
| **Job Limits** | Memory-bound | Disk-bound | Redis-bound |
|
|
467
|
+
| **Worker Mechanism** | Immediate | Polling-based | Event-driven |
|
|
468
|
+
| **Cron Jobs** | ⚠️ Basic | ⚠️ Basic | ✅ Full |
|
|
469
|
+
| **Distributed Locking** | ❌ | ❌ | ✅ |
|
|
470
|
+
| **Rate Limiting** | ⚠️ Basic | ⚠️ Basic | ✅ Advanced |
|
|
471
|
+
|
|
472
|
+
---
|
|
473
|
+
|
|
474
|
+
### 18. Memory Adapter (Testing & Development)
|
|
475
|
+
|
|
476
|
+
The `IgniterJobsMemoryAdapter` provides a full-featured in-memory implementation perfect for unit tests and quick development.
|
|
477
|
+
|
|
478
|
+
#### 18.1 When to Use
|
|
479
|
+
|
|
480
|
+
- **Unit Tests:** Fast, isolated, no external dependencies.
|
|
481
|
+
- **Integration Tests:** When you need predictable job behavior.
|
|
482
|
+
- **Development:** Quick experimentation without Redis.
|
|
483
|
+
- **CI/CD Pipelines:** Zero-config test environments.
|
|
484
|
+
|
|
485
|
+
#### 18.2 Installation
|
|
486
|
+
|
|
487
|
+
No additional dependencies required. The adapter is built into `@igniter-js/jobs`.
|
|
488
|
+
|
|
489
|
+
#### 18.3 Basic Usage
|
|
490
|
+
|
|
491
|
+
```typescript
|
|
492
|
+
import { IgniterJobs } from "@igniter-js/jobs";
|
|
493
|
+
import { IgniterJobsMemoryAdapter } from "@igniter-js/jobs/adapters";
|
|
494
|
+
|
|
495
|
+
const adapter = IgniterJobsMemoryAdapter.create();
|
|
496
|
+
|
|
497
|
+
const jobs = IgniterJobs.create()
|
|
498
|
+
.withAdapter(adapter)
|
|
499
|
+
.withService("test")
|
|
500
|
+
.withContext(async () => ({ db: mockDb }))
|
|
501
|
+
.addQueue(emailQueue)
|
|
502
|
+
.build();
|
|
503
|
+
```
|
|
504
|
+
|
|
505
|
+
#### 18.4 Configuration Options
|
|
506
|
+
|
|
507
|
+
```typescript
|
|
508
|
+
interface IgniterJobsMemoryAdapterOptions {
|
|
509
|
+
/** Maximum jobs to retain in history (default: 1000) */
|
|
510
|
+
maxJobHistory?: number;
|
|
511
|
+
}
|
|
14
512
|
```
|
|
15
|
-
packages/jobs/
|
|
16
|
-
├── src/
|
|
17
|
-
│ ├── adapters/ # Adapter implementations + barrel
|
|
18
|
-
│ ├── builders/ # Fluent builders (jobs, queue, worker)
|
|
19
|
-
│ ├── core/ # Runtime classes & proxies
|
|
20
|
-
│ ├── errors/ # IgniterJobsError hierarchy
|
|
21
|
-
│ ├── types/ # Public type surface (adapter/config/events/queue/worker)
|
|
22
|
-
│ └── utils/ # Static utility classes (id/prefix helpers)
|
|
23
|
-
├── AGENTS.md
|
|
24
|
-
# AGENTS.md - @igniter-js/jobs
|
|
25
513
|
|
|
26
|
-
|
|
27
|
-
> **Version:** 0.0.1 (bootstrap)
|
|
514
|
+
#### 18.5 Testing Patterns
|
|
28
515
|
|
|
29
|
-
|
|
516
|
+
```typescript
|
|
517
|
+
// Access internal state for assertions
|
|
518
|
+
const adapter = IgniterJobsMemoryAdapter.create();
|
|
30
519
|
|
|
31
|
-
|
|
520
|
+
// Dispatch a job
|
|
521
|
+
const jobId = await adapter.dispatch({
|
|
522
|
+
queue: "email",
|
|
523
|
+
jobName: "send",
|
|
524
|
+
input: { to: "test@example.com" },
|
|
525
|
+
});
|
|
32
526
|
|
|
33
|
-
|
|
527
|
+
// Verify job was created
|
|
528
|
+
const job = await adapter.getJob(jobId);
|
|
529
|
+
expect(job?.status).toBe("waiting");
|
|
34
530
|
|
|
35
|
-
|
|
531
|
+
// Simulate worker processing
|
|
532
|
+
const worker = await adapter.createWorker({
|
|
533
|
+
queues: ["email"],
|
|
534
|
+
concurrency: 1,
|
|
535
|
+
});
|
|
36
536
|
|
|
537
|
+
// Wait for processing
|
|
538
|
+
await new Promise((r) => setTimeout(r, 100));
|
|
539
|
+
|
|
540
|
+
// Verify completion
|
|
541
|
+
const completed = await adapter.getJob(jobId);
|
|
542
|
+
expect(completed?.status).toBe("completed");
|
|
543
|
+
|
|
544
|
+
// Cleanup
|
|
545
|
+
await worker.close();
|
|
546
|
+
```
|
|
547
|
+
|
|
548
|
+
#### 18.6 Limitations
|
|
549
|
+
|
|
550
|
+
- **Non-persistent:** All data lost on process restart.
|
|
551
|
+
- **Single-process:** Cannot share state across workers.
|
|
552
|
+
- **Memory-bound:** Large job volumes may cause OOM.
|
|
553
|
+
|
|
554
|
+
---
|
|
555
|
+
|
|
556
|
+
### 19. SQLite Adapter (Desktop, CLI, MCP Servers)
|
|
557
|
+
|
|
558
|
+
The `IgniterJobsSQLiteAdapter` provides persistent job storage using SQLite, ideal for local environments where Redis is impractical.
|
|
559
|
+
|
|
560
|
+
#### 19.1 When to Use
|
|
561
|
+
|
|
562
|
+
- **Desktop Applications:** Tauri, Electron apps needing background tasks.
|
|
563
|
+
- **CLI Tools:** Long-running commands with progress tracking.
|
|
564
|
+
- **MCP Servers:** Model Context Protocol servers with job queues.
|
|
565
|
+
- **Edge Deployments:** Single-node deployments.
|
|
566
|
+
- **Local Development:** When you want jobs to survive restarts.
|
|
567
|
+
|
|
568
|
+
#### 19.2 Installation
|
|
569
|
+
|
|
570
|
+
```bash
|
|
571
|
+
npm install better-sqlite3
|
|
572
|
+
# or
|
|
573
|
+
bun add better-sqlite3
|
|
574
|
+
```
|
|
575
|
+
|
|
576
|
+
The `better-sqlite3` package is an optional peer dependency.
|
|
577
|
+
|
|
578
|
+
#### 19.3 Basic Usage
|
|
579
|
+
|
|
580
|
+
```typescript
|
|
581
|
+
import { IgniterJobs } from "@igniter-js/jobs";
|
|
582
|
+
import { IgniterJobsSQLiteAdapter } from "@igniter-js/jobs/adapters";
|
|
583
|
+
|
|
584
|
+
// File-based (persistent)
|
|
585
|
+
const adapter = IgniterJobsSQLiteAdapter.create({
|
|
586
|
+
path: "./jobs.sqlite",
|
|
587
|
+
});
|
|
588
|
+
|
|
589
|
+
// In-memory (for tests)
|
|
590
|
+
const memoryAdapter = IgniterJobsSQLiteAdapter.create({
|
|
591
|
+
path: ":memory:",
|
|
592
|
+
});
|
|
593
|
+
|
|
594
|
+
const jobs = IgniterJobs.create()
|
|
595
|
+
.withAdapter(adapter)
|
|
596
|
+
.withService("desktop-app")
|
|
597
|
+
.withContext(async () => ({ settings: loadSettings() }))
|
|
598
|
+
.addQueue(syncQueue)
|
|
599
|
+
.build();
|
|
600
|
+
```
|
|
601
|
+
|
|
602
|
+
#### 19.4 Configuration Options
|
|
603
|
+
|
|
604
|
+
```typescript
|
|
605
|
+
interface IgniterJobsSQLiteAdapterOptions {
|
|
606
|
+
/**
|
|
607
|
+
* Path to the SQLite database file.
|
|
608
|
+
* Use ":memory:" for in-memory database.
|
|
609
|
+
* @example "./data/jobs.sqlite"
|
|
610
|
+
*/
|
|
611
|
+
path: string;
|
|
612
|
+
|
|
613
|
+
/**
|
|
614
|
+
* Polling interval in milliseconds for workers to check for new jobs.
|
|
615
|
+
* Lower values = more responsive, higher CPU usage.
|
|
616
|
+
* @default 500
|
|
617
|
+
*/
|
|
618
|
+
pollingInterval?: number;
|
|
619
|
+
|
|
620
|
+
/**
|
|
621
|
+
* Enable WAL (Write-Ahead Logging) mode for better concurrent performance.
|
|
622
|
+
* Recommended for production use.
|
|
623
|
+
* @default true
|
|
624
|
+
*/
|
|
625
|
+
enableWAL?: boolean;
|
|
626
|
+
}
|
|
627
|
+
```
|
|
628
|
+
|
|
629
|
+
#### 19.5 Database Schema
|
|
630
|
+
|
|
631
|
+
The adapter automatically creates and manages the following tables:
|
|
632
|
+
|
|
633
|
+
```sql
|
|
634
|
+
-- Main jobs table
|
|
635
|
+
CREATE TABLE IF NOT EXISTS jobs (
|
|
636
|
+
id TEXT PRIMARY KEY,
|
|
637
|
+
queue TEXT NOT NULL,
|
|
638
|
+
name TEXT NOT NULL,
|
|
639
|
+
input TEXT, -- JSON serialized
|
|
640
|
+
result TEXT, -- JSON serialized
|
|
641
|
+
error TEXT,
|
|
642
|
+
status TEXT NOT NULL DEFAULT 'waiting',
|
|
643
|
+
progress INTEGER DEFAULT 0,
|
|
644
|
+
priority INTEGER DEFAULT 0,
|
|
645
|
+
attempts_made INTEGER DEFAULT 0,
|
|
646
|
+
max_attempts INTEGER DEFAULT 1,
|
|
647
|
+
delay_until INTEGER, -- Unix timestamp
|
|
648
|
+
metadata TEXT, -- JSON serialized
|
|
649
|
+
scope_type TEXT,
|
|
650
|
+
scope_id TEXT,
|
|
651
|
+
created_at INTEGER NOT NULL,
|
|
652
|
+
started_at INTEGER,
|
|
653
|
+
completed_at INTEGER
|
|
654
|
+
);
|
|
655
|
+
|
|
656
|
+
-- Indexes for efficient queries
|
|
657
|
+
CREATE INDEX IF NOT EXISTS idx_jobs_queue_status ON jobs(queue, status);
|
|
658
|
+
CREATE INDEX IF NOT EXISTS idx_jobs_status ON jobs(status);
|
|
659
|
+
CREATE INDEX IF NOT EXISTS idx_jobs_priority ON jobs(priority DESC);
|
|
660
|
+
CREATE INDEX IF NOT EXISTS idx_jobs_delay ON jobs(delay_until);
|
|
661
|
+
|
|
662
|
+
-- Job logs table
|
|
663
|
+
CREATE TABLE IF NOT EXISTS job_logs (
|
|
664
|
+
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
665
|
+
job_id TEXT NOT NULL,
|
|
666
|
+
level TEXT NOT NULL,
|
|
667
|
+
message TEXT NOT NULL,
|
|
668
|
+
timestamp INTEGER NOT NULL,
|
|
669
|
+
FOREIGN KEY (job_id) REFERENCES jobs(id) ON DELETE CASCADE
|
|
670
|
+
);
|
|
671
|
+
|
|
672
|
+
-- Paused queues tracking
|
|
673
|
+
CREATE TABLE IF NOT EXISTS paused_queues (
|
|
674
|
+
name TEXT PRIMARY KEY,
|
|
675
|
+
paused_at INTEGER NOT NULL
|
|
676
|
+
);
|
|
677
|
+
```
|
|
678
|
+
|
|
679
|
+
#### 19.6 Worker Polling Mechanism
|
|
680
|
+
|
|
681
|
+
Unlike event-driven adapters (BullMQ), the SQLite adapter uses a polling-based approach:
|
|
682
|
+
|
|
683
|
+
```
|
|
684
|
+
┌─────────────────────────────────────────────────────────────────┐
|
|
685
|
+
│ Worker Polling Loop │
|
|
686
|
+
├─────────────────────────────────────────────────────────────────┤
|
|
687
|
+
│ │
|
|
688
|
+
│ ┌─────────┐ ┌─────────────┐ ┌─────────────┐ │
|
|
689
|
+
│ │ Wait │────▶│ Check Jobs │────▶│ Process Job │ │
|
|
690
|
+
│ │ Polling │ │ Available? │ │ (if any) │ │
|
|
691
|
+
│ │ Interval│ └─────────────┘ └─────────────┘ │
|
|
692
|
+
│ └─────────┘ │ │ │
|
|
693
|
+
│ ▲ │ No jobs │ Done │
|
|
694
|
+
│ │ │ ▼ │
|
|
695
|
+
│ └────────────────┴───────────────────┘ │
|
|
696
|
+
│ │
|
|
697
|
+
│ Query Priority: │
|
|
698
|
+
│ 1. Delayed jobs ready to run (delay_until <= NOW) │
|
|
699
|
+
│ 2. Waiting jobs, ordered by priority DESC │
|
|
700
|
+
│ 3. Retry jobs (failed but attempts < max_attempts) │
|
|
701
|
+
│ │
|
|
702
|
+
└─────────────────────────────────────────────────────────────────┘
|
|
703
|
+
```
|
|
704
|
+
|
|
705
|
+
#### 19.7 Complete Example
|
|
706
|
+
|
|
707
|
+
```typescript
|
|
708
|
+
import { IgniterJobsSQLiteAdapter } from "@igniter-js/jobs/adapters";
|
|
709
|
+
|
|
710
|
+
async function main() {
|
|
711
|
+
// 1. Create adapter
|
|
712
|
+
const adapter = IgniterJobsSQLiteAdapter.create({
|
|
713
|
+
path: "./my-app-jobs.sqlite",
|
|
714
|
+
pollingInterval: 500,
|
|
715
|
+
enableWAL: true,
|
|
716
|
+
});
|
|
717
|
+
|
|
718
|
+
// 2. Register jobs
|
|
719
|
+
adapter.registerJob("sync", "upload", {
|
|
720
|
+
handler: async ({ input }) => {
|
|
721
|
+
const { fileId } = input as { fileId: string };
|
|
722
|
+
// Simulate upload
|
|
723
|
+
await uploadFile(fileId);
|
|
724
|
+
return { uploaded: true };
|
|
725
|
+
},
|
|
726
|
+
onProgress: async ({ progress }) => {
|
|
727
|
+
console.log(`Upload progress: ${progress}%`);
|
|
728
|
+
},
|
|
729
|
+
});
|
|
730
|
+
|
|
731
|
+
// 3. Create worker
|
|
732
|
+
const worker = await adapter.createWorker({
|
|
733
|
+
queues: ["sync"],
|
|
734
|
+
concurrency: 2,
|
|
735
|
+
handlers: {
|
|
736
|
+
onActive: ({ job }) => console.log(`Processing: ${job.id}`),
|
|
737
|
+
onSuccess: ({ job }) => console.log(`Completed: ${job.id}`),
|
|
738
|
+
onFailure: ({ job, error }) => console.error(`Failed: ${job.id}`, error),
|
|
739
|
+
},
|
|
740
|
+
});
|
|
741
|
+
|
|
742
|
+
// 4. Dispatch jobs
|
|
743
|
+
const jobId = await adapter.dispatch({
|
|
744
|
+
queue: "sync",
|
|
745
|
+
jobName: "upload",
|
|
746
|
+
input: { fileId: "file_123" },
|
|
747
|
+
priority: 10,
|
|
748
|
+
});
|
|
749
|
+
|
|
750
|
+
console.log(`Dispatched job: ${jobId}`);
|
|
751
|
+
|
|
752
|
+
// 5. Monitor status
|
|
753
|
+
const counts = await adapter.getQueueJobCounts("sync");
|
|
754
|
+
console.log(`Queue status:`, counts);
|
|
755
|
+
|
|
756
|
+
// 6. Graceful shutdown
|
|
757
|
+
process.on("SIGINT", async () => {
|
|
758
|
+
await worker.close();
|
|
759
|
+
await adapter.shutdown();
|
|
760
|
+
process.exit(0);
|
|
761
|
+
});
|
|
762
|
+
}
|
|
763
|
+
|
|
764
|
+
main().catch(console.error);
|
|
37
765
|
```
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
766
|
+
|
|
767
|
+
#### 19.8 Persistence & Recovery
|
|
768
|
+
|
|
769
|
+
Jobs are automatically persisted and survive process restarts:
|
|
770
|
+
|
|
771
|
+
```typescript
|
|
772
|
+
// Session 1: Dispatch job and exit
|
|
773
|
+
const adapter = IgniterJobsSQLiteAdapter.create({ path: "./jobs.sqlite" });
|
|
774
|
+
await adapter.dispatch({
|
|
775
|
+
queue: "emails",
|
|
776
|
+
jobName: "send",
|
|
777
|
+
input: { to: "user@example.com" },
|
|
778
|
+
});
|
|
779
|
+
await adapter.shutdown();
|
|
780
|
+
// Process exits
|
|
781
|
+
|
|
782
|
+
// Session 2: Jobs are still there
|
|
783
|
+
const adapter2 = IgniterJobsSQLiteAdapter.create({ path: "./jobs.sqlite" });
|
|
784
|
+
const jobs = await adapter2.searchJobs({ queue: "emails", status: ["waiting"] });
|
|
785
|
+
console.log(jobs); // Shows the pending job from session 1
|
|
54
786
|
```
|
|
55
787
|
|
|
56
|
-
|
|
788
|
+
#### 19.9 Limitations
|
|
57
789
|
|
|
58
|
-
- **
|
|
59
|
-
- **
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
3) Job enqueue → adapter schedules → worker consumes → lifecycle events emit (internal + telemetry).
|
|
63
|
-
4) Errors bubble as `IgniterJobsError` with codes; runtime logs/telemetry should not crash workers.
|
|
64
|
-
- **Telemetry:** Optional peer; events emitted via lightweight helper (fire-and-forget). Event names `igniter.jobs.<group>.<event>`; attributes prefixed with `ctx.`. No actors; single scope (e.g., queue/workspace) only.
|
|
65
|
-
- **Scheduling:** `queue.addCron()` produces repeatable jobs; advanced scheduling metadata (skip weekends/business hours) attaches via `metadata.advancedScheduling`. BullMQ repeat options must be preserved.
|
|
790
|
+
- **Single Process:** WAL mode helps, but true concurrent writes from multiple processes may cause issues.
|
|
791
|
+
- **Polling Overhead:** Unlike event-driven systems, there's CPU overhead from polling.
|
|
792
|
+
- **No Distributed Locking:** Cannot guarantee exactly-once processing across processes.
|
|
793
|
+
- **Limited Cron:** Basic cron support without advanced scheduling features.
|
|
66
794
|
|
|
67
|
-
|
|
795
|
+
---
|
|
68
796
|
|
|
69
|
-
|
|
70
|
-
- **Type Safety:** No implicit `any` on public API. Use `Prettify` helpers for flattened types. Align with `packages/core/src/types/jobs.interface.ts` and keep parity.
|
|
71
|
-
- **Adapters:** `IgniterJobsAdapter` defines capabilities. BullMQ adapter must keep: rate limiting, scheduling (cron/repeat), retries/backoff, job metadata, progress, bulk ops, management APIs (pause/resume/drain/clean/obliterate), hooks/callbacks. Memory adapter is test-only.
|
|
72
|
-
- **Docs:** TSDoc (English) with examples/edge cases for every exported symbol. Keep README + AGENTS + CHANGELOG in sync with behavior.
|
|
73
|
-
- **Utils:** Keep shared helpers in `src/utils`; add tests. Prefer pure functions or static classes.
|
|
74
|
-
- **Telemetry:** Optional dependency; interface lives in types to avoid hard dependency. Telemetry errors must never fail job execution.
|
|
797
|
+
### 20. BullMQ Adapter (Production Scale)
|
|
75
798
|
|
|
76
|
-
|
|
799
|
+
The `IgniterJobsBullMQAdapter` (external package) provides Redis-based job queuing for production environments.
|
|
77
800
|
|
|
78
|
-
|
|
79
|
-
- **src/core/**: Runtime wrappers that orchestrate builders + adapters. Handles lifecycle hooks, logging/telemetry, error translation.
|
|
80
|
-
- **src/adapters/**: Adapter implementations + barrel. BullMQ mirrors `@igniter-js/adapter-bullmq`; memory adapter minimal but API-compatible for tests.
|
|
81
|
-
- **src/errors/**: `IgniterJobsError` codes. Add new codes here first; keep messages actionable.
|
|
82
|
-
- **src/telemetry/**: Event descriptors + emit helpers. Keep names stable and attributes prefixed `ctx.`.
|
|
83
|
-
- **src/types/**: Single source of truth for public API (config, events, adapters, queue/worker types). Update before implementation.
|
|
84
|
-
- **src/utils/**: ID generation, prefix handling, validation, backoff helpers. Pure and tested.
|
|
801
|
+
#### 20.1 When to Use
|
|
85
802
|
|
|
86
|
-
|
|
803
|
+
- **Production Deployments:** High availability and reliability.
|
|
804
|
+
- **Distributed Systems:** Multiple workers across nodes.
|
|
805
|
+
- **High Throughput:** Thousands of jobs per second.
|
|
806
|
+
- **Advanced Features:** Rate limiting, priorities, delayed jobs, cron.
|
|
87
807
|
|
|
88
|
-
|
|
89
|
-
- **Job events:** `enqueued`, `scheduled`, `started`, `completed`, `failed`, `progress`, `retrying`.
|
|
90
|
-
- **Worker events:** `started`, `stopped`, `idle`, `paused`, `resumed`.
|
|
91
|
-
- **Queue events:** `paused`, `resumed`, `drained`, `cleaned`, `obliterated`.
|
|
92
|
-
- **Attributes:** always include `ctx.job.id`, `ctx.job.queue`, plus duration/progress where applicable. Add `ctx.worker.id` for worker lifecycle.
|
|
93
|
-
- **Behavior:** Emission is best-effort; never throw. Keep helper tolerant (swallow telemetry errors, log if logger available).
|
|
808
|
+
#### 20.2 Installation
|
|
94
809
|
|
|
95
|
-
|
|
810
|
+
```bash
|
|
811
|
+
npm install @igniter-js/adapter-bullmq bullmq ioredis
|
|
812
|
+
```
|
|
96
813
|
|
|
97
|
-
|
|
98
|
-
- `bun test` (if bun is configured in scripts)
|
|
99
|
-
- `npx tsc --noEmit` for types
|
|
814
|
+
#### 20.3 Basic Usage
|
|
100
815
|
|
|
101
|
-
|
|
816
|
+
```typescript
|
|
817
|
+
import { IgniterJobs } from "@igniter-js/jobs";
|
|
818
|
+
import { IgniterJobsBullMQAdapter } from "@igniter-js/adapter-bullmq";
|
|
819
|
+
import Redis from "ioredis";
|
|
820
|
+
|
|
821
|
+
const redis = new Redis(process.env.REDIS_URL);
|
|
822
|
+
|
|
823
|
+
const adapter = IgniterJobsBullMQAdapter.create({ redis });
|
|
824
|
+
|
|
825
|
+
const jobs = IgniterJobs.create()
|
|
826
|
+
.withAdapter(adapter)
|
|
827
|
+
.withService("api-server")
|
|
828
|
+
.withContext(async () => ({ db, cache }))
|
|
829
|
+
.addQueue(emailQueue)
|
|
830
|
+
.addQueue(analyticsQueue)
|
|
831
|
+
.build();
|
|
832
|
+
```
|
|
102
833
|
|
|
103
|
-
|
|
104
|
-
- **Config:** `tsconfig.json` extends shared base; `tsup.config.ts` declares multiple entries if needed; keep barrels aligned.
|
|
105
|
-
- **External deps:** BullMQ/Redis via adapter; avoid pulling heavy deps into core runtime. Telemetry is optional peer.
|
|
834
|
+
#### 20.4 Reference
|
|
106
835
|
|
|
107
|
-
|
|
836
|
+
See `@igniter-js/adapter-bullmq` package documentation for full configuration options.
|
|
108
837
|
|
|
109
|
-
|
|
110
|
-
2. Update types (public surface) first; then implementation; then tests.
|
|
111
|
-
3. Preserve BullMQ feature parity; log any intentional deltas in CHANGELOG.
|
|
112
|
-
4. Keep telemetry names/attributes stable; update descriptors + tests if changed.
|
|
113
|
-
5. Add/adjust TSDoc for every exported symbol and public function.
|
|
114
|
-
6. Run tests and typecheck (`npm test --filter @igniter-js/jobs`, `npx tsc --noEmit`).
|
|
115
|
-
7. Update README + AGENTS + CHANGELOG when behavior or API changes.
|
|
838
|
+
---
|
|
116
839
|
|
|
117
|
-
|
|
840
|
+
### 21. Implementing Custom Adapters
|
|
841
|
+
|
|
842
|
+
To create a custom adapter, implement the `IgniterJobsAdapter` interface:
|
|
843
|
+
|
|
844
|
+
```typescript
|
|
845
|
+
import type {
|
|
846
|
+
IgniterJobsAdapter,
|
|
847
|
+
IgniterJobsDispatchParams,
|
|
848
|
+
IgniterJobsWorkerConfig,
|
|
849
|
+
IgniterJobsWorkerHandle,
|
|
850
|
+
IgniterJobSearchResult,
|
|
851
|
+
IgniterJobsQueueInfo,
|
|
852
|
+
} from "@igniter-js/jobs";
|
|
853
|
+
|
|
854
|
+
export class CustomAdapter implements IgniterJobsAdapter {
|
|
855
|
+
// ... implement all required methods
|
|
856
|
+
}
|
|
857
|
+
```
|
|
858
|
+
|
|
859
|
+
#### 21.1 Required Method Signatures
|
|
860
|
+
|
|
861
|
+
See `src/types/adapter.ts` for the complete interface definition.
|
|
862
|
+
|
|
863
|
+
#### 21.2 Testing Custom Adapters
|
|
864
|
+
|
|
865
|
+
Use the existing test suites as a reference:
|
|
866
|
+
- `src/adapters/memory.adapter.spec.ts`
|
|
867
|
+
- `src/adapters/sqlite.adapter.spec.ts`
|
|
868
|
+
|
|
869
|
+
---
|
|
870
|
+
|
|
871
|
+
## V. EXAMPLES & USE CASES
|
|
872
|
+
|
|
873
|
+
### 22. Example Directory Structure
|
|
874
|
+
|
|
875
|
+
The `examples/` directory contains runnable examples:
|
|
876
|
+
|
|
877
|
+
```
|
|
878
|
+
packages/jobs/examples/
|
|
879
|
+
├── README.md # Example documentation
|
|
880
|
+
└── sqlite-example.ts # Complete SQLite adapter demo
|
|
881
|
+
```
|
|
882
|
+
|
|
883
|
+
### 23. Running the SQLite Example
|
|
884
|
+
|
|
885
|
+
```bash
|
|
886
|
+
cd packages/jobs
|
|
887
|
+
npx tsx examples/sqlite-example.ts
|
|
888
|
+
```
|
|
889
|
+
|
|
890
|
+
This example demonstrates:
|
|
891
|
+
- Creating an SQLite adapter with file persistence
|
|
892
|
+
- Registering multiple job types
|
|
893
|
+
- Priority-based job processing
|
|
894
|
+
- Delayed job scheduling
|
|
895
|
+
- Worker metrics and lifecycle handlers
|
|
896
|
+
- Queue status monitoring
|
|
897
|
+
- Graceful shutdown
|
|
898
|
+
|
|
899
|
+
---
|
|
900
|
+
|
|
901
|
+
## VI. MIGRATION GUIDE
|
|
902
|
+
|
|
903
|
+
### 24. Migrating from Memory to SQLite
|
|
904
|
+
|
|
905
|
+
For development to production-like local testing:
|
|
906
|
+
|
|
907
|
+
```typescript
|
|
908
|
+
// Before: Memory adapter
|
|
909
|
+
import { IgniterJobsMemoryAdapter } from "@igniter-js/jobs/adapters";
|
|
910
|
+
const adapter = IgniterJobsMemoryAdapter.create();
|
|
911
|
+
|
|
912
|
+
// After: SQLite adapter
|
|
913
|
+
import { IgniterJobsSQLiteAdapter } from "@igniter-js/jobs/adapters";
|
|
914
|
+
const adapter = IgniterJobsSQLiteAdapter.create({
|
|
915
|
+
path: "./dev-jobs.sqlite",
|
|
916
|
+
});
|
|
917
|
+
|
|
918
|
+
// Rest of the code remains identical!
|
|
919
|
+
```
|
|
920
|
+
|
|
921
|
+
### 25. Migrating from SQLite to BullMQ
|
|
922
|
+
|
|
923
|
+
For local development to production:
|
|
924
|
+
|
|
925
|
+
```typescript
|
|
926
|
+
// Before: SQLite adapter
|
|
927
|
+
import { IgniterJobsSQLiteAdapter } from "@igniter-js/jobs/adapters";
|
|
928
|
+
const adapter = IgniterJobsSQLiteAdapter.create({
|
|
929
|
+
path: "./jobs.sqlite",
|
|
930
|
+
});
|
|
931
|
+
|
|
932
|
+
// After: BullMQ adapter
|
|
933
|
+
import { IgniterJobsBullMQAdapter } from "@igniter-js/adapter-bullmq";
|
|
934
|
+
const adapter = IgniterJobsBullMQAdapter.create({
|
|
935
|
+
redis: new Redis(process.env.REDIS_URL),
|
|
936
|
+
});
|
|
937
|
+
|
|
938
|
+
// Rest of the code remains identical!
|
|
939
|
+
```
|
|
940
|
+
|
|
941
|
+
---
|
|
942
|
+
|
|
943
|
+
## VII. MAINTENANCE CHANGELOG
|
|
944
|
+
|
|
945
|
+
### Version 0.2.0 (2026-01-17)
|
|
946
|
+
|
|
947
|
+
- **Added:** `IgniterJobsSQLiteAdapter` for persistent local job queues
|
|
948
|
+
- **Added:** SQLite adapter comprehensive test suite (71 tests)
|
|
949
|
+
- **Added:** `examples/` directory with runnable examples
|
|
950
|
+
- **Added:** `better-sqlite3` as optional peer dependency
|
|
951
|
+
- **Updated:** Adapter exports to include SQLite adapter
|
|
952
|
+
- **Updated:** AGENTS.md with comprehensive adapter documentation
|
|
953
|
+
|
|
954
|
+
### Version 0.1.2 (2025-12-23)
|
|
955
|
+
|
|
956
|
+
- Initial stable release
|
|
957
|
+
- Memory adapter for testing
|
|
958
|
+
- BullMQ adapter reference
|
|
959
|
+
|
|
960
|
+
---
|
|
961
|
+
|
|
962
|
+
## VIII. FAQ & TROUBLESHOOTING
|
|
963
|
+
|
|
964
|
+
### 26. Frequently Asked Questions
|
|
965
|
+
|
|
966
|
+
#### Q: Which adapter should I choose?
|
|
967
|
+
|
|
968
|
+
| Environment | Recommended Adapter |
|
|
969
|
+
|-------------|---------------------|
|
|
970
|
+
| Unit/Integration Tests | Memory Adapter |
|
|
971
|
+
| Desktop App (Tauri/Electron) | SQLite Adapter |
|
|
972
|
+
| CLI Tool | SQLite Adapter |
|
|
973
|
+
| MCP Server | SQLite Adapter |
|
|
974
|
+
| Local Development | SQLite Adapter |
|
|
975
|
+
| Production (Single Node) | SQLite or BullMQ |
|
|
976
|
+
| Production (Multi Node) | BullMQ Adapter |
|
|
977
|
+
| Serverless | BullMQ Adapter |
|
|
978
|
+
|
|
979
|
+
#### Q: Can I switch adapters without changing business logic?
|
|
980
|
+
|
|
981
|
+
**Yes!** All adapters implement the same `IgniterJobsAdapter` interface. Your job definitions, handlers, and queue configurations remain identical.
|
|
982
|
+
|
|
983
|
+
#### Q: How do I handle job failures?
|
|
984
|
+
|
|
985
|
+
```typescript
|
|
986
|
+
adapter.registerJob("email", "send", {
|
|
987
|
+
handler: async ({ input }) => {
|
|
988
|
+
// Your logic here
|
|
989
|
+
},
|
|
990
|
+
attempts: 3, // Retry up to 3 times
|
|
991
|
+
onFailure: async ({ error, isFinalAttempt }) => {
|
|
992
|
+
if (isFinalAttempt) {
|
|
993
|
+
// Send alert, log to monitoring, etc.
|
|
994
|
+
await alertOps(`Job failed permanently: ${error.message}`);
|
|
995
|
+
}
|
|
996
|
+
},
|
|
997
|
+
});
|
|
998
|
+
```
|
|
999
|
+
|
|
1000
|
+
#### Q: How do I track job progress?
|
|
1001
|
+
|
|
1002
|
+
```typescript
|
|
1003
|
+
adapter.registerJob("import", "csv", {
|
|
1004
|
+
handler: async ({ input, job }) => {
|
|
1005
|
+
const rows = await parseCSV(input.file);
|
|
1006
|
+
|
|
1007
|
+
for (let i = 0; i < rows.length; i++) {
|
|
1008
|
+
await processRow(rows[i]);
|
|
1009
|
+
|
|
1010
|
+
// Update progress
|
|
1011
|
+
await adapter.updateJobProgress(job.id, Math.floor((i / rows.length) * 100));
|
|
1012
|
+
}
|
|
1013
|
+
|
|
1014
|
+
return { processed: rows.length };
|
|
1015
|
+
},
|
|
1016
|
+
onProgress: async ({ progress, message }) => {
|
|
1017
|
+
console.log(`Progress: ${progress}%`);
|
|
1018
|
+
},
|
|
1019
|
+
});
|
|
1020
|
+
```
|
|
1021
|
+
|
|
1022
|
+
#### Q: Can SQLite handle high throughput?
|
|
1023
|
+
|
|
1024
|
+
SQLite with WAL mode can handle moderate throughput (100-1000 jobs/minute) on a single machine. For higher throughput or distributed processing, use BullMQ.
|
|
1025
|
+
|
|
1026
|
+
### 27. Common Issues & Solutions
|
|
1027
|
+
|
|
1028
|
+
#### Issue: "Cannot find module 'better-sqlite3'"
|
|
1029
|
+
|
|
1030
|
+
**Cause:** The `better-sqlite3` peer dependency is not installed.
|
|
1031
|
+
|
|
1032
|
+
**Solution:**
|
|
1033
|
+
```bash
|
|
1034
|
+
npm install better-sqlite3
|
|
1035
|
+
# If compilation fails on macOS:
|
|
1036
|
+
npm install better-sqlite3 --build-from-source
|
|
1037
|
+
```
|
|
1038
|
+
|
|
1039
|
+
#### Issue: Jobs stuck in "active" status after restart
|
|
1040
|
+
|
|
1041
|
+
**Cause:** Process crashed while job was processing.
|
|
1042
|
+
|
|
1043
|
+
**Solution:**
|
|
1044
|
+
```typescript
|
|
1045
|
+
// On startup, reset stale active jobs
|
|
1046
|
+
const staleJobs = await adapter.searchJobs({
|
|
1047
|
+
queue: "my-queue",
|
|
1048
|
+
status: ["active"],
|
|
1049
|
+
});
|
|
1050
|
+
|
|
1051
|
+
for (const job of staleJobs) {
|
|
1052
|
+
await adapter.retryJob(job.id);
|
|
1053
|
+
}
|
|
1054
|
+
```
|
|
1055
|
+
|
|
1056
|
+
#### Issue: Worker not processing jobs
|
|
1057
|
+
|
|
1058
|
+
**Cause:** Worker queues don't match job queue names.
|
|
1059
|
+
|
|
1060
|
+
**Solution:**
|
|
1061
|
+
```typescript
|
|
1062
|
+
// Ensure queue names match
|
|
1063
|
+
adapter.registerJob("email", "send", { ... }); // Queue: "email"
|
|
1064
|
+
|
|
1065
|
+
// Worker must listen to "email" queue
|
|
1066
|
+
const worker = await adapter.createWorker({
|
|
1067
|
+
queues: ["email"], // Must include "email"
|
|
1068
|
+
concurrency: 1,
|
|
1069
|
+
});
|
|
1070
|
+
```
|
|
1071
|
+
|
|
1072
|
+
#### Issue: SQLite database locked
|
|
1073
|
+
|
|
1074
|
+
**Cause:** Multiple processes trying to write simultaneously.
|
|
1075
|
+
|
|
1076
|
+
**Solution:**
|
|
1077
|
+
- Use WAL mode (enabled by default)
|
|
1078
|
+
- Ensure only one process writes at a time
|
|
1079
|
+
- Consider BullMQ for multi-process scenarios
|
|
1080
|
+
|
|
1081
|
+
---
|
|
1082
|
+
|
|
1083
|
+
## IX. PERFORMANCE CONSIDERATIONS
|
|
1084
|
+
|
|
1085
|
+
### 28. SQLite Adapter Performance Tips
|
|
1086
|
+
|
|
1087
|
+
1. **Use WAL Mode:** Always enable WAL mode for better read/write concurrency.
|
|
1088
|
+
|
|
1089
|
+
2. **Tune Polling Interval:**
|
|
1090
|
+
- Lower interval (100ms) = more responsive, higher CPU
|
|
1091
|
+
- Higher interval (1000ms) = less responsive, lower CPU
|
|
1092
|
+
|
|
1093
|
+
3. **Batch Operations:** When dispatching many jobs, consider batching:
|
|
1094
|
+
```typescript
|
|
1095
|
+
const jobIds = [];
|
|
1096
|
+
for (const item of items) {
|
|
1097
|
+
jobIds.push(await adapter.dispatch({
|
|
1098
|
+
queue: "process",
|
|
1099
|
+
jobName: "item",
|
|
1100
|
+
input: item,
|
|
1101
|
+
}));
|
|
1102
|
+
}
|
|
1103
|
+
```
|
|
1104
|
+
|
|
1105
|
+
4. **Clean Old Jobs:** Periodically clean completed/failed jobs:
|
|
1106
|
+
```typescript
|
|
1107
|
+
await adapter.cleanQueue("email", {
|
|
1108
|
+
status: ["completed", "failed"],
|
|
1109
|
+
olderThan: 7 * 24 * 60 * 60 * 1000, // 7 days
|
|
1110
|
+
});
|
|
1111
|
+
```
|
|
1112
|
+
|
|
1113
|
+
5. **Index Usage:** The adapter creates optimized indexes for common queries.
|
|
1114
|
+
|
|
1115
|
+
### 29. Memory Considerations
|
|
1116
|
+
|
|
1117
|
+
- **Memory Adapter:** All jobs stored in RAM; limit maxJobHistory.
|
|
1118
|
+
- **SQLite Adapter:** Disk-based; memory usage scales with active jobs.
|
|
1119
|
+
- **BullMQ Adapter:** Redis memory; configure maxmemory policy.
|
|
1120
|
+
|
|
1121
|
+
---
|
|
1122
|
+
|
|
1123
|
+
## X. SECURITY BEST PRACTICES
|
|
1124
|
+
|
|
1125
|
+
### 30. Input Validation
|
|
1126
|
+
|
|
1127
|
+
Always validate job inputs using schemas:
|
|
1128
|
+
|
|
1129
|
+
```typescript
|
|
1130
|
+
import { z } from "zod";
|
|
1131
|
+
|
|
1132
|
+
const emailQueue = IgniterQueue.create("email")
|
|
1133
|
+
.addJob("send", {
|
|
1134
|
+
input: z.object({
|
|
1135
|
+
to: z.string().email(),
|
|
1136
|
+
subject: z.string().max(200),
|
|
1137
|
+
body: z.string(),
|
|
1138
|
+
}),
|
|
1139
|
+
handler: async ({ input }) => {
|
|
1140
|
+
// input is validated and typed
|
|
1141
|
+
},
|
|
1142
|
+
})
|
|
1143
|
+
.build();
|
|
1144
|
+
```
|
|
118
1145
|
|
|
119
|
-
|
|
120
|
-
- **Cron/repeat drift:** Verify repeat options passed untouched to BullMQ; advanced scheduling metadata must survive serialization.
|
|
121
|
-
- **Type regressions:** Compare against `packages/core/src/types/jobs.interface.ts`. No `any` leakage in public exports.
|
|
122
|
-
- **Adapter parity gaps:** Cross-check with `packages/adapter-bullmq/src/bullmq.adapter.ts` for missing capabilities.
|
|
123
|
-
- **Flaky integration tests:** Prefer Redis-mock/stub; isolate time-based tests with fake timers.
|
|
1146
|
+
### 31. Sensitive Data Handling
|
|
124
1147
|
|
|
125
|
-
|
|
1148
|
+
- **Never log raw inputs** containing passwords, tokens, or PII.
|
|
1149
|
+
- **Use job IDs** for reference rather than storing sensitive data in jobs.
|
|
1150
|
+
- **Encrypt at rest** if storing sensitive metadata.
|
|
126
1151
|
|
|
127
|
-
|
|
128
|
-
- `packages/adapter-bullmq/src/bullmq.adapter.ts` — legacy feature set to mirror.
|
|
129
|
-
- `packages/store`, `packages/telemetry` — builder/test patterns and redaction/sampling ideas.
|
|
1152
|
+
---
|
|
130
1153
|
|
|
131
|
-
Stay consistent, type-safe, and keep docs/tests in lockstep with the code.
|