@igniter-js/jobs 0.1.0 → 0.1.12

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (44) hide show
  1. package/AGENTS.md +1006 -410
  2. package/CHANGELOG.md +13 -0
  3. package/README.md +2092 -277
  4. package/dist/adapter-CXZxomI9.d.mts +529 -0
  5. package/dist/adapter-CXZxomI9.d.ts +529 -0
  6. package/dist/adapters/bullmq.adapter.d.mts +56 -111
  7. package/dist/adapters/bullmq.adapter.d.ts +56 -111
  8. package/dist/adapters/bullmq.adapter.js +432 -530
  9. package/dist/adapters/bullmq.adapter.js.map +1 -1
  10. package/dist/adapters/bullmq.adapter.mjs +432 -530
  11. package/dist/adapters/bullmq.adapter.mjs.map +1 -1
  12. package/dist/adapters/index.d.mts +142 -4
  13. package/dist/adapters/index.d.ts +142 -4
  14. package/dist/adapters/index.js +1701 -939
  15. package/dist/adapters/index.js.map +1 -1
  16. package/dist/adapters/index.mjs +1699 -938
  17. package/dist/adapters/index.mjs.map +1 -1
  18. package/dist/adapters/memory.adapter.d.mts +53 -99
  19. package/dist/adapters/memory.adapter.d.ts +53 -99
  20. package/dist/adapters/memory.adapter.js +554 -464
  21. package/dist/adapters/memory.adapter.js.map +1 -1
  22. package/dist/adapters/memory.adapter.mjs +554 -464
  23. package/dist/adapters/memory.adapter.mjs.map +1 -1
  24. package/dist/index.d.mts +524 -887
  25. package/dist/index.d.ts +524 -887
  26. package/dist/index.js +2917 -860
  27. package/dist/index.js.map +1 -1
  28. package/dist/index.mjs +2914 -860
  29. package/dist/index.mjs.map +1 -1
  30. package/dist/shim.d.mts +36 -0
  31. package/dist/shim.d.ts +36 -0
  32. package/dist/shim.js +75 -0
  33. package/dist/shim.js.map +1 -0
  34. package/dist/shim.mjs +67 -0
  35. package/dist/shim.mjs.map +1 -0
  36. package/dist/telemetry/index.d.mts +281 -0
  37. package/dist/telemetry/index.d.ts +281 -0
  38. package/dist/telemetry/index.js +97 -0
  39. package/dist/telemetry/index.js.map +1 -0
  40. package/dist/telemetry/index.mjs +95 -0
  41. package/dist/telemetry/index.mjs.map +1 -0
  42. package/package.json +71 -20
  43. package/dist/adapter-CcQCatSa.d.mts +0 -1411
  44. package/dist/adapter-CcQCatSa.d.ts +0 -1411
package/AGENTS.md CHANGED
@@ -1,557 +1,1153 @@
1
- # @igniter-js/jobs - Agent Manual
1
+ # AGENTS.md - @igniter-js/jobs
2
2
 
3
- > **Last Updated:** 2025-01-01
4
- > **Version:** 0.1.0
3
+ > **Last Updated:** 2026-01-29
4
+ > **Version:** 1.0.0-alpha.0
5
+ > **Goal:** This document serves as the complete operational manual for Code Agents maintaining and consuming the @igniter-js/jobs package.
5
6
 
6
- ## 1. Package Overview
7
+ ---
7
8
 
8
- ### Purpose
9
+ ## 1. Package Vision & Context
9
10
 
10
- `@igniter-js/jobs` provides type-safe background job processing for TypeScript applications, powered by BullMQ and Redis. It follows the Igniter Builder API pattern for maximum type safety and developer experience.
11
+ `@igniter-js/jobs` is the definitive background processing and task scheduling solution for the Igniter.js ecosystem. It transforms the often-unpredictable world of distributed queues into a type-safe, observable, and highly structured domain that feels like a native extension of the application's business logic.
11
12
 
12
- ### Core Concepts
13
+ ### Why This Package Exists
13
14
 
14
- | Concept | Description |
15
- |---------|-------------|
16
- | **Queue** | Named container for jobs (e.g., `email`, `notifications`) |
17
- | **Job** | Unit of work with typed payload and handler |
18
- | **Cron Job** | Scheduled job with pattern or interval |
19
- | **Worker** | Process that executes jobs from queues |
20
- | **Scope** | Multi-tenant context (e.g., organization, workspace) |
21
- | **Actor** | Entity that triggered the job (e.g., user, system) |
22
- | **Adapter** | Implementation for queue backend (BullMQ, Memory) |
15
+ In modern distributed systems, background processing is usually the first point of failure and the last point of visibility. Traditional queue libraries often suffer from:
23
16
 
24
- ### Package Exports
17
+ - **Type Erasure:** Job payloads are treated as `any` or raw JSON, leading to "poison pill" jobs that crash workers after being enqueued from a different version of the app.
18
+ - **Context Isolation:** Accessing core resources like database connections, mailers, or configurations inside a worker usually requires brittle global variables or complex dependency injection hacks.
19
+ - **Observability Gaps:** Tracking a job from the moment it's enqueued in an API request to its final completion on a remote worker often requires manual, inconsistent instrumentation.
20
+ - **Multi-Tenant Leakage:** In SaaS environments, it's dangerously easy for a job intended for Organization A to leak context or data into Organization B's processing loop.
25
21
 
26
- ```typescript
27
- // Main entry point
28
- import { IgniterJobs, IgniterQueue, IgniterJobsError } from '@igniter-js/jobs'
22
+ ### The Igniter.js Solution
29
23
 
30
- // Adapters
31
- import { BullMQAdapter, MemoryAdapter } from '@igniter-js/jobs/adapters'
32
- ```
24
+ `@igniter-js/jobs` solves these problems by enforcing a strict **Infrastructure-as-Code** approach to background tasks:
25
+
26
+ 1. **End-to-End Type Safety:** By using TypeScript's powerful inference engine, the package ensures that the `input` passed to `.dispatch()` is exactly what the `handler` expects.
27
+ 2. **Resource-Aware Context:** The `withContext()` factory ensures that every job execution starts with a fresh, validated set of resources, exactly like a standard API request.
28
+ 3. **Multi-Platform Support (Adapters):** Whether using **BullMQ** for massive production scale or an **In-Memory** adapter for unit tests, the business logic remains identical.
29
+ 4. **Native Observability:** Telemetry and internal event publishing are "baked in," providing immediate visibility into enqueuing, execution times, failures, and retries without writing a single line of logging code.
33
30
 
34
31
  ---
35
32
 
36
- ## 2. Architecture
33
+ ## I. MAINTAINER GUIDE (Internal Architecture)
37
34
 
38
- ### File Structure
35
+ ### 2. FileSystem Topology (Maintenance)
39
36
 
40
- ```
41
- packages/jobs/
42
- ├── src/
43
- │ ├── index.ts # Main exports
44
- │ ├── types/
45
- │ │ ├── index.ts # Type exports
46
- │ │ ├── scope.ts # Scope/Actor types
47
- │ │ ├── job.ts # Job definition types
48
- │ │ ├── queue.ts # Queue config types
49
- │ │ ├── events.ts # Event types
50
- │ │ ├── adapter.ts # Adapter interface
51
- │ │ ├── config.ts # Builder state types
52
- │ │ └── worker.ts # Worker types
53
- │ ├── errors/
54
- │ │ ├── index.ts # Error exports
55
- │ │ └── igniter-jobs.error.ts # Error class and codes
56
- │ ├── builders/
57
- │ │ ├── index.ts # Builder exports
58
- │ │ ├── igniter-queue.builder.ts # Queue builder
59
- │ │ ├── igniter-jobs.builder.ts # Jobs builder
60
- │ │ └── igniter-worker.builder.ts # Worker builder
61
- │ ├── core/
62
- │ │ ├── index.ts # Core exports
63
- │ │ └── igniter-jobs.ts # Runtime class
64
- │ └── adapters/
65
- │ ├── index.ts # Adapter exports
66
- │ ├── bullmq.adapter.ts # BullMQ implementation
67
- │ └── memory.adapter.ts # Memory implementation
68
- ├── tests/
69
- │ ├── igniter-queue.builder.test.ts
70
- │ ├── igniter-jobs.builder.test.ts
71
- │ ├── igniter-jobs.runtime.test.ts
72
- │ └── memory.adapter.test.ts
73
- ├── package.json
74
- ├── tsconfig.json
75
- ├── tsup.config.ts
76
- ├── vitest.config.ts
77
- ├── README.md
78
- └── AGENTS.md # This file
79
- ```
37
+ Maintainers must respect the following directory structure and responsibilities:
80
38
 
81
- ### Builder Pattern Flow
39
+ - **`src/builders/`**: The configuration factory.
40
+ - `main.builder.ts`: `IgniterJobsBuilder`. Manages immutable state accumulation and generic type narrowing for context and queues.
41
+ - `queue.builder.ts`: `IgniterQueueBuilder`. Fluent API for defining jobs and crons within a queue.
42
+ - `worker.builder.ts`: `IgniterWorkerBuilder`. Configures worker concurrency and lifecycle handlers.
43
+ - **`src/core/`**: The runtime heart.
44
+ - `manager.ts`: `IgniterJobsManager`. Implements the proxy-based API and the handler-wrapping logic.
45
+ - `queue.ts`: `IgniterQueue` facade for the queue builder.
46
+ - **`src/adapters/`**: The infrastructure boundary.
47
+ - `memory.adapter.ts`: Production-grade in-memory implementation for tests (non-persistent).
48
+ - `sqlite.adapter.ts`: SQLite-based persistent adapter for local/desktop/CLI environments.
49
+ - `bullmq.adapter.ts`: Reference to external `@igniter-js/adapter-bullmq` for production Redis-based queues.
50
+ - **`src/errors/`**: Resiliency definitions.
51
+ - `jobs.error.ts`: `IgniterJobsError`. Extends `IgniterError` with metadata-rich payloads and authoritative error codes.
52
+ - **`src/telemetry/`**: Observability registry.
53
+ - `index.ts`: defines Zod-validated telemetry events for the package.
54
+ - **`src/types/`**: Pure contract definitions.
55
+ - `adapter.ts`: `IgniterJobsAdapter` interface.
56
+ - `runtime.ts`: Accessor interfaces for the proxy API.
57
+ - `job.ts`: authoritative types for job definitions, contexts, and hooks.
58
+ - **`src/utils/`**: Static helper library.
59
+ - `events.utils.ts`: Pub/sub event helpers and channel composition.
60
+ - `id-generator.ts`: Deterministic ID utilities for adapters.
61
+ - `prefix.ts`: Consistency logic for queue names and event channels.
62
+ - `scope.ts`: Scope merging and extraction utilities.
63
+ - `telemetry.ts`: Telemetry helper utilities.
64
+ - `validation.ts`: Runtime schema validation utilities.
82
65
 
83
- ```
84
- 1. IgniterQueue.create('name')
85
- └─► addJob() / addCron()
86
- └─► build() → IgniterQueueBuiltConfig
87
-
88
- 2. IgniterJobs.create()
89
- └─► withAdapter()
90
- └─► addQueue()
91
- └─► addScope() / addActor()
92
- └─► withDefaults()
93
- └─► build() → IgniterJobsRuntime (with Proxy)
94
-
95
- 3. jobs.queueName.jobName.dispatch(data)
96
- └─► Proxy intercepts access
97
- └─► Returns typed dispatch function
98
- ```
66
+ ### 3. Architecture Deep-Dive
99
67
 
100
- ### Proxy-Based API
68
+ #### 3.1 The Handler-Wrapper Pattern
101
69
 
102
- The runtime uses JavaScript Proxy to provide type-safe access:
70
+ The core reliability of the package comes from the fact that user handlers are never called directly by the adapter. Instead, `core/manager.ts` wraps every handler in a sophisticated pipeline:
103
71
 
104
- ```typescript
105
- // How it works internally:
106
- jobs.email.sendWelcome.dispatch(data)
107
- │ │ │
108
- │ │ └── Job proxy with dispatch/schedule methods
109
- │ └── Queue proxy returning job proxies
110
- └── Root proxy returning queue proxies
111
- ```
72
+ 1. **Telemetry Injection:** A "Started" event is emitted before the handler runs.
73
+ 2. **Context Resolution:** The `contextFactory` provided in the builder is invoked to create the execution environment.
74
+ 3. **Metadata Unpacking:** The scope (tenant ID) is extracted from the job metadata and injected into the context.
75
+ 4. **Schema Enforcement:** If an `input` schema is defined, the payload is validated _inside the worker_ before being passed to the handler.
76
+ 5. **Lifecycle Management:** The wrapper catches errors, determines if it was the final attempt, and emits the corresponding "Success" or "Failure" telemetry.
77
+
78
+ #### 3.2 Proxy-Based Accessors
79
+
80
+ The `IgniterJobsRuntime` uses a dynamic property accessor pattern. When you call `jobs.email.sendWelcome`, you aren't accessing a hardcoded property. Instead, the runtime maps the `email` queue and `sendWelcome` job from its internal configuration. This allows for a fluent, "discoverable" API that is 100% type-safe without needing code generation.
81
+
82
+ ### 4. Operational Flow Mapping (Pipelines)
83
+
84
+ #### 4.1 Method: `IgniterJobsBuilder.build()`
85
+
86
+ 1. **Argument Validation:** Adapter, Service, Environment, and Context Factory must be present.
87
+ 2. **Internal Logic:** Builds `IgniterJobsConfig` from builder state.
88
+ 3. **Runtime Creation:** Instantiates `IgniterJobsManager` and calls `.toRuntime()`.
89
+ 4. **Adapter Registration:** `IgniterJobsManager.ensureRegistered()` registers jobs and crons on the adapter.
90
+ 5. **Result Formatting:** Returns a proxy runtime with queue/job accessors.
91
+
92
+ #### 4.2 Method: `job.dispatch(params)`
93
+
94
+ 1. **Argument Validation:** Validates `input` against job schema.
95
+ 2. **Telemetry (Enqueued):** Emits `igniter.jobs.job.enqueued` when telemetry is configured.
96
+ 3. **Internal Logic:** Calls `IgniterJobsScopeUtils.mergeMetadataWithScope`.
97
+ 4. **Adapter Call:** Calls `adapter.dispatch()`.
98
+ 5. **Telemetry (Success):** Emits success event with `jobId`.
99
+
100
+ #### 4.3 Method: `worker.start()`
101
+
102
+ 1. **Internal Logic:** Consolidates queues, handlers, and concurrency.
103
+ 2. **Adapter Call:** Calls `adapter.createWorker()`.
104
+ 3. **Result Formatting:** Returns a worker handle for control.
105
+
106
+ #### 4.4 Method: `queue.pause()`
107
+
108
+ 1. **Internal Logic:** Resolves queue name.
109
+ 2. **Adapter Call:** Calls `adapter.pauseQueue()`.
110
+
111
+ #### 4.5 Method: `job.get(id).retrieve()`
112
+
113
+ 1. **Internal Logic:** Resolves queue and job name.
114
+ 2. **Adapter Call:** Calls `adapter.getJob()`.
115
+ 3. **Result Formatting:** Maps adapter result to `JobSearchResult`.
116
+
117
+ #### 4.6 Method: `job.get(id).retry()`
118
+
119
+ 1. **Adapter Call:** Calls `adapter.retryJob()`.
120
+ 2. **Result:** Adapter handles retry; telemetry emission depends on adapter implementation.
121
+
122
+ ### 5. Dependency & Type Graph
123
+
124
+ The package is designed for maximum portability with minimal dependency creep.
125
+
126
+ - **`@igniter-js/common`**: For base `IgniterError`, `IgniterLogger`, and the `StandardSchemaV1` interface.
127
+ - **`@igniter-js/telemetry`**: Optional peer dependency for operational monitoring.
128
+
129
+ Type Flow:
130
+ `IgniterJobsBuilder` -> `IgniterJobsConfig` -> `IgniterJobsRuntime` -> `QueueAccessor` -> `JobAccessor`
131
+
132
+ ### 6. Maintenance Checklist
133
+
134
+ 1. **Parity Check:** If you add a new feature to the `IgniterJobsAdapter` interface, ensure it is implemented in `memory.adapter.ts`.
135
+ 2. **Telemetry Audit:** Job lifecycle emits telemetry in `core/manager.ts`. Queue/worker telemetry depends on adapter behavior.
136
+ 3. **Error Propagation:** Ensure that errors thrown in user hooks (`onSuccess`, `onFailure`) do not crash the worker loop.
137
+ 4. **Inference Test:** Run `npm run typecheck` and verify that the `main.builder.spec.ts` can still autocomplete queue names.
138
+
139
+ ### 7. Maintainer Troubleshooting
140
+
141
+ #### Issue: Context factory is called too many times
142
+
143
+ - **Check:** Is the adapter caching the context factory result per job execution?
144
+ - **Fix:** Move factory invocation to the wrapper logic in `core/manager.ts`.
145
+
146
+ #### Issue: Telemetry attributes are missing
147
+
148
+ - **Check:** Are the attributes prefixed with `ctx.job.` or `ctx.worker.`?
149
+ - **Fix:** Follow the schema in `telemetry/index.ts`.
112
150
 
113
151
  ---
114
152
 
115
- ## 3. Key Components
153
+ ## II. CONSUMER GUIDE (Developer Manual)
154
+
155
+ ### 8. Distribution Anatomy (Consumption)
156
+
157
+ The package provides organized subpath exports for optimized bundling:
116
158
 
117
- ### 3.1 IgniterQueue Builder
159
+ - **`@igniter-js/jobs`**: The main entry point. Use for `IgniterJobs` and `IgniterQueue`.
160
+ - **`@igniter-js/jobs/adapters`**: Built-in adapters (Memory, SQLite, BullMQ wrapper).
161
+ - **`@igniter-js/jobs/telemetry`**: Telemetry registry for registration.
118
162
 
119
- **Location:** `src/builders/igniter-queue.builder.ts`
163
+ ### 9. Quick Start & Common Patterns
120
164
 
121
- **Purpose:** Define queue configuration with type-safe job definitions.
165
+ #### Pattern: The Singleton Instance
122
166
 
123
- **Key Methods:**
124
- - `create(name, options?)` - Initialize builder
125
- - `addJob(name, config)` - Add job with schema and handler
126
- - `addCron(name, config)` - Add cron job
127
- - `build()` - Return frozen config
167
+ Setup your jobs in a central `src/services/jobs.ts` file:
128
168
 
129
- **Type Flow:**
130
169
  ```typescript
131
- IgniterQueue.create('email')
132
- .addJob('send', { schema, handler })
133
- // Jobs type accumulates: { send: {...} }
134
- .build()
135
- // Returns: IgniterQueueBuiltConfig<'email', Jobs>
170
+ import { IgniterJobs } from "@igniter-js/jobs";
171
+ import { IgniterJobsBullMQAdapter } from "@igniter-js/jobs/adapters";
172
+
173
+ export const jobs = IgniterJobs.create()
174
+ .withAdapter(IgniterJobsBullMQAdapter.create({ redis }))
175
+ .withService("api")
176
+ .withEnvironment("production")
177
+ .withContext(async () => ({ db }))
178
+ .addQueue(myQueue)
179
+ .build();
136
180
  ```
137
181
 
138
- ### 3.2 IgniterJobs Builder
182
+ #### Pattern: Scheduled Retries
183
+
184
+ Prevent permanent failures in external API calls:
185
+
186
+ ```typescript
187
+ .addJob("sync", {
188
+ attempts: 5,
189
+ delay: 5000,
190
+ handler: async () => { /* ... */ }
191
+ })
192
+ ```
193
+
194
+ ### 10. Real-World Use Case Library
195
+
196
+ #### Case 1: E-commerce Order Expiry
139
197
 
140
- **Location:** `src/builders/igniter-jobs.builder.ts`
198
+ - **Scenario:** Cancel unpaid orders after 1 hour.
199
+ - **Implementation:** `jobs.orders.cancel.schedule({ input: { id }, delay: 3600000 })`.
141
200
 
142
- **Purpose:** Create jobs runtime with adapter, context, and queues.
201
+ #### Case 2: Fintech Nightly Reconciliation
143
202
 
144
- **Key Methods:**
145
- - `create<Context>()` - Initialize with context type
146
- - `withAdapter(adapter)` - Set queue adapter (required)
147
- - `withContext(factory)` - Set context factory for handlers
148
- - `addQueue(queue)` - Add queue configuration
149
- - `addScope(name, config)` - Define scope type
150
- - `addActor(name, config)` - Define actor type
151
- - `withTelemetry(adapter)` - Enable tracing
152
- - `withDefaults(options)` - Default job options
153
- - `build()` - Return runtime instance
203
+ - **Scenario:** Verify 1M transactions against bank API.
204
+ - **Implementation:** Cron job at 3 AM. Dispatches batch jobs with 100 transactions each.
154
205
 
155
- ### 3.3 IgniterJobs Runtime
206
+ #### Case 3: Social Media Media Transcoding
156
207
 
157
- **Location:** `src/core/igniter-jobs.ts`
208
+ - **Scenario:** Generate 5 video qualities after upload.
209
+ - **Implementation:** High-priority queue for 360p, low-priority for 4k.
158
210
 
159
- **Purpose:** Main runtime class with proxy-based access.
211
+ #### Case 4: Healthcare Appointment Reminders
160
212
 
161
- **Key Properties:**
162
- - `[queueName]` - Queue proxy (via Proxy)
163
- - `job` - Job management methods
164
- - `queue` - Queue management methods
165
- - `subscribe` - Event subscription
166
- - `search` - Search methods
167
- - `worker` - Worker builder access
168
- - `shutdown()` - Graceful shutdown
213
+ - **Scenario:** Send SMS 24h before appointment.
214
+ - **Implementation:** `.schedule({ at: reminderDate })`.
169
215
 
170
- ### 3.4 Adapters
216
+ #### Case 5: SaaS Multi-Tenant CSV Import
171
217
 
172
- **BullMQ Adapter:** `src/adapters/bullmq.adapter.ts`
173
- - Production-ready with Redis
174
- - Supports all BullMQ features
175
- - Lazy loads BullMQ module
218
+ - **Scenario:** Process 100k row CSV.
219
+ - **Implementation:** `job.progress(percent)` every 100 rows for real-time progress bar.
176
220
 
177
- **Memory Adapter:** `src/adapters/memory.adapter.ts`
178
- - For testing/development
179
- - No external dependencies
180
- - Simulates job processing
221
+ ### 11. Domain-Specific Guidance
222
+
223
+ - **Financial Systems:** Always use `attempts` and `backoff`. Implement `onFailure` to alert engineering.
224
+ - **High-Volume Caching:** Use `delay` to batch invalidation events.
225
+ - **AI Processing:** Set concurrency to match your GPU/API limits.
226
+
227
+ ### 12. Best Practices & Anti-Patterns
228
+
229
+ | Practice | Why? | Example |
230
+ | :------------------- | :------------------- | :-------------------------------------- |
231
+ | ✅ Small Payloads | Reduces overhead. | `input: { id: '123' }` |
232
+ | ✅ Context Injection | Keeps handlers pure. | `withContext(() => ({ db }))` |
233
+ | ✅ Idempotency | Jobs WILL retry. | Check status before taking action. |
234
+ | ❌ Sync I/O | Kills throughput. | `readFileSync(...)` |
235
+ | ❌ Global DB | Untestable. | `import { db } from './db'` in handler. |
181
236
 
182
237
  ---
183
238
 
184
- ## 4. Type System
239
+ ## III. TECHNICAL REFERENCE & RESILIENCE
185
240
 
186
- ### Core Type Hierarchy
241
+ ### 13. Exhaustive API Reference
187
242
 
188
- ```typescript
189
- // Job Definition (queue level)
190
- interface IgniterJobDefinition<TData, TResult, TContext> {
191
- schema: ZodSchema<TData>
192
- handler: IgniterJobHandler<TData, TResult, TContext>
193
- options?: IgniterJobOptions
194
- }
243
+ | Class/Method | Parameters | Returns | Description |
244
+ | :--------------------- | :--------------- | :---------------- | :--------------------- |
245
+ | `IgniterJobs.create()` | - | `Builder` | Starts the fluent API. |
246
+ | `dispatch()` | `params` | `Promise<string>` | Enqueues a job. |
247
+ | `schedule()` | `params` | `Promise<string>` | Schedules a job. |
248
+ | `search()` | `target, filter` | `Promise<T[]>` | Management API. |
249
+ | `worker.create()` | - | `WorkerBuilder` | Configures a worker. |
195
250
 
196
- // Handler Context (passed to handler)
197
- interface IgniterJobHandlerContext<TData, TContext> {
198
- data: TData
199
- context: TContext
200
- job: { id: string; name: string; queue: string; attempt: number }
201
- log: (level, message) => Promise<void>
202
- updateProgress: (progress) => Promise<void>
203
- scope?: IgniterJobScopeEntry
204
- actor?: IgniterJobActorEntry
205
- }
251
+ ### 14. Telemetry & Observability Registry
206
252
 
207
- // Dispatch Options
208
- interface DispatchOptions {
209
- delay?: number
210
- priority?: number
211
- attempts?: number
212
- backoff?: IgniterJobBackoff
213
- jobId?: string
214
- scope?: { type: string; id: string }
215
- actor?: { type: string; id: string }
216
- }
217
- ```
253
+ | Event Name | Group | Meaning |
254
+ | :--------------- | :----- | :---------------- |
255
+ | `job.enqueued` | job | Task accepted. |
256
+ | `job.started` | job | Task started. |
257
+ | `job.completed` | job | Task finished. |
258
+ | `job.failed` | job | Task threw error. |
259
+ | `worker.started` | worker | Worker is online. |
218
260
 
219
- ### Type Inference Chain
261
+ ### 15. Troubleshooting & Error Code Library
220
262
 
221
- ```typescript
222
- // 1. Define schema
223
- const schema = z.object({ email: z.string() })
224
- // Type: z.ZodObject<{ email: z.ZodString }>
263
+ #### JOBS_ADAPTER_REQUIRED
264
+
265
+ - **Context:** `build()`.
266
+ - **Cause:** No adapter provided.
267
+ - **Solution:** Call `.withAdapter()`.
268
+
269
+ #### JOBS_SERVICE_REQUIRED
270
+
271
+ - **Context:** `build()`.
272
+ - **Cause:** No service name.
273
+ - **Solution:** Call `.withService()`.
274
+
275
+ #### JOBS_CONTEXT_REQUIRED
276
+
277
+ - **Context:** `build()`.
278
+ - **Cause:** No context factory.
279
+ - **Solution:** Call `.withContext()`.
280
+
281
+ #### JOBS_CONFIGURATION_INVALID
225
282
 
226
- // 2. Schema infers handler data type
227
- handler: async ({ data }) => { ... }
228
- // data type: { email: string }
283
+ - **Context:** Various.
284
+ - **Cause:** Missing environment or invalid scope.
285
+ - **Solution:** Follow builder hints.
229
286
 
230
- // 3. Queue builder accumulates job types
231
- .addJob('sendWelcome', { schema, handler })
232
- // Jobs type: { sendWelcome: { data: { email: string } } }
287
+ #### JOBS_QUEUE_NOT_FOUND
233
288
 
234
- // 4. Runtime proxy provides typed dispatch
235
- jobs.email.sendWelcome.dispatch({ email: 'x' })
236
- // Argument type: { email: string }
289
+ - **Context:** Worker creation.
290
+ - **Cause:** Wrong queue name.
291
+ - **Solution:** Check `addQueue()` vs `worker.addQueue()`.
292
+
293
+ #### JOBS_QUEUE_DUPLICATE
294
+
295
+ - **Context:** Initialization.
296
+ - **Cause:** Same name twice.
297
+ - **Solution:** Use unique names.
298
+
299
+ #### JOBS_INVALID_DEFINITION
300
+
301
+ - **Context:** Job registration.
302
+ - **Cause:** Malformed job object.
303
+ - **Solution:** Ensure `handler` exists.
304
+
305
+ #### JOBS_HANDLER_REQUIRED
306
+
307
+ - **Context:** Job registration.
308
+ - **Cause:** Handler is null.
309
+ - **Solution:** Provide async function.
310
+
311
+ #### JOBS_DUPLICATE_JOB
312
+
313
+ - **Context:** Queue building.
314
+ - **Cause:** Two jobs with same name.
315
+ - **Solution:** Rename one.
316
+
317
+ #### JOBS_NOT_FOUND
318
+
319
+ - **Context:** `get(id)`.
320
+ - **Cause:** Invalid ID.
321
+ - **Solution:** Verify ID or check retention.
322
+
323
+ #### JOBS_NOT_REGISTERED
324
+
325
+ - **Context:** Worker execution.
326
+ - **Cause:** Worker doesn't know job.
327
+ - **Solution:** Check runtime config parity.
328
+
329
+ #### JOBS_EXECUTION_FAILED
330
+
331
+ - **Context:** Worker.
332
+ - **Cause:** Handler threw error.
333
+ - **Solution:** Check business logic.
334
+
335
+ #### JOBS_TIMEOUT
336
+
337
+ - **Context:** Worker.
338
+ - **Cause:** Job took too long.
339
+ - **Solution:** Increase `timeout` in job opts.
340
+
341
+ #### JOBS_CONTEXT_FACTORY_FAILED
342
+
343
+ - **Context:** Worker.
344
+ - **Cause:** Factory threw error.
345
+ - **Solution:** Check DB/resource health.
346
+
347
+ #### JOBS_VALIDATION_FAILED
348
+
349
+ - **Context:** `dispatch`.
350
+ - **Cause:** Schema mismatch.
351
+ - **Solution:** Correct the input.
352
+
353
+ #### JOBS_INVALID_INPUT
354
+
355
+ - **Context:** Runtime.
356
+ - **Cause:** Malformed data.
357
+ - **Solution:** Validate early.
358
+
359
+ #### JOBS_INVALID_CRON
360
+
361
+ - **Context:** Registration.
362
+ - **Cause:** Bad syntax.
363
+ - **Solution:** Check cron expression.
364
+
365
+ #### JOBS_INVALID_SCHEDULE
366
+
367
+ - **Context:** `schedule`.
368
+ - **Cause:** Past date.
369
+ - **Solution:** Use future date.
370
+
371
+ #### JOBS_SCOPE_ALREADY_DEFINED
372
+
373
+ - **Context:** Builder.
374
+ - **Cause:** Two scopes added.
375
+ - **Solution:** Only one allowed.
376
+
377
+ #### JOBS_WORKER_FAILED
378
+
379
+ - **Context:** Worker.
380
+ - **Cause:** Connection failure.
381
+ - **Solution:** Check Redis health.
382
+
383
+ #### JOBS_ADAPTER_ERROR
384
+
385
+ - **Context:** Generic.
386
+ - **Cause:** Backend failure.
387
+ - **Solution:** Check Redis/Adapter logs.
388
+
389
+ #### JOBS_ADAPTER_CONNECTION_FAILED
390
+
391
+ - **Context:** Generic.
392
+ - **Cause:** Redis down.
393
+ - **Solution:** Restore connectivity.
394
+
395
+ #### JOBS_SUBSCRIBE_FAILED
396
+
397
+ - **Context:** `subscribe`.
398
+ - **Cause:** Pub/Sub failure.
399
+ - **Solution:** Check Redis ACLs.
400
+
401
+ ---
402
+
403
+ ## IV. ADAPTERS REFERENCE
404
+
405
+ The `@igniter-js/jobs` package supports multiple adapters for different use cases and environments. This section provides comprehensive documentation for each adapter.
406
+
407
+ ### 16. Adapter Architecture Overview
408
+
409
+ All adapters implement the `IgniterJobsAdapter` interface defined in `src/types/adapter.ts`. This interface ensures consistent behavior across different storage backends.
410
+
411
+ ```
412
+ ┌─────────────────────────────────────────────────────────────────┐
413
+ │ IgniterJobsAdapter Interface │
414
+ ├─────────────────────────────────────────────────────────────────┤
415
+ │ Job Management │
416
+ │ ├── registerJob(queue, name, definition) │
417
+ │ ├── registerCron(queue, name, definition) │
418
+ │ ├── dispatch(params) → jobId │
419
+ │ ├── schedule(params) → jobId │
420
+ │ ├── getJob(id) → job | null │
421
+ │ ├── getJobState(id) → status | null │
422
+ │ ├── getJobLogs(id) → logs[] │
423
+ │ ├── getJobProgress(id) → number │
424
+ │ ├── removeJob(id) │
425
+ │ ├── retryJob(id) │
426
+ │ ├── promoteJob(id) │
427
+ │ └── moveJobToFailed(id, error) │
428
+ ├─────────────────────────────────────────────────────────────────┤
429
+ │ Queue Management │
430
+ │ ├── listQueues() → queue[] │
431
+ │ ├── getQueueInfo(name) → info | null │
432
+ │ ├── getQueueJobCounts(name) → counts │
433
+ │ ├── pauseQueue(name) │
434
+ │ ├── resumeQueue(name) │
435
+ │ ├── drainQueue(name) → removed │
436
+ │ ├── cleanQueue(name, options) → cleaned │
437
+ │ ├── obliterateQueue(name, options) │
438
+ │ └── retryAllInQueue(name) → retried │
439
+ ├─────────────────────────────────────────────────────────────────┤
440
+ │ Worker Management │
441
+ │ ├── createWorker(config) → WorkerHandle │
442
+ │ ├── getWorkers() → Map<id, worker> │
443
+ │ └── searchWorkers(filter) → workers[] │
444
+ ├─────────────────────────────────────────────────────────────────┤
445
+ │ Search & Discovery │
446
+ │ ├── searchJobs(filter) → jobs[] │
447
+ │ └── searchQueues(filter) → queues[] │
448
+ ├─────────────────────────────────────────────────────────────────┤
449
+ │ Pub/Sub │
450
+ │ ├── publishEvent(channel, data) │
451
+ │ └── subscribeEvent(channel, handler) → unsubscribe │
452
+ ├─────────────────────────────────────────────────────────────────┤
453
+ │ Lifecycle │
454
+ │ └── shutdown() │
455
+ └─────────────────────────────────────────────────────────────────┘
237
456
  ```
238
457
 
458
+ ### 17. Adapter Comparison Matrix
459
+
460
+ | Feature | Memory | SQLite | BullMQ |
461
+ |---------|--------|--------|--------|
462
+ | **Persistence** | ❌ In-memory only | ✅ File-based | ✅ Redis |
463
+ | **Multi-process** | ❌ Single process | ⚠️ Single process (WAL) | ✅ Distributed |
464
+ | **Installation** | Built-in | `better-sqlite3` | `bullmq` + Redis |
465
+ | **Use Case** | Tests, demos | Desktop, CLI, MCP | Production, scale |
466
+ | **Job Limits** | Memory-bound | Disk-bound | Redis-bound |
467
+ | **Worker Mechanism** | Immediate | Polling-based | Event-driven |
468
+ | **Cron Jobs** | ⚠️ Basic | ⚠️ Basic | ✅ Full |
469
+ | **Distributed Locking** | ❌ | ❌ | ✅ |
470
+ | **Rate Limiting** | ⚠️ Basic | ⚠️ Basic | ✅ Advanced |
471
+
239
472
  ---
240
473
 
241
- ## 5. Error Handling
474
+ ### 18. Memory Adapter (Testing & Development)
242
475
 
243
- ### Error Codes
476
+ The `IgniterJobsMemoryAdapter` provides a full-featured in-memory implementation perfect for unit tests and quick development.
244
477
 
245
- **Configuration Errors:**
246
- - `JOBS_MISSING_ADAPTER` - No adapter provided
247
- - `JOBS_INVALID_QUEUE_NAME` - Invalid queue name
248
- - `JOBS_INVALID_JOB_NAME` - Invalid job name
249
- - `JOBS_DUPLICATE_QUEUE` - Queue already registered
250
- - `JOBS_DUPLICATE_JOB` - Job already registered
478
+ #### 18.1 When to Use
251
479
 
252
- **Queue Errors:**
253
- - `JOBS_QUEUE_NOT_FOUND` - Queue doesn't exist
254
- - `JOBS_QUEUE_PAUSED` - Queue is paused
255
- - `JOBS_QUEUE_FAILED_TO_PAUSE` - Pause operation failed
480
+ - **Unit Tests:** Fast, isolated, no external dependencies.
481
+ - **Integration Tests:** When you need predictable job behavior.
482
+ - **Development:** Quick experimentation without Redis.
483
+ - **CI/CD Pipelines:** Zero-config test environments.
256
484
 
257
- **Job Errors:**
258
- - `JOBS_JOB_NOT_FOUND` - Job doesn't exist
259
- - `JOBS_JOB_VALIDATION_FAILED` - Payload validation failed
260
- - `JOBS_JOB_HANDLER_FAILED` - Handler threw error
261
- - `JOBS_JOB_TIMEOUT` - Job exceeded timeout
485
+ #### 18.2 Installation
262
486
 
263
- **Dispatch Errors:**
264
- - `JOBS_DISPATCH_FAILED` - Dispatch operation failed
265
- - `JOBS_SCHEDULE_FAILED` - Schedule operation failed
266
- - `JOBS_INVALID_DISPATCH_PAYLOAD` - Invalid payload
487
+ No additional dependencies required. The adapter is built into `@igniter-js/jobs`.
267
488
 
268
- ### Error Class
489
+ #### 18.3 Basic Usage
269
490
 
270
491
  ```typescript
271
- throw new IgniterJobsError({
272
- code: 'JOBS_JOB_NOT_FOUND',
273
- message: `Job "${jobId}" not found in queue "${queue}"`,
274
- statusCode: 404,
275
- details: { jobId, queue },
276
- })
492
+ import { IgniterJobs } from "@igniter-js/jobs";
493
+ import { IgniterJobsMemoryAdapter } from "@igniter-js/jobs/adapters";
494
+
495
+ const adapter = IgniterJobsMemoryAdapter.create();
496
+
497
+ const jobs = IgniterJobs.create()
498
+ .withAdapter(adapter)
499
+ .withService("test")
500
+ .withContext(async () => ({ db: mockDb }))
501
+ .addQueue(emailQueue)
502
+ .build();
277
503
  ```
278
504
 
279
- ---
505
+ #### 18.4 Configuration Options
280
506
 
281
- ## 6. Development Guidelines
507
+ ```typescript
508
+ interface IgniterJobsMemoryAdapterOptions {
509
+ /** Maximum jobs to retain in history (default: 1000) */
510
+ maxJobHistory?: number;
511
+ }
512
+ ```
282
513
 
283
- ### Adding New Features
514
+ #### 18.5 Testing Patterns
284
515
 
285
- 1. **Add Types First**
286
- - Define in appropriate `src/types/*.ts` file
287
- - Export from `src/types/index.ts`
288
- - Export from `src/index.ts` if public
516
+ ```typescript
517
+ // Access internal state for assertions
518
+ const adapter = IgniterJobsMemoryAdapter.create();
519
+
520
+ // Dispatch a job
521
+ const jobId = await adapter.dispatch({
522
+ queue: "email",
523
+ jobName: "send",
524
+ input: { to: "test@example.com" },
525
+ });
526
+
527
+ // Verify job was created
528
+ const job = await adapter.getJob(jobId);
529
+ expect(job?.status).toBe("waiting");
530
+
531
+ // Simulate worker processing
532
+ const worker = await adapter.createWorker({
533
+ queues: ["email"],
534
+ concurrency: 1,
535
+ });
536
+
537
+ // Wait for processing
538
+ await new Promise((r) => setTimeout(r, 100));
539
+
540
+ // Verify completion
541
+ const completed = await adapter.getJob(jobId);
542
+ expect(completed?.status).toBe("completed");
543
+
544
+ // Cleanup
545
+ await worker.close();
546
+ ```
289
547
 
290
- 2. **Update Builders**
291
- - Add method to builder class
292
- - Update state type if needed
293
- - Maintain fluent return type
548
+ #### 18.6 Limitations
294
549
 
295
- 3. **Update Runtime**
296
- - Implement in `src/core/igniter-jobs.ts`
297
- - Add proxy handling if needed
550
+ - **Non-persistent:** All data lost on process restart.
551
+ - **Single-process:** Cannot share state across workers.
552
+ - **Memory-bound:** Large job volumes may cause OOM.
298
553
 
299
- 4. **Update Adapter Interface**
300
- - Add method to `IgniterJobsAdapter`
301
- - Implement in BullMQ adapter
302
- - Implement in Memory adapter
554
+ ---
303
555
 
304
- 5. **Add Tests**
305
- - Unit test for builder method
306
- - Integration test for runtime
307
- - Adapter test for implementation
556
+ ### 19. SQLite Adapter (Desktop, CLI, MCP Servers)
308
557
 
309
- ### Testing Strategy
558
+ The `IgniterJobsSQLiteAdapter` provides persistent job storage using SQLite, ideal for local environments where Redis is impractical.
559
+
560
+ #### 19.1 When to Use
561
+
562
+ - **Desktop Applications:** Tauri, Electron apps needing background tasks.
563
+ - **CLI Tools:** Long-running commands with progress tracking.
564
+ - **MCP Servers:** Model Context Protocol servers with job queues.
565
+ - **Edge Deployments:** Single-node deployments.
566
+ - **Local Development:** When you want jobs to survive restarts.
567
+
568
+ #### 19.2 Installation
310
569
 
311
570
  ```bash
312
- # Run all tests
313
- npm test
571
+ npm install better-sqlite3
572
+ # or
573
+ bun add better-sqlite3
574
+ ```
575
+
576
+ The `better-sqlite3` package is an optional peer dependency.
577
+
578
+ #### 19.3 Basic Usage
579
+
580
+ ```typescript
581
+ import { IgniterJobs } from "@igniter-js/jobs";
582
+ import { IgniterJobsSQLiteAdapter } from "@igniter-js/jobs/adapters";
314
583
 
315
- # Watch mode
316
- npm run test:watch
584
+ // File-based (persistent)
585
+ const adapter = IgniterJobsSQLiteAdapter.create({
586
+ path: "./jobs.sqlite",
587
+ });
317
588
 
318
- # Coverage
319
- npm run test:coverage
589
+ // In-memory (for tests)
590
+ const memoryAdapter = IgniterJobsSQLiteAdapter.create({
591
+ path: ":memory:",
592
+ });
593
+
594
+ const jobs = IgniterJobs.create()
595
+ .withAdapter(adapter)
596
+ .withService("desktop-app")
597
+ .withContext(async () => ({ settings: loadSettings() }))
598
+ .addQueue(syncQueue)
599
+ .build();
320
600
  ```
321
601
 
322
- **Test Files:**
323
- - `tests/igniter-queue.builder.test.ts` - Queue builder
324
- - `tests/igniter-jobs.builder.test.ts` - Jobs builder
325
- - `tests/igniter-jobs.runtime.test.ts` - Integration
326
- - `tests/memory.adapter.test.ts` - Memory adapter
602
+ #### 19.4 Configuration Options
327
603
 
328
- ### Code Style
604
+ ```typescript
605
+ interface IgniterJobsSQLiteAdapterOptions {
606
+ /**
607
+ * Path to the SQLite database file.
608
+ * Use ":memory:" for in-memory database.
609
+ * @example "./data/jobs.sqlite"
610
+ */
611
+ path: string;
612
+
613
+ /**
614
+ * Polling interval in milliseconds for workers to check for new jobs.
615
+ * Lower values = more responsive, higher CPU usage.
616
+ * @default 500
617
+ */
618
+ pollingInterval?: number;
619
+
620
+ /**
621
+ * Enable WAL (Write-Ahead Logging) mode for better concurrent performance.
622
+ * Recommended for production use.
623
+ * @default true
624
+ */
625
+ enableWAL?: boolean;
626
+ }
627
+ ```
329
628
 
330
- - Use JSDoc for all public APIs
331
- - Follow existing patterns exactly
332
- - No `any` types in public API
333
- - Export types explicitly
334
- - Use `readonly` where appropriate
629
+ #### 19.5 Database Schema
630
+
631
+ The adapter automatically creates and manages the following tables:
632
+
633
+ ```sql
634
+ -- Main jobs table
635
+ CREATE TABLE IF NOT EXISTS jobs (
636
+ id TEXT PRIMARY KEY,
637
+ queue TEXT NOT NULL,
638
+ name TEXT NOT NULL,
639
+ input TEXT, -- JSON serialized
640
+ result TEXT, -- JSON serialized
641
+ error TEXT,
642
+ status TEXT NOT NULL DEFAULT 'waiting',
643
+ progress INTEGER DEFAULT 0,
644
+ priority INTEGER DEFAULT 0,
645
+ attempts_made INTEGER DEFAULT 0,
646
+ max_attempts INTEGER DEFAULT 1,
647
+ delay_until INTEGER, -- Unix timestamp
648
+ metadata TEXT, -- JSON serialized
649
+ scope_type TEXT,
650
+ scope_id TEXT,
651
+ created_at INTEGER NOT NULL,
652
+ started_at INTEGER,
653
+ completed_at INTEGER
654
+ );
655
+
656
+ -- Indexes for efficient queries
657
+ CREATE INDEX IF NOT EXISTS idx_jobs_queue_status ON jobs(queue, status);
658
+ CREATE INDEX IF NOT EXISTS idx_jobs_status ON jobs(status);
659
+ CREATE INDEX IF NOT EXISTS idx_jobs_priority ON jobs(priority DESC);
660
+ CREATE INDEX IF NOT EXISTS idx_jobs_delay ON jobs(delay_until);
661
+
662
+ -- Job logs table
663
+ CREATE TABLE IF NOT EXISTS job_logs (
664
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
665
+ job_id TEXT NOT NULL,
666
+ level TEXT NOT NULL,
667
+ message TEXT NOT NULL,
668
+ timestamp INTEGER NOT NULL,
669
+ FOREIGN KEY (job_id) REFERENCES jobs(id) ON DELETE CASCADE
670
+ );
671
+
672
+ -- Paused queues tracking
673
+ CREATE TABLE IF NOT EXISTS paused_queues (
674
+ name TEXT PRIMARY KEY,
675
+ paused_at INTEGER NOT NULL
676
+ );
677
+ ```
335
678
 
336
- ---
679
+ #### 19.6 Worker Polling Mechanism
337
680
 
338
- ## 7. Adapter Implementation Guide
681
+ Unlike event-driven adapters (BullMQ), the SQLite adapter uses a polling-based approach:
339
682
 
340
- ### Required Methods
683
+ ```
684
+ ┌─────────────────────────────────────────────────────────────────┐
685
+ │ Worker Polling Loop │
686
+ ├─────────────────────────────────────────────────────────────────┤
687
+ │ │
688
+ │ ┌─────────┐ ┌─────────────┐ ┌─────────────┐ │
689
+ │ │ Wait │────▶│ Check Jobs │────▶│ Process Job │ │
690
+ │ │ Polling │ │ Available? │ │ (if any) │ │
691
+ │ │ Interval│ └─────────────┘ └─────────────┘ │
692
+ │ └─────────┘ │ │ │
693
+ │ ▲ │ No jobs │ Done │
694
+ │ │ │ ▼ │
695
+ │ └────────────────┴───────────────────┘ │
696
+ │ │
697
+ │ Query Priority: │
698
+ │ 1. Delayed jobs ready to run (delay_until <= NOW) │
699
+ │ 2. Waiting jobs, ordered by priority DESC │
700
+ │ 3. Retry jobs (failed but attempts < max_attempts) │
701
+ │ │
702
+ └─────────────────────────────────────────────────────────────────┘
703
+ ```
341
704
 
342
- When implementing a new adapter, implement ALL methods from `IgniterJobsAdapter`:
705
+ #### 19.7 Complete Example
343
706
 
344
707
  ```typescript
345
- interface IgniterJobsAdapter {
346
- // Connection
347
- readonly client: unknown
348
-
349
- // Job Operations
350
- dispatch(params: AdapterDispatchParams): Promise<string>
351
- schedule(params: AdapterScheduleParams): Promise<string>
352
- getJob(queue: string, jobId: string): Promise<IgniterJobInfo | null>
353
- getJobState(queue: string, jobId: string): Promise<IgniterJobStatus | null>
354
- getJobProgress(queue: string, jobId: string): Promise<number>
355
- getJobLogs(queue: string, jobId: string): Promise<IgniterJobLog[]>
356
- retryJob(queue: string, jobId: string): Promise<void>
357
- removeJob(queue: string, jobId: string): Promise<void>
358
- promoteJob(queue: string, jobId: string): Promise<void>
359
- moveJob(queue: string, jobId: string, state: string, reason?: string): Promise<void>
360
- retryJobs(queue: string, jobIds: string[]): Promise<void>
361
- removeJobs(queue: string, jobIds: string[]): Promise<void>
362
-
363
- // Queue Operations
364
- getQueue(queue: string): Promise<IgniterQueueInfo>
365
- pauseQueue(queue: string): Promise<void>
366
- resumeQueue(queue: string): Promise<void>
367
- drainQueue(queue: string): Promise<number>
368
- cleanQueue(queue: string, options: IgniterQueueCleanOptions): Promise<number>
369
- obliterateQueue(queue: string, options?: { force?: boolean }): Promise<void>
370
- retryAllFailed(queue: string): Promise<number>
371
- getJobCounts(queue: string): Promise<IgniterJobCounts>
372
- listJobs(queue: string, options?: {...}): Promise<IgniterJobSearchResult[]>
373
- pauseJobType(queue: string, jobName: string): Promise<void>
374
- resumeJobType(queue: string, jobName: string): Promise<void>
375
-
376
- // Events
377
- subscribe(pattern: string, handler: IgniterJobEventHandler): Promise<IgniterJobUnsubscribeFn>
378
-
379
- // Workers
380
- createWorker(config: AdapterWorkerConfig, handler: AdapterJobHandler): Promise<AdapterWorkerHandle>
381
-
382
- // Search
383
- searchJobs(filter: {...}): Promise<IgniterJobSearchResult[]>
384
- searchQueues(filter: {...}): Promise<IgniterQueueInfo[]>
385
-
386
- // Lifecycle
387
- shutdown(): Promise<void>
708
+ import { IgniterJobsSQLiteAdapter } from "@igniter-js/jobs/adapters";
709
+
710
+ async function main() {
711
+ // 1. Create adapter
712
+ const adapter = IgniterJobsSQLiteAdapter.create({
713
+ path: "./my-app-jobs.sqlite",
714
+ pollingInterval: 500,
715
+ enableWAL: true,
716
+ });
717
+
718
+ // 2. Register jobs
719
+ adapter.registerJob("sync", "upload", {
720
+ handler: async ({ input }) => {
721
+ const { fileId } = input as { fileId: string };
722
+ // Simulate upload
723
+ await uploadFile(fileId);
724
+ return { uploaded: true };
725
+ },
726
+ onProgress: async ({ progress }) => {
727
+ console.log(`Upload progress: ${progress}%`);
728
+ },
729
+ });
730
+
731
+ // 3. Create worker
732
+ const worker = await adapter.createWorker({
733
+ queues: ["sync"],
734
+ concurrency: 2,
735
+ handlers: {
736
+ onActive: ({ job }) => console.log(`Processing: ${job.id}`),
737
+ onSuccess: ({ job }) => console.log(`Completed: ${job.id}`),
738
+ onFailure: ({ job, error }) => console.error(`Failed: ${job.id}`, error),
739
+ },
740
+ });
741
+
742
+ // 4. Dispatch jobs
743
+ const jobId = await adapter.dispatch({
744
+ queue: "sync",
745
+ jobName: "upload",
746
+ input: { fileId: "file_123" },
747
+ priority: 10,
748
+ });
749
+
750
+ console.log(`Dispatched job: ${jobId}`);
751
+
752
+ // 5. Monitor status
753
+ const counts = await adapter.getQueueJobCounts("sync");
754
+ console.log(`Queue status:`, counts);
755
+
756
+ // 6. Graceful shutdown
757
+ process.on("SIGINT", async () => {
758
+ await worker.close();
759
+ await adapter.shutdown();
760
+ process.exit(0);
761
+ });
388
762
  }
763
+
764
+ main().catch(console.error);
389
765
  ```
390
766
 
767
+ #### 19.8 Persistence & Recovery
768
+
769
+ Jobs are automatically persisted and survive process restarts:
770
+
771
+ ```typescript
772
+ // Session 1: Dispatch job and exit
773
+ const adapter = IgniterJobsSQLiteAdapter.create({ path: "./jobs.sqlite" });
774
+ await adapter.dispatch({
775
+ queue: "emails",
776
+ jobName: "send",
777
+ input: { to: "user@example.com" },
778
+ });
779
+ await adapter.shutdown();
780
+ // Process exits
781
+
782
+ // Session 2: Jobs are still there
783
+ const adapter2 = IgniterJobsSQLiteAdapter.create({ path: "./jobs.sqlite" });
784
+ const jobs = await adapter2.searchJobs({ queue: "emails", status: ["waiting"] });
785
+ console.log(jobs); // Shows the pending job from session 1
786
+ ```
787
+
788
+ #### 19.9 Limitations
789
+
790
+ - **Single Process:** WAL mode helps, but true concurrent writes from multiple processes may cause issues.
791
+ - **Polling Overhead:** Unlike event-driven systems, there's CPU overhead from polling.
792
+ - **No Distributed Locking:** Cannot guarantee exactly-once processing across processes.
793
+ - **Limited Cron:** Basic cron support without advanced scheduling features.
794
+
391
795
  ---
392
796
 
393
- ## 8. Common Patterns
797
+ ### 20. BullMQ Adapter (Production Scale)
798
+
799
+ The `IgniterJobsBullMQAdapter` (external package) provides Redis-based job queuing for production environments.
394
800
 
395
- ### Multi-tenant Job Dispatch
801
+ #### 20.1 When to Use
802
+
803
+ - **Production Deployments:** High availability and reliability.
804
+ - **Distributed Systems:** Multiple workers across nodes.
805
+ - **High Throughput:** Thousands of jobs per second.
806
+ - **Advanced Features:** Rate limiting, priorities, delayed jobs, cron.
807
+
808
+ #### 20.2 Installation
809
+
810
+ ```bash
811
+ npm install @igniter-js/adapter-bullmq bullmq ioredis
812
+ ```
813
+
814
+ #### 20.3 Basic Usage
396
815
 
397
816
  ```typescript
817
+ import { IgniterJobs } from "@igniter-js/jobs";
818
+ import { IgniterJobsBullMQAdapter } from "@igniter-js/adapter-bullmq";
819
+ import Redis from "ioredis";
820
+
821
+ const redis = new Redis(process.env.REDIS_URL);
822
+
823
+ const adapter = IgniterJobsBullMQAdapter.create({ redis });
824
+
398
825
  const jobs = IgniterJobs.create()
399
826
  .withAdapter(adapter)
827
+ .withService("api-server")
828
+ .withContext(async () => ({ db, cache }))
400
829
  .addQueue(emailQueue)
401
- .addScope('organization', {
402
- resolver: (id) => db.org.findUnique({ where: { id } }),
403
- })
404
- .addActor('user', {
405
- resolver: (id) => db.user.findUnique({ where: { id } }),
406
- })
407
- .build()
408
-
409
- // Dispatch with scope/actor
410
- await jobs.email.sendWelcome.dispatch(
411
- { email: 'test@example.com' },
412
- {
413
- scope: { type: 'organization', id: 'org-123' },
414
- actor: { type: 'user', id: 'user-456' },
415
- }
416
- )
830
+ .addQueue(analyticsQueue)
831
+ .build();
417
832
  ```
418
833
 
419
- ### Job With Progress Updates
834
+ #### 20.4 Reference
835
+
836
+ See `@igniter-js/adapter-bullmq` package documentation for full configuration options.
837
+
838
+ ---
839
+
840
+ ### 21. Implementing Custom Adapters
841
+
842
+ To create a custom adapter, implement the `IgniterJobsAdapter` interface:
420
843
 
421
844
  ```typescript
422
- const queue = IgniterQueue.create('exports')
423
- .addJob('generateReport', {
424
- schema: z.object({ reportId: z.string() }),
425
- handler: async ({ data, updateProgress, log }) => {
426
- await updateProgress(10)
427
- await log('info', 'Starting report generation')
845
+ import type {
846
+ IgniterJobsAdapter,
847
+ IgniterJobsDispatchParams,
848
+ IgniterJobsWorkerConfig,
849
+ IgniterJobsWorkerHandle,
850
+ IgniterJobSearchResult,
851
+ IgniterJobsQueueInfo,
852
+ } from "@igniter-js/jobs";
853
+
854
+ export class CustomAdapter implements IgniterJobsAdapter {
855
+ // ... implement all required methods
856
+ }
857
+ ```
428
858
 
429
- // ... process
859
+ #### 21.1 Required Method Signatures
430
860
 
431
- await updateProgress(50)
432
- await log('info', 'Halfway done')
861
+ See `src/types/adapter.ts` for the complete interface definition.
433
862
 
434
- // ... finish
863
+ #### 21.2 Testing Custom Adapters
435
864
 
436
- await updateProgress(100)
437
- return { success: true }
438
- },
439
- })
440
- .build()
865
+ Use the existing test suites as a reference:
866
+ - `src/adapters/memory.adapter.spec.ts`
867
+ - `src/adapters/sqlite.adapter.spec.ts`
868
+
869
+ ---
870
+
871
+ ## V. EXAMPLES & USE CASES
872
+
873
+ ### 22. Example Directory Structure
874
+
875
+ The `examples/` directory contains runnable examples:
876
+
877
+ ```
878
+ packages/jobs/examples/
879
+ ├── README.md # Example documentation
880
+ └── sqlite-example.ts # Complete SQLite adapter demo
881
+ ```
882
+
883
+ ### 23. Running the SQLite Example
884
+
885
+ ```bash
886
+ cd packages/jobs
887
+ npx tsx examples/sqlite-example.ts
441
888
  ```
442
889
 
443
- ### Scheduled Jobs
890
+ This example demonstrates:
891
+ - Creating an SQLite adapter with file persistence
892
+ - Registering multiple job types
893
+ - Priority-based job processing
894
+ - Delayed job scheduling
895
+ - Worker metrics and lifecycle handlers
896
+ - Queue status monitoring
897
+ - Graceful shutdown
898
+
899
+ ---
900
+
901
+ ## VI. MIGRATION GUIDE
902
+
903
+ ### 24. Migrating from Memory to SQLite
904
+
905
+ For development to production-like local testing:
444
906
 
445
907
  ```typescript
446
- // Schedule at specific time
447
- await jobs.email.sendWelcome.schedule({
448
- data: { email: 'test@example.com' },
449
- at: new Date('2025-12-25T00:00:00Z'),
450
- })
908
+ // Before: Memory adapter
909
+ import { IgniterJobsMemoryAdapter } from "@igniter-js/jobs/adapters";
910
+ const adapter = IgniterJobsMemoryAdapter.create();
451
911
 
452
- // Schedule with delay
453
- await jobs.email.sendWelcome.schedule({
454
- data: { email: 'test@example.com' },
455
- delay: 60000, // 1 minute
456
- })
912
+ // After: SQLite adapter
913
+ import { IgniterJobsSQLiteAdapter } from "@igniter-js/jobs/adapters";
914
+ const adapter = IgniterJobsSQLiteAdapter.create({
915
+ path: "./dev-jobs.sqlite",
916
+ });
917
+
918
+ // Rest of the code remains identical!
457
919
  ```
458
920
 
459
- ### Worker with Rate Limiting
921
+ ### 25. Migrating from SQLite to BullMQ
922
+
923
+ For local development to production:
460
924
 
461
925
  ```typescript
462
- const worker = await jobs.worker
463
- .forQueues(['email'])
464
- .withConcurrency(10)
465
- .withLimiter({
466
- max: 100, // max jobs
467
- duration: 60000, // per minute
468
- })
469
- .withLockDuration(30000)
470
- .onIdle(() => console.log('No jobs to process'))
471
- .start()
926
+ // Before: SQLite adapter
927
+ import { IgniterJobsSQLiteAdapter } from "@igniter-js/jobs/adapters";
928
+ const adapter = IgniterJobsSQLiteAdapter.create({
929
+ path: "./jobs.sqlite",
930
+ });
931
+
932
+ // After: BullMQ adapter
933
+ import { IgniterJobsBullMQAdapter } from "@igniter-js/adapter-bullmq";
934
+ const adapter = IgniterJobsBullMQAdapter.create({
935
+ redis: new Redis(process.env.REDIS_URL),
936
+ });
937
+
938
+ // Rest of the code remains identical!
472
939
  ```
473
940
 
474
941
  ---
475
942
 
476
- ## 9. Version History
943
+ ## VII. MAINTENANCE CHANGELOG
477
944
 
478
- ### 0.1.0 (Initial Release)
945
+ ### Version 0.2.0 (2026-01-17)
479
946
 
480
- - IgniterQueue builder with addJob/addCron
481
- - IgniterJobs builder with full configuration
482
- - IgniterWorker builder with fluent API
483
- - Proxy-based runtime for type-safe access
484
- - BullMQ adapter with full feature support
485
- - Memory adapter for testing
486
- - ✅ Scope and actor support
487
- - Event subscription system
488
- - ✅ Job and queue search
489
- - Error codes and classes
947
+ - **Added:** `IgniterJobsSQLiteAdapter` for persistent local job queues
948
+ - **Added:** SQLite adapter comprehensive test suite (71 tests)
949
+ - **Added:** `examples/` directory with runnable examples
950
+ - **Added:** `better-sqlite3` as optional peer dependency
951
+ - **Updated:** Adapter exports to include SQLite adapter
952
+ - **Updated:** AGENTS.md with comprehensive adapter documentation
953
+
954
+ ### Version 0.1.2 (2025-12-23)
955
+
956
+ - Initial stable release
957
+ - Memory adapter for testing
958
+ - BullMQ adapter reference
490
959
 
491
960
  ---
492
961
 
493
- ## 10. Troubleshooting
962
+ ## VIII. FAQ & TROUBLESHOOTING
963
+
964
+ ### 26. Frequently Asked Questions
494
965
 
495
- ### Common Issues
966
+ #### Q: Which adapter should I choose?
496
967
 
497
- **"JOBS_MISSING_ADAPTER"**
498
- - Ensure `withAdapter()` is called before `build()`
499
- - Check adapter is properly instantiated
968
+ | Environment | Recommended Adapter |
969
+ |-------------|---------------------|
970
+ | Unit/Integration Tests | Memory Adapter |
971
+ | Desktop App (Tauri/Electron) | SQLite Adapter |
972
+ | CLI Tool | SQLite Adapter |
973
+ | MCP Server | SQLite Adapter |
974
+ | Local Development | SQLite Adapter |
975
+ | Production (Single Node) | SQLite or BullMQ |
976
+ | Production (Multi Node) | BullMQ Adapter |
977
+ | Serverless | BullMQ Adapter |
500
978
 
501
- **"JOBS_QUEUE_NOT_FOUND"**
502
- - Queue must be registered with `addQueue()`
503
- - Check queue name spelling
979
+ #### Q: Can I switch adapters without changing business logic?
504
980
 
505
- **"JOBS_JOB_VALIDATION_FAILED"**
506
- - Payload doesn't match Zod schema
507
- - Check `details.issues` for validation errors
981
+ **Yes!** All adapters implement the same `IgniterJobsAdapter` interface. Your job definitions, handlers, and queue configurations remain identical.
508
982
 
509
- **TypeScript errors on dispatch**
510
- - Ensure queue is added before build
511
- - Check job name exists in queue
512
- - Verify payload matches schema
983
+ #### Q: How do I handle job failures?
513
984
 
514
- ### Debug Tips
985
+ ```typescript
986
+ adapter.registerJob("email", "send", {
987
+ handler: async ({ input }) => {
988
+ // Your logic here
989
+ },
990
+ attempts: 3, // Retry up to 3 times
991
+ onFailure: async ({ error, isFinalAttempt }) => {
992
+ if (isFinalAttempt) {
993
+ // Send alert, log to monitoring, etc.
994
+ await alertOps(`Job failed permanently: ${error.message}`);
995
+ }
996
+ },
997
+ });
998
+ ```
999
+
1000
+ #### Q: How do I track job progress?
515
1001
 
516
1002
  ```typescript
517
- // Enable BullMQ debug logging
518
- process.env.DEBUG = 'bullmq:*'
1003
+ adapter.registerJob("import", "csv", {
1004
+ handler: async ({ input, job }) => {
1005
+ const rows = await parseCSV(input.file);
1006
+
1007
+ for (let i = 0; i < rows.length; i++) {
1008
+ await processRow(rows[i]);
1009
+
1010
+ // Update progress
1011
+ await adapter.updateJobProgress(job.id, Math.floor((i / rows.length) * 100));
1012
+ }
1013
+
1014
+ return { processed: rows.length };
1015
+ },
1016
+ onProgress: async ({ progress, message }) => {
1017
+ console.log(`Progress: ${progress}%`);
1018
+ },
1019
+ });
1020
+ ```
519
1021
 
520
- // Check job state
521
- const state = await jobs.job.state('queue', 'job-id')
522
- console.log('Job state:', state)
1022
+ #### Q: Can SQLite handle high throughput?
523
1023
 
524
- // Get job logs
525
- const logs = await jobs.job.logs('queue', 'job-id')
526
- console.log('Job logs:', logs)
1024
+ SQLite with WAL mode can handle moderate throughput (100-1000 jobs/minute) on a single machine. For higher throughput or distributed processing, use BullMQ.
527
1025
 
528
- // Subscribe to all events
529
- await jobs.subscribe('*', (event) => {
530
- console.log('Event:', event.type, event.data)
531
- })
1026
+ ### 27. Common Issues & Solutions
1027
+
1028
+ #### Issue: "Cannot find module 'better-sqlite3'"
1029
+
1030
+ **Cause:** The `better-sqlite3` peer dependency is not installed.
1031
+
1032
+ **Solution:**
1033
+ ```bash
1034
+ npm install better-sqlite3
1035
+ # If compilation fails on macOS:
1036
+ npm install better-sqlite3 --build-from-source
1037
+ ```
1038
+
1039
+ #### Issue: Jobs stuck in "active" status after restart
1040
+
1041
+ **Cause:** Process crashed while job was processing.
1042
+
1043
+ **Solution:**
1044
+ ```typescript
1045
+ // On startup, reset stale active jobs
1046
+ const staleJobs = await adapter.searchJobs({
1047
+ queue: "my-queue",
1048
+ status: ["active"],
1049
+ });
1050
+
1051
+ for (const job of staleJobs) {
1052
+ await adapter.retryJob(job.id);
1053
+ }
532
1054
  ```
533
1055
 
1056
+ #### Issue: Worker not processing jobs
1057
+
1058
+ **Cause:** Worker queues don't match job queue names.
1059
+
1060
+ **Solution:**
1061
+ ```typescript
1062
+ // Ensure queue names match
1063
+ adapter.registerJob("email", "send", { ... }); // Queue: "email"
1064
+
1065
+ // Worker must listen to "email" queue
1066
+ const worker = await adapter.createWorker({
1067
+ queues: ["email"], // Must include "email"
1068
+ concurrency: 1,
1069
+ });
1070
+ ```
1071
+
1072
+ #### Issue: SQLite database locked
1073
+
1074
+ **Cause:** Multiple processes trying to write simultaneously.
1075
+
1076
+ **Solution:**
1077
+ - Use WAL mode (enabled by default)
1078
+ - Ensure only one process writes at a time
1079
+ - Consider BullMQ for multi-process scenarios
1080
+
534
1081
  ---
535
1082
 
536
- ## 11. Dependencies
1083
+ ## IX. PERFORMANCE CONSIDERATIONS
537
1084
 
538
- ### Peer Dependencies
1085
+ ### 28. SQLite Adapter Performance Tips
539
1086
 
540
- | Package | Version | Purpose |
541
- |---------|---------|---------|
542
- | `bullmq` | `^5.0.0` | Job queue implementation |
543
- | `ioredis` | `^5.0.0` | Redis client |
544
- | `zod` | `^3.0.0` | Schema validation |
545
- | `@igniter-js/telemetry` | `^0.1.0` | Optional tracing |
1087
+ 1. **Use WAL Mode:** Always enable WAL mode for better read/write concurrency.
1088
+
1089
+ 2. **Tune Polling Interval:**
1090
+ - Lower interval (100ms) = more responsive, higher CPU
1091
+ - Higher interval (1000ms) = less responsive, lower CPU
1092
+
1093
+ 3. **Batch Operations:** When dispatching many jobs, consider batching:
1094
+ ```typescript
1095
+ const jobIds = [];
1096
+ for (const item of items) {
1097
+ jobIds.push(await adapter.dispatch({
1098
+ queue: "process",
1099
+ jobName: "item",
1100
+ input: item,
1101
+ }));
1102
+ }
1103
+ ```
1104
+
1105
+ 4. **Clean Old Jobs:** Periodically clean completed/failed jobs:
1106
+ ```typescript
1107
+ await adapter.cleanQueue("email", {
1108
+ status: ["completed", "failed"],
1109
+ olderThan: 7 * 24 * 60 * 60 * 1000, // 7 days
1110
+ });
1111
+ ```
1112
+
1113
+ 5. **Index Usage:** The adapter creates optimized indexes for common queries.
1114
+
1115
+ ### 29. Memory Considerations
1116
+
1117
+ - **Memory Adapter:** All jobs stored in RAM; limit maxJobHistory.
1118
+ - **SQLite Adapter:** Disk-based; memory usage scales with active jobs.
1119
+ - **BullMQ Adapter:** Redis memory; configure maxmemory policy.
1120
+
1121
+ ---
1122
+
1123
+ ## X. SECURITY BEST PRACTICES
1124
+
1125
+ ### 30. Input Validation
1126
+
1127
+ Always validate job inputs using schemas:
1128
+
1129
+ ```typescript
1130
+ import { z } from "zod";
1131
+
1132
+ const emailQueue = IgniterQueue.create("email")
1133
+ .addJob("send", {
1134
+ input: z.object({
1135
+ to: z.string().email(),
1136
+ subject: z.string().max(200),
1137
+ body: z.string(),
1138
+ }),
1139
+ handler: async ({ input }) => {
1140
+ // input is validated and typed
1141
+ },
1142
+ })
1143
+ .build();
1144
+ ```
546
1145
 
547
- ### Dev Dependencies
1146
+ ### 31. Sensitive Data Handling
548
1147
 
549
- | Package | Version | Purpose |
550
- |---------|---------|---------|
551
- | `typescript` | `^5.0.0` | TypeScript compiler |
552
- | `vitest` | `^1.0.0` | Testing framework |
553
- | `tsup` | `^8.0.0` | Build tool |
1148
+ - **Never log raw inputs** containing passwords, tokens, or PII.
1149
+ - **Use job IDs** for reference rather than storing sensitive data in jobs.
1150
+ - **Encrypt at rest** if storing sensitive metadata.
554
1151
 
555
1152
  ---
556
1153
 
557
- **Remember:** Always follow existing patterns. When in doubt, check how similar features are implemented in `@igniter-js/store` or `@igniter-js/core`.