@igniter-js/jobs 0.1.13 → 0.1.15
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +119 -243
- package/README.md +352 -158
- package/dist/{adapter-CXZxomI9.d.mts → adapter-DDyMVche.d.mts} +125 -20
- package/dist/{adapter-CXZxomI9.d.ts → adapter-DDyMVche.d.ts} +125 -20
- package/dist/adapters/bun.d.mts +101 -0
- package/dist/adapters/bun.d.ts +101 -0
- package/dist/adapters/bun.js +1048 -0
- package/dist/adapters/bun.js.map +1 -0
- package/dist/adapters/bun.mjs +1046 -0
- package/dist/adapters/bun.mjs.map +1 -0
- package/dist/adapters/{memory.adapter.d.ts → mock.d.mts} +7 -3
- package/dist/adapters/{memory.adapter.d.mts → mock.d.ts} +7 -3
- package/dist/adapters/{memory.adapter.js → mock.js} +122 -25
- package/dist/adapters/mock.js.map +1 -0
- package/dist/adapters/{memory.adapter.mjs → mock.mjs} +122 -25
- package/dist/adapters/mock.mjs.map +1 -0
- package/dist/adapters/{bullmq.adapter.d.mts → node.d.mts} +8 -3
- package/dist/adapters/{bullmq.adapter.d.ts → node.d.ts} +8 -3
- package/dist/adapters/{bullmq.adapter.js → node.js} +194 -40
- package/dist/adapters/node.js.map +1 -0
- package/dist/adapters/{bullmq.adapter.mjs → node.mjs} +194 -40
- package/dist/adapters/node.mjs.map +1 -0
- package/dist/index.d.mts +41 -38
- package/dist/index.d.ts +41 -38
- package/dist/index.js +145 -1856
- package/dist/index.js.map +1 -1
- package/dist/index.mjs +146 -1854
- package/dist/index.mjs.map +1 -1
- package/package.json +29 -41
- package/CHANGELOG.md +0 -13
- package/dist/adapters/bullmq.adapter.js.map +0 -1
- package/dist/adapters/bullmq.adapter.mjs.map +0 -1
- package/dist/adapters/index.d.mts +0 -143
- package/dist/adapters/index.d.ts +0 -143
- package/dist/adapters/index.js +0 -1891
- package/dist/adapters/index.js.map +0 -1
- package/dist/adapters/index.mjs +0 -1887
- package/dist/adapters/index.mjs.map +0 -1
- package/dist/adapters/memory.adapter.js.map +0 -1
- package/dist/adapters/memory.adapter.mjs.map +0 -1
package/AGENTS.md
CHANGED
|
@@ -27,6 +27,7 @@ In modern distributed systems, background processing is usually the first point
|
|
|
27
27
|
2. **Resource-Aware Context:** The `withContext()` factory ensures that every job execution starts with a fresh, validated set of resources, exactly like a standard API request.
|
|
28
28
|
3. **Multi-Platform Support (Adapters):** Whether using **BullMQ** for massive production scale or an **In-Memory** adapter for unit tests, the business logic remains identical.
|
|
29
29
|
4. **Native Observability:** Telemetry and internal event publishing are "baked in," providing immediate visibility into enqueuing, execution times, failures, and retries without writing a single line of logging code.
|
|
30
|
+
5. **Typed Job Streams:** Jobs can emit typed per-job stream events for live consumers and optional persisted replay.
|
|
30
31
|
|
|
31
32
|
---
|
|
32
33
|
|
|
@@ -45,7 +46,7 @@ Maintainers must respect the following directory structure and responsibilities:
|
|
|
45
46
|
- `queue.ts`: `IgniterQueue` facade for the queue builder.
|
|
46
47
|
- **`src/adapters/`**: The infrastructure boundary.
|
|
47
48
|
- `memory.adapter.ts`: Production-grade in-memory implementation for tests (non-persistent).
|
|
48
|
-
- `sqlite.adapter.ts`:
|
|
49
|
+
- `bun-sqlite.adapter.ts`: Bun-native SQLite adapter for local, desktop, CLI, and embedded workloads.
|
|
49
50
|
- `bullmq.adapter.ts`: Reference to external `@igniter-js/adapter-bullmq` for production Redis-based queues.
|
|
50
51
|
- **`src/errors/`**: Resiliency definitions.
|
|
51
52
|
- `jobs.error.ts`: `IgniterJobsError`. Extends `IgniterError` with metadata-rich payloads and authoritative error codes.
|
|
@@ -55,6 +56,7 @@ Maintainers must respect the following directory structure and responsibilities:
|
|
|
55
56
|
- `adapter.ts`: `IgniterJobsAdapter` interface.
|
|
56
57
|
- `runtime.ts`: Accessor interfaces for the proxy API.
|
|
57
58
|
- `job.ts`: authoritative types for job definitions, contexts, and hooks.
|
|
59
|
+
- `stream.ts`: typed per-job stream definitions, emit/read/subscribe contracts, and inference helpers.
|
|
58
60
|
- **`src/utils/`**: Static helper library.
|
|
59
61
|
- `events.utils.ts`: Pub/sub event helpers and channel composition.
|
|
60
62
|
- `id-generator.ts`: Deterministic ID utilities for adapters.
|
|
@@ -157,7 +159,9 @@ Type Flow:
|
|
|
157
159
|
The package provides organized subpath exports for optimized bundling:
|
|
158
160
|
|
|
159
161
|
- **`@igniter-js/jobs`**: The main entry point. Use for `IgniterJobs` and `IgniterQueue`.
|
|
160
|
-
- **`@igniter-js/jobs/adapters`**:
|
|
162
|
+
- **`@igniter-js/jobs/adapters/node`**: Node.js adapters (`BullMQ` wrapper).
|
|
163
|
+
- **`@igniter-js/jobs/adapters/bun`**: Bun-native SQLite adapter.
|
|
164
|
+
- **`@igniter-js/jobs/adapters/mock`**: In-memory adapter for tests and local mocks.
|
|
161
165
|
- **`@igniter-js/jobs/telemetry`**: Telemetry registry for registration.
|
|
162
166
|
|
|
163
167
|
### 9. Quick Start & Common Patterns
|
|
@@ -168,7 +172,7 @@ Setup your jobs in a central `src/services/jobs.ts` file:
|
|
|
168
172
|
|
|
169
173
|
```typescript
|
|
170
174
|
import { IgniterJobs } from "@igniter-js/jobs";
|
|
171
|
-
import { IgniterJobsBullMQAdapter } from "@igniter-js/jobs/adapters";
|
|
175
|
+
import { IgniterJobsBullMQAdapter } from "@igniter-js/jobs/adapters/node";
|
|
172
176
|
|
|
173
177
|
export const jobs = IgniterJobs.create()
|
|
174
178
|
.withAdapter(IgniterJobsBullMQAdapter.create({ redis }))
|
|
@@ -216,7 +220,7 @@ Prevent permanent failures in external API calls:
|
|
|
216
220
|
#### Case 5: SaaS Multi-Tenant CSV Import
|
|
217
221
|
|
|
218
222
|
- **Scenario:** Process 100k row CSV.
|
|
219
|
-
- **Implementation:** `job.
|
|
223
|
+
- **Implementation:** `job.updateProgress?.(percent)` every 100 rows for real-time progress bar.
|
|
220
224
|
|
|
221
225
|
### 11. Domain-Specific Guidance
|
|
222
226
|
|
|
@@ -457,17 +461,17 @@ All adapters implement the `IgniterJobsAdapter` interface defined in `src/types/
|
|
|
457
461
|
|
|
458
462
|
### 17. Adapter Comparison Matrix
|
|
459
463
|
|
|
460
|
-
| Feature
|
|
461
|
-
|
|
462
|
-
| **Persistence**
|
|
463
|
-
| **Multi-process**
|
|
464
|
-
| **Installation**
|
|
465
|
-
| **Use Case**
|
|
466
|
-
| **Job Limits**
|
|
467
|
-
| **Worker Mechanism**
|
|
468
|
-
| **Cron Jobs**
|
|
469
|
-
| **Distributed Locking** | ❌
|
|
470
|
-
| **Rate Limiting**
|
|
464
|
+
| Feature | Memory | Bun SQLite | BullMQ |
|
|
465
|
+
| ----------------------- | ----------------- | ------------------------ | ----------------- |
|
|
466
|
+
| **Persistence** | ❌ In-memory only | ✅ File-based | ✅ Redis |
|
|
467
|
+
| **Multi-process** | ❌ Single process | ⚠️ Single process/local | ✅ Distributed |
|
|
468
|
+
| **Installation** | Built-in | Bundled with package | `bullmq` + Redis |
|
|
469
|
+
| **Use Case** | Tests, demos | Desktop, CLI, local apps | Production, scale |
|
|
470
|
+
| **Job Limits** | Memory-bound | Disk-bound | Redis-bound |
|
|
471
|
+
| **Worker Mechanism** | Immediate | Embedded worker runtime | Event-driven |
|
|
472
|
+
| **Cron Jobs** | ⚠️ Basic | ✅ Supported | ✅ Supported |
|
|
473
|
+
| **Distributed Locking** | ❌ | ❌ | ✅ |
|
|
474
|
+
| **Rate Limiting** | ⚠️ Basic | ✅ Supported | ✅ Advanced |
|
|
471
475
|
|
|
472
476
|
---
|
|
473
477
|
|
|
@@ -490,7 +494,7 @@ No additional dependencies required. The adapter is built into `@igniter-js/jobs
|
|
|
490
494
|
|
|
491
495
|
```typescript
|
|
492
496
|
import { IgniterJobs } from "@igniter-js/jobs";
|
|
493
|
-
import { IgniterJobsMemoryAdapter } from "@igniter-js/jobs/adapters";
|
|
497
|
+
import { IgniterJobsMemoryAdapter } from "@igniter-js/jobs/adapters/mock";
|
|
494
498
|
|
|
495
499
|
const adapter = IgniterJobsMemoryAdapter.create();
|
|
496
500
|
|
|
@@ -553,47 +557,36 @@ await worker.close();
|
|
|
553
557
|
|
|
554
558
|
---
|
|
555
559
|
|
|
556
|
-
### 19. SQLite Adapter (Desktop, CLI,
|
|
560
|
+
### 19. Bun SQLite Adapter (Desktop, CLI, Local Apps)
|
|
557
561
|
|
|
558
|
-
The `
|
|
562
|
+
The `IgniterJobsBunSQLiteAdapter` is the local persistent adapter for Bun runtimes. It is the recommended choice for desktop applications, embedded tools, local orchestrators, and Bun services that want SQLite-backed jobs without Redis.
|
|
559
563
|
|
|
560
564
|
#### 19.1 When to Use
|
|
561
565
|
|
|
562
|
-
- **Desktop Applications:** Tauri
|
|
563
|
-
- **CLI Tools:** Long-running commands with progress tracking.
|
|
564
|
-
- **
|
|
565
|
-
- **
|
|
566
|
-
- **Local Development:** When you want jobs to survive restarts.
|
|
566
|
+
- **Desktop Applications:** Tauri or Electron apps with a Bun sidecar/runtime.
|
|
567
|
+
- **CLI Tools:** Long-running commands with progress tracking and durable jobs.
|
|
568
|
+
- **Local Services:** Single-node Bun services that need persistence and recurring jobs.
|
|
569
|
+
- **Embedded Workloads:** Sidecar workers, local automation, and job-heavy scripts.
|
|
567
570
|
|
|
568
571
|
#### 19.2 Installation
|
|
569
572
|
|
|
570
|
-
|
|
571
|
-
npm install better-sqlite3
|
|
572
|
-
# or
|
|
573
|
-
bun add better-sqlite3
|
|
574
|
-
```
|
|
575
|
-
|
|
576
|
-
The `better-sqlite3` package is an optional peer dependency.
|
|
573
|
+
No extra adapter package is required. Install `@igniter-js/jobs`, run the app in Bun, and import the adapter from `@igniter-js/jobs/adapters/bun`.
|
|
577
574
|
|
|
578
575
|
#### 19.3 Basic Usage
|
|
579
576
|
|
|
580
577
|
```typescript
|
|
581
578
|
import { IgniterJobs } from "@igniter-js/jobs";
|
|
582
|
-
import {
|
|
579
|
+
import { IgniterJobsBunSQLiteAdapter } from "@igniter-js/jobs/adapters/bun";
|
|
583
580
|
|
|
584
|
-
|
|
585
|
-
const adapter = IgniterJobsSQLiteAdapter.create({
|
|
581
|
+
const adapter = IgniterJobsBunSQLiteAdapter.create({
|
|
586
582
|
path: "./jobs.sqlite",
|
|
587
|
-
|
|
588
|
-
|
|
589
|
-
// In-memory (for tests)
|
|
590
|
-
const memoryAdapter = IgniterJobsSQLiteAdapter.create({
|
|
591
|
-
path: ":memory:",
|
|
583
|
+
durable: true,
|
|
592
584
|
});
|
|
593
585
|
|
|
594
586
|
const jobs = IgniterJobs.create()
|
|
595
587
|
.withAdapter(adapter)
|
|
596
588
|
.withService("desktop-app")
|
|
589
|
+
.withEnvironment("production")
|
|
597
590
|
.withContext(async () => ({ settings: loadSettings() }))
|
|
598
591
|
.addQueue(syncQueue)
|
|
599
592
|
.build();
|
|
@@ -602,144 +595,52 @@ const jobs = IgniterJobs.create()
|
|
|
602
595
|
#### 19.4 Configuration Options
|
|
603
596
|
|
|
604
597
|
```typescript
|
|
605
|
-
interface
|
|
606
|
-
/**
|
|
607
|
-
* Path to the SQLite database file.
|
|
608
|
-
* Use ":memory:" for in-memory database.
|
|
609
|
-
* @example "./data/jobs.sqlite"
|
|
610
|
-
*/
|
|
598
|
+
interface IgniterJobsBunSQLiteAdapterOptions {
|
|
611
599
|
path: string;
|
|
612
|
-
|
|
613
|
-
|
|
614
|
-
|
|
615
|
-
|
|
616
|
-
|
|
617
|
-
|
|
618
|
-
pollingInterval?: number;
|
|
619
|
-
|
|
620
|
-
/**
|
|
621
|
-
* Enable WAL (Write-Ahead Logging) mode for better concurrent performance.
|
|
622
|
-
* Recommended for production use.
|
|
623
|
-
* @default true
|
|
624
|
-
*/
|
|
625
|
-
enableWAL?: boolean;
|
|
600
|
+
durable?: boolean;
|
|
601
|
+
heartbeatInterval?: number;
|
|
602
|
+
pollTimeout?: number;
|
|
603
|
+
batchSize?: number;
|
|
604
|
+
lockDuration?: number;
|
|
605
|
+
maxStalledCount?: number;
|
|
626
606
|
}
|
|
627
607
|
```
|
|
628
608
|
|
|
629
|
-
#### 19.5
|
|
630
|
-
|
|
631
|
-
The adapter automatically creates and manages the following tables:
|
|
632
|
-
|
|
633
|
-
```sql
|
|
634
|
-
-- Main jobs table
|
|
635
|
-
CREATE TABLE IF NOT EXISTS jobs (
|
|
636
|
-
id TEXT PRIMARY KEY,
|
|
637
|
-
queue TEXT NOT NULL,
|
|
638
|
-
name TEXT NOT NULL,
|
|
639
|
-
input TEXT, -- JSON serialized
|
|
640
|
-
result TEXT, -- JSON serialized
|
|
641
|
-
error TEXT,
|
|
642
|
-
status TEXT NOT NULL DEFAULT 'waiting',
|
|
643
|
-
progress INTEGER DEFAULT 0,
|
|
644
|
-
priority INTEGER DEFAULT 0,
|
|
645
|
-
attempts_made INTEGER DEFAULT 0,
|
|
646
|
-
max_attempts INTEGER DEFAULT 1,
|
|
647
|
-
delay_until INTEGER, -- Unix timestamp
|
|
648
|
-
metadata TEXT, -- JSON serialized
|
|
649
|
-
scope_type TEXT,
|
|
650
|
-
scope_id TEXT,
|
|
651
|
-
created_at INTEGER NOT NULL,
|
|
652
|
-
started_at INTEGER,
|
|
653
|
-
completed_at INTEGER
|
|
654
|
-
);
|
|
655
|
-
|
|
656
|
-
-- Indexes for efficient queries
|
|
657
|
-
CREATE INDEX IF NOT EXISTS idx_jobs_queue_status ON jobs(queue, status);
|
|
658
|
-
CREATE INDEX IF NOT EXISTS idx_jobs_status ON jobs(status);
|
|
659
|
-
CREATE INDEX IF NOT EXISTS idx_jobs_priority ON jobs(priority DESC);
|
|
660
|
-
CREATE INDEX IF NOT EXISTS idx_jobs_delay ON jobs(delay_until);
|
|
661
|
-
|
|
662
|
-
-- Job logs table
|
|
663
|
-
CREATE TABLE IF NOT EXISTS job_logs (
|
|
664
|
-
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
665
|
-
job_id TEXT NOT NULL,
|
|
666
|
-
level TEXT NOT NULL,
|
|
667
|
-
message TEXT NOT NULL,
|
|
668
|
-
timestamp INTEGER NOT NULL,
|
|
669
|
-
FOREIGN KEY (job_id) REFERENCES jobs(id) ON DELETE CASCADE
|
|
670
|
-
);
|
|
671
|
-
|
|
672
|
-
-- Paused queues tracking
|
|
673
|
-
CREATE TABLE IF NOT EXISTS paused_queues (
|
|
674
|
-
name TEXT PRIMARY KEY,
|
|
675
|
-
paused_at INTEGER NOT NULL
|
|
676
|
-
);
|
|
677
|
-
```
|
|
609
|
+
#### 19.5 Runtime Characteristics
|
|
678
610
|
|
|
679
|
-
|
|
611
|
+
- Uses an embedded SQLite-backed runtime under Bun.
|
|
612
|
+
- Persists jobs to a local file path.
|
|
613
|
+
- Supports immediate dispatch, `at`, `delay`, `cron`, and `every` scheduling.
|
|
614
|
+
- Provides progress updates, logs, worker lifecycle events, and queue inspection APIs.
|
|
615
|
+
- Designed for local and embedded topologies, not distributed Redis-style clusters.
|
|
680
616
|
|
|
681
|
-
|
|
682
|
-
|
|
683
|
-
```
|
|
684
|
-
┌─────────────────────────────────────────────────────────────────┐
|
|
685
|
-
│ Worker Polling Loop │
|
|
686
|
-
├─────────────────────────────────────────────────────────────────┤
|
|
687
|
-
│ │
|
|
688
|
-
│ ┌─────────┐ ┌─────────────┐ ┌─────────────┐ │
|
|
689
|
-
│ │ Wait │────▶│ Check Jobs │────▶│ Process Job │ │
|
|
690
|
-
│ │ Polling │ │ Available? │ │ (if any) │ │
|
|
691
|
-
│ │ Interval│ └─────────────┘ └─────────────┘ │
|
|
692
|
-
│ └─────────┘ │ │ │
|
|
693
|
-
│ ▲ │ No jobs │ Done │
|
|
694
|
-
│ │ │ ▼ │
|
|
695
|
-
│ └────────────────┴───────────────────┘ │
|
|
696
|
-
│ │
|
|
697
|
-
│ Query Priority: │
|
|
698
|
-
│ 1. Delayed jobs ready to run (delay_until <= NOW) │
|
|
699
|
-
│ 2. Waiting jobs, ordered by priority DESC │
|
|
700
|
-
│ 3. Retry jobs (failed but attempts < max_attempts) │
|
|
701
|
-
│ │
|
|
702
|
-
└─────────────────────────────────────────────────────────────────┘
|
|
703
|
-
```
|
|
704
|
-
|
|
705
|
-
#### 19.7 Complete Example
|
|
617
|
+
#### 19.6 Complete Example
|
|
706
618
|
|
|
707
619
|
```typescript
|
|
708
|
-
import {
|
|
620
|
+
import { IgniterJobsBunSQLiteAdapter } from "@igniter-js/jobs/adapters/bun";
|
|
709
621
|
|
|
710
622
|
async function main() {
|
|
711
|
-
|
|
712
|
-
const adapter = IgniterJobsSQLiteAdapter.create({
|
|
623
|
+
const adapter = IgniterJobsBunSQLiteAdapter.create({
|
|
713
624
|
path: "./my-app-jobs.sqlite",
|
|
714
|
-
|
|
715
|
-
|
|
625
|
+
durable: true,
|
|
626
|
+
batchSize: 25,
|
|
716
627
|
});
|
|
717
628
|
|
|
718
|
-
// 2. Register jobs
|
|
719
629
|
adapter.registerJob("sync", "upload", {
|
|
720
|
-
handler: async ({ input }) => {
|
|
630
|
+
handler: async ({ input, job }) => {
|
|
721
631
|
const { fileId } = input as { fileId: string };
|
|
722
|
-
|
|
632
|
+
await job.updateProgress?.(25, "Uploading metadata");
|
|
723
633
|
await uploadFile(fileId);
|
|
634
|
+
await job.updateProgress?.(100, "Upload completed");
|
|
724
635
|
return { uploaded: true };
|
|
725
636
|
},
|
|
726
|
-
onProgress: async ({ progress }) => {
|
|
727
|
-
console.log(`Upload progress: ${progress}%`);
|
|
728
|
-
},
|
|
729
637
|
});
|
|
730
638
|
|
|
731
|
-
// 3. Create worker
|
|
732
639
|
const worker = await adapter.createWorker({
|
|
733
640
|
queues: ["sync"],
|
|
734
641
|
concurrency: 2,
|
|
735
|
-
handlers: {
|
|
736
|
-
onActive: ({ job }) => console.log(`Processing: ${job.id}`),
|
|
737
|
-
onSuccess: ({ job }) => console.log(`Completed: ${job.id}`),
|
|
738
|
-
onFailure: ({ job, error }) => console.error(`Failed: ${job.id}`, error),
|
|
739
|
-
},
|
|
740
642
|
});
|
|
741
643
|
|
|
742
|
-
// 4. Dispatch jobs
|
|
743
644
|
const jobId = await adapter.dispatch({
|
|
744
645
|
queue: "sync",
|
|
745
646
|
jobName: "upload",
|
|
@@ -747,50 +648,26 @@ async function main() {
|
|
|
747
648
|
priority: 10,
|
|
748
649
|
});
|
|
749
650
|
|
|
750
|
-
console.log(
|
|
751
|
-
|
|
752
|
-
// 5. Monitor status
|
|
753
|
-
const counts = await adapter.getQueueJobCounts("sync");
|
|
754
|
-
console.log(`Queue status:`, counts);
|
|
651
|
+
console.log(jobId);
|
|
755
652
|
|
|
756
|
-
// 6. Graceful shutdown
|
|
757
653
|
process.on("SIGINT", async () => {
|
|
758
654
|
await worker.close();
|
|
759
655
|
await adapter.shutdown();
|
|
760
|
-
process.exit(0);
|
|
761
656
|
});
|
|
762
657
|
}
|
|
763
|
-
|
|
764
|
-
main().catch(console.error);
|
|
765
658
|
```
|
|
766
659
|
|
|
767
|
-
#### 19.
|
|
660
|
+
#### 19.7 Persistence & Recovery
|
|
768
661
|
|
|
769
|
-
Jobs
|
|
662
|
+
- Jobs remain on disk across process restarts.
|
|
663
|
+
- Delayed and recurring work is persisted in the local SQLite backend.
|
|
664
|
+
- Worker shutdown should always call `adapter.shutdown()` during graceful exits.
|
|
770
665
|
|
|
771
|
-
|
|
772
|
-
// Session 1: Dispatch job and exit
|
|
773
|
-
const adapter = IgniterJobsSQLiteAdapter.create({ path: "./jobs.sqlite" });
|
|
774
|
-
await adapter.dispatch({
|
|
775
|
-
queue: "emails",
|
|
776
|
-
jobName: "send",
|
|
777
|
-
input: { to: "user@example.com" },
|
|
778
|
-
});
|
|
779
|
-
await adapter.shutdown();
|
|
780
|
-
// Process exits
|
|
781
|
-
|
|
782
|
-
// Session 2: Jobs are still there
|
|
783
|
-
const adapter2 = IgniterJobsSQLiteAdapter.create({ path: "./jobs.sqlite" });
|
|
784
|
-
const jobs = await adapter2.searchJobs({ queue: "emails", status: ["waiting"] });
|
|
785
|
-
console.log(jobs); // Shows the pending job from session 1
|
|
786
|
-
```
|
|
787
|
-
|
|
788
|
-
#### 19.9 Limitations
|
|
666
|
+
#### 19.8 Limitations
|
|
789
667
|
|
|
790
|
-
- **
|
|
791
|
-
- **
|
|
792
|
-
- **
|
|
793
|
-
- **Limited Cron:** Basic cron support without advanced scheduling features.
|
|
668
|
+
- **Local Topology:** The adapter is intended for Bun-local deployments and desktop-class runtimes.
|
|
669
|
+
- **No Redis-style Distribution:** Use BullMQ when you need multi-node horizontal workers.
|
|
670
|
+
- **Runtime Requirement:** This adapter runs only under Bun.
|
|
794
671
|
|
|
795
672
|
---
|
|
796
673
|
|
|
@@ -863,8 +740,9 @@ See `src/types/adapter.ts` for the complete interface definition.
|
|
|
863
740
|
#### 21.2 Testing Custom Adapters
|
|
864
741
|
|
|
865
742
|
Use the existing test suites as a reference:
|
|
743
|
+
|
|
866
744
|
- `src/adapters/memory.adapter.spec.ts`
|
|
867
|
-
- `src/
|
|
745
|
+
- `src/core/telemetry.spec.ts`
|
|
868
746
|
|
|
869
747
|
---
|
|
870
748
|
|
|
@@ -877,18 +755,19 @@ The `examples/` directory contains runnable examples:
|
|
|
877
755
|
```
|
|
878
756
|
packages/jobs/examples/
|
|
879
757
|
├── README.md # Example documentation
|
|
880
|
-
└── sqlite-example.ts
|
|
758
|
+
└── bun-sqlite-example.ts # Complete Bun SQLite adapter demo
|
|
881
759
|
```
|
|
882
760
|
|
|
883
|
-
### 23. Running the SQLite Example
|
|
761
|
+
### 23. Running the Bun SQLite Example
|
|
884
762
|
|
|
885
763
|
```bash
|
|
886
764
|
cd packages/jobs
|
|
887
|
-
|
|
765
|
+
bun run examples/bun-sqlite-example.ts
|
|
888
766
|
```
|
|
889
767
|
|
|
890
768
|
This example demonstrates:
|
|
891
|
-
|
|
769
|
+
|
|
770
|
+
- Creating a Bun SQLite adapter with file persistence
|
|
892
771
|
- Registering multiple job types
|
|
893
772
|
- Priority-based job processing
|
|
894
773
|
- Delayed job scheduling
|
|
@@ -900,32 +779,32 @@ This example demonstrates:
|
|
|
900
779
|
|
|
901
780
|
## VI. MIGRATION GUIDE
|
|
902
781
|
|
|
903
|
-
### 24. Migrating from Memory to SQLite
|
|
782
|
+
### 24. Migrating from Memory to Bun SQLite
|
|
904
783
|
|
|
905
784
|
For development to production-like local testing:
|
|
906
785
|
|
|
907
786
|
```typescript
|
|
908
787
|
// Before: Memory adapter
|
|
909
|
-
import { IgniterJobsMemoryAdapter } from "@igniter-js/jobs/adapters";
|
|
788
|
+
import { IgniterJobsMemoryAdapter } from "@igniter-js/jobs/adapters/mock";
|
|
910
789
|
const adapter = IgniterJobsMemoryAdapter.create();
|
|
911
790
|
|
|
912
|
-
// After: SQLite adapter
|
|
913
|
-
import {
|
|
914
|
-
const adapter =
|
|
791
|
+
// After: Bun SQLite adapter
|
|
792
|
+
import { IgniterJobsBunSQLiteAdapter } from "@igniter-js/jobs/adapters/bun";
|
|
793
|
+
const adapter = IgniterJobsBunSQLiteAdapter.create({
|
|
915
794
|
path: "./dev-jobs.sqlite",
|
|
916
795
|
});
|
|
917
796
|
|
|
918
797
|
// Rest of the code remains identical!
|
|
919
798
|
```
|
|
920
799
|
|
|
921
|
-
### 25. Migrating from SQLite to BullMQ
|
|
800
|
+
### 25. Migrating from Bun SQLite to BullMQ
|
|
922
801
|
|
|
923
802
|
For local development to production:
|
|
924
803
|
|
|
925
804
|
```typescript
|
|
926
|
-
// Before: SQLite adapter
|
|
927
|
-
import {
|
|
928
|
-
const adapter =
|
|
805
|
+
// Before: Bun SQLite adapter
|
|
806
|
+
import { IgniterJobsBunSQLiteAdapter } from "@igniter-js/jobs/adapters/bun";
|
|
807
|
+
const adapter = IgniterJobsBunSQLiteAdapter.create({
|
|
929
808
|
path: "./jobs.sqlite",
|
|
930
809
|
});
|
|
931
810
|
|
|
@@ -944,11 +823,9 @@ const adapter = IgniterJobsBullMQAdapter.create({
|
|
|
944
823
|
|
|
945
824
|
### Version 0.2.0 (2026-01-17)
|
|
946
825
|
|
|
947
|
-
- **Added:** `
|
|
948
|
-
- **Added:** SQLite adapter comprehensive test suite (71 tests)
|
|
826
|
+
- **Added:** `IgniterJobsBunSQLiteAdapter` for persistent local Bun job queues
|
|
949
827
|
- **Added:** `examples/` directory with runnable examples
|
|
950
|
-
- **
|
|
951
|
-
- **Updated:** Adapter exports to include SQLite adapter
|
|
828
|
+
- **Updated:** Adapter exports to include Bun SQLite adapter
|
|
952
829
|
- **Updated:** AGENTS.md with comprehensive adapter documentation
|
|
953
830
|
|
|
954
831
|
### Version 0.1.2 (2025-12-23)
|
|
@@ -965,16 +842,16 @@ const adapter = IgniterJobsBullMQAdapter.create({
|
|
|
965
842
|
|
|
966
843
|
#### Q: Which adapter should I choose?
|
|
967
844
|
|
|
968
|
-
| Environment
|
|
969
|
-
|
|
970
|
-
| Unit/Integration Tests
|
|
971
|
-
| Desktop App (Tauri/Electron) | SQLite Adapter
|
|
972
|
-
| CLI Tool
|
|
973
|
-
| MCP Server
|
|
974
|
-
| Local Development
|
|
975
|
-
| Production (Single Node)
|
|
976
|
-
| Production (Multi Node)
|
|
977
|
-
| Serverless
|
|
845
|
+
| Environment | Recommended Adapter |
|
|
846
|
+
| ---------------------------- | -------------------- |
|
|
847
|
+
| Unit/Integration Tests | Memory Adapter |
|
|
848
|
+
| Desktop App (Tauri/Electron) | Bun SQLite Adapter |
|
|
849
|
+
| CLI Tool | Bun SQLite Adapter |
|
|
850
|
+
| MCP Server | Bun SQLite Adapter |
|
|
851
|
+
| Local Development | Bun SQLite Adapter |
|
|
852
|
+
| Production (Single Node) | Bun SQLite or BullMQ |
|
|
853
|
+
| Production (Multi Node) | BullMQ Adapter |
|
|
854
|
+
| Serverless | BullMQ Adapter |
|
|
978
855
|
|
|
979
856
|
#### Q: Can I switch adapters without changing business logic?
|
|
980
857
|
|
|
@@ -1003,14 +880,14 @@ adapter.registerJob("email", "send", {
|
|
|
1003
880
|
adapter.registerJob("import", "csv", {
|
|
1004
881
|
handler: async ({ input, job }) => {
|
|
1005
882
|
const rows = await parseCSV(input.file);
|
|
1006
|
-
|
|
883
|
+
|
|
1007
884
|
for (let i = 0; i < rows.length; i++) {
|
|
1008
885
|
await processRow(rows[i]);
|
|
1009
|
-
|
|
886
|
+
|
|
1010
887
|
// Update progress
|
|
1011
|
-
await
|
|
888
|
+
await job.updateProgress?.(Math.floor((i / rows.length) * 100));
|
|
1012
889
|
}
|
|
1013
|
-
|
|
890
|
+
|
|
1014
891
|
return { processed: rows.length };
|
|
1015
892
|
},
|
|
1016
893
|
onProgress: async ({ progress, message }) => {
|
|
@@ -1019,28 +896,24 @@ adapter.registerJob("import", "csv", {
|
|
|
1019
896
|
});
|
|
1020
897
|
```
|
|
1021
898
|
|
|
1022
|
-
#### Q: Can SQLite handle high throughput?
|
|
899
|
+
#### Q: Can Bun SQLite handle high throughput?
|
|
1023
900
|
|
|
1024
|
-
SQLite
|
|
901
|
+
The Bun SQLite adapter handles local persistent workloads very well, especially for desktop, CLI, and embedded apps. For distributed workers or multi-node scale, use BullMQ.
|
|
1025
902
|
|
|
1026
903
|
### 27. Common Issues & Solutions
|
|
1027
904
|
|
|
1028
|
-
#### Issue:
|
|
905
|
+
#### Issue: Bun SQLite adapter throws a runtime error in Node.js
|
|
1029
906
|
|
|
1030
|
-
**Cause:** The
|
|
907
|
+
**Cause:** The Bun SQLite adapter only runs inside Bun.
|
|
1031
908
|
|
|
1032
|
-
**Solution:**
|
|
1033
|
-
```bash
|
|
1034
|
-
npm install better-sqlite3
|
|
1035
|
-
# If compilation fails on macOS:
|
|
1036
|
-
npm install better-sqlite3 --build-from-source
|
|
1037
|
-
```
|
|
909
|
+
**Solution:** Use the BullMQ adapter in Node.js runtimes, or run the local adapter from Bun.
|
|
1038
910
|
|
|
1039
911
|
#### Issue: Jobs stuck in "active" status after restart
|
|
1040
912
|
|
|
1041
913
|
**Cause:** Process crashed while job was processing.
|
|
1042
914
|
|
|
1043
915
|
**Solution:**
|
|
916
|
+
|
|
1044
917
|
```typescript
|
|
1045
918
|
// On startup, reset stale active jobs
|
|
1046
919
|
const staleJobs = await adapter.searchJobs({
|
|
@@ -1058,6 +931,7 @@ for (const job of staleJobs) {
|
|
|
1058
931
|
**Cause:** Worker queues don't match job queue names.
|
|
1059
932
|
|
|
1060
933
|
**Solution:**
|
|
934
|
+
|
|
1061
935
|
```typescript
|
|
1062
936
|
// Ensure queue names match
|
|
1063
937
|
adapter.registerJob("email", "send", { ... }); // Queue: "email"
|
|
@@ -1069,40 +943,43 @@ const worker = await adapter.createWorker({
|
|
|
1069
943
|
});
|
|
1070
944
|
```
|
|
1071
945
|
|
|
1072
|
-
#### Issue: SQLite database locked
|
|
946
|
+
#### Issue: Bun SQLite database locked
|
|
1073
947
|
|
|
1074
948
|
**Cause:** Multiple processes trying to write simultaneously.
|
|
1075
949
|
|
|
1076
950
|
**Solution:**
|
|
1077
|
-
|
|
1078
|
-
- Ensure
|
|
1079
|
-
-
|
|
951
|
+
|
|
952
|
+
- Ensure the local runtime is the primary writer to the database path.
|
|
953
|
+
- Avoid multiple unrelated Bun processes sharing the same local queue file.
|
|
954
|
+
- Consider BullMQ for multi-process or multi-node scenarios.
|
|
1080
955
|
|
|
1081
956
|
---
|
|
1082
957
|
|
|
1083
958
|
## IX. PERFORMANCE CONSIDERATIONS
|
|
1084
959
|
|
|
1085
|
-
### 28. SQLite Adapter Performance Tips
|
|
960
|
+
### 28. Bun SQLite Adapter Performance Tips
|
|
1086
961
|
|
|
1087
|
-
1. **Use
|
|
962
|
+
1. **Use durable mode only for critical jobs:** durable writes trade throughput for stronger persistence.
|
|
1088
963
|
|
|
1089
|
-
2. **Tune
|
|
1090
|
-
- Lower interval (100ms) = more responsive, higher CPU
|
|
1091
|
-
- Higher interval (1000ms) = less responsive, lower CPU
|
|
964
|
+
2. **Tune worker batch settings:** `batchSize`, `pollTimeout`, and `heartbeatInterval` affect local throughput.
|
|
1092
965
|
|
|
1093
966
|
3. **Batch Operations:** When dispatching many jobs, consider batching:
|
|
967
|
+
|
|
1094
968
|
```typescript
|
|
1095
969
|
const jobIds = [];
|
|
1096
970
|
for (const item of items) {
|
|
1097
|
-
jobIds.push(
|
|
1098
|
-
|
|
1099
|
-
|
|
1100
|
-
|
|
1101
|
-
|
|
971
|
+
jobIds.push(
|
|
972
|
+
await adapter.dispatch({
|
|
973
|
+
queue: "process",
|
|
974
|
+
jobName: "item",
|
|
975
|
+
input: item,
|
|
976
|
+
}),
|
|
977
|
+
);
|
|
1102
978
|
}
|
|
1103
979
|
```
|
|
1104
980
|
|
|
1105
981
|
4. **Clean Old Jobs:** Periodically clean completed/failed jobs:
|
|
982
|
+
|
|
1106
983
|
```typescript
|
|
1107
984
|
await adapter.cleanQueue("email", {
|
|
1108
985
|
status: ["completed", "failed"],
|
|
@@ -1110,12 +987,12 @@ await adapter.cleanQueue("email", {
|
|
|
1110
987
|
});
|
|
1111
988
|
```
|
|
1112
989
|
|
|
1113
|
-
5. **
|
|
990
|
+
5. **Split high-volume workloads into dedicated queues:** local SQLite workloads benefit from clear queue boundaries.
|
|
1114
991
|
|
|
1115
992
|
### 29. Memory Considerations
|
|
1116
993
|
|
|
1117
994
|
- **Memory Adapter:** All jobs stored in RAM; limit maxJobHistory.
|
|
1118
|
-
- **SQLite Adapter:** Disk-based; memory usage scales with active
|
|
995
|
+
- **Bun SQLite Adapter:** Disk-based; memory usage scales with active workers and buffered operations.
|
|
1119
996
|
- **BullMQ Adapter:** Redis memory; configure maxmemory policy.
|
|
1120
997
|
|
|
1121
998
|
---
|
|
@@ -1150,4 +1027,3 @@ const emailQueue = IgniterQueue.create("email")
|
|
|
1150
1027
|
- **Encrypt at rest** if storing sensitive metadata.
|
|
1151
1028
|
|
|
1152
1029
|
---
|
|
1153
|
-
|