@basaltkit/queue 1.5.0 → 2.0.0

package/README.md

@@ -20,15 +20,20 @@ This module gives you three things you'd normally have to build by hand:
 
 1. **Declarative, type-safe jobs** — you define each job once with `defineJob` (name, validation schema, number of attempts) and then call `MyJob.dispatch(data)` anywhere in the application. Data is validated with Zod *before* entering the queue, so invalid data never reaches the worker.
 2. **Context propagation** — information from the current request (`requestId`, `tenantId`, `userId`, etc.) automatically travels along with the job and is restored inside the worker. Your logs and tenant checks work in the worker the same way they did in the HTTP request.
-3. **Two interchangeable drivers** — in production, use **BullMQ** (over Redis, with real retries and delays); in development and tests, use the **sync** driver, which runs the job immediately, in the same process, without needing Redis installed.
+3. **Interchangeable drivers** — in production, use **BullMQ** (over Redis, with real retries and delays), or RabbitMQ/SQS/Kafka through a driver package; in development and tests, use the **sync** driver, which runs the job immediately, in the same process, without needing Redis installed. The core itself is backend-neutral: it depends on no broker client.
 
 ## Installation
 
 ```bash
 pnpm add @basaltkit/queue
+
+# only if you use the BullMQ/Redis driver (`connection` or `BullmqQueueDriver`):
+pnpm add bullmq
 ```
 
-The package depends on `@basaltkit/core` and `@basaltkit/events` (installed automatically). For production you also need an accessible **Redis** server (BullMQ stores the queues there). For development and tests you need nothing.
+The package depends on `@basaltkit/core` and `@basaltkit/events` (installed automatically). For development and tests you need nothing else — the sync driver has no dependencies.
+
+`bullmq` is an **optional peer dependency**, not a dependency: `@basaltkit/queue` is the driver-agnostic core, so an application running on [RabbitMQ](https://www.npmjs.com/package/@basaltkit/queue-rabbitmq), [SQS](https://www.npmjs.com/package/@basaltkit/queue-sqs), [Kafka](https://www.npmjs.com/package/@basaltkit/queue-kafka) or the sync driver never installs (or loads) BullMQ and its ioredis transitive weight. Install `bullmq` yourself the moment you pass `connection` or construct `BullmqQueueDriver`; forget it and boot fails with a `MissingQueueDriverPackageError` telling you exactly that. For production with BullMQ you also need an accessible **Redis** server (it stores the queues there).
 
 ## Get started in 5 minutes
 
@@ -399,7 +404,7 @@ Creates the event→job bridge. Returns the subscription cancel function.
 ### Drivers
 
 - **`class SyncQueueDriver`** — runs inline on `dispatch`, honors `attempts` (immediate retry). Public property `executed: { queue, jobName, attempts }[]` with the execution history (capped at 1000 entries). For testing and dev without Redis.
-- **`class BullmqQueueDriver`** — production over Redis; see the options table below.
+- **`class BullmqQueueDriver`** — production over Redis; see the options table below. Imported from its own entry point, `@basaltkit/queue/bullmq` (one import path per backend, like the RabbitMQ/SQS/Kafka driver packages), and requires the optional `bullmq` peer.
 - **`interface QueueDriver`** (Advanced) — contract for custom drivers: `setExecutor(executor)`, `add(queue, jobName, data, options: AddJobOptions)`, `startWorker(queue, { concurrency? })`, optional `stats(queue)` / `retryFailed(queue, { limit? })` / `list(queue, options)`, `close()`, plus the optional `name` and `capabilities` fields. Helper types: `AddJobOptions`, `JobExecutor`, `QueueStats`, `DriverCapabilities`, `JobState`, `JobSummary`, `JobEnvelope`, `ListJobsOptions`.
 
   The three optional methods are a deliberate pattern: a driver **omits** what its
@@ -417,6 +422,12 @@ Creates the event→job bridge. Returns the subscription cancel function.
 
 #### `new BullmqQueueDriver(options: BullmqDriverOptions)`
 
+```ts
+import { BullmqQueueDriver } from '@basaltkit/queue/bullmq'   // needs `pnpm add bullmq`
+```
+
+(The `BullmqDriverOptions` **type** is also re-exported from `@basaltkit/queue` — types are erased at build, so they cost nothing.)
+
 | Option | Type | Default | Purpose |
 |---|---|---|---|
 | `connection` | `string \| ConnectionOptions` | — (required) | Redis URL (`redis://…`, `rediss://…` → TLS) or ioredis options. |
@@ -477,7 +488,8 @@ Passing your own `driver` bypasses that forwarding — a supplied driver owns it
 so set them in its constructor:
 
 ```ts
-import { queuePlugin, BullmqQueueDriver } from '@basaltkit/queue'
+import { queuePlugin } from '@basaltkit/queue'
+import { BullmqQueueDriver } from '@basaltkit/queue/bullmq'
 
 queuePlugin({
   jobs: [SendWelcomeEmail],
@@ -496,6 +508,7 @@ queuePlugin({
 | `JobNotRegisteredError` | `QUEUE_JOB_NOT_REGISTERED` | `job.dispatch()` was called before the job was registered in a `QueueManager`. |
 | `UnknownJobError` | `QUEUE_UNKNOWN_JOB` | A job reached the worker but is not registered in that process — producer and worker registered different job lists. |
 | `UnsupportedJobOptionError` | `QUEUE_UNSUPPORTED_OPTION` | With `onUnsupported: 'throw'`, a dispatch used an option the active driver's `capabilities` do not include. `status = 500`. |
+| `MissingQueueDriverPackageError` | `QUEUE_MISSING_DRIVER_PACKAGE` | `queuePlugin({ connection })` selected the BullMQ driver but the optional `bullmq` peer is not installed. Thrown at boot, with the fix in the message; `.cause` holds the original resolution error. |
 
 Errors thrown outside these classes come from the driver's client (ioredis, amqplib, kafkajs,
 the AWS SDK) and reach you through that driver's `onError`.

package/dist/index.d.ts

@@ -1,14 +1,24 @@
+import { BasaltError } from '@basaltkit/core';
 import { type QueueDriver } from './driver.js';
-import { type BullmqDriverOptions } from './drivers/bullmq.js';
+import type { BullmqDriverOptions } from './drivers/bullmq.js';
 import type { JobDefinition, JobRetention } from './job.js';
 import { QueueManager, type UnsupportedPolicy } from './manager.js';
 export { defineJob, JobValidationError, JobNotRegisteredError, type JobDefinition, type JobSchema, type JobBackoff, type JobRetention, type DispatchOptions, } from './job.js';
 export { QueueManager, UnknownJobError, UnsupportedJobOptionError, type UnsupportedPolicy, type QueueManagerOptions, } from './manager.js';
 export { queuedOn, type QueuedListenerOptions } from './bridge.js';
 export { SyncQueueDriver } from './drivers/sync.js';
-export { BullmqQueueDriver, type BullmqDriverOptions } from './drivers/bullmq.js';
+/**
+ * The BullMQ driver's *types* stay on the barrel (erased at build, so they cost
+ * a consumer nothing). The CLASS lives at its own entry point:
+ * `import { BullmqQueueDriver } from '@basaltkit/queue/bullmq'` — the same
+ * shape as the RabbitMQ/SQS/Kafka driver packages, one import path per backend.
+ */
+export type { BullmqDriverOptions } from './drivers/bullmq.js';
 export { readJobEnvelope, DEFAULT_LIST_STATES, DEFAULT_LIST_LIMIT, MAX_LIST_LIMIT, type QueueDriver, type QueueStats, type AddJobOptions, type JobExecutor, type DriverCapabilities, type JobState, type JobSummary, type JobEnvelope, type ListJobsOptions, } from './driver.js';
 export declare const QUEUE: import("@basaltkit/core").Token<QueueManager>;
+export declare class MissingQueueDriverPackageError extends BasaltError {
+    constructor(options?: ErrorOptions);
+}
 export interface QueuePluginOptions {
     /** Jobs known to this process (producer and/or worker). */
     jobs?: JobDefinition<unknown>[];

package/dist/index.js

@@ -1,25 +1,59 @@
-import { createToken, definePlugin, ensureMetadata } from '@basaltkit/core';
+import { BasaltError, createToken, definePlugin, ensureMetadata } from '@basaltkit/core';
 import { DEFAULT_LIST_LIMIT, DEFAULT_LIST_STATES, } from './driver.js';
-import { BullmqQueueDriver } from './drivers/bullmq.js';
 import { SyncQueueDriver } from './drivers/sync.js';
 import { QueueManager } from './manager.js';
 export { defineJob, JobValidationError, JobNotRegisteredError, } from './job.js';
 export { QueueManager, UnknownJobError, UnsupportedJobOptionError, } from './manager.js';
 export { queuedOn } from './bridge.js';
 export { SyncQueueDriver } from './drivers/sync.js';
-export { BullmqQueueDriver } from './drivers/bullmq.js';
 export { readJobEnvelope, DEFAULT_LIST_STATES, DEFAULT_LIST_LIMIT, MAX_LIST_LIMIT, } from './driver.js';
 export const QUEUE = createToken('queue');
+let bullmqDriverModule;
+/** Actionable guidance instead of a bare ERR_MODULE_NOT_FOUND from deep inside the driver. */
+const MISSING_BULLMQ = '`queuePlugin({ connection })` selects the BullMQ driver, which needs the `bullmq` package — ' +
+    'an optional peer dependency of @basaltkit/queue that is not installed. ' +
+    'Either install it (`pnpm add bullmq`), or pass an explicit `driver:` — ' +
+    '`@basaltkit/queue-rabbitmq`, `@basaltkit/queue-sqs`, `@basaltkit/queue-kafka`, ' +
+    'or `new SyncQueueDriver()` for dev/tests.';
+export class MissingQueueDriverPackageError extends BasaltError {
+    constructor(options) {
+        super('QUEUE_MISSING_DRIVER_PACKAGE', MISSING_BULLMQ, options);
+    }
+}
+async function loadBullmqDriver() {
+    if (bullmqDriverModule)
+        return bullmqDriverModule;
+    try {
+        bullmqDriverModule = await import('./drivers/bullmq.js');
+    }
+    catch (error) {
+        throw new MissingQueueDriverPackageError({ cause: error });
+    }
+    return bullmqDriverModule;
+}
 export function queuePlugin(options = {}) {
     return definePlugin({
         name: 'basalt:queue',
-        register({ container }) {
+        async register({ container }) {
+            // Resolve the BullMQ driver MODULE here — `BasaltApp.boot()` awaits
+            // `register`, so the class is in hand before anything can resolve QUEUE,
+            // and the container factory below stays synchronous. This is still a
+            // bindings-only phase: loading a module is not I/O against the world —
+            // no Redis connection is opened until the singleton is first resolved.
+            // Doing it in `boot` instead would leave a hole: a plugin booting earlier
+            // that resolves QUEUE would hit an unloaded driver.
+            if (!options.driver && options.connection)
+                await loadBullmqDriver();
             registerQueueCommands(container);
             container.singleton(QUEUE, () => {
                 let driver = options.driver;
                 if (!driver) {
                     if (options.connection) {
-                        driver = new BullmqQueueDriver({
+                        // Defensive: only reachable if `register`'s promise was dropped by
+                        // a non-standard host instead of awaited.
+                        if (!bullmqDriverModule)
+                            throw new MissingQueueDriverPackageError();
+                        driver = new bullmqDriverModule.BullmqQueueDriver({
                             connection: options.connection,
                             ...(options.onError !== undefined ? { onError: options.onError } : {}),
                             ...(options.onJobFailed !== undefined ? { onJobFailed: options.onJobFailed } : {}),

package/package.json

@@ -1,10 +1,10 @@
 {
   "name": "@basaltkit/queue",
-  "version": "1.5.0",
+  "version": "2.0.0",
   "engines": {
     "node": ">=22.5.0"
   },
-  "description": "Basalt queues on top of BullMQ: declarative jobs with Zod payloads, context propagation (tenant/requestId) to workers and a sync driver for tests.",
+  "description": "Driver-agnostic Basalt queues: declarative jobs with Zod payloads, context propagation (tenant/requestId) to workers, a BullMQ/Redis driver behind an optional peer, and a sync driver for tests.",
   "license": "MIT",
   "type": "module",
   "sideEffects": false,
@@ -12,18 +12,30 @@
     ".": {
       "types": "./dist/index.d.ts",
       "import": "./dist/index.js"
+    },
+    "./bullmq": {
+      "types": "./dist/drivers/bullmq.d.ts",
+      "import": "./dist/drivers/bullmq.js"
     }
   },
   "files": [
     "dist"
   ],
   "dependencies": {
-    "bullmq": "^6.2.1",
-    "@basaltkit/core": "^1.3.1",
-    "@basaltkit/events": "^1.1.1"
+    "@basaltkit/events": "^1.1.1",
+    "@basaltkit/core": "^1.3.1"
+  },
+  "peerDependencies": {
+    "bullmq": "^6.2.1"
+  },
+  "peerDependenciesMeta": {
+    "bullmq": {
+      "optional": true
+    }
   },
   "devDependencies": {
     "@types/node": "^26.3.0",
+    "bullmq": "^6.2.1",
     "typescript": "^7.0.2",
     "vitest": "^4.1.11",
     "zod": "^3.24.0 || ^4.0.0",