@basaltkit/queue 1.4.1 → 2.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -20,15 +20,20 @@ This module gives you three things you'd normally have to build by hand:
20
20
 
21
21
  1. **Declarative, type-safe jobs** — you define each job once with `defineJob` (name, validation schema, number of attempts) and then call `MyJob.dispatch(data)` anywhere in the application. Data is validated with Zod *before* entering the queue, so invalid data never reaches the worker.
22
22
  2. **Context propagation** — information from the current request (`requestId`, `tenantId`, `userId`, etc.) automatically travels along with the job and is restored inside the worker. Your logs and tenant checks work in the worker the same way they did in the HTTP request.
23
- 3. **Two interchangeable drivers** — in production, use **BullMQ** (over Redis, with real retries and delays); in development and tests, use the **sync** driver, which runs the job immediately, in the same process, without needing Redis installed.
23
+ 3. **Interchangeable drivers** — in production, use **BullMQ** (over Redis, with real retries and delays), or RabbitMQ/SQS/Kafka through a driver package; in development and tests, use the **sync** driver, which runs the job immediately, in the same process, without needing Redis installed. The core itself is backend-neutral: it depends on no broker client.
24
24
 
25
25
  ## Installation
26
26
 
27
27
  ```bash
28
28
  pnpm add @basaltkit/queue
29
+
30
+ # only if you use the BullMQ/Redis driver (`connection` or `BullmqQueueDriver`):
31
+ pnpm add bullmq
29
32
  ```
30
33
 
31
- The package depends on `@basaltkit/core` and `@basaltkit/events` (installed automatically). For production you also need an accessible **Redis** server (BullMQ stores the queues there). For development and tests you need nothing.
34
+ The package depends on `@basaltkit/core` and `@basaltkit/events` (installed automatically). For development and tests you need nothing else the sync driver has no dependencies.
35
+
36
+ `bullmq` is an **optional peer dependency**, not a dependency: `@basaltkit/queue` is the driver-agnostic core, so an application running on [RabbitMQ](https://www.npmjs.com/package/@basaltkit/queue-rabbitmq), [SQS](https://www.npmjs.com/package/@basaltkit/queue-sqs), [Kafka](https://www.npmjs.com/package/@basaltkit/queue-kafka) or the sync driver never installs (or loads) BullMQ and its ioredis transitive weight. Install `bullmq` yourself the moment you pass `connection` or construct `BullmqQueueDriver`; forget it and boot fails with a `MissingQueueDriverPackageError` telling you exactly that. For production with BullMQ you also need an accessible **Redis** server (it stores the queues there).
32
37
 
33
38
  ## Get started in 5 minutes
34
39
 
@@ -195,17 +200,35 @@ If a job reaches a worker that hasn't registered it, `UnknownJobError` is thrown
195
200
 
196
201
  ### CLI commands (`basalt queue:*`)
197
202
 
198
- Registering `queuePlugin` also wires three CLI commands (run via the `@basaltkit/cli` runner):
203
+ Registering `queuePlugin` also wires four CLI commands (run via the `@basaltkit/cli` runner):
199
204
 
200
205
  ```bash
201
206
  basalt queue:work --queue=default --concurrency=5 # run a worker until Ctrl+C
202
207
  basalt queue:stats --queue=billing # waiting/active/completed/failed/delayed
203
208
  basalt queue:retry --queue=billing --limit=100 # re-enqueue failed jobs
209
+ basalt queue:jobs --queue=billing --states=failed # list individual jobs
210
+ ```
211
+
212
+ `queue:jobs` flags:
213
+
214
+ | Flag | Default | Description |
215
+ |---|---|---|
216
+ | `--queue` | `default` | Queue to inspect. |
217
+ | `--states` | `completed,failed,waiting,active` | Comma-separated states from `waiting`, `active`, `completed`, `failed`, `delayed`. An unknown state is rejected with the valid list, never silently dropped. |
218
+ | `--limit` | `20` | Maximum rows in **total** (newest first), capped at `1000`. |
219
+ | `--payload` | off | Also print each job's payload (truncated). **Off by default: payloads can contain personal data.** |
220
+
221
+ ```
222
+ id name state attempts age
223
+ 1042 email.welcome completed 1 3s
224
+ 1041 email.welcome failed 3 2m
225
+ 2 job(s) on "billing" (completed, failed, waiting, active, limit 20). Payloads hidden — add --payload to include them.
204
226
  ```
205
227
 
206
- `queue:stats` and `queue:retry` need a driver that can introspect job state — the
207
- **BullMQ** driver (a Redis `connection`). With the inline `sync` driver they
208
- report the operation as unsupported (it keeps no job state), rather than guessing.
228
+ `queue:stats`, `queue:retry` and `queue:jobs` need a driver that can introspect job
229
+ state — the **BullMQ** driver (a Redis `connection`). With the inline `sync` driver,
230
+ or a broker driver that cannot read a job without consuming it (RabbitMQ, SQS, Kafka),
231
+ they report the operation as unsupported rather than guessing.
209
232
 
210
233
  ### Manual use without a plugin (e.g. in tests)
211
234
 
@@ -300,6 +323,7 @@ const manager = app.container.get(QUEUE) // get the QueueManager from the contai
300
323
  | `work` | `(queue = 'default', { concurrency? }?) => void` | Starts a worker for the queue (no-op on the sync driver). |
301
324
  | `stats` | `(queue = 'default') => Promise<QueueStats \| undefined>` | Job counts per state, or `undefined` when the driver can't introspect (sync). |
302
325
  | `retryFailed` | `(queue = 'default', { limit? }?) => Promise<number \| undefined>` | Re-enqueues failed jobs; returns the count, or `undefined` when the driver doesn't support it. |
326
+ | `list` | `(queue = 'default', options?: ListJobsOptions) => Promise<JobSummary[] \| undefined>` | Lists individual jobs, newest first, payload already unwrapped from the dispatch envelope. `undefined` when the driver can't list. |
303
327
  | `close` | `() => Promise<void>` | Closes workers and connections. |
304
328
 
305
329
  `QueueManagerOptions`:
@@ -311,6 +335,60 @@ const manager = app.container.get(QUEUE) // get the QueueManager from the contai
311
335
  | `removeOnComplete` | `JobRetention` | driver default | Default retention for completed jobs. |
312
336
  | `removeOnFail` | `JobRetention` | driver default | Default retention for failed jobs. |
313
337
 
338
+ ### Listing jobs — `manager.list(queue?, options?)`
339
+
340
+ The supported way to see *which* jobs are on a queue, without reaching into the
341
+ broker's own client (which re-couples your app to one backend):
342
+
343
+ ```ts
344
+ import { QUEUE } from '@basaltkit/queue'
345
+
346
+ const jobs = await app.container.get(QUEUE).list('billing', { states: ['failed'], limit: 10 })
347
+ if (!jobs) {
348
+ // the active driver cannot list — handle it, don't assume an empty queue
349
+ } else {
350
+ for (const job of jobs) console.log(job.id, job.name, job.state, job.payload)
351
+ }
352
+ ```
353
+
354
+ `ListJobsOptions`:
355
+
356
+ | Field | Type | Default | Description |
357
+ |---|---|---|---|
358
+ | `states` | `JobState[]` | `['completed', 'failed', 'waiting', 'active']` | Which states to look in. `JobState` = `'waiting' \| 'active' \| 'completed' \| 'failed' \| 'delayed'` — the same vocabulary as `QueueStats`. |
359
+ | `limit` | `number` | `20` (max `1000`) | Maximum jobs returned in **total**, newest first — not per state. Each state is read up to `limit` and the merged result is truncated, so one busy state can't starve the others. |
360
+
361
+ **Why `completed` and `failed` are in the default.** A worker drains `waiting` in
362
+ milliseconds, so a healthy queue shows `waiting: 0, active: 0` almost always.
363
+ Defaulting to only those would answer "did my job run?" with an empty list on a
364
+ queue that is working perfectly. `delayed` is *not* in the default — it is a
365
+ separate question; ask for it explicitly.
366
+
367
+ `JobSummary` (driver-neutral — never the backend's own job object):
368
+
369
+ | Field | Type | Description |
370
+ |---|---|---|
371
+ | `id` | `string` | Backend-assigned job id. |
372
+ | `name` | `string` | The job name from `defineJob({ name })`. |
373
+ | `state` | `JobState` | Where the job was found. |
374
+ | `attemptsMade` | `number` | Attempts so far (`0` before the first run). |
375
+ | `timestamp` | `number` | Creation time, epoch ms. |
376
+ | `payload` | `unknown` | **Your** payload — already unwrapped from the `{ payload, context }` dispatch envelope. |
377
+ | `context` | `RequestContext \| undefined` | The request context captured at dispatch (`requestId`, `tenantId`, …), when there was one. |
378
+ | `failedReason` | `string \| undefined` | Why it failed — only for `state: 'failed'`. |
379
+
380
+ **A `JobSummary` carries the job's payload, so it can carry personal data.** Treat
381
+ the result like the records it came from: never log it wholesale, and if you expose
382
+ it over HTTP, require authentication (`meta: { auth: true }`), authorize it per
383
+ tenant, and put raw payloads behind an explicit opt-in. `basalt queue:jobs` follows
384
+ the same rule — it hides payloads unless you pass `--payload`.
385
+
386
+ `readJobEnvelope(data)` is exported for custom drivers implementing `list`: it
387
+ opens the `{ payload, context }` envelope defensively, treating anything that
388
+ isn't one (an older producer, a hand-written job) as the payload itself.
389
+ `DEFAULT_LIST_STATES`, `DEFAULT_LIST_LIMIT` and `MAX_LIST_LIMIT` are exported so
390
+ a custom driver defaults the same way BullMQ does.
391
+
314
392
  ### `queuedOn<T>(bus, manager, event, handler, options?): () => void`
315
393
 
316
394
  Creates the event→job bridge. Returns the subscription cancel function.
@@ -326,11 +404,30 @@ Creates the event→job bridge. Returns the subscription cancel function.
326
404
  ### Drivers
327
405
 
328
406
  - **`class SyncQueueDriver`** — runs inline on `dispatch`, honors `attempts` (immediate retry). Public property `executed: { queue, jobName, attempts }[]` with the execution history (capped at 1000 entries). For testing and dev without Redis.
329
- - **`class BullmqQueueDriver`** — production over Redis; see the options table below.
330
- - **`interface QueueDriver`** (Advanced) — contract for custom drivers: `setExecutor(executor)`, `add(queue, jobName, data, options: AddJobOptions)`, `startWorker(queue, { concurrency? })`, optional `stats(queue)` / `retryFailed(queue, { limit? })`, `close()`, plus the optional `name` and `capabilities` fields. Helper types: `AddJobOptions`, `JobExecutor`, `QueueStats`, `DriverCapabilities`.
407
+ - **`class BullmqQueueDriver`** — production over Redis; see the options table below. Imported from its own entry point, `@basaltkit/queue/bullmq` (one import path per backend, like the RabbitMQ/SQS/Kafka driver packages), and requires the optional `bullmq` peer.
408
+ - **`interface QueueDriver`** (Advanced) — contract for custom drivers: `setExecutor(executor)`, `add(queue, jobName, data, options: AddJobOptions)`, `startWorker(queue, { concurrency? })`, optional `stats(queue)` / `retryFailed(queue, { limit? })` / `list(queue, options)`, `close()`, plus the optional `name` and `capabilities` fields. Helper types: `AddJobOptions`, `JobExecutor`, `QueueStats`, `DriverCapabilities`, `JobState`, `JobSummary`, `JobEnvelope`, `ListJobsOptions`.
409
+
410
+ The three optional methods are a deliberate pattern: a driver **omits** what its
411
+ backend cannot do, `QueueManager` returns `undefined`, and the caller gets an
412
+ honest "unsupported" instead of a guess. Omit `list` rather than implement it
413
+ with a destructive read — merely *looking* at a queue must never change it.
414
+
415
+ | Driver | `stats` | `retryFailed` | `list` | Why |
416
+ |---|:---:|:---:|:---:|---|
417
+ | `bullmq` | ✅ | ✅ | ✅ | Redis keeps jobs; reading them is non-destructive. |
418
+ | `sync` | ❌ | ❌ | ❌ | Runs inline and stores nothing. |
419
+ | [`rabbitmq`](https://www.npmjs.com/package/@basaltkit/queue-rabbitmq) | ❌ | ❌ | ❌ | AMQP has no non-destructive read: `basic.get`/consume hide the message from real workers and mark it redelivered. |
420
+ | [`sqs`](https://www.npmjs.com/package/@basaltkit/queue-sqs) | ❌ | ❌ | ❌ | `ReceiveMessage` starts the visibility timeout and bumps `ApproximateReceiveCount` — peeking could redrive jobs to the DLQ. |
421
+ | [`kafka`](https://www.npmjs.com/package/@basaltkit/queue-kafka) | ❌ | ❌ | ❌ | Reading is non-destructive, but a log has no per-message state — any job states would be invented. |
331
422
 
332
423
  #### `new BullmqQueueDriver(options: BullmqDriverOptions)`
333
424
 
425
+ ```ts
426
+ import { BullmqQueueDriver } from '@basaltkit/queue/bullmq' // needs `pnpm add bullmq`
427
+ ```
428
+
429
+ (The `BullmqDriverOptions` **type** is also re-exported from `@basaltkit/queue` — types are erased at build, so they cost nothing.)
430
+
334
431
  | Option | Type | Default | Purpose |
335
432
  |---|---|---|---|
336
433
  | `connection` | `string \| ConnectionOptions` | — (required) | Redis URL (`redis://…`, `rediss://…` → TLS) or ioredis options. |
@@ -391,7 +488,8 @@ Passing your own `driver` bypasses that forwarding — a supplied driver owns it
391
488
  so set them in its constructor:
392
489
 
393
490
  ```ts
394
- import { queuePlugin, BullmqQueueDriver } from '@basaltkit/queue'
491
+ import { queuePlugin } from '@basaltkit/queue'
492
+ import { BullmqQueueDriver } from '@basaltkit/queue/bullmq'
395
493
 
396
494
  queuePlugin({
397
495
  jobs: [SendWelcomeEmail],
@@ -410,6 +508,7 @@ queuePlugin({
410
508
  | `JobNotRegisteredError` | `QUEUE_JOB_NOT_REGISTERED` | `job.dispatch()` was called before the job was registered in a `QueueManager`. |
411
509
  | `UnknownJobError` | `QUEUE_UNKNOWN_JOB` | A job reached the worker but is not registered in that process — producer and worker registered different job lists. |
412
510
  | `UnsupportedJobOptionError` | `QUEUE_UNSUPPORTED_OPTION` | With `onUnsupported: 'throw'`, a dispatch used an option the active driver's `capabilities` do not include. `status = 500`. |
511
+ | `MissingQueueDriverPackageError` | `QUEUE_MISSING_DRIVER_PACKAGE` | `queuePlugin({ connection })` selected the BullMQ driver but the optional `bullmq` peer is not installed. Thrown at boot, with the fix in the message; `.cause` holds the original resolution error. |
413
512
 
414
513
  Errors thrown outside these classes come from the driver's client (ioredis, amqplib, kafkajs,
415
514
  the AWS SDK) and reach you through that driver's `onError`.
@@ -441,6 +540,13 @@ The `onUnsupported` policy caught a dispatch option the active driver can't hono
441
540
  **A job failed for good and nothing was logged.**
442
541
  Only the BullMQ driver reports exhausted jobs, via `onJobFailed` (default `console.error`). If you replaced it with a no-op, you removed the only signal. On the broker drivers, inspect the dead-letter destination (`q.dead`, `<topic>.dead`, `<queue>-dead`).
443
542
 
543
+ **How do I see what's actually on the queue?**
544
+ `basalt queue:stats` for the counts and `basalt queue:jobs` for the individual jobs
545
+ (or `QUEUE`'s `manager.stats()` / `manager.list()` in code). Don't open the broker's
546
+ own client — that couples your app to one backend and hands you the raw dispatch
547
+ envelope instead of your payload. On a driver that can't introspect, both return
548
+ `undefined` / print "Not supported" — an honest gap, not an empty queue.
549
+
444
550
  **Do I need Redis to run the tests?**
445
551
  No. Without `connection`, the plugin uses `SyncQueueDriver`. You can also instantiate the driver directly and inspect `driver.executed`.
446
552
 
package/dist/driver.d.ts CHANGED
@@ -1,3 +1,4 @@
1
+ import type { RequestContext } from '@basaltkit/core';
1
2
  /** Driver-neutral retention: `true`/`false`, a count, or `{ ageMs, count }`. */
2
3
  export type RetentionOption = boolean | number | {
3
4
  ageMs?: number;
@@ -42,6 +43,80 @@ export interface QueueStats {
42
43
  failed: number;
43
44
  delayed: number;
44
45
  }
46
+ /**
47
+ * Driver-neutral job state — deliberately the same vocabulary as
48
+ * {@link QueueStats}, so `stats()` and `list()` describe the same lifecycle.
49
+ * Backend-specific states are mapped onto these by the driver.
50
+ */
51
+ export type JobState = 'waiting' | 'active' | 'completed' | 'failed' | 'delayed';
52
+ /**
53
+ * States {@link QueueDriver.list} returns when the caller picks none.
54
+ *
55
+ * `completed` and `failed` come FIRST and are included on purpose: a worker
56
+ * drains `waiting` in milliseconds, so a healthy queue has `waiting: 0,
57
+ * active: 0` nearly always. Defaulting to only those would answer "did my job
58
+ * run?" with an empty list on a queue that is working perfectly — the classic
59
+ * false alarm. `delayed` is left out of the default (it is a deliberate,
60
+ * separate question); ask for it explicitly.
61
+ */
62
+ export declare const DEFAULT_LIST_STATES: readonly JobState[];
63
+ /** Default `limit` for {@link QueueDriver.list} — a screenful, not a dump. */
64
+ export declare const DEFAULT_LIST_LIMIT = 20;
65
+ /** Hard ceiling on `limit`, so an inspection call can't turn into a full scan. */
66
+ export declare const MAX_LIST_LIMIT = 1000;
67
+ export interface ListJobsOptions {
68
+ /** States to look in. Default {@link DEFAULT_LIST_STATES}. */
69
+ states?: JobState[] | undefined;
70
+ /**
71
+ * Maximum jobs returned in TOTAL (newest first), not per state. Default
72
+ * {@link DEFAULT_LIST_LIMIT}, capped at {@link MAX_LIST_LIMIT}.
73
+ */
74
+ limit?: number | undefined;
75
+ }
76
+ /**
77
+ * A driver-neutral view of ONE job. Drivers must return this — never their
78
+ * backend's own job object (a BullMQ `Job`, an amqplib message, …). Leaking
79
+ * that would invert the coupling this API exists to remove: the app would be
80
+ * back to writing broker-specific code, just through a Basalt method.
81
+ */
82
+ export interface JobSummary {
83
+ /** Backend-assigned job id, as a string. */
84
+ id: string;
85
+ /** The job name — what `defineJob({ name })` declared. */
86
+ name: string;
87
+ /** Which state the job was found in. */
88
+ state: JobState;
89
+ /** Attempts made so far (0 before the first run). */
90
+ attemptsMade: number;
91
+ /** When the job was created, epoch milliseconds. */
92
+ timestamp: number;
93
+ /** The APP's payload — unwrapped from the dispatch envelope. */
94
+ payload: unknown;
95
+ /** The request context captured at dispatch (requestId, tenantId, …), if any. */
96
+ context?: RequestContext | undefined;
97
+ /** Why it failed — only meaningful for `state: 'failed'`. */
98
+ failedReason?: string | undefined;
99
+ }
100
+ /**
101
+ * The wire shape `QueueManager.dispatch` hands to {@link QueueDriver.add} as
102
+ * `data`: the app's payload plus a snapshot of the request context, so the
103
+ * context survives the hop to the worker. Drivers store it opaquely; only
104
+ * `list()` needs to open it — via {@link readJobEnvelope}.
105
+ */
106
+ export interface JobEnvelope {
107
+ payload: unknown;
108
+ context?: RequestContext | undefined;
109
+ }
110
+ /**
111
+ * Opens a dispatch envelope for {@link JobSummary}. Defensive on purpose: a
112
+ * queue can also hold data written by an older version or by a non-Basalt
113
+ * producer, and an inspection command must never throw on it — anything that
114
+ * is not an envelope is reported as the payload itself.
115
+ */
116
+ export declare function readJobEnvelope(data: unknown): {
117
+ payload: unknown;
118
+ context?: RequestContext | undefined;
119
+ };
45
120
  /** Queue driver contract. BullMQ in production; sync in tests/dev. */
46
121
  export interface QueueDriver {
47
122
  /** Short identifier used in diagnostics (e.g. 'bullmq', 'sync'). */
@@ -68,5 +143,17 @@ export interface QueueDriver {
68
143
  retryFailed?(queue: string, options?: {
69
144
  limit?: number;
70
145
  }): Promise<number>;
146
+ /**
147
+ * Optional: list individual jobs (`basalt queue:jobs`) as driver-neutral
148
+ * {@link JobSummary} objects — newest first, `payload` unwrapped from the
149
+ * dispatch envelope via {@link readJobEnvelope}.
150
+ *
151
+ * Omit it when the backend cannot read a job WITHOUT consuming it: faking
152
+ * the capability with a destructive read (an AMQP `basic.get`, an SQS
153
+ * `ReceiveMessage`) would make merely *looking* at a queue change it. The
154
+ * QueueManager then returns `undefined` and the CLI reports the operation as
155
+ * unsupported — an honest gap beats a dangerous guess.
156
+ */
157
+ list?(queue: string, options?: ListJobsOptions): Promise<JobSummary[]>;
71
158
  close(): Promise<void>;
72
159
  }
package/dist/driver.js CHANGED
@@ -1 +1,31 @@
1
- export {};
1
+ /**
2
+ * States {@link QueueDriver.list} returns when the caller picks none.
3
+ *
4
+ * `completed` and `failed` come FIRST and are included on purpose: a worker
5
+ * drains `waiting` in milliseconds, so a healthy queue has `waiting: 0,
6
+ * active: 0` nearly always. Defaulting to only those would answer "did my job
7
+ * run?" with an empty list on a queue that is working perfectly — the classic
8
+ * false alarm. `delayed` is left out of the default (it is a deliberate,
9
+ * separate question); ask for it explicitly.
10
+ */
11
+ export const DEFAULT_LIST_STATES = ['completed', 'failed', 'waiting', 'active'];
12
+ /** Default `limit` for {@link QueueDriver.list} — a screenful, not a dump. */
13
+ export const DEFAULT_LIST_LIMIT = 20;
14
+ /** Hard ceiling on `limit`, so an inspection call can't turn into a full scan. */
15
+ export const MAX_LIST_LIMIT = 1000;
16
+ /**
17
+ * Opens a dispatch envelope for {@link JobSummary}. Defensive on purpose: a
18
+ * queue can also hold data written by an older version or by a non-Basalt
19
+ * producer, and an inspection command must never throw on it — anything that
20
+ * is not an envelope is reported as the payload itself.
21
+ */
22
+ export function readJobEnvelope(data) {
23
+ if (typeof data === 'object' && data !== null && 'payload' in data) {
24
+ const envelope = data;
25
+ return {
26
+ payload: envelope.payload,
27
+ ...(envelope.context !== undefined ? { context: envelope.context } : {}),
28
+ };
29
+ }
30
+ return { payload: data };
31
+ }
@@ -1,5 +1,5 @@
1
1
  import { type ConnectionOptions } from 'bullmq';
2
- import type { AddJobOptions, JobExecutor, QueueDriver, QueueStats } from '../driver.js';
2
+ import { type AddJobOptions, type JobExecutor, type JobSummary, type ListJobsOptions, type QueueDriver, type QueueStats } from '../driver.js';
3
3
  export interface BullmqDriverOptions {
4
4
  /** Redis URL (redis://... or rediss://...) or ioredis connection options. */
5
5
  connection: string | ConnectionOptions;
@@ -48,6 +48,17 @@ export declare class BullmqQueueDriver implements QueueDriver {
48
48
  retryFailed(queue: string, options?: {
49
49
  limit?: number;
50
50
  }): Promise<number>;
51
+ /**
52
+ * Reads jobs WITHOUT consuming them — Redis keeps finished jobs (subject to
53
+ * retention), so inspection here is non-destructive, which is why BullMQ can
54
+ * offer `list` at all.
55
+ *
56
+ * One `getJobs` per state, each capped at `limit`: that keeps every job's
57
+ * state known without an extra `job.getState()` round-trip per job, and
58
+ * stops one busy state (usually `completed`) from starving the others. The
59
+ * merged result is sorted newest-first and truncated to `limit`.
60
+ */
61
+ list(queue: string, options?: ListJobsOptions): Promise<JobSummary[]>;
51
62
  close(): Promise<void>;
52
63
  private queue;
53
64
  }
@@ -1,4 +1,5 @@
1
1
  import { Queue, Worker } from 'bullmq';
2
+ import { DEFAULT_LIST_LIMIT, DEFAULT_LIST_STATES, MAX_LIST_LIMIT, readJobEnvelope, } from '../driver.js';
2
3
  /** Map the driver-neutral retention to BullMQ's (age in seconds), falling back to a default. */
3
4
  function toBullRetention(retention, fallback) {
4
5
  if (retention === undefined)
@@ -84,6 +85,32 @@ export class BullmqQueueDriver {
84
85
  }
85
86
  return retried;
86
87
  }
88
+ /**
89
+ * Reads jobs WITHOUT consuming them — Redis keeps finished jobs (subject to
90
+ * retention), so inspection here is non-destructive, which is why BullMQ can
91
+ * offer `list` at all.
92
+ *
93
+ * One `getJobs` per state, each capped at `limit`: that keeps every job's
94
+ * state known without an extra `job.getState()` round-trip per job, and
95
+ * stops one busy state (usually `completed`) from starving the others. The
96
+ * merged result is sorted newest-first and truncated to `limit`.
97
+ */
98
+ async list(queue, options = {}) {
99
+ const requested = options.states?.length ? options.states : DEFAULT_LIST_STATES;
100
+ const states = [...new Set(requested)];
101
+ const limit = Math.min(Math.max(1, Math.trunc(options.limit ?? DEFAULT_LIST_LIMIT)), MAX_LIST_LIMIT);
102
+ const summaries = [];
103
+ for (const state of states) {
104
+ const jobs = await this.queue(queue).getJobs([state], 0, limit - 1);
105
+ for (const job of jobs) {
106
+ // getJobs can yield holes when a job is removed mid-read (retention).
107
+ if (job)
108
+ summaries.push(toSummary(job, state));
109
+ }
110
+ }
111
+ summaries.sort((a, b) => b.timestamp - a.timestamp);
112
+ return summaries.slice(0, limit);
113
+ }
87
114
  async close() {
88
115
  await Promise.all(this.workers.map((worker) => worker.close()));
89
116
  await Promise.all([...this.queues.values()].map((queue) => queue.close()));
@@ -98,6 +125,24 @@ export class BullmqQueueDriver {
98
125
  return queue;
99
126
  }
100
127
  }
128
+ /**
129
+ * BullMQ `Job` → driver-neutral {@link JobSummary}. The BullMQ object never
130
+ * escapes the driver, and `job.data` (the dispatch envelope) is opened so the
131
+ * caller sees its own payload.
132
+ */
133
+ function toSummary(job, state) {
134
+ const { payload, context } = readJobEnvelope(job.data);
135
+ return {
136
+ id: String(job.id ?? ''),
137
+ name: job.name,
138
+ state,
139
+ attemptsMade: job.attemptsMade ?? 0,
140
+ timestamp: job.timestamp ?? 0,
141
+ payload,
142
+ ...(context !== undefined ? { context } : {}),
143
+ ...(job.failedReason ? { failedReason: job.failedReason } : {}),
144
+ };
145
+ }
101
146
  function parseRedisUrl(url) {
102
147
  const parsed = new URL(url);
103
148
  return {
package/dist/index.d.ts CHANGED
@@ -1,14 +1,24 @@
1
- import { type BullmqDriverOptions } from './drivers/bullmq.js';
2
- import type { QueueDriver } from './driver.js';
1
+ import { BasaltError } from '@basaltkit/core';
2
+ import { type QueueDriver } from './driver.js';
3
+ import type { BullmqDriverOptions } from './drivers/bullmq.js';
3
4
  import type { JobDefinition, JobRetention } from './job.js';
4
5
  import { QueueManager, type UnsupportedPolicy } from './manager.js';
5
6
  export { defineJob, JobValidationError, JobNotRegisteredError, type JobDefinition, type JobSchema, type JobBackoff, type JobRetention, type DispatchOptions, } from './job.js';
6
7
  export { QueueManager, UnknownJobError, UnsupportedJobOptionError, type UnsupportedPolicy, type QueueManagerOptions, } from './manager.js';
7
8
  export { queuedOn, type QueuedListenerOptions } from './bridge.js';
8
9
  export { SyncQueueDriver } from './drivers/sync.js';
9
- export { BullmqQueueDriver, type BullmqDriverOptions } from './drivers/bullmq.js';
10
- export type { QueueDriver, QueueStats, AddJobOptions, JobExecutor, DriverCapabilities } from './driver.js';
10
+ /**
11
+ * The BullMQ driver's *types* stay on the barrel (erased at build, so they cost
12
+ * a consumer nothing). The CLASS lives at its own entry point:
13
+ * `import { BullmqQueueDriver } from '@basaltkit/queue/bullmq'` — the same
14
+ * shape as the RabbitMQ/SQS/Kafka driver packages, one import path per backend.
15
+ */
16
+ export type { BullmqDriverOptions } from './drivers/bullmq.js';
17
+ export { readJobEnvelope, DEFAULT_LIST_STATES, DEFAULT_LIST_LIMIT, MAX_LIST_LIMIT, type QueueDriver, type QueueStats, type AddJobOptions, type JobExecutor, type DriverCapabilities, type JobState, type JobSummary, type JobEnvelope, type ListJobsOptions, } from './driver.js';
11
18
  export declare const QUEUE: import("@basaltkit/core").Token<QueueManager>;
19
+ export declare class MissingQueueDriverPackageError extends BasaltError {
20
+ constructor(options?: ErrorOptions);
21
+ }
12
22
  export interface QueuePluginOptions {
13
23
  /** Jobs known to this process (producer and/or worker). */
14
24
  jobs?: JobDefinition<unknown>[];
package/dist/index.js CHANGED
@@ -1,23 +1,59 @@
1
- import { createToken, definePlugin, ensureMetadata } from '@basaltkit/core';
2
- import { BullmqQueueDriver } from './drivers/bullmq.js';
1
+ import { BasaltError, createToken, definePlugin, ensureMetadata } from '@basaltkit/core';
2
+ import { DEFAULT_LIST_LIMIT, DEFAULT_LIST_STATES, } from './driver.js';
3
3
  import { SyncQueueDriver } from './drivers/sync.js';
4
4
  import { QueueManager } from './manager.js';
5
5
  export { defineJob, JobValidationError, JobNotRegisteredError, } from './job.js';
6
6
  export { QueueManager, UnknownJobError, UnsupportedJobOptionError, } from './manager.js';
7
7
  export { queuedOn } from './bridge.js';
8
8
  export { SyncQueueDriver } from './drivers/sync.js';
9
- export { BullmqQueueDriver } from './drivers/bullmq.js';
9
+ export { readJobEnvelope, DEFAULT_LIST_STATES, DEFAULT_LIST_LIMIT, MAX_LIST_LIMIT, } from './driver.js';
10
10
  export const QUEUE = createToken('queue');
11
+ let bullmqDriverModule;
12
+ /** Actionable guidance instead of a bare ERR_MODULE_NOT_FOUND from deep inside the driver. */
13
+ const MISSING_BULLMQ = '`queuePlugin({ connection })` selects the BullMQ driver, which needs the `bullmq` package — ' +
14
+ 'an optional peer dependency of @basaltkit/queue that is not installed. ' +
15
+ 'Either install it (`pnpm add bullmq`), or pass an explicit `driver:` — ' +
16
+ '`@basaltkit/queue-rabbitmq`, `@basaltkit/queue-sqs`, `@basaltkit/queue-kafka`, ' +
17
+ 'or `new SyncQueueDriver()` for dev/tests.';
18
+ export class MissingQueueDriverPackageError extends BasaltError {
19
+ constructor(options) {
20
+ super('QUEUE_MISSING_DRIVER_PACKAGE', MISSING_BULLMQ, options);
21
+ }
22
+ }
23
+ async function loadBullmqDriver() {
24
+ if (bullmqDriverModule)
25
+ return bullmqDriverModule;
26
+ try {
27
+ bullmqDriverModule = await import('./drivers/bullmq.js');
28
+ }
29
+ catch (error) {
30
+ throw new MissingQueueDriverPackageError({ cause: error });
31
+ }
32
+ return bullmqDriverModule;
33
+ }
11
34
  export function queuePlugin(options = {}) {
12
35
  return definePlugin({
13
36
  name: 'basalt:queue',
14
- register({ container }) {
37
+ async register({ container }) {
38
+ // Resolve the BullMQ driver MODULE here — `BasaltApp.boot()` awaits
39
+ // `register`, so the class is in hand before anything can resolve QUEUE,
40
+ // and the container factory below stays synchronous. This is still a
41
+ // bindings-only phase: loading a module is not I/O against the world —
42
+ // no Redis connection is opened until the singleton is first resolved.
43
+ // Doing it in `boot` instead would leave a hole: a plugin booting earlier
44
+ // that resolves QUEUE would hit an unloaded driver.
45
+ if (!options.driver && options.connection)
46
+ await loadBullmqDriver();
15
47
  registerQueueCommands(container);
16
48
  container.singleton(QUEUE, () => {
17
49
  let driver = options.driver;
18
50
  if (!driver) {
19
51
  if (options.connection) {
20
- driver = new BullmqQueueDriver({
52
+ // Defensive: only reachable if `register`'s promise was dropped by
53
+ // a non-standard host instead of awaited.
54
+ if (!bullmqDriverModule)
55
+ throw new MissingQueueDriverPackageError();
56
+ driver = new bullmqDriverModule.BullmqQueueDriver({
21
57
  connection: options.connection,
22
58
  ...(options.onError !== undefined ? { onError: options.onError } : {}),
23
59
  ...(options.onJobFailed !== undefined ? { onJobFailed: options.onJobFailed } : {}),
@@ -58,14 +94,52 @@ export function queuePlugin(options = {}) {
58
94
  },
59
95
  });
60
96
  }
97
+ /** The states `queue:jobs --states` accepts — the driver-neutral vocabulary. */
98
+ const LIST_STATES = ['waiting', 'active', 'completed', 'failed', 'delayed'];
99
+ /** Parses `--states failed,waiting`. Throws on an unknown state rather than silently dropping it. */
100
+ function parseStates(raw) {
101
+ if (typeof raw !== 'string' || raw.trim() === '')
102
+ return undefined;
103
+ const states = raw.split(',').map((part) => part.trim()).filter(Boolean);
104
+ const unknown = states.filter((state) => !LIST_STATES.includes(state));
105
+ if (unknown.length > 0) {
106
+ throw new Error(`Unknown job state(s): ${unknown.join(', ')}. Valid states: ${LIST_STATES.join(', ')}.`);
107
+ }
108
+ return states;
109
+ }
110
+ /** Compact age for the CLI table ('3s', '12m', '4h', '2d') — friendlier than an epoch. */
111
+ function formatAge(timestamp, now) {
112
+ const seconds = Math.max(0, Math.round((now - timestamp) / 1000));
113
+ if (seconds < 60)
114
+ return `${seconds}s`;
115
+ if (seconds < 3600)
116
+ return `${Math.round(seconds / 60)}m`;
117
+ if (seconds < 86_400)
118
+ return `${Math.round(seconds / 3600)}h`;
119
+ return `${Math.round(seconds / 86_400)}d`;
120
+ }
121
+ /** One-line, length-capped rendering of an arbitrary value for a table cell. */
122
+ function preview(value, max = 120) {
123
+ let text;
124
+ try {
125
+ text = typeof value === 'string' ? value : JSON.stringify(value);
126
+ }
127
+ catch {
128
+ text = String(value);
129
+ }
130
+ text = (text ?? String(value)).replace(/\s+/g, ' ');
131
+ return text.length > max ? `${text.slice(0, max - 1)}…` : text;
132
+ }
61
133
  /**
62
- * Registers `queue:work`, `queue:stats` and `queue:retry` into the CLI command
63
- * bucket. Commands resolve the manager lazily, so they work with whatever driver
134
+ * Registers `queue:work`, `queue:stats`, `queue:retry` and `queue:jobs` into the
135
+ * CLI command bucket. Commands resolve the manager lazily, so they work with whatever driver
64
136
  * the app configured. Registered structurally to avoid a hard @basaltkit/cli dep.
65
137
  */
66
138
  function registerQueueCommands(container) {
67
139
  const manager = () => container.get(QUEUE);
68
140
  const unsupported = 'Not supported by the active queue driver — the inline sync driver keeps no job state. Use the BullMQ driver (a Redis `connection`).';
141
+ const unsupportedList = 'Not supported by the active queue driver — listing jobs needs a backend that can read a job WITHOUT consuming it. ' +
142
+ 'Use the BullMQ driver (a Redis `connection`); the sync driver keeps no job state, and the RabbitMQ/SQS/Kafka drivers cannot list non-destructively.';
69
143
  ensureMetadata(container).add('commands', {
70
144
  name: 'queue:work',
71
145
  description: 'Run a worker that processes jobs for a queue (Ctrl+C to stop)',
@@ -104,4 +178,44 @@ function registerQueueCommands(container) {
104
178
  io.log(`Re-enqueued ${retried} failed job(s) on "${queue}".`);
105
179
  },
106
180
  });
181
+ ensureMetadata(container).add('commands', {
182
+ name: 'queue:jobs',
183
+ description: 'List individual jobs on a queue (id/name/state/attempts); --payload also prints their data',
184
+ async handle({ io, flags, }) {
185
+ const queue = typeof flags['queue'] === 'string' ? flags['queue'] : 'default';
186
+ const states = parseStates(flags['states']);
187
+ const limit = typeof flags['limit'] === 'string' ? Number(flags['limit']) : undefined;
188
+ const jobs = await manager().list(queue, {
189
+ ...(states !== undefined ? { states } : {}),
190
+ ...(limit !== undefined ? { limit } : {}),
191
+ });
192
+ if (!jobs) {
193
+ io.log(unsupportedList);
194
+ return;
195
+ }
196
+ const asked = (states ?? DEFAULT_LIST_STATES).join(', ');
197
+ if (jobs.length === 0) {
198
+ io.log(`No jobs on "${queue}" in ${asked}.`);
199
+ return;
200
+ }
201
+ // Payloads can hold personal data, so they are NOT printed by default —
202
+ // `--payload` is the deliberate opt-in (same posture as the docs' warning).
203
+ const withPayload = flags['payload'] === true || flags['payload'] === 'true';
204
+ const now = Date.now();
205
+ const anyFailed = jobs.some((job) => job.failedReason !== undefined);
206
+ io.table(jobs.map((job) => ({
207
+ id: job.id,
208
+ name: job.name,
209
+ state: job.state,
210
+ attempts: job.attemptsMade,
211
+ age: formatAge(job.timestamp, now),
212
+ ...(anyFailed ? { reason: job.failedReason ? preview(job.failedReason, 60) : '' } : {}),
213
+ ...(withPayload ? { payload: preview(job.payload) } : {}),
214
+ })));
215
+ io.log(`${jobs.length} job(s) on "${queue}" (${asked}, limit ${limit ?? DEFAULT_LIST_LIMIT}).` +
216
+ (withPayload
217
+ ? ' Payloads shown — they can contain personal data.'
218
+ : ' Payloads hidden — add --payload to include them.'));
219
+ },
220
+ });
107
221
  }
package/dist/manager.d.ts CHANGED
@@ -1,5 +1,5 @@
1
1
  import { BasaltError } from '@basaltkit/core';
2
- import type { QueueDriver, QueueStats } from './driver.js';
2
+ import type { JobSummary, ListJobsOptions, QueueDriver, QueueStats } from './driver.js';
3
3
  import { type DispatchOptions, type JobDefinition, type JobDispatcher, type JobRetention } from './job.js';
4
4
  export declare class UnknownJobError extends BasaltError {
5
5
  constructor(job: string);
@@ -55,6 +55,19 @@ export declare class QueueManager implements JobDispatcher {
55
55
  retryFailed(queue?: string, options?: {
56
56
  limit?: number;
57
57
  }): Promise<number | undefined>;
58
+ /**
59
+ * Lists individual jobs on the queue — newest first, with each job's own
60
+ * payload already unwrapped from the dispatch envelope. Returns `undefined`
61
+ * when the driver can't list (the sync driver keeps no state; a broker that
62
+ * cannot read a message without consuming it deliberately omits `list`).
63
+ *
64
+ * This is the supported alternative to reaching around the framework into
65
+ * the broker's own client, which couples the app to one backend.
66
+ *
67
+ * A summary carries the job's payload, so it can carry personal data —
68
+ * treat the result as sensitive (see the queues guide).
69
+ */
70
+ list(queue?: string, options?: ListJobsOptions): Promise<JobSummary[] | undefined>;
58
71
  close(): Promise<void>;
59
72
  /** Executes a job received from the driver: validates, restores the context, runs the handler. */
60
73
  private execute;
package/dist/manager.js CHANGED
@@ -128,6 +128,21 @@ export class QueueManager {
128
128
  async retryFailed(queue = 'default', options = {}) {
129
129
  return this.driver.retryFailed?.(queue, options);
130
130
  }
131
+ /**
132
+ * Lists individual jobs on the queue — newest first, with each job's own
133
+ * payload already unwrapped from the dispatch envelope. Returns `undefined`
134
+ * when the driver can't list (the sync driver keeps no state; a broker that
135
+ * cannot read a message without consuming it deliberately omits `list`).
136
+ *
137
+ * This is the supported alternative to reaching around the framework into
138
+ * the broker's own client, which couples the app to one backend.
139
+ *
140
+ * A summary carries the job's payload, so it can carry personal data —
141
+ * treat the result as sensitive (see the queues guide).
142
+ */
143
+ async list(queue = 'default', options = {}) {
144
+ return this.driver.list?.(queue, options);
145
+ }
131
146
  async close() {
132
147
  await this.driver.close();
133
148
  }
package/package.json CHANGED
@@ -1,10 +1,10 @@
1
1
  {
2
2
  "name": "@basaltkit/queue",
3
- "version": "1.4.1",
3
+ "version": "2.0.0",
4
4
  "engines": {
5
5
  "node": ">=22.5.0"
6
6
  },
7
- "description": "Basalt queues on top of BullMQ: declarative jobs with Zod payloads, context propagation (tenant/requestId) to workers and a sync driver for tests.",
7
+ "description": "Driver-agnostic Basalt queues: declarative jobs with Zod payloads, context propagation (tenant/requestId) to workers, a BullMQ/Redis driver behind an optional peer, and a sync driver for tests.",
8
8
  "license": "MIT",
9
9
  "type": "module",
10
10
  "sideEffects": false,
@@ -12,18 +12,30 @@
12
12
  ".": {
13
13
  "types": "./dist/index.d.ts",
14
14
  "import": "./dist/index.js"
15
+ },
16
+ "./bullmq": {
17
+ "types": "./dist/drivers/bullmq.d.ts",
18
+ "import": "./dist/drivers/bullmq.js"
15
19
  }
16
20
  },
17
21
  "files": [
18
22
  "dist"
19
23
  ],
20
24
  "dependencies": {
21
- "bullmq": "^6.2.1",
22
- "@basaltkit/core": "^1.3.1",
23
- "@basaltkit/events": "^1.1.1"
25
+ "@basaltkit/events": "^1.1.1",
26
+ "@basaltkit/core": "^1.3.1"
27
+ },
28
+ "peerDependencies": {
29
+ "bullmq": "^6.2.1"
30
+ },
31
+ "peerDependenciesMeta": {
32
+ "bullmq": {
33
+ "optional": true
34
+ }
24
35
  },
25
36
  "devDependencies": {
26
37
  "@types/node": "^26.3.0",
38
+ "bullmq": "^6.2.1",
27
39
  "typescript": "^7.0.2",
28
40
  "vitest": "^4.1.11",
29
41
  "zod": "^3.24.0 || ^4.0.0",