queue-jobs-worker 1.0.4 → 1.0.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +51 -78
- package/README.md +111 -77
- package/assets/queue-jobs-worker-demo.gif +0 -0
- package/dist/core/client.d.ts.map +1 -1
- package/dist/core/queue.d.ts +3 -1
- package/dist/core/queue.d.ts.map +1 -1
- package/dist/index.cjs +188 -139
- package/dist/index.cjs.map +1 -1
- package/dist/index.js +188 -139
- package/dist/index.js.map +1 -1
- package/dist/lib/scripts/claim.lua +27 -0
- package/dist/lib/scripts/index.d.ts +5 -0
- package/dist/lib/scripts/index.d.ts.map +1 -0
- package/dist/lib/scripts/rate-limit.lua +36 -0
- package/dist/lib/scripts/recover-stalled.lua +40 -0
- package/dist/lib/scripts/renew-lock.lua +16 -0
- package/dist/storage/mysql.adapter.d.ts.map +1 -1
- package/dist/storage/postgres.adapter.d.ts.map +1 -1
- package/dist/storage/redis.adapter.d.ts.map +1 -1
- package/package.json +1 -1
- package/assets/queue-jobs-worker-demo.mp4 +0 -0
package/CHANGELOG.md
CHANGED
|
@@ -4,22 +4,22 @@ All notable changes to **queue-jobs-worker** will be documented in this file.
|
|
|
4
4
|
|
|
5
5
|
The format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
|
|
6
6
|
This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
|
-
|
|
7
|
+
|
|
8
|
+
## [1.0.5] — 2026-09-15
|
|
8
9
|
|
|
9
10
|
### Core
|
|
10
11
|
|
|
11
12
|
### Fixed
|
|
12
13
|
|
|
13
|
-
-
|
|
14
|
+
- **Storage Initialization Enforcement prior to Queue Creation & Execution** ([#14](https://github.com/rafidahmed870/queue-jobs-worker/issues/14))
|
|
14
15
|
|
|
15
|
-
Previously,
|
|
16
|
+
Previously, `QueueClient.createQueue()` allowed queues to be created before `await client.init()` was called when using external storage dialects (e.g. Redis, PostgreSQL, MySQL). This caused the created queue to bind to the temporary `InMemoryStorageAdapter` instance. When `client.init()` was subsequently called, the real external storage adapter replaced the internal storage field on `QueueClient`, rendering previously enqueued jobs lost or inaccessible.
|
|
16
17
|
|
|
17
18
|
After the fix:
|
|
18
19
|
|
|
19
|
-
- `
|
|
20
|
-
-
|
|
21
|
-
-
|
|
22
|
-
- User processors can monitor `signal.aborted` or pass `signal` to async operations (e.g. `fetch`, database queries, timers) for cooperative cancellation.
|
|
20
|
+
- `QueueClient.createQueue()` checks `isInitialised` before creating a queue. For external dialects and custom adapters, attempting to create a queue before `await client.init()` throws an explicit error.
|
|
21
|
+
- Queue operations (`enqueue`, `getJob`, `getJobs`, `getJobCounts`, and `createWorker`) enforce initialization status checks before executing, preventing operations on uninitialized storage.
|
|
22
|
+
- In-memory dialect continues to auto-initialize synchronously, preserving convenient single-line setup for tests and local development.
|
|
23
23
|
|
|
24
24
|
---
|
|
25
25
|
|
|
@@ -36,106 +36,77 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
|
|
|
36
36
|
<!-- Links -->
|
|
37
37
|
[1.0.0]: https://github.com/rafidahmed870/queue-jobs-worker/releases/tag/v1.0.0
|
|
38
38
|
|
|
39
|
-
###
|
|
40
|
-
|
|
41
|
-
### Fixed
|
|
42
|
-
|
|
43
|
-
- **`recoverStalledJobs()` race condition — stale recovery overwrites a live job** ([#6](https://github.com/rafidahmed870/queue-jobs-worker/issues/6))
|
|
39
|
+
### Lib
|
|
44
40
|
|
|
45
|
-
|
|
41
|
+
### Added
|
|
46
42
|
|
|
47
|
-
|
|
48
|
-
|
|
43
|
+
- **`lib/scripts/` — Redis Lua scripts extracted to standalone `.lua` files**
|
|
44
|
+
- `claim.lua` — atomic job claim with delayed-job promotion.
|
|
45
|
+
- `recover-stalled.lua` — compare-and-swap stalled job recovery.
|
|
46
|
+
- `renew-lock.lua` — atomic lock renewal with ownership guard.
|
|
47
|
+
- `rate-limit.lua` — atomic rate limit decision, reset, and counter increment.
|
|
48
|
+
- `lib/scripts/index.ts` re-exports scripts as named string constants (`CLAIM_LUA`, `RECOVER_STALLED_LUA`, `RENEW_LOCK_LUA`, `RATE_LIMIT_LUA`).
|
|
49
|
+
- Scripts are embedded into the CJS/ESM distribution bundles at build time via `tsup`'s `loader: { ".lua": "text" }`.
|
|
49
50
|
|
|
50
|
-
|
|
51
|
-
its lock. The write pipeline had no knowledge of that change and would
|
|
52
|
-
unconditionally overwrite the job back to `"waiting"`, causing duplicate
|
|
53
|
-
processing or data loss.
|
|
51
|
+
---
|
|
54
52
|
|
|
55
|
-
|
|
56
|
-
per-job Lua script (`RECOVER_STALLED_LUA`) that implements a
|
|
57
|
-
**compare-and-swap (CAS)** guard. The script atomically re-reads
|
|
58
|
-
`lockExpiresAt`, `lockId`, and `status` from the hash and aborts if any of
|
|
59
|
-
the three values differ from what the caller observed in the read phase.
|
|
60
|
-
Because Redis executes Lua scripts as a single indivisible command, no
|
|
61
|
-
concurrent write can slip between the re-read and the state update. The
|
|
62
|
-
pre-filter (skip jobs whose lock has not yet expired) is preserved as an
|
|
63
|
-
optimisation to avoid unnecessary Lua round-trips.
|
|
53
|
+
### Storage
|
|
64
54
|
|
|
65
|
-
|
|
66
|
-
tick so the race is theoretical, but an equivalent CAS guard has been added
|
|
67
|
-
for consistency: `lockId` and `lockExpiresAt` are snapshotted at decision
|
|
68
|
-
time and re-validated immediately before the write. Any interleaving that
|
|
69
|
-
mutated those fields will cause the recovery to be skipped.
|
|
55
|
+
### Added
|
|
70
56
|
|
|
71
|
-
|
|
57
|
+
- **Consistent timestamps across Lua scripts**
|
|
58
|
+
- Changed `RedisStorageAdapter` to use `now_iso` from `ARGV[3]` for `updatedAt` in `CLAIM_LUA` and `RECOVER_STALLED_LUA`.
|
|
59
|
+
- Previously, `updatedAt` was sometimes derived from `lockExpiresAt`, which could differ from the actual time of the operation.
|
|
72
60
|
|
|
73
|
-
|
|
61
|
+
### Fixed
|
|
74
62
|
|
|
75
|
-
|
|
63
|
+
- **Atomic rate limiting across Redis, PostgreSQL, and MySQL adapters**
|
|
64
|
+
- `RedisStorageAdapter`: Implemented `rate-limit.lua` (`RATE_LIMIT_LUA`) script to perform window check, expiry reset, counter evaluation, increment, and TTL renewal atomically inside Redis.
|
|
65
|
+
- `PostgreSQLStorageAdapter`: Wrapped `checkAndIncrementRateLimit` in a pool client transaction (`BEGIN ... COMMIT`) utilizing `INSERT ... ON CONFLICT DO NOTHING` and `SELECT ... FOR UPDATE` row locking.
|
|
66
|
+
- `MySQLStorageAdapter`: Wrapped `checkAndIncrementRateLimit` in a connection transaction (`beginTransaction ... commit`) utilizing `INSERT ... ON DUPLICATE KEY UPDATE` and `SELECT ... FOR UPDATE` row locking.
|
|
76
67
|
|
|
77
|
-
### Fixed
|
|
78
68
|
|
|
79
|
-
|
|
69
|
+
---
|
|
80
70
|
|
|
81
|
-
|
|
71
|
+
### Types
|
|
82
72
|
|
|
83
|
-
|
|
73
|
+
### Added
|
|
84
74
|
|
|
85
|
-
|
|
86
|
-
-
|
|
87
|
-
-
|
|
88
|
-
- User processors can monitor `signal.aborted` or pass `signal` to async operations (e.g. `fetch`, database queries, timers) for cooperative cancellation.
|
|
75
|
+
- **`lua.d.ts` — ambient module declaration for `.lua` imports**
|
|
76
|
+
- Declares `declare module "*.lua"` so TypeScript recognises `.lua` files as `string`-exporting modules.
|
|
77
|
+
- Required by `src/lib/scripts/index.ts` to import Lua scripts directly without type errors.
|
|
89
78
|
|
|
90
|
-
|
|
79
|
+
---
|
|
91
80
|
|
|
92
|
-
###
|
|
81
|
+
### Tests
|
|
93
82
|
|
|
94
|
-
|
|
83
|
+
### Added
|
|
95
84
|
|
|
96
|
-
|
|
85
|
+
- **`lua-scripts.test.ts` — unit tests for Redis Lua scripts**
|
|
86
|
+
- 22 tests covering all four Lua scripts (`CLAIM_LUA`, `RECOVER_STALLED_LUA`, `RENEW_LOCK_LUA`, `RATE_LIMIT_LUA`).
|
|
87
|
+
- Verifies each script loads as a non-empty string from `src/lib/scripts/index.ts`.
|
|
88
|
+
- Asserts presence of critical Redis commands (`ZPOPMIN`, `ZRANGEBYSCORE`, `SADD`, `HSET`, `SREM`, `ZADD`, `INCR`, `EXPIRE`) and CAS/RateLimit guard conditions.
|
|
89
|
+
- **`vitest.config.ts` — `rawLuaPlugin` added**
|
|
90
|
+
- Custom Vite transform plugin that loads `.lua` files as raw text strings during tests, mirroring `tsup`'s `loader: { ".lua": "text" }` used at build time.
|
|
97
91
|
|
|
98
92
|
---
|
|
99
93
|
|
|
100
|
-
## [1.0.
|
|
94
|
+
## [1.0.4] — 2026-09-13
|
|
101
95
|
|
|
102
96
|
### Core
|
|
103
97
|
|
|
104
98
|
### Fixed
|
|
105
99
|
|
|
106
|
-
- **`Worker` —
|
|
100
|
+
- **`Worker` — Job timeout cooperative cancellation via `AbortSignal`** ([#12](https://github.com/rafidahmed870/queue-jobs-worker/issues/12))
|
|
107
101
|
|
|
108
|
-
`
|
|
109
|
-
a try/catch. If the import failed — or if the resolved `Cron` class was not a
|
|
110
|
-
function — the code silently fell back to `Date.now() + 60_000`, scheduling
|
|
111
|
-
the next run 60 seconds later regardless of the configured cron expression.
|
|
112
|
-
The same silent fallback was also triggered for invalid cron expressions that
|
|
113
|
-
caused the `Cron` constructor to throw.
|
|
102
|
+
Previously, when a job attempt reached its configured `timeout`, the worker rejected the internal execution promise and marked the attempt as failed (or scheduled a retry), but the underlying processor `Promise` continued running in the background. This could lead to duplicate side effects when retries overlapped with timed-out attempts.
|
|
114
103
|
|
|
115
104
|
After the fix:
|
|
116
105
|
|
|
117
|
-
- `
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
-
|
|
121
|
-
`worker:error` event is emitted and re-enqueue is skipped. The worker
|
|
122
|
-
remains running.
|
|
123
|
-
- If `cronInstance.nextRun()` returns `null` (the schedule has no future
|
|
124
|
-
occurrences), a `worker:error` is emitted and re-enqueue is skipped. Again,
|
|
125
|
-
the worker keeps running.
|
|
126
|
-
- The 1-minute fallback path has been removed entirely — there is no silent
|
|
127
|
-
fallback under any failure condition.
|
|
128
|
-
|
|
129
|
-
- **`Worker` — rate-limit quota no longer consumed on empty-queue polls** ([#4](https://github.com/rafidahmed870/queue-jobs-worker/issues/4))
|
|
130
|
-
|
|
131
|
-
`claimNext()` previously called `checkAndIncrementRateLimit()` before
|
|
132
|
-
attempting to claim a job. This meant every poll cycle against an empty queue
|
|
133
|
-
burned a quota slot, potentially exhausting the configured window budget
|
|
134
|
-
before any real work was done. After the fix, the storage `claim()` call
|
|
135
|
-
happens first; the rate-limit counter is only incremented when a job is
|
|
136
|
-
actually claimed for processing. If the rate limit is reached at that point
|
|
137
|
-
the lock is immediately released via `releaseLock()` so the job remains
|
|
138
|
-
reclaimable on the next window.
|
|
106
|
+
- `Processor` type signature is updated: `type Processor<TPayload = unknown> = (job: Job<TPayload>, signal: AbortSignal) => Promise<void>`.
|
|
107
|
+
- An `AbortController` is created for each job attempt.
|
|
108
|
+
- When job execution times out, the worker aborts the `AbortSignal` with a timeout error before rejecting the wrapper promise.
|
|
109
|
+
- User processors can monitor `signal.aborted` or pass `signal` to async operations (e.g. `fetch`, database queries, timers) for cooperative cancellation.
|
|
139
110
|
|
|
140
111
|
---
|
|
141
112
|
|
|
@@ -151,6 +122,8 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
|
|
|
151
122
|
|
|
152
123
|
<!-- Links -->
|
|
153
124
|
|
|
125
|
+
[1.0.5]: https://github.com/rafidahmed870/queue-jobs-worker/compare/v1.0.0...v1.0.5
|
|
126
|
+
[1.0.0]: https://github.com/rafidahmed870/queue-jobs-worker/releases/tag/v1.0.0
|
|
154
127
|
[1.0.4]: https://github.com/rafidahmed870/queue-jobs-worker/compare/v1.0.0...v1.0.4
|
|
155
128
|
[1.0.0]: https://github.com/rafidahmed870/queue-jobs-worker/releases/tag/v1.0.0
|
|
156
129
|
[1.0.3]: https://github.com/rafidahmed870/queue-jobs-worker/compare/v1.0.2...v1.0.3
|
package/README.md
CHANGED
|
@@ -6,13 +6,16 @@ A durable, TypeScript-first job queue for Node.js built for asynchronous work, r
|
|
|
6
6
|
|
|
7
7
|
<p align="center">
|
|
8
8
|
<a href="https://www.npmjs.com/package/queue-jobs-worker">
|
|
9
|
-
<img src="https://img.shields.io/npm/
|
|
9
|
+
<img src="https://img.shields.io/npm/dm/queue-jobs-worker.svg?label=npm%20downloads" alt="npm downloads">
|
|
10
|
+
</a>
|
|
11
|
+
<a href="https://www.npmjs.com/package/queue-jobs-worker">
|
|
12
|
+
<img src="https://img.shields.io/npm/v/queue-jobs-worker.svg?label=npm%20version" alt="npm version">
|
|
10
13
|
</a>
|
|
11
14
|
<a href="./LICENSE">
|
|
12
15
|
<img src="https://img.shields.io/npm/l/queue-jobs-worker.svg" alt="license">
|
|
13
16
|
</a>
|
|
14
17
|
<a href="https://nodejs.org">
|
|
15
|
-
<img src="https://img.shields.io/node/v/queue-jobs-worker.svg" alt="node">
|
|
18
|
+
<img src="https://img.shields.io/node/v/queue-jobs-worker.svg" alt="node version">
|
|
16
19
|
</a>
|
|
17
20
|
</p>
|
|
18
21
|
|
|
@@ -31,7 +34,7 @@ It supports all major local and production-friendly backends:
|
|
|
31
34
|
|
|
32
35
|
For a detailed feature breakdown, see [FEATURES.md](./FEATURES.md).
|
|
33
36
|
|
|
34
|
-
|
|
37
|
+
<img src="./assets/queue-jobs-worker-demo.gif" alt="demo" />
|
|
35
38
|
|
|
36
39
|
## Installation
|
|
37
40
|
|
|
@@ -114,26 +117,26 @@ const client = new QueueClient({
|
|
|
114
117
|
await client.init();
|
|
115
118
|
```
|
|
116
119
|
|
|
117
|
-
| Dialect
|
|
118
|
-
|
|
119
|
-
| `memory`
|
|
120
|
-
| `redis`
|
|
121
|
-
| `postgres` | `postgresql://user:pass@host:5432/dbname`
|
|
122
|
-
| `mysql`
|
|
120
|
+
| Dialect | Connection Format | `client.init()` Behavior |
|
|
121
|
+
| ---------- | --------------------------------------------- | --------------------------------- |
|
|
122
|
+
| `memory` | N/A | No-op (transient memory store) |
|
|
123
|
+
| `redis` | `redis://...`, `rediss://...` (TLS), Auth URL | Connects & verifies with `PING` |
|
|
124
|
+
| `postgres` | `postgresql://user:pass@host:5432/dbname` | `SELECT 1` check & creates schema |
|
|
125
|
+
| `mysql` | `mysql://user:pass@host:3306/dbname` | Connection check & creates schema |
|
|
123
126
|
|
|
124
127
|
---
|
|
125
128
|
|
|
126
129
|
## Core Concepts
|
|
127
130
|
|
|
128
|
-
| Concept
|
|
129
|
-
|
|
130
|
-
| `QueueClient`
|
|
131
|
-
| `Queue`
|
|
132
|
-
| `Job`
|
|
133
|
-
| `Worker`
|
|
134
|
-
| `Processor`
|
|
135
|
-
| `StorageAdapter` | A backend abstraction for durable storage
|
|
136
|
-
| `DLQ`
|
|
131
|
+
| Concept | Description |
|
|
132
|
+
| ---------------- | -------------------------------------------------------- |
|
|
133
|
+
| `QueueClient` | Entry point that owns configuration, storage, and queues |
|
|
134
|
+
| `Queue` | A separate job stream with its own settings |
|
|
135
|
+
| `Job` | A unit of work passed to your processor |
|
|
136
|
+
| `Worker` | Claims and executes jobs |
|
|
137
|
+
| `Processor` | Your async function, e.g. `async (job) => { ... }` |
|
|
138
|
+
| `StorageAdapter` | A backend abstraction for durable storage |
|
|
139
|
+
| `DLQ` | Dead Letter Queue for permanently failed jobs |
|
|
137
140
|
|
|
138
141
|
---
|
|
139
142
|
|
|
@@ -180,13 +183,17 @@ const queue = client.createQueue("notifications");
|
|
|
180
183
|
|
|
181
184
|
await queue.enqueue("send-push", { userId: "u_123" });
|
|
182
185
|
|
|
183
|
-
await queue.enqueue(
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
186
|
+
await queue.enqueue(
|
|
187
|
+
"send-push",
|
|
188
|
+
{ userId: "u_123" },
|
|
189
|
+
{
|
|
190
|
+
attempts: 5,
|
|
191
|
+
retryDelay: 2000,
|
|
192
|
+
backoff: "linear",
|
|
193
|
+
timeout: 10_000,
|
|
194
|
+
priority: 10,
|
|
195
|
+
},
|
|
196
|
+
);
|
|
190
197
|
```
|
|
191
198
|
|
|
192
199
|
If you want TypeScript type safety for `job.data`, pass a generic when creating the queue, such as `client.createQueue<{ userId: string }>("notifications")`.
|
|
@@ -386,19 +393,45 @@ You can provide a custom backend by implementing the `StorageAdapter` interface.
|
|
|
386
393
|
const { QueueClient } = require("queue-jobs-worker");
|
|
387
394
|
|
|
388
395
|
class MongoStorageAdapter {
|
|
389
|
-
async initialize() {
|
|
390
|
-
|
|
391
|
-
|
|
392
|
-
async
|
|
393
|
-
|
|
394
|
-
|
|
395
|
-
async
|
|
396
|
-
|
|
397
|
-
|
|
398
|
-
async
|
|
399
|
-
|
|
400
|
-
|
|
401
|
-
async
|
|
396
|
+
async initialize() {
|
|
397
|
+
/* connect, create indexes */
|
|
398
|
+
}
|
|
399
|
+
async close() {
|
|
400
|
+
/* disconnect */
|
|
401
|
+
}
|
|
402
|
+
async enqueue(input) {
|
|
403
|
+
/* ... */
|
|
404
|
+
}
|
|
405
|
+
async claim(input) {
|
|
406
|
+
/* atomic claim */
|
|
407
|
+
}
|
|
408
|
+
async complete(jobId) {
|
|
409
|
+
/* ... */
|
|
410
|
+
}
|
|
411
|
+
async requeue(input) {
|
|
412
|
+
/* ... */
|
|
413
|
+
}
|
|
414
|
+
async moveToDlq(input) {
|
|
415
|
+
/* ... */
|
|
416
|
+
}
|
|
417
|
+
async releaseLock(jobId) {
|
|
418
|
+
/* ... */
|
|
419
|
+
}
|
|
420
|
+
async recoverStalledJobs(queue, now) {
|
|
421
|
+
/* ... */
|
|
422
|
+
}
|
|
423
|
+
async getJob(jobId) {
|
|
424
|
+
/* ... */
|
|
425
|
+
}
|
|
426
|
+
async getJobs(filter) {
|
|
427
|
+
/* ... */
|
|
428
|
+
}
|
|
429
|
+
async getJobCounts(queue) {
|
|
430
|
+
/* ... */
|
|
431
|
+
}
|
|
432
|
+
async checkAndIncrementRateLimit(queue, max, windowMs, now) {
|
|
433
|
+
/* ... */
|
|
434
|
+
}
|
|
402
435
|
}
|
|
403
436
|
|
|
404
437
|
const client = QueueClient.withAdapter(new MongoStorageAdapter(), {
|
|
@@ -414,19 +447,19 @@ await client.init();
|
|
|
414
447
|
|
|
415
448
|
### `new QueueClient(options?)`
|
|
416
449
|
|
|
417
|
-
| Option
|
|
418
|
-
|
|
419
|
-
| `dialect`
|
|
420
|
-
| `connectionString`
|
|
421
|
-
| `defaults.attempts`
|
|
422
|
-
| `defaults.retryDelay`
|
|
423
|
-
| `defaults.backoff`
|
|
424
|
-
| `defaults.timeout`
|
|
425
|
-
| `defaults.concurrency`
|
|
426
|
-
| `defaults.pollInterval`
|
|
427
|
-
| `defaults.stalledInterval` | `number`
|
|
428
|
-
| `defaults.lockDuration`
|
|
429
|
-
| `defaults.rateLimit`
|
|
450
|
+
| Option | Type | Default | Description |
|
|
451
|
+
| -------------------------- | ---------------------------------------------- | --------------- | ----------------------------------- |
|
|
452
|
+
| `dialect` | `"memory" \| "redis" \| "postgres" \| "mysql"` | `"memory"` | Storage backend |
|
|
453
|
+
| `connectionString` | `string` | — | Required for Redis/PostgreSQL/MySQL |
|
|
454
|
+
| `defaults.attempts` | `number` | `3` | Max retries per job |
|
|
455
|
+
| `defaults.retryDelay` | `number` | `1000` | Base retry delay in ms |
|
|
456
|
+
| `defaults.backoff` | `"fixed" \| "linear" \| "exponential"` | `"exponential"` | Retry strategy |
|
|
457
|
+
| `defaults.timeout` | `number` | `30000` | Per-attempt timeout in ms |
|
|
458
|
+
| `defaults.concurrency` | `number` | `10` | Default worker concurrency |
|
|
459
|
+
| `defaults.pollInterval` | `number` | `1000` | Poll interval in ms |
|
|
460
|
+
| `defaults.stalledInterval` | `number` | `30000` | Stalled-job check interval in ms |
|
|
461
|
+
| `defaults.lockDuration` | `number` | `60000` | Lock TTL in ms |
|
|
462
|
+
| `defaults.rateLimit` | `{ max, duration }` | — | Optional rate limiting |
|
|
430
463
|
|
|
431
464
|
### `client.init()`
|
|
432
465
|
|
|
@@ -454,16 +487,16 @@ Creates a client using a custom storage backend.
|
|
|
454
487
|
|
|
455
488
|
### `queue.enqueue(type, payload, options?)`
|
|
456
489
|
|
|
457
|
-
| Option
|
|
458
|
-
|
|
459
|
-
| `attempts`
|
|
460
|
-
| `retryDelay`
|
|
461
|
-
| `backoff`
|
|
462
|
-
| `timeout`
|
|
463
|
-
| `priority`
|
|
464
|
-
| `schedule.delay` | `number`
|
|
465
|
-
| `schedule.runAt` | `string \| number` | Absolute run time
|
|
466
|
-
| `schedule.cron`
|
|
490
|
+
| Option | Type | Description |
|
|
491
|
+
| ---------------- | ------------------ | ------------------------------------- |
|
|
492
|
+
| `attempts` | `number` | Maximum attempts for this job |
|
|
493
|
+
| `retryDelay` | `number` | Base retry delay in ms |
|
|
494
|
+
| `backoff` | `string` | Retry backoff strategy |
|
|
495
|
+
| `timeout` | `number` | Per-attempt timeout in ms |
|
|
496
|
+
| `priority` | `number` | Higher values are processed first |
|
|
497
|
+
| `schedule.delay` | `number` | Delay before the job becomes eligible |
|
|
498
|
+
| `schedule.runAt` | `string \| number` | Absolute run time |
|
|
499
|
+
| `schedule.cron` | `string` | Cron expression for recurring jobs |
|
|
467
500
|
|
|
468
501
|
### `queue.process(type, processor)`
|
|
469
502
|
|
|
@@ -471,30 +504,31 @@ Registers an async processor for a job type. The processor signature is `async (
|
|
|
471
504
|
|
|
472
505
|
### `queue.createWorker(options?)`
|
|
473
506
|
|
|
474
|
-
| Option
|
|
475
|
-
|
|
476
|
-
| `concurrency`
|
|
477
|
-
| `shutdownTimeout` | `number` | `30000`
|
|
507
|
+
| Option | Type | Default | Description |
|
|
508
|
+
| ----------------- | -------- | ------------ | ----------------------------------- |
|
|
509
|
+
| `concurrency` | `number` | queue config | Maximum simultaneous job executions |
|
|
510
|
+
| `shutdownTimeout` | `number` | `30000` | Graceful shutdown wait time in ms |
|
|
478
511
|
|
|
479
512
|
### `queue.getJob(id)` / `queue.getJobs(status?, limit?, offset?)`
|
|
513
|
+
|
|
480
514
|
### `queue.getJobCounts()`
|
|
481
515
|
|
|
482
516
|
---
|
|
483
517
|
|
|
484
518
|
## Storage Support Matrix
|
|
485
519
|
|
|
486
|
-
| Feature
|
|
487
|
-
|
|
488
|
-
| Persistence
|
|
489
|
-
| Atomic claim
|
|
490
|
-
| Priority ordering
|
|
491
|
-
| Delayed jobs
|
|
492
|
-
| Retry + backoff
|
|
493
|
-
| DLQ
|
|
494
|
-
| Stalled recovery
|
|
495
|
-
| Rate limiting
|
|
496
|
-
| Connection check on init |
|
|
497
|
-
| Auto-create schema
|
|
520
|
+
| Feature | Memory | Redis | PostgreSQL | MySQL |
|
|
521
|
+
| ------------------------ | :----: | :-----: | :-------------: | :-------------: |
|
|
522
|
+
| Persistence | — | ✓ | ✓ | ✓ |
|
|
523
|
+
| Atomic claim | ✓ | ✓ (Lua) | ✓ (SKIP LOCKED) | ✓ (SKIP LOCKED) |
|
|
524
|
+
| Priority ordering | ✓ | ✓ | ✓ | ✓ |
|
|
525
|
+
| Delayed jobs | ✓ | ✓ | ✓ | ✓ |
|
|
526
|
+
| Retry + backoff | ✓ | ✓ | ✓ | ✓ |
|
|
527
|
+
| DLQ | ✓ | ✓ | ✓ | ✓ |
|
|
528
|
+
| Stalled recovery | ✓ | ✓ | ✓ | ✓ |
|
|
529
|
+
| Rate limiting | ✓ | ✓ | ✓ | ✓ |
|
|
530
|
+
| Connection check on init | — | ✓ PING | ✓ SELECT 1 | ✓ SELECT 1 |
|
|
531
|
+
| Auto-create schema | — | — | ✓ | ✓ |
|
|
498
532
|
|
|
499
533
|
---
|
|
500
534
|
|
|
Binary file
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"client.d.ts","sourceRoot":"","sources":["../../src/core/client.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;GAsBG;AAEH,OAAO,KAAK,EAAE,kBAAkB,EAAkB,MAAM,0BAA0B,CAAC;AACnF,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,yBAAyB,CAAC;AAC5D,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,2BAA2B,CAAC;AAChE,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,0BAA0B,CAAC;AAC5D,OAAO,EAAE,KAAK,EAAE,MAAM,YAAY,CAAC;AAwBnC,qBAAa,WAAW;IACtB,0DAA0D;IAC1D,QAAQ,EAAE,cAAc,CAAC;IAEzB,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAoB;IAC5C,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAA2B;IACpD,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAS;IACjC,OAAO,CAAC,QAAQ,CAAC,gBAAgB,CAAqB;IACtD,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAqC;IAC5D,OAAO,CAAC,WAAW,CAAS;IAC5B,OAAO,CAAC,MAAM,CAAS;IACvB,8GAA8G;IAC9G,OAAO,CAAC,cAAc,CAAS;gBAEnB,OAAO,GAAE,kBAAuB;
|
|
1
|
+
{"version":3,"file":"client.d.ts","sourceRoot":"","sources":["../../src/core/client.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;GAsBG;AAEH,OAAO,KAAK,EAAE,kBAAkB,EAAkB,MAAM,0BAA0B,CAAC;AACnF,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,yBAAyB,CAAC;AAC5D,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,2BAA2B,CAAC;AAChE,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,0BAA0B,CAAC;AAC5D,OAAO,EAAE,KAAK,EAAE,MAAM,YAAY,CAAC;AAwBnC,qBAAa,WAAW;IACtB,0DAA0D;IAC1D,QAAQ,EAAE,cAAc,CAAC;IAEzB,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAoB;IAC5C,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAA2B;IACpD,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAS;IACjC,OAAO,CAAC,QAAQ,CAAC,gBAAgB,CAAqB;IACtD,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAqC;IAC5D,OAAO,CAAC,WAAW,CAAS;IAC5B,OAAO,CAAC,MAAM,CAAS;IACvB,8GAA8G;IAC9G,OAAO,CAAC,cAAc,CAAS;gBAEnB,OAAO,GAAE,kBAAuB;IAkB5C,OAAO,CAAC,yBAAyB;IAajC;;;;;;;;;;;;;OAaG;IACG,IAAI,IAAI,OAAO,CAAC,IAAI,CAAC;IAY3B,gCAAgC;IAC1B,UAAU,IAAI,OAAO,CAAC,IAAI,CAAC;YAInB,cAAc;IAkC5B;;;;;OAKG;IACH,WAAW,CAAC,QAAQ,GAAG,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,YAAY,GAAG,KAAK,CAAC,QAAQ,CAAC;IAwBtF,6DAA6D;IAC7D,QAAQ,CAAC,QAAQ,GAAG,OAAO,EAAE,IAAI,EAAE,MAAM,GAAG,KAAK,CAAC,QAAQ,CAAC,GAAG,SAAS;IAIvE,wDAAwD;IACxD,YAAY,CAAC,QAAQ,GAAG,OAAO,EAAE,IAAI,EAAE,MAAM,GAAG,KAAK,CAAC,QAAQ,CAAC;IAQ/D,qDAAqD;IACrD,IAAI,UAAU,IAAI,MAAM,EAAE,CAEzB;IAED,wDAAwD;IACxD,IAAI,aAAa,IAAI,OAAO,CAE3B;IAMD,EAAE,CAAC,CAAC,SAAS,MAAM,WAAW,EAAE,KAAK,EAAE,CAAC,EAAE,QAAQ,EAAE,CAAC,GAAG,IAAI,EAAE,WAAW,CAAC,CAAC,CAAC,KAAK,IAAI,GAAG,IAAI;IAK5F,IAAI,CAAC,CAAC,SAAS,MAAM,WAAW,EAAE,KAAK,EAAE,CAAC,EAAE,QAAQ,EAAE,CAAC,GAAG,IAAI,EAAE,WAAW,CAAC,CAAC,CAAC,KAAK,IAAI,GAAG,IAAI;IAK9F,GAAG,CAAC,CAAC,SAAS,MAAM,WAAW,EAAE,KAAK,EAAE,CAAC,EAAE,QAAQ,EAAE,CAAC,GAAG,IAAI,EAAE,WAAW,CAAC,CAAC,CAAC,KAAK,IAAI,GAAG,IAAI;IAS7F;;;OAGG;IACG,KAAK,IAAI,OAAO,CAAC,IAAI,CAAC;IAa5B;;;;;;;;OAQG;IACH,MAAM,CAAC,WAAW,CAChB,OAAO,EAAE,cAAc,EACvB,OAAO,GAAE,IAAI,CAAC,kBAAkB,EAAE,SAAS,GAAG,kBAAkB,CAAM,GACrE,WAAW;CAOf"}
|
package/dist/core/queue.d.ts
CHANGED
|
@@ -26,11 +26,13 @@ export declare class Queue<TPayload = unknown> {
|
|
|
26
26
|
private readonly emitter;
|
|
27
27
|
private readonly resolvedConfig;
|
|
28
28
|
private readonly clientDefaults;
|
|
29
|
+
private readonly isInitialisedCheck?;
|
|
29
30
|
/** Registered processors keyed by job type. */
|
|
30
31
|
private readonly processors;
|
|
31
32
|
/** Active worker instances created by this queue. */
|
|
32
33
|
private readonly workers;
|
|
33
|
-
constructor(name: string, storage: StorageAdapter, emitter: QueueEventEmitter, options: QueueOptions | undefined, defaults: ResolvedDefaults);
|
|
34
|
+
constructor(name: string, storage: StorageAdapter, emitter: QueueEventEmitter, options: QueueOptions | undefined, defaults: ResolvedDefaults, isInitialisedCheck?: (() => boolean) | undefined);
|
|
35
|
+
private assertInitialised;
|
|
34
36
|
/**
|
|
35
37
|
* Add a new job to the queue.
|
|
36
38
|
*
|
package/dist/core/queue.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"queue.d.ts","sourceRoot":"","sources":["../../src/core/queue.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;GAWG;AAEH,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,2BAA2B,CAAC;AAChE,OAAO,KAAK,EAAE,UAAU,EAAE,SAAS,EAAE,MAAM,uBAAuB,CAAC;AACnE,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,yBAAyB,CAAC;AAC5D,OAAO,KAAK,EAAE,aAAa,EAAE,SAAS,EAAE,MAAM,0BAA0B,CAAC;AACzE,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,0BAA0B,CAAC;AAC/D,OAAO,EAAE,GAAG,EAAE,MAAM,UAAU,CAAC;AAC/B,OAAO,EAAE,MAAM,EAAE,MAAM,aAAa,CAAC;AAErC,OAAO,KAAK,EAAE,iBAAiB,EAAE,MAAM,sBAAsB,CAAC;AAE9D,KAAK,gBAAgB,GAAG,QAAQ,CAAC,cAAc,CAAC,CAAC;AA2BjD,qBAAa,KAAK,CAAC,QAAQ,GAAG,OAAO;IACnC,4DAA4D;IAC5D,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IAEtB,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAiB;IACzC,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAoB;IAC5C,OAAO,CAAC,QAAQ,CAAC,cAAc,CAAyB;IACxD,OAAO,CAAC,QAAQ,CAAC,cAAc,CAAmB;
|
|
1
|
+
{"version":3,"file":"queue.d.ts","sourceRoot":"","sources":["../../src/core/queue.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;GAWG;AAEH,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,2BAA2B,CAAC;AAChE,OAAO,KAAK,EAAE,UAAU,EAAE,SAAS,EAAE,MAAM,uBAAuB,CAAC;AACnE,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,yBAAyB,CAAC;AAC5D,OAAO,KAAK,EAAE,aAAa,EAAE,SAAS,EAAE,MAAM,0BAA0B,CAAC;AACzE,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,0BAA0B,CAAC;AAC/D,OAAO,EAAE,GAAG,EAAE,MAAM,UAAU,CAAC;AAC/B,OAAO,EAAE,MAAM,EAAE,MAAM,aAAa,CAAC;AAErC,OAAO,KAAK,EAAE,iBAAiB,EAAE,MAAM,sBAAsB,CAAC;AAE9D,KAAK,gBAAgB,GAAG,QAAQ,CAAC,cAAc,CAAC,CAAC;AA2BjD,qBAAa,KAAK,CAAC,QAAQ,GAAG,OAAO;IACnC,4DAA4D;IAC5D,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IAEtB,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAiB;IACzC,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAoB;IAC5C,OAAO,CAAC,QAAQ,CAAC,cAAc,CAAyB;IACxD,OAAO,CAAC,QAAQ,CAAC,cAAc,CAAmB;IAClD,OAAO,CAAC,QAAQ,CAAC,kBAAkB,CAAC,CAA8B;IAElE,+CAA+C;IAC/C,OAAO,CAAC,QAAQ,CAAC,UAAU,CAAyC;IAEpE,qDAAqD;IACrD,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAgB;gBAGtC,IAAI,EAAE,MAAM,EACZ,OAAO,EAAE,cAAc,EACvB,OAAO,EAAE,iBAAiB,EAC1B,OAAO,EAAE,YAAY,GAAG,SAAS,EACjC,QAAQ,EAAE,gBAAgB,EAC1B,kBAAkB,CAAC,EAAE,CAAC,MAAM,OAAO,CAAC,GAAG,SAAS;IAUlD,OAAO,CAAC,iBAAiB;IAYzB;;;;;;OAMG;IACG,OAAO,CAAC,IAAI,EAAE,MAAM,EAAE,OAAO,EAAE,QAAQ,EAAE,OAAO,CAAC,EAAE,UAAU,GAAG,OAAO,CAAC,GAAG,CAAC,QAAQ,CAAC,CAAC;IAsC5F;;;;;OAKG;IACH,OAAO,CAAC,CAAC,GAAG,QAAQ,EAAE,IAAI,EAAE,MAAM,EAAE,SAAS,EAAE,SAAS,CAAC,CAAC,CAAC,GAAG,IAAI;IAQlE;;;;OAIG;IACH,YAAY,CAAC,OAAO,CAAC,EAAE,aAAa,GAAG,MAAM;IAqB7C,8EAA8E;IACxE,MAAM,CAAC,KAAK,EAAE,MAAM,GAAG,OAAO,CAAC,GAAG,CAAC,QAAQ,CAAC,GAAG,IAAI,CAAC;IAO1D,iEAAiE;IAC3D,OAAO,CAAC,MAAM,CAAC,EAAE,SAAS,EAAE,KAAK,SAAM,EAAE,MAAM,SAAI,GAAG,OAAO,CAAC,GAAG,CAAC,QAAQ,CAAC,EAAE,CAAC;IAWpF,+CAA+C;IACzC,YAAY,IAAI,OAAO,CAAC,MAAM,CAAC,SAAS,EAAE,MAAM,CAAC,CAAC;IASxD;;;OAGG;IACG,KAAK,IAAI,OAAO,CAAC,IAAI,CAAC;IAI5B,wDAAwD;IACxD,UAAU,IAAI,SAAS,MAAM,EAAE;CAGhC"}
|