bunqueue 2.8.60 → 2.9.0

package/README.md

@@ -13,8 +13,8 @@
 </p>
 
 <p align="center">
-  High-performance job queue for Bun. Built for AI agents and automation.<br/>
-  Zero external infrastructure. MCP-native. TypeScript-first.
+  High-performance job queue for Bun. Memory or one-file SQLite; PostgreSQL 15–18 multi-broker when you scale.<br/>
+  DLQ, cron, SQLite S3 backups, and native MCP. Built for AI agents and automation. No Redis.
 </p>
 
 <p align="center">
@@ -48,14 +48,13 @@ await app.add('send', { to: 'alice@example.com' });
 ```
 
 That's it. Queue + Worker in one object, persisted to a single SQLite file.
-No Redis, no config, no setup. The install is 5.5 MB, 7 packages, 2 runtime
-dependencies (`croner` + `msgpackr`) — SQLite, S3, HTTP and WebSocket are Bun
-built-ins.
+No Redis, no config, no setup. `msgpackr` is the only runtime dependency;
+cron, SQLite, S3, HTTP and WebSocket use Bun's built-ins.
 
 ### Not on Bun? Run the server, connect from anywhere
 
-The queue also runs as a standalone server — one command, nothing else to
-operate:
+The queue also runs as a standalone server. Memory is the zero-configuration
+default; SQLite is the zero-infrastructure persistent option:
 
 ```bash
 # in-memory without --data-path; pass it to persist jobs to SQLite
@@ -67,8 +66,24 @@ docker run -d -p 6789:6789 -p 6790:6790 \
   ghcr.io/egeominotti/bunqueue:latest
 ```
 
+For multiple active brokers, the repository includes a topology pinned to
+PostgreSQL 18.6:
+
+```bash
+POSTGRES_PASSWORD='replace-me' \
+  BUNQUEUE_POSTGRES_URL='postgres://bunqueue:replace-me@postgres:5432/bunqueue' \
+  docker compose -f docker-compose.postgres.yml up --build -d
+```
+
+It starts two brokers against one database/namespace. PostgreSQL is server-only;
+embedded mode keeps using memory/SQLite. Supply both Compose values when the
+password changes, percent-encoding reserved characters in the URL only. MySQL
+is not supported. CI validates PostgreSQL 15, 16, 17, and the pinned/recommended
+18.6 release. See the
+[storage guide](https://bunqueue.dev/guide/databases/).
+
 Release images also carry the exact package version. For reproducible
-deployments, pin `ghcr.io/egeominotti/bunqueue:2.8.60`; `latest` points to the
+deployments, pin `ghcr.io/egeominotti/bunqueue:2.9.0`; `latest` points to the
 same multi-arch image at release time.
 
 Then produce and process from the language you already use:
@@ -80,7 +95,7 @@ npm install bunqueue-client    # Node.js ≥ 20, Deno ≥ 2, Bun, Cloudflare Wor
 ```typescript
 import { Queue, Worker } from 'bunqueue-client';
 
-const queue = new Queue('emails');                       // localhost:6789 by default
+const queue = new Queue('emails'); // localhost:6789 by default
 await queue.add('welcome', { to: 'user@example.com' });
 
 new Worker('emails', async (job) => ({ sent: true }), { concurrency: 10 });
@@ -96,14 +111,18 @@ Python, PHP, Go, Rust and Elixir clients speak the same protocol — see
 
 ## Why bunqueue?
 
-| Library      | Requires    | AI-native |
-| ------------ | ----------- | --------- |
-| BullMQ       | Redis       | No        |
-| Agenda       | MongoDB     | No        |
-| pg-boss      | PostgreSQL  | No        |
-| **bunqueue** | **Nothing** | **Yes**   |
-
-- **Zero external infrastructure** — one process, one SQLite file. `cp` to back up
+| Library      | Requires                                   | AI-native |
+| ------------ | ------------------------------------------ | --------- |
+| BullMQ       | Redis                                      | No        |
+| Agenda       | MongoDB                                    | No        |
+| pg-boss      | PostgreSQL                                 | No        |
+| **bunqueue** | **Nothing (memory/SQLite) · PostgreSQL optional** | **Yes**   |
+
+- **Zero external infrastructure by default** — memory by default; one SQLite
+  file when local persistence is configured
+- **PostgreSQL 15–18 multi-broker mode** — PostgreSQL 18.6 is recommended;
+  database-authoritative claims, fenced
+  leases, shared limits, cron, workers, job-state/lifecycle metrics, and failover state
 - **BullMQ-compatible API** — same `Queue`, `Worker`, `QueueEvents`; [migrating takes minutes](https://bunqueue.dev/guide/migration/)
 - **MCP server included** — 73 tools; AI agents get full queue control out of the box
 - **Everything server-side** — retries with backoff, priorities, cron, rate limits, dead letter queue
@@ -111,24 +130,24 @@ Python, PHP, Go, Rust and Elixir clients speak the same protocol — see
   in-memory batch push, 186K jobs/sec public on-disk Embedded `addBulk`, and
   159K jobs/sec TCP `PUSHB`; [methodology and distributions](https://bunqueue.dev/guide/benchmarks/)
 
-**Great for:** single-server deployments, AI agents that need a scheduler,
-prototypes and MVPs, embedded use cases (CLI tools, edge, serverless), teams
+**Great for:** embedded and single-server deployments, PostgreSQL-backed broker
+fleets, AI agents that need a scheduler, edge/serverless spooling, and teams
 that don't want to operate Redis.
 
-**Not ideal for:** multi-region distributed systems requiring HA or automatic
-failover today. If you already run Redis and BullMQ works for you, keep it.
+**Not ideal for:** multi-region consensus or deployments that require MySQL as
+the queue store. If you already run Redis and BullMQ works for you, keep it.
 
 [When to choose bunqueue →](https://bunqueue.dev/guide/comparison/)
 
 ## Two Modes
 
-|                  | Embedded                              | Server (TCP)                                 |
-| ---------------- | ------------------------------------- | -------------------------------------------- |
-| **How it works** | Queue runs inside your process        | Standalone server, clients connect via TCP   |
-| **Setup**        | `bun add bunqueue`                    | `docker run` or `bunqueue start`             |
-| **Performance**  | 186K jobs/sec on-disk `addBulk`; 729K internal in-memory batch | 159K jobs/sec TCP `PUSHB`; 17K jobs/sec worker drain |
-| **Best for**     | Single-process apps, CLIs, serverless | Multiple workers, separate producer/consumer |
-| **Scaling**      | Same process only                     | Multiple clients across machines             |
+|                  | Embedded                                                       | Server (TCP)                                             |
+| ---------------- | -------------------------------------------------------------- | -------------------------------------------------------- |
+| **How it works** | Queue runs inside your process                                 | Standalone server, clients connect via TCP               |
+| **Setup**        | `bun add bunqueue`                                             | `docker run` or `bunqueue start`                         |
+| **Performance**  | 186K jobs/sec on-disk `addBulk`; 729K internal in-memory batch | SQLite: 159K TCP `PUSHB`; 17K jobs/sec worker drain      |
+| **Best for**     | Single-process apps, CLIs, serverless                          | Multiple workers, separate producer/consumer             |
+| **Scaling**      | Same process only                                              | Multiple clients; multiple brokers with PostgreSQL 15–18 |
 
 ### Embedded
 
@@ -180,18 +199,20 @@ await queue.add('process', { data: 'hello' });
 
 ## One Queue, Any Language (SDKs)
 
-The server does all the heavy lifting. Official client SDKs speak the native
-TCP protocol with full feature parity, so producers and workers can live
-anywhere in your stack — add a job from TypeScript, process it from Python:
-
-| Where your code runs | Install |
-| -------------------- | ------- |
-| **Node.js ≥ 20, Deno ≥ 2, Bun, Cloudflare Workers** | [`npm install bunqueue-client`](https://www.npmjs.com/package/bunqueue-client) |
-| **Python ≥ 3.9** | [`pip install bunqueue-client`](https://pypi.org/project/bunqueue-client/) |
-| **PHP ≥ 8.1** | [`composer require bunqueue/client`](https://packagist.org/packages/bunqueue/client) |
-| **Go ≥ 1.26.5** | `go get github.com/egeominotti/bunqueue/sdk/go` |
-| **Rust ≥ 1.85** | [`cargo add bunqueue-client`](https://crates.io/crates/bunqueue-client) |
-| **Elixir ≥ 1.15** | Hex coming soon — today: use [`sdk/elixir`](./sdk/elixir) as a path dependency |
+The server does all the heavy lifting. Official client SDKs share the
+protocol-conformant core Queue, Worker, and Flow surface, so producers and
+workers can live anywhere in your stack — add a job from TypeScript, process it
+from Python. Language-specific capabilities are tracked in the
+[SDK guide](https://bunqueue.dev/guide/sdks/).
+
+| Where your code runs                                | Install                                                                                |
+| --------------------------------------------------- | -------------------------------------------------------------------------------------- |
+| **Node.js ≥ 20, Deno ≥ 2, Bun, Cloudflare Workers** | [`npm install bunqueue-client`](https://www.npmjs.com/package/bunqueue-client)         |
+| **Python ≥ 3.9**                                    | [`pip install bunqueue-client`](https://pypi.org/project/bunqueue-client/)             |
+| **PHP ≥ 8.1**                                       | [`composer require bunqueue/client`](https://packagist.org/packages/bunqueue/client)   |
+| **Go ≥ 1.26.5**                                     | `go get github.com/egeominotti/bunqueue/sdk/go`                                        |
+| **Rust ≥ 1.85**                                     | [`cargo add bunqueue-client`](https://crates.io/crates/bunqueue-client)                |
+| **Elixir ≥ 1.15**                                   | Hex coming soon — today: use [`sdk/elixir`](./sdk/elixir) as a path dependency         |
 
 ```typescript
 // Node.js / Deno / Cloudflare Workers
@@ -237,21 +258,24 @@ const root = await flows.add({
   ],
 });
 
-console.log(root.job.id, root.children?.map(({ job }) => job.id));
+console.log(
+  root.job.id,
+  root.children?.map(({ job }) => job.id)
+);
 await flows.close();
 ```
 
 The repository records the contracts and the test strategy beside each
 implementation:
 
-| SDK | Runtime invariants | Generated tests | Mutation engine |
-| --- | --- | --- | --- |
-| [TypeScript](./sdk/typescript/README.md) | [contract](./sdk/typescript/INVARIANTS.md) | fast-check | none¹ |
-| [Python](./sdk/python/README.md) | [contract](./sdk/python/INVARIANTS.md) | Hypothesis | mutmut |
-| [PHP](./sdk/php/README.md) | [contract](./sdk/php/INVARIANTS.md) | Eris | Infection |
-| [Go](./sdk/go/README.md) | [contract](./sdk/go/INVARIANTS.md) | Rapid | Gremlins |
-| [Rust](./sdk/rust/README.md) | [contract](./sdk/rust/INVARIANTS.md) | proptest | cargo-mutants |
-| [Elixir](./sdk/elixir/README.md) | [contract](./sdk/elixir/INVARIANTS.md) | StreamData | Muex |
+| SDK                                      | Runtime invariants                         | Generated tests | Mutation engine |
+| ---------------------------------------- | ------------------------------------------ | --------------- | --------------- |
+| [TypeScript](./sdk/typescript/README.md) | [contract](./sdk/typescript/INVARIANTS.md) | fast-check      | none¹           |
+| [Python](./sdk/python/README.md)         | [contract](./sdk/python/INVARIANTS.md)     | Hypothesis      | mutmut          |
+| [PHP](./sdk/php/README.md)               | [contract](./sdk/php/INVARIANTS.md)        | Eris            | Infection       |
+| [Go](./sdk/go/README.md)                 | [contract](./sdk/go/INVARIANTS.md)         | Rapid           | Gremlins        |
+| [Rust](./sdk/rust/README.md)             | [contract](./sdk/rust/INVARIANTS.md)       | proptest        | cargo-mutants   |
+| [Elixir](./sdk/elixir/README.md)         | [contract](./sdk/elixir/INVARIANTS.md)     | StreamData      | Muex            |
 
 ¹ The TypeScript SDK has no mutation engine. StrykerJS was removed because its
 dependency graph produced every advisory the weekly audit had to answer for,
@@ -314,17 +338,25 @@ human-in-the-loop signals — built on bunqueue, no new infrastructure:
 import { Workflow, Engine } from 'bunqueue/workflow';
 
 const orderFlow = new Workflow('order-pipeline')
-  .step('reserve-stock', async () => {
-    await inventory.reserve();
-    return { reserved: true };
-  }, {
-    compensate: async () => await inventory.release(), // auto-rollback on failure
-  })
-  .step('charge', async () => {
-    return { txId: await payments.charge() };
-  }, {
-    compensate: async () => await payments.refund(),
-  })
+  .step(
+    'reserve-stock',
+    async () => {
+      await inventory.reserve();
+      return { reserved: true };
+    },
+    {
+      compensate: async () => await inventory.release(), // auto-rollback on failure
+    }
+  )
+  .step(
+    'charge',
+    async () => {
+      return { txId: await payments.charge() };
+    },
+    {
+      compensate: async () => await payments.refund(),
+    }
+  )
   .waitFor('manager-approval', { timeout: 86_400_000 }) // human-in-the-loop
   .step('confirm', async (ctx) => {
     return { txId: (ctx.steps['charge'] as { txId: string }).txId };
@@ -336,13 +368,13 @@ const run = await engine.start('order-pipeline', { orderId: 'ORD-1' });
 await engine.signal(run.id, 'manager-approval', { approved: true });
 ```
 
-| | **bunqueue** | **Temporal** | **Inngest** | **Trigger.dev** |
-|---|---|---|---|---|
-| **Infrastructure** | None (embedded) | PostgreSQL + 7 services | Cloud-only | Redis + PostgreSQL |
-| **Saga compensation** | Built-in | Manual | Manual | Manual |
-| **Human-in-the-loop** | `.waitFor()` | Signals API | `step.waitForEvent()` | Waitpoint tokens |
-| **Self-hosted** | Zero-config | Complex | No | Complex |
-| **Pricing** | Free (MIT) | Free / Cloud $$ | Per-execution | Free tier, then $50/mo+ |
+|                       | **bunqueue**    | **Temporal**            | **Inngest**           | **Trigger.dev**         |
+| --------------------- | --------------- | ----------------------- | --------------------- | ----------------------- |
+| **Infrastructure**    | None (embedded) | PostgreSQL + 7 services | Cloud-only            | Redis + PostgreSQL      |
+| **Saga compensation** | Built-in        | Manual                  | Manual                | Manual                  |
+| **Human-in-the-loop** | `.waitFor()`    | Signals API             | `step.waitForEvent()` | Waitpoint tokens        |
+| **Self-hosted**       | Zero-config     | Complex                 | No                    | Complex                 |
+| **Pricing**           | Free (MIT)      | Free / Cloud $$         | Per-execution         | Free tier, then $50/mo+ |
 
 Also included: nested workflows, `doUntil`/`doWhile` loops, `forEach` over
 dynamic lists, schema validation (Zod, ArkType, Valibot or any `.parse()`),
@@ -374,8 +406,8 @@ claude mcp add bunqueue -- bunx bunqueue-mcp
 }
 ```
 
-Then just ask: *"Schedule a cleanup job every day at 3 AM"* · *"Show me all
-failed jobs and retry them"* · *"Set rate limit to 50/sec on api-calls"*.
+Then just ask: _"Schedule a cleanup job every day at 3 AM"_ · _"Show me all
+failed jobs and retry them"_ · _"Set rate limit to 50/sec on api-calls"_.
 
 [MCP guide →](https://bunqueue.dev/guide/mcp/)
 
@@ -399,13 +431,13 @@ https://github.com/user-attachments/assets/e8a8d38e-b4a6-4dc8-8360-876c0f24d116
 
 Native Ryzen 9 9950X3D, Bun 1.3.14; medians from repeated fresh processes:
 
-| Workload | Mode | Median | Persistence |
-| --- | --- | ---: | --- |
-| Internal batched push, 1M jobs | Embedded | 729,395 jobs/sec | In-memory, no `dataPath` |
-| Public sustained `addBulk`, 50K cell | Embedded | 186,384 jobs/sec | On-disk buffered SQLite |
-| `PUSHB`, fresh 50K sample | TCP | 158,779 jobs/sec | On-disk buffered SQLite |
-| No-work worker drain, concurrency 50 | TCP | 17,256 jobs/sec | Full pull/process/ACK |
-| Linear Workflow Engine | Embedded / TCP | 2,700 / 3,187 workflows/sec | Workflow SQLite + 3 queue nodes |
+| Workload                             | Mode           |                      Median | Persistence                     |
+| ------------------------------------ | -------------- | --------------------------: | ------------------------------- |
+| Internal batched push, 1M jobs       | Embedded       |            729,395 jobs/sec | In-memory, no `dataPath`        |
+| Public sustained `addBulk`, 50K cell | Embedded       |            186,384 jobs/sec | On-disk buffered SQLite         |
+| `PUSHB`, fresh 50K sample            | TCP            |            158,779 jobs/sec | On-disk buffered SQLite         |
+| No-work worker drain, concurrency 50 | TCP            |             17,256 jobs/sec | Full pull/process/ACK           |
+| Linear Workflow Engine               | Embedded / TCP | 2,700 / 3,187 workflows/sec | Workflow SQLite + 3 queue nodes |
 
 These operations do different work; the internal in-memory result is not an
 SQLite or public-API claim. Run `bun run bench`, `bun run bench:tcp`, or

package/dist/application/background/dlq.js

@@ -5,6 +5,8 @@ export function performDlqMaintenance(ctx) {
         shards: ctx.shards,
         jobIndex: ctx.jobIndex,
         jobResults: ctx.jobResults,
+        dependencyResults: ctx.dependencyResults,
+        customIdMap: ctx.customIdMap,
         jobLogs: ctx.jobLogs,
         storage: ctx.storage,
     };

package/dist/application/contextFactory.js

@@ -161,6 +161,7 @@ export class ContextFactory {
             jobLocks: this.deps.jobLocks,
             clientJobs: this.deps.clientJobs,
             dependencyResults: this.deps.dependencyResults,
+            customIdMap: this.deps.customIdMap,
             depCompletions: this.deps.depCompletions,
             maxDependencyCompletions: this.deps.config.maxCompletedJobs,
             webhookManager: this.deps.webhookManager,
@@ -188,6 +189,8 @@ export class ContextFactory {
             shards: this.deps.shards,
             jobIndex: this.deps.jobIndex,
             jobResults: this.deps.jobResults,
+            dependencyResults: this.deps.dependencyResults,
+            customIdMap: this.deps.customIdMap,
             jobLogs: this.deps.jobLogs,
             storage: this.deps.storage,
         };
@@ -201,6 +204,8 @@ export class ContextFactory {
             completedJobs: this.deps.completedJobs,
             completedJobsData: this.deps.completedJobsData,
             jobResults: this.deps.jobResults,
+            dependencyResults: this.deps.dependencyResults,
+            customIdMap: this.deps.customIdMap,
         };
     }
     getLogsContext() {

package/dist/application/dlqManager.d.ts

@@ -14,6 +14,14 @@ export interface DlqContext {
     jobResults: {
         delete(id: JobId): boolean;
     };
+    dependencyResults: {
+        releaseConsumer(id: JobId): void;
+    };
+    customIdMap: {
+        get(id: string): JobId | undefined;
+        set(id: string, jobId: JobId): void;
+        delete(id: string): boolean;
+    };
     jobLogs: {
         delete(id: JobId): boolean;
     };
@@ -30,6 +38,8 @@ export declare function getDlqStats(queue: string, ctx: DlqContext): DlqStats;
 export declare function purgeExpiredDlq(queue: string, ctx: DlqContext): number;
 /** Purge all jobs from DLQ */
 export declare function purgeDlqJobs(queue: string, ctx: DlqContext): number;
+/** Remove one DLQ entry without re-enqueuing it. */
+export declare function removeDlqJob(queue: string, id: JobId, ctx: DlqContext): boolean;
 /** Configure DLQ for a queue */
 export declare function configureDlq(queue: string, ctx: DlqContext, config: Partial<DlqConfig>): void;
 /** Get DLQ configuration */

package/dist/application/dlqManager.js

@@ -83,6 +83,40 @@ export function purgeDlqJobs(queue, ctx) {
     shard.clearDlq(queue);
     return entries.length;
 }
+/** Remove one DLQ entry without re-enqueuing it. */
+export function removeDlqJob(queue, id, ctx) {
+    const shard = ctx.shards[shardIndex(queue)];
+    const entries = shard.getDlqEntries(queue).filter((entry) => entry.job.id === id);
+    if (entries.length === 0)
+        return false;
+    const location = ctx.jobIndex.get(id);
+    const terminalJobIds = location === undefined || (location.type === 'dlq' && location.queueName === queue) ? [id] : [];
+    // Keep the in-memory entry visible until the durable transaction succeeds.
+    // This prevents a failed SQLite write from leaving memory and disk out of sync.
+    ctx.storage?.purgeDlqEntries(queue, [id], terminalJobIds, false);
+    shard.removeAllFromDlq(queue, id);
+    cleanupTerminalEntries(queue, entries, terminalJobIds, ctx);
+    return true;
+}
+function cleanupTerminalEntries(queue, entries, terminalJobIds, ctx) {
+    const terminalIds = new Set(terminalJobIds);
+    for (const entry of entries) {
+        if (!terminalIds.has(entry.job.id))
+            continue;
+        const customId = entry.job.customId;
+        if (customId && ctx.customIdMap.get(customId) === entry.job.id) {
+            ctx.customIdMap.delete(customId);
+        }
+    }
+    for (const jobId of terminalJobIds) {
+        const location = ctx.jobIndex.get(jobId);
+        if (location?.type === 'dlq' && location.queueName === queue)
+            ctx.jobIndex.delete(jobId);
+        ctx.jobResults.delete(jobId);
+        ctx.jobLogs.delete(jobId);
+        ctx.dependencyResults.releaseConsumer(jobId);
+    }
+}
 /** Remove every durable and global reference owned by terminal DLQ entries. */
 function cleanupPurgedEntries(queue, entries, ctx, clearQueue) {
     const jobIds = [...new Set(entries.map((entry) => entry.job.id))];
@@ -95,14 +129,7 @@ function cleanupPurgedEntries(queue, entries, ctx, clearQueue) {
     // A newer live generation can reuse the same custom id; its jobs row and
     // auxiliary state must survive cleanup of the stale terminal generation.
     ctx.storage?.purgeDlqEntries(queue, jobIds, terminalJobIds, clearQueue);
-    for (const jobId of terminalJobIds) {
-        const location = ctx.jobIndex.get(jobId);
-        if (location?.type === 'dlq' && location.queueName === queue) {
-            ctx.jobIndex.delete(jobId);
-        }
-        ctx.jobResults.delete(jobId);
-        ctx.jobLogs.delete(jobId);
-    }
+    cleanupTerminalEntries(queue, entries, terminalJobIds, ctx);
 }
 /** Configure DLQ for a queue */
 export function configureDlq(queue, ctx, config) {
@@ -145,6 +172,13 @@ export function retryCompletedJobs(queue, ctx, jobId, options = {}) {
 }
 /** Requeue a single completed job */
 function requeueCompletedJob(job, ctx) {
+    const currentLocation = ctx.jobIndex.get(job.id);
+    if (currentLocation?.type !== 'completed' || currentLocation.queueName !== job.queue)
+        return 0;
+    const shard = ctx.shards[shardIndex(job.queue)];
+    const keyOwner = job.uniqueKey ? shard.getUniqueKeyEntry(job.queue, job.uniqueKey) : null;
+    if (keyOwner && keyOwner.jobId !== job.id)
+        return 0;
     const now = Date.now();
     const timeline = [...job.timeline];
     if (timeline.length < MAX_TIMELINE_ENTRIES) {
@@ -166,7 +200,6 @@ function requeueCompletedJob(job, ctx) {
     // A SQLite failure therefore leaves the completed generation authoritative.
     ctx.storage?.requeueCompletedJob(requeued);
     const idx = shardIndex(requeued.queue);
-    const shard = ctx.shards[idx];
     shard.getQueue(requeued.queue).push(requeued);
     shard.incrementQueued(requeued.id, false, requeued.createdAt, requeued.queue, requeued.runAt);
     ctx.jobIndex.set(requeued.id, {
@@ -177,6 +210,11 @@ function requeueCompletedJob(job, ctx) {
     ctx.completedJobs.delete(requeued.id);
     ctx.completedJobsData.delete(requeued.id);
     ctx.jobResults.delete(requeued.id);
+    if (requeued.customId)
+        ctx.customIdMap.set(requeued.customId, requeued.id);
+    if (requeued.uniqueKey) {
+        shard.registerUniqueKeyWithTtl(requeued.queue, requeued.uniqueKey, requeued.id, requeued.deduplicationTtl ?? undefined);
+    }
     shard.notify(requeued.queue);
     return 1;
 }

package/dist/application/dlqRetry.js

@@ -3,33 +3,62 @@ import { MAX_TIMELINE_ENTRIES } from '../domain/types/job';
 import { retainDlqRetryState, scheduleNextRetry, setDlqRetryState } from '../domain/types/dlq';
 import { shardIndex } from '../shared/hash';
 function prepareRetry(job, now) {
-    job.attempts = 0;
-    job.runAt = now;
-    job.startedAt = null;
-    job.stallCount = 0;
-    job.lastHeartbeat = now;
-    if (job.timeline.length < MAX_TIMELINE_ENTRIES) {
-        job.timeline.push({ state: 'waiting', timestamp: now });
-    }
+    const timeline = [...job.timeline];
+    if (timeline.length < MAX_TIMELINE_ENTRIES)
+        timeline.push({ state: 'waiting', timestamp: now });
+    return {
+        ...job,
+        attempts: 0,
+        lastHeartbeat: now,
+        runAt: now,
+        stallCount: 0,
+        startedAt: null,
+        timeline,
+    };
+}
+function canRetry(queue, job, ctx) {
+    const idx = shardIndex(queue);
+    const shard = ctx.shards[idx];
+    const location = ctx.jobIndex.get(job.id);
+    if (location?.type !== 'dlq' || location.queueName !== queue)
+        return false;
+    const customOwner = job.customId ? ctx.customIdMap.get(job.customId) : undefined;
+    if (customOwner && customOwner !== job.id)
+        return false;
+    const keyOwner = job.uniqueKey ? shard.getUniqueKeyEntry(queue, job.uniqueKey) : null;
+    return keyOwner === null || keyOwner.jobId === job.id;
 }
-function enqueueRetry(queue, job, now, ctx) {
+function publishRetry(queue, job, ctx) {
     const idx = shardIndex(queue);
     const shard = ctx.shards[idx];
-    prepareRetry(job, now);
     shard.getQueue(queue).push(job);
-    shard.incrementQueued(job.id, job.runAt > now, job.createdAt, queue, job.runAt);
+    shard.incrementQueued(job.id, false, job.createdAt, queue, job.runAt);
     ctx.jobIndex.set(job.id, { type: 'queue', shardIdx: idx, queueName: queue });
+    if (job.customId)
+        ctx.customIdMap.set(job.customId, job.id);
+    if (job.uniqueKey) {
+        shard.registerUniqueKeyWithTtl(queue, job.uniqueKey, job.id, job.deduplicationTtl ?? undefined);
+    }
+}
+function retryEntry(queue, entry, ctx, now, prepareState) {
+    if (!canRetry(queue, entry.job, ctx))
+        return null;
+    const job = prepareRetry(entry.job, now);
+    prepareState(job);
     ctx.storage?.requeueDlqJob(job);
+    const removed = ctx.shards[shardIndex(queue)].removeFromDlq(queue, entry.job.id);
+    if (!removed)
+        throw new Error('DLQ entry disappeared after its retry committed');
+    publishRetry(queue, job, ctx);
+    return job;
 }
 /** Retry a single job from DLQ and reset the automatic retry generation. */
 export function retryDlqJob(queue, jobId, ctx) {
     const shard = ctx.shards[shardIndex(queue)];
-    const entry = shard.removeFromDlq(queue, jobId);
+    const entry = shard.getDlqEntries(queue).find((candidate) => candidate.job.id === jobId);
     if (!entry)
         return null;
-    setDlqRetryState(entry.job, null);
-    enqueueRetry(queue, entry.job, Date.now(), ctx);
-    return entry.job;
+    return retryEntry(queue, entry, ctx, Date.now(), (job) => setDlqRetryState(job, null));
 }
 /** Retry jobs from DLQ, optionally bounded to the oldest `limit` entries. */
 export function retryDlqJobs(queue, ctx, jobId, limit) {
@@ -49,29 +78,21 @@ export function retryDlqJobs(queue, ctx, jobId, limit) {
         }
         return retried;
     }
-    const entries = [...shard.getDlqEntries(queue)];
-    shard.clearDlq(queue);
-    ctx.storage?.clearDlqQueue(queue);
-    const now = Date.now();
-    for (const entry of entries) {
-        setDlqRetryState(entry.job, null);
-        enqueueRetry(queue, entry.job, now, ctx);
-    }
-    return entries.length;
+    const ids = shard.getDlqEntries(queue).map((entry) => entry.job.id);
+    let retried = 0;
+    for (const id of ids)
+        if (retryDlqJob(queue, id, ctx))
+            retried++;
+    return retried;
 }
 /** Retry all DLQ entries selected by a metadata filter. */
 export function retryDlqByFilter(queue, ctx, filter) {
     const shard = ctx.shards[shardIndex(queue)];
     const entries = [...shard.getDlqFiltered(queue, filter)];
-    const now = Date.now();
     let retried = 0;
     for (const entry of entries) {
-        const removed = shard.removeFromDlq(queue, entry.job.id);
-        if (!removed)
-            continue;
-        setDlqRetryState(removed.job, null);
-        enqueueRetry(queue, removed.job, now, ctx);
-        retried++;
+        if (retryDlqJob(queue, entry.job.id, ctx))
+            retried++;
     }
     return retried;
 }
@@ -85,13 +106,13 @@ export function processAutoRetry(queue, ctx) {
     const entries = [...shard.getAutoRetryEntries(queue, now)];
     let retried = 0;
     for (const entry of entries) {
-        const removed = shard.removeFromDlq(queue, entry.job.id);
-        if (!removed)
-            continue;
-        scheduleNextRetry(removed, config);
-        retainDlqRetryState(removed.job, removed);
-        enqueueRetry(queue, removed.job, now, ctx);
-        retried++;
+        const scheduled = { ...entry, attempts: [...entry.attempts] };
+        scheduleNextRetry(scheduled, config);
+        const job = retryEntry(queue, entry, ctx, now, (candidate) => {
+            retainDlqRetryState(candidate, scheduled);
+        });
+        if (job)
+            retried++;
     }
     return retried;
 }

package/dist/application/eventsManager.d.ts

@@ -27,7 +27,7 @@ export declare class EventsManager {
      * Wait for a specific job to complete - event-driven, no polling
      * Returns true if job completed, false if timeout
      */
-    waitForJobCompletion(jobId: JobId, timeoutMs: number): Promise<boolean>;
+    waitForJobCompletion(jobId: JobId, timeoutMs: number, signal?: AbortSignal): Promise<boolean>;
     /** Check if broadcast has any listeners - for batch optimizations */
     needsBroadcast(): boolean;
     /** Broadcast event to all subscribers - optimized to skip work when no listeners */