@boringnode/queue 0.5.2 → 0.6.0

package/README.md

@@ -26,6 +26,7 @@ npm install @boringnode/queue
 - **Priority Queues**: Process high-priority jobs first
 - **Bulk Dispatch**: Efficiently dispatch thousands of jobs at once
 - **Job Grouping**: Organize related jobs for monitoring
+- **Job Deduplication**: Prevent duplicate jobs with custom IDs
 - **Retry with Backoff**: Exponential, linear, or fixed backoff strategies
 - **Job Timeout**: Fail or retry jobs that exceed a time limit
 - **Job History**: Retain completed/failed jobs for debugging
@@ -131,6 +132,85 @@ await SendEmailJob.dispatchMany(recipients).group('newsletter-jan-2025')
 
 The `groupId` is stored with job data and accessible via `job.data.groupId`.
 
+## Job Deduplication
+
+Prevent the same job from being pushed multiple times. Four modes, all via `.dedup()`:
+
+### Simple (skip while job exists)
+
+```typescript
+// First dispatch - job is created
+await SendInvoiceJob.dispatch({ orderId: 123 }).dedup({ id: 'order-123' }).run()
+
+// Second dispatch with same dedup ID - silently skipped
+await SendInvoiceJob.dispatch({ orderId: 123 }).dedup({ id: 'order-123' }).run()
+```
+
+### Throttle (skip within TTL window)
+
+```typescript
+// Within 5s, duplicates are skipped. After 5s, a new job is created.
+await SendEmailJob.dispatch({ to: 'user@example.com' })
+  .dedup({ id: 'welcome-123', ttl: '5s' })
+  .run()
+```
+
+### Extend (reset TTL on duplicate)
+
+```typescript
+// Each duplicate push resets the TTL timer.
+await RateLimitJob.dispatch({ userId: 42 }).dedup({ id: 'rate-42', ttl: '1m', extend: true }).run()
+```
+
+### Debounce (replace payload + reset TTL)
+
+```typescript
+// Within the 2s window, the latest payload overwrites the previous pending job.
+await SaveDraftJob.dispatch({ content: 'latest draft' })
+  .dedup({ id: 'draft-42', ttl: '2s', replace: true, extend: true })
+  .run()
+```
+
+### Inspecting the outcome
+
+`DispatchResult` tells you what happened:
+
+```typescript
+const { jobId, deduped } = await SaveDraftJob.dispatch({ content: '...' })
+  .dedup({ id: 'draft-42', ttl: '2s', replace: true })
+  .run()
+
+// deduped: 'added' | 'skipped' | 'replaced' | 'extended'
+// jobId: the UUID of the job (the existing one when deduped)
+```
+
+### How it works
+
+- The dedup ID is automatically prefixed with the job name (`SendInvoiceJob::order-123`), so different job types can reuse the same key.
+- The user-supplied `id` must be ≤ 400 characters, and the combined `<jobName>::<id>` key must be ≤ 510 characters (constrained by the Knex storage column). Both limits are validated at `.dedup()` time.
+- `ttl` accepts a Duration (`'5s'`, `'1m'`) or milliseconds, and must be **positive** when provided. Use `0` or omit `ttl` if you want no expiry — `ttl: 0` is rejected to avoid an ambiguous "expired immediately vs no-expiry" interpretation across engines.
+- `extend` and `replace` **require** `ttl` — calling them without `ttl` throws.
+- `replace` only applies to jobs in `pending` or `delayed` state. Jobs that are active (executing) or retained in history (`completed`/`failed` with retention) are left alone; the dispatch returns `{ deduped: 'skipped' }`.
+- `replace` swaps the **payload only** — priority, queue, delay, groupId, and stored dedup options of the existing job are retained. To change those, use a different dedup id or wait for the TTL to expire.
+- `extend` resets the TTL clock but never changes the window length. The window length is fixed to the `ttl` from the first dispatch that created the dedup slot. Later dispatches that pass a different `ttl` only reset the clock; their `ttl` value is ignored. To resize the window, let the slot expire and start over with a new dispatch.
+- `extend` works in **all states** — even when the existing job is `active` (executing) or retained in history. Unlike `replace` (which is no-op on non-replaceable states), `extend` always refreshes the dedup TTL window. Use this when you want the dedup slot to keep blocking new dispatches for the lifetime of a long-running job.
+- `extend` requires the **first** dispatch to have set a `ttl`. If the slot was created without a `ttl`, later `extend` dispatches have no window to refresh and return `{ deduped: 'skipped' }` instead of `'extended'`.
+- `retryJob` does not touch the dedup entry — a retried job continues to occupy the dedup slot. TTL runs on wall-clock time, so long-running retries may outlive the TTL window. Use a generous TTL or no TTL if retries must stay deduped.
+- Atomic and race-free:
+  - **Redis**: a single Lua script per dispatch performs the dedup-key lookup, state check (pending/delayed ZSCORE), payload swap, and TTL refresh atomically.
+  - **Knex**: transactional `SELECT ... FOR UPDATE` + insert/update inside a transaction. A nested savepoint catches unique-constraint violations under concurrent inserts and returns `{ deduped: 'skipped' }` pointing at the winner.
+  - **SyncAdapter**: executes inline, no dedup support.
+
+### Caveats
+
+- Without `.dedup()`, jobs use auto-generated UUIDs and are never deduplicated.
+- The **Sync adapter** ignores `.dedup()` entirely — every dispatch executes inline and `deduped` is always `undefined` on the result. Use Redis or Knex if you need real deduplication.
+- `.dedup()` is only available on single dispatch. `dispatchMany` / `pushManyOn` reject jobs with a `dedup` field.
+- Scheduled jobs (`.schedule()`) do not support dedup — each cron/interval fire is an independent dispatch.
+- With no `ttl`, dedup persists until the job is removed (completed/failed without retention). When retention keeps the record, re-dispatch stays blocked until the record is pruned.
+- With `ttl`, dedup expires after the window — a new job (new UUID) is created. The old job still runs.
+- Knex MySQL concurrent race: MySQL does not support partial unique indexes, so two `pushOn` calls with the same dedup id firing at the exact same instant can both succeed. Serialize at the app layer if strict guarantees are required, or use Postgres / SQLite / Redis (all of which serialize correctly via the partial unique index or Lua atomicity).
+
 ## Job History & Retention
 
 Keep completed and failed jobs for debugging:
@@ -536,7 +616,7 @@ import * as boringqueue from '@boringnode/queue'
 
 const instrumentation = new QueueInstrumentation({
   messagingSystem: 'boringqueue', // default
-  executionSpanLinkMode: 'link',  // or 'parent'
+  executionSpanLinkMode: 'link', // or 'parent'
 })
 
 instrumentation.enable()
@@ -549,19 +629,19 @@ The instrumentation patches `QueueManager.init()` to automatically inject its wr
 
 The instrumentation uses standard [OTel messaging semantic conventions](https://opentelemetry.io/docs/specs/semconv/messaging/messaging-spans/) where they map cleanly, plus a few queue-specific custom attributes.
 
-| Attribute                       | Kind    | Description                                |
-| ------------------------------- | ------- | ------------------------------------------ |
-| `messaging.system`              | Semconv | `'boringqueue'` (configurable)             |
-| `messaging.operation.name`      | Semconv | `'publish'` or `'process'`                 |
-| `messaging.destination.name`    | Semconv | Queue name                                 |
-| `messaging.message.id`          | Semconv | Job ID for single-message spans            |
-| `messaging.batch.message_count` | Semconv | Number of jobs in a batch dispatch         |
-| `messaging.message.retry.count` | Custom  | Retry count (0-based) for a job attempt    |
-| `messaging.job.name`            | Custom  | Job class name (e.g. `SendEmailJob`)       |
-| `messaging.job.status`          | Custom  | `'completed'`, `'failed'`, or `'retrying'` |
-| `messaging.job.group_id`        | Custom  | Queue-specific group identifier            |
-| `messaging.job.priority`        | Custom  | Queue-specific job priority                |
-| `messaging.job.delay_ms`        | Custom  | Delay before the job becomes available     |
+| Attribute                       | Kind    | Description                                   |
+| ------------------------------- | ------- | --------------------------------------------- |
+| `messaging.system`              | Semconv | `'boringqueue'` (configurable)                |
+| `messaging.operation.name`      | Semconv | `'publish'` or `'process'`                    |
+| `messaging.destination.name`    | Semconv | Queue name                                    |
+| `messaging.message.id`          | Semconv | Job ID for single-message spans               |
+| `messaging.batch.message_count` | Semconv | Number of jobs in a batch dispatch            |
+| `messaging.message.retry.count` | Custom  | Retry count (0-based) for a job attempt       |
+| `messaging.job.name`            | Custom  | Job class name (e.g. `SendEmailJob`)          |
+| `messaging.job.status`          | Custom  | `'completed'`, `'failed'`, or `'retrying'`    |
+| `messaging.job.group_id`        | Custom  | Queue-specific group identifier               |
+| `messaging.job.priority`        | Custom  | Queue-specific job priority                   |
+| `messaging.job.delay_ms`        | Custom  | Delay before the job becomes available        |
 | `messaging.job.queue_time_ms`   | Custom  | Time spent waiting in queue before processing |
 
 ### Trace Context Propagation

package/build/chunk-6IO4P6RB.js

@@ -0,0 +1,145 @@
+// src/drivers/redis_job_storage.ts
+function hydrateRedisJob(data, overlay) {
+  const jobData = JSON.parse(data);
+  const parsedOverlay = overlay ? JSON.parse(overlay) : void 0;
+  if (parsedOverlay?.payloadUndefined) {
+    jobData.payload = void 0;
+  } else if (parsedOverlay?.payload !== void 0) {
+    jobData.payload = JSON.parse(parsedOverlay.payload);
+  }
+  return {
+    ...jobData,
+    ...parsedOverlay?.attempts !== void 0 ? { attempts: parsedOverlay.attempts } : {},
+    ...parsedOverlay?.stalledCount !== void 0 ? { stalledCount: parsedOverlay.stalledCount } : {}
+  };
+}
+function encodeRedisJobPayloadOverlay(payload) {
+  const encoded = JSON.stringify(payload);
+  return encoded === void 0 ? ["", "1"] : [encoded, "0"];
+}
+var REDIS_JOB_STORAGE_LUA = `
+  local function read_job_overlay(overlay_key, job_id)
+    local overlay_json = redis.call('HGET', overlay_key, job_id)
+    if overlay_json then
+      return cjson.decode(overlay_json)
+    end
+    return {}
+  end
+
+  local function write_job_overlay(overlay_key, job_id, overlay)
+    local has_overlay = false
+    for _ in pairs(overlay) do
+      has_overlay = true
+      break
+    end
+
+    if has_overlay then
+      redis.call('HSET', overlay_key, job_id, cjson.encode(overlay))
+    else
+      redis.call('HDEL', overlay_key, job_id)
+    end
+  end
+
+  local function store_job_data(data_key, overlay_key, job_id, job_data)
+    redis.call('HDEL', overlay_key, job_id)
+    redis.call('HSET', data_key, job_id, job_data)
+  end
+
+  local function delete_job_data(data_key, overlay_key, job_id)
+    redis.call('HDEL', data_key, job_id)
+    redis.call('HDEL', overlay_key, job_id)
+  end
+
+  local function delete_jobs_data(data_key, overlay_key, job_ids)
+    if #job_ids > 0 then
+      redis.call('HDEL', data_key, unpack(job_ids))
+      redis.call('HDEL', overlay_key, unpack(job_ids))
+    end
+  end
+
+  local function encode_job_result(job_data, overlay_key, job_id, result)
+    local encoded = result or {}
+    encoded.data = job_data
+
+    local overlay_json = redis.call('HGET', overlay_key, job_id)
+    if overlay_json then
+      encoded.overlay = overlay_json
+    end
+
+    return cjson.encode(encoded)
+  end
+
+  local function apply_payload_overlay(overlay, payload_data, payload_is_undefined)
+    if payload_is_undefined == 1 then
+      overlay.payload = nil
+      overlay.payloadUndefined = true
+    else
+      overlay.payload = payload_data
+      overlay.payloadUndefined = nil
+    end
+  end
+`;
+var REDIS_DEDUP_LUA = `
+  local function resolve_dedup_existing_job(
+    data_key,
+    state_key_a,
+    state_key_b,
+    overlay_key,
+    dedup_key,
+    extend,
+    replace,
+    payload_data,
+    payload_is_undefined
+  )
+    local existing = redis.call('GET', dedup_key)
+    if not existing then
+      return nil
+    end
+
+    local existing_data = redis.call('HGET', data_key, existing)
+    if not existing_data then
+      return nil
+    end
+
+    local in_state_a = redis.call('ZSCORE', state_key_a, existing)
+    local in_state_b = redis.call('ZSCORE', state_key_b, existing)
+    local replaceable = in_state_a or in_state_b
+    local ok, existing_job = pcall(cjson.decode, existing_data)
+    local original_ttl = nil
+
+    if ok and type(existing_job) == 'table' and existing_job.dedup then
+      original_ttl = tonumber(existing_job.dedup.ttl)
+    end
+
+    if replace == 1 and replaceable then
+      if ok and existing_job then
+        local overlay = read_job_overlay(overlay_key, existing)
+        apply_payload_overlay(overlay, payload_data, payload_is_undefined)
+        write_job_overlay(overlay_key, existing, overlay)
+
+        if extend == 1 and original_ttl and original_ttl > 0 then
+          redis.call('PEXPIRE', dedup_key, original_ttl)
+        end
+
+        return {'replaced', existing}
+      end
+
+      return {'skipped', existing}
+    end
+
+    if extend == 1 and original_ttl and original_ttl > 0 then
+      redis.call('PEXPIRE', dedup_key, original_ttl)
+      return {'extended', existing}
+    end
+
+    return {'skipped', existing}
+  end
+`;
+
+export {
+  hydrateRedisJob,
+  encodeRedisJobPayloadOverlay,
+  REDIS_JOB_STORAGE_LUA,
+  REDIS_DEDUP_LUA
+};
+//# sourceMappingURL=chunk-6IO4P6RB.js.map

package/build/chunk-6IO4P6RB.js.map

@@ -0,0 +1 @@
+{"version":3,"sources":["../src/drivers/redis_job_storage.ts"],"sourcesContent":["import type { JobData } from '../types/main.js'\n\n/**\n * Redis stores the original job JSON as an opaque payload in the data hash.\n * Lua scripts write only mutable Redis-side overrides in this overlay so\n * legacy jobs without overlays keep decoding from their original data.\n */\nexport type RedisJobOverlay = Partial<Pick<JobData, 'attempts' | 'stalledCount'>> & {\n  payload?: string\n  payloadUndefined?: boolean\n}\n\nexport function hydrateRedisJob(data: string, overlay?: string | null | false): JobData {\n  const jobData = JSON.parse(data) as JobData\n  const parsedOverlay = overlay ? (JSON.parse(overlay) as RedisJobOverlay) : undefined\n\n  if (parsedOverlay?.payloadUndefined) {\n    jobData.payload = undefined\n  } else if (parsedOverlay?.payload !== undefined) {\n    jobData.payload = JSON.parse(parsedOverlay.payload)\n  }\n\n  return {\n    ...jobData,\n    ...(parsedOverlay?.attempts !== undefined ? { attempts: parsedOverlay.attempts } : {}),\n    ...(parsedOverlay?.stalledCount !== undefined\n      ? { stalledCount: parsedOverlay.stalledCount }\n      : {}),\n  }\n}\n\nexport function encodeRedisJobPayloadOverlay(payload: JobData['payload']): [string, string] {\n  const encoded = JSON.stringify(payload)\n\n  return encoded === undefined ? ['', '1'] : [encoded, '0']\n}\n\nexport const REDIS_JOB_STORAGE_LUA = `\n  local function read_job_overlay(overlay_key, job_id)\n    local overlay_json = redis.call('HGET', overlay_key, job_id)\n    if overlay_json then\n      return cjson.decode(overlay_json)\n    end\n    return {}\n  end\n\n  local function write_job_overlay(overlay_key, job_id, overlay)\n    local has_overlay = false\n    for _ in pairs(overlay) do\n      has_overlay = true\n      break\n    end\n\n    if has_overlay then\n      redis.call('HSET', overlay_key, job_id, cjson.encode(overlay))\n    else\n      redis.call('HDEL', overlay_key, job_id)\n    end\n  end\n\n  local function store_job_data(data_key, overlay_key, job_id, job_data)\n    redis.call('HDEL', overlay_key, job_id)\n    redis.call('HSET', data_key, job_id, job_data)\n  end\n\n  local function delete_job_data(data_key, overlay_key, job_id)\n    redis.call('HDEL', data_key, job_id)\n    redis.call('HDEL', overlay_key, job_id)\n  end\n\n  local function delete_jobs_data(data_key, overlay_key, job_ids)\n    if #job_ids > 0 then\n      redis.call('HDEL', data_key, unpack(job_ids))\n      redis.call('HDEL', overlay_key, unpack(job_ids))\n    end\n  end\n\n  local function encode_job_result(job_data, overlay_key, job_id, result)\n    local encoded = result or {}\n    encoded.data = job_data\n\n    local overlay_json = redis.call('HGET', overlay_key, job_id)\n    if overlay_json then\n      encoded.overlay = overlay_json\n    end\n\n    return cjson.encode(encoded)\n  end\n\n  local function apply_payload_overlay(overlay, payload_data, payload_is_undefined)\n    if payload_is_undefined == 1 then\n      overlay.payload = nil\n      overlay.payloadUndefined = true\n    else\n      overlay.payload = payload_data\n      overlay.payloadUndefined = nil\n    end\n  end\n`\n\nexport const REDIS_DEDUP_LUA = `\n  local function resolve_dedup_existing_job(\n    data_key,\n    state_key_a,\n    state_key_b,\n    overlay_key,\n    dedup_key,\n    extend,\n    replace,\n    payload_data,\n    payload_is_undefined\n  )\n    local existing = redis.call('GET', dedup_key)\n    if not existing then\n      return nil\n    end\n\n    local existing_data = redis.call('HGET', data_key, existing)\n    if not existing_data then\n      return nil\n    end\n\n    local in_state_a = redis.call('ZSCORE', state_key_a, existing)\n    local in_state_b = redis.call('ZSCORE', state_key_b, existing)\n    local replaceable = in_state_a or in_state_b\n    local ok, existing_job = pcall(cjson.decode, existing_data)\n    local original_ttl = nil\n\n    if ok and type(existing_job) == 'table' and existing_job.dedup then\n      original_ttl = tonumber(existing_job.dedup.ttl)\n    end\n\n    if replace == 1 and replaceable then\n      if ok and existing_job then\n        local overlay = read_job_overlay(overlay_key, existing)\n        apply_payload_overlay(overlay, payload_data, payload_is_undefined)\n        write_job_overlay(overlay_key, existing, overlay)\n\n        if extend == 1 and original_ttl and original_ttl > 0 then\n          redis.call('PEXPIRE', dedup_key, original_ttl)\n        end\n\n        return {'replaced', existing}\n      end\n\n      return {'skipped', existing}\n    end\n\n    if extend == 1 and original_ttl and original_ttl > 0 then\n      redis.call('PEXPIRE', dedup_key, original_ttl)\n      return {'extended', existing}\n    end\n\n    return {'skipped', existing}\n  end\n`\n"],"mappings":";AAYO,SAAS,gBAAgB,MAAc,SAA0C;AACtF,QAAM,UAAU,KAAK,MAAM,IAAI;AAC/B,QAAM,gBAAgB,UAAW,KAAK,MAAM,OAAO,IAAwB;AAE3E,MAAI,eAAe,kBAAkB;AACnC,YAAQ,UAAU;AAAA,EACpB,WAAW,eAAe,YAAY,QAAW;AAC/C,YAAQ,UAAU,KAAK,MAAM,cAAc,OAAO;AAAA,EACpD;AAEA,SAAO;AAAA,IACL,GAAG;AAAA,IACH,GAAI,eAAe,aAAa,SAAY,EAAE,UAAU,cAAc,SAAS,IAAI,CAAC;AAAA,IACpF,GAAI,eAAe,iBAAiB,SAChC,EAAE,cAAc,cAAc,aAAa,IAC3C,CAAC;AAAA,EACP;AACF;AAEO,SAAS,6BAA6B,SAA+C;AAC1F,QAAM,UAAU,KAAK,UAAU,OAAO;AAEtC,SAAO,YAAY,SAAY,CAAC,IAAI,GAAG,IAAI,CAAC,SAAS,GAAG;AAC1D;AAEO,IAAM,wBAAwB;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AA+D9B,IAAM,kBAAkB;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;","names":[]}

package/build/{chunk-VRXHCWNK.js → chunk-AHUVTAI7.js}

@@ -603,6 +603,7 @@ var JobDispatcher = class {
   #delay;
   #priority;
   #groupId;
+  #dedup;
   /**
    * Create a new job dispatcher.
    *
@@ -694,6 +695,76 @@ var JobDispatcher = class {
     this.#groupId = groupId;
     return this;
   }
+  /**
+   * Configure deduplication for this job.
+   *
+   * Modes:
+   * - **Simple** (`{ id }`): skip duplicates while the job exists.
+   * - **Throttle** (`{ id, ttl }`): skip duplicates within a TTL window.
+   * - **Extend** (`{ id, ttl, extend: true }`): reset the TTL clock on each duplicate.
+   *   The window length stays at the original ttl from the first dispatch.
+   * - **Replace** (`{ id, ttl, replace: true }`): swap the payload of the existing
+   *   pending/delayed job on duplicate within TTL. Active jobs and retained
+   *   completed/failed jobs return `'skipped'`. Only `payload` changes —
+   *   priority/queue/delay/groupId are preserved.
+   * - **Debounce** (`{ id, ttl, replace: true, extend: true }`): replace + reset TTL.
+   *
+   * The id is automatically prefixed with the job name to prevent collisions
+   * between different job types.
+   *
+   * @param options.id - Unique deduplication key
+   * @param options.ttl - TTL as Duration ('5s', 5000). Required for extend/replace.
+   * @param options.extend - Reset the TTL clock on duplicate within window. Window
+   *   length stays at the original ttl; this option's `ttl` arg is ignored on extend.
+   * @param options.replace - Swap payload of existing pending/delayed job within
+   *   window. Active and retained jobs are not modified.
+   *
+   * @example
+   * ```typescript
+   * // Simple dedup
+   * await SendInvoiceJob.dispatch({ orderId: 123 })
+   *   .dedup({ id: 'order-123' })
+   *
+   * // Throttle: 5 second window
+   * await SendEmailJob.dispatch({ to: 'x' })
+   *   .dedup({ id: 'welcome', ttl: '5s' })
+   *
+   * // Debounce: replace payload within window
+   * await SaveDraftJob.dispatch({ content: 'latest' })
+   *   .dedup({ id: 'draft-42', ttl: '2s', replace: true, extend: true })
+   * ```
+   */
+  dedup(options) {
+    if (!options.id) {
+      throw new Error("Dedup ID must be a non-empty string");
+    }
+    if (options.id.length > 400) {
+      throw new Error("Dedup ID must be 400 characters or less");
+    }
+    const prefixedLength = this.#name.length + 2 + options.id.length;
+    if (prefixedLength > 510) {
+      throw new Error(
+        `Dedup ID combined with job name exceeds 510 characters (got ${prefixedLength}). Shorten either the job name or the dedup id.`
+      );
+    }
+    if ((options.extend || options.replace) && options.ttl === void 0) {
+      throw new Error("dedup.ttl is required when extend or replace is set");
+    }
+    let parsedTtl;
+    if (options.ttl !== void 0) {
+      parsedTtl = parse(options.ttl);
+      if (!Number.isFinite(parsedTtl) || parsedTtl <= 0) {
+        throw new Error("dedup.ttl must be a positive duration");
+      }
+    }
+    this.#dedup = {
+      id: options.id,
+      ttl: parsedTtl,
+      extend: options.extend,
+      replace: options.replace
+    };
+    return this;
+  }
   /**
    * Use a specific adapter for this job.
    *
@@ -726,6 +797,7 @@ var JobDispatcher = class {
    */
   async run() {
     const id = randomUUID();
+    const dedupId = this.#dedup ? `${this.#name}::${this.#dedup.id}` : void 0;
     debug_default("dispatching job %s with id %s using payload %s", this.#name, id, this.#payload);
     const adapter = this.#getAdapterInstance();
     const wrapInternal = QueueManager.getInternalOperationWrapper();
@@ -737,16 +809,31 @@ var JobDispatcher = class {
       attempts: 0,
       priority: this.#priority,
       groupId: this.#groupId,
-      createdAt: Date.now()
+      createdAt: Date.now(),
+      ...dedupId ? {
+        dedup: {
+          id: dedupId,
+          ttl: this.#dedup.ttl,
+          extend: this.#dedup.extend,
+          replace: this.#dedup.replace
+        }
+      } : {}
     };
     const message = { jobs: [jobData], queue: this.#queue, delay: parsedDelay };
+    let pushResult;
     await dispatchChannel.tracePromise(async () => {
-      if (parsedDelay !== void 0) {
-        await wrapInternal(() => adapter.pushLaterOn(this.#queue, jobData, parsedDelay));
-      } else {
-        await wrapInternal(() => adapter.pushOn(this.#queue, jobData));
+      const result = parsedDelay !== void 0 ? await wrapInternal(() => adapter.pushLaterOn(this.#queue, jobData, parsedDelay)) : await wrapInternal(() => adapter.pushOn(this.#queue, jobData));
+      if (result && typeof result === "object" && "outcome" in result) {
+        pushResult = { outcome: result.outcome, jobId: result.jobId };
+        message.dedupOutcome = result.outcome;
       }
     }, message);
+    if (pushResult && this.#dedup) {
+      return {
+        jobId: pushResult.jobId,
+        deduped: pushResult.outcome
+      };
+    }
     return { jobId: id };
   }
   /**
@@ -1297,7 +1384,9 @@ var FakeAdapter = class {
   #pendingTimeouts = /* @__PURE__ */ new Set();
   #schedules = /* @__PURE__ */ new Map();
   #pushedJobs = [];
+  #dedupIndex = /* @__PURE__ */ new Map();
   #onDispose;
+  #workerId = "";
   /**
    * Set the function to call when the fake is disposed
    */
@@ -1308,7 +1397,8 @@ var FakeAdapter = class {
   [Symbol.dispose]() {
     this.#onDispose?.();
   }
-  setWorkerId(_workerId) {
+  setWorkerId(workerId) {
+    this.#workerId = workerId;
   }
   getPushedJobs() {
     return [...this.#pushedJobs];
@@ -1334,6 +1424,7 @@ var FakeAdapter = class {
     this.#failedJobs.clear();
     this.#schedules.clear();
     this.#pushedJobs = [];
+    this.#dedupIndex.clear();
   }
   assertPushed(matcher, query) {
     const record = this.findPushed(matcher, query);
@@ -1366,21 +1457,33 @@ var FakeAdapter = class {
     return this.pushOn("default", jobData);
   }
   async pushOn(queue, jobData) {
+    const deduped = await this.#applyDedup(queue, jobData);
+    if (deduped) return deduped;
     this.#recordPush(queue, jobData);
     this.#enqueue(queue, jobData);
+    if (jobData.dedup) {
+      return { outcome: "added", jobId: jobData.id };
+    }
   }
   async pushLater(jobData, delay) {
     return this.pushLaterOn("default", jobData, delay);
   }
-  pushLaterOn(queue, jobData, delay) {
+  async pushLaterOn(queue, jobData, delay) {
+    const deduped = await this.#applyDedup(queue, jobData);
+    if (deduped) return deduped;
     this.#recordPush(queue, jobData, delay);
     this.#schedulePush(queue, jobData, delay);
-    return Promise.resolve();
+    if (jobData.dedup) {
+      return { outcome: "added", jobId: jobData.id };
+    }
   }
   async pushMany(jobs) {
     return this.pushManyOn("default", jobs);
   }
   async pushManyOn(queue, jobs) {
+    if (jobs.some((j) => j.dedup)) {
+      throw new Error("dedup is not supported in batch dispatch; use single dispatch");
+    }
     for (const job of jobs) {
       await this.pushOn(queue, job);
     }
@@ -1407,7 +1510,7 @@ var FakeAdapter = class {
       return null;
     }
     const acquiredAt = Date.now();
-    this.#activeJobs.set(job.id, { job, acquiredAt, queue });
+    this.#activeJobs.set(job.id, { job, acquiredAt, queue, workerId: this.#workerId });
     return { ...job, acquiredAt };
   }
   async completeJob(jobId, queue, removeOnComplete) {
@@ -1415,6 +1518,7 @@ var FakeAdapter = class {
     if (!active) return;
     this.#activeJobs.delete(jobId);
     if (removeOnComplete === void 0 || removeOnComplete === true) {
+      this.#cleanupDedupForJob(queue, active.job);
       return;
     }
     this.#storeHistory(queue, "completed", active.job, removeOnComplete);
@@ -1424,6 +1528,7 @@ var FakeAdapter = class {
     if (!active) return;
     this.#activeJobs.delete(jobId);
     if (removeOnFail === void 0 || removeOnFail === true) {
+      this.#cleanupDedupForJob(queue, active.job);
       return;
     }
     this.#storeHistory(queue, "failed", active.job, removeOnFail, error);
@@ -1459,6 +1564,7 @@ var FakeAdapter = class {
       const currentStalledCount = active.job.stalledCount ?? 0;
       if (currentStalledCount >= maxStalledCount) {
         this.#activeJobs.delete(jobId);
+        this.#cleanupDedupForJob(active.queue, active.job);
         continue;
       }
       this.#activeJobs.delete(jobId);
@@ -1471,6 +1577,18 @@ var FakeAdapter = class {
     }
     return recovered;
   }
+  async renewJobs(queue, jobIds) {
+    const now = Date.now();
+    let renewed = 0;
+    for (const jobId of jobIds) {
+      const active = this.#activeJobs.get(jobId);
+      if (active && active.queue === queue && active.workerId === this.#workerId) {
+        active.acquiredAt = now;
+        renewed++;
+      }
+    }
+    return renewed;
+  }
   async getJob(jobId, queue) {
     const active = this.#activeJobs.get(jobId);
     if (active && active.queue === queue) {
@@ -1627,23 +1745,110 @@ var FakeAdapter = class {
     const records = store.get(queue);
     records.push(record);
     if (retention && retention !== true) {
-      this.#applyRetention(records, retention);
+      this.#applyRetention(records, retention, queue);
     }
   }
-  #applyRetention(records, retention) {
+  #applyRetention(records, retention, queue) {
     if (retention === false || retention === true) {
       return;
     }
+    const pruned = [];
     if (retention.age !== void 0) {
       const maxAgeMs = parse(retention.age);
       if (maxAgeMs > 0) {
         const cutoff = Date.now() - maxAgeMs;
-        const filtered = records.filter((record) => (record.finishedAt ?? 0) >= cutoff);
-        records.splice(0, records.length, ...filtered);
+        const kept = [];
+        for (const record of records) {
+          if ((record.finishedAt ?? 0) >= cutoff) {
+            kept.push(record);
+          } else {
+            pruned.push(record);
+          }
+        }
+        records.splice(0, records.length, ...kept);
       }
     }
     if (retention.count !== void 0 && retention.count > 0 && records.length > retention.count) {
-      records.splice(0, records.length - retention.count);
+      const excess = records.length - retention.count;
+      pruned.push(...records.slice(0, excess));
+      records.splice(0, excess);
+    }
+    for (const record of pruned) {
+      this.#cleanupDedupForJob(queue, record.data);
+    }
+  }
+  #applyDedup(queue, jobData) {
+    if (!jobData.dedup) return null;
+    const dedupId = jobData.dedup.id;
+    const now = Date.now();
+    const entry = this.#getDedupEntry(queue, dedupId);
+    if (entry) {
+      const withinTtl = !entry.ttl || now - entry.createdAt < entry.ttl;
+      if (withinTtl) {
+        const existing = this.#findJobById(queue, entry.jobId);
+        if (existing) {
+          const replaceable = existing.location === "pending" || existing.location === "delayed";
+          if (jobData.dedup.replace && replaceable) {
+            existing.job.payload = structuredClone(jobData.payload);
+            if (jobData.dedup.extend && entry.ttl) {
+              entry.createdAt = now;
+            }
+            return { outcome: "replaced", jobId: entry.jobId };
+          }
+          if (jobData.dedup.extend && entry.ttl) {
+            entry.createdAt = now;
+            return { outcome: "extended", jobId: entry.jobId };
+          }
+          return { outcome: "skipped", jobId: entry.jobId };
+        }
+      }
+    }
+    this.#setDedupEntry(queue, dedupId, {
+      jobId: jobData.id,
+      createdAt: now,
+      ttl: jobData.dedup.ttl,
+      replace: jobData.dedup.replace,
+      extend: jobData.dedup.extend
+    });
+    return null;
+  }
+  #findJobById(queue, jobId) {
+    const active = this.#activeJobs.get(jobId);
+    if (active && active.queue === queue) {
+      return { job: active.job, location: "active" };
+    }
+    const pending = this.#queues.get(queue)?.find((j) => j.id === jobId);
+    if (pending) {
+      return { job: pending, location: "pending" };
+    }
+    const delayed = this.#delayedJobs.get(queue)?.get(jobId);
+    if (delayed) {
+      return { job: delayed.job, location: "delayed" };
+    }
+    const completed = this.#findHistory(this.#completedJobs, queue, jobId);
+    if (completed) {
+      return { job: completed.data, location: "completed" };
+    }
+    const failed = this.#findHistory(this.#failedJobs, queue, jobId);
+    if (failed) {
+      return { job: failed.data, location: "failed" };
+    }
+    return null;
+  }
+  #getDedupEntry(queue, dedupId) {
+    return this.#dedupIndex.get(queue)?.get(dedupId);
+  }
+  #setDedupEntry(queue, dedupId, entry) {
+    if (!this.#dedupIndex.has(queue)) {
+      this.#dedupIndex.set(queue, /* @__PURE__ */ new Map());
+    }
+    this.#dedupIndex.get(queue).set(dedupId, entry);
+  }
+  #cleanupDedupForJob(queue, job) {
+    if (!job.dedup) return;
+    const entry = this.#getDedupEntry(queue, job.dedup.id);
+    if (entry && entry.jobId === job.id) {
+      this.#dedupIndex.get(queue)?.delete(job.dedup.id);
     }
   }
   #findHistory(store, queue, jobId) {
@@ -1721,4 +1926,4 @@ export {
   ScheduleBuilder,
   Job
 };
-//# sourceMappingURL=chunk-VRXHCWNK.js.map
+//# sourceMappingURL=chunk-AHUVTAI7.js.map