@nicnocquee/dataqueue 1.35.0-beta.20260224075710 → 1.35.0-beta.20260224112317

package/ai/docs-content.json

@@ -33,25 +33,25 @@
     "slug": "api/job-options",
     "title": "JobOptions",
     "description": "",
-    "content": "The `JobOptions` interface defines the options for creating a new job in the queue.\n\n## Fields\n\n- `jobType`: _string_ — The type of the job.\n- `payload`: _any_ — The payload for the job, type-safe per job\n  type.\n- `maxAttempts?`: _number_ — Maximum number of attempts for\n  this job (default: 3).\n- `priority?`: _number_ — Priority of the job (higher runs\n  first, default: 0).\n- `runAt?`: _Date | null_ — When to run the job (default: now).\n- `timeoutMs?`: _number_ — Timeout for this job in milliseconds.\n  If not set, uses the processor default or unlimited.\n- `forceKillOnTimeout?`: _boolean_ — If true, the job will be forcefully terminated (using Worker Threads) when timeout is reached. If false (default), the job will only receive an AbortSignal and must handle the abort gracefully.\n\n  **⚠️ Runtime Requirements**: This option requires **Node.js** and will **not work** in Bun or other runtimes without worker thread support. See [Force Kill on Timeout](/usage/force-kill-timeout) for details.\n\n- `tags?`: _string[]_ — Tags for this job. Used for grouping, searching, or batch operations.\n- `idempotencyKey?`: _string_ — Optional idempotency key. When provided, ensures that only one job exists for a given key. If a job with the same key already exists, `addJob` returns the existing job's ID instead of creating a duplicate. See [Idempotency](/usage/add-job#idempotency) for details.\n\n## Example\n\n```ts\nconst job = {\n  jobType: 'email',\n  payload: { to: 'user@example.com', subject: 'Hello' },\n  maxAttempts: 5,\n  priority: 10,\n  runAt: new Date(Date.now() + 60000), // run in 1 minute\n  timeoutMs: 30000, // 30 seconds\n  forceKillOnTimeout: false, // Use graceful shutdown (default)\n  tags: ['welcome', 'user'], // tags for grouping/searching\n  idempotencyKey: 'welcome-email-user-123', // prevent duplicate jobs\n};\n```"
+    "content": "The `JobOptions` interface defines the options for creating a new job in the queue.\n\n## Fields\n\n- `jobType`: _string_ — The type of the job.\n- `payload`: _any_ — The payload for the job, type-safe per job\n  type.\n- `maxAttempts?`: _number_ — Maximum number of attempts for\n  this job (default: 3).\n- `priority?`: _number_ — Priority of the job (higher runs\n  first, default: 0).\n- `runAt?`: _Date | null_ — When to run the job (default: now).\n- `timeoutMs?`: _number_ — Timeout for this job in milliseconds.\n  If not set, uses the processor default or unlimited.\n- `forceKillOnTimeout?`: _boolean_ — If true, the job will be forcefully terminated (using Worker Threads) when timeout is reached. If false (default), the job will only receive an AbortSignal and must handle the abort gracefully.\n\n  **⚠️ Runtime Requirements**: This option requires **Node.js** and will **not work** in Bun or other runtimes without worker thread support. See [Force Kill on Timeout](/usage/force-kill-timeout) for details.\n\n- `tags?`: _string[]_ — Tags for this job. Used for grouping, searching, or batch operations.\n- `idempotencyKey?`: _string_ — Optional idempotency key. When provided, ensures that only one job exists for a given key. If a job with the same key already exists, `addJob` returns the existing job's ID instead of creating a duplicate. See [Idempotency](/usage/add-job#idempotency) for details.\n- `deadLetterJobType?`: _string_ — Optional dead-letter destination job type. When the job exhausts retries, DataQueue creates a new pending job in this job type with an envelope payload containing source metadata, original payload, and failure context.\n\n## Example\n\n```ts\nconst job = {\n  jobType: 'email',\n  payload: { to: 'user@example.com', subject: 'Hello' },\n  maxAttempts: 5,\n  priority: 10,\n  runAt: new Date(Date.now() + 60000), // run in 1 minute\n  timeoutMs: 30000, // 30 seconds\n  forceKillOnTimeout: false, // Use graceful shutdown (default)\n  tags: ['welcome', 'user'], // tags for grouping/searching\n  idempotencyKey: 'welcome-email-user-123', // prevent duplicate jobs\n  deadLetterJobType: 'email_dead_letter', // route exhausted failures\n};\n```"
   },
   {
     "slug": "api/job-queue",
     "title": "JobQueue",
     "description": "",
-    "content": "## Initialization\n\n### initJobQueue\n\n```ts\ninitJobQueue(config: JobQueueConfig): JobQueue\n```\n\nInitializes the job queue system with the provided configuration. The `JobQueueConfig` is a discriminated union -- you provide either a PostgreSQL or Redis configuration.\n\n#### PostgresJobQueueConfig\n\nProvide either `databaseConfig` (the library creates a pool) or `pool` (bring your own `pg.Pool`). At least one must be set.\n\n```ts\ninterface PostgresJobQueueConfig {\n  backend?: 'postgres'; // Optional, defaults to 'postgres'\n  databaseConfig?: {\n    connectionString?: string;\n    host?: string;\n    port?: number;\n    database?: string;\n    user?: string;\n    password?: string;\n    ssl?: DatabaseSSLConfig;\n  };\n  pool?: import('pg').Pool; // Bring your own pool\n  verbose?: boolean;\n}\n```\n\n#### RedisJobQueueConfig\n\nProvide either `redisConfig` (the library creates an ioredis client) or `client` (bring your own). At least one must be set.\n\n```ts\ninterface RedisJobQueueConfig {\n  backend: 'redis'; // Required\n  redisConfig?: {\n    url?: string;\n    host?: string;\n    port?: number;\n    password?: string;\n    db?: number;\n    tls?: RedisTLSConfig;\n    keyPrefix?: string; // Default: 'dq:'\n  };\n  client?: unknown; // Bring your own ioredis client\n  keyPrefix?: string; // Key prefix when using external client (default: 'dq:')\n  verbose?: boolean;\n}\n```\n\n#### JobQueueConfig\n\n```ts\ntype JobQueueConfig = PostgresJobQueueConfig | RedisJobQueueConfig;\n```\n\n#### DatabaseSSLConfig\n\n```ts\ninterface DatabaseSSLConfig {\n  ca?: string;\n  cert?: string;\n  key?: string;\n  rejectUnauthorized?: boolean;\n}\n```\n\n- `ca` - Client certificate authority (CA) as PEM string or file path. If the value starts with 'file://', it will be loaded from file, otherwise treated as PEM string.\n- `cert` - Client certificate as PEM string or file path. If the value starts with 'file://', it will be loaded from file, otherwise treated as PEM string.\n- `key` - Client private key as PEM string or file path. If the value starts with 'file://', it will be loaded from file, otherwise treated as PEM string.\n- `rejectUnauthorized` - Whether to reject unauthorized certificates (default: true)\n\n#### RedisTLSConfig\n\n```ts\ninterface RedisTLSConfig {\n  ca?: string;\n  cert?: string;\n  key?: string;\n  rejectUnauthorized?: boolean;\n}\n```\n\n---\n\n## Adding Jobs\n\n### addJob\n\n```ts\naddJob(job: JobOptions, options?: AddJobOptions): Promise<number>\n```\n\nAdds a job to the queue. Returns the job ID.\n\n#### JobOptions\n\n```ts\ninterface JobOptions {\n  jobType: string;\n  payload: any;\n  maxAttempts?: number;\n  priority?: number;\n  runAt?: Date | null;\n  timeoutMs?: number;\n  tags?: string[];\n  idempotencyKey?: string;\n  retryDelay?: number; // Base delay between retries in seconds (default: 60)\n  retryBackoff?: boolean; // Use exponential backoff (default: true)\n  retryDelayMax?: number; // Max delay cap in seconds (default: none)\n}\n```\n\n- `retryDelay` - Base delay between retries in seconds. When `retryBackoff` is true, this is the base for exponential backoff (`retryDelay * 2^attempts`). When false, retries use this fixed delay. Default: `60`.\n- `retryBackoff` - Whether to use exponential backoff. When true, delay doubles with each attempt and includes jitter. Default: `true`.\n- `retryDelayMax` - Maximum delay cap in seconds. Only meaningful when `retryBackoff` is true. No limit when omitted.\n\n#### AddJobOptions\n\n```ts\ninterface AddJobOptions {\n  db?: DatabaseClient;\n}\n```\n\n- `db` — An external database client (e.g., a `pg.PoolClient` inside a transaction). When provided, the INSERT runs on this client instead of the internal pool. **PostgreSQL only.** Throws if used with the Redis backend.\n\n### addJobs\n\n```ts\naddJobs(jobs: JobOptions[], options?: AddJobOptions): Promise<number[]>\n```\n\nAdds multiple jobs to the queue in a single operation. More efficient than calling `addJob` in a loop because it batches the INSERT into a single database round-trip (PostgreSQL) or a single atomic Lua script (Redis).\n\nReturns an array of job IDs in the same order as the input array.\n\nEach job can independently have its own `priority`, `runAt`, `tags`, `idempotencyKey`, and other options. Idempotency keys are handled per-job — duplicates resolve to the existing job's ID without creating a new row.\n\nPassing an empty array returns `[]` immediately without touching the database.\n\n```ts\nconst jobIds = await jobQueue.addJobs([\n  {\n    jobType: 'email',\n    payload: { to: 'a@example.com', subject: 'Hi', body: '...' },\n  },\n  {\n    jobType: 'email',\n    payload: { to: 'b@example.com', subject: 'Hi', body: '...' },\n    priority: 10,\n  },\n  {\n    jobType: 'report',\n    payload: { reportId: '123', userId: '456' },\n    tags: ['monthly'],\n  },\n]);\n// jobIds = [1, 2, 3]\n```\n\nThe `{ db }` option works the same as `addJob` — pass a transactional client to batch-insert within an existing transaction (PostgreSQL only).\n\n#### DatabaseClient\n\n```ts\ninterface DatabaseClient {\n  query(\n    text: string,\n    values?: any[],\n  ): Promise<{ rows: any[]; rowCount: number | null }>;\n}\n```\n\nAny object matching this interface works — `pg.Pool`, `pg.PoolClient`, `pg.Client`, or ORM query runners that expose a raw `query()` method.\n\n---\n\n## Retrieving Jobs\n\n### getJob\n\n```ts\ngetJob(id: number): Promise<JobRecord | null>\n```\n\nRetrieves a job by its ID.\n\n### getJobs\n\n```ts\ngetJobs(\n  filters?: {\n    jobType?: string;\n    priority?: number;\n    runAt?: Date | { gt?: Date; gte?: Date; lt?: Date; lte?: Date; eq?: Date };\n    tags?: { values: string[]; mode?: 'all' | 'any' | 'none' | 'exact' };\n  },\n  limit?: number,\n  offset?: number\n): Promise<JobRecord[]>\n```\n\nRetrieves jobs matching the provided filters, with optional pagination.\n\n### getJobsByStatus\n\n```ts\ngetJobsByStatus(status: string, limit?: number, offset?: number): Promise<JobRecord[]>\n```\n\nRetrieves jobs by their status, with pagination.\n\n### getAllJobs\n\n```ts\ngetAllJobs(limit?: number, offset?: number): Promise<JobRecord[]>\n```\n\nRetrieves all jobs, with optional pagination.\n\n### getJobsByTags\n\n```ts\ngetJobsByTags(tags: string[], mode?: TagQueryMode, limit?: number, offset?: number): Promise<JobRecord[]>\n```\n\nRetrieves jobs by tag(s).\n\n---\n\n## Managing Jobs\n\n### retryJob\n\n```ts\nretryJob(jobId: number): Promise<void>\n```\n\nRetries a job given its ID.\n\n### cancelJob\n\n```ts\ncancelJob(jobId: number): Promise<void>\n```\n\nCancels a job given its ID.\n\n### editJob\n\n```ts\neditJob(jobId: number, updates: EditJobOptions): Promise<void>\n```\n\nEdits a pending job given its ID. Only works for jobs with status 'pending'. Silently fails for other statuses (processing, completed, failed, cancelled).\n\n#### EditJobOptions\n\n```ts\ninterface EditJobOptions {\n  payload?: any;\n  maxAttempts?: number;\n  priority?: number;\n  runAt?: Date | null;\n  timeoutMs?: number;\n  tags?: string[];\n  retryDelay?: number | null;\n  retryBackoff?: boolean | null;\n  retryDelayMax?: number | null;\n}\n```\n\nAll fields are optional - only provided fields will be updated. Note that `jobType` cannot be changed. Set retry fields to `null` to revert to legacy default behavior.\n\n#### Example\n\n```ts\n// Edit a pending job's payload and priority\nawait jobQueue.editJob(jobId, {\n  payload: { to: 'newemail@example.com', subject: 'Updated' },\n  priority: 10,\n});\n\n// Edit only the scheduled run time\nawait jobQueue.editJob(jobId, {\n  runAt: new Date(Date.now() + 60000), // Run in 1 minute\n});\n\n// Edit multiple fields at once\nawait jobQueue.editJob(jobId, {\n  payload: { to: 'updated@example.com' },\n  priority: 5,\n  maxAttempts: 10,\n  timeoutMs: 30000,\n  tags: ['urgent', 'priority'],\n});\n```\n\n### editAllPendingJobs\n\n```ts\neditAllPendingJobs(\n  filters?: {\n    jobType?: string;\n    priority?: number;\n    runAt?: Date | { gt?: Date; gte?: Date; lt?: Date; lte?: Date; eq?: Date };\n    tags?: { values: string[]; mode?: 'all' | 'any' | 'none' | 'exact' };\n  },\n  updates: EditJobOptions\n): Promise<number>\n```\n\nEdits all pending jobs that match the filters. Only works for jobs with status 'pending'. Non-pending jobs are not affected. Returns the number of jobs that were edited.\n\n#### Parameters\n\n- `filters` (optional): Filters to select which jobs to edit. If not provided, all pending jobs are edited.\n  - `jobType`: Filter by job type\n  - `priority`: Filter by priority\n  - `runAt`: Filter by scheduled run time (supports `gt`, `gte`, `lt`, `lte`, `eq` operators or exact Date match)\n  - `tags`: Filter by tags with mode ('all', 'any', 'none', 'exact')\n- `updates`: The fields to update (same as `EditJobOptions`). All fields are optional - only provided fields will be updated.\n\n#### Returns\n\nThe number of jobs that were successfully edited.\n\n#### Examples\n\n```ts\n// Edit all pending jobs\nconst editedCount = await jobQueue.editAllPendingJobs(undefined, {\n  priority: 10,\n});\n\n// Edit all pending email jobs\nconst editedCount = await jobQueue.editAllPendingJobs(\n  { jobType: 'email' },\n  {\n    priority: 5,\n  },\n);\n\n// Edit all pending jobs with 'urgent' tag\nconst editedCount = await jobQueue.editAllPendingJobs(\n  { tags: { values: ['urgent'], mode: 'any' } },\n  {\n    priority: 10,\n    maxAttempts: 5,\n  },\n);\n\n// Edit all pending jobs scheduled in the future\nconst editedCount = await jobQueue.editAllPendingJobs(\n  { runAt: { gte: new Date() } },\n  {\n    priority: 10,\n  },\n);\n\n// Edit with combined filters\nconst editedCount = await jobQueue.editAllPendingJobs(\n  {\n    jobType: 'email',\n    tags: { values: ['urgent'], mode: 'any' },\n  },\n  {\n    priority: 10,\n    maxAttempts: 5,\n  },\n);\n```\n\n**Note:** Only pending jobs are edited. Jobs with other statuses (processing, completed, failed, cancelled) are not affected. Edit events are recorded for each affected job, just like single job edits.\n\n### cancelAllUpcomingJobs\n\n```ts\ncancelAllUpcomingJobs(filters?: {\n  jobType?: string;\n  priority?: number;\n  runAt?: Date | { gt?: Date; gte?: Date; lt?: Date; lte?: Date; eq?: Date };\n  tags?: { values: string[]; mode?: 'all' | 'any' | 'none' | 'exact' };\n}): Promise<number>\n```\n\nCancels all upcoming jobs that match the filters. Returns the number of jobs cancelled.\n\n### cleanupOldJobs\n\n```ts\ncleanupOldJobs(daysToKeep?: number): Promise<number>\n```\n\nCleans up jobs older than the specified number of days. Returns the number of jobs removed.\n\n### reclaimStuckJobs\n\n```ts\nreclaimStuckJobs(maxProcessingTimeMinutes?: number): Promise<number>\n```\n\nReclaims jobs stuck in 'processing' for too long. Returns the number of jobs reclaimed. If a job has a `timeoutMs` that is longer than the `maxProcessingTimeMinutes` threshold, the job's own timeout is used instead, preventing premature reclamation of long-running jobs.\n\n---\n\n## Job Events\n\n### getJobEvents\n\n```ts\ngetJobEvents(jobId: number): Promise<JobEvent[]>\n```\n\nRetrieves the job events for a job.\n\n#### JobEvent\n\n```ts\ninterface JobEvent {\n  id: number;\n  jobId: number;\n  eventType: JobEventType;\n  createdAt: Date;\n  metadata: any;\n}\n```\n\n#### JobEventType\n\n```ts\nenum JobEventType {\n  Added = 'added',\n  Processing = 'processing',\n  Completed = 'completed',\n  Failed = 'failed',\n  Cancelled = 'cancelled',\n  Retried = 'retried',\n  Edited = 'edited',\n}\n```\n\n---\n\n## Event Hooks\n\nDataQueue emits real-time events for job lifecycle transitions. Register listeners using `on`, `once`, `off`, and `removeAllListeners`. Works identically with both PostgreSQL and Redis backends.\n\n### QueueEventMap\n\n```ts\ninterface QueueEventMap {\n  'job:added': { jobId: number; jobType: string };\n  'job:processing': { jobId: number; jobType: string };\n  'job:completed': { jobId: number; jobType: string };\n  'job:failed': {\n    jobId: number;\n    jobType: string;\n    error: Error;\n    willRetry: boolean;\n  };\n  'job:cancelled': { jobId: number };\n  'job:retried': { jobId: number };\n  'job:waiting': { jobId: number; jobType: string };\n  'job:progress': { jobId: number; progress: number };\n  error: Error;\n}\n```\n\n### on\n\n```ts\non(event: QueueEventName, listener: (data) => void): void\n```\n\nRegister a listener that fires every time the event is emitted.\n\n### once\n\n```ts\nonce(event: QueueEventName, listener: (data) => void): void\n```\n\nRegister a one-time listener that auto-removes after the first invocation.\n\n### off\n\n```ts\noff(event: QueueEventName, listener: (data) => void): void\n```\n\nRemove a previously registered listener. Pass the exact function reference used with `on` or `once`.\n\n### removeAllListeners\n\n```ts\nremoveAllListeners(event?: QueueEventName): void\n```\n\nRemove all listeners for a specific event, or all listeners for all events when called without arguments.\n\nSee [Event Hooks](/usage/event-hooks) for detailed usage examples.\n\n---\n\n## Processing Jobs\n\n### createProcessor\n\n```ts\ncreateProcessor(\n  handlers: JobHandlers,\n  options?: ProcessorOptions\n): Processor\n```\n\nCreates a job processor with the provided handlers and options.\n\n#### ProcessorOptions\n\n```ts\ninterface ProcessorOptions {\n  workerId?: string;\n  batchSize?: number;\n  concurrency?: number;\n  pollInterval?: number;\n  onError?: (error: Error) => void;\n  verbose?: boolean;\n  jobType?: string | string[];\n}\n```\n\n---\n\n## Background Supervisor\n\n### createSupervisor\n\n```ts\ncreateSupervisor(options?: SupervisorOptions): Supervisor\n```\n\nCreates a background supervisor that automatically runs maintenance tasks on a configurable interval: reclaiming stuck jobs, cleaning up old completed jobs/events, and expiring timed-out waitpoint tokens.\n\n#### SupervisorOptions\n\n```ts\ninterface SupervisorOptions {\n  intervalMs?: number; // default: 60000\n  stuckJobsTimeoutMinutes?: number; // default: 10\n  cleanupJobsDaysToKeep?: number; // default: 30 (0 to disable)\n  cleanupEventsDaysToKeep?: number; // default: 30 (0 to disable)\n  cleanupBatchSize?: number; // default: 1000\n  reclaimStuckJobs?: boolean; // default: true\n  expireTimedOutTokens?: boolean; // default: true\n  onError?: (error: Error) => void; // default: console.error\n  verbose?: boolean;\n}\n```\n\n#### Supervisor\n\n```ts\ninterface Supervisor {\n  start(): Promise<SupervisorRunResult>;\n  startInBackground(): void;\n  stop(): void;\n  stopAndDrain(timeoutMs?: number): Promise<void>;\n  isRunning(): boolean;\n}\n```\n\n- `start()` runs all tasks once and returns the results (serverless-friendly).\n- `startInBackground()` starts a background loop that runs every `intervalMs`.\n- `stopAndDrain()` stops the loop and waits for the current run to finish.\n\n#### SupervisorRunResult\n\n```ts\ninterface SupervisorRunResult {\n  reclaimedJobs: number;\n  cleanedUpJobs: number;\n  cleanedUpEvents: number;\n  expiredTokens: number;\n}\n```\n\nSee [Long-Running Server](/usage/long-running-server#background-supervisor) for usage examples.\n\n---\n\n## Accessing the Underlying Client\n\n### getPool\n\n```ts\ngetPool(): Pool\n```\n\nReturns the PostgreSQL connection pool instance. Only available when using the PostgreSQL backend.\n\n> **Note:** Throws an error if called when using the Redis backend.\n\n### getRedisClient\n\n```ts\ngetRedisClient(): Redis\n```\n\nReturns the `ioredis` client instance. Only available when using the Redis backend.\n\n> **Note:** Throws an error if called when using the PostgreSQL backend."
+    "content": "## Initialization\n\n### initJobQueue\n\n```ts\ninitJobQueue(config: JobQueueConfig): JobQueue\n```\n\nInitializes the job queue system with the provided configuration. The `JobQueueConfig` is a discriminated union -- you provide either a PostgreSQL or Redis configuration.\n\n#### PostgresJobQueueConfig\n\nProvide either `databaseConfig` (the library creates a pool) or `pool` (bring your own `pg.Pool`). At least one must be set.\n\n```ts\ninterface PostgresJobQueueConfig {\n  backend?: 'postgres'; // Optional, defaults to 'postgres'\n  databaseConfig?: {\n    connectionString?: string;\n    host?: string;\n    port?: number;\n    database?: string;\n    user?: string;\n    password?: string;\n    ssl?: DatabaseSSLConfig;\n  };\n  pool?: import('pg').Pool; // Bring your own pool\n  verbose?: boolean;\n}\n```\n\n#### RedisJobQueueConfig\n\nProvide either `redisConfig` (the library creates an ioredis client) or `client` (bring your own). At least one must be set.\n\n```ts\ninterface RedisJobQueueConfig {\n  backend: 'redis'; // Required\n  redisConfig?: {\n    url?: string;\n    host?: string;\n    port?: number;\n    password?: string;\n    db?: number;\n    tls?: RedisTLSConfig;\n    keyPrefix?: string; // Default: 'dq:'\n  };\n  client?: unknown; // Bring your own ioredis client\n  keyPrefix?: string; // Key prefix when using external client (default: 'dq:')\n  verbose?: boolean;\n}\n```\n\n#### JobQueueConfig\n\n```ts\ntype JobQueueConfig = PostgresJobQueueConfig | RedisJobQueueConfig;\n```\n\n#### DatabaseSSLConfig\n\n```ts\ninterface DatabaseSSLConfig {\n  ca?: string;\n  cert?: string;\n  key?: string;\n  rejectUnauthorized?: boolean;\n}\n```\n\n- `ca` - Client certificate authority (CA) as PEM string or file path. If the value starts with 'file://', it will be loaded from file, otherwise treated as PEM string.\n- `cert` - Client certificate as PEM string or file path. If the value starts with 'file://', it will be loaded from file, otherwise treated as PEM string.\n- `key` - Client private key as PEM string or file path. If the value starts with 'file://', it will be loaded from file, otherwise treated as PEM string.\n- `rejectUnauthorized` - Whether to reject unauthorized certificates (default: true)\n\n#### RedisTLSConfig\n\n```ts\ninterface RedisTLSConfig {\n  ca?: string;\n  cert?: string;\n  key?: string;\n  rejectUnauthorized?: boolean;\n}\n```\n\n---\n\n## Adding Jobs\n\n### addJob\n\n```ts\naddJob(job: JobOptions, options?: AddJobOptions): Promise<number>\n```\n\nAdds a job to the queue. Returns the job ID.\n\n#### JobOptions\n\n```ts\ninterface JobOptions {\n  jobType: string;\n  payload: any;\n  maxAttempts?: number;\n  priority?: number;\n  runAt?: Date | null;\n  timeoutMs?: number;\n  tags?: string[];\n  idempotencyKey?: string;\n  retryDelay?: number; // Base delay between retries in seconds (default: 60)\n  retryBackoff?: boolean; // Use exponential backoff (default: true)\n  retryDelayMax?: number; // Max delay cap in seconds (default: none)\n  deadLetterJobType?: string; // Route exhausted failures to this job type\n  group?: { id: string; tier?: string }; // Optional group for global concurrency limits\n}\n```\n\n- `retryDelay` - Base delay between retries in seconds. When `retryBackoff` is true, this is the base for exponential backoff (`retryDelay * 2^attempts`). When false, retries use this fixed delay. Default: `60`.\n- `retryBackoff` - Whether to use exponential backoff. When true, delay doubles with each attempt and includes jitter. Default: `true`.\n- `retryDelayMax` - Maximum delay cap in seconds. Only meaningful when `retryBackoff` is true. No limit when omitted.\n- `deadLetterJobType` - Optional dead-letter destination. When retries are exhausted, a new pending job is created in this job type with an envelope payload (`originalJob`, `originalPayload`, `failure`).\n- `group` - Optional grouping metadata. Use `group.id` to enforce global per-group limits with `ProcessorOptions.groupConcurrency`. `group.tier` is reserved for future policies.\n\n#### AddJobOptions\n\n```ts\ninterface AddJobOptions {\n  db?: DatabaseClient;\n}\n```\n\n- `db` — An external database client (e.g., a `pg.PoolClient` inside a transaction). When provided, the INSERT runs on this client instead of the internal pool. **PostgreSQL only.** Throws if used with the Redis backend.\n\n### addJobs\n\n```ts\naddJobs(jobs: JobOptions[], options?: AddJobOptions): Promise<number[]>\n```\n\nAdds multiple jobs to the queue in a single operation. More efficient than calling `addJob` in a loop because it batches the INSERT into a single database round-trip (PostgreSQL) or a single atomic Lua script (Redis).\n\nReturns an array of job IDs in the same order as the input array.\n\nEach job can independently have its own `priority`, `runAt`, `tags`, `idempotencyKey`, and other options. Idempotency keys are handled per-job — duplicates resolve to the existing job's ID without creating a new row.\n\nPassing an empty array returns `[]` immediately without touching the database.\n\n```ts\nconst jobIds = await jobQueue.addJobs([\n  {\n    jobType: 'email',\n    payload: { to: 'a@example.com', subject: 'Hi', body: '...' },\n  },\n  {\n    jobType: 'email',\n    payload: { to: 'b@example.com', subject: 'Hi', body: '...' },\n    priority: 10,\n  },\n  {\n    jobType: 'report',\n    payload: { reportId: '123', userId: '456' },\n    tags: ['monthly'],\n  },\n]);\n// jobIds = [1, 2, 3]\n```\n\nThe `{ db }` option works the same as `addJob` — pass a transactional client to batch-insert within an existing transaction (PostgreSQL only).\n\n#### DatabaseClient\n\n```ts\ninterface DatabaseClient {\n  query(\n    text: string,\n    values?: any[],\n  ): Promise<{ rows: any[]; rowCount: number | null }>;\n}\n```\n\nAny object matching this interface works — `pg.Pool`, `pg.PoolClient`, `pg.Client`, or ORM query runners that expose a raw `query()` method.\n\n---\n\n## Retrieving Jobs\n\n### getJob\n\n```ts\ngetJob(id: number): Promise<JobRecord | null>\n```\n\nRetrieves a job by its ID.\n\n### getJobs\n\n```ts\ngetJobs(\n  filters?: {\n    jobType?: string;\n    priority?: number;\n    runAt?: Date | { gt?: Date; gte?: Date; lt?: Date; lte?: Date; eq?: Date };\n    tags?: { values: string[]; mode?: 'all' | 'any' | 'none' | 'exact' };\n  },\n  limit?: number,\n  offset?: number\n): Promise<JobRecord[]>\n```\n\nRetrieves jobs matching the provided filters, with optional pagination.\n\n### getJobsByStatus\n\n```ts\ngetJobsByStatus(status: string, limit?: number, offset?: number): Promise<JobRecord[]>\n```\n\nRetrieves jobs by their status, with pagination.\n\n### getAllJobs\n\n```ts\ngetAllJobs(limit?: number, offset?: number): Promise<JobRecord[]>\n```\n\nRetrieves all jobs, with optional pagination.\n\n### getJobsByTags\n\n```ts\ngetJobsByTags(tags: string[], mode?: TagQueryMode, limit?: number, offset?: number): Promise<JobRecord[]>\n```\n\nRetrieves jobs by tag(s).\n\n---\n\n## Managing Jobs\n\n### retryJob\n\n```ts\nretryJob(jobId: number): Promise<void>\n```\n\nRetries a job given its ID.\n\n### cancelJob\n\n```ts\ncancelJob(jobId: number): Promise<void>\n```\n\nCancels a job given its ID.\n\n### editJob\n\n```ts\neditJob(jobId: number, updates: EditJobOptions): Promise<void>\n```\n\nEdits a pending job given its ID. Only works for jobs with status 'pending'. Silently fails for other statuses (processing, completed, failed, cancelled).\n\n#### EditJobOptions\n\n```ts\ninterface EditJobOptions {\n  payload?: any;\n  maxAttempts?: number;\n  priority?: number;\n  runAt?: Date | null;\n  timeoutMs?: number;\n  tags?: string[];\n  retryDelay?: number | null;\n  retryBackoff?: boolean | null;\n  retryDelayMax?: number | null;\n  deadLetterJobType?: string | null;\n}\n```\n\nAll fields are optional - only provided fields will be updated. Note that `jobType` cannot be changed. Set retry fields to `null` to revert to legacy default behavior. Set `deadLetterJobType` to `null` to clear dead-letter routing for pending jobs.\n\n#### Example\n\n```ts\n// Edit a pending job's payload and priority\nawait jobQueue.editJob(jobId, {\n  payload: { to: 'newemail@example.com', subject: 'Updated' },\n  priority: 10,\n});\n\n// Edit only the scheduled run time\nawait jobQueue.editJob(jobId, {\n  runAt: new Date(Date.now() + 60000), // Run in 1 minute\n});\n\n// Edit multiple fields at once\nawait jobQueue.editJob(jobId, {\n  payload: { to: 'updated@example.com' },\n  priority: 5,\n  maxAttempts: 10,\n  timeoutMs: 30000,\n  tags: ['urgent', 'priority'],\n});\n```\n\n### editAllPendingJobs\n\n```ts\neditAllPendingJobs(\n  filters?: {\n    jobType?: string;\n    priority?: number;\n    runAt?: Date | { gt?: Date; gte?: Date; lt?: Date; lte?: Date; eq?: Date };\n    tags?: { values: string[]; mode?: 'all' | 'any' | 'none' | 'exact' };\n  },\n  updates: EditJobOptions\n): Promise<number>\n```\n\nEdits all pending jobs that match the filters. Only works for jobs with status 'pending'. Non-pending jobs are not affected. Returns the number of jobs that were edited.\n\n#### Parameters\n\n- `filters` (optional): Filters to select which jobs to edit. If not provided, all pending jobs are edited.\n  - `jobType`: Filter by job type\n  - `priority`: Filter by priority\n  - `runAt`: Filter by scheduled run time (supports `gt`, `gte`, `lt`, `lte`, `eq` operators or exact Date match)\n  - `tags`: Filter by tags with mode ('all', 'any', 'none', 'exact')\n- `updates`: The fields to update (same as `EditJobOptions`). All fields are optional - only provided fields will be updated.\n\n#### Returns\n\nThe number of jobs that were successfully edited.\n\n#### Examples\n\n```ts\n// Edit all pending jobs\nconst editedCount = await jobQueue.editAllPendingJobs(undefined, {\n  priority: 10,\n});\n\n// Edit all pending email jobs\nconst editedCount = await jobQueue.editAllPendingJobs(\n  { jobType: 'email' },\n  {\n    priority: 5,\n  },\n);\n\n// Edit all pending jobs with 'urgent' tag\nconst editedCount = await jobQueue.editAllPendingJobs(\n  { tags: { values: ['urgent'], mode: 'any' } },\n  {\n    priority: 10,\n    maxAttempts: 5,\n  },\n);\n\n// Edit all pending jobs scheduled in the future\nconst editedCount = await jobQueue.editAllPendingJobs(\n  { runAt: { gte: new Date() } },\n  {\n    priority: 10,\n  },\n);\n\n// Edit with combined filters\nconst editedCount = await jobQueue.editAllPendingJobs(\n  {\n    jobType: 'email',\n    tags: { values: ['urgent'], mode: 'any' },\n  },\n  {\n    priority: 10,\n    maxAttempts: 5,\n  },\n);\n```\n\n**Note:** Only pending jobs are edited. Jobs with other statuses (processing, completed, failed, cancelled) are not affected. Edit events are recorded for each affected job, just like single job edits.\n\n### cancelAllUpcomingJobs\n\n```ts\ncancelAllUpcomingJobs(filters?: {\n  jobType?: string;\n  priority?: number;\n  runAt?: Date | { gt?: Date; gte?: Date; lt?: Date; lte?: Date; eq?: Date };\n  tags?: { values: string[]; mode?: 'all' | 'any' | 'none' | 'exact' };\n}): Promise<number>\n```\n\nCancels all upcoming jobs that match the filters. Returns the number of jobs cancelled.\n\n### cleanupOldJobs\n\n```ts\ncleanupOldJobs(daysToKeep?: number): Promise<number>\n```\n\nCleans up jobs older than the specified number of days. Returns the number of jobs removed.\n\n### reclaimStuckJobs\n\n```ts\nreclaimStuckJobs(maxProcessingTimeMinutes?: number): Promise<number>\n```\n\nReclaims jobs stuck in 'processing' for too long. Returns the number of jobs reclaimed. If a job has a `timeoutMs` that is longer than the `maxProcessingTimeMinutes` threshold, the job's own timeout is used instead, preventing premature reclamation of long-running jobs.\n\n---\n\n## Job Events\n\n### getJobEvents\n\n```ts\ngetJobEvents(jobId: number): Promise<JobEvent[]>\n```\n\nRetrieves the job events for a job.\n\n#### JobEvent\n\n```ts\ninterface JobEvent {\n  id: number;\n  jobId: number;\n  eventType: JobEventType;\n  createdAt: Date;\n  metadata: any;\n}\n```\n\n#### JobEventType\n\n```ts\nenum JobEventType {\n  Added = 'added',\n  Processing = 'processing',\n  Completed = 'completed',\n  Failed = 'failed',\n  Cancelled = 'cancelled',\n  Retried = 'retried',\n  Edited = 'edited',\n}\n```\n\n---\n\n## Event Hooks\n\nDataQueue emits real-time events for job lifecycle transitions. Register listeners using `on`, `once`, `off`, and `removeAllListeners`. Works identically with both PostgreSQL and Redis backends.\n\n### QueueEventMap\n\n```ts\ninterface QueueEventMap {\n  'job:added': { jobId: number; jobType: string };\n  'job:processing': { jobId: number; jobType: string };\n  'job:completed': { jobId: number; jobType: string };\n  'job:failed': {\n    jobId: number;\n    jobType: string;\n    error: Error;\n    willRetry: boolean;\n  };\n  'job:cancelled': { jobId: number };\n  'job:retried': { jobId: number };\n  'job:waiting': { jobId: number; jobType: string };\n  'job:progress': { jobId: number; progress: number };\n  error: Error;\n}\n```\n\n### on\n\n```ts\non(event: QueueEventName, listener: (data) => void): void\n```\n\nRegister a listener that fires every time the event is emitted.\n\n### once\n\n```ts\nonce(event: QueueEventName, listener: (data) => void): void\n```\n\nRegister a one-time listener that auto-removes after the first invocation.\n\n### off\n\n```ts\noff(event: QueueEventName, listener: (data) => void): void\n```\n\nRemove a previously registered listener. Pass the exact function reference used with `on` or `once`.\n\n### removeAllListeners\n\n```ts\nremoveAllListeners(event?: QueueEventName): void\n```\n\nRemove all listeners for a specific event, or all listeners for all events when called without arguments.\n\nSee [Event Hooks](/usage/event-hooks) for detailed usage examples.\n\n---\n\n## Processing Jobs\n\n### createProcessor\n\n```ts\ncreateProcessor(\n  handlers: JobHandlers,\n  options?: ProcessorOptions\n): Processor\n```\n\nCreates a job processor with the provided handlers and options.\n\n#### ProcessorOptions\n\n```ts\ninterface ProcessorOptions {\n  workerId?: string;\n  batchSize?: number;\n  concurrency?: number;\n  groupConcurrency?: number;\n  pollInterval?: number;\n  onError?: (error: Error) => void;\n  verbose?: boolean;\n  jobType?: string | string[];\n}\n```\n\n- `groupConcurrency` - Optional global per-group concurrency limit (positive integer). Applies only to jobs with `group.id`; ungrouped jobs are unaffected.\n\n---\n\n## Background Supervisor\n\n### createSupervisor\n\n```ts\ncreateSupervisor(options?: SupervisorOptions): Supervisor\n```\n\nCreates a background supervisor that automatically runs maintenance tasks on a configurable interval: reclaiming stuck jobs, cleaning up old completed jobs/events, and expiring timed-out waitpoint tokens.\n\n#### SupervisorOptions\n\n```ts\ninterface SupervisorOptions {\n  intervalMs?: number; // default: 60000\n  stuckJobsTimeoutMinutes?: number; // default: 10\n  cleanupJobsDaysToKeep?: number; // default: 30 (0 to disable)\n  cleanupEventsDaysToKeep?: number; // default: 30 (0 to disable)\n  cleanupBatchSize?: number; // default: 1000\n  reclaimStuckJobs?: boolean; // default: true\n  expireTimedOutTokens?: boolean; // default: true\n  onError?: (error: Error) => void; // default: console.error\n  verbose?: boolean;\n}\n```\n\n#### Supervisor\n\n```ts\ninterface Supervisor {\n  start(): Promise<SupervisorRunResult>;\n  startInBackground(): void;\n  stop(): void;\n  stopAndDrain(timeoutMs?: number): Promise<void>;\n  isRunning(): boolean;\n}\n```\n\n- `start()` runs all tasks once and returns the results (serverless-friendly).\n- `startInBackground()` starts a background loop that runs every `intervalMs`.\n- `stopAndDrain()` stops the loop and waits for the current run to finish.\n\n#### SupervisorRunResult\n\n```ts\ninterface SupervisorRunResult {\n  reclaimedJobs: number;\n  cleanedUpJobs: number;\n  cleanedUpEvents: number;\n  expiredTokens: number;\n}\n```\n\nSee [Long-Running Server](/usage/long-running-server#background-supervisor) for usage examples.\n\n---\n\n## Accessing the Underlying Client\n\n### getPool\n\n```ts\ngetPool(): Pool\n```\n\nReturns the PostgreSQL connection pool instance. Only available when using the PostgreSQL backend.\n\n> **Note:** Throws an error if called when using the Redis backend.\n\n### getRedisClient\n\n```ts\ngetRedisClient(): Redis\n```\n\nReturns the `ioredis` client instance. Only available when using the Redis backend.\n\n> **Note:** Throws an error if called when using the PostgreSQL backend."
   },
   {
     "slug": "api/job-record",
     "title": "JobRecord",
     "description": "",
-    "content": "The `JobRecord` interface represents a job stored in the queue, including its status, attempts, and metadata.\n\n## Fields\n\n- `id`: _number_ — Unique job ID.\n- `jobType`: _string_ — The type of the job.\n- `payload`: _any_ — The job payload.\n- `status`:\n  _'pending' | 'processing' | 'completed' | 'failed' | 'cancelled'_ —\n  Current job status.\n- `createdAt`: _Date_ — When the job was created.\n- `updated_at`: _Date_ — When the job was last updated.\n- `locked_at`: _Date | null_ — When the job was locked for\n  processing.\n- `locked_by`: _string | null_ — Worker that locked the job.\n- `attempts`: _number_ — Number of attempts so far.\n- `maxAttempts`: _number_ — Maximum allowed attempts.\n- `nextAttemptAt`: _Date | null_ — When the next attempt is\n  scheduled.\n- `priority`: _number_ — Job priority.\n- `runAt`: _Date_ — When the job is scheduled to run.\n- `pendingReason?`: _string | null_ — Reason for pending\n  status.\n- `errorHistory?`: _\\{ message: string; timestamp: string \\}[]_ — Error history for the job.\n- `timeoutMs?`: _number | null_ — Timeout for this job in\n  milliseconds.\n- `failureReason?`: _FailureReason | null_ — Reason for last\n  failure, if any.\n- `completedAt`: _Date | null_ — When the job was completed.\n- `startedAt`: _Date | null_ — When the job was first picked up\n  for processing.\n- `lastRetriedAt`: _Date | null_ — When the job was last\n  retried.\n- `lastFailedAt`: _Date | null_ — When the job last failed.\n- `lastCancelledAt`: _Date | null_ — When the job was last\n  cancelled.\n- `tags?`: _string[]_ — Tags for this job. Used for grouping, searching, or batch operations.\n- `idempotencyKey?`: _string | null_ — The idempotency key for this job, if one was provided when the job was created.\n- `progress?`: _number | null_ — Progress percentage (0–100) reported by the handler via `ctx.setProgress()`. `null` if no progress has been reported. See [Progress Tracking](/usage/progress-tracking).\n- `output?`: _unknown_ — Handler output stored via `ctx.setOutput(data)` or by returning a value from the handler. `null` if no output has been stored. See [Job Output](/usage/job-output).\n\n## Example\n\n```json\n{\n  \"id\": 1,\n  \"jobType\": \"email\",\n  \"payload\": { \"to\": \"user@example.com\", \"subject\": \"Hello\" },\n  \"status\": \"completed\",\n  \"createdAt\": \"2024-06-01T12:00:00Z\",\n  \"tags\": [\"welcome\", \"user\"],\n  \"idempotencyKey\": \"welcome-email-user-123\",\n  \"progress\": 100,\n  \"output\": { \"messageId\": \"abc-123\", \"sentAt\": \"2024-06-01T12:00:05Z\" }\n}\n```"
+    "content": "The `JobRecord` interface represents a job stored in the queue, including its status, attempts, and metadata.\n\n## Fields\n\n- `id`: _number_ — Unique job ID.\n- `jobType`: _string_ — The type of the job.\n- `payload`: _any_ — The job payload.\n- `status`:\n  _'pending' | 'processing' | 'completed' | 'failed' | 'cancelled' | 'waiting'_ —\n  Current job status.\n- `createdAt`: _Date_ — When the job was created.\n- `updated_at`: _Date_ — When the job was last updated.\n- `locked_at`: _Date | null_ — When the job was locked for\n  processing.\n- `locked_by`: _string | null_ — Worker that locked the job.\n- `attempts`: _number_ — Number of attempts so far.\n- `maxAttempts`: _number_ — Maximum allowed attempts.\n- `nextAttemptAt`: _Date | null_ — When the next attempt is\n  scheduled.\n- `priority`: _number_ — Job priority.\n- `runAt`: _Date_ — When the job is scheduled to run.\n- `pendingReason?`: _string | null_ — Reason for pending\n  status.\n- `errorHistory?`: _\\{ message: string; timestamp: string \\}[]_ — Error history for the job.\n- `timeoutMs?`: _number | null_ — Timeout for this job in\n  milliseconds.\n- `failureReason?`: _FailureReason | null_ — Reason for last\n  failure, if any.\n- `completedAt`: _Date | null_ — When the job was completed.\n- `startedAt`: _Date | null_ — When the job was first picked up\n  for processing.\n- `lastRetriedAt`: _Date | null_ — When the job was last\n  retried.\n- `lastFailedAt`: _Date | null_ — When the job last failed.\n- `lastCancelledAt`: _Date | null_ — When the job was last\n  cancelled.\n- `tags?`: _string[]_ — Tags for this job. Used for grouping, searching, or batch operations.\n- `idempotencyKey?`: _string | null_ — The idempotency key for this job, if one was provided when the job was created.\n- `progress?`: _number | null_ — Progress percentage (0–100) reported by the handler via `ctx.setProgress()`. `null` if no progress has been reported. See [Progress Tracking](/usage/progress-tracking).\n- `output?`: _unknown_ — Handler output stored via `ctx.setOutput(data)` or by returning a value from the handler. `null` if no output has been stored. See [Job Output](/usage/job-output).\n- `deadLetterJobType?`: _string | null_ — Configured dead-letter destination job type for this job.\n- `deadLetteredAt?`: _Date | null_ — Timestamp when this job was routed to a dead-letter job.\n- `deadLetterJobId?`: _number | null_ — Linked dead-letter job ID created when retries were exhausted.\n\n## Example\n\n```json\n{\n  \"id\": 1,\n  \"jobType\": \"email\",\n  \"payload\": { \"to\": \"user@example.com\", \"subject\": \"Hello\" },\n  \"status\": \"completed\",\n  \"createdAt\": \"2024-06-01T12:00:00Z\",\n  \"tags\": [\"welcome\", \"user\"],\n  \"idempotencyKey\": \"welcome-email-user-123\",\n  \"progress\": 100,\n  \"output\": { \"messageId\": \"abc-123\", \"sentAt\": \"2024-06-01T12:00:05Z\" }\n}\n```"
   },
   {
     "slug": "api/processor",
     "title": "Processor",
     "description": "",
-    "content": "The `Processor` interface represents a job processor that can process jobs from the queue, either in the background or synchronously.\n\n## Creating a processor\n\nCreate a processor by calling `createProcessor` on the queue.\n\n```ts\nconst jobQueue = getJobQueue();\nconst processor = queue.createProcessor(handlers, options);\n```\n\n### ProcessorOptions\n\n```ts\ninterface ProcessorOptions {\n  workerId?: string;\n  batchSize?: number;\n  concurrency?: number;\n  pollInterval?: number;\n  onError?: (error: Error) => void;\n  verbose?: boolean;\n  jobType?: string | string[];\n}\n```\n\n## Methods\n\n### startInBackground\n\n```ts\nstartInBackground(): void\n```\n\nStart the job processor in the background. This will run continuously and process jobs as they become available. It polls for new jobs every `pollInterval` milliseconds (default: 5 seconds).\n\n### stop\n\n```ts\nstop(): void\n```\n\nStop the job processor that runs in the background. Does not wait for in-flight jobs to finish.\n\n### stopAndDrain\n\n```ts\nstopAndDrain(timeoutMs?: number): Promise<void>\n```\n\nStop the processor and wait for the current in-flight batch to finish before resolving. Accepts an optional timeout in milliseconds (default: `30000`). If the batch does not complete within the timeout, the promise resolves anyway so your process is not stuck indefinitely. Useful for graceful shutdown (e.g., SIGTERM handling). See [Long-Running Server](/usage/long-running-server) for a full example.\n\n### isRunning\n\n```ts\nisRunning(): boolean\n```\n\nCheck if the job processor is running.\n\n### start\n\n```ts\nstart(): Promise<number>\n```\n\nStart the job processor synchronously. This will process jobs immediately and then stop. Returns the number of jobs processed."
+    "content": "The `Processor` interface represents a job processor that can process jobs from the queue, either in the background or synchronously.\n\n## Creating a processor\n\nCreate a processor by calling `createProcessor` on the queue.\n\n```ts\nconst jobQueue = getJobQueue();\nconst processor = queue.createProcessor(handlers, options);\n```\n\n### ProcessorOptions\n\n```ts\ninterface ProcessorOptions {\n  workerId?: string;\n  batchSize?: number;\n  concurrency?: number;\n  groupConcurrency?: number;\n  pollInterval?: number;\n  onError?: (error: Error) => void;\n  verbose?: boolean;\n  jobType?: string | string[];\n}\n```\n\n- `groupConcurrency` sets a global per-group concurrency cap across all workers/instances for jobs with `group.id`.\n- Must be a positive integer when provided.\n- Jobs without `group.id` are not affected.\n\n## Methods\n\n### startInBackground\n\n```ts\nstartInBackground(): void\n```\n\nStart the job processor in the background. This will run continuously and process jobs as they become available. It polls for new jobs every `pollInterval` milliseconds (default: 5 seconds).\n\n### stop\n\n```ts\nstop(): void\n```\n\nStop the job processor that runs in the background. Does not wait for in-flight jobs to finish.\n\n### stopAndDrain\n\n```ts\nstopAndDrain(timeoutMs?: number): Promise<void>\n```\n\nStop the processor and wait for the current in-flight batch to finish before resolving. Accepts an optional timeout in milliseconds (default: `30000`). If the batch does not complete within the timeout, the promise resolves anyway so your process is not stuck indefinitely. Useful for graceful shutdown (e.g., SIGTERM handling). See [Long-Running Server](/usage/long-running-server) for a full example.\n\n### isRunning\n\n```ts\nisRunning(): boolean\n```\n\nCheck if the job processor is running.\n\n### start\n\n```ts\nstart(): Promise<number>\n```\n\nStart the job processor synchronously. This will process jobs immediately and then stop. Returns the number of jobs processed."
   },
   {
     "slug": "api/tags",
@@ -117,7 +117,7 @@
     "slug": "intro/comparison",
     "title": "Comparison",
     "description": "How DataQueue compares to BullMQ and Trigger.dev",
-    "content": "Choosing a job queue depends on your stack, infrastructure preferences, and the features you need. Here is a side-by-side comparison of **DataQueue**, **BullMQ**, and **Trigger.dev**.\n\n| Feature                 | DataQueue                                       | BullMQ                                      | Trigger.dev                             |\n| ----------------------- | ----------------------------------------------- | ------------------------------------------- | --------------------------------------- |\n| **Backend**             | PostgreSQL or Redis                             | Redis only                                  | Cloud or self-hosted (Postgres + Redis) |\n| **Type Safety**         | Full generic `PayloadMap`                       | Basic types                                 | Full TypeScript tasks                   |\n| **Scheduling**          | `runAt`, Cron                                   | Cron, delayed, recurring                    | Cron, delayed                           |\n| **Retries**             | Exponential backoff, configurable `maxAttempts` | Exponential backoff, custom strategies, DLQ | Auto retries, bulk replay, DLQ          |\n| **Priority**            | Integer priority                                | Priority levels                             | Queue-based priority                    |\n| **Concurrency Control** | `batchSize` + `concurrency`                     | Built-in                                    | Per-task + shared limits                |\n| **Rate Limiting**       | -                                               | Yes                                         | Via concurrency limits                  |\n| **Job Flows / DAGs**    | -                                               | Parent-child flows                          | Workflows                               |\n| **Dashboard**           | Built-in Next.js package                        | Third-party (Bull Board, etc.)              | Built-in web dashboard                  |\n| **Wait / Pause Jobs**   | `waitFor`, `waitUntil`, token system            | -                                           | Durable execution                       |\n| **Human-in-the-Loop**   | Token system                                    | -                                           | Yes                                     |\n| **Progress Tracking**   | Yes (0-100%)                                    | Yes                                         | Yes (realtime)                          |\n| **Serverless-First**    | Yes                                             | No (needs long-running process)             | Yes (cloud)                             |\n| **Self-Hosted**         | Yes                                             | Yes (your Redis)                            | Yes (containers)                        |\n| **Cloud Option**        | -                                               | -                                           | Yes                                     |\n| **License**             | MIT                                             | MIT                                         | Apache-2.0                              |\n| **Pricing**             | Free (OSS)                                      | Free (OSS)                                  | Free tier + paid plans                  |\n| **Infrastructure**      | Your own Postgres or Redis                      | Your own Redis                              | Their cloud or your infra               |\n\n## Where DataQueue shines\n\n- **Serverless-first** — designed from the ground up for Vercel, AWS Lambda, and other serverless platforms. No long-running process required.\n- **Use your existing database** — back your queue with PostgreSQL or Redis. No additional infrastructure to provision or pay for.\n- **Wait and token system** — pause jobs with `waitFor`, `waitUntil`, or token-based waits for human-in-the-loop workflows, all within a single handler function.\n- **Type-safe PayloadMap** — a generic `PayloadMap` gives you compile-time validation of every job type and its payload, catching bugs before they reach production.\n- **Built-in Next.js dashboard** — add a full admin UI to your Next.js app with a single route file. No separate service to deploy."
+    "content": "Choosing a job queue depends on your stack, infrastructure preferences, and the features you need. Here is a side-by-side comparison of **DataQueue**, **BullMQ**, and **Trigger.dev**.\n\n| Feature                 | DataQueue                                               | BullMQ                                      | Trigger.dev                             |\n| ----------------------- | ------------------------------------------------------- | ------------------------------------------- | --------------------------------------- |\n| **Backend**             | PostgreSQL or Redis                                     | Redis only                                  | Cloud or self-hosted (Postgres + Redis) |\n| **Type Safety**         | Full generic `PayloadMap`                               | Basic types                                 | Full TypeScript tasks                   |\n| **Scheduling**          | `runAt`, Cron                                           | Cron, delayed, recurring                    | Cron, delayed                           |\n| **Retries**             | Exponential backoff, configurable `maxAttempts`         | Exponential backoff, custom strategies, DLQ | Auto retries, bulk replay, DLQ          |\n| **Priority**            | Integer priority                                        | Priority levels                             | Queue-based priority                    |\n| **Concurrency Control** | `batchSize` + `concurrency` + global `groupConcurrency` | Built-in                                    | Per-task + shared limits                |\n| **Rate Limiting**       | -                                                       | Yes                                         | Via concurrency limits                  |\n| **Job Flows / DAGs**    | -                                                       | Parent-child flows                          | Workflows                               |\n| **Dashboard**           | Built-in Next.js package                                | Third-party (Bull Board, etc.)              | Built-in web dashboard                  |\n| **Wait / Pause Jobs**   | `waitFor`, `waitUntil`, token system                    | -                                           | Durable execution                       |\n| **Human-in-the-Loop**   | Token system                                            | -                                           | Yes                                     |\n| **Progress Tracking**   | Yes (0-100%)                                            | Yes                                         | Yes (realtime)                          |\n| **Serverless-First**    | Yes                                                     | No (needs long-running process)             | Yes (cloud)                             |\n| **Self-Hosted**         | Yes                                                     | Yes (your Redis)                            | Yes (containers)                        |\n| **Cloud Option**        | -                                                       | -                                           | Yes                                     |\n| **License**             | MIT                                                     | MIT                                         | Apache-2.0                              |\n| **Pricing**             | Free (OSS)                                              | Free (OSS)                                  | Free tier + paid plans                  |\n| **Infrastructure**      | Your own Postgres or Redis                              | Your own Redis                              | Their cloud or your infra               |\n\n## Where DataQueue shines\n\n- **Serverless-first** — designed from the ground up for Vercel, AWS Lambda, and other serverless platforms. No long-running process required.\n- **Use your existing database** — back your queue with PostgreSQL or Redis. No additional infrastructure to provision or pay for.\n- **Wait and token system** — pause jobs with `waitFor`, `waitUntil`, or token-based waits for human-in-the-loop workflows, all within a single handler function.\n- **Type-safe PayloadMap** — a generic `PayloadMap` gives you compile-time validation of every job type and its payload, catching bugs before they reach production.\n- **Built-in Next.js dashboard** — add a full admin UI to your Next.js app with a single route file. No separate service to deploy."
   },
   {
     "slug": "intro",
@@ -147,7 +147,7 @@
     "slug": "usage/building-with-ai",
     "title": "Building with AI",
     "description": "Tools and resources for building DataQueue projects with AI coding assistants.",
-    "content": "We provide multiple tools to help AI coding assistants write correct DataQueue code. Use one or all of them for the best developer experience.\n\n## Quick Setup\n\n### 1. Install Skills\n\nPortable instruction sets that teach any AI coding assistant DataQueue best practices.\n\n```bash\nnpx dataqueue-cli install-skills\n```\n\nSkills are installed as `SKILL.md` files into your AI tool's skills directory (`.cursor/skills/`, `.claude/skills/`, etc.). They cover core patterns, advanced features (waits, cron, tokens), and React/Dashboard integration.\n\n### 2. Install Agent Rules\n\nComprehensive rule sets installed directly into your AI client's config files.\n\n```bash\nnpx dataqueue-cli install-rules\n```\n\nThe installer prompts you to choose your AI client and writes rules to the appropriate location:\n\n| Client         | Installs to                       |\n| -------------- | --------------------------------- |\n| Cursor         | `.cursor/rules/dataqueue-*.mdc`   |\n| Claude Code    | `CLAUDE.md`                       |\n| AGENTS.md      | `AGENTS.md`                       |\n| GitHub Copilot | `.github/copilot-instructions.md` |\n| Windsurf       | `CONVENTIONS.md`                  |\n\n### 3. Install MCP Server\n\nGive your AI assistant direct access to DataQueue documentation — search docs, fetch specific pages, and list all available topics.\n\n```bash\nnpx dataqueue-cli install-mcp\n```\n\nThe installer prompts you to choose your AI client and writes the MCP config to the appropriate location. Currently supported clients:\n\n| Client            | Installs to                           |\n| ----------------- | ------------------------------------- |\n| Cursor            | `.cursor/mcp.json`                    |\n| Claude Code       | `.mcp.json`                           |\n| VS Code (Copilot) | `.vscode/mcp.json`                    |\n| Windsurf          | `~/.codeium/windsurf/mcp_config.json` |\n\nThe MCP server runs via `npx dataqueue-cli mcp` and communicates over stdio. It exposes three tools:\n\n| Tool             | Description                              |\n| ---------------- | ---------------------------------------- |\n| `search-docs`    | Full-text search across all doc pages    |\n| `get-doc-page`   | Fetch a specific doc page by slug        |\n| `list-doc-pages` | List all available doc pages with titles |\n\n## Skills vs Agent Rules vs MCP\n\n|                   | **Skills**                           | **Agent Rules**                        | **MCP Server**                         |\n| :---------------- | :----------------------------------- | :------------------------------------- | :------------------------------------- |\n| **What it does**  | Drops skill files into your project  | Installs rule sets into client config  | Runs a live server your AI connects to |\n| **Installs to**   | `.cursor/skills/`, `.claude/skills/` | `.cursor/rules/`, `CLAUDE.md`, etc.    | `.cursor/mcp.json`, `.mcp.json`, etc.  |\n| **Best for**      | Teaching patterns and best practices | Comprehensive code generation guidance | Live documentation search              |\n| **Works offline** | Yes                                  | Yes                                    | Yes (runs locally)                     |\n\n**Recommendation:** Install all three. Skills and Agent Rules teach your AI _how_ to write code. The MCP Server lets it _look up_ the docs when it needs specifics.\n\n## llms.txt\n\nWe publish machine-readable documentation for LLM consumption:\n\n- [docs.dataqueue.dev/llms.txt](https://docs.dataqueue.dev/llms.txt) — concise overview\n- [docs.dataqueue.dev/llms-full.txt](https://docs.dataqueue.dev/llms-full.txt) — full documentation\n\nThese follow the [llms.txt standard](https://llmstxt.org) and can be fed directly into any LLM context window.\n\n## Project-Level Context Snippet\n\nIf you prefer a lightweight approach, paste this snippet into a context file at the root of your project:\n\n| File                              | Read by                       |\n| :-------------------------------- | :---------------------------- |\n| `CLAUDE.md`                       | Claude Code                   |\n| `AGENTS.md`                       | OpenAI Codex, Jules, OpenCode |\n| `.cursor/rules/*.md`              | Cursor                        |\n| `.github/copilot-instructions.md` | GitHub Copilot                |\n| `CONVENTIONS.md`                  | Windsurf, Cline, and others   |\n\n```markdown\n# DataQueue rules\n\n## Imports\n\nAlways import from `@nicnocquee/dataqueue`.\n\n## PayloadMap pattern\n\nDefine a type map of job types to payload shapes for full type safety:\n\n\\`\\`\\`ts\ntype JobPayloadMap = {\nsend_email: { to: string; subject: string; body: string };\n};\n\\`\\`\\`\n\n## Initialization (singleton)\n\nNever call initJobQueue per request — use a module-level singleton:\n\n\\`\\`\\`ts\nimport { initJobQueue } from '@nicnocquee/dataqueue';\nlet queue: ReturnType<typeof initJobQueue<JobPayloadMap>> | null = null;\nexport const getJobQueue = () => {\nif (!queue) {\nqueue = initJobQueue<JobPayloadMap>({\ndatabaseConfig: { connectionString: process.env.PG_DATAQUEUE_DATABASE },\n});\n}\nreturn queue;\n};\n\\`\\`\\`\n\n## Handler pattern\n\nType handlers as `JobHandlers<PayloadMap>` — TypeScript enforces a handler for every job type.\n\n## Processing\n\n- Serverless: `processor.start()` (one-shot)\n- Long-running: `processor.startInBackground()` + `stopAndDrain()` on SIGTERM\n\n## Common mistakes\n\n1. Creating initJobQueue per request (creates a DB pool each time)\n2. Missing handler for a job type (fails with NoHandler)\n3. Not checking signal.aborted in long handlers\n4. Forgetting reclaimStuckJobs() — crashed workers leave jobs stuck\n5. Skipping migrations (PostgreSQL requires `dataqueue-cli migrate`)\n```"
+    "content": "We provide multiple tools to help AI coding assistants write correct DataQueue code. Use one or all of them for the best developer experience.\n\n## Quick Setup\n\n### 1. Install Skills\n\nPortable instruction sets that teach any AI coding assistant DataQueue best practices.\n\n```bash\nnpx dataqueue-cli install-skills\n```\n\nSkills are installed as `SKILL.md` files into your AI tool's skills directory (`.cursor/skills/`, `.claude/skills/`, etc.). They cover core patterns, advanced features (waits, cron, tokens), and React/Dashboard integration.\n\n### 2. Install Agent Rules\n\nComprehensive rule sets installed directly into your AI client's config files.\n\n```bash\nnpx dataqueue-cli install-rules\n```\n\nThe installer prompts you to choose your AI client and writes rules to the appropriate location:\n\n| Client         | Installs to                       |\n| -------------- | --------------------------------- |\n| Cursor         | `.cursor/rules/dataqueue-*.mdc`   |\n| Claude Code    | `CLAUDE.md`                       |\n| AGENTS.md      | `AGENTS.md`                       |\n| GitHub Copilot | `.github/copilot-instructions.md` |\n| Windsurf       | `CONVENTIONS.md`                  |\n\n### 3. Install MCP Server\n\nGive your AI assistant direct access to DataQueue documentation — search docs, fetch specific pages, and list all available topics.\n\n```bash\nnpx dataqueue-cli install-mcp\n```\n\nThe installer prompts you to choose your AI client and writes the MCP config to the appropriate location. Currently supported clients:\n\n| Client            | Installs to                           |\n| ----------------- | ------------------------------------- |\n| Cursor            | `.cursor/mcp.json`                    |\n| Claude Code       | `.mcp.json`                           |\n| VS Code (Copilot) | `.vscode/mcp.json`                    |\n| Windsurf          | `~/.codeium/windsurf/mcp_config.json` |\n\nThe MCP server runs via `npx dataqueue-cli mcp` and communicates over stdio. It exposes three tools:\n\n| Tool             | Description                              |\n| ---------------- | ---------------------------------------- |\n| `search-docs`    | Full-text search across all doc pages    |\n| `get-doc-page`   | Fetch a specific doc page by slug        |\n| `list-doc-pages` | List all available doc pages with titles |\n\n## Skills vs Agent Rules vs MCP\n\n|                   | **Skills**                           | **Agent Rules**                        | **MCP Server**                         |\n| :---------------- | :----------------------------------- | :------------------------------------- | :------------------------------------- |\n| **What it does**  | Drops skill files into your project  | Installs rule sets into client config  | Runs a live server your AI connects to |\n| **Installs to**   | `.cursor/skills/`, `.claude/skills/` | `.cursor/rules/`, `CLAUDE.md`, etc.    | `.cursor/mcp.json`, `.mcp.json`, etc.  |\n| **Best for**      | Teaching patterns and best practices | Comprehensive code generation guidance | Live documentation search              |\n| **Works offline** | Yes                                  | Yes                                    | Yes (runs locally)                     |\n\n**Recommendation:** Install all three. Skills and Agent Rules teach your AI _how_ to write code. The MCP Server lets it _look up_ the docs when it needs specifics.\n\n## llms.txt\n\nWe publish machine-readable documentation for LLM consumption:\n\n- [docs.dataqueue.dev/llms.txt](https://docs.dataqueue.dev/llms.txt) — concise overview\n- [docs.dataqueue.dev/llms-full.txt](https://docs.dataqueue.dev/llms-full.txt) — full documentation\n\nThese follow the [llms.txt standard](https://llmstxt.org) and can be fed directly into any LLM context window.\n\n## Project-Level Context Snippet\n\nIf you prefer a lightweight approach, paste this snippet into a context file at the root of your project:\n\n| File                              | Read by                       |\n| :-------------------------------- | :---------------------------- |\n| `CLAUDE.md`                       | Claude Code                   |\n| `AGENTS.md`                       | OpenAI Codex, Jules, OpenCode |\n| `.cursor/rules/*.md`              | Cursor                        |\n| `.github/copilot-instructions.md` | GitHub Copilot                |\n| `CONVENTIONS.md`                  | Windsurf, Cline, and others   |\n\n```markdown\n# DataQueue rules\n\n## Imports\n\nAlways import from `@nicnocquee/dataqueue`.\n\n## PayloadMap pattern\n\nDefine a type map of job types to payload shapes for full type safety:\n\n\\`\\`\\`ts\ntype JobPayloadMap = {\nsend_email: { to: string; subject: string; body: string };\n};\n\\`\\`\\`\n\n## Initialization (singleton)\n\nNever call initJobQueue per request — use a module-level singleton:\n\n\\`\\`\\`ts\nimport { initJobQueue } from '@nicnocquee/dataqueue';\nlet queue: ReturnType<typeof initJobQueue<JobPayloadMap>> | null = null;\nexport const getJobQueue = () => {\nif (!queue) {\nqueue = initJobQueue<JobPayloadMap>({\ndatabaseConfig: { connectionString: process.env.PG_DATAQUEUE_DATABASE },\n});\n}\nreturn queue;\n};\n\\`\\`\\`\n\n## Handler pattern\n\nType handlers as `JobHandlers<PayloadMap>` — TypeScript enforces a handler for every job type.\n\n## Processing\n\n- Serverless: `processor.start()` (one-shot)\n- Long-running: `processor.startInBackground()` + `stopAndDrain()` on SIGTERM\n\n## Common mistakes\n\n1. Creating initJobQueue per request (creates a DB pool each time)\n2. Missing handler for a job type (fails with NoHandler)\n3. Not checking signal.aborted in long handlers\n4. Forgetting dead-letter routing for critical jobs — set `deadLetterJobType` so exhausted failures are inspectable/replayable\n5. Forgetting reclaimStuckJobs() — crashed workers leave jobs stuck\n6. Skipping migrations (PostgreSQL requires `dataqueue-cli migrate`)\n```"
   },
   {
     "slug": "usage/cancel-jobs",
@@ -195,7 +195,7 @@
     "slug": "usage/failed-jobs",
     "title": "Failed Jobs",
     "description": "",
-    "content": "A job handler can fail for many reasons, such as a bug in the code or running out of resources.\n\nWhen a job fails, it is marked as `failed` and retried up to `maxAttempts` times (default: 3). You can view the error history for a job in its `errorHistory` field.\n\n## Retry configuration\n\nYou can control the retry behavior per-job using three options:\n\n| Option          | Type      | Default | Description                                    |\n| --------------- | --------- | ------- | ---------------------------------------------- |\n| `retryDelay`    | `number`  | `60`    | Base delay between retries in **seconds**      |\n| `retryBackoff`  | `boolean` | `true`  | Use exponential backoff (doubles each attempt) |\n| `retryDelayMax` | `number`  | _none_  | Maximum cap for the delay in **seconds**       |\n\n### Fixed delay\n\nSet `retryBackoff: false` to use a constant delay between retries:\n\n```ts\nawait jobQueue.addJob({\n  jobType: 'email',\n  payload: { to: 'user@example.com' },\n  maxAttempts: 5,\n  retryDelay: 30, // 30 seconds between each retry\n  retryBackoff: false,\n});\n```\n\nEvery retry will wait exactly 30 seconds.\n\n### Exponential backoff (default)\n\nWhen `retryBackoff` is `true` (the default), the delay doubles with each attempt. A small amount of random jitter is added to prevent thundering herd problems:\n\n```ts\nawait jobQueue.addJob({\n  jobType: 'email',\n  payload: { to: 'user@example.com' },\n  maxAttempts: 5,\n  retryDelay: 10, // base: 10 seconds\n  retryBackoff: true, // enabled by default\n});\n```\n\nThis produces approximate delays of 10s, 20s, 40s, 80s, ... (with jitter).\n\n### Capping the delay\n\nUse `retryDelayMax` to prevent the delay from growing unbounded:\n\n```ts\nawait jobQueue.addJob({\n  jobType: 'email',\n  payload: { to: 'user@example.com' },\n  maxAttempts: 10,\n  retryDelay: 5,\n  retryBackoff: true,\n  retryDelayMax: 300, // never wait more than 5 minutes\n});\n```\n\nDelays: ~5s, ~10s, ~20s, ~40s, ~80s, ~160s, ~300s, ~300s, ...\n\n### Default behavior\n\nIf none of the retry options are set, the legacy formula `2^attempts * 1 minute` is used. This means the first retry is after ~2 minutes, then ~4 minutes, then ~8 minutes, and so on.\n\n## Jitter\n\nWhen exponential backoff is enabled, each computed delay is multiplied by a random factor between 0.5 and 1.0. This prevents multiple failed jobs from retrying at exactly the same time, which could overload downstream services.\n\n## Cron schedules\n\nRetry configuration can also be set on cron schedules. Every job enqueued by the schedule inherits the retry settings:\n\n```ts\nawait jobQueue.addCronJob({\n  scheduleName: 'daily-report',\n  cronExpression: '0 9 * * *',\n  jobType: 'report',\n  payload: { type: 'daily' },\n  retryDelay: 60,\n  retryBackoff: true,\n  retryDelayMax: 600,\n});\n```\n\n## Editing retry config\n\nYou can update the retry configuration of a pending job:\n\n```ts\nawait jobQueue.editJob(jobId, {\n  retryDelay: 15,\n  retryBackoff: false,\n});\n```"
+    "content": "A job handler can fail for many reasons, such as a bug in the code or running out of resources.\n\nWhen a job fails, it is marked as `failed` and retried up to `maxAttempts` times (default: 3). You can view the error history for a job in its `errorHistory` field.\n\n## Dead-letter queues\n\nYou can route permanently failed jobs to a dead-letter job type using `deadLetterJobType`.\n\nWhen a job exhausts retries (`attempts >= maxAttempts`), DataQueue:\n\n1. Keeps the source job as `failed`.\n2. Creates a new pending dead-letter job in `deadLetterJobType`.\n3. Stores linkage metadata on the source job (`deadLetteredAt`, `deadLetterJobId`).\n\n```ts\nawait jobQueue.addJob({\n  jobType: 'email',\n  payload: { to: 'user@example.com' },\n  maxAttempts: 3,\n  deadLetterJobType: 'email_dead_letter',\n});\n```\n\nThe dead-letter job payload is an envelope:\n\n```ts\n{\n  originalJob: { id, jobType, attempts, maxAttempts },\n  originalPayload: { ... }, // original job payload\n  failure: { message, reason, failedAt },\n}\n```\n\nIf `deadLetterJobType` is not set, behavior is unchanged: exhausted jobs remain failed without creating a dead-letter job.\n\n## Retry configuration\n\nYou can control the retry behavior per-job using three options:\n\n| Option          | Type      | Default | Description                                    |\n| --------------- | --------- | ------- | ---------------------------------------------- |\n| `retryDelay`    | `number`  | `60`    | Base delay between retries in **seconds**      |\n| `retryBackoff`  | `boolean` | `true`  | Use exponential backoff (doubles each attempt) |\n| `retryDelayMax` | `number`  | _none_  | Maximum cap for the delay in **seconds**       |\n\n### Fixed delay\n\nSet `retryBackoff: false` to use a constant delay between retries:\n\n```ts\nawait jobQueue.addJob({\n  jobType: 'email',\n  payload: { to: 'user@example.com' },\n  maxAttempts: 5,\n  retryDelay: 30, // 30 seconds between each retry\n  retryBackoff: false,\n});\n```\n\nEvery retry will wait exactly 30 seconds.\n\n### Exponential backoff (default)\n\nWhen `retryBackoff` is `true` (the default), the delay doubles with each attempt. A small amount of random jitter is added to prevent thundering herd problems:\n\n```ts\nawait jobQueue.addJob({\n  jobType: 'email',\n  payload: { to: 'user@example.com' },\n  maxAttempts: 5,\n  retryDelay: 10, // base: 10 seconds\n  retryBackoff: true, // enabled by default\n});\n```\n\nThis produces approximate delays of 10s, 20s, 40s, 80s, ... (with jitter).\n\n### Capping the delay\n\nUse `retryDelayMax` to prevent the delay from growing unbounded:\n\n```ts\nawait jobQueue.addJob({\n  jobType: 'email',\n  payload: { to: 'user@example.com' },\n  maxAttempts: 10,\n  retryDelay: 5,\n  retryBackoff: true,\n  retryDelayMax: 300, // never wait more than 5 minutes\n});\n```\n\nDelays: ~5s, ~10s, ~20s, ~40s, ~80s, ~160s, ~300s, ~300s, ...\n\n### Default behavior\n\nIf none of the retry options are set, the legacy formula `2^attempts * 1 minute` is used. This means the first retry is after ~2 minutes, then ~4 minutes, then ~8 minutes, and so on.\n\n## Jitter\n\nWhen exponential backoff is enabled, each computed delay is multiplied by a random factor between 0.5 and 1.0. This prevents multiple failed jobs from retrying at exactly the same time, which could overload downstream services.\n\n## Cron schedules\n\nRetry configuration can also be set on cron schedules. Every job enqueued by the schedule inherits the retry settings:\n\n```ts\nawait jobQueue.addCronJob({\n  scheduleName: 'daily-report',\n  cronExpression: '0 9 * * *',\n  jobType: 'report',\n  payload: { type: 'daily' },\n  retryDelay: 60,\n  retryBackoff: true,\n  retryDelayMax: 600,\n});\n```\n\n## Editing retry config\n\nYou can update the retry configuration of a pending job:\n\n```ts\nawait jobQueue.editJob(jobId, {\n  retryDelay: 15,\n  retryBackoff: false,\n});\n```"
   },
   {
     "slug": "usage/force-kill-timeout",
@@ -249,7 +249,7 @@
     "slug": "usage/process-jobs",
     "title": "Process Jobs",
     "description": "",
-    "content": "So far, we haven't actually performed any jobs—we've only added them to the queue. Now, let's process those jobs.\n\n> **Note:** This page covers processing jobs in a **serverless environment** using\n  cron-triggered API routes. If you're running a long-lived server (Express,\n  Fastify, etc.), see [Long-Running Server](/usage/long-running-server).\n\nIn a serverless environment, we can't have a long-running process that constantly monitors and processes the queue.\n\nInstead, we create an API endpoint that checks the queue and processes jobs in batches. This endpoint is then triggered by a cron job. For example, you can create an API endpoint at `app/api/cron/process` to process jobs in batches:\n\n```typescript title=\"@/app/api/cron/process.ts\"\nimport { jobHandlers } from '@/lib/job-handler';\nimport { getJobQueue } from '@/lib/queue';\nimport { NextResponse } from 'next/server';\n\nexport async function GET(request: Request) {\n  // Secure the cron route: https://vercel.com/docs/cron-jobs/manage-cron-jobs#securing-cron-jobs\n  const authHeader = request.headers.get('authorization');\n  if (authHeader !== `Bearer ${process.env.CRON_SECRET}`) {\n    return NextResponse.json({ message: 'Unauthorized' }, { status: 401 });\n  }\n\n  try {const jobQueue = getJobQueue();\n\n    // Control how many jobs are processed in parallel per batch using the `concurrency` option.\n    // For example, to process up to 3 jobs in parallel per batch:\n    const processor = jobQueue.createProcessor(jobHandlers, {\n      workerId: `cron-${Date.now()}`,\n      batchSize: 10, // up to 10 jobs per batch\n      concurrency: 3, // up to 3 jobs processed in parallel\n      verbose: true,\n    });\n\n    const processed = await processor.start();\n\n    return NextResponse.json({\n      message: 'Job processing completed',\n      processed,\n    });\n  } catch (error) {\n    console.error('Error processing jobs:', error);\n    return NextResponse.json(\n      { message: 'Failed to process jobs' },\n      { status: 500 },\n    );\n  }\n}\n```\n\nIn the example above, we use the `createProcessor` method to create a processor. When you call the processor's `start` function, it processes jobs in the queue up to the `batchSize` limit.\n\n### Batch Size\n\nServerless platforms like Vercel limit how long a function can run. If you set `batchSize` too high, the function might run too long and get killed. Choose a `batchSize` that fits your use case.\n\nYou can also process only certain job types by setting the `jobType` option. If a job type is more resource-intensive, use a lower `batchSize` for that type.\n\nFor example, you can define two endpoints: one for low-resource jobs and another for high-resource jobs, each with different `batchSize` and `concurrency` values.\n\n### Concurrency\n\nSome jobs are resource-intensive, like image processing, LLM calls, or calling a rate-limited external service. In these cases, set the `concurrency` option to control how many jobs run in parallel per batch.\n\nThe default is `3`. Set it to `1` to process jobs one at a time. Use a lower value to avoid exhausting resources in constrained environments.\n\n### Triggering the Processor via Cron\n\nDefining an endpoint isn't enough—you need to trigger it regularly. For example, use Vercel cron to trigger the endpoint every minute by adding this to your `vercel.json`:\n\n```json title=\"vercel.json\"\n{\n  \"$schema\": \"https://openapi.vercel.sh/vercel.json\",\n  \"crons\": [\n    {\n      \"path\": \"/api/cron/process\",\n      \"schedule\": \"* * * * *\"\n    }\n  ]\n}\n```\n\nFor Vercel cron, set the `CRON_SECRET` environment variable, as it's sent in the `authorization` header. If you use a different cron service, set the `authorization` header to the value of `CRON_SECRET`:\n\n```\nAuthorization: Bearer <VALUE_OF_CRON_SECRET>\n```\n\nDuring development, you can create a small script to run the cron job continuously in the background. For example, you can create a `cron.sh` file like [this one](https://github.com/nicnocquee/dataqueue/blob/main/apps/demo/cron.sh), then add it to your `package.json` scripts:\n\n```json title=\"package.json\"\n{\n  \"scripts\": {\n    \"cron\": \"bash cron.sh\"\n  }\n}\n```\n\nThen, you can run the cron job by running `pnpm cron` from the apps/demo directory."
+    "content": "So far, we haven't actually performed any jobs—we've only added them to the queue. Now, let's process those jobs.\n\n> **Note:** This page covers processing jobs in a **serverless environment** using\n  cron-triggered API routes. If you're running a long-lived server (Express,\n  Fastify, etc.), see [Long-Running Server](/usage/long-running-server).\n\nIn a serverless environment, we can't have a long-running process that constantly monitors and processes the queue.\n\nInstead, we create an API endpoint that checks the queue and processes jobs in batches. This endpoint is then triggered by a cron job. For example, you can create an API endpoint at `app/api/cron/process` to process jobs in batches:\n\n```typescript title=\"@/app/api/cron/process.ts\"\nimport { jobHandlers } from '@/lib/job-handler';\nimport { getJobQueue } from '@/lib/queue';\nimport { NextResponse } from 'next/server';\n\nexport async function GET(request: Request) {\n  // Secure the cron route: https://vercel.com/docs/cron-jobs/manage-cron-jobs#securing-cron-jobs\n  const authHeader = request.headers.get('authorization');\n  if (authHeader !== `Bearer ${process.env.CRON_SECRET}`) {\n    return NextResponse.json({ message: 'Unauthorized' }, { status: 401 });\n  }\n\n  try {const jobQueue = getJobQueue();\n\n    // Control how many jobs are processed in parallel per batch using the `concurrency` option.\n    // For example, to process up to 3 jobs in parallel per batch:\n    const processor = jobQueue.createProcessor(jobHandlers, {\n      workerId: `cron-${Date.now()}`,\n      batchSize: 10, // up to 10 jobs per batch\n      concurrency: 3, // up to 3 jobs processed in parallel\n      groupConcurrency: 2, // optional: max 2 concurrent jobs per group.id globally\n      verbose: true,\n    });\n\n    const processed = await processor.start();\n\n    return NextResponse.json({\n      message: 'Job processing completed',\n      processed,\n    });\n  } catch (error) {\n    console.error('Error processing jobs:', error);\n    return NextResponse.json(\n      { message: 'Failed to process jobs' },\n      { status: 500 },\n    );\n  }\n}\n```\n\nIn the example above, we use the `createProcessor` method to create a processor. When you call the processor's `start` function, it processes jobs in the queue up to the `batchSize` limit.\n\n### Batch Size\n\nServerless platforms like Vercel limit how long a function can run. If you set `batchSize` too high, the function might run too long and get killed. Choose a `batchSize` that fits your use case.\n\nYou can also process only certain job types by setting the `jobType` option. If a job type is more resource-intensive, use a lower `batchSize` for that type.\n\nFor example, you can define two endpoints: one for low-resource jobs and another for high-resource jobs, each with different `batchSize` and `concurrency` values.\n\n### Concurrency\n\nSome jobs are resource-intensive, like image processing, LLM calls, or calling a rate-limited external service. In these cases, set the `concurrency` option to control how many jobs run in parallel per batch.\n\nThe default is `3`. Set it to `1` to process jobs one at a time. Use a lower value to avoid exhausting resources in constrained environments.\n\n### Group-Based Concurrency\n\nWhen jobs should be limited per tenant/customer/account across all workers, set `groupConcurrency` on the processor and provide `group.id` when enqueuing jobs:\n\n```typescript\nawait jobQueue.addJob({\n  jobType: 'send_email',\n  payload: { to: 'user@example.com' },\n  group: { id: 'tenant_123' },\n});\n\nconst processor = jobQueue.createProcessor(jobHandlers, {\n  batchSize: 20,\n  concurrency: 10,\n  groupConcurrency: 2,\n});\n```\n\nWith this configuration, at most 2 jobs from the same `group.id` run at once globally, while ungrouped jobs continue to flow normally.\n\n### Triggering the Processor via Cron\n\nDefining an endpoint isn't enough—you need to trigger it regularly. For example, use Vercel cron to trigger the endpoint every minute by adding this to your `vercel.json`:\n\n```json title=\"vercel.json\"\n{\n  \"$schema\": \"https://openapi.vercel.sh/vercel.json\",\n  \"crons\": [\n    {\n      \"path\": \"/api/cron/process\",\n      \"schedule\": \"* * * * *\"\n    }\n  ]\n}\n```\n\nFor Vercel cron, set the `CRON_SECRET` environment variable, as it's sent in the `authorization` header. If you use a different cron service, set the `authorization` header to the value of `CRON_SECRET`:\n\n```\nAuthorization: Bearer <VALUE_OF_CRON_SECRET>\n```\n\nDuring development, you can create a small script to run the cron job continuously in the background. For example, you can create a `cron.sh` file like [this one](https://github.com/nicnocquee/dataqueue/blob/main/apps/demo/cron.sh), then add it to your `package.json` scripts:\n\n```json title=\"package.json\"\n{\n  \"scripts\": {\n    \"cron\": \"bash cron.sh\"\n  }\n}\n```\n\nThen, you can run the cron job by running `pnpm cron` from the apps/demo directory."
   },
   {
     "slug": "usage/progress-tracking",
@@ -279,7 +279,7 @@
     "slug": "usage/scaling",
     "title": "Scaling to Thousands of Jobs",
     "description": "",
-    "content": "DataQueue is designed to handle high-volume workloads out of the box. This page covers how to tune your setup for thousands (or more) of concurrent jobs, whether you're using PostgreSQL or Redis.\n\n## How Throughput Works\n\nWhen a processor runs, it follows this cycle:\n\n1. **Claim** a batch of ready jobs from the database (atomically, using `FOR UPDATE SKIP LOCKED` in PostgreSQL or Lua scripts in Redis).\n2. **Process** up to `concurrency` jobs in parallel from that batch.\n3. **Repeat** immediately if the batch was full, or wait `pollInterval` before checking again.\n\nThe theoretical maximum throughput of a single processor instance is:\n\n```\nthroughput = batchSize / (avgJobDuration + pollInterval)\n```\n\nFor example, with `batchSize: 20`, `concurrency: 10`, `pollInterval: 2000ms`, and jobs averaging 500ms each, a single processor can handle roughly **10 jobs/second** (600/minute).\n\n> **Note:** When a full batch is returned, the processor immediately polls again without\n  waiting. This means it can drain backlogs much faster than the formula\n  suggests during peak load.\n\n## Tuning Processor Settings\n\n### Batch Size\n\n`batchSize` controls how many jobs are claimed per polling cycle.\n\n| Environment                 | Recommended | Why                                    |\n| --------------------------- | ----------- | -------------------------------------- |\n| Serverless (Vercel, Lambda) | 1-10        | Functions have execution time limits   |\n| Long-running server         | 10-50       | Larger batches reduce polling overhead |\n| High-throughput worker      | 50-100      | Maximizes throughput per poll cycle    |\n\n### Concurrency\n\n`concurrency` controls how many jobs from the batch run in parallel.\n\n- **CPU-bound jobs** (image processing, compression): keep concurrency low (1-4) to avoid CPU saturation.\n- **IO-bound jobs** (API calls, email sending): higher concurrency (5-20) works well since jobs spend most time waiting.\n- **Rate-limited APIs**: match concurrency to the API's rate limit to avoid throttling.\n\n### Poll Interval\n\n`pollInterval` controls how often the processor checks for new jobs when idle.\n\n- **1000ms**: Low latency, higher database load. Good for real-time workloads.\n- **5000ms** (default): Balanced. Good for most use cases.\n- **10000-30000ms**: Gentle on the database. Use when latency tolerance is high.\n\n```typescript\nconst processor = jobQueue.createProcessor(jobHandlers, {\n  batchSize: 30,\n  concurrency: 10,\n  pollInterval: 2000,\n});\n```\n\n## Horizontal Scaling with Multiple Workers\n\nDataQueue supports running **multiple processor instances** simultaneously -- on the same server, across multiple servers, or in separate containers. No coordination is needed between workers.\n\n### How It Works\n\n- Each processor gets a unique `workerId` (auto-generated, or set manually).\n- PostgreSQL uses `FOR UPDATE SKIP LOCKED` to ensure no two workers claim the same job.\n- Redis uses atomic Lua scripts for the same guarantee.\n- Workers can safely run on different machines pointing at the same database.\n\n### Example: Multiple Workers\n\n```typescript\n// Worker 1 (e.g., on server A)\nconst processor1 = jobQueue.createProcessor(jobHandlers, {\n  workerId: 'worker-a',\n  batchSize: 20,\n  concurrency: 5,\n});\nprocessor1.startInBackground();\n\n// Worker 2 (e.g., on server B)\nconst processor2 = jobQueue.createProcessor(jobHandlers, {\n  workerId: 'worker-b',\n  batchSize: 20,\n  concurrency: 5,\n});\nprocessor2.startInBackground();\n```\n\n### Specialized Workers\n\nUse the `jobType` filter to create workers dedicated to specific job types. This lets you scale different workloads independently:\n\n```typescript\n// Fast worker for lightweight jobs\nconst emailWorker = jobQueue.createProcessor(jobHandlers, {\n  jobType: 'email',\n  batchSize: 50,\n  concurrency: 20,\n  pollInterval: 1000,\n});\n\n// Slow worker for heavy jobs\nconst reportWorker = jobQueue.createProcessor(jobHandlers, {\n  jobType: 'report',\n  batchSize: 5,\n  concurrency: 1,\n  pollInterval: 5000,\n});\n```\n\n## PostgreSQL Scaling\n\n### Connection Pool\n\nEach processor instance uses database connections from the pool. The default `max` is 10, which works for a single processor. If you run multiple processors in the same process, increase the pool size:\n\n```typescript\nconst jobQueue = initJobQueue({\n  databaseConfig: {\n    connectionString: process.env.DATABASE_URL,\n    max: 20, // increase for multiple processors\n  },\n});\n```\n\n> **Note:** If you run processors on separate servers, each has its own pool. The total\n  connections across all servers should stay within your database's\n  `max_connections` setting (typically 100 for managed databases like Neon or\n  Supabase).\n\n### Table Maintenance\n\nAs completed jobs accumulate, the `job_queue` table grows. This doesn't affect processing speed (claim queries use partial indexes that only cover active jobs), but it does increase storage and slow down full-table queries like `getJobs()`.\n\n**Run cleanup regularly:**\n\n```typescript\n// Delete completed jobs older than 30 days\nawait jobQueue.cleanupOldJobs(30);\n\n// Delete old job events\nawait jobQueue.cleanupOldJobEvents(30);\n```\n\nCleanup operations are batched internally (1000 rows at a time) so they won't lock the table or timeout even with hundreds of thousands of old jobs.\n\n**Run reclaim regularly:**\n\n```typescript\n// Reclaim jobs stuck in 'processing' for more than 10 minutes\nawait jobQueue.reclaimStuckJobs(10);\n```\n\nThis recovers jobs from workers that crashed or timed out.\n\n### Monitoring Table Size\n\nYou can monitor the `job_queue` table size with this query:\n\n```sql\nSELECT\n  pg_size_pretty(pg_total_relation_size('job_queue')) AS total_size,\n  (SELECT count(*) FROM job_queue WHERE status = 'pending') AS pending,\n  (SELECT count(*) FROM job_queue WHERE status = 'processing') AS processing,\n  (SELECT count(*) FROM job_queue WHERE status = 'completed') AS completed,\n  (SELECT count(*) FROM job_queue WHERE status = 'failed') AS failed;\n```\n\n### Performance Indexes\n\nDataQueue includes optimized partial indexes out of the box:\n\n- `idx_job_queue_claimable` -- speeds up job claiming (pending jobs by priority).\n- `idx_job_queue_failed_retry` -- speeds up retry scheduling.\n- `idx_job_queue_stuck` -- speeds up stuck job reclamation.\n- `idx_job_queue_cleanup` -- speeds up cleanup of old completed jobs.\n\nThese are created automatically when you run migrations.\n\n## Redis Scaling\n\n### Memory Usage\n\nRedis stores everything in memory. Estimate memory usage as:\n\n| Jobs    | Approximate Memory |\n| ------- | ------------------ |\n| 1,000   | 1-2 MB             |\n| 10,000  | 10-20 MB           |\n| 100,000 | 100-200 MB         |\n\nThis assumes payloads under 1 KB each. Larger payloads increase memory proportionally.\n\n### Key Best Practices\n\n- **Enable persistence**: Use AOF (Append Only File) for durability. Without it, a Redis restart loses all jobs.\n- **Use `keyPrefix`** to isolate multiple queues in the same Redis instance:\n\n```typescript\nconst jobQueue = initJobQueue({\n  backend: 'redis',\n  redisConfig: {\n    url: process.env.REDIS_URL,\n    keyPrefix: 'myapp:jobs:',\n  },\n});\n```\n\n- **Run cleanup regularly** to free memory from completed jobs. Like PostgreSQL, Redis cleanup is batched internally using cursor-based scanning, so it's safe to run even with a large number of completed jobs.\n\n## Payload Best Practices\n\nKeep payloads small. Store references (IDs, URLs) rather than full data:\n\n```typescript\n// Good: small payload with reference\nawait jobQueue.addJob({\n  jobType: 'processImage',\n  payload: { imageId: 'img_abc123', bucket: 's3://uploads' },\n});\n\n// Avoid: large payload with embedded data\nawait jobQueue.addJob({\n  jobType: 'processImage',\n  payload: { imageData: '<base64 string of 5MB image>' }, // too large\n});\n```\n\nA good target is **under 10 KB per payload**. This keeps database queries fast and Redis memory predictable.\n\n## Example: Processing 10,000 Jobs per Hour\n\nHere's a configuration that can comfortably process ~10,000 jobs per hour, assuming each job takes about 1 second:\n\n```typescript title=\"worker.ts\"\nimport { initJobQueue } from '@nicnocquee/dataqueue';\nimport { jobHandlers } from './job-handlers';\n\nconst jobQueue = initJobQueue({\n  databaseConfig: {\n    connectionString: process.env.DATABASE_URL,\n    max: 15, // room for processor + maintenance queries\n  },\n});\n\nconst processor = jobQueue.createProcessor(jobHandlers, {\n  batchSize: 30,\n  concurrency: 10, // 10 jobs in parallel\n  pollInterval: 2000,\n  onError: (error) => {\n    console.error('Processor error:', error);\n  },\n});\n\nprocessor.startInBackground();\n\n// Maintenance\nsetInterval(\n  async () => {\n    try {\n      await jobQueue.reclaimStuckJobs(10);\n    } catch (e) {\n      console.error(e);\n    }\n  },\n  10 * 60 * 1000,\n);\n\nsetInterval(\n  async () => {\n    try {\n      await jobQueue.cleanupOldJobs(30);\n      await jobQueue.cleanupOldJobEvents(30);\n    } catch (e) {\n      console.error(e);\n    }\n  },\n  24 * 60 * 60 * 1000,\n);\n\n// Graceful shutdown\nprocess.on('SIGTERM', async () => {\n  await processor.stopAndDrain(30000);\n  jobQueue.getPool().end();\n  process.exit(0);\n});\n```\n\nWith 10 concurrent jobs and ~1s each, a single worker handles roughly **10 jobs/second** = **36,000 jobs/hour**. For higher throughput, add more worker instances.\n\n## Quick Reference\n\n| Scale               | Workers | batchSize | concurrency | pollInterval | Notes                            |\n| ------------------- | ------- | --------- | ----------- | ------------ | -------------------------------- |\n| < 100 jobs/hour     | 1       | 10        | 3           | 5000ms       | Default settings work fine       |\n| 100-1,000/hour      | 1       | 20        | 5           | 3000ms       | Single worker is sufficient      |\n| 1,000-10,000/hour   | 1-2     | 30        | 10          | 2000ms       | Add a second worker if needed    |\n| 10,000-100,000/hour | 2-5     | 50        | 15          | 1000ms       | Multiple workers recommended     |\n| 100,000+/hour       | 5+      | 50-100    | 20          | 1000ms       | Specialized workers per job type |"
+    "content": "DataQueue is designed to handle high-volume workloads out of the box. This page covers how to tune your setup for thousands (or more) of concurrent jobs, whether you're using PostgreSQL or Redis.\n\n## How Throughput Works\n\nWhen a processor runs, it follows this cycle:\n\n1. **Claim** a batch of ready jobs from the database (atomically, using `FOR UPDATE SKIP LOCKED` in PostgreSQL or Lua scripts in Redis).\n2. **Process** up to `concurrency` jobs in parallel from that batch.\n3. **Repeat** immediately if the batch was full, or wait `pollInterval` before checking again.\n\nThe theoretical maximum throughput of a single processor instance is:\n\n```\nthroughput = batchSize / (avgJobDuration + pollInterval)\n```\n\nFor example, with `batchSize: 20`, `concurrency: 10`, `pollInterval: 2000ms`, and jobs averaging 500ms each, a single processor can handle roughly **10 jobs/second** (600/minute).\n\n> **Note:** When a full batch is returned, the processor immediately polls again without\n  waiting. This means it can drain backlogs much faster than the formula\n  suggests during peak load.\n\n## Tuning Processor Settings\n\n### Batch Size\n\n`batchSize` controls how many jobs are claimed per polling cycle.\n\n| Environment                 | Recommended | Why                                    |\n| --------------------------- | ----------- | -------------------------------------- |\n| Serverless (Vercel, Lambda) | 1-10        | Functions have execution time limits   |\n| Long-running server         | 10-50       | Larger batches reduce polling overhead |\n| High-throughput worker      | 50-100      | Maximizes throughput per poll cycle    |\n\n### Concurrency\n\n`concurrency` controls how many jobs from the batch run in parallel.\n\n- **CPU-bound jobs** (image processing, compression): keep concurrency low (1-4) to avoid CPU saturation.\n- **IO-bound jobs** (API calls, email sending): higher concurrency (5-20) works well since jobs spend most time waiting.\n- **Rate-limited APIs**: match concurrency to the API's rate limit to avoid throttling.\n\n### Group Concurrency\n\n`groupConcurrency` caps how many jobs with the same `group.id` can be in `processing` at the same time across all worker instances.\n\n- Use this for multi-tenant fairness and per-account rate protection.\n- It is global coordination (PostgreSQL + Redis), not per-process.\n- Ungrouped jobs are unaffected.\n\n```typescript\nawait jobQueue.addJob({\n  jobType: 'send_email',\n  payload: { to: 'user@example.com' },\n  group: { id: 'tenant_abc' },\n});\n\nconst processor = jobQueue.createProcessor(jobHandlers, {\n  batchSize: 30,\n  concurrency: 10,\n  groupConcurrency: 2,\n  pollInterval: 2000,\n});\n```\n\n### Poll Interval\n\n`pollInterval` controls how often the processor checks for new jobs when idle.\n\n- **1000ms**: Low latency, higher database load. Good for real-time workloads.\n- **5000ms** (default): Balanced. Good for most use cases.\n- **10000-30000ms**: Gentle on the database. Use when latency tolerance is high.\n\n```typescript\nconst processor = jobQueue.createProcessor(jobHandlers, {\n  batchSize: 30,\n  concurrency: 10,\n  pollInterval: 2000,\n});\n```\n\n## Horizontal Scaling with Multiple Workers\n\nDataQueue supports running **multiple processor instances** simultaneously -- on the same server, across multiple servers, or in separate containers. No coordination is needed between workers.\n\n### How It Works\n\n- Each processor gets a unique `workerId` (auto-generated, or set manually).\n- PostgreSQL uses `FOR UPDATE SKIP LOCKED` to ensure no two workers claim the same job.\n- Redis uses atomic Lua scripts for the same guarantee.\n- Workers can safely run on different machines pointing at the same database.\n\n### Example: Multiple Workers\n\n```typescript\n// Worker 1 (e.g., on server A)\nconst processor1 = jobQueue.createProcessor(jobHandlers, {\n  workerId: 'worker-a',\n  batchSize: 20,\n  concurrency: 5,\n});\nprocessor1.startInBackground();\n\n// Worker 2 (e.g., on server B)\nconst processor2 = jobQueue.createProcessor(jobHandlers, {\n  workerId: 'worker-b',\n  batchSize: 20,\n  concurrency: 5,\n});\nprocessor2.startInBackground();\n```\n\n### Specialized Workers\n\nUse the `jobType` filter to create workers dedicated to specific job types. This lets you scale different workloads independently:\n\n```typescript\n// Fast worker for lightweight jobs\nconst emailWorker = jobQueue.createProcessor(jobHandlers, {\n  jobType: 'email',\n  batchSize: 50,\n  concurrency: 20,\n  pollInterval: 1000,\n});\n\n// Slow worker for heavy jobs\nconst reportWorker = jobQueue.createProcessor(jobHandlers, {\n  jobType: 'report',\n  batchSize: 5,\n  concurrency: 1,\n  pollInterval: 5000,\n});\n```\n\n## PostgreSQL Scaling\n\n### Connection Pool\n\nEach processor instance uses database connections from the pool. The default `max` is 10, which works for a single processor. If you run multiple processors in the same process, increase the pool size:\n\n```typescript\nconst jobQueue = initJobQueue({\n  databaseConfig: {\n    connectionString: process.env.DATABASE_URL,\n    max: 20, // increase for multiple processors\n  },\n});\n```\n\n> **Note:** If you run processors on separate servers, each has its own pool. The total\n  connections across all servers should stay within your database's\n  `max_connections` setting (typically 100 for managed databases like Neon or\n  Supabase).\n\n### Table Maintenance\n\nAs completed jobs accumulate, the `job_queue` table grows. This doesn't affect processing speed (claim queries use partial indexes that only cover active jobs), but it does increase storage and slow down full-table queries like `getJobs()`.\n\n**Run cleanup regularly:**\n\n```typescript\n// Delete completed jobs older than 30 days\nawait jobQueue.cleanupOldJobs(30);\n\n// Delete old job events\nawait jobQueue.cleanupOldJobEvents(30);\n```\n\nCleanup operations are batched internally (1000 rows at a time) so they won't lock the table or timeout even with hundreds of thousands of old jobs.\n\n**Run reclaim regularly:**\n\n```typescript\n// Reclaim jobs stuck in 'processing' for more than 10 minutes\nawait jobQueue.reclaimStuckJobs(10);\n```\n\nThis recovers jobs from workers that crashed or timed out.\n\n### Monitoring Table Size\n\nYou can monitor the `job_queue` table size with this query:\n\n```sql\nSELECT\n  pg_size_pretty(pg_total_relation_size('job_queue')) AS total_size,\n  (SELECT count(*) FROM job_queue WHERE status = 'pending') AS pending,\n  (SELECT count(*) FROM job_queue WHERE status = 'processing') AS processing,\n  (SELECT count(*) FROM job_queue WHERE status = 'completed') AS completed,\n  (SELECT count(*) FROM job_queue WHERE status = 'failed') AS failed;\n```\n\n### Performance Indexes\n\nDataQueue includes optimized partial indexes out of the box:\n\n- `idx_job_queue_claimable` -- speeds up job claiming (pending jobs by priority).\n- `idx_job_queue_failed_retry` -- speeds up retry scheduling.\n- `idx_job_queue_stuck` -- speeds up stuck job reclamation.\n- `idx_job_queue_cleanup` -- speeds up cleanup of old completed jobs.\n\nThese are created automatically when you run migrations.\n\n## Redis Scaling\n\n### Memory Usage\n\nRedis stores everything in memory. Estimate memory usage as:\n\n| Jobs    | Approximate Memory |\n| ------- | ------------------ |\n| 1,000   | 1-2 MB             |\n| 10,000  | 10-20 MB           |\n| 100,000 | 100-200 MB         |\n\nThis assumes payloads under 1 KB each. Larger payloads increase memory proportionally.\n\n### Key Best Practices\n\n- **Enable persistence**: Use AOF (Append Only File) for durability. Without it, a Redis restart loses all jobs.\n- **Use `keyPrefix`** to isolate multiple queues in the same Redis instance:\n\n```typescript\nconst jobQueue = initJobQueue({\n  backend: 'redis',\n  redisConfig: {\n    url: process.env.REDIS_URL,\n    keyPrefix: 'myapp:jobs:',\n  },\n});\n```\n\n- **Run cleanup regularly** to free memory from completed jobs. Like PostgreSQL, Redis cleanup is batched internally using cursor-based scanning, so it's safe to run even with a large number of completed jobs.\n\n## Payload Best Practices\n\nKeep payloads small. Store references (IDs, URLs) rather than full data:\n\n```typescript\n// Good: small payload with reference\nawait jobQueue.addJob({\n  jobType: 'processImage',\n  payload: { imageId: 'img_abc123', bucket: 's3://uploads' },\n});\n\n// Avoid: large payload with embedded data\nawait jobQueue.addJob({\n  jobType: 'processImage',\n  payload: { imageData: '<base64 string of 5MB image>' }, // too large\n});\n```\n\nA good target is **under 10 KB per payload**. This keeps database queries fast and Redis memory predictable.\n\n## Example: Processing 10,000 Jobs per Hour\n\nHere's a configuration that can comfortably process ~10,000 jobs per hour, assuming each job takes about 1 second:\n\n```typescript title=\"worker.ts\"\nimport { initJobQueue } from '@nicnocquee/dataqueue';\nimport { jobHandlers } from './job-handlers';\n\nconst jobQueue = initJobQueue({\n  databaseConfig: {\n    connectionString: process.env.DATABASE_URL,\n    max: 15, // room for processor + maintenance queries\n  },\n});\n\nconst processor = jobQueue.createProcessor(jobHandlers, {\n  batchSize: 30,\n  concurrency: 10, // 10 jobs in parallel\n  pollInterval: 2000,\n  onError: (error) => {\n    console.error('Processor error:', error);\n  },\n});\n\nprocessor.startInBackground();\n\n// Maintenance\nsetInterval(\n  async () => {\n    try {\n      await jobQueue.reclaimStuckJobs(10);\n    } catch (e) {\n      console.error(e);\n    }\n  },\n  10 * 60 * 1000,\n);\n\nsetInterval(\n  async () => {\n    try {\n      await jobQueue.cleanupOldJobs(30);\n      await jobQueue.cleanupOldJobEvents(30);\n    } catch (e) {\n      console.error(e);\n    }\n  },\n  24 * 60 * 60 * 1000,\n);\n\n// Graceful shutdown\nprocess.on('SIGTERM', async () => {\n  await processor.stopAndDrain(30000);\n  jobQueue.getPool().end();\n  process.exit(0);\n});\n```\n\nWith 10 concurrent jobs and ~1s each, a single worker handles roughly **10 jobs/second** = **36,000 jobs/hour**. For higher throughput, add more worker instances.\n\n## Quick Reference\n\n| Scale               | Workers | batchSize | concurrency | pollInterval | Notes                            |\n| ------------------- | ------- | --------- | ----------- | ------------ | -------------------------------- |\n| < 100 jobs/hour     | 1       | 10        | 3           | 5000ms       | Default settings work fine       |\n| 100-1,000/hour      | 1       | 20        | 5           | 3000ms       | Single worker is sufficient      |\n| 1,000-10,000/hour   | 1-2     | 30        | 10          | 2000ms       | Add a second worker if needed    |\n| 10,000-100,000/hour | 2-5     | 50        | 15          | 1000ms       | Multiple workers recommended     |\n| 100,000+/hour       | 5+      | 50-100    | 20          | 1000ms       | Specialized workers per job type |"
   },
   {
     "slug": "usage/wait",

package/ai/rules/advanced.md

@@ -116,6 +116,21 @@ await queue.addJob({
 - No config — legacy `2^attempts * 60s` formula (backward compatible).
 - Cron schedules propagate retry config to enqueued jobs.
 
+## Dead-Letter Routing
+
+Configure dead-letter capture with `deadLetterJobType`:
+
+```typescript
+await queue.addJob({
+  jobType: 'email',
+  payload,
+  maxAttempts: 3,
+  deadLetterJobType: 'email_dead_letter',
+});
+```
+
+When retries are exhausted, DataQueue creates a pending dead-letter job with envelope payload containing `originalJob`, `originalPayload`, and `failure`. Source jobs remain `failed` and store linkage metadata (`deadLetteredAt`, `deadLetterJobId`).
+
 ## Event Hooks
 
 Subscribe to real-time lifecycle events via `on`, `once`, `off`, `removeAllListeners`. Works with both Postgres and Redis.
@@ -137,6 +152,7 @@ Events: `job:added`, `job:processing`, `job:completed`, `job:failed` (with `will
 ## Scaling
 
 - Increase `batchSize` and `concurrency` for higher throughput.
+- Use `group: { id }` on jobs with `groupConcurrency` on processors when you need global per-tenant/per-account fairness.
 - Run multiple processor instances with unique `workerId` values — `FOR UPDATE SKIP LOCKED` (PostgreSQL) or Lua scripts (Redis) prevent double-claiming.
 - Use `jobType` filter for specialized workers.
 - Use `createSupervisor()` to automate maintenance (reclaim stuck jobs, cleanup, token expiry). Safe to run across multiple instances.

package/ai/rules/basic.md

@@ -86,7 +86,7 @@ const ids = await queue.addJobs([
 // ids[i] corresponds to the i-th input job
 ```
 
-Both support `idempotencyKey`, `priority`, `runAt`, `tags`, and `{ db }` for transactional inserts (PostgreSQL only).
+Both support `idempotencyKey`, `priority`, `runAt`, `tags`, optional `group: { id, tier? }`, and `{ db }` for transactional inserts (PostgreSQL only).
 
 ## Handlers
 
@@ -113,6 +113,7 @@ Handler signature: `(payload: T, signal: AbortSignal, ctx: JobContext) => Promis
 const processor = queue.createProcessor(handlers, {
   batchSize: 10,
   concurrency: 3,
+  groupConcurrency: 2, // optional global cap per group.id
 });
 await processor.start();
 ```
@@ -149,6 +150,21 @@ Control retry behavior per-job with optional fields on `addJob`:
 
 When none are set, the legacy `2^attempts * 60s` formula is used.
 
+## Dead-Letter Queue
+
+Use `deadLetterJobType` for jobs that must be captured after exhausting retries:
+
+```typescript
+await queue.addJob({
+  jobType: 'email',
+  payload: { to: 'user@example.com', subject: 'Hi', body: '...' },
+  maxAttempts: 3,
+  deadLetterJobType: 'email_dead_letter',
+});
+```
+
+On exhaustion, the source job stays `failed` and a new pending dead-letter job is created with envelope payload: `{ originalJob, originalPayload, failure }`.
+
 ## Common Mistakes
 
 1. Creating `initJobQueue` per request — use a singleton.
@@ -157,3 +173,4 @@ When none are set, the legacy `2^attempts * 60s` formula is used.
 4. Skipping maintenance — use `createSupervisor()` to automate reclaim, cleanup, and token expiry. Without it, stuck jobs and old data accumulate.
 5. Skipping migrations (PostgreSQL) — run `dataqueue-cli migrate` first. Redis needs none.
 6. Using `stop()` instead of `stopAndDrain()` — leaves in-flight jobs stuck.
+7. Expecting dead-letter routing without setting `deadLetterJobType` — DLQ is opt-in.

package/ai/skills/dataqueue-advanced/SKILL.md

@@ -239,6 +239,28 @@ await queue.cancelAllUpcomingJobs({
 
 Tag query modes: `'exact'`, `'all'`, `'any'`, `'none'`.
 
+## Group-Based Concurrency
+
+Use job `group.id` plus processor `groupConcurrency` to enforce a global cap per group across all workers/instances (PostgreSQL and Redis).
+
+```typescript
+await queue.addJob({
+  jobType: 'email',
+  payload: {
+    /* ... */
+  },
+  group: { id: 'tenant_abc', tier: 'gold' }, // tier is optional/reserved
+});
+
+const processor = queue.createProcessor(handlers, {
+  batchSize: 20,
+  concurrency: 10,
+  groupConcurrency: 2,
+});
+```
+
+Ungrouped jobs are unaffected by `groupConcurrency`.
+
 ## Idempotency
 
 ```typescript
@@ -342,6 +364,29 @@ await queue.addCronJob({
 
 Every job enqueued by the schedule inherits the retry settings.
 
+### Dead-letter routing
+
+Set `deadLetterJobType` on jobs (or cron schedules) to route exhausted failures:
+
+```typescript
+await queue.addJob({
+  jobType: 'email',
+  payload: { to: 'user@example.com' },
+  maxAttempts: 3,
+  deadLetterJobType: 'email_dead_letter',
+});
+```
+
+Dead-letter jobs receive envelope payload:
+
+```typescript
+{
+  originalJob: { id, jobType, attempts, maxAttempts },
+  originalPayload: {...},
+  failure: { message, reason, failedAt }
+}
+```
+
 ### Default behavior
 
 When no retry options are set, the legacy formula `2^attempts * 60 seconds` is used. This is fully backward compatible.

package/ai/skills/dataqueue-core/SKILL.md

@@ -114,6 +114,7 @@ const jobId = await queue.addJob({
   runAt: new Date(Date.now() + 5000),
   tags: ['welcome'],
   idempotencyKey: 'welcome-user-123',
+  group: { id: 'tenant_123' }, // optional: for global per-group concurrency limits
 });
 ```
 
@@ -182,6 +183,21 @@ await queue.addJob({
 - **Exponential backoff** (default): delay doubles each attempt with jitter.
 - **Default**: when no retry options are set, legacy `2^attempts * 60s` is used.
 
+### Dead-letter queues
+
+Route exhausted failures into a dedicated job type with `deadLetterJobType`:
+
+```typescript
+await queue.addJob({
+  jobType: 'send_email',
+  payload: { to: 'user@example.com', subject: 'Hi', body: 'Hello' },
+  maxAttempts: 3,
+  deadLetterJobType: 'email_dead_letter',
+});
+```
+
+When retries are exhausted, DataQueue keeps the source job as `failed` and creates a new pending dead-letter job with envelope payload: `{ originalJob, originalPayload, failure }`.
+
 ## Step 5: Process Jobs
 
 ### Serverless (one-shot)
@@ -190,6 +206,7 @@ await queue.addJob({
 const processor = queue.createProcessor(handlers, {
   batchSize: 10,
   concurrency: 3,
+  groupConcurrency: 2, // optional global cap per group.id across all workers
 });
 const processed = await processor.start();
 ```
@@ -233,3 +250,4 @@ process.on('SIGTERM', async () => {
 6. **Not calling `stopAndDrain` on shutdown** — use `stopAndDrain()` (not `stop()`) for graceful shutdown to avoid stuck jobs.
 7. **Forgetting to commit/rollback when using `db` option** — the `addJob` INSERT sits in an open transaction. If you never `COMMIT` or `ROLLBACK`, the connection leaks and the job is invisible to other sessions.
 8. **Using `db` option with Redis** — transactional job creation is PostgreSQL only. The Redis backend throws if `db` is provided.
+9. **Expecting dead-letter routing without configuration** — DLQ is opt-in. Set `deadLetterJobType` on jobs (or cron schedules) that require dead-letter capture.