@chidchanun/bcp 0.2.9 → 0.2.10

package/docs/api-reference.md

@@ -66,30 +66,38 @@ Related guides: [Authentication](authentication.md), [Auth Session Stores](auth-
 
 ## `bcp/jobs`
 
-Server-only Background Jobs and Job Scheduling APIs.
+Server-only Background Jobs, Job Scheduling and Durable Jobs APIs.
 
-Queue APIs:
+Core queue APIs:
 
 ```ts
 import {
     createJobQueue,
     createMemoryJobQueueAdapter,
     type BackgroundJobQueue,
+    type CleanupJobsOptions,
+    type DeadLetterJobRecord,
     type EnqueueJobOptions,
+    type HeartbeatJobOptions,
     type JobHandler,
     type JobHandlerContext,
     type JobQueueAdapter,
     type JobQueueOptions,
+    type JobQueueStats,
     type JobRecord,
     type JobRetryDelay,
     type JobState,
     type JobWorker,
     type MemoryJobQueueAdapter,
+    type ProcessNextJobOptions,
+    type RecoverStaleJobsOptions,
+    type RequeueDeadLetterOptions,
+    type ReserveJobOptions,
     type StartJobWorkerOptions,
 } from "bcp/jobs";
 ```
 
-Scheduling APIs added in `0.2.9`:
+Scheduling APIs:
 
 ```ts
 import {
@@ -110,15 +118,68 @@ import {
 } from "bcp/jobs";
 ```
 
-`createJobQueue()` provides delayed enqueueing, retry/backoff, cancellation, manual `processNext()` execution and concurrent workers.
+Durable Redis-compatible APIs added in `0.2.10`:
 
-`createJobScheduler()` adds recurring interval and UTC five-field cron schedules. The scheduler leases due records from `JobScheduleStore`, then enqueues normal queue jobs using deterministic occurrence ids.
+```ts
+import {
+    createRedisJobQueueAdapter,
+    createRedisJobScheduleStore,
+    type RedisCommandClient,
+    type RedisJobQueueAdapter,
+    type RedisJobsAdapterOptions,
+    type RedisJobScheduleStore,
+} from "bcp/jobs";
+```
+
+### Queue lifecycle
+
+`createJobQueue()` supports:
+
+```text
+immediate/delayed enqueue
+retry/backoff
+cancellation
+manual processNext()
+worker concurrency
+visibility timeout
+heartbeat lease renewal
+stale-running recovery
+DLQ inspection/requeue
+terminal retention cleanup
+queue statistics
+```
+
+Durable adapter methods such as `heartbeat()`, `recoverStale()`, `listDeadLetters()`, `requeueDeadLetter()`, `cleanup()` and `stats()` are optional so earlier `JobQueueAdapter` implementations remain compatible.
+
+`reserve()` receives an optional second `ReserveJobOptions` argument containing `ownerId` and `visibilityTimeoutMs`. Production durable adapters should use it to create an atomic visibility lease.
+
+### Scheduling
+
+`createJobScheduler()` supports recurring interval and UTC five-field cron schedules.
+
+The scheduler leases due records from `JobScheduleStore`, then enqueues normal jobs using deterministic occurrence IDs. `nextCronTime()` and `nextScheduleTime()` are available for tooling and tests.
+
+### Redis-compatible durable adapters
+
+`createRedisJobQueueAdapter()` implements the queue contract on a shared Redis-compatible command client.
+
+`createRedisJobScheduleStore()` implements the schedule-store contract using the same minimal client shape:
+
+```ts
+interface RedisCommandClient {
+    sendCommand(
+        command: string[]
+    ): Promise<unknown>;
+}
+```
+
+BCP does not install a Redis library and does not create the connection. Applications own authentication, TLS, Cluster/Sentinel configuration and connection shutdown.
 
-The built-in queue adapter and schedule store are process-local. Durable multi-instance deployments should implement both `JobQueueAdapter` and `JobScheduleStore` against shared infrastructure. `JobQueueAdapter.reserve()` and `JobScheduleStore.acquireDue()` are the respective atomic concurrency boundaries.
+The reference adapters use atomic Lua state transitions. The default namespace `bcp:{jobs}` uses a Redis Cluster hash tag so all job/schedule keys share one slot.
 
-Cron evaluation in `0.2.9` uses UTC. `nextCronTime()` and `nextScheduleTime()` are available for tooling and tests.
+The processing model is at-least-once. Handlers that perform non-idempotent external side effects should use application-level idempotency protection.
 
-Related guides: [Background Jobs Platform](background-jobs.md), [Job Scheduling Platform](job-scheduling.md), [Observability Platform v2](observability.md).
+Related guides: [Background Jobs Platform](background-jobs.md), [Job Scheduling Platform](job-scheduling.md), [Durable Jobs Platform](durable-jobs.md), [Observability Platform v2](observability.md).
 
 ## `bcp/observability`
 

package/docs/docs-web-manifest.json

@@ -1,7 +1,7 @@
 {
   "schemaVersion": 1,
   "framework": "bcp",
-  "versionTarget": "0.2.9",
+  "versionTarget": "0.2.10",
   "releaseState": "unreleased",
   "sections": [
     {
@@ -57,7 +57,7 @@
     {
       "id": "runtime",
       "title": "Runtime & Infrastructure",
-      "description": "Middleware, background jobs, scheduling, observability, logging, caching, security and production hardening.",
+      "description": "Middleware, durable background jobs, scheduling, observability, logging, caching, security and production hardening.",
       "pages": [
         { "route": "/docs/middleware", "source": "middleware.md", "title": "Middleware" },
         { "route": "/docs/hydration", "source": "hydration.md", "title": "Hydration" },
@@ -65,6 +65,7 @@
         { "route": "/docs/observability", "source": "observability.md", "title": "Observability Platform v2" },
         { "route": "/docs/background-jobs", "source": "background-jobs.md", "title": "Background Jobs Platform" },
         { "route": "/docs/job-scheduling", "source": "job-scheduling.md", "title": "Job Scheduling Platform" },
+        { "route": "/docs/durable-jobs", "source": "durable-jobs.md", "title": "Durable Jobs Platform" },
         { "route": "/docs/caching", "source": "caching.md", "title": "Caching" },
         { "route": "/docs/security", "source": "security.md", "title": "Security" },
         { "route": "/docs/production-hardening", "source": "production-hardening.md", "title": "Production Hardening" }
@@ -110,7 +111,8 @@
     }
   ],
   "releases": [
-    { "route": "/releases/0.2.9", "source": "releases/0.2.9.md", "version": "0.2.9", "state": "unreleased" },
+    { "route": "/releases/0.2.10", "source": "releases/0.2.10.md", "version": "0.2.10", "state": "unreleased" },
+    { "route": "/releases/0.2.9", "source": "releases/0.2.9.md", "version": "0.2.9" },
     { "route": "/releases/0.2.8", "source": "releases/0.2.8.md", "version": "0.2.8" },
     { "route": "/releases/0.2.7", "source": "releases/0.2.7.md", "version": "0.2.7" },
     { "route": "/releases/0.2.6", "source": "releases/0.2.6.md", "version": "0.2.6" },

package/docs/durable-jobs.md

@@ -0,0 +1,359 @@
+# Durable Jobs Platform
+
+BCP Framework `0.2.10` extends `bcp/jobs` with production-oriented worker lifecycle primitives, dead-letter handling, retention utilities and Redis-compatible queue/scheduler adapters.
+
+The public entrypoint remains:
+
+```ts
+import {
+    createJobQueue,
+    createJobScheduler,
+} from "bcp/jobs";
+```
+
+`0.2.10` is additive to the queue contract introduced in `0.2.8` and the scheduling contract introduced in `0.2.9`.
+
+## What 0.2.10 adds
+
+```text
+visibility timeout
+worker lease owner
+heartbeat renewal
+stale-running recovery
+dead-letter queue (DLQ)
+dead-letter requeue
+terminal-job cleanup
+queue statistics
+Redis-compatible JobQueueAdapter
+Redis-compatible JobScheduleStore
+```
+
+The built-in memory adapter implements the same lifecycle for development and deterministic tests. Durable multi-process deployments should use a shared adapter such as the Redis-compatible adapter or another implementation of the public contracts.
+
+## Worker visibility timeout
+
+Workers now reserve jobs with a visibility lease:
+
+```ts
+const worker =
+    jobs.startWorker({
+        workerId: "email-worker",
+        concurrency: 4,
+        visibilityTimeoutMs: 30_000,
+        heartbeatIntervalMs: 10_000,
+        pollIntervalMs: 250,
+    });
+```
+
+A durable adapter can persist:
+
+```text
+leaseOwner
+leaseUntil
+heartbeatAt
+```
+
+While a handler is running, BCP renews the lease when the adapter implements `heartbeat()`.
+
+If the process disappears and the lease expires, `recoverStale()` can move the job back to `queued` so another worker can process it.
+
+This is an **at-least-once** model. Handlers that perform non-idempotent side effects should use application-level idempotency keys or transactions.
+
+## Manual stale recovery
+
+```ts
+const recovered =
+    await jobs.recoverStale({
+        limit: 100,
+    });
+```
+
+Workers also request stale-job recovery before reserving work when the adapter implements the capability.
+
+A recovered job keeps its attempt count. It may therefore reach `maxAttempts` after repeated crashes or visibility-timeout recoveries.
+
+## Dead-letter queue
+
+When a handler exhausts `maxAttempts`, the memory and Redis adapters keep the job in terminal `failed` state and also add it to the DLQ index.
+
+```ts
+const failedJobs =
+    await jobs.deadLetters();
+```
+
+Each result includes:
+
+```ts
+{
+    state: "failed",
+    deadLetteredAt: number,
+    // normal JobRecord fields
+}
+```
+
+Requeue a DLQ job:
+
+```ts
+await jobs.requeueDeadLetter(
+    jobId,
+    {
+        delayMs: 5_000,
+        resetAttempts: true,
+    }
+);
+```
+
+`resetAttempts` defaults to `true`.
+
+## Queue statistics
+
+```ts
+const stats =
+    await jobs.stats();
+```
+
+Shape:
+
+```ts
+{
+    total: number;
+    queued: number;
+    running: number;
+    succeeded: number;
+    failed: number;
+    cancelled: number;
+    deadLetters: number;
+}
+```
+
+Adapters may provide an optimized `stats()` implementation. Otherwise BCP derives statistics from `list()` plus the optional DLQ contract.
+
+## Retention cleanup
+
+Terminal jobs can be removed after an application-defined retention window:
+
+```ts
+const sevenDaysAgo =
+    Date.now() -
+    7 * 24 * 60 * 60 * 1000;
+
+await jobs.cleanup({
+    before: sevenDaysAgo,
+    states: [
+        "succeeded",
+        "cancelled",
+    ],
+});
+```
+
+If `states` is omitted, cleanup considers:
+
+```text
+succeeded
+failed
+cancelled
+```
+
+For failed jobs, cleanup also removes the DLQ index entry.
+
+The Redis reference adapter removes at most 1,000 matching terminal records per cleanup call. Large installations can invoke cleanup repeatedly from a scheduled maintenance job.
+
+## Redis-compatible queue adapter
+
+BCP does not install or own a Redis client library. The adapter accepts a minimal command client:
+
+```ts
+interface RedisCommandClient {
+    sendCommand(
+        command: string[]
+    ): Promise<unknown>;
+}
+```
+
+Create the queue adapter:
+
+```ts
+import {
+    createJobQueue,
+    createRedisJobQueueAdapter,
+} from "bcp/jobs";
+
+const adapter =
+    createRedisJobQueueAdapter({
+        client: redisCommandClient,
+        namespace: "my-app:{jobs}",
+    });
+
+export const jobs =
+    createJobQueue({
+        adapter,
+    });
+```
+
+The default namespace is:
+
+```text
+bcp:{jobs}
+```
+
+The `{jobs}` Redis hash tag keeps all adapter keys in the same Redis Cluster slot so Lua operations can remain atomic.
+
+### Connection lifecycle
+
+Applications own the Redis connection. When desired, provide a close hook:
+
+```ts
+const adapter =
+    createRedisJobQueueAdapter({
+        client: redisCommandClient,
+        close: async () => {
+            await redisClient.quit();
+        },
+    });
+```
+
+Then:
+
+```ts
+await jobs.close();
+```
+
+will stop BCP workers and invoke the adapter close hook.
+
+If queue and scheduler share one Redis connection, normally provide the close hook to only one owner or coordinate connection shutdown at the application layer.
+
+## Redis environment configuration
+
+A typical application can use:
+
+```dotenv
+REDIS_URL=redis://localhost:6379
+```
+
+`bcp/jobs` does **not** read `REDIS_URL` automatically. Connection creation remains application-owned so credentials, TLS, Sentinel/Cluster configuration and client-library choice stay explicit.
+
+## Atomic Redis behavior
+
+The reference Redis adapter uses Lua `EVAL` for state-changing operations that require atomicity, including:
+
+```text
+enqueue duplicate protection
+job reservation
+complete/fail/cancel
+heartbeat lease renewal
+stale-running recovery
+DLQ requeue
+retention cleanup
+schedule upsert/remove
+schedule lease acquisition
+schedule completion/release
+```
+
+Queue reservation atomically removes one eligible job from the available index, marks it running and writes the visibility lease before another worker can reserve it.
+
+## Redis-backed scheduler
+
+The same Redis command client can back the scheduling contract:
+
+```ts
+import {
+    createJobScheduler,
+    createRedisJobScheduleStore,
+} from "bcp/jobs";
+
+const scheduleStore =
+    createRedisJobScheduleStore({
+        client: redisCommandClient,
+        namespace: "my-app:{jobs}",
+    });
+
+export const scheduler =
+    createJobScheduler({
+        queue: jobs,
+        store: scheduleStore,
+        ownerId: "scheduler-a",
+    });
+```
+
+`acquireDue()` is implemented with an atomic Redis lease, allowing multiple scheduler processes to share the same schedule store.
+
+## Recommended production topology
+
+```text
+Web/API instances
+      |
+      | enqueue / schedule
+      v
+Redis-compatible shared job state
+      |
+      +-------------------+
+      |                   |
+      v                   v
+Worker A              Worker B
+visibility lease      visibility lease
+heartbeat             heartbeat
+      |
+      +---- failure ----> retry / DLQ
+
+Scheduler A -----+
+                 +---- shared schedule lease
+Scheduler B -----+
+```
+
+Queue workers and schedulers can run in the same Node process for small deployments, but separate worker processes/containers are recommended when job load can affect web latency.
+
+## Adapter compatibility
+
+Existing `0.2.8` `JobQueueAdapter` implementations remain valid. Durable methods are optional:
+
+```ts
+heartbeat?()
+recoverStale?()
+listDeadLetters?()
+requeueDeadLetter?()
+cleanup?()
+stats?()
+```
+
+The `reserve()` method now receives an optional second argument with the worker owner ID and visibility timeout. Existing JavaScript adapters that ignore the optional argument continue to work.
+
+For production durable adapters, implementations should support the lease contract rather than ignoring it.
+
+## Failure model
+
+BCP jobs intentionally use practical at-least-once delivery semantics.
+
+Applications should assume that a job can execute more than once when:
+
+- a worker completes external side effects but crashes before recording success,
+- a visibility lease expires during a long or blocked handler,
+- a DLQ job is manually requeued,
+- infrastructure retries a command after an ambiguous network result.
+
+Use idempotency keys, unique database constraints or transactional outbox/inbox patterns for side effects where duplicates are unsafe.
+
+## Observability
+
+`jobs.stats()` can feed application metrics from `bcp/observability`:
+
+```ts
+const stats =
+    await jobs.stats();
+
+queueDepth.set(
+    stats.queued
+);
+
+dlqDepth.set(
+    stats.deadLetters
+);
+```
+
+For Redis deployments, infrastructure-level Redis monitoring should complement application-level BCP metrics.
+
+## Related guides
+
+- [Background Jobs Platform](background-jobs.md)
+- [Job Scheduling Platform](job-scheduling.md)
+- [Observability Platform v2](observability.md)
+- [Production Hardening](production-hardening.md)

package/docs/platform-manifest.json

@@ -1,9 +1,9 @@
 {
   "schemaVersion": 1,
   "framework": "bcp",
-  "version": "0.2.9",
+  "version": "0.2.10",
   "releaseState": "unreleased",
-  "baseline": "job-scheduling-platform",
+  "baseline": "durable-jobs-platform",
   "runtime": {
     "node": ">=24.11.0",
     "react": "19",
@@ -80,6 +80,16 @@
     "jobScheduleStoreContract": true,
     "schedulerLeases": true,
     "scheduledRunDeduplication": true,
+    "durableJobsPlatform": true,
+    "redisJobQueueAdapter": true,
+    "redisJobScheduleStore": true,
+    "jobVisibilityTimeout": true,
+    "jobHeartbeats": true,
+    "staleJobRecovery": true,
+    "deadLetterQueue": true,
+    "deadLetterRequeue": true,
+    "jobRetentionCleanup": true,
+    "jobQueueStatistics": true,
     "databaseMigrations": true,
     "databaseAdapterContract": true,
     "databasePostgresql": true,
@@ -119,7 +129,7 @@
     "s3-compatible"
   ],
   "compatibility": {
-    "previousBaseline": "0.2.8",
+    "previousBaseline": "0.2.9",
     "intentionalBreakingChangesFromPreviousBaseline": false,
     "migrationGuide": "migration-0.2.md"
   },
@@ -138,7 +148,8 @@
     "observability": "observability.md",
     "backgroundJobs": "background-jobs.md",
     "jobScheduling": "job-scheduling.md",
+    "durableJobs": "durable-jobs.md",
     "migrationGuide": "migration-0.2.md",
-    "releaseNotes": "releases/0.2.9.md"
+    "releaseNotes": "releases/0.2.10.md"
   }
 }

package/docs/releases/0.2.10.md

@@ -0,0 +1,148 @@
+# 0.2.10 — Durable Jobs Platform
+
+**State:** unreleased development target
+
+BCP Framework `0.2.10` extends the Background Jobs and Job Scheduling platforms with durable worker lifecycle contracts, dead-letter handling, retention/statistics APIs and Redis-compatible reference adapters.
+
+## Highlights
+
+### Durable worker leases
+
+Workers can reserve jobs with a visibility timeout and stable worker identity.
+
+New job record metadata:
+
+```text
+leaseOwner
+leaseUntil
+heartbeatAt
+recoveredAt
+```
+
+When an adapter supports `heartbeat()`, BCP workers renew the lease while a handler is running.
+
+When an adapter supports `recoverStale()`, expired running jobs can be returned to the queue for at-least-once recovery.
+
+### Dead-letter queue
+
+Retry-exhausted jobs remain terminal `failed` jobs and can also be indexed in the adapter DLQ.
+
+New queue APIs:
+
+```ts
+jobs.deadLetters()
+jobs.requeueDeadLetter()
+```
+
+The built-in memory adapter and Redis-compatible adapter implement both APIs.
+
+### Queue maintenance
+
+New operational APIs:
+
+```ts
+jobs.stats()
+jobs.cleanup()
+jobs.recoverStale()
+```
+
+Statistics include queue state totals and DLQ depth.
+
+Cleanup can remove old succeeded, failed and cancelled records after an application-defined retention period.
+
+### Redis-compatible queue adapter
+
+New public API:
+
+```ts
+createRedisJobQueueAdapter()
+```
+
+The adapter consumes a minimal application-owned Redis command client exposing:
+
+```ts
+sendCommand(command: string[]): Promise<unknown>
+```
+
+BCP does not add a Redis client runtime dependency.
+
+The Redis adapter uses atomic Lua operations for:
+
+- duplicate-safe enqueue,
+- job reservation,
+- complete/fail/cancel transitions,
+- heartbeat lease renewal,
+- stale-job recovery,
+- DLQ requeue,
+- retention cleanup.
+
+The default Redis namespace is `bcp:{jobs}` so all adapter keys share one Redis Cluster hash slot.
+
+### Redis-compatible scheduler store
+
+New public API:
+
+```ts
+createRedisJobScheduleStore()
+```
+
+The scheduler store implements the `0.2.9` `JobScheduleStore` contract on shared Redis state and uses atomic schedule leasing for multi-instance schedulers.
+
+### Compiled npm runtime
+
+The published `bcp/jobs` default runtime continues to use generated `jobs.mjs` rather than requiring consumers to execute framework TypeScript directly.
+
+The Durable Jobs package smoke imports that compiled runtime and verifies the new public APIs.
+
+## Public API additions
+
+`bcp/jobs` adds:
+
+```text
+createRedisJobQueueAdapter
+createRedisJobScheduleStore
+RedisCommandClient
+RedisJobQueueAdapter
+RedisJobScheduleStore
+RedisJobsAdapterOptions
+
+CleanupJobsOptions
+DeadLetterJobRecord
+HeartbeatJobOptions
+JobQueueStats
+ProcessNextJobOptions
+RecoverStaleJobsOptions
+RequeueDeadLetterOptions
+ReserveJobOptions
+```
+
+Existing queue and scheduling APIs remain available.
+
+## Compatibility
+
+The release is intended to be backward compatible with `0.2.9`.
+
+Existing `JobQueueAdapter` implementations remain valid because durable capabilities are optional and the new reservation metadata is provided as an optional second parameter to `reserve()`.
+
+Adapters that need production visibility-timeout semantics should implement the new durable methods.
+
+## Delivery semantics
+
+Durable jobs use at-least-once processing semantics.
+
+Applications should make side-effecting handlers idempotent when duplicate execution would be unsafe.
+
+## Validation target
+
+Before tagging/publishing `0.2.10`, run:
+
+```bash
+npm run typecheck
+npm run test:unit
+npm run test:integration
+npm run test:e2e
+npm run test:package
+npm run rc:check
+```
+
+The final release tag must point to the exact commit that passed the complete RC sequence.