@chidchanun/bcp 0.2.9 → 0.2.11

package/docs/releases/0.2.10.md

@@ -0,0 +1,148 @@
+# 0.2.10 — Durable Jobs Platform
+
+**State:** unreleased development target
+
+BCP Framework `0.2.10` extends the Background Jobs and Job Scheduling platforms with durable worker lifecycle contracts, dead-letter handling, retention/statistics APIs and Redis-compatible reference adapters.
+
+## Highlights
+
+### Durable worker leases
+
+Workers can reserve jobs with a visibility timeout and stable worker identity.
+
+New job record metadata:
+
+```text
+leaseOwner
+leaseUntil
+heartbeatAt
+recoveredAt
+```
+
+When an adapter supports `heartbeat()`, BCP workers renew the lease while a handler is running.
+
+When an adapter supports `recoverStale()`, expired running jobs can be returned to the queue for at-least-once recovery.
+
+### Dead-letter queue
+
+Retry-exhausted jobs remain terminal `failed` jobs and can also be indexed in the adapter DLQ.
+
+New queue APIs:
+
+```ts
+jobs.deadLetters()
+jobs.requeueDeadLetter()
+```
+
+The built-in memory adapter and Redis-compatible adapter implement both APIs.
+
+### Queue maintenance
+
+New operational APIs:
+
+```ts
+jobs.stats()
+jobs.cleanup()
+jobs.recoverStale()
+```
+
+Statistics include queue state totals and DLQ depth.
+
+Cleanup can remove old succeeded, failed and cancelled records after an application-defined retention period.
+
+### Redis-compatible queue adapter
+
+New public API:
+
+```ts
+createRedisJobQueueAdapter()
+```
+
+The adapter consumes a minimal application-owned Redis command client exposing:
+
+```ts
+sendCommand(command: string[]): Promise<unknown>
+```
+
+BCP does not add a Redis client runtime dependency.
+
+The Redis adapter uses atomic Lua operations for:
+
+- duplicate-safe enqueue,
+- job reservation,
+- complete/fail/cancel transitions,
+- heartbeat lease renewal,
+- stale-job recovery,
+- DLQ requeue,
+- retention cleanup.
+
+The default Redis namespace is `bcp:{jobs}` so all adapter keys share one Redis Cluster hash slot.
+
+### Redis-compatible scheduler store
+
+New public API:
+
+```ts
+createRedisJobScheduleStore()
+```
+
+The scheduler store implements the `0.2.9` `JobScheduleStore` contract on shared Redis state and uses atomic schedule leasing for multi-instance schedulers.
+
+### Compiled npm runtime
+
+The published `bcp/jobs` default runtime continues to use generated `jobs.mjs` rather than requiring consumers to execute framework TypeScript directly.
+
+The Durable Jobs package smoke imports that compiled runtime and verifies the new public APIs.
+
+## Public API additions
+
+`bcp/jobs` adds:
+
+```text
+createRedisJobQueueAdapter
+createRedisJobScheduleStore
+RedisCommandClient
+RedisJobQueueAdapter
+RedisJobScheduleStore
+RedisJobsAdapterOptions
+
+CleanupJobsOptions
+DeadLetterJobRecord
+HeartbeatJobOptions
+JobQueueStats
+ProcessNextJobOptions
+RecoverStaleJobsOptions
+RequeueDeadLetterOptions
+ReserveJobOptions
+```
+
+Existing queue and scheduling APIs remain available.
+
+## Compatibility
+
+The release is intended to be backward compatible with `0.2.9`.
+
+Existing `JobQueueAdapter` implementations remain valid because durable capabilities are optional and the new reservation metadata is provided as an optional second parameter to `reserve()`.
+
+Adapters that need production visibility-timeout semantics should implement the new durable methods.
+
+## Delivery semantics
+
+Durable jobs use at-least-once processing semantics.
+
+Applications should make side-effecting handlers idempotent when duplicate execution would be unsafe.
+
+## Validation target
+
+Before tagging/publishing `0.2.10`, run:
+
+```bash
+npm run typecheck
+npm run test:unit
+npm run test:integration
+npm run test:e2e
+npm run test:package
+npm run rc:check
+```
+
+The final release tag must point to the exact commit that passed the complete RC sequence.

package/docs/releases/0.2.11.md

@@ -0,0 +1,180 @@
+# BCP Framework 0.2.11 — Workflow Orchestration
+
+> Release state: unreleased development target until RC validation, tagging and npm publication complete.
+
+`0.2.11` adds server-side workflow orchestration on top of the durable jobs foundation introduced in `0.2.8`–`0.2.10`.
+
+## Highlights
+
+- new `bcp/workflow` server-only public entrypoint,
+- sequential workflow steps,
+- parallel step groups,
+- per-step retry policies,
+- persisted workflow delays,
+- manual resume/retry/cancel controls,
+- saga-style compensation in reverse completion order,
+- `WorkflowStore` persistence contract,
+- workflow run leases via atomic `claim()` / `release()`,
+- optional `bcp/jobs` queue-backed execution,
+- delayed workflow continuation through the job queue,
+- compiled `workflow.mjs` runtime in the npm package,
+- browser/client bundle boundary enforcement,
+- unit and prepared-package smoke coverage.
+
+## Basic API
+
+```ts
+import {
+    createWorkflow,
+} from "bcp/workflow";
+
+const onboarding =
+    createWorkflow<{
+        userId: number;
+    }>(
+        "user.onboarding",
+        workflow => {
+            workflow.step(
+                "profile",
+                createProfile
+            );
+
+            workflow.parallel(
+                "initialize",
+                parallel => {
+                    parallel.step(
+                        "preferences",
+                        createPreferences
+                    );
+                    parallel.step(
+                        "workspace",
+                        createWorkspace
+                    );
+                }
+            );
+
+            workflow.delay(
+                "cooldown",
+                1_000
+            );
+        }
+    );
+```
+
+## Queue-backed workflows
+
+Applications may provide an existing `BackgroundJobQueue`:
+
+```ts
+const workflow =
+    createWorkflow(
+        "order.fulfillment",
+        defineWorkflow,
+        {
+            queue: jobs,
+            store:
+                workflowStore,
+        }
+    );
+```
+
+`start()`, `retry()` and `resume()` then enqueue workflow execution rather than executing the run inline. Delay steps enqueue delayed continuation jobs.
+
+The queue retains responsibility for delivery, worker concurrency, visibility timeout, heartbeat and stale-job recovery.
+
+## Persistence
+
+`createMemoryWorkflowStore()` is included for development/tests.
+
+Production multi-instance deployments should implement `WorkflowStore` with shared durable storage. `claim()` must atomically lease a workflow run so multiple workers cannot execute the same run concurrently.
+
+## Retry behavior
+
+A step can define:
+
+```ts
+{
+    maxAttempts: 3,
+    retryDelayMs:
+        attempt =>
+            attempt * 1_000,
+}
+```
+
+A run that remains failed after its step retry policy is exhausted can later be retried with:
+
+```ts
+await workflow.retry(
+    runId
+);
+```
+
+Already-successful preceding steps remain complete.
+
+## Compensation
+
+Steps can define compensation handlers:
+
+```ts
+workflow.step(
+    "reserve-stock",
+    reserveStock,
+    {
+        compensate:
+            releaseStock,
+    }
+);
+```
+
+Then:
+
+```ts
+await workflow.compensate(
+    runId
+);
+```
+
+Successful compensatable steps execute in reverse completion order.
+
+## Delays
+
+Delay steps persist `waiting` state and `waitUntil` rather than holding a long-running process timer.
+
+Without a queue, application code calls `resume()` after the due time. With a queue, BCP schedules a delayed workflow continuation job.
+
+## Compatibility
+
+`0.2.11` is additive relative to `0.2.10`.
+
+Existing `bcp/jobs`, scheduler, Redis durable adapters and all prior public entrypoints remain supported.
+
+No intentional breaking changes are introduced from the `0.2.10` baseline.
+
+## Validation
+
+Before release:
+
+```bash
+npm run typecheck
+npm run test:unit
+npm run test:integration
+npm run test:e2e
+npm run test:package
+npm run rc:check
+```
+
+Workflow validation covers:
+
+- sequential step execution,
+- transient step retry,
+- parallel groups,
+- persisted delay/resume semantics,
+- manual failed-run retry,
+- reverse compensation order,
+- queue-backed execution,
+- workflow store lease behavior,
+- browser bundle rejection,
+- compiled `workflow.mjs` package execution,
+- API/platform/docs manifest parity.
+
+The final release tag must point to the exact commit that passed the complete RC sequence.

package/docs/workflow-orchestration.md

@@ -0,0 +1,374 @@
+# Workflow Orchestration
+
+BCP `0.2.11` adds a server-only workflow orchestration layer through `bcp/workflow`.
+
+The workflow runtime is designed for application processes that need to coordinate multiple backend operations as one observable run instead of manually chaining unrelated jobs.
+
+## Public entrypoint
+
+```ts
+import {
+    createMemoryWorkflowStore,
+    createWorkflow,
+} from "bcp/workflow";
+```
+
+`bcp/workflow` is server-only and must not be imported into page/client bundles.
+
+## Basic workflow
+
+```ts
+import {
+    createWorkflow,
+} from "bcp/workflow";
+
+export const onboarding =
+    createWorkflow<{
+        userId: number;
+    }>(
+        "user.onboarding",
+        workflow => {
+            workflow.step(
+                "create-profile",
+                async ({ input }) => {
+                    await createProfile(
+                        input.userId
+                    );
+                }
+            );
+
+            workflow.step(
+                "send-email",
+                async ({ input }) => {
+                    await sendWelcomeEmail(
+                        input.userId
+                    );
+                }
+            );
+        }
+    );
+```
+
+Start a run:
+
+```ts
+const run =
+    await onboarding.start({
+        userId: 42,
+    });
+```
+
+Without a background queue, `start()` executes immediately until the workflow either succeeds, fails, is cancelled or reaches a delay.
+
+## Step retries
+
+Each step can define its own retry policy:
+
+```ts
+workflow.step(
+    "charge-card",
+    chargeCard,
+    {
+        maxAttempts: 3,
+        retryDelayMs:
+            attempt =>
+                attempt * 1_000,
+    }
+);
+```
+
+The runtime persists the step attempt count and terminal error in the workflow run record.
+
+Step retries happen inside the workflow execution attempt. Long-running production workflows should prefer idempotent handlers so a process restart or at-least-once queue delivery can safely repeat work.
+
+## Parallel steps
+
+Use `parallel()` when independent operations can run together:
+
+```ts
+workflow.parallel(
+    "initialize",
+    parallel => {
+        parallel.step(
+            "preferences",
+            createPreferences
+        );
+
+        parallel.step(
+            "workspace",
+            createDefaultWorkspace
+        );
+    }
+);
+```
+
+All child step records are persisted under the parent parallel step.
+
+The parallel group succeeds only when every child succeeds.
+
+## Delays
+
+A workflow delay is persisted instead of blocking a process with a long timer:
+
+```ts
+workflow.delay(
+    "cooldown",
+    5 * 60 * 1000
+);
+```
+
+The run transitions to:
+
+```text
+waiting
+```
+
+with a persisted `waitUntil` timestamp.
+
+For an in-process workflow, resume after the timestamp:
+
+```ts
+await workflow.resume(
+    run.id
+);
+```
+
+Calling `resume()` before `waitUntil` returns the current waiting run unchanged.
+
+For administrative tooling, a forced resume is available:
+
+```ts
+await workflow.resume(
+    run.id,
+    {
+        force: true,
+    }
+);
+```
+
+## Queue-backed execution
+
+A workflow can use the BCP background job queue as its execution layer:
+
+```ts
+import {
+    createJobQueue,
+} from "bcp/jobs";
+import {
+    createWorkflow,
+} from "bcp/workflow";
+
+const jobs =
+    createJobQueue({
+        adapter:
+            redisQueueAdapter,
+    });
+
+export const workflow =
+    createWorkflow(
+        "order.fulfillment",
+        defineOrderWorkflow,
+        {
+            queue: jobs,
+            store:
+                workflowStore,
+        }
+    );
+```
+
+When a queue is configured:
+
+- `start()` persists the run and enqueues execution,
+- `retry()` enqueues another execution attempt,
+- `resume()` enqueues continuation,
+- workflow delays enqueue a delayed continuation job,
+- worker concurrency, visibility leases, heartbeat and stale-job recovery remain owned by `bcp/jobs`.
+
+This keeps job delivery and workflow state as separate contracts.
+
+## Workflow store
+
+`WorkflowStore` is the persistence boundary:
+
+```ts
+interface WorkflowStore {
+    create(run): Promise<void>;
+    update(run): Promise<void>;
+    get(id): Promise<WorkflowRunRecord | null>;
+    list(workflowName?): Promise<WorkflowRunRecord[]>;
+
+    claim(
+        id,
+        {
+            ownerId,
+            now,
+            leaseMs,
+        }
+    ): Promise<boolean>;
+
+    release(
+        id,
+        ownerId
+    ): Promise<void>;
+}
+```
+
+The built-in:
+
+```ts
+createMemoryWorkflowStore()
+```
+
+is intended for local development and deterministic tests.
+
+Production multi-instance deployments should use a shared durable store whose `claim()` operation is atomic.
+
+The claim is the workflow concurrency boundary. Only one owner should execute a run while its lease is active.
+
+## Run states
+
+Workflow runs use:
+
+```text
+pending
+running
+waiting
+succeeded
+failed
+cancelled
+compensating
+compensated
+```
+
+Step records use the corresponding lifecycle states where applicable.
+
+## Inspect runs
+
+```ts
+const run =
+    await workflow.get(
+        runId
+    );
+
+const runs =
+    await workflow.list();
+```
+
+A run includes:
+
+- workflow name,
+- input payload,
+- current state,
+- timestamps,
+- current step,
+- step attempts/errors,
+- parallel child state,
+- `waitUntil`,
+- completion order,
+- current workflow lease metadata.
+
+## Manual retry
+
+A failed run can retry its failed step without rerunning already successful preceding steps:
+
+```ts
+await workflow.retry(
+    runId
+);
+```
+
+The failed step attempt counter is reset for the new manual retry cycle.
+
+## Compensation
+
+A successful step can provide a compensation handler:
+
+```ts
+workflow.step(
+    "reserve-stock",
+    reserveStock,
+    {
+        compensate:
+            releaseStock,
+    }
+);
+```
+
+If a later step fails, application code can request compensation:
+
+```ts
+await workflow.compensate(
+    runId
+);
+```
+
+Compensation runs successful compensatable steps in reverse completion order.
+
+This is a saga-style compensation mechanism. It does not provide database transaction rollback across external services.
+
+Cancellation can also request compensation:
+
+```ts
+await workflow.cancel(
+    runId,
+    {
+        compensate: true,
+    }
+);
+```
+
+## Idempotency
+
+Workflow state and background queues provide orchestration and delivery coordination, but external side effects should still be idempotent.
+
+For example, payment and email handlers should use application-level idempotency keys where duplicate execution would be harmful.
+
+A durable queue provides at-least-once delivery semantics, not exactly-once external side effects.
+
+## Recommended production topology
+
+```text
+API / application
+       |
+       v
+WorkflowStore (shared durable state)
+       |
+       v
+BCP Workflow
+       |
+       v
+Durable JobQueueAdapter
+       |
+   +---+---+
+   |       |
+Worker A Worker B
+```
+
+The workflow store and queue do not have to use the same backend.
+
+## Lifecycle
+
+```ts
+await workflow.close();
+await jobs.close();
+```
+
+`workflow.close()` unregisters its internal queue handler and closes the configured workflow store when the store exposes `close()`.
+
+The application remains responsible for lifecycle of external database/Redis clients used by custom stores/adapters.
+
+## Current scope
+
+`0.2.11` intentionally focuses on the orchestration core:
+
+- sequential steps,
+- parallel steps,
+- step retries,
+- persisted delays,
+- manual retry/resume/cancel,
+- compensation,
+- workflow state persistence,
+- run leases,
+- optional `bcp/jobs` execution.
+
+Transactional outbox/event delivery remains a separate future milestone so workflow orchestration does not pretend to make a database transaction and external queue publish atomic.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@chidchanun/bcp",
-  "version": "0.2.9",
+  "version": "0.2.11",
   "description": "BCP Framework - a React full-stack framework with file-based routing, SSR, APIs, middleware, islands, caching and standalone production builds.",
   "type": "module",
   "license": "MIT",
@@ -67,6 +67,11 @@
       "browser": "./packages/client/src/server-only.browser.mjs",
       "default": "./packages/client/src/jobs.mjs"
     },
+    "./workflow": {
+      "types": "./packages/client/src/workflow.ts",
+      "browser": "./packages/client/src/server-only.browser.mjs",
+      "default": "./packages/client/src/workflow.mjs"
+    },
     "./observability": {
       "types": "./packages/client/src/observability.ts",
       "browser": "./packages/client/src/server-only.browser.mjs",

package/packages/bundler/src/client-boundary.ts

@@ -29,6 +29,7 @@ const SERVER_ONLY_IMPORTS =
         "bcp/database",
         "bcp/auth",
         "bcp/jobs",
+        "bcp/workflow",
         "bcp/observability",
     ]);