@chidchanun/bcp 0.2.11 → 0.2.12

package/docs/releases/0.2.12.md

@@ -0,0 +1,147 @@
+# BCP Framework 0.2.12 — Transactional Outbox & Events
+
+Release state: unreleased development target.
+
+`0.2.12` adds a server-only transactional outbox and event-delivery platform that bridges BCP Database transactions with durable jobs or application-owned publishers without performing external side effects inside the business transaction.
+
+## New public entrypoint
+
+```ts
+import {
+    createEventBus,
+    createMemoryOutboxStore,
+    createOutboxDispatcher,
+    createOutboxMigrationSql,
+    createSqlOutboxStore,
+    createTransactionalOutbox,
+} from "bcp/events";
+```
+
+`bcp/events` is server-only and receives the same browser/client boundary protection as `bcp/database`, `bcp/jobs` and `bcp/workflow`.
+
+## Transaction-bound outbox writes
+
+Application code can insert business rows and outbox rows through the same `TransactionDatabase`:
+
+```ts
+await db.transaction(async tx => {
+    await tx.execute(
+        "INSERT INTO orders ..."
+    );
+
+    await outbox.publish(
+        tx,
+        "order.created",
+        {
+            orderId: 42,
+        }
+    );
+});
+```
+
+`createSqlOutboxStore()` writes through the caller transaction rather than the root database connection.
+
+## SQL providers
+
+Provider-specific migration SQL is available for:
+
+```text
+mysql
+postgresql
+sqlite
+```
+
+The SQL store persists:
+
+- event ID/type/payload,
+- metadata,
+- correlation/causation/aggregate IDs,
+- state,
+- attempts/max attempts,
+- created/available/processing/published/failed timestamps,
+- delivery errors,
+- lease owner/expiry.
+
+## Dispatcher
+
+`createOutboxDispatcher()` adds:
+
+- batched claiming,
+- dispatcher leases,
+- stale-lease recovery,
+- retry/backoff,
+- terminal failed state,
+- durable job-queue handoff,
+- custom publisher delivery,
+- in-process EventBus delivery,
+- polling runner lifecycle,
+- cleanup/statistics through the store.
+
+## Durable job handoff
+
+With `queue: jobs`, event type:
+
+```text
+order.created
+```
+
+is handed off as:
+
+```text
+event.order.created
+```
+
+The queued payload contains the event envelope and uses stable job ID:
+
+```text
+outbox:<event-id>
+```
+
+A successful enqueue marks the outbox event `published`. Downstream job completion remains the responsibility of the job runtime.
+
+## Event bus
+
+`createEventBus()` provides local server-side handler registration and sequential event delivery for tests, local composition and single-process handlers.
+
+It is not a distributed durable broker.
+
+## States
+
+```text
+pending
+processing
+published
+failed
+```
+
+Expired `processing` leases can return to `pending` for another dispatcher attempt.
+
+## Delivery semantics
+
+The platform closes the database-commit / external-publish gap but intentionally uses at-least-once delivery semantics.
+
+Consumers should use stable event IDs and application-level idempotency when duplicate external side effects are unsafe.
+
+## Backward compatibility
+
+`0.2.12` is additive relative to `0.2.11`:
+
+- existing `bcp/jobs` APIs remain unchanged,
+- existing `bcp/workflow` APIs remain unchanged,
+- no database adapter contract method was removed,
+- existing applications do not need an outbox unless they opt into `bcp/events`.
+
+## Validation
+
+The release candidate must pass:
+
+```bash
+npm run typecheck
+npm run test:unit
+npm run test:integration
+npm run test:e2e
+npm run test:package
+npm run rc:check
+```
+
+Package validation executes the compiled `events.mjs` runtime and verifies the `bcp/events` export, transaction/outbox API, dispatcher delivery and migration helper.

package/docs/transactional-outbox-events.md

@@ -0,0 +1,465 @@
+# Transactional Outbox & Events
+
+BCP `0.2.12` adds a server-only `bcp/events` entrypoint for storing integration events in the same database transaction as application data and dispatching those events after commit.
+
+## Why an outbox exists
+
+This is unsafe when the database and message system are separate resources:
+
+```ts
+await db.transaction(async tx => {
+    await tx.execute(
+        "INSERT INTO orders ..."
+    );
+
+    await jobs.enqueue(
+        "event.order.created",
+        payload
+    );
+});
+```
+
+The database can commit while the queue publish fails, or the queue can publish before the database transaction ultimately rolls back.
+
+The transactional outbox changes the write path to:
+
+```text
+Database transaction
+    |
+    +-- business row
+    +-- outbox event row
+             |
+          COMMIT
+             |
+             v
+      Outbox Dispatcher
+             |
+      queue / publisher
+```
+
+The business write and outbox write share one database transaction. External delivery happens later.
+
+## Public entrypoint
+
+```ts
+import {
+    createOutboxDispatcher,
+    createOutboxMigrationSql,
+    createSqlOutboxStore,
+    createTransactionalOutbox,
+} from "bcp/events";
+```
+
+`bcp/events` is server-only and must not be imported into page/client bundles.
+
+## Create the outbox table
+
+Generate provider-specific SQL:
+
+```ts
+import {
+    createOutboxMigrationSql,
+} from "bcp/events";
+
+const sql =
+    createOutboxMigrationSql(
+        "postgresql"
+    );
+```
+
+Supported providers match the SQL Database Platform:
+
+```text
+mysql
+postgresql
+sqlite
+```
+
+The default table name is:
+
+```text
+bcp_outbox_events
+```
+
+A custom table name must be a simple SQL identifier.
+
+Use the generated SQL in a normal BCP migration. The outbox table stores event identity/type/payload/metadata, delivery state, retry counters, timestamps and dispatcher lease information.
+
+## Create a SQL outbox
+
+```ts
+import {
+    db,
+} from "bcp/database";
+
+import {
+    createSqlOutboxStore,
+    createTransactionalOutbox,
+} from "bcp/events";
+
+export const outboxStore =
+    createSqlOutboxStore({
+        database: db,
+        driver: "postgresql",
+    });
+
+export const outbox =
+    createTransactionalOutbox({
+        store: outboxStore,
+        defaultMaxAttempts: 5,
+    });
+```
+
+## Publish inside the business transaction
+
+Pass the `TransactionDatabase` supplied by `db.transaction()` directly to `outbox.publish()`:
+
+```ts
+await db.transaction(
+    async tx => {
+        await tx.execute(
+            `INSERT INTO orders (
+                id,
+                user_id,
+                total
+            ) VALUES ($1, $2, $3)`,
+            [
+                order.id,
+                order.userId,
+                order.total,
+            ]
+        );
+
+        await outbox.publish(
+            tx,
+            "order.created",
+            {
+                orderId: order.id,
+                userId: order.userId,
+            },
+            {
+                aggregateId:
+                    String(order.id),
+                correlationId:
+                    requestId,
+            }
+        );
+    }
+);
+```
+
+For MySQL or SQLite, use the placeholder syntax expected by that database in your business SQL. `createSqlOutboxStore()` handles its own provider-specific placeholders internally.
+
+Do not publish to Redis, RabbitMQ, HTTP or another external system from inside this transaction. Only write the outbox row there.
+
+## Event metadata
+
+`publish()` accepts:
+
+```ts
+await outbox.publish(
+    tx,
+    "payment.captured",
+    payload,
+    {
+        id: eventId,
+        metadata: {
+            source: "checkout",
+        },
+        correlationId: requestId,
+        causationId: previousEventId,
+        aggregateId: paymentId,
+        delayMs: 1_000,
+        maxAttempts: 8,
+    }
+);
+```
+
+Fields have these intended uses:
+
+- `id`: stable event/idempotency identifier.
+- `correlationId`: groups work belonging to one request or distributed operation.
+- `causationId`: identifies the event/command that caused this event.
+- `aggregateId`: identifies the domain resource associated with the event.
+- `metadata`: application-defined non-secret event metadata.
+
+## Dispatcher
+
+The dispatcher claims due events using a lease, delivers them, then marks the outbox row published or schedules another attempt.
+
+```ts
+import {
+    createOutboxDispatcher,
+} from "bcp/events";
+
+export const dispatcher =
+    createOutboxDispatcher({
+        store: outboxStore,
+        queue: jobs,
+        ownerId: "outbox-a",
+        leaseMs: 30_000,
+        batchSize: 100,
+        pollIntervalMs: 1_000,
+    });
+```
+
+Start it:
+
+```ts
+const runner =
+    dispatcher.start();
+```
+
+Shutdown:
+
+```ts
+await runner.stop();
+await dispatcher.close();
+```
+
+## Durable Jobs delivery
+
+When `queue` is configured, event type `order.created` is enqueued as:
+
+```text
+event.order.created
+```
+
+Register the consumer normally:
+
+```ts
+jobs.register<{
+    eventId: string;
+    type: string;
+    payload: {
+        orderId: number;
+    };
+}>(
+    "event.order.created",
+    async ({ payload }) => {
+        await notifyOrderCreated(
+            payload.payload.orderId
+        );
+    }
+);
+```
+
+The job payload is an envelope containing:
+
+```text
+eventId
+type
+payload
+metadata
+correlationId
+causationId
+aggregateId
+createdAt
+```
+
+The queue job ID is derived from the outbox event ID:
+
+```text
+outbox:<event-id>
+```
+
+This provides a stable handoff identity. Durable job handlers should still be idempotent because the overall delivery model is at-least-once.
+
+## Custom publisher
+
+A dispatcher can publish to an application-owned broker/API instead of the BCP job queue:
+
+```ts
+const dispatcher =
+    createOutboxDispatcher({
+        store: outboxStore,
+        publish: async event => {
+            await broker.publish(
+                event.type,
+                event
+            );
+        },
+    });
+```
+
+BCP intentionally does not require RabbitMQ, Kafka, NATS or another event-broker dependency.
+
+## In-process event bus
+
+For local application delivery:
+
+```ts
+import {
+    createEventBus,
+} from "bcp/events";
+
+const bus =
+    createEventBus();
+
+bus.on(
+    "account.updated",
+    async ({ payload }) => {
+        await refreshAccountCache(
+            payload
+        );
+    }
+);
+```
+
+Then pass `eventBus: bus` to `createOutboxDispatcher()`.
+
+The in-process bus is not a durable distributed broker. Use it for local handlers, tests or composition, not as a replacement for a shared queue in multi-instance deployments.
+
+## Retry and terminal failure
+
+A claimed event increments `attempts`.
+
+If delivery throws and attempts remain, the dispatcher returns the event to `pending` with a future `availableAt`.
+
+The default delay is exponential with a 60-second cap. Override it:
+
+```ts
+createOutboxDispatcher({
+    store: outboxStore,
+    publish,
+    retryDelayMs:
+        attempt =>
+            attempt * 2_000,
+});
+```
+
+When `maxAttempts` is exhausted the event becomes:
+
+```text
+failed
+```
+
+The record remains available for inspection/retention until cleanup.
+
+## Leases and stale recovery
+
+Claimed events use:
+
+```text
+state = processing
+leaseOwner
+leaseUntil
+```
+
+If a dispatcher crashes before acknowledgement, another dispatcher can recover the expired lease and return the event to `pending`.
+
+Manual recovery:
+
+```ts
+await dispatcher.recoverStale(
+    100
+);
+```
+
+Production SQL stores perform claim/update operations through the configured BCP database transaction boundary. PostgreSQL/MySQL claims also use `FOR UPDATE SKIP LOCKED`; SQLite uses its transactional write serialization and conditional state updates.
+
+## States
+
+```text
+pending
+   |
+   v
+processing
+   | \
+   |  \ delivery error + attempts left
+   |   -----------------> pending
+   |
+   +--------------------> published
+   |
+   + retry exhausted ---> failed
+```
+
+`published` means the configured dispatcher destination accepted the event. For job-queue delivery, it means enqueue succeeded; it does not mean the downstream job handler has completed.
+
+## Statistics
+
+```ts
+const stats =
+    await outboxStore.stats();
+```
+
+Shape:
+
+```ts
+{
+    total: number;
+    pending: number;
+    processing: number;
+    published: number;
+    failed: number;
+}
+```
+
+## Retention cleanup
+
+Remove old terminal records:
+
+```ts
+await outboxStore.cleanup({
+    before:
+        Date.now() -
+        7 * 24 * 60 * 60 * 1000,
+});
+```
+
+Only `published` and `failed` records are eligible. Pending or processing records are preserved.
+
+## Memory store
+
+For tests and single-process development:
+
+```ts
+import {
+    createMemoryOutboxStore,
+} from "bcp/events";
+```
+
+The memory store demonstrates the `OutboxStore` contract but is not transactionally durable. Its `append()` accepts the transaction parameter for API compatibility, but its data lives only in process memory.
+
+Use `createSqlOutboxStore()` when the outbox must participate in the same durable database transaction as application writes.
+
+## Delivery guarantees
+
+The outbox prevents the database-write / external-publish gap, but it does not provide exactly-once side effects.
+
+Treat delivery as:
+
+```text
+at-least-once
+```
+
+Use stable event IDs and consumer-level idempotency for external side effects such as charging cards, sending one-time notifications or calling third-party APIs.
+
+## Production topology
+
+```text
+API / Form Action
+      |
+      v
+Database transaction
+      |
+      +-- business data
+      +-- outbox event
+      |
+    COMMIT
+      |
+      v
+Shared SQL Outbox
+      |
+  +---+---+
+  |       |
+Dispatcher A  Dispatcher B
+      |
+      v
+Durable queue / broker
+      |
+  +---+---+
+Worker A  Worker B
+```
+
+This allows multiple API, dispatcher and worker instances without coupling the business transaction to the external broker availability window.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@chidchanun/bcp",
-  "version": "0.2.11",
+  "version": "0.2.12",
   "description": "BCP Framework - a React full-stack framework with file-based routing, SSR, APIs, middleware, islands, caching and standalone production builds.",
   "type": "module",
   "license": "MIT",
@@ -72,6 +72,11 @@
       "browser": "./packages/client/src/server-only.browser.mjs",
       "default": "./packages/client/src/workflow.mjs"
     },
+    "./events": {
+      "types": "./packages/client/src/events.ts",
+      "browser": "./packages/client/src/server-only.browser.mjs",
+      "default": "./packages/client/src/events.mjs"
+    },
     "./observability": {
       "types": "./packages/client/src/observability.ts",
       "browser": "./packages/client/src/server-only.browser.mjs",

package/packages/bundler/src/client-boundary.ts

@@ -30,6 +30,7 @@ const SERVER_ONLY_IMPORTS =
         "bcp/auth",
         "bcp/jobs",
         "bcp/workflow",
+        "bcp/events",
         "bcp/observability",
     ]);