@chidchanun/bcp 0.2.11 → 0.2.13

package/docs/releases/0.2.13.md

@@ -0,0 +1,122 @@
+# BCP Framework 0.2.13
+
+Status: unreleased development target.
+
+## Realtime Platform
+
+`0.2.13` introduces `bcp/realtime`, a server-only provider-neutral realtime layer for channels/rooms, presence, cross-hub broker delivery, WebSocket adapter integration, SSE responses and heartbeat lifecycle.
+
+## Public API
+
+New entrypoint:
+
+```ts
+import {
+    createMemoryRealtimeBroker,
+    createMemoryRealtimePresenceStore,
+    createRealtime,
+    createRealtimeSseResponse,
+} from "bcp/realtime";
+```
+
+Key contracts:
+
+- `RealtimeHub`
+- `RealtimeConnection`
+- `RealtimeBroker`
+- `RealtimePresenceStore`
+- `RealtimeSocket`
+- `RealtimeEnvelope`
+- `RealtimeEventHandler`
+- `RealtimeAuthenticate`
+- `RealtimeAuthorizeChannel`
+
+## Channels and presence
+
+Connections can join/leave named channels and attach typed presence metadata. Hub broadcasts deliver only to local connections that joined the channel, while `RealtimeBroker` distributes envelopes across hubs.
+
+`createMemoryRealtimeBroker()` and `createMemoryRealtimePresenceStore()` provide deterministic local/test implementations.
+
+## Authentication and authorization
+
+`createRealtime()` accepts:
+
+- `authenticate` for request/data-based connection identity,
+- `getUserId` for presence identity,
+- `authorizeChannel` for private-channel join checks.
+
+Authentication policy remains application-owned so public and authenticated realtime endpoints can share the same runtime primitives.
+
+## WebSocket integration
+
+BCP does not add a mandatory WebSocket dependency. `RealtimeSocket` defines the minimal transport surface required by `attachSocket()`.
+
+The built-in JSON protocol supports:
+
+- `join`
+- `leave`
+- `event`
+- `ping`
+
+Valid client messages refresh connection liveness. Ping receives `realtime.pong`.
+
+## Server-Sent Events
+
+`RealtimeHub.sse()` and `createRealtimeSseResponse()` produce Web-standard streaming `Response` objects with event-stream headers, optional retry hints, event filtering and keep-alive comments.
+
+## Heartbeat lifecycle
+
+`startHeartbeat()` periodically sends `realtime.ping` and sweeps stale connections. `sweepStale()` can also be invoked manually for deterministic infrastructure loops and tests.
+
+## Multi-instance contract
+
+The memory broker and presence store are process-local.
+
+Production multi-instance applications can implement shared `RealtimeBroker` and `RealtimePresenceStore` adapters using infrastructure such as Redis, NATS or another service without coupling BCP to one provider/client library.
+
+## Integration model
+
+Realtime complements, but does not replace, durable platform primitives:
+
+```text
+Database / Transactional Outbox
+            |
+            v
+     Durable Jobs / Workflow
+            |
+            v
+       Realtime Hub
+            |
+      WebSocket / SSE
+```
+
+Realtime broadcasts are transient. Applications that require reconnect catch-up should refetch durable state/history after reconnect.
+
+## Package/runtime
+
+- `bcp/realtime` is server-only.
+- Client page/island graphs reject direct realtime imports.
+- publish preparation compiles `packages/client/src/realtime.ts` to `realtime.mjs`.
+- no new runtime dependency is required.
+
+## Validation
+
+The release adds unit/package coverage for:
+
+- cross-hub broker broadcasts,
+- channel membership,
+- presence metadata,
+- authentication identity,
+- channel authorization,
+- socket join/event/ping protocol,
+- stale heartbeat cleanup,
+- SSE streaming,
+- server-only boundary enforcement,
+- compiled package runtime execution,
+- public-entrypoint manifest parity.
+
+## Compatibility
+
+Previous baseline: `0.2.12`.
+
+There are no intentional breaking changes from `0.2.12`.

package/docs/transactional-outbox-events.md

@@ -0,0 +1,465 @@
+# Transactional Outbox & Events
+
+BCP `0.2.12` adds a server-only `bcp/events` entrypoint for storing integration events in the same database transaction as application data and dispatching those events after commit.
+
+## Why an outbox exists
+
+This is unsafe when the database and message system are separate resources:
+
+```ts
+await db.transaction(async tx => {
+    await tx.execute(
+        "INSERT INTO orders ..."
+    );
+
+    await jobs.enqueue(
+        "event.order.created",
+        payload
+    );
+});
+```
+
+The database can commit while the queue publish fails, or the queue can publish before the database transaction ultimately rolls back.
+
+The transactional outbox changes the write path to:
+
+```text
+Database transaction
+    |
+    +-- business row
+    +-- outbox event row
+             |
+          COMMIT
+             |
+             v
+      Outbox Dispatcher
+             |
+      queue / publisher
+```
+
+The business write and outbox write share one database transaction. External delivery happens later.
+
+## Public entrypoint
+
+```ts
+import {
+    createOutboxDispatcher,
+    createOutboxMigrationSql,
+    createSqlOutboxStore,
+    createTransactionalOutbox,
+} from "bcp/events";
+```
+
+`bcp/events` is server-only and must not be imported into page/client bundles.
+
+## Create the outbox table
+
+Generate provider-specific SQL:
+
+```ts
+import {
+    createOutboxMigrationSql,
+} from "bcp/events";
+
+const sql =
+    createOutboxMigrationSql(
+        "postgresql"
+    );
+```
+
+Supported providers match the SQL Database Platform:
+
+```text
+mysql
+postgresql
+sqlite
+```
+
+The default table name is:
+
+```text
+bcp_outbox_events
+```
+
+A custom table name must be a simple SQL identifier.
+
+Use the generated SQL in a normal BCP migration. The outbox table stores event identity/type/payload/metadata, delivery state, retry counters, timestamps and dispatcher lease information.
+
+## Create a SQL outbox
+
+```ts
+import {
+    db,
+} from "bcp/database";
+
+import {
+    createSqlOutboxStore,
+    createTransactionalOutbox,
+} from "bcp/events";
+
+export const outboxStore =
+    createSqlOutboxStore({
+        database: db,
+        driver: "postgresql",
+    });
+
+export const outbox =
+    createTransactionalOutbox({
+        store: outboxStore,
+        defaultMaxAttempts: 5,
+    });
+```
+
+## Publish inside the business transaction
+
+Pass the `TransactionDatabase` supplied by `db.transaction()` directly to `outbox.publish()`:
+
+```ts
+await db.transaction(
+    async tx => {
+        await tx.execute(
+            `INSERT INTO orders (
+                id,
+                user_id,
+                total
+            ) VALUES ($1, $2, $3)`,
+            [
+                order.id,
+                order.userId,
+                order.total,
+            ]
+        );
+
+        await outbox.publish(
+            tx,
+            "order.created",
+            {
+                orderId: order.id,
+                userId: order.userId,
+            },
+            {
+                aggregateId:
+                    String(order.id),
+                correlationId:
+                    requestId,
+            }
+        );
+    }
+);
+```
+
+For MySQL or SQLite, use the placeholder syntax expected by that database in your business SQL. `createSqlOutboxStore()` handles its own provider-specific placeholders internally.
+
+Do not publish to Redis, RabbitMQ, HTTP or another external system from inside this transaction. Only write the outbox row there.
+
+## Event metadata
+
+`publish()` accepts:
+
+```ts
+await outbox.publish(
+    tx,
+    "payment.captured",
+    payload,
+    {
+        id: eventId,
+        metadata: {
+            source: "checkout",
+        },
+        correlationId: requestId,
+        causationId: previousEventId,
+        aggregateId: paymentId,
+        delayMs: 1_000,
+        maxAttempts: 8,
+    }
+);
+```
+
+Fields have these intended uses:
+
+- `id`: stable event/idempotency identifier.
+- `correlationId`: groups work belonging to one request or distributed operation.
+- `causationId`: identifies the event/command that caused this event.
+- `aggregateId`: identifies the domain resource associated with the event.
+- `metadata`: application-defined non-secret event metadata.
+
+## Dispatcher
+
+The dispatcher claims due events using a lease, delivers them, then marks the outbox row published or schedules another attempt.
+
+```ts
+import {
+    createOutboxDispatcher,
+} from "bcp/events";
+
+export const dispatcher =
+    createOutboxDispatcher({
+        store: outboxStore,
+        queue: jobs,
+        ownerId: "outbox-a",
+        leaseMs: 30_000,
+        batchSize: 100,
+        pollIntervalMs: 1_000,
+    });
+```
+
+Start it:
+
+```ts
+const runner =
+    dispatcher.start();
+```
+
+Shutdown:
+
+```ts
+await runner.stop();
+await dispatcher.close();
+```
+
+## Durable Jobs delivery
+
+When `queue` is configured, event type `order.created` is enqueued as:
+
+```text
+event.order.created
+```
+
+Register the consumer normally:
+
+```ts
+jobs.register<{
+    eventId: string;
+    type: string;
+    payload: {
+        orderId: number;
+    };
+}>(
+    "event.order.created",
+    async ({ payload }) => {
+        await notifyOrderCreated(
+            payload.payload.orderId
+        );
+    }
+);
+```
+
+The job payload is an envelope containing:
+
+```text
+eventId
+type
+payload
+metadata
+correlationId
+causationId
+aggregateId
+createdAt
+```
+
+The queue job ID is derived from the outbox event ID:
+
+```text
+outbox:<event-id>
+```
+
+This provides a stable handoff identity. Durable job handlers should still be idempotent because the overall delivery model is at-least-once.
+
+## Custom publisher
+
+A dispatcher can publish to an application-owned broker/API instead of the BCP job queue:
+
+```ts
+const dispatcher =
+    createOutboxDispatcher({
+        store: outboxStore,
+        publish: async event => {
+            await broker.publish(
+                event.type,
+                event
+            );
+        },
+    });
+```
+
+BCP intentionally does not require RabbitMQ, Kafka, NATS or another event-broker dependency.
+
+## In-process event bus
+
+For local application delivery:
+
+```ts
+import {
+    createEventBus,
+} from "bcp/events";
+
+const bus =
+    createEventBus();
+
+bus.on(
+    "account.updated",
+    async ({ payload }) => {
+        await refreshAccountCache(
+            payload
+        );
+    }
+);
+```
+
+Then pass `eventBus: bus` to `createOutboxDispatcher()`.
+
+The in-process bus is not a durable distributed broker. Use it for local handlers, tests or composition, not as a replacement for a shared queue in multi-instance deployments.
+
+## Retry and terminal failure
+
+A claimed event increments `attempts`.
+
+If delivery throws and attempts remain, the dispatcher returns the event to `pending` with a future `availableAt`.
+
+The default delay is exponential with a 60-second cap. Override it:
+
+```ts
+createOutboxDispatcher({
+    store: outboxStore,
+    publish,
+    retryDelayMs:
+        attempt =>
+            attempt * 2_000,
+});
+```
+
+When `maxAttempts` is exhausted the event becomes:
+
+```text
+failed
+```
+
+The record remains available for inspection/retention until cleanup.
+
+## Leases and stale recovery
+
+Claimed events use:
+
+```text
+state = processing
+leaseOwner
+leaseUntil
+```
+
+If a dispatcher crashes before acknowledgement, another dispatcher can recover the expired lease and return the event to `pending`.
+
+Manual recovery:
+
+```ts
+await dispatcher.recoverStale(
+    100
+);
+```
+
+Production SQL stores perform claim/update operations through the configured BCP database transaction boundary. PostgreSQL/MySQL claims also use `FOR UPDATE SKIP LOCKED`; SQLite uses its transactional write serialization and conditional state updates.
+
+## States
+
+```text
+pending
+   |
+   v
+processing
+   | \
+   |  \ delivery error + attempts left
+   |   -----------------> pending
+   |
+   +--------------------> published
+   |
+   + retry exhausted ---> failed
+```
+
+`published` means the configured dispatcher destination accepted the event. For job-queue delivery, it means enqueue succeeded; it does not mean the downstream job handler has completed.
+
+## Statistics
+
+```ts
+const stats =
+    await outboxStore.stats();
+```
+
+Shape:
+
+```ts
+{
+    total: number;
+    pending: number;
+    processing: number;
+    published: number;
+    failed: number;
+}
+```
+
+## Retention cleanup
+
+Remove old terminal records:
+
+```ts
+await outboxStore.cleanup({
+    before:
+        Date.now() -
+        7 * 24 * 60 * 60 * 1000,
+});
+```
+
+Only `published` and `failed` records are eligible. Pending or processing records are preserved.
+
+## Memory store
+
+For tests and single-process development:
+
+```ts
+import {
+    createMemoryOutboxStore,
+} from "bcp/events";
+```
+
+The memory store demonstrates the `OutboxStore` contract but is not transactionally durable. Its `append()` accepts the transaction parameter for API compatibility, but its data lives only in process memory.
+
+Use `createSqlOutboxStore()` when the outbox must participate in the same durable database transaction as application writes.
+
+## Delivery guarantees
+
+The outbox prevents the database-write / external-publish gap, but it does not provide exactly-once side effects.
+
+Treat delivery as:
+
+```text
+at-least-once
+```
+
+Use stable event IDs and consumer-level idempotency for external side effects such as charging cards, sending one-time notifications or calling third-party APIs.
+
+## Production topology
+
+```text
+API / Form Action
+      |
+      v
+Database transaction
+      |
+      +-- business data
+      +-- outbox event
+      |
+    COMMIT
+      |
+      v
+Shared SQL Outbox
+      |
+  +---+---+
+  |       |
+Dispatcher A  Dispatcher B
+      |
+      v
+Durable queue / broker
+      |
+  +---+---+
+Worker A  Worker B
+```
+
+This allows multiple API, dispatcher and worker instances without coupling the business transaction to the external broker availability window.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@chidchanun/bcp",
-  "version": "0.2.11",
+  "version": "0.2.13",
   "description": "BCP Framework - a React full-stack framework with file-based routing, SSR, APIs, middleware, islands, caching and standalone production builds.",
   "type": "module",
   "license": "MIT",
@@ -72,6 +72,16 @@
       "browser": "./packages/client/src/server-only.browser.mjs",
       "default": "./packages/client/src/workflow.mjs"
     },
+    "./events": {
+      "types": "./packages/client/src/events.ts",
+      "browser": "./packages/client/src/server-only.browser.mjs",
+      "default": "./packages/client/src/events.mjs"
+    },
+    "./realtime": {
+      "types": "./packages/client/src/realtime.ts",
+      "browser": "./packages/client/src/server-only.browser.mjs",
+      "default": "./packages/client/src/realtime.mjs"
+    },
     "./observability": {
       "types": "./packages/client/src/observability.ts",
       "browser": "./packages/client/src/server-only.browser.mjs",

package/packages/bundler/src/client-boundary.ts

@@ -30,6 +30,8 @@ const SERVER_ONLY_IMPORTS =
         "bcp/auth",
         "bcp/jobs",
         "bcp/workflow",
+        "bcp/events",
+        "bcp/realtime",
         "bcp/observability",
     ]);