@nestarc/inbox 0.1.0 → 0.2.0

package/CHANGELOG.md

@@ -2,6 +2,27 @@
 
 User-facing changes and upgrade actions for `@nestarc/inbox`.
 
+## 0.2.0 — 2026-10-28
+
+Adds application integration-test helpers and reproducible operational recovery checks. Runtime processing contracts and SQL schema v2 are unchanged; upgrading from 0.1.0 requires no database migration.
+
+### Testing helpers
+
+- Added `@nestarc/inbox/testing` with `drainInbox()`, `waitForInboxRecord()`, and typed `InboxTestError` diagnostics. Helpers wait through persisted retry schedules and bound database reads and worker calls with a timeout, cancellation signal, and pass limit. They use the real service and scoped admin API; they do not create databases or cancel transactions already in flight. [Testing guide](docs/testing.md)
+- Verified the subpath with unit tests, PostgreSQL integration tests, and a separately installed tarball consumer using strict TypeScript, CommonJS, and dynamic ESM.
+
+### Operational validation
+
+- Added actual TCP disconnect/blackhole, connection-pool saturation, and child-process SIGTERM recovery tests.
+- Added PostgreSQL physical standby promotion after confirmed WAL replay and real SQLSTATE `53100` in a bounded disposable tablespace, including rollback and recovery assertions. [Database operations](docs/db-operations.md)
+- Added RabbitMQ process restarts, unacknowledged redelivery, persisted retry/TTL delivery, and dead-letter routing tests. [Broker recovery](docs/broker-recovery.md)
+- Added a disk-backed sustained benchmark with simultaneous ingress and processing, pool pressure, hot/cold tenants, retry recovery, and receipt/payload storage measurements. It verifies business effects separately from receipt completion. These bounded local measurements do not establish production capacity or an SLA.
+- Added a Node 24/PostgreSQL 16 operational recovery gate to CI and release validation. Real-service adoption remains outside this release's validation scope.
+
+### Release automation
+
+- Wait up to 20 minutes for npm's post-publication scanning before verifying registry integrity. Pending 404 responses are retried every 30 seconds; integrity mismatches and other lookup failures still stop immediately. This prevents successful uploads from being reported as failed after only 16 seconds. No runtime API or schema changes.
+
 ## 0.1.0 — 2026-10-28
 
 First public release version. It promotes the runtime implementation and schema verified as `0.1.0-preview.1`, with version metadata and documentation updates. Preview.0 and Preview.1 were local development baselines, not published releases. APIs and schema may change before 1.0.

package/README.md

@@ -4,9 +4,9 @@
 
 A transactional inbox for **NestJS, Prisma, and PostgreSQL**. Store an incoming event durably, acknowledge its delivery, then commit your business writes and inbox completion in one database transaction. Retries and crash recovery use the same durable record.
 
-**Version `0.1.0` · SQL schema v2.** APIs and schema may change before 1.0; review the [changelog](CHANGELOG.md) and migration steps when upgrading.
+**Version `0.2.0` · SQL schema v2.** APIs and schema may change before 1.0; review the [changelog](CHANGELOG.md) and migration steps when upgrading.
 
-[Install](#1-install-the-package) · [Try it locally](#try-it-locally) · [Integrate with NestJS](#integrate-with-nestjs) · [Reliability evidence](#reliability-evidence) · [Performance](#performance) · [Usage guide](docs/usage.md)
+[Install](#1-install-the-package) · [Try it locally](#try-it-locally) · [Integrate with NestJS](#integrate-with-nestjs) · [Reliability evidence](#reliability-evidence) · [Performance](#performance) · [Usage guide](docs/usage.md) · [Testing helpers](docs/testing.md)
 
 ## Why use Inbox?
 
@@ -69,7 +69,7 @@ npm run test:package
 This builds a tarball, installs it in a separate consumer, starts a disposable PostgreSQL database, and checks types, imports, SQL assets, deduplication, rollback, replay, and scoped administration. A successful run ends with:
 
 ```text
-Installed @nestarc/inbox 0.1.0 tarball verification passed.
+Installed @nestarc/inbox 0.2.0 tarball verification passed.
 ```
 
 The runner ignores an existing application `DATABASE_URL`, uses a random local port, and removes its container afterward. The first run may download dependencies and a database image. Docker or database failures fail the check rather than skipping it.
@@ -78,6 +78,11 @@ To explore the integrations with temporary databases and brokers:
 
 ```sh
 npm run test:examples
+npm run test:operations
+npm run test:operations:database
+# Install example dependencies once for the RabbitMQ recovery fixture:
+npm install --prefix examples --no-audit --no-fund
+npm run test:broker-recovery
 ```
 
 That runs all eight [examples](examples/README.md), including an outbox → inbox → business write → outbox chain. You can also follow the examples guide to run one integration at a time.
@@ -97,7 +102,9 @@ The implementation is checked against real databases and installed package artif
 
 Recorded checks for the local Preview.1 baseline include **85 unit tests**, **38 database/crash tests on each of PostgreSQL 16 and 18**, **41 release checks on Node 22 and 24**, and **eight runnable integrations**. GitHub-hosted [CI run 40684612217](https://github.com/nestarc/inbox/actions/runs/40684612217) passed all four Node/PostgreSQL combinations and the examples at commit `664b4fc`. Version 0.1.0 promotes that runtime implementation and schema; the promotion changes version metadata and documentation. Counts describe the recorded runs, not a guarantee of every runtime or failure scenario. [Read the test-to-contract mapping and remaining validation](docs/validation.md).
 
-A lost commit response is tested by error injection after a real commit. Actual network partitions, database failover, disk exhaustion, and sustained pool saturation remain separate operational exercises.
+The 0.2.0 follow-up adds **20 testing-helper unit checks**, **three PostgreSQL helper cases**, and actual TCP faults, pool saturation, SIGTERM, PostgreSQL standby promotion, bounded tablespace exhaustion, and RabbitMQ restart/retry/DLQ recovery checks. The [current validation record](docs/validation.md) distinguishes each tested condition from deployment-specific guarantees. Existing 0.1.0 installations keep schema v2 without a migration.
+
+A lost commit response is tested by error injection after a real commit. Separate operational tests exercise actual connection loss and blackholes, standby promotion after confirmed WAL replay, business-tablespace ENOSPC, and bounded pool pressure. Automatic failover routing, unreplicated commit loss, WAL-volume exhaustion, and application-specific soak tests remain deployment concerns.
 
 ## Performance
 
@@ -121,6 +128,8 @@ The implementation uses partial indexes for due work and expired leases, `SKIP L
 npm run benchmark
 ```
 
+For simultaneous ingress and processing on a disk-backed fixture, run `npm run benchmark:sustained`. The [sustained load report](docs/benchmarks/2026-10-28-sustained-local.md) records overload recovery, pool waiting, hot/cold tenant latency, and payload retention. It is a separate workload from the short acceptance benchmark above.
+
 ## Integrate with NestJS
 
 The following steps target an **existing NestJS 11 application with Prisma 7**, using the versions in the compatibility table above. Your application owns its Prisma client, database migrations, transport credentials, and shutdown lifecycle.
@@ -130,14 +139,14 @@ The following steps target an **existing NestJS 11 application with Prisma 7**,
 In your application directory, install Inbox and the adapter matching your Prisma client:
 
 ```sh
-npm install @nestarc/inbox@0.1.0
+npm install @nestarc/inbox@0.2.0
 npm install @prisma/adapter-pg@7.10.1
 ```
 
 For a local source build, run `npm ci` and `npm pack` in this repository checkout. `prepack` builds the library. Install the resulting tarball in your application instead:
 
 ```sh
-npm install /absolute/path/to/inbox/nestarc-inbox-0.1.0.tgz
+npm install /absolute/path/to/inbox/nestarc-inbox-0.2.0.tgz
 ```
 
 ### 2. Apply the schema
@@ -285,11 +294,16 @@ npm run test:release
 npm run test:integration
 npm run test:package
 npm run test:examples
+npm run test:operations
+npm run test:operations:database
+# Install example dependencies once for the RabbitMQ recovery fixture:
+npm install --prefix examples --no-audit --no-fund
+npm run test:broker-recovery
 ```
 
-Database checks require Docker. Set `INBOX_TEST_POSTGRES_IMAGE=postgres:18-alpine` to select PostgreSQL 18; the fixture default is 16.
+Database checks require Docker. Set `INBOX_TEST_POSTGRES_IMAGE=postgres:18-alpine` to select PostgreSQL 18 for the general fixture; its default is 16. The dedicated `test:operations:database` primary/standby and ENOSPC suite is pinned to PostgreSQL 16.
 
-CI and releases share a Node 22/24 × PostgreSQL 16/18 validation workflow. A GitHub Release is configured to publish the exact tested tarball through npm OIDC: prereleases to `next`, release versions such as 0.1.0 to `latest`. Manual runs only validate and dry-run. [Release setup and publishing procedure](docs/releasing.md)
+CI and releases share a Node 22/24 × PostgreSQL 16/18 validation workflow. A GitHub Release is configured to publish the exact tested tarball through npm OIDC: prereleases to `next`, release versions such as 0.2.0 to `latest`. A separate Node 24/PostgreSQL 16 gate checks operational recovery. Manual runs only validate and dry-run. [Release setup and publishing procedure](docs/releasing.md)
 
 [Changelog](CHANGELOG.md) · [Full usage guide](docs/usage.md) · [Validation record](docs/validation.md) · [Architecture decision](docs/adr/0001-durable-inbox.md) · [Roadmap](docs/implementation-roadmap.md)
 

package/dist/testing/index.d.ts

@@ -0,0 +1,40 @@
+import type { InboxRecord, InboxStatus } from "../contracts/types";
+import type { InboxAdminView } from "../inbox-admin.service";
+import type { InboxService } from "../inbox.service";
+export interface InboxTestOptions {
+    /** Overall wait budget, including database reads and worker execution. Default: 5,000. */
+    timeoutMs?: number;
+    /** Delay between state checks. Default: 10. */
+    pollIntervalMs?: number;
+    /** Maximum state-check passes. Default: 1,000. */
+    maxPasses?: number;
+    signal?: AbortSignal;
+}
+export interface InboxWaitOptions extends InboxTestOptions {
+    /** The terminal state to wait for. Default: COMPLETED. */
+    status?: "COMPLETED" | "DEAD";
+}
+/** Safe diagnostics; deliberately excludes payloads, scope and handler error text. */
+export interface InboxTestSnapshot {
+    id: string;
+    status: InboxStatus | null;
+    attemptCount: number | null;
+    cycleAttemptCount: number | null;
+}
+export type InboxTestErrorCode = "INVALID_OPTIONS" | "TIMEOUT" | "ABORTED" | "PASS_LIMIT" | "UNEXPECTED_TERMINAL_STATE";
+export declare class InboxTestError extends Error {
+    readonly code: InboxTestErrorCode;
+    readonly records: readonly InboxTestSnapshot[];
+    constructor(code: InboxTestErrorCode, message: string, records?: readonly InboxTestSnapshot[]);
+}
+type Reader = Pick<InboxAdminView, "get">;
+type Runner = Pick<InboxService, "runOnce">;
+/**
+ * Run the real worker until the requested receipts are COMPLETED.
+ * IDs select assertions, not claims: runOnce may process other owned receipts.
+ * Timeout/cancellation stops waiting and new polls; it cannot cancel an in-flight transaction.
+ */
+export declare function drainInbox(inbox: Runner, admin: Reader, ids: readonly string[], options?: InboxTestOptions): Promise<InboxRecord[]>;
+/** Observe an existing worker without starting processing or changing the receipt. */
+export declare function waitForInboxRecord(admin: Reader, id: string, options?: InboxWaitOptions): Promise<InboxRecord>;
+export {};

package/dist/testing/index.js

@@ -0,0 +1,120 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.InboxTestError = void 0;
+exports.drainInbox = drainInbox;
+exports.waitForInboxRecord = waitForInboxRecord;
+const abort_1 = require("../internal/reliability-core/abort");
+class InboxTestError extends Error {
+    code;
+    records;
+    constructor(code, message, records = []) {
+        super(message);
+        this.code = code;
+        this.name = "InboxTestError";
+        this.records = records.map((record) => ({ ...record }));
+    }
+}
+exports.InboxTestError = InboxTestError;
+function positiveInteger(value, name) {
+    if (!Number.isSafeInteger(value) || value < 1 || value > 2147483647)
+        throw new InboxTestError("INVALID_OPTIONS", `${name} must be a positive 32-bit integer.`);
+    return value;
+}
+function receiptIds(ids) {
+    if (!Array.isArray(ids) ||
+        ids.length > 1000 ||
+        Array.from(ids).some((id) => typeof id !== "string" ||
+            !/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(id)))
+        throw new InboxTestError("INVALID_OPTIONS", "Provide at most 1,000 receipt UUIDs.");
+    return [...new Set(ids.map((id) => id.toLowerCase()))];
+}
+function pause(ms, signal) {
+    return new Promise((resolve, reject) => {
+        const aborted = () => {
+            clearTimeout(timer);
+            signal.removeEventListener("abort", aborted);
+            reject(signal.reason);
+        };
+        const timer = setTimeout(() => {
+            signal.removeEventListener("abort", aborted);
+            resolve();
+        }, ms);
+        signal.addEventListener("abort", aborted, { once: true });
+        if (signal.aborted)
+            aborted();
+    });
+}
+async function awaitRecords(reader, ids, options, runner) {
+    const timeoutMs = positiveInteger(options.timeoutMs ?? 5000, "timeoutMs");
+    const pollIntervalMs = positiveInteger(options.pollIntervalMs ?? 10, "pollIntervalMs");
+    const maxPasses = positiveInteger(options.maxPasses ?? 1000, "maxPasses");
+    const status = options.status ?? "COMPLETED";
+    if (status !== "COMPLETED" && status !== "DEAD")
+        throw new InboxTestError("INVALID_OPTIONS", "status must be COMPLETED or DEAD.");
+    const snapshots = ids.map((id) => ({
+        id,
+        status: null,
+        attemptCount: null,
+        cycleAttemptCount: null,
+    }));
+    const controller = new AbortController();
+    const fail = (code, message) => new InboxTestError(code, message, snapshots);
+    const aborted = () => controller.abort(fail("ABORTED", "Inbox test wait was aborted."));
+    options.signal?.addEventListener("abort", aborted, { once: true });
+    if (options.signal?.aborted)
+        aborted();
+    const timer = setTimeout(() => controller.abort(fail("TIMEOUT", `Inbox test did not reach ${status} within ${timeoutMs}ms.`)), timeoutMs);
+    const run = (fn) => (0, abort_1.withAbort)(controller.signal, fn);
+    try {
+        return await run(async () => {
+            if (ids.length === 0)
+                return [];
+            for (let pass = 0; pass < maxPasses; pass++) {
+                const records = [];
+                for (let index = 0; index < ids.length; index++) {
+                    const record = await run(() => reader.get(ids[index]));
+                    snapshots[index] = {
+                        id: ids[index],
+                        status: record?.status ?? null,
+                        attemptCount: record?.attemptCount ?? null,
+                        cycleAttemptCount: record?.cycleAttemptCount ?? null,
+                    };
+                    if (record)
+                        records.push(record);
+                    if (record &&
+                        record.status !== status &&
+                        (record.status === "DEAD" || record.status === "COMPLETED"))
+                        throw fail("UNEXPECTED_TERMINAL_STATE", `Inbox receipt ${record.id} reached ${record.status}; expected ${status}.`);
+                }
+                if (records.length === ids.length &&
+                    records.every((record) => record.status === status))
+                    return records;
+                if (pass + 1 === maxPasses)
+                    break;
+                if (runner)
+                    await run(() => runner.runOnce());
+                // A zero-claim poll may mean a scheduled retry or a live lease, not completion.
+                await pause(pollIntervalMs, controller.signal);
+            }
+            throw fail("PASS_LIMIT", `Inbox test did not reach ${status} after ${maxPasses} state checks.`);
+        });
+    }
+    finally {
+        clearTimeout(timer);
+        options.signal?.removeEventListener("abort", aborted);
+    }
+}
+/**
+ * Run the real worker until the requested receipts are COMPLETED.
+ * IDs select assertions, not claims: runOnce may process other owned receipts.
+ * Timeout/cancellation stops waiting and new polls; it cannot cancel an in-flight transaction.
+ */
+async function drainInbox(inbox, admin, ids, options = {}) {
+    return awaitRecords(admin, receiptIds(ids), { ...options, status: "COMPLETED" }, inbox);
+}
+/** Observe an existing worker without starting processing or changing the receipt. */
+async function waitForInboxRecord(admin, id, options = {}) {
+    const records = await awaitRecords(admin, receiptIds([id]), options);
+    return records[0];
+}
+//# sourceMappingURL=index.js.map

package/dist/testing/index.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../../src/testing/index.ts"],"names":[],"mappings":";;;AA4LA,gCAYC;AAGD,gDAOC;AA/MD,8DAA+D;AAgC/D,MAAa,cAAe,SAAQ,KAAK;IAI5B;IAHF,OAAO,CAA+B;IAE/C,YACW,IAAwB,EACjC,OAAe,EACf,UAAwC,EAAE;QAE1C,KAAK,CAAC,OAAO,CAAC,CAAC;QAJN,SAAI,GAAJ,IAAI,CAAoB;QAKjC,IAAI,CAAC,IAAI,GAAG,gBAAgB,CAAC;QAC7B,IAAI,CAAC,OAAO,GAAG,OAAO,CAAC,GAAG,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC,EAAE,GAAG,MAAM,EAAE,CAAC,CAAC,CAAC;IAC1D,CAAC;CACF;AAZD,wCAYC;AAKD,SAAS,eAAe,CAAC,KAAa,EAAE,IAAY;IAClD,IAAI,CAAC,MAAM,CAAC,aAAa,CAAC,KAAK,CAAC,IAAI,KAAK,GAAG,CAAC,IAAI,KAAK,GAAG,UAAU;QACjE,MAAM,IAAI,cAAc,CACtB,iBAAiB,EACjB,GAAG,IAAI,qCAAqC,CAC7C,CAAC;IACJ,OAAO,KAAK,CAAC;AACf,CAAC;AAED,SAAS,UAAU,CAAC,GAAsB;IACxC,IACE,CAAC,KAAK,CAAC,OAAO,CAAC,GAAG,CAAC;QACnB,GAAG,CAAC,MAAM,GAAG,IAAI;QACjB,KAAK,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,IAAI,CAClB,CAAC,EAAE,EAAE,EAAE,CACL,OAAO,EAAE,KAAK,QAAQ;YACtB,CAAC,iEAAiE,CAAC,IAAI,CACrE,EAAE,CACH,CACJ;QAED,MAAM,IAAI,cAAc,CACtB,iBAAiB,EACjB,sCAAsC,CACvC,CAAC;IACJ,OAAO,CAAC,GAAG,IAAI,GAAG,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,EAAE,EAAE,EAAE,CAAC,EAAE,CAAC,WAAW,EAAE,CAAC,CAAC,CAAC,CAAC;AACzD,CAAC;AAED,SAAS,KAAK,CAAC,EAAU,EAAE,MAAmB;IAC5C,OAAO,IAAI,OAAO,CAAC,CAAC,OAAO,EAAE,MAAM,EAAE,EAAE;QACrC,MAAM,OAAO,GAAG,GAAG,EAAE;YACnB,YAAY,CAAC,KAAK,CAAC,CAAC;YACpB,MAAM,CAAC,mBAAmB,CAAC,OAAO,EAAE,OAAO,CAAC,CAAC;YAC7C,MAAM,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC;QACxB,CAAC,CAAC;QACF,MAAM,KAAK,GAAG,UAAU,CAAC,GAAG,EAAE;YAC5B,MAAM,CAAC,mBAAmB,CAAC,OAAO,EAAE,OAAO,CAAC,CAAC;YAC7C,OAAO,EAAE,CAAC;QACZ,CAAC,EAAE,EAAE,CAAC,CAAC;QACP,MAAM,CAAC,gBAAgB,CAAC,OAAO,EAAE,OAAO,EAAE,EAAE,IAAI,EAAE,IAAI,EAAE,CAAC,CAAC;QAC1D,IAAI,MAAM,CAAC,OAAO;YAAE,OAAO,EAAE,CAAC;IAChC,CAAC,CAAC,CAAC;AACL,CAAC;AAED,KAAK,UAAU,YAAY,CACzB,MAAc,EACd,GAAsB,EACtB,OAAyB,EACzB,MAAe;IAEf,MAAM,SAAS,GAAG,eAAe,CAAC,OAAO,CAAC,SAAS,IAAI,IAAI,EAAE,WAAW,CAAC,CAAC;IAC1E,MAAM,cAAc,GAAG,eAAe,CACpC,OAAO,CAAC,cAAc,IAAI,EAAE,EAC5B,gBAAgB,CACjB,CAAC;IACF,MAAM,SAAS,GAAG,eAAe,CAAC,OAAO,CAAC,SAAS,IAAI,IAAI,EAAE,WAAW,CAAC,CAAC;IAC1E,MAAM,MAAM,GAAG,OAAO,CAAC,MAAM,IAAI,WAAW,CAAC;IAC7C,IAAI,MAAM,KAAK,WAAW,IAAI,MAAM,KAAK,MAAM;QAC7C,MAAM,IAAI,cAAc,CACtB,iBAAiB,EACjB,mCAAmC,CACpC,CAAC;IACJ,MAAM,SAAS,GAAwB,GAAG,CAAC,GAAG,CAAC,CAAC,EAAE,EAAE,EAAE,CAAC,CAAC;QACtD,EAAE;QACF,MAAM,EAAE,IAAI;QACZ,YAAY,EAAE,IAAI;QAClB,iBAAiB,EAAE,IAAI;KACxB,CAAC,CAAC,CAAC;IACJ,MAAM,UAAU,GAAG,IAAI,eAAe,EAAE,CAAC;IACzC,MAAM,IAAI,GAAG,CAAC,IAAwB,EAAE,OAAe,EAAE,EAAE,CACzD,IAAI,cAAc,CAAC,IAAI,EAAE,OAAO,EAAE,SAAS,CAAC,CAAC;IAC/C,MAAM,OAAO,GAAG,GAAG,EAAE,CACnB,UAAU,CAAC,KAAK,CAAC,IAAI,CAAC,SAAS,EAAE,8BAA8B,CAAC,CAAC,CAAC;IACpE,OAAO,CAAC,MAAM,EAAE,gBAAgB,CAAC,OAAO,EAAE,OAAO,EAAE,EAAE,IAAI,EAAE,IAAI,EAAE,CAAC,CAAC;IACnE,IAAI,OAAO,CAAC,MAAM,EAAE,OAAO;QAAE,OAAO,EAAE,CAAC;IACvC,MAAM,KAAK,GAAG,UAAU,CACtB,GAAG,EAAE,CACH,UAAU,CAAC,KAAK,CACd,IAAI,CACF,SAAS,EACT,4BAA4B,MAAM,WAAW,SAAS,KAAK,CAC5D,CACF,EACH,SAAS,CACV,CAAC;IACF,MAAM,GAAG,GAAG,CAAI,EAAoB,EAAE,EAAE,CAAC,IAAA,iBAAS,EAAC,UAAU,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC;IAC1E,IAAI,CAAC;QACH,OAAO,MAAM,GAAG,CAAC,KAAK,IAAI,EAAE;YAC1B,IAAI,GAAG,CAAC,MAAM,KAAK,CAAC;gBAAE,OAAO,EAAE,CAAC;YAChC,KAAK,IAAI,IAAI,GAAG,CAAC,EAAE,IAAI,GAAG,SAAS,EAAE,IAAI,EAAE,EAAE,CAAC;gBAC5C,MAAM,OAAO,GAAkB,EAAE,CAAC;gBAClC,KAAK,IAAI,KAAK,GAAG,CAAC,EAAE,KAAK,GAAG,GAAG,CAAC,MAAM,EAAE,KAAK,EAAE,EAAE,CAAC;oBAChD,MAAM,MAAM,GAAG,MAAM,GAAG,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,GAAG,CAAC,GAAG,CAAC,KAAK,CAAE,CAAC,CAAC,CAAC;oBACxD,SAAS,CAAC,KAAK,CAAC,GAAG;wBACjB,EAAE,EAAE,GAAG,CAAC,KAAK,CAAE;wBACf,MAAM,EAAE,MAAM,EAAE,MAAM,IAAI,IAAI;wBAC9B,YAAY,EAAE,MAAM,EAAE,YAAY,IAAI,IAAI;wBAC1C,iBAAiB,EAAE,MAAM,EAAE,iBAAiB,IAAI,IAAI;qBACrD,CAAC;oBACF,IAAI,MAAM;wBAAE,OAAO,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;oBACjC,IACE,MAAM;wBACN,MAAM,CAAC,MAAM,KAAK,MAAM;wBACxB,CAAC,MAAM,CAAC,MAAM,KAAK,MAAM,IAAI,MAAM,CAAC,MAAM,KAAK,WAAW,CAAC;wBAE3D,MAAM,IAAI,CACR,2BAA2B,EAC3B,iBAAiB,MAAM,CAAC,EAAE,YAAY,MAAM,CAAC,MAAM,cAAc,MAAM,GAAG,CAC3E,CAAC;gBACN,CAAC;gBACD,IACE,OAAO,CAAC,MAAM,KAAK,GAAG,CAAC,MAAM;oBAC7B,OAAO,CAAC,KAAK,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,MAAM,CAAC,MAAM,KAAK,MAAM,CAAC;oBAEnD,OAAO,OAAO,CAAC;gBACjB,IAAI,IAAI,GAAG,CAAC,KAAK,SAAS;oBAAE,MAAM;gBAClC,IAAI,MAAM;oBAAE,MAAM,GAAG,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC;gBAC9C,gFAAgF;gBAChF,MAAM,KAAK,CAAC,cAAc,EAAE,UAAU,CAAC,MAAM,CAAC,CAAC;YACjD,CAAC;YACD,MAAM,IAAI,CACR,YAAY,EACZ,4BAA4B,MAAM,UAAU,SAAS,gBAAgB,CACtE,CAAC;QACJ,CAAC,CAAC,CAAC;IACL,CAAC;YAAS,CAAC;QACT,YAAY,CAAC,KAAK,CAAC,CAAC;QACpB,OAAO,CAAC,MAAM,EAAE,mBAAmB,CAAC,OAAO,EAAE,OAAO,CAAC,CAAC;IACxD,CAAC;AACH,CAAC;AAED;;;;GAIG;AACI,KAAK,UAAU,UAAU,CAC9B,KAAa,EACb,KAAa,EACb,GAAsB,EACtB,UAA4B,EAAE;IAE9B,OAAO,YAAY,CACjB,KAAK,EACL,UAAU,CAAC,GAAG,CAAC,EACf,EAAE,GAAG,OAAO,EAAE,MAAM,EAAE,WAAW,EAAE,EACnC,KAAK,CACN,CAAC;AACJ,CAAC;AAED,sFAAsF;AAC/E,KAAK,UAAU,kBAAkB,CACtC,KAAa,EACb,EAAU,EACV,UAA4B,EAAE;IAE9B,MAAM,OAAO,GAAG,MAAM,YAAY,CAAC,KAAK,EAAE,UAAU,CAAC,CAAC,EAAE,CAAC,CAAC,EAAE,OAAO,CAAC,CAAC;IACrE,OAAO,OAAO,CAAC,CAAC,CAAE,CAAC;AACrB,CAAC"}

package/docs/benchmarks/2026-10-28-sustained-local.md

@@ -0,0 +1,104 @@
+# Sustained ingress, recovery, and retention on a disk-backed database
+
+This report records two local runs of the `0.2.0` candidate. Each run uses a fresh PostgreSQL container and a dedicated Docker volume, with simultaneous receipt acceptance and worker processing. The runs exercise a bounded overload and recovery window; they are not a production capacity estimate or an hours-long soak test.
+
+## Workload and environment
+
+| Setting           | Value                                                                                          |
+| ----------------- | ---------------------------------------------------------------------------------------------- |
+| Node / Prisma     | 24.22.1 / 7.10.1                                                                               |
+| PostgreSQL        | 16.15, official `postgres:16-alpine` image                                                     |
+| Host              | macOS arm64, Apple M1 Pro, 10 logical CPUs                                                     |
+| Docker            | 29.9.2; database limited to 2 CPUs and 1 GiB                                                   |
+| Database storage  | Fresh named volume on the Docker VM disk; no PostgreSQL tmpfs                                  |
+| Durability        | `fsync`, `synchronous_commit`, and `full_page_writes` enabled                                  |
+| Existing history  | 5,000 completed receipts with retained 1 KiB payload bodies                                    |
+| Ingress           | 120 seconds: 30 s at 80/s, 60 s at 240/s, 30 s at 80/s                                         |
+| Concurrency       | At most 16 acceptance requests, 4 handler tasks                                                |
+| Worker polling    | Continuous awaited `runOnce()` calls, with a 5 ms pause after an empty poll                    |
+| Connections       | Shared application pool of 8; separate measurement pool of 2                                   |
+| Pool pressure     | Hold 4 application connections from about second 48 to 78                                      |
+| Tenants           | One hot tenant receives 95%; one cold tenant receives 5%                                       |
+| Handler           | 15 ms SQL sleep followed by one business insert in the inbox transaction                       |
+| Retry             | Every 50th message within each tenant fails after its first insert, forcing rollback and retry |
+| Recovery deadline | 180 seconds after ingress stops                                                                |
+| Retention         | Purge completed payloads in batches of 500, then ordinary `VACUUM (ANALYZE)`                   |
+
+The effect table has an independent sequence primary key. Inbox IDs and message IDs have no unique constraint in that table, so duplicate committed effects cannot be hidden by a business uniqueness check. Verification checks every accepted receipt for exactly one committed effect and a completed status, as well as the expected retries and zero unexpected errors.
+
+Receipt payloads use 64 random 1 KiB bodies. The acceptance scheduler is bounded: when all 16 request slots are occupied, it waits rather than generating an unbounded client queue. The JSON reports distinguish target submissions, actual attempts, accepted receipts, and unsent target submissions. Sampled pool queue depth and measured connection acquisition time describe this shared pool; they do not measure a remote production pool.
+
+## Results
+
+Both runs passed. Across **38,398 accepted receipts**, the database contained exactly **38,398 completed receipts and business effects**, with zero missing or duplicate committed effects. The **770 intentional first-attempt failures** rolled back and retried successfully. There were no unexpected acceptance, worker, lease-loss, or DEAD outcomes.
+
+| Measurement                                       |                      Run 1 |                      Run 2 |
+| ------------------------------------------------- | -------------------------: | -------------------------: |
+| Accepted / completed / business effects           |   19,199 / 19,199 / 19,199 |   19,199 / 19,199 / 19,199 |
+| Intentional rollbacks and retries                 |                        385 |                        385 |
+| Receive requests/s during the 120 s ingress phase |                    159.992 |                    159.991 |
+| Acceptance p50 / p95 / p99                        | 3.480 / 20.615 / 25.915 ms | 3.573 / 21.736 / 27.094 ms |
+| Shared-pool acquisition p95 / p99                 |         11.084 / 20.255 ms |         11.652 / 21.132 ms |
+| One-second samples with queued pool acquisition   |                         25 |                         22 |
+| Highest sampled pool queue depth                  |                          6 |                          6 |
+| Highest sampled backlog                           |                      7,613 |                      7,750 |
+| Backlog when ingress stopped                      |                      7,573 |                      7,676 |
+| Recovery after ingress stopped                    |                   94.675 s |                   94.191 s |
+| Total ingress plus recovery                       |                  214.675 s |                  214.191 s |
+| Cold tenant completed / completion p95            |             960 / 95.571 s |             960 / 95.478 s |
+| Hot tenant completed / completion p95             |          18,239 / 95.574 s |          18,239 / 95.483 s |
+| Missing / duplicate committed effects             |                      0 / 0 |                      0 / 0 |
+
+Both offered schedules had 19,200 boundary-inclusive target slots. The strict 120-second cutoff submitted 19,199 of them; the unsubmitted final slot is recorded in each JSON rather than counted as an accepted message. The approximate 160/s receipt rate is the configured mixed offered load, not a maximum-throughput finding. The 95-second completion p95 includes accumulated queue delay under the deliberate overload; it is distinct from the acceptance latency.
+
+The original reports preserve every one-second sample, exact counters, environment details, and latency distributions:
+
+- [Run 1](2026-10-28-sustained-run1.json): completed at `2026-10-28T13:21:12.858Z` / 22:21:12.858 KST.
+- [Run 2](2026-10-28-sustained-run2.json): completed at `2026-10-28T13:25:04.040Z` / 22:25:04.040 KST.
+
+Both used image ID `sha256:f4610795587492cd4fb887c2dc3de301b813278bb6f616992061141b5f15ac36`. Each runner exited successfully, and a final inspection found no remaining benchmark-labeled containers or volumes. Earlier 10-second harness development checks are not included in these measurements.
+
+## Retention guidance
+
+Payload cleanup preserves the receipt identity and fingerprint. The benchmark re-delivers 40 original envelopes after cleanup and checks that they remain duplicates with the original completed receipt and no new business effect. It also compares a checksum of every stored receipt ID and fingerprint before and after cleanup.
+
+| Storage measurement                                   |             Run 1 |             Run 2 |
+| ----------------------------------------------------- | ----------------: | ----------------: |
+| Seeded history: 5,000 receipts, total relation size   |          7.98 MiB |          7.98 MiB |
+| Completed receipts before cleanup                     |            24,199 |            24,199 |
+| Logical payload storage before → after cleanup        |     24.55 → 0 MiB |     24.55 → 0 MiB |
+| Heap before cleanup                                   |         44.68 MiB |         44.98 MiB |
+| Indexes before → after cleanup                        |  9.75 → 10.31 MiB |  9.70 → 10.39 MiB |
+| Total relation size before → after cleanup and vacuum | 54.48 → 55.04 MiB | 54.73 → 55.41 MiB |
+| Cleanup time, 500-row batches                         |        723.099 ms |        715.353 ms |
+| Receipt/fingerprint checksum preserved                |               Yes |               Yes |
+| Duplicate re-delivery probes after cleanup            |           40 / 40 |           40 / 40 |
+
+The growth includes receipt insertion, claim/completion updates, intentional retries, and index maintenance. It is not a stable bytes-per-message coefficient for other applications. Cleanup removed all logical payload bytes but left physical allocation available for reuse; the small index increase during cleanup is included rather than hidden.
+
+Use bounded purge batches and measure table, index, and WAL growth in the application workload. Deleting a payload does not remove the deduplication identity or its indexes. Ordinary vacuum makes dead tuple space reusable and does not promise that allocated relation bytes shrink immediately. Retention capacity therefore needs to account for continuing receipt and index growth even when payloads are purged.
+
+The hot/cold tenant measurements expose queue delay under shared scheduling. They do not establish a fairness bound. A tenant with low traffic can still wait behind accumulated work; measure the application's tail latency before choosing any additional scheduling policy.
+
+## Reproduce
+
+With Docker running and repository dependencies installed:
+
+```sh
+npm run benchmark:sustained --silent > /tmp/inbox-sustained-run1.json
+npm run benchmark:sustained --silent > /tmp/inbox-sustained-run2.json
+```
+
+The runner builds the current checkout and generates the test Prisma client. Setup and progress go to stderr; stdout contains only the JSON report. It ignores an existing application `DATABASE_URL`, creates a randomly named database, binds its port only on loopback, and removes its own container and volume in cleanup. The database is capped at 2 CPUs and 1 GiB; commands and the benchmark child have bounded execution times.
+
+For a short harness check, use `INBOX_BENCHMARK_SECONDS=10`. Allowed ingress durations are 10–300 seconds; `INBOX_BENCHMARK_RECOVERY_SECONDS` is also bounded to 10–300 seconds. `INBOX_BENCHMARK_RETAINED` controls the seeded history, from 0 to 50,000 receipts. A run exits unsuccessfully if correctness checks fail or the intended pool-queue/backlog conditions were not observed.
+
+## Interpretation limits
+
+- Two runs on one local Docker VM cover a several-minute overload and recovery window. They do not replace application-specific soak tests or a production capacity plan.
+- The volume is disk-backed, but host/VM caches and virtualized flush behavior remain. This is not a cold-cache or physical disk I/O benchmark.
+- Intentional SQL delay and retries make the backlog observable. The recorded throughput is specific to that handler and is not the package's maximum throughput.
+- The ingress target is an offered schedule with bounded concurrency; actual acceptance throughput and missed targets are reported separately.
+- Tenant results describe this 95/5 mix and the current shared scheduling behavior. No fairness SLA is claimed.
+- This workload excludes broker traffic, RLS hooks, outbox writes, network faults, and failover. Separate operational scenarios verify those behaviors.
+- Other repository test containers were stopped before the measured runs. Unrelated host activity was not controlled.