@spine-event-engine/storage-postgres 2.0.0-snapshot.22 → 2.0.0-snapshot.24

package/README.md

@@ -8,6 +8,10 @@ with live acceptance recorded against PostgreSQL 16.15 and 18.6.
 This is an experimental snapshot package. Use Node 24 or newer. For the precise
 public API and operational rules, see the [coding-agent reference](REFERENCE.md).
 
+Agent repositories can use this factory to retain accepted work and conversation
+history across restarts. Their state and execution completion are saved together
+in PostgreSQL; a model call does not keep a database transaction open.
+
 ## Install
 
 ```sh
@@ -60,14 +64,15 @@ tenant or database targets are rejected during construction.
 ## Lifecycle and operations
 
 Tables are initialized lazily and checked against the expected PostgreSQL
-layout. Each acquired client is released and each transaction is completed or
-rolled back. Call `factory.close()` when the application stops; it is
+layout. Each acquired client is released; history maintenance applies deletion
+pages independently. Call `factory.close()` when the application stops; it is
 idempotent and drains live handles and pools. A closed factory cannot create new
 handles.
 
 The provider supports record CRUD, compare-and-set, immutable append, Entity
-state and event history, delivery cleanup, declared-column filtering, sorting,
-limits, offsets, and continuations through the common storage API. It uses
+state and event history, declared-column filtering, sorting, limits, offsets,
+and continuations through the common storage API. Eligible Inbox cleanup deletes
+the message ID directly through record storage. It uses
 PostgreSQL `BYTEA`, parameterized `$1`-style values, and PostgreSQL's configured
 text collation. Collation ordering can differ from JavaScript ordering, so
 applications should choose and document their database collation deliberately.
@@ -83,6 +88,10 @@ collisions are rejected before the database is accessed.
 ## Live integration checks
 
 The live suite is opt-in and requires an already running PostgreSQL service.
+Use disposable test databases and a PostgreSQL superuser for this suite. The Agent
+history regressions alter index catalog flags in isolated temporary schemas to
+check rejection of invalid or unfinished indexes; these schemas are removed after
+each test. The production adapter does not require superuser access.
 No container or database is started automatically:
 
 ```sh

package/REFERENCE.md

@@ -1,7 +1,7 @@
 # @spine-event-engine/storage-postgres reference
 
 This reference describes the public PostgreSQL adapter contract for coding
-agents and maintainers.
+agents.
 
 ## Public entry point
 
@@ -42,7 +42,7 @@ altered. Missing, extra, or incompatible columns, defaults, nullability,
 primary keys, or unique constraints reject with `PostgresStorageSchemaError`.
 
 The adapter uses `$1` parameters, `IS NOT DISTINCT FROM`, PostgreSQL upsert,
-transaction advisory locks, and a bounded normalized query plan. It preserves
+independent bounded history maintenance pages, and a normalized query plan. It preserves
 the common 1,000-bind plan budget and 10,000-row default with one
 overflow lookahead row. The database collation controls text ordering; the
 provider does not invent a JavaScript-compatible collation.
@@ -60,13 +60,64 @@ plain, non-reserved ASCII identifiers only. Reserved and quoted-required names
 preserve their spelling; all names reject NUL, unsafe values, 63-byte overflow,
 and collisions before DDL, DML, or catalog inspection.
 
+## Entity commit behavior
+
+The provider captures and validates one Entity commit input before database I/O,
+then prepares its record families. It writes state history, diagnostic Events,
+canonical delivery Events, and finally the current state as ordered independent
+records. A later failure leaves the completed prefix in place; an Event row
+alone does not prove that the current state was written. Concurrent commits can
+interleave, including their current-state writes. This is not one SQL
+transaction across the four families.
+
+After preparation succeeds, a PostgreSQL deadlock (`40P01`) or serialization
+failure (`40001`) during the ordered record writes triggers one replay of the
+captured input. A failure during preparation is not retried. Commit does not
+rerun the handler or create new
+versions or Event IDs. An immutable retry accepts an existing row only when its
+wire bytes match; the same ID with different bytes is a collision. A caller
+retry must likewise reuse the original versions and Event IDs. See the
+[shared storage contract](../storage/REFERENCE.md) for the provider SPI and
+cross-provider limits.
+
+## Agent history
+
+The provider-only Agent history handle stores `AgentHistoryRecord` in one
+`agent_history` record family per selected tenant database. Its payload retains
+the original complete `AgentHistoryEntry` and full state type and Agent key.
+SHA-256 digests provide bounded physical record IDs and indexed scope keys;
+reads also compare the complete state type and Agent key. Three validated B-tree
+indexes cover scope plus full order, scope plus category plus full order, and
+scope plus conversation digest plus full order. Conversation reads also compare
+the complete conversation key. `order_key` encodes original seconds,
+nanoseconds, category, and unsigned UTF-8 record ID under PostgreSQL `C`
+collation. The handle rejects incompatible existing same-name indexes; native
+B-tree entry limits still apply to exceptionally long record IDs. Pages use
+bounded SQL chunks and sum serialized entry bytes; a fetched chunk can be
+larger than the returned byte budget.
+
+## Agent execution
+
+Agent execution records and per-instance pending entries are stored in separate
+tenant database tables. Indexed queries select the next eligible instance and
+the first unresolved signal within it. Required B-tree indexes use complete
+binary-comparable ordering keys and are checked before admission.
+
+Claims, history updates and conditional Entity completion use the same native
+transaction connection. Completion compares the original Entity Version before
+writing the resulting state, history, preferences and outgoing Commands. Reopening
+a factory can read the saved work. No database transaction remains open while a
+model request runs.
+
 ## Lifecycle and errors
 
 `factory.close()` is idempotent, closes registered handles, prevents new handle
 creation, and drains pools. Record and Entity handles can close independently.
-Acquired clients are released in `finally` blocks and transactions are always
-committed or rolled back before release. Errors do not include URLs,
-credentials, SQL text, or driver internals.
+Acquired clients are released in `finally` blocks. History maintenance applies
+each deletion page independently. Top-level provider error messages and public
+responses omit URLs, credentials, SQL text, and driver details. The original
+driver failure remains available as the internal `cause`; do not expose it in
+public responses or unfiltered logs.
 
 Custom table creation may use `useOperationFactory(factory)` with the exported
 `PostgresTableSpec`, `PostgresCreateOperation`, and
@@ -77,7 +128,10 @@ compiler internals.
 
 ## Live verification
 
-The infrastructure suite is intentionally excluded from ordinary CI. Supply
+The infrastructure suite is intentionally excluded from ordinary CI. Its Agent
+history regressions require a superuser to alter index catalog flags within
+disposable temporary schemas, which are removed after each test. This is a test
+requirement; the production adapter does not require superuser access. Supply
 explicit `SPINE_TS_POSTGRESQL_URL`, `SPINE_TS_POSTGRESQL_TENANT_A_URL`, and
 `SPINE_TS_POSTGRESQL_TENANT_B_URL` values naming three test databases, then run
 the package's `test:postgresql:16`

package/dist/postgres/agent-execution.d.ts

@@ -0,0 +1,285 @@
+import { type Message } from "@bufbuild/protobuf";
+import type { Timestamp } from "@bufbuild/protobuf/wkt";
+import type { CommandId } from "@spine-event-engine/proto";
+import { type AgentAcceptedInvocation, type AgentExecutionHead, type AgentExecutionRecord, type AgentInvocationKey } from "@spine-event-engine/proto/generated/spine/server/agent/execution_record_pb.js";
+import type { AgentHistoryRecord } from "@spine-event-engine/proto/generated/spine/server/agent/history_record_pb.js";
+import type { EntityRecord } from "@spine-event-engine/proto/generated/spine/server/entity/entity_pb.js";
+import { type AgentExecutionStorage, type AgentExecutionCapacity, type AgentExecutionStorageInput, type AgentExecutionClaim, type AgentExecutionUpdate, type AgentExecutionComplete, type AgentPendingRead, type AgentPendingPage } from "@spine-event-engine/storage/provider";
+import { PostgresRecordStorage } from "./record-storage.js";
+import { PostgresEntityCommitStorage } from "./entity-commit.js";
+/**
+ * Persists accepted Agent work, exclusive claims and deterministic evidence.
+ *
+ * @typeParam I Typed Agent identifier.
+ * @typeParam S Generated Agent state.
+ */
+export declare class PostgresAgentExecution<I, S extends Message> implements AgentExecutionStorage<I, S> {
+    #private;
+    private readonly input;
+    private readonly invocation;
+    private readonly head;
+    private readonly history;
+    private readonly current;
+    private readonly commits;
+    /**
+     * PostgreSQL has no tighter fixed row limit in this provider contract.
+     */
+    readonly capacity: AgentExecutionCapacity;
+    /**
+     * Binds prepared record families to one Agent repository and tenant boundary.
+     *
+     * @param input Agent repository and current Entity layout.
+     * @param invocation Mutable invocation records.
+     * @param head Per-instance claim and current preference records.
+     * @param history Immutable Agent conversation, System and domain entries.
+     * @param current Existing Entity current records used for no-op version checks.
+     * @param commits Opens the existing conditional Entity commit coordinator.
+     */
+    constructor(input: AgentExecutionStorageInput<I, S>, invocation: PostgresRecordStorage<string, AgentExecutionRecord>, head: PostgresRecordStorage<string, AgentExecutionHead>, history: PostgresRecordStorage<string, AgentHistoryRecord>, current: PostgresRecordStorage<I, EntityRecord>, commits: () => PostgresEntityCommitStorage<I, S>);
+    /**
+     * Persists original accepted work idempotently under its per-instance lock.
+     *
+     * @param accepted Complete source signal and selected bindings.
+     * @returns Detached stored invocation.
+     */
+    admit(accepted: AgentAcceptedInvocation): Promise<AgentExecutionRecord>;
+    /**
+     * Reads a bounded indexed page of eligible accepted or interrupted work.
+     *
+     * @param request Complete continuation and page count.
+     * @returns Detached records and continuation state.
+     */
+    pending(request: AgentPendingRead): Promise<AgentPendingPage>;
+    /**
+     * Builds one binary-collated native pending-head query.
+     *
+     * @param request Bounded pending-page request.
+     * @param asOf Fixed provider-time eligibility cutoff.
+     * @param table Resolved head table in the current schema.
+     * @returns Native SQL and exact bound values.
+     */
+    private pendingQuery;
+    /**
+     * Reads one complete persisted invocation.
+     *
+     * @param key Original source and Agent instance identity.
+     * @returns Detached record if present.
+     */
+    read(key: AgentInvocationKey): Promise<AgentExecutionRecord | undefined>;
+    /**
+     * Acquires only earliest unresolved same-instance work under a fresh transaction clock.
+     *
+     * @param key Original source and Agent instance identity.
+     * @param token New exclusive token.
+     * @param expiresAt Requested future expiry.
+     * @returns Claimed record and current preferences, when eligible.
+     */
+    claim(key: AgentInvocationKey, token: string, expiresAt: Timestamp): Promise<AgentExecutionClaim | undefined>;
+    /**
+     * Checks the locked instance and persists one exclusive lease.
+     *
+     * @param client Native client in the current transaction.
+     * @param rows Tenant record executors.
+     * @param key Original Agent invocation identity.
+     * @param token New exclusive token.
+     * @param expiresAt Requested future lease expiry.
+     * @returns Claimed record and preferences when eligible.
+     */
+    private claimLocked;
+    /**
+     * Checks the current head and earliest same-instance source.
+     *
+     * @param client Native client in the current transaction.
+     * @param rows Tenant record executors.
+     * @param head Per-instance pending and lease record.
+     * @param key Original Agent invocation identity.
+     * @param now Fresh time read under the per-instance lock.
+     * @returns Whether this invocation may receive the next lease.
+     */
+    private eligibleClaim;
+    /**
+     * Updates one still-current lease under the per-instance transaction lock.
+     *
+     * @param key Original source and Agent instance identity.
+     * @param token Current exclusive token.
+     * @param expiresAt Requested later expiry.
+     * @returns Whether renewal succeeded.
+     */
+    renew(key: AgentInvocationKey, token: string, expiresAt: Timestamp): Promise<boolean>;
+    /**
+     * Persists one execution update and its immutable history rows in the same transaction.
+     *
+     * @param input Exact read image, next execution record and history entries.
+     * @returns Completion after fenced evidence is persisted.
+     */
+    update(input: AgentExecutionUpdate): Promise<void>;
+    /**
+     * Commits Entity state, history, preferences and outgoing obligations together.
+     *
+     * @param input Complete conditional transition.
+     * @returns Completion after conditional Entity and evidence commit.
+     */
+    complete(input: AgentExecutionComplete<I, S>): Promise<void>;
+    /**
+     * Applies every conditional completion row on one transaction client.
+     *
+     * @param client Native database client in the current transaction.
+     * @param input Requested fenced execution change.
+     * @param prepared Prepared Entity state and Version write.
+     * @param rows Provider record handles bound to the tenant.
+     * @returns Completion after Entity and execution evidence commit.
+     */
+    private completeLocked;
+    /**
+     * Validates one Version increment or no-op and a nonterminal prior phase.
+     *
+     * @param input Requested fenced execution change.
+     * @param prior Previously persisted execution record.
+     * @returns Expected initial Entity Version.
+     */
+    private validateCompletion;
+    /**
+     * Marks original outgoing IDs accepted by normal delivery.
+     *
+     * @param key Original source and Agent instance identity.
+     * @param token Current exclusive token.
+     * @param expectedRecordBytes Exact previous record bytes.
+     * @param signals Original accepted output IDs.
+     * @returns Completion after original output IDs are marked delivered.
+     */
+    markDelivered(key: AgentInvocationKey, token: string, expectedRecordBytes: Uint8Array, signals: readonly CommandId[]): Promise<void>;
+    /**
+     * Marks original output IDs while the per-instance native lock is held.
+     *
+     * @param client Native client in the current transaction.
+     * @param rows Tenant record executors.
+     * @param key Original Agent invocation identity.
+     * @param token Current exclusive claim token.
+     * @param expectedRecordBytes Exact prior execution record bytes.
+     * @param signals Original accepted output IDs.
+     * @returns Completion after persisted output status is updated.
+     */
+    private deliverLocked;
+    /**
+     * Closes all provider record handles without deleting persisted work.
+     */
+    close(): void;
+    /**
+     * Rejects calls after handle closure.
+     */
+    private requireOpen;
+    /**
+     * Prepares three existing record families and required native indexes once.
+     *
+     * @returns Completion after provider capabilities are checked.
+     */
+    private ready;
+    /**
+     * Ensures full scoped order and expiry indexes before accepted work.
+     *
+     * @returns Completion after native tables and indexes are prepared.
+     */
+    private initialize;
+    /**
+     * Acquires a transaction-scoped lock even when the head row is absent.
+     *
+     * @param client Native database client in the current transaction.
+     * @param key Original Agent invocation identity.
+     * @param rows Provider record handles bound to the tenant.
+     * @returns Completion after locking the per-instance head.
+     */
+    private lockHead;
+    /**
+     * Reads one head and rejects a physical digest collision.
+     *
+     * @param client Native database client in the current transaction.
+     * @param key Original Agent invocation identity.
+     * @param rows Provider record handles bound to the tenant.
+     * @returns Per-instance head when present.
+     */
+    private currentHead;
+    /**
+     * Reads one invocation and rejects a physical digest collision.
+     *
+     * @param client Native database client in the current transaction.
+     * @param key Original Agent invocation identity.
+     * @param rows Provider record handles bound to the tenant.
+     * @returns Stored invocation when present.
+     */
+    private existing;
+    /**
+     * Checks fresh Time, exact token and complete previous record bytes.
+     *
+     * @param checkNext Whether to validate the proposed next record.
+     * @param client Native database client in the current transaction.
+     * @param input Requested fenced execution change.
+     * @param rows Provider record handles bound to the tenant.
+     * @returns Current fenced record and head.
+     */
+    private expected;
+    /**
+     * Reads claimed rows under the already acquired per-instance lock.
+     *
+     * @param client Native database client in the current transaction.
+     * @param key Original Agent invocation identity.
+     * @param rows Provider record handles bound to the tenant.
+     * @param token Exclusive claim token.
+     * @returns Current record and head if the token remains valid.
+     */
+    private claimed;
+    /**
+     * Checks no earlier unresolved signal in the same full Agent scope.
+     *
+     * @param client Native database client in the current transaction.
+     * @param key Original Agent invocation identity.
+     * @param rows Provider record handles bound to the tenant.
+     * @returns Whether this invocation is earliest unresolved work.
+     */
+    private earliest;
+    /**
+     * Adds an earlier accepted source without resetting an existing eligibility.
+     *
+     * @param head Per-instance pending and lease record.
+     * @param record Persisted Agent execution record.
+     */
+    private offerCandidate;
+    /**
+     * Updates the head to one bounded native successor before terminal publication.
+     *
+     * @param client Native database client in the current transaction.
+     * @param head Per-instance pending and lease record.
+     * @param key Original Agent invocation identity.
+     * @param order Original Inbox order.
+     * @param rows Provider record handles bound to the tenant.
+     * @returns Completion after updating the next pending head.
+     */
+    private advanceCandidate;
+    /**
+     * Reads at most two rows per unresolved status in original Inbox order.
+     *
+     * @param client Native database client in the current transaction.
+     * @param exclude Invocation omitted while selecting the successor.
+     * @param key Original Agent invocation identity.
+     * @param rows Provider record handles bound to the tenant.
+     * @returns Earliest unresolved invocation when present.
+     */
+    private successor;
+    /**
+     * Builds the bounded native query for one unresolved invocation status.
+     *
+     * @param table Resolved invocation table in the current schema.
+     * @returns Complete indexed successor query.
+     */
+    private successorSql;
+    /**
+     * Validates current Version and applies optional Entity rows in this transaction.
+     *
+     * @param client Native database client in the current transaction.
+     * @param expectedVersion Expected current Entity Version.
+     * @param input Requested fenced execution change.
+     * @param prepared Prepared Entity state and Version write.
+     * @returns Completion after conditional Entity state and Version write.
+     */
+    private completeEntity;
+}
+//# sourceMappingURL=agent-execution.d.ts.map

package/dist/postgres/agent-execution.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"agent-execution.d.ts","sourceRoot":"","sources":["../../src/postgres/agent-execution.ts"],"names":[],"mappings":"AAcA,OAAO,EAAoB,KAAK,OAAO,EAAE,MAAM,oBAAoB,CAAC;AACpE,OAAO,KAAK,EAAE,SAAS,EAAE,MAAM,wBAAwB,CAAC;AAExD,OAAO,KAAK,EAAE,SAAS,EAAE,MAAM,2BAA2B,CAAC;AAE3D,OAAO,EAIL,KAAK,uBAAuB,EAC5B,KAAK,kBAAkB,EACvB,KAAK,oBAAoB,EACzB,KAAK,kBAAkB,EACxB,MAAM,+EAA+E,CAAC;AACvF,OAAO,KAAK,EAAE,kBAAkB,EAAE,MAAM,6EAA6E,CAAC;AACtH,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,sEAAsE,CAAC;AACzG,OAAO,EAKL,KAAK,qBAAqB,EAC1B,KAAK,sBAAsB,EAC3B,KAAK,0BAA0B,EAC/B,KAAK,mBAAmB,EACxB,KAAK,oBAAoB,EACzB,KAAK,sBAAsB,EAC3B,KAAK,gBAAgB,EACrB,KAAK,gBAAgB,EACtB,MAAM,sCAAsC,CAAC;AAI9C,OAAO,EAAE,qBAAqB,EAA+B,MAAM,qBAAqB,CAAC;AACzF,OAAO,EAAE,2BAA2B,EAAE,MAAM,oBAAoB,CAAC;AAWjE;;;;;GAKG;AACH,qBAAa,sBAAsB,CAAC,CAAC,EAAE,CAAC,SAAS,OAAO,CAAE,YAAW,qBAAqB,CAAC,CAAC,EAAE,CAAC,CAAC;;IAqB5F,OAAO,CAAC,QAAQ,CAAC,KAAK;IACtB,OAAO,CAAC,QAAQ,CAAC,UAAU;IAC3B,OAAO,CAAC,QAAQ,CAAC,IAAI;IACrB,OAAO,CAAC,QAAQ,CAAC,OAAO;IACxB,OAAO,CAAC,QAAQ,CAAC,OAAO;IACxB,OAAO,CAAC,QAAQ,CAAC,OAAO;IArB1B;;OAEG;IACH,QAAQ,CAAC,QAAQ,EAAE,sBAAsB,CAAqB;IAE9D;;;;;;;;;OASG;gBAEgB,KAAK,EAAE,0BAA0B,CAAC,CAAC,EAAE,CAAC,CAAC,EACvC,UAAU,EAAE,qBAAqB,CAAC,MAAM,EAAE,oBAAoB,CAAC,EAC/D,IAAI,EAAE,qBAAqB,CAAC,MAAM,EAAE,kBAAkB,CAAC,EACvD,OAAO,EAAE,qBAAqB,CAAC,MAAM,EAAE,kBAAkB,CAAC,EAC1D,OAAO,EAAE,qBAAqB,CAAC,CAAC,EAAE,YAAY,CAAC,EAC/C,OAAO,EAAE,MAAM,2BAA2B,CAAC,CAAC,EAAE,CAAC,CAAC;IAGnE;;;;;OAKG;IACG,KAAK,CAAC,QAAQ,EAAE,uBAAuB,GAAG,OAAO,CAAC,oBAAoB,CAAC;IA0B7E;;;;;OAKG;IACG,OAAO,CAAC,OAAO,EAAE,gBAAgB,GAAG,OAAO,CAAC,gBAAgB,CAAC;IA4BnE;;;;;;;OAOG;IACH,OAAO,CAAC,YAAY;IAoBpB;;;;;OAKG;IACG,IAAI,CAAC,GAAG,EAAE,kBAAkB,GAAG,OAAO,CAAC,oBAAoB,GAAG,SAAS,CAAC;IAU9E;;;;;;;OAOG;IACG,KAAK,CACT,GAAG,EAAE,kBAAkB,EACvB,KAAK,EAAE,MAAM,EACb,SAAS,EAAE,SAAS,GACnB,OAAO,CAAC,mBAAmB,GAAG,SAAS,CAAC;IAU3C;;;;;;;;;OASG;YACW,WAAW;IA+BzB;;;;;;;;;OASG;YACW,aAAa;IAoB3B;;;;;;;OAOG;IACG,KAAK,CAAC,GAAG,EAAE,kBAAkB,EAAE,KAAK,EAAE,MAAM,EAAE,SAAS,EAAE,SAAS,GAAG,OAAO,CAAC,OAAO,CAAC;IAoB3F;;;;;OAKG;IACG,MAAM,CAAC,KAAK,EAAE,oBAAoB,GAAG,OAAO,CAAC,IAAI,CAAC;IAyBxD;;;;;OAKG;IACG,QAAQ,CAAC,KAAK,EAAE,sBAAsB,CAAC,CAAC,EAAE,CAAC,CAAC,GAAG,OAAO,CAAC,IAAI,CAAC;IAkBlE;;;;;;;;OAQG;YACW,cAAc;IA6B5B;;;;;;OAMG;IACH,OAAO,CAAC,kBAAkB;IA+B1B;;;;;;;;OAQG;IACG,aAAa,CACjB,GAAG,EAAE,kBAAkB,EACvB,KAAK,EAAE,MAAM,EACb,mBAAmB,EAAE,UAAU,EAC/B,OAAO,EAAE,SAAS,SAAS,EAAE,GAC5B,OAAO,CAAC,IAAI,CAAC;IAShB;;;;;;;;;;OAUG;YACW,aAAa;IAiC3B;;OAEG;IACH,KAAK,IAAI,IAAI;IASb;;OAEG;IACH,OAAO,CAAC,WAAW;IAInB;;;;OAIG;IACH,OAAO,CAAC,KAAK;IASb;;;;OAIG;YACW,UAAU;IAsBxB;;;;;;;OAOG;YACW,QAAQ;IAatB;;;;;;;OAOG;YACW,WAAW;IAsBzB;;;;;;;OAOG;YACW,QAAQ;IAkBtB;;;;;;;;OAQG;YACW,QAAQ;IAoBtB;;;;;;;;OAQG;YACW,OAAO;IAoBrB;;;;;;;OAOG;YACW,QAAQ;IAatB;;;;;OAKG;IACH,OAAO,CAAC,cAAc;IAItB;;;;;;;;;OASG;YACW,gBAAgB;IAW9B;;;;;;;;OAQG;YACW,SAAS;IAmCvB;;;;;OAKG;IACH,OAAO,CAAC,YAAY;IAQpB;;;;;;;;OAQG;YACW,cAAc;CAkB7B"}