@dudousxd/nestjs-agent-store-drizzle 0.23.0 → 0.25.0

package/README.md

@@ -16,15 +16,33 @@ pnpm add @dudousxd/nestjs-agent-store-drizzle drizzle-orm
 ## Use
 
 The host app owns the connection and passes in an already-opened Drizzle handle — this module never
-opens one itself. Any SQLite-dialect driver works (better-sqlite3, libsql, D1, …).
+opens one itself. **SQLite, Postgres and MySQL** all work — CI runs every store suite on
+better-sqlite3, node-postgres (Postgres 16) and mysql2 (MySQL 8.4), and the other Drizzle drivers of
+each dialect speak the same query builder. Build the handle with the schema object for your dialect;
+every store reads the dialect off the handle.
 
 ```ts
+// SQLite
 import Database from 'better-sqlite3';
 import { drizzle } from 'drizzle-orm/better-sqlite3';
-import { DrizzleAgentStoreModule, agentSchema, ensureAgentSchema } from '@dudousxd/nestjs-agent-store-drizzle';
-
+import { agentSchema } from '@dudousxd/nestjs-agent-store-drizzle';
 const db = drizzle(new Database('app.db'), { schema: agentSchema });
-await ensureAgentSchema(db); // idempotent CREATE TABLE IF NOT EXISTS — or run your own migrations
+
+// Postgres
+import { drizzle } from 'drizzle-orm/node-postgres';
+import { pgAgentSchema } from '@dudousxd/nestjs-agent-store-drizzle';
+const db = drizzle(pool, { schema: pgAgentSchema });
+
+// MySQL (8.0+)
+import { drizzle } from 'drizzle-orm/mysql2';
+import { mysqlAgentSchema } from '@dudousxd/nestjs-agent-store-drizzle';
+const db = drizzle(pool, { schema: mysqlAgentSchema, mode: 'default' });
+```
+
+```ts
+import { DrizzleAgentStoreModule, ensureAgentSchema } from '@dudousxd/nestjs-agent-store-drizzle';
+
+await ensureAgentSchema(db); // idempotent: creates what is missing — or run your own migrations
 
 @Module({
   imports: [
@@ -35,10 +53,31 @@ await ensureAgentSchema(db); // idempotent CREATE TABLE IF NOT EXISTS — or run
 export class AppModule {}
 ```
 
-The package ships the `agentSchema` (Drizzle tables), `ensureAgentSchema` (a non-destructive
-DDL helper for a quick start), `DrizzleAgentStore`, `DrizzleGovernanceQueries`, and
-`DrizzleRagIngestionLog`. For production,
-prefer your normal drizzle-kit migrations over the `ensureAgentSchema` helper.
+The package ships the tables for each dialect (`agentSchema` · `pgAgentSchema` · `mysqlAgentSchema`
+— same table, column and property names), `ensureAgentSchema` (a non-destructive DDL helper),
+`DrizzleAgentStore`, `DrizzleGovernanceQueries`, and `DrizzleRagIngestionLog`. For production, prefer
+your normal drizzle-kit migrations over the `ensureAgentSchema` helper: point drizzle-kit at the
+schema object for your dialect.
+
+### What each dialect stores
+
+| | SQLite | Postgres | MySQL |
+|---|---|---|---|
+| JSON columns | `text` (JSON) | `jsonb` | `json` |
+| timestamps | epoch-ms `integer` | `timestamptz(3)` | `datetime(3)`, UTC |
+| long text (content, reasoning, errors) | `text` | `text` | `longtext` |
+| keys and labels | `text` | `text` | `varchar(255)` |
+| prices / costs | `real` | `double precision` | `double` |
+| id / actor comparison | exact | exact | exact (`utf8mb4_bin`) |
+
+`ensureAgentSchema` renders its DDL from these schema objects, creates missing tables, columns and
+indexes, and never drops or alters anything. On Postgres and MySQL it holds a cross-replica lock
+(`pg_advisory_xact_lock` / `get_lock`) for the duration, so replicas booting an empty database at
+once do not race their `CREATE TABLE`s.
+
+Messages carry a per-thread `seq`, assigned on append, which is what orders a transcript — two
+messages of one turn routinely share a timestamp. A row written before the column existed has none
+and sorts first, by `created_at`, as before.
 
 ## The read a turn makes
 
@@ -56,7 +95,7 @@ a client rendering a transcript.
 ## Memory
 
 `DrizzleMemoryProvider` is a `MemoryProvider` over `agent_memory`, a table `ensureAgentSchema`
-creates alongside the other six.
+creates alongside the other agent tables.
 
 ```ts
 import { DrizzleAgentStoreModule, DrizzleMemoryProvider } from '@dudousxd/nestjs-agent-store-drizzle';
@@ -198,3 +237,100 @@ for the full model.
 ## License
 
 MIT © Davide Carvalho
+
+
+Action preflight confirmations persist in the nullable `agent_tool_call.confirmation` JSON column.
+The schema helper adds this column to existing databases. Hosts managing their own migrations must
+add it before upgrading (`JSONB` on PostgreSQL, `JSON` on MySQL, JSON text on SQLite). The stored
+confirmation is returned in `StoredMessage.approvals` so a reload uses the same wording.
+
+### Durable action proposals
+
+`DrizzleAgentStore` also implements the optional core `ActionProposalStore` capability.
+Its `createActionProposal`, scoped get/list, immutable decision, lease claim/renewal,
+and settlement methods persist to the independent `agent_action_proposal` table;
+no persisted thread, message, or run is required. The standalone
+`DrizzleActionProposalStore` exposes the same methods when only this capability is needed.
+Both accept an optional second constructor argument `{ clock: () => number }` for a
+trusted server clock (default `Date.now`); never pass a request timestamp as that clock.
+
+Creation is replay-safe against the original canonical JSON snapshot. A proposal ID
+is globally unique, and every read or mutation checks tenant, actor, and thread scope.
+Approval and queued execution commit together in one version-fenced SQL update.
+Claims/recovery increment a generation and issue a new token; renewal and settlement
+require both and an unexpired lease. Expiry is inclusive (`now >= expiresAt`).
+A contended caller transaction can return `conflict` after bounded CAS retries;
+retry from a fresh transaction when its repeatable-read snapshot is stale.
+The stable `idempotencyKey` survives recovery: delivery is at least once, so the tool
+must enforce idempotency for its external effect. This storage capability does not
+start a worker or change the agent loop.
+
+For existing databases, run `ensureAgentSchema(db)` to add the new table and its
+scope/creation-order index without altering existing data. Lists filter and limit in
+SQL (default 100, maximum 1000), with UTF16 lexical ID ordering for equal creation
+times. Logical proposal IDs are limited to 255 UTF16 code units by the core contract.
+If you own migrations,
+include `agent_action_proposal` from `agentSchema`, `pgAgentSchema`, or
+`mysqlAgentSchema` (or the corresponding `agentSchemaDdl(dialect)` statements).
+The physical primary key and scope key are SHA-256 digests; original identifiers
+remain in the canonical JSON text snapshot and are matched exactly, including
+trailing spaces,
+independently of MySQL collation padding. Text storage preserves valid JSON strings
+containing NUL or lone surrogates on every dialect. Decision audit and execution
+work share
+the snapshot column, fenced by the numeric `version` column. Existing table schemas
+and fixture callers remain compatible.
+
+Worker discovery is available on `DrizzleAgentStore` and `DrizzleActionProposalStore`
+through the privileged `claimNextActionProposal({ workerId, leaseMs })` and
+`expireActionProposals({ limit })` methods. Do not expose these unscoped methods on
+public routes. Discovery selects at most 32 queued or expired-lease candidates;
+expiry selects at most `limit` (1–1000) pending proposals due at the trusted store
+clock. Existing scoped, version-fenced mutations decide which worker wins.
+
+Upgrading an existing proposal table requires the additive nullable
+`execution_status`, `lease_expires_at`, and `proposal_expires_at` columns, the
+`discovery_index_version` column with default 0, and the execution/expiry indexes
+from the exported schema. `ensureAgentSchema` adds these definitions; hosts using
+Drizzle migrations must include them in their forward migration. Stop old writers,
+apply the DDL, then repeat the bounded backfill before starting new workers:
+
+```ts
+while (await store.backfillActionProposalDiscoveryIndex({ limit: 1000 })) {
+  // Each batch derives metadata from the canonical payload under its row version.
+}
+```
+
+New writes maintain index version 1 and all discovery metadata atomically with the
+proposal snapshot. Backfill increments the row version without changing the
+proposal payload, timestamps, or audit. It does not invoke transitions or execute
+tools. This upgrade does not support mixed old and new writers: old writers cannot
+maintain the new indexes. Discovery deliberately excludes rows awaiting backfill;
+it never performs an implicit, unbounded migration.
+
+### Independent approval runtime
+
+`actionApprovalMode: 'independent'` requires proposal discovery, atomic replacement,
+terminal outcome admission and the thread queue on the same database. This adapter
+advertises admission support for better-sqlite3, node-postgres, mysql2 and libSQL;
+other Drizzle drivers retain blocking behavior and fail independent configuration.
+Synchronous SQLite transaction callbacks complete synchronously, including both
+assistant insertion and the delivery compare-and-set. PostgreSQL/MySQL admission
+locks the proposal and conversation before checking the delivery lease again.
+
+The updated schemas add nullable `delivery_status`, `delivery_lease_expires_at`,
+`replacement_group_key`, `outcome_id_key` and their indexes to proposals,
+`action_proposal_outcome` to messages, `proposal_id` to tool calls and
+`ui_capabilities` to queued messages. `ensureAgentSchema` adds missing columns and
+indexes; applications owning drizzle-kit migrations must include the same additions.
+Outcomes use escaped TEXT/LONGTEXT metadata to preserve JSON strings exactly.
+
+Delivery writes one assistant fact and its UI together with its admitted marker,
+waits while a user turn owns the conversation and deduplicates retries. Replacement
+only closes pending proposals in the same exact scope, tool and replacement group.
+An existing transaction passed as the database handle remains the transaction
+owner: a caller rollback also rolls back replacement or admission. External side
+effects remain at least once; use the stable tool-context idempotency key.
+
+See [the shared setup and rollout guide](../../docs/independent-approvals.md) for
+policies, text/channel decisions, remembered approvals and component capabilities.