@godspeedai/cognate-store-sqlite 0.1.1 → 0.2.0

package/README.md

@@ -1,40 +1,77 @@
 # @godspeedai/cognate-store-sqlite
 
-Authoritative `DurableStore` on `bun:sqlite` (spec §13).
-
-## Tables (spec §13.2)
-
-`schema_versions` (migrations), `event_streams` (per-stream version), `events` (envelope only —
-position, event_id, stream_id/sequence, type, timing, causality, semantic ref, metadata),
-`payloads` (event_id → JSON payload, separate from the envelope so `erasePayload` can delete
-content without touching causality), `idempotency` (scope+key → fingerprint/result/event_ids),
-`records` (collection+id → version/JSON value, CAS), `outbox` (id, topic, payload, status,
-attempts, `claimed_until` lease), `projection_checkpoints`, `projection_state`, `blobs`
-(sha256-addressed bytes).
-
-## Transaction boundary
-
-`commit()` runs the entire decision — envelope validation, idempotency lookup, stream/record CAS,
-event/record/outbox writes, idempotency recording — inside one `BEGIN IMMEDIATE` transaction.
-Any failure throws and rolls back everything; `onCommitted` listeners fire only after `COMMIT`
-returns. `commitProjection` and `erasePayload` are each their own single transaction. bun:sqlite
-is synchronous and the store performs no `await` before entering a transaction, so concurrent
-`commit()` calls in one process serialize naturally through the JS event loop — exactly one wins
-a stream-version race, the rest see `ConcurrencyError`.
-
-## Durability settings
-
-`journal_mode=WAL`, `synchronous=FULL` (fsync on every commit — durability over speed, per spec
-§31.2 "fail closed": a decision must never be reported committed before SQLite confirms it),
-`foreign_keys=ON`, `busy_timeout=5000`. `secure_delete=ON` so `erasePayload` overwrites freed
-pages instead of merely unlinking them; it also runs `PRAGMA wal_checkpoint(TRUNCATE)` so no
-plaintext lingers in the WAL either.
-
-## Limits
-
-Single-process writer: correctness relies on one Node/Bun process holding the connection and on
-bun:sqlite's synchronous API for in-process serialization, not on SQLite's file locking for
-cross-process write concurrency. Multiple processes writing the same file concurrently will hit
-`SQLITE_BUSY` after `busy_timeout` rather than being serialized by this store. Back up by copying
-the `.sqlite` file plus its `-wal`/`-shm` siblings only after a checkpoint (or use the SQLite
-backup API); a bare copy of the main file while WAL is active can miss committed data.
+Durable, transactional event storage for Cognate applications, built on `bun:sqlite`. One SQLite
+file holds an application's entire authoritative history: events, records, idempotency results,
+outbox intents, and blobs — committed atomically per decision.
+
+**When to use this package:** any Cognate app that must survive restarts. Use
+[`@godspeedai/cognate-store-memory`](https://www.npmjs.com/package/@godspeedai/cognate-store-memory)
+for tests and throwaway prototypes instead. Both implement the same
+`@godspeedai/cognate-events` store contract, so an app can switch stores with one line.
+
+## Installation
+
+```sh
+bun add @godspeedai/cognate-store-sqlite
+```
+
+Requires [Bun](https://bun.sh) >= 1.4.0 (the store uses Bun's built-in `bun:sqlite`).
+
+## Quick start
+
+```ts
+import { openSqliteStore } from "@godspeedai/cognate-store-sqlite";
+
+const store = openSqliteStore({ path: "./app.sqlite" });
+```
+
+Pass the store to `createRuntime` from
+[`@godspeedai/cognate-runtime-bun`](https://www.npmjs.com/package/@godspeedai/cognate-runtime-bun)
+to make an entire agent runtime durable — `createRuntime` also accepts a file path directly
+(`store: "./app.sqlite"`), which selects this store:
+
+```ts
+import { createRuntime } from "@godspeedai/cognate-runtime-bun";
+
+const runtime = await createRuntime({ store: "./app.sqlite" /* agents, policy, ... */ });
+```
+
+The store implements the `DurableStore` contract from
+[`@godspeedai/cognate-events`](https://www.npmjs.com/package/@godspeedai/cognate-events):
+`commit` applies an atomic batch per decision (stream appends with expected versions, record
+compare-and-swap writes, outbox intents, idempotency results), and `ConcurrencyError` is thrown
+when a concurrent writer loses a stream-version race.
+
+## What the schema guarantees
+
+Every committed decision is one SQLite transaction (`BEGIN IMMEDIATE`): envelope validation,
+per-stream version checks, record compare-and-swap writes, outbox intents, and idempotency
+recording succeed or roll back together. A duplicate idempotency key returns the originally
+recorded result instead of re-applying the decision.
+
+Durability is configured for correctness over speed: `journal_mode=WAL` with
+`synchronous=FULL` (fsync on every commit), so a decision is never reported committed before
+SQLite has confirmed it. Payload erasure overwrites freed pages (`secure_delete`) and
+checkpoints the WAL, so erased content does not linger in sidecar files.
+
+## Limitations
+
+- **One writer process.** Concurrent `commit` calls in one process serialize naturally, but
+  multiple processes writing the same file will hit `SQLITE_BUSY` after the 5-second busy
+  timeout rather than being serialized by this store.
+- **Back up carefully.** Copy the `.sqlite` file plus its `-wal`/`-shm` siblings only after a
+  WAL checkpoint (or use SQLite's backup API) — a bare copy while WAL is active can miss
+  committed data.
+
+## Related packages
+
+- [`@godspeedai/cognate-events`](https://www.npmjs.com/package/@godspeedai/cognate-events) — the
+  store contract and conformance kit this package implements.
+- [`@godspeedai/cognate-store-memory`](https://www.npmjs.com/package/@godspeedai/cognate-store-memory) —
+  the in-memory implementation of the same contract.
+- [`@godspeedai/cognate`](https://www.npmjs.com/package/@godspeedai/cognate) — the SDK umbrella;
+  the easiest way to get a runtime backed by this store.
+
+## License
+
+Apache-2.0

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@godspeedai/cognate-store-sqlite",
-  "version": "0.1.1",
+  "version": "0.2.0",
   "description": "Authoritative DurableStore on bun:sqlite (spec §13).",
   "type": "module",
   "license": "Apache-2.0",
@@ -22,6 +22,6 @@
     ".": "./src/index.ts"
   },
   "dependencies": {
-    "@godspeedai/cognate-events": "^0.1.1"
+    "@godspeedai/cognate-events": "^0.2.0"
   }
 }

package/src/index.ts

@@ -15,6 +15,7 @@ import type {
   OutboxSettlement,
   ProjectionCheckpoint,
   ProjectionCommit,
+  ProjectionSnapshot,
   Json,
 } from "@godspeedai/cognate-events";
 import { migrate } from "./schema.ts";
@@ -313,6 +314,15 @@ export function openSqliteStore(options: SqliteStoreOptions): DurableStore {
     db.query("DELETE FROM payloads WHERE event_id = ?").run(eventId);
     db.query("UPDATE events SET erasure_reason = ?, erasure_at = ? WHERE event_id = ?").run(reason, erasedAt, eventId);
 
+    // Idempotency results are derived copies: a stored result may quote the erased payload (the
+    // continuation-transition shape) or fold earlier events of the same stream (the updateState
+    // shape). Erasure tombstones such results so replay can never re-disclose the content.
+    db.query(
+      `UPDATE idempotency SET result = ?
+       WHERE EXISTS (SELECT 1 FROM json_each(idempotency.event_ids) je WHERE je.value = ?)
+          OR EXISTS (SELECT 1 FROM json_each(idempotency.event_ids) je JOIN events e ON e.event_id = je.value WHERE e.stream_id = ?)`,
+    ).run(JSON.stringify({ __erased: { reason, erasedAt } }), eventId, original.stream_id);
+
     ensureStream("$erasures");
     const sequence = streamVersionRow("$erasures") + 1;
     const tombstone = insertEventRow(
@@ -472,6 +482,29 @@ export function openSqliteStore(options: SqliteStoreOptions): DurableStore {
       return rows.map((row) => ({ key: row.key, value: JSON.parse(row.value) as Json }));
     },
 
+    async readProjectionSnapshot(name: string): Promise<ProjectionSnapshot> {
+      assertOpen();
+      // One statement, one snapshot: the checkpoint can never be ahead of the rows it is returned
+      // with, even with another process folding concurrently (DEBT-021).
+      const row = db
+        .query(
+          `SELECT
+             (SELECT json_object('name', name, 'version', version, 'position', position, 'status', status,
+                                 'error_position', error_position, 'error_event_id', error_event_id, 'error_message', error_message)
+              FROM projection_checkpoints WHERE name = ?1) AS checkpoint,
+             (SELECT json_group_array(json_object('key', ordered.key, 'value', json(ordered.value)))
+              FROM (SELECT key, value FROM projection_state WHERE name = ?1 ORDER BY key) AS ordered) AS rows`,
+        )
+        .get(name) as Row;
+      const checkpoint = row.checkpoint ? JSON.parse(row.checkpoint as string) : undefined;
+      return {
+        checkpoint: checkpoint
+          ? { ...checkpoint, error: checkpoint.error_event_id != null ? { position: checkpoint.error_position, eventId: checkpoint.error_event_id, message: checkpoint.error_message } : null }
+          : undefined,
+        rows: (JSON.parse(row.rows as string) as { key: string; value: Json }[]).map((entry) => ({ key: entry.key, value: entry.value })),
+      };
+    },
+
     async resetProjection(name: string): Promise<void> {
       assertOpen();
       const txn = db.transaction((n: string): void => {