@happyvertical/smrt-core 0.51.28 → 0.51.29

package/AGENTS.md

@@ -16,6 +16,7 @@ and repository rules.
 | `src/schema/`, `src/migrations/`, `src/cascade.ts`, `src/system/` | DDL parity, indexes, migrations, delete integrity, retention | [agents/schema-paths.md](agents/schema-paths.md) |
 | `src/postgres-permissions.ts` | Explicit PostgreSQL ACL plans and atomic reconciliation | [deployment permission contract](../../docs/content/postgres-permissions.md) |
 | `src/change-feed.ts` | Durable changes, cursors, table versions, retention | [agents/change-feed.md](agents/change-feed.md) |
+| `src/change-feed-authz.ts` | Consumer table/row change-feed authorization hooks (#3020) | [agents/change-feed.md](agents/change-feed.md) |
 | `src/change-signals.ts` | Signal bus, replica fan-out, SSE | [agents/change-signals.md](agents/change-signals.md) |
 | `src/generators/`, `src/vite-plugin/web-collections.ts` | REST/CLI/MCP generation, manifest hashes, ETags | [agents/generators.md](agents/generators.md) |
 | `src/vite-plugin/`, `src/consumer-plugin/`, `src/knowledge.ts` | Decorator UI hints, knowledge projection, generation snapshots | [agents/build-knowledge.md](agents/build-knowledge.md) |
@@ -23,6 +24,7 @@ and repository rules.
 | `src/object.ts`, `src/collection.ts`, `src/learning/memory.ts` | Context memory and semantic search | [agents/memory.md](agents/memory.md) |
 | `src/system/diagnostics.ts` | SELECT-only `_smrt_*` diagnostics reader behind smrt-dev-mcp runtime tools (#1824) | [agents/system-diagnostics.md](agents/system-diagnostics.md) |
 | `src/system/registry-snapshot.ts` | Sanitized plain-JSON projection of the booted `ObjectRegistry` for the smrt-dev-mcp runtime dev-plane (#1831); never constructors, validators, values, or absolute paths | [agents/registry-snapshot.md](agents/registry-snapshot.md) |
+| `src/run-once.ts` | Shared idempotency seam: insert-only claim + caller work in one transaction, replay on retry, typed in-flight/unknown-outcome answers (#3080) | [agents/run-once.md](agents/run-once.md) |
 
 ## Cross-module invariants
 
@@ -53,6 +55,24 @@ and repository rules.
 - `withDatabase(db, callback)` restores only database bindings (including public
   `options.db`); `withTransaction(callback)` also restores identity/revision
   metadata after rollback. Do not use a bound instance concurrently.
+- `runOnce({ db, tenantId, actor, token, content }, work)` (`src/run-once.ts`,
+  #3080) claims a key derived from all four inputs — never the token alone,
+  which collapses distinct submissions — via `INSERT ... ON CONFLICT
+  (claim_key) DO NOTHING RETURNING claim_key` into the hand-DDL
+  `_smrt_run_once_claims` system table, and runs `work` in the SAME
+  transaction; a completed claim replays its stored result instead of
+  re-running, a failed `work` rolls the claim back with the transaction, and
+  an unresolved claim raises typed `RunOnceClaimError`
+  (`RUN_ONCE_IN_FLIGHT` / `RUN_ONCE_OUTCOME_UNKNOWN`) rather than guessing.
+  The claim insert must never raise on conflict: PostgreSQL aborts the whole
+  transaction on any raised error, which would take the recovery `SELECT`
+  down with it too — this is why it is `DO NOTHING`, not a plain `INSERT`
+  caught for `isUniqueViolationError`. Completed claims are bounded by
+  `runRetentionSweep()`'s `run-once-claims` task (`system/retention.ts`,
+  `DEFAULT_RETENTION_POLICY.runOnceClaims`, default 30 days on
+  `completed_at`; `in_progress` rows are never matched) — the window is a
+  replay deadline, not just disk hygiene: a token replayed after its claim
+  is swept runs `work()` again. See `agents/run-once.md`.
 - `ensureSystemTables(db, typeHint?)` provisions framework tables idempotently;
   call it on a base PostgreSQL connection before caller-owned transactions.
   Bootstrap uses a transaction-scoped advisory lock, taken only when the

package/README.md

@@ -67,6 +67,52 @@ const results = await products.list({
 
 ```
 
+### Run an action exactly once per submission (`runOnce`)
+
+A double-tapped submit, a retried request, or two concurrent requests should
+never write the same record twice. `runOnce()` (#3080) is the shared
+idempotency seam: an insert-only claim, written in the SAME transaction as
+the work it guards, so a retry either runs the work once or replays the first
+attempt's stored result.
+
+```typescript
+import { runOnce, RunOnceClaimError } from '@happyvertical/smrt-core';
+
+const purchaseOrderId = await runOnce(
+  {
+    db,
+    tenantId: session.tenantId,
+    actor: session.userId,
+    token: formData.get('submissionToken') as string, // minted once per rendered form
+    content: { supplierId, lines }, // only its digest is persisted, never the raw value
+  },
+  async (tx) => {
+    const po = await createPurchaseOrder(tx, { supplierId, lines });
+    return po.id; // must be JSON-serializable — this is what a replay returns
+  },
+);
+```
+
+- **Key derivation is `(tenantId, actor, token, contentDigest)` — never the
+  token alone.** A token-only key collapses two genuinely different
+  submissions that happen to reuse one form's token. `contentDigest` is a
+  sha256 of the sorted-key-JSON encoding of `content`, so key order never
+  changes it.
+- The claim lives in the hand-migrated `_smrt_run_once_claims` system table
+  (never an application table you migrate yourself) and only ever stores
+  digests plus the JSON-serialized `work()` result — never the raw token or
+  raw content.
+- A claim whose work is still in flight, or whose outcome cannot be resolved,
+  rejects with `RunOnceClaimError` (`code: 'RUN_ONCE_IN_FLIGHT'` or
+  `'RUN_ONCE_OUTCOME_UNKNOWN'`) instead of silently re-running or guessing —
+  mirroring the refused-vs-unknown-outcome distinction from #2990. Retry with
+  the SAME token and content; `runOnce()` never retries on its own.
+- If `work()` throws, the transaction that inserted the claim rolls it back
+  too — a failed attempt never strands a claim, and a retry with the same key
+  runs again.
+
+See [`agents/run-once.md`](agents/run-once.md) for the full contract.
+
 ### Bounded multi-collection reads
 
 When one request needs several independent collections, use a keyed read plan
@@ -527,6 +573,7 @@ Every `SmrtObject`/`SmrtCollection` can persist learned knowledge via `remember(
 | `AIError` | AI provider failures |
 | `ValidationError` | Field/object validation failures |
 | `RuntimeError` | General runtime failures |
+| `RunOnceClaimError` | `runOnce()` claim in flight or unresolvable (`RUN_ONCE_IN_FLIGHT` / `RUN_ONCE_OUTCOME_UNKNOWN`) |
 | `ErrorUtils` | Retry policy (`withRetry`, `isRetryable`) plus sanitization helpers |
 
 ### Database error classification

package/agents/build-knowledge.md

@@ -15,6 +15,22 @@ schemas. No schema/persistence/security effect — `sensitive`/`readPermission`
 stay the security rail, and `sensitive`/`transient` fields never emit to the
 client at all.
 
+## Worker registration artifact (#3117)
+
+With `svelteKit.enabled`, a production `vite build` compiles the generated
+`smrt-register.ts` into `.smrt/runtime/register.js` after the server build
+(`src/vite-plugin/worker-registration.ts`, called once from `closeBundle`).
+adapter-node re-bundles the server from its own entries, so the web bundle is
+never importable by a separate Node process; this artifact is how a deployed
+task/schedule worker registers the app's local objects (and, through the
+`.smrt/register.js` it imports, consumed packages' objects) with the same
+qualified identities and isolated manifests as the web server. It is a nested
+SSR build with no consumer plugins: only aliases pointing into the project
+sources (e.g. `$lib`) apply, and every other bare import stays external so the
+worker shares the installed `@happyvertical/smrt-core` registry. SvelteKit
+virtual modules (`$env/*`, `$app/*`) are not available to a worker; an object
+importing one fails at worker startup. A failure fails the build.
+
 ## Lightweight discovery
 
 `src/knowledge-discovery.ts`, exported through `smrt-core/knowledge`, enumerates

package/agents/change-feed.md

@@ -94,10 +94,101 @@ through a resolver hook on the leaf module (so `change-feed.ts` never imports
 the registry), sees the exact name being recorded with no derivation to keep in
 sync, and declares it — closing the read path and signal bus for that table.
 
-Scope note: the generated `_changes`/`_events` routes still gate on an
-authenticated principal only — there is no per-table permission check, and
-tenant scoping (`getTenantScopedChangesSince`, fail-closed) remains the sole
-row-level filter for ordinary tables.
+Scope note: the generated `_changes`/`_events` routes — both the SvelteKit
+routes and the runtime REST generator's — authorize on an authenticated
+principal plus tenant scoping (`getTenantScopedChangesSince`, fail-closed) by
+default. A consumer-supplied table/row authorization seam beyond that —
+`change-feed-authz.ts` (#3020, below) — is optional and off until registered.
+
+## Consumer table/row authorization seam (#3020)
+
+Tenant scoping keeps one tenant from seeing another's rows, but it cannot
+express "table X is readable by some principals of the tenant but not others"
+or "principal P only sees their own rows of table X" (e.g. a bay-tablet
+station versus the back office). `?tables=` on `_changes` was always a
+client-chosen filter, never enforcement — a table the client did not ask for
+was withheld only because it did not ask, not because it could not have.
+
+`src/change-feed-authz.ts` closes that gap with two independent, optional
+hooks, dependency-inverted onto `globalThis` exactly like
+`resolveDispatchTenantScope` (`dispatch/tenant-resolver.ts`), so a consumer
+registers them once with no change to the generated "DO NOT EDIT" route files.
+Both hooks are evaluated identically by **every** transport — the generated
+SvelteKit `_changes`/`_events` routes AND the runtime REST generator's
+`_changes`/`_events` (`generators/changes-route.ts`, `generators/events-route.ts`)
+— so a consumer serving both cannot register a hook that protects one and
+leaks through the other:
+
+- `setChangeFeedAuthorizer(authorizeChangeFeed)` — table-level.
+  `authorizeChangeFeed({ locals, request, tables }) => allowedTables` returns
+  the tables the requester may read from the feed at all; the result is
+  INTERSECTED with the client's `?tables=` filter (never unioned), so a table
+  the hook does not name is never queried, let alone returned.
+- `setChangeFeedEntryVisibility(isChangeFeedEntryVisible)` — row-level, for
+  scope narrower than a whole table. Applied per entry, after the table
+  filter, before anything is serialized — including a live `_events` signal.
+
+`locals` is SvelteKit's `event.locals`; `request` is SvelteKit's
+`event.request` or, on REST, the `authMiddleware`-processed `Request`. REST
+has no `locals` concept, so REST routes pass `locals: undefined` and a
+REST-hosting consumer identifies the principal from `request` instead
+(whatever the `authMiddleware` attached — headers, a decoded token, etc.).
+Every generated route supplies `request`.
+
+Both hooks are optional and independent; unregistered, every transport
+behaves exactly as before (tenant-scoped only). A hook that throws or returns
+something other than the documented shape fails closed to "authorizes
+nothing" for that call — same posture as a throwing `resolveDispatchTenantScope`
+resolver, and the same 200-empty-page shape the sensitivity filter above
+already gives a request naming only tables it may not see (never a 5xx, which
+would tell a caller its own hook is broken — indistinguishable from "denied"
+from the caller's side).
+
+Every filter narrows the *returned* `changes`; nothing ever touches
+`cursor`/`resyncRequired`/`resyncCursor` — mirrors the sensitivity filter's
+"filters affect what is returned, never how the cursor advances" and for the
+same reason: a client must advance past a denied entry without re-polling it
+forever, and the entry's row id/timing must never be inferable from a stalled
+cursor. Table denial sets `getChangesSince`'s explicit `denyAllTables: true`
+option rather than passing an empty `tables` array, because `getChangesSince`
+treats an omitted/empty `tables` as "no filter" and would otherwise widen the
+read. `denyAllTables` skips the `table_name IN (...)` clause (and the query)
+entirely — an earlier version instead passed a guaranteed-no-match sentinel
+table name through that clause, which broke on PostgreSQL (it rejects a NUL
+byte in a text parameter, turning a denied request into a 500 instead of the
+required 200 empty page, #3020 P1 follow-up). `denyAllTables` still runs every
+resync/pruned-cursor check unfiltered first, so the real horizon computation
+runs and the cursor still advances — it just matches zero rows.
+`change-feed-authz.ts`'s `isChangeFeedDenyAll()` is the one place that decides
+"empty allow-list" (deny all) vs. "no hook registered" (no filter); both
+`readAuthorized()` (pull side) and `buildChangeEventStream()`'s catch-up loop
+(push side) call it rather than re-deriving the distinction. With NO hook
+registered, `resolveAuthorizedChangeFeedTables()` normalizes an explicit empty
+requested `tables: []` to `undefined` before that decision runs: `?tables=`
+with nothing in it is documented as equivalent to an omitted filter, and
+without a hook there is no authorized answer to narrow anything with — feeding
+`[]` through unchanged would let `isChangeFeedDenyAll()` read it as "deny
+every table" and silently break the promised unchanged default (#3020
+follow-up). Once a hook IS registered, its own answer is what decides
+deny-all, exactly as above.
+
+A registered hook's returned table list is also validated for USABLE
+strings, not just string-typed ones: any entry that is empty or contains a
+NUL byte fails the whole call closed to `[]` (deny all), the same posture as
+a non-array or non-string result. This closes the same NUL-byte hazard the
+`denyAllTables` sentinel removal above fixed for a *synthesized* value — here
+the NUL byte would come from the hook's OWN answer, which otherwise flows
+straight into the ordinary `table_name IN (...)` clause and 500s on
+PostgreSQL (#3020 Copilot follow-up).
+
+`getAuthorizedChangesSince()` / `getAuthorizedTenantScopedChangesSince()` wrap
+`getChangesSince()` / `getTenantScopedChangesSince()` with both hooks applied;
+BOTH `_changes` routes (SvelteKit's `vite-plugin/changes-route.ts` and REST's
+`generators/changes-route.ts` `handleChangesRoute`) call the tenant-scoped
+variant, and `buildChangeEventStream()` (`change-signals.md`) — shared by
+BOTH `_events` routes — applies the same hooks to catch-up replay and live
+signal delivery, resolving the table allow-list ONCE at connection open
+(mirrors captured tenant scope).
 
 ## Compatible bulk mutations (#2818)
 

package/agents/change-signals.md

@@ -10,5 +10,10 @@ The push companion to the change feed (`src/change-signals.ts` + the generated `
 
 - **Change-signal bus** (`src/change-signals.ts`): every framework save/delete that appends a durable feed row also publishes a coarse `ChangeSignal` `{ table, operation, rowId, tenantId, seq }` — **never a row payload** (authorization stays on the read path). Structurally mirrors the collection cache's notify/listen path. `subscribeToChangeSignals(db, listener) → unsubscribe`; `publishChangeSignal`/`broadcastChangeSignal`/the listener loop stay internal. `appendChange` now returns the allocated `seq` (was `void`); the signal carries it as a coarse resume cursor (`bumpChangeFeed` ignores the return). The publish runs only after the append SUCCEEDS (no signal without a durable feed row) and in its own log-and-swallow try/catch, so a signal problem never fails the user's write. `_smrt_*` writes never signal (the writer skips them). Delivery is synchronous per-listener with per-listener try/catch (one throwing SSE controller never blocks others); no per-subscriber queue — backpressure rides the platform `ReadableStream`.
 - **In-process + cross-replica**: locally-published and peer-received signals go through the SAME `deliverLocally` path. Cross-replica fan-out rides the db adapter's optional notification capability (`db.notifications`, a NEW `smrt_change_signals` channel distinct from the cache channel) with echo-avoidance by `PROCESS_ID`. No capability → in-process only, warn-once, **never an error, never blocks the write** (subscribers on other replicas fall back to cursor polling).
-- **Generated `_events` SSE route**: REST (`GET {basePath}/_events`, requires `authMiddleware`, otherwise 401 — fail-closed, per-model `api.public` does NOT apply; 405 non-GET; 503 no db or subscriber capacity reached) and SvelteKit (`{routesDir}/_events/+server.ts`, requires an authenticated principal on `locals`; opt out via `sveltekit.eventsRoute.enabled: false`; cap via `sveltekit.eventsRoute.maxSubscribers`, where `0` means unlimited). Tenant scope is captured ONCE at connection open (`resolveDispatchTenantScope`) and filtered server-side per signal via `signalVisibleToTenant` (same rule as `getChangesSince`'s tenant filter) before any byte hits the wire — delivery runs outside any tenant ALS context, so it must use the captured value. The stream lifecycle lives in `buildChangeEventStream(db, { cursor, tenantScope, heartbeatMs?, manifestHash? })` (exported; the SvelteKit route imports it so it stays thin): subscribe-before-catch-up (closes the gap window; overlap is deduped by the SSE `id:`/seq client-side), `retry: 3000`, optional connection-open `event: manifest` carrying `{ manifestHash }` for live contract detection, cursor catch-up via `getChangesSince` filtered by the CAPTURED scope (not re-resolved from ALS, so it matches the live-signal filter exactly and can't replay another tenant's rows) (`Last-Event-ID` header beats `?since=`; default = live-forward only; `resyncRequired` → `event: resync`), heartbeat (`DEFAULT_EVENTS_HEARTBEAT_MS` = 15s), and `cancel()` teardown (clears heartbeat + unsubscribes). SSE change frame: `id: <seq>\nevent: change\ndata: {table,operation,rowId,tenantId}\n\n` — seq is ONLY in the `id:` line, never the data JSON. Subscriber cap default is `DEFAULT_EVENTS_MAX_SUBSCRIBERS` = 1000; over-cap connections return retryable 503 + `Retry-After`, and existing subscribers are unaffected. **Cross-origin is opt-in and fail-closed (#1861)**: same-origin only by default. REST wraps `_events` in its CORS layer — set `enableCors` + an explicit `allowedOrigins` allowlist + `allowCredentials: true` and an allow-listed browser can subscribe with a credentialed `EventSource` (`withCredentials: true`); the response echoes the specific `Origin` (never `*`) plus `Access-Control-Allow-Credentials: true`. SvelteKit mirrors this via `sveltekit.eventsRoute.allowedOrigins` + `allowCredentials` (the generated route bakes the allowlist into a `Set`, echoes only a member origin, and answers a credentialed `OPTIONS` preflight). CORS never authorizes — the fail-closed auth guard and captured tenant scope are unchanged, so the read posture holds identically across origins; it only lets an allow-listed browser's cookies reach the guard. Client disconnect through the Node `createServer` bridge now cancels the response reader (was a teardown leak) so `cancel()` fires and the subscription is released.
+- **Generated `_events` SSE route**: REST (`GET {basePath}/_events`, requires `authMiddleware`, otherwise 401 — fail-closed, per-model `api.public` does NOT apply; 405 non-GET; 503 no db or subscriber capacity reached) and SvelteKit (`{routesDir}/_events/+server.ts`, requires an authenticated principal on `locals`; opt out via `sveltekit.eventsRoute.enabled: false`; cap via `sveltekit.eventsRoute.maxSubscribers`, where `0` means unlimited). Tenant scope is captured ONCE at connection open (`resolveDispatchTenantScope`) and filtered server-side per signal via `signalVisibleToTenant` (same rule as `getChangesSince`'s tenant filter) before any byte hits the wire — delivery runs outside any tenant ALS context, so it must use the captured value. The stream lifecycle lives in `buildChangeEventStream(db, { cursor, tenantScope, heartbeatMs?, manifestHash?, locals? })` (exported; the SvelteKit route imports it so it stays thin): subscribe-before-catch-up (closes the gap window; overlap is deduped by the SSE `id:`/seq client-side), `retry: 3000`, optional connection-open `event: manifest` carrying `{ manifestHash }` for live contract detection, cursor catch-up via `getChangesSince` filtered by the CAPTURED scope (not re-resolved from ALS, so it matches the live-signal filter exactly and can't replay another tenant's rows) (`Last-Event-ID` header beats `?since=`; default = live-forward only; `resyncRequired` → `event: resync`), heartbeat (`DEFAULT_EVENTS_HEARTBEAT_MS` = 15s), and `cancel()` teardown (clears heartbeat + unsubscribes). SSE change frame: `id: <seq>\nevent: change\ndata: {table,operation,rowId,tenantId}\n\n` — seq is ONLY in the `id:` line, never the data JSON. Subscriber cap default is `DEFAULT_EVENTS_MAX_SUBSCRIBERS` = 1000; over-cap connections return retryable 503 + `Retry-After`, and existing subscribers are unaffected. **Cross-origin is opt-in and fail-closed (#1861)**: same-origin only by default. REST wraps `_events` in its CORS layer — set `enableCors` + an explicit `allowedOrigins` allowlist + `allowCredentials: true` and an allow-listed browser can subscribe with a credentialed `EventSource` (`withCredentials: true`); the response echoes the specific `Origin` (never `*`) plus `Access-Control-Allow-Credentials: true`. SvelteKit mirrors this via `sveltekit.eventsRoute.allowedOrigins` + `allowCredentials` (the generated route bakes the allowlist into a `Set`, echoes only a member origin, and answers a credentialed `OPTIONS` preflight). CORS never authorizes — the fail-closed auth guard and captured tenant scope are unchanged, so the read posture holds identically across origins; it only lets an allow-listed browser's cookies reach the guard. Client disconnect through the Node `createServer` bridge now cancels the response reader (was a teardown leak) so `cancel()` fires and the subscription is released.
+- **Table/row authorization** (#3020, `change-feed-authz.ts`), evaluated on BOTH transports: the SvelteKit route forwards `{ locals, request }` and REST's `handleEventsRoute` forwards `{ locals: undefined, request: authResult }` (the `authMiddleware`-processed request — REST has no `locals`, so a hook a REST-hosting consumer registers identifies the principal from `request`) into `buildChangeEventStream`. If the consumer registered `authorizeChangeFeed`/`isChangeFeedEntryVisible`, `start()` resolves the allowed-tables set ONCE at connection open — same reasoning and same code shape as captured `tenantScope` — and applies both hooks to catch-up replay (via `getChangesSince`'s `tables` filter, passing `denyAllTables: true` — never a synthesized sentinel table name — when the resolved allow-list is empty; `isChangeFeedDenyAll()` makes that call the same way `change-feed-authz.ts`'s own `readAuthorized()` does; see `change-feed.md`'s #3020 section) and to every live signal before `controller.enqueue`. Table checks stay synchronous (a `Set.has`); the row-visibility check is async, so live delivery chains it through a per-connection `deliveryQueue` promise to keep order — but that chain is only ever allocated when a row hook is actually registered, so the unconfigured default keeps the pre-#3020 fully-synchronous per-listener delivery ("delivery is synchronous per-listener" above still holds when no row hook is registered). `start()` re-checks `closed` immediately after that authorization `await` and bails out without subscribing when the client disconnected while it was pending (#3020 P2) — `cancel()` may run `teardown()` before any subscription exists, and subscribing after `closed` is already `true` would register a `change-signals` listener no later `teardown()` call could ever remove (it short-circuits on `closed`), permanently inflating `changeSignalSubscriberCount`.
+
+**Live-forward gap fill during the authorizer await** (#3020 follow-up): `unsubscribe` above is not set until AFTER the table-authorizer `await` resolves, so a write that commits (and signals) while a hook is pending has no listener to reach it. With an explicit `cursor`, this is harmless — the catch-up phase (below) starts from that pre-await cursor and replays anything committed in the gap. In live-forward-only mode (`cursor === null`) there is no catch-up phase at all, so that write would be lost forever, not merely delayed. When a table hook is registered AND `cursor === null`, `start()` therefore captures the feed's current head BEFORE the authorizer `await` (one `denyAllTables: true` `getChangesSince` bounds query — cheap, no row scan) as a gap-fill cursor; once the allow-list resolves, catch-up runs from that captured cursor exactly as it would for an explicit one, replaying only entries appended during the await (applying the same resolved table/row hooks) before the stream goes fully live. A supplied `cursor` needs no separate capture since its own catch-up already starts before the same await.
+
+Subscribe still happens before the gap-fill catch-up query runs, so a write landing in that narrower window (between subscribe attaching and the catch-up read) can again be delivered twice — once live, once replayed — exactly like the ordinary subscribe-then-catch-up overlap above; the client dedupes both cases identically via the SSE `id:`/seq.
 - **Known gaps** (documented in the module): raw-SQL writes don't signal (same gap as the feed); live signals for caller-managed-transaction writes are best-effort (the append + signal fire pre-commit, so a rolled-back write may emit a signal and its freed seq is later reused) — the autocommit default path is exact, and clients reconcile via full catch-up/resync (inherits the change feed's transaction caveat).

package/agents/run-once.md

@@ -0,0 +1,268 @@
+# runOnce: shared idempotency seam (#3080)
+
+`src/run-once.ts`. A DB/transaction-level primitive — it takes a
+`DatabaseInterface`, not a `SmrtObject`/`SmrtCollection` — so it works outside
+SvelteKit and outside the ORM. Any caller with a database handle and a
+JSON-serializable work result can use it.
+
+## Why this exists, and why it is not the token alone
+
+Every consumer with a form, a retried request, or a double-tapped button
+needs the same guard, and a first hand-rolled attempt at it (reported against
+this issue) keyed the claim on the per-form token alone. That silently
+collapsed two genuinely different submissions that happened to reuse one
+form's token — the exact bug this primitive exists to prevent everyone from
+re-introducing.
+
+## Key derivation
+
+```
+normalized    = normalizeRunOnceContent(content)   // one pass; FormData captured as a node
+contentDigest = containsFormData(normalized)
+  ? sha256("\u0001run-once-formdata-v1\n" + stableStringify(toFormDataDomain(normalized)))
+  : sha256(stableStringify(normalized))              // plain JSON: unchanged since #3080
+claimKey      = sha256(stableStringify([tenantId, actor, token, contentDigest]))
+```
+
+`stableStringify` (`../src/knowledge-graph.ts`) is sorted-key JSON: property
+insertion order never changes the digest. Content is normalized first, following
+JSON serialization exactly: `toJSON(key)` is called with the property name or
+array index (a `Date` digests as its ISO string, not `{}`), and `undefined`,
+function and symbol values are omitted from objects and written as `null` in
+arrays. A `FormData` has no enumerable properties, so
+JSON would reduce every form to `{}`; it digests as its ordered entry list
+instead (repeated fields are significant; a `File` digests by name, type and
+size), hashed in a separate domain (the input is prefixed with a byte sorted-key
+JSON never starts with, and user keys starting with NUL are escaped) so no plain
+object can forge it and plain-JSON digests are unchanged. `Map`, `Set`,
+`RegExp`, any other non-plain object without `toJSON()`, and `bigint` throw a
+`TypeError` rather than silently hashing as `{}` (#3136). Folding in `tenantId`
+and `actor`
+means two tenants or two actors can never collide on the same claim even if a
+token were somehow shared between them; folding in `contentDigest` means a
+retry of the SAME submission (same token, same content) replays, while a
+DIFFERENT submission through the same token (edited content before a
+resubmit) runs as its own claim.
+
+Neither the raw token nor the raw content is ever persisted — only their
+digests — so `_smrt_run_once_claims` cannot leak submitted business data.
+
+## Placement: `smrt-core`, not `smrt-svelte`
+
+The issue posed this as a judgement call. `runOnce()` lives in `smrt-core`
+beside the other write primitives (`object.ts`, `collection.ts`) because:
+
+- It is DB/transaction-level, not SvelteKit-specific — any caller with a
+  `DatabaseInterface` can use it, including CLI jobs, MCP tool handlers, and
+  non-web callers.
+- The claim table must exist for every `smrt-core` consumer, not only ones
+  that also depend on `smrt-svelte`.
+
+No SvelteKit form-token helper shipped alongside it. The token contract
+(mint a random, unguessable value once per rendered form; carry it as a
+hidden field or equivalent; pass it straight through to `runOnce()`) is a few
+lines any consumer already owns as part of rendering its own form, and there
+was no existing svelte-facing package this fit into without adding one. If a
+second consumer needs the exact same minting/reading helper, add it to
+`smrt-svelte`'s forms module then — do not add a package for one caller.
+
+## Schema: a hand-DDL system table, not an `@smrt()` model
+
+`_smrt_run_once_claims` is declared in `src/system/schema.ts`
+(`CREATE_SMRT_RUN_ONCE_CLAIMS_TABLE`), added to `ALL_SYSTEM_TABLES`, and
+created by `ensureSystemTables()` / `bootstrapSystemTables()`
+(`src/system/bootstrap.ts`) — the same mechanism that creates
+`_smrt_migrations` and `_smrt_contexts`. This is "Mechanism A" in the
+`_smrt_` prefix classification ([schema-paths.md](schema-paths.md)), not
+"Mechanism B" (an `@smrt()`-decorated model migrated via `db:migrate`, like
+`_smrt_jobs`).
+
+This was a deliberate choice, not the default: `runOnce()` must work for any
+`smrt-core` consumer without that consumer registering a class and running a
+migration for a table they've never heard of. A Mechanism-B table only exists
+once an app imports and registers its owning class; a Mechanism-A table
+exists for every `smrt-core` database unconditionally, which is what a
+framework-wide primitive needs.
+
+Adding the table required:
+
+- A new DDL constant in `ALL_SYSTEM_TABLES`.
+- Bumping `SMRT_SCHEMA_VERSION` (1.10.1 → 1.11.0) — `CREATE TABLE IF NOT
+  EXISTS` is a no-op on an existing database, so nothing upgrades without the
+  bump.
+- A new `SMRT_SCHEMA_DDL_CHECKSUMS` entry — `system-schema-evolution.test.ts`
+  recomputes the DDL hash and fails the build if it does not match the
+  recorded value for the current version.
+
+No new column was added to an existing system table, so no
+`addColumnIfMissing()` entry in `system/compatibility.ts` was needed. Live
+schema parity (`checkLiveSchemaParity`, `smrt doctor --db` /
+`db:status --parity`) picks the table up automatically: system-table shape
+parsing (`schema/system-table-shapes.ts`) derives its expectation from the
+same DDL rather than a hand-maintained list.
+
+## Table shape
+
+```sql
+CREATE TABLE IF NOT EXISTS _smrt_run_once_claims (
+  claim_key TEXT PRIMARY KEY,       -- the derived key above
+  tenant_id TEXT NOT NULL,          -- redundant with claim_key; kept for admin queries and a defense-in-depth tenant filter on lookup
+  actor TEXT NOT NULL,              -- same
+  content_digest TEXT NOT NULL,     -- same, plus lets an operator distinguish "same token, different content" incidents
+  status TEXT NOT NULL DEFAULT 'in_progress',  -- 'in_progress' | 'completed'
+  result TEXT,                      -- JSON-serialized work() return value, set only when status = 'completed'
+  created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
+  completed_at TIMESTAMP
+);
+```
+
+`claim_key` is the PRIMARY KEY — the "insert-only against a UNIQUE column"
+the issue asked for. A second `INSERT` for the same key never adopts the
+existing row.
+
+**Why the claim insert is raw SQL (`INSERT ... ON CONFLICT (claim_key) DO
+NOTHING RETURNING claim_key`), not a plain `INSERT` caught for a classified
+`unique_violation`:** on PostgreSQL, a statement that raises an error aborts
+the WHOLE transaction ("current transaction is aborted, commands ignored
+until end of transaction block") — every later statement in that same
+transaction fails too, including the `SELECT` `runOnce()` needs to read the
+winner's stored result back. A first version of this primitive did exactly
+that (plain `INSERT` + catch classified via `isUniqueViolationError`,
+`../src/db-errors.ts`) and every replay/double-submit on PostgreSQL broke:
+review caught it before merge. `DO NOTHING` never raises — it reports zero
+rows via `RETURNING` instead — so the transaction's error state is never
+touched. SQLite and DuckDB support the identical `ON CONFLICT ... DO
+NOTHING` / `RETURNING` syntax (already used elsewhere in this codebase for
+`_smrt_migrations`, `system/bootstrap.ts`), so one query is portable across
+every adapter without engine branching.
+
+## Atomicity and rollback
+
+`runOnce()` opens `db.transaction()` (or nests under a caller-supplied
+transaction, per `@happyvertical/sql`'s own SAVEPOINT /
+`NestedTransactionError` rules) and, inside it: inserts the claim
+(`in_progress`), runs `work(tx)` with that SAME transaction-bound handle, then
+updates the claim to `completed` with the JSON-serialized result — all one
+transaction.
+
+If `work()` throws, the callback throws, and `db.transaction()`'s own
+commit-on-success/rollback-on-error contract undoes the claim insert along
+with whatever `work()` partially wrote. A claim is never stranded
+`in_progress` by a failed attempt; a retry with the same key can always run
+again. No compensating delete is needed — this falls out of using one
+transaction for the whole claim+work+completion sequence, not a special case
+in `runOnce()`'s own code.
+
+## Concurrency
+
+`@happyvertical/sql`'s single-connection adapters (SQLite, DuckDB, JSON)
+serialize `transaction()` calls on one connection: a second, concurrent
+`runOnce()` call does not begin its own transaction until the first ends.
+PostgreSQL pools, but the claim insert's arbiter still has to check the
+unique index against another transaction's uncommitted row of the same
+PRIMARY KEY, and blocks until that transaction resolves, then either no-ops
+(committed — `DO NOTHING` fires, zero rows) or inserts (rolled back). On
+every adapter, two concurrent callers with the same key never both run
+`work()`, and the loser never sees a PostgreSQL error for it — only zero
+rows back, at which point it reads the now-committed, now-`completed` claim
+row and replays its stored result.
+
+This is also why the "in flight" and "unknown outcome" branches below are
+normally unreachable through `runOnce()`-to-`runOnce()` races alone on a
+single connection — the losing transaction can only ever observe a
+COMMITTED row, and this module only ever commits a row once it is
+`completed`. They are kept as defined answers (not assumptions) for a claim
+row that reached that state some other way, and are exercised in
+`src/__tests__/issue-3080-run-once.test.ts` by seeding the table directly and
+by unit-testing the pure resolver (`resolveExistingRunOnceClaim`).
+
+## Typed answers: refused vs. unknown outcome (mirrors #2990)
+
+A conflict on insert is a *definite* decision — the row exists — but whether
+it is safe to hand back a result depends on `status`:
+
+| Row state after conflict | Result |
+|---|---|
+| `status: 'completed'` | Replay the stored `result`. No error. |
+| exists, not completed | `RunOnceClaimError.inFlight()` — `code: 'RUN_ONCE_IN_FLIGHT'`. Some execution holds this key right now; retry later with the SAME token and content. |
+| conflict reported, no row readable | `RunOnceClaimError.outcomeUnknown()` — `code: 'RUN_ONCE_OUTCOME_UNKNOWN'`. Genuinely indeterminate; retrying with the SAME token and content is still safe. |
+
+Both error codes live on one `RunOnceClaimError` class (`src/errors.ts`),
+following this codebase's existing error-hierarchy convention (one class,
+multiple codes via static factories — see `TenantIsolationError`,
+`DatabaseError`) rather than a bespoke class hierarchy. Neither is retried
+automatically by `runOnce()` itself: only the caller knows whether retrying
+inline is appropriate for its own request.
+
+## Retention: completed claims are pruned; the window is a replay deadline
+
+`_smrt_run_once_claims` is append-only in normal operation — every submission
+adds a row and nothing else ever deletes one — so, like `_smrt_changes`,
+`_smrt_ai_usage`, and `_smrt_dispatch`, it is wired into
+`system/retention.ts`'s `runRetentionSweep()` as a built-in task
+(`'run-once-claims'`), covered by `DEFAULT_RETENTION_POLICY.runOnceClaims`
+(default `{ maxAgeDays: 30 }`, overridable per policy or disabled with
+`runOnceClaims: false`). `pruneRunOnceClaims()` deletes only rows with
+`status = 'completed'` AND `completed_at` older than the window; an
+`'in_progress'` row has no `completed_at` and can never match this predicate,
+so a claim held by a live (or abandoned) transaction is never pruned by age
+alone. The predicate is covered by
+`idx_smrt_run_once_claims_status_completed` (`status, completed_at`), added
+in `SMRT_SCHEMA_VERSION` 1.11.1.
+
+**The window is a replay deadline, not just disk hygiene.** `runOnce()`'s
+entire contract — a retry or double-submit with the SAME token and content
+replays the first attempt's stored result instead of running `work()` again
+— depends on the claim row still existing. Once retention sweeps a completed
+claim, a client that replays that exact request (same `tenantId`, `actor`,
+`token`, and content) no longer collides with anything: `runOnce()` sees no
+existing row, wins the insert, and runs `work()` a second time. This is
+silent — there is no error, just a second execution of whatever `work()`
+does. The default 30-day window is chosen to be comfortably longer than any
+realistic client retry window (a browser tab left open, a queued mobile
+request, a webhook redelivery schedule) for the same reason the other
+default windows in `DEFAULT_RETENTION_POLICY` are generous relative to their
+consumers (see `system/retention.ts`'s module doc). A consumer with a
+longer-lived retry/replay window (e.g. an at-least-once delivery queue with a
+multi-week redelivery tail) must raise `runOnceClaims.maxAgeDays` to exceed
+it, not rely on the default.
+
+Tests: `src/__tests__/issue-2375-retention.test.ts` covers
+`pruneRunOnceClaims()` end-to-end through `runRetentionSweep()` — an old
+completed claim is removed, a recent completed claim and an old
+`in_progress` claim both survive.
+
+## Tests
+
+`src/__tests__/issue-3080-run-once.test.ts` (real in-memory SQLite via
+`getTestDatabase({ classes: [] })` — no `@smrt()` model is needed, only the
+system table):
+
+- Runs work once; returns its result.
+- Retry after completion replays the stored result without re-running.
+- Concurrent double submit (`Promise.all`) produces one claim row and the
+  same result for both callers.
+- Same token, different content → two separate claims (guards the
+  token-alone regression this issue was filed against).
+- Same token/content, different actor → separate claim.
+- Same token/content, different tenant → separate claim.
+- Work throws → claim rolls back; a subsequent retry with the same key runs.
+- A claim seeded directly as `in_progress` → `RunOnceClaimError.inFlight()`,
+  `work()` never called.
+- Content-digest stability (key order) and sensitivity (value change) are
+  unit-tested directly.
+- `resolveExistingRunOnceClaim` (pure resolver) is unit-tested for all three
+  branches, including the otherwise-unreachable `outcomeUnknown` case.
+
+`src/__tests__/issue-3080-run-once-postgres.optional.test.ts` (real
+PostgreSQL, `SMRT_TEST_POSTGRES_URL`, `describe.skipIf`, following
+`change-feed-concurrency.optional.test.ts`'s established pattern: distinct
+`dbid`s for two genuinely concurrent connections, `getTestDatabase({ db,
+classes: [] })` to bootstrap system tables): replay after completion, and
+concurrent double submit across the two real connections — both would
+reject with a PostgreSQL `DatabaseError` ("Failed to retrieve record from
+table", the driver's wrapping of 25P02 "current transaction is aborted")
+against the pre-fix plain-`INSERT`-and-catch version; verified locally
+against a disposable `postgres:18-alpine` container (revert via patch file,
+confirmed both tests red, restored, confirmed both green) since the SQLite
+suite cannot reach this PostgreSQL-specific failure mode at all.

package/agents/schema-paths.md

@@ -737,15 +737,19 @@ the component is uniformly UUID and the planner emits nothing.
 
 ### Retention (`src/system/retention.ts`)
 
-`runRetentionSweep(db, policy)` runs four built-ins in fixed order, then
+`runRetentionSweep(db, policy)` runs five built-ins in fixed order (`changes`,
+`ai-usage`, `contexts`, `dispatch`, `run-once-claims`), then
 `registerRetentionTask()` contributions. A failed task records its result and
 continues; a missing table is `unavailable`. The `globalThis` registry avoids
 split registrations under duplicate core resolution; package tasks exist only
 after importing the package. CLI prune optionally imports jobs/users.
 
 Defaults are opt-out: changes 30 days, AI usage 90 days, completed dispatch 30
-days/failed dispatch 90 days, contexts by `expires_at`. `smrt.configure({
-retention })` tunes built-ins; contributed tasks own their defaults/options.
+days/failed dispatch 90 days, contexts by `expires_at`, run-once claims 30
+days on `completed_at` (#3080; `in_progress` rows never match — the window is
+a replay deadline for `runOnce()`, not just disk hygiene, see
+`agents/run-once.md`). `smrt.configure({ retention })` tunes built-ins;
+contributed tasks own their defaults/options.
 Jobs defaults are 7 days terminal, 30 failed, 30 events via
 `registerJobRetentionTasks()` or runner `retention.jobs`; expired credentials
 have no extra window. Disable a table/task with `false` or the whole policy with
@@ -760,8 +764,9 @@ approximate under concurrency. Overlapping change/AI-usage bounds exclude rows
 already counted, including dry runs.
 
 Retention indexes belong in system DDL and its versioned replay:
-`_smrt_contexts(expires_at)`, `_smrt_ai_usage(tenant_id, created_at)`, and dispatch
-`(status, processed_at)` / `(status, updated_at)`. Jobs `(status, completed_at)`
+`_smrt_contexts(expires_at)`, `_smrt_ai_usage(tenant_id, created_at)`, dispatch
+`(status, processed_at)` / `(status, updated_at)`, and
+`_smrt_run_once_claims(status, completed_at)`. Jobs `(status, completed_at)`
 belongs in `ensureJobsSystemTableCompatibility()` on each collection initialize,
 since decorated jobs tables do not exist at bootstrap.
 

package/dist/browser.js

@@ -4,7 +4,8 @@ import { MetricsAdapter } from "./adapters/metrics.js";
 import { PubSubAdapter } from "./adapters/pubsub.js";
 import "./adapters/index.js";
 import { classifyDatabaseError, classifyDialectMessage, isAbortedTransactionError, isDeterministicDatabaseError, isNotNullViolationError, isTransientDatabaseError, isUniqueViolationError } from "./db-errors.js";
-import { AIError, ConfigurationError, DatabaseError, ErrorUtils, FilesystemError, NetworkError, RuntimeError, SmrtError, TenantIsolationError, ValidationError, ValidationReport, ValidationUtils } from "./errors.js";
+import { AIError, ConfigurationError, DatabaseError, ErrorUtils, FilesystemError, NetworkError, RunOnceClaimError, RuntimeError, SmrtError, TenantIsolationError, ValidationError, ValidationReport, ValidationUtils } from "./errors.js";
+import { SMRT_SCHEMA_VERSION } from "./system/schema.js";
 import { config } from "./config.js";
 import { SignalSanitizer } from "./signals/sanitizer.js";
 import { SignalBus } from "./signals/bus.js";
@@ -24,7 +25,7 @@ import { SmrtPolymorphicAssociation } from "./polymorphic-association.js";
 import "./signals/index.js";
 import { DIAGNOSTICS_DEFAULT_LIMIT, SYSTEM_DIAGNOSTICS_TABLES, readDispatchHealth, readJobHealth, readMigrationStatus, readRecentChanges, readRegistryDrift, readScheduleHealth, readSystemDiagnostics } from "./system/diagnostics.js";
 import { BOOTED_PROVENANCE, assertPlainJson, sanitizeMessagePaths, snapshotRegistry } from "./system/registry-snapshot.js";
-import { DEFAULT_RETENTION_POLICY, clearRetentionTasks, getRetentionTasks, pruneAiUsage, pruneExpiredContexts, registerRetentionTask, runRetentionSweep, unregisterRetentionTask } from "./system/retention.js";
+import { DEFAULT_RETENTION_POLICY, clearRetentionTasks, getRetentionTasks, pruneAiUsage, pruneExpiredContexts, pruneRunOnceClaims, registerRetentionTask, runRetentionSweep, unregisterRetentionTask } from "./system/retention.js";
 import "./system/index.js";
 import "./tools/index.js";
-export { AIError, AiUsageCollector, AiUsagePersistenceHandler, BOOTED_PROVENANCE, ConfigurationError, DEFAULT_AI_COST_RATES, DEFAULT_FACET_LIMIT, DEFAULT_RETENTION_POLICY, DIAGNOSTICS_DEFAULT_LIMIT, DatabaseError, EmbeddingUnavailableError, ErrorUtils, FilesystemError, MAX_FACET_FIELDS, MAX_FACET_LIMIT, MAX_STI_READ_SCOPE_TYPES, MetricsAdapter, NetworkError, ObjectRegistry, PubSubAdapter, RuntimeError, SMRT_COLLECTION_BASE_NAMES, SYSTEM_DIAGNOSTICS_TABLES, SignalBus, SignalSanitizer, SmrtClass, SmrtCollection, SmrtError, SmrtHierarchical, SmrtJunction, SmrtObject, SmrtPolymorphicAssociation, TenantIsolationError, ValidationError, ValidationReport, ValidationUtils, assertPlainJson, assertPostgresSystemTimestampsCurrent, classifyDatabaseError, classifyDialectMessage, clearRetentionTasks, config, convertTypeToJsonSchema, ensureBootstrapSystemTableCompatibility, ensureDeferredSystemTableCompatibility, ensureDispatchSubscriptionsSystemTableCompatibility, ensureDispatchSystemTableCompatibility, ensureJobEventsSystemTableCompatibility, ensureJobsSystemTableCompatibility, ensureLegacySystemTableCompatibility, ensureSystemTables, estimateAiUsageCost, executeToolCall, executeToolCalls, formatToolResults, generateToolFromMethod, generateToolManifest, getDatabaseEngine, getRetentionTasks, isAbortedTransactionError, isDeterministicDatabaseError, isFrameworkBaseClass, isNotNullViolationError, isSmrtCollectionExtendsName, isTransientDatabaseError, isUniqueViolationError, migratePostgresSystemTimestamps, planPostgresSystemTimestampMigrations, pruneAiUsage, pruneExpiredContexts, readDispatchHealth, readJobHealth, readMigrationStatus, readRecentChanges, readRegistryDrift, readScheduleHealth, readSystemDiagnostics, registerRetentionTask, runRetentionSweep, sanitizeMessagePaths, shouldIncludeMethod, smrt, smrt as smrtRegistry, snapshotRegistry, tableExists, unregisterRetentionTask, validateToolCall };
+export { AIError, AiUsageCollector, AiUsagePersistenceHandler, BOOTED_PROVENANCE, ConfigurationError, DEFAULT_AI_COST_RATES, DEFAULT_FACET_LIMIT, DEFAULT_RETENTION_POLICY, DIAGNOSTICS_DEFAULT_LIMIT, DatabaseError, EmbeddingUnavailableError, ErrorUtils, FilesystemError, MAX_FACET_FIELDS, MAX_FACET_LIMIT, MAX_STI_READ_SCOPE_TYPES, MetricsAdapter, NetworkError, ObjectRegistry, PubSubAdapter, RunOnceClaimError, RuntimeError, SMRT_COLLECTION_BASE_NAMES, SMRT_SCHEMA_VERSION, SYSTEM_DIAGNOSTICS_TABLES, SignalBus, SignalSanitizer, SmrtClass, SmrtCollection, SmrtError, SmrtHierarchical, SmrtJunction, SmrtObject, SmrtPolymorphicAssociation, TenantIsolationError, ValidationError, ValidationReport, ValidationUtils, assertPlainJson, assertPostgresSystemTimestampsCurrent, classifyDatabaseError, classifyDialectMessage, clearRetentionTasks, config, convertTypeToJsonSchema, ensureBootstrapSystemTableCompatibility, ensureDeferredSystemTableCompatibility, ensureDispatchSubscriptionsSystemTableCompatibility, ensureDispatchSystemTableCompatibility, ensureJobEventsSystemTableCompatibility, ensureJobsSystemTableCompatibility, ensureLegacySystemTableCompatibility, ensureSystemTables, estimateAiUsageCost, executeToolCall, executeToolCalls, formatToolResults, generateToolFromMethod, generateToolManifest, getDatabaseEngine, getRetentionTasks, isAbortedTransactionError, isDeterministicDatabaseError, isFrameworkBaseClass, isNotNullViolationError, isSmrtCollectionExtendsName, isTransientDatabaseError, isUniqueViolationError, migratePostgresSystemTimestamps, planPostgresSystemTimestampMigrations, pruneAiUsage, pruneExpiredContexts, pruneRunOnceClaims, readDispatchHealth, readJobHealth, readMigrationStatus, readRecentChanges, readRegistryDrift, readScheduleHealth, readSystemDiagnostics, registerRetentionTask, runRetentionSweep, sanitizeMessagePaths, shouldIncludeMethod, smrt, smrt as smrtRegistry, snapshotRegistry, tableExists, unregisterRetentionTask, validateToolCall };