@happyvertical/smrt-core 0.45.1 → 0.45.2

package/agents/schema-paths.md

@@ -4,16 +4,11 @@ Module semantics for `src/schema/` — which `SchemaGenerator` entry point reach
 a real database, what each one emits, and the rules that keep them in step.
 Package orientation, the cross-module invariants, and the traps that apply
 before editing anything live in [../AGENTS.md](../AGENTS.md) — read that first;
-its "Schema paths" section is the short form of everything below.
-
-Written from the 2026-08-17 database-layer gap assessment (epic #2382). Symbol
-names here are stable; the line numbers the assessment quotes are not, so trust
-this call graph and re-grep before citing a location.
+it links the relevant runtime and generation contracts.
 
 ## Four entry points, two of which ship
 
-`src/schema/generator.ts` exposes four index-emitting entry points. They do not
-produce the same schema for the same class.
+`src/schema/generator.ts` exposes four index-emitting entry points. Their columns and indexes must agree for the same class.
 
 | Entry point | Selected by | Status |
 |---|---|---|
@@ -22,15 +17,6 @@ produce the same schema for the same class.
 | `generateSTISchemaFromRegistry` | `src/testing/database.ts` (`getTestDatabase()`), `src/schema/utils.ts` (`generateSchema`; `ensureSchema` only as a fallback) | tests + runtime helpers |
 | `generateSchemaFromRegistry` | the same two callers | tests + runtime helpers |
 
-A fifth entry point, the build-time AST `generateSchema(objectDef)`, existed
-until #2380: it fed only the `smrt:schema` virtual module, which had no
-consumer, had rotted relative to the four paths above (an `idx_`-prefixed
-naming scheme none of the others use, and no conflict-index emission at all),
-and was deleted rather than wired up. `SchemaOverrideSystem`
-(`schema/override-system.ts`) — unwired, and its two non-generic methods
-hard-coded a schema extension for a project outside this monorepo — was
-deleted alongside it. See rule 9 and the new rule at the end of this file.
-
 Production DDL takes the manifest route:
 
 ```
@@ -43,14 +29,6 @@ Production DDL takes the manifest route:
                         use; no in-repo caller outside its own tests)
 ```
 
-The suite takes the registry route. Before #2359 the registry route emitted
-indexes the manifest route did not — per-column foreign-key indexes, and STI
-partial FK indexes filtered by `_meta_type` — so tests ran against a richer
-schema than any deployment received, the manifest STI path populated a
-`fkColumnsByClass` map it never read, and the manifest CTI path had no FK loop
-at all. `src/testing/database.ts`'s "same as migrations" comment described an
-intent, not the code.
-
 Since #2359 the two families share one set of index helpers and
 `src/schema/schema-path-parity.test.ts` runs the same fixture manifest through
 the manifest paths, through `ObjectRegistry.registerFromManifest()` + the
@@ -85,66 +63,40 @@ divergence is a bug in the generator, not an exception to add to the test.
   row a slug resolves to). The tenant-led default key below counts as serving
   it (`servesSlugLookup()`): a tenant-scoped slug lookup carries the tenant
   predicate (#2365) and is served by the prefix, so no second index.
-- **Tenant-scoped tables key per tenant (#2360).** A tenant-scoped class with
-  no explicit `conflictColumns` upserts on, and indexes,
-  `(tenant_id, slug, context)` — `(tenant_id, slug, context, _meta_type)` for
-  an STI hierarchy — resolved by one rule on both paths:
-  `ManifestGenerator.normalizeConflictColumns()` materializes it into
-  `decoratorConfig.conflictColumns` for the manifest paths (so the manifest,
-  the schema, `smrt-knowledge.json` and the runtime read one value), and
-  `ObjectRegistry.getConflictColumns()` derives the same value at runtime from
-  the schema owner's `tenantScoped` config (`ObjectRegistry.getTenantColumn()`;
-  an STI child resolves through its root; a `@report` class through its
-  group/bucket columns; a custom primary key through that key). Explicit
-  `conflictColumns` are never rewritten. `src/schema/conflict-target.ts` holds
-  the shared helpers. Consequences: the index NAME stays
-  `<table>_slug_context_idx` / `_slug_context_meta_type_idx`, so the differ
-  swaps the columns of an existing global unique in place by name (a superset
-  key — creating it cannot fail on existing rows); the tenant-led key also
-  serves the tenant column, so `<table>_tenant_id_idx` is no longer emitted
-  for those tables (an existing one is an orphan the differ drops only with
-  `--drop-indexes`); NULL-tenant rows (`mode: 'optional'` outside a tenant
-  context) dedup among themselves through the SDK's null-aware upsert
-  (`IS NOT DISTINCT FROM` under a PostgreSQL advisory lock / an in-process
-  lock on SQLite) — application-enforced now, where the old global index was
-  database-enforced: the tenant-led index treats NULLs as distinct, so raw SQL
-  can insert two global rows with one slug, and a raw
-  `ON CONFLICT (slug, context…)` against such a table no longer binds (use
-  `WHERE NOT EXISTS`, plus an advisory lock on PostgreSQL). Emitting
-  `NULLS NOT DISTINCT` on PostgreSQL ≥ 15 (the SDK already detects it) would
-  restore the database arbiter — a follow-up. The `save()` path serializes an
-  unset tenant field as an explicit `NULL` whatever its registered type,
-  because the SDK rejects an upsert whose conflict column is missing from the
-  row.
-- **Rolling the tenant-led key out (#2360).** There is no mixed-version state:
-  new code against the old index fails every NEW-object create on a
-  tenant-scoped default-key table (PostgreSQL 42P10, SQLite "ON CONFLICT
-  clause does not match…"), and old code against the new index fails the same
-  way, because the conflict target must match the unique index's column set
-  exactly; only persisted objects (upsert on `id`) keep saving. Deploy the code
-  and run `smrt db:migrate` in the same maintenance step. The plan is one
-  `DROP INDEX` + `CREATE UNIQUE INDEX` per table under the SAME name (a
-  superset key, so the build cannot fail when the old same-name index was a
-  valid UNIQUE over the subset key; a #1165-class table whose old index was
-  non-unique or missing may hold duplicates that a superset UNIQUE rejects —
-  `db:diff` shows which tables' old index is non-unique or missing; dedupe
-  those rows before migrating). Atomic mode swaps every table in one
-  transaction: `DROP INDEX` takes ACCESS EXCLUSIVE and holds it until commit,
-  which blocks ALL access to those tables — reads included — for the batch;
-  size `statementTimeout` for the largest tenant-scoped table. That is the
-  maintenance window this rollout requires anyway (no mixed-version state), so
-  run this wave — the #2359 index wave included — in atomic mode inside it;
-  the "roll out with `--postgres-safe`" advice above applies to a #2359-only
-  wave, because `--postgres-safe` runs the two statements sequentially per
-  table, so each table has NO conflict index between them and a failed rebuild
-  leaves it without one until the re-run. The recreate has no automatic
-  DOWN: reverting the code means re-creating the old index by hand. And
-  legacy NULL-tenant rows fork rather than get adopted — a tenant-context save
-  whose slug matches a `(NULL, slug, ctx)` row now inserts `(tenant, slug,
-  ctx)` beside it, and that tenant no longer sees the legacy row — so backfill
-  `tenant_id` (anytown: `SET tenant_id = context::uuid`) BEFORE this release.
-  Ingestion that relied on natural-key dedup across tenants now inserts one
-  row per tenant (release note).
+- **Tenant default keys** are `(tenant_id, slug, context)`, plus `_meta_type`
+  for STI. `ManifestGenerator.normalizeConflictColumns()` and
+  `ObjectRegistry.getConflictColumns()` share `src/schema/conflict-target.ts`:
+  resolve tenant fields through the schema owner/STI root, report group/bucket
+  columns through the report, and custom PKs through their key. Explicit
+  `conflictColumns` remain unchanged. The manifest, schema, knowledge, and
+  runtime must carry the same value.
+  Names remain `<table>_slug_context_idx` / `_slug_context_meta_type_idx`, so
+  migration replaces a same-name global unique with tenant-led columns. That
+  prefix serves tenant and tenant-scoped slug reads; a legacy standalone tenant
+  index is dropped only with `--drop-indexes`.
+- **Optional NULL tenants** dedup through SDK null-aware upsert (PostgreSQL
+  `IS NOT DISTINCT FROM` plus advisory lock; SQLite process lock), not the
+  unique index: raw SQL can duplicate NULL-tenant keys. Raw global inserts need
+  `WHERE NOT EXISTS` and a PostgreSQL advisory lock; an old global `ON CONFLICT`
+  target no longer binds. Save serializes an unset tenant explicitly as NULL,
+  because every conflict column must be present. PostgreSQL `NULLS NOT DISTINCT`
+  remains a potential follow-up, not current enforcement.
+- **Tenant-key rollout requires a maintenance window.** Old code/new indexes
+  and new code/old indexes both fail new-object saves because conflict column
+  sets must match exactly; persisted ID-based saves still work. Backfill legacy
+  NULL tenants first or scoped ingestion creates separate rows and cannot see
+  the old global ones. Cross-tenant natural-key dedup now creates one row per
+  tenant. Deploy code and migrate together in atomic mode: each table drops
+  and recreates its same-name unique index, holding ACCESS EXCLUSIVE locks
+  (including against reads) until commit. Size `statementTimeout` for the
+  largest table. A valid old subset unique guarantees the superset build;
+  missing/nonunique old indexes may contain duplicates and need dedup first.
+  Include the reference-index wave in that atomic window. `--postgres-safe` is
+  suitable for an additive reference-index-only wave, but a key replacement
+  leaves a per-table gap between drop/build and a failed build leaves no arbiter
+  until rerun. There is no automatic DOWN; reverting code requires deliberately
+  recreating its old indexes.
+
 - **STI `@field({ unique: true })` is enforced through indexes** (the differ can
   add an index to an existing table, never a column constraint): a full
   `<table>_<col>_unique_idx` when the STI base declares it, one
@@ -172,138 +124,31 @@ divergence is a bug in the generator, not an exception to add to the test.
   `getAllSchemasAsDefinitions()` table definition, and only falls back to
   `generateSchema()` when no schema is registered at all.
 
-So a normal build keeps the manifest schema through `db:setup`, and a
-registry-derived schema is a dev/test artifact. `smrt-content` shows what one
-looks like: `packages/content/src/hooks.server.ts` `bootstrapSchema()` calls
-`generateSchema()` for every registered class and then `ensureSchema()` from the
-SvelteKit `handle` hook on any `/api/*` request, so that process holds
-registry-derived schemas rather than the manifest ones. It reaches only that
-package's own `vite dev` app — the library build excludes the file and the
-package never exports it — but it is the shape to recognize. Check which route a
-process actually took before trusting a reproduction.
-
-## Why the drift stayed invisible
-
-Every drift oracle compares a database with the same artifact that dropped the
-index:
-
-- `verifyPersistenceTable()` (`src/schema/table-verifier.ts`) calls
-  `db.tableExists()` and nothing else. "Runtime verifies schema" has always meant
-  existence-only — no column, type, constraint, or index comparison.
-- `smrt doctor` never opens a database connection.
-- `db:status` and `db:diff` diff the live database against
-  `getAllSchemasAsDefinitions()`, i.e. the manifest projection.
-
-An index the manifest never emitted is "in sync" by construction. That is how a
-production database reached 164 unindexed `tenant_id` columns while `db:status`
-reported no drift (#2356 → #2359). The assessment's other counts — 196/231
-`@foreignKey` and 91/92 `@crossPackageRef` columns with no production index,
-238/238 tables carrying a redundant index on the primary key, zero DB-level
-foreign-key constraints on any engine — come from regenerating every package's
-schema against a live database, so re-measure rather than quote them once the
-epic's fixes land.
-
-## Rules
-
-### 1. Verify against the production path, not the test path
-
-Any change to column or index emission goes on **all** paths that ship and is
-proven by the path-parity test (`src/schema/schema-path-parity.test.ts`, #2359)
-— extend its fixture; a green suite otherwise proves the registry paths only.
-Read the call graph before believing a comment: "same as migrations" was wrong
-for years.
-
-### 2. Every new query predicate ships with its index
-
-Collection methods, poll loops, auth lookups, junction right-side filters, and
-polymorphic owner lookups all count — or write down why the predicate does not
-need one. For list workloads, EXPLAIN on a PostgreSQL snapshot; the measured
-spread on the assessed workload was 21 ms → 0.1 ms.
-
-### 3. Run the PostgreSQL lane
-
-Anything touching numeric types, uuid casts, upsert conflict targets, timestamps,
-or migrations runs the package's `test:postgres` script:
-
-```bash
-pnpm --filter @happyvertical/smrt-<pkg> test:postgres
-```
-
-core, cli, users, sales, marketing, analytics, and vitest carry the lane.
-SQLite's type affinity accepts values PostgreSQL rejects — a money field declared
-`number = 0` compiles to INTEGER and only fails on PG (#2361).
-
-### 4. Read the built artifact, not the source
-
-What a decorator produced is in `dist/manifest.json` and in regenerated schemas:
-`integer` vs `decimal`, the actual index list, the actual conflict columns. When
-the question is "how many tables/columns/indexes", regenerate and count across
-every package; do not sample a few and extrapolate.
-
-### 5. Index intent belongs on both the constraint and the read path
-
-A conflict target is not automatically a unique index, and a unique index is not
-automatically the index a read path uses. Custom `conflictColumns` used to
-replace the `(slug, context)` index while `loadFromSlug`/`getId` still queried
-slug+context, and STI dropped `@field({ unique: true })` — both fixed in #2359,
-see "Index rules" above. Check the pair, not the declaration.
-
-### 6. Multi-tenancy is a whole-path property
-
-Every unique constraint and every conflict target on a tenant-scoped table
-includes the tenant column — otherwise a second tenant's `save()` of the same
-natural key updates the first tenant's row through `DO UPDATE SET` (#2360; the
-default key now does, see "Index rules" — an explicit `conflictColumns` that
-omits the tenant column is the class author's own key and is not rewritten).
-And every read path is interceptor-aware: hydration
-(`loadFromId`/`loadFromSlug`), get-by-slug, vector search, and collection
-memory, not only `list()` (#2365).
-
-### 7. Retry only transient errors
-
-Classify through the cause chain (SQLSTATE), never on a message substring, and
-never retry inside an aborted PostgreSQL transaction (`25P02`). Test the
-contract end to end against a real database, not only the classifier (#2366).
+## Verification
 
-### 8. Thread new decorator options through every config-rebuild site
+Extend `src/schema/schema-path-parity.test.ts` for every generator change;
+manifest, registry, and merged migration schemas must agree. Inspect regenerated
+`dist/manifest.json` and schemas across affected packages, not only decorators.
+Runtime `verifyPersistenceTable()` checks table existence only. Database drift
+checks compare with generated artifacts; they cannot detect an omission shared
+by those artifacts. Use `smrt doctor --db` / `db:status --parity` for live parity.
 
-A new `@smrt()` or `@field()` option that affects schema must reach the
-`SchemaGeneratorConfig` type in `src/schema/generator.ts` and every site that
-rebuilds that config — `src/schema/utils.ts` and `src/testing/database.ts` — or
-it is silently dropped on the paths that rebuild it (#2357).
+Every new query predicate needs its index or an explicit reason none is needed.
+Run `pnpm --filter @happyvertical/smrt-core test:postgres` for numeric types,
+UUID casts, conflict targets, timestamps, or migrations. Schema-affecting options
+must reach `SchemaGeneratorConfig` and both config rebuild sites:
+`src/schema/utils.ts` and `src/testing/database.ts`.
 
-### 9. Delete or wire dead paths, and write docs to what the code does
+Tenant uniqueness and conflict targets must include the tenant column; explicit
+`conflictColumns` are author-owned and never rewritten. All reads, including
+hydration, slug lookup, vector search, and memory, remain interceptor-aware.
+Retry only transient errors classified through the cause chain; never retry an
+aborted PostgreSQL transaction (`25P02`).
 
-Dead code that looks canonical misleads the next agent: the AST `generateSchema`
-path, `SchemaOverrideSystem`, and the never-emitted `triggers: []` all read as
-supported surfaces (#2380). Documentation follows the implementation, not the
-intent — say "verifies the table exists" when that is what runs.
-
-### 10. Untracked "known limitation" comments are bugs nobody will read
-
-File the issue and link it from the comment. A `products` comment explaining why
-a conflict-column change was refrained from sat there for months — and
-misdescribed the failure mode the whole time.
-
-### 11. Consumer repair scripts are signals
-
-Downstream repair tooling (anytown's `db-repair-plan.ts` carried column-type
-repairs, missing STI columns and indexes, and `tenant_id` backfills since April)
-is the consumer-side record of framework gaps. Mine it during triage.
-
-### 12. Try to falsify before filing, and treat operations as correctness
-
-Re-verify a finding at source before it becomes an issue — one assessment
-candidate claimed conflict indexes past two columns were narrowed to two
-columns, when only the index *name* is shortened. And an index fix that ships
-without a bounded-timeout, `CONCURRENTLY`-capable migrate path can take
-production down on rollout (#2362).
-
-### 13. Composite indexes are declared, not inferred (#2357)
+### Composite indexes are declared, not inferred (#2357)
 
 The generated set only covers foreign keys, unique/conflict columns, the STI
-discriminator, reference columns (#2359), the default list ordering (rule 18
-below), and single columns opted in with `@field({ indexed: true })`. A list
+discriminator, reference columns (#2359), default list ordering, and single columns opted in with `@field({ indexed: true })`. A list
 workload's access path is composite, so declare it:
 
 ```ts
@@ -322,14 +167,14 @@ scans a btree either way, so an ascending index also serves the matching
 (partial index) are honoured.
 
 `appendDeclaredIndexes()` runs first on all four entry points, ahead of
-`ensureDefaultListOrderingIndex()` (rule 18) and `ensureReferenceColumnIndexes()`,
+`ensureDefaultListOrderingIndex()` (default ordering below) and `ensureReferenceColumnIndexes()`,
 so a declared composite leading with the tenant column (or any reference column)
 replaces the automatic standalone index rather than duplicating it.
 Unknown columns, malformed entries, and a name collision with a different index
 all fail generation — a silently dropped index only surfaces later as a
-production slowdown. Rule 8 above is why this works at runtime at all.
+production slowdown. Keep both config rebuild sites aligned.
 
-### 14. Relationship targets resolve to a class name on both paths
+### Relationship targets resolve to a class name on both paths
 
 `@foreignKey`/`@oneToMany`/`@manyToMany` accept a class, a name string, or a
 `() => Target` thunk. The decorator invokes the thunk and throws when the target
@@ -339,7 +184,7 @@ costs the relationship edge, `loadRelated()`, and the FK-derived index (#2379).
 A thunk resolves at decoration time, so a target declared later in the same
 module is still in its temporal dead zone — use the string form there.
 
-### 15. A SQLite type change is a table rebuild (#2370)
+### A SQLite type change is a table rebuild (#2370)
 
 SQLite has no `ALTER TABLE ... ALTER COLUMN ... TYPE`, so
 `src/migrations/sqlite-rebuild.ts` answers a `type_upgrade` on SQLite with the
@@ -417,7 +262,7 @@ and always reports what it will not touch:
   on a populated one it is added nullable and the `NOT NULL` is reported as a
   manual follow-up on every engine.
 - **SQLite** has no `ALTER COLUMN`: nullability/default alterations are manual
-  (comment SQL → `db:migrate` exit 1). The #2370 rebuild (rule 15) consumes
+  (comment SQL → `db:migrate` exit 1). The SQLite rebuild consumes
   only `type_upgrade` placeholders today; extending it to rewrite constraints
   would lift this.
 - Defaults compare through `canonicalizeDefault()`, which folds engine
@@ -427,7 +272,7 @@ and always reports what it will not touch:
   round-trip test (create from each DDL strategy → compare → zero changes) in
   `src/migrations/__tests__/issue-2369-*.test.ts` guards this.
 
-### 16. `schema.ddl` is a preview, not the table
+### `schema.ddl` is a preview, not the table
 
 `SchemaDefinition.ddl` / `manifest.json` `schema.ddl` is the engine-neutral
 CREATE TABLE string from `SchemaGenerator.generateSQL()` with no engine: no
@@ -446,7 +291,7 @@ string, and do not write a private CREATE INDEX renderer — the retired ones
 dropped `where` and `jsonPath` (#2358). Every DDL strategy also spells out
 `PRIMARY KEY NOT NULL`: SQLite lets a bare non-INTEGER PRIMARY KEY hold NULL.
 
-### 17. The merged table shape is registration-order independent (#2372)
+### The merged table shape is registration-order independent (#2372)
 
 `getAllSchemas()` and `getAllSchemasAsDefinitions()` fold every class that
 shares a physical table — the whole STI hierarchy — into one shape. Both route
@@ -454,16 +299,9 @@ through `buildMergedTableSchemas()`, which groups contributors by table and
 then merges them in a **deterministic** order: the STI base first, then
 ancestors before descendants, then by qualified name.
 
-That order matters because the first contributor seeds the table: it supplies
-the fallback base columns, the `idType`, the conflict columns and the cached
-DDL, and its columns win every merge conflict. When registration order decided
-it, an STI child that carries no manifest `schema` — the external- and
-consumer-manifest case — seeded the table from bare fallback columns and the
-base class's richer ones were skipped when it registered later, yielding
-`context TEXT` instead of `context TEXT NOT NULL DEFAULT ''` and timestamps
-with no NOT NULL/DEFAULT. The shipped content manifest lists `Article` before
-`Content`, so the losing order was the one that shipped, and the differ
-compares types only, so the weak fresh-create was never repaired.
+The first contributor supplies fallback columns, `idType`, conflict columns,
+cached DDL, and wins column conflicts. Keep base-first ordering even when a
+child without manifest schema registers first.
 
 Two invariants keep the two assembly paths agreeing:
 
@@ -488,49 +326,26 @@ When adding a class-level input to the merged shape, take it from the seeding
 contributor rather than "whichever class arrives first", and cover it with a
 child-first/base-first equality test.
 
-### 18. The generator owns the index for its own default ordering (#2363)
-
-Every generated list surface — REST, MCP, the SvelteKit list route — pages with
-`ORDER BY created_at DESC, <pk> ASC` (`DEFAULT_LIST_ORDER_BY`, #2367), and
-until #2363 no schema path indexed `created_at` (the AST path, deleted in
-#2380, indexed `updated_at`), so the framework's own default page was a
-sequential scan plus a top-N sort. `ensureDefaultListOrderingIndex()` now runs
-on all four entry points and emits:
-
-- `(<tenant column>, created_at)` on a tenant-scoped table — the tenancy
-  interceptor puts `tenant_id = ?` in front of every list, so the tenant column
-  leads and `created_at` orders within it. This composite **replaces** the
-  standalone tenant index from #2359: a B-tree serves every prefix of its
-  column list, so `ensureDefaultListOrderingIndex()` is called first and
-  `ensureReferenceColumnIndexes()` then sees the column as already served. The
-  tenant column is found by `referenceKind === 'tenantId'`, never by the
-  `tenant_id` spelling — `@smrt({ tenantScoped: { field } })` renames it.
-- `(created_at)` otherwise.
-
-Three deliberate omissions, so nobody "fixes" them later:
-
-- **No `DESC`.** `IndexDefinition` carries no per-column direction and
-  PostgreSQL scans a B-tree backwards just as cheaply.
-- **No primary-key tiebreak column.** The default order mixes directions
-  (`created_at DESC, id ASC`), so no single-direction index satisfies the whole
-  key; the leading columns already turn a full sort into an index scan plus an
-  incremental sort over rows sharing a timestamp.
-- **Not scoped per STI subtype.** `(_meta_type, created_at)` would serve a
-  child collection's list but not the base class's polymorphic one, which
-  carries no discriminator predicate — the same reasoning that keeps STI
-  reference indexes plain (#2359). One unqualified index per shared table.
-
-An existing UNQUALIFIED index that already leads with the same columns
-suppresses it — a partial or JSON-path index never counts. That is how a
-declared `@smrt({ indexes: [...] })` composite (#2357) takes over: declaring
-`(tenant_id, created_at, status)` replaces the generated pair, while declaring
-a different sort column such as `(tenant_id, publish_date)` sits **beside** it,
-because that index cannot order the default page. Declared indexes are appended
-before this helper for exactly that reason; anything that appends an index in
-future goes in the same slot, ahead of `ensureDefaultListOrderingIndex()` and
-`ensureReferenceColumnIndexes()`.
-
-### 19. One conflict-target rule, applied on every producer
+### Default list ordering indexes
+
+All four generators index `DEFAULT_LIST_ORDER_BY` (`created_at DESC, <pk> ASC`):
+`ensureDefaultListOrderingIndex()` emits `(tenant column, created_at)` when
+scoped, otherwise `(created_at)`. Resolve tenant columns by `referenceKind ===
+'tenantId'`, not spelling. The tenant-leading pair also serves the reference
+index requirement.
+
+Only an unqualified, non-JSON-path index with the same leading columns suppresses
+it. Append declared composites first, then default ordering, then reference
+indexes. `(tenant_id, created_at, status)` replaces the default pair;
+`(tenant_id, publish_date)` does not. Emit one plain index per STI table, since
+base polymorphic reads lack `_meta_type` predicates.
+
+Do not add direction or PK columns by inference: `IndexDefinition` has no
+per-column directions, backward B-tree scans serve descending timestamps, and
+the mixed-direction PK tie-break still needs incremental sorting within equal
+timestamps.
+
+### One conflict-target rule, applied on every producer
 
 `save()` upserts on `ObjectRegistry.getConflictColumns()`; the schema must
 carry exactly one unique index over those columns (or they must be the
@@ -546,15 +361,10 @@ schema does not index is a hard PostgreSQL error (42P10) on the first save,
 and a key the schema indexes without the tenant column is the silent
 cross-tenant overwrite this rule exists for.
 
-### 20. Every generated index name is length-guarded before it leaves a path (#2374)
+### Every generated index name is length-guarded before it leaves a path (#2374)
 
-PostgreSQL truncates any identifier past 63 **bytes** and reports nothing;
-SQLite and DuckDB do not, so the entire test suite was blind to it. The 66-byte
-`content_contribution_revisions_contribution_id_revision_number_idx` shipped
-that way — only the differ's signature-equivalence check kept it from emitting
-`add_index` on every run. Two names agreeing for 63 bytes is the real hazard:
-`CREATE INDEX IF NOT EXISTS` no-ops against the wrong index, and the second
-index is never created.
+PostgreSQL truncates identifiers beyond 63 bytes; two generated names sharing
+that prefix can make `CREATE INDEX IF NOT EXISTS` silently skip an index.
 
 `schema/index-utils.ts` owns the guard, and it splits by who owns the name:
 
@@ -567,17 +377,9 @@ index is never created.
   worse than refusing it, and `SchemaComparer` matches indexes **by name**
   first, so a 70-byte declaration could never match the 63-byte index
   PostgreSQL stored and `db:migrate` would emit `add_index` forever.
-- **Table and column names** → deliberately **not** guarded. PostgreSQL
-  truncates identifiers *consistently on every reference*: `CREATE TABLE
-  "<80 bytes>"` and a later `SELECT ... FROM "<the same 80 bytes>"` both resolve
-  to the same stored 63-byte name, so one long name round-trips fine end to end.
-  `smrt-users` depends on this — it ships an intentional 80-byte
-  `@smrt({ tableName })` (`permission_policy_table_name_that_is_far_too_long…`)
-  and derives unique Postgres RLS policy names from it. An earlier revision of
-  this rule hard-errored here on the theory that the runtime resolves tables by
-  name and would break; that theory is wrong for the reason above, and the error
-  broke `packages/users`. The residual collision risk is over a name the
-  developer chose, not one the generator manufactured.
+- **Table and column names** are not guarded: PostgreSQL truncates their
+  declarations and references consistently. `smrt-users` tests intentionally
+  long table names; collision risk remains with the author.
 
 `enforceIdentifierLimits()` is the single call site per path, placed **after**
 `ensureReferenceColumnIndexes()` — nothing may lengthen a name after it. Doing
@@ -611,7 +413,7 @@ can exceed 63 bytes even when the table and column each fit. SMRT never names
 it, and PostgreSQL disambiguates its own truncations by appending a counter
 rather than collapsing them, so there is no silent-collision hazard there.
 
-### 21. The `_smrt_` prefix does not mean "system table" (#2376)
+### The `_smrt_` prefix does not mean "system table" (#2376)
 
 `bootstrapSystemTables()` owns nine hand-written tables; ~25 more `_smrt_*`
 tables belong to `@smrt()` models and are created by `db:migrate` (feature
@@ -685,6 +487,12 @@ and indexes but deliberately emits no physical constraint, avoiding circular
 package DDL. Tenant markers follow the same non-constraint rule because a
 tenant is a scope, not an ownership edge.
 
+Same-package archival identifiers may explicitly use `@foreignKey(Target, {
+constraint: false })`: preserve relationship loading and indexing, but omit
+physical constraints, schema dependencies, and application cascade/preflight
+so the identifier survives parent deletion. Document the retention reason at
+the field; ordinary references remain constrained.
+
 For a same-package relationship whose semantics are portable but whose physical
 constraint shape is not, `@foreignKey(Target, { constraint: { engines: [...] } })`
 is the public exception. The allowlist scopes physical DDL and dependency
@@ -700,6 +508,13 @@ tables first and adds their named constraints afterward. DuckDB refuses cycles,
 self-references, `CASCADE`, and `SET NULL` with an actionable error because its
 current ALTER/constraint support cannot enforce those shapes safely.
 
+Rollback drops children before parents, removes deferred PostgreSQL cycle
+constraints first, and defers SQLite checks while dropping populated cycles.
+Aggregation that filters a parent also removes a retained child's physical FK.
+PostgreSQL deferred constraint adds are idempotent. Generated `ON UPDATE
+CASCADE` remains the default; DuckDB/JSON must refuse unsupported actions
+rather than silently stripping them.
+
 For existing tables, PostgreSQL checks the exact child table/column against the
 exact referenced table/column before adding a constraint as `NOT VALID` and
 then validating it. The probe uses distinct child/parent aliases and, when both
@@ -717,16 +532,9 @@ no-op.
 
 ### Pre-R11 `text` ids converge to `uuid` before any FK statement (#2608)
 
-R11 made SMRT identifiers and references native `uuid` on PostgreSQL. A
-database created before that change still stores its `id` columns as `text`
-while every reference column added afterwards materializes as `uuid`.
-PostgreSQL cannot implement a foreign key across two different physical types —
-FK DDL admits no cast — so `ADD CONSTRAINT … NOT VALID` fails with SQLSTATE
-42804 and aborts every later statement in the same migration batch.
-
-Two rails handle it, and both are PostgreSQL-only. SQLite stores UUIDs as text
-by design and DuckDB cannot rewrite a column type in place, so neither engine
-emits anything for this drift.
+PostgreSQL FK columns must have matching physical types. Legacy text IDs may
+meet newer native UUID references; neither SQLite (text UUID by design) nor
+DuckDB (no in-place type rewrite) emits this convergence.
 
 **The runtime guard fails closed.** `SchemaManager.ensurePostgresForeignKey()`
 reads both live column types and refuses to emit `ADD CONSTRAINT` when they
@@ -744,12 +552,8 @@ tolerated pre-R11 deployment and is left alone; its foreign keys are
 type-compatible today, and the R11 uuid/text equivalence in
 `migrations/differ.ts` keeps it out of the column diff.
 
-Components, not individual pairs, are the unit of decision: one legacy `text`
-primary key can be referenced by several children, and converting it for one
-of them would break every sibling that is still `text`. A self-referential
-table falls out of the same grouping because both endpoints land in one
-component. Convergence is relationship-driven, so a legacy `text` id that
-nothing references keeps its R11 tolerance.
+Converge entire relationship components, including siblings and self-references;
+an unreferenced legacy text ID retains its UUID/text equivalence tolerance.
 
 The planner never coerces data. Before emitting anything it probes each column
 it would rewrite for values that are not uuid-shaped (the same `~*` canonical
@@ -814,173 +618,79 @@ align them deliberately.
 The conversion is one-time and idempotent: once the column is native `uuid`,
 the component is uniformly UUID and the planner emits nothing.
 
-Properties to keep if you touch that module:
-
-- **The plan is registry-derived and rebuilt per delete.** Registration is
-  incremental — manifests load lazily and tests register classes between cases —
-  so a cached plan would silently skip a table that registered later. Cache it
-  only behind an invalidation hook that every registration path calls.
-- **A class with nothing pointing at it skips the transaction entirely — but
-  `CascadePlan.isEmpty` requires no polymorphic association class anywhere in
-  the process, not just no typed references.** `buildCascadePlan()` pushes
-  *every* registered `SmrtPolymorphicAssociation` subclass into
-  `plan.polymorphic` unconditionally (`cascade.ts` around
-  `isPolymorphicAssociationClass`): a `metaType` column can point at any class
-  at runtime, so there is no static metadata to scope it by the target being
-  deleted. One registered polymorphic class anywhere makes `isEmpty` false for
-  every delete in that process — do not read "the common case skips the
-  transaction" as "most deletes in a real app skip it"; in a multi-package app
-  that registers even one polymorphic association, almost none do.
-  `runCascadeDelete()` builds the plan for `getResolvedQualifiedName()` (not the
-  bare constructor name — two packages can register the same simple name).
-- **Cascaded rows are removed set-based.** Their `beforeDelete`/`afterDelete`
-  hooks and interceptors do not run and no change-feed tombstone is written for
-  them, which is exactly what a DB-level `ON DELETE CASCADE` does. Only the
-  object `delete()` was called on runs the lifecycle. Do not "improve" this into
-  a per-row model delete without deciding what that means for sync consumers.
-- **Everything is one transaction where the adapter has one**, including the
-  object's own `DELETE`, whenever there is anything to cascade. The `RESTRICT`
-  checks run first, before any mutation, so a refusal costs nothing; the
-  transaction is what makes a refusal *deeper* in the graph safe.
-- **`_smrt_embeddings` and `_smrt_contexts` are matched by id *and* a
-  class-name candidate set, not id alone.** Their class columns store the
-  *runtime* constructor name, which for an STI hierarchy is a concrete
-  subclass rather than the class the cascade planned from — id-alone matching
-  looked STI-safe, but let two unrelated classes using `idType: 'text'`
-  (non-UUID, not guaranteed globally unique) collide on a shared id value and
-  delete each other's rows (review fix). `ownerClassCandidates()` expands to
-  every STI hierarchy member of the class the ids actually belong to, in both
-  qualified and simple form. A failure to clean them is logged, never raised —
-  an application database may predate the table, and losing derived rows must
-  not fail a valid delete.
-
-`_smrt_changes`, `_smrt_ai_usage`, `_smrt_signals` and the dispatch tables are
-deliberately **not** cascaded. They are append-only logs; the change feed in
-particular receives the delete's own tombstone, so cascading it would erase the
-record that tells sync clients the row is gone.
-
-### 22. System tables get a retention policy, not just a prune function (#2375)
-
-Four framework-owned tables grow with traffic and nothing used to remove a row:
-`_smrt_changes` (one per save/delete), `_smrt_ai_usage` (one per AI call, and
-persistence is on by default), `_smrt_contexts` (whose `expires_at` nothing
-enforced) and `_smrt_dispatch` (an operator-only `dispatch:cleanup`).
-`src/system/retention.ts` is now the single place that bounds them.
-
-- **`runRetentionSweep(db, policy)` is the entry point.** It runs the four
-  built-in tasks in a fixed order, then every task other packages contributed
-  via `registerRetentionTask()` — `@happyvertical/smrt-jobs` registers
-  `_smrt_jobs`/`_smrt_job_events`, `@happyvertical/smrt-users` registers
-  session/magic-link/CLI-auth expiry. A task that throws is recorded on its own
-  result and the sweep continues; a missing table reports `unavailable`, so a
-  sweep is safe against a partially bootstrapped database.
-- **A contributed task only exists in a process that loaded its package.** Both
-  packages register on import from their entry point, and the registry lives on
-  `globalThis` (like `ObjectRegistry`) so a duplicated `smrt-core` resolution
-  cannot split it. `smrt db:prune` optionally imports both packages for exactly
-  this reason — a project that installs neither correctly gets neither task.
-- **Defaults are opt-out, not opt-in.** `DEFAULT_RETENTION_POLICY` covers the
-  four built-in tables (changes 30 days, AI usage 90 days, dispatch 30 days
-  completed / 90 days failed, contexts strictly by their own `expires_at`), and
-  those are the ones `smrt.configure({ retention })` tunes. Contributed tasks
-  carry their own defaults and their own window options —
-  `DEFAULT_JOB_RETENTION` (7 days terminal / 30 days failed / 30 days events,
-  set through `registerJobRetentionTasks()` or the runner's `retention.jobs`),
-  and expired credentials, which have no window because an expired credential
-  has nothing worth retaining. Every task, built-in or contributed, can be
-  turned off: a table set to `false`, a task set to `false` under `tasks`, or
-  `enabled: false` for the whole sweep — through `smrt.configure`,
-  `smrt db:prune --skip`, or the runner's `retention` config.
-- **Contributed task names are prefixed with the owning package's short name**
-  (`jobs-records`, `jobs-events`, `users-sessions`, …) because the registry is
-  one process-global namespace.
-- **Scheduling lives outside core.** A running `TaskRunner` sweeps every six
-  hours (`retention: false` opts out) and `smrt db:prune` is the cron entry
-  point. The first runner sweep is one interval after `start()`, never at
-  start: a crash-looping worker must not become a delete loop.
-- **Every prune counts before it deletes.** `rowCount` is not reliably
-  populated across the engines SMRT supports, so counting is both what gives a
-  usable figure and what lets `dryRun` preview the *same* predicate rather than
-  an approximation of it. Count and delete are two statements and deliberately
-  not one transaction — a maintenance pass must not hold a write lock over a
-  large delete — so the figure is approximate under concurrent writers. Where
-  two bounds can select the same row (`pruneChangeFeed`, `pruneAiUsage`), the
-  second bound excludes what the first already accounted for, so a dry run does
-  not count an entry twice.
-- **Every retention predicate ships with its index** (rule 2 applies to
-  maintenance SQL too): `_smrt_contexts(expires_at)`,
-  `_smrt_ai_usage(tenant_id, created_at)` — which is also the subscriptions
-  billing meter's range scan — `_smrt_dispatch(status, processed_at)` and
-  `(status, updated_at)` come from the system DDL, so they reach existing
-  databases through the `SMRT_SCHEMA_VERSION` bump that replays it.
-  `_smrt_jobs(status, completed_at)` comes from
-  `ensureJobsSystemTableCompatibility()` instead, because `_smrt_jobs` is
-  generated from a decorated class and does not exist yet when bootstrap runs;
-  the jobs collection calls that path on every `initialize()`.
-- **Expiry enforcement is prune-side only.** `recall()`/`recallAll()` keep
-  their documented "expiry is not applied at read time" contract — changing it
-  would change read semantics for existing callers, which is a different issue
-  from bounding storage. `LearningMemory` filters expired rows itself.
-
-### 23. Dead generation surfaces were deleted, not wired (#2380)
-
-Rule 9 named three surfaces that read as canonical but were not: the AST
-`generateSchema(objectDef)` entry point, `SchemaOverrideSystem`, and the
-never-emitted `triggers: []`. Resolution, so a future agent does not re-open
-what was deliberately decided:
-
-- **The AST path is gone.** `SchemaGenerator.generateSchema(objectDef)` and its
-  AST-only private helpers (`generateIndexes`, `generateTriggers`,
-  `extractDependencies`, `generateVersion`, `getTableName`,
-  `extractPackageName`) were deleted from `schema/generator.ts`, along with
-  their sole caller, `generateSchemaModule()` in `vite-plugin/index.ts`, and the
-  `smrt:schema` / `@happyvertical/smrt-virt-schema` virtual module registration
-  that fed. Nothing else called it — grep the deleted method's exact name
-  before assuming a caller was missed; the path-parity fixture and every other
-  rule above already speak only of the four surviving entry points.
-- **`SchemaOverrideSystem` is gone**, file and all
-  (`schema/override-system.ts` no longer exists). It was never called from
-  anywhere in this repository outside its own now-deleted exports, and two of
-  its five public methods (`createPraecoContentOverride`,
-  `createPraecoMeetingOverride`) hard-coded a schema extension for a
-  consuming project outside this monorepo — scaffolding that never belonged in
-  the framework, not a generic feature with a missing caller. `SchemaOverride`
-  (the type) went with it; `ColumnDefinition`/`IndexDefinition`/
-  `TriggerDefinition`, which it merely referenced, did not.
-- **The DDL-strategy trigger machinery was kept, not deleted.**
-  `TriggerDefinition`, `SchemaDefinition.triggers`, and every DDL strategy's
-  `generateTriggers()` / `generateTriggerStatement()` / `supportsTriggers()`
-  (`schema/ddl/*.ts`) are real, engine-uniform, directly-tested rendering code
-  that runs on **every** table creation via `strategy.generateTriggers(schema)`
-  — unlike the AST path, this is not an orphaned call graph. It is kept for the
-  same reason rule 16 keeps the cached `schema.ddl` string: `SchemaDefinition`
-  is part of the shape third-party tooling and published manifests may already
-  depend on, and `EngineSpecificDDL`/`MultiEngineDDL` (`schema/ddl/types.ts`)
-  carry `triggers` as part of that same contract. Deleting a published field is
-  a different (and unjustified) risk from deleting a virtual module nothing
-  ever imported.
-- **What changed is what is documented, not what runs.** `schema.triggers` is
-  now explicitly documented (`schema/types.ts`) as always `[]` on every schema
-  a `@smrt()` class can produce, and why: there is no `@smrt()`/`@field()`
-  option that populates it (unlike `indexes`, #2357), `updated_at` is
-  maintained at the application layer (`SmrtObject.save()`), and
-  `migrations/differ.ts` never diffs triggers — so even a hand-populated one
-  would only apply to a newly `CREATE TABLE`d table and never retrofit an
-  existing one. Wiring live trigger emission was considered and rejected for
-  this issue: it is a migration-rollout feature (retrofitting 238+ existing
-  production tables needs the same `SMRT_SCHEMA_VERSION`-replay or differ
-  support rule 21/rule 22's system-table work required), not a cleanup, and
-  nothing in the epic depended on it the way #2359 depended on FK indexes
-  actually shipping.
-- **`_smrt_signals` and `ObjectRegistry.persistToDatabase()`/`loadFromDatabase()`**
-  — named in the original finding alongside triggers — were already handled by
-  #2376 before this issue landed: see rule 21 and `system/schema.ts`'s
-  `RETIRED_SYSTEM_TABLES`. Nothing further to do there.
-- **The two config-rebuild-site comments** (`schema/utils.ts`,
-  `testing/database.ts`) rule 8 requires were already in place, added by
-  #2357/#2360; the `testing/database.ts` "same as migrations" overclaim rule 1
-  quotes was already corrected by #2359, and doctor's `experimentalDecorators`
-  check was already fixed by #2368/#2399 (see `packages/cli/AGENTS.md`
-  Gotchas). Re-verify against current source before repeating any of these —
-  the epic's PRs landed across one evening and a stale assessment line is not
-  proof a fix is still needed.
+### Application cascade invariants (`src/cascade.ts`)
+
+- Rebuild the registry-derived plan on every delete; manifests register lazily.
+  Caching requires invalidation across every registration path.
+- Plan from `getResolvedQualifiedName()`. Every registered polymorphic
+  association class participates, since its runtime target can be any class;
+  `CascadePlan.isEmpty` requires no such class anywhere and no typed references.
+  Only an empty plan skips the transaction.
+- Cascades are set-based: child hooks/interceptors and change-feed tombstones do
+  not run. Only the explicitly deleted object runs its lifecycle. RESTRICT
+  checks precede mutations; the parent DELETE and cascades share one transaction
+  where supported, so deeper refusals roll back.
+- Derived `_smrt_embeddings` / `_smrt_contexts` cleanup matches IDs AND
+  `ownerClassCandidates()` (qualified and simple STI member names), never IDs
+  alone: unrelated text-ID classes can collide. Cleanup failures are logged,
+  not raised, because these tables may not exist in older databases.
+- Never cascade append-only `_smrt_changes`, `_smrt_ai_usage`, `_smrt_signals`,
+  or dispatch logs; deleting change tombstones would break sync.
+
+### Retention (`src/system/retention.ts`)
+
+`runRetentionSweep(db, policy)` runs four built-ins in fixed order, then
+`registerRetentionTask()` contributions. A failed task records its result and
+continues; a missing table is `unavailable`. The `globalThis` registry avoids
+split registrations under duplicate core resolution; package tasks exist only
+after importing the package. CLI prune optionally imports jobs/users.
+
+Defaults are opt-out: changes 30 days, AI usage 90 days, completed dispatch 30
+days/failed dispatch 90 days, contexts by `expires_at`. `smrt.configure({
+retention })` tunes built-ins; contributed tasks own their defaults/options.
+Jobs defaults are 7 days terminal, 30 failed, 30 events via
+`registerJobRetentionTasks()` or runner `retention.jobs`; expired credentials
+have no extra window. Disable a table/task with `false` or the whole policy with
+`enabled: false`; CLI `--skip` and runner configuration expose these controls.
+Task names use package prefixes (`jobs-records`, `users-sessions`).
+
+Core does not schedule sweeps. TaskRunner runs every six hours, first one
+interval after start; `retention: false` opts out. `smrt db:prune` supports cron.
+Every prune counts then deletes using the same predicate; `rowCount` is not
+portable. The two statements deliberately are not transactional, so counts are
+approximate under concurrency. Overlapping change/AI-usage bounds exclude rows
+already counted, including dry runs.
+
+Retention indexes belong in system DDL and its versioned replay:
+`_smrt_contexts(expires_at)`, `_smrt_ai_usage(tenant_id, created_at)`, and dispatch
+`(status, processed_at)` / `(status, updated_at)`. Jobs `(status, completed_at)`
+belongs in `ensureJobsSystemTableCompatibility()` on each collection initialize,
+since decorated jobs tables do not exist at bootstrap.
+
+Expiry remains prune-side for object/collection `recall()`/`recallAll()`;
+`LearningMemory` separately filters it at read time.
+
+## Supported generation surfaces
+
+The four entry points above are the supported generator paths. The unused AST
+`generateSchema(objectDef)`, `smrt:schema` / `@happyvertical/smrt-virt-schema`
+virtual modules, and project-specific `SchemaOverrideSystem` were removed.
+
+Keep published `SchemaDefinition.triggers`, `TriggerDefinition`, and the DDL
+strategies' trigger renderers: they support hand-authored new-table schemas.
+Generated `@smrt()` schemas always emit `triggers: []`; no decorator populates
+it, `save()` maintains `updated_at`, and the differ never retrofits triggers.
+Adding live trigger generation requires a migration rollout design.
+`_smrt_signals` and database-persisted registry APIs are retired; see
+`RETIRED_SYSTEM_TABLES` in `src/system/schema.ts`.
+
+## PostgreSQL migration execution
+
+`MigrationTracker.applyAll({ atomic: true })` sets local lock/statement timeouts
+before any DDL. `postgresSafe: true` commits non-index DDL atomically, then runs
+indexes CONCURRENTLY on `db.acquireSession()` so settings and DDL share a
+connection. This mode is not atomic. Unfinished indexes are `failed`, not
+`running`; `[smrt: concurrent-index phase 1 committed]` in `error_message`
+allows reruns to resume index work without replaying committed DDL. Inspect
+`pg_index.indisvalid` and drop INVALID indexes before rebuild (`pg_indexes`
+alone cannot detect them). Operational commands: `packages/cli/AGENTS.md`.