@jarenjs/db 0.49.2 → 0.66.1

package/ARCHITECTURE.md

@@ -1,21 +1,21 @@
 # @jarenjs/db — architecture
 
-The public documentation promises SQLite and nothing else. This file
-documents the two seams that make that promise cheap to keep and cheap
-to outgrow — an undocumented seam decays into an accident.
+The package separates Store behavior from driver ownership and dialect spelling.
 
 ## The driver seam (`src/driver.js`, `src/drivers/*`)
 
-A driver is `{ name, dialect, open(path, options) }` returning a
-`Connection` (see `docs/MODEL-FORMAT.md` §4). Three bindings exist,
-one per subpath — `./node` (`node:sqlite`), `./bun` (`bun:sqlite`),
-`./wasm` (an injected handle) — and each imports its runtime builtin
-**lazily inside `open()`**, never at module scope. The release gate
-imports every export subpath under Node *and* Bun; Bun ships no
-`node:sqlite` and Node cannot resolve `bun:` specifiers, so a
-top-level builtin import would turn the gate red in both directions.
-`open()` is where "this binding does not exist here" becomes the coded
-`JD0003`.
+A driver is `{ name, dialect, open(path, options) }` returning a Connection
+(`docs/MODEL-FORMAT.md` §4). Runtime bindings live behind public Node, Bun, wasm,
+PostgreSQL, Node worker and Node pool subpaths. Node and Bun builtins load lazily
+inside `open()`, so importing a subpath remains portable and an unavailable
+binding becomes `JD0003`. The worker endpoint's static Node imports are reached
+only when the Node worker factory launches that endpoint.
+
+Worker protocol, scheduling and snapshot storage sit below `finishConnection`;
+query planning, capture and transaction scopes remain in the Store. Compiled
+operation metadata classifies reads for the pool. The wasm oo1 adapter shares one
+session mapping across OPFS and atomic IndexedDB snapshots. [Execution hosts](docs/HOSTS.md)
+documents bounds, native-call shutdown limits, capability probes and measurements.
 
 Every connection and statement method may return a value or a promise.
 The store composes through the sync-capable `chain` helper, which
@@ -46,10 +46,54 @@ test-double dialect whose quoting, parameter style and type names all
 differ: the same model produces correspondingly different SQL.
 
 Deliberately *not* dialect concerns, because they are behavioural
-rather than syntactic, and live in the capability table instead:
-whether functions register per connection, whether change capture
-exists and in what form, and whether tables can be restructured in
-place.
+rather than syntactic, and live in the CONNECTION capability table
+instead: whether functions register per connection, whether change
+capture exists and in what form, and whether tables can be
+restructured in place.
+
+The dialect has a capability table of its own, and it is a closed set
+(`DIALECT_CAPABILITIES`): `createDialect` refuses a name outside it, so
+a misspelled capability cannot read as a quiet `false` on the engine
+that has the feature. Every entry answers a SYNTACTIC question — does
+this engine have a binary JSON type, indexable generated columns, a
+column with no scalar type, `RETURNING`, upsert, savepoints, an
+up-front write lock, a result alias in `GROUP BY`, in-place `ALTER`,
+virtual tables, triggers, a configuration vocabulary, a stored CREATE
+text, always-on referential integrity, an insertion-ordered row
+identity. What the store does where the answer is `false` is a
+FALLBACK or a coded refusal on the common path, never a skip:
+
+| Capability off | What the common path does instead |
+|---|---|
+| `pragmas` | no configuration statement is issued; an explicitly requested pragma is `JD0007`, a defaulted one is dropped, and the effective record reads `null` throughout |
+| `declaredSqlText` | the drift check is the structural one — columns and their types, indexes and their covered columns in order, foreign-key tuples — and an object the model never declared is no longer detectable |
+| `foreignKeysAlwaysOn` (on) | the open path neither sets nor reads back the referential-integrity switch |
+| `virtualTables` / `triggers` | a `physical: 'rtree'` column set maps back onto the B-tree over the four edge columns, and `explain().prefilters[].via` says so |
+| `untypedColumns` | a comparison against a member the schema does not type reads the member out of the document instead of the column: same answer, unindexed |
+| `indexableGeneratedColumns` | the derived-column mapping is the STORED one — the store writes the values and the columns are ordinary |
+| `rowIdentity` (with a declared identity column) | the dialect adds that column to every table it creates, and it IS `rowIdentity()` |
+| `immediateTransactions` | `tx.beginImmediate` is `tx.begin`, and a read-then-write body takes its lock when it writes |
+
+Two subsystems are SQLite's own in 0.1 and say so on the connection
+rather than in the dialect: the durable job queue (`capabilities.jobs`)
+and the change ledger (`capabilities.changeCapture`) write their own
+statements. Opening a store that asks for either on a connection that
+declares it absent is a coded refusal at open — `JD0003` and `JD0051` —
+not a failure at the first statement.
+
+`test/db/dialect-conformance.js` is the kit every dialect passes: it
+asserts behaviour rather than text, because a second dialect that
+passed a golden-SQL comparison would have to spell PostgreSQL the way
+SQLite spells it. A quoted identifier cannot close its own quoting; a
+write statement binds through the dialect's own parameter form; a
+generated column carries the storage word its engine has; an optional
+feature is absent in BOTH halves or present in both.
+`test/db/dialect-census.test.js` is the other side of the same claim: a
+character scanner reads every string and template literal of every
+module, keeps the ones that begin as SQL, and fails on a SQLite
+spelling outside `src/dialects/` and `src/drivers/` unless the module
+is named — with the capability that makes it honest — in a short
+approved list.
 
 The SQLite dialect stores documents as JSONB in a `BLOB` column of a
 `STRICT` table, projects each indexed path into a virtual generated
@@ -57,6 +101,175 @@ column over `jsonb_extract`, and renders reads back to text through
 `json()`. Parameters are positional because every shipped binding
 binds arrays.
 
+### The PostgreSQL dialect (`src/dialects/postgres.js`)
+
+`postgresDialect()` is the second spelling, and the reason the contract
+above exists. It imports no client and no runtime builtin, so
+`@jarenjs/db/postgres` resolves in a browser bundle and type-checks with
+nothing installed; transport is the injected driver's business.
+
+Documents are `jsonb`. An indexed path becomes a **STORED** generated
+column — PostgreSQL has no indexable virtual one — typed from the
+schema, and its expression guards the member with `jsonb_typeof` inside
+a `CASE`, because a generated column's expression must be IMMUTABLE and
+a cast that can raise is not one: a document whose member is the wrong
+JSON type stores `NULL` rather than failing its INSERT. Text columns and
+text comparisons carry `COLLATE "C"`, which is byte order, which is what
+SQLite's default `BINARY` collation is — without it the same half-open
+prefix range would hold different rows on a server initialised in
+another locale. Every table the dialect creates carries `rid bigserial`,
+because a collection is a SEQUENCE and PostgreSQL has no per-row
+identity that survives an UPDATE (`ctid` moves).
+
+Three mapping decisions cost something, and each is stated rather than
+worked around:
+
+| Decision | Why | What it costs |
+|---|---|---|
+| `integer` and `number` are both `numeric` | `bigint` would reject a document the model accepts — `3.5` in a member the schema types `integer` — inside a generated column's cast, and a storage decision that can refuse a valid document is not one | introspection cannot tell the two apart on the way back |
+| a `boolean` member is `smallint` holding 1 or 0 | the shared forms compare a boolean against those integers (a type test, a projected pair's type name); a `boolean` column would make each an operator-resolution error rather than a row | introspection reads the column back as a number |
+| a path the schema does not type is `jsonb` | there is no honest scalar type for it, and `capabilities.untypedColumns: false` says so | a comparison against such a member reads the document rather than the column: the same answer, unindexed |
+
+Two engine differences the emitter had to be taught, rather than the
+dialect papering over:
+
+- **The type discriminator's vocabulary.** `jsonb_typeof` answers six
+  names; the row decoder reads SQLite's in JavaScript. The dialect maps
+  the two that differ (`string` → `text`, `boolean` → `true`/`false`)
+  and declares `numericTypeNames: ['number']`, which is what the
+  emitter's numeric guard reads instead of `('integer', 'real')`.
+- **An external operand's parameter.** A PostgreSQL parameter's type is
+  resolved once for the whole statement, so one placeholder cannot be a
+  text member's operand in one branch and a numeric member's in
+  another — and the guard that keeps a row out of the wrong branch does
+  not stop the COERCION, because `AND` does not short-circuit. The
+  dialect declares `externalEncoding: 'json'`: the store binds the
+  value's JSON text, and the two branches compare in a space that holds
+  every scalar (text against text under `C`, number against number in
+  `jsonb`'s own numeric order).
+
+An identifier longer than 63 bytes is REFUSED rather than emitted:
+PostgreSQL truncates one silently, and two generated column names that
+share a prefix would become one, which is an index quietly serving
+another path.
+
+### Database → model (`src/introspect.js`)
+
+`store.introspect(options)` — and `introspectModel(connection, options)`
+under it — reads a live database's catalog and derives the
+`jaren-model` document that would produce it, beside a report of
+everything it could not.
+
+It is **read-only**, and a statement trace is what says so: every
+statement it issues begins `SELECT` (or `PRAGMA`, on SQLite). A derived
+model is an ANSWER; applying it is the migration planner's job and the
+operator's decision, and the two are deliberately not one act.
+
+It **never claims a byte-perfect round trip**. A physical shape carries
+less than the model that made it, and each gap is a report row with a
+stable code, an object and a detail, sorted, once:
+
+| code | what the shape does not carry |
+|---|---|
+| `document-members` | a document's unindexed members leave no trace, so the derived schema holds only what an index or the key names |
+| `key-source` | a text key column cannot say which document member filled it; `identity: 'uuid'` is derived and `options.keys` supplies the pointer |
+| `unmapped-view` | a view is not a shape a model can declare |
+| `unmapped-table` | a table with neither a document column nor a key |
+| `unmapped-column` | a generated column whose expression is not a member path this dialect wrote |
+| `unmapped-index` | an index over an expression, a predicate, or a column no member path explains |
+| `unmapped-type` | a column type no schema type maps back from — the member is derived untyped |
+| `ambiguous-relation` | a foreign key says which entity it points at; which side declared the edge, and whether the other holds many, is not in the shape |
+
+`strict: true` refuses instead of returning a partial model, because a
+caller about to diff the result against a declared model needs to know
+the difference is real.
+
+What IS exact: every index, by name and by covered member paths in
+order; the type of every member an index covers, placed at the DEPTH
+the path names it at (so `$.nested.deep` rebuilds
+`nested.properties.deep`, and the column type round-trips); a
+database-allocated integer key; an entity's primary key, its unique and
+indexed columns, and its foreign keys with their on-delete behaviour.
+
+Two things make the two engines answer the same question. The dialect's
+catalog statements hand back tables, columns, indexes and foreign keys
+in ONE row shape — so the IR in the middle is engine-neutral — and only
+the dialect reads its own generated-column expression back into a member
+path (`memberPathOf`), because only the dialect wrote it. Indexes come
+back sorted by name on both, because a physical shape carries no record
+of the order a model declared them in and a derivation that inherited
+one engine's listing order would not be a derivation.
+
+### The portability matrix
+
+Everything a dialect may decline, and what the store does instead. A
+`—` is a feature the engine does not have; the store's behaviour is the
+common fallback, never a skip.
+
+| Capability | SQLite | PostgreSQL |
+|---|---|---|
+| `jsonb` | yes (`BLOB` + the JSONB family) | yes (`jsonb`) |
+| `generatedColumns` / `indexableGeneratedColumns` | yes, `VIRTUAL` | yes, `STORED` |
+| `untypedColumns` | yes (`ANY`) | — (the member is read from the document) |
+| `returning`, `upsert`, `savepoints` | yes | yes |
+| `immediateTransactions` | yes (`BEGIN IMMEDIATE`) | — (`tx.beginImmediate` is `BEGIN`) |
+| `groupByAlias` | yes | yes |
+| `alterTableFull` | — | yes |
+| `virtualTables` / `triggers` | yes (the R\*Tree mapping) | — (`physical: 'rtree'` maps back onto the four edge columns) |
+| `pragmas` | yes (the closed set) | — (a server is configured by its operator) |
+| `declaredSqlText` | yes | — (the drift check is the structural one) |
+| `foreignKeysAlwaysOn` | — (set and verified per connection) | yes |
+| `rowIdentity` | yes (`rowid`) | yes (the declared `rid` column) |
+| `savepointStartsTransaction` | yes | — (a top-level transaction is `BEGIN`/`COMMIT`) |
+
+And the two subsystems that are SQLite's own, declared on the
+CONNECTION rather than the dialect because they are a library's
+capability and not a syntax: the durable job queue (`jobs`) and the
+change ledger (`changeCapture`). A store asked for either on a
+connection that declares it absent is refused at open by name.
+
+One behavioural difference is neither a capability nor a fallback,
+because nothing can paper over it: **PostgreSQL aborts a transaction a
+statement failed in**, and every later statement in it is `25P02` until
+it ends. SQLite carries on. A body that means to catch a write's failure
+and continue has to say so — `scope.transaction()` opens a savepoint the
+failure rolls back to — and where it does not, the store reports the
+engine's own condition under `JD2088` (`class: 'aborted'`, not
+retryable) rather than a confusing failure of the next statement.
+
+### The injected driver (`src/drivers/postgres.js`)
+
+The driver imports nothing either. It takes a connection SOURCE —
+`{ connect() }` answering `{ query(text, values), release?() }`, which a
+`pg.Pool` is verbatim — acquires ONE client at open, holds it for the
+store's life and releases it exactly once at `close()`. Holding one is
+not a simplification: a connection owns one savepoint stack, and a store
+that took a different pooled client per statement would have `BEGIN` on
+one and `COMMIT` on another.
+
+Statements are prepared by name, from a counter that is monotonic across
+the PROCESS rather than across one adapter — a prepared statement
+belongs to the session, and a pooled client's session outlives the store
+that borrowed it, so a per-adapter counter would re-use a name for
+different SQL and the server refuses that. The names are deallocated at
+`close()`, so the session goes back to the pool clean. A cached plan
+whose result type changed under it (`0A000`, after a migration) is
+re-run unnamed rather than raised.
+
+Three normalizations, each forced by the wire rather than chosen:
+`int8` and `numeric` arrive as strings (they can exceed a double) and
+become JavaScript numbers, which is the same ceiling SQLite's INTEGER
+has; `json`/`jsonb` arrive parsed, because the client's type parsers are
+the host's configuration, and the row decoder reads text; and a
+JavaScript boolean is bound as 1 or 0, because a boolean member is 1 or
+0 in this mapping.
+
+`schema` is where the store lives. It is set on the acquired connection
+AND given to the dialect, so the DDL and the catalog reads agree — one
+value, two consumers. A migration's SHADOW replay is a second database,
+which on a file engine is `:memory:` and on a server has to be another
+schema: `migrate(..., { shadowDriver })` is where the host names one.
+
 ## What sits on top
 
 - `src/ddl.js` — a normalized collection to its physical plan. Index
@@ -141,9 +354,13 @@ composition; `$exists`/`$empty`; `$starts-with`/`$ends-with`/
 `$contains` on schema-typed string paths with literal patterns;
 `$orderby` over singular schema-typed paths (`$dir`, `$empty`, no
 collation); a top-level `$subsequence` window with literal bounds; the
-top-level aggregates `$count` (bare-binding return only) and
-`$sum`/`$avg`/`$min`/`$max` over a singular schema-typed path; and the
-whole-document projection that returns the bare binding.
+top-level aggregates `$count` and `$sum`/`$avg`/`$min`/`$max` over a
+singular schema-typed path; the whole-document projection that returns
+the bare binding; ONE member path, projected as its value beside its
+JSON type; and a nested SHAPE of objects, arrays, literals and member
+paths, projected as one value/type pair per distinct leaf and rebuilt
+by the decoder — never by parsing a JSON text the database assembled,
+which could not tell an absent member from a present `null`.
 
 Plus the spatial predicates a **derived** index makes decidable:
 `$bbox-intersects` against a literal or external region, a geohash
@@ -176,11 +393,14 @@ mode, `knn`, beside native, row and set.
 
 | construct | reason |
 |---|---|
-| `$let` bindings, `$fold`, `$groupby`, positional/window bindings | no equivalence proof exists yet; residual by default |
-| a second `$for` binding, joins, non-singular path expansion | one relation per plan in this version |
+| `$let` bindings, `$fold`, positional/window bindings | no equivalence proof exists yet; residual by default |
+| a `$groupby` whose key is untyped or nullable, whose `$return` reads the binding, or whose `$orderby` names anything but a key | SQL's grouping and the engine's need not agree on an untyped key; after a grouping the binding holds the group's ROWS, which an object member cannot take |
+| a window over the GROUPS, or an aggregate of them | the plan groups whole; a `LIMIT` over the groups would cut a different set |
+| a `$for` binding nothing joins to — a cartesian product | the engine builds the product; a plan that emitted one by accident is the thing an equi-join graph exists to prevent |
+| non-singular path expansion | one relation per binding in this version |
 | `$match` and other unlisted operators, `$call` | no native spelling proven equivalent |
 | `$orderby` with a `$collation` | a collation the dialect cannot reproduce is refused, not approximated |
-| projections other than `'$it'` | run per row (the row residual) — pushed, ordered and windowed rows, projected by the engine |
+| a projection the tree cannot rebuild: an operator over a member, a reference to the binding itself, a non-singular path, a projection with no member path at all | the WHOLE projection runs per row (the row residual) — pushed, ordered and windowed rows, projected by the engine; promoting the part that composes would answer a shape nobody asked for |
 | string operators with an external pattern | the pattern's type is unknowable at plan time and the engine ERRORS on non-string patterns |
 | comparisons where both sides are paths | join territory |
 | array/object literals in comparisons | deep-equality has no guarded native form |
@@ -247,14 +467,21 @@ index-usable spelling and read every row of the collection.
 
 The guards make the forms sound for typed AND untyped paths alike —
 the schema type's job is choosing the generated COLUMN (the index),
-never weakening the guard. Two documented preconditions: string
-operators, aggregates AND ordering are only promoted on schema-typed
-paths because the engine ERRORS on non-conforming operands where SQL
-would coerce or sort — a stored `null` under an ordered key, a
-non-string under a string operator — so on an unvalidated store, rows
-violating the collection schema can make the engine throw where the
-database answers; keep `compileSchema` injected if that distinction
-matters to you.
+never weakening the guard. Two documented preconditions. String
+operators, aggregates and ordering are only promoted on schema-typed
+paths that cannot hold `null` and are not boolean, because the engine
+ERRORS on non-conforming operands where SQL would coerce or sort — a
+`null` under an ordered key, a non-string under a string operator —
+so a path whose schema admits `null` is a named residual for those
+forms and the two engines keep answering alike over conforming data.
+And the guards protect the JSON TYPE, not the VALUE: a typed
+generated column carries SQLite affinity, so on an unvalidated store a
+row that violates the schema — the string `'020'` under an `integer`
+path — reads as the integer `20` in the column while `json_type` still
+says text, and a pushed comparison answers rows the engine (comparing
+the JSON value) does not. Keep `compileSchema` injected: with the hook
+no such row is ever stored, and without it the pushdown is exact only
+over documents that happen to conform.
 
 **Bind-time diversion.** SQLite cannot bind a boolean, and a
 `null`-valued external needs Jaren's null semantics, not SQL's. At
@@ -400,15 +627,19 @@ wants the engine's refusal for every row keeps `compileSchema` injected
 and nothing else, which is what makes the two agree on every row it
 does fetch.
 
-**The probe.** A literal vector must be the column's width at plan time
+**The probe.** A literal vector must be a DECLARED width at plan time
 (another width is not recognized, and the reason says both widths). An
-external probe is checked at call time by the binder's own rule: a
-bound value that is not a vector of the column's width — another width,
-a non-finite component, not an array at all — DIVERTS the call to the
-full-collection residual, where the engine answers what it answers
-everywhere (empty keys, so the secondary keys order every row; or its
-own `JQ2001` for a non-array). The plan never raises on the engine's
-behalf.
+external probe's width is only knowable when it is bound, so the plan
+carries one ALTERNATIVE per declared width over the member — one emitted
+statement each, prepared once and kept with the plan — and the bind
+picks the alternative whose width the probe has. A `CASE` across the
+columns would read every one of them per row, and a statement per call
+would give up the prepared cache; neither is emitted. A bound value no
+declared width takes — another width, a non-finite component, not an
+array at all — DIVERTS the call to the full-collection residual, where
+the engine answers what it answers everywhere (empty keys, so the
+secondary keys order every row; or its own `JQ2001` for a non-array).
+The plan never raises on the engine's behalf.
 
 That diversion is correct and it is expensive: the residual reads every
 document, and `explain()` still reports `knn`, because the plan is the
@@ -467,17 +698,66 @@ is the ENGINE running the caller's own document, decides:
 
 | operator | what bounds the fetch |
 |---|---|
+| `$overlaps` over a declared interval | the half-open conjunction over the two bound columns, plus every row the operator RAISES on (below) |
 | `$resample` (fill, calendar, `first`/`last`) | the spec's own `start`/`end`, plus the operand's `$where` |
 | `$rolling` | the operand's `$where` alone — a window measured in time answers once per input instant |
-| `$asof` with the collection on the RIGHT | the probes' own span (`at <= max` backward, `at >= min` forward, both sides under a `tolerance`) and a membership test over the probes' `by` keys |
+| `$asof` with the collection on the RIGHT | the probes' own span (`at <= max` backward, `at >= min` forward, both sides under a `tolerance`) and a membership test over the probes' `by` keys, plus the ANCHOR below |
 | `$asof` with the collection on the LEFT | nothing — a join answers once per LEFT row, so every left row is needed |
 
-The as-of bound is one STATEMENT, whatever the probes number, which is
-what `test/db/statement-count.test.js` pins: the failure mode a batch
-exists to refuse is one seek per left row. Without a `tolerance` a
-backward join can only be bounded ABOVE, so that one statement can read
-most of a long history — `benchmark/series.js` publishes the candidate
-count beside the timing rather than netting it out.
+The as-of bound is a FIXED number of statements, whatever the probes
+number, which is what `test/db/statement-count.test.js` pins: the
+failure mode a batch exists to refuse is one seek per left row.
+
+**The anchor.** Without a `tolerance` the open side has no bound the
+probes imply: the row that answers the earliest probe is the last one at
+or before it, however far back that lies. The tight bound is the data's
+own, so the plan ASKS for it — `plan.seeks` carries one aggregate read
+through the same declared index, and the statement binds its answer
+through the typed slot (`emit.js`'s fourth `ParamSlot` kind), which
+needs no bind-time type branch because the column's type is declared.
+
+Per group the anchor is `MAX(at) WHERE at <= min(probes)`; the scalar is
+the LEAST of those over the groups. A single global `MAX` would be
+unsound — it can come from one group and drop another group's only
+candidate — and matches only move forward as the probe does, so no row
+below the fold can answer any probe. The comparison stays inclusive, so
+rows sharing the anchor instant reach the kernel and its duplicate rule
+decides among them. A seek that finds nothing binds its own probe: it
+proved there is no row on that side. `forward` anchors the other end,
+`nearest` both, and a `tolerance` asks for none — arithmetic already
+closed both sides. What the anchor saves depends on where the probes
+sit; `benchmark/series.js` publishes the candidate count beside the
+timing rather than netting it out, over probes spread evenly across the
+whole span, which is the anchor at its worst.
+
+**`$overlaps` and the row that raises.** Two half-open spans share an
+instant when each starts before the other ends, which over declared
+bound columns is two comparisons the emitter already spells. What makes
+it a pre-filter rather than an exact translation is the kernel's other
+half: a span that is empty or reversed is not `false`, it RAISES
+(`JQ2001`). A conjunction alone would drop `[500, 100)` for a probe of
+`[100, 200)` — answering where the engine errors, which no pushdown may
+do — so the statement keeps every row whose own span is inverted
+(`start >= end`) and the operator decides over the candidates. That
+disjunct compares two COLUMNS, which no index bounds, so the fetch
+scans; what it still buys is that the pruned rows never reach the
+engine at all. The other malformed cases are SCHEMA ones, exactly as the
+spatial promotions' precondition is: the member must be typed as an
+object whose `start` and `end` are both REQUIRED and both numeric, and a
+write is validated against that schema, so a bound that is absent,
+textual or null is a row the collection cannot hold. An unmapped pair,
+an untyped one, or a probe that is not itself a half-open span pushes
+nothing and names why. ROADMAP carries the declared-interval `CHECK`
+that would make the conjunction exact and the fetch a seek.
+
+**Reading a declared column without its guard.** A temporal refinement's
+own bounds (`p: 'colCmp'`) carry no `json_type` beside them: the model
+declared the column, the comparison only NARROWS, and the kernel decides
+over what comes back. Without that, the statement parsed every scanned
+row's document to discriminate a member the column already carried — at
+the benchmark's largest shape that guard was most of the fetch's cost.
+Narrowing is what makes it safe: SQL's own ordering keeps a row the
+guard would have dropped, never the other way about.
 
 **A group with no instant.** A row whose instant member is missing or
 is not a number groups under SQL `NULL`; the kernel REFUSES such a row
@@ -571,8 +851,9 @@ collection binding, so the planner passes that binding's **name** in:
 wrapped under any other name every reference would read as an external,
 the determinism rule would reject the fragment, and the hatch would
 silently not engage — no error and no reason in `explain()`.
-Registration is keyed by `contentKey(fragment)`
-so identical fragments share one registration, and the planner MUST
+Registration is keyed by `semanticKey(fragment)` (the
+order-insensitive identity from `@jarenjs/core/object`) so identical
+fragments share one registration, and the planner MUST
 produce a correct plan with the capability disabled (tested that way).
 Preference order: native SQL → deterministic function → residual, and
 `explain()` names the choice. **Index-form UDFs are deliberately not
@@ -585,11 +866,15 @@ UDF-expression indexes.
 ### The statement cache
 
 A caller of the core primitives, not an eighth implementation:
-`createBoundedCache` keyed by `contentKey(document)` plus collection,
-dialect and strictness. `contentKey` is the memo-grade key
-(`hashContent(stableStringify(x) ?? '')` — drops `undefined` members,
-no cycle guard; both properties acceptable for a cache key), never
-`canonicalizeJson` (signature-grade, throws on `undefined`).
+`createSemanticCache` keyed by the whole discriminating tuple — the
+document plus collection, dialect, strictness, the pushdown switch and
+the profile — where the identity is the tuple's COMPLETE
+serialization, never a fingerprint of it: a 32-bit content hash
+collides after tens of thousands of documents, and a collision here
+answers one query with another query's plan and rows (the
+cache-identity test exists to keep that key abolished).
+`store.stats()` exposes hits, misses and evictions so the cache is
+proven rather than assumed.
 `store.stats()` exposes hits, misses and evictions, so the cache is
 proven rather than assumed.
 
@@ -621,8 +906,12 @@ diff engine and maps operations to minimal statements (column writes,
 fallback); inserts batch parent-first, deletes run child-first,
 foreign-key cycles among the changed set are `JD0040`, and a declared
 `version` property turns every update into an optimistic
-`WHERE version = ?` with `JD2040` on conflict. The tracker mutates
-ONLY after commit: a failed save retries.
+`WHERE version = ?` with `JD2040` on conflict. A save that FAILS leaves
+the tracker exactly as it was, so it retries; a save whose statements
+all succeed advances at once — inside an enclosing transaction too,
+where the database already holds the rows — and registers the exact
+withdrawal of that advance on the owning scope, whose rollback takes it
+back so the retry plans the same statements again.
 
 ## The migration engine, relationally (`src/migrate.js`, `src/cli.js`)
 
@@ -711,22 +1000,40 @@ re-query to keep a promise rather than a number.
 
 ## Durable runs and the job queue (`src/jobs.js`, `src/dag-job.js`)
 
-The queue's whole correctness story is one guarded statement: the
-claim UPDATE selects the earliest eligible row (pending, failed, or an
-expired lease — so recovery IS the next claim, not a sweeper) whose
-kind the worker registered, sets it leased with a deadline, and
-RETURNs it. One statement is one transaction, so no two workers claim
-the same job without any distributed lock; every later transition
-wears `state='leased' AND lease_owner=?`, making execution
-at-least-once and completion exactly-once (proven by four workers on
-four connections over one WAL file). The `@jarenjs/flow` composition
-injects `compileDag` (db never imports flow — an import-graph test
-enforces it) and binds a per-job checkpoint store; a DAG run's
-completion records the result, marks the job done and prunes the
-checkpoint rows in ONE transaction, so a crash resumes from its
-checkpointed nodes rather than restarting. Non-goals stated plainly: a
-shared SQLite file over a network filesystem is not a safe
-coordination substrate.
+The queue's correctness story has two halves. The claim UPDATE selects
+the earliest eligible row (pending, failed, or an expired lease — so
+recovery IS the next claim, not a sweeper) whose kind the worker
+registered, MINTS the fence — a fresh opaque `lease_token` and the next
+`lease_generation` — sets the row leased with a deadline, and RETURNs
+it. One statement is one transaction, so no two workers claim the same
+job without any distributed lock. Every later settling transition —
+renew, checkpoint save, complete, fail — then wears the fence:
+`state='leased' AND lease_token=? AND lease_until > ?`, the token plus
+CURRENT validity, never the owner (one worker reuses one owner for
+every attempt it ever makes, so an owner cannot say which attempt is
+speaking). A lease is immutable — `renew` answers a replacement and the
+superseded token settles nothing — and a guard that matches nothing is
+a coded refusal naming which of the three reasons applied (`JD2065`
+settled/unknown, `JD2066` superseded, `JD2067` expired), never a silent
+`false`. A renewal that fails for any OTHER reason is a storage problem
+the next renewal may survive; only the three fence codes prove loss.
+Checkpoint rows are stamped with the attempt's generation: `load` reads
+up to it, and a settlement prunes no further, so a stale attempt cannot
+erase a live one's work. Execution stays at-least-once; settlement is
+exactly-once AGAINST THE STORE (proven by four workers on four
+connections over one WAL file) — an external effect a handler already
+made is still the handler's to make idempotent. Ownership follows the
+handle: root `store.jobs.*` and all worker control I/O take the store
+gate, while `tx.jobs` runs as its exact transaction scope — the
+transactional outbox. The `@jarenjs/flow` composition injects
+`compileDag` (db never imports flow — an import-graph test enforces it)
+and binds a per-ATTEMPT checkpoint store that follows the current
+lease; a DAG run's completion records the result, marks the job done
+and prunes the checkpoint rows in ONE transaction, so a crash resumes
+from its checkpointed nodes rather than restarting, and a resume must
+match the persisted workflow revision and input hash (`JD2069`).
+Non-goals stated plainly: a shared SQLite file over a network
+filesystem is not a safe coordination substrate.
 
 ## The wasm driver (`src/drivers/wasm.js`)
 
@@ -746,3 +1053,45 @@ studio's throwaway-store operation, holds the wasm build to the
 JavaScript engine's recorded answers in every one of those engines,
 OPFS or not, because an entry seeds its own store and needs execution,
 not persistence.
+
+## Operating the store (`src/pragmas.js`, `src/maintenance.js`, `src/backup.js`, `src/cancellation.js`, `src/errors.js`)
+
+The operability surface is five small modules and one rule each,
+consumed by `src/store.js` and never reached by a query:
+
+- `src/pragmas.js` — the closed table of configurable connection
+  pragmas (option name, SQL spelling, validator, read-back mapping,
+  default, the two store-kind flags). The open sequence applies the
+  requests in table order and reads every declared pragma back; an
+  explicit request the engine did not take refuses the open, a default
+  it could not take is reported as read. The dialect spells exactly one
+  set statement and one read statement for it.
+- `src/maintenance.js` — `checkpoint`, `integrityCheck`,
+  `foreignKeyCheck`, `optimize` over a connection, answering the engine's
+  own row as typed data; availability per operation is the binding's
+  declaration and, for the two that write, the store's read-only flag —
+  refused by code exactly where the report says `false`.
+- `src/backup.js` — the online backup over the driver's primitive triple
+  (`copy`, `rename`, `remove`; the Node binding's), written to a
+  temporary sibling and renamed only at verified completion, cleaned up
+  on every other branch. The root closure imports no builtin: the
+  primitives are the driver's.
+- `src/cancellation.js` — the one check every lifecycle runs between its
+  units of work (a statement, a row, a step, a batch, a page), on the
+  clock the caller was given; each lifecycle owns its abort code, the
+  deadline code is one. `capabilities.cancellation` states the
+  granularity per lifecycle and that nothing interrupts mid-statement.
+- `src/errors.js` — `classifyDriverError`, the one table every path
+  consults (writes, the query engines through their boundary and the
+  cursor's `wrap`, maintenance and backup under their own code, the open
+  sequence): class, retryability, code. The query path's only special
+  case is the int64 overflow of a pushed aggregate, answered by the
+  engine as a coded residual.
+
+The open path itself creates or verifies the shape inside an IMMEDIATE
+transaction with idempotent DDL (one dialect spelling, applied on the
+open path only) and retries once on a classed busy failure, so two
+processes creating one fresh file never meet the deferred-upgrade
+`SQLITE_BUSY` the busy handler cannot retry. The job queue's
+administration (`page`, `cancel`, `requeue`, `sweep`) lives in
+`src/jobs.js` beside the fence it authorises through (JOBS-FORMAT §10).