@jarenjs/db 0.73.0 → 0.83.2

package/ARCHITECTURE.md

@@ -177,9 +177,19 @@ stable code, an object and a detail, sorted, once:
 | `unmapped-table` | a table with neither a document column nor a key |
 | `unmapped-column` | a generated column whose expression is not a member path this dialect wrote |
 | `unmapped-index` | an index over an expression, a predicate, or a column no member path explains |
+| `unmapped-constraint` | a CHECK that is not a complete scalar enum of a mapped entity column |
 | `unmapped-type` | a column type no schema type maps back from — the member is derived untyped |
 | `ambiguous-relation` | a foreign key says which entity it points at; which side declared the edge, and whether the other holds many, is not in the shape |
 
+Scalar entity enum CHECKs are recovered on both dialects. SQLite reads its
+stored CREATE text; PostgreSQL reads `pg_constraint` expressions, including
+its `ANY (ARRAY[...])` rendering and numeric literal casts. The parser accepts
+only the entire enum expression. Unknown predicates, NULL lists, non-binary SQLite collations, nondeterministic
+PostgreSQL collations and casts whose meaning is not proven remain `unmapped-constraint`. Partial indexes and
+expression terms retain their catalog flags; they are never converted into
+unconditional uniqueness. Optional `introspect.checks`/`readChecks` hooks leave
+injected dialects compatible with the earlier catalog surface.
+
 `strict: true` refuses instead of returning a partial model, because a
 caller about to diff the result against a declared model needs to know
 the difference is real.
@@ -351,7 +361,8 @@ A single-binding FLWOR over the collection (`$for: { <name>: '$[*]' }`
 — the examples here write `it`, but the binding is the **document's** to
 name and nothing translates differently under another one)
 with: comparison predicates (`$eq $ne $lt $le $gt $ge`) between a
-singular member path and a literal or external; `$and`/`$or`/`$not`
+singular member path and a literal or external, or two paths in the same
+non-null number/string family; `$and`/`$or`/`$not`
 composition; `$exists`/`$empty`; `$starts-with`/`$ends-with`/
 `$contains` on schema-typed string paths with literal patterns;
 `$orderby` over singular schema-typed paths (`$dir`, `$empty`, no
@@ -359,11 +370,33 @@ collation); a top-level `$subsequence` window with literal bounds; the
 top-level aggregates `$count` and `$sum`/`$avg`/`$min`/`$max` over a
 singular schema-typed path; the whole-document projection that returns
 the bare binding; ONE member path, projected as its value beside its
-JSON type; and a nested SHAPE of objects, arrays, literals and member
+JSON type; and a nested SHAPE of objects, arrays, literals, whole collection bindings and member
 paths, projected as one value/type pair per distinct leaf and rebuilt
 by the decoder — never by parsing a JSON text the database assembled,
 which could not tell an absent member from a present `null`.
 
+Constant projection trees fetch a row marker instead of a document. A
+window over a single path removes absent members before its SQL limit;
+a window over an opaque projection stays set-residual because that projection
+may emit zero or several items per source row.
+
+General scalar grouping preserves absent and null keys separately through
+value/type pairs. Null and boolean keys may group, but ordering them stays
+residual to preserve `JQ2005`. Explicit group ordering accepts orderable keys and proven count/min/max
+expressions, including aggregates absent from the return; first appearance
+breaks ties. SUM/AVG ordering remains residual: compensated or differently
+ordered SQL accumulation can change which group sorts first. Literal windows lower over singleton group constructors;
+`$count` over such constructors with only row-count aggregates counts a grouped
+subquery. `$distinct` over a typed scalar projection uses the same key relation, with
+absent members removed first. Unordered inputs retain first appearance; an
+ordering composed solely of the projected path lowers to group-key order.
+
+Entity projection trees, constants, counts and single-path windows follow the
+same cardinality rules. An equijoin graph can additionally filter through
+boolean trees whose leaves belong to individual bindings. The entity emitter
+resolves each leaf's binding alias before composing the total predicates;
+a disjunction never supplies a mandatory join edge.
+
 Plus the spatial predicates a **derived** index makes decidable:
 `$bbox-intersects` against a literal or external region, a geohash
 prefix no longer than a `derive: 'geohash'` column's precision, and the
@@ -396,15 +429,16 @@ mode, `knn`, beside native, row and set.
 | construct | reason |
 |---|---|
 | `$let` bindings, `$fold`, positional/window bindings | no equivalence proof exists yet; residual by default |
-| a `$groupby` whose key is untyped or nullable, whose `$return` reads the binding, or whose `$orderby` names anything but a key | SQL's grouping and the engine's need not agree on an untyped key; after a grouping the binding holds the group's ROWS, which an object member cannot take |
-| a window over the GROUPS, or an aggregate of them | the plan groups whole; a `LIMIT` over the groups would cut a different set |
+| a `$groupby` whose key is untyped, whose `$return` reads the binding, or whose `$orderby` names anything but an orderable key or count/min/max | SQL's grouping and the engine's need not agree on an untyped key; after a grouping the binding holds the group's ROWS, which an object member cannot take |
+| a window over a group return that may omit an item, or a group aggregate beyond the proven constructor count | SQL group cardinality must equal the projected item cardinality; numeric and error semantics need their own proof |
 | a `$for` binding nothing joins to — a cartesian product | the engine builds the product; a plan that emitted one by accident is the thing an equi-join graph exists to prevent |
 | non-singular path expansion | one relation per binding in this version |
 | `$match` and other unlisted operators, `$call` | no native spelling proven equivalent |
 | `$orderby` with a `$collation` | a collation the dialect cannot reproduce is refused, not approximated |
-| a projection the tree cannot rebuild: an operator over a member, a reference to the binding itself, a non-singular path, a projection with no member path at all | the WHOLE projection runs per row (the row residual) — pushed, ordered and windowed rows, projected by the engine; promoting the part that composes would answer a shape nobody asked for |
+| a projection the tree cannot rebuild: an operator over a member, a reference to the binding itself, a non-singular path | the WHOLE projection runs per row (the row residual) — pushed and ordered rows, projected by the engine; a window over these items runs in the set residual; promoting the part that composes would answer a shape nobody asked for |
 | string operators with an external pattern | the pattern's type is unknowable at plan time and the engine ERRORS on non-string patterns |
-| comparisons where both sides are paths | join territory |
+| path comparisons with untyped, nullable, boolean or differing comparison families | the total typed comparison proof does not cover these shapes |
+| `$distinct` over untyped/compound projections or ordering by other paths | first-occurrence order and structural equality need additional lowering |
 | array/object literals in comparisons | deep-equality has no guarded native form |
 | `$within` over a `derive: 'bbox'` column | a bounding-box pre-filter is pushed; exact containment refines in the engine |
 | a bounded `$distance` over a `derive: 'bbox'` column | a geodesic-circle box pre-filter is pushed; the exact distance refines in the engine |
@@ -788,7 +822,7 @@ exactly what an honest explain may not print.
 ### The two residual modes
 
 - **Row residual** — only the projection is untranslated: predicates,
-  ordering and the window are fully pushed; each fetched row runs
+  and ordering are fully pushed, with no output window; each fetched row runs
   `{ $for: { <the document's own binding>: '$[*]' },
   $return: [ <the document's $return> ] }` (the array wrapper keeps
   array-valued items unambiguous) and the items concatenate in row
@@ -1116,3 +1150,32 @@ by refreshing affected parents from bounded leaves. Source and result payloads
 consume row and byte credits; unsupported shapes keep named reruns. See
 [the replication contract](docs/REPLICATION-FORMAT.md) and
 [the strategy matrix](docs/LIVE-FORMAT.md).
+
+Parameterized distance bounds use the closed `circleAxis` derived parameter:
+`{ kind: 'circleAxis', centre: { external } | { literal }, radius: { external }
+| { literal }, axis: 'w' | 's' | 'e' | 'n' }`. Both inputs are named in every
+edge slot. The binder uses the shared geographic kernel and diverts the whole
+query for missing/invalid values, negative radii, polar or antimeridian boxes.
+The exact distance predicate still refines the candidates. The shared oracle
+covers SQLite column/R*Tree indexes and PostgreSQL, including repeated cached
+calls with different bound values.
+
+
+## Explicit relational adoption
+
+`introspect.js` owns physical inventory independently of model derivation.
+`physical.js` compiles column codecs and verifies declarations against that
+inventory. The existing entity core, tracker and graph row merger execute both
+hybrid and column layouts; there is no separate relational store. The query
+planner reports decoded evaluation for physical codecs, and refuses physical
+keyset continuation until its identity semantics are qualified.
+
+`sql.js` binds trusted statements to `store.js` transaction views. It reuses the
+read classifier's tokenizer and the driver's scope owner. Writes invalidate all
+clean tracked entities; pending edits and incomplete capture populations refuse.
+`invariants.js` uses the shared Query compiler for store rules; the dialect lowers
+a bounded database subset into ordered trigger bodies. `migrate.js` reuses its
+existing rebuild/receipt transaction and verifies preservation before publication.
+The backup publisher remains shared by Node online and Bun serialized snapshots.
+
+`src/search.js` composes core lexical mechanics and the JSON predicate compiler over complete bounded entity snapshots. Committed capture and data-version checks invalidate derived state; SHA-256 source content validates persisted caches across reopen. Snapshot storage uses existing collection transactions. See [search execution](docs/SEARCH.md); native full-text dialects remain unqualified.

package/README.md

@@ -1,5 +1,7 @@
 # @jarenjs/db
 
+For existing-file adoption with native queries, receipts and jobs, start with the [combined public recipe and evidence ledger](../../docs/ADOPTION-EVIDENCE.md). Adopted-trigger capture, physical keysets and PostgreSQL subsystem gaps retain their documented refusals; synthetic SQLite proof does not retire downstream SQL.
+
 Documents AND entities in SQLite. A **model document** declares
 collections (a JSON Schema, a key, indexes) and — since phase B —
 **entities**: keys, typed columns, relations, defaults and an
@@ -214,7 +216,7 @@ no continuation to emit.
   `explain().projection` names the path or the leaf `paths`, and
   reading a shape no longer reads every document. A shape the tree
   cannot rebuild — an operator over a member, a reference to the
-  binding itself, a projection with no path at all — refuses WHOLE and
+  whole entity binding, a non-singular path — refuses WHOLE and
   runs per row, with `explain().residualProjection` naming what stayed
   behind: promoting the half that composes would answer a shape nobody
   asked for. Every `explain()` also carries `budget`: the profile
@@ -227,6 +229,10 @@ no continuation to emit.
   arithmetic deviation declared rather than hidden (MODEL-FORMAT §10.6: SQLite's
   compensated `SUM` and the engine's naive one differ in the last
   bit). `strict: true` turns any residual into a compile error.
+  Constant trees fetch only a row marker. Counts over entity projection trees
+  lower too; a single-path count or window excludes absent members before
+  counting items. A window over a projection that can emit several items runs
+  in the engine.
 - **Registered operators, correct in the residual, pushed where it
   pays.** Open with a registry (`operators:
   createJsltRegistry().use(mathPack).use(financePack)`) and a query may
@@ -254,6 +260,12 @@ no continuation to emit.
   `openStore` by name — and the promotion still needs a numeric member
   the schema forbids `null` on, because SQL cannot tell a stored `null`
   from an absent one and the engine can.
+- **Typed path comparisons and distinct scalar projections.** Comparisons
+  between two paths in the same non-null number/string family lower without
+  a UDF. `$distinct` over a typed scalar projection groups by its value and JSON
+  type, excluding missing members. Unordered inputs retain first occurrence;
+  ordering solely by the projected path also lowers. Literal windows apply
+  to the distinct items.
 - **Grouping and joins lower whole, or not at all.** A `$groupby` over
   schema-typed member keys becomes a real `GROUP BY`: the keys come back
   with their JSON types beside them, so a group whose key is ABSENT
@@ -262,13 +274,20 @@ no continuation to emit.
   in SQL under the ENGINE's empty rules (`0` for a count or a sum, no
   member at all for the other three), and the groups come out in the
   engine's own order of first appearance unless an `$orderby` over the
-  keys says otherwise. Entity queries join any number of bindings: every
+  keys or count/min/max says otherwise. Entity queries join any number of bindings: every
   binding past the first must be attached by a column equality to one
   already joined, which is what makes the plan a nested loop the engine
   can be compared against — a binding nothing attaches would be a
   cartesian product, so it is the residual, named, and `strict: true`
   refuses it. `explain()` lists the join order with the equalities that
   attached each binding, and the group's keys, aggregates and order.
+  Nullable scalar keys also group; ordering null or boolean keys remains
+  residual to preserve the engine's error. Windows over singleton group
+  constructors lower, and counts over constructors containing only keys,
+  literals and row counts count the grouped subquery. Ties under a partial
+  group ordering retain first appearance.
+  Boolean predicates may span joined bindings when each leaf belongs to one
+  binding; the equality graph still establishes every join.
   A join predicate that is not an equality — a range between two mapped
   columns of one family — refines a match it never makes: the anchor is
   still an equality, so a range alone stays the residual. A projected
@@ -1177,6 +1196,13 @@ be declared (`unmapped-view`), a foreign key cannot say which side
 declared the edge (`ambiguous-relation`). `strict: true` refuses rather
 than answering a partial model.
 
+Scalar enum CHECKs on mapped entity columns are recovered from both
+catalogs. A partial index or an expression term is reported as `unmapped-index`
+without inventing unconditional uniqueness. CHECKs outside the complete scalar
+enum grammar, including NULL lists, collation-dependent equality and unproven
+type conversions, remain
+`unmapped-constraint`.
+
 The derived model is usable as a migration's `from`: read a database,
 plan against your declared model, and an unchanged shape plans nothing.
 SQLite and PostgreSQL derive the same logical model from equivalent
@@ -1309,10 +1335,10 @@ its side-effect-free status read are in
   capture after a disposable live probe. Failed session probes select journal
   capture. OPFS, IndexedDB snapshots and memory have explicit capability and
   durability differences; see [execution hosts](docs/HOSTS.md).
-- **Named future work, not silent gaps**: `$groupby` pushdown beyond
-  the `$time-bucket` ladder, a many-to-many hop on the chain, membership
-  on an auto-keyed pending insert, additional join/group shapes, other SQL dialects,
-  transport policy, database introspection (MODEL-FORMAT §10.6, the roadmap).
+- **Remaining boundaries**: untyped and opaque grouped/projection shapes,
+  federation beyond two sources, richer incremental live queries, portable
+  capture and introspection outside the model vocabulary (MODEL-FORMAT §10.6
+  and the roadmap).
 
 The normative formats are
 [docs/MODEL-FORMAT.md](docs/MODEL-FORMAT.md) (storage §§1–7, safe
@@ -1333,6 +1359,7 @@ Every subpath a consumer can import, derived from the manifest by
 <!--fact:exports.db-->
 | Import | Kind | Declarations |
 |---|---|---|
+| `@jarenjs/db/search` | JavaScript | declared |
 | `@jarenjs/db` | JavaScript | declared |
 | `@jarenjs/db/node` | JavaScript | declared |
 | `@jarenjs/db/postgres` | JavaScript | declared |
@@ -1342,6 +1369,7 @@ Every subpath a consumer can import, derived from the manifest by
 | `@jarenjs/db/app` | JavaScript | declared |
 | `@jarenjs/db/schemas/jaren-migration.draft-07.schema.json` | schema | — |
 | `@jarenjs/db/schemas/jaren-migration.schema.json` | schema | — |
+| `@jarenjs/db/schemas/jaren-model.authoring.schema.json` | schema | — |
 | `@jarenjs/db/schemas/jaren-model.draft-07.schema.json` | schema | — |
 | `@jarenjs/db/schemas/jaren-model.schema.json` | schema | — |
 | `@jarenjs/db/schemas/jaren-replication-snapshot.draft-07.schema.json` | schema | — |
@@ -1368,3 +1396,38 @@ The output is one atomically published collection bundle; read it with
 `--format collections`. Assertion plans expose provider, ordered-fold and
 bounded materialization strategies through `onAssertionPlan`.
 See [MIGRATION-FORMAT §6 and §11](docs/MIGRATION-FORMAT.md).
+
+Collection constructor projections can include a whole row beside member
+paths. Group ordering lowers count/min/max expressions, including aggregates
+used only for sorting; SUM/AVG ordering remains in the engine to preserve
+floating-point accumulation order. Parameterized distance bounds bind both
+centre and radius through the spatial index, with full-query fallback for
+invalid, polar and antimeridian probes.
+
+Live collection groups now maintain multiple keys, unordered typed scalar
+distinct projections, and count/sum/avg/min/max over canonical group results.
+Only affected groups are reevaluated in source order; a final aggregate folds
+the retained group outputs in first-appearance order. Input documents and
+group outputs consume both state-entry and byte credits. See
+[LIVE-FORMAT](docs/LIVE-FORMAT.md) for the supported shapes and measured costs.
+
+
+## Existing relational SQLite files
+
+Use `readSchema(connection)` for physical inventory, then declare an entity's
+`physical` table, ordered keys and column codecs and open with `{ adopt: true }`.
+Opening verifies the existing shape and emits no DDL. Ordinary column tables need
+no document column; integer identities, exact hexadecimal BLOBs, database defaults
+and read-only views have explicit contracts in [MODEL-FORMAT](docs/MODEL-FORMAT.md#12-existing-column-layouts).
+
+Inside `store.transaction`, `tx.sql.prepare(text, { access: 'read' | 'write' })`
+shares the entity/outbox connection and savepoint owner. Statements expire with
+the scope. [The client recipe](../linq/docs/DB-CLIENT.md#trusted-sql-during-adoption)
+documents trust, invalidation and synchronous execution. Schema changes use
+`planPhysicalMigration` and `migrate`; [preservation and forward recovery](docs/MIGRATION-FORMAT.md#existing-physical-files-and-forward-recovery)
+require explicit dispositions and assertions. `planInvariants` supplies declared
+SQLite constraint/audit triggers for installation through that migration boundary.
+
+Native column reads and bounded mutation documents are specified in [NATIVE-PLANS](docs/NATIVE-PLANS.md), including SQL census coverage, resource accounting and refusals.
+
+`@jarenjs/db/search` composes the resident ranker with bounded authoritative entity reads and optional atomic snapshot storage. See [persisted search](docs/SEARCH.md).

package/docs/HOSTS.md

@@ -272,3 +272,20 @@ Checked and dropped:
 - Read-only WAL workers do not accept writes, nested scopes do not migrate,
   transient cursors do not exhaust the statement cap, and failed writes are not
   automatically replayed. The lifecycle and fault corpus exercises each boundary.
+
+
+## Existing-file adoption and backup
+
+Node and Bun qualify explicit SQLite column mappings and scoped prepared SQL.
+The synchronous transaction API exists only when the driver's observed
+`synchronous` capability is true; worker and other async-only hosts expose no
+sync twin. Physical adoption on PostgreSQL refuses pending a separate mapping
+and codec qualification. Unknown application-trigger effects do not qualify
+capture or replication; those combinations refuse before an adoption claim.
+
+Node backups use the built-in online snapshot. Bun uses `Database.serialize()`
+under the store gate, writes and flushes a sibling temporary, then uses the shared
+atomic publisher. Both include committed WAL. Bun's snapshot holds the whole
+image in memory and cancellation takes effect between phases. Process-kill tests
+cover rebuild copy, table drop, commit and backup publication on both hosts;
+these tests do not establish power-loss durability or native executable packaging.

package/docs/JOBS-FORMAT.md

@@ -504,3 +504,29 @@ a cancellation policy, a retry policy are the host's.
 only by `cancel()`; `counts()` reports it. A transaction view's `jobs`
 (the outbox, §3) carries none of the four: an administration call is a
 root call.
+
+## Business receipt and external-effect composition
+
+`jobs.assertLease(lease)` checks current execution authority without a write or
+renewal. It uses the same token/expiry guard and `JD2065`/`JD2066`/`JD2067` refusals
+as settlement; malformed authority is `JD2068`. Inside `tx.jobs`, this read and
+subsequent mapped writes share the transaction lock. Workers receive
+`context.lease()` to read their current token after automatic renewal.
+
+`createWorker({ effectSafety(job, context), ... })` optionally gates each handler
+admission. Only `true` admits; any other resolved value pauses the queue attempt
+in its existing `cancelled` state without running the handler. The worker
+`context.pause()` exposes the same operation. An authorized operator can
+explicitly `requeue` and claim it for reconciliation; it never becomes an
+unreclaimable completed job merely because its external outcome is unknown.
+A thrown policy error follows the ordinary failure path and is checked again on
+a later attempt. `createDagJobRunner` forwards the same hook. The hook supplements
+the mapped effect store's mandatory pre-dispatch fence and durable intent; it is
+not a replacement for them.
+
+Business receipts and external intents belong to application-mapped tables via
+`@jarenjs/linq/db`. Job reset/sweep operate on queue/checkpoint rows only and do
+not erase those facts. An expired lease, reset or retryable job failure cannot
+provide permission to resend unresolved external writes. Local co-commit does
+not mean exactly-once remote delivery. See the contract package's
+[durable composition](../../contract/docs/DURABLE.md).

package/docs/LIVE-FORMAT.md

@@ -192,11 +192,11 @@ Choosing how much history to keep is the host's decision
 (`retention`); what the reader owes is that when rows go, it reports
 the gap instead of hiding it.
 
-**Replication is not built here**, and this log alone does not make
-it safe: there is no conflict resolution, no site identity, no causal
-ordering across writers. The bounded reader with its watermarks and
-its explicit gap is the precondition a replication protocol would be
-built on — not the protocol. That sentence is the whole claim.
+**The log alone is not replication.** Replica identity, causal frontiers,
+durable receipts, conflicts and bounded reset snapshots belong to the opt-in
+[replication subsystem](REPLICATION-FORMAT.md). Its protocol builds on these
+watermarks; a change-log cursor by itself supplies no conflict policy or causal
+ordering across writers.
 
 ## 6. Cross-connection behaviour and non-claims
 
@@ -208,8 +208,8 @@ which changes when ANOTHER connection commits; poll it and treat a
 change as "re-read what you care about". Cross-tab delivery is §7's
 story (the live-query layer).
 
-Non-claims, in one place: no replication, no conflict resolution, no
-capture of writes made by other connections, no capture on stores
+Capture-layer non-claims: no conflict resolution or capture of writes made
+by other connections, no capture on stores
 opened without `capture`, and no statement-level ordering within a
 commit (§2).
 
@@ -255,9 +255,8 @@ the maintenance are this table's — an entity chain re-runs, declared —
 and this document stays the only place they are decided.
 
 **This table is normative.** Every row is implemented and tested;
-nothing outside it is attempted. Classification reads the compiled
-PLAN (never the raw document), so "extractable" below means exactly
-what the pushdown planner already means by it.
+nothing outside it is attempted. Classification combines canonical document shapes with the compiled selection
+plan; "extractable" below means what the pushdown planner already proves.
 
 | Construct (as planned) | Strategy | Maintained state |
 |---|---|---|
@@ -265,7 +264,10 @@ what the pushdown planner already means by it.
 | the same with a per-row `select` projection (row-mode plan) | **incremental rows**: the affected row alone is recomputed; a source row may project to several items | result rows, grouped by source key |
 | `orderBy` over extractable paths, optional `limit`, offset 0 | **maintained window**: a sorted structure; ties broken by the collection key, appended as the final sort term; an insert sorting beyond a full window is a no-op | the window rows and their sort keys |
 | whole-query `count` / `sum` / `avg` / `min` / `max` (the plan's aggregate), optional `where` | **running accumulator** plus a per-row contribution map — a delete can only be answered from retained contributions (§3: a `remove` carries no old value). `min`/`max` removal of the last extremum holder FALLS BACK to a recompute over the retained contributions; the accumulator alone cannot answer, and this fallback is the documented cost | one contribution per matching row |
-| single-level `groupBy` with aggregate returns, in the canonical form below | **per-group deltas**: the accumulator machinery, one instance per group; groups appear in first-appearance order, exactly the engine's order | per-group, per-row contributions |
+| single-level `groupBy` with one or more keys and canonical returns below | **group maintenance**: reevaluate affected groups through the engine in source-row order; first surviving source occurrence determines group order | source documents and group outputs, with entry and byte credits |
+| unordered SQL-native typed scalar distinct projections | **distinct maintenance**: keep the source holders of each value; deleting its earliest holder may move the value in first-occurrence order | source documents and unique outputs, with entry and byte credits |
+| count/sum/avg/min/max over canonical unwindowed groups | **group maintenance** followed by an engine fold over the retained group outputs in first-occurrence order | source documents and group outputs, with entry and byte credits |
+| ordered/windowed distinct or other aggregates over groups | **re-run on invalidation** | the previous result, for diffing |
 | `where` whose spatial predicate is **refined** (the plan pushed a bounding-box or cell-range pre-filter and left the exact `$within`, bounded `$distance` or over-long cell prefix to the residual — `explain().prefilters` with `exact: false`), no order, no aggregate; optional per-row `select` | **incremental rows** — the geofence: the initial fetch narrows through the derived index, and every touched row is re-evaluated by the engine's EXACT predicate, so a point emits `add` when it enters the region, `remove` when it leaves, and nothing while it moves within (a whole-document return sees a `replace` carrying the new position) | the result rows |
 | a refined spatial predicate over a collection with **no document key** (`key: null`, rowid identity) | **re-run on invalidation** — the per-row strategy tracks a row by its declared key, and a rowid is not one; the reason says `rows without a document key cannot be tracked` | the previous result, for diffing |
 | `orderBy` beside a refined spatial predicate — over `$distance` (not a path) or over a member (the set residual drops the planner's order terms) | **re-run on invalidation**, the ordering named as the reason | the previous result, for diffing |
@@ -326,8 +328,8 @@ nested groups have a separate bounded strategy below):
 
 After `$groupby`, `$it` is the group's item sequence and `$g` its key;
 return members are the group key or an aggregate over `$it` (a path
-below it selects the aggregated member). Anything else in the return
-is not canonical and re-runs.
+below it selects the aggregated member). A bare key or a single aggregate is also canonical; other return shapes
+re-run. Multiple declared keys follow the same rule.
 
 ## 8. Invalidation
 
@@ -657,3 +659,40 @@ Count, sum, average, minimum and maximum recompute from only the affected
 parent's bounded leaves. An offset, an unsupported operator, a global input to
 the nested group, or a group-of-groups LINQ emission remains a named rerun.
 Replicated writes enter the same committed capture stream as local writes.
+
+### Group maintenance details
+
+Canonical single-level groups accept one or more key expressions over one
+collection binding, an optional fully translated filter, and either a bare
+key, one count/sum/avg/min/max expression, or an object containing key,
+`$default: [key, null]`, and aggregate members. Global-root reads, ordering,
+and windows remain rerun shapes. Aggregate expressions are evaluated by the
+query engine, including empty results and errors; maintenance does not
+substitute JavaScript arithmetic for query operators.
+
+Keys preserve structural equality and distinguish missing from null. Source
+holders retain physical row positions: changing a holder preserves its
+position, and deleting a group's earliest holder may move that group's
+output. Unaffected group outputs retain reference identity. Only affected
+groups are reevaluated, followed by an optional final aggregate over all group
+outputs. This costs work proportional to the affected groups plus group-output
+ordering/folding, rather than constant-time arithmetic deltas.
+
+`live.maxMaintained` counts source holders and group output items;
+`live.maxBytes` counts their serialized documents. This includes document
+members omitted from a small aggregate result. Exceeding a bound refuses
+registration or closes an active query with `JD2060`; the last delivered
+result remains unchanged. Engine failures follow the same invalidation path.
+
+`node --expose-gc benchmark/changeflow.js` compares every mutation against a
+fresh query and a patch-only consumer. The
+[generated comparison table](REPLICATION-FORMAT.md#measurements) includes
+multiple-key groups, an aggregate of groups and distinct beside forced reruns.
+It reports initialization allocations and both median and tail mutation time;
+heap deltas before collection are not precise retained-state sizes. The fixture
+has 200 initial rows and 15 mutations per case, so these measurements establish
+correctness and costs for that fixture rather than a universal crossover.
+
+## Structural range adapter
+
+`@jarenjs/linq/db` exposes bounded ranges over captured hybrid entity roots, using the existing keyset pager and committed observer. Its live event is an explicit source reset with monotone revision; this does not promote offset windows or the remaining incremental shapes. See [COLLECTION-PROVIDER](../../app/docs/COLLECTION-PROVIDER.md) for credits, snapshot identity and disposal. Maintained top windows check offscreen rows against their bound and release retained state on close.

package/docs/MIGRATION-FORMAT.md

@@ -598,3 +598,37 @@ difference between a puzzled afternoon and a five-minute fix.
 Down migrations REMAIN a non-goal (§7's reasoning is unchanged): a
 down migration is a data-loss generator wearing a seatbelt; recovery
 is a backup restored plus the forward chain.
+
+## Existing physical files and forward recovery
+
+`planPhysicalMigration(connection, fromModel, toModel, options)` records the
+source schema and explicit DDL/SQL/rebuild steps as an ordinary migration document.
+`options` supplies an `id`, `steps`, a disposition for every source `type:name`
+(`preserve`, `replace`, or `drop`), and optional `{ sql, params, expected }`
+preservation assertions. Assertions are SELECTs evaluated before and after the
+steps. They should cover committed identities, exact BLOB hex and application
+history facts. Unknown objects cannot disappear without a declared disposition.
+Automatic hybrid model diffing refuses column layouts; an explicit plan is required.
+
+Apply through `migrate(target, [plan], { baseline, model, shadow: false })`.
+A physical plan must be qualified on an explicit backup and fresh-target fixture;
+an empty model-generated shadow cannot recreate the original file's application
+programs. The runner uses its existing immediate transaction, ordered steps,
+checksummed receipt and FK checks. It checks the source schema before destructive
+steps and preserved objects and assertions before publication. Target mapped
+columns and declared invariant triggers are verified. A changed source is
+`JD0020`, a lost object/fact is `JD0023`, and an edited applied receipt is `JD0022`.
+
+Migration history is created inside the applying transaction only when needed.
+An identical second run executes no DDL or DML. Failed steps and failed commits
+roll back; after a process kill SQLite recovery leaves the source or the committed
+target. Re-running resumes from committed receipts. Forward repair plans start
+from the newest file, including later application edits; restoring an older
+backup does not qualify as forward repair.
+
+`backupTo()` publishes a sibling temporary only after a complete snapshot.
+Node uses online backup. Bun uses its native serialized SQLite snapshot under
+the store gate, then flushes and atomically renames through the same publisher.
+Bun holds a full database image in memory and cannot offer page-granular copy
+cancellation. Both snapshots include committed WAL; interruption before rename
+leaves the previous destination valid, while a leftover temporary is not published.