@jarenjs/db 0.67.0 → 0.72.2

package/ARCHITECTURE.md

@@ -258,9 +258,11 @@ re-run unnamed rather than raised.
 
 Three normalizations, each forced by the wire rather than chosen:
 `int8` and `numeric` arrive as strings (they can exceed a double) and
-become JavaScript numbers, which is the same ceiling SQLite's INTEGER
-has; `json`/`jsonb` arrive parsed, because the client's type parsers are
-the host's configuration, and the row decoder reads text; and a
+become JavaScript numbers. An `int8` outside the safe integer range
+refuses with `JD2005`; allocated collection inserts decode `RETURNING`
+inside a transaction so a refused key rolls its row back. PostgreSQL
+sequences still advance on rollback. `json`/`jsonb` arrive parsed, because
+the client's type parsers are the host's configuration, and the row decoder reads text; and a
 JavaScript boolean is bound as 1 or 0, because a boolean member is 1 or
 0 in this mapping.
 
@@ -529,7 +531,7 @@ member as an array or an object **and nothing else** — §8.14 answers
 | `{$le\|$lt: [{$distance: [<path>, <literal>]}, r]}` | the same four comparisons against `circleBounds(probe, r)` | implied | every position within `r` metres lies inside the circle's box, which the kernel computes on the same sphere and the same `EARTH_RADIUS` the engine measures with — so the two cannot disagree by model. `$ge`/`$gt` is NOT promoted: no box narrows "farther than r" |
 | `{$starts-with: [{$geohash: [<path>, k]}, "<cell>"]}`, cell length ≤ k | `<c> IN ("<cell>")` or `<c> >= "<cell>" AND <c> < successor` | **exact** | the column HOLDS `$geohash(row, k)`, and geohash is a prefix code, so a prefix test on the expression is the same test on the column |
 | the same, cell length > k | the cell truncated to k | implied | the column can only confirm its own first k characters |
-| `{$exists: {$index-of: [{$geohash-neighbours: "<cell>"}, {$geohash: [<path>, k]}]}}`, cell length = k | `<c> IN (…the nine cells…)` | **exact** | the membership test compares whole strings and the column is exactly one of them. Nine cells, never one: two points ten metres apart can differ in the FIRST character of their cell (D7), so a single prefix is bucketing and only the neighbourhood is proximity |
+| `{$exists: {$index-of: [{$geohash-neighbours: "<cell>"}, {$geohash: [<path>, k]}]}}`, cell length = k | `<c> IS NULL OR <c> IN (…the nine cells…)` | **inexact** | Unbounded members remain candidates so the row predicate preserves the empty-search-item error. An explicit `$exists` guard excludes them safely. Nine cells cover neighbours across a cell boundary; one prefix does not. |
 
 **The proof is a proof about BOXES, not about SQL**, so the physical
 mapping (MODEL-FORMAT §2.1, `physical`) does not enter it: the same box

package/README.md

@@ -297,7 +297,9 @@ no continuation to emit.
   `$within`, a `$bbox-intersects`, a bounded `$distance` or a geohash
   probe over such a collection narrows **in SQLite** through the index
   and refines **in the engine**. `$bbox-intersects` and a geohash cell
-  test are exact and need no refinement; `$within` and a bounded
+  prefix test are exact and need no refinement; neighbourhood membership
+  retains unbounded candidates and preserves the engine's empty-item error.
+  `$within` and a bounded
   `$distance` push a bounding box the truth table proves they imply,
   and the exact predicate re-runs over the narrowed candidates —
   `explain().prefilters` says which, over what columns, and whether it
@@ -431,8 +433,10 @@ how.scanNarrative;
 The region arrives as a bound parameter: a GeoJSON object is not a
 value any database can bind, so what binds is one edge of its box per
 slot, computed at bind time from the same kernel the stored columns
-came from. `$bbox-intersects` and a geohash cell test are exact and
-need no refinement; `$within` and a bounded `$distance` push the box
+came from. `$bbox-intersects` and a geohash prefix no longer than the
+indexed cell are exact and need no refinement; neighbourhood membership
+refines to preserve errors on
+unbounded candidates. `$within` and a bounded `$distance` push the box
 they provably imply and re-run the exact predicate over the narrowed
 candidates. A circle that reaches a pole or crosses the antimeridian
 pushes **nothing** — there is no single box to push — and the answer is
@@ -688,6 +692,15 @@ opinion: <!--fact:vector.ceiling-->5.142 ns per vector component — one query r
 Past that this design is the wrong tool and no margin changes it; what
 lies beyond is an approximate index, and this store does not have one.
 
+The [labelled retrieval comparison](../../benchmark/README.md#labelled-recall-and-repeated-refinement)
+also scores an optional storage adapter with approximate candidates against
+exact cosine on cached live embeddings. The AI ledger reports the adapter's
+algorithm, exhaustive flag and returned candidate count, validates candidate
+identity/vector shape, and re-scores candidates itself. The DB ledger recipe
+continues to normalize as an exact adapter. No ANN runtime dependency is added.
+
+Index decision: <!--fact:recall.annDecision-->0/6 contender rows cleared all bars; retain exact. Required exact-top-10 recall ≥ 0.95, p95 speedup ≥ 2×, and a measured exact p95 ≥ 100 ms. The largest reference corpus contains 5183 documents; scale beyond it remains unmeasured.<!--/fact-->
+
 **One document, three executors, proven to agree.** As with the spatial
 family, the k-nearest shapes of a committed corpus
 (`test/json/fixtures/vector-corpus.json`) run through the JavaScript
@@ -1028,6 +1041,10 @@ shutdown limits, the browser persistence matrix and measured latency/memory loss
   and `worker.stop()` quiesces every claim, renewal, checkpoint and
   settlement before it resolves, so closing the store releases the
   database file deterministically.
+  DAG jobs inspect persisted workflow/input/task identity before reading
+  checkpoint values. `jobs.reset(id, { expectedGeneration })` explicitly
+  discards an inactive run's checkpoints and restarts it under a new fence;
+  live leases and stale observations refuse (JOBS-FORMAT §10).
 - **The browser** (`@jarenjs/db/wasm`): the same store, the same
   queries, the same live updates run on the official SQLite wasm build
   over a probed persistence ladder: isolated SharedArrayBuffer OPFS,
@@ -1072,13 +1089,13 @@ disagree.
 <!--fact:live.eventTimeTable-->
 | view | maintained | re-run | ratio |
 |---|---:|---:|---:|
-| bucket (60 s ladder, mean), 1000 rows | 119 µs | 1.09 ms | 9.2× |
-| rolling (5 min window, mean), 1000 rows | 407 µs | 3.86 ms | 9.5× |
-| bucket (60 s ladder, mean), 10000 rows | 136 µs | 10.9 ms | 80.2× |
-| rolling (5 min window, mean), 10000 rows | 13.7 ms | 62.7 ms | 4.6× |
+| bucket (60 s ladder, mean), 1000 rows | 123 µs | 911 µs | 7.4× |
+| rolling (5 min window, mean), 1000 rows | 411 µs | 3.67 ms | 8.9× |
+| bucket (60 s ladder, mean), 10000 rows | 136 µs | 8.12 ms | 59.6× |
+| rolling (5 min window, mean), 10000 rows | 13.4 ms | 60.9 ms | 4.5× |
 <!--/fact-->
 
-The gain is <!--fact:live.eventTimeBand-->80.2× for the bucket and 4.6× for the rolling at 10,000 readings<!--/fact-->. A bucket
+The gain is <!--fact:live.eventTimeBand-->59.6× for the bucket and 4.5× for the rolling at 10,000 readings<!--/fact-->. A bucket
 view is nearly flat in the series length, because a write folds one
 bucket again and the rest of the ladder is untouched. A rolling view is
 not, and the table says so: its answer is one row per reading, so the
@@ -1335,3 +1352,19 @@ Every subpath a consumer can import, derived from the manifest by
 | `@jarenjs/db/node-worker` | JavaScript | declared |
 | `@jarenjs/db/node-pool` | JavaScript | declared |
 <!--/fact-->
+
+
+The [durable AI ledger recipe](../ai/README.md#a-durable-ledger-over-jarenjsdb)
+exposes optional atomic namespace mutation as well as its base storage and rank
+capabilities. It reads and publishes through one immediate transaction, using the
+transaction's synchronous collection facade so another SQLite connection on the
+same event loop cannot block an awaiting writer. A rejected callback changes
+neither records nor counters; ledger id minting and guarded multi-record updates
+use that boundary.
+
+Document-only migrations also accept named files:
+`jaren-db documents --migrations migrations --in users=users.json --in events=events.jsonl --out migrated.json`.
+The output is one atomically published collection bundle; read it with
+`--format collections`. Assertion plans expose provider, ordered-fold and
+bounded materialization strategies through `onAssertionPlan`.
+See [MIGRATION-FORMAT §6 and §11](docs/MIGRATION-FORMAT.md).

package/docs/HOSTS.md

@@ -6,6 +6,11 @@ capabilities are observed when the connection opens. The common oracle and
 lifecycle corpus is `test/db/store-hosts.test.js`; it checks values and coded
 errors across Node, worker, pool, wasm sessions and wasm journal fallback.
 
+Concurrent PostgreSQL first opens can race while creating the same collection
+or entity table. Each initialization transaction retries one catalog-creation
+collision after rollback, then re-reads and verifies the winning shape. Ordinary
+constraint failures remain errors, and a repeated collision refuses the open.
+
 ## Node workers
 
 ```js

package/docs/JOBS-FORMAT.md

@@ -390,7 +390,10 @@ await store.jobs.enqueue('sync-report', { input: { day: '2026-08-05' } });
 
   **The legacy rule is deterministic, and never reads unknown as equal.**
   A run checkpointed by a release that did not record task identity has
-  no `taskVersionsHash`. If it recorded no node value yet, there is
+  no `taskVersionsHash`. Identity inspection reads only the reserved metadata
+  value and probes whether other rows exist; it does not load or parse node
+  values. Missing metadata with saved values refuses `JD2069`. Refused
+  legacy upgrades leave the original identity untouched. If it recorded no node value yet, there is
   nothing that could be replayed wrongly, so the identity is upgraded in
   place and the run proceeds. If it DID record values, the implementation
   that produced them cannot be confirmed and the resume is refused,
@@ -471,6 +474,16 @@ a cancellation policy, a retry policy are the host's.
   and counts it under `stats().cancellations`; an attempt held by
   another process meets the fence at its next settling call and records
   a loss, as any superseded attempt does.
+- **`reset(id, { expectedGeneration })`** — explicitly discard one inactive
+  run's checkpoints and restart its attempts at zero. Read the job's
+  `leaseGeneration` first and pass that observation. One transaction
+  conditionally updates the job, increments its fence generation and deletes
+  its checkpoint rows. A concurrent generation change refuses `JD2066`, a
+  live lease `JD2068`, an unknown or completed job `JD2065`; no refusal
+  discards values. The next claim recomputes the workflow under current
+  identity. Reset does not undo external task effects; their idempotency
+  keys remain the host's responsibility. It is root-only administration,
+  with the same signal/deadline options as requeue.
 - **`requeue(id)`** — returns a failed, dead, cancelled or lease-expired
   job to `pending` at the clock's instant (the queue's own order). The
   attempt history is KEPT: `attempts` counts every claim the job ever

package/docs/MIGRATION-FORMAT.md

@@ -107,6 +107,39 @@ and the column entry carries the width the value is packed to:
 
 ## 3. Planning and the widening/narrowing rule
 
+### Repairing a legacy spatial member expression
+
+SQLite derived spatial columns read members as JSON values, preserving strings
+and booleans as well as geometry objects and coordinate arrays. A legacy column
+declared with `json(jsonb_extract(...))` has different SQL and remains a
+`JD0002` at open; opening never rewrites an existing table. Rebuild those
+derived columns through an ordinary migration. The intermediate model below
+removes only spatial indexes; the two plans are combined into one transaction,
+including any R\*Tree trigger removal and backfill:
+
+```js
+const withoutSpatial = structuredClone(model);
+for (const collection of Object.values(withoutSpatial.collections ?? {})) {
+  collection.indexes = (collection.indexes ?? [])
+    .filter((index) => !['geohash', 'bbox'].includes(index.derive));
+}
+const remove = planMigration(model, withoutSpatial, { dialect: sqliteDialect }).migration;
+const restore = planMigration(withoutSpatial, model, { dialect: sqliteDialect }).migration;
+const repair = {
+  ...remove, id: 'spatial-member-json', to: restore.to,
+  steps: [...remove.steps, ...restore.steps],
+};
+await migrate({ driver, path }, [...previousMigrations, repair], { baseline, model });
+```
+
+Keep the historical baseline and full migration list. With no previous migrations,
+`baseline` is `model`. The logical model hash stays the same; the explicit
+artifact changes the physical expression. Shadow verification and final shape
+validation run normally, stored documents remain intact, and replay skips the
+recorded repair. The stored-column mapping does not need this repair.
+
+### Model differences
+
 `planMigration(fromModel, toModel, { dialect, id, derived })` produces
 `{ migration, report }` by diffing the two models' PHYSICAL plans. The
 from-model is the previous model — the previous model FILE, or, under
@@ -235,25 +268,34 @@ hash of the `baseline` model when no migration has run.
 
   | strategy | which assertions | what it costs |
   |---|---|---|
-  | per-document | a FLWOR over `$[*]` whose `$where`/`$return` read only the binding | one keyset batch at a time; fails fast at the first batch that violates |
-  | fold | exactly one of `$count`, `$sum`, `$min`, `$max` over the root | one batch at a time; each batch is answered by the ENGINE and the partial answers combine |
-  | materialize | everything else (`$let`, `$distinct`, a nested `$for`, two aggregates) | every document at once, under `assertionBounds` |
-
-  A fold is sound because the operator is associative: the answer over a
-  collection is the combination of the answers over any partition of it.
-  Nothing reimplements an operator — each batch is evaluated by the same
-  compiled query the whole-collection path would use, and only the
-  COMBINE step is written here, so null handling, empty-sequence answers
-  and type coercions are the engine's. A suite runs every fold shape both
-  ways, over ten corpora and six partitions, and requires the value and
-  the verdict to be indistinguishable; a shape that cannot pass it is not
-  in the set.
+  | provider | a collection `$count` that the existing query planner proves native without a typed intermediate schema | one aggregate result, with no assertion document fetch |
+  | per-document | an independent, unwindowed FLWOR over `$[*]`, with the default empty-sequence expectation | one keyset batch; fails at the first violation |
+  | fold | `$count`, `$sum`, `$avg`, `$min`, `$max` over a partition-independent operand; sequence EBV of an independent FLWOR | one batch plus fixed accumulator state |
+  | fold with bounds | `$distinct` over such an operand with an explicit positive `maxDistinct` | one batch plus unique items under cardinality and byte bounds |
+  | materialize | global or positional operands, nested scans, unsupported shapes, or unbounded distinct | the collection under `assertionBounds` |
+
+  Folds feed the operand's sequence items, in original order, into the
+  query engine's shared accumulator. Floating-point addition is **not
+  associative**: combining batch totals changes answers. The ordered state
+  preserves the whole-query result, including singleton arrays, negative
+  zero, NaN, mixed-type refusals and Unicode code-point ordering. Sequence
+  EBV is checked once across all batches, including an empty source.
+  `$[0]`, root-dependent filters and positional bindings never qualify as
+  independent merely because they occur inside an aggregate.
+
+  `classifyAssertion(query, { expect, maxDistinct })` exposes the portable
+  strategy. `onAssertionPlan(plan)` reports migration, step, collection,
+  strategy, shape, reason and bounds before the assertion reads documents;
+  Store dry-run query statements also expose strategy and reason. Provider
+  promotion reuses the query planner with no assumed typed columns. Typed
+  SQL aggregates still need a trustworthy intermediate schema and proof
+  that their numeric/order semantics match; otherwise the ordered fold runs.
 
   **A materializing assertion is bounded.** `options.assertionBounds`
   defaults to `{ maxRows: 100000, maxBytes: 67108864 }` and is crossed
   BEFORE the excess is held — the walk stops at the row that would break
   it, refusing `JD2007` (rows) or `JD2076` (bytes) and naming the two
-  assertion shapes that are answered in batches instead. `null` on either
+  assertion strategies that are answered in batches instead. `null` on either
   member removes that bound, which a caller must ask for: an unbounded
   read nobody declared is exactly what this classification removes. This
   is a deliberate behavior change — a migration that used to read a very
@@ -265,14 +307,11 @@ hash of the `baseline` model when no migration has run.
   transformed | derived | asserted }`, one event per batch), and never
   hold the whole collection in memory. A PER-DOCUMENT assertion — a
   FLWOR over `$[*]` whose `$where` and `$return` read only the binding
-  — walks the same batches and fails fast at the first batch that
-  violates, because its answer over each batch is its answer over the
-  whole. A cross-document assertion (one that reads the root: `$count:
-  '$[*]'`, a `$let`, a `$distinct`, a nested `$for`) reads the whole
-  collection into one array — a stated cost; keep such assertions
-  early, before the data grows. A cross-document assertion that is one
-  associative aggregate no longer costs that read at all — see the
-  classification table above.
+  — with the default empty expectation walks the same batches and fails
+  fast. Other assertions follow the classification table above.
+  `batchSize` must be a positive safe integer. `maxDistinct` opts into a
+  distinct fold and bounds retained unique items; `maxBytes` applies to
+  those items. Duplicates do not consume additional cardinality.
 - A transform MUST NOT change a caller-keyed document's key member —
   the key column would go stale; the run refuses (`JD0023`).
 
@@ -510,10 +549,23 @@ jaren-db documents --migrations <dir> --in <file|-> (--out <file|-> | --in-place
   line-delimited, everything else one JSON array) unless `--format` /
   `--out-format` says otherwise, and stdio defaults to JSONL. Input and
   output encodings are independent, so this is also the converter.
-  - **A file holds ONE collection.** The migrations name it; a chain
-    whose document steps touch more than one cannot be applied to a
-    file, and is refused rather than partly run. `--collection` asserts
-    which collection the file holds and refuses a mismatch.
+  - **Named sources share one collection bundle.** Repeat
+    `--in users=users.json --in events=events.jsonl --out migrated.json`.
+    Output defaults to `--out-format collections`: a standard JSON object
+    mapping collection names to arrays, including empty collections.
+    Publication is one atomic rename after every collection succeeds.
+    Multiple source files cannot use `--in-place`, because separate
+    renames cannot give this guarantee. Single-source `--collection`
+    still asserts the collection name and refuses a mismatch.
+  - **A bundle can be read explicitly** with `--format collections`, then
+    replaced using `--in-place --yes`. Bundle input is materialized under
+    a whole-file `maxBytes` ceiling before parsing and per-collection
+    `maxRows` validation after parsing. JSON and JSONL single-collection
+    sources remain streaming. `--max-rows`, `--max-bytes`, and
+    `--max-distinct` configure assertion admission too; the first two
+    accept `none` to remove their limit deliberately. A materializing
+    file run admits rows before retaining them. Physical-step and query
+    compilation refusals precede every input read.
   - **`--out` writes elsewhere; `--in-place` replaces the input and
     needs `--yes`.** Either way the documents land in a sibling
     temporary that is renamed over the target only once every document

package/docs/MODEL-FORMAT.md

@@ -379,7 +379,7 @@ rather than degrading silently.
 
 | capability | value | mapping | drivers |
 |---|---|---|---|
-| `deterministicIndexableFunctions` | `true` | a **virtual generated column** whose expression calls a deterministic function the store registers at open — `jaren_geohash(<member>, <precision>)`, `jaren_bbox_w(<member>)`, … over `json(jsonb_extract("doc", '<path>'))` | `node`, `wasm` |
+| `deterministicIndexableFunctions` | `true` | a **virtual generated column** whose expression calls a deterministic function the store registers at open — `jaren_geohash(<member>, <precision>)`, `jaren_bbox_w(<member>)`, … over `json(("doc" -> '<path>'))` | `node`, `wasm` |
 | `deterministicIndexableFunctions` | `false` | a **stored column** the store writes on every insert, upsert and patch, computed in JavaScript from the same kernel call | `bun` |
 | `rtree` | `false` | a `derive: 'bbox'` column set that declared `physical: 'rtree'` (§2.1) is planned, created and verified as the **B-tree over its four columns**, and `explain().prefilters[].via` reports `'columns'` beside `store.capabilities.rtree === false` | any build without `ENABLE_RTREE` |
 
@@ -451,9 +451,18 @@ no positions at all, or one whose coordinates are not positions, which
 is what a non-finite coordinate becomes: JSON cannot carry `NaN`, so it
 arrives as `null` and is no longer a number.
 
+A string also has no position: an untyped member can store WKT or even the
+string `"[1,2]"` without interpreting it as a coordinate array. Virtual and
+stored columns both derive `NULL`. Existing SQLite expressions that passed raw
+SQL strings to `json()` need the explicit column repair described in
+[MIGRATION-FORMAT §3](MIGRATION-FORMAT.md#repairing-a-legacy-spatial-member-expression).
+
 The consequence is stated here rather than discovered later: **a row
-whose derived column is `NULL` is not found by a predicate pushed to
-that column.** For the spatial predicates the planner promotes
+whose derived column is `NULL` is not found by ordinary predicates pushed to
+that column.** Neighbourhood membership retains those rows as candidates and
+re-runs its predicate: `$index-of` raises for an empty search item, so discarding
+the row would hide that error. Guard the geohash with `$exists` before the
+membership test when unbounded rows should be excluded. For the other spatial predicates the planner promotes
 (ARCHITECTURE.md, "The implied conjunct") that is not a divergence —
 §8.14 measures a value by its representative position, and that
 position is missing in exactly the cases the box is, so `$within`,

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@jarenjs/db",
   "private": false,
-  "version": "0.67.0",
+  "version": "0.72.2",
   "type": "module",
   "main": "./src/index.js",
   "types": "./types/index.d.ts",
@@ -84,9 +84,9 @@
     "prepack": "npm run build:types"
   },
   "dependencies": {
-    "@jarenjs/core": "^0.67.0",
-    "@jarenjs/json": "^0.67.0",
-    "@jarenjs/validate": "^0.67.0"
+    "@jarenjs/core": "^0.72.2",
+    "@jarenjs/json": "^0.72.2",
+    "@jarenjs/validate": "^0.72.2"
   },
   "bin": {
     "jaren-db": "./src/cli.js"

package/src/algebra.js

@@ -57,7 +57,7 @@ export const PLAN_VERSION = 2;
  *   { p: 'udf', name: string, key: string } |
  *   { p: 'bboxOverlap', columns: { w: string, s: string, e: string,
  *     n: string }, probe: { box: number[] } | { ext: string } } |
- *   { p: 'cellIn', column: string, cells: string[] } |
+ *   { p: 'cellIn', column: string, cells: string[], keepEmpty?: boolean } |
  *   { p: 'cellPrefix', column: string, prefix: string } |
  *   { p: 'interval', columns: { start: string, end: string },
  *     probe: { from: number, to: number } } |
@@ -75,6 +75,9 @@ export const PLAN_VERSION = 2;
  *   — but each is TOTAL through its own `IS NOT NULL`, so a row with no
  *   box or no cell answers FALSE rather than SQL's NULL and negation
  *   still composes classically.
+ *   A neighbourhood membership uses `keepEmpty: true` instead: NULL
+ *   cells remain candidates so its residual preserves the query's error
+ *   for an empty search item. Such a predicate is never exact.
  *
  *   `colCmp` is the same idea one comparison wide: a bound over a
  *   DECLARED column, with no `json_type` beside it. The planner builds

package/src/cli.js

@@ -21,8 +21,10 @@ import {
   migrateDocuments, streamDocuments, classifyAssertion,
 } from './index.js';
 import { nodeDriver } from './drivers/node.js';
+import { prepareDocumentRun } from './documents.js';
+import { normalizeAssertionBounds, createAssertionBoundGuard } from './document-steps.js';
 import {
-  readDocuments, openAtomicTarget, openStreamTarget, openNullTarget,
+  readDocuments, readCollectionBundle, openAtomicTarget, openStreamTarget, openNullTarget,
   formatOf, DOCUMENT_FORMATS,
 } from './document-files.js';
 
@@ -37,7 +39,9 @@ Usage:
   jaren-db check    --model <model> --store <db> [--migrations <dir>] [--snapshot <file>]
   jaren-db shape    --model <model>
   jaren-db documents --migrations <dir> --in <file|-> (--out <file|-> | --in-place --yes | --check)
-                     [--format json|jsonl] [--out-format json|jsonl] [--collection <name>] [--batch-size <n>]
+                     [--format json|jsonl|collections] [--out-format json|jsonl|collections]
+                     [--in collection=file ...] [--collection <name>] [--batch-size <n>]
+                     [--max-rows <n|none>] [--max-bytes <n|none>] [--max-distinct <n>]
 
 A <model> or a migration is a .json file, or a MODULE (.js, .mjs, .cjs —
 or .ts where Node strips types) whose default export, or its 'model' /
@@ -96,6 +100,7 @@ function parseArgs(argv) {
     command: argv[2], from: null, to: null, model: null, store: null,
     baseline: null, migrations: null, id: null, out: null,
     snapshot: null, types: null,
+    inputs: [], maxRows: undefined, maxBytes: undefined, maxDistinct: undefined,
     in: null, format: null, outFormat: null, collection: null,
     batchSize: null, inPlace: false, check: false,
     dryRun: false, yes: false, help: false,
@@ -112,7 +117,10 @@ function parseArgs(argv) {
       case '--out': options.out = argv[++i]; break;
       case '--snapshot': options.snapshot = argv[++i]; break;
       case '--types': options.types = argv[++i]; break;
-      case '--in': options.in = argv[++i]; break;
+      case '--in': options.in = argv[++i]; options.inputs.push(options.in); break;
+      case '--max-rows': options.maxRows = argv[++i]; break;
+      case '--max-bytes': options.maxBytes = argv[++i]; break;
+      case '--max-distinct': options.maxDistinct = argv[++i]; break;
       case '--out-format': options.outFormat = argv[++i]; break;
       case '--format': options.format = argv[++i]; break;
       case '--collection': options.collection = argv[++i]; break;
@@ -435,94 +443,107 @@ async function commandShape(options) {
  */
 async function commandDocuments(options) {
   if (options.migrations === null) misuse('documents needs --migrations <dir>');
-  if (options.in === null) misuse('documents needs --in <file> (or - for standard input)');
+  if (options.inputs.length === 0) misuse('documents needs --in <file> (or - for standard input)');
+  if (options.inputs.some((input) => typeof input !== 'string' || input.startsWith('--')))
+    misuse('--in requires a file or collection=file');
+  const multiple = options.inputs.length > 1 || /^[A-Za-z_][A-Za-z0-9_]*=/.test(options.in);
+  if (multiple && options.inPlace)
+    misuse('multiple sources require one --out collection bundle; several in-place renames are not atomic');
   const sinks = [options.out !== null, options.inPlace, options.check].filter(Boolean).length;
-  if (sinks === 0)
-    misuse('documents needs one of --out <file>, --in-place or --check');
-  if (sinks > 1)
-    misuse('documents takes exactly one of --out, --in-place and --check');
-  if (options.inPlace && options.in === '-')
-    misuse('--in-place needs a file to replace, not standard input');
-  if (options.inPlace && !options.yes)
-    misuse('--in-place rewrites the input file — pass --yes to confirm, or --out to write elsewhere');
+  if (sinks !== 1) misuse('documents takes exactly one of --out <file>, --in-place and --check');
+  if (options.inPlace && options.in === '-') misuse('--in-place needs a file to replace, not standard input');
+  if (options.inPlace && !options.yes) misuse('--in-place rewrites the input file — pass --yes to confirm, or --out to write elsewhere');
   const batchSize = options.batchSize === null ? 500 : Number(options.batchSize);
-  if (!Number.isInteger(batchSize) || batchSize < 1)
-    misuse(`--batch-size must be a positive integer, not '${options.batchSize}'`);
-
-  const fromStdin = options.in === '-';
-  const inFormat = options.format ?? (fromStdin ? 'jsonl' : formatOf(options.in));
-  if (!DOCUMENT_FORMATS.includes(inFormat))
-    misuse(`--format must be one of ${DOCUMENT_FORMATS.join(', ')}, not '${inFormat}'`);
+  if (!Number.isSafeInteger(batchSize) || batchSize < 1) misuse('--batch-size must be a positive integer');
+  const declared = {};
+  for (const name of ['maxRows', 'maxBytes', 'maxDistinct']) {
+    if (options[name] === undefined) continue;
+    declared[name] = options[name] === 'none' ? null : Number(options[name]);
+  }
+  let assertionBounds;
+  try { assertionBounds = normalizeAssertionBounds(declared); }
+  catch (error) { misuse(error.message); }
+  const inFormat = options.format;
+  const bundleInput = inFormat === 'collections';
+  if (inFormat !== null && ![...DOCUMENT_FORMATS, 'collections'].includes(inFormat))
+    misuse('--format must be one of json, jsonl or collections');
+  if (bundleInput && multiple) misuse('--format collections takes one bundle file');
   const target = options.inPlace ? options.in : options.out;
   const toStdout = target === '-';
-  const outFormat = options.outFormat
-    ?? (options.check || toStdout ? inFormat : formatOf(/** @type {string} */ (target)));
-  if (!DOCUMENT_FORMATS.includes(outFormat))
-    misuse(`--out-format must be one of ${DOCUMENT_FORMATS.join(', ')}, not '${outFormat}'`);
-  if (!fromStdin && !fs.existsSync(options.in)) fail(`no such file: '${options.in}'`);
-
   const migrations = await loadMigrationsDir(options.migrations);
   if (migrations.length === 0) fail(`no migration documents in '${options.migrations}'`);
-
-  // A document file holds ONE collection. The migrations say which:
-  // every document step must name it, or this chain cannot be applied to
-  // a file at all — running only the steps that match would leave the
-  // rest silently unapplied, which is the one outcome a migration runner
-  // may never produce.
   const documentSteps = migrations.flatMap((migration) => migration.steps)
     .filter((step) => step.kind === 'jslt' || step.kind === 'query');
   const named = [...new Set(documentSteps.map((step) => step.collection))];
-  if (named.length > 1) {
-    fail(`a document file holds one collection, and these migrations touch ${named.length} `
-      + `(${named.join(', ')}) — run them against a store, or split the chain so each `
-      + 'migration touches the collection its file holds');
-  }
-  if (options.collection !== null && named.length === 1 && options.collection !== named[0]) {
-    misuse(`--collection names '${options.collection}', but these migrations touch `
-      + `'${named[0]}' — is this the right file for them?`);
-  }
-  // with no document step at all there is no collection to infer; the
-  // run still proceeds, because a physical step must be REFUSED by name
-  // rather than reported as a missing collection
-  const collection = named[0] ?? options.collection ?? 'documents';
-
-  // only a MATERIALIZING assertion needs the collection at once; a
-  // per-document predicate and an associative aggregate are both
-  // answered one batch at a time, so they stream
-  const materializes = documentSteps.some((step) => step.collection === collection
-    && step.kind === 'query' && classifyAssertion(step.assert).strategy === 'materialize');
-
-  const source = () => (fromStdin ? process.stdin : options.in);
+  const files = {};
+  if (multiple) {
+    if (options.collection !== null) misuse('named --in sources already declare their collections');
+    for (const input of options.inputs) {
+      const match = /^([A-Za-z_][A-Za-z0-9_]*)=(.+)$/.exec(input);
+      if (match === null) misuse('each named --in must be collection=file');
+      if (Object.hasOwn(files, match[1])) misuse(`duplicate input collection '${match[1]}'`);
+      Object.defineProperty(files, match[1], { value: match[2], enumerable: true });
+    }
+    if (Object.values(files).filter((file) => file === '-').length > 1) misuse('standard input can supply only one collection');
+  }
+  else if (!bundleInput) {
+    if (named.length > 1) fail(`a document file holds one collection, but these migrations touch ${named.join(', ')}; supply --in collection=file for every collection, or --format collections`);
+    const name = named[0] ?? options.collection ?? 'documents';
+    if (options.collection !== null && options.collection !== name) misuse(`--collection names '${options.collection}', but these migrations touch '${name}'`);
+    Object.defineProperty(files, name, { value: options.in, enumerable: true });
+  }
+  const runOptions = { batchSize, assertionBounds };
+  // Physical and compilation refusals precede every document read, including
+  // the materializing route. A bundle's actual collection inventory is checked
+  // again after its bounded parse.
+  prepareDocumentRun(bundleInput ? named : Object.keys(files), migrations, runOptions);
+  let state;
+  if (bundleInput) state = await readCollectionBundle(options.in === '-' ? process.stdin : options.in, assertionBounds);
+  const names = bundleInput ? Object.keys(state) : Object.keys(files);
+  prepareDocumentRun(names, migrations, runOptions);
+  const outFormat = options.outFormat ?? (multiple || bundleInput ? 'collections'
+    : options.check || toStdout ? inFormat ?? (options.in === '-' ? 'jsonl' : formatOf(options.in)) : formatOf(target));
+  if (!['json', 'jsonl', 'collections'].includes(outFormat)) misuse('--out-format must be json, jsonl or collections');
+  if (names.length > 1 && outFormat !== 'collections') misuse('multiple collections require --out-format collections');
+  const materializes = bundleInput || documentSteps.some((step) => step.kind === 'query'
+    && classifyAssertion(step.assert, { expect: step.expect, maxDistinct: assertionBounds.maxDistinct }).strategy === 'materialize');
+  const sources = Object.fromEntries(Object.entries(files).map(([name, file]) => [name,
+    readDocuments(file === '-' ? process.stdin : file, inFormat ?? (file === '-' ? 'jsonl' : formatOf(file)))]));
   let sink;
   if (options.check) sink = openNullTarget();
-  else if (toStdout) sink = openStreamTarget(process.stdout, outFormat);
-  else sink = await openAtomicTarget(/** @type {string} */ (target), outFormat);
-
+  else if (toStdout) sink = openStreamTarget(process.stdout, outFormat, { collections: names });
+  else sink = await openAtomicTarget(target, outFormat, { collections: names });
   try {
     let report;
     if (materializes) {
-      const documents = [];
-      for await (const document of readDocuments(source(), inFormat)) documents.push(document);
-      const out = await migrateDocuments({ [collection]: documents }, migrations, { batchSize });
+      if (!bundleInput) {
+        state = {};
+        for (const name of names) {
+          const documents = [], guard = createAssertionBoundGuard(assertionBounds, name, 'the file runner');
+          for await (const document of sources[name]) { guard.admit(document); documents.push(document); }
+          Object.defineProperty(state, name, { value: documents, enumerable: true });
+        }
+      }
+      const out = await migrateDocuments(state, migrations, runOptions);
       report = out.report;
-      for (const document of out.documents[collection]) await sink.write(document);
-    }
-    else {
-      report = await streamDocuments({ [collection]: readDocuments(source(), inFormat) },
-        migrations, { batchSize, write: (name, document) => sink.write(document) });
+      for (const name of names) for (const document of out.documents[name]) await sink.write(document, name);
     }
+    else report = await streamDocuments(sources, migrations,
+      { ...runOptions, write: (name, document) => sink.write(document, name) });
     const written = await sink.commit();
-    const counts = report.counts[collection] ?? { read: 0, transformed: 0, asserted: 0 };
-    console.log(`${options.check ? 'checked' : 'migrated'} '${collection}': `
-      + `${counts.read} read, ${counts.transformed} transformed, ${counts.asserted} asserted `
-      + `(${report.strategy[collection]})`);
+    for (const name of names) {
+      const counts = report.counts[name] ?? { read: 0, transformed: 0, asserted: 0 };
+      console.log(`${options.check ? 'checked' : 'migrated'} '${name}': ${counts.read} read, `
+        + `${counts.transformed} transformed, ${counts.asserted} asserted (${report.strategy[name]})`);
+    }
     console.log(`applied: ${report.applied.join(', ')}`);
     if (options.check) console.log('checked only — nothing was written');
     else if (toStdout) console.log(`wrote ${written.documents} document(s) to standard output`);
     else console.log(`wrote ${written.documents} document(s) to ${target} (${written.bytes} bytes)`);
   }
   catch (error) {
-    await sink.abort();
+    try { await sink.abort(); }
+    catch (cleanupError) { error.cleanupError = cleanupError; }
     return fail(error.message);
   }
 }