@omgbase/oqx 0.14.0 → 0.14.1

package/CHANGELOG.md

@@ -4,6 +4,70 @@ All notable changes to `@omgbase/oqx` are recorded here. The format follows
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versions follow
 [Semantic Versioning](https://semver.org/) (pre-1.0: a minor bump may break).
 
+## [0.14.1] - 2026-10-09
+
+### Patch
+- Relational patterns in nested blocks stop being quadratic. The in-memory engine
+  now runs every nested block through an optimizer (`optimize/`: a logical
+  `BlockPlan` per block, a `Rule` interface, a fixpoint driver) with four rules,
+  each proven unobservable — same rows, lifts and errors as the scan:
+
+  - **Correlated equality → hash probe.** `^customers first { where id ==
+    ^customer_id }`, `^orders collect { … where customer_id == ^id }`,
+    `^customers exists { where id == ^customer_id }`, `${customers} first { … }`:
+    a top-level `&&` conjunct `local == outer` (either side) is answered from a
+    hash index on the receiver built once per run (per collection and path),
+    with the residual conjuncts evaluated over the bucket in receiver order.
+    Multiple equalities intersect smallest-first. The index reproduces §5
+    equality exactly (absent ≡ null, `-0` ≡ `0`, `NaN` matches nothing, objects
+    by reference). The sound rule: every conjunct left of the equality must be
+    raise-free (no call, no `single`, no lift, no bound, no out-of-range binding)
+    and the outer side must be raise-free and read nothing from the block's row;
+    otherwise the scan runs. A receiver is indexed when it is statically stable
+    (reads only the root/bindings) or when the same collection object is probed
+    twice; a receiver that yields a fresh value per enclosing row is scanned.
+    5k × 5k zip: ~9 ms (the scan: seconds).
+  - **Invariant block hoisting.** A block that reads nothing from any enclosing
+    row and lifts nothing is evaluated once per run and its value reused.
+  - **Semi/anti-join short-circuit** is kept over the probe's bucket (`exists`
+    stops at the first bucket row that passes).
+  - **Cardinality.** `exists`/`none`/`count` with no residual predicate are
+    answered from the bucket (or receiver) size without entering a row.
+
+  `DataContext.indexFor(collection, path)` (TypeScript) / `index_for` (Rust) is
+  a new optional seam for pre-built indexes; `IndexedCollection` keys under §5
+  equality (it kept `null` and `undefined` apart and matched `NaN` to itself)
+  and exposes `context()` (TypeScript) so a residual's nested blocks see the
+  root and reuse its indexes. `InMemoryEngine` takes `{ rules, trace }`
+  (TypeScript) / `with_rules` (Rust); `rules: []` is the naive scan, and the
+  conformance suites prove optimized ≡ naive — result or error, stage and
+  message — over every `spec/oqx` fixture. No language change.
+
+  A `RowIndex` may also implement `lookupRows(value)` (TypeScript) /
+  `lookup_rows` (Rust): the matching rows themselves, in receiver order. For a
+  statically stable receiver the engine asks the context's index **before**
+  reading the collection and, when it answers rows, never materializes the
+  receiver (one correlation is probed — the first whose outer side is a `^`
+  reference rather than a literal — and the other conjuncts, remaining equalities
+  included, stay residual in their original order; a `TraceEvent` of kind
+  `lookup` reports it). A statically stable receiver is now also read once per
+  run when nothing is correlated, instead of once per enclosing row. Rust:
+  `index_for` returns `Option<Rc<dyn RowIndex>>` and `lookup` returns
+  `Vec<usize>`, so a context can create indexes on demand.
+
+  `DataContext::materialize(value)` (Rust) / `materialize?(value)` (TypeScript)
+  is a new optional seam, default identity: a context that hands out a stand-in
+  for a collection it has not read yet (a lazy table handle) resolves it here.
+  The engine calls it on every value it is about to observe as a value — an
+  operand, an argument, a projected item, an `order by`/`distinct` key, a
+  `where` scalar, a lift — and never in row position (the source, a receiver, a
+  `from`, a `follow` destination), where the value reaches `to_rows`/`index_for`
+  as handed out. The Rust engine's execution path was folded to mirror the
+  TypeScript one: one `access` for every consumer (the `follow` seeds included),
+  `scan`/`materialize`, a `memoized` helper for invariant plans, `outer_value`,
+  and `lookup_order`/`residual_without` in `optimize`; optimized ≡ naive over
+  every fixture, same perf.
+
 ## [0.14.0] - 2026-10-06
 
 ### Minor

package/README.md

@@ -781,6 +781,117 @@ blocks/nodes, the relations table, `$` intrinsics, `WITH RECURSIVE` for `follow`
 becomes a `QueryPlanner`, while oqx contributes the parser, IR, semantics
 contract, and residual executor.
 
+## Performance: relational patterns
+
+A nested block runs once per enclosing row, so the natural way to join two
+collections —
+
+```js
+oqx`select id, total, customer: ${customers} first { where id == ^customer_id } from ${orders}`;
+oqx`select name, orders: ${orders} collect { id, total where customer_id == ^id } from ${customers}`;
+oqx`select id from ${orders} where ${customers} exists { where id == ^customer_id }`;
+```
+
+— would be quadratic if the block scanned its receiver each time. It does not:
+the in-memory engine runs every nested block through an **optimizer**
+(`src/optimize/`), a small logical plan per block plus rules that rewrite it,
+each with a proof that the rewrite is unobservable — same rows in the same
+order, same lifts, same error (same message) at the same point. The planned
+engine and the naive engine (`new InMemoryEngine(ctx, { rules: [] })`) are run
+over every `spec/oqx` fixture by `test/conformance.test.ts` and must agree on
+results and on errors.
+
+**What is optimized**
+
+- **Correlated equality → hash probe.** A top-level `&&` conjunct of the block's
+  `where` of the form `local == outer` (either side) — `local` a bare identifier
+  or member chain on the block's row (`customer_id`, `meta.id`, `$value`),
+  `outer` anything that reads nothing from that row (`^customer_id`, `^^x.id`,
+  a `${…}` binding, a literal, arithmetic over them) — is answered from a hash
+  index on the receiver built **once per run** per collection and path. The
+  remaining conjuncts run over the bucket only, in receiver order; several
+  equalities intersect their buckets smallest first. The index keys are exactly
+  `==` (SEMANTICS §5): absent ≡ `null`, `-0` ≡ `0`, `NaN` matches nothing,
+  objects and arrays by reference. 5 000 orders × 5 000 customers zip in a few
+  milliseconds; the scan takes seconds.
+- **Invariant blocks run once.** A block that reads nothing from any enclosing
+  row (`^customers collect { name values }` from a top-level row) and lifts
+  nothing is evaluated the first time it is reached and its value reused for
+  every other row (the same array/record by reference).
+- **Semi/anti-join short-circuit.** `exists` and `none` stop at the first bucket
+  row that passes the residual, as they stop at the first matching row of a scan.
+- **Cardinality only.** `exists` / `none` / `count` with nothing left to check
+  per row (no `where`, or every conjunct correlated; no `distinct`, no `order
+  by`) are answered from the bucket's size after the bound without entering a
+  row.
+
+**The sound rule.** `&&` is strictly left to right and short-circuiting (§13),
+so a probe may skip a row only if everything the scan would have evaluated for
+it could neither raise nor bind: an equality is hoisted only when **every
+conjunct to its left is raise-free** — no function or method call, no `single`,
+no lift, no `limit`/`offset`, no out-of-range `${…}`, only `exists`/`none`/
+`count` blocks that are themselves raise-free — and its outer side is raise-free
+too. A conjunct that could raise *right* of the equality is simply evaluated
+over the bucket, where the scan would have evaluated it on the same rows. A
+block whose `where` lifts anywhere is never probed (a lift could change what
+the outer side reads between rows). Otherwise the block scans as before, so
+`where nope(x) && id == ^id` still raises `unknown function`.
+
+Property reads are treated as total (§23: evaluation is otherwise total). A
+`DataContext.get` that throws is caught where the engine relies on reads it
+would not otherwise perform — building an index, evaluating the probe value —
+and the block falls back to the scan, which raises or not in its own order.
+
+**Which receivers are indexed.** A receiver that reads only the root scope and
+bindings (`^customers`, `${customers}`, `^^config.items`) is evaluated once per
+run — its rows are read once and reused across the enclosing rows, with or
+without a correlation — and indexed once per path even if the context hands out
+a fresh array per read. Any other receiver is indexed the second time the
+**same object** is probed, so a per-row relation (`lines collect { … where cid
+== ^id }` with a different `lines` array per row) is scanned, not indexed. A
+scalar receiver is one row.
+
+**Pre-built indexes.** `DataContext.indexFor(collection, path)` is an optional
+seam: return a `RowIndex` (`lookup(value) → ascending positions into
+toRows(collection)`) for a collection you have already indexed, or `undefined`
+to let the engine build one. A `RowIndex` may also implement
+`lookupRows(value) → the matching rows, in receiver order`: for a stable
+receiver the engine then probes it **before** reading the collection, so a
+context backed by a store (a lazy handle for a table, an index on a column)
+answers the probe with one indexed lookup and the table is never materialized.
+One correlation is probed this way — the first whose outer side is a `^`
+reference rather than a literal — and the other conjuncts, remaining
+equalities included, are evaluated per selected row in their original order.
+
+**Lazy handles.** A context that hands out a stand-in for a collection it has
+not read yet may also implement `materialize(value)`: the engine calls it on
+every value it is about to observe *as a value* (an operand of `==`/`in`/
+arithmetic, a function or method argument, a projected item, an `order by` or
+`distinct` key, a `where` scalar, a lift) and never on a value in *row
+position* (the source, a block receiver, a body-level `from`, a `follow`
+destination), which reaches `toRows` / `indexFor` as handed out. So the probe
+above never reads the table, and the language never sees the stand-in. A
+handle that already behaves as the collection (a `Proxy` over an array, as
+omgbase's does) needs no `materialize`; the Rust crate's `DataContext::
+materialize` is the same seam for a `Value` that cannot be lazy on its own.
+`IndexedCollection` does the positional form for its fields:
+
+```js
+import { parse, run, IndexedCollection } from "@omgbase/oqx";
+const orders = new IndexedCollection("orders", orderRows, ["customer_id"]);
+run(parse("select name, o: ^orders collect { id values where customer_id == ^id } from customers"),
+    { context: orders.context({ customers }) });   // probes the pre-built index
+```
+
+**Still a scan.** A block with a `from` re-projection or a `follow`; a `where`
+whose equality sits right of a conjunct that could raise; inequalities,
+`in`, `contains()` and other non-equality correlations; `order by … limit N`
+(sorted in full, then bounded); `distinct` (hashes projections, not an index);
+top-level queries (a `QueryPlanner` is the seam for those). `trace` in
+`InMemoryEngine`'s options receives one event per index built, probe, direct
+lookup, memo hit, cardinality answer or fallback, if you want to see what a
+query did.
+
 ## Exports
 
 Everything below is exported from `@omgbase/oqx` (`src/index.ts`); the SQLite
@@ -827,10 +938,18 @@ adapter lives on the `@omgbase/oqx/sqlite` subpath.
 **Engines and contexts**
 
 - `Engine` — `{ run(query, bindings): OqxResult }`; what `run({ engine })` accepts.
-- `InMemoryEngine` — `new InMemoryEngine(context?)`; the reference engine over a
-  `DataContext` (tier 1/2).
+- `InMemoryEngine` — `new InMemoryEngine(context?, options?)`; the reference
+  engine over a `DataContext` (tier 1/2). `InMemoryEngineOptions`: `rules`
+  (the optimizer rules for nested blocks, default `DEFAULT_RULES`; `[]` is the
+  naive scan) and `trace` (a `(event: TraceEvent) => void` sink for index /
+  probe / memo / cardinality / fallback events). See [Performance](#performance-relational-patterns).
 - `DataContext` — the tier-2 interface: `root`, `get`, `toRows`, `identity`,
-  plus optional `callFunction` / `callMethod` (above) and an optional
+  plus optional `callFunction` / `callMethod` (above), an optional
+  `indexFor(collection, path)` returning a `RowIndex` for a pre-built equality
+  index (`lookup` → positions; optionally `lookupRows` → rows, probed without
+  materializing the collection), an optional `materialize(value)` resolving a
+  lazy handle the engine is about to observe as a value (see
+  [Performance](#performance-relational-patterns)), and an optional
   `regexDialect` (`"oqx"` default | `"native"`). The engine dispatches
   `matches` through `callMethod`, so a custom context that wants the native
   dialect answers `matches` itself with `semantics.regexMatches(recv, args,
@@ -873,17 +992,31 @@ adapter lives on the `@omgbase/oqx/sqlite` subpath.
   its native query using `constValue` for parameters, then returns
   `{ rows, residual: residualQuery(query, residual) }`.
 - `IndexedCollection` — `new IndexedCollection(rootName, rows, indexFields)`; a
-  `QueryPlanner` that hash-indexes `rows` on `indexFields` and answers
-  `field == value` predicates on them from the index, leaving the rest residual:
+  `QueryPlanner` that hash-indexes `rows` on `indexFields` (under `==`: absent ≡
+  null, `NaN` matches nothing) and answers `field == value` predicates on them
+  from the index, leaving the rest residual. `indexFor(collection, path)` serves
+  the same indexes to the engine's correlated probes and `context(extraRoots?)`
+  is a `DataContext` that resolves the root and wires `indexFor` up; the plans
+  it returns carry that context, so a residual's nested blocks see the root.
 
   ```js
   import { parse, run, IndexedCollection, PlannedEngine } from "@omgbase/oqx";
   const planner = new IndexedCollection("people", people, ["city", "title"]);
-  const engine = new PlannedEngine(planner);
+  const engine = new PlannedEngine(planner, planner.context());
   run(parse('name from people where city == "NYC" && age > 30'), { engine });
   // city probe from the index; `age > 30` finished in-memory over the candidates
   ```
 
+**Optimizer (nested blocks)**
+
+- `DEFAULT_RULES`, `correlatedEqualityProbe`, `stableReceiver`, `invariantBlock`,
+  `cardinalityOnly` — the shipped `Rule`s (`(plan: BlockPlan, ctx: RuleContext)
+  => BlockPlan | null`); `logicalBlock(op, depth)` is the unoptimized plan of a
+  block and `optimizeBlock(op, depth, ctx, rules)` runs rules to a fixpoint.
+  `BlockPlan` / `Correlation` / `RuleContext` are the types; `HashIndex` is the
+  engine's `RowIndex` implementation (`add(value, position)`, `lookup(value)`),
+  reusable by a context implementing `indexFor`.
+
 - `SqliteTable` (from `@omgbase/oqx/sqlite`) —
   `new SqliteTable(db, tableName, options)` over a `node:sqlite` `DatabaseSync`.
   `SqliteTableOptions`:

package/dist/adapters/indexed.d.ts

@@ -1,11 +1,23 @@
 import type { Query } from "../ast.js";
+import type { DataContext } from "../context.js";
 import type { Plan, QueryPlanner } from "../planner.js";
+import type { RowIndex } from "../optimize/hash-index.js";
 export declare class IndexedCollection implements QueryPlanner {
     private name;
     private rows;
     private indexes;
-    /** Index `rows` (exposed as root `name`) on each field in `indexFields`. */
+    /** Index `rows` (exposed as root `name`) on each field in `indexFields`. A
+     * field is read as the engine reads a bare identifier (own property; array
+     * element for an integer-spelled name); a row without it keys as absent. */
     constructor(name: string, rows: readonly unknown[], indexFields: readonly string[]);
+    /** The pre-built index for `collection` on `path` when `collection` is this
+     * collection's rows and `path` is one indexed field — the `DataContext.indexFor`
+     * seam, which `context()` wires up. */
+    indexFor(collection: unknown, path: readonly string[]): RowIndex | undefined;
+    /** A `DataContext` over plain objects that serves this collection as the
+     * root `name` (plus `extraRoots`) and exposes the indexes through
+     * `indexFor`, so the engine's correlated probes reuse them. */
+    context(extraRoots?: Record<string, unknown>): DataContext;
     plan(query: Query, params: readonly unknown[]): Plan | null;
 }
 //# sourceMappingURL=indexed.d.ts.map

package/dist/adapters/indexed.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"indexed.d.ts","sourceRoot":"","sources":["../../src/adapters/indexed.ts"],"names":[],"mappings":"AAUA,OAAO,KAAK,EAAE,KAAK,EAAE,MAAM,WAAW,CAAC;AACvC,OAAO,KAAK,EAAE,IAAI,EAAE,YAAY,EAAE,MAAM,eAAe,CAAC;AAGxD,qBAAa,iBAAkB,YAAW,YAAY;IACpD,OAAO,CAAC,IAAI,CAAS;IACrB,OAAO,CAAC,IAAI,CAAqB;IACjC,OAAO,CAAC,OAAO,CAA8C;IAE7D,4EAA4E;gBAChE,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,SAAS,OAAO,EAAE,EAAE,WAAW,EAAE,SAAS,MAAM,EAAE;IAelF,IAAI,CAAC,KAAK,EAAE,KAAK,EAAE,MAAM,EAAE,SAAS,OAAO,EAAE,GAAG,IAAI,GAAG,IAAI;CAoB5D"}
+{"version":3,"file":"indexed.d.ts","sourceRoot":"","sources":["../../src/adapters/indexed.ts"],"names":[],"mappings":"AAqBA,OAAO,KAAK,EAAE,KAAK,EAAE,MAAM,WAAW,CAAC;AACvC,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,eAAe,CAAC;AAEjD,OAAO,KAAK,EAAE,IAAI,EAAE,YAAY,EAAE,MAAM,eAAe,CAAC;AAExD,OAAO,KAAK,EAAE,QAAQ,EAAE,MAAM,2BAA2B,CAAC;AAG1D,qBAAa,iBAAkB,YAAW,YAAY;IACpD,OAAO,CAAC,IAAI,CAAS;IACrB,OAAO,CAAC,IAAI,CAAqB;IACjC,OAAO,CAAC,OAAO,CAAgC;IAE/C;;gFAE4E;gBAChE,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,SAAS,OAAO,EAAE,EAAE,WAAW,EAAE,SAAS,MAAM,EAAE;IAWlF;;2CAEuC;IACvC,QAAQ,CAAC,UAAU,EAAE,OAAO,EAAE,IAAI,EAAE,SAAS,MAAM,EAAE,GAAG,QAAQ,GAAG,SAAS;IAK5E;;mEAE+D;IAC/D,OAAO,CAAC,UAAU,GAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAM,GAAG,WAAW;IAI9D,IAAI,CAAC,KAAK,EAAE,KAAK,EAAE,MAAM,EAAE,SAAS,OAAO,EAAE,GAAG,IAAI,GAAG,IAAI;CAoB5D"}

package/dist/adapters/indexed.js

@@ -1,33 +1,57 @@
 // An in-memory optimizing planner: it hash-indexes a named root collection on
-// chosen fields and, for a query whose where-clause contains equality predicates
-// on those fields, answers from the index (candidate intersection) instead of a
-// full scan. Everything it can't turn into an index probe is left as a residual
-// query the in-memory engine finishes over the candidate rows.
+// chosen fields and, for a query whose where-clause contains equality
+// predicates on those fields, answers from the index (candidate intersection)
+// instead of a full scan. Everything it can't turn into an index probe is left
+// as a residual query the in-memory engine finishes over the candidate rows.
 //
 // This is the smallest honest demonstration of plan optimization: same answers
 // as a naive scan, far fewer rows examined, and partial-pushdown correctness via
 // the residual.
-import { partitionPushable, residualQuery, asEquality, constValue } from "../plan.js";
+//
+// The same indexes serve the engine's correlated probes: `context()` is a
+// `DataContext` that resolves the root and answers `indexFor(collection, path)`
+// from them, so a nested `^emp first { where id == ^manager_id }` probes the
+// pre-built index instead of building one per run. The planner's own plans
+// carry that context too, so a residual's nested blocks still see every root.
+//
+// Buckets are keyed under OQX equality (`HashIndex`): absent ≡ null, `-0` ≡ `0`,
+// `NaN` matches nothing, objects by reference — exactly what `==` compares, so
+// a probe returns the rows the scan would keep (a plain `Map` would keep `null`
+// and `undefined` apart and match `NaN` to itself).
+import { DefaultContext } from "../context.js";
+import { partitionPushable, residualQuery, asEquality, constValue, ROWS_ROOT } from "../plan.js";
+import { HashIndex, intersectPositions } from "../optimize/hash-index.js";
 export class IndexedCollection {
     name;
     rows;
     indexes = new Map();
-    /** Index `rows` (exposed as root `name`) on each field in `indexFields`. */
+    /** Index `rows` (exposed as root `name`) on each field in `indexFields`. A
+     * field is read as the engine reads a bare identifier (own property; array
+     * element for an integer-spelled name); a row without it keys as absent. */
     constructor(name, rows, indexFields) {
         this.name = name;
         this.rows = rows;
+        const read = new DefaultContext();
         for (const field of indexFields) {
-            const idx = new Map();
-            for (const row of rows) {
-                const key = row?.[field];
-                let bucket = idx.get(key);
-                if (!bucket)
-                    idx.set(key, (bucket = []));
-                bucket.push(row);
-            }
+            const idx = new HashIndex();
+            rows.forEach((row, i) => idx.add(read.get(row, field), i));
             this.indexes.set(field, idx);
         }
     }
+    /** The pre-built index for `collection` on `path` when `collection` is this
+     * collection's rows and `path` is one indexed field — the `DataContext.indexFor`
+     * seam, which `context()` wires up. */
+    indexFor(collection, path) {
+        if (collection !== this.rows || path.length !== 1)
+            return undefined;
+        return this.indexes.get(path[0]);
+    }
+    /** A `DataContext` over plain objects that serves this collection as the
+     * root `name` (plus `extraRoots`) and exposes the indexes through
+     * `indexFor`, so the engine's correlated probes reuse them. */
+    context(extraRoots = {}) {
+        return new IndexedContext(this, { ...extraRoots, [this.name]: this.rows });
+    }
     plan(query, params) {
         if (query.source.kind !== "ident" || query.source.name !== this.name)
             return null;
@@ -40,18 +64,25 @@ export class IndexedCollection {
         if (pushed.length === 0)
             return null; // no index probe available — let the scan handle it
         // Intersect the candidate sets from each indexed equality (smallest first).
-        let candidate = null;
-        for (const e of pushed) {
+        const buckets = pushed.map((e) => {
             const eq = asEquality(e);
-            const bucket = this.indexes.get(eq.field).get(constValue(eq.value, params)) ?? [];
-            candidate = candidate === null ? bucket.slice() : intersect(candidate, bucket);
-        }
-        const rows = candidate ?? [];
-        return { rows: () => rows, residual: residualQuery(query, residual) };
+            return this.indexes.get(eq.field).lookup(constValue(eq.value, params));
+        }).sort((a, b) => a.length - b.length);
+        let candidate = buckets[0];
+        for (let i = 1; i < buckets.length && candidate.length > 0; i++)
+            candidate = intersectPositions(candidate, buckets[i]);
+        const rows = candidate.map((i) => this.rows[i]);
+        return { rows: () => rows, residual: residualQuery(query, residual), context: this.context({ [ROWS_ROOT]: rows }) };
     }
 }
-function intersect(a, b) {
-    const set = new Set(b);
-    return a.filter((x) => set.has(x));
+class IndexedContext extends DefaultContext {
+    owner;
+    constructor(owner, roots) {
+        super(roots);
+        this.owner = owner;
+    }
+    indexFor(collection, path) {
+        return this.owner.indexFor(collection, path);
+    }
 }
 //# sourceMappingURL=indexed.js.map

package/dist/adapters/indexed.js.map

@@ -1 +1 @@
-{"version":3,"file":"indexed.js","sourceRoot":"","sources":["../../src/adapters/indexed.ts"],"names":[],"mappings":"AAAA,8EAA8E;AAC9E,iFAAiF;AACjF,gFAAgF;AAChF,gFAAgF;AAChF,+DAA+D;AAC/D,EAAE;AACF,+EAA+E;AAC/E,iFAAiF;AACjF,gBAAgB;AAIhB,OAAO,EAAE,iBAAiB,EAAE,aAAa,EAAE,UAAU,EAAE,UAAU,EAAE,MAAM,YAAY,CAAC;AAEtF,MAAM,OAAO,iBAAiB;IACpB,IAAI,CAAS;IACb,IAAI,CAAqB;IACzB,OAAO,GAAG,IAAI,GAAG,EAAmC,CAAC;IAE7D,4EAA4E;IAC5E,YAAY,IAAY,EAAE,IAAwB,EAAE,WAA8B;QAChF,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC;QACjB,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC;QACjB,KAAK,MAAM,KAAK,IAAI,WAAW,EAAE,CAAC;YAChC,MAAM,GAAG,GAAG,IAAI,GAAG,EAAsB,CAAC;YAC1C,KAAK,MAAM,GAAG,IAAI,IAAI,EAAE,CAAC;gBACvB,MAAM,GAAG,GAAI,GAA+B,EAAE,CAAC,KAAK,CAAC,CAAC;gBACtD,IAAI,MAAM,GAAG,GAAG,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;gBAC1B,IAAI,CAAC,MAAM;oBAAE,GAAG,CAAC,GAAG,CAAC,GAAG,EAAE,CAAC,MAAM,GAAG,EAAE,CAAC,CAAC,CAAC;gBACzC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;YACnB,CAAC;YACD,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,KAAK,EAAE,GAAG,CAAC,CAAC;QAC/B,CAAC;IACH,CAAC;IAED,IAAI,CAAC,KAAY,EAAE,MAA0B;QAC3C,IAAI,KAAK,CAAC,MAAM,CAAC,IAAI,KAAK,OAAO,IAAI,KAAK,CAAC,MAAM,CAAC,IAAI,KAAK,IAAI,CAAC,IAAI;YAAE,OAAO,IAAI,CAAC;QAClF,IAAI,KAAK,CAAC,IAAI,CAAC,MAAM,GAAG,CAAC,IAAI,KAAK,CAAC,MAAM;YAAE,OAAO,IAAI,CAAC;QAEvD,MAAM,EAAE,MAAM,EAAE,QAAQ,EAAE,GAAG,iBAAiB,CAAC,KAAK,CAAC,KAAK,EAAE,CAAC,CAAC,EAAE,EAAE;YAChE,MAAM,EAAE,GAAG,UAAU,CAAC,CAAC,CAAC,CAAC;YACzB,OAAO,EAAE,IAAI,IAAI,IAAI,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,EAAE,CAAC,KAAK,CAAC,CAAC;QAClD,CAAC,CAAC,CAAC;QACH,IAAI,MAAM,CAAC,MAAM,KAAK,CAAC;YAAE,OAAO,IAAI,CAAC,CAAC,oDAAoD;QAE1F,4EAA4E;QAC5E,IAAI,SAAS,GAAqB,IAAI,CAAC;QACvC,KAAK,MAAM,CAAC,IAAI,MAAM,EAAE,CAAC;YACvB,MAAM,EAAE,GAAG,UAAU,CAAC,CAAC,CAAE,CAAC;YAC1B,MAAM,MAAM,GAAG,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,EAAE,CAAC,KAAK,CAAE,CAAC,GAAG,CAAC,UAAU,CAAC,EAAE,CAAC,KAAK,EAAE,MAAM,CAAC,CAAC,IAAI,EAAE,CAAC;YACnF,SAAS,GAAG,SAAS,KAAK,IAAI,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC,SAAS,CAAC,SAAS,EAAE,MAAM,CAAC,CAAC;QACjF,CAAC;QACD,MAAM,IAAI,GAAG,SAAS,IAAI,EAAE,CAAC;QAC7B,OAAO,EAAE,IAAI,EAAE,GAAG,EAAE,CAAC,IAAI,EAAE,QAAQ,EAAE,aAAa,CAAC,KAAK,EAAE,QAAQ,CAAC,EAAE,CAAC;IACxE,CAAC;CACF;AAED,SAAS,SAAS,CAAC,CAAY,EAAE,CAAY;IAC3C,MAAM,GAAG,GAAG,IAAI,GAAG,CAAC,CAAC,CAAC,CAAC;IACvB,OAAO,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC;AACrC,CAAC"}
+{"version":3,"file":"indexed.js","sourceRoot":"","sources":["../../src/adapters/indexed.ts"],"names":[],"mappings":"AAAA,8EAA8E;AAC9E,sEAAsE;AACtE,8EAA8E;AAC9E,+EAA+E;AAC/E,6EAA6E;AAC7E,EAAE;AACF,+EAA+E;AAC/E,iFAAiF;AACjF,gBAAgB;AAChB,EAAE;AACF,0EAA0E;AAC1E,gFAAgF;AAChF,6EAA6E;AAC7E,2EAA2E;AAC3E,8EAA8E;AAC9E,EAAE;AACF,iFAAiF;AACjF,+EAA+E;AAC/E,gFAAgF;AAChF,oDAAoD;AAIpD,OAAO,EAAE,cAAc,EAAE,MAAM,eAAe,CAAC;AAE/C,OAAO,EAAE,iBAAiB,EAAE,aAAa,EAAE,UAAU,EAAE,UAAU,EAAE,SAAS,EAAE,MAAM,YAAY,CAAC;AAEjG,OAAO,EAAE,SAAS,EAAE,kBAAkB,EAAE,MAAM,2BAA2B,CAAC;AAE1E,MAAM,OAAO,iBAAiB;IACpB,IAAI,CAAS;IACb,IAAI,CAAqB;IACzB,OAAO,GAAG,IAAI,GAAG,EAAqB,CAAC;IAE/C;;gFAE4E;IAC5E,YAAY,IAAY,EAAE,IAAwB,EAAE,WAA8B;QAChF,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC;QACjB,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC;QACjB,MAAM,IAAI,GAAG,IAAI,cAAc,EAAE,CAAC;QAClC,KAAK,MAAM,KAAK,IAAI,WAAW,EAAE,CAAC;YAChC,MAAM,GAAG,GAAG,IAAI,SAAS,EAAE,CAAC;YAC5B,IAAI,CAAC,OAAO,CAAC,CAAC,GAAG,EAAE,CAAC,EAAE,EAAE,CAAC,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,GAAG,EAAE,KAAK,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC;YAC3D,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,KAAK,EAAE,GAAG,CAAC,CAAC;QAC/B,CAAC;IACH,CAAC;IAED;;2CAEuC;IACvC,QAAQ,CAAC,UAAmB,EAAE,IAAuB;QACnD,IAAI,UAAU,KAAK,IAAI,CAAC,IAAI,IAAI,IAAI,CAAC,MAAM,KAAK,CAAC;YAAE,OAAO,SAAS,CAAC;QACpE,OAAO,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,CAAE,CAAC,CAAC;IACpC,CAAC;IAED;;mEAE+D;IAC/D,OAAO,CAAC,aAAsC,EAAE;QAC9C,OAAO,IAAI,cAAc,CAAC,IAAI,EAAE,EAAE,GAAG,UAAU,EAAE,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,IAAI,CAAC,IAAI,EAAE,CAAC,CAAC;IAC7E,CAAC;IAED,IAAI,CAAC,KAAY,EAAE,MAA0B;QAC3C,IAAI,KAAK,CAAC,MAAM,CAAC,IAAI,KAAK,OAAO,IAAI,KAAK,CAAC,MAAM,CAAC,IAAI,KAAK,IAAI,CAAC,IAAI;YAAE,OAAO,IAAI,CAAC;QAClF,IAAI,KAAK,CAAC,IAAI,CAAC,MAAM,GAAG,CAAC,IAAI,KAAK,CAAC,MAAM;YAAE,OAAO,IAAI,CAAC;QAEvD,MAAM,EAAE,MAAM,EAAE,QAAQ,EAAE,GAAG,iBAAiB,CAAC,KAAK,CAAC,KAAK,EAAE,CAAC,CAAC,EAAE,EAAE;YAChE,MAAM,EAAE,GAAG,UAAU,CAAC,CAAC,CAAC,CAAC;YACzB,OAAO,EAAE,IAAI,IAAI,IAAI,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,EAAE,CAAC,KAAK,CAAC,CAAC;QAClD,CAAC,CAAC,CAAC;QACH,IAAI,MAAM,CAAC,MAAM,KAAK,CAAC;YAAE,OAAO,IAAI,CAAC,CAAC,oDAAoD;QAE1F,4EAA4E;QAC5E,MAAM,OAAO,GAAG,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE;YAC/B,MAAM,EAAE,GAAG,UAAU,CAAC,CAAC,CAAE,CAAC;YAC1B,OAAO,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,EAAE,CAAC,KAAK,CAAE,CAAC,MAAM,CAAC,UAAU,CAAC,EAAE,CAAC,KAAK,EAAE,MAAM,CAAC,CAAC,CAAC;QAC1E,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,MAAM,GAAG,CAAC,CAAC,MAAM,CAAC,CAAC;QACvC,IAAI,SAAS,GAAG,OAAO,CAAC,CAAC,CAAE,CAAC;QAC5B,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,OAAO,CAAC,MAAM,IAAI,SAAS,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC,EAAE;YAAE,SAAS,GAAG,kBAAkB,CAAC,SAAS,EAAE,OAAO,CAAC,CAAC,CAAE,CAAC,CAAC;QACxH,MAAM,IAAI,GAAG,SAAS,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC;QAChD,OAAO,EAAE,IAAI,EAAE,GAAG,EAAE,CAAC,IAAI,EAAE,QAAQ,EAAE,aAAa,CAAC,KAAK,EAAE,QAAQ,CAAC,EAAE,OAAO,EAAE,IAAI,CAAC,OAAO,CAAC,EAAE,CAAC,SAAS,CAAC,EAAE,IAAI,EAAE,CAAC,EAAE,CAAC;IACtH,CAAC;CACF;AAED,MAAM,cAAe,SAAQ,cAAc;IACjC,KAAK,CAAoB;IAEjC,YAAY,KAAwB,EAAE,KAA8B;QAClE,KAAK,CAAC,KAAK,CAAC,CAAC;QACb,IAAI,CAAC,KAAK,GAAG,KAAK,CAAC;IACrB,CAAC;IAED,QAAQ,CAAC,UAAmB,EAAE,IAAuB;QACnD,OAAO,IAAI,CAAC,KAAK,CAAC,QAAQ,CAAC,UAAU,EAAE,IAAI,CAAC,CAAC;IAC/C,CAAC;CACF"}

package/dist/context.d.ts

@@ -1,4 +1,5 @@
 import type { RegexDialect } from "./regex.js";
+import type { RowIndex } from "./optimize/hash-index.js";
 /** Result of a context-provided function/method call: `handled: false` tells the
  * engine to fall back to the builtin table (or error if none). */
 export interface CallResult {
@@ -17,6 +18,18 @@ export interface DataContext {
     get(row: unknown, key: string): unknown;
     /** Coerce a relation/source value into rows (may be lazy). */
     toRows(value: unknown): Iterable<unknown>;
+    /** Optional: a value this context handed out as a stand-in for a collection
+     * it has not read yet (a lazy table handle), resolved to what it stands for.
+     * The engine calls it on every value it is about to observe AS A VALUE — an
+     * operand of `==`/`in`/arithmetic, a function or method argument, a projected
+     * item, an `order by` or `distinct` key, a `where` scalar, a lift — and never
+     * on a value it reads in ROW POSITION (the query source, a block receiver, a
+     * body-level `from`, a `follow` destination), which reaches `toRows` and
+     * `indexFor` as handed out, so a store-backed context can answer a probe on
+     * the handle without reading the table and still never lets the stand-in be
+     * seen by the language. Absent = identity; a context whose handle already
+     * behaves as the collection (a `Proxy` over an array) needs none. */
+    materialize?(value: unknown): unknown;
     /** Identity of a row for `follow` cycle detection / dedup. */
     identity(row: unknown): unknown;
     /** Optional custom free function; return `{ handled: false }` to defer. */
@@ -31,6 +44,17 @@ export interface DataContext {
      * `regexMatches(recv, args, dialect)` is the helper). `BUILTIN_METHODS.matches`
      * is always the baseline. */
     readonly regexDialect?: RegexDialect;
+    /** Optional: a pre-built equality index over `collection` (a value this
+     * context served as a root or relation) on the property path `path`
+     * (`["customer_id"]`, `["meta", "id"]`; `[]` keys by the row itself). The
+     * engine asks before building its own hash index for a correlated equality
+     * in a nested block (`where id == ^customer_id`); return `undefined` to let
+     * it build one. `lookup(value)` must return the ascending positions, into
+     * `toRows(collection)` in order, of the rows whose value at `path` equals
+     * `value` under OQX equality (SEMANTICS §5: absent ≡ null, `-0` ≡ `0`, `NaN`
+     * matches nothing, objects by reference). `IndexedCollection.context()`
+     * implements it over its indexes. */
+    indexFor?(collection: unknown, path: readonly string[]): RowIndex | undefined;
 }
 export interface DefaultContextOptions {
     /** See `DataContext.regexDialect`. Default `"oqx"`. */

package/dist/context.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"context.d.ts","sourceRoot":"","sources":["../src/context.ts"],"names":[],"mappings":"AAWA,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,YAAY,CAAC;AAQ/C;kEACkE;AAClE,MAAM,WAAW,UAAU;IACzB,OAAO,EAAE,OAAO,CAAC;IACjB,KAAK,CAAC,EAAE,OAAO,CAAC;CACjB;AAED,MAAM,WAAW,WAAW;IAC1B,gFAAgF;IAChF,IAAI,CAAC,IAAI,EAAE,MAAM,GAAG,OAAO,CAAC;IAC5B;;;;;kFAK8E;IAC9E,GAAG,CAAC,GAAG,EAAE,OAAO,EAAE,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC;IACxC,8DAA8D;IAC9D,MAAM,CAAC,KAAK,EAAE,OAAO,GAAG,QAAQ,CAAC,OAAO,CAAC,CAAC;IAC1C,8DAA8D;IAC9D,QAAQ,CAAC,GAAG,EAAE,OAAO,GAAG,OAAO,CAAC;IAChC,2EAA2E;IAC3E,YAAY,CAAC,CAAC,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,OAAO,EAAE,GAAG,UAAU,CAAC;IACzD,oEAAoE;IACpE,UAAU,CAAC,CAAC,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,OAAO,EAAE,IAAI,EAAE,OAAO,EAAE,GAAG,UAAU,CAAC;IACtE;;;;;;iCAM6B;IAC7B,QAAQ,CAAC,YAAY,CAAC,EAAE,YAAY,CAAC;CACtC;AAED,MAAM,WAAW,qBAAqB;IACpC,uDAAuD;IACvD,YAAY,CAAC,EAAE,YAAY,CAAC;CAC7B;AAED;;;0EAG0E;AAC1E,qBAAa,cAAe,YAAW,WAAW;IAChD,OAAO,CAAC,KAAK,CAA0B;IACvC,QAAQ,CAAC,YAAY,EAAE,YAAY,CAAC;gBAExB,KAAK,GAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAM,EAAE,OAAO,GAAE,qBAA0B;IAKpF,IAAI,CAAC,IAAI,EAAE,MAAM,GAAG,OAAO;IAI3B,GAAG,CAAC,GAAG,EAAE,OAAO,EAAE,GAAG,EAAE,MAAM,GAAG,OAAO;IAMvC,MAAM,CAAC,KAAK,EAAE,OAAO,GAAG,QAAQ,CAAC,OAAO,CAAC;IAIzC,QAAQ,CAAC,GAAG,EAAE,OAAO,GAAG,OAAO;IAK/B,YAAY,CAAC,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,OAAO,EAAE,GAAG,UAAU;IAKvD,UAAU,CAAC,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,OAAO,EAAE,IAAI,EAAE,OAAO,EAAE,GAAG,UAAU;CAKrE"}
+{"version":3,"file":"context.d.ts","sourceRoot":"","sources":["../src/context.ts"],"names":[],"mappings":"AAWA,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,YAAY,CAAC;AAC/C,OAAO,KAAK,EAAE,QAAQ,EAAE,MAAM,0BAA0B,CAAC;AAQzD;kEACkE;AAClE,MAAM,WAAW,UAAU;IACzB,OAAO,EAAE,OAAO,CAAC;IACjB,KAAK,CAAC,EAAE,OAAO,CAAC;CACjB;AAED,MAAM,WAAW,WAAW;IAC1B,gFAAgF;IAChF,IAAI,CAAC,IAAI,EAAE,MAAM,GAAG,OAAO,CAAC;IAC5B;;;;;kFAK8E;IAC9E,GAAG,CAAC,GAAG,EAAE,OAAO,EAAE,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC;IACxC,8DAA8D;IAC9D,MAAM,CAAC,KAAK,EAAE,OAAO,GAAG,QAAQ,CAAC,OAAO,CAAC,CAAC;IAC1C;;;;;;;;;;yEAUqE;IACrE,WAAW,CAAC,CAAC,KAAK,EAAE,OAAO,GAAG,OAAO,CAAC;IACtC,8DAA8D;IAC9D,QAAQ,CAAC,GAAG,EAAE,OAAO,GAAG,OAAO,CAAC;IAChC,2EAA2E;IAC3E,YAAY,CAAC,CAAC,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,OAAO,EAAE,GAAG,UAAU,CAAC;IACzD,oEAAoE;IACpE,UAAU,CAAC,CAAC,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,OAAO,EAAE,IAAI,EAAE,OAAO,EAAE,GAAG,UAAU,CAAC;IACtE;;;;;;iCAM6B;IAC7B,QAAQ,CAAC,YAAY,CAAC,EAAE,YAAY,CAAC;IACrC;;;;;;;;;yCASqC;IACrC,QAAQ,CAAC,CAAC,UAAU,EAAE,OAAO,EAAE,IAAI,EAAE,SAAS,MAAM,EAAE,GAAG,QAAQ,GAAG,SAAS,CAAC;CAC/E;AAED,MAAM,WAAW,qBAAqB;IACpC,uDAAuD;IACvD,YAAY,CAAC,EAAE,YAAY,CAAC;CAC7B;AAED;;;0EAG0E;AAC1E,qBAAa,cAAe,YAAW,WAAW;IAChD,OAAO,CAAC,KAAK,CAA0B;IACvC,QAAQ,CAAC,YAAY,EAAE,YAAY,CAAC;gBAExB,KAAK,GAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAM,EAAE,OAAO,GAAE,qBAA0B;IAKpF,IAAI,CAAC,IAAI,EAAE,MAAM,GAAG,OAAO;IAI3B,GAAG,CAAC,GAAG,EAAE,OAAO,EAAE,GAAG,EAAE,MAAM,GAAG,OAAO;IAMvC,MAAM,CAAC,KAAK,EAAE,OAAO,GAAG,QAAQ,CAAC,OAAO,CAAC;IAIzC,QAAQ,CAAC,GAAG,EAAE,OAAO,GAAG,OAAO;IAK/B,YAAY,CAAC,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,OAAO,EAAE,GAAG,UAAU;IAKvD,UAAU,CAAC,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,OAAO,EAAE,IAAI,EAAE,OAAO,EAAE,GAAG,UAAU;CAKrE"}

package/dist/context.js.map

@@ -1 +1 @@
-{"version":3,"file":"context.js","sourceRoot":"","sources":["../src/context.ts"],"names":[],"mappings":"AAAA,gFAAgF;AAChF,iFAAiF;AACjF,gFAAgF;AAChF,4EAA4E;AAC5E,gFAAgF;AAChF,gFAAgF;AAChF,EAAE;AACF,gFAAgF;AAChF,2EAA2E;AAE3E,OAAO,EAAE,gBAAgB,EAAE,YAAY,EAAE,iBAAiB,EAAE,eAAe,EAAE,MAAM,gBAAgB,CAAC;AAGpG,0EAA0E;AAC1E,+DAA+D;AAC/D,SAAS,UAAU,CAAC,GAAW;IAC7B,OAAO,gBAAgB,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;AACpC,CAAC;AA0CD;;;0EAG0E;AAC1E,MAAM,OAAO,cAAc;IACjB,KAAK,CAA0B;IAC9B,YAAY,CAAe;IAEpC,YAAY,QAAiC,EAAE,EAAE,UAAiC,EAAE;QAClF,IAAI,CAAC,KAAK,GAAG,KAAK,CAAC;QACnB,IAAI,CAAC,YAAY,GAAG,OAAO,CAAC,YAAY,IAAI,KAAK,CAAC;IACpD,CAAC;IAED,IAAI,CAAC,IAAY;QACf,OAAO,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,KAAK,EAAE,IAAI,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;IACxE,CAAC;IAED,GAAG,CAAC,GAAY,EAAE,GAAW;QAC3B,IAAI,GAAG,IAAI,IAAI,IAAI,OAAO,GAAG,KAAK,QAAQ;YAAE,OAAO,SAAS,CAAC;QAC7D,IAAI,KAAK,CAAC,OAAO,CAAC,GAAG,CAAC;YAAE,OAAO,UAAU,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;QAC9E,OAAO,MAAM,CAAC,MAAM,CAAC,GAAG,EAAE,GAAG,CAAC,CAAC,CAAC,CAAE,GAA+B,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;IACrF,CAAC;IAED,MAAM,CAAC,KAAc;QACnB,OAAO,gBAAgB,CAAC,KAAK,CAAC,CAAC;IACjC,CAAC;IAED,QAAQ,CAAC,GAAY;QACnB,IAAI,GAAG,IAAI,IAAI,IAAI,OAAO,GAAG,KAAK,QAAQ,IAAI,IAAI,IAAI,GAAG;YAAE,OAAQ,GAA+B,CAAC,EAAE,CAAC;QACtG,OAAO,GAAG,CAAC;IACb,CAAC;IAED,YAAY,CAAC,IAAY,EAAE,IAAe;QACxC,MAAM,EAAE,GAAG,MAAM,CAAC,MAAM,CAAC,iBAAiB,EAAE,IAAI,CAAC,CAAC,CAAC,CAAC,iBAAiB,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;QACxF,OAAO,EAAE,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,IAAI,EAAE,KAAK,EAAE,EAAE,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,KAAK,EAAE,CAAC;IACtE,CAAC;IAED,UAAU,CAAC,IAAY,EAAE,IAAa,EAAE,IAAe;QACrD,IAAI,IAAI,KAAK,SAAS;YAAE,OAAO,EAAE,OAAO,EAAE,IAAI,EAAE,KAAK,EAAE,YAAY,CAAC,IAAI,EAAE,IAAI,EAAE,IAAI,CAAC,YAAY,CAAC,EAAE,CAAC;QACrG,MAAM,EAAE,GAAG,MAAM,CAAC,MAAM,CAAC,eAAe,EAAE,IAAI,CAAC,CAAC,CAAC,CAAC,eAAe,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;QACpF,OAAO,EAAE,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,IAAI,EAAE,KAAK,EAAE,EAAE,CAAC,IAAI,EAAE,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,KAAK,EAAE,CAAC;IAC5E,CAAC;CACF"}
+{"version":3,"file":"context.js","sourceRoot":"","sources":["../src/context.ts"],"names":[],"mappings":"AAAA,gFAAgF;AAChF,iFAAiF;AACjF,gFAAgF;AAChF,4EAA4E;AAC5E,gFAAgF;AAChF,gFAAgF;AAChF,EAAE;AACF,gFAAgF;AAChF,2EAA2E;AAE3E,OAAO,EAAE,gBAAgB,EAAE,YAAY,EAAE,iBAAiB,EAAE,eAAe,EAAE,MAAM,gBAAgB,CAAC;AAIpG,0EAA0E;AAC1E,+DAA+D;AAC/D,SAAS,UAAU,CAAC,GAAW;IAC7B,OAAO,gBAAgB,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;AACpC,CAAC;AAiED;;;0EAG0E;AAC1E,MAAM,OAAO,cAAc;IACjB,KAAK,CAA0B;IAC9B,YAAY,CAAe;IAEpC,YAAY,QAAiC,EAAE,EAAE,UAAiC,EAAE;QAClF,IAAI,CAAC,KAAK,GAAG,KAAK,CAAC;QACnB,IAAI,CAAC,YAAY,GAAG,OAAO,CAAC,YAAY,IAAI,KAAK,CAAC;IACpD,CAAC;IAED,IAAI,CAAC,IAAY;QACf,OAAO,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,KAAK,EAAE,IAAI,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;IACxE,CAAC;IAED,GAAG,CAAC,GAAY,EAAE,GAAW;QAC3B,IAAI,GAAG,IAAI,IAAI,IAAI,OAAO,GAAG,KAAK,QAAQ;YAAE,OAAO,SAAS,CAAC;QAC7D,IAAI,KAAK,CAAC,OAAO,CAAC,GAAG,CAAC;YAAE,OAAO,UAAU,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;QAC9E,OAAO,MAAM,CAAC,MAAM,CAAC,GAAG,EAAE,GAAG,CAAC,CAAC,CAAC,CAAE,GAA+B,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;IACrF,CAAC;IAED,MAAM,CAAC,KAAc;QACnB,OAAO,gBAAgB,CAAC,KAAK,CAAC,CAAC;IACjC,CAAC;IAED,QAAQ,CAAC,GAAY;QACnB,IAAI,GAAG,IAAI,IAAI,IAAI,OAAO,GAAG,KAAK,QAAQ,IAAI,IAAI,IAAI,GAAG;YAAE,OAAQ,GAA+B,CAAC,EAAE,CAAC;QACtG,OAAO,GAAG,CAAC;IACb,CAAC;IAED,YAAY,CAAC,IAAY,EAAE,IAAe;QACxC,MAAM,EAAE,GAAG,MAAM,CAAC,MAAM,CAAC,iBAAiB,EAAE,IAAI,CAAC,CAAC,CAAC,CAAC,iBAAiB,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;QACxF,OAAO,EAAE,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,IAAI,EAAE,KAAK,EAAE,EAAE,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,KAAK,EAAE,CAAC;IACtE,CAAC;IAED,UAAU,CAAC,IAAY,EAAE,IAAa,EAAE,IAAe;QACrD,IAAI,IAAI,KAAK,SAAS;YAAE,OAAO,EAAE,OAAO,EAAE,IAAI,EAAE,KAAK,EAAE,YAAY,CAAC,IAAI,EAAE,IAAI,EAAE,IAAI,CAAC,YAAY,CAAC,EAAE,CAAC;QACrG,MAAM,EAAE,GAAG,MAAM,CAAC,MAAM,CAAC,eAAe,EAAE,IAAI,CAAC,CAAC,CAAC,CAAC,eAAe,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;QACpF,OAAO,EAAE,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,IAAI,EAAE,KAAK,EAAE,EAAE,CAAC,IAAI,EAAE,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,KAAK,EAAE,CAAC;IAC5E,CAAC;CACF"}

package/dist/engine.d.ts

@@ -1,5 +1,6 @@
-import type { Query } from "./ast.js";
+import type { Query, OpNode } from "./ast.js";
 import type { DataContext } from "./context.js";
+import type { Rule } from "./optimize/index.js";
 /** The shaped result of a top-level query, discriminated by consumer. */
 export type OqxResult = {
     consumer: "collect";
@@ -24,9 +25,47 @@ export type OqxResult = {
 export interface Engine {
     run(query: Query, bindings: readonly unknown[]): OqxResult;
 }
+/** What the optimizer did at run time (for tests and diagnostics): an index
+ * built (or taken from the context) for a collection and path, a probe that
+ * selected a bucket, a context index that answered a probe with rows directly
+ * (`lookupRows`: the collection was never materialized), a memoized invariant
+ * block reused, a consumer answered from a cardinality, or a probe that fell
+ * back to the scan. */
+export type TraceEvent = {
+    kind: "index";
+    path: readonly string[];
+    rows: number;
+    source: "engine" | "context";
+} | {
+    kind: "probe";
+    paths: readonly (readonly string[])[];
+    candidates: number;
+} | {
+    kind: "lookup";
+    path: readonly string[];
+    candidates: number;
+} | {
+    kind: "memo";
+    op: OpNode;
+} | {
+    kind: "cardinality";
+    count: number;
+} | {
+    kind: "fallback";
+    reason: "unstable" | "index" | "probe-value" | "not-a-collection";
+};
+export interface InMemoryEngineOptions {
+    /** The optimizer rules to apply to nested blocks (default `DEFAULT_RULES`);
+     * `[]` runs the naive scan everywhere. */
+    rules?: readonly Rule[];
+    /** Receives a `TraceEvent` for each optimizer action during `run`. */
+    trace?: (event: TraceEvent) => void;
+}
 export declare class InMemoryEngine implements Engine {
     private ctx;
-    constructor(context?: DataContext);
+    private rules;
+    private trace;
+    constructor(context?: DataContext, options?: InMemoryEngineOptions);
     run(query: Query, bindings?: readonly unknown[]): OqxResult;
     private boundOf;
     private matches;
@@ -41,13 +80,27 @@ export declare class InMemoryEngine implements Engine {
     private itemValue;
     private evalWhere;
     private evalWhereOp;
+    private evalWhereOpPlanned;
     private opRows;
     private anyMatch;
     private iterMatchRows;
     private matchRows;
+    private planOf;
+    private memoized;
+    private access;
+    private scan;
+    private reproject;
+    private materialize;
+    private lookupRows;
+    private probe;
+    private outerValue;
+    private contextIndex;
+    private indexFor;
     private evalCollectValue;
+    private evalCollectValuePlanned;
     private sortScopes;
     private evalExpr;
+    private evalRaw;
     private resolveIn;
     private evalCall;
 }

package/dist/engine.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"engine.d.ts","sourceRoot":"","sources":["../src/engine.ts"],"names":[],"mappings":"AAeA,OAAO,KAAK,EACV,KAAK,EACN,MAAM,UAAU,CAAC;AAClB,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,cAAc,CAAC;AAOhD,yEAAyE;AACzE,MAAM,MAAM,SAAS,GACjB;IAAE,QAAQ,EAAE,SAAS,CAAC;IAAC,IAAI,EAAE,OAAO,EAAE,CAAA;CAAE,GACxC;IAAE,QAAQ,EAAE,QAAQ,CAAC;IAAC,MAAM,EAAE,OAAO,CAAA;CAAE,GACvC;IAAE,QAAQ,EAAE,MAAM,CAAC;IAAC,IAAI,EAAE,OAAO,CAAA;CAAE,GACnC;IAAE,QAAQ,EAAE,OAAO,CAAC;IAAC,KAAK,EAAE,MAAM,CAAA;CAAE,GACpC;IAAE,QAAQ,EAAE,OAAO,CAAC;IAAC,GAAG,EAAE,OAAO,GAAG,IAAI,CAAA;CAAE,GAC1C;IAAE,QAAQ,EAAE,QAAQ,CAAC;IAAC,GAAG,EAAE,OAAO,GAAG,IAAI,CAAA;CAAE,CAAC;AAEhD,6EAA6E;AAC7E,MAAM,WAAW,MAAM;IACrB,GAAG,CAAC,KAAK,EAAE,KAAK,EAAE,QAAQ,EAAE,SAAS,OAAO,EAAE,GAAG,SAAS,CAAC;CAC5D;AA4BD,qBAAa,cAAe,YAAW,MAAM;IAC3C,OAAO,CAAC,GAAG,CAAc;gBAEb,OAAO,GAAE,WAAkC;IAIvD,GAAG,CAAC,KAAK,EAAE,KAAK,EAAE,QAAQ,GAAE,SAAS,OAAO,EAAO,GAAG,SAAS;IAmD/D,OAAO,CAAC,OAAO;IAef,OAAO,CAAC,OAAO;IAKf,OAAO,CAAC,MAAM;IAId,OAAO,CAAC,KAAK;IASb,OAAO,CAAC,KAAK;IAab,OAAO,CAAC,SAAS;IAiBjB,OAAO,CAAC,iBAAiB;IAuCzB,OAAO,CAAC,UAAU;IA4ElB,OAAO,CAAC,KAAK;IAgBb,OAAO,CAAC,UAAU;IASlB,OAAO,CAAC,SAAS;IAMjB,OAAO,CAAC,SAAS;IAYjB,OAAO,CAAC,WAAW;IAiCnB,OAAO,CAAC,MAAM;IAQd,OAAO,CAAC,QAAQ;IAKhB,OAAO,CAAE,aAAa;IAStB,OAAO,CAAC,SAAS;IAKjB,OAAO,CAAC,gBAAgB;IA2BxB,OAAO,CAAC,UAAU;IAwBlB,OAAO,CAAC,QAAQ;IAsEhB,OAAO,CAAC,SAAS;IAQjB,OAAO,CAAC,QAAQ;CAYjB;AAuGD,8EAA8E;AAC9E,wBAAgB,QAAQ,CAAC,KAAK,EAAE,KAAK,EAAE,QAAQ,EAAE,SAAS,OAAO,EAAE,EAAE,KAAK,EAAE,OAAO,GAAG,SAAS,CAG9F"}
+{"version":3,"file":"engine.d.ts","sourceRoot":"","sources":["../src/engine.ts"],"names":[],"mappings":"AAqBA,OAAO,KAAK,EACV,KAAK,EAAe,MAAM,EAC3B,MAAM,UAAU,CAAC;AAClB,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,cAAc,CAAC;AAMhD,OAAO,KAAK,EAA0B,IAAI,EAAY,MAAM,qBAAqB,CAAC;AAGlF,yEAAyE;AACzE,MAAM,MAAM,SAAS,GACjB;IAAE,QAAQ,EAAE,SAAS,CAAC;IAAC,IAAI,EAAE,OAAO,EAAE,CAAA;CAAE,GACxC;IAAE,QAAQ,EAAE,QAAQ,CAAC;IAAC,MAAM,EAAE,OAAO,CAAA;CAAE,GACvC;IAAE,QAAQ,EAAE,MAAM,CAAC;IAAC,IAAI,EAAE,OAAO,CAAA;CAAE,GACnC;IAAE,QAAQ,EAAE,OAAO,CAAC;IAAC,KAAK,EAAE,MAAM,CAAA;CAAE,GACpC;IAAE,QAAQ,EAAE,OAAO,CAAC;IAAC,GAAG,EAAE,OAAO,GAAG,IAAI,CAAA;CAAE,GAC1C;IAAE,QAAQ,EAAE,QAAQ,CAAC;IAAC,GAAG,EAAE,OAAO,GAAG,IAAI,CAAA;CAAE,CAAC;AAEhD,6EAA6E;AAC7E,MAAM,WAAW,MAAM;IACrB,GAAG,CAAC,KAAK,EAAE,KAAK,EAAE,QAAQ,EAAE,SAAS,OAAO,EAAE,GAAG,SAAS,CAAC;CAC5D;AA2CD;;;;;uBAKuB;AACvB,MAAM,MAAM,UAAU,GAClB;IAAE,IAAI,EAAE,OAAO,CAAC;IAAC,IAAI,EAAE,SAAS,MAAM,EAAE,CAAC;IAAC,IAAI,EAAE,MAAM,CAAC;IAAC,MAAM,EAAE,QAAQ,GAAG,SAAS,CAAA;CAAE,GACtF;IAAE,IAAI,EAAE,OAAO,CAAC;IAAC,KAAK,EAAE,SAAS,CAAC,SAAS,MAAM,EAAE,CAAC,EAAE,CAAC;IAAC,UAAU,EAAE,MAAM,CAAA;CAAE,GAC5E;IAAE,IAAI,EAAE,QAAQ,CAAC;IAAC,IAAI,EAAE,SAAS,MAAM,EAAE,CAAC;IAAC,UAAU,EAAE,MAAM,CAAA;CAAE,GAC/D;IAAE,IAAI,EAAE,MAAM,CAAC;IAAC,EAAE,EAAE,MAAM,CAAA;CAAE,GAC5B;IAAE,IAAI,EAAE,aAAa,CAAC;IAAC,KAAK,EAAE,MAAM,CAAA;CAAE,GACtC;IAAE,IAAI,EAAE,UAAU,CAAC;IAAC,MAAM,EAAE,UAAU,GAAG,OAAO,GAAG,aAAa,GAAG,kBAAkB,CAAA;CAAE,CAAC;AAE5F,MAAM,WAAW,qBAAqB;IACpC;8CAC0C;IAC1C,KAAK,CAAC,EAAE,SAAS,IAAI,EAAE,CAAC;IACxB,sEAAsE;IACtE,KAAK,CAAC,EAAE,CAAC,KAAK,EAAE,UAAU,KAAK,IAAI,CAAC;CACrC;AAcD,qBAAa,cAAe,YAAW,MAAM;IAC3C,OAAO,CAAC,GAAG,CAAc;IACzB,OAAO,CAAC,KAAK,CAAkB;IAC/B,OAAO,CAAC,KAAK,CAA4C;gBAE7C,OAAO,GAAE,WAAkC,EAAE,OAAO,GAAE,qBAA0B;IAM5F,GAAG,CAAC,KAAK,EAAE,KAAK,EAAE,QAAQ,GAAE,SAAS,OAAO,EAAO,GAAG,SAAS;IAwD/D,OAAO,CAAC,OAAO;IAef,OAAO,CAAC,OAAO;IAKf,OAAO,CAAC,MAAM;IAId,OAAO,CAAC,KAAK;IASb,OAAO,CAAC,KAAK;IAab,OAAO,CAAC,SAAS;IAiBjB,OAAO,CAAC,iBAAiB;IAuCzB,OAAO,CAAC,UAAU;IA4ElB,OAAO,CAAC,KAAK;IAgBb,OAAO,CAAC,UAAU;IASlB,OAAO,CAAC,SAAS;IAMjB,OAAO,CAAC,SAAS;IAYjB,OAAO,CAAC,WAAW;IAOnB,OAAO,CAAC,kBAAkB;IAsC1B,OAAO,CAAC,MAAM;IASd,OAAO,CAAC,QAAQ;IAOhB,OAAO,CAAE,aAAa;IAQtB,OAAO,CAAC,SAAS;IAWjB,OAAO,CAAC,MAAM;IAMd,OAAO,CAAC,QAAQ;IAgBhB,OAAO,CAAC,MAAM;IA4Bd,OAAO,CAAC,IAAI;IAIZ,OAAO,CAAC,SAAS;IAKjB,OAAO,CAAC,WAAW;IAOnB,OAAO,CAAC,UAAU;IAgBlB,OAAO,CAAC,KAAK;IAoBb,OAAO,CAAC,UAAU;IAOlB,OAAO,CAAC,YAAY;IAWpB,OAAO,CAAC,QAAQ;IAqBhB,OAAO,CAAC,gBAAgB;IAMxB,OAAO,CAAC,uBAAuB;IA4B/B,OAAO,CAAC,UAAU;IA4BlB,OAAO,CAAC,QAAQ;IAQhB,OAAO,CAAC,OAAO;IAsEf,OAAO,CAAC,SAAS;IAQjB,OAAO,CAAC,QAAQ;CAYjB;AA+HD,8EAA8E;AAC9E,wBAAgB,QAAQ,CAAC,KAAK,EAAE,KAAK,EAAE,QAAQ,EAAE,SAAS,OAAO,EAAE,EAAE,KAAK,EAAE,OAAO,GAAG,SAAS,CAG9F"}