@omgbase/oqx 0.14.1 → 0.18.0

package/CHANGELOG.md

@@ -4,6 +4,47 @@ All notable changes to `@omgbase/oqx` are recorded here. The format follows
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versions follow
 [Semantic Versioning](https://semver.org/) (pre-1.0: a minor bump may break).
 
+## [0.18.0] - 2026-10-10
+
+### Minor
+- **OQX 0.17 — join syntax sugar** (`spec/oqx` 0.16 → 0.17; `cases/sugar.json`, 134 new cases, 1131 in all; GRAMMAR §2/§4, SEMANTICS §5b/§14, AST.md). Everything that is pure sugar desugars **in the parser** to existing nodes, so the optimizer, the pushdown planners and `print` see the explicit form; two shapes are new.
+
+  - **A receiver block without a consumer is `collect`**: `x { … }` ≡ `x collect { … }` in select position, where position and as a whole query (`docs { age > 15 }`). Carve-out: after the last `follow` destination a brace is still the options block (`follow children { depth 2 }`); a destination block keeps its consumer.
+  - **Where-first bodies**: a block whose leading expression is *syntactically* a predicate (a comparison, `in`, `&&`/`||`, a prefix `!`/`is`/`not`, an infix `is`, a call, a literal, a binding, a parenthesized expression, a postfix `!` or bracket chain, a consumer test — anything that is not a bare name, a dotted navigation or a lift) drops `where`: `people exists { age > 50 }`, `jobs collect { pay > 2 }`. Bare names stay projections (`docs { is active }` is the bare-field filter); `{ name where age >= 18 }` keeps the keyword; `{ age >= 18, name }` is an error suggesting that form. The top level is unchanged: `age > 15 from people` is still the parse error (where would precede from).
+  - **Brackets**: `x[p]` → `x first { where p }`; `x[p]!` → `x single { where p }` under a `required` (zero matches is the required error, many the `single` error); `x[n]` / `x[${i}]` → `x first { offset n }` (out of range is absent; a non-integer binding is the `offset` eval error); `x[n]!` → required positional. Chains compose (`refs(company)[0].name`, `jobs[pay > 2][0]`); a negative or fractional literal index is a parse error; `[`/`]` are tokens.
+  - **Postfix `!` = required** — new node `{ kind: "required", expr }` / `Expr::Required`: the value, or the eval error `` `<expr>!` is absent `` (`on <id>` when the row has a scalar identity). Never a filter or a coercion (`0!`, `""!` are values); tightest precedence (`refs(c)[0]!.name` vs `refs(c)[0].name!`); a `!` right after a complete operand is postfix, a `!` at operand start is the negation (`!=` stays one token: write `a! == b` with the space).
+  - **`is` / `not` / `and` / `or`** become keywords (reserved: `from where select is not and or` + the literal words). Prefix `is x` ≡ `!!x`, `not x` ≡ `!x`. Infix `x is y` / `x is not y` — new `binary` ops `"is"` / `"is not"` (`BinaryOp::Is` / `IsNot`) — compare **identity** per SEMANTICS §16 (a row's `id` when present, else structural), so `$it is ^$it` compares rows and for scalars `is` ≡ `==`; comparison precedence, non-chainable. `and` / `or` are exact synonyms of `&&` / `||` (same nodes, precedence, short-circuit and value semantics; `print` still writes the symbols). `in` over rows is unchanged (deferred).
+  - **Alias reuse in `select`**: an item may reference the items to its left (`boss: ^people[id == ^manager], bossName: boss.name`), inlined by `resolveAliases` / `resolve_aliases` like a `where` alias; a reference to an item to the right is a parse error (a cycle when it refers back). **Behavior change:** `a: b, b: a` is now the cycle error (it was a field swap), and `senior: age > 50, s: senior` projects the alias (it was the row field `senior`).
+  - **A directive may stand in value position** (`jobs first { }.pay`): this is how `print` writes a bracket chain back; `op` joins the `Expr` family by position (`isExpr` stays false for it; `transform` rebuilds it without handing it to the mapping). AST.md §8 records the additions (minor; nothing renamed or removed).
+  - Existing fixtures that turned from error into meaning: `people exists { age > 50 }`-style blocks (`where` ×4, `errors-parse` consumer-after-projection re-queried as `{ name, jobs exists { } }`, `projection` call-in-block re-queried as `{ name, has(budget) }`), the `[` lex-error case (now a token).
+  - **omgbase binding**: the pushdown planners decline `is` / `is not` (identity is the engine's structural notion, not SQL's `IS`), `required` and value-position directives (they may raise), so they stay residual; `query_syntax` gains a "Sugar" block; `docs/query-language.md` §3.6. The surface fixture `implicit-where-in-block` re-queried as `{ kind == "md:task", name }` (still the loud error pointing at `where`).
+  - `@omgbase/oqx-syntax`: `[`/`]` are bracket punctuation (no longer lex errors), `is`/`not`/`and`/`or` are keywords highlighted as word operators, `LANGUAGE_VERSION` 0.17; assets regenerated.
+- **OQX 0.18 — the root scope is a row, absolute scope references, canonical `and`/`or`** (`spec/oqx` 0.17 → 0.18; `cases/absolute-refs.json` new (24), `outer-refs` +6 (the `$it is absent at the root scope` case re-queried as the root-object cases), `limit-offset` +2: 1163 cases in all; GRAMMAR §1/§3/§4/§5/§6, SEMANTICS §2/§8/§18/§19, AST.md §1/§3/§6/§8, README).
+
+  - **The root scope is a row: the host's root object.** SEMANTICS §2 step 1 no longer reads "absent at the root": `$it` at the root scope is the host's root object, so from a top-level row `^$it` is that object, `entries(^$it)` enumerates the named roots in insertion order, `^$it.people` ≡ `^people` (and works as a receiver), `^^$it` from depth two is the same object, and `$it values from $it` is the object as one row. The seam gained **`DataContext.rootObject?(): unknown`** (TypeScript, optional — absent keeps the root row absent) / **`fn root_object(&self) -> Value`** (Rust, provided method defaulting to `Value::Undefined`); `DefaultContext` returns the roots record it was constructed with. Step 5 at the root still reads `ctx.root(name)` — a bare name is never a property of the root object — so hosts with lazy roots are unaffected, and a host-provided root-level name (omgbase will expose the repository id as `$id`, cashing in `$repo.$id` → `^$id` from depth one, `^^$id` from depth two) is readable through `root(name)`. A top-level `limit`/`offset` is evaluated in the root scope *without* its row (SEMANTICS §18): `^$it` reached through a bracket inside the bound is absent, pinned.
+  - **`N^name` — absolute scope references.** An unsigned integer literal immediately followed by `^` (no whitespace) names a scope by its syntactic depth (root 0, top-level rows 1, +1 per block body, a destination block's rows one below the frontier row, a follow `where` one deeper than the frontier row — the `visit` depth): `0^docs`, `0^$it`, `1^$path`, `0^$id`, and as a lift target `1^tasks: text`. It **desugars in the parser** to the existing `outer` node (or lift) with `levels = currentDepth − N`; no new AST shape, `print` writes the carets. `N ≥ currentDepth` is the parse error `scope N does not enclose this block (the current scope is depth D)`; `1 ^x` (a space) and `1.0^x` stay the stray-`^` error. Composes with brackets, calls and receivers (`0^docs[0]`, `x[p == 0^flag]`, `entries(0^$it)`, `0^$it.docs collect { … }`, `follow 0^people collect { where manager == 1^id }`, `follow next { where prev == 1^id && 0^k == 2 }`, `limit 1^n`). Both parsers now track the scope depth of every position.
+  - **Canonical `print` writes `and` / `or`** for the `where` tree's `and`/`or` nodes and for `logical` expressions (both engines); `!` stays `!`, `is`/`is not` were already words, the JSON `logical.op` stays `"&&"`/`"||"`. Consequence: every query text omgbase's `graph` macro prints flips from `||` to `or` — the pinned `graph` query strings in `spec/surface` (`reads.json`, and the `omgbase-surface` unit test for the multi-root seed) are regenerated; nothing else in the surface fixtures changes.
+  - **omgbase binding** (`@omgbase/core` `oqx-js/context.ts`, `omgbase-surface` `context.rs`): the root object is the repository root (`$repo`), so `^$it` from a top-level row is `$repo` and `^$it.docs` the docs scan. The `^docs`/`$repo` redesign and the `$id` root name are a separate change.
+  - `@omgbase/oqx-syntax`: the `N^` lexeme highlights as one outer-reference token (`lift`), `LANGUAGE_VERSION` 0.18; assets regenerated.
+
+## [0.16.0] - 2026-10-09
+
+### Minor
+- **OQX 0.16 — the AST is a first-class contract** (`spec/oqx/AST.md`, fixtures `cases/ast.json` + `cases/ast-spans.json`, 70 new cases; 959 in all). Both engines now produce one normalized tree a tool can reflect on without re-parsing:
+
+  - **Spans on every node**: `span: [start, end)` in Unicode code points over the raw source (a template's `rawSource`, where a binding occupies its `${n}` marker). The TypeScript lexer counted UTF-16 units; it now counts code points like the Rust one, so the `(at offset N)` of a lex/parse error is the same number in both engines. `toUtf16(span, source)` converts for editors.
+  - **Normalized shape**: `kind` on every node (`query`, `subquery`, `follow`, `order` added), optionals materialized (`where`/`limit`/`offset`/`countCmp: null`, `distinct`/`values: false`), operators and consumers serialized as their source words, the never-produced `Expr.index` variant removed. `toJSON(query)` / `ast_to_json(&query)` emit `{ "oqx": "0.16", "kind": "query", … }`; the Rust crate derives serde under the `json` feature (`query_from_json` reads it back).
+  - **`where` keeps its surface form**: the parser no longer inlines `select` aliases into `where` (it only validates them — every `errors-parse` fixture is unchanged); the substitution is the exported pure `resolveAliases` / `resolve_aliases`, applied exactly once by the run entry points (`oqx`, `run`, `execute`, `runQuery`; `run_query`, `execute`). **An `Engine.run` now evaluates the query it is given** — code that drives `InMemoryEngine`/`PlannedEngine` directly with a query whose `where` uses aliases must resolve first (resolution is not idempotent: `name: name.upper() … where name`).
+  - **Traversal**: `visit(root, { enter, leave })` with `{ path, clause, depth }` (scope depth, 0 = root) and `transform(root, f)`, both driven by one child-key table (`CHILDREN`; Rust `walk::visit`/`transform` with a `Visitor` trait, `Node`, `Clause`); `stripSpans` / `strip_spans`; builders `build.*` / `oqx::build::*` for hand-made nodes (empty span `[0, 0]`).
+  - **Canonical printer**: `print(node)` / `print_query` — single spaces, `select` written at the top level, every item aliased, double-quoted strings, minimal parentheses from the precedence table; `printTemplate` → `{ strings, count, indices }` for a tree with bindings (`print` throws, new error stage `print`). Law, enforced by both spec runners over every fixture query: `strip(parse(print(parse(q)))) ≡ strip(parse(q))`.
+  - **Migration** for anyone constructing AST literals (after 0.14's `receiver` → `destinations`): add `kind` to `Query`/`Subquery`/`Follow`/`OrderSpec`, `span` to every node, write the materialized fields (`limit: null`, `countCmp: null`, `distinct: false`, `values: false`), drop `index`; or use `build.*`. Rust: struct variants gained `span` (match with `..`), `Expr::Lit(v)` is `Expr::Lit { value, span }`, `Token` gained `end`.
+  - **omgbase** (`@omgbase/core`, `omgbase-surface`): the runner's `$self` row-function rewrite is a `transform`, the `semantic("…")` scan a `visit`, aliases are resolved once before the runner's own rewrites; the `graph` macro builds its `follow` walk with the builders and `print`s it instead of splicing strings — the generated query text and every `spec/surface` result are unchanged. `@omgbase/oqx-syntax`'s `LANGUAGE_VERSION` moves to `0.16` (assets regenerated).
+
+## [0.15.0] - 2026-10-09
+
+### Minor
+- OQX language **0.15**: the intrinsic naming the current item is **`$it`**; `$value` is no longer an intrinsic and has no synonym. `$it` is the scope's row itself — the top-level row, a nested block's inner item, each element of a scalar collection, the entry's value inside `entries(x)` — absent at the root, with `^$it` the enclosing row, `$it.field` navigation, `$it values`, `$it` as a `follow … { by $it }` identity. A row property literally named `$it` is shadowed by the intrinsic. `$value` is now an ordinary property name under SEMANTICS §2 step 5, exactly like the non-metadata `$` names since 0.13: `{ "$value": 3 }` projects `3`, a row without one reads absent, over scalars it is absent, and inside an `entries()` scope it is a property of the entry's *value*. Why: "self" implies a fixed referent while the row changes per scope; "it" is the pronoun for whatever is being handled; keeping `$value` as a synonym would spell one concept two ways. `$key` is unchanged, and the `{ key, value }` record of `entries(x)` used as a plain value is unchanged (that is a value shape, not the intrinsic). **Migration:** replace `$value` with `$it` in every query (`s/\$value/\$it/g`); nothing else moves. New `intrinsics` fixtures pin the rule; every other fixture query was rewritten with its expectation untouched. omgbase's query surface (`query`, `query_syntax`, the CLI help, `docs/query-language.md`, the tutorial, `spec/surface` fixture queries) follows: a `$value` in an omgbase query now reads a frontmatter/attrs key so named.
+
 ## [0.14.1] - 2026-10-09
 
 ### Patch

package/README.md

@@ -16,7 +16,7 @@ JavaScript tagged template.
 > OQX is a language with more than one implementation; the specification is
 > [`spec/oqx`](https://github.com/omgbase/omgbase/tree/main/spec/oqx). The
 > language version is the package version's `major.minor` (`LANGUAGE_VERSION`,
-> `"0.14"`); the patch digit is this implementation's own.
+> `"0.17"`); the patch digit is this implementation's own.
 > Requirements: Node ≥ 22.13 for `@omgbase/oqx/sqlite`; Node ≥ 22.18 to run the
 > test suite (see [Requirements](#requirements)).
 
@@ -65,9 +65,12 @@ from <collection>          from ${people}
 
 **`select` is the only keyword you may drop, and only when the projection comes
 first** (`name, id from ${people}`). Every other clause always carries its
-keyword — in particular a predicate is never implicit, so a block filters with
-`where` (`jobs exists { where !end }`), and `from ${people} count` is an error
-rather than a projection of a field called `count`. Writing a clause out of
+keyword — with one piece of sugar (0.17): a **block** whose leading expression
+is syntactically a predicate is a `where`-first body (`jobs exists { !end }` ≡
+`jobs exists { where !end }`; a bare name still projects, so `jobs { employer }`
+projects and `jobs { is current }` filters — see §10). At the top level a
+predicate always needs `where`, and `from ${people} count` is an error rather
+than a projection of a field called `count`. Writing a clause out of
 order is a parse error that names the order (`` `select` must come before
 `from` — OQX clause order is select, from, where, follow, order by, limit,
 offset ``). The same body grammar applies inside every consumer block
@@ -151,45 +154,53 @@ projection and inside any `collect` / `first` / `single` block, and composes wit
 employers as strings, not `{ employer }` records). An alias, if present, is
 ignored; a lift (`^name:`) cannot be combined with it.
 
-**`$value` — the current item.** Every scope has a current value; `$value` is
+**`$it` — the current item.** Every scope has a current value; `$it` is
 that exact value, whatever its type (an object row or a plain scalar). Bare names
-still navigate it (`name` ≡ `$value.name`), so `$value` matters exactly where
+still navigate it (`name` ≡ `$it.name`), so `$it` matters exactly where
 there is nothing to navigate: collections of numbers or strings, or handing the
 whole row somewhere. Together with `values` this makes scalar collections
 first-class:
 
 ```js
 const scores = [10, 60, 70, 45];
-oqx`$value values from ${scores} where $value > 50`;              // [60, 70]
-oqx`$value values from ${scores} order by $value desc`;           // [70, 60, 45, 10]
+oqx`$it values from ${scores} where $it > 50`;              // [60, 70]
+oqx`$it values from ${scores} order by $it desc`;           // [70, 60, 45, 10]
 
 const players = [{ name: "Ann", scores: [10, 60, 70] }, { name: "Ben", scores: [45] }];
-oqx`name, big: scores collect { $value values where $value > 50 } from ${players}`;
+oqx`name, big: scores collect { $it values where $it > 50 } from ${players}`;
 // [{ name: "Ann", big: [60, 70] }, { name: "Ben", big: [] }]
 
-oqx`employee: $value from ${people} where name == "Bob"`;         // [{ employee: <Bob> }]
+oqx`employee: $it from ${people} where name == "Bob"`;         // [{ employee: <Bob> }]
 ```
 
-Inside a nested block `$value` is the inner item; the enclosing row is `^$value`
-(§3). At the root scope (before any row) it is absent.
+Inside a nested block `$it` is the inner item; the enclosing row is `^$it`
+(§3). At the root scope `$it` is the context's **root object** (language 0.18;
+`DataContext.rootObject()` — `DefaultContext` returns the roots record, so
+`^$it` from a top-level row is `{ people, … }` and `entries(^$it)` lists the
+named roots). `$it` is the only name
+for the current item (language 0.15; through 0.14 it was spelled `$value`, and
+there is no synonym): `$value` is now an ordinary property read, like any other
+non-metadata `$name` — a row `{ "$value": 3 }` projects `3`, a row without one
+reads absent. A row that itself owns a `$it` property is shadowed by the
+intrinsic; reach it with `^` from a nested scope or rename it upstream.
 
 **`entries(x)` and `$key` — records to collections, explicitly.** A plain object
 is **not** iterable: `from ${obj}` is one row (the object). `entries(obj)`
 converts it into a collection of entries, and inside such a scope the current
-item is the property's **value** — `$value` and bare names read it — while
+item is the property's **value** — `$it` and bare names read it — while
 **`$key`** is the property's key:
 
 ```js
 const settings = { theme: "dark", fontSize: 14, autosave: true };
-oqx`key: $key, value: $value from entries(${settings})`;
+oqx`key: $key, value: $it from entries(${settings})`;
 // [{ key: "theme", value: "dark" }, { key: "fontSize", value: 14 }, { key: "autosave", value: true }]
-oqx`$key values from entries(${settings}) where $value != "dark"`;   // ["fontSize", "autosave"]
+oqx`$key values from entries(${settings}) where $it != "dark"`;   // ["fontSize", "autosave"]
 
 const flags = { beta: { on: true }, legacy: { on: false } };
 oqx`$key values from entries(${flags}) where on`;                    // ["beta"]  (bare `on` reads the value)
 
-oqx`name, on: entries(prefs) collect { $key values where $value } from ${users}`;   // as a nested receiver
-oqx`name from ${users} where entries(prefs) exists { where $key == "dark" && $value }`;
+oqx`name, on: entries(prefs) collect { $key values where $it } from ${users}`;   // as a nested receiver
+oqx`name from ${users} where entries(prefs) exists { where $key == "dark" && $it }`;
 ```
 
 `entries(array)` yields numeric index keys, a `Map` yields its entries, and
@@ -254,9 +265,10 @@ against the value of `x`; a string in `x` can't inject operators or identifiers.
 
 **`where` sees the `select` aliases.** A name defined in the same body's
 projection may be used in its `where`; an alias shadows a same-named field
-there. It is a compile-time rewrite — the alias's expression is substituted
-inline — so it costs nothing at run time and a storage planner still sees an
-ordinary predicate:
+there. The alias's expression is substituted inline before evaluation
+(`resolveAliases`, applied once by the run entry points — the parsed tree keeps
+what you wrote), so it costs nothing at run time and a storage planner still
+sees an ordinary predicate:
 
 ```js
 oqx`select name, adult: age >= 18 from ${people} where adult`;        // ≡ where age >= 18
@@ -275,7 +287,11 @@ an alias. `order by` is not rewritten — it reads row fields (§8).
 against the **current row only**. If the row lacks that property the value is
 absent — it never falls through to an enclosing row. To correlate with an
 enclosing scope you say so explicitly with `^name` ("exactly one scope out";
-`^^name` for two, and so on):
+`^^name` for two, and so on) — or by **depth** with `N^name` (language 0.18):
+`0^people` is the root scope's `people` from any depth, `1^name` the top-level
+row's `name`, `0^$it` the root object. `N^` is an integer touching a caret
+(`1 ^x` is not one); it is pure sugar for the caret form with `levels = depth −
+N`, and `N ≥ the current depth` is a parse error.
 
 ```js
 const accounts = [
@@ -524,7 +540,7 @@ fully), not a short-circuiting `exists`.
 ### 8. Ordering
 
 `order by <expr> [asc|desc]`, comma-separated for tie-breaks. Absent values sort
-last. The sort expression reads the row (fields, `$value`, recursion
+last. The sort expression reads the row (fields, `$it`, recursion
 intrinsics) — it is not rewritten against the `select` aliases, so
 `order by decade` sorts by a field called `decade`, not by `decade: age / 10`.
 
@@ -637,22 +653,69 @@ with `^` the enclosing scope, as the body does. Give `follow` a stable identity
 (`.id` or `by`) when your relation returns fresh objects rather than shared
 references.
 
+### 10. Sugar (0.17)
+
+Every form below is shorthand for an explicit directive — it **desugars in the
+parser** to the nodes you already know (`print` writes the explicit form, the
+optimizer and any pushdown planner see it), so nothing here has semantics of its
+own except the postfix `!` and the infix `is`:
+
+```js
+// A receiver block without a consumer is `collect`; a block whose LEADING
+// expression is syntactically a predicate drops `where`.
+oqx`name, current: jobs { !end_date } from ${people}`
+//   ≡ name, current: jobs collect { where !end_date } from ${people}
+oqx`${people} { age > 40 }`                  // ≡ ${people} collect { where age > 40 }
+oqx`${people} count { jobs exists { } }`     // a consumer test is predicate-shaped too
+oqx`${people} first { name }`                // a bare name still projects — unchanged
+oqx`${people} first { is active }`           // the bare-field filter: `is x` ≡ `!!x`, `not x` ≡ `!x`
+
+// Brackets: `[p]` is `first { where p }`, `[n]` is `first { offset n }`;
+// a trailing `!` requires the result (and makes the predicate form `single`).
+oqx`name, boss: ^people[id == ^manager].name from ${people}`   // the first match, navigated
+oqx`name, first_job: jobs[0], primary: jobs[pay > 2]! from ${people}`
+//   jobs[0]          ≡ jobs first { offset 0 }        (out of range ⇒ absent)
+//   jobs[pay > 2]!   ≡ jobs single { where pay > 2 }!  (zero or many matches ⇒ an eval error)
+
+// Postfix `!` — required: the value, or an eval error naming the expression
+// (and the row's id). Never a filter, never a coercion: 0! and ""! are values.
+oqx`select id!, title from ${docs}`           // insist on an identity before a write
+// Tightest precedence: `refs(c)[0]!.name` requires the lookup, `refs(c)[0].name!` the name.
+
+// `is` / `is not` compare IDENTITY (a row's id when present, else structurally):
+oqx`name from ${people} where manager is null`            // absent (same as == null)
+oqx`${people} count { ^people exists { $it is ^$it } }`   // rows compare; `==` leaves that unspecified
+// `and` / `or` are exact synonyms of `&&` / `||`: `title or path` coalesces.
+// `print` writes the words (language 0.18); `!` stays `!`.
+
+// A select item may use the items to its left (inlined like a `where` alias):
+oqx`boss: ^people[id == ^manager], bossName: boss.name from ${people}`
+```
+
+Reserved words, never a bare field name: `from where select is not and or` and
+the literal words `true false null`. The top level is unchanged — `age > 15
+from ${people}` stays an error, because `where` would precede `from`.
+
 ### Cheat-sheet
 
 ```
 select … from … where … follow … order by … limit N offset N   the fixed clause order (each at most once)
 name, alias: expr, nested: rel collect { … }   projection (`select` may be dropped only here, in first position)
 from ${source}                                  source collection (required at the top level)
-where a == b && rel exists { where … } || !c    predicate tree + nested ops (`where` is never implicit)
-where alias                                      `where` may use this body's select aliases (inlined; an alias shadows a field)
+where a == b && rel exists { where … } || !c    predicate tree + nested ops (`and`/`or` ≡ `&&`/`||`)
+rel { pay > 2 }  /  rel { employer }            a block without a consumer is collect; a leading predicate is where-first, a bare name projects
+rel[p]  /  rel[p]!  /  rel[0]  /  rel[0]!       first match / exactly one (required) / positional / required positional
+x!                                               required: the value, or an error when absent (tightest; `a!.b` vs `a.b!`)
+is x / not x  /  x is y / x is not y             truthiness (≡ !!x / !x) / identity (a row's id, else structural; absent ≡ null)
+where alias  /  a: expr, b: a                    `where` and later items may use this body's select aliases (resolved before evaluation; an alias shadows a field)
 where rel none { … }                             zero rows (≡ !rel exists { … }; "all" = none over the complement)
 where x in lo..hi / lo...hi / ..hi / lo..        range membership (incl. / excl. / open-ended)
 where x in range(field)                          coerce a string field to a range, then test coverage
 name                                             the CURRENT row's field only (never climbs)
-$value                                           the current item itself (a scalar row, or the whole object)
-from entries(obj) … $key / $value                a record's properties as a collection (key + value; bare names read the value)
+$it                                              the current item itself (a scalar row, or the whole object)
+from entries(obj) … $key / $it                   a record's properties as a collection (key + value; bare names read the value)
 <expr> values                                    scalar projection: the value, not a { name: value } record
-^name / ^^name                                   read an enclosing row's field (exactly N scopes out); ^$value = the enclosing row
+^name / ^^name                                   read an enclosing row's field (exactly N scopes out); ^$it = the enclosing row
 ^name: expr  /  ^^name: expr                     lift/export a value N scopes out (flatten-append)
 ^rel collect { … }  /  ^^root exists { … }       nested consumer over an enclosing row's relation / a named root
 entries(rel) exists { … }                        a free-function call may be a receiver
@@ -664,6 +727,78 @@ follow ^people collect { where manager == ^id } a destination block: successors
 ${source} <collect|exists|none|count|first|single> { … }   whole-query consumer
 ```
 
+## The AST: reflecting on a query
+
+`parse(source)` returns the query's abstract syntax tree, and since language
+0.16 that tree is a **public contract** shared by both implementations
+([`spec/oqx/AST.md`](https://github.com/omgbase/omgbase/tree/main/spec/oqx/AST.md)):
+a tool reflects on a query — which relations it names, which clauses it has,
+where in the text each piece sits — without re-parsing. The objects are plain
+data (no parent pointers, no methods) and *are* the JSON shape.
+
+- **Every node has `kind` and `span`.** `span` is `[start, end)` in Unicode
+  **code points** over the raw source (for a template, `rawSource(strings)`,
+  where a binding occupies its `${n}` marker); `toUtf16(span, source)` converts
+  for an editor that counts UTF-16 units. A parenthesized operand spans its
+  parentheses; a block (`subquery`) spans its braces.
+- **Optionals are materialized** — `where: null`, `limit: null`, `countCmp: null`,
+  `distinct: false`, `values: false` — so nothing has to be defaulted.
+- **`where` keeps its surface form.** A `select` alias used in `where` stays an
+  `ident` in the tree; `resolveAliases(query)` is the pure substitution the run
+  entry points (`oqx`, `run`, `execute`, `runQuery`) apply exactly once before
+  evaluation. An `Engine.run` evaluates the query it is given — if you drive an
+  engine yourself, resolve first.
+
+```js
+import { parse, visit, transform, print, printTemplate, toJSON, stripSpans, build } from "@omgbase/oqx";
+
+const q = parse('select name, n: jobs collect { employer } from people where jobs exists { where !end }');
+
+// visit: every node, with its ancestors, the clause it sits in, and the scope depth
+const relations = [];
+visit(q, {
+  enter(node, { clause, depth }) {
+    if (node.kind === "op") relations.push({ relation: print(node.receiver), clause, depth });
+  },
+});
+// → [{ relation: "jobs", clause: "select", depth: 1 }, { relation: "jobs", clause: "where", depth: 1 }]
+
+// transform: rebuild with every expression mapped (children first; untouched nodes keep identity + spans)
+const rewritten = transform(q, (e) => (e.kind === "ident" && e.name === "name" ? build.member(build.ident("person"), "name") : e));
+
+// print: the canonical source — single spaces, `select` written, minimal parentheses
+print(rewritten);
+// 'select name: person.name, n: jobs collect { employer: employer } from people where jobs exists { where !end }'
+
+toJSON(q);        // { oqx: "0.17", kind: "query", span: [0, 93], … } — the shared JSON document
+stripSpans(q);    // the same tree without spans, for shape comparisons
+```
+
+`visit(root, { enter?, leave? })` walks depth-first in canonical source order;
+`enter` may return `false` to skip a subtree. Its context is `{ path, clause,
+depth }` — the ancestors, the clause (`source` · `from` · `where` · `select` ·
+`orderBy` · `limit` · `offset` · `follow` · `follow.destination` · `follow.where`
+· `follow.frontier` · `follow.by`; `null` at the root) and the **scope depth**
+the node is evaluated at (0 = the root scope: a top-level source or bound; 1 =
+a top-level row; a block's body is one deeper than its receiver). Both walks
+are driven by one child-key table, `CHILDREN`, so a future field is added in one
+place.
+
+`print(node)` is canonical, and the law `stripSpans(parse(print(parse(q))))`
+≡ `stripSpans(parse(q))` holds for every spec fixture (both runners check it);
+`print(parse(q)) === q` is *not* a law — spelling is normalized. A binding is a
+value, never source text: `print` throws (`OqxError`, stage `"print"`) on one,
+and `printTemplate(node)` returns `{ strings, count, indices }` — the fragments
+around each binding and the binding each gap stands for (the canonical clause
+order can move one past another), to re-run with `parseTemplate(strings,
+count)` and the values permuted by `indices`.
+
+`build.*` constructs nodes with the empty span `[0, 0]` and the materialized
+defaults (`build.query(source, { select, where, … })`, `build.op(receiver,
+"collect", build.subquery({ … }))`, `build.field`, `build.collect`,
+`build.follow`, `build.ident`, `build.lit`, `build.binary`, `build.and`, …), so
+a tree you assemble prints with `print` and has the shape of a parsed one.
+
 ## Data context: string queries and named roots
 
 When you don't need interpolation, `execute` runs a plain string query against a
@@ -682,7 +817,8 @@ the full discriminated result (`{ consumer, … }`).
 ## Architecture: adapting to other storage & query systems
 
 OQX is layered so it can be the front-end for query systems far beyond in-memory
-objects. The parsed `Query` AST is the host-agnostic IR; execution is pluggable.
+objects. The parsed `Query` AST is the host-agnostic contract (above); execution
+is pluggable.
 
 ```
 Query AST  ─┬─ InMemoryEngine(DataContext)     tier 1/2 — drive any data model
@@ -700,8 +836,10 @@ than approximate it. The conformance suite verifies this.
 
 The engine never touches host objects directly; it asks a `DataContext` to
 resolve named roots, read properties/relations, coerce results to rows, and
-compute identity. Implement it to query an ORM graph, a remote API, or lazily
-loaded relations — the query *semantics* stay in OQX. Name resolution is simple
+compute identity (and, optionally, to supply a `rootObject()` — the root
+scope's row, what `$it` is at the root and `^$it` from a top-level row; without
+one the root row is absent). Implement it to query an ORM graph, a remote API,
+or lazily loaded relations — the query *semantics* stay in OQX. Name resolution is simple
 for a context: a bare `field`, a `.field` segment, and a `^field` outer
 reference each become one `get(row, key)` against exactly the row of the scope
 they name, so a computed relation only needs `get` to know about it:
@@ -805,7 +943,7 @@ results and on errors.
 
 - **Correlated equality → hash probe.** A top-level `&&` conjunct of the block's
   `where` of the form `local == outer` (either side) — `local` a bare identifier
-  or member chain on the block's row (`customer_id`, `meta.id`, `$value`),
+  or member chain on the block's row (`customer_id`, `meta.id`, `$it`),
   `outer` anything that reads nothing from that row (`^customer_id`, `^^x.id`,
   a `${…}` binding, a literal, arithmetic over them) — is answered from a hash
   index on the receiver built **once per run** per collection and path. The
@@ -903,7 +1041,8 @@ adapter lives on the `@omgbase/oqx/sqlite` subpath.
   consumer-shaped value: an array for `collect`, a boolean for `exists` / `none`,
   a number for `count`, a record or `null` for `first` / `single`.
 - `parse(source)` — a query string → reusable `Query` AST (an `OqxError` with
-  `stage: "lex" | "parse"` on bad input).
+  `stage: "lex" | "parse"` on bad input); `parseTemplate(strings, count)` the
+  tagged-template form, with `binding` nodes for the `${…}` slots.
 - `execute(source, roots?)` — parse and run a string against named roots
   (`{ people }`), returning the consumer-shaped value.
 - `run(query, opts?)` — run a parsed `Query`, returning the full `OqxResult`.
@@ -1027,11 +1166,30 @@ adapter lives on the `@omgbase/oqx/sqlite` subpath.
   - `map?: (raw) => row` — a custom raw-SQL-row → query-row mapper (overrides
     `jsonColumns`).
 
+**The AST** (see [above](#the-ast-reflecting-on-a-query); `spec/oqx/AST.md` is normative)
+
+- `visit(root, { enter?, leave? })` / `transform(root, f)` — the generic walks
+  (`Visitor`, `VisitContext`, `Clause` are the types); `CHILDREN` — the
+  child-key table per node kind that drives them.
+- `print(node)` / `printTemplate(node)` — the canonical printer (`Template` is
+  `{ strings, count, indices }`).
+- `resolveAliases(query)` — the `select`-alias substitution the entry points
+  apply before evaluation (SEMANTICS §14).
+- `toJSON(query)` — the tree stamped with `oqx: LANGUAGE_VERSION`; `stripSpans(node)`
+  — the tree without spans (`Stripped<T>`); `toUtf16(span, source)` and
+  `codePointLength(s)` — code-point ↔ UTF-16 offsets; `rawSource(strings)` —
+  the display source of a template.
+- `build` — the node builders (`build.query`, `build.op`, `build.subquery`,
+  `build.field`, `build.ident`, …).
+- `LANGUAGE_VERSION` — the `spec/oqx/VERSION` this implementation conforms to (`"0.17"`).
+
 **Types**
 
 - Every AST node type from `src/ast.ts` is re-exported (`Query`, `Subquery`,
-  `Where`, `Expr`, `OpNode`, `SelectItem`, `OrderSpec`, `Follow`, …) for
-  planners that walk the IR.
+  `Where`, `Expr`, `OpNode`, `SelectItem`, `OrderSpec`, `Follow`,
+  `AstNode`, `AstKind`, `Span`, …) for tools that reflect on queries and
+  planners that walk the tree; `isExpr(node)` tells an expression from the
+  structural nodes.
 
 ## Requirements
 

package/dist/ast.d.ts

@@ -1,91 +1,123 @@
+/** A half-open source range `[start, end)` in Unicode code points over the raw
+ * source (`rawSource` for a template, where a binding occupies its `${n}`
+ * marker). The empty span `[0, 0]` marks a node that did not come from source. */
+export type Span = [start: number, end: number];
 /** Query consumers — how a (sub)query's row set is shaped. `none` is the
  * zero-cardinality complement of `exists` (true iff the block yields no rows). */
 export type Consumer = "collect" | "exists" | "none" | "count" | "first" | "single";
 /** Comparison operators usable in a `count { … } <op> <int>` test. */
 export type RelOp = "==" | "!=" | "<" | "<=" | ">" | ">=";
+/** The identity operators (SEMANTICS §5b, since 0.17). */
+export type IdentityOp = "is" | "is not";
+/** The operator of a `binary` expression: a comparison, an identity test, or arithmetic. */
+export type BinaryOp = RelOp | IdentityOp | "+" | "-" | "*" | "/" | "%";
+/** The operator of a `unary` expression. */
+export type UnaryOp = "!" | "-";
+/** The operator of a `logical` expression. */
+export type LogicalOp = "&&" | "||";
 /** Scalar value/predicate expression, evaluated against a row scope + bindings. */
 export type Expr = {
     kind: "lit";
+    span: Span;
     value: string | number | boolean | null;
 } | {
     kind: "ident";
+    span: Span;
     name: string;
 } | {
     kind: "outer";
+    span: Span;
     levels: number;
     name: string;
 } | {
     kind: "binding";
+    span: Span;
     index: number;
 } | {
     kind: "member";
+    span: Span;
     recv: Expr;
     name: string;
-} | {
-    kind: "index";
-    recv: Expr;
-    index: Expr;
 } | {
     kind: "call";
+    span: Span;
     recv: Expr | null;
     name: string;
     args: Expr[];
 } | {
     kind: "unary";
-    op: "!" | "-";
+    span: Span;
+    op: UnaryOp;
+    expr: Expr;
+} | {
+    kind: "required";
+    span: Span;
     expr: Expr;
 } | {
     kind: "binary";
-    op: string;
+    span: Span;
+    op: BinaryOp;
     left: Expr;
     right: Expr;
 } | {
     kind: "logical";
-    op: "&&" | "||";
+    span: Span;
+    op: LogicalOp;
     left: Expr;
     right: Expr;
 } | {
     kind: "in";
+    span: Span;
     left: Expr;
     right: Expr;
 } | {
     kind: "range";
+    span: Span;
     lo: Expr | null;
     hi: Expr | null;
     exclusiveEnd: boolean;
-};
+} | OpNode;
+/** One `order by` term. */
 export interface OrderSpec {
+    kind: "order";
+    span: Span;
     expr: Expr;
     desc: boolean;
 }
 /** One projection item: a named scalar/navigation value, or a named nested
- * collection consumer. `lift` marks a `^name` one-scope lift. */
+ * collection consumer. `lift` marks a `^name` lift. */
 export type SelectItem = {
     kind: "field";
+    span: Span;
     name: string;
     expr: Expr;
     lift: number;
 } | {
     kind: "collect";
+    span: Span;
     name: string;
     op: OpNode;
 };
+/** The `count { … } <op> <int>` test attached to an `OpNode` — an attribute of
+ * the directive, not a node of its own. */
+export interface CountCmp {
+    op: RelOp;
+    value: number;
+}
 /** A postfix consumer directive over a receiver collection:
  * `<receiver> <op> { <sub> }`, optionally `count { … } <relop> <int>`. */
 export interface OpNode {
     kind: "op";
+    span: Span;
     receiver: Expr;
     op: Consumer;
     sub: Subquery;
-    countCmp?: {
-        op: RelOp;
-        value: number;
-    };
+    countCmp: CountCmp | null;
     /** `distinct` — dedup the rows this directive consumes by their projected
      * value (the `select`), so `count distinct { … }` counts distinct projections
      * and `collect distinct { … }` yields distinct rows. Empty select ⇒ dedup by
      * row identity. Spellable as `<op> distinct { … }` or `{ select distinct … }`. */
-    distinct?: boolean;
+    distinct: boolean;
 }
 /** One `follow` destination: a relation of the current row (an `Expr` receiver), or a
  * destination block — a select-position directive (`collect`/`first`/`single`) re-evaluated
@@ -93,6 +125,8 @@ export interface OpNode {
 export type FollowDestination = Expr | OpNode;
 /** The recursive `follow` clause. */
 export interface Follow {
+    kind: "follow";
+    span: Span;
     destinations: FollowDestination[];
     distinct: boolean;
     where: Expr | null;
@@ -100,8 +134,10 @@ export interface Follow {
     depth: number | null;
     by: Expr | null;
 }
-/** A nested (sub)query body. */
+/** A nested (sub)query body — the `{ … }` of a directive. */
 export interface Subquery {
+    kind: "subquery";
+    span: Span;
     from: Expr[];
     where: Where | null;
     select: SelectItem[];
@@ -109,34 +145,40 @@ export interface Subquery {
     follow: Follow | null;
     /** `values` — scalar projection mode: the (single) projected expression is the
      * row's result itself rather than being wrapped in a `{ name: value }` record,
-     * so `name values` yields `["Bob", …]` and `$value values` yields the rows. */
-    values?: boolean;
+     * so `name values` yields `["Bob", …]` and `$it values` yields the rows. */
+    values: boolean;
     /** `limit N` / `offset N` — bound the row set AFTER where/order/distinct and
      * BEFORE the consumer reduces it, so `count { … limit 5 }` is at most 5 and
      * `first { … offset 1 }` is the second row. Each is a value expression
      * (a literal, a `${…}` binding, or an outer reference) read as part of the
      * block — `^n` is the enclosing row's field, as everywhere inside `{ … }` —
      * and must yield a non-negative integer. */
-    limit?: Expr;
-    offset?: Expr;
+    limit: Expr | null;
+    offset: Expr | null;
 }
 /** The where-clause boolean tree: OQX owns &&/||/!/grouping so consumer ops
  * (invisible to the scalar evaluator) compose with scalar predicates. */
 export type Where = {
     kind: "and";
+    span: Span;
     parts: Where[];
 } | {
     kind: "or";
+    span: Span;
     parts: Where[];
 } | {
     kind: "not";
+    span: Span;
     expr: Where;
 } | {
     kind: "scalar";
+    span: Span;
     expr: Expr;
 } | OpNode;
 /** A top-level OQX query. */
 export interface Query {
+    kind: "query";
+    span: Span;
     source: Expr;
     from: Expr[];
     where: Where | null;
@@ -145,11 +187,23 @@ export interface Query {
     consumer: Consumer;
     follow: Follow | null;
     /** `distinct` — dedup the result rows by their projected value (see OpNode). */
-    distinct?: boolean;
+    distinct: boolean;
     /** `values` — scalar projection mode (see Subquery). */
-    values?: boolean;
+    values: boolean;
     /** `limit N` / `offset N` (see Subquery); evaluated at the root scope. */
-    limit?: Expr;
-    offset?: Expr;
+    limit: Expr | null;
+    offset: Expr | null;
 }
+/** Every node of the tree. `AstNode["kind"]` is the closed set of node kinds. */
+export type AstNode = Query | Subquery | OpNode | Follow | OrderSpec | SelectItem | Where | Expr;
+/** The closed set of node kinds. */
+export type AstKind = AstNode["kind"];
+/** Whether a node is a scalar `Expr` (an `op` in expression position is not: it
+ * is an expression only by position, see AST.md §1). */
+export declare function isExpr(node: AstNode): node is Expr;
+/** The parsed query's JSON form (`toJSON`): the tree with the language version
+ * it was produced under. */
+export type QueryJson = Query & {
+    oqx: string;
+};
 //# sourceMappingURL=ast.d.ts.map