@omgbase/oqx 0.15.0 → 0.20.0

package/CHANGELOG.md

@@ -4,6 +4,66 @@ All notable changes to `@omgbase/oqx` are recorded here. The format follows
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versions follow
 [Semantic Versioning](https://semver.org/) (pre-1.0: a minor bump may break).
 
+## [0.20.0] - 2026-10-11
+
+### Minor
+- OQX 0.20 (`spec/oqx` 0.19 → 0.20; 1188 → 1258 fixtures, three new suites `tuples`, `identity`, `cardinality`):
+
+  - **`follow` loses its options block.** A brace after *any* destination opens that destination's block (`follow children { depth 2 }`, `follow a, children { id != "root" }`, `follow entries(kids) { where … }`), with the 0.17 body rules. Two clauses belong to a destination block alone, after `where` in the fixed order: `depth <1..8>` (this destination expands a frontier row only while its `$depth < n`) and `frontier <expr>` (an occurrence reached through this destination for which it holds is a frontier; read with `^` the walk's enclosing scope; seeds are never frontiers). The former `where` option is the block's `where`; `by` is gone. When several destinations reach a row in one step the first governs the occurrence. **The body that carries the `follow` may carry `frontier <expr>` too** (Brendan, 2026-10-11): it applies to the SEEDS — a seed for which it holds (read in the seed's own scope, `^` the enclosing scope, like `where`) is `$stop == "frontier"` and is not expanded, while it stays in the result — so every block that feeds rows into the walk says which of its rows are frontiers, and the frontier relevant to the current `$it` is defined separately from each destination's. The clause order of every body is `select from where frontier follow order by limit offset` (a destination block's adds `depth` after `frontier`); a body-level `frontier` without a `follow` is the parse error `` `frontier` needs a `follow` ``, a body-level `depth` an error naming the destination block; inside a destination block `frontier` is always the destination's. AST: `follow` keeps only `destinations`/`distinct`; `subquery` gains `frontier`/`depth`, `query` gains `frontier`; `print` writes `follow x collect { where … depth 2 }` and a body's `frontier` between `where` and `follow`.
+  - **Tuples.** `(e1, e2, …)` with two or more elements is a value and a receiver: the elements concatenated with one level of flattening (a collection contributes its items, a row itself; an absent element nothing, a `null` is kept; duplicates kept). `size((a, b))`, `"x" in (tags, aliases)`, `(mom, dad, siblings) { name values }`, `follow (before, after) { depth 2 }` (one destination over the union). New AST node `tuple { items }`.
+  - **Identity is `$key`.** One meaning — the key a row is known by — for `follow` cycle detection, the per-step union, `follow distinct`, `distinct` over unprojected rows, `is`, and reading `$key` on any row. The default is the host's identity through a new `DataContext` seam (TS: optional `identityOf(row)`, replacing the required `identity`, which stays as a deprecated alias for one minor; Rust: `identity` now has a structural default), else the row's structural value; an entry's identity is its key. The language never names an `id` property: the "`id` if present" rule is removed, and the in-memory `DefaultContext` names no identity. A block may project `$key: <expr>` to give the rows it yields that identity (`select name, $key: email from people follow next { $key: email }`); in a destination block `$key:` is not a column. Caveat: identities compare as values across the whole walk, so destinations that can reach the same rows should project the same `$key`.
+  - **Result shape by receiver cardinality.** In row position an array is a collection and a non-array object is one row: a `collect` over it — `doc { title }`, `title from doc`, a nested `d: dst { title }` — yields the one record (or absent when filtered out; `doc { status == "active" }` is a conditional reach-through), not a one-element list; `values` the one value; `first`/`single` that object. A scalar in row position is an eval error (`` `s` is a string ("abc"), not a collection or an object ``) instead of a one-row collection — `list(s)` says it explicitly. A `from` re-projection or a `follow` makes the result plural. Hosts say which of their objects are collections through the new optional `DataContext.isSingle` (TS; default: a non-iterable object is one row) / `is_single` (Rust; default `true`). TS `OqxResult` for `collect` gains `single: boolean` (`unwrap` yields the record or `null`); Rust gains `OqxResult::Record(Option<Value>)`.
+  - **Sugar closures.** `x { … }` ≡ `x collect { … }` wherever a value is accepted, function arguments included (`size(children { is checked })`); `x distinct { … }` ≡ `x collect distinct { … }`.
+
+  omgbase binding (surface stays 3.0; the dialect follows the language):
+
+  - `DataContext.identityOf` (TS) / `identity` (Rust) answers a store row's id for every kind (`doc_id`, `block_id`, `node_id`, `edge_id`), so `$key` on any hit is its id and walks dedup by it as before; the binding no longer relies on the removed `id`-property rule.
+  - The statically single relations `doc`, `block` and `dst` hand out an absent-single marker when they have no row, so `dst { title }` on a link row is one record and, on an external or dangling link, absent — not `[]`; as a value the marker is absent (`dst == null`, `has(dst)`, `dst.title` unchanged). The Rust context's `is_single` marks the lazy root scans as collections.
+  - The `graph` tool compiles to `follow distinct <dir>_links collect { from <dst|doc> depth <n> }`; `query_syntax`, the `query` tool description, both help cards, `docs/query-language.md`, `docs/graph-and-query.md`, `docs/mcp-api.md` and the OQX tutorial describe the 0.20 forms (body-level `frontier` included); the planners treat a destination block's `frontier`/`depth` as its own, never push a tuple down, and a body with `frontier` plans like one with a `follow` — in memory (pinned on both sides). `spec/surface` `query-follow::body-frontier-keeps-seeds-unexpanded` walks the processes with the unverified one kept but not expanded.
+  - `spec/surface` fixtures using the old options block are rewritten to the destination-block forms (`… collect { from dst depth 2 }`; `by type` → `{ $key: type from dst }`) and regenerated; `spec/cli` and `interop.json` regenerated with them.
+  - `@omgbase/oqx-syntax`: `LANGUAGE_VERSION` 0.20; `FOLLOW_OPTIONS` → `DESTINATION_CLAUSES` (`frontier`, `depth`; `by` is no longer a word of the language), assets regenerated.
+
+## [0.19.0] - 2026-10-10
+
+### Minor
+- OQX language 0.19 — outer-reference `follow` destinations take the implicit `collect`. A `follow` destination whose receiver is headed by an outer reference (`^docs`, `^^root.rel`, `0^docs`, `^$it.docs`, with any member chain) can never be a plain destination (plain destinations are relations of the current row), so a `{` right after it now opens a **destination block** with the implicit `collect`: `follow refs(before), ^docs { where ^$path in list(after) }` walks `before` references forward and, backward, every document whose `after` names the frontier row — exactly `^docs collect { … }` with the 0.17 body rules (where-first, projections). Until 0.18 that query failed with the outer-reference error, because the 0.17 carve-out reserved a brace after the last destination for the options block. Any other receiver (a bare relation, a call like `refs(before)` or `entries(kids)`, a binding) keeps that reading: a brace after it is the options block when it is the last destination (`follow children { depth 2 }`, `follow a, children { depth 2 }`) and the "a destination block needs its consumer" error when the brace holds a predicate. An options block may still follow a trailing destination block (`follow ^docs { where … } { depth 2 }`). A plain `follow ^children` (no brace, no consumer) is still an error; its message now names both block spellings: `` `follow` takes a relation of the current row (`follow <relation>`); an outer reference walks a root collection only as a destination block — `follow ^docs { where … }` or `follow ^docs collect { … }` ``. Fixtures: 25 new cases across `follow`, `sugar`, `errors-parse` and `ast` (1188 total); the 0.17 sugar case "an outer-reference destination with a brace is the outer-reference error" is replaced by its positive form. `LANGUAGE_VERSION` is 0.19 in both engines and in `@omgbase/oqx-syntax`; the `query_syntax` reference (`@omgbase/core`, `omgbase-surface`) describes the new rule.
+
+## [0.18.0] - 2026-10-10
+
+### Minor
+- **OQX 0.17 — join syntax sugar** (`spec/oqx` 0.16 → 0.17; `cases/sugar.json`, 134 new cases, 1131 in all; GRAMMAR §2/§4, SEMANTICS §5b/§14, AST.md). Everything that is pure sugar desugars **in the parser** to existing nodes, so the optimizer, the pushdown planners and `print` see the explicit form; two shapes are new.
+
+  - **A receiver block without a consumer is `collect`**: `x { … }` ≡ `x collect { … }` in select position, where position and as a whole query (`docs { age > 15 }`). Carve-out: after the last `follow` destination a brace is still the options block (`follow children { depth 2 }`); a destination block keeps its consumer.
+  - **Where-first bodies**: a block whose leading expression is *syntactically* a predicate (a comparison, `in`, `&&`/`||`, a prefix `!`/`is`/`not`, an infix `is`, a call, a literal, a binding, a parenthesized expression, a postfix `!` or bracket chain, a consumer test — anything that is not a bare name, a dotted navigation or a lift) drops `where`: `people exists { age > 50 }`, `jobs collect { pay > 2 }`. Bare names stay projections (`docs { is active }` is the bare-field filter); `{ name where age >= 18 }` keeps the keyword; `{ age >= 18, name }` is an error suggesting that form. The top level is unchanged: `age > 15 from people` is still the parse error (where would precede from).
+  - **Brackets**: `x[p]` → `x first { where p }`; `x[p]!` → `x single { where p }` under a `required` (zero matches is the required error, many the `single` error); `x[n]` / `x[${i}]` → `x first { offset n }` (out of range is absent; a non-integer binding is the `offset` eval error); `x[n]!` → required positional. Chains compose (`refs(company)[0].name`, `jobs[pay > 2][0]`); a negative or fractional literal index is a parse error; `[`/`]` are tokens.
+  - **Postfix `!` = required** — new node `{ kind: "required", expr }` / `Expr::Required`: the value, or the eval error `` `<expr>!` is absent `` (`on <id>` when the row has a scalar identity). Never a filter or a coercion (`0!`, `""!` are values); tightest precedence (`refs(c)[0]!.name` vs `refs(c)[0].name!`); a `!` right after a complete operand is postfix, a `!` at operand start is the negation (`!=` stays one token: write `a! == b` with the space).
+  - **`is` / `not` / `and` / `or`** become keywords (reserved: `from where select is not and or` + the literal words). Prefix `is x` ≡ `!!x`, `not x` ≡ `!x`. Infix `x is y` / `x is not y` — new `binary` ops `"is"` / `"is not"` (`BinaryOp::Is` / `IsNot`) — compare **identity** per SEMANTICS §16 (a row's `id` when present, else structural), so `$it is ^$it` compares rows and for scalars `is` ≡ `==`; comparison precedence, non-chainable. `and` / `or` are exact synonyms of `&&` / `||` (same nodes, precedence, short-circuit and value semantics; `print` still writes the symbols). `in` over rows is unchanged (deferred).
+  - **Alias reuse in `select`**: an item may reference the items to its left (`boss: ^people[id == ^manager], bossName: boss.name`), inlined by `resolveAliases` / `resolve_aliases` like a `where` alias; a reference to an item to the right is a parse error (a cycle when it refers back). **Behavior change:** `a: b, b: a` is now the cycle error (it was a field swap), and `senior: age > 50, s: senior` projects the alias (it was the row field `senior`).
+  - **A directive may stand in value position** (`jobs first { }.pay`): this is how `print` writes a bracket chain back; `op` joins the `Expr` family by position (`isExpr` stays false for it; `transform` rebuilds it without handing it to the mapping). AST.md §8 records the additions (minor; nothing renamed or removed).
+  - Existing fixtures that turned from error into meaning: `people exists { age > 50 }`-style blocks (`where` ×4, `errors-parse` consumer-after-projection re-queried as `{ name, jobs exists { } }`, `projection` call-in-block re-queried as `{ name, has(budget) }`), the `[` lex-error case (now a token).
+  - **omgbase binding**: the pushdown planners decline `is` / `is not` (identity is the engine's structural notion, not SQL's `IS`), `required` and value-position directives (they may raise), so they stay residual; `query_syntax` gains a "Sugar" block; `docs/query-language.md` §3.6. The surface fixture `implicit-where-in-block` re-queried as `{ kind == "md:task", name }` (still the loud error pointing at `where`).
+  - `@omgbase/oqx-syntax`: `[`/`]` are bracket punctuation (no longer lex errors), `is`/`not`/`and`/`or` are keywords highlighted as word operators, `LANGUAGE_VERSION` 0.17; assets regenerated.
+- **OQX 0.18 — the root scope is a row, absolute scope references, canonical `and`/`or`** (`spec/oqx` 0.17 → 0.18; `cases/absolute-refs.json` new (24), `outer-refs` +6 (the `$it is absent at the root scope` case re-queried as the root-object cases), `limit-offset` +2: 1163 cases in all; GRAMMAR §1/§3/§4/§5/§6, SEMANTICS §2/§8/§18/§19, AST.md §1/§3/§6/§8, README).
+
+  - **The root scope is a row: the host's root object.** SEMANTICS §2 step 1 no longer reads "absent at the root": `$it` at the root scope is the host's root object, so from a top-level row `^$it` is that object, `entries(^$it)` enumerates the named roots in insertion order, `^$it.people` ≡ `^people` (and works as a receiver), `^^$it` from depth two is the same object, and `$it values from $it` is the object as one row. The seam gained **`DataContext.rootObject?(): unknown`** (TypeScript, optional — absent keeps the root row absent) / **`fn root_object(&self) -> Value`** (Rust, provided method defaulting to `Value::Undefined`); `DefaultContext` returns the roots record it was constructed with. Step 5 at the root still reads `ctx.root(name)` — a bare name is never a property of the root object — so hosts with lazy roots are unaffected, and a host-provided root-level name (omgbase will expose the repository id as `$id`, cashing in `$repo.$id` → `^$id` from depth one, `^^$id` from depth two) is readable through `root(name)`. A top-level `limit`/`offset` is evaluated in the root scope *without* its row (SEMANTICS §18): `^$it` reached through a bracket inside the bound is absent, pinned.
+  - **`N^name` — absolute scope references.** An unsigned integer literal immediately followed by `^` (no whitespace) names a scope by its syntactic depth (root 0, top-level rows 1, +1 per block body, a destination block's rows one below the frontier row, a follow `where` one deeper than the frontier row — the `visit` depth): `0^docs`, `0^$it`, `1^$path`, `0^$id`, and as a lift target `1^tasks: text`. It **desugars in the parser** to the existing `outer` node (or lift) with `levels = currentDepth − N`; no new AST shape, `print` writes the carets. `N ≥ currentDepth` is the parse error `scope N does not enclose this block (the current scope is depth D)`; `1 ^x` (a space) and `1.0^x` stay the stray-`^` error. Composes with brackets, calls and receivers (`0^docs[0]`, `x[p == 0^flag]`, `entries(0^$it)`, `0^$it.docs collect { … }`, `follow 0^people collect { where manager == 1^id }`, `follow next { where prev == 1^id && 0^k == 2 }`, `limit 1^n`). Both parsers now track the scope depth of every position.
+  - **Canonical `print` writes `and` / `or`** for the `where` tree's `and`/`or` nodes and for `logical` expressions (both engines); `!` stays `!`, `is`/`is not` were already words, the JSON `logical.op` stays `"&&"`/`"||"`. Consequence: every query text omgbase's `graph` macro prints flips from `||` to `or` — the pinned `graph` query strings in `spec/surface` (`reads.json`, and the `omgbase-surface` unit test for the multi-root seed) are regenerated; nothing else in the surface fixtures changes.
+  - **omgbase binding** (`@omgbase/core` `oqx-js/context.ts`, `omgbase-surface` `context.rs`): the root object is the repository root (`$repo`), so `^$it` from a top-level row is `$repo` and `^$it.docs` the docs scan. The `^docs`/`$repo` redesign and the `$id` root name are a separate change.
+  - `@omgbase/oqx-syntax`: the `N^` lexeme highlights as one outer-reference token (`lift`), `LANGUAGE_VERSION` 0.18; assets regenerated.
+
+## [0.16.0] - 2026-10-09
+
+### Minor
+- **OQX 0.16 — the AST is a first-class contract** (`spec/oqx/AST.md`, fixtures `cases/ast.json` + `cases/ast-spans.json`, 70 new cases; 959 in all). Both engines now produce one normalized tree a tool can reflect on without re-parsing:
+
+  - **Spans on every node**: `span: [start, end)` in Unicode code points over the raw source (a template's `rawSource`, where a binding occupies its `${n}` marker). The TypeScript lexer counted UTF-16 units; it now counts code points like the Rust one, so the `(at offset N)` of a lex/parse error is the same number in both engines. `toUtf16(span, source)` converts for editors.
+  - **Normalized shape**: `kind` on every node (`query`, `subquery`, `follow`, `order` added), optionals materialized (`where`/`limit`/`offset`/`countCmp: null`, `distinct`/`values: false`), operators and consumers serialized as their source words, the never-produced `Expr.index` variant removed. `toJSON(query)` / `ast_to_json(&query)` emit `{ "oqx": "0.16", "kind": "query", … }`; the Rust crate derives serde under the `json` feature (`query_from_json` reads it back).
+  - **`where` keeps its surface form**: the parser no longer inlines `select` aliases into `where` (it only validates them — every `errors-parse` fixture is unchanged); the substitution is the exported pure `resolveAliases` / `resolve_aliases`, applied exactly once by the run entry points (`oqx`, `run`, `execute`, `runQuery`; `run_query`, `execute`). **An `Engine.run` now evaluates the query it is given** — code that drives `InMemoryEngine`/`PlannedEngine` directly with a query whose `where` uses aliases must resolve first (resolution is not idempotent: `name: name.upper() … where name`).
+  - **Traversal**: `visit(root, { enter, leave })` with `{ path, clause, depth }` (scope depth, 0 = root) and `transform(root, f)`, both driven by one child-key table (`CHILDREN`; Rust `walk::visit`/`transform` with a `Visitor` trait, `Node`, `Clause`); `stripSpans` / `strip_spans`; builders `build.*` / `oqx::build::*` for hand-made nodes (empty span `[0, 0]`).
+  - **Canonical printer**: `print(node)` / `print_query` — single spaces, `select` written at the top level, every item aliased, double-quoted strings, minimal parentheses from the precedence table; `printTemplate` → `{ strings, count, indices }` for a tree with bindings (`print` throws, new error stage `print`). Law, enforced by both spec runners over every fixture query: `strip(parse(print(parse(q)))) ≡ strip(parse(q))`.
+  - **Migration** for anyone constructing AST literals (after 0.14's `receiver` → `destinations`): add `kind` to `Query`/`Subquery`/`Follow`/`OrderSpec`, `span` to every node, write the materialized fields (`limit: null`, `countCmp: null`, `distinct: false`, `values: false`), drop `index`; or use `build.*`. Rust: struct variants gained `span` (match with `..`), `Expr::Lit(v)` is `Expr::Lit { value, span }`, `Token` gained `end`.
+  - **omgbase** (`@omgbase/core`, `omgbase-surface`): the runner's `$self` row-function rewrite is a `transform`, the `semantic("…")` scan a `visit`, aliases are resolved once before the runner's own rewrites; the `graph` macro builds its `follow` walk with the builders and `print`s it instead of splicing strings — the generated query text and every `spec/surface` result are unchanged. `@omgbase/oqx-syntax`'s `LANGUAGE_VERSION` moves to `0.16` (assets regenerated).
+
 ## [0.15.0] - 2026-10-09
 
 ### Minor