@jarenjs/json 0.9.2 → 0.34.0

package/docs/QUERY-FORMAT.md

@@ -78,6 +78,13 @@ Two conformance roles exist:
 5. A sequence of exactly one item is identified with that item ("singleton ≡
    item"). A literal `42` and a one-item sequence containing `42` are
    indistinguishable.
+6. The **input document** a query is applied to MUST be an item — that is, a
+   JSON value. An implementation is NOT required to verify this (a deep check
+   would cost a full walk per call), so a non-JSON value passed in may simply
+   flow through as an opaque item. It MUST, however, reject a *missing*
+   document (JavaScript `undefined`) with `JQ2011`: a host API that spends
+   the absent value on the empty sequence would otherwise answer "empty" for
+   a query whose existence test says otherwise.
 
 ### 2.2 Effective boolean value (EBV)
 
@@ -190,11 +197,15 @@ evaluating to the empty sequence contributes none.
 Against a store with four book prices, this constructs a single 7-element
 array: `1`, the four prices, `2`, `3`.
 
-The same flattening applies to the member value expressions of a map
-constructor and to every other expression position; it is a property of the
-sequence data model, not of the array constructor alone. To embed an array
-*as a value* without evaluation, use `$const` (§3.5.1); to bind one without
-iteration, use `$let` (§6.3).
+This flattening is a property of the sequence data model, not of the array
+constructor alone; it applies in every expression position where a sequence can
+flow. A **map-constructor member value** is the one position that cannot flatten
+— a JSON member holds exactly one value — so it takes the single-value
+cardinality rule instead: a member value that evaluates to the empty sequence
+**omits the member**, a singleton becomes the member's value, and a value of two
+or more items is runtime error `JQ2001`. To embed an array *as a value* without
+evaluation, use `$const` (§3.5.1); to bind one without iteration, use `$let`
+(§6.3).
 
 ### 3.5 Escape hatches
 
@@ -226,9 +237,12 @@ order:
 
 - *keyExpr* MUST evaluate to a single string; any other result (empty
   sequence, non-string, multi-item sequence) is runtime error `JQ2004`.
-- *valueExpr* is evaluated as an ordinary expression (its result flows per
-  §3.4).
-- Later pairs win on duplicate keys.
+- *valueExpr* is evaluated and taken as the member value under §3.4's
+  member-value cardinality: an empty result **omits the pair**, a singleton is
+  the value, and two or more items are runtime error `JQ2001`.
+- Later pairs win on duplicate keys — but because an omitted (empty-valued) pair
+  is never written, an earlier pair for the same key **survives** a later pair
+  for that key whose value is the empty sequence.
 
 Each pair MUST be an array of exactly two expressions (`JQ0003` otherwise).
 An empty pair list constructs the empty object.
@@ -296,6 +310,17 @@ this, and later work uses it for static cardinality checks.
 An absolute path in any expression position reads from the one input
 document; there is no context-item drift — `$` is always the document root.
 
+> **Footgun (variable-rooted filters).** A filter selector filters the
+> *children* of each node it is applied to (RFC 9535 §2.3.5). So a filter
+> written directly against a `$let`/`$for` variable tests that item's members,
+> not the item itself. Given `{"$let": {"books": "$.store.book[*]"}}`, the path
+> `"$books[?@.price > 20]"` applies the filter to the *members* of each book
+> object and asks each member for a `.price` child — no member has one, so every
+> book contributes the empty sequence and the whole path is empty. To filter the
+> books themselves, place the predicate one level up, where the books are the
+> children being filtered — `"$.store.book[?@.price > 20]"` — or iterate with
+> `$for` and test in `$where` (§6.4).
+
 ### 5.2 The two filter dialects
 
 Two boolean mini-languages coexist in this format, and both keep the exact
@@ -340,13 +365,14 @@ choose the dialect deliberately when absent members are possible.
 
 ### 6.1 Shape and clause order
 
-A FLWOR phrase is an operator phrase containing `$for` and/or `$let`, plus
-optional clauses, plus the REQUIRED `$return`:
+A FLWOR phrase is an operator phrase containing `$fold`, `$for` and/or
+`$let`, plus optional clauses, plus the REQUIRED `$return`:
 
 | Key | Value | Presence |
 |---|---|---|
-| `$for` | binding object (§6.2) | at least one of `$for`, `$let` |
-| `$let` | binding object (§6.3) | at least one of `$for`, `$let` |
+| `$fold` | one-member accumulator binding (§6.9) | at least one of `$fold`, `$for`, `$let` |
+| `$for` | binding object (§6.2) | at least one of `$fold`, `$for`, `$let` |
+| `$let` | binding object (§6.3) | at least one of `$fold`, `$for`, `$let` |
 | `$as` | schema assertion object (§6.8) | OPTIONAL |
 | `$where` | expression | OPTIONAL |
 | `$groupby` | binding object (§6.5) | OPTIONAL |
@@ -358,7 +384,7 @@ The clauses apply in **fixed semantic order regardless of their order in the
 JSON document** (**D7**):
 
 ```
-$for → $let → $as → $where → $groupby → $orderby → $count → $return
+$fold → $for → $let → $as → $where → $groupby → $orderby → $count → $return
 ```
 
 JSON key order is not interoperable — several ecosystems (e.g. Go maps)
@@ -377,7 +403,9 @@ which is standard XQuery practice anyway:
 ```
 
 The FLWOR phrase evaluates to the concatenation of the `$return` results over
-the surviving tuple stream, in tuple order.
+the surviving tuple stream, in tuple order — unless the phrase has a `$fold`
+clause (§6.9), in which case it evaluates to the final accumulator and
+`$return` names the accumulator's next value instead of an output item.
 
 ### 6.2 `$for` — iteration bindings
 
@@ -395,8 +423,8 @@ array, `$for` unpacks it into its members — one level only — and iterates
 those. This is the ergonomic default for the JSON data model, where `[*]`
 already produced values, not nodes. To bind an array *as a value*, use `$let`
 or `{"$const": [...]}`. Objects are NOT auto-iterated; iterate an object's
-values with a `[*]` path segment (a future `$entries` operator is the planned
-counterpart for member pairs).
+values with a `[*]` path segment, or its member pairs with the `$entries`
+operator (§8.9).
 
 **Multiple bindings** nest left-to-right in document key order, exactly like
 consecutive XQuery `for` clauses, and **may be correlated**: a later source
@@ -407,16 +435,18 @@ may reference variables bound earlier in the same `$for` object.
 > key-order-hostile stacks SHOULD emit one binding per phrase, nesting
 > phrases, instead of relying on multi-key binding objects.
 
-**Extended binding form** — a source written as
+**Extended binding form** — a source written as an object with an `$in`
+member iterates `$in` like a plain source, with options:
 
 ```json
 { "$in": expr, "$at": "posName" }
 ```
 
-iterates `expr` like a plain source and additionally binds *posName* to the
-**0-based** (D6) position of the current item within the iterated sequence.
-Both keys are REQUIRED in this form (a positionless binding is simply the
-plain form); *posName* MUST be a valid variable name.
+binds *posName* to the **0-based** (D6) position of the current item within
+the iterated sequence. `$in` is the only REQUIRED key (so a bare
+`{"$in": expr}` is just the long spelling of the plain form); *posName* MUST
+be a valid variable name. The two remaining options — `$allowing-empty` and
+the `$window` family — are §6.10.
 
 ```json
 { "$for": { "b": { "$in": "$.store.book[*]", "$at": "i" } },
@@ -462,10 +492,14 @@ XQuery 3.1 group-by semantics:
 
 - Each *keyExpr* is evaluated per tuple; each *name* becomes a
   **grouping-key variable**, bound in every subsequent clause to that group's
-  key value (a singleton per group).
+  key value (a singleton per group). A *keyExpr* MUST evaluate to the empty
+  sequence or a single item; a two-or-more-item key is runtime error `JQ2001`.
+  An **empty key is allowed** and groups with the other empty keys.
 - Tuples with equal key combinations form one group. Key equality is **deep
-  structural JSON equality** (`equalsJson`-grade, the same relation as `$eq`
-  item equality, §8.4), with numbers compared mathematically.
+  structural JSON equality** (`equalsJson`-grade), with numbers compared
+  mathematically. Unlike `$eq` item equality (§8.4), grouping treats **`NaN` as
+  equal to itself** (the XQuery grouping rule; §8.9's `$distinct` and `$sort`
+  cite the same relation).
 - Every other variable bound in the phrase is **rebound to the sequence** of
   its values across the group's tuples, in tuple order.
 - The tuple stream after `$groupby` has one tuple per group, in order of
@@ -497,11 +531,21 @@ A *keySpec* is either an expression (shorthand for ascending, empty-least) or
 the explicit form
 
 ```json
-{ "$key": expr, "$dir": "asc" | "desc", "$empty": "least" | "greatest" }
+{ "$key": expr, "$dir": "asc" | "desc", "$empty": "least" | "greatest",
+  "$collation": "name" }
 ```
 
-with `$key` REQUIRED and `$dir` (default `"asc"`) and `$empty` (default
-`"least"`) OPTIONAL.
+with `$key` REQUIRED and `$dir` (default `"asc"`), `$empty` (default
+`"least"`) and `$collation` OPTIONAL.
+
+`$collation` names a **registered pure compare function**
+(`options.collations`, a trusted host capability like `options.functions`,
+§8.12) applied to this key's *string* comparisons — the natural-language
+orders (`Intl.Collator('nl').compare`, say) that the format's default
+code-point order (§8.9) deliberately does not build in. Number keys keep
+numeric order; an unregistered name is compile error `JQ0010`. Registered
+collations appear in `query.dependencies`/`query.explain()` (§8.12), so a
+saved rule declares the collations it needs.
 
 - An array value of `$orderby` is **always a list of key specs**, ordered
   major to minor, and MUST NOT be empty. Consequently a *single* key spec is
@@ -515,7 +559,9 @@ with `$key` REQUIRED and `$dir` (default `"asc"`) and `$empty` (default
   mathematically, strings by **Unicode scalar values** (code point order).
   Comparing any other combination (number with string, or a key that is a
   boolean, `null`, array, object, or multi-item sequence) is runtime error
-  `JQ2005`.
+  `JQ2005`. A `NaN` key is a number: the comparator is total over `NaN`, ordering
+  it **equal to itself and less than every other number**, then falling through
+  to the next key on ties (§8.9's `$sort` cites this same rule).
 - Empty key sequences sort per `$empty`: `"least"` (default) places them
   first ascending / last descending; `"greatest"` the reverse.
 
@@ -577,6 +623,124 @@ instead.
   "$return": "$b.title" }
 ```
 
+### 6.9 `$fold` — the accumulator clause
+
+```json
+"$fold": { name: initExpr }
+```
+
+`$fold` turns the phrase from a *map* into a **reduction**. It binds exactly
+one accumulator variable (a second member is `JQ0003`):
+
+- *initExpr* is evaluated **once**, in the phrase's **enclosing** scope,
+  before the tuple stream starts. It therefore cannot reference this phrase's
+  own `$for`/`$let` bindings — such a name is free, and resolves as an
+  external (§9) rather than as the binding.
+- *name* is in scope from `$let` onward — in `$let`, `$where`, `$groupby`,
+  `$orderby` and `$return` — but **not** in `$for` sources, which are
+  iterated once and must not depend on a value that changes per tuple.
+- For each surviving tuple, `$return` is evaluated and the accumulator is
+  **rebound to its result**. `$return` names the accumulator's next value,
+  not an item of the output.
+- The phrase evaluates to the **final accumulator**. If no tuple survives,
+  that is the initial value.
+
+*name* counts as a binding of this phrase for the duplicate rule (`JQ0007`).
+Because the accumulator is a binding rather than a lambda parameter, this
+gives the language a general fold without giving the JSON encoding a way to
+spell a **function value** — folds, running totals and pointer walks are
+expressible; passing a function to an operator still is not.
+
+`$fold` composes with every other clause. With `$orderby` the reduction runs
+over the *sorted* tuples; with `$groupby` it updates once per group; `$where`
+selects which tuples update it at all.
+
+```json
+{ "$fold": { "total": 0 },
+  "$for": { "b": "$.store.book[*]" },
+  "$where": { "$lt": ["$b.price", 10] },
+  "$return": { "$add": ["$total", "$b.price"] } }
+```
+
+sums the prices of the cheap books. Because `$get` (§8.9) is a real dynamic
+lookup, a fold over a runtime path is a pointer walk:
+
+```json
+{ "$fold": { "cur": "$.doc" },
+  "$for": { "seg": "$.path[*]" },
+  "$return": { "$get": ["$cur", "$seg"] } }
+```
+
+resolves `$.path` — a sequence of member names and array indexes — against
+`$.doc`, one segment per tuple.
+
+### 6.10 `$allowing-empty` and window clauses
+
+Two further options of the extended `$for` binding form (§6.2).
+
+**`$allowing-empty`** makes a binding **outer-join-style**: when the source
+would yield no tuple at all, the clause emits exactly **one** tuple with the
+variable bound to the **empty sequence**, so the enclosing tuple survives.
+
+```json
+{ "$in": expr, "$allowing-empty": true }
+```
+
+The condition is "yields no tuple", not "the sequence was empty" — an item
+that is an empty array contributes no members under D4 unpacking (§6.2) and
+so also triggers it. When the binding also has `$at`, that tuple's position
+is **-1**: every real position is a non-negative 0-based one (D6), so -1 is
+the only value free to mean "no position".
+
+Note the source is what must be empty. Because a path filter's `$` is the
+input document and never a query variable (§3.2), a *correlated* source is
+written as a nested phrase:
+
+```json
+{ "$for": { "b": "$.store.book[*]",
+            "r": { "$in": { "$for": { "x": "$.ratings[*]" },
+                            "$where": { "$eq": ["$x.isbn", "$b.isbn"] },
+                            "$return": "$x" },
+                   "$allowing-empty": true } },
+  "$return": { "title": "$b.title", "stars": "$r.stars" } }
+```
+
+keeps every book, rated or not; an unrated book's `stars` member is omitted
+because an empty member value omits the member (§3.4).
+
+**Windows** iterate consecutive *runs* of the item stream instead of single
+items:
+
+```json
+{ "$in": expr, "$window": "tumbling" | "sliding", "$size": n, "$step": m, "$at": "w" }
+```
+
+`$size` is REQUIRED with `$window` and MUST be a positive **integer literal**,
+as MUST `$step` when present; they are not expressions, because a width that
+varied per tuple could not be compiled into a specialized loop. `$size`/`$step`
+without `$window`, or `$window` without `$size`, is `JQ0003`. The variable
+binds the window's items **as a sequence** (not an array item); `$at` binds
+the 0-based window number.
+
+A window starts at item 0 and every `$step` items thereafter; `$step`
+defaults to `$size` for `tumbling` and to `1` for `sliding`. The two kinds
+differ only in what happens at the end of the stream:
+
+- **`tumbling`** *partitions* the stream — every item belongs to exactly one
+  window — so a short final window IS emitted; dropping it would silently
+  lose data.
+- **`sliding`** is a moving view of fixed width, so a short window is not one
+  of them: only full-width windows are emitted.
+
+```json
+{ "$for": { "w": { "$in": "$.readings[*]", "$window": "sliding", "$size": 3 } },
+  "$return": { "$avg": "$w" } }
+```
+
+is a 3-point moving average; the same document with `"tumbling"` and no
+`$step` averages disjoint blocks of three, including a final block of one or
+two if the stream does not divide evenly.
+
 ---
 
 ## 7. Quantifier phrases
@@ -715,7 +879,10 @@ All arithmetic is IEEE double arithmetic (**D1**):
   XQuery *double* division semantics, not the decimal FOAR0001 error.
 - `$idiv` is **truncating division** (quotient rounded toward zero to an
   integral double); `$idiv` or `$mod` with a zero divisor is runtime error
-  `JQ2002`.
+  `JQ2002`. Only a **zero divisor** errors: an `Infinity` or `NaN` *dividend*
+  over a finite non-zero divisor is not an error — both operators keep IEEE
+  double behavior (`Math.trunc` of the quotient, `%` for `$mod`) and yield
+  `NaN`, in keeping with D1's IEEE arithmetic.
 - `$mod` takes the sign of the dividend (XQuery `mod` semantics).
 - `$neg` is unary minus.
 
@@ -805,7 +972,7 @@ clause/operator collision note.
 | `$min` | empty | The least item (`fn:min`). Items MUST be all numbers or all strings (`JQ2001` otherwise, including mixed); numbers compare mathematically, strings by Unicode scalar values. A `NaN` item makes the result `NaN` (F&O). |
 | `$max` | empty | The greatest item (`fn:max`), same rules as `$min`. |
 
-### 8.9 Sequence operators — `$distinct $reverse $sort $head $tail $subsequence $index-of $range $get`
+### 8.9 Sequence operators — `$distinct $reverse $sort $head $tail $subsequence $index-of $range $get $entries $from-entries`
 
 | Operator | Signature | Definition |
 |---|---|---|
@@ -818,6 +985,8 @@ clause/operator collision note.
 | `$index-of` | `[seq, item]` | The 0-based (D6) positions in *seq* of the items deep-equal to *item*, as a sequence, in order (`fn:index-of`). Equality is the `$eq` item relation (D2) — `NaN` matches nothing. *item* MUST be exactly one item (`JQ2001`). |
 | `$range` | `[start, end]` | The integers from *start* to *end* **inclusive** (the XQuery `to` operator). Either operand empty → empty; *start* > *end* → empty. A non-integral or non-number operand is `JQ2001`. A result of more than 2³² items is runtime error `JQ2007` (resource guard). |
 | `$get` | `[target, key]` | Dynamic lookup, the runtime counterpart of a path leaf: an object *target* with a string *key* yields the member value or empty; an array *target* with an integer *key* yields the element at that 0-based (D6) index — a negative index counts from the end, like the RFC 9535 index selector — or empty. **Every other combination** (wrong type pairing, non-integral index, empty or multi-item operands) is simply the empty sequence, never an error. |
+| `$entries` | `{"$entries": e}` | The member-pair counterpart of a `[*]` path segment (which yields values only): each OBJECT item of the operand contributes one `{"key": name, "value": v}` pair per member, in member order; non-object items contribute nothing, and an empty operand yields the empty sequence. `$from-entries` is the inverse; the `$map` constructor (§3.5.2) is the fixed-arity form for computed keys. |
+| `$from-entries` | `{"$from-entries": e}` | The inverse of `$entries`: assembles ONE object from the operand's `{"key": name, "value": v}` items, in sequence order — later pairs win on duplicate keys, exactly like the `$map` constructor. Items without a string `key` contribute nothing; a pair missing its `value` member reads as `null`; an empty operand constructs the empty object. |
 
 Combined example (also exercises the aggregates of §8.8):
 
@@ -927,6 +1096,265 @@ README's "`$query` — cross-field assertions" section.
 
 ---
 
+### 8.12 Registered functions, operators, collations, and execution limits
+
+Three compile options make a compilation's **trusted host capabilities**
+explicit — none of them changes the closed format: a document using them
+compiles only against a host that registered them, and a host that
+registered nothing keeps exactly the spec vocabulary.
+
+**`options.functions` and `$call`.** A registry of named pure functions;
+the `$call` phrase invokes one:
+
+```json
+{ "$call": ["upper", "$b.title"] }
+```
+
+The first item MUST be a literal string naming a registered function
+(`JQ0010` otherwise). Argument expressions evaluate first; each crosses
+the boundary as plain JSON — a sequence as an array of items, the empty
+sequence as `undefined`. The function's return value is one item;
+`undefined` is the empty sequence. A throwing function is runtime error
+`JQ2010`. Functions MUST be pure over JSON: they are part of the query's
+semantics, not an effect hatch.
+
+**`options.extensions` and host `$`-operators.** A registry of named
+operator entries that extend the closed operator vocabulary (§8) with
+host-provided `$`-operators. Each entry has the same shape the built-in
+operators use — `{ params, result, resultType, compile, normalize? }` —
+so a registered `{ "$sqrt": "$b.x" }` normalizes, type-annotates (its
+declared `resultType` flows through `annotateTypes`, Appendix C.8) and
+compiles exactly like a built-in, and appears in
+`query.dependencies.operators`. A name that collides with the closed
+vocabulary is a `TypeError` at compile — a host programming error, never
+a `JQ` document error. The intended way to build this registry is
+`createJsltRegistry()` from `@jarenjs/json/jslt` (JSLT-FORMAT §13), which
+composes plain-data packs of pure `@jarenjs/core` functions (math,
+finance, statistics) into `{ extensions, functions }` and works across
+stylesheets, bare queries and `@jarenjs/linq` — and, through
+`@jarenjs/db`, in the query residual and (for the `pushable:'scalar'`
+subset) as SQLite deterministic UDFs (MODEL-FORMAT §8.1–8.2). A document
+compiled *without* a registry keeps exactly the spec vocabulary — `$sqrt`
+is then `JQ0002`.
+
+**`options.collations`.** A registry of named pure compare functions for
+`$orderby`'s `$collation` member (§6.6).
+
+**`options.limits`.** Deterministic limits enforced *inside* the
+synchronous engine. Two of them are **output caps**: they bound what a
+phrase or the query hands onward, NOT memory, fan-out or intermediate
+accumulation.
+
+- `sequenceItems` — bounds every FLWOR phrase materialization (the
+  sequence a phrase *returns*) and tightens `$range`'s resource guard
+  below its 2³² ceiling; exceeding it is `JQ2009` (`$range` keeps its
+  historical `JQ2007`). A `$groupby`/`$orderby` barrier may accumulate
+  arbitrarily many items — and a collation may run arbitrarily many
+  comparisons — behind a small final output; bare paths, `$count` and
+  other operators can likewise materialize above the cap internally.
+- `resultItems` — bounds the final result at the query boundary
+  (`JQ2009`), checked after evaluation; `first()`, `exists()` and
+  `ebv()` deliberately bypass it.
+
+The other two bound the **query** rather than its output:
+
+- `steps` — bounds **expression-node evaluations** (`JQ2009`). A step is
+  one node evaluation, not one primitive operation: a node that loops
+  internally — materializing a `$range`, a general comparison's cross
+  product, a sort's comparisons — counts once. It is the only limit that
+  bounds work rather than output, and the only one that costs: setting
+  it compiles a counter check into every node. The counter resets per
+  evaluation, so a compiled query stays reusable.
+- `depth` — bounds **expression nesting**, and is checked at **compile
+  time** (`JQ0011`). The language has no recursion — no user-defined
+  functions, no self-reference — so the compiled closure tree's maximum
+  evaluation depth IS the document's static nesting. Checking it once is
+  therefore exact, and costs nothing to evaluate.
+
+An unknown limit name, or a value that is not a positive integer, is a
+host programming error (`TypeError`), because an accepted-but-unenforced
+limit would be a silent false guarantee.
+
+Together these make trusted, developer-authored rules diagnosable, and
+give user- or model-authored queries a deterministic work bound. They
+still are not a sandbox: `steps` bounds evaluations, not memory, and a
+wall-clock or CPU limit is out of scope by design — a synchronous run on
+the caller's thread cannot be preempted, so a host needing hard
+termination owns a worker or isolate.
+
+**Dependencies and explanation.** The compiled query reports what it
+needs: `query.dependencies` is a frozen `{ externals, operators,
+functions, collations }`, and `query.explain()` returns that plus the
+enforced limits as fresh plain JSON — the vetting surface for saved or
+machine-authored rules.
+
+### 8.13 Dates and times
+
+JSON has no date type, so dates are **RFC 3339 strings** and these operators
+are ordinary string operators with a calendar's worth of rules. Every one of
+them is a **pure function of its operand**: there is deliberately no
+`current-dateTime`, because a compiled query must give the same answer for
+the same input document forever — it is cached by document identity, saved as
+a rule, and usable as a validation keyword.
+
+**Type predicates.** Unary `{"$is-date": e}`, `$is-time`, `$is-datetime`,
+`$is-duration` — `true` iff `e` is a **singleton string** in that lexical
+form (`full-date`, `full-time`, `date-time`, and the RFC 3339 Appendix A
+duration grammar). Like the §8.10 `$is-*` family these never raise: the empty
+sequence, a multi-item sequence and a non-string are all `false`. The forms
+are disjoint — a `date-time` is not a `date`.
+
+**Components.** Unary `$year`, `$month`, `$day`, `$hours`, `$minutes`,
+`$seconds`, `$offset`. Each propagates the empty sequence and returns a
+number. Components are read **lexically, in the value's own offset** — the
+`fn:year-from-dateTime` reading, and the one that makes "group by month"
+mean what an author expects. `$seconds` carries the fraction (`5.5`).
+`$offset` is minutes east of UTC, and is the one component that yields the
+**empty sequence** rather than an error when absent, because RFC 3339 leaves
+a bare `full-date` offset-less.
+
+Asking a value for a component of a half it does not have — the `$hours` of a
+`full-date`, the `$year` of a `full-time` — is runtime error `JQ2001`, as is
+an operand that is not an RFC 3339 value at all.
+
+**Instants.** Unary `$epoch` maps a value carrying a date to **milliseconds
+since 1970-01-01T00:00:00Z**, and `$datetime` maps such a number back to a
+canonical UTC `date-time` string. `$epoch` is the one place a value is
+shifted to UTC, which makes it the way to compare or subtract across
+offsets — as *strings*, `"…T14:00:00+02:00"` sorts after `"…T12:00:00Z"`
+though they are the same instant. A `full-time` has no instant to place
+(`JQ2001`); so does a number outside the range RFC 3339 can spell.
+
+**Calendar arithmetic.** `$date-add` and `$date-sub` shift a value, either
+by an ISO 8601 duration (`[date, "P1M"]`) or by an amount and a unit
+(`[date, 3, "day"]`). `$start-of` and `$end-of` truncate to a unit, and
+`$date-diff` counts whole units from one value to another. The unit is
+**data**, not vocabulary — one of `year`, `quarter`, `month`, `week`, `day`,
+`hour`, `minute`, `second`, `millisecond` — so an unknown one is `JQ2001`
+rather than a compile error.
+
+Two rules make these predictable:
+
+- **The lexical form is preserved.** A `full-date` shifted by a day is still
+  a `full-date`, and a `date-time` keeps its own offset rather than being
+  normalized to UTC. A query that buckets dates must not silently start
+  producing date-times. Consequently `$end-of` on a `full-date` yields that
+  unit's last *day*, where on a `date-time` it yields the last millisecond —
+  a `full-date` has nowhere to put one.
+- **Month arithmetic clamps.** `2026-01-31` plus one month is `2026-02-28`,
+  because the alternative — overflowing into March — makes adding a month
+  non-monotonic. `$date-diff` counts months to match, so adding its result
+  back never overshoots: `2026-01-31` to `2026-02-28` is **one** month.
+
+`$date-format` renders a value through a **Unicode LDML** pattern
+(`yyyy-MM-dd`, not moment's `YYYY-MM-DD`); a literal pattern compiles once
+with the query. Patterns are limited to the locale-independent tokens: month
+and weekday *names* would need locale data this format does not carry, so
+`MMMM`, `MMM`, `EEEE`, `EEE` and `a` are rejected — `JQ0003` for a literal
+pattern, `JQ2001` for one computed at runtime. Localized rendering belongs to
+the presentation layer, not to a query.
+
+| Operator | Definition |
+|---|---|
+| `$is-date` `$is-time` `$is-datetime` `$is-duration` | singleton string in that RFC 3339 form → `true`; anything else → `false` |
+| `$year` `$month` `$day` | lexical date components; a value with no date is `JQ2001` |
+| `$hours` `$minutes` `$seconds` | lexical time components, `$seconds` including its fraction; a value with no time is `JQ2001` |
+| `$offset` | minutes east of UTC; a bare `full-date` → empty |
+| `$week` `$week-year` | ISO 8601 week number and its week-numbering year, which is not always the calendar year (2027-01-01 is week 53 of 2026) |
+| `$quarter` `$weekday` | calendar quarter 1-4; ISO weekday 1 (Monday) to 7 (Sunday) |
+| `$epoch` | date or date-time → milliseconds since the epoch (UTC); a `full-time` → `JQ2001` |
+| `$datetime` | epoch milliseconds → canonical UTC `date-time`; out of RFC 3339 range → `JQ2001` |
+| `$date-add` `$date-sub` | `[date, duration]` or `[date, amount, unit]` → a value of the same lexical form |
+| `$start-of` `$end-of` | `[date, unit]` → the unit's first / last instant, in the same lexical form |
+| `$date-diff` | `[from, to, unit]` → whole units, negative when `to` precedes `from` |
+| `$date-format` | `[date, pattern]` → the value rendered through an LDML pattern |
+
+Every operator that *produces* a date produces it in the same canonical
+spelling, so one query can never emit two forms of one instant: a fractional
+second appears only when non-zero and without trailing zeros
+(`…:05.5Z`, never `…:05.500Z`).
+
+Durations are recognized and applied, but never *decomposed* into a number:
+`P1M` is not a fixed count of milliseconds, so there is no honest length to
+report without a calendar anchor. Applying one to a date is where the anchor
+exists, which is what `$date-add` is for; fixed-width spans go through
+`$epoch` and ordinary `$sub`.
+
+```json
+{ "$for": { "e": "$.events[*]" },
+  "$groupby": { "w": { "$start-of": ["$e.on", "week"] } },
+  "$orderby": ["$w"],
+  "$return": { "week": "$w", "count": { "$count": "$e" } } }
+```
+
+buckets events into ISO weeks — the shape components alone could not express,
+because a week boundary is arithmetic, not a field.
+
+```json
+{ "$for": { "e": "$.events[*]" },
+  "$where": { "$is-datetime": "$e.at" },
+  "$groupby": { "y": { "$year": "$e.at" }, "m": { "$month": "$e.at" } },
+  "$orderby": ["$y", "$m"],
+  "$return": { "year": "$y", "month": "$m", "count": { "$count": "$e" } } }
+```
+
+### 8.14 Spatial
+
+Geography enters the language the way dates did: through the format the data
+already has. Operands are **GeoJSON** ([RFC 7946](https://datatracker.ietf.org/doc/html/rfc7946))
+— a bare position `[longitude, latitude]`, a geometry, a `Feature`, or a
+`FeatureCollection` — because that is what a JSON document holds. There is no
+geometry type to construct, and a wrapper is unwrapped for you: passing a
+`Feature` where a geometry is wanted is not an error, it is the common case.
+
+Coordinates are longitude then latitude, in WGS 84 decimal degrees. RFC 7946
+removed alternative coordinate reference systems, so there is nothing to
+configure and no `crs` to honour.
+
+**Measurements are geodesic, never planar.** A degree of longitude spans about
+111 km at the equator and 68 km at 52°N, so a Euclidean answer over raw degrees
+is wrong by two thirds over a kilometre at Dutch latitudes. `$distance`,
+`$area` and `$length` answer in metres and square metres on the WGS 84 sphere,
+accurate to under half a percent. Convert with ordinary arithmetic —
+`{"$idiv": [{"$distance": [a, b]}, 1000]}` for kilometres.
+
+`$distance` and `$within` measure a value by its **representative position**: a
+bare position or `Point` is itself, anything else is its centroid. That is
+stated rather than inferred because the alternative — the minimum distance
+between two shapes — needs point-to-segment geodesics and is a much larger
+piece of work this format does not yet do.
+
+`$bbox-intersects` is named for exactly what it tests. An operator called
+`$intersects` that compared only bounding boxes would be a lie the first time
+two L-shaped regions shared a box and nothing else; real geometry-to-geometry
+intersection is overlay work and is deliberately absent.
+
+| Operator | Definition |
+|---|---|
+| `$bbox` | any value → `[west, south, east, north]`; a value with no positions → empty |
+| `$area` | square metres of the value's polygons, exterior rings less holes; anything without a surface → `0` |
+| `$length` | metres of the value's lines and ring perimeters; a point → `0` |
+| `$centroid` | the mean of the value's positions, as a position. **Not** the area-weighted centre of mass: for a concave shape it can fall outside the polygon |
+| `$distance` | `[a, b]` → metres between the two representative positions |
+| `$within` | `[a, b]` → is `a`'s representative position inside `b`'s surface? Only a polygon has an inside, so a line or point as `b` is `false` |
+| `$bbox-intersects` | `[a, b]` → do the two bounding boxes overlap? Touching edges count |
+| `$geohash` | `[value]` or `[value, precision]` → the base-32 cell string; precision is 1-12, default 9 |
+
+Note what needs **no** operator. A geohash is a string, so proximity is
+`$starts-with` on a prefix and spatial bucketing is `$groupby` over
+`$substring` — the existing vocabulary already indexes, groups and orders them.
+
+```json
+{ "$for": { "c": "$.cities[*]" },
+  "$where": { "$within": ["$c.at", "$.region"] },
+  "$orderby": [{ "$key": { "$distance": ["$c.at", "$.centre"] } }],
+  "$return": { "name": "$c.name",
+               "km": { "$idiv": [{ "$distance": ["$c.at", "$.centre"] }, 1000] } } }
+```
+
+selects the cities inside a region and orders them by how far they are from a
+point — a spatial filter and a spatial sort, in the language's own clauses.
+
 ## 9. Variables, scoping, and external parameters
 
 1. Variables are introduced by `$for`, `$let`, `$at`, `$count`, `$groupby`
@@ -952,6 +1380,33 @@ README's "`$query` — cross-field assertions" section.
 
 `$minPrice` is free — an external the caller binds at call time.
 
+**Closed-world compilation.** Because use is the declaration, a typo in a
+variable name is not an error: it quietly becomes a new external. A host that
+knows the parameters it intends to expose MAY compile **closed-world**,
+declaring them (`options.externals` in this implementation). Every free
+variable that is not declared is then compile error `JQ0005` at its own
+reference site, and an empty declaration list forbids externals entirely.
+This changes no document semantics — a document that compiles closed-world
+behaves identically compiled open — it only decides which documents compile.
+
+> **Worked example (clause order and `$count`).** Scope follows the
+> **semantic** clause order of §6.1, not document key order, and `$count`
+> binds *after* `$where`. So a `$where` that mentions the phrase's own
+> `$count` name does not see the tuple number — the name is not in scope
+> yet, and rule 3 makes it an **external**:
+>
+> ```json
+> { "$for": { "b": "$[*]" },
+>   "$where": { "$lt": ["$n", 2] },
+>   "$count": "n",
+>   "$return": ["$b", "$n"] }
+> ```
+>
+> `$n` in `$where` is external (the compiled query reports `["n"]`); `$n` in
+> `$return` is the tuple number. This is correct and surprising, which is why
+> it is worth stating: filtering by position is done with `$at` (§6.2), whose
+> variable is in scope from the binding onward, not with `$count`.
+
 ---
 
 ## 10. Errors
@@ -974,11 +1429,13 @@ runtime errors as `JsonQueryRuntimeError`. Every error carries:
 | `JQ0002` | Unknown operator / `$`-key outside the vocabulary | XPST0017 |
 | `JQ0003` | Known phrase with bad arity, value shape, or key combination | XPST0003 |
 | `JQ0004` | String starting `$` is not a valid path or escape (§3.2) | XPST0003 |
-| `JQ0005` | Variable reference that is neither bound nor collectible as an external (reserved for closed-world compilation modes; see §9); also an `$as` member naming a variable not bound by its phrase's `$for`/`$let` (§6.8) | XPST0008 |
+| `JQ0005` | Variable reference that is neither bound nor a declared external under a closed-world compilation (§9); also an `$as` member naming a variable not bound by its phrase's `$for`/`$let` (§6.8) | XPST0008 |
 | `JQ0006` | Version envelope with unknown or non-string `$query` (§4.2) | XQST0031 |
 | `JQ0007` | Duplicate variable binding within one phrase (§6.3) | XQST0089 |
 | `JQ0008` | Schema operator (`$valid`/`$assert`/`$as`) in a query compiled without a type-test compiler (§8.11) | XQST0009 |
 | `JQ0009` | Schema literal rejected by the type-test compiler (invalid embedded schema, §8.11) | XQST0059 |
+| `JQ0010` | `$call`/`$collation` naming no registered function/collation (§8.12, §6.6) | XPST0017 |
+| `JQ0011` | Expression nesting deeper than `limits.depth` (§8.12) | XPDY0130 |
 
 ### 10.3 Runtime errors (`JQ2xxx`)
 
@@ -992,6 +1449,9 @@ runtime errors as `JsonQueryRuntimeError`. Every error carries:
 | `JQ2006` | Reference to an unbound external parameter (§9) | XPDY0002 |
 | `JQ2007` | Resource guard: an operator result exceeding an implementation limit (`$range` over 2³² items, §8.9) | XPDY0130 |
 | `JQ2008` | Schema assertion failure: an item rejected by `$assert`'s schema, or a bound variable rejected by its `$as` schema (§6.8, §8.11) | XPTY0004 |
+| `JQ2009` | An execution limit exceeded: `limits.sequenceItems` on a phrase materialization, `limits.resultItems` at the query boundary, or `limits.steps` expression evaluations (§8.12) | XPDY0130 |
+| `JQ2010` | A registered `$call` function threw (§8.12) | FOER0000 |
+| `JQ2011` | The input document is `undefined`, which is not a JSON value (§2.1) | XPDY0002 |
 
 ---
 
@@ -1061,8 +1521,9 @@ The schema cannot express, and therefore leaves to the compiler (stated in
 `description`s in the artifacts): fixed clause ordering (semantic, not
 structural — every key order is valid JSON), variable scoping and duplicate
 detection (`JQ0005`/`JQ0007`), `$as` name binding (`JQ0005`), full grammar
-of variable-rooted path segments (only the head is pattern-checked), and all
-runtime typing rules. Schema-literal positions (§8.11) validate as `true` —
+of variable-rooted path segments (only the head is pattern-checked), the
+co-occurrence rules of the extended `$for` binding (`$size`/`$step` require
+`$window`, `$window` requires `$size`, §6.10), and all runtime typing rules. Schema-literal positions (§8.11) validate as `true` —
 draft-neutral by definition; embedded JSON Schemas are deliberately **not**
 meta-validated by these artifacts (the type-test compiler is authoritative,
 `JQ0009`). Where the schema and this text disagree, this text wins and the
@@ -1210,6 +1671,20 @@ emit a bespoke query DSL as free text and parsing it hopefully — and the
 trade is: grammar errors eliminated by construction, semantic errors reduced
 to a machine-checkable, machine-repairable residue.
 
+Provider structured-output modes support different JSON Schema subsets
+regardless of the draft they advertise, and the strict ones enforce neither
+`patternProperties`/`propertyNames` nor asserted `format`s and reject
+`oneOf`. For those, a third artifact ships beside the two of §12.1:
+`packages/json/schemas/jaren-query.llm-profile.schema.json`, `$id`
+`https://jarenjs.dev/schemas/jaren-query/0.1/llm-profile` — a mechanically
+derived, pure *relaxation* in which those constraints are removed (each
+restated in the nearest `description`, which the model still reads) and
+every `oneOf` becomes `anyOf`. Every canonical-valid query document
+validates under the profile; the reverse is deliberately not guaranteed.
+Hand the profile to the decoder, keep validating locally against the
+canonical schema — the package [README](../README.md#generating-queries-with-llms)
+walks the full pipeline.
+
 Because query documents are plain JSON, they also travel well through the
 rest of an LLM toolchain: function-call arguments, retrieval filters, and
 audit logs all speak JSON already, and a generated query can be validated,
@@ -1219,3 +1694,190 @@ The same constrained-decoding model applies to complete recursive
 stylesheets; see
 [JSLT-FORMAT Appendix B](./JSLT-FORMAT.md#appendix-b-llm-structured-output-non-normative)
 and its mechanically query-derived schema twins.
+
+## Appendix C. The normalized form (normative)
+
+The engine compiles in two stages: stage 1 **normalizes** a query
+document into a frozen abstract syntax tree (the grammar of §§3–9
+resolved — object partitioning, phrase classification, string forms,
+scope resolution, cardinality analysis); stage 2 specializes that tree
+into closures. Stage 2 is an optimisation artifact and changes freely.
+**Stage 1 is the language resolved, and this appendix publishes it as a
+contract**: `analyzeQuery` (the package's `./query` subpath) returns the
+normalized tree so a consumer — a translator, a planner, an analyzer —
+can walk *the engine's own reading* of a document instead of inventing a
+second one.
+
+### C.1 The analysis entry point
+
+```js
+analyzeQuery(doc, options) -> {
+  astVersion,   // integer; see the compatibility policy (C.7)
+  root,         // the frozen node tree (C.3)
+  externals,    // [{ name, slot }] in order of first appearance (§9)
+  frameSize,    // total frame slots the tree addresses (C.5)
+  dependencies, // { externals, operators, functions, collations }
+  limits,       // the normalized limits record, or null
+}
+```
+
+`analyzeQuery` accepts the same options as `compileJsonQuery` and
+applies the same JQ0xxx rejections, with one deliberate difference:
+**schema literals do not require `options.compileTypeTest`**. Where
+compilation without the hook is `JQ0008`, analysis normalizes the schema
+literal to its `raw` node carrying the frozen schema with **no compiled
+predicate** (`test` is `null`), so a consumer can analyse a document it
+could not execute. When the hook IS supplied, analysis compiles the
+predicate exactly as compilation would (and can therefore still raise
+`JQ0009`). Everything else — `$call`/`$collation` registry resolution,
+closed-world externals, limits validation — behaves identically in both
+modes.
+
+A caller wanting both pays for one normalization:
+`compileJsonQuery(doc, { analysis: true })` exposes the same record at
+`query.analysis` on the compiled query.
+
+`NODE_KINDS` (same subpath) is the frozen, sorted list of the twelve
+node kinds of C.3; `AST_VERSION` is the current version integer.
+
+### C.2 The cardinality lattice
+
+Every node carries `card`, a static **upper approximation** of its
+runtime sequence length:
+
+| value | constant | meaning |
+|---|---|---|
+| 0 | `CARD_ZERO` | statically the empty sequence |
+| 1 | `CARD_ONE` | always exactly one item |
+| 2 | `CARD_OPT` | zero or one item |
+| 3 | `CARD_MANY` | any number of items (the top) |
+
+Two combinators are part of the contract: `joinCard(a, b)` is the least
+upper bound (the cardinality of "one branch or the other", `$if`), and
+`sumCard(a, b)` is concatenation (`$seq`): `ZERO` is its identity and
+any two non-`ZERO` contributions give `MANY`. Path nodes additionally
+carry `singular` (C.3), the RFC 9535 singular-query judgement of their
+segment list.
+
+### C.3 The twelve node kinds
+
+Every node is a plain frozen object carrying at least
+`{ kind, card, docPath }`. The complete field sets:
+
+| kind | fields beyond `kind`/`card`/`docPath` |
+|---|---|
+| `literal` | `value` — a frozen JSON value (scalars, `$const` payloads, non-`$` strings) |
+| `var` | `slot`, `external` (boolean), `name` (`'$'` for the input document, slot 0) |
+| `path` | `name` (root variable), `rootSlot`, `external`, `rootCard`, `segments` (the frozen RFC 9535 segment list of `parseJSONPath`), `singular` |
+| `object` | `entries` — `[{ name, expr }]`, the map constructor of Rule 1 |
+| `map` | `pairs` — `[{ key, value }]`, the general `$map` constructor |
+| `array` | `elements` — `[node]`, the Rule 3 array constructor |
+| `raw` | `value` — a frozen verbatim JSON value (operator `raw`/`name`/`schema` argument positions); schema positions also carry `test` — the compiled predicate, or `null` under analysis without a hook |
+| `op` | `name`, `args` — `[node]`; `$range` under `limits` also carries `limits`; host extension operators also carry `entry` (the host's registry entry, an opaque host value) |
+| `call` | `name`, `fn` (the registered host function — an opaque host value), `args` |
+| `let` | `bindings` — `[{ name, slot, expr }]`, `ret` (the degenerate `{$let, $return}` phrase) |
+| `quant` | `some` (boolean), `bindings` — `[{ name, slot, expr }]`, `satisfies` |
+| `flwor` | see C.4 |
+
+**Host-valued members.** `raw.test`, `call.fn`, `op.entry` and an
+orderby spec's `collation` are the four places the tree carries host
+functions rather than JSON. A consumer that serializes or diffs the tree
+MUST treat them as opaque presence/absence facts; everything else in the
+tree is plain JSON.
+
+### C.4 The `flwor` node
+
+The full phrase (§6) normalizes to one node with the clauses in their
+fixed semantic order regardless of JSON key order:
+
+- `fold` — `null` or `{ name, slot, expr, docPath }` (§6.9); the initial
+  value is normalized in the ENCLOSING scope.
+- `forBindings` — `[{ name, slot, expr, atSlot, allowingEmpty, window }]`;
+  `atSlot` is `-1` without `$at`; `window` is `null` or
+  `{ sliding, size, step }` (§6.10).
+- `letBindings` — `[{ name, slot, expr }]`.
+- `asChecks` — `null` or `[{ name, slot, isLet, schema, test, docPath }]`
+  (§6.8); `test` is `null` under analysis without a hook.
+- `where` — `null` or a node.
+- `groupby` — `null` or `{ keys: [{ name, slot, expr, docPath }],
+  docPath, accSlots }`; `accSlots` is the frozen list of pre-group
+  binding slots that later clauses actually read (barrier liveness — an
+  over-approximation-free artifact of `collectReadSlots`).
+- `orderby` — `null` or `{ specs: [{ key, desc, emptyGreatest,
+  collation, collationName, docPath }], docPath, liveSlots }`;
+  `liveSlots` is the analogous snapshot liveness list at the sort
+  barrier.
+- `count` — `null` or `{ name, slot }`.
+- `ret` — the `$return` node.
+- `limits` — the normalized limits record, or `null`.
+
+### C.5 Slots, frames, and external resolution
+
+A compiled query evaluates against one frame array. **Slot 0 is the
+input document.** Every binding site — `$for` names, `$at` names, `$let`
+names, `$fold`'s accumulator, `$groupby` key names, `$count` — and every
+external parameter is allocated the next slot from a single counter, in
+normalization order; nested phrases keep allocating in the same frame.
+`frameSize` is the final counter (one more than the highest slot; under
+`limits.steps` one extra slot holds the step counter). Scoping is
+lexical: a name resolves to the innermost binding; a free name is an
+**external parameter**, allocated a slot at its first appearance (use is
+the declaration, §9) and reported in `externals` in that order. Under a
+closed-world compilation (`options.externals`) a free name outside the
+declared set is `JQ0005` at its own reference site. An external's `card`
+is `CARD_ONE` (the caller binds one JSON value).
+
+### C.6 `docPath` and the freezing guarantee
+
+Every node's `docPath` is an RFC 6901 JSON Pointer into the query
+document as written: `''` is the document root (or `/$expr`-prefixed
+under the version envelope, §4), object member names are
+pointer-escaped, operator argument positions append the operator key and
+the array index (`/$where/$eq/0`). It is the same pointer surface the
+`JQ0xxx` errors carry.
+
+The tree is **deeply frozen at every level** — nodes, binding records,
+segment lists, captured values. That is a guarantee, not an
+implementation detail: a consumer may hold, share and index the tree
+without defensive copies, and MUST NOT mutate it (annotation passes
+return new trees; see `annotateTypes`). Captured document fragments
+(`literal`/`raw` values, schemas) are deep-frozen COPIES — the caller's
+objects are never frozen.
+
+### C.7 The compatibility policy
+
+`AST_VERSION` (currently **1**) is bumped when a node kind is added or
+removed, a published field is removed or retyped, or an invariant of
+this appendix changes.
+
+- Adding a **new optional field** to a node is NOT a version bump;
+  consumers must tolerate unknown fields.
+- Adding a **node kind** IS a version bump: an exhaustive consumer
+  dispatching on `kind` must fail loudly on a kind it does not know,
+  and the version tells it why.
+- Explicitly **not** promised: the compiled closures, the operator
+  registry's internals (`compile` bodies), evaluation order beyond what
+  §§2–9 already require, and the two liveness lists' exact contents
+  beyond "the slots later clauses read".
+
+### C.8 Type annotation (optional pass)
+
+The tree carries cardinality but no value types. `annotateTypes`
+(same subpath) is a separate, optional pass:
+
+```js
+annotateTypes(analysis, { typeOf }) -> analysis'
+```
+
+It returns a NEW analysis whose tree mirrors the input with a frozen
+`type` tag on every node — `{ type: 'unknown' | 'null' | 'boolean' |
+'number' | 'integer' | 'string' | 'array' | 'object', optional:
+boolean }` — never mutating the input and never running during
+compilation. `typeOf(pathNode)` is the caller's answer for path nodes
+(a store schema, a model — whatever the caller knows); returning
+`null`/`undefined` means unknown. Literals and constructors type
+themselves; quantifiers are boolean; operators propagate through the
+registry's declared `resultType` families (comparison, arithmetic,
+string, aggregate — everything undeclared yields `unknown`).
+**`unknown` is always a safe answer; a wrong tag is a defect.** General
+inference beyond these rules is out of scope here.