@xcitedbs/client 0.2.14 → 0.2.16

package/unquery-grammar.md

@@ -0,0 +1,414 @@
+# Unquery — grammar and parse tree (parser-faithful)
+
+This document is the **ground truth** for Unquery syntax as implemented in the XCiteDB engine. It is derived from `TParser` in [`XCiteDB2/XCiteDB/src/TemplateParser.cpp`](../XCiteDB2/XCiteDB/src/TemplateParser.cpp) and the AST in [`XCiteDB2/XCiteDB/include/TemplateQuery.h`](../XCiteDB2/XCiteDB/include/TemplateQuery.h). Use it for **syntax checking** and **conservative static typing** of expressions inside JSON templates.
+
+Human-oriented prose and examples: [Unquery reference manual](../web/src/docs/content/unquery-language-reference.html) (Asciidoc source: `web/src/docs/unquery-source/unquery-language-reference.adoc`).
+
+---
+
+## 1. Two-stage parsing
+
+### 1.1 Stage 1 — JSON template (`JSONToTQ`)
+
+The outer query is a **JSON value** (object, array, or leaf). The host parses JSON; Unquery then walks it.
+
+| JSON kind | AST / node | Notes |
+|-----------|------------|--------|
+| `array` | `TQArray` | Each element is a child `TemplateQuery`. |
+| `object` | `TQObject` | Each member: parse **key string** then **value**. |
+| `string` (top-level fragment) | `TQValue` / `TQValueWithCond` | Entire string passed to `TParser::value()`. |
+| `number` | `TQValue` + `TExprIntConst` / `TExprDoubleConst` | Not parsed as Unquery text. |
+| `true` / `false` | `TQValue` + `TExprBoolConst` | Not parsed as Unquery text. |
+| `null` / missing | `TQValue` + `TExprJSONConst(null)` | Fallback when `JSONToTQ` gets an unsupported JSON kind. |
+
+**Object member processing** (`JSONToTQ`, object branch):
+
+1. `name = m.name.GetString()` — the JSON **key** is a string.
+2. `TParser parse_name(name, st)` — same lexer/parser as values.
+3. `key = parse_name.key()` then `context_mod = parse_name.context_mod(true, ContextModMode::Start)`.
+4. If `m.value` is a **string**: parse with `TParser parse_val(...)` according to `key->getKeyType()` (see §7).
+5. If `m.value` is not a string: `JSONToTQ(m.value)` recursively; optional `context_mod->replace(val)`.
+
+**Symbol table**: `TSymTable` holds `funcs: map<string,int>` (name → parameter count). Populated when a `#func` key is parsed; used when parsing `$userfunc(...)` in expressions.
+
+### 1.2 Stage 2 — string sublanguages
+
+| Sublanguage | Entry point | Typical use |
+|-------------|---------------|-------------|
+| Object **key** (before context modifiers) | `TParser::key()` | JSON object member name. |
+| Context modifier chain after key | `TParser::context_mod()` | Suffix after key in the same string. |
+| Object **value** (string leaf) or directive body | `TParser::value()` | Expression + optional `?` + optional constraint + optional `@sort`. |
+| `#if` value only | `TParser::condition()` | No `value()` — no sort suffix on the condition string. |
+
+**Principle (evaluation)**: result shape mirrors template shape, except where the manual documents exceptions (`#return`, `#returnif`, dynamic keys, `||` context-or, `**`, etc.) — see §9.
+
+---
+
+## 2. Lexical grammar (`TParser::nextToken`)
+
+Whitespace (`isspace`) is skipped between tokens. After a token is consumed, trailing spaces are skipped (`_pos` advanced).
+
+### 2.1 Tokens
+
+**Numbers** — if first char is digit:
+
+- Consume digits and at most one `.` in the numeric part.
+- If the next char is alphanumeric or `_`, extend the token as an **identifier-like** run (so tokens like `123abc` exist as a single token).
+
+**Identifiers** — if first char is `isalnum(c) || c=='_'` OR `$` OR `%`:
+
+- Consume while `isalnum_` (`[A-Za-z0-9_]`).
+
+**Double-quoted string** — starts with `"`:
+
+- Run until closing `"` that is not preceded by `\` (backslash escapes the quote).
+- Error if EOF before closing quote.
+
+**Single-quoted string** — starts with `'`:
+
+- Run until next `'` (no escape handling in lexer).
+- Error if EOF.
+
+**Backtick string** — starts with `` ` ``:
+
+- Run until next `` ` ``.
+- Error if EOF.
+
+**Single-character punctuation** (each is one token unless merged below):
+
+`()[]{};,.#?!:$%`
+
+Special: if char is `.` and next is `.`, emit token `..` (two dots).
+
+**Two-character operators** (only these merge; otherwise adjacent operators are separate tokens):
+
+| Token | Condition |
+|-------|-----------|
+| `<=` | `<` followed by `=` |
+| `>=` | `>` followed by `=` |
+| `<<` | `<` followed by `<` |
+| `->` | `-` followed by `>` |
+| `\|\|` | `\|` followed by `\|` |
+| `**` | `*` followed by `*` |
+
+**Important**: `<-` is **not** one token — it lexes as `<` then `-` (comment in parser: avoids invalid `<-`).
+
+Any other punctuation: single character only.
+
+---
+
+## 3. String value grammar (`TParser::value`)
+
+Parser entry: read one **expression**, then optional pieces, then **must** be at end of string (`eos()`).
+
+```ebnf
+value       ::= expression ( '?' condition )? constraint? sortSpec? EOF
+constraint  ::= baseCondition   (* only if next token is not '@' and not EOF; see below *)
+sortSpec    ::= '@' ( 'ascending' | 'descending' | 'unique_ascending' | 'unique_descending' )
+                ( '(' INTEGER ')' )?
+```
+
+**Constraint coupling** (implementation detail, affects syntax):
+
+- After parsing `expression` and optional `?condition`, if the input is **not** at EOF and the next token is **not** `@`, the parser calls `baseCondition(exp)` where `exp` is the already-parsed expression (first operand of the comparison / test).
+- So forms like `"Field1?predicate Field1=2"` are parsed; the constraint shares the leading expression as LHS.
+- AST: outer `TQValue` may be wrapped in `TQValueWithCond` for the `?` part; the **constraint** from `value()`’s `second` member is attached separately in `JSONToTQ` (object field vs `KeyType::Values` — see §7).
+
+**Sort order** (`@`): must be one of the four keywords above; optional `( n )` where `n` is a single integer token (`stoll`). Unknown sort keyword → parse error.
+
+---
+
+## 4. Expression grammar (`expression`, `baseExpression`)
+
+### 4.1 Precedence (`expression(int prec)`)
+
+Climbing parser levels:
+
+| `prec` | Operators | Associates |
+|--------|-----------|------------|
+| 0 | `+`, `-` | Left via loop: each new rhs parsed with `expression(1)`. |
+| 1 | `*`, `/`, `mod` | Left; rhs parsed with `expression(2)`. |
+| 3 | Postfix `.token` and `[ expr ]` | Postfix chain on current `res`. |
+
+Parentheses: `(` `expression(0)` `)` as primary.
+
+### 4.2 EBNF sketch
+
+```ebnf
+expression   ::= '(' expression ')'
+                | baseExpression ( ( '+' | '-' ) expression(1)
+                                 | ( '*' | '/' | 'mod' ) expression(2) )*
+                | baseExpression ( '.' fieldSegment | '[' expression ']' )*
+
+fieldSegment ::= IDENT | BACKTICK_STR | '$' IDENT ...   (* lexer token after '.'; special case: if token starts with '$', subfield parse backs up — see pathWithBrackets *)
+```
+
+**Postfix** (prec ≤ 3): after `consume()` of `.` or `[`, if `[` then parse `expression(0)` unless next is `]`; require `]`; build `TExprSubfield(res, expr_or_empty, is_index)`.
+
+### 4.3 `baseExpression` — alternatives (first token disambiguation)
+
+Roughly in parse order:
+
+| Prefix / form | AST | Notes |
+|-----------------|-----|--------|
+| `[` optional `]` | `TExprSubfield(TExprField("."), expr?, is_index)` | `[]` = whole array on current `.`; `[e]` = index. |
+| `$` `(` `expression` `)` | `TExprField(expr)` | Evaluate / path from dynamic name (`$()`). |
+| `$if` `(` `condition` `,` `expression` `,` `expression` `)` | `TExprITE` | |
+| `$call` `(` IDENT `)` | `TExprCall(name)` | Zero-arg user function by name. |
+| `$var` `(` IDENT `)` | `TExprVar` | |
+| `$file` `(` `expression` `)` | `TExprFile` | |
+| `$csv` `(` `expression` ( `,` delim? `,` header? )? `)` | `TExprCSV` | Optional delim: next token strip quotes; optional header: `boolean()`. |
+| `$prev` `(` `expression` `)` | `TExprPrev` | Aggregate. |
+| `$lower` / `$upper` `(` `expression` `)` | `TExprChangeCase` | |
+| `$path` | `TExprPath` | |
+| `$index` | `TExprIndex` | |
+| `$key` | `TExprKey` | |
+| `$reskey` | `TExprReskey` | |
+| `$filename` | `TExprFilename` | |
+| `$env` `(` `expression` `)` | `TExprEnv` | |
+| `$identifier` ( `(` INT `)` )? | `TExprIdentifier(parts)` | |
+| `$xml` / `$xml_no_children` | `TExprXML` | |
+| `$node` | `TExprNode` | |
+| `$attr` `(` QUOTED `)` | `TExprAttr` | Param must be quoted then stripped. |
+| `$child` `(` QUOTED `)` | `TExprChild` | |
+| `$xpath` / `$lxpath` `(` QUOTED `)` | `TExprXPath` | |
+| `$in_filter` `(` QUOTED `)` | `TExprInFilter` | |
+| `$text` `(` `expression` `)` | `TExprText` | |
+| `$size` / `$length` `(` `expression` `)` | `TExprSize` / `TExprLength` | |
+| `$to_time` / `$time_to_str` `(` `expression` ( `,` QUOTED )? `)` | **Bug in parser**: inner `fmt` is declared twice in block; format may be ignored — treat as optional second arg. |
+| `$now` | `TExprIntConst(epoch)` | Evaluated at parse time in parser. |
+| `$string` / `$number` / `$int` / `$float` / `$bool` `(` `expression` `)` | `TExprTypeCast` | |
+| `$node_date` | `TExprLastChange(false,false)` | |
+| `$data_date` ( `(` `expression` `)` )? | `TExprLastChange(true,false,arg)` | No parens → arg is `TExprField(".")`. |
+| `$count` | `TExprCount` | Aggregate, no args. |
+| `$sum` / `$avg` `(` `expression` `)` | `TExprSum` / `TExprAvg` | Aggregates. |
+| `$min` / `$max` `(` `expression` `)` | `TExprMinmax` | Aggregate. |
+| `$substr` `(` `expression` `,` `expression(0)` ( `,` `expression(0)` )? `)` | `TExprSubstr` | |
+| `$find` / `$ifind` `(` `expression` `,` `expression(0)` `)` | `TExprFind` | |
+| `$replace` `(` expr `,` expr `,` expr ( `,` boolean )? `)` | `TExprReplace` | |
+| `$split` `(` expr `,` expr `)` | `TExprSplit` | |
+| `$join` `(` expr ( `,` expr )? `)` | `TExprJoin` | If no second arg, delimiter is empty string const. |
+| `$D` QUOTED | `TExprIntConst` | Date literal → epoch via `stringToTime`. |
+| `%` IDENT | `TExprVar` | Same as `$var(name)` when name is rest of token. |
+| `$` IDENT `(` arg ( `,` arg )* `)` | `TExprCall` | **User-defined** function: name must exist in `sym_table->funcs`; arity must match. |
+| `"..."` / `'...'` | `TExprStringConst` | Strip outer quotes. |
+| number | `TExprIntConst` / `TExprDoubleConst` | If token contains `.` → double. |
+| `true` / `false` | `TExprBoolConst` | |
+| `/` `baseExpression` | `TExprChangepath(..., ROOT)` | |
+| `..` ( `/` `baseExpression` )? | `TExprChangepath(..., UP)` | Without `/`, inner arg defaults to `TExprField(".")`. |
+| `<<` `baseExpression` | `TExprChangepath(..., PREVID)` | |
+| IDENT … | `TExprField(path)` | Via `pathWithBrackets(firstToken)`. |
+| `` `...` `` | `TExprField` | Strip + `escape_field_name`. |
+| `.` | `TExprField(".")` | |
+| `-` number | `TExprIntConst(-n)` | **Only** if next token is a number token. |
+
+**Backtick in postfix position**: if the next token after an infix position starts with `` ` ``, lexer returns it; `expression()` treats it as `op` and runs `escape_field_name(stripQuotes_(op))` for the field segment string.
+
+---
+
+## 5. Path segments (`pathId`, `pathWithBrackets`)
+
+```ebnf
+pathId   ::= IDENT | BACKTICK_STR | '"' ... '"' | '\'' ... '\''
+pathMore ::= '.' ( IDENT | BACKTICK_STR | '$' ... )   (* if next token after '.' starts with '$', pathWithBrackets stops — Field.$x is NOT continued as path *)
+          | '[' INTEGER ']'
+```
+
+**`pathWithBrackets`**: uses save/restore; only consumes `.field` or `[n]` when syntactically valid; numeric index must be integer token followed by `]`.
+
+**Note**: Dynamic index in **path text** at primary position uses postfix `[` `expression` `]` in `expression()`, not `pathWithBrackets` (which only accepts **integer literals** in bracket segments).
+
+### 5.1 Identifier expressions (after `->`)
+
+Parsed when the text after `->` is unambiguously an identifier path (see §8.2 dispatch). Builds a **`TQIdExpr`**: a list of `TQIdStep` operations (`IdStepKind`) evaluated at runtime to produce **zero or more** identifier strings (set-valued, like JSON-path iteration). A path that collapses to a single identifier is the trivial one-element case.
+
+```ebnf
+identifierExpression ::= idAnchor idSuffix
+                       | '[' ']' idSuffix                    (* anchor-less: same as `->./[]` … *)
+                       | '**' idSuffix                       (* anchor-less: same as `->./**` … *)
+                       | idSegment ( '/' idSegment )+ idSuffix   (* unanchored under current id *)
+
+idAnchor               ::= '/' | '.' '/' | ( '..' '/' )+
+
+idSuffix               ::= ( '/' idSegment | '[' ']' | '**' )*
+
+idSegment              ::= IDENT | BACKTICK_STR | QUOTED_STR
+                       | '$' '(' expression ')'
+                       | '%' IDENT
+                       | '.'                                  (* no-op segment *)
+                       | '[' ']'                              (* children of preceding identifier *)
+                       | '**'                                 (* self + all descendants of preceding *)
+```
+
+**Not allowed:** mid-path `..` (only the leading `../` anchor run); `[` `N` `]` / `[` anything other than `]` (no index or pattern brackets); mid-path `..` as a segment; `[]` / `**` immediately inside `->$each(...)` (each argument is a JSON expression, not an identifier path). Going “above” `/` with `../` yields an empty identifier at runtime (no match, no exception).
+
+**`[]` semantics:** expand to every **direct child** identifier under the path built so far (same as legacy `->$children` from that node). When the `id_hier` LMDB sub-db is present on overlay and base, evaluation uses `XMLReader::listChildrenFromHierMerged` for efficiency; otherwise it falls back to the same `SimpleQuery` + `match_start` scan as before.
+
+**`**` semantics:** expand to the identifier built so far **plus every descendant** identifier under it (same shape as JSON-path `**` under the current path — includes self).
+
+**`->$each(expr)`** is a separate arrow form (`ArrowOp::Each`): `expr.getJSON(ctx)` must be a string or array of strings; each string is used as an identifier. **`$each(...)` must not be followed by `/` in the same arrow modifier** (parse error); chain another `->` or put path logic inside the expression.
+
+---
+
+## 6. Conditions (`condition`, `baseCondition`)
+
+### 6.1 Boolean condition grammar
+
+```ebnf
+condition      ::= condOr
+condOr         ::= condAnd ( '|' condAnd )*          (* prec <= 1 *)
+condAnd        ::= condNot ( '&' condNot )*         (* prec <= 2 *)
+condNot        ::= '!' condNot                      (* prec <= 3 *)
+                | condAtom
+condAtom       ::= '(' disambig ')'
+disambig       ::= try parse expression(0); if next ')' then ')' and baseCondition(expr)
+                | else condition(0) ')'             (* parenthesized sub-condition *)
+                | baseCondition()
+```
+
+**`baseCondition`** (after optional lhs from caller):
+
+1. If no `lhs` and next is `(`, parse condition inside parens (retry path on parse error — see source).
+2. Else if no `lhs`, `lhs = expression()`.
+3. `op = nextToken()`.
+4. If `op == "!"` and next is `=`, consume `=` → op `!=`.
+5. Else if `op == "!"` alone → **`TQExistsTest(lhs)`** — no RHS expression.
+6. Else if `op` is one of `is_array`, `is_object`, `is_literal`, `is_string`, `is_number`, `is_int`, `is_float`, `is_bool` → **`TQTypeTest`**, no RHS.
+7. Else `rhs = expression()`; then dispatch on `op` for compare / string / `in` / `not_in`.
+8. **Static rule**: if `(lhs->isAggregate(0) && !rhs->isLiteral()) || (rhs->isAggregate(0) && !lhs->isLiteral())` → **parse error** `"Aggregates can only be compared with literals"`.
+
+Compare ops: `=`, `!=`, `<`, `>`, `<=`, `>=`.  
+String ops: `contains`, `starts_with`, `ends_with`, `matches`.  
+Set ops: `in`, `not_in`.
+
+**Note**: condition OR is **single** `|`, not `||` (`||` is lexer merge for **context modifiers** only).
+
+---
+
+## 7. Keys (`TParser::key`) and `JSONToTQ` string dispatch
+
+### 7.1 Key grammar
+
+```ebnf
+key ::= '$' '(' expression ')'              (* dynamic key → TQParamKey *)
+     | '$' IDENT ... | '%' IDENT ...       (* whole thing parsed as expression() → TQParamKey *)
+     | '{' '}'                             (* TQRegexKey empty regex *)
+     | '{' QUOTED '}'                      (* TQRegexKey *)
+     | '#' directive
+     | pathId                              (* TQSimpleKey, name may include quoted segments from pathId *)
+
+directive ::= 'if'
+           | 'func' IDENT ( '(' IDENT ( ',' IDENT )* ')' )?   (* register arity; TQFuncDefinition *)
+           | 'var' IDENT
+           | 'assign' IDENT
+           | 'exists' | 'notexists' | 'return' | 'returnif'
+```
+
+Unknown `#name` → error.
+
+### 7.2 `KeyType` and how string **values** are parsed (`JSONToTQ`)
+
+For object members whose JSON **value** is a string, `JSONToTQ` chooses the parser by `key->getKeyType()`:
+
+| Key kind | `getKeyType()` | String value parsed with | `obj->add(key, q, condq)` behavior |
+|----------|----------------|----------------------------|-------------------------------------|
+| `#if` | `Cond` | `condition()` only → `TQConditionP` stored in local `cond` | `q` stays empty. If `cond` set, `condq = TQCondWrapper(cond)` (with optional `context_mod` on `condq`). `add(#if, nullptr, condq)` — `TQObject::add` inserts a synthetic `KeyType::Cond` field holding the wrapper so `processData` tests it before other members. |
+| `TQSimpleKey`, `TQParamKey`, `TQRegexKey` | **`Values`** (inherits default `TQKey::getKeyType`) | `value()` → `vc.first` = template value, `vc.second` = optional **constraint** from same string | If `vc.second`: `condq = TQCondWrapper` + context mod; `q = vc.first` + context mod. Constraint is **object-level** (wrapper), not inside `TQValueWithCond`. |
+| `#var`, `#assign`, `#exists`, `#notexists`, `#return`, `#returnif` | `Variable`, `Assign`, `Exists`, `Notexists`, `Return`, `ReturnIf` | `else` branch: `value()` | If `vc.second`, entire `q` is `TQValueWithCond(vc.first, vc.second)` — constraint is **on the value**. |
+| `#func` | `Func` | `else` branch: `value()` (body is a full template subtree when JSON is object/array) | Same wrapping as other directives; registration uses `TQFuncDefinition` + `sym_table`. |
+
+**Summary**: Trailing **constraint** after the main expression in a field value is either a separate **`TQCondWrapper`** (normal / dynamic keys) or embedded in **`TQValueWithCond`** (directives). `#if` has no `value()` — its string is only a `condition()`.
+
+---
+
+## 8. Context modifiers (`TParser::context_mod`)
+
+Parsed **after** `key()` from the **same** JSON key string. Mode parameter `ContextModMode` controls whether leading `:` / `->` / `[` / `{` / `.` is required or forbidden (start vs continuation vs `||` branch).
+
+### 8.1 Modifier atoms (after optional leading `:`)
+
+| Form | `ContextMode` | `arrow` / payload |
+|------|---------------|-------------------|
+| `{` `}` or `{` `regex` `}` | `Regex` | String context; empty regex = all keys |
+| `**` | `AllPaths` | |
+| `$` `(` `expression` `)` | `Eval` | `expr` |
+| `$…` or `%…` expression | `Eval` | `expr` from `expression()` |
+| `->` … | `Arrow` | See §8.2 (`identifierExpression`, builtins, legacy JSON-field lookup, `->$each`); often `new_frame = true` |
+| `[` `]` | `Array` | traversal |
+| `[` INT `]` | *(string context)* | literal index path segment |
+| `.` | *(path)* | context string `"."` |
+| IDENT… `[` `]`? | `Array` if trailing `[]`, else path | Empty path + no `[` → **`Reskey`** mode |
+| Optional `?` `condition` after each atom | — | Wraps inner mod chain in `TQValueWithCond` |
+
+Recursion: `context_mod(...)` parses the **rest** of the chain; result wrapped in `TQContextMod` (expr or string form).
+
+### 8.2 Arrow (`->`) targets
+
+Dispatch (after `->`):
+
+| Next shape | Behaviour |
+|------------|-----------|
+| `$self`, `$parent`, `$children`, `$descendants`, `$descendants_and_self`, `$ancestors`, `$ancestors_and_self`, `$all` | **Parser sugar:** lowered to a constant `TQIdExpr` and stored as **`ArrowOp::IdExpr`** (same runtime as path syntax). |
+| `$date(…)`, `$branch(…)`, `$file(…)`, `$csv(…)` | Built-ins unchanged (`getArrowOp` + parsing of args). |
+| `$var` `(` … `)` | `ArrowOp::Var` with `context` from `pathWithBrackets`. |
+| `%ident` (standalone) | `ArrowOp::Var`. |
+| `$each` `(` `expression` `)` | **`ArrowOp::Each`** — iterate string/array of identifier strings from `expr.getJSON(ctx)`. Must not be followed by `/` in the same modifier. |
+| `/`, `./`, `../`, `[` `]`, `**`, anchors + `identifierExpression` triggers (see `startsIdentifierExpression`) | **`identifierExpression`** → **`ArrowOp::IdExpr`** + `TQIdExpr`. |
+| Single quoted path segment with no following `/` | **`identifierExpression`** (one `Literal` step). |
+| `IDENT` / `pathWithBrackets` with **no** `/` that triggered `identifierExpression`; bare `$(expr)` with no following `/` | **Legacy** JSON-field lookup: `ctx.getJSON(name)` where `name` is field name or `expr` string (`ArrowOp::None`). **Deprecation:** parser sets `legacy_warn` on the `TQContextMod`; runtime prints a one-time-per-node warning to `stderr`. |
+
+**Chaining**: `||` after a **complete** modifier started with `:` (see source): builds `TQContextModOr` with `TQShared` placeholder so alternatives share downstream value.
+
+---
+
+## 9. Result shape (static intuition)
+
+- **Objects**: one result object per template object; keys may be repeated dynamically (`$(...)` / `{}` keys).
+- **Arrays**: appends one element per evaluation pass as per engine rules.
+- **`#return`**: first `#return` in object wins; flattens shape (manual §8.6).
+- **`#returnif`**: first non-empty wins among returnifs (manual §8.7).
+- **Context traversal** (`:[]`, `:{}`, `**`, arrow iterators): may run sub-template many times → many contributions to arrays / dynamic keys.
+- **`||`**: union of contexts for the **same** value subtree.
+
+---
+
+## 10. Static type-checking checklist (for AIs)
+
+1. **End of string**: `value()` must consume entire string; trailing garbage → `"Could not parse text at the end"`.
+2. **Aggregates in conditions**: one side must be **literal** if the other is aggregate (`isAggregate(0)`).
+3. **`#if` value**: parse as **condition**, not `value()` — no `@sort` on that string.
+4. **`#func f(a,b)`**: register arity; every `$f(...)` must supply exactly that many comma-separated `expression` arguments.
+5. **Built-in arity**: match §4.3 table; `$join` allows 1–2 args; `$replace` allows 3–4; `$csv` allows 1–3; `$identifier` allows 0–1 (parens with int).
+6. **`<-`**: not a token; write `<` and unary `-` separately — almost certainly wrong for “less than minus”.
+7. **`..`**: token is two dots; field access after parent must use `../` (`..` then `/` then `baseExpression`).
+8. **Sort**: `@sort` must be last in a **value** string; cannot follow non-`@` trailing constraint without parsing constraint first (constraint is **not** after `@` — `@` ends value parsing).
+9. **Keys**: `#func` name must not duplicate earlier definition in same template symbol table pass.
+10. **`%x`**: same as `$var(x)` for expression typing.
+11. **Path literals in keys**: `pathId` allows quoted strings as path segments (stored as part of key name string); unusual but legal lexer-wise.
+12. **Condition OR**: use single `|`, not `||`.
+13. **`is_string`**: implemented in parser / `Operator` enum; reference manual may not list it — engine supports it.
+14. **Identifier expressions**: `[` `N` `]` and `[` non-`]` forms are rejected inside `idSegment` and immediately after a completed `identifierExpression` (no `[` before `?` / `||` / end) except the allowed `[]` children step.
+15. **`../` past root**: evaluates to empty identifier → no match (no throw).
+16. **`[]` / `**`**: the only bracket / star multi-value forms in identifier expressions; `[N]`, `[$(x)]`, mid-path `..`, and `[]` / `**` inside `$each(...)` are parse errors.
+
+---
+
+## 11. Appendix — AST classes (quick index)
+
+**Template / structure**: `TemplateQuery`, `TQPlaceholder`, `TQObject`, `TQArray`, `TQValue`, `TQValueWithCond`, `TQCondWrapper`, `TQContextMod`, `TQContextModOr`, `TQShared`, `TQInnerValue`, …
+
+**Keys**: `TQKey`, `TQSimpleKey`, `TQParamKey`, `TQRegexKey`, `TQDirectiveKey`, `TQFuncDefinition`
+
+**Expressions**: `TExpression`, `TExprField`, `TExprSubfield`, `TExprBinaryOp`, `TExprChangepath`, `TExprITE`, literals (`TExprStringConst`, `TExprIntConst`, `TExprDoubleConst`, `TExprBoolConst`, `TExprJSONConst`), `TExprCall`, `TExprVar`, `TExprFile`, `TExprCSV`, `TExprPrev`, `TExprCount`, `TExprSum`, `TExprAvg`, `TExprMinmax`, string/XML helpers (`TExprPath`, `TExprIndex`, `TExprKey`, …), `TExprTypeCast`, `TExprLastChange`, `TExprIdentifier`, `TExprIdentifierUp`, …
+
+**Context modifiers**: `TQContextMod` carries optional `legacy_warn` (deprecated legacy `->field` / bare `->$(expr)` JSON-field branch). **`ArrowOp`**: `None`, `Date`, `Branch`, `Var`, `File`, `Other`, **`IdExpr`** (with `TQIdExprP id_expr`), **`Each`**. Structural `->$self` / `$children` / … are not distinct enum values — they parse to `IdExpr`.
+
+**Conditions**: `TQCondition`, `TQCondBool`, `TQCompareTest`, `TQStringTest`, `TQJSONTest`, `TQExistsTest`, `TQTypeTest`
+
+**Operators** (shared enum): `Operator` in `TemplateQuery.h` — includes arithmetic, compare, `ROOT`, `UP`, `PREVID`, etc.
+
+---
+
+## Document history
+
+- Introduced as canonical grammar reference for tooling and AI-assisted editing; keep in sync when `TemplateParser.cpp` changes.