@spintax/core 0.6.1 → 0.8.0

package/CHANGELOG.md

@@ -3,6 +3,469 @@
 All notable changes to `@spintax/core` are documented here. This project adheres to
 [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## 0.8.0 — 2026-09-13
+
+Two defects found by the ports while they mirrored 0.7.0, both places where this engine read a
+template differently from the PHP engines — and both measured on those engines before anything
+changed: the character classes of the post-process, and the key that decides which constructs are
+re-read as text (#80).
+
+**The post-process reads characters the way PHP does.** PHP compiles every pattern of the cosmetic
+stage with `/u`, and `/u` turns on PCRE2_UCP: `\s`, `\d`, `\w` and `\b` are Unicode classes there.
+This engine took them for ASCII — the belief was written into its own comments ("no PCRE_UCP") — and
+the three ports that copied the classes did the same. Measured on both PHP engines:
+
+| input | 0.7.0 | both PHP engines, and now this one |
+|---|---|---|
+| `и т.д. и т.п. всё` | `И т. Д. И т. П. Всё` | `И т.д. и т.п. всё` |
+| `открой пример.рф сегодня` | `Открой пример. Рф сегодня` | `Открой пример.рф сегодня` |
+| `пишите на info@сайт.рф` | `Пишите на info@сайт. Рф` | `Пишите на info@сайт.рф` |
+| `end.<NBSP>next` | `End. <NBSP>next` | `End.<NBSP>Next` |
+| `в 5<NBSP>тыс.<NBSP>руб.` | `В 5<NBSP>тыс. <NBSP>руб.` | `В 5<NBSP>тыс.<NBSP>руб.` |
+
+**A conditional or a config reference inside `{…}`/`[…]` is text before the split, too (#80).** 0.7.0
+re-read a construct as text when a `%var%` sat directly in it, and decided that from a key narrower
+than its own rule:
+
+| template | 0.7.0 | both PHP engines, and now this one |
+|---|---|---|
+| `[<minsize=%n%;maxsize=%n%>a\|b\|c]`, `n=1` | all three elements | one element |
+| `[<sep=%S%>a\|b]`, `S=", "` | `b a` | `b, a` |
+| `[{?f?a\|b\|x}\|c]` | `c b\|x` — a raw pipe | three elements |
+| `[<sep=", ";lastsep=" and ">{?f?live casino}\|slots\|poker]` | `slots, poker and ` | `poker and slots` |
+| `[<sep=", ">slots\|{live casino\|}\|poker]`, the empty option picked | `slots, , poker` | `slots, poker` |
+
+Minor rather than patch: rendered text changes for every template of these shapes, and
+`AST_VERSION` moves to 4.
+
+### Fixed
+
+- **The post-process classes are PHP's UCP classes** (`internal/charclass.ts`). Whitespace is PCRE2's
+  `\s` — NBSP, the thin spaces, NEL — and not JavaScript's, which adds U+FEFF and misses U+0085 and
+  U+180E. `\b` counts a letter of any script as a word character. The digit in the two spacing
+  lookaheads is any `\p{Nd}`. The decimal shield stays ASCII: PHP writes that one pattern without `/u`.
+- **Conditional truthiness uses the same whitespace.** A value of U+FEFF alone is truthy (it was blank
+  here); a value of U+0085 alone is blank (it was truthy).
+- **The permutation-config patterns go the other way.** PHP writes them without `/u`, so their `\s` is
+  ASCII, while JavaScript's `\s` is Unicode on any flags. A no-break space around `=` is no longer
+  config whitespace — `[<minsize<NBSP>=<NBSP>1>a|b|c]` is one separator, as in PHP — and the validator
+  agrees: `[<foo<NBSP>=1>a|b]` is valid, `[<minsize=2<NBSP>>a|b]` is `permutation.minsize-not-integer`.
+- **A reference anywhere in a permutation's `<config>` marks it for the re-read** (#80). 0.7.0 tested
+  the PARSED `sep` and `lastsep`, where a size reference never arrives (a size that is not digits
+  parses to nothing) and neither does an unquoted separator (it parses to the default). The raw
+  header is tested now, so `minsize=%n%`, `maxsize=%n%` and `sep=%S%` take their values from the
+  context, as they always have in PHP. So does a whole `{?…}` there or in a per-element separator —
+  `[<lastsep="{?en? and | и }">a|b|c]` printed the raw conditional unless a neighbour happened to
+  trigger the re-read (found in review). A bare `{?` does not mark: the first cut of that check made
+  every level of `[<{?}>a|[<{?}>a|…]]` rescan its body for a conditional that was not there, twice
+  0.7.0's time on deep nesting (found by the Codex gate).
+- **A `{?…}` conditional directly in `{…}`/`[…]` marks the construct** (#80), whatever its branches
+  hold — 0.7.0 marked it only when a `%var%` sat in a branch. The plugin resolves conditionals at
+  Stage 6a, before any bracket is read, so a taken branch's `|` separates options, an empty branch
+  leaves an empty permutation element that is dropped, and whitespace at a branch's edge is the
+  element's edge. 0.7.0's notes kept the empty element on purpose as a pathological divergence; a
+  list item gated by a flag is an ordinary template, and in one it rendered `Есть покер, слоты и.`.
+- **A permutation element is its rendered text, trimmed, and one that renders empty is dropped** (#80).
+  The plugin resolves every nested enumeration before it splits a permutation, so an optional item
+  written `{live casino|}` that picks its empty option leaves `slots||poker` there — two elements —
+  while this engine kept a blank third and printed `slots, , poker`; `[<minsize=3;maxsize=3>a|{b|}|c]`
+  counted three and printed `a  c`. The conditional fix above did not reach it: the re-read drops a
+  part that is empty as TEXT, and `{b|}` is not empty until it is rendered. Found by the differential
+  below after the conditional half was in; the size pick and the shuffle count the elements that
+  remain, and an element that is neither empty nor padded draws as before.
+- **`AST_VERSION` 3 → 4.** The node shape is 3's, but an `Ast` cached by 0.7.0 lacks `raw` exactly
+  where the wider key puts it and would render the old output; the guard turns that into
+  `AstVersionError`. Nothing persists a handle across versions.
+- **Three patterns no longer scan a long run once per character** (found in review). Trimming each
+  assembled element with `/[…]+$/` retried from every position of a whitespace run inside the text —
+  the first cut of the element fix made `[a%v%y|z]` over 51 000 spaces take 3 s, and 13 s from a
+  362-byte macro template; the PHP trims are loops now, in the parser as well, where the same regex
+  already cost seconds on a re-read. And two post-process patterns that were quadratic IN 0.7.0 got
+  new triggers from the wider whitespace class: removing whitespace before punctuation rescanned a run
+  with no punctuation after it (100 000 form feeds: 10 s in 0.7.0; a macro of NBSP or U+3000: 10–31 s
+  on the first cut), and capitalizing after a line break rescanned a run of breaks and spaces with no
+  letter after it (20 000 of `\n` plus a space: 4 s in 0.7.0). Each now starts a match only where the
+  run starts — the same matches, checked over 500 000 generated strings — and all of these take
+  milliseconds.
+- **`parse()` and `render()` no longer throw on a long tag-shaped permutation config.** Whether
+  `[<name>…]` opens with an HTML tag was decided by compiling `</name\s*>` from the name itself, and V8
+  will not compile that past about 7.8 KB of name: `SyntaxError: Invalid regular expression … Stack
+  overflow`, straight out of `parse()`. A 7.8 KB source was enough — under the reference Worker's 8 KB
+  cap — and so were 333 bytes of `#set` doubling a letter inside a re-read config, which spell the name
+  at render time. §9.2 says the engine never throws on content; 0.7.0 threw on both.
+  The closing tag is found by a scan now, matching exactly what the pattern matched: any case, and the
+  two non-ASCII letters `iu` folds into ASCII (U+017F, U+212A). Found by the scaling probe behind the
+  note on template scans below.
+- **`constructor` and `__proto__` are ordinary variable names.** The variable and definition maps were
+  plain JavaScript objects, so both names found `Object`'s own members, and assigning `__proto__` stored
+  nothing. `render('a %constructor% b')` threw `TypeError` — §9.2 lets `render()` throw only on a resolver
+  or a foreign `Ast` — while `validate()` called the name merely undefined, so nothing could screen it out.
+  `%__proto__%` rendered as nothing, `{?constructor?…}` and `{?__proto__?…}` took the then-branch, a
+  `#set`, a `#def` or a context value under `__proto__` was lost, a runtime list under it spliced
+  `[object Object]`, and the validator withheld `plural.arity`, `plural.count-macro` and
+  `plural.nested-brackets` for a `#set` of either name — valid where both PHP validators say invalid.
+  Both PHP engines read variables from arrays and always treated them as ordinary names. The maps have
+  no prototype now (`internal/name-map.ts`). Found by the Codex gate on the nesting rewrite, where
+  `%__proto__%` had left the expansion budget `NaN`.
+
+### Changed, visibly
+
+- A no-break space behaves like a space around punctuation: removed before `,.;:!?`, no second space
+  inserted after one, and the next sentence capitalized through it. French typography with U+202F
+  before `!` or `?` loses it — as a plain `bonjour !` always has, in every engine.
+- A glued Cyrillic sentence stays glued: `конец.Начало` matches the bare-domain shield in PHP and now
+  here, exactly as `compact.Game` always has in ASCII. Whether the shield should take a capitalized
+  TLD at all is #79.
+
+### Corpus
+
+Twenty fixtures, every expectation taken from BOTH PHP engines (docker, real code): fifteen in
+`render-postprocess.json` — Cyrillic multi-dot abbreviations, an IDN domain, email and TLD, NBSP before
+and after punctuation and after a whitelisted abbreviation, a URL that ends at NBSP, an Arabic-Indic
+digit, NEL as whitespace and U+FEFF as not, a glued Cyrillic sentence, `_` before an abbreviation — plus
+`conditional/bom-only-is-truthy`, `conditional/nel-only-is-falsy`,
+`perm/config-nbsp-is-not-config-whitespace` and its two `validate/*` twins. Nineteen fail on 0.7.0; the
+twentieth is a negative guard. No earlier fixture put a non-ASCII character next to a boundary or a
+whitespace class — every domain, email and multi-dot case was Latin — which is how a premise stated in
+a comment stayed green for two months. `postprocess.test.ts` had pinned the mangled `т.д.` "in both
+engines", with an assertion that passed on the capital letter alone.
+
+Seven more `splice/*` fixtures for #80, again from both PHP engines: a size and an unquoted separator
+taken from a variable, a taken branch carrying a pipe in `[…]` and in `{…}`, a flag-gated list item,
+a branch trimmed at an element's edge — six fail on 0.7.0 — and the negative
+`splice/conditional-without-pipes-keeps-draws`, which pins that a conditional changing nothing
+structural draws exactly where 0.7.0 drew. Three `perm/*` fixtures for an element that renders empty or
+padded: `perm/nested-empty-option-drops-element`, `perm/nested-option-edge-whitespace-trimmed` and
+`perm/dropped-element-narrows-the-size-range`, all failing on 0.7.0. Four from the review, all failing on
+0.7.0: `splice/conditional-in-per-element-separator`, `splice/conditional-in-config-lastsep`, and the
+separator of a dropped element — `perm/dropped-element-takes-its-separator` (`[a<1>|{x|}|b]` is `a b`)
+and `perm/dropped-element-passes-on-its-trailing-separator` (`[a<1>|{x|}<2>|b]` is `a2b`).
+
+**The corpus can pin how many** (#74). `validate` cases take an optional `diagnosticCount` — the exact
+number of diagnostics per code — where the subset match used to be the only assertion, and that
+subset match is what passed two million circular-reference diagnostics for eleven days (#59). Two
+cases carry it: `validate/cycle-diamond-terminates` (22, one per name) and the new
+`validate/plural-count-macro-per-reference`, which records #73's decision: `plural.count-macro` once
+per tainted reference in the count slot, as this engine, `spintax-core` and both PHP validators already
+emit it. No engine output changed for either.
+
+Twelve fixtures for the prototype names, every expectation taken from both PHP engines: eight in
+`render-semantics.json` — undefined references, conditionals, a `#set` of each name, context keys, a
+`#def` rolled once, a `#def` naming them undefined, a runtime list spliced into a permutation, a form
+slot — and four in `validate.json`: the three withheld plural verdicts and the negative
+`validate/prototype-names-defined-and-undefined`. Eleven fail on 0.7.0. A JSON-decoded context carries
+`__proto__` as an ordinary key in every engine's harness; a JavaScript object literal would not.
+
+### Notes
+
+**Cost.** Against 0.7.0, the post-process of this whole batch is faster on prose and on HTML blocks and
+about a third slower on shield-heavy text — V8 runs a Unicode-class lookbehind more slowly than its ASCII
+`\b`, and every shield pattern carries one now. Per 1 000 calls, the four builds side by side:
+
+| input | 0.7.0 | the classes alone (regexes) | first linear scanners | now |
+|---|---|---|---|---|
+| prose, 857 characters | 34 ms | 32 ms | 108 ms | 19 ms |
+| HTML blocks, about 700 characters | 33 ms | 31 ms | 86 ms | 27 ms |
+| a URL, email, domain, decimal and abbreviation line, 237 characters | 36 ms | 46 ms | 70 ms | 49 ms |
+| the same line ×3 200, 756 KB, once | 113 ms | 151 ms | 223 ms | 158 ms |
+
+**The post-process is linear — and was a live denial of service in every release before this.** Eight
+of its passes were global regex replaces that a long run made retry from every start inside it: the
+email shield (one long word), the bare-domain shield (a dotted chain in any script), the
+trailing-punctuation cut of a URL, the space after a run of sentence marks, and the three capitalizers
+after a sentence end, a block tag and a line break (an unclosed `<`, a run of `<p>`). None of those
+runs needs a long template: 336 bytes of `#set` doubling one letter expand to 131 000 characters, whose
+post-process took 25 s, and the dotted, mark and tag units ran longer — so any host rendering untrusted
+templates with post-process on, the reference Worker and the hosted MCP server among them, could be held
+for as long as its CPU limit allowed. The expansion budget bounded the size of that text, never the time
+to post-process it. One more stage was quadratic on its own terms (found by the Codex gate): the restore
+of a text that carries U+0000, where #54 keeps the reference loop's reading — one `split`/`join` per
+placeholder — so a few hundred bytes doubling a NUL and a decimal took 2.1 s at 16 000 decimals, four
+times that per doubling.
+
+The shields and the capitalizers are scanners now. They run the plugin's own patterns at the same starts,
+in the same order, and skip only starts that provably fail. Every start inside one run of email-local
+characters meets the same `@` or none, so the email shield works back from each `@` — a text without one
+costs nothing. An attempt that fails at the start of a chain of labels fails at every later start in that
+chain, because the chain's own labels prefix any match further in, so the domain shield is one regex that
+matches at the starts it tries and consumes the chain when there is no domain — and is skipped when no dot
+in the text is followed by a label's first character. A lead has exactly one tokenization — a tag ends at
+the first `>` — so the capitalizers find their boundaries natively, walk a short lead, and build an index of
+where each lead ends, from the right, only when a lead holds a tag or runs past 32 characters; one index
+serves all three passes unless a capital grew the text (`ß` → `SS`). The URL cut is a loop, and the
+sentence-mark space starts a match only where its run starts. Output is byte-identical: 7.9 million strings
+against the regex implementation — exhaustive over small alphabets aimed at each pass, plus random soup — and
+5.8 million more aimed at the final form of each scanner, with every real control mutation caught, and the
+20 000-input post-process differential against both PHP engines unchanged at zero. Those macro templates now
+render half a megabyte to a megabyte in 0.1–0.2 s.
+
+The first cut of these scanners paid for linearity on every call: a regex call per word, a JavaScript step
+per character, three times — 3.4 times the regex passes on a short paragraph (the table above). The scaling
+bench, one large text, could not show that, and this note first said the bench was no slower on prose. The
+shapes above are what came after measuring a thousand short renders instead.
+
+The NUL-path restore computes the loop's result in one pass. An occurrence of a key is always a stretch
+of the original text between two `\x00`s — no stored value holds a `\x00`, and no whole stored value
+fits inside a key name — so the only way one replacement touches another is by taking a delimiter they
+share, and visiting the candidates in the loop's key order, each only while both its delimiters survive,
+is the loop. 1.9 million NUL-carrying strings come out identical, three control mutations were caught,
+and the 16 000-decimal case takes 130 ms.
+
+**The template scans are linear too — and part of that was live the same way.** They had the
+post-process's flaw. The parser matched each `{` and `[` by counting forward to its closer, which
+for an opener that never closes is the end of the text, from every such opener; the plural scan did the
+same for `{plural …}`, and the validator ran `/\[<([^>]*?)>/` from every `[<`. Lazy or overlapping
+patterns restarted from every character of a run in four more places: a `/# … #/` comment that never
+closes, a `#set`/`#def` value with a long whitespace run inside, a tag-shaped config holding a quoted
+`>`, and the validator's config-key test on a long word. The re-read of a construct hands the parser
+whatever its macros spell, so the parser's half was reachable from a few hundred bytes:
+
+| input (Node 22) | 0.7.0 | now |
+|---|---|---|
+| 333–341 bytes of `#set` spelling `[<`, `{?a?`, `[<li>`, `[<sep="` or `{plural 1:` inside `{…}` — render | 1.3–6.2 s, ×4 per doubling | 63–117 ms |
+| 387 bytes spelling form feeds into a tag-shaped config that holds a quoted `>` — render | 8.6 s, ×4 per doubling | 117 ms |
+| `[<` repeated, 32 KB — render / validate | 1.4 s / 0.27 s | 29 ms / 23 ms |
+| `{plural 1:` repeated, 32 KB — render / validate | 0.32 s / 0.20 s | 41 ms / 17 ms |
+| `/#a` repeated, 32 KB — parse / validate | 0.14 s / 0.14 s | 10 ms / 12 ms |
+| a `#set` value holding 32 KB of spaces — parse / validate | 2.8 s / 11.8 s | 4 ms / 14 ms |
+| `[<a`, 32 KB of spaces, a quoted `>` — parse | 2.0 s | 10 ms |
+| `[<` and a 32 KB word `>` — validate | 0.9 s | 10 ms |
+
+Brackets are paired once for the whole text, and the plural scan and the renderer's conditional pass use
+the same tables — the deep-nesting note below says how a construct is read now. The comment strip and the
+config scan are `indexOf` loops. The directive value ends at its last non-blank character instead of
+growing lazily, the tag pattern takes one whitespace character where two runs used to trade them, and a
+config key starts only where a word does — each matching exactly what it matched before. Output is
+byte-identical to the code before: 4.3 million generated strings through `parse`, `render` (three rng
+strategies, with and without post-process), `validate`, `analyze`, `extract` and the plural scan —
+exhaustive over alphabets aimed at each rewritten scan, plus random soup — with all twelve control
+mutations caught first.
+
+**Deep nesting no longer costs its depth again at every level — and the re-read had made that a live denial
+of service.** #68 kept nesting super-linear on the ground that depth costs source: at the hosted 8 KB cap,
+4 000 levels answered in about a second. 0.7.0's re-read took that ground away, because a construct re-read
+as text hands the parser whatever its macros spell (found by the Codex gate):
+
+| input (Node 22, the two measured side by side) | 0.7.0 | now |
+|---|---|---|
+| 705 bytes: `{%o15%x%c15%\|y}` over fifteen `#set` doublings of `{` and of `}` — render | 40 s | 0.14 s |
+| a `#def` rolling those 32 768 levels, referenced once — render | 40 s | 0.28 s |
+| `{a…x…b}` 16 384 levels deep, every level adding text — render | 19 s | 0.14 s |
+| `[a\|…x…]` 16 384 levels deep — render | 20 s | 0.24 s |
+| `{a%u%…x…}` 8 192 levels, each marked for the re-read by an undefined reference — render | 20 s | 0.10 s |
+| 20 000 nested conditionals naming a `#def` of 2^17 form feeds — render | 13 s | 0.30 s |
+| `{` 100 000 levels deep — parse | > 90 s | 0.21 s |
+
+Four costs were paid once per level, and each is gone:
+
+- **The parser read every construct's content in full** — to find its closer, to split it on top-level
+  pipes, to find a conditional's pipe, a config's end, a closing tag, a trailing separator — and handed its
+  children copies. A construct's children are spans of one text now, and every one of those questions is
+  answered for a span from tables built once over the text (`internal/text-index.ts`): bracket pairs; the
+  top-level pipes grouped by the brace and bracket totals in front of them, since `split_top_level`'s two
+  signed counters split exactly where both totals equal the ones at the span's start; a conditional's pipe
+  by jumping each opener to where one stack of both bracket kinds closes it, which is what the clamped
+  counter counts; the quote parity of every `>`; every closing tag by name; every `<` and `>`. Most of those
+  answers are a binary search in a sorted list of positions, so a template of n characters parses in
+  O(n log n) at worst rather than O(n) — the logarithm shows on many small siblings: 200 000 `{a|b}` parse in
+  150 ms against 0.7.0's 137, 100 000 `[<sep=",">a|b]` in 208 against 161 (found by the Codex gate).
+- **The walk joined each frame's output into a new string**, copying a construct's text once for every
+  level above it. Frames hand fragments up instead — joins, and windows for a trimmed permutation element,
+  every join knowing how many PHP trim characters sit at its ends — and the text is made once.
+- **A construct marked for the re-read ran both conditional passes and the fixpoint over its body to find
+  nothing to change**, once per marked level. The index answers first: the body can change only if it holds
+  a conditional the pass would resolve — a well-formed head whose brace closes inside it — or a reference
+  expansion would substitute, a name the variable map defines while the budget lasts. When neither, the
+  splice returns what it always returned. The conditional pass finds its heads once, too, instead of
+  searching the text again from the start of every span.
+- **Truthiness scanned a value's leading whitespace at every conditional**; it is computed once per
+  variable map.
+
+Output is byte-identical to the code before this change: the 4.3 million alphabet strings above again, and
+20 000 generated documents — long outputs, padded and empty elements, constructs marked for the re-read
+with defined, undefined and budget-starved references, conditionals in bodies, count slots and `#def`
+rolls, nests up to 300 deep — through eight probes each (`parse`, `validate`, `analyze`, four rng
+strategies with and without post-process, and a pre-parsed `Ast`), with the control mutations caught first:
+a join that stops counting trim characters after its first piece, one that never counts them at its end, a
+trim window a character late, a re-read check that forgets conditionals or inverted heads or compares names
+case-sensitively, one truthiness cache for every map, a conditional pass that skips a head at a span's start,
+top-level pipes grouped by one depth, a flipped quote parity, a closing tag on the span's end counted
+outside it. The PHP engines are no linear reference here: both resolve innermost-first with a
+full-text pass per level and stop at 10 000 levels with an exception — in `spintax/core`, 8 192 nested
+permutations took 5.9 s and 32 768 nested enumerations threw. Real templates did not pay for any of this —
+the render cost note below has the numbers. An `Ast` copied out of the process (it is not a serialization
+format) loses the side table the index lives in, and renders the same text by reading bodies as before.
+
+**Two costs bounded by the source, recorded rather than changed.** The same gate found `validate()`
+formatting a large definition cycle in quadratic time — each name walks the cycle to count what its
+printed route leaves out: 8 000 names 6.5 s, the 320 that fit in 8 KB 23 ms — and `render()` ordering
+`#def`s in quadratic-to-cubic time: 1 600 in a chain 2 s, 400 in 90 ms. A macro cannot spell either,
+both were so in 0.7.0, and the hosted surfaces cap a source at 8 KB.
+
+**Two PHP builds differ at the margin.** PCRE2 10.43 made non-spacing marks and connector punctuation
+word characters, so PHP 8.3 sees a boundary between `x` and U+0301 that PHP 8.4 does not. The corpus
+runs PHP 8.4 and this engine follows it; recorded in the conformance README.
+
+**What the wider re-read does not move.** A construct re-read because it holds a conditional draws
+exactly as the tree did whenever the taken branch changes nothing structural — measured on 0.7.0
+before the change, 500 seeds over five shapes (a permutation with separators, one with a size range,
+an enumeration, a nested permutation in a branch, an else branch): identical renders. What changes is
+the three kinds of text the plugin always produced here — a pipe in a branch, an empty element, a
+trimmed edge — and only in the renders where one occurs: a permutation whose elements all render
+non-empty and unpadded picks and shuffles exactly as before. `validate()` still reports `minsize=%n%`
+as `permutation.minsize-not-integer`, as both PHP validators do: a verdict about the template as
+written, unchanged.
+
+**Verified against PHP, not only against 0.7.0.** A generator of construct-heavy templates —
+conditionals with empty, piped and padded branches, references in configs and elements, macros
+carrying conditionals, per-element separators — 3 000 renders with `rng: first` and `last` through
+both PHP engines: 0.7.0 differed on 416, this engine on 0. The first cut of #80 still differed on
+147, every one an element that rendered empty through a nested construct; that is how the trim-and-drop
+above was found. And 20 000 generated post-process inputs (Latin, Cyrillic, marks, every space the two
+dialects disagree on, shield triggers): 0.7.0 differed on 6 229, this engine on none but the recorded
+final `trim`, which accounts for all 3 689 of its edge-only differences. A zero holds for what those
+generators can build, and is stated here as that.
+
+**On a production host's templates.** 2 522 template-and-flag combinations taken from a live
+deployment's migrations, 15 132 renders against 0.7.0: 30 change, and `validate`/`analyze`/`extract`
+change on none. They are the two defects, in real copy: Russian brand articles stop printing
+`Т. Е. Каждый перевод…` and `…х5, х7 и т. Д.`, and a payment FAQ whose list gates one item behind
+`{?CasinoHasCrypto?…|}` stops printing `…are offered.. Traditional methods…` for a casino without
+crypto — the blank element and its `. ` separator, which post-process had turned into a double stop.
+The same generated documents without any of the changed shapes — 600 of them, 3 600 renders plus
+`validate`/`analyze`/`extract` — are byte-identical, and the harness caught all three deliberate control
+mutations first.
+
+**Render cost.** A construct re-read because it holds a conditional is parsed again on every render, so
+the payment FAQ whose list gates one item behind a flag is the one real template that got slower: 1 000
+renders take 120 ms against 0.7.0's 86. Everything else measured got faster once the index answered most
+re-reads without reading, the walk stopped copying and the post-process lost its per-call overhead: the FAQ
+with a providers gate four times over, 513 ms against 1 322; plain prose, 145 against 198; all 1 262
+construct-bearing literals of the production host's migrations, rendered once each, 178 ms against 262.
+
+**The prototype names, verified against a renamed oracle.** 0.7.0's output under those names was the
+defect, so the fixed build is compared with the build before it on each document with `__proto__` and
+`constructor` renamed to ordinary names of the same length, renamed back in the output: 20 000 generated
+documents, 19 875 carrying one of the names in a reference, a conditional, a `#set`, a `#def`, a plural
+slot or a context key, through `parse`, `validate`, `analyze` and five renders — no difference, with three
+control mutations caught first (an unrelated truthiness change, and the alias map or the `#def` map left a
+plain object). The 449 009 alphabet strings, which name neither, are unchanged. The maps are merged by a
+loop: `Object.assign` into a map with no prototype took five times the spread it replaced, once per
+`#def` — 1 000 renders of a template with 60 context variables and 20 definitions went from 920 ms to
+1 336 — and the loop brings that to 968. The production templates, the FAQ and prose measure as before.
+
+**Recorded, not closed** — in the conformance README, under the known divergences. A value carrying an
+unbalanced bracket (`[a|{%L%}]` with `L = "x}|y"`) re-cuts the enclosing construct in PHP and only its
+own here. Four shapes of the same family, where a nested pick leaves markup the permutation's reader
+sees in PHP and a tree parsed before the pick: a leading element that renders empty exposes its
+`<…>` as the permutation's config, a pick ending in `<…>` becomes a per-element separator, a
+construct inside the config is resolved first, and a nested permutation joined by `|` re-splits the
+outer one (`[[<|>a|b]|c]`). All four were already so in 0.7.0. A generated differential aimed at exactly these meets
+the first twice in 3 000 renders. And the Unicode tables: Node 22's are 17.0, PCRE2 10.44's are 15.0,
+so a character assigned since then can sit on the other side of a boundary.
+
+## 0.7.0 — 2026-09-12
+
+**A `%variable%` written directly inside `{…}` or `[…]` is now spliced as TEXT before the construct
+is split** — so a pipe-joined value is a list of options or elements, as it has always been in the
+PHP engines. Reported from production: a brand preset
+`[<minsize=5;maxsize=7;sep=", ";lastsep=" and ">%CasinoProvidersList%]` over a 57-name runtime
+list rendered all 57 names joined with `|` — no size pick, no shuffle, no separators — in 131
+published rows across 15 tenants. Minor rather than patch: rendered text changes for every template
+of that shape, and `AST_VERSION` moves.
+
+### Fixed
+
+- **The defect.** The tree is built before any value exists, so `[<…>%list%]` was ONE element
+  holding a variable node; `resolveVariable` handed a construct-free value back as finished text,
+  and the `|` that separates elements in every PHP engine — whose `expand_variables` runs over the
+  whole text before any bracket is read — was never seen. Same for `{%list%}`, for `[a|%list%|b]`,
+  and for a `#set` or `#def` wrapping the construct.
+- **The rule, restated for a tree walk.** The parser keeps a construct's raw body when it holds a
+  *direct* reference — at the top level of an option, inside a conditional's branches (the plugin
+  resolves `{?…}` before it expands, Stage 6a), or in a separator string (`<sep="%S%">`,
+  `a <%S%> |`). At render time such a construct is re-read from its expanded text in the plugin's
+  own order: conditionals → expansion → conditionals → parse. Every other construct keeps the tree
+  it was parsed into, and with it the RNG order the corpus pins; a spliced value with no structural
+  characters re-reads to the same tree, so those renders are unchanged too.
+- **The hop budget inside a bracket equals the one outside.** The textual fixpoint runs the
+  plugin's own `<= MAX_VARIABLE_DEPTH` — 51 passes, minus the hops a macro re-parse already spent —
+  and whatever it leaves is frozen for the whole subtree, so the plugin's 51 hops hold in every
+  shape: the mutual cycle leaves `%b%`, `#set %b% = x%b%y` leaves 51 pairs, a 51-deep chain into
+  `x|y` reaches the body as text and is split — inside `{…}` as at top level (#57's pins, moved
+  inside). **The plural slots now use the same arithmetic.** They ran a flat 50 passes and rendered
+  the picked form unfrozen, so a 51-deep alias chain in the count slot erased a block the plugin
+  renders, and a 52-deep chain in a form resolved to its end where the plugin leaves `%a52%`.
+  Pre-existing, found by the review gate, closed here.
+
+### Changed
+
+- `AST_VERSION` 2 → 3: `EnumerationNode` and `PermutationNode` gained `raw`. A handle from an
+  older parser would render the old, wrong output, so the version guard turns it into
+  `AstVersionError` — the same reason `#def` bumped it. Nothing persists a handle across versions.
+- **Every substitution now charges the expansion budget (#69), as it does in PHP.**
+  `resolveVariable` used to charge only construct-bearing values — harmless while a plain value
+  could only be a leaf, and the one door left open once a re-read construct could hand it
+  references its own fixpoint had cut off: 2^12 references to a 1 KiB value produced 4 MiB out of
+  a 1 MiB allowance, and 2^20 is an out-of-memory abort. Found by the review gate before release.
+  A template referencing a 65 KB plain list sixteen times now reaches the budget, as it already
+  did in PHP; the references past it stay literal.
+- **The truncated shape of an expansion bomb is half what it was.** The substitution at the depth
+  cap is charged like every other now (it was free), so the 62-character bomb renders 0.57 MB where it
+  rendered 1.14 MB. Not parity-gated — the conformance README says what a truncated explosion looks
+  like is engine-specific — and the contract holds unchanged: terminates, never throws, bounded output,
+  unaffordable references left literal. The hosted MCP server's output cap now stops that bomb at
+  variant 4 rather than 2 (its test moved with it).
+- `neutralize()` docs: the pipe is deliberately not shielded. A neutralized value the author places
+  inside `{…}`/`[…]` is split on its `|` in every engine — this one was immune only by the defect.
+
+### Corpus
+
+Nineteen `splice/*` fixtures in `render-semantics.json`, expected outputs taken from BOTH PHP
+engines (docker, real code): the inline permutation, the `#set` wrapper over a runtime list,
+literals around the variable, `{%L%}`, a conditional branch carrying the list, a `#def` wrapper
+(rolled once, then spliced), `<sep="%S%">`, `lastsep="%S%"` and a per-element `<%S%>`, a macro
+value with brackets AND a pipe, three hop-budget pins (a 50-alias chain into `x|y` split on the 51st
+hop inside `{…}`, the same chain reaching its number in a plural count slot, a 51-alias chain left
+literal in a form), the production
+preset shape — and three negatives: a top-level reference is NOT split, an undefined name stays one
+literal element, a nested construct splits at its own level only. No prior fixture had a variable
+inside a construct, which is how this hid in four tree-walk engines (TS, Python, Object Pascal,
+.NET) while the PHP engines were right all along. The siblings fail the new cases until they mirror
+the rule.
+
+### Notes
+
+**What stays as it was, on purpose.** Two pre-existing tree-walk divergences in the *non-triggered*
+path are unchanged: an element that renders to empty (a nested conditional yielding `''`) is kept
+here and dropped by PHP, and PHP's top-level scan splits the fullwidth plural fallback on its pipes.
+Both pathological, neither reported. A *triggered* construct drops the empty element as PHP does,
+because the text is what PHP sees — and trims a conditional's taken branch at an element's edge,
+as the plugin trims after Stage 6a. The fullwidth fallback is the exception in BOTH paths: the plugin
+resolves plurals before it reads a bracket, so a fallback's ASCII pipes reach the enclosing construct
+there, while here the re-read makes the plural a node again and it renders whole. Only a template
+`validate()` already rejects (`plural.arity`, `plural.nested-brackets`) can reach it; recorded in the
+conformance README under the known divergences rather than closed (review finding, kept open on
+purpose).
+
+**The hop budget, corrected in review.** The first cut gave the fixpoint 50 passes and rendered the
+leftover at the depth cap, which spliced it once more as finished text: a 52nd hop when reached through
+a macro, and one that hid a structural terminal value from the split (a 51-deep chain into `x|y`
+inside `{…}` rendered `x|y`, where the plugin's 51st pass yields `{x|y}` and picks). Now the fixpoint
+runs the plugin's own `<= MAX_VARIABLE_DEPTH` — 51 passes, minus the hops a macro re-parse already
+spent — and whatever is left is frozen for the whole subtree.
+
+**Verified.** The 0.6.1 build against this one over generated documents (seeded LCG, every
+construct, `render` in both post-process modes, `analyze`, `extract`) — the harness first shown to
+catch two deliberate control mutations. Documents with no direct reference anywhere: byte-identical,
+and `analyze`/`extract` identical on every document. Documents whose constructs DO carry a direct
+reference, with plain values and no conditional: byte-identical over 1 788 renders and three seeds —
+that is the claim that a spliced value with no structural characters re-reads to the same tree. With
+conditionals and construct-bearing definition values allowed, 159 of 922 triggered renders differ, in
+exactly the three ways the PHP text has always differed from the old tree: whitespace only (a taken
+branch trimmed at an element's edge), an element that became empty dropped before the shuffle (so
+every later draw shifts), or a `|` carried in by a rolled `#def`.
+
 ## 0.6.1 — 2026-08-19
 
 **The engine no longer throws on deeply nested content** (issue #68). `parse()`, `render()` and

package/README.md

@@ -14,11 +14,12 @@ parse, render, validate, extract, analyze, and neutralize spintax templates.
 
 - **Zero runtime dependencies.** Runs unchanged on Cloudflare Workers, Node 18+, and in the browser.
 - **ESM-first, dual CJS.** Ships `.d.ts` types for both.
-- **One syntax contract, [five independent engines](https://spintax.net/spintax-engines/).**
+- **One syntax contract, [six independent engines](https://spintax.net/spintax-engines/).**
   This package, [`spintax/core`](https://packagist.org/packages/spintax/core) (PHP),
-  [`spintax-core`](https://pypi.org/project/spintax-core/) (Python), an Object Pascal engine, and
+  [`spintax-core`](https://pypi.org/project/spintax-core/) (Python), an Object Pascal engine,
+  [`Spintax.Core`](https://www.nuget.org/packages/Spintax.Core/) (.NET), and
   the [Spintax WordPress plugin](https://wordpress.org/plugins/spintax/) are all held to the same
-  behavior contract by a **shared golden corpus of 234 fixtures** — deterministic verdicts, plural
+  behavior contract by a **shared golden corpus of 258 fixtures** — deterministic verdicts, plural
   buckets, conditionals, `#set`/`#def` semantics and post-processing agree everywhere, so a
   template is an asset you can move between runtimes, not a lock-in.
 - **MIT** licensed.
@@ -80,10 +81,10 @@ dedupe in the host and cap the retries: a template may simply not have N combina
 | --- | --- | --- |
 | Enumeration | `{a\|b\|c}` | pick one (nestable: `{a\|{b\|c}}`) |
 | Permutation | `[a\|b\|c]` | pick N, shuffle, join — `[<minsize=1;maxsize=2;sep=", ">a\|b\|c]` |
-| Variable | `%var%` | substitute a context value |
+| Variable | `%var%` | substitute a context value — inside `{…}`/`[…]` the value is spliced as text first, so a `|` it carries separates options |
 | Local set | `#set %v% = value` | define a macro — re-picked at every use |
 | Local def | `#def %v% = value` | define a value — picked once per render, held at every use |
-| Conditional | `{?VAR?then\|else}` | `then` if `VAR` is truthy, else `else` |
+| Conditional | `{?VAR?then\|else}` | `then` if `VAR` is truthy, else `else` — inside `{…}`/`[…]` it resolves first, so a `\|` in the taken branch separates options and an empty one drops its element |
 | Plural | `{plural %n%: one\|few\|many}` | grammatical agreement by locale |
 | Include | `#include "slug-or-id"` | embed another template (host-resolved) |
 | Comment | `/# … #/` | stripped before rendering |
@@ -153,6 +154,11 @@ render('%bio%', { context: { bio: neutralize('Save {50|60}% today') }, postProce
 // → "Save {50|60}% today"   (the braces stay literal, not a random pick)
 ```
 
+The pipe is deliberately **not** shielded: it means nothing outside a construct, and inside one it
+is the separator. A neutralized value an author places in `{…}`/`[…]` is still split on its `|`, in
+this engine as in every other in the family. Keep such a value out of a construct if it must stay
+one option.
+
 ## Using it without writing code
 
 - **n8n** — install [`n8n-nodes-spintax`](https://www.npmjs.com/package/n8n-nodes-spintax)