@spintax/core 0.5.3 → 0.6.1

package/CHANGELOG.md

@@ -3,6 +3,92 @@
 All notable changes to `@spintax/core` are documented here. This project adheres to
 [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## 0.6.1 — 2026-08-19
+
+**The engine no longer throws on deeply nested content** (issue #68). `parse()`, `render()` and
+`analyze()` raised `RangeError: Maximum call stack size exceeded` at about 2000 levels of nesting —
+a **3.9 KB** template — while §9.2 promises a bad construct comes back as text with fullwidth
+braces, never as an exception. No other engine in the family threw.
+
+### Fixed
+
+Three walks recursed once per level of nesting, and each is a frame stack now:
+
+- `parseSequence` — a construct returns a *plan* (its child texts, and how to assemble the node)
+  and a stack drives it. The shape mirrors the Python port's `_plan_*` functions, written that way
+  from the start for this reason.
+- `renderNodes` — the same. **The RNG order is contract and did not move**: an enumeration picks
+  before descending, so an unpicked branch never consumes RNG; a permutation renders every element
+  first and consumes its own picks after. Seeded renders are reproducible within the engine (§3.2)
+  and corpus fixtures pin exact picks with `rng: { sequence }`.
+- `walk` (`ast.ts`) — pre-order, children in source order, because `analyze`'s `constructs` counts
+  and `refs` order come out of it.
+
+Fixing only the parser would have moved the wall rather than removed it: with parsing iterative,
+`render` and `analyze` still threw at ~5000. All three now return at 2 000, 9 000 and 50 000 levels.
+
+`validate()` and `extract()` never threw, which made this worse than it looked — a caller could not
+pre-screen for it by validating first.
+
+### Notes
+
+**Output is byte-identical.** 400 generated documents covering every construct, dumped across
+parse / extract / validate×2 / analyze / render×2 / neutralize before and after — no difference.
+The harness itself was corrected first: of three deliberate control mutations it caught two and
+missed one, because the generator never emitted the per-element `[a < or > | b]` separator — which
+is precisely the code this release moved. A green that cannot see the changed line is not evidence.
+
+**Deep nesting is still super-linear**, in this engine and in the family: 9 000 levels cost seconds
+here where the PHP engine costs 382 ms, because `findMatchingClose`, the inner slice and
+`splitTopLevel` each rescan a construct's content once per level. Removing that means threading
+spans through five helpers whose separator rules deliberately differ from one another, so it is
+recorded rather than guessed at — the same call as #71. Bounding input remains a host job (§9.3);
+the reference Worker caps a source at 8192 characters.
+
+`AST_VERSION` is unchanged — the node shape did not move.
+
+## 0.6.0 — 2026-08-18
+
+**`validate()` now emits ONE `variable.circular-reference` per NAME that takes part in, or leads
+to, a cycle** — it used to emit one per PATH (issue #59). Minor rather than patch: diagnostic
+output visibly changes, though no verdict moves and no API does.
+
+### Fixed
+
+Per-path emission is exponential on a converging diamond, because the number of routes into a
+cycle is exponential in the diamond's depth:
+
+| shape | template | before | after |
+|---|---|---|---|
+| 20 definitions, each referencing the previous one twice, feeding a 2-cycle | **507 B** | **2 097 152 diagnostics, 5.9 s** | **22 diagnostics, 6 ms** |
+| the same at depth 200 | 5 KB | (never finished) | 202 diagnostics, ms |
+| live `POST /validate-template` | **547 B** | **HTTP 503** | 200, with a verdict |
+
+The second half of the same issue: one cycle of N names printed an N-name route in each of N
+messages. A 43 KB template of one giant cycle carried **29 MB of message text**; the route is now
+capped at 8 names and becomes a count — `n0 → n1 → … → n7 → … (1992 more)` — which brings that to
+200 KB. A real cycle is two to five names and reads exactly as it did.
+
+**Messages are unchanged for every shape a human writes.** The colour walk that already existed as
+a prune now also records one witness edge per name, and following those edges reproduces the route
+the per-path walk used to print: `a → b → a`, `d0 → c1 → c2 → c1`. Only the duplicates are gone.
+`Diagnostic.data` carries `{ name }` so a consumer can group without parsing the message.
+
+### Notes
+
+This reverses a deliberate decision, which is why it is recorded rather than quietly shipped. The
+per-path shape was pinned on purpose in three engines' own suites — "an implementation that
+deduplicates repeated edges, memoizes per root, or reorders roots goes red here, not silently
+green" — and `spintax-win` aligned to it on 2026-08-07. It could not be kept and bounded: re-walking
+every route *is* the emission. `spintax-core` (Python) already emitted per name and was the one
+engine immune; the rest followed it, and its counts are now the reference — verified by diffing 400
+generated definition graphs against it, and against `spintax/core`, with zero mismatches.
+
+Mirrored into `spintax/core` 0.8.0 and the WordPress plugin. The corpus gains a fixture for the
+diamond whose real assertion is that the engine *answers*; it cannot gate the count, because
+diagnostics are asserted as a subset — and `packages/conformance/README.md` now says that
+multiplicity is deliberately not part of the contract.
+
 ## 0.5.3 — 2026-08-18
 
 The expansion budget added in 0.5.2 was created per rendered AST, and an `#include` renders one

package/dist/index.cjs

@@ -3,19 +3,32 @@
 // src/internal/ast.ts
 var AST_VERSION = 2;
 function walk(nodes, visit) {
-  for (const n of nodes) {
+  const stack = [{ list: nodes, i: 0 }];
+  while (stack.length > 0) {
+    const top = stack[stack.length - 1];
+    if (top.i >= top.list.length) {
+      stack.pop();
+      continue;
+    }
+    const n = top.list[top.i];
+    top.i += 1;
     visit(n);
+    let children;
     switch (n.type) {
       case "enumeration":
-        for (const opt of n.options) walk(opt, visit);
+        children = n.options;
         break;
       case "conditional":
-        walk(n.then, visit);
-        walk(n.else, visit);
+        children = [n.then, n.else];
         break;
       case "permutation":
-        for (const opt of n.options) walk(opt.nodes, visit);
+        children = n.options.map((opt) => opt.nodes);
         break;
+      default:
+        continue;
+    }
+    for (let k = children.length - 1; k >= 0; k -= 1) {
+      stack.push({ list: children[k], i: 0 });
     }
   }
 }
@@ -92,65 +105,102 @@ function stripComments(text) {
   return text.replace(/\/#[\s\S]*?#\//g, "");
 }
 function parseSequence(text) {
-  const nodes = [];
-  let literal = "";
-  let i = 0;
-  const flushLiteral = () => {
-    if (literal !== "") {
-      nodes.push({ type: "literal", value: literal });
-      literal = "";
-    }
-  };
-  while (i < text.length) {
-    const ch = text.charAt(i);
-    if (ch === "{") {
-      const end = findMatchingClose(text, i, "{", "}");
-      if (end === -1) {
-        literal += ch;
-        i += 1;
+  const frame = (t) => ({ text: t, i: 0, literal: "", nodes: [], plan: null, parts: [] });
+  const stack = [frame(text)];
+  while (stack.length > 0) {
+    const f = stack[stack.length - 1];
+    if (f.plan !== null) {
+      if (f.parts.length < f.plan.texts.length) {
+        stack.push(frame(f.plan.texts[f.parts.length]));
         continue;
       }
-      const content = text.slice(i + 1, end);
-      flushLiteral();
-      nodes.push(parseBraceConstruct(content));
-      i = end + 1;
+      f.nodes.push(f.plan.build(f.parts));
+      f.plan = null;
+      f.parts = [];
       continue;
     }
-    if (ch === "[") {
-      const end = findMatchingClose(text, i, "[", "]");
-      if (end === -1) {
-        literal += ch;
-        i += 1;
-        continue;
+    const flushLiteral = () => {
+      if (f.literal !== "") {
+        f.nodes.push({ type: "literal", value: f.literal });
+        f.literal = "";
       }
-      flushLiteral();
-      nodes.push(parsePermutation(text.slice(i + 1, end)));
-      i = end + 1;
-      continue;
-    }
-    if (ch === "%") {
-      const name = VARIABLE_RE.exec(text.slice(i))?.[1];
-      if (name !== void 0) {
+    };
+    let planned = null;
+    while (f.i < f.text.length && planned === null) {
+      const ch = f.text.charAt(f.i);
+      if (ch === "{" || ch === "[") {
+        const close = ch === "{" ? "}" : "]";
+        const end = findMatchingClose(f.text, f.i, ch, close);
+        if (end === -1) {
+          f.literal += ch;
+          f.i += 1;
+          continue;
+        }
+        const inner = f.text.slice(f.i + 1, end);
         flushLiteral();
-        nodes.push({ type: "variable", name });
-        i += name.length + 2;
+        planned = ch === "{" ? planBraceConstruct(inner) : planPermutation(inner);
+        f.i = end + 1;
         continue;
       }
+      if (ch === "%") {
+        const name = VARIABLE_RE.exec(f.text.slice(f.i))?.[1];
+        if (name !== void 0) {
+          flushLiteral();
+          f.nodes.push({ type: "variable", name });
+          f.i += name.length + 2;
+          continue;
+        }
+      }
+      f.literal += ch;
+      f.i += 1;
+    }
+    if (planned !== null) {
+      if ("node" in planned) f.nodes.push(planned.node);
+      else f.plan = planned;
+      continue;
     }
-    literal += ch;
-    i += 1;
+    flushLiteral();
+    stack.pop();
+    const parent = stack[stack.length - 1];
+    if (parent !== void 0) parent.parts.push(f.nodes);
+    else return f.nodes;
   }
-  flushLiteral();
-  return nodes;
+  return [];
 }
-function parseBraceConstruct(content) {
+function planBraceConstruct(content) {
   if (content.charAt(0) === "?") {
-    const cond = tryParseConditional(content);
-    if (cond) return cond;
+    const parts = splitConditional(content);
+    if (parts !== null) {
+      return {
+        texts: [parts.thenRaw, parts.elseRaw],
+        build: (children) => ({
+          type: "conditional",
+          name: parts.name,
+          inverted: parts.inverted,
+          then: children[0] ?? [],
+          else: children[1] ?? []
+        })
+      };
+    }
   } else if (content.startsWith(PLURAL_PREFIX) && content.slice(PLURAL_PREFIX.length).includes(":")) {
-    return parsePlural(content.slice(PLURAL_PREFIX.length));
+    return { node: parsePlural(content.slice(PLURAL_PREFIX.length)) };
   }
-  return { type: "enumeration", options: splitTopLevel(content).map((o) => parseSequence(o)) };
+  return {
+    texts: splitTopLevel(content),
+    build: (children) => ({ type: "enumeration", options: children })
+  };
+}
+function planPermutation(rawInner) {
+  const { config, content } = extractPermutationConfig(rawInner);
+  const { texts, separators } = permutationElements(splitTopLevel(content));
+  return {
+    texts,
+    build: (children) => ({
+      type: "permutation",
+      config,
+      options: children.map((nodes, i) => ({ nodes, separator: separators[i] ?? null }))
+    })
+  };
 }
 function recognizeConditional(text, contentStart, contentEnd) {
   let p = contentStart + 1;
@@ -179,17 +229,6 @@ function splitConditional(content) {
     elseRaw: sep < 0 ? "" : body.slice(sep + 1)
   };
 }
-function tryParseConditional(content) {
-  const parts = splitConditional(content);
-  if (parts === null) return null;
-  return {
-    type: "conditional",
-    name: parts.name,
-    inverted: parts.inverted,
-    then: parseSequence(parts.thenRaw),
-    else: parseSequence(parts.elseRaw)
-  };
-}
 function parsePlural(afterPrefix) {
   const colon = afterPrefix.indexOf(":");
   return { type: "plural", countRaw: afterPrefix.slice(0, colon), formsRaw: afterPrefix.slice(colon + 1) };
@@ -204,11 +243,6 @@ var PER_ELEM_HTML_RE = /^[a-zA-Z][a-zA-Z0-9]*\s/;
 function defaultPermConfig() {
   return { minsize: null, maxsize: null, sep: " ", lastsep: null };
 }
-function parsePermutation(rawInner) {
-  const { config, content } = extractPermutationConfig(rawInner);
-  const options = extractPerElementSeparators(splitTopLevel(content));
-  return { type: "permutation", config, options };
-}
 function extractPermutationConfig(content) {
   const trimmed = phpLtrim(content);
   if (trimmed === "" || trimmed.charAt(0) !== "<") {
@@ -252,8 +286,9 @@ function looksLikeHtmlStartTag(tagText, remaining) {
   const tagName = (m[1] ?? "").toLowerCase();
   return new RegExp(`</${escapeRegExp(tagName)}\\s*>`, "iu").test(remaining);
 }
-function extractPerElementSeparators(rawParts) {
-  const options = [];
+function permutationElements(rawParts) {
+  const texts = [];
+  const separators = [];
   let pendingSep = null;
   rawParts.forEach((part, i) => {
     let text = part;
@@ -267,11 +302,12 @@ function extractPerElementSeparators(rawParts) {
     }
     const trimmed = phpTrim(text);
     if (trimmed !== "") {
-      options.push({ nodes: parseSequence(trimmed), separator: pendingSep });
+      texts.push(trimmed);
+      separators.push(pendingSep);
     }
     pendingSep = trailingSep;
   });
-  return options;
+  return { texts, separators };
 }
 function extractTrailingSep(part) {
   const trimmed = phpRtrim(part);
@@ -750,9 +786,14 @@ function checkVariableReferences(text, idx, known, out) {
     for (const m of value.matchAll(/%(\w+)%/gu)) refs.push((m[1] ?? "").toLowerCase());
     refsOf.set(name, refs);
   }
-  const reachesCycle = namesThatReachACycle(defs, refsOf);
+  const { reaches: reachesCycle, via } = namesThatReachACycle(defs, refsOf);
   for (const name of defs.keys()) {
-    detectCycle(name, defs, refsOf, reachesCycle, defPos.get(name) ?? somewhere, out);
+    if (!reachesCycle.has(name)) continue;
+    out.push(err(
+      "variable.circular-reference",
+      `Circular variable reference: ${cyclePath(name, via)}.`,
+      { ...defPos.get(name) ?? somewhere, data: { name } }
+    ));
   }
   const body = text.replace(/^[ \t]*#(?:set|def)[ \t]+%\w+%[ \t]*=[ \t]*.*?$/gmu, (m) => m.replace(/[^\n]/g, " "));
   const seen = /* @__PURE__ */ new Set();
@@ -776,6 +817,12 @@ function namesThatReachACycle(defs, refsOf) {
   const BLACK = 2;
   const color = /* @__PURE__ */ new Map();
   const reaches = /* @__PURE__ */ new Set();
+  const via = /* @__PURE__ */ new Map();
+  const mark = (name, witness) => {
+    if (reaches.has(name)) return;
+    reaches.add(name);
+    via.set(name, witness);
+  };
   for (const root of defs.keys()) {
     if (color.has(root)) continue;
     const stack = [{ name: root, refs: refsOf.get(root) ?? [], i: 0 }];
@@ -787,9 +834,9 @@ function namesThatReachACycle(defs, refsOf) {
         top.i += 1;
         if (ref === top.name || !defs.has(ref)) continue;
         const c = color.get(ref);
-        if (c === GREY) reaches.add(top.name);
+        if (c === GREY) mark(top.name, ref);
         else if (c === BLACK) {
-          if (reaches.has(ref)) reaches.add(top.name);
+          if (reaches.has(ref)) mark(top.name, ref);
         } else {
           stack.push({ name: ref, refs: refsOf.get(ref) ?? [], i: 0 });
           color.set(ref, GREY);
@@ -797,42 +844,41 @@ function namesThatReachACycle(defs, refsOf) {
       } else {
         stack.pop();
         color.set(top.name, BLACK);
-        if (stack.length > 0 && reaches.has(top.name)) reaches.add(stack[stack.length - 1].name);
+        if (stack.length > 0 && reaches.has(top.name)) mark(stack[stack.length - 1].name, top.name);
       }
     }
   }
-  return reaches;
-}
-function detectCycle(root, defs, refsOf, reachesCycle, rootPos, out) {
-  if (!reachesCycle.has(root)) return;
-  const path = [root];
-  const onPath = /* @__PURE__ */ new Set([root]);
-  const stack = [{ name: root, refs: refsOf.get(root) ?? [], i: 0 }];
-  const leave = (frame) => {
-    stack.pop();
-    onPath.delete(frame.name);
-    path.pop();
-  };
-  while (stack.length > 0) {
-    const top = stack[stack.length - 1];
-    if (top.i >= top.refs.length) {
-      leave(top);
-      continue;
+  return { reaches, via };
+}
+var CYCLE_PATH_LIMIT = 8;
+function cyclePath(name, via) {
+  const seen = /* @__PURE__ */ new Set([name]);
+  const shown = [name];
+  let current = name;
+  for (; ; ) {
+    const next = via.get(current);
+    if (next === void 0) break;
+    if (seen.has(next)) {
+      shown.push(next);
+      return shown.join(" \u2192 ");
     }
-    const ref = top.refs[top.i];
-    top.i += 1;
-    if (ref === top.name) continue;
-    if (onPath.has(ref)) {
-      out.push(err("variable.circular-reference", `Circular variable reference: ${[...path, ref].join(" \u2192 ")}.`, rootPos));
-      leave(top);
-      continue;
-    }
-    if (defs.has(ref) && reachesCycle.has(ref)) {
-      stack.push({ name: ref, refs: refsOf.get(ref) ?? [], i: 0 });
-      onPath.add(ref);
-      path.push(ref);
+    if (shown.length >= CYCLE_PATH_LIMIT) {
+      let more = 0;
+      let walk2 = next;
+      while (!seen.has(walk2)) {
+        seen.add(walk2);
+        more += 1;
+        const step = via.get(walk2);
+        if (step === void 0) break;
+        walk2 = step;
+      }
+      return `${shown.join(" \u2192 ")} \u2192 \u2026 (${more} more)`;
     }
+    seen.add(next);
+    shown.push(next);
+    current = next;
   }
+  return shown.join(" \u2192 ");
 }
 function checkIncludeTargets(text, idx, known, out) {
   const set = new Set(known);
@@ -1006,9 +1052,34 @@ function directReferences(text) {
   return [...text.matchAll(/%(\w+)%/gu)].map((match) => (match[1] ?? "").toLowerCase());
 }
 function renderNodes(nodes, opts) {
-  let out = "";
-  for (const node of nodes) out += renderNode(node, opts);
-  return out;
+  const frame = (list) => ({ nodes: list, i: 0, out: [], pending: null });
+  const stack = [frame(nodes)];
+  while (stack.length > 0) {
+    const f = stack[stack.length - 1];
+    if (f.pending !== null) {
+      if (f.pending.done.length < f.pending.lists.length) {
+        stack.push(frame(f.pending.lists[f.pending.done.length]));
+        continue;
+      }
+      f.out.push(f.pending.assemble(f.pending.done));
+      f.pending = null;
+      continue;
+    }
+    if (f.i >= f.nodes.length) {
+      const text = f.out.join("");
+      stack.pop();
+      const parent = stack[stack.length - 1];
+      if (parent === void 0) return text;
+      parent.pending.done.push(text);
+      continue;
+    }
+    const node = f.nodes[f.i];
+    f.i += 1;
+    const step = renderNode(node, opts);
+    if (typeof step === "string") f.out.push(step);
+    else f.pending = step;
+  }
+  return "";
 }
 function renderNode(node, opts) {
   switch (node.type) {
@@ -1060,7 +1131,7 @@ function conditionalTakesThen(name, inverted, opts) {
 }
 function renderConditional(node, opts) {
   const truthy = conditionalTakesThen(node.name, node.inverted, opts);
-  return renderNodes(truthy ? node.then : node.else, opts);
+  return { lists: [truthy ? node.then : node.else], done: [], assemble: (parts) => parts[0] ?? "" };
 }
 function resolveCountConditionals(text, opts) {
   if (!text.includes("{?")) return text;
@@ -1145,7 +1216,7 @@ function renderPlural(node, opts) {
     return fullwidthVerbatim(countRaw, formsRaw);
   }
   const picked = pluralFor(base, Number.parseInt(count, 10), forms);
-  return renderNodes(parseSequence(picked), opts);
+  return { lists: [parseSequence(picked)], done: [], assemble: (parts) => parts[0] ?? "" };
 }
 function rawConstruct(countRaw, formsRaw) {
   return `{plural ${countRaw}:${formsRaw}}`;
@@ -1156,13 +1227,19 @@ function fullwidthVerbatim(countRaw, formsRaw) {
 function renderEnumeration(options, opts) {
   if (options.length === 0) return "";
   const picked = options[randomInt(opts.rng, 0, options.length - 1)];
-  return picked ? renderNodes(picked, opts) : "";
+  if (!picked) return "";
+  return { lists: [picked], done: [], assemble: (parts) => parts[0] ?? "" };
 }
 function renderPermutation(node, opts) {
-  const elements = node.options.map((o) => ({
-    text: renderNodes(o.nodes, opts),
-    sep: o.separator
-  }));
+  if (node.options.length === 0) return "";
+  return {
+    lists: node.options.map((o) => o.nodes),
+    done: [],
+    assemble: (parts) => assemblePermutation(node, parts, opts)
+  };
+}
+function assemblePermutation(node, rendered, opts) {
+  const elements = node.options.map((o, i) => ({ text: rendered[i] ?? "", sep: o.separator }));
   const total = elements.length;
   if (total === 0) return "";
   const { config } = node;