@spintax/core 0.4.0 → 0.5.1

package/CHANGELOG.md

@@ -3,6 +3,120 @@
 All notable changes to `@spintax/core` are documented here. This project adheres to
 [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## 0.5.1 — 2026-08-18
+
+Two crashes in the 0.5.0 form-counting path, both reachable from template text, both found by
+review rather than by a failing test. **Upgrade from 0.5.0.**
+
+### Fixed
+
+- **A 62-character template could take `validate()` out with an out-of-memory crash.**
+
+  ```
+  #set %a% = %b% %b%
+  #set %b% = %a% %a%
+  {plural 2: one|%a%}
+  ```
+
+  Counting plural forms expands definitions, and the expansion was bounded by 51 *passes* with
+  nothing bounding the *size*. That template doubles the text on every pass, so 51 passes is 2^51.
+  Every engine in the family died on it — this one and the Python port with an out-of-memory
+  crash, both PHP engines with a memory fatal. A non-cyclic doubling chain does the same, so the
+  circular-reference guard never got a chance.
+
+  The expansion now stops at 64 KB and reports the count as unknowable, which is the same silence
+  this validator already uses for every input it cannot pin down. A form list is a handful of
+  plural forms; nothing legitimate approaches the ceiling.
+
+  `validate` is exposed by public HTTP surfaces — this was a remote denial of service, not a
+  local footgun.
+
+- **`validate()` threw `RangeError` on a long `#set` chain.** The macro walk added in 0.5.0 was
+  recursive: roughly 9000 links (a 205 KB template) exhausted the call stack, while the Python
+  port gave up at ~1000 links (20 KB) and both PHP engines answered normally — one input, three
+  behaviours. The walk is iterative now.
+
+  It is a pre-order depth-first walk in source order, and it has to stay one: the first non-clean
+  answer wins, so a graph holding both an opaque macro and a brackety one gives different
+  diagnostics depending on which is met first. Checked by leaving the PHP engine recursive and
+  diffing 300 generated macro graphs against it — no mismatches.
+
+### Notes
+
+No API change and no verdict change for any template that was not crashing. Both fixes are
+mirrored into `spintax/core`, `spintax-core` and the WordPress plugin, and pinned by two corpus
+fixtures whose real assertion is that the engine answers at all — a runner that hangs fails them.
+
+## 0.5.0 — 2026-08-18
+
+Two plural fixes, both cross-engine, both found by measuring one engine against another rather
+than by a failing test. Minor rather than patch: validation output and render output both change.
+
+### Fixed
+
+- **`validate` counted plural forms before `%variable%` expansion; `render` counts them after**
+  (issue #66, found by the Object Pascal port and confirmed in all five engines).
+
+  `#def %tail% = few|many` with `{plural 2: one|%tail%}` under `ru` renders correctly as three
+  forms, and every validator reported `plural.arity` for the two it could see in the source. The
+  count now substitutes definition values first — every reference per pass, as the renderer does —
+  and splits the result.
+
+  Only where the count is **provably invariant**. A value carrying any bracket — all four, and
+  conditionals too — suppresses the count-based verdicts rather than guessing: `{a|b}` really does
+  always freeze to one form, but `{?flag?a|b|c}` freezes as `a` or as `b|c` and the two cannot be
+  told apart without evaluating the construct. Predicting the roll was tried first and produced a
+  fresh crop of false errors. Silence on an unknowable input is the trade.
+
+  One case is not a prediction: a `#set` named **directly** in the form slot is substituted
+  verbatim and is still spintax when the plural is decided, so its brackets keep earning
+  `plural.nested-brackets` — through a `#def` it is rolled first and earns nothing.
+
+- **A conditional in the plural COUNT slot deleted the block, silently.**
+
+  ```
+  #set %flag% =
+  #set %n% = {?flag?1|2}
+  start {plural %n%: one|two} end
+  ```
+
+  rendered `start end` — no fallback braces, no diagnostic, `validate` returning `[]`. Both PHP
+  engines have always rendered `start two end`: they run the conditional stage over the whole text
+  before plurals, so a plain number reaches the slot. This engine expanded variables into the raw
+  slot and left constructs literal, so the conditional failed the numeric test and the block was
+  erased. `plural.count-macro` exempts conditionals *because* they resolve before plurals — the
+  validator was written to a renderer behaviour that was not implemented here.
+
+  The branch is substituted, never rendered: enumerations resolve AFTER plurals, so a branch
+  yielding `{a|b}` still reaches the numeric test intact and still erases the block, exactly as the
+  plugin does. Rendering it would invent a count no engine has.
+
+  The **form** slot is deliberately untouched. There the engines genuinely disagree — both PHP
+  renderers resolve a conditional that expansion introduces inside a form list, this engine and the
+  Python one do not — and picking a side is issue #67, not a bug fix.
+
+### Performance
+
+- The count-slot pass is linear and iterative. Written recursively it made `render()` throw
+  `RangeError` at ~9000 levels of nesting, breaking the §9.2 promise that render never throws on
+  content; matching braces per `{?` made it quadratic, and an unbalanced count slot is legal input
+  (only the whole `{plural …}` block has to balance, and the slot is cut at the first `:`), so a
+  78 KB template cost 3 seconds. Both were reachable from template text through a public endpoint.
+  78 KB now costs 42 ms and 1 MB costs 372 ms.
+
+  Deep *balanced* nesting is still super-linear, in every engine of the family — the reference
+  implementation scans the body per level too. Bounding input size is a host job (§9.3); the
+  reference Worker now refuses a template over 8192 characters with 413, the same cap the hosted
+  MCP server has always applied.
+
+### Notes
+
+Mirrored where each engine needed it: the counting fix into `spintax/core`, the WordPress plugin
+and `spintax-core` (Python); the count-slot fix into the Python engine only, the PHP engines
+having been right all along. Pinned by 20 new corpus fixtures — 12 for the counting rule, 8 for the
+count slot, of which 4 are negative controls — taking the corpus from 235 to 255, green in
+TypeScript, Python and both PHP engines.
+
 ## 0.4.0 — 2026-08-17
 
 ### Added

package/dist/index.cjs

@@ -51,7 +51,7 @@ function stripSentinels(text) {
 // src/internal/parser.ts
 var VARIABLE_RE = /^%(\w+)%/;
 var DIRECTIVE_RE = /^[ \t]*#(set|def)[ \t]+%(\w+)%[ \t]*=[ \t]*(.*?)[ \t]*\r?$/gmu;
-var CONDITIONAL_NAME_RE = /^[A-Za-z_]\w*/;
+var CONDITIONAL_NAME_RE = /[A-Za-z_]\w*/y;
 var PLURAL_PREFIX = "plural ";
 function parseTemplate(src) {
   const { body, setDefs, defDefs } = extractDirectives(stripComments(stripSentinels(src)));
@@ -152,28 +152,42 @@ function parseBraceConstruct(content) {
   }
   return { type: "enumeration", options: splitTopLevel(content).map((o) => parseSequence(o)) };
 }
-function tryParseConditional(content) {
-  let p = 1;
+function recognizeConditional(text, contentStart, contentEnd) {
+  let p = contentStart + 1;
   let inverted = false;
-  if (content.charAt(p) === "!") {
+  if (text.charAt(p) === "!") {
     inverted = true;
     p += 1;
   }
-  const name = CONDITIONAL_NAME_RE.exec(content.slice(p))?.[0];
-  if (name === void 0) return null;
+  CONDITIONAL_NAME_RE.lastIndex = p;
+  const name = CONDITIONAL_NAME_RE.exec(text)?.[0];
+  if (name === void 0 || p + name.length > contentEnd) return null;
   p += name.length;
-  if (content.charAt(p) !== "?") return null;
+  if (text.charAt(p) !== "?") return null;
   p += 1;
-  const body = content.slice(p);
-  const sep = firstTopLevelPipe(body);
-  const thenRaw = sep < 0 ? body : body.slice(0, sep);
-  const elseRaw = sep < 0 ? "" : body.slice(sep + 1);
+  return { name, inverted, bodyStart: p, sepIndex: firstTopLevelPipe(text, p, contentEnd) };
+}
+function splitConditional(content) {
+  const head = recognizeConditional(content, 0, content.length);
+  if (head === null) return null;
+  const body = content.slice(head.bodyStart);
+  const sep = head.sepIndex < 0 ? -1 : head.sepIndex - head.bodyStart;
+  return {
+    name: head.name,
+    inverted: head.inverted,
+    thenRaw: sep < 0 ? body : body.slice(0, sep),
+    elseRaw: sep < 0 ? "" : body.slice(sep + 1)
+  };
+}
+function tryParseConditional(content) {
+  const parts = splitConditional(content);
+  if (parts === null) return null;
   return {
     type: "conditional",
-    name,
-    inverted,
-    then: parseSequence(thenRaw),
-    else: parseSequence(elseRaw)
+    name: parts.name,
+    inverted: parts.inverted,
+    then: parseSequence(parts.thenRaw),
+    else: parseSequence(parts.elseRaw)
   };
 }
 function parsePlural(afterPrefix) {
@@ -333,9 +347,9 @@ function splitTopLevel(inner) {
   parts.push(cur);
   return parts;
 }
-function firstTopLevelPipe(body) {
+function firstTopLevelPipe(body, from = 0, to = body.length) {
   let depth = 0;
-  for (let j = 0; j < body.length; j += 1) {
+  for (let j = from; j < to; j += 1) {
     const ch = body.charAt(j);
     if (ch === "{" || ch === "[") {
       depth += 1;
@@ -429,7 +443,7 @@ function validateTemplate(src, opts = {}) {
   checkBrackets(text, diagnostics);
   checkDirectives(text, diagnostics);
   checkPermutationConfigs(text, idx, diagnostics);
-  checkPlurals(text, idx, opts.locale, diagnostics);
+  checkPlurals(text, idx, opts.locale, opts.knownVariables, diagnostics);
   checkVariableReferences(text, idx, opts.knownVariables, diagnostics);
   if (opts.knownIncludes && opts.knownIncludes.length > 0) {
     checkIncludeTargets(text, idx, opts.knownIncludes, diagnostics);
@@ -563,10 +577,12 @@ function checkPermutationConfigs(text, idx, out) {
     }
   }
 }
-function checkPlurals(text, idx, locale, out) {
+function checkPlurals(text, idx, locale, knownVariables, out) {
   const base = locale && locale !== "" ? normalizeBaseLang(locale) : "";
   const arity = base !== "" ? pluralArity(base) : 0;
   const tainted = macroTaintedNames(text);
+  const { setDefs: macros, defDefs: defs } = extractDirectives(text);
+  const hostNames = new Set((knownVariables ?? []).map((n) => n.toLowerCase()));
   for (const block of findPluralBlocks(text)) {
     const at = span(idx, block.start, block.end - block.start);
     for (const m of block.countSlot.matchAll(/%(\w+)%/gu)) {
@@ -582,7 +598,13 @@ function checkPlurals(text, idx, locale, out) {
       out.push(err("plural.nested-brackets", "{plural ...}: forms must not contain nested spintax brackets. Extract via #def first \u2014 a #set is substituted verbatim and would put the brackets straight back.", at));
       continue;
     }
-    const count = block.formsRaw.split("|").length;
+    const expanded = expandFormsForCounting(block.formsRaw, defs, macros, hostNames);
+    if (expanded.directMacroSpintax) {
+      out.push(err("plural.nested-brackets", "{plural ...}: forms must not contain nested spintax brackets. Extract via #def first \u2014 a #set is substituted verbatim and would put the brackets straight back.", at));
+      continue;
+    }
+    if (expanded.unresolved) continue;
+    const count = expanded.forms;
     if (arity > 0) {
       if (count !== arity) {
         out.push(err(
@@ -592,15 +614,90 @@ function checkPlurals(text, idx, locale, out) {
         ));
       }
     } else if (count !== DEFAULT_PLURAL_ARITY) {
+      const unusable = locale !== void 0 && locale !== "";
       out.push(warn(
         "plural.locale-missing",
-        `{plural ...}: ${count} forms, but no locale was supplied. render defaults to ${DEFAULT_PLURAL_ARITY} forms and leaves this block unresolved \u2014 pass the locale you will render with.`,
-        { ...at, data: { got: count, defaultArity: DEFAULT_PLURAL_ARITY } }
+        unusable ? `{plural ...}: ${count} forms, and the locale '${locale}' does not normalize to a language the engine knows, so render falls back to ${DEFAULT_PLURAL_ARITY} forms and leaves this block unresolved \u2014 pass a base tag such as 'en' or 'ru'.` : `{plural ...}: ${count} forms, but no locale was supplied. render defaults to ${DEFAULT_PLURAL_ARITY} forms and leaves this block unresolved \u2014 pass the locale you will render with.`,
+        {
+          ...at,
+          data: {
+            got: count,
+            defaultArity: DEFAULT_PLURAL_ARITY,
+            ...unusable ? { locale } : {}
+          }
+        }
       ));
     }
   }
 }
 var UNRESOLVED_AT_PLURAL_TIME = /\[|\{(?!\?)/u;
+var ANY_BRACKET = /[[\]{}]/u;
+var FORM_EXPANSION_PASSES = 51;
+var FORM_EXPANSION_MAX_GROWTH = 64 * 1024;
+function expandFormsForCounting(formsRaw, defs, macros, hostNames) {
+  const unknown = (directMacroSpintax = false) => ({
+    forms: 0,
+    unresolved: true,
+    directMacroSpintax
+  });
+  const seenMacro = /* @__PURE__ */ new Set();
+  const refsOf = (text2) => [...text2.matchAll(/%(\w+)%/gu)].map((m) => (m[1] ?? "").toLowerCase());
+  const walkMacros = (source) => {
+    const stack = [{ refs: refsOf(source), i: 0 }];
+    while (stack.length > 0) {
+      const frame = stack[stack.length - 1];
+      if (frame.i >= frame.refs.length) {
+        stack.pop();
+        continue;
+      }
+      const name = frame.refs[frame.i++];
+      if (defs[name] !== void 0 || hostNames.has(name)) continue;
+      const macro = macros[name];
+      if (macro === void 0 || seenMacro.has(name)) continue;
+      seenMacro.add(name);
+      if (/\{\?/u.test(macro)) return "opaque";
+      if (ANY_BRACKET.test(macro)) return "brackets";
+      stack.push({ refs: refsOf(macro), i: 0 });
+    }
+    return "clean";
+  };
+  const verbatim = walkMacros(formsRaw);
+  if (verbatim === "brackets") return unknown(true);
+  if (verbatim === "opaque") return unknown();
+  let text = formsRaw;
+  const budget = formsRaw.length + FORM_EXPANSION_MAX_GROWTH;
+  for (let pass = 0; pass < FORM_EXPANSION_PASSES; pass++) {
+    let sawReference = false;
+    let bailed = false;
+    const parts = [];
+    let total = 0;
+    let cursor = 0;
+    const re = /%(\w+)%/gu;
+    let m;
+    while ((m = re.exec(text)) !== null) {
+      sawReference = true;
+      const name = (m[1] ?? "").toLowerCase();
+      const value = hostNames.has(name) ? void 0 : defs[name] ?? macros[name];
+      if (value === void 0 || ANY_BRACKET.test(value)) {
+        bailed = true;
+        break;
+      }
+      parts.push(text.slice(cursor, m.index), value);
+      total += m.index - cursor + value.length;
+      cursor = m.index + m[0].length;
+      if (total > budget) return unknown();
+    }
+    if (bailed) return unknown();
+    parts.push(text.slice(cursor));
+    total += text.length - cursor;
+    if (total > budget) return unknown();
+    text = parts.join("");
+    if (!sawReference) {
+      return { forms: text.split("|").length, unresolved: false, directMacroSpintax: false };
+    }
+  }
+  return unknown();
+}
 function macroTaintedNames(text) {
   const macros = extractDirectives(text).setDefs;
   const tainted = /* @__PURE__ */ new Set();
@@ -950,14 +1047,63 @@ function expandVarsOnly(text, opts) {
   }
   return out;
 }
-function renderConditional(node, opts) {
-  const value = opts.vars[node.name.toLowerCase()];
+function conditionalTakesThen(name, inverted, opts) {
+  const value = opts.vars[name.toLowerCase()];
   const baseTruthy = value !== void 0 && /\S/u.test(value);
-  const truthy = node.inverted ? !baseTruthy : baseTruthy;
+  return inverted ? !baseTruthy : baseTruthy;
+}
+function renderConditional(node, opts) {
+  const truthy = conditionalTakesThen(node.name, node.inverted, opts);
   return renderNodes(truthy ? node.then : node.else, opts);
 }
+function resolveCountConditionals(text, opts) {
+  if (!text.includes("{?")) return text;
+  const close = matchBraces(text);
+  const out = [];
+  const pending = [[0, text.length]];
+  while (pending.length > 0) {
+    const segment = pending.pop();
+    const segEnd = segment[1];
+    let i = segment[0];
+    while (i < segEnd) {
+      const open = text.indexOf("{?", i);
+      if (open === -1 || open + 1 >= segEnd) {
+        out.push(text.slice(i, segEnd));
+        break;
+      }
+      const shut = close[open] ?? -1;
+      const head = shut === -1 || shut >= segEnd ? null : recognizeConditional(text, open + 1, shut);
+      if (head === null) {
+        out.push(text.slice(i, Math.min(open + 2, segEnd)));
+        i = open + 2;
+        continue;
+      }
+      out.push(text.slice(i, open));
+      const branchEnd = head.sepIndex < 0 ? shut : head.sepIndex;
+      const [from, to] = conditionalTakesThen(head.name, head.inverted, opts) ? [head.bodyStart, branchEnd] : [head.sepIndex < 0 ? shut : head.sepIndex + 1, shut];
+      pending.push([shut + 1, segEnd]);
+      pending.push([from, to]);
+      break;
+    }
+  }
+  return out.join("");
+}
+function matchBraces(text) {
+  const close = new Int32Array(text.length).fill(-1);
+  const opens = [];
+  for (let i = 0; i < text.length; i += 1) {
+    const ch = text.charAt(i);
+    if (ch === "{") {
+      opens.push(i);
+    } else if (ch === "}") {
+      const open = opens.pop();
+      if (open !== void 0) close[open] = i;
+    }
+  }
+  return close;
+}
 function renderPlural(node, opts) {
-  const countRaw = expandVarsOnly(node.countRaw, opts);
+  const countRaw = resolveCountConditionals(expandVarsOnly(node.countRaw, opts), opts);
   const formsRaw = expandVarsOnly(node.formsRaw, opts);
   const base = normalizeBaseLang(opts.locale);
   const report = (issue) => opts.onPluralError?.(issue);