@hyperfixi/testing-framework 4.5.0 → 4.6.0

package/CHANGELOG.md

@@ -7,6 +7,131 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [4.6.0] - 2026-10-15
+
+### Added
+
+- **`@lokascript/semantic`: `ask` and `answer`** in every language's own word (es `preguntar` /
+  `responder`, ja `尋ねる` / `答える`, de `frage` / `antworte`): upstream's two dialog commands
+  (`ask "Name?"` prompts; `answer "Save?" with "Yes" or "No"` confirms, the choices after the language's
+  `with`). Both were refused in every language. Listed for native review in
+  `packages/semantic/NATIVE_REVIEW_NEEDED.md`.
+- **`@lokascript/semantic`: `scroll … by` with a direction** in every language's own words
+  (`scroll #box down by 200px`: es `desplazamiento #box abajo por 200px`, ja `#boxを 下 200px スクロール`,
+  ru `прокрутить #box вниз 200px`). Upstream's relative scroll was refused in every language: no
+  direction had a word, and scroll's schema modeled `scroll to <el>` alone. Listed for native review in
+  `packages/semantic/NATIVE_REVIEW_NEEDED.md`.
+
+### Changed
+
+- **`@lokascript/framework`: a domain DSL's parse refuses input it did not read.**
+  `createMultilingualDSL`'s `parse` (and so `compile`, `validate`, `translate`) tries every pattern and
+  takes the best match that reads the WHOLE input: highest priority, then confidence. When the only
+  matches leave input unread, it throws `UnreadInputError` (exported; `code: 'unread-input'`, with the
+  longest match's `reading` and the `unread` text). It used to return the first match however much it
+  read, so a trailing word was dropped without a word: the domain gate (lokascript-domains) found
+  1,744 of 1,744 renders accepted with an unknown word after them, and `ask what is 2+2` asked the
+  model "what". Against the gate (domains 4.0.0, measured unpublished with `gate:upstream`): 1,527
+  cases better, 62 worse, every one `pass → refused` (a value such as `click`'s that the generator
+  ignored, now refused as unread rather than dropped); no case became silent, and no accepted parse
+  changed its reading. A multi-word llm question is quoted: `ask "what is 2+2"`.
+- **`@lokascript/framework`: a domain DSL's `translate` refuses a rendering that does not read
+  back.** It reads its output in the target language and requires the source command: the same
+  action and role values, or the same compiled code. Otherwise it throws `TranslationReadBackError`
+  (exported; the text under `partial`, what it reads as under `reading`), unless `{ lossy: 'allow' }`,
+  as semantic's `translate`. `DomainRegistry`'s tools (the MCP `parse_` / `compile_` / `validate_` /
+  `translate_<domain>` tools) report refusals as `ok: false` with an `UNREAD_INPUT` or
+  `TRANSLATION_DOES_NOT_READ_BACK` diagnostic instead of a generic tool error, and `translate_<domain>`
+  reads its own rendering back the same way. Translating every domain gate command from English: 46
+  renders that used to be returned are now refused (45 do not parse in the target language).
+- **`@lokascript/semantic`: the parser is split into modules.** `semantic-parser.ts` went from 9,680
+  lines to 751, across 22 modules under `parser/`, and each question about an end word (does it close a
+  block, does it stop a clause) has one reader. Each step was checked against 27,194 parses; the steps
+  that change a reading are listed under Fixed. `SemanticParseError` keeps its import path.
+
+### Fixed
+
+- **`@lokascript/semantic`: a handler's count** (`on click 1 …`, the first click only; `on click 2 …`)
+  is read in every language. The renders already wrote it (es `al clic 1 ocultar`), but every reader
+  outside English dropped it, so the translation was refused in all 23 languages. Bengali, Italian and
+  Thai also read every clause between a handler's event and its body (`on click elsewhere`,
+  `on mutation of attributes`, `on foo queue first`), which their head pattern skipped. In the command-shape
+  gate, 375 lanes that were refused now pass.
+- **`@lokascript/semantic`: a query of a pseudo-class alone** (`<:checked/>`, `<:not(.done)/>`) reads
+  as a query in every language, English too. Every tokenizer read `<` and then a local variable
+  `:checked`, so a translation was refused. In the command-shape gate, 161 lanes that were refused now
+  pass.
+- **`@lokascript/semantic`: value spellings from upstream's own tests** read in every language,
+  English too: an escaped class name (`.group-\[:nth-of-type\(3\)_\&\]:block`), a query of several
+  selectors (`<#d1, #d2/>`), a unit after parentheses (`(startWidth)px`), `[@data-foo]` and an
+  attribute after a pronoun (`set it @disabled to true`, written `its @disabled`), a dotted property
+  before `of` (`set parentNode.innerHTML of #d1 to "foo"`), and `set {bar: 2, baz: 3} on obj`. Each was
+  refused. `get` no longer has a destination slot, which no reader used and which took Italian's `in`:
+  `get #p1 in me` read back as `get #p1 on me` in Italian, and `get result as JSONString` was refused
+  in Quechua. In the command-shape gate, 202 lanes are better, 3 of them silent before.
+- **`@lokascript/semantic`: upstream's collection expressions** read in English and translate:
+  `set :items to <span/> in #box where it matches .a`,
+  `put matches mapped to its textContent into #out`, `repeat for x in :items where x.val > 10 …`. They were refused in every language, English
+  too, or kept as an English clause after the command. A translation writes the language's own `where`,
+  as it already did in a condition (es `donde`, de `wo`), and keeps `mapped to` in English until native
+  review. Also read now: an element-scoped variable (`:items`) inside a longer value, the article in
+  `in the closest <table/>` and `checked of the :boxes`, and zh/tl's `no` before a sigil variable. Quechua and
+  Vietnamese read their own `where` back (it read as `target`, or split at its `_`), which also clears a
+  silent pair in each. In the command-shape gate, 154 lanes are better, 2 of them silent before.
+- **`@lokascript/semantic`: a query's scope inside a phrase** reads in English and translates:
+  `put the length of <:checked/> in #box into me` (the value ended at the query and `in` was taken as
+  put's marker), `bind $n to the length of <:checked/> in the next <tbody/> …` (a positional scope),
+  `put <span/> in #box where it matches .a into me` (English came back silently as `… into #box …`), and
+  `put the length of <p/> into #out` and `put 'x' into the next <div/>'s textContent`, which the lossy
+  check refused because it read the `'s` after `<p/>` as a quote. In the command-shape gate, 63 lanes are
+  better, 1 of them silent before.
+- **`@lokascript/semantic`: no translation reads back as other English without saying so.** The
+  command-shape gate's last five silent pairs read back as their source: pl's `closest .bar to .foo`
+  (pl's own `to` is "it"), `closest @foo` in pl and sw, a Quechua chain of `'s` (`$data's inner's val`,
+  read as a string), and Indonesian's `saya punya @data-x`. English `set closest @foo to "doh"` and
+  `put closest @foo into me`, refused in all 24 languages, read now too.
+- **`@lokascript/semantic`: commands that semantic already models, refused in some languages,** read
+  and translate. In the command-shape gate, 168 lanes that were refused now pass; with the rows above,
+  the refused lanes of shapes semantic models went from 911 to 86, and the gate reads 26,531 pass,
+  709 refused and 0 silent. The commands:
+  - `fetch $u as JSON`, with a `$` or `:` variable as the URL (23 languages);
+  - `toggle my *display`, `*opacity` and `*visibility` (ar, de, fr, he, ja, tr);
+  - a `js(…) … end` block after `then` behind a handler head (bn, es, it, ko, pl, ru, th, tl, uk);
+  - a snake_case name (`call global_function()`), which 11 tokenizers split at its `_`;
+  - a custom event after a handler head's `or` (`on click or foo`: bn, it, ja, ko, th, tr, zh). English
+    no longer drops the second event of `on foo or bar`;
+  - `put … at end of` in Vietnamese, and in bn, hi, ja, qu and tr after a bare name
+    (`put response at end of #results`);
+  - a `set` target with no marker after it, in it, pl, ru and uk (`set .divs.innerHTML to "foo"`,
+    `#d1.parentNode`, `bar["count"]`), and `on keydown[key=="Enter"]` in ar, de, fr and he;
+  - `fetch … as Object` and `as Number` behind a bn, hi or ko handler head;
+  - a top-level `for item in items …`, which bn, hi, ko and sw read as a handler on `item`;
+  - a loop's bottom test (`repeat … until x is 3 end`) in de, fr and ko;
+  - a Korean second handler on a parameterized event (`on exception(error)`), and Polish
+    `then throw` after `do` (Polish for `into`);
+  - bn, th and vi's own word for `checked`.
+- **`@lokascript/semantic`: `at end of` inside a behavior or a def** reads in every language:
+  `behavior B on click put "x" at end of me end` was refused, English to English too, because the
+  `end` of `at end of` closed the block. A variable spelled like an end word in such a body reads too
+  (es `fin`).
+- **`@lokascript/semantic`: two copied type lists that had lost `property-path`.** Chinese reads
+  `put the textContent of last <.picked/> in #list`, which it refused, and ja, ko and zh write the count
+  of `repeat the textContent of #a times` in their own words (`#aのtextContent`), not English.
+- **`@lokascript/semantic`: Quechua's `repeat until event … from …`** writes Quechua
+  (`kama ruway pointerup ta kutipay #box manta …`), not a leftover of the retired i18n transformer with
+  English `until event` and `repeat` in it.
+- **Test runs no longer fill the temp directory**: `scripts/vitest-run.sh` gives each run its own
+  `TMPDIR` and removes it. Vitest 5.0.2 leaves a copy of every transformed module in `$TMPDIR` per run.
+
+### Removed
+
+- **`@hyperfixi/mcp-server`: the jsx domain's four tools** (`parse_jsx`, `compile_jsx`, `validate_jsx`,
+  `translate_jsx`), and `jsx` as a `domain` for `lse_validate_and_feedback`, `generate_training_data`
+  and the `lse_generate` prompt. The server now depends on `@lokascript/domains` ^4.0.0, which retired
+  jsx with six other preview domains (`docs-internal/DOMAINS_RETHINK.md`, D1). 8 domains remain (sql,
+  bdd, behaviorspec, llm, flow, voice, todo, learn): 32 domain tools, 102 tools defined, 97 listed by
+  default.
+
 ## [4.5.0] - 2026-10-11
 
 Pronouns, suffixes and handler heads read as their language. In 4.4.0 every translation wrote `me` and
@@ -32,9 +157,9 @@ review in `packages/semantic/NATIVE_REVIEW_NEEDED.md`; none has been reviewed ye
 
 - **`@lokascript/semantic` writes a pronoun in the case its marker takes.** Free forms: es `en mí`,
   pt `a mim`, it `a me`, tl `sa akin`, pl `do mnie`, ru `ко мне` / `во мне`, uk `до мене`, de `zu mir`
-  / `in mich` (direction) / `auf mir` (location), hi `मुझ में`; `it` after a marker (ru `к этому`, pl `do
-  tego`, de `dazu` / `darin` / `darauf`, fr `à lui`, hi `इस में`); `me` as a verb's object (de `zeige
-  mich`, ru `покажи меня`). Fused forms: tr `bana` / `beni` / `benden`, `ona` / `onu`; ar `إليّ` / `فيه`;
+  / `in mich` (direction) / `auf mir` (location), hi `मुझ में`; `it` after a marker (ru `к этому`, pl
+  `do tego`, de `dazu` / `darin` / `darauf`, fr `à lui`, hi `इस में`); `me` as a verb's object (de
+  `zeige mich`, ru `покажи меня`). Fused forms: tr `bana` / `beni` / `benden`, `ona` / `onu`; ar `إليّ` / `فيه`;
   he `אליי` / `ממני`; bn `আমাকে` / `আমাতে`; hi `मुझे`; tl `dito`; pt `nele`. Object clitics on the verb:
   es `mostrarme`, it `metterlo`, pt `colocá-lo`, ar `ضعه`; fr writes the object pronoun before the
   infinitive (`le mettre`, `l'afficher`, `me cacher`). Every reader takes the nominative and the English
@@ -88,8 +213,8 @@ language has a word for. The words are the owner's vocabulary sheet
   counts, per render, the English grammar words (the tokens the engine's own parser matched in the
   source), English event names the language's lexicon has a word for, and a nominative `me` beside a
   marker in the 13 languages where that case is wrong. Two shrink-only baselines (the corpus, in
-  `test:canonical`, and the command-shape cases, in their shards); `tools/regen-english-leaks-baseline.ts
-  --report` prints each language against M2's targets. An English alternative a profile keeps for
+  `test:canonical`, and the command-shape cases, in their shards);
+  `tools/regen-english-leaks-baseline.ts --report` prints each language against M2's targets. An English alternative a profile keeps for
   code-switched input, beside a primary that is the language's own, counts as English.
 - **`@lokascript/semantic`: `transition … from … to`** reads and writes in every language (a `source`
   role), and the bare command reads in English. The schema's `destination` role is gone: upstream has
@@ -106,8 +231,8 @@ language has a word for. The words are the owner's vocabulary sheet
   back in English as upstream's first person), `does not match`, `fetch … do not throw`, `on first`,
   `go back`, `from elsewhere`, a loop's `index`, `start view transition`, a query's `in`, a conversion's
   `as` (the type name stays English), `the X of Y` inside an expression (es `valor de #price`, ja
-  `#priceの値`), `catch` and `finally`, `element x`, and `def` (as the noun "function", es `función
-  greet(name)`). bn, th and vi have their own `document` and `window`; he has its own words for all of
+  `#priceの値`), `catch` and `finally`, `element x`, and `def` (as the noun "function", es
+  `función greet(name)`). bn, th and vi have their own `document` and `window`; he has its own words for all of
   these and for `matches`, `exists`, `or`, `not`, `first`, `next`, `previous`, `closest`, `empty`, `is`,
   `document` and `window`; and the 15 languages that wrote English `forever` have their own (es
   `repetir siempre`, de `endlos`, ja `ずっと`). Every reader still takes the English words. Where a word
@@ -1611,7 +1736,8 @@ _Synchronized version release. See git history for details._
 - npm access token stored in GitHub Secrets
 - 2FA recommended for npm organization
 
-[Unreleased]: https://github.com/codetalcott/hyperfixi/compare/v4.4.0...HEAD
+[Unreleased]: https://github.com/codetalcott/hyperfixi/compare/v4.6.0...HEAD
+[4.6.0]: https://github.com/codetalcott/hyperfixi/compare/v4.5.0...v4.6.0
 [4.5.0]: https://github.com/codetalcott/hyperfixi/compare/v4.4.0...v4.5.0
 [4.4.0]: https://github.com/codetalcott/hyperfixi/compare/v4.3.0...v4.4.0
 [4.3.0]: https://github.com/codetalcott/hyperfixi/compare/v4.2.0...v4.3.0

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@hyperfixi/testing-framework",
-  "version": "4.5.0",
+  "version": "4.6.0",
   "description": "Cross-platform behavior testing suite for LokaScript applications",
   "main": "dist/index.js",
   "module": "dist/index.mjs",
@@ -57,10 +57,10 @@
   "author": "LokaScript Contributors",
   "license": "MIT",
   "dependencies": {
-    "@hyperfixi/patterns-reference": "^4.5.0",
-    "@lokascript/compilation-service": "^4.5.0",
-    "@lokascript/i18n": "^4.5.0",
-    "@lokascript/semantic": "^4.5.0",
+    "@hyperfixi/patterns-reference": "^4.6.0",
+    "@lokascript/compilation-service": "^4.6.0",
+    "@lokascript/i18n": "^4.6.0",
+    "@lokascript/semantic": "^4.6.0",
     "diff": "^8.0.3",
     "esbuild": "^0.25.12",
     "happy-dom": "^20.14.5",
@@ -72,8 +72,8 @@
     "vite": "^8.3.1"
   },
   "devDependencies": {
-    "@hyperfixi/engine": "^4.5.0",
-    "@lokascript/hyperscript-adapter": "^4.5.0",
+    "@hyperfixi/engine": "^4.6.0",
+    "@lokascript/hyperscript-adapter": "^4.6.0",
     "@types/diff": "^5.0.0",
     "@types/node": "^26.6.3",
     "@vitest/coverage-v8": "^5.0.0",

package/src/multilingual/command-shapes.ts

@@ -573,13 +573,6 @@ function someNode(plain: unknown, test: (node: PlainNode) => boolean): boolean {
 }
 
 export const LOUD: readonly LoudFamily[] = [
-  {
-    name: 'ask-answer',
-    reason:
-      'semantic has no schema and no dictionary word for `ask` / `answer`; adding them is an ' +
-      'owner vocabulary decision (the M2 sheet)',
-    matches: (_source, types) => types.has('askCommand') || types.has('answerCommand'),
-  },
   {
     name: 'templated-selector',
     reason:
@@ -600,14 +593,6 @@ export const LOUD: readonly LoudFamily[] = [
     matches: source => /'s(?=\s+(then|end)\b|\s*$)/.test(source),
   }, // M1 step 3 (2026-10-07): the English refusals left after the value and
   // command-form fixes, each family with why it is not carried.
-  {
-    name: 'scroll-by',
-    reason:
-      "scroll's relative form (`scroll #box down by 200px`) names a direction, which no " +
-      "language's dictionary has: adding them is an owner vocabulary decision (the M2 sheet)",
-    matches: (_source, _types, plain) =>
-      someNode(plain, node => node.type === 'scrollCommand' && 'by' in node),
-  },
   {
     name: 'beep',
     reason:
@@ -623,44 +608,13 @@ export const LOUD: readonly LoudFamily[] = [
       'a halt-only reading (OPEN_ITEMS P56)',
     matches: source => /\bhalt\s+(default\b|the event's\s+(bubbling|default)\b)/.test(source),
   },
-  {
-    name: 'collection-expressions',
-    reason:
-      "upstream 0.9.93's collection expressions (`… where it matches .a`, `mapped to`) are an " +
-      "expression form semantic's reader does not have: a parser-track item (OPEN_ITEMS P57)",
-    // Not `if no <q/> in … where …`, which English reads (its foreign lanes are
-    // the value-words family).
-    matches: (source, types) => types.has('collectionExpression') && !/\bif\s+no\s+</.test(source),
-  },
-  {
-    name: 'pseudo-class-query',
-    reason:
-      'a query of a pseudo-class (`<:checked/>`): the tokenizers read `:checked` as a local ' +
-      'variable; a tokenizer item, rare in pages',
-    matches: source => /<:[\w-]+\/>/.test(source),
-  },
   {
     name: 'closest-forms',
-    reason:
-      "`closest @attr` (an ancestor's attribute) and `closest parent <q/>`: the forms of " +
-      '`closest` besides a query, rare in pages',
+    // `closest @attr` (an ancestor's attribute) left this family when the positional
+    // run took an attribute query (2026-10-13).
+    reason: '`closest parent <q/>`: the form of `closest` that starts at the parent, rare in pages',
     matches: (_source, _types, plain) =>
-      someNode(
-        plain,
-        node =>
-          node.type === 'closestExpr' &&
-          // Not `closest @foo to #d1`, which English reads (value-words).
-          (node.parentSearch === true ||
-            (String(node.css ?? '').startsWith('[') &&
-              (node.to as { type?: unknown } | undefined)?.type === 'implicitMeTarget'))
-      ),
-  },
-  {
-    name: 'possessive-after-query',
-    reason:
-      "a possessive after a positional query (`the next <div/>'s textContent`), rare in pages; " +
-      'the `of` form (`the textContent of the next <div/>`) is carried',
-    matches: (source, types) => types.has('relativePositionalExpression') && /\/>'s\b/.test(source),
+      someNode(plain, node => node.type === 'closestExpr' && node.parentSearch === true),
   },
   {
     name: 'colon-event-arguments',
@@ -683,18 +637,6 @@ export const LOUD: readonly LoudFamily[] = [
         source
       ),
   },
-  {
-    name: 'rare-values',
-    reason:
-      'value spellings rare in pages: an escaped class name (`.group-\\[…\\]`), a comma ' +
-      'list in a query (`<#d1, #d2/>`), an attribute after a bare or bracketed reference ' +
-      '(`it @a`, `my [@a]`), a dotted path before `of`, a unit after parentheses ' +
-      '(`(w)px`), `set {…} on <obj>`',
-    matches: source =>
-      /\\\[|<[^<>]*,[^<>]*\/>|\b(my|its|your)\s+\[@|\bset\s+(it|me|you)\s+@|\b[a-z]\w*\.[a-z]\w*\s+of\b|\)(px|em|rem|%)|\bset\s+\{[^}]*\}\s+on\b/i.test(
-        source
-      ),
-  },
   {
     name: 'rare-commands',
     reason:
@@ -723,27 +665,9 @@ export interface SilentFamily {
   ids: readonly string[];
 }
 
-export const SILENT_FAMILIES: readonly SilentFamily[] = [
-  {
-    name: 'value-words',
-    reason:
-      "a language's word inside a value reads back as another English word: pl `to` → " +
-      '`it` (its `it` word, after `closest`), qu `where` → `target`, vi `where` split at `_`',
-    ids: ['d74a86f3a0', 'da24c9d768', 'b948ed872b'],
-  },
-  {
-    name: 'it-locative',
-    reason: 'it `in me` (a query scope) reads back as `on me`',
-    ids: ['2159462e64', '0ff2f70080', 'd7b35fefe6'],
-  },
-  {
-    name: 'possessive-split',
-    reason:
-      "a possessive reads back split or reworded: qu `$data 's inner' s val`, id `my` → " +
-      '`me punya`',
-    ids: ['76130fbf7d', '36e2294330', 'b839394853'],
-  },
-];
+// Empty since 2026-10-13: the last two families (pl `to` after `closest`, qu and id
+// possessives) were reader fixes. A new silent pair needs a family here, with its reason.
+export const SILENT_FAMILIES: readonly SilentFamily[] = [];
 
 /** The silent family a case belongs to, if any. */
 export function silentFamilyOf(id: string): SilentFamily | undefined {

package/src/multilingual/value-matrix.ts

@@ -596,7 +596,7 @@ const CODE_NAMES: readonly string[] = ['userData'];
  * Not a name the window already holds (`focus`, `length`), nor `then`: the
  * fixture sets each name there, and a `then` on the window makes upstream
  * take it for a promise. Not yet `if` or `end`, which open and close a block
- * in every language's reader (OPEN_ITEMS P53: 93 and 505 failing pairs).
+ * in every language's reader (OPEN_ITEMS P53: 93 and 501 failing pairs, 2026-10-13).
  */
 export const KEYWORD_NAMES: readonly string[] = [
   'click',