@hyperfixi/testing-framework 4.3.0 → 4.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +130 -1
- package/package.json +8 -8
- package/src/multilingual/README.md +35 -0
- package/src/multilingual/command-shapes-gate.ts +32 -0
- package/src/multilingual/command-shapes.ts +22 -23
- package/src/multilingual/direct-path-shapes.ts +60 -26
- package/src/multilingual/english-leaks.corpus.test.ts +74 -0
- package/src/multilingual/english-leaks.test.ts +463 -0
- package/src/multilingual/english-leaks.ts +1000 -0
- package/src/multilingual/hand-written-input.tasks.json +162 -0
- package/src/multilingual/hand-written-input.test.ts +62 -0
- package/src/multilingual/hand-written-input.ts +201 -0
- package/src/vocab/batch3-roundtrip.test.ts +2 -1
package/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,132 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [4.5.0] - 2026-10-11
|
|
11
|
+
|
|
12
|
+
Pronouns, suffixes and handler heads read as their language. In 4.4.0 every translation wrote `me` and
|
|
13
|
+
`it` in the nominative beside a marker (es `a yo`, de `zu ich`, tr `ben e`, ar `إلى أنا`), wrote every
|
|
14
|
+
case particle as a word of its own (ko `#out 에`, ja `"x" を`), and wrote a Japanese, Bengali or Turkish
|
|
15
|
+
handler head with two markers (`クリック を で`, `ক্লিক তে`, `tıklama i üzerinde`). Each of these is now
|
|
16
|
+
the language's own form, and every reader still takes the old one. The forms are listed for native
|
|
17
|
+
review in `packages/semantic/NATIVE_REVIEW_NEEDED.md`; none has been reviewed yet.
|
|
18
|
+
|
|
19
|
+
### Added
|
|
20
|
+
|
|
21
|
+
- **`@lokascript/testing-framework`: the English-leak gate counts pronoun case and marker spacing.**
|
|
22
|
+
`case:it` and an unmarked object pronoun (`case:me-object`, `case:it-object`) beside `case:me`; and
|
|
23
|
+
`spacing:<marker>`, a particle ko, ja, bn or tr attaches to its noun written as a word of its own.
|
|
24
|
+
Both shrink-only, both halves (corpus and command shapes), with per-language exit targets in
|
|
25
|
+
`LEAK_TARGETS`.
|
|
26
|
+
- **`@lokascript/semantic`: `toggle … between`** in every language's own words (ja `の間で`, hi `के बीच`,
|
|
27
|
+
he `בין`), with the style form's element written in the language.
|
|
28
|
+
- **`docs-internal/multilingual/FUSED_FORMS.md`**: the design for pronouns fused with their markers and
|
|
29
|
+
suffixes attached to words, with the owner's four questions and their answers.
|
|
30
|
+
|
|
31
|
+
### Changed
|
|
32
|
+
|
|
33
|
+
- **`@lokascript/semantic` writes a pronoun in the case its marker takes.** Free forms: es `en mí`,
|
|
34
|
+
pt `a mim`, it `a me`, tl `sa akin`, pl `do mnie`, ru `ко мне` / `во мне`, uk `до мене`, de `zu mir`
|
|
35
|
+
/ `in mich` (direction) / `auf mir` (location), hi `मुझ में`; `it` after a marker (ru `к этому`, pl `do
|
|
36
|
+
tego`, de `dazu` / `darin` / `darauf`, fr `à lui`, hi `इस में`); `me` as a verb's object (de `zeige
|
|
37
|
+
mich`, ru `покажи меня`). Fused forms: tr `bana` / `beni` / `benden`, `ona` / `onu`; ar `إليّ` / `فيه`;
|
|
38
|
+
he `אליי` / `ממני`; bn `আমাকে` / `আমাতে`; hi `मुझे`; tl `dito`; pt `nele`. Object clitics on the verb:
|
|
39
|
+
es `mostrarme`, it `metterlo`, pt `colocá-lo`, ar `ضعه`; fr writes the object pronoun before the
|
|
40
|
+
infinitive (`le mettre`, `l'afficher`, `me cacher`). Every reader takes the nominative and the English
|
|
41
|
+
forms as before.
|
|
42
|
+
- **`@lokascript/semantic` writes a case particle onto its word** in ko (`"x"을 #out에 넣다`, `나를`,
|
|
43
|
+
each particle in the form the syllable takes), ja (`#outに`, `.aを`) and bn (`#outএ`, `.xকে`), and tr
|
|
44
|
+
fuses a suffix with its own reference words (`sonucu`, `hedefe`, `belgeye`) while code keeps its
|
|
45
|
+
suffix apart (`#out e`) until a reviewer picks a code word's harmony. tr also reads a suffix on code
|
|
46
|
+
after an apostrophe (`#out'a`, `.active'i`).
|
|
47
|
+
- **`@lokascript/semantic`: a custom event in a wait takes the language's `event` word** (`esperar evento foo`,
|
|
48
|
+
ja `イベント foo 待つ`), as `repeat until event X` already does; a known DOM event keeps `esperar transitionend`.
|
|
49
|
+
`wait for <custom event>` was refused in every language. A possessive whose property is a reference word
|
|
50
|
+
(`its.detail`) now reads back as that property, not the language's word for it.
|
|
51
|
+
- **`@lokascript/semantic`: ms and pl read their lexicon's `undefined` word** (`tidak_ditakrifkan`, `niezdefiniowane`),
|
|
52
|
+
which read back as a variable name.
|
|
53
|
+
- **`@lokascript/testing-framework`: the hand-written-input gate** (`hand-written-input.test.ts`): twenty tasks a
|
|
54
|
+
native speaker writes from English, run as written on the engine against upstream; committed with no inputs.
|
|
55
|
+
- **`@lokascript/semantic` writes a handler head as one word**: ja `クリックで` (was `クリック を で`),
|
|
56
|
+
bn `ক্লিকে` (the locative in the form the word takes: `জমাতে` after a vowel; was `ক্লিক তে`),
|
|
57
|
+
tr `tıklamada` (the locative in the form harmony gives: `göndermede`, `değişiklikte`; code keeps it apart, `keyup da`; was `tıklama i üzerinde`). The readers keep taking the two-marker heads.
|
|
58
|
+
- **`@lokascript/semantic` writes a query's scope before the query** in bn, hi, tr, ja and ko (tr
|
|
59
|
+
`bendeki <p/>`, hi `मुझ में स्थित <p/>`, ja `自分 の中の <p/>`), where it wrote English's order.
|
|
60
|
+
- **`@lokascript/semantic`: es and pt write the `focus` and `blur` commands as verbs** (`enfocarme`,
|
|
61
|
+
`focar-me`); the nouns were the events' and still read.
|
|
62
|
+
- **`@lokascript/semantic`: ja's head pattern outranks its fused handler patterns**, so a ja handler's
|
|
63
|
+
body always takes the clause walk: a `js … end` block after `then` and `fetch … as T` round-trip
|
|
64
|
+
where the fused path dropped them. The 2026-08 transformer's verb-first `繰り返し イベント X を まで`
|
|
65
|
+
body no longer reads in ja (the render since 2026-08-28 does).
|
|
66
|
+
|
|
67
|
+
- **`@lokascript/htmx-adapter`: the tr vocab gains `hx-da` as an alias of `hx-on`;** `hx-üzerinde` stays the name
|
|
68
|
+
(the profile's `on` word moved to the locative; a suffix is the wrong register for an attribute name).
|
|
69
|
+
|
|
70
|
+
### Fixed
|
|
71
|
+
|
|
72
|
+
- **`publish.yml`: a repair run skips a version npm is still processing** (`E409 previously staged`)
|
|
73
|
+
instead of failing the whole run.
|
|
74
|
+
|
|
75
|
+
## [4.4.0] - 2026-10-10
|
|
76
|
+
|
|
77
|
+
Translations now read as their language. In 4.3.0, 19% of the corpus's translations and 24% of the
|
|
78
|
+
command-shape scripts' translations kept at least one English word, and a third of the corpus kept an
|
|
79
|
+
English event name. Counting only the English a recorded decision does not keep (type names, `url`,
|
|
80
|
+
`debounced at`, …), every one of the 23 languages is now at or under 5% of corpus renders (0.7–4.6%)
|
|
81
|
+
and 10% of command-shape renders (3.9–6.2%), and no translation writes an English event name its
|
|
82
|
+
language has a word for. The words are the owner's vocabulary sheet
|
|
83
|
+
(`docs-internal/multilingual/VOCABULARY_SHEET.md`), decided 2026-10-08.
|
|
84
|
+
|
|
85
|
+
### Added
|
|
86
|
+
|
|
87
|
+
- **`@hyperfixi/testing-framework`: the English left in a translation is gated.** `english-leaks.ts`
|
|
88
|
+
counts, per render, the English grammar words (the tokens the engine's own parser matched in the
|
|
89
|
+
source), English event names the language's lexicon has a word for, and a nominative `me` beside a
|
|
90
|
+
marker in the 13 languages where that case is wrong. Two shrink-only baselines (the corpus, in
|
|
91
|
+
`test:canonical`, and the command-shape cases, in their shards); `tools/regen-english-leaks-baseline.ts
|
|
92
|
+
--report` prints each language against M2's targets. An English alternative a profile keeps for
|
|
93
|
+
code-switched input, beside a primary that is the language's own, counts as English.
|
|
94
|
+
- **`@lokascript/semantic`: `transition … from … to`** reads and writes in every language (a `source`
|
|
95
|
+
role), and the bare command reads in English. The schema's `destination` role is gone: upstream has
|
|
96
|
+
none, and SOV destination markers read an owned property's owner as an element (tr and qu swapped
|
|
97
|
+
`#foo's *width` silently). English writes `using` for an easing.
|
|
98
|
+
- **`@lokascript/semantic`: `add/remove/hide/show … when <condition>`** in every language, through a
|
|
99
|
+
`condition` role marked with each profile's `when`. It was English in every translation, and refused
|
|
100
|
+
as a bare command.
|
|
101
|
+
|
|
102
|
+
### Changed
|
|
103
|
+
|
|
104
|
+
- **`@lokascript/semantic` writes each language's own words** for: loop heads (`repeat for x in`,
|
|
105
|
+
`while`, `until`; es `repetir para x en .items`), `I match` / `I am` (written as the third person, and
|
|
106
|
+
back in English as upstream's first person), `does not match`, `fetch … do not throw`, `on first`,
|
|
107
|
+
`go back`, `from elsewhere`, a loop's `index`, `start view transition`, a query's `in`, a conversion's
|
|
108
|
+
`as` (the type name stays English), `the X of Y` inside an expression (es `valor de #price`, ja
|
|
109
|
+
`#priceの値`), `catch` and `finally`, `element x`, and `def` (as the noun "function", es `función
|
|
110
|
+
greet(name)`). bn, th and vi have their own `document` and `window`; he has its own words for all of
|
|
111
|
+
these and for `matches`, `exists`, `or`, `not`, `first`, `next`, `previous`, `closest`, `empty`, `is`,
|
|
112
|
+
`document` and `window`; and the 15 languages that wrote English `forever` have their own (es
|
|
113
|
+
`repetir siempre`, de `endlos`, ja `ずっと`). Every reader still takes the English words. Where a word
|
|
114
|
+
would read back as another (es `en` is also a marker), the translation keeps English's.
|
|
115
|
+
- **`@lokascript/semantic` writes native event names** in he, hi, it, pl, qu, ru, th, uk and vi where
|
|
116
|
+
the lexicon's word reads back; the rest are denylisted with their reasons.
|
|
117
|
+
- **`@lokascript/semantic`: a foreign `put X at start of Y` is written with the language's `prepend`**,
|
|
118
|
+
which English writes back as `put … at start of`.
|
|
119
|
+
|
|
120
|
+
### Fixed
|
|
121
|
+
|
|
122
|
+
- **`@lokascript/semantic` reads `me` after a marker in the case the marker takes**, as authors write it:
|
|
123
|
+
de `zu mir`, pt `a mim`, pl `do mnie`, ru `ко мне`, uk `до мене`, hi `मुझ में`. Each was read as a
|
|
124
|
+
variable (`zu mir` ran as `to mir`) or refused. Translations still write the nominative.
|
|
125
|
+
|
|
126
|
+
- **`@lokascript/semantic`: `put X at start of Y` translates.** It was refused in all 23 languages: no
|
|
127
|
+
language but English had a pattern for it, and the render wrote `at end of` in its place.
|
|
128
|
+
- **`@lokascript/semantic`: a one-line handler's `put … at end of`** translates in es, id, it, pt, tl and
|
|
129
|
+
uk. The fused handler pattern took the end noun for the destination wherever the language's `at` word
|
|
130
|
+
is also a destination marker.
|
|
131
|
+
- **`@lokascript/semantic`: `^x is …` keeps its copula** in ar, hi and th, which read the `is` word after a
|
|
132
|
+
`^` variable as the pronoun `it`.
|
|
133
|
+
- **`@lokascript/semantic`: an SOV loop body may open with a variable named `index`** (ja `index を …`),
|
|
134
|
+
and `repeat` is never read as an event name.
|
|
135
|
+
|
|
10
136
|
## [4.3.0] - 2026-10-07
|
|
11
137
|
|
|
12
138
|
More of what upstream reads now translates, and what does not is refused instead of lost. On the
|
|
@@ -1485,7 +1611,10 @@ _Synchronized version release. See git history for details._
|
|
|
1485
1611
|
- npm access token stored in GitHub Secrets
|
|
1486
1612
|
- 2FA recommended for npm organization
|
|
1487
1613
|
|
|
1488
|
-
[Unreleased]: https://github.com/codetalcott/hyperfixi/compare/v4.
|
|
1614
|
+
[Unreleased]: https://github.com/codetalcott/hyperfixi/compare/v4.4.0...HEAD
|
|
1615
|
+
[4.5.0]: https://github.com/codetalcott/hyperfixi/compare/v4.4.0...v4.5.0
|
|
1616
|
+
[4.4.0]: https://github.com/codetalcott/hyperfixi/compare/v4.3.0...v4.4.0
|
|
1617
|
+
[4.3.0]: https://github.com/codetalcott/hyperfixi/compare/v4.2.0...v4.3.0
|
|
1489
1618
|
[4.2.0]: https://github.com/codetalcott/hyperfixi/compare/v4.1.0...v4.2.0
|
|
1490
1619
|
[4.1.0]: https://github.com/codetalcott/hyperfixi/compare/v4.0.1...v4.1.0
|
|
1491
1620
|
[4.0.1]: https://github.com/codetalcott/hyperfixi/compare/v4.0.0...v4.0.1
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@hyperfixi/testing-framework",
|
|
3
|
-
"version": "4.
|
|
3
|
+
"version": "4.5.0",
|
|
4
4
|
"description": "Cross-platform behavior testing suite for LokaScript applications",
|
|
5
5
|
"main": "dist/index.js",
|
|
6
6
|
"module": "dist/index.mjs",
|
|
@@ -40,7 +40,7 @@
|
|
|
40
40
|
"analyze-failures": "tsx src/multilingual/tools/analyze-failures.ts",
|
|
41
41
|
"typecheck": "tsc --noEmit",
|
|
42
42
|
"test:check": "VITEST_TIMEOUT=240 VITEST_QUIET=1 bash ../../scripts/vitest-run.sh --reporter=dot",
|
|
43
|
-
"test:canonical": "FOREIGN_CANONICAL_VALIDITY=1 VITEST_TIMEOUT=300 bash ../../scripts/vitest-run.sh --reporter=dot src/multilingual/canonical-validity.test.ts src/multilingual/foreign-canonical-validity.test.ts src/multilingual/render-fidelity.test.ts src/multilingual/bare-render-fidelity.test.ts src/multilingual/en-reference-preservation.test.ts",
|
|
43
|
+
"test:canonical": "FOREIGN_CANONICAL_VALIDITY=1 VITEST_TIMEOUT=300 bash ../../scripts/vitest-run.sh --reporter=dot src/multilingual/canonical-validity.test.ts src/multilingual/foreign-canonical-validity.test.ts src/multilingual/render-fidelity.test.ts src/multilingual/bare-render-fidelity.test.ts src/multilingual/en-reference-preservation.test.ts src/multilingual/english-leaks.corpus.test.ts",
|
|
44
44
|
"test:shipped-sources": "VITEST_TIMEOUT=300 bash ../../scripts/vitest-run.sh --reporter=dot src/multilingual/shipped-sources-engine.test.ts src/multilingual/shipped-sources-localized.test.ts src/multilingual/shipped-examples-execution.test.ts"
|
|
45
45
|
},
|
|
46
46
|
"keywords": [
|
|
@@ -57,10 +57,10 @@
|
|
|
57
57
|
"author": "LokaScript Contributors",
|
|
58
58
|
"license": "MIT",
|
|
59
59
|
"dependencies": {
|
|
60
|
-
"@hyperfixi/patterns-reference": "^4.
|
|
61
|
-
"@lokascript/compilation-service": "^4.
|
|
62
|
-
"@lokascript/i18n": "^4.
|
|
63
|
-
"@lokascript/semantic": "^4.
|
|
60
|
+
"@hyperfixi/patterns-reference": "^4.5.0",
|
|
61
|
+
"@lokascript/compilation-service": "^4.5.0",
|
|
62
|
+
"@lokascript/i18n": "^4.5.0",
|
|
63
|
+
"@lokascript/semantic": "^4.5.0",
|
|
64
64
|
"diff": "^8.0.3",
|
|
65
65
|
"esbuild": "^0.25.12",
|
|
66
66
|
"happy-dom": "^20.14.5",
|
|
@@ -72,8 +72,8 @@
|
|
|
72
72
|
"vite": "^8.3.1"
|
|
73
73
|
},
|
|
74
74
|
"devDependencies": {
|
|
75
|
-
"@hyperfixi/engine": "^4.
|
|
76
|
-
"@lokascript/hyperscript-adapter": "^4.
|
|
75
|
+
"@hyperfixi/engine": "^4.5.0",
|
|
76
|
+
"@lokascript/hyperscript-adapter": "^4.5.0",
|
|
77
77
|
"@types/diff": "^5.0.0",
|
|
78
78
|
"@types/node": "^26.6.3",
|
|
79
79
|
"@vitest/coverage-v8": "^5.0.0",
|
|
@@ -294,6 +294,41 @@ npx tsx tools/regen-value-matrix-baseline.ts --dry-run --results /tmp/matrix.jso
|
|
|
294
294
|
A lane that fails in `en-rt` fails in nearly every translation: semantic's English
|
|
295
295
|
parse lost the value, and every translation is rendered from it. Fix that first.
|
|
296
296
|
|
|
297
|
+
## English left in a translation
|
|
298
|
+
|
|
299
|
+
Every gate above asks whether a translation means what the source means; none asks
|
|
300
|
+
whether it reads as the language. `english-leaks.ts` counts, in each render, four kinds
|
|
301
|
+
of finding:
|
|
302
|
+
|
|
303
|
+
| Finding | What it is |
|
|
304
|
+
| ----------------------------------- | --------------------------------------------------------------------------------------------- |
|
|
305
|
+
| `<word>` | an English word the engine reads as grammar in the source, written in English |
|
|
306
|
+
| `event:<name>` | an English event name where the language's lexicon has a word and the renderer may use it |
|
|
307
|
+
| `case:me` | the nominative `me` beside a role marker, in the 13 languages where that form is wrong |
|
|
308
|
+
| `case:it` | the nominative `it` beside a role marker that wants another form (`CASE_RULES`) |
|
|
309
|
+
| `case:me-object` / `case:it-object` | the nominative pronoun as a command's unmarked object, where that form is wrong |
|
|
310
|
+
| `spacing:<marker>` | a marker ko, ja, bn or tr attaches to its noun, written apart (`#out 에`; `ATTACHED_MARKERS`) |
|
|
311
|
+
|
|
312
|
+
"Grammar" is derived per source: the tokens the engine's own parser matched as words, plus
|
|
313
|
+
the references in its parse. A word the language spells the same (es `a`, de `in`) and a
|
|
314
|
+
name spelled like a keyword are never counted; strings, `js` blocks and brace interiors
|
|
315
|
+
are not read. The report splits each word by context (`clause`: in a clause semantic keeps
|
|
316
|
+
as written; `bracket`, `call`, `property`, `plain`) and by whether the language has its own
|
|
317
|
+
word for it.
|
|
318
|
+
|
|
319
|
+
Two halves, each with a shrink-only baseline: `baselines/english-leaks.corpus.json` (every
|
|
320
|
+
non-markup corpus row, `render(parse(en), L)`; in `test:canonical`, after `populate`) and
|
|
321
|
+
`baselines/english-leaks.shapes.json` (the renders the command-shape shards already make).
|
|
322
|
+
|
|
323
|
+
```bash
|
|
324
|
+
# The burn-down: per language, per word (with contexts), per event
|
|
325
|
+
npx tsx tools/regen-english-leaks-baseline.ts --report
|
|
326
|
+
|
|
327
|
+
# After a fix: prune the findings that are gone (refuses to add new ones)
|
|
328
|
+
npx tsx tools/regen-english-leaks-baseline.ts # both halves (corpus needs a fresh populate)
|
|
329
|
+
npx tsx tools/regen-english-leaks-baseline.ts --shapes # one half
|
|
330
|
+
```
|
|
331
|
+
|
|
297
332
|
## Direct-path shapes on the text path
|
|
298
333
|
|
|
299
334
|
Core's `src/multilingual/*-direct-path.test.ts` files pin shapes core's direct path ran
|
|
@@ -13,6 +13,10 @@
|
|
|
13
13
|
*
|
|
14
14
|
* The cases file, the baseline's own consistency and the oracle are checked
|
|
15
15
|
* in command-shapes.cases.test.ts and command-shapes.oracle.test.ts.
|
|
16
|
+
*
|
|
17
|
+
* Each shard also judges the English left in its renders (english-leaks.ts),
|
|
18
|
+
* against baselines/english-leaks.shapes.json: the same renders, a second
|
|
19
|
+
* question, so the gate costs no second translation pass.
|
|
16
20
|
*/
|
|
17
21
|
|
|
18
22
|
import { readFileSync } from 'node:fs';
|
|
@@ -27,6 +31,16 @@ import {
|
|
|
27
31
|
type CommandShapesBaseline,
|
|
28
32
|
type PairChange,
|
|
29
33
|
} from './command-shapes';
|
|
34
|
+
import {
|
|
35
|
+
diffLeakBaseline,
|
|
36
|
+
gainedLeaks,
|
|
37
|
+
prunableLeaks,
|
|
38
|
+
initLeakScanner,
|
|
39
|
+
loadLeakBaseline,
|
|
40
|
+
shapeLeaks,
|
|
41
|
+
type LeakChange,
|
|
42
|
+
type LeakResults,
|
|
43
|
+
} from './english-leaks';
|
|
30
44
|
|
|
31
45
|
export const COMMAND_SHAPE_SHARDS = 3;
|
|
32
46
|
|
|
@@ -47,11 +61,16 @@ export function describeCommandShapesShard(shard: number): void {
|
|
|
47
61
|
`${c.id} [${c.lane}] ${c.listed} → ${c.got}: ${sources.get(c.id)?.slice(0, 80)}` +
|
|
48
62
|
(c.detail ? ` ⇒ ${JSON.stringify(collapse(c.detail).slice(0, 80))}` : '');
|
|
49
63
|
|
|
64
|
+
const leakLine = (c: LeakChange, words: string[]): string =>
|
|
65
|
+
`${c.id} [${c.language}] ${words.join(' ')}: ${sources.get(c.id)?.slice(0, 80)}`;
|
|
66
|
+
|
|
50
67
|
describe(`command shapes [${shard + 1}/${COMMAND_SHAPE_SHARDS}]`, () => {
|
|
51
68
|
let results: CaseResult[] = [];
|
|
69
|
+
let leaks: LeakResults = new Map();
|
|
52
70
|
|
|
53
71
|
beforeAll(async () => {
|
|
54
72
|
results = await runCommandShapes(own);
|
|
73
|
+
leaks = shapeLeaks(await initLeakScanner(), results, own);
|
|
55
74
|
const pairs = results.reduce((n, r) => n + Object.keys(r.lanes).length, 0);
|
|
56
75
|
const failing = results.reduce(
|
|
57
76
|
(n, r) => n + Object.values(r.lanes).filter(l => l.outcome !== 'pass').length,
|
|
@@ -72,5 +91,18 @@ export function describeCommandShapesShard(shard: number): void {
|
|
|
72
91
|
const { better } = diffBaseline(results, loadCommandShapesBaseline());
|
|
73
92
|
expect(capped(better.map(line))).toEqual([]);
|
|
74
93
|
});
|
|
94
|
+
|
|
95
|
+
it('no render holds English the leak baseline does not list (english-leaks.ts)', () => {
|
|
96
|
+
expect(leaks.size).toBeGreaterThan(300);
|
|
97
|
+
const gained = gainedLeaks(diffLeakBaseline(leaks, loadLeakBaseline('shapes')));
|
|
98
|
+
expect(capped(gained.map(c => leakLine(c, c.added)))).toEqual([]);
|
|
99
|
+
});
|
|
100
|
+
|
|
101
|
+
it('no listed English is gone, nor a listed refusal rendering now (prune with tools/regen-english-leaks-baseline.ts --shapes)', () => {
|
|
102
|
+
const prune = prunableLeaks(diffLeakBaseline(leaks, loadLeakBaseline('shapes')));
|
|
103
|
+
expect(capped(prune.map(c => leakLine(c, c.newRender ? ['now renders'] : c.gone)))).toEqual(
|
|
104
|
+
[]
|
|
105
|
+
);
|
|
106
|
+
});
|
|
75
107
|
});
|
|
76
108
|
}
|
|
@@ -111,6 +111,8 @@ export interface LaneResult {
|
|
|
111
111
|
/** The lane's English (a silent lane), or the error's first line (a refused one). */
|
|
112
112
|
detail?: string;
|
|
113
113
|
kind?: SilentKind;
|
|
114
|
+
/** The language's render (`translate(src, 'en', L)`), when there was one: what english-leaks.ts reads. */
|
|
115
|
+
rendered?: string;
|
|
114
116
|
}
|
|
115
117
|
|
|
116
118
|
export interface CaseResult {
|
|
@@ -497,11 +499,6 @@ export async function initCommandShapes(): Promise<CommandShapeRunner> {
|
|
|
497
499
|
const readers = await loadEngineReaders();
|
|
498
500
|
const { translate } = (await import('@lokascript/semantic')) as { translate: Translate };
|
|
499
501
|
|
|
500
|
-
const lane = (source: string, language: string): string =>
|
|
501
|
-
language === 'en'
|
|
502
|
-
? translate(source, 'en', 'en')
|
|
503
|
-
: translate(translate(source, 'en', language), language, 'en');
|
|
504
|
-
|
|
505
502
|
return {
|
|
506
503
|
runCase(shape, lanes = LANES) {
|
|
507
504
|
const engine = readers[readerOf(shape)];
|
|
@@ -516,14 +513,26 @@ export async function initCommandShapes(): Promise<CommandShapeRunner> {
|
|
|
516
513
|
lanes: {},
|
|
517
514
|
};
|
|
518
515
|
for (const language of lanes) {
|
|
516
|
+
let rendered: string | undefined;
|
|
519
517
|
let english: string;
|
|
520
518
|
try {
|
|
521
|
-
english =
|
|
519
|
+
if (language === 'en') english = translate(shape.source, 'en', 'en');
|
|
520
|
+
else {
|
|
521
|
+
rendered = translate(shape.source, 'en', language);
|
|
522
|
+
english = translate(rendered, language, 'en');
|
|
523
|
+
}
|
|
522
524
|
} catch (e) {
|
|
523
|
-
result.lanes[language] = {
|
|
525
|
+
result.lanes[language] = {
|
|
526
|
+
outcome: 'refused',
|
|
527
|
+
detail: firstLine(e),
|
|
528
|
+
...(rendered !== undefined ? { rendered } : {}),
|
|
529
|
+
};
|
|
524
530
|
continue;
|
|
525
531
|
}
|
|
526
|
-
result.lanes[language] =
|
|
532
|
+
result.lanes[language] = {
|
|
533
|
+
...judge(engine, want, wantPlain, english),
|
|
534
|
+
...(rendered !== undefined ? { rendered } : {}),
|
|
535
|
+
};
|
|
527
536
|
}
|
|
528
537
|
return result;
|
|
529
538
|
},
|
|
@@ -718,19 +727,9 @@ export const SILENT_FAMILIES: readonly SilentFamily[] = [
|
|
|
718
727
|
{
|
|
719
728
|
name: 'value-words',
|
|
720
729
|
reason:
|
|
721
|
-
"a language's word inside a value reads back as another English word:
|
|
722
|
-
'`it
|
|
723
|
-
|
|
724
|
-
ids: [
|
|
725
|
-
'a30e7d6174',
|
|
726
|
-
'69537d01ae',
|
|
727
|
-
'd74a86f3a0',
|
|
728
|
-
'da24c9d768',
|
|
729
|
-
'b948ed872b',
|
|
730
|
-
'a4c56acf1e',
|
|
731
|
-
'87250fbc5e',
|
|
732
|
-
'54a65f4602',
|
|
733
|
-
],
|
|
730
|
+
"a language's word inside a value reads back as another English word: pl `to` → " +
|
|
731
|
+
'`it` (its `it` word, after `closest`), qu `where` → `target`, vi `where` split at `_`',
|
|
732
|
+
ids: ['d74a86f3a0', 'da24c9d768', 'b948ed872b'],
|
|
734
733
|
},
|
|
735
734
|
{
|
|
736
735
|
name: 'it-locative',
|
|
@@ -741,8 +740,8 @@ export const SILENT_FAMILIES: readonly SilentFamily[] = [
|
|
|
741
740
|
name: 'possessive-split',
|
|
742
741
|
reason:
|
|
743
742
|
"a possessive reads back split or reworded: qu `$data 's inner' s val`, id `my` → " +
|
|
744
|
-
|
|
745
|
-
ids: ['76130fbf7d', '36e2294330', 'b839394853'
|
|
743
|
+
'`me punya`',
|
|
744
|
+
ids: ['76130fbf7d', '36e2294330', 'b839394853'],
|
|
746
745
|
},
|
|
747
746
|
];
|
|
748
747
|
|
|
@@ -57,6 +57,11 @@ export interface DirectPathCase {
|
|
|
57
57
|
event: string | null;
|
|
58
58
|
/** The languages the core test ran; `[]` = English only. */
|
|
59
59
|
languages: 'all23' | string[];
|
|
60
|
+
/**
|
|
61
|
+
* The source defines a behavior of this name (a hand-written task): it is
|
|
62
|
+
* defined under a fresh name and installed with no arguments.
|
|
63
|
+
*/
|
|
64
|
+
behaviorName?: string;
|
|
60
65
|
}
|
|
61
66
|
|
|
62
67
|
const CASES_PATH = path.resolve(
|
|
@@ -234,6 +239,12 @@ export function caseFails(r: DirectPathResult): boolean {
|
|
|
234
239
|
|
|
235
240
|
export interface DirectPathHosts {
|
|
236
241
|
runCase(c: DirectPathCase): Promise<DirectPathResult>;
|
|
242
|
+
/**
|
|
243
|
+
* One translation someone WROTE (the hand-written-input gate), run as
|
|
244
|
+
* runCase runs a render: `written` as written on the engine with the
|
|
245
|
+
* adapter, against upstream on the English source.
|
|
246
|
+
*/
|
|
247
|
+
runWritten(c: DirectPathCase, lang: string, written: string): Promise<PairResult>;
|
|
237
248
|
/** Restore the console, the rejection listeners and EventTarget. */
|
|
238
249
|
close(): Promise<void>;
|
|
239
250
|
}
|
|
@@ -321,14 +332,17 @@ export async function initDirectPathHosts(): Promise<DirectPathHosts> {
|
|
|
321
332
|
document.documentElement.removeAttribute('lang');
|
|
322
333
|
try {
|
|
323
334
|
let install = code;
|
|
324
|
-
|
|
335
|
+
const coreBehavior = c.id.startsWith('behavior-handlers');
|
|
336
|
+
if (coreBehavior || c.behaviorName) {
|
|
325
337
|
// The source defines a behavior: define it (under a fresh name), then
|
|
326
|
-
// install it in English, as the core test does.
|
|
338
|
+
// install it in English, as the core test does. A hand-written task
|
|
339
|
+
// names its behavior (`Hider`) and installs it with no arguments.
|
|
327
340
|
const name = `Demo${letter(behaviorSerial)}${letter(Math.floor(behaviorSerial / 26))}`;
|
|
328
341
|
behaviorSerial++;
|
|
329
|
-
const
|
|
342
|
+
const pattern = coreBehavior ? /Demo[A-Z]+/g : new RegExp(`\\b${c.behaviorName}\\b`, 'g');
|
|
343
|
+
const definition = code.replace(pattern, name);
|
|
330
344
|
host.evaluate(lang === 'en' ? definition : preprocess(definition, lang));
|
|
331
|
-
install = `install ${name}(h: 1)`;
|
|
345
|
+
install = coreBehavior ? `install ${name}(h: 1)` : `install ${name}()`;
|
|
332
346
|
lang = 'en';
|
|
333
347
|
}
|
|
334
348
|
document.body.innerHTML = c.fixture;
|
|
@@ -386,15 +400,44 @@ export async function initDirectPathHosts(): Promise<DirectPathHosts> {
|
|
|
386
400
|
}
|
|
387
401
|
};
|
|
388
402
|
|
|
389
|
-
|
|
390
|
-
|
|
391
|
-
|
|
392
|
-
|
|
403
|
+
/** Upstream on the English source: the oracle, or why there is none. */
|
|
404
|
+
type Oracle = { rejected: string | undefined; want: string };
|
|
405
|
+
const oracle = async (c: DirectPathCase): Promise<Oracle> => {
|
|
406
|
+
// A behavior definition is not a handler: it is checked by running it.
|
|
407
|
+
const rejected =
|
|
408
|
+
c.id.startsWith('behavior-handlers') || c.behaviorName
|
|
393
409
|
? undefined
|
|
394
410
|
: rejects(upstream, c.source);
|
|
395
|
-
|
|
396
|
-
|
|
397
|
-
|
|
411
|
+
const want = rejected
|
|
412
|
+
? `✗upstream rejects: ${rejected}`
|
|
413
|
+
: await runAll(upstream, c, c.source, 'en');
|
|
414
|
+
return { rejected, want };
|
|
415
|
+
};
|
|
416
|
+
/** One translation, written or rendered, on the engine against the oracle. */
|
|
417
|
+
const pairFor = async (
|
|
418
|
+
c: DirectPathCase,
|
|
419
|
+
{ rejected, want }: Oracle,
|
|
420
|
+
lang: string,
|
|
421
|
+
written: string
|
|
422
|
+
): Promise<PairResult> => {
|
|
423
|
+
let english: string;
|
|
424
|
+
try {
|
|
425
|
+
english = preprocess(written, lang);
|
|
426
|
+
} catch (e) {
|
|
427
|
+
english = `✗${(e as Error).message}`;
|
|
428
|
+
}
|
|
429
|
+
const got = await runAll(engine, c, written, lang);
|
|
430
|
+
let ref = want;
|
|
431
|
+
if (rejected && !english.startsWith('✗') && !rejects(upstream, english)) {
|
|
432
|
+
ref = await runAll(upstream, c, english, 'en');
|
|
433
|
+
}
|
|
434
|
+
const ok = got === ref && !EMPTY.test(ref) && !ref.startsWith('✗');
|
|
435
|
+
return { lang, written, english, got, ref, ok };
|
|
436
|
+
};
|
|
437
|
+
|
|
438
|
+
return {
|
|
439
|
+
async runCase(c) {
|
|
440
|
+
const o = await oracle(c);
|
|
398
441
|
const eng = await runAll(engine, c, c.source, 'en');
|
|
399
442
|
const languages = c.languages === 'all23' ? [...FOREIGN_LANGUAGES] : c.languages;
|
|
400
443
|
const node = parseSemantic(c.source, 'en').node;
|
|
@@ -407,24 +450,15 @@ export async function initDirectPathHosts(): Promise<DirectPathHosts> {
|
|
|
407
450
|
written = null;
|
|
408
451
|
}
|
|
409
452
|
if (written === null) {
|
|
410
|
-
pairs.push({ lang, written, english: '', got: '✗untranslatable', ref: want, ok: false });
|
|
453
|
+
pairs.push({ lang, written, english: '', got: '✗untranslatable', ref: o.want, ok: false });
|
|
411
454
|
continue;
|
|
412
455
|
}
|
|
413
|
-
|
|
414
|
-
try {
|
|
415
|
-
english = preprocess(written, lang);
|
|
416
|
-
} catch (e) {
|
|
417
|
-
english = `✗${(e as Error).message}`;
|
|
418
|
-
}
|
|
419
|
-
const got = await runAll(engine, c, written, lang);
|
|
420
|
-
let ref = want;
|
|
421
|
-
if (rejected && !english.startsWith('✗') && !rejects(upstream, english)) {
|
|
422
|
-
ref = await runAll(upstream, c, english, 'en');
|
|
423
|
-
}
|
|
424
|
-
const ok = got === ref && !EMPTY.test(ref) && !ref.startsWith('✗');
|
|
425
|
-
pairs.push({ lang, written, english, got, ref, ok });
|
|
456
|
+
pairs.push(await pairFor(c, o, lang, written));
|
|
426
457
|
}
|
|
427
|
-
return { id: c.id, want, eng, pairs };
|
|
458
|
+
return { id: c.id, want: o.want, eng, pairs };
|
|
459
|
+
},
|
|
460
|
+
async runWritten(c, lang, written) {
|
|
461
|
+
return pairFor(c, await oracle(c), lang, written);
|
|
428
462
|
},
|
|
429
463
|
async close() {
|
|
430
464
|
await new Promise(resolve => setTimeout(resolve, 0));
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* The English-leak gate, corpus half (see english-leaks.ts): every non-markup
|
|
3
|
+
* corpus row rendered into each language as `populate` writes it, judged
|
|
4
|
+
* against baselines/english-leaks.corpus.json. Fails on a finding a pair did
|
|
5
|
+
* not have, and on a listed one that is gone (prune with
|
|
6
|
+
* tools/regen-english-leaks-baseline.ts --corpus), so the list only shrinks.
|
|
7
|
+
*
|
|
8
|
+
* Needs FOREIGN_CANONICAL_VALIDITY=1 and a freshly populated patterns.db: the
|
|
9
|
+
* SET of rows depends on `populate`, the same contract the other corpus gates
|
|
10
|
+
* carry (`npm run test:canonical`).
|
|
11
|
+
*
|
|
12
|
+
* @vitest-environment node
|
|
13
|
+
*/
|
|
14
|
+
import { describe, it, expect, beforeAll } from 'vitest';
|
|
15
|
+
import { getAllPatterns } from '@hyperfixi/patterns-reference';
|
|
16
|
+
import {
|
|
17
|
+
corpusLeaks,
|
|
18
|
+
diffLeakBaseline,
|
|
19
|
+
gainedLeaks,
|
|
20
|
+
prunableLeaks,
|
|
21
|
+
initLeakScanner,
|
|
22
|
+
loadLeakBaseline,
|
|
23
|
+
type LeakResults,
|
|
24
|
+
} from './english-leaks';
|
|
25
|
+
|
|
26
|
+
const DB_FRESHLY_POPULATED = process.env.FOREIGN_CANONICAL_VALIDITY === '1';
|
|
27
|
+
|
|
28
|
+
/** At most `max` lines, then a count: a broad regression must not print thousands. */
|
|
29
|
+
function capped(lines: string[], max = 40): string[] {
|
|
30
|
+
return lines.length <= max ? lines : [...lines.slice(0, max), `… and ${lines.length - max} more`];
|
|
31
|
+
}
|
|
32
|
+
|
|
33
|
+
describe.skipIf(!DB_FRESHLY_POPULATED)('english-leaks: corpus renders', () => {
|
|
34
|
+
let results: LeakResults = new Map();
|
|
35
|
+
let skipped: string[] = [];
|
|
36
|
+
const sources = new Map<string, string>();
|
|
37
|
+
|
|
38
|
+
beforeAll(async () => {
|
|
39
|
+
const rows = await getAllPatterns({ limit: 5000 });
|
|
40
|
+
for (const row of rows) sources.set(row.id, row.rawCode.replace(/\s+/g, ' ').trim());
|
|
41
|
+
({ results, skipped } = await corpusLeaks(await initLeakScanner(), rows));
|
|
42
|
+
}, 300_000);
|
|
43
|
+
|
|
44
|
+
it('scores the corpus (sanity: guards a false green)', () => {
|
|
45
|
+
expect(results.size).toBeGreaterThan(140);
|
|
46
|
+
// Rows the engine rejects have no grammar to read (worker, socket, eventsource, …).
|
|
47
|
+
expect(skipped.length).toBeLessThan(10);
|
|
48
|
+
});
|
|
49
|
+
|
|
50
|
+
it('no render holds English the baseline does not list', () => {
|
|
51
|
+
const gained = gainedLeaks(diffLeakBaseline(results, loadLeakBaseline('corpus')));
|
|
52
|
+
expect(
|
|
53
|
+
capped(
|
|
54
|
+
gained.map(
|
|
55
|
+
c => `${c.id} [${c.language}] ${c.added.join(' ')}: ${sources.get(c.id)?.slice(0, 80)}`
|
|
56
|
+
)
|
|
57
|
+
)
|
|
58
|
+
).toEqual([]);
|
|
59
|
+
});
|
|
60
|
+
|
|
61
|
+
it('no listed English is gone (prune it with tools/regen-english-leaks-baseline.ts --corpus)', () => {
|
|
62
|
+
const prune = prunableLeaks(diffLeakBaseline(results, loadLeakBaseline('corpus')));
|
|
63
|
+
expect(
|
|
64
|
+
capped(
|
|
65
|
+
prune.map(c => `${c.id} [${c.language}] ${c.newRender ? 'now renders' : c.gone.join(' ')}`)
|
|
66
|
+
)
|
|
67
|
+
).toEqual([]);
|
|
68
|
+
});
|
|
69
|
+
|
|
70
|
+
it('keeps no entry for a row the corpus no longer holds', () => {
|
|
71
|
+
const stale = Object.keys(loadLeakBaseline('corpus').entries).filter(id => !results.has(id));
|
|
72
|
+
expect(stale).toEqual([]);
|
|
73
|
+
});
|
|
74
|
+
});
|