@kohaku-ui/evals 0.1.0 → 0.2.0

package/CHANGELOG.md

@@ -0,0 +1,22 @@
+# @kohaku-ui/evals
+
+## 0.2.0
+
+### Minor Changes
+
+- [#23](https://github.com/yosuque/kohaku/pull/23) [`cf2623c`](https://github.com/yosuque/kohaku/commit/cf2623cd2688969db1156d0817ffff08bbe3f610) Thanks [@yosuque](https://github.com/yosuque)! - **`PROMPT_REVISION` bumped `"12"` → `"13"`** (`packages/composer/src/prompt.ts` / the Python mirror `python/kohaku/src/kohaku/composer/prompt.py`): the L2 design brief's spacing line no longer presupposes a "## Design system" section that may not be in the prompt (it now reads "one consistent spacing scale throughout; use the design tokens or kit utilities when the prompt supplies them" instead of "consistent spacing from the design tokens (…)"); `designKitPromptFragment`/`design_kit_prompt_fragment` gained an empty-input guard (an empty `classes`/`utilities`/`namespaces` no longer emits a dangling, self-contradicting heading); the "Skeleton of a well-formed widget body" section is now shown **iff a kit declares its own `skeleton`**, with no fallback of any kind to the built-in `k-card`/`k-grid-3`/`k-table` skeleton for a different kit (the built-in `DEFAULT_KIT_VOCABULARY` now declares its own `skeleton`/`skeleton=` field, rather than the function special-casing it by object identity — the identity check silently dropped the skeleton, and therefore changed the prompt, for any structurally-equivalent-but-not-identical `DEFAULT_KIT_VOCABULARY` reference, e.g. across an ESM dual-package install or a `ComposePolicy` that round-trips through JSON); kit classes are now presented **sorted by name** rather than in `Object.entries`/dict insertion order (closing a real TS/Python divergence: JS sorts integer-like keys like `"2col"` ahead of every other key regardless of declaration order, while Python dict iteration keeps insertion order); and "component class(es)" was renamed to "kit class(es)" throughout the L2 prompt and its repair feedback, to stop colliding with the Spec's own `ComponentDefinition` vocabulary. The built-in kit's own prompt bytes are unchanged by the empty-input guard and the skeleton-declaration change (it always showed its skeleton, before and after) — the sort and the rename are the only bytes that moved for it. **What to expect:** every cached L2 generation is separated by this revision bump, so the first request after upgrading pays a fresh L1/L2 generation cost instead of a cache hit (as with every prior `PROMPT_REVISION` bump); no action is required, and already-promoted/fixated artifacts are unaffected (only the *generation* cache is keyed by this). A product supplying its **own** design kit (`designSystem.kit`) that relied on the old fallback (omitting its own `skeleton` and expecting to see the built-in one) will now see no Skeleton section at all — pass your kit's own `skeleton` explicitly if you want one (see docs/user-guide.md's bring-your-own-kit section).
+  
+  `policyFingerprint`/`policy_fingerprint`'s design-kit material no longer carries a `classesOrder` entry (`packages/composer/src/context.ts` / `context.py`): now that kit classes are presented sorted by name, insertion order no longer affects the L2 prompt, so the material that used to carry it separately has nothing left to protect. Separately, the material's `skeleton` entry for the built-in kit changes from an absent key to `DEFAULT_KIT_SKELETON`'s text, reflecting the field it now declares. Both are one-time fingerprint changes for any caller using `designSystem.kit` (same "your compose cache is repartitioned once" caveat as any `policyFingerprint` material change; combined with the `PROMPT_REVISION` bump above, the practical effect is the same fresh-generation cost already described).
+  
+  `l2PromotionRubric` (the L2 promotion judge) moves to version `"0.3"`: `visual_quality`'s weight moves `0.1` → `0.2` and `generality`'s moves `0.15` → `0.05` (`generality` and `schema_inferability` are two ways of asking "is this a reusable, well-made component", which made `generality` the natural weight source for `visual_quality`'s increase); its description now also covers `never use fixed pixel widths — fill the container width` and `loading` states (previously part of the L2 generation brief but not scored), and replaces the unscorable two-word "restrained color" with the L2 brief's own countable wording ("use the primary color for one emphasis at most; tone colors only when they carry meaning"). `RubricCriterion`/`Criterion` also gains an optional `floor`, and `l2PromotionRubric`'s `safety` criterion sets one at `0.5`: previously a `safety` score of `0` could still be outvoted by the other five criteria under the weighted average (e.g. full marks elsewhere clears the default `passScore` of `0.6` even at `safety=0`) — `safety` is the one criterion that reads the generated HTML's semantic behavior the way a human reviewer would, so a low score there should never be compensated away by good looks elsewhere. `scoreWithRubric`'s verdict now fails (`pass: false`) whenever any floored criterion's score falls strictly below its floor, regardless of the weighted-average score, and reports which criteria vetoed via the new `JudgeVerdict.vetoedBy`/`JudgeVerdict.vetoed_by` (always present, `[]` when nothing vetoed). **What to expect:** a judge call using the default rubric can now flip from `pass: true` to `pass: false` purely because of a low `safety` score, even when the overall weighted score is unchanged — check `vetoedBy` when auditing a promotion-review outcome that looks surprising. Existing consumers also see a further score shift from the weight rebalance and the new description wording (on top of the `"0.2"` rubric's own shift when `visual_quality` was added). `rubricVersion`/`rubric_version` is an audit stamp only — no data migration. A consumer who wants the exact pre-`"0.3"` behavior (weights, description, and no floor) can pin it explicitly: `judge({ ..., rubric: l2PromotionRubricV0_2 })` (Python: `rubric=l2_promotion_rubric_v0_2`), newly exported alongside the existing `l2PromotionRubricV0_1`/`l2_promotion_rubric_v0_1`.
+
+- [#23](https://github.com/yosuque/kohaku/pull/23) [`b922703`](https://github.com/yosuque/kohaku/commit/b9227038c12dcaf790697b5ca8f1e70c98abc154) Thanks [@yosuque](https://github.com/yosuque)! - `l2PromotionRubric` (L2 promotion judge) gains a `visual_quality` criterion — clear visual hierarchy, consistent spacing, restrained color, right-aligned tabular numeric columns, notice-style empty/error states, no browser-default table/button/input styling, and (when the generation prompt supplied design tokens or a design kit) styles expressed with them rather than hard-coded values. The rubric moves to version `"0.2"` and existing weights are rebalanced (`safety` 0.3→0.25, `schema_inferability` 0.2→0.15, `visual_quality` 0.1 new) so all six criteria still sum to 1.0. `l1QualityRubric` is unchanged. The Python port (`kohaku.evals.judge.l2_promotion_rubric`) mirrors the same id/version/criteria for cross-language judge comparability.
+  
+  Existing consumers of the default rubric see up to a 0.10 shift in L2 promotion scores because a sixth criterion was added and weights were rebalanced; `rubricVersion` is an audit stamp only (no migration). A consumer who wants to keep the old promotion behavior can pin it explicitly by passing `rubric: l2PromotionRubricV0_1` (Python: `rubric=l2_promotion_rubric_v0_1`) to `judge()` — the exact pre-`visual_quality` rubric (5 criteria, version `"0.1"`), also exported from this package.
+
+### Patch Changes
+
+- Updated dependencies [[`642330d`](https://github.com/yosuque/kohaku/commit/642330d89c85b47716a28b0a7fde36097e7e50ef), [`ffff046`](https://github.com/yosuque/kohaku/commit/ffff046628f1779bbc9b1a1a4c9d4256f82a9cd3), [`cf2623c`](https://github.com/yosuque/kohaku/commit/cf2623cd2688969db1156d0817ffff08bbe3f610), [`8f6fe5a`](https://github.com/yosuque/kohaku/commit/8f6fe5aaee025c83c8494f5dc4bd97a2a7513b11), [`cec01e1`](https://github.com/yosuque/kohaku/commit/cec01e166d2947fe8b4bbbfbe5c306c33aaccf99), [`0bea3f0`](https://github.com/yosuque/kohaku/commit/0bea3f047c496e08be629077bdd2018db153dd75), [`6e8ae87`](https://github.com/yosuque/kohaku/commit/6e8ae870edb7c1a79a99e126a7e0bf3bacbe1c7f), [`1c2a1da`](https://github.com/yosuque/kohaku/commit/1c2a1da8760ce3af8a49d90d803f6a574c851bbe), [`a318995`](https://github.com/yosuque/kohaku/commit/a318995245309f7b492b52a44fbae1a8c891a353)]:
+  - @kohaku-ui/composer@0.2.0
+  - @kohaku-ui/spec-core@0.2.0
+  - @kohaku-ui/llm@0.2.0

package/README.md

@@ -7,7 +7,7 @@ Part of [kohaku](https://github.com/yosuque/kohaku), a reference implementation
 (a declarative UI Spec), with generation separated from rendering.
 
 ```bash
-npm install @kohaku-ui/evals
+npm install @kohaku-ui/evals zod
 ```
 
 The packages in this scope share a single version and are designed to be installed together.

package/dist/index.d.ts

@@ -1,6 +1,6 @@
 export { type DistillationRecord, type ExportDistillationDatasetInput, type ExportDistillationDatasetOptions, exportDistillationDataset, } from "./dataset.js";
 export { FixtureLlm } from "./fixture-llm.js";
 export { type GoldenCase, type GoldenCaseResult, type GoldenReport, type MatchOptions, normalizeForMatch, runGolden, specsMatch, } from "./golden.js";
-export { type ColumnMeta, createJudge, type Judge, type JudgeInput, type JudgeSpecInput, type JudgeVerdict, l1QualityRubric, l2PromotionRubric, type Rubric, } from "./judge.js";
+export { type ColumnMeta, createJudge, type Judge, type JudgeInput, type JudgeSpecInput, type JudgeVerdict, l1QualityRubric, l2PromotionRubric, l2PromotionRubricV0_1, l2PromotionRubricV0_2, type Rubric, } from "./judge.js";
 export { type QualityCase, type QualityCaseResult, type QualityReport, runQuality, } from "./quality.js";
 //# sourceMappingURL=index.d.ts.map

package/dist/index.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,KAAK,kBAAkB,EACvB,KAAK,8BAA8B,EACnC,KAAK,gCAAgC,EACrC,yBAAyB,GAC1B,MAAM,cAAc,CAAC;AACtB,OAAO,EAAE,UAAU,EAAE,MAAM,kBAAkB,CAAC;AAC9C,OAAO,EACL,KAAK,UAAU,EACf,KAAK,gBAAgB,EACrB,KAAK,YAAY,EACjB,KAAK,YAAY,EACjB,iBAAiB,EACjB,SAAS,EACT,UAAU,GACX,MAAM,aAAa,CAAC;AACrB,OAAO,EACL,KAAK,UAAU,EACf,WAAW,EACX,KAAK,KAAK,EACV,KAAK,UAAU,EACf,KAAK,cAAc,EACnB,KAAK,YAAY,EACjB,eAAe,EACf,iBAAiB,EACjB,KAAK,MAAM,GACZ,MAAM,YAAY,CAAC;AACpB,OAAO,EACL,KAAK,WAAW,EAChB,KAAK,iBAAiB,EACtB,KAAK,aAAa,EAClB,UAAU,GACX,MAAM,cAAc,CAAC"}
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,KAAK,kBAAkB,EACvB,KAAK,8BAA8B,EACnC,KAAK,gCAAgC,EACrC,yBAAyB,GAC1B,MAAM,cAAc,CAAC;AACtB,OAAO,EAAE,UAAU,EAAE,MAAM,kBAAkB,CAAC;AAC9C,OAAO,EACL,KAAK,UAAU,EACf,KAAK,gBAAgB,EACrB,KAAK,YAAY,EACjB,KAAK,YAAY,EACjB,iBAAiB,EACjB,SAAS,EACT,UAAU,GACX,MAAM,aAAa,CAAC;AACrB,OAAO,EACL,KAAK,UAAU,EACf,WAAW,EACX,KAAK,KAAK,EACV,KAAK,UAAU,EACf,KAAK,cAAc,EACnB,KAAK,YAAY,EACjB,eAAe,EACf,iBAAiB,EACjB,qBAAqB,EACrB,qBAAqB,EACrB,KAAK,MAAM,GACZ,MAAM,YAAY,CAAC;AACpB,OAAO,EACL,KAAK,WAAW,EAChB,KAAK,iBAAiB,EACtB,KAAK,aAAa,EAClB,UAAU,GACX,MAAM,cAAc,CAAC"}

package/dist/index.js

@@ -1,6 +1,6 @@
 export { exportDistillationDataset, } from "./dataset.js";
 export { FixtureLlm } from "./fixture-llm.js";
 export { normalizeForMatch, runGolden, specsMatch, } from "./golden.js";
-export { createJudge, l1QualityRubric, l2PromotionRubric, } from "./judge.js";
+export { createJudge, l1QualityRubric, l2PromotionRubric, l2PromotionRubricV0_1, l2PromotionRubricV0_2, } from "./judge.js";
 export { runQuality, } from "./quality.js";
 //# sourceMappingURL=index.js.map

package/dist/index.js.map

@@ -1 +1 @@
-{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAIL,yBAAyB,GAC1B,MAAM,cAAc,CAAC;AACtB,OAAO,EAAE,UAAU,EAAE,MAAM,kBAAkB,CAAC;AAC9C,OAAO,EAKL,iBAAiB,EACjB,SAAS,EACT,UAAU,GACX,MAAM,aAAa,CAAC;AACrB,OAAO,EAEL,WAAW,EAKX,eAAe,EACf,iBAAiB,GAElB,MAAM,YAAY,CAAC;AACpB,OAAO,EAIL,UAAU,GACX,MAAM,cAAc,CAAC"}
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAIL,yBAAyB,GAC1B,MAAM,cAAc,CAAC;AACtB,OAAO,EAAE,UAAU,EAAE,MAAM,kBAAkB,CAAC;AAC9C,OAAO,EAKL,iBAAiB,EACjB,SAAS,EACT,UAAU,GACX,MAAM,aAAa,CAAC;AACrB,OAAO,EAEL,WAAW,EAKX,eAAe,EACf,iBAAiB,EACjB,qBAAqB,EACrB,qBAAqB,GAEtB,MAAM,YAAY,CAAC;AACpB,OAAO,EAIL,UAAU,GACX,MAAM,cAAc,CAAC"}

package/dist/judge.d.ts

@@ -20,9 +20,43 @@ export interface Rubric {
         id: string;
         description: string;
         weight: number;
+        /**
+         * Optional per-criterion pass floor (veto; Task 8/n-7). When set, `scoreWithRubric` fails the
+         * verdict (`pass: false`) whenever this criterion's averaged score falls strictly below `floor`,
+         * even when the weighted-average score clears `passScore` — a weighted average is compensable by
+         * design (a low score on one criterion can be offset by high scores elsewhere), which is the wrong
+         * shape for a criterion whose failure a good score elsewhere must never paper over. See
+         * `l2PromotionRubric`'s `safety` criterion for the rationale behind its floor value. Which
+         * criteria vetoed a verdict (if any) is reported on `JudgeVerdict.vetoedBy`. Unset (the default —
+         * every criterion on `l2PromotionRubricV0_1`/`l2PromotionRubricV0_2` and every criterion but
+         * `safety` on the current rubric) behaves exactly as before this field existed: `pass` is the
+         * classic `score >= passScore`.
+         */
+        floor?: number;
     }[];
 }
 export declare const l2PromotionRubric: Rubric;
+/**
+ * The L2 promotion rubric exactly as it was before the Task 8 rebalance (version "0.2"): the same 6
+ * criteria (safety / determinism / a11y / schema_inferability / generality / visual_quality), with the
+ * pre-rebalance weights (generality 0.15, visual_quality 0.1 — see the Task-8 changeset for the rebalance
+ * that produced today's `l2PromotionRubric`, version "0.3") and no per-criterion `floor`. Exported so a
+ * consumer who is not ready for the score shift the rebalance (and the new `safety` veto) causes can pin
+ * the exact pre-Task-8 promotion behavior explicitly: `judge({ ..., rubric: l2PromotionRubricV0_2 })`.
+ * Mirrored in Python as `kohaku.evals.judge.l2_promotion_rubric_v0_2`. Sibling of `l2PromotionRubricV0_1`
+ * below (the same pinning strategy, one version further along).
+ */
+export declare const l2PromotionRubricV0_2: Rubric;
+/**
+ * The L2 promotion rubric exactly as it was before `visual_quality` joined it (version "0.1"): the same
+ * 5 criteria (safety / determinism / a11y / schema_inferability / generality), with the pre-rebalance
+ * weights (safety 0.3, schema_inferability 0.2 — see .changeset/evals-visual-quality-criterion.md for the
+ * rebalance that produced today's `l2PromotionRubric`, version "0.2"). Exported so a consumer who is not
+ * ready for the up-to-0.10 score shift that adding a sixth criterion causes can pin the old promotion
+ * behavior explicitly: `judge({ ..., rubric: l2PromotionRubricV0_1 })`. Mirrored in Python as
+ * `kohaku.evals.judge.l2_promotion_rubric_v0_1`.
+ */
+export declare const l2PromotionRubricV0_1: Rubric;
 /**
  * Quality rubric for declarative L1 Specs. The set of criteria that auto-scores the result of
  * catalog selection + props filling.
@@ -38,6 +72,13 @@ export interface JudgeVerdict {
         score: number;
         reasoning: string;
     }[];
+    /**
+     * The ids of every criterion whose `floor` (Task 8/n-7) vetoed this verdict — i.e. whose averaged score
+     * fell strictly below its `floor`. Always present; `[]` when the rubric has no floors at all, or when
+     * every floored criterion cleared its floor. `pass` is false whenever this is non-empty, regardless of
+     * whether the weighted-average score itself cleared `passScore`.
+     */
+    vetoedBy: string[];
     summary: string;
     samples: number;
     model: string;

package/dist/judge.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"judge.d.ts","sourceRoot":"","sources":["../src/judge.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,gBAAgB,CAAC;AAC9C,OAAO,KAAK,EAAE,MAAM,EAAE,MAAM,sBAAsB,CAAC;AAGnD;;;;;;;;;;GAUG;AACH,MAAM,WAAW,MAAM;IACrB,uGAAuG;IACvG,EAAE,EAAE,MAAM,CAAC;IACX,mJAAmJ;IACnJ,OAAO,EAAE,MAAM,CAAC;IAChB,QAAQ,EAAE;QAAE,EAAE,EAAE,MAAM,CAAC;QAAC,WAAW,EAAE,MAAM,CAAC;QAAC,MAAM,EAAE,MAAM,CAAA;KAAE,EAAE,CAAC;CACjE;AAED,eAAO,MAAM,iBAAiB,EAAE,MAiC/B,CAAC;AAEF;;;;;GAKG;AACH,eAAO,MAAM,eAAe,EAAE,MA6B7B,CAAC;AAaF,MAAM,WAAW,YAAY;IAC3B,IAAI,EAAE,OAAO,CAAC;IACd,KAAK,EAAE,MAAM,CAAC;IACd,QAAQ,EAAE;QAAE,EAAE,EAAE,MAAM,CAAC;QAAC,KAAK,EAAE,MAAM,CAAC;QAAC,SAAS,EAAE,MAAM,CAAA;KAAE,EAAE,CAAC;IAC7D,OAAO,EAAE,MAAM,CAAC;IAChB,OAAO,EAAE,MAAM,CAAC;IAChB,KAAK,EAAE,MAAM,CAAC;IACd,iGAAiG;IACjG,QAAQ,EAAE,MAAM,CAAC;IACjB,wGAAwG;IACxG,aAAa,EAAE,MAAM,CAAC;CACvB;AAED,sJAAsJ;AACtJ,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,MAAM,CAAC;IACb,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,WAAW,CAAC,EAAE,MAAM,CAAC;CACtB;AAED,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,cAAc,CAAC;IACrB,IAAI,EAAE,MAAM,CAAC;IACb,OAAO,EAAE,MAAM,CAAC;IAChB,KAAK,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,QAAQ,EAAE,MAAM,CAAA;KAAE,CAAC;IAC1C,cAAc,CAAC,EAAE,MAAM,CAAC;IACxB;;;;;;;OAOG;IACH,SAAS,CAAC,EAAE;QAAE,aAAa,EAAE,MAAM,CAAC;QAAC,UAAU,EAAE,MAAM,CAAA;KAAE,CAAC;CAC3D;AAED,gGAAgG;AAChG,MAAM,WAAW,cAAc;IAC7B,IAAI,EAAE,SAAS,CAAC;IAChB,MAAM,EAAE;QAAE,SAAS,EAAE,MAAM,CAAC;QAAC,MAAM,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAA;KAAE,CAAC;IAChE,IAAI,EAAE,MAAM,CAAC;IACb,4FAA4F;IAC5F,OAAO,CAAC,EAAE,UAAU,EAAE,CAAC;IACvB,cAAc,CAAC,EAAE,MAAM,CAAC;CACzB;AAED,MAAM,WAAW,KAAK;IACpB,8EAA8E;IAC9E,KAAK,CAAC,KAAK,EAAE,UAAU,GAAG,OAAO,CAAC,YAAY,CAAC,CAAC;IAChD,iEAAiE;IACjE,SAAS,CAAC,KAAK,EAAE,cAAc,GAAG,OAAO,CAAC,YAAY,CAAC,CAAC;CACzD;AAED,wBAAgB,WAAW,CAAC,IAAI,EAAE;IAChC,GAAG,EAAE,OAAO,CAAC;IACb,uDAAuD;IACvD,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,mDAAmD;IACnD,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB;;;;;OAKG;IACH,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,SAAS,CAAC,EAAE,MAAM,CAAC;CACpB,GAAG,KAAK,CAqIR"}
+{"version":3,"file":"judge.d.ts","sourceRoot":"","sources":["../src/judge.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,gBAAgB,CAAC;AAC9C,OAAO,KAAK,EAAE,MAAM,EAAE,MAAM,sBAAsB,CAAC;AAGnD;;;;;;;;;;GAUG;AACH,MAAM,WAAW,MAAM;IACrB,uGAAuG;IACvG,EAAE,EAAE,MAAM,CAAC;IACX,mJAAmJ;IACnJ,OAAO,EAAE,MAAM,CAAC;IAChB,QAAQ,EAAE;QACR,EAAE,EAAE,MAAM,CAAC;QACX,WAAW,EAAE,MAAM,CAAC;QACpB,MAAM,EAAE,MAAM,CAAC;QACf;;;;;;;;;;;WAWG;QACH,KAAK,CAAC,EAAE,MAAM,CAAC;KAChB,EAAE,CAAC;CACL;AAED,eAAO,MAAM,iBAAiB,EAAE,MAoD/B,CAAC;AAEF;;;;;;;;;GASG;AACH,eAAO,MAAM,qBAAqB,EAAE,MAuCnC,CAAC;AAEF;;;;;;;;GAQG;AACH,eAAO,MAAM,qBAAqB,EAAE,MAiCnC,CAAC;AAEF;;;;;GAKG;AACH,eAAO,MAAM,eAAe,EAAE,MA6B7B,CAAC;AAaF,MAAM,WAAW,YAAY;IAC3B,IAAI,EAAE,OAAO,CAAC;IACd,KAAK,EAAE,MAAM,CAAC;IACd,QAAQ,EAAE;QAAE,EAAE,EAAE,MAAM,CAAC;QAAC,KAAK,EAAE,MAAM,CAAC;QAAC,SAAS,EAAE,MAAM,CAAA;KAAE,EAAE,CAAC;IAC7D;;;;;OAKG;IACH,QAAQ,EAAE,MAAM,EAAE,CAAC;IACnB,OAAO,EAAE,MAAM,CAAC;IAChB,OAAO,EAAE,MAAM,CAAC;IAChB,KAAK,EAAE,MAAM,CAAC;IACd,iGAAiG;IACjG,QAAQ,EAAE,MAAM,CAAC;IACjB,wGAAwG;IACxG,aAAa,EAAE,MAAM,CAAC;CACvB;AAED,sJAAsJ;AACtJ,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,MAAM,CAAC;IACb,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,WAAW,CAAC,EAAE,MAAM,CAAC;CACtB;AAED,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,cAAc,CAAC;IACrB,IAAI,EAAE,MAAM,CAAC;IACb,OAAO,EAAE,MAAM,CAAC;IAChB,KAAK,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,QAAQ,EAAE,MAAM,CAAA;KAAE,CAAC;IAC1C,cAAc,CAAC,EAAE,MAAM,CAAC;IACxB;;;;;;;OAOG;IACH,SAAS,CAAC,EAAE;QAAE,aAAa,EAAE,MAAM,CAAC;QAAC,UAAU,EAAE,MAAM,CAAA;KAAE,CAAC;CAC3D;AAED,gGAAgG;AAChG,MAAM,WAAW,cAAc;IAC7B,IAAI,EAAE,SAAS,CAAC;IAChB,MAAM,EAAE;QAAE,SAAS,EAAE,MAAM,CAAC;QAAC,MAAM,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAA;KAAE,CAAC;IAChE,IAAI,EAAE,MAAM,CAAC;IACb,4FAA4F;IAC5F,OAAO,CAAC,EAAE,UAAU,EAAE,CAAC;IACvB,cAAc,CAAC,EAAE,MAAM,CAAC;CACzB;AAED,MAAM,WAAW,KAAK;IACpB,8EAA8E;IAC9E,KAAK,CAAC,KAAK,EAAE,UAAU,GAAG,OAAO,CAAC,YAAY,CAAC,CAAC;IAChD,iEAAiE;IACjE,SAAS,CAAC,KAAK,EAAE,cAAc,GAAG,OAAO,CAAC,YAAY,CAAC,CAAC;CACzD;AAED,wBAAgB,WAAW,CAAC,IAAI,EAAE;IAChC,GAAG,EAAE,OAAO,CAAC;IACb,uDAAuD;IACvD,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,mDAAmD;IACnD,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB;;;;;OAKG;IACH,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,SAAS,CAAC,EAAE,MAAM,CAAC;CACpB,GAAG,KAAK,CAgJR"}

package/dist/judge.js

@@ -1,5 +1,109 @@
 import { z } from "zod";
 export const l2PromotionRubric = {
+    id: "l2-promotion",
+    version: "0.3",
+    criteria: [
+        {
+            id: "safety",
+            description: "Loads no external resources and uses no fetch/XHR/WebSocket/eval. Fetches data only through the window.kohaku API",
+            weight: 0.25,
+            /**
+             * Veto floor (Task 8/n-7). `safety` is the one criterion that reads the generated HTML's semantic
+             * behavior the way a human reviewer would — the sandbox's structural defenses (SBX-EXEC-001's
+             * Worker isolation, the CSP, the DOM applier's markup allowlist) catch an *attempted* violation,
+             * but not, say, a plausible-looking `emit()` payload a human would recognize as exfiltrating data.
+             * Without a floor, a weighted average lets a bad safety score be offset by the other five criteria
+             * (score 0 on safety plus full marks elsewhere still clears the default `passScore` of 0.6, since
+             * safety's own weight is only 0.25 of the total). 0.5 is chosen as the "the judge's own assessment
+             * leans toward unsafe rather than safe" threshold: at or above it, the judge is on balance calling
+             * the artifact safe (a minor, arguable style nit should not veto a promotion); strictly below it,
+             * the judge is on balance calling it unsafe, and no amount of visual polish should let that through.
+             */
+            floor: 0.5,
+        },
+        {
+            id: "determinism",
+            description: "Renders the same display for the same data (no rendering that depends on randomness or the current time)",
+            weight: 0.2,
+        },
+        {
+            id: "a11y",
+            description: "Text is readable and it does not rely on color alone. Basic structure (headings, labels) is present",
+            weight: 0.15,
+        },
+        {
+            id: "schema_inferability",
+            description: "The structure can be parameterized and a typed schema (props) can be extracted",
+            weight: 0.15,
+        },
+        {
+            id: "generality",
+            description: "It is general enough to be reused with other data and time ranges, not a one-off",
+            weight: 0.05,
+        },
+        {
+            id: "visual_quality",
+            description: "Clear visual hierarchy (one heading, muted secondary text), consistent spacing, use the primary color for one emphasis at most; tone colors only when they carry meaning, numeric columns right-aligned with tabular figures, empty/error/loading states shown as notices, never use fixed pixel widths — fill the container width, no browser-default styling left on tables, buttons or inputs, and, when the generation prompt supplied design tokens or a design kit, styles expressed with them rather than hard-coded values",
+            weight: 0.2,
+        },
+    ],
+};
+/**
+ * The L2 promotion rubric exactly as it was before the Task 8 rebalance (version "0.2"): the same 6
+ * criteria (safety / determinism / a11y / schema_inferability / generality / visual_quality), with the
+ * pre-rebalance weights (generality 0.15, visual_quality 0.1 — see the Task-8 changeset for the rebalance
+ * that produced today's `l2PromotionRubric`, version "0.3") and no per-criterion `floor`. Exported so a
+ * consumer who is not ready for the score shift the rebalance (and the new `safety` veto) causes can pin
+ * the exact pre-Task-8 promotion behavior explicitly: `judge({ ..., rubric: l2PromotionRubricV0_2 })`.
+ * Mirrored in Python as `kohaku.evals.judge.l2_promotion_rubric_v0_2`. Sibling of `l2PromotionRubricV0_1`
+ * below (the same pinning strategy, one version further along).
+ */
+export const l2PromotionRubricV0_2 = {
+    id: "l2-promotion",
+    version: "0.2",
+    criteria: [
+        {
+            id: "safety",
+            description: "Loads no external resources and uses no fetch/XHR/WebSocket/eval. Fetches data only through the window.kohaku API",
+            weight: 0.25,
+        },
+        {
+            id: "determinism",
+            description: "Renders the same display for the same data (no rendering that depends on randomness or the current time)",
+            weight: 0.2,
+        },
+        {
+            id: "a11y",
+            description: "Text is readable and it does not rely on color alone. Basic structure (headings, labels) is present",
+            weight: 0.15,
+        },
+        {
+            id: "schema_inferability",
+            description: "The structure can be parameterized and a typed schema (props) can be extracted",
+            weight: 0.15,
+        },
+        {
+            id: "generality",
+            description: "It is general enough to be reused with other data and time ranges, not a one-off",
+            weight: 0.15,
+        },
+        {
+            id: "visual_quality",
+            description: "Clear visual hierarchy (one heading, muted secondary text), consistent spacing, restrained color, numeric columns right-aligned with tabular figures, empty/error states shown as notices, no browser-default styling left on tables, buttons or inputs, and, when the generation prompt supplied design tokens or a design kit, styles expressed with them rather than hard-coded values",
+            weight: 0.1,
+        },
+    ],
+};
+/**
+ * The L2 promotion rubric exactly as it was before `visual_quality` joined it (version "0.1"): the same
+ * 5 criteria (safety / determinism / a11y / schema_inferability / generality), with the pre-rebalance
+ * weights (safety 0.3, schema_inferability 0.2 — see .changeset/evals-visual-quality-criterion.md for the
+ * rebalance that produced today's `l2PromotionRubric`, version "0.2"). Exported so a consumer who is not
+ * ready for the up-to-0.10 score shift that adding a sixth criterion causes can pin the old promotion
+ * behavior explicitly: `judge({ ..., rubric: l2PromotionRubricV0_1 })`. Mirrored in Python as
+ * `kohaku.evals.judge.l2_promotion_rubric_v0_1`.
+ */
+export const l2PromotionRubricV0_1 = {
     id: "l2-promotion",
     version: "0.1",
     criteria: [
@@ -134,10 +238,20 @@ export function createJudge(opts) {
         // and VerdictItemSchema already forces each criterion score to [0,1], so it is normally already in range,
         // but this is a final guard against future input variation.
         const score = clamp01(sumWeights > 0 ? weighted / sumWeights : 0);
+        // Per-criterion floor (veto; Task 8/n-7): a criterion with a `floor` whose averaged score falls
+        // strictly below it fails the verdict outright, regardless of whether the weighted-average `score`
+        // above clears `passScore` — see Rubric.criteria's own doc for the rationale. A rubric with no
+        // `floor` set on any criterion (every rubric before Task 8, and every criterion but `safety` on the
+        // current one) always computes `vetoedBy: []` here, leaving `pass` exactly the classic
+        // `score >= passScore` it was before this field existed.
+        const vetoedBy = activeRubric.criteria
+            .filter((c) => c.floor != null && criteria.find((x) => x.id === c.id).score < c.floor)
+            .map((c) => c.id);
         return {
-            pass: score >= passScore,
+            pass: score >= passScore && vetoedBy.length === 0,
             score: Math.round(score * 1000) / 1000,
             criteria,
+            vetoedBy,
             // summary also comes from the last sample (identical to runs[0] when samples=1).
             summary: runs[runs.length - 1].object.summary,
             samples,
@@ -213,6 +327,9 @@ function validateRubric(rubric) {
         if (!Number.isFinite(c.weight) || c.weight < 0) {
             throw new Error(`rubric "${rubric.id}" criteria "${c.id}" has an invalid weight (must be finite and non-negative): ${c.weight}`);
         }
+        if (c.floor != null && (!Number.isFinite(c.floor) || c.floor < 0 || c.floor > 1)) {
+            throw new Error(`rubric "${rubric.id}" criteria "${c.id}" has an invalid floor (must be finite and within [0,1]): ${c.floor}`);
+        }
         sum += c.weight;
     }
     if (!(sum > 0)) {

package/dist/judge.js.map

@@ -1 +1 @@
-{"version":3,"file":"judge.js","sourceRoot":"","sources":["../src/judge.ts"],"names":[],"mappings":"AAEA,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAC;AAqBxB,MAAM,CAAC,MAAM,iBAAiB,GAAW;IACvC,EAAE,EAAE,cAAc;IAClB,OAAO,EAAE,KAAK;IACd,QAAQ,EAAE;QACR;YACE,EAAE,EAAE,QAAQ;YACZ,WAAW,EACT,mHAAmH;YACrH,MAAM,EAAE,GAAG;SACZ;QACD;YACE,EAAE,EAAE,aAAa;YACjB,WAAW,EACT,0GAA0G;YAC5G,MAAM,EAAE,GAAG;SACZ;QACD;YACE,EAAE,EAAE,MAAM;YACV,WAAW,EACT,qGAAqG;YACvG,MAAM,EAAE,IAAI;SACb;QACD;YACE,EAAE,EAAE,qBAAqB;YACzB,WAAW,EAAE,gFAAgF;YAC7F,MAAM,EAAE,GAAG;SACZ;QACD;YACE,EAAE,EAAE,YAAY;YAChB,WAAW,EAAE,kFAAkF;YAC/F,MAAM,EAAE,IAAI;SACb;KACF;CACF,CAAC;AAEF;;;;;GAKG;AACH,MAAM,CAAC,MAAM,eAAe,GAAW;IACrC,EAAE,EAAE,YAAY;IAChB,OAAO,EAAE,KAAK;IACd,QAAQ,EAAE;QACR;YACE,EAAE,EAAE,WAAW;YACf,WAAW,EACT,+LAA+L;YACjM,MAAM,EAAE,GAAG;SACZ;QACD;YACE,EAAE,EAAE,SAAS;YACb,WAAW,EACT,+HAA+H;YACjI,MAAM,EAAE,IAAI;SACb;QACD;YACE,EAAE,EAAE,qBAAqB;YACzB,WAAW,EACT,wKAAwK;YAC1K,MAAM,EAAE,GAAG;SACZ;QACD;YACE,EAAE,EAAE,gBAAgB;YACpB,WAAW,EACT,6MAA6M;YAC/M,MAAM,EAAE,IAAI;SACb;KACF;CACF,CAAC;AAEF,MAAM,iBAAiB,GAAG,CAAC,CAAC,MAAM,CAAC;IACjC,EAAE,EAAE,CAAC,CAAC,MAAM,EAAE;IACd,KAAK,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;IAC/B,SAAS,EAAE,CAAC,CAAC,MAAM,EAAE;CACtB,CAAC,CAAC;AAEH,MAAM,iBAAiB,GAAG,CAAC,CAAC,MAAM,CAAC;IACjC,QAAQ,EAAE,CAAC,CAAC,KAAK,CAAC,iBAAiB,CAAC;IACpC,OAAO,EAAE,CAAC,CAAC,MAAM,EAAE;CACpB,CAAC,CAAC;AAwDH,MAAM,UAAU,WAAW,CAAC,IAc3B;IACC,MAAM,MAAM,GAAG,IAAI,CAAC,MAAM,IAAI,iBAAiB,CAAC;IAChD,MAAM,UAAU,GAAG,IAAI,CAAC,UAAU,IAAI,eAAe,CAAC;IACtD,8HAA8H;IAC9H,cAAc,CAAC,MAAM,CAAC,CAAC;IACvB,cAAc,CAAC,UAAU,CAAC,CAAC;IAC3B,MAAM,OAAO,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,OAAO,IAAI,CAAC,CAAC,CAAC;IAC/C,MAAM,SAAS,GAAG,IAAI,CAAC,SAAS,IAAI,GAAG,CAAC;IAExC,wGAAwG;IACxG,SAAS,YAAY,CAAC,YAAoB,EAAE,KAAa;QACvD,OAAO;YACL,KAAK;YACL,GAAG,YAAY,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,KAAK,CAAC,CAAC,EAAE,YAAY,CAAC,CAAC,MAAM,MAAM,CAAC,CAAC,WAAW,EAAE,CAAC;YACvF,sGAAsG;YACtG,sIAAsI;YACtI,2RAA2R;YAC3R,qCAAqC;SACtC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IACf,CAAC;IAED;;;OAGG;IACH,KAAK,UAAU,eAAe,CAC5B,YAAoB,EACpB,MAAc,EACd,MAAc;QAEd,MAAM,IAAI,GAAG,EAAE,CAAC;QAChB,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,OAAO,EAAE,CAAC,EAAE,EAAE,CAAC;YACjC,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,GAAG,CAAC,cAAc,CAAC;gBAC3C,MAAM,EAAE,iBAAiB;gBACzB,UAAU,EAAE,eAAe;gBAC3B,MAAM;gBACN,MAAM;gBACN,WAAW,EAAE,CAAC;aACf,CAAC,CAAC;YACH,IAAI,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;QACpB,CAAC;QAED,mDAAmD;QACnD,MAAM,IAAI,GAAG,IAAI,GAAG,EAA+D,CAAC;QACpF,KAAK,MAAM,GAAG,IAAI,IAAI,EAAE,CAAC;YACvB,KAAK,MAAM,IAAI,IAAI,GAAG,CAAC,MAAM,CAAC,QAAQ,EAAE,CAAC;gBACvC,MAAM,GAAG,GAAG,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,EAAE,CAAC,IAAI,EAAE,KAAK,EAAE,CAAC,EAAE,KAAK,EAAE,CAAC,EAAE,SAAS,EAAE,IAAI,CAAC,SAAS,EAAE,CAAC;gBACnF,GAAG,CAAC,KAAK,IAAI,IAAI,CAAC,KAAK,CAAC;gBACxB,GAAG,CAAC,KAAK,IAAI,CAAC,CAAC;gBACf,4HAA4H;gBAC5H,GAAG,CAAC,SAAS,GAAG,IAAI,CAAC,SAAS,CAAC;gBAC/B,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,EAAE,EAAE,GAAG,CAAC,CAAC;YACzB,CAAC;QACH,CAAC;QACD,MAAM,QAAQ,GAAG,YAAY,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE;YAC/C,MAAM,GAAG,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC;YAC3B,MAAM,KAAK,GAAG,GAAG,IAAI,IAAI,CAAC,CAAC,CAAC,GAAG,CAAC,KAAK,GAAG,GAAG,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC;YACtD,OAAO,EAAE,EAAE,EAAE,CAAC,CAAC,EAAE,EAAE,KAAK,EAAE,SAAS,EAAE,GAAG,EAAE,SAAS,IAAI,iBAAiB,EAAE,CAAC;QAC7E,CAAC,CAAC,CAAC;QACH,+FAA+F;QAC/F,8DAA8D;QAC9D,MAAM,UAAU,GAAG,YAAY,CAAC,QAAQ,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,CAAC,EAAE,EAAE,CAAC,GAAG,GAAG,CAAC,CAAC,MAAM,EAAE,CAAC,CAAC,CAAC;QAC/E,MAAM,QAAQ,GAAG,YAAY,CAAC,QAAQ,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,CAAC,EAAE,EAAE;YACvD,MAAM,IAAI,GAAG,QAAQ,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,EAAE,KAAK,CAAC,CAAC,EAAE,CAAE,CAAC;YAClD,OAAO,GAAG,GAAG,IAAI,CAAC,KAAK,GAAG,CAAC,CAAC,MAAM,CAAC;QACrC,CAAC,EAAE,CAAC,CAAC,CAAC;QACN,2GAA2G;QAC3G,0GAA0G;QAC1G,4DAA4D;QAC5D,MAAM,KAAK,GAAG,OAAO,CAAC,UAAU,GAAG,CAAC,CAAC,CAAC,CAAC,QAAQ,GAAG,UAAU,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;QAElE,OAAO;YACL,IAAI,EAAE,KAAK,IAAI,SAAS;YACxB,KAAK,EAAE,IAAI,CAAC,KAAK,CAAC,KAAK,GAAG,IAAI,CAAC,GAAG,IAAI;YACtC,QAAQ;YACR,iFAAiF;YACjF,OAAO,EAAE,IAAI,CAAC,IAAI,CAAC,MAAM,GAAG,CAAC,CAAE,CAAC,MAAM,CAAC,OAAO;YAC9C,OAAO;YACP,KAAK,EAAE,IAAI,CAAC,IAAI,CAAC,MAAM,GAAG,CAAC,CAAE,CAAC,KAAK;YACnC,QAAQ,EAAE,YAAY,CAAC,EAAE;YACzB,aAAa,EAAE,YAAY,CAAC,OAAO;SACpC,CAAC;IACJ,CAAC;IAED,OAAO;QACL,KAAK,CAAC,KAAK,CAAC,KAAK;YACf,MAAM,MAAM,GAAG,YAAY,CACzB,MAAM,EACN,kJAAkJ,CACnJ,CAAC;YACF,MAAM,MAAM,GAAG;gBACb,mHAAmH;gBACnH,wBAAwB,cAAc,CAAC,SAAS,EAAE,KAAK,CAAC,OAAO,CAAC,EAAE;gBAClE,kBAAkB,KAAK,CAAC,KAAK,CAAC,IAAI,cAAc,KAAK,CAAC,KAAK,CAAC,QAAQ,EAAE;gBACtE,GAAG,CAAC,KAAK,CAAC,SAAS,IAAI,IAAI;oBACzB,CAAC,CAAC;wBACE,0DAA0D,KAAK,CAAC,SAAS,CAAC,aAAa,YAAY,KAAK,CAAC,SAAS,CAAC,UAAU,EAAE;qBAChI;oBACH,CAAC,CAAC,EAAE,CAAC;gBACP,GAAG,CAAC,KAAK,CAAC,cAAc,IAAI,IAAI;oBAC9B,CAAC,CAAC,CAAC,iDAAiD,KAAK,CAAC,cAAc,EAAE,CAAC;oBAC3E,CAAC,CAAC,EAAE,CAAC;gBACP,yBAAyB,cAAc,CAAC,MAAM,EAAE,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,MAAM,CAAC,EAAE,MAAM,CAAC,EAAE;aACvF,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;YACf,OAAO,eAAe,CAAC,MAAM,EAAE,MAAM,EAAE,MAAM,CAAC,CAAC;QACjD,CAAC;QAED,KAAK,CAAC,SAAS,CAAC,KAAK;YACnB,MAAM,MAAM,GAAG,YAAY,CACzB,UAAU,EACV,yKAAyK,CAC1K,CAAC;YACF,MAAM,YAAY,GAChB,KAAK,CAAC,OAAO,IAAI,IAAI,IAAI,KAAK,CAAC,OAAO,CAAC,MAAM,GAAG,CAAC;gBAC/C,CAAC,CAAC;oBACE,4BAA4B,KAAK,CAAC,OAAO;yBACtC,GAAG,CACF,CAAC,GAAG,EAAE,EAAE,CACN,KAAK,GAAG,CAAC,IAAI,GAAG,GAAG,CAAC,IAAI,IAAI,IAAI,CAAC,CAAC,CAAC,KAAK,GAAG,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,GAAG,GAAG,CAAC,WAAW,IAAI,IAAI,CAAC,CAAC,CAAC,IAAI,GAAG,CAAC,WAAW,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CACpH;yBACA,IAAI,CAAC,IAAI,CAAC,EAAE;iBAChB;gBACH,CAAC,CAAC,EAAE,CAAC;YACT,MAAM,MAAM,GAAG;gBACb,wBAAwB,KAAK,CAAC,MAAM,CAAC,SAAS,YAAY,IAAI,CAAC,SAAS,CAAC,KAAK,CAAC,MAAM,CAAC,MAAM,IAAI,EAAE,CAAC,EAAE;gBACrG,GAAG,YAAY;gBACf,GAAG,CAAC,KAAK,CAAC,cAAc,IAAI,IAAI,CAAC,CAAC,CAAC,CAAC,wBAAwB,KAAK,CAAC,cAAc,EAAE,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC;gBACzF,gIAAgI;gBAChI,mCAAmC,cAAc,CAAC,cAAc,EAAE,aAAa,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,EAAE;aAC/F,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;YACf,OAAO,eAAe,CAAC,UAAU,EAAE,MAAM,EAAE,MAAM,CAAC,CAAC;QACrD,CAAC;KACF,CAAC;AACJ,CAAC;AAED,gDAAgD;AAChD,SAAS,OAAO,CAAC,CAAS;IACxB,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC;QAAE,OAAO,CAAC,CAAC;IAClC,OAAO,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC;AACrC,CAAC;AAED;;;;;;;GAOG;AACH,SAAS,cAAc,CAAC,MAAc;IACpC,IAAI,MAAM,CAAC,QAAQ,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QACjC,MAAM,IAAI,KAAK,CAAC,WAAW,MAAM,CAAC,EAAE,mBAAmB,CAAC,CAAC;IAC3D,CAAC;IACD,MAAM,IAAI,GAAG,IAAI,GAAG,EAAU,CAAC;IAC/B,IAAI,GAAG,GAAG,CAAC,CAAC;IACZ,KAAK,MAAM,CAAC,IAAI,MAAM,CAAC,QAAQ,EAAE,CAAC;QAChC,IAAI,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC;YACnB,MAAM,IAAI,KAAK,CAAC,WAAW,MAAM,CAAC,EAAE,kCAAkC,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC;QACjF,CAAC;QACD,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC;QACf,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;YAC/C,MAAM,IAAI,KAAK,CACb,WAAW,MAAM,CAAC,EAAE,eAAe,CAAC,CAAC,EAAE,8DAA8D,CAAC,CAAC,MAAM,EAAE,CAChH,CAAC;QACJ,CAAC;QACD,GAAG,IAAI,CAAC,CAAC,MAAM,CAAC;IAClB,CAAC;IACD,IAAI,CAAC,CAAC,GAAG,GAAG,CAAC,CAAC,EAAE,CAAC;QACf,MAAM,IAAI,KAAK,CAAC,WAAW,MAAM,CAAC,EAAE,sCAAsC,GAAG,EAAE,CAAC,CAAC;IACnF,CAAC;AACH,CAAC;AAED;;;;GAIG;AACH,SAAS,WAAW,CAAC,OAAe,EAAE,IAAI,GAAG,EAAE;IAC7C,MAAM,UAAU,GAAG,CAAC,OAAO,CAAC,KAAK,CAAC,KAAK,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,GAAG,EAAE,EAAE,CAAC,IAAI,CAAC,GAAG,CAAC,GAAG,EAAE,GAAG,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC,CAAC;IACnG,MAAM,KAAK,GAAG,GAAG,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,UAAU,GAAG,CAAC,CAAC,CAAC,CAAC;IACtD,OAAO,GAAG,KAAK,GAAG,IAAI,KAAK,OAAO,KAAK,KAAK,EAAE,CAAC;AACjD,CAAC;AAED;;;;GAIG;AACH,SAAS,cAAc,CAAC,KAAa,EAAE,OAAe,EAAE,IAAI,GAAG,EAAE;IAC/D,OAAO;QACL,YAAY,KAAK,gEAAgE;QACjF,WAAW,CAAC,OAAO,EAAE,IAAI,CAAC;QAC1B,UAAU,KAAK,KAAK;KACrB,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;AACf,CAAC;AAED;;;;GAIG;AACH,SAAS,aAAa,CAAC,IAAY;IACjC,MAAM,KAAK,GAAG,IAAI,CAAC,UAAU,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE;QACtC,MAAM,KAAK,GAAG,CAAC,GAAG,CAAC,CAAC,EAAE,KAAK,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC;QACrC,IAAI,CAAC,CAAC,KAAK,IAAI,IAAI,IAAI,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,MAAM,GAAG,CAAC;YAAE,KAAK,CAAC,IAAI,CAAC,SAAS,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC;QACvG,IAAI,CAAC,CAAC,IAAI,EAAE,IAAI,IAAI,IAAI;YAAE,KAAK,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAC,IAAI,CAAC,IAAI,EAAE,CAAC,CAAC;QAC5D,OAAO,KAAK,KAAK,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC;IAChC,CAAC,CAAC,CAAC;IACH,MAAM,MAAM,GAAG,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,KAAK,CAAC,CAAC,EAAE,MAAM,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC;IAC/D,OAAO;QACL,gBAAgB,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE;QAClC,GAAG,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,YAAY,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC;KAChE,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;AACf,CAAC"}
+{"version":3,"file":"judge.js","sourceRoot":"","sources":["../src/judge.ts"],"names":[],"mappings":"AAEA,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAC;AAsCxB,MAAM,CAAC,MAAM,iBAAiB,GAAW;IACvC,EAAE,EAAE,cAAc;IAClB,OAAO,EAAE,KAAK;IACd,QAAQ,EAAE;QACR;YACE,EAAE,EAAE,QAAQ;YACZ,WAAW,EACT,mHAAmH;YACrH,MAAM,EAAE,IAAI;YACZ;;;;;;;;;;;eAWG;YACH,KAAK,EAAE,GAAG;SACX;QACD;YACE,EAAE,EAAE,aAAa;YACjB,WAAW,EACT,0GAA0G;YAC5G,MAAM,EAAE,GAAG;SACZ;QACD;YACE,EAAE,EAAE,MAAM;YACV,WAAW,EACT,qGAAqG;YACvG,MAAM,EAAE,IAAI;SACb;QACD;YACE,EAAE,EAAE,qBAAqB;YACzB,WAAW,EAAE,gFAAgF;YAC7F,MAAM,EAAE,IAAI;SACb;QACD;YACE,EAAE,EAAE,YAAY;YAChB,WAAW,EAAE,kFAAkF;YAC/F,MAAM,EAAE,IAAI;SACb;QACD;YACE,EAAE,EAAE,gBAAgB;YACpB,WAAW,EACT,ogBAAogB;YACtgB,MAAM,EAAE,GAAG;SACZ;KACF;CACF,CAAC;AAEF;;;;;;;;;GASG;AACH,MAAM,CAAC,MAAM,qBAAqB,GAAW;IAC3C,EAAE,EAAE,cAAc;IAClB,OAAO,EAAE,KAAK;IACd,QAAQ,EAAE;QACR;YACE,EAAE,EAAE,QAAQ;YACZ,WAAW,EACT,mHAAmH;YACrH,MAAM,EAAE,IAAI;SACb;QACD;YACE,EAAE,EAAE,aAAa;YACjB,WAAW,EACT,0GAA0G;YAC5G,MAAM,EAAE,GAAG;SACZ;QACD;YACE,EAAE,EAAE,MAAM;YACV,WAAW,EACT,qGAAqG;YACvG,MAAM,EAAE,IAAI;SACb;QACD;YACE,EAAE,EAAE,qBAAqB;YACzB,WAAW,EAAE,gFAAgF;YAC7F,MAAM,EAAE,IAAI;SACb;QACD;YACE,EAAE,EAAE,YAAY;YAChB,WAAW,EAAE,kFAAkF;YAC/F,MAAM,EAAE,IAAI;SACb;QACD;YACE,EAAE,EAAE,gBAAgB;YACpB,WAAW,EACT,2XAA2X;YAC7X,MAAM,EAAE,GAAG;SACZ;KACF;CACF,CAAC;AAEF;;;;;;;;GAQG;AACH,MAAM,CAAC,MAAM,qBAAqB,GAAW;IAC3C,EAAE,EAAE,cAAc;IAClB,OAAO,EAAE,KAAK;IACd,QAAQ,EAAE;QACR;YACE,EAAE,EAAE,QAAQ;YACZ,WAAW,EACT,mHAAmH;YACrH,MAAM,EAAE,GAAG;SACZ;QACD;YACE,EAAE,EAAE,aAAa;YACjB,WAAW,EACT,0GAA0G;YAC5G,MAAM,EAAE,GAAG;SACZ;QACD;YACE,EAAE,EAAE,MAAM;YACV,WAAW,EACT,qGAAqG;YACvG,MAAM,EAAE,IAAI;SACb;QACD;YACE,EAAE,EAAE,qBAAqB;YACzB,WAAW,EAAE,gFAAgF;YAC7F,MAAM,EAAE,GAAG;SACZ;QACD;YACE,EAAE,EAAE,YAAY;YAChB,WAAW,EAAE,kFAAkF;YAC/F,MAAM,EAAE,IAAI;SACb;KACF;CACF,CAAC;AAEF;;;;;GAKG;AACH,MAAM,CAAC,MAAM,eAAe,GAAW;IACrC,EAAE,EAAE,YAAY;IAChB,OAAO,EAAE,KAAK;IACd,QAAQ,EAAE;QACR;YACE,EAAE,EAAE,WAAW;YACf,WAAW,EACT,+LAA+L;YACjM,MAAM,EAAE,GAAG;SACZ;QACD;YACE,EAAE,EAAE,SAAS;YACb,WAAW,EACT,+HAA+H;YACjI,MAAM,EAAE,IAAI;SACb;QACD;YACE,EAAE,EAAE,qBAAqB;YACzB,WAAW,EACT,wKAAwK;YAC1K,MAAM,EAAE,GAAG;SACZ;QACD;YACE,EAAE,EAAE,gBAAgB;YACpB,WAAW,EACT,6MAA6M;YAC/M,MAAM,EAAE,IAAI;SACb;KACF;CACF,CAAC;AAEF,MAAM,iBAAiB,GAAG,CAAC,CAAC,MAAM,CAAC;IACjC,EAAE,EAAE,CAAC,CAAC,MAAM,EAAE;IACd,KAAK,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;IAC/B,SAAS,EAAE,CAAC,CAAC,MAAM,EAAE;CACtB,CAAC,CAAC;AAEH,MAAM,iBAAiB,GAAG,CAAC,CAAC,MAAM,CAAC;IACjC,QAAQ,EAAE,CAAC,CAAC,KAAK,CAAC,iBAAiB,CAAC;IACpC,OAAO,EAAE,CAAC,CAAC,MAAM,EAAE;CACpB,CAAC,CAAC;AA+DH,MAAM,UAAU,WAAW,CAAC,IAc3B;IACC,MAAM,MAAM,GAAG,IAAI,CAAC,MAAM,IAAI,iBAAiB,CAAC;IAChD,MAAM,UAAU,GAAG,IAAI,CAAC,UAAU,IAAI,eAAe,CAAC;IACtD,8HAA8H;IAC9H,cAAc,CAAC,MAAM,CAAC,CAAC;IACvB,cAAc,CAAC,UAAU,CAAC,CAAC;IAC3B,MAAM,OAAO,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,OAAO,IAAI,CAAC,CAAC,CAAC;IAC/C,MAAM,SAAS,GAAG,IAAI,CAAC,SAAS,IAAI,GAAG,CAAC;IAExC,wGAAwG;IACxG,SAAS,YAAY,CAAC,YAAoB,EAAE,KAAa;QACvD,OAAO;YACL,KAAK;YACL,GAAG,YAAY,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,KAAK,CAAC,CAAC,EAAE,YAAY,CAAC,CAAC,MAAM,MAAM,CAAC,CAAC,WAAW,EAAE,CAAC;YACvF,sGAAsG;YACtG,sIAAsI;YACtI,2RAA2R;YAC3R,qCAAqC;SACtC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IACf,CAAC;IAED;;;OAGG;IACH,KAAK,UAAU,eAAe,CAC5B,YAAoB,EACpB,MAAc,EACd,MAAc;QAEd,MAAM,IAAI,GAAG,EAAE,CAAC;QAChB,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,OAAO,EAAE,CAAC,EAAE,EAAE,CAAC;YACjC,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,GAAG,CAAC,cAAc,CAAC;gBAC3C,MAAM,EAAE,iBAAiB;gBACzB,UAAU,EAAE,eAAe;gBAC3B,MAAM;gBACN,MAAM;gBACN,WAAW,EAAE,CAAC;aACf,CAAC,CAAC;YACH,IAAI,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;QACpB,CAAC;QAED,mDAAmD;QACnD,MAAM,IAAI,GAAG,IAAI,GAAG,EAA+D,CAAC;QACpF,KAAK,MAAM,GAAG,IAAI,IAAI,EAAE,CAAC;YACvB,KAAK,MAAM,IAAI,IAAI,GAAG,CAAC,MAAM,CAAC,QAAQ,EAAE,CAAC;gBACvC,MAAM,GAAG,GAAG,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,EAAE,CAAC,IAAI,EAAE,KAAK,EAAE,CAAC,EAAE,KAAK,EAAE,CAAC,EAAE,SAAS,EAAE,IAAI,CAAC,SAAS,EAAE,CAAC;gBACnF,GAAG,CAAC,KAAK,IAAI,IAAI,CAAC,KAAK,CAAC;gBACxB,GAAG,CAAC,KAAK,IAAI,CAAC,CAAC;gBACf,4HAA4H;gBAC5H,GAAG,CAAC,SAAS,GAAG,IAAI,CAAC,SAAS,CAAC;gBAC/B,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,EAAE,EAAE,GAAG,CAAC,CAAC;YACzB,CAAC;QACH,CAAC;QACD,MAAM,QAAQ,GAAG,YAAY,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE;YAC/C,MAAM,GAAG,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC;YAC3B,MAAM,KAAK,GAAG,GAAG,IAAI,IAAI,CAAC,CAAC,CAAC,GAAG,CAAC,KAAK,GAAG,GAAG,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC;YACtD,OAAO,EAAE,EAAE,EAAE,CAAC,CAAC,EAAE,EAAE,KAAK,EAAE,SAAS,EAAE,GAAG,EAAE,SAAS,IAAI,iBAAiB,EAAE,CAAC;QAC7E,CAAC,CAAC,CAAC;QACH,+FAA+F;QAC/F,8DAA8D;QAC9D,MAAM,UAAU,GAAG,YAAY,CAAC,QAAQ,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,CAAC,EAAE,EAAE,CAAC,GAAG,GAAG,CAAC,CAAC,MAAM,EAAE,CAAC,CAAC,CAAC;QAC/E,MAAM,QAAQ,GAAG,YAAY,CAAC,QAAQ,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,CAAC,EAAE,EAAE;YACvD,MAAM,IAAI,GAAG,QAAQ,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,EAAE,KAAK,CAAC,CAAC,EAAE,CAAE,CAAC;YAClD,OAAO,GAAG,GAAG,IAAI,CAAC,KAAK,GAAG,CAAC,CAAC,MAAM,CAAC;QACrC,CAAC,EAAE,CAAC,CAAC,CAAC;QACN,2GAA2G;QAC3G,0GAA0G;QAC1G,4DAA4D;QAC5D,MAAM,KAAK,GAAG,OAAO,CAAC,UAAU,GAAG,CAAC,CAAC,CAAC,CAAC,QAAQ,GAAG,UAAU,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;QAElE,gGAAgG;QAChG,mGAAmG;QACnG,+FAA+F;QAC/F,oGAAoG;QACpG,uFAAuF;QACvF,yDAAyD;QACzD,MAAM,QAAQ,GAAG,YAAY,CAAC,QAAQ;aACnC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,KAAK,IAAI,IAAI,IAAI,QAAQ,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,EAAE,KAAK,CAAC,CAAC,EAAE,CAAE,CAAC,KAAK,GAAG,CAAC,CAAC,KAAK,CAAC;aACtF,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC;QAEpB,OAAO;YACL,IAAI,EAAE,KAAK,IAAI,SAAS,IAAI,QAAQ,CAAC,MAAM,KAAK,CAAC;YACjD,KAAK,EAAE,IAAI,CAAC,KAAK,CAAC,KAAK,GAAG,IAAI,CAAC,GAAG,IAAI;YACtC,QAAQ;YACR,QAAQ;YACR,iFAAiF;YACjF,OAAO,EAAE,IAAI,CAAC,IAAI,CAAC,MAAM,GAAG,CAAC,CAAE,CAAC,MAAM,CAAC,OAAO;YAC9C,OAAO;YACP,KAAK,EAAE,IAAI,CAAC,IAAI,CAAC,MAAM,GAAG,CAAC,CAAE,CAAC,KAAK;YACnC,QAAQ,EAAE,YAAY,CAAC,EAAE;YACzB,aAAa,EAAE,YAAY,CAAC,OAAO;SACpC,CAAC;IACJ,CAAC;IAED,OAAO;QACL,KAAK,CAAC,KAAK,CAAC,KAAK;YACf,MAAM,MAAM,GAAG,YAAY,CACzB,MAAM,EACN,kJAAkJ,CACnJ,CAAC;YACF,MAAM,MAAM,GAAG;gBACb,mHAAmH;gBACnH,wBAAwB,cAAc,CAAC,SAAS,EAAE,KAAK,CAAC,OAAO,CAAC,EAAE;gBAClE,kBAAkB,KAAK,CAAC,KAAK,CAAC,IAAI,cAAc,KAAK,CAAC,KAAK,CAAC,QAAQ,EAAE;gBACtE,GAAG,CAAC,KAAK,CAAC,SAAS,IAAI,IAAI;oBACzB,CAAC,CAAC;wBACE,0DAA0D,KAAK,CAAC,SAAS,CAAC,aAAa,YAAY,KAAK,CAAC,SAAS,CAAC,UAAU,EAAE;qBAChI;oBACH,CAAC,CAAC,EAAE,CAAC;gBACP,GAAG,CAAC,KAAK,CAAC,cAAc,IAAI,IAAI;oBAC9B,CAAC,CAAC,CAAC,iDAAiD,KAAK,CAAC,cAAc,EAAE,CAAC;oBAC3E,CAAC,CAAC,EAAE,CAAC;gBACP,yBAAyB,cAAc,CAAC,MAAM,EAAE,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,MAAM,CAAC,EAAE,MAAM,CAAC,EAAE;aACvF,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;YACf,OAAO,eAAe,CAAC,MAAM,EAAE,MAAM,EAAE,MAAM,CAAC,CAAC;QACjD,CAAC;QAED,KAAK,CAAC,SAAS,CAAC,KAAK;YACnB,MAAM,MAAM,GAAG,YAAY,CACzB,UAAU,EACV,yKAAyK,CAC1K,CAAC;YACF,MAAM,YAAY,GAChB,KAAK,CAAC,OAAO,IAAI,IAAI,IAAI,KAAK,CAAC,OAAO,CAAC,MAAM,GAAG,CAAC;gBAC/C,CAAC,CAAC;oBACE,4BAA4B,KAAK,CAAC,OAAO;yBACtC,GAAG,CACF,CAAC,GAAG,EAAE,EAAE,CACN,KAAK,GAAG,CAAC,IAAI,GAAG,GAAG,CAAC,IAAI,IAAI,IAAI,CAAC,CAAC,CAAC,KAAK,GAAG,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,GAAG,GAAG,CAAC,WAAW,IAAI,IAAI,CAAC,CAAC,CAAC,IAAI,GAAG,CAAC,WAAW,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CACpH;yBACA,IAAI,CAAC,IAAI,CAAC,EAAE;iBAChB;gBACH,CAAC,CAAC,EAAE,CAAC;YACT,MAAM,MAAM,GAAG;gBACb,wBAAwB,KAAK,CAAC,MAAM,CAAC,SAAS,YAAY,IAAI,CAAC,SAAS,CAAC,KAAK,CAAC,MAAM,CAAC,MAAM,IAAI,EAAE,CAAC,EAAE;gBACrG,GAAG,YAAY;gBACf,GAAG,CAAC,KAAK,CAAC,cAAc,IAAI,IAAI,CAAC,CAAC,CAAC,CAAC,wBAAwB,KAAK,CAAC,cAAc,EAAE,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC;gBACzF,gIAAgI;gBAChI,mCAAmC,cAAc,CAAC,cAAc,EAAE,aAAa,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,EAAE;aAC/F,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;YACf,OAAO,eAAe,CAAC,UAAU,EAAE,MAAM,EAAE,MAAM,CAAC,CAAC;QACrD,CAAC;KACF,CAAC;AACJ,CAAC;AAED,gDAAgD;AAChD,SAAS,OAAO,CAAC,CAAS;IACxB,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC;QAAE,OAAO,CAAC,CAAC;IAClC,OAAO,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC;AACrC,CAAC;AAED;;;;;;;GAOG;AACH,SAAS,cAAc,CAAC,MAAc;IACpC,IAAI,MAAM,CAAC,QAAQ,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QACjC,MAAM,IAAI,KAAK,CAAC,WAAW,MAAM,CAAC,EAAE,mBAAmB,CAAC,CAAC;IAC3D,CAAC;IACD,MAAM,IAAI,GAAG,IAAI,GAAG,EAAU,CAAC;IAC/B,IAAI,GAAG,GAAG,CAAC,CAAC;IACZ,KAAK,MAAM,CAAC,IAAI,MAAM,CAAC,QAAQ,EAAE,CAAC;QAChC,IAAI,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC;YACnB,MAAM,IAAI,KAAK,CAAC,WAAW,MAAM,CAAC,EAAE,kCAAkC,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC;QACjF,CAAC;QACD,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC;QACf,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;YAC/C,MAAM,IAAI,KAAK,CACb,WAAW,MAAM,CAAC,EAAE,eAAe,CAAC,CAAC,EAAE,8DAA8D,CAAC,CAAC,MAAM,EAAE,CAChH,CAAC;QACJ,CAAC;QACD,IAAI,CAAC,CAAC,KAAK,IAAI,IAAI,IAAI,CAAC,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,KAAK,GAAG,CAAC,IAAI,CAAC,CAAC,KAAK,GAAG,CAAC,CAAC,EAAE,CAAC;YACjF,MAAM,IAAI,KAAK,CACb,WAAW,MAAM,CAAC,EAAE,eAAe,CAAC,CAAC,EAAE,6DAA6D,CAAC,CAAC,KAAK,EAAE,CAC9G,CAAC;QACJ,CAAC;QACD,GAAG,IAAI,CAAC,CAAC,MAAM,CAAC;IAClB,CAAC;IACD,IAAI,CAAC,CAAC,GAAG,GAAG,CAAC,CAAC,EAAE,CAAC;QACf,MAAM,IAAI,KAAK,CAAC,WAAW,MAAM,CAAC,EAAE,sCAAsC,GAAG,EAAE,CAAC,CAAC;IACnF,CAAC;AACH,CAAC;AAED;;;;GAIG;AACH,SAAS,WAAW,CAAC,OAAe,EAAE,IAAI,GAAG,EAAE;IAC7C,MAAM,UAAU,GAAG,CAAC,OAAO,CAAC,KAAK,CAAC,KAAK,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,GAAG,EAAE,EAAE,CAAC,IAAI,CAAC,GAAG,CAAC,GAAG,EAAE,GAAG,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC,CAAC;IACnG,MAAM,KAAK,GAAG,GAAG,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,UAAU,GAAG,CAAC,CAAC,CAAC,CAAC;IACtD,OAAO,GAAG,KAAK,GAAG,IAAI,KAAK,OAAO,KAAK,KAAK,EAAE,CAAC;AACjD,CAAC;AAED;;;;GAIG;AACH,SAAS,cAAc,CAAC,KAAa,EAAE,OAAe,EAAE,IAAI,GAAG,EAAE;IAC/D,OAAO;QACL,YAAY,KAAK,gEAAgE;QACjF,WAAW,CAAC,OAAO,EAAE,IAAI,CAAC;QAC1B,UAAU,KAAK,KAAK;KACrB,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;AACf,CAAC;AAED;;;;GAIG;AACH,SAAS,aAAa,CAAC,IAAY;IACjC,MAAM,KAAK,GAAG,IAAI,CAAC,UAAU,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE;QACtC,MAAM,KAAK,GAAG,CAAC,GAAG,CAAC,CAAC,EAAE,KAAK,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC;QACrC,IAAI,CAAC,CAAC,KAAK,IAAI,IAAI,IAAI,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,MAAM,GAAG,CAAC;YAAE,KAAK,CAAC,IAAI,CAAC,SAAS,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC;QACvG,IAAI,CAAC,CAAC,IAAI,EAAE,IAAI,IAAI,IAAI;YAAE,KAAK,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAC,IAAI,CAAC,IAAI,EAAE,CAAC,CAAC;QAC5D,OAAO,KAAK,KAAK,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC;IAChC,CAAC,CAAC,CAAC;IACH,MAAM,MAAM,GAAG,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,KAAK,CAAC,CAAC,EAAE,MAAM,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC;IAC/D,OAAO;QACL,gBAAgB,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE;QAClC,GAAG,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,YAAY,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC;KAChE,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;AACf,CAAC"}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@kohaku-ui/evals",
-  "version": "0.1.0",
+  "version": "0.2.0",
   "description": "Evaluation harness for kohaku: golden Spec regression, LLM-as-judge scoring and a record/replay LLM.",
   "type": "module",
   "exports": {
@@ -11,19 +11,19 @@
     "./package.json": "./package.json"
   },
   "dependencies": {
-    "@kohaku-ui/llm": "^0.1.0",
-    "@kohaku-ui/spec-core": "^0.1.0",
-    "@kohaku-ui/composer": "^0.1.0"
+    "@kohaku-ui/spec-core": "^0.2.0",
+    "@kohaku-ui/composer": "^0.2.0",
+    "@kohaku-ui/llm": "^0.2.0"
   },
   "peerDependencies": {
-    "zod": "^4.6.2"
+    "zod": "^4.6.5"
   },
   "devDependencies": {
-    "@types/node": "^25.9.6",
+    "@kohaku-ui/registry": "^0.2.0",
+    "@types/node": "^26.5.1",
     "typescript": "7.0.2",
-    "vitest": "^4.1.11",
-    "zod": "^4.6.2",
-    "@kohaku-ui/registry": "^0.1.0"
+    "vitest": "^5.0.1",
+    "zod": "^4.6.5"
   },
   "license": "Apache-2.0",
   "homepage": "https://github.com/yosuque/kohaku/tree/main/packages/evals#readme",
@@ -42,7 +42,8 @@
   "files": [
     "dist",
     "LICENSE",
-    "README.md"
+    "README.md",
+    "CHANGELOG.md"
   ],
   "publishConfig": {
     "access": "public"