@smthrs/evals 0.0.0-stage → 1.0.0-rc.4

package/CHANGELOG.md

@@ -0,0 +1,142 @@
+# @smthrs/evals
+
+## [Unreleased]
+
+### Added
+
+- `Trials`: repeated-run statistics for live evaluations. `passAtK` is the
+  unbiased pass@k estimator, `passHatK` the pass^k consistency measure, and
+  `summarize` and `aggregate` report both per case and across cases. Bad
+  counts throw the new `EvalError` code `invalid_trials`.
+
+- `Runner.run` executes under the `Eval.run` span and each case under
+  `Eval.case`.
+
+- Added `Report.Data`, `Report.data`, and `Report.version`: the serialized
+  report shape, the projection that builds it, and the format version stamped
+  on it as `reportVersion`. It versions the wire independently of the nested
+  `Baseline.version`.
+
+### Changed
+
+- Report cells, gate summaries, and runner diagnostics share one scorer label.
+  A scorer declared without a name now labels as its key in runner
+  diagnostics, the way report cells and gate summaries already labelled it,
+  rather than as `scorer (first 8 of the key)`. One unnamed scorer no longer
+  appears under two spellings in one report.
+
+- **Breaking:** `CaseExecutor.make(run)` takes only the callback. The
+  `{ run }` and `{ execute }` object forms and `CaseExecutor.Implementation`
+  are gone; pass the function itself. `CaseExecutor.CaseInput` is gone too;
+  use `Suite.Case`. 1.0.0-rc.0 retired the `execute` spelling from
+  `CaseExecutor.Service` but kept it as a `make` alias, so the package still
+  had three spellings for one callback.
+
+- **Breaking:** `Report.json` serializes `Report.Data` rather than the
+  in-memory `Regression.Report` graph. `run.observations` is now the
+  observation table and everything else refers to a row by index:
+  `run.cases[i].observations` is a list of indexes and a regression or
+  nondeterminism entry's `actual` is a single index. `samples` and
+  `inconclusive` are gone from the wire because both are filters of
+  `run.observations` by `kind`. The graph carried each observation in up to
+  four places, so the artifact was about five times one copy of the
+  observations; a run at the declared case ceiling produced 49.2 MiB of JSON
+  for 9.9 MiB of observations. `Regression.Report` itself is unchanged, so a
+  gate still reads `samples` and `inconclusive` in memory.
+
+### Fixed
+
+- Bound the canonical encoder's total traversal at 2000000 values and 33554432
+  output code units, leaving `[budget exceeded]` where a budget runs out. The
+  cycle detector tracks ancestors only, so a value referenced twice is expanded
+  twice: 24 nested two-child wrappers over one shared leaf are 25 objects at
+  depth 24, far below the depth ceiling of 64, and expanded to a multi-megabyte
+  artifact in 16 seconds. The budget is spent in traversal order, which the key
+  sort fixes, so one value always truncates at the same place.
+
+- Load version-1 baselines without a top-level `suite` when every record names
+  the same suite. Reject empty or ambiguous legacy artifacts with an explicit
+  `invalid_baseline` reason at `suite`.
+- Compute each baseline record's serialization sort key once per write.
+
+## [1.0.0-rc.0] - 2026-09-01
+
+### Added
+
+- Added fixed suites, deterministic evaluation runs, baselines, regression
+  reports, and score gates.
+- Added `Runner.makeInline` and `Runner.layerInline`, the in-process batch runner
+  a run now scores with by default, so a suite with pure scorers no longer has to
+  hand-copy an adapter.
+- Added `Suite.limits`, the declared ceilings on concurrency, case count, and
+  JSON Lines fixture size.
+- Added `Observation.scorerName` and `BaselineRecord.scorerName`, so a report can
+  be read without grepping for a scorer digest.
+- Added `Runner.BatchResult` and the optional `ScoreBatchRunner.runBatchCorrelated`,
+  which tags each observation with its job identity. A run correlates by identity
+  when a runner implements it, so results may come back in any order.
+- Added `Baseline.suite`, which records artifact ownership even when the baseline
+  holds no records. It is required in validated baselines and newly written
+  artifacts. `Baseline.load` accepts older version-1 artifacts without it only
+  when their nonempty records all name the same suite.
+- Added package-owned documentation in `docs/`, gated by `test/docs.test.ts`.
+
+### Changed
+
+- **Breaking:** `Runner.run` requires only `CaseExecutor`. It resolved the
+  `Runner` service with `Effect.serviceOption` while declaring it in its
+  requirements, which made the in-process scoring path unreachable.
+- **Breaking:** `CaseExecutor.Service` is one `run` callback. The twin
+  `run`/`execute` pair could diverge, and `make` with neither silently degraded
+  to the unavailable executor; it now throws a `TypeError`.
+- **Breaking:** `Baseline.Record` is `Baseline.BaselineRecord`, so the global
+  `Record` utility type is usable again.
+- **Breaking:** `Regression.MissingObservation.stepKey` is required, and
+  `Regression.Report`'s `samples` and `inconclusive` are narrowed to the
+  observation kinds they actually hold.
+- Verify the batch-runner protocol instead of trusting array position. A runner
+  that returns the wrong number of observations, or observations identifying
+  other jobs, now fails the run with `scorer_protocol` rather than attributing
+  one case's score to another. `(targetStepKey, scorerKey)` alone is not a job
+  identity: two cases graded by one scorer at one step key still permuted
+  silently, so an order-only runner now fails with the new `ambiguous_score_job`
+  code before it is called, and a correlated runner is paired by job identity.
+- Decode a batch observation's whole shape rather than spot-checking its score.
+  An inconclusive observation with no reason produced an `Observation` that
+  violated its own type and made `Report.markdown` throw a `TypeError`.
+- Turn a batch score that is not finite and inside `[0, 1]` into an inconclusive
+  observation naming the scorer and the offending value.
+- Fail `Regression.compare` with `invalid_baseline` when the artifact or any of
+  its records belongs to another suite. `Baseline.load` derives ownership for
+  older suite-less version-1 artifacts only when all records agree on one suite;
+  it rejects empty or ambiguous legacy artifacts. Comparisons with the wrong
+  suite used to report zero findings.
+- Snapshot and freeze suite cases and bindings, including a binding's sampling
+  policy, and reject data that cannot be structured-cloned, so a run stays
+  reproducible from the validated suite. `Suite.make` validated at call time and
+  copied at execution time, so the two could see different data.
+- Read every option, case, binding, and baseline record field exactly once.
+  Reading a field twice let a getter return the value that passed validation and
+  then a different one into the suite or the committed artifact.
+- Serialize reports and baselines through one bounded, cycle-safe canonical
+  encoder. A cyclic case output used to throw a `RangeError` out of a function
+  typed `string`, and a hostile proxy still threw out of the encoder that
+  replaced it.
+- Render an `Error` as an object carrying its own enumerable fields, so
+  `EvalError.code` and `EvalError.path` survive into `Report.json`. The previous
+  `[Name: message]` string dropped the stable branch point a caller reads.
+- Preserve the executor's own code and message on a failed case, and print the
+  code in the gate summary, which used to read as a tautology.
+- Name every missing, inconclusive, and failed case in `Report.markdown` instead
+  of only counting them, and escape, flatten, and cap every rendered cell.
+- Encode every tuple key injectively instead of joining on `U+0000`, and reject
+  control characters in suite and case names.
+- Print the scorer as `name (first 8 of the key)` in gate messages as well as in
+  the Markdown report. A missing observation used to name a 64-character digest
+  with no way back to the scorer.
+- Split the error vocabulary so a caller can branch on it: added
+  `invalid_run_options`, `invalid_tolerance`, `ambiguous_score_job`,
+  `scorer_protocol`, and `scorer_unavailable`; added `EvalError.path`; removed
+  the unraised `missing_ground_truth`.
+- Raised coverage thresholds to 100% on branches, functions, lines, and
+  statements.

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 William Cory and the Smithers Flows contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,63 @@
-# Temporary Holding Version
+# @smthrs/evals
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+Release candidate scope, host requirements and compatibility review are defined in the [library support policy](https://github.com/smithersai/smithers/blob/main/RELEASE_SUPPORT.md).
+
+**Documentation:** https://evals.smithers.sh
+
+Fixed-suite evaluation for flows: it connects target execution and scorer runners
+to validated suites, committed baselines, regression comparison, reports, and CI
+gates.
+
+A unit test asserts equality. A flow that calls a model produces an answer that
+is rarely equal to anything, only better or worse than the last one. This package
+scores the answer, compares the score with a committed baseline, and turns the
+comparison into a CI exit code.
+
+Runtime grading belongs to `@smthrs/scorers/ScoreGate`: score samples, verdicts,
+threshold checks, CI grades, and `ScoreGateError` share one pure contract.
+`@smthrs/testing/ScoreGate` is its test facade and also supplies a fixed-suite
+runner. Runtime evaluation code imports scorers directly.
+
+## Install
+
+The package is at 1.0.0-rc.0 and is not on the npm registry yet. It is a
+workspace package of https://github.com/smithersai/smithers, so you use it from
+a package in a clone of that repository. The steps are at
+https://evals.smithers.sh/installation/.
+
+## Example
+
+```ts
+import { CaseExecutor, Runner, Suite } from "@smthrs/evals"
+import { Effect, Layer } from "effect"
+
+const greet = { name: "greet" }
+
+const executor = CaseExecutor.make((suiteCase) =>
+  Effect.succeed({
+    output: `Hello, ${(suiteCase.input as { readonly name: string }).name}`,
+    stepKey: suiteCase.name,
+    latencyMs: 0,
+    target: greet
+  })
+)
+
+const program = Effect.gen(function*() {
+  const suite = yield* Suite.make({
+    name: "smoke",
+    cases: [{ name: "hello", input: { name: "Ada" } }],
+    concurrency: 1
+  })
+  return yield* Runner.run(suite, { runId: "nightly-2026-01-01", at: "2026-01-01T00:00:00.000Z" })
+}).pipe(Effect.provide(Layer.succeed(CaseExecutor.CaseExecutor)(executor)))
+```
+
+`Runner.run` needs only `CaseExecutor`: scoring runs in process by default.
+Baselines, regression comparison, reports, and gates complete the loop;
+https://evals.smithers.sh/quickstart/ walks all of it end to end.
+
+## Reference
+
+https://evals.smithers.sh/reference/api/ documents every export, the stable
+failure codes, the batch-runner protocol, the step-key rule that separates a
+regression from nondeterminism, and the declared size and concurrency limits.

package/dist/cjs/Baseline.d.ts

@@ -0,0 +1,118 @@
+/**
+ * Canonical committed evaluation baselines.
+ *
+ * A baseline is the record of what a suite used to score, committed beside the
+ * suite it belongs to. It holds only the fields a comparison reads, and
+ * validation rebuilds every record from those fields, so nothing a caller
+ * happened to attach to an object travels into a committed artifact.
+ *
+ * @since 0.1.0
+ */
+import * as Effect from "effect/Effect";
+import { EvalError } from "./EvalError.ts";
+import type { RunResult } from "./Runner.ts";
+/**
+ * Current committed baseline artifact version.
+ *
+ * @category models
+ * @since 0.1.0
+ */
+export declare const version: 1;
+/**
+ * One successful score retained by a baseline.
+ *
+ * `scorer` is the scorer key a comparison matches on. `scorerName` is the
+ * optional readable name of the same scorer.
+ *
+ * @category models
+ * @since 0.1.0
+ */
+export interface BaselineRecord {
+    readonly suite: string;
+    readonly case: string;
+    readonly scorer: string;
+    readonly scorerName?: string | undefined;
+    readonly stepKey: string;
+    readonly score: number;
+}
+/**
+ * Canonical committed evaluation baseline.
+ *
+ * `suite` records artifact ownership even when `records` is empty.
+ *
+ * @category models
+ * @since 0.1.0
+ */
+export interface Baseline {
+    readonly version: typeof version;
+    readonly suite: string;
+    readonly records: ReadonlyArray<BaselineRecord>;
+}
+/**
+ * The declared ceilings a committed baseline is validated against.
+ *
+ * `length` bounds the JSON text `load` will parse, so an oversized committed
+ * file fails with a sentence instead of exhausting a CI runner's memory.
+ *
+ * @category models
+ * @since 0.1.0
+ */
+export declare const limits: {
+    /** Longest baseline JSON text, in UTF-16 code units. */
+    readonly length: number;
+};
+/**
+ * Builds and validates a baseline from a run's successful observations.
+ *
+ * Inconclusive observations are dropped: a baseline records what was measured,
+ * and an inconclusive observation measured nothing.
+ *
+ * @category constructors
+ * @since 0.1.0
+ */
+export declare const fromRun: (run: RunResult) => Effect.Effect<Baseline, EvalError>;
+/**
+ * Validates an in-memory baseline.
+ *
+ * The result is a snapshot: every known field is read once, then records are
+ * rebuilt from those validated values and the array is frozen. Mutating the
+ * array or the objects that were passed in cannot change the validated
+ * baseline, and a stateful getter cannot substitute a value after validation.
+ *
+ * Fails with `invalid_baseline` carrying the record index and field name in
+ * `path` for a wrong version, a non-array `records`, a record that is not an
+ * object, an identity field that is not a string or holds a control
+ * character, or a score that is not finite in `[0, 1]`.
+ *
+ * @category constructors
+ * @since 0.1.0
+ */
+export declare const make: (baseline: Omit<Baseline, "version"> & {
+    readonly version?: typeof version;
+}) => Effect.Effect<Baseline, EvalError>;
+/**
+ * Serializes a baseline with recursively sorted keys and stable numbers.
+ *
+ * Records are ordered by an injective encoding of `(suite, case, scorer,
+ * stepKey)`, so two records whose names differ only in where a separator falls
+ * still sort apart. The result ends with a newline and never throws: a
+ * validated baseline holds only strings and finite numbers.
+ *
+ * @category serialization
+ * @since 0.1.0
+ */
+export declare const write: (baseline: Baseline) => string;
+/**
+ * Loads and validates baseline JSON.
+ *
+ * A version-1 artifact without a top-level `suite` is accepted only when its
+ * nonempty records all name the same suite. That suite becomes the artifact's
+ * owner, so writing the result includes it. Empty or ambiguous legacy artifacts
+ * fail with `invalid_baseline` at `suite`. Text longer than `limits.length`
+ * fails with `invalid_baseline` at `text` before any of it is parsed.
+ *
+ * @category serialization
+ * @since 0.1.0
+ */
+export declare const load: (text: string) => Effect.Effect<Baseline, EvalError>;
+//# sourceMappingURL=Baseline.d.ts.map

package/dist/cjs/Baseline.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"Baseline.d.ts","sourceRoot":"","sources":["../../src/Baseline.ts"],"names":[],"mappings":"AAAA;;;;;;;;;GASG;AAEH,OAAO,KAAK,MAAM,MAAM,eAAe,CAAA;AACvC,OAAO,EAAE,SAAS,EAAE,MAAM,gBAAgB,CAAA;AAI1C,OAAO,KAAK,EAAe,SAAS,EAAE,MAAM,aAAa,CAAA;AAEzD;;;;;GAKG;AACH,eAAO,MAAM,OAAO,EAAG,CAAU,CAAA;AAEjC;;;;;;;;GAQG;AACH,MAAM,WAAW,cAAc;IAC7B,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAA;IACtB,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAA;IACrB,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAA;IACvB,QAAQ,CAAC,UAAU,CAAC,EAAE,MAAM,GAAG,SAAS,CAAA;IACxC,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAA;IACxB,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAA;CACvB;AAED;;;;;;;GAOG;AACH,MAAM,WAAW,QAAQ;IACvB,QAAQ,CAAC,OAAO,EAAE,OAAO,OAAO,CAAA;IAChC,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAA;IACtB,QAAQ,CAAC,OAAO,EAAE,aAAa,CAAC,cAAc,CAAC,CAAA;CAChD;AAED;;;;;;;;GAQG;AACH,eAAO,MAAM,MAAM;IACjB,wDAAwD;;CAEhD,CAAA;AAiGV;;;;;;;;GAQG;AACH,eAAO,MAAM,OAAO,GAAI,KAAK,SAAS,KAAG,MAAM,CAAC,MAAM,CAAC,QAAQ,EAAE,SAAS,CAczE,CAAA;AAED;;;;;;;;;;;;;;;GAeG;AACH,eAAO,MAAM,IAAI,GACf,UAAU,IAAI,CAAC,QAAQ,EAAE,SAAS,CAAC,GAAG;IAAE,QAAQ,CAAC,OAAO,CAAC,EAAE,OAAO,OAAO,CAAA;CAAE,KAC1E,MAAM,CAAC,MAAM,CAAC,QAAQ,EAAE,SAAS,CAInC,CAAA;AAED;;;;;;;;;;GAUG;AACH,eAAO,MAAM,KAAK,GAAI,UAAU,QAAQ,KAAG,MAW1C,CAAA;AAED;;;;;;;;;;;GAWG;AACH,eAAO,MAAM,IAAI,GAAI,MAAM,MAAM,KAAG,MAAM,CAAC,MAAM,CAAC,QAAQ,EAAE,SAAS,CAMhC,CAAA"}

package/dist/cjs/Baseline.js

@@ -0,0 +1,158 @@
+"use strict";
+var __create = Object.create;
+var __defProp = Object.defineProperty;
+var __getOwnPropDesc = Object.getOwnPropertyDescriptor;
+var __getOwnPropNames = Object.getOwnPropertyNames;
+var __getProtoOf = Object.getPrototypeOf;
+var __hasOwnProp = Object.prototype.hasOwnProperty;
+var __export = (target, all) => {
+  for (var name in all)
+    __defProp(target, name, { get: all[name], enumerable: true });
+};
+var __copyProps = (to, from, except, desc) => {
+  if (from && typeof from === "object" || typeof from === "function") {
+    for (let key of __getOwnPropNames(from))
+      if (!__hasOwnProp.call(to, key) && key !== except)
+        __defProp(to, key, { get: () => from[key], enumerable: !(desc = __getOwnPropDesc(from, key)) || desc.enumerable });
+  }
+  return to;
+};
+var __toESM = (mod, isNodeMode, target) => (target = mod != null ? __create(__getProtoOf(mod)) : {}, __copyProps(
+  // If the importer is in node compatibility mode or this is not an ESM
+  // file that has been converted to a CommonJS file using a Babel-
+  // compatible transform (i.e. "__esModule" has not been set), then set
+  // "default" to the CommonJS "module.exports" for node compatibility.
+  isNodeMode || !mod || !mod.__esModule ? __defProp(target, "default", { value: mod, enumerable: true }) : target,
+  mod
+));
+var __toCommonJS = (mod) => __copyProps(__defProp({}, "__esModule", { value: true }), mod);
+var Baseline_exports = {};
+__export(Baseline_exports, {
+  fromRun: () => fromRun,
+  limits: () => limits,
+  load: () => load,
+  make: () => make,
+  version: () => version,
+  write: () => write
+});
+module.exports = __toCommonJS(Baseline_exports);
+var Effect = __toESM(require("effect/Effect"), 1);
+var import_EvalError = require("./EvalError.js");
+var import_CanonicalJson = require("./internal/CanonicalJson.js");
+var import_controlCharacters = require("./internal/controlCharacters.js");
+var import_tupleKey = require("./internal/tupleKey.js");
+const version = 1;
+const limits = {
+  /** Longest baseline JSON text, in UTF-16 code units. */
+  length: 16 * 1024 * 1024
+};
+const fail = (message, path) => Effect.fail(
+  new import_EvalError.EvalError({ code: "invalid_baseline", message, ...path === void 0 ? {} : { path } })
+);
+const stringField = (value, field, path) => {
+  if (typeof value !== "string") {
+    return fail(`Baseline record field '${field}' must be a string, got ${typeof value}`, path);
+  }
+  const control = (0, import_controlCharacters.controlCharacter)(value);
+  return control === void 0 ? Effect.succeed(value) : fail(`Baseline record field '${field}' must not contain the control character ${control}`, path);
+};
+const decodeRecord = (value, index) => Effect.gen(function* () {
+  const at = `records[${index}]`;
+  if (typeof value !== "object" || value === null || Array.isArray(value)) {
+    return yield* fail(`Baseline record must be an object, got ${value === null ? "null" : typeof value}`, at);
+  }
+  const source = value;
+  const rawSuite = source.suite;
+  const rawCase = source.case;
+  const rawScorer = source.scorer;
+  const rawScorerName = source.scorerName;
+  const rawStepKey = source.stepKey;
+  const rawScore = source.score;
+  const suite = yield* stringField(rawSuite, "suite", `${at}.suite`);
+  const caseName = yield* stringField(rawCase, "case", `${at}.case`);
+  const scorer = yield* stringField(rawScorer, "scorer", `${at}.scorer`);
+  const stepKey = yield* stringField(rawStepKey, "stepKey", `${at}.stepKey`);
+  const score = rawScore;
+  if (typeof score !== "number" || !Number.isFinite(score) || score < 0 || score > 1) {
+    return yield* fail(
+      `Baseline record field 'score' must be a finite number in [0, 1], got ${String(score)}`,
+      `${at}.score`
+    );
+  }
+  const scorerName = rawScorerName === void 0 ? void 0 : yield* stringField(rawScorerName, "scorerName", `${at}.scorerName`);
+  return {
+    suite,
+    case: caseName,
+    scorer,
+    ...scorerName === void 0 ? {} : { scorerName },
+    stepKey,
+    score: Object.is(score, -0) ? 0 : score
+  };
+});
+const validate = (value) => Effect.gen(function* () {
+  if (typeof value !== "object" || value === null || Array.isArray(value)) {
+    return yield* fail("Baseline must be an object");
+  }
+  const artifact = value;
+  const rawVersion = artifact.version;
+  const rawSuite = artifact.suite;
+  const rawRecords = artifact.records;
+  if (rawVersion !== version) {
+    return yield* fail(`Baseline version must be ${version}, got ${String(rawVersion)}`, "version");
+  }
+  if (rawSuite !== void 0 && typeof rawSuite !== "string") {
+    return yield* fail(`Baseline field 'suite' must be a string, got ${typeof rawSuite}`, "suite");
+  }
+  const suiteControl = rawSuite === void 0 ? void 0 : (0, import_controlCharacters.controlCharacter)(rawSuite);
+  if (suiteControl !== void 0) {
+    return yield* fail(`Baseline field 'suite' must not contain the control character ${suiteControl}`, "suite");
+  }
+  if (!Array.isArray(rawRecords)) {
+    return yield* fail("Baseline records must be an array", "records");
+  }
+  const records = yield* Effect.forEach(rawRecords, decodeRecord);
+  let suite = rawSuite;
+  if (suite === void 0) {
+    const first = records[0];
+    if (first === void 0) {
+      return yield* fail("Cannot infer baseline suite: legacy artifact has no records", "suite");
+    }
+    if (records.some((record) => record.suite !== first.suite)) {
+      return yield* fail("Cannot infer baseline suite: legacy records name multiple suites", "suite");
+    }
+    suite = first.suite;
+  }
+  return { version, suite, records: Object.freeze(records) };
+});
+const fromRun = (run) => {
+  const records = run.observations.flatMap(
+    (observation) => observation.kind === "score" ? [{
+      suite: run.suite,
+      case: observation.case,
+      scorer: observation.scorer,
+      ...observation.scorerName === void 0 ? {} : { scorerName: observation.scorerName },
+      stepKey: observation.stepKey,
+      score: observation.score
+    }] : []
+  );
+  return validate({ version, suite: run.suite, records });
+};
+const make = (baseline) => {
+  const artifactVersion = baseline.version ?? version;
+  const records = baseline.records;
+  return validate({ version: artifactVersion, suite: baseline.suite, records });
+};
+const write = (baseline) => {
+  const sortKey = (record) => (0, import_tupleKey.tupleKey)(record.suite, record.case, record.scorer, record.stepKey);
+  const records = baseline.records.map((record) => [sortKey(record), record]).sort(([left], [right]) => import_CanonicalJson.CanonicalJson.compareText(left, right)).map(([, record]) => record);
+  return import_CanonicalJson.CanonicalJson.stringify({
+    version: baseline.version,
+    suite: baseline.suite,
+    records
+  });
+};
+const load = (text) => text.length > limits.length ? fail(`Baseline must be at most ${limits.length} characters, got ${text.length}`, "text") : Effect.try({
+  try: () => JSON.parse(text),
+  catch: (cause) => new import_EvalError.EvalError({ code: "invalid_baseline", message: "Baseline is not valid JSON", cause })
+}).pipe(Effect.flatMap(validate));
+//# sourceMappingURL=Baseline.js.map

package/dist/cjs/Baseline.js.map

@@ -0,0 +1,7 @@
+{
+  "version": 3,
+  "sources": ["../../src/Baseline.ts"],
+  "sourcesContent": ["/**\n * Canonical committed evaluation baselines.\n *\n * A baseline is the record of what a suite used to score, committed beside the\n * suite it belongs to. It holds only the fields a comparison reads, and\n * validation rebuilds every record from those fields, so nothing a caller\n * happened to attach to an object travels into a committed artifact.\n *\n * @since 0.1.0\n */\n\nimport * as Effect from \"effect/Effect\"\nimport { EvalError } from \"./EvalError.ts\"\nimport { CanonicalJson } from \"./internal/CanonicalJson.ts\"\nimport { controlCharacter } from \"./internal/controlCharacters.ts\"\nimport { tupleKey } from \"./internal/tupleKey.ts\"\nimport type { Observation, RunResult } from \"./Runner.ts\"\n\n/**\n * Current committed baseline artifact version.\n *\n * @category models\n * @since 0.1.0\n */\nexport const version = 1 as const\n\n/**\n * One successful score retained by a baseline.\n *\n * `scorer` is the scorer key a comparison matches on. `scorerName` is the\n * optional readable name of the same scorer.\n *\n * @category models\n * @since 0.1.0\n */\nexport interface BaselineRecord {\n  readonly suite: string\n  readonly case: string\n  readonly scorer: string\n  readonly scorerName?: string | undefined\n  readonly stepKey: string\n  readonly score: number\n}\n\n/**\n * Canonical committed evaluation baseline.\n *\n * `suite` records artifact ownership even when `records` is empty.\n *\n * @category models\n * @since 0.1.0\n */\nexport interface Baseline {\n  readonly version: typeof version\n  readonly suite: string\n  readonly records: ReadonlyArray<BaselineRecord>\n}\n\n/**\n * The declared ceilings a committed baseline is validated against.\n *\n * `length` bounds the JSON text `load` will parse, so an oversized committed\n * file fails with a sentence instead of exhausting a CI runner's memory.\n *\n * @category models\n * @since 0.1.0\n */\nexport const limits = {\n  /** Longest baseline JSON text, in UTF-16 code units. */\n  length: 16 * 1024 * 1024\n} as const\n\nconst fail = (message: string, path?: string): Effect.Effect<never, EvalError> =>\n  Effect.fail(\n    new EvalError({ code: \"invalid_baseline\", message, ...(path === undefined ? {} : { path }) })\n  )\n\n// A committed record is read by a gate whose summary goes to a CI log, so it\n// follows the same control-character rule as a suite name: rejected where it\n// enters the system, not flattened later.\nconst stringField = (value: unknown, field: string, path: string): Effect.Effect<string, EvalError> => {\n  if (typeof value !== \"string\") {\n    return fail(`Baseline record field '${field}' must be a string, got ${typeof value}`, path)\n  }\n  const control = controlCharacter(value)\n  return control === undefined\n    ? Effect.succeed(value)\n    : fail(`Baseline record field '${field}' must not contain the control character ${control}`, path)\n}\n\nconst decodeRecord = (value: unknown, index: number): Effect.Effect<BaselineRecord, EvalError> =>\n  Effect.gen(function*() {\n    const at = `records[${index}]`\n    if (typeof value !== \"object\" || value === null || Array.isArray(value)) {\n      return yield* fail(`Baseline record must be an object, got ${value === null ? \"null\" : typeof value}`, at)\n    }\n    const source = value as { readonly [key: string]: unknown }\n    const rawSuite = source.suite\n    const rawCase = source.case\n    const rawScorer = source.scorer\n    const rawScorerName = source.scorerName\n    const rawStepKey = source.stepKey\n    const rawScore = source.score\n    const suite = yield* stringField(rawSuite, \"suite\", `${at}.suite`)\n    const caseName = yield* stringField(rawCase, \"case\", `${at}.case`)\n    const scorer = yield* stringField(rawScorer, \"scorer\", `${at}.scorer`)\n    const stepKey = yield* stringField(rawStepKey, \"stepKey\", `${at}.stepKey`)\n    const score = rawScore\n    if (typeof score !== \"number\" || !Number.isFinite(score) || score < 0 || score > 1) {\n      return yield* fail(\n        `Baseline record field 'score' must be a finite number in [0, 1], got ${String(score)}`,\n        `${at}.score`\n      )\n    }\n    const scorerName = rawScorerName === undefined\n      ? undefined\n      : yield* stringField(rawScorerName, \"scorerName\", `${at}.scorerName`)\n    // Every record is rebuilt field by field. Keeping the caller's object would\n    // carry unknown keys, and any getter among them, straight into a committed\n    // artifact.\n    return {\n      suite,\n      case: caseName,\n      scorer,\n      ...(scorerName === undefined ? {} : { scorerName }),\n      stepKey,\n      score: Object.is(score, -0) ? 0 : score\n    }\n  })\n\nconst validate = (value: unknown): Effect.Effect<Baseline, EvalError> =>\n  Effect.gen(function*() {\n    if (typeof value !== \"object\" || value === null || Array.isArray(value)) {\n      return yield* fail(\"Baseline must be an object\")\n    }\n    const artifact = value as { readonly version?: unknown; readonly suite?: unknown; readonly records?: unknown }\n    const rawVersion = artifact.version\n    const rawSuite = artifact.suite\n    const rawRecords = artifact.records\n    if (rawVersion !== version) {\n      return yield* fail(`Baseline version must be ${version}, got ${String(rawVersion)}`, \"version\")\n    }\n    if (rawSuite !== undefined && typeof rawSuite !== \"string\") {\n      return yield* fail(`Baseline field 'suite' must be a string, got ${typeof rawSuite}`, \"suite\")\n    }\n    const suiteControl = rawSuite === undefined ? undefined : controlCharacter(rawSuite)\n    if (suiteControl !== undefined) {\n      return yield* fail(`Baseline field 'suite' must not contain the control character ${suiteControl}`, \"suite\")\n    }\n    if (!Array.isArray(rawRecords)) {\n      return yield* fail(\"Baseline records must be an array\", \"records\")\n    }\n    const records = yield* Effect.forEach(rawRecords, decodeRecord)\n    let suite = rawSuite\n    if (suite === undefined) {\n      const first = records[0]\n      if (first === undefined) {\n        return yield* fail(\"Cannot infer baseline suite: legacy artifact has no records\", \"suite\")\n      }\n      if (records.some((record) => record.suite !== first.suite)) {\n        return yield* fail(\"Cannot infer baseline suite: legacy records name multiple suites\", \"suite\")\n      }\n      suite = first.suite\n    }\n    return { version, suite, records: Object.freeze(records) }\n  })\n\n/**\n * Builds and validates a baseline from a run's successful observations.\n *\n * Inconclusive observations are dropped: a baseline records what was measured,\n * and an inconclusive observation measured nothing.\n *\n * @category constructors\n * @since 0.1.0\n */\nexport const fromRun = (run: RunResult): Effect.Effect<Baseline, EvalError> => {\n  const records: Array<BaselineRecord> = run.observations.flatMap((observation: Observation) =>\n    observation.kind === \"score\"\n      ? [{\n        suite: run.suite,\n        case: observation.case,\n        scorer: observation.scorer,\n        ...(observation.scorerName === undefined ? {} : { scorerName: observation.scorerName }),\n        stepKey: observation.stepKey,\n        score: observation.score\n      }]\n      : []\n  )\n  return validate({ version, suite: run.suite, records })\n}\n\n/**\n * Validates an in-memory baseline.\n *\n * The result is a snapshot: every known field is read once, then records are\n * rebuilt from those validated values and the array is frozen. Mutating the\n * array or the objects that were passed in cannot change the validated\n * baseline, and a stateful getter cannot substitute a value after validation.\n *\n * Fails with `invalid_baseline` carrying the record index and field name in\n * `path` for a wrong version, a non-array `records`, a record that is not an\n * object, an identity field that is not a string or holds a control\n * character, or a score that is not finite in `[0, 1]`.\n *\n * @category constructors\n * @since 0.1.0\n */\nexport const make = (\n  baseline: Omit<Baseline, \"version\"> & { readonly version?: typeof version }\n): Effect.Effect<Baseline, EvalError> => {\n  const artifactVersion = baseline.version ?? version\n  const records = baseline.records\n  return validate({ version: artifactVersion, suite: baseline.suite, records })\n}\n\n/**\n * Serializes a baseline with recursively sorted keys and stable numbers.\n *\n * Records are ordered by an injective encoding of `(suite, case, scorer,\n * stepKey)`, so two records whose names differ only in where a separator falls\n * still sort apart. The result ends with a newline and never throws: a\n * validated baseline holds only strings and finite numbers.\n *\n * @category serialization\n * @since 0.1.0\n */\nexport const write = (baseline: Baseline): string => {\n  const sortKey = (record: BaselineRecord): string => tupleKey(record.suite, record.case, record.scorer, record.stepKey)\n  const records = baseline.records\n    .map((record) => [sortKey(record), record] as const)\n    .sort(([left], [right]) => CanonicalJson.compareText(left, right))\n    .map(([, record]) => record)\n  return CanonicalJson.stringify({\n    version: baseline.version,\n    suite: baseline.suite,\n    records\n  })\n}\n\n/**\n * Loads and validates baseline JSON.\n *\n * A version-1 artifact without a top-level `suite` is accepted only when its\n * nonempty records all name the same suite. That suite becomes the artifact's\n * owner, so writing the result includes it. Empty or ambiguous legacy artifacts\n * fail with `invalid_baseline` at `suite`. Text longer than `limits.length`\n * fails with `invalid_baseline` at `text` before any of it is parsed.\n *\n * @category serialization\n * @since 0.1.0\n */\nexport const load = (text: string): Effect.Effect<Baseline, EvalError> =>\n  text.length > limits.length\n    ? fail(`Baseline must be at most ${limits.length} characters, got ${text.length}`, \"text\")\n    : Effect.try({\n      try: () => JSON.parse(text) as unknown,\n      catch: (cause) => new EvalError({ code: \"invalid_baseline\", message: \"Baseline is not valid JSON\", cause })\n    }).pipe(Effect.flatMap(validate))\n"],
+  "mappings": ";;;;;;;;;;;;;;;;;;;;;;;;;;;;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAWA,aAAwB;AACxB,uBAA0B;AAC1B,2BAA8B;AAC9B,+BAAiC;AACjC,sBAAyB;AASlB,MAAM,UAAU;AA2ChB,MAAM,SAAS;AAAA;AAAA,EAEpB,QAAQ,KAAK,OAAO;AACtB;AAEA,MAAM,OAAO,CAAC,SAAiB,SAC7B,OAAO;AAAA,EACL,IAAI,2BAAU,EAAE,MAAM,oBAAoB,SAAS,GAAI,SAAS,SAAY,CAAC,IAAI,EAAE,KAAK,EAAG,CAAC;AAC9F;AAKF,MAAM,cAAc,CAAC,OAAgB,OAAe,SAAmD;AACrG,MAAI,OAAO,UAAU,UAAU;AAC7B,WAAO,KAAK,0BAA0B,KAAK,2BAA2B,OAAO,KAAK,IAAI,IAAI;AAAA,EAC5F;AACA,QAAM,cAAU,2CAAiB,KAAK;AACtC,SAAO,YAAY,SACf,OAAO,QAAQ,KAAK,IACpB,KAAK,0BAA0B,KAAK,4CAA4C,OAAO,IAAI,IAAI;AACrG;AAEA,MAAM,eAAe,CAAC,OAAgB,UACpC,OAAO,IAAI,aAAY;AACrB,QAAM,KAAK,WAAW,KAAK;AAC3B,MAAI,OAAO,UAAU,YAAY,UAAU,QAAQ,MAAM,QAAQ,KAAK,GAAG;AACvE,WAAO,OAAO,KAAK,0CAA0C,UAAU,OAAO,SAAS,OAAO,KAAK,IAAI,EAAE;AAAA,EAC3G;AACA,QAAM,SAAS;AACf,QAAM,WAAW,OAAO;AACxB,QAAM,UAAU,OAAO;AACvB,QAAM,YAAY,OAAO;AACzB,QAAM,gBAAgB,OAAO;AAC7B,QAAM,aAAa,OAAO;AAC1B,QAAM,WAAW,OAAO;AACxB,QAAM,QAAQ,OAAO,YAAY,UAAU,SAAS,GAAG,EAAE,QAAQ;AACjE,QAAM,WAAW,OAAO,YAAY,SAAS,QAAQ,GAAG,EAAE,OAAO;AACjE,QAAM,SAAS,OAAO,YAAY,WAAW,UAAU,GAAG,EAAE,SAAS;AACrE,QAAM,UAAU,OAAO,YAAY,YAAY,WAAW,GAAG,EAAE,UAAU;AACzE,QAAM,QAAQ;AACd,MAAI,OAAO,UAAU,YAAY,CAAC,OAAO,SAAS,KAAK,KAAK,QAAQ,KAAK,QAAQ,GAAG;AAClF,WAAO,OAAO;AAAA,MACZ,wEAAwE,OAAO,KAAK,CAAC;AAAA,MACrF,GAAG,EAAE;AAAA,IACP;AAAA,EACF;AACA,QAAM,aAAa,kBAAkB,SACjC,SACA,OAAO,YAAY,eAAe,cAAc,GAAG,EAAE,aAAa;AAItE,SAAO;AAAA,IACL;AAAA,IACA,MAAM;AAAA,IACN;AAAA,IACA,GAAI,eAAe,SAAY,CAAC,IAAI,EAAE,WAAW;AAAA,IACjD;AAAA,IACA,OAAO,OAAO,GAAG,OAAO,EAAE,IAAI,IAAI;AAAA,EACpC;AACF,CAAC;AAEH,MAAM,WAAW,CAAC,UAChB,OAAO,IAAI,aAAY;AACrB,MAAI,OAAO,UAAU,YAAY,UAAU,QAAQ,MAAM,QAAQ,KAAK,GAAG;AACvE,WAAO,OAAO,KAAK,4BAA4B;AAAA,EACjD;AACA,QAAM,WAAW;AACjB,QAAM,aAAa,SAAS;AAC5B,QAAM,WAAW,SAAS;AAC1B,QAAM,aAAa,SAAS;AAC5B,MAAI,eAAe,SAAS;AAC1B,WAAO,OAAO,KAAK,4BAA4B,OAAO,SAAS,OAAO,UAAU,CAAC,IAAI,SAAS;AAAA,EAChG;AACA,MAAI,aAAa,UAAa,OAAO,aAAa,UAAU;AAC1D,WAAO,OAAO,KAAK,gDAAgD,OAAO,QAAQ,IAAI,OAAO;AAAA,EAC/F;AACA,QAAM,eAAe,aAAa,SAAY,aAAY,2CAAiB,QAAQ;AACnF,MAAI,iBAAiB,QAAW;AAC9B,WAAO,OAAO,KAAK,iEAAiE,YAAY,IAAI,OAAO;AAAA,EAC7G;AACA,MAAI,CAAC,MAAM,QAAQ,UAAU,GAAG;AAC9B,WAAO,OAAO,KAAK,qCAAqC,SAAS;AAAA,EACnE;AACA,QAAM,UAAU,OAAO,OAAO,QAAQ,YAAY,YAAY;AAC9D,MAAI,QAAQ;AACZ,MAAI,UAAU,QAAW;AACvB,UAAM,QAAQ,QAAQ,CAAC;AACvB,QAAI,UAAU,QAAW;AACvB,aAAO,OAAO,KAAK,+DAA+D,OAAO;AAAA,IAC3F;AACA,QAAI,QAAQ,KAAK,CAAC,WAAW,OAAO,UAAU,MAAM,KAAK,GAAG;AAC1D,aAAO,OAAO,KAAK,oEAAoE,OAAO;AAAA,IAChG;AACA,YAAQ,MAAM;AAAA,EAChB;AACA,SAAO,EAAE,SAAS,OAAO,SAAS,OAAO,OAAO,OAAO,EAAE;AAC3D,CAAC;AAWI,MAAM,UAAU,CAAC,QAAuD;AAC7E,QAAM,UAAiC,IAAI,aAAa;AAAA,IAAQ,CAAC,gBAC/D,YAAY,SAAS,UACjB,CAAC;AAAA,MACD,OAAO,IAAI;AAAA,MACX,MAAM,YAAY;AAAA,MAClB,QAAQ,YAAY;AAAA,MACpB,GAAI,YAAY,eAAe,SAAY,CAAC,IAAI,EAAE,YAAY,YAAY,WAAW;AAAA,MACrF,SAAS,YAAY;AAAA,MACrB,OAAO,YAAY;AAAA,IACrB,CAAC,IACC,CAAC;AAAA,EACP;AACA,SAAO,SAAS,EAAE,SAAS,OAAO,IAAI,OAAO,QAAQ,CAAC;AACxD;AAkBO,MAAM,OAAO,CAClB,aACuC;AACvC,QAAM,kBAAkB,SAAS,WAAW;AAC5C,QAAM,UAAU,SAAS;AACzB,SAAO,SAAS,EAAE,SAAS,iBAAiB,OAAO,SAAS,OAAO,QAAQ,CAAC;AAC9E;AAaO,MAAM,QAAQ,CAAC,aAA+B;AACnD,QAAM,UAAU,CAAC,eAAmC,0BAAS,OAAO,OAAO,OAAO,MAAM,OAAO,QAAQ,OAAO,OAAO;AACrH,QAAM,UAAU,SAAS,QACtB,IAAI,CAAC,WAAW,CAAC,QAAQ,MAAM,GAAG,MAAM,CAAU,EAClD,KAAK,CAAC,CAAC,IAAI,GAAG,CAAC,KAAK,MAAM,mCAAc,YAAY,MAAM,KAAK,CAAC,EAChE,IAAI,CAAC,CAAC,EAAE,MAAM,MAAM,MAAM;AAC7B,SAAO,mCAAc,UAAU;AAAA,IAC7B,SAAS,SAAS;AAAA,IAClB,OAAO,SAAS;AAAA,IAChB;AAAA,EACF,CAAC;AACH;AAcO,MAAM,OAAO,CAAC,SACnB,KAAK,SAAS,OAAO,SACjB,KAAK,4BAA4B,OAAO,MAAM,oBAAoB,KAAK,MAAM,IAAI,MAAM,IACvF,OAAO,IAAI;AAAA,EACX,KAAK,MAAM,KAAK,MAAM,IAAI;AAAA,EAC1B,OAAO,CAAC,UAAU,IAAI,2BAAU,EAAE,MAAM,oBAAoB,SAAS,8BAA8B,MAAM,CAAC;AAC5G,CAAC,EAAE,KAAK,OAAO,QAAQ,QAAQ,CAAC;",
+  "names": []
+}

package/dist/cjs/CaseExecutor.d.ts

@@ -0,0 +1,85 @@
+/**
+ * Injectable target-flow execution boundary.
+ *
+ * A suite says what to run; this service says how. Keeping it injectable is
+ * what lets the same suite run against a real flow, a scripted host, or nothing
+ * at all, and it is the only place a target's failure is converted into a typed
+ * evaluation failure.
+ *
+ * @since 0.1.0
+ */
+import type { Flow } from "@smthrs/core";
+import * as Context from "effect/Context";
+import * as Effect from "effect/Effect";
+import * as Layer from "effect/Layer";
+import { EvalError } from "./EvalError.ts";
+import type { Case } from "./Suite.ts";
+/**
+ * The result of executing one target-flow case.
+ *
+ * `target` is the flow value the case actually executed. A run matches it
+ * against each binding's `appliesTo` by reference identity, so it has to be the
+ * declared flow itself rather than a copy of it.
+ *
+ * @category models
+ * @since 0.1.0
+ */
+export interface Execution {
+    readonly output: unknown;
+    readonly stepKey: string;
+    readonly latencyMs: number;
+    readonly target: Pick<Flow.Any, "name">;
+}
+/**
+ * The one callback a case executor is.
+ *
+ * @category models
+ * @since 0.1.0
+ */
+export type Run = (suiteCase: Case) => Effect.Effect<Execution, EvalError>;
+/**
+ * Runtime shape for an injectable target-flow executor.
+ *
+ * @category services
+ * @since 0.1.0
+ */
+export interface Service {
+    readonly run: Run;
+}
+declare const CaseExecutor_base: Context.ServiceClass<CaseExecutor, "flows/evals/CaseExecutor", Service>;
+/**
+ * Injectable execution boundary for a target flow.
+ *
+ * @category services
+ * @since 0.1.0
+ */
+export declare class CaseExecutor extends CaseExecutor_base {
+}
+/**
+ * Builds an executor from its one callback.
+ *
+ * Throws a `TypeError` when there is no callback. An executor that silently
+ * degraded to {@link makeNoop} turned one wiring mistake into a whole suite of
+ * cases failing with `executor`, which reads as a broken target rather than a
+ * missing one.
+ *
+ * @category constructors
+ * @since 0.1.0
+ */
+export declare const make: (run: Run) => Service;
+/**
+ * Builds an executor that fails every case with a typed executor error.
+ *
+ * @category constructors
+ * @since 0.1.0
+ */
+export declare const makeNoop: () => Service;
+/**
+ * Provides the unavailable executor.
+ *
+ * @category layers
+ * @since 0.1.0
+ */
+export declare const layerNoop: Layer.Layer<CaseExecutor>;
+export {};
+//# sourceMappingURL=CaseExecutor.d.ts.map

package/dist/cjs/CaseExecutor.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"CaseExecutor.d.ts","sourceRoot":"","sources":["../../src/CaseExecutor.ts"],"names":[],"mappings":"AAAA;;;;;;;;;GASG;AAEH,OAAO,KAAK,EAAE,IAAI,EAAE,MAAM,cAAc,CAAA;AACxC,OAAO,KAAK,OAAO,MAAM,gBAAgB,CAAA;AACzC,OAAO,KAAK,MAAM,MAAM,eAAe,CAAA;AACvC,OAAO,KAAK,KAAK,MAAM,cAAc,CAAA;AACrC,OAAO,EAAE,SAAS,EAAE,MAAM,gBAAgB,CAAA;AAC1C,OAAO,KAAK,EAAE,IAAI,EAAE,MAAM,YAAY,CAAA;AAEtC;;;;;;;;;GASG;AACH,MAAM,WAAW,SAAS;IACxB,QAAQ,CAAC,MAAM,EAAE,OAAO,CAAA;IACxB,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAA;IACxB,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAA;IAC1B,QAAQ,CAAC,MAAM,EAAE,IAAI,CAAC,IAAI,CAAC,GAAG,EAAE,MAAM,CAAC,CAAA;CACxC;AAED;;;;;GAKG;AACH,MAAM,MAAM,GAAG,GAAG,CAAC,SAAS,EAAE,IAAI,KAAK,MAAM,CAAC,MAAM,CAAC,SAAS,EAAE,SAAS,CAAC,CAAA;AAE1E;;;;;GAKG;AACH,MAAM,WAAW,OAAO;IACtB,QAAQ,CAAC,GAAG,EAAE,GAAG,CAAA;CAClB;;AAED;;;;;GAKG;AACH,qBAAa,YAAa,SAAQ,iBAAoE;CAAG;AAEzG;;;;;;;;;;GAUG;AACH,eAAO,MAAM,IAAI,GAAI,KAAK,GAAG,KAAG,OAK/B,CAAA;AAED;;;;;GAKG;AACH,eAAO,MAAM,QAAQ,QAAO,OAUxB,CAAA;AAEJ;;;;;GAKG;AACH,eAAO,MAAM,SAAS,EAAE,KAAK,CAAC,KAAK,CAAC,YAAY,CAA2C,CAAA"}