eval-quality 1.0.0 → 1.2.0

package/README.md

@@ -1,36 +1,40 @@
 # `eval-quality`
 
 **[Documentation](https://bmad-code-org.github.io/bmad-eval-quality/)** ·
-[Getting started](https://bmad-code-org.github.io/bmad-eval-quality/tutorials/getting-started/) ·
+[Start here](https://bmad-code-org.github.io/bmad-eval-quality/tutorials/getting-started/) ·
 [CLI reference](https://bmad-code-org.github.io/bmad-eval-quality/reference/cli-commands/) ·
 [npm](https://www.npmjs.com/package/eval-quality)
 
+Write the eval. Hide the bug. See if the eval catches it.
+
 ```bash
 npx eval-quality --help
 ```
 
-### `eval-quality` does four things
+## The idea
 
-1. **Compile**: validate and normalize an eval contract into a machine-readable artifact.
-2. **Seal**: render the brief for the independent evaluator while hiding the planted bug and scoring answer.
-3. **Preflight**: verify baseline environment readiness and probe reachability before running an evaluator.
-4. **Score**: compare the evaluator's completed findings against the hidden bug signature and mint a versioned evidence artifact.
+An AI evaluation can pass and prove nothing. It sends a request, sees something plausible come back, and reports success while the failure it was written to catch sits right next to the thing it looked at. That is a blind spot, and a green run never shows you one.
 
-It executes nothing. No agent, no judge, and no system under test runs inside it; your harness runs the evaluation and hands over a sealed run record.
+Mutation testing finds blind spots in ordinary tests: keep the tests fixed, plant a known defect in the code, run the tests again, and ask whether they caught it. `eval-quality` points the same idea at evaluations.
 
-### What is the evaluation contract?
+```text
+clean system    → evaluation → should pass
+mutated system  → evaluation → should degrade
+                                    ↓
+                       did the evaluation catch it?
+```
 
-It is the test.
+An evaluation that caught the planted defect is sensitive to that failure. One that stayed green has a blind spot, and now you know where.
 
-More precisely, it is the evaluator’s instructions for how to expose a failure and what evidence counts as finding it. The long name is Behavioral Evaluation Contract; the docs shorten it to eval contract or evaluation contract.
+## The evaluation contract
 
-It defines:
+The contract is the test: the evaluator's instructions for how to expose a failure and what evidence counts as finding it. The long name is Behavioral Evaluation Contract; the docs shorten it to eval contract. It is a JSON document that declares:
 
 - the behavior being evaluated;
 - the probes the evaluator should perform;
 - the evidence it should inspect;
 - the negative behavior it must rule out;
-- the oracle that determines pass or fail.
+- the oracle that decides pass or fail.
 
 For example:
 
@@ -48,10 +52,6 @@ A weak eval checks only the response and misses the bug.
 
 A strong eval checks the response **and** persistence, so it catches the bug.
 
-### Caveman summary
-
-Write the eval. Hide the bug. See if the eval catches it.
-
 ## The core flow, in eight nouns
 
 Every run of an evaluation walks the same order:
@@ -66,14 +66,16 @@ evaluation contract → probe → observation → preflight → evidence → ora
 | **Probe** | How to poke the system to produce evidence: a test case, a call, a step. In scoring, a probe also names the defect it seeded. | "Update note n-1, then read it back." |
 | **Observation** | What actually happened when the system was poked: the recorded status, headers, and body of one call. | `PATCH` returned 200 with the new title; the later `GET` returned the old one. |
 | **Preflight** | Whether the environment and the observations are fit for meaningful measurement. | Both operations reachable, the fixture reset, the clean control clean. |
-| **Evidence** | The recorded output, trajectory, and artifacts from the evaluation run: what the evaluator saw and what it claimed. | A finding citing the two observations above. |
+| **Evidence** | The recorded output, trajectory, and artifacts from the evaluation run: what the oracles resolve against. | A finding citing the two observations above. |
 | **Oracle** | The assertion: the relation that has to hold over the evidence. | The title sent equals the title read back. |
 | **Rubric** | The grading guide for judgment-heavy quality, with anchored criteria a judge scores against. The judge runs outside the package and its scores arrive in the sealed run record. | Present only when a contract declares one. |
 | **Score / verdict** | The combined result: did the evaluation catch the planted defect? `PASS`, `WAIVED`, `CONCERNS`, or `FAIL`, or Invalid when the run produced no verdict. | `FAIL`, exit code 2. |
 
 The word evidence is used twice on purpose. The evidence in the flow is what the evaluator produced, and it reaches `score` inside a sealed run record. The evidence artifact is what `score` mints at the end: the outcomes, the verdict, the strength vector, and the exit code.
 
-The four commands sit on that flow like this:
+## What the tool does
+
+`eval-quality` is a Node package and a command line binary. It gives you four commands, which sit on the flow like this:
 
 | Command | Reads | Writes |
 | --- | --- | --- |
@@ -82,69 +84,11 @@ The four commands sit on that flow like this:
 | `preflight` | a contract, a probe list, observations | `preflight-verdict.json`, fit or unfit to measure |
 | `score` | a sealed run record, the contract, a probe, the preflight verdict, a scoring policy, a caller-attested corpus digest, and the isolation manifest and evaluator configuration the record was produced under | `evidence-artifact.json`, and the verdict's own exit code |
 
-## Elaboration
-
-Compile disciplined agent eval contracts, then check whether those contracts can catch known bugs.
-
-An agent can produce an answer that reads as correct and is materially wrong. An eval can make the same mistake.
-
-Weak oracle:
-
-```text
-Check malformed input is handled correctly.
-```
-
-An evaluator given that instruction sends one malformed request, sees an error come back, and reports success. The record that should never have been created was created anyway. Nobody looked.
-
-Strong oracle:
-
-```text
-Send malformed input. Verify the request fails, inspect the full response body,
-confirm the specific error, and confirm no record was created.
-```
-
-A passing eval says little when the contract never asked for the probe that would expose the failure. Testing whether the eval can catch a failure you already know about is the first check worth running.
-
-The loop that does that is a twin run. Keep the contract, the probes, the oracles, and the scoring policy fixed. Run the evaluator once against the clean system and once against the same system carrying one known defect. Score both runs. The contract is strong when the clean run passes and the mutated run degrades, and it has a blind spot when both stay green.
-
-## What each part provides
-
-`eval-quality` provides:
-
-- the Behavioral Evaluation Contract schema
-- the oracle vocabulary and authoring rules
-- the contract compiler
-- the environment pre-flight
-- Eval Contract strength scoring: the AD-7 rate vector and dominance relation, implemented in
-  `src/core/score/strength.ts` and reached by the `score` command
-- PASS / WAIVED / CONCERNS / FAIL governance: both verdict ladders, implemented and total in
-  `src/core/score/ladder.ts`, and the `score` command's own exit code
-- versioned evidence output: `evidence-artifact.json`, implemented in `src/core/emit/emit.ts` and
-  minted by the `score` command
-
-The caller provides:
-
-- execution of its chosen agent, harness, or person
-- repeated trials
-- cost accounting
-- the live system and environment-probe implementation
-- a sealed run record returned for ingestion
-
-`eval-quality` executes nothing: it never spawns a process, calls a model, drives a system under test,
-or invokes a judge. Its six stages are compile, seal, ingest, pre-flight, score, and emit, all pure,
-and every one is reachable through the CLI and the library alike. That list is the declared stage
-order; on the clock, ingest follows the evaluator run, so it sits after pre-flight and just before
-score. Compile, seal, and pre-flight each
-have their own command and their own exported function. `ingest`, `score`, and `emit` are reached
-through the one `score` command and the one exported `runScore` call that chains them, per AD-14's
-rule that a command exposes no more than the library itself calls. Pre-flight probes the fixture through the environment-probe port, so a contract that declares a fixture reset
-needs the caller's probe policy to authorize that operation's method as well as the read methods
-every other pre-flight leg uses. Engine integration is a later adapter behind a port. See
-[ADR-004](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-29/ADR-004-execution-boundary.md).
+It executes nothing. No agent, no judge, and no system under test runs inside it. You run the agent or harness, the repeated trials, and the live system with its environment probe, and you hand over a sealed run record. `eval-quality` compiles, seals, preflights, and scores.
 
 ## Who it is for
 
-Teams shipping AI agents, coding skills, review bots, MCP-based assistants, or automated test-generation systems, and teams operating human-on-the-loop or dark-factory delivery.
+Teams shipping AI agents, coding skills, review bots, MCP-based assistants, or automated test-generation systems, and teams that let agents ship with little or no human review.
 
 Use `eval-quality` when all three are true:
 
@@ -154,57 +98,43 @@ Use `eval-quality` when all three are true:
 
 Deterministic work does not need it and already has cheaper, stronger evidence from unit, integration, contract, E2E and performance testing.
 
-## Behavioral Evaluation Contracts
+## The authoring discipline
 
-A **Behavioral Evaluation Contract** is a versioned specification of the behaviors to probe, the evidence to collect, the negative cases to exercise, and the rules that decide whether the system passes or fails. **Eval Contract** is the shorthand used from here on. The individual checks inside it are **oracles**. The contract carries no prescribed action sequence; the evaluator chooses its own path.
+A contract carries no prescribed action sequence; the evaluator chooses its own path. What the contract fixes is seven rules that survived the experiments: separate the success indicator from the body, read the whole body, probe malformed and negative inputs, verify per record, cross-check sibling parameters and sibling tools, check for omissions and completeness, and read a state change back after writing it.
 
-The authoring discipline is a small set of rules that survived the experiments: separate the success indicator from the body, read the whole body, probe malformed and negative inputs, verify per record, and cross-check sibling parameters and sibling tools.
+The compiler enforces those rules against the contract artifact, in three classes. Structural errors fail compilation. Coverage gaps score down without blocking. A waived rule is allowed when the waiver records the rule name, a rationale, a machine-checkable condition, and the approval.
 
-A compiler enforces these rules mechanically against the contract artifact, in three classes. Structural errors fail compilation. Coverage gaps score down without blocking. A waived pattern is allowed when it records the named rule, a rationale, a machine-checkable condition, and the approval.
+Rubrics compile under the same discipline: an anchored scale, named criteria that each state a question, and evidence pointers that resolve against the declared interfaces.
 
-Rubrics compile under the same discipline: an anchored scale, a bounded length, named failure-mode penalties, rubric identifiers unique across the contract and criterion identifiers unique inside their own rubric, every criterion stating a question, and every criterion's evidence pointer resolving against the declared interfaces. Authored rubric text that asks a judge to grade the subject's own stated reasoning fails a closed-vocabulary check over the wording.
+## How contract strength is scored
 
-## How Eval Contract strength scoring works
+Do not trust a contract because it looks thorough. Put a known defect behind it, run the evaluator, and check whether the contract's oracles caused the defect to be caught.
 
-The `score` command and its `runScore` library call compute it; `npm run generate:worked-example`
-runs the same functions over the committed worked chain, and
-[the full walkthrough](https://bmad-code-org.github.io/bmad-eval-quality/how-to/author-behavioral-contracts/)
-reads the result field by field. Do not trust a contract because it looks thorough. Put a known defect behind it, run the evaluator, and check whether the contract's oracles caused the defect to be caught.
-
-Two probe classes go behind a contract, and a strong contract rejects both:
+Three probe classes go behind a contract, and a strong contract catches all three:
 
 - **Defect probes**, where the behavior is simply wrong.
 - **Gameability probes**, where the behavior looks compliant while dodging the oracle's intent. A test that raises coverage while asserting nothing is the familiar version of this.
+- **Zero-action probes**, where the system does nothing and reports success.
 
-Probes come from qualified historical defects or verified controlled mutations. The corpus separates a visible development set from an immutable sealed set for each scoring version. Only the development set exists today; the sealed set is part of the design and ships in no release yet.
+Canary probes and clean controls are run too, and never enter the strength vector.
 
-Every required oracle check resolves to exactly one state, and the state travels with the result, so
-"the check reported" is never sufficient on its own: `caught`, `confirmed`, `missed`,
-`passed-clean-control`, `false-positive`, `abstained`, `bypassed`, `unreached`, `oracle-error`,
-`judge-error`, `infrastructure-error`, or `not-applicable`.
+Every required oracle check resolves to exactly one of twelve states, and the state travels with the result: `caught`, `confirmed`, `missed`, `passed-clean-control`, `false-positive`, `abstained`, `bypassed`, `unreached`, `oracle-error`, `judge-error`, `infrastructure-error`, or `not-applicable`. A required oracle that missed, abstained, errored, or is absent prevents PASS, and a high overall score never overrides it. An infrastructure error or a failed pre-flight invalidates the run; run it again.
 
-A required oracle that missed, abstained, errored, or is absent prevents PASS, and a high overall score never overrides it. An infrastructure error or a failed environment pre-flight is not a behavioral result at all; it invalidates the run, and the run is re-executed.
+A caught defect is decided by evidence. A finding counts as detection only when the probe's declared defect signature matches an observation the finding cites; the evaluator's own claim does not settle it.
 
-Repeated runs of one probe are trials, and they reduce to one result per probe before any rate is computed. The `score` stage takes a trial set; the `score` command and `runScore` hand it one sealed run record per call, so a run scored from the published surface completes one trial, and against a policy declaring a minimum of three, as the worked example's does, its strength vector is reported and marked non-comparable.
+Repeated runs of one probe are trials, reduced to one result per probe before any rate is computed. The `score` command and `runScore` hand the stage one sealed run record per call, so against a policy that asks for more than one trial the strength vector is reported and marked non-comparable.
 
-## Using it
+[The full walkthrough](https://bmad-code-org.github.io/bmad-eval-quality/how-to/author-behavioral-contracts/) reads a scored run field by field, down to its verdict and exit code.
 
-`eval-quality` is its own repository and package, with no framework around it.
-
-The **library** is the primary surface. It exports the artifact types, the compiler, the pre-flight, `runScore`, the canonical digest, the lineage validator, and the failure-code and verdict registries. The Zod schemas themselves are not exported; they are published as JSON Schema under `eval-quality/schemas/*`. The published typed schema is what lets coding agents author contracts correctly by default, which is how the discipline scales beyond the people who went looking for the tool.
-
-The **CLI** wraps the same library for callers that cannot import TypeScript: CI jobs, GitHub Actions, PR-review and unit-test bots, other frameworks' skills, and any agent permitted to run a shell command.
-
-### What the CLI Commands Do
+## Using it
 
-- **`compile`**: Typechecks an authored `eval-contract.json`. Verifies that all behaviors, oracles, rubrics, and sensitivity witnesses comply with structural and authoring rules.
-- **`seal`**: Generates a `sealed-evaluator-brief.json` by stripping secret defect signatures, planted answers, and author commentary. The brief carries only the directions and safety bounds the evaluator needs.
-- **`preflight`**: Reduces caller-supplied probe observations against the contract to verify environment baseline readiness and probe reachability. All four of `--contract`, `--probes`, `--observations`, and `--run-id` are required. Halts early with exit code `3` if the environment is unready. `schemas/probe.schema.json` gives the shape of one probe in the list; each observation echoes a planned leg's id back as `probeId`, and the getting-started tutorial writes six by hand.
-- **`score`**: Chains ingest, score, and emit over one sealed run record, minting `evidence-artifact.json` and exiting with the AD-21 verdict's own exit code. `schemas/sealed-run-record.schema.json` gives the record's shape, and the committed worked chain under `_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-29/spike-worked-example/` carries one, scored. `--record`, `--contract`, `--probe`, `--preflight-verdict`, `--policy`, and `--corpus-digest` are required; `--isolation-manifest` and `--evaluator-configuration` are each optional and their absence invalidates the run and the command still parses; `--private-manifest` is optional and, when given, each entry's declared digest is checked against its resolved bytes. `--corpus-root` names the directory a private reference resolves under, and is required only when `--private-manifest` or a private-storage isolation-manifest reference is actually present.
+Node.js 22.20.0 or newer. `zod` is the only production dependency.
 
-### Running the CLI
+```bash
+npm install eval-quality
+```
 
-Every command runs through `npx` without installing anything:
+Every command runs through `npx`:
 
 ```bash
 npx eval-quality compile --in contract.json --out ./eval-out
@@ -221,18 +151,7 @@ npx eval-quality score --record record.json --contract contract.json \
   --out ./eval-out
 ```
 
-Every command is non-interactive: no prompt, no terminal check, and no behaviour that differs when
-stdin is a pipe. Each one is a single call into the library plus artifact serialization.
-
-**Input and output.** `--in` is the only input that falls back to stdin: `compile` and `seal` read it
-when `--in` is left out. `-` names stdin explicitly on any input, and at most one input may be `-` per
-invocation. `compile` and `seal` each take one input; `preflight` takes three, all required; `score`
-takes eight, three of them optional (`--isolation-manifest`, `--evaluator-configuration`, and
-`--private-manifest`). Without `--out` the artifact goes to stdout, so a command composes with a pipe.
-An `--out` ending in `.json` is a file path; anything else is a directory, and the artifact is
-written to `<target>/<kind>.json` where `kind` is `eval-contract`, `sealed-evaluator-brief`,
-`preflight-verdict`, or `evidence-artifact`. Diagnostics and errors go to stderr, always, so stdout
-carries the artifact alone.
+Every command is non-interactive. Without `--out` the artifact goes to stdout, so a command composes with a pipe. An `--out` ending in `.json` is a file path; anything else is a directory, and the artifact lands at `<target>/<kind>.json`. Diagnostics and errors go to stderr, so stdout carries the artifact alone.
 
 **Exit codes.**
 
@@ -246,81 +165,51 @@ carries the artifact alone.
 | `5` | runtime fault |
 | `64` | usage error |
 
-`--strict` never promotes a CONCERNS whose firing conditions are all evidence conditions: those
-conditions report that the measurement fell short of the policy. Codes 1 and 2 report a verdict
-`score`'s ladder resolved, read directly off `LadderResolution.exitCode`; every other invalidating
-condition behind code 3 is reachable through `score` too, alongside the failed pre-flight `preflight`
-itself reports.
+`--strict` never promotes a CONCERNS whose firing conditions are all evidence conditions: those report that the measurement fell short of the policy. Codes 1 and 2 come from `score`'s verdict ladder. Code 3 comes from a failed pre-flight, which `preflight` reports, or from any other invalidating condition `score` finds.
+
+`--strict` is accepted on every command. `--strict-inputs` and `--no-strict-inputs` are a different switch: they set the compiler's input strictness, on by default, and only `compile` and `seal` accept them.
 
-`--strict` is the gate-promotion flag and is accepted on every command. `--strict-inputs` and
-`--no-strict-inputs` are a different switch: they set the compiler's input strictness, which is on
-by default, and `preflight` and `score` each reject both with exit `64` because neither has a compile
-step.
+The [CLI reference](https://bmad-code-org.github.io/bmad-eval-quality/reference/cli-commands/) has every flag, every parsing rule, and the package exports.
 
-**The published JSON Schema.** A consumer that does not read TypeScript validates against the
-twelve generated documents, published at the `eval-quality/schemas/*` subpath:
+**The library** exports the same stages as functions, plus the artifact types, the canonical digest, the lineage validator, and the failure-code and verdict registries. The schemas are published as JSON Schema under `eval-quality/schemas/*`, which is what lets a coding agent author contracts correctly by default:
 
 ```ts
 import spec from 'eval-quality/schemas/eval-contract.schema.json' with { type: 'json' }
 ```
 
-The import attribute is required: ESM on Node 22 and 24 both throw `ERR_IMPORT_ATTRIBUTE_MISSING`
-without it. The development corpus ships the same way, at `eval-quality/corpus/dev/`, so an adopter
-can read real compiled contracts and one compiled-and-sealed pair without cloning this repository.
-`eval-quality/adapters` is one of the five published subpaths, holding the four reference adapters
-the conformance suite runs against.
-
-Eleven of the twelve published schemas carry a `schemaVersion`. `artifact-reference` is exempt: it
-is embedded inside other artifacts, so it has no version to break.
+The import attribute is required: ESM on Node 22 and 24 both throw `ERR_IMPORT_ATTRIBUTE_MISSING` without it. The development corpus ships the same way, at `eval-quality/corpus/dev/`, so you can read twenty-one real contracts and one compiled-and-sealed pair without cloning this repository.
 
-`schemaVersion` is declared as any integer at or above 1, so a document at an unexpected version
-parses. The bumps in the next release each add a required field, which is why an older document
-fails; the version itself is compared in exactly one place, `validateLineageChain`, over
-lineage-chain members, and nowhere on the command path. The package is pre-1.0, so pin exactly.
-`CHANGELOG.md` records what each release breaks.
+Version 1.0 is out and the published surface is stable: a breaking change to a command, an export, or a schema is a major version bump. `compile` refuses a contract whose `schemaVersion` differs from the one this build reads, so check the stamp on anything you did not author against this version. `CHANGELOG.md` records what each release breaks.
 
 ## Relationship with BMad and TEA
 
-The dependency runs one way: TEA uses `eval-quality`, and `eval-quality` knows nothing about TEA.
+TEA is the BMad Test Architect. The dependency runs one way: TEA uses `eval-quality`, and `eval-quality` knows nothing about TEA.
 
 ```mermaid
 graph LR
   TEA["TEA<br/>(reference authoring client)"] -- "drafts a contract, then calls" --> EQ["eval-quality<br/>(this package)"]
 ```
 
-TEA is the reference authoring client. It reads BMad planning artifacts, notices eval-relevant work, drafts a contract, and calls this package. It is not co-installed, and `eval-quality` holds no knowledge of TEA, BMad, or any planning-artifact format.
+TEA is the reference authoring client. It reads BMad planning artifacts, notices eval-relevant work, drafts a contract, and calls this package. Any human, bot, CI job, skill, or other framework can author a contract and use `eval-quality` directly. The compiler judges the artifact, whoever produced it.
 
-Any human, bot, CI job, skill, or other framework can author a contract and use `eval-quality` directly. The discipline still applies, because the compiler judges the artifact, whoever produced it.
+### Example: testing a `bmad-tea` knowledge harness
 
-Evaluator runs remain isolated to prevent builder-context leakage and preserve traceability. Stronger contract oracles produced the measured detection improvement.
-
-### Real-World Walkthrough: Testing a `bmad-tea` Knowledge Harness
-1. Author an `eval-contract.json` declaring required knowledge step files (e.g. `playwright-utils-mandate.md`).
-2. Run `eval-quality compile --in contract.json` to validate contract structure and discipline rules.
-3. Run `eval-quality seal --in contract.json --out ./run` to generate `sealed-evaluator-brief.json`.
-4. Probe the harness's environment and run `eval-quality preflight` over the observations; a verdict that does not pass is exit `3`, and the run stops there.
-5. Pass `sealed-evaluator-brief.json` to `bmad-tea` to execute the task without seeing answer keys, once against the clean harness and once against a harness with one known step file removed.
+1. Author an `eval-contract.json` declaring the required knowledge step files (for example `playwright-utils-mandate.md`).
+2. Run `eval-quality compile --in contract.json` to check the contract's structure and discipline rules.
+3. Run `eval-quality seal --in contract.json --out ./run` to produce `sealed-evaluator-brief.json`.
+4. Probe the harness's environment and run `eval-quality preflight` over the observations. A verdict that does not pass is exit `3`, and the run stops there.
+5. Hand `sealed-evaluator-brief.json` to `bmad-tea` to execute the task without seeing answer keys, once against the clean harness and once against a harness with one known step file removed.
 6. Seal each evaluator run into a `sealed-run-record.json` and run `eval-quality score` over it with the probe that names the removed file as the seeded defect. The clean run should pass; the mutated run should degrade, and the exit code says which.
 
 ## Evidence and limitations
 
-Holding the model, the budget, the system, and the defects fixed, and changing only how the Eval Contract was authored, sealed-evaluator detection moved from **0.33 to 1.00** across three naturally occurring defects, three repetitions per arm, 19 scored runs.
+Holding the model, the budget, the system, and the defects fixed, and changing only how the eval contract was authored, sealed-evaluator detection moved from **0.33 to 1.00** across three naturally occurring defects, three repetitions per arm, 19 scored runs.
 
-Both experiment rounds missed at least one preregistered gate. Round 1 recorded `DARK-FACTORY REJECTED`; round 2 block 1 recorded `CONTRACT-DISCIPLINE NOT SUPPORTED`, failing one gate of five on a single unreplicated clean control. The separation comes from two of the three defects, since both arms detected the third in every repetition, and both separating cases carry a recorded measurement-layer confound. The sample covers three defects, one system, and one model. This supports a product-direction decision at narrow scale. Certification would require broader replication.
+Both experiment rounds missed at least one preregistered gate; round 2 failed one gate of five on a single unreplicated clean control. The separation comes from two of the three defects, since both arms detected the third in every repetition, and both separating cases carry a recorded measurement-layer confound. The sample covers three defects, one system, and one model. This supports a product-direction decision at narrow scale. Certification would require broader replication.
 
 Read the [product brief](_bmad-output/planning-artifacts/briefs/brief-eval-quality-2026-07-17/brief.md) for the product rationale and the [PRD](_bmad-output/planning-artifacts/prds/prd-eval-quality-2026-07-17/prd.md) for build requirements. The experiment record includes the [round 1 verdict](experiments/hypothesis-validation/DECISION.md), [round 2 results](experiments/hypothesis-validation/PHASE2-RESULTS.md), [metric summary](experiments/hypothesis-validation/results/summary.md), and [protocol](experiments/hypothesis-validation/HYPOTHESIS_VALIDATION_PLAN.md).
 
-## Architecture status
-
-The [architecture spine](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-29/ARCHITECTURE-SPINE.md) is split by pipeline half, and its own status line still reads that the compile-and-seal half is epic-ready while the score half is not. The code has moved past that line. Epic 7 delivered AD-21, AD-33, and AD-40 as pure functions with generated tables, and epic 8 shipped the `ingest`, `score`, and `emit` stages, the `score` command, and `runScore` over them, which closes every item the spine's *Owed to the reference implementation* section listed. The `score` stage consumes a trial set; the command and `runScore` hand it one record per call, so a run scored from the published surface completes one trial, and whenever the policy's declared minimum exceeds one its strength vector is reported and marked non-comparable. Gate C closed at zero blocking authoring points and 14 of 14 declaration-only predicates. Gate D's generated-current-fields arm matched the hand-written positive control at 3 of 3 seeded-defect catches, so `seal` joins the stage-one order without adding an evidence-precondition field.
-
-Contract strength scoring has been open since [ADR-007](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-29/ADR-007-compile-score-split.md): three rounds of external review established that the catch rate was 1.00 by construction, because nothing matched a finding to the defect its probe seeded. That input now exists, and so does the mapping that reads it: `src/core/score/witness.ts` is AD-40's witness match, delivered by epic 7. What is still owed is its validation against the block-2 replication, which the spine records as committed and not yet run.
-
-Contract compilation was declared ready in ADR-007 and a fourth review withdrew that claim in [ADR-008](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-29/ADR-008-compile-half-owed-to-calibration.md). The named calibration is now complete. The absent local-only mut2 arm was reconstructed from its recorded base, reproduced its prior black-box behavior, and ran under a pre-registered three-arm, three-repetition design. All three arms composed filters and detected the seeded defect in every valid repetition. This closes the calibration gate narrowly; it does not generalize the historical 0.33-to-1.00 effect beyond one behavior and one controlled mutation.
-
-Both are documented as defects, because four rounds have shown that a confidently worded revision is the thing that goes wrong here.
-
-The decision record, in order: [ADR-001](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-22/ADR-001-evaluator-isolation-boundary.md) on evaluator isolation, [ADR-002](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-22/ADR-002-contract-authoring-discipline.md) on why authoring discipline is the product, [ADR-003](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-29/ADR-003-measurement-mechanics.md) on measurement mechanics, [ADR-004](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-29/ADR-004-execution-boundary.md) on why this package executes nothing, [ADR-005](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-29/ADR-005-review-round-corrections.md) and [ADR-006](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-29/ADR-006-interaction-plan.md) on what review and hand-authoring corrected, [ADR-007](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-29/ADR-007-compile-score-split.md) on the split, [ADR-008](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-29/ADR-008-compile-half-owed-to-calibration.md) on why the other half stopped claiming to be finished too, and [ADR-009](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-29/ADR-009-adversarial-gate-corrections.md) on the seventeen places where two conforming implementations still disagreed. Review triage lives in [`reviews/`](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-29/reviews/).
+The design record is the [architecture spine](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-29/ARCHITECTURE-SPINE.md), its nine decision records, and the review triage in [`reviews/`](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-29/reviews/).
 
 ## Not building now
 
@@ -330,97 +219,32 @@ Out of scope entirely: a new eval engine, a hosted service, a dashboard or GUI,
 
 ## Development
 
-Node `>=22.20.0`, which `package.json` declares as the engine floor. `zod` is the only production
-dependency.
-
 ```bash
 npm install
-npm run validate            # build, typecheck, lint, docs, doc invocations, shareable, spine, vectors, schemas, both code registries, the AD-21, AD-31 and AD-33 tables, layers, lineage, boundary, corpus, worked chain, website deps, tests with coverage
+npm run validate            # the whole gate: build, typecheck, lint, every drift check, tests with coverage
 npm run build               # emit to dist/
+npm run test                # run the suite once
 npm run lint:fix            # auto-fix with Biome
-npm run test:coverage       # run the suite and fail below AD-30's 90 percent statement and branch floor on core/
-npm run generate:schemas    # rebuild schemas/*.schema.json from the Zod source
-npm run check:schemas       # fail if the committed schemas differ from the source by one byte
-npm run check:ad5-registry  # fail if the compile-time failure-code list drifts from the AD-5 table
-npm run check:ad28-registry # fail if the runtime fault-code list drifts from the AD-28 table
-npm run check:lineage       # fail if a module outside the stage table writes an artifact's lineage fields
-npm run check:boundary      # fail if anything the tarball carries references the planning system that produced it
-npm run generate:ad21-table # rebuild docs/ad21-verdict-decision.generated.md from the two verdict ladders
-npm run check:ad21-table    # fail if the committed AD-21 table differs from the builder by one byte
-npm run generate:ad31-table # rebuild docs/ad31-coverage-predicates.generated.md from the predicates
-npm run check:ad31-table    # fail if the committed AD-31 table differs from the builder by one byte
-npm run generate:ad33-table # rebuild docs/ad33-outcome-decision.generated.md from the decision procedure
-npm run check:ad33-table    # fail if the committed AD-33 table differs from the builder by one byte
-npm run generate:dev-corpus # rebuild corpus/dev/ from the contract fixtures through the shipped compile and seal
-npm run check:corpus        # fail if the committed corpus differs from the builder by one byte
-npm run generate:worked-example # rebuild the spike worked chain by running the compile, seal, score, and emit functions over it
-npm run check:worked-example    # fail if the committed worked chain differs from the builder by one byte
-npm run build:shareable     # render the planning artifacts to self-contained HTML
 npm run test:conformance    # run the published port conformance suite against every shipped adapter
 ```
 
-`schemas/` holds the twelve published JSON Schema documents, generated from the Zod definitions and
-committed. They are the contract for consumers who do not read TypeScript, so they are proven
-equivalent to the source: a byte-exact drift check, a rejection suite
-asserting the validator keyword and instance path for every negative fixture, a differential check
-comparing Zod's verdict against a third-party validator's over a generated corpus, and a
-keyword-mutation sweep that deletes each published constraint and requires some fixture to notice.
-Edit the Zod schema and regenerate; never hand-edit a file under `schemas/`.
-
-Every artifact the library hands back is deep-frozen, so it cannot be changed in place. This package
-is ES modules, which are always strict, so an attempt throws a `TypeError` there; a sloppy-mode
-caller sees the write fail silently. A revision is minted as a new artifact carrying its parent's
-digest and a revision count one greater. `check:lineage` fails the build when a lineage field is
-written outside `src/core/schemas/`, `src/core/lineage/`, and the modules the AD-24 stage table
-names as that artifact's producer, which today are `src/core/seal/seal.ts`,
-`src/core/preflight/reduce.ts`, and `src/core/emit/emit.ts`.
-
-The `eval-quality/conformance` subpath publishes the port boundary: the four port types, the message
-shapes they carry, the AD-28 `RUNTIME_FAULT_CODES` registry and `RuntimeFaultCode` type a conforming
-adapter throws against, and an executable conformance suite. An adapter is conforming when
-`runCorpusPortConformance`, `runClockPortConformance`, `runFileSystemPortConformance`,
-`runEnvironmentProbePortConformance`, or `runCommandLineProbeConformance` returns a report whose
-`passed` is true, which is the definition; each returns a report, so the suite carries no test
-framework and runs under whichever one you already use.
+Several files are generated from the code and guarded byte for byte, so a hand edit fails the build. Regenerate them:
 
-```ts
-import { runCorpusPortConformance, type CorpusPort } from 'eval-quality/conformance'
-```
+| Generated | Rebuild | Guard |
+| --- | --- | --- |
+| `schemas/*.schema.json`, from the Zod source | `npm run generate:schemas` | `npm run check:schemas` |
+| `corpus/dev/`, from the contract fixtures | `npm run generate:dev-corpus` | `npm run check:corpus` |
+| `docs/ad21-verdict-decision.generated.md`, the two verdict ladders | `npm run generate:ad21-table` | `npm run check:ad21-table` |
+| `docs/ad31-coverage-predicates.generated.md`, the coverage predicates | `npm run generate:ad31-table` | `npm run check:ad31-table` |
+| `docs/ad33-outcome-decision.generated.md`, the outcome decision procedure | `npm run generate:ad33-table` | `npm run check:ad33-table` |
+| the committed worked chain | `npm run generate:worked-example` | `npm run check:worked-example` |
+| `_bmad-output/shareable/`, this README, CONTRIBUTING, and the planning artifacts as standalone HTML | `npm run build:shareable` | `npm run check:shareable` |
+
+Every artifact the library hands back is deep-frozen. A revision is a new artifact carrying its parent's digest and a revision count one greater, and `npm run check:lineage` fails the build when a lineage field is written outside the modules that own it. `npm run check:boundary` fails it when anything the tarball carries references the planning system that produced it.
+
+`eval-quality/conformance` publishes the port boundary: the four port types, the fault registry a conforming adapter throws against, and an executable conformance suite that returns a report and runs under whichever test framework you already use. The [CLI reference](https://bmad-code-org.github.io/bmad-eval-quality/reference/cli-commands/) describes the ports and the suite.
 
-The suite drives a subject through four scenarios and checks six assertions per port method: a
-mechanism failure is a typed fault, exactly one underlying call happens on success and on failure, an
-aborted signal rejects promptly, an in-band error value is thrown as a fault, and a
-successful call returns a response the published schema accepts. `EnvironmentProbePort` adds a second
-arm per mechanism: `runEnvironmentProbePortConformance` (`api`) adds thirteen more from AD-35's
-default-deny target policy, and `runCommandLineProbeConformance` (`cli`) adds nine, once
-`CommandTargetPolicy` gave that mechanism something to authorize — an unmapped interface, an
-unmapped executable, an unauthorized subcommand path, a non-zero exit read as an observation, a
-shell-metacharacter argument proven to reach the process as one literal token, a declared artifact
-captured, and both caps enforced. `npm run test:conformance` runs the suite against the four
-adapters this package ships and against two in-repository probe subjects, one per mechanism, that
-exist only as the suite's own subjects.
-
-`docs/ad21-verdict-decision.generated.md` holds AD-21's two published verdict ladders, production and
-contract-scoring, emitted from the rule tables in `src/core/score/ladder.ts` together with the
-fixtures that exercise them, and guarded by `npm run check:ad21-table`. Each row carries its
-condition, its rung, the guard in prose, and whether `--strict` may promote it.
-
-`docs/ad31-coverage-predicates.generated.md` holds AD-31's published predicate table, emitted from
-the seven relevance predicates and their seven satisfaction twins run over a hand-authored contract
-corpus. It is generated by `npm run generate:ad31-table` and guarded by `npm run check:ad31-table`,
-a byte-exact drift check that fails when a predicate changes and the committed document does not, so
-the table is evidence the predicates produce. A hand edit fails the check; regenerate.
-
-`docs/ad33-outcome-decision.generated.md` holds AD-33's published decision table: the ten
-invalidating conditions, the twenty-row outcome ladder, the two waiver rules, the eight
-corroboration rules, the named structural constraints with the infeasible input pairs derived from
-them, and five censuses over the fixture set. It is generated by `npm run generate:ad33-table` and
-guarded by `npm run check:ad33-table`, the same byte-exact drift check, and the builder refuses to
-publish a census cell at zero, so a rule or a state losing its last fixture fails the build. AD-33
-puts a cell-per-input-tuple table out of arithmetic reach, so what is published is the enumerated
-output of the total function itself. A hand edit fails the check; regenerate.
-
-`build:shareable` renders this README, the product brief, the PRD, the architecture spine, all nine ADRs, and every document those pages link to (contributing, code of conduct, security, licence, and the four experiment records) to `_bmad-output/shareable/` as standalone styled HTML for sharing outside the repo. Rendering the linked documents is what lets a recipient without repository access follow the evidence, contribution, security, and licence links; anything that has no page of its own, such as a directory, is marked in the export as needing repository access. A hand edit fails the check; regenerate: `check:shareable` fails the build when the committed export is stale or carries a repository URL that is not the canonical one. Mermaid diagrams render as code blocks there, which is a known limitation.
+[CONTRIBUTING.md](CONTRIBUTING.md) covers the gate and the release process.
 
 ## Contributing
 

package/corpus/dev/README.md

@@ -1,43 +1,39 @@
 # Development corpus
 
-Nineteen contracts and one compiled-and-sealed pair, published so an adopter can read real input to
+Twenty-one contracts and one compiled-and-sealed pair, published so an adopter can read real input to
 this package without cloning the repository. Everything here is generated by
 `npm run generate:dev-corpus` and checked byte for byte by `npm run check:corpus`.
 
 ## What is here
 
-- `contracts/<contractId>.json`: twenty-one contracts. Nineteen are one per AD-20 discipline
-  rule in each declaration state, and two describe a system under test that runs behind a command
-  rather than over HTTP. Eighteen are published only after this package's own compile stage accepts
-  them, so every one of those is a contract the compiler admits. Three fail compilation by design;
-  those ship as authored input, and `index.json` records the failure code each one raises.
+- `contracts/<contractId>.json`: twenty-one contracts. Nineteen are one per discipline rule in each
+  declaration state, and two describe a system under test that runs behind a command. Eighteen are
+  published only after this package's own compile stage accepts them, so every one of those is a
+  contract the compiler admits. Three fail compilation by design; those ship as authored input, and
+  `index.json` records the failure code each one raises.
 - `compile-seal-example/contract.json` and `compile-seal-example/brief.json`: one contract and
   the brief this package's compile-then-seal boundary produces from it.
-- `index.json`: every file above, its kind, and the AD-27 digest of the exact bytes on disk.
+- `index.json`: every file above, its kind, and the digest of the exact bytes on disk.
 
 ## These contracts are visible and diagnostic
 
-AD-38 calls a development corpus visible and diagnostic. Nothing here is a holdout: every contract
-is published, readable, and meant to be read while writing your own. A holdout set that measures a
-contract's strength is a separate thing this package does not ship.
-
-## What is absent, and why
-
-**The qualified-probe dimensions are absent.** AD-38 asks for at least one qualified probe per
-probe class and per `expectedClean` state. The probe schema now carries both halves qualification
-needs: AD-9's per-route qualification record and AD-40's machine-readable defect signature, with a
-corpus gate that admits a probe only when the two agree with its class. The trial reducer and the
-score stage that reads it are both shipped, so an admitted probe can be scored end to end today;
-what is still missing is this directory's own gate widening to require at least one such probe. The
-dimension arrives with the change that adds that gate.
-
-**Three of the four artifacts in AD-38's end-to-end example are absent here.** The example there is
-a sealed brief, a conforming sealed run record, an isolation manifest, and an evaluator
-configuration. The last three are inputs the shipped `ingest` stage consumes, authored for the
-worked example in `scripts/worked-example-target.ts` and exercised there through
-`ingest`/`score`/`emit`. Only the run record among them is committed, as
-`spike-worked-example/sealed-run-record.json`; the isolation manifest and the evaluator
-configuration exist only as the authored values that build passes to `ingest`, never serialized to
-a file. This directory still ships only the compile-and-seal pair, scoped to what a corpus of
-contracts needs,
-under a name that does not claim AD-38's term.
+Nothing here is a holdout: every contract is published, readable, and meant to be read while writing
+your own. A holdout set that measures a contract's strength is a separate thing this package does not
+ship.
+
+## What is absent
+
+**The qualified-probe dimensions are absent.** A qualified probe is one whose qualification record and
+defect signature agree with its class, and a probe corpus would carry at least one per probe class and
+per `expectedClean` state. The probe schema carries both halves, and the trial reducer and the
+score stage that reads it are both shipped, so an admitted probe can be scored end to end today. This
+directory's own gate does not yet require one.
+
+**Three of the four artifacts in an end-to-end example are absent here.** Such an example is a sealed
+brief, a conforming sealed run record, an isolation manifest, and an evaluator configuration. The last
+three are inputs the shipped `ingest` stage consumes, authored for the worked example in
+`scripts/worked-example-target.ts` and exercised there through `ingest`/`score`/`emit`. Only
+the run record among them is committed, as `spike-worked-example/sealed-run-record.json`; the
+isolation manifest and the evaluator configuration exist only as the authored values that build
+passes to `ingest`. This directory ships only the compile-and-seal pair, scoped to what a corpus of
+contracts needs.

package/corpus/dev/index.json

@@ -1 +1 @@
-{"entries":[{"digest":"sha256:07cb7353bf4cec88e209d6d893f4618294268b8bead9dc9e804c166038cef213","kind":"readme","path":"corpus/dev/README.md"},{"digest":"sha256:2a5a248cea57e55ba6f79eaddc5e48e156917b4b91866fa349985c8d651a94cb","kind":"sealed-evaluator-brief","path":"corpus/dev/compile-seal-example/brief.json"},{"digest":"sha256:8f9dea3220baedd8a07da83ca4dd804c943cebda99c6754734694e48b8007e1c","kind":"contract","path":"corpus/dev/compile-seal-example/contract.json"},{"digest":"sha256:ec77b19ba1d3c8537ff74cc656f67c4ac46f0d0040e65422b3d742687a97e39c","kind":"contract","path":"corpus/dev/contracts/absent-collection-locations.json"},{"digest":"sha256:411c6a8c074ab6e524368605812532cc932fffc1ff20e68a65b50145fc294181","kind":"contract","path":"corpus/dev/contracts/absent-sibling-groups.json"},{"digest":"sha256:b6d22168553303903df5513da24edd01bf2f780a7f8dc3c0ed0d66a0bc6c7ba8","kind":"contract","path":"corpus/dev/contracts/absent-success-indicator.json"},{"digest":"sha256:eca39a841b40a67595e06343119f63be471af8110139b0245d6b18d18d0be4bb","kind":"contract","path":"corpus/dev/contracts/empty-channel-roles.json"},{"digest":"sha256:56aa60b674f70b4f96be8b982ffe384b7825caf886d1ae8107e2019beeee5d19","kind":"contract","path":"corpus/dev/contracts/empty-collection-locations.json"},{"digest":"sha256:c6ba5efeac1e3c5dfa096375225b98fd80a6fee4f28f4546df21f18baacce663","kind":"contract","path":"corpus/dev/contracts/empty-request-shapes.json","structuralFailure":"unreachable-check-evidence"},{"digest":"sha256:703d3e1468b6e058d728fd909d170f130d9539dc27822b16c539c5e9a3c908c1","kind":"contract","path":"corpus/dev/contracts/empty-sibling-groups.json"},{"digest":"sha256:f7983813b294cc114c35efd3efeb1db315d40ada56ae24f22b575046e6a76a14","kind":"contract","path":"corpus/dev/contracts/fragment-selection.json"},{"digest":"sha256:3e912f4423aa529fec21364155197334506316cdd7435ed1b59969eaab9cadf4","kind":"contract","path":"corpus/dev/contracts/no-collection-quantifier.json"},{"digest":"sha256:10e142a5ea8130c83c8dbe3be1c8df67c2997e8b57da0394cccbcfc43f44326f","kind":"contract","path":"corpus/dev/contracts/no-operation-inventory.json","structuralFailure":"unreachable-check-evidence"},{"digest":"sha256:8eace1550e26534eb886c9f217d4a8999891072643535b11afda44efc21d736f","kind":"contract","path":"corpus/dev/contracts/no-read-back-relation.json"},{"digest":"sha256:f66980583dde3213b65d5460491fe2dc34961cc9ee44600a40a2d10ba260efed","kind":"contract","path":"corpus/dev/contracts/no-state-change-marker.json","structuralFailure":"undeclared-mandatory-input"},{"digest":"sha256:deb36540d39aa99ae1d5d7e2e2808211ed40319fcc0d5519500e478659d8d232","kind":"contract","path":"corpus/dev/contracts/no-type-violating-step.json"},{"digest":"sha256:aa0ffb76c69870798b949e2e5a6a93b0bd46507fa7124505878b45a0672bc7e5","kind":"contract","path":"corpus/dev/contracts/per-key-split-oracles.json"},{"digest":"sha256:e4eaeb0e38357912dffb4bc7d29730797f9a1e485b4d98616fc934e83a57b732","kind":"contract","path":"corpus/dev/contracts/review-corpus.json"},{"digest":"sha256:8f9dea3220baedd8a07da83ca4dd804c943cebda99c6754734694e48b8007e1c","kind":"contract","path":"corpus/dev/contracts/satisfied-declarations.json"},{"digest":"sha256:951a90c269944c9e29c1b06ba4d634dbe6d2a1847308a951e7a5b6e4d271e2ff","kind":"contract","path":"corpus/dev/contracts/single-required-response-key.json"},{"digest":"sha256:fd696c819b7e2ada4dfb821fb4bbeeb0210a7826f0dd3c845e70476e7cab6a4e","kind":"contract","path":"corpus/dev/contracts/split-indicator-oracle.json"},{"digest":"sha256:70fa47876aa4318fb758203c77d2f234cdf26fe404d382dcc49e8f7dbf0c378e","kind":"contract","path":"corpus/dev/contracts/unaddressed-parameter-sibling.json"},{"digest":"sha256:9a64e4c32aad92a08da1f64a7d5616b4b96b60bd7070cdeaef7afbd9723e573c","kind":"contract","path":"corpus/dev/contracts/unnamed-reference-set.json"},{"digest":"sha256:fbb9e557e77311f8d608e003e019fe9c59fcf3e11fd737bf97cd0a3ff1e2e6b6","kind":"contract","path":"corpus/dev/contracts/wrong-cardinality-form.json"}]}
+{"entries":[{"digest":"sha256:419dfab17baf109b84e2f971ab85c399ff94712f562bf6e74d1f3e9093b68b18","kind":"readme","path":"corpus/dev/README.md"},{"digest":"sha256:2a5a248cea57e55ba6f79eaddc5e48e156917b4b91866fa349985c8d651a94cb","kind":"sealed-evaluator-brief","path":"corpus/dev/compile-seal-example/brief.json"},{"digest":"sha256:8f9dea3220baedd8a07da83ca4dd804c943cebda99c6754734694e48b8007e1c","kind":"contract","path":"corpus/dev/compile-seal-example/contract.json"},{"digest":"sha256:ec77b19ba1d3c8537ff74cc656f67c4ac46f0d0040e65422b3d742687a97e39c","kind":"contract","path":"corpus/dev/contracts/absent-collection-locations.json"},{"digest":"sha256:411c6a8c074ab6e524368605812532cc932fffc1ff20e68a65b50145fc294181","kind":"contract","path":"corpus/dev/contracts/absent-sibling-groups.json"},{"digest":"sha256:b6d22168553303903df5513da24edd01bf2f780a7f8dc3c0ed0d66a0bc6c7ba8","kind":"contract","path":"corpus/dev/contracts/absent-success-indicator.json"},{"digest":"sha256:eca39a841b40a67595e06343119f63be471af8110139b0245d6b18d18d0be4bb","kind":"contract","path":"corpus/dev/contracts/empty-channel-roles.json"},{"digest":"sha256:56aa60b674f70b4f96be8b982ffe384b7825caf886d1ae8107e2019beeee5d19","kind":"contract","path":"corpus/dev/contracts/empty-collection-locations.json"},{"digest":"sha256:c6ba5efeac1e3c5dfa096375225b98fd80a6fee4f28f4546df21f18baacce663","kind":"contract","path":"corpus/dev/contracts/empty-request-shapes.json","structuralFailure":"unreachable-check-evidence"},{"digest":"sha256:703d3e1468b6e058d728fd909d170f130d9539dc27822b16c539c5e9a3c908c1","kind":"contract","path":"corpus/dev/contracts/empty-sibling-groups.json"},{"digest":"sha256:f7983813b294cc114c35efd3efeb1db315d40ada56ae24f22b575046e6a76a14","kind":"contract","path":"corpus/dev/contracts/fragment-selection.json"},{"digest":"sha256:3e912f4423aa529fec21364155197334506316cdd7435ed1b59969eaab9cadf4","kind":"contract","path":"corpus/dev/contracts/no-collection-quantifier.json"},{"digest":"sha256:10e142a5ea8130c83c8dbe3be1c8df67c2997e8b57da0394cccbcfc43f44326f","kind":"contract","path":"corpus/dev/contracts/no-operation-inventory.json","structuralFailure":"unreachable-check-evidence"},{"digest":"sha256:8eace1550e26534eb886c9f217d4a8999891072643535b11afda44efc21d736f","kind":"contract","path":"corpus/dev/contracts/no-read-back-relation.json"},{"digest":"sha256:f66980583dde3213b65d5460491fe2dc34961cc9ee44600a40a2d10ba260efed","kind":"contract","path":"corpus/dev/contracts/no-state-change-marker.json","structuralFailure":"undeclared-mandatory-input"},{"digest":"sha256:deb36540d39aa99ae1d5d7e2e2808211ed40319fcc0d5519500e478659d8d232","kind":"contract","path":"corpus/dev/contracts/no-type-violating-step.json"},{"digest":"sha256:aa0ffb76c69870798b949e2e5a6a93b0bd46507fa7124505878b45a0672bc7e5","kind":"contract","path":"corpus/dev/contracts/per-key-split-oracles.json"},{"digest":"sha256:e4eaeb0e38357912dffb4bc7d29730797f9a1e485b4d98616fc934e83a57b732","kind":"contract","path":"corpus/dev/contracts/review-corpus.json"},{"digest":"sha256:8f9dea3220baedd8a07da83ca4dd804c943cebda99c6754734694e48b8007e1c","kind":"contract","path":"corpus/dev/contracts/satisfied-declarations.json"},{"digest":"sha256:951a90c269944c9e29c1b06ba4d634dbe6d2a1847308a951e7a5b6e4d271e2ff","kind":"contract","path":"corpus/dev/contracts/single-required-response-key.json"},{"digest":"sha256:fd696c819b7e2ada4dfb821fb4bbeeb0210a7826f0dd3c845e70476e7cab6a4e","kind":"contract","path":"corpus/dev/contracts/split-indicator-oracle.json"},{"digest":"sha256:70fa47876aa4318fb758203c77d2f234cdf26fe404d382dcc49e8f7dbf0c378e","kind":"contract","path":"corpus/dev/contracts/unaddressed-parameter-sibling.json"},{"digest":"sha256:9a64e4c32aad92a08da1f64a7d5616b4b96b60bd7070cdeaef7afbd9723e573c","kind":"contract","path":"corpus/dev/contracts/unnamed-reference-set.json"},{"digest":"sha256:fbb9e557e77311f8d608e003e019fe9c59fcf3e11fd737bf97cd0a3ff1e2e6b6","kind":"contract","path":"corpus/dev/contracts/wrong-cardinality-form.json"}]}

package/dist/adapters/command-line-adapter.d.ts

@@ -30,7 +30,19 @@ export type CommandMechanism = {
     readonly run: (request: CommandRunRequest, signal: AbortSignal) => Promise<CommandRunResult>;
     readonly readArtifact: (path: string, maxBytes: number) => Promise<ArtifactRead>;
 };
-/** Options first as `--{key}`, positionals after, both in the record's own key order. Exported for its own unit tests: this is the one place shell-injection safety is decided. */
+/**
+ * Options first as `--{key}`, positionals after, both in the record's own key
+ * order. Exported for its own unit tests: this is the one place
+ * shell-injection safety is decided.
+ *
+ * An array value is the repeatable spelling. A great many command-line tools
+ * accept an option more than once and collect the values (`--env-pass A
+ * --env-pass B`), and a channel record holds one value per key, so before this
+ * a caller could send exactly one. The single JSON token an array used to
+ * produce (`--env-pass ["A","B"]`) reaches no parser that understands it, so
+ * nothing can depend on the old spelling. An empty array emits nothing, the
+ * same as `false`: there is no value to pass.
+ */
 export declare function buildArgv(channels: CommandProbeRequest['channels']): string[];
 /**
  * The real mechanism: an actual child process, an actual file read. Exported,

package/dist/adapters/command-line-adapter.js

@@ -16,11 +16,13 @@
  * 2. `argument` and `option` build the argv the same way a well-behaved CLI
  *    parser reads one: options first as `--{key}` (a boolean `true` is a
  *    bare flag, `false` is omitted, anything else gets one value token),
- *    positionals after in the record's own key order. `environment` passes
- *    through as declared, plus the host's own `PATH` so a `target` naming a
- *    bare command still resolves; a declared `PATH` key wins over that
- *    default. `stdin` is written and the stream is closed; `absent` closes
- *    it with nothing written.
+ *    positionals after in the record's own key order. An array value is the
+ *    repeatable spelling: `--{key}` is emitted once per element, and an
+ *    array positional contributes one token per element. `environment`
+ *    passes through as declared, plus the host's own `PATH` so a `target`
+ *    naming a bare command still resolves; a declared `PATH` key wins over
+ *    that default. `stdin` is written and the stream is closed; `absent`
+ *    closes it with nothing written.
  * 3. `maxElapsedMs` and `maxOutputBytes` are enforced by this adapter, not
  *    borrowed from `AbortSignal`: exceeding either kills the process with
  *    `SIGKILL` and throws `budget-exhausted`, exactly as an HTTP cap does.
@@ -47,17 +49,37 @@ function stringifyScalar(value) {
         return String(value);
     return JSON.stringify(value);
 }
-/** Options first as `--{key}`, positionals after, both in the record's own key order. Exported for its own unit tests: this is the one place shell-injection safety is decided. */
+/**
+ * Options first as `--{key}`, positionals after, both in the record's own key
+ * order. Exported for its own unit tests: this is the one place
+ * shell-injection safety is decided.
+ *
+ * An array value is the repeatable spelling. A great many command-line tools
+ * accept an option more than once and collect the values (`--env-pass A
+ * --env-pass B`), and a channel record holds one value per key, so before this
+ * a caller could send exactly one. The single JSON token an array used to
+ * produce (`--env-pass ["A","B"]`) reaches no parser that understands it, so
+ * nothing can depend on the old spelling. An empty array emits nothing, the
+ * same as `false`: there is no value to pass.
+ */
 export function buildArgv(channels) {
     const optionTokens = [];
     for (const [key, value] of Object.entries(channels.option)) {
         if (value === false)
             continue;
+        if (Array.isArray(value)) {
+            for (const element of value) {
+                optionTokens.push(`--${key}`, stringifyScalar(element));
+            }
+            continue;
+        }
         optionTokens.push(`--${key}`);
         if (value !== true)
             optionTokens.push(stringifyScalar(value));
     }
-    const argumentTokens = Object.values(channels.argument).map(stringifyScalar);
+    const argumentTokens = Object.values(channels.argument).flatMap((value) => Array.isArray(value)
+        ? value.map(stringifyScalar)
+        : [stringifyScalar(value)]);
     return [...optionTokens, ...argumentTokens];
 }
 function buildEnv(declared) {

package/dist/cli/render.d.ts

@@ -18,6 +18,6 @@ export declare function renderError(error: unknown): string;
  * AD-21's seven exit codes, one line each. The `--help` output and the README
  * table are this text, so the two cannot drift.
  */
-export declare const EXIT_CODE_TABLE = "Exit codes (AD-21):\n  0   success, and every verdict other than FAIL or a promoted CONCERNS\n  1   CONCERNS promoted by --strict\n  2   FAIL\n  3   invalid: a failed pre-flight, or any other AD-21 invalidating condition\n  4   structural failure\n  5   runtime fault\n  64  usage error\n\n  --strict never promotes a CONCERNS whose firing conditions are all evidence\n  conditions: those conditions report that the measurement fell short of the\n  policy. 1 and 2 report a verdict the score command's ladder resolved; every\n  other invalidating condition behind 3 is reachable there too, alongside the\n  failed pre-flight the preflight command itself reports.";
+export declare const EXIT_CODE_TABLE = "Exit codes (AD-21):\n  0   success, and every verdict other than FAIL or a promoted CONCERNS\n  1   CONCERNS promoted by --strict\n  2   FAIL\n  3   invalid: a failed pre-flight, or any other AD-21 invalidating condition\n  4   structural failure\n  5   runtime fault\n  64  usage error\n\n  --strict never promotes a CONCERNS whose firing conditions are all evidence\n  conditions: those report that the measurement fell short of the policy.\n  1 and 2 come from the score command's verdict ladder. 3 comes from a failed\n  pre-flight, which the preflight command reports, or from any other\n  invalidating condition score finds.";
 /** `eval-quality: usage: <message>` */
 export declare function renderUsage(message: string): string;

package/dist/cli/render.js

@@ -95,10 +95,10 @@ export const EXIT_CODE_TABLE = `Exit codes (AD-21):
   64  usage error
 
   --strict never promotes a CONCERNS whose firing conditions are all evidence
-  conditions: those conditions report that the measurement fell short of the
-  policy. 1 and 2 report a verdict the score command's ladder resolved; every
-  other invalidating condition behind 3 is reachable there too, alongside the
-  failed pre-flight the preflight command itself reports.`;
+  conditions: those report that the measurement fell short of the policy.
+  1 and 2 come from the score command's verdict ladder. 3 comes from a failed
+  pre-flight, which the preflight command reports, or from any other
+  invalidating condition score finds.`;
 /** `eval-quality: usage: <message>` */
 export function renderUsage(message) {
     return `${PREFIX}: usage: ${message}`;

package/dist/index.d.ts

@@ -12,4 +12,4 @@ export type { ScoringPolicy } from './core/schemas/scoring-policy.ts';
 export type { SealedEvaluatorBrief } from './core/schemas/sealed-evaluator-brief.ts';
 export type { SealedRunRecord } from './core/schemas/sealed-run-record.ts';
 export type { FixtureReset, ManifestationWitness, SensitivityWitness, SensitivityWitnessLeg, WitnessChannel, WitnessInputs, } from './core/schemas/sensitivity-witness.ts';
-export declare const VERSION = "1.0.0";
+export declare const VERSION = "1.2.0";

package/dist/index.js

@@ -19,4 +19,4 @@
 // subpath, where AD-37 puts the conformance definition an adapter author
 // reads; the reference adapters stay at `eval-quality/adapters`.
 export * from './application/index.js';
-export const VERSION = '1.0.0';
+export const VERSION = '1.2.0';

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "eval-quality",
-  "version": "1.0.0",
+  "version": "1.2.0",
   "description": "Compile disciplined Behavioral Evaluation Contracts and score their ability to catch known defects.",
   "author": "Murat Ozcan",
   "license": "Apache-2.0",