eval-quality 0.1.0 → 0.3.0

package/README.md

@@ -1,18 +1,28 @@
 # `eval-quality`
 
-### `eval-quality` does three things
+**[Documentation](https://bmad-code-org.github.io/bmad-eval-quality/)** ·
+[Getting started](https://bmad-code-org.github.io/bmad-eval-quality/tutorials/getting-started/) ·
+[CLI reference](https://bmad-code-org.github.io/bmad-eval-quality/reference/cli-commands/) ·
+[npm](https://www.npmjs.com/package/eval-quality)
+
+```bash
+npx eval-quality --help
+```
+
+### `eval-quality` does four things
 
 1. **Compile**: validate and normalize an eval contract into a machine-readable artifact.
 2. **Seal**: render the brief for the independent evaluator while hiding the planted bug and scoring answer.
 3. **Preflight**: verify baseline environment readiness and probe reachability before running an evaluator.
+4. **Score**: compare the evaluator's completed findings against the hidden bug signature and mint a versioned evidence artifact.
 
-Scoring is the next milestone: comparing the evaluator’s completed findings with the hidden bug signature to determine whether the bug was actually caught.
+It executes nothing. No agent, no judge, and no system under test runs inside it; your harness runs the evaluation and hands over a sealed run record.
 
-### What is the eval spec?
+### What is the evaluation contract?
 
 It is the test.
 
-More precisely, it is the evaluator’s instructions for how to expose a failure and what evidence counts as finding it.
+More precisely, it is the evaluator’s instructions for how to expose a failure and what evidence counts as finding it. The long name is Behavioral Evaluation Contract; the docs shorten it to eval contract or evaluation contract.
 
 It defines:
 
@@ -42,25 +52,35 @@ A strong eval checks the response **and** persistence, so it catches the bug.
 
 Write the eval. Hide the bug. See if the eval catches it.
 
-## Key Concepts
+## The core flow, in eight nouns
+
+Every run of an evaluation walks the same order:
 
-Understanding `eval-quality` requires three core artifacts:
+```text
+evaluation contract → probe → observation → preflight → evidence → oracle → rubric → score / verdict
+```
 
-| Concept | What it is | Example |
+| Noun | What it is | Example |
 | --- | --- | --- |
-| **Contract** (`eval-contract.json`) | The test specification defining expected behaviors, oracles (checks), permitted tools, and evidence rules. | "Verify API rejects invalid JWT and creates zero database records." |
-| **Probe** (`probe.json`) | A diagnostic request sent to the environment to test baseline state, reachability, or fault injection. | A request sending an expired token to `/api/v1/resource`. |
-| **Observation** (`observation.json`) | The empirical response evidence recorded when a probe is executed against the environment. | `{ responseStatus: 401, responseBody: { error: "token_expired" } }` |
+| **Evaluation contract** | What we want to measure: the behaviors, the checks, the interfaces a probe may touch, and the bounds a run stays inside. | "A PATCH that reports success has persisted the change." |
+| **Probe** | How to poke the system to produce evidence: a test case, a call, a step. In scoring, a probe also names the defect it seeded. | "Update note n-1, then read it back." |
+| **Observation** | What actually happened when the system was poked: the recorded status, headers, and body of one call. | `PATCH` returned 200 with the new title; the later `GET` returned the old one. |
+| **Preflight** | Whether the environment and the observations are fit for meaningful measurement. | Both operations reachable, the fixture reset, the clean control clean. |
+| **Evidence** | The recorded output, trajectory, and artifacts from the evaluation run: what the evaluator saw and what it claimed. | A finding citing the two observations above. |
+| **Oracle** | The assertion: the relation that has to hold over the evidence. | The title sent equals the title read back. |
+| **Rubric** | The grading guide for judgment-heavy quality, with anchored criteria a judge scores against. The judge runs outside the package and its scores arrive in the sealed run record. | Present only when a contract declares one. |
+| **Score / verdict** | The combined result: did the evaluation catch the planted defect? `PASS`, `WAIVED`, `CONCERNS`, or `FAIL`, or Invalid when the run produced no verdict. | `FAIL`, exit code 2. |
 
-### How They Fit Together
+The word evidence is used twice on purpose. The evidence in the flow is what the evaluator produced, and it reaches `score` inside a sealed run record. The evidence artifact is what `score` mints at the end: the outcomes, the verdict, the strength vector, and the exit code.
 
-```text
-┌────────────────────────┐      ┌────────────────────────┐      ┌────────────────────────┐
-│     Eval Contract      │      │         Probe          │      │      Observation       │
-│  (The Specification)   │ ───► │ (Diagnostic Request)   │ ───► │   (Empirical Result)   │
-│  "What should happen"  │      │  "Send malformed JWT"  │      │   "Got 401, 0 records" │
-└────────────────────────┘      └────────────────────────┘      └────────────────────────┘
-```
+The four commands sit on that flow like this:
+
+| Command | Reads | Writes |
+| --- | --- | --- |
+| `compile` | an authored contract | `eval-contract.json`, checked against the schema and the discipline rules |
+| `seal` | a contract | `sealed-evaluator-brief.json`, the contract minus everything that would give the answer away |
+| `preflight` | a contract, a probe list, observations | `preflight-verdict.json`, fit or unfit to measure |
+| `score` | a sealed run record, the contract, a probe, the preflight verdict, a scoring policy, a caller-attested corpus digest, and the isolation manifest and evaluator configuration the record was produced under | `evidence-artifact.json`, and the verdict's own exit code |
 
 ## Elaboration
 
@@ -85,13 +105,7 @@ confirm the specific error, and confirm no record was created.
 
 A passing eval says little when the contract never asked for the probe that would expose the failure. Testing whether the eval can catch a failure you already know about is the first check worth running.
 
-```text
-product spec
-  → Behavioral Evaluation Contract
-  → known defect or gameability probe
-  → independent evaluator
-  → per-oracle evidence and a gate decision
-```
+The loop that does that is a twin run. Keep the contract, the probes, the oracles, and the scoring policy fixed. Run the evaluator once against the clean system and once against the same system carrying one known defect. Score both runs. The contract is strong when the clean run passes and the mutated run degrades, and it has a blind spot when both stay green.
 
 ## What each part provides
 
@@ -101,8 +115,12 @@ product spec
 - the oracle vocabulary and authoring rules
 - the contract compiler
 - the environment pre-flight
-- Eval Contract strength scoring (next milestone)
-- versioned evidence output and PASS / WAIVED / CONCERNS / FAIL governance (next milestone)
+- Eval Contract strength scoring: the AD-7 rate vector and dominance relation, implemented in
+  `src/core/score/strength.ts` and reached by the `score` command
+- PASS / WAIVED / CONCERNS / FAIL governance: both verdict ladders, implemented and total in
+  `src/core/score/ladder.ts`, and the `score` command's own exit code
+- versioned evidence output: `evidence-artifact.json`, implemented in `src/core/emit/emit.ts` and
+  minted by the `score` command
 
 The caller provides:
 
@@ -113,12 +131,15 @@ The caller provides:
 - a sealed run record returned for ingestion
 
 `eval-quality` executes nothing: it never spawns a process, calls a model, drives a system under test,
-or invokes a judge. Its pure stages are compile, seal, ingest, pre-flight, score, and emit; compile,
-seal, and pre-flight ship, and ingest, score, and emit are the next milestone. Pre-flight probes the
-fixture through the environment-probe port, so a contract that declares a fixture reset
+or invokes a judge. Its six stages are compile, seal, ingest, pre-flight, score, and emit, all pure,
+and every one is reachable through the CLI and the library alike. That list is the declared stage
+order; on the clock, ingest follows the evaluator run, so it sits after pre-flight and just before
+score. Compile, seal, and pre-flight each
+have their own command and their own exported function. `ingest`, `score`, and `emit` are reached
+through the one `score` command and the one exported `runScore` call that chains them, per AD-14's
+rule that a command exposes no more than the library itself calls. Pre-flight probes the fixture through the environment-probe port, so a contract that declares a fixture reset
 needs the caller's probe policy to authorize that operation's method as well as the read methods
-every other pre-flight leg uses. Engine integration is a later adapter behind a port, not a v0
-dependency. See
+every other pre-flight leg uses. Engine integration is a later adapter behind a port. See
 [ADR-004](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-29/ADR-004-execution-boundary.md).
 
 ## Who it is for
@@ -145,27 +166,32 @@ Rubrics compile under the same discipline: an anchored scale, a bounded length,
 
 ## How Eval Contract strength scoring works
 
-Do not trust a contract because it looks thorough. Put a known defect behind it, run the evaluator, and check whether the contract's oracles caused the defect to be caught.
+The `score` command and its `runScore` library call compute it; `npm run generate:worked-example`
+runs the same functions over the committed worked chain, and
+[the full walkthrough](https://bmad-code-org.github.io/bmad-eval-quality/how-to/author-behavioral-contracts/)
+reads the result field by field. Do not trust a contract because it looks thorough. Put a known defect behind it, run the evaluator, and check whether the contract's oracles caused the defect to be caught.
 
 Two probe classes go behind a contract, and a strong contract rejects both:
 
 - **Defect probes**, where the behavior is simply wrong.
 - **Gameability probes**, where the behavior looks compliant while dodging the oracle's intent. A test that raises coverage while asserting nothing is the familiar version of this.
 
-Probes come from qualified historical defects or verified controlled mutations. The corpus separates a visible development set from an immutable sealed set for each scoring version.
+Probes come from qualified historical defects or verified controlled mutations. The corpus separates a visible development set from an immutable sealed set for each scoring version. Only the development set exists today; the sealed set is part of the design and ships in no release yet.
 
 Every required oracle check resolves to exactly one state, and the state travels with the result, so
 "the check reported" is never sufficient on its own: `caught`, `confirmed`, `missed`,
 `passed-clean-control`, `false-positive`, `abstained`, `bypassed`, `unreached`, `oracle-error`,
 `judge-error`, `infrastructure-error`, or `not-applicable`.
 
-A required oracle that missed, abstained, errored, or is absent prevents PASS, and a high overall score never overrides it. An infrastructure error or a failed environment pre-flight is not a behavioral result at all; it invalidates the run and is re-executed rather than scored.
+A required oracle that missed, abstained, errored, or is absent prevents PASS, and a high overall score never overrides it. An infrastructure error or a failed environment pre-flight is not a behavioral result at all; it invalidates the run, and the run is re-executed.
+
+Repeated runs of one probe are trials, and they reduce to one result per probe before any rate is computed. The `score` stage takes a trial set; the `score` command and `runScore` hand it one sealed run record per call, so a run scored from the published surface completes one trial, and against a policy declaring a minimum of three, as the worked example's does, its strength vector is reported and marked non-comparable.
 
 ## Using it
 
-`eval-quality` is its own repository and package, not a plugin inside another framework.
+`eval-quality` is its own repository and package, with no framework around it.
 
-The **library** is the primary surface. It exports the contract schema, the oracle vocabulary, the compiler, the pre-flight, and the evidence types. The published typed schema is what lets coding agents author contracts correctly by default, which is how the discipline scales beyond the people who went looking for the tool.
+The **library** is the primary surface. It exports the artifact types, the compiler, the pre-flight, `runScore`, the canonical digest, the lineage validator, and the failure-code and verdict registries. The Zod schemas themselves are not exported; they are published as JSON Schema under `eval-quality/schemas/*`. The published typed schema is what lets coding agents author contracts correctly by default, which is how the discipline scales beyond the people who went looking for the tool.
 
 The **CLI** wraps the same library for callers that cannot import TypeScript: CI jobs, GitHub Actions, PR-review and unit-test bots, other frameworks' skills, and any agent permitted to run a shell command.
 
@@ -173,7 +199,8 @@ The **CLI** wraps the same library for callers that cannot import TypeScript: CI
 
 - **`compile`**: Typechecks an authored `eval-contract.json`. Verifies that all behaviors, oracles, rubrics, and sensitivity witnesses comply with structural and authoring rules.
 - **`seal`**: Generates a `sealed-evaluator-brief.json` by stripping secret defect signatures, planted answers, and author commentary. The brief carries only the directions and safety bounds the evaluator needs.
-- **`preflight`**: Reduces caller-supplied probe observations against the contract to verify environment baseline readiness and probe reachability. Halts early with exit code `3` if the environment is unready.
+- **`preflight`**: Reduces caller-supplied probe observations against the contract to verify environment baseline readiness and probe reachability. All four of `--contract`, `--probes`, `--observations`, and `--run-id` are required. Halts early with exit code `3` if the environment is unready. `schemas/probe.schema.json` gives the shape of one probe in the list; each observation echoes a planned leg's id back as `probeId`, and the getting-started tutorial writes six by hand.
+- **`score`**: Chains ingest, score, and emit over one sealed run record, minting `evidence-artifact.json` and exiting with the AD-21 verdict's own exit code. `schemas/sealed-run-record.schema.json` gives the record's shape, and the committed worked chain under `_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-29/spike-worked-example/` carries one, scored. `--record`, `--contract`, `--probe`, `--preflight-verdict`, `--policy`, and `--corpus-digest` are required; `--isolation-manifest` and `--evaluator-configuration` are each optional and their absence invalidates the run and the command still parses; `--private-manifest` is optional and, when given, each entry's declared digest is checked against its resolved bytes. `--corpus-root` names the directory a private reference resolves under, and is required only when `--private-manifest` or a private-storage isolation-manifest reference is actually present.
 
 ### Running the CLI
 
@@ -187,17 +214,25 @@ npx eval-quality seal --in contract.json --out ./eval-out
 npx eval-quality preflight --contract contract.json \
   --probes probes.json --observations observations.json \
   --run-id 2026-08-28-a --out ./eval-out
+
+npx eval-quality score --record record.json --contract contract.json \
+  --probe probe.json --preflight-verdict preflight-verdict.json \
+  --policy policy.json --corpus-digest <digest> \
+  --out ./eval-out
 ```
 
 Every command is non-interactive: no prompt, no terminal check, and no behaviour that differs when
 stdin is a pipe. Each one is a single call into the library plus artifact serialization.
 
-**Input and output.** An input flag left out reads stdin, and `-` names stdin explicitly; at most one
-input may be `-`. Without `--out` the artifact goes to stdout, so a command composes with a pipe.
+**Input and output.** `--in` is the only input that falls back to stdin: `compile` and `seal` read it
+when `--in` is left out. `-` names stdin explicitly on any input, and at most one input may be `-` per
+invocation. `compile` and `seal` each take one input; `preflight` takes three, all required; `score`
+takes eight, three of them optional (`--isolation-manifest`, `--evaluator-configuration`, and
+`--private-manifest`). Without `--out` the artifact goes to stdout, so a command composes with a pipe.
 An `--out` ending in `.json` is a file path; anything else is a directory, and the artifact is
-written to `<target>/<kind>.json` where `kind` is `eval-contract`, `sealed-evaluator-brief`, or
-`preflight-verdict`. Diagnostics and errors go to stderr, always, so stdout carries the artifact
-alone.
+written to `<target>/<kind>.json` where `kind` is `eval-contract`, `sealed-evaluator-brief`,
+`preflight-verdict`, or `evidence-artifact`. Diagnostics and errors go to stderr, always, so stdout
+carries the artifact alone.
 
 **Exit codes.**
 
@@ -206,18 +241,21 @@ alone.
 | `0` | success, and every verdict other than FAIL or a promoted CONCERNS |
 | `1` | CONCERNS promoted by `--strict` |
 | `2` | FAIL |
-| `3` | invalid: a pre-flight verdict that did not pass |
+| `3` | invalid: a failed pre-flight, or any other AD-21 invalidating condition |
 | `4` | structural failure |
 | `5` | runtime fault |
 | `64` | usage error |
 
-Codes 1 and 2 report a scored verdict. Scoring ships in a later release, so no command here reaches
-either yet, and `--strict` changes no code this binary produces. The flag and the two codes are part
-of the published contract, so they are documented now and wired now.
+`--strict` never promotes a CONCERNS whose firing conditions are all evidence conditions: those
+conditions report that the measurement fell short of the policy. Codes 1 and 2 report a verdict
+`score`'s ladder resolved, read directly off `LadderResolution.exitCode`; every other invalidating
+condition behind code 3 is reachable through `score` too, alongside the failed pre-flight `preflight`
+itself reports.
 
 `--strict` is the gate-promotion flag and is accepted on every command. `--strict-inputs` and
 `--no-strict-inputs` are a different switch: they set the compiler's input strictness, which is on
-by default.
+by default, and `preflight` and `score` each reject both with exit `64` because neither has a compile
+step.
 
 **The published JSON Schema.** A consumer that does not read TypeScript validates against the
 twelve generated documents, published at the `eval-quality/schemas/*` subpath:
@@ -229,6 +267,17 @@ import spec from 'eval-quality/schemas/eval-contract.schema.json' with { type: '
 The import attribute is required: ESM on Node 22 and 24 both throw `ERR_IMPORT_ATTRIBUTE_MISSING`
 without it. The development corpus ships the same way, at `eval-quality/corpus/dev/`, so an adopter
 can read real compiled contracts and one compiled-and-sealed pair without cloning this repository.
+`eval-quality/adapters` is one of the five published subpaths, holding the three reference adapters
+the conformance suite runs against.
+
+Eleven of the twelve published schemas carry a `schemaVersion`. `artifact-reference` is exempt: it
+is embedded inside other artifacts, so it has no version to break.
+
+`schemaVersion` is declared as any integer at or above 1, so a document at an unexpected version
+parses. The bumps in the next release each add a required field, which is why an older document
+fails; the version itself is compared in exactly one place, `validateLineageChain`, over
+lineage-chain members, and nowhere on the command path. The package is pre-1.0, so pin exactly.
+`CHANGELOG.md` records what each release breaks.
 
 ## Relationship with BMad and TEA
 
@@ -241,7 +290,7 @@ graph LR
 
 TEA is the reference authoring client. It reads BMad planning artifacts, notices eval-relevant work, drafts a contract, and calls this package. It is not co-installed, and `eval-quality` holds no knowledge of TEA, BMad, or any planning-artifact format.
 
-Any human, bot, CI job, skill, or other framework can author a contract and use `eval-quality` directly. The discipline still applies, because the compiler judges the artifact rather than trusting whoever produced it.
+Any human, bot, CI job, skill, or other framework can author a contract and use `eval-quality` directly. The discipline still applies, because the compiler judges the artifact, whoever produced it.
 
 Evaluator runs remain isolated to prevent builder-context leakage and preserve traceability. Stronger contract oracles produced the measured detection improvement.
 
@@ -249,7 +298,9 @@ Evaluator runs remain isolated to prevent builder-context leakage and preserve t
 1. Author an `eval-contract.json` declaring required knowledge step files (e.g. `playwright-utils-mandate.md`).
 2. Run `eval-quality compile --in contract.json` to validate contract structure and discipline rules.
 3. Run `eval-quality seal --in contract.json --out ./run` to generate `sealed-evaluator-brief.json`.
-4. Pass `sealed-evaluator-brief.json` to `bmad-tea` to execute the task without seeing answer keys.
+4. Probe the harness's environment and run `eval-quality preflight` over the observations; a verdict that does not pass is exit `3`, and the run stops there.
+5. Pass `sealed-evaluator-brief.json` to `bmad-tea` to execute the task without seeing answer keys, once against the clean harness and once against a harness with one known step file removed.
+6. Seal each evaluator run into a `sealed-run-record.json` and run `eval-quality score` over it with the probe that names the removed file as the seeded defect. The clean run should pass; the mutated run should degrade, and the exit code says which.
 
 ## Evidence and limitations
 
@@ -261,13 +312,13 @@ Read the [product brief](_bmad-output/planning-artifacts/briefs/brief-eval-quali
 
 ## Architecture status
 
-The [architecture spine](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-29/ARCHITECTURE-SPINE.md) is split by pipeline half: the compile-and-seal half is epic-ready, while the score half is not. Gate C closed at zero blocking authoring points and 14 of 14 declaration-only predicates. Gate D's generated-current-fields arm matched the hand-written positive control at 3 of 3 seeded-defect catches, so `seal` joins the stage-one order without adding an evidence-precondition field.
+The [architecture spine](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-29/ARCHITECTURE-SPINE.md) is split by pipeline half, and its own status line still reads that the compile-and-seal half is epic-ready while the score half is not. The code has moved past that line. Epic 7 delivered AD-21, AD-33, and AD-40 as pure functions with generated tables, and epic 8 shipped the `ingest`, `score`, and `emit` stages, the `score` command, and `runScore` over them, which closes every item the spine's *Owed to the reference implementation* section listed. The `score` stage consumes a trial set; the command and `runScore` hand it one record per call, so a run scored from the published surface completes one trial, and whenever the policy's declared minimum exceeds one its strength vector is reported and marked non-comparable. Gate C closed at zero blocking authoring points and 14 of 14 declaration-only predicates. Gate D's generated-current-fields arm matched the hand-written positive control at 3 of 3 seeded-defect catches, so `seal` joins the stage-one order without adding an evidence-precondition field.
 
-Contract strength scoring has been open since [ADR-007](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-29/ADR-007-compile-score-split.md): three rounds of external review established that the catch rate was 1.00 by construction, because nothing matched a finding to the defect its probe seeded. That input now exists and the mapping that reads it is owed to a reference implementation.
+Contract strength scoring has been open since [ADR-007](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-29/ADR-007-compile-score-split.md): three rounds of external review established that the catch rate was 1.00 by construction, because nothing matched a finding to the defect its probe seeded. That input now exists, and so does the mapping that reads it: `src/core/score/witness.ts` is AD-40's witness match, delivered by epic 7. What is still owed is its validation against the block-2 replication, which the spine records as committed and not yet run.
 
 Contract compilation was declared ready in ADR-007 and a fourth review withdrew that claim in [ADR-008](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-29/ADR-008-compile-half-owed-to-calibration.md). The named calibration is now complete. The absent local-only mut2 arm was reconstructed from its recorded base, reproduced its prior black-box behavior, and ran under a pre-registered three-arm, three-repetition design. All three arms composed filters and detected the seeded defect in every valid repetition. This closes the calibration gate narrowly; it does not generalize the historical 0.33-to-1.00 effect beyond one behavior and one controlled mutation.
 
-Both are documented as defects rather than dressed as decisions, because four rounds have shown that a confidently worded revision is the thing that goes wrong here.
+Both are documented as defects, because four rounds have shown that a confidently worded revision is the thing that goes wrong here.
 
 The decision record, in order: [ADR-001](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-22/ADR-001-evaluator-isolation-boundary.md) on evaluator isolation, [ADR-002](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-22/ADR-002-contract-authoring-discipline.md) on why authoring discipline is the product, [ADR-003](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-29/ADR-003-measurement-mechanics.md) on measurement mechanics, [ADR-004](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-29/ADR-004-execution-boundary.md) on why this package executes nothing, [ADR-005](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-29/ADR-005-review-round-corrections.md) and [ADR-006](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-29/ADR-006-interaction-plan.md) on what review and hand-authoring corrected, [ADR-007](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-29/ADR-007-compile-score-split.md) on the split, [ADR-008](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-29/ADR-008-compile-half-owed-to-calibration.md) on why the other half stopped claiming to be finished too, and [ADR-009](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-29/ADR-009-adversarial-gate-corrections.md) on the seventeen places where two conforming implementations still disagreed. Review triage lives in [`reviews/`](_bmad-output/planning-artifacts/architecture/architecture-eval-quality-2026-07-29/reviews/).
 
@@ -279,28 +330,38 @@ Out of scope entirely: a new eval engine, a hosted service, a dashboard or GUI,
 
 ## Development
 
+Node `>=22.20.0`, which `package.json` declares as the engine floor. `zod` is the only production
+dependency.
+
 ```bash
 npm install
-npm run validate            # typecheck, lint, docs, shareable, spine, vectors, schemas, registries, AD-31 table, layers, lineage, boundary, corpus, tests with coverage
+npm run validate            # build, typecheck, lint, docs, doc invocations, shareable, spine, vectors, schemas, both code registries, the AD-21, AD-31 and AD-33 tables, layers, lineage, boundary, corpus, worked chain, website deps, tests with coverage
 npm run build               # emit to dist/
 npm run lint:fix            # auto-fix with Biome
 npm run test:coverage       # run the suite and fail below AD-30's 90 percent statement and branch floor on core/
 npm run generate:schemas    # rebuild schemas/*.schema.json from the Zod source
 npm run check:schemas       # fail if the committed schemas differ from the source by one byte
-npm run check:ad5-registry  # fail if the failure-code list drifts from the AD-5 table
+npm run check:ad5-registry  # fail if the compile-time failure-code list drifts from the AD-5 table
+npm run check:ad28-registry # fail if the runtime fault-code list drifts from the AD-28 table
 npm run check:lineage       # fail if a module outside the stage table writes an artifact's lineage fields
 npm run check:boundary      # fail if anything the tarball carries references the planning system that produced it
+npm run generate:ad21-table # rebuild docs/ad21-verdict-decision.generated.md from the two verdict ladders
+npm run check:ad21-table    # fail if the committed AD-21 table differs from the builder by one byte
 npm run generate:ad31-table # rebuild docs/ad31-coverage-predicates.generated.md from the predicates
 npm run check:ad31-table    # fail if the committed AD-31 table differs from the builder by one byte
+npm run generate:ad33-table # rebuild docs/ad33-outcome-decision.generated.md from the decision procedure
+npm run check:ad33-table    # fail if the committed AD-33 table differs from the builder by one byte
 npm run generate:dev-corpus # rebuild corpus/dev/ from the contract fixtures through the shipped compile and seal
 npm run check:corpus        # fail if the committed corpus differs from the builder by one byte
+npm run generate:worked-example # rebuild the spike worked chain by running the compile, seal, score, and emit functions over it
+npm run check:worked-example    # fail if the committed worked chain differs from the builder by one byte
 npm run build:shareable     # render the planning artifacts to self-contained HTML
 npm run test:conformance    # run the published port conformance suite against every shipped adapter
 ```
 
 `schemas/` holds the twelve published JSON Schema documents, generated from the Zod definitions and
 committed. They are the contract for consumers who do not read TypeScript, so they are proven
-equivalent to the source rather than assumed to be: a byte-exact drift check, a rejection suite
+equivalent to the source: a byte-exact drift check, a rejection suite
 asserting the validator keyword and instance path for every negative fixture, a differential check
 comparing Zod's verdict against a third-party validator's over a generated corpus, and a
 keyword-mutation sweep that deletes each published constraint and requires some fixture to notice.
@@ -311,15 +372,16 @@ is ES modules, which are always strict, so an attempt throws a `TypeError` there
 caller sees the write fail silently. A revision is minted as a new artifact carrying its parent's
 digest and a revision count one greater. `check:lineage` fails the build when a lineage field is
 written outside `src/core/schemas/`, `src/core/lineage/`, and the modules the AD-24 stage table
-names as that artifact's producer, which today are `src/core/seal/seal.ts` and
-`src/core/preflight/reduce.ts`.
+names as that artifact's producer, which today are `src/core/seal/seal.ts`,
+`src/core/preflight/reduce.ts`, and `src/core/emit/emit.ts`.
 
 The `eval-quality/conformance` subpath publishes the port boundary: the four port types, the message
-shapes they carry, and an executable conformance suite. An adapter is conforming when
+shapes they carry, the AD-28 `RUNTIME_FAULT_CODES` registry and `RuntimeFaultCode` type a conforming
+adapter throws against, and an executable conformance suite. An adapter is conforming when
 `runCorpusPortConformance`, `runClockPortConformance`, `runFileSystemPortConformance`, or
-`runEnvironmentProbePortConformance` returns a report whose `passed` is true, which is the definition
-rather than a paraphrase of one; each returns a report instead of asserting, so the suite carries no
-test framework and runs under whichever one you already use.
+`runEnvironmentProbePortConformance` returns a report whose `passed` is true, which is the definition;
+each returns a report, so the suite carries no test framework and runs under whichever one you
+already use.
 
 ```ts
 import { runCorpusPortConformance, type CorpusPort } from 'eval-quality/conformance'
@@ -327,20 +389,33 @@ import { runCorpusPortConformance, type CorpusPort } from 'eval-quality/conforma
 
 The suite drives a subject through four scenarios and checks six assertions per port method: a
 mechanism failure is a typed fault, exactly one underlying call happens on success and on failure, an
-aborted signal rejects promptly, an in-band error value is thrown rather than returned, and a
+aborted signal rejects promptly, an in-band error value is thrown as a fault, and a
 successful call returns a response the published schema accepts. The environment-probe port adds
 thirteen more from AD-35's default-deny target policy. `npm run test:conformance` runs the suite
 against the three adapters this package ships and against an in-repository probe subject that exists
 only as the suite's own subject.
 
+`docs/ad21-verdict-decision.generated.md` holds AD-21's two published verdict ladders, production and
+contract-scoring, emitted from the rule tables in `src/core/score/ladder.ts` together with the
+fixtures that exercise them, and guarded by `npm run check:ad21-table`. Each row carries its
+condition, its rung, the guard in prose, and whether `--strict` may promote it.
+
 `docs/ad31-coverage-predicates.generated.md` holds AD-31's published predicate table, emitted from
 the seven relevance predicates and their seven satisfaction twins run over a hand-authored contract
 corpus. It is generated by `npm run generate:ad31-table` and guarded by `npm run check:ad31-table`,
 a byte-exact drift check that fails when a predicate changes and the committed document does not, so
-the table is evidence the predicates produce rather than documentation kept beside them. Regenerate
-rather than hand-edit it.
-
-`build:shareable` renders this README, the product brief, the PRD, the architecture spine, all nine ADRs, and every document those pages link to (contributing, code of conduct, security, licence, and the four experiment records) to `_bmad-output/shareable/` as standalone styled HTML for sharing outside the repo. Rendering the linked documents is what lets a recipient without repository access follow the evidence, contribution, security, and licence links instead of hitting a 404; anything that has no page of its own, such as a directory, is marked in the export as needing repository access. Regenerate rather than hand-edit those files: `check:shareable` fails the build when the committed export is stale or carries a repository URL that is not the canonical one. Mermaid diagrams render as code blocks there, which is a known limitation.
+the table is evidence the predicates produce. A hand edit fails the check; regenerate.
+
+`docs/ad33-outcome-decision.generated.md` holds AD-33's published decision table: the ten
+invalidating conditions, the twenty-row outcome ladder, the two waiver rules, the eight
+corroboration rules, the named structural constraints with the infeasible input pairs derived from
+them, and five censuses over the fixture set. It is generated by `npm run generate:ad33-table` and
+guarded by `npm run check:ad33-table`, the same byte-exact drift check, and the builder refuses to
+publish a census cell at zero, so a rule or a state losing its last fixture fails the build. AD-33
+puts a cell-per-input-tuple table out of arithmetic reach, so what is published is the enumerated
+output of the total function itself. A hand edit fails the check; regenerate.
+
+`build:shareable` renders this README, the product brief, the PRD, the architecture spine, all nine ADRs, and every document those pages link to (contributing, code of conduct, security, licence, and the four experiment records) to `_bmad-output/shareable/` as standalone styled HTML for sharing outside the repo. Rendering the linked documents is what lets a recipient without repository access follow the evidence, contribution, security, and licence links; anything that has no page of its own, such as a directory, is marked in the export as needing repository access. A hand edit fails the check; regenerate: `check:shareable` fails the build when the committed export is stale or carries a repository URL that is not the canonical one. Mermaid diagrams render as code blocks there, which is a known limitation.
 
 ## Contributing
 

package/corpus/dev/README.md

@@ -6,8 +6,9 @@ this package without cloning the repository. Everything here is generated by
 
 ## What is here
 
-- `contracts/<contractId>.json`: nineteen contracts, one per AD-20 discipline rule in each
-  declaration state. Sixteen are published only after this package's own compile stage accepts
+- `contracts/<contractId>.json`: twenty-one contracts. Nineteen are one per AD-20 discipline
+  rule in each declaration state, and two describe a system under test that runs behind a command
+  rather than over HTTP. Eighteen are published only after this package's own compile stage accepts
   them, so every one of those is a contract the compiler admits. Three fail compilation by design;
   those ship as authored input, and `index.json` records the failure code each one raises.
 - `compile-seal-example/contract.json` and `compile-seal-example/brief.json`: one contract and
@@ -23,14 +24,20 @@ contract's strength is a separate thing this package does not ship.
 ## What is absent, and why
 
 **The qualified-probe dimensions are absent.** AD-38 asks for at least one qualified probe per
-probe class and per `expectedClean` state. Qualifying a probe needs a trial reducer, which Owed
-item 1 records as not yet built, and a defect signature for the probe, which Owed item 7 records as
-missing for the only probe this repository names. Both dimensions arrive with the stage that adds
-them.
-
-**Three of the four artifacts in AD-38's end-to-end example are absent.** The example there is a
-sealed brief, a conforming sealed run record, an isolation manifest, and an evaluator
-configuration. The last three are produced by ingest, which does not exist yet, and Owed item 7
-forbids hand-filling downstream values: the chain must be regenerated from the reference reducer
-once that reducer exists. So this directory ships the compile-and-seal pair under a name that does
-not claim AD-38's term.
+probe class and per `expectedClean` state. The probe schema now carries both halves qualification
+needs: AD-9's per-route qualification record and AD-40's machine-readable defect signature, with a
+corpus gate that admits a probe only when the two agree with its class. The trial reducer and the
+score stage that reads it are both shipped, so an admitted probe can be scored end to end today;
+what is still missing is this directory's own gate widening to require at least one such probe. The
+dimension arrives with the change that adds that gate.
+
+**Three of the four artifacts in AD-38's end-to-end example are absent here.** The example there is
+a sealed brief, a conforming sealed run record, an isolation manifest, and an evaluator
+configuration. The last three are inputs the shipped `ingest` stage consumes, authored for the
+worked example in `scripts/worked-example-target.ts` and exercised there through
+`ingest`/`score`/`emit`. Only the run record among them is committed, as
+`spike-worked-example/sealed-run-record.json`; the isolation manifest and the evaluator
+configuration exist only as the authored values that build passes to `ingest`, never serialized to
+a file. This directory still ships only the compile-and-seal pair, scoped to what a corpus of
+contracts needs,
+under a name that does not claim AD-38's term.

package/corpus/dev/compile-seal-example/brief.json

@@ -1 +1 @@
-{"behaviors":[{"description":"A created thing is readable back in the list of things.","id":"B-001","observableSuccessCriterion":"A list call after a create returns one element per seeded thing, carrying the name the create call sent.","oracles":["O-001","O-002","O-003","O-004","O-005","O-006","O-007"],"requirementLinks":[{"id":"REQ-1","scheme":"local"}],"riskLinks":[{"id":"RISK-1","scheme":"local-risk"}],"severity":"critical"}],"budgets":{"maxCostUsd":"0.25","maxToolCalls":20,"maxWallClockMinutes":5},"contractDigest":"sha256:1f7c657db755e9550ee78c57bdb81ea9f9de9ca227dbeaf823939a0313cbd781","directions":[{"oracleId":"O-001","text":"Its items field from the list things endpoint (with the supplied query limit) is asserted to satisfy the declared \"covers-by-key\" condition. The declared polarity expects this relation to hold. One list call over the seeded set. A list omitting a seeded thing, or repeating one is treated as a defect."},{"oracleId":"O-002","text":"The create thing endpoint (with the supplied body name): its error field, its id field, and its ok field is asserted to satisfy every declared condition together. The declared polarity expects this relation to hold. The whole create response. A create reporting success with no identifier, or with a diagnostic beside it is treated as a defect."},{"oracleId":"O-003","text":"The list things endpoint (with the supplied query limit): its error field and its items field is asserted to satisfy every declared condition together. The declared polarity expects this relation to hold. The list response taken as a whole. A list carrying items alongside a diagnostic field is treated as a defect."},{"oracleId":"O-004","text":"Its error field from the create thing endpoint (with a malformed body name value) and its error field from the list things endpoint (with a malformed query limit value) is asserted to satisfy every declared condition together. The declared polarity expects this relation to hold. Both sibling operations, each given an input that violates its declared type. One sibling rejecting the malformed input while the other accepts it is treated as a defect."},{"oracleId":"O-005","text":"The body name value you sent to the create thing endpoint (with the supplied body name), compared with the query limit value you sent to the list things endpoint (with the supplied query limit) is asserted to satisfy every declared condition together. The declared polarity expects this relation to hold. The two sibling parameters, as sent. One parameter carried and the other dropped is treated as a defect."},{"oracleId":"O-006","text":"The body name value you sent to the create thing endpoint (with the supplied body name), compared with its items field from the list things endpoint (with the supplied query limit) is asserted to contain the declared member. The declared polarity expects this relation to hold. The list read after the create, against the name the create sent. A create reporting success whose thing never appears in a later list is treated as a defect."},{"oracleId":"O-007","text":"Every element reachable through its items field from the list things endpoint (with the supplied query limit) is asserted to meet the declared condition. The declared polarity expects this relation to hold. Every element of the returned list. A list whose first element carries an identifier and whose later elements do not is treated as a defect."}],"parentDigest":null,"permittedInterfaces":[{"kind":"api","logicalId":"thing-api"}],"probeStepBound":8,"revisionCount":0,"safetyLimits":["No request to any host other than the mapped thing-api target."],"schemaVersion":1,"scopedResources":[]}
+{"behaviors":[{"description":"A created thing is readable back in the list of things.","id":"B-001","observableSuccessCriterion":"A list call after a create returns one element per seeded thing, carrying the name the create call sent.","oracles":["O-001","O-002","O-003","O-004","O-005","O-006","O-007"],"requirementLinks":[{"id":"REQ-1","scheme":"local"}],"riskLinks":[{"id":"RISK-1","scheme":"local-risk"}],"severity":"critical"}],"budgets":{"maxCostUsd":"0.25","maxToolCalls":20,"maxWallClockMinutes":5},"contractDigest":"sha256:7ca80def8afa456f9afa171dbdbe7be9c7c9c9b799cf7bedfa5f6a0db71cd0a9","directions":[{"oracleId":"O-001","text":"Its items field from the list things endpoint (with the supplied query limit) is asserted to satisfy the declared \"covers-by-key\" condition. The declared polarity expects this relation to hold. One list call over the seeded set. A list omitting a seeded thing, or repeating one is treated as a defect."},{"oracleId":"O-002","text":"The create thing endpoint (with the supplied body name): its error field, its id field, and its ok field is asserted to satisfy every declared condition together. The declared polarity expects this relation to hold. The whole create response. A create reporting success with no identifier, or with a diagnostic beside it is treated as a defect."},{"oracleId":"O-003","text":"The list things endpoint (with the supplied query limit): its error field and its items field is asserted to satisfy every declared condition together. The declared polarity expects this relation to hold. The list response taken as a whole. A list carrying items alongside a diagnostic field is treated as a defect."},{"oracleId":"O-004","text":"Its error field from the create thing endpoint (with a malformed body name value) and its error field from the list things endpoint (with a malformed query limit value) is asserted to satisfy every declared condition together. The declared polarity expects this relation to hold. Both sibling operations, each given an input that violates its declared type. One sibling rejecting the malformed input while the other accepts it is treated as a defect."},{"oracleId":"O-005","text":"The body name value you sent to the create thing endpoint (with the supplied body name), compared with the query limit value you sent to the list things endpoint (with the supplied query limit) is asserted to satisfy every declared condition together. The declared polarity expects this relation to hold. The two sibling parameters, as sent. One parameter carried and the other dropped is treated as a defect."},{"oracleId":"O-006","text":"The body name value you sent to the create thing endpoint (with the supplied body name), compared with its items field from the list things endpoint (with the supplied query limit) is asserted to contain the declared member. The declared polarity expects this relation to hold. The list read after the create, against the name the create sent. A create reporting success whose thing never appears in a later list is treated as a defect."},{"oracleId":"O-007","text":"Every element reachable through its items field from the list things endpoint (with the supplied query limit) is asserted to meet the declared condition. The declared polarity expects this relation to hold. Every element of the returned list. A list whose first element carries an identifier and whose later elements do not is treated as a defect."}],"parentDigest":null,"permittedInterfaces":[{"kind":"api","logicalId":"thing-api"}],"principals":[],"probeStepBound":8,"revisionCount":0,"safetyLimits":["No request to any host other than the mapped thing-api target."],"schemaVersion":2,"scopedResources":[]}

package/corpus/dev/compile-seal-example/contract.json

@@ -1 +1 @@
-{"behaviors":[{"description":"A created thing is readable back in the list of things.","id":"B-001","observableSuccessCriterion":"A list call after a create returns one element per seeded thing, carrying the name the create call sent.","oracles":["O-001","O-002","O-003","O-004","O-005","O-006","O-007"],"requirementLinks":[{"id":"REQ-1","scheme":"local"}],"riskLinks":[{"id":"RISK-1","scheme":"local-risk"}],"severity":"critical"}],"budgets":{"maxCostUsd":"0.25","maxToolCalls":20,"maxWallClockMinutes":5},"contractId":"satisfied-declarations","fixtureReset":null,"forbiddenInputs":["original-spec","source-code","repository","builder-transcript","implementation-logs","comparator-results","human-labels"],"interactionPlan":[{"after":null,"inputBinding":{"body":{"name":{"matcher":"any"}},"header":null,"path":null,"query":null},"operationId":"create-thing","stepId":"create"},{"after":"create","inputBinding":{"body":null,"header":null,"path":null,"query":{"limit":{"literal":10}}},"operationId":"list-things","stepId":"list"},{"after":null,"inputBinding":{"body":{"name":{"matcher":"type-violating"}},"header":null,"path":null,"query":null},"operationId":"create-thing","stepId":"malformed-create"},{"after":null,"inputBinding":{"body":null,"header":null,"path":null,"query":{"limit":{"matcher":"type-violating"}}},"operationId":"list-things","stepId":"malformed-list"}],"oracles":[{"check":{"actualKey":"id","expectedKey":"id","op":"covers-by-key","operands":[{"referenceSet":"expected-things"},{"pointer":"/interactions/list/response-body/items"}]},"commentary":"Reconciles the whole list against the declared set.","direction":{"evidenceTargets":["/interactions/list/response-body/items"],"negativeDomain":"A list omitting a seeded thing, or repeating one.","polarity":"expects-hold","relation":"covers-by-key","scope":"One list call over the seeded set."},"id":"O-001","polarity":"expects-hold"},{"check":{"op":"all","operands":[{"op":"existence","operands":[{"pointer":"/interactions/create/response-body/ok"}]},{"op":"existence","operands":[{"pointer":"/interactions/create/response-body/id"}]},{"op":"absence","operands":[{"pointer":"/interactions/create/response-body/error"}]}]},"commentary":null,"direction":{"evidenceTargets":["/interactions/create/response-body/ok","/interactions/create/response-body/id","/interactions/create/response-body/error"],"negativeDomain":"A create reporting success with no identifier, or with a diagnostic beside it.","polarity":"expects-hold","relation":"all","scope":"The whole create response."},"id":"O-002","polarity":"expects-hold"},{"check":{"op":"all","operands":[{"op":"existence","operands":[{"pointer":"/interactions/list/response-body/items"}]},{"op":"absence","operands":[{"pointer":"/interactions/list/response-body/error"}]}]},"commentary":null,"direction":{"evidenceTargets":["/interactions/list/response-body/items","/interactions/list/response-body/error"],"negativeDomain":"A list carrying items alongside a diagnostic field.","polarity":"expects-hold","relation":"all","scope":"The list response taken as a whole."},"id":"O-003","polarity":"expects-hold"},{"check":{"op":"all","operands":[{"op":"existence","operands":[{"pointer":"/interactions/malformed-create/response-body/error"}]},{"op":"existence","operands":[{"pointer":"/interactions/malformed-list/response-body/error"}]}]},"commentary":null,"direction":{"evidenceTargets":["/interactions/malformed-create/response-body/error","/interactions/malformed-list/response-body/error"],"negativeDomain":"One sibling rejecting the malformed input while the other accepts it.","polarity":"expects-hold","relation":"all","scope":"Both sibling operations, each given an input that violates its declared type."},"id":"O-004","polarity":"expects-hold"},{"check":{"op":"all","operands":[{"op":"existence","operands":[{"pointer":"/interactions/create/call-inputs/body/name"}]},{"op":"existence","operands":[{"pointer":"/interactions/list/call-inputs/query/limit"}]}]},"commentary":null,"direction":{"evidenceTargets":["/interactions/create/call-inputs/body/name","/interactions/list/call-inputs/query/limit"],"negativeDomain":"One parameter carried and the other dropped.","polarity":"expects-hold","relation":"all","scope":"The two sibling parameters, as sent."},"id":"O-005","polarity":"expects-hold"},{"check":{"op":"containment","operands":[{"pointer":"/interactions/list/response-body/items"},{"pointer":"/interactions/create/call-inputs/body/name"}]},"commentary":null,"direction":{"evidenceTargets":["/interactions/list/response-body/items","/interactions/create/call-inputs/body/name"],"negativeDomain":"A create reporting success whose thing never appears in a later list.","polarity":"expects-hold","relation":"containment","scope":"The list read after the create, against the name the create sent."},"id":"O-006","polarity":"expects-hold"},{"check":{"collection":{"pointer":"/interactions/list/response-body/items"},"op":"for-all","predicate":{"op":"existence","operands":[{"pointer":"@/id"}]}},"commentary":null,"direction":{"evidenceTargets":["/interactions/list/response-body/items"],"negativeDomain":"A list whose first element carries an identifier and whose later elements do not.","polarity":"expects-hold","relation":"for-all","scope":"Every element of the returned list."},"id":"O-007","polarity":"expects-hold"}],"parentDigest":null,"permittedInterfaces":[{"kind":"api","logicalId":"thing-api","operations":[{"method":"POST","operationId":"create-thing","pathTemplate":"/things","requestShape":{"body":{"permittedKeys":["name"],"requiredKeys":["name"],"types":{"name":"string"}},"header":{"permittedKeys":[],"requiredKeys":[],"types":{}},"path":{"permittedKeys":[],"requiredKeys":[],"types":{}},"query":{"permittedKeys":[],"requiredKeys":[],"types":{}}},"responseDescriptor":{"channelRoles":{"/error":"diagnostic","/id":"payload","/ok":"success-indicator"},"collectionLocations":[],"permittedKeys":["id","ok","error"],"requiredKeys":["id","ok"],"successIndicator":"/ok","types":{"error":"string","id":"string","ok":"boolean"}},"sensitivityWitness":{"channel":"body","legs":[{"inputs":{"body":{"kind":"json","value":{"name":"alpha"}},"header":{},"path":{},"query":{}},"legId":"create-witness-a"},{"inputs":{"body":{"kind":"json","value":{"name":"beta"}},"header":{},"path":{},"query":{}},"legId":"create-witness-b"}],"relation":{"op":"not","operands":[{"op":"deep-equality","operands":[{"pointer":"/interactions/create-witness-a/response-body"},{"pointer":"/interactions/create-witness-b/response-body"}]}]},"witnessId":"create-thing-sensitivity"},"stateChangeMarker":true,"volatilePointers":["/id"]},{"method":"GET","operationId":"list-things","pathTemplate":"/things","requestShape":{"body":{"permittedKeys":[],"requiredKeys":[],"types":{}},"header":{"permittedKeys":[],"requiredKeys":[],"types":{}},"path":{"permittedKeys":[],"requiredKeys":[],"types":{}},"query":{"permittedKeys":["limit"],"requiredKeys":[],"types":{"limit":"number"}}},"responseDescriptor":{"channelRoles":{"/error":"diagnostic","/items":"collection"},"collectionLocations":[{"expectedCardinality":{"count":3,"mode":"exact"},"pointer":"/items","referenceSet":"expected-things"}],"permittedKeys":["items","error"],"requiredKeys":["items"],"successIndicator":"/items","types":{"error":"string","items":"array"}},"sensitivityWitness":{"channel":"query","legs":[{"inputs":{"body":{"kind":"absent"},"header":{},"path":{},"query":{"limit":1}},"legId":"list-witness-a"},{"inputs":{"body":{"kind":"absent"},"header":{},"path":{},"query":{"limit":2}},"legId":"list-witness-b"}],"relation":{"op":"not","operands":[{"op":"deep-equality","operands":[{"pointer":"/interactions/list-witness-a/response-body"},{"pointer":"/interactions/list-witness-b/response-body"}]}]},"witnessId":"list-things-sensitivity"},"stateChangeMarker":false,"volatilePointers":[]}]}],"probeStepBound":8,"referenceSets":{"expected-things":{"commentary":null,"keys":["id"],"members":[{"id":"t-1"},{"id":"t-2"},{"id":"t-3"}]}},"requiredEvidence":["Request and response pair for every call, in order."],"revisionCount":0,"rubrics":[{"criteria":[{"evidence":"/interactions/list/response-body/items","id":"RC-001","text":"Does the returned list carry every expected identifier?"}],"failureModePenalties":[{"description":"An expected thing is missing.","name":"omission"}],"id":"R-001","maxLength":400,"scaleLevels":[{"anchor":"Every expected thing is present.","level":1}]}],"safetyLimits":["No request to any host other than the mapped thing-api target."],"schemaVersion":1,"scopedResources":null,"siblingGroups":{"operations":[["create-thing","list-things"]],"parameters":[["limit","name"]]},"sourceSpecDigest":null,"testData":{"cleanup":"Delete every thing created during the run.","setup":"Seed exactly three things with identifiers t-1, t-2, t-3."},"waivers":[{"approval":"gate-c-reviewer","condition":null,"expiresAt":"2027-01-01T00:00:00Z","id":"W-001","rationale":"The upstream seed is unavailable in the sandbox environment.","rule":"omission-and-completeness"}]}
+{"behaviors":[{"description":"A created thing is readable back in the list of things.","id":"B-001","observableSuccessCriterion":"A list call after a create returns one element per seeded thing, carrying the name the create call sent.","oracles":["O-001","O-002","O-003","O-004","O-005","O-006","O-007"],"requirementLinks":[{"id":"REQ-1","scheme":"local"}],"riskLinks":[{"id":"RISK-1","scheme":"local-risk"}],"severity":"critical"}],"budgets":{"maxCostUsd":"0.25","maxToolCalls":20,"maxWallClockMinutes":5},"contractId":"satisfied-declarations","fixtureReset":null,"forbiddenInputs":["original-spec","source-code","repository","builder-transcript","implementation-logs","comparator-results","human-labels"],"interactionPlan":[{"after":null,"cardinality":"exactly-one","inputBinding":{"body":{"name":{"matcher":"any"}},"header":null,"path":null,"query":null},"operationId":"create-thing","stepId":"create"},{"after":"create","cardinality":"exactly-one","inputBinding":{"body":null,"header":null,"path":null,"query":{"limit":{"literal":10}}},"operationId":"list-things","stepId":"list"},{"after":null,"cardinality":"exactly-one","inputBinding":{"body":{"name":{"matcher":"type-violating"}},"header":null,"path":null,"query":null},"operationId":"create-thing","stepId":"malformed-create"},{"after":null,"cardinality":"exactly-one","inputBinding":{"body":null,"header":null,"path":null,"query":{"limit":{"matcher":"type-violating"}}},"operationId":"list-things","stepId":"malformed-list"}],"oracles":[{"check":{"actualKey":"id","expectedKey":"id","op":"covers-by-key","operands":[{"referenceSet":"expected-things"},{"pointer":"/interactions/list/response-body/items"}]},"commentary":"Reconciles the whole list against the declared set.","direction":{"evidenceTargets":["/interactions/list/response-body/items"],"negativeDomain":"A list omitting a seeded thing, or repeating one.","polarity":"expects-hold","relation":"covers-by-key","scope":"One list call over the seeded set."},"id":"O-001","polarity":"expects-hold"},{"check":{"op":"all","operands":[{"op":"existence","operands":[{"pointer":"/interactions/create/response-body/ok"}]},{"op":"existence","operands":[{"pointer":"/interactions/create/response-body/id"}]},{"op":"absence","operands":[{"pointer":"/interactions/create/response-body/error"}]}]},"commentary":null,"direction":{"evidenceTargets":["/interactions/create/response-body/ok","/interactions/create/response-body/id","/interactions/create/response-body/error"],"negativeDomain":"A create reporting success with no identifier, or with a diagnostic beside it.","polarity":"expects-hold","relation":"all","scope":"The whole create response."},"id":"O-002","polarity":"expects-hold"},{"check":{"op":"all","operands":[{"op":"existence","operands":[{"pointer":"/interactions/list/response-body/items"}]},{"op":"absence","operands":[{"pointer":"/interactions/list/response-body/error"}]}]},"commentary":null,"direction":{"evidenceTargets":["/interactions/list/response-body/items","/interactions/list/response-body/error"],"negativeDomain":"A list carrying items alongside a diagnostic field.","polarity":"expects-hold","relation":"all","scope":"The list response taken as a whole."},"id":"O-003","polarity":"expects-hold"},{"check":{"op":"all","operands":[{"op":"existence","operands":[{"pointer":"/interactions/malformed-create/response-body/error"}]},{"op":"existence","operands":[{"pointer":"/interactions/malformed-list/response-body/error"}]}]},"commentary":null,"direction":{"evidenceTargets":["/interactions/malformed-create/response-body/error","/interactions/malformed-list/response-body/error"],"negativeDomain":"One sibling rejecting the malformed input while the other accepts it.","polarity":"expects-hold","relation":"all","scope":"Both sibling operations, each given an input that violates its declared type."},"id":"O-004","polarity":"expects-hold"},{"check":{"op":"all","operands":[{"op":"existence","operands":[{"pointer":"/interactions/create/call-inputs/body/name"}]},{"op":"existence","operands":[{"pointer":"/interactions/list/call-inputs/query/limit"}]}]},"commentary":null,"direction":{"evidenceTargets":["/interactions/create/call-inputs/body/name","/interactions/list/call-inputs/query/limit"],"negativeDomain":"One parameter carried and the other dropped.","polarity":"expects-hold","relation":"all","scope":"The two sibling parameters, as sent."},"id":"O-005","polarity":"expects-hold"},{"check":{"op":"containment","operands":[{"pointer":"/interactions/list/response-body/items"},{"pointer":"/interactions/create/call-inputs/body/name"}]},"commentary":null,"direction":{"evidenceTargets":["/interactions/list/response-body/items","/interactions/create/call-inputs/body/name"],"negativeDomain":"A create reporting success whose thing never appears in a later list.","polarity":"expects-hold","relation":"containment","scope":"The list read after the create, against the name the create sent."},"id":"O-006","polarity":"expects-hold"},{"check":{"collection":{"pointer":"/interactions/list/response-body/items"},"op":"for-all","predicate":{"op":"existence","operands":[{"pointer":"@/id"}]}},"commentary":null,"direction":{"evidenceTargets":["/interactions/list/response-body/items"],"negativeDomain":"A list whose first element carries an identifier and whose later elements do not.","polarity":"expects-hold","relation":"for-all","scope":"Every element of the returned list."},"id":"O-007","polarity":"expects-hold"}],"parentDigest":null,"permittedInterfaces":[{"kind":"api","logicalId":"thing-api","operations":[{"method":"POST","operationId":"create-thing","pathTemplate":"/things","requestShape":{"body":{"permittedKeys":["name"],"requiredKeys":["name"],"types":{"name":"string"}},"header":{"permittedKeys":[],"requiredKeys":[],"types":{}},"path":{"permittedKeys":[],"requiredKeys":[],"types":{}},"query":{"permittedKeys":[],"requiredKeys":[],"types":{}}},"responseDescriptor":{"channelRoles":{"/error":"diagnostic","/id":"payload","/ok":"success-indicator"},"collectionLocations":[],"permittedKeys":["id","ok","error"],"requiredKeys":["id","ok"],"successIndicator":"/ok","types":{"error":"string","id":"string","ok":"boolean"}},"sensitivityWitness":{"channel":"body","legs":[{"inputs":{"body":{"kind":"json","value":{"name":"alpha"}},"header":{},"path":{},"query":{}},"legId":"create-witness-a"},{"inputs":{"body":{"kind":"json","value":{"name":"beta"}},"header":{},"path":{},"query":{}},"legId":"create-witness-b"}],"relation":{"op":"not","operands":[{"op":"deep-equality","operands":[{"pointer":"/interactions/create-witness-a/response-body"},{"pointer":"/interactions/create-witness-b/response-body"}]}]},"witnessId":"create-thing-sensitivity"},"stateChangeMarker":true,"volatilePointers":["/id"]},{"method":"GET","operationId":"list-things","pathTemplate":"/things","requestShape":{"body":{"permittedKeys":[],"requiredKeys":[],"types":{}},"header":{"permittedKeys":[],"requiredKeys":[],"types":{}},"path":{"permittedKeys":[],"requiredKeys":[],"types":{}},"query":{"permittedKeys":["limit"],"requiredKeys":[],"types":{"limit":"number"}}},"responseDescriptor":{"channelRoles":{"/error":"diagnostic","/items":"collection"},"collectionLocations":[{"expectedCardinality":{"count":3,"mode":"exact"},"pointer":"/items","referenceSet":"expected-things"}],"permittedKeys":["items","error"],"requiredKeys":["items"],"successIndicator":"/items","types":{"error":"string","items":"array"}},"sensitivityWitness":{"channel":"query","legs":[{"inputs":{"body":{"kind":"absent"},"header":{},"path":{},"query":{"limit":1}},"legId":"list-witness-a"},{"inputs":{"body":{"kind":"absent"},"header":{},"path":{},"query":{"limit":2}},"legId":"list-witness-b"}],"relation":{"op":"not","operands":[{"op":"deep-equality","operands":[{"pointer":"/interactions/list-witness-a/response-body"},{"pointer":"/interactions/list-witness-b/response-body"}]}]},"witnessId":"list-things-sensitivity"},"stateChangeMarker":false,"volatilePointers":[]}]}],"probeStepBound":8,"referenceSets":{"expected-things":{"commentary":null,"keys":["id"],"members":[{"id":"t-1"},{"id":"t-2"},{"id":"t-3"}]}},"requiredEvidence":["Request and response pair for every call, in order."],"revisionCount":0,"rubrics":[{"criteria":[{"evidence":"/interactions/list/response-body/items","id":"RC-001","text":"Does the returned list carry every expected identifier?"}],"failureModePenalties":[{"description":"An expected thing is missing.","name":"omission"}],"id":"R-001","maxLength":400,"scaleLevels":[{"anchor":"Every expected thing is present.","level":1}]}],"safetyLimits":["No request to any host other than the mapped thing-api target."],"schemaVersion":4,"scopedResources":null,"siblingGroups":{"operations":[["create-thing","list-things"]],"parameters":[["limit","name"]]},"sourceSpecDigest":null,"testData":{"cleanup":"Delete every thing created during the run.","principals":null,"resources":null,"setup":"Seed exactly three things with identifiers t-1, t-2, t-3."},"waivers":[{"approval":"gate-c-reviewer","condition":null,"expiresAt":"2027-01-01T00:00:00Z","id":"W-001","rationale":"The upstream seed is unavailable in the sandbox environment.","rule":"omission-and-completeness"}]}