@event4u/agent-config 9.8.0 → 9.10.0

package/docs/contracts/audit-log-v1.md

@@ -1,150 +0,0 @@
----
-stability: beta
-keep-beta-until: 2026-08-12
----
-
-# Audit-log v1
-
-**Purpose.** Pin the append-only schema that the audit-as-memory
-pipeline writes per `/work` and `/implement-ticket` run, so the
-pattern-extraction script ([`extract_audit_patterns.py`](../../src/scripts/extract_audit_patterns.py))
-and the `learning-to-rule-or-skill` skill can mine repeated successful
-patterns without re-reading conversation bodies.
-
-**Scope.** Defines the JSONL line shape, file location, redaction
-floor, append-only invariant, and the producer/consumer contract.
-Does **not** define how patterns are scored or how proposals are
-promoted — that is the consumer's responsibility (see Cross-references).
-
-Last refreshed: 2026-05-11.
-
-## Producer / consumer split
-
-| Side | Responsibility | Where |
-|---|---|---|
-| **Producer** | One JSONL line per phase end, derived from the phase's [`decision-trace-v1.md`](decision-trace-v1.md) JSON and the [`memory-visibility-v1.md`](memory-visibility-v1.md) counts. | `work_engine` hook on phase boundary. |
-| **Consumer** | Pattern mining + human review gate. Never edits a written line; corrections are new lines with `type=supersede`. | [`extract_audit_patterns.py`](../../src/scripts/extract_audit_patterns.py). |
-
-## File location
-
-```
-agents/runtime/state/audit/<YYYY-MM>.jsonl
-```
-
-One file per UTC month. Files are append-only — `merge=union` via
-`.gitattributes` (same mechanism as the archived
-`agents/memory/intake/*.jsonl` recipe). Files MAY be gitignored in
-consumer projects; the contract does not require commit.
-
-## Line shape
-
-One JSON object per line, UTF-8, no trailing whitespace:
-
-```json
-{"schema_version":1,"id":"01HXY...","ts":"2026-05-11T12:34:56Z","work_id":"PROJ-123-2026-05-11T12-30-00Z","phase":"verify","outcome":"success","confidence_band":"high","risk_class":"low","memory":{"asks":3,"hits":2},"verify":{"claims":1,"first_try_passes":1},"rules_applied":["verify-before-complete","commit-policy"],"persona":"backend","input_kind":"ticket","type":"phase"}
-```
-
-Single-line. The pretty-printed reference shape:
-
-```json
-{
-  "schema_version": 1,
-  "id": "01HXY7K9...",
-  "ts": "2026-05-11T12:34:56Z",
-  "work_id": "PROJ-123-2026-05-11T12-30-00Z",
-  "phase": "verify",
-  "outcome": "success",
-  "confidence_band": "high",
-  "risk_class": "low",
-  "memory": { "asks": 3, "hits": 2 },
-  "verify": { "claims": 1, "first_try_passes": 1 },
-  "rules_applied": ["verify-before-complete", "commit-policy"],
-  "persona": "backend",
-  "input_kind": "ticket",
-  "type": "phase"
-}
-```
-
-## Field semantics
-
-| Field | Type | Meaning |
-|---|---|---|
-| `schema_version` | int | Always `1` for this contract. Major bump on breaking changes. |
-| `id` | string | ULID or content hash. Stable, deduplicated by the **reader**, never by in-place edit. |
-| `ts` | string | ISO-8601 UTC timestamp of phase end. |
-| `work_id` | string | Matches the `WorkState` directory id from [`decision-trace-v1.md`](decision-trace-v1.md). Allows cross-trace correlation. |
-| `phase` | enum | One of `refine` · `memory` · `analyze` · `plan` · `implement` · `test` · `verify` · `report`. |
-| `outcome` | enum | One of `success` · `blocked` · `skipped` · `error`. Mirrors `Outcome` from `work_engine.directives`. |
-| `confidence_band` | enum | One of `low` · `medium` · `high`. Sourced from decision-trace. |
-| `risk_class` | enum | One of `low` · `medium` · `high`. Inherits max risk from touched files. |
-| `memory.asks` / `memory.hits` | int | Counts only — never ids, never bodies. |
-| `verify.claims` / `verify.first_try_passes` | int | Verify-gate counts. |
-| `rules_applied` | string[] | Stable rule ids whose Iron Law fired this phase. Bounded to ≤ 32; remainder dropped silently. |
-| `persona` | string \| null | Resolved `roles.active_role` from `.agent-settings.yml` at phase start. |
-| `input_kind` | enum | One of `prompt` · `ticket` · `orchestration`. Matches `WorkState.input.kind`. |
-| `type` | enum | One of `phase` · `supersede` · `note`. `supersede` carries an extra `supersedes` field with the prior `id`. |
-
-Unknown trailing fields are forward-compat extensions; readers MUST
-NOT raise on them.
-
-A line MAY carry an optional `orchestration` object when the run was
-produced by the auto-dispatch orchestration layer. It is additive and
-non-breaking — `schema_version` is unchanged, no existing field is
-removed or renamed, and readers that do not understand it ignore it per
-the forward-compat rule above. Its shape lives in
-[`orchestration-telemetry.md`](../../src/agent-src/contexts/execution/orchestration-telemetry.md).
-
-## Privacy floor
-
-Lines MUST NOT contain:
-
-- Conversation **bodies**, summaries, prompts, or quoted snippets.
-- Memory entry bodies, ids, or content. Only **counts** travel here —
-  ids stay in `decision-trace-*.json` per [`memory-visibility-v1.md`](memory-visibility-v1.md).
-- Secrets, tokens, environment values, file contents.
-- Paths outside the package's `agents/runtime/state/` and `tests/` allowlist.
-
-The floor is enforced by
-`tests/contracts/test_audit_log_redaction.py` — any new producer-side
-edit that touches this schema adds a fixture there.
-
-## Append-only invariant
-
-- New entries append. Existing lines are **immutable**.
-- A correction is a **new** entry with `type=supersede` and a
-  `supersedes` field carrying the prior `id`. The reader applies the
-  chain.
-- Deletion is forbidden at the producer layer. Operators rotate
-  monthly files; archived months MAY be purged out-of-band by the
-  consumer project's retention policy.
-
-## Cadence
-
-One line per phase end. The producer hook fires on the same
-`post_tool_use` / `stop` boundary that emits the visibility line
-([`memory-visibility-v1.md`](memory-visibility-v1.md) § Cadence).
-
-Cost-profile interaction:
-
-| Cost profile | Line emission |
-|---|---|
-| `lean` | suppress unless `outcome != success` OR `risk_class >= medium` |
-| `standard` | always |
-| `verbose` | always |
-
-## Stability
-
-Beta. Breaking changes between v1 and v2 are allowed in a minor
-release if the change appears in `CHANGELOG.md` under a `### Breaking`
-heading. Consumers MUST gate on `schema_version` and refuse unknown
-majors.
-
-## Cross-references
-
-- Input feed (counts + ids): [`memory-visibility-v1.md`](memory-visibility-v1.md).
-- Per-phase JSON the producer reads: [`decision-trace-v1.md`](decision-trace-v1.md).
-- Pattern-extraction consumer: [`extract_audit_patterns.py`](../../src/scripts/extract_audit_patterns.py).
-- Skill that consumes promoted patterns:
-  [`learning-to-rule-or-skill`](../../.agent-src.uncondensed/skills/learning-to-rule-or-skill/SKILL.md).
-- Append-only JSONL precedent:
-  [`adr-chat-history-split.md`](adr-chat-history-split.md).

package/docs/contracts/auto-orchestration-v1.md

@@ -1,70 +0,0 @@
----
-stability: beta
-keep-beta-until: 2026-09-23
----
-
-# Auto-Orchestration v1
-
-**Purpose.** Pin the data shapes the automatic subagent-orchestration layer
-depends on — the host-capability manifest, the spawn brief, and the
-orchestration telemetry object — so producers and consumers agree without
-re-deriving them. Behaviour (when to dispatch, how to route, how to verify)
-lives in the context docs; this contract pins the **shapes**.
-
-**Scope.** The three JSON shapes below + the activation precondition. Does not
-define classification rules, routing policy, or the verification budget — those
-are the cited context docs.
-
-Last refreshed: 2026-06-23. Decided in
-[ADR-105](../decisions/ADR-105-automatic-subagent-orchestration.md).
-
-## Host-capability manifest
-
-Resolved once per session, cached. Safe default = all-false.
-
-```json
-{ "schema_version": 1, "subagent_spawn": false, "parallel_spawn": false, "status_polling": false, "separate_quota_pool": false }
-```
-
-Full semantics: [`host-capability-manifest`](../../src/agent-src/contexts/execution/host-capability-manifest.md).
-
-## Spawn brief
-
-Composed per delegated sub-task; knowledge is references only, capped.
-
-```json
-{ "task": "<sub-task>", "role_mode": "reviewer|null", "profile": "<id|null>", "personas": ["<id>"], "knowledge_refs": ["<id-or-path>"] }
-```
-
-Full semantics: [`subagent-spawn-contract`](../../src/agent-src/contexts/execution/subagent-spawn-contract.md).
-
-## Orchestration telemetry
-
-Optional `orchestration` object on an [`audit-log-v1`](audit-log-v1.md) line —
-additive, non-breaking, counts + ids only.
-
-```json
-{ "task_size_estimate": 0, "spawn_count": 0, "tiers": [], "token_delta": 0, "wall_clock_ms": 0, "outcome": "DONE", "verify_mode": "deterministic" }
-```
-
-Full semantics: [`orchestration-telemetry`](../../src/agent-src/contexts/execution/orchestration-telemetry.md).
-
-## Activation precondition
-
-Auto-dispatch is attempted only when:
-
-```
-subagents.enabled AND subagents.auto != off AND manifest.subagent_spawn AND task-is-delegable
-```
-
-Any failing → in-session. Full gate:
-[`auto-orchestration-activation`](../../src/agent-src/contexts/execution/auto-orchestration-activation.md).
-
-## Cross-references
-
-- [ADR-105](../decisions/ADR-105-automatic-subagent-orchestration.md) — the decisions.
-- [`subagent-routing`](../../src/agent-src/contexts/execution/subagent-routing.md) — model downshift + quota bonus.
-- [`auto-dispatch-classification`](../../src/agent-src/contexts/execution/auto-dispatch-classification.md) — delegability rules.
-- [`verify-budget`](../../src/agent-src/contexts/execution/verify-budget.md) — verification budget.
-- [`subagent-steering`](../../src/agent-src/contexts/execution/subagent-steering.md) — guardrails + kill-switch.
-- [`orchestration-benchmark-gate`](../../src/agent-src/contexts/execution/orchestration-benchmark-gate.md) — the default-flip gate.

package/docs/contracts/benchmark-ab-contract.md

@@ -1,106 +0,0 @@
----
-stability: beta
-keep-beta-until: 2026-09-04
----
-
-# Package-Impact A/B Benchmark — Contract
-
-> Lifecycle, cache invalidation, and reader-side semantics for `docs/benchmark.md` and the underlying A/B bench reports. Companion to `docs/contracts/benchmark-report-schema.md` (which owns the per-report JSON shape this contract layers an axis onto).
-
-## Scope
-
-This contract covers the **variant axis** (`with` vs. `without` agent-config) that the package-impact A/B bench adds on top of the existing version-over-time bench. It does NOT redefine the underlying report schema — see `benchmark-report-schema.md` for the per-report JSON shape.
-
-## Producer / consumer surface
-
-| Concern | Owner |
-|---|---|
-| Materialising the variant clones | `scripts/bench_ab_clone.py` |
-| Verifying the clones differ only at the agent-config surface | `scripts/bench_ab_integrity.py` |
-| Track A (behavioural) runner | `scripts/bench_ab_tracka_run.py` |
-| Track B (task) runner | `scripts/bench_ab_task_runner.py` |
-| Track B scoring | `scripts/_lib/bench_ab_scoring.py` |
-| Cache key + lookup | `scripts/_lib/bench_ab_cache.py` |
-| Variant diff | `scripts/bench_ab_diff.py` |
-| Rendered report | `scripts/render_benchmark_md.py` → `docs/benchmark.md` |
-| Corpus / doc linter | `scripts/lint_bench_ab.py` |
-| Task orchestration | `taskfiles/bench-ab.yml` (`task bench:ab*`) |
-
-## When `docs/benchmark.md` regenerates
-
-`docs/benchmark.md` is **derived**. It regenerates from the latest paired reports under `internal/bench/reports/ab/` whenever:
-
-1. `task bench:ab` runs (the full pipeline ends with `bench:ab:diff` which calls the renderer).
-2. `task bench:ab:diff` runs alone (no fresh bench, just re-render).
-3. The renderer is invoked directly: `./scripts-run src/scripts/render_benchmark_md`.
-
-The renderer is deterministic: same reports → same output. It never runs a bench. If no reports exist, it writes a placeholder document — never errors out.
-
-## Cache key + invalidation
-
-The Phase 2 cache exists so a daily `task bench:ab` does not re-run the expensive `without` arm when nothing the model would see has changed. The cache key is a tuple:
-
-```
-(corpus_hash, claude_cli_version, target_shape_hash)
-```
-
-A cached `without` report is **fresh** when its recorded `cache_key` matches the current key for every component. Otherwise it is **stale** and the `--reuse-cache` path either refreshes (default in interactive mode) or reuses with a stale flag (`--non-interactive`).
-
-Invalidation triggers:
-
-| Trigger | Component that drifts |
-|---|---|
-| Edit `internal/bench/corpora/ab-tracka.yaml` or `ab-trackb.yaml` | `corpus_hash` |
-| Upgrade the local `claude` CLI | `claude_cli_version` |
-| Add / remove a `WITH_SURFACE` in `scripts/bench_ab_clone.py` | `target_shape_hash` |
-| Edit any file under `internal/bench/ab/fixture/` | `target_shape_hash` |
-
-## How a reader interprets the staleness flag
-
-`docs/benchmark.md` carries a methodology section naming the cache key for the latest run. A reader who suspects the page is stale should:
-
-1. Check the **Last rendered** timestamp at the bottom of the Methodology section.
-2. Inspect `internal/bench/reports/ab/` for newer reports than what the doc reflects.
-3. Re-run `task bench:ab:diff` to re-render from the latest reports (cheap; no bench).
-4. Re-run `task bench:ab` for a full refresh (re-runs the `with` arm, reuses the `without` baseline if the cache is fresh).
-
-## Modes — what the reader can trust
-
-Track B carries an explicit `mode` in every report header:
-
-- `dry-run` — no `claude` CLI invocation; transcripts are stubs. Both variants score 0/N by construction. This is the CI-cheap mode; **do NOT cite Track B numbers in a dry-run report as evidence of package impact**.
-- `live` — real `claude --print` invocation per task. The numbers are real; the cost is real. This is the mode that produces evidence.
-
-The renderer surfaces `mode` prominently in the Track B section. A `dry-run` Track B block is a plumbing-health check, not a measurement.
-
-## Track A is always cheap, always meaningful
-
-Track A measures **surface availability** — does the rule/skill body the prompt would activate exist in the agent's reachable context? This is the precondition for the rule-router to fire. It does not need the `claude` CLI; it is a file-grep over the materialised clone. By construction:
-
-- `without` MUST score 0% — no agent-config surface present → no expected_target file exists.
-- `with` should score close to 100% — every expected_target file should exist with the expected_keywords.
-
-The integrity check `bench_ab_tracka_run.py::integrity_check` fails the run if `without` ever scores non-zero. That's the safety boundary: if `without` scores anything, the variant axis leaked and the bench is invalid.
-
-## Acceptance criteria for a "real" run
-
-A bench run counts as a real measurement (rather than a plumbing health check) when:
-
-- `./scripts-run src/scripts/bench_ab_integrity` exits 0 (the variant axis is clean).
-- Track A runs in BOTH variants and produces `integrity_ok: true` (with scoring close to 100%, without scoring 0%).
-- Track B runs in BOTH variants in `--mode live` (not dry-run).
-- `./scripts-run src/scripts/lint_bench_ab` passes.
-- The rendered `docs/benchmark.md` carries the Headline + Track A + Track B + Methodology + History sections.
-
-## Out of scope for this contract
-
-- The per-report JSON schema details — see `docs/contracts/benchmark-report-schema.md`.
-- Statistical significance / sample-size policy — `--samples N` is opt-in per run; the bench default is n=1 (daily-quick-read goal, not a research-grade study).
-- LLM-judge scoring — explicitly deferred to a follow-up roadmap.
-- Cross-model comparison — out of scope; the bench measures one model (whatever the local `claude` CLI points to).
-
-## See also
-
-- the `road-to-package-impact-benchmark` roadmap (archived) — built this surface.
-- `internal/bench/ab/README.md` — the Shape A vs. Shape B decision and the layout.
-- `docs/contracts/benchmark-report-schema.md` — per-report JSON shape (sibling contract).

package/docs/contracts/benchmark-corpus-spec.md

@@ -1,125 +0,0 @@
----
-stability: beta
-keep-beta-until: 2026-08-14
----
-
-# Benchmark Corpus Spec — step-4 Phase 1
-
-Parser-visible contract for the golden corpus consumed by
-[`scripts/bench_runner.py`](../../src/scripts/bench_runner.py) and the
-upcoming `scripts/lint_bench_corpus.py`. Defines composition, schema,
-and validation invariants.
-
-## Path decision
-
-Roadmap `step-4-measurement-and-benchmark.md`
-Phase 1 Step 2 names `internal/bench/corpus.yaml`. The existing benchmark
-infrastructure (runner + non-dev corpus + `task bench`) lives under
-`tests/eval/` and `scripts/bench_runner.py` hardcodes that directory.
-**Canonical location:** `tests/eval/corpus-<id>.yaml`. The `internal/bench/`
-directory is reserved for **reports + pricing** (Phase 2 deliverables).
-Migration to `internal/bench/corpus.yaml` is a no-op rename if downstream Phase
-2 work proves the consolidation is worth the diff cost.
-
-## Composition (25 prompts)
-
-| Bucket | Count | Purpose |
-|---|---|---|
-| **Routing-canonical** | 10 | One prompt per major skill cluster — exact-match scoring |
-| **Ambiguous** | 8 | Multiple plausible skills — set-intersection ≥ 0.7 scoring |
-| **Destructive / security carve-out** | 5 | Triggers a safety floor — selection must surface the floor skill |
-| **Long-context** | 2 | ≥ 4 k input tokens — exercises retrieval under context pressure |
-
-The 10 routing-canonical prompts MUST cover the kernel + tier-1 skill
-clusters used by the dev profile (`developer.yml`). The 8 ambiguous
-prompts MUST each declare ≥ 2 acceptable skills in `expected_skills`.
-The 5 destructive / security prompts MUST declare an
-`expected_carve_outs` value (e.g. `security-sensitive-stop`,
-`non-destructive-by-default`).
-
-## Schema
-
-```yaml
-version: 1                              # corpus format version (int)
-corpus_id: <id>                         # short kebab-case identifier
-selection_accuracy_target: 0.60         # 0.0–1.0; runner exits non-zero below
-prompts:
-  - id: <bucket>-<NN>                   # e.g. canonical-01, ambiguous-03
-    category: <bucket>                  # canonical | ambiguous | destructive | long-context | router-coverage
-    user_type_candidates: [<slug>, ...] # optional; informational
-    language: en                        # en | de — per language-and-tone
-    prompt: "<text>"                    # the agent-facing prompt
-    expected_skills: [<slug>, ...]      # ≥ 1 entry; non-empty
-    expected_carve_outs: [<slug>, ...]  # required when category == destructive
-    intended_triggers: [<rule_id>, ...] # router-telemetry attribution — rule ids the
-                                        # corpus author expects to activate AND that the
-                                        # deterministic replay can verify (keyword / phrase
-                                        # / command / path with supplied open_files or
-                                        # command context). Replay checks intended vs
-                                        # observed; drift surfaces as a finding, not
-                                        # silently. OPTIONAL on non-router-coverage corpora.
-                                        # (Council R3 honesty floor — pass-3 onwards.)
-    replay_opaque_triggers: [<rule_id>, ...] # rule ids the author expects to fire at
-                                        # RUNTIME but only via an `intent` trigger (or a
-                                        # router coverage gap) the static replay cannot
-                                        # see. Reported separately by the telemetry — NOT
-                                        # counted as missed_intended (would be false drift)
-                                        # nor as unintended_activations. A rule may sit in
-                                        # at most one bucket. router-coverage tasks need at
-                                        # least one of {intended_triggers,
-                                        # replay_opaque_triggers} non-empty.
-    open_files: [<path>, ...]           # optional — paths the agent has "open" when
-                                        # the prompt fires. Used to drive `path_prefix`
-                                        # and `file_pattern` triggers in router replay.
-    command: "<slash-command>"          # optional — exact command invoked (e.g.
-                                        # `/roadmap:process-step`). Drives `command:`
-                                        # triggers in router replay.
-    rubric:                             # optional structural assertion
-      must_include: ["<phrase>", ...]   # all phrases must appear in output
-      must_not_include: ["<phrase>", ...]
-      length_words: { min: 0, max: 0 }
-    quality_assertion: "<regex>"        # optional regex over agent output
-```
-
-### Invariants (lint-bench gate)
-
-| Drift | `reason` | Example |
-|---|---|---|
-| Missing top-level `version` / `corpus_id` / `prompts` | `missing_top_level` | — |
-| `version` not in `{1}` | `unsupported_version` | `version: 2` |
-| `selection_accuracy_target` outside `[0.0, 1.0]` | `target_out_of_range` | `1.5` |
-| Duplicate `id` across prompts | `duplicate_id` | two `canonical-01` |
-| `id` does not match `^[a-z][a-z0-9-]*-\d{2}$` | `bad_id_format` | `Canonical_1` |
-| `category` not in `{canonical, ambiguous, destructive, long-context, router-coverage}` | `bad_category` | `category: misc` |
-| `language` not in `{en, de}` | `bad_language` | `language: fr` |
-| `expected_skills` empty / missing (**except `category == router-coverage`**, which tests trigger activation, not skill selection) | `empty_expected` | `expected_skills: []` |
-| `expected_skills` references an unknown skill slug | `unknown_skill` | `expected_skills: [imaginary]` |
-| `category == destructive` without `expected_carve_outs` | `missing_carve_out` | — |
-| Prompt text empty / whitespace-only | `empty_prompt` | — |
-| `intended_triggers` / `replay_opaque_triggers` references a rule that doesn't exist in `dist/router.json` | `unknown_intended_trigger` | `intended_triggers: [no-such-rule]` |
-| `intended_triggers` present but not a list | `bad_intended_triggers_shape` | `intended_triggers: foo` |
-| `replay_opaque_triggers` present but not a list | `bad_replay_opaque_triggers_shape` | `replay_opaque_triggers: foo` |
-| Same rule id in both `intended_triggers` and `replay_opaque_triggers` | `trigger_in_both_buckets` | `intended:[x]` + `opaque:[x]` |
-| `category == router-coverage` with **both** `intended_triggers` and `replay_opaque_triggers` empty/absent | `missing_intended_triggers` | — |
-
-The linter MUST run with `--quiet` honour per the script-output
-convention and emit one violation per line in non-quiet mode.
-
-## Composition gates (25-prompt-complete state)
-
-Once `corpus-dev.yaml` reaches the 25-prompt target, the linter
-additionally enforces the per-bucket counts above. Until then, the
-linter only enforces per-prompt invariants — partial corpora are
-valid during Phase 1 build-out.
-
-The composition gate is opt-in via `--require-full` to keep the
-reduced 10-prompt suite (Phase 1 Step 4) usable during development
-without tripping CI.
-
-## Cross-references
-
-- Runner — [`scripts/bench_runner.py`](../../src/scripts/bench_runner.py)
-- Linter — `scripts/lint_bench_corpus.py` (Phase 1 Step 3)
-- Existing non-dev corpus — [`tests/eval/corpus-non-dev.yaml`](../../tests/eval/corpus-non-dev.yaml)
-- Language gate — [`language-and-tone`](../../.agent-src.uncondensed/rules/language-and-tone.md)
-- Report schema — `docs/contracts/benchmark-report-schema.md` (Phase 2 Step 4)

package/docs/contracts/benchmark-report-schema.md

@@ -1,113 +0,0 @@
----
-stability: beta
-keep-beta-until: 2026-08-14
----
-
-# Benchmark Report Schema — step-4 Phase 2
-
-Parser-visible contract for the JSON + Markdown reports emitted by
-[`scripts/bench_run.py`](../../src/scripts/bench_run.py). Every `task bench`
-run writes one `internal/bench/reports/<ts>-<corpus_id>.json` + matching `.md`.
-
-## File layout
-
-```
-internal/bench/
-├── pricing.yaml                       # per-1M model rates + sourced_on dates
-└── reports/
-    ├── 2026-05-16T10-30-00Z-dev.json  # machine-readable
-    ├── 2026-05-16T10-30-00Z-dev.md    # human-readable
-    └── ...
-```
-
-Filename format: `<UTC ISO-8601 with `:` → `-`>-<corpus_id>.{json,md}`.
-Sortable lexicographically.
-
-## JSON schema (v1)
-
-```yaml
-schema_version: 1
-generated_at: <ISO-8601 UTC>
-corpus:
-  id: <corpus_id>
-  path: tests/eval/corpus-<id>.yaml
-  prompt_count: <int>
-runner:
-  bench_run_version: <semver>
-  baseline_collector: scripts/bench_runner.py     # selection-accuracy floor
-  baseline_collector_sha: <git-sha-or-mtime>
-selection:
-  top_k: 3
-  prompts_hit: <int>
-  prompts_total: <int>
-  selection_accuracy: <float 0.0-1.0>             # hits / total
-  target: <float>                                 # from corpus
-  passed: <bool>                                  # accuracy >= target
-  per_prompt:                                     # one entry per corpus prompt
-    - id: canonical-01
-      expected_skills: [...]
-      top_k_ranked: [...]
-      hit: <bool>
-cost:
-  source: agents/cost-tracking/sessions.jsonl     # or "unavailable"
-  sessions_scanned: <int>
-  totals:
-    input_tokens: <int>
-    output_tokens: <int>
-    cache_read_input_tokens: <int>
-    cache_creation_input_tokens: <int>
-    total_cost_usd: <float>
-  per_tier:                                        # haiku / sonnet / opus / unknown
-    sonnet: { messages: <int>, cost_usd: <float> }
-    ...
-  pricing_sourced_on: <ISO date from internal/bench/pricing.yaml>
-quality:
-  source: <path-or-"not_collected">
-  prompts_with_assertion: <int>
-  prompts_passing: <int>
-  quality_score: <float 0.0-1.0>                  # passing / total OR 0.0 if not_collected
-  per_prompt:
-    - id: canonical-01
-      assertion: <regex-string>
-      assertion_kind: rubric.must_include | quality_assertion
-      passed: <bool | "not_collected">
-verdict:
-  selection: pass | fail
-  quality: pass | fail | not_collected
-  overall: pass | fail | partial                   # partial = quality not_collected
-```
-
-## Markdown shape
-
-Headers in order:
-
-1. `# Benchmark Report — <corpus_id> · <generated_at>`
-2. `## Headline` — three-line summary (selection · tokens · quality).
-3. `## Selection accuracy` — table per prompt with hit/miss + expected/got.
-4. `## Token usage` — per-tier message counts + token totals; "unavailable"
-   block if no session jsonl was found. The monetary (USD) comparison is
-   **intentionally not rendered** — per-call API pricing misleads
-   subscription users; tokens are the currency-neutral metric that matters.
-5. `## Quality probe` — per-prompt assertion pass/fail; `not_collected`
-   block when no agent-output path was passed.
-6. `## Notes` — `corpus path` and the versioned filename for citation.
-
-## Invariants
-
-- **No silent drops.** Missing token source → emit `source: unavailable`
-  with a marker; never omit the section.
-- **Quality stub honesty.** When agent outputs are not provided, set
-  `quality.source: not_collected` and `verdict.overall: partial`. Score
-  stays `0.0`; never inflate by assuming pass.
-- **Tokens, not money.** The rendered report shows token counts only. The
-  JSON still carries the `cost` block (`total_cost_usd`, per-tier `cost_usd`)
-  for back-compat with downstream consumers, but no USD figure is rendered in
-  the Markdown headline, sections, or footer.
-
-## Cross-references
-
-- Runner — [`scripts/bench_run.py`](../../src/scripts/bench_run.py)
-- Baseline collector — [`scripts/bench_runner.py`](../../src/scripts/bench_runner.py)
-- Corpus contract — [`benchmark-corpus-spec.md`](benchmark-corpus-spec.md)
-- Pricing source — [`internal/bench/pricing.yaml`](../../bench/pricing.yaml)
-- Cost session reader (live sessions) — [`scripts/cost/track.mjs`](../../src/scripts/cost/track.mjs)