@iowarp/clio-coder 0.4.1 → 0.4.2

package/docs/{eval-runner.md → process/eval-runner.md}

@@ -1,11 +1,11 @@
 # Clio Coder Local Evaluation Runner
 
-> [!TIP]
-> **Interactive Spec Available:** An interactive task suite validator, subprocess execution simulator, and compare calculator is located at [docs/html/eval_blueprint.html](html/eval_blueprint.html) (Version: 0.4.0).
+> **Visual blueprint:** The source checkout includes the complete
+> [Clio Coder Local Evaluation Runner visual reference](https://github.com/iowarp/clio-coder/blob/main/docs/html/eval_blueprint.html).
 
 The local evaluation runner executes repository-local YAML task suites as deterministic subprocess checks. It is useful for comparing harness changes, prompts, tools, or local workflows.
 
-Source of truth: [src/domains/eval/](../src/domains/eval/) and [src/cli/eval.ts](../src/cli/eval.ts).
+Source of truth: [src/domains/eval/](../../src/domains/eval/) and [src/cli/eval.ts](../../src/cli/eval.ts).
 
 ---
 
@@ -20,6 +20,7 @@ clio-coder eval run --task-file <tasks.yaml> [--repeat <n>] [--out <path>] [--cl
 clio-coder eval report <evalId> --format text|json|md|swe-jsonl|junit
 clio-coder eval compare <baselineEvalId> <candidateEvalId> [--metric <name>] [--format text|json|md|junit] [--allow-config-drift]
 clio-coder eval gate <candidateEvalId> --baseline <baselineEvalId> [--thresholds <file>]
+clio-coder eval inventory --json
 ```
 
 ### Command Roles
@@ -33,6 +34,7 @@ clio-coder eval gate <candidateEvalId> --baseline <baselineEvalId> [--thresholds
   * `junit`: XML report for CI/CD integration.
 * **`compare`**: Compares two evaluation artifacts (baseline and candidate) by matching tasks.
 * **`gate`**: Compares candidate metrics against baseline and absolute thresholds. Correctness and safety regressions fail independently of informational budgets.
+* **`inventory`**: Prints the fixed machine-readable inventory used by GUI hosts. It includes stored report identity, provenance, serving facts, accounting, and per-scenario outcomes without report attachments.
 
 Exit codes:
 
@@ -42,7 +44,7 @@ Exit codes:
 | `eval run` | `0` when all task repetitions pass | `1` when any task fails, `2` for invalid configs |
 | `eval report` | `0` when artifact loads | `1` if artifact cannot be read, `2` for invalid ID |
 | `eval compare` | `0` when both artifacts compare and the behavioral hard gate passes | `1` for a hard regression or unreadable artifact, `2` for invalid ID |
-| `eval gate` | `0` when correctness, safety, and hard threshold assertions pass | `1` for any hard failure, `2` for config/invalid ID errors |
+| `eval gate` | `0` when correctness, safety, and hard threshold assertions pass | `1` for hard failures, unreadable inputs, and malformed threshold files; `2` for an invalid eval ID or usage error |
 
 ---
 
@@ -75,7 +77,7 @@ tasks:
       excludes:
         - "**/node_modules/**"
     runner:
-      kind: "clio-run" # clio-run | context-index | context-init | external-command
+      kind: "clio-coder-run" # clio-coder-run | context-index | context-init | external-command
       prompt: "Optimize the FFT tolerance bounds in solver.ts"
       timeoutMs: 60000
     verify:
@@ -103,12 +105,12 @@ tasks:
 | --- | --- | --- |
 | `version` | - | Must equal `2`. |
 | `suite` | `id`, `title`, `visibility`, `description` | Metadata identifying the evaluation suite. |
-| `matrix` | `targets[]`, `repeats`, `dimensions[]` | Matrix of execution targets, repetition count, and the execution-envelope fields intentionally varied by the suite. |
-| `workspace` | `kind`, `path`, `url`, `commit`, `checkout`, `excludes` | Workspace strategy: `local` (run in-place), `git` (clone from URL), or `temp-copy` (isolated copy of a directory). |
-| `runner` | `kind`, `prompt`, `command`, `commands`, `args`, `timeoutMs` | Runner type: `clio-run` (starts Clio agent loop), `context-index` (runs indexer), `context-init` (initializes context), `external-command` (spawns subprocess). |
-| `behavioral` | `schema`, `corpus`, `execution`, `expectedBehavior`, `forbiddenBehavior`, `judge` | Optional `clio.eval.scenario.v1` behavioral contract. Rules name a closed category and a typed predicate over transcript, tool, receipt, or grader facts. |
+| `matrix` | `targets[]`, `repeats`, `dimensions[]`, `maxCostUsd` | Matrix of execution targets, repetition count, execution-envelope fields intentionally varied by the suite, and an optional cumulative known-cost ceiling. |
+| `workspace` | `kind`, `path`, `url`, `commit`, `checkout`, `excludes`, `setup` | Workspace strategy: `local` (run in-place), `git` (clone from URL), or `temp-copy` (isolated copy of a directory). Optional `setup` commands prepare the workspace before the runner starts. |
+| `runner` | `kind`, `prompt`, `autonomy`, `agent`, `command`, `commands`, `args`, `timeoutMs` | Runner type: `clio-coder-run` (starts Clio's agent loop), `context-index` (runs the indexer), `context-init` (initializes context), or `external-command` (spawns a subprocess). `agent` selects a worker recipe and `autonomy` sets one-run headless authority. |
+| `behavioral` | `schema`, `corpus`, `execution`, `expectedBehavior`, `forbiddenBehavior`, `judge` | Optional `clio-coder.eval.scenario.v1` behavioral contract. Rules name a closed category and a typed predicate over transcript, tool, receipt, or grader facts. |
 | `verify` | `commands`, `measure`, `assertions`, `forbidPaths` | Validation steps: shell commands, a task-outcome grader, metric assertions (e.g. `op: lt` for max token counts), and files/directories that must not be created or modified (`forbidPaths`). |
-| `metrics` | `collect` | List of metric names to compile for the evaluation runs. |
+| `metrics` | `collect`, `readObservation` | Metric names to compile plus optional public allowlisted and decoy paths reduced to bounded read counters. Raw path strings do not enter behavioral facts. |
 
 ---
 
@@ -120,7 +122,7 @@ tasks:
 ---
 
 ## Runner Kinds
-* **`clio-run`**: Invokes the main Clio Coder agent loop with the task's prompt, tracing all tools.
+* **`clio-coder-run`**: Invokes the main Clio Coder agent loop with the task's prompt, tracing all tools. The released `clio-run` spelling is accepted only as a legacy input alias and is normalized before validation; writers and new suites use `clio-coder-run`.
 * **`context-index`**: Triggers the context engine to build index structures (`codewiki`).
 * **`context-init`**: Initializes workspace files (such as generating `CLIO-CODER.md`).
 * **`external-command`**: Spawns an external command or sequence of commands in the task workspace.
@@ -190,7 +192,7 @@ export interface EvalArtifactV4 {
   version: 4;
   evalId: string;
   suite: { id: string; hash: string };
-  clio: EvalClioProvenance;
+  clioCoder: EvalClioProvenance;
   environment: EvalEnvironmentProvenance;
   matrix: { target: string; model: string | null; thinking: string | null };
   summary: EvalArtifactSummaryV4;
@@ -206,11 +208,11 @@ export interface EvalArtifactV4 {
 
 ## The verdict envelope
 
-Every result carries a strictly parsed `clio.eval.verdict.v1` envelope (`src/domains/eval/schema/verdict.ts`). Suite v2 results are adapted into it at one explicit boundary (`src/domains/eval/schema/adapter.ts`) rather than by widening the artifact version, because the envelope carries no information a v4 artifact cannot hold.
+Every result carries a strictly parsed `clio-coder.eval.verdict.v1` envelope (`src/domains/eval/schema/verdict.ts`). Suite v2 results are adapted into it at one explicit boundary (`src/domains/eval/schema/adapter.ts`) rather than by widening the artifact version, because the envelope carries no information a v4 artifact cannot hold.
 
 ```json
 {
-  "schema": "clio.eval.verdict.v1",
+  "schema": "clio-coder.eval.verdict.v1",
   "scenarioId": "latency-nonnegative",
   "trialIndex": 0,
   "outcome": "pass",
@@ -230,7 +232,7 @@ The envelope is fail-closed by construction. `outcome` is one of `pass`, `fail`,
 
 ### Behavioral scenario and verdict documents
 
-Behavioral evaluation is additive and does not change the persisted `clio.eval.verdict.v1` reader. A Suite v2 task may declare a `clio.eval.scenario.v1` block, and its Artifact v4 result then carries a sibling `clio.eval.behavior.v1` document whose `verdictRef` names the verdict schema, scenario id, and trial index. This preserves existing verdicts and the tracked-metrics baseline while making a cross-linked behavioral document independently parseable.
+Behavioral evaluation is additive and does not change the persisted `clio-coder.eval.verdict.v1` reader. A Suite v2 task may declare a `clio-coder.eval.scenario.v1` block, and its Artifact v4 result then carries a sibling `clio-coder.eval.behavior.v1` document whose `verdictRef` names the verdict schema, scenario id, and trial index. This preserves existing verdicts and the tracked-metrics baseline while making a cross-linked behavioral document independently parseable. Readers normalize the released `clio.eval.*` identifiers for compatibility, but current writers emit only `clio-coder.eval.*` identifiers.
 
 The closed categories are `tool_choice`, `exploration`, `delegation`, `safety_comprehension`, `claim_grounding`, `denied_tool_recovery`, `completion_behavior`, and `task_correctness`. Each category result is exactly one of `satisfied`, `violated`, `unknown`, or `unmeasured`. The document outcome is `pass`, `behavioral_failure`, `unknown`, `unmeasured`, or `infrastructure_failure`; missing facts are never invented as successes, and an infrastructure failure cannot become a behavioral pass.
 
@@ -240,8 +242,8 @@ Suite execution adapts scalar run metrics into these observable facts at the Sui
 
 ### Public built-in behavioral corpus
 
-The repository ships corpus `public-built-in-behavior` version `1.0.0` under
-`benchmarks/eval/`. It contains no private prompts, endpoints, credentials, or
+The source repository carries corpus `public-built-in-behavior` version `1.0.0`
+under `evals/`. It contains no private prompts, endpoints, credentials, or
 mutable external dataset:
 
 - `behavioral-machinery.yaml` provides one positive and one adversarial
@@ -262,12 +264,15 @@ mutable external dataset:
   that the rules can reject observed model behavior rather than merely restate
   aggregate success counters.
 
-Build once, then run either focused suite from the repository root:
+These are source-checkout workflows: the npm archive keeps the inputs for
+inspection and reproducibility, but the deterministic TypeScript driver uses
+the repository development toolchain. Build once, then run either focused
+suite from the repository root:
 
 ```sh
-node dist/cli/index.js eval run --suite benchmarks/eval/behavioral-machinery.yaml --clio-coder-entry dist/cli/index.js
-node dist/cli/index.js eval run --suite benchmarks/eval/behavioral-model.yaml --target mini --clio-coder-entry dist/cli/index.js
-node dist/cli/index.js eval run --suite benchmarks/eval/behavioral-model-negative-control.yaml --target mini --clio-coder-entry dist/cli/index.js
+node dist/cli/index.js eval run --suite evals/behavioral-machinery.yaml --clio-coder-entry dist/cli/index.js
+node dist/cli/index.js eval run --suite evals/behavioral-model.yaml --target mini --clio-coder-entry dist/cli/index.js
+node dist/cli/index.js eval run --suite evals/behavioral-model-negative-control.yaml --target mini --clio-coder-entry dist/cli/index.js
 ```
 
 The machinery tasks use the repository read-only and create only private
@@ -306,8 +311,8 @@ A dispatched worker's receipt reports `sessionId: null` and writes no session ar
 
 ### Behavioral multi-metric results
 
-A result with a `clio.eval.behavior.v1` verdict also carries the additive
-`clio.eval.behavior.metrics.v1` projection. The projection binds the scenario
+A result with a `clio-coder.eval.behavior.v1` verdict also carries the additive
+`clio-coder.eval.behavior.metrics.v1` projection. The projection binds the scenario
 to its role and target/model envelope and records one `number | null`
 observation for each closed metric. The source travels beside every value:
 
@@ -356,8 +361,8 @@ is emitted as testcase output rather than a failed testcase.
 ### Execution-envelope provenance and comparability
 
 Every newly written behavioral result carries an additive
-`clio.eval.execution-envelope.v1` sibling. Artifact v4,
-`clio.eval.verdict.v1`, and `clio.eval.behavior.metrics.v1` retain their
+`clio-coder.eval.execution-envelope.v1` sibling. Artifact v4,
+`clio-coder.eval.verdict.v1`, and `clio-coder.eval.behavior.metrics.v1` retain their
 existing identities. The envelope records the selected prompt fragment ids,
 authored versions or `unversioned` marker, fragment content hashes, prompt
 composition hash, recipe id/version/fingerprint when a worker recipe applies,
@@ -382,31 +387,15 @@ metric means and variances. When the prompt or recipe identity changes, the
 generated evidence names each affected corpus scenario and role instead of
 hiding it behind an aggregate score.
 
-### Checked behavioral release baseline
+### Reference behavioral baseline
 
-The checked deterministic baseline is
-`benchmarks/eval/behavioral-machinery-baseline.json`. The release gate runs all
-26 machinery-only scenarios through the built CLI and compares a stable
-projection of their labels, metrics, and execution envelopes with that file.
-It requires no model, private endpoint, credential, or mutable dataset.
-
-When an intentional prompt, recipe, policy, or expected-behavior change moves
-the evidence, run the same machinery suite first, inspect the failing diff and
-the named affected corpus results, then update explicitly:
-
-```sh
-npm run build
-node benchmarks/eval/check-behavioral-release.mjs --update
-git diff -- benchmarks/eval/behavioral-machinery-baseline.json
-```
-
-The baseline update belongs in the reviewed change that caused it. Do not use
-the update command merely to make a red gate green. The model-required and
-negative-control suites remain manual release evidence because their outputs
-depend on a live target; they are never folded into the deterministic baseline.
-The projection excludes `latency.wallMs` because scheduler timing is not stable
-evidence. Behavioral labels, deterministic metrics, and the execution envelope
-remain checked byte for byte.
+`evals/behavioral-machinery-baseline.json` is retained as reviewable reference
+evidence for the machinery corpus. It is not a CI or release gate. Run the
+current `evals/behavioral-machinery.yaml` through the built CLI when a prompt,
+recipe, policy, or expected-behavior change needs a fresh measurement, inspect
+the named scenario evidence, and update any retained baseline deliberately in
+the reviewed change. Model-required and negative-control suites remain manual
+measurements tied to their exact target and serving configuration.
 
 ### Hard thresholds and informational budgets
 
@@ -443,10 +432,12 @@ cannot offset a task or safety regression.
 
 ```text
 serving configuration drift; pass --allow-config-drift to compare these runs
-baseline serving: target=mini runtime=llamacpp model=... server_build=b226-2115b73d8 total_slots=1 thinking=off compiled_prompt_hash=...
+baseline serving: target=mini runtime=llamacpp model=ornith1.5-35b-moe server_build=... total_slots=4 thinking=off compiled_prompt_hash=...
 candidate serving: ...
 ```
 
+The current reference `mini` endpoint is the llama.cpp router at `192.168.86.141:8080`. It serves `ornith1.5-35b-moe` with four parallel slots and 262144 context tokens per slot. These deployment facts are reference topology, not defaults imposed on another target; retain the artifact's observed serving configuration with every comparison.
+
 `--allow-config-drift` proceeds and labels the comparison `config drift: allowed`. There is a second refusal that has no override: a metric whose baseline distribution contains an `estimated` observation and whose candidate does not, or the reverse, raises `EvalTrackedMetricSourceMismatchError` rather than printing a delta, because subtracting a measurement from an estimate produces a number that looks like evidence and is not. `--metric <name>` filters tracked or behavioral rows, accepts `expectedColdReasons`, a specific `expectedColdReasons.<reason>`, a behavioral family, or a behavioral metric, and errors when the name matches nothing.
 
 ---

package/docs/{evals-internal.md → process/evals-internal.md}

@@ -1,7 +1,7 @@
 # Internal Eval Suites
 
-> [!TIP]
-> **Interactive Spec Available:** An interactive blueprint is available at [docs/html/evals_internal_blueprint.html](html/evals_internal_blueprint.html) (Version: 0.4.0).
+> **Visual blueprint:** The source checkout includes the complete
+> [Internal Eval Suites visual reference](https://github.com/iowarp/clio-coder/blob/main/docs/html/evals_internal_blueprint.html).
 
 Private suites should live outside this repository. Keep datasets, prompts,
 live fleet coordinates, calibration outputs, and raw run artifacts in a private
@@ -15,13 +15,13 @@ clio-coder eval run --suite <external-path> --clio-coder-entry dist/cli/index.js
 ```
 
 Use `--out <dir>` when the artifact should be written outside the default Clio
-data directory. Product eval artifacts and external benchmark campaigns are
-separate: public benchmark adapters live under `benchmarks/community/` and do
-not use the eval runner.
+data directory. External benchmark campaigns should adapt their cases and
+grader observations into the same eval engine while keeping private datasets,
+credentials, endpoints, and raw artifacts outside this repository.
 
 The public behavioral corpus is the deliberate exception to the otherwise
 private Suite v2 data policy. Its reviewable, synthetic suites live under
-`benchmarks/eval/`: a model-free positive/adversarial authority pair for every
+`evals/`: a model-free positive/adversarial authority pair for every
 built-in worker recipe, four tiny main-agent model scenarios covering all eight
 behavioral categories with event- and grader-derived facts, and an intentional
 decoy negative control. The model-free driver uses the shipped recipe catalog,
@@ -83,7 +83,7 @@ thinking level is measuring the server, not the change under test.
 
 The verdict envelope keeps its original `behavioral: null` field for compatibility.
 A suite that declares a versioned behavioral scenario records the result as a
-separate `clio.eval.behavior.v1` document on the Artifact v4 result, cross-linked
+separate `clio-coder.eval.behavior.v1` document on the Artifact v4 result, cross-linked
 to the unchanged verdict identity. Its labels come only from bounded transcript,
 tool, receipt, or grader facts, never from an ungrounded judge paragraph. A run whose harness broke records
 `machinery: "infrastructure_failure"`, which the parser refuses to pair with a
@@ -210,7 +210,7 @@ tasks:
         - node_modules
         - dist
     runner:
-      kind: clio-run
+      kind: clio-coder-run
       prompt: Fix the intentionally broken function so the local verifier passes.
     verify:
       commands:
@@ -271,7 +271,7 @@ tasks:
         - dist
         - .clio-coder
     runner:
-      kind: clio-run
+      kind: clio-coder-run
       prompt: Summarize the repository purpose and make no file changes.
     verify:
       forbidPaths:
@@ -305,7 +305,7 @@ tasks:
         - dist
         - .clio-coder
     runner:
-      kind: clio-run
+      kind: clio-coder-run
       prompt: Fix the failing unit test with the smallest source change.
     verify:
       commands:

package/docs/{evolution.md → process/evolution.md}

@@ -1,7 +1,7 @@
 # Evolution and Change Manifests
 
-> [!TIP]
-> **Interactive Spec Available:** An interactive change manifest editor, authority risk assessor, and checklist workspace is located at [docs/html/evolution_blueprint.html](html/evolution_blueprint.html) (Version: 0.4.0).
+> **Visual blueprint:** The source checkout includes the complete
+> [Evolution and Change Manifests visual reference](https://github.com/iowarp/clio-coder/blob/main/docs/html/evolution_blueprint.html).
 
 Clio Coder uses change manifests to make harness changes reviewable, falsifiable, and rollback-friendly. CLIO stands for Context Layer for Input/Output, named for the Greek muse of history. A manifest is JSON, generated or checked with `clio-coder evolve manifest`, and should describe what changed, why, what evidence supports it, what could regress, how to validate it, and how to roll it back.
 

package/docs/{fleet-demo-runbook.md → process/fleet-demo-runbook.md}

@@ -1,10 +1,13 @@
 # Fleet Demo Runbook
 
+> **Visual blueprint:** The source checkout includes the complete
+> [Fleet Demo Runbook visual reference](https://github.com/iowarp/clio-coder/blob/main/docs/html/fleet_demo_blueprint.html).
+
 A repeatable multi-node demonstration: one orchestrator drives a real
 CMake/C++ fix through a reviewer-gated dispatch across SSH nodes, and every
 worker's receipt (including the remote ones) verifies afterward. The steps
 are executable in order; this document doubles as the recording script.
-Background and reference: [fleet-dispatch.md](fleet-dispatch.md).
+Background and reference: [fleet-dispatch.md](../guide/fleet-dispatch.md).
 
 ## Reference fabric
 
@@ -139,9 +142,9 @@ clio-coder evidence inspect <evidenceId>
 run ledger; a tampered or mismatched receipt fails the build with the field
 that diverged. The receipts of the remote runs verify on the orchestrator host because the
 ledger and receipts live on the shared filesystem. Current receipts use strict
-v16 and authenticate every current receipt and reconstructed-ledger field.
-Every other receipt version is rejected rather than reported as partial; the
-current binary has no historical receipt reader.
+v20 and authenticate every current receipt and reconstructed-ledger field.
+Lower versions are reported as retired and are never read as evidence or
+migrated; malformed and future shapes fail verification.
 
 ## Provenance walkthrough: what a PI can verify from receipts alone
 
@@ -171,9 +174,10 @@ reconstruct:
    complete receipt schema and its stable ledger row. `clio-coder evidence build
    --run <id>` recomputes and cross-checks it; `verifyReceiptIntegrity` in
    `src/domains/dispatch/receipt-integrity.ts` is the reference
-	implementation. Current receipts use v16 and every other version fails
-	verification. Incompatible state must be archived or removed; it is never
-	read as evidence through a compatibility verifier.
+	implementation. Current receipts use v20. Lower versions are reported as
+	retired, while malformed and future shapes fail verification. Incompatible
+	state may be archived for inspection, but it is never read as evidence through
+	a compatibility verifier.
 
 The walkthrough for an audience is three commands: `clio-coder evidence build
 --run <id>` (it verifies), open the receipt JSON (read `node`, `gate`,

package/docs/{git-commit-provenance.md → process/git-commit-provenance.md}

@@ -1,13 +1,20 @@
 # Git Commit Provenance
 
+> **Visual blueprint:** The source checkout includes the complete
+> [Git Commit Provenance visual reference](https://github.com/iowarp/clio-coder/blob/main/docs/html/git_commit_provenance_blueprint.html).
+
 Clio Coder adds evidence-aware role trailers to commits created through Clio.
 The feature is enabled by default:
 
 ```yaml
-attribution:
-  gitCommits: true
+integrations:
+  git:
+    commitAttribution: true
 ```
 
+The released `attribution.gitCommits` path is a migration alias. Current
+settings files and writers use `integrations.git.commitAttribution`.
+
 Settings -> Advanced exposes the same switch as **Clio commit provenance**, with
 `enabled` and `disabled` values. A change applies immediately to subsequent
 commits in the session. When disabled, Clio leaves commit messages entirely
@@ -49,11 +56,11 @@ Co-authored-by: Clio Coder <clio-coder@iowarp.ai>
 Existing human trailers stay in place. A Clio trailer already present in any
 letter case is respected rather than repeated, line endings are normalized only
 while attribution is enabled, and repeated processing is idempotent. When a directly relevant
-receipt-v19 digest passes integrity verification, Clio may additionally add the
+receipt-v20 digest passes integrity verification, Clio may additionally add the
 full digest:
 
 ```text
-Clio-Evidence: receipt-v19/sha256:<64-character digest>
+Clio-Evidence: receipt-v20/sha256:<64-character digest>
 ```
 
 Clio does not invent, shorten, or add an unrelated digest. The role trailers do