@iowarp/clio-coder 0.3.0

package/docs/evidence-and-memory.md

@@ -0,0 +1,243 @@
+# Evidence Corpus and Long-Term Memory
+
+> [!TIP]
+> **Interactive Spec Available:** An interactive memory lifecycle dashboard and simulator is located at [docs/html/memory_blueprint.html](html/memory_blueprint.html) (Version: 0.3.0). Use it to design, validate, and simulate memory proposals, approval loops, pruning rules, and token budgets.
+
+Clio Coder treats run claims and agent lessons as structured artifacts to support reproducibility and scientific provenance. In evaluations such as [SWE-bench](https://www.swebench.com), capturing granular execution evidence is essential for validating agent claims. Evidence corpora are deterministic directories built from run ledgers, receipts, sessions, audits, and eval artifacts. In v0.3.0, forensic evidence auto-builds on dispatch run completion: when a run finalizes, the observability domain automatically compiles the evidence bundle under `<dataDir>/evidence/run-<id>/` and updates a compact sidecar index row in `<stateDir>/evidence-index.json`. Long-term memory records are local, evidence-linked, and only injected after explicit approval. Use the TUI [`/view`](observability.md) command for interactive inspection of receipts, dispatch output, durable tool output, compaction summaries, and session accountability before building or citing evidence.
+
+Source of truth: `src/domains/evidence/**`, `src/domains/memory/**`, `src/cli/evidence.ts`, and `src/cli/memory.ts`.
+
+---
+
+## Evidence CLI
+
+```bash
+clio-coder evidence build --run <runId>
+clio-coder evidence build --session <sessionId>
+clio-coder evidence build --eval <evalId>
+clio-coder evidence inspect <evidenceId>
+clio-coder evidence list
+```
+
+`clio-coder evidence inspect <id>` requires a valid evidence artifact ID. If the requested artifact does not exist on disk, it outputs `error: evidence artifact not found: <id> (see clio-coder evidence list)` and exits with code 1.
+
+Evidence IDs are deterministic:
+
+| Source | ID shape |
+| --- | --- |
+| Run | `run-<runId>` |
+| Session | `session-<sessionId>` |
+| Eval | `eval-<evalId>` |
+
+Rebuilding the same evidence ID rewrites the same directory under `<dataDir>/evidence/`.
+
+
+---
+
+## Evidence directory layout
+
+Run/session evidence files:
+
+```text
+<dataDir>/evidence/<evidenceId>/
+├── overview.json
+├── transcript.md
+├── trace.raw.jsonl
+├── trace.cleaned.jsonl
+├── tool-events.jsonl
+├── audit-linked.jsonl
+├── receipt.json
+├── gate-decisions.json
+├── protected-artifacts.json
+├── findings.json
+└── findings.md
+```
+
+Eval evidence adds `eval-result.json` and uses empty receipt/protected-artifact placeholders when no linked receipts exist.
+
+### Core files
+
+| File | Purpose |
+| --- | --- |
+| `overview.json` | Stable summary: source, runs, sessions, statuses, tasks, models, totals, tags, and file list. |
+| `transcript.md` | Human-readable run/session/eval transcript. |
+| `trace.raw.jsonl` | Raw run ledger/receipt/eval rows. |
+| `trace.cleaned.jsonl` | Compact normalized rows plus findings. |
+| `tool-events.jsonl` | Tool summaries from session entries, audit rows, receipts, or eval commands. |
+| `audit-linked.jsonl` | Audit rows linked to run/session context when available. |
+| `receipt.json` | Receipt bundle (`{ version: 1, receipts: [...] }`); only receipts that pass integrity verification contribute verified fields. |
+| `gate-decisions.json` | Integrity-verified review verdicts, compete winner selections, and winner confirmations discovered from linked receipt ids. |
+| `protected-artifacts.json` | Protected artifact state/events. |
+| `findings.json` / `findings.md` | Structured and readable findings. |
+
+When a run was chained (pipeline), composed with a persona override, or escalated for a permission, `transcript.md` and `trace.cleaned.jsonl` surface the receipt's provenance field sets, and `clio-coder evidence inspect` prints them as a `provenance <runId>:` block. The field paths, types, and stability labels are documented in the [receipt provenance schema](./observability.md#receipt-fields-for-dispatch-provenance).
+
+---
+
+## Evidence Tag Taxonomy and Failure Causes
+
+Clio Coder classifies every run, session, and eval record using a closed set of 25 canonical tags. These tags distinguish general execution characteristics (such as lineage linkages) from actual failure causes.
+
+### Complete Taxonomy
+
+| Tag | Category | Trigger / Meaning |
+| --- | --- | --- |
+| `audit-linked` | Provenance | Audit logs successfully linked to this run or session. |
+| `audit-missing` | Provenance | No matching audit logs were found. |
+| `best-effort-link` | Provenance | Inspection commands or logs linked via heuristics. |
+| `timeout` | Failure | Execution exceeded the maximum duration limit. |
+| `context-overflow` | Constraint | Model context limit was exceeded. |
+| `provider-transient` | Transient | Temporary API or model gateway connection error. |
+| `missing-dependency`| Failure | Python, Node, or system package dependency was missing. |
+| `wrong-runtime` | Configuration | Execution failed due to incorrect compiler or runtime environment. |
+| `proxy-validation` | Validation | Weak validation (e.g. only file-presence check rather than execution). |
+| `no-validation` | Validation | Succeeded turn or run did not execute any verification commands. |
+| `destructive-cleanup`| Precaution | Clean-up rules triggered to prevent workspace pollution or damage. |
+| `blocked-tool` | Failure | The safety net blocked a tool call requested by the model. |
+| `escalation` | Precaution | A worker permission escalation timed out or was denied; see the receipt provenance schema below. |
+| `receipt-integrity` | Security | Forensic verification detected receipt modification or checksum mismatch. |
+| `protected-artifact`| Precaution | Mutating a path protected by project or system safety policies. |
+| `tool-loop` | Constraint | The model repeatedly called the same tool with identical arguments. |
+| `test-failure` | Failure | A verification command containing test/lint keywords exited non-zero. |
+| `build-failure` | Failure | A verification command containing build keywords exited non-zero. |
+| `cwd-missing` | Configuration | The directory target specified for execution did not exist. |
+| `session-linked` | Provenance | The run is linked back to its originating parent session. |
+| `session-missing` | Provenance | No parent session could be resolved for this run. |
+| `auth-failure` | Failure | Missing or invalid credentials/API keys. |
+| `external-bypass` | Security | An external runner bypassed standard safety gates. |
+| `external-approximation`| Validation | An external runner approximated results rather than fully executing. |
+| `unknown` | Undefined | Unclassified execution failure. |
+
+---
+
+### Failure-Cause Tag Subset
+
+A subset of the taxonomy represents actual failure causes (governed by the `FAILURE_CAUSE_TAG_ORDER` array). These are the only tags included in the receipt summaries and TUI observability histograms:
+
+1. **`timeout`**: Triggered if the run outcome is `"timed_out"` or `"stalled"`, or if the error/failure text contains `"timed out"` or `"timeout"`.
+2. **`auth-failure`**: Triggered if failure text contains keywords like `"auth"`, `"api key"`, `"credential"`, or `"unauthorized"`.
+3. **`missing-dependency`**: Triggered if failure logs contain `"module not found"`, `"missing package"`, or `"missing dependency"`.
+4. **`build-failure`**: Triggered in receipt summaries when a non-zero receipt exit is paired with build tool names in `toolStats` (e.g. `build`, `compile`, `make`, `cmake`, `cargo`, `gradle`, `ninja`, `tsc`). Forensic evidence can also classify a non-zero run from build language in the recorded task text.
+5. **`test-failure`**: Triggered in receipt summaries when a non-zero receipt exit is paired with test or lint tool names in `toolStats` (e.g. `pytest`, `ctest`, `jest`, `vitest`, `test`, `lint`, `typecheck`). Forensic evidence can also classify a non-zero run from validation language in the recorded task text.
+6. **`blocked-tool`**: Triggered if tool execution statistics show a blocked count greater than `0`.
+
+---
+
+## Receipt findingsSummary
+
+Each run receipt (persisted under `<stateDir>/receipts/<runId>.json`) carries an optional `findingsSummary` block. This block provides a cheap, integrity-covered summary of the run's findings:
+
+```json
+"findingsSummary": {
+  "tags": ["test-failure"],
+  "firstPassSuccess": false,
+  "findingCount": 1
+}
+```
+
+### Computation and Lifecycle
+- **Circular Dependency Prevention**: To prevent circular dependencies, `findingsSummary` is calculated **cheaply in-memory** at receipt-record time using the draft envelope and tool statistics (in `src/domains/dispatch/receipt-findings.ts`). It never reads from disk or calls `buildEvidence`.
+- **First-Pass Success**: Calculated as `true` only if the terminal outcome was `"succeeded"`, the lineage attempt was `0` (no dispatch retries), the tool stats confirm at least one successful validation tool was executed, and no failure-cause tags were detected.
+- **Cryptographic Coverage**: Current receipts use strict v15 and authenticate every current receipt field, including briefing and steering provenance, routing intent and decision, route quality, worker identity, execution role, and result-contract conformance, against the reconstructed ledger. Every version other than v15 is rejected; there is no historical receipt reader.
+
+| Version | Verification policy | Compatibility policy |
+|---|---|---|
+| v15 | Current canonical projection; every current receipt and reconstructible ledger field is authenticated | Accepted |
+| Any other version | No reader | Rejected; remove or archive the incompatible state rather than expecting migration |
+
+Receipt integrity and evidence verification answer different questions. The
+former proves that a receipt matches its ledger envelope; the latter records
+whether applicable validation evidence was observed. Briefing provenance is
+also distinct from bounded project-context provenance: both can be absent or
+present independently, and neither hash is evidence for the other.
+
+### Mutation-Report Grounding
+
+Mutation-report receipts are grounded directly against observed tool events recorded in the run ledger:
+- When a worker run concludes, claimed modified files are validated against the actual write set observed from `edit` and `write` tool invocations.
+- If a target file was untouched during the run but already existed on disk, the result contract seals route quality as `unmeasured` rather than `fail`.
+- If a write attempt was refused or denied, mutation validation marks the outcome as `unmeasured`.
+- If an unattempted mutation target does not exist on disk, postcondition validation fails.
+
+---
+
+
+## Memory CLI
+
+```bash
+clio-coder memory list
+clio-coder memory propose --from-evidence <evidenceId>
+clio-coder memory approve <memoryId>
+clio-coder memory reject <memoryId>
+clio-coder memory prune --stale
+```
+
+Memory records live in:
+
+```text
+<dataDir>/memory/records.json
+```
+
+The store is capped at `500` records and is sorted by scope, key, creation time, and id for stable writes.
+
+---
+
+## Memory record lifecycle
+
+```mermaid
+stateDiagram-v2
+    evidence --> proposed: propose --from-evidence
+    proposed --> approved: approve <id>
+    proposed --> rejected: reject <id>
+    approved --> rejected: reject <id>
+    proposed --> pruned: prune --stale after 30 days
+    rejected --> pruned: prune --stale after 30 days
+    approved --> pruned: prune --stale after 180 days since lastVerifiedAt/createdAt
+```
+
+Records must cite at least one evidence ID to be considered for prompt injection. Rejected records remain in the store until stale pruning so the same bad lesson is not immediately re-proposed from the same evidence.
+
+---
+
+## Prompt injection rules
+
+The chat loop loads memory synchronously from the bounded local store and calls `buildMemoryPromptSection()`.
+
+Defaults:
+
+| Constraint | Default |
+| --- | --- |
+| Scopes | `global`, `repo` |
+| Token budget | `400` estimated tokens |
+| Max records | `5` |
+| Required status | `approved: true` |
+| Required provenance | At least one `evidenceRefs[]` entry |
+| Suppression | Records with active `regressions[]` entries are skipped |
+
+Rendered memory lines always cite record ID, scope, lesson, and evidence IDs. The prompt tells the model not to extrapolate beyond cited findings.
+
+### Repository-scoped identity
+
+Repository memory is selected by an exact canonical absolute-path identity. The interactive orchestrator and `clio-coder run --agent` compute that identity from the active working directory; symlink aliases collapse to the same key. A repository move, a different Git worktree path, a subdirectory launch, a malformed identity, or a missing identity does not inherit another repository's memory. Global records are unaffected.
+
+Every `scope: "repo"` record must carry:
+
+```json
+"repository": { "kind": "canonical-path", "key": "/canonical/absolute/repository/path" }
+```
+
+The structured `repository` field is the only applicability mechanism: store validation rejects repo records without it, and `appliesWhen` tokens never grant repository applicability. There is intentionally no automatic path rewrite for moved repositories or worktrees: a filesystem move produces a different identity and the record simply stops applying until it is re-scoped with new evidence.
+
+---
+
+## Recommended workflow
+
+1. Build evidence from the run/session/eval that taught the lesson.
+2. Inspect the evidence and findings.
+3. Propose memory from the evidence.
+4. Review the proposed lesson for correctness and scope.
+5. Approve only if it is durable and useful.
+6. Reject incorrect or overbroad records.
+7. Prune stale records periodically.
+
+Memory is meant to reduce repeated mistakes, not to become an unreviewed second instruction system.

package/docs/evolution.md

@@ -0,0 +1,143 @@
+# Evolution and Change Manifests
+
+> [!TIP]
+> **Interactive Spec Available:** An interactive change manifest editor, authority risk assessor, and checklist workspace is located at [docs/html/evolution_blueprint.html](html/evolution_blueprint.html) (Version: 0.3.0).
+
+Clio Coder uses change manifests to make harness changes reviewable, falsifiable, and rollback-friendly. CLIO stands for Context Layer for Input/Output, named for the Greek muse of history. A manifest is JSON, generated or checked with `clio-coder evolve manifest`, and should describe what changed, why, what evidence supports it, what could regress, how to validate it, and how to roll it back.
+
+Source of truth: `src/domains/evolution/manifest.ts`, `src/domains/evolution/validate.ts`, and `src/cli/evolve.ts`.
+
+---
+
+## CLI
+
+```bash
+clio-coder evolve manifest init > change-manifest.json
+clio-coder evolve manifest validate change-manifest.json
+clio-coder evolve manifest summarize change-manifest.json
+```
+
+`init` prints a template. `validate` exits non-zero and reports JSON paths when fields are missing or invalid. In `v0.2.9`, it additionally resolves each non-empty `evidenceRefs` entry (which must match `run-<id>` or `session-<id>` formatting) against the local evidence store, failing validation if a referenced bundle is not found. `summarize` prints the iteration id, base SHA, authority levels, components, files, predicted regressions, validation-step count, and reports evidence refs resolution.
+
+---
+
+## Minimal manifest
+
+```json
+{
+  "version": 1,
+  "iterationId": "exploratory-1",
+  "baseGitSha": "0000000000000000000000000000000000000000",
+  "createdAt": "2026-04-29T00:00:00.000Z",
+  "changes": [
+    {
+      "id": "change-1",
+      "componentIds": ["context-file:CLIO-CODER.md"],
+      "filesChanged": ["CLIO-CODER.md"],
+      "authorityLevel": "prompt",
+      "evidenceRefs": [],
+      "rootCause": "First exploratory iteration; no evidence corpus exists yet.",
+      "targetedFix": "Describe the smallest proposed harness change.",
+      "predictedFixes": ["One expected improvement."],
+      "predictedRegressions": [],
+      "validationPlan": ["npm run test"],
+      "rollbackPlan": "Revert the filesChanged entries for this change.",
+      "expectedBudgetImpact": {
+        "risk": "same"
+      }
+    }
+  ]
+}
+```
+
+Only the first exploratory iteration (`iterationId: "exploratory-1"`) is permitted to use an empty `evidenceRefs` array. For all other iterations, the manifest must cite valid evidence of verification.
+
+### Evidence-Linked Validation (Slice 5a)
+
+During `clio-coder evolve manifest validate` and `summarize` commands, Clio Coder validates the referenced evidence bundles:
+- **Format Verification**: Every reference in the `evidenceRefs` array must follow the format `run-<id>` or `session-<id>`.
+- **Durable Store Resolution**: Each reference must correspond to a folder that actually exists under `<dataDir>/evidence/`. If any referenced bundle is missing, validation fails and reports a dangling reference issue.
+- **Engine Boundaries**: To maintain domain boundaries (`check:boundaries`), the validation function `validateChangeManifest` is completely decoupled. It accepts a `resolveEvidenceRef` predicate option. The CLI passes a resolver connected to the evidence store, keeping the evolution domain from directly importing the evidence domain.
+
+### Self-Edit Gate Deferral (Slice 5b)
+
+Enforcement of high-authority self-edits (Slice 5b) is deferred per the status and design in `src/domains/evolution/SELF_EDIT_GATE.md`.
+
+#### Why it is Deferred
+To enforce that Clio Coder cannot modify its own codebase (specifically paths mapped to `HIGH_AUTHORITY_LEVELS` such as `src/engine/**` or `src/domains/safety/**`) without a validated change manifest, the harness must distinguish between:
+1. **Agent-Initiated Self-Edits**: Clio editing its own harness paths autonomously.
+2. **Operator-Requested Edits**: The developer instructing Clio to modify files in the repository.
+
+Currently, the middleware lacks a reliable signal to separate these two scenarios. Implementing a naive path-glob filter would block developers from making ordinary source changes, which is forbidden by the release specification.
+
+#### Concrete Gate Design and Re-entry
+When the required signal (`selfEditOrigin: "operator-requested" | "agent-initiated"`) is implemented, Slice 5b will run as a `before_tool` middleware registration. For any high-authority tool call classified as `agent-initiated`, it will require a change manifest whose `evidenceRefs` validate against the local evidence store, returning a `block_tool` effect if missing.
+
+
+---
+
+## Schema notes
+
+| Field | Type | Requirement |
+| --- | --- | --- |
+| `version` | literal `1` | Required. |
+| `iterationId` | non-empty string | Required. `exploratory-1` has special empty-evidence handling. |
+| `baseGitSha` | non-empty string | Required. Use the commit the change was based on. |
+| `createdAt` | non-empty string | Required; ISO timestamp recommended. |
+| `changes` | array | Required. |
+| `changes[].id` | non-empty string | Required. |
+| `changes[].componentIds` | string array | Required; can be empty only if `filesChanged` is non-empty. |
+| `changes[].filesChanged` | string array | Required; can be empty only if `componentIds` is non-empty. |
+| `changes[].authorityLevel` | enum | Required; see below. |
+| `changes[].evidenceRefs` | string array | Required; empty only for `exploratory-1`. |
+| `changes[].rootCause` | non-empty string | Required. |
+| `changes[].targetedFix` | non-empty string | Required. |
+| `changes[].predictedFixes` | string array | Required. |
+| `changes[].predictedRegressions` | string array | Required; high-authority changes require at least one entry. |
+| `changes[].validationPlan` | string array | Required. |
+| `changes[].rollbackPlan` | non-empty string | Required. |
+| `changes[].expectedBudgetImpact` | object | Optional; when present, `risk` is required. |
+
+Authority levels:
+
+```text
+prompt | tool-description | tool-implementation | middleware | memory | runtime | safety | schema | cli
+```
+
+High-authority levels are:
+
+```text
+tool-implementation | middleware | runtime | safety | schema | cli
+```
+
+High-authority changes must list predicted regressions. This is deliberate: the point of the manifest is not to promise safety, but to make risk reviewable.
+
+Budget impact risk values:
+
+```text
+lower | same | higher
+```
+
+Optional budget deltas:
+
+```json
+"expectedBudgetImpact": {
+  "tokenDelta": 500,
+  "wallTimeDeltaMs": 2000,
+  "risk": "higher"
+}
+```
+
+---
+
+## Recommended workflow
+
+1. Capture the current commit: `git rev-parse HEAD`.
+2. Run `clio-coder components snapshot --out before.json` if the change affects prompts, tools, runtimes, safety, schemas, or recipes.
+3. Draft the change manifest.
+4. Implement the smallest change set.
+5. Run the validation plan and update `evidenceRefs` with resolved evidence bundle IDs such as `run-<id>` or `session-<id>` where available.
+6. Run `clio-coder evolve manifest validate` and include the manifest in review notes.
+7. If the change fails, use `rollbackPlan` rather than ad hoc cleanup.
+
+Change manifests are especially useful for experimental CLIO work because they separate evidence-backed claims from plans, hypotheses, and future milestones.

package/docs/exit-codes-and-output.md

@@ -0,0 +1,74 @@
+# Exit Codes & Machine-Readable Output Contracts
+
+This document specifies the process exit codes, machine-readable JSON streaming formats, standard I/O separation rules, and `--help` conventions across all Clio Coder CLI commands in `v0.3.0`.
+
+Source implementations: `src/cli/` and `src/entry/`.
+
+---
+
+## 1. Global Exit Codes
+
+Clio Coder follows a deterministic exit code taxonomy across all commands:
+
+| Exit Code | Meaning | Typical Causes & Conditions |
+| :--- | :--- | :--- |
+| **`0`** | **Success** | Successful command execution, clean run settlement, `--version`, `--help` invocation, or missing trace database notice without an explicit `--db` flag. |
+| **`1`** | **Operational Failure** | Execution error, model target unreachable, doctor diagnosis with unresolved issues, explicit `--db` path not found, or evaluation rubric failure (`fail` or `error` verdict). |
+| **`2`** | **Syntax / Usage Error** | Unknown subcommand, invalid flag, missing required positional arguments, global flag placed after subcommand, or data mutation SQL keyword passed to `clio-coder trace sql`. |
+| **`3`** | **Unmeasured / Harness State** | Specific to `clio-coder skills eval`: rubric could not be evaluated due to unparseable judge output, timeout, or evidence archive write failure (distinct from a failure/regression). |
+
+---
+
+## 2. The `--help` Standard
+
+Every subcommand in Clio Coder adheres to the strict `--help` convention:
+
+1. **Standard Output**: Usage instructions and options are printed exclusively to `stdout`.
+2. **Zero Exit**: The process exits with code `0`.
+3. **Zero Side Effects**: Running `clio-coder <subcommand> --help` executes no runtime setup, initiates no network probes, and mutates no state files.
+
+### Global vs Subcommand Flag Positioning
+
+Global options (such as `--cwd`, `--config-dir`, `--state-dir`, `--profile`, and `--debug`) must precede the subcommand. If a global flag is placed after the subcommand name, Clio prints a remediation guide to `stderr` and exits with code `2`:
+
+```text
+--config-dir is a global option and must come before the subcommand: clio-coder --config-dir <path> <command> ...
+```
+
+---
+
+## 3. Standard I/O Separation & Headless Execution
+
+In headless execution (`clio-coder run`):
+
+1. **Standard Output (`stdout`)**: Reserved strictly for the final answer, deliverable artifact content, or machine-readable JSON streams.
+2. **Standard Error (`stderr`)**: Reserved for progress notifications, permission denial advisories, telemetry warnings, and error diagnostics.
+3. **Headless Permission Denials**: When a tool requires permission that cannot be granted in headless mode, Clio emits `HEADLESS_PERMISSION_DENIED_REASON`:
+   ```text
+   clio-coder run cannot confirm permission requests; rerun interactively to approve this action.
+   ```
+   The denial is delivered to the LLM as a tool result so the agent can adapt or report the limitation. If the run completes after the denial, the process exits `0` with the answer on `stdout`.
+
+---
+
+## 4. Machine-Readable Output Formats (`--json` & `--json-events`)
+
+Many Clio CLI subcommands provide structured JSON output for integration with scripts, CI pipelines, and external orchestrators.
+
+### Subcommand JSON Summary
+
+| Subcommand | Flag | Output Structure |
+| :--- | :--- | :--- |
+| `clio-coder run` | `--json` | Stream of incremental NDJSON event frames (`message`, `tool_call`, `tool_result`, `terminal`). |
+| `clio-coder run` | `--json-events terminal` | Filters event stream to emit only the final terminal run receipt. |
+| `clio-coder agents` | `--json` | JSON array of registered agent recipe metadata objects. |
+| `clio-coder targets` | `--json` | JSON object containing the configured `targets` array. |
+| `clio-coder models` | `--json` | JSON array of catalog models with capability flags. |
+| `clio-coder fleet status` | `--json` | JSON snapshot of cluster nodes, active leases, and drain status. |
+| `clio-coder trace runs` | `--json` | JSON array of trace run records. |
+| `clio-coder trace sql` | Positional query | JSON array of rows returned by the read-only SQLite `SELECT` query. |
+| `clio-coder paths` | `--json` | JSON object mapping platform directory names to absolute paths. |
+
+### Incremental Streaming Invariant
+
+The `--json` stream from `clio-coder run` emits **deltas and increments**, never repeated whole-message snapshots. This guarantees that consumers receive stream tokens linearly without duplicating memory or bandwidth.