@iowarp/clio-coder 0.3.0

package/docs/context-engine.md

@@ -0,0 +1,236 @@
+# Context Engine
+
+> [!TIP]
+> **Interactive Spec Available:** An interactive dashboard is located at [docs/html/context_blueprint.html](html/context_blueprint.html) (Version: 0.3.0).
+
+Clio Coder tracks context pressure, records per-turn snapshots, and protects the provider context with bounded tool results plus single-threshold compaction.
+
+Source of truth lives in `src/domains/session/context-accounting.ts`, `src/domains/session/context-ledger.ts`, `src/domains/session/compaction/`, `src/domains/session/migrations/index.ts`, and the chat-loop integration in `src/interactive/chat-loop.ts`.
+
+## Context window resolution
+
+Each target has a declared, desired, and effective context window. The effective window is the operating ceiling used by budget checks and compaction. It can come from a live loaded model config, a probe, a target override, a model hint, catalog knowledge, a local-native default, or a descriptor default.
+
+Local-native runtimes use a recommended minimum desired window of 128,000 tokens. If the live model reports a smaller loaded context window, Clio re-resolves the target so accounting uses the actual ceiling.
+
+## Token accounting and snapshots
+
+The estimator in `context-accounting.ts` uses a four-characters-per-token family for hot-path accounting. It estimates system prompt, tools, messages, pending input, and runtime categories without calling a model tokenizer on every TUI refresh.
+
+At submit time, Clio captures a context snapshot and persists a slim JSONL record under the session directory as `context-snapshots.jsonl`. The slim record keeps token counts, segment metadata, signatures, and hashes, not the heavy prompt or transcript text. When provider usage arrives, `reconcileSnapshot` folds actual input and output counts back into the ledger.
+
+Session metadata enforces session format version 3 (`CURRENT_SESSION_FORMAT_VERSION = 3`). Before resuming any session, Clio checks `sessionFormatVersion`; earlier formats are rejected outright with an error rather than silently migrated.
+
+The `/context` overlay and footer meter read the same ledger categories: `system`, `tools`, `agents`, `skills`, `memory`, `project`, `messages`, `pending`, `reserve`, `free`, and `streaming`.
+
+## Single-threshold compaction
+
+Auto-compaction is controlled by one pressure threshold. Pressure is `estimated_tokens / context_window`. The default threshold is `0.8`.
+
+When `compaction.auto` is enabled and pressure crosses the threshold before a request, Clio first masks stale tool observations and stale thinking older than `excludeLastTurns`. This is a cheap local rewrite. Tool call and result structure remain present, but the observation body is replaced with a marker and stale assistant thinking content is dropped from replay.
+
+Marker format:
+
+```text
+[Observation masked: <tool> output was <lines> lines, <chars> chars - contents masked to save context. Re-run the tool for current content.] Preview: <preview>
+```
+
+Already-compacted entries are not masked again. Recent turns keep their full observations and thinking. If masking drops pressure below the threshold, Clio sends the request without an LLM summary. If pressure remains above the threshold, Clio runs the summary compaction path, appends a compaction summary entry, refreshes replay messages from the session, and continues.
+
+Manual `/context compact`, `CLIO_CODER_FORCE_COMPACT=1`, and overflow recovery force the LLM summary path directly. The overflow guard runs before the user turn is committed, so a blocked oversized request does not leave an unanswered user entry in the ledger.
+
+## Cache-divergence honesty
+
+Compaction rewrites the replayed history. On a local backend with a single prefix-cache slot, the next turn after compaction is expected to be cold because the byte prefix changed. Dispatch traffic can disturb the same slot.
+
+Clio records these disturbances once on the next assistant entry as `promptCache.expectedColdReasons`. The user sees one dim notice, and the same reasons persist on that entry in the session ledger next to the per-call cache data.
+
+Per-call cache verdicts are `hot`, `partial`, `cold`, and `small`. They are derived from provider usage and persisted with `timing { ttftMs, apiMs }` and `promptCache { input, cacheRead, cacheWrite, backendVerdict }` when available.
+
+## Settings
+
+The public settings block has one threshold and one recent-turn horizon:
+
+```yaml
+compaction:
+  auto: true
+  threshold: 0.8
+  excludeLastTurns: 6
+  # model: provider/summary-model-id
+  # systemPrompt: ~/.config/clio-coder/prompts/compaction.md
+```
+
+`auto` controls the pre-request trigger. Manual `/context compact` still runs when `auto` is false. `model` optionally selects a dedicated summarization model. `systemPrompt` optionally points at a prompt override file for compaction.
+
+Settings validation is strict: an older file still carrying the removed `compaction.thresholds` block fails to load with the exact key path during normal startup. Edit removed or unknown keys deliberately; `clio-coder doctor --fix` does not transform settings into the current schema.
+
+---
+
+## Project-context preload class
+
+The compiled session prompt preloads the full rendered project context (the `CLIO-CODER.md` fragment plus project-type and codewiki markers) only when a parseable `CLIO-CODER.md` exists and the rendered text stays within 8000 characters and 220 lines; otherwise it preloads a compact synopsis. The rule lives in `src/domains/prompts/preload.ts` and every reporting surface classifies with it:
+
+- `/context init` and `clio-coder context init` print `preload: full (N.NkB, N lines)` or `preload: synopsis (reason: size|lines)` after the summary, and warn when a full preload is within 10% of either limit.
+- `clio-coder config inspect` shows the preload class in the `CLIO-CODER.md` entry's detail.
+- The `/context` overlay shows a `project preload:` line under the category legend once a session prompt has compiled.
+
+## Context refresh
+
+`/context refresh` and `clio-coder context refresh` rebuild the structural codewiki
+and restamp `.clio-coder/state.json` without reading or writing `CLIO-CODER.md`. The CLI
+flag `--wiki` is the only refresh path that may update the Markdown wiki, and
+it only runs when an existing wiki metadata file is present. Regenerating or
+updating handbook prose stays with `/context init`.
+
+`clio-coder context init` is model-driven by default. The `--heuristic` flag is the sole deterministic flag for offline handbook generation. The `--propose` flag writes ignored drafts to `.clio-coder/proposals/`, `--apply` updates from the existing handbook, and `--rewrite` generates a fresh handbook.
+
+When bootstrapping across local runtimes such as `llamacpp` where strict grammar/schema enforcement might be rejected by the endpoint, generator logic retries automatically using a bounded prompt-parser fallback. If `--rewrite` was requested but the model generation fails to produce a valid handbook rewrite, `clio-coder context init` prints a notice and exits with code 1 rather than leaving an inconsistent state.
+
+
+---
+
+## Codewiki and Wiki
+
+Project context has two local layers. The structural layer is model-free and
+feeds navigation. The Markdown wiki layer is agent-authored and exists only
+when the operator explicitly asks for it.
+
+| Layer | Artifact | Producer | Model use | Prompt surfacing |
+| --- | --- | --- | --- | --- |
+| Structural codewiki | `.clio-coder/codewiki.json` plus `.clio-coder/state.json` | `context init`, `context refresh`, `context index`, session freshness checks, and incremental mutation observers | None | `<codewiki>available...; use code_nav</codewiki>` |
+| Markdown wiki | `.clio-coder/wiki/**/*.md` plus `.clio-coder/wiki/meta.json` | `clio-coder context wiki` or `clio-coder context refresh --wiki` | Yes, one planning dispatch plus one dispatch per page | `<wiki>N pages at .clio-coder/wiki (start: quickstart.md)...</wiki>` |
+
+### Structural Index
+
+`.clio-coder/codewiki.json` uses schema v5 and is written as compact JSON. File
+records contain a stable id, path, language, line count, role, per-file content
+hash, extracted import specifiers, and an optional first docstring/JSDoc
+summary. Symbol records store declaration-level symbols only (such as classes, interfaces, types, global functions, and methods) and intentionally skip function-local symbols. Each record stores name, kind, file id, line, and optional signature. Edges are built from imports and record either an internal file id target or an external module string.
+
+In Git workspaces, the indexer uses the same visible file set across full builds,
+incremental updates, fingerprints, and project profiles: tracked files plus
+untracked, unignored work in progress. It excludes symlinks, submodule gitlinks,
+generated output, scratch space, and local-state directories such as `.git`,
+`.clio-coder`, `.superpowers`, `.codex`, `.claude`, `.clio-coder-benchmark`, `node_modules`,
+`dist`, `build`, `coverage`, virtualenvs, `target`, and `vendor`. Non-Git
+workspaces use a bounded filesystem walk with the same directory exclusions.
+Source coverage spans TypeScript, JavaScript, Python, Rust, Go, C, C++, CUDA
+(`.cu` and `.cuh`), Java, Ruby, and C#, with config entries for manifests such
+as `package.json`, `pyproject.toml`, `Cargo.toml`, `go.mod`, `pom.xml`,
+`CMakeLists.txt`, `Gemfile`, and `*.csproj`.
+
+Extraction is async and tree-sitter-first. Clio loads WASM grammars for the ten
+source language grammars above, extracts symbols/imports/exports from the parsed tree,
+and merges regex import extraction for languages with regex extractors. If a
+tree-sitter parse fails for one file, that file falls back to the available
+regex extractor instead of aborting the whole build. C# is covered by the
+tree-sitter C# grammar.
+
+Ambiguous `.h` files are classified deterministically by `classifyCHeaderLanguage` (`src/core/c-header-language.ts`). The scanner removes comments before checking C++-only standard-library includes, and removes comments and literals before checking C++ syntax markers such as templates, namespaces, class/member declarations, scope resolution, C++ casts, and C++ qualifiers. If any C++ marker is present, the header is indexed as `c++`; otherwise, it defaults to `c`. This guarantees that both full indexing and incremental file syncs assign identical language tags to `.h` files. Declaration-only C/C++ APIs are indexed so header-heavy MPI, CUDA, and scientific libraries remain navigable even when implementations live elsewhere.
+
+Incremental updates are real updates, not a full rebuild hidden behind the
+name. Successful file-mutating tools report changed paths through the
+middleware observer. The context domain coalesces those paths, checks per-file
+content hashes and reparses only changed files (`perf(context)` optimization),
+reads only the changed indexable files for path-based updates, replaces their file and symbol records, removes
+deleted records, and rebuilds edges from the merged import set. Non-indexable
+paths are no-ops.
+
+### Markdown Wiki & `code_nav` Resolution
+
+The wiki lives under `.clio-coder/wiki/` as a nested tree and is written by the
+`wiki-writer` agent. Model agents resolve pages dynamically through `code_nav`
+with `mode: "wiki"`; an optional query resolves a page id or title, where the id
+is the page's path without its extension (`domains/dispatch`), and returns its
+summary and path. That gives deterministic on-demand navigation without loading
+whole pages into prompt context.
+
+The unit of work is one page, not one wiki. A run makes a single planning
+dispatch, then one dispatch per page, each with a fresh context holding only that
+page's plan entry, its anchor sources, and the sibling paths it may link to. The
+repository-wide payload, including the codewiki digest, appears only in the
+planning prompt. Because a static prompt is re-sent on every round of a run, this
+is what keeps prefill cost from growing quadratically with the size of the wiki.
+
+`_plan.json` is the skeleton and the checkpoint. It is derived deterministically
+from the codewiki index, so a usable plan exists before any model runs; the
+planning dispatch may merge, split, rename, drop, or re-anchor entries by
+rewriting it, and a malformed rewrite falls back to the candidate. The harness
+owns each entry's status and rewrites the file after every page, so a run that
+ends early records exactly which pages are still owed. Staging survives such a
+run and the next one resumes from it.
+
+Every page opens with front matter carrying `title`, `summary`, `sources`,
+`symbols`, `tests`, `invariants`, and `validate`. That is the retrieval layer:
+`quickstart.md`, every directory `index.md`, and the task-routing table are
+generated from it after each run, so navigation cannot drift or miss a page and
+no writer has to remember to update it.
+
+Assembly repairs rather than rejects. A missing H1, absent or malformed front
+matter, a dangling `sources` entry, a link to a page that was never written, and
+a citation to a path that does not exist are all mechanically fixable, so each is
+fixed or recorded in a `<!-- clio:wiki ... -->` marker and reported; none fails a
+run. An empty page is dropped and its plan entry stays owed. An update run's
+scope is computed, not guessed: a page is rewritten when git reports a change to
+one of the sources its own front matter claims.
+
+`meta.json` records `updatedAt`, `gitHead`, the indexed source-tree hash, the
+model label, a content hash over the page tree, the page list, and the plan.
+`generation.pagesPlanned` and `generation.pagesWritten` say whether a run
+finished; when they differ, `clio-coder context wiki --update` completes the rest.
+
+`clio-coder context wiki` creates a wiki when no metadata exists and updates one when metadata is present. During wiki generation, Clio automatically refreshes a stale codewiki index before grounding the model run. The decision to write new pages and update metadata is a no-op if the newly generated content's hash matches the existing content hash. `clio-coder context wiki --update` requests update mode explicitly. `clio-coder context wiki --status` only reads metadata and does not run a model. `clio-coder context refresh --wiki` first rebuilds the structural codewiki and then updates an existing wiki when `.clio-coder/wiki/meta.json` exists; when no wiki metadata exists, it performs no wiki generation.
+
+### Lifecycle Matrix
+
+| Event | Structural codewiki behavior | Markdown wiki behavior |
+| --- | --- | --- |
+| Session start | If state or `.clio-coder/codewiki.json` already exists, Clio checks freshness best-effort and performs a full rebuild when the index is stale, missing, unreadable, or needs v5 backfill. Never-indexed directories are skipped. | No generation or update. Existing wiki status may surface in the welcome dashboard. |
+| In-session edits | Successful file mutations enqueue changed paths for incremental `updateCodewikiPaths`; the queue is serialized and best-effort. | No automatic update. |
+| Session stop | Drains the incremental queue, then rebuilds only when state is stale, the index is missing, or v5 backfill is needed. State records `lastSessionAt`, `lastIndexedAt` when applicable, and `codewikiVersion`. | No automatic update. |
+| `/context init` or `clio-coder context init` | Performs a full codewiki rebuild before generating, preserving, proposing, or previewing `CLIO-CODER.md`; writes state with the fingerprint and codewiki version when it writes state. | No wiki generation. |
+| `/context refresh` or `clio-coder context refresh` | Performs a full codewiki rebuild and writes state. Does not touch `CLIO-CODER.md`. | If an existing wiki is stale and `--wiki` was not passed on the CLI, prints a hint to run `clio-coder context refresh --wiki` or `clio-coder context wiki --update`. |
+| `clio-coder context refresh --wiki` | Performs the same full codewiki rebuild and state write. | Updates an existing wiki through the model-backed page dispatches. No wiki metadata means no wiki model call. |
+| `clio-coder context wiki` | Automatically refreshes the codewiki index if stale before composing the wiki prompt. | Plans, then writes each owed page in its own dispatch, assembles and promotes whatever landed (no-op if content hashes match), and records any pages still owed. |
+| `clio-coder context wiki --status` | No index rebuild. | Reads metadata and reports page count, update time, recorded git head, git-head drift, and how many planned pages remain unwritten. |
+
+### Staleness
+
+Codewiki staleness is controlled by one predicate:
+`isStale(prev, curr)` compares only `fingerprint.treeHash`. The fingerprint
+hash is mtime-aware: it walks the repository, excludes generated/local-state
+directories and lock/archive files, and hashes each included relative path,
+file size, and floored `mtimeMs`. The fingerprint also records `gitHead` and
+`loc`; `loc` comes from the codewiki artifact when available, otherwise from a
+line count over source extensions. Those fields are reporting data, not the
+stale predicate.
+
+`.clio-coder/state.json` stores the fingerprint and optional `codewikiVersion`.
+Legacy v2/v3/v4 codewiki files can still be read as degraded v5 artifacts, but
+their missing or deliberately invalidated per-file hashes make `codewikiNeedsBackfill` true. The
+next session freshness check, `code_nav` demand load, wiki generation, or
+explicit refresh rebuilds them into full v5.
+
+Wiki staleness is separate. New `.clio-coder/wiki/meta.json` files record both the git
+head and the indexed source-tree hash used when the wiki content last changed.
+Clio reports drift at the same git head when tracked or untracked source files
+change, and combines committed and working-tree evidence in its changed-file
+count. Older metadata without a source-tree hash retains the git-head-only
+check. Git-less or unreadable git states degrade to `fresh` with a warning when
+Clio cannot prove drift.
+
+### Surfacing and Navigation
+
+The compiled prompt surfaces only markers, never the codewiki JSON or wiki page
+contents. Fresh codewiki renders as `<codewiki>available; use code_nav</codewiki>`.
+A stale codewiki marker adds `(stale; run /context refresh)`. A valid wiki marker
+names the page count and `quickstart.md`; a stale wiki marker adds `(stale; run
+clio-coder context wiki --update)`.
+
+`clio-coder context` prints a structural digest from `renderCodewikiDigest`: schema
+version, project language, file/config/symbol/edge counts, language and role
+counts, top areas, entry points, key symbols, and dependency samples. The
+welcome dashboard shows module count, wiki page count and freshness, and a
+small entry-point excerpt from the same digest. Agents query the structural
+layer through the read-only `code_nav` tool. See [tool-usage.md](tool-usage.md)
+for the full mode reference.

package/docs/dispatch-architecture-rationale.md

@@ -0,0 +1,126 @@
+# Dispatch Architecture Rationale
+
+Why `src/domains/dispatch/` is one domain, why one import out of it looks
+irregular and is allowed to, and why the repository has no barrel-only import
+convention. No code moved as a result of this document. It exists so that a
+later split is argued from invariants rather than from file counts.
+
+Counts verified against the current tree: 65 TypeScript files in
+`src/domains/dispatch/`, a 137-line barrel at `src/domains/dispatch/index.ts`,
+and one dispatch → eval import.
+
+---
+
+## Size does not argue for a split
+
+A five-way split by responsibility label is the obvious proposal and the wrong
+one. The invariants in this domain are not partitioned by the labels such a
+split would use. They cross them.
+
+### Invariants that cross the proposed seams
+
+| Invariant | Crosses | Evidence |
+| --- | --- | --- |
+| A retry is `recovery`, and rebinds its reservation to the node and cost bound it actually resolved | routing, admission, retries, receipts | `execution-role.ts`, `capacity-lease.ts`, `routing-intent.ts` |
+| A plan slot belongs to an assignment, never an attempt, so a retry never queues behind itself | admission, scheduling, retries | `admission-queue.ts` |
+| Whole-plan preflight and reservation happen before any spawn | scheduling, admission, write boundaries | `execution-scheduler.ts` |
+| Route history keys on capability, and a receipt's `quality` block must be run-local | routing, receipts, quality | `route-history.ts`, `route-quality.ts` |
+| Write-boundary attribution is per scheduling *window*, so the compiler refuses a wave with two writers | scheduling, write boundaries, plan compilation | `execution-plan.ts`, `write-boundary.ts` |
+| A loop's later nodes are `unneeded`, decided by the scheduler, not the plan | plan compilation, scheduling, receipts | `fleet-plan.ts`, `execution-scheduler.ts` |
+| Staleness revalidation re-runs a verification a later workspace step invalidated | scheduling, plan compilation, code steps | `execution-scheduler.ts` |
+| Receipt integrity v15 seals normalized routing intent | routing, receipts | `receipt-integrity.ts`, `routing-intent.ts` |
+
+The write-boundary and loop rows are the sharpest. Both are properties of a
+*wave*, which is a scheduling concept computed by the plan compiler and enforced
+by the scheduler. A split putting plan compilation and scheduling in different
+modules puts the two halves of one invariant on opposite sides of a module
+boundary, where nothing but convention keeps them agreeing.
+
+### What a split would have to preserve
+
+Any future split must carry these forward. A split proposal that does not
+address every one of them is not ready:
+
+1. **One hashed plan.** `compileExecutionPlan` produces one deterministic hashed
+   DAG including unrolled loops. Wave computation, boundary-attribution refusal,
+   and authority grants are all decided there. Splitting compilation from
+   scheduling requires the wave contract to become an explicit, versioned
+   interface rather than an in-process assumption.
+2. **Admission is serialized by one cross-process lock.** Lease acquisition,
+   retry rebinding, heartbeat, drain, and reservation transfer are one critical
+   section. A split that puts any of them behind a separate module's API must
+   not introduce a second lock or a lock-free path.
+3. **Attempt identity.** Assignment id and terminal run id are distinct, and
+   role derivation (`recovery` for every attempt after the first) is read by
+   routing, receipts, history, and the ledger. This is a shared vocabulary, not a
+   routing detail.
+4. **Fail-closed defaults.** No-ready-candidate, manual pins, `failover: none`,
+   missing authority grants, and unverifiable boundaries all refuse. A split must
+   not create a module whose default answer is permissive.
+5. **Receipt sealing is the authority.** The orchestrator's validation, not the
+   worker's, decides conformance. Any split must keep sealing on the
+   orchestrator side of the new seam.
+
+### Recommendation
+
+Do not split cross-domain, and do not split `dispatch` on responsibility labels.
+Internal cohesion is available and safe: extracting a pure reducer or a
+single-owner helper into another file *within* `src/domains/dispatch` costs
+nothing and needs only the existing tests. `route-quality.ts` and
+`routing-intent.ts` are already this shape and are the model to follow.
+
+A genuine split, if it is ever wanted, should be argued from the wave contract
+outward, because that is the one seam the invariants above actually respect.
+
+---
+
+## Dispatch reads an eval parser the eval barrel does not export
+
+`src/domains/dispatch/route-observer.ts` imports `parseEvalArtifactV4` from
+`../eval/artifacts/store.js`. The eval barrel does not export it. This is the
+only dispatch → eval import in the domain.
+
+This is coupling worth recording, not a violation. It breaks none of the five
+enforced boundary rules, and the direction is defensible: the routing quality
+reducer treats an eval artifact as evidence, so it must parse one, and
+`parseEvalArtifactV4` is the strict fail-closed parser rather than a convenience
+reader. Routing quality reading eval evidence through the artifact's own
+validating parser is better than routing quality inventing a second reader that
+could accept an artifact the eval domain would reject.
+
+Widening the eval barrel to export it would be a public-surface change made only
+to satisfy import form, which is exactly what the barrel decision below rejects.
+If the coupling is ever to be removed, the honest fix is for the eval domain to
+own a narrow "read an artifact as routing evidence" function and export that,
+which is a design change needing its own justification.
+
+---
+
+## The repository has no barrel-only import convention
+
+Direct subpath imports are permitted. This is a decision, not a postponement.
+
+The evidence that decides it:
+
+- Measured across `src/domains/**`, counting an import as cross-domain when the
+  importing file and the resolved target sit in different `src/domains/<name>`
+  directories: **110** cross-domain subpath imports against **29** cross-domain
+  barrel imports. Direct subpath import is the majority pattern by roughly four
+  to one, not an exception to a rule.
+- All five enforced boundary rules
+  (`tests/boundaries/check-boundaries.ts`) constrain dependency **direction**:
+  who may depend on whom. Not one constrains import **form**. There is no rule
+  to be half-consistent with.
+- `src/domains/dispatch/execution-plan.ts` imports `AgentAutomationAuthority`
+  from `../agents/spec.js`, and `src/domains/agents/index.ts` does not export
+  it. A barrel-only rule would have to widen the agents barrel for no reason but
+  import style.
+
+A barrel-only sixth rule would require widening many barrels to re-export
+symbols currently reached directly. Every one of those is a public-surface
+addition justified by nothing but import style, and it would rewrite every
+affected import site for no behavioral gain. A boundary rule should protect an
+invariant. "Always import through the barrel" protects a preference.
+
+What is *not* permitted is anything the five direction rules forbid, and those
+stay enforced by `npm run check:boundaries`.

package/docs/documentation-coverage.md

@@ -0,0 +1,46 @@
+# Clio Coder Documentation Coverage Matrix
+
+This matrix maps every top-level directory in `src/` and every domain directory under `src/domains/` to its authoritative documentation page. It records coverage status (`documented`, `partial`, `undocumented`), missing concepts, and key source contracts for `v0.3.0`.
+
+## Coverage Matrix
+
+| Source Area | Primary Subsystems / Modules | Owning Documentation | Status | Coverage Details & Gap Analysis |
+| :--- | :--- | :--- | :--- | :--- |
+| `src/cli/` | Command line routing, subcommands, flag parsing, exit codes, machine-readable JSON streaming | [commands-and-modes.md](commands-and-modes.md), [exit-codes-and-output.md](exit-codes-and-output.md) | `documented` | Covered by CLI reference and the dedicated exit codes and machine-readable output contract guide. |
+| `src/core/` | Invariants, constants, configuration defaults, headless permissions, tool definitions, response schema contracts | [architecture.md](architecture.md), [configuration-and-targets.md](configuration-and-targets.md), [installation-and-lifecycle.md](installation-and-lifecycle.md), [safety-model.md](safety-model.md) | `documented` | Fully documented across architecture, configuration, lifecycle, and safety model pages. |
+| `src/engine/` (Core) | Engine turn loop, prompt priming, streaming message adapters, turn execution | [architecture.md](architecture.md), [context-engine.md](context-engine.md) | `documented` | Documented across architecture and context engine guides. |
+| `src/engine/acp/` | ACP protocol server, transport adapters, tool mediators, permission forwarding, error taxonomy | [acp.md](acp.md) | `documented` | Dedicated ACP specification covering server wiring, permission mediation, timeouts, error taxonomy, and security boundaries. |
+| `src/entry/` | Application bootstrapping, CLI router, interactive loop entry point | [architecture.md](architecture.md), [installation-and-lifecycle.md](installation-and-lifecycle.md) | `documented` | Documented in architecture compilation boundaries and lifecycle guides. |
+| `src/interactive/` | TUI architecture, screens, overlays, keybindings, panels, theme tokens, width matrices | [tui-design.md](tui-design.md), [commands-and-modes.md](commands-and-modes.md) | `documented` | Fully documented in TUI design specification and commands reference. |
+| `src/tools/` | 19 built-in tools across 7 planes, registry, policy engine bindings, observation envelope bounds | [tool-usage.md](tool-usage.md), [prompt-envelope-and-tools.md](prompt-envelope-and-tools.md) | `documented` | Comprehensive 19-tool reference with schemas, examples, and envelope size constraints. |
+| `src/utils/` | Image manipulation, photon operations, git execution utilities | [architecture.md](architecture.md), [tool-usage.md](tool-usage.md) | `documented` | Utility helpers documented within tool usage and architectural boundaries. |
+| `src/worker/` | Worker subprocess lifecycle, NDJSON transport, heartbeat timers, control lane demuxing, spec contracts | [worker-dispatch-mechanics.md](worker-dispatch-mechanics.md) | `documented` | Complete reference for NDJSON socket protocols, watchdog timers, and exit status mapping. |
+| `src/domains/agents/` | 12 built-in recipes, agent catalog, recipe schema, fleet commands, fleet contract v4 | [built-in-agents.md](built-in-agents.md), [fleet-dispatch.md](fleet-dispatch.md) | `documented` | Documented in built-in agent recipes guide and fleet dispatch architecture. |
+| `src/domains/components/` | Component scanning, snapshots, hashing, diffing | [middleware-and-components.md](middleware-and-components.md) | `documented` | Documented in active component snapshot and middleware guide. |
+| `src/domains/config/` | Configuration contracts, file watcher, keybinding definitions, setting classifiers | [configuration-and-targets.md](configuration-and-targets.md), [commands-and-modes.md](commands-and-modes.md) | `documented` | Documented in configuration targets and command/keybinding reference. |
+| `src/domains/context/` | `CLIO-CODER.md` bootstrap, codewiki generation, prompt context assembly, project rules | [context-engine.md](context-engine.md) | `documented` | Documented in context window, token accounting, and compaction reference. |
+| `src/domains/dispatch/` | Fleet orchestration, assignment store, batch tracker, admission, route planner, receipt integrity v15 | [fleet-dispatch.md](fleet-dispatch.md), [dispatch-architecture-rationale.md](dispatch-architecture-rationale.md), [worker-dispatch-mechanics.md](worker-dispatch-mechanics.md) | `documented` | Multi-node fleet dispatch, admission invariants, and receipt verification fully documented. |
+| `src/domains/eval/` | Suite v2 YAML schema, eval runner, metrics, reporters, workspace sandboxing | [eval-runner.md](eval-runner.md), [evals-internal.md](evals-internal.md) | `documented` | Documented in eval runner and soak benchmark guides. |
+| `src/domains/evidence/` | Evidence bundles, findings taxonomy, provenance store, failure attribution | [evidence-and-memory.md](evidence-and-memory.md) | `documented` | Documented in evidence directory structures and memory retrieval guide. |
+| `src/domains/evolution/` | Falsifiable Change Manifest JSON templates and `clio-coder evolve` self-edit gates | [evolution.md](evolution.md) | `documented` | Documented in evolution manifest reference and mutation validation rules. |
+| `src/domains/extensions/` | Extension manifest schemas, resource roots, portable share archives | [extensions-and-sharing.md](extensions-and-sharing.md) | `documented` | Documented in extensions and sharing guide. |
+| `src/domains/lifecycle/` | Platform initialization, upgrade, reset, uninstall, migrations, doctor diagnostics | [installation-and-lifecycle.md](installation-and-lifecycle.md) | `documented` | Platform directories, permissions, initialization, and diagnostic commands documented. |
+| `src/domains/memory/` | Proactive task memory, three-tier bank, two-phase policy grammar, handoffs | [proactive-memory.md](proactive-memory.md), [evidence-and-memory.md](evidence-and-memory.md) | `documented` | Proactive task bank, intervention rules, and persistence fully documented. |
+| `src/domains/middleware/` | Middleware hooks (`turn_start`, `tool_call`, `tool_result`, `turn_end`), reminders, budgets | [middleware-and-components.md](middleware-and-components.md) | `documented` | Documented in middleware hooks and active component snapshot guide. |
+| `src/domains/observability/` | Trace store (`node:sqlite` WAL mirror), metrics, cost accounting, evidence index | [trace-store.md](trace-store.md), [observability.md](observability.md) | `documented` | Database schema, rowid cursor queries, and receipt provenance documented. |
+| `src/domains/prompts/` | Prompt compiler, fragment loaders, static cache stability, memory intervention injection | [prompt-envelope-and-tools.md](prompt-envelope-and-tools.md) | `documented` | Documented in prompt envelope and tool delivery guide. |
+| `src/domains/providers/` | Runtime adapters, capability probes, model catalog, thinking control, ALCF OAuth | [configuration-and-targets.md](configuration-and-targets.md), [model-catalog.md](model-catalog.md), [provider-adapter-cookbook.md](provider-adapter-cookbook.md), [alcf-provider.md](alcf-provider.md) | `documented` | Complete provider adapter contracts, model catalogs, and ALCF Globus targets documented. |
+| `src/domains/resources/` | Skill package discovery, marketplace index resolution, prompt resources | [skills-marketplace.md](skills-marketplace.md), [extensions-and-sharing.md](extensions-and-sharing.md) | `documented` | Skills marketplace, publishing flows, and resource managers documented. |
+| `src/domains/safety/` | Policy engine, action classifiers, damage-control rules, path policies, finish contract, audit log | [safety-model.md](safety-model.md), [scientific-validation.md](scientific-validation.md) | `documented` | Policy evaluation order, 10-step sequence, write containment, and finish contract documented. |
+| `src/domains/scheduling/` | Capacity lease acquisition, heartbeats, expiry, cross-process locks, cluster scheduling | [capacity-and-scheduling.md](capacity-and-scheduling.md), [fleet-dispatch.md](fleet-dispatch.md) | `documented` | Dedicated capacity leasing, heartbeat TTL, and cross-process lock reference. |
+| `src/domains/session/` | Context ledger v3, tree branching (`/tree`), `/fork`, `/resume`, checkpoints, protected-artifact journal | [session-lifecycle.md](session-lifecycle.md) | `documented` | Dedicated session lifecycle guide covering ledger format v3, branching, journal, and recovery. |
+| `src/domains/share/` | Portable share archive bundles, manifest verification, import/export flows | [extensions-and-sharing.md](extensions-and-sharing.md) | `documented` | Share archives and portable bundle formats documented in extensions guide. |
+| `src/domains/webhook/` | Empty directory | None (Inert) | `inert` | Directory contains no active modules or exports in v0.3.0. |
+
+## Cross-Cutting Reference Guides
+
+In addition to source subsystem mappings, the documentation set includes cross-cutting contracts:
+
+1. [artifact-versions.md](artifact-versions.md): Canonical version registry and migration contract for all 9 serialized artifact schemas across Clio Coder.
+2. [glossary.md](glossary.md): Formal definitions of 17 core architectural concepts mapped to their TypeScript types in `src/`.
+3. [troubleshooting.md](troubleshooting.md): Comprehensive diagnostic and remediation guide keyed by exact user-facing error strings.

package/docs/documentation-guide.md

@@ -0,0 +1,166 @@
+# Documentation Standards and Codebase Alignment
+
+> [!TIP]
+> **Interactive Spec Available:** An interactive documentation link linter, phrasing/claim evaluator, and alignment portal is located at [docs/html/documentation_blueprint.html](html/documentation_blueprint.html) (Version: 0.3.0).
+
+Clio Coder is an experimental community alpha. Documentation should help contributors and early users work from the source of truth without overstating maturity. When docs drift, prefer the current source and tests over older prose or aspirational roadmap notes.
+
+---
+
+## Source-first documentation rule
+
+Before changing public docs, inspect the relevant implementation:
+
+1. `git log --oneline -- <area>` for recent intent and release context.
+2. `src/**` for current behavior.
+3. `tests/**` for executable contracts and edge cases.
+4. `README.md`, `CHANGELOG.md`, and `docs/*.md` for existing public wording.
+
+Classify claims clearly:
+
+| Claim class | How to word it |
+| --- | --- |
+| Shipped and tested | State directly and link to source/tests. |
+| Implemented but experimental | Say alpha/experimental and name sharp edges. |
+| Typed contract exists, default runtime is inert | Say the schema exists but no public loader/rules are active. |
+| Planned/future | Put in roadmap language; do not present as available behavior. |
+
+---
+
+## Documentation map
+
+| Guide | Primary source references | What it should cover |
+| --- | --- | --- |
+| [README.md](../README.md) | `CHANGELOG.md`, package metadata, release receipts | Product overview, install, first run, alpha framing, and release status. |
+| [docs/README.md](README.md) | This docs directory | Documentation hub. |
+| [commands-and-modes.md](commands-and-modes.md) | `src/cli/index.ts`, `src/cli/args.ts`, `src/interactive/slash-commands.ts`, `src/domains/dispatch/**` | CLI commands, headless run flags (`--session`, `--continue`, `--json-events`), session continuity, `--json` wire projection promise, slash commands, keybindings, live steering. |
+| [context-engine.md](context-engine.md) | `src/domains/context/**`, `src/domains/session/context-accounting.ts`, `src/domains/session/context-ledger.ts`, `src/domains/session/compaction/` | Context window resolution, per-model probe capabilities, token accounting, snapshots, progressive compaction, model-driven `clio-coder context init`, format v3 session enforcement. |
+| [architecture.md](architecture.md) | `tests/boundaries/check-boundaries.ts`, `src/core/domain-loader.ts`, `src/engine/**`, `src/worker/**` | Source layout, 5 enforced boundary rules (dependency direction vs import form), runtime flow mermaid diagram, event/audit model, detect-and-rollback write boundaries. |
+| [dispatch-architecture-rationale.md](dispatch-architecture-rationale.md) | `src/domains/dispatch/**`, `tests/boundaries/check-boundaries.ts` | Design rationale, not behavior: invariants that cross the seams a dispatch split would use, what any future split must preserve, the one dispatch→eval import, and the closed barrel-import decision. |
+| [configuration-and-targets.md](configuration-and-targets.md) | `src/core/defaults.ts`, `src/core/config.ts`, `src/domains/providers/**`, `src/cli/configure.ts`, `src/cli/targets.ts`, `src/cli/models.ts`, `src/cli/auth.ts` | TargetDescriptor, contextWindowProvenance (`configured`, `discovered`, `catalog`, `runtime-default`), settings.yaml, strict validation, saved defaults vs live routing. |
+| [safety-model.md](safety-model.md) | `src/domains/safety/**`, `src/tools/registry.ts`, `src/tools/policy.ts`, `src/tools/verify/**`, `src/entry/orchestrator.ts`, `src/domains/dispatch/write-boundary.ts` | Operating posture, `resolveEffectiveAutonomy` / `resolveBaselineAutonomy`, detect-and-rollback write boundaries, approval axes, damage control, typed validation. |
+| [prompt-envelope-and-tools.md](prompt-envelope-and-tools.md) | `src/domains/prompts/compiler.ts`, `src/interactive/chat-loop.ts`, `src/core/tool-names.ts`, `src/tools/registry.ts`, `src/tools/observation.ts`, `src/tools/agent-tools.ts` | Prompt envelope reuse, canonical tool delivery via single `agent-tools.ts` adapter, seven-plane tool surface, observation envelope, strict `ToolName` keying. |
+| [tool-usage.md](tool-usage.md) | `src/tools/agent-tools.ts`, `src/tools/registry.ts`, `src/tools/observation.ts` | In-depth reference for all 19 worker tools: parameters, typical payloads, `prepareArguments` normalizers, and error examples. |
+| [provider-adapter-cookbook.md](provider-adapter-cookbook.md) | `src/domains/providers/registry.ts`, `src/domains/providers/types/runtime-descriptor.ts` | RuntimeDescriptor, probe(), probeReasoning(), synthesizeModel(), thinking mechanisms. |
+| [alcf-provider.md](alcf-provider.md) | `src/domains/providers/runtimes/cloud/alcf.ts`, `src/engine/alcf-oauth.ts` | Globus PKCE OAuth, openAuthStorage(), Sophia vLLM, Metis API, chatTemplateKwargsUnsupported. |
+| [environment-variables.md](environment-variables.md) | `src/core/guardrails.ts`, `src/core/xdg.ts`, `src/domains/providers/knowledge-base-path.ts` | Comprehensive env var matrix: guardrail overrides, directory layout (CLIO_CODER_HOME), debug toggles, and internal plumbing. |
+| [built-in-agents.md](built-in-agents.md) | `src/domains/agents/**`, `src/domains/agents/builtins/*.md`, `src/domains/dispatch/**` | Builtin agent recipes, discovery roots, frontmatter schema, fleet contract shadowing (`.clio-coder/fleets/<name>.md`), active route automation. |
+| [fleet-dispatch.md](fleet-dispatch.md) | `src/domains/dispatch/**` | Multi-node SSH dispatch: process-safe admission, capacity leases, Contract v4 write boundaries (detect-and-rollback), bounded check/repair loops (`loop_bound_exhausted`), deterministic code steps, attestation, receipts v15. |
+| [capacity-and-scheduling.md](capacity-and-scheduling.md) | `src/domains/scheduling/**`, `src/domains/dispatch/capacity-lease.ts`, `src/domains/dispatch/reservation-store.ts` | Multi-process capacity leases (`dispatch-admission.json`), heartbeat TTLs, cross-process transaction locks (`dispatch-admission.json.lock`), and cluster drain controls. |
+| [worker-dispatch-mechanics.md](worker-dispatch-mechanics.md) | `src/worker/**` | NDJSON parent-child socket protocols, control/bulk lane demuxing, watchdog timers, worker attestation (13 protocol fields), permission parking, exit codes. |
+| [fleet-demo-runbook.md](fleet-demo-runbook.md) | `src/domains/dispatch/**` | Multi-node fleet demo: SSH setup, C++ build/repair workflow, reviewer gates, receipt verification v15. |
+| [session-lifecycle.md](session-lifecycle.md) | `src/engine/session.ts`, `src/domains/session/**` | Session lifecycle, on-disk ledger format v3 (`current.jsonl`), tree branching (`tree.json`), active-path lineage selection, `/fork`, `/resume`, checkpoints, and write-ahead protected-artifact journal. |
+| [acp.md](acp.md) | `src/engine/acp/**`, `src/cli/acp.ts` | Agent Client Protocol (ACP) server over stdio, tool mediation, non-stall permission handling, timeout bounds, and error taxonomy. |
+| [artifact-versions.md](artifact-versions.md) | `src/domains/dispatch/receipt-integrity.ts`, `src/engine/session.ts`, `src/worker/spec-contract.ts`, `src/domains/agents/fleet-contract.ts`, `src/domains/eval/schema/`, `src/domains/observability/trace-store.ts` | Version registry and migration policies for all 9 serialized artifact schemas across Clio Coder. |
+| [exit-codes-and-output.md](exit-codes-and-output.md) | `src/cli/**`, `src/entry/**` | Global process exit codes (0, 1, 2, 3), `--help` standard on stdout, machine-readable JSON streaming (`--json`, `--json-events`), and headless stdout deliverable contracts. |
+| [troubleshooting.md](troubleshooting.md) | `src/core/**`, `src/cli/**`, `src/domains/**` | Actionable error remediation and diagnostics keyed by exact user-facing messages. |
+| [glossary.md](glossary.md) | `src/domains/dispatch/types.ts`, `src/tools/**`, `src/domains/agents/**`, `src/core/**` | Canonical definitions of 17 core architectural concepts mapped to `src/` types. |
+| [documentation-coverage.md](documentation-coverage.md) | `src/**` | Complete source-to-documentation mapping matrix and subsystem coverage status. |
+| [tui-design.md](tui-design.md) | `src/interactive/theme/tokens.ts`, `src/interactive/theme/glyphs.ts` | TUI color system, glyph vocabulary (`contextReserve`), structural layouts, state choreography, code ink. |
+| [installation-and-lifecycle.md](installation-and-lifecycle.md) | `src/cli/paths.ts`, `src/cli/doctor.ts`, `src/cli/uninstall.ts`, `src/cli/removal.ts` | Installation, upgrade, reset, uninstallation, launcher ownership and what `--remove-binary` will and will not remove, partial-failure behavior, configuration folders (`credentials.yaml` `0o600`), and permissions. |
+| [release-cut-checklist.md](release-cut-checklist.md) | `scripts/check-release.mjs`, `scripts/lifecycle-matrix.mjs`, `package.json` | Ordered release-cut steps with an explicit authorization boundary: everything external or irreversible is marked not run and needs an operator decision. |
+| [observability.md](observability.md) | `src/domains/observability/**`, `src/interactive/view/**`, `src/domains/dispatch/**`, `src/core/bus-events.ts` | `/view` artifact browsing, receipt verification, worker diagnostics, event routing, and cost snapshots. |
+| [evidence-and-memory.md](evidence-and-memory.md) | `src/domains/evidence/**`, `src/domains/memory/**`, `src/cli/evidence.ts`, `src/cli/memory.ts` | Evidence corpus layout, findings, memory lifecycle and prompt injection. |
+| [proactive-memory.md](proactive-memory.md) | `src/domains/memory/**` | Proactive task memory architecture, session task bank, intervention rules, and handoff carrying. |
+| [trace-store.md](trace-store.md) | `src/cli/trace.ts`, `src/domains/observability/trace-store.ts` | WAL SQLite trace mirror database schema, rowid cursor queries, rebuildability, 6 `clio-coder trace` subcommands (`runs`, `phases`, `tail`, `procs`, read-only `sql` SELECT, `ui`). |
+| [eval-runner.md](eval-runner.md) | `src/domains/eval/**`, `src/cli/eval.ts` | Local YAML eval tasks, dual token accountings (`tokens.*` wire vs `receiptUsage.*` journal), fail-closed null totals, EvalArtifactV4 format, `verify.measure` task outcome recording. |
+| [evals-internal.md](evals-internal.md) | `src/domains/eval/**`, `benchmarks/soak/**` | Private context index determinism, target smoke matrices, soak machinery benchmark suite (4 suites: `clio-soak`, `clio-soak-boundary`, `clio-soak-chaos`, `clio-soak-loop`). |
+| [extensions-and-sharing.md](extensions-and-sharing.md) | `src/domains/extensions/**`, `src/domains/resources/**`, `src/domains/share/**`, `src/cli/extensions.ts`, `src/cli/share.ts` | Prompt and skill resources, extension manifests, portable share archives. |
+| [skills-marketplace.md](skills-marketplace.md) | `src/interactive/overlays/skills-hub.ts`, `src/domains/resources/skills/marketplace.ts` | Skills Hub marketplace discovery through the install resolver, empty state, install actions, publishing flow. |
+| [model-catalog.md](model-catalog.md) | `src/domains/providers/catalog.ts`, `src/domains/providers/models/**`, `src/domains/providers/probe/**`, `src/domains/providers/model-capabilities.ts` | Model catalog, live probes (`--offline` toggle), exact-id selector `probeCapabilitiesForModel`, field-note promotion. |
+| [middleware-and-components.md](middleware-and-components.md) | `src/domains/components/**`, `src/domains/middleware/**`, `src/cli/components.ts` | Active component snapshots, phase-aware middleware hook budgets (`DEFAULT_MIDDLEWARE_HOOK_BUDGETS_MS`). |
+| [scientific-validation.md](scientific-validation.md) | `src/domains/safety/rigor.ts`, `src/domains/safety/finish-contract.ts` | Advisory validation-contract patterns for scientific artifacts and HPC assumptions. |
+| [evolution.md](evolution.md) | `src/domains/evolution/**`, `src/cli/evolve.ts` | Falsifiable Change Manifest JSON templates, evidence-linked validation, and `clio-coder evolve`. |
+| [config-knobs-audit.md](config-knobs-audit.md) | `src/cli/config.ts` | Point-in-time inventory of legacy environment variables (Historical Appendix). |
+
+---
+
+## Style conventions
+
+### Alpha framing
+
+Use direct, honest language:
+
+- "experimental community alpha"
+- "source-build path"
+- "current runtime is conservative/inert"
+- "advisory contract"
+- "planned/future milestone"
+
+Avoid phrases that imply managed production stability, full plugin maturity, or automatic scientific validation when the current code does not provide it.
+
+### Markdown structure
+
+- Prefer short sections with tables for command and schema references.
+- Use fenced examples that can be copied.
+- Keep links relative and repository-portable; do not use absolute `file:///home/...` links.
+- Mention source file paths in backticks instead of editor-specific absolute URLs.
+
+### GitHub alerts
+
+Use alerts sparingly:
+
+> [!NOTE]
+> Context or caveats that prevent misinterpretation.
+
+> [!WARNING]
+> Sharp edges, alpha limitations, or behavior that can surprise contributors.
+
+> [!CAUTION]
+> Safety, data loss, or security-sensitive constraints.
+
+---
+
+## Blueprint Coverage & Format Strategy
+
+Clio Coder maintains two complementary documentation formats:
+
+1. **Markdown Documents (`docs/*.md`)**: The single canonical reference for coding agents, developers, and maintainers. They optimize for retrievability, exact enumerations, schema tables, typed TypeScript contracts, and source citations (`src/...:line`).
+2. **Interactive HTML Blueprints (`docs/html/*.html`)**: Visual reference cards and client-side simulators designed for human operators exploring dynamic behaviors (such as safety rule evaluation, token compaction calculation, YAML target validation, and prompt structure).
+
+### Blueprint Creation Policy
+
+Not every markdown document earns a standalone interactive HTML blueprint. Reference specifications—such as `artifact-versions.md`, `exit-codes-and-output.md`, `glossary.md`, `troubleshooting.md`, `session-lifecycle.md`, `acp.md`, `capacity-and-scheduling.md`, and `documentation-coverage.md`—are authoritative tabular contracts and state machine specifications. Building client-side JavaScript simulators for these documents would duplicate runtime validation logic and introduce synchronization hazards across releases. These pages are therefore linked directly from `docs/html/index.html` as Markdown Reference Specifications with explicit visual distinction from interactive simulator blueprints.
+
+---
+
+## Update checklist
+
+When a feature changes:
+
+1. Identify the source owner (`src/cli`, `src/interactive`, `src/tools`, or a domain).
+2. Check whether public CLI help changed.
+3. Update the mapped guide in the same PR.
+4. If behavior affects safety, sessions, receipts, prompts, targets, or dispatch, update both README-level user docs and the deeper guide.
+5. Run a lightweight link check for changed Markdown.
+6. For release docs, verify version badges/sections match `package.json` and `CHANGELOG.md`.
+
+Suggested local link check:
+
+```bash
+python3 - <<'PY'
+import pathlib, re
+for md in list(pathlib.Path('docs').glob('*.md')) + [pathlib.Path('README.md')]:
+    text = md.read_text()
+    for m in re.finditer(r'\[[^\]]+\]\(([^)]+)\)', text):
+        link = m.group(1)
+        if link.startswith(('http://', 'https://', 'mailto:', '#')):
+            continue
+        target = link.split('#')[0]
+        if target and not (md.parent / target).exists():
+            line = text.count('\n', 0, m.start()) + 1
+            print(f'{md}:{line}: missing {link}')
+PY
+```
+
+---
+
+## Community documentation priorities
+
+Clio users tend to be early adopters running real repositories, local models, and scientific/HPC code. Good docs should therefore prioritize:
+
+- reproducible first-run and target configuration;
+- local model/runtime field notes with exact versions and serving settings;
+- safety receipts and redaction guidance for issue reports;
+- small examples for project-local `CLIO-CODER.md`, `.clio-coder/safety.yaml`, prompts, skills, and agents;
+- clear labels for experimental surfaces such as middleware and scientific validation contracts.