@iowarp/clio-coder 0.3.8 → 0.4.0

package/docs/installation-and-lifecycle.md

@@ -3,7 +3,19 @@
 Clio Coder is designed to be self-contained and platform-compliant. This document outlines the default directory paths, file purposes, permission levels, and lifecycle commands (`install`, `reset`, `upgrade`, and `uninstall`). Clio Coder installs from npm as `@iowarp/clio-coder` (`npm install -g @iowarp/clio-coder`, published since v0.3.0) or from a source checkout with a deterministic local symlink; the CLI classifies both install kinds and `clio-coder upgrade` handles each.
 
 > [!TIP]
-> **Interactive Spec Available:** An interactive dashboard with a path simulator and visual flowcharts is located at [docs/html/lifecycle_blueprint.html](html/lifecycle_blueprint.html) (Version: 0.3.8). You can open it directly in any web browser to view details dynamically.
+> **Interactive Spec Available:** An interactive dashboard with a path simulator and visual flowcharts is located at [docs/html/lifecycle_blueprint.html](html/lifecycle_blueprint.html) (Version: 0.4.0). You can open it directly in any web browser to view details dynamically.
+
+### Optional dependency: the Claude Agent SDK
+
+`@anthropic-ai/claude-agent-sdk` is an `optionalDependencies` entry, not a hard dependency. Its platform package carries a proprietary binary of roughly 224MB per platform, and only the `claude-sdk` runtime uses it. Skip it with:
+
+```bash
+npm install -g @iowarp/clio-coder --omit=optional
+```
+
+Measured on Linux x64 with a production install (`--omit=dev`): 387MB across 118 packages with the SDK, 143MB across 109 packages without it. That is 244MB and nine packages saved, a 63% smaller tree, and what remains is fully open-licensed.
+
+Everything except the `claude-sdk` runtime works on the smaller install: boot, `clio-coder doctor`, every other target and worker runtime. Dispatching a `claude-sdk` target on an install that omitted the package fails that run with a diagnostic naming the package and the command that fixes it (`npm install @anthropic-ai/claude-agent-sdk@0.3.186`); nothing else degrades, and nothing fails at startup.
 
 ---
 
@@ -84,6 +96,7 @@ The core files are created automatically during the first run. `credentials.yaml
 | **State** | `install.json` | Install metadata: Clio version, node, platform, `installedAt` (written once at first install), `upgradedAt` and `upgradedFrom` (stamped on a version change), and `noticedVersion` (the version whose one-time upgrade notice the interactive launch has shown). | Writer/umask default | Removed by uninstall / `reset --state`. |
 | **State** | `migrations.json` | Log of successfully applied schema/state migrations. | Writer/umask default | Removed by uninstall / `reset --state`. |
 | **Data** | `memory/records.json` | Long-term learning memories (up to 500 records) proposed/approved from runs. | Writer/umask default | Removed by uninstall / `reset --data`. |
+| **Data** | `tools/<id>/<version>/` | One pinned external program Clio downloaded on request (`clio-coder tools install <id>`), with its upstream license text and a `clio-install.json` recording url, sha256, platform and install time. Binaries `0o755`, documents `0o644`. Only the pinned version is kept: a successful install prunes the versions it supersedes. | `0o755` dir | `clio-coder tools remove <id>` deletes every version of one tool; removed by uninstall / `reset --data`. |
 | **State** | `audit/YYYY-MM-DD.jsonl` | Daily safety audit logs showing allowed/blocked tool actions. | Writer/umask default | Removed by uninstall / `reset --state`. |
 | **State** | `sessions/<cwdHash>/<id>/` | Session details: `meta.json`, `current.jsonl`, and fork hierarchies `tree.json`. | Writer/umask default | Removed by uninstall / `reset --state`. |
 
@@ -256,7 +269,7 @@ inventory of what a level covers, because a remembered list drifts as soon as a
 new artifact is written into a root.
 
 *   `--state` *(Default)*: Deletes the state root only. It holds every session transcript and the audit trail beside it, so a reset is the end of `resume`, `/view`, and their history. This is the level a bare `clio-coder reset` selects, and it carries that note in its preview.
-*   `--data`: Deletes the data root only: memory, evidence, evals (durable products).
+*   `--data`: Deletes the data root only: memory, evidence, evals, and any vendored external tools (durable products). The vendored tools are the one entry a reset cannot regenerate locally; `clio-coder tools install <id>` downloads them again.
 *   `--cache`: Deletes the cache root only.
 *   `--auth`: Deletes `credentials.yaml`. Removes all saved keys.
 *   `--config`: Deletes `settings.yaml` to revert preferences to default.

package/docs/middleware-and-components.md

@@ -1,7 +1,7 @@
 # Middleware and Component Registry
 
 > [!TIP]
-> **Interactive Spec Available:** An interactive dashboard with an interactive component scanner and a dynamic hook-and-effect pipeline is located at [docs/html/middleware_blueprint.html](html/middleware_blueprint.html) (Version: 0.3.8).
+> **Interactive Spec Available:** An interactive dashboard with an interactive component scanner and a dynamic hook-and-effect pipeline is located at [docs/html/middleware_blueprint.html](html/middleware_blueprint.html) (Version: 0.4.0).
 
 Clio Coder has two related but separate surfaces:
 

package/docs/model-catalog.md

@@ -1,7 +1,7 @@
 # Model Catalog, Runtime Refresh, and Field Notes
 
 > [!TIP]
-> **Interactive Spec Available:** An interactive dashboard mapping capabilities, probe discovery, and target resolution is located at [docs/html/models_blueprint.html](html/models_blueprint.html) (Version: 0.3.8).
+> **Interactive Spec Available:** An interactive dashboard mapping capabilities, probe discovery, and target resolution is located at [docs/html/models_blueprint.html](html/models_blueprint.html) (Version: 0.4.0).
 
 Clio Coder treats a selectable model as the intersection of three sources:
 
@@ -161,6 +161,15 @@ The Context Engine evaluates thinking mechanisms per model target and manages li
 
 - **Ollama Native (`ollama-native`):** Ollama utilizes the native `thinking` field in the request and response payloads. The engine handles Ollama-specific effort levels and streams reasoning increments cleanly through the native thinking channel.
 - **LM Studio (`lmstudio`):** Chat uses the OpenAI-compatible `/v1/chat/completions` surface, including its `reasoning` stream field. Clio controls thinking only with `reasoning_effort` and never sends `chat_template_kwargs` to LM Studio. See <https://lmstudio.ai/docs/developer/openai-compat/chat-completions>.
+- **LiteLLM (`litellm`):** This is a gateway runtime, not an `openai-compat`
+  alias. Discovery checks `/health/liveliness`, reads aliases and capability
+  metadata from `/v1/model/info`, and records the physical deployment reported
+  by `x-litellm-*` response headers. Defaults stay conservative when metadata is
+  absent: tools, vision, and reasoning are not inferred. The runtime advertises
+  standard `json_schema` structured output and treats schema-plus-tools as a
+  conflict because the gateway alias does not identify one stable upstream.
+  Residency is observe-only because LiteLLM owns loading and eviction behind the
+  alias.
 - **OpenAI Completions (`openai-completions`):** The OpenAI-compatible completions provider preserves reasoning blocks within assistant messages. It replays thinking blocks via the `reasoning_content` parameter in the message history, ensuring that the model maintains its chain-of-thought across conversational turns without stripping the data.
 - **Anthropic OAuth / API (`anthropic-max`):** Uses the `anthropic-extended` thinking format. The engine supports Anthropic's native extended thinking block protocol, streaming thinking increments and outputting them wrapped appropriately or natively depending on target capabilities.
 - **Reasoning-Never Models (`thinking.mechanism: none`):** When a model is configured or cataloged with `thinking.mechanism: none`, it is treated as a reasoning-never model. For these models, Clio must not send any thinking fields or parameters in requests, must not replay thinking blocks, must not surface thinking events to the TUI, and must not preserve or log reasoning token usage in metrics.

package/docs/observability.md

@@ -1,7 +1,7 @@
 # Observability Viewer
 
 > [!TIP]
-> **Interactive Spec Available:** An interactive dashboard is located at [docs/html/observability_blueprint.html](html/observability_blueprint.html) (Version: 0.3.8).
+> **Interactive Spec Available:** An interactive dashboard is located at [docs/html/observability_blueprint.html](html/observability_blueprint.html) (Version: 0.4.0).
 
 `/view` is the interactive artifact viewer for a Clio session. It keeps the live transcript compact while preserving a full inspection path for durable artifacts, task ledgers, and successful workspace outputs.
 
@@ -13,6 +13,35 @@
 
 `/view` opens a full-screen split viewer. The left pane groups artifacts by category and supports type-to-filter. The right pane renders the selected artifact with pager controls. `Tab` or `Shift+Tab` switches between the artifact list and details. `Left` and `Right` jump to the previous or next non-empty category from either pane; `Up` and `Down` select artifacts in the list or scroll details in the content pane. Category jumps honor the active filter and wrap at the ends. `v` verifies a selected receipt. `o` shows the absolute backing path through the notice channel when the selected artifact has one; pathless artifacts produce a warning notice instead. In the list pane, `Esc` clears a non-empty filter before a second `Esc` closes the viewer.
 
+## Trace retention and state usage
+
+The SQLite trace mirror at `<state-dir>/trace.sqlite` is rebuildable and bounded. By default Clio retains terminal runs for 30 days and limits the allocated database to 128 MiB (134,217,728 bytes), whichever limit is reached first. The policy runs after each dispatch or interactive turn becomes terminal. It deletes a run as one unit across `runs`, `phases`, `events`, `envelopes`, `gate_results`, `agent_sessions`, and `processes`. A `queued` or `running` run is never a candidate, even when its start time is older than the age cutoff or its rows put the store over the byte limit.
+
+Two environment variables configure the automatic policy:
+
+| Variable | Default | Valid values |
+| --- | ---: | --- |
+| `CLIO_CODER_TRACE_RETENTION_DAYS` | `30` | An integer of at least 1. |
+| `CLIO_CODER_TRACE_MAX_BYTES` | `134217728` | An integer of at least 1,048,576. |
+
+An operator can apply the current policy immediately or supply one-command overrides:
+
+```bash
+clio-coder trace prune
+clio-coder trace prune --max-age-days 14 --max-bytes 67108864
+clio-coder trace prune --json
+```
+
+The command reports terminal runs removed, total rows removed across the seven run-owned tables, physical bytes reclaimed from `trace.sqlite` and its WAL sidecars, whether `VACUUM` ran, and how many live runs were protected. Age pruning uses a terminal run's `ended_at`. Size pruning removes the oldest terminal runs until the live database pages fit or no terminal candidate remains.
+
+Deleting SQLite rows creates reusable pages but does not normally reduce the file. Clio runs `VACUUM` when at least 20 percent of allocated pages are reclaimable, or whenever reclaiming deleted pages is necessary to enforce the 128 MiB bound. It then truncates the WAL. Smaller deletions remain available for SQLite to reuse and avoid rewriting the whole database on every completed run.
+
+`clio-coder doctor` includes a `state storage` row with the recursive byte total for the state directory and the largest top-level contributor. For example:
+
+```text
+OK   state storage          96.4 MiB (101,082,624 bytes); largest contributor trace.sqlite at 89.9 MiB (94,248,960 bytes)
+```
+
 ---
 
 ## The Evidence Spine End-to-End
@@ -69,8 +98,28 @@ An `EvidenceIndexRow` has the following schema:
 | `turns` | `turns in window` and the `tokens` fact | Folded calls that were turns, so labelled calls are subtracted exactly as `/cost` subtracts them. |
 | `sideQuestions` | `side questions in window` and the `tokens` fact | `/btw` rounds in the window. |
 | `handoffs` | `handoffs in window` and the `tokens` fact | `/handoff` extraction rounds in the window. |
+| `prewarms` | `pre-warms in window` and the `tokens` fact | Prompt pre-warm rounds in the window. |
+| `backgroundMemorySteps` | `background memory steps in window` and the `tokens` fact | Proactive-memory model steps in the window. |
+
+The last five fields appear only when at least one labelled call falls in the window, and each individual line is printed only when its own count is above zero. An archive with no labelled call in it renders exactly as it did before those fields existed, so their presence is itself the signal that money was spent beside a session. All four labelled kinds are subtracted from `turns` the same way, so a session's turn count never includes a round the operator did not take.
+
+The report also prints a prompt-cache block, one row per session that recorded any cache telemetry:
+
+```text
+  prompt cache by session (from backend timings and persisted verdicts)
+  session       uncached prefill  hot/partial/cold/small
+  3vpu6z19ee7t  130353            4/3/2/0
+```
 
-The last three fields appear only when at least one labelled call falls in the window. An archive with no `/btw` or `/handoff` round in it renders exactly as it did before those fields existed, so their presence is itself the signal that money was spent beside a session.
+`uncached prefill` is the sum of the backend's own newly evaluated prompt tokens across every persisted call in that session, and reads `n/a` rather than `0` when the server reported no cache figure to subtract. The four counts are the per-call verdicts. Both facts are also in `--json` under a `session-cache` fact per session.
+
+`clio-coder doctor` reports the same evidence for the latest session only, as one row, so a cache problem is visible without opening the TUI or a report:
+
+```text
+OK cache telemetry  last session 3vpu6z19ee7t: hot 4 · partial 3 · cold 2 · small 0; top expected reason dispatch (3)
+```
+
+The row reads `top expected reason none` when the session recorded verdicts but no expected-cold reason, and it degrades to a warning saying `no prompt-cache telemetry recorded` when the latest session has none at all, which is the honest answer for a target whose backend reports nothing rather than a claim of a perfect cache. "Latest" selects the most recent `current.jsonl` by its newest entry timestamp, falling back to the file's mtime, so the other diagnostic JSONL files in a session directory cannot be mistaken for the conversation.
 
 ### The Out-of-Turn Usage Store
 
@@ -102,6 +151,8 @@ A row has the following schema:
 
 `repoIdentity` is the same cwd hash the session ledger is filed under, which is what lets `usage report --repo <path>` select these rows with the hash it already computes for the ledgers.
 
+`label` is one of `side-question`, `handoff`, `prewarm`, or `background-memory`. The last two joined for the same reason as the first two: a prompt pre-warm and a proactive-memory step are provider calls the operator did not ask for and would otherwise never see, and neither appends anything to the session JSONL. A row may also carry `timing { durationMs }` and a `promptCache` block built from the backend's own prefill facts when the server reported them; a backend that reports no timings simply omits the block, as LM Studio's OpenAI-compatible port does.
+
 ---
 
 ## Artifact Categories and Path Layouts
@@ -206,8 +257,8 @@ The base provenance sets, steering, routing, quality, worker identity, result-co
 | `safety.toolTelemetry.ingestionErrors` | `number` | Current dispatch receipts | Malformed or lost frames, event-fold/source errors, and drain timeouts that make otherwise mediated telemetry incomplete | experimental |
 | `safety.toolTelemetry.unfinished` | `{ tool, count }[]` | Current dispatch receipts | Tool starts that had no matching finish when the receipt sealed | experimental |
 | `safety.toolTelemetry.workspaceMutationPossible` | `boolean` | Current dispatch receipts | Whether incomplete or unavailable telemetry could conceal a shared-workspace mutation; retry admission fails closed when true | experimental |
-| `autonomyEnforcement.grade` | `string` | Always in v0.3.7 | The autonomy grade level enforced for the run | experimental |
-| `autonomyEnforcement.autonomy` | `string` | Always in v0.3.7 | The effective autonomy level name (e.g. auto-edit, suggest, read-only, full-auto) | experimental |
+| `autonomyEnforcement.grade` | `string` | Always in v0.4.0 | The autonomy grade level enforced for the run | experimental |
+| `autonomyEnforcement.autonomy` | `string` | Always in v0.4.0 | The effective autonomy level name (e.g. auto-edit, suggest, read-only, full-auto) | experimental |
 | `autonomyEnforcement.externalMode` | `string` | When running external worker | The execution mode of the external worker runtime | experimental |
 | `autonomyEnforcement.dangerousBypass` | `boolean` | When running external worker | Whether a safety bypass was explicitly activated | experimental |
 | `validationGrounding.claimed` | `number` | Validation grounding evaluated | Count of validations claimed by worker | experimental |

package/docs/proactive-memory.md

@@ -1,6 +1,6 @@
 # Proactive task memory
 
-> **Interactive Spec Available:** An interactive memory lifecycle dashboard and simulator is located at [docs/html/memory_blueprint.html](html/memory_blueprint.html) (Version: 0.3.8).
+> **Interactive Spec Available:** An interactive memory lifecycle dashboard and simulator is located at [docs/html/memory_blueprint.html](html/memory_blueprint.html) (Version: 0.4.0).
 
 Clio's proactive task memory protects long-running work from behavioral state
 decay: a requirement, environment fact, failed attempt, or diagnosis can still
@@ -120,7 +120,7 @@ malformed response, or telemetry failure is silent and never blocks a tool.
 - `memory.intervention.everyNTools` (default `10`): Minimum completed-tool interval between background interventions.
 - `memory.intervention.windowSteps` (default `8`): Completed tool-trajectory window analyzed during background evaluation.
 - `memory.intervention.maxTokens` (default `400`): Bounds the rendered memory-bank and reminder context budget; the policy model output cap is a separate fixed `4,000`-token contract in `task-memory-policy.ts`, sized so that a model which reasons anyway still reaches its envelope.
-- `memory.intervention.timeoutMs` (default `180000`): Wall-clock limit for one background memory-policy request. The step is detached, so this deadline never delays a turn; set it above the observed step time for your route or finished work is discarded as a timeout. Step latency on a small local route is long-tailed rather than tightly clustered, so size this off a high percentile and not off a median.
+- `memory.intervention.timeoutMs` (default `30000`): Wall-clock limit for one background memory-policy request. The step is detached, so this deadline never delays a turn, but it does hold a request slot on a real inference endpoint that your own turns and your dispatched workers queue against. The default is what a turn boundary can wait for rather than what a long-tailed route eventually answers in: on the reference route below, 23 of 60 steps ran past 30 seconds and 531 of the measured 1,666 seconds were spent beyond that mark. Raise it only if you have measured that your route's slow steps are the ones producing reminders, and read the trade in "Cost and the default decision" first.
 
 ## Trigger semantics
 
@@ -203,6 +203,88 @@ Thus `last` remains `injected` across such continuations until a later
 tool-bearing or explicitly triggered memory step produces a new outcome (e.g.,
 a healthy tool leading to `silent`).
 
+## Cost and the default decision
+
+The LLM tier costs real tokens, real seconds of model time, and a request slot on
+a server that is usually the same machine the operator's own turns run on. Every
+step is therefore accounted for the way a `/btw` side question is: one cost entry
+under the `background-memory` label, which `/cost` shows as its own `memory steps`
+row, and one durable row in `<stateDir>/usage/out-of-turn.jsonl` carrying the
+usage, the call's duration, and the backend's prefill facts, which
+`clio-coder usage report` folds after the process exits. `/memory` shows the
+lifetime figures folded from `steps.jsonl`: steps, tokens, model time, and the
+hit rate.
+
+### The measurement
+
+From one operator's `steps.jsonl`, 274 rows spanning 2026-08-14 to 2026-08-29 on
+a small local background route:
+
+| Figure | Value |
+| --- | --- |
+| Model-tier steps | 60 |
+| Tokens | 137,205 |
+| Model time | 1,666.6 s |
+| Step latency | median 18.7 s, p90 70.2 s, max 102.5 s |
+| Injections produced by the model tier | 6 |
+| Hit rate | 10.0 percent |
+| Cost per injection | 22,868 tokens and 278 s of model time |
+| Model-tier injections in the last 5 days | 0 of 4 steps |
+
+Four further injections in the same window came from the free rules tier, so the
+lifetime total of 10 injections is not the model tier's score. Rules-tier
+injections cost nothing.
+
+### The decision
+
+The default does not change, and it is a deliberate default rather than an
+unexamined one:
+
+- `memory.intervention.enabled` stays `true`. It runs the rules tier, which makes
+  no model calls, spends no tokens, and produced 4 of the 10 injections.
+- The LLM tier stays opt-in through `background.target` and `background.model`,
+  which is already the case: an unset background role never resolves a client.
+  A 10 percent hit rate at 22,868 tokens per injection does not earn a default-on
+  position, and it is not so poor that it earns removal from an operator who has
+  measured their own route and wants it.
+- The step deadline drops from 180 s to 30 s. This is the one behavioral change,
+  and it is a genuine trade: at 30 s, two of the six observed injections, at
+  53.6 s and 57.7 s, would have been cut, while 531 s of the 1,666 s spent would
+  not have been spent at all. The deadline is the bound on what one optional call
+  may hold a shared local server for, not a prediction of when a route answers.
+- A step that would run on the endpoint the chat target is streaming against is
+  skipped with reason `endpoint_busy`, and the skip is recorded. On a single-slot
+  llama.cpp router the alternative is queueing behind the operator's own decoding
+  or evicting the resident model, and neither is a cost an optional call may
+  impose.
+
+### What a background target costs on a shared local server
+
+If `background.target` names the same server as `orchestrator.target`, that
+server's slots are shared. On a llama.cpp router started with `--parallel 1`
+there is exactly one, and the memory step and the operator's turn contend for it.
+
+The consequence is worth stating plainly: a shared endpoint suppresses the model
+tier rather than merely delaying it. A step is started from the `turn_end` hook,
+which fires inside the streaming run at `agent_end`
+(`src/interactive/turn-runtime.ts`), while the chat loop still holds its
+foreground registration on that endpoint; the loop releases the hold afterwards,
+in the `finally` around the run (`src/interactive/chat-loop.ts`). Every boundary
+therefore finds the endpoint busy and records `dropped`/`endpoint_busy`. That is
+the intended trade: an optional call may not take the one slot the operator's own
+turn is using, and it may not make the server swap the resident model out. The
+`/memory` step list and `steps.jsonl` say so on every boundary, so the tier is
+visibly declining rather than quietly idle.
+
+The second mechanism is an `expected cold` stamp, for the case where a step did
+run on the chat endpoint. Its prompt is a trajectory rather than the chat prefix,
+so the next turn's prefill is expected to be cold; `/context` names
+`background_memory` as the reason instead of reporting an unexplained cold
+prefix.
+
+Pointing the background role at a second machine avoids both effects and is the
+arrangement the tier is designed for.
+
 ## Choosing a background model
 
 Memory reads a trajectory and writes a fixed envelope. It does not plan, and it
@@ -257,7 +339,7 @@ memory:
     everyNTools: 10
     windowSteps: 8
     maxTokens: 400
-    timeoutMs: 180000
+    timeoutMs: 30000
 ```
 
 With `background.target` and `background.model` unset, Clio stays in the
@@ -289,12 +371,12 @@ percentile of 79.9, and a 95th of 131.6. Capability is not the constraint;
 latency is, its spread is wide, and the detached step above is what makes the
 tier usable anyway.
 
-Size `timeoutMs` off that tail rather than off the median. The shipped 180000
-captures roughly the whole distribution on this route. A 20000 setting looks
-generous against an 18.6-second median and in practice discarded about half of
-all steps, since the request is aborted on timeout and its work is thrown
-away. A route whose steps mostly record `timeout` is a misconfigured deadline
-before it is a slow model.
+The deadline is a bound on what an optional call may hold that server for, not a
+figure sized to capture the tail. The shipped 30000 sits above the median and
+below the tail deliberately, and a step that exceeds it records `timeout` with
+its work discarded. A route whose steps mostly record `timeout` is a
+misconfigured deadline before it is a slow model, so read the ledger before
+raising it: `/memory` shows the hit rate the raise would be buying.
 
 The target ID is not hard-coded. Any configured orchestrator-eligible local
 target and wire model can fill the background role. Before enabling it, use the
@@ -316,6 +398,25 @@ For an immediate kill switch, set `memory.intervention.enabled` to `false` in
 `/settings`. Removing the background target instead returns to rules-only
 operation while leaving deterministic protection active.
 
+## Where what the tier writes ends up
+
+A bank entry lives and dies with its session. When a reminder actually reaches
+the operator, the entries it cited are also proposed into the durable store at
+`<dataDir>/memory/records.json`, unapproved, scoped to the repository the session
+is working in, with provenance naming the session and the source entry. That is
+the one automatic writer of that file; everything else about it is unchanged.
+`/memory` and `clio-coder memory list` show the proposal, and
+`clio-coder memory approve <id>` is still a separate operator action, so nothing
+the background plane produced reaches a system prompt without review. A step with
+no session, or one running outside a canonical repository, proposes nothing:
+global scope broadens applicability to every future session and is not a claim a
+background step may make on the operator's behalf.
+
+Rules-tier reminders are not proposed. Their entries are this middleware's own
+one-line records of a repeated tool failure, and filing each one as a durable
+lesson would fill the review queue with rows nobody asked for. They remain
+promotable by hand from `/memory`.
+
 ## What the LLM tier actually writes
 
 Measured on the shipped prompt against `google/gemma-4-26b-a4b-qat`, across ten
@@ -392,11 +493,23 @@ keeps one previous generation as `steps.jsonl.1`. Every exact-schema record has:
 - `silent`, `injected`, `gated`, `timeout`, `malformed`, or `dropped` decision;
 - count of cited entries, input/output/total memory-model tokens, and latency.
 
-`dropped` is the one outcome that ran no step: the boundary triggered while an
-earlier step still held the single in-flight slot. It costs no tokens and no
-latency, its triggers survive to the next free boundary, and it does not replace
-the operator-visible last decision. Counting `dropped` rows against `llm` rows
-over a session is how a starved cadence becomes visible.
+The same steps are also billed. See "Cost and the default decision" for the
+`/cost` row, the durable out-of-turn usage row, and the lifetime figures `/memory`
+folds out of this file.
+
+`dropped` is the one outcome that ran no step. It has two causes, separated by
+the row's reason: `step_in_flight` means the boundary triggered while an earlier
+step still held the single in-flight slot, and `endpoint_busy` means the step
+would have called the endpoint the chat target was streaming against. Both cost
+no tokens and no latency, both leave their triggers pending for the next free
+boundary, and neither replaces the operator-visible last decision. Counting
+`dropped` rows against `llm` rows over a session is how a starved cadence becomes
+visible.
+
+A step that exceeds the deadline records `timeout`, never `silent`: reason
+`deadline` when the policy's own race fired first, and `timed_out` when the
+transport aborted at its deadline. Both are distinct from `client_error`, which
+is a route that refused rather than a route that was slow.
 
 The log contains no task, trajectory, bank, error, or reminder text. File creation,
 rotation, serialization, and injected sinks are all best effort; a read-only

package/docs/prompt-envelope-and-tools.md

@@ -1,7 +1,7 @@
 # Prompt Envelope and Tools
 
 > [!TIP]
-> **Interactive Spec Available:** An interactive dashboard is located at [docs/html/tools_blueprint.html](html/tools_blueprint.html) (Version: 0.3.8).
+> **Interactive Spec Available:** An interactive dashboard is located at [docs/html/tools_blueprint.html](html/tools_blueprint.html) (Version: 0.4.0).
 
 Clio Coder keeps the model-facing envelope stable and moves enforcement into the runtime registry and safety policy.
 
@@ -13,6 +13,24 @@ The chat loop compiles one provider-facing system prompt for a session. The comp
 
 The compiled prompt is reused byte-for-byte on ordinary submits. It recompiles only when that key changes or when config hot-reload invalidates the prompt cache. Path-scoped project rules can therefore recompile the prompt when a matching file enters working context. When recompilation changes the text, the session ledger records a `promptRecompiled` entry with the previous hash, new hash, and token estimate.
 
+## Section order: stable prefix first
+
+The compiled prompt lays its sections down in `SESSION_PROMPT_SECTION_ORDER` (`src/domains/prompts/compiler.ts`): identity, operating contract, delegation, skills, safety, tool contract, fleet, retrieval hints, project context, memory, runtime, then the operator-editable tail fragments (workspace root, Clio repo awareness, project rules, operator profile) in their own order.
+
+One rule fixes that list. A section goes as late as its volatility, and anything that reads a clock, a probe, or a mutable store goes after everything that does not. Every backend Clio targets caches by exact prefix and re-prefills from the earliest changed byte, so a section that can change between two turns must not sit ahead of sections that cannot. The runtime block is last of the compiled sections because its `Context window: N` moves when the backend reloads a model or a co-residency clamp lands; memory sits just ahead of it because an approved memory record rewrites that section mid-session; project rules are dead last because path-scoped rules join the prompt when a matching file enters working context.
+
+`Context window: N` is the window the backend will actually serve. A recorded loaded window outranks a probe, which reports a figure the target advertises without saying it is what is open, so a resumed session states the window its ledger measured rather than a re-probed server-wide number. Each prompt-manifest record carries that window and the layer that answered it (`contextWindow`, `contextWindowSource`) alongside a `version` for the prompt layout itself, so a recompile whose only cause was the window moving is explained by the record rather than inferred.
+
+`PROMPT_MANIFEST_VERSION` (`src/domains/session/prompt-manifest.ts`) is `2` as of this release, and the reordering above is what moved it. The field is additive: a record written by 0.3.8 carries no `version` and reads back as version 1, so a `prompt-manifest.jsonl` from an older session still parses. The rule for the field is that it tracks the layout rather than the inputs. Bump it when the compiled text moves for a reason other than a changed fragment, a changed tool surface, or a changed setting, so that a resumed session has the version in hand to explain the single `promptRecompiled` entry its first compile writes.
+
+### What not to add to the prefix
+
+Two additions look free and are not.
+
+The first is a terseness rule. It is tempting to cap the prose a model emits between tool calls, because that text is generated tokens on every hop of a long turn. Anthropic measured that exact change on Claude Code and reported a 3 percent quality regression, so a word-count or verbosity limit on inter-tool text is a bad trade: the tokens it saves are the cheapest ones in the turn, and the model's own narration of what it is about to do is load-bearing for what it then does. Bound tool results instead, where a single `grep` can cost thousands of tokens and the envelope caps already do the work.
+
+The second is anything that varies with the wall clock or the working tree. No timestamp, no `git status`, no branch name, no session id, and no run id belongs anywhere in the compiled prefix. Every backend Clio targets caches by exact prefix and re-prefills from the earliest changed byte, so one such field turns the whole prompt into a cache miss on every turn for no information the model could not have asked a tool for. On the sprint's measurement server that is a whole 2,778-token prompt re-prefilled at 2.6 s where the same change behind the stable sections cost 516 tokens and 0.72 s. Volatile facts belong in the user message, in a tool result, or in the runtime block, which is last for this reason.
+
 The disk fragments under `src/domains/prompts/fragments/` are layered by who reads them. `identity.clio` and `operating.contract` are constitutional: they render for every reader, name no tool, and state what is always true about Clio and her harness. `operating.delegation` (fleet coordination, receipts, spot-checks, shared `[worker result]` notes) renders only when `dispatch` is on the session's tool surface, and `operating.skills` (skill-shaped tasks, `/skill <name>` suggestions) only when `context` is; a fragment that teaches a tool is absent when the tool is, the same rule the Fleet block follows. `identity.docs-routing`, the directive to call `context(scope="docs")` before answering a question about Clio herself, follows the `context` gate too, while `identity.self-awareness` (installed paths, code outranks docs, configuration locations) names no tool and is unconditional. `operating.worker` (the assigned-task contract) renders only for dispatched workers, which never see the coordinator fragments. `safety.<level>` states what runs, what is approval-required, and what is blocked at the effective autonomy, in the safety net's action-class vocabulary (read, write, command, `system_modify`, `git_destructive`) and never by tool name, so the same body is true on every surface; the session and every worker read that one body, and what "approval-required" resolves to is the only role text (one operator confirmation for the session, the worker's `onPermission` routing for a worker).
 
 Prompt extensions can add dynamic fragments for project rules, the operator profile, and Clio source-tree awareness. Pending skill requests and middleware reminders are visible text in the user message, not hidden prompt machinery.

package/docs/provider-adapter-cookbook.md

@@ -1,7 +1,7 @@
 # Provider Adapter Cookbook
 
 > [!TIP]
-> **Interactive Spec Available:** An interactive runtime adapter descriptor builder and probe sequence capability checklist is located at [docs/html/provider_adapter_blueprint.html](html/provider_adapter_blueprint.html) (Version: 0.3.8).
+> **Interactive Spec Available:** An interactive runtime adapter descriptor builder and probe sequence capability checklist is located at [docs/html/provider_adapter_blueprint.html](html/provider_adapter_blueprint.html) (Version: 0.4.0).
 
 This cookbook guides developers through implementing custom model runtimes and inference server integrations within Clio Coder. It explains the runtime descriptor interfaces, probing protocols, model synthesis, and how to configure reasoning and thinking behaviors.
 

package/docs/release-cut-checklist.md

@@ -39,8 +39,14 @@ Run against the exact final candidate with `NO_COLOR` unset and
 7. `npm run ci` (runs 1 through 6)
 8. `npm run ci:release` (7 plus `scripts/check-release.mjs`: dist shebang
    integrity, version coherence between `package.json` and the top
-   `CHANGELOG.md` heading, the forbidden-file list, the required runtime
-   resources, and the tarball and unpacked size budgets)
+   `CHANGELOG.md` heading, the deterministic 26-scenario behavioral machinery
+   corpus against its checked baseline, the forbidden-file list, the required
+   runtime resources, and the tarball and unpacked size budgets). A baseline
+   mismatch prints reviewable evidence and names prompt- or recipe-affected
+   corpus results. For an intentional change, inspect that diff, run
+   `node benchmarks/eval/check-behavioral-release.mjs --update` (with `TMPDIR` on a disk-backed path if `/tmp` is a small tmpfs),
+   review `benchmarks/eval/behavioral-machinery-baseline.json`, and commit it
+   with the change.
 9. Optional: step 8 again under Node 24. Hosted CI gates on Node 22 alone,
    the `engines` floor; the weekly `flake-hunt` workflow carries Node 24.
    Repeat locally only when the cut touches runtime-sensitive code.
@@ -58,7 +64,17 @@ Run against the exact final candidate with `NO_COLOR` unset and
 12. Install that tarball into a clean temporary prefix with an empty
     `CLIO_CODER_HOME` and verify `--version`, `--help`, an empty-state non-TTY
     launch, `doctor`, and `uninstall --dry-run` without developer-local state.
-13. Interactive release testing, which this cut added because the release is
+13. Before interactive release testing, run the model-required public
+    behavioral corpus manually against the release target and built CLI:
+    `node dist/cli/index.js eval run --suite benchmarks/eval/behavioral-model.yaml --target mini --clio-coder-entry dist/cli/index.js`
+    and
+    `node dist/cli/index.js eval run --suite benchmarks/eval/behavioral-model-negative-control.yaml --target mini --clio-coder-entry dist/cli/index.js`.
+    Retain both Artifact v4 files as release evidence. The positive corpus must
+    report its scenario and role rows without an undeclared envelope mismatch;
+    the negative control must still record violated exploration and safety
+    labels. These model-dependent runs are manual and are never required by
+    ordinary deterministic CI. Continue with interactive release testing,
+    which this cut added because the release is
     almost entirely interactive surface: a tester agent drives the step-12
     install through real TUI sessions in a throwaway repository, one session
     per shipped feature, against local targets for the main session and a

package/docs/safety-model.md

@@ -1,7 +1,7 @@
 # Clio Coder Safety Model
 
 > [!TIP]
-> **Interactive Spec Available:** An interactive dashboard is located at [docs/html/safety_blueprint.html](html/safety_blueprint.html) (Version: 0.3.8).
+> **Interactive Spec Available:** An interactive dashboard is located at [docs/html/safety_blueprint.html](html/safety_blueprint.html) (Version: 0.4.0).
 
 Clio Coder's safety posture is code-enforced, not prompt-only. As the orchestrator coding agent in the [IOWarp](https://iowarp.ai) ecosystem developed by the [Gnosis Research Center](https://grc.iit.edu) at Illinois Tech under NSF Award [#2411318](https://www.nsf.gov/awardsearch/showAward?AWD_ID=2411318), Clio gates execution by target capabilities, the tool registry, the safety policy engine, project policies, protected-artifact checks, and audit receipts.
 
@@ -13,7 +13,7 @@ Source of truth: `src/domains/safety/**`, `src/tools/registry.ts`, `src/tools/bo
 
 The `autonomy` setting (`read-only` | `suggest` | `auto-edit` | `full-auto`) is an enforced dial. It controls exactly one thing: which action classes run immediately, which park for operator approval, and which are auto-denied. The safety net (damage-control rules, path policy, protected artifacts, loop guard, dispatch scope admission) is independent of the dial and identical at every level. When a `[safety-net]` notice appears at full-auto, that is the always-on net working as designed, not a contradiction of the level.
 
-In Clio Coder v0.3.7, effective autonomy resolution is strictly centralized in `src/entry/orchestrator.ts` through `resolveEffectiveAutonomy` and `resolveBaselineAutonomy`. Every admission surface (tool registry admission, dispatch plan provenance, and ACP session snapshots) delegates to this pair of functions so that fallback paths cannot diverge across execution contexts. `resolveBaselineAutonomy` evaluates dispatch settings overrides, headless CLI options, and configuration settings before applying the default `auto-edit` level. `resolveEffectiveAutonomy` combines any active ACP session autonomy level with the baseline resolution.
+In Clio Coder v0.4.0, effective autonomy resolution is strictly centralized in `src/entry/orchestrator.ts` through `resolveEffectiveAutonomy` and `resolveBaselineAutonomy`. Every admission surface (tool registry admission, dispatch plan provenance, and ACP session snapshots) delegates to this pair of functions so that fallback paths cannot diverge across execution contexts. `resolveBaselineAutonomy` evaluates dispatch settings overrides, headless CLI options, and configuration settings before applying the default `auto-edit` level. `resolveEffectiveAutonomy` combines any active ACP session autonomy level with the baseline resolution.
 
 ### Autonomy levels
 

package/docs/scientific-validation.md

@@ -1,11 +1,11 @@
 # Clio Coder Scientific Validation Contracts
 
 > [!TIP]
-> **Interactive Spec Available:** An interactive numerical tolerance calculator and HPC queue execution simulator is located at [docs/html/validation_blueprint.html](html/validation_blueprint.html) (Version: 0.3.8).
+> **Interactive Spec Available:** An interactive numerical tolerance calculator and HPC queue execution simulator is located at [docs/html/validation_blueprint.html](html/validation_blueprint.html) (Version: 0.4.0).
 
 Scientific software development cannot treat simple file presence as proof of correctness. A simulation script that crashes on rank 48, or writes out NetCDF arrays filled with `NaN`s, may still successfully write a file to the disk. 
 
-Clio Coder recognizes **scientific validation contract files** as an opt-in signal for a higher evidence bar. In v0.3.7, the session rigor resolver does not parse or enforce a scientific contract schema. The presence of `.clio-coder/validation.yaml`, `.clio-coder/validation.yml`, `validation.yaml`, `validation.yml`, or `VALIDATION.md` at the workspace root raises the default rigor level to `high`; the file contents are advisory material for developers, project agents, and external validators.
+Clio Coder recognizes **scientific validation contract files** as an opt-in signal for a higher evidence bar. In v0.4.0, the session rigor resolver does not parse or enforce a scientific contract schema. The presence of `.clio-coder/validation.yaml`, `.clio-coder/validation.yml`, `validation.yaml`, `validation.yml`, or `VALIDATION.md` at the workspace root raises the default rigor level to `high`; the file contents are advisory material for developers, project agents, and external validators.
 
 This advisory convention is separate from the executable project verifier catalog at `.clio-coder/verifiers.yaml`. The verifier catalog has a strict version-1 schema and admits exact argv vectors to the `verify` tool. Scientific validation contracts and handbook expectations do not grant command authority: prose such as `validators: ["python tools/check_grid.py"]` remains guidance until the project owner confirms the equivalent argv, cwd, timeout, and tags in `verifiers.yaml`. The executable catalog does not interpret numerical tolerances or artifact expectations; it only runs the explicitly declared process vector through safe-exec.
 
@@ -95,7 +95,7 @@ Comparing floating-point values in scientific computations must accommodate roun
 
 ## Common Scientific Artifact Families
 
-The following labels are useful project conventions for validation contracts and reports. They are not a closed, core-enforced enum in v0.3.7:
+The following labels are useful project conventions for validation contracts and reports. They are not a closed, core-enforced enum in v0.4.0:
 
 - **`HDF5` / `NetCDF` / `Zarr`:** Multi-dimensional scientific array files.
 - **`FITS`:** Flexible Image Transport System (used in astrophysics).

package/docs/session-lifecycle.md

@@ -1,6 +1,6 @@
 # Session Lifecycle
 
-This document is the authoritative specification for Clio Coder interactive and headless session lifecycles, on-disk ledger structures, tree-based conversation branching, checkpoints, and recovery protocols in `v0.3.7`.
+This document is the authoritative specification for Clio Coder interactive and headless session lifecycles, on-disk ledger structures, tree-based conversation branching, checkpoints, and recovery protocols in `v0.4.0`.
 
 Source implementations: `src/engine/session.ts` and `src/domains/session/`.
 

package/docs/skills-marketplace.md

@@ -1,7 +1,7 @@
 # Skills Marketplace
 
 > [!TIP]
-> **Interactive Spec Available:** An interactive dashboard is located at [docs/html/skills_blueprint.html](html/skills_blueprint.html) (Version: 0.3.8).
+> **Interactive Spec Available:** An interactive dashboard is located at [docs/html/skills_blueprint.html](html/skills_blueprint.html) (Version: 0.4.0).
 
 The Skills Hub (`/skill`) shows project skills, user skills, and the marketplace. Every marketplace row comes from the same local lookup that `clio-coder skills install <name>` and `/skill <name>` resolve through, so the hub lists nothing it cannot install.
 

package/docs/tool-usage.md

@@ -1,11 +1,11 @@
 # Tool Usage Reference
 
 > [!TIP]
-> **Interactive Spec Available:** An interactive seven-plane tool atlas and observation envelope truncation/offload calculator is located at [docs/html/tool_usage_blueprint.html](html/tool_usage_blueprint.html) (Version: 0.3.8).
+> **Interactive Spec Available:** An interactive seven-plane tool atlas and observation envelope truncation/offload calculator is located at [docs/html/tool_usage_blueprint.html](html/tool_usage_blueprint.html) (Version: 0.4.0).
 
 This is the deep usage reference behind the deliberately terse tool descriptions in the prompt envelope. Toolkit v2 keeps rich guidance out of tool descriptions and puts it here, where `context(scope="docs", query=...)` retrieves it section by section. Each tool below has its own self-contained `##` section covering the argument surface, defaults, truncation and continuation behavior, and concrete calls. Source of truth is `src/tools/`.
 
-In Clio Coder v0.3.7, `src/tools/agent-tools.ts` serves as the single agent-tool adapter across both orchestrator and worker runtimes. Both surfaces resolve their executable tools through the exact same `effectiveToolNames` narrowing, ensuring that attested tool schemas never drift from the tools available at runtime. Tools are keyed strictly by the `ToolName` union with no alias table. Argument leniency for weak-model callers is provided exclusively by per-tool `prepareArguments` normalizers declared on `ToolSpec`.
+In Clio Coder v0.4.0, `src/tools/agent-tools.ts` serves as the single agent-tool adapter across both orchestrator and worker runtimes. Both surfaces resolve their executable tools through the exact same `effectiveToolNames` narrowing, ensuring that attested tool schemas never drift from the tools available at runtime. Tools are keyed strictly by the `ToolName` union with no alias table. Argument leniency for weak-model callers is provided exclusively by per-tool `prepareArguments` normalizers declared on `ToolSpec`.
 
 ## Observation envelope: truncation notices, offload, next hints, and the turn budget
 
@@ -253,7 +253,7 @@ Arguments:
 - `cwd` (optional). Default agent working directory.
 - `timeout_ms` (optional). Aborts the whole dispatch; in sequential mode remaining tasks are skipped and the skip is reported.
 - `briefing` (optional string, top-level default or per-task override). Parent-composed context/data, not worker instructions: it cannot replace `task`. It is trimmed and omitted when blank, rejected above 12,000 UTF-8 bytes, sent as its own delimited untrusted dynamic message, and retained only as byte/hash provenance. The shared value applies to string tasks and object tasks without an override; an object-level briefing wins.
-- `intent` (optional object, top-level default or per-task override). Declares `read_roots`, `write_roots`, `relevant_paths`, `expected_outputs`, and `verification`. Path arrays contain normalized repository-relative POSIX paths. Verification entries contain a declared `check` id and optional `timeout_ms`; ids are resolved from package scripts and `.clio-coder/verifiers.yaml` before approval. Checks are ids, not shell commands.
+- `intent` (object, top-level default or per-task override, and the default way to dispatch). Declares `read_roots`, `write_roots`, `relevant_paths`, `expected_outputs`, and `verification`. Path arrays contain normalized repository-relative POSIX paths. Verification entries contain a declared `check` id and optional `timeout_ms`; ids are resolved from package scripts and `.clio-coder/verifiers.yaml` before approval. Checks are ids, not shell commands. Declared paths select the project rules that apply to them and pin worker context; omitting `intent` falls back to reading path-like tokens out of the task and briefing, which can miss an applicable rule. In a batch, per-task `intent` shallow-merges over the top-level object and is then checked against it as a ceiling: a task may narrow the shared scope and is refused with `intent_scope_widening` if it reaches outside. Declaring `write_roots` that disagree with a legacy `writeRoots`, an `expected_outputs` entry outside every declared write root, or an `intent.version` other than 2 are each terminal refusals carrying a stable reason code. See [dispatch-typed-intent.md](dispatch-typed-intent.md).
 - `gate` (optional string, top-level default or per-task override). Exact shorthand for `intent.verification=[{check: gate}]`. Supplying it together with `intent.verification` is refused.
 - `max_output_bytes` (optional). Summary byte budget; default 20000, split across runs with at least 1024 bytes each.
 
@@ -269,13 +269,23 @@ Sealed receipts are the durable evidence; worker prose remains advisory until ve
 
 ```text
 dispatch(list=true)
-dispatch(agent="debugger", task="Adversarially verify the strict v19 receipt boundary", briefing="Prior receipt R1 cited receipt-integrity.ts and left these claims unresolved", detach=true)
-dispatch(tasks=["Run the contract tests in tests/contracts/dispatch.test.ts and report each failure with its assertion"])
+dispatch(agent="debugger", task="Adversarially verify the strict v19 receipt boundary", briefing="Prior receipt R1 cited receipt-integrity.ts and left these claims unresolved", intent={read_roots: ["src/domains/dispatch/"]}, detach=true)
 dispatch(tasks=[
-  {agent: "researcher", task: "Map every caller of finalizeObservation and summarize the envelope shapes"},
-  {agent: "coder", task: "Fix the failing assertion in tests/contracts/safety.test.ts", intent: {write_roots: ["tests/contracts"], verification: [{check: "test"}]}}
+  {task: "Run the contract tests in tests/contracts/dispatch.test.ts and report each failure with its assertion",
+   intent: {read_roots: ["tests/contracts/", "src/domains/dispatch/"], verification: [{check: "test"}]}}
+])
+dispatch(tasks=[
+  {agent: "researcher", task: "Map every caller of finalizeObservation and summarize the envelope shapes",
+   intent: {read_roots: ["src/domains/"]}},
+  {agent: "coder", task: "Fix the failing assertion in tests/contracts/safety.test.ts",
+   intent: {write_roots: ["tests/contracts/"], expected_outputs: ["tests/contracts/safety.test.ts"], verification: [{check: "test"}]}}
 ], mode="parallel")
-dispatch(tasks=["Refactor step 1", "Refactor step 2"], mode="sequential", timeout_ms=600000)
+dispatch(
+  intent={read_roots: ["src/"], write_roots: ["src/domains/"]},
+  tasks=[
+    {task: "Refactor step 1", intent: {write_roots: ["src/domains/dispatch/"]}},
+    {task: "Refactor step 2", intent: {write_roots: ["src/domains/context/"]}}
+  ], mode="sequential", timeout_ms=600000)
 ```
 
 ## verify: run declared verification checks

package/docs/trace-store.md

@@ -1,7 +1,7 @@
 # Trace store contract
 
 > [!TIP]
-> **Interactive Spec Available:** An interactive trace database viewer, schema inspector, and SQL query validator simulator is located at [docs/html/trace_blueprint.html](html/trace_blueprint.html) (Version: 0.3.8).
+> **Interactive Spec Available:** An interactive trace database viewer, schema inspector, and SQL query validator simulator is located at [docs/html/trace_blueprint.html](html/trace_blueprint.html) (Version: 0.4.0).
 
 Clio's trace database is a rebuildable, queryable mirror. Receipts, session
 ledgers, gate artifacts, and evidence remain the source of truth. Removing