specpi 0.29.0 → 0.30.1

package/CHANGELOG.md

@@ -1,5 +1,51 @@
 # Changelog
 
+## Unreleased
+
+## 0.30.1 - 2026-09-22
+
+- Repair Jev's task context: read the active contract structurally, fall back to the bounded current request, and clear stale context on task, branch and session changes.
+- Allocate the existing 1 KiB state budget by field. Preserve result samples and candidate/question correspondence; abstain locally when required evidence cannot fit. Record sample coverage and local refusals separately from classifier calls.
+- Renew transmission consent with an accurate disclosure of sampled file contents, command output and request text. Redact question text as well as state, replace outside-workspace absolute paths, and use opaque source/cluster IDs. Consent now binds the transport scheme as well as the host.
+- Rank delegation sources per `packet.jobs[]`, respecting the declared mode and preserving every selected source and ungated position. Store gap-triage opinions alongside the original observation, with a bounded existing-cluster shortlist. Opinions cannot change priority, canonical identity or human improvement authority.
+- Retain object-shaped Score distributions, reject malformed probabilities and require a Choice distribution before applying its margin gate. Separate warnings from real elisions and count delivered effects rather than undelivered advice. Shell results are no longer eligible for shortening because their commands may not be safe to repeat.
+- Close review findings at asynchronous boundaries: bind reports before consent and root lookup, recheck collection before sending and advisory authority inside locked writes, and abandon stale tool-result lookups. Reject transport redirects, redact quoted credentials and multiline keys before sampling, and retain committed-effect accounting when later notifications or report rendering fail.
+- Keep all advisor systems off by default, production thresholds unchanged, compaction guidance withdrawn and the command guard separate. These repairs do not establish a task-quality or cost benefit; new controlled evaluations are still required.
+- Isolate the base-package installer tests' home directory as well as their agent directory. The old fixtures could switch off the real command guard during validation; a regression now checks that a synthetic caller's enabled guard remains byte-for-byte unchanged.
+- Trim the evaluations page to results, a short method and chart notes, without changing the published data. Add expandable diagrams for the six Jev systems, with theme-aware colors and contained scrolling on narrow screens.
+
+## 0.30.0 - 2026-09-21
+
+- Replace the evaluations page with Terminal-Bench 2.0. The suite that filled it was written in this repository, and it was retired for the reason it kept reporting: nearly every harness passed nearly every task, so the thing it was built to measure was the thing it could not see, and three rounds of harder tasks did not move it. A suite cannot mark its own homework. The page now carries somebody else's benchmark on tasks nobody here chose -- Pi with no harness, SpecPi + Jev, Oh My Pi and Claude Code, 334 scored attempts on one model through one endpoint.
+- Report the result the outside benchmark actually gives, which is that it does not separate the harnesses either. Solve rate spans 0.767 to 0.838 across the four arms; Pi against SpecPi + Jev is Fisher p = 0.83, and exactly 30/39 each on the slice the comparison rests on. That is the second suite to find no difference in capability, and this one's tasks were not chosen here and are failed about a fifth of the time, so the usual explanation -- that the tasks are too easy -- does not cover it.
+- Make spend the column the page is about, because it is the one that separates. SpecPi + Jev reaches the same solve rate as bare Pi on 26% fewer prompt tokens and 23% less cost per attempt, and cost per attempt spans 1.7x across the three rows whose cached share could be measured. Score is the control here rather than the finding.
+- Publish the run unfinished, and say so in the first paragraph. Terminal-Bench 2.0's hard split and a SWE-Gym slice have not run, plain SpecPi is not an arm so nothing separates the harness from the advisory layer, and Oh My Pi's widened slice holds six attempts per task rather than three because two launches of the same script raced into one output directory. Section 07 of the page lists each of these rather than leaving the reader to infer them from a table that looks complete.
+- Publish Claude Code's cached share as absent rather than as zero. Its traffic crosses the Messages/chat-completions translation, which recorded no cached tokens at all on these runs, and a zero would have read as "Claude Code destroys cache efficiency" -- which the run does not show, since Pi and SpecPi see 94% on the same endpoint in the same sitting. Its cost is published as an upper bound for the same reason, because the cached share is what would discount it.
+- Map Anthropic's cache tokens onto the pair Anthropic reports them as. `scripts/eval-anthropic.mjs` was passing the provider's prompt total through as `input_tokens` and emitting no `cache_read_input_tokens`, but Anthropic counts cache reads outside the input total rather than inside it, so the two have to split the prompt. This is the fault above, fixed; the rows it spoiled predate it and are labelled rather than silently re-priced, because the tokens are not recoverable from an archived aggregate.
+- Count auxiliary calls as spend. `modelRequests()` answers how many turns an attempt took, and the token totals were reading the same filtered list, so the session-title call Claude Code makes on every attempt cost nothing in any report. Billing and conversation counting are separate questions now: totals and pricing cover the auxiliary calls, while the turn series and the per-turn figures stay conversation-only, because an auxiliary is not a turn.
+- Fail a Responses reply the provider reported as failed. `scripts/eval-responses.mjs` inspected only `incomplete_details`, so an HTTP 200 stream carrying `response.failed` was assembled into an ordinary reply and returned with a `stop` finish reason -- a provider failure arriving at the harness as a short answer, which is the one shape that cannot be told from the harness giving up. The proxy returns 502 with the provider's own code and message instead. `max_output_tokens` stays a finish reason rather than a failure, because that turn happened and its text is real.
+- Admit a row to the controlled-window comparison only when the window was enforced. `scripts/tier6-metrics.mjs` accepted any attempt that had compacted at all, which is evidence that the harness compacts and not that it compacted at the declared window: an 80,000-token attempt with one compaction was being compared against attempts held to 24,000. The bound is measured rather than picked -- at that window, enforced attempts peak at 1.58x it and provider-default attempts start at 1.89x -- and it excludes rather than admits when it is unsure.
+- Let an eval run drop one part of the SpecPi layer. `SPECPI_EVAL_ABLATE` takes `agents`, `tools` and `goal`, combinable, so a run can strip the harness to Pi's exact tool surface and measure what that alone accounts for. It was built to test a specific claim of mine -- that SpecPi's four extra tools explained a collapse on one model -- and it refuted it, which is the use an ablation is for.
+- Delete the retired suite's publishers rather than leaving them aimed at sections that no longer exist. `scripts/tb-metrics.mjs`, `scripts/tb-site.mjs` and `scripts/tier6-site.mjs` go with the three data files they wrote. `scripts/eval-site.mjs` keeps the part with other callers -- reading run reports, and the chart and table renderers the new page draws with -- and loses the page-specific rendering it no longer has a page for. `scripts/tier6-metrics.mjs` still runs, writing beside the runs it describes instead of into a published directory.
+
+- Add Claude Code to the eval suite as `--harness=claude-code`. It is the first harness that speaks the Anthropic Messages API, and the provider the whole suite runs against is chat-completions only, so measuring it at all meant translating both directions. `scripts/eval-anthropic.mjs` does that and nothing else.
+- Translate at the edge rather than throughout. The request becomes a chat-completions request before the proxy records anything, and the reply is converted back only on the way out, so `summarizeRequest`, `toolOutcomeOf`, `extractUsage`, `extractToolCalls` and the cost pipeline all keep reading the one shape they have always read. The alternative -- teaching each of them a second shape -- is six places to drift instead of one place to read.
+- Rebuild the stream rather than piping it. `startProxy` already buffers the whole upstream reply before answering, so there is no streaming fidelity to lose: the OpenAI deltas are collected into one finished message and re-emitted as a well-formed Anthropic event sequence. `tests/eval-anthropic.test.mjs` covers the cases where a wrong conversion would be scored as the harness failing the task rather than as the suite losing its work -- several tool results in one turn, a failed tool result, fragmented tool arguments, and arguments that never parse.
+- Read no Anthropic credential and spend none. Claude Code normally authenticates against a stored subscription login, so a run against a real configuration could fall back to it and bill a subscription for an eval. The row uses a disposable `CLAUDE_CONFIG_DIR` with no login to fall back on, points `ANTHROPIC_BASE_URL` at the logging proxy, and the proxy discards the client's token and sends `EVAL_FORWARD_KEY` upstream. Runs bill the same OpenCode Go subscription as every other harness; the probe confirms it, reporting `apiKeySource: none`.
+- Answer the reachability probe instead of recording it. Claude Code sends `HEAD /api/hello` before it starts. It carries no body and names no model, so recording it added a turn with no tokens, and forwarding it posted an empty payload to the provider's completions endpoint and billed for the reply.
+- Keep the session-title call out of the turn count. A Messages request carrying no tool schema is the harness titling its own session, which is the rule `conversationSummary` has always used to find the real first call; it is now tagged so its spend stays in the cost column while it stops inflating turns. Claude Code is the first proxy harness to make one -- OpenCode makes one too but reports its own per-step usage rather than going through this log. Uncorrected, a single-prompt attempt read as three turns instead of one.
+- Stop losing a finished attempt to a directory that will not delete. On Windows a directory stays locked while any process holds it as a working directory, and a harness that leaves a helper alive for a moment after its own exit keeps the attempt's workspace locked with it. The attempt's result is already computed by then, so throwing discarded a real measurement to report a temporary file the operating system cleans up anyway. It now retries for longer, then says so and moves on.
+- Note what this row cannot do. Claude Code exposes no context-window setting -- its compaction triggers off the model's own window -- so it cannot be held to a tier's declared window, and tier 6 reads its attempts as unwindowed the way it did for Codex and OpenCode before those learned to read one.
+- Withdraw compaction guidance from the Jev layer. It was the layer's only system that had ever applied a verdict at scale, and the only measurement of it says it cost solve rate. Tier 6 ran the pair comparison twice, months apart on two versions: SpecPi 0.28.0 gave 5/8 against plain SpecPi's 8/8 on the long-session task, and 0.29.0 gave 3/8 against 6/8. Pooled that is 8/16 against 14/16, Fisher exact p = 0.054. The shorter task showed nothing either time. `extensions/jev-advisor/questions/compaction.mjs`, both hooks that used it, its gate thresholds and its budget are gone, and the managed set drops with them.
+- Attribute the loss before removing the thing. Over those attempts `compaction` accounted for 139 of the 142 verdicts the layer applied; `retention` applied nothing across 287 calls and `progress` applied three. Whatever separated the two arms was almost entirely this one system's doing, which is what made a removal defensible on a comparison that never reached significance.
+- Say plainly what this does not establish. p = 0.054 is not significance. A within-arm check of whether heavier compaction predicts failure reaches only p = 0.251. The mechanism was never isolated: the system could not delete anything -- it appended one sentence to the summariser's prompt, leaving Pi's own cut point and token budget alone -- so any harm ran through a summary that kept the wrong material, most likely because the sentence told the summariser to *prioritise* one kind of content on a task built so that a fact from the first turn must survive. What carried the decision was three weak signals agreeing and the asymmetry of the bet, not a result.
+- Take the `/tree` branch labels with it. They rode the compaction switch by design, on the reasoning that someone who has decided the advisor may steer a summary has decided that once, and the same `decide()` that built the harmful sentence also fed the branch-summary path. Nothing measured the labels either way; they are removed because the machinery they shared is.
+- Bump the settings schema to 5 and drop both `systems.compaction` and `budgets.compaction` on the way through, the way schema 4 dropped the guard keys. Nothing anyone had switched on is being switched off in any meaningful sense: the hooks those keys gated no longer exist, so a retained preference could only describe a system that cannot run. The panel in SpecPi Chat drops the row with it, and a test pins the panel's shape to the advisor's so the two cannot drift.
+- Keep the runs that led here in the record rather than deleting them. The tier-6 tables carried the compaction row and both pair comparisons until this release retired that page; they remain in this repository's history, and the Jev page points at this changelog for the comparison and the p-values. A measurement that led to a removal is the reason the removal can be defended, so it outlives the page it was published on.
+- Grade tier 1 and 2 answers by content rather than by the line endings they were written with. A file written through a Windows shell redirection ends its lines CRLF and one written through a file tool ends them LF; six checkers compared bytes, so they failed correct work for the route it took. Five of the six tier 1-2 failures in the published DeepSeek run were this, all on Codex, which published Codex at 30/37 and a 0.742 score against the 35/37 and 0.862 the regrade measures. Files a task says not to touch stay byte-exact, because rewriting one with different endings is still rewriting it.
+- Supersede eval cells rather than pooling them. Each attempt's workspace is discarded, so a cell cannot be re-graded, only re-run; the site now keeps one cell per harness, task and tier and takes the last report that measured it. Without that rule the regrade above would have been averaged with the readings it corrects.
+- Reach a provider on the wire it actually speaks. Muse Spark serves only the Responses API and returns 503 "Endpoint is unavailable" to a chat-completions request, as gpt-5.6-luna and grok-4.6 do; Codex already spoke Responses and reached it unaided, so the model was measurable on one row out of eight. `scripts/eval-responses.mjs` translates at the edge, the seam `eval-anthropic.mjs` already uses, and `EVAL_FORWARD_WIRE` declares the wire rather than probing for it, so latency is not doubled and the wire stays a property of the run. The report records which wire was used and the method says so, because seven rows having been translated is a material fact about how they were measured.
+- Mint an OpenCode session only for the endpoint that routes on one. The gate was on forwarding rather than on the endpoint, so a run against any other provider spent an OpenCode call per attempt, billed its tokens to the run, and needed a login the run was not otherwise using, to produce an id the provider ignores. The published method claimed every forwarded run minted one, which was never true of a direct provider.
 ## 0.29.0 - 2026-09-20
 
 - Return the command guard to `specpi-jev-guard`, and leave it there. 0.28.0 built the guard into the layer as its eighth system, for three stated reasons that were properties of the package as it stood: a global configuration file with no session scope, a key read from the environment only, and a fail-closed posture. Two of the three are fixed upstream. Guard 0.3.0 ships `/jev-guard setup | on | off [--global] | check | model | backend`, so it has a session scope and a switch of its own, and it resolves Pi's saved login before the environment, so `/login openrouter` serves it exactly as it serves the advisor. What remains is fail-closed, which is a disclosed trade rather than a defect. `extensions/jev-advisor/risk.mjs` and `questions/guard.mjs` are gone, the package is pinned again, and the base is eight.

package/NPM_RELEASE.md

@@ -6,7 +6,7 @@ Publishing, tags, deprecation, ownership changes and GitHub Releases require exp
 
 ## Prepare
 
-1. This candidate is SpecPi **0.29.0**, paired with the Chat **0.14.0** VSIX. Browser QA **0.3.0** is already independently published; do not republish or edit its source as part of this release. Choose an unused version; update `package.json`, the dated `CHANGELOG.md` entry, README, and the website. Check Chat's version and VSIX download links. Preserve historical changelog entries.
+1. This candidate is SpecPi **0.30.1**, paired with the Chat **0.14.0** VSIX. Browser QA **0.3.0** is already independently published; do not republish or edit its source as part of this release. Choose an unused version; update `package.json`, the dated `CHANGELOG.md` entry, README, and the website. Check Chat's version and VSIX download links. Preserve historical changelog entries.
 2. Validate:
 
     ```sh
@@ -20,7 +20,7 @@ Publishing, tags, deprecation, ownership changes and GitHub Releases require exp
     git diff --check
     ```
 
-    Installer/Pi lifecycle tests must use disposable state, never a live profile. `check:base` requires network access and verifies the real six-package base with isolated home/configuration paths. The base check now invokes the installed Browser QA Node bin for Chromium setup and offline doctor checks; scope `PLAYWRIGHT_BROWSERS_PATH` to a disposable test cache. Verify setup skip, core-only behavior, failure rollback, and BetterWright ownership migration in the installer regressions. Browser-cache/package bytes are outside managed rollback and uninstall. Review all default version changes, upstream lifecycle scripts, compatibility, and notices in `THIRD_PARTY.md`. Local dry runs disable provenance because they lack GitHub OIDC.
+    Installer/Pi lifecycle tests must use disposable state, never a live profile. `check:base` requires network access and verifies the real eight-package base with isolated home/configuration paths. The base check now invokes the installed Browser QA Node bin for Chromium setup and offline doctor checks; scope `PLAYWRIGHT_BROWSERS_PATH` to a disposable test cache. Verify setup skip, core-only behavior, failure rollback, and BetterWright ownership migration in the installer regressions. Browser-cache/package bytes are outside managed rollback and uninstall. Review all default version changes, upstream lifecycle scripts, compatibility, and notices in `THIRD_PARTY.md`. Local dry runs disable provenance because they lack GitHub OIDC.
 
 3. Review the diff, exact package manifest and artifact; obtain fresh read-only review for lifecycle, permissions and packaging changes.
 4. After approval and passing PR checks, merge to `main`, then create the matching `v<version>` tag at the merge commit. Build Chat's VSIX with `npm --prefix vscode run package` and attach it to the GitHub Release. Publish the release to start the npm workflow and approve the protected `npm` environment if it requests review. Verify the website deployment and download links. If publication cannot complete, revert the release merge and use a new version for the next attempt.

package/README.md

@@ -28,7 +28,7 @@
 
 ---
 
-SpecPi is a small starting point for the [Pi coding agent](https://pi.dev/). It is one opinionated setup for how the agent should work, not a marketplace of plugins.
+SpecPi 0.30.1 is a small starting point for the [Pi coding agent](https://pi.dev/). It is one opinionated setup for how the agent should work, not a marketplace of plugins.
 
 At the center are two built-in extensions. **Scope control** keeps each task to the files it said it would touch. The **improvement loop** turns repeated friction into small, tested changes to the setup, instead of letting prompts and workarounds pile up. Around those are eight hand-picked packages, each locked to an exact version and checked before anything installs, plus **SpecPi Chat**, a VS Code panel for working alongside the agent.
 
@@ -73,36 +73,34 @@ The gap between the two SpecPi bars comes from a few separate switches, so the e
 ## Harness evaluations
 
 The chart above counts characters. It says nothing about what a harness costs to
-actually use, or whether it finishes the job. That is what the eval suite is for:
-the same tasks, the same model and the same frozen price list, with only the
-harness changing.
+actually use, or whether it finishes the job. That is what the evals are for:
+[Terminal-Bench 2.0](https://www.tbench.ai/), the same model and the same frozen
+price list, with only the harness changing.
 
 <!-- eval-summary -->
 
-**259 attempts across 7 harnesses and 24 tasks**, all on `deepseek-v4.1-flash`.
+**334 scored attempts across 20 tasks and 4 harnesses**,
+all on `deepseek-v4.1-flash`.
 
-| Harness | Solved | Cost/attempt | Prompt tokens | Sent before any work |
-| --- | --- | --- | --- | --- |
-| Pi | 34/37 | $0.0038 | 112,681 | 5,522 |
-| SpecPi | 35/37 | $0.0042 | 134,859 | 13,799 |
-| OpenCode | 31/37 | $0.0043 | 143,830 | 38,097 |
-| SpecPi + Jev | 35/37 | $0.0046 | 153,077 | 13,799 |
-| Codex CLI | 30/37 | $0.0067 | 242,031 | 38,148 |
-| Oh My Pi | 35/37 | $0.0070 | 257,783 | 62,612 |
-| DeepSeek Harness | 35/37 | $0.0078 | 281,676 | 31,750 |
+| Harness | Solved | Rate | Cost/attempt | Prompt tokens | Cache hit |
+| --- | --- | --- | --- | --- | --- |
+| SpecPi + Jev | 62/74 | 0.838 | $0.0125 | 381,153 | 94.5% |
+| Pi (base) | 60/74 | 0.811 | $0.0161 | 512,903 | 93.7% |
+| Oh My Pi | 90/113 | 0.796 | $0.0216 | 1,103,136 | 97.4% |
+| Claude Code | 56/73 | 0.767 | $0.1128 or less | 630,410 | not measured |
 
-Cost is the harness's own model spend, priced from recorded usage against a
-dated price file. The last column is the tool schema plus system instructions
-riding every single request, which is the fixed toll a harness charges before
-the model does anything.
+Solve rate does not separate these harnesses -- Pi against SpecPi + Jev is Fisher
+p = 0.83 -- so the column worth reading is what each one spent reaching the same
+answers. Cost is recomputed from recorded tokens against a dated price file, never
+taken from a harness's self-report. Claude Code's cached share was not recorded on
+this run, so its cost is an upper bound.
 
 <!-- /eval-summary -->
 
-Read the method, the charts and the caveats on the
-[evaluations page](https://tannermidd.github.io/SpecPi/evaluations/). The runner,
-the tasks and the recorded results are in [evals/](evals/), and this table is
-regenerated from those reports by `node scripts/eval-site.mjs`, so it cannot
-drift from the published figures.
+This run is still in progress. See the results and brief method on the
+[evaluations page](https://tannermidd.github.io/SpecPi/evaluations/). The table above is regenerated
+from the run data by `node scripts/tb2-site.mjs`, so it cannot drift from the
+published figures.
 
 ## Install
 
@@ -123,7 +121,7 @@ Full setup options, package details, and requirements: [website](https://tannerm
 
 | | |
 | --- | --- |
-| [Packages](https://tannermidd.github.io/SpecPi/#packages) | The seven pinned packages and what each provides |
+| [Packages](https://tannermidd.github.io/SpecPi/#packages) | The eight pinned packages and what each provides |
 | [Scope control](https://tannermidd.github.io/SpecPi/wiki/#scope) | `/scope` commands and drift monitoring |
 | [Improvement loop](https://tannermidd.github.io/SpecPi/#loop) | Local wishlist, `/harness-improvement`, and retirement with evidence |
 | [SpecPi Chat](https://tannermidd.github.io/SpecPi/#vscode-chat) | VS Code frontend and VSIX install · [Chat guide](https://github.com/TannerMidd/SpecPi/blob/main/vscode/README.md) |

package/SECURITY_MODEL.md

@@ -18,19 +18,23 @@ Optional tool groups ship withdrawn. The `request_capability` tool lets the mode
 
 The advisor is the only part of SpecPi that sends anything off this machine, and it ships off. With the master switch off there is no network call, no key read, no consent read and no prompt injection: the harness behaves exactly as it did before the extension existed.
 
-What is sent: a summary object of at most 1 KB per call to `openrouter.ai`, which is where Jev is published and the default route, or to `api.typesafe.ai` when `JEV_BACKEND=typesafe` selects the direct API. Either way, over HTTPS. It carries tool names, byte counts, relative paths, entry counts, short descriptions and a bounded sample of lines from the material being judged. `sanitize.mjs` is the only module permitted to build one, so the rule is enforced in one place rather than promised in several: credentials, tokens, JWTs, emails and URLs are redacted, absolute paths are relativized, and anything still over budget is truncated rather than sent. Redaction reduces exposure; it does not guarantee that a model-authored string carries nothing sensitive.
+What is sent: a summary object of at most 1 KB per call to `openrouter.ai`, which is where Jev is published and the default route, or to `api.typesafe.ai` when `JEV_BACKEND=typesafe` selects the direct API. Either way, over HTTPS. It carries tool names, byte counts, relative paths, entry counts, short descriptions and a bounded sample of lines from the material being judged. `sanitize.mjs` builds the wire state and sanitizes question text. Known credential, token, JWT, email and URL patterns are redacted, including quoted credential values and multiline private-key blocks before sampling; workspace paths become relative and other absolute paths are replaced. Source and cluster questions use opaque IDs, with their descriptions inside the state budget. Required fields have explicit allocations: samples may be shortened, but missing objectives, missing samples or candidate lists that cannot fit cause local abstention, not an evidence-free classifier call. Redaction reduces exposure; it does not guarantee that a model-authored string carries nothing sensitive.
 
-Earlier versions of this document said file contents and command output were "refused outright". That was never accurate and is corrected here. Deciding whether a tool result is spent, or whether a fetched page is addressing the agent, cannot be done from byte counts alone, so `outline()` has always sent a sample of the result's own lines: up to six from the head and two from the tail, each collapsed to at most 80 characters and passed through the same redaction as everything else. As of 0.27.0 it also samples up to four lines evenly spaced through the middle, because an instruction planted in a fetched page is rarely in its first six lines and a digest that could never contain one would ask system 7 a question its own state made unanswerable. So the accurate statement is: **a bounded, redacted sample of at most twelve short lines per result, inside a 1 KB total budget** — not the whole content, and not none of it. What the sample cannot include is anything beyond that budget, which is enforced by truncation rather than by intention.
+Earlier versions of this document said file contents and command output were "refused outright". That was never accurate and is corrected here. Deciding whether a tool result is spent, or whether a fetched page is addressing the agent, cannot be done from byte counts alone, so `outline()` has always sent a sample of the result's own lines: up to six from the head and two from the tail, each collapsed to at most 80 characters and passed through the same redaction as everything else. As of 0.27.0 it also samples up to four lines evenly spaced through the middle, because an instruction planted in a fetched page is rarely in its first six lines and a digest that could never contain one would ask system 7 a question its own state made unanswerable. So the accurate statement is: **a bounded, redacted sample of at most twelve short lines per result, inside a 1 KB total budget** — not the whole content, and not none of it. What the sample cannot include is anything beyond that budget. The ledger records offered and retained sample counts and whether required evidence was present; that does not establish that the sample contains every relevant fact.
 
-Consent is separate from the switch. The first time any system would transmit, an interactive dialog names the endpoint, the shape of the data and the byte budget. Without an interactive human nothing is sent, so an unattended run cannot be the thing that starts transmitting. The grant lives in `<agent-dir>/specpi/jev/consent.json` and is bound to the host it was given for, so switching backends asks again rather than carrying an old answer to a new destination. `/jev forget` revokes it.
+Consent is separate from the switch. The first time any system would transmit, an interactive dialog names the endpoint, the shape of the data and the byte budget. Without an existing valid grant, a noninteractive run cannot start transmitting. A previously granted consent may be reused without a UI. The grant lives in `<agent-dir>/specpi/jev/consent.json` and is bound to the destination origin, including its transport scheme. Transport rejects redirects rather than forwarding evidence to an unapproved destination. Schema 2 requires renewed consent because the earlier dialog incorrectly promised never to send file contents or command output. It now explicitly discloses sampled content, the current request/objective, best-effort redaction and any non-HTTPS endpoint override. `/jev forget` revokes it.
 
-Every call appends one line to `<agent-dir>/specpi/jev/transmissions.jsonl`: timestamp, system, question keys, state byte count, a SHA-256 of the exact payload, latency and the gated outcome. The payload itself is never written. `/jev ledger` reads it back, so "only digests are sent" is checkable rather than asserted. The ledger is user data and survives uninstall.
+Every advisor request attempt appends one line to `<agent-dir>/specpi/jev/transmissions.jsonl`: timestamp, system, question keys, state byte count, a SHA-256 of the sanitized state and questions, latency and the gated outcome. Local evidence refusals are marked `sent: false`, consume no call budget, and are counted separately from calls. Effect tags distinguish shortening, warnings, recorded assessments, notifications, capability proposals and queued steering; queueing does not prove the model consumed a message. The payload itself is never written. `/jev ledger` reads it back, so "only digests are sent" is checkable rather than asserted. The ledger is user data and survives uninstall.
 
 The advisor also keeps `<agent-dir>/specpi/jev/usage.json`, the running call count for the current session, written atomically with owner-only permissions. It exists because the ledger has no session boundary in it, so nothing outside the advisor's own process could say what *this* session had spent, and the count is what SpecPi Chat shows beside each budget. It is the one file in this layer meant to be read by another process, and it is deliberately the least interesting one: counts per system, the budgets they are counted against, a session identifier and two timestamps. No state, no questions, no answers, not even the ledger's digests. It is written only while the master switch is on, so a layer nobody has enabled leaves no trace of having been installed, and the last session's counts are kept rather than deleted at shutdown, because "this has never run" and "the session that just ended spent its whole budget" are different facts and a reader should be able to tell them apart. One file serves the directory, as the settings file does, so where several Pi sessions share an agent directory it describes whichever wrote to it last; it carries a session identifier, two timestamps and an active flag so a reader can say which rather than having to assume.
 
-The advisor holds no authority. It never grants a capability, never calls a tool and never allows one. Its only blocking action is to refuse a capability-gap report that looks like it carries a credential, which asks the model to rewrite its own text and discards nothing. Failure is silent, not closed: a timeout, HTTP error, missing key, missing consent, an exhausted per-system or session budget, or ungated confidence produces no advice, and the existing code path runs unchanged. Tool-result retention can shorten a large read-only result before it is appended; it never alters a write, edit or error result, and the replacement says the output is recoverable by re-running.
+The advisor holds no authority. It never grants a capability, never calls a tool and never allows one. Its only blocking action is to refuse a capability-gap report that looks like it carries a credential, which asks the model to rewrite its own text and discards nothing. Failure is silent, not closed: a timeout, HTTP error, missing key, missing consent, an exhausted per-system or session budget, or ungated confidence produces no advice, and the existing code path runs unchanged. Tool-result retention can shorten a large observational result before it is appended; it never shortens a write, edit, shell or error result. Shell success does not establish that a command is read-only or safe to repeat. Other replacements still suggest re-reading; preserving an immutable original result is not implemented.
 
-Two of the seven systems do something the other five do not, and both are off by default like everything else here.
+Task context comes from the workflow owner's validated active contract, not a rendered heading or a search through stored conversations. A bounded current-request fallback is refreshed when the task changes and cleared on branch/session changes. Delegation ranking adapts `packet.jobs[]` independently, uses each declared mode, and preserves the exact source multiset and ungated positions.
+
+Gap triage runs only after local collection is enabled. Reports bind their task and session before any awaited consent or root lookup; collection is rechecked before transmission. Locked writes recheck task authority and strip the advisory assessment if Jev was disabled while waiting, without preventing otherwise authorized local collection. A bounded shortlist comes from sanitized wishlist observations, never from Pi state. A persisted `assessment` is a Jev opinion based on the reporting model's account, **not independent evidence**. It cannot change the original impact or suggested fix, canonical identity, priority, qualification, selection or retirement. Suggested cluster matches require a human merge decision. Model-authored report arguments cannot inject these assessments.
+
+Two of the six systems can add fixed advice to model context, and both are off by default like everything else here.
 
 **System 5, progress detection**, is the only one that can change what the model does. On a confident verdict that a session has stopped making progress it adds one fixed, code-written line at a turn boundary. It ships set to `notify`, which tells the person and cannot change the session; `message`, which appends the line for the model to read, is a deliberate opt-in. It is written once per session and never retracted, it never aborts or blocks, and it is suppressed entirely when the same call reports that a person would have to answer something first. Nothing the model produced ever enters the transcript: the line is chosen from a fixed table by failure mode.
 

package/extensions/jev-advisor/broker.mjs

@@ -6,8 +6,8 @@
 // inside sanitize.
 //
 // AWAIT ONLY THE SYSTEMS THAT MUTATE WHAT THEY INSPECT. Retention must be awaited, because its
-// answer replaces the tool result it was asked about; so must compaction, the branch hook and the
-// two tool_call systems, which return a patch or edit `event.input` in place. A system that acts on
+// answer replaces the tool result it was asked about; so must the two tool_call systems, which
+// return a patch or edit `event.input` in place. A system that acts on
 // a later turn must not be awaited: at roughly 300ms a call, a turn-level system firing thirty
 // times would add nine seconds to an attempt that takes a hundred and thirty, to deliver advice
 // that was never going to change the turn it was asked during.
@@ -15,14 +15,13 @@
 import { randomUUID } from "node:crypto";
 import { SYSTEM_NAMES, loadSettings } from "./config.mjs";
 import { ensureConsent } from "./consent.mjs";
-import { buildState } from "./sanitize.mjs";
+import { buildQuestions, buildState } from "./sanitize.mjs";
 import { ask } from "./client.mjs";
 import { payloadDigest, record } from "./ledger.mjs";
 import { writeUsage } from "./usage.mjs";
 
 export const SYSTEM_LABELS = Object.freeze({
     retention: "Tool-result retention",
-    compaction: "Compaction guidance",
     gap: "Capability-gap triage",
     sources: "Delegation source ranking",
     progress: "Progress and thrash detection",
@@ -165,8 +164,27 @@ export function createBroker(options = {}) {
      * is the only point where the answers and the audit line exist together; its `decision` is
      * handed back so the caller does not gate the same answers twice.
      */
-    const request = async ({ system, state, questions, ctx, root, maxBytes, timeoutMs, signal, decide }) => {
+    const request = async ({
+        system,
+        state,
+        questions,
+        ctx,
+        root,
+        profile,
+        maxBytes,
+        timeoutMs,
+        signal = ctx?.signal,
+        decide,
+        apply,
+        isCurrent = () => true,
+    }) => {
+        const startedGeneration = generation;
+        const currentRequest = () => startedGeneration === generation && !signal?.aborted && isCurrent();
         const settings = readSettings();
+        if (!currentRequest()) {
+            return { ok: false, reason: "context-changed", answers: {} };
+        }
+
         if (!settings.master) {
             return { ok: false, reason: "master-off", answers: {} };
         }
@@ -189,6 +207,25 @@ export function createBroker(options = {}) {
             return { ok: false, reason: "budget-exhausted", answers: {} };
         }
 
+        const built = buildState(state, { root, maxBytes, profile });
+        const wireQuestions = buildQuestions(questions, root);
+        if (!built.ok || !wireQuestions) {
+            const reason = !built.ok ? built.reason : "invalid-questions";
+            // A local abstention is not a transmission and consumes no call budget. Counts and
+            // coverage only: never record the material that could not safely be sent.
+            write({
+                system,
+                sent: false,
+                ok: false,
+                reason,
+                outcome: reason,
+                applied: false,
+                coverage: built.coverage,
+            });
+
+            return { ok: false, reason, answers: {} };
+        }
+
         const consented = await resolveConsent(ctx, SYSTEM_LABELS[system] ?? system);
         if (!consented) {
             return { ok: false, reason: "no-consent", answers: {} };
@@ -196,22 +233,41 @@ export function createBroker(options = {}) {
 
         // Settings can change while the dialog is open, and a session can end under it.
         const current = readSettings();
+        if (!currentRequest()) {
+            return { ok: false, reason: "context-changed", answers: {} };
+        }
+
         if (!current.master || current.systems[system] !== true) {
             return { ok: false, reason: "master-off", answers: {} };
         }
 
-        const built = buildState(state, { root, maxBytes });
-        const questionKeys = Object.keys(questions);
+        // Parallel result hooks can wait on the same consent prompt. Reserve only after rechecking.
+        if ((usedBySystem.get(system) ?? 0) >= current.budgets[system]) {
+            return { ok: false, reason: "system-budget-exhausted", answers: {} };
+        }
+
+        if (callsUsed >= current.budgets.total) {
+            return { ok: false, reason: "budget-exhausted", answers: {} };
+        }
+
+        const questionKeys = Object.keys(wireQuestions);
         callsUsed += 1;
         usedBySystem.set(system, (usedBySystem.get(system) ?? 0) + 1);
-        const startedGeneration = generation;
-        const result = await transport(built.state, questions, { timeoutMs, signal });
+        let result;
+        try {
+            result = await transport(built.state, wireQuestions, { timeoutMs, signal });
+        } catch {
+            result = { ok: false, reason: "transport-error", answers: {} };
+        }
+
         // The session can end under a call that was never awaited, which is the normal shape of a
         // turn-level system: the payload has already left the machine, and the answer now belongs
         // to a session that no longer exists. It must not be acted on. It must still be recorded --
         // the ledger's whole claim is that every transmission appears in it, and a run that sent 44
         // and logged 43 is how this was found. So the line is written either way and says which.
-        const stale = startedGeneration !== generation;
+        const mayApply = () => currentRequest() && readSettings().master && readSettings().systems[system] === true;
+        let stale = !mayApply();
+        const committedEffects = new Set();
 
         // A gate that throws must not turn into a failed call: the caller's own catch would have
         // swallowed it anyway, and recording it as unapplied is the truthful line.
@@ -219,17 +275,38 @@ export function createBroker(options = {}) {
         if (!stale && result.ok && typeof decide === "function") {
             try {
                 outcome = decide(result.answers) ?? { applied: false };
+                if (typeof apply === "function") {
+                    const delivered = await apply(outcome.decision, {
+                        isCurrent: mayApply,
+                        recordEffect: (effect) => committedEffects.add(effect),
+                    });
+                    outcome = { ...outcome, applied: false, ...delivered };
+                }
             } catch {
                 outcome = { applied: false, gateThrew: true };
             }
         }
 
+        // A later notification or report-rendering failure cannot undo an already committed effect.
+        if (committedEffects.size > 0) {
+            outcome = {
+                ...outcome,
+                applied: true,
+                effects: [...new Set([...(outcome.effects ?? []), ...committedEffects])],
+            };
+        }
+
+        // An application may await a dialog or a locked wishlist write. Do not attribute its
+        // delivered effects to a different session if that session changed while awaiting it.
+        stale ||= !currentRequest() || !readSettings().master || readSettings().systems[system] !== true;
         write({
             system,
+            sent: true,
             questionKeys,
+            coverage: built.coverage,
             stateBytes: built.bytes,
             stateTruncated: built.truncated,
-            payloadSha256: payloadDigest({ state: built.state, questions }),
+            payloadSha256: payloadDigest({ state: built.state, questions: wireQuestions }),
             ok: result.ok,
             reason: result.ok ? undefined : result.reason,
             // A sent payload whose answer arrived too late to use. Distinguished from a refusal,
@@ -238,6 +315,7 @@ export function createBroker(options = {}) {
             // Whether the advice changed anything, and what it saved when the change was a
             // shortening. Zero is a real answer here and means "asked, and kept the result whole".
             applied: outcome.applied === true,
+            effects: Array.isArray(outcome.effects) ? outcome.effects : [],
             // And why not, when nothing changed. Without this a system that asks and never acts is
             // indistinguishable from one whose gate can never be satisfied, which is the exact
             // failure the calibration pass had to go looking for by hand.

package/extensions/jev-advisor/client.mjs

@@ -74,30 +74,51 @@ function unavailable(reason) {
  * A Noul has no confidence, and inventing one would let a caller gate on a number the model never
  * reported, so it stays undefined.
  */
-function normalizeAnswer(raw) {
-    if (!raw || typeof raw !== "object") {
+function normalizeAnswer(raw, question) {
+    if (!raw || typeof raw !== "object" || !question) {
         return undefined;
     }
 
-    if (typeof raw.noul === "number") {
+    const probability = (value) => Number.isFinite(value) && value >= 0 && value <= 1;
+    const confidence = probability(raw.confidence) ? raw.confidence : undefined;
+    const distribution = raw.probabilities;
+    const validDistribution =
+        distribution &&
+        typeof distribution === "object" &&
+        Object.keys(distribution).length > 0 &&
+        Object.values(distribution).every(probability);
+
+    if (question.type === "noul" && probability(raw.noul)) {
         return { kind: "noul", value: raw.noul, probabilities: undefined, confidence: undefined };
     }
 
-    if (typeof raw.choice === "string") {
+    if (question.type === "choice" && typeof raw.choice === "string" && Object.hasOwn(question.criteria, raw.choice)) {
         return {
             kind: "choice",
             value: raw.choice,
-            probabilities: raw.probabilities && typeof raw.probabilities === "object" ? raw.probabilities : undefined,
-            confidence: typeof raw.confidence === "number" ? raw.confidence : undefined,
+            probabilities:
+                validDistribution &&
+                !Array.isArray(distribution) &&
+                Object.keys(distribution).length === Object.keys(question.criteria).length &&
+                Object.keys(distribution).every((key) => Object.hasOwn(question.criteria, key))
+                    ? distribution
+                    : undefined,
+            confidence,
         };
     }
 
-    if (typeof raw.score === "number") {
+    if (
+        question.type === "score" &&
+        Number.isFinite(raw.score) &&
+        raw.score >= 0 &&
+        raw.score <= question.criteria.length - 1
+    ) {
         return {
             kind: "score",
             value: raw.score,
-            probabilities: Array.isArray(raw.probabilities) ? raw.probabilities : undefined,
-            confidence: typeof raw.confidence === "number" ? raw.confidence : undefined,
+            // The documented Score distribution is an object; older routes also return arrays.
+            probabilities: validDistribution ? distribution : undefined,
+            confidence,
         };
     }
 
@@ -126,6 +147,8 @@ export async function ask(state, questions, options = {}) {
     try {
         const response = await fetch(endpoint(), {
             method: "POST",
+            // Consent covers this origin, not a redirect target that could receive the same body.
+            redirect: "error",
             headers: {
                 "content-type": "application/json",
                 authorization: `Bearer ${key}`,
@@ -143,7 +166,7 @@ export async function ask(state, questions, options = {}) {
         const body = await response.json();
         const answers = {};
         for (const [name, raw] of Object.entries(body?.answers ?? {})) {
-            const normalized = normalizeAnswer(raw);
+            const normalized = normalizeAnswer(raw, Object.hasOwn(questions, name) ? questions[name] : undefined);
             if (normalized) {
                 answers[name] = normalized;
             }

package/extensions/jev-advisor/config.mjs

@@ -18,15 +18,7 @@ import path from "node:path";
 import { randomUUID } from "node:crypto";
 
 /** Systems that may run inside a session. Offline scripts are not gated here. */
-export const SYSTEM_NAMES = Object.freeze([
-    "retention",
-    "compaction",
-    "gap",
-    "sources",
-    "progress",
-    "untrusted",
-    "capability",
-]);
+export const SYSTEM_NAMES = Object.freeze(["retention", "gap", "sources", "progress", "untrusted", "capability"]);
 
 /**
  * What a confident stuck verdict is allowed to do. `notify` tells the person and cannot be wrong in
@@ -57,14 +49,14 @@ const MAX_TOTAL_BUDGET = 2048;
  * session, and which one won would be decided by event ordering rather than by anyone's policy.
  *
  * So the ceiling is two-level: each system gets its own, and the total is a real constraint because
- * it is deliberately less than their sum -- 2048 against 2322. That relationship is the invariant,
+ * it is deliberately less than their sum -- 2048 against 2274. That relationship is the invariant,
  * not either number: raising the total without raising the per-system ceilings would leave a total
  * no combination of systems could ever reach, which is a limit that reads as a limit and is not
  * one. `tests/jev-advisor.test.mjs` pins the inequality so a future change to one has to consider
  * the other. Running out of one system's budget stops that system and nothing else.
  *
  * The per-system numbers follow how often each one can fire: retention on every large read-only
- * result, compaction once or twice in a long session, gap per report, sources per delegation batch.
+ * result, gap per report, sources per delegation batch.
  */
 export const DEFAULT_BUDGETS = Object.freeze({
     // A backstop, not a working limit, and the number says which. Measured, a full tier-3 task -- a
@@ -92,7 +84,6 @@ export const DEFAULT_BUDGETS = Object.freeze({
     // add up to is not a constraint at all -- see below. They are scaled rather than re-derived:
     // each one's rationale is a firing frequency, and none of those frequencies changed.
     retention: 832,
-    compaction: 48,
     gap: 192,
     sources: 128,
     // Turn-level, but gated behind local signals and a four-turn cooldown, so it only spends on
@@ -144,7 +135,7 @@ export function regularFile(file, label) {
 /** Every unknown shape collapses to the same all-off default rather than a partial enable. */
 export function defaultSettings() {
     return {
-        schema: 4,
+        schema: 5,
         master: false,
         startup: false,
         systems: Object.fromEntries(SYSTEM_NAMES.map((name) => [name, false])),
@@ -206,6 +197,29 @@ function migrateToFour(raw) {
     return { ...rest, schema: 4, systems };
 }
 
+/**
+ * Schema 5 removes compaction guidance, which was withdrawn rather than fixed.
+ *
+ * Two tier-6 runs, on SpecPi 0.28.0 and 0.29.0, both measured the arm carrying it solving fewer
+ * long-session tasks than plain SpecPi: 14/16 against 8/16 pooled, Fisher exact p = 0.054. Over the
+ * same attempts compaction was 55 of 56 applied verdicts, so the arm's behaviour was almost entirely
+ * this system's, and no other system in the layer applied enough to be a candidate.
+ *
+ * The evidence never reached significance and the mechanism was never isolated. The system was
+ * removed anyway, because a system that steers a summary has to earn the risk it takes, and one
+ * whose only measurement says it costs solve rate has not.
+ *
+ * Both the `systems.compaction` switch and the per-system `budgets.compaction` ceiling go. Dropping
+ * them turns nothing off that a user had on in any meaningful sense: the hooks they gated no longer
+ * exist, so a retained preference could only describe a system that cannot run.
+ */
+function migrateToFive(raw) {
+    const { compaction: _system, ...systems } = raw?.systems ?? {};
+    const { compaction: _budget, ...budgets } = raw?.budgets ?? {};
+
+    return { ...raw, schema: 5, systems, budgets };
+}
+
 /**
  * What the advisor will read, given a settings object, without writing it anywhere.
  *
@@ -222,15 +236,16 @@ export function normalizeSettings(raw) {
 
 function normalize(raw) {
     const one = raw?.schema === 1 ? migrate(raw) : raw;
-    const source = one?.schema === 2 || one?.schema === 3 ? migrateToFour(one) : one;
-    if (source?.schema !== 4) {
+    const four = one?.schema === 2 || one?.schema === 3 ? migrateToFour(one) : one;
+    const source = four?.schema === 4 ? migrateToFive(four) : four;
+    if (source?.schema !== 5) {
         return defaultSettings();
     }
 
     const systems = Object.fromEntries(SYSTEM_NAMES.map((name) => [name, source.systems?.[name] === true]));
 
     return {
-        schema: 4,
+        schema: 5,
         master: source.master === true,
         startup: source.startup === true,
         systems,
@@ -268,7 +283,7 @@ export function writeFileAtomic(file, contents) {
 export function saveSettings(settings) {
     // A caller handing back an older shape is migrated rather than reset, so a round trip through
     // an old reader cannot quietly disable the layer.
-    const next = normalize([1, 2, 3].includes(settings?.schema) ? settings : { ...settings, schema: 4 });
+    const next = normalize([1, 2, 3, 4].includes(settings?.schema) ? settings : { ...settings, schema: 5 });
     const file = settingsFile();
     if (fs.existsSync(file)) {
         regularFile(file, "Jev settings");