pi-jev-lens 0.2.0 → 0.3.0

package/README.md

@@ -1,16 +1,17 @@
 # pi-jev-lens
 
-**Your coding agent reads a 600-line file to change one function. jev-lens sends the model the outline and that
-function.** The rest is one `recall` away, and the agent knows it.
+Your coding agent reads a 600-line file to change one function. jev-lens sends the model the outline of the file and
+that one function. The agent can ask for the rest with the `recall` tool, and the footer of the result tells it so.
 
-A [pi](https://github.com/earendil-works/pi-mono) extension. Tool output is most of what a coding agent pays for:
-every `cat`, every test run, every `grep` lands in the prompt in full and stays there, cached, for the rest of the
-session. jev-lens steps in before that first send. Code builds a handful of candidate views of the output, and
-[jev](https://docs.typesafe.ai), TypeSafe's System One judgment model, picks the smallest one that still lets the agent
-do its next step. Nothing is generated or summarized: every view is lines of the original, with line numbers, so the
-agent can ask for exactly the part it is missing.
+jev-lens is an extension for [pi](https://github.com/earendil-works/pi-mono), the coding agent. Tool output is most
+of what a coding agent pays for. Every `cat`, every test run and every `grep` goes into the prompt in full and stays
+there for the rest of the session. jev-lens acts before that first send. Code builds a small set of candidate views
+of the output. A view is a subset of the lines of the output, with line numbers. Then [jev](https://docs.typesafe.ai),
+the judgment model from TypeSafe, picks the smallest view that still lets the agent do its next step. Nothing is
+generated or summarized. Every view is made of lines from the original, so the agent can ask for exactly the part
+that it does not have.
 
-What the model sees instead of a 1.5k-token file:
+This is what the model sees instead of a file of 1.5k tokens:
 
 ```
   1│ import { parse } from "./parse.js";
@@ -29,51 +30,69 @@ full output, or recall(id, lines: "a-b") / recall(id, pattern: "...") for a slic
 
 ## What the numbers say
 
-We did not guess the defaults; we measured them on 500 real agent trajectories (OpenHands on SWE-rebench, 3300 large
-tool results, 11.6 million tokens) and on our own day-to-day pi sessions. The research log is STATUS.md; the headlines:
+We measured the defaults on 500 real agent trajectories (OpenHands on SWE-rebench: 3300 large tool results, 11.6
+million tokens) and on our own daily pi sessions. A token is the unit that a model provider bills. The research log
+is STATUS.md. These are the main results:
 
-| | |
+| result | source |
 |---|---|
-| **79 % fewer tokens** sent for large tool results across the 500 benchmark trajectories | 11.6M → 2.4M |
-| **88 % on command output** (test runs, grep, build logs), 58 % on docs, 47 % on listings, 31 % on code | per kind |
-| **31 % of large-result tokens** in our own sessions with gpt-6-astra, which reads code through `cat` and runs few tests | real use |
-| **2 of 26 later edits** missed their block; 0.3 % of results had a dropped line quoted; 2.2 % had a dropped identifier used | the harm side |
-| **8 % lower cost, 17 % smaller final prompt, same pass rate, zero recalls** on live end-to-end runs where big files get read | pi headless, 3 runs each |
-
-Read the two savings numbers together. The benchmark agent spends its output budget on pytest runs and grep, which
-compress to almost nothing; an agent that mostly reads source code sits closer to the code number, because code is
-the one thing we refuse to compress unless jev is confident. And both are shares of *large tool results*: over a
-whole session, with the system prompt, the conversation and every small result counted, the footer will show a lower
-percentage. What you save depends on what your agent reads.
-
-Three things we learned that shaped the design:
-
-- **Compress before the first send, not after.** Pruning old results later looks great on token counts (−19 %) and costs
-  *more* money (+17 %), because every rewrite breaks the prompt cache. Post-send pruning is still in the code, off by default.
-- **Code is different.** Test logs and grep output can lose 90 % and nobody misses it. Code is edited from, and an edit
-  whose old text the model never saw fails. So code views keep every retained line byte-exact, and code is sent full
-  unless jev is confident. The tempting always-outline policy saved more and missed 17 % of later edits; it is opt-in.
-- **Views built in code beat prompt tuning.** Four rounds of letting a researcher model rewrite jev's prompts and
-  thresholds moved nothing that held up on held-out data. Every gain that lasted was a new kind of view: failing tests
-  only, grep match groups, JSON keys, the file the agent `cat`-ed through bash.
+| 79 % fewer tokens sent for large tool results (11.6M became 2.4M) | the 500 benchmark trajectories |
+| 88 % fewer on command output (test runs, grep, build logs), 58 % on docs, 47 % on listings, 31 % on code | the same, per kind of output |
+| 31 % fewer tokens for large tool results | our own sessions with gpt-6-astra, which reads code with `cat` and runs few tests |
+| 2 of 26 later edits missed their block, 0.3 % of results had a dropped line quoted, 2.2 % had a dropped identifier used | the harm side, same 500 trajectories |
+| 8 % lower cost, 17 % smaller final prompt, the same pass rate, zero recalls | live end-to-end runs in pi where the agent reads big files, 3 runs each |
+
+Read the two savings numbers together. The benchmark agent spends its output budget on pytest runs and grep, and
+those compress to almost nothing. An agent that mostly reads source code sits closer to the code number, because
+code is the one thing that jev-lens does not compress unless jev is confident. Both numbers are shares of large tool
+results only. Over a whole session, the footer counts the system prompt, the conversation and every small result too,
+so it will show a lower percentage. What you save depends on what your agent reads.
+
+Three results shaped the design:
+
+1. Compress before the first send, not after. Pruning old results later removes 19 % of the tokens and costs 17 %
+   more money, because every rewrite of an old message breaks the prompt cache. The prompt cache is the provider's
+   discount for a prompt prefix that did not change since the last call. Post-send pruning is still in the code, but
+   it is off by default.
+2. Code is different. Test logs and grep output can lose 90 % of their lines and the agent does not miss them. The
+   agent edits from code, and an edit fails when its old text is a line that the model never saw. So views of code
+   keep every retained line byte for byte, and code is sent in full unless jev is confident. The policy that always
+   sends an outline saved more and missed 17 % of later edits. It is available, but you must turn it on.
+3. Views built in code beat prompt tuning. In four rounds, a researcher model rewrote the questions to jev and the
+   thresholds. None of its changes held up on data that it had not seen. Every gain that held was a new kind of view:
+   only the failing tests, grep match groups, JSON keys, the file that the agent read with `cat`.
 
 ## Install
 
+Use pi 0.84.3 or newer. The extension uses pi's built-in tool renderers for results that it does not compress.
+
 ```sh
 pi install npm:pi-jev-lens                    # from npm
-pi install git:github.com/dizk/pi-jev-lens    # or straight from GitHub
+pi install git:github.com/dizk/pi-jev-lens    # or from GitHub
 ```
 
-jev needs a TypeSafe API key (get one at [console.typesafe.ai](https://console.typesafe.ai)). Three ways to provide it,
-in the order they are tried:
+jev needs a TypeSafe API key. You can get one at [console.typesafe.ai](https://console.typesafe.ai). jev-lens looks
+for the key in this order:
 
 1. `TYPESAFE_API_KEY` in the environment.
-2. `/jev-lens key` inside pi: prompts for the key (or `/jev-lens key ts_...`) and stores it in
-   `~/.pi/agent/jev-lens.json`, readable only by you. jev is active from the next tool result, no restart.
-3. A `.env` file next to the installed package (development).
+2. A `.env` file next to the installed package. This is for development and supplies environment values that are not already set.
+3. The key that you stored with `/jev-lens key` inside pi. In terminal mode, the command opens a masked input field.
+   The key is stored in `~/.pi/agent/jev-lens.json`. New files are readable only by you.
+
+Use `/jev-lens key` without an argument to keep the key out of command history. The field displays only `*` characters.
+Type or paste the key, then press Enter to save. Press Esc to cancel without changing the stored key.
+In RPC or noninteractive mode, set `TYPESAFE_API_KEY`. These modes do not fall back to a visible input field.
+The key file still stores the key as plain text. Masking protects the terminal display, not the file.
+
+You can still use `/jev-lens key ts_...`, but that exposes the key in the editor and can retain it in command history.
+A new key takes effect without a restart, but the command does not validate it.
+If `TYPESAFE_API_KEY` is set, that value takes priority again after reload.
+If mock mode is forced or compression is disabled, storing a key does not change those settings.
 
-Without a key the extension warns at startup and runs a mock classifier that compresses nothing. For development,
-clone the repo and load it directly:
+If no key is found, the extension shows a warning at startup and uses a deterministic mock classifier.
+The mock can compress results without API calls. It is not the jev model.
+
+For development, clone the repository and load it directly:
 
 ```sh
 git clone https://github.com/dizk/pi-jev-lens.git && cd pi-jev-lens && npm install
@@ -83,68 +102,88 @@ pi -e ./index.ts
 
 ## How it works
 
-Every text tool result of at least 1200 estimated tokens (about 5 kB) passes through pi's `tool_result` hook before
-it is stored or sent. Small results are never touched. Code builds candidate **views**: strict subsets of the output,
-with line numbers and omission markers, never generated text.
+Every text tool result of at least 1200 estimated tokens (about 5 kB) goes through pi's `tool_result` hook before
+pi stores it or sends it. jev-lens never touches smaller results. Code builds the candidate views. Each view is a
+subset of the lines of the output, with line numbers and markers for the omitted lines. No text is generated.
 
 | view | for | keeps |
 |---|---|---|
 | `outline` | code, prose | imports, exports, signatures, headings, doc comments |
-| `relevant` | code, command output | outline or section headers plus the full bodies jev says the agent will need (second jev step) |
-| `sections` | command output | the first line of every section: grep match groups, JSON keys, headings, `COMMAND:`-style markers, paragraphs |
+| `relevant` | code, command output | the outline or the section headers, plus the full bodies that jev says the agent will need (second jev step) |
+| `sections` | command output | the first line of every section: grep match groups, JSON keys, headings, markers like `COMMAND:`, paragraphs |
 | `signals` | command output | errors, warnings, failing tests, summary lines, the tail |
 | `testlog` | test output | failures, assertions, tracebacks and summaries |
-| `matches` | search output | first matches per file, with omission counts |
+| `matches` | search output | the first matches per file, with a count of the omitted ones |
 | `log` | repetitive output | representative repeated lines, errors and the tail |
-| `tree` | directory listings | a sample of entries per directory, with omission counts |
-| `focus` | anything | lines mentioning identifiers from the task and the tool call, with context |
-| `sample` | tabular or log-like data | header, a dozen rows, the count |
-| `head_tail` | anything | first and last lines |
-
-jev then answers two questions over the task, the assistant's text before the call and a preview of each view:
-*which view is the smallest that still suffices* (a Choice) and *will the next step need the exact full text* (a
-yes/no). When an outline or `sections` view is chosen, a second request asks, per block or section, whether the agent
-will need its body, and those bodies are put back. If that reaches 90 % of the original, full text is sent instead.
-
-What makes it safe to edit from a view:
-
-- Views of code and prose keep every retained line exactly, so an edit whose old text was copied from the view still
-  matches the file. Views of command output shorten decorative bars and very long lines.
-- Files the agent reads through bash (`cat a.py b.py`, `sed -n '1,80p' x.ts`, `head`, brace groups, globs) are typed as
-  code or prose and get the same views as `read`. Anything mixed with other commands stays command output.
-- The agent's own `edit` and `write` results are never reduced.
-- Code is sent full unless jev is confident a view suffices (`gate` policy). The always-outline policy saves more but
-  missed 17 % of later edits on real trajectories, so it is opt-in.
-
-Code structure comes from tree-sitter (grammars from `@vscode/tree-sitter-wasm` plus `@binclusive/tree-sitter-kotlin-wasm`):
-TypeScript, TSX, JavaScript, Kotlin, Java, Rust, Python, Go, C, C++, C#, Ruby, PHP, Bash, CSS. Large classes are split
-into their members. Other languages fall back to regex heuristics.
-
-**Recall.** When a result is compressed, its full output is kept in the result's `details` (persisted in the session,
-never sent to the model). The footer names a `recall` tool that serves it back by id, line range or pattern. Every
-recall is logged as feedback that a view was too small.
+| `tree` | directory listings | a sample of entries per directory, with a count of the omitted ones |
+| `focus` | anything | the lines that mention identifiers from the task and the tool call, with context |
+| `sample` | tabular or log-like data | the header, a dozen rows, the count |
+| `head_tail` | anything | the first and the last lines |
+
+jev then answers two questions. It sees the task, the text that the assistant wrote before the call, and a preview of
+each view. The questions are: which view is the smallest one that is still enough (a choice), and will the next step
+need the exact full text (yes or no). If jev chose an outline or a `sections` view, a second request asks, for each
+block or section, whether the agent will need its body. Those bodies go back into the view. If the result reaches
+90 % of the original size, jev-lens sends the full text instead.
+
+These rules make it safe for the agent to edit from a view:
+
+- Views of code and prose keep every retained line exactly as it is. An edit whose old text was copied from the view
+  still matches the file. Views of command output shorten decorative bars and very long lines.
+- Files that the agent reads through bash (`cat a.py b.py`, `sed -n '1,80p' x.ts`, `head`, brace groups, globs) count
+  as code or prose and get the same views as the `read` tool. If the command also does something else, the output
+  stays command output.
+- jev-lens never reduces the results of the agent's own `edit` and `write` tools.
+- jev-lens sends code in full unless jev is confident that a view is enough. This is the `gate` policy. The
+  `outline` policy always sends an outline plus expanded bodies. It saves more, but it missed 17 % of later edits on
+  real trajectories, so you must turn it on yourself.
+
+The code structure comes from tree-sitter (grammars from `@vscode/tree-sitter-wasm` and
+`@binclusive/tree-sitter-kotlin-wasm`): TypeScript, TSX, JavaScript, Kotlin, Java, Rust, Python, Go, C, C++, C#,
+Ruby, PHP, Bash, CSS. Large classes are split into their members. For other languages, jev-lens uses regular
+expressions that know the common declaration keywords.
+
+When jev-lens compresses a result, it keeps the full output in the result's `details`. pi persists that in the
+session but never sends it to the model. The footer names the `recall` tool, which serves the full output back by
+id, by line range or by pattern. jev-lens logs every recall as a signal that a view was too small.
 
 ## In pi
 
-The footer shows the share of the session's input tokens jev kept out of the prompt, and what it did:
+The footer shows the share of the session's input tokens that jev kept out of the prompt, and what it did:
 
 ```
 jev-lens −38% of input (presend −12.3k · 5/8 · 1 recalls)
 ```
 
-The share is cut / (sent + cut): sent is the provider's own input plus cache-read counts over all calls, cut is what
-every compressed result saved on every call it was part of.
+The share is cut divided by sent plus cut. Sent counts input and cache-read tokens reported by the provider since the last load.
+Cut estimates what compressed results saved on those calls, including results restored from the session.
+The percentage counts repeated savings when the same result appears in later prompts. The `presend` total counts each result once.
+
+After `/reload` or resume, the footer includes saved tokens from restored compressed results.
+For example, `0/3 new · 5 restored` means no new compressions among three candidates, plus five restored compressed results.
+New-result counts and recall counts start at zero after loading. `/jev-lens stats` shows new and restored savings separately.
 
-In the transcript a compressed result shows a header like `⌁ jev-lens outline · 179 of 1524 tokens (−88 %)` and,
-expanded (ctrl+e), exactly what the model saw. Commands:
+In the transcript, a compressed result shows a header like `⌁ jev-lens outline · 179 of 1524 tokens (−88 %)`. When
+you expand it with ctrl+e, you see exactly what the model saw. These commands are available:
 
-- `/jev-lens` stats, and where the key comes from
-- `/jev-lens list` the latest 200 compressed results with tokens before and after
-- `/jev-lens diff [n]` overlay of the n-th latest: the original with the lines the model did not get marked `−`
-  (`t` switches to what was sent, `Esc` closes)
-- `/jev-lens key` store the API key
+Type `/jev-lens ` and press Tab to complete subcommands. After `diff `, completion offers available result numbers.
+Use `/reload` after installing the package in a running pi session.
 
-Every decision is logged to `<project>/.pi/jev-lens.log` (JSON lines). `JEV_LENS_UI=0` keeps pi's own tool rendering.
+- `/jev-lens` or `/jev-lens stats` shows the statistics, active configuration, and key source.
+- `/jev-lens help` shows command usage.
+- `/jev-lens decisions` shows post-send pruning decisions. Post-send pruning is off by default.
+- `/jev-lens list` lists the latest 200 compressed results with the tokens before and after.
+- `/jev-lens diff [n]` opens an overlay for the n-th latest result. It shows the original with the lines that the
+  model did not get marked with `−`. Press `t` to see what was sent, and `Esc` to close.
+- `/jev-lens key` stores the API key.
+
+If compression fails, jev-lens keeps the full output and shows a warning. A failed post-send classification leaves that result unchanged.
+Warnings appear at most once per stage per session. The footer shows `degraded` until a later attempt in that stage succeeds.
+Use `/jev-lens stats` to see failure counts and recovery status. Cancellation does not count as a failure.
+
+Uncompressed results, errors, and streaming updates use pi's built-in tool renderers.
+jev-lens logs every decision to `<project>/.pi/jev-lens.log` as JSON lines. Set `JEV_LENS_UI=0` to disable the
+custom savings headers and tool overrides.
 
 ### Configuration (environment)
 
@@ -152,29 +191,34 @@ Every decision is logged to `<project>/.pi/jev-lens.log` (JSON lines). `JEV_LENS
 |---|---|---|
 | `JEV_LENS_PRESEND` | `1` | `0` turns compression off |
 | `JEV_LENS_PRESEND_MIN_TOKENS` | `1200` | smaller results are always sent in full |
-| `JEV_LENS_PRESEND_NEEDS_FULL_ABOVE` / `_FULL_MASS_ABOVE` | `0.5` / `0.5` | send full when P(needs full) or P(full view) exceeds these |
-| `JEV_LENS_PRESEND_CODE_POLICY` | `gate` | `outline`: always send an outline plus expanded bodies (more savings, more edit-misses) |
-| `JEV_LENS_PRESEND_CODE_NEEDS_FULL_ABOVE` | `0.5` | code uses the minimum of this and the general needs-full threshold |
-| `JEV_LENS_PRESEND_EXPAND_ABOVE` | `0.5` | expand a code block's body when P(needed) exceeds this |
-| `JEV_LENS_PRESEND_COMMAND_NEEDS_FULL_ABOVE` | `0.65` | needs-full threshold for command output |
-| `JEV_LENS_PRESEND_COMMAND_POLICY` | `sections` | when jev picks full for command output but needs-full is low, send section headers and expand the needed sections. `gate`: jev's choice stands |
-| `JEV_LENS_PRESEND_SECTION_EXPAND_ABOVE` | `0.5` | expand a section when P(needed) exceeds this |
-| `JEV_LENS_PRESEND_SECTION_FLOOR` | `0.3` | send full when no section reaches this probability (jev could not tell); `0` allows headers alone |
-| `JEV_LENS_PRESEND_MIN_CONFIDENCE` | `0` | send full below this choice confidence (0 = off) |
-| `JEV_LENS_MODEL` | `jev-latest` | jev model |
-| `JEV_LENS_CLASSIFIER` | unset | `mock` forces the deterministic classifier, no API calls |
-| `JEV_LENS_LOG` | `1` | `0` disables logging |
-| `JEV_LENS_UI` | `1` | `0` disables the custom tool rendering |
-| `JEV_LENS_VARIANT` | unset | JSON file with `config`, `prompts` and `views` overrides, as produced by the autoresearch loop |
+| `JEV_LENS_PRESEND_NEEDS_FULL_ABOVE` / `_FULL_MASS_ABOVE` | `0.5` / `0.5` | send full when P(needs full) or P(full view) is above these |
+| `JEV_LENS_PRESEND_CODE_POLICY` | `gate` | `outline`: always send an outline plus expanded bodies (more savings, more missed edits) |
+| `JEV_LENS_PRESEND_CODE_NEEDS_FULL_ABOVE` | `0.5` | code uses the lower of this and the general needs-full threshold |
+| `JEV_LENS_PRESEND_EXPAND_ABOVE` | `0.5` | expand the body of a code block when P(needed) is above this |
+| `JEV_LENS_PRESEND_COMMAND_NEEDS_FULL_ABOVE` | `0.65` | the needs-full threshold for command output |
+| `JEV_LENS_PRESEND_COMMAND_POLICY` | `sections` | when jev picks full for command output but needs-full is low, send the section headers and expand the needed sections. `gate`: keep jev's choice |
+| `JEV_LENS_PRESEND_SECTION_EXPAND_ABOVE` | `0.5` | expand a section when P(needed) is above this |
+| `JEV_LENS_PRESEND_SECTION_FLOOR` | `0.3` | send full when no section reaches this probability, because jev could not tell. `0` allows headers alone |
+| `JEV_LENS_PRESEND_MIN_CONFIDENCE` | `0` | send full below this choice confidence. `0` turns the check off |
+| `JEV_LENS_MODEL` | `jev-latest` | the jev model |
+| `JEV_LENS_CLASSIFIER` | unset | `mock` forces the deterministic classifier, with no API calls |
+| `JEV_LENS_LOG` | `1` | `0` turns logging off |
+| `JEV_LENS_UI` | `1` | `0` turns the custom tool rendering off |
+| `JEV_LENS_VARIANT` | unset | a JSON file with `config`, `prompts` and `views` overrides, as the autoresearch loop writes it |
 | `JEV_LENS_MODE` | `off` | optional post-send pruning, see below |
 
 ## Evaluation
 
-Every change here is scored against what the agent actually did next in a recorded trajectory, which is the only
-honest judge of "did it need that text". The benchmark is real OpenHands trajectories (`eval/bench/`, data fetched by
-`eval/bench/fetch.sh`), and the metrics are: **edit-miss** (it edited a line the view had dropped), **quote-miss** (it quoted dropped text),
-**ref-miss** (it used an identifier that only existed in the dropped part). Edit-misses are weighted five times in
-the objective, and they are rare, so anything that touches code views must be scored on the 500-trajectory slice:
+Every change is scored against what the agent did next in a recorded trajectory. That is the only honest judge of
+whether the agent needed the text. The benchmark uses real OpenHands trajectories in `eval/bench/`. The script
+`eval/bench/fetch.sh` downloads the data. The metrics are:
+
+- edit-miss: the agent later edited a line that the view had dropped.
+- quote-miss: the agent quoted text that the view had dropped.
+- ref-miss: the agent used an identifier that only existed in the dropped part.
+
+An edit-miss counts five times in the objective, and edit-misses are rare. So you must score every change that
+touches code views on the 500-trajectory slice, not only on the 100-trajectory holdout.
 
 | slice | large results | saved | edit-miss | quote-miss | ref-miss |
 |---|---|---|---|---|---|
@@ -183,52 +227,56 @@ the objective, and they are rare, so anything that touches code views must be sc
 
 ```sh
 npm test                                                # unit tests, mock classifier
-node --import tsx eval/bench/run.ts --from 200 --to 300 # holdout, ~4 min
-node --import tsx eval/bench/run.ts --from 300 --to 800 # the 500-trajectory slice, ~20 min
+node --import tsx eval/bench/run.ts --from 200 --to 300 # the holdout, about 4 minutes
+node --import tsx eval/bench/run.ts --from 300 --to 800 # the 500-trajectory slice, about 20 minutes
 node --import tsx eval/presend-replay.ts <session dir>  # replay your own pi sessions from ~/.pi/agent/sessions
-node --import tsx eval/bench/autoresearch.ts --iterations 8   # a researcher model tunes prompts and thresholds
+node --import tsx eval/bench/autoresearch.ts --iterations 8   # a researcher model tunes the prompts and thresholds
 ```
 
-STATUS.md is the research log: every variant tried, its numbers, and why the defaults are what they are. The short
-version: new code-built views moved the numbers, prompt wording did not, and the small holdout was wrong about code
-until the slice was five times larger.
+STATUS.md is the research log. It lists every variant that we tried, its numbers, and why the defaults are what they
+are. In short: new code-built views moved the numbers, prompt wording did not, and the small holdout was wrong about
+code until the slice was five times larger.
 
 ## Optional: post-send pruning
 
-`JEV_LENS_MODE=budget` (or `rolling`, `batch`) turns on a second layer: after the agent has reacted to a tool result,
-jev judges whether it is still needed, and the result is trimmed to head and tail or replaced by a one-line stub in
-later prompts. Decisions are persisted and frozen once applied, so the cached prefix is rewritten as rarely as
-possible; `budget` mode only rewrites when the pending prunes remove at least half of the tail they would touch.
-Measured on real sessions this frees context but does not save money under prompt-cache pricing, which is why it is
-off by default. Its settings: `JEV_LENS_BUDGET_FRACTION` / `_BUDGET_MIN_TOKENS` (`0.5` / `1000`), `JEV_LENS_FORGET_BELOW`
-(`0.25`), `JEV_LENS_TRIM_BELOW` / `_TRIM_ABOVE` (`0.5` / `0.6`), `JEV_LENS_MIN_TOKENS` (`150`), `JEV_LENS_TRIM_HEAD` /
-`_TRIM_TAIL` (`15` / `15`), `JEV_LENS_CLASSIFY_WAIT_MS` (`2500`), `JEV_LENS_CACHE_TTL_MS` (`300000`),
-`JEV_LENS_STATE_HEAD` / `_STATE_TAIL` (`2500` / `800`), `JEV_LENS_DISABLED=1` (new decisions become `keep`).
-`/jev-lens decisions` lists them. Tool results are never removed, only rewritten.
+`JEV_LENS_MODE=budget` (or `rolling`, or `batch`) turns on a second layer. After the agent reacted to a tool result,
+jev judges whether the result is still needed. In later prompts, a result that is not needed is cut to its head and
+tail, or replaced by a stub of one line. jev-lens persists each decision and freezes it after the first use, so the
+cached prefix is rewritten as rarely as possible. The `budget` mode only rewrites when the pending cuts remove at least
+half of the tail that they would touch. On real sessions, this layer frees context but does not save money under
+prompt-cache pricing. That is why it is off by default.
+
+Its settings are `JEV_LENS_BUDGET_FRACTION` and `JEV_LENS_BUDGET_MIN_TOKENS` (`0.5` and `1000`), `JEV_LENS_FORGET_BELOW`
+(`0.25`), `JEV_LENS_TRIM_BELOW` and `JEV_LENS_TRIM_ABOVE` (`0.5` and `0.6`), `JEV_LENS_MIN_TOKENS` (`150`),
+`JEV_LENS_TRIM_HEAD` and `JEV_LENS_TRIM_TAIL` (`15` and `15`), `JEV_LENS_CLASSIFY_WAIT_MS` (`2500`),
+`JEV_LENS_CACHE_TTL_MS` (`300000`), `JEV_LENS_STATE_HEAD` and `JEV_LENS_STATE_TAIL` (`2500` and `800`), and
+`JEV_LENS_DISABLED=1` (new decisions become `keep`). The command `/jev-lens decisions` lists the decisions. jev-lens
+never removes a tool result. It only rewrites it.
 
 ## Using this as a reference
 
-The pieces are independent of pi and can be lifted into another agent:
+The pieces do not depend on pi. You can lift them into another agent:
 
 | piece | file | depends on |
 |---|---|---|
-| candidate views | `src/views.ts` | nothing; async code views optionally load `src/treesitter.ts` |
+| candidate views | `src/views.ts` | nothing. The async code views can load `src/treesitter.ts` |
 | tree-sitter blocks and signatures | `src/treesitter.ts` | `web-tree-sitter`, `@vscode/tree-sitter-wasm`, `@binclusive/tree-sitter-kotlin-wasm` |
-| the jev questions, state shape, decision rule, block expansion | `src/presend.ts` | `@typesafe-ai/sdk` |
-| bash display-command parser | `src/shell-display.ts` | nothing |
-| the hook wiring for pi (tool_result, recall tool, UI) | `index.ts`, `src/ui.ts` | pi |
-| benchmark and metrics on real trajectories | `eval/presend-score.ts`, `eval/bench/` | run `eval/bench/fetch.sh` first |
+| the jev questions, the state shape, the decision rule, block expansion | `src/presend.ts` | `@typesafe-ai/sdk` |
+| the parser for bash display commands | `src/shell-display.ts` | nothing |
+| the hook wiring for pi (tool_result, the recall tool, the UI) | `index.ts`, `src/ui.ts` | pi |
+| the benchmark and the metrics on real trajectories | `eval/presend-score.ts`, `eval/bench/` | run `eval/bench/fetch.sh` first |
 | post-send decisions and the frozen ledger | `src/classifier.ts`, `src/policy.ts`, `src/ledger.ts` | `@typesafe-ai/sdk` |
 
-In one paragraph: when a large tool result arrives, build views from the text (code, no model), ask jev which view
-suffices and whether exact text is needed, apply the selection policy, and optionally expand blocks or sections in a
-second request. If a reduced view wins, replace the content with that view plus a footer naming `recall`, and keep the
-full text in the result's details.
+The sequence is this. When a large tool result arrives, code builds the views from the text. jev says which view is
+enough and whether the exact text is needed. Code applies the selection policy. If needed, a second request expands
+blocks or sections. If a reduced view wins, the content becomes that view plus a footer that names `recall`, and the
+full text stays in the details of the result.
 
 ## Contributing and license
 
 Issues and pull requests are welcome at [github.com/dizk/pi-jev-lens](https://github.com/dizk/pi-jev-lens).
-`npm test` runs the unit tests with the mock classifier; `npm run typecheck` runs tsc. Changes to how views are built
-or chosen should come with benchmark numbers, on the 500-trajectory slice when they touch code.
+`npm test` runs the unit tests with the mock classifier. `npm run typecheck` runs tsc. A change to how views are
+built or chosen must come with benchmark numbers. If the change touches code views, you must use the 500-trajectory
+slice.
 
 MIT, see LICENSE.

package/index.ts

@@ -11,11 +11,11 @@
  */
 import { appendFileSync, mkdirSync } from "node:fs";
 import { join } from "node:path";
-import type { ExtensionAPI, ExtensionContext, Theme } from "@earendil-works/pi-coding-agent";
+import type { ExtensionAPI, ExtensionContext, ToolDefinition } from "@earendil-works/pi-coding-agent";
 import type { AgentMessage } from "./src/pi-types.ts";
 import { CONFIG_DIR_NAME } from "@earendil-works/pi-coding-agent";
 import { Type } from "typebox";
-import { createBashTool, createFindTool, createGrepTool, createLsTool, createReadTool } from "@earendil-works/pi-coding-agent";
+import { createBashToolDefinition, createFindToolDefinition, createGrepToolDefinition, createLsToolDefinition, createReadToolDefinition } from "@earendil-works/pi-coding-agent";
 import { Text } from "@earendil-works/pi-tui";
 import { DiffOverlay, listLines, savingsLine, type CompressedRecord } from "./src/ui.ts";
 import { TypeSafeClient } from "@typesafe-ai/sdk";
@@ -23,6 +23,9 @@ import { buildItemState, JevClassifier, MockClassifier, type Classifier } from "
 import { buildPresendState, decideView, DEFAULT_PROMPTS, expandRelevantBlocks, JevPresend, MockPresend, type PresendClassifier, type PromptVariant } from "./src/presend.ts";
 import { buildCandidatesAsync, extractTerms, footer } from "./src/views.ts";
 import { keyFilePath, loadConfigWithVariant, storeKey, type Config } from "./src/config.ts";
+import { Health } from "./src/health.ts";
+import { SecretInput } from "./src/secret-input.ts";
+import { commandCompletions, commandHelp } from "./src/commands.ts";
 import { ENTRY_TYPE, rebuildLedger } from "./src/ledger.ts";
 import { applyLedger, decideBucket, pendingPrunable, shouldApplyPending } from "./src/policy.ts";
 import { contentText, describeToolCall, estimateTokensOfText, toolCallsOf, truncate } from "./src/text.ts";
@@ -37,6 +40,7 @@ export default function (pi: ExtensionAPI) {
 	const { cfg, variant } = loadConfigWithVariant();
 	const prompts: PromptVariant = { ...DEFAULT_PROMPTS, ...((variant.prompts ?? {}) as Partial<PromptVariant>), viewDescriptions: { ...DEFAULT_PROMPTS.viewDescriptions, ...(((variant.prompts ?? {}) as Partial<PromptVariant>).viewDescriptions ?? {}) } };
 	const viewParams = variant.views ?? {};
+	let keySource = process.env.TYPESAFE_API_KEY === cfg.apiKey && cfg.apiKey ? "env" : cfg.apiKey ? keyFilePath() : "none";
 	let usingMock = cfg.forceMock || !cfg.apiKey;
 	let classifier: Classifier = usingMock ? new MockClassifier() : new JevClassifier(cfg);
 	let presend: PresendClassifier = usingMock ? new MockPresend() : new JevPresend(new TypeSafeClient({ apiKey: cfg.apiKey }), cfg.model, prompts);
@@ -54,7 +58,9 @@ export default function (pi: ExtensionAPI) {
 	const recordById = new Map<string, CompressedRecord>();
 	const remember = (r: CompressedRecord) => { records.push(r); recordById.set(r.id, r); if (records.length > 200) { const old = records.shift(); if (old) recordById.delete(old.id); } };
 	let lastAssistantText = "";
+	let health = new Health();
 	let presendTotals = { considered: 0, compressed: 0, tokensSaved: 0, recalls: 0 };
+	let restored = { compressed: 0, tokensSaved: 0 };
 
 	let ledger = new Map<string, Decision>();
 	/** Classifications launched but not yet resolved, keyed by toolCallId. */
@@ -96,14 +102,18 @@ export default function (pi: ExtensionAPI) {
 		return sent > 0 ? Math.round((100 * kept) / (sent + kept)) : undefined;
 	};
 	const statusText = () => {
-		const tag = usingMock ? "jev-lens(mock)" : "jev-lens";
+		const tag = !cfg.enabled ? "jev-lens(disabled)" : usingMock ? "jev-lens(mock)" : "jev-lens";
 		const pct = cutShare();
-		const lead = pct === undefined ? tag : `${tag} −${pct}% of input`;
+		const label = health.failing ? `${tag}(degraded)` : tag;
+		const lead = pct === undefined ? label : `${label} −${pct}% of input`;
 		const pruned = cfg.mode === "off" ? "" : `, pruned −${(totals.pruned / 1000).toFixed(1)}k · ${totals.applied}`;
-		return `${lead} (presend −${(presendTotals.tokensSaved / 1000).toFixed(1)}k · ${presendTotals.compressed}/${presendTotals.considered} · ${presendTotals.recalls} recalls${pruned})`;
+		const saved = presendTotals.tokensSaved + restored.tokensSaved;
+		const counts = `${presendTotals.compressed}/${presendTotals.considered}${restored.compressed ? ` new · ${restored.compressed} restored` : ""}`;
+		return `${lead} (presend −${(saved / 1000).toFixed(1)}k · ${counts} · ${presendTotals.recalls} recalls${pruned})`;
 	};
 	const status = (ctx: ExtensionContext) => {
 		if (!ctx.hasUI) return;
+		for (const warning of health.warnings()) ctx.ui.notify(warning, "warning");
 		ctx.ui.setStatus("jev-lens", statusText());
 	};
 
@@ -116,6 +126,7 @@ export default function (pi: ExtensionAPI) {
 		sessionAbort.abort();
 		sessionAbort = new AbortController();
 		lastAssistantText = "";
+		health = new Health();
 		ledger = rebuildLedger(ctx.sessionManager.getEntries());
 		buffer = [];
 		inflight.clear();
@@ -130,6 +141,7 @@ export default function (pi: ExtensionAPI) {
 		records.length = 0;
 		recordById.clear();
 		presendTotals = { considered: 0, compressed: 0, tokensSaved: 0, recalls: 0 };
+		restored = { compressed: 0, tokensSaved: 0 };
 		try {
 			mkdirSync(join(ctx.cwd, CONFIG_DIR_NAME), { recursive: true });
 			logPath = join(ctx.cwd, CONFIG_DIR_NAME, "jev-lens.log");
@@ -147,6 +159,8 @@ export default function (pi: ExtensionAPI) {
 				if (d?.full) {
 					fullOutputs.set(entry.message.toolCallId, { text: d.full, toolName: entry.message.toolName, args: d.args, view: d.view ?? "?" });
 					const sent = contentText(entry.message.content).replace(/\n\n\[jev-lens:[\s\S]*$/, "");
+					restored.compressed++;
+					restored.tokensSaved += Math.max(0, estimateTokensOfText(d.full) - estimateTokensOfText(sent));
 					remember({ id: entry.message.toolCallId, toolName: entry.message.toolName, args: d.args, kind: d.kind ?? "?", view: d.view ?? "?", tokensBefore: estimateTokensOfText(d.full), tokensAfter: estimateTokensOfText(sent), full: d.full, sent, included: d.included ?? [], needsFull: d.needsFull, pFull: d.p?.full, recalls: 0, at: entry.message.timestamp });
 				}
 			}
@@ -206,20 +220,19 @@ export default function (pi: ExtensionAPI) {
 		}
 	});
 
-	pi.on("agent_end", async () => {
-		const epoch = generation;
+	pi.on("agent_end", async (_event, ctx) => {
 		// No further assistant reaction is coming for the last results; classify with what we have.
 		const toClassify = buffer;
 		buffer = [];
-		for (const item of toClassify) launchClassification(item, "", [], undefined);
+		for (const item of toClassify) launchClassification(item, "", [], ctx);
 		await waitForWork([...inflight.values()]);
-		void epoch;
 	});
 
 	function launchClassification(item: PendingResult, afterText: string, afterCalls: { name: string; arguments: unknown }[], ctx?: ExtensionContext) {
 		const m = item.message;
 		if (cfg.mode === "off" || sessionAbort.signal.aborted || m.content.some((c) => c.type !== "text")) return;
 		const epoch = generation;
+		const signal = workSignal(ctx?.signal);
 		if (ledger.has(m.toolCallId) || inflight.has(m.toolCallId)) return;
 		const output = contentText(m.content);
 		const tokens = estimateTokensOfText(output);
@@ -238,9 +251,11 @@ export default function (pi: ExtensionAPI) {
 		const summary = describeToolCall(m.toolName, item.args, output.length, lines);
 		const started = Date.now();
 		const p = classifier
-			.classifyToolResult(state, workSignal(ctx?.signal))
+			.classifyToolResult(state, signal)
 			.then((probs) => {
-				if (epoch !== generation) return;
+				if (epoch !== generation || signal.aborted) return;
+				health.success("postsend");
+				if (ctx) status(ctx);
 				const decision: Decision = {
 					id: m.toolCallId,
 					toolName: m.toolName,
@@ -256,7 +271,10 @@ export default function (pi: ExtensionAPI) {
 				log({ event: "decision", id: decision.id, tool: m.toolName, bucket: decision.bucket, p: probs, tokens, ms: Date.now() - started, summary });
 			})
 			.catch((err) => {
-				if (epoch === generation) log({ event: "classify_error", id: m.toolCallId, error: String(err?.message ?? err) });
+				if (epoch !== generation || signal.aborted) return;
+				health.failure("postsend", err);
+				log({ event: "classify_error", id: m.toolCallId, error: health.lines()[1] });
+				if (ctx) status(ctx);
 			})
 			.finally(() => { if (epoch === generation) inflight.delete(m.toolCallId); });
 		inflight.set(m.toolCallId, p);
@@ -348,16 +366,16 @@ export default function (pi: ExtensionAPI) {
 		if (event.content.some((c) => c.type === "image")) return;
 		presendTotals.considered++;
 		const started = Date.now();
-		const terms = extractTerms(latestUser, lastAssistantText, JSON.stringify(event.input ?? {}));
-		const cands = await buildCandidatesAsync(event.toolName, event.input, text, terms, viewParams);
-		if (epoch !== generation || signal.aborted) return;
-		if (cands.views.length < 2) {
-			log({ event: "presend", id: event.toolCallId, tool: event.toolName, tokens, view: "full", reason: "no-candidates" });
-			return;
-		}
-		const totalLines = text.split("\n").length;
-		const state = buildPresendState(cfg, { firstUser, latestUser, agentText: lastAssistantText, toolName: event.toolName, args: event.input, isError: event.isError, cands, totalLines, totalChars: text.length });
 		try {
+			const terms = extractTerms(latestUser, lastAssistantText, JSON.stringify(event.input ?? {}));
+			const cands = await buildCandidatesAsync(event.toolName, event.input, text, terms, viewParams);
+			if (epoch !== generation || signal.aborted) return;
+			if (cands.views.length < 2) {
+				log({ event: "presend", id: event.toolCallId, tool: event.toolName, tokens, view: "full", reason: "no-candidates" });
+				return;
+			}
+			const totalLines = text.split("\n").length;
+			const state = buildPresendState(cfg, { firstUser, latestUser, agentText: lastAssistantText, toolName: event.toolName, args: event.input, isError: event.isError, cands, totalLines, totalChars: text.length });
 			const answer = await presend.choose(state, cands.views.map((v) => v.kind), signal);
 			if (epoch !== generation || signal.aborted) return;
 			let view = decideView(answer, cands, cfg);
@@ -368,6 +386,8 @@ export default function (pi: ExtensionAPI) {
 				if (epoch !== generation || signal.aborted) return;
 				if (ex) { view = ex.view; expanded = ex.probs.map((p, i) => (p > above ? i : -1)).filter((i) => i >= 0); }
 			}
+			health.success("presend");
+			status(ctx);
 			log({ event: "presend", id: event.toolCallId, tool: event.toolName, kind: cands.kind, tokens, view: view.kind, viewTokens: estimateTokensOfText(view.text), chosen: answer.choice, needsFull: answer.needsFull, p: answer.probabilities, confidence: answer.confidence, expanded, candidates: cands.views.map((v) => `${v.kind}:${v.chars}`), ms: Date.now() - started });
 			if (view.kind === "full") return;
 			presendTotals.compressed++;
@@ -378,7 +398,10 @@ export default function (pi: ExtensionAPI) {
 			status(ctx);
 			return { content: [{ type: "text", text: view.text + footer(view, event.toolCallId, totalLines) }], details };
 		} catch (err) {
-			if (epoch === generation) log({ event: "presend_error", id: event.toolCallId, error: String((err as Error)?.message ?? err) });
+			if (epoch !== generation || signal.aborted) return;
+			health.failure("presend", err);
+			log({ event: "presend_error", id: event.toolCallId, error: health.lines()[0] });
+			status(ctx);
 			return;
 		}
 	});
@@ -427,71 +450,69 @@ export default function (pi: ExtensionAPI) {
 
 	if (process.env.JEV_LENS_UI !== "0") {
 		const cwd = process.cwd();
-		const originals: Record<string, ReturnType<typeof createReadTool>> = {
-			read: createReadTool(cwd) as ReturnType<typeof createReadTool>,
-			bash: createBashTool(cwd) as unknown as ReturnType<typeof createReadTool>,
-			grep: createGrepTool(cwd) as unknown as ReturnType<typeof createReadTool>,
-			find: createFindTool(cwd) as unknown as ReturnType<typeof createReadTool>,
-			ls: createLsTool(cwd) as unknown as ReturnType<typeof createReadTool>,
-		};
-		for (const [name, original] of Object.entries(originals)) {
-			const o = original as unknown as { description: string; parameters: never; execute: (...a: unknown[]) => Promise<unknown>; renderCall?: (...a: unknown[]) => unknown; renderResult?: (...a: unknown[]) => unknown; promptSnippet?: string; promptGuidelines?: string[] };
+		// Tool definitions include pi's renderers; create*Tool() strips them.
+		const originals: ToolDefinition<any, any>[] = [
+			createReadToolDefinition(cwd), createBashToolDefinition(cwd),
+			createGrepToolDefinition(cwd), createFindToolDefinition(cwd), createLsToolDefinition(cwd),
+		];
+		for (const original of originals) {
 			pi.registerTool({
-				name,
-				label: name,
-				description: o.description,
-				parameters: o.parameters,
-				promptSnippet: o.promptSnippet,
-				promptGuidelines: o.promptGuidelines,
-				async execute(toolCallId: string, params: unknown, signal: AbortSignal | undefined, onUpdate: unknown, ctx: unknown) {
-					return (o.execute as (id: string, p: unknown, s: unknown, u: unknown, c: unknown) => Promise<never>)(toolCallId, params, signal, onUpdate, ctx);
-				},
-				renderCall(args: unknown, theme: Theme, context: unknown) {
-					if (o.renderCall) return (o.renderCall as (a: unknown, t: unknown, c: unknown) => never)(args, theme, context);
-					const a = args as Record<string, unknown>;
-					const what = typeof a.path === "string" ? a.path : typeof a.command === "string" ? a.command : typeof a.pattern === "string" ? a.pattern : "";
-					return new Text(theme.fg("toolTitle", theme.bold(`${name} `)) + theme.fg("accent", String(what)), 0, 0);
-				},
-				renderResult(result: { content: unknown }, options: { expanded: boolean }, theme: Theme, context: { toolCallId: string }) {
+				...original,
+				renderResult(result, options, theme, context) {
 					const rec = recordById.get(context.toolCallId);
-					if (!rec) {
-						if (o.renderResult) return (o.renderResult as (r: unknown, op: unknown, t: unknown, c: unknown) => never)(result, options, theme, context);
-						const text = contentText(result.content);
-						const lines = text.split("\n");
-						let out = theme.fg("success", `${lines.length} lines`);
-						if (options.expanded) out += "\n" + lines.slice(0, 200).join("\n");
-						else out += theme.fg("dim", "  " + lines[0]?.slice(0, 80));
-						return new Text(out, 0, 0);
+					if (!rec || options.isPartial || context.isError) {
+						return original.renderResult!(result, options, theme, context);
 					}
 					let out = savingsLine(rec, theme);
 					if (options.expanded) out += "\n" + rec.sent;
 					else out += "\n" + theme.fg("dim", rec.sent.split("\n").slice(0, 3).join("\n"));
 					return new Text(out, 0, 0);
 				},
-			} as never);
+			});
 		}
 	}
 
 	// ---- commands ----------------------------------------------------------------------
 
 	pi.registerCommand("jev-lens", {
-		description: "jev-lens: stats | list (compressed results) | diff [n] (original vs sent, overlay) | decisions | key [api-key] (store your TypeSafe key)",
+		description: "Inspect compression and setup: stats | list | diff [n] | decisions | key | help",
+		getArgumentCompletions: (prefix) => commandCompletions(prefix, records),
 		handler: async (args, ctx) => {
 			const sub = (args ?? "").trim();
-			if (sub === "key" || sub.startsWith("key ")) {
+			if (sub === "help" || sub === "--help" || sub === "-h") {
+				ctx.ui.notify(commandHelp, "info");
+				return;
+			}
+			if (/^key(?:\s|$)/.test(sub)) {
 				let key = sub.slice(3).trim();
-				if (!key) key = ((await ctx.ui.input("TypeSafe API key (from console.typesafe.ai):", "ts_...")) ?? "").trim();
-				if (!key) { ctx.ui.notify("no key entered", "info"); return; }
-				const where = storeKey(key);
+				if (!key && (!ctx.hasUI || ctx.mode !== "tui")) { ctx.ui.notify("Masked key input requires terminal mode. Set TYPESAFE_API_KEY or run /jev-lens key in interactive pi.", "warning"); return; }
+				if (!key) key = ((await ctx.ui.custom<string | undefined>((tui, theme, keys, done) =>
+					new SecretInput(theme, keys, done, () => tui.requestRender()),
+				)) ?? "").trim();
+				if (!key) { ctx.ui.notify("Key setup cancelled. The current key is unchanged.", "info"); return; }
+				let where: string;
+				try { where = storeKey(key); }
+				catch {
+					ctx.ui.notify(`Could not store the key in ${keyFilePath()}. Check directory permissions or set TYPESAFE_API_KEY.`, "error");
+					return;
+				}
 				useKey(key);
-				ctx.ui.notify(`jev-lens: key stored in ${where}; jev is active from the next tool result`, "info");
+				keySource = where;
+				const next = cfg.forceMock ? "Mock mode remains active. Unset JEV_LENS_CLASSIFIER and reload pi to use jev." : !cfg.enabled || !cfg.presend ? "Pre-send compression is disabled. See /jev-lens stats." : "jev will use this key from the next tool result. The key has not been validated.";
+				ctx.ui.notify(`jev-lens: key stored in ${where}. ${next}${process.env.TYPESAFE_API_KEY ? " TYPESAFE_API_KEY takes priority again after reload." : ""}`, "info");
 				status(ctx);
 				return;
 			}
-			if (sub.startsWith("diff")) {
-				const n = Number(sub.slice(4).trim() || "1");
-				const rec = records[records.length - (Number.isFinite(n) && n >= 1 ? n : 1)];
-				if (!rec) { ctx.ui.notify("no compressed tool result to show yet", "info"); return; }
+			if (/^diff(?:\s|$)/.test(sub)) {
+				const arg = sub.slice(4).trim();
+				const n = Number(arg || "1");
+				if ((arg && !/^\d+$/.test(arg)) || !Number.isSafeInteger(n) || n < 1) {
+					ctx.ui.notify("Usage: /jev-lens diff [n]. Use a positive whole number. 1 is the newest result.", "warning");
+					return;
+				}
+				if (!records.length) { ctx.ui.notify("No compressed results yet. Use /jev-lens stats to inspect compression settings.", "info"); return; }
+				const rec = records[records.length - n];
+				if (!rec) { ctx.ui.notify(`Result ${n} is not available. Choose 1-${records.length} from /jev-lens list.`, "warning"); return; }
 				if (!ctx.hasUI || ctx.mode !== "tui") { ctx.ui.notify(listLines([rec], { fg: (_c, t) => t, bold: (t) => t }).join("\n"), "info"); return; }
 				await ctx.ui.custom<void>((tui, theme, _kb, done) => {
 					const height = Math.max(12, Math.floor(((tui as { terminalHeight?: number }).terminalHeight ?? process.stdout.rows ?? 40) * 0.85));
@@ -506,17 +527,23 @@ export default function (pi: ExtensionAPI) {
 			}
 			if (sub === "decisions") {
 				const rows = [...ledger.values()].map((d) => `${d.status === "applied" ? "●" : "○"} ${d.bucket.padEnd(6)} n=${d.p.needed.toFixed(2)} o=${d.p.outcomeOnly.toFixed(2)} ${d.tokensBefore}t ${d.summary}`);
-				ctx.ui.notify(rows.join("\n") || "(no decisions yet)", "info");
+				ctx.ui.notify(rows.join("\n") || (cfg.mode === "off" ? "Post-send pruning is off (the default). Pre-send compression is separate: see /jev-lens stats." : "No post-send decisions yet."), "info");
+				return;
+			}
+			if (sub && sub !== "stats") {
+				ctx.ui.notify("Unknown subcommand or extra arguments. Run /jev-lens help for usage.", "warning");
 				return;
 			}
 			const hit = totals.input + totals.cacheRead > 0 ? Math.round((100 * totals.cacheRead) / (totals.input + totals.cacheRead)) : 0;
 			ctx.ui.notify(
 				[
-					`mode=${cfg.mode} enabled=${cfg.enabled} classifier=${usingMock ? "mock (no key: /jev-lens key)" : cfg.model} key=${process.env.TYPESAFE_API_KEY ? "env" : cfg.apiKey ? keyFilePath() : "none"}`,
-					`presend: ${presendTotals.compressed}/${presendTotals.considered} large results compressed, ≈${presendTotals.tokensSaved} tokens saved, ${presendTotals.recalls} recalls`,
+					`mode=${cfg.mode} enabled=${cfg.enabled} presend=${cfg.presend} classifier=${usingMock ? cfg.forceMock ? "mock (forced by JEV_LENS_CLASSIFIER)" : "mock (no key: /jev-lens key)" : cfg.model} key=${keySource}`,
+					`presend since load: ${presendTotals.compressed}/${presendTotals.considered} large results compressed, ≈${presendTotals.tokensSaved} tokens saved, ${presendTotals.recalls} recalls`,
+					`restored from session: ${restored.compressed} compressed results, ≈${restored.tokensSaved} tokens saved (included in footer savings)`,
 					`post-send: calls=${totals.calls} decisions=${ledger.size} applied=${totals.applied} pruned≈${totals.pruned} tokens`,
+					...health.lines(),
 					`cache: read=${totals.cacheRead} uncached=${totals.input} hit=${hit}%`,
-					`input cut: ${cutShare() ?? 0}% of the session's input tokens (≈${cut.presend + cut.pruned} of ${totals.input + totals.cacheRead + cut.presend + cut.pruned}: presend ${cut.presend}, pruned ${cut.pruned}, summed over ${totals.calls} calls)`,
+					`input cut: ${cutShare() ?? 0}% of input tokens counted since load (≈${cut.presend + cut.pruned} of ${totals.input + totals.cacheRead + cut.presend + cut.pruned}: presend ${cut.presend}, pruned ${cut.pruned}, summed over ${totals.calls} calls)`,
 				].join("\n"),
 				"info",
 			);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-jev-lens",
-  "version": "0.2.0",
+  "version": "0.3.0",
   "description": "pi extension that compresses large tool results before they reach the model: jev picks the view (outline, relevant blocks, sections, signals, testlog), full text stays recallable",
   "author": "Didrik Rognstad",
   "license": "MIT",
@@ -51,7 +51,7 @@
     "web-tree-sitter": "^0.27.0"
   },
   "peerDependencies": {
-    "@earendil-works/pi-coding-agent": "*",
+    "@earendil-works/pi-coding-agent": ">=0.84.3",
     "@earendil-works/pi-tui": "*",
     "typebox": "*"
   },

package/src/commands.ts

@@ -0,0 +1,35 @@
+import type { AutocompleteItem } from "@earendil-works/pi-tui";
+import type { CompressedRecord } from "./ui.ts";
+
+const commands = [
+	{ value: "stats", label: "stats", description: "Show statistics and active configuration" },
+	{ value: "list", label: "list", description: "List recent compressed results" },
+	{ value: "diff", label: "diff", description: "Compare original and sent output: diff [n], newest = 1" },
+	{ value: "decisions", label: "decisions", description: "Show post-send pruning decisions" },
+	{ value: "key", label: "key", description: "Store a TypeSafe API key" },
+	{ value: "help", label: "help", description: "Show commands and usage" },
+];
+
+export const commandHelp = [
+	"/jev-lens [stats] — Show statistics and active configuration.",
+	...commands.slice(1).map((c) => `/jev-lens ${c.value === "diff" ? "diff [n]" : c.value} — ${c.description}.`),
+	"For diff, 1 is the newest result. Use /jev-lens list to find a number.",
+	"Press Tab after /jev-lens to complete a subcommand.",
+].join("\n");
+
+/** Pi replaces the entire argument prefix, so diff values include the subcommand. */
+export function commandCompletions(prefix: string, records: CompressedRecord[]): AutocompleteItem[] | null {
+	const input = prefix.trimStart();
+	const diff = /^diff\s+(\d*)$/.exec(input);
+	let items: AutocompleteItem[];
+	if (diff) {
+		items = [...records].reverse().map((r, i) => ({
+			value: `diff ${i + 1}`,
+			label: `diff ${i + 1}`,
+			description: `${r.toolName} · ${r.view} · ${r.tokensBefore} → ${r.tokensAfter} tokens`,
+		})).filter((_, i) => String(i + 1).startsWith(diff[1]));
+	} else {
+		items = commands.filter((c) => c.value.startsWith(input));
+	}
+	return items.length ? items : null;
+}

package/src/health.ts

@@ -0,0 +1,38 @@
+export type Stage = "presend" | "postsend";
+
+type StageHealth = { failures: number; failing: boolean; reason: string; notified: boolean };
+
+/** Session-local failure counters. Never expose provider error messages or credentials. */
+export class Health {
+	private stages: Record<Stage, StageHealth> = {
+		presend: { failures: 0, failing: false, reason: "", notified: false },
+		postsend: { failures: 0, failing: false, reason: "", notified: false },
+	};
+
+	failure(stage: Stage, error: unknown): void {
+		const status = error && typeof error === "object" && "status" in error ? error.status : undefined;
+		const reason = status === 401 || status === 403 ? "Check your TypeSafe API key."
+			: status === 429 ? "TypeSafe rejected the request because of a usage limit."
+			: "Check your connection and TypeSafe service availability.";
+		Object.assign(this.stages[stage], { failures: this.stages[stage].failures + 1, failing: true, reason });
+	}
+
+	success(stage: Stage): void { this.stages[stage].failing = false; }
+
+	get failing(): boolean { return Object.values(this.stages).some((s) => s.failing); }
+
+	/** At most one warning per stage per session, including work completed without a UI context. */
+	warnings(): string[] {
+		return (Object.entries(this.stages) as [Stage, StageHealth][]).flatMap(([stage, state]) => {
+			if (!state.failures || state.notified) return [];
+			state.notified = true;
+			const effect = stage === "presend" ? "Full output was kept." : "The affected result was not pruned.";
+			return [`jev-lens: ${stage === "presend" ? "Pre-send compression" : "Post-send classification"} failed. ${effect} ${state.reason} See /jev-lens stats. Further failures appear there without repeated warnings.`];
+		});
+	}
+
+	lines(): string[] {
+		return (Object.entries(this.stages) as [Stage, StageHealth][]).map(([stage, state]) =>
+			`${stage} failures: ${state.failures}${state.failures ? state.failing ? ` (last attempt failed). ${state.reason}` : " (a later attempt succeeded)" : ""}`);
+	}
+}

package/src/secret-input.ts

@@ -0,0 +1,124 @@
+import { CURSOR_MARKER, decodeKittyPrintable, truncateToWidth, type Component, type Focusable, type KeybindingsManager } from "@earendil-works/pi-tui";
+import type { ThemeLike } from "./ui.ts";
+
+const PASTE_START = "\x1b[200~", PASTE_END = "\x1b[201~";
+const MAX_LENGTH = 4096;
+
+/** A secret-only editor: no plaintext rendering, history, clipboard, undo, or reveal action. */
+export class SecretInput implements Component, Focusable {
+	focused = false;
+	private value: string[] = [];
+	private cursor = 0;
+	private paste: string | undefined;
+	private pasteTooLong = false;
+	private error = "";
+	private closed = false;
+
+	constructor(
+		private theme: ThemeLike,
+		private keys: Pick<KeybindingsManager, "matches" | "getKeys">,
+		private done: (value: string | undefined) => void,
+		private requestRender: () => void,
+	) {}
+
+	private insert(text: string): void {
+		const chars = Array.from(text);
+		if (/[\x00-\x1f\x7f-\x9f]/.test(text)) {
+			this.error = "Paste a single-line API key. Control characters are not allowed.";
+		} else if (this.value.length + chars.length > MAX_LENGTH) {
+			this.error = `The key is too long. Maximum: ${MAX_LENGTH} characters.`;
+		} else {
+			this.value.splice(this.cursor, 0, ...chars);
+			this.cursor += chars.length;
+			this.error = "";
+		}
+	}
+
+	private finish(value?: string): void {
+		this.dispose();
+		this.done(value);
+	}
+
+	/** Drop references on submit, cancel, or external teardown. JS cannot guarantee memory erasure. */
+	dispose(): void {
+		this.value.fill("");
+		this.value = [];
+		this.cursor = 0;
+		this.paste = undefined;
+		this.error = "";
+		this.closed = true;
+	}
+
+	handleInput(data: string): void {
+		if (this.closed) return;
+		if (this.paste === undefined && data.startsWith(PASTE_START)) {
+			this.paste = "";
+			this.pasteTooLong = false;
+			data = data.slice(PASTE_START.length);
+		}
+		if (this.paste !== undefined) {
+			this.paste += data;
+			const end = this.paste.indexOf(PASTE_END);
+			if (end >= 0) {
+				const text = this.paste.slice(0, end);
+				const remaining = this.paste.slice(end + PASTE_END.length);
+				this.paste = undefined;
+				if (this.pasteTooLong) this.error = "The pasted key is too long. Paste only the API key.";
+				else this.insert(text.trim());
+				if (remaining) this.handleInput(remaining);
+			} else if (this.paste.length > MAX_LENGTH + PASTE_END.length) {
+				this.pasteTooLong = true;
+				this.paste = this.paste.slice(-PASTE_END.length); // retain a possible split end marker
+			}
+		} else if (this.keys.matches(data, "tui.select.cancel")) {
+			this.finish();
+		} else if (this.keys.matches(data, "tui.input.submit")) {
+			if (!this.error) this.finish(this.value.join("").trim() || undefined);
+		} else if (this.keys.matches(data, "tui.editor.cursorLeft")) {
+			this.cursor = Math.max(0, this.cursor - 1);
+		} else if (this.keys.matches(data, "tui.editor.cursorRight")) {
+			this.cursor = Math.min(this.value.length, this.cursor + 1);
+		} else if (this.keys.matches(data, "tui.editor.cursorLineStart")) {
+			this.cursor = 0;
+		} else if (this.keys.matches(data, "tui.editor.cursorLineEnd")) {
+			this.cursor = this.value.length;
+		} else if (this.keys.matches(data, "tui.editor.deleteCharBackward")) {
+			if (this.cursor) this.value.splice(--this.cursor, 1);
+			this.error = "";
+		} else if (this.keys.matches(data, "tui.editor.deleteCharForward")) {
+			this.value.splice(this.cursor, 1);
+			this.error = "";
+		} else if (this.keys.matches(data, "tui.editor.deleteToLineStart")) {
+			this.value.splice(0, this.cursor);
+			this.cursor = 0;
+			this.error = "";
+		} else if (this.keys.matches(data, "tui.editor.deleteToLineEnd")) {
+			this.value.splice(this.cursor);
+			this.error = "";
+		} else {
+			const text = decodeKittyPrintable(data) ?? data;
+			if (!/[\x00-\x1f\x7f-\x9f]/.test(text)) this.insert(text);
+		}
+		this.requestRender();
+	}
+
+	invalidate(): void {}
+
+	render(width: number): string[] {
+		if (width < 1) return [""];
+		const room = Math.max(1, width - 2);
+		const start = Math.max(0, this.cursor - room + 1);
+		const before = "*".repeat(this.cursor - start);
+		const after = "*".repeat(Math.min(this.value.length - this.cursor, room - before.length - 1));
+		const cursor = this.cursor < this.value.length ? "*" : " ";
+		const marker = this.focused ? CURSOR_MARKER : "";
+		const field = `${width > 2 ? "> " : ""}${before}${marker}${this.focused ? `\x1b[7m${cursor}\x1b[27m` : cursor}${after}`;
+		return [
+			this.theme.fg("accent", "TypeSafe API key (masked)"),
+			this.theme.fg("dim", "Get a key at console.typesafe.ai. Type or paste it below."),
+			field,
+			this.theme.fg("dim", `${this.keys.getKeys("tui.input.submit").join("/")}: save · ${this.keys.getKeys("tui.select.cancel").join("/")}: cancel`),
+			...(this.error ? [this.theme.fg("warning", this.error)] : []),
+		].map((line) => truncateToWidth(line, width));
+	}
+}