pi-warden 0.26.0 → 0.34.1

package/CHANGELOG.md

@@ -8,6 +8,230 @@ How to keep this current: add the entry in the same pull request as the change,
 
 <!-- Empty. Next release starts here. -->
 
+## 0.34.1
+
+### Fixed
+- `scripts/calibrate-action.mjs`: `--yes` now spends what the corpus needs instead of stopping at the 2000-request default; `--max-requests N` is an explicit cap that `--yes` does not lift, and a run it stops early says how many replays it skipped and writes `report-latest-partial.md`.
+
+### Docs
+- Action guard calibration on 315 recorded sessions at 0.33.3 under `eval/reports/2026-09-21-calibration-0.33.3/`, with the headline table in `docs/guards.md`.
+- README measurement paragraph reworded for readability, same numbers.
+
+## 0.34.0
+
+### Added
+- `scripts/hold-stats.mjs`: read-only script reporting per-project and total hold statistics from the SQLite database, with `--json` output and `PI_WARDEN_DB` support.
+- `/warden status` now shows a lifetime hold sentence for the current project root (e.g. "Lifetime here: 55 holds, 11 labeled, 0 stood (0/11), 114 allowed accepted, 0 regretted.").
+
+### Docs
+- README "Does it actually help?" section now includes a measurement-on-real-use paragraph with current hold numbers, date, and the link to the floor-evidence decision in 0.33.0.
+
+## 0.33.4
+
+### Fixed
+- Rules guard no longer judges prose-only fallback documents (README.md, CLAUDE.md, AGENTS.md) as one rule; a fallback with no rule-shaped sections is skipped.
+
+## 0.33.3
+
+### Tests
+- Live smoke stuck suite now exercises the Jev-judged path: two new cases reach `source: "typesafe"` and print real scores; the existing offline-repeat case prints `offline repeat` instead of three `undefined` values.
+
+## 0.33.2
+
+### Fixed
+- Tests and eval runs now write to an isolated database via `PI_WARDEN_DB` instead of polluting the user's `holds.db`.
+
+## 0.33.1
+
+### Tests
+- Live smoke suite cap raised from 60 to 100 requests; summary line now reports budget-error misses.
+
+## 0.33.0
+
+### Changed
+- Built-in pattern floor is now evidence when a judge answers (`action.floor: "evidence"`, default). Built-in hits (shell rules, rm classifier, sensitive-path, outside-project) are fed to the judge as `floor_hits` in the request state and traced as `(evidence)`, but they no longer override the judge's `irreversible` score. User-declared rules keep their declared action. `action.floor: "level"` restores the legacy behaviour. Replay on 52 real holds: evidence mode drops held count from 52 to 28; level mode preserves 44 confirm + 8 warn.
+
+### Fixed
+- Deferred sensitive-path hit now warns (instead of silently allowing) when the judge fails in evidence mode with `failOpen: true`.
+
+## 0.32.0
+
+### Fixed
+- `replanned` outcomes now persist to SQLite, closing the gap where re-plan labels were set in memory but never written to the database.
+- `command_preview` stores the redacted command or path (capped at 200 chars) instead of the bare tool name.
+- Outcome known at record time (dialog-approved/declined) is no longer lost to a race between `noteOutcomes` and `learningIds.set`.
+- Set `PRAGMA busy_timeout` on the SQLite connection and skip VACUUM when no rows are pruned, preventing SQLITE_BUSY on startup with concurrent sessions. Fixes #36.
+
+### Changed
+- Judged allowed calls (`held = 0`) are now stored in SQLite alongside held calls, enabling precision and false-negative rate computation.
+- Disclosure and `docs/data-handling.md` updated to reflect that judged allowed calls are stored with `held = 0`.
+
+## 0.31.0
+
+### Added
+- `/warden audit` command: agent-driven workspace audit that sends a prompt to the session model. The model reads source, finds concrete Jev opportunities, produces measurable evidence, and writes an HTML report to `.pi-warden/audit-report.html`.
+
+### Changed
+- Bumped pi-typesafe dependency to ^0.6.1.
+
+## 0.30.4
+
+### Changed
+- Made `should_proceed` trace-only by default until calibrated, so low scores no longer ask the agent to pause. Set `action.shouldProceed.steer: true` to restore the existing steer; the `hold` threshold is unchanged.
+
+## 0.30.3
+
+### Fixed
+- Live status follows the newest verdict, including guards without sentence tokens; stack entries retain guard update order within their severity groups.
+- Action warning and hold sentences use verdict reasons instead of inferring irreversibility from the level or unrelated scores.
+
+## 0.30.2
+
+### Fixed
+- The live widget bar wraps its sentence to the pane width; one over-wide line tripped pi's render-width guard and aborted the session. Fixes #29.
+
+## 0.30.1
+
+### Fixed
+- Approval question now accepts generic task-level approval ("proceed", "yes", "go ahead", "sure") instead of requiring explicit approval of each specific action. Fixes the pattern where the model keeps holding after the user approved the whole task.
+
+### Changed
+- Approval question uses intent-based wording that lets Jev reason about approval directly rather than matching specific words.
+- Bumped pi-typesafe dependency to ^0.6.1.
+
+## 0.30.0
+
+### Added
+- Violation pipeline: deterministic per-violation authorization, escalation, and aggregation for pattern-detected and rules-guard violations.
+- `Violation`, `ViolationScope`, `Authorization`, `EscalatedViolation` types for the violation pipeline.
+- `authorize()`, `isNegated()`, `scopeMatches()`, `isAuthEligible()` (severity-based) for deterministic authorization.
+- `escalateBlastRadius()` and `escalateRulesViolation()` for Jev-backed severity escalation.
+- `aggregateLevel()` for final tool-call level from remaining violations.
+- `resolveRulesFile()` and `extractRules()` for token-aware rules file resolution.
+- `checkPiWardenMissing()` for first-run warning support.
+- `escalationThreshold` config key in `ActionGuardConfig` (default 0.85).
+- `/warden init` command: scaffolds a starter pi-warden.md with safety rules and project-type-specific rules.
+- `writeStarterRules()`, `generateStarterRules()`, `detectProjectType()`, `buildProjectContext()` in `src/init.ts`.
+- First-run warning when pi-warden.md is missing and a fallback is active.
+- Per-violation noul questions sent to Jev on the same request, returning P(yes) as a confidence value. Answers drive escalation (`escalateBlastRadius`, `escalateRulesViolation`) and aggregation (`aggregateLevel`) in the action guard pipeline. Calibrated: AUC 0.73 against regret, 0.42 against rejected turns (600 sessions, 2026-09-20).
+- `parseViolationJudgments()` for safe parsing of Jev noul responses with defaults for missing/malformed data; supports legacy choice fallback.
+- Resolved rules file (`rules`, `rulesSource`) passed to Jev in the request state.
+- `should_proceed` noul question on every action request: unified gate covering rule violations, unrequested scope, explicit constraint breaches, and material user decisions. Calibrated: AUC 0.26 against regret, 0.58 against rejected turns (600 sessions, 2026-09-20).
+- `action.shouldProceed` config key `{ hold: number }` (default 0.6): steer threshold for `should_proceed`.
+- `shouldProceedQuestion` exported from `src/guard.ts`.
+- `Verdict.shouldProceedSteer` flag for the unified gate steer.
+- `Judgment.shouldProceed` field for traceability.
+- `matchPathRules` is now exported for use by conscience-loader.
+
+### Changed
+- `ViolationScope` no longer has a `labels` field; scope matching uses paths only.
+- `Violation` no longer has an `authEligible` field; eligibility is derived from severity in `authorize()`.
+- `isAuthEligible()` now takes a `Severity` parameter instead of a `PatternHit`.
+- `escalateRulesViolation()` threshold check now matches `escalateBlastRadius()` style (guard clause for no-escalation).
+
+### Tests
+- 38 tests for authorization, escalation, aggregation, violation judgment parsing, rules-file resolution, and first-run warning.
+
+## 0.29.3
+
+### Fixed
+
+- Trace-only off-task findings no longer reach the agent through dedicated steers, generic headless warnings, or the headless `/warden test` report. Independent warning and confirmation reasons on the same call still deliver normally.
+
+## 0.29.2
+
+### Added
+
+- GitHub Actions CI for pull requests, main pushes, version tags, and manual runs: workflow lint plus typecheck, offline tests, and build on Node 22.19.0, 24, and 26.
+- Package installation smoke checks and downloadable npm tarballs with SHA-256 checksums; validated version tags prepare draft GitHub releases without publishing to npm.
+
+### Docs
+
+- Documented CI checks and manual release steps, and corrected the contributor version-bump instructions.
+
+## 0.29.1
+
+### Changed
+
+- Extracted duplicated credential-key regex alternation in `redact.ts` into a shared `CREDENTIAL_KEYS` constant.
+
+## 0.29.0
+
+### Changed
+
+- Off-task is now gated on the categorical scope answer instead of the score alone (AUC 0.51). `expected_step` vetoes off-task entirely; `unrelated` always warns; `plausible_side_step` warns trace-only. All off-task steers are trace-only until AUC clears 0.51.
+- Ledger records now include a `callExcerpt` (tool + path) for auditing off-task and intent-mismatch warns.
+
+## 0.28.4
+
+### Fixed
+
+- Extension no longer crashes on load when node:sqlite is unavailable (e.g. some Node v25 builds). Learning features disabled gracefully; all guards still work.
+
+## 0.28.3
+
+### Fixed
+
+- Temp output directories from saveOutput() are now cleaned up at session start (~2.5 MB/session leak).
+
+## 0.28.2
+
+### Fixed
+
+- One-time warning when confirm mode falls back to steer in headless sessions.
+
+## 0.28.1
+
+### Fixed
+
+- TypeSafe consent disclosure now mentions the SQLite hold database, its contents, and the retention policy.
+
+## 0.28.0
+
+### Added
+
+- \`learning.retentionDays\` config key (default 365) — prunes hold records older than this on startup. \`0\` disables pruning.
+
+### Fixed
+
+- SQLite hold database grew unboundedly; now pruned on startup with VACUUM.
+- Empty catch blocks in learning.ts now log warnings instead of silently swallowing errors.
+
+## 0.27.2
+
+### Fixed
+
+- Redact user prompts before storing in SQLite hold records (JSONL was already safe).
+
+## 0.27.1
+
+### Removed
+
+- Dead `SessionCompressionTracker` class — `getMultiplier()` and `record()` were never called outside tests. ~50 lines removed.
+
+## 0.27.0
+
+### Added
+
+- Sentence-style status bar templates (70+ across all 9 guards). `pickSentenceTemplate()` selects the best template from token context; zero LLM cost.
+- `widget.barMode: "live"` (default): status bar shows one sentence like *"warden allowed bash action — write to src/config.ts"* instead of data-style tokens.
+- `widget.barMode: "stack"`: the previous multi-guard data-style view.
+- Compact sidebar mode: trace entries default to 1–3 lines; press `d` to toggle detailed view.
+- Interactive config panel (`/warden config`): arrow-key navigation, Enter to toggle booleans or edit values, `s` to save.
+- `/warden config set <key.path> <value>` and `/warden config get <key.path>` for quick CLI edits.
+- `TraceEntry.tokens` field for live-mode re-rendering with sentence templates.
+- `formatVerdictTokens()` returns both rendered line and raw tokens.
+- `setNestedValue`, `getNestedValue`, `parseConfigValue` config helpers.
+
+### Changed
+
+- `widget.barMode` defaults to `"live"`.
+- Removed 7 exported-but-never-called functions and 3 non-functional blocks (proactive guidance, adaptive thresholds, intent prediction).
+
+### Fixed
+
+- `/warden config` now works before any guard activity fires (`lastUi` set on session start).
+
 ## 0.26.0
 
 ### Added

package/README.md

@@ -1,174 +1,59 @@
 # pi-warden
 
-Your `AGENTS.md` is a request. pi-warden is the check behind it.
+**Stop babysitting your coding agent.**
 
-On every write and edit, [Jev](https://typesafe.ai) judges the change against the rules in your `pi-warden.md` and quotes the broken rule back into the agent's own context. The same channel names slop, breaks retry loops, calls out "done" when no test ran, keeps large tool output small, triages async subagent reports, and holds the rare destructive command. You keep working. The agent gets a nudge instead of you getting a dialog.
+pi-warden supervises Pi while it works, catching risky actions, ignored rules, stuck loops, unverified "done" claims, security issues, runaway output, wasted context, and more.
 
-Measured: 150 paired headless runs broke a project rule **6 times with pi-warden off and 0 times with it on**. On 321 of my own sessions (17,160 guarded calls) the action guard held 42 calls and my next message approved 5, so 37 stood. A judgment costs about 1,000 input tokens (roughly $0.00004) and lands in about 0.3 s, which is why it can fire on every guarded call.
+Instead of interrupting you for every problem, Warden usually feeds the issue back to the agent so it can correct itself and keep going.
 
 ![Real verdicts from pi-warden: the same command gets a different verdict depending on what the user asked for](https://raw.githubusercontent.com/DevMortimer/pi-warden/main/docs/preview.png)
 
-The verdicts above are real output from `npm run test:live`. Independent project; not affiliated with TypeSafe AI or the Pi authors. Built on [pi-typesafe](https://github.com/DevMortimer/pi-typesafe).
-
-## What it watches
-
-| Guard | Watches | Does |
-| --- | --- | --- |
-| **Rules** | every `write` and `edit` | judges the written code against your project's Markdown rules (`pi-warden.md`) and names the violated rule to the agent |
-| **Slop** | written code and final replies | names stubs, restating comments, dead code, hedging, padded replies; the agent fixes them in the next edit |
-| **Stuck** | tool results | notices three failures with the same strategy and asks for a new hypothesis |
-| **Done-check** | the final message | catches "done" claims after code changes when no test, build, or lint passed |
-| **Context saver** | large or repeated tool output | keeps the exact lines that matter, stores the rest in a file, points recalls at a search |
-| **Action** | every `bash`, `write`, `edit` (and context-mode's `ctx_execute*`) before it runs | holds irreversible calls and tells the agent why, so it re-plans or asks you; steers it back when a change is unrelated to your request |
-| **Security** | written code and tool output | flags hardcoded secrets, disabled TLS, unsafe interpolation; marks injected instructions in tool output; fixture and documentation stand-ins are traced, never announced |
-| **Runaway** | the reply stream | stops a reply that repeats the same block over and over (code only, no request) |
-| **Steer budget** | the agent's attention | bounds the steers one run can demand: repeats and notices past the budget are recorded in the trace only, and a final reply that mostly restates an earlier reply of the same run is counted, never steered |
-| **Subagent triage** | async subagent reports | Jev decides whether a child report is worth waking the agent for; routine progress stays in context and costs no request ([detail](https://github.com/DevMortimer/pi-warden/blob/main/docs/guards.md#subagent-triage)) |
-| **Notifications** (opt-in) | moments that need you | desktop notification for a held call, a confirm dialog, a runaway stop |
-
-Three jobs, one channel: keep the agent on your rules, keep slop out of the code and the replies, keep the context small. The action guard is the seatbelt underneath: on 17,000 recorded calls it held 42 times. Everything arrives through the agent's own context, so the agent gets smarter and you are not the approval button. Full detail per guard, with thresholds and tuning numbers, is in [docs/guards.md](https://github.com/DevMortimer/pi-warden/blob/main/docs/guards.md).
-
-## In one minute
+## Install
 
 ```bash
 pi install npm:pi-warden
 ```
 
-Then, inside Pi:
-
-1. `/warden enable`. Read the data notice and confirm. If no key is stored yet, paste one from [console.typesafe.ai](https://console.typesafe.ai) (hidden input, saved owner-only, shared with pi-typesafe).
-2. `/warden test` shows one synthetic verdict and what the agent would be told.
-3. Work as usual. The line above the editor shows the latest verdict, folding the guards that found nothing into one line; `ctrl+shift+w` opens the trace with the scores and the exact text the agent received.
-
-Requires Pi 0.85 or newer and Node.js 22.19 or newer. Without a key, the offline parts still run: the pattern list, the runaway stop, duplicate-output notes, sensitive-path notes, and the credential-shape warnings. The [examples folder](examples/README.md) has a starter rules file and both config files.
-
-## Does it improve the code?
-
-**Short answer: the warden cell did not break a rule in 150 paired runs. The control cells did, in 6 of them.**
-
-`npm run eval:ab` runs the same fixture tasks through headless Pi twice. The control cell gets the rules as prose in `AGENTS.md`; the warden cell gets the same file plus pi-warden enforcing `pi-warden.md`. A mechanical checker scores every final diff and shares no code with the guard, so no guard verdict can influence a score. Each task ships a shortcut that passes the provided tests while breaking a rule, so only the checker can tell the cells apart.
-
-| Model | Runs per cell | Control runs with a rule violation | Warden runs with a rule violation | Tests fully passing (control / warden) |
-| --- | --- | --- | --- | --- |
-| glm-5.3-flash | 60 | **5** | **0** | 54 of 60 / 55 of 60 |
-| deepseek-v4.1-flash | 90 | **1** | **0** | 84 of 90 / 82 of 90 |
-
-The rules the traps break are the ones no linter can check and no model knows from training: an endpoint pasted inline instead of imported from the config module, a `TODO` without an issue, `console.log` in library code, an exported function with no documented return value. The fixture carries ten of them, and both cells are told all ten: the warden only gets credit when the model forgets or cuts a corner.
-
-The clipping task shows the shape of the effect. Without the warden, 6 of 10 glm runs cut user-visible text to fit. With it, 1 of 10, and every run that received a clip steer ended clean.
-
-Run it on your own model: `npm run eval:ab -- --repeats 3 --model <model>` (it asks you to pick a model on purpose, because a full batch spends real tokens). Per-batch reports, including the runs where nothing changed, live in [eval/reports/](eval/reports/README.md).
-
-The guards are also stable over repeated invocations. A continuous overnight run (2026-09-18) evaluated every guard across **109 cycles, 13,952 cases, 100% pass rate, zero score drift**. The same synthetic input produces the same judgment within rounding noise across all runs. Full numbers: [docs/overnight-eval.md](docs/overnight-eval.md) and [eval/reports/2026-09-18-overnight-stability/](eval/reports/2026-09-18-overnight-stability/).
-
-### Why a strong model does not make this pointless
-
-A guard like this is a smoke alarm. Most days it says nothing, and that is the design: it only pays off on the run where the model is tired, compacted, four hours in, or confident about a shortcut you would have caught in review. Four of its ten guards do not depend on the model being weak at all:
-
-- **Context saver**: large tool output is replaced by the lines that matter plus a file pointer. That is your token bill, on any model.
-- **Runaway stop** and **stuck** break a loop the model cannot see itself repeating, on any model.
-- **Done-check** reads the final message against the checks that actually ran, so "all tests pass" gets checked, not trusted.
-- **Subagent triage** scans async child reports so routine progress does not wake the parent agent or cost a request.
-
-The rules side is calibrated for the tail, not the average. Judge it on the runs where it fires, which is what the [trace panel](https://github.com/DevMortimer/pi-warden/blob/main/docs/guards.md) and `/warden status` hold precision are for.
-
-### What it does not do yet
-
-Published because the negative is as useful as the positive. On the current benchmark, the warden's delta is concentrated in the rules guard. Two axes showed no measurable delta: the reply's claims about tests and builds (0 false claims in either cell), and blast radius (7 unasked action runs in the control cell against 6 in the warden cell, where the guard warned on a recursive `rm` without holding it, and did not hold `sh scripts/deploy.sh` on a "deploy it" prompt). A 12-turn decay arc also found no rule-violation growth in either cell. Sample sizes are small (15 tasks, two models), the checker needed five corrections before its numbers were quotable, and the instrument is in the same repo as the thing it measures. Re-run it and quote the folder.
-
-## Two moments from real sessions
-
-Both are from my own work sessions, taken from the session logs. Project names are redacted, paths shortened, secrets replaced. The other two, a placeholder the agent talked itself out of removing and a retry loop broken mid-run, are in [docs/examples.md](https://github.com/DevMortimer/pi-warden/blob/main/docs/examples.md).
-
-### It quoted a project rule back, and the agent fixed it in the next turn
-
-The project's `pi-warden.md` rule "No hardcoded secrets" says connection URLs come from settings or the environment. The agent wrote a demo script with the local database DSN as a fallback default.
-
-```text
-assistant
-  [write] scripts/demo_rail_decisions.py   (186 lines)
-    DSN = os.getenv("SUPABASE_DB_URL", "postgresql://[local default]")
-
-pi-warden (steer message to the agent)
-  pi-warden: the content just written to scripts/demo_rail_decisions.py violates
-  project rule from pi-warden.md: "No hardcoded secrets" (0.88): Source and config
-  code must not contain passwords, API keys, tokens, or connection URLs (including
-  Supabase DSNs and `FLY_API_TOKEN` values). They come from settings
-  (`utils/config.py`) or the environ… [77 more chars]. Fix it in your next edit.
-
-assistant
-  The warden flags the hardcoded local DSN. Right call. Making the script require
-  `SUPABASE_DB_URL` from the environment instead:
-    DSN = os.getenv("SUPABASE_DB_URL")
-    if not DSN:
-        sys.exit("Set SUPABASE_DB_URL to the local stack's DB_URL (supabase status)")
-```
-
-The rule is three lines of Markdown in the repo. No linter knows what a Supabase DSN looks like in a default argument; Jev does, and the agent got the rule text, not a red X.
-
-### It held a database reset, and the agent finished everything else first
-
-A feature branch with a migration. The agent decided to replay the local database. `supabase db reset` is legitimate in that repo, but it wipes local state and nobody had asked for it.
-
-```text
-assistant
-  [bash] cd .claude/worktrees/con-142 && supabase db reset 2>&1 | tail -6
-
-tool result (written by pi-warden; the command did not run)
-  pi-warden held this bash call before it ran: irreversible 0.85. Do not retry it
-  unchanged. Either (1) reach the goal with a recoverable alternative that stays
-  inside the project (a targeted path, a dry run, a move instead of a delete, a
-  normal push), or (2) if this exact action is genuinely required, stop and tell
-  the user in one or two sentences what it does, what cannot be undone, and why
-  it is needed, then wait for their reply. [...the template continues]
-
-assistant
-  The warden held `supabase db reset`. It's the sanctioned local-stack replay
-  (loopback docker only, no dev/prod reach), but I'll pause that step and ask at
-  the end of this update. Continuing with lanes that don't need it:
-  [bash] uv run pytest tests/canon tests/services/messaging ...   92 passed in 24.80s
-
-assistant (final message, about 16 tool calls later)
-  Draft PR is up, work committed and pushed. One decision is pending for you.
-  PR: redacted#2640 (draft) ...
-```
-
-No dialog, no interruption. The agent moved the risky step to the end and asked in one sentence.
-
-## How it decides
+1. `/warden enable` to activate. Paste a key from [console.typesafe.ai](https://console.typesafe.ai) if you have one (hidden input, saved owner-only), or skip for offline-only mode.
+2. `/warden test` to see one synthetic verdict.
+3. Work as usual. Status line shows verdicts; `ctrl+shift+w` opens the trace sidebar.
 
-Every guarded call (`bash`, `write`, `edit`, and shell-like tools such as context-mode's `ctx_execute`) goes through four steps:
+Works without any key (offline guards: pattern list, runaway stop, sensitive-path notes, credential warnings). Requires Pi 0.85+, Node 22.19+.
 
-1. **Read-only? Skip.** `git status && ls` or a `read`: no request, no line in the trace.
-2. **Pattern list, offline.** Force push, `git reset --hard`, recursive `rm` outside the project, SQL `DROP`/`TRUNCATE`/`DELETE FROM`, `curl | sh`, publishing, infrastructure destroys: the call is held. Risky but recoverable things (`rm -rf` on a project path, `sudo`, `--no-verify`, `gh pr merge`, touching `.env`) warn. Text that is data, such as a commit message that mentions `git push --force`, is not a command.
-3. **Jev, with your consent.** One request carries your request, up to eight earlier messages, the agent's own words right before the call (its plan), and the call itself. Jev answers fixed questions with probabilities: irreversible, off task, does it change anything, how it relates to the task, does it contradict the plan, is the effect visible outside the working tree. On a write the slop and security questions ride along, and the rules check is a second request in parallel. About 0.3 s before the tool runs.
-4. **Act.** Three things hold a call: a destructive pattern, a `deny` command rule (blocked outright, no dialog), or `irreversible` at 0.7 or above. Everything else (off task, plan mismatch, slop, a rule violation, a security smell) tells the agent and lets the call run.
+## What Warden catches
 
-In the default `steer` mode a hold goes back to the agent as its tool result with two acceptable next moves: find a recoverable way, or explain the action to you in a sentence and wait. If your reply approves it, the retry goes through. `confirm` mode shows you a dialog instead; `advise` never holds.
-
-Three rules hold all of this together. Patterns set the floor and Jev can only raise it. The agent's plan can add a nudge but never remove a hold. Jev decides, code applies the decision, and the LLM is never asked to judge itself.
-
-**Why Jev and not a second LLM call.** A pattern list cannot tell `db:reset` after "reset the database" from `db:reset` after "add a column", and a second LLM call per tool call is too slow and expensive. Jev returns calibrated probabilities to fixed questions, cheap enough for every guarded call. The full argument, and how large a request gets, is in [docs/guards.md#why-jev](https://github.com/DevMortimer/pi-warden/blob/main/docs/guards.md#why-jev-and-not-a-second-llm-call).
-
-## What the numbers say
-
-`node scripts/calibrate-action.mjs --all` replays every guarded call in your recorded Pi sessions through the guard and asks Jev, once per turn, whether your next message approved a hold, regretted a call that ran, or rejected the turn. On 321 of my own sessions (17,160 guarded calls): **42 holds, and my next message approved 5**, so 37 stood; **20 regretted calls (2% of turns)**, all scope rather than data loss; and off-task caused 56 of the 139 replay holds without drawing a complaint, so it now steers and never holds.
+| Guard | Watches | Does |
+| --- | --- | --- |
+| **Action** | `bash`, `write`, `edit` before run | Holds irreversible calls, records trace-only off-task findings |
+| **Rules** | every `write` and `edit` | Judges against project Markdown rules, quotes the broken rule |
+| **Slop** | code and replies | Names stubs, restating comments, dead code, hedging |
+| **Stuck** | tool results | Repeated failures using the same strategy |
+| **Done-check** | final message | "Done" claims with no test/build/lint behind them |
+| **Security** | code and output | Hardcoded secrets, injection risks |
+| **Runaway** | reply stream | Stops replies that begin repeating themselves |
+| **Subagent triage** | async child reports | Keeps noisy background work from waking the parent |
 
-The labelling never stops: what you do after each hold (approve, decline, redirect, complain) becomes a label, and `/warden status` shows hold precision for your own sessions. The signal ranking, the intent threshold, the candidate questions that lost, and what 67 live steers did to the agent are in [docs/guards.md#calibration](https://github.com/DevMortimer/pi-warden/blob/main/docs/guards.md#calibration).
+## How intervention works
 
-## Questions people asked
+1. **Read-only? Skip.** `git status`, `ls`, `read` means no request and no trace entry.
+2. **Known-dangerous pattern? Catch it locally.** Force push, `git reset --hard`, recursive `rm`, SQL `DROP` held instantly.
+3. **Needs judgment?** Warden evaluates the action in context.
+4. **Steer or hold.** Most issues go back to the agent so it can correct itself. Irreversible actions can be stopped before they run.
 
-**How is this different from putting the rules in AGENTS.md?**
-An `AGENTS.md` is read once and asks the model to remember. Long sessions, compaction, and cheap models forget. pi-warden re-reads your rules file when it changes and judges every write against it at the moment it happens, then quotes the rule the agent broke. The rules can be things a linter cannot check: "exported functions declare return types", "a TODO must name a ticket", "comments must not restate the code". Keep the AGENTS.md; this is the check behind it.
+In the default `steer` mode, Warden talks to the agent rather than interrupting you. `confirm` asks you directly; `advise` never blocks.
 
-**Do I need a TypeSafe key?**
-Only for Jev judgments. Without one you still get the pattern list, the runaway stop, duplicate-output notes, sensitive-path notes, and credential-shape warnings. Keys come through a waitlist at [console.typesafe.ai](https://console.typesafe.ai); install now and run `/warden enable` when yours arrives.
+## More than guardrails
 
-The other answers (regex, Claude Code's auto mode, cost, permissions, what is sent to TypeSafe, project overrides, headless CI, and the size of a Jev request) are in [docs/faq.md](https://github.com/DevMortimer/pi-warden/blob/main/docs/faq.md).
+- **Learns from holds**, tracking outcomes and recommending policy changes via `/warden recommend`.
+- **Custom policies**: command, path, and arming rules for your own workflow.
+- **Context saver** trims oversized tool output while keeping the full result retrievable.
+- **Multiple judgment backends**: TypeSafe by default, with OpenRouter support.
+- **Full traceability**: inspect what Warden saw, decided, and told the agent.
+- **Desktop alerts** for events that need you.
 
-## Rules for your project
+## Custom rules
 
-Put `pi-warden.md` at the project root. Each heading is one rule; the text under it is the specification. An optional `paths:` line limits the rule to matching files.
+Put `pi-warden.md` at the project root. Each heading is one rule:
 
 ```markdown
 # No console statements
@@ -179,45 +64,44 @@ paths: src/**/*.ts
 Every exported function declares its return type.
 ```
 
-On every write and edit, Jev judges the change against each rule: compliant, violation, not applicable, or insufficient context. A violation at 0.7 or above is quoted back to the agent (heading, rule text, and which edit); the write goes through, because a held write would leave a half-written file. The third hit of one rule in a session is called out as a standing rule, and files are re-read when they change.
+Jev judges every write and quotes violations back. Rules can be things no linter checks: "a TODO must name a ticket", "comments must not restate the code". Without `pi-warden.md`, the first of `README.md`/`CLAUDE.md`/`AGENTS.md` is judged instead. [Examples and config details.](docs/configuration.md)
 
-Without a `pi-warden.md`, the first of `README.md`, `CLAUDE.md`, `AGENTS.md` is judged as one document instead, so existing context files work from day one. Starter file with a dozen rules: [`examples/pi-warden.md`](examples/pi-warden.md). Path globs, sensitive-path notes, and the tuning set: [docs/guards.md#rules](https://github.com/DevMortimer/pi-warden/blob/main/docs/guards.md#rules).
+## Does it actually help?
 
-## Commands
+In 150 paired agent runs, the control setup violated the tested project rule **6 times**. With Warden: **0**.
 
-Seven commands: `/warden status`, `enable`, `disable`, `mode steer|confirm|advise`, `config`, `test`, `trace`. `status` reports guard state, session counts, steer counts per guard, thresholds, hold precision, and config paths; `trace` toggles the sidebar with the exact text the agent received. The full table is in [docs/commands.md](https://github.com/DevMortimer/pi-warden/blob/main/docs/commands.md).
+An overnight stability run covered **13,952 guard cases across 109 cycles** with no score drift.
 
-## Configuration
+These are project-maintained benchmarks, not universal claims. Raw reports and reproducible eval tooling are in the repo.
 
-Two optional files: your defaults in `~/.pi/agent/pi-warden/config.json` (`/warden config` opens it) and per-repo overrides in `.pi/pi-warden.json`. Common tweaks are `mode`, `steerVisible`, `notices`, `notify.enabled`, `slop.prose.audience`, `rules.skip`, and `context.enabled`. Every key with its default, what a project file may and may not change, and the environment variables: [docs/configuration.md](https://github.com/DevMortimer/pi-warden/blob/main/docs/configuration.md). Worked examples of all three files: [examples/README.md](examples/README.md).
+### Does it get in the way? Measured, not promised.
 
-## Status line and trace
+**3 holds per 1,000 calls. The other 997 run.** Across 315 recorded sessions and 18,075 guarded calls, the action hold fired 48 times, each for an action that is hard to undo. On the maintainer's machine, 452 allowed calls have run since outcome tracking began and not one was regretted afterward. When a hold was wrong, one retry cleared it: 2 of 13 labeled holds stood, the other 11 cleared on retry. The 27 calls the maintainer later regretted in the replay were ordinary edits and commits, not the kind a pre-call hold can see coming; those belong to the rules guard, the done-check, and the regret label, which are measured too. Snapshot numbers: `scripts/hold-stats.mjs` refreshes them, and the full replay is in [`eval/reports/2026-09-21-calibration-0.33.3/`](eval/reports/2026-09-21-calibration-0.33.3/).
 
-The line above the editor shows the latest verdict per guard, the verdict leading as a chip: `WARN action write · irreversible 0.09 · off-task 0.95 · unrelated · off task`. The guards that found nothing fold into one line per verdict, so a quiet turn costs one line. `/warden trace`, `ctrl+shift+w`, or a click on the line opens a right-hand sidebar with the full trace, newest first. Clicks need Pi's fullscreen mode (`tuiMode: "fullscreen"` in `/settings`). Widget templates and placement: [docs/configuration.md#status-line-and-trace-sidebar](https://github.com/DevMortimer/pi-warden/blob/main/docs/configuration.md#status-line-and-trace-sidebar).
+[See the evals ->](eval/reports/)
 
-## For extension authors
+## Privacy
 
-Every guard is a plain function you can call with any object that has pi-typesafe's `evaluate` method as the judge, and the library has no dependency on Pi's runtime, so it is safe in tests. Calls go through pi-typesafe's `ask`, so timeouts, aborts, and error shapes are pi-typesafe's contract:
+Depending on the configured judgment backend, Warden sends to your provider: a redacted summary of each guarded call, the agent's plan, up to eight redacted prior messages, and redacted tool-output samples. Secrets are stripped before anything leaves the machine. [Full details.](docs/data-handling.md)
 
-```ts
-import { evaluateAction, defaultConfig } from "pi-warden";
-import { createTypeSafe } from "pi-typesafe";
+## Documentation
 
-const verdict = await evaluateAction(
-  { tool: "bash", input: { command: "git push --force" }, cwd: process.cwd(), task: "push my branch" },
-  { config: defaultConfig().action, judge: createTypeSafe() },   // omit judge for pattern checks only
-);
-```
-
-The rules guard, the session-level `ActionGuard` and `RulesGuard`, and the other exports are in [docs/extension-authors.md](https://github.com/DevMortimer/pi-warden/blob/main/docs/extension-authors.md).
+- [docs/guards.md](docs/guards.md): every guard, thresholds, calibration, Jev questions
+- [docs/configuration.md](docs/configuration.md): all config keys, defaults, status line templates
+- [docs/commands.md](docs/commands.md): command reference
+- [docs/faq.md](docs/faq.md): cost, permissions, headless CI, project overrides
+- [docs/data-handling.md](docs/data-handling.md): what leaves the machine, what stays
+- [docs/examples.md](docs/examples.md): real session examples
+- [examples/](examples/): starter rules file, both config files
 
 ## Development
 
-`npm install`, then `npm run check` (typecheck, 185 offline tests on a mocked transport, build). `npm run test:live` spends real judgments across the guards; `node scripts/calibrate-action.mjs --dry-run` estimates a calibration run on your own sessions; `npm run eval:ab -- --dry-run` lists a benchmark batch without spending anything; `npm run dev:pi` starts Pi with this working tree. Release history, including the measured numbers behind each change: [CHANGELOG.md](CHANGELOG.md). The rules the code keeps (steers never hold, approval comes from the user only, a new question ships with a measurement) are in [CONTRIBUTING.md](CONTRIBUTING.md).
-
-## Credits
-
-Rules as headings and the four outcomes come from [jevrealtimecodecheck](https://github.com/MrDesjardins/jevrealtimecodecheck); path globs, sensitive paths, and "judge the change, not its story" from [wince](https://github.com/TinyFrontier/wince).
+```bash
+npm install
+npm run check          # typecheck + offline tests + build
+npm run test:live      # live tests (costs real judgments)
+npm run eval:ab        # A/B benchmark (costs real tokens)
+```
 
 ## License
 

package/dist/audit.d.ts

@@ -0,0 +1,22 @@
+export { discoverSourceFiles, findProjects } from "./discover.js";
+/**
+ * Build the user message that sends the workspace to the session model for an
+ * agent-driven audit.  The model reads source, finds Jev opportunities, produces
+ * measurable evidence, and writes an HTML report.
+ *
+ * `projects` comes from {@link findProjects}.  `cwd` is the workspace root.
+ */
+export declare function buildAuditPrompt(cwd: string, projects: string[]): string;
+export interface ReportSnapshot {
+    exists: boolean;
+    mtimeMs: number;
+}
+/**
+ * Take a point-in-time snapshot of the audit report file.  Call before
+ * sending the prompt and again after the agent finishes.
+ */
+export declare function snapshotReport(reportPath: string): ReportSnapshot;
+/**
+ * Determine whether the audit produced a new or updated report.
+ */
+export declare function reportOutcome(before: ReportSnapshot, after: ReportSnapshot): "written" | "stale" | "missing";