pi-warden 0.64.1 → 0.71.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -8,6 +8,101 @@ How to keep this current: add the entry in the same pull request as the change,
8
8
 
9
9
  <!-- Empty. Next release starts here. -->
10
10
 
11
+ ## 0.71.0
12
+
13
+ ### Added
14
+
15
+ - A project rule may set its own cutoff and severity in header lines at the top of its body, beside `paths:`: `threshold: 0.8` (a number from 0 to 1) and `severity: high|normal|low` (default `normal`). A rule with a `threshold:` fires at its own cutoff instead of `rules.threshold`; severity only orders findings (high, normal, low, then score), in the steer and in the trace. A bad value is ignored and `/warden rules` reports the line.
16
+ - `rules.softThreshold` (default `0`, off): a score from this value up to a rule's cutoff becomes a soft "please double-check" finding. Soft findings never hold, get one short separate sentence in the same steer (`Also check whether "<rule>" applies here (0.62).`), count against the steer budget like any other steer, and are recorded in the rules log with `soft: true`. When only soft findings exist the steer is that sentence alone.
17
+
18
+ ### Tests
19
+
20
+ - Rules bench (`eval/rules-bench/`): 161 labelled cases (107 tune, 54 holdout) and eight new cases: six add a new violation of a rule the file already breaks and two touch only a line next to an old violation. Three fixture rules carry header lines (`severity:` on two, `threshold: 0.9` on the boolean-name rule). The scorer reads those headers and reports each rule at its own cutoff and the soft tier under it. The run is in `eval/reports/2026-09-27-rules-tiers/`, with case `r15-04` recorded as a still-open miss.
21
+
22
+ ### Docs
23
+
24
+ - `docs/guards.md` → Calibration records the rules-tier measurement, and `docs/configuration.md` documents `rules.softThreshold`.
25
+
26
+ ## 0.70.2
27
+
28
+ ### Fixed
29
+
30
+ - Two store writes in one process no longer race. Every write to a loops, prefs, user-config, steer-stats, or rules-log file now goes to a temporary file unique to the process and the write before the rename, and the read-modify-write changes to the loops and prefs files run one after another, so a `warden_loops` or `warden_remember` call can no longer fail with `ENOENT` or silently lose the other call's change. A second Pi process writing the same file is still not guarded.
31
+
32
+ ## 0.70.1
33
+
34
+ ### Tests
35
+
36
+ - Rules bench (`eval/rules-bench/`, `npm run eval:rules`): 17 fixture rules and 153 labelled cases (four violations, three compliant near-misses and two cases where the rule does not apply for each rule), split into a fixed `tune` set (102) and a fixed `holdout` set (51), one request per case. `--split` selects the set, `--budget` caps requests, `--concurrency` sets parallel requests, `--dry-run` lists the plan and sends nothing, and `--rescore DIR` re-scores saved answers without spending. The scorer reports recall, false-alarm rate, precision and counts at cutoffs 0.3 to 0.9, AUC per rule, the rules whose clean cases sit in the 0.3 to 0.5 band, and the two-tier view (raise at 0.7 against a 0.5 to 0.7 double-check). Cases where a path-scoped rule is not asked at all are counted apart from the metrics. The first run is in `eval/reports/2026-09-26-rules-bench/`. The guard is unchanged.
37
+
38
+ ## 0.70.0
39
+
40
+ ### Added
41
+
42
+ - `/warden rules check` asks Jev which of the active rules the rules guard cannot judge well, and prints one line per rule that needs attention with the reason, its score, and one suggestion (`move it to your linter`, `split it so the changed file alone shows the violation`), then `N fine, M need attention`. Two questions per rule: whether it can be judged from one changed file's content alone (`from_change_alone`, `needs_other_files`, `needs_task_or_history`, `too_vague`) and whether a standard linter, formatter, or type checker could enforce it exactly (cutoff 0.70, measured in `docs/guards.md` → Calibration). Rule names, text, and `paths:` scopes are sent redacted, with no file content and no task text; with no key or no consent it says so and sends nothing, and plain `/warden rules` stays local. Advice only: no rule is changed, disabled, or skipped, and the guard's questions, thresholds, and defaults are untouched.
43
+
44
+ ## 0.69.0
45
+
46
+ ### Added
47
+
48
+ - `/warden report [--days N]` reads a local rules verdict log and reports, per rule, how often it was judged, fired, was cleared after a fire, and its mean violation score, with a flag for `never fires`, `fires on everything`, or `undecided`, then the rules in the current set with no records. Default 30 days; local only, nothing is sent. Each judgment from the rules guard is now recorded (time, session id, project-relative path, tool, rule id and name, outcome, P(violation), the threshold, whether it was a finding, and whether it cleared an earlier finding), keyed by a hash of the project path under `~/.pi/agent/pi-warden/rules/`; the file keeps the newest 5,000 records and a write failure is silent to the agent and shown once in the trace.
49
+
50
+ ### Docs
51
+
52
+ - README rewritten shorter: badges, one headline claim, a table of what the agent does and what pi-warden does, and the measured numbers in one list.
53
+
54
+ ## 0.68.0
55
+
56
+ ### Added
57
+
58
+ - `/warden rules` prints the active parsed rule ids, path scopes and source files, plus the dropped count and `rules.exclude` patterns, so the local rule set can be inspected without sending anything to Jev. With `rules.enabled: false` the output leads with `Rules guard is off (rules.enabled: false). These would apply:` and the same details follow. A fallback document judged as one aggregate rule reports its condensed size only.
59
+
60
+ ## 0.67.0
61
+
62
+ ### Added
63
+
64
+ - **Stuck evidence.** The stuck guard now sends the judge a compact structured `evidence` section next to today's fields. Per run: the failing test, error, location, summary and exit code, parsed generically from the run's own output (TAP/node test, jest, vitest, pytest, tsc, eslint, cargo, go, a `make test` script, Playwright), the run number of the earlier run that failed the same way after durations, clock times, temp paths, line:column positions and ordering are normalised, and a 300-character head plus 300-character tail fallback when no parser knows the output. Per `edit`/`write`: the path and a diff of the change capped at 600 characters. A digest gives the failed runs, the distinct failures, the runs that repeat an earlier command, the edits between the first and last failed run, and the information-gathering calls after the first failure. Every string is redacted, the whole object is capped at 4 KB, and the oldest runs are dropped first. The measurement behind it is in `eval/reports/2026-09-26-stuck-evidence/`: on the same 40 stuck bench cases, accuracy 0.725 → 1.000, right where the old state was wrong on 11 cases and wrong where it was right on none (sign test p = 0.001), no latency change at p50. Edit diffs (capped, redacted) now leave the machine for stuck checks; `docs/data-handling.md` lists them.
65
+ - Config `stuck.evidence` (default `true`, valid in the user and the project file). `false` sends the output tails only, as before. The guard's questions, thresholds and when it fires are unchanged.
66
+
67
+ ### Tests
68
+
69
+ - Judge bench: the stuck arms are now the shipped guard itself. Arm A is `buildStuckRequest` with `stuck.evidence` on (the default), arm C is the same builder with it off, and arm B keeps A and enlarges the raw slices, so a run compares the shipped state against its predecessor on the same cases. The generic parser and the failure signature moved out of `eval/judge-bench/` into `src/evidence.ts`, so the bench exercises the shipped code; the bench's own parser module is gone.
70
+ - New tests: the request without evidence equals the state sent before the change, one sample of each runner format names its failing test, an unknown output falls back to head and tail, noise-only differences (line numbers, times, temp paths, ordering) are one failure while a changed failure is two, a credential in an output or in a diff never reaches the state, the 4 KB cap holds and drops the oldest runs first then the oldest edits, and `stuck.evidence` is read from the user file and the project file.
71
+
72
+ ## 0.66.1
73
+
74
+ ### Tests
75
+
76
+ - Judge bench (`eval/judge-bench/`, `npm run eval:judge`): 80 labelled cases for the stuck and done guards (40 each, half positive), each sent to the real judge in three request states: A, the state the guards build today; B, the same shape with larger raw output slices; C, today's fields plus a compact `evidence` object from a generic failure parser (failing tests, errors, locations, edit diffs, file types, checks after the last edit). 36 traces come from real runs of node test, tsc, cargo, go, sbcl and scripts in throwaway projects; the rest are written in the runners' own formats. Scoring uses the guards' own thresholds and gate. `--dry-run` lists the plan and sends nothing; `--budget` caps requests. Guard behaviour does not change.
77
+ - First judge bench run (`eval/reports/2026-09-26-judge-bench/`, 720 requests, 3 repeats): on stuck, the structured state is right where today's state is wrong on 11 cases and never the reverse (sign test p = 0.001; accuracy 0.725 → 0.992), mostly progressing windows that today's state calls stuck; its digest is close to the label rule, so this is an upper bound. On done, no arm beats today's state beyond noise, because the five not-done cases with a passing check after the last edit never reach the judge. Larger raw slices change one verdict. Latency p50 is 245–257 ms in every arm.
78
+
79
+ ## 0.66.0
80
+
81
+ ### Added
82
+
83
+ - Call-waste notes. Each tool call re-reads the whole conversation, so four patterns that spend calls for nothing now earn one advisory sentence each, attached to the tool result that triggers them: a second `sleep` poll inside 10 calls, a third ranged read of one file whose ranges are adjacent or overlapping, a third search of one file with the same or an overlapping pattern, and a check re-run with a different output filter after an earlier run was cut by a pipe and showed no failure. The note is text added to a result the model is about to read, so it costs no request; it never holds, blocks, or warns, never spends `steerBudget`, and is not adapted by `steers`. One note per detector per 20 calls. On the first run of a session one tip is appended to the system prompt when `waste.tip` is on: read files in large ranges or whole, run a check once without a pipe and search its output, and wait with one blocking command instead of repeated sleeps. New config section `waste`: `enabled` (default `true`), `tip` (default `false`, opt-in), `every` (20), and one switch per detector (`sleep`, `paging`, `search`, `recheck`, all default `true`). Every note is recorded in the trace as the `waste` guard with its detector. Thresholds come from an offline measurement over the tool calls of 1175 existing sessions; the weak-model bench that ran the guard on and off is under `eval/reports/2026-09-26-waste-nudges/`.
84
+
85
+ ### Changed
86
+
87
+ - The session tip is opt-in: `waste.tip` defaults to `false`. The five-repeat A/B on the weak suite (`eval/reports/2026-09-26-waste-tip-5x/`, 80 runs) put the tip's arm 0.5% apart on tokens and 0.6% apart on turns from the control, with a paired sign test over the task medians at p = 1.0 for tokens and for turns, with and without the long task, so the prompt tax is not paid by default. The four call-waste notes stay on by default and are unaffected; `waste.enabled` still gates both.
88
+
89
+ ### Tests
90
+
91
+ - The same two cells re-run at five repeats on the weak suite (`eval/reports/2026-09-26-waste-tip-5x/`, 80 runs) do not confirm the two-repeat numbers: tokens −0.5%, turns +0.6%, tool calls −10%, no detector note fired in either cell, tip delivered in all 40 waste-on runs, success 33 against 30 of 40 and harm 2 against 4. A paired sign test over the task medians gives p = 1.00 for tokens and for turns, with and without the long task, so the −43% tokens and −26% turns of the batch above were small-sample spread rather than a guard effect. One waste-on run was killed at the runner's 12-minute timeout; dropping it does not change the verdict.
92
+
93
+ ## 0.65.1
94
+
95
+ ### Fixed
96
+
97
+ - The rules guard no longer blames an edit for code that the edit kept. Each edit's request now carries `after`, the lines of `before` with the edit applied, and each rule question asks whether the change introduces a violation, judged on `after`, instead of judging `newText` alone with `before` as context only. In the weak-model bench, three of the four rules steers that were noise flagged a missing `@returns` for an edit to a function body or an added import, with the JSDoc still directly above the function. An edit that drops a required `@returns` is still flagged.
98
+ - A rules steer names the rule and the written file only; it no longer names the rules file (`from pi-warden.md`). After a false rules steer in the bench, one weak model spent 6 calls reading pi-warden's own config and hold log.
99
+
100
+ ## 0.65.0
101
+
102
+ ### Added
103
+
104
+ - Adaptive steers per model. pi-warden counts, per model and per steer kind (`intent-mismatch`, `off-task`, `rules`, `slop`, `stuck`, `conscience`, and the other advisory notes), the steers it sent, the ones the agent followed (a course change in its next two messages), and the ones it disputed ("as I said", "false positive"). After 30 or more steers, a kind followed under 20% of the time or disputed over 40% becomes trace-only for that model; 1 in 5 is still sent, and every further 30 steers the probes decide whether it is sent again. Holds, confirm, deny, security masking, the credential notice on masked output, and the done-check are never trace-only, and per-call judgments never change. The counts stay in `steer-stats.json` in pi-warden's data folder. `/warden status` lists the trace-only kinds per model with their rates; `/warden unmute <kind> [model]` resets a pair. New config section `steers`: `adaptive` (default `true`), `minSteers` (30), `minFollowed` (0.2), `maxDisputed` (0.4), `recheckEvery` (30), `probeEvery` (5). A replay of 937 recorded sessions would make `intent-mismatch` trace-only for three models (6 to 11% followed) and nothing else.
105
+
11
106
  ## 0.64.1
12
107
 
13
108
  ### Fixed
package/README.md CHANGED
@@ -1,16 +1,28 @@
1
- # pi-warden
1
+ <div align="center">
2
2
 
3
- **Stop babysitting your coding agent.**
3
+ # pi-warden
4
4
 
5
- pi-warden supervises Pi while it works, catching risky actions, ignored rules, stuck loops, unverified "done" claims, security issues, runaway output, wasted context, and more.
5
+ **Your coding agent says "done". It didn't run the tests.**<br>
6
+ **pi-warden catches it, and the agent fixes it without you.**
6
7
 
7
- Instead of interrupting you for every problem, Warden usually feeds the issue back to the agent so it can correct itself and keep going.
8
+ [![npm](https://img.shields.io/npm/v/pi-warden?color=cb3837&logo=npm)](https://www.npmjs.com/package/pi-warden)
9
+ [![downloads](https://img.shields.io/npm/dm/pi-warden?color=blue)](https://www.npmjs.com/package/pi-warden)
10
+ [![stars](https://img.shields.io/github/stars/DevMortimer/pi-warden?style=flat&logo=github)](https://github.com/DevMortimer/pi-warden/stargazers)
11
+ [![license](https://img.shields.io/badge/license-MIT-green)](LICENSE)
12
+ [![Pi](https://img.shields.io/badge/Pi-0.85%2B-8a2be2)](https://github.com/earendil-works/pi)
13
+ [![judged by Jev](https://img.shields.io/badge/judged%20by-Jev-orange)](https://typesafe.ai)
8
14
 
9
- **In nine days of real use: 743 sessions, 193 risky actions held before they ran, 1,292 notes sent to the agent. After a "done" with no test behind it, the agent ran one 78% of the time.**
15
+ </div>
10
16
 
11
17
  ![pi-warden tells the agent what it got wrong and the agent fixes it: 65 untested "done" claims, 51 then ran the tests, 5 found a failure it had missed; after a hold the agent found a safer way 33 times](https://raw.githubusercontent.com/DevMortimer/pi-warden/main/docs/hero.png)
12
18
 
13
- <sub>Numbers from the first nine days of real use (2026-09-16 to 2026-09-24, 743 sessions). The quoted lines are verbatim. Counted with `scripts/field-usage.mjs`; the [field report](eval/reports/2026-09-24-field-usage/) has the method and what was noise.</sub>
19
+ <div align="center">
20
+
21
+ **759 sessions · 220 risky actions stopped before they ran · 76% of fake "done"s turned into real test runs**
22
+
23
+ <sub>Nine days of the maintainer's real use across all their projects, not just this one, 2026-09-16 to 2026-09-24. Method and noise: [field report](eval/reports/2026-09-24-field-usage/).</sub>
24
+
25
+ </div>
14
26
 
15
27
  ## Install
16
28
 
@@ -18,115 +30,59 @@ Instead of interrupting you for every problem, Warden usually feeds the issue ba
18
30
  pi install npm:pi-warden
19
31
  ```
20
32
 
21
- 1. `/warden enable` to turn judgments on. Paste a key from [console.typesafe.ai](https://console.typesafe.ai) (hidden input, saved owner-only), or skip for offline-only mode.
22
- 2. `/warden init` to write a starter `pi-warden.md`. The rules guard enforces what you put there.
23
- 3. `/warden index` (recommended) so the conscience knows your skills and tools from their real descriptions.
24
- 4. `/warden test` to see one synthetic verdict, then work as usual. The status line shows verdicts; `ctrl+shift+w` opens the trace sidebar.
25
-
26
- The conscience is beta and off by default. Turn it on with `conscience.enabled: true` in the user config.
27
-
28
- Works without any key (offline guards: pattern list, runaway stop, sensitive-path notes, credential warnings). Requires Pi 0.85+, Node 22.19+.
29
-
30
- ## What Warden catches
31
-
32
- | Guard | Watches | Does |
33
- | --- | --- | --- |
34
- | **Action** | `bash`, `write`, `edit` before run | Holds irreversible calls, records trace-only off-task findings |
35
- | **Rules** | every `write` and `edit` | Judges against project Markdown rules, quotes the broken rule |
36
- | **Slop** | code and replies | Names stubs, restating comments, dead code, hedging |
37
- | **Stuck** | tool results | Repeated failures using the same strategy |
38
- | **Done-check** | final message | "Done" claims with no test/build/lint behind them, or UI changes nobody looked at |
39
- | **Security** | code and output | Hardcoded secrets, injection risks |
40
- | **Runaway** | reply stream | Stops replies that begin repeating themselves |
41
- | **Subagent triage** | async child reports | Keeps noisy background work from waking the parent |
42
- | **Conscience** (beta, off) | each operator prompt | Recommends a skill or tool from the capability index before the agent acts; enable with `conscience.enabled: true` |
43
-
44
- ## How intervention works
45
-
46
- ![Real verdicts from pi-warden: the same command gets a different verdict depending on what the user asked for](https://raw.githubusercontent.com/DevMortimer/pi-warden/main/docs/preview.png)
33
+ Then `/warden enable` (paste a [TypeSafe](https://console.typesafe.ai) key) and `/warden init` (writes a starter `pi-warden.md`). That's it. No key? The offline guards still run.
47
34
 
48
- 1. **Read-only? Skip.** `git status`, `ls`, `read` means no request and no trace entry.
49
- 2. **Known-dangerous pattern? Catch it locally.** Force push, `git reset --hard`, recursive `rm`, SQL `DROP` held instantly. An agent deleting temp-directory scratch it created in the same session is not held (macOS and Windows).
50
- 3. **Needs judgment?** Warden evaluates the action in context.
51
- 4. **Steer or hold.** Most issues go back to the agent so it can correct itself. Irreversible actions can be stopped before they run.
35
+ ## It steers. It doesn't nag.
52
36
 
53
- In the default `steer` mode, Warden talks to the agent rather than interrupting you. `confirm` asks you directly; `advise` never blocks.
37
+ Most guardrails stop and ask you. pi-warden tells **the agent** what it got wrong, and the agent corrects itself. You are pulled in only when something can't be undone: **3 holds per 1,000 calls. The other 997 just run.**
54
38
 
55
- ## More than guardrails
39
+ | Your agent… | pi-warden… |
40
+ | --- | --- |
41
+ | says "done" with no test, build, or lint behind it | sends it back to prove it |
42
+ | breaks a rule in your `pi-warden.md` or `AGENTS.md` | quotes the exact rule it broke |
43
+ | is about to `git push --force`, `reset --hard`, `rm -rf`, `DROP` | holds it before it runs |
44
+ | retries the same failing fix for the third time | asks for a new hypothesis |
45
+ | writes stubs, restating comments, hardcoded secrets | names them on the spot |
46
+ | floods its context with a 40k-line log | keeps the lines that matter, stores the rest |
47
+ | starts repeating itself forever | stops the reply |
56
48
 
57
- - **Learns from holds**, tracking outcomes and recommending policy changes via `/warden recommend`.
58
- - **Custom policies**: command, path, and arming rules for your own workflow.
59
- - **Context saver** trims oversized tool output while keeping the full result retrievable.
60
- - **Multiple judgment backends**: TypeSafe by default, with OpenRouter support.
61
- - **Honest about being off**: when judgments cannot run (no consent, no key, a rejected key, budget spent) Warden says so once, with the fix. A failing backend is paused after repeated errors instead of costing a timeout per action.
62
- - **Full traceability**: inspect what Warden saw, decided, and told the agent.
63
- - **Desktop alerts** for events that need you.
49
+ [Every guard, with its thresholds and calibration →](docs/guards.md)
64
50
 
65
- ## Custom rules
66
-
67
- Put `pi-warden.md` at the project root. Each heading is one rule:
51
+ ## Rules no linter can check
68
52
 
69
53
  ```markdown
70
- # No console statements
71
- Code must not contain `console.log` or `console.debug`. Use the logger.
54
+ # A TODO names a ticket
55
+ A bare `TODO` or `FIXME` without a ticket reference is a violation.
72
56
 
73
- # Exported functions must have explicit return types
74
- paths: src/**/*.ts
75
- Every exported function declares its return type.
57
+ # Errors never reach the user raw
58
+ paths: src/api/**/*.ts
59
+ Catch errors at the handler and return a message a user can act on.
76
60
  ```
77
61
 
78
- Jev judges every write and quotes violations back. Rules can be things no linter checks: "a TODO must name a ticket", "comments must not restate the code". Without `pi-warden.md`, the first of `AGENTS.md`/`CLAUDE.md`/`README.md` is judged instead. [Examples and config details.](docs/configuration.md)
79
-
80
- ## Does it actually help?
81
-
82
- ### In daily use
83
-
84
- Four days of the maintainer's own work, 397 sessions ([report](eval/reports/2026-09-24-field-usage/), [raw numbers](eval/reports/2026-09-24-field-usage/usage.json), `scripts/field-usage.mjs` to run it on your own logs):
85
-
86
- - **Done-check:** 47 nudges; in 35 the agent then ran a test or build.
87
- - **Holds:** 143 of 19,695 actions. Of the 44 with a recorded outcome, the agent took a safer route 33 times.
88
- - **Rules:** 195 corrections naming a project rule, including a process wait with no timeout and a home path in a public test fixture.
89
- - **Context:** about 123,000 tokens of oversized output trimmed in the traced sessions, and no agent needed the full output back.
90
-
91
- The report also lists what was noise that week (scratch deletes, credential notices on source code, two over-broad rules) and what changed because of it.
62
+ Each `#` heading is one rule. Every write and edit is judged against it in about a quarter of a second by [Jev](https://typesafe.ai), a model that returns a probability, not prose. No `pi-warden.md`? Your `AGENTS.md`, `CLAUDE.md`, or `README.md` is used instead.
92
63
 
93
- ### In benchmarks
64
+ ## Receipts
94
65
 
95
- In 150 paired agent runs, the control setup violated the tested project rule **6 times**. With Warden: **0**.
66
+ - **Rules:** in 150 paired agent runs, the agent without pi-warden broke the tested rule **6 times**. With it: **0**.
67
+ - **Done-check:** after a nudge, the agent ran a check **57 of 75** times, and sometimes found a failure it had missed.
68
+ - **Holds:** when the agent was stopped, it found a safer way **40 of 65** times; you approved 24.
69
+ - **Stability:** 13,952 guard cases over 109 overnight cycles, no score drift.
96
70
 
97
- An overnight stability run covered **13,952 guard cases across 109 cycles** with no score drift.
98
-
99
- These are project-maintained benchmarks, not universal claims. Raw reports and reproducible eval tooling are in the repo.
100
-
101
- ### Does it get in the way? Measured, not promised.
102
-
103
- **3 holds per 1,000 calls. The other 997 run.** Across 315 recorded sessions and 18,075 guarded calls, the action hold fired 48 times, each for an action that is hard to undo. On the maintainer's machine, 452 allowed calls have run since outcome tracking began and not one was regretted afterward. When a hold was wrong, one retry cleared it: 2 of 13 labeled holds stood, the other 11 cleared on retry. The 27 calls the maintainer later regretted in the replay were ordinary edits and commits, not the kind a pre-call hold can see coming; those belong to the rules guard, the done-check, and the regret label, which are measured too. Snapshot numbers: `scripts/hold-stats.mjs` refreshes them, and the full replay is in [`eval/reports/2026-09-21-calibration-0.33.3/`](eval/reports/2026-09-21-calibration-0.33.3/).
104
-
105
- [See the evals ->](eval/reports/)
71
+ Every number has a script and a raw report in [`eval/reports/`](eval/reports/). They are the maintainer's measurements, not a universal promise, and the reports list what was noise.
106
72
 
107
73
  ## Privacy
108
74
 
109
- Depending on the configured judgment backend, Warden sends to your provider: a redacted summary of each guarded call, the agent's plan, up to eight redacted prior messages, the task spine (your first request in the thread and up to four earlier ones, capped at 1,200 characters), and redacted tool-output samples. Secrets are stripped before anything leaves the machine. [Full details.](docs/data-handling.md) Security work — pentesting, incident response, CTF — has its own configuration recipe for keeping samples local and for the friction that credentials and lab setups cause: [Recipe: security work](docs/configuration.md#recipe-security-work).
75
+ Secrets and unshown paths are stripped before anything leaves your machine. [Exactly what is sent →](docs/data-handling.md)
110
76
 
111
- ## Documentation
77
+ ## Docs
112
78
 
113
- - [docs/guards.md](docs/guards.md): every guard, thresholds, calibration, Jev questions
114
- - [docs/configuration.md](docs/configuration.md): all config keys, defaults, status line templates
115
- - [docs/commands.md](docs/commands.md): command reference
116
- - [docs/faq.md](docs/faq.md): cost, permissions, headless CI, project overrides
117
- - [docs/data-handling.md](docs/data-handling.md): what leaves the machine, what stays
118
- - [docs/examples.md](docs/examples.md): real session examples
119
- - [examples/](examples/): starter rules file, both config files
79
+ [Guards](docs/guards.md) · [Configuration](docs/configuration.md) · [Commands](docs/commands.md) · [FAQ](docs/faq.md) · [Data handling](docs/data-handling.md) · [Examples](docs/examples.md)
120
80
 
121
81
  ## Development
122
82
 
123
83
  ```bash
124
84
  npm install
125
- npm run check # typecheck + offline tests + build
126
- npm run test:live # live tests (costs real judgments)
127
- npm run eval:ab # A/B benchmark (costs real tokens)
85
+ npm run check # typecheck + offline tests + build
128
86
  ```
129
87
 
130
- ## License
131
-
132
- MIT
88
+ MIT licensed.
@@ -0,0 +1,132 @@
1
+ import type { SteersConfig } from "./config.js";
2
+ /**
3
+ * Per-model steer adaptation. A steer kind that a model follows rarely or disputes often becomes trace-only for that
4
+ * model: on recorded sessions intent-mismatch notices were followed 8% of the time, the done-check 77%, and strong
5
+ * models dispute noise ("as I said …"). Holds and per-call judgments never change; only whether a note is sent.
6
+ */
7
+ /** One kind of note to the agent. Finer than the steer guard: the `action` guard sends several unrelated notes. */
8
+ export type SteerKind = "intent-mismatch" | "off-task" | "should-proceed" | "warn-headless" | "large-output" | "slop" | "rules" | "sensitive-path" | "security-write" | "stuck" | "repeat" | "done" | "prose" | "conscience" | "loops" | "runaway" | "subagent" | "hold" | "confirm" | "deny" | "masking" | "credential-notice";
9
+ export declare const STEER_KINDS: readonly SteerKind[];
10
+ /**
11
+ * Never trace-only, whatever a model does with them: holds, confirm and deny verdicts, security masking, the credential
12
+ * notice on masked output, and the done-check. Runaway recovery and subagent wakes start a turn of their own; muting
13
+ * them would change what the run does, not only what the agent is told.
14
+ */
15
+ export declare const NEVER_MUTED: ReadonlySet<SteerKind>;
16
+ interface Counts {
17
+ sent: number;
18
+ followed: number;
19
+ disputed: number;
20
+ }
21
+ export interface PairState extends Counts {
22
+ muted: boolean;
23
+ /** Steers asked for since the pair became trace-only or was last re-checked. */
24
+ asked: number;
25
+ /** Outcomes of the probe steers sent since then. */
26
+ probe: Counts;
27
+ }
28
+ export interface MutedPair {
29
+ model: string;
30
+ kind: SteerKind;
31
+ sent: number;
32
+ followed: number;
33
+ disputed: number;
34
+ }
35
+ /** Where the counts live: next to the user config, in pi-warden's data folder. */
36
+ export declare function steerStatsPath(): string;
37
+ export declare const DEFAULT_STEERS: SteersConfig;
38
+ /** True when the counts put the pair under the follow floor or over the dispute ceiling. */
39
+ export declare function failing(kind: SteerKind, counts: Counts, config: SteersConfig): boolean;
40
+ export type SteerDecision = "send" | "probe" | "trace-only";
41
+ /** The counts per (model, kind) pair, read once and written after every change. */
42
+ export declare class SteerStats {
43
+ private readonly path;
44
+ private data;
45
+ constructor(path?: string);
46
+ private load;
47
+ private save;
48
+ pair(model: string, kind: SteerKind): PairState | undefined;
49
+ private ensure;
50
+ /**
51
+ * Called before a steer of this kind goes out. A trace-only pair still sends the first of every `probeEvery` steers;
52
+ * after `recheckEvery` steers the probes decide: rates within the thresholds end trace-only, and the probe counts
53
+ * become the pair's new counts, so a fresh `minSteers` must pass before it can be muted again.
54
+ */
55
+ request(model: string, kind: SteerKind, config: SteersConfig): SteerDecision;
56
+ /**
57
+ * The outcome of one sent steer. Crossing the thresholds at `minSteers` or later makes the pair trace-only. Kinds in
58
+ * NEVER_MUTED are counted, for the status, and never muted.
59
+ */
60
+ observe(model: string, kind: SteerKind, outcome: {
61
+ followed: boolean;
62
+ disputed: boolean;
63
+ }, config: SteersConfig): void;
64
+ /** Drops the pair's counts; its steers are sent again. True when there was something to reset. */
65
+ unmute(model: string, kind: SteerKind): boolean;
66
+ /** Every trace-only pair, per model. */
67
+ muted(): MutedPair[];
68
+ /** Models with a pair of this kind, for `/warden unmute` without a model. */
69
+ modelsWith(kind: SteerKind): string[];
70
+ }
71
+ export declare function formatMuted(pairs: readonly MutedPair[], config: SteersConfig): string;
72
+ /** One assistant message after the steer: its text and its tool calls. */
73
+ export interface AssistantView {
74
+ text: string;
75
+ calls: Array<{
76
+ name: string;
77
+ input: Record<string, unknown>;
78
+ }>;
79
+ }
80
+ /** What the steer was about, for the follow check. */
81
+ export interface SteerSubject {
82
+ /** The file the note named (rules, slop, security). */
83
+ path?: string;
84
+ /** The call the note was about (stuck, repeat): a different next call is a course change. */
85
+ call?: {
86
+ name: string;
87
+ input: Record<string, unknown>;
88
+ };
89
+ /** The tool or skill the conscience named. */
90
+ capability?: {
91
+ kind: "tool" | "skill";
92
+ id: string;
93
+ };
94
+ }
95
+ /** Agent replies that argue with a note instead of acting on it. The same pattern the field replays used. */
96
+ export declare const DISPUTE: RegExp;
97
+ /**
98
+ * Whether the next assistant messages (at most two) changed course after a steer of this kind, and whether they
99
+ * disputed it. A port of the heuristic the field replays used; it is a proxy and does not prove the note caused the change.
100
+ */
101
+ export declare function steerOutcome(kind: SteerKind, subject: SteerSubject, views: readonly AssistantView[]): {
102
+ followed: boolean;
103
+ disputed: boolean;
104
+ };
105
+ /**
106
+ * Sent steers waiting for the agent's next two messages. A steer queued for the next turn waits past the next user
107
+ * message; any other steer that meets a user message first is settled on what it saw, or dropped when it saw nothing.
108
+ */
109
+ export declare class SteerWatch {
110
+ private pending;
111
+ add(model: string, kinds: readonly SteerKind[], subject?: SteerSubject, options?: {
112
+ nextTurn?: boolean;
113
+ }): void;
114
+ /** Feeds one assistant message; returns the steers it settled. */
115
+ assistant(view: AssistantView): Array<{
116
+ model: string;
117
+ kind: SteerKind;
118
+ followed: boolean;
119
+ disputed: boolean;
120
+ }>;
121
+ /** A user message arrived. */
122
+ user(): Array<{
123
+ model: string;
124
+ kind: SteerKind;
125
+ followed: boolean;
126
+ disputed: boolean;
127
+ }>;
128
+ clear(): void;
129
+ }
130
+ /** The text and tool calls of an assistant message as Pi stores it. */
131
+ export declare function assistantView(content: unknown): AssistantView;
132
+ export {};