pi-warden 0.25.0 → 0.28.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -6,6 +6,93 @@ How to keep this current: add the entry in the same pull request as the change,
6
6
 
7
7
  ## Unreleased
8
8
 
9
+ <!-- Empty. Next release starts here. -->
10
+
11
+ ## 0.28.4
12
+
13
+ ### Fixed
14
+
15
+ - Extension no longer crashes on load when node:sqlite is unavailable (e.g. some Node v25 builds). Learning features disabled gracefully; all guards still work.
16
+
17
+ ## 0.28.3
18
+
19
+ ### Fixed
20
+
21
+ - Temp output directories from saveOutput() are now cleaned up at session start (~2.5 MB/session leak).
22
+
23
+ ## 0.28.2
24
+
25
+ ### Fixed
26
+
27
+ - One-time warning when confirm mode falls back to steer in headless sessions.
28
+
29
+ ## 0.28.1
30
+
31
+ ### Fixed
32
+
33
+ - TypeSafe consent disclosure now mentions the SQLite hold database, its contents, and the retention policy.
34
+
35
+ ## 0.28.0
36
+
37
+ ### Added
38
+
39
+ - \`learning.retentionDays\` config key (default 365) — prunes hold records older than this on startup. \`0\` disables pruning.
40
+
41
+ ### Fixed
42
+
43
+ - SQLite hold database grew unboundedly; now pruned on startup with VACUUM.
44
+ - Empty catch blocks in learning.ts now log warnings instead of silently swallowing errors.
45
+
46
+ ## 0.27.2
47
+
48
+ ### Fixed
49
+
50
+ - Redact user prompts before storing in SQLite hold records (JSONL was already safe).
51
+
52
+ ## 0.27.1
53
+
54
+ ### Removed
55
+
56
+ - Dead `SessionCompressionTracker` class — `getMultiplier()` and `record()` were never called outside tests. ~50 lines removed.
57
+
58
+ ## 0.27.0
59
+
60
+ ### Added
61
+
62
+ - Sentence-style status bar templates (70+ across all 9 guards). `pickSentenceTemplate()` selects the best template from token context; zero LLM cost.
63
+ - `widget.barMode: "live"` (default): status bar shows one sentence like *"warden allowed bash action — write to src/config.ts"* instead of data-style tokens.
64
+ - `widget.barMode: "stack"`: the previous multi-guard data-style view.
65
+ - Compact sidebar mode: trace entries default to 1–3 lines; press `d` to toggle detailed view.
66
+ - Interactive config panel (`/warden config`): arrow-key navigation, Enter to toggle booleans or edit values, `s` to save.
67
+ - `/warden config set <key.path> <value>` and `/warden config get <key.path>` for quick CLI edits.
68
+ - `TraceEntry.tokens` field for live-mode re-rendering with sentence templates.
69
+ - `formatVerdictTokens()` returns both rendered line and raw tokens.
70
+ - `setNestedValue`, `getNestedValue`, `parseConfigValue` config helpers.
71
+
72
+ ### Changed
73
+
74
+ - `widget.barMode` defaults to `"live"`.
75
+ - Removed 7 exported-but-never-called functions and 3 non-functional blocks (proactive guidance, adaptive thresholds, intent prediction).
76
+
77
+ ### Fixed
78
+
79
+ - `/warden config` now works before any guard activity fires (`lastUi` set on session start).
80
+
81
+ ## 0.26.0
82
+
83
+ ### Added
84
+
85
+ - OpenRouter as a configurable judgment backend (`typesafeBackend` in user config). Set to `"openrouter"` to route Jev decisions through OpenRouter's decisions API instead of api.typesafe.ai. New `src/backend.ts` module; `OPENROUTER_API_KEY` env var; project overrides cannot redirect judgments.
86
+
87
+ ### Changed
88
+
89
+ - Bumped pi-typesafe to ^0.6.0 (native `backend` option on `TypeSafeOptions`).
90
+ - Deduplicated `resolveBackend` call in `shape.ts`.
91
+
92
+ ### Fixed
93
+
94
+ - Host-TUI capability guard: the extension no longer crashes when the host's bundled `pi-tui` lacks the `MouseRegion` component (e.g. omp 18.2.5). A missing named import was a link-time error that silently killed every guard, the `/warden` command, and the shortcut. The import is now a namespace property read (`tuiModule.MouseRegion`), which degrades to `undefined` instead of failing. The status widget renders the same guard text without the clickable wrapper.
95
+
9
96
  ## 0.25.0
10
97
 
11
98
  ### Added
package/README.md CHANGED
@@ -1,174 +1,59 @@
1
1
  # pi-warden
2
2
 
3
- Your `AGENTS.md` is a request. pi-warden is the check behind it.
3
+ **Stop babysitting your coding agent.**
4
4
 
5
- On every write and edit, [Jev](https://typesafe.ai) judges the change against the rules in your `pi-warden.md` and quotes the broken rule back into the agent's own context. The same channel names slop, breaks retry loops, calls out "done" when no test ran, keeps large tool output small, triages async subagent reports, and holds the rare destructive command. You keep working. The agent gets a nudge instead of you getting a dialog.
5
+ pi-warden supervises Pi while it works, catching risky actions, ignored rules, stuck loops, unverified "done" claims, security issues, runaway output, wasted context, and more.
6
6
 
7
- Measured: 150 paired headless runs broke a project rule **6 times with pi-warden off and 0 times with it on**. On 321 of my own sessions (17,160 guarded calls) the action guard held 42 calls and my next message approved 5, so 37 stood. A judgment costs about 1,000 input tokens (roughly $0.00004) and lands in about 0.3 s, which is why it can fire on every guarded call.
7
+ Instead of interrupting you for every problem, Warden usually feeds the issue back to the agent so it can correct itself and keep going.
8
8
 
9
9
  ![Real verdicts from pi-warden: the same command gets a different verdict depending on what the user asked for](https://raw.githubusercontent.com/DevMortimer/pi-warden/main/docs/preview.png)
10
10
 
11
- The verdicts above are real output from `npm run test:live`. Independent project; not affiliated with TypeSafe AI or the Pi authors. Built on [pi-typesafe](https://github.com/DevMortimer/pi-typesafe).
12
-
13
- ## What it watches
14
-
15
- | Guard | Watches | Does |
16
- | --- | --- | --- |
17
- | **Rules** | every `write` and `edit` | judges the written code against your project's Markdown rules (`pi-warden.md`) and names the violated rule to the agent |
18
- | **Slop** | written code and final replies | names stubs, restating comments, dead code, hedging, padded replies; the agent fixes them in the next edit |
19
- | **Stuck** | tool results | notices three failures with the same strategy and asks for a new hypothesis |
20
- | **Done-check** | the final message | catches "done" claims after code changes when no test, build, or lint passed |
21
- | **Context saver** | large or repeated tool output | keeps the exact lines that matter, stores the rest in a file, points recalls at a search |
22
- | **Action** | every `bash`, `write`, `edit` (and context-mode's `ctx_execute*`) before it runs | holds irreversible calls and tells the agent why, so it re-plans or asks you; steers it back when a change is unrelated to your request |
23
- | **Security** | written code and tool output | flags hardcoded secrets, disabled TLS, unsafe interpolation; marks injected instructions in tool output; fixture and documentation stand-ins are traced, never announced |
24
- | **Runaway** | the reply stream | stops a reply that repeats the same block over and over (code only, no request) |
25
- | **Steer budget** | the agent's attention | bounds the steers one run can demand: repeats and notices past the budget are recorded in the trace only, and a final reply that mostly restates an earlier reply of the same run is counted, never steered |
26
- | **Subagent triage** | async subagent reports | Jev decides whether a child report is worth waking the agent for; routine progress stays in context and costs no request ([detail](https://github.com/DevMortimer/pi-warden/blob/main/docs/guards.md#subagent-triage)) |
27
- | **Notifications** (opt-in) | moments that need you | desktop notification for a held call, a confirm dialog, a runaway stop |
28
-
29
- Three jobs, one channel: keep the agent on your rules, keep slop out of the code and the replies, keep the context small. The action guard is the seatbelt underneath: on 17,000 recorded calls it held 42 times. Everything arrives through the agent's own context, so the agent gets smarter and you are not the approval button. Full detail per guard, with thresholds and tuning numbers, is in [docs/guards.md](https://github.com/DevMortimer/pi-warden/blob/main/docs/guards.md).
30
-
31
- ## In one minute
11
+ ## Install
32
12
 
33
13
  ```bash
34
14
  pi install npm:pi-warden
35
15
  ```
36
16
 
37
- Then, inside Pi:
38
-
39
- 1. `/warden enable`. Read the data notice and confirm. If no key is stored yet, paste one from [console.typesafe.ai](https://console.typesafe.ai) (hidden input, saved owner-only, shared with pi-typesafe).
40
- 2. `/warden test` shows one synthetic verdict and what the agent would be told.
41
- 3. Work as usual. The line above the editor shows the latest verdict, folding the guards that found nothing into one line; `ctrl+shift+w` opens the trace with the scores and the exact text the agent received.
42
-
43
- Requires Pi 0.85 or newer and Node.js 22.19 or newer. Without a key, the offline parts still run: the pattern list, the runaway stop, duplicate-output notes, sensitive-path notes, and the credential-shape warnings. The [examples folder](examples/README.md) has a starter rules file and both config files.
44
-
45
- ## Does it improve the code?
46
-
47
- **Short answer: the warden cell did not break a rule in 150 paired runs. The control cells did, in 6 of them.**
48
-
49
- `npm run eval:ab` runs the same fixture tasks through headless Pi twice. The control cell gets the rules as prose in `AGENTS.md`; the warden cell gets the same file plus pi-warden enforcing `pi-warden.md`. A mechanical checker scores every final diff and shares no code with the guard, so no guard verdict can influence a score. Each task ships a shortcut that passes the provided tests while breaking a rule, so only the checker can tell the cells apart.
50
-
51
- | Model | Runs per cell | Control runs with a rule violation | Warden runs with a rule violation | Tests fully passing (control / warden) |
52
- | --- | --- | --- | --- | --- |
53
- | glm-5.3-flash | 60 | **5** | **0** | 54 of 60 / 55 of 60 |
54
- | deepseek-v4.1-flash | 90 | **1** | **0** | 84 of 90 / 82 of 90 |
55
-
56
- The rules the traps break are the ones no linter can check and no model knows from training: an endpoint pasted inline instead of imported from the config module, a `TODO` without an issue, `console.log` in library code, an exported function with no documented return value. The fixture carries ten of them, and both cells are told all ten: the warden only gets credit when the model forgets or cuts a corner.
57
-
58
- The clipping task shows the shape of the effect. Without the warden, 6 of 10 glm runs cut user-visible text to fit. With it, 1 of 10, and every run that received a clip steer ended clean.
59
-
60
- Run it on your own model: `npm run eval:ab -- --repeats 3 --model <model>` (it asks you to pick a model on purpose, because a full batch spends real tokens). Per-batch reports, including the runs where nothing changed, live in [eval/reports/](eval/reports/README.md).
61
-
62
- The guards are also stable over repeated invocations. A continuous overnight run (2026-09-18) evaluated every guard across **109 cycles, 13,952 cases, 100% pass rate, zero score drift**. The same synthetic input produces the same judgment within rounding noise across all runs. Full numbers: [docs/overnight-eval.md](docs/overnight-eval.md) and [eval/reports/2026-09-18-overnight-stability/](eval/reports/2026-09-18-overnight-stability/).
63
-
64
- ### Why a strong model does not make this pointless
65
-
66
- A guard like this is a smoke alarm. Most days it says nothing, and that is the design: it only pays off on the run where the model is tired, compacted, four hours in, or confident about a shortcut you would have caught in review. Four of its ten guards do not depend on the model being weak at all:
67
-
68
- - **Context saver**: large tool output is replaced by the lines that matter plus a file pointer. That is your token bill, on any model.
69
- - **Runaway stop** and **stuck** break a loop the model cannot see itself repeating, on any model.
70
- - **Done-check** reads the final message against the checks that actually ran, so "all tests pass" gets checked, not trusted.
71
- - **Subagent triage** scans async child reports so routine progress does not wake the parent agent or cost a request.
72
-
73
- The rules side is calibrated for the tail, not the average. Judge it on the runs where it fires, which is what the [trace panel](https://github.com/DevMortimer/pi-warden/blob/main/docs/guards.md) and `/warden status` hold precision are for.
74
-
75
- ### What it does not do yet
76
-
77
- Published because the negative is as useful as the positive. On the current benchmark, the warden's delta is concentrated in the rules guard. Two axes showed no measurable delta: the reply's claims about tests and builds (0 false claims in either cell), and blast radius (7 unasked action runs in the control cell against 6 in the warden cell, where the guard warned on a recursive `rm` without holding it, and did not hold `sh scripts/deploy.sh` on a "deploy it" prompt). A 12-turn decay arc also found no rule-violation growth in either cell. Sample sizes are small (15 tasks, two models), the checker needed five corrections before its numbers were quotable, and the instrument is in the same repo as the thing it measures. Re-run it and quote the folder.
78
-
79
- ## Two moments from real sessions
80
-
81
- Both are from my own work sessions, taken from the session logs. Project names are redacted, paths shortened, secrets replaced. The other two, a placeholder the agent talked itself out of removing and a retry loop broken mid-run, are in [docs/examples.md](https://github.com/DevMortimer/pi-warden/blob/main/docs/examples.md).
82
-
83
- ### It quoted a project rule back, and the agent fixed it in the next turn
84
-
85
- The project's `pi-warden.md` rule "No hardcoded secrets" says connection URLs come from settings or the environment. The agent wrote a demo script with the local database DSN as a fallback default.
86
-
87
- ```text
88
- assistant
89
- [write] scripts/demo_rail_decisions.py (186 lines)
90
- DSN = os.getenv("SUPABASE_DB_URL", "postgresql://[local default]")
91
-
92
- pi-warden (steer message to the agent)
93
- pi-warden: the content just written to scripts/demo_rail_decisions.py violates
94
- project rule from pi-warden.md: "No hardcoded secrets" (0.88): Source and config
95
- code must not contain passwords, API keys, tokens, or connection URLs (including
96
- Supabase DSNs and `FLY_API_TOKEN` values). They come from settings
97
- (`utils/config.py`) or the environ… [77 more chars]. Fix it in your next edit.
98
-
99
- assistant
100
- The warden flags the hardcoded local DSN. Right call. Making the script require
101
- `SUPABASE_DB_URL` from the environment instead:
102
- DSN = os.getenv("SUPABASE_DB_URL")
103
- if not DSN:
104
- sys.exit("Set SUPABASE_DB_URL to the local stack's DB_URL (supabase status)")
105
- ```
106
-
107
- The rule is three lines of Markdown in the repo. No linter knows what a Supabase DSN looks like in a default argument; Jev does, and the agent got the rule text, not a red X.
108
-
109
- ### It held a database reset, and the agent finished everything else first
110
-
111
- A feature branch with a migration. The agent decided to replay the local database. `supabase db reset` is legitimate in that repo, but it wipes local state and nobody had asked for it.
112
-
113
- ```text
114
- assistant
115
- [bash] cd .claude/worktrees/con-142 && supabase db reset 2>&1 | tail -6
116
-
117
- tool result (written by pi-warden; the command did not run)
118
- pi-warden held this bash call before it ran: irreversible 0.85. Do not retry it
119
- unchanged. Either (1) reach the goal with a recoverable alternative that stays
120
- inside the project (a targeted path, a dry run, a move instead of a delete, a
121
- normal push), or (2) if this exact action is genuinely required, stop and tell
122
- the user in one or two sentences what it does, what cannot be undone, and why
123
- it is needed, then wait for their reply. [...the template continues]
124
-
125
- assistant
126
- The warden held `supabase db reset`. It's the sanctioned local-stack replay
127
- (loopback docker only, no dev/prod reach), but I'll pause that step and ask at
128
- the end of this update. Continuing with lanes that don't need it:
129
- [bash] uv run pytest tests/canon tests/services/messaging ... 92 passed in 24.80s
130
-
131
- assistant (final message, about 16 tool calls later)
132
- Draft PR is up, work committed and pushed. One decision is pending for you.
133
- PR: redacted#2640 (draft) ...
134
- ```
135
-
136
- No dialog, no interruption. The agent moved the risky step to the end and asked in one sentence.
137
-
138
- ## How it decides
139
-
140
- Every guarded call (`bash`, `write`, `edit`, and shell-like tools such as context-mode's `ctx_execute`) goes through four steps:
17
+ 1. `/warden enable` to activate. Paste a key from [console.typesafe.ai](https://console.typesafe.ai) if you have one (hidden input, saved owner-only), or skip for offline-only mode.
18
+ 2. `/warden test` to see one synthetic verdict.
19
+ 3. Work as usual. Status line shows verdicts; `ctrl+shift+w` opens the trace sidebar.
141
20
 
142
- 1. **Read-only? Skip.** `git status && ls` or a `read`: no request, no line in the trace.
143
- 2. **Pattern list, offline.** Force push, `git reset --hard`, recursive `rm` outside the project, SQL `DROP`/`TRUNCATE`/`DELETE FROM`, `curl | sh`, publishing, infrastructure destroys: the call is held. Risky but recoverable things (`rm -rf` on a project path, `sudo`, `--no-verify`, `gh pr merge`, touching `.env`) warn. Text that is data, such as a commit message that mentions `git push --force`, is not a command.
144
- 3. **Jev, with your consent.** One request carries your request, up to eight earlier messages, the agent's own words right before the call (its plan), and the call itself. Jev answers fixed questions with probabilities: irreversible, off task, does it change anything, how it relates to the task, does it contradict the plan, is the effect visible outside the working tree. On a write the slop and security questions ride along, and the rules check is a second request in parallel. About 0.3 s before the tool runs.
145
- 4. **Act.** Three things hold a call: a destructive pattern, a `deny` command rule (blocked outright, no dialog), or `irreversible` at 0.7 or above. Everything else (off task, plan mismatch, slop, a rule violation, a security smell) tells the agent and lets the call run.
21
+ Works without any key (offline guards: pattern list, runaway stop, sensitive-path notes, credential warnings). Requires Pi 0.85+, Node 22.19+.
146
22
 
147
- In the default `steer` mode a hold goes back to the agent as its tool result with two acceptable next moves: find a recoverable way, or explain the action to you in a sentence and wait. If your reply approves it, the retry goes through. `confirm` mode shows you a dialog instead; `advise` never holds.
23
+ ## What Warden catches
148
24
 
149
- Three rules hold all of this together. Patterns set the floor and Jev can only raise it. The agent's plan can add a nudge but never remove a hold. Jev decides, code applies the decision, and the LLM is never asked to judge itself.
150
-
151
- **Why Jev and not a second LLM call.** A pattern list cannot tell `db:reset` after "reset the database" from `db:reset` after "add a column", and a second LLM call per tool call is too slow and expensive. Jev returns calibrated probabilities to fixed questions, cheap enough for every guarded call. The full argument, and how large a request gets, is in [docs/guards.md#why-jev](https://github.com/DevMortimer/pi-warden/blob/main/docs/guards.md#why-jev-and-not-a-second-llm-call).
152
-
153
- ## What the numbers say
154
-
155
- `node scripts/calibrate-action.mjs --all` replays every guarded call in your recorded Pi sessions through the guard and asks Jev, once per turn, whether your next message approved a hold, regretted a call that ran, or rejected the turn. On 321 of my own sessions (17,160 guarded calls): **42 holds, and my next message approved 5**, so 37 stood; **20 regretted calls (2% of turns)**, all scope rather than data loss; and off-task caused 56 of the 139 replay holds without drawing a complaint, so it now steers and never holds.
25
+ | Guard | Watches | Does |
26
+ | --- | --- | --- |
27
+ | **Action** | `bash`, `write`, `edit` before run | Holds irreversible calls, steers off-task changes |
28
+ | **Rules** | every `write` and `edit` | Judges against project Markdown rules, quotes the broken rule |
29
+ | **Slop** | code and replies | Names stubs, restating comments, dead code, hedging |
30
+ | **Stuck** | tool results | Repeated failures using the same strategy |
31
+ | **Done-check** | final message | "Done" claims with no test/build/lint behind them |
32
+ | **Security** | code and output | Hardcoded secrets, injection risks |
33
+ | **Runaway** | reply stream | Stops replies that begin repeating themselves |
34
+ | **Subagent triage** | async child reports | Keeps noisy background work from waking the parent |
156
35
 
157
- The labelling never stops: what you do after each hold (approve, decline, redirect, complain) becomes a label, and `/warden status` shows hold precision for your own sessions. The signal ranking, the intent threshold, the candidate questions that lost, and what 67 live steers did to the agent are in [docs/guards.md#calibration](https://github.com/DevMortimer/pi-warden/blob/main/docs/guards.md#calibration).
36
+ ## How intervention works
158
37
 
159
- ## Questions people asked
38
+ 1. **Read-only? Skip.** `git status`, `ls`, `read` means no request and no trace entry.
39
+ 2. **Known-dangerous pattern? Catch it locally.** Force push, `git reset --hard`, recursive `rm`, SQL `DROP` held instantly.
40
+ 3. **Needs judgment?** Warden evaluates the action in context.
41
+ 4. **Steer or hold.** Most issues go back to the agent so it can correct itself. Irreversible actions can be stopped before they run.
160
42
 
161
- **How is this different from putting the rules in AGENTS.md?**
162
- An `AGENTS.md` is read once and asks the model to remember. Long sessions, compaction, and cheap models forget. pi-warden re-reads your rules file when it changes and judges every write against it at the moment it happens, then quotes the rule the agent broke. The rules can be things a linter cannot check: "exported functions declare return types", "a TODO must name a ticket", "comments must not restate the code". Keep the AGENTS.md; this is the check behind it.
43
+ In the default `steer` mode, Warden talks to the agent rather than interrupting you. `confirm` asks you directly; `advise` never blocks.
163
44
 
164
- **Do I need a TypeSafe key?**
165
- Only for Jev judgments. Without one you still get the pattern list, the runaway stop, duplicate-output notes, sensitive-path notes, and credential-shape warnings. Keys come through a waitlist at [console.typesafe.ai](https://console.typesafe.ai); install now and run `/warden enable` when yours arrives.
45
+ ## More than guardrails
166
46
 
167
- The other answers (regex, Claude Code's auto mode, cost, permissions, what is sent to TypeSafe, project overrides, headless CI, and the size of a Jev request) are in [docs/faq.md](https://github.com/DevMortimer/pi-warden/blob/main/docs/faq.md).
47
+ - **Learns from holds**, tracking outcomes and recommending policy changes via `/warden recommend`.
48
+ - **Custom policies**: command, path, and arming rules for your own workflow.
49
+ - **Context saver** trims oversized tool output while keeping the full result retrievable.
50
+ - **Multiple judgment backends**: TypeSafe by default, with OpenRouter support.
51
+ - **Full traceability**: inspect what Warden saw, decided, and told the agent.
52
+ - **Desktop alerts** for events that need you.
168
53
 
169
- ## Rules for your project
54
+ ## Custom rules
170
55
 
171
- Put `pi-warden.md` at the project root. Each heading is one rule; the text under it is the specification. An optional `paths:` line limits the rule to matching files.
56
+ Put `pi-warden.md` at the project root. Each heading is one rule:
172
57
 
173
58
  ```markdown
174
59
  # No console statements
@@ -179,45 +64,40 @@ paths: src/**/*.ts
179
64
  Every exported function declares its return type.
180
65
  ```
181
66
 
182
- On every write and edit, Jev judges the change against each rule: compliant, violation, not applicable, or insufficient context. A violation at 0.7 or above is quoted back to the agent (heading, rule text, and which edit); the write goes through, because a held write would leave a half-written file. The third hit of one rule in a session is called out as a standing rule, and files are re-read when they change.
183
-
184
- Without a `pi-warden.md`, the first of `README.md`, `CLAUDE.md`, `AGENTS.md` is judged as one document instead, so existing context files work from day one. Starter file with a dozen rules: [`examples/pi-warden.md`](examples/pi-warden.md). Path globs, sensitive-path notes, and the tuning set: [docs/guards.md#rules](https://github.com/DevMortimer/pi-warden/blob/main/docs/guards.md#rules).
67
+ Jev judges every write and quotes violations back. Rules can be things no linter checks: "a TODO must name a ticket", "comments must not restate the code". Without `pi-warden.md`, the first of `README.md`/`CLAUDE.md`/`AGENTS.md` is judged instead. [Examples and config details.](docs/configuration.md)
185
68
 
186
- ## Commands
69
+ ## Does it actually help?
187
70
 
188
- Seven commands: `/warden status`, `enable`, `disable`, `mode steer|confirm|advise`, `config`, `test`, `trace`. `status` reports guard state, session counts, steer counts per guard, thresholds, hold precision, and config paths; `trace` toggles the sidebar with the exact text the agent received. The full table is in [docs/commands.md](https://github.com/DevMortimer/pi-warden/blob/main/docs/commands.md).
71
+ In 150 paired agent runs, the control setup violated the tested project rule **6 times**. With Warden: **0**.
189
72
 
190
- ## Configuration
73
+ An overnight stability run covered **13,952 guard cases across 109 cycles** with no score drift.
191
74
 
192
- Two optional files: your defaults in `~/.pi/agent/pi-warden/config.json` (`/warden config` opens it) and per-repo overrides in `.pi/pi-warden.json`. Common tweaks are `mode`, `steerVisible`, `notices`, `notify.enabled`, `slop.prose.audience`, `rules.skip`, and `context.enabled`. Every key with its default, what a project file may and may not change, and the environment variables: [docs/configuration.md](https://github.com/DevMortimer/pi-warden/blob/main/docs/configuration.md). Worked examples of all three files: [examples/README.md](examples/README.md).
75
+ These are project-maintained benchmarks, not universal claims. Raw reports and reproducible eval tooling are in the repo.
193
76
 
194
- ## Status line and trace
77
+ [See the evals ->](eval/reports/)
195
78
 
196
- The line above the editor shows the latest verdict per guard, the verdict leading as a chip: `WARN action write · irreversible 0.09 · off-task 0.95 · unrelated · off task`. The guards that found nothing fold into one line per verdict, so a quiet turn costs one line. `/warden trace`, `ctrl+shift+w`, or a click on the line opens a right-hand sidebar with the full trace, newest first. Clicks need Pi's fullscreen mode (`tuiMode: "fullscreen"` in `/settings`). Widget templates and placement: [docs/configuration.md#status-line-and-trace-sidebar](https://github.com/DevMortimer/pi-warden/blob/main/docs/configuration.md#status-line-and-trace-sidebar).
79
+ ## Privacy
197
80
 
198
- ## For extension authors
81
+ Depending on the configured judgment backend, Warden sends to your provider: a redacted summary of each guarded call, the agent's plan, up to eight redacted prior messages, and redacted tool-output samples. Secrets are stripped before anything leaves the machine. [Full details.](docs/data-handling.md)
199
82
 
200
- Every guard is a plain function you can call with any object that has pi-typesafe's `evaluate` method as the judge, and the library has no dependency on Pi's runtime, so it is safe in tests. Calls go through pi-typesafe's `ask`, so timeouts, aborts, and error shapes are pi-typesafe's contract:
83
+ ## Documentation
201
84
 
202
- ```ts
203
- import { evaluateAction, defaultConfig } from "pi-warden";
204
- import { createTypeSafe } from "pi-typesafe";
205
-
206
- const verdict = await evaluateAction(
207
- { tool: "bash", input: { command: "git push --force" }, cwd: process.cwd(), task: "push my branch" },
208
- { config: defaultConfig().action, judge: createTypeSafe() }, // omit judge for pattern checks only
209
- );
210
- ```
211
-
212
- The rules guard, the session-level `ActionGuard` and `RulesGuard`, and the other exports are in [docs/extension-authors.md](https://github.com/DevMortimer/pi-warden/blob/main/docs/extension-authors.md).
85
+ - [docs/guards.md](docs/guards.md): every guard, thresholds, calibration, Jev questions
86
+ - [docs/configuration.md](docs/configuration.md): all config keys, defaults, status line templates
87
+ - [docs/commands.md](docs/commands.md): command reference
88
+ - [docs/faq.md](docs/faq.md): cost, permissions, headless CI, project overrides
89
+ - [docs/data-handling.md](docs/data-handling.md): what leaves the machine, what stays
90
+ - [docs/examples.md](docs/examples.md): real session examples
91
+ - [examples/](examples/): starter rules file, both config files
213
92
 
214
93
  ## Development
215
94
 
216
- `npm install`, then `npm run check` (typecheck, 185 offline tests on a mocked transport, build). `npm run test:live` spends real judgments across the guards; `node scripts/calibrate-action.mjs --dry-run` estimates a calibration run on your own sessions; `npm run eval:ab -- --dry-run` lists a benchmark batch without spending anything; `npm run dev:pi` starts Pi with this working tree. Release history, including the measured numbers behind each change: [CHANGELOG.md](CHANGELOG.md). The rules the code keeps (steers never hold, approval comes from the user only, a new question ships with a measurement) are in [CONTRIBUTING.md](CONTRIBUTING.md).
217
-
218
- ## Credits
219
-
220
- Rules as headings and the four outcomes come from [jevrealtimecodecheck](https://github.com/MrDesjardins/jevrealtimecodecheck); path globs, sensitive paths, and "judge the change, not its story" from [wince](https://github.com/TinyFrontier/wince).
95
+ ```bash
96
+ npm install
97
+ npm run check # typecheck + offline tests + build
98
+ npm run test:live # live tests (costs real judgments)
99
+ npm run eval:ab # A/B benchmark (costs real tokens)
100
+ ```
221
101
 
222
102
  ## License
223
103
 
@@ -0,0 +1,23 @@
1
+ import type { TypeSafeOptions } from "pi-typesafe";
2
+ /** The judgment backend that receives pi-warden's Jev requests. */
3
+ export type JudgmentBackend = "typesafe" | "openrouter";
4
+ /** The host the consent disclosure names as the destination. */
5
+ export declare function backendHost(backend: JudgmentBackend): string;
6
+ /** Adapt the consent text to the active backend by substituting the destination host. */
7
+ export declare function disclosureFor(backend: JudgmentBackend, disclosure: string): string;
8
+ /**
9
+ * Whether the right credential exists for the chosen backend.
10
+ * TypeSafe uses the pi-typesafe keystore; other backends use an environment variable.
11
+ */
12
+ export declare function keyAvailable(backend: JudgmentBackend, env: Record<string, string | undefined>, keystoreUsable: () => boolean): boolean;
13
+ /**
14
+ * Build the options forwarded to `createTypeSafe`.
15
+ * The caller must gate on `keyAvailable` before calling `createTypeSafe`.
16
+ */
17
+ export declare function judgeOptions(config: {
18
+ maxRequests: number;
19
+ timeoutMs: number;
20
+ typesafeBackend: JudgmentBackend;
21
+ }): TypeSafeOptions;
22
+ /** Resolve the effective backend from a raw config value; invalid values fall back to "typesafe". */
23
+ export declare function resolveBackend(raw: unknown): JudgmentBackend;
@@ -0,0 +1,36 @@
1
+ import { authState } from "pi-typesafe";
2
+ const BACKENDS = {
3
+ typesafe: { host: "api.typesafe.ai" },
4
+ openrouter: { host: "openrouter.ai", keyEnv: "OPENROUTER_API_KEY" },
5
+ };
6
+ /** The host the consent disclosure names as the destination. */
7
+ export function backendHost(backend) {
8
+ return BACKENDS[backend].host;
9
+ }
10
+ /** Adapt the consent text to the active backend by substituting the destination host. */
11
+ export function disclosureFor(backend, disclosure) {
12
+ return backend === "typesafe" ? disclosure : disclosure.replace(BACKENDS.typesafe.host, BACKENDS[backend].host);
13
+ }
14
+ /**
15
+ * Whether the right credential exists for the chosen backend.
16
+ * TypeSafe uses the pi-typesafe keystore; other backends use an environment variable.
17
+ */
18
+ export function keyAvailable(backend, env, keystoreUsable) {
19
+ const keyEnv = BACKENDS[backend].keyEnv;
20
+ return keyEnv ? (env[keyEnv]?.trim() ?? "") !== "" : keystoreUsable();
21
+ }
22
+ /**
23
+ * Build the options forwarded to `createTypeSafe`.
24
+ * The caller must gate on `keyAvailable` before calling `createTypeSafe`.
25
+ */
26
+ export function judgeOptions(config) {
27
+ return {
28
+ maxRequests: config.maxRequests,
29
+ timeoutMs: config.timeoutMs,
30
+ ...(config.typesafeBackend === "typesafe" ? {} : { backend: config.typesafeBackend }),
31
+ };
32
+ }
33
+ /** Resolve the effective backend from a raw config value; invalid values fall back to "typesafe". */
34
+ export function resolveBackend(raw) {
35
+ return raw === "typesafe" || raw === "openrouter" ? raw : "typesafe";
36
+ }
package/dist/config.d.ts CHANGED
@@ -1,3 +1,4 @@
1
+ import type { JudgmentBackend } from "./backend.js";
1
2
  import type { WidgetConfig } from "./widget.js";
2
3
  export interface Threshold {
3
4
  /** P(yes) at or above this shows a warning and continues. */
@@ -212,11 +213,25 @@ export interface SubagentConfig {
212
213
  export type RecallTool = "auto" | "rg" | "ag" | "ugrep" | "git-grep" | "grep" | "select-string" | "findstr" | "none";
213
214
  export declare function isRecallTool(value: unknown): value is RecallTool;
214
215
  export type WardenMode = "steer" | "confirm" | "advise";
216
+ export interface LearningConfig {
217
+ /** Enable adaptive thresholds based on learning data. */
218
+ adaptiveThresholds: boolean;
219
+ /** Enable pattern analysis and recommendations. */
220
+ patternAnalysis: boolean;
221
+ /** Minimum number of holds before adaptive thresholds kick in. */
222
+ minHoldsForAdaptive: number;
223
+ /** How aggressively to adjust thresholds (0-1). Higher values mean faster adaptation. */
224
+ adaptationRate: number;
225
+ /** Days to keep hold records in SQLite before pruning. 0 disables pruning. */
226
+ retentionDays: number;
227
+ }
215
228
  export interface WardenConfig {
216
229
  /** Master switch. false disables every guard, including offline pattern checks. */
217
230
  enabled: boolean;
218
231
  /** Consent to send task and action summaries to api.typesafe.ai. Set by /warden enable; never by a project file. */
219
232
  typesafe: boolean;
233
+ /** The decisions service the judgments go to. User file only: a project must not redirect judgments to another vendor. */
234
+ typesafeBackend: JudgmentBackend;
220
235
  /**
221
236
  * steer (default): a confirm-level call is held and the agent receives the judgment as its tool result, so it re-plans or asks
222
237
  * the user in chat. confirm: open a dialog and let the user decide (falls back to steer without a UI). advise: never hold; report only.
@@ -248,6 +263,8 @@ export interface WardenConfig {
248
263
  * steer costs at least one LLM turn, and a closing run that collects six notices collects six restatements of the final
249
264
  * status. 0 disables the budget. Critical guards (stuck, done, runaway, subagent wake) always deliver. */
250
265
  steerBudget: number;
266
+ /** Learning and adaptation settings. */
267
+ learning: LearningConfig;
251
268
  }
252
269
  export declare const PACKAGE_NAME = "pi-warden";
253
270
  /** Bumped when WardenConfig gains a section; extension.ts checks it so a half-updated module graph is reported, not crashed on. */
@@ -274,4 +291,8 @@ export declare function readUserConfig(): Json;
274
291
  export declare function writeUserConfig(raw: Json): string;
275
292
  /** Persists one top-level user setting without disturbing the rest of the file. */
276
293
  export declare function setUserSetting(key: "typesafe" | "enabled" | "mode", value: boolean | WardenMode): string;
294
+ export declare function setNestedValue(obj: Record<string, unknown>, path: string, value: unknown): Record<string, unknown>;
295
+ export declare function getNestedValue(obj: Record<string, unknown>, path: string): unknown;
296
+ /** Coerce a CLI string into a JSON primitive so `/warden config set` stays ergonomic. */
297
+ export declare function parseConfigValue(value: string): unknown;
277
298
  export {};
package/dist/config.js CHANGED
@@ -1,6 +1,7 @@
1
1
  import { chmodSync, mkdirSync, readFileSync, renameSync, writeFileSync } from "node:fs";
2
2
  import { homedir } from "node:os";
3
3
  import { dirname, join } from "node:path";
4
+ import { resolveBackend } from "./backend.js";
4
5
  import { COMMAND_TOOLS } from "./tools.js";
5
6
  import { defaultWidgetConfig } from "./widget.js";
6
7
  const RECALL_TOOLS = ["auto", "rg", "ag", "ugrep", "git-grep", "grep", "select-string", "findstr", "none"];
@@ -15,6 +16,7 @@ export function defaultConfig() {
15
16
  return {
16
17
  enabled: true,
17
18
  typesafe: false,
19
+ typesafeBackend: "typesafe",
18
20
  mode: "steer",
19
21
  timeoutMs: 5000,
20
22
  maxRequests: 500,
@@ -47,6 +49,7 @@ export function defaultConfig() {
47
49
  steerVisible: false,
48
50
  notices: false,
49
51
  steerBudget: 3,
52
+ learning: { adaptiveThresholds: true, patternAnalysis: true, minHoldsForAdaptive: 20, adaptationRate: 0.1, retentionDays: 365 },
50
53
  };
51
54
  }
52
55
  /** Mirrors Pi's agent directory rule so the file sits next to pi-typesafe's auth.json. */
@@ -349,6 +352,7 @@ function applyWidget(base, raw) {
349
352
  return {
350
353
  enabled: boolean(raw.enabled, base.enabled),
351
354
  placement: raw.placement === "belowEditor" || raw.placement === "aboveEditor" ? raw.placement : base.placement,
355
+ barMode: raw.barMode === "live" || raw.barMode === "stack" ? raw.barMode : base.barMode,
352
356
  shortcut: typeof raw.shortcut === "string" ? raw.shortcut.trim() : base.shortcut,
353
357
  panelWidth: typeof raw.panelWidth === "number" && Number.isSafeInteger(raw.panelWidth) && raw.panelWidth >= 20 ? raw.panelWidth
354
358
  : typeof raw.panelWidth === "string" && /^[1-9]\d?%$/.test(raw.panelWidth.trim()) ? raw.panelWidth.trim() : base.panelWidth,
@@ -404,6 +408,7 @@ export function applyUserOverrides(base, raw) {
404
408
  return {
405
409
  enabled: boolean(raw.enabled, base.enabled),
406
410
  typesafe: boolean(raw.typesafe, base.typesafe),
411
+ typesafeBackend: resolveBackend(raw.typesafeBackend),
407
412
  mode: isMode(raw.mode) ? raw.mode : base.mode,
408
413
  ...shared,
409
414
  ...applyGuards(base, raw, shared.timeoutMs, "user"),
@@ -411,6 +416,18 @@ export function applyUserOverrides(base, raw) {
411
416
  steerVisible: boolean(raw.steerVisible, base.steerVisible),
412
417
  notices: boolean(raw.notices, base.notices),
413
418
  steerBudget: typeof raw.steerBudget === "number" && Number.isInteger(raw.steerBudget) && raw.steerBudget >= 0 ? raw.steerBudget : base.steerBudget,
419
+ learning: applyLearning(base.learning, raw.learning),
420
+ };
421
+ }
422
+ function applyLearning(base, raw) {
423
+ if (!isObject(raw))
424
+ return base;
425
+ return {
426
+ adaptiveThresholds: boolean(raw.adaptiveThresholds, base.adaptiveThresholds),
427
+ patternAnalysis: boolean(raw.patternAnalysis, base.patternAnalysis),
428
+ minHoldsForAdaptive: Math.max(5, positiveInteger(raw.minHoldsForAdaptive, base.minHoldsForAdaptive)),
429
+ adaptationRate: Math.max(0, Math.min(1, probability(raw.adaptationRate, base.adaptationRate))),
430
+ retentionDays: Math.max(0, positiveInteger(raw.retentionDays, base.retentionDays)),
414
431
  };
415
432
  }
416
433
  /** Project files may tune the guards but cannot grant TypeSafe consent, change the mode, or raise budgets. */
@@ -442,3 +459,37 @@ export function writeUserConfig(raw) {
442
459
  export function setUserSetting(key, value) {
443
460
  return writeUserConfig({ ...readUserConfig(), [key]: value });
444
461
  }
462
+ export function setNestedValue(obj, path, value) {
463
+ const keys = path.split(".");
464
+ const result = { ...obj };
465
+ let current = result;
466
+ for (let i = 0; i < keys.length - 1; i++) {
467
+ const key = keys[i];
468
+ current[key] = { ...(current[key] ?? {}) };
469
+ current = current[key];
470
+ }
471
+ current[keys.at(-1)] = value;
472
+ return result;
473
+ }
474
+ export function getNestedValue(obj, path) {
475
+ const keys = path.split(".");
476
+ let current = obj;
477
+ for (const key of keys) {
478
+ if (typeof current !== "object" || current === null)
479
+ return undefined;
480
+ current = current[key];
481
+ }
482
+ return current;
483
+ }
484
+ /** Coerce a CLI string into a JSON primitive so `/warden config set` stays ergonomic. */
485
+ export function parseConfigValue(value) {
486
+ if (value === "true")
487
+ return true;
488
+ if (value === "false")
489
+ return false;
490
+ if (value === "null")
491
+ return null;
492
+ if (/^-?\d+(\.\d+)?$/.test(value))
493
+ return Number(value);
494
+ return value;
495
+ }
@@ -2,7 +2,7 @@ import type { ExtensionAPI, ExtensionContext } from "@earendil-works/pi-coding-a
2
2
  import type { ToolCallRef } from "./action-guard.js";
3
3
  import type { SlopSymptom, Verdict } from "./guard.js";
4
4
  import type { ShapeResult } from "./shape.js";
5
- export declare const disclosure = "With TypeSafe judgments enabled, pi-warden sends to api.typesafe.ai: your latest request and up to eight redacted prior user/assistant text messages for task context, plus a redacted, truncated summary of each guarded bash, write, or edit call before it runs, with the agent's own words from the message that makes the call (its stated plan); for a write or edit in a project with a rules file (pi-warden.md, the configured files, or README/CLAUDE/AGENTS as fallback), a larger redacted sample of the written content with the current file around each edit and the rule text; the last few tool calls and output tails when the agent keeps failing; the agent's final message when it reports completion without running checks; redacted tool-output samples for security and context saving (retention and output format); a redacted sample of an async subagent report that names a failure, a stop, or a question, with your latest request, when warden decides whether that report should wake the agent; and, on the first guarded call after your reply, the redacted summaries of the calls allowed in the previous turn, so Jev can say whether your reply regrets one of them. Compression and duplicate notes store an exact, owner-only copy in a temporary file on this machine; the hold feedback log stores tool names, pattern ids, scores, and outcomes (never commands) in an owner-only file under Pi's agent directory. Requests may incur charges. Secret redaction is best-effort. Results are model judgments, not proof or authorization; offline pattern checks stay active either way.";
5
+ export declare const disclosure = "With TypeSafe judgments enabled, pi-warden sends to api.typesafe.ai: your latest request and up to eight redacted prior user/assistant text messages for task context, plus a redacted, truncated summary of each guarded bash, write, or edit call before it runs, with the agent's own words from the message that makes the call (its stated plan); for a write or edit in a project with a rules file (pi-warden.md, the configured files, or README/CLAUDE/AGENTS as fallback), a larger redacted sample of the written content with the current file around each edit and the rule text; the last few tool calls and output tails when the agent keeps failing; the agent's final message when it reports completion without running checks; redacted tool-output samples for security and context saving (retention and output format); a redacted sample of an async subagent report that names a failure, a stop, or a question, with your latest request, when warden decides whether that report should wake the agent; and, on the first guarded call after your reply, the redacted summaries of the calls allowed in the previous turn, so Jev can say whether your reply regrets one of them. Compression and duplicate notes store an exact, owner-only copy in a temporary file on this machine; the hold feedback log stores tool names, pattern ids, scores, and outcomes (never commands) in an owner-only file under Pi's agent directory; an owner-only SQLite database under Pi's agent directory stores redacted hold context (plan, summary, reason) for learning and retention (configurable, default 365 days). Requests may incur charges. Secret redaction is best-effort. Results are model judgments, not proof or authorization; offline pattern checks stay active either way.";
6
6
  /** Which guard spent the user's attention. The status line reports one count per guard. */
7
7
  export type SteerGuard = "action" | "rules" | "security" | "stuck" | "done" | "prose" | "runaway" | "subagent";
8
8
  /**