pi-warden 0.8.0 → 0.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +162 -105
- package/dist/config.d.ts +20 -1
- package/dist/config.js +24 -1
- package/dist/extension.d.ts +1 -1
- package/dist/extension.js +50 -8
- package/dist/guard.d.ts +13 -0
- package/dist/guard.js +122 -6
- package/dist/index.d.ts +7 -5
- package/dist/index.js +4 -3
- package/dist/panel.js +1 -1
- package/dist/rules.d.ts +164 -0
- package/dist/rules.js +479 -0
- package/dist/shape.d.ts +1 -1
- package/dist/shape.js +5 -2
- package/dist/trace.d.ts +3 -1
- package/dist/trace.js +14 -0
- package/dist/widget.d.ts +5 -0
- package/dist/widget.js +18 -0
- package/package.json +2 -2
package/README.md
CHANGED
|
@@ -1,34 +1,30 @@
|
|
|
1
1
|
# pi-warden
|
|
2
2
|
|
|
3
|
-
A second pair of eyes for [Pi](https://pi.dev)
|
|
3
|
+
A second pair of eyes for [Pi](https://pi.dev). It watches what the agent does, asks [Jev](https://typesafe.ai) small typed questions about it (about 250 ms each), and acts through the agent's own context: a held call, a short steer message, a compressed output. You keep working; the agent gets smarter.
|
|
4
4
|
|
|
5
5
|

|
|
6
6
|
|
|
7
|
-
The verdicts above are real output from `npm run test:live`. Independent project; not affiliated with TypeSafe AI or the Pi authors.
|
|
7
|
+
The verdicts above are real output from `npm run test:live`. Independent project; not affiliated with TypeSafe AI or the Pi authors. Built on [pi-typesafe](https://github.com/DevMortimer/pi-typesafe).
|
|
8
8
|
|
|
9
|
-
##
|
|
10
|
-
|
|
11
|
-
pi-warden is a harness for the agent, not a gate for the user. It watches every tool call, tool result, and reply, asks Jev small typed questions about them, and acts through the agent's own context — a held call, a steer message, a compressed output — rather than through dialogs. Jev decides; code applies the decision; the LLM is never asked to judge itself. Guard decisions are visible in the trace panel. Existing session entries are never rewritten; output warnings and excerpts are applied before a new tool result enters the session.
|
|
12
|
-
|
|
13
|
-
Three jobs, in order of maturity:
|
|
9
|
+
## What it does
|
|
14
10
|
|
|
15
|
-
|
|
|
11
|
+
| Guard | Watches | Does |
|
|
16
12
|
| --- | --- | --- |
|
|
17
|
-
| **
|
|
18
|
-
| **
|
|
19
|
-
| **
|
|
13
|
+
| **Action** | every `bash`, `write`, `edit` (and context-mode's `ctx_execute*`) before it runs | holds irreversible or off-task calls and tells the agent why, so it re-plans or asks you |
|
|
14
|
+
| **Rules** | every `write` and `edit` | judges the written code against your project's Markdown rules (`pi-warden.md`) and names the violated rule to the agent |
|
|
15
|
+
| **Slop** | written code and final replies | names stubs, restating comments, dead code, hedging, padded replies; the agent fixes them in the next edit |
|
|
16
|
+
| **Security** | written code and tool output | flags hardcoded secrets, disabled TLS, unsafe interpolation; marks injected instructions in tool output |
|
|
17
|
+
| **Stuck** | tool results | notices three failures with the same strategy and asks for a new hypothesis |
|
|
18
|
+
| **Runaway** | the reply stream | stops a reply that repeats the same block over and over (code only, no request) |
|
|
19
|
+
| **Done-check** | the final message | catches "done" claims after code changes when no test, build, or lint passed |
|
|
20
|
+
| **Context saver** | large or repeated tool output | keeps the exact lines that matter, stores the rest in a file, points recalls at a search |
|
|
21
|
+
| **Notifications** | moments that need you | desktop notification for a held call, a confirm dialog, a runaway stop |
|
|
20
22
|
|
|
21
|
-
|
|
23
|
+
Every verdict lands on a status line above the editor. Click it (fullscreen mode), press `ctrl+shift+w`, or run `/warden trace` for a live sidebar with the scores, the reasons, and exactly what the agent was told.
|
|
22
24
|
|
|
23
|
-
|
|
25
|
+
## Why Jev
|
|
24
26
|
|
|
25
|
-
|
|
26
|
-
- **Scope drift gets flagged.** Poems in a bugfix, refactors nobody asked for, dependency installs unrelated to the task: warned to you at 0.6, held at 0.85 when Jev also calls the action `unrelated`.
|
|
27
|
-
- **Loops get broken.** Three failures with the same strategy (same command with cosmetic changes, same error after each edit) earn the agent a steer: re-read the error, form a new hypothesis, or report the blocker. Exact repeats are caught offline; Jev tells "investigating between failures" from "flailing" — 0.32 vs 0.93 in the smoke run.
|
|
28
|
-
- **"Done" gets checked.** When the final message reports completion after file changes and no test, build, or lint passed in that run, the agent is asked to verify before you read a false "all green". A claim that tests passed when none ran is called out as such. Once per prompt, so it cannot loop.
|
|
29
|
-
- **Slop gets a name.** Written code is scored per symptom on the same request as the action guard (zero extra latency): stubs and fake-data mocks, comments that restate the code, dead or duplicated code, hedging notes. The agent is told which symptom and how to fix it, never held; the third repeat in a session becomes a standing rule. Replies are scored too — wordy, clichéd, or too technical for the configured audience — and a trend across replies earns a nudge that shapes the next one without spending a turn.
|
|
30
|
-
- **Nothing slows down the boring calls.** Read-only shell lines are skipped without a request; anything else costs one request of ~600 input tokens.
|
|
31
|
-
- **It degrades gracefully.** Offline pattern checks and exact-repeat detection run with no account at all. On an API timeout or outage the call is allowed with a warning (configurable), and reasons never include your command text or upstream error bodies.
|
|
27
|
+
Agents pick the next command well and notice badly when that command is out of proportion to the request. Pattern lists catch `rm -rf /` and force pushes; they cannot tell `db:reset` after "reset the database" from `db:reset` after "add a column". A generative model can, but a second LLM call per tool call is slow and expensive. Jev is a System One model: it returns calibrated probabilities instead of text, in a quarter of a second, for a fraction of a cent. That is cheap enough to sit in front of every guarded call. Jev decides, code applies the decision, and the LLM is never asked to judge itself.
|
|
32
28
|
|
|
33
29
|
## Install
|
|
34
30
|
|
|
@@ -36,115 +32,143 @@ Agents are good at picking the next command and bad at noticing when that comman
|
|
|
36
32
|
pi install npm:pi-warden
|
|
37
33
|
```
|
|
38
34
|
|
|
39
|
-
Requires Pi 0.85 or newer and Node.js 22.19 or newer. Pattern checks work with no account. Jev judgments need a TypeSafe API key and are billed to your TypeSafe account (a judgment is roughly 600 input tokens; output is free).
|
|
35
|
+
Requires Pi 0.85 or newer and Node.js 22.19 or newer. Pattern checks work with no account. Jev judgments need a TypeSafe API key and are billed to your TypeSafe account (a judgment is roughly 600 input tokens; a rules request with 8 rules is about 2000; output is free).
|
|
40
36
|
|
|
41
37
|
## Setup
|
|
42
38
|
|
|
43
39
|
1. Get a key at [console.typesafe.ai](https://console.typesafe.ai) (API Keys).
|
|
44
|
-
2. Run `/warden enable`. Read the data notice and confirm
|
|
45
|
-
3.
|
|
40
|
+
2. Run `/warden enable`. Read the data notice and confirm. If no key is stored yet, paste it at the hidden prompt. The key is verified and saved to `~/.pi/agent/pi-typesafe/auth.json` (owner-only, shared with pi-typesafe). Consent is saved to `~/.pi/agent/pi-warden/config.json` and stays on until `/warden disable`.
|
|
41
|
+
3. Optional: `/warden test` shows one synthetic verdict.
|
|
42
|
+
|
|
43
|
+
`TYPESAFE_API_KEY` in the environment takes precedence over the stored key. Headless runs give consent with `PI_WARDEN_ENABLED=1`.
|
|
44
|
+
|
|
45
|
+
## Philosophy
|
|
46
46
|
|
|
47
|
-
|
|
47
|
+
pi-warden is a harness for the agent, not a gate for you. Three jobs:
|
|
48
48
|
|
|
49
|
-
|
|
49
|
+
| Pillar | What it means |
|
|
50
|
+
| --- | --- |
|
|
51
|
+
| **Security** | Holds and offline patterns protect risky operations. Output checks flag injected instructions and credential shapes. Insecure written code gets a targeted steer. Advisory, not a sandbox. |
|
|
52
|
+
| **Deslopify** | Written code is scored per symptom and judged against your project rules. Replies are scored for wordiness, clichés, and jargon. Repeats become a standing rule. |
|
|
53
|
+
| **Context saving** | Only the newest tool result is ever touched, so the prompt cache prefix stays byte-identical. Large output is cut to exact lines and stored; recalls go to a search, not a whole-file read. |
|
|
54
|
+
|
|
55
|
+
## The guards
|
|
56
|
+
|
|
57
|
+
### Action guard
|
|
50
58
|
|
|
51
|
-
|
|
59
|
+
Runs on `tool_call`, before the tool executes.
|
|
52
60
|
|
|
53
|
-
1. **Skip** read-only tools
|
|
54
|
-
2. **
|
|
55
|
-
|
|
61
|
+
1. **Skip** read-only tools and read-only shell lines (`git status && ls`): no request, no widget line.
|
|
62
|
+
2. **Patterns**, offline: force pushes, `git reset --hard`, `git clean`, recursive `rm` on absolute, home, variable, or parent paths, SQL `DROP`/`TRUNCATE`/`DELETE FROM`, block-device writes, `chmod -R 777`, fork bombs, `curl | sh`, `kill -1`, shutdown, package publishing, infrastructure destroys hold the call. `rm -rf` on a project path, `git checkout -- .`, `git branch -D`, `git stash drop`, `find -delete`, `sudo` warn. Reads or writes of `.env`, SSH, AWS, npm, kube, and other credential files warn. A `write` that overwrites a file outside the project holds; creating or editing outside the project warns.
|
|
63
|
+
|
|
64
|
+
Text that is data is not a command. A heredoc body written to a file, a quoted `echo`/`printf` argument, a `grep` pattern, or a `git commit -m` message can mention `git push --force` without a hold. The same text fed to `sh`, `bash -c`, `eval`, `xargs`, or a `python3 - <<EOF` script that calls `os.system` keeps every hit.
|
|
65
|
+
3. **Jev**, with consent: one request with `{ task, context, action }` and four questions. `irreversible` (yes/no), `off_task` (yes/no), `mutates` (does it change anything), `scope` (expected step, plausible side step, unrelated, unclear). Defaults: irreversible at 0.5 warns and at 0.7 holds; off-task at 0.6 warns and at 0.85 with `unrelated` holds, but only when the action can change something. An unrelated `grep` is warned about, never held. Patterns set the floor; Jev can only raise it.
|
|
56
66
|
4. **Act**, by mode:
|
|
57
|
-
- `steer` (default): a hold blocks the call and returns the judgment to the agent as
|
|
58
|
-
- `confirm`: a
|
|
59
|
-
- `advise`: never holds
|
|
67
|
+
- `steer` (default): a hold blocks the call and returns the judgment to the agent as its tool result, with the two acceptable next moves: find a recoverable alternative, or explain the action to you and wait. If your reply approves it, the retry goes through (Jev reads your reply; offline, a yes/go-ahead heuristic does).
|
|
68
|
+
- `confirm`: a `ctx.ui.confirm` dialog. No blocks with a short reason. Falls back to `steer` without a UI.
|
|
69
|
+
- `advise`: never holds, reports only.
|
|
60
70
|
|
|
61
|
-
|
|
71
|
+
The action guard receives your latest message plus up to eight earlier user and assistant messages (750 redacted characters each) so follow-ups and side comments do not replace the task. Sibling tool calls in one assistant message are judged together in one round trip. If TypeSafe cannot answer, the call is allowed with a warning (`failOpen: true`; set it to `false` to hold instead).
|
|
62
72
|
|
|
63
|
-
###
|
|
73
|
+
### Rules
|
|
64
74
|
|
|
65
|
-
|
|
75
|
+
Write your project's rules as Markdown headings in `pi-warden.md` at the project root:
|
|
66
76
|
|
|
67
|
-
|
|
77
|
+
```markdown
|
|
78
|
+
# No console statements
|
|
79
|
+
Code must not contain `console.log` or `console.debug`. Use the logger.
|
|
68
80
|
|
|
69
|
-
|
|
81
|
+
# Exported functions must have explicit return types
|
|
82
|
+
paths: src/**/*.ts
|
|
83
|
+
Every exported function declares its return type.
|
|
70
84
|
|
|
71
|
-
|
|
85
|
+
# TODO comments need a reference
|
|
86
|
+
A `TODO` or `FIXME` must name a ticket, for example `TODO(APP-123): ...`.
|
|
87
|
+
```
|
|
72
88
|
|
|
73
|
-
|
|
89
|
+
Each heading is one rule; the text under it is the specification. A `paths:` line right under the heading limits the rule to matching files (`**` matches any depth, `*` stays within one segment). Content inside code fences is never read as a heading. The highest heading level present in the file delimits rules, so `##` rules under a `#` title work too.
|
|
74
90
|
|
|
75
|
-
|
|
91
|
+
On every `write` and `edit`, pi-warden sends its own request, in parallel with the action guard's, carrying the written content and one question per rule: `compliant`, `violation`, `not_applicable`, or `insufficient_context`. A `write` is sampled at 6000 characters (head, middle, tail); an `edit` sends each new text plus about 40 lines of the current file around the replaced text. With two or more edits, one more question asks which edit contains the violation.
|
|
76
92
|
|
|
77
|
-
|
|
93
|
+
Any rule with P(violation) at or above `rules.threshold` (0.7) is named to the agent: the heading, up to 200 characters of the rule text, and the edit when located. The write goes through; a held write would leave a half-written file. The third hit of one rule in a session says so and asks the agent to treat it as a standing rule. When slop also fires on the same write, both arrive as one message.
|
|
78
94
|
|
|
79
|
-
|
|
80
|
-
2. Any symptom ≥ `slop.threshold` (0.7) trips the note. The write **goes through** — holding it would leave a half-written file.
|
|
81
|
-
3. The agent receives a steer message naming the symptoms and their fixes, e.g. *"pi-warden: the content just written to `src/x.ts` has stub or placeholder code where a working implementation is needed; hedging or vague notes. Fix it in your next edit: replace stubs, placeholders, and hard-coded fake data with the working implementation, or state in your reply exactly what is left unimplemented and why; replace \"should work\", \"for now\", and TODOs without a plan with a definite statement or a concrete follow-up."* The third time a symptom appears in a session, the note says so and asks the agent to treat it as a standing rule.
|
|
95
|
+
Sources, in order: `pi-warden.md` at the root; else the files listed in `rules.files` (all sent in one request); else, with `rules.fallback` (default true), the first of `README.md`, `CLAUDE.md`, `AGENTS.md`, judged as one document with one question. A fallback document is cut to `rules.maxChars` (8000) with every heading and the head of each section kept, so a very long AGENTS.md still fits. Files are re-read when they change; no restart needed. At most 31 rules are asked per request (TypeSafe's cap is 32); the rest are ignored and `/warden status` says how many.
|
|
82
96
|
|
|
83
|
-
|
|
97
|
+
Path scoping in config: `rules.exclude` globs are never sent to Jev (secrets, generated, vendored files); `rules.skip` globs are files the rules do not apply to (tests, docs); a per-rule `paths:` line narrows one rule. `rules.sensitivePaths` maps a glob to a note, for example `"migrations/**": "Tell the user this touches a migration and add a rollback"`; a write or edit under a matching path gives the agent that note once per path, offline, with no request.
|
|
84
98
|
|
|
85
|
-
|
|
99
|
+
From the tuning set (`scripts/rules-cases.mjs`, 8 rules, 13 cases, all as expected): `console.log` in code scores 1.00, a bare TODO 1.00 and a `TODO(QUEUE-41)` 0.00, an empty catch 0.99, a `switch` without `default` 0.96, a hardcoded token 0.88, a missing return type 0.99 with a bad boolean name 0.96 on the same write. A compliant module, a test that mentions `console.log` in a string, and a Markdown doc about `console.log` score nothing. The locator points at the right one of two edits. Not covered: code written through shell heredocs and content past the sample limits.
|
|
86
100
|
|
|
87
|
-
|
|
101
|
+
Ideas borrowed with thanks from [jevrealtimecodecheck](https://github.com/MrDesjardins/jevrealtimecodecheck) (rules as headings, the four outcomes) and [wince](https://github.com/TinyFrontier/wince) (path globs, sensitive paths, judge the change and not its story, the locator question).
|
|
88
102
|
|
|
89
|
-
###
|
|
103
|
+
### Slop
|
|
90
104
|
|
|
91
|
-
|
|
105
|
+
**In code.** When the agent calls `write` or `edit`, four yes/no questions ride on the action guard's request (no extra latency), one per symptom: `slop_stub` (placeholder or fake-data code where a working implementation is needed), `slop_comments` (comments that restate the code), `slop_dead` (commented-out code, unused imports, duplicated logic, unreachable branches), `slop_hedging` ("should work", "for now", TODOs without a plan). Jev sees a 1500-character head/middle/tail sample of a `write` or the first three replacement texts of an `edit`. Any symptom at or above `slop.threshold` (0.7) sends the agent a steer naming the symptom and its fix. The write is never held. The third repeat of a symptom becomes a standing rule.
|
|
92
106
|
|
|
93
|
-
|
|
107
|
+
From the tuning set (`scripts/slop-cases.mjs`): a `// TODO: implement later` stub scores stub 0.99; restating comments 0.97 while an explanatory why-comment scores 0.08; commented-out code scores dead 0.96; a mock inside a test file scores stub 0.51 (below threshold, correctly).
|
|
94
108
|
|
|
95
|
-
|
|
109
|
+
**In replies.** The final reply (200 characters or more) is judged against `slop.prose.audience`: `wordy`, `cliches`, `jargon`. `audience` is `technical` (default), `plain`, or free text such as "a founder without programming background". A symptom must appear in `trend` (2) of the last 3 replies before the agent is nudged; the nudge is queued for your next prompt so it shapes the next reply without spending a turn. A padded reply scores wordy 0.97 / clichés 0.99; a dense three-point summary 0.25 / 0.05.
|
|
110
|
+
|
|
111
|
+
### Security
|
|
112
|
+
|
|
113
|
+
Written code gets a `security_risk` question on the action request: hardcoded credentials, disabled TLS checks, unsafe shell or SQL interpolation, broad permissions, bypassed verification. A threshold crossing (0.7) warns you and steers the agent; it does not block.
|
|
114
|
+
|
|
115
|
+
Tool output from content-bearing tools (`read`, fetch and search tools, named MCP equivalents) is checked for instructions that redirect the assistant or ask for private data; other tools from 2048 characters. Jev receives a redacted 6000-character head/tail sample. A score at or above `security.threshold` wraps the text in an untrusted-data notice and steers the agent. Credential-shape checks work offline and warn not to echo or commit possible secrets. This is advisory, not a sandbox.
|
|
116
|
+
|
|
117
|
+
### Stuck
|
|
118
|
+
|
|
119
|
+
Keeps the last 12 tool results for the current prompt. When the latest result failed and at least 3 failures have accumulated, exact repeats are caught offline (same call, same output with timings and addresses normalised). Otherwise one request judges the sequence: `same_strategy`, `approach_change` (identical / cosmetic / meaningfully different), `progress`. Same strategy at 0.7 counts as stuck: you get a notification and the agent gets a steer asking for a new hypothesis or a blocker report. At most one check per 3 results. In the smoke run, "investigating between failures" scored 0.32 and "flailing" 0.93.
|
|
96
120
|
|
|
97
|
-
|
|
121
|
+
### Runaway
|
|
98
122
|
|
|
99
|
-
|
|
123
|
+
A model that degenerates mid-reply repeats the same lines until the token limit or you press Esc; no tool runs and no turn ends, so nothing else stops it. pi-warden reads the stream as it arrives: per token it appends to a buffer; every 256 characters it counts identical paragraphs (24 characters or more) and checks for one unit repeated back to back at the end. A block repeated `repeats` (4) times in the reply, or `thinkingRepeats` (10) times in thinking, aborts the run. Code only: nothing is sent anywhere. With `recover: true` (default) the agent gets one follow-up turn that names the repeat and asks for the one next step; a second runaway for the same prompt is stopped and left for you. Calibrated on 43,000 local assistant messages: ordinary replies repeat a paragraph twice at most, thinking up to seven times, and the one real runaway repeated its block 28 times. The guard stopped only that one, at half its length.
|
|
100
124
|
|
|
101
|
-
|
|
125
|
+
### Done-check
|
|
102
126
|
|
|
103
|
-
|
|
127
|
+
Tracks each run's evidence: code changes (`write`, `edit`) and check commands (`npm test`, `pytest`, `cargo test`, `tsc`, `eslint`, `go test`, `make test`, and similar) with pass or fail; context-mode's inline `Command exited with code N` counts as a failure. When a run ends with a normal message after code changes and no passing check, one request judges the message: `claims_done`, `claims_verified`, `verification_applies`, `outcome`. A completion claim at 0.7 or above for a task where checks mean something is reported as unverified; a claim that tests passed when none ran is called a false claim. With `nudge: true` the agent gets one follow-up turn asking it to run the checks or say plainly that nothing was verified. Once per prompt.
|
|
104
128
|
|
|
105
|
-
|
|
129
|
+
### Context saver
|
|
106
130
|
|
|
107
|
-
|
|
131
|
+
Only the newest tool result is ever changed, before it enters the session, so the prompt cache prefix and all earlier entries stay as they were.
|
|
108
132
|
|
|
109
|
-
**
|
|
133
|
+
- **Duplicates** (code only). A text result of at least `context.duplicateMinChars` (2000) that is identical to an earlier result of this session becomes a short note naming the earlier tool and the size, plus a recall footer. Re-running the same failing test is the typical case.
|
|
134
|
+
- **Retention and format** (Jev decides, code applies). For a single text block of at least `context.tailMinChars` (12000), Jev picks `all`, `errors_and_summary`, or `summary_only`, and names the format (`vitest_jest`, `node_test`, `tsc`, `eslint`, `pytest`, `git_diff`, `git_log`, `npm_install`, `other`). When a format is confident (0.7) and its markers are present, a parser keeps the exact lines that matter: failing tests with their assertions, compiler and linter errors with file and line, changed files with counts, package notices, the summary line. Otherwise bounded head, diagnostic, and tail excerpts. Nothing is paraphrased. The full output is written to an owner-only temporary file first; the excerpt links to it. If storage fails, the original stays. Multiple text blocks are not compressed.
|
|
135
|
+
- **Recall through search.** The footer names the file and a search command that exists on this machine (probed once: `rg`, `ag`, `ugrep`, `git grep --no-index`, `grep`, `Select-String`, `findstr`). `context.recallTool` pins one or `none`.
|
|
136
|
+
- **Measuring it.** `/warden status` shows how many outputs were candidates, how many were compressed or dropped as duplicates, the bytes removed, the token-turns spared, and the recalls, split into whole-file reads (which give the saving back) and scoped accesses. A recall rate above about 10% means `context.confidence` is too low for your work.
|
|
110
137
|
|
|
111
|
-
|
|
138
|
+
Set `context.enabled: false` to turn it off. Full-output files can contain secrets and stay in the OS temporary directory until removed.
|
|
112
139
|
|
|
113
|
-
|
|
140
|
+
### Desktop notifications
|
|
141
|
+
|
|
142
|
+
A held call the agent will ask you about, a confirm dialog waiting for an answer, and a runaway stop reach the desktop. macOS uses `osascript`; Linux tries `notify-send`, `dunstify`, `gdbus`, `kdialog`, `zenity`, then `powershell.exe` for WSL; Windows shows a toast through PowerShell. Interactive sessions only, one notification per `cooldownMs` (10 s), the reason but never the command. `"notify": { "enabled": false }` turns it off; `"command": ["curl", "-d", "{body}", "https://ntfy.sh/your-topic"]` in the user file replaces the desktop tool with your own relay (no shell; `{title}` and `{body}` are replaced and set as `PI_WARDEN_TITLE` / `PI_WARDEN_BODY`). A project file may switch notifications off but never names a command.
|
|
114
143
|
|
|
115
|
-
###
|
|
144
|
+
### Steer messages
|
|
116
145
|
|
|
117
|
-
|
|
146
|
+
Nudges from the rules, slop, stuck, done, and prose guards are custom messages in the agent's context. By default they are hidden from the transcript (`steerVisible: false`); the notification tells you a nudge happened and the trace panel shows the exact text. When the per-session request budget is spent, pi-warden says so once and continues with offline checks.
|
|
118
147
|
|
|
119
148
|
## Commands
|
|
120
149
|
|
|
121
150
|
| Command | Effect |
|
|
122
151
|
| --- | --- |
|
|
123
|
-
| `/warden status` | Guard state, consent
|
|
124
|
-
| `/warden enable` |
|
|
152
|
+
| `/warden status` | Guard state, consent and key source, session counts, thresholds, rules source, config paths, last verdicts |
|
|
153
|
+
| `/warden enable` | Data notice, key prompt if none is stored, consent saved |
|
|
125
154
|
| `/warden disable` | Stop Jev judgments; pattern checks continue |
|
|
126
|
-
| `/warden mode steer\|confirm\|advise` |
|
|
127
|
-
| `/warden config` | Edit the user config JSON in Pi's editor
|
|
128
|
-
| `/warden test` |
|
|
155
|
+
| `/warden mode steer\|confirm\|advise` | How holds are handled; without an argument, show the current mode |
|
|
156
|
+
| `/warden config` | Edit the user config JSON in Pi's editor |
|
|
157
|
+
| `/warden test` | One synthetic destructive action, its verdict, and what the agent would be told |
|
|
129
158
|
| `/warden trace` | Toggle the trace sidebar (or print the last 20 events without a UI) |
|
|
130
159
|
|
|
131
160
|
## Status line and trace sidebar
|
|
132
161
|
|
|
133
|
-
The line above the editor shows the latest verdict per guard, for example `warden · bash · irreversible 0.84 · off-task 0.86 · unrelated · confirm`.
|
|
162
|
+
The line above the editor shows the latest verdict per guard, for example `warden · bash · irreversible 0.84 · off-task 0.86 · unrelated · confirm`. `/warden trace`, `ctrl+shift+w`, and a click on the line each toggle a right-hand sidebar with the full trace, newest first, live. The sidebar does not take the keyboard; click inside it for arrow keys and PgUp/PgDn, `c` clears, Esc hands input back, `q` closes. `widget.panelWidth` sets its width.
|
|
134
163
|
|
|
135
|
-
|
|
164
|
+
Clicks and the wheel need Pi's fullscreen mode (`tuiMode: "fullscreen"` in `/settings`). In macOS Terminal.app enable View → Allow Mouse Reporting.
|
|
136
165
|
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
Templates in `config.widget` control the text. Segments are separated by ` · `; a segment whose token has no value for that verdict is dropped, so optional information disappears together with its label:
|
|
166
|
+
Templates in `config.widget` control the text. Segments are separated by ` · `; a segment whose token has no value is dropped:
|
|
140
167
|
|
|
141
168
|
```json
|
|
142
169
|
"widget": {
|
|
143
|
-
"enabled": true,
|
|
144
|
-
"placement": "aboveEditor",
|
|
145
|
-
"shortcut": "ctrl+shift+w",
|
|
146
|
-
"panelWidth": "40%",
|
|
147
170
|
"action": "warden · {tool} · irreversible {irreversible} · off-task {offTask} · {scope} · slop: {slop} · patterns: {patterns} · {flags} · {level}",
|
|
171
|
+
"rules": "warden · rules · {tool} {path} · {asked} rules · {violations} · {status}",
|
|
148
172
|
"stuck": "warden · stuck · {failures} failures · same strategy {sameStrategy} · change {approachChange} · progress {progress} · {flags} · {status}",
|
|
149
173
|
"done": "warden · done-check · {changes} changes · {checksPassed}/{checks} checks passed · claims done {claimsDone} · claims verified {claimsVerified} · checks apply {checksApply} · {outcome} · {status}",
|
|
150
174
|
"prose": "warden · prose · wordy {wordy} · clichés {cliches} · jargon {jargon} · {flags} · {status}",
|
|
@@ -154,7 +178,7 @@ Templates in `config.widget` control the text. Segments are separated by ` · `;
|
|
|
154
178
|
}
|
|
155
179
|
```
|
|
156
180
|
|
|
157
|
-
Tokens
|
|
181
|
+
Tokens per guard: action `tool level source irreversible offTask scope approved slop slopStub slopComments slopDead slopHedging patterns reasons path model ms flags time`; rules `tool path asked violations status source reasons model ms flags time`; prose `wordy cliches jargon status reasons model ms flags time`; stuck `failures sameStrategy approachChange progress status source reasons model ms flags time`; done `changes checks checksPassed claimsDone claimsVerified checksApply outcome status reasons model ms flags time`; security `tool injection exfiltration status`; context `tool retention bytesSaved`; runaway `kind count chars signal block status time`. `"enabled": false` hides the line; `"shortcut": ""` disables the keybinding.
|
|
158
182
|
|
|
159
183
|
## Configuration
|
|
160
184
|
|
|
@@ -174,68 +198,101 @@ User file `~/.pi/agent/pi-warden/config.json` (owner-only). Missing keys use the
|
|
|
174
198
|
"irreversible": { "warn": 0.5, "confirm": 0.7 },
|
|
175
199
|
"offTask": { "warn": 0.6, "confirm": 0.85 }
|
|
176
200
|
},
|
|
177
|
-
"
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
201
|
+
"rules": {
|
|
202
|
+
"enabled": true,
|
|
203
|
+
"threshold": 0.7,
|
|
204
|
+
"files": [],
|
|
205
|
+
"fallback": true,
|
|
206
|
+
"maxChars": 8000,
|
|
207
|
+
"exclude": [],
|
|
208
|
+
"skip": [],
|
|
209
|
+
"sensitivePaths": {}
|
|
210
|
+
},
|
|
183
211
|
"slop": {
|
|
184
212
|
"enabled": true,
|
|
185
213
|
"threshold": 0.7,
|
|
186
214
|
"prose": { "enabled": true, "audience": "technical", "threshold": 0.7, "trend": 2, "minChars": 200 }
|
|
187
215
|
},
|
|
216
|
+
"security": { "enabled": true, "threshold": 0.7 },
|
|
217
|
+
"stuck": { "enabled": true, "window": 12, "minFailures": 3, "cooldown": 3, "sameStrategy": 0.7, "nudge": true },
|
|
218
|
+
"done": { "enabled": true, "claimsDone": 0.7, "nudge": true },
|
|
219
|
+
"context": { "enabled": true, "tailMinChars": 12000, "confidence": 0.8, "duplicateMinChars": 2000, "recallTool": "auto", "formatConfidence": 0.7 },
|
|
220
|
+
"runaway": { "enabled": true, "repeats": 4, "thinkingRepeats": 10, "minChars": 400, "recover": true },
|
|
221
|
+
"notify": { "enabled": true, "cooldownMs": 10000, "command": [] },
|
|
188
222
|
"widget": { "enabled": true, "placement": "aboveEditor", "shortcut": "ctrl+shift+w", "panelWidth": "40%" },
|
|
189
223
|
"steerVisible": false
|
|
190
224
|
}
|
|
191
225
|
```
|
|
192
226
|
|
|
193
|
-
A project may add `.pi/pi-warden.json` with `enabled` and per-guard overrides
|
|
227
|
+
A project may add `.pi/pi-warden.json` with `enabled` and per-guard overrides: stricter thresholds, extra guarded tools, `rules.files`, `rules.skip`, `rules.sensitivePaths`, or `"done": { "enabled": false }`. Project files are read only when Pi trusts the project. They can never grant `typesafe` consent, change `mode`, raise `timeoutMs` or `maxRequests`, or set `notify.command`. Environment: `PI_WARDEN_ENABLED=1` (consent), `PI_WARDEN_MODE=steer|confirm|advise`.
|
|
228
|
+
|
|
229
|
+
A wince-style setup for a backend repo:
|
|
230
|
+
|
|
231
|
+
```json
|
|
232
|
+
{
|
|
233
|
+
"rules": {
|
|
234
|
+
"skip": ["tests/**", "**/*.test.*", "docs/**", "**/*.md"],
|
|
235
|
+
"exclude": ["secrets/**", "**/*.pem"],
|
|
236
|
+
"sensitivePaths": {
|
|
237
|
+
"migrations/**": "This touches a migration: tell the user and add a rollback path",
|
|
238
|
+
"**/permissions*": "Access control changed: ask the user for a security review before merging"
|
|
239
|
+
}
|
|
240
|
+
}
|
|
241
|
+
}
|
|
242
|
+
```
|
|
194
243
|
|
|
195
244
|
## Data handling
|
|
196
245
|
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
-
|
|
200
|
-
-
|
|
246
|
+
With consent, requests go to `https://api.typesafe.ai`. What is sent:
|
|
247
|
+
|
|
248
|
+
- **Action guard**: your latest prompt (1500 characters), up to eight earlier user and assistant messages (750 redacted characters each), the tool name, the command (2000 characters) or the file path (relative inside the project, `~`-shortened outside), whether the file exists, a 1500-character head/middle/tail sample of a `write`, the first three edit pairs (400 characters each) of an `edit`.
|
|
249
|
+
- **Rules**: the project-relative path, a 6000-character sample of a `write` or each edit's new text (1500 characters) with about 40 lines of the current file around the replaced text, and the rule text from your rules file or the condensed fallback document (`rules.maxChars`). No task text. Files under `rules.exclude` are never sent.
|
|
250
|
+
- **Stuck**: the last 12 tool calls (300 characters each) with 400-character output tails.
|
|
251
|
+
- **Done-check and prose**: the agent's final message (2000 and 2500 characters), the run's check commands, the audience description.
|
|
252
|
+
- **Output checks**: a redacted head/tail sample up to 6000 characters plus size, line counts, and tool name.
|
|
253
|
+
- **Nothing** for duplicate detection, the runaway guard, sensitive-path notes, or pattern checks.
|
|
201
254
|
|
|
202
|
-
|
|
255
|
+
Obvious credentials (`Authorization` headers, `TOKEN=` and `SECRET=` assignments, `sk-`, `ghp_`, `AKIA`, JWTs, URL passwords, PEM blocks) are replaced with `[redacted]` before sending. Best-effort; do not rely on it for prompts that contain secrets. Text steered to the agent names the tool, the reasons, and the scores, not the command. UI errors never include upstream response bodies or keys. Judgments are model output; thresholds are yours to tune.
|
|
203
256
|
|
|
204
|
-
|
|
257
|
+
## After updating the package
|
|
258
|
+
|
|
259
|
+
Run `/reload` in each open Pi session after an update. If a session reports `Cannot read properties of undefined (reading 'enabled')` while a fresh session works, restart it: the loaded modules do not match the files on disk. Since 0.5.2 the extension checks the shape of the config it receives; a missing section (an extension module and a config module from different versions in one process) switches that guard off and prints one warning naming the sections and the schema numbers. If you see it, restart Pi.
|
|
205
260
|
|
|
206
261
|
## For extension authors
|
|
207
262
|
|
|
208
|
-
|
|
263
|
+
Every guard is a plain function you can call with any object that has pi-typesafe's `evaluate` method as the judge:
|
|
209
264
|
|
|
210
265
|
```ts
|
|
211
|
-
import { evaluateAction, defaultConfig } from "pi-warden";
|
|
266
|
+
import { evaluateAction, evaluateRules, RuleStore, defaultConfig } from "pi-warden";
|
|
212
267
|
import { createTypeSafe } from "pi-typesafe";
|
|
213
268
|
|
|
269
|
+
const judge = createTypeSafe();
|
|
214
270
|
const verdict = await evaluateAction(
|
|
215
271
|
{ tool: "bash", input: { command: "git push --force" }, cwd: process.cwd(), task: "push my branch" },
|
|
216
|
-
{ config: defaultConfig().action, judge
|
|
272
|
+
{ config: defaultConfig().action, judge }, // omit judge for pattern checks only
|
|
217
273
|
);
|
|
218
274
|
verdict.level; // "allow" | "warn" | "confirm" (confirm = hold in steer mode)
|
|
219
275
|
verdict.reasons; // ["destructive: git force push", "irreversible 0.91"]
|
|
220
|
-
verdict.judgment; // { irreversible, offTask, scope, scopeConfidence, approved?, model, elapsedMs }
|
|
221
|
-
steerReason(verdict, { canApprove: true }); // the text the agent receives for a hold
|
|
222
|
-
```
|
|
223
276
|
|
|
224
|
-
|
|
277
|
+
const set = new RuleStore().load(process.cwd(), defaultConfig().rules);
|
|
278
|
+
const rules = await evaluateRules("write", { path: "src/a.ts", content: "console.log(1)" }, { cwd: process.cwd(), config: defaultConfig().rules, set, judge, timeoutMs: 5000 });
|
|
279
|
+
rules.findings; // [{ id: "no-console-statements", name, violation: 0.97, body }]
|
|
280
|
+
```
|
|
225
281
|
|
|
226
|
-
Also exported: `matchPatterns`, `isReadOnlyCommand`, `describeAction`, `redact`, `formatVerdict`, the question sets, the stuck detector
|
|
282
|
+
What spans calls in a session lives in `ActionGuard` (hold, reply, retry approval, sibling batching) and `RulesGuard` (rule cache, sibling prejudging, repeat counts, sensitive-path notes). Also exported: `matchPatterns`, `isReadOnlyCommand`, `stripDataText`, `describeAction`, `parseRules`, `matchGlob`, `redact`, `formatVerdict`, the question sets, the stuck detector, the runaway guard, the notifier, the done-check, and the config helpers.
|
|
227
283
|
|
|
228
284
|
## Development
|
|
229
285
|
|
|
230
286
|
```bash
|
|
231
287
|
npm install
|
|
232
|
-
npm run check
|
|
233
|
-
npm run test:live
|
|
234
|
-
node scripts/
|
|
235
|
-
node scripts/
|
|
236
|
-
|
|
237
|
-
node scripts/
|
|
238
|
-
npm run
|
|
288
|
+
npm run check # typecheck, offline tests (mocked transport), build
|
|
289
|
+
npm run test:live # billable synthetic judgments across the guards; pass action|slop|approval|stuck|done|security|context for one group
|
|
290
|
+
node scripts/rules-cases.mjs # 13 billable cases against an 8-rule fixture file
|
|
291
|
+
node scripts/slop-cases.mjs # 22 billable cases for the slop and prose questions
|
|
292
|
+
node scripts/security-cases.mjs # 9 output-security and task-continuity cases
|
|
293
|
+
node scripts/context-cases.mjs # 15 labelled outputs for retention and format
|
|
294
|
+
npm run dev:pi # start Pi with this working tree plus an installed pi-typesafe
|
|
295
|
+
npm run preview # re-render docs/preview.png (needs a Chrome binary)
|
|
239
296
|
```
|
|
240
297
|
|
|
241
298
|
## License
|
package/dist/config.d.ts
CHANGED
|
@@ -58,6 +58,24 @@ export interface SecurityConfig {
|
|
|
58
58
|
/** P(injection or exfiltration) at or above this adds an untrusted-output notice. */
|
|
59
59
|
threshold: number;
|
|
60
60
|
}
|
|
61
|
+
export interface RulesConfig {
|
|
62
|
+
/** Judge each write and edit against the project's Markdown rules on its own Jev request; steer, never hold. */
|
|
63
|
+
enabled: boolean;
|
|
64
|
+
/** P(violation) at or above this names the rule to the agent. */
|
|
65
|
+
threshold: number;
|
|
66
|
+
/** Project-relative Markdown rule files, used when the root pi-warden.md is absent. All are sent in one request. */
|
|
67
|
+
files: string[];
|
|
68
|
+
/** With no rules file, README.md, CLAUDE.md, or AGENTS.md (first found) is judged as one document. */
|
|
69
|
+
fallback: boolean;
|
|
70
|
+
/** Characters of a fallback document sent per request; every heading and the head of each section are kept within it. */
|
|
71
|
+
maxChars: number;
|
|
72
|
+
/** Globs of files whose content is never sent to Jev for rules (secrets, generated, vendored). */
|
|
73
|
+
exclude: string[];
|
|
74
|
+
/** Globs of files the rules do not apply to, for example tests or docs. */
|
|
75
|
+
skip: string[];
|
|
76
|
+
/** Glob → note. A write or edit under a matching path steers the agent with the note once per path; code only. */
|
|
77
|
+
sensitivePaths: Record<string, string>;
|
|
78
|
+
}
|
|
61
79
|
export interface ContextConfig {
|
|
62
80
|
enabled: boolean;
|
|
63
81
|
/** Only new tool output is compressed; warm history and system prompts are never changed. */
|
|
@@ -116,6 +134,7 @@ export interface WardenConfig {
|
|
|
116
134
|
done: DoneGuardConfig;
|
|
117
135
|
slop: SlopGuardConfig;
|
|
118
136
|
security: SecurityConfig;
|
|
137
|
+
rules: RulesConfig;
|
|
119
138
|
context: ContextConfig;
|
|
120
139
|
runaway: RunawayConfig;
|
|
121
140
|
notify: NotifyConfig;
|
|
@@ -126,7 +145,7 @@ export interface WardenConfig {
|
|
|
126
145
|
}
|
|
127
146
|
export declare const PACKAGE_NAME = "pi-warden";
|
|
128
147
|
/** Bumped when WardenConfig gains a section; extension.ts checks it so a half-updated module graph is reported, not crashed on. */
|
|
129
|
-
export declare const CONFIG_SCHEMA =
|
|
148
|
+
export declare const CONFIG_SCHEMA = 5;
|
|
130
149
|
export declare const PROJECT_CONFIG_FILE = "pi-warden.json";
|
|
131
150
|
export declare function defaultConfig(): WardenConfig;
|
|
132
151
|
/** Mirrors Pi's agent directory rule so the file sits next to pi-typesafe's auth.json. */
|