pi-warden 0.2.1 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +38 -5
- package/dist/config.d.ts +3 -0
- package/dist/config.js +18 -1
- package/dist/done.d.ts +2 -2
- package/dist/done.js +11 -13
- package/dist/extension.js +72 -19
- package/dist/guard.d.ts +2 -2
- package/dist/guard.js +12 -22
- package/dist/index.d.ts +6 -0
- package/dist/index.js +3 -0
- package/dist/panel.d.ts +44 -0
- package/dist/panel.js +121 -0
- package/dist/stuck.d.ts +6 -3
- package/dist/stuck.js +11 -14
- package/dist/tools.d.ts +16 -0
- package/dist/tools.js +37 -0
- package/dist/trace.d.ts +29 -0
- package/dist/trace.js +81 -0
- package/dist/widget.d.ts +35 -0
- package/dist/widget.js +101 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1,11 +1,23 @@
|
|
|
1
1
|
# pi-warden
|
|
2
2
|
|
|
3
|
-
A second pair of eyes for [Pi](https://pi.dev) that makes the agent smarter instead of interrupting you. Before the agent runs a `bash`, `write`, or `edit` call, pi-warden asks [Jev](https://typesafe.ai) two questions about it in ~250 ms: *would this destroy something that cannot be recovered?* and *is this what the user actually asked for?* The answers are probabilities, so `rm -rf dist` after "rebuild from scratch" sails through while `npm run db:reset` after "add a column" is held, and the agent is told why so it re-plans or asks you in chat. Three more guards watch the run: a **stuck-loop** detector, a **done-check** for completion claims that no test backed up, and a **slop** note for stubs and filler. Built on [pi-typesafe](https://github.com/DevMortimer/pi-typesafe).
|
|
3
|
+
A second pair of eyes for [Pi](https://pi.dev) that makes the agent smarter instead of interrupting you. Before the agent runs a `bash`, `write`, or `edit` call (or a shell command through [context-mode](https://www.npmjs.com/package/context-mode)'s `ctx_execute`), pi-warden asks [Jev](https://typesafe.ai) two questions about it in ~250 ms: *would this destroy something that cannot be recovered?* and *is this what the user actually asked for?* The answers are probabilities, so `rm -rf dist` after "rebuild from scratch" sails through while `npm run db:reset` after "add a column" is held, and the agent is told why so it re-plans or asks you in chat. Three more guards watch the run: a **stuck-loop** detector, a **done-check** for completion claims that no test backed up, and a **slop** note for stubs and filler. Built on [pi-typesafe](https://github.com/DevMortimer/pi-typesafe).
|
|
4
4
|
|
|
5
5
|

|
|
6
6
|
|
|
7
7
|
The verdicts above are real output from `npm run test:live`. Independent project; not affiliated with TypeSafe AI or the Pi authors.
|
|
8
8
|
|
|
9
|
+
## Philosophy
|
|
10
|
+
|
|
11
|
+
pi-warden is a harness for the agent, not a gate for the user. It watches every tool call, tool result, and reply, asks Jev small typed questions about them, and acts through the agent's own context — a held call, a steer message, a compressed output — rather than through dialogs. Jev decides; code applies the decision; the LLM is never asked to judge itself. Everything warden does is visible in the trace panel, and nothing is rewritten in your session file.
|
|
12
|
+
|
|
13
|
+
Three jobs, in order of maturity:
|
|
14
|
+
|
|
15
|
+
| Pillar | What it means | Status |
|
|
16
|
+
| --- | --- | --- |
|
|
17
|
+
| **Security** | Irreversible or off-task actions are held with a reason the agent can act on; credential files, force pushes, destructive SQL, and remote-script execution are caught offline; secrets are redacted before anything leaves the machine. Next: injected instructions inside tool output (web pages, issues, files) flagged and quarantined; insecure patterns in written code steered. | built · extending |
|
|
18
|
+
| **Deslopify** | Stubs, placeholders, filler, and restating comments in written code earn a scored nudge; the agent fixes them on the next turn. Next: per-symptom scores instead of one quality number, and wordy or AI-flavoured replies scored against an audience setting. | built for code · prose planned |
|
|
19
|
+
| **Context saving** | Jev judges which tool outputs are worth keeping; code compresses them. Only at moments that cannot cost a prompt-cache miss: when an output arrives (before it enters the cache), when the cache is measurably cold, or when Pi compacts anyway. Works alongside prompt-cache optimizers because it never touches the system prompt or a warm prefix. | planned |
|
|
20
|
+
|
|
9
21
|
## Why a coding agent needs this
|
|
10
22
|
|
|
11
23
|
Agents are good at picking the next command and bad at noticing when that command is out of proportion to the request. Pattern lists catch `rm -rf /` and force pushes; they cannot tell `db:reset` when you asked for a reset from `db:reset` when you asked for a column. A generative model can, but a second LLM call per tool call is slow and expensive. Jev is a System One model: it returns calibrated probabilities instead of text, in a quarter of a second, for a fraction of a cent. That makes it cheap enough to sit in front of **every** guarded call:
|
|
@@ -38,7 +50,7 @@ A `TYPESAFE_API_KEY` environment variable takes precedence over the stored key.
|
|
|
38
50
|
|
|
39
51
|
### Action guard (`tool_call`)
|
|
40
52
|
|
|
41
|
-
1. **Skip** read-only tools, and read-only shell lines such as `git status && ls` (no network, no widget).
|
|
53
|
+
1. **Skip** read-only tools, and read-only shell lines such as `git status && ls` (no network, no widget). Guarded by default: `bash`, `powershell`, `write`, `edit`, and context-mode's `ctx_execute`, `ctx_execute_file`, `ctx_batch_execute` (their `code`/`commands` fields are read as the command; non-shell code such as JavaScript is judged but never skipped as read-only). Add other tools with `action.tools`; unknown tools are judged from their JSON input.
|
|
42
54
|
2. **Pattern pass** (offline): force pushes, `git reset --hard`, `git clean -f`, recursive `rm` on absolute/home/variable/parent paths, SQL `DROP`/`TRUNCATE`/`DELETE FROM`, block-device writes, `chmod -R 777`, fork bombs, `curl | sh`, `kill -1`, shutdown, package publishing, infrastructure destroys → **hold**. `rm -rf` on a project path, `git checkout -- .`, `git branch -D`, `git stash drop`, `find -delete`, `sudo` → **warn**. Reads or writes of `.env`, SSH, AWS, npm, kube, and other credential files → **warn**. A `write` that overwrites an existing file outside the project → **hold**; creating or editing outside the project → **warn**.
|
|
43
55
|
3. **Jev judgment** (with consent): one request with the state `{ task, action }` and three questions — `irreversible` (Noul), `off_task` (Noul), `scope` (Choice: expected step, plausible side step, unrelated, unclear). Defaults: irreversible ≥ 0.5 warns, ≥ 0.7 holds; off-task ≥ 0.6 warns, ≥ 0.85 with `unrelated` holds. Pattern results set the floor; Jev can only raise it. For `write`/`edit`, two slop questions ride on the same request.
|
|
44
56
|
4. **Act**, by mode:
|
|
@@ -54,7 +66,7 @@ Keeps the last 12 tool results for the current prompt. When the latest result fa
|
|
|
54
66
|
|
|
55
67
|
### Done-check (`agent_end`)
|
|
56
68
|
|
|
57
|
-
Tracks each run's evidence: code changes (`write`, `edit`) and check commands (`npm test`, `pytest`, `cargo test`, `tsc`, `eslint`, `go test`, `make test`, and similar) with pass/fail. When a run ends with a normal assistant message after code changes and no passing check, one Jev request judges the message: `claims_done`, `claims_verified`, `verification_applies` (Noul), `outcome` (Choice: complete / partial / blocked / other). A completion claim ≥ 0.7 that is not a blocker or question, for a task where checks would mean something (≥ 0.5; prose and file housekeeping score ~0.05), is reported as unverified; a verification claim with no check run is reported as a false claim. With `nudge: true` (default) the agent receives one follow-up turn asking it to run the checks or say plainly that nothing was verified — once per user prompt.
|
|
69
|
+
Tracks each run's evidence: code changes (`write`, `edit`) and check commands (`npm test`, `pytest`, `cargo test`, `tsc`, `eslint`, `go test`, `make test`, and similar, whether run through `bash` or `ctx_execute`) with pass/fail; context-mode's inline `Command exited with code N` counts as a failure. When a run ends with a normal assistant message after code changes and no passing check, one Jev request judges the message: `claims_done`, `claims_verified`, `verification_applies` (Noul), `outcome` (Choice: complete / partial / blocked / other). A completion claim ≥ 0.7 that is not a blocker or question, for a task where checks would mean something (≥ 0.5; prose and file housekeeping score ~0.05), is reported as unverified; a verification claim with no check run is reported as a false claim. With `nudge: true` (default) the agent receives one follow-up turn asking it to run the checks or say plainly that nothing was verified — once per user prompt.
|
|
58
70
|
|
|
59
71
|
### Slop (`write`/`edit`, same request as the action guard)
|
|
60
72
|
|
|
@@ -78,6 +90,26 @@ If TypeSafe cannot answer (timeout after 5 s, outage, budget), an action-guard c
|
|
|
78
90
|
| `/warden mode steer\|confirm\|advise` | Choose how holds are handled; without an argument, show the current mode |
|
|
79
91
|
| `/warden config` | Edit the user config JSON in Pi's editor and save it |
|
|
80
92
|
| `/warden test` | Evaluate one synthetic destructive action and show the verdict and what the agent would be told |
|
|
93
|
+
| `/warden trace` | Open the trace panel (or print the last 20 events without a UI) |
|
|
94
|
+
|
|
95
|
+
## Status line and trace panel
|
|
96
|
+
|
|
97
|
+
The line above the editor shows the latest verdict per guard, for example `warden · bash · irreversible 0.84 · off-task 0.86 · unrelated · confirm`. Every verdict is also kept in a session trace with what was inspected (redacted command or path), the pattern hits, Jev's scores with model and latency, the reasons, and the exact text the agent was told. Open the panel with `/warden trace`, `ctrl+shift+w`, or by clicking the status line (clicks need Pi's fullscreen mode: `tuiMode: "fullscreen"` in settings, because regular mode leaves the mouse to the terminal). The panel is a right-hand overlay, newest first, live-updating while the agent works; ↑↓/PgUp/PgDn scroll, `c` clears, Esc closes.
|
|
98
|
+
|
|
99
|
+
Templates in `config.widget` control the text. Segments are separated by ` · `; a segment whose token has no value for that verdict is dropped, so optional information disappears together with its label:
|
|
100
|
+
|
|
101
|
+
```json
|
|
102
|
+
"widget": {
|
|
103
|
+
"enabled": true,
|
|
104
|
+
"placement": "aboveEditor",
|
|
105
|
+
"shortcut": "ctrl+shift+w",
|
|
106
|
+
"action": "warden · {tool} · irreversible {irreversible} · off-task {offTask} · {scope} · slop {slopQuality} · stub {slopStub} · patterns: {patterns} · {flags} · {level}",
|
|
107
|
+
"stuck": "warden · stuck · {failures} failures · same strategy {sameStrategy} · change {approachChange} · progress {progress} · {flags} · {status}",
|
|
108
|
+
"done": "warden · done-check · {changes} changes · {checksPassed}/{checks} checks passed · claims done {claimsDone} · claims verified {claimsVerified} · checks apply {checksApply} · {outcome} · {status}"
|
|
109
|
+
}
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
Tokens — action: `tool level source irreversible offTask scope approved slopQuality slopStub patterns reasons path model ms flags time`; stuck: `failures sameStrategy approachChange progress status source reasons model ms flags time`; done: `changes checks checksPassed claimsDone claimsVerified checksApply outcome status reasons model ms flags time`. A minimal line: `"action": "⚔ {tool} {level} · {irreversible}/{offTask}"`. Set `"enabled": false` to hide the line (the trace and panel keep working); `"shortcut": ""` disables the keybinding.
|
|
81
113
|
|
|
82
114
|
## Configuration
|
|
83
115
|
|
|
@@ -92,14 +124,15 @@ User file `~/.pi/agent/pi-warden/config.json` (owner-only). Missing keys use the
|
|
|
92
124
|
"maxRequests": 500,
|
|
93
125
|
"action": {
|
|
94
126
|
"enabled": true,
|
|
95
|
-
"tools": ["bash", "write", "edit"],
|
|
127
|
+
"tools": ["bash", "powershell", "ctx_execute", "ctx_batch_execute", "ctx_execute_file", "write", "edit"],
|
|
96
128
|
"failOpen": true,
|
|
97
129
|
"irreversible": { "warn": 0.5, "confirm": 0.7 },
|
|
98
130
|
"offTask": { "warn": 0.6, "confirm": 0.85 }
|
|
99
131
|
},
|
|
100
132
|
"stuck": { "enabled": true, "window": 12, "minFailures": 3, "cooldown": 3, "sameStrategy": 0.7, "nudge": true },
|
|
101
133
|
"done": { "enabled": true, "claimsDone": 0.7, "nudge": true },
|
|
102
|
-
"slop": { "enabled": true, "quality": 1.5, "placeholder": 0.7 }
|
|
134
|
+
"slop": { "enabled": true, "quality": 1.5, "placeholder": 0.7 },
|
|
135
|
+
"widget": { "enabled": true, "placement": "aboveEditor", "shortcut": "ctrl+shift+w" }
|
|
103
136
|
}
|
|
104
137
|
```
|
|
105
138
|
|
package/dist/config.d.ts
CHANGED
|
@@ -1,3 +1,4 @@
|
|
|
1
|
+
import type { WidgetConfig } from "./widget.js";
|
|
1
2
|
export interface Threshold {
|
|
2
3
|
/** P(yes) at or above this shows a warning and continues. */
|
|
3
4
|
warn: number;
|
|
@@ -62,6 +63,8 @@ export interface WardenConfig {
|
|
|
62
63
|
stuck: StuckGuardConfig;
|
|
63
64
|
done: DoneGuardConfig;
|
|
64
65
|
slop: SlopGuardConfig;
|
|
66
|
+
/** The status line above the editor and the trace panel. */
|
|
67
|
+
widget: WidgetConfig;
|
|
65
68
|
}
|
|
66
69
|
export declare const PACKAGE_NAME = "pi-warden";
|
|
67
70
|
export declare const PROJECT_CONFIG_FILE = "pi-warden.json";
|
package/dist/config.js
CHANGED
|
@@ -1,6 +1,8 @@
|
|
|
1
1
|
import { chmodSync, mkdirSync, readFileSync, renameSync, writeFileSync } from "node:fs";
|
|
2
2
|
import { homedir } from "node:os";
|
|
3
3
|
import { dirname, join } from "node:path";
|
|
4
|
+
import { COMMAND_TOOLS } from "./tools.js";
|
|
5
|
+
import { defaultWidgetConfig } from "./widget.js";
|
|
4
6
|
export const PACKAGE_NAME = "pi-warden";
|
|
5
7
|
export const PROJECT_CONFIG_FILE = `${PACKAGE_NAME}.json`;
|
|
6
8
|
export function defaultConfig() {
|
|
@@ -12,7 +14,7 @@ export function defaultConfig() {
|
|
|
12
14
|
maxRequests: 500,
|
|
13
15
|
action: {
|
|
14
16
|
enabled: true,
|
|
15
|
-
tools: [
|
|
17
|
+
tools: [...COMMAND_TOOLS, "write", "edit"],
|
|
16
18
|
failOpen: true,
|
|
17
19
|
timeoutMs: 5000,
|
|
18
20
|
irreversible: { warn: 0.5, confirm: 0.7 },
|
|
@@ -21,6 +23,7 @@ export function defaultConfig() {
|
|
|
21
23
|
stuck: { enabled: true, window: 12, minFailures: 3, cooldown: 3, sameStrategy: 0.7, nudge: true },
|
|
22
24
|
done: { enabled: true, claimsDone: 0.7, nudge: true },
|
|
23
25
|
slop: { enabled: true, quality: 1.5, placeholder: 0.7 },
|
|
26
|
+
widget: defaultWidgetConfig(),
|
|
24
27
|
};
|
|
25
28
|
}
|
|
26
29
|
/** Mirrors Pi's agent directory rule so the file sits next to pi-typesafe's auth.json. */
|
|
@@ -105,6 +108,19 @@ function applySlop(base, raw) {
|
|
|
105
108
|
return base;
|
|
106
109
|
return { enabled: boolean(raw.enabled, base.enabled), quality: score(raw.quality, base.quality), placeholder: probability(raw.placeholder, base.placeholder) };
|
|
107
110
|
}
|
|
111
|
+
function applyWidget(base, raw) {
|
|
112
|
+
if (!isObject(raw))
|
|
113
|
+
return base;
|
|
114
|
+
const template = (value, fallback) => (typeof value === "string" && value.trim() ? value : fallback);
|
|
115
|
+
return {
|
|
116
|
+
enabled: boolean(raw.enabled, base.enabled),
|
|
117
|
+
placement: raw.placement === "belowEditor" || raw.placement === "aboveEditor" ? raw.placement : base.placement,
|
|
118
|
+
shortcut: typeof raw.shortcut === "string" ? raw.shortcut.trim() : base.shortcut,
|
|
119
|
+
action: template(raw.action, base.action),
|
|
120
|
+
stuck: template(raw.stuck, base.stuck),
|
|
121
|
+
done: template(raw.done, base.done),
|
|
122
|
+
};
|
|
123
|
+
}
|
|
108
124
|
/** Shared request settings; `action.timeoutMs`/`action.maxRequests` from 0.1.x files are still honoured. */
|
|
109
125
|
function applyShared(base, raw) {
|
|
110
126
|
const legacy = isObject(raw.action) ? raw.action : {};
|
|
@@ -132,6 +148,7 @@ export function applyUserOverrides(base, raw) {
|
|
|
132
148
|
mode: isMode(raw.mode) ? raw.mode : base.mode,
|
|
133
149
|
...shared,
|
|
134
150
|
...applyGuards(base, raw, shared.timeoutMs),
|
|
151
|
+
widget: applyWidget(base.widget, raw.widget),
|
|
135
152
|
};
|
|
136
153
|
}
|
|
137
154
|
/** Project files may tune the guards but cannot grant TypeSafe consent, change the mode, or raise budgets. */
|
package/dist/done.d.ts
CHANGED
|
@@ -14,7 +14,7 @@ export interface RunEvidence {
|
|
|
14
14
|
}>;
|
|
15
15
|
}
|
|
16
16
|
export declare function emptyEvidence(): RunEvidence;
|
|
17
|
-
export declare function recordOutcome(evidence: RunEvidence, outcome: ToolOutcome, input: Record<string, unknown
|
|
17
|
+
export declare function recordOutcome(evidence: RunEvidence, outcome: ToolOutcome, input: Record<string, unknown>, tool?: string): void;
|
|
18
18
|
interface MessageLike {
|
|
19
19
|
role: string;
|
|
20
20
|
content?: unknown;
|
|
@@ -82,5 +82,5 @@ export interface DoneOptions {
|
|
|
82
82
|
export declare function evaluateDone(task: string | undefined, finalMessage: string, evidence: RunEvidence, options: DoneOptions): Promise<DoneVerdict>;
|
|
83
83
|
/** Follow-up for the agent: verify or say plainly that nothing was verified. */
|
|
84
84
|
export declare function doneNudge(verdict: DoneVerdict): string;
|
|
85
|
-
export declare function formatDone(verdict: DoneVerdict): string;
|
|
85
|
+
export declare function formatDone(verdict: DoneVerdict, template?: string): string;
|
|
86
86
|
export {};
|
package/dist/done.js
CHANGED
|
@@ -1,6 +1,8 @@
|
|
|
1
1
|
import { choice, noul, TypeSafeIntegrationError } from "pi-typesafe";
|
|
2
2
|
import { isReadOnlyCommand } from "./guard.js";
|
|
3
3
|
import { redact } from "./redact.js";
|
|
4
|
+
import { commandOf } from "./tools.js";
|
|
5
|
+
import { DEFAULT_TEMPLATES, doneTokens, renderTemplate } from "./widget.js";
|
|
4
6
|
/** Commands whose success is evidence that the work was verified. */
|
|
5
7
|
const CHECK_COMMAND = /\b(?:(?:npm|pnpm|yarn|bun)\s+(?:run\s+)?(?:test|check|lint|typecheck|build|verify|ci)\b|(?:npx|pnpm|bunx)\s+(?:tsc|jest|vitest|mocha|eslint|biome|prettier\s+--check)\b|pytest|jest|vitest|mocha|tsc|eslint|biome\s+check|ruff|mypy|flake8|pylint|black\s+--check|cargo\s+(?:test|check|build|clippy)|go\s+(?:test|vet|build)|make\s+(?:test|check|lint|build)|mvn\s+(?:test|verify)|gradle\w*\s+(?:test|check|build)|dotnet\s+(?:test|build)|node\s+--test|deno\s+(?:test|check|lint)|rspec|rake\s+test|mix\s+test|phpunit|swift\s+(?:test|build)|xcodebuild\s+test|ctest|zig\s+(?:test|build))\b/;
|
|
6
8
|
/**
|
|
@@ -12,20 +14,22 @@ export function classifyToolResult(tool, input, failed) {
|
|
|
12
14
|
return "mutation";
|
|
13
15
|
if (tool === "read" || tool === "grep" || tool === "find" || tool === "ls")
|
|
14
16
|
return "read";
|
|
15
|
-
|
|
17
|
+
const view = commandOf(tool, input);
|
|
18
|
+
if (!view)
|
|
16
19
|
return "unknown";
|
|
17
|
-
if (CHECK_COMMAND.test(
|
|
20
|
+
if (CHECK_COMMAND.test(view.command))
|
|
18
21
|
return failed ? "check-fail" : "check-pass";
|
|
19
|
-
return isReadOnlyCommand(
|
|
22
|
+
return view.shell && isReadOnlyCommand(view.command) ? "read" : "unknown";
|
|
20
23
|
}
|
|
21
24
|
export function emptyEvidence() {
|
|
22
25
|
return { mutations: 0, checks: [] };
|
|
23
26
|
}
|
|
24
|
-
export function recordOutcome(evidence, outcome, input) {
|
|
27
|
+
export function recordOutcome(evidence, outcome, input, tool = "bash") {
|
|
25
28
|
if (outcome === "mutation")
|
|
26
29
|
evidence.mutations++;
|
|
27
30
|
if (outcome === "check-pass" || outcome === "check-fail") {
|
|
28
|
-
const
|
|
31
|
+
const command = commandOf(tool, input)?.command;
|
|
32
|
+
const call = command !== undefined ? redact(command.length > 200 ? `${command.slice(0, 200)}…` : command) : "check";
|
|
29
33
|
evidence.checks.push({ call, passed: outcome === "check-pass" });
|
|
30
34
|
}
|
|
31
35
|
}
|
|
@@ -110,12 +114,6 @@ export function doneNudge(verdict) {
|
|
|
110
114
|
const detail = failed.length ? `The last check that ran failed: ${failed.at(-1).call}. Fix that first.` : "Run the project's tests, build, or lint (whatever exists) on what you changed.";
|
|
111
115
|
return `pi-warden: ${verdict.reasons.join("; ")}. ${detail} Then report the actual result. If no check exists or can run, say so explicitly instead of presenting the work as done.`;
|
|
112
116
|
}
|
|
113
|
-
export function formatDone(verdict) {
|
|
114
|
-
|
|
115
|
-
if (verdict.judgment)
|
|
116
|
-
parts.push(`claims done ${verdict.judgment.claimsDone.toFixed(2)}`, `claims verified ${verdict.judgment.claimsVerified.toFixed(2)}`, `checks apply ${verdict.judgment.verificationApplies.toFixed(2)}`, verdict.judgment.outcome);
|
|
117
|
-
if (verdict.error)
|
|
118
|
-
parts.push("typesafe error");
|
|
119
|
-
parts.push(verdict.falseClaim ? "false claim" : verdict.unverified ? "unverified" : "ok");
|
|
120
|
-
return parts.join(" · ");
|
|
117
|
+
export function formatDone(verdict, template = DEFAULT_TEMPLATES.done) {
|
|
118
|
+
return renderTemplate(template, doneTokens(verdict));
|
|
121
119
|
}
|
package/dist/extension.js
CHANGED
|
@@ -1,9 +1,13 @@
|
|
|
1
|
+
import { MouseRegion, Text } from "@earendil-works/pi-tui";
|
|
1
2
|
import { createTypeSafe, resolveApiKey } from "pi-typesafe";
|
|
2
3
|
import { ensureApiKey } from "pi-typesafe/ui";
|
|
3
4
|
import { applyUserOverrides, defaultConfig, isMode, loadConfig, PACKAGE_NAME, projectConfigPath, readUserConfig, setUserSetting, userConfigPath, writeUserConfig } from "./config.js";
|
|
4
5
|
import { classifyToolResult, doneNudge, emptyEvidence, evaluateDone, finalAssistantText, formatDone, needsDoneCheck, recordOutcome } from "./done.js";
|
|
5
6
|
import { evaluateAction, formatVerdict, steerReason, textApproves } from "./guard.js";
|
|
6
7
|
import { AttemptWindow, evaluateStuck, formatStuck, makeAttempt, resultFailed, stuckNudge } from "./stuck.js";
|
|
8
|
+
import { openTracePanel } from "./panel.js";
|
|
9
|
+
import { actionDetails, doneDetails, stuckDetails, Trace } from "./trace.js";
|
|
10
|
+
import { TOKEN_NAMES } from "./widget.js";
|
|
7
11
|
export const disclosure = "With TypeSafe judgments enabled, pi-warden sends to api.typesafe.ai: your latest request plus a redacted, truncated summary of each guarded bash, write, or edit call before it runs; the last few tool calls and output tails when the agent keeps failing; and the agent's final message when it reports completion without running checks. Requests may incur charges. Secret redaction is best-effort. Results are model judgments, not proof or authorization; offline pattern checks stay active either way.";
|
|
8
12
|
const WIDGET = PACKAGE_NAME;
|
|
9
13
|
const CONFIRM_TEXT_LIMIT = 500;
|
|
@@ -51,6 +55,9 @@ export default function wardenExtension(pi) {
|
|
|
51
55
|
let budgetExhausted = false;
|
|
52
56
|
let stats = freshStats();
|
|
53
57
|
const widget = new Map();
|
|
58
|
+
const trace = new Trace();
|
|
59
|
+
let panelOpen = false;
|
|
60
|
+
let lastUi;
|
|
54
61
|
/** Calls held in steer mode, with the user prompt current at that time; a retry after the user replies asks Jev about approval. */
|
|
55
62
|
const held = new Map();
|
|
56
63
|
let attempts = new AttemptWindow(defaultConfig().stuck.window);
|
|
@@ -71,10 +78,33 @@ export default function wardenExtension(pi) {
|
|
|
71
78
|
if (ctx.hasUI)
|
|
72
79
|
ctx.ui.notify(`warden: ${message}${budgetExhausted ? " Pattern checks continue without TypeSafe for the rest of this session." : ""}`, "warning");
|
|
73
80
|
};
|
|
74
|
-
const
|
|
81
|
+
const openPanel = (ui) => {
|
|
82
|
+
if (!ui || panelOpen)
|
|
83
|
+
return;
|
|
84
|
+
panelOpen = true;
|
|
85
|
+
openTracePanel(ui, trace).catch(() => undefined).finally(() => { panelOpen = false; });
|
|
86
|
+
};
|
|
87
|
+
const paint = (ctx, config) => {
|
|
88
|
+
if (!ctx.hasUI)
|
|
89
|
+
return;
|
|
90
|
+
lastUi = ctx.ui;
|
|
91
|
+
if (!config.widget.enabled || widget.size === 0) {
|
|
92
|
+
ctx.ui.setWidget(WIDGET, undefined);
|
|
93
|
+
return;
|
|
94
|
+
}
|
|
95
|
+
const lines = [...widget.values()];
|
|
96
|
+
// A custom component so a click (fullscreen mode) opens the trace panel; plain lines otherwise look the same.
|
|
97
|
+
ctx.ui.setWidget(WIDGET, (_tui, theme) => new MouseRegion(new Text(lines.map(line => theme.fg("muted", line)).join("\n"), 0, 0), event => {
|
|
98
|
+
if (event.type !== "click" || event.button !== "left")
|
|
99
|
+
return undefined;
|
|
100
|
+
openPanel(lastUi);
|
|
101
|
+
return { handled: true };
|
|
102
|
+
}), { placement: config.widget.placement });
|
|
103
|
+
};
|
|
104
|
+
const record = (ctx, config, guard, line, details) => {
|
|
75
105
|
widget.set(guard, line);
|
|
76
|
-
|
|
77
|
-
|
|
106
|
+
trace.push({ at: Date.now(), guard, line, details });
|
|
107
|
+
paint(ctx, config);
|
|
78
108
|
};
|
|
79
109
|
const steer = (content) => pi.sendMessage({ customType: `${PACKAGE_NAME}-steer`, content, display: true }, { deliverAs: "steer" });
|
|
80
110
|
pi.on("session_start", async (_event, ctx) => {
|
|
@@ -82,6 +112,7 @@ export default function wardenExtension(pi) {
|
|
|
82
112
|
budgetExhausted = false;
|
|
83
113
|
stats = freshStats();
|
|
84
114
|
widget.clear();
|
|
115
|
+
trace.clear();
|
|
85
116
|
held.clear();
|
|
86
117
|
attempts.reset();
|
|
87
118
|
evidence = emptyEvidence();
|
|
@@ -125,8 +156,10 @@ export default function wardenExtension(pi) {
|
|
|
125
156
|
stats.approved++;
|
|
126
157
|
held.delete(key);
|
|
127
158
|
}
|
|
159
|
+
const mode = activeMode(config, ctx.hasUI);
|
|
160
|
+
const told = verdict.level === "confirm" && mode === "steer" ? steerReason(verdict, { canApprove: judge !== undefined }) : undefined;
|
|
128
161
|
if (verdict.source !== "read-only")
|
|
129
|
-
|
|
162
|
+
record(ctx, config, "action", formatVerdict(verdict, config.widget.action), actionDetails(verdict, { mode, ...(told ? { told } : {}) }));
|
|
130
163
|
if (verdict.slopReasons?.length) {
|
|
131
164
|
stats.slop++;
|
|
132
165
|
const where = verdict.summary.path ?? event.toolName;
|
|
@@ -142,7 +175,6 @@ export default function wardenExtension(pi) {
|
|
|
142
175
|
}
|
|
143
176
|
if (verdict.level !== "confirm")
|
|
144
177
|
return undefined;
|
|
145
|
-
const mode = activeMode(config, ctx.hasUI);
|
|
146
178
|
const reasons = verdict.reasons.join("; ");
|
|
147
179
|
if (mode === "advise") {
|
|
148
180
|
stats.warned++;
|
|
@@ -161,15 +193,15 @@ export default function wardenExtension(pi) {
|
|
|
161
193
|
held.set(key, task);
|
|
162
194
|
if (ctx.hasUI)
|
|
163
195
|
ctx.ui.notify(`warden · held ${event.toolName}: ${reasons}. The agent was told why and asked to re-plan or ask you.`, "warning");
|
|
164
|
-
return { block: true, reason: steerReason(verdict, { canApprove: judge !== undefined }) };
|
|
196
|
+
return { block: true, reason: told ?? steerReason(verdict, { canApprove: judge !== undefined }) };
|
|
165
197
|
});
|
|
166
198
|
pi.on("tool_result", async (event, ctx) => {
|
|
167
199
|
const config = configFor(ctx);
|
|
168
200
|
if (!config.enabled)
|
|
169
201
|
return;
|
|
170
|
-
const failed = resultFailed(event.isError, event.details);
|
|
202
|
+
const failed = resultFailed(event.isError, event.details, event.content);
|
|
171
203
|
if (config.done.enabled)
|
|
172
|
-
recordOutcome(evidence, classifyToolResult(event.toolName, event.input, failed), event.input);
|
|
204
|
+
recordOutcome(evidence, classifyToolResult(event.toolName, event.input, failed), event.input, event.toolName);
|
|
173
205
|
if (!config.stuck.enabled)
|
|
174
206
|
return;
|
|
175
207
|
attempts.push(makeAttempt(event.toolName, event.input, event.content, failed));
|
|
@@ -181,14 +213,15 @@ export default function wardenExtension(pi) {
|
|
|
181
213
|
noteError(ctx, verdict.error, undefined);
|
|
182
214
|
if (verdict.source === "repeat" && !verdict.stuck)
|
|
183
215
|
return;
|
|
184
|
-
|
|
216
|
+
const nudge = verdict.stuck && config.stuck.nudge ? stuckNudge(verdict) : undefined;
|
|
217
|
+
record(ctx, config, "stuck", formatStuck(verdict, config.widget.stuck), stuckDetails(verdict, attempts.attempts, nudge));
|
|
185
218
|
if (!verdict.stuck)
|
|
186
219
|
return;
|
|
187
220
|
stats.stuck++;
|
|
188
221
|
if (ctx.hasUI)
|
|
189
|
-
ctx.ui.notify(`warden · stuck: ${verdict.reasons.join("; ")}${
|
|
190
|
-
if (
|
|
191
|
-
steer(
|
|
222
|
+
ctx.ui.notify(`warden · stuck: ${verdict.reasons.join("; ")}${nudge ? " (agent nudged)" : ""}`, "warning");
|
|
223
|
+
if (nudge)
|
|
224
|
+
steer(nudge);
|
|
192
225
|
});
|
|
193
226
|
pi.on("agent_end", async (event, ctx) => {
|
|
194
227
|
const config = configFor(ctx);
|
|
@@ -202,20 +235,30 @@ export default function wardenExtension(pi) {
|
|
|
202
235
|
const verdict = await evaluateDone(latestUserPrompt(ctx), finalMessage, evidence, { config: config.done, judge, timeoutMs: config.timeoutMs, signal: ctx.signal });
|
|
203
236
|
if (verdict.error)
|
|
204
237
|
noteError(ctx, verdict.error, undefined);
|
|
205
|
-
|
|
238
|
+
const nudge = verdict.unverified && config.done.nudge ? doneNudge(verdict) : undefined;
|
|
239
|
+
record(ctx, config, "done", formatDone(verdict, config.widget.done), doneDetails(verdict, finalMessage, nudge));
|
|
206
240
|
if (!verdict.unverified)
|
|
207
241
|
return;
|
|
208
242
|
stats.unverified++;
|
|
209
243
|
if (ctx.hasUI)
|
|
210
|
-
ctx.ui.notify(`warden · done-check: ${verdict.reasons.join("; ")}${
|
|
211
|
-
if (
|
|
244
|
+
ctx.ui.notify(`warden · done-check: ${verdict.reasons.join("; ")}${nudge ? " (agent asked to verify)" : ""}`, "warning");
|
|
245
|
+
if (nudge) {
|
|
212
246
|
doneNudged = true;
|
|
213
|
-
pi.sendMessage({ customType: `${PACKAGE_NAME}-steer`, content:
|
|
247
|
+
pi.sendMessage({ customType: `${PACKAGE_NAME}-steer`, content: nudge, display: true }, { deliverAs: "followUp", triggerTurn: true });
|
|
214
248
|
}
|
|
215
249
|
});
|
|
216
|
-
|
|
250
|
+
// Shortcuts are registered once at load; a shape check keeps a typo in the config file from being registered.
|
|
251
|
+
const shortcut = loadConfig().widget.shortcut;
|
|
252
|
+
if (/^(?:(?:ctrl|shift|alt|super)\+)+[a-z0-9]+$|^f\d{1,2}$/i.test(shortcut)) {
|
|
253
|
+
pi.registerShortcut(shortcut, {
|
|
254
|
+
description: "Open the pi-warden trace panel",
|
|
255
|
+
handler: async (ctx) => { if (ctx.hasUI)
|
|
256
|
+
openPanel(ctx.ui); },
|
|
257
|
+
});
|
|
258
|
+
}
|
|
259
|
+
const actions = ["status", "enable", "disable", "mode", "config", "test", "trace"];
|
|
217
260
|
pi.registerCommand("warden", {
|
|
218
|
-
description: "pi-warden status, TypeSafe consent, steer/confirm/advise mode, config editor, and a synthetic guard test",
|
|
261
|
+
description: "pi-warden status, TypeSafe consent, steer/confirm/advise mode, config editor, trace panel, and a synthetic guard test",
|
|
219
262
|
getArgumentCompletions(prefix) {
|
|
220
263
|
const matches = actions.filter(action => action.startsWith(prefix)).map(action => ({ value: action, label: action }));
|
|
221
264
|
return matches.length ? matches : null;
|
|
@@ -241,9 +284,19 @@ export default function wardenExtension(pi) {
|
|
|
241
284
|
`Thresholds: irreversible warn ${config.action.irreversible.warn} / hold ${config.action.irreversible.confirm}; off-task warn ${config.action.offTask.warn} / hold ${config.action.offTask.confirm}; stuck same-strategy ${config.stuck.sameStrategy} after ${config.stuck.minFailures} failures; done claims ${config.done.claimsDone}; slop quality ${config.slop.quality}, stub ${config.slop.placeholder}; failOpen ${config.action.failOpen}.`,
|
|
242
285
|
`Config: ${userConfigPath()}${ctx.isProjectTrusted() ? ` and ${projectConfigPath(ctx.cwd)}` : ""}.`,
|
|
243
286
|
widget.size ? `Last: ${[...widget.values()].join(" | ")}` : "No guarded activity yet this session.",
|
|
287
|
+
`Trace: ${trace.entries().length} events (/warden trace${shortcut ? `, ${shortcut}` : ""}, or click the status line in fullscreen mode). Widget templates in config.widget: action tokens ${TOKEN_NAMES.action.map(name => `{${name}}`).join(" ")}.`,
|
|
244
288
|
].join(" "));
|
|
245
289
|
return;
|
|
246
290
|
}
|
|
291
|
+
if (action === "trace") {
|
|
292
|
+
if (!ctx.hasUI) {
|
|
293
|
+
const entries = trace.entries();
|
|
294
|
+
report(entries.length ? entries.slice(-20).map(entry => `${new Date(entry.at).toTimeString().slice(0, 8)} ${entry.guard}: ${entry.line}${entry.details.length ? `\n ${entry.details.join("\n ")}` : ""}`).join("\n") : "No guarded activity yet this session.");
|
|
295
|
+
return;
|
|
296
|
+
}
|
|
297
|
+
openPanel(ctx.ui);
|
|
298
|
+
return;
|
|
299
|
+
}
|
|
247
300
|
if (action === "enable") {
|
|
248
301
|
if (!ctx.hasUI) {
|
|
249
302
|
report("Consent needs an interactive session. For headless runs set PI_WARDEN_ENABLED=1 and TYPESAFE_API_KEY explicitly.", "warning");
|
|
@@ -310,7 +363,7 @@ export default function wardenExtension(pi) {
|
|
|
310
363
|
if (judge && ctx.hasUI && !await ctx.ui.confirm("Send one synthetic pi-warden test request?", `A synthetic action ("rm -rf /tmp/pi-warden-demo" for the task "Clean up the demo directory") goes to api.typesafe.ai and may incur charges. ${disclosure}`))
|
|
311
364
|
return;
|
|
312
365
|
const verdict = await evaluateAction({ tool: "bash", input: { command: "rm -rf /tmp/pi-warden-demo" }, cwd: ctx.cwd, task: "Clean up the demo directory" }, { config: { ...config.action, enabled: true, tools: ["bash"] }, judge });
|
|
313
|
-
|
|
366
|
+
record(ctx, config, "action", formatVerdict(verdict, config.widget.action), actionDetails(verdict, { mode: activeMode(config, ctx.hasUI) }));
|
|
314
367
|
report(`${formatVerdict(verdict)}${verdict.reasons.length ? ` — ${verdict.reasons.join("; ")}` : ""}${judge ? "" : " (pattern checks only: TypeSafe judgments are not enabled or no key is configured)"}${verdict.error ? ` — ${verdict.error}` : ""}`);
|
|
315
368
|
if (verdict.level === "confirm") {
|
|
316
369
|
const mode = activeMode(config, ctx.hasUI);
|
package/dist/guard.d.ts
CHANGED
|
@@ -129,5 +129,5 @@ export declare function textApproves(task: string | undefined): boolean;
|
|
|
129
129
|
export declare function steerReason(verdict: Verdict, options: {
|
|
130
130
|
canApprove: boolean;
|
|
131
131
|
}): string;
|
|
132
|
-
/** One-line rendering for widgets and logs. Includes no command text. */
|
|
133
|
-
export declare function formatVerdict(verdict: Verdict): string;
|
|
132
|
+
/** One-line rendering for widgets and logs. Includes no command text. Templates: see widget.ts. */
|
|
133
|
+
export declare function formatVerdict(verdict: Verdict, template?: string): string;
|
package/dist/guard.js
CHANGED
|
@@ -3,6 +3,8 @@ import { homedir } from "node:os";
|
|
|
3
3
|
import { isAbsolute, relative, resolve, sep } from "node:path";
|
|
4
4
|
import { choice, noul, score, TypeSafeIntegrationError } from "pi-typesafe";
|
|
5
5
|
import { redact } from "./redact.js";
|
|
6
|
+
import { commandOf } from "./tools.js";
|
|
7
|
+
import { actionTokens, DEFAULT_TEMPLATES, renderTemplate } from "./widget.js";
|
|
6
8
|
const LEVEL_RANK = { allow: 0, warn: 1, confirm: 2 };
|
|
7
9
|
const higher = (a, b) => (LEVEL_RANK[a] >= LEVEL_RANK[b] ? a : b);
|
|
8
10
|
const TASK_LIMIT = 1500;
|
|
@@ -72,7 +74,7 @@ export function matchPatterns(tool, input, cwd) {
|
|
|
72
74
|
const hits = new Map();
|
|
73
75
|
const add = (hit) => { if (hit && !hits.has(hit.id))
|
|
74
76
|
hits.set(hit.id, hit); };
|
|
75
|
-
const command =
|
|
77
|
+
const command = commandOf(tool, input)?.command;
|
|
76
78
|
if (command) {
|
|
77
79
|
for (const rule of SHELL_RULES)
|
|
78
80
|
if (rule.test.test(command))
|
|
@@ -159,9 +161,10 @@ function displayPath(target, cwd) {
|
|
|
159
161
|
}
|
|
160
162
|
export function describeAction(tool, input, cwd) {
|
|
161
163
|
const summary = { tool };
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
164
|
+
const view = commandOf(tool, input);
|
|
165
|
+
if (view)
|
|
166
|
+
summary.command = redact(truncate(view.command, COMMAND_LIMIT));
|
|
167
|
+
if (typeof input.path === "string" && input.path.trim() && tool !== "ctx_execute_file") {
|
|
165
168
|
const shown = displayPath(input.path, cwd);
|
|
166
169
|
summary.path = shown.path;
|
|
167
170
|
summary.location = shown.location;
|
|
@@ -260,7 +263,8 @@ export async function evaluateAction(action, options) {
|
|
|
260
263
|
reasons.push(`${action.tool === "write" ? "creates" : "changes"} a file outside the project`);
|
|
261
264
|
}
|
|
262
265
|
}
|
|
263
|
-
|
|
266
|
+
const view = commandOf(action.tool, action.input);
|
|
267
|
+
if (view?.shell && patterns.length === 0 && isReadOnlyCommand(view.command)) {
|
|
264
268
|
return { level, source: "read-only", summary, patterns, reasons };
|
|
265
269
|
}
|
|
266
270
|
if (!judge)
|
|
@@ -346,21 +350,7 @@ export function steerReason(verdict, options) {
|
|
|
346
350
|
lines.push("pi-warden allows the same call again once the user has replied with approval.");
|
|
347
351
|
return lines.join(" ");
|
|
348
352
|
}
|
|
349
|
-
/** One-line rendering for widgets and logs. Includes no command text. */
|
|
350
|
-
export function formatVerdict(verdict) {
|
|
351
|
-
|
|
352
|
-
if (verdict.judgment)
|
|
353
|
-
parts.push(`irreversible ${percent(verdict.judgment.irreversible)}`, `off-task ${percent(verdict.judgment.offTask)}`, verdict.judgment.scope.replace(/_/g, " "));
|
|
354
|
-
if (verdict.slop)
|
|
355
|
-
parts.push(`slop ${verdict.slop.quality.toFixed(1)}/2`, `stub ${percent(verdict.slop.placeholder)}`);
|
|
356
|
-
if (verdict.patterns.length)
|
|
357
|
-
parts.push(`patterns: ${verdict.patterns.map(hit => hit.id).join(", ")}`);
|
|
358
|
-
if (verdict.approvedByUser)
|
|
359
|
-
parts.push("user approved");
|
|
360
|
-
if (verdict.source === "error")
|
|
361
|
-
parts.push("typesafe error");
|
|
362
|
-
if (verdict.source === "read-only")
|
|
363
|
-
parts.push("read-only");
|
|
364
|
-
parts.push(verdict.level);
|
|
365
|
-
return parts.join(" · ");
|
|
353
|
+
/** One-line rendering for widgets and logs. Includes no command text. Templates: see widget.ts. */
|
|
354
|
+
export function formatVerdict(verdict, template = DEFAULT_TEMPLATES.action) {
|
|
355
|
+
return renderTemplate(template, actionTokens(verdict));
|
|
366
356
|
}
|
package/dist/index.d.ts
CHANGED
|
@@ -7,3 +7,9 @@ export type { ToolOutcome, RunEvidence, DoneJudgment, DoneVerdict, DoneOptions }
|
|
|
7
7
|
export { redact } from "./redact.js";
|
|
8
8
|
export { defaultConfig, loadConfig, applyUserOverrides, applyProjectOverrides, isMode, userConfigPath, projectConfigPath, readUserConfig, writeUserConfig, setUserSetting, PACKAGE_NAME } from "./config.js";
|
|
9
9
|
export type { ActionGuardConfig, StuckGuardConfig, DoneGuardConfig, SlopGuardConfig, Threshold, WardenConfig, WardenMode, LoadOptions } from "./config.js";
|
|
10
|
+
export { commandOf, outputReportsFailure, COMMAND_TOOLS } from "./tools.js";
|
|
11
|
+
export type { CommandView } from "./tools.js";
|
|
12
|
+
export { renderTemplate, actionTokens, stuckTokens, doneTokens, defaultWidgetConfig, DEFAULT_TEMPLATES, TOKEN_NAMES } from "./widget.js";
|
|
13
|
+
export type { WidgetConfig, WidgetPlacement, Tokens } from "./widget.js";
|
|
14
|
+
export { Trace, actionDetails, stuckDetails, doneDetails } from "./trace.js";
|
|
15
|
+
export type { TraceEntry, GuardName } from "./trace.js";
|
package/dist/index.js
CHANGED
|
@@ -3,3 +3,6 @@ export { AttemptWindow, makeAttempt, resultFailed, resultText, evaluateStuck, bu
|
|
|
3
3
|
export { classifyToolResult, recordOutcome, emptyEvidence, needsDoneCheck, finalAssistantText, evaluateDone, buildDoneRequest, doneQuestions, doneNudge, formatDone } from "./done.js";
|
|
4
4
|
export { redact } from "./redact.js";
|
|
5
5
|
export { defaultConfig, loadConfig, applyUserOverrides, applyProjectOverrides, isMode, userConfigPath, projectConfigPath, readUserConfig, writeUserConfig, setUserSetting, PACKAGE_NAME } from "./config.js";
|
|
6
|
+
export { commandOf, outputReportsFailure, COMMAND_TOOLS } from "./tools.js";
|
|
7
|
+
export { renderTemplate, actionTokens, stuckTokens, doneTokens, defaultWidgetConfig, DEFAULT_TEMPLATES, TOKEN_NAMES } from "./widget.js";
|
|
8
|
+
export { Trace, actionDetails, stuckDetails, doneDetails } from "./trace.js";
|
package/dist/panel.d.ts
ADDED
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
import type { Component, TuiMouseEvent, TuiMouseEventResult } from "@earendil-works/pi-tui";
|
|
2
|
+
import type { Trace } from "./trace.js";
|
|
3
|
+
interface ThemeLike {
|
|
4
|
+
fg(color: string, text: string): string;
|
|
5
|
+
bold(text: string): string;
|
|
6
|
+
}
|
|
7
|
+
/**
|
|
8
|
+
* Side panel listing the trace newest-first, live-updating while open. Keys: ↑/↓/j/k scroll a line, PgUp/PgDn a page,
|
|
9
|
+
* Home/End jump, c clears the trace, Esc or q closes. Wheel scrolling works in fullscreen mode.
|
|
10
|
+
*/
|
|
11
|
+
export declare class TracePanel implements Component {
|
|
12
|
+
private readonly trace;
|
|
13
|
+
private readonly theme;
|
|
14
|
+
private readonly close;
|
|
15
|
+
private readonly requestRender;
|
|
16
|
+
private readonly title;
|
|
17
|
+
focused: boolean;
|
|
18
|
+
private scroll;
|
|
19
|
+
private viewport;
|
|
20
|
+
private readonly unsubscribe;
|
|
21
|
+
constructor(trace: Trace, theme: ThemeLike, close: () => void, requestRender: () => void, title?: string);
|
|
22
|
+
dispose(): void;
|
|
23
|
+
/** Rendering is cheap and derived from the trace each time; nothing is cached. */
|
|
24
|
+
invalidate(): void;
|
|
25
|
+
private lines;
|
|
26
|
+
render(width: number): string[];
|
|
27
|
+
/** The overlay tells us how tall we may be through the layout; fall back to a fixed viewport otherwise. */
|
|
28
|
+
setViewport(rows: number): void;
|
|
29
|
+
handleInput(data: string): void;
|
|
30
|
+
handleMouse(event: TuiMouseEvent): TuiMouseEventResult | undefined;
|
|
31
|
+
}
|
|
32
|
+
export interface PanelUi {
|
|
33
|
+
custom<T>(factory: (tui: {
|
|
34
|
+
requestRender(): void;
|
|
35
|
+
terminal?: {
|
|
36
|
+
rows: number;
|
|
37
|
+
};
|
|
38
|
+
}, theme: ThemeLike, keybindings: unknown, done: (result: T) => void) => Component & {
|
|
39
|
+
dispose?(): void;
|
|
40
|
+
}, options?: Record<string, unknown>): Promise<T>;
|
|
41
|
+
}
|
|
42
|
+
/** Open the trace as a right-hand overlay. Resolves when the user closes it. */
|
|
43
|
+
export declare function openTracePanel(ui: PanelUi, trace: Trace): Promise<void>;
|
|
44
|
+
export {};
|
package/dist/panel.js
ADDED
|
@@ -0,0 +1,121 @@
|
|
|
1
|
+
import { Key, matchesKey, truncateToWidth, wrapTextWithAnsi } from "@earendil-works/pi-tui";
|
|
2
|
+
const LEVEL_COLOR = { allow: "success", ok: "success", warn: "warning", unverified: "warning", confirm: "error", stuck: "error", "false claim": "error" };
|
|
3
|
+
/**
|
|
4
|
+
* Side panel listing the trace newest-first, live-updating while open. Keys: ↑/↓/j/k scroll a line, PgUp/PgDn a page,
|
|
5
|
+
* Home/End jump, c clears the trace, Esc or q closes. Wheel scrolling works in fullscreen mode.
|
|
6
|
+
*/
|
|
7
|
+
export class TracePanel {
|
|
8
|
+
trace;
|
|
9
|
+
theme;
|
|
10
|
+
close;
|
|
11
|
+
requestRender;
|
|
12
|
+
title;
|
|
13
|
+
focused = true;
|
|
14
|
+
scroll = 0;
|
|
15
|
+
viewport = 20;
|
|
16
|
+
unsubscribe;
|
|
17
|
+
constructor(trace, theme, close, requestRender, title = "pi-warden trace") {
|
|
18
|
+
this.trace = trace;
|
|
19
|
+
this.theme = theme;
|
|
20
|
+
this.close = close;
|
|
21
|
+
this.requestRender = requestRender;
|
|
22
|
+
this.title = title;
|
|
23
|
+
this.unsubscribe = trace.subscribe(() => this.requestRender());
|
|
24
|
+
}
|
|
25
|
+
dispose() {
|
|
26
|
+
this.unsubscribe();
|
|
27
|
+
}
|
|
28
|
+
/** Rendering is cheap and derived from the trace each time; nothing is cached. */
|
|
29
|
+
invalidate() { }
|
|
30
|
+
lines(width) {
|
|
31
|
+
const { theme } = this;
|
|
32
|
+
const entries = this.trace.entries();
|
|
33
|
+
const out = [];
|
|
34
|
+
out.push(theme.bold(theme.fg("accent", ` ${this.title}`)) + theme.fg("muted", ` · ${entries.length} event${entries.length === 1 ? "" : "s"} · ↑↓ PgUp PgDn scroll · c clear · esc close`));
|
|
35
|
+
out.push(theme.fg("muted", "─".repeat(Math.max(0, width))));
|
|
36
|
+
if (!entries.length) {
|
|
37
|
+
out.push(theme.fg("muted", " No guarded activity yet this session. Verdicts, Jev scores, and what the agent was told will appear here."));
|
|
38
|
+
return out.map(line => truncateToWidth(line, width, ""));
|
|
39
|
+
}
|
|
40
|
+
for (let index = entries.length - 1; index >= 0; index--) {
|
|
41
|
+
const entry = entries[index];
|
|
42
|
+
const status = entry.line.split(" · ").at(-1) ?? "";
|
|
43
|
+
const color = LEVEL_COLOR[status] ?? "text";
|
|
44
|
+
const head = `${theme.fg("muted", new Date(entry.at).toTimeString().slice(0, 8))} ${theme.fg(color, theme.bold(entry.guard.padEnd(6)))} `;
|
|
45
|
+
const headWidth = 9 + 7;
|
|
46
|
+
const body = wrapTextWithAnsi(theme.fg(color, entry.line), Math.max(10, width - headWidth));
|
|
47
|
+
out.push(head + (body[0] ?? ""));
|
|
48
|
+
for (const continuation of body.slice(1))
|
|
49
|
+
out.push(" ".repeat(headWidth) + continuation);
|
|
50
|
+
for (const detail of entry.details) {
|
|
51
|
+
const wrapped = wrapTextWithAnsi(detail, Math.max(10, width - headWidth - 2));
|
|
52
|
+
for (const [n, line] of wrapped.entries())
|
|
53
|
+
out.push(" ".repeat(headWidth) + theme.fg("muted", n === 0 ? "· " : " ") + theme.fg("dim", line));
|
|
54
|
+
}
|
|
55
|
+
out.push("");
|
|
56
|
+
}
|
|
57
|
+
return out.map(line => truncateToWidth(line, width, ""));
|
|
58
|
+
}
|
|
59
|
+
render(width) {
|
|
60
|
+
const all = this.lines(width);
|
|
61
|
+
const maxScroll = Math.max(0, all.length - this.viewport);
|
|
62
|
+
if (this.scroll > maxScroll)
|
|
63
|
+
this.scroll = maxScroll;
|
|
64
|
+
const visible = all.slice(this.scroll, this.scroll + this.viewport);
|
|
65
|
+
if (all.length > this.viewport) {
|
|
66
|
+
const last = visible.length - 1;
|
|
67
|
+
visible[last] = truncateToWidth(theme_fg(this.theme, "muted", ` … ${all.length - this.scroll - this.viewport > 0 ? `${all.length - this.scroll - this.viewport} more lines below` : "end"} · ${this.scroll} above`), width, "");
|
|
68
|
+
}
|
|
69
|
+
return visible;
|
|
70
|
+
}
|
|
71
|
+
/** The overlay tells us how tall we may be through the layout; fall back to a fixed viewport otherwise. */
|
|
72
|
+
setViewport(rows) {
|
|
73
|
+
this.viewport = Math.max(5, rows);
|
|
74
|
+
}
|
|
75
|
+
handleInput(data) {
|
|
76
|
+
if (matchesKey(data, Key.escape) || data === "q" || matchesKey(data, Key.ctrl("c"))) {
|
|
77
|
+
this.close();
|
|
78
|
+
return;
|
|
79
|
+
}
|
|
80
|
+
if (matchesKey(data, Key.up) || data === "k")
|
|
81
|
+
this.scroll = Math.max(0, this.scroll - 1);
|
|
82
|
+
else if (matchesKey(data, Key.down) || data === "j")
|
|
83
|
+
this.scroll += 1;
|
|
84
|
+
else if (matchesKey(data, Key.pageUp))
|
|
85
|
+
this.scroll = Math.max(0, this.scroll - this.viewport);
|
|
86
|
+
else if (matchesKey(data, Key.pageDown))
|
|
87
|
+
this.scroll += this.viewport;
|
|
88
|
+
else if (matchesKey(data, Key.home))
|
|
89
|
+
this.scroll = 0;
|
|
90
|
+
else if (matchesKey(data, Key.end))
|
|
91
|
+
this.scroll = Number.MAX_SAFE_INTEGER;
|
|
92
|
+
else if (data === "c")
|
|
93
|
+
this.trace.clear();
|
|
94
|
+
else
|
|
95
|
+
return;
|
|
96
|
+
this.requestRender();
|
|
97
|
+
}
|
|
98
|
+
handleMouse(event) {
|
|
99
|
+
if (event.type === "wheel") {
|
|
100
|
+
this.scroll = Math.max(0, this.scroll + (event.wheelDelta ?? 0) * 3);
|
|
101
|
+
return { handled: true, render: true };
|
|
102
|
+
}
|
|
103
|
+
return undefined;
|
|
104
|
+
}
|
|
105
|
+
}
|
|
106
|
+
function theme_fg(theme, color, text) {
|
|
107
|
+
return theme.fg(color, text);
|
|
108
|
+
}
|
|
109
|
+
/** Open the trace as a right-hand overlay. Resolves when the user closes it. */
|
|
110
|
+
export function openTracePanel(ui, trace) {
|
|
111
|
+
return ui.custom((tui, theme, _keybindings, done) => {
|
|
112
|
+
const panel = new TracePanel(trace, theme, () => done(), () => tui.requestRender());
|
|
113
|
+
const rows = tui.terminal?.rows;
|
|
114
|
+
if (typeof rows === "number")
|
|
115
|
+
panel.setViewport(Math.floor(rows * 0.9) - 2);
|
|
116
|
+
return panel;
|
|
117
|
+
}, {
|
|
118
|
+
overlay: true,
|
|
119
|
+
overlayOptions: { anchor: "right-center", width: "48%", minWidth: 56, maxHeight: "90%", margin: { right: 1 } },
|
|
120
|
+
});
|
|
121
|
+
}
|
package/dist/stuck.d.ts
CHANGED
|
@@ -32,8 +32,11 @@ export declare function resultText(content: ReadonlyArray<{
|
|
|
32
32
|
type: string;
|
|
33
33
|
text?: string;
|
|
34
34
|
}>): string;
|
|
35
|
-
/** Non-zero exit codes count as failures even when the tool did not flag an error. */
|
|
36
|
-
export declare function resultFailed(isError: boolean, details: unknown
|
|
35
|
+
/** Non-zero exit codes count as failures even when the tool did not flag an error; context-mode reports them in the text. */
|
|
36
|
+
export declare function resultFailed(isError: boolean, details: unknown, content?: ReadonlyArray<{
|
|
37
|
+
type: string;
|
|
38
|
+
text?: string;
|
|
39
|
+
}>): boolean;
|
|
37
40
|
export declare function makeAttempt(tool: string, input: Record<string, unknown>, content: ReadonlyArray<{
|
|
38
41
|
type: string;
|
|
39
42
|
text?: string;
|
|
@@ -85,4 +88,4 @@ export interface StuckOptions {
|
|
|
85
88
|
export declare function evaluateStuck(window: AttemptWindow, task: string | undefined, options: StuckOptions): Promise<StuckVerdict>;
|
|
86
89
|
/** Steering text for the agent. Names the pattern and asks for a change of method, not another retry. */
|
|
87
90
|
export declare function stuckNudge(verdict: StuckVerdict): string;
|
|
88
|
-
export declare function formatStuck(verdict: StuckVerdict): string;
|
|
91
|
+
export declare function formatStuck(verdict: StuckVerdict, template?: string): string;
|
package/dist/stuck.js
CHANGED
|
@@ -1,6 +1,8 @@
|
|
|
1
1
|
import { createHash } from "node:crypto";
|
|
2
2
|
import { noul, score, TypeSafeIntegrationError } from "pi-typesafe";
|
|
3
3
|
import { redact } from "./redact.js";
|
|
4
|
+
import { commandOf, outputReportsFailure } from "./tools.js";
|
|
5
|
+
import { DEFAULT_TEMPLATES, renderTemplate, stuckTokens } from "./widget.js";
|
|
4
6
|
const CALL_LIMIT = 300;
|
|
5
7
|
const OUTPUT_LIMIT = 400;
|
|
6
8
|
function head(text, limit) {
|
|
@@ -13,12 +15,14 @@ function tail(text, limit) {
|
|
|
13
15
|
export function resultText(content) {
|
|
14
16
|
return content.filter(part => part.type === "text" && typeof part.text === "string").map(part => part.text).join("\n");
|
|
15
17
|
}
|
|
16
|
-
/** Non-zero exit codes count as failures even when the tool did not flag an error. */
|
|
17
|
-
export function resultFailed(isError, details) {
|
|
18
|
+
/** Non-zero exit codes count as failures even when the tool did not flag an error; context-mode reports them in the text. */
|
|
19
|
+
export function resultFailed(isError, details, content = []) {
|
|
18
20
|
if (isError)
|
|
19
21
|
return true;
|
|
20
22
|
const exitCode = details && typeof details === "object" ? details.exitCode : undefined;
|
|
21
|
-
|
|
23
|
+
if (typeof exitCode === "number" && exitCode !== 0)
|
|
24
|
+
return true;
|
|
25
|
+
return outputReportsFailure(resultText(content));
|
|
22
26
|
}
|
|
23
27
|
/** Durations, timestamps, PIDs, and addresses change between identical runs; counts and line numbers stay. */
|
|
24
28
|
function normaliseOutput(text) {
|
|
@@ -29,7 +33,8 @@ function normaliseOutput(text) {
|
|
|
29
33
|
.replace(/\d{4}-\d{2}-\d{2}[T ]\d{2}:\d{2}(?::\d{2}(?:\.\d+)?)?Z?/g, "#date");
|
|
30
34
|
}
|
|
31
35
|
export function makeAttempt(tool, input, content, failed) {
|
|
32
|
-
const
|
|
36
|
+
const command = commandOf(tool, input)?.command;
|
|
37
|
+
const call = command !== undefined ? command
|
|
33
38
|
: typeof input.path === "string" ? `${tool} ${input.path}`
|
|
34
39
|
: JSON.stringify(input);
|
|
35
40
|
const text = resultText(content).trim();
|
|
@@ -135,14 +140,6 @@ export async function evaluateStuck(window, task, options) {
|
|
|
135
140
|
export function stuckNudge(verdict) {
|
|
136
141
|
return `pi-warden: ${verdict.reasons.join("; ")}. Stop retrying. Re-read the last error output carefully, state a new hypothesis about the cause, and either gather the missing information (read the relevant file, check versions or paths) or try a different method. If two different methods have failed, report the blocker to the user with the exact error instead of trying again.`;
|
|
137
142
|
}
|
|
138
|
-
export function formatStuck(verdict) {
|
|
139
|
-
|
|
140
|
-
if (verdict.judgment)
|
|
141
|
-
parts.push(`same strategy ${verdict.judgment.sameStrategy.toFixed(2)}`, `change ${verdict.judgment.approachChange.toFixed(1)}/2`, `progress ${verdict.judgment.progress.toFixed(2)}`);
|
|
142
|
-
if (verdict.source === "repeat" && verdict.stuck)
|
|
143
|
-
parts.push("exact repeat");
|
|
144
|
-
if (verdict.source === "error")
|
|
145
|
-
parts.push("typesafe error");
|
|
146
|
-
parts.push(verdict.stuck ? "stuck" : "ok");
|
|
147
|
-
return parts.join(" · ");
|
|
143
|
+
export function formatStuck(verdict, template = DEFAULT_TEMPLATES.stuck) {
|
|
144
|
+
return renderTemplate(template, stuckTokens(verdict));
|
|
148
145
|
}
|
package/dist/tools.d.ts
ADDED
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Knowledge of which tool inputs carry shell commands. Pi's built-ins plus context-mode's ctx_* tools, which many Pi users
|
|
3
|
+
* run instead of bash. Unknown tools return nothing and are judged from their JSON input only.
|
|
4
|
+
*/
|
|
5
|
+
export interface CommandView {
|
|
6
|
+
/** Shell text to pattern-match and judge. Several commands are joined with newlines. */
|
|
7
|
+
command: string;
|
|
8
|
+
/** False when the code is not shell (JavaScript, Python …); pattern rules still run, the read-only shortcut does not. */
|
|
9
|
+
shell: boolean;
|
|
10
|
+
}
|
|
11
|
+
/** Extract the command text a tool call would execute, or undefined for tools that run no commands. */
|
|
12
|
+
export declare function commandOf(tool: string, input: Record<string, unknown>): CommandView | undefined;
|
|
13
|
+
/** Tools whose calls carry commands and should be guarded by default. */
|
|
14
|
+
export declare const COMMAND_TOOLS: string[];
|
|
15
|
+
/** Text that signals failure in tool output when the tool itself did not flag an error (context-mode reports exit codes inline). */
|
|
16
|
+
export declare function outputReportsFailure(text: string): boolean;
|
package/dist/tools.js
ADDED
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Knowledge of which tool inputs carry shell commands. Pi's built-ins plus context-mode's ctx_* tools, which many Pi users
|
|
3
|
+
* run instead of bash. Unknown tools return nothing and are judged from their JSON input only.
|
|
4
|
+
*/
|
|
5
|
+
const SHELL_LANGUAGES = new Set(["shell", "bash", "sh", "zsh", "powershell"]);
|
|
6
|
+
function commandsOf(value) {
|
|
7
|
+
if (!Array.isArray(value))
|
|
8
|
+
return [];
|
|
9
|
+
return value.map(item => (item && typeof item === "object" ? item.command : undefined)).filter((command) => typeof command === "string");
|
|
10
|
+
}
|
|
11
|
+
/** Extract the command text a tool call would execute, or undefined for tools that run no commands. */
|
|
12
|
+
export function commandOf(tool, input) {
|
|
13
|
+
switch (tool) {
|
|
14
|
+
case "bash":
|
|
15
|
+
case "powershell":
|
|
16
|
+
return typeof input.command === "string" ? { command: input.command, shell: true } : undefined;
|
|
17
|
+
case "ctx_execute":
|
|
18
|
+
case "ctx_execute_file": {
|
|
19
|
+
if (typeof input.code !== "string")
|
|
20
|
+
return undefined;
|
|
21
|
+
const language = typeof input.language === "string" ? input.language.toLowerCase() : "";
|
|
22
|
+
return { command: input.code, shell: SHELL_LANGUAGES.has(language) };
|
|
23
|
+
}
|
|
24
|
+
case "ctx_batch_execute": {
|
|
25
|
+
const commands = commandsOf(input.commands);
|
|
26
|
+
return commands.length ? { command: commands.join("\n"), shell: true } : undefined;
|
|
27
|
+
}
|
|
28
|
+
default:
|
|
29
|
+
return undefined;
|
|
30
|
+
}
|
|
31
|
+
}
|
|
32
|
+
/** Tools whose calls carry commands and should be guarded by default. */
|
|
33
|
+
export const COMMAND_TOOLS = ["bash", "powershell", "ctx_execute", "ctx_batch_execute", "ctx_execute_file"];
|
|
34
|
+
/** Text that signals failure in tool output when the tool itself did not flag an error (context-mode reports exit codes inline). */
|
|
35
|
+
export function outputReportsFailure(text) {
|
|
36
|
+
return /(?:^|\n)\s*Command exited with code [1-9]\d*\b/.test(text) || /(?:^|\n)\s*\(timed out\)\s*$/.test(text);
|
|
37
|
+
}
|
package/dist/trace.d.ts
ADDED
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
import type { DoneVerdict } from "./done.js";
|
|
2
|
+
import type { Verdict } from "./guard.js";
|
|
3
|
+
import type { Attempt, StuckVerdict } from "./stuck.js";
|
|
4
|
+
export type GuardName = "action" | "stuck" | "done";
|
|
5
|
+
export interface TraceEntry {
|
|
6
|
+
at: number;
|
|
7
|
+
guard: GuardName;
|
|
8
|
+
/** The widget line for this event. */
|
|
9
|
+
line: string;
|
|
10
|
+
/** Redacted details: what was inspected, what Jev answered, what the agent was told. */
|
|
11
|
+
details: string[];
|
|
12
|
+
}
|
|
13
|
+
/** Session memory of guard decisions; the widget shows the latest line per guard, the panel shows the history. */
|
|
14
|
+
export declare class Trace {
|
|
15
|
+
private readonly limit;
|
|
16
|
+
private readonly items;
|
|
17
|
+
private readonly listeners;
|
|
18
|
+
constructor(limit?: number);
|
|
19
|
+
push(entry: TraceEntry): void;
|
|
20
|
+
entries(): readonly TraceEntry[];
|
|
21
|
+
clear(): void;
|
|
22
|
+
subscribe(listener: () => void): () => void;
|
|
23
|
+
}
|
|
24
|
+
export declare function actionDetails(verdict: Verdict, extra?: {
|
|
25
|
+
mode?: string;
|
|
26
|
+
told?: string;
|
|
27
|
+
}): string[];
|
|
28
|
+
export declare function stuckDetails(verdict: StuckVerdict, attempts: readonly Attempt[], told?: string): string[];
|
|
29
|
+
export declare function doneDetails(verdict: DoneVerdict, finalMessage: string, told?: string): string[];
|
package/dist/trace.js
ADDED
|
@@ -0,0 +1,81 @@
|
|
|
1
|
+
/** Session memory of guard decisions; the widget shows the latest line per guard, the panel shows the history. */
|
|
2
|
+
export class Trace {
|
|
3
|
+
limit;
|
|
4
|
+
items = [];
|
|
5
|
+
listeners = new Set();
|
|
6
|
+
constructor(limit = 100) {
|
|
7
|
+
this.limit = limit;
|
|
8
|
+
}
|
|
9
|
+
push(entry) {
|
|
10
|
+
this.items.push(entry);
|
|
11
|
+
if (this.items.length > this.limit)
|
|
12
|
+
this.items.splice(0, this.items.length - this.limit);
|
|
13
|
+
for (const listener of this.listeners)
|
|
14
|
+
listener();
|
|
15
|
+
}
|
|
16
|
+
entries() {
|
|
17
|
+
return this.items;
|
|
18
|
+
}
|
|
19
|
+
clear() {
|
|
20
|
+
this.items.length = 0;
|
|
21
|
+
for (const listener of this.listeners)
|
|
22
|
+
listener();
|
|
23
|
+
}
|
|
24
|
+
subscribe(listener) {
|
|
25
|
+
this.listeners.add(listener);
|
|
26
|
+
return () => { this.listeners.delete(listener); };
|
|
27
|
+
}
|
|
28
|
+
}
|
|
29
|
+
const clip = (text, limit) => (text.length <= limit ? text : `${text.slice(0, limit)}…`);
|
|
30
|
+
const percent = (value) => value.toFixed(2);
|
|
31
|
+
export function actionDetails(verdict, extra = {}) {
|
|
32
|
+
const { summary, judgment } = verdict;
|
|
33
|
+
const lines = [];
|
|
34
|
+
if (summary.command !== undefined)
|
|
35
|
+
lines.push(`ran: ${clip(summary.command.replace(/\s+/g, " "), 300)}`);
|
|
36
|
+
if (summary.path !== undefined)
|
|
37
|
+
lines.push(`${summary.tool} ${summary.path}${summary.location === "outside_project" ? " (outside the project)" : ""}${summary.exists === false ? " (new file)" : ""}${summary.bytes !== undefined ? ` · ${summary.bytes} bytes` : ""}${summary.editCount !== undefined ? ` · ${summary.editCount} edit${summary.editCount === 1 ? "" : "s"}` : ""}`);
|
|
38
|
+
if (summary.input !== undefined)
|
|
39
|
+
lines.push(`input: ${clip(summary.input, 300)}`);
|
|
40
|
+
if (verdict.patterns.length)
|
|
41
|
+
lines.push(`patterns: ${verdict.patterns.map(hit => `${hit.id} (${hit.severity})`).join(", ")}`);
|
|
42
|
+
if (judgment) {
|
|
43
|
+
lines.push(`jev: irreversible ${percent(judgment.irreversible)} · off-task ${percent(judgment.offTask)} · ${judgment.scope.replace(/_/g, " ")} (${percent(judgment.scopeConfidence)})${judgment.approved !== undefined ? ` · approved ${percent(judgment.approved)}` : ""} · ${judgment.model} · ${judgment.elapsedMs} ms`);
|
|
44
|
+
}
|
|
45
|
+
if (verdict.slop)
|
|
46
|
+
lines.push(`slop: quality ${verdict.slop.quality.toFixed(2)}/2 · placeholder ${percent(verdict.slop.placeholder)}${verdict.slopReasons?.length ? ` → ${verdict.slopReasons.join("; ")}` : ""}`);
|
|
47
|
+
if (verdict.reasons.length)
|
|
48
|
+
lines.push(`why: ${verdict.reasons.join("; ")}`);
|
|
49
|
+
if (verdict.error)
|
|
50
|
+
lines.push(`typesafe: ${verdict.error}`);
|
|
51
|
+
if (extra.mode && verdict.level === "confirm")
|
|
52
|
+
lines.push(`mode: ${extra.mode}`);
|
|
53
|
+
if (extra.told)
|
|
54
|
+
lines.push(`agent told: ${clip(extra.told, 400)}`);
|
|
55
|
+
return lines;
|
|
56
|
+
}
|
|
57
|
+
export function stuckDetails(verdict, attempts, told) {
|
|
58
|
+
const lines = attempts.map((attempt, index) => `${index + 1}. ${attempt.failed ? "✗" : "✓"} ${clip(attempt.call.replace(/\s+/g, " "), 120)}${attempt.failed && attempt.output ? ` → ${clip(attempt.output.replace(/\s+/g, " "), 120)}` : ""}`);
|
|
59
|
+
if (verdict.judgment)
|
|
60
|
+
lines.push(`jev: same strategy ${percent(verdict.judgment.sameStrategy)} · approach change ${verdict.judgment.approachChange.toFixed(2)}/2 · progress ${percent(verdict.judgment.progress)} · ${verdict.judgment.model} · ${verdict.judgment.elapsedMs} ms`);
|
|
61
|
+
if (verdict.reasons.length)
|
|
62
|
+
lines.push(`why: ${verdict.reasons.join("; ")}`);
|
|
63
|
+
if (verdict.error)
|
|
64
|
+
lines.push(`typesafe: ${verdict.error}`);
|
|
65
|
+
if (told)
|
|
66
|
+
lines.push(`agent told: ${clip(told, 400)}`);
|
|
67
|
+
return lines;
|
|
68
|
+
}
|
|
69
|
+
export function doneDetails(verdict, finalMessage, told) {
|
|
70
|
+
const lines = [`final message: ${clip(finalMessage.replace(/\s+/g, " "), 300)}`];
|
|
71
|
+
lines.push(`evidence: ${verdict.evidence.mutations} code change${verdict.evidence.mutations === 1 ? "" : "s"}${verdict.evidence.checks.length ? `; checks: ${verdict.evidence.checks.map(check => `${clip(check.call, 60)} → ${check.passed ? "passed" : "failed"}`).join(", ")}` : "; no checks ran"}`);
|
|
72
|
+
if (verdict.judgment)
|
|
73
|
+
lines.push(`jev: claims done ${percent(verdict.judgment.claimsDone)} · claims verified ${percent(verdict.judgment.claimsVerified)} · checks apply ${percent(verdict.judgment.verificationApplies)} · ${verdict.judgment.outcome} · ${verdict.judgment.model} · ${verdict.judgment.elapsedMs} ms`);
|
|
74
|
+
if (verdict.reasons.length)
|
|
75
|
+
lines.push(`why: ${verdict.reasons.join("; ")}`);
|
|
76
|
+
if (verdict.error)
|
|
77
|
+
lines.push(`typesafe: ${verdict.error}`);
|
|
78
|
+
if (told)
|
|
79
|
+
lines.push(`agent told: ${clip(told, 400)}`);
|
|
80
|
+
return lines;
|
|
81
|
+
}
|
package/dist/widget.d.ts
ADDED
|
@@ -0,0 +1,35 @@
|
|
|
1
|
+
import type { DoneVerdict } from "./done.js";
|
|
2
|
+
import type { Verdict } from "./guard.js";
|
|
3
|
+
import type { StuckVerdict } from "./stuck.js";
|
|
4
|
+
export type WidgetPlacement = "aboveEditor" | "belowEditor";
|
|
5
|
+
export interface WidgetConfig {
|
|
6
|
+
enabled: boolean;
|
|
7
|
+
placement: WidgetPlacement;
|
|
8
|
+
/** Keyboard shortcut that opens the trace panel; empty string disables it. */
|
|
9
|
+
shortcut: string;
|
|
10
|
+
/** Templates per guard. Segments are separated by " · "; a segment whose token has no value is dropped. */
|
|
11
|
+
action: string;
|
|
12
|
+
stuck: string;
|
|
13
|
+
done: string;
|
|
14
|
+
}
|
|
15
|
+
export declare const DEFAULT_TEMPLATES: {
|
|
16
|
+
readonly action: "warden · {tool} · irreversible {irreversible} · off-task {offTask} · {scope} · slop {slopQuality} · stub {slopStub} · patterns: {patterns} · {flags} · {level}";
|
|
17
|
+
readonly stuck: "warden · stuck · {failures} failures · same strategy {sameStrategy} · change {approachChange} · progress {progress} · {flags} · {status}";
|
|
18
|
+
readonly done: "warden · done-check · {changes} changes · {checksPassed}/{checks} checks passed · claims done {claimsDone} · claims verified {claimsVerified} · checks apply {checksApply} · {outcome} · {status}";
|
|
19
|
+
};
|
|
20
|
+
export declare function defaultWidgetConfig(): WidgetConfig;
|
|
21
|
+
export type Tokens = Record<string, string | undefined>;
|
|
22
|
+
/**
|
|
23
|
+
* Fill `{token}` placeholders. The template is split on " · "; a segment is dropped when any of its tokens is empty,
|
|
24
|
+
* so optional information disappears together with its label. Unknown tokens render as empty (and drop their segment).
|
|
25
|
+
*/
|
|
26
|
+
export declare function renderTemplate(template: string, tokens: Tokens): string;
|
|
27
|
+
export declare function actionTokens(verdict: Verdict, at?: number): Tokens;
|
|
28
|
+
export declare function stuckTokens(verdict: StuckVerdict, at?: number): Tokens;
|
|
29
|
+
export declare function doneTokens(verdict: DoneVerdict, at?: number): Tokens;
|
|
30
|
+
/** Token names users can put in templates, for /warden status and the README. */
|
|
31
|
+
export declare const TOKEN_NAMES: {
|
|
32
|
+
readonly action: readonly ["tool", "level", "source", "irreversible", "offTask", "scope", "approved", "slopQuality", "slopStub", "patterns", "reasons", "path", "model", "ms", "flags", "time", "guard"];
|
|
33
|
+
readonly stuck: readonly ["failures", "sameStrategy", "approachChange", "progress", "status", "source", "reasons", "model", "ms", "flags", "time", "guard"];
|
|
34
|
+
readonly done: readonly ["changes", "checks", "checksPassed", "claimsDone", "claimsVerified", "checksApply", "outcome", "status", "reasons", "model", "ms", "flags", "time", "guard"];
|
|
35
|
+
};
|
package/dist/widget.js
ADDED
|
@@ -0,0 +1,101 @@
|
|
|
1
|
+
export const DEFAULT_TEMPLATES = {
|
|
2
|
+
action: "warden · {tool} · irreversible {irreversible} · off-task {offTask} · {scope} · slop {slopQuality} · stub {slopStub} · patterns: {patterns} · {flags} · {level}",
|
|
3
|
+
stuck: "warden · stuck · {failures} failures · same strategy {sameStrategy} · change {approachChange} · progress {progress} · {flags} · {status}",
|
|
4
|
+
done: "warden · done-check · {changes} changes · {checksPassed}/{checks} checks passed · claims done {claimsDone} · claims verified {claimsVerified} · checks apply {checksApply} · {outcome} · {status}",
|
|
5
|
+
};
|
|
6
|
+
export function defaultWidgetConfig() {
|
|
7
|
+
return { enabled: true, placement: "aboveEditor", shortcut: "ctrl+shift+w", ...DEFAULT_TEMPLATES };
|
|
8
|
+
}
|
|
9
|
+
const SEPARATOR = " · ";
|
|
10
|
+
const TOKEN = /\{([a-zA-Z]+)\}/g;
|
|
11
|
+
/**
|
|
12
|
+
* Fill `{token}` placeholders. The template is split on " · "; a segment is dropped when any of its tokens is empty,
|
|
13
|
+
* so optional information disappears together with its label. Unknown tokens render as empty (and drop their segment).
|
|
14
|
+
*/
|
|
15
|
+
export function renderTemplate(template, tokens) {
|
|
16
|
+
const segments = [];
|
|
17
|
+
for (const segment of template.split(SEPARATOR)) {
|
|
18
|
+
let missing = false;
|
|
19
|
+
const rendered = segment.replace(TOKEN, (_match, name) => {
|
|
20
|
+
const value = tokens[name];
|
|
21
|
+
if (value === undefined || value === "")
|
|
22
|
+
missing = true;
|
|
23
|
+
return value ?? "";
|
|
24
|
+
});
|
|
25
|
+
if (!missing && rendered.trim())
|
|
26
|
+
segments.push(rendered.trim());
|
|
27
|
+
}
|
|
28
|
+
return segments.join(SEPARATOR);
|
|
29
|
+
}
|
|
30
|
+
const fixed = (value, digits = 2) => (value === undefined ? undefined : value.toFixed(digits));
|
|
31
|
+
const time = (at) => new Date(at).toTimeString().slice(0, 8);
|
|
32
|
+
export function actionTokens(verdict, at = Date.now()) {
|
|
33
|
+
const flags = [
|
|
34
|
+
verdict.approvedByUser ? "user approved" : undefined,
|
|
35
|
+
verdict.source === "error" ? "typesafe error" : undefined,
|
|
36
|
+
verdict.source === "read-only" ? "read-only" : undefined,
|
|
37
|
+
].filter(Boolean).join(", ");
|
|
38
|
+
return {
|
|
39
|
+
guard: "action",
|
|
40
|
+
time: time(at),
|
|
41
|
+
tool: verdict.summary.tool,
|
|
42
|
+
level: verdict.level,
|
|
43
|
+
source: verdict.source,
|
|
44
|
+
irreversible: fixed(verdict.judgment?.irreversible),
|
|
45
|
+
offTask: fixed(verdict.judgment?.offTask),
|
|
46
|
+
scope: verdict.judgment?.scope.replace(/_/g, " "),
|
|
47
|
+
approved: fixed(verdict.judgment?.approved),
|
|
48
|
+
slopQuality: verdict.slop ? `${verdict.slop.quality.toFixed(1)}/2` : undefined,
|
|
49
|
+
slopStub: fixed(verdict.slop?.placeholder),
|
|
50
|
+
patterns: verdict.patterns.length ? verdict.patterns.map(hit => hit.id).join(", ") : undefined,
|
|
51
|
+
reasons: verdict.reasons.length ? verdict.reasons.join("; ") : undefined,
|
|
52
|
+
path: verdict.summary.path,
|
|
53
|
+
model: verdict.judgment?.model,
|
|
54
|
+
ms: verdict.judgment ? String(verdict.judgment.elapsedMs) : undefined,
|
|
55
|
+
flags: flags || undefined,
|
|
56
|
+
};
|
|
57
|
+
}
|
|
58
|
+
export function stuckTokens(verdict, at = Date.now()) {
|
|
59
|
+
const flags = [
|
|
60
|
+
verdict.source === "repeat" && verdict.stuck ? "exact repeat" : undefined,
|
|
61
|
+
verdict.source === "error" ? "typesafe error" : undefined,
|
|
62
|
+
].filter(Boolean).join(", ");
|
|
63
|
+
return {
|
|
64
|
+
guard: "stuck",
|
|
65
|
+
time: time(at),
|
|
66
|
+
failures: String(verdict.failures),
|
|
67
|
+
sameStrategy: fixed(verdict.judgment?.sameStrategy),
|
|
68
|
+
approachChange: verdict.judgment ? `${verdict.judgment.approachChange.toFixed(1)}/2` : undefined,
|
|
69
|
+
progress: fixed(verdict.judgment?.progress),
|
|
70
|
+
status: verdict.stuck ? "stuck" : "ok",
|
|
71
|
+
source: verdict.source,
|
|
72
|
+
reasons: verdict.reasons.length ? verdict.reasons.join("; ") : undefined,
|
|
73
|
+
model: verdict.judgment?.model,
|
|
74
|
+
ms: verdict.judgment ? String(verdict.judgment.elapsedMs) : undefined,
|
|
75
|
+
flags: flags || undefined,
|
|
76
|
+
};
|
|
77
|
+
}
|
|
78
|
+
export function doneTokens(verdict, at = Date.now()) {
|
|
79
|
+
return {
|
|
80
|
+
guard: "done",
|
|
81
|
+
time: time(at),
|
|
82
|
+
changes: String(verdict.evidence.mutations),
|
|
83
|
+
checks: String(verdict.evidence.checks.length),
|
|
84
|
+
checksPassed: String(verdict.evidence.checks.filter(check => check.passed).length),
|
|
85
|
+
claimsDone: fixed(verdict.judgment?.claimsDone),
|
|
86
|
+
claimsVerified: fixed(verdict.judgment?.claimsVerified),
|
|
87
|
+
checksApply: fixed(verdict.judgment?.verificationApplies),
|
|
88
|
+
outcome: verdict.judgment?.outcome,
|
|
89
|
+
status: verdict.falseClaim ? "false claim" : verdict.unverified ? "unverified" : "ok",
|
|
90
|
+
reasons: verdict.reasons.length ? verdict.reasons.join("; ") : undefined,
|
|
91
|
+
model: verdict.judgment?.model,
|
|
92
|
+
ms: verdict.judgment ? String(verdict.judgment.elapsedMs) : undefined,
|
|
93
|
+
flags: verdict.error ? "typesafe error" : undefined,
|
|
94
|
+
};
|
|
95
|
+
}
|
|
96
|
+
/** Token names users can put in templates, for /warden status and the README. */
|
|
97
|
+
export const TOKEN_NAMES = {
|
|
98
|
+
action: ["tool", "level", "source", "irreversible", "offTask", "scope", "approved", "slopQuality", "slopStub", "patterns", "reasons", "path", "model", "ms", "flags", "time", "guard"],
|
|
99
|
+
stuck: ["failures", "sameStrategy", "approachChange", "progress", "status", "source", "reasons", "model", "ms", "flags", "time", "guard"],
|
|
100
|
+
done: ["changes", "checks", "checksPassed", "claimsDone", "claimsVerified", "checksApply", "outcome", "status", "reasons", "model", "ms", "flags", "time", "guard"],
|
|
101
|
+
};
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pi-warden",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.3.0",
|
|
4
4
|
"description": "Guardrails for Pi built on pi-typesafe that steer the agent instead of interrupting you: Jev judges irreversible and off-task tool calls before they run, detects stuck loops, checks unverified done claims, and flags slop.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "MIT",
|