pi-warden 0.3.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +29 -14
- package/dist/config.d.ts +16 -4
- package/dist/config.js +17 -5
- package/dist/extension.d.ts +3 -1
- package/dist/extension.js +57 -13
- package/dist/guard.d.ts +18 -9
- package/dist/guard.js +41 -16
- package/dist/index.d.ts +7 -4
- package/dist/index.js +4 -2
- package/dist/panel.js +1 -1
- package/dist/prose.d.ts +57 -0
- package/dist/prose.js +98 -0
- package/dist/trace.d.ts +3 -1
- package/dist/trace.js +13 -1
- package/dist/widget.d.ts +7 -2
- package/dist/widget.js +23 -4
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -15,7 +15,7 @@ Three jobs, in order of maturity:
|
|
|
15
15
|
| Pillar | What it means | Status |
|
|
16
16
|
| --- | --- | --- |
|
|
17
17
|
| **Security** | Irreversible or off-task actions are held with a reason the agent can act on; credential files, force pushes, destructive SQL, and remote-script execution are caught offline; secrets are redacted before anything leaves the machine. Next: injected instructions inside tool output (web pages, issues, files) flagged and quarantined; insecure patterns in written code steered. | built · extending |
|
|
18
|
-
| **Deslopify** |
|
|
18
|
+
| **Deslopify** | Written code is scored per symptom — stubs, restating comments, dead code, hedging — and the agent gets a nudge naming the symptom and its fix; repeats become a standing rule. Replies are scored for wordiness, assistant clichés, and jargon against an `audience` setting, and a trend across replies shapes the next one. | built |
|
|
19
19
|
| **Context saving** | Jev judges which tool outputs are worth keeping; code compresses them. Only at moments that cannot cost a prompt-cache miss: when an output arrives (before it enters the cache), when the cache is measurably cold, or when Pi compacts anyway. Works alongside prompt-cache optimizers because it never touches the system prompt or a warm prefix. | planned |
|
|
20
20
|
|
|
21
21
|
## Why a coding agent needs this
|
|
@@ -26,7 +26,7 @@ Agents are good at picking the next command and bad at noticing when that comman
|
|
|
26
26
|
- **Scope drift gets flagged.** Poems in a bugfix, refactors nobody asked for, dependency installs unrelated to the task: warned to you at 0.6, held at 0.85 when Jev also calls the action `unrelated`.
|
|
27
27
|
- **Loops get broken.** Three failures with the same strategy (same command with cosmetic changes, same error after each edit) earn the agent a steer: re-read the error, form a new hypothesis, or report the blocker. Exact repeats are caught offline; Jev tells "investigating between failures" from "flailing" — 0.32 vs 0.93 in the smoke run.
|
|
28
28
|
- **"Done" gets checked.** When the final message reports completion after file changes and no test, build, or lint passed in that run, the agent is asked to verify before you read a false "all green". A claim that tests passed when none ran is called out as such. Once per prompt, so it cannot loop.
|
|
29
|
-
- **Slop gets a
|
|
29
|
+
- **Slop gets a name.** Written code is scored per symptom on the same request as the action guard (zero extra latency): stubs and fake-data mocks, comments that restate the code, dead or duplicated code, hedging notes. The agent is told which symptom and how to fix it, never held; the third repeat in a session becomes a standing rule. Replies are scored too — wordy, clichéd, or too technical for the configured audience — and a trend across replies earns a nudge that shapes the next one without spending a turn.
|
|
30
30
|
- **Nothing slows down the boring calls.** Read-only shell lines are skipped without a request; anything else costs one request of ~600 input tokens.
|
|
31
31
|
- **It degrades gracefully.** Offline pattern checks and exact-repeat detection run with no account at all. On an API timeout or outage the call is allowed with a warning (configurable), and reasons never include your command text or upstream error bodies.
|
|
32
32
|
|
|
@@ -68,15 +68,23 @@ Keeps the last 12 tool results for the current prompt. When the latest result fa
|
|
|
68
68
|
|
|
69
69
|
Tracks each run's evidence: code changes (`write`, `edit`) and check commands (`npm test`, `pytest`, `cargo test`, `tsc`, `eslint`, `go test`, `make test`, and similar, whether run through `bash` or `ctx_execute`) with pass/fail; context-mode's inline `Command exited with code N` counts as a failure. When a run ends with a normal assistant message after code changes and no passing check, one Jev request judges the message: `claims_done`, `claims_verified`, `verification_applies` (Noul), `outcome` (Choice: complete / partial / blocked / other). A completion claim ≥ 0.7 that is not a blocker or question, for a task where checks would mean something (≥ 0.5; prose and file housekeeping score ~0.05), is reported as unverified; a verification claim with no check run is reported as a false claim. With `nudge: true` (default) the agent receives one follow-up turn asking it to run the checks or say plainly that nothing was verified — once per user prompt.
|
|
70
70
|
|
|
71
|
-
### Slop (`write`/`edit`, same request as the action guard)
|
|
71
|
+
### Slop in code (`write`/`edit`, same request as the action guard)
|
|
72
72
|
|
|
73
73
|
pi-warden cannot rewrite code — Jev only judges — so slop is handled as a feedback loop through the agent:
|
|
74
74
|
|
|
75
|
-
1. When the agent calls `write` or `edit`,
|
|
76
|
-
2.
|
|
77
|
-
3. The
|
|
75
|
+
1. When the agent calls `write` or `edit`, four Noul questions ride on the action guard's request (no extra latency), one per symptom, each with concrete yes/no criteria: `slop_stub` (placeholder, mock, or "implement later" code where `task` needs a working implementation), `slop_comments` (explanatory comments that restate what the adjacent code shows), `slop_dead` (commented-out code, unused imports or variables, duplicated logic, unreachable branches), `slop_hedging` ("should work", "for now", TODOs without a plan). Jev sees a head/middle/tail sample of a `write` (1500 characters) or the first three replacement texts of an `edit`, plus your request.
|
|
76
|
+
2. Any symptom ≥ `slop.threshold` (0.7) trips the note. The write **goes through** — holding it would leave a half-written file.
|
|
77
|
+
3. The agent receives a steer message naming the symptoms and their fixes, e.g. *"pi-warden: the content just written to `src/x.ts` has stub or placeholder code where a working implementation is needed; hedging or vague notes. Fix it in your next edit: replace stubs, placeholders, and hard-coded fake data with the working implementation, or state in your reply exactly what is left unimplemented and why; replace \"should work\", \"for now\", and TODOs without a plan with a definite statement or a concrete follow-up."* The third time a symptom appears in a session, the note says so and asks the agent to treat it as a standing rule.
|
|
78
78
|
|
|
79
|
-
From the
|
|
79
|
+
Per-symptom scoring is what makes this precise. From the tuning set (`scripts/slop-cases.mjs`, 16 code cases): a `// TODO: implement later` stub scores stub 0.99; restating comments score comments 0.97 while an explanatory why-comment scores 0.08; commented-out code scores dead 0.96 and comments 0.43; a mock inside a *test file* scores stub 0.51 (below threshold, correctly); a stub explicitly outside the task scores 0.24. Not covered: code written through shell heredocs and content past the sample limits.
|
|
80
|
+
|
|
81
|
+
### Slop in replies (`agent_end`)
|
|
82
|
+
|
|
83
|
+
The run's final reply (≥ 200 characters) is judged against `slop.prose.audience`: `wordy` (preamble, restating the request, closing summary, filler), `cliches` ("Great question", "I hope this helps", "it's worth noting", unrequested caveat lists, emoji headings), `jargon` (unexplained terms for the audience). `audience` is `technical` (default), `plain`, or any free-text description such as *"a founder without programming background"*. One long answer to a long question is not punished: a symptom must appear in `trend` (2) of the last 3 replies before the agent is nudged, the nudge is queued for the next user prompt so it shapes the next reply without spending a turn, and two replies pass before the same nudge can fire again. From the tuning set: a padded reply scores wordy 0.97 / clichés 0.99; a dense three-point summary 0.25 / 0.05; the same status update scores jargon 0.94 for a plain audience and 0.24 for a developer.
|
|
84
|
+
|
|
85
|
+
### Steer messages
|
|
86
|
+
|
|
87
|
+
Nudges from the stuck, done, slop, and prose guards are custom messages in the agent's context. By default they are **hidden from the transcript** (`steerVisible: false`) so the conversation stays yours; the notification tells you a nudge happened and the trace panel shows the exact text. Set `steerVisible: true` to see them inline.
|
|
80
88
|
|
|
81
89
|
If TypeSafe cannot answer (timeout after 5 s, outage, budget), an action-guard call is allowed with a warning (`failOpen: true`; set it to `false` to hold instead), and the other guards simply skip. When the per-session request budget is spent, pi-warden says so once and continues with offline checks. Consent in headless runs comes from `PI_WARDEN_ENABLED=1`; `PI_WARDEN_MODE` overrides the mode.
|
|
82
90
|
|
|
@@ -103,13 +111,14 @@ Templates in `config.widget` control the text. Segments are separated by ` · `;
|
|
|
103
111
|
"enabled": true,
|
|
104
112
|
"placement": "aboveEditor",
|
|
105
113
|
"shortcut": "ctrl+shift+w",
|
|
106
|
-
"action": "warden · {tool} · irreversible {irreversible} · off-task {offTask} · {scope} · slop {
|
|
114
|
+
"action": "warden · {tool} · irreversible {irreversible} · off-task {offTask} · {scope} · slop: {slop} · patterns: {patterns} · {flags} · {level}",
|
|
107
115
|
"stuck": "warden · stuck · {failures} failures · same strategy {sameStrategy} · change {approachChange} · progress {progress} · {flags} · {status}",
|
|
108
|
-
"done": "warden · done-check · {changes} changes · {checksPassed}/{checks} checks passed · claims done {claimsDone} · claims verified {claimsVerified} · checks apply {checksApply} · {outcome} · {status}"
|
|
116
|
+
"done": "warden · done-check · {changes} changes · {checksPassed}/{checks} checks passed · claims done {claimsDone} · claims verified {claimsVerified} · checks apply {checksApply} · {outcome} · {status}",
|
|
117
|
+
"prose": "warden · prose · wordy {wordy} · clichés {cliches} · jargon {jargon} · {flags} · {status}"
|
|
109
118
|
}
|
|
110
119
|
```
|
|
111
120
|
|
|
112
|
-
Tokens — action: `tool level source irreversible offTask scope approved
|
|
121
|
+
Tokens — action: `tool level source irreversible offTask scope approved slop slopStub slopComments slopDead slopHedging patterns reasons path model ms flags time`; prose: `wordy cliches jargon status reasons model ms flags time`; stuck: `failures sameStrategy approachChange progress status source reasons model ms flags time`; done: `changes checks checksPassed claimsDone claimsVerified checksApply outcome status reasons model ms flags time`. A minimal line: `"action": "⚔ {tool} {level} · {irreversible}/{offTask}"`. Set `"enabled": false` to hide the line (the trace and panel keep working); `"shortcut": ""` disables the keybinding.
|
|
113
122
|
|
|
114
123
|
## Configuration
|
|
115
124
|
|
|
@@ -131,8 +140,13 @@ User file `~/.pi/agent/pi-warden/config.json` (owner-only). Missing keys use the
|
|
|
131
140
|
},
|
|
132
141
|
"stuck": { "enabled": true, "window": 12, "minFailures": 3, "cooldown": 3, "sameStrategy": 0.7, "nudge": true },
|
|
133
142
|
"done": { "enabled": true, "claimsDone": 0.7, "nudge": true },
|
|
134
|
-
"slop": {
|
|
135
|
-
|
|
143
|
+
"slop": {
|
|
144
|
+
"enabled": true,
|
|
145
|
+
"threshold": 0.7,
|
|
146
|
+
"prose": { "enabled": true, "audience": "technical", "threshold": 0.7, "trend": 2, "minChars": 200 }
|
|
147
|
+
},
|
|
148
|
+
"widget": { "enabled": true, "placement": "aboveEditor", "shortcut": "ctrl+shift+w" },
|
|
149
|
+
"steerVisible": false
|
|
136
150
|
}
|
|
137
151
|
```
|
|
138
152
|
|
|
@@ -140,7 +154,7 @@ A project may add `.pi/pi-warden.json` with `enabled` and per-guard overrides (f
|
|
|
140
154
|
|
|
141
155
|
## Data handling
|
|
142
156
|
|
|
143
|
-
- With consent, each guarded call sends to `https://api.typesafe.ai`: your latest prompt (truncated to 1500 characters), the tool name, the command (truncated to 2000 characters) or the file path (relative inside the project, `~`-shortened outside), whether the file exists, a 1500-character
|
|
157
|
+
- With consent, each guarded call sends to `https://api.typesafe.ai`: your latest prompt (truncated to 1500 characters), the tool name, the command (truncated to 2000 characters) or the file path (relative inside the project, `~`-shortened outside), whether the file exists, a 1500-character head/middle/tail sample for `write`, and the first three edit pairs (400 characters each) for `edit`. The stuck detector sends the last 12 tool calls (300 characters each) with 400-character output tails; the done-check and the prose check send the agent's final message (2000 and 2500 characters) with the run's check commands and the audience description. No other files, history, or telemetry.
|
|
144
158
|
- Obvious credentials in the action (`Authorization` headers, `TOKEN=`/`SECRET=` assignments, `sk-`, `ghp_`, `AKIA`, JWTs, URL passwords, PEM blocks) are replaced with `[redacted]` before sending. This is best-effort; do not rely on it for prompts that contain secrets.
|
|
145
159
|
- Text returned or steered to the agent names the tool, the reasons, and the scores, not the command text. UI errors never include upstream response bodies or keys.
|
|
146
160
|
- Judgments are model output. Thresholds are yours to tune; a hold is information for the agent and for you, not a verdict on either.
|
|
@@ -170,8 +184,9 @@ Also exported: `matchPatterns`, `isReadOnlyCommand`, `describeAction`, `redact`,
|
|
|
170
184
|
```bash
|
|
171
185
|
npm install
|
|
172
186
|
npm run check # typecheck, offline tests (mocked transport), build
|
|
173
|
-
npm run test:live #
|
|
187
|
+
npm run test:live # 31 billable synthetic judgments across all guards (key from .env or the stored login); pass action|slop|approval|stuck|done for one group
|
|
174
188
|
npm run dev:pi # start Pi with this working tree plus an installed pi-typesafe
|
|
189
|
+
node scripts/slop-cases.mjs # 22 billable cases that tune the slop and prose questions
|
|
175
190
|
npm run preview # re-render docs/preview.png from the recorded live verdicts (needs a Chrome binary)
|
|
176
191
|
```
|
|
177
192
|
|
package/dist/config.d.ts
CHANGED
|
@@ -36,12 +36,22 @@ export interface DoneGuardConfig {
|
|
|
36
36
|
/** Also send the agent a follow-up asking it to verify. Triggers one more LLM turn. */
|
|
37
37
|
nudge: boolean;
|
|
38
38
|
}
|
|
39
|
+
export interface ProseConfig {
|
|
40
|
+
enabled: boolean;
|
|
41
|
+
/** Who reads the agent's replies: "technical", "plain", or a free-text description. Drives the jargon question. */
|
|
42
|
+
audience: string;
|
|
43
|
+
/** P(symptom) at or above this counts as a hit. */
|
|
44
|
+
threshold: number;
|
|
45
|
+
/** A symptom must hit in this many of the last three replies before the agent is nudged. */
|
|
46
|
+
trend: number;
|
|
47
|
+
/** Replies with fewer characters are not judged. */
|
|
48
|
+
minChars: number;
|
|
49
|
+
}
|
|
39
50
|
export interface SlopGuardConfig {
|
|
40
51
|
enabled: boolean;
|
|
41
|
-
/**
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
placeholder: number;
|
|
52
|
+
/** P(symptom) at or above this is reported for written code: stub, comments, dead, hedging. */
|
|
53
|
+
threshold: number;
|
|
54
|
+
prose: ProseConfig;
|
|
45
55
|
}
|
|
46
56
|
export type WardenMode = "steer" | "confirm" | "advise";
|
|
47
57
|
export interface WardenConfig {
|
|
@@ -65,6 +75,8 @@ export interface WardenConfig {
|
|
|
65
75
|
slop: SlopGuardConfig;
|
|
66
76
|
/** The status line above the editor and the trace panel. */
|
|
67
77
|
widget: WidgetConfig;
|
|
78
|
+
/** Show steer messages in the transcript. They are always visible in the trace panel. */
|
|
79
|
+
steerVisible: boolean;
|
|
68
80
|
}
|
|
69
81
|
export declare const PACKAGE_NAME = "pi-warden";
|
|
70
82
|
export declare const PROJECT_CONFIG_FILE = "pi-warden.json";
|
package/dist/config.js
CHANGED
|
@@ -22,8 +22,9 @@ export function defaultConfig() {
|
|
|
22
22
|
},
|
|
23
23
|
stuck: { enabled: true, window: 12, minFailures: 3, cooldown: 3, sameStrategy: 0.7, nudge: true },
|
|
24
24
|
done: { enabled: true, claimsDone: 0.7, nudge: true },
|
|
25
|
-
slop: { enabled: true,
|
|
25
|
+
slop: { enabled: true, threshold: 0.7, prose: { enabled: true, audience: "technical", threshold: 0.7, trend: 2, minChars: 200 } },
|
|
26
26
|
widget: defaultWidgetConfig(),
|
|
27
|
+
steerVisible: false,
|
|
27
28
|
};
|
|
28
29
|
}
|
|
29
30
|
/** Mirrors Pi's agent directory rule so the file sits next to pi-typesafe's auth.json. */
|
|
@@ -68,9 +69,6 @@ function positiveInteger(value, fallback) {
|
|
|
68
69
|
export function isMode(value) {
|
|
69
70
|
return value === "steer" || value === "confirm" || value === "advise";
|
|
70
71
|
}
|
|
71
|
-
function score(value, fallback) {
|
|
72
|
-
return typeof value === "number" && Number.isFinite(value) && value >= 0 && value <= 2 ? value : fallback;
|
|
73
|
-
}
|
|
74
72
|
function applyAction(base, raw, timeoutMs) {
|
|
75
73
|
const withTimeout = { ...base, timeoutMs };
|
|
76
74
|
if (!isObject(raw))
|
|
@@ -103,10 +101,22 @@ function applyDone(base, raw) {
|
|
|
103
101
|
return base;
|
|
104
102
|
return { enabled: boolean(raw.enabled, base.enabled), claimsDone: probability(raw.claimsDone, base.claimsDone), nudge: boolean(raw.nudge, base.nudge) };
|
|
105
103
|
}
|
|
104
|
+
function applyProse(base, raw) {
|
|
105
|
+
if (!isObject(raw))
|
|
106
|
+
return base;
|
|
107
|
+
return {
|
|
108
|
+
enabled: boolean(raw.enabled, base.enabled),
|
|
109
|
+
audience: typeof raw.audience === "string" && raw.audience.trim() ? raw.audience.trim() : base.audience,
|
|
110
|
+
threshold: probability(raw.threshold, base.threshold),
|
|
111
|
+
trend: Math.min(3, positiveInteger(raw.trend, base.trend)),
|
|
112
|
+
minChars: positiveInteger(raw.minChars, base.minChars),
|
|
113
|
+
};
|
|
114
|
+
}
|
|
106
115
|
function applySlop(base, raw) {
|
|
107
116
|
if (!isObject(raw))
|
|
108
117
|
return base;
|
|
109
|
-
|
|
118
|
+
// 0.2.x used `placeholder` for the stub threshold; it still sets the shared threshold.
|
|
119
|
+
return { enabled: boolean(raw.enabled, base.enabled), threshold: probability(raw.threshold ?? raw.placeholder, base.threshold), prose: applyProse(base.prose, raw.prose) };
|
|
110
120
|
}
|
|
111
121
|
function applyWidget(base, raw) {
|
|
112
122
|
if (!isObject(raw))
|
|
@@ -119,6 +129,7 @@ function applyWidget(base, raw) {
|
|
|
119
129
|
action: template(raw.action, base.action),
|
|
120
130
|
stuck: template(raw.stuck, base.stuck),
|
|
121
131
|
done: template(raw.done, base.done),
|
|
132
|
+
prose: template(raw.prose, base.prose),
|
|
122
133
|
};
|
|
123
134
|
}
|
|
124
135
|
/** Shared request settings; `action.timeoutMs`/`action.maxRequests` from 0.1.x files are still honoured. */
|
|
@@ -149,6 +160,7 @@ export function applyUserOverrides(base, raw) {
|
|
|
149
160
|
...shared,
|
|
150
161
|
...applyGuards(base, raw, shared.timeoutMs),
|
|
151
162
|
widget: applyWidget(base.widget, raw.widget),
|
|
163
|
+
steerVisible: boolean(raw.steerVisible, base.steerVisible),
|
|
152
164
|
};
|
|
153
165
|
}
|
|
154
166
|
/** Project files may tune the guards but cannot grant TypeSafe consent, change the mode, or raise budgets. */
|
package/dist/extension.d.ts
CHANGED
|
@@ -1,6 +1,8 @@
|
|
|
1
1
|
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
|
2
|
-
import type { Verdict } from "./guard.js";
|
|
2
|
+
import type { SlopSymptom, Verdict } from "./guard.js";
|
|
3
3
|
export declare const disclosure = "With TypeSafe judgments enabled, pi-warden sends to api.typesafe.ai: your latest request plus a redacted, truncated summary of each guarded bash, write, or edit call before it runs; the last few tool calls and output tails when the agent keeps failing; and the agent's final message when it reports completion without running checks. Requests may incur charges. Secret redaction is best-effort. Results are model judgments, not proof or authorization; offline pattern checks stay active either way.";
|
|
4
4
|
export declare function confirmMessage(verdict: Verdict): string;
|
|
5
|
+
/** Names each symptom and its fix; repeats in the session turn the note into a standing rule. */
|
|
6
|
+
export declare function slopSteer(where: string, symptoms: readonly SlopSymptom[], counts: Record<SlopSymptom, number>): string;
|
|
5
7
|
/** Native Pi registration; importing the root library does not load this module. */
|
|
6
8
|
export default function wardenExtension(pi: ExtensionAPI): void;
|
package/dist/extension.js
CHANGED
|
@@ -3,15 +3,16 @@ import { createTypeSafe, resolveApiKey } from "pi-typesafe";
|
|
|
3
3
|
import { ensureApiKey } from "pi-typesafe/ui";
|
|
4
4
|
import { applyUserOverrides, defaultConfig, isMode, loadConfig, PACKAGE_NAME, projectConfigPath, readUserConfig, setUserSetting, userConfigPath, writeUserConfig } from "./config.js";
|
|
5
5
|
import { classifyToolResult, doneNudge, emptyEvidence, evaluateDone, finalAssistantText, formatDone, needsDoneCheck, recordOutcome } from "./done.js";
|
|
6
|
-
import { evaluateAction, formatVerdict, steerReason, textApproves } from "./guard.js";
|
|
6
|
+
import { evaluateAction, formatVerdict, SLOP_LABELS, steerReason, textApproves } from "./guard.js";
|
|
7
|
+
import { evaluateProse, proseNudge, ProseTrend } from "./prose.js";
|
|
7
8
|
import { AttemptWindow, evaluateStuck, formatStuck, makeAttempt, resultFailed, stuckNudge } from "./stuck.js";
|
|
8
9
|
import { openTracePanel } from "./panel.js";
|
|
9
|
-
import { actionDetails, doneDetails, stuckDetails, Trace } from "./trace.js";
|
|
10
|
-
import { TOKEN_NAMES } from "./widget.js";
|
|
10
|
+
import { actionDetails, doneDetails, proseDetails, stuckDetails, Trace } from "./trace.js";
|
|
11
|
+
import { proseTokens, renderTemplate, TOKEN_NAMES } from "./widget.js";
|
|
11
12
|
export const disclosure = "With TypeSafe judgments enabled, pi-warden sends to api.typesafe.ai: your latest request plus a redacted, truncated summary of each guarded bash, write, or edit call before it runs; the last few tool calls and output tails when the agent keeps failing; and the agent's final message when it reports completion without running checks. Requests may incur charges. Secret redaction is best-effort. Results are model judgments, not proof or authorization; offline pattern checks stay active either way.";
|
|
12
13
|
const WIDGET = PACKAGE_NAME;
|
|
13
14
|
const CONFIRM_TEXT_LIMIT = 500;
|
|
14
|
-
const freshStats = () => ({ inspected: 0, judged: 0, warned: 0, held: 0, approved: 0, slop: 0, stuckChecks: 0, stuck: 0, doneChecks: 0, unverified: 0, errors: 0 });
|
|
15
|
+
const freshStats = () => ({ inspected: 0, judged: 0, warned: 0, held: 0, approved: 0, slop: 0, stuckChecks: 0, stuck: 0, doneChecks: 0, unverified: 0, proseChecks: 0, proseNudges: 0, errors: 0 });
|
|
15
16
|
function latestUserPrompt(ctx) {
|
|
16
17
|
const entries = ctx.sessionManager.getBranch();
|
|
17
18
|
for (let index = entries.length - 1; index >= 0; index--) {
|
|
@@ -49,6 +50,19 @@ export function confirmMessage(verdict) {
|
|
|
49
50
|
return lines.join("\n");
|
|
50
51
|
}
|
|
51
52
|
const callKey = (tool, input) => `${tool}\0${JSON.stringify(input)}`;
|
|
53
|
+
const SLOP_FIXES = {
|
|
54
|
+
stub: "replace stubs, placeholders, and hard-coded fake data with the working implementation, or state in your reply exactly what is left unimplemented and why",
|
|
55
|
+
comments: "delete comments that restate the code; keep only those that explain intent, constraints, or non-obvious behaviour",
|
|
56
|
+
dead: "remove commented-out code, unused imports and variables, duplicated logic, and unreachable branches",
|
|
57
|
+
hedging: "replace \"should work\", \"for now\", and TODOs without a plan with a definite statement or a concrete follow-up",
|
|
58
|
+
};
|
|
59
|
+
/** Names each symptom and its fix; repeats in the session turn the note into a standing rule. */
|
|
60
|
+
export function slopSteer(where, symptoms, counts) {
|
|
61
|
+
const named = symptoms.map(symptom => `${SLOP_LABELS[symptom]}${counts[symptom] >= 3 ? ` (${counts[symptom]}th time this session)` : ""}`).join("; ");
|
|
62
|
+
const fixes = symptoms.map(symptom => SLOP_FIXES[symptom]).join("; ");
|
|
63
|
+
const standing = symptoms.some(symptom => counts[symptom] >= 3) ? " Treat this as a standing rule for the rest of the session." : "";
|
|
64
|
+
return `pi-warden: the content just written to ${where} has ${named}. Fix it in your next edit: ${fixes}.${standing}`;
|
|
65
|
+
}
|
|
52
66
|
/** Native Pi registration; importing the root library does not load this module. */
|
|
53
67
|
export default function wardenExtension(pi) {
|
|
54
68
|
let client;
|
|
@@ -63,6 +77,8 @@ export default function wardenExtension(pi) {
|
|
|
63
77
|
let attempts = new AttemptWindow(defaultConfig().stuck.window);
|
|
64
78
|
let evidence = emptyEvidence();
|
|
65
79
|
let doneNudged = false;
|
|
80
|
+
const prose = new ProseTrend();
|
|
81
|
+
const slopCounts = { stub: 0, comments: 0, dead: 0, hedging: 0 };
|
|
66
82
|
const configFor = (ctx) => loadConfig({ cwd: ctx.cwd, projectTrusted: ctx.isProjectTrusted() });
|
|
67
83
|
const consentGiven = (config) => config.typesafe || process.env.PI_WARDEN_ENABLED === "1";
|
|
68
84
|
const consentSource = (config) => config.typesafe ? "/warden enable" : process.env.PI_WARDEN_ENABLED === "1" ? "PI_WARDEN_ENABLED" : undefined;
|
|
@@ -106,7 +122,7 @@ export default function wardenExtension(pi) {
|
|
|
106
122
|
trace.push({ at: Date.now(), guard, line, details });
|
|
107
123
|
paint(ctx, config);
|
|
108
124
|
};
|
|
109
|
-
const steer = (content) => pi.sendMessage({ customType: `${PACKAGE_NAME}-steer`, content, display:
|
|
125
|
+
const steer = (config, content, options = { deliverAs: "steer" }) => pi.sendMessage({ customType: `${PACKAGE_NAME}-steer`, content, display: config.steerVisible }, options);
|
|
110
126
|
pi.on("session_start", async (_event, ctx) => {
|
|
111
127
|
client = undefined;
|
|
112
128
|
budgetExhausted = false;
|
|
@@ -117,6 +133,9 @@ export default function wardenExtension(pi) {
|
|
|
117
133
|
attempts.reset();
|
|
118
134
|
evidence = emptyEvidence();
|
|
119
135
|
doneNudged = false;
|
|
136
|
+
prose.reset();
|
|
137
|
+
for (const symptom of Object.keys(slopCounts))
|
|
138
|
+
slopCounts[symptom] = 0;
|
|
120
139
|
if (ctx.hasUI)
|
|
121
140
|
ctx.ui.setWidget(WIDGET, undefined);
|
|
122
141
|
});
|
|
@@ -160,12 +179,14 @@ export default function wardenExtension(pi) {
|
|
|
160
179
|
const told = verdict.level === "confirm" && mode === "steer" ? steerReason(verdict, { canApprove: judge !== undefined }) : undefined;
|
|
161
180
|
if (verdict.source !== "read-only")
|
|
162
181
|
record(ctx, config, "action", formatVerdict(verdict, config.widget.action), actionDetails(verdict, { mode, ...(told ? { told } : {}) }));
|
|
163
|
-
if (verdict.
|
|
182
|
+
if (verdict.slopSymptoms?.length && verdict.slopReasons) {
|
|
164
183
|
stats.slop++;
|
|
184
|
+
for (const symptom of verdict.slopSymptoms)
|
|
185
|
+
slopCounts[symptom]++;
|
|
165
186
|
const where = verdict.summary.path ?? event.toolName;
|
|
166
187
|
if (ctx.hasUI)
|
|
167
188
|
ctx.ui.notify(`warden · slop · ${where}: ${verdict.slopReasons.join("; ")}`, "warning");
|
|
168
|
-
steer(
|
|
189
|
+
steer(config, slopSteer(where, verdict.slopSymptoms, slopCounts));
|
|
169
190
|
}
|
|
170
191
|
if (verdict.level === "warn") {
|
|
171
192
|
stats.warned++;
|
|
@@ -221,16 +242,39 @@ export default function wardenExtension(pi) {
|
|
|
221
242
|
if (ctx.hasUI)
|
|
222
243
|
ctx.ui.notify(`warden · stuck: ${verdict.reasons.join("; ")}${nudge ? " (agent nudged)" : ""}`, "warning");
|
|
223
244
|
if (nudge)
|
|
224
|
-
steer(nudge);
|
|
245
|
+
steer(config, nudge);
|
|
225
246
|
});
|
|
226
247
|
pi.on("agent_end", async (event, ctx) => {
|
|
227
248
|
const config = configFor(ctx);
|
|
228
|
-
if (!config.enabled
|
|
249
|
+
if (!config.enabled)
|
|
229
250
|
return;
|
|
230
251
|
const finalMessage = finalAssistantText(event.messages);
|
|
231
252
|
const judge = judgeFor(config);
|
|
232
253
|
if (!finalMessage || !judge)
|
|
233
254
|
return;
|
|
255
|
+
if (config.slop.enabled && config.slop.prose.enabled && finalMessage.length >= config.slop.prose.minChars) {
|
|
256
|
+
stats.proseChecks++;
|
|
257
|
+
const verdict = await evaluateProse(latestUserPrompt(ctx), finalMessage, { config: config.slop.prose, judge, timeoutMs: config.timeoutMs, signal: ctx.signal });
|
|
258
|
+
if (verdict.error)
|
|
259
|
+
noteError(ctx, verdict.error, undefined);
|
|
260
|
+
else
|
|
261
|
+
prose.record(verdict.flagged);
|
|
262
|
+
const due = verdict.error ? [] : prose.due(config.slop.prose.trend);
|
|
263
|
+
const nudge = due.length ? proseNudge(due, config.slop.prose.audience, prose.counts) : undefined;
|
|
264
|
+
if (nudge) {
|
|
265
|
+
verdict.nudged = true;
|
|
266
|
+
prose.markNudged();
|
|
267
|
+
stats.proseNudges++;
|
|
268
|
+
}
|
|
269
|
+
record(ctx, config, "prose", renderTemplate(config.widget.prose, proseTokens(verdict)), proseDetails(verdict, finalMessage, config.slop.prose.audience, nudge));
|
|
270
|
+
if (nudge) {
|
|
271
|
+
if (ctx.hasUI)
|
|
272
|
+
ctx.ui.notify(`warden · prose: ${due.join(", ")} in ${config.slop.prose.trend} of the last 3 replies (agent nudged for the next reply)`, "warning");
|
|
273
|
+
steer(config, nudge, { deliverAs: "nextTurn" });
|
|
274
|
+
}
|
|
275
|
+
}
|
|
276
|
+
if (!config.done.enabled || doneNudged || !needsDoneCheck(evidence))
|
|
277
|
+
return;
|
|
234
278
|
stats.doneChecks++;
|
|
235
279
|
const verdict = await evaluateDone(latestUserPrompt(ctx), finalMessage, evidence, { config: config.done, judge, timeoutMs: config.timeoutMs, signal: ctx.signal });
|
|
236
280
|
if (verdict.error)
|
|
@@ -244,7 +288,7 @@ export default function wardenExtension(pi) {
|
|
|
244
288
|
ctx.ui.notify(`warden · done-check: ${verdict.reasons.join("; ")}${nudge ? " (agent asked to verify)" : ""}`, "warning");
|
|
245
289
|
if (nudge) {
|
|
246
290
|
doneNudged = true;
|
|
247
|
-
|
|
291
|
+
steer(config, nudge, { deliverAs: "followUp", triggerTurn: true });
|
|
248
292
|
}
|
|
249
293
|
});
|
|
250
294
|
// Shortcuts are registered once at load; a shape check keeps a typo in the config file from being registered.
|
|
@@ -277,11 +321,11 @@ export default function wardenExtension(pi) {
|
|
|
277
321
|
const key = resolveApiKey();
|
|
278
322
|
const source = consentSource(config);
|
|
279
323
|
const usage = client?.getUsage();
|
|
280
|
-
const guards = [config.action.enabled && "action", config.stuck.enabled && "stuck", config.done.enabled && "done-check", config.slop.enabled && "slop"].filter(Boolean).join(", ");
|
|
324
|
+
const guards = [config.action.enabled && "action", config.stuck.enabled && "stuck", config.done.enabled && "done-check", config.slop.enabled && "slop", config.slop.enabled && config.slop.prose.enabled && `prose (${config.slop.prose.audience})`].filter(Boolean).join(", ");
|
|
281
325
|
report([
|
|
282
326
|
`pi-warden: ${config.enabled ? `guarding ${config.action.tools.join(", ")} (${guards})` : "off"}; mode ${activeMode(config, ctx.hasUI)}; TypeSafe judgments ${source ? `enabled via ${source}` : "disabled (run /warden enable)"}; key ${key ? key.source === "stored" ? "stored (shared with pi-typesafe)" : "from TYPESAFE_API_KEY" : "missing (run /warden enable)"}.`,
|
|
283
|
-
`Session: ${stats.inspected} inspected, ${stats.judged} judged, ${stats.warned} warned, ${stats.held} held, ${stats.approved} approved on retry, ${stats.slop} slop notes, ${stats.stuck}/${stats.stuckChecks} stuck, ${stats.unverified}/${stats.doneChecks} unverified done, ${stats.errors} TypeSafe errors; ${usage?.requestsStarted ?? 0}/${config.maxRequests} requests.`,
|
|
284
|
-
`Thresholds: irreversible warn ${config.action.irreversible.warn} / hold ${config.action.irreversible.confirm}; off-task warn ${config.action.offTask.warn} / hold ${config.action.offTask.confirm}; stuck same-strategy ${config.stuck.sameStrategy} after ${config.stuck.minFailures} failures; done claims ${config.done.claimsDone}; slop
|
|
327
|
+
`Session: ${stats.inspected} inspected, ${stats.judged} judged, ${stats.warned} warned, ${stats.held} held, ${stats.approved} approved on retry, ${stats.slop} slop notes, ${stats.stuck}/${stats.stuckChecks} stuck, ${stats.unverified}/${stats.doneChecks} unverified done, ${stats.proseNudges}/${stats.proseChecks} prose nudges, ${stats.errors} TypeSafe errors; ${usage?.requestsStarted ?? 0}/${config.maxRequests} requests. Steers are ${config.steerVisible ? "shown in the transcript" : "hidden from the transcript (trace panel shows them)"}.`,
|
|
328
|
+
`Thresholds: irreversible warn ${config.action.irreversible.warn} / hold ${config.action.irreversible.confirm}; off-task warn ${config.action.offTask.warn} / hold ${config.action.offTask.confirm}; stuck same-strategy ${config.stuck.sameStrategy} after ${config.stuck.minFailures} failures; done claims ${config.done.claimsDone}; slop ${config.slop.threshold}, prose ${config.slop.prose.threshold} in ${config.slop.prose.trend}/3 replies; failOpen ${config.action.failOpen}.`,
|
|
285
329
|
`Config: ${userConfigPath()}${ctx.isProjectTrusted() ? ` and ${projectConfigPath(ctx.cwd)}` : ""}.`,
|
|
286
330
|
widget.size ? `Last: ${[...widget.values()].join(" | ")}` : "No guarded activity yet this session.",
|
|
287
331
|
`Trace: ${trace.entries().length} events (/warden trace${shortcut ? `, ${shortcut}` : ""}, or click the status line in fullscreen mode). Widget templates in config.widget: action tokens ${TOKEN_NAMES.action.map(name => `{${name}}`).join(" ")}.`,
|
package/dist/guard.d.ts
CHANGED
|
@@ -42,12 +42,15 @@ export interface Judgment {
|
|
|
42
42
|
model: string;
|
|
43
43
|
elapsedMs: number;
|
|
44
44
|
}
|
|
45
|
+
/** One probability per slop symptom; the steer names the symptoms above the threshold. */
|
|
45
46
|
export interface SlopJudgment {
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
47
|
+
stub: number;
|
|
48
|
+
comments: number;
|
|
49
|
+
dead: number;
|
|
50
|
+
hedging: number;
|
|
50
51
|
}
|
|
52
|
+
export type SlopSymptom = keyof SlopJudgment;
|
|
53
|
+
export declare const SLOP_SYMPTOMS: readonly SlopSymptom[];
|
|
51
54
|
export interface Verdict {
|
|
52
55
|
level: Level;
|
|
53
56
|
source: "skipped" | "read-only" | "pattern" | "typesafe" | "error";
|
|
@@ -57,7 +60,8 @@ export interface Verdict {
|
|
|
57
60
|
reasons: string[];
|
|
58
61
|
judgment?: Judgment;
|
|
59
62
|
slop?: SlopJudgment;
|
|
60
|
-
/**
|
|
63
|
+
/** Symptoms at or above the slop threshold, strongest first. The level itself is never raised by slop. */
|
|
64
|
+
slopSymptoms?: SlopSymptom[];
|
|
61
65
|
slopReasons?: string[];
|
|
62
66
|
/** True when a previously held call was allowed because the user's latest message approves it. */
|
|
63
67
|
approvedByUser?: boolean;
|
|
@@ -91,9 +95,12 @@ export declare const questions: {
|
|
|
91
95
|
}>;
|
|
92
96
|
};
|
|
93
97
|
export declare const slopQuestions: {
|
|
94
|
-
|
|
95
|
-
|
|
98
|
+
slop_stub: import("pi-typesafe").NoulQuestion;
|
|
99
|
+
slop_comments: import("pi-typesafe").NoulQuestion;
|
|
100
|
+
slop_dead: import("pi-typesafe").NoulQuestion;
|
|
101
|
+
slop_hedging: import("pi-typesafe").NoulQuestion;
|
|
96
102
|
};
|
|
103
|
+
export declare const SLOP_LABELS: Record<SlopSymptom, string>;
|
|
97
104
|
export declare const approvalQuestion: {
|
|
98
105
|
approved: import("pi-typesafe").NoulQuestion;
|
|
99
106
|
};
|
|
@@ -107,8 +114,10 @@ export declare function buildRequest(summary: ActionSummary, task: string | unde
|
|
|
107
114
|
};
|
|
108
115
|
questions: {
|
|
109
116
|
approved?: import("pi-typesafe").NoulQuestion;
|
|
110
|
-
|
|
111
|
-
|
|
117
|
+
slop_stub?: import("pi-typesafe").NoulQuestion;
|
|
118
|
+
slop_comments?: import("pi-typesafe").NoulQuestion;
|
|
119
|
+
slop_dead?: import("pi-typesafe").NoulQuestion;
|
|
120
|
+
slop_hedging?: import("pi-typesafe").NoulQuestion;
|
|
112
121
|
irreversible: import("pi-typesafe").NoulQuestion;
|
|
113
122
|
off_task: import("pi-typesafe").NoulQuestion;
|
|
114
123
|
scope: import("pi-typesafe").ChoiceQuestion<{
|
package/dist/guard.js
CHANGED
|
@@ -5,6 +5,7 @@ import { choice, noul, score, TypeSafeIntegrationError } from "pi-typesafe";
|
|
|
5
5
|
import { redact } from "./redact.js";
|
|
6
6
|
import { commandOf } from "./tools.js";
|
|
7
7
|
import { actionTokens, DEFAULT_TEMPLATES, renderTemplate } from "./widget.js";
|
|
8
|
+
export const SLOP_SYMPTOMS = ["stub", "comments", "dead", "hedging"];
|
|
8
9
|
const LEVEL_RANK = { allow: 0, warn: 1, confirm: 2 };
|
|
9
10
|
const higher = (a, b) => (LEVEL_RANK[a] >= LEVEL_RANK[b] ? a : b);
|
|
10
11
|
const TASK_LIMIT = 1500;
|
|
@@ -14,6 +15,16 @@ const EDIT_LIMIT = 400;
|
|
|
14
15
|
function truncate(text, limit) {
|
|
15
16
|
return text.length <= limit ? text : `${text.slice(0, limit)}… [${text.length - limit} more chars]`;
|
|
16
17
|
}
|
|
18
|
+
/** Head, a slice from the middle, and the tail, so stubs at the end of a long file are still seen. */
|
|
19
|
+
function sample(text, limit) {
|
|
20
|
+
if (text.length <= limit)
|
|
21
|
+
return text;
|
|
22
|
+
const head = Math.floor(limit * 0.6);
|
|
23
|
+
const mid = Math.floor(limit * 0.2);
|
|
24
|
+
const tail = limit - head - mid;
|
|
25
|
+
const middleStart = Math.floor(text.length / 2 - mid / 2);
|
|
26
|
+
return `${text.slice(0, head)}\n… [${middleStart - head} chars] …\n${text.slice(middleStart, middleStart + mid)}\n… [${text.length - tail - (middleStart + mid)} chars] …\n${text.slice(-tail)}`;
|
|
27
|
+
}
|
|
17
28
|
const SHELL_RULES = [
|
|
18
29
|
{ id: "git-force-push", severity: "destructive", label: "git force push", test: /\bgit\s+push\b[^\n;&|]*\s(?:-f|--force)(?![-\w])/ },
|
|
19
30
|
{ id: "git-force-with-lease", severity: "risky", label: "git push --force-with-lease", test: /\bgit\s+push\b[^\n;&|]*--force-with-lease/ },
|
|
@@ -172,7 +183,7 @@ export function describeAction(tool, input, cwd) {
|
|
|
172
183
|
}
|
|
173
184
|
if (tool === "write" && typeof input.content === "string") {
|
|
174
185
|
summary.bytes = Buffer.byteLength(input.content, "utf8");
|
|
175
|
-
summary.excerpt = redact(
|
|
186
|
+
summary.excerpt = redact(sample(input.content, EXCERPT_LIMIT));
|
|
176
187
|
}
|
|
177
188
|
if (tool === "edit" && Array.isArray(input.edits)) {
|
|
178
189
|
summary.editCount = input.edits.length;
|
|
@@ -207,12 +218,28 @@ export const questions = {
|
|
|
207
218
|
}),
|
|
208
219
|
};
|
|
209
220
|
export const slopQuestions = {
|
|
210
|
-
|
|
211
|
-
"
|
|
212
|
-
"
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
|
|
221
|
+
slop_stub: noul("Does the content `action` writes leave placeholder, stub, mock, or \"implement later\" code where `task` needs a working implementation?", {
|
|
222
|
+
true: "Yes: a function returns a constant, null, or fake data instead of doing its job; a TODO or \"implement later\" stands where the logic should be; a mock is hard-coded where a real call is needed.",
|
|
223
|
+
false: "No: the code does what `task` asks, or the incomplete part is clearly outside what `task` asked for.",
|
|
224
|
+
}),
|
|
225
|
+
slop_comments: noul("Do the explanatory comments in the content `action` writes mostly restate what the adjacent code already shows, instead of explaining intent, constraints, or non-obvious behaviour? Commented-out code is not an explanatory comment and is judged elsewhere.", {
|
|
226
|
+
true: "Yes: comments such as \"// increment the counter\" above counter++, \"// return the result\", \"// loop over items\", banners repeating the function name, or doc comments that only repeat parameter names.",
|
|
227
|
+
false: "No: there are few or no comments, or the comments explain why, an invariant, a workaround, a limitation, a reference, or a decision a reader could not infer from the code.",
|
|
228
|
+
}),
|
|
229
|
+
slop_dead: noul("Does the content `action` writes include dead or redundant code: commented-out code, unused imports or variables, duplicated logic, or branches that cannot be reached?", {
|
|
230
|
+
true: "Yes: blocks of commented-out code, variables or imports that are never used, the same logic written twice, or checks that are always true or false.",
|
|
231
|
+
false: "No: every line participates in the behaviour.",
|
|
232
|
+
}),
|
|
233
|
+
slop_hedging: noul("Does the content `action` writes contain vague or hedging text: \"this should work\", \"for now\", \"might need changes\", TODO notes without a plan, or explanations that say nothing specific?", {
|
|
234
|
+
true: "Yes: uncertain or apologetic notes in code or docs, placeholders like \"TBD\", or prose that does not commit to what the code does.",
|
|
235
|
+
false: "No: the text states what the code does and why, or there is no such text.",
|
|
236
|
+
}),
|
|
237
|
+
};
|
|
238
|
+
export const SLOP_LABELS = {
|
|
239
|
+
stub: "stub or placeholder code where a working implementation is needed",
|
|
240
|
+
comments: "comments that restate the code",
|
|
241
|
+
dead: "dead or duplicated code",
|
|
242
|
+
hedging: "hedging or vague notes",
|
|
216
243
|
};
|
|
217
244
|
export const approvalQuestion = {
|
|
218
245
|
approved: noul("Does `task` (the user's latest message) explicitly approve running `action`, which was held earlier for the user's decision?", {
|
|
@@ -300,15 +327,13 @@ export async function evaluateAction(action, options) {
|
|
|
300
327
|
reasons.push(`off-task ${percent(judgment.offTask)} (${judgment.scope.replace(/_/g, " ")})`);
|
|
301
328
|
}
|
|
302
329
|
const verdict = { level, source: "typesafe", summary, patterns, reasons, judgment };
|
|
303
|
-
if (options.slop?.enabled &&
|
|
304
|
-
verdict.slop = {
|
|
305
|
-
const
|
|
306
|
-
if (
|
|
307
|
-
|
|
308
|
-
|
|
309
|
-
|
|
310
|
-
if (slopReasons.length)
|
|
311
|
-
verdict.slopReasons = slopReasons;
|
|
330
|
+
if (options.slop?.enabled && SLOP_SYMPTOMS.every(symptom => typeof answers[`slop_${symptom}`]?.noul === "number")) {
|
|
331
|
+
verdict.slop = { stub: answers.slop_stub.noul, comments: answers.slop_comments.noul, dead: answers.slop_dead.noul, hedging: answers.slop_hedging.noul };
|
|
332
|
+
const flagged = SLOP_SYMPTOMS.filter(symptom => verdict.slop[symptom] >= options.slop.threshold).sort((a, b) => verdict.slop[b] - verdict.slop[a]);
|
|
333
|
+
if (flagged.length) {
|
|
334
|
+
verdict.slopSymptoms = flagged;
|
|
335
|
+
verdict.slopReasons = flagged.map(symptom => `${SLOP_LABELS[symptom]} (${percent(verdict.slop[symptom])})`);
|
|
336
|
+
}
|
|
312
337
|
}
|
|
313
338
|
if (level === "confirm" && judgment.approved !== undefined && judgment.approved >= APPROVAL_THRESHOLD) {
|
|
314
339
|
verdict.level = "allow";
|
package/dist/index.d.ts
CHANGED
|
@@ -1,15 +1,18 @@
|
|
|
1
1
|
export { evaluateAction, describeAction, matchPatterns, isReadOnlyCommand, buildRequest, formatVerdict, steerReason, textApproves, questions, slopQuestions, approvalQuestion } from "./guard.js";
|
|
2
|
-
export type { ActionInput, ActionSummary, EvaluateOptions, Judge, Judgment, SlopJudgment, Level, PatternHit, ScopeLabel, Severity, Verdict } from "./guard.js";
|
|
2
|
+
export type { ActionInput, ActionSummary, EvaluateOptions, Judge, Judgment, SlopJudgment, SlopSymptom, Level, PatternHit, ScopeLabel, Severity, Verdict } from "./guard.js";
|
|
3
|
+
export { SLOP_SYMPTOMS, SLOP_LABELS } from "./guard.js";
|
|
4
|
+
export { evaluateProse, buildProseRequest, proseQuestions, proseNudge, describeAudience, ProseTrend, PROSE_SYMPTOMS, PROSE_LABELS } from "./prose.js";
|
|
5
|
+
export type { ProseSymptom, ProseVerdict, ProseOptions } from "./prose.js";
|
|
3
6
|
export { AttemptWindow, makeAttempt, resultFailed, resultText, evaluateStuck, buildStuckRequest, stuckQuestions, stuckNudge, formatStuck } from "./stuck.js";
|
|
4
7
|
export type { Attempt, StuckJudgment, StuckVerdict, StuckOptions } from "./stuck.js";
|
|
5
8
|
export { classifyToolResult, recordOutcome, emptyEvidence, needsDoneCheck, finalAssistantText, evaluateDone, buildDoneRequest, doneQuestions, doneNudge, formatDone } from "./done.js";
|
|
6
9
|
export type { ToolOutcome, RunEvidence, DoneJudgment, DoneVerdict, DoneOptions } from "./done.js";
|
|
7
10
|
export { redact } from "./redact.js";
|
|
8
11
|
export { defaultConfig, loadConfig, applyUserOverrides, applyProjectOverrides, isMode, userConfigPath, projectConfigPath, readUserConfig, writeUserConfig, setUserSetting, PACKAGE_NAME } from "./config.js";
|
|
9
|
-
export type { ActionGuardConfig, StuckGuardConfig, DoneGuardConfig, SlopGuardConfig, Threshold, WardenConfig, WardenMode, LoadOptions } from "./config.js";
|
|
12
|
+
export type { ActionGuardConfig, StuckGuardConfig, DoneGuardConfig, SlopGuardConfig, ProseConfig, Threshold, WardenConfig, WardenMode, LoadOptions } from "./config.js";
|
|
10
13
|
export { commandOf, outputReportsFailure, COMMAND_TOOLS } from "./tools.js";
|
|
11
14
|
export type { CommandView } from "./tools.js";
|
|
12
|
-
export { renderTemplate, actionTokens, stuckTokens, doneTokens, defaultWidgetConfig, DEFAULT_TEMPLATES, TOKEN_NAMES } from "./widget.js";
|
|
15
|
+
export { renderTemplate, actionTokens, stuckTokens, doneTokens, proseTokens, defaultWidgetConfig, DEFAULT_TEMPLATES, TOKEN_NAMES } from "./widget.js";
|
|
13
16
|
export type { WidgetConfig, WidgetPlacement, Tokens } from "./widget.js";
|
|
14
|
-
export { Trace, actionDetails, stuckDetails, doneDetails } from "./trace.js";
|
|
17
|
+
export { Trace, actionDetails, stuckDetails, doneDetails, proseDetails } from "./trace.js";
|
|
15
18
|
export type { TraceEntry, GuardName } from "./trace.js";
|
package/dist/index.js
CHANGED
|
@@ -1,8 +1,10 @@
|
|
|
1
1
|
export { evaluateAction, describeAction, matchPatterns, isReadOnlyCommand, buildRequest, formatVerdict, steerReason, textApproves, questions, slopQuestions, approvalQuestion } from "./guard.js";
|
|
2
|
+
export { SLOP_SYMPTOMS, SLOP_LABELS } from "./guard.js";
|
|
3
|
+
export { evaluateProse, buildProseRequest, proseQuestions, proseNudge, describeAudience, ProseTrend, PROSE_SYMPTOMS, PROSE_LABELS } from "./prose.js";
|
|
2
4
|
export { AttemptWindow, makeAttempt, resultFailed, resultText, evaluateStuck, buildStuckRequest, stuckQuestions, stuckNudge, formatStuck } from "./stuck.js";
|
|
3
5
|
export { classifyToolResult, recordOutcome, emptyEvidence, needsDoneCheck, finalAssistantText, evaluateDone, buildDoneRequest, doneQuestions, doneNudge, formatDone } from "./done.js";
|
|
4
6
|
export { redact } from "./redact.js";
|
|
5
7
|
export { defaultConfig, loadConfig, applyUserOverrides, applyProjectOverrides, isMode, userConfigPath, projectConfigPath, readUserConfig, writeUserConfig, setUserSetting, PACKAGE_NAME } from "./config.js";
|
|
6
8
|
export { commandOf, outputReportsFailure, COMMAND_TOOLS } from "./tools.js";
|
|
7
|
-
export { renderTemplate, actionTokens, stuckTokens, doneTokens, defaultWidgetConfig, DEFAULT_TEMPLATES, TOKEN_NAMES } from "./widget.js";
|
|
8
|
-
export { Trace, actionDetails, stuckDetails, doneDetails } from "./trace.js";
|
|
9
|
+
export { renderTemplate, actionTokens, stuckTokens, doneTokens, proseTokens, defaultWidgetConfig, DEFAULT_TEMPLATES, TOKEN_NAMES } from "./widget.js";
|
|
10
|
+
export { Trace, actionDetails, stuckDetails, doneDetails, proseDetails } from "./trace.js";
|
package/dist/panel.js
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
import { Key, matchesKey, truncateToWidth, wrapTextWithAnsi } from "@earendil-works/pi-tui";
|
|
2
|
-
const LEVEL_COLOR = { allow: "success", ok: "success", warn: "warning", unverified: "warning", confirm: "error", stuck: "error", "false claim": "error" };
|
|
2
|
+
const LEVEL_COLOR = { allow: "success", ok: "success", warn: "warning", unverified: "warning", nudged: "warning", confirm: "error", stuck: "error", "false claim": "error" };
|
|
3
3
|
/**
|
|
4
4
|
* Side panel listing the trace newest-first, live-updating while open. Keys: ↑/↓/j/k scroll a line, PgUp/PgDn a page,
|
|
5
5
|
* Home/End jump, c clears the trace, Esc or q closes. Wheel scrolling works in fullscreen mode.
|
package/dist/prose.d.ts
ADDED
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
import type { ProseConfig } from "./config.js";
|
|
2
|
+
import type { Judge } from "./guard.js";
|
|
3
|
+
export type ProseSymptom = "wordy" | "cliches" | "jargon";
|
|
4
|
+
export declare const PROSE_SYMPTOMS: readonly ProseSymptom[];
|
|
5
|
+
export declare const proseQuestions: {
|
|
6
|
+
wordy: import("pi-typesafe").NoulQuestion;
|
|
7
|
+
cliches: import("pi-typesafe").NoulQuestion;
|
|
8
|
+
jargon: import("pi-typesafe").NoulQuestion;
|
|
9
|
+
};
|
|
10
|
+
export declare const PROSE_LABELS: Record<ProseSymptom, string>;
|
|
11
|
+
/** Map the two built-in audience names to descriptions Jev can judge against; anything else is used as written. */
|
|
12
|
+
export declare function describeAudience(audience: string): string;
|
|
13
|
+
export interface ProseVerdict {
|
|
14
|
+
scores?: Record<ProseSymptom, number>;
|
|
15
|
+
/** Symptoms at or above the threshold for this reply, strongest first. */
|
|
16
|
+
flagged: ProseSymptom[];
|
|
17
|
+
/** Set by the caller when a nudge was sent. */
|
|
18
|
+
nudged?: boolean;
|
|
19
|
+
model?: string;
|
|
20
|
+
elapsedMs?: number;
|
|
21
|
+
error?: string;
|
|
22
|
+
}
|
|
23
|
+
export declare function buildProseRequest(task: string | undefined, reply: string, audience: string): {
|
|
24
|
+
state: {
|
|
25
|
+
task: string;
|
|
26
|
+
audience: string;
|
|
27
|
+
reply: string;
|
|
28
|
+
};
|
|
29
|
+
questions: {
|
|
30
|
+
wordy: import("pi-typesafe").NoulQuestion;
|
|
31
|
+
cliches: import("pi-typesafe").NoulQuestion;
|
|
32
|
+
jargon: import("pi-typesafe").NoulQuestion;
|
|
33
|
+
};
|
|
34
|
+
};
|
|
35
|
+
export interface ProseOptions {
|
|
36
|
+
config: ProseConfig;
|
|
37
|
+
judge: Judge;
|
|
38
|
+
timeoutMs: number;
|
|
39
|
+
signal?: AbortSignal | undefined;
|
|
40
|
+
}
|
|
41
|
+
export declare function evaluateProse(task: string | undefined, reply: string, options: ProseOptions): Promise<ProseVerdict>;
|
|
42
|
+
/**
|
|
43
|
+
* Remembers the last three replies' symptoms. A nudge needs a symptom in `trend` of them, so one long answer to a long
|
|
44
|
+
* question is not punished, and after a nudge the next two replies are given time to change.
|
|
45
|
+
*/
|
|
46
|
+
export declare class ProseTrend {
|
|
47
|
+
private readonly history;
|
|
48
|
+
private cooldown;
|
|
49
|
+
readonly counts: Record<ProseSymptom, number>;
|
|
50
|
+
record(flagged: readonly ProseSymptom[]): void;
|
|
51
|
+
/** Symptoms that crossed the trend requirement, or an empty list during cool-down. */
|
|
52
|
+
due(trend: number): ProseSymptom[];
|
|
53
|
+
markNudged(): void;
|
|
54
|
+
reset(): void;
|
|
55
|
+
}
|
|
56
|
+
/** Queued for the next user prompt, so it shapes the next reply without spending a turn. */
|
|
57
|
+
export declare function proseNudge(symptoms: readonly ProseSymptom[], audience: string, counts: Record<ProseSymptom, number>): string;
|
package/dist/prose.js
ADDED
|
@@ -0,0 +1,98 @@
|
|
|
1
|
+
import { noul, TypeSafeIntegrationError } from "pi-typesafe";
|
|
2
|
+
import { redact } from "./redact.js";
|
|
3
|
+
export const PROSE_SYMPTOMS = ["wordy", "cliches", "jargon"];
|
|
4
|
+
export const proseQuestions = {
|
|
5
|
+
wordy: noul("Is `reply` longer than its content requires: a preamble, restating the request, summarising what it just said, or filler sentences that add no information?", {
|
|
6
|
+
true: "Yes: it opens with what it is about to do, repeats the question, ends with a summary of the same points, or pads with sentences that could be removed without losing anything.",
|
|
7
|
+
false: "No: each sentence carries information the reader needs; length matches the substance.",
|
|
8
|
+
}),
|
|
9
|
+
cliches: noul("Does `reply` use assistant clichés or filler phrasing: \"Great question\", \"Certainly!\", \"I hope this helps\", \"it's worth noting\", \"delve\", \"let me know if\", unrequested bullet lists of caveats, or emoji headings?", {
|
|
10
|
+
true: "Yes: one or more such phrases or patterns are present.",
|
|
11
|
+
false: "No: the wording is direct and specific to the subject.",
|
|
12
|
+
}),
|
|
13
|
+
jargon: noul("Would a reader described by `audience` struggle with `reply` because of unexplained technical terms or assumed background?", {
|
|
14
|
+
true: "Yes: terms, acronyms, or concepts appear without explanation that this audience would not know.",
|
|
15
|
+
false: "No: the reply matches what this audience can follow, or the audience is technical.",
|
|
16
|
+
}),
|
|
17
|
+
};
|
|
18
|
+
export const PROSE_LABELS = {
|
|
19
|
+
wordy: "longer than the content needs (preamble, restating, summarising, filler)",
|
|
20
|
+
cliches: "assistant clichés or filler phrasing",
|
|
21
|
+
jargon: "unexplained technical terms for the configured audience",
|
|
22
|
+
};
|
|
23
|
+
const AUDIENCES = {
|
|
24
|
+
technical: "a software developer who knows this codebase and its tools",
|
|
25
|
+
plain: "a non-programmer who owns the product and reads the reply as a status update",
|
|
26
|
+
};
|
|
27
|
+
/** Map the two built-in audience names to descriptions Jev can judge against; anything else is used as written. */
|
|
28
|
+
export function describeAudience(audience) {
|
|
29
|
+
return AUDIENCES[audience.toLowerCase()] ?? audience;
|
|
30
|
+
}
|
|
31
|
+
export function buildProseRequest(task, reply, audience) {
|
|
32
|
+
return {
|
|
33
|
+
state: {
|
|
34
|
+
task: task?.trim() ? (task.trim().length > 1000 ? `${task.trim().slice(0, 1000)}…` : task.trim()) : "(no user request recorded in this session)",
|
|
35
|
+
audience: describeAudience(audience),
|
|
36
|
+
reply: redact(reply.length > 2500 ? `${reply.slice(0, 2500)}…` : reply),
|
|
37
|
+
},
|
|
38
|
+
questions: proseQuestions,
|
|
39
|
+
};
|
|
40
|
+
}
|
|
41
|
+
export async function evaluateProse(task, reply, options) {
|
|
42
|
+
try {
|
|
43
|
+
const timeout = AbortSignal.timeout(options.timeoutMs);
|
|
44
|
+
const result = await options.judge.evaluate(buildProseRequest(task, reply, options.config.audience), { signal: options.signal ? AbortSignal.any([options.signal, timeout]) : timeout });
|
|
45
|
+
const scores = { wordy: result.answers.wordy.noul, cliches: result.answers.cliches.noul, jargon: result.answers.jargon.noul };
|
|
46
|
+
const flagged = PROSE_SYMPTOMS.filter(symptom => scores[symptom] >= options.config.threshold).sort((a, b) => scores[b] - scores[a]);
|
|
47
|
+
return { scores, flagged, model: result.model, elapsedMs: result.elapsedMs };
|
|
48
|
+
}
|
|
49
|
+
catch (error) {
|
|
50
|
+
return { flagged: [], error: error instanceof TypeSafeIntegrationError ? error.message : "TypeSafe request failed." };
|
|
51
|
+
}
|
|
52
|
+
}
|
|
53
|
+
/**
|
|
54
|
+
* Remembers the last three replies' symptoms. A nudge needs a symptom in `trend` of them, so one long answer to a long
|
|
55
|
+
* question is not punished, and after a nudge the next two replies are given time to change.
|
|
56
|
+
*/
|
|
57
|
+
export class ProseTrend {
|
|
58
|
+
history = [];
|
|
59
|
+
cooldown = 0;
|
|
60
|
+
counts = { wordy: 0, cliches: 0, jargon: 0 };
|
|
61
|
+
record(flagged) {
|
|
62
|
+
this.history.push([...flagged]);
|
|
63
|
+
if (this.history.length > 3)
|
|
64
|
+
this.history.shift();
|
|
65
|
+
for (const symptom of flagged)
|
|
66
|
+
this.counts[symptom]++;
|
|
67
|
+
if (this.cooldown > 0)
|
|
68
|
+
this.cooldown--;
|
|
69
|
+
}
|
|
70
|
+
/** Symptoms that crossed the trend requirement, or an empty list during cool-down. */
|
|
71
|
+
due(trend) {
|
|
72
|
+
if (this.cooldown > 0)
|
|
73
|
+
return [];
|
|
74
|
+
return PROSE_SYMPTOMS.filter(symptom => this.history.filter(set => set.includes(symptom)).length >= trend);
|
|
75
|
+
}
|
|
76
|
+
markNudged() {
|
|
77
|
+
this.cooldown = 2;
|
|
78
|
+
}
|
|
79
|
+
reset() {
|
|
80
|
+
this.history.length = 0;
|
|
81
|
+
this.cooldown = 0;
|
|
82
|
+
for (const symptom of PROSE_SYMPTOMS)
|
|
83
|
+
this.counts[symptom] = 0;
|
|
84
|
+
}
|
|
85
|
+
}
|
|
86
|
+
/** Queued for the next user prompt, so it shapes the next reply without spending a turn. */
|
|
87
|
+
export function proseNudge(symptoms, audience, counts) {
|
|
88
|
+
const parts = symptoms.map(symptom => `${PROSE_LABELS[symptom]}${counts[symptom] >= 3 ? ` (${counts[symptom]} replies this session)` : ""}`);
|
|
89
|
+
const fixes = [];
|
|
90
|
+
if (symptoms.includes("wordy"))
|
|
91
|
+
fixes.push("lead with the answer, cut preambles and closing summaries, and stop when the information is complete");
|
|
92
|
+
if (symptoms.includes("cliches"))
|
|
93
|
+
fixes.push("drop filler phrases and unrequested caveat lists");
|
|
94
|
+
if (symptoms.includes("jargon"))
|
|
95
|
+
fixes.push(`write for ${describeAudience(audience)}: explain or replace technical terms`);
|
|
96
|
+
const standing = symptoms.some(symptom => counts[symptom] >= 3) ? " Treat this as a standing rule for the rest of the session." : "";
|
|
97
|
+
return `pi-warden: your recent replies read as ${parts.join("; ")}. From the next reply on, ${fixes.join("; ")}.${standing}`;
|
|
98
|
+
}
|
package/dist/trace.d.ts
CHANGED
|
@@ -1,7 +1,8 @@
|
|
|
1
1
|
import type { DoneVerdict } from "./done.js";
|
|
2
2
|
import type { Verdict } from "./guard.js";
|
|
3
|
+
import type { ProseVerdict } from "./prose.js";
|
|
3
4
|
import type { Attempt, StuckVerdict } from "./stuck.js";
|
|
4
|
-
export type GuardName = "action" | "stuck" | "done";
|
|
5
|
+
export type GuardName = "action" | "stuck" | "done" | "prose";
|
|
5
6
|
export interface TraceEntry {
|
|
6
7
|
at: number;
|
|
7
8
|
guard: GuardName;
|
|
@@ -27,3 +28,4 @@ export declare function actionDetails(verdict: Verdict, extra?: {
|
|
|
27
28
|
}): string[];
|
|
28
29
|
export declare function stuckDetails(verdict: StuckVerdict, attempts: readonly Attempt[], told?: string): string[];
|
|
29
30
|
export declare function doneDetails(verdict: DoneVerdict, finalMessage: string, told?: string): string[];
|
|
31
|
+
export declare function proseDetails(verdict: ProseVerdict, reply: string, audience: string, told?: string): string[];
|
package/dist/trace.js
CHANGED
|
@@ -43,7 +43,7 @@ export function actionDetails(verdict, extra = {}) {
|
|
|
43
43
|
lines.push(`jev: irreversible ${percent(judgment.irreversible)} · off-task ${percent(judgment.offTask)} · ${judgment.scope.replace(/_/g, " ")} (${percent(judgment.scopeConfidence)})${judgment.approved !== undefined ? ` · approved ${percent(judgment.approved)}` : ""} · ${judgment.model} · ${judgment.elapsedMs} ms`);
|
|
44
44
|
}
|
|
45
45
|
if (verdict.slop)
|
|
46
|
-
lines.push(`slop:
|
|
46
|
+
lines.push(`slop: stub ${percent(verdict.slop.stub)} · comments ${percent(verdict.slop.comments)} · dead ${percent(verdict.slop.dead)} · hedging ${percent(verdict.slop.hedging)}${verdict.slopReasons?.length ? ` → ${verdict.slopReasons.join("; ")}` : ""}`);
|
|
47
47
|
if (verdict.reasons.length)
|
|
48
48
|
lines.push(`why: ${verdict.reasons.join("; ")}`);
|
|
49
49
|
if (verdict.error)
|
|
@@ -79,3 +79,15 @@ export function doneDetails(verdict, finalMessage, told) {
|
|
|
79
79
|
lines.push(`agent told: ${clip(told, 400)}`);
|
|
80
80
|
return lines;
|
|
81
81
|
}
|
|
82
|
+
export function proseDetails(verdict, reply, audience, told) {
|
|
83
|
+
const lines = [`reply: ${clip(reply.replace(/\s+/g, " "), 300)}`, `audience: ${audience}`];
|
|
84
|
+
if (verdict.scores)
|
|
85
|
+
lines.push(`jev: wordy ${percent(verdict.scores.wordy)} · clichés ${percent(verdict.scores.cliches)} · jargon ${percent(verdict.scores.jargon)}${verdict.model ? ` · ${verdict.model} · ${verdict.elapsedMs} ms` : ""}`);
|
|
86
|
+
if (verdict.flagged.length)
|
|
87
|
+
lines.push(`flagged: ${verdict.flagged.join(", ")}`);
|
|
88
|
+
if (verdict.error)
|
|
89
|
+
lines.push(`typesafe: ${verdict.error}`);
|
|
90
|
+
if (told)
|
|
91
|
+
lines.push(`agent told: ${clip(told, 400)}`);
|
|
92
|
+
return lines;
|
|
93
|
+
}
|
package/dist/widget.d.ts
CHANGED
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
import type { DoneVerdict } from "./done.js";
|
|
2
2
|
import type { Verdict } from "./guard.js";
|
|
3
|
+
import type { ProseVerdict } from "./prose.js";
|
|
3
4
|
import type { StuckVerdict } from "./stuck.js";
|
|
4
5
|
export type WidgetPlacement = "aboveEditor" | "belowEditor";
|
|
5
6
|
export interface WidgetConfig {
|
|
@@ -11,11 +12,13 @@ export interface WidgetConfig {
|
|
|
11
12
|
action: string;
|
|
12
13
|
stuck: string;
|
|
13
14
|
done: string;
|
|
15
|
+
prose: string;
|
|
14
16
|
}
|
|
15
17
|
export declare const DEFAULT_TEMPLATES: {
|
|
16
|
-
readonly action: "warden · {tool} · irreversible {irreversible} · off-task {offTask} · {scope} · slop {
|
|
18
|
+
readonly action: "warden · {tool} · irreversible {irreversible} · off-task {offTask} · {scope} · slop: {slop} · patterns: {patterns} · {flags} · {level}";
|
|
17
19
|
readonly stuck: "warden · stuck · {failures} failures · same strategy {sameStrategy} · change {approachChange} · progress {progress} · {flags} · {status}";
|
|
18
20
|
readonly done: "warden · done-check · {changes} changes · {checksPassed}/{checks} checks passed · claims done {claimsDone} · claims verified {claimsVerified} · checks apply {checksApply} · {outcome} · {status}";
|
|
21
|
+
readonly prose: "warden · prose · wordy {wordy} · clichés {cliches} · jargon {jargon} · {flags} · {status}";
|
|
19
22
|
};
|
|
20
23
|
export declare function defaultWidgetConfig(): WidgetConfig;
|
|
21
24
|
export type Tokens = Record<string, string | undefined>;
|
|
@@ -27,9 +30,11 @@ export declare function renderTemplate(template: string, tokens: Tokens): string
|
|
|
27
30
|
export declare function actionTokens(verdict: Verdict, at?: number): Tokens;
|
|
28
31
|
export declare function stuckTokens(verdict: StuckVerdict, at?: number): Tokens;
|
|
29
32
|
export declare function doneTokens(verdict: DoneVerdict, at?: number): Tokens;
|
|
33
|
+
export declare function proseTokens(verdict: ProseVerdict, at?: number): Tokens;
|
|
30
34
|
/** Token names users can put in templates, for /warden status and the README. */
|
|
31
35
|
export declare const TOKEN_NAMES: {
|
|
32
|
-
readonly action: readonly ["tool", "level", "source", "irreversible", "offTask", "scope", "approved", "
|
|
36
|
+
readonly action: readonly ["tool", "level", "source", "irreversible", "offTask", "scope", "approved", "slop", "slopStub", "slopComments", "slopDead", "slopHedging", "patterns", "reasons", "path", "model", "ms", "flags", "time", "guard"];
|
|
37
|
+
readonly prose: readonly ["wordy", "cliches", "jargon", "status", "reasons", "model", "ms", "flags", "time", "guard"];
|
|
33
38
|
readonly stuck: readonly ["failures", "sameStrategy", "approachChange", "progress", "status", "source", "reasons", "model", "ms", "flags", "time", "guard"];
|
|
34
39
|
readonly done: readonly ["changes", "checks", "checksPassed", "claimsDone", "claimsVerified", "checksApply", "outcome", "status", "reasons", "model", "ms", "flags", "time", "guard"];
|
|
35
40
|
};
|
package/dist/widget.js
CHANGED
|
@@ -1,7 +1,8 @@
|
|
|
1
1
|
export const DEFAULT_TEMPLATES = {
|
|
2
|
-
action: "warden · {tool} · irreversible {irreversible} · off-task {offTask} · {scope} · slop {
|
|
2
|
+
action: "warden · {tool} · irreversible {irreversible} · off-task {offTask} · {scope} · slop: {slop} · patterns: {patterns} · {flags} · {level}",
|
|
3
3
|
stuck: "warden · stuck · {failures} failures · same strategy {sameStrategy} · change {approachChange} · progress {progress} · {flags} · {status}",
|
|
4
4
|
done: "warden · done-check · {changes} changes · {checksPassed}/{checks} checks passed · claims done {claimsDone} · claims verified {claimsVerified} · checks apply {checksApply} · {outcome} · {status}",
|
|
5
|
+
prose: "warden · prose · wordy {wordy} · clichés {cliches} · jargon {jargon} · {flags} · {status}",
|
|
5
6
|
};
|
|
6
7
|
export function defaultWidgetConfig() {
|
|
7
8
|
return { enabled: true, placement: "aboveEditor", shortcut: "ctrl+shift+w", ...DEFAULT_TEMPLATES };
|
|
@@ -45,8 +46,11 @@ export function actionTokens(verdict, at = Date.now()) {
|
|
|
45
46
|
offTask: fixed(verdict.judgment?.offTask),
|
|
46
47
|
scope: verdict.judgment?.scope.replace(/_/g, " "),
|
|
47
48
|
approved: fixed(verdict.judgment?.approved),
|
|
48
|
-
|
|
49
|
-
slopStub: fixed(verdict.slop?.
|
|
49
|
+
slop: verdict.slopSymptoms?.length ? verdict.slopSymptoms.map(symptom => `${symptom} ${verdict.slop[symptom].toFixed(2)}`).join(", ") : verdict.slop ? "none" : undefined,
|
|
50
|
+
slopStub: fixed(verdict.slop?.stub),
|
|
51
|
+
slopComments: fixed(verdict.slop?.comments),
|
|
52
|
+
slopDead: fixed(verdict.slop?.dead),
|
|
53
|
+
slopHedging: fixed(verdict.slop?.hedging),
|
|
50
54
|
patterns: verdict.patterns.length ? verdict.patterns.map(hit => hit.id).join(", ") : undefined,
|
|
51
55
|
reasons: verdict.reasons.length ? verdict.reasons.join("; ") : undefined,
|
|
52
56
|
path: verdict.summary.path,
|
|
@@ -93,9 +97,24 @@ export function doneTokens(verdict, at = Date.now()) {
|
|
|
93
97
|
flags: verdict.error ? "typesafe error" : undefined,
|
|
94
98
|
};
|
|
95
99
|
}
|
|
100
|
+
export function proseTokens(verdict, at = Date.now()) {
|
|
101
|
+
return {
|
|
102
|
+
guard: "prose",
|
|
103
|
+
time: time(at),
|
|
104
|
+
wordy: fixed(verdict.scores?.wordy),
|
|
105
|
+
cliches: fixed(verdict.scores?.cliches),
|
|
106
|
+
jargon: fixed(verdict.scores?.jargon),
|
|
107
|
+
status: verdict.nudged ? "nudged" : verdict.flagged.length ? verdict.flagged.join(", ") : "ok",
|
|
108
|
+
reasons: verdict.flagged.length ? verdict.flagged.join(", ") : undefined,
|
|
109
|
+
model: verdict.model,
|
|
110
|
+
ms: verdict.elapsedMs === undefined ? undefined : String(verdict.elapsedMs),
|
|
111
|
+
flags: verdict.error ? "typesafe error" : undefined,
|
|
112
|
+
};
|
|
113
|
+
}
|
|
96
114
|
/** Token names users can put in templates, for /warden status and the README. */
|
|
97
115
|
export const TOKEN_NAMES = {
|
|
98
|
-
action: ["tool", "level", "source", "irreversible", "offTask", "scope", "approved", "
|
|
116
|
+
action: ["tool", "level", "source", "irreversible", "offTask", "scope", "approved", "slop", "slopStub", "slopComments", "slopDead", "slopHedging", "patterns", "reasons", "path", "model", "ms", "flags", "time", "guard"],
|
|
117
|
+
prose: ["wordy", "cliches", "jargon", "status", "reasons", "model", "ms", "flags", "time", "guard"],
|
|
99
118
|
stuck: ["failures", "sameStrategy", "approachChange", "progress", "status", "source", "reasons", "model", "ms", "flags", "time", "guard"],
|
|
100
119
|
done: ["changes", "checks", "checksPassed", "claimsDone", "claimsVerified", "checksApply", "outcome", "status", "reasons", "model", "ms", "flags", "time", "guard"],
|
|
101
120
|
};
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pi-warden",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.4.0",
|
|
4
4
|
"description": "Guardrails for Pi built on pi-typesafe that steer the agent instead of interrupting you: Jev judges irreversible and off-task tool calls before they run, detects stuck loops, checks unverified done claims, and flags slop.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "MIT",
|