pi-warden 0.9.1 → 0.10.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -74,6 +74,8 @@ Runs on `tool_call`, before the tool executes.
74
74
 
75
75
  The action guard receives your latest message plus up to eight earlier user and assistant messages (750 redacted characters each) so follow-ups and side comments do not replace the task. Sibling tool calls in one assistant message are judged together in one round trip. If TypeSafe cannot answer, the call is allowed with a warning (`failOpen: true`; set it to `false` to hold instead).
76
76
 
77
+ **Hold feedback.** What you do next labels each judgment, so hold precision is measured on your sessions rather than assumed. A hold your reply releases (or the confirm dialog allows) was a false positive; a hold you decline, or that nobody approves after you replied and the next turn ended, stood. An allowed call your next message tells the agent to stop, undo, or revert was a miss: one `regretted` question rides the first action request after your reply, with the redacted summaries of last turn's allowed calls (a locator names the one when there are several); offline, a stop-word heuristic stands in. `/warden status` shows the counts and the precision, the trace entry of each call gets its outcome, and every judged call is written with its scores and outcome to an owner-only per-session file under `~/.pi/agent/pi-warden/holds/` (tool, pattern ids, scores, level, mode, outcome; never the command). `"action": { "feedbackLog": false }` keeps the counts and skips the file.
78
+
77
79
  ### Rules
78
80
 
79
81
  Write your project's rules as Markdown headings in `pi-warden.md` at the project root (a starter file with a dozen rules is in [`examples/pi-warden.md`](examples/pi-warden.md)):
@@ -153,7 +155,7 @@ Nudges from the rules, slop, stuck, done, and prose guards are custom messages i
153
155
 
154
156
  | Command | Effect |
155
157
  | --- | --- |
156
- | `/warden status` | Guard state, consent and key source, session counts, thresholds, rules source, config paths, last verdicts |
158
+ | `/warden status` | Guard state, consent and key source, session counts, thresholds, hold precision and feedback log, rules source, config paths, last verdicts |
157
159
  | `/warden enable` | Data notice, key prompt if none is stored, consent saved |
158
160
  | `/warden disable` | Stop Jev judgments; pattern checks continue |
159
161
  | `/warden mode steer\|confirm\|advise` | How holds are handled; without an argument, show the current mode |
@@ -200,7 +202,8 @@ User file `~/.pi/agent/pi-warden/config.json` (owner-only). Missing keys use the
200
202
  "tools": ["bash", "powershell", "ctx_execute", "ctx_batch_execute", "ctx_execute_file", "write", "edit"],
201
203
  "failOpen": true,
202
204
  "irreversible": { "warn": 0.5, "confirm": 0.7 },
203
- "offTask": { "warn": 0.6, "confirm": 0.85 }
205
+ "offTask": { "warn": 0.6, "confirm": 0.85 },
206
+ "feedbackLog": true
204
207
  },
205
208
  "rules": {
206
209
  "enabled": true,
@@ -249,13 +252,15 @@ A wince-style setup for a backend repo (the full version is [`examples/pi-warden
249
252
 
250
253
  With consent, requests go to `https://api.typesafe.ai`. What is sent:
251
254
 
252
- - **Action guard**: your latest prompt (1500 characters), up to eight earlier user and assistant messages (750 redacted characters each), the tool name, the command (2000 characters) or the file path (relative inside the project, `~`-shortened outside), whether the file exists, a 1500-character head/middle/tail sample of a `write`, the first three edit pairs (400 characters each) of an `edit`.
255
+ - **Action guard**: your latest prompt (1500 characters), up to eight earlier user and assistant messages (750 redacted characters each), the tool name, the command (2000 characters) or the file path (relative inside the project, `~`-shortened outside), whether the file exists, a 1500-character head/middle/tail sample of a `write`, the first three edit pairs (400 characters each) of an `edit`. On the first guarded call after your reply, the tool names and commands (300 characters) or paths of up to six calls allowed in the previous turn, for the regret question.
253
256
  - **Rules**: the project-relative path, a 6000-character sample of a `write` or each edit's new text (1500 characters) with about 40 lines of the current file around the replaced text, and the rule text from your rules file or the condensed fallback document (`rules.maxChars`). No task text. Files under `rules.exclude` are never sent.
254
257
  - **Stuck**: the last 12 tool calls (300 characters each) with 400-character output tails.
255
258
  - **Done-check and prose**: the agent's final message (2000 and 2500 characters), the run's check commands, the audience description.
256
259
  - **Output checks**: a redacted head/tail sample up to 6000 characters plus size, line counts, and tool name.
257
260
  - **Nothing** for duplicate detection, the runaway guard, sensitive-path notes, or pattern checks.
258
261
 
262
+ The hold feedback log under `~/.pi/agent/pi-warden/holds/` stays on this machine, owner-only: one JSON line per judged call with tool, pattern ids, scores, level, mode, and outcome; never the command, path, or prompt.
263
+
259
264
  Obvious credentials (`Authorization` headers, `TOKEN=` and `SECRET=` assignments, `sk-`, `ghp_`, `AKIA`, JWTs, URL passwords, PEM blocks) are replaced with `[redacted]` before sending. Best-effort; do not rely on it for prompts that contain secrets. Text steered to the agent names the tool, the reasons, and the scores, not the command. UI errors never include upstream response bodies or keys. Judgments are model output; thresholds are yours to tune.
260
265
 
261
266
  ## After updating the package
@@ -290,7 +295,7 @@ What spans calls in a session lives in `ActionGuard` (hold, reply, retry approva
290
295
  ```bash
291
296
  npm install
292
297
  npm run check # typecheck, offline tests (mocked transport), build
293
- npm run test:live # billable synthetic judgments across the guards; pass action|slop|approval|stuck|done|security|context for one group
298
+ npm run test:live # billable synthetic judgments across the guards; pass action|slop|approval|regret|stuck|done|security|context for one group
294
299
  node scripts/rules-cases.mjs # 13 billable cases against an 8-rule fixture file
295
300
  node scripts/slop-cases.mjs # 22 billable cases for the slop and prose questions
296
301
  node scripts/security-cases.mjs # 9 output-security and task-continuity cases
@@ -1,5 +1,5 @@
1
1
  import type { ActionGuardConfig, SecurityConfig, SlopGuardConfig } from "./config.js";
2
- import type { TaskMessage, Verdict } from "./guard.js";
2
+ import type { PreviousAction, TaskMessage, Verdict } from "./guard.js";
3
3
  import type { Judge } from "./jev.js";
4
4
  /** One tool call as the agent proposed it. `id` is Pi's tool call id, stable across hooks and retries. */
5
5
  export interface ToolCallRef {
@@ -23,6 +23,8 @@ export interface InspectOptions {
23
23
  signal?: AbortSignal | undefined;
24
24
  slop?: SlopGuardConfig | undefined;
25
25
  security?: SecurityConfig | undefined;
26
+ /** Calls allowed in the previous turn; the regret question about them rides this call's request, never a sibling's. */
27
+ previousActions?: readonly PreviousAction[] | undefined;
26
28
  }
27
29
  /**
28
30
  * The Action guard for one session. `evaluateAction` judges a single call; this module owns what spans calls:
@@ -20,7 +20,7 @@ export class ActionGuard {
20
20
  const { task } = conversation;
21
21
  // A hold happened under an earlier prompt and the user has since replied: ask whether the reply approves this action.
22
22
  const retryAfterHold = this.holdPending && this.lastHoldPrompt !== task;
23
- const judgeCall = (tool, input) => evaluateAction({ tool, input, cwd: options.cwd, task, context: conversation.context }, { config: options.config, judge: options.judge, signal: options.signal, slop: options.slop, security: options.security, retryAfterHold });
23
+ const judgeCall = (tool, input, previousActions) => evaluateAction({ tool, input, cwd: options.cwd, task, context: conversation.context }, { config: options.config, judge: options.judge, signal: options.signal, slop: options.slop, security: options.security, retryAfterHold, previousActions });
24
24
  // A retry after a hold stays sequential because an approval consumed by one sibling changes the question for the next.
25
25
  if (options.judge && !retryAfterHold) {
26
26
  for (const sibling of conversation.siblings ?? []) {
@@ -33,7 +33,7 @@ export class ActionGuard {
33
33
  }
34
34
  const key = JSON.stringify(call.input);
35
35
  const ready = this.prejudged.get(call.id);
36
- const pending = ready && !ready.used && ready.key === key && !retryAfterHold ? ready.verdict : judgeCall(call.tool, call.input);
36
+ const pending = ready && !ready.used && ready.key === key && !retryAfterHold ? ready.verdict : judgeCall(call.tool, call.input, options.previousActions);
37
37
  this.prejudged.set(call.id, { key, verdict: pending, used: true });
38
38
  const verdict = await pending;
39
39
  if (retryAfterHold && !options.judge && verdict.level === "confirm" && textApproves(task)) {
package/dist/config.d.ts CHANGED
@@ -15,6 +15,8 @@ export interface ActionGuardConfig {
15
15
  timeoutMs: number;
16
16
  irreversible: Threshold;
17
17
  offTask: Threshold;
18
+ /** Write each judged call and what the user did next (approved, declined, re-planned, regretted) to an owner-only per-session file under the agent directory; redacted, never the command. */
19
+ feedbackLog: boolean;
18
20
  }
19
21
  export interface StuckGuardConfig {
20
22
  enabled: boolean;
package/dist/config.js CHANGED
@@ -25,6 +25,7 @@ export function defaultConfig() {
25
25
  timeoutMs: 5000,
26
26
  irreversible: { warn: 0.5, confirm: 0.7 },
27
27
  offTask: { warn: 0.6, confirm: 0.85 },
28
+ feedbackLog: true,
28
29
  },
29
30
  stuck: { enabled: true, window: 12, minFailures: 3, cooldown: 3, sameStrategy: 0.7, nudge: true },
30
31
  done: { enabled: true, claimsDone: 0.7, nudge: true },
@@ -92,6 +93,7 @@ function applyAction(base, raw, timeoutMs) {
92
93
  timeoutMs,
93
94
  irreversible: threshold(raw.irreversible, base.irreversible),
94
95
  offTask: threshold(raw.offTask, base.offTask),
96
+ feedbackLog: boolean(raw.feedbackLog, base.feedbackLog),
95
97
  };
96
98
  }
97
99
  function applyStuck(base, raw) {
@@ -2,7 +2,7 @@ import type { ExtensionAPI, ExtensionContext } from "@earendil-works/pi-coding-a
2
2
  import type { ToolCallRef } from "./action-guard.js";
3
3
  import type { SlopSymptom, Verdict } from "./guard.js";
4
4
  import type { ShapeResult } from "./shape.js";
5
- export declare const disclosure = "With TypeSafe judgments enabled, pi-warden sends to api.typesafe.ai: your latest request and up to eight redacted prior user/assistant text messages for task context, plus a redacted, truncated summary of each guarded bash, write, or edit call before it runs; for a write or edit in a project with a rules file (pi-warden.md, the configured files, or README/CLAUDE/AGENTS as fallback), a larger redacted sample of the written content with the current file around each edit and the rule text; the last few tool calls and output tails when the agent keeps failing; the agent's final message when it reports completion without running checks; and redacted tool-output samples for security and context saving (retention and output format). Compression and duplicate notes store an exact, owner-only copy in a temporary file on this machine. Requests may incur charges. Secret redaction is best-effort. Results are model judgments, not proof or authorization; offline pattern checks stay active either way.";
5
+ export declare const disclosure = "With TypeSafe judgments enabled, pi-warden sends to api.typesafe.ai: your latest request and up to eight redacted prior user/assistant text messages for task context, plus a redacted, truncated summary of each guarded bash, write, or edit call before it runs; for a write or edit in a project with a rules file (pi-warden.md, the configured files, or README/CLAUDE/AGENTS as fallback), a larger redacted sample of the written content with the current file around each edit and the rule text; the last few tool calls and output tails when the agent keeps failing; the agent's final message when it reports completion without running checks; redacted tool-output samples for security and context saving (retention and output format); and, on the first guarded call after your reply, the redacted summaries of the calls allowed in the previous turn, so Jev can say whether your reply regrets one of them. Compression and duplicate notes store an exact, owner-only copy in a temporary file on this machine; the hold feedback log stores tool names, pattern ids, scores, and outcomes (never commands) in an owner-only file under Pi's agent directory. Requests may incur charges. Secret redaction is best-effort. Results are model judgments, not proof or authorization; offline pattern checks stay active either way.";
6
6
  /**
7
7
  * Tool calls of the assistant message being preflighted. Pi runs `tool_call` hooks for sibling calls one after another,
8
8
  * so judging them one request at a time costs one round trip per call; judging them together costs one round trip.
package/dist/extension.js CHANGED
@@ -6,6 +6,7 @@ import * as configModule from "./config.js";
6
6
  import { applyUserOverrides, defaultConfig, isMode, loadConfig, PACKAGE_NAME, projectConfigPath, readUserConfig, setUserSetting, userConfigPath, writeUserConfig } from "./config.js";
7
7
  import { classifyToolResult, doneNudge, emptyEvidence, evaluateDone, finalAssistantText, formatDone, needsDoneCheck, recordOutcome } from "./done.js";
8
8
  import { evaluateAction, formatVerdict, SLOP_LABELS, steerReason } from "./guard.js";
9
+ import { formatHolds, HoldLedger, HoldLog, holdLogPath, outcomeNote, regretsAt, textRegrets } from "./holds.js";
9
10
  import { evaluateProse, proseNudge, ProseTrend } from "./prose.js";
10
11
  import { compressOutput, duplicateNote, evaluateOutput, outputKey, saveOutput, securityNotice } from "./output.js";
11
12
  import { classifyRecall, detectSearchTool, recallInstruction } from "./recall.js";
@@ -19,7 +20,7 @@ import { completeConfig, shapeWarning } from "./shape.js";
19
20
  import { ContextLedger, formatLedger } from "./saver.js";
20
21
  import { actionDetails, doneDetails, proseDetails, rulesDetails, runawayDetails, stuckDetails, Trace } from "./trace.js";
21
22
  import { DEFAULT_TEMPLATES, proseTokens, renderTemplate, TOKEN_NAMES } from "./widget.js";
22
- export const disclosure = "With TypeSafe judgments enabled, pi-warden sends to api.typesafe.ai: your latest request and up to eight redacted prior user/assistant text messages for task context, plus a redacted, truncated summary of each guarded bash, write, or edit call before it runs; for a write or edit in a project with a rules file (pi-warden.md, the configured files, or README/CLAUDE/AGENTS as fallback), a larger redacted sample of the written content with the current file around each edit and the rule text; the last few tool calls and output tails when the agent keeps failing; the agent's final message when it reports completion without running checks; and redacted tool-output samples for security and context saving (retention and output format). Compression and duplicate notes store an exact, owner-only copy in a temporary file on this machine. Requests may incur charges. Secret redaction is best-effort. Results are model judgments, not proof or authorization; offline pattern checks stay active either way.";
23
+ export const disclosure = "With TypeSafe judgments enabled, pi-warden sends to api.typesafe.ai: your latest request and up to eight redacted prior user/assistant text messages for task context, plus a redacted, truncated summary of each guarded bash, write, or edit call before it runs; for a write or edit in a project with a rules file (pi-warden.md, the configured files, or README/CLAUDE/AGENTS as fallback), a larger redacted sample of the written content with the current file around each edit and the rule text; the last few tool calls and output tails when the agent keeps failing; the agent's final message when it reports completion without running checks; redacted tool-output samples for security and context saving (retention and output format); and, on the first guarded call after your reply, the redacted summaries of the calls allowed in the previous turn, so Jev can say whether your reply regrets one of them. Compression and duplicate notes store an exact, owner-only copy in a temporary file on this machine; the hold feedback log stores tool names, pattern ids, scores, and outcomes (never commands) in an owner-only file under Pi's agent directory. Requests may incur charges. Secret redaction is best-effort. Results are model judgments, not proof or authorization; offline pattern checks stay active either way.";
23
24
  const WIDGET = PACKAGE_NAME;
24
25
  const CONFIRM_TEXT_LIMIT = 500;
25
26
  const freshStats = () => ({ inspected: 0, judged: 0, warned: 0, held: 0, approved: 0, slop: 0, ruleChecks: 0, ruleViolations: 0, pathNotes: 0, stuckChecks: 0, stuck: 0, doneChecks: 0, unverified: 0, proseChecks: 0, proseNudges: 0, runaway: 0, errors: 0 });
@@ -148,6 +149,12 @@ export default function wardenExtension(pi) {
148
149
  let lastUi;
149
150
  const actionGuard = new ActionGuard();
150
151
  const rulesGuard = new RulesGuard();
152
+ // Hold feedback: what the user did after each judged call, the trace entry each label lands on, and the per-session log.
153
+ const holds = new HoldLedger();
154
+ const traceOf = new WeakMap();
155
+ let holdLog;
156
+ // Allowed calls of the turn the user just replied to; the regret question about them rides the next action request.
157
+ let regretCandidates = [];
151
158
  let attempts = new AttemptWindow(defaultConfig().stuck.window);
152
159
  let evidence = emptyEvidence();
153
160
  let doneNudged = false;
@@ -224,8 +231,25 @@ export default function wardenExtension(pi) {
224
231
  };
225
232
  const record = (ctx, config, guard, line, details) => {
226
233
  widget.set(guard, line);
227
- trace.push({ at: Date.now(), guard, line, details });
234
+ const entry = { at: Date.now(), guard, line, details };
235
+ trace.push(entry);
228
236
  paint(ctx, config);
237
+ return entry;
238
+ };
239
+ /** Labels landed on earlier calls: their trace entries say so and the session log is rewritten. */
240
+ const noteOutcomes = (config, records) => {
241
+ for (const item of records) {
242
+ const entry = traceOf.get(item);
243
+ if (entry)
244
+ trace.amend(entry, outcomeNote(item));
245
+ }
246
+ if (config.action.feedbackLog && holds.records().length)
247
+ void holdLog?.save(holds.records());
248
+ };
249
+ /** The user's reply was read for regret, by Jev or by the offline heuristic; the candidates are labelled once. */
250
+ const settleRegret = (config, result) => {
251
+ regretCandidates = [];
252
+ noteOutcomes(config, holds.regret(result));
229
253
  };
230
254
  const steer = (config, content, options = { deliverAs: "steer" }) => pi.sendMessage({ customType: `${PACKAGE_NAME}-steer`, content, display: config.steerVisible }, options);
231
255
  /**
@@ -255,6 +279,9 @@ export default function wardenExtension(pi) {
255
279
  panel?.close();
256
280
  actionGuard.reset();
257
281
  rulesGuard.reset();
282
+ holds.reset();
283
+ regretCandidates = [];
284
+ holdLog = new HoldLog(holdLogPath(typeof ctx.sessionManager.getSessionId === "function" ? ctx.sessionManager.getSessionId() : String(process.pid)));
258
285
  attempts.reset();
259
286
  evidence = emptyEvidence();
260
287
  doneNudged = false;
@@ -272,14 +299,20 @@ export default function wardenExtension(pi) {
272
299
  ctx.ui.setWidget(WIDGET, undefined);
273
300
  });
274
301
  // A new user prompt starts a new attempt history and a new done-check budget.
275
- pi.on("before_agent_start", async (_event, ctx) => {
276
- attempts = new AttemptWindow(configFor(ctx).stuck.window);
302
+ pi.on("before_agent_start", async (event, ctx) => {
303
+ const config = configFor(ctx);
304
+ attempts = new AttemptWindow(config.stuck.window);
277
305
  doneNudged = false;
278
306
  runaway.reset();
279
307
  runawayStops = 0;
280
308
  pendingRunaway = undefined;
281
309
  actionGuard.turnEnd();
282
310
  rulesGuard.turnEnd();
311
+ // Holds the user never approved are re-plans now; last turn's allowed calls wait for the regret question.
312
+ noteOutcomes(config, holds.promptArrived());
313
+ regretCandidates = holds.candidates();
314
+ if (regretCandidates.length && !judgeFor(config))
315
+ settleRegret(config, { regretted: textRegrets(event.prompt), via: "text" });
283
316
  });
284
317
  // Each assistant message is judged on its own; Pi does not forward the stream's own "start" event, so this is the reset.
285
318
  pi.on("message_start", async (event) => {
@@ -345,9 +378,12 @@ export default function wardenExtension(pi) {
345
378
  ? rulesGuard.inspect(call, siblings, { cwd: ctx.cwd, config: config.rules, judge, timeoutMs: config.timeoutMs, signal: ctx.signal })
346
379
  : undefined;
347
380
  rulesCheck?.catch(() => undefined);
348
- const verdict = await actionGuard.inspect(call, { task, context: recentTaskContext(ctx), siblings }, { config: config.action, cwd: ctx.cwd, judge, signal: ctx.signal, slop: config.slop, security: config.security });
381
+ const verdict = await actionGuard.inspect(call, { task, context: recentTaskContext(ctx), siblings }, { config: config.action, cwd: ctx.cwd, judge, signal: ctx.signal, slop: config.slop, security: config.security, previousActions: regretCandidates.length ? regretCandidates : undefined });
349
382
  if (verdict.source === "skipped")
350
383
  return;
384
+ if (regretCandidates.length && verdict.judgment?.regretted !== undefined) {
385
+ settleRegret(config, { regretted: regretsAt(verdict.judgment.regretted), target: verdict.judgment.regretTarget, probability: verdict.judgment.regretted, via: "jev" });
386
+ }
351
387
  // Notes for the agent about the content it just wrote: slop, rule violations, and sensitive paths arrive as one message.
352
388
  const notes = [];
353
389
  if (verdict.judgment?.securityRisk !== undefined && verdict.judgment.securityRisk >= config.security.threshold) {
@@ -357,12 +393,23 @@ export default function wardenExtension(pi) {
357
393
  stats.judged++;
358
394
  if (verdict.source === "error")
359
395
  noteError(ctx, verdict.error ?? "TypeSafe request failed.", verdict.errorCode);
360
- if (verdict.approvedByUser)
361
- stats.approved++;
362
396
  const mode = activeMode(config, ctx.hasUI);
397
+ if (verdict.approvedByUser) {
398
+ stats.approved++;
399
+ const released = holds.approved(event.toolName);
400
+ if (released)
401
+ noteOutcomes(config, [released]);
402
+ }
363
403
  const told = verdict.level === "confirm" && mode === "steer" ? steerReason(verdict, { canApprove: judge !== undefined }) : undefined;
364
- if (verdict.source !== "read-only")
365
- record(ctx, config, "action", formatVerdict(verdict, config.widget.action), actionDetails(verdict, { mode, ...(told ? { told } : {}) }));
404
+ const entry = verdict.source !== "read-only" ? record(ctx, config, "action", formatVerdict(verdict, config.widget.action), actionDetails(verdict, { mode, ...(told ? { told } : {}) })) : undefined;
405
+ // What happens to this call is the label for its scores; a decision made in the dialog lands at once, a steer-mode hold waits for the user.
406
+ const track = (held, outcome, via) => {
407
+ if (!entry)
408
+ return;
409
+ const item = holds.record(verdict, { held, mode, outcome, via });
410
+ traceOf.set(item, entry);
411
+ noteOutcomes(config, outcome ? [item] : []);
412
+ };
366
413
  if (verdict.slopSymptoms?.length && verdict.slopReasons) {
367
414
  stats.slop++;
368
415
  for (const symptom of verdict.slopSymptoms)
@@ -403,27 +450,35 @@ export default function wardenExtension(pi) {
403
450
  stats.warned++;
404
451
  if (ctx.hasUI)
405
452
  ctx.ui.notify(`warden · ${event.toolName}: ${verdict.reasons.join("; ")}`, "warning");
453
+ track(false);
406
454
  return undefined;
407
455
  }
408
- if (verdict.level !== "confirm")
456
+ if (verdict.level !== "confirm") {
457
+ track(false);
409
458
  return undefined;
459
+ }
410
460
  const reasons = verdict.reasons.join("; ");
411
461
  if (mode === "advise") {
412
462
  stats.warned++;
413
463
  if (ctx.hasUI)
414
464
  ctx.ui.notify(`warden · ${event.toolName} (advise mode, not held): ${reasons}`, "warning");
465
+ track(false);
415
466
  return undefined;
416
467
  }
417
468
  if (mode === "confirm") {
418
469
  notifyDesktop(ctx, config, `Waiting for you: allow this ${event.toolName} call? ${reasons}`);
419
470
  const allowed = await ctx.ui.confirm(`warden: allow this ${event.toolName} call?`, confirmMessage(verdict), ctx.signal ? { signal: ctx.signal } : {});
420
- if (allowed)
471
+ if (allowed) {
472
+ track(true, "approved", "dialog");
421
473
  return undefined;
474
+ }
422
475
  stats.held++;
476
+ track(true, "declined", "dialog");
423
477
  return { block: true, reason: `pi-warden: the user declined this ${event.toolName} call (${reasons}). Do not retry it unchanged; ask the user how to proceed.` };
424
478
  }
425
479
  stats.held++;
426
480
  actionGuard.hold(task);
481
+ track(true);
427
482
  if (ctx.hasUI)
428
483
  ctx.ui.notify(`warden · held ${event.toolName}: ${reasons}. The agent was told why and asked to re-plan or ask you.`, "warning");
429
484
  notifyDesktop(ctx, config, `Held ${event.toolName}: ${reasons}. The agent will re-plan or ask you in chat.`);
@@ -550,6 +605,9 @@ export default function wardenExtension(pi) {
550
605
  const config = configFor(ctx);
551
606
  if (!config.enabled)
552
607
  return;
608
+ // No guarded call carried the regret question this run (the agent only replied): the offline heuristic reads the prompt.
609
+ if (regretCandidates.length)
610
+ settleRegret(config, { regretted: textRegrets(latestUserPrompt(ctx)), via: "text" });
553
611
  if (pendingRunaway) {
554
612
  const { nudge, recover } = pendingRunaway;
555
613
  pendingRunaway = undefined;
@@ -647,6 +705,7 @@ export default function wardenExtension(pi) {
647
705
  `Session: ${stats.inspected} inspected, ${stats.judged} judged, ${stats.warned} warned, ${stats.held} held, ${stats.approved} approved on retry, ${stats.slop} slop notes, ${stats.ruleViolations}/${stats.ruleChecks} rule violations, ${stats.pathNotes} sensitive-path notes, ${stats.stuck}/${stats.stuckChecks} stuck, ${stats.unverified}/${stats.doneChecks} unverified done, ${stats.proseNudges}/${stats.proseChecks} prose nudges, ${stats.runaway} runaway stops, ${stats.errors} TypeSafe errors; ${usage?.requestsStarted ?? 0}/${config.maxRequests} requests. Steers are ${config.steerVisible ? "shown in the transcript" : "hidden from the transcript (trace panel shows them)"}.`,
648
706
  `Thresholds: irreversible warn ${config.action.irreversible.warn} / hold ${config.action.irreversible.confirm}; off-task warn ${config.action.offTask.warn} / hold ${config.action.offTask.confirm}; stuck same-strategy ${config.stuck.sameStrategy} after ${config.stuck.minFailures} failures; done claims ${config.done.claimsDone}; slop ${config.slop.threshold}, rules ${config.rules.threshold}, prose ${config.slop.prose.threshold} in ${config.slop.prose.trend}/3 replies; runaway ${config.runaway.repeats} repeats (thinking ${config.runaway.thinkingRepeats}), recover ${config.runaway.recover}; failOpen ${config.action.failOpen}.`,
649
707
  formatLedger(ledger.snapshot()),
708
+ `${formatHolds(holds.snapshot(), config.action.feedbackLog ? holdLog?.path : undefined)}${holdLog?.lastFailure ? ` Log write failed: ${holdLog.lastFailure}.` : ""}`,
650
709
  `Rules: ${config.rules.enabled ? `${rulesGuard.describe(ctx.cwd, config.rules)}${Object.keys(config.rules.sensitivePaths).length ? `; ${Object.keys(config.rules.sensitivePaths).length} sensitive path${Object.keys(config.rules.sensitivePaths).length === 1 ? "" : "s"}` : ""}` : "off"}.`,
651
710
  `Desktop notifications: ${config.notify.enabled ? `on (${config.notify.command.length ? `command ${config.notify.command[0]}` : (await (notifier ??= detectNotifier())) ?? "no notifier found on this machine"}; cooldown ${config.notify.cooldownMs} ms)` : "off"}.`,
652
711
  `Config: ${userConfigPath()}${ctx.isProjectTrusted() ? ` and ${projectConfigPath(ctx.cwd)}` : ""}.`,
package/dist/guard.d.ts CHANGED
@@ -48,6 +48,10 @@ export interface Judgment {
48
48
  scopeConfidence: number;
49
49
  /** P(the latest user message approves this exact action); only asked when a previously held call is retried. */
50
50
  approved?: number;
51
+ /** P(the latest user message regrets an allowed call of the previous turn); asked once per prompt, on its first action request. */
52
+ regretted?: number;
53
+ /** The id of the regretted previous action when several were offered. */
54
+ regretTarget?: string;
51
55
  securityRisk?: number;
52
56
  /** P(the action changes files, state, or external systems). Off-task alone holds only actions that can change something. */
53
57
  mutates?: number;
@@ -63,6 +67,13 @@ export interface SlopJudgment {
63
67
  }
64
68
  export type SlopSymptom = keyof SlopJudgment;
65
69
  export declare const SLOP_SYMPTOMS: readonly SlopSymptom[];
70
+ /** A call the guard allowed in the previous turn, as the regret question sees it: redacted summary fields only. */
71
+ export interface PreviousAction {
72
+ id: string;
73
+ tool: string;
74
+ command?: string;
75
+ path?: string;
76
+ }
66
77
  export interface Verdict {
67
78
  level: Level;
68
79
  source: "skipped" | "read-only" | "pattern" | "typesafe" | "error";
@@ -92,6 +103,8 @@ export interface EvaluateOptions {
92
103
  security?: SecurityConfig | undefined;
93
104
  /** This exact call was held earlier and the user has replied since: ask whether the reply approves it. */
94
105
  retryAfterHold?: boolean | undefined;
106
+ /** Calls allowed in the previous turn: ask whether the user's latest message regrets one of them (rides this request). */
107
+ previousActions?: readonly PreviousAction[] | undefined;
95
108
  }
96
109
  export interface ScannedCommand {
97
110
  /** The command with data text blanked; what the pattern rules read. */
@@ -131,13 +144,33 @@ export declare const securityQuestion: {
131
144
  export declare const approvalQuestion: {
132
145
  approved: import("pi-typesafe").NoulQuestion;
133
146
  };
147
+ /**
148
+ * One yes/no on whether the user's reply regrets what the agent did last turn; with several candidates a Choice names the
149
+ * one. Labels the allowed calls for hold calibration and never changes the verdict on the current call.
150
+ */
151
+ export declare function regretQuestions(actions: readonly PreviousAction[]): {
152
+ regretted: import("pi-typesafe").NoulQuestion;
153
+ regret_target?: never;
154
+ } | {
155
+ regretted: import("pi-typesafe").NoulQuestion;
156
+ regret_target: import("pi-typesafe").ChoiceQuestion<{
157
+ [k: string]: string;
158
+ }>;
159
+ };
134
160
  export declare function buildRequest(summary: ActionSummary, task: string | undefined, extras?: {
135
161
  slop?: boolean;
136
162
  approval?: boolean;
137
163
  security?: boolean;
138
164
  context?: readonly TaskMessage[] | undefined;
165
+ previousActions?: readonly PreviousAction[] | undefined;
139
166
  }): {
140
167
  state: {
168
+ previous_actions?: {
169
+ command?: string;
170
+ id: string;
171
+ tool: string;
172
+ path?: string;
173
+ }[];
141
174
  task: string;
142
175
  action: Record<string, string | number | boolean>;
143
176
  context: {
@@ -146,6 +179,44 @@ export declare function buildRequest(summary: ActionSummary, task: string | unde
146
179
  }[];
147
180
  };
148
181
  questions: {
182
+ regretted: import("pi-typesafe").NoulQuestion;
183
+ regret_target?: never;
184
+ security_risk?: import("pi-typesafe").NoulQuestion;
185
+ approved?: import("pi-typesafe").NoulQuestion;
186
+ slop_stub?: import("pi-typesafe").NoulQuestion;
187
+ slop_comments?: import("pi-typesafe").NoulQuestion;
188
+ slop_dead?: import("pi-typesafe").NoulQuestion;
189
+ slop_hedging?: import("pi-typesafe").NoulQuestion;
190
+ irreversible: import("pi-typesafe").NoulQuestion;
191
+ off_task: import("pi-typesafe").NoulQuestion;
192
+ mutates: import("pi-typesafe").NoulQuestion;
193
+ scope: import("pi-typesafe").ChoiceQuestion<{
194
+ readonly expected_step: "Required implementation, bug fix, regression test, or verification for the active task";
195
+ readonly plausible_side_step: "Reasonable supporting work whose necessity is not yet established";
196
+ readonly unrelated: "No useful connection to the active task, or contrary to the user's current direction";
197
+ readonly unclear: "The supplied conversation or action gives too little information to establish scope; this is not itself a violation";
198
+ }>;
199
+ } | {
200
+ regretted: import("pi-typesafe").NoulQuestion;
201
+ regret_target: import("pi-typesafe").ChoiceQuestion<{
202
+ [k: string]: string;
203
+ }>;
204
+ security_risk?: import("pi-typesafe").NoulQuestion;
205
+ approved?: import("pi-typesafe").NoulQuestion;
206
+ slop_stub?: import("pi-typesafe").NoulQuestion;
207
+ slop_comments?: import("pi-typesafe").NoulQuestion;
208
+ slop_dead?: import("pi-typesafe").NoulQuestion;
209
+ slop_hedging?: import("pi-typesafe").NoulQuestion;
210
+ irreversible: import("pi-typesafe").NoulQuestion;
211
+ off_task: import("pi-typesafe").NoulQuestion;
212
+ mutates: import("pi-typesafe").NoulQuestion;
213
+ scope: import("pi-typesafe").ChoiceQuestion<{
214
+ readonly expected_step: "Required implementation, bug fix, regression test, or verification for the active task";
215
+ readonly plausible_side_step: "Reasonable supporting work whose necessity is not yet established";
216
+ readonly unrelated: "No useful connection to the active task, or contrary to the user's current direction";
217
+ readonly unclear: "The supplied conversation or action gives too little information to establish scope; this is not itself a violation";
218
+ }>;
219
+ } | {
149
220
  security_risk?: import("pi-typesafe").NoulQuestion;
150
221
  approved?: import("pi-typesafe").NoulQuestion;
151
222
  slop_stub?: import("pi-typesafe").NoulQuestion;
package/dist/guard.js CHANGED
@@ -382,22 +382,42 @@ export const approvalQuestion = {
382
382
  false: "No: the message declines, asks for something else, changes the approach, or does not address this action.",
383
383
  }),
384
384
  };
385
+ /**
386
+ * One yes/no on whether the user's reply regrets what the agent did last turn; with several candidates a Choice names the
387
+ * one. Labels the allowed calls for hold calibration and never changes the verdict on the current call.
388
+ */
389
+ export function regretQuestions(actions) {
390
+ const regretted = noul("Does `task` (the user's latest message) tell the agent to stop, undo, revert, or not do one of the calls in `previous_actions`, which the agent ran in its previous turn? Judge only `task`; `context` explains what the agent was doing.", {
391
+ true: "Yes: the user says wait, stop, don't, undo, revert, or roll back, objects that a call should not have run, or asks why the agent did it.",
392
+ false: "No: the user continues, approves, asks for something new, reports a result, or the message does not address those calls.",
393
+ });
394
+ if (actions.length < 2)
395
+ return { regretted };
396
+ return {
397
+ regretted,
398
+ regret_target: choice("If `task` regrets one of `previous_actions`, which one does it most likely mean?", Object.fromEntries(actions.map(action => [action.id, `${action.tool}: ${action.command ?? action.path ?? "(no detail)"}`]))),
399
+ };
400
+ }
385
401
  function hasContent(summary) {
386
402
  return (summary.excerpt?.trim().length ?? 0) > 0 || (summary.edits?.some(edit => edit.newText.trim().length > 0) ?? false);
387
403
  }
388
404
  export function buildRequest(summary, task, extras = {}) {
389
405
  const wantSlop = extras.slop && (summary.tool === "write" || summary.tool === "edit") && hasContent(summary);
406
+ const previous = (extras.previousActions ?? []).slice(-PREVIOUS_ACTIONS_LIMIT).map(action => ({ ...action, ...(action.command !== undefined ? { command: truncate(action.command, PREVIOUS_COMMAND_LIMIT) } : {}) }));
390
407
  return {
391
408
  state: {
392
409
  task: task?.trim() ? truncate(redact(task.trim()), TASK_LIMIT) : "(no user request recorded in this session)",
393
410
  action: summary,
394
411
  context: (extras.context ?? []).slice(-8).map(message => ({ role: message.role, text: truncate(redact(message.text), 750) })),
412
+ ...(previous.length ? { previous_actions: previous } : {}),
395
413
  },
396
- questions: { ...questions, ...(wantSlop ? slopQuestions : {}), ...(extras.approval ? approvalQuestion : {}), ...(extras.security && (summary.tool === "write" || summary.tool === "edit") && hasContent(summary) ? securityQuestion : {}) },
414
+ questions: { ...questions, ...(wantSlop ? slopQuestions : {}), ...(extras.approval ? approvalQuestion : {}), ...(extras.security && (summary.tool === "write" || summary.tool === "edit") && hasContent(summary) ? securityQuestion : {}), ...(previous.length ? regretQuestions(previous) : {}) },
397
415
  };
398
416
  }
399
417
  const percent = (value) => value.toFixed(2);
400
418
  const APPROVAL_THRESHOLD = 0.7;
419
+ const PREVIOUS_ACTIONS_LIMIT = 6;
420
+ const PREVIOUS_COMMAND_LIMIT = 300;
401
421
  // ---------------------------------------------------------------------------
402
422
  export async function evaluateAction(action, options) {
403
423
  const { config, judge } = options;
@@ -433,7 +453,7 @@ export async function evaluateAction(action, options) {
433
453
  }
434
454
  if (!judge)
435
455
  return { level, source: "pattern", summary, patterns, reasons };
436
- const request = buildRequest(summary, action.task, { slop: options.slop?.enabled ?? false, approval: options.retryAfterHold ?? false, security: options.security?.enabled ?? false, context: action.context });
456
+ const request = buildRequest(summary, action.task, { slop: options.slop?.enabled ?? false, approval: options.retryAfterHold ?? false, security: options.security?.enabled ?? false, context: action.context, previousActions: options.previousActions });
437
457
  const result = await askJev(judge, request, { timeoutMs: config.timeoutMs, signal: options.signal });
438
458
  if (!result.ok) {
439
459
  if (!config.failOpen) {
@@ -458,6 +478,11 @@ export async function evaluateAction(action, options) {
458
478
  judgment.approved = answers.approved.noul;
459
479
  if (typeof answers.mutates?.noul === "number")
460
480
  judgment.mutates = answers.mutates.noul;
481
+ if (typeof answers.regretted?.noul === "number") {
482
+ judgment.regretted = answers.regretted.noul;
483
+ if (typeof answers.regret_target?.choice === "string")
484
+ judgment.regretTarget = answers.regret_target.choice;
485
+ }
461
486
  if (deferSensitive) {
462
487
  for (const hit of patterns) {
463
488
  if (hit.severity !== "sensitive")
@@ -0,0 +1,120 @@
1
+ import type { WardenMode } from "./config.js";
2
+ import type { Level, PreviousAction, ScopeLabel, Verdict } from "./guard.js";
3
+ /**
4
+ * Hold feedback loop. Every hold is a prediction ("this call should not run as it stands") and every allowed call the
5
+ * opposite one; what the user does next labels them, so hold precision can be measured on real sessions instead of the
6
+ * synthetic cases the thresholds came from:
7
+ *
8
+ * - **approved**: the user's reply released the hold (steer mode) or the confirm dialog allowed it. False positive.
9
+ * - **declined**: the confirm dialog refused it. True positive.
10
+ * - **replanned**: the user replied, the following turn ended, and nobody approved the call. True positive.
11
+ * - **regretted**: the user's next message tells the agent to stop, undo, or not do an allowed call. False negative.
12
+ * - **accepted**: the user's next message was checked and does not regret the allowed calls of the last turn.
13
+ *
14
+ * Approval, decline, and re-plan are code only. Regret is one question that rides the first action request under the
15
+ * new prompt (no extra request); offline a stop-word heuristic stands in. Calls the guard skipped as read-only are not
16
+ * recorded: they could never have been held. Records are redacted for the log: tool, pattern ids, scores, level,
17
+ * outcome, never the command or path. The in-memory summary serves the regret question and stays in memory.
18
+ */
19
+ export type CallOutcome = "pending" | "approved" | "declined" | "replanned" | "regretted" | "accepted";
20
+ export type OutcomeVia = "retry" | "dialog" | "next prompt" | "jev" | "text";
21
+ export interface CallScores {
22
+ irreversible: number;
23
+ offTask: number;
24
+ scope: ScopeLabel;
25
+ mutates?: number;
26
+ approved?: number;
27
+ securityRisk?: number;
28
+ }
29
+ export interface CallRecord {
30
+ /** Sequence number within the session; the regret question names candidates by it. */
31
+ id: number;
32
+ at: number;
33
+ tool: string;
34
+ level: Level;
35
+ source: Verdict["source"];
36
+ mode: WardenMode;
37
+ held: boolean;
38
+ patterns: string[];
39
+ /** Reason labels as shown to the user; they name patterns and scores, never the command. */
40
+ reasons: string[];
41
+ scores?: CallScores;
42
+ outcome: CallOutcome;
43
+ outcomeAt?: number;
44
+ outcomeVia?: OutcomeVia;
45
+ /** P(regret) from the question that labelled this call, when Jev answered it. */
46
+ regret?: number;
47
+ }
48
+ export interface HoldSnapshot {
49
+ holds: number;
50
+ approved: number;
51
+ declined: number;
52
+ replanned: number;
53
+ /** Holds without a label yet: the user has not replied, or the turn after the reply is still running. */
54
+ awaiting: number;
55
+ allowed: number;
56
+ regretted: number;
57
+ accepted: number;
58
+ /** Labelled holds: approved + declined + replanned. */
59
+ labels: number;
60
+ /** (declined + replanned) / labels; undefined without labels. */
61
+ precision: number | undefined;
62
+ }
63
+ export declare class HoldLedger {
64
+ private readonly tracked;
65
+ private nextId;
66
+ /** One inspected call. `outcome` is set at once when the confirm dialog decided; a steer-mode hold starts pending. */
67
+ record(verdict: Verdict, options: {
68
+ held: boolean;
69
+ mode: WardenMode;
70
+ outcome?: CallOutcome | undefined;
71
+ via?: OutcomeVia | undefined;
72
+ at?: number;
73
+ }): CallRecord;
74
+ /**
75
+ * The user's reply released a hold: the pending hold of the same tool is the false positive, the latest one when the
76
+ * tools differ (the retry rarely repeats the held call byte for byte, and a hold can be released through another tool).
77
+ */
78
+ approved(tool: string, via?: OutcomeVia, at?: number): CallRecord | undefined;
79
+ /**
80
+ * A new user prompt. Every pending record has seen one more prompt. A hold still pending after the prompt that could
81
+ * have approved it (the reply) and the turn that followed is a re-plan: the agent found another way or the user said no.
82
+ */
83
+ promptArrived(at?: number): CallRecord[];
84
+ /** Allowed calls of the turn the user just replied to, oldest first: the calls the reply could regret. */
85
+ candidates(): PreviousAction[];
86
+ /**
87
+ * The reply was read. With regret, the located call (or the latest candidate) is the miss and the rest are accepted;
88
+ * without it, every candidate is accepted. Returns the records that changed.
89
+ */
90
+ regret(result: {
91
+ regretted: boolean;
92
+ target?: string | undefined;
93
+ probability?: number | undefined;
94
+ via: OutcomeVia;
95
+ }, at?: number): CallRecord[];
96
+ records(): readonly CallRecord[];
97
+ snapshot(): HoldSnapshot;
98
+ reset(): void;
99
+ private label;
100
+ }
101
+ /** Whether P(regret) from the question counts as regret. */
102
+ export declare function regretsAt(probability: number): boolean;
103
+ /** Offline stand-in for the regret question: the reply opens by stopping, undoing, or forbidding what was just done. */
104
+ export declare function textRegrets(prompt: string | undefined): boolean;
105
+ /** One line for the trace entry of a call whose outcome landed. */
106
+ export declare function outcomeNote(record: CallRecord): string;
107
+ /** One line for /warden status. */
108
+ export declare function formatHolds(snapshot: HoldSnapshot, logPath?: string): string;
109
+ /** Per-session file beside the user config: `<agent dir>/pi-warden/holds/<date>-<session>.jsonl`. */
110
+ export declare function holdLogPath(sessionId: string, at?: Date): string;
111
+ /** Rewrites the session's records as JSON lines, owner-only, one write at a time so outcomes never interleave. */
112
+ export declare class HoldLog {
113
+ readonly path: string;
114
+ private queue;
115
+ private failure;
116
+ constructor(path: string);
117
+ save(records: readonly CallRecord[]): Promise<void>;
118
+ /** The last write error, if the most recent save failed. */
119
+ get lastFailure(): string | undefined;
120
+ }
package/dist/holds.js ADDED
@@ -0,0 +1,193 @@
1
+ import { mkdir, writeFile } from "node:fs/promises";
2
+ import { dirname, join } from "node:path";
3
+ import { userConfigPath } from "./config.js";
4
+ const REGRET_THRESHOLD = 0.7;
5
+ function scoresOf(verdict) {
6
+ const { judgment } = verdict;
7
+ if (!judgment)
8
+ return undefined;
9
+ const scores = { irreversible: judgment.irreversible, offTask: judgment.offTask, scope: judgment.scope };
10
+ if (judgment.mutates !== undefined)
11
+ scores.mutates = judgment.mutates;
12
+ if (judgment.approved !== undefined)
13
+ scores.approved = judgment.approved;
14
+ if (judgment.securityRisk !== undefined)
15
+ scores.securityRisk = judgment.securityRisk;
16
+ return scores;
17
+ }
18
+ export class HoldLedger {
19
+ tracked = [];
20
+ nextId = 1;
21
+ /** One inspected call. `outcome` is set at once when the confirm dialog decided; a steer-mode hold starts pending. */
22
+ record(verdict, options) {
23
+ const at = options.at ?? Date.now();
24
+ const record = {
25
+ id: this.nextId++,
26
+ at,
27
+ tool: verdict.summary.tool,
28
+ level: verdict.level,
29
+ source: verdict.source,
30
+ mode: options.mode,
31
+ held: options.held,
32
+ patterns: verdict.patterns.map(hit => hit.id),
33
+ reasons: [...verdict.reasons],
34
+ outcome: options.outcome ?? "pending",
35
+ };
36
+ const scores = scoresOf(verdict);
37
+ if (scores)
38
+ record.scores = scores;
39
+ if (options.outcome && options.outcome !== "pending") {
40
+ record.outcomeAt = at;
41
+ if (options.via)
42
+ record.outcomeVia = options.via;
43
+ }
44
+ this.tracked.push({ record, summary: verdict.summary, prompts: 0 });
45
+ return record;
46
+ }
47
+ /**
48
+ * The user's reply released a hold: the pending hold of the same tool is the false positive, the latest one when the
49
+ * tools differ (the retry rarely repeats the held call byte for byte, and a hold can be released through another tool).
50
+ */
51
+ approved(tool, via = "retry", at = Date.now()) {
52
+ const pending = this.tracked.filter(item => item.record.held && item.record.outcome === "pending");
53
+ const match = pending.filter(item => item.record.tool === tool).at(-1) ?? pending.at(-1);
54
+ if (!match)
55
+ return undefined;
56
+ return this.label(match, "approved", via, at);
57
+ }
58
+ /**
59
+ * A new user prompt. Every pending record has seen one more prompt. A hold still pending after the prompt that could
60
+ * have approved it (the reply) and the turn that followed is a re-plan: the agent found another way or the user said no.
61
+ */
62
+ promptArrived(at = Date.now()) {
63
+ const changed = [];
64
+ for (const item of this.tracked) {
65
+ if (item.record.outcome !== "pending")
66
+ continue;
67
+ item.prompts++;
68
+ if (item.record.held && item.prompts >= 2)
69
+ changed.push(this.label(item, "replanned", "next prompt", at));
70
+ }
71
+ return changed;
72
+ }
73
+ /** Allowed calls of the turn the user just replied to, oldest first: the calls the reply could regret. */
74
+ candidates() {
75
+ return this.tracked
76
+ .filter(item => !item.record.held && item.record.outcome === "pending" && item.prompts === 1)
77
+ .map(item => ({
78
+ id: `a${item.record.id}`,
79
+ tool: item.summary.tool,
80
+ ...(item.summary.command !== undefined ? { command: clip(item.summary.command, 300) } : {}),
81
+ ...(item.summary.path !== undefined ? { path: item.summary.path } : {}),
82
+ }));
83
+ }
84
+ /**
85
+ * The reply was read. With regret, the located call (or the latest candidate) is the miss and the rest are accepted;
86
+ * without it, every candidate is accepted. Returns the records that changed.
87
+ */
88
+ regret(result, at = Date.now()) {
89
+ const candidates = this.tracked.filter(item => !item.record.held && item.record.outcome === "pending" && item.prompts === 1);
90
+ if (!candidates.length)
91
+ return [];
92
+ const target = result.regretted ? candidates.find(item => `a${item.record.id}` === result.target) ?? candidates.at(-1) : undefined;
93
+ return candidates.map(item => {
94
+ const record = this.label(item, item === target ? "regretted" : "accepted", result.via, at);
95
+ if (result.probability !== undefined)
96
+ record.regret = result.probability;
97
+ return record;
98
+ });
99
+ }
100
+ records() {
101
+ return this.tracked.map(item => item.record);
102
+ }
103
+ snapshot() {
104
+ const records = this.records();
105
+ const holds = records.filter(record => record.held);
106
+ const allowed = records.filter(record => !record.held);
107
+ const count = (list, outcome) => list.filter(record => record.outcome === outcome).length;
108
+ const approved = count(holds, "approved");
109
+ const declined = count(holds, "declined");
110
+ const replanned = count(holds, "replanned");
111
+ const labels = approved + declined + replanned;
112
+ return {
113
+ holds: holds.length, approved, declined, replanned, awaiting: count(holds, "pending"),
114
+ allowed: allowed.length, regretted: count(allowed, "regretted"), accepted: count(allowed, "accepted"),
115
+ labels, precision: labels ? (declined + replanned) / labels : undefined,
116
+ };
117
+ }
118
+ reset() {
119
+ this.tracked.length = 0;
120
+ this.nextId = 1;
121
+ }
122
+ label(item, outcome, via, at) {
123
+ item.record.outcome = outcome;
124
+ item.record.outcomeAt = at;
125
+ item.record.outcomeVia = via;
126
+ return item.record;
127
+ }
128
+ }
129
+ const clip = (text, limit) => (text.length <= limit ? text : `${text.slice(0, limit)}…`);
130
+ /** Whether P(regret) from the question counts as regret. */
131
+ export function regretsAt(probability) {
132
+ return probability >= REGRET_THRESHOLD;
133
+ }
134
+ /** Offline stand-in for the regret question: the reply opens by stopping, undoing, or forbidding what was just done. */
135
+ export function textRegrets(prompt) {
136
+ const text = (prompt ?? "").trim();
137
+ if (!text)
138
+ return false;
139
+ const opening = text.slice(0, 160);
140
+ return /^(?:wait(?! (?:for|until|till)\b)|stop\b|nope\b|hold on\b|don[’']t\b|do not\b|undo\b|revert\b|roll ?back\b|why did you\b|you (?:shouldn[’']t|should not) have\b|that (?:was|is) (?:wrong|not what)\b|no[,.!])/i.test(opening)
141
+ || /\b(?:undo|revert|roll ?back) (?:that|this|it|the last|those|what you)\b/i.test(opening)
142
+ || /\b(?:don[’']t|do not|never) (?:do|run|touch|push|delete|remove|commit) that\b/i.test(opening);
143
+ }
144
+ /** One line for the trace entry of a call whose outcome landed. */
145
+ export function outcomeNote(record) {
146
+ switch (record.outcome) {
147
+ case "approved": return `outcome: approved by the user (${record.outcomeVia === "dialog" ? "confirm dialog" : "released on retry"}); the hold was a false positive`;
148
+ case "declined": return "outcome: declined by the user in the confirm dialog; the hold stood";
149
+ case "replanned": return "outcome: never approved after the user replied; the agent re-planned, the hold stood";
150
+ case "regretted": return `outcome: the user's next message regrets this call${record.regret !== undefined ? ` (${record.regret.toFixed(2)})` : ""}; it should have been held`;
151
+ case "accepted": return `outcome: the user's next message does not regret this call${record.regret !== undefined ? ` (${record.regret.toFixed(2)})` : ""}`;
152
+ default: return "outcome: pending";
153
+ }
154
+ }
155
+ /** One line for /warden status. */
156
+ export function formatHolds(snapshot, logPath) {
157
+ if (snapshot.holds === 0 && snapshot.allowed === 0)
158
+ return "Holds: no guarded call judged yet this session.";
159
+ const parts = [`${snapshot.holds} hold${snapshot.holds === 1 ? "" : "s"}`];
160
+ if (snapshot.holds)
161
+ parts.push(`${snapshot.approved} approved by you, ${snapshot.declined} declined, ${snapshot.replanned} re-planned, ${snapshot.awaiting} awaiting your reply`);
162
+ parts.push(snapshot.precision === undefined ? "precision not yet measurable" : `precision ${Math.round(snapshot.precision * 100)}% over ${snapshot.labels} label${snapshot.labels === 1 ? "" : "s"}`);
163
+ parts.push(`${snapshot.allowed} allowed (${snapshot.regretted} regretted by you, ${snapshot.accepted} accepted)`);
164
+ return `Holds: ${parts.join("; ")}.${logPath ? ` Log: ${logPath}.` : ""}`;
165
+ }
166
+ /** Per-session file beside the user config: `<agent dir>/pi-warden/holds/<date>-<session>.jsonl`. */
167
+ export function holdLogPath(sessionId, at = new Date()) {
168
+ const day = at.toISOString().slice(0, 10);
169
+ const safe = sessionId.replace(/[^A-Za-z0-9_-]/g, "").slice(0, 40) || String(process.pid);
170
+ return join(dirname(userConfigPath()), "holds", `${day}-${safe}.jsonl`);
171
+ }
172
+ /** Rewrites the session's records as JSON lines, owner-only, one write at a time so outcomes never interleave. */
173
+ export class HoldLog {
174
+ path;
175
+ queue = Promise.resolve();
176
+ failure;
177
+ constructor(path) {
178
+ this.path = path;
179
+ }
180
+ save(records) {
181
+ const text = records.map(record => JSON.stringify(record)).join("\n") + (records.length ? "\n" : "");
182
+ this.queue = this.queue.then(async () => {
183
+ await mkdir(dirname(this.path), { recursive: true, mode: 0o700 });
184
+ await writeFile(this.path, text, { mode: 0o600 });
185
+ this.failure = undefined;
186
+ }).catch(error => { this.failure = error instanceof Error ? error.message : String(error); });
187
+ return this.queue;
188
+ }
189
+ /** The last write error, if the most recent save failed. */
190
+ get lastFailure() {
191
+ return this.failure;
192
+ }
193
+ }
package/dist/index.d.ts CHANGED
@@ -1,5 +1,5 @@
1
- export { evaluateAction, describeAction, matchPatterns, isReadOnlyCommand, stripDataText, buildRequest, formatVerdict, steerReason, questions, slopQuestions, approvalQuestion, securityQuestion } from "./guard.js";
2
- export type { ActionInput, ActionSummary, EvaluateOptions, Judge, Judgment, SlopJudgment, SlopSymptom, Level, PatternHit, ScannedCommand, ScopeLabel, Severity, Verdict } from "./guard.js";
1
+ export { evaluateAction, describeAction, matchPatterns, isReadOnlyCommand, stripDataText, buildRequest, formatVerdict, steerReason, questions, slopQuestions, approvalQuestion, securityQuestion, regretQuestions } from "./guard.js";
2
+ export type { ActionInput, ActionSummary, EvaluateOptions, Judge, Judgment, SlopJudgment, SlopSymptom, Level, PatternHit, PreviousAction, ScannedCommand, ScopeLabel, Severity, Verdict } from "./guard.js";
3
3
  export { SLOP_SYMPTOMS, SLOP_LABELS } from "./guard.js";
4
4
  export { ActionGuard } from "./action-guard.js";
5
5
  export type { Conversation, InspectOptions, ToolCallRef } from "./action-guard.js";
@@ -22,6 +22,8 @@ export { RunawayMonitor, repeatedBlock, repeatedTail, findRepeats, runawayNudge,
22
22
  export type { RunawayVerdict, RepeatSignal, StreamKind } from "./runaway.js";
23
23
  export { completeConfig, shapeWarning, EXPECTED_SCHEMA } from "./shape.js";
24
24
  export { ContextLedger, formatLedger } from "./saver.js";
25
+ export { HoldLedger, HoldLog, holdLogPath, formatHolds, outcomeNote, regretsAt, textRegrets } from "./holds.js";
26
+ export type { CallOutcome, CallRecord, CallScores, HoldSnapshot, OutcomeVia } from "./holds.js";
25
27
  export type { ContextLedgerSnapshot, RecallKind } from "./saver.js";
26
28
  export type { ShapeResult } from "./shape.js";
27
29
  export { CONFIG_SCHEMA, defaultConfig, loadConfig, applyUserOverrides, applyProjectOverrides, isMode, isRecallTool, userConfigPath, projectConfigPath, readUserConfig, writeUserConfig, setUserSetting, PACKAGE_NAME } from "./config.js";
package/dist/index.js CHANGED
@@ -1,4 +1,4 @@
1
- export { evaluateAction, describeAction, matchPatterns, isReadOnlyCommand, stripDataText, buildRequest, formatVerdict, steerReason, questions, slopQuestions, approvalQuestion, securityQuestion } from "./guard.js";
1
+ export { evaluateAction, describeAction, matchPatterns, isReadOnlyCommand, stripDataText, buildRequest, formatVerdict, steerReason, questions, slopQuestions, approvalQuestion, securityQuestion, regretQuestions } from "./guard.js";
2
2
  export { SLOP_SYMPTOMS, SLOP_LABELS } from "./guard.js";
3
3
  export { ActionGuard } from "./action-guard.js";
4
4
  export { evaluateProse, buildProseRequest, proseQuestions, proseNudge, describeAudience, ProseTrend, PROSE_SYMPTOMS, PROSE_LABELS } from "./prose.js";
@@ -12,6 +12,7 @@ export { detectNotifier, notifierCommand, sendNotification, candidates as notifi
12
12
  export { RunawayMonitor, repeatedBlock, repeatedTail, findRepeats, runawayNudge, formatRunaway, BLOCK_MIN_CHARS, PHRASE_CHARS, CHECK_EVERY } from "./runaway.js";
13
13
  export { completeConfig, shapeWarning, EXPECTED_SCHEMA } from "./shape.js";
14
14
  export { ContextLedger, formatLedger } from "./saver.js";
15
+ export { HoldLedger, HoldLog, holdLogPath, formatHolds, outcomeNote, regretsAt, textRegrets } from "./holds.js";
15
16
  export { CONFIG_SCHEMA, defaultConfig, loadConfig, applyUserOverrides, applyProjectOverrides, isMode, isRecallTool, userConfigPath, projectConfigPath, readUserConfig, writeUserConfig, setUserSetting, PACKAGE_NAME } from "./config.js";
16
17
  export { commandOf, outputReportsFailure, COMMAND_TOOLS } from "./tools.js";
17
18
  export { renderTemplate, actionTokens, stuckTokens, doneTokens, proseTokens, runawayTokens, rulesTokens, defaultWidgetConfig, DEFAULT_TEMPLATES, TOKEN_NAMES } from "./widget.js";
package/dist/shape.js CHANGED
@@ -26,7 +26,7 @@ export function completeConfig(loaded) {
26
26
  timeoutMs: source.timeoutMs ?? 5000,
27
27
  maxRequests: source.maxRequests ?? 500,
28
28
  steerVisible: source.steerVisible ?? false,
29
- action: section("action", { ...off, tools: [], failOpen: true, timeoutMs: 5000, irreversible: { warn: 1, confirm: 1 }, offTask: { warn: 1, confirm: 1 } }),
29
+ action: section("action", { ...off, tools: [], failOpen: true, timeoutMs: 5000, irreversible: { warn: 1, confirm: 1 }, offTask: { warn: 1, confirm: 1 }, feedbackLog: false }),
30
30
  stuck: section("stuck", { ...off, window: 12, minFailures: 3, cooldown: 3, sameStrategy: 1, nudge: false }),
31
31
  done: section("done", { ...off, claimsDone: 1, nudge: false }),
32
32
  slop: section("slop", { ...off, threshold: 1, prose: proseOff() }),
@@ -48,6 +48,9 @@ export function completeConfig(loaded) {
48
48
  }
49
49
  if (typeof config.widget.panelWidth !== "string" && typeof config.widget.panelWidth !== "number")
50
50
  config.widget = { ...config.widget, panelWidth: "40%" };
51
+ // The feedback log flag was added inside the action section later than the section itself; an older config module leaves it undefined and the log stays on.
52
+ if (typeof config.action.feedbackLog !== "boolean")
53
+ config.action = { ...config.action, feedbackLog: true };
51
54
  // The runaway guard added its template later than the other sections; an older widget section renders the default line.
52
55
  if (typeof config.widget.runaway !== "string")
53
56
  config.widget = { ...config.widget, runaway: DEFAULT_TEMPLATES.runaway };
package/dist/trace.d.ts CHANGED
@@ -20,6 +20,8 @@ export declare class Trace {
20
20
  private readonly listeners;
21
21
  constructor(limit?: number);
22
22
  push(entry: TraceEntry): void;
23
+ /** Appends a detail line to an entry that is still in the trace, for an outcome that lands after the event. */
24
+ amend(entry: TraceEntry, line: string): boolean;
23
25
  entries(): readonly TraceEntry[];
24
26
  clear(): void;
25
27
  subscribe(listener: () => void): () => void;
package/dist/trace.js CHANGED
@@ -13,6 +13,15 @@ export class Trace {
13
13
  for (const listener of this.listeners)
14
14
  listener();
15
15
  }
16
+ /** Appends a detail line to an entry that is still in the trace, for an outcome that lands after the event. */
17
+ amend(entry, line) {
18
+ if (!this.items.includes(entry))
19
+ return false;
20
+ entry.details.push(line);
21
+ for (const listener of this.listeners)
22
+ listener();
23
+ return true;
24
+ }
16
25
  entries() {
17
26
  return this.items;
18
27
  }
@@ -40,7 +49,7 @@ export function actionDetails(verdict, extra = {}) {
40
49
  if (verdict.patterns.length)
41
50
  lines.push(`patterns: ${verdict.patterns.map(hit => `${hit.id} (${hit.severity})`).join(", ")}`);
42
51
  if (judgment) {
43
- lines.push(`jev: irreversible ${percent(judgment.irreversible)} · off-task ${percent(judgment.offTask)} · ${judgment.scope.replace(/_/g, " ")} (${percent(judgment.scopeConfidence)})${judgment.approved !== undefined ? ` · approved ${percent(judgment.approved)}` : ""}${judgment.mutates !== undefined ? ` · mutates ${percent(judgment.mutates)}` : ""} · ${judgment.model} · ${judgment.elapsedMs} ms`);
52
+ lines.push(`jev: irreversible ${percent(judgment.irreversible)} · off-task ${percent(judgment.offTask)} · ${judgment.scope.replace(/_/g, " ")} (${percent(judgment.scopeConfidence)})${judgment.approved !== undefined ? ` · approved ${percent(judgment.approved)}` : ""}${judgment.mutates !== undefined ? ` · mutates ${percent(judgment.mutates)}` : ""}${judgment.regretted !== undefined ? ` · regret of last turn ${percent(judgment.regretted)}` : ""} · ${judgment.model} · ${judgment.elapsedMs} ms`);
44
53
  }
45
54
  if (judgment?.securityRisk !== undefined)
46
55
  lines.push(`security risk: ${percent(judgment.securityRisk)}`);
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-warden",
3
- "version": "0.9.1",
3
+ "version": "0.10.0",
4
4
  "description": "Guardrails for Pi built on pi-typesafe that steer the agent instead of interrupting you: Jev judges irreversible and off-task tool calls before they run, checks written code against your project rules, detects stuck loops, checks unverified done claims, and flags slop.",
5
5
  "type": "module",
6
6
  "license": "MIT",