@wyattjoh/demur 0.4.2 → 0.6.0

package/README.md

@@ -48,6 +48,12 @@ or sensitive names. Review TypeSafe's service terms and data-handling policy
 before enabling demur in a sensitive repository. Do not run secrets directly in
 shell arguments when the guard is active.
 
+Pi training capture stores the complete command, working directory, operating
+mode, full verdict and judgments, and resulting host action locally. These
+records can therefore contain secrets or sensitive names from shell arguments
+and paths. Training capture is off by default and is unavailable while the Pi
+integration is disabled.
+
 ## Requirements
 
 - [Bun](https://bun.sh/) 1.4 or newer
@@ -92,7 +98,7 @@ Pin a specific release when reproducibility matters:
 
 <!-- x-release-please-start-version -->
 ```sh
-pi install npm:@wyattjoh/demur@0.4.2
+pi install npm:@wyattjoh/demur@0.6.0
 ```
 <!-- x-release-please-end -->
 
@@ -109,23 +115,34 @@ command request and resulting verdict cross its local stdio pipes. `ask` opens
 an interactive confirmation dialog; without an interactive UI, demur blocks the
 command.
 
-Use `/demur` to open the extension menu. It can enable or disable demur and
-change what Pi does when demur cannot obtain a trustworthy judgment because of
-a missing credential, timeout, API error, malformed worker response, or
-unexpected guard failure:
+Use `/demur` to open the extension menu. Its global operating mode is:
+
+- `enforce` (default) applies `allow`, `ask`, and `deny` decisions normally.
+- `passive` still judges every Bash call and prints the diagnostic, but never
+  prompts or blocks because of the verdict.
+- `disabled` bypasses the worker and allows Bash calls without judgment.
+
+Training capture can be enabled independently in `enforce` or `passive` mode.
+It is automatically turned off when the integration is disabled. Pi's bottom
+status bar always shows the current mode and whether training is active so every
+bypass or local recording state remains visible.
+
+The menu also controls what enforce mode does when demur cannot obtain a
+trustworthy judgment because of a missing credential, timeout, API error,
+malformed worker response, or unexpected guard failure:
 
 - `block` (default) fails closed.
 - `ask` requests interactive confirmation and blocks when no UI is available.
 - `allow` fails open without confirmation.
 
-Disabling demur bypasses the worker and allows Bash calls without judgment. Pi's
-bottom status bar always shows `demur: enabled` or `demur: disabled` so this
-bypass remains visible.
-
-Both settings are stored globally at `$XDG_CONFIG_HOME/demur/config.json`, or
+Settings are stored globally at `$XDG_CONFIG_HOME/demur/config.json`, or
 `~/.config/demur/config.json` when `XDG_CONFIG_HOME` is unset, and apply to
-future Pi sessions. While demur is enabled, the failure policy never changes a
-completed `deny` policy judgment; those commands remain blocked.
+future Pi sessions. Set `DEMUR_CONFIG_HOME` to use an isolated demur directory;
+`config.json` is read and written directly beneath it. This demur-specific
+override takes precedence over the XDG and home-directory locations. While
+demur is enforcing, the failure policy never changes a completed `deny` policy
+judgment; those commands remain blocked. Passive mode reports failures but does
+not apply the failure policy because it never blocks.
 
 After each run, Pi's interactive UI prints the decision, submitted input-token
 count, the run's estimated input cost, the accumulated global estimate, and the
@@ -142,6 +159,16 @@ file before an atomic rename so the total cannot be partially written or lose a
 concurrent increment. Cost-accounting failures do not change demur's guard
 decision; the status reports `accumulated unavailable` instead.
 
+Training evaluations are appended as private, versioned JSONL records at
+`$XDG_STATE_HOME/demur/training.jsonl`, with the same home-directory fallback.
+Set `DEMUR_STATE_HOME` to place `usage.json`, `training.jsonl`, and
+`training-reviews.jsonl` directly beneath an isolated directory instead. This
+demur-specific override takes precedence over the XDG and home-directory
+locations. Training records are retained until the user removes them. Human
+reviews are appended separately; accepted and corrected records are linked by a
+stable record ID, leaving the original evidence unchanged. A training-write
+failure is reported but never changes whether the command runs.
+
 Pi packages execute with the user's full system permissions. Review this
 repository before installing it.
 
@@ -182,16 +209,63 @@ Claude Code's `hookSpecificOutput.permissionDecision` response.
 
 ### CLI
 
-Manage the stored credential or judge a single command without a host
-integration:
+Launch the central interface, manage the stored credential, or judge a single
+command without a host integration:
 
 ```sh
+demur
 demur auth login
 demur auth status
 demur auth logout
+demur training review
+demur training list --status=unreviewed --cwd=/workspace
+demur training list --status=deny --json
+demur training review <record-id> --decision=deny --note="would destroy work" --json
 demur judge "git reset --hard HEAD~3"
 ```
 
+Bare `demur` opens the central OpenTUI interface when stdin and stdout are
+interactive. `demur training review` remains an explicit alias for the same
+interface. `[` opens Reviews and `]` opens Settings. You can also navigate Up
+to the top-level section strip, use Left and Right to switch sections, and press
+Down or Enter to open one. Settings exposes every option from Pi's `/demur`
+menu—operating mode, training capture,
+and failure policy—and atomically saves each change to the same global
+configuration file. Use Up and Down to select a setting, Left and Right to
+change it in either direction, or Enter/Space to choose its next value.
+Disabling demur also turns training capture off, and training remains unavailable
+until an active mode is selected.
+
+The Reviews header shows the persisted global estimated cost, and each queue row
+shows the model decision in a muted semantic color. It starts in an `all` view;
+Tab and Shift-Tab rotate between `all`, `not reviewed`, `approved` (`allow`), `ask`,
+and `deny` views. The queue is focused initially: arrow keys navigate it, Up
+from its first result focuses a fuzzy working-directory filter, and another Up
+focuses the tab strip. Left and Right select adjacent focused tabs, while Down
+returns through the filter to the queue. Right from the queue focuses the
+scrollable detail pane.
+The detail pane supports arrows or `j`/`k`; Left returns to the queue. Page Up
+and Page Down page within the focused pane, and queue navigation stops at its
+first and last entries. Mouse clicks select tabs, records, the filter, or either
+pane; the wheel scrolls the queue and detail pane. Enter selects the original
+decision, `1`/`2`/`3` choose `allow`/`ask`/`deny`, `s`
+leaves a record for a later pass, and `q` or Escape stops. Previously reviewed
+records remain available, and changing an answer appends a review revision while
+preserving its visible history. The TUI remains open when a view is empty and
+polls training state for newly captured or externally reviewed evaluations.
+
+Flag-based training commands provide the same review operations without the
+TUI. `demur training list` accepts `--status` and fuzzy `--cwd` filters. Status
+is derived from the latest human review, so `allow`, `ask`, and `deny` select
+reviewed records while `unreviewed` selects records without a review. Record a
+new append-only review revision by passing a record ID, a required
+`--decision=<allow|ask|deny>`, and an optional `--note`. Add `--json` to either
+operation to emit one versioned JSON document; JSON failures are written to
+stdout with a nonzero exit code. Interactive `demur training review` still opens
+the TUI, but without a terminal it requires an explicit list or record-ID review
+operation. Reviews remain separate from the original evidence so they can later
+be curated into independently licensed eval fixtures.
+
 From a development checkout, `bun run judge "<command>"` remains available.
 
 ## Configuration
@@ -200,6 +274,8 @@ From a development checkout, `bun run judge "<command>"` remains available.
 | --- | --- | --- |
 | `TYPESAFE_API_KEY` | stored credential | Optional TypeSafe API credential override. Missing keys fail closed. |
 | `DEMUR_TIMEOUT_MS` | `4000` | Per-attempt model timeout in milliseconds. |
+| `DEMUR_CONFIG_HOME` | XDG/home config | Demur-specific directory containing `config.json`. |
+| `DEMUR_STATE_HOME` | XDG/home state | Demur-specific directory containing usage and training state. |
 | `DEMUR_DISABLE` | unset | Emergency bypass. `1` or `true` allows every command. |
 
 ## Failure posture
@@ -209,12 +285,12 @@ timeout, API failure, malformed response, or unexpected guard error returns
 `deny` with a reason that identifies the guard failure rather than presenting it
 as a policy judgment. The Claude Code adapter and CLI preserve that verdict.
 
-The Pi extension defaults to enabled with the same fail-closed behavior, but
-its explicit `/demur` menu can globally change how Pi handles guard failures or
-disable the extension entirely. The failure-policy override applies only when
-no trustworthy judgment was produced; it cannot loosen a completed policy
-denial while demur is enabled. The bottom status bar makes the enabled state
-visible.
+The Pi extension defaults to enforce mode with the same fail-closed behavior,
+but its explicit `/demur` menu can globally select enforce, passive, or disabled
+mode. The failure-policy override applies only in enforce mode when no
+trustworthy judgment was produced; it cannot loosen a completed policy denial.
+Passive mode always continues after reporting the underlying verdict, while the
+bottom status bar keeps the active mode and training state visible.
 
 `DEMUR_DISABLE=1` remains the cross-host emergency bypass. It disables judgment
 and protection entirely and should remain unset during normal use.
@@ -248,7 +324,9 @@ and deterministic policy controls alongside demur.
 - `src/key.ts` — environment precedence and operating-system credential storage
 - `src/guard.internal.ts` — Effect-native orchestration and fail-closed recovery
 - `src/guard.ts` — managed runtime and Promise boundary
-- `extensions/demur/` — Pi `tool_call` integration
+- `src/training-review-model.ts` — historical review status and cwd filtering
+- `src/training-review-tui.tsx` — interactive OpenTUI training-review queue
+- `extensions/demur/` — Pi `tool_call` integration and training-state storage
 - `src/adapters/claude-code.ts` — Claude Code `PreToolUse` integration
 - `eval/` — safe synthetic contrast cases and the live evaluation runner
 

package/extensions/demur/cost-tracker.ts

@@ -9,6 +9,7 @@ import {
 } from "node:fs/promises";
 import { homedir } from "node:os";
 import { dirname, join } from "node:path";
+import { getDemurStateDirectory } from "./paths.ts";
 
 const LOCK_RETRY_MS = 10;
 const LOCK_TIMEOUT_MS = 5_000;
@@ -31,9 +32,9 @@ export type CostTotals = {
 };
 
 /**
- * Resolve the global demur usage file according to the XDG state convention.
+ * Resolve the global demur usage file using demur-specific and XDG conventions.
  *
- * @param environment - Process environment used to resolve `XDG_STATE_HOME`
+ * @param environment - Process environment used to resolve demur and XDG overrides
  * @param homeDirectory - Home directory used when the XDG override is absent
  * @returns Absolute path to demur's usage state file
  */
@@ -41,8 +42,10 @@ export function getCostStatePath(
   environment: NodeJS.ProcessEnv = process.env,
   homeDirectory: string = homedir(),
 ): string {
-  const stateDirectory = environment.XDG_STATE_HOME || join(homeDirectory, ".local", "state");
-  return join(stateDirectory, "demur", "usage.json");
+  return join(
+    getDemurStateDirectory(environment, homeDirectory),
+    "usage.json",
+  );
 }
 
 /**
@@ -57,6 +60,17 @@ export function estimateInputCostUsd(inputTokens: number): number {
   return (inputTokens * JEV_INPUT_COST_USD_PER_MILLION) / 1_000_000;
 }
 
+/**
+ * Format an estimated US-dollar cost without hiding sub-cent evaluations.
+ *
+ * @param value - Estimated cost in US dollars
+ * @returns Dollar-prefixed cost with up to nine fractional digits
+ */
+export function formatUsd(value: number): string {
+  const decimal = value.toFixed(9).replace(/0+$/, "").replace(/\.$/, "");
+  return `$${decimal}`;
+}
+
 /**
  * Atomically add one judgment's usage to the global accumulated estimate.
  *
@@ -80,7 +94,7 @@ export async function recordInputCost(
   const release = await acquireLock(`${statePath}.lock`);
 
   try {
-    const current = await readTotals(statePath);
+    const current = await loadCostTotals(statePath);
     const next: CostTotals = {
       version: 1,
       totalInputTokens: current.totalInputTokens + inputTokens,
@@ -114,7 +128,18 @@ async function acquireLock(lockPath: string): Promise<() => Promise<void>> {
   }
 }
 
-async function readTotals(statePath: string): Promise<CostTotals> {
+/**
+ * Load the persisted global usage and estimated-cost totals.
+ *
+ * A missing state file represents zero recorded usage. Invalid state fails
+ * rather than returning a misleading total.
+ *
+ * @param statePath - Usage file to read
+ * @returns Persisted totals, or zero totals when the file does not exist
+ */
+export async function loadCostTotals(
+  statePath: string = getCostStatePath(),
+): Promise<CostTotals> {
   let content: string;
   try {
     content = await readFile(statePath, "utf8");

package/extensions/demur/index.ts

@@ -10,17 +10,21 @@ import {
 import type { Verdict } from "../../src/types.ts";
 import {
   estimateInputCostUsd,
+  formatUsd,
   recordInputCost,
 } from "./cost-tracker.ts";
 import {
   DEFAULT_DEMUR_SETTINGS,
+  DEMUR_MODES,
   FAILURE_POLICIES,
   loadDemurSettings,
+  parseDemurMode,
   parseFailurePolicy,
   saveDemurSettings,
   type DemurSettings,
   type FailurePolicy,
 } from "./settings.ts";
+import { recordTrainingEvaluation } from "./training-store.ts";
 
 const WORKER_PATH = fileURLToPath(
   new URL("../../src/adapters/pi-worker.ts", import.meta.url),
@@ -44,7 +48,7 @@ export async function handleToolCall(
   settings: DemurSettings = DEFAULT_DEMUR_SETTINGS,
 ): Promise<ToolCallEventResult | undefined> {
   if (!isToolCallEventType("bash", event)) return undefined;
-  if (!settings.enabled) return undefined;
+  if (settings.mode === "disabled") return undefined;
 
   const command = event.input.command ?? "";
   if (command.trim() === "") return undefined;
@@ -70,28 +74,45 @@ export async function handleToolCall(
       };
     }
 
-    return handleGuardFailure(
-      `demur: guard worker crashed — ${errorDetail(error)}`,
+    verdict = {
+      decision: "deny",
+      reason: `demur: guard worker crashed — ${errorDetail(error)}`,
+      judgments: undefined,
+      failure: "unexpected",
+      latencyMs: evaluationMs,
+      usage: undefined,
+    };
+  }
+
+  const evaluationMs = performance.now() - evaluationStartedAt;
+  const inputTokens = verdict.usage?.inputTokens;
+  const accumulatedCostUsd = await recordAccumulatedCost(inputTokens);
+  const result = settings.mode === "passive"
+    ? resolvePassiveVerdict(
+      verdict,
+      ctx,
+      accumulatedCostUsd,
+      evaluationMs,
+    )
+    : await resolveVerdict(
+      verdict,
       command,
       ctx,
       settings.failurePolicy,
-      undefined,
-      undefined,
+      accumulatedCostUsd,
       evaluationMs,
     );
-  }
 
-  const evaluationMs = performance.now() - evaluationStartedAt;
-  const inputTokens = verdict.usage?.inputTokens;
-  const accumulatedCostUsd = await recordAccumulatedCost(inputTokens);
-  return resolveVerdict(
-    verdict,
-    command,
-    ctx,
-    settings.failurePolicy,
-    accumulatedCostUsd,
-    evaluationMs,
-  );
+  if (settings.training) {
+    await recordTrainingResult(
+      command,
+      ctx,
+      settings.mode,
+      verdict,
+      result,
+    );
+  }
+  return result;
 }
 
 /**
@@ -185,6 +206,40 @@ export async function resolveVerdict(
   return { block: true, reason: `${verdict.reason} Declined by the user.` };
 }
 
+/**
+ * Report a completed verdict without allowing it to affect execution.
+ *
+ * Passive mode never prompts and never returns a block result. Failures remain
+ * visible as failures rather than being mapped through the enforcement-only
+ * failure policy.
+ *
+ * @param verdict - Completed demur guard result
+ * @param ctx - Pi extension context used for notifications
+ * @param accumulatedCostUsd - Persisted global estimate after this run
+ * @param evaluationMs - Wall-clock time spent obtaining the guard verdict
+ * @returns Nothing so Pi continues with the command
+ */
+export function resolvePassiveVerdict(
+  verdict: Verdict,
+  ctx: ExtensionContext,
+  accumulatedCostUsd: number | undefined,
+  evaluationMs: number,
+): undefined {
+  const source = verdict.failure === undefined
+    ? verdict.decision.toUpperCase()
+    : "FAILURE";
+  const result = `PASSIVE: ${source} · NOT ENFORCED`;
+  notifyRun(
+    ctx,
+    result,
+    verdict.usage?.inputTokens,
+    accumulatedCostUsd,
+    evaluationMs,
+    source === "ALLOW" ? "info" : "warning",
+  );
+  return undefined;
+}
+
 /**
  * Format the compact status Pi prints after each demur run.
  *
@@ -326,14 +381,40 @@ export default function demur(pi: ExtensionAPI): void {
       }
 
       await refreshSettings(ctx);
-      const toggleLabel = settings.enabled ? "Disable demur" : "Enable demur";
+      const modeLabel = `Change mode (current: ${settings.mode})`;
+      const trainingLabel = settings.training
+        ? "Disable training capture"
+        : "Enable training capture";
       const policyLabel = `Change failure policy (current: ${settings.failurePolicy})`;
-      const action = await ctx.ui.select("demur", [toggleLabel, policyLabel]);
+      const actions = settings.mode === "disabled"
+        ? [modeLabel, policyLabel]
+        : [modeLabel, trainingLabel, policyLabel];
+      const action = await ctx.ui.select("demur", actions);
       if (action === undefined) return;
 
-      if (action === toggleLabel) {
+      if (action === modeLabel) {
+        const selection = await ctx.ui.select(
+          `demur mode (current: ${settings.mode})`,
+          [...DEMUR_MODES],
+        );
+        if (selection === undefined) return;
+
+        const mode = parseDemurMode(selection);
+        if (mode === undefined) return;
+        await persistSettings(
+          {
+            ...settings,
+            mode,
+            training: mode === "disabled" ? false : settings.training,
+          },
+          ctx,
+        );
+        return;
+      }
+
+      if (action === trainingLabel) {
         await persistSettings(
-          { ...settings, enabled: !settings.enabled },
+          { ...settings, training: !settings.training },
           ctx,
         );
         return;
@@ -366,7 +447,7 @@ export default function demur(pi: ExtensionAPI): void {
     } catch (error: unknown) {
       settings = { ...DEFAULT_DEMUR_SETTINGS };
       ctx.ui.notify(
-        `Could not load demur settings; using enabled/block: ${errorDetail(error)}`,
+        `Could not load demur settings; using enforce/block with training off: ${errorDetail(error)}`,
         "warning",
       );
     }
@@ -395,14 +476,42 @@ function updateStatus(
   ctx: ExtensionContext,
   settings: DemurSettings,
 ): void {
-  const status = settings.enabled ? "enabled" : "disabled";
-  const color = settings.enabled ? "success" : "warning";
-  ctx.ui.setStatus("demur", ctx.ui.theme.fg(color, `demur: ${status}`));
+  const training = settings.training ? " + training" : "";
+  const color = settings.mode === "enforce" && !settings.training
+    ? "success"
+    : "warning";
+  ctx.ui.setStatus(
+    "demur",
+    ctx.ui.theme.fg(color, `demur: ${settings.mode}${training}`),
+  );
 }
 
 function settingsNotification(settings: DemurSettings): string {
-  const status = settings.enabled ? "enabled" : "disabled";
-  return `demur ${status} globally; failure policy: ${settings.failurePolicy}.`;
+  const training = settings.training ? "on" : "off";
+  return `demur mode: ${settings.mode}; training: ${training}; failure policy: ${settings.failurePolicy}.`;
+}
+
+async function recordTrainingResult(
+  command: string,
+  ctx: ExtensionContext,
+  mode: "enforce" | "passive",
+  verdict: Verdict,
+  result: ToolCallEventResult | undefined,
+): Promise<void> {
+  try {
+    await recordTrainingEvaluation({
+      command,
+      cwd: ctx.cwd,
+      mode,
+      verdict,
+      hostAction: result?.block === true ? "block" : "allow",
+    });
+  } catch (error: unknown) {
+    ctx.ui.notify(
+      `demur: could not record training evaluation — ${errorDetail(error)}`,
+      "warning",
+    );
+  }
 }
 
 async function handleGuardFailure(
@@ -521,26 +630,31 @@ function formatDecimal(value: number, fractionDigits: number): string {
     .replace(/\.0+$/, "");
 }
 
-function formatUsd(value: number): string {
-  const decimal = value.toFixed(9).replace(/0+$/, "").replace(/\.$/, "");
-  return `$${decimal}`;
-}
-
 function parseVerdict(output: string): Verdict {
   const value: unknown = JSON.parse(output);
   if (value === null || typeof value !== "object") {
     throw new Error("verdict must be an object");
   }
 
-  const { decision, reason } = value as Record<string, unknown>;
+  const { decision, reason, judgments, failure, latencyMs, usage } =
+    value as Record<string, unknown>;
   if (
     (decision !== "allow" && decision !== "ask" && decision !== "deny") ||
-    typeof reason !== "string"
+    typeof reason !== "string" ||
+    typeof latencyMs !== "number" ||
+    !Number.isFinite(latencyMs)
   ) {
-    throw new Error("verdict must contain a valid decision and reason");
+    throw new Error("verdict must contain a valid decision, reason, and latency");
   }
 
-  return value as Verdict;
+  return {
+    decision,
+    reason,
+    judgments: judgments as Verdict["judgments"],
+    failure: failure as Verdict["failure"],
+    latencyMs,
+    usage: usage as Verdict["usage"],
+  };
 }
 
 function errorDetail(error: unknown): string {

package/extensions/demur/paths.ts

@@ -0,0 +1,46 @@
+import { homedir } from "node:os";
+import { join } from "node:path";
+
+/**
+ * Resolve demur's configuration directory.
+ *
+ * `DEMUR_CONFIG_HOME` takes precedence over the XDG config directory and the
+ * standard home-directory fallback. Unlike `XDG_CONFIG_HOME`, the demur-specific
+ * override names demur's directory directly.
+ *
+ * @param environment - Process environment used to resolve directory overrides
+ * @param homeDirectory - Home directory used when overrides are absent
+ * @returns Absolute or caller-provided path to demur's configuration directory
+ */
+export function getDemurConfigDirectory(
+  environment: NodeJS.ProcessEnv = process.env,
+  homeDirectory: string = homedir(),
+): string {
+  if (environment.DEMUR_CONFIG_HOME) return environment.DEMUR_CONFIG_HOME;
+
+  const configHome = environment.XDG_CONFIG_HOME ||
+    join(homeDirectory, ".config");
+  return join(configHome, "demur");
+}
+
+/**
+ * Resolve demur's persistent state directory.
+ *
+ * `DEMUR_STATE_HOME` takes precedence over the XDG state directory and the
+ * standard home-directory fallback. Unlike `XDG_STATE_HOME`, the demur-specific
+ * override names demur's directory directly.
+ *
+ * @param environment - Process environment used to resolve directory overrides
+ * @param homeDirectory - Home directory used when overrides are absent
+ * @returns Absolute or caller-provided path to demur's persistent state directory
+ */
+export function getDemurStateDirectory(
+  environment: NodeJS.ProcessEnv = process.env,
+  homeDirectory: string = homedir(),
+): string {
+  if (environment.DEMUR_STATE_HOME) return environment.DEMUR_STATE_HOME;
+
+  const stateHome = environment.XDG_STATE_HOME ||
+    join(homeDirectory, ".local", "state");
+  return join(stateHome, "demur");
+}