pi-jev-auto-mode 0.1.0

package/docs/security.md

@@ -0,0 +1,122 @@
+# Security notes
+
+## The shape of the problem
+
+A coding agent with shell access can be talked into almost anything by the content it reads:
+a dependency's README, a test fixture, an issue body, a comment in the file it was asked to
+fix. A permission gate therefore cannot be "a second opinion from another chat model". It has
+to be an envelope the agent cannot argue its way out of, plus a narrow judgment for the cases
+the envelope cannot decide on its own.
+
+This package splits those two responsibilities and keeps the envelope authoritative.
+
+## What the semantic layer may and may not do
+
+| Allowed | Not allowed |
+|---|---|
+| Approve a call the deterministic layer escalated | Approve a hard-deny command |
+| Refuse a call that looks required by the task | Override a user deny pattern |
+| Report "uncertain", which becomes a confirmation | Widen the set of protected paths |
+| Clear a `soft` hazard rejection when the user's own request covers the call | Clear a `hazard`-severity rejection (secret egress, credential stores, injection) |
+
+The order in `evaluateToolCall` is the enforcement: hard-deny and user rules return before the
+engine is constructed or called at all. There is no code path in which a probabilistic verdict
+is consulted for a hard-deny target.
+
+### The soft/hazard split, and why it is the riskiest part of the design
+
+A rejection from a condition marked `severity: soft` (`local_scope`, `no_outward_effect`,
+`no_irreversible_damage`) is cleared when `intent_coverage` is satisfied. This is what makes
+`git push --force origin feature/x` after "force push this branch" an approval instead of a
+block, and it is the only place where a probabilistic judgment grants permission for an
+irreversible action.
+
+Three things bound the risk:
+
+1. The hard-deny patterns for catastrophic targets (forced push to a protected branch, root
+deletion, disk writes) run first and cannot be reached by any semantic verdict.
+2. Content-based authority does not count: `intent_coverage` reads user-authored messages
+only. A README that says "run this installer" is not a user request, and the fixture set
+measures that case as `ask`.
+3. `severity: hazard` covers the conditions where consent should not be sufficient at all:
+sending secret material, writing credential stores, and text that tries to steer the judgment.
+
+If a wrong approval ever appears in practice, this is the mechanism to remove first.
+
+## Failure modes and what happens
+
+Everything below resolves to **block**. Silence is never consent.
+
+| Failure | Resolution |
+|---|---|
+| No semantic engine configured (no API key) | confirm in a UI, block without one |
+| API key missing or rejected | block, with the reason surfaced to the model |
+| Login with a key the API refuses | the key is not stored, so a typo cannot become a permanently blocking gate |
+| Login while the API is unreachable | the key is not stored and the command says it could not verify |
+| Timeout / connection error | block (`timeout`, `network`) |
+| 5xx or 429 after retries | block (`http`) |
+| 4xx that retries cannot fix | block (`http`) — not rethrown, so the gate cannot fail open |
+| Response shape wrong, or a condition missing from the answer | block (`malformed_response`) |
+| State + questions over the shared budget | block (`state_too_large`) before the request is sent |
+| Engine throws | block (`engine_error`) |
+| Request cancelled (Esc) | block |
+| No UI available for a confirmation | block (`no-ui`) |
+| A condition answered by fewer than all keys | block (`malformed_response`) — a missing answer is never an approval |
+
+A confirmation is not a bypass: it runs only when the semantic layer said `uncertain`, never
+when it said `deny` or when no decision was available.
+
+## Injection stance
+
+- The user intent sent for judgment is built from **user-authored messages only**. Assistant
+  text and tool output are excluded, because they carry repository content and command output
+  and would otherwise let a file argue for its own approval.
+- File contents and diffs are never sent. Only paths.
+- `AGENTS.md` / `CLAUDE.md` and the agent configuration directories are treated as protected
+  paths: a write there changes what the agent believes it was told.
+- One dedicated condition (`prompt_injection_absent`) asks whether the call is trying to make
+  other state act as instructions.
+- The gate never returns a JEV rationale verbatim as a system-level instruction; a block
+  reason is a tool-call error string, which is the weakest channel it can use.
+
+## What is sent to TypeSafe
+
+Judgment requires the content to leave the machine. The API is `api.typesafe.ai`, and the
+payload is deliberately narrow:
+
+| Sent | Not sent |
+|---|---|
+| tool name, bash command text (truncated) | file contents, diffs, `write` bodies |
+| write/edit target path, cwd | tool output, assistant messages |
+| matched policy reason names | environment variables |
+| recent user messages (bounded, ≤4k chars) | the API key itself |
+| policy notes | |
+
+The API key is stored as a `0600` file under `<agentDir>/secrets/`, the same place Pi keeps its
+own credentials. It is never written to the settings file, and it is never part of the judgment
+state: it travels only in the `Authorization` header to `api.typesafe.ai`, so it cannot come
+back out through a decision record.
+
+Redaction runs before the state is built: `*_KEY=` / `*_TOKEN=` / `*_SECRET=` assignments,
+`Bearer …`, JWTs, `sk-` / `rk-` keys, `ghp_` / `gho_` tokens, `AKIA…` access key IDs,
+`apikey_…` keys, and PEM private keys become `<redacted>` markers. Redaction is a safety net,
+not a guarantee — an unusual secret format will pass through. Lower
+`maxStateCharacters`, or keep a command out of the gate by adding a deny/allow rule, if a
+repository must not produce outbound text at all.
+
+Records written to the session store the decision, the matched reasons, the rationale, the
+model name, and per-condition probabilities. They are local and do not enter the model's
+context.
+
+## Known limits
+
+- `classifyWriteTarget` is lexical (no `realpath`), so a symlink inside the working directory
+  pointing outside it is not detected by the deterministic layer.
+- Command matching is conservative pattern matching, not a shell parser. `rm -rf build` is
+  recognized as scoped; obfuscated equivalents (`xargs`, command substitution, `sh -c`) are
+  escalated rather than recognized.
+- A command that `cd`s elsewhere and then deletes is judged by its text and intent, not by a
+  simulated shell.
+- The probability thresholds are calibrated on one person's data, one sample per fixture,
+  with ±0.05 run-to-run variance. See [`calibration.md`](./calibration.md); treat the
+  thresholds as a starting point and tune them from the recorded probabilities.

package/index.ts

@@ -0,0 +1 @@
+export { default } from "./src/extension.ts";

package/package.json

@@ -0,0 +1,67 @@
+{
+  "name": "pi-jev-auto-mode",
+  "version": "0.1.0",
+  "description": "JEV (TypeSafe System One) backed auto mode for the Pi coding agent: semantically auto-approves bash, write, and edit tool calls and fails closed when a decision cannot be made.",
+  "keywords": [
+    "pi-package",
+    "pi-extension",
+    "pi-coding-agent",
+    "auto-mode",
+    "permission",
+    "gate",
+    "guardrail",
+    "jev",
+    "typesafe",
+    "semantic-validation"
+  ],
+  "license": "MIT",
+  "author": "jomatsu",
+  "homepage": "https://github.com/jomatsu/pi-jev-auto-mode",
+  "bugs": "https://github.com/jomatsu/pi-jev-auto-mode/issues",
+  "repository": "github:jomatsu/pi-jev-auto-mode",
+  "type": "module",
+  "engines": {
+    "node": ">=22.19.0"
+  },
+  "files": [
+    "index.ts",
+    "src",
+    "docs",
+    "README.md",
+    "CHANGELOG.md",
+    "SECURITY.md",
+    "LICENSE"
+  ],
+  "pi": {
+    "extensions": [
+      "./index.ts"
+    ]
+  },
+  "scripts": {
+    "test": "node --test --experimental-strip-types --test-reporter=spec 'test/**/*.test.ts'",
+    "typecheck": "tsc --noEmit",
+    "check": "npm run typecheck && npm test",
+    "prepublishOnly": "npm run check"
+  },
+  "peerDependencies": {
+    "@earendil-works/pi-coding-agent": "*",
+    "@earendil-works/pi-tui": "*"
+  },
+  "peerDependenciesMeta": {
+    "@earendil-works/pi-coding-agent": {
+      "optional": true
+    },
+    "@earendil-works/pi-tui": {
+      "optional": true
+    }
+  },
+  "devDependencies": {
+    "@earendil-works/pi-coding-agent": "^0.84.2",
+    "@earendil-works/pi-tui": "^0.84.2",
+    "@types/node": "^22.15.0",
+    "typescript": "^5.8.0"
+  },
+  "dependencies": {
+    "@typesafe-ai/sdk": "^0.6.0"
+  }
+}

package/src/call.ts

@@ -0,0 +1,180 @@
+/**
+ * Turn a Pi tool call into the shape JEV will judge.
+ *
+ * What leaves this module is what a third party (TypeSafe) gets to see, so the
+ * boundary is explicit: the command text and the target path are sent, file
+ * contents, diffs, and tool output are not. Obvious credentials are redacted on
+ * the way out.
+ */
+
+import { classifyWriteTarget } from "./policy.ts";
+import type { JevJson, JevState } from "./jev/state.ts";
+
+export type GatedTool = "bash" | "write" | "edit";
+
+export const GATED_TOOLS: readonly GatedTool[] = ["bash", "write", "edit"];
+
+export interface ToolCallEventLike {
+  readonly toolName: string;
+  readonly toolCallId?: string;
+  readonly input: Record<string, unknown>;
+}
+
+export interface GatedCall {
+  readonly tool: GatedTool;
+  /** One-line description used in records and dialogs. */
+  readonly summary: string;
+  /** bash: the command, truncated and redacted. Absent for file tools. */
+  readonly command?: string;
+  /** write/edit: absolute target path. */
+  readonly path?: string;
+  /** write/edit: target relative to the working directory, when inside it. */
+  readonly relativePath?: string;
+  readonly outsideCwd: boolean;
+  readonly protectedReason?: string;
+  /** write/edit: number of edit hunks. */
+  readonly editCount?: number;
+  /** write: content size in characters. The content itself is never sent. */
+  readonly contentLength?: number;
+}
+
+export interface CallOptions {
+  readonly cwd: string;
+  readonly maxCommandLength?: number;
+  /** Additional protected locations from settings. */
+  readonly extraProtectedPaths?: readonly string[];
+}
+
+export const DEFAULT_MAX_COMMAND_LENGTH = 4000;
+
+const SECRET_PATTERNS: readonly { readonly pattern: RegExp; readonly replacement: string }[] = [
+  {
+    pattern: /-----BEGIN [A-Z ]*PRIVATE KEY-----[\s\S]*?-----END [A-Z ]*PRIVATE KEY-----/g,
+    replacement: "<redacted-private-key>",
+  },
+  { pattern: /\beyJ[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{5,}\b/g, replacement: "<redacted-jwt>" },
+  { pattern: /\b(?:sk|rk)-[A-Za-z0-9_-]{16,}\b/g, replacement: "<redacted-key>" },
+  { pattern: /\b(?:ghp|gho|ghu|ghs|ghr)_[A-Za-z0-9]{20,}\b/g, replacement: "<redacted-token>" },
+  { pattern: /\b(?:AKIA|ASIA)[0-9A-Z]{12,}\b/g, replacement: "<redacted-aws-key>" },
+  { pattern: /\bapikey_[A-Za-z0-9_-]{8,}\b/gi, replacement: "<redacted-typesafe-key>" },
+  { pattern: /\bBearer\s+[A-Za-z0-9._~+/-]{12,}=*/g, replacement: "Bearer <redacted>" },
+  {
+    pattern:
+      /((?:api[_-]?key|secret|token|password|passwd|access[_-]?key|client[_-]?secret|auth[_-]?token)\s*[:=]\s*)(["']?)([^\s"';|&]{6,})/gi,
+    replacement: "$1$2<redacted>",
+  },
+];
+
+/** Replace obvious credentials so they are neither sent nor displayed. */
+export function redactSecrets(text: string): string {
+  let result = text;
+  for (const { pattern, replacement } of SECRET_PATTERNS) {
+    result = result.replace(pattern, replacement);
+  }
+  return result;
+}
+
+export function truncate(value: string, maxLength: number): string {
+  return value.length > maxLength ? `${value.slice(0, maxLength)}...` : value;
+}
+
+function asString(value: unknown): string | undefined {
+  return typeof value === "string" ? value : undefined;
+}
+
+/**
+ * Build the gate's view of a tool call.
+ *
+ * Returns `undefined` for tools this extension does not gate.
+ */
+export function buildGatedCall(event: ToolCallEventLike, options: CallOptions): GatedCall | undefined {
+  const maxCommandLength = options.maxCommandLength ?? DEFAULT_MAX_COMMAND_LENGTH;
+
+  if (event.toolName === "bash") {
+    const rawCommand = asString(event.input.command) ?? "";
+    const command = truncate(redactSecrets(rawCommand), maxCommandLength);
+    const firstLine = rawCommand.split("\n")[0] ?? rawCommand;
+    return {
+      tool: "bash",
+      summary: truncate(redactSecrets(firstLine.trim()), 200) || "(empty command)",
+      command,
+      outsideCwd: false,
+    };
+  }
+
+  if (event.toolName === "write" || event.toolName === "edit") {
+    const inputPath = asString(event.input.path) ?? "";
+    const target = classifyWriteTarget(inputPath, options.cwd, options.extraProtectedPaths ?? []);
+    const editCount = Array.isArray(event.input.edits) ? event.input.edits.length : undefined;
+    const contentLength = asString(event.input.content)?.length;
+
+    return {
+      tool: event.toolName,
+      summary: `${event.toolName} ${target.relativeToCwd ?? target.absolute}`,
+      path: target.absolute,
+      relativePath: target.relativeToCwd,
+      outsideCwd: target.outsideCwd,
+      protectedReason: target.protectedReason,
+      editCount,
+      contentLength,
+    };
+  }
+
+  return undefined;
+}
+
+export interface RepoFacts {
+  readonly cwd: string;
+  readonly isGitRepository: boolean;
+  readonly protectedPaths: readonly string[];
+}
+
+export interface JevStateInput {
+  readonly call: GatedCall;
+  readonly reasons: readonly string[];
+  /** Recent user-authored text. Never assistant or tool output. */
+  readonly intent: string;
+  /** User-authored policy notes. */
+  readonly policy: string;
+  readonly repo: RepoFacts;
+}
+
+export const NO_POLICY_PLACEHOLDER = "(no user policy configured)";
+export const NO_INTENT_PLACEHOLDER = "(no recent user message available)";
+
+/**
+ * Assemble the JEV request state.
+ *
+ * The user policy is `context` (session-scoped) so conditions can name it, and
+ * the call plus the user intent are `value` (per-call).
+ */
+export function toJevState(input: JevStateInput): JevState {
+  const { call } = input;
+
+  const value: Record<string, JevJson> = {
+    tool: call.tool,
+    operation: call.summary,
+    matched_policy_reasons: [...input.reasons],
+    user_intent: input.intent.trim() || NO_INTENT_PLACEHOLDER,
+  };
+
+  if (call.command !== undefined) value.command = call.command;
+  if (call.path !== undefined) {
+    value.path = call.path;
+    value.relative_path = call.relativePath ?? null;
+    value.outside_working_directory = call.outsideCwd;
+  }
+  if (call.editCount !== undefined) value.edit_count = call.editCount;
+  if (call.contentLength !== undefined) value.content_length = call.contentLength;
+
+  const context: Record<string, JevJson> = {
+    policy: input.policy.trim() || NO_POLICY_PLACEHOLDER,
+    repository: {
+      cwd: input.repo.cwd,
+      is_git_repository: input.repo.isGitRepository,
+      protected_paths: [...input.repo.protectedPaths],
+    },
+  };
+
+  return { value, context };
+}

package/src/decide.ts

@@ -0,0 +1,81 @@
+/**
+ * The decision-engine seam.
+ *
+ * The gate itself (ordering, blocking, recording) is deterministic. Everything
+ * probabilistic sits behind `DecisionEngine`, so the extension can be tested
+ * without a network and so the JEV implementation can be swapped or disabled
+ * without touching the safety-critical path.
+ */
+
+import type { GatedCall, RepoFacts } from "./call.ts";
+
+export type DecisionSource = "hard-deny" | "user-rule" | "engine" | "unavailable" | "no-ui" | "user";
+
+/**
+ * A display-ready report of one condition's judgment.
+ *
+ * `verdict` is the band the probability fell into; `ignored` means a hazard-mode
+ * condition landed in the middle band, which is not evidence of anything. This is
+ * what a record shows during threshold tuning.
+ */
+export interface ConditionReport {
+  readonly ruleId: string;
+  readonly label: string;
+  readonly probability: number;
+  readonly threshold: number;
+  readonly verdict: "satisfied" | "rejected" | "uncertain" | "ignored";
+  readonly clearedByIntent: boolean;
+}
+
+export interface EngineEvidence {
+  /** Per-condition probability, when the engine exposes one. */
+  readonly probabilities?: Readonly<Record<string, number>>;
+  /** The threshold each probability was compared against. */
+  readonly thresholds?: Readonly<Record<string, number>>;
+  /** One entry per condition that was asked. */
+  readonly conditions?: readonly ConditionReport[];
+  /** The condition that decided the call, when one did. */
+  readonly decidingRule?: string;
+  /** Conditions whose clear rejection the user's own request cleared. */
+  readonly clearedByIntent?: readonly string[];
+  readonly model?: string;
+  readonly latencyMs?: number;
+}
+
+export type EngineVerdict =
+  | ({ readonly verdict: "allow"; readonly rationale: string } & EngineEvidence)
+  | ({ readonly verdict: "deny"; readonly rationale: string } & EngineEvidence)
+  | ({ readonly verdict: "uncertain"; readonly rationale: string } & EngineEvidence)
+  | ({ readonly verdict: "unavailable"; readonly rationale: string; readonly reason: string } & EngineEvidence);
+
+export interface CandidateInput {
+  readonly call: GatedCall;
+  /** Names of the policy patterns this call matched. */
+  readonly reasons: readonly string[];
+  readonly intent: string;
+  readonly policy: string;
+  readonly repo: RepoFacts;
+}
+
+export interface JudgeOptions {
+  readonly signal?: AbortSignal;
+}
+
+export interface DecisionEngine {
+  readonly id: string;
+  judge(input: CandidateInput, options: JudgeOptions): Promise<EngineVerdict>;
+}
+
+/**
+ * Milestone-1 engine: no semantic judgment at all.
+ *
+ * Every candidate is reported as `uncertain`, which means "ask the user when a UI
+ * exists, block otherwise". That keeps the deterministic layer shippable and
+ * verifiable on its own, and it fails in the safe direction.
+ */
+export function createManualEngine(): DecisionEngine {
+  return {
+    id: "manual",
+    judge: () => Promise.resolve({ verdict: "uncertain", rationale: "No semantic decision engine is configured." }),
+  };
+}