@agent-corpus/policy-manager 0.4.0 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -46,6 +46,18 @@ Setup merges only the managed hook definitions. Existing custom hooks, permissio
46
46
 
47
47
  Start a new session after setup. Codex still requires explicit review and trust through `/hooks`; setup does not grant that trust. Existing sandbox controls and host permission rules continue to apply. Ambiguous requests retain the documented classification or human fallback; setup does not enable automatic AI fallback across unsupported native interfaces.
48
48
 
49
+ ### Codex hook trust
50
+
51
+ Codex runs a non-managed hook only after you review and trust its exact definition, and setup never grants that trust. After `setup-hook-codex`, start Codex and open `/hooks`, review the five policy-manager hooks, and trust them. Until then Codex skips them and prints a warning at startup, so no policy decision is made.
52
+
53
+ Trust is recorded per hook in `config.toml` under `[hooks.state."<hooks.json path>:<event>:0:0"]` as `trusted_hash = "sha256:<hex>"`. The hash covers the event, the matcher, and the hook's command and timeout, so:
54
+
55
+ - Trust is tied to the command text only: Codex hashes the command line, not the script behind it. An upgrade that keeps the same CLI path (for example an in-place `npm install -g` upgrade) keeps trust while the code behind the hooks changes, so review a new version before upgrading in place. An upgrade that changes the command, such as installing into a per-version directory or a Node upgrade that changes `process.execPath` (it is part of the command), makes Codex mark all five hooks as changed and skip them until you trust them again. Re-running `setup-hook-codex` for the same installed version changes nothing and keeps trust.
56
+ - Codex hashes the effective timeout, which it limits to 1 to 3 seconds for `SessionEnd`. The generated `SessionEnd` timeout is 3 seconds, the same effective value as the earlier 8, so changing it did not by itself invalidate trust.
57
+ - `--dangerously-bypass-hook-trust` runs enabled hooks without persisted trust for one invocation. Use it only for automation that already vets the hook source.
58
+
59
+ Codex hook trust was checked against Codex CLI 0.167.1; see the [Codex hooks reference](https://developers.openai.com/codex/hooks) for the release behavior.
60
+
49
61
  ## Request protocol
50
62
 
51
63
  Pass one JSON object on standard input:
@@ -70,6 +82,35 @@ The session's policy-manager agent returns the following object for an ambiguous
70
82
 
71
83
  Categories are `code-read`, `worktree-write`, `cloud-read`, `git-read`, `impactful-write`, `destructive`, `sensitive`, and `unknown`. The threshold is 0.99 plus deterministic verification and category agreement. Explicitly listed metadata operations with permitted selectors can qualify through AI classification. Endpoint overrides and cloud writes cannot qualify. Unknown commands stay human decisions. No classifier starts a second harness or calls an external model API.
72
84
 
85
+ ## AI classification of ambiguous requests (experiment)
86
+
87
+ Under a `permissive` profile, a person is asked only for what code says must stay with a person. Every other escalation is _ambiguous_: the deterministic engine cannot decide it, so the session's AI decides it instead of the person. Strict mode, and calls made outside the hooks (`policy-manager review`), are unchanged.
88
+
89
+ How a hook call flows (Claude Code, Codex and Copilot `PreToolUse`):
90
+
91
+ 1. **First call.** The command is _not run_ and the person is not asked. The hook blocks it with a reason that tells the model to ask the session's policy-manager agent whether the exact command is recoverable and non-sensitive, to record the answer with `policy-manager classify`, and to retry the same command.
92
+ 2. **Classified.** `classify` takes `{requestHash, category, confidence, recoverable, rationale}`. The request is approved only when `recoverable` is `true`, `confidence` is at least 0.95, and the category is not `sensitive`, `destructive` or `unknown`. Anything else is a **deny with a reason** the model sees and works around. No person is involved either way. The outcome is remembered for five minutes for that exact request, and a verdict is final in that time: a deny cannot be reclassified into an approve.
93
+ 3. **Fallback.** If the classifier is unavailable or its answer is invalid, the retry reaches a person through the normal prompt (under Codex, which cannot ask, it is a deny that tells the model to have the user run the command themselves or change the profile). This happens at most once per request, so there is never a loop and never a lost prompt. After 50 AI approvals in a session, further ambiguous requests go to a person.
94
+
95
+ **What always stays with a person, in code, and is never AI-relaxable** (these are the owner's calls; they list exactly what this experiment does not change):
96
+
97
+ - Credential and secret retrieval, protected paths and configuration such as the policy profile and hook settings (category `sensitive`).
98
+ - Destructive operations: unrecoverable deletion, unmerged-work loss, cloud deletion (category `destructive`).
99
+ - Commands matching the profile's `humanPatterns`.
100
+ - Writes outside a linked feature worktree (main clones and protected branches).
101
+ - Cloud operations outside the read catalog. The profile's `cloudWrites: "allow"` is the way to automate these.
102
+ - Requests the engine cannot trust: incomplete or invalid requests, a different execution directory, unrecognized patches, and environment or loader overrides such as `LD_PRELOAD`.
103
+
104
+ **Known weakness.** The classifier is the working model's own policy-manager agent, and the working model records its answer, so a model that wanted to approve itself could. Confidence is self-reported. This is why the experiment records everything and is reviewed after two days of use rather than trusted.
105
+
106
+ **Recording.** The audit log gets `classification-requested` (with the command, redacted: secret-looking values, URL credentials and long tokens are replaced and the text is capped), `classification` (category, confidence, recoverable, redacted rationale), `decision` (with the stage `requested`, `approved`, `denied` or `fallback`), and `classification-fallback` (with the reason `unavailable`, `invalid-result` or `limit`). Review them across sessions:
107
+
108
+ ```sh
109
+ policy-manager review-classifications [--since 2026-11-12T00:00:00Z] [--json]
110
+ ```
111
+
112
+ The report shows how many ambiguous requests were approved, denied, fell back to a person (and why), and never resolved; a confidence histogram; every AI-approved command with its outcome; approved commands that then failed; and commands a person later denied. It reads the session logs under the temporary directory, which the operating system clears after a few days, so run it within two days of starting. `replay` prints how many escalations in recorded transcripts would go to the classifier instead of a person.
113
+
73
114
  ## Policy
74
115
 
75
116
  - Ordinary file reads must resolve inside a Git repository.
@@ -129,7 +170,7 @@ The cache belongs to the service process and is scoped to harness plus session I
129
170
 
130
171
  `SessionStart` reports the private temporary directory containing `audit.jsonl`. Each request and decision is recorded before an approval is returned. The log stores versioned event envelopes, readable action summaries, opaque session identifiers, request hashes, and hashed targets. It excludes paths, argument values, source text, credentials, and model explanations. Compare request hashes against the original request in the harness transcript during an investigation. Logs rotate at 10 MiB per segment and stop accepting writes at 100 MiB total. Earlier segments remain available through `audit-log query` and `audit-log export`; no records are automatically deleted. Logs remain after normal shutdown and can disappear during operating-system cleanup. Copy them before cleanup when needed.
131
172
 
132
- Logging failure, malformed requests, service failure, and unknown semantics never produce an automatic approval. Permission hooks fall through to native human handling. Pre-tool failures exit 2. Host timeout behavior differs: Copilot command-hook timeouts fall through to its normal permission flow. Codex does not support `ask` in `PreToolUse`, so human recommendations add context and leave native permission handling unchanged.
173
+ Logging failure, malformed requests, service failure, and unknown semantics never produce an automatic approval. Permission hooks fall through to native human handling. Pre-tool failures exit 2. Host timeout behavior differs: Copilot command-hook timeouts fall through to its normal permission flow. Codex does not support `ask` in `PreToolUse`: it parses the value, marks the hook failed and runs the tool anyway. A decision that needs a human is therefore returned to Codex as a `deny` whose reason says the command was not run and that approving it in chat does not unblock it (the decision is deterministic, so a retry is denied again); there is no in-Codex button to approve it. The user runs the command themselves or changes the policy profile. An ambiguous cloud request is the one exception: its reason points to the session policy-manager agent, and a classification of approved is accepted for the session, so a retry passes. Codex's own approval prompt (`PermissionRequest`) is left to Codex for the requests it would ask about, and an approval is returned as `allow`.
133
174
 
134
175
  ## Audit API and reported outcomes
135
176
 
@@ -159,9 +200,9 @@ Or an execution report:
159
200
  }
160
201
  ```
161
202
 
162
- Human responses are `approved` or `denied`. Execution statuses are `completed`, `succeeded`, `failed`, or `cancelled`. Reports use the source `policy-manager.adapter-report`. They are observations declared by the caller and never modify permission decisions or caches. The API does not authenticate a human identity. A denial policy remains a human decision even after an approval report is recorded.
203
+ Human responses are `approved` or `denied`. Execution statuses are `completed`, `succeeded`, `failed`, `cancelled`, or `unknown`. `unknown` is recorded for Codex when the hook payload exposes no exit status, which is the usual case for a Bash result. Reports use the source `policy-manager.adapter-report`. They are observations declared by the caller and never modify permission decisions or caches. The API does not authenticate a human identity. A denial policy remains a human decision even after an approval report is recorded.
163
204
 
164
- Generated post-tool hooks record execution reports for the supported tool matcher. Claude and Copilot also get failure hooks. Codex uses `PostToolUse`, which can carry nonzero exit codes. When an exit code is absent, the adapter records `completed` rather than inventing success. Output text and error bodies are discarded. Human responses require an explicit report from the host; the native hooks do not provide a universal human-response signal.
205
+ Generated post-tool hooks record execution reports for the supported tool matcher. Claude and Copilot also get failure hooks. Codex uses `PostToolUse`, which also runs after commands that exit non-zero. Codex sends a Bash result as the command's output text only, with no exit code and no duration, so the adapter records `unknown` for Codex rather than `completed` or `succeeded`; a structured `exit_code` is used when one is present. Claude and Copilot record `completed` when an exit code is absent. Output text and error bodies are discarded. Human responses require an explicit report from the host; the native hooks do not provide a universal human-response signal.
165
206
 
166
207
  The shared logger redacts structured sensitive fields, validates events, locks concurrent access, and flushes each append. Queries cover active and rotated segments. Busy storage, a stale lock, incomplete records, capacity limits, or write failures prevent automatic approval. The initial API is a library plus CLI; an additional resident logging server is deferred until another consumer needs one.
167
208
 
@@ -172,7 +213,7 @@ Tests exercise real service processes and native-shaped hook payloads for all th
172
213
  Verified documentation, November 5, 2026:
173
214
 
174
215
  - [Claude Code hooks](https://code.claude.com/docs/en/hooks)
175
- - [Codex hooks](https://learn.chatgpt.com/docs/hooks)
216
+ - [Codex hooks](https://developers.openai.com/codex/hooks)
176
217
  - [Copilot hooks](https://docs.github.com/en/copilot/reference/hooks-reference)
177
218
 
178
219
  Exit codes: 0 means a protocol response was returned; inspect `decision` before acting. 1 means invalid input or service failure. Pre-tool hook failures use 2. Diagnostics use stderr and contain no raw request data.
@@ -1,5 +1,15 @@
1
1
  import type { Decision, OutcomeReport, Request } from "./types.js";
2
2
  export declare function normalize(harness: Request["harness"], payload: Record<string, unknown>): Request;
3
- export declare function hookResponse(harness: Request["harness"], phase: string, decision: Decision): Record<string, unknown>;
3
+ export interface HookContext {
4
+ cliPath: string;
5
+ sessionId: string;
6
+ }
7
+ /**
8
+ * The exact command the model is told to run to record a classification. A here-string, not a
9
+ * pipe: `printf ... | node ...` is "piping into an interpreter" and is itself escalated, so the
10
+ * round trip could never finish. A test evaluates this exact string and requires `approve`.
11
+ */
12
+ export declare function classifyCommand(harness: Request["harness"], context?: HookContext, json?: string): string;
13
+ export declare function hookResponse(harness: Request["harness"], phase: string, decision: Decision, context?: HookContext): Record<string, unknown>;
4
14
  export declare function executionReport(request: Request, phase: string, payload: Record<string, unknown>): OutcomeReport;
5
15
  //# sourceMappingURL=adapters.d.ts.map
package/dist/adapters.js CHANGED
@@ -15,7 +15,55 @@ export function normalize(harness, payload) {
15
15
  input: args,
16
16
  };
17
17
  }
18
- export function hookResponse(harness, phase, decision) {
18
+ /**
19
+ * The exact command the model is told to run to record a classification. A here-string, not a
20
+ * pipe: `printf ... | node ...` is "piping into an interpreter" and is itself escalated, so the
21
+ * round trip could never finish. A test evaluates this exact string and requires `approve`.
22
+ */
23
+ export function classifyCommand(harness, context, json = "<json>") {
24
+ // A Windows path (`D:\\a\\cli.js`) is written with forward slashes: Node accepts them, and a
25
+ // backslash inside double quotes is a shell escape the policy cannot judge, which would
26
+ // escalate the command that finishes the round trip.
27
+ const cliPath = context && /^[A-Za-z]:\\/.test(context.cliPath)
28
+ ? context.cliPath.replace(/\\/g, "/")
29
+ : context?.cliPath;
30
+ const safe = context &&
31
+ cliPath &&
32
+ /^[^"$`\\\n']+$/.test(cliPath) &&
33
+ /^[A-Za-z0-9_.-]+$/.test(context.sessionId);
34
+ return safe
35
+ ? `node "${cliPath}" classify ${harness} ${context.sessionId} <<< '${json}'`
36
+ : `policy-manager classify ${harness} <session-id> <<< '${json}'`;
37
+ }
38
+ /** What a blocked ambiguous request tells the model to do. Never ask for human approval here. */
39
+ function classifyInstruction(harness, decision, context) {
40
+ const command = classifyCommand(harness, context);
41
+ return `Not run yet: this request needs classification before it can run (request ${decision.requestHash}; ${decision.reason}). Ask the session's policy-manager agent whether the exact command is recoverable and non-sensitive. It answers with {"requestHash":"${decision.requestHash}","category":"code-read|worktree-write|cloud-read|git-read|impactful-write|destructive|sensitive|unknown","confidence":0.0-1.0,"recoverable":true|false,"rationale":"one short sentence without quotes"}. Record that answer with this command, replacing <json> with the answer and using no single quotes in it: ${command} Then retry the same command unchanged. If you cannot get a classification, retry once and the user will be asked.`;
42
+ }
43
+ export function hookResponse(harness, phase, decision, context) {
44
+ if (decision.decision === "deny") {
45
+ const reason = decision.classification === "requested"
46
+ ? classifyInstruction(harness, decision, context)
47
+ : decision.reason;
48
+ if (phase === "PermissionRequest")
49
+ return harness === "copilot"
50
+ ? {}
51
+ : {
52
+ hookSpecificOutput: {
53
+ hookEventName: phase,
54
+ decision: { behavior: "deny", message: reason },
55
+ },
56
+ };
57
+ return harness === "copilot"
58
+ ? { permissionDecision: "deny", permissionDecisionReason: reason }
59
+ : {
60
+ hookSpecificOutput: {
61
+ hookEventName: phase,
62
+ permissionDecision: "deny",
63
+ permissionDecisionReason: reason,
64
+ },
65
+ };
66
+ }
19
67
  if (harness === "copilot") {
20
68
  if (phase === "PermissionRequest")
21
69
  return decision.decision === "approve" ? { behavior: "allow" } : {};
@@ -29,13 +77,24 @@ export function hookResponse(harness, phase, decision) {
29
77
  : {};
30
78
  if (decision.decision === "approve")
31
79
  return { hookSpecificOutput: { hookEventName: phase, permissionDecision: "allow" } };
32
- if (harness === "codex")
80
+ // Codex parses permissionDecision "ask" but does not support it: it marks the hook failed and
81
+ // runs the tool anyway (https://developers.openai.com/codex/hooks). Only "deny" stops a command,
82
+ // so a decision that needs a human is a deny. Approving it in chat does not change the decision,
83
+ // so the reason says so; the one in-session way through is classifying an ambiguous cloud
84
+ // request (needsClassification), which a retry then sees as approved. Codex's own approval prompt
85
+ // (PermissionRequest, above) still handles the requests it would ask about.
86
+ if (harness === "codex") {
87
+ const next = decision.needsClassification === true && decision.classification !== "fallback"
88
+ ? "Classify it with the session policy-manager agent and retry. If it is not classified as approved, ask the user to run it themselves or to change the policy profile."
89
+ : "Approving it in this session does not unblock it: ask the user to run it themselves or to change the policy profile.";
33
90
  return {
34
91
  hookSpecificOutput: {
35
92
  hookEventName: phase,
36
- additionalContext: `${decision.reason}. Request ${decision.requestHash}. The policy manager grants no approval. Classify ambiguous requests with the session policy-manager agent. Preserve the native human approval flow.`,
93
+ permissionDecision: "deny",
94
+ permissionDecisionReason: `Policy manager: ${decision.reason}. This needs a human decision and was not run. ${next} Request ${decision.requestHash}.`,
37
95
  },
38
96
  };
97
+ }
39
98
  return {
40
99
  hookSpecificOutput: {
41
100
  hookEventName: phase,
@@ -51,6 +110,10 @@ export function executionReport(request, phase, payload) {
51
110
  const exitCode = typeof code === "number" && Number.isSafeInteger(code) && Math.abs(code) <= 2147483647
52
111
  ? code
53
112
  : undefined;
113
+ // Codex sends a Bash result as the output text alone: no exit code and no duration, even for a
114
+ // command that exited non-zero. That is not "completed"; the outcome is unknown. Claude and
115
+ // Copilot keep their existing behaviour.
116
+ const unknown = exitCode === undefined && request.harness === "codex";
54
117
  const duration = payload.duration_ms ?? payload.durationMs;
55
118
  return {
56
119
  requestHash: hash(request),
@@ -60,7 +123,9 @@ export function executionReport(request, phase, payload) {
60
123
  ? "cancelled"
61
124
  : "failed"
62
125
  : exitCode === undefined
63
- ? "completed"
126
+ ? unknown
127
+ ? "unknown"
128
+ : "completed"
64
129
  : exitCode === 0
65
130
  ? "succeeded"
66
131
  : "failed",
package/dist/cli.js CHANGED
@@ -7,6 +7,7 @@ import { executionReport, hookResponse, normalize } from "./adapters.js";
7
7
  import { hookConfig } from "./config.js";
8
8
  import { defaultProfilePath, loadProfile, parseProfile } from "./profile.js";
9
9
  import { replayTranscripts } from "./replay.js";
10
+ import { formatReview, reviewClassifications, sessionDirectories } from "./review.js";
10
11
  import { send, serve, start, stop } from "./service.js";
11
12
  import { SetupError, setupHook } from "./setup.js";
12
13
  const usage = `policy-manager usage [--json]
@@ -20,6 +21,7 @@ policy-manager config <claude|codex|copilot> <absolute-installed-cli-path>
20
21
  policy-manager profile validate <file> # check a policy profile without installing it
21
22
  policy-manager profile install <file> # install it to the user config directory outside Git
22
23
  policy-manager replay <transcript-dir-or-file...> [--profile <file>] [--files <n>] # approval rate over recorded Bash commands; nothing is executed
24
+ policy-manager review-classifications [--since <iso-time>] [--dir <session-dir>...] [--json] # risk review of AI-classified ambiguous requests
23
25
  policy-manager stop <claude|codex|copilot> <session-id>
24
26
  A per-session background service checks policy and writes private temporary audit.jsonl logs.
25
27
  AI classification comes from the session's policy-manager agent, never an external API.
@@ -90,12 +92,26 @@ try {
90
92
  const report = replayTranscripts(positionals, profile, Number(values.files ?? 50));
91
93
  const pct = (n) => `${((100 * n) / Math.max(report.commands, 1)).toFixed(1)}%`;
92
94
  console.log(`${report.commands} unique commands; approved ${pct(report.approved)} (strict ${pct(report.strictApproved)})`);
95
+ const humans = report.commands - report.approved;
96
+ console.log(`${report.classified} of ${humans} escalations go to AI classification instead of a person; ${humans - report.classified} stay with a human by rule`);
93
97
  for (const row of report.reasons.slice(0, 15)) {
94
98
  console.log(`${row.count}\t${row.reason}`);
95
99
  for (const example of row.examples)
96
100
  console.log(`\t ${example}`);
97
101
  }
98
102
  }
103
+ else if (action === "review-classifications") {
104
+ const { values } = parseArgs({
105
+ args: args.slice(1),
106
+ options: {
107
+ since: { type: "string" },
108
+ dir: { type: "string", multiple: true },
109
+ json: { type: "boolean" },
110
+ },
111
+ });
112
+ const review = await reviewClassifications(sessionDirectories(values.dir ?? []), values.since);
113
+ console.log(values.json ? JSON.stringify(review, null, 2) : formatReview(review));
114
+ }
99
115
  else if (action === "serve")
100
116
  await serve(harness, phase ?? "");
101
117
  else if (action === "stop")
@@ -120,7 +136,7 @@ try {
120
136
  else {
121
137
  const directory = await start(harness, request.sessionId, cliPath);
122
138
  if (phase === "SessionStart") {
123
- const context = `Policy service started. Audit log: ${join(directory, "audit.jsonl")}. Native hooks route tool and permission requests directly to the deterministic service. For ambiguous requests, use a configured policy-manager agent or retain native human approval. If .agent-config/agents/policy-manager.md exists, read it before creating a read-only policy-manager agent. Never ask an agent to approve its own tools.`;
139
+ const context = `Policy service started. Audit log: ${join(directory, "audit.jsonl")}. Native hooks route tool and permission requests directly to the deterministic service. An ambiguous request is blocked once with instructions to have the policy-manager agent classify it and then retry; the person is asked only if classification fails. If .agent-config/agents/policy-manager.md exists, read it before creating a read-only policy-manager agent. Never ask an agent to approve its own tools.`;
124
140
  console.log(JSON.stringify(harness === "copilot"
125
141
  ? { additionalContext: context }
126
142
  : { hookSpecificOutput: { hookEventName: phase, additionalContext: context } }));
@@ -136,8 +152,9 @@ try {
136
152
  const decision = (await send(harness, request.sessionId, {
137
153
  action: "review",
138
154
  request,
155
+ phase,
139
156
  }));
140
- console.log(JSON.stringify(hookResponse(harness, phase, decision)));
157
+ console.log(JSON.stringify(hookResponse(harness, phase, decision, { cliPath, sessionId: request.sessionId })));
141
158
  }
142
159
  else
143
160
  throw new Error("Unsupported hook event");
package/dist/config.js CHANGED
@@ -16,6 +16,8 @@ export function hookConfig(harness, cliPath) {
16
16
  "SessionEnd",
17
17
  ];
18
18
  for (const phase of phases) {
19
+ // Codex limits SessionEnd to 1 to 3 seconds and warns about anything longer.
20
+ const timeout = harness === "codex" && phase === "SessionEnd" ? 3 : 8;
19
21
  if (harness === "copilot") {
20
22
  const event = phase[0]?.toLowerCase() + phase.slice(1);
21
23
  hooks[event] = [
@@ -24,7 +26,7 @@ export function hookConfig(harness, cliPath) {
24
26
  ...(phase === "PreToolUse" || phase.startsWith("PostToolUse") ? { matcher } : {}),
25
27
  exec: process.execPath,
26
28
  args: [cliPath, "hook", harness, phase, managedHookFlag],
27
- timeoutSec: 8,
29
+ timeoutSec: timeout,
28
30
  },
29
31
  ];
30
32
  }
@@ -36,7 +38,7 @@ export function hookConfig(harness, cliPath) {
36
38
  {
37
39
  type: "command",
38
40
  command: `"${process.execPath}" "${cliPath}" hook ${harness} ${phase} ${managedHookFlag}`,
39
- timeout: 8,
41
+ timeout,
40
42
  },
41
43
  ],
42
44
  },
package/dist/policy.d.ts CHANGED
@@ -3,6 +3,8 @@ import type { Category, Classification, Decision, Request } from "./types.js";
3
3
  export declare function hash(value: unknown): string;
4
4
  export declare function words(command: string): string[] | undefined;
5
5
  export declare function result(request: Request, category: Category, reason: string, approve?: boolean): Decision;
6
+ /** Self-reported confidence needed for the AI to approve an ambiguous request. */
7
+ export declare const AMBIGUOUS_CONFIDENCE = 0.95;
6
8
  export declare function evaluate(request: Request, ai?: Classification, profile?: Profile): Decision;
7
9
  export declare function evaluateArgv(request: Request, argv: string[]): Decision;
8
10
  //# sourceMappingURL=policy.d.ts.map
package/dist/policy.js CHANGED
@@ -4,6 +4,7 @@ import { resolve } from "node:path";
4
4
  import { branchEvidence, canonical, git, inside, pathEvidence } from "./evidence.js";
5
5
  import { commandDecision, permissiveRead, permissiveWrite } from "./permissive.js";
6
6
  import { strictProfile } from "./profile.js";
7
+ import { redactText } from "./redact.js";
7
8
  export function hash(value) {
8
9
  return createHash("sha256").update(JSON.stringify(value)).digest("hex");
9
10
  }
@@ -172,8 +173,64 @@ export function result(request, category, reason, approve = false) {
172
173
  cached: false,
173
174
  };
174
175
  }
176
+ /** Self-reported confidence needed for the AI to approve an ambiguous request. */
177
+ export const AMBIGUOUS_CONFIDENCE = 0.95;
178
+ /**
179
+ * Escalations that stay with a human in every profile. Categories `sensitive` (credentials,
180
+ * secrets, protected paths and configuration) and `destructive` (unrecoverable loss) are excluded
181
+ * by category; these are the remaining boundaries: the request itself cannot be trusted, an
182
+ * organization or the profile asked for a human, or a repository rule forbids the write.
183
+ */
184
+ const HUMAN_BOUNDARIES = [
185
+ /^Incomplete request/,
186
+ /^Invalid tool input/,
187
+ /^Execution directory differs/,
188
+ /^Unrecognized patch format/,
189
+ /^Patch has no verified target paths/,
190
+ /^Environment override requires human review/,
191
+ /^Custom execution environment requires human review/,
192
+ /^Matches an organization human-approval pattern/,
193
+ /^Write requires a linked worktree on a feature branch/,
194
+ /^Cloud operation is outside the verified inspection catalog/,
195
+ ];
196
+ /**
197
+ * Under the permissive profile, every escalation that is not a boundary is ambiguous: the
198
+ * deterministic engine cannot decide it, so the session's AI classifier does. Strict mode is
199
+ * unchanged.
200
+ */
201
+ function markAmbiguous(decision, profile) {
202
+ if (profile.mode !== "permissive" ||
203
+ decision.decision !== "human" ||
204
+ decision.needsClassification ||
205
+ decision.category === "sensitive" ||
206
+ decision.category === "destructive" ||
207
+ HUMAN_BOUNDARIES.some((pattern) => pattern.test(decision.reason)))
208
+ return decision;
209
+ return { ...decision, needsClassification: true, tier: "ambiguous" };
210
+ }
211
+ function resolveAmbiguous(decision, ai) {
212
+ const safe = ai.recoverable === true &&
213
+ ai.confidence >= AMBIGUOUS_CONFIDENCE &&
214
+ !["sensitive", "destructive", "unknown"].includes(ai.category);
215
+ const base = { ...decision, needsClassification: false, cache: "session", ai };
216
+ if (safe)
217
+ return {
218
+ ...base,
219
+ decision: "approve",
220
+ category: ai.category,
221
+ classification: "approved",
222
+ reason: "Ambiguous request judged recoverable and non-sensitive by the same-session AI",
223
+ };
224
+ const why = ai.rationale ? ` Rationale: ${redactText(ai.rationale, 200)}` : "";
225
+ return {
226
+ ...base,
227
+ decision: "deny",
228
+ classification: "denied",
229
+ reason: `Not run: the classifier did not approve this request (category ${ai.category}, confidence ${ai.confidence}, recoverable ${ai.recoverable ?? "not stated"}).${why} Use a recoverable, non-sensitive alternative, or explain to the user what is needed.`,
230
+ };
231
+ }
175
232
  export function evaluate(request, ai, profile = strictProfile) {
176
- const decision = evaluateCore(request, profile);
233
+ const decision = markAmbiguous(evaluateCore(request, profile), profile);
177
234
  if (ai) {
178
235
  // Self-reported confidence is not a probability or a substitute for verified effects.
179
236
  if (![
@@ -197,6 +254,8 @@ export function evaluate(request, ai, profile = strictProfile) {
197
254
  reason: "Invalid or mismatched AI classification",
198
255
  };
199
256
  decision.ai = ai;
257
+ if (decision.tier === "ambiguous")
258
+ return resolveAmbiguous(decision, ai);
200
259
  if (decision.category === "cloud-read" &&
201
260
  decision.needsClassification === true &&
202
261
  ai.category === "cloud-read" &&
@@ -0,0 +1,3 @@
1
+ /** Redacts a shell command or other free text, keeping it recognisable and at most `max` long. */
2
+ export declare function redactText(text: string, max?: number): string;
3
+ //# sourceMappingURL=redact.d.ts.map
package/dist/redact.js ADDED
@@ -0,0 +1,140 @@
1
+ // Redaction for text that is written to the audit log: the exact command of an ambiguous request
2
+ // and the classifier's free-text rationale. The audit log is private, but it is also what a human
3
+ // reads in a risk review, so values that look like secrets never reach it.
4
+ //
5
+ // Text is split into words by hand (quotes keep a quoted value together) and every pattern is
6
+ // anchored or linear, so hostile input cannot make the redaction slow.
7
+ /** Names that carry a secret as a substring: `DB_PASSWORD`, `--api-key`, `x-auth-token`. */
8
+ const SECRET_NAME = /(?:token|secret|pass(?:word|wd|phrase)?|pwd|api[-_]?key|apikey|credential|bearer|cookie|private[-_]?key|access[-_]?key|signature)/i;
9
+ /** `auth` only as a whole word, so `--author` is not a secret but `--auth` and `Authorization` are. */
10
+ const AUTH_NAME = /(?:^|[^a-z])auth(?:orization)?(?:[^a-z]|$)/i;
11
+ const TOKEN_PREFIX = /^(?:gh[pousr]_|github_pat_|sk-|xox[baprs]-|AKIA|ASIA|AIza|eyJ|glpat-|npm_|shpat_)/;
12
+ const LONG_OPAQUE = /^[A-Za-z0-9+/=_-]{32,}$/;
13
+ /** Programs whose `-pPASSWORD` attaches the password to the flag. Elsewhere `-p…` is not secret. */
14
+ const ATTACHED_PASSWORD = /^(?:mysql|mysqldump|mysqladmin|mysqlcheck|mariadb|mariadb-dump)$/;
15
+ /** Flags whose next word is a credential, whatever the word looks like. */
16
+ const CREDENTIAL_FLAG = /^(?:-u|-U|--user|--proxy-user|--oauth2-bearer)$/;
17
+ const isSecretName = (name) => SECRET_NAME.test(name) || AUTH_NAME.test(name);
18
+ function words(text) {
19
+ const out = [];
20
+ let current = "";
21
+ let quote = "";
22
+ let inWord = false;
23
+ const flush = () => {
24
+ if (inWord)
25
+ out.push(current);
26
+ current = "";
27
+ inWord = false;
28
+ };
29
+ for (const char of text) {
30
+ if (quote) {
31
+ current += char;
32
+ if (char === quote)
33
+ quote = "";
34
+ }
35
+ else if (char === '"' || char === "'") {
36
+ quote = char;
37
+ current += char;
38
+ inWord = true;
39
+ }
40
+ else if (/\s/.test(char)) {
41
+ flush();
42
+ out.push(char);
43
+ inWord = false;
44
+ }
45
+ else {
46
+ current += char;
47
+ inWord = true;
48
+ }
49
+ }
50
+ flush();
51
+ return out;
52
+ }
53
+ /** `https://user:pass@host/x?token=1` keeps host and path but loses userinfo and query values. */
54
+ function redactUrl(word) {
55
+ const scheme = word.match(/^["']?[a-z][a-z0-9+.-]*:\/\//i)?.[0];
56
+ if (!scheme)
57
+ return word;
58
+ const rest = word.slice(scheme.length);
59
+ const cut = rest.search(/[?#]/);
60
+ const head = cut < 0 ? rest : rest.slice(0, cut);
61
+ const slash = head.indexOf("/");
62
+ const authority = slash < 0 ? head : head.slice(0, slash);
63
+ const path = slash < 0 ? "" : head.slice(slash);
64
+ const at = authority.lastIndexOf("@");
65
+ const host = at < 0 ? authority : `[REDACTED]@${authority.slice(at + 1)}`;
66
+ return `${scheme}${host}${path}${cut >= 0 && rest[cut] === "?" ? "?[REDACTED]" : ""}`;
67
+ }
68
+ /** `-H "X-Api-Key: abc"` and `"Authorization: Bearer abc"`, one quoted word. */
69
+ function redactHeader(word) {
70
+ const colon = word.indexOf(":");
71
+ if (colon <= 0 || word.startsWith("//", colon + 1))
72
+ return undefined;
73
+ const name = word.slice(0, colon).replace(/^["']/, "");
74
+ if (!/^[A-Za-z][\w-]*$/.test(name) || !isSecretName(name))
75
+ return undefined;
76
+ const quote = /["']$/.test(word) ? word.at(-1) : "";
77
+ return `${word.slice(0, colon + 1)} [REDACTED]${quote}`;
78
+ }
79
+ /** Redacts a shell command or other free text, keeping it recognisable and at most `max` long. */
80
+ export function redactText(text, max = 300) {
81
+ let hideNext = false;
82
+ let program = "";
83
+ let atStart = true;
84
+ const out = words(text).map((raw) => {
85
+ if (/^\s$/.test(raw))
86
+ return raw;
87
+ if (/^(?:\||\|\||&&|;|&)$/.test(raw)) {
88
+ atStart = true;
89
+ return raw;
90
+ }
91
+ if (atStart && !/^[A-Za-z_][A-Za-z0-9_]*=/.test(raw)) {
92
+ program =
93
+ raw
94
+ .replace(/^["']|["']$/g, "")
95
+ .split(/[\\/]/)
96
+ .pop() ?? "";
97
+ atStart = false;
98
+ }
99
+ if (hideNext) {
100
+ // `Authorization: Bearer abc` carries the secret in the word after the scheme.
101
+ hideNext = /^["']?(?:bearer|basic|token)["']?$/i.test(raw);
102
+ return hideNext ? raw : "[REDACTED]";
103
+ }
104
+ // URL credentials first: a `?token=` in the query must not hide userinfo before it.
105
+ const piece = redactUrl(raw);
106
+ if (CREDENTIAL_FLAG.test(piece)) {
107
+ hideNext = true;
108
+ return piece;
109
+ }
110
+ // `Authorization:` alone: the value, and a scheme before it, are the words that follow.
111
+ if (/^["']?[A-Za-z][\w-]*:["']?$/.test(piece) && isSecretName(piece.replace(/[:"']/g, ""))) {
112
+ hideNext = true;
113
+ return piece;
114
+ }
115
+ const header = redactHeader(piece);
116
+ if (header)
117
+ return header;
118
+ const eq = piece.indexOf("=");
119
+ const name = eq >= 0 ? piece.slice(0, eq) : piece;
120
+ // `mysql -pSECRET`: a short option with its value attached.
121
+ if (/^-p./.test(piece) && eq < 0 && ATTACHED_PASSWORD.test(program))
122
+ return "-p[REDACTED]";
123
+ if (/^-[A-Za-z]./.test(piece) && eq < 0 && isSecretName(piece))
124
+ return `${piece.slice(0, 2)}[REDACTED]`;
125
+ if (isSecretName(name)) {
126
+ if (eq >= 0)
127
+ return `${name}=[REDACTED]`;
128
+ // A bare flag takes its value from the next word; a header name waits for its value.
129
+ if (/^-{1,2}[A-Za-z]/.test(piece) || /:["']?$/.test(piece))
130
+ hideNext = true;
131
+ return piece;
132
+ }
133
+ // A quoted token is still a token: judge the word without its surrounding quote.
134
+ const bare = piece.replace(/^["']|["']$/g, "");
135
+ return TOKEN_PREFIX.test(bare) || LONG_OPAQUE.test(bare) ? "[REDACTED]" : piece;
136
+ });
137
+ const joined = out.join("");
138
+ return joined.length > max ? `${joined.slice(0, max)}…` : joined;
139
+ }
140
+ //# sourceMappingURL=redact.js.map
package/dist/replay.d.ts CHANGED
@@ -3,6 +3,8 @@ export interface ReplayReport {
3
3
  commands: number;
4
4
  approved: number;
5
5
  strictApproved: number;
6
+ /** Escalations the AI classifier handles instead of a person (permissive profile only). */
7
+ classified: number;
6
8
  reasons: Array<{
7
9
  reason: string;
8
10
  count: number;
package/dist/replay.js CHANGED
@@ -27,7 +27,13 @@ function transcriptFiles(paths, limit) {
27
27
  export function replayTranscripts(paths, profile, files = 50) {
28
28
  const seen = new Set();
29
29
  const reasons = new Map();
30
- const report = { commands: 0, approved: 0, strictApproved: 0, reasons: [] };
30
+ const report = {
31
+ commands: 0,
32
+ approved: 0,
33
+ strictApproved: 0,
34
+ classified: 0,
35
+ reasons: [],
36
+ };
31
37
  for (const file of transcriptFiles(paths, files))
32
38
  for (const line of readFileSync(file, "utf8").split("\n")) {
33
39
  if (!line.includes('"tool_use"'))
@@ -66,6 +72,8 @@ export function replayTranscripts(paths, profile, files = 50) {
66
72
  report.approved++;
67
73
  continue;
68
74
  }
75
+ if (decision.tier === "ambiguous")
76
+ report.classified++;
69
77
  const bucket = reasons.get(decision.reason) ?? { count: 0, examples: [] };
70
78
  bucket.count++;
71
79
  if (bucket.examples.length < 3)
@@ -0,0 +1,55 @@
1
+ /** One classification the AI gave for a request. A request can have several (a replay). */
2
+ export interface Attempt {
3
+ time: string;
4
+ category?: string;
5
+ confidence?: number;
6
+ recoverable?: boolean;
7
+ rationale?: string;
8
+ /** `invalid` when the answer was rejected before it could decide anything. */
9
+ outcome: "approved" | "denied" | "invalid";
10
+ }
11
+ /** One ambiguous request, followed through classification to its outcome. */
12
+ export interface ClassifiedRequest {
13
+ session: string;
14
+ requestHash: string;
15
+ time: string;
16
+ command?: string;
17
+ reason?: string;
18
+ /** Every classification, in order. Later ones never overwrite earlier ones. */
19
+ attempts: Attempt[];
20
+ /** What happened to the request: `approved`/`denied` by the AI, `human` after a fallback. */
21
+ outcome: "approved" | "denied" | "fallback" | "unresolved";
22
+ fallbackReason?: string;
23
+ execution?: {
24
+ status?: string;
25
+ exitCode?: number;
26
+ };
27
+ humanResponse?: string;
28
+ }
29
+ export interface ClassificationReview {
30
+ sessions: number;
31
+ requested: number;
32
+ approved: number;
33
+ denied: number;
34
+ fallback: number;
35
+ fallbackReasons: Record<string, number>;
36
+ unresolved: number;
37
+ /** Share of requested ambiguous requests that reached a person. */
38
+ fallbackRate: number;
39
+ histogram: Array<{
40
+ range: string;
41
+ count: number;
42
+ }>;
43
+ aiApproved: ClassifiedRequest[];
44
+ aiDenied: ClassifiedRequest[];
45
+ approvedButFailed: ClassifiedRequest[];
46
+ humanDenied: ClassifiedRequest[];
47
+ /** Requests classified more than once. A replay is the signal a risk review looks for first. */
48
+ reclassified: ClassifiedRequest[];
49
+ /** Requests the AI denied and later approved. */
50
+ deniedThenApproved: ClassifiedRequest[];
51
+ }
52
+ export declare function sessionDirectories(extra?: string[]): string[];
53
+ export declare function reviewClassifications(directories: string[], since?: string): Promise<ClassificationReview>;
54
+ export declare function formatReview(review: ClassificationReview): string;
55
+ //# sourceMappingURL=review.d.ts.map
package/dist/review.js ADDED
@@ -0,0 +1,231 @@
1
+ import { existsSync, lstatSync, readdirSync } from "node:fs";
2
+ import { tmpdir } from "node:os";
3
+ import { join } from "node:path";
4
+ import { AuditLog } from "@agent-corpus/audit-log";
5
+ const BUCKETS = [
6
+ ["0.00-0.50", 0, 0.5],
7
+ ["0.50-0.80", 0.5, 0.8],
8
+ ["0.80-0.90", 0.8, 0.9],
9
+ ["0.90-0.95", 0.9, 0.95],
10
+ ["0.95-0.99", 0.95, 0.99],
11
+ ["0.99-1.00", 0.99, 1.0001],
12
+ ];
13
+ export function sessionDirectories(extra = []) {
14
+ const root = tmpdir();
15
+ const found = existsSync(root)
16
+ ? readdirSync(root)
17
+ .filter((name) => /^acp-[A-Za-z0-9]+-[a-f0-9]{24}$/.test(name))
18
+ .map((name) => join(root, name))
19
+ : [];
20
+ return [...found, ...extra].filter((path) => {
21
+ try {
22
+ return lstatSync(path).isDirectory();
23
+ }
24
+ catch {
25
+ return false;
26
+ }
27
+ });
28
+ }
29
+ const text = (value) => typeof value === "string" ? value : undefined;
30
+ const num = (value) => typeof value === "number" && Number.isFinite(value) ? value : undefined;
31
+ export async function reviewClassifications(directories, since) {
32
+ const requests = [];
33
+ let sessions = 0;
34
+ for (const directory of directories) {
35
+ const records = [];
36
+ try {
37
+ let after;
38
+ for (;;) {
39
+ const page = await new AuditLog(directory).query({
40
+ ...(since ? { since } : {}),
41
+ ...(after ? { after } : {}),
42
+ limit: 10000,
43
+ });
44
+ records.push(...page.records);
45
+ if (!page.nextCursor)
46
+ break;
47
+ after = page.nextCursor;
48
+ }
49
+ }
50
+ catch {
51
+ continue; // an unreadable or foreign directory is skipped, never trusted
52
+ }
53
+ if (!records.length)
54
+ continue;
55
+ sessions++;
56
+ const byRequest = new Map();
57
+ const entry = (record) => {
58
+ if (!record.requestId)
59
+ return undefined;
60
+ let value = byRequest.get(record.requestId);
61
+ if (!value) {
62
+ value = {
63
+ session: record.sessionId,
64
+ requestHash: record.requestId,
65
+ time: record.timestamp,
66
+ attempts: [],
67
+ outcome: "unresolved",
68
+ };
69
+ byRequest.set(record.requestId, value);
70
+ }
71
+ return value;
72
+ };
73
+ for (const record of records) {
74
+ const a = record.attributes;
75
+ // Only requests that entered the classification flow are reviewed.
76
+ if (record.eventName !== "classification-requested" && !byRequest.has(record.requestId ?? ""))
77
+ continue;
78
+ if (record.eventName === "classification-requested") {
79
+ const value = entry(record);
80
+ if (!value)
81
+ continue;
82
+ value.time = record.timestamp;
83
+ const command = text(a.redactedCommand);
84
+ if (command)
85
+ value.command = command;
86
+ const reason = text(a.reason);
87
+ if (reason)
88
+ value.reason = reason;
89
+ if (!requests.includes(value))
90
+ requests.push(value);
91
+ }
92
+ else if (record.eventName === "classification") {
93
+ const value = entry(record);
94
+ if (!value)
95
+ continue;
96
+ const attempt = { time: record.timestamp, outcome: "invalid" };
97
+ const category = text(a.category);
98
+ if (category)
99
+ attempt.category = category;
100
+ const confidence = num(a.confidence);
101
+ if (confidence !== undefined)
102
+ attempt.confidence = confidence;
103
+ if (typeof a.recoverable === "boolean")
104
+ attempt.recoverable = a.recoverable;
105
+ const rationale = text(a.rationale);
106
+ if (rationale)
107
+ attempt.rationale = rationale;
108
+ value.attempts.push(attempt);
109
+ }
110
+ else if (record.eventName === "decision") {
111
+ const value = entry(record);
112
+ if (!value)
113
+ continue;
114
+ const latest = value.attempts.at(-1);
115
+ if (a.classification === "approved" || a.classification === "denied") {
116
+ // The decision that follows a classification settles that attempt; a retry's decision
117
+ // repeats the same outcome and changes nothing.
118
+ if (latest?.outcome === "invalid")
119
+ latest.outcome = a.classification;
120
+ value.outcome = a.classification;
121
+ }
122
+ else if (a.classification === "fallback")
123
+ value.outcome = "fallback";
124
+ }
125
+ else if (record.eventName === "classification-fallback") {
126
+ const value = entry(record);
127
+ if (!value)
128
+ continue;
129
+ value.outcome = "fallback";
130
+ const reason = text(a.fallbackReason);
131
+ if (reason)
132
+ value.fallbackReason = reason;
133
+ }
134
+ else if (record.eventName === "execution-result") {
135
+ const value = entry(record);
136
+ if (!value)
137
+ continue;
138
+ value.execution = {
139
+ ...(text(a.status) ? { status: text(a.status) } : {}),
140
+ ...(num(a.exitCode) !== undefined ? { exitCode: num(a.exitCode) } : {}),
141
+ };
142
+ }
143
+ else if (record.eventName === "human-response") {
144
+ const value = entry(record);
145
+ if (value)
146
+ value.humanResponse = text(a.response) ?? "";
147
+ }
148
+ }
149
+ }
150
+ const approved = requests.filter((r) => r.outcome === "approved");
151
+ const denied = requests.filter((r) => r.outcome === "denied");
152
+ const fallback = requests.filter((r) => r.outcome === "fallback");
153
+ const fallbackReasons = {};
154
+ for (const r of fallback) {
155
+ const key = r.fallbackReason ?? "unavailable";
156
+ fallbackReasons[key] = (fallbackReasons[key] ?? 0) + 1;
157
+ }
158
+ return {
159
+ sessions,
160
+ requested: requests.length,
161
+ approved: approved.length,
162
+ denied: denied.length,
163
+ fallback: fallback.length,
164
+ fallbackReasons,
165
+ unresolved: requests.length - approved.length - denied.length - fallback.length,
166
+ fallbackRate: requests.length ? fallback.length / requests.length : 0,
167
+ histogram: BUCKETS.map(([range, low, high]) => ({
168
+ range,
169
+ count: requests
170
+ .flatMap((r) => r.attempts)
171
+ .filter((t) => t.confidence !== undefined && t.confidence >= low && t.confidence < high)
172
+ .length,
173
+ })),
174
+ aiApproved: approved,
175
+ aiDenied: denied,
176
+ approvedButFailed: approved.filter((r) => r.execution &&
177
+ (r.execution.status === "failed" ||
178
+ (r.execution.exitCode !== undefined && r.execution.exitCode !== 0))),
179
+ humanDenied: requests.filter((r) => r.humanResponse === "denied"),
180
+ reclassified: requests.filter((r) => r.attempts.length > 1),
181
+ deniedThenApproved: requests.filter((r) => {
182
+ const first = r.attempts.findIndex((t) => t.outcome === "denied");
183
+ return first >= 0 && r.attempts.slice(first + 1).some((t) => t.outcome === "approved");
184
+ }),
185
+ };
186
+ }
187
+ function row(r) {
188
+ const outcome = r.execution
189
+ ? ` -> ${r.execution.status ?? "ran"}${r.execution.exitCode !== undefined ? ` (exit ${r.execution.exitCode})` : ""}`
190
+ : "";
191
+ const attempts = r.attempts
192
+ .map((t, i) => ` ${r.attempts.length > 1 ? `attempt ${i + 1}: ` : ""}${t.outcome} ${t.category ?? "?"} ${t.confidence ?? "?"} recoverable=${t.recoverable ?? "?"}${t.rationale ? ` why: ${t.rationale}` : ""}`)
193
+ .join("\n");
194
+ return ` ${r.time} ${r.command ?? r.requestHash}${outcome}${attempts ? `\n${attempts}` : ""}`;
195
+ }
196
+ export function formatReview(review) {
197
+ const pct = (n) => `${((100 * n) / Math.max(review.requested, 1)).toFixed(1)}%`;
198
+ const lines = [
199
+ `Sessions scanned: ${review.sessions}`,
200
+ `Ambiguous requests sent to classification: ${review.requested}`,
201
+ ` AI approved: ${review.approved} (${pct(review.approved)})`,
202
+ ` AI denied: ${review.denied} (${pct(review.denied)})`,
203
+ ` Fell back to human: ${review.fallback} (${pct(review.fallback)})${Object.entries(review.fallbackReasons)
204
+ .map(([k, v]) => ` ${k}=${v}`)
205
+ .join("")}`,
206
+ ` Never resolved: ${review.unresolved} (${pct(review.unresolved)}; the model abandoned the command)`,
207
+ "",
208
+ "Confidence histogram (classifier self-report):",
209
+ ...review.histogram.map((b) => ` ${b.range} ${b.count}`),
210
+ "",
211
+ `Requests classified more than once (${review.reclassified.length}):`,
212
+ ...review.reclassified.map(row),
213
+ "",
214
+ `AI-denied, then approved on a later classification (${review.deniedThenApproved.length}):`,
215
+ ...review.deniedThenApproved.map(row),
216
+ "",
217
+ `AI-approved commands that failed (${review.approvedButFailed.length}):`,
218
+ ...review.approvedButFailed.map(row),
219
+ "",
220
+ `Commands a person later denied (${review.humanDenied.length}):`,
221
+ ...review.humanDenied.map(row),
222
+ "",
223
+ `All AI-approved commands (${review.aiApproved.length}):`,
224
+ ...review.aiApproved.map(row),
225
+ "",
226
+ `AI-denied commands (${review.aiDenied.length}):`,
227
+ ...review.aiDenied.map(row),
228
+ ];
229
+ return lines.join("\n");
230
+ }
231
+ //# sourceMappingURL=review.js.map
package/dist/service.d.ts CHANGED
@@ -5,6 +5,7 @@ interface Message {
5
5
  request?: Request;
6
6
  classification?: Classification;
7
7
  report?: OutcomeReport;
8
+ phase?: string;
8
9
  }
9
10
  export declare function serve(harness: string, sessionId: string): Promise<void>;
10
11
  export declare function send(harness: string, sessionId: string, message: Omit<Message, "token">): Promise<unknown>;
package/dist/service.js CHANGED
@@ -64,7 +64,7 @@ export async function serve(harness, sessionId) {
64
64
  server.close();
65
65
  }
66
66
  else if (message.action === "review" && message.request)
67
- result = session.review(message.request);
67
+ result = session.review(message.request, undefined, message.phase);
68
68
  else if (message.action === "classify" && message.classification)
69
69
  result = session.classify(message.classification);
70
70
  else if (message.action === "record" && message.report) {
package/dist/session.d.ts CHANGED
@@ -1,5 +1,7 @@
1
1
  import { type Profile } from "./profile.js";
2
2
  import type { Classification, Decision, OutcomeReport, Request } from "./types.js";
3
+ /** AI approvals per session; beyond this, ambiguous requests go to a human. */
4
+ export declare const MAX_AI_APPROVALS = 50;
3
5
  export declare class PolicySession {
4
6
  readonly directory: string;
5
7
  readonly harness: string;
@@ -7,8 +9,18 @@ export declare class PolicySession {
7
9
  readonly profile: Profile;
8
10
  private cache;
9
11
  private pending;
12
+ private states;
13
+ /** Requests the AI approved, counted once each however often they are retried. */
14
+ private approvedByAi;
10
15
  constructor(directory: string, harness: string, sessionId: string, profile?: Profile);
11
- review(request: Request, ai?: Classification): Decision;
16
+ private state;
17
+ private remember;
18
+ /**
19
+ * `phase` is the hook event. Only `PreToolUse` runs the classification flow: the first time an
20
+ * ambiguous request arrives it is blocked with an instruction to classify and retry; the next
21
+ * time, with no usable classification, a human is asked and the reason is recorded.
22
+ */
23
+ review(request: Request, ai?: Classification, phase?: string): Decision;
12
24
  record(report: OutcomeReport): void;
13
25
  classify(classification: Classification): Decision;
14
26
  }
package/dist/session.js CHANGED
@@ -1,6 +1,11 @@
1
1
  import { audit, requestSummary } from "./audit.js";
2
2
  import { evaluate, hash } from "./policy.js";
3
3
  import { strictProfile } from "./profile.js";
4
+ import { redactText } from "./redact.js";
5
+ /** How long a classification outcome for one exact request is remembered. */
6
+ const CLASSIFICATION_TTL_MS = 5 * 60_000;
7
+ /** AI approvals per session; beyond this, ambiguous requests go to a human. */
8
+ export const MAX_AI_APPROVALS = 50;
4
9
  export class PolicySession {
5
10
  directory;
6
11
  harness;
@@ -8,13 +13,42 @@ export class PolicySession {
8
13
  profile;
9
14
  cache = new Map();
10
15
  pending = new Map();
16
+ states = new Map();
17
+ /** Requests the AI approved, counted once each however often they are retried. */
18
+ approvedByAi = new Set();
11
19
  constructor(directory, harness, sessionId, profile = strictProfile) {
12
20
  this.directory = directory;
13
21
  this.harness = harness;
14
22
  this.sessionId = sessionId;
15
23
  this.profile = profile;
16
24
  }
17
- review(request, ai) {
25
+ state(requestHash) {
26
+ const state = this.states.get(requestHash);
27
+ if (state && state.expires <= Date.now()) {
28
+ this.states.delete(requestHash);
29
+ return undefined;
30
+ }
31
+ return state;
32
+ }
33
+ remember(requestHash, change) {
34
+ if (this.states.size >= 512)
35
+ this.states.clear();
36
+ this.states.set(requestHash, {
37
+ requested: false,
38
+ invalid: false,
39
+ fallback: false,
40
+ limited: false,
41
+ ...this.states.get(requestHash),
42
+ ...change,
43
+ expires: Date.now() + CLASSIFICATION_TTL_MS,
44
+ });
45
+ }
46
+ /**
47
+ * `phase` is the hook event. Only `PreToolUse` runs the classification flow: the first time an
48
+ * ambiguous request arrives it is blocked with an instruction to classify and retry; the next
49
+ * time, with no usable classification, a human is asked and the reason is recorded.
50
+ */
51
+ review(request, ai, phase) {
18
52
  if (request.sessionId !== this.sessionId || request.harness !== this.harness)
19
53
  throw new Error("Session mismatch");
20
54
  audit(this.directory, { event: "request", ...requestSummary(request) });
@@ -31,6 +65,8 @@ export class PolicySession {
31
65
  ].includes(ai.category) ||
32
66
  typeof ai.confidence !== "number" ||
33
67
  !Number.isFinite(ai.confidence) ||
68
+ (ai.recoverable !== undefined && typeof ai.recoverable !== "boolean") ||
69
+ (ai.rationale !== undefined && typeof ai.rationale !== "string") ||
34
70
  !/^[a-f0-9]{64}$/.test(ai.requestHash))) {
35
71
  audit(this.directory, {
36
72
  event: "decision",
@@ -44,14 +80,72 @@ export class PolicySession {
44
80
  const key = hash([hash(request), "policy-v1", this.profile.digest ?? this.profile.mode]);
45
81
  const entry = this.cache.get(key);
46
82
  const cachedAi = entry && entry.expires > Date.now() ? entry.ai : undefined;
83
+ const requestHash = hash(request);
84
+ const known = ai ? undefined : this.state(requestHash);
47
85
  if (ai)
48
86
  audit(this.directory, {
49
87
  event: "classification",
50
- requestHash: hash(request),
88
+ requestHash,
51
89
  category: ai.category,
52
90
  confidence: ai.confidence,
91
+ ...(ai.recoverable === undefined ? {} : { recoverable: ai.recoverable }),
92
+ ...(ai.rationale ? { rationale: redactText(ai.rationale, 200) } : {}),
53
93
  });
54
- const decision = evaluate(request, ai ?? cachedAi, this.profile);
94
+ let decision = evaluate(request, ai ?? cachedAi ?? known?.ai, this.profile);
95
+ if (decision.tier === "ambiguous") {
96
+ if (ai && !decision.classification)
97
+ this.remember(requestHash, { invalid: true });
98
+ else if (ai) {
99
+ // The cap is checked when the approval is first given and stored with the outcome, so a
100
+ // retry cannot turn a limited request into an approval.
101
+ if (decision.classification === "approved" &&
102
+ !this.approvedByAi.has(requestHash) &&
103
+ this.approvedByAi.size >= MAX_AI_APPROVALS) {
104
+ decision = {
105
+ ...decision,
106
+ decision: "human",
107
+ classification: "fallback",
108
+ fallbackReason: "limit",
109
+ reason: "Session limit on AI approvals reached; a human decides",
110
+ };
111
+ this.remember(requestHash, { limited: true });
112
+ }
113
+ else {
114
+ if (decision.classification === "approved")
115
+ this.approvedByAi.add(requestHash);
116
+ this.remember(requestHash, { ai });
117
+ }
118
+ }
119
+ if (decision.needsClassification && !ai && phase === "PreToolUse") {
120
+ if (!known?.requested) {
121
+ this.remember(requestHash, { requested: true });
122
+ audit(this.directory, {
123
+ event: "classification-requested",
124
+ requestHash,
125
+ tier: "ambiguous",
126
+ reason: decision.reason,
127
+ redactedCommand: redactText(String(request.input.command ?? request.input.cmd ?? request.tool), 300),
128
+ });
129
+ decision = { ...decision, decision: "deny", classification: "requested" };
130
+ }
131
+ else {
132
+ const fallbackReason = known.limited
133
+ ? "limit"
134
+ : known.invalid
135
+ ? "invalid-result"
136
+ : "unavailable";
137
+ if (!known.fallback) {
138
+ this.remember(requestHash, { fallback: true });
139
+ audit(this.directory, {
140
+ event: "classification-fallback",
141
+ requestHash,
142
+ fallbackReason,
143
+ });
144
+ }
145
+ decision = { ...decision, classification: "fallback", fallbackReason };
146
+ }
147
+ }
148
+ }
55
149
  if (decision.decision === "human")
56
150
  this.cache.delete(key);
57
151
  // Evaluation always precedes lookup. Filesystem evidence is never cached.
@@ -66,8 +160,10 @@ export class PolicySession {
66
160
  });
67
161
  }
68
162
  }
69
- if (decision.decision === "human" &&
70
- ["unknown", "cloud-read"].includes(decision.category) &&
163
+ // A denied request is final for its remembered state: only a human decision or a request
164
+ // still waiting for its first classification can be classified.
165
+ if ((decision.decision === "human" || decision.classification === "requested") &&
166
+ (["unknown", "cloud-read"].includes(decision.category) || decision.needsClassification) &&
71
167
  !ai) {
72
168
  if (this.pending.size >= 64)
73
169
  this.pending.delete(this.pending.keys().next().value ?? "");
@@ -103,7 +199,7 @@ export class PolicySession {
103
199
  });
104
200
  }
105
201
  else if (report.event === "execution-result") {
106
- if (!["completed", "succeeded", "failed", "cancelled"].includes(report.status ?? ""))
202
+ if (!["completed", "succeeded", "failed", "cancelled", "unknown"].includes(report.status ?? ""))
107
203
  throw new Error("Invalid execution status");
108
204
  if (report.exitCode !== undefined &&
109
205
  (!Number.isSafeInteger(report.exitCode) || Math.abs(report.exitCode) > 2147483647))
@@ -127,8 +223,19 @@ export class PolicySession {
127
223
  const request = this.pending.get(classification.requestHash);
128
224
  if (!request)
129
225
  throw new Error("No pending request matches the classification");
226
+ // One classification per request: a verdict, deny or approve, cannot be replaced.
227
+ if (this.state(classification.requestHash)?.ai)
228
+ throw new Error("The request is already classified");
130
229
  this.pending.delete(classification.requestHash);
131
- return this.review(request, classification);
230
+ try {
231
+ return this.review(request, classification);
232
+ }
233
+ catch (error) {
234
+ // A malformed result is not an approval; the retry will fall back to a human, with the
235
+ // reason recorded for the review.
236
+ this.remember(classification.requestHash, { invalid: true });
237
+ throw error;
238
+ }
132
239
  }
133
240
  }
134
241
  //# sourceMappingURL=session.js.map
package/dist/setup.js CHANGED
@@ -127,7 +127,9 @@ export function setupHook(harness, cliPath, options = {}) {
127
127
  "Start a new harness session to load the hooks.",
128
128
  ...(harness === "codex"
129
129
  ? [
130
- "Open /hooks and explicitly review and trust the installed hooks. Setup does not grant hook trust.",
130
+ "Open /hooks and explicitly review and trust the installed hooks. Setup does not grant hook trust, and Codex skips hooks it has not trusted.",
131
+ "Codex trusts the command text, not the code behind it: an upgrade at the same CLI path keeps trust, so review a new version before upgrading in place. A changed command (a new install path or Node binary) is skipped until you trust it again.",
132
+ "Codex does not support ask in PreToolUse, so a decision that needs a human is returned as a deny and the command does not run; approving it in chat does not unblock it.",
131
133
  ]
132
134
  : []),
133
135
  "Native sandbox controls and existing permission rules remain in effect.",
package/dist/types.d.ts CHANGED
@@ -10,22 +10,34 @@ export interface Classification {
10
10
  category: Category;
11
11
  confidence: number;
12
12
  requestHash: string;
13
+ /** Whether the action can be undone (merged work, tracked files, anything a backup restores). */
14
+ recoverable?: boolean;
15
+ /** Short free-text reason from the classifier; capped and redacted before it is logged. */
16
+ rationale?: string;
13
17
  }
18
+ /** Where an ambiguous request stands in the AI classification flow. */
19
+ export type ClassificationStage = "requested" | "approved" | "denied" | "fallback";
14
20
  export interface Decision {
15
- decision: "approve" | "human";
21
+ /** `deny` is a decision the model sees and works around; `human` asks the person. */
22
+ decision: "approve" | "human" | "deny";
16
23
  category: Category;
17
24
  reason: string;
18
25
  requestHash: string;
19
26
  cache: "none" | "session";
20
27
  cached: boolean;
21
28
  needsClassification?: boolean;
29
+ /** `ambiguous`: the deterministic engine could not decide and nothing in code forbids it. */
30
+ tier?: "ambiguous";
31
+ classification?: ClassificationStage;
32
+ /** Why a human was asked after classification could not decide. */
33
+ fallbackReason?: "unavailable" | "invalid-result" | "limit";
22
34
  ai?: Classification;
23
35
  }
24
36
  export interface OutcomeReport {
25
37
  requestHash: string;
26
38
  event: "human-response" | "execution-result";
27
39
  response?: "approved" | "denied";
28
- status?: "completed" | "succeeded" | "failed" | "cancelled";
40
+ status?: "completed" | "succeeded" | "failed" | "cancelled" | "unknown";
29
41
  exitCode?: number;
30
42
  durationMs?: number;
31
43
  }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@agent-corpus/policy-manager",
3
- "version": "0.4.0",
3
+ "version": "0.5.0",
4
4
  "description": "Deterministic, session-scoped permission decisions with audit logs and bounded AI classification.",
5
5
  "keywords": [
6
6
  "coding-agent",