pi-pr-review 1.1.2 → 1.3.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -63,15 +63,19 @@ The `/pr-review-config` command maps three labels to models:
63
63
  ```
64
64
  /pr-review-config # open the settings menu (like /settings & /nervous:config)
65
65
  /pr-review-config show # print the current mapping
66
- /pr-review-config light=<spec> medium=<spec> heavy=<spec> # set non-interactively
66
+ /pr-review-config light=<spec> medium=<spec> heavy=<spec> # set primary tier models
67
+ /pr-review-config heavy_fallbacks=<spec>,<spec> # retry chain for quota/rate-limit failures
68
+ /pr-review-config light_tool_policy=none # tier default when a pass omits tool_policy
67
69
  /pr-review-config medium=unset # clear a tier (back to pi default)
68
- /pr-review-config tools=read,bash,grep,find,ls # tools granted to each subagent
70
+ /pr-review-config heavy_fallbacks=unset # clear a fallback chain
71
+ /pr-review-config light_tool_policy=unset # restore legacy configured-tool behavior
72
+ /pr-review-config tools=read,bash,grep,find,ls # allowlist used by configured policy
69
73
  ```
70
74
 
71
75
  Running `/pr-review-config` with no arguments in the TUI opens an interactive settings menu that mirrors pi's `/settings` and the NERVous `/nervous:config`:
72
76
 
73
- - One row per tier (`light` / `medium` / `heavy` model) plus a `subagent tools` row.
74
- - Press Enter on a tier to pick a model from a searchable list (or `(unset — pi default)`); Enter/Space on `subagent tools` cycles presets.
77
+ - One primary-model, fallback-model, and tool-policy row per tier (`light` / `medium` / `heavy`) plus a configured-tool allowlist row.
78
+ - Press Enter on a primary or fallback row to pick a model from a searchable list (or unset it); Enter/Space cycles tool policies and allowlist presets. The menu sets one fallback model at a time; use the `key=value` form for longer fallback chains.
75
79
  - Selections apply and persist **immediately**; Esc closes the menu.
76
80
  - Type to search, and tab-completion is available for the `key=value` form.
77
81
 
@@ -91,15 +95,27 @@ Example `pr-review.json`:
91
95
  "medium": "<balanced-model-spec>",
92
96
  "heavy": "<strong-model-spec:high>"
93
97
  },
98
+ "fallbacks": {
99
+ "light": ["<backup-fast-model>"],
100
+ "medium": ["<backup-balanced-model>"],
101
+ "heavy": ["<backup-strong-model:high>", "<balanced-model-spec>"]
102
+ },
103
+ "toolPolicies": {
104
+ "light": "none",
105
+ "medium": "configured",
106
+ "heavy": "configured"
107
+ },
94
108
  "tools": ["read", "bash", "grep", "find", "ls"]
95
109
  }
96
110
  ```
97
111
 
98
- Each tier runs in an **isolated `pi` subprocess** on its configured model. The `review_subagents` batch tool runs independent passes concurrently (default `max_parallel: 4`, capped at 6) and returns ordered per-pass results; the older single-pass `review_subagent` tool remains available as a compatibility fallback. If a tier is unset, that subagent falls back to the nearest configured tier, then to your pi default model.
112
+ Each tier runs in an **isolated `pi` subprocess** on its configured model. The `review_subagents` batch tool runs independent passes concurrently (default `max_parallel: 4`, capped at 6) and returns ordered per-pass results; the older single-pass `review_subagent` tool remains available as a compatibility fallback. If a tier model fails with a retryable quota/rate-limit/capacity error, the subprocess retries that tier's configured `fallbacks` in order. Non-quota failures do not blindly cycle through fallbacks. If a tier is unset, that subagent falls back to the nearest configured tier, then to your pi default model.
113
+
114
+ Tool policy is additive and backward compatible: `none` emits Pi's explicit `--no-tools`; `configured` uses the existing `tools` allowlist. A tool call's optional `tool_policy` overrides `toolPolicies[tier]`, which in turn falls back to legacy `configured` behavior. The shipped `/pr-review` prompt explicitly uses `none` only for overview because its complete evidence is supplied in context. Conventions/maintainability and both heavy specialist passes use `configured` repository-context tools so they can inspect surrounding files when needed. Fallback model attempts keep the original pass policy.
99
115
 
100
- ### 2. The orchestrator / fallback model
116
+ ### 2. The orchestrator / inline-fallback model
101
117
 
102
- The orchestrator (which fetches the PR, merges findings, and emits the JSON) and the inline-fallback path both run on your pi session model:
118
+ The orchestrator (which fetches the PR, merges findings, and emits the JSON) and the inline fallback path both run on your pi session model:
103
119
 
104
120
  - **Per run:** `pi --model <model-id> "/pr-review 123"`
105
121
  - **Persistent default (user):** `~/.pi/agent/settings.json` → `{ "defaultModel": "<model-id>", "defaultThinkingLevel": "high" }`
@@ -174,15 +190,19 @@ Severity tags: `[P0]` blocking/drop-everything · `[P1]` blocking/urgent · `[P2
174
190
  pi-pr-review/
175
191
  ├─ package.json # pi manifest: prompts + extensions
176
192
  ├─ prompts/pr-review.md # the /pr-review orchestrator prompt
193
+ ├─ lib/pr-review-policy.ts # pure tool-policy resolution/argv helpers
177
194
  ├─ extensions/pr-review-subagent.ts # review_subagents/review_subagent tools + /pr-review-config command
178
- └─ extensions/review-table.ts # renders the final JSON as a table (TUI only)
195
+ ├─ extensions/review-table.ts # renders the final JSON as a table (TUI only)
196
+ └─ tests/pr-review-policy.test.ts # focused policy compatibility tests
179
197
  ```
180
198
 
181
- ## Security & cost notes
199
+ ## Speed, security & cost notes
182
200
 
183
- - The `review_subagents` batch tool and `review_subagent` fallback spawn isolated `pi` subprocesses (`--mode json -p --no-session`) on your configured tier models. Subagents are read-only reviewers (`read,bash,grep,find,ls` by default) and never post comments or edit files.
201
+ - The four independent review lenses remain intact. Only overview runs context-only with `--no-tools`; medium and both heavy specialists retain configured tools for surrounding-file validation. All subprocesses use `--no-context-files` because the orchestrator supplies the base review context explicitly, and convention excerpts are sent only to the medium pass instead of every model.
202
+ - Tool results include effective `toolPolicy` and `elapsedMs` telemetry (plus per-attempt timing) so repeated representative runs can be compared at p50/p95 without guessing. Restore tools for a custom pass by sending `tool_policy: "configured"`; callers that omit policy retain legacy behavior unless `toolPolicies` config says otherwise.
203
+ - The `review_subagents` batch tool and `review_subagent` fallback spawn isolated `pi` subprocesses (`--mode json -p --no-session`) on your configured tier models. Reviewer prompts prohibit modifications, but a configured allowlist containing `bash` is not technically read-only; use a narrower allowlist if stronger enforcement is required.
184
204
  - Project-local `pr-review.json` is only read when the project is trusted.
185
- - Tiered review calls multiple models per PR, now concurrently for independent passes. Point `light` at a cheap model for overview/risk scan; reserve `heavy` for the deep passes.
205
+ - Tiered review calls multiple models per PR, concurrently for independent passes. Point `light` at a cheap model for overview/risk scan; reserve `heavy` for deep passes, and configure per-tier `fallbacks` only for acceptable backup models because retries can increase cost.
186
206
 
187
207
  ## Design notes
188
208
 
@@ -4,7 +4,8 @@
4
4
  * Adds configurable, tiered review subagents to the /pr-review workflow.
5
5
  *
6
6
  * - Config surface: `pr-review.json` (user: ~/.pi/agent, project: <repo>/.pi) maps
7
- * the labels `light` / `medium` / `heavy` to whatever models you choose.
7
+ * the labels `light` / `medium` / `heavy` to whatever models you choose,
8
+ * plus optional per-tier fallback chains for quota/capacity failures.
8
9
  * No model names are hardcoded here — you configure them.
9
10
  * - Tool: `review_subagent` spawns one isolated `pi` subprocess on the model
10
11
  * bound to the requested tier and returns its review report.
@@ -45,6 +46,13 @@ import {
45
46
  Text,
46
47
  } from "@earendil-works/pi-tui";
47
48
  import { Type } from "typebox";
49
+ import {
50
+ appendToolPolicyArgs,
51
+ buildReviewBaseArgs,
52
+ normalizeToolPolicy,
53
+ resolveToolPolicy,
54
+ type ToolPolicy,
55
+ } from "../lib/pr-review-policy.ts";
48
56
 
49
57
  // ---------------------------------------------------------------------------
50
58
  // Config
@@ -54,6 +62,8 @@ type Tier = "light" | "medium" | "heavy";
54
62
  const TIERS: Tier[] = ["light", "medium", "heavy"];
55
63
 
56
64
  const UNSET = "(unset — pi default)";
65
+ const INHERIT_TOOL_POLICY = "(inherit — configured tools)";
66
+ const TOOL_POLICIES: ToolPolicy[] = ["configured", "none"];
57
67
  const TOOLS_PRESETS = ["read,bash,grep,find,ls", "read,grep,find,ls", "read"];
58
68
  const DEFAULT_BATCH_PARALLEL = 4;
59
69
  const MAX_BATCH_PARALLEL = 6;
@@ -66,7 +76,11 @@ const TIER_PURPOSE: Record<Tier, string> = {
66
76
  interface PrReviewConfig {
67
77
  /** Tier label -> model spec (e.g. "anthropic/model", "openai/model:high"). */
68
78
  tiers: Partial<Record<Tier, string>>;
69
- /** Tools granted to each review subagent process. */
79
+ /** Tier label -> ordered fallback model specs used for quota/capacity failures. */
80
+ fallbacks: Partial<Record<Tier, string[]>>;
81
+ /** Optional tier-level policy used when a tool call does not override it. */
82
+ toolPolicies: Partial<Record<Tier, ToolPolicy>>;
83
+ /** Tools granted to review subagents whose effective policy is configured. */
70
84
  tools: string[];
71
85
  }
72
86
 
@@ -91,6 +105,37 @@ function readConfigFile(filePath: string): Partial<PrReviewConfig> {
91
105
  }
92
106
  }
93
107
 
108
+ function normalizeFallbacks(
109
+ raw: Partial<Record<Tier, unknown>> | undefined,
110
+ preserveEmpty = false,
111
+ ): Partial<Record<Tier, string[]>> {
112
+ const out: Partial<Record<Tier, string[]>> = {};
113
+ if (!raw || typeof raw !== "object") return out;
114
+ for (const tier of TIERS) {
115
+ const value = raw[tier];
116
+ if (value === undefined) continue;
117
+ const list = Array.isArray(value)
118
+ ? value.map((v) => String(v).trim()).filter(Boolean)
119
+ : typeof value === "string"
120
+ ? value.split(",").map((v) => v.trim()).filter(Boolean)
121
+ : [];
122
+ if (list.length > 0 || preserveEmpty) out[tier] = [...new Set(list)];
123
+ }
124
+ return out;
125
+ }
126
+
127
+ function normalizeToolPolicies(
128
+ raw: Partial<Record<Tier, unknown>> | undefined,
129
+ ): Partial<Record<Tier, ToolPolicy>> {
130
+ const out: Partial<Record<Tier, ToolPolicy>> = {};
131
+ if (!raw || typeof raw !== "object") return out;
132
+ for (const tier of TIERS) {
133
+ const policy = normalizeToolPolicy(raw[tier]);
134
+ if (policy) out[tier] = policy;
135
+ }
136
+ return out;
137
+ }
138
+
94
139
  /** User config, overlaid by project config when the project is trusted. */
95
140
  function loadConfig(ctx: Pick<ExtensionContext, "cwd" | "isProjectTrusted">): PrReviewConfig {
96
141
  const user = readConfigFile(userConfigPath());
@@ -102,6 +147,11 @@ function loadConfig(ctx: Pick<ExtensionContext, "cwd" | "isProjectTrusted">): Pr
102
147
  }
103
148
  return {
104
149
  tiers: { ...(user.tiers ?? {}), ...(project.tiers ?? {}) },
150
+ fallbacks: { ...normalizeFallbacks(user.fallbacks, true), ...normalizeFallbacks(project.fallbacks, true) },
151
+ toolPolicies: {
152
+ ...normalizeToolPolicies(user.toolPolicies),
153
+ ...normalizeToolPolicies(project.toolPolicies),
154
+ },
105
155
  tools: project.tools ?? user.tools ?? DEFAULT_TOOLS,
106
156
  };
107
157
  }
@@ -109,7 +159,12 @@ function loadConfig(ctx: Pick<ExtensionContext, "cwd" | "isProjectTrusted">): Pr
109
159
  /** User-level config only (the scope the config command edits), with defaults. */
110
160
  function readUserConfig(): PrReviewConfig {
111
161
  const raw = readConfigFile(userConfigPath());
112
- return { tiers: { ...(raw.tiers ?? {}) }, tools: raw.tools ?? [...DEFAULT_TOOLS] };
162
+ return {
163
+ tiers: { ...(raw.tiers ?? {}) },
164
+ fallbacks: normalizeFallbacks(raw.fallbacks),
165
+ toolPolicies: normalizeToolPolicies(raw.toolPolicies),
166
+ tools: raw.tools ?? [...DEFAULT_TOOLS],
167
+ };
113
168
  }
114
169
 
115
170
  function writeUserConfig(next: PrReviewConfig): string {
@@ -119,19 +174,50 @@ function writeUserConfig(next: PrReviewConfig): string {
119
174
  return filePath;
120
175
  }
121
176
 
122
- /** Resolve the model spec for a tier, falling back to the nearest configured tier. */
123
- function resolveModelSpec(config: PrReviewConfig, tier: Tier): { spec?: string; usedTier?: Tier } {
124
- if (config.tiers[tier]) return { spec: config.tiers[tier], usedTier: tier };
125
- // Preference order: search outward from the requested tier.
126
- const order: Record<Tier, Tier[]> = {
127
- light: ["light", "medium", "heavy"],
128
- medium: ["medium", "heavy", "light"],
129
- heavy: ["heavy", "medium", "light"],
177
+ interface ModelAttempt {
178
+ spec?: string;
179
+ usedTier?: Tier;
180
+ kind: "primary" | "fallback" | "nearest" | "default";
181
+ fallbackIndex?: number;
182
+ }
183
+
184
+ const NEAREST_TIER_ORDER: Record<Tier, Tier[]> = {
185
+ light: ["light", "medium", "heavy"],
186
+ medium: ["medium", "heavy", "light"],
187
+ heavy: ["heavy", "medium", "light"],
188
+ };
189
+
190
+ /** Resolve the ordered model attempts for a tier, preserving nearest-tier/default behavior when the tier is unset. */
191
+ function resolveModelAttempts(config: PrReviewConfig, tier: Tier): ModelAttempt[] {
192
+ const attempts: ModelAttempt[] = [];
193
+ const seen = new Set<string>();
194
+ const add = (attempt: ModelAttempt) => {
195
+ const key = attempt.spec ?? "__pi_default__";
196
+ if (seen.has(key)) return;
197
+ seen.add(key);
198
+ attempts.push(attempt);
130
199
  };
131
- for (const candidate of order[tier]) {
132
- if (config.tiers[candidate]) return { spec: config.tiers[candidate], usedTier: candidate };
200
+
201
+ const primary = config.tiers[tier];
202
+ if (primary) {
203
+ add({ spec: primary, usedTier: tier, kind: "primary" });
204
+ } else {
205
+ let foundNearest = false;
206
+ for (const candidate of NEAREST_TIER_ORDER[tier]) {
207
+ const spec = config.tiers[candidate];
208
+ if (!spec) continue;
209
+ add({ spec, usedTier: candidate, kind: "nearest" });
210
+ foundNearest = true;
211
+ break;
212
+ }
213
+ if (!foundNearest) add({ kind: "default" });
133
214
  }
134
- return {};
215
+
216
+ for (const [index, spec] of (config.fallbacks[tier] ?? []).entries()) {
217
+ add({ spec, usedTier: tier, kind: "fallback", fallbackIndex: index });
218
+ }
219
+
220
+ return attempts;
135
221
  }
136
222
 
137
223
  // ---------------------------------------------------------------------------
@@ -295,6 +381,20 @@ interface SubagentPassRequest {
295
381
  tier: Tier;
296
382
  objective: string;
297
383
  context?: string;
384
+ toolPolicy?: ToolPolicy;
385
+ }
386
+
387
+ interface ModelAttemptReport {
388
+ kind: ModelAttempt["kind"];
389
+ spec?: string;
390
+ usedTier?: Tier;
391
+ model?: string;
392
+ exitCode: number;
393
+ status: "completed" | "failed";
394
+ retryable: boolean;
395
+ elapsedMs: number;
396
+ stopReason?: string;
397
+ errorMessage?: string;
298
398
  }
299
399
 
300
400
  interface SubagentPassResult {
@@ -309,39 +409,67 @@ interface SubagentPassResult {
309
409
  stderr?: string;
310
410
  stopReason?: string;
311
411
  errorMessage?: string;
412
+ attempts: ModelAttemptReport[];
413
+ fallbackUsed: boolean;
414
+ retryableFailure: boolean;
415
+ toolPolicy: ToolPolicy;
416
+ elapsedMs: number;
312
417
  }
313
418
 
314
- function noticeForTier(tier: Tier, spec: string | undefined, usedTier: Tier | undefined): string {
315
- if (spec) {
316
- return usedTier === tier
317
- ? `tier=${tier} model=${spec}`
318
- : `tier=${tier} (not configured; using ${usedTier} model=${spec})`;
419
+ function noticeForAttempt(tier: Tier, attempt: ModelAttempt): string {
420
+ if (!attempt.spec) {
421
+ return `tier=${tier} (no tier configured; using pi default model — run /pr-review-config to set tiers)`;
319
422
  }
320
- return `tier=${tier} (no tier configured; using pi default model — run /pr-review-config to set tiers)`;
423
+ if (attempt.kind === "fallback") {
424
+ return `tier=${tier} fallback #${(attempt.fallbackIndex ?? 0) + 1} model=${attempt.spec}`;
425
+ }
426
+ if (attempt.usedTier && attempt.usedTier !== tier) {
427
+ return `tier=${tier} (not configured; using ${attempt.usedTier} model=${attempt.spec})`;
428
+ }
429
+ return `tier=${tier} model=${attempt.spec}`;
430
+ }
431
+
432
+ function noticeForResult(tier: Tier, attempts: ModelAttemptReport[], finalNotice: string): string {
433
+ const failedBeforeSuccess = attempts.filter((a) => a.status === "failed").length;
434
+ if (failedBeforeSuccess > 0 && attempts.some((a) => a.status === "completed")) {
435
+ return `${finalNotice} (after ${failedBeforeSuccess} retry${failedBeforeSuccess === 1 ? "" : "ies"})`;
436
+ }
437
+ return finalNotice;
321
438
  }
322
439
 
323
440
  function buildPassTask(objective: string, context: string | undefined): string {
324
441
  return context ? `Objective: ${objective}\n\n--- PR context / diff ---\n${context}` : `Objective: ${objective}`;
325
442
  }
326
443
 
327
- async function runSubagentPass(
444
+ function isRetryableModelFailure(result: RunResult): boolean {
445
+ if (result.stopReason === "aborted") return false;
446
+ const haystack = [result.errorMessage, result.stderr, result.text, result.stopReason]
447
+ .filter(Boolean)
448
+ .join("\n")
449
+ .toLowerCase();
450
+ if (!haystack) return false;
451
+ return /(?:\b429\b|rate[\s_-]*limit(?:ed)?|too many requests|quota|usage[\s_-]*limit|insufficient[\s_-]*quota|resource[\s_-]*exhausted|out of credits|insufficient credits|credit limit|billing quota|billing hard limit|at capacity|overloaded|temporarily unavailable|model (?:is )?(?:temporarily|currently) unavailable|service unavailable|try again later)/i.test(
452
+ haystack,
453
+ );
454
+ }
455
+
456
+ async function runSubagentAttempt(
328
457
  config: PrReviewConfig,
329
458
  ctx: Pick<ExtensionContext, "cwd">,
330
459
  pass: SubagentPassRequest,
460
+ attempt: ModelAttempt,
461
+ toolPolicy: ToolPolicy,
331
462
  signal: AbortSignal | undefined,
332
463
  onText?: (text: string) => void,
333
- ): Promise<SubagentPassResult> {
334
- const tier = pass.tier;
335
- const { spec, usedTier } = resolveModelSpec(config, tier);
336
- const notice = noticeForTier(tier, spec, usedTier);
464
+ ): Promise<{ result: RunResult; notice: string; elapsedMs: number }> {
337
465
  let tmp: { dir: string; filePath: string } | undefined;
338
-
466
+ const startedAt = Date.now();
339
467
  try {
340
- const args = ["--mode", "json", "-p", "--no-session"];
341
- if (spec) args.push("--model", spec);
342
- if (config.tools.length > 0) args.push("--tools", config.tools.join(","));
468
+ const args = buildReviewBaseArgs();
469
+ if (attempt.spec) args.push("--model", attempt.spec);
470
+ appendToolPolicyArgs(args, toolPolicy, config.tools);
343
471
 
344
- tmp = await writeTempPrompt(tier, buildSubagentSystemPrompt(tier));
472
+ tmp = await writeTempPrompt(pass.tier, buildSubagentSystemPrompt(pass.tier));
345
473
  args.push("--append-system-prompt", tmp.filePath);
346
474
  args.push(buildPassTask(pass.objective, pass.context));
347
475
 
@@ -349,32 +477,12 @@ async function runSubagentPass(
349
477
  const result = await runReviewSubprocess(invocation.command, invocation.args, ctx.cwd, signal, (text) => {
350
478
  onText?.(text);
351
479
  });
352
-
353
- const failed = result.exitCode !== 0 || result.stopReason === "error" || result.stopReason === "aborted";
354
- return {
355
- id: pass.id ?? tier,
356
- tier,
357
- usedTier,
358
- model: result.model ?? spec,
359
- exitCode: result.exitCode,
360
- status: failed ? "failed" : "completed",
361
- notice,
362
- text: result.text || (failed ? "" : "NO FINDINGS."),
363
- stderr: result.stderr || undefined,
364
- stopReason: result.stopReason,
365
- errorMessage: result.errorMessage,
366
- };
480
+ return { result, notice: noticeForAttempt(pass.tier, attempt), elapsedMs: Date.now() - startedAt };
367
481
  } catch (e) {
368
482
  return {
369
- id: pass.id ?? tier,
370
- tier,
371
- usedTier,
372
- model: spec,
373
- exitCode: 1,
374
- status: "failed",
375
- notice,
376
- text: "",
377
- errorMessage: errMessage(e),
483
+ result: { text: "", exitCode: 1, stderr: "", errorMessage: errMessage(e) },
484
+ notice: noticeForAttempt(pass.tier, attempt),
485
+ elapsedMs: Date.now() - startedAt,
378
486
  };
379
487
  } finally {
380
488
  if (tmp) {
@@ -387,6 +495,93 @@ async function runSubagentPass(
387
495
  }
388
496
  }
389
497
 
498
+ async function runSubagentPass(
499
+ config: PrReviewConfig,
500
+ ctx: Pick<ExtensionContext, "cwd">,
501
+ pass: SubagentPassRequest,
502
+ signal: AbortSignal | undefined,
503
+ onText?: (text: string) => void,
504
+ ): Promise<SubagentPassResult> {
505
+ const startedAt = Date.now();
506
+ const tier = pass.tier;
507
+ const toolPolicy = resolveToolPolicy(pass.toolPolicy, config.toolPolicies[tier]);
508
+ const attempts = resolveModelAttempts(config, tier);
509
+ const reports: ModelAttemptReport[] = [];
510
+ let lastResult: RunResult | undefined;
511
+ let lastNotice = noticeForAttempt(tier, attempts[0]!);
512
+
513
+ for (const attempt of attempts) {
514
+ const { result, notice, elapsedMs } = await runSubagentAttempt(
515
+ config,
516
+ ctx,
517
+ pass,
518
+ attempt,
519
+ toolPolicy,
520
+ signal,
521
+ onText,
522
+ );
523
+ const failed = result.exitCode !== 0 || result.stopReason === "error" || result.stopReason === "aborted";
524
+ const retryable = failed && isRetryableModelFailure(result);
525
+ lastResult = result;
526
+ lastNotice = notice;
527
+ reports.push({
528
+ kind: attempt.kind,
529
+ spec: attempt.spec,
530
+ usedTier: attempt.usedTier,
531
+ model: result.model ?? attempt.spec,
532
+ exitCode: result.exitCode,
533
+ status: failed ? "failed" : "completed",
534
+ retryable,
535
+ elapsedMs,
536
+ stopReason: result.stopReason,
537
+ errorMessage: result.errorMessage,
538
+ });
539
+
540
+ if (!failed) {
541
+ return {
542
+ id: pass.id ?? tier,
543
+ tier,
544
+ usedTier: attempt.usedTier,
545
+ model: result.model ?? attempt.spec,
546
+ exitCode: result.exitCode,
547
+ status: "completed",
548
+ notice: noticeForResult(tier, reports, notice),
549
+ text: result.text || "NO FINDINGS.",
550
+ stderr: result.stderr || undefined,
551
+ stopReason: result.stopReason,
552
+ errorMessage: result.errorMessage,
553
+ attempts: reports,
554
+ fallbackUsed: attempt.kind === "fallback" || reports.length > 1,
555
+ retryableFailure: false,
556
+ toolPolicy,
557
+ elapsedMs: Date.now() - startedAt,
558
+ };
559
+ }
560
+
561
+ if (!retryable || signal?.aborted) break;
562
+ }
563
+
564
+ const final = lastResult ?? { text: "", exitCode: 1, stderr: "", errorMessage: "No model attempts were available." };
565
+ return {
566
+ id: pass.id ?? tier,
567
+ tier,
568
+ usedTier: reports.at(-1)?.usedTier,
569
+ model: reports.at(-1)?.model,
570
+ exitCode: final.exitCode,
571
+ status: "failed",
572
+ notice: noticeForResult(tier, reports, lastNotice),
573
+ text: final.text || "",
574
+ stderr: final.stderr || undefined,
575
+ stopReason: final.stopReason,
576
+ errorMessage: final.errorMessage,
577
+ attempts: reports,
578
+ fallbackUsed: reports.length > 1,
579
+ retryableFailure: reports.at(-1)?.retryable ?? false,
580
+ toolPolicy,
581
+ elapsedMs: Date.now() - startedAt,
582
+ };
583
+ }
584
+
390
585
  function normalizeMaxParallel(raw: unknown, count: number): number {
391
586
  if (count <= 0) return 0;
392
587
  const requested = typeof raw === "number" && Number.isFinite(raw) ? Math.floor(raw) : DEFAULT_BATCH_PARALLEL;
@@ -410,6 +605,13 @@ async function runWithConcurrency<T, R>(
410
605
  return results;
411
606
  }
412
607
 
608
+ function formatAttemptSummary(result: SubagentPassResult): string {
609
+ if (result.attempts.length <= 1) return "";
610
+ return `attempts: ${result.attempts
611
+ .map((a) => `${a.status === "completed" ? "✓" : a.retryable ? "↻" : "✗"} ${a.spec ?? "pi default"}`)
612
+ .join(" → ")}`;
613
+ }
614
+
413
615
  function formatBatchResults(results: SubagentPassResult[], maxParallel: number): string {
414
616
  const failed = results.filter((r) => r.status === "failed");
415
617
  const lines = [
@@ -421,7 +623,16 @@ function formatBatchResults(results: SubagentPassResult[], maxParallel: number):
421
623
  );
422
624
  }
423
625
  for (const result of results) {
424
- lines.push("", `## Pass: ${result.id}`, `status: ${result.status}`, result.notice);
626
+ lines.push(
627
+ "",
628
+ `## Pass: ${result.id}`,
629
+ `status: ${result.status}`,
630
+ `tool_policy: ${result.toolPolicy}`,
631
+ `elapsed_ms: ${result.elapsedMs}`,
632
+ result.notice,
633
+ );
634
+ const attemptSummary = formatAttemptSummary(result);
635
+ if (attemptSummary) lines.push(attemptSummary);
425
636
  if (result.status === "failed") {
426
637
  const detail = result.errorMessage || result.stderr || result.text || "(no output)";
427
638
  lines.push(`error: ${detail}`);
@@ -457,13 +668,19 @@ const ReviewSubagentParams = Type.Object({
457
668
  "Shared context for the subagent, typically the PR title/description plus the unified diff to review.",
458
669
  }),
459
670
  ),
671
+ tool_policy: Type.Optional(
672
+ StringEnum(["none", "configured"] as const, {
673
+ description:
674
+ "Tool access for this pass. none emits --no-tools; configured uses the configured allowlist. Request override wins over tier config; omission preserves configured legacy behavior unless tier policy is set.",
675
+ }),
676
+ ),
460
677
  });
461
678
 
462
679
  const ReviewSubagentsParams = Type.Object({
463
680
  context: Type.Optional(
464
681
  Type.String({
465
682
  description:
466
- "Shared PR context for every pass, typically PR title/body/metadata, in-scope convention-file summaries, and the unified diff.",
683
+ "Shared PR context for every pass, typically PR title/body/metadata, cross-cutting requirements, and the unified diff. Put specialist-only convention excerpts in the medium pass context.",
467
684
  }),
468
685
  ),
469
686
  max_parallel: Type.Optional(
@@ -490,6 +707,12 @@ const ReviewSubagentsParams = Type.Object({
490
707
  description: "Optional pass-specific context appended after the shared context.",
491
708
  }),
492
709
  ),
710
+ tool_policy: Type.Optional(
711
+ StringEnum(["none", "configured"] as const, {
712
+ description:
713
+ "Tool access for this pass. none emits --no-tools; configured uses the configured allowlist. The resolved policy remains fixed across fallback model attempts.",
714
+ }),
715
+ ),
493
716
  }),
494
717
  {
495
718
  description: "Independent review passes to run concurrently. Results are returned in this same order.",
@@ -520,7 +743,12 @@ export default function (pi: ExtensionAPI) {
520
743
  const result = await runSubagentPass(
521
744
  loadConfig(ctx),
522
745
  ctx,
523
- { tier, objective: params.objective, context: params.context },
746
+ {
747
+ tier,
748
+ objective: params.objective,
749
+ context: params.context,
750
+ toolPolicy: normalizeToolPolicy(params.tool_policy),
751
+ },
524
752
  signal,
525
753
  (text) => onUpdate?.({ content: [{ type: "text", text }] }),
526
754
  );
@@ -535,6 +763,11 @@ export default function (pi: ExtensionAPI) {
535
763
  usedTier: result.usedTier,
536
764
  model: result.model,
537
765
  exitCode: result.exitCode,
766
+ fallbackUsed: result.fallbackUsed,
767
+ retryableFailure: result.retryableFailure,
768
+ toolPolicy: result.toolPolicy,
769
+ elapsedMs: result.elapsedMs,
770
+ attempts: result.attempts,
538
771
  },
539
772
  };
540
773
  }
@@ -546,6 +779,10 @@ export default function (pi: ExtensionAPI) {
546
779
  usedTier: result.usedTier,
547
780
  model: result.model,
548
781
  exitCode: result.exitCode,
782
+ fallbackUsed: result.fallbackUsed,
783
+ toolPolicy: result.toolPolicy,
784
+ elapsedMs: result.elapsedMs,
785
+ attempts: result.attempts,
549
786
  },
550
787
  };
551
788
  },
@@ -587,6 +824,7 @@ export default function (pi: ExtensionAPI) {
587
824
  tier,
588
825
  objective: pass.objective,
589
826
  context: combineContexts(sharedContext, typeof pass.context === "string" ? pass.context : undefined),
827
+ toolPolicy: normalizeToolPolicy(pass.tool_policy),
590
828
  };
591
829
  });
592
830
  const maxParallel = normalizeMaxParallel(params.max_parallel, passes.length);
@@ -622,6 +860,11 @@ export default function (pi: ExtensionAPI) {
622
860
  status: r.status,
623
861
  stopReason: r.stopReason,
624
862
  errorMessage: r.errorMessage,
863
+ fallbackUsed: r.fallbackUsed,
864
+ retryableFailure: r.retryableFailure,
865
+ toolPolicy: r.toolPolicy,
866
+ elapsedMs: r.elapsedMs,
867
+ attempts: r.attempts,
625
868
  outputChars: r.text.length,
626
869
  })),
627
870
  },
@@ -630,7 +873,7 @@ export default function (pi: ExtensionAPI) {
630
873
  });
631
874
 
632
875
  pi.registerCommand("pr-review-config", {
633
- description: "Open the review-tier settings menu, or show/set light/medium/heavy models for /pr-review",
876
+ description: "Open the review-tier settings menu, or show/set light/medium/heavy models and fallbacks for /pr-review",
634
877
  handler: async (args, ctx) => {
635
878
  const raw = (args ?? "").trim();
636
879
  const parsed = parseConfigArgs(raw);
@@ -640,7 +883,7 @@ export default function (pi: ExtensionAPI) {
640
883
  }
641
884
 
642
885
  try {
643
- // Direct set: `/pr-review-config light=... heavy=...`
886
+ // Direct set: `/pr-review-config light=... heavy=... heavy_fallbacks=...`
644
887
  if (parsed.hasChanges) {
645
888
  const next = applyConfigPatch(readUserConfig(), parsed.patch);
646
889
  writeUserConfig(next);
@@ -701,7 +944,15 @@ function shouldOpenConfigMenu(args: string, ctx: ExtensionContext): boolean {
701
944
 
702
945
  const CONFIG_COMPLETIONS: Array<{ value: string; label: string }> = [
703
946
  ...TIERS.map((t) => ({ value: `${t}=`, label: `${t}=<model> — ${TIER_PURPOSE[t]}` })),
704
- { value: "tools=", label: "tools=read,bash,grep,find,ls — tools granted to review subagents" },
947
+ ...TIERS.map((t) => ({
948
+ value: `${t}_fallbacks=`,
949
+ label: `${t}_fallbacks=<model1,model2> — retry chain for quota/rate-limit failures`,
950
+ })),
951
+ ...TIERS.map((t) => ({
952
+ value: `${t}_tool_policy=`,
953
+ label: `${t}_tool_policy=<none|configured|unset> — default tool access when a pass does not override it`,
954
+ })),
955
+ { value: "tools=", label: "tools=read,bash,grep,find,ls — allowlist used by configured policy" },
705
956
  { value: "show", label: "show — print the current tier config" },
706
957
  ];
707
958
 
@@ -734,13 +985,45 @@ function splitConfigArgs(input: string): string[] {
734
985
  return tokens;
735
986
  }
736
987
 
988
+ function splitCommaList(value: string): string[] {
989
+ return value.split(",").map((s) => s.trim()).filter(Boolean);
990
+ }
991
+
992
+ function isFallbackKey(key: string): boolean {
993
+ for (const tier of TIERS) {
994
+ if (key === `${tier}_fallbacks` || key === `${tier}-fallbacks` || key === `${tier}.fallbacks`) {
995
+ return true;
996
+ }
997
+ }
998
+ return false;
999
+ }
1000
+
1001
+ function isToolPolicyKey(key: string): boolean {
1002
+ for (const tier of TIERS) {
1003
+ if (
1004
+ key === `${tier}_tool_policy` ||
1005
+ key === `${tier}-tool-policy` ||
1006
+ key === `${tier}.toolpolicy`
1007
+ ) {
1008
+ return true;
1009
+ }
1010
+ }
1011
+ return false;
1012
+ }
1013
+
1014
+ function tierFromCompoundKey(key: string): Tier {
1015
+ return key.split(/[_\-.]/)[0] as Tier;
1016
+ }
1017
+
737
1018
  interface ConfigPatch {
738
1019
  tiers: Partial<Record<Tier, string | null>>;
1020
+ fallbacks: Partial<Record<Tier, string[] | null>>;
1021
+ toolPolicies: Partial<Record<Tier, ToolPolicy | null>>;
739
1022
  tools?: string[];
740
1023
  }
741
1024
 
742
1025
  function parseConfigArgs(args: string): { patch: ConfigPatch; hasChanges: boolean; errors: string[] } {
743
- const patch: ConfigPatch = { tiers: {} };
1026
+ const patch: ConfigPatch = { tiers: {}, fallbacks: {}, toolPolicies: {} };
744
1027
  const errors: string[] = [];
745
1028
  const tokens = splitConfigArgs(args).filter((t) => !["show", "get", "current"].includes(t.toLowerCase()));
746
1029
  for (const token of tokens) {
@@ -753,28 +1036,65 @@ function parseConfigArgs(args: string): { patch: ConfigPatch; hasChanges: boolea
753
1036
  const value = token.slice(eq + 1);
754
1037
  if ((TIERS as string[]).includes(key)) {
755
1038
  patch.tiers[key as Tier] = value === "" || value === "unset" ? null : value;
1039
+ } else if (isFallbackKey(key)) {
1040
+ const tier = tierFromCompoundKey(key);
1041
+ patch.fallbacks[tier] = value === "" || value === "unset" || value === "none" ? null : splitCommaList(value);
1042
+ } else if (isToolPolicyKey(key)) {
1043
+ const tier = tierFromCompoundKey(key);
1044
+ if (value === "" || value === "unset" || value === "inherit") patch.toolPolicies[tier] = null;
1045
+ else {
1046
+ const policy = normalizeToolPolicy(value);
1047
+ if (policy) patch.toolPolicies[tier] = policy;
1048
+ else errors.push(`invalid ${key} "${value}" (expected none|configured|unset)`);
1049
+ }
756
1050
  } else if (key === "tools") {
757
- patch.tools = value.split(",").map((s) => s.trim()).filter(Boolean);
1051
+ patch.tools = splitCommaList(value);
758
1052
  } else {
759
- errors.push(`unknown key "${key}" (expected light|medium|heavy|tools)`);
1053
+ errors.push(
1054
+ `unknown key "${key}" (expected light|medium|heavy|<tier>_fallbacks|<tier>_tool_policy|tools)`,
1055
+ );
760
1056
  }
761
1057
  }
762
- const hasChanges = Object.keys(patch.tiers).length > 0 || patch.tools !== undefined;
1058
+ const hasChanges =
1059
+ Object.keys(patch.tiers).length > 0 ||
1060
+ Object.keys(patch.fallbacks).length > 0 ||
1061
+ Object.keys(patch.toolPolicies).length > 0 ||
1062
+ patch.tools !== undefined;
763
1063
  return { patch, hasChanges, errors };
764
1064
  }
765
1065
 
766
1066
  function applyConfigPatch(base: PrReviewConfig, patch: ConfigPatch): PrReviewConfig {
767
- const next: PrReviewConfig = { tiers: { ...base.tiers }, tools: [...base.tools] };
1067
+ const next: PrReviewConfig = {
1068
+ tiers: { ...base.tiers },
1069
+ fallbacks: normalizeFallbacks(base.fallbacks),
1070
+ toolPolicies: normalizeToolPolicies(base.toolPolicies),
1071
+ tools: [...base.tools],
1072
+ };
768
1073
  for (const tier of TIERS) {
769
- if (!(tier in patch.tiers)) continue;
770
- const value = patch.tiers[tier];
771
- if (value === null || value === undefined) delete next.tiers[tier];
772
- else next.tiers[tier] = value;
1074
+ if (tier in patch.tiers) {
1075
+ const value = patch.tiers[tier];
1076
+ if (value === null || value === undefined) delete next.tiers[tier];
1077
+ else next.tiers[tier] = value;
1078
+ }
1079
+ if (tier in patch.fallbacks) {
1080
+ const value = patch.fallbacks[tier];
1081
+ if (value === null || value === undefined || value.length === 0) delete next.fallbacks[tier];
1082
+ else next.fallbacks[tier] = [...new Set(value)];
1083
+ }
1084
+ if (tier in patch.toolPolicies) {
1085
+ const value = patch.toolPolicies[tier];
1086
+ if (value === null || value === undefined) delete next.toolPolicies[tier];
1087
+ else next.toolPolicies[tier] = value;
1088
+ }
773
1089
  }
774
1090
  if (patch.tools) next.tools = patch.tools;
775
1091
  return next;
776
1092
  }
777
1093
 
1094
+ function formatModelList(models: string[] | undefined): string {
1095
+ return models && models.length > 0 ? `\`${models.join(",")}\`` : "_none_";
1096
+ }
1097
+
778
1098
  function summarizeConfig(
779
1099
  user: PrReviewConfig,
780
1100
  effective: PrReviewConfig,
@@ -799,7 +1119,14 @@ function summarizeConfig(
799
1119
  (t) =>
800
1120
  `| \`${t}\` | ${user.tiers[t] ? `\`${user.tiers[t]}\`` : "_unset_"} | ${effective.tiers[t] ? `\`${effective.tiers[t]}\`` : "_pi default_"} | ${TIER_PURPOSE[t]} |`,
801
1121
  ),
802
- `| \`tools\` | \`${user.tools.join(",")}\` | \`${effective.tools.join(",")}\` | tools granted to each review subagent |`,
1122
+ `| \`tools\` | \`${user.tools.join(",")}\` | \`${effective.tools.join(",")}\` | allowlist used when policy is \`configured\` |`,
1123
+ "",
1124
+ "| Tier | Your fallbacks | Effective fallbacks | Tool policy |",
1125
+ "|---|---|---|---|",
1126
+ ...TIERS.map(
1127
+ (t) =>
1128
+ `| \`${t}\` | ${formatModelList(user.fallbacks[t])} | ${formatModelList(effective.fallbacks[t])} | ${user.toolPolicies[t] ? `\`${user.toolPolicies[t]}\`` : "_inherit configured_"} → \`${effective.toolPolicies[t] ?? "configured"}\` |`,
1129
+ ),
803
1130
  "",
804
1131
  `User config: \`${userConfigPath()}\``,
805
1132
  ];
@@ -810,7 +1137,11 @@ function summarizeConfig(
810
1137
  "- Open the settings menu: `/pr-review-config`",
811
1138
  "- Print this summary: `/pr-review-config show`",
812
1139
  "- Set directly: `/pr-review-config light=provider/model heavy=provider/model:high`",
1140
+ "- Set fallback chain: `/pr-review-config heavy_fallbacks=provider/backup:high,provider/backup2`",
1141
+ "- Set tier tool policy: `/pr-review-config light_tool_policy=none`",
813
1142
  "- Clear a tier: `/pr-review-config medium=unset`",
1143
+ "- Clear fallback chain: `/pr-review-config heavy_fallbacks=unset`",
1144
+ "- Restore legacy tier tool behavior: `/pr-review-config light_tool_policy=unset`",
814
1145
  "- A `<model>` is any pi model pattern (`provider/model` or `provider/model:thinking`).",
815
1146
  );
816
1147
  return lines.join("\n");
@@ -828,10 +1159,14 @@ const MODEL_LIST_ROWS = 10;
828
1159
  * search UX and fuzzyFilter's slash-token matching, so typing a model name substring
829
1160
  * finds `provider/model`). The SelectList is rebuilt as the query changes.
830
1161
  */
831
- function buildModelSubmenu(available: string[], currentSpec: string | undefined) {
1162
+ function buildModelSubmenu(
1163
+ available: string[],
1164
+ currentSpec: string | undefined,
1165
+ unsetDescription = "Fall back to the nearest configured tier, then the pi default.",
1166
+ ) {
832
1167
  return (_currentValue: string, done: (selectedValue?: string) => void) => {
833
1168
  const allItems: SelectItem[] = [
834
- { value: "__unset__", label: UNSET, description: "Fall back to the nearest configured tier, then the pi default." },
1169
+ { value: "__unset__", label: UNSET, description: unsetDescription },
835
1170
  ...available.map((spec) => ({ value: spec, label: spec })),
836
1171
  ];
837
1172
  const theme = getSelectListTheme();
@@ -890,14 +1225,35 @@ function configMenuItems(cfg: PrReviewConfig, available: string[]): SettingItem[
890
1225
  currentValue: cfg.tiers[tier] ?? UNSET,
891
1226
  submenu: buildModelSubmenu(available, cfg.tiers[tier]),
892
1227
  }));
1228
+ const fallbackItems: SettingItem[] = TIERS.map((tier) => ({
1229
+ id: `${tier}_fallbacks`,
1230
+ label: `${tier} fallback model`,
1231
+ description:
1232
+ "Retry model for quota/rate-limit/capacity failures. Press Enter to set one fallback; use key=value for chains.",
1233
+ currentValue: cfg.fallbacks[tier]?.join(",") || "(none)",
1234
+ submenu: buildModelSubmenu(
1235
+ available,
1236
+ cfg.fallbacks[tier]?.[0],
1237
+ "Clear this tier's fallback chain. Use key=value form to set multiple fallbacks.",
1238
+ ),
1239
+ }));
1240
+ const policyItems: SettingItem[] = TIERS.map((tier) => ({
1241
+ id: `${tier}_tool_policy`,
1242
+ label: `${tier} tool policy`,
1243
+ description: "Default when a pass does not explicitly set tool_policy. Enter/Space cycles values.",
1244
+ currentValue: cfg.toolPolicies[tier] ?? INHERIT_TOOL_POLICY,
1245
+ values: [INHERIT_TOOL_POLICY, ...TOOL_POLICIES],
1246
+ }));
893
1247
  const current = cfg.tools.join(",");
894
1248
  const toolValues = [current, ...TOOLS_PRESETS.filter((p) => p !== current)];
895
1249
  return [
896
1250
  ...tierItems,
1251
+ ...fallbackItems,
1252
+ ...policyItems,
897
1253
  {
898
1254
  id: "tools",
899
- label: "subagent tools",
900
- description: "Tools granted to each review subagent. Enter/Space cycles presets.",
1255
+ label: "configured tool allowlist",
1256
+ description: "Tools available when effective policy is configured. Enter/Space cycles presets.",
901
1257
  currentValue: current,
902
1258
  values: toolValues,
903
1259
  },
@@ -920,7 +1276,11 @@ async function showConfigMenu(
920
1276
 
921
1277
  let settingsList: SettingsList;
922
1278
  const refresh = () => {
923
- for (const tier of TIERS) settingsList.updateValue(tier, draft.tiers[tier] ?? UNSET);
1279
+ for (const tier of TIERS) {
1280
+ settingsList.updateValue(tier, draft.tiers[tier] ?? UNSET);
1281
+ settingsList.updateValue(`${tier}_fallbacks`, draft.fallbacks[tier]?.join(",") || "(none)");
1282
+ settingsList.updateValue(`${tier}_tool_policy`, draft.toolPolicies[tier] ?? INHERIT_TOOL_POLICY);
1283
+ }
924
1284
  settingsList.updateValue("tools", draft.tools.join(","));
925
1285
  };
926
1286
 
@@ -928,15 +1288,32 @@ async function showConfigMenu(
928
1288
  if ((TIERS as string[]).includes(id)) {
929
1289
  if (newValue === "__unset__") delete draft.tiers[id as Tier];
930
1290
  else draft.tiers[id as Tier] = newValue;
1291
+ } else if (isFallbackKey(id)) {
1292
+ const tier = tierFromCompoundKey(id);
1293
+ if (newValue === "__unset__") delete draft.fallbacks[tier];
1294
+ else draft.fallbacks[tier] = [newValue];
1295
+ } else if (isToolPolicyKey(id)) {
1296
+ const tier = tierFromCompoundKey(id);
1297
+ if (newValue === INHERIT_TOOL_POLICY) delete draft.toolPolicies[tier];
1298
+ else {
1299
+ const policy = normalizeToolPolicy(newValue);
1300
+ if (policy) draft.toolPolicies[tier] = policy;
1301
+ }
931
1302
  } else if (id === "tools") {
932
- draft.tools = newValue.split(",").map((s) => s.trim()).filter(Boolean);
1303
+ draft.tools = splitCommaList(newValue);
933
1304
  } else {
934
1305
  return;
935
1306
  }
936
1307
  try {
937
1308
  writeUserConfig(draft);
938
1309
  refresh();
939
- const shown = id === "tools" ? draft.tools.join(",") : (draft.tiers[id as Tier] ?? UNSET);
1310
+ const shown = id === "tools"
1311
+ ? draft.tools.join(",")
1312
+ : isFallbackKey(id)
1313
+ ? (draft.fallbacks[tierFromCompoundKey(id)]?.join(",") ?? "(none)")
1314
+ : isToolPolicyKey(id)
1315
+ ? (draft.toolPolicies[tierFromCompoundKey(id)] ?? INHERIT_TOOL_POLICY)
1316
+ : (draft.tiers[id as Tier] ?? UNSET);
940
1317
  ctx.ui.notify(`PR review config: ${id} = ${shown}`, "info");
941
1318
  } catch (e) {
942
1319
  ctx.ui.notify(`pr-review-config failed: ${errMessage(e)}`, "error");
@@ -946,7 +1323,7 @@ async function showConfigMenu(
946
1323
 
947
1324
  settingsList = new SettingsList(
948
1325
  configMenuItems(draft, available),
949
- 6,
1326
+ 12,
950
1327
  getSettingsListTheme(),
951
1328
  (id, newValue) => persist(id, newValue),
952
1329
  () => done(undefined),
@@ -0,0 +1,32 @@
1
+ export type ToolPolicy = "none" | "configured";
2
+
3
+ /** Isolated review subprocesses receive all review context explicitly. */
4
+ export function buildReviewBaseArgs(): string[] {
5
+ return ["--mode", "json", "-p", "--no-session", "--no-context-files"];
6
+ }
7
+
8
+ export function normalizeToolPolicy(value: unknown): ToolPolicy | undefined {
9
+ return value === "none" || value === "configured" ? value : undefined;
10
+ }
11
+
12
+ /** Request override wins, then explicit tier config, then legacy configured behavior. */
13
+ export function resolveToolPolicy(
14
+ requested: ToolPolicy | undefined,
15
+ configured: ToolPolicy | undefined,
16
+ ): ToolPolicy {
17
+ return requested ?? configured ?? "configured";
18
+ }
19
+
20
+ /** Preserve legacy configured behavior; only `none` explicitly disables every tool. */
21
+ export function appendToolPolicyArgs(
22
+ args: string[],
23
+ policy: ToolPolicy,
24
+ configuredTools: string[],
25
+ ): string[] {
26
+ if (policy === "none") {
27
+ args.push("--no-tools");
28
+ } else if (configuredTools.length > 0) {
29
+ args.push("--tools", configuredTools.join(","));
30
+ }
31
+ return args;
32
+ }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-pr-review",
3
- "version": "1.1.2",
3
+ "version": "1.3.1",
4
4
  "description": "Model-agnostic GitHub PR code review prompt for pi. Pass a PR number; it derives the base and head branches from the PR in the current repo, reviews the diff, and returns structured JSON findings.",
5
5
  "keywords": [
6
6
  "pi-package",
@@ -14,6 +14,9 @@
14
14
  "type": "git",
15
15
  "url": "git+https://github.com/10ego/pi-pr-review.git"
16
16
  },
17
+ "scripts": {
18
+ "test": "bun test"
19
+ },
17
20
  "peerDependencies": {
18
21
  "@earendil-works/pi-ai": "*",
19
22
  "@earendil-works/pi-coding-agent": "*",
@@ -21,11 +24,11 @@
21
24
  "typebox": "*"
22
25
  },
23
26
  "pi": {
24
- "prompts": [
25
- "./prompts"
26
- ],
27
27
  "extensions": [
28
28
  "./extensions"
29
+ ],
30
+ "prompts": [
31
+ "./prompts"
29
32
  ]
30
33
  }
31
34
  }
@@ -20,14 +20,16 @@ You are the **orchestrator**. You own all GitHub access, skip decisions, convent
20
20
 
21
21
  If the `review_subagents` batch tool is available, prefer it over multiple single-pass calls. Fetch PR metadata and the unified diff once, gather any relevant convention-file excerpts, then call `review_subagents` with shared `context`, `max_parallel`, and ordered `passes`. This guarantees bounded parallel fan-out instead of depending on whether the tool interface runs separate calls concurrently. Use these pass assignments:
22
22
 
23
- | Pass id | Tier label | Scope |
24
- |---------|------------|-------|
25
- | `overview` | `light` | Step 3 overview, strengths, and high-level risk areas |
26
- | `conventions-maintainability` | `medium` | Step 5 convention compliance, readability, maintainability, test gaps, and nits |
27
- | `correctness` | `heavy` | Step 5 compile/parse, logic, error handling, lifecycle, concurrency, and edge-case defects |
28
- | `security-performance` | `heavy` | Step 5 security vulnerabilities and performance regressions |
23
+ | Pass id | Tier label | Tool policy | Scope |
24
+ |---------|------------|-------------|-------|
25
+ | `overview` | `light` | `none` | Step 3 overview, strengths, and high-level risk areas |
26
+ | `conventions-maintainability` | `medium` | `configured` | Step 5 convention compliance, readability, maintainability, test gaps, and nits |
27
+ | `correctness` | `heavy` | `configured` | Step 5 compile/parse, logic, error handling, lifecycle, concurrency, and edge-case defects |
28
+ | `security-performance` | `heavy` | `configured` | Step 5 security vulnerabilities and performance regressions |
29
29
 
30
- Call `review_subagents` with `{ context, max_parallel: 4, passes: [...] }`. If any pass returns `status: failed`, treat the review evidence as incomplete: rerun the failed pass with `review_subagent` or perform that pass inline before finalizing. Always put the PR title/description, relevant metadata, convention-file excerpts, and unified diff in shared `context` so subagents do not refetch anything.
30
+ Call `review_subagents` with `{ context, max_parallel: 4, passes: [...] }`, setting each pass's `tool_policy` exactly as shown. Shared `context` contains the PR title/description, relevant metadata, complete unified diff, and strictly cross-cutting review requirements. Put convention-file paths and excerpts only in the `conventions-maintainability` pass's own `context`; they are appended to shared context by the tool. This avoids sending specialist-only material to every model while still allowing the medium reviewer to inspect surrounding files when maintainability or test-gap analysis requires it.
31
+
32
+ If deterministic context assembly reports that required input is missing or truncated before dispatch (for example, an incomplete diff or unreadable applicable convention file), set `tool_policy: configured` for that affected pass instead of `none`. Do not ask the model to self-diagnose and rerun. If any pass returns `status: failed`, treat the review evidence as incomplete: rerun the failed pass with `review_subagent` using the same tool policy, or perform that pass inline before finalizing.
31
33
 
32
34
  If `review_subagents` is unavailable but `review_subagent` is available, run the same pass assignments as individual `review_subagent` calls; emit independent calls in the same turn when the interface supports parallel tool calls. If neither subagent tool is available, perform every pass yourself inline on the current session model.
33
35
 
@@ -78,25 +80,25 @@ List (do not dump contents yet) the repository convention files that could gover
78
80
  - The root convention file (`CLAUDE.md`, and/or `AGENTS.md` if present).
79
81
  - Any convention file living in a directory that contains a file modified by this PR.
80
82
 
81
- When you evaluate compliance for a given changed file, only apply convention files that share that file's path or a parent path. Read a convention file's contents when a changed file falls under its scope, and include the relevant rule excerpts (with file paths) in the shared context for the medium pass.
83
+ When you evaluate compliance for a given changed file, only apply convention files that share that file's path or a parent path. Read a convention file's contents when a changed file falls under its scope, and include the relevant rule excerpts (with file paths) only in the `conventions-maintainability` pass-specific `context`, not the batch's shared context.
82
84
 
83
85
  ## Step 3 — Overview & strengths (`light` reviewer)
84
86
 
85
87
  Write a short **overview** (1–3 short paragraphs) of what the PR does and how, grounded in the diff and PR title/description — enough to understand author intent. Also collect a list of genuine **strengths** (good tests, nice consolidation, correct reuse of helpers, etc.) and any high-level risk areas for the specialist passes. Strengths are part of the output, and understanding intent is what lets you tell an intentional change from a bug.
86
88
 
87
- When `review_subagents` is available, include this as the `overview` pass in the same batch as the Step 5 review passes instead of waiting for a separate sequential call.
89
+ When `review_subagents` is available, include this as the `overview` pass in the same batch as the Step 5 review passes instead of waiting for a separate sequential call. Set `tool_policy: none`; the complete PR metadata and diff are already supplied.
88
90
 
89
91
  ## Step 4 — (reserved)
90
92
 
91
93
  ## Step 5 — Review passes (dispatch by tier, then merge results)
92
94
 
93
- Run these independent passes over the diff, each on its tier reviewer (or inline if subagent tools are unavailable). Give every pass the shared PR context from Step 1 plus the relevant convention-file excerpts from Step 2, and instruct each pass to report issues **at every severity, including nits**, within its own scope. Do **not** ask every pass to audit everything; the objective boundaries below reduce duplicate work while preserving coverage.
95
+ Run these independent passes over the diff, each on its tier reviewer (or inline if subagent tools are unavailable). Give every pass the shared PR metadata and complete diff from Step 1; give only the medium pass the relevant convention-file excerpts from Step 2. Instruct each pass to report issues **at every severity, including nits**, within its own scope. Do **not** ask every pass to audit everything; the objective and tool-policy boundaries below reduce duplicate work while preserving coverage.
94
96
 
95
97
  When `review_subagents` is available, dispatch the Step 3 `overview` pass and these Step 5 passes in one batch with `max_parallel: 4`, then combine the ordered outputs:
96
98
 
97
- 1. **Convention, readability & maintainability pass — `medium` reviewer.** Audit changed lines against the in-scope convention files from Step 2. Flag violations (quote the rule), softer deviations from documented style as nits, naming/dead-code/comment/typo issues, minor duplication, test gaps, and "worth confirming" observations (e.g. "no current callers — confirm intended", "confirm this generated file came from codegen not a hand-edit"). Do not duplicate deep correctness or security analysis unless needed to explain a convention/maintainability issue.
98
- 2. **Bug & correctness pass — `heavy` reviewer.** Hunt for defects in the introduced code: compile/parse failures, logic errors, wrong results, off-by-one, error handling, resource/lifecycle, concurrency, and edge cases. Include lower-severity correctness smells and missing edge cases as P2/P3. Do not duplicate pure style nits covered by the medium pass.
99
- 3. **Security & performance pass — `heavy` reviewer.** Look for security issues (injection, authz, secrets, unsafe deserialization) and performance regressions in the changed code. Note minor ones too. Do not duplicate ordinary correctness/readability findings unless they are security/performance relevant.
99
+ 1. **Convention, readability & maintainability pass — `medium` reviewer, `tool_policy: configured`.** Audit changed lines against the in-scope convention excerpts supplied in this pass's context. Flag violations (quote the rule), softer deviations from documented style as nits, naming/dead-code/comment/typo issues, minor duplication, test gaps, and "worth confirming" observations (e.g. "no current callers — confirm intended", "confirm this generated file came from codegen not a hand-edit"). Use configured tools to inspect surrounding files, tests, callers, or generated-file context when needed; do not modify files or duplicate deep correctness/security analysis unless needed to explain a convention/maintainability issue.
100
+ 2. **Bug & correctness pass — `heavy` reviewer, `tool_policy: configured`.** Hunt for defects in the introduced code: compile/parse failures, logic errors, wrong results, off-by-one, error handling, resource/lifecycle, concurrency, and edge cases. Include lower-severity correctness smells and missing edge cases as P2/P3. Do not duplicate pure style nits covered by the medium pass. Use configured repository-context tools when surrounding files or callers are needed to confirm impact; reviewing remains non-modifying even if the allowlist includes `bash`.
101
+ 3. **Security & performance pass — `heavy` reviewer, `tool_policy: configured`.** Look for security issues (injection, authz, secrets, unsafe deserialization) and performance regressions in the changed code. Note minor ones too. Do not duplicate ordinary correctness/readability findings unless they are security/performance relevant. Use configured tools when repository context is needed to validate a candidate; do not modify files.
100
102
 
101
103
  ## Step 6 — Verification (best-effort, orchestrator-owned, non-destructive)
102
104
 
@@ -0,0 +1,72 @@
1
+ import { describe, expect, test } from "bun:test";
2
+ import {
3
+ appendToolPolicyArgs,
4
+ buildReviewBaseArgs,
5
+ normalizeToolPolicy,
6
+ resolveToolPolicy,
7
+ } from "../lib/pr-review-policy.ts";
8
+
9
+ describe("tool policy resolution", () => {
10
+ test("request override wins over tier config", () => {
11
+ expect(resolveToolPolicy("none", "configured")).toBe("none");
12
+ expect(resolveToolPolicy("configured", "none")).toBe("configured");
13
+ });
14
+
15
+ test("tier config applies when request omits policy", () => {
16
+ expect(resolveToolPolicy(undefined, "none")).toBe("none");
17
+ });
18
+
19
+ test("omission preserves legacy configured behavior", () => {
20
+ expect(resolveToolPolicy(undefined, undefined)).toBe("configured");
21
+ });
22
+
23
+ test("normalization rejects unknown values", () => {
24
+ expect(normalizeToolPolicy("none")).toBe("none");
25
+ expect(normalizeToolPolicy("configured")).toBe("configured");
26
+ expect(normalizeToolPolicy("auto")).toBeUndefined();
27
+ });
28
+ });
29
+
30
+ describe("tool policy argv", () => {
31
+ test("base args isolate explicit review context", () => {
32
+ expect(buildReviewBaseArgs()).toEqual([
33
+ "--mode",
34
+ "json",
35
+ "-p",
36
+ "--no-session",
37
+ "--no-context-files",
38
+ ]);
39
+ });
40
+
41
+ test("none emits explicit --no-tools", () => {
42
+ const args = ["--mode", "json"];
43
+ expect(appendToolPolicyArgs(args, "none", ["read", "bash"])).toEqual([
44
+ "--mode",
45
+ "json",
46
+ "--no-tools",
47
+ ]);
48
+ });
49
+
50
+ test("configured emits the configured allowlist", () => {
51
+ const args = ["--mode", "json"];
52
+ expect(appendToolPolicyArgs(args, "configured", ["read", "grep"])).toEqual([
53
+ "--mode",
54
+ "json",
55
+ "--tools",
56
+ "read,grep",
57
+ ]);
58
+ });
59
+
60
+ test("configured with an empty list preserves legacy omitted-flag behavior", () => {
61
+ const args = ["--mode", "json"];
62
+ expect(appendToolPolicyArgs(args, "configured", [])).toEqual(["--mode", "json"]);
63
+ });
64
+
65
+ test("one resolved policy can be reused across fallback attempts", () => {
66
+ const policy = resolveToolPolicy("none", "configured");
67
+ const first = appendToolPolicyArgs([], policy, ["read"]);
68
+ const fallback = appendToolPolicyArgs([], policy, ["read"]);
69
+ expect(first).toEqual(["--no-tools"]);
70
+ expect(fallback).toEqual(["--no-tools"]);
71
+ });
72
+ });