pi-prompt-template-model-enhanced 0.13.0 → 0.14.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,30 @@
1
1
  # Changelog
2
2
 
3
+ ## [0.14.0](https://github.com/Nabsku/pi-prompt-template-model-enhanced/compare/v0.13.1...v0.14.0) (2026-06-22)
4
+
5
+
6
+ ### Features
7
+
8
+ * add best-of-n preset resolution ([e0f45e6](https://github.com/Nabsku/pi-prompt-template-model-enhanced/commit/e0f45e64490e6874e6693e7b6450954d7f9dde64))
9
+
10
+
11
+ ### Bug Fixes
12
+
13
+ * address best-of-n preset review comments ([0dfdf80](https://github.com/Nabsku/pi-prompt-template-model-enhanced/commit/0dfdf8065c0a241f0786f7271800b1c877b117cc))
14
+ * close preset review edge cases ([df85bf3](https://github.com/Nabsku/pi-prompt-template-model-enhanced/commit/df85bf3d86c2f43bcbec8d9fdb150e2fde37e61f))
15
+ * close preset validation review gaps ([1822061](https://github.com/Nabsku/pi-prompt-template-model-enhanced/commit/1822061ae90ee9b5ee2d7ea222ce9cabc9698e79))
16
+ * harden best-of-n preset overrides ([caa0981](https://github.com/Nabsku/pi-prompt-template-model-enhanced/commit/caa0981936d5e264f5e2dd334d9f0e1abe59aa59))
17
+ * harden project preset resolution ([04cddc9](https://github.com/Nabsku/pi-prompt-template-model-enhanced/commit/04cddc919256b0c40b28a51b67c185a592f899fb))
18
+ * prevent invalid project presets from falling through ([8b0f589](https://github.com/Nabsku/pi-prompt-template-model-enhanced/commit/8b0f5899ca5ebd1350218cd0d87d24af891a9656))
19
+ * validate presets from prompt cwd ([ce97729](https://github.com/Nabsku/pi-prompt-template-model-enhanced/commit/ce9772947bf31abbc1fc4481bdc13bf51385da62))
20
+
21
+ ## [0.13.1](https://github.com/Nabsku/pi-prompt-template-model-enhanced/compare/v0.13.0...v0.13.1) (2026-06-21)
22
+
23
+
24
+ ### Bug Fixes
25
+
26
+ * harden prompt-library parity coverage ([1666691](https://github.com/Nabsku/pi-prompt-template-model-enhanced/commit/16666916eff1757cefa4b743cdbb18ea704b95a5))
27
+
3
28
  ## [0.13.0](https://github.com/Nabsku/pi-prompt-template-model-enhanced/compare/v0.12.0...v0.13.0) (2026-06-21)
4
29
 
5
30
  ### Prompt library
package/README.md CHANGED
@@ -32,6 +32,41 @@ No more manually switching models, copying standard instructions between prompts
32
32
  - **Pi-native TUI**: browse templates and inspect dry-run output interactively in Pi TUI mode without typing template names.
33
33
  - **Execution control**: loops, model rotation, fresh context, boomerang collapse, delegated subagents, chains, and best-of-N compare prompts.
34
34
 
35
+ ## Differences from upstream
36
+
37
+ This package is an enhanced fork of [`pi-prompt-template-model`](https://github.com/nicobailon/pi-prompt-template-model), with compatibility preserved where practical and new workflow features kept opt-in where possible.
38
+
39
+ ### Additions
40
+
41
+ - Prompt includes from shared Markdown partials and prompt-library roots.
42
+ - Multiple skill injection via `skills`, plus constrained wildcard skill selectors.
43
+ - Dry-run previews and a Pi-native TUI picker/inspector for prompt templates.
44
+ - Loop controls, model rotation, fresh context, and boomerang context collapse.
45
+ - Chain templates, deterministic prompt steps, delegated subagents, and parallel delegation helpers.
46
+ - Best-of-N compare prompts with reusable preset catalogs.
47
+ - Richer `/validate-prompts` diagnostics, include graphs, and source summaries.
48
+
49
+ ### Behavior changes and stricter validation
50
+
51
+ - Invalid prompt config is reported and skipped more consistently instead of running with silently degraded behavior.
52
+ - Prompt-library commands have extra trust checks because they can come from project-local reusable libraries.
53
+ - Duplicate prompt and prompt-library precedence is deterministic and reported with diagnostics.
54
+ - Runtime flags are scoped to prompt types that support them; for example, `--preset` only affects compare prompts.
55
+ - Project best-of-N presets require session approval and invalid project presets fail closed instead of falling back to same-named user presets.
56
+
57
+ ### Breaking or migration notes
58
+
59
+ - Compare prompt templates must use nested `bestOfN:` frontmatter. Top-level `workers`, `reviewers`, and `finalApplier` fields are rejected with migration diagnostics.
60
+ - `skills:` must be a list. Use `skill: name` for a single skill.
61
+ - Some fields that upstream accepted loosely are now type-checked before registration, including prompt includes, chain declarations, loop values, cwd paths, compare lineups, and preset catalogs.
62
+ - Best-of-N presets intentionally cannot set execution-policy fields such as `task`, `taskSuffix`, `cwd`, `finalApplier`, `worktree`, dirty/report/commit policy, or other prompt-owned behavior.
63
+
64
+ ### Small tweaks
65
+
66
+ - Better command descriptions and runtime warnings for ignored or unsupported flags.
67
+ - More deterministic model/thinking fallback behavior in dry-run and execution paths.
68
+ - Packaged examples, authoring skill docs, and validation output are kept aligned with shipped behavior.
69
+
35
70
  ## Installation
36
71
 
37
72
  ```bash
@@ -182,7 +217,8 @@ All fields are optional. Templates that don't use any extension features (no `mo
182
217
  | `subagent` | — | Delegate execution to a subagent instead of running in the current session. `true` uses the default `delegate` agent; a string value like `reviewer` targets that specific agent. Requires [pi-subagents](https://github.com/nicobailon/pi-subagents/). |
183
218
  | `inheritContext` | `false` | Only meaningful with `subagent`. When `true`, the subagent receives a fork of the current conversation context instead of starting fresh. |
184
219
  | `parallel` | — | Delegated prompts only. Repeats the same subagent in parallel `N` times. Each copy gets a slot header like `[Parallel subagent 2/3]` prepended to the task. Must be an integer greater than or equal to 2. |
185
- | `bestOfN` | — | Compare templates only. Nested compare authoring block with `workers`, `reviewers`, optional `finalApplier`, and optional `worktree`. Top-level compare fields are not supported in templates. |
220
+ | `bestOfN` | — | Compare templates only. Nested compare authoring block with `workers`, `reviewers`, optional `preset`, optional `finalApplier`, and optional `worktree`. Top-level compare fields are not supported in templates. |
221
+ | `bestOfN.preset` | — | Name of a best-of-N preset from `~/.pi/agent/best-of-n-presets.json` or `<compare-cwd>/.pi/best-of-n-presets.json`. Presets can supply worker/reviewer agents, models, counts, `defaultModel`, and `maxModelCalls`; prompt templates still own task text, `cwd`, final apply, dirty/report/commit policy, and other execution policy. |
186
222
  | `bestOfN.workers` | — | Ordered worker lineup used for the worker phase. Each slot object supports optional `agent`/`subagent`, optional `model`, optional `task`, optional `taskSuffix`, optional `cwd`, and optional `count`. If both `agent` and `subagent` are omitted, the default agent is `delegate`. |
187
223
  | `bestOfN.reviewers` | — | Ordered reviewer lineup used after worker aggregation. Slot shape matches workers. If both `agent` and `subagent` are omitted, the default agent is `reviewer`. |
188
224
  | `bestOfN.finalApplier` | — | Optional single-slot final apply phase that edits the real branch after reviewers. Supports optional `agent`/`subagent`, optional `model`, optional `task`, and optional `taskSuffix`. If both `agent` and `subagent` are omitted, the default agent is `delegate`. `count` and `cwd` are not supported. Requires `bestOfN.worktree: true` at runtime. |
@@ -588,6 +624,7 @@ Prompt-template frontmatter authoring uses `bestOfN:`. Runtime overrides stay on
588
624
  - `--workers=<json-array>` / `--reviewers=<json-array>` replace the corresponding frontmatter lineup.
589
625
  - `--workers-append=<json-array>` / `--reviewers-append=<json-array>` append to the corresponding lineup.
590
626
  - `--final-applier=<json-object-or-one-element-array>` replaces the optional final apply slot.
627
+ - `--preset=<name>` / `--preset <name>` selects a best-of-N preset for compare prompts only; it is ignored on non-compare prompts.
591
628
 
592
629
  Each worker/reviewer JSON array entry must be an object with either `subagent` or `agent`, plus optional `model`, `task`, `taskSuffix`, `cwd`, and `count`. In worker slots, `"subagent": true` maps to `delegate`. In reviewer slots, `"subagent": true` maps to `reviewer`. `--final-applier=` accepts one slot object (or a one-element array) with `subagent`/`agent`, optional `model`, optional `task`, and optional `taskSuffix`; for this final slot, `"subagent": true` maps to `delegate`, and both `count` and `cwd` are not supported.
593
630
 
@@ -615,7 +652,45 @@ Compare prompt templates are authored under `bestOfN:`. Top-level `workers`, `re
615
652
  3. Optional final apply phase: if `finalApplier` is configured, run one delegated apply step on the real compare repo (`compareCwd`) to pick a winner or synthesize/cherry-pick and apply the final patch.
616
653
  4. If all reviewers fail but `finalApplier` exists, the final apply step still runs with fallback context from workers plus reviewer failure summaries.
617
654
 
618
- Worker/reviewer lineups are fully configurable from `bestOfN` frontmatter or runtime overrides, so there is no fixed three-model worker assumption. If a compare prompt omits `bestOfN.workers`, it falls back to one `delegate` worker using the current/main model. If it omits `bestOfN.reviewers`, it falls back to one `reviewer` slot. `bestOfN.finalApplier` is optional, and compare runs reject an effective final applier unless `bestOfN.worktree: true` is set.
655
+ Worker/reviewer lineups are fully configurable from `bestOfN` frontmatter, presets, or runtime overrides, so there is no fixed three-model worker assumption. If a compare prompt omits `bestOfN.workers`, it falls back to one `delegate` worker using the current/main model. If it omits `bestOfN.reviewers`, it falls back to one `reviewer` slot. `bestOfN.finalApplier` is optional, and compare runs reject an effective final applier unless `bestOfN.worktree: true` is set.
656
+
657
+ ### Best-of-N presets
658
+
659
+ Presets keep expensive lineup choices reusable without letting project config own prompt policy. Define them in either place:
660
+
661
+ - User presets: `~/.pi/agent/best-of-n-presets.json`
662
+ - Project presets: `<compare-cwd>/.pi/best-of-n-presets.json`
663
+
664
+ Project presets override user presets of the same name, but running a project preset asks for session approval. Compare prompts that set `cwd`, runtime `--cwd`, or use `parallel-patch-compare-at-path` resolve project presets from the effective compare cwd. `/validate-prompts` mirrors prompt `cwd` where it can be known statically.
665
+
666
+ ```json
667
+ {
668
+ "presets": {
669
+ "quick": {
670
+ "description": "Two cheap workers, one reviewer",
671
+ "defaultModel": "openai-codex/gpt-5.4-mini:low",
672
+ "maxModelCalls": 3,
673
+ "workers": [{ "agent": "delegate", "count": 2 }],
674
+ "reviewers": [{ "agent": "reviewer" }]
675
+ }
676
+ }
677
+ }
678
+ ```
679
+
680
+ Use from a prompt:
681
+
682
+ ```yaml
683
+ bestOfN:
684
+ preset: quick
685
+ ```
686
+
687
+ Or at runtime:
688
+
689
+ ```bash
690
+ /best-of-n --preset quick refactor the parser
691
+ ```
692
+
693
+ Preset slot fields are intentionally limited to `agent`/`subagent`, `model`, and `count`. Presets cannot set `task`, `taskSuffix`, `cwd`, `finalApplier`, `worktree`, dirty/report/commit behavior, or other execution policy. Invalid selected presets fail closed instead of falling back to same-named user presets, and `maxModelCalls` caps the expanded worker + reviewer calls before any subagents start.
619
694
 
620
695
  For same-model best-of-N, use `count: N` on one worker slot:
621
696
 
package/args.ts CHANGED
@@ -21,6 +21,7 @@ export interface SubagentOverrideExtraction {
21
21
  override?: SubagentOverride;
22
22
  cwd?: string;
23
23
  model?: string;
24
+ preset?: string;
24
25
  fork?: boolean;
25
26
  }
26
27
 
@@ -240,10 +241,65 @@ export function extractWorktreeFlag(argsString: string): { args: string; worktre
240
241
  return { args, worktree: found };
241
242
  }
242
243
 
244
+ const LINEUP_OVERRIDE_VALUE_PREFIXES = [
245
+ "--workers=",
246
+ "--workers-append=",
247
+ "--reviewers=",
248
+ "--reviewers-append=",
249
+ "--final-applier=",
250
+ ] as const;
251
+
252
+ function readJsonLikeFlagTokenEnd(argsString: string, tokenStart: number): number {
253
+ const plainEnd = () => {
254
+ let end = tokenStart;
255
+ while (end < argsString.length && !/\s/.test(argsString[end])) end++;
256
+ return end;
257
+ };
258
+ const prefix = LINEUP_OVERRIDE_VALUE_PREFIXES.find((candidate) => argsString.startsWith(candidate, tokenStart));
259
+ if (!prefix) return plainEnd();
260
+ let i = tokenStart + prefix.length;
261
+ if (argsString[i] === '"' || argsString[i] === "'") {
262
+ const quote = argsString[i++];
263
+ while (i < argsString.length) {
264
+ if (argsString[i] === "\\") {
265
+ i += 2;
266
+ continue;
267
+ }
268
+ if (argsString[i] === quote) return i + 1;
269
+ i++;
270
+ }
271
+ return i;
272
+ }
273
+ const opener = argsString[i];
274
+ const closer = opener === "[" ? "]" : opener === "{" ? "}" : undefined;
275
+ if (!closer) return plainEnd();
276
+ let depth = 0;
277
+ let inString = false;
278
+ for (; i < argsString.length; i++) {
279
+ const char = argsString[i];
280
+ if (inString) {
281
+ if (char === "\\") i++;
282
+ else if (char === '"') inString = false;
283
+ continue;
284
+ }
285
+ if (char === '"') {
286
+ inString = true;
287
+ continue;
288
+ }
289
+ if (char === opener) depth++;
290
+ else if (char === closer) {
291
+ depth--;
292
+ if (depth === 0) return i + 1;
293
+ }
294
+ }
295
+ return i;
296
+ }
297
+
243
298
  export function extractSubagentOverride(argsString: string): SubagentOverrideExtraction {
244
299
  let override: SubagentOverride | undefined;
245
300
  let cwdRaw: string | undefined;
246
301
  let modelRaw: string | undefined;
302
+ let presetRaw: string | undefined;
247
303
  let fork = false;
248
304
  const tokensToRemove: Array<{ start: number; end: number }> = [];
249
305
 
@@ -265,7 +321,7 @@ export function extractSubagentOverride(argsString: string): SubagentOverrideExt
265
321
  }
266
322
 
267
323
  const tokenStart = i;
268
- while (i < argsString.length && !/\s/.test(argsString[i])) i++;
324
+ i = readJsonLikeFlagTokenEnd(argsString, tokenStart);
269
325
  const token = argsString.slice(tokenStart, i);
270
326
 
271
327
  if (token === "--subagent") {
@@ -295,6 +351,31 @@ export function extractSubagentOverride(argsString: string): SubagentOverrideExt
295
351
  continue;
296
352
  }
297
353
 
354
+ if (token.startsWith("--preset=")) {
355
+ tokensToRemove.push({ start: tokenStart, end: i });
356
+ const value = token.slice("--preset=".length);
357
+ presetRaw = value || undefined;
358
+ continue;
359
+ }
360
+
361
+ if (token === "--preset") {
362
+ let lookahead = i;
363
+ while (lookahead < argsString.length && /\s/.test(argsString[lookahead])) lookahead++;
364
+ if (lookahead < argsString.length && argsString[lookahead] !== '"' && argsString[lookahead] !== "'") {
365
+ const valueStart = lookahead;
366
+ while (lookahead < argsString.length && !/\s/.test(argsString[lookahead])) lookahead++;
367
+ const value = argsString.slice(valueStart, lookahead);
368
+ if (value && !value.startsWith("--")) {
369
+ tokensToRemove.push({ start: tokenStart, end: i }, { start: valueStart, end: lookahead });
370
+ presetRaw = value;
371
+ i = lookahead;
372
+ continue;
373
+ }
374
+ }
375
+ tokensToRemove.push({ start: tokenStart, end: i });
376
+ continue;
377
+ }
378
+
298
379
  if (token === "--fork") {
299
380
  tokensToRemove.push({ start: tokenStart, end: i });
300
381
  fork = true;
@@ -317,6 +398,7 @@ export function extractSubagentOverride(argsString: string): SubagentOverrideExt
317
398
  ...(override ? { override } : {}),
318
399
  ...(cwdRaw !== undefined ? { cwd: cwdRaw } : {}),
319
400
  ...(modelRaw !== undefined ? { model: modelRaw } : {}),
401
+ ...(presetRaw !== undefined ? { preset: presetRaw } : {}),
320
402
  ...(fork ? { fork: true } : {}),
321
403
  };
322
404
  }
@@ -0,0 +1,256 @@
1
+ import { existsSync, readFileSync } from "node:fs";
2
+ import { homedir } from "node:os";
3
+ import { join, resolve } from "node:path";
4
+ import type { DelegationLineupSlot, PromptLoaderDiagnostic, PromptSource } from "./prompt-loader.js";
5
+
6
+ export interface BestOfNPreset {
7
+ description?: string;
8
+ defaultModel?: string;
9
+ maxModelCalls?: number;
10
+ workers?: DelegationLineupSlot[];
11
+ reviewers?: DelegationLineupSlot[];
12
+ }
13
+
14
+ export interface ResolvedBestOfNPreset extends BestOfNPreset {
15
+ name: string;
16
+ source: PromptSource;
17
+ filePath: string;
18
+ }
19
+
20
+ export interface BestOfNPresetCatalog {
21
+ presets: Map<string, ResolvedBestOfNPreset>;
22
+ invalidPresetNames: Set<string>;
23
+ diagnostics: PromptLoaderDiagnostic[];
24
+ projectFileInvalid: boolean;
25
+ }
26
+
27
+ function createPresetDiagnostic(code: string, filePath: string, source: PromptSource, message: string): PromptLoaderDiagnostic {
28
+ return {
29
+ code,
30
+ filePath,
31
+ source,
32
+ message,
33
+ key: `${code}:${filePath}:${message}`,
34
+ };
35
+ }
36
+
37
+ function isRecord(value: unknown): value is Record<string, unknown> {
38
+ return value !== null && typeof value === "object" && !Array.isArray(value);
39
+ }
40
+
41
+ function isValidModelSelectionSpec(spec: string): boolean {
42
+ if (!spec || spec.includes("*") || /\s/.test(spec)) return false;
43
+ const slashIndex = spec.indexOf("/");
44
+ if (slashIndex === -1) return true;
45
+ if (slashIndex === 0) return false;
46
+ const modelId = spec.slice(slashIndex + 1);
47
+ if (modelId.length === 0) return false;
48
+ if (modelId.split("/").some((segment) => segment.length === 0)) return false;
49
+ return true;
50
+ }
51
+
52
+ function rejectUnsupportedKeys(
53
+ value: Record<string, unknown>,
54
+ allowedKeys: Set<string>,
55
+ presetName: string,
56
+ filePath: string,
57
+ source: PromptSource,
58
+ diagnostics: PromptLoaderDiagnostic[],
59
+ context: string,
60
+ ): boolean {
61
+ const unsupportedKeys = Object.keys(value).filter((key) => !allowedKeys.has(key));
62
+ if (unsupportedKeys.length === 0) return false;
63
+ diagnostics.push(createPresetDiagnostic("invalid-best-of-n-preset", filePath, source, `Ignoring preset ${JSON.stringify(presetName)} in ${filePath}: ${context} has unsupported field(s): ${unsupportedKeys.join(", ")}.`));
64
+ return true;
65
+ }
66
+
67
+ const PRESET_KEYS = new Set(["description", "defaultModel", "maxModelCalls", "workers", "reviewers"]);
68
+ const PRESET_SLOT_KEYS = new Set(["agent", "subagent", "model", "count"]);
69
+
70
+ function normalizePresetSlot(
71
+ value: unknown,
72
+ field: "workers" | "reviewers",
73
+ presetName: string,
74
+ filePath: string,
75
+ source: PromptSource,
76
+ diagnostics: PromptLoaderDiagnostic[],
77
+ index: number,
78
+ ): DelegationLineupSlot | undefined {
79
+ if (!isRecord(value)) {
80
+ diagnostics.push(createPresetDiagnostic("invalid-best-of-n-preset", filePath, source, `Ignoring preset ${JSON.stringify(presetName)} in ${filePath}: ${field} slot ${index + 1} must be an object.`));
81
+ return undefined;
82
+ }
83
+ if (rejectUnsupportedKeys(value, PRESET_SLOT_KEYS, presetName, filePath, source, diagnostics, `${field} slot ${index + 1}`)) {
84
+ return undefined;
85
+ }
86
+ if (value.agent !== undefined && value.subagent !== undefined) {
87
+ diagnostics.push(createPresetDiagnostic("invalid-best-of-n-preset", filePath, source, `Ignoring preset ${JSON.stringify(presetName)} in ${filePath}: ${field} slot ${index + 1} cannot combine "agent" and "subagent".`));
88
+ return undefined;
89
+ }
90
+
91
+ let agent: string | undefined;
92
+ if (typeof value.agent === "string" && value.agent.trim()) {
93
+ agent = value.agent.trim();
94
+ } else if (value.agent !== undefined) {
95
+ diagnostics.push(createPresetDiagnostic("invalid-best-of-n-preset", filePath, source, `Ignoring preset ${JSON.stringify(presetName)} in ${filePath}: ${field} slot ${index + 1} requires a non-empty string "agent".`));
96
+ return undefined;
97
+ }
98
+ if (!agent && value.subagent !== undefined) {
99
+ if (value.subagent === true) {
100
+ agent = field === "reviewers" ? "reviewer" : "delegate";
101
+ } else if (typeof value.subagent === "string" && value.subagent.trim()) {
102
+ agent = value.subagent.trim();
103
+ } else {
104
+ diagnostics.push(createPresetDiagnostic("invalid-best-of-n-preset", filePath, source, `Ignoring preset ${JSON.stringify(presetName)} in ${filePath}: ${field} slot ${index + 1} requires "subagent" to be true or a non-empty string.`));
105
+ return undefined;
106
+ }
107
+ }
108
+ if (!agent) agent = field === "reviewers" ? "reviewer" : "delegate";
109
+
110
+ const slot: DelegationLineupSlot = { agent };
111
+ if (value.model !== undefined) {
112
+ if (typeof value.model !== "string" || !value.model.trim() || !isValidModelSelectionSpec(value.model.trim())) {
113
+ diagnostics.push(createPresetDiagnostic("invalid-best-of-n-preset", filePath, source, `Ignoring preset ${JSON.stringify(presetName)} in ${filePath}: ${field} slot ${index + 1} has an invalid "model".`));
114
+ return undefined;
115
+ }
116
+ slot.model = value.model.trim();
117
+ }
118
+ if (value.count !== undefined) {
119
+ let count: number | undefined;
120
+ if (typeof value.count === "number") count = value.count;
121
+ else if (typeof value.count === "string" && /^\d+$/.test(value.count.trim())) count = parseInt(value.count.trim(), 10);
122
+ if (count === undefined || !Number.isInteger(count) || count < 1) {
123
+ diagnostics.push(createPresetDiagnostic("invalid-best-of-n-preset", filePath, source, `Ignoring preset ${JSON.stringify(presetName)} in ${filePath}: ${field} slot ${index + 1} "count" must be an integer greater than or equal to 1.`));
124
+ return undefined;
125
+ }
126
+ slot.count = count;
127
+ }
128
+ return slot;
129
+ }
130
+
131
+ function normalizePresetLineup(
132
+ value: unknown,
133
+ field: "workers" | "reviewers",
134
+ presetName: string,
135
+ filePath: string,
136
+ source: PromptSource,
137
+ diagnostics: PromptLoaderDiagnostic[],
138
+ ): DelegationLineupSlot[] | undefined {
139
+ if (value === undefined) return undefined;
140
+ if (!Array.isArray(value) || value.length === 0) {
141
+ diagnostics.push(createPresetDiagnostic("invalid-best-of-n-preset", filePath, source, `Ignoring preset ${JSON.stringify(presetName)} in ${filePath}: ${field} must be a non-empty array.`));
142
+ return undefined;
143
+ }
144
+ const slots: DelegationLineupSlot[] = [];
145
+ for (let i = 0; i < value.length; i++) {
146
+ const slot = normalizePresetSlot(value[i], field, presetName, filePath, source, diagnostics, i);
147
+ if (!slot) return undefined;
148
+ slots.push(slot);
149
+ }
150
+ return slots;
151
+ }
152
+
153
+ function normalizePreset(name: string, value: unknown, filePath: string, source: PromptSource, diagnostics: PromptLoaderDiagnostic[]): BestOfNPreset | undefined {
154
+ if (!/^[A-Za-z0-9][A-Za-z0-9._-]*$/.test(name)) {
155
+ diagnostics.push(createPresetDiagnostic("invalid-best-of-n-preset", filePath, source, `Ignoring invalid best-of-N preset name ${JSON.stringify(name)} in ${filePath}.`));
156
+ return undefined;
157
+ }
158
+ if (!isRecord(value)) {
159
+ diagnostics.push(createPresetDiagnostic("invalid-best-of-n-preset", filePath, source, `Ignoring preset ${JSON.stringify(name)} in ${filePath}: preset must be an object.`));
160
+ return undefined;
161
+ }
162
+ if (rejectUnsupportedKeys(value, PRESET_KEYS, name, filePath, source, diagnostics, "preset")) {
163
+ return undefined;
164
+ }
165
+ const preset: BestOfNPreset = {};
166
+ if (value.description !== undefined) {
167
+ if (typeof value.description !== "string") {
168
+ diagnostics.push(createPresetDiagnostic("invalid-best-of-n-preset", filePath, source, `Ignoring preset ${JSON.stringify(name)} in ${filePath}: description must be a string.`));
169
+ return undefined;
170
+ }
171
+ const description = value.description.trim();
172
+ if (description) preset.description = description;
173
+ }
174
+ if (value.defaultModel !== undefined) {
175
+ if (typeof value.defaultModel !== "string" || !value.defaultModel.trim() || !isValidModelSelectionSpec(value.defaultModel.trim())) {
176
+ diagnostics.push(createPresetDiagnostic("invalid-best-of-n-preset", filePath, source, `Ignoring preset ${JSON.stringify(name)} in ${filePath}: defaultModel must be a valid model spec string.`));
177
+ return undefined;
178
+ }
179
+ preset.defaultModel = value.defaultModel.trim();
180
+ }
181
+ if (value.maxModelCalls !== undefined) {
182
+ if (typeof value.maxModelCalls !== "number" || !Number.isInteger(value.maxModelCalls) || value.maxModelCalls < 1) {
183
+ diagnostics.push(createPresetDiagnostic("invalid-best-of-n-preset", filePath, source, `Ignoring preset ${JSON.stringify(name)} in ${filePath}: maxModelCalls must be an integer greater than or equal to 1.`));
184
+ return undefined;
185
+ }
186
+ preset.maxModelCalls = value.maxModelCalls;
187
+ }
188
+ const workers = normalizePresetLineup(value.workers, "workers", name, filePath, source, diagnostics);
189
+ if (value.workers !== undefined && !workers) return undefined;
190
+ const reviewers = normalizePresetLineup(value.reviewers, "reviewers", name, filePath, source, diagnostics);
191
+ if (value.reviewers !== undefined && !reviewers) return undefined;
192
+ if (workers) preset.workers = workers;
193
+ if (reviewers) preset.reviewers = reviewers;
194
+ if (!preset.workers && !preset.reviewers && value.workers === undefined && value.reviewers === undefined) {
195
+ diagnostics.push(createPresetDiagnostic("invalid-best-of-n-preset", filePath, source, `Ignoring preset ${JSON.stringify(name)} in ${filePath}: expected workers or reviewers.`));
196
+ return undefined;
197
+ }
198
+ if (!preset.workers && !preset.reviewers) return undefined;
199
+ return preset;
200
+ }
201
+
202
+ function readPresetFile(filePath: string, source: PromptSource): BestOfNPresetCatalog {
203
+ const presets = new Map<string, ResolvedBestOfNPreset>();
204
+ const invalidPresetNames = new Set<string>();
205
+ const diagnostics: PromptLoaderDiagnostic[] = [];
206
+ if (!existsSync(filePath)) return { presets, invalidPresetNames, diagnostics, projectFileInvalid: false };
207
+ let projectFileInvalid = false;
208
+ let parsed: unknown;
209
+ try {
210
+ parsed = JSON.parse(readFileSync(filePath, "utf-8"));
211
+ } catch (error) {
212
+ projectFileInvalid = source === "project";
213
+ diagnostics.push(createPresetDiagnostic("invalid-best-of-n-presets-file", filePath, source, `Skipping best-of-N presets file ${filePath}: ${error instanceof Error ? error.message : String(error)}.`));
214
+ return { presets, invalidPresetNames, diagnostics, projectFileInvalid };
215
+ }
216
+ if (!isRecord(parsed) || !isRecord(parsed.presets)) {
217
+ projectFileInvalid = source === "project";
218
+ diagnostics.push(createPresetDiagnostic("invalid-best-of-n-presets-file", filePath, source, `Skipping best-of-N presets file ${filePath}: expected top-level object with a "presets" object.`));
219
+ return { presets, invalidPresetNames, diagnostics, projectFileInvalid };
220
+ }
221
+ for (const [name, rawPreset] of Object.entries(parsed.presets)) {
222
+ const preset = normalizePreset(name, rawPreset, filePath, source, diagnostics);
223
+ if (!preset) {
224
+ invalidPresetNames.add(name);
225
+ continue;
226
+ }
227
+ presets.set(name, { name, source, filePath, ...preset });
228
+ }
229
+ return { presets, invalidPresetNames, diagnostics, projectFileInvalid };
230
+ }
231
+
232
+ export function getBestOfNPresetPaths(cwd: string): { user: string; project: string } {
233
+ return {
234
+ user: join(homedir(), ".pi", "agent", "best-of-n-presets.json"),
235
+ project: resolve(cwd, ".pi", "best-of-n-presets.json"),
236
+ };
237
+ }
238
+
239
+ export function loadBestOfNPresetCatalog(cwd: string): BestOfNPresetCatalog {
240
+ const paths = getBestOfNPresetPaths(cwd);
241
+ const user = readPresetFile(paths.user, "user");
242
+ const project = readPresetFile(paths.project, "project");
243
+ const presets = project.projectFileInvalid ? new Map(project.presets) : new Map([...user.presets, ...project.presets]);
244
+ for (const name of project.invalidPresetNames) presets.delete(name);
245
+ return {
246
+ presets,
247
+ invalidPresetNames: new Set([...user.invalidPresetNames, ...project.invalidPresetNames]),
248
+ diagnostics: [...user.diagnostics, ...project.diagnostics],
249
+ projectFileInvalid: project.projectFileInvalid,
250
+ };
251
+ }
252
+
253
+ export function applyPresetDefaultModel(slots: DelegationLineupSlot[] | undefined, defaultModel: string | undefined): DelegationLineupSlot[] | undefined {
254
+ if (!slots || !defaultModel) return slots?.map((slot) => ({ ...slot }));
255
+ return slots.map((slot) => ({ ...slot, model: slot.model ?? defaultModel }));
256
+ }
package/index.ts CHANGED
@@ -15,6 +15,7 @@ import {
15
15
  type LineupOverrideAction,
16
16
  type SubagentOverride,
17
17
  } from "./args.js";
18
+ import { loadBestOfNPresetCatalog, applyPresetDefaultModel, type ResolvedBestOfNPreset } from "./best-of-n-presets.js";
18
19
  import { parseChainSteps, parseChainDeclaration, type ChainStep, type ChainStepOrParallel, type ParallelChainStep } from "./chain-parser.js";
19
20
  import { generateBoomerangSummary, generateChainStepSummary, generateIterationSummary, didIterationMakeChanges, getIterationEntries, wasIterationAborted } from "./loop-utils.js";
20
21
  import { selectModelCandidate } from "./model-selection.js";
@@ -130,6 +131,7 @@ export default function promptModelExtension(pi: ExtensionAPI) {
130
131
  let lastDiagnostics = "";
131
132
  let storedCommandCtx: ExtensionCommandContext | null = null;
132
133
  const approvedProjectPromptLibraryCwds = new Set<string>();
134
+ const approvedProjectPresetCwds = new Set<string>();
133
135
  const UNLIMITED_LOOP_CAP = 999;
134
136
 
135
137
  const toolManager = createToolManager(pi, {
@@ -194,6 +196,49 @@ export default function promptModelExtension(pi: ExtensionAPI) {
194
196
  return true;
195
197
  }
196
198
 
199
+ async function ensureProjectPresetApproved(preset: ResolvedBestOfNPreset, ctx: ExtensionCommandContext, catalogCwd: string): Promise<boolean> {
200
+ if (preset.source !== "project") return true;
201
+ const cwdKey = resolvePath(catalogCwd);
202
+ if (approvedProjectPresetCwds.has(cwdKey)) return true;
203
+ const message = `Best-of-N preset \`${preset.name}\` is loaded from ${preset.filePath}. Approve project best-of-N presets for this session?`;
204
+ if (!ctx.hasUI || typeof (ctx.ui as { confirm?: unknown }).confirm !== "function") {
205
+ notify(ctx, `${message} Run in an interactive UI session and approve it, or move trusted presets to ~/.pi/agent/best-of-n-presets.json.`, "error");
206
+ return false;
207
+ }
208
+ const approved = await ctx.ui.confirm("Approve project best-of-N preset", message, { timeout: 30_000 });
209
+ if (!approved) {
210
+ notify(ctx, `Project best-of-N preset \`${preset.name}\` was not approved.`, "warning");
211
+ return false;
212
+ }
213
+ approvedProjectPresetCwds.add(cwdKey);
214
+ return true;
215
+ }
216
+
217
+ async function resolveBestOfNPresetLineup(
218
+ prompt: PromptWithModel,
219
+ runtimePreset: string | undefined,
220
+ ctx: ExtensionCommandContext,
221
+ catalogCwd: string,
222
+ ): Promise<{ workers?: DelegationLineupSlot[]; reviewers?: DelegationLineupSlot[]; maxModelCalls?: number } | undefined> {
223
+ const presetName = runtimePreset ?? prompt.preset;
224
+ if (!presetName) return {};
225
+ const catalog = loadBestOfNPresetCatalog(catalogCwd);
226
+ for (const diagnostic of catalog.diagnostics) {
227
+ notify(ctx, diagnostic.message, "warning");
228
+ }
229
+ const preset = catalog.presets.get(presetName);
230
+ if (!preset) {
231
+ notify(ctx, `Best-of-N preset \`${presetName}\` was not found. Define it in ~/.pi/agent/best-of-n-presets.json or .pi/best-of-n-presets.json.`, "error");
232
+ return undefined;
233
+ }
234
+ if (!(await ensureProjectPresetApproved(preset, ctx, catalogCwd))) return undefined;
235
+ return {
236
+ workers: applyPresetDefaultModel(preset.workers, preset.defaultModel),
237
+ reviewers: applyPresetDefaultModel(preset.reviewers, preset.defaultModel),
238
+ maxModelCalls: preset.maxModelCalls,
239
+ };
240
+ }
241
+
197
242
  pi.registerMessageRenderer<SkillLoadedDetails>("skill-loaded", renderSkillLoaded);
198
243
  pi.registerMessageRenderer(PROMPT_TEMPLATE_SUBAGENT_MESSAGE_TYPE, renderDelegatedSubagentResult);
199
244
  pi.registerMessageRenderer(PROMPT_TEMPLATE_DETERMINISTIC_MESSAGE_TYPE, renderDeterministicResult);
@@ -519,6 +564,12 @@ export default function promptModelExtension(pi: ExtensionAPI) {
519
564
  return slots?.map((slot) => ({ ...slot }));
520
565
  }
521
566
 
567
+ function expandedLineupCallCount(slots: DelegationLineupSlot[]): number {
568
+ let total = 0;
569
+ for (const slot of slots) total += slot.count ?? 1;
570
+ return total;
571
+ }
572
+
522
573
  function expandLineupCounts(slots: DelegationLineupSlot[]): DelegationLineupSlot[] {
523
574
  const expanded: DelegationLineupSlot[] = [];
524
575
  for (const slot of slots) {
@@ -729,7 +780,7 @@ export default function promptModelExtension(pi: ExtensionAPI) {
729
780
  args: string,
730
781
  ctx: ExtensionCommandContext,
731
782
  currentModel: Model<any> | undefined,
732
- runtime: { cwd?: string; model?: string; subagentOverride?: SubagentOverride; fork?: boolean },
783
+ runtime: { cwd?: string; model?: string; preset?: string; subagentOverride?: SubagentOverride; fork?: boolean },
733
784
  ) {
734
785
  if (!(await ensureProjectPromptLibraryApproved(prompt, ctx))) return;
735
786
 
@@ -782,24 +833,30 @@ export default function promptModelExtension(pi: ExtensionAPI) {
782
833
  }
783
834
  const sharedTask = rendered.content;
784
835
 
785
- const requestedWorkers = applyLineupActions(prompt.workers, lineupExtraction.actions, "workers") ?? [];
786
- const requestedReviewers = applyLineupActions(prompt.reviewers, lineupExtraction.actions, "reviewers") ?? [];
836
+ const presetLineup = await resolveBestOfNPresetLineup(prompt, runtime.preset, ctx, compareCwd);
837
+ if (!presetLineup) return;
838
+ const requestedWorkers = applyLineupActions(presetLineup.workers ?? prompt.workers, lineupExtraction.actions, "workers") ?? [];
839
+ const requestedReviewers = applyLineupActions(presetLineup.reviewers ?? prompt.reviewers, lineupExtraction.actions, "reviewers") ?? [];
787
840
  const requestedFinalApplier = applyFinalApplierAction(prompt.finalApplier, lineupExtraction.actions);
788
841
  if (requestedFinalApplier && prompt.worktree !== true) {
789
842
  notify(ctx, "Compare prompts with finalApplier require worktree: true.", "error");
790
843
  return;
791
844
  }
792
845
 
793
- const workerSlots = expandLineupCounts(
794
- requestedWorkers.length > 0
795
- ? requestedWorkers
796
- : [{ agent: DEFAULT_SUBAGENT_NAME }],
797
- );
798
- const reviewerSlots = expandLineupCounts(
799
- requestedReviewers.length > 0
800
- ? requestedReviewers
801
- : [{ agent: "reviewer" }],
802
- );
846
+ const effectiveWorkerSlots = requestedWorkers.length > 0
847
+ ? requestedWorkers
848
+ : [{ agent: DEFAULT_SUBAGENT_NAME }];
849
+ const effectiveReviewerSlots = requestedReviewers.length > 0
850
+ ? requestedReviewers
851
+ : [{ agent: "reviewer" }];
852
+ const presetModelCalls = expandedLineupCallCount(effectiveWorkerSlots) + expandedLineupCallCount(effectiveReviewerSlots);
853
+ if (presetLineup.maxModelCalls !== undefined && presetModelCalls > presetLineup.maxModelCalls) {
854
+ notify(ctx, `Best-of-N preset model-call cap exceeded: requested ${presetModelCalls} worker/reviewer call(s), but preset allows ${presetLineup.maxModelCalls}.`, "error");
855
+ return;
856
+ }
857
+
858
+ const workerSlots = expandLineupCounts(effectiveWorkerSlots);
859
+ const reviewerSlots = expandLineupCounts(effectiveReviewerSlots);
803
860
 
804
861
  const normalizedWorkers = normalizeLineupCwds(workerSlots, compareCwd, ctx);
805
862
  if (!normalizedWorkers) return;
@@ -1499,7 +1556,7 @@ export default function promptModelExtension(pi: ExtensionAPI) {
1499
1556
 
1500
1557
  function getDryRunUnsupportedReason(prompt: PromptWithModel): string | undefined {
1501
1558
  if (prompt.chain) return DRY_RUN_CHAIN_UNSUPPORTED;
1502
- if (prompt.workers !== undefined || prompt.reviewers !== undefined || prompt.finalApplier !== undefined) return DRY_RUN_COMPARE_UNSUPPORTED;
1559
+ if (prompt.workers !== undefined || prompt.reviewers !== undefined || prompt.finalApplier !== undefined || prompt.preset !== undefined) return DRY_RUN_COMPARE_UNSUPPORTED;
1503
1560
  if (prompt.deterministic) return DRY_RUN_DETERMINISTIC_UNSUPPORTED;
1504
1561
  return undefined;
1505
1562
  }
@@ -1659,7 +1716,7 @@ export default function promptModelExtension(pi: ExtensionAPI) {
1659
1716
  }
1660
1717
  }
1661
1718
 
1662
- const hasCompareLineup = prompt.workers !== undefined || prompt.reviewers !== undefined || prompt.finalApplier !== undefined;
1719
+ const hasCompareLineup = prompt.workers !== undefined || prompt.reviewers !== undefined || prompt.finalApplier !== undefined || prompt.preset !== undefined;
1663
1720
  if (hasCompareLineup) {
1664
1721
  await runComparePrompt(
1665
1722
  name,
@@ -1670,6 +1727,7 @@ export default function promptModelExtension(pi: ExtensionAPI) {
1670
1727
  {
1671
1728
  cwd: runtimeCwd,
1672
1729
  model: subagent.model,
1730
+ preset: subagent.preset,
1673
1731
  subagentOverride: subagent.override,
1674
1732
  fork: subagent.fork,
1675
1733
  },
@@ -1677,6 +1735,10 @@ export default function promptModelExtension(pi: ExtensionAPI) {
1677
1735
  return;
1678
1736
  }
1679
1737
 
1738
+ if (subagent.preset) {
1739
+ notify(ctx, `--preset is only supported on compare prompts (ignored)`, "warning");
1740
+ }
1741
+
1680
1742
  if (prompt.chain) {
1681
1743
  if (subagent.model) notify(ctx, `--model is not supported on chain prompts (ignored)`, "warning");
1682
1744
  if (subagent.fork) notify(ctx, `--fork is not supported on chain prompts (ignored)`, "warning");
@@ -1791,6 +1853,7 @@ export default function promptModelExtension(pi: ExtensionAPI) {
1791
1853
  function resetSessionScopedState(ctx: ExtensionContext) {
1792
1854
  storedCommandCtx = null;
1793
1855
  approvedProjectPromptLibraryCwds.clear();
1856
+ approvedProjectPresetCwds.clear();
1794
1857
  pendingSkillMessage = undefined;
1795
1858
  previousModel = undefined;
1796
1859
  previousThinking = undefined;
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-prompt-template-model-enhanced",
3
- "version": "0.13.0",
3
+ "version": "0.14.0",
4
4
  "type": "module",
5
5
  "description": "Prompt template model selector extension for pi coding agent",
6
6
  "author": "Nico Bailon",
@@ -27,6 +27,7 @@
27
27
  "files": [
28
28
  "index.ts",
29
29
  "args.ts",
30
+ "best-of-n-presets.ts",
30
31
  "chain-parser.ts",
31
32
  "loop-utils.ts",
32
33
  "model-selection.ts",
package/prompt-dry-run.ts CHANGED
@@ -209,7 +209,7 @@ function errorResult(
209
209
  }
210
210
 
211
211
  function hasCompareLineup(prompt: PromptWithModel): boolean {
212
- return prompt.workers !== undefined || prompt.reviewers !== undefined || prompt.finalApplier !== undefined;
212
+ return prompt.workers !== undefined || prompt.reviewers !== undefined || prompt.finalApplier !== undefined || prompt.preset !== undefined;
213
213
  }
214
214
 
215
215
  function shouldDelegatePrompt(prompt: Pick<PromptWithModel, "subagent">, override?: SubagentOverride): boolean {
package/prompt-loader.ts CHANGED
@@ -104,6 +104,7 @@ export interface PromptWithModel {
104
104
  workers?: DelegationLineupSlot[];
105
105
  reviewers?: DelegationLineupSlot[];
106
106
  finalApplier?: DelegationLineupSlot;
107
+ preset?: string;
107
108
  source: PromptSource;
108
109
  rootKind: PromptRootKind;
109
110
  subdir?: string;
@@ -1884,9 +1885,11 @@ function loadPromptsWithModelFromDir(
1884
1885
  const workers = normalizeLineup(hasBestOfN ? bestOfN?.workers : undefined, "workers", fullPath, source, diagnostics);
1885
1886
  const reviewers = normalizeLineup(hasBestOfN ? bestOfN?.reviewers : undefined, "reviewers", fullPath, source, diagnostics);
1886
1887
  const finalApplier = normalizeFinalApplier(hasBestOfN ? bestOfN?.finalApplier : undefined, fullPath, source, diagnostics);
1888
+ const preset = normalizeStringField("best-of-n-preset", hasBestOfN ? bestOfN?.preset : undefined, fullPath, source, diagnostics);
1887
1889
  let safeWorkers = workers;
1888
1890
  let safeReviewers = reviewers;
1889
1891
  let safeFinalApplier = finalApplier;
1892
+ let safePreset = preset;
1890
1893
  if (chain && subagent !== undefined) {
1891
1894
  diagnostics.push(
1892
1895
  createDiagnostic(
@@ -1909,7 +1912,7 @@ function loadPromptsWithModelFromDir(
1909
1912
  );
1910
1913
  deterministic = undefined;
1911
1914
  }
1912
- if (chain && (safeWorkers !== undefined || safeReviewers !== undefined || safeFinalApplier !== undefined)) {
1915
+ if (chain && (safeWorkers !== undefined || safeReviewers !== undefined || safeFinalApplier !== undefined || safePreset !== undefined)) {
1913
1916
  diagnostics.push(
1914
1917
  createDiagnostic(
1915
1918
  "invalid-lineup-chain",
@@ -1921,8 +1924,9 @@ function loadPromptsWithModelFromDir(
1921
1924
  safeWorkers = undefined;
1922
1925
  safeReviewers = undefined;
1923
1926
  safeFinalApplier = undefined;
1927
+ safePreset = undefined;
1924
1928
  }
1925
- if (subagent !== undefined && (safeWorkers !== undefined || safeReviewers !== undefined || safeFinalApplier !== undefined)) {
1929
+ if (subagent !== undefined && (safeWorkers !== undefined || safeReviewers !== undefined || safeFinalApplier !== undefined || safePreset !== undefined)) {
1926
1930
  diagnostics.push(
1927
1931
  createDiagnostic(
1928
1932
  "invalid-lineup-subagent",
@@ -1934,6 +1938,7 @@ function loadPromptsWithModelFromDir(
1934
1938
  safeWorkers = undefined;
1935
1939
  safeReviewers = undefined;
1936
1940
  safeFinalApplier = undefined;
1941
+ safePreset = undefined;
1937
1942
  }
1938
1943
  if (subagent !== undefined && deterministic !== undefined) {
1939
1944
  diagnostics.push(
@@ -1979,7 +1984,7 @@ function loadPromptsWithModelFromDir(
1979
1984
  );
1980
1985
  safeParallel = undefined;
1981
1986
  }
1982
- if (safeParallel !== undefined && (safeWorkers !== undefined || safeReviewers !== undefined || safeFinalApplier !== undefined)) {
1987
+ if (safeParallel !== undefined && (safeWorkers !== undefined || safeReviewers !== undefined || safeFinalApplier !== undefined || safePreset !== undefined)) {
1983
1988
  diagnostics.push(
1984
1989
  createDiagnostic(
1985
1990
  "invalid-lineup-parallel",
@@ -1991,6 +1996,7 @@ function loadPromptsWithModelFromDir(
1991
1996
  safeWorkers = undefined;
1992
1997
  safeReviewers = undefined;
1993
1998
  safeFinalApplier = undefined;
1999
+ safePreset = undefined;
1994
2000
  }
1995
2001
  if (safeParallel !== undefined && deterministic !== undefined) {
1996
2002
  diagnostics.push(
@@ -2003,7 +2009,7 @@ function loadPromptsWithModelFromDir(
2003
2009
  );
2004
2010
  deterministic = undefined;
2005
2011
  }
2006
- const hasLineup = safeWorkers !== undefined || safeReviewers !== undefined || safeFinalApplier !== undefined;
2012
+ const hasLineup = safeWorkers !== undefined || safeReviewers !== undefined || safeFinalApplier !== undefined || safePreset !== undefined;
2007
2013
  if (!hasBestOfN && hasLegacyCompareFields) {
2008
2014
  diagnostics.push(
2009
2015
  createDiagnostic(
@@ -2223,6 +2229,7 @@ function loadPromptsWithModelFromDir(
2223
2229
  workers: safeWorkers,
2224
2230
  reviewers: safeReviewers,
2225
2231
  finalApplier: safeFinalApplier,
2232
+ preset: safePreset,
2226
2233
  source,
2227
2234
  rootKind,
2228
2235
  subdir: subdir || undefined,
@@ -1,6 +1,8 @@
1
+ import { resolve as resolvePath } from "node:path";
2
+ import { loadBestOfNPresetCatalog, type ResolvedBestOfNPreset } from "./best-of-n-presets.js";
1
3
  import { parseChainDeclaration, type ChainStep, type ChainStepOrParallel } from "./chain-parser.js";
2
4
  import { collectPromptIncludeGraphs, type PromptIncludeGraph, type PromptIncludeGraphEdge, type PromptIncludeGraphNode } from "./prompt-includes.js";
3
- import { collectPromptSourceRecords, discoverFilesystemSkills, loadPromptsWithModel, readSkillContent, resolveSkillPath, type PromptLoaderDiagnostic, type PromptSource, type PromptSourceRecord } from "./prompt-loader.js";
5
+ import { collectPromptSourceRecords, discoverFilesystemSkills, loadPromptsWithModel, readSkillContent, resolveSkillPath, type PromptLoaderDiagnostic, type PromptSource, type PromptSourceRecord, type PromptWithModel } from "./prompt-loader.js";
4
6
 
5
7
  export interface RegisteredPromptSkill {
6
8
  skillName: string;
@@ -314,8 +316,57 @@ function validatePromptChains(cwd: string, result: PromptValidationResult, promp
314
316
  }
315
317
  }
316
318
 
319
+ function expandedSlotCount(slots: Array<{ count?: number }> | undefined): number {
320
+ return (slots ?? []).reduce((total, slot) => total + (slot.count ?? 1), 0);
321
+ }
322
+
323
+ function presetExpandedCallCount(preset: ResolvedBestOfNPreset, prompt: PromptWithModel): number {
324
+ const workerSlots = preset.workers ?? prompt.workers;
325
+ const reviewerSlots = preset.reviewers ?? prompt.reviewers;
326
+ return (workerSlots && workerSlots.length > 0 ? expandedSlotCount(workerSlots) : 1)
327
+ + (reviewerSlots && reviewerSlots.length > 0 ? expandedSlotCount(reviewerSlots) : 1);
328
+ }
329
+
317
330
  function validateComparePrompts(cwd: string, result: PromptValidationResult, prompts: ReturnType<typeof loadPromptsWithModel>["prompts"]) {
331
+ const catalogByCwd = new Map<string, ReturnType<typeof loadBestOfNPresetCatalog>>();
332
+ function catalogFor(catalogCwd: string) {
333
+ const key = resolvePath(catalogCwd);
334
+ let catalog = catalogByCwd.get(key);
335
+ if (!catalog) {
336
+ catalog = loadBestOfNPresetCatalog(catalogCwd);
337
+ catalogByCwd.set(key, catalog);
338
+ result.diagnostics.push(...catalog.diagnostics);
339
+ }
340
+ return catalog;
341
+ }
318
342
  for (const prompt of prompts.values()) {
343
+ if (prompt.preset) {
344
+ const presetCatalog = catalogFor(prompt.cwd ?? cwd);
345
+ const preset = presetCatalog.presets.get(prompt.preset);
346
+ if (!preset) {
347
+ result.diagnostics.push(
348
+ createValidationDiagnostic(
349
+ "best-of-n-preset-not-found",
350
+ prompt.filePath,
351
+ prompt.source,
352
+ `Prompt template ${prompt.filePath} references missing best-of-N preset ${JSON.stringify(prompt.preset)}.`,
353
+ ),
354
+ );
355
+ } else if (preset.maxModelCalls !== undefined) {
356
+ const modelCalls = presetExpandedCallCount(preset, prompt);
357
+ if (modelCalls > preset.maxModelCalls) {
358
+ result.diagnostics.push(
359
+ createValidationDiagnostic(
360
+ "best-of-n-preset-cap-exceeded",
361
+ prompt.filePath,
362
+ prompt.source,
363
+ `Prompt template ${prompt.filePath} references best-of-N preset ${JSON.stringify(prompt.preset)}, but its expanded worker/reviewer calls (${modelCalls}) exceed maxModelCalls (${preset.maxModelCalls}).`,
364
+ ),
365
+ );
366
+ }
367
+ }
368
+ }
369
+
319
370
  if (prompt.finalApplier && prompt.worktree !== true) {
320
371
  result.diagnostics.push(
321
372
  createValidationDiagnostic(
@@ -153,18 +153,29 @@ Supports exact IDs, `provider/model-id` pairs, wildcards (`anthropic/*`), and co
153
153
 
154
154
  ## Best-of-N Compare
155
155
 
156
- Run multiple workers, aggregate with reviewers, optionally apply final changes:
156
+ Run multiple workers, aggregate with reviewers, optionally apply final changes. Put workflow policy in the prompt and reusable lineup choices in optional presets:
157
+
158
+ ```json
159
+ {
160
+ "presets": {
161
+ "quick": {
162
+ "defaultModel": "openai-codex/gpt-5.4-mini:low",
163
+ "maxModelCalls": 3,
164
+ "workers": [{ "agent": "delegate", "count": 2 }],
165
+ "reviewers": [{ "agent": "reviewer" }]
166
+ }
167
+ }
168
+ }
169
+ ```
170
+
171
+ Preset files live at `~/.pi/agent/best-of-n-presets.json` and `<compare-cwd>/.pi/best-of-n-presets.json`. Project presets override user presets after approval. Preset slots only support `agent`/`subagent`, `model`, and `count`; keep `task`, `taskSuffix`, `cwd`, `finalApplier`, `worktree`, and dirty/report/commit policy in prompt templates.
157
172
 
158
173
  ```yaml
159
174
  ---
160
175
  description: Best-of-N code review
161
176
  bestOfN:
177
+ preset: quick
162
178
  worktree: true # required if using finalApplier
163
- workers:
164
- - model: openai-codex/gpt-5.4-mini:low
165
- count: 2
166
- reviewers:
167
- - model: anthropic/claude-sonnet-4-20250514:medium
168
179
  finalApplier:
169
180
  agent: delegate
170
181
  model: anthropic/claude-sonnet-4-20250514:high
@@ -185,5 +196,6 @@ Override frontmatter at invocation:
185
196
  - `--cwd=/absolute/path` — working directory override when the prompt supports `cwd`
186
197
  - `--chain-context` — pass summaries to later delegated chain steps
187
198
  - `--worktree` — use git worktrees for parallel delegated work
199
+ - `--preset=<name>` / `--preset <name>` — select a best-of-N preset for compare prompts only
188
200
 
189
201
  When stuck, check `README.md` and `examples/best-of-n.md` in this extension.