pi-gauntlet 5.11.0 → 5.12.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,13 @@
1
1
  # Changelog
2
2
 
3
+ ## v5.12.1 - 2026-09-19
4
+
5
+ - Fixed: `gauntlet-telemetry-salvage` and `gauntlet-performance` no longer crash with `ERR_UNSUPPORTED_NODE_MODULES_TYPE_STRIPPING` when run from an npm-installed copy - both bins are now committed esbuild bundles (sources in `src/bins/`, rebuild with `npm run build:bins`), guarded by a CI freshness check, bundle pack assertions, and a packed-install smoke test. (#39)
6
+
7
+ ## v5.12.0 - 2026-09-19
8
+
9
+ - New human-only `/skill:gauntlet-performance` and parse-only `gauntlet-performance` bin: the CLI digests committed telemetry records (current repo, plus `--dir <path>` repos; `--since <version>` narrows; `--json`) into per-run rows and per-version p50/max, and the skill turns the digest into one example-led recommendation, cornerstones, and a three-item menu (render on request, ticket via `shape-ticket`, drill-down). `gauntlet-telemetry-salvage` now stamps a record still `in_progress` with no ship phase as `status: shipped` + `shipped_at` at landing (`present`/`restored ... (marked shipped)`, `unfinished` under `--check`); `finishing-a-development-branch` and `gatekeep-pr` carry the new lines. The gh-37 record on this repo is backfilled to `shipped`. (#35)
10
+
3
11
  ## v5.11.0 - 2026-09-18
4
12
 
5
13
  - `checkoutOf` falls back to `jj root` when `git rev-parse` fails, so `plan_check`, Guard 2, and telemetry resolve the checkout inside a plain (non-colocated) jj workspace; colocated jj repos still resolve through git first. The settings loader goes through the same resolution via a new sync `checkoutOfSync`, and telemetry records bound to a jj workspace carry a one-time `record written, not committed: not a git checkout` warning instead of a failed `git commit` per checkpoint. (#38)
package/README.md CHANGED
@@ -69,9 +69,9 @@ Everything between gate 1 and gate 2 - task breakdown, implementation, both revi
69
69
 
70
70
  pi-gauntlet ships three kinds of pieces, layered on top of pi-cohort's dispatch:
71
71
 
72
- - **18 skills** - the workflow logic. Thirteen activate automatically when pi sees the matching kind of task, and each one gates the next: `brainstorming`, `writing-plans`, `roasting-the-spec`, `test-driven-development`, `subagent-driven-development`, `dispatching-parallel-agents`, `verification-before-completion`, `requesting-code-review`, `receiving-code-review`, `using-git-worktrees`, `finishing-a-development-branch`, `writing-skills`, `linear` (reads/searches/comments on/manages Linear tickets via the `linearis` CLI; owns all linearis mechanics and the `## Issue tracker` overrides schema; tracker-facing skills route to it). Five more are explicit-invocation-only (`disable-model-invocation: true`): `shape-ticket` creates or repairs one tracker issue per run against a Context/Problem/Idea/Acceptance-Criteria template, gated by an AC integrity check, a cheap council roast, and a single human-confirmed write - run it with `/skill:shape-ticket`. `gatekeep-pr` is consent-gated pre-merge verification of a PR against its issue - read-only gathering, verification evidence resolved CI-first (green checks on the exact assessed head count as evidence; the project's verification command runs only as fallback), a rubric-based review, then a deterministic authorship-aware menu with stable finding IDs (P#/L#/C#/F#) and numbered pre-composed courses (fixes execute as a single parallel-safe wave: one gate run, one re-review, one push); nothing mutates (fixes, pushes, reviews, merges) until you pick a row - run it with `/skill:gatekeep-pr <pr>`. `check-delivery` is a post-merge detective control: proves an issue actually shipped (default-branch landing, delivery target, per-AC evidence) before its tracker status advances; it never writes a terminal status - run it with `/skill:check-delivery <ref>`. `chase-bug` is human-only bug triage: read-only root-cause discovery to an evidenced verdict menu (real bug -> ticket/brainstorm/hotfix/respond; five negative verdicts), then a gated response to the reporter for addressable origins (GitHub issue / tracker ticket) and a rendered verdict summary otherwise - it never fixes during triage; the hotfix row hands off to `skills/chase-bug/hotfix.md` after the menu - run it with `/skill:chase-bug`. `gauntlet-resume` is the only way back into an interrupted flow from a fresh session: it takes a pi-cohort `/handoff` brief (file or pasted) or a bare worktree that already holds a spec, restores phase/plan tracker state through the legal arming sequence (`start brainstorm`, `skip` with `resume:` reasons, `plan_check` before implement-or-later), never creates a worktree, and never infers approval from artifacts - run it with `/skill:gauntlet-resume [<brief>] [<worktree>]`.
72
+ - **19 skills** - the workflow logic. Thirteen activate automatically when pi sees the matching kind of task, and each one gates the next: `brainstorming`, `writing-plans`, `roasting-the-spec`, `test-driven-development`, `subagent-driven-development`, `dispatching-parallel-agents`, `verification-before-completion`, `requesting-code-review`, `receiving-code-review`, `using-git-worktrees`, `finishing-a-development-branch`, `writing-skills`, `linear` (reads/searches/comments on/manages Linear tickets via the `linearis` CLI; owns all linearis mechanics and the `## Issue tracker` overrides schema; tracker-facing skills route to it). Six more are explicit-invocation-only (`disable-model-invocation: true`): `shape-ticket` creates or repairs one tracker issue per run against a Context/Problem/Idea/Acceptance-Criteria template, gated by an AC integrity check, a cheap council roast, and a single human-confirmed write - run it with `/skill:shape-ticket`. `gatekeep-pr` is consent-gated pre-merge verification of a PR against its issue - read-only gathering, verification evidence resolved CI-first (green checks on the exact assessed head count as evidence; the project's verification command runs only as fallback), a rubric-based review, then a deterministic authorship-aware menu with stable finding IDs (P#/L#/C#/F#) and numbered pre-composed courses (fixes execute as a single parallel-safe wave: one gate run, one re-review, one push); nothing mutates (fixes, pushes, reviews, merges) until you pick a row - run it with `/skill:gatekeep-pr <pr>`. `check-delivery` is a post-merge detective control: proves an issue actually shipped (default-branch landing, delivery target, per-AC evidence) before its tracker status advances; it never writes a terminal status - run it with `/skill:check-delivery <ref>`. `chase-bug` is human-only bug triage: read-only root-cause discovery to an evidenced verdict menu (real bug -> ticket/brainstorm/hotfix/respond; five negative verdicts), then a gated response to the reporter for addressable origins (GitHub issue / tracker ticket) and a rendered verdict summary otherwise - it never fixes during triage; the hotfix row hands off to `skills/chase-bug/hotfix.md` after the menu - run it with `/skill:chase-bug`. `gauntlet-performance` reads the committed run telemetry (current repo by default, `--dir <path>` adds others) through the parse-only `gauntlet-performance` CLI and answers with one example-led recommendation, 3-5 cornerstone numbers, and a menu of at most three actions (render a report file, open the recommendation as a ticket via `shape-ticket`, drill into one run); it writes nothing unless you pick render - run it with `/skill:gauntlet-performance [--dir <path>]... [--since <version>]`. `gauntlet-resume` is the only way back into an interrupted flow from a fresh session: it takes a pi-cohort `/handoff` brief (file or pasted) or a bare worktree that already holds a spec, restores phase/plan tracker state through the legal arming sequence (`start brainstorm`, `skip` with `resume:` reasons, `plan_check` before implement-or-later), never creates a worktree, and never infers approval from artifacts - run it with `/skill:gauntlet-resume [<brief>] [<worktree>]`.
73
73
  - **7 subagent personas** - the specialized child agents the skills dispatch via pi-cohort: `implementer`, `code-reviewer`, `spec-reviewer`, `conformance-reviewer`, `spec-summarizer`, `spec-council-member`, `spec-council-synthesizer`. See [doc/personas.md](./doc/personas.md) for what each one does and why its permissions are scoped the way they are.
74
- - **4 runtime extensions** - the enforcement layer. `plan-tracker` and `phase-tracker` are tools skills call to track progress (with a TUI widget); `verify-before-ship` is a hook that warns if you push or open a PR without a passing test run since your last edit; a phase-tracker flow guard reminds on implement-phase commits missing spec/code review. In a brainstorming-entered flow, phase-tracker rejects `implement` or `verify` completion while tracker tasks remain pending or in progress; see [its configuration reference](./doc/configuration.md#phase-tracker). phase-tracker also registers `plan_check`, which verifies a plan against its spec and against the grammar in [skills/writing-plans/reference/plan-contract.md](./skills/writing-plans/reference/plan-contract.md), including that each task's `Tests:` commands are selective and never the full suite; a pass stamps the plan for implementation. `telemetry` records one committed YAML record per gauntlet run (phase timing, models, personas, gate/fix rounds, diff at ship) that ships in the squash beside the spec - finishing and the PR gate restore a stripped record with `gauntlet-telemetry-salvage` - and blocks a brainstorm `write` into an already-shipped spec; see [its configuration reference](./doc/configuration.md#telemetry). See [doc/configuration.md](./doc/configuration.md) for the settings each one reads.
74
+ - **4 runtime extensions** - the enforcement layer. `plan-tracker` and `phase-tracker` are tools skills call to track progress (with a TUI widget); `verify-before-ship` is a hook that warns if you push or open a PR without a passing test run since your last edit; a phase-tracker flow guard reminds on implement-phase commits missing spec/code review. In a brainstorming-entered flow, phase-tracker rejects `implement` or `verify` completion while tracker tasks remain pending or in progress; see [its configuration reference](./doc/configuration.md#phase-tracker). phase-tracker also registers `plan_check`, which verifies a plan against its spec and against the grammar in [skills/writing-plans/reference/plan-contract.md](./skills/writing-plans/reference/plan-contract.md), including that each task's `Tests:` commands are selective and never the full suite; a pass stamps the plan for implementation. `telemetry` records one committed YAML record per gauntlet run (phase timing, models, personas, gate/fix rounds, diff at ship) that ships in the squash beside the spec - finishing and the PR gate restore a stripped record, and stamp a record left `in_progress` with no ship phase as `shipped`, with `gauntlet-telemetry-salvage` - and blocks a brainstorm `write` into an already-shipped spec; see [its configuration reference](./doc/configuration.md#telemetry). See [doc/configuration.md](./doc/configuration.md) for the settings each one reads.
75
75
 
76
76
  pi-gauntlet is **opinionated**: every non-trivial change is *meant* to ride this one pipeline, entered through `brainstorming`. Enforcement is opt-in by entry, not ambient: once brainstorming starts a flow, the phase-tracker extension mechanically blocks a phase from closing before its gate runs, and warns once if the main loop writes code during implement (subagents own implement-phase edits). A change made *without* entering the flow (a typo, a formatting run, a dependency bump - see "When to use / when NOT to use") is not gated; the discipline of routing real work through the pipeline is a convention the tooling supports, not a trap it springs on every edit.
77
77
 
@@ -122,6 +122,10 @@ Pin an exact release with `npm:pi-gauntlet@X.Y.Z`. See [doc/install-internals.md
122
122
 
123
123
  `gauntlet-spec-index` provides lexical search across `doc/specs/*.md` at the repository root and one service level down. From a repository worktree, run `node <pi-gauntlet-package>/bin/gauntlet-spec-index.mjs --query "<text>" [--limit N]`; it requires Node >=24.15.0, refreshes its FTS5 index on every query, and prints tab-separated `score`, `path`, `service`, `title`, `status`, `shipped_at`, `files`, and `snippet` columns. The per-worktree cache lives at `.pi/gauntlet/index.sqlite`, and its first creation adds `/.pi/gauntlet/index.sqlite*` to Git's `info/exclude` so the database and SQLite sidecars stay out of `git status`.
124
124
 
125
+ ## Performance digest
126
+
127
+ `gauntlet-performance` digests telemetry records without judging them: `node <pi-gauntlet-package>/bin/gauntlet-performance.mjs [--dir <repo root or telemetry dir>]... [--since <version>] [--json]` prints a `corpus:` line, one row per run (`run_id`, repo, spec, version, status - `shipped*` when the record has no ship phase - wall, per-phase minutes, tokens, cost, models, dispatches, `fix_round_grants`, reopens, conformance loops, reviewer findings, council dispatches), a `by version` block of p50/max over shipped, non-truncated runs, and `skipped:` lines for files it could not use. Default corpus is the current repo's telemetry dir; each `--dir` adds a repo root (its own `telemetry.dir` setting is honoured) or a bare telemetry dir. `--since` compares versions numerically. Exit 0 always except usage errors. `/skill:gauntlet-performance` is the reasoning layer on top.
128
+
125
129
  For local development against a checkout instead of npm:
126
130
 
127
131
  ```bash
@@ -0,0 +1,327 @@
1
+ #!/usr/bin/env node
2
+
3
+ // src/bins/gauntlet-performance.mjs
4
+ import { existsSync, readdirSync, readFileSync, realpathSync, statSync } from "node:fs";
5
+ import { homedir } from "node:os";
6
+ import { basename, join, resolve } from "node:path";
7
+ import { spawnSync } from "node:child_process";
8
+ import process from "node:process";
9
+ import { parse as parseYaml } from "yaml";
10
+
11
+ // extensions/lib/gauntlet-settings.ts
12
+ import path from "node:path";
13
+ function mergeGauntlet(preset, repo) {
14
+ return { ...preset ?? {}, ...repo ?? {} };
15
+ }
16
+ var nonEmptyString = (v) => typeof v === "string" && v.trim().length > 0;
17
+ var joinWarn = (ws) => ws.length ? ws.join("; ") : void 0;
18
+ var DEFAULT_TELEMETRY_DIR = ".pi/gauntlet/telemetry";
19
+ var DEFAULT_TELEMETRY_BUCKETS = [
20
+ ["test", ["**/test/**", "**/tests/**", "**/__tests__/**", "**/*.test.*", "**/*.spec.*", "**/*_test.*"]],
21
+ ["docs", ["**/*.md"]],
22
+ ["config", ["**/*.json", "**/*.yaml", "**/*.yml", "**/*.toml", "**/*.lock", "**/*-lock.*"]]
23
+ ];
24
+ function resolveTelemetry(g) {
25
+ const t = g.telemetry;
26
+ const warnings = [];
27
+ const enabled = t?.enabled !== false;
28
+ let dir = DEFAULT_TELEMETRY_DIR;
29
+ if (t?.dir !== void 0) {
30
+ const value = nonEmptyString(t.dir) ? t.dir.trim().replace(/\/+$/, "") : "";
31
+ const canonical = value.replace(/\\/g, "/");
32
+ const normalized = path.posix.normalize(canonical);
33
+ if (value && path.win32.parse(canonical).root === "" && normalized !== ".." && !normalized.startsWith("../")) dir = normalized;
34
+ else warnings.push("telemetry.dir must be a non-empty path relative to the git toplevel; using the default");
35
+ }
36
+ let buckets = DEFAULT_TELEMETRY_BUCKETS;
37
+ if (t?.buckets !== void 0) {
38
+ const b = t.buckets;
39
+ const valid = b !== null && typeof b === "object" && !Array.isArray(b) && Object.keys(b).length > 0 && Object.values(b).every((v) => Array.isArray(v) && v.length > 0 && v.every(nonEmptyString));
40
+ if (valid) buckets = Object.entries(b).map(([name, globs]) => [name, [...globs]]);
41
+ else warnings.push("telemetry.buckets is not an object of non-empty glob arrays; using the defaults");
42
+ }
43
+ return { enabled, dir, buckets, warning: joinWarn(warnings) };
44
+ }
45
+
46
+ // src/bins/gauntlet-performance.mjs
47
+ var PHASES = ["brainstorm", "plan", "implement", "verify", "ship"];
48
+ var TOKEN_KEYS = ["input", "output", "cache_read", "cache_write"];
49
+ var RUN_HEADER = ["run_id", "repo", "spec", "version", "status", "wall", "b/p/i/v/s min", "tokens", "cost", "models", "disp", "grants", "reopens", "loops", "findings", "council"];
50
+ var VERSION_HEADER = ["version", "n", "shipped", "truncated", "wall p50/max", "tokens p50/max", "cost p50/max", "disp p50", "grants p50/max", "reopens p50/max", "loops p50/max", "findings p50 b/M/m", "models"];
51
+ var usage = () => {
52
+ process.stderr.write("usage: gauntlet-performance [--dir <repo root or telemetry dir>]... [--since <version>] [--json]\n");
53
+ process.exit(1);
54
+ };
55
+ var semver = (s) => {
56
+ const m = /^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)$/.exec(typeof s === "string" ? s.trim() : "");
57
+ return m ? [Number(m[1]), Number(m[2]), Number(m[3])] : void 0;
58
+ };
59
+ var cmpSemver = (a, b) => a[0] - b[0] || a[1] - b[1] || a[2] - b[2];
60
+ function parseArgs(argv) {
61
+ const opts = { dirs: [], since: void 0, json: false };
62
+ for (let i = 0; i < argv.length; i++) {
63
+ const a = argv[i];
64
+ if (a === "--dir" && argv[i + 1] !== void 0 && !argv[i + 1].startsWith("--")) opts.dirs.push(argv[++i]);
65
+ else if (a === "--since" && argv[i + 1] !== void 0 && !argv[i + 1].startsWith("--")) opts.since = argv[++i];
66
+ else if (a === "--json") opts.json = true;
67
+ else usage();
68
+ }
69
+ if (opts.since !== void 0 && !semver(opts.since)) usage();
70
+ return opts;
71
+ }
72
+ function readLayer(file) {
73
+ if (!existsSync(file)) return {};
74
+ try {
75
+ return JSON.parse(readFileSync(file, "utf8"));
76
+ } catch (e) {
77
+ process.stderr.write(`warning: ${file}: ${e.message}; using {} for this layer
78
+ `);
79
+ return {};
80
+ }
81
+ }
82
+ function telemetryDirOf(root) {
83
+ const agentDir = process.env.PI_CODING_AGENT_DIR || join(homedir(), ".pi", "agent");
84
+ const preset = readLayer(join(agentDir, "settings.json"));
85
+ const repo = readLayer(join(root, ".pi", "settings.json"));
86
+ const t = resolveTelemetry(mergeGauntlet(preset?.piGauntlet, repo?.piGauntlet));
87
+ if (t.warning) process.stderr.write(`warning: ${t.warning}
88
+ `);
89
+ return join(root, t.dir);
90
+ }
91
+ var gitToplevel = (cwd) => {
92
+ const r = spawnSync("git", ["-C", cwd, "rev-parse", "--show-toplevel"], { encoding: "utf8", timeout: 1e4 });
93
+ return r.status === 0 ? r.stdout.trim() : void 0;
94
+ };
95
+ function corpusFor(path2) {
96
+ const label = basename(resolve(path2));
97
+ if (!existsSync(path2)) return { label, skip: "not found" };
98
+ const abs = realpathSync(path2);
99
+ const top = gitToplevel(abs);
100
+ if (top !== void 0 && realpathSync(top) === abs) {
101
+ const dir = telemetryDirOf(abs);
102
+ return existsSync(dir) ? { label, dir } : { label, skip: "no telemetry dir" };
103
+ }
104
+ if (!statSync(abs).isDirectory()) return { label, skip: "not a directory" };
105
+ return { label, dir: abs };
106
+ }
107
+ function* yamlFiles(dir) {
108
+ if (!existsSync(dir)) return;
109
+ for (const e of readdirSync(dir, { withFileTypes: true }).sort((a, b) => a.name.localeCompare(b.name))) {
110
+ const p = join(dir, e.name);
111
+ if (e.isDirectory()) yield* yamlFiles(p);
112
+ else if (e.isFile() && e.name.endsWith(".yaml")) yield p;
113
+ }
114
+ }
115
+ var num = (v) => typeof v === "number" && Number.isFinite(v) ? v : null;
116
+ var str = (v) => typeof v === "string" ? v : null;
117
+ var obj = (v) => v && typeof v === "object" && !Array.isArray(v) ? v : null;
118
+ var sumOrNull = (vals) => {
119
+ const xs = vals.filter((v) => v !== null);
120
+ return xs.length ? xs.reduce((a, b) => a + b, 0) : null;
121
+ };
122
+ function loadRecord(file, label) {
123
+ let doc;
124
+ try {
125
+ doc = parseYaml(readFileSync(file, "utf8"));
126
+ } catch {
127
+ return { skip: "unparseable" };
128
+ }
129
+ const d = obj(doc);
130
+ if (!d) return { skip: "unparseable" };
131
+ if (d.schema !== 1) return { skip: `schema ${d.schema === void 0 ? "missing" : String(d.schema)}` };
132
+ if (typeof d.spec !== "string" || typeof d.run_id !== "string") return { skip: "not a record" };
133
+ const derived = obj(d.derived) ?? {};
134
+ const phases = obj(derived.phases) ?? {};
135
+ const ph = (p) => obj(phases[p]);
136
+ const tok = (p) => obj(ph(p)?.tokens);
137
+ const personas = obj(derived.personas) ?? {};
138
+ const reviews = obj(derived.reviews);
139
+ const gates = obj(derived.gates) ?? {};
140
+ const version = str(obj(d.versions)?.["pi-gauntlet"]);
141
+ const reviewEntries = reviews === null ? null : Object.values(reviews);
142
+ const findings = reviewEntries === null ? null : reviewEntries.length === 0 ? { blocker: 0, major: 0, minor: 0 } : Object.fromEntries(["blocker", "major", "minor"].map((severity) => {
143
+ const counters = reviewEntries.map((r) => {
144
+ const review = obj(r);
145
+ if (review === null) return null;
146
+ if (!Object.hasOwn(review, "findings")) return 0;
147
+ const reviewFindings = obj(review.findings);
148
+ if (reviewFindings === null) return null;
149
+ return Object.hasOwn(reviewFindings, severity) ? num(reviewFindings[severity]) : 0;
150
+ });
151
+ return [severity, counters.includes(null) ? null : counters.reduce((a, b) => a + b, 0)];
152
+ }));
153
+ const truncated = ph("ship") === null;
154
+ return {
155
+ row: {
156
+ run_id: d.run_id,
157
+ repo: label,
158
+ spec: basename(file, ".yaml"),
159
+ version: semver(version) ? version.trim() : "unknown",
160
+ status: str(d.status),
161
+ created_at: str(d.created_at),
162
+ truncated,
163
+ wall_s: truncated ? null : num(derived.duration_s),
164
+ phase_min: Object.fromEntries(PHASES.map((p) => {
165
+ const s = num(ph(p)?.duration_s);
166
+ return [p, s === null ? null : s / 60];
167
+ })),
168
+ tokens: sumOrNull(PHASES.flatMap((p) => TOKEN_KEYS.map((k) => num(tok(p)?.[k])))),
169
+ cost: sumOrNull(PHASES.map((p) => num(tok(p)?.cost))),
170
+ models: [...new Set(PHASES.map((p) => str(ph(p)?.model)).filter((m) => m !== null))].sort(),
171
+ dispatches: sumOrNull(Object.values(personas).map((p) => num(obj(p)?.dispatches))),
172
+ grants: num(gates.fix_round_grants),
173
+ reopens: num(gates.task_reopens),
174
+ spec_rounds: num(gates.spec_rounds),
175
+ plan_rounds: num(gates.plan_rounds),
176
+ loops: num(derived.conformance_loops),
177
+ open_gaps: num(derived.conformance_open_gaps),
178
+ findings,
179
+ council: num(obj(personas["spec-council-member"])?.dispatches),
180
+ ship_option: str(gates.ship_option),
181
+ tests: str(obj(derived.tests)?.result)
182
+ }
183
+ };
184
+ }
185
+ var p50 = (xs) => {
186
+ const s = [...xs].sort((a, b) => a - b);
187
+ const m = s.length >> 1;
188
+ return s.length % 2 ? s[m] : (s[m - 1] + s[m]) / 2;
189
+ };
190
+ var stat = (rows, pick) => {
191
+ const xs = rows.map(pick).filter((v) => v !== null && v !== void 0);
192
+ return xs.length ? { p50: p50(xs), max: Math.max(...xs) } : { p50: null, max: null };
193
+ };
194
+ var versionOrder = (a, b) => {
195
+ const x = semver(a), y = semver(b);
196
+ return x && y ? cmpSemver(x, y) : x ? -1 : y ? 1 : 0;
197
+ };
198
+ function aggregate(rows) {
199
+ const groups = /* @__PURE__ */ new Map();
200
+ for (const r of rows) {
201
+ if (!groups.has(r.version)) groups.set(r.version, []);
202
+ groups.get(r.version).push(r);
203
+ }
204
+ return [...groups.entries()].sort(([a], [b]) => versionOrder(a, b)).map(([version, all]) => {
205
+ const shipped = all.filter((r) => r.status === "shipped" && !r.truncated);
206
+ const models = {};
207
+ for (const r of shipped) for (const m of r.models) models[m] = (models[m] ?? 0) + 1;
208
+ return {
209
+ version,
210
+ n: all.length,
211
+ shipped: shipped.length,
212
+ truncated: all.filter((r) => r.truncated).length,
213
+ wall_s: stat(shipped, (r) => r.wall_s),
214
+ tokens: stat(shipped, (r) => r.tokens),
215
+ cost: stat(shipped, (r) => r.cost),
216
+ dispatches: { p50: stat(shipped, (r) => r.dispatches).p50 },
217
+ grants: stat(shipped, (r) => r.grants),
218
+ reopens: stat(shipped, (r) => r.reopens),
219
+ loops: stat(shipped, (r) => r.loops),
220
+ findings: {
221
+ blocker: { p50: stat(shipped, (r) => r.findings?.blocker ?? null).p50 },
222
+ major: { p50: stat(shipped, (r) => r.findings?.major ?? null).p50 },
223
+ minor: { p50: stat(shipped, (r) => r.findings?.minor ?? null).p50 }
224
+ },
225
+ models
226
+ };
227
+ });
228
+ }
229
+ var dash = (v) => v === null || v === void 0 ? "-" : String(v);
230
+ var fmtMin = (s) => s === null ? "-" : `${Math.round(s / 60)}m`;
231
+ var fmtCount = (n) => n === null ? "-" : n >= 1e6 ? `${(n / 1e6).toFixed(1)}M` : n >= 1e3 ? `${(n / 1e3).toFixed(1)}k` : String(n);
232
+ var fmtCost = (c) => c === null ? "-" : c.toFixed(2);
233
+ var pair = (s, f) => `${f(s.p50)}/${f(s.max)}`;
234
+ var findingsCell = (f) => f === null ? "-" : `${dash(f.blocker)}/${dash(f.major)}/${dash(f.minor)}`;
235
+ var table = (header, rows) => {
236
+ const w = header.map((h, i) => Math.max(h.length, ...rows.map((r) => r[i].length)));
237
+ return [header, ...rows].map((r) => r.map((c, i) => c.padEnd(w[i])).join(" ").trimEnd()).join("\n");
238
+ };
239
+ var runRow = (r) => [
240
+ r.run_id.slice(0, 8),
241
+ r.repo,
242
+ r.spec,
243
+ r.version,
244
+ `${dash(r.status)}${r.truncated ? "*" : ""}`,
245
+ fmtMin(r.wall_s),
246
+ PHASES.map((p) => r.phase_min[p] === null ? "-" : String(Math.round(r.phase_min[p]))).join("/"),
247
+ fmtCount(r.tokens),
248
+ fmtCost(r.cost),
249
+ r.models.join(",") || "-",
250
+ dash(r.dispatches),
251
+ dash(r.grants),
252
+ dash(r.reopens),
253
+ dash(r.loops),
254
+ findingsCell(r.findings),
255
+ dash(r.council)
256
+ ];
257
+ var versionRow = (g) => [
258
+ g.version,
259
+ String(g.n),
260
+ String(g.shipped),
261
+ String(g.truncated),
262
+ pair(g.wall_s, fmtMin),
263
+ pair(g.tokens, fmtCount),
264
+ pair(g.cost, fmtCost),
265
+ dash(g.dispatches.p50),
266
+ pair(g.grants, dash),
267
+ pair(g.reopens, dash),
268
+ pair(g.loops, dash),
269
+ `${dash(g.findings.blocker.p50)}/${dash(g.findings.major.p50)}/${dash(g.findings.minor.p50)}`,
270
+ Object.entries(g.models).map(([m, n]) => `${m}:${n}`).join(",") || "-"
271
+ ];
272
+ function main() {
273
+ const opts = parseArgs(process.argv.slice(2));
274
+ const corpora = [];
275
+ const skipped = [];
276
+ const top = gitToplevel(process.cwd());
277
+ if (top === void 0) process.stderr.write(`not a git checkout: ${process.cwd()}
278
+ `);
279
+ else corpora.push({ label: basename(top), dir: telemetryDirOf(top) });
280
+ for (const p of opts.dirs) {
281
+ const c = corpusFor(p);
282
+ if (c.skip) skipped.push({ file: p, reason: c.skip });
283
+ else corpora.push(c);
284
+ }
285
+ const since = opts.since === void 0 ? void 0 : semver(opts.since);
286
+ const seen = /* @__PURE__ */ new Set();
287
+ const corpus = {};
288
+ const rows = [];
289
+ for (const c of corpora) {
290
+ corpus[c.label] = corpus[c.label] ?? 0;
291
+ for (const file of yamlFiles(c.dir)) {
292
+ const res = loadRecord(file, c.label);
293
+ if (res.skip) {
294
+ skipped.push({ file, reason: res.skip });
295
+ continue;
296
+ }
297
+ if (seen.has(res.row.run_id)) {
298
+ skipped.push({ file, reason: "duplicate run_id" });
299
+ continue;
300
+ }
301
+ seen.add(res.row.run_id);
302
+ const v = semver(res.row.version);
303
+ if (since && (!v || cmpSemver(v, since) < 0)) continue;
304
+ corpus[c.label]++;
305
+ rows.push(res.row);
306
+ }
307
+ }
308
+ rows.sort((a, b) => (a.created_at ?? "").localeCompare(b.created_at ?? "") || a.run_id.localeCompare(b.run_id));
309
+ const byVersion = rows.length ? aggregate(rows) : [];
310
+ if (opts.json) {
311
+ console.log(JSON.stringify({ corpus, since: opts.since ?? null, runs: rows, by_version: byVersion, skipped }, null, 2));
312
+ return;
313
+ }
314
+ console.log(`corpus: ${Object.entries(corpus).map(([k, v]) => `${k}=${v}`).join(", ")}${opts.since === void 0 ? "" : ` since: ${opts.since}`}`);
315
+ if (rows.length === 0) {
316
+ console.log(`no records found in ${corpora.map((c) => c.dir).join(", ") || process.cwd()}`);
317
+ } else {
318
+ console.log(`runs (${rows.length})`);
319
+ console.log(table(RUN_HEADER, rows.map(runRow)));
320
+ console.log("");
321
+ console.log("by version");
322
+ console.log(table(VERSION_HEADER, byVersion.map(versionRow)));
323
+ }
324
+ if (rows.length && skipped.length) console.log("");
325
+ if (rows.length) for (const s of skipped) console.log(`skipped: ${s.file}: ${s.reason}`);
326
+ }
327
+ main();
@@ -0,0 +1,350 @@
1
+ import { test } from "node:test";
2
+ import assert from "node:assert/strict";
3
+ import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from "node:fs";
4
+ import { join, dirname } from "node:path";
5
+ import { tmpdir } from "node:os";
6
+ import { spawnSync } from "node:child_process";
7
+ import { fileURLToPath } from "node:url";
8
+
9
+ const CLI = join(dirname(fileURLToPath(import.meta.url)), "gauntlet-performance.mjs");
10
+ const TDIR = ".pi/gauntlet/telemetry/doc/specs";
11
+ const EMPTY_AGENT = mkdtempSync(join(tmpdir(), "gp-empty-agent-"));
12
+ process.on("exit", () => rmSync(EMPTY_AGENT, { recursive: true, force: true }));
13
+
14
+ const write = (root, rel, text) => {
15
+ mkdirSync(join(root, dirname(rel)), { recursive: true });
16
+ writeFileSync(join(root, rel), text);
17
+ };
18
+ const gitRepo = (prefix = "gp-") => {
19
+ const root = mkdtempSync(join(tmpdir(), prefix));
20
+ spawnSync("git", ["init", "-q"], { cwd: root });
21
+ return root;
22
+ };
23
+ const invoke = (cwd, args = [], env = {}) => {
24
+ const r = spawnSync(process.execPath, [CLI, ...args], {
25
+ cwd, encoding: "utf8", env: { ...process.env, PI_CODING_AGENT_DIR: EMPTY_AGENT, ...env },
26
+ });
27
+ return { status: r.status, stdout: r.stdout, stderr: r.stderr.trim(), lines: r.stdout.split("\n").filter(Boolean) };
28
+ };
29
+ const json = (cwd, args = []) => JSON.parse(invoke(cwd, ["--json", ...args]).stdout);
30
+ const cleanup = (t, ...roots) => t.after(() => roots.forEach((r) => rmSync(r, { recursive: true, force: true })));
31
+
32
+ const phase = (extra) => `{ started_at: 2026-09-01T00:00:00Z, completed_at: 2026-09-01T00:10:00Z, duration_s: 600, ${extra} }`;
33
+ const tokens = (input, output, cacheRead, cacheWrite, cost) =>
34
+ `tokens: { input: ${input}, output: ${output}, cache_read: ${cacheRead}, cache_write: ${cacheWrite}, cost: ${cost} }`;
35
+
36
+ // Complete run: cache-dominated tokens, an unphased block that must be ignored, two models.
37
+ const COMPLETE = `schema: 1
38
+ spec: doc/specs/a.md
39
+ run_id: aaaaaaaa-0000-0000-0000-000000000001
40
+ status: shipped
41
+ created_at: 2026-09-01T00:00:00Z
42
+ shipped_at: 2026-09-01T01:00:00Z
43
+ versions: { pi-gauntlet: 5.9.1 }
44
+ derived:
45
+ duration_s: 3600
46
+ phases:
47
+ brainstorm: ${phase(`model: m/alpha, ${tokens(100, 10, 5000, 200, 1.5)}`)}
48
+ plan: ${phase(`model: m/alpha, ${tokens(50, 5, 1000, 100, 0.5)}`)}
49
+ implement: ${phase(`model: m/beta, ${tokens(10, 1, 100, 10, 0.25)}`)}
50
+ verify: ${phase(`model: m/beta`)}
51
+ ship: ${phase(`model: m/beta`)}
52
+ unphased: { ${tokens(99999, 99999, 99999, 99999, 999)} }
53
+ personas:
54
+ worker: { dispatches: 3, models: [m/beta] }
55
+ spec-council-member: { dispatches: 4, models: [m/alpha] }
56
+ reviews:
57
+ spec-reviewer: { dispatches: 1, nonzero_exit: 0, findings: { blocker: 1, major: 2, minor: 3 } }
58
+ code-reviewer: { dispatches: 1, nonzero_exit: 0 }
59
+ conformance_loops: 2
60
+ conformance_open_gaps: 0
61
+ gates: { spec_rounds: 1, plan_rounds: 1, fix_round_grants: 1, task_reopens: 0, ship_option: squash }
62
+ tests: { command: npm test, result: pass }
63
+ accumulators: {}
64
+ events: []
65
+ `;
66
+ // Truncated run: stamped shipped by salvage, stale duration_s, no ship phase, zero gates.
67
+ const TRUNCATED = `schema: 1
68
+ spec: doc/specs/b.md
69
+ run_id: bbbbbbbb-0000-0000-0000-000000000002
70
+ status: shipped
71
+ created_at: 2026-09-02T00:00:00Z
72
+ shipped_at: 2026-09-02T02:00:00Z
73
+ versions: { pi-gauntlet: 5.9.1 }
74
+ derived:
75
+ duration_s: 2501
76
+ phases:
77
+ brainstorm: ${phase(`model: m/alpha, ${tokens(1, 1, 1, 1, 0.01)}`)}
78
+ plan: { started_at: 2026-09-02T00:10:00Z, model: m/alpha }
79
+ personas: {}
80
+ conformance_loops: 0
81
+ gates: { spec_rounds: 0, plan_rounds: 0, fix_round_grants: 0, task_reopens: 0 }
82
+ accumulators: {}
83
+ events: []
84
+ `;
85
+ const IN_PROGRESS = `schema: 1
86
+ spec: doc/specs/c.md
87
+ run_id: cccccccc-0000-0000-0000-000000000003
88
+ status: in_progress
89
+ created_at: 2026-09-03T00:00:00Z
90
+ versions: { pi-gauntlet: 5.11.0 }
91
+ derived:
92
+ duration_s: 10
93
+ phases: { brainstorm: { started_at: 2026-09-03T00:00:00Z } }
94
+ personas: {}
95
+ conformance_loops: 0
96
+ gates: { spec_rounds: 0, plan_rounds: 0, fix_round_grants: 0, task_reopens: 0 }
97
+ accumulators: {}
98
+ events: []
99
+ `;
100
+ // Second complete run on 5.9.1 with different numbers: makes p50 over an even count.
101
+ const COMPLETE_2 = COMPLETE
102
+ .replace("run_id: aaaaaaaa-0000-0000-0000-000000000001", "run_id: dddddddd-0000-0000-0000-000000000004")
103
+ .replace("spec: doc/specs/a.md", "spec: doc/specs/d.md")
104
+ .replace("created_at: 2026-09-01T00:00:00Z", "created_at: 2026-09-04T00:00:00Z")
105
+ .replace("duration_s: 3600", "duration_s: 7200")
106
+ .replace("fix_round_grants: 1", "fix_round_grants: 3")
107
+ .replace(/reviews:[\s\S]*?conformance_loops/, "conformance_loops");
108
+ const OLD_VERSION = COMPLETE
109
+ .replace("run_id: aaaaaaaa-0000-0000-0000-000000000001", "run_id: eeeeeeee-0000-0000-0000-000000000005")
110
+ .replace("spec: doc/specs/a.md", "spec: doc/specs/e.md")
111
+ .replace("pi-gauntlet: 5.9.1", "pi-gauntlet: 5.10.0");
112
+ const BAD_GATES = IN_PROGRESS
113
+ .replace("run_id: cccccccc-0000-0000-0000-000000000003", "run_id: ffffffff-0000-0000-0000-000000000006")
114
+ .replace("spec: doc/specs/c.md", "spec: doc/specs/f.md")
115
+ .replace(/gates: \{[^}]*\}/, 'gates: "nope"');
116
+ const EMPTY_REVIEWS = IN_PROGRESS
117
+ .replace("run_id: cccccccc-0000-0000-0000-000000000003", "run_id: gggggggg-0000-0000-0000-000000000007")
118
+ .replace("spec: doc/specs/c.md", "spec: doc/specs/g.md")
119
+ .replace("created_at: 2026-09-03T00:00:00Z", "created_at: 2026-09-05T00:00:00Z")
120
+ .replace(" conformance_loops: 0", " reviews: {}\n conformance_loops: 0");
121
+ const REVIEW_WITHOUT_FINDINGS = IN_PROGRESS
122
+ .replace("run_id: cccccccc-0000-0000-0000-000000000003", "run_id: hhhhhhhh-0000-0000-0000-000000000008")
123
+ .replace("spec: doc/specs/c.md", "spec: doc/specs/h.md")
124
+ .replace(" conformance_loops: 0", " reviews: { code-reviewer: { dispatches: 1, nonzero_exit: 0 } }\n conformance_loops: 0");
125
+
126
+ const corpus = (root) => {
127
+ write(root, `${TDIR}/a.yaml`, COMPLETE);
128
+ write(root, `${TDIR}/b.yaml`, TRUNCATED);
129
+ write(root, `${TDIR}/c.yaml`, IN_PROGRESS);
130
+ write(root, `${TDIR}/d.yaml`, COMPLETE_2);
131
+ write(root, `${TDIR}/e.yaml`, OLD_VERSION);
132
+ write(root, `${TDIR}/f.yaml`, BAD_GATES);
133
+ write(root, `${TDIR}/present-empty-reviews.yaml`, EMPTY_REVIEWS);
134
+ write(root, `${TDIR}/broken.yaml`, "schema: 1\nspec: [\n");
135
+ write(root, `${TDIR}/v2.yaml`, "schema: 2\nspec: doc/specs/z.md\nrun_id: z\n");
136
+ write(root, `${TDIR}/norec.yaml`, "schema: 1\nspec: doc/specs/z.md\n");
137
+ return root;
138
+ };
139
+ const byId = (runs, prefix) => runs.find((r) => r.run_id.startsWith(prefix));
140
+
141
+ test("per-run fields: typed reads, unphased excluded, truncated wall null, findings null vs 0/0/0", (t) => {
142
+ const root = corpus(gitRepo()); cleanup(t, root);
143
+ const d = json(root);
144
+ assert.deepEqual(d.corpus, { [root.split("/").pop()]: 7 });
145
+ assert.equal(d.since, null);
146
+ const a = byId(d.runs, "aaaaaaaa");
147
+ assert.equal(a.repo, root.split("/").pop());
148
+ assert.equal(a.spec, "a");
149
+ assert.equal(a.version, "5.9.1");
150
+ assert.equal(a.status, "shipped");
151
+ assert.equal(a.truncated, false);
152
+ assert.equal(a.wall_s, 3600);
153
+ assert.deepEqual(a.phase_min, { brainstorm: 10, plan: 10, implement: 10, verify: 10, ship: 10 });
154
+ assert.equal(a.tokens, 100 + 10 + 5000 + 200 + 50 + 5 + 1000 + 100 + 10 + 1 + 100 + 10);
155
+ assert.equal(a.cost, 2.25);
156
+ assert.deepEqual(a.models, ["m/alpha", "m/beta"]);
157
+ assert.equal(a.dispatches, 7);
158
+ assert.equal(a.grants, 1);
159
+ assert.equal(a.reopens, 0);
160
+ assert.equal(a.spec_rounds, 1);
161
+ assert.equal(a.plan_rounds, 1);
162
+ assert.equal(a.loops, 2);
163
+ assert.equal(a.open_gaps, 0);
164
+ assert.deepEqual(a.findings, { blocker: 1, major: 2, minor: 3 });
165
+ assert.equal(a.council, 4);
166
+ assert.equal(a.ship_option, "squash");
167
+ assert.equal(a.tests, "pass");
168
+ const b = byId(d.runs, "bbbbbbbb");
169
+ assert.equal(b.truncated, true);
170
+ assert.equal(b.wall_s, null);
171
+ assert.deepEqual(b.phase_min, { brainstorm: 10, plan: null, implement: null, verify: null, ship: null });
172
+ assert.equal(b.tokens, 4);
173
+ assert.equal(b.dispatches, null);
174
+ assert.equal(b.findings, null);
175
+ assert.equal(b.council, null);
176
+ assert.equal(b.ship_option, null);
177
+ assert.equal(b.tests, null);
178
+ const dd = byId(d.runs, "dddddddd");
179
+ assert.equal(dd.findings, null);
180
+ const f = byId(d.runs, "ffffffff");
181
+ assert.equal(f.grants, null);
182
+ assert.equal(f.status, "in_progress");
183
+ const emptyReviews = byId(d.runs, "gggggggg");
184
+ assert.deepEqual(emptyReviews.findings, { blocker: 0, major: 0, minor: 0 });
185
+ });
186
+
187
+ test("review entries without findings contribute zero counters", (t) => {
188
+ const root = gitRepo(); cleanup(t, root);
189
+ write(root, `${TDIR}/review-without-findings.yaml`, REVIEW_WITHOUT_FINDINGS);
190
+ const d = json(root);
191
+ assert.deepEqual(d.runs[0].findings, { blocker: 0, major: 0, minor: 0 });
192
+ });
193
+
194
+ test("malformed findings counters remain null in JSON and render as dashes in text", (t) => {
195
+ const root = gitRepo(); cleanup(t, root);
196
+ const malformed = COMPLETE.replace(
197
+ "findings: { blocker: 1, major: 2, minor: 3 }",
198
+ 'findings: { blocker: "bad", major: 2, minor: 3 }',
199
+ );
200
+ write(root, `${TDIR}/malformed-findings.yaml`, malformed);
201
+ const d = json(root);
202
+ assert.deepEqual(d.runs[0].findings, { blocker: null, major: 2, minor: 3 });
203
+ const row = invoke(root).lines.find((line) => line.startsWith("aaaaaaaa"));
204
+ assert.match(row, /\s-\/2\/3\s+4$/);
205
+ });
206
+
207
+ test("malformed reviewer entries and findings objects contribute null counters", (t) => {
208
+ const root = gitRepo(); cleanup(t, root);
209
+ const malformedFindings = COMPLETE.replace(
210
+ "findings: { blocker: 1, major: 2, minor: 3 }",
211
+ 'findings: "bad"',
212
+ );
213
+ const malformedReviewer = COMPLETE.replace(
214
+ "spec-reviewer: { dispatches: 1, nonzero_exit: 0, findings: { blocker: 1, major: 2, minor: 3 } }",
215
+ 'spec-reviewer: "bad"',
216
+ );
217
+ write(root, `${TDIR}/malformed-findings-object.yaml`, malformedFindings);
218
+ write(root, `${TDIR}/malformed-reviewer.yaml`, malformedReviewer.replace(
219
+ "run_id: aaaaaaaa-0000-0000-0000-000000000001",
220
+ "run_id: 22222222-0000-0000-0000-000000000012",
221
+ ));
222
+ const d = json(root);
223
+ assert.deepEqual(byId(d.runs, "aaaaaaaa").findings, { blocker: null, major: null, minor: null });
224
+ assert.deepEqual(byId(d.runs, "22222222").findings, { blocker: null, major: null, minor: null });
225
+ });
226
+
227
+ test("phase minutes remain unrounded in JSON", (t) => {
228
+ const root = gitRepo(); cleanup(t, root);
229
+ write(root, `${TDIR}/phase-61.yaml`, COMPLETE.replace("duration_s: 600", "duration_s: 61"));
230
+ const d = json(root);
231
+ assert.equal(d.runs[0].phase_min.brainstorm, 61 / 60);
232
+ });
233
+
234
+ test("aggregates: only shipped non-truncated rows feed p50/max; even-count p50 is the mean; models tally; unknown group", (t) => {
235
+ const root = corpus(gitRepo()); cleanup(t, root);
236
+ write(root, `${TDIR}/g.yaml`, COMPLETE.replace("run_id: aaaaaaaa-0000-0000-0000-000000000001", "run_id: 99999999-0000-0000-0000-000000000009").replace("versions: { pi-gauntlet: 5.9.1 }", "versions: {}"));
237
+ const d = json(root);
238
+ assert.deepEqual(d.by_version.map((g) => g.version), ["5.9.1", "5.10.0", "5.11.0", "unknown"]);
239
+ const v591 = d.by_version[0];
240
+ assert.equal(v591.n, 3);
241
+ assert.equal(v591.shipped, 2);
242
+ assert.equal(v591.truncated, 1);
243
+ assert.deepEqual(v591.wall_s, { p50: 5400, max: 7200 });
244
+ assert.deepEqual(v591.grants, { p50: 2, max: 3 });
245
+ assert.deepEqual(v591.dispatches, { p50: 7 });
246
+ assert.deepEqual(v591.findings, { blocker: { p50: 1 }, major: { p50: 2 }, minor: { p50: 3 } });
247
+ assert.deepEqual(v591.models, { "m/alpha": 2, "m/beta": 2 });
248
+ const v5110 = d.by_version[2];
249
+ assert.equal(v5110.n, 3);
250
+ assert.equal(v5110.shipped, 0);
251
+ assert.deepEqual(v5110.wall_s, { p50: null, max: null });
252
+ assert.deepEqual(v5110.models, {});
253
+ assert.equal(d.by_version[3].shipped, 1);
254
+ });
255
+
256
+ test("row order is created_at then run_id; skipped lists unparseable, schema, not a record", (t) => {
257
+ const root = corpus(gitRepo()); cleanup(t, root);
258
+ const d = json(root);
259
+ assert.deepEqual(d.runs.map((r) => r.run_id[0]), ["a", "e", "b", "c", "f", "d", "g"]);
260
+ const reasons = Object.fromEntries(d.skipped.map((s) => [s.file.split("/").pop(), s.reason]));
261
+ assert.equal(reasons["broken.yaml"], "unparseable");
262
+ assert.equal(reasons["v2.yaml"], "schema 2");
263
+ assert.equal(reasons["norec.yaml"], "not a record");
264
+ });
265
+
266
+ test("text output: header, shipped* marker, - cells, by version block, skipped lines", (t) => {
267
+ const root = corpus(gitRepo()); cleanup(t, root);
268
+ const r = invoke(root);
269
+ assert.equal(r.status, 0, r.stderr);
270
+ assert.equal(r.lines[0], `corpus: ${root.split("/").pop()}=7`);
271
+ assert.equal(r.lines[1], "runs (7)");
272
+ assert.match(r.lines[2], /^run_id\s+repo\s+spec\s+version\s+status\s+wall\s+b\/p\/i\/v\/s min\s+tokens\s+cost\s+models\s+disp\s+grants\s+reopens\s+loops\s+findings\s+council$/);
273
+ const bRow = r.lines.find((l) => l.startsWith("bbbbbbbb"));
274
+ assert.match(bRow, /\s5\.9\.1\s+shipped\*\s+-\s+10\/-\/-\/-\/-\s/);
275
+ assert.match(bRow, /\s-\s*$/);
276
+ const aRow = r.lines.find((l) => l.startsWith("aaaaaaaa"));
277
+ assert.match(aRow, /\sshipped\s+60m\s+10\/10\/10\/10\/10\s+6\.6k\s+2\.25\s+m\/alpha,m\/beta\s+7\s+1\s+0\s+2\s+1\/2\/3\s+4$/);
278
+ assert.ok(r.lines.includes("by version"));
279
+ const v = r.lines.find((l) => l.startsWith("5.9.1"));
280
+ assert.match(v, /^5\.9\.1\s+3\s+2\s+1\s+90m\/120m\s/);
281
+ assert.ok(r.lines.some((l) => /^skipped: .*broken\.yaml: unparseable$/.test(l)));
282
+ });
283
+
284
+ test("--since numeric compare: 5.10.0 > 5.9.1, unknown and malformed versions dropped; no rows -> no records found, exit 0", (t) => {
285
+ const root = corpus(gitRepo()); cleanup(t, root);
286
+ write(root, `${TDIR}/g.yaml`, COMPLETE.replace("run_id: aaaaaaaa-0000-0000-0000-000000000001", "run_id: 99999999-0000-0000-0000-000000000009").replace("versions: { pi-gauntlet: 5.9.1 }", "versions: {}"));
287
+ write(root, `${TDIR}/malformed-version.yaml`, COMPLETE.replace("run_id: aaaaaaaa-0000-0000-0000-000000000001", "run_id: 11111111-0000-0000-0000-000000000011").replace("pi-gauntlet: 5.9.1", "pi-gauntlet: 5.9.1garbage"));
288
+ write(root, `${TDIR}/leading-zero-version.yaml`, COMPLETE.replace("run_id: aaaaaaaa-0000-0000-0000-000000000001", "run_id: 33333333-0000-0000-0000-000000000013").replace("pi-gauntlet: 5.9.1", "pi-gauntlet: 5.09.1"));
289
+ assert.equal(byId(json(root).runs, "11111111").version, "unknown");
290
+ assert.equal(byId(json(root).runs, "33333333").version, "unknown");
291
+ const sinceFive = json(root, ["--since", "5.0.0"]);
292
+ assert.equal(byId(sinceFive.runs, "33333333"), undefined);
293
+ const d = json(root, ["--since", "5.10.0"]);
294
+ assert.equal(d.since, "5.10.0");
295
+ assert.deepEqual(d.runs.map((r) => r.version).sort(), ["5.10.0", "5.11.0", "5.11.0", "5.11.0"]);
296
+ const r = invoke(root, ["--since", "9.9.9"]);
297
+ assert.equal(r.status, 0);
298
+ assert.equal(r.lines[0], `corpus: ${root.split("/").pop()}=0 since: 9.9.9`);
299
+ assert.match(r.lines[1], /^no records found in .*\.pi\/gauntlet\/telemetry$/);
300
+ assert.equal(r.lines.length, 2);
301
+ });
302
+
303
+ test("--dir: repo root with telemetry dir, repo root without (skipped, checkout not walked), bare telemetry dir; duplicate run_id collapses", (t) => {
304
+ const root = corpus(gitRepo());
305
+ const other = gitRepo("gp-other-");
306
+ write(other, `${TDIR}/dup.yaml`, COMPLETE);
307
+ write(other, `${TDIR}/h.yaml`, COMPLETE.replace("run_id: aaaaaaaa-0000-0000-0000-000000000001", "run_id: 88888888-0000-0000-0000-000000000008"));
308
+ const empty = gitRepo("gp-empty-");
309
+ write(empty, "doc/specs/decoy.yaml", COMPLETE.replace("run_id: aaaaaaaa-0000-0000-0000-000000000001", "run_id: 77777777-0000-0000-0000-000000000007"));
310
+ const bare = mkdtempSync(join(tmpdir(), "gp-bare-"));
311
+ write(bare, "x.yaml", COMPLETE.replace("run_id: aaaaaaaa-0000-0000-0000-000000000001", "run_id: 66666666-0000-0000-0000-000000000006"));
312
+ cleanup(t, root, other, empty, bare);
313
+ const d = json(root, ["--dir", other, "--dir", empty, "--dir", bare, "--dir", join(bare, "missing")]);
314
+ assert.equal(d.corpus[other.split("/").pop()], 1);
315
+ assert.equal(d.corpus[bare.split("/").pop()], 1);
316
+ assert.equal(d.corpus[empty.split("/").pop()], undefined);
317
+ assert.ok(d.runs.some((r) => r.run_id.startsWith("88888888")));
318
+ assert.ok(d.runs.some((r) => r.run_id.startsWith("66666666")));
319
+ assert.ok(!d.runs.some((r) => r.run_id.startsWith("77777777")));
320
+ assert.equal(d.runs.filter((r) => r.run_id.startsWith("aaaaaaaa")).length, 1);
321
+ assert.ok(d.skipped.some((s) => s.file.endsWith("dup.yaml") && s.reason === "duplicate run_id"));
322
+ assert.ok(d.skipped.some((s) => s.file === empty && s.reason === "no telemetry dir"));
323
+ assert.ok(d.skipped.some((s) => s.file === join(bare, "missing") && s.reason === "not found"));
324
+ });
325
+
326
+ test("--dir skips an existing regular file as not a directory", (t) => {
327
+ const root = gitRepo();
328
+ const file = join(root, "telemetry.yaml");
329
+ writeFileSync(file, COMPLETE);
330
+ cleanup(t, root);
331
+ const r = invoke(root, ["--json", "--dir", file]);
332
+ assert.equal(r.status, 0, r.stderr);
333
+ const d = JSON.parse(r.stdout);
334
+ assert.ok(d.skipped.some((s) => s.file === file && s.reason === "not a directory"));
335
+ });
336
+
337
+ test("repo .pi/settings.json telemetry.dir is honoured for the default corpus", (t) => {
338
+ const root = gitRepo(); cleanup(t, root);
339
+ write(root, ".pi/settings.json", JSON.stringify({ piGauntlet: { telemetry: { dir: "custom/dir" } } }));
340
+ write(root, "custom/dir/doc/specs/a.yaml", COMPLETE);
341
+ const d = json(root);
342
+ assert.equal(d.runs.length, 1);
343
+ });
344
+
345
+ test("usage error exits 1", (t) => {
346
+ const root = gitRepo(); cleanup(t, root);
347
+ assert.equal(invoke(root, ["--bogus"]).status, 1);
348
+ assert.equal(invoke(root, ["--since", "abc"]).status, 1);
349
+ assert.equal(invoke(root, ["--since", "5.09.1"]).status, 1);
350
+ });
@@ -1,76 +1,203 @@
1
1
  #!/usr/bin/env node
2
- // Restore a gauntlet telemetry record that a plan strip or a gate fix commit removed
3
- // from the branch. The recorder pathspec-commits the record at every checkpoint, so the
4
- // newest copy is always in the deleting commit's parent. Exit 0 on every outcome: the
5
- // callers are ship paths and this script never blocks one.
6
- import { existsSync, readFileSync, rmSync } from "node:fs";
2
+
3
+ // src/bins/gauntlet-telemetry-salvage.mjs
4
+ import { existsSync, readFileSync, rmSync, writeFileSync } from "node:fs";
7
5
  import { homedir } from "node:os";
8
6
  import { isAbsolute, join } from "node:path";
9
7
  import { spawnSync } from "node:child_process";
10
8
  import process from "node:process";
11
- import { mergeGauntlet, resolveTelemetry } from "../extensions/lib/gauntlet-settings.ts";
12
- import { isSpecPath, recordPathFor } from "../extensions/lib/telemetry-paths.ts";
13
- import { BASE_REFS } from "../extensions/lib/telemetry-ship.ts";
14
9
 
15
- const GIT_TIMEOUT_MS = 10_000;
16
- const COMMIT_TIMEOUT_MS = Number(process.env.GAUNTLET_SALVAGE_COMMIT_TIMEOUT_MS) || 30_000;
17
- const GIT_ENV = { ...process.env, GIT_TERMINAL_PROMPT: "0", GIT_EDITOR: "true" };
10
+ // extensions/lib/gauntlet-settings.ts
11
+ import path from "node:path";
12
+ function mergeGauntlet(preset, repo) {
13
+ return { ...preset ?? {}, ...repo ?? {} };
14
+ }
15
+ var nonEmptyString = (v) => typeof v === "string" && v.trim().length > 0;
16
+ var joinWarn = (ws) => ws.length ? ws.join("; ") : void 0;
17
+ var DEFAULT_TELEMETRY_DIR = ".pi/gauntlet/telemetry";
18
+ var DEFAULT_TELEMETRY_BUCKETS = [
19
+ ["test", ["**/test/**", "**/tests/**", "**/__tests__/**", "**/*.test.*", "**/*.spec.*", "**/*_test.*"]],
20
+ ["docs", ["**/*.md"]],
21
+ ["config", ["**/*.json", "**/*.yaml", "**/*.yml", "**/*.toml", "**/*.lock", "**/*-lock.*"]]
22
+ ];
23
+ function resolveTelemetry(g) {
24
+ const t = g.telemetry;
25
+ const warnings = [];
26
+ const enabled = t?.enabled !== false;
27
+ let dir = DEFAULT_TELEMETRY_DIR;
28
+ if (t?.dir !== void 0) {
29
+ const value = nonEmptyString(t.dir) ? t.dir.trim().replace(/\/+$/, "") : "";
30
+ const canonical = value.replace(/\\/g, "/");
31
+ const normalized = path.posix.normalize(canonical);
32
+ if (value && path.win32.parse(canonical).root === "" && normalized !== ".." && !normalized.startsWith("../")) dir = normalized;
33
+ else warnings.push("telemetry.dir must be a non-empty path relative to the git toplevel; using the default");
34
+ }
35
+ let buckets = DEFAULT_TELEMETRY_BUCKETS;
36
+ if (t?.buckets !== void 0) {
37
+ const b = t.buckets;
38
+ const valid = b !== null && typeof b === "object" && !Array.isArray(b) && Object.keys(b).length > 0 && Object.values(b).every((v) => Array.isArray(v) && v.length > 0 && v.every(nonEmptyString));
39
+ if (valid) buckets = Object.entries(b).map(([name, globs]) => [name, [...globs]]);
40
+ else warnings.push("telemetry.buckets is not an object of non-empty glob arrays; using the defaults");
41
+ }
42
+ return { enabled, dir, buckets, warning: joinWarn(warnings) };
43
+ }
44
+
45
+ // extensions/lib/phase-tracker-helpers.ts
46
+ var STMT_START = "(?:^|[\\n;&|(])\\s*";
47
+
48
+ // extensions/lib/telemetry-paths.ts
49
+ var isSpecPath = (rel) => /(^|\/)doc\/specs\/[^/]+\.md$/.test(rel);
50
+ var recordPathFor = (dir, specRel) => `${dir}/${specRel.replace(/\.md$/, ".yaml")}`;
51
+ var GIT_FLAGS = "git\\s+(?:-\\S+(?:\\s+\\S+)?\\s+)*";
52
+ var SHIP_RE = new RegExp(STMT_START + "(" + GIT_FLAGS + "(?:merge\\s+--squash|push)(?=\\s|$)|gh\\s+pr\\s+create)");
53
+ var DISCARD_RE = new RegExp(STMT_START + "(" + GIT_FLAGS + "(?:worktree\\s+remove|branch\\s+-D)(?=\\s|$))");
54
+ var SQUASH_RE = new RegExp("^" + GIT_FLAGS + "merge\\s+--squash");
55
+
56
+ // extensions/lib/telemetry-ship.ts
57
+ var BASE_REFS = ["origin/HEAD", "main", "master"];
58
+
59
+ // extensions/lib/telemetry-record.ts
60
+ import { Document, isCollection, isMap, isSeq, parse as parseYaml, stringify as stringifyYaml } from "yaml";
61
+ var emptyAccumulators = () => ({ phases: {}, personas: {}, reviews: {}, spec_writes: {}, gates: { spec_rounds: 0, plan_rounds: 0, fix_round_grants: 0, task_reopens: 0 }, conformance_loops: 0, amendments: 0, spec_edits_after_ship: 0, events_dropped: 0 });
62
+ function newRecord(o) {
63
+ return { schema: 1, spec: o.spec, run_id: o.runId, branch: o.branch, status: "in_progress", created_at: o.now, sessions: [o.session], derived: { duration_s: 0, phases: {}, personas: {}, conformance_loops: 0, gates: { spec_rounds: 0, plan_rounds: 0, fix_round_grants: 0, task_reopens: 0 }, amendments: 0, spec_edits_after_ship: 0, spec_writes: {}, events_dropped: 0 }, accumulators: {}, events: [] };
64
+ }
65
+ var stripUndefined = (v) => {
66
+ if (Array.isArray(v)) return v.map(stripUndefined);
67
+ if (v && typeof v === "object") return Object.fromEntries(Object.entries(v).filter(([, x]) => x !== void 0).map(([k, x]) => [k, stripUndefined(x)]));
68
+ return v;
69
+ };
70
+ var flowLeafChildren = (map, blockLists = /* @__PURE__ */ new Set()) => {
71
+ for (const pair of map.items) {
72
+ const key = String(pair.key);
73
+ const child = pair.value;
74
+ if (!isCollection(child) || blockLists.has(key)) continue;
75
+ if (isMap(child) && child.items.length > 0 && child.items.every((item) => isMap(item.value))) {
76
+ for (const item of child.items) if (isCollection(item.value)) item.value.flow = true;
77
+ } else {
78
+ child.flow = true;
79
+ }
80
+ }
81
+ };
82
+ var flowModelLists = (node) => {
83
+ if (isMap(node)) {
84
+ for (const pair of node.items) {
85
+ if (String(pair.key) === "models" && isSeq(pair.value)) pair.value.flow = true;
86
+ flowModelLists(pair.value);
87
+ }
88
+ } else if (isSeq(node)) {
89
+ for (const item of node.items) flowModelLists(item);
90
+ }
91
+ };
92
+ function serializeRecord(rec) {
93
+ const { derived, accumulators, events, ...head } = rec;
94
+ const body = stripUndefined({ ...head, derived, accumulators });
95
+ const doc = new Document(body);
96
+ const derivedNode = doc.get("derived", true);
97
+ if (isMap(derivedNode)) flowLeafChildren(derivedNode, /* @__PURE__ */ new Set(["modified_files"]));
98
+ const accumulatorNode = doc.get("accumulators", true);
99
+ if (isMap(accumulatorNode)) {
100
+ for (const session of accumulatorNode.items) if (isMap(session.value)) flowLeafChildren(session.value);
101
+ }
102
+ flowModelLists(doc.contents);
103
+ const bodyText = doc.toString({ lineWidth: 0 });
104
+ const eventLines = events.map((e) => " - " + stringifyYaml(stripUndefined(e), { collectionStyle: "flow", lineWidth: 0 }).trim());
105
+ return bodyText + "events:\n" + (eventLines.length ? eventLines.join("\n") + "\n" : "");
106
+ }
107
+ function parseRecord(text) {
108
+ let doc;
109
+ try {
110
+ doc = parseYaml(text);
111
+ } catch {
112
+ return void 0;
113
+ }
114
+ if (!doc || typeof doc !== "object") return void 0;
115
+ const d = doc;
116
+ if (d.schema !== 1 || typeof d.spec !== "string" || typeof d.run_id !== "string") return void 0;
117
+ const accumulators = Object.fromEntries(Object.entries(d.accumulators ?? {}).flatMap(
118
+ ([session, block]) => block && typeof block === "object" && !Array.isArray(block) ? [[session, { ...emptyAccumulators(), ...block }]] : []
119
+ ));
120
+ return { ...d, sessions: Array.isArray(d.sessions) ? d.sessions : [], derived: d.derived ?? newRecord({ spec: d.spec, session: "", now: d.created_at ?? "", runId: d.run_id }).derived, accumulators, events: Array.isArray(d.events) ? d.events : [] };
121
+ }
18
122
 
19
- const usage = () => {
123
+ // src/bins/gauntlet-telemetry-salvage.mjs
124
+ var GIT_TIMEOUT_MS = 1e4;
125
+ var COMMIT_TIMEOUT_MS = Number(process.env.GAUNTLET_SALVAGE_COMMIT_TIMEOUT_MS) || 3e4;
126
+ var GIT_ENV = { ...process.env, GIT_TERMINAL_PROMPT: "0", GIT_EDITOR: "true" };
127
+ var usage = () => {
20
128
  process.stderr.write("usage: gauntlet-telemetry-salvage --worktree <abs path> [--base <ref>] [--dir <telemetry dir>] [--check]\n");
21
129
  process.exit(1);
22
130
  };
23
-
24
131
  function parseArgs(argv) {
25
- const opts = { worktree: undefined, base: undefined, dir: undefined, check: false };
132
+ const opts = { worktree: void 0, base: void 0, dir: void 0, check: false };
26
133
  for (let i = 0; i < argv.length; i++) {
27
134
  const a = argv[i];
28
- if (a === "--worktree" && argv[i + 1] !== undefined && !argv[i + 1].startsWith("--")) opts.worktree = argv[++i];
29
- else if (a === "--base" && argv[i + 1] !== undefined && !argv[i + 1].startsWith("--")) opts.base = argv[++i];
30
- else if (a === "--dir" && argv[i + 1] !== undefined && !argv[i + 1].startsWith("--")) opts.dir = argv[++i];
135
+ if (a === "--worktree" && argv[i + 1] !== void 0 && !argv[i + 1].startsWith("--")) opts.worktree = argv[++i];
136
+ else if (a === "--base" && argv[i + 1] !== void 0 && !argv[i + 1].startsWith("--")) opts.base = argv[++i];
137
+ else if (a === "--dir" && argv[i + 1] !== void 0 && !argv[i + 1].startsWith("--")) opts.dir = argv[++i];
31
138
  else if (a === "--check") opts.check = true;
32
139
  else usage();
33
140
  }
34
141
  if (!opts.worktree || !isAbsolute(opts.worktree)) usage();
35
142
  return opts;
36
143
  }
37
-
38
144
  function git(cwd, args, timeout = GIT_TIMEOUT_MS) {
39
145
  const r = spawnSync("git", args, { cwd, encoding: "utf8", env: GIT_ENV, timeout });
40
146
  const timedOut = r.error?.code === "ETIMEDOUT";
41
147
  const stderr = timedOut ? "timed out" : (r.stderr ?? "").trim().split("\n")[0] || r.error?.message || `git exited ${r.status}`;
42
148
  return { ok: !timedOut && r.status === 0, stdout: (r.stdout ?? "").trim(), stderr, timedOut };
43
149
  }
44
-
45
150
  function readLayer(file) {
46
151
  if (!existsSync(file)) return {};
47
152
  try {
48
153
  return JSON.parse(readFileSync(file, "utf8"));
49
154
  } catch (e) {
50
- process.stderr.write(`warning: ${file}: ${e.message}; using {} for this layer\n`);
155
+ process.stderr.write(`warning: ${file}: ${e.message}; using {} for this layer
156
+ `);
51
157
  return {};
52
158
  }
53
159
  }
54
-
55
- // Same two layers the recorder reads (gauntlet-settings-loader.ts), without the pi import.
56
160
  function telemetrySettings(root) {
57
161
  const agentDir = process.env.PI_CODING_AGENT_DIR || join(homedir(), ".pi", "agent");
58
162
  const preset = readLayer(join(agentDir, "settings.json"));
59
163
  const repo = readLayer(join(root, ".pi", "settings.json"));
60
164
  return resolveTelemetry(mergeGauntlet(preset?.piGauntlet, repo?.piGauntlet));
61
165
  }
62
-
63
166
  function resolveBase(root, explicit) {
64
167
  for (const ref of explicit ? [explicit] : BASE_REFS) {
65
168
  if (git(root, ["rev-parse", "--verify", "-q", `${ref}^{commit}`]).ok) return ref;
66
169
  }
67
- return undefined;
170
+ return void 0;
171
+ }
172
+ var unfinished = (rec) => {
173
+ const phases = rec?.derived?.phases;
174
+ return rec?.status === "in_progress" && (phases === null || typeof phases !== "object" || !("ship" in phases));
175
+ };
176
+ var stamped = (rec, now) => serializeRecord({ ...rec, status: "shipped", shipped_at: now });
177
+ function stampPresent(root, rec, check) {
178
+ const line = `present ${rec}`;
179
+ const abs = join(root, rec);
180
+ if (!existsSync(abs) || !git(root, ["diff", "--quiet", "HEAD", "--", rec]).ok || !git(root, ["diff", "--quiet", "--cached", "HEAD", "--", rec]).ok) return line;
181
+ const parsed = parseRecord(readFileSync(abs, "utf8"));
182
+ if (!unfinished(parsed)) return line;
183
+ if (check) return `unfinished ${rec}`;
184
+ const prior = readFileSync(abs);
185
+ const failed = (reason) => {
186
+ writeFileSync(abs, prior);
187
+ git(root, ["reset", "-q", "--", rec]);
188
+ return `restore failed ${rec}: ${reason}`;
189
+ };
190
+ writeFileSync(abs, stamped(parsed, (/* @__PURE__ */ new Date()).toISOString()));
191
+ const add = git(root, ["add", "-f", "--", rec]);
192
+ if (!add.ok) return failed(add.stderr);
193
+ const commit = git(root, ["commit", "-q", "-m", `telemetry: mark ${rec} shipped at landing`, "--", rec], COMMIT_TIMEOUT_MS);
194
+ if (!commit.ok) return failed(commit.stderr);
195
+ return `${line} (marked shipped)`;
68
196
  }
69
-
70
197
  function salvage(root, rec, base, check) {
71
198
  const presence = git(root, ["cat-file", "-e", `HEAD:${rec}`]);
72
199
  if (presence.timedOut) return `restore failed ${rec}: timed out`;
73
- if (presence.ok) return `present ${rec}`;
200
+ if (presence.ok) return stampPresent(root, rec, check);
74
201
  const history = git(root, ["log", `${base}..HEAD`, "--grep=^telemetry: ", "-1", "--format=%H"]);
75
202
  if (history.timedOut) return `restore failed ${rec}: timed out`;
76
203
  if (!history.stdout) return `no telemetry run ${rec}`;
@@ -79,17 +206,20 @@ function salvage(root, rec, base, check) {
79
206
  const del = deletion.stdout;
80
207
  if (!del) return `never written ${rec}`;
81
208
  if (check) return `stripped ${rec} in ${del}`;
82
-
83
209
  const abs = join(root, rec);
84
210
  const onDisk = existsSync(abs);
85
211
  const preStaged = git(root, ["ls-files", "--", rec]).stdout !== "";
86
- // A pre-staged record is user-owned and can only be committed when the worktree matches it.
87
212
  if (preStaged && (!onDisk || !git(root, ["diff", "--quiet", "--", rec]).ok)) {
88
213
  return `restore failed ${rec}: staged copy differs from worktree`;
89
214
  }
90
215
  let created = false;
91
216
  let stagedByScript = false;
217
+ let priorBytes;
92
218
  const rollback = () => {
219
+ if (priorBytes !== void 0) {
220
+ writeFileSync(abs, priorBytes);
221
+ if (preStaged) git(root, ["add", "-f", "--", rec]);
222
+ }
93
223
  if (stagedByScript) git(root, ["reset", "-q", "--", rec]);
94
224
  if (created) rmSync(abs, { force: true });
95
225
  };
@@ -97,36 +227,39 @@ function salvage(root, rec, base, check) {
97
227
  rollback();
98
228
  return `restore failed ${rec}: ${reason}`;
99
229
  };
100
-
101
230
  if (!onDisk) {
102
231
  const co = git(root, ["checkout", `${del}^`, "--", rec]);
103
232
  if (!co.ok) return `restore failed ${rec}: ${co.stderr}`;
104
233
  created = true;
105
- stagedByScript = true;
106
- } else if (!preStaged) {
107
- const add = git(root, ["add", "-f", "--", rec]);
108
- if (!add.ok) return failed(add.stderr);
109
- stagedByScript = true;
110
234
  }
235
+ stagedByScript = !preStaged;
236
+ let suffix = "";
237
+ const parsed = parseRecord(readFileSync(abs, "utf8"));
238
+ if (unfinished(parsed)) {
239
+ priorBytes = readFileSync(abs);
240
+ writeFileSync(abs, stamped(parsed, (/* @__PURE__ */ new Date()).toISOString()));
241
+ suffix = " (marked shipped)";
242
+ }
243
+ const add = git(root, ["add", "-f", "--", rec]);
244
+ if (!add.ok) return failed(add.stderr);
111
245
  const commit = git(root, ["commit", "-q", "-m", `telemetry: restore record stripped in ${del.slice(0, 10)}`, "--", rec], COMMIT_TIMEOUT_MS);
112
246
  if (!commit.ok) return failed(commit.stderr);
113
- return `restored ${rec} from ${del}`;
247
+ return `restored ${rec} from ${del}${suffix}`;
114
248
  }
115
-
116
249
  function main() {
117
250
  const opts = parseArgs(process.argv.slice(2));
118
- const override = opts.dir === undefined
119
- ? undefined
120
- : resolveTelemetry({ telemetry: { enabled: true, dir: opts.dir } });
251
+ const override = opts.dir === void 0 ? void 0 : resolveTelemetry({ telemetry: { enabled: true, dir: opts.dir } });
121
252
  if (override?.warning) usage();
122
253
  const top = git(opts.worktree, ["rev-parse", "--show-toplevel"]);
123
254
  if (!existsSync(opts.worktree) || !top.ok) {
124
- process.stderr.write(`not a git worktree: ${opts.worktree}\n`);
255
+ process.stderr.write(`not a git worktree: ${opts.worktree}
256
+ `);
125
257
  return;
126
258
  }
127
259
  const root = top.stdout;
128
260
  const telemetry = telemetrySettings(root);
129
- if (telemetry.warning) process.stderr.write(`warning: ${telemetry.warning}\n`);
261
+ if (telemetry.warning) process.stderr.write(`warning: ${telemetry.warning}
262
+ `);
130
263
  if (!telemetry.enabled) return console.log("telemetry disabled");
131
264
  const dir = override?.dir ?? telemetry.dir;
132
265
  const base = resolveBase(root, opts.base);
@@ -137,5 +270,4 @@ function main() {
137
270
  if (specs.length === 0) return console.log("no spec on branch");
138
271
  for (const spec of specs) console.log(salvage(root, recordPathFor(dir, spec), base, opts.check));
139
272
  }
140
-
141
273
  main();
@@ -11,6 +11,11 @@
11
11
  const REC = ".pi/gauntlet/telemetry/doc/specs/x.yaml";
12
12
  const RECORD_V1 = "schema: 1\nspec: doc/specs/x.md\nstatus: in_progress\n";
13
13
  const RECORD_V2 = "schema: 1\nspec: doc/specs/x.md\nstatus: shipped\n";
14
+ // Records parseRecord accepts (run_id present). UNFINISHED: in_progress, no ship phase.
15
+ const UNFINISHED = "schema: 1\nspec: doc/specs/x.md\nrun_id: r-1\nstatus: in_progress\ncreated_at: 2026-09-17T16:11:45Z\nderived:\n duration_s: 60\n phases:\n brainstorm: { started_at: 2026-09-17T16:11:45Z, duration_s: 60 }\n";
16
+ const SHIP_PHASE = UNFINISHED.replace(" brainstorm:", " ship: { started_at: 2026-09-17T16:12:45Z }\n brainstorm:");
17
+ const SCALAR_PHASES = UNFINISHED.replace(" phases:\n brainstorm: { started_at: 2026-09-17T16:11:45Z, duration_s: 60 }", " phases: nope");
18
+ const FINISHED = UNFINISHED.replace("status: in_progress", "status: shipped\nshipped_at: 2026-09-17T18:56:13Z");
14
19
 
15
20
  const git = (cwd, args, env = {}) =>
16
21
  spawnSync("git", ["-c", "user.name=t", "-c", "user.email=t@t", "-c", "commit.gpgsign=false", ...args], {
@@ -32,7 +37,7 @@
32
37
  return out(root, ["rev-parse", "HEAD"]);
33
38
  };
34
39
  // main: README only. feat: spec (+ record when withRecord).
35
- const repo = ({ withRecord = true } = {}) => {
40
+ const repo = ({ withRecord = true, record = RECORD_V1 } = {}) => {
36
41
  const root = mkdtempSync(join(tmpdir(), "gts-"));
37
42
  git(root, ["init", "-q", "-b", "main"]);
38
43
  write(root, "README.md", "# fixture\n");
@@ -41,7 +46,7 @@
41
46
  write(root, SPEC, "# Spec X\n\n**Goal:** g.\n");
42
47
  commit(root, "Add spec", SPEC);
43
48
  if (withRecord) {
44
- write(root, REC, RECORD_V1);
49
+ write(root, REC, record);
45
50
  commit(root, "telemetry: doc/specs/x.md", REC);
46
51
  }
47
52
  return root;
@@ -348,6 +353,103 @@ PATH="${process.env.PATH.split(":").filter((part) => part !== bin).join(":")}" e
348
353
  assert.equal(out(remote, ["for-each-ref"]), remoteBefore);
349
354
  });
350
355
 
356
+ test("present + unfinished -> stamped shipped, one telemetry: commit, clean tree", (t) => {
357
+ const root = repo({ record: UNFINISHED }); cleanup(t, root);
358
+ const n = commitCount(root);
359
+ const r = run(root);
360
+ assert.equal(r.status, 0, r.stderr);
361
+ assert.deepEqual(r.lines, [`present ${REC} (marked shipped)`]);
362
+ const text = readFileSync(join(root, REC), "utf8");
363
+ assert.match(text, /^status: shipped$/m);
364
+ assert.match(text, /^shipped_at: \d{4}-\d{2}-\d{2}T/m);
365
+ assert.doesNotMatch(text, /in_progress/);
366
+ assert.equal(commitCount(root), n + 1);
367
+ assert.equal(headSubject(root), `telemetry: mark ${REC} shipped at landing`);
368
+ assert.deepEqual(headFiles(root), [REC]);
369
+ assert.equal(porcelain(root), "");
370
+ });
371
+
372
+ test("present + scalar phases -> stamped shipped, exit 0", (t) => {
373
+ const root = repo({ record: SCALAR_PHASES }); cleanup(t, root);
374
+ const r = run(root);
375
+ assert.equal(r.status, 0, r.stderr);
376
+ assert.deepEqual(r.lines, [`present ${REC} (marked shipped)`]);
377
+ assert.match(readFileSync(join(root, REC), "utf8"), /^status: shipped$/m);
378
+ });
379
+
380
+ test("present + in_progress with a ship phase -> untouched", (t) => {
381
+ const root = repo({ record: SHIP_PHASE }); cleanup(t, root);
382
+ const before = head(root);
383
+ const r = run(root);
384
+ assert.deepEqual(r.lines, [`present ${REC}`]);
385
+ assert.equal(head(root), before);
386
+ assert.equal(readFileSync(join(root, REC), "utf8"), SHIP_PHASE);
387
+ });
388
+
389
+ test("present + unfinished but worktree copy dirty -> untouched, present", (t) => {
390
+ const root = repo({ record: UNFINISHED }); cleanup(t, root);
391
+ write(root, REC, UNFINISHED + "# local edit\n");
392
+ const before = head(root);
393
+ const r = run(root);
394
+ assert.deepEqual(r.lines, [`present ${REC}`]);
395
+ assert.equal(head(root), before);
396
+ assert.equal(readFileSync(join(root, REC), "utf8"), UNFINISHED + "# local edit\n");
397
+ });
398
+
399
+ test("present + already shipped -> byte-identical, no commit", (t) => {
400
+ const root = repo({ record: FINISHED }); cleanup(t, root);
401
+ const before = head(root);
402
+ const r = run(root);
403
+ assert.deepEqual(r.lines, [`present ${REC}`]);
404
+ assert.equal(head(root), before);
405
+ assert.equal(readFileSync(join(root, REC), "utf8"), FINISHED);
406
+ });
407
+
408
+ test("restore with scalar phases -> stamped shipped, exit 0", (t) => {
409
+ const root = repo({ record: SCALAR_PHASES }); cleanup(t, root);
410
+ const del = rmCommit(root, "strip", REC);
411
+ const r = run(root);
412
+ assert.equal(r.status, 0, r.stderr);
413
+ assert.deepEqual(r.lines, [`restored ${REC} from ${del} (marked shipped)`]);
414
+ assert.match(out(root, ["show", `HEAD:${REC}`]), /^status: shipped$/m);
415
+ });
416
+
417
+ test("restore of an unfinished record -> one commit whose bytes carry the stamp", (t) => {
418
+ const root = repo({ record: UNFINISHED }); cleanup(t, root);
419
+ const del = rmCommit(root, "strip", REC);
420
+ write(root, REC, UNFINISHED);
421
+ const n = commitCount(root);
422
+ const r = run(root);
423
+ assert.deepEqual(r.lines, [`restored ${REC} from ${del} (marked shipped)`]);
424
+ assert.equal(commitCount(root), n + 1);
425
+ const committed = out(root, ["show", `HEAD:${REC}`]);
426
+ assert.match(committed, /^status: shipped$/m);
427
+ assert.match(committed, /^shipped_at: /m);
428
+ assert.equal(porcelain(root), "");
429
+ });
430
+
431
+ test("--check on present + unfinished -> unfinished <rec>, nothing changes", (t) => {
432
+ const root = repo({ record: UNFINISHED }); cleanup(t, root);
433
+ const before = head(root);
434
+ const r = run(root, ["--check"]);
435
+ assert.deepEqual(r.lines, [`unfinished ${REC}`]);
436
+ assert.equal(head(root), before);
437
+ assert.equal(readFileSync(join(root, REC), "utf8"), UNFINISHED);
438
+ assert.equal(porcelain(root), "");
439
+ });
440
+
441
+ test("present + unfinished, rejecting pre-commit hook -> restore failed, HEAD bytes and index restored", (t) => {
442
+ const root = repo({ record: UNFINISHED }); cleanup(t, root);
443
+ write(root, ".git/hooks/pre-commit", "#!/bin/sh\nexit 1\n");
444
+ chmodSync(join(root, ".git/hooks/pre-commit"), 0o755);
445
+ const before = head(root);
446
+ const r = run(root);
447
+ assert.match(r.stdout, new RegExp(`^restore failed ${REC.replace(/\./g, "\\.")}: `));
448
+ assert.equal(head(root), before);
449
+ assert.equal(readFileSync(join(root, REC), "utf8"), UNFINISHED);
450
+ assert.equal(porcelain(root), "");
451
+ });
452
+
351
453
  test("finishing Option 1 fixture: strip-on-branch + salvage + merge --squash leaves the record, not the plan, staged", (t) => {
352
454
  const root = repo(); cleanup(t, root);
353
455
  const plan = "doc/plans/x.md";
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-gauntlet",
3
- "version": "5.11.0",
3
+ "version": "5.12.1",
4
4
  "description": "Opinionated, gated workflow skills, subagent personas, and runtime extensions for the pi coding agent.",
5
5
  "author": "Jacek Juraszek",
6
6
  "type": "module",
@@ -33,7 +33,8 @@
33
33
  ],
34
34
  "bin": {
35
35
  "gauntlet-spec-index": "bin/gauntlet-spec-index.mjs",
36
- "gauntlet-telemetry-salvage": "bin/gauntlet-telemetry-salvage.mjs"
36
+ "gauntlet-telemetry-salvage": "bin/gauntlet-telemetry-salvage.mjs",
37
+ "gauntlet-performance": "bin/gauntlet-performance.mjs"
37
38
  },
38
39
  "engines": {
39
40
  "node": ">=24.15.0"
@@ -57,9 +58,13 @@
57
58
  "scripts": {
58
59
  "postinstall": "node ./bin/install-agents.mjs",
59
60
  "link-agents": "node ./bin/install-agents.mjs",
61
+ "build:bins": "node scripts/build-bins.mjs",
60
62
  "test": "node scripts/ci.mjs"
61
63
  },
62
64
  "dependencies": {
63
65
  "yaml": "^2.9.0"
66
+ },
67
+ "devDependencies": {
68
+ "esbuild": "0.28.2"
64
69
  }
65
70
  }
@@ -177,7 +177,7 @@ fi
177
177
  node <bin>/gauntlet-telemetry-salvage.mjs --worktree "$WORKTREE" --base <base-branch>
178
178
  ```
179
179
 
180
- The salvage prints one line per spec on the branch (`present`, `restored <path> from <sha>`, `no telemetry run`, `never written`, `restore failed <path>: <reason>`) and always exits 0. Print its stdout verbatim in the ship completion message. A `restore failed` line is reported, never retried, and never blocks the ship - the record stays recoverable from the branch ref.
180
+ The salvage prints one line per spec on the branch (`present`, `present <path> (marked shipped)`, `restored <path> from <sha>`, `restored <path> from <sha> (marked shipped)`, `no telemetry run`, `never written`, `restore failed <path>: <reason>`) and always exits 0. `(marked shipped)` means the record was still `in_progress` with no ship phase (the recorder lost its binding) and the salvage committed `status: shipped` + `shipped_at` as one `telemetry:` commit; it rides the squash or push like any branch commit. Print its stdout verbatim in the ship completion message. A `restore failed` line is reported, never retried, and never blocks the ship - the record stays recoverable from the branch ref.
181
181
 
182
182
  #### Option 1: Squash-merge to base
183
183
 
@@ -174,6 +174,7 @@ verification command may write to the tree while the Reviewer reads it):
174
174
  squash. On a cell with no push row (fork overlay, report-only states) the same
175
175
  finding is a non-blocking follow-up instead: the record stays recoverable from the
176
176
  PR head ref after merge, and blocking would stop a ship the gate cannot repair.
177
+ `unfinished <path>` (record still `in_progress` with no ship phase) also lands in `## Evidence` as one line and is non-blocking: pre-landing `in_progress` is normal, and the merge course's salvage run stamps it.
177
178
  - **Evidence:** On the CI path, list each satisfying check's name, conclusion, assessed SHA, and run URL - there is no command or raw_tail to paste. On the local path, paste each run's `command` and `raw_tail` verbatim, fenced - never
178
179
  paraphrased. Any authored summary is labeled as a summary and never substitutes for
179
180
  `raw_tail`.
@@ -256,7 +257,7 @@ are never bundled into one selection, with one scoped exception: the selected me
256
257
  course first runs `node <bin>/gauntlet-telemetry-salvage.mjs --worktree <provisioned
257
258
  path> --base origin/<baseRefName>` (no `--check`). `present` -> merge as-is. `restored
258
259
  <path> from <sha>` -> push that single `telemetry: restore` commit as part of this
259
- course, re-fetch `headRefOid`, and pass the new SHA to `--match-head-commit`. `restore
260
+ course, re-fetch `headRefOid`, and pass the new SHA to `--match-head-commit`. A line ending `(marked shipped)` (`present` or `restored`) is handled the same way: push that single `telemetry:` commit, re-fetch `headRefOid`, pass the new SHA. `restore
260
261
  failed` -> merge proceeds, the reason is printed, and the follow-up names recovery
261
262
  from the PR head ref.
262
263
 
@@ -508,7 +509,7 @@ The menu is a state machine, not a one-shot report:
508
509
  reviewed doc edits selected alongside them (one commit, or one per batch
509
510
  sequentially; subjects name the fixes) - then re-resolves the evidence for the new head **once** (the brief's stale-head row: prior evidence is stale; the local command executes only on a fallback/opt-out resolution). Before the push, run
510
511
  `node <bin>/gauntlet-telemetry-salvage.mjs --worktree <provisioned path> --base
511
- origin/<baseRefName>` (no `--check`); a `restored` commit rides the wave's single
512
+ origin/<baseRefName>` (no `--check`); a `restored` or `(marked shipped)` commit rides the wave's single
512
513
  push and the pushed SHA becomes the assessed head under the course's-own-push rule
513
514
  in step 1. Print its stdout in the re-rendered report's `## Evidence`. **On
514
515
  green**, push **once**; gate and push are per-wave invariants, never per-fix or
@@ -0,0 +1,43 @@
1
+ ---
2
+ name: gauntlet-performance
3
+ description: Use when a human asks how gauntlet runs perform across the recorded telemetry - explicit invocation only (/skill:gauntlet-performance [--dir <path>]... [--since <version>]).
4
+ disable-model-invocation: true
5
+ ---
6
+
7
+ # Gauntlet Performance
8
+
9
+ The CLI parses and aggregates; you reason. Never open a telemetry YAML record yourself.
10
+
11
+ ## Run the digest
12
+
13
+ 1. Resolve the CLI: `<directory of this SKILL.md>/../../bin/gauntlet-performance.mjs`.
14
+ 2. Run it from the repo root, passing the user's `--dir` and `--since` arguments verbatim:
15
+ `node <bin>/gauntlet-performance.mjs [--dir <path>]... [--since <version>]`.
16
+ Default corpus is the current repo; each `--dir` adds a repo root or a telemetry dir.
17
+ 3. If the command is missing or the output contains `no records found`, relay that line and stop - no menu.
18
+
19
+ ## Read the digest
20
+
21
+ `runs` has one row per record; `shipped*` marks a truncated run (no ship phase recorded - salvage stamped it at landing; its `wall` is `-` and it feeds no aggregate). `by version` groups by pi-gauntlet version: `n` counts every row, `shipped` counts the rows behind the p50/max columns. `grants` is `fix_round_grants` - the fix-round proxy (human-granted extra review rounds); schema 1 has no code-review round count.
22
+
23
+ ## Reply - exactly this, in this order
24
+
25
+ 1. **Recommendation** (2-4 sentences). One claim, led by the run that exemplifies it: quote its `spec` slug, `run_id`, and the 1-3 numbers that carry the claim. When `grants` is the evidence, call it the fix-round proxy. If no version group has `shipped >= 2`, the recommendation is "sample too small" with the `n`/`shipped` counts per version.
26
+ 2. **Cornerstones**: 3-5 bullets of aggregate facts from `by version` - corpus size, truncated count, the p50s and model tallies that moved between versions.
27
+ 3. **Menu**, numbered, at most 3 items, rendered exactly as:
28
+ - `1. render report` - ask for a target path; write markdown there: the digest verbatim, then the recommendation and cornerstones above. If the file exists, ask before overwriting. Write nothing unless this item is chosen.
29
+ - `2. open recommendation as ticket` - hand the claim and its numbers to `/skill:shape-ticket`; never create a ticket directly.
30
+ - `3. drill into <slug>` - re-run the CLI with `--json` and show that run's fields.
31
+
32
+ Nothing else: no preamble, no restated digest, no file written before item 1 is chosen.
33
+
34
+ ## Project overrides
35
+
36
+ If a gauntlet overrides file exists - checked in order:
37
+ `.pi/gauntlet-overrides.md`, `<repo root>/gauntlet-overrides.md`,
38
+ `<repo root>/doc/gauntlet-overrides.md`; first found wins - read it. Any
39
+ sections relevant to this skill - by name match, by topic (routing,
40
+ verification, worktrees, etc.), or by workflow convention - override or
41
+ extend the instructions above. Project-local `AGENTS.md` is already in
42
+ context - check it for project-specific routing tables, service paths, and
43
+ verification commands.