pi-gauntlet 5.18.7 → 5.20.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +16 -0
- package/README.md +5 -2
- package/bin/gauntlet-telemetry-seal.mjs +346 -0
- package/bin/gauntlet-telemetry-seal.test.mjs +271 -0
- package/extensions/lib/telemetry-diff.test.ts +108 -0
- package/extensions/lib/telemetry-diff.ts +61 -0
- package/extensions/lib/telemetry-paths.test.ts +11 -35
- package/extensions/lib/telemetry-paths.ts +13 -42
- package/extensions/lib/telemetry-record.test.ts +0 -6
- package/extensions/lib/telemetry-ship.test.ts +1 -5
- package/extensions/lib/telemetry-ship.ts +1 -4
- package/extensions/telemetry.test.ts +243 -135
- package/extensions/telemetry.ts +148 -201
- package/package.json +2 -2
- package/skills/brainstorming/SKILL.md +1 -1
- package/skills/finishing-a-development-branch/SKILL.md +19 -16
- package/skills/gatekeep-pr/reference/assessment.md +0 -13
- package/skills/gatekeep-pr/reference/decision-menu.md +1 -2
- package/skills/gatekeep-pr/reference/post-selection-loop.md +3 -3
- package/skills/gauntlet-performance/SKILL.md +1 -1
- package/skills/gauntlet-resume/SKILL.md +46 -32
- package/skills/gauntlet-resume/reference/brief-contract.md +12 -8
- package/skills/gauntlet-resume/reference/reconstruction.md +10 -3
- package/skills/gauntlet-resume/reference/seed.md +47 -0
- package/bin/gauntlet-telemetry-salvage.mjs +0 -273
- package/bin/gauntlet-telemetry-salvage.test.mjs +0 -466
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,21 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## v5.20.0 - 2026-09-26
|
|
4
|
+
|
|
5
|
+
### Changed
|
|
6
|
+
|
|
7
|
+
- Telemetry is gauntlet-bound: the recorder binds only after `phase_tracker start brainstorm` armed the flow (a spec written outside the gauntlet gets no record), never stages or commits the record, and hides it through `<git-common-dir>/info/exclude` until `finishing-a-development-branch` seals it. The new `gauntlet-telemetry-seal` bin replaces `gauntlet-telemetry-salvage`: it stamps `shipped`, computes the diff, re-derives, and commits the record once before the PR push or the squash; a nonzero exit stops the landing. `gatekeep-pr` no longer mentions telemetry. Records are no longer marked `abandoned` on discard, and the recorder does not bind to already sealed records on disk; `spec_edits_after_ship` is no longer produced (the field remains declared and old records parse). Plain jj workspaces keep sealing on `phase complete ship`. In-flight branches with a committed record show it as modified until the seal.
|
|
8
|
+
|
|
9
|
+
## v5.19.0 - 2026-09-26
|
|
10
|
+
|
|
11
|
+
### Added
|
|
12
|
+
|
|
13
|
+
- `gauntlet-resume` accepts a spec seed: a one-token `.md` file tracked on main under a configured spec dir. The route (`skills/gauntlet-resume/reference/seed.md`) checks primary session, tracked, and not shipped, derives the branch name from the spec basename minus its date, reuses the worktree registered for that branch or creates it through `/skill:using-git-worktrees`, pins the spec, and continues in reconstruction. Reconstruction pairs a candidate plan with a spec tracked from base; a plan already at base reports `plan predates base; no post-base task evidence`.
|
|
14
|
+
|
|
15
|
+
### Changed
|
|
16
|
+
|
|
17
|
+
- `gauntlet-resume` arguments classify by token count then content: a file whose line 1 starts `# Handoff:` is a brief regardless of extension, a `.md` under a spec dir is a seed, an existing git worktree root or bare name is a worktree. "Never creates a worktree" narrows to "never `git worktree add` directly". Supersedes the matching clauses of `doc/specs/2026-09-17-gh-31-gauntlet-resume.md` and `doc/specs/2026-09-20-gh-40-gauntlet-handoff-shared-brief-contract.md`.
|
|
18
|
+
|
|
3
19
|
## v5.18.7 - 2026-09-24
|
|
4
20
|
|
|
5
21
|
### Added
|
package/README.md
CHANGED
|
@@ -69,9 +69,9 @@ Everything between gate 1 and gate 2 - task breakdown, implementation, both revi
|
|
|
69
69
|
|
|
70
70
|
pi-gauntlet ships three kinds of pieces, layered on top of pi-cohort's dispatch:
|
|
71
71
|
|
|
72
|
-
- **20 skills** - the workflow logic. Thirteen activate automatically when pi sees the matching kind of task, and each one gates the next: `brainstorming`, `writing-plans`, `roasting-the-spec`, `test-driven-development`, `subagent-driven-development`, `dispatching-parallel-agents`, `verification-before-completion`, `requesting-code-review`, `receiving-code-review`, `using-git-worktrees`, `finishing-a-development-branch`, `writing-skills`, `linear` (reads/searches/comments on/manages Linear tickets via the `linearis` CLI; owns all linearis mechanics and the `## Issue tracker` overrides schema; tracker-facing skills route to it). Seven more are explicit-invocation-only (`disable-model-invocation: true`): `shape-ticket` creates or repairs one tracker issue per run against a Context/Problem/Idea/Acceptance-Criteria template, gated by an AC integrity check, a cheap council roast, and a single human-confirmed write - run it with `/skill:shape-ticket`. `gatekeep-pr` is consent-gated pre-merge verification of a PR against its issue - read-only gathering, verification evidence resolved CI-first (green checks on the exact assessed head count as evidence; the project's verification command runs only as fallback), a rubric-based review, then a deterministic authorship-aware menu with stable finding IDs (P#/L#/C#/F#) and numbered pre-composed courses (fixes execute as a single parallel-safe wave: one gate run, one re-review, one push); nothing mutates (fixes, pushes, reviews, merges) until you pick a row - run it with `/skill:gatekeep-pr <pr>`. `check-delivery` is a post-merge detective control: proves an issue actually shipped (default-branch landing, delivery target, per-AC evidence) before its tracker status advances; it never writes a terminal status - run it with `/skill:check-delivery <ref>`. `chase-bug` is human-only bug triage: read-only root-cause discovery to an evidenced verdict menu (real bug -> ticket/brainstorm/hotfix/respond; five negative verdicts), then a gated response to the reporter for addressable origins (GitHub issue / tracker ticket) and a rendered verdict summary otherwise - it never fixes during triage; the hotfix row hands off to `skills/chase-bug/hotfix.md` after the menu - run it with `/skill:chase-bug`. `gauntlet-performance` reads the committed run telemetry (current repo by default, `--dir <path>` adds others) through the parse-only `gauntlet-performance` CLI and answers with one example-led recommendation, 3-5 cornerstone numbers, and a menu of at most three actions (render a report file, open the recommendation as a ticket via `shape-ticket`, drill into one run); it writes nothing unless you pick render - run it with `/skill:gauntlet-performance [--dir <path>]... [--since <version>]`. `gauntlet-handoff` ends a session whose flow a fresh session will continue: it invokes pi-cohort's `handoff` skill (`--out <path>` or `--key <name>`, default key = the run worktree's branch with `/` flattened to `-`, default file `<tmpdir>/pi-handoff/<key>.md`), then appends the gauntlet process-state section (phase/plan tracker status) per the shared contract `skills/gauntlet-resume/reference/brief-contract.md` - run it with `/skill:gauntlet-handoff [--out <path> | --key <name>]`. `gauntlet-resume` is the only way back into an interrupted flow from a fresh session: it takes a `gauntlet-handoff` brief (a file path, pasted text, or - with no arguments - a pick from the briefs under `<tmpdir>/pi-handoff/`)
|
|
72
|
+
- **20 skills** - the workflow logic. Thirteen activate automatically when pi sees the matching kind of task, and each one gates the next: `brainstorming`, `writing-plans`, `roasting-the-spec`, `test-driven-development`, `subagent-driven-development`, `dispatching-parallel-agents`, `verification-before-completion`, `requesting-code-review`, `receiving-code-review`, `using-git-worktrees`, `finishing-a-development-branch`, `writing-skills`, `linear` (reads/searches/comments on/manages Linear tickets via the `linearis` CLI; owns all linearis mechanics and the `## Issue tracker` overrides schema; tracker-facing skills route to it). Seven more are explicit-invocation-only (`disable-model-invocation: true`): `shape-ticket` creates or repairs one tracker issue per run against a Context/Problem/Idea/Acceptance-Criteria template, gated by an AC integrity check, a cheap council roast, and a single human-confirmed write - run it with `/skill:shape-ticket`. `gatekeep-pr` is consent-gated pre-merge verification of a PR against its issue - read-only gathering, verification evidence resolved CI-first (green checks on the exact assessed head count as evidence; the project's verification command runs only as fallback), a rubric-based review, then a deterministic authorship-aware menu with stable finding IDs (P#/L#/C#/F#) and numbered pre-composed courses (fixes execute as a single parallel-safe wave: one gate run, one re-review, one push); nothing mutates (fixes, pushes, reviews, merges) until you pick a row - run it with `/skill:gatekeep-pr <pr>`. `check-delivery` is a post-merge detective control: proves an issue actually shipped (default-branch landing, delivery target, per-AC evidence) before its tracker status advances; it never writes a terminal status - run it with `/skill:check-delivery <ref>`. `chase-bug` is human-only bug triage: read-only root-cause discovery to an evidenced verdict menu (real bug -> ticket/brainstorm/hotfix/respond; five negative verdicts), then a gated response to the reporter for addressable origins (GitHub issue / tracker ticket) and a rendered verdict summary otherwise - it never fixes during triage; the hotfix row hands off to `skills/chase-bug/hotfix.md` after the menu - run it with `/skill:chase-bug`. `gauntlet-performance` reads the committed run telemetry (current repo by default, `--dir <path>` adds others) through the parse-only `gauntlet-performance` CLI and answers with one example-led recommendation, 3-5 cornerstone numbers, and a menu of at most three actions (render a report file, open the recommendation as a ticket via `shape-ticket`, drill into one run); it writes nothing unless you pick render - run it with `/skill:gauntlet-performance [--dir <path>]... [--since <version>]`. `gauntlet-handoff` ends a session whose flow a fresh session will continue: it invokes pi-cohort's `handoff` skill (`--out <path>` or `--key <name>`, default key = the run worktree's branch with `/` flattened to `-`, default file `<tmpdir>/pi-handoff/<key>.md`), then appends the gauntlet process-state section (phase/plan tracker status) per the shared contract `skills/gauntlet-resume/reference/brief-contract.md` - run it with `/skill:gauntlet-handoff [--out <path> | --key <name>]`. `gauntlet-resume` is the only way back into an interrupted flow from a fresh session: it takes a `gauntlet-handoff` brief (a file path, pasted text, or - with no arguments - a pick from the briefs under `<tmpdir>/pi-handoff/`), a bare worktree that already holds a spec, or a spec tracked on main under a configured spec dir (a spec seed: it creates or reuses the worktree through `using-git-worktrees` and pins the spec), restores phase/plan tracker state through the legal arming sequence (`start brainstorm`, `skip` with `resume:` reasons, `plan_check` before implement-or-later), and never infers approval from artifacts - run it with `/skill:gauntlet-resume [<brief-file> | <spec>.md | <worktree-name-or-path>]`.
|
|
73
73
|
- **7 subagent personas** - the specialized child agents the skills dispatch via pi-cohort: `implementer`, `code-reviewer`, `spec-reviewer`, `conformance-reviewer`, `spec-summarizer`, `spec-council-member`, `spec-council-synthesizer`. See [doc/personas.md](./doc/personas.md) for what each one does and why its permissions are scoped the way they are.
|
|
74
|
-
- **4 runtime extensions** - the enforcement layer. `plan-tracker` and `phase-tracker` are tools skills call to track progress (with a TUI widget); `verify-before-ship` is a hook that warns if you push or open a PR without a passing test run since your last edit; a phase-tracker flow guard reminds on implement-phase commits missing spec/code review. In a brainstorming-entered flow, phase-tracker rejects `implement` or `verify` completion while tracker tasks remain pending or in progress; see [its configuration reference](./doc/configuration.md#phase-tracker). phase-tracker also registers `plan_check`, which verifies a plan against its spec and against the grammar in [skills/writing-plans/reference/plan-contract.md](./skills/writing-plans/reference/plan-contract.md), including that each task's `Tests:` commands are selective and never the full suite; a pass stamps the plan for implementation. `telemetry` records one
|
|
74
|
+
- **4 runtime extensions** - the enforcement layer. `plan-tracker` and `phase-tracker` are tools skills call to track progress (with a TUI widget); `verify-before-ship` is a hook that warns if you push or open a PR without a passing test run since your last edit; a phase-tracker flow guard reminds on implement-phase commits missing spec/code review. In a brainstorming-entered flow, phase-tracker rejects `implement` or `verify` completion while tracker tasks remain pending or in progress; see [its configuration reference](./doc/configuration.md#phase-tracker). phase-tracker also registers `plan_check`, which verifies a plan against its spec and against the grammar in [skills/writing-plans/reference/plan-contract.md](./skills/writing-plans/reference/plan-contract.md), including that each task's `Tests:` commands are selective and never the full suite; a pass stamps the plan for implementation. `telemetry` records one YAML record per gauntlet run (phase timing, models, personas, gate/fix rounds, diff at ship) that ships in the squash beside the spec - it is written only inside a brainstorming-entered flow, kept untracked and git-excluded until `finishing-a-development-branch` seals and commits it once with `gauntlet-telemetry-seal` - and blocks a brainstorm `write` into an already-shipped spec; see [its configuration reference](./doc/configuration.md#telemetry). See [doc/configuration.md](./doc/configuration.md) for the settings each one reads.
|
|
75
75
|
|
|
76
76
|
pi-gauntlet is **opinionated**: every non-trivial change is *meant* to ride this one pipeline, entered through `brainstorming`. Enforcement is opt-in by entry, not ambient: once brainstorming starts a flow, the phase-tracker extension mechanically blocks a phase from closing before its gate runs, and warns once if the main loop writes code during implement (subagents own implement-phase edits). A change made *without* entering the flow (a typo, a formatting run, a dependency bump - see "When to use / when NOT to use") is not gated; the discipline of routing real work through the pipeline is a convention the tooling supports, not a trap it springs on every edit.
|
|
77
77
|
|
|
@@ -90,6 +90,9 @@ Run by a human against pi-cohort >= 7.1.0, before a pi-gauntlet release claims t
|
|
|
90
90
|
3. Hotfix context with non-pending trackers: the brief ends at `## Skills loaded`; resume takes the `chase-bug` route.
|
|
91
91
|
4. Fresh session, zero arguments: the listing shows the briefs from 1-3 and a pick restores; `/skill:gauntlet-resume /nonexistent.md` stops naming that path without listing anything.
|
|
92
92
|
5. Branch `hotfix/x`: the default file is `pi-handoff/hotfix-x.md` and appears in the zero-argument listing.
|
|
93
|
+
6. Primary session, idle trackers, spec tracked on main: `/skill:gauntlet-resume doc/specs/<x>.md` creates the worktree and asks the approval question; approval leaves `phase_tracker` at plan in_progress with brainstorm `⊘ (resume: ...)`.
|
|
94
|
+
7. Fresh idle session, same seed: the registered worktree is reused (no creation) and the question repeats. Run once on a consumer whose `## using-git-worktrees` override lays worktrees out as sibling dirs, once on `.worktrees/`.
|
|
95
|
+
8. After writing-plans commits the plan beside the seeded spec: `/skill:gauntlet-handoff` then a fresh `/skill:gauntlet-resume <brief>` restores plan phase; a fresh `/skill:gauntlet-resume <worktree>` reaches "Spec with plan" with task evidence from the plan commit.
|
|
93
96
|
|
|
94
97
|
## Key concepts
|
|
95
98
|
|
|
@@ -0,0 +1,346 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
|
|
3
|
+
// src/bins/gauntlet-telemetry-seal.mjs
|
|
4
|
+
import { existsSync, readFileSync, realpathSync, writeFileSync } from "node:fs";
|
|
5
|
+
import { homedir } from "node:os";
|
|
6
|
+
import { isAbsolute as isAbsolute2, join } from "node:path";
|
|
7
|
+
import { spawnSync } from "node:child_process";
|
|
8
|
+
import process from "node:process";
|
|
9
|
+
|
|
10
|
+
// extensions/lib/gauntlet-settings.ts
|
|
11
|
+
import path from "node:path";
|
|
12
|
+
function mergeGauntlet(preset, repo) {
|
|
13
|
+
return { ...preset ?? {}, ...repo ?? {} };
|
|
14
|
+
}
|
|
15
|
+
var nonEmptyString = (v) => typeof v === "string" && v.trim().length > 0;
|
|
16
|
+
var joinWarn = (ws) => ws.length ? ws.join("; ") : void 0;
|
|
17
|
+
var DEFAULT_TELEMETRY_DIR = ".pi/gauntlet/telemetry";
|
|
18
|
+
var DEFAULT_TELEMETRY_BUCKETS = [
|
|
19
|
+
["test", ["**/test/**", "**/tests/**", "**/__tests__/**", "**/*.test.*", "**/*.spec.*", "**/*_test.*"]],
|
|
20
|
+
["docs", ["**/*.md"]],
|
|
21
|
+
["config", ["**/*.json", "**/*.yaml", "**/*.yml", "**/*.toml", "**/*.lock", "**/*-lock.*"]]
|
|
22
|
+
];
|
|
23
|
+
function resolveTelemetry(g) {
|
|
24
|
+
const t = g.telemetry;
|
|
25
|
+
const warnings = [];
|
|
26
|
+
const enabled = t?.enabled !== false;
|
|
27
|
+
let dir = DEFAULT_TELEMETRY_DIR;
|
|
28
|
+
if (t?.dir !== void 0) {
|
|
29
|
+
const value = nonEmptyString(t.dir) ? t.dir.trim().replace(/\/+$/, "") : "";
|
|
30
|
+
const canonical = value.replace(/\\/g, "/");
|
|
31
|
+
const normalized = path.posix.normalize(canonical);
|
|
32
|
+
if (value && path.win32.parse(canonical).root === "" && normalized !== ".." && !normalized.startsWith("../")) dir = normalized;
|
|
33
|
+
else warnings.push("telemetry.dir must be a non-empty path relative to the git toplevel; using the default");
|
|
34
|
+
}
|
|
35
|
+
let buckets = DEFAULT_TELEMETRY_BUCKETS;
|
|
36
|
+
if (t?.buckets !== void 0) {
|
|
37
|
+
const b = t.buckets;
|
|
38
|
+
const valid = b !== null && typeof b === "object" && !Array.isArray(b) && Object.keys(b).length > 0 && Object.values(b).every((v) => Array.isArray(v) && v.length > 0 && v.every(nonEmptyString));
|
|
39
|
+
if (valid) buckets = Object.entries(b).map(([name, globs]) => [name, [...globs]]);
|
|
40
|
+
else warnings.push("telemetry.buckets is not an object of non-empty glob arrays; using the defaults");
|
|
41
|
+
}
|
|
42
|
+
return { enabled, dir, buckets, warning: joinWarn(warnings) };
|
|
43
|
+
}
|
|
44
|
+
|
|
45
|
+
// extensions/lib/telemetry-paths.ts
|
|
46
|
+
import { isAbsolute, matchesGlob, relative, resolve } from "node:path";
|
|
47
|
+
|
|
48
|
+
// extensions/lib/phase-tracker-helpers.ts
|
|
49
|
+
var STMT_START = "(?:^|[\\n;&|(])\\s*";
|
|
50
|
+
|
|
51
|
+
// extensions/lib/telemetry-paths.ts
|
|
52
|
+
var toPosix = (p) => p.split("\\").join("/");
|
|
53
|
+
var isSpecPath = (rel) => /(^|\/)doc\/specs\/[^/]+\.md$/.test(rel);
|
|
54
|
+
var recordPathFor = (dir, specRel) => `${dir}/${specRel.replace(/\.md$/, ".yaml")}`;
|
|
55
|
+
function repoRelativeToolPath(toplevel, cwd, p) {
|
|
56
|
+
const abs = isAbsolute(p) ? p : resolve(cwd, p);
|
|
57
|
+
const rel = toPosix(relative(toplevel, abs));
|
|
58
|
+
if (rel === "" || rel.startsWith("../") || rel === ".." || isAbsolute(rel)) return void 0;
|
|
59
|
+
return rel;
|
|
60
|
+
}
|
|
61
|
+
var GIT_FLAGS = "git\\s+(?:-\\S+(?:\\s+\\S+)?\\s+)*";
|
|
62
|
+
var SHIP_RE = new RegExp(STMT_START + "(" + GIT_FLAGS + "(?:merge\\s+--squash|push)(?=\\s|$)|gh\\s+pr\\s+create)");
|
|
63
|
+
var SQUASH_RE = new RegExp("^" + GIT_FLAGS + "merge\\s+--squash");
|
|
64
|
+
function classifyBucket(rel, buckets) {
|
|
65
|
+
for (const [name, globs] of buckets) if (globs.some((g) => matchesGlob(rel, g))) return name;
|
|
66
|
+
return "code";
|
|
67
|
+
}
|
|
68
|
+
function numstatPath(raw) {
|
|
69
|
+
const braced = raw.replace(/\{([^{}]*) => ([^{}]*)\}/g, (_m, _a, b) => b).replace(/\/\//g, "/");
|
|
70
|
+
const arrow = braced.indexOf(" => ");
|
|
71
|
+
return arrow >= 0 ? braced.slice(arrow + 4) : braced;
|
|
72
|
+
}
|
|
73
|
+
|
|
74
|
+
// extensions/lib/telemetry-ship.ts
|
|
75
|
+
function modifiedFilesFrom(nameOnly, spec, dir) {
|
|
76
|
+
const dirPrefix = dir.replace(/\/+$/, "") + "/";
|
|
77
|
+
return nameOnly.split("\n").filter((f) => f && f !== spec && !/(^|\/)doc\/plans\//.test(f) && !f.startsWith(dirPrefix)).sort();
|
|
78
|
+
}
|
|
79
|
+
|
|
80
|
+
// extensions/lib/telemetry-diff.ts
|
|
81
|
+
function aggregateNumstat(numstat, files, buckets) {
|
|
82
|
+
const out = {};
|
|
83
|
+
for (const line of numstat.split("\n")) {
|
|
84
|
+
if (!line.trim()) continue;
|
|
85
|
+
const [ins, del, ...rest] = line.split(" ");
|
|
86
|
+
const path2 = numstatPath(rest.join(" "));
|
|
87
|
+
if (!files.has(path2)) continue;
|
|
88
|
+
const bucket = classifyBucket(path2, buckets);
|
|
89
|
+
const stat = out[bucket] ??= { files: 0, insertions: 0, deletions: 0 };
|
|
90
|
+
stat.files += 1;
|
|
91
|
+
stat.insertions += ins === "-" ? 0 : Number(ins) || 0;
|
|
92
|
+
stat.deletions += del === "-" ? 0 : Number(del) || 0;
|
|
93
|
+
}
|
|
94
|
+
return out;
|
|
95
|
+
}
|
|
96
|
+
var firstLine = (s) => s.trim().split("\n")[0];
|
|
97
|
+
async function computeGitDiff(o) {
|
|
98
|
+
const mb = await o.git(["merge-base", "HEAD", o.base], o.cwd);
|
|
99
|
+
if (mb.code !== 0 || !mb.stdout.trim()) return { warning: `diff omitted: merge-base failed: ${firstLine(mb.stderr)}` };
|
|
100
|
+
const base = mb.stdout.trim();
|
|
101
|
+
const names = await o.git(["diff", "--name-only", `${base}...HEAD`], o.cwd);
|
|
102
|
+
const files = modifiedFilesFrom(names.stdout, o.spec, o.dir);
|
|
103
|
+
const numstat = await o.git(["diff", "--numstat", `${base}...HEAD`], o.cwd);
|
|
104
|
+
const count = await o.git(["rev-list", "--count", "--invert-grep", "--grep=^telemetry: ", `${base}..HEAD`], o.cwd);
|
|
105
|
+
return { modified_files: files, diff: { base, commits: Number(count.stdout.trim()) || 0, buckets: aggregateNumstat(numstat.stdout, new Set(files), o.buckets) } };
|
|
106
|
+
}
|
|
107
|
+
|
|
108
|
+
// extensions/lib/telemetry-record.ts
|
|
109
|
+
import { Document, isCollection, isMap, isSeq, parse as parseYaml, stringify as stringifyYaml } from "yaml";
|
|
110
|
+
var PHASES = ["brainstorm", "plan", "implement", "verify", "ship"];
|
|
111
|
+
var emptyAccumulators = () => ({ phases: {}, personas: {}, reviews: {}, spec_writes: {}, gates: { spec_rounds: 0, plan_rounds: 0, fix_round_grants: 0, task_reopens: 0 }, conformance_loops: 0, amendments: 0, spec_edits_after_ship: 0, events_dropped: 0 });
|
|
112
|
+
var addTokens = (a, b) => ({ input: (a?.input ?? 0) + b.input, output: (a?.output ?? 0) + b.output, cache_read: (a?.cache_read ?? 0) + b.cache_read, cache_write: (a?.cache_write ?? 0) + b.cache_write, cost: (a?.cost ?? 0) + b.cost });
|
|
113
|
+
var optSum = (a, b) => a === void 0 && b === void 0 ? void 0 : (a ?? 0) + (b ?? 0);
|
|
114
|
+
var optMax = (a, b) => a === void 0 ? b : b === void 0 ? a : Math.max(a, b);
|
|
115
|
+
var compact = (o) => Object.fromEntries(Object.entries(o).filter(([, v]) => v !== void 0));
|
|
116
|
+
function foldAccumulators(a, b) {
|
|
117
|
+
const out = emptyAccumulators();
|
|
118
|
+
for (const k of /* @__PURE__ */ new Set([...Object.keys(a.phases), ...Object.keys(b.phases)])) {
|
|
119
|
+
const x = a.phases[k] ?? {}, y = b.phases[k] ?? {};
|
|
120
|
+
out.phases[k] = compact({ tokens: x.tokens && y.tokens ? addTokens(x.tokens, y.tokens) : x.tokens ?? y.tokens, compactions: optSum(x.compactions, y.compactions), user_messages: optSum(x.user_messages, y.user_messages), peak_context: optMax(x.peak_context, y.peak_context) });
|
|
121
|
+
}
|
|
122
|
+
for (const k of /* @__PURE__ */ new Set([...Object.keys(a.personas), ...Object.keys(b.personas)])) {
|
|
123
|
+
const x = a.personas[k], y = b.personas[k];
|
|
124
|
+
out.personas[k] = compact({ dispatches: (x?.dispatches ?? 0) + (y?.dispatches ?? 0), models: [.../* @__PURE__ */ new Set([...x?.models ?? [], ...y?.models ?? []])], tokens: x?.tokens && y?.tokens ? addTokens(x.tokens, y.tokens) : x?.tokens ?? y?.tokens });
|
|
125
|
+
}
|
|
126
|
+
for (const k of /* @__PURE__ */ new Set([...Object.keys(a.reviews), ...Object.keys(b.reviews)])) {
|
|
127
|
+
const x = a.reviews[k], y = b.reviews[k];
|
|
128
|
+
const findings = x?.findings || y?.findings ? { blocker: (x?.findings?.blocker ?? 0) + (y?.findings?.blocker ?? 0), major: (x?.findings?.major ?? 0) + (y?.findings?.major ?? 0), minor: (x?.findings?.minor ?? 0) + (y?.findings?.minor ?? 0) } : void 0;
|
|
129
|
+
out.reviews[k] = compact({ dispatches: (x?.dispatches ?? 0) + (y?.dispatches ?? 0), nonzero_exit: (x?.nonzero_exit ?? 0) + (y?.nonzero_exit ?? 0), findings });
|
|
130
|
+
}
|
|
131
|
+
for (const k of /* @__PURE__ */ new Set([...Object.keys(a.spec_writes), ...Object.keys(b.spec_writes)])) {
|
|
132
|
+
const x = a.spec_writes[k], y = b.spec_writes[k];
|
|
133
|
+
out.spec_writes[k] = { count: (x?.count ?? 0) + (y?.count ?? 0), last_sha256: y?.last_sha256 ?? x?.last_sha256 ?? "" };
|
|
134
|
+
}
|
|
135
|
+
for (const g of Object.keys(out.gates)) out.gates[g] = a.gates[g] + b.gates[g];
|
|
136
|
+
out.conformance_loops = a.conformance_loops + b.conformance_loops;
|
|
137
|
+
out.conformance_open_gaps = b.conformance_open_gaps ?? a.conformance_open_gaps;
|
|
138
|
+
out.amendments = a.amendments + b.amendments;
|
|
139
|
+
out.spec_edits_after_ship = a.spec_edits_after_ship + b.spec_edits_after_ship;
|
|
140
|
+
out.events_dropped = a.events_dropped + b.events_dropped;
|
|
141
|
+
return out;
|
|
142
|
+
}
|
|
143
|
+
function newRecord(o) {
|
|
144
|
+
return { schema: 1, spec: o.spec, run_id: o.runId, branch: o.branch, status: "in_progress", created_at: o.now, sessions: [o.session], derived: { duration_s: 0, phases: {}, personas: {}, conformance_loops: 0, gates: { spec_rounds: 0, plan_rounds: 0, fix_round_grants: 0, task_reopens: 0 }, amendments: 0, spec_edits_after_ship: 0, spec_writes: {}, events_dropped: 0 }, accumulators: {}, events: [] };
|
|
145
|
+
}
|
|
146
|
+
var totalAccumulators = (rec) => Object.values(rec.accumulators).reduce((acc, block) => foldAccumulators(acc, block), emptyAccumulators());
|
|
147
|
+
var seconds = (a, b) => Math.max(0, Math.round((Date.parse(b) - Date.parse(a)) / 1e3));
|
|
148
|
+
function liveShipEvent(events) {
|
|
149
|
+
let live;
|
|
150
|
+
for (const e of events) {
|
|
151
|
+
if (e.kind === "ship") live = e;
|
|
152
|
+
else if (e.kind === "ship_failed") live = void 0;
|
|
153
|
+
}
|
|
154
|
+
return live;
|
|
155
|
+
}
|
|
156
|
+
function derive(rec, now) {
|
|
157
|
+
const acc = totalAccumulators(rec), phases = {};
|
|
158
|
+
for (const e of rec.events) {
|
|
159
|
+
if (e.kind !== "phase") continue;
|
|
160
|
+
if (e.action === "reset") {
|
|
161
|
+
for (const p of PHASES) if (phases[p]) phases[p] = compact({ ...phases[p], started_at: void 0, completed_at: void 0, duration_s: void 0, model: void 0, thinking: void 0 });
|
|
162
|
+
continue;
|
|
163
|
+
}
|
|
164
|
+
if (!e.name) continue;
|
|
165
|
+
const cur = phases[e.name] ?? {};
|
|
166
|
+
if (e.action === "start") phases[e.name] = compact({ ...cur, started_at: e.ts, completed_at: void 0, duration_s: void 0, model: e.model, thinking: e.thinking });
|
|
167
|
+
else if (cur.started_at) phases[e.name] = { ...cur, completed_at: e.ts, duration_s: seconds(cur.started_at, e.ts) };
|
|
168
|
+
}
|
|
169
|
+
for (const k of Object.keys(acc.phases)) phases[k] = compact({ ...phases[k] ?? {}, ...acc.phases[k] });
|
|
170
|
+
const live = liveShipEvent(rec.events);
|
|
171
|
+
return compact({ duration_s: seconds(rec.created_at, rec.shipped_at ?? rec.abandoned_at ?? now), phases, personas: acc.personas, reviews: Object.keys(acc.reviews).length ? acc.reviews : void 0, conformance_loops: acc.conformance_loops, conformance_open_gaps: acc.conformance_open_gaps, gates: compact({ ...acc.gates, ship_option: live?.option }), plan: rec.derived.plan, tests: rec.derived.tests, amendments: acc.amendments, spec_edits_after_ship: acc.spec_edits_after_ship, diff: rec.derived.diff, modified_files: rec.derived.modified_files, spec_writes: acc.spec_writes, events_dropped: acc.events_dropped });
|
|
172
|
+
}
|
|
173
|
+
var stripUndefined = (v) => {
|
|
174
|
+
if (Array.isArray(v)) return v.map(stripUndefined);
|
|
175
|
+
if (v && typeof v === "object") return Object.fromEntries(Object.entries(v).filter(([, x]) => x !== void 0).map(([k, x]) => [k, stripUndefined(x)]));
|
|
176
|
+
return v;
|
|
177
|
+
};
|
|
178
|
+
var flowLeafChildren = (map, blockLists = /* @__PURE__ */ new Set()) => {
|
|
179
|
+
for (const pair of map.items) {
|
|
180
|
+
const key = String(pair.key);
|
|
181
|
+
const child = pair.value;
|
|
182
|
+
if (!isCollection(child) || blockLists.has(key)) continue;
|
|
183
|
+
if (isMap(child) && child.items.length > 0 && child.items.every((item) => isMap(item.value))) {
|
|
184
|
+
for (const item of child.items) if (isCollection(item.value)) item.value.flow = true;
|
|
185
|
+
} else {
|
|
186
|
+
child.flow = true;
|
|
187
|
+
}
|
|
188
|
+
}
|
|
189
|
+
};
|
|
190
|
+
var flowModelLists = (node) => {
|
|
191
|
+
if (isMap(node)) {
|
|
192
|
+
for (const pair of node.items) {
|
|
193
|
+
if (String(pair.key) === "models" && isSeq(pair.value)) pair.value.flow = true;
|
|
194
|
+
flowModelLists(pair.value);
|
|
195
|
+
}
|
|
196
|
+
} else if (isSeq(node)) {
|
|
197
|
+
for (const item of node.items) flowModelLists(item);
|
|
198
|
+
}
|
|
199
|
+
};
|
|
200
|
+
function serializeRecord(rec) {
|
|
201
|
+
const { derived, accumulators, events, ...head } = rec;
|
|
202
|
+
const body = stripUndefined({ ...head, derived, accumulators });
|
|
203
|
+
const doc = new Document(body);
|
|
204
|
+
const derivedNode = doc.get("derived", true);
|
|
205
|
+
if (isMap(derivedNode)) flowLeafChildren(derivedNode, /* @__PURE__ */ new Set(["modified_files"]));
|
|
206
|
+
const accumulatorNode = doc.get("accumulators", true);
|
|
207
|
+
if (isMap(accumulatorNode)) {
|
|
208
|
+
for (const session of accumulatorNode.items) if (isMap(session.value)) flowLeafChildren(session.value);
|
|
209
|
+
}
|
|
210
|
+
flowModelLists(doc.contents);
|
|
211
|
+
const bodyText = doc.toString({ lineWidth: 0 });
|
|
212
|
+
const eventLines = events.map((e) => " - " + stringifyYaml(stripUndefined(e), { collectionStyle: "flow", lineWidth: 0 }).trim());
|
|
213
|
+
return bodyText + "events:\n" + (eventLines.length ? eventLines.join("\n") + "\n" : "");
|
|
214
|
+
}
|
|
215
|
+
function parseRecord(text) {
|
|
216
|
+
let doc;
|
|
217
|
+
try {
|
|
218
|
+
doc = parseYaml(text);
|
|
219
|
+
} catch {
|
|
220
|
+
return void 0;
|
|
221
|
+
}
|
|
222
|
+
if (!doc || typeof doc !== "object") return void 0;
|
|
223
|
+
const d = doc;
|
|
224
|
+
if (d.schema !== 1 || typeof d.spec !== "string" || typeof d.run_id !== "string") return void 0;
|
|
225
|
+
const accumulators = Object.fromEntries(Object.entries(d.accumulators ?? {}).flatMap(
|
|
226
|
+
([session, block]) => block && typeof block === "object" && !Array.isArray(block) ? [[session, { ...emptyAccumulators(), ...block }]] : []
|
|
227
|
+
));
|
|
228
|
+
return { ...d, sessions: Array.isArray(d.sessions) ? d.sessions : [], derived: d.derived ?? newRecord({ spec: d.spec, session: "", now: d.created_at ?? "", runId: d.run_id }).derived, accumulators, events: Array.isArray(d.events) ? d.events : [] };
|
|
229
|
+
}
|
|
230
|
+
|
|
231
|
+
// src/bins/gauntlet-telemetry-seal.mjs
|
|
232
|
+
var GIT_TIMEOUT_MS = 1e4;
|
|
233
|
+
var COMMIT_TIMEOUT_MS = 3e4;
|
|
234
|
+
var GIT_ENV = { ...process.env, GIT_TERMINAL_PROMPT: "0", GIT_EDITOR: "true" };
|
|
235
|
+
var OPTIONS = /* @__PURE__ */ new Set(["pr", "squash"]);
|
|
236
|
+
var fail = (code, message) => {
|
|
237
|
+
process.stderr.write(message + "\n");
|
|
238
|
+
process.exit(code);
|
|
239
|
+
};
|
|
240
|
+
var usage = () => fail(1, "usage: gauntlet-telemetry-seal --worktree <abs path> --option <pr|squash> --base <ref> [--spec <spec path inside the checkout>] [--dir <telemetry dir>]");
|
|
241
|
+
function parseArgs(argv) {
|
|
242
|
+
const opts = { worktree: void 0, option: void 0, base: void 0, spec: void 0, dir: void 0 };
|
|
243
|
+
for (let i = 0; i < argv.length; i++) {
|
|
244
|
+
const a = argv[i];
|
|
245
|
+
const value = argv[i + 1];
|
|
246
|
+
const takes = (key) => {
|
|
247
|
+
if (value === void 0 || value.startsWith("--")) usage();
|
|
248
|
+
opts[key] = argv[++i];
|
|
249
|
+
};
|
|
250
|
+
if (a === "--worktree") takes("worktree");
|
|
251
|
+
else if (a === "--option") takes("option");
|
|
252
|
+
else if (a === "--base") takes("base");
|
|
253
|
+
else if (a === "--spec") takes("spec");
|
|
254
|
+
else if (a === "--dir") takes("dir");
|
|
255
|
+
else usage();
|
|
256
|
+
}
|
|
257
|
+
if (!opts.worktree || !isAbsolute2(opts.worktree) || !opts.base || !OPTIONS.has(opts.option)) usage();
|
|
258
|
+
return opts;
|
|
259
|
+
}
|
|
260
|
+
function git(cwd, args, timeout = GIT_TIMEOUT_MS) {
|
|
261
|
+
const r = spawnSync("git", args, { cwd, encoding: "utf8", env: GIT_ENV, timeout });
|
|
262
|
+
const timedOut = r.error?.code === "ETIMEDOUT";
|
|
263
|
+
const stderr = timedOut ? "timed out" : (r.stderr ?? "").trim().split("\n")[0] || r.error?.message || `git exited ${r.status}`;
|
|
264
|
+
return { ok: !timedOut && r.status === 0, code: timedOut ? 1 : r.status ?? 1, stdout: (r.stdout ?? "").trim(), stderr };
|
|
265
|
+
}
|
|
266
|
+
var gitRunner = (args, cwd) => {
|
|
267
|
+
const r = git(cwd, args);
|
|
268
|
+
return { code: r.code, stdout: r.stdout, stderr: r.stderr };
|
|
269
|
+
};
|
|
270
|
+
function readLayer(file) {
|
|
271
|
+
if (!existsSync(file)) return {};
|
|
272
|
+
try {
|
|
273
|
+
return JSON.parse(readFileSync(file, "utf8"));
|
|
274
|
+
} catch (e) {
|
|
275
|
+
process.stderr.write(`warning: ${file}: ${e.message}; using {} for this layer
|
|
276
|
+
`);
|
|
277
|
+
return {};
|
|
278
|
+
}
|
|
279
|
+
}
|
|
280
|
+
function telemetrySettings(root) {
|
|
281
|
+
const agentDir = process.env.PI_CODING_AGENT_DIR || join(homedir(), ".pi", "agent");
|
|
282
|
+
const preset = readLayer(join(agentDir, "settings.json"));
|
|
283
|
+
const repo = readLayer(join(root, ".pi", "settings.json"));
|
|
284
|
+
return resolveTelemetry(mergeGauntlet(preset?.piGauntlet, repo?.piGauntlet));
|
|
285
|
+
}
|
|
286
|
+
var isoNow = () => (/* @__PURE__ */ new Date()).toISOString().replace(/\.\d{3}Z$/, "Z");
|
|
287
|
+
var trackedAndClean = (root, rec) => git(root, ["ls-files", "--error-unmatch", "--", rec]).ok && git(root, ["diff", "--quiet", "HEAD", "--", rec]).ok;
|
|
288
|
+
async function seal(root, rec, o) {
|
|
289
|
+
const abs = join(root, rec);
|
|
290
|
+
if (!existsSync(abs)) fail(2, `no record at ${rec}`);
|
|
291
|
+
const prior = readFileSync(abs);
|
|
292
|
+
const parsed = parseRecord(prior.toString("utf8"));
|
|
293
|
+
if (!parsed) fail(2, `unparseable record ${rec}: schema 1 with spec and run_id required`);
|
|
294
|
+
if (parsed.status !== "in_progress") {
|
|
295
|
+
if (parsed.status !== "shipped" || trackedAndClean(root, rec)) return console.log(`already sealed ${rec}`);
|
|
296
|
+
} else {
|
|
297
|
+
const now = isoNow();
|
|
298
|
+
parsed.status = "shipped";
|
|
299
|
+
parsed.shipped_at = now;
|
|
300
|
+
parsed.events.push({ ts: now, session: parsed.sessions.at(-1) ?? "seal", phase: "ship", kind: "ship", option: o.option });
|
|
301
|
+
const out = await computeGitDiff({ git: gitRunner, cwd: root, spec: parsed.spec, dir: o.dir, buckets: o.buckets, base: o.base });
|
|
302
|
+
if (out.warning) parsed.events.push({ ts: now, session: parsed.sessions.at(-1) ?? "seal", phase: "ship", kind: "warning", message: out.warning });
|
|
303
|
+
parsed.derived.diff = out.diff;
|
|
304
|
+
parsed.derived.modified_files = out.modified_files;
|
|
305
|
+
parsed.derived = derive(parsed, now);
|
|
306
|
+
writeFileSync(abs, serializeRecord(parsed));
|
|
307
|
+
}
|
|
308
|
+
const rollback = (reason) => {
|
|
309
|
+
writeFileSync(abs, prior);
|
|
310
|
+
git(root, ["reset", "-q", "--", rec]);
|
|
311
|
+
fail(1, `seal failed ${rec}: ${reason}`);
|
|
312
|
+
};
|
|
313
|
+
const add = git(root, ["add", "-f", "--", rec]);
|
|
314
|
+
if (!add.ok) rollback(add.stderr);
|
|
315
|
+
const commit = git(root, ["commit", "-q", "-m", `telemetry: ${parsed.spec}`, "--", rec], COMMIT_TIMEOUT_MS);
|
|
316
|
+
if (!commit.ok) rollback(commit.stderr);
|
|
317
|
+
console.log(`sealed ${rec}`);
|
|
318
|
+
}
|
|
319
|
+
async function main() {
|
|
320
|
+
const opts = parseArgs(process.argv.slice(2));
|
|
321
|
+
const override = opts.dir === void 0 ? void 0 : resolveTelemetry({ telemetry: { enabled: true, dir: opts.dir } });
|
|
322
|
+
if (override?.warning) usage();
|
|
323
|
+
const top = existsSync(opts.worktree) ? git(opts.worktree, ["rev-parse", "--show-toplevel"]) : { ok: false };
|
|
324
|
+
if (!top.ok) fail(1, "not a git checkout");
|
|
325
|
+
const root = top.stdout;
|
|
326
|
+
let rel;
|
|
327
|
+
if (opts.spec) {
|
|
328
|
+
const specAbs = isAbsolute2(opts.spec) && existsSync(opts.spec) ? realpathSync(opts.spec) : opts.spec;
|
|
329
|
+
rel = repoRelativeToolPath(root, root, specAbs);
|
|
330
|
+
if (!rel || !isSpecPath(rel)) usage();
|
|
331
|
+
}
|
|
332
|
+
const telemetry = telemetrySettings(root);
|
|
333
|
+
if (telemetry.warning) process.stderr.write(`warning: ${telemetry.warning}
|
|
334
|
+
`);
|
|
335
|
+
if (!telemetry.enabled) return console.log("telemetry disabled");
|
|
336
|
+
const dir = override?.dir ?? telemetry.dir;
|
|
337
|
+
if (!git(root, ["rev-parse", "--verify", "-q", `${opts.base}^{commit}`]).ok) fail(1, `no base ref ${opts.base}`);
|
|
338
|
+
const o = { option: opts.option, base: opts.base, dir, buckets: telemetry.buckets };
|
|
339
|
+
if (opts.spec) return seal(root, recordPathFor(dir, rel), o);
|
|
340
|
+
const diff = git(root, ["diff", "--name-only", `${opts.base}...HEAD`]);
|
|
341
|
+
if (!diff.ok) fail(1, `git diff failed: ${diff.stderr}`);
|
|
342
|
+
const records = diff.stdout.split("\n").filter((f) => f && isSpecPath(f)).map((f) => recordPathFor(dir, f)).filter((rec) => existsSync(join(root, rec)));
|
|
343
|
+
if (records.length === 0) return console.log("no telemetry run");
|
|
344
|
+
for (const rec of records) await seal(root, rec, o);
|
|
345
|
+
}
|
|
346
|
+
await main();
|